Cargando…

Quality assessment of higher resolution images and videos with remote testing

In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be...

Descripción completa

Detalles Bibliográficos
Autores principales: Göring, Steve, Rao, Rakesh Rao Ramachandra, Raake, Alexander
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer International Publishing 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10098993/
https://www.ncbi.nlm.nih.gov/pubmed/37073257
http://dx.doi.org/10.1007/s41233-023-00055-6
_version_ 1785024949548744704
author Göring, Steve
Rao, Rakesh Rao Ramachandra
Raake, Alexander
author_facet Göring, Steve
Rao, Rakesh Rao Ramachandra
Raake, Alexander
author_sort Göring, Steve
collection PubMed
description In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be one approach to get such quality annotations. They are usually performed in well-defined and controlled environments to ensure high reliability. However, this high reliability comes at a cost of higher time consumption and costs incurred. To mitigate this, crowd or online tests could be used. Usually, online tests cover a wider range of end devices, environmental conditions, or participants, which may have an impact on the ratings. To verify whether such online tests can be used for visual quality assessment, we designed three online tests. These online tests are based on previously conducted lab tests as this enables comparison of the results of both test paradigms. Our focus is on the quality assessment of high-resolution images and videos. The online tests use AVrate Voyager, which is a publicly accessible framework for online tests. To transform the lab tests into online tests, dedicated adaptations in the test methodologies are required. The considered modifications are, for example, a patch-based or centre cropping of the images and videos, or a randomly sub-sampling of the to-be-rated stimuli. Based on the analysis of the test results in terms of correlation and SOS analysis it is shown that online tests can be used as a reliable replacement for lab tests albeit with some limitations. These limitations relate to, e.g., lack of appropriate display devices, limitation of web technologies, and modern browsers considering support for different video codecs and formats.
format Online
Article
Text
id pubmed-10098993
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Springer International Publishing
record_format MEDLINE/PubMed
spelling pubmed-100989932023-04-14 Quality assessment of higher resolution images and videos with remote testing Göring, Steve Rao, Rakesh Rao Ramachandra Raake, Alexander Qual User Exp Research Article In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be one approach to get such quality annotations. They are usually performed in well-defined and controlled environments to ensure high reliability. However, this high reliability comes at a cost of higher time consumption and costs incurred. To mitigate this, crowd or online tests could be used. Usually, online tests cover a wider range of end devices, environmental conditions, or participants, which may have an impact on the ratings. To verify whether such online tests can be used for visual quality assessment, we designed three online tests. These online tests are based on previously conducted lab tests as this enables comparison of the results of both test paradigms. Our focus is on the quality assessment of high-resolution images and videos. The online tests use AVrate Voyager, which is a publicly accessible framework for online tests. To transform the lab tests into online tests, dedicated adaptations in the test methodologies are required. The considered modifications are, for example, a patch-based or centre cropping of the images and videos, or a randomly sub-sampling of the to-be-rated stimuli. Based on the analysis of the test results in terms of correlation and SOS analysis it is shown that online tests can be used as a reliable replacement for lab tests albeit with some limitations. These limitations relate to, e.g., lack of appropriate display devices, limitation of web technologies, and modern browsers considering support for different video codecs and formats. Springer International Publishing 2023-04-13 2023 /pmc/articles/PMC10098993/ /pubmed/37073257 http://dx.doi.org/10.1007/s41233-023-00055-6 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Research Article
Göring, Steve
Rao, Rakesh Rao Ramachandra
Raake, Alexander
Quality assessment of higher resolution images and videos with remote testing
title Quality assessment of higher resolution images and videos with remote testing
title_full Quality assessment of higher resolution images and videos with remote testing
title_fullStr Quality assessment of higher resolution images and videos with remote testing
title_full_unstemmed Quality assessment of higher resolution images and videos with remote testing
title_short Quality assessment of higher resolution images and videos with remote testing
title_sort quality assessment of higher resolution images and videos with remote testing
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10098993/
https://www.ncbi.nlm.nih.gov/pubmed/37073257
http://dx.doi.org/10.1007/s41233-023-00055-6
work_keys_str_mv AT goringsteve qualityassessmentofhigherresolutionimagesandvideoswithremotetesting
AT raorakeshraoramachandra qualityassessmentofhigherresolutionimagesandvideoswithremotetesting
AT raakealexander qualityassessmentofhigherresolutionimagesandvideoswithremotetesting