Cargando…

Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling

Some of the ‘best practice’ approaches to ensuring reproducibility of research can be difficult to implement in the developmental and clinical domains, where sample sizes and session lengths are constrained by the practicalities of recruitment and testing. For this reason, an important area of impro...

Descripción completa

Detalles Bibliográficos
Autores principales: Burton, Nichola, Burton, Michael, Fisher, Carmen, Peña, Patricia González, Rhodes, Gillian, Ewing, Louise
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer US 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8516750/
https://www.ncbi.nlm.nih.gov/pubmed/33821456
http://dx.doi.org/10.3758/s13428-021-01566-w
_version_ 1784583859751354368
author Burton, Nichola
Burton, Michael
Fisher, Carmen
Peña, Patricia González
Rhodes, Gillian
Ewing, Louise
author_facet Burton, Nichola
Burton, Michael
Fisher, Carmen
Peña, Patricia González
Rhodes, Gillian
Ewing, Louise
author_sort Burton, Nichola
collection PubMed
description Some of the ‘best practice’ approaches to ensuring reproducibility of research can be difficult to implement in the developmental and clinical domains, where sample sizes and session lengths are constrained by the practicalities of recruitment and testing. For this reason, an important area of improvement to target is the reliability of measurement. Here we demonstrate that best–worst scaling (BWS) provides a superior alternative to Likert ratings for measuring children’s subjective impressions. Seventy-three children aged 5–6 years rated the trustworthiness of faces using either Likert ratings or BWS over two sessions. Individual children’s ratings in the BWS condition were significantly more consistent from session 1 to session 2 than those in the Likert condition, a finding we also replicate with a large adult sample (N = 72). BWS also produced more reliable ratings at the group level than Likert ratings in the child sample. These findings indicate that BWS is a developmentally appropriate response format that can deliver substantial improvements in reliability of measurement, which can increase our confidence in the robustness of findings with children.
format Online
Article
Text
id pubmed-8516750
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Springer US
record_format MEDLINE/PubMed
spelling pubmed-85167502021-10-29 Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling Burton, Nichola Burton, Michael Fisher, Carmen Peña, Patricia González Rhodes, Gillian Ewing, Louise Behav Res Methods Article Some of the ‘best practice’ approaches to ensuring reproducibility of research can be difficult to implement in the developmental and clinical domains, where sample sizes and session lengths are constrained by the practicalities of recruitment and testing. For this reason, an important area of improvement to target is the reliability of measurement. Here we demonstrate that best–worst scaling (BWS) provides a superior alternative to Likert ratings for measuring children’s subjective impressions. Seventy-three children aged 5–6 years rated the trustworthiness of faces using either Likert ratings or BWS over two sessions. Individual children’s ratings in the BWS condition were significantly more consistent from session 1 to session 2 than those in the Likert condition, a finding we also replicate with a large adult sample (N = 72). BWS also produced more reliable ratings at the group level than Likert ratings in the child sample. These findings indicate that BWS is a developmentally appropriate response format that can deliver substantial improvements in reliability of measurement, which can increase our confidence in the robustness of findings with children. Springer US 2021-04-05 2021 /pmc/articles/PMC8516750/ /pubmed/33821456 http://dx.doi.org/10.3758/s13428-021-01566-w Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Burton, Nichola
Burton, Michael
Fisher, Carmen
Peña, Patricia González
Rhodes, Gillian
Ewing, Louise
Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title_full Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title_fullStr Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title_full_unstemmed Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title_short Beyond Likert ratings: Improving the robustness of developmental research measurement using best–worst scaling
title_sort beyond likert ratings: improving the robustness of developmental research measurement using best–worst scaling
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8516750/
https://www.ncbi.nlm.nih.gov/pubmed/33821456
http://dx.doi.org/10.3758/s13428-021-01566-w
work_keys_str_mv AT burtonnichola beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling
AT burtonmichael beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling
AT fishercarmen beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling
AT penapatriciagonzalez beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling
AT rhodesgillian beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling
AT ewinglouise beyondlikertratingsimprovingtherobustnessofdevelopmentalresearchmeasurementusingbestworstscaling