Cargando…

Bias of Two-Level Scalability Coefficients and Their Standard Errors

Two-level Mokken scale analysis is a generalization of Mokken scale analysis for multi-rater data. The bias of estimated scalability coefficients for two-level Mokken scale analysis, the bias of their estimated standard errors, and the coverage of the confidence intervals has been investigated, unde...

Descripción completa

Detalles Bibliográficos
Autores principales: Koopman, Letty, Zijlstra, Bonne J. H., de Rooij, Mark, van der Ark, L. Andries
Formato: Online Artículo Texto
Lenguaje:English
Publicado: SAGE Publications 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7174805/
https://www.ncbi.nlm.nih.gov/pubmed/32341607
http://dx.doi.org/10.1177/0146621619843821
_version_ 1783524703804063744
author Koopman, Letty
Zijlstra, Bonne J. H.
de Rooij, Mark
van der Ark, L. Andries
author_facet Koopman, Letty
Zijlstra, Bonne J. H.
de Rooij, Mark
van der Ark, L. Andries
author_sort Koopman, Letty
collection PubMed
description Two-level Mokken scale analysis is a generalization of Mokken scale analysis for multi-rater data. The bias of estimated scalability coefficients for two-level Mokken scale analysis, the bias of their estimated standard errors, and the coverage of the confidence intervals has been investigated, under various testing conditions. It was found that the estimated scalability coefficients were unbiased in all tested conditions. For estimating standard errors, the delta method and the cluster bootstrap were compared. The cluster bootstrap structurally underestimated the standard errors of the scalability coefficients, with low coverage values. Except for unequal numbers of raters across subjects and small sets of items, the delta method standard error estimates had negligible bias and good coverage. Post hoc simulations showed that the cluster bootstrap does not correctly reproduce the sampling distribution of the scalability coefficients, and an adapted procedure was suggested. In addition, the delta method standard errors can be slightly improved if the harmonic mean is used for unequal numbers of raters per subject rather than the arithmetic mean.
format Online
Article
Text
id pubmed-7174805
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher SAGE Publications
record_format MEDLINE/PubMed
spelling pubmed-71748052021-05-01 Bias of Two-Level Scalability Coefficients and Their Standard Errors Koopman, Letty Zijlstra, Bonne J. H. de Rooij, Mark van der Ark, L. Andries Appl Psychol Meas Articles Two-level Mokken scale analysis is a generalization of Mokken scale analysis for multi-rater data. The bias of estimated scalability coefficients for two-level Mokken scale analysis, the bias of their estimated standard errors, and the coverage of the confidence intervals has been investigated, under various testing conditions. It was found that the estimated scalability coefficients were unbiased in all tested conditions. For estimating standard errors, the delta method and the cluster bootstrap were compared. The cluster bootstrap structurally underestimated the standard errors of the scalability coefficients, with low coverage values. Except for unequal numbers of raters across subjects and small sets of items, the delta method standard error estimates had negligible bias and good coverage. Post hoc simulations showed that the cluster bootstrap does not correctly reproduce the sampling distribution of the scalability coefficients, and an adapted procedure was suggested. In addition, the delta method standard errors can be slightly improved if the harmonic mean is used for unequal numbers of raters per subject rather than the arithmetic mean. SAGE Publications 2019-05-14 2020-05 /pmc/articles/PMC7174805/ /pubmed/32341607 http://dx.doi.org/10.1177/0146621619843821 Text en © The Author(s) 2019 http://www.creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
spellingShingle Articles
Koopman, Letty
Zijlstra, Bonne J. H.
de Rooij, Mark
van der Ark, L. Andries
Bias of Two-Level Scalability Coefficients and Their Standard Errors
title Bias of Two-Level Scalability Coefficients and Their Standard Errors
title_full Bias of Two-Level Scalability Coefficients and Their Standard Errors
title_fullStr Bias of Two-Level Scalability Coefficients and Their Standard Errors
title_full_unstemmed Bias of Two-Level Scalability Coefficients and Their Standard Errors
title_short Bias of Two-Level Scalability Coefficients and Their Standard Errors
title_sort bias of two-level scalability coefficients and their standard errors
topic Articles
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7174805/
https://www.ncbi.nlm.nih.gov/pubmed/32341607
http://dx.doi.org/10.1177/0146621619843821
work_keys_str_mv AT koopmanletty biasoftwolevelscalabilitycoefficientsandtheirstandarderrors
AT zijlstrabonnejh biasoftwolevelscalabilitycoefficientsandtheirstandarderrors
AT derooijmark biasoftwolevelscalabilitycoefficientsandtheirstandarderrors
AT vanderarklandries biasoftwolevelscalabilitycoefficientsandtheirstandarderrors