Cargando…
A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been pro...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
SAGE Publications
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9386881/ https://www.ncbi.nlm.nih.gov/pubmed/35989730 http://dx.doi.org/10.1177/00131644211045351 |
_version_ | 1784769909439332352 |
---|---|
author | Nagy, Gabriel Ulitzsch, Esther |
author_facet | Nagy, Gabriel Ulitzsch, Esther |
author_sort | Nagy, Gabriel |
collection | PubMed |
description | Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model. |
format | Online Article Text |
id | pubmed-9386881 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | SAGE Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-93868812022-08-19 A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models Nagy, Gabriel Ulitzsch, Esther Educ Psychol Meas Article Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model. SAGE Publications 2021-09-13 2022-10 /pmc/articles/PMC9386881/ /pubmed/35989730 http://dx.doi.org/10.1177/00131644211045351 Text en © The Author(s) 2021 https://creativecommons.org/licenses/by-nc/4.0/This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). |
spellingShingle | Article Nagy, Gabriel Ulitzsch, Esther A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title | A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title_full | A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title_fullStr | A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title_full_unstemmed | A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title_short | A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models |
title_sort | multilevel mixture irt framework for modeling response times as predictors or indicators of response engagement in irt models |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9386881/ https://www.ncbi.nlm.nih.gov/pubmed/35989730 http://dx.doi.org/10.1177/00131644211045351 |
work_keys_str_mv | AT nagygabriel amultilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels AT ulitzschesther amultilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels AT nagygabriel multilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels AT ulitzschesther multilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels |