Cargando…

A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models

Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been pro...

Descripción completa

Detalles Bibliográficos
Autores principales: Nagy, Gabriel, Ulitzsch, Esther
Formato: Online Artículo Texto
Lenguaje:English
Publicado: SAGE Publications 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9386881/
https://www.ncbi.nlm.nih.gov/pubmed/35989730
http://dx.doi.org/10.1177/00131644211045351
_version_ 1784769909439332352
author Nagy, Gabriel
Ulitzsch, Esther
author_facet Nagy, Gabriel
Ulitzsch, Esther
author_sort Nagy, Gabriel
collection PubMed
description Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model.
format Online
Article
Text
id pubmed-9386881
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher SAGE Publications
record_format MEDLINE/PubMed
spelling pubmed-93868812022-08-19 A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models Nagy, Gabriel Ulitzsch, Esther Educ Psychol Meas Article Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model. SAGE Publications 2021-09-13 2022-10 /pmc/articles/PMC9386881/ /pubmed/35989730 http://dx.doi.org/10.1177/00131644211045351 Text en © The Author(s) 2021 https://creativecommons.org/licenses/by-nc/4.0/This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
spellingShingle Article
Nagy, Gabriel
Ulitzsch, Esther
A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title_full A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title_fullStr A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title_full_unstemmed A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title_short A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models
title_sort multilevel mixture irt framework for modeling response times as predictors or indicators of response engagement in irt models
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9386881/
https://www.ncbi.nlm.nih.gov/pubmed/35989730
http://dx.doi.org/10.1177/00131644211045351
work_keys_str_mv AT nagygabriel amultilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels
AT ulitzschesther amultilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels
AT nagygabriel multilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels
AT ulitzschesther multilevelmixtureirtframeworkformodelingresponsetimesaspredictorsorindicatorsofresponseengagementinirtmodels