Cargando…
Cognitive Diagnostic Models for Rater Effects
In recent decades, cognitive diagnostic models (CDMs) have been intensively researched and applied to various educational and psychological tests. However, because existing CDMs fail to consider rater effects, the application of CDMs to constructed-response (CR) items that involve human raters is se...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7105832/ https://www.ncbi.nlm.nih.gov/pubmed/32265810 http://dx.doi.org/10.3389/fpsyg.2020.00525 |
_version_ | 1783512484110401536 |
---|---|
author | Li, Xiaomin Wang, Wen-Chung Xie, Qin |
author_facet | Li, Xiaomin Wang, Wen-Chung Xie, Qin |
author_sort | Li, Xiaomin |
collection | PubMed |
description | In recent decades, cognitive diagnostic models (CDMs) have been intensively researched and applied to various educational and psychological tests. However, because existing CDMs fail to consider rater effects, the application of CDMs to constructed-response (CR) items that involve human raters is seriously limited. Given the popularity of CR items, it is desirable to develop new CDMs that are capable of describing and estimating rater effects on CR items. In this study, we developed such new CDMs within the frameworks of facets models and hierarchical rater models, using the log-linear cognitive diagnosis model as a template. The parameters of the new models were estimated with the Markov chain Monte Carlo methods implemented in the freeware JAGS. Simulations were conducted to evaluate the parameter recovery of the new models. Results showed that the parameters were recovered fairly well and the more data there were, the better the recovery. Implications and applications of the new models were illustrated with an empirical study that adopted a fine-grained checklist to assess English academic essays. |
format | Online Article Text |
id | pubmed-7105832 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-71058322020-04-07 Cognitive Diagnostic Models for Rater Effects Li, Xiaomin Wang, Wen-Chung Xie, Qin Front Psychol Psychology In recent decades, cognitive diagnostic models (CDMs) have been intensively researched and applied to various educational and psychological tests. However, because existing CDMs fail to consider rater effects, the application of CDMs to constructed-response (CR) items that involve human raters is seriously limited. Given the popularity of CR items, it is desirable to develop new CDMs that are capable of describing and estimating rater effects on CR items. In this study, we developed such new CDMs within the frameworks of facets models and hierarchical rater models, using the log-linear cognitive diagnosis model as a template. The parameters of the new models were estimated with the Markov chain Monte Carlo methods implemented in the freeware JAGS. Simulations were conducted to evaluate the parameter recovery of the new models. Results showed that the parameters were recovered fairly well and the more data there were, the better the recovery. Implications and applications of the new models were illustrated with an empirical study that adopted a fine-grained checklist to assess English academic essays. Frontiers Media S.A. 2020-03-24 /pmc/articles/PMC7105832/ /pubmed/32265810 http://dx.doi.org/10.3389/fpsyg.2020.00525 Text en Copyright © 2020 Li, Wang and Xie. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Psychology Li, Xiaomin Wang, Wen-Chung Xie, Qin Cognitive Diagnostic Models for Rater Effects |
title | Cognitive Diagnostic Models for Rater Effects |
title_full | Cognitive Diagnostic Models for Rater Effects |
title_fullStr | Cognitive Diagnostic Models for Rater Effects |
title_full_unstemmed | Cognitive Diagnostic Models for Rater Effects |
title_short | Cognitive Diagnostic Models for Rater Effects |
title_sort | cognitive diagnostic models for rater effects |
topic | Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7105832/ https://www.ncbi.nlm.nih.gov/pubmed/32265810 http://dx.doi.org/10.3389/fpsyg.2020.00525 |
work_keys_str_mv | AT lixiaomin cognitivediagnosticmodelsforratereffects AT wangwenchung cognitivediagnosticmodelsforratereffects AT xieqin cognitivediagnosticmodelsforratereffects |