Cargando…
Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis
BACKGROUND: Progress testing is an assessment tool used to periodically assess all students at the end-of-curriculum level. Because students cannot know everything, it is important that they recognize their lack of knowledge. For that reason, the formula-scoring method has usually been used. However...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5679154/ https://www.ncbi.nlm.nih.gov/pubmed/29121888 http://dx.doi.org/10.1186/s12909-017-1051-8 |
_version_ | 1783277554404163584 |
---|---|
author | Cecilio-Fernandes, Dario Medema, Harro Collares, Carlos Fernando Schuwirth, Lambert Cohen-Schotanus, Janke Tio, René A. |
author_facet | Cecilio-Fernandes, Dario Medema, Harro Collares, Carlos Fernando Schuwirth, Lambert Cohen-Schotanus, Janke Tio, René A. |
author_sort | Cecilio-Fernandes, Dario |
collection | PubMed |
description | BACKGROUND: Progress testing is an assessment tool used to periodically assess all students at the end-of-curriculum level. Because students cannot know everything, it is important that they recognize their lack of knowledge. For that reason, the formula-scoring method has usually been used. However, where partial knowledge needs to be taken into account, the number-right scoring method is used. Research comparing both methods has yielded conflicting results. As far as we know, in all these studies, Classical Test Theory or Generalizability Theory was used to analyze the data. In contrast to these studies, we will explore the use of the Rasch model to compare both methods. METHODS: A 2 × 2 crossover design was used in a study where 298 students from four medical schools participated. A sample of 200 previously used questions from the progress tests was selected. The data were analyzed using the Rasch model, which provides fit parameters, reliability coefficients, and response option analysis. RESULTS: The fit parameters were in the optimal interval ranging from 0.50 to 1.50, and the means were around 1.00. The person and item reliability coefficients were higher in the number-right condition than in the formula-scoring condition. The response option analysis showed that the majority of dysfunctional items emerged in the formula-scoring condition. CONCLUSIONS: The findings of this study support the use of number-right scoring over formula scoring. Rasch model analyses showed that tests with number-right scoring have better psychometric properties than formula scoring. However, choosing the appropriate scoring method should depend not only on psychometric properties but also on self-directed test-taking strategies and metacognitive skills. |
format | Online Article Text |
id | pubmed-5679154 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-56791542017-11-17 Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis Cecilio-Fernandes, Dario Medema, Harro Collares, Carlos Fernando Schuwirth, Lambert Cohen-Schotanus, Janke Tio, René A. BMC Med Educ Research Article BACKGROUND: Progress testing is an assessment tool used to periodically assess all students at the end-of-curriculum level. Because students cannot know everything, it is important that they recognize their lack of knowledge. For that reason, the formula-scoring method has usually been used. However, where partial knowledge needs to be taken into account, the number-right scoring method is used. Research comparing both methods has yielded conflicting results. As far as we know, in all these studies, Classical Test Theory or Generalizability Theory was used to analyze the data. In contrast to these studies, we will explore the use of the Rasch model to compare both methods. METHODS: A 2 × 2 crossover design was used in a study where 298 students from four medical schools participated. A sample of 200 previously used questions from the progress tests was selected. The data were analyzed using the Rasch model, which provides fit parameters, reliability coefficients, and response option analysis. RESULTS: The fit parameters were in the optimal interval ranging from 0.50 to 1.50, and the means were around 1.00. The person and item reliability coefficients were higher in the number-right condition than in the formula-scoring condition. The response option analysis showed that the majority of dysfunctional items emerged in the formula-scoring condition. CONCLUSIONS: The findings of this study support the use of number-right scoring over formula scoring. Rasch model analyses showed that tests with number-right scoring have better psychometric properties than formula scoring. However, choosing the appropriate scoring method should depend not only on psychometric properties but also on self-directed test-taking strategies and metacognitive skills. BioMed Central 2017-11-09 /pmc/articles/PMC5679154/ /pubmed/29121888 http://dx.doi.org/10.1186/s12909-017-1051-8 Text en © The Author(s). 2017 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Article Cecilio-Fernandes, Dario Medema, Harro Collares, Carlos Fernando Schuwirth, Lambert Cohen-Schotanus, Janke Tio, René A. Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title | Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title_full | Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title_fullStr | Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title_full_unstemmed | Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title_short | Comparison of formula and number-right scoring in undergraduate medical training: a Rasch model analysis |
title_sort | comparison of formula and number-right scoring in undergraduate medical training: a rasch model analysis |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5679154/ https://www.ncbi.nlm.nih.gov/pubmed/29121888 http://dx.doi.org/10.1186/s12909-017-1051-8 |
work_keys_str_mv | AT ceciliofernandesdario comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis AT medemaharro comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis AT collarescarlosfernando comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis AT schuwirthlambert comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis AT cohenschotanusjanke comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis AT tiorenea comparisonofformulaandnumberrightscoringinundergraduatemedicaltrainingaraschmodelanalysis |