Cargando…
Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial
BACKGROUND: Little is known regarding the psychometric properties of computerized long-menu formats in comparison to classic formats. We compared single-best-answer (Type A) and long-menu formats using identical question stems during the computer-based, summative, intermediate clinical-clerkship exa...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6582573/ https://www.ncbi.nlm.nih.gov/pubmed/31215430 http://dx.doi.org/10.1186/s12909-019-1651-6 |
_version_ | 1783428351935905792 |
---|---|
author | Cerutti, Bernard Stollar, Fabiola Escher, Monica Blondon, Katherine Aujesky, Susanne Nendaz, Mathieu Galetto-Lacour, Annick |
author_facet | Cerutti, Bernard Stollar, Fabiola Escher, Monica Blondon, Katherine Aujesky, Susanne Nendaz, Mathieu Galetto-Lacour, Annick |
author_sort | Cerutti, Bernard |
collection | PubMed |
description | BACKGROUND: Little is known regarding the psychometric properties of computerized long-menu formats in comparison to classic formats. We compared single-best-answer (Type A) and long-menu formats using identical question stems during the computer-based, summative, intermediate clinical-clerkship exams for nine disciplines. METHODS: In this randomised sequential trial, we assigned the examinees for every summative exam to either the Type A or long-menu format (four different experimental questions, otherwise identical). The primary outcome was the power of discrimination. The study was carried out at the Faculty of Medicine, University of Geneva, Switzerland, and included all the students enrolled for the exams that were part of the study. Examinees were surveyed about the long-menu format at the end of the trial. RESULTS: The trial was stopped for futility (p = 0.7948) after 22 exams including 88 experimental items. The long-menu format had a similar discriminatory power but was more difficult than the Type A format (71.45% vs 77.80%; p = 0.0001). Over half of the options (54.4%) chosen by the examinees in long-menu formats were not proposed as distractors in the Type A formats. Most examinees agreed that their reasoning strategy was different. CONCLUSIONS: In a non-selected population of examinees taking summative exams, long-menu questions have the same discriminatory power as classic Type A questions, but they are slightly more difficult. They are perceived to be closer to real practice, which could have a positive educational impact. We would recommend their use in the final years of the curriculum, within realistic key-feature problems, to assess clinical reasoning and patient management skills. |
format | Online Article Text |
id | pubmed-6582573 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-65825732019-06-26 Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial Cerutti, Bernard Stollar, Fabiola Escher, Monica Blondon, Katherine Aujesky, Susanne Nendaz, Mathieu Galetto-Lacour, Annick BMC Med Educ Research Article BACKGROUND: Little is known regarding the psychometric properties of computerized long-menu formats in comparison to classic formats. We compared single-best-answer (Type A) and long-menu formats using identical question stems during the computer-based, summative, intermediate clinical-clerkship exams for nine disciplines. METHODS: In this randomised sequential trial, we assigned the examinees for every summative exam to either the Type A or long-menu format (four different experimental questions, otherwise identical). The primary outcome was the power of discrimination. The study was carried out at the Faculty of Medicine, University of Geneva, Switzerland, and included all the students enrolled for the exams that were part of the study. Examinees were surveyed about the long-menu format at the end of the trial. RESULTS: The trial was stopped for futility (p = 0.7948) after 22 exams including 88 experimental items. The long-menu format had a similar discriminatory power but was more difficult than the Type A format (71.45% vs 77.80%; p = 0.0001). Over half of the options (54.4%) chosen by the examinees in long-menu formats were not proposed as distractors in the Type A formats. Most examinees agreed that their reasoning strategy was different. CONCLUSIONS: In a non-selected population of examinees taking summative exams, long-menu questions have the same discriminatory power as classic Type A questions, but they are slightly more difficult. They are perceived to be closer to real practice, which could have a positive educational impact. We would recommend their use in the final years of the curriculum, within realistic key-feature problems, to assess clinical reasoning and patient management skills. BioMed Central 2019-06-18 /pmc/articles/PMC6582573/ /pubmed/31215430 http://dx.doi.org/10.1186/s12909-019-1651-6 Text en © The Author(s). 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Article Cerutti, Bernard Stollar, Fabiola Escher, Monica Blondon, Katherine Aujesky, Susanne Nendaz, Mathieu Galetto-Lacour, Annick Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title | Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title_full | Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title_fullStr | Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title_full_unstemmed | Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title_short | Comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
title_sort | comparison of long-menu and single-best-answer multiple choice questions in computer-based summative assessments: a randomised controlled trial |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6582573/ https://www.ncbi.nlm.nih.gov/pubmed/31215430 http://dx.doi.org/10.1186/s12909-019-1651-6 |
work_keys_str_mv | AT ceruttibernard comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT stollarfabiola comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT eschermonica comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT blondonkatherine comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT aujeskysusanne comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT nendazmathieu comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial AT galettolacourannick comparisonoflongmenuandsinglebestanswermultiplechoicequestionsincomputerbasedsummativeassessmentsarandomisedcontrolledtrial |