Cargando…
Fairness and objectivity of a multiple scenario objective structured clinical examination
Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led...
Autores principales: | , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
German Medical Science GMS Publishing House
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6545613/ https://www.ncbi.nlm.nih.gov/pubmed/31211221 http://dx.doi.org/10.3205/zma001234 |
_version_ | 1783423408710615040 |
---|---|
author | Spanke, Johannes Raus, Christina Haase, Annekathrin Angelow, Aniela Ludwig, Fabian Weckmann, Gesine Schmidt, Carsten Oliver Chenot, Jean-Francois |
author_facet | Spanke, Johannes Raus, Christina Haase, Annekathrin Angelow, Aniela Ludwig, Fabian Weckmann, Gesine Schmidt, Carsten Oliver Chenot, Jean-Francois |
author_sort | Spanke, Johannes |
collection | PubMed |
description | Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE (MS-OSCE) where all students had to manage the same chief complaint at a station but it’s underlying scenarios being randomly changed during students’ rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios. Methods: We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on students’ grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model. Results: A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach’s alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students’ skills. Conclusions: The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students. |
format | Online Article Text |
id | pubmed-6545613 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | German Medical Science GMS Publishing House |
record_format | MEDLINE/PubMed |
spelling | pubmed-65456132019-06-17 Fairness and objectivity of a multiple scenario objective structured clinical examination Spanke, Johannes Raus, Christina Haase, Annekathrin Angelow, Aniela Ludwig, Fabian Weckmann, Gesine Schmidt, Carsten Oliver Chenot, Jean-Francois GMS J Med Educ Article Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE (MS-OSCE) where all students had to manage the same chief complaint at a station but it’s underlying scenarios being randomly changed during students’ rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios. Methods: We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on students’ grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model. Results: A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach’s alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students’ skills. Conclusions: The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students. German Medical Science GMS Publishing House 2019-05-16 /pmc/articles/PMC6545613/ /pubmed/31211221 http://dx.doi.org/10.3205/zma001234 Text en Copyright © 2019 Spanke et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/. |
spellingShingle | Article Spanke, Johannes Raus, Christina Haase, Annekathrin Angelow, Aniela Ludwig, Fabian Weckmann, Gesine Schmidt, Carsten Oliver Chenot, Jean-Francois Fairness and objectivity of a multiple scenario objective structured clinical examination |
title | Fairness and objectivity of a multiple scenario objective structured clinical examination |
title_full | Fairness and objectivity of a multiple scenario objective structured clinical examination |
title_fullStr | Fairness and objectivity of a multiple scenario objective structured clinical examination |
title_full_unstemmed | Fairness and objectivity of a multiple scenario objective structured clinical examination |
title_short | Fairness and objectivity of a multiple scenario objective structured clinical examination |
title_sort | fairness and objectivity of a multiple scenario objective structured clinical examination |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6545613/ https://www.ncbi.nlm.nih.gov/pubmed/31211221 http://dx.doi.org/10.3205/zma001234 |
work_keys_str_mv | AT spankejohannes fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT rauschristina fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT haaseannekathrin fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT angelowaniela fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT ludwigfabian fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT weckmanngesine fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT schmidtcarstenoliver fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination AT chenotjeanfrancois fairnessandobjectivityofamultiplescenarioobjectivestructuredclinicalexamination |