Cargando…
An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study
BACKGROUND: Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students’ performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinic...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7240440/ https://www.ncbi.nlm.nih.gov/pubmed/32374267 http://dx.doi.org/10.2196/15444 |
_version_ | 1783536883139084288 |
---|---|
author | Staziaki, Pedro Vinícius Sarangi, Rutuparna Parikh, Ujas N Brooks, Jeffrey G LeBedis, Christina Alexandra Shaffer, Kitt |
author_facet | Staziaki, Pedro Vinícius Sarangi, Rutuparna Parikh, Ujas N Brooks, Jeffrey G LeBedis, Christina Alexandra Shaffer, Kitt |
author_sort | Staziaki, Pedro Vinícius |
collection | PubMed |
description | BACKGROUND: Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students’ performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE: This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students’ radiology knowledge and the reproducibility of this assessment tool. METHODS: We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students’ clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS: A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (−0.02; P=.48), adjusting for the medical school year. CONCLUSIONS: Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested. |
format | Online Article Text |
id | pubmed-7240440 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-72404402020-06-01 An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study Staziaki, Pedro Vinícius Sarangi, Rutuparna Parikh, Ujas N Brooks, Jeffrey G LeBedis, Christina Alexandra Shaffer, Kitt JMIR Med Educ Original Paper BACKGROUND: Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students’ performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE: This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students’ radiology knowledge and the reproducibility of this assessment tool. METHODS: We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students’ clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS: A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (−0.02; P=.48), adjusting for the medical school year. CONCLUSIONS: Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested. JMIR Publications 2020-05-06 /pmc/articles/PMC7240440/ /pubmed/32374267 http://dx.doi.org/10.2196/15444 Text en ©Pedro Vinícius Staziaki, Rutuparna Sarangi, Ujas N Parikh, Jeffrey G Brooks, Christina Alexandra LeBedis, Kitt Shaffer. Originally published in JMIR Medical Education (http://mededu.jmir.org), 06.05.2020. https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on http://mededu.jmir.org/, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Staziaki, Pedro Vinícius Sarangi, Rutuparna Parikh, Ujas N Brooks, Jeffrey G LeBedis, Christina Alexandra Shaffer, Kitt An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title | An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title_full | An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title_fullStr | An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title_full_unstemmed | An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title_short | An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study |
title_sort | objective structured clinical examination for medical student radiology clerkships: reproducibility study |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7240440/ https://www.ncbi.nlm.nih.gov/pubmed/32374267 http://dx.doi.org/10.2196/15444 |
work_keys_str_mv | AT staziakipedrovinicius anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT sarangirutuparna anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT parikhujasn anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT brooksjeffreyg anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT lebedischristinaalexandra anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT shafferkitt anobjectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT staziakipedrovinicius objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT sarangirutuparna objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT parikhujasn objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT brooksjeffreyg objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT lebedischristinaalexandra objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy AT shafferkitt objectivestructuredclinicalexaminationformedicalstudentradiologyclerkshipsreproducibilitystudy |