Cargando…
Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study
BACKGROUND: In-training examination (ITE) has been widely adopted as an assessment tool to measure residents' competency. We incorporated different formats of assessments into the emergency medicine (EM) residency training program to form a multimodal, multistation ITE. This study was conducted...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8959571/ https://www.ncbi.nlm.nih.gov/pubmed/35355591 http://dx.doi.org/10.3389/fmed.2022.840721 |
_version_ | 1784677186613018624 |
---|---|
author | Liu, Pin Chen, Shou-Yen Chang, Yu-Che Ng, Chip-Jin Chaou, Chung-Hsien |
author_facet | Liu, Pin Chen, Shou-Yen Chang, Yu-Che Ng, Chip-Jin Chaou, Chung-Hsien |
author_sort | Liu, Pin |
collection | PubMed |
description | BACKGROUND: In-training examination (ITE) has been widely adopted as an assessment tool to measure residents' competency. We incorporated different formats of assessments into the emergency medicine (EM) residency training program to form a multimodal, multistation ITE. This study was conducted to examine the cost and effectiveness of its different testing formats. METHODS: We conducted a longitudinal study in a tertiary teaching hospital in Taiwan. Nine EM residents were enrolled and followed for 4 years, and the biannual ITE scores were recorded and analyzed. Each ITE consisted of 8–10 stations and was categorized into four formats: multiple-choice question (MCQ), question and answer (QA), oral examination (OE), and high-fidelity simulation (HFS) formats. The learner satisfaction, validity, reliability, and costs were analyzed. RESULTS: 486 station scores were recorded during the 4 years. The numbers of MCQ, OE, QA, and HFS stations were 45 (9.26%), 90 (18.5%), 198 (40.7%), and 135 (27.8%), respectively. The overall Cronbach's alpha reached 0.968, indicating good overall internal consistency. The correlation with EM board examination was highest for HFS (ρ = 0.657). The average costs of an MCQ station, an OE station, and an HFS station were ~3, 14, and 21 times that of a QA station. CONCLUSIONS: Multi-dimensional assessment contributes to good reliability. HFS correlates best with the final training exam score but is also the most expensive format among ITEs. Increased testing domains with various formats improve ITE's overall reliability. Program directors must understand each test format's strengths and limitations to bring forth the best combination of exams under the local context. |
format | Online Article Text |
id | pubmed-8959571 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-89595712022-03-29 Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study Liu, Pin Chen, Shou-Yen Chang, Yu-Che Ng, Chip-Jin Chaou, Chung-Hsien Front Med (Lausanne) Medicine BACKGROUND: In-training examination (ITE) has been widely adopted as an assessment tool to measure residents' competency. We incorporated different formats of assessments into the emergency medicine (EM) residency training program to form a multimodal, multistation ITE. This study was conducted to examine the cost and effectiveness of its different testing formats. METHODS: We conducted a longitudinal study in a tertiary teaching hospital in Taiwan. Nine EM residents were enrolled and followed for 4 years, and the biannual ITE scores were recorded and analyzed. Each ITE consisted of 8–10 stations and was categorized into four formats: multiple-choice question (MCQ), question and answer (QA), oral examination (OE), and high-fidelity simulation (HFS) formats. The learner satisfaction, validity, reliability, and costs were analyzed. RESULTS: 486 station scores were recorded during the 4 years. The numbers of MCQ, OE, QA, and HFS stations were 45 (9.26%), 90 (18.5%), 198 (40.7%), and 135 (27.8%), respectively. The overall Cronbach's alpha reached 0.968, indicating good overall internal consistency. The correlation with EM board examination was highest for HFS (ρ = 0.657). The average costs of an MCQ station, an OE station, and an HFS station were ~3, 14, and 21 times that of a QA station. CONCLUSIONS: Multi-dimensional assessment contributes to good reliability. HFS correlates best with the final training exam score but is also the most expensive format among ITEs. Increased testing domains with various formats improve ITE's overall reliability. Program directors must understand each test format's strengths and limitations to bring forth the best combination of exams under the local context. Frontiers Media S.A. 2022-03-09 /pmc/articles/PMC8959571/ /pubmed/35355591 http://dx.doi.org/10.3389/fmed.2022.840721 Text en Copyright © 2022 Liu, Chen, Chang, Ng and Chaou. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Medicine Liu, Pin Chen, Shou-Yen Chang, Yu-Che Ng, Chip-Jin Chaou, Chung-Hsien Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title | Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title_full | Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title_fullStr | Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title_full_unstemmed | Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title_short | Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study |
title_sort | multimodal in-training examination in an emergency medicine residency training program: a longitudinal observational study |
topic | Medicine |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8959571/ https://www.ncbi.nlm.nih.gov/pubmed/35355591 http://dx.doi.org/10.3389/fmed.2022.840721 |
work_keys_str_mv | AT liupin multimodalintrainingexaminationinanemergencymedicineresidencytrainingprogramalongitudinalobservationalstudy AT chenshouyen multimodalintrainingexaminationinanemergencymedicineresidencytrainingprogramalongitudinalobservationalstudy AT changyuche multimodalintrainingexaminationinanemergencymedicineresidencytrainingprogramalongitudinalobservationalstudy AT ngchipjin multimodalintrainingexaminationinanemergencymedicineresidencytrainingprogramalongitudinalobservationalstudy AT chaouchunghsien multimodalintrainingexaminationinanemergencymedicineresidencytrainingprogramalongitudinalobservationalstudy |