Cargando…

Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

Purpose  Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful reside...

Descripción completa

Detalles Bibliográficos
Autores principales: Gudgel, Brett M., Melson, Andrew T., Dvorak, Justin, Ding, Kai, Siatkowski, R. Michael
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Thieme Medical Publishers, Inc. 2021
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9928014/
https://www.ncbi.nlm.nih.gov/pubmed/37388830
http://dx.doi.org/10.1055/s-0041-1733932
_version_ 1784888566845800448
author Gudgel, Brett M.
Melson, Andrew T.
Dvorak, Justin
Ding, Kai
Siatkowski, R. Michael
author_facet Gudgel, Brett M.
Melson, Andrew T.
Dvorak, Justin
Ding, Kai
Siatkowski, R. Michael
author_sort Gudgel, Brett M.
collection PubMed
description Purpose  Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS). Design  Retrospective analysis of application characteristics versus residency performance was conducted. Participants  Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study. Methods  Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS. Main Outcome Measures  Characteristics from residency applications that correlate with resident performance were the primary outcome of this study. Results  Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance. Conclusions  Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.
format Online
Article
Text
id pubmed-9928014
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Thieme Medical Publishers, Inc.
record_format MEDLINE/PubMed
spelling pubmed-99280142023-06-29 Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency Gudgel, Brett M. Melson, Andrew T. Dvorak, Justin Ding, Kai Siatkowski, R. Michael J Acad Ophthalmol (2017) Purpose  Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS). Design  Retrospective analysis of application characteristics versus residency performance was conducted. Participants  Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study. Methods  Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS. Main Outcome Measures  Characteristics from residency applications that correlate with resident performance were the primary outcome of this study. Results  Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance. Conclusions  Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking. Thieme Medical Publishers, Inc. 2021-11-10 /pmc/articles/PMC9928014/ /pubmed/37388830 http://dx.doi.org/10.1055/s-0041-1733932 Text en The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. ( https://creativecommons.org/licenses/by-nc-nd/4.0/ ) https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License, which permits unrestricted reproduction and distribution, for non-commercial purposes only; and use and reproduction, but not distribution, of adapted material for non-commercial purposes only, provided the original work is properly cited.
spellingShingle Gudgel, Brett M.
Melson, Andrew T.
Dvorak, Justin
Ding, Kai
Siatkowski, R. Michael
Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title_full Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title_fullStr Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title_full_unstemmed Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title_short Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency
title_sort correlation of ophthalmology residency application characteristics with subsequent performance in residency
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9928014/
https://www.ncbi.nlm.nih.gov/pubmed/37388830
http://dx.doi.org/10.1055/s-0041-1733932
work_keys_str_mv AT gudgelbrettm correlationofophthalmologyresidencyapplicationcharacteristicswithsubsequentperformanceinresidency
AT melsonandrewt correlationofophthalmologyresidencyapplicationcharacteristicswithsubsequentperformanceinresidency
AT dvorakjustin correlationofophthalmologyresidencyapplicationcharacteristicswithsubsequentperformanceinresidency
AT dingkai correlationofophthalmologyresidencyapplicationcharacteristicswithsubsequentperformanceinresidency
AT siatkowskirmichael correlationofophthalmologyresidencyapplicationcharacteristicswithsubsequentperformanceinresidency