Evaluating applicants to a new emergency medicine residency program: subjective assessment of applicant characteristics

BACKGROUND: Because of the Accreditation Council for Graduate Medical Education (ACGME) and the Residency Review Committee (RRC) approval timelines, new residency programs cannot use Electronic Residency Application Service (ERAS) during their first year of applicants. AIM: We sought to identify differences between program directors’ subjective ratings of applicants from an emergency medicine (EM) residency program’s first year (in which ERAS was not used) to their ratings of applicants the following year in which ERAS was used. METHOD: The University of Utah Emergency Medicine Residency Program received approval from the ACGME in 2004. Applicants for the entering class of 2005 (year 1) did not use ERAS, submitting a separate application, while those applying for the following year (year 2) used ERAS. Residency program directors rated applicants using subjective components of their applications, assigning scores on scales from 0–10 or 0–5 (10 or 5 = highest score) for select components of the application. We retrospectively reviewed and compared these ratings between the 2 years of applicants. RESULTS: A total of 130 and 458 prospective residents applied during year 1 and year 2, respectively. Applicants were similar in average scores for research (1.65 vs. 1.81, scale 0–5, p = 0.329) and volunteer work (5.31 vs. 5.56, scale 0–10, p = 0.357). Year 1 applicants received higher scores for their personal statement (3.21 vs. 2.22, scale 0–5, p < 0.001), letters of recommendation (7.0 vs. 5.94, scale 0–10, p < 0.001), dean’s letter (3.5 vs. 2.7, scale 1–5, p < 0.001), and in their potential contribution to class characteristics (4.64 vs. 3.34, scale 0–10, p < 0.001). CONCLUSION: While the number of applicants increased, the use of ERAS in a new residency program did not improve the overall subjective ratings of residency applicants. Year 1 applicants received higher scores for the written components of their applications and in their potential contributions to class characteristics.