Impact of the final adjective in the Medical Student Performance Evaluation on determination of applicant desirability

Background: The Medical Student Performance Evaluation (MSPE) is a primary source of information used by residency programs in their selection of trainees. The MSPE contains a narrative description of the applicant’s performance during medical school. In 2002, the Association of American Medical Colleges’ guideline for preparation of the MSPE recommended inclusion of a comparative summative assessment of the student’s overall performance relative to his/her peers (final adjective). Objective: We hypothesize that the inclusion of a final adjective in the MSPE affects a reviewer’s assessment of the applicant’s desirability more than the narrative description of performance and designed a study to evaluate this hypothesis. Design: Fifty-six faculty members from the Departments of Pediatrics and Medicine with experience reviewing MSPEs as part of the intern selection process reviewed two pairs of mock MSPE letters. In each pair, the narrative in one letter was superior to that in the other. Two final adjectives describing relative class ranks were created. Each subject was first presented with a pair of letters with mismatched final adjective (study), i.e., the letter with the stronger narrative was presented with the weaker final adjective and vice versa. The subject was then presented with a second pair of letters without final adjectives (control). Subjects ranked the relative desirability of the two applicants in each pair. Results: The proportion of rankings congruent with the strength of the narratives under study and control conditions were compared. Subjects were significantly less likely to rank the applicants congruent with the strength of the narratives when the strength of the final adjectives conflicted with the strength of the narrative; 42.9% of study letters were ranked congruent with the narrative versus 82.1% of controls (p = 0.0001). Conclusion: The MSPE final adjective had a greater impact than the narrative description of performance on the determination of applicant desirability. Abbreviations: MSPE: Medical Student Performance Evaluation; AAMC: Association of American Medical Colleges; BCM: Baylor College of Medicine