Tracking operative autonomy and performance in otolaryngology training using smartphone technology: A single institution pilot study

BACKGROUND: In the era of duty hour restrictions, otolaryngology residents may not gain the operative experience necessary to function autonomously by the end of training. This study quantifies residents' autonomy during key indicator cases, defined by the Accreditation Council for Graduate Medical Education. STUDY DESIGN: Prospective cohort study. METHODS: Faculty and residents at a large academic institution were surveyed on the surgical autonomy trainees should achieve for otolaryngology key indicator surgeries at each training level. Residents and faculty used the mobile application “System for Improving and Measuring Procedural Learning” (SIMPL) between December 2017 and July 2018 to log trainees' operative autonomy during cases on a validated four‐level Zwisch scale, from “show and tell” to “supervision only.” RESULTS: The study included 40 participants (23 residents and 17 attendings). The survey response rate was 83%. In surveys, residents overestimated the autonomy PGY5 residents should achieve for parotidectomy, rhinoplasty, thyroid/parathyroidectomy, and airway procedures compared with faculty (P < .05). Using SIMPL, 833 evaluations were logged of which 253 were paired evaluations for key indicator cases. Comparing survey predictions with actual cases logged in SIMPL, residents and faculty overestimated the autonomy achieved by senior trainees performing mastoidectomy (PGY5, P < .05) and ethmoidectomy (PGY4/5, P < .05); both felt that senior residents should operate with between “passive help” and “supervision only” whereas residents actually had “passive help.” Residents overestimated their autonomy during rhinoplasty (PGY5, P = .017) and parotidectomy (PGY5, P = .007) while attendings accurately expected chief residents to have “passive help.” CONCLUSIONS: Resident surgical autonomy varies across otolaryngology procedures. Multicenter studies are needed to elucidate national trends. LEVEL OF EVIDENCE: 2