106 Neurosurgery Resident Feedback through Artificial-Intelligence

OBJECTIVES/GOALS: Surgical training is constrained by duty hour limits, bias, and a trial-and-error learning process. Surgeon skill variation is a healthcare system disparity that can impact patient outcomes. Incorporating validated, standardized assessment tools and machine learning (ML) algorithms may help to standardize and reduce bias in surgeon education. METHODS/STUDY POPULATION: To support assessment tool and ML algorithm development, we are curating an annotated video registry of neurosurgical procedures. Point-of-view video of resident and attending neurosurgeons performing craniotomies is recorded via an eye-tracking headset. A Delphi panel of neurosurgeons will review the video and determine which represent expert versus trainee performance. Neurosurgery attendings will be interviewed to provide descriptions of craniotomies which will be used to develop an assessment rubric. A Delphi panel will determine what rubric components should be maintained. New craniotomy videos will be viewed by attendings in a blinded fashion while completing the assessment rubric. An online feedback platform is being developed allowing residents to prospectively track assessment data. RESULTS/ANTICIPATED RESULTS: We anticipate development of an annotated, institutional video database featuring craniotomies performed by residents and attending neurosurgeons. Using a Delphi approach, we anticipate achieving consensus on which videos reflect expert versus trainee performance. We anticipate development of a novel craniotomy assessment rubric that is both valid and reliable. Our online feedback platform will allow prospective tracking of assessment data from multiple sources and enhanced transparency in the feedback process. The video registry and assessment data will enable development of novel ML algorithms able to recognize craniotomy segments and estimate operator skill. DISCUSSION/SIGNIFICANCE: Building a video registry of procedures, validated assessment tools, and a prototype feedback platform enables a pipeline for ML algorithm development. Together these tools will help to standardize and optimize resident education translating to earlier operative independence, improved patient safety, and reduced bias during surgical training.