Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

OBJECTIVES: Efficient and accurate emergency department (ED) triage is critical to prioritize the sickest patients and manage department flow. We explored the use of electronic health record data and advanced predictive analytics to improve triage performance. METHODS: Using a data set of over 5 million ED encounters of patients 18 years and older across 21 EDs from 2016 to 2020, we derived triage models using deep learning to predict 2 outcomes: hospitalization (primary outcome) and fast‐track eligibility (exploratory outcome), defined as ED discharge with <2 resource types used (eg, laboratory or imaging studies) and no critical events (eg, resuscitative medications use or intensive care unit [ICU] admission). We report area under the receiver operator characteristic curve (AUC) and 95% confidence intervals (CI) for models using (1) triage variables alone (demographics and vital signs), (2) triage nurse clinical assessment alone (unstructured notes), and (3) triage variables plus clinical assessment for each prediction target. RESULTS: We found 12.7% of patients were hospitalized (n = 673,659) and 37.0% were fast‐track eligible (n = 1,966,615). The AUC was lowest for models using triage variables alone: AUC 0.77 (95% CI 0.77–0.78) and 0.70 (95% CI 0.70–0.71) for hospitalization and fast‐track eligibility, respectively, and highest for models incorporating clinical assessment with triage variables for both hospitalization and fast‐track eligibility: AUC 0.87 (95% CI 0.87–0.87) for both prediction targets. CONCLUSION: Our findings highlight the potential to use advanced predictive analytics to accurately predict key ED triage outcomes. Predictive accuracy was optimized when clinical assessments were added to models using simple structured variables alone.