Validation of deep-learning-based triage and acuity score using a large national dataset

AIM: Triage is important in identifying high-risk patients amongst many less urgent patients as emergency department (ED) overcrowding has become a national crisis recently. This study aims to validate that a Deep-learning-based Triage and Acuity Score (DTAS) identifies high-risk patients more accurately than existing triage and acuity scores using a large national dataset. METHODS: We conducted a retrospective observational cohort study using data from the Korean National Emergency Department Information System (NEDIS), which collected data on visits in real time from 151 EDs. The NEDIS data was split into derivation data (January 2014-June 2016) and validation data (July-December 2016). We also used data from the Sejong General Hospital (SGH) for external validation (January-December 2017). We predicted in-hospital mortality, critical care, and hospitalization using initial information of ED patients (age, sex, chief complaint, time from symptom onset to ED visit, arrival mode, trauma, initial vital signs and mental status as predictor variables). RESULTS: A total of 11,656,559 patients were included in this study. The primary outcome was in-hospital mortality. The Area Under the Receiver Operating Characteristic curve (AUROC) and Area Under the Precision and Recall Curve (AUPRC) of DTAS were 0.935 and 0.264. It significantly outperformed Korean triage and acuity score (AUROC:0.785, AUPRC:0.192), modified early warning score (AUROC:0.810, AUPRC:0.116), logistic regression (AUROC:0.903, AUPRC:0.209), and random forest (AUROC:0.910, AUPRC:0.179). CONCLUSION: Deep-learning-based Triage and Acuity Score predicted in-hospital mortality, critical care, and hospitalization more accurately than existing triages and acuity, and it was validated using a large, multicenter dataset.