Comparison of Automated Sepsis Identification Methods and Electronic Health Record–based Sepsis Phenotyping: Improving Case Identification Accuracy by Accounting for Confounding Comorbid Conditions

To develop and evaluate a novel strategy that automates the retrospective identification of sepsis using electronic health record data. DESIGN: Retrospective cohort study of emergency department and in-hospital patient encounters from 2014 to 2018. SETTING: One community and two academic hospitals in Maryland. PATIENTS: All patients 18 years old or older presenting to the emergency department or admitted to any acute inpatient medical or surgical unit including patients discharged from the emergency department. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: From the electronic health record, 233,252 emergency department and inpatient encounters were identified. Patient data were used to develop and validate electronic health record–based sepsis phenotyping, an adaptation of “the Centers for Disease Control Adult Sepsis Event toolkit” that accounts for comorbid conditions when identifying sepsis patients. The performance of this novel system was then compared with 1) physician case review and 2) three other commonly used strategies using metrics of sensitivity and precision relative to sepsis billing codes, termed “billing code sensitivity” and “billing code predictive value.” Physician review of electronic health record–based sepsis phenotyping identified cases confirmed 79% as having sepsis; 88% were confirmed or had a billing code for sepsis; and 99% were confirmed, had a billing code, or received at least 4 days of antibiotics. At comparable billing code sensitivity (0.91; 95% CI, 0.88–0.93), electronic health record–based sepsis phenotyping had a higher billing code predictive value (0.32; 95% CI, 0.30–0.34) than either the Centers for Medicare and Medicaid Services Sepsis Core Measure (SEP-1) definition or the Sepsis-3 consensus definition (0.12; 95% CI, 0.11–0.13; and 0.07; 95% CI, 0.07–0.08, respectively). When compared with electronic health record–based sepsis phenotyping, Adult Sepsis Event had a lower billing code sensitivity (0.75; 95% CI, 0.72–0.78) and similar billing code predictive value (0.29; 95% CI, 0.26–0.31). Electronic health record–based sepsis phenotyping identified patients with higher in-hospital mortality and nearly one-half as many false-positive cases when compared with SEP-1 and Sepsis-3. CONCLUSIONS: By accounting for comorbid conditions, electronic health record–based sepsis phenotyping exhibited better performance when compared with other automated definitions of sepsis.