Using Statistical and Machine Learning Methods to Evaluate the Prognostic Accuracy of SIRS and qSOFA

OBJECTIVES: The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. METHODS: This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and naïve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. RESULTS: Of 132,704 eligible patient visits, 14% died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95% confidence interval [CI], 2.96–3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95% CI, 1.18–1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63). For qSOFA, the sensitivity and specificity were DT = 0.39, LR = 0.64, NB = 0.62 and DT = 0.89, LR = 0.63, NB = 0.66, respectively. For SIRS, the sensitivity and specificity were DT = 0.46, LR = 0.62, NB = 0.62 and DT = 0.70, LR = 0.59, NB = 0.58, respectively. CONCLUSIONS: The evidences suggest that qSOFA is a better diagnostic criteria than SIRS. The low sensitivity of qSOFA can be improved by carefully selecting the threshold to translate the predicted probabilities into labels. These findings can guide healthcare providers in selecting risk-stratification measures for patients presenting to an ED with sepsis.