The prognostic accuracy evaluation of SAPS 3, SOFA and APACHE II scores for mortality prediction in the surgical ICU: an external validation study and decision-making analysis

BACKGROUND: The early postoperative period is critical for surgical patients. SOFA, SAPS 3 and APACHE II are prognostic scores widely used to predict mortality in ICU patients. This study aimed to evaluate these index tests for their prognostic accuracy for intra-ICU and in-hospital mortalities as target conditions in patients admitted to ICU after urgent or elective surgeries and to test whether they aid in decision-making. The process comprised the assessment of discrimination through analysis of the areas under the receiver operating characteristic curves and calibration of the prognostic models for the target conditions. After, the clinical relevance of applying them was evaluated through the measurement of the net benefit of their use in the clinical decision. RESULTS: Index tests were found to discriminate regular for both target conditions with a poor calibration (C statistics—intra-ICU mortality AUROCs: APACHE II 0.808, SAPS 3 0.821 and SOFA 0.797/in-hospital mortality AUROCs: APACHE II 0.772, SAPS 3 0.790 and SOFA 0.742). Calibration assessment revealed a weak correlation between the observed and expected number of cases in several thresholds of risk, calculated by each model, for both tested outcomes. The net benefit analysis showed that all score’s aggregate value in the clinical decision when the calculated probabilities of death ranged between 10 and 40%. CONCLUSIONS: In this study, we observed that the tested ICU prognostic scores are fair tools for intra-ICU and in-hospital mortality prediction in a cohort of postoperative surgical patients. Also, they may have some potential to be used as ancillary data to support decision-making by physicians and families regarding the level of therapeutic investment and palliative care. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s13613-019-0488-9) contains supplementary material, which is available to authorized users.