Validation of acute physiologic and chronic health evaluation II scoring system software developed at The Aga Khan University, Pakistan

OBJECTIVE: To assess the predictive performance of Acute Physiologic and Chronic Health Evaluation II (APACHE II) software available on the hospital intranet and analyze interrater reliability of calculating the APACHE II score by the gold standard manual method or automatically using the software. MATERIALS AND METHODS: An expert scorer not involved in the data collection had calculated APACHE II score of 213 patients admitted to surgical Intensive Care Unit using the gold standard manual method for a previous study performed in the department. The same data were entered into the computer software available on the hospital intranet (http://intranet/apacheii) to recalculate the APACHE II score automatically along with the predicted mortality. Receiver operating characteristic curve (ROC), Hosmer-Lemeshow goodness-of-fit statistical test and Pearson's correlation coefficient was computed. RESULTS: The 213 patients had an average APACHE II score of 17.20 ± 8.24, the overall mortality rate was 32.8% and standardized mortality ratio was 1.00. The area under the ROC curve of 0.827 was significantly >0.5 (P < 0.01) and had confidence interval of 0.77-0.88. The goodness-of-fit test showed a good calibration (H = 5.46, P = 0.71). Interrater reliability using Pearson's product moment correlations demonstrated a strong positive relationship between the computer and the manual expert scorer (r = 0.98, P = 0.0005). CONCLUSION: APACHE II software available on the hospital's intranet has satisfactory calibration and discrimination and interrater reliability is good when compared with the gold standard manual method.