Automated Detection of Healthcare Associated Infections: External Validation and Updating of a Model for Surveillance of Drain-Related Meningitis

OBJECTIVE: Automated surveillance of healthcare-associated infections can improve efficiency and reliability of surveillance. The aim was to validate and update a previously developed multivariable prediction model for the detection of drain-related meningitis (DRM). DESIGN: Retrospective cohort study using traditional surveillance by infection control professionals as reference standard. PATIENTS: Patients receiving an external cerebrospinal fluid drain, either ventricular (EVD) or lumbar (ELD) in a tertiary medical care center. Children, patients with simultaneous drains, <1 day of follow-up or pre-existing meningitis were excluded leaving 105 patients in validation set (2010–2011) and 653 in updating set (2004–2011). METHODS: For validation, the original model was applied. Discrimination, classification and calibration were assessed. For updating, data from all available years was used to optimally re-estimate coefficients and determine whether extension with new predictors is necessary. The updated model was validated and adjusted for optimism (overfitting) using bootstrapping techniques. RESULTS: In model validation, the rate of DRM was 17.4/1000 days at risk. All cases were detected by the model. The area under the ROC curve was 0.951. The positive predictive value was 58.8% (95% CI 40.7–75.4) and calibration was good. The revised model also includes Gram stain results. Area under the ROC curve after correction for optimism was 0.963 (95% CI 0.953– 0.974). Group-level prediction was adequate. CONCLUSIONS: The previously developed multivariable prediction model maintains discriminatory power and calibration in an independent patient population. The updated model incorporates all available data and performs well, also after elaborate adjustment for optimism.