Retrospective derivation and validation of a search algorithm to identify extubation failure in the intensive care unit

BACKGROUND: Development and validation of automated electronic medical record (EMR) search strategies is important in identifying extubation failure in the intensive care unit (ICU). We developed and validated an automated search algorithm (strategy) for extubation failure in critically ill patients. METHODS: The EMR search algorithm was created through sequential steps with keywords applied to an institutional EMR database. The search strategy was derived retrospectively through secondary analysis of a 100-patient subset from the 978 patient cohort admitted to a neurological ICU from January 1, 2002, through December 31, 2011(derivation subset). It was, then, validated against an additional 100-patient subset (validation subset). Sensitivity, specificity, negative and positive predictive values of the automated search algorithm were compared with a manual medical record review (the reference standard) for data extraction of extubation failure. RESULTS: In the derivation subset of 100 random patients, the initial automated electronic search strategy achieved a sensitivity of 85% (95% CI, 56%-97%) and a specificity of 95% (95% CI, 87%-98%). With refinements in the search algorithm, the final sensitivity was 93% (95% CI, 64%-99%) and specificity increased to 100% (95% CI, 95%-100%) in this subset. In validation of the algorithm through a separate 100 random patient subset, the reported sensitivity and specificity were 94% (95% CI, 69%-99%) and 98% (95% CI, 92%-99%) respectively. CONCLUSIONS: Use of electronic search algorithms allows for correct extraction of extubation failure in the ICU, with high degrees of sensitivity and specificity. Such search algorithms are a reliable alternative to manual chart review for identification of extubation failure.