Counterintuitive results from observational data: a case study and discussion

OBJECTIVE: To explore the issue of counterintuitive data via analysis of a representative case in which the data obtained was unexpected and inconsistent with current knowledge. We then discuss the issue of counterintuitive data while developing a framework for approaching such findings. DESIGN: The case study is a retrospective analysis of a cohort of coronary artery bypass graft (CABG) patients. Regression was used to examine the association between perceived pain in the intensive care unit (ICU) and selected outcomes. SETTING: Medical Information Mart for Intensive Care-III, a publicly available, de-identified critical care patient database. PARTICIPANTS: 844 adult patients from the database who underwent CABG surgery and were extubated within 24 hours after ICU admission. OUTCOMES: 30 day mortality, 1 year mortality and hospital length of stay (LOS). RESULTS: Increased pain levels were found to be significantly associated with reduced mortality at 30 days and 1 year, and shorter hospital LOS. A one-point increase in mean pain level was found to be associated with a reduction in the odds of 30 day and 1 year mortality by a factor of 0.457 (95% CI 0.304 to 0.687, p<0.01) and 0.710 (95% CI 0.571 to 0.881, p<0.01) respectively, and a 0.916 (95% CI −1.159 to –0.673, p<0.01) day decrease in hospital LOS. CONCLUSION: The finding of an association between increased pain and improved outcomes was unexpected and clinically counterintuitive. In an increasingly digitised age of medical big data, such results are likely to become more common. The reliability of such counterintuitive results must be carefully examined. We suggest several issues to consider in this analytic process. If the data is determined to be valid, consideration must then be made towards alternative explanations for the counterintuitive results observed. Such results may in fact indicate that current clinical knowledge is incomplete or not have been firmly based on empirical evidence and function to inspire further research into the factors involved.