Use of electronic medical records in development and validation of risk prediction models of hospital readmission: systematic review

OBJECTIVE: To provide focused evaluation of predictive modeling of electronic medical record (EMR) data to predict 30 day hospital readmission. DESIGN: Systematic review. DATA SOURCE: Ovid Medline, Ovid Embase, CINAHL, Web of Science, and Scopus from January 2015 to January 2019. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: All studies of predictive models for 28 day or 30 day hospital readmission that used EMR data. OUTCOME MEASURES: Characteristics of included studies, methods of prediction, predictive features, and performance of predictive models. RESULTS: Of 4442 citations reviewed, 41 studies met the inclusion criteria. Seventeen models predicted risk of readmission for all patients and 24 developed predictions for patient specific populations, with 13 of those being developed for patients with heart conditions. Except for two studies from the UK and Israel, all were from the US. The total sample size for each model ranged between 349 and 1 195 640. Twenty five models used a split sample validation technique. Seventeen of 41 studies reported C statistics of 0.75 or greater. Fifteen models used calibration techniques to further refine the model. Using EMR data enabled final predictive models to use a wide variety of clinical measures such as laboratory results and vital signs; however, use of socioeconomic features or functional status was rare. Using natural language processing, three models were able to extract relevant psychosocial features, which substantially improved their predictions. Twenty six studies used logistic or Cox regression models, and the rest used machine learning methods. No statistically significant difference (difference 0.03, 95% confidence interval −0.0 to 0.07) was found between average C statistics of models developed using regression methods (0.71, 0.68 to 0.73) and machine learning (0.74, 0.71 to 0.77). CONCLUSIONS: On average, prediction models using EMR data have better predictive performance than those using administrative data. However, this improvement remains modest. Most of the studies examined lacked inclusion of socioeconomic features, failed to calibrate the models, neglected to conduct rigorous diagnostic testing, and did not discuss clinical impact.