Comparative Effectiveness of New Approaches to Improve Mortality Risk Models From Medicare Claims Data

IMPORTANCE: Risk adjustment models using claims-based data are central in evaluating health care performance. Although US Centers for Medicare & Medicaid Services (CMS) models apply well-vetted statistical approaches, recent changes in the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) coding system and advances in computational capabilities may provide an opportunity for enhancement. OBJECTIVE: To examine whether changes using already available data would enhance risk models and yield greater discrimination in hospital-level performance measures. DESIGN, SETTING, AND PARTICIPANTS: This comparative effectiveness study used ICD-9-CM codes from all Medicare fee-for-service beneficiary claims for hospitalizations for acute myocardial infarction (AMI), heart failure (HF), or pneumonia among patients 65 years and older from July 1, 2013, through September 30, 2015. Changes to current CMS mortality risk models were applied incrementally to patient-level models, and the best model was tested on hospital performance measures to model 30-day mortality. Analyses were conducted from April 19, 2018, to September 19, 2018. MAIN OUTCOMES AND MEASURES: The main outcome was all-cause death within 30 days of hospitalization for AMI, HF, or pneumonia, examined using 3 changes to current CMS mortality risk models: (1) incorporating present on admission coding to better exclude potential complications of care, (2) separating index admission diagnoses from those of the 12-month history, and (3) using ungrouped ICD-9-CM codes. RESULTS: There were 361 175 hospital admissions (mean [SD] age, 78.6 [8.4] years; 189 225 [52.4%] men) for AMI, 716 790 hospital admissions (mean [SD] age, 81.1 [8.4] years; 326 825 [45.6%] men) for HF, and 988 225 hospital admissions (mean [SD] age, 80.7 [8.6] years; 460 761 [46.6%] men) for pneumonia during the study; mean 30-day mortality rates were 13.8% for AMI, 12.1% for HF, and 16.1% for pneumonia. Each change to the models was associated with incremental gains in C statistics. The best model, incorporating all changes, was associated with significantly improved patient-level C statistics, from 0.720 to 0.826 for AMI, 0.685 to 0.776 for HF, and 0.715 to 0.804 for pneumonia. Compared with current CMS models, the best model produced wider predicted probabilities with better calibration and Brier scores. Hospital risk-standardized mortality rates had wider distributions, with more hospitals identified as good or bad performance outliers. CONCLUSIONS AND RELEVANCE: Incorporating present on admission coding and using ungrouped index and historical ICD-9-CM codes were associated with improved patient-level and hospital-level risk models for mortality compared with the current CMS models for all 3 conditions.