When Stars Do Not Align: Overall Hospital Quality Star Ratings and the Volume-Outcome Association

BACKGROUND: Volume-outcome relationships are well established for coronary artery bypass grafting and total joint arthroplasty surgery. Although the U.S. Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings program includes outcome quality measures for these procedures, these outcome quality measures are not counted toward the star ratings for low-volume hospitals. We sought to assess whether excluding low-volume hospitals from surgical quality measures with known volume-outcome relationships affects the star ratings. METHODS: We identified quality measures used in CMS’s star ratings that are related to surgical procedures with a known volume-outcome relationship and tested for the presence of the volume-outcome association for each of these measures. We then imputed missing values for low-volume hospitals for each measure and otherwise identically repeated the CMS calculations in order to assess the percentages of hospitals with the same, better, or worse ratings. RESULTS: Among the measures used to calculate star ratings, we identified 4 quality measures (2 related to coronary artery bypass grafting and 2 related to total joint arthroplasty) with known volume-outcome relationships that were excluded from the calculations of the star ratings for low-volume hospitals. We confirmed a volume-outcome association in the CMS data for all 4 measures. When total joint arthroplasty complications were imputed for low-volume hospitals and then included in the calculation of the star ratings, over one-third of hospitals received a different rating; both low-volume and other hospitals were more often hurt than helped. Imputing the other 3 quality measures among low-volume hospitals left the ratings unchanged. CONCLUSIONS: The CMS star ratings do not fully represent the risks of undergoing procedures at low-volume hospitals, potentially misrepresent quality across facilities, and hence are of uncertain utility to consumers.