Comparison of Consumer Rankings With Centers for Medicare & Medicaid Services Five-Star Rankings of Nursing Homes

IMPORTANCE: The Centers for Medicare & Medicaid Services (CMS) Five-Star measure for nursing homes is designed with input from expert panels about the importance of multiple quality indicators. Consumers may assign different values to these indicators, creating different 5-star ratings. OBJECTIVE: To compare nursing homes’ rankings based on the CMS Five-Star measure with rankings based on consumers’ judgment about the importance of the same quality indicators. DESIGN, SETTING, AND PARTICIPANTS: In this quality improvement study, CMS Five-Star data were linked with a measure calculated from CMS quality indicators and consumer values obtained from a national survey. Data covered the last quarter of 2016 and the first three quarters of 2017. The study included 10 676 nursing homes, comprising 69.8% of those with reported Five-Star measures. The national survey included adults, either nursing home residents or their family members who reported being familiar with the quality of care their relative received. Data analysis was performed from January 2019 to February 2020. MAIN OUTCOMES AND MEASURES: The contingent valuation method was administered via the survey to obtain consumers’ relative values of the quality indicators, and statistical analyses were used to create the contingent valuation measure. Agreement in nursing home rankings was assessed using the Five-Star measure, which is based on weights developed by expert panels, compared with rankings based on the contingent valuation measure. RESULTS: Among 10 676 study nursing homes with a mean (SD) of 119.4 (59.4) beds, 7845 (73.5%) were for profit, 6424 (61.8%) were part of a chain, and 8009 (75.0%) were urban. The 4310 survey respondents (mean [SD] age, 39.9 [15.6] years; 1143 [26.5%] men; 3448 [80%] white) included mostly family members (3879 participants [90.0%]). The Pearson correlation coefficient (0.65) and weighted κ statistics (0.48) indicated only moderate agreement between ranking of nursing homes’ performance by the 2 measures and disagreement on ranking for approximately one-half of the nursing homes. CONCLUSIONS AND RELEVANCE: Current nursing home report cards might not reflect consumers’ values and the relative importance consumers place on each of the quality indicators that compose the overall Five-Star measure. Quality report cards might be more relevant to consumers by augmenting the Five-Star measure with a measure reflecting consumers’ preferences. It is unknown whether these conclusions are generalizable to other report cards, such as Hospital and Home Health Compare, without conducting similar studies for these report cards.