Template matching for benchmarking hospital performance in the veterans affairs healthcare system

Comparing hospital performance in a health system is traditionally done with multilevel regression models that adjust for differences in hospitals’ patient case-mix. In contrast, “template matching” compares outcomes of similar patients at different hospitals but has been used only in limited patient settings. Our objective was to test a basic template matching approach in the nationwide Veterans Affairs healthcare system (VA), compared with a more standard regression approach. We performed various simulations using observational data from VA electronic health records whereby we randomly assigned patients to “pseudo hospitals,” eliminating true hospital level effects. We randomly selected a representative template of 240 patients and matched 240 patients on demographic and physiological factors from each pseudo hospital to the template. We varied hospital performance for different simulations such that some pseudo hospitals negatively impacted patient mortality. Electronic health record data of 460,213 hospitalizations at 111 VA hospitals across the United States in 2015. We assessed 30-day mortality at each pseudo hospital and identified lowest quintile hospitals by template matching and regression. The regression model adjusted for predicted 30-day mortality (as a measure of illness severity). Regression identified the lowest quintile hospitals with 100% accuracy compared with 80.3% to 82.0% for template matching when systematic differences in 30-day mortality existed. The current standard practice of risk-adjusted regression incorporating patient-level illness severity was better able to identify lower-performing hospitals than the simplistic template matching algorithm.