A method for measuring the effect of certified electronic health record technology on childhood immunization status scores among Medicaid managed care network providers

OBJECTIVE: To provide a methodology for estimating the effect of U.S.-based Certified Electronic Health Records Technology (CEHRT) implemented by primary care physicians (PCPs) on a Healthcare Effectiveness Data and Information Set (HEDIS) measure for childhood immunization delivery. MATERIALS AND METHODS: This study integrates multiple health care administrative data sources from 2010 through 2014, analyzed through an interrupted time series design and a hierarchical Bayesian model. We compared managed care physicians using CEHRT to propensity-score matched comparisons from network physicians who did not adopt CEHRT. Inclusion criteria for physicians using CEHRT included attesting to the Childhood Immunization Status clinical quality measure in addition to meeting “Meaningful Use” (MU) during calendar year 2013. We used a first-presence patient attribution approach to develop provider-specific immunization scores. RESULTS: We evaluated 147 providers using CEHRT, with 147 propensity-score matched providers selected from a pool of 1253 PCPs practicing in Maryland. The estimate for change in odds of increasing immunization rates due to CEHRT was 1.2 (95% credible set, 0.88–1.73). DISCUSSION: We created a method for estimating immunization quality scores using Bayesian modeling. Our approach required linking separate administrative data sets, constructing a propensity-score matched cohort, and using first-presence, claims-based childhood visit information for patient attribution. In the absence of integrated data sets and precise and accurate patient attribution, this is a reusable method for researchers and health system administrators to estimate the impact of health information technology on individual, provider-level, process-based, though outcomes-focused, quality measures. CONCLUSION: This research has provided evidence for using Bayesian analysis of propensity-score matched provider populations to estimate the impact of CEHRT on outcomes-based quality measures such as childhood immunization delivery.