Evaluating the Reliability of EHR-Generated Clinical Outcomes Reports: A Case Study

INTRODUCTION: Quality incentive programs, such as Meaningful Use, operate under the assumption that clinical quality measures can be reliably extracted from EHRs. Safety Net providers, particularly Federally Qualified Health Centers and Look-Alikes, tend to be high adopters of EHRs; however, recent reports have shown that only about 9% of FQHCs and Look-Alikes were demonstrating meaningful use as of 2013. Our experience working with the Crescent City Beacon Community (CCBC) found that many health centers relied on chart audits to report quality measures as opposed to electronically generating reports directly from their EHRs due to distrust in the data. This paper describes a step-by-step process for improving the reliability of data extracted from EHRs to increase reliability of quality measure reports, to support quality improvement, and to achieve alignment with national clinical quality reporting requirements. BACKGROUND: Lack of standardization in data capture and reporting within EHRs drives distrust in EHR-reported data. Practices or communities attempting to achieve standardization may look to CCBC’s experience for guidance on where to start and the level of resources required in order to execute a data standardization project. During the time of this data standardization project, CCBC was launching an HIE. Lack of trust in EHR data was a driver for distrust in the HIE data. METHODS: We present a case study where a five-step process was used to harmonize measures, reduce data errors, and increase trust in EHR clinical outcomes reports among a community of Safety Net providers using a common EHR. Primary outcomes were the incidence of reporting errors and the potential effect of error types on quality measure percentages. The activities and level of resources required to achieve these results were also documented by the CCBC program. FINDINGS: Implementation of a community-wide data reporting project resulted in measure harmonization, reduced reporting burden, and error reduction in EHR-generated clinical outcomes reporting across participating clinics over a nine-month period. Increased accuracy of clinical outcomes reports provided physicians and clinical care teams with better information to guide their decision-making around quality improvement planning. DISCUSSION: A number of challenges exist to achieving reliable population level quality reporting from EHRs at the practice, vendor, and community levels. Our experience demonstrates that quality measure reporting from EHRs is not a straightforward process, and it requires time and close collaboration between clinics and vendors to improve reliability of reports. Our experience found that practices valued the opportunity and step-wise process to validate their data locally (out of their EHRs) prior to reporting out of the HIE. CONCLUSION AND NEXT STEPS: Communities can achieve higher levels of confidence in quality measure reporting at the population level by establishing collaborative user groups that work with EHR vendors as partners and use technical assistance to build relationships and trust in EHR-generated reports. While this paper describes the first phase of our work around improving standardization and reliability of EHR reports, vendors should continue to explore modifications for improving data capture (at the front-end) via standardized data entry templates.