Field methods in medical record abstraction: assessing the properties of comparative effectiveness estimates

BACKGROUND: Comparative effectiveness studies using Medicare claims data are vulnerable to treatment selection biases and supplemental data from a sample of patients has been recommended for examining the magnitude of this bias. Previous research using nationwide Medicare claims data has typically relied on the Medicare Current Beneficiary Survey (MCBS) for supplemental data. Because many important clinical variables for our specific research question are not available in the MCBS, we collected medical record data from a subsample of patients to assess the validity of assumptions and to aid in the interpretation of our estimates. This paper seeks to describe and document the process used to collect and validate this supplemental information. METHODS: Medicare claims data files for all patients with fee-for-service Medicare benefits who had an acute myocardial infarction (AMI) in 2007 or 2008 were obtained. Medical records were obtained and abstracted for a stratified subsample of 1,601 of these patients, using strata defined by claims-based measures of physician prescribing practices and drug treatment combinations. The abstraction tool was developed collaboratively by study clinicians and researchers, leveraging important elements from previously validated tools. RESULTS: Records for 2,707 AMI patients were requested from the admitting hospitals and 1,751 were received for an overall response rate of 65%; 1,601 cases were abstracted by trained personnel at a contracted firm. Data were collected with overall 96% inter-abstractor agreement across all variables. Some non-response bias was detected at the patient and facility level. CONCLUSION: Although Medicare claims data are a potentially powerful resource for conducting comparative effectiveness analyses, observational databases are vulnerable to treatment selection biases. This study demonstrates that it is feasible to abstract medical records for Medicare patients nationwide and collect high quality data, to design the sampling purposively to address specific research questions, and to more thoroughly evaluate the appropriateness of care delivered to AMI patients.