Validation of the treatment identification strategy of the HEDIS addiction quality measures: concordance with medical record review

BACKGROUND: Strategies to accurately identify the occurrence of specific health care events in administrative data is central to many quality improvement and research efforts. Many health care quality measures have treatment identification strategies based on diagnosis and procedure codes - an approach that is inexpensive and feasible but usually of unknown validity. In this study, we examined if the diagnosis/procedure code combinations used in the 2006 HEDIS Initiation and Engagement quality measures to identify instances of addiction treatment have high concordance with documentation of addiction treatment in clinical progress notes. METHODS: Four type of records were randomly sampled from VHA electronic medical data: (a) Outpatient records from a substance use disorder (SUD) specialty clinic with a HEDIS-qualified substance use disorder (SUD) diagnosis/CPT code combination (n = 700), (b) Outpatient records from a non-SUD setting with a HEDIS-qualified SUD diagnosis/CPT code combination (n = 592), (c) Specialty SUD Inpatient/residential records that included a SUD diagnosis (n = 700), and (d) Non-SUD specialty Inpatient/residential records that included a SUD diagnosis (n = 700). Clinical progress notes for the sampled records were extracted and two raters classified each as documenting or not documenting addiction treatment. Rates of concordance between the HEDIS addiction treatment identification strategy and the raters' judgments were calculated for each record type. RESULTS: Within SUD outpatient clinics and SUD inpatient specialty units, 92% and 98% of sampled records had chart evidence of addiction treatment. Of outpatient encounters with a qualifying diagnosis/procedure code combination outside of SUD clinics, 63% had chart evidence of addiction treatment. Within non-SUD specialty inpatient units, only 46% of sampled records had chart evidence of addiction treatment. CONCLUSIONS: For records generated in SUD specialty settings, the HEDIS strategy of identifying SUD treatment with diagnosis and procedure codes has a high concordance with chart review. The concordance rate outside of SUD specialty settings is much lower and highly variable between facilities. Therefore, some patients may be counted as meeting the 2006 HEDIS Initiation and Engagement criteria without having received the specified amount (or any) addiction treatment.