Methods for evaluating adverse drug event preventability in emergency department patients

BACKGROUND: There is a high degree of variability in assessing the preventability of adverse drug events, limiting the ability to compare rates of preventable adverse drug events across different studies. We compared three methods for determining preventability of adverse drug events in emergency department patients and explored their strengths and weaknesses. METHODS: This mixed-methods study enrolled emergency department patients diagnosed with at least one adverse drug event from three prior prospective studies. A clinical pharmacist and physician reviewed the medical and research records of all patients, and independently rated each event’s preventability using a “best practice-based” approach, an “error-based” approach, and an “algorithm-based” approach. Raters discussed discordant ratings until reaching consensus. We assessed the inter-rater agreement between clinicians using the same assessment method, and between different assessment methods using Cohen’s kappa with 95% confidence intervals (95% CI). Qualitative researchers observed discussions, took field notes, and reviewed free text comments made by clinicians in a “comment” box in the data collection form. We developed a coding structure and iteratively analyzed qualitative data for emerging themes regarding the application of each preventability assessment method using NVivo. RESULTS: Among 1356 adverse drug events, a best practice-based approach rated 64.1% (95% CI: 61.5–66.6%) of events as preventable, an error-based approach rated 64.3% (95% CI: 61.8–66.9%) of events as preventable, and an algorithm-based approach rated 68.8% (95% CI: 66.1–71.1%) of events as preventable. When applying the same method, the inter-rater agreement between clinicians was 0.53 (95% CI: 0.48–0.59), 0.55 (95%CI: 0.50–0.60) and 0.55 (95% CI: 0.49–0.55) for the best practice-, error-, and algorithm-based approaches, respectively. The inter-rater agreement between different assessment methods using consensus ratings for each ranged between 0.88 (95% CI 0.85–0.91) and 0.99 (95% CI 0.98–1.00). Compared to a best practice-based assessment, clinicians believed the algorithm-based assessment was too rigid. It did not account for the complexities of and variations in clinical practice, and frequently was too definitive when assigning preventability ratings. CONCLUSION: There was good agreement between all three methods of determining the preventability of adverse drug events. However, clinicians found the algorithmic approach constraining, and preferred a best practice-based assessment method. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-018-0617-4) contains supplementary material, which is available to authorized users.