Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events

BACKGROUND: Adverse events (AEs) in clinical trials may be reported in multiple sources. Different methods for reporting adverse events across trials or across sources for a single trial may produce inconsistent information about the adverse events associated with interventions. METHODS: We compared the methods authors use to decide which AEs to include in a particular source (i.e., “selection criteria”), including the number of different types of AEs reported (i.e., rather than the number of events). We compared sources (e.g., journal articles, clinical study reports (CSRs)) of trials for two drug-indications—gabapentin for neuropathic pain and quetiapine for bipolar depression. Electronic searches were completed in 2015. We identified selection criteria and assessed how criteria affected AE reporting. RESULTS: We identified 21 gabapentin and 7 quetiapine trials. We found 6 gabapentin CSRs and 2 quetiapine CSRs, all written by drug manufacturers. All CSRs reported all AEs without applying selection criteria; by comparison, no other source reported all AEs, and 15/68 (22%) gabapentin sources and 19/48 (40%) quetiapine sources reported using selection criteria. Selection criteria greatly affected the number of AEs reported. For example, 67/316 (21%) AEs in one quetiapine trial met the criterion “occurring in ≥2% of participants in any treatment group,” while only 5/316 (2%) AEs met the criterion “occurring in ≥10% of quetiapine-treated patients and twice as frequent in the quetiapine group as the placebo group.” CONCLUSIONS: Selection criteria for reporting AEs vary across trials and across sources for individual trials. If investigators do not pre-specify selection criteria, they might “cherry-pick” AEs based on results. Even if investigators pre-specify selection criteria, selective reporting will produce biased meta-analyses and clinical practice guidelines. Data about all AEs identified in clinical trials should be publicly available; however, sharing data will not solve all the problems identified in this study. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s13063-019-3581-3) contains supplementary material, which is available to authorized users.