Effect of important modifiers on harmful effects in evidence synthesis practice of adverse events were insufficiently investigated: an empirical investigation

BACKGROUND: Safety is important in the assessment of health interventions, while the results of adverse events are often susceptive to potential effect modifiers since the event risk tends to be rare. In this study, we investigated whether the potential impact of the important effect modifiers on harmful effects was analyzed in meta-analyses of adverse events. METHODS: Systematic reviews of healthcare interventions, had adverse events as the exclusive outcomes, had at least one meta-analysis, and published between 1(st) January 2015, and 1(st) January 2020 were collected. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. Six effect modifiers that are the most important for harmful effects were identified by a group discussion. The proportions of eligible systematic reviews that investigated the potential impact of the six effect modifiers on harmful effects were summarized. RESULTS: We identified 279 systematic reviews eligible for this study. Except for the modifier of interventions/controls (70.61%, 197/279), most of the systematic reviews failed to investigate the potential impact of treatment duration (21.15%, 59/279), dosage (24.73%, 69/279), age (11.47%, 32/279), risk of bias (6.45%, 18/279), and source of funding (1.08%, 3/279) on harmful effects. Systematic reviews with meta-analyses containing more studies were more likely to investigate the potential impacts of these modifiers on the effects, but the proportion was still low (2.3% to 33.3%). Systematic reviews that developed a protocol were significantly more likely to investigate the potential impact of all these effect modifiers (e.g. treatment duration: odds ratio = 5.08, 95% CI: 2.76 to 9.35) on the results. CONCLUSIONS: Current systematic reviews rarely investigated the potential impact of the important effect modifiers on harmful effects. Methodological guidelines for meta-analysis of adverse events should consider “effect modifier” as one of the domains to help systematic review authors better investigate harmful effects. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-023-01928-2.