A study of the value of requesting information from drug manufacturers for systematic reviews; 9 years of experience from the drug effectiveness review project

BACKGROUND: Systematic reviews (SRs) depend on comprehensive searches for evidence to provide balanced, accurate results. Requesting published and unpublished studies from pharmaceutical manufacturers has been proposed as a method to engage industry stakeholders and potentially reduce reporting bias. The Drug Effectiveness Review Project (DERP) has been requesting such evidence since 2003; the purpose of this study was to retrospectively evaluate the type and impact of the evidence received. METHODS: Data from “dossiers” submitted by pharmaceutical manufacturers for a set of 40 SRs conducted for DERP from July 2006 to June 2015 were retrospectively evaluated. Characteristics of data submitted in dossiers, including numbers, types, and characteristics of studies submitted and then included in DERP SRs, were abstracted. Time trends, study quality, publication status, and whether the submission represented a unique study or supplemental data to a published study were assessed. The impact of this evidence on SR conclusions was assessed using dual review. Differences were resolved through a consensus. RESULTS: Over 9 years, 160 dossiers were received, relating to 40 DERP SRs. Out of 7360 studies/datasets submitted, 2.2% (160) were included in a SR. The ratio of submitted-to-included increased over time. Most were unique studies (23% were supplemental data sets), and almost 42% of the studies were unpublished. The majority of the studies were rated fair quality, with 7.3% rated good and 14% rated poor quality by the original SR authors. Considering all literature search sources, 7.2% of all studies included in the 40 SRs came from a dossier, and 16% of dossier studies were included in a meta-analysis. The dossier studies resulted in changes to conclusions in 42% of the SRs. Out of 46 unpublished unique studies included in a SR, 25 (54%) influenced the conclusions in favor of the manufacturers drug, 8% favored a competitor drug, and 40% favored neither. In 92% of cases favoring the manufacturer’s drug, the dossier study was the only evidence for that drug in a specific population or outcome. CONCLUSIONS: In SRs conducted for DERP, few studies submitted by pharmaceutical manufacturers were ultimately included in a SR. The included data helped to reduce reporting and publication bias by filling important gaps and in some cases led to altered conclusions. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s13643-018-0834-2) contains supplementary material, which is available to authorized users.