A protocol for a systematic review on the impact of unpublished studies and studies published in the gray literature in meta-analyses

BACKGROUND: Meta-analyses are particularly vulnerable to the effects of publication bias. Despite methodologists’ best efforts to locate all evidence for a given topic the most comprehensive searches are likely to miss unpublished studies and studies that are published in the gray literature only. If the results of the missing studies differ systematically from the published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention’s effects. As part of the OPEN project (http://www.open-project.eu) we will conduct a systematic review with the following objectives: ▪ To assess the impact of studies that are not published or published in the gray literature on pooled effect estimates in meta-analyses (quantitative measure). ▪ To assess whether the inclusion of unpublished studies or studies published in the gray literature leads to different conclusions in meta-analyses (qualitative measure). METHODS/DESIGN: Inclusion criteria: Methodological research projects of a cohort of meta-analyses which compare the effect of the inclusion or exclusion of unpublished studies or studies published in the gray literature. Literature search: To identify relevant research projects we will conduct electronic searches in Medline, Embase and The Cochrane Library; check reference lists; and contact experts. Outcomes: 1) The extent to which the effect estimate in a meta-analyses changes with the inclusion or exclusion of studies that were not published or published in the gray literature; and 2) the extent to which the inclusion of unpublished studies impacts the meta-analyses’ conclusions. Data collection: Information will be collected on the area of health care; the number of meta-analyses included in the methodological research project; the number of studies included in the meta-analyses; the number of study participants; the number and type of unpublished studies; studies published in the gray literature and published studies; the sources used to retrieve studies that are unpublished, published in the gray literature, or commercially published; and the validity of the methodological research project. Data synthesis: Data synthesis will involve descriptive and statistical summaries of the findings of the included methodological research projects. DISCUSSION: Results are expected to be publicly available in the middle of 2013.