Accumulating Research: A Systematic Account of How Cumulative Meta-Analyses Would Have Provided Knowledge, Improved Health, Reduced Harm and Saved Resources

BACKGROUND: “Cumulative meta-analysis” describes a statistical procedure to calculate, retrospectively, summary estimates from the results of similar trials every time the results of a further trial in the series had become available. In the early 1990s, comparisons of cumulative meta-analyses of treatments for myocardial infarction with advice promulgated through medical textbooks showed that research had continued long after robust estimates of treatment effects had accumulated, and that medical textbooks had overlooked strong, existing evidence from trials. Cumulative meta-analyses have subsequently been used to assess what could have been known had new studies been informed by systematic reviews of relevant existing evidence and how waste might have been reduced. METHODS AND FINDINGS: We used a systematic approach to identify and summarise the findings of cumulative meta-analyses of studies of the effects of clinical interventions, published from 1992 to 2012. Searches were done of PubMed, MEDLINE, EMBASE, the Cochrane Methodology Register and Science Citation Index. A total of 50 eligible reports were identified, including more than 1,500 cumulative meta-analyses. A variety of themes are illustrated with specific examples. The studies showed that initially positive results became null or negative in meta-analyses as more trials were done; that early null or negative results were over-turned; that stable results (beneficial, harmful and neutral) would have been seen had a meta-analysis been done before the new trial; and that additional trials had been much too small to resolve the remaining uncertainties. CONCLUSIONS: This large, unique collection of cumulative meta-analyses highlights how a review of the existing evidence might have helped researchers, practitioners, patients and funders make more informed decisions and choices about new trials over decades of research. This would have led to earlier uptake of effective interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.