Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer

BACKGROUND: Multiple treatments are frequently available for a given condition, and clinicians and patients need a comprehensive, up-to-date synthesis of evidence for all competing treatments. We aimed to quantify the waste of research related to the failure of systematic reviews to provide a complete and up-to-date evidence synthesis over time. METHODS: We performed a series of systematic overviews and networks of randomized trials assessing the gap between evidence covered by systematic reviews and available trials of second-line treatments for advanced non-small cell lung cancer. We searched the Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, MEDLINE, EMBASE, and other resources sequentially by year from 2009 to March 2, 2015. We sequentially compared the amount of evidence missing from systematic reviews to the randomized evidence available for inclusion each year. We constructed cumulative networks of randomized evidence over time and evaluated the proportion of trials, patients, treatments, and treatment comparisons not covered by systematic reviews on December 31 each year from 2009 to 2015. RESULTS: We identified 77 trials (28,636 patients) assessing 47 treatments with 54 comparisons and 29 systematic reviews (13 published after 2013). From 2009 to 2015, the evidence covered by existing systematic reviews was consistently incomplete: 45 % to 70 % of trials; 30 % to 58 % of patients; 40 % to 66 % of treatments; and 38 % to 71 % of comparisons were missing. In the cumulative networks of randomized evidence, 10 % to 17 % of treatment comparisons were partially covered by systematic reviews and 55 % to 85 % were partially or not covered. CONCLUSIONS: We illustrate how systematic reviews of a given condition provide a fragmented, out-of-date panorama of the evidence for all treatments. This waste of research might be reduced by the development of live cumulative network meta-analyses. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12916-016-0555-0) contains supplementary material, which is available to authorized users.