Include or not to include conference abstracts in systematic reviews? Lessons learned from a large Cochrane network meta-analysis including 585 trials

BACKGROUND: Systematic reviews attempt to gather all available evidence. Controversy exists regarding effort and benefit of including study results presented at conferences only. We recently published a Cochrane network meta-analysis (NMA) including 585 randomized controlled trials comparing drugs for prevention of postoperative nausea and vomiting (PONV). Studies published as conference abstracts only were excluded. This study aimed to include all eligible studies published as abstracts only, assessing their added value regarding reporting quality and effect on the review’s interpretation. METHODS: Conference abstracts were searched in the review’s excluded studies and conference proceedings of anaesthesiologic societies. We assessed their reporting quality regarding review’s eligibility criteria, Cochrane ‘risk of bias’ assessment tool 1.0, and adherence to CONSORT (Consolidated Standards of Reporting Trials) for abstracts. Abstracts were included in sensitivity NMA, and impact on the NMA structure was investigated. RESULTS: We identified 90 abstracts. A total of 14% (13/90) were eligible. A total of 86% (77/90) are awaiting classification due to insufficient reporting of review’s eligibility criteria. In abstracts awaiting classification, sufficient information was missing on standardization of anaesthesia in 71% (55/77), age of participants in 56% (43/77), and outcome details in 46% (36/77). A total of 73% (66/90) of abstracts lacked sufficient information on 15/25 data extraction items. Reported study characteristics of abstracts were comparable to included studies of the review. A total of 62% (56/90) of abstract trials were assessed as overall high risk of bias due to poor reporting. Median adherence to CONSORT for abstracts was 24% (IQR, 18 to 29%). Six of the 13 eligible abstracts reported relevant outcome data in sufficient detail for NMA on seven outcomes of the Cochrane review. Inclusion of abstracts did not substantially change the network structure, network effect estimates, ranking of treatments, or the conclusion. Certainty of evidence for headache on palonosetron use was upgraded from very low to low. CONCLUSIONS: Most conference abstracts on PONV were insufficiently reported regarding review’s narrow inclusion criteria and could not be included in NMA. The resource-intensive search and evaluation of abstracts did not substantially extent the full-text evidence base of the review, given the few adequately reported abstracts. Conferences should oblige authors to adhere to CONSORT for abstracts. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13643-022-02048-6.