How should systematic reviewers handle conference abstracts? A view from the trenches

BACKGROUND: While identifying and cataloging unpublished studies from conference proceedings is generally recognized as a good practice during systematic reviews, controversy remains whether to include study results that are reported in conference abstracts. Existing guidelines provide conflicting recommendations. MAIN BODY: The main argument for including conference abstracts in systematic reviews is that abstracts with positive results are preferentially published, and published sooner, as full-length articles compared with other abstracts. Arguments against including conference abstracts are that (1) searching for abstracts is resource-intensive, (2) abstracts may not contain adequate information, and (3) the information in abstracts may not be dependable. However, studies comparing conference abstracts and fully published articles of the same study find only minor differences, usually with conference abstracts presenting preliminary results. Other studies that have examined differences in treatment estimates of meta-analyses with and without conference abstracts report changes in precision, but usually not in the treatment effect estimate. However, in some cases, including conference abstracts has made a difference in the estimate of the treatment effect, not just its precision. Instead of arbitrarily deciding to include or exclude conference abstracts in systematic reviews, we suggest that systematic reviewers should consider the availability of evidence informing the review. If available evidence is sparse or conflicting, it may be worthwhile to search for conference abstracts. Further, attempts to contact authors of abstracts or search for protocols or trial registers to supplement the information presented in conference abstracts is prudent. If unique information from conference abstracts is included in a meta-analysis, a sensitivity analysis with and without the unique results should be conducted. CONCLUSIONS: Under given circumstances, it is worthwhile to search for and include results from conference abstracts in systematic reviews.