An evaluation of harvest plots to display results of meta-analyses in overviews of reviews: a cross-sectional study

BACKGROUND: Harvest plots are used to graphically display evidence from complex and diverse studies or results. Overviews of reviews bring together evidence from two or more systematic reviews. Our objective was to determine the feasibility of using harvest plots to depict complex results of overviews of reviews. METHODS: We conducted a survey of 279 members of Cochrane Child Health to determine their preferences for graphical display of data, and their understanding of data presented in the form of harvest plots. Preferences were rated on a scale of 0–100 (100 most preferred) and tabulated using descriptive statistics. Knowledge and accuracy were assessed by tabulating the number of correctly answered questions for harvest plots and traditional data summary tables; t-tests were used to compare responses between formats. RESULTS: 53 individuals from 7 countries completed the survey (19 %): 60 % were females; the majority had an MD (38 %), PhD (47 %), or equivalent. Respondents had published a median of 3 systematic reviews (inter-quartile range 1 to 8). There were few differences between harvest plots and tables in terms of being: well-suited to summarize and display results from meta-analysis (52 vs. 56); easy to understand (53 vs. 51); and, intuitive (49 vs. 44). Harvest plots were considered more aesthetically pleasing (56 vs. 44, p = 0.03). 40 % felt the harvest plots could be used in conjunction with tables to display results from meta-analyses; additionally, 45 % felt the harvest plots could be used with some improvement. There was no statistically significant difference in percentage of knowledge questions answered correctly for harvest plots compared with tables. When considering both types of data display, 21 % of knowledge questions were answered incorrectly. CONCLUSIONS: Neither harvest plots nor standard summary tables were ranked highly in terms of being easy to understand or intuitive, reflecting that neither format is ideal to summarize the results of meta-analyses in overviews of reviews. Responses to knowledge questions showed some misinterpretation of results of meta-analyses. Reviewers should ensure that messages are clearly articulated and summarized in the text to avoid misinterpretation. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12874-015-0084-0) contains supplementary material, which is available to authorized users.