Risk of bias judgements and strength of conclusions in meta-evidence from the Cochrane Colorectal Cancer Group

BACKGROUND: The Cochrane Collaboration records risk of bias (ROB) judgements on the original studies it analyses. The aim of this review is to perform an audit of all literature produced by the Cochrane Colorectal Cancer Group (CCCG), focusing on whether intervention type has any relationship with ROB and the ability of a review to inform clinical practice. METHODS: The most recent version of every CCCG review from January 2000 to the end of July 2018 was included. Conclusions were categorized as informing clinical practice (I) or not (N). Both I and N categories were divided into firm (F) or tempered (T) based on the definitiveness of their language. ROB judgements were aggregated. Reviews were classed as Medical (M), Surgical (S), Medical & Surgical (MS) or Other (O) based on their intervention, with O reviews then excluded. Data were analyzed in SPSS. RESULTS: Ninety-five reviews were included, covering 1892 studies. Sixty-two percent (n = 59/95) informed clinical practice (I). Thirty-eight percent (n = 36/95) did not inform clinical practice (N). Of the N group, 53% (n = 19/36) were completely equivocal (firm) while 47% (n = 17/36) were moderately so (tempered). In the I group, 46% (n = 27/59) gave a conclusion that was firm and 54% (n = 32/59) were tempered. Seven thousand five hundred sixty-four cases of bias were assessed. Risk of bias was low in 43%, high in 20% and unclear in 37%. A review that regarded a medical intervention alone was significantly more likely to be comprised of studies with a low risk of bias than a review that included a surgical intervention (p < 0.001). CONCLUSION: The Cochrane Colorectal Cancer Group finds the risk of bias to be low in less than half of its judgements. A review that included a surgical intervention was less likely to display low risk of bias. Risk of bias was associated with whether a review informed clinical practice, but intervention type was not. Readers of colorectal literature should be cautious when considering original and meta-evidence in this field, particularly where a surgical intervention is assessed. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s13643-019-1001-0) contains supplementary material, which is available to authorized users.