Adjusting trial results for biases in meta‐analysis: combining data‐based evidence on bias with detailed trial assessment

Flaws in the conduct of randomized trials can lead to biased estimation of the intervention effect. Methods for adjustment of within‐trial biases in meta‐analysis include the use of empirical evidence from an external collection of meta‐analyses, and the use of expert opinion informed by the assessment of detailed trial information. Our aim is to present methods to combine these two approaches to gain the advantages of both. We make use of the risk of bias information that is routinely available in Cochrane reviews, by obtaining empirical distributions for the bias associated with particular bias profiles (combinations of risk of bias judgements). We propose three methods: a formal combination of empirical evidence and opinion in a Bayesian analysis; asking experts to give an opinion on bias informed by both summary trial information and a bias distribution from the empirical evidence, either numerically or by selecting areas of the empirical distribution. The methods are demonstrated through application to two example binary outcome meta‐analyses. Bias distributions based on opinion informed by trial information alone were most dispersed on average, and those based on opinions obtained by selecting areas of the empirical distribution were narrowest. Although the three methods for combining empirical evidence with opinion vary in ease and speed of implementation, they yielded similar results in the two examples.