Empirical evidence of study design biases in nutrition randomised controlled trials: a meta-epidemiological study

BACKGROUND: Instruments to critically appraise randomised controlled trials (RCTs) are based on evidence from meta-epidemiological studies. We aim to conduct a meta-epidemiological study on the average bias associated with reported methodological trial characteristics such as random sequence generation, allocation concealment, blinding, incomplete outcome data, selective reporting, and compliance of RCTs in nutrition research. METHODS: We searched the Cochrane Database of Systematic Reviews, for systematic reviews of RCTs, published between 01 January 2010 and 31 December 2019. We combined the estimates of the average bias (e.g. ratio of risk ratios [RRR] or differences in standardised mean differences) in meta-analyses using the random-effects model. Subgroup analyses were conducted to investigate the potential differences among the RCTs with low versus high/unclear risk of bias with respect to the different types of interventions (e.g. micronutrients, fatty acids, dietary approach), outcomes (e.g. mortality, pregnancy outcomes), and type of outcome (objective, subjective). Heterogeneity was assessed through I(2) and τ(2), and prediction intervals were calculated. RESULTS: We included 27 Cochrane nutrition reviews with 77 meta-analyses (n = 927 RCTs). The available evidence suggests that intervention effect estimates may not be exaggerated in RCTs with high/unclear risk of bias (versus low) judgement for sequence generation (RRR 0.97, 95% CI 0.93 to 1.02; I(2) = 28%; τ(2) = 0.002), allocation concealment (RRR 1.00, 95% CI 0.96 to 1.04; I(2) = 27%; τ(2) = 0.001), blinding of participants and personnel (RRR 0.95, 95% CI 0.91 to 1.00; I(2) = 23%; τ(2) = 0), selective reporting (RRR 0.97, 95% CI 0.92 to 1.02; I(2) = 24%; τ(2) = 0), and compliance (RRR 0.95, 95% CI 0.89 to 1.02; I(2) = 0%; τ(2) = 0). Intervention effect estimates seemed to be exaggerated in RCTs with a high/unclear risk of bias judgement for blinding of outcome assessment (RRR 0.81, 95% CI 0.70 to 0.94; I(2) = 26%; τ(2) = 0.03), which was predominately driven by subjective outcomes, and incomplete outcome data (RRR 0.92, 95% CI 0.88 to 0.97; I(2) = 22%; τ(2) = 0.001). For continuous outcomes, no differences were observed, except for selective reporting. CONCLUSIONS: On average, most characteristics of nutrition RCTs may not exaggerate intervention effect estimates, but the average bias appears to be greatest in trials of subjective outcomes. Replication of this study is suggested in this field to keep this conclusion updated. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12916-022-02540-9.