Most published meta-regression analyses based on aggregate data suffer from methodological pitfalls: a meta-epidemiological study

BACKGROUND: Due to clinical and methodological diversity, clinical studies included in meta-analyses often differ in ways that lead to differences in treatment effects across studies. Meta-regression analysis is generally recommended to explore associations between study-level characteristics and treatment effect, however, three key pitfalls of meta-regression may lead to invalid conclusions. Our aims were to determine the frequency of these three pitfalls of meta-regression analyses, examine characteristics associated with the occurrence of these pitfalls, and explore changes between 2002 and 2012. METHODS: A meta-epidemiological study of studies including aggregate data meta-regression analysis in the years 2002 and 2012. We assessed the prevalence of meta-regression analyses with at least 1 of 3 pitfalls: ecological fallacy, overfitting, and inappropriate methods to regress treatment effects against the risk of the analysed outcome. We used logistic regression to investigate study characteristics associated with pitfalls and examined differences between 2002 and 2012. RESULTS: Our search yielded 580 studies with meta-analyses, of which 81 included meta-regression analyses with aggregated data. 57 meta-regression analyses were found to contain at least one pitfall (70%): 53 were susceptible to ecological fallacy (65%), 14 had a risk of overfitting (17%), and 5 inappropriately regressed treatment effects against the risk of the analysed outcome (6%). We found no difference in the prevalence of meta-regression analyses with methodological pitfalls between 2002 and 2012, nor any study-level characteristic that was clearly associated with the occurrence of any of the pitfalls. CONCLUSION: The majority of meta-regression analyses based on aggregate data contain methodological pitfalls that may result in misleading findings. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-021-01310-0.