Methodological and Clinical Heterogeneity and Extraction Errors in Meta‐Analyses of Catheter Ablation for Atrial Fibrillation in Heart Failure

BACKGROUND: Meta‐analyses are expected to follow a standardized process, and thus, they have become highly formulaic, although there is little evidence that such regimentation yields high‐quality results. METHODS AND RESULTS: This article describes the results of a critical examination of 14 published meta‐analyses of catheter ablation for atrial fibrillation in heart failure that were based on a nearly identical core set of 4 to 6 primary trials. Methodological issues included (1) the neglect of primary data or the failure to report any primary data; (2) the inaccurate recording of the number of randomized patients; (3) the lack of attention to data missingness or baseline imbalances; (4) the failure to contact investigators of primary trials for additional data; (5) the incorrect extraction of data, the misidentification of events, and the assignment of events to the wrong treatment groups; (6) the calculation of summary estimates based on demonstrably heterogenous data, methods of differing reliability, or unrelated end points; and (7) the development of conclusions based on sparse numbers of events or overly reliant on the results of 1 dominant trial. CONCLUSIONS: These findings reinforce existing concerns about the methodological validity of meta‐analyses and their current status in the hierarchy of medical evidence, and they raise new questions about the process by which meta‐analyses undergo peer review by medical journals.