The aprotinin saga and the risks of conducting meta-analyses on small randomised controlled trials – a critique of a Cochrane review

BACKGROUND: Aprotinin for reducing blood loss during coronary artery bypass surgery was withdrawn from the market after early termination of a large randomised controlled trial (RCT) showing excess mortality for patients receiving aprotinin compared to lysine analogues. Several meta-analyses of small RCTs did not show excess mortality and even indicated reduced mortality, while several observational studies showed excess mortality. The aim of this paper is to review the quality of the meta-analysis of a Cochrane report. METHODS: The 52 studies included in the meta-analysis of the Cochrane report were reviewed according to whether an objective to study mortality was formulated in advance, whether follow-up method or time were specified, and whether the study had statistical power to show any effect. RESULTS: The Cochrane report restricted the analysis to RCTs, but the largest study should not have been included given that it was a prospective observational study with 1 784 patients rather than an RCT. None of the RCTs had sufficient statistical power to detect differences in mortality. Most studies had fewer than 100 patients. Seven out of 51 RCTs had mortality outcome as one of their objectives. Only very few described follow-up method or time. CONCLUSION: It is doubtful whether small studies should be included in meta-analyses if they do not have the purpose of studying the specified outcome and if the follow-up method or time are not adequately described. The aprotinin saga shows overconfidence in small RCTs of inferior quality compared to well-conducted observational studies.