Strength in numbers? The fragility index of studies from the Scandinavian knee ligament registries

PURPOSE: The fragility index (FI) is a metric to evaluate the robustness of statistically significant results. It describes the number of patients who would need to change from a non-event to an event to change a result from significant to non-significant. This systematic survey aimed to evaluate the feasibility of applying the FI to findings related to anterior cruciate ligament (ACL) reconstruction in the Scandinavian knee ligament registries. METHODS: The PubMed, EMBASE, Cochrane Library and AMED databases were searched. Studies from the Scandinavian knee ligament registers were eligible if they reported a statistically significant result (p < 0.05) for any of the following dichotomous outcomes; ACL revision, contralateral ACL reconstruction or the presence of postoperative knee laxity. Only studies with a two-arm comparative analysis were included. Eligibility assessment, data extraction and quality assessment were performed by two independent reviewers. The dichotomous analyses were stratified according to the grouping variable for the two comparative arms as follows; age, patient sex, activity at injury, graft choice, drilling technique, graft fixation, single- versus double-bundle, concomitant cartilage injury and country. The two-sided Fisher’s exact test was used to calculate the FI of all statistically significant analyses. RESULTS: From 158 identified studies, 13 studies were included. They reported statistical significance for a total of 56 dichotomous analyses, of which all but two had been determined by a time-to-event analysis. The median sample size for the arms was 5540 (range 92–38,666). The mean FI for all 56 dichotomous analyses was 80.6 (median 34.5), which means that a mean of 80.6 patients were needed to change outcome status to generate a non-significant result instead of a significant one. Seventeen analyses (30.4%) immediately became non-significant when performing the two-sided Fisher’s exact test and, therefore, had an FI of 0. The analyses related to age were the most robust, with a mean FI of 178.5 (median 116, range 1–1089). The mean FI of the other grouping variables ranged from 0.5 to 48.0. CONCLUSION: There was large variability in the FI in analyses from the Scandinavian knee ligament registries and almost one third of the analyses had an FI of zero. The FI is a rough measurement of robustness when applied to registry studies, however, future studies are needed to determine the most appropriate metric for robustness in registry studies. The use of the FI can provide clinicians with a deeper understanding of significant study results and promotes an evidence-based approach in the clinical care of patients. LEVEL OF EVIDENCE: Systematic review of prospective cohort studies, Level II. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00167-019-05551-x) contains supplementary material, which is available to authorized users.