Quality of systematic reviews: an example of studies comparing artificial disc replacement with fusion in the cervical spine

Cervical artificial disc replacement (C-ADR) is now an alternative to anterior cervical discectomy and fusion (ACDF). Many studies have evaluated the efficacy of C-ADR compared with ACDF. This led to a series of systematic reviews and meta-analyses to evaluate the evidence of the superiority of one intervention against the other. The aim of the study presented here was to evaluate the quality of these reviews and meta-analyses. Medline via Ovid, Embase, and Cochrane Library were searched using the keywords: (total disk replacement, prosthesis, implantation, discectomy, and arthroplasty) AND (cervical vertebrae, cervical spine, and spine) AND (systematic reviews, reviews, and meta-analysis). Screening and data extraction were conducted by two reviewers independently. Two reviewers then assessed the quality of the selected reviews and meta-analysis using 11-item AMSTAR score which is a validated measurement tool to assess the methodological quality of systematic reviews. Screening of full reports of 46 relevant abstracts resulted in the selection of 15 systematic reviews and/or meta-analyses as eligible for this study. The two reviewers’ inter-rater agreement level was high as indicated by kappa of >0.72. The AMSTAR score of the reviews ranged from 3 to 11. Only one study (a Cochrane review) scored 100% (AMSTAR 11). Five studies scored below (AMSTAR 5) indicating low-quality reviews. The most significant drawbacks of reviews of a score below 5 were not using an extensive search strategy, failure to use the scientific quality of the included studies appropriately in formulating a conclusion, not assessing publication bias, and not reporting the excluded studies. With a significant exception of a Cochrane review, the methodological quality of systematic reviews evaluating the evidence of C-ADR versus ACDF has to be improved.