Reliability and External Validity of AMSTAR in Assessing Quality of TCM Systematic Reviews

Objective. The aim of this study is to measure the reliability and external validity of AMSTAR by applying it to a sample of TCM systematic reviews. Study Design and Methods. We tested the agreement, reliability, construct validity, and feasibility of AMSTAR through comparisons with OQAQ. Statistical analyses were performed by using SPSS 13.0. Results. A random of sample with 41 TCM systematic reviews was selected from a database. The interrater agreement of the individual items of AMSTAR was moderate with a mean kappa of 0.50 (95% CI: 0.26, 0.73). The ICC for AMSTAR against OQAQ (total score of 9 items, excluding item 10) was 0.87 (95% CI: 0.76, 0.93). Conclusions. Although there is room for improvement on few items, the new tool is reliable, valid, and easy to use for methodological quality assessment of systematic reviews on TCM.