Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis

BACKGROUND: More and more automated efficient ultrasound image analysis techniques, such as ultrasound-based computer-aided diagnosis system (CAD), were developed to obtain accurate, reproducible, and more objective diagnosis results for thyroid nodules. So far, whether the diagnostic performance of existing CAD systems can reach the diagnostic level of experienced radiologists is still controversial. The aim of the meta-analysis was to evaluate the accuracy of CAD for thyroid nodules’ diagnosis by reviewing current literatures and summarizing the research status. METHODS: A detailed literature search on PubMed, Embase, and Cochrane Libraries for articles published until December 2018 was carried out. The diagnostic performances of CAD systems vs radiologist were evaluated by meta-analysis. We determined the sensitivity and the specificity across studies, calculated positive and negative likelihood ratios and constructed summary receiver-operating characteristic (SROC) curves. Meta-analysis of studies was performed using a mixed-effect, hierarchical logistic regression model. RESULTS: Five studies with 536 patients and 723 thyroid nodules were included in this meta-analysis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio (DOR) for CAD system were 0.87 (95% confidence interval [CI], 0.73–0.94), 0.79 (95% CI 0.63–0.89), 4.1 (95% CI 2.5–6.9), 0.17 (95% CI 0.09–0.32), and 25 (95% CI 15–42), respectively. The SROC curve indicated that the area under the curve was 0.90 (95% CI 0.87–0.92). The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and DOR for experienced radiologists were 0.82 (95% CI 0.69–0.91), 0.83 (95% CI 0.76–0.89), 4.9 (95% CI 3.4–7.0), 0.22 (95% CI 0.12–0.38), and 23 (95% CI 11–46), respectively. The SROC curve indicated that the area under the curve was 0.96 (95% CI 0.94–0.97). CONCLUSION: The sensitivity of the CAD system in the diagnosis of thyroid nodules was similar to that of experienced radiologists. However, the CAD system had lower specificity and DOR than experienced radiologists. The CAD system may play the potential role as a decision-making assistant alongside radiologists in the thyroid nodules’ diagnosis. Future technical improvements would be helpful to increase the accuracy as well as diagnostic efficiency.