Concordance assessment of Watson for Oncology in breast cancer chemotherapy: first China experience

BACKGROUND: IBM Watson for Oncology (WFO) is an artificial intelligence cognitive computing system that provides confidence-ranked, evidence-based treatment recommendations for cancer. We examine the level of agreement for breast cancer chemotherapy between WFO recommended and clinical use in a large population of breast cancer cases. METHODS: A total of 1,301 breast cancer patients were reviewed in The First Affiliated Hospital with Nanjing Medical University, China from June 2013 to December 2017. Patients’ data were entered manually into WFO by the trained senior oncology fellows. Chemotherapy recommendations were provided in 3 categories, “Recommended”, “For Consideration”, and “Not Recommended”. Concordance was achieved when oncologists’ treatment suggestions were in the “Recommended” or “For Consideration” categories. RESULTS: The chemotherapy regimen concordance was 69.4% among all breast cancer cases, 65.0% among the cases in adjuvant chemotherapy (AC) group and 96.7% among the cases in neoadjuvant chemotherapy (NAC) group. The concordance varied greatly in subset analysis with respect to TNM stage and molecular subtype. AC recommendations were concordant in 92.3% of stage III breast cancer and 50.8% of stage I. However, the concordance varied by molecular subtype, which was higher for triple negative breast cancer (89.3%) than others. The chemotherapy regimen concordance declined significantly with increasing age, except for the age group 41–50 years. CONCLUSIONS: Chemotherapy regimens provided by WFO did not exhibit a high degree of agreement with those suggested by oncologists in clinical practice in the hospital in China. The current effort is underway to enhance WFO’s capabilities as a cognitive decision support tool by incorporating regional guidelines, enabling oncologists and patients to benefit from WFO worldwide.