Prediction of prognostic risk factors in hepatocellular carcinoma with transarterial chemoembolization using multi-modal multi-task deep learning

BACKGROUND: Due to heterogeneity of hepatocellular carcinoma (HCC), outcome assessment of HCC with transarterial chemoembolization (TACE) is challenging. METHODS: We built histologic-related scores to determine microvascular invasion (MVI) and Edmondson-Steiner grade by training CT radiomics features using machine learning classifiers in a cohort of 494 HCCs with hepatic resection. Meanwhile, we developed a deep learning (DL)-score for disease-specific survival by training CT imaging using DL networks in a cohort of 243 HCCs with TACE. Then, three newly built imaging hallmarks with clinicoradiologic factors were analyzed with a Cox-Proportional Hazard (Cox-PH) model. FINDINGS: In HCCs with hepatic resection, two imaging hallmarks resulted in areas under the curve (AUCs) of 0.79 (95% confidence interval [CI]: 0.71–0.85) and 0.72 (95% CI: 0.64–0.79) for predicting MVI and Edmondson-Steiner grade, respectively, using test data. In HCCs with TACE, higher DL-score (hazard ratio [HR]: 3.01; 95% CI: 2.02–4.50), American Joint Committee on Cancer (AJCC) stage III+IV (HR: 1.71; 95% CI: 1.12–2.61), Response Evaluation Criteria in Solid Tumors (RECIST) with stable disease + progressive disease (HR: 2.72; 95% CI: 1.84–4.01), and TACE-course > 3 (HR: 0.65; 95% CI: 0.45–0.76) were independent prognostic factors. Using these factors via a Cox-PH model resulted in a concordance index of 0.73 (95% CI: 0.71–0.76) for predicting overall survival and AUCs of 0.85 (95% CI: 0.81–0.89), 0.90 (95% CI: 0.86–0.94), and 0.89 (95% CI: 0.84–0.92), respectively, for predicting 3-year, 5-year, and 10-year survival. INTERPRETATION: Our study offers a DL-based, noninvasive imaging hallmark to predict outcome of HCCs with TACE. FUNDING: This work was supported by the key research and development program of Jiangsu Province (Grant number: BE2017756).