Advanced Deep Learning Approach to Automatically Segment Malignant Tumors and Ablation Zone in the Liver With Contrast-Enhanced CT

OBJECTIVE: Liver cancer is one of the most commonly diagnosed cancer, and energy-based tumor ablation is a widely accepted treatment. Automatic and robust segmentation of liver tumors and ablation zones would facilitate the evaluation of treatment success. The purpose of this study was to develop and evaluate an automatic deep learning based method for (1) segmentation of liver and liver tumors in both arterial and portal venous phase for pre-treatment CT, and (2) segmentation of liver and ablation zones in both arterial and portal venous phase for after ablation treatment. MATERIALS AND METHODS: 252 CT images from 63 patients undergoing liver tumor ablation at a large University Hospital were retrospectively included; each patient had pre-treatment and post-treatment multi-phase CT images. 3D voxel-wise manual segmentation of the liver, tumors and ablation region by the radiologist provided reference standard. Deep learning models for liver and lesion segmentation were initially trained on the public Liver Tumor Segmentation Challenge (LiTS) dataset to obtain base models. Then, transfer learning was applied to adapt the base models on the clinical training-set, to obtain tumor and ablation segmentation models both for arterial and portal venous phase images. For modeling, 2D residual-attention Unet (RA-Unet) was employed for liver segmentation and a multi-scale patch-based 3D RA-Unet for tumor and ablation segmentation. RESULTS: On the independent test-set, the proposed method achieved a dice similarity coefficient (DSC) of 0.96 and 0.95 for liver segmentation on arterial and portal venous phase, respectively. For liver tumors, the model on arterial phase achieved detection sensitivity of 71%, DSC of 0.64, and on portal venous phase sensitivity of 82%, DSC of 0.73. For liver tumors >0.5cm(3) performance improved to sensitivity 79%, DSC 0.65 on arterial phase and, sensitivity 86%, DSC 0.72 on portal venous phase. For ablation zone, the model on arterial phase achieved detection sensitivity of 90%, DSC of 0.83, and on portal venous phase sensitivity of 90%, DSC of 0.89. CONCLUSION: The proposed deep learning approach can provide automated segmentation of liver tumors and ablation zones on multi-phase (arterial and portal venous) and multi-time-point (before and after treatment) CT enabling quantitative evaluation of treatment success.