Automatic renal mass segmentation and classification on CT images based on 3D U-Net and ResNet algorithms

PURPOSE: To automatically evaluate renal masses in CT images by using a cascade 3D U-Net- and ResNet-based method to accurately segment and classify focal renal lesions. MATERIAL AND METHODS: We used an institutional dataset comprising 610 CT image series from 490 patients from August 2009 to August 2021 to train and evaluate the proposed method. We first determined the boundaries of the kidneys on the CT images utilizing a 3D U-Net-based method to be used as a region of interest to search for renal mass. An ensemble learning model based on 3D U-Net was then used to detect and segment the masses, followed by a ResNet algorithm for classification. Our algorithm was evaluated with an external validation dataset and kidney tumor segmentation (KiTS21) challenge dataset. RESULTS: The algorithm achieved a Dice similarity coefficient (DSC) of 0.99 for bilateral kidney boundary segmentation in the test set. The average DSC for renal mass delineation using the 3D U-Net was 0.75 and 0.83. Our method detected renal masses with recalls of 84.54% and 75.90%. The classification accuracy in the test set was 86.05% for masses (<5 mm) and 91.97% for masses (≥5 mm). CONCLUSION: We developed a deep learning-based method for fully automated segmentation and classification of renal masses in CT images. Testing of this algorithm showed that it has the capability of accurately localizing and classifying renal masses.