MS-ResNet: disease-specific survival prediction using longitudinal CT images and clinical data

PURPOSE: Medical imaging data of lung cancer in different stages contain a large amount of time information related to its evolution (emergence, development, or extinction). We try to explore the evolution process of lung images in time dimension to improve the prediction of lung cancer survival by using longitudinal CT images and clinical data jointly. METHODS: In this paper, we propose an innovative multi-branch spatiotemporal residual network (MS-ResNet) for disease-specific survival (DSS) prediction by integrating the longitudinal computed tomography (CT) images at different times and clinical data. Specifically, we first extract the deep features from the multi-period CT images by an improved residual network. Then, the feature selection algorithm is used to select the most relevant feature subset from the clinical data. Finally, we integrate the deep features and feature subsets to take full advantage of the complementarity between the two types of data to generate the final prediction results. RESULTS: The experimental results demonstrate that our MS-ResNet model is superior to other methods, achieving a promising 86.78% accuracy in the classification of short-survivor, med-survivor, and long-survivor. CONCLUSION: In computer-aided prognostic analysis of cancer, the time dimension features of the course of disease and the integration of patient clinical data and CT data can effectively improve the prediction accuracy.