Development and External Validation of Machine Learning-Based Models for Predicting Lung Metastasis in Kidney Cancer: A Large Population-Based Study

The accuracy of indices widely used to evaluate lung metastasis (LM) in patients with kidney cancer (KC) is insufficient. Therefore, we aimed at developing a model to estimate the risk of developing LM in KC based on a large population size and machine learning algorithms. Demographic and clinicopathologic variables of patients with KC diagnosed between 2004 and 2017 were retrospectively analyzed. We performed a univariate logistic regression analysis to identify risk factors for LM in patients with KC. Six machine learning (ML) classifiers were established and tuned using the ten-fold cross-validation method. External validation was performed using clinicopathologic information from 492 patients from the Southwest Hospital, Chongqing, China. Algorithm performance was estimated by analyzing the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, precision, recall, F1 score, clinical decision analysis (DCA), and clinical utility curve (CUC). A total of 52,714 eligible patients diagnosed with KC were enrolled, of whom 2,618 developed LM. Variables of age, sex, race, T stage, N stage, tumor size, histology, and grade were identified as important for the prediction of LM. The extreme gradient boosting (XGB) algorithm performed better than other models in both the internal validation (AUC: 0.913, sensitivity: 0.873, specificity: 0.809, and F1 score: 0.325) and the external validation (AUC: 0.904, sensitivity: 0.750, specificity: 0.878, and F1 score: 0.364). This study established a predictive model for LM in KC patients based on ML algorithms which showed high accuracy and applicative value. A web-based predictor was built using the XGB model to help clinicians make more rational and personalized decisions.