Does including machine learning predictions in ALS clinical trial analysis improve statistical power?

OBJECTIVE: Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease which leads to progressive muscle weakness and eventually death. The increasing availability of large ALS clinical trial datasets have generated much interest in developing predictive models for disease progression. However, the utility of predictive modeling on clinical trial analysis has not been thoroughly evaluated. METHODS: We evaluated a predictive modeling approach for ALS disease progression measured by ALSFRS‐R using the PRO‐ACT database and validated our findings in a novel test set from a former clinical trial. We examined clinical trial scenarios where model predictions could improve statistical power for detecting treatment effects with simulated clinical trials. RESULTS: Models constructed with imputed PRO‐ACT data have better external validation results than those fitted with complete observations. When fitted with imputed data, super learner (R (2) = 0.71, MSPE = 19.7) and random forest (R (2) = 0.70, MSPE = 19.6) have similar performance in the external validation and slightly outperform the linear mixed effects model (R (2) = 0.69, MSPE = 20.5). Simulation studies suggest including machine learning predictions as a covariate in the analysis model of a 12‐month clinical study can increase the trial's effective sample size by 16% when there is a hypothetical treatment effect of 25% reduction in ALSFRS‐R mean rate of change. INTERPRETATION: Predictive modeling approaches for ALSFRS‐R are able to explain a moderate amount of variability in longitudinal change, which is improved by robust missing data handling for baseline characteristics. Including ALSFRS‐R post‐baseline model prediction results as a covariate in the model for primary analysis may increase power under moderate treatment effects.