Challenges of using historical data in clinical trials

Clinical trials’ attempts to improve design and analysis in order to reduce the experimental burden, providing more rapid and valid answers, are common, especially in rare or pediatric diseases, or in the setting of an emerging disease such as COVID-19. One promising though underused approach is to increase the information by borrowing external data on control and/or on treatment effect, which is almost always available. However, the objectives and extent of use of such external data differ widely across studies. They may be dedicated to extrapolate from different populations (from adults to children, across geographic regions, etc.) or treatments. They may adaptively apply during an ongoing trial or be only used at the time of analysis of a trial after study enrollment is closed. They can use only outcome data from a historical control group for either a complete or partial replacement of the current control arm, or focusing on estimated treatment benefit from a whole historical sample. They make various assumptions regarding the exchangeability or commensality of the historical and current populations, with various diagnostics for checking those assumptions. Finally, they use either a frequentist or Bayesian inference, based on aggregated data or requiring individual patient data (IPD). Notably, borrowing of external data often uses Bayesian approaches, by modelling external information into a prior, with increasing complexity to face heterogeneity across populations. Similar frequentist methods based on weighted tests or joint models, have also been proposed, as well as hierarchical models, either Bayesian or frequentist, similar to a meta-analysis setting. Last, methods based on propensity scores and those based on matching-adjusted indirect comparisons could also been used, depending on the availability of IPD. We aim to provide some comparison of the competing approaches for incorporating historical data into a contemporary trial. Data from randomized clinical trials performed in patients with COVID-19 will serve as illustrative examples.