Comparing internal and external validation in the discovery of qualitative treatment-subgroup effects using two small clinical trials

In a two-arm randomized trial where both arms receive active treatment (i.e., treatments A and B), often the primary goal is to determine which of the treatments, on average, is more effective. A supplementary objective is to understand possible heterogeneity in the treatment effect by identifying multivariable subgroups of patients for whom A is more effective than B and, conversely, patients for whom B is more effective than A, known as a qualitative interaction. This is the objective of the qualitative interaction trees (QUINT) algorithm developed by Dusseldorp et al (Statistics in Medicine, 2014). We apply QUINT to a small randomized trial comparing facilitated relaxation meditation to facilitated life completion and preparation among patients with life-limiting illness (n = 135). We then conduct an internal validation of the QUINT solution using bootstrap resampling and compare it to an external validation with another, similarly conducted small randomized trial. Internal and external validation showed the apparent range in effect sizes was over-estimated, and subgroups identified were not consistent between the two trials. While the qualitative interaction trees algorithm is a promising area of data-driven multivariable subgroup discovery, our analyses illustrate the importance of validating the solution, particularly for trials with smaller numbers of participants.