Are morbidity and mortality estimates from randomized controlled trials externally valid? A comparison of outcomes among infants enrolled into an RCT or a cohort study in Botswana

BACKGROUND: The external validity of the randomized controlled trial (RCT) refers to the extent to which the results of the RCT apply to the relevant, non-trial population and is impacted by its eligibility criteria, its organization, and its delivery of the intervention. Here, we compared the outcomes of mortality and hospitalization between an RCT and a cohort study that concurrently enrolled HIV-exposed uninfected (HEU) newborns in Botswana. METHODS: The Mpepu Study (the RCT) was a clinical trial which determined that co-trimoxazole (CTX) provided no survival benefit for HEUs, allowing both arms of the RCT to be used. The Maikaelelo study (the cohort study) was a prospective observational study that enrolled HEU newborns with telephone follow-up and no in-person visits. Rates of death and hospitalization in the pooled population, were modeled using cox-proportional hazards models for time to death or time to first hospitalization, with study setting (RCT vs. cohort study) as an independent variable. The causal effect of study setting on morbidity and mortality was obtained through a treatment effects approach. RESULTS: In total, 4,010 infants were included; 1,306 were enrolled into the cohort study and 2,704 were enrolled into the RCT. No significant differences in mortality were observed between the two study settings (HR: 1.28, 95% CI: 0.76, 2.13), but RCT participants had a lower risk of hospitalization (HR: 0.72, 95% CI: 0.58, 0.89) that decreased with age. However, RCT participants had a higher risk of hospitalization within the first six months of life. The causal risk difference in hospitalizations attributable to the RCT setting was -0.03 (95% CI: -0.05, -0.01). CONCLUSIONS: Children in an RCT with rigorous application of national standard of care guidelines experienced a significantly lower risk of hospitalization than children participating in a cohort study that did not alter clinical care. Future research is needed to further investigate outcome disparities when real-world results fail to mirror those achieved in a clinical trial. Trial registration The Mpepu Trial was funded by the U.S. National Institutes of Health (No. NCT01229761) and the Maikaelelo Study was funded primarily by the U.S. Centers for Disease Control and Prevention (32AI007433-21).