Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies

BACKGROUND: Digital health interventions have demonstrated efficacy for several conditions including for pediatric chronic pain. However, the process of making interventions available to end users in an efficient and sustained way is challenging and remains a new area of research. To advance this field, comprehensive frameworks have been created. OBJECTIVE: The aim of this study is to compare the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) and Behavior Interventions using Technology (BIT) frameworks with data collected from the web-based management of adolescent pain (WebMAP Mobile; WMM) randomized controlled trial (RCT). METHODS: We conducted a hybrid effectiveness-implementation cluster RCT with a stepped wedge design in which the intervention was sequentially implemented in 8 clinics, following a usual care period. Participants were 143 youths (mean age 14.5 years, SD 1.9; 117/143, 81.8% female) with chronic pain, from which 73 were randomized to receive the active intervention. Implementation outcomes were assessed using the RE-AIM and BIT frameworks. RESULTS: According to the RE-AIM framework, the WMM showed excellent reach, recruiting a sample 19% larger than the size originally planned and consenting 79.0% (143/181) of eligible referred adolescents. Effectiveness was limited, with only global impression of change showing significantly greater improvements in the treatment group; however, greater treatment engagement was associated with greater reductions in pain and disability. Adoption was excellent (all the invited clinics participated and referred patients). Implementation was acceptable, showing good user engagement and moderate adherence and positive attitudes of providers. Costs were similar to planned, with a 7% increase in funds needed to make the WMM publicly available. Maintenance was evidenced by 56 new patients downloading the app during the maintenance period and by all clinics agreeing to continue making referrals and all, but one, making new referrals. According to the BIT, 82% (60/73) of adolescents considered the treatment acceptable. In terms of adoption, 93% (68/73) downloaded the app, and all of them used it after their first log-in. In terms of appropriateness at the user level, 2 participants were unable to download the app. Perceptions of the appearance, navigation, and theme were positive. Providers perceived the WMM as a good fit for their clinic, beneficial, helpful, and resource efficient. In terms of feasibility, no technical issues were reported. In terms of fidelity, 40% (29/73) completed the treatment. Implementation costs were 7% above the budget. With regard to penetration, 56 new users accessed the app during the maintenance period. In terms of sustainability, 88% (7/8) of clinics continued recommending the WMM after the end of the study. CONCLUSIONS: For the first time, a real-world digital health intervention was used as a proof of concept to test all the domains in the RE-AIM and BIT frameworks, allowing for comparisons. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1016/j.cct.2018.10.003