Effect of Smartphone App–Based Education on Clinician Prescribing Habits in a Learning Health Care System: A Randomized Cluster Crossover Trial

IMPORTANCE: Effective methods for engaging clinicians in continuing education for learning-based practice improvement remain unknown. OBJECTIVE: To determine whether a smartphone-based app using spaced education with retrieval practice is an effective method to increase evidence-based practice. DESIGN, SETTING, AND PARTICIPANTS: A prospective, unblinded, single-center, crossover randomized clinical trial was conducted at a single academic medical center from January 6 to April 24, 2020. Vanderbilt University Medical Center clinicians prescribing intravenous fluids were invited to participate in this study. INTERVENTIONS: All clinicians received two 4-week education modules: 1 on prescribing intravenous fluids and 1 on prescribing opioid and nonopioid medications (counterbalancing measure), over a 12-week period. The order of delivery was randomized 1:1 such that 1 group received the fluid management module first, followed by the pain management module after a 4-week break, and the other group received the pain management module first, followed by the fluid management module after a 4-week break. MAIN OUTCOMES AND MEASURES: The primary outcome was evidence-based clinician prescribing behavior concerning intravenous fluids in the inpatient setting and pain medication prescribing on discharge from the hospital. RESULTS: A total of 354 participants were enrolled and randomized, with 177 in group 1 (fluid then pain management education) and 177 in group 2 (pain management then fluid education). During the overall study period, 16 868 questions were sent to 349 learners, with 11 783 (70.0%) being opened: 10 885 (92.4%) of those opened were answered and 7175 (65.9%) of those answered were answered correctly. The differences between groups changed significantly over time, indicated by the significant interaction between educational intervention and time (P = .002). Briefly, at baseline evidence-concordant IV fluid ordered 7.2% less frequently in group 1 than group 2 (95% CI, −19.2% to 4.9%). This was reversed after training at 4% higher (95% CI, −8.2% to 16.0%) in group 1 than group 2, a more than doubling in the odds of evidence-concordant ordering (OR, 2.56, 95% CI, 0.80-8.21). Postintervention, all gains had been reversed with less frequent ordering in group 1 than group 2 (−9.5%, 95% CI, −21.6% to 2.7%). There was no measurable change in opioid prescribing behaviors at any time point. CONCLUSIONS AND RELEVANCE: In this randomized clinical trial, use of smartphone app learning modules resulted in statistically significant short-term improvement in some prescribing behaviors. However, this effect was not sustained over the long-term. Additional research is needed to understand how to sustain improvements in care delivery as a result of continuous professional development at the institutional level. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03771482