Examining Predictors of Real-World User Engagement with Self-Guided eHealth Interventions: Analysis of Mobile Apps and Websites Using a Novel Dataset

BACKGROUND: The literature suggests that the product design of self-guided electronic health (eHealth) interventions impacts user engagement. Traditional trial settings, however, do not enable the examination of these relationships in real-world use. OBJECTIVE: This study aimed to examine whether the qualities of product design, research evidence, and publicly available data predict real-world user engagement with mobile and Web-based self-guided eHealth interventions. METHODS: This analysis included self-guided mobile and Web-based eHealth interventions available to the public—with their qualities assessed using the Enlight suite of scales. Scales included Usability, Visual Design, User Engagement, Content, Therapeutic Persuasiveness, Therapeutic Alliance, Credibility, and Research Evidence. Behavioral data on real-world usage were obtained from a panel that provides aggregated nonpersonal information on user engagement with websites and mobile apps, based on a time window of 18 months that was set between November 1, 2016 and April 30, 2018. Real-world user engagement variables included average usage time (for both mobile apps and websites) and mobile app user retention 30 days after download. RESULTS: The analysis included 52 mobile apps (downloads median 38,600; interquartile range [IQR] 116,000) and 32 websites (monthly unique visitors median 5689; IQR 30,038). Results point to moderate correlations between Therapeutic Persuasiveness, Therapeutic Alliance, and the 3 user engagement variables (.31≤rs≤.51; Ps≤.03). Visual Design, User Engagement, and Content demonstrated similar degrees of correlation with mobile app engagement variables (.25≤rs≤.49; Ps≤.04) but not with average usage time of Web-based interventions. Positive correlations were also found between the number of reviews on Google Play and average app usage time (r=.58; P<.001) and user retention after 30 days (r=.23; P=.049). Although several product quality ratings were positively correlated with research evidence, the latter was not significantly correlated with real-world user engagement. Hierarchical stepwise regression analysis revealed that either Therapeutic Persuasiveness or Therapeutic Alliance explained 15% to 26% of user engagement variance. Data on Google Play (number of reviews) explained 15% of the variance of mobile app usage time above Enlight ratings; however, publicly available data did not significantly contribute to explaining the variance of the other 2 user-engagement variables. CONCLUSIONS: Results indicate that the qualities of product design predict real-world user engagement with eHealth interventions. The use of real-world behavioral datasets is a novel way to learn about user behaviors, creating new avenues for eHealth intervention research.