The Comparative Effectiveness of Mobile Phone Interventions in Improving Health Outcomes: Meta-Analytic Review

BACKGROUND: As mobile technology continues expanding, researchers have been using mobile phones to conduct health interventions (mobile health—mHealth—interventions). The multiple features of mobile phones offer great opportunities to disseminate large-scale, cost-efficient, and tailored messages to participants. However, the interventions to date have shown mixed results, with a large variance of effect sizes (Cohen d=−0.62 to 1.65). OBJECTIVE: The study aimed to generate cumulative knowledge that informs mHealth intervention research. The aims were twofold: (1) to calculate an overall effect magnitude for mHealth interventions compared with alternative interventions or conditions, and (2) to analyze potential moderators of mHealth interventions’ comparative efficacy. METHODS: Comprehensive searches of the Communication & Mass Media Complete, PsycINFO, Web of Knowledge, Academic Search Premier, PubMed and MEDLINE databases were conducted to identify potentially eligible studies in peer-reviewed journals, conference proceedings, and dissertations and theses. Search queries were formulated using a combination of search terms: “intervention” (Title or Abstract) AND “health” (Title or Abstract) AND “*phone*” OR “black-berr*” (OR mHealth OR “application*” OR app* OR mobile OR cellular OR “short messag*” OR palm* OR iPhone* OR MP3* OR MP4* OR iPod*) (Title or Abstract). Cohen d was computed as the basic unit of analysis, and the variance-weighted analysis was implemented to compute the overall effect size under a random-effects model. Analysis of variance–like and meta-regression models were conducted to analyze categorical and continuous moderators, respectively. RESULTS: The search resulted in 3424 potential studies, the abstracts (and full text, as necessary) of which were reviewed for relevance. Studies were screened in multiple stages using explicit inclusion and exclusion criteria, and citations were evaluated for inclusion of qualified studies. A total of 64 studies were included in the current meta-analysis. Results showed that mHealth interventions are relatively more effective than comparison interventions or conditions, with a small but significant overall weighted effect size (Cohen d=0.31). In addition, the effects of interventions are moderated by theoretical paradigm, 3 engagement types (ie, changing personal environment, reinforcement tracking, social presentation), mobile use type, intervention channel, and length of follow-up. CONCLUSIONS: To the best of our knowledge, this is the most comprehensive meta-analysis to date that examined the overall effectiveness of mHealth interventions across health topics and is the first study that statistically tested moderators. Our findings not only shed light on intervention design using mobile phones, but also provide new directions for research in health communication and promotion using new media. Future research scholarship is needed to examine the effectiveness of mHealth interventions across various health issues, especially those that have not yet been investigated (eg, substance use, sexual health), engaging participants using social features on mobile phones, and designing tailored mHealth interventions for diverse subpopulations to maximize effects.