The Effectiveness of Web-Based vs. Non-Web-Based Interventions: A Meta-Analysis of Behavioral Change Outcomes

BACKGROUND: A primary focus of self-care interventions for chronic illness is the encouragement of an individual's behavior change necessitating knowledge sharing, education, and understanding of the condition. The use of the Internet to deliver Web-based interventions to patients is increasing rapidly. In a 7-year period (1996 to 2003), there was a 12-fold increase in MEDLINE citations for “Web-based therapies.” The use and effectiveness of Web-based interventions to encourage an individual's change in behavior compared to non-Web-based interventions have not been substantially reviewed. OBJECTIVE: This meta-analysis was undertaken to provide further information on patient/client knowledge and behavioral change outcomes after Web-based interventions as compared to outcomes seen after implementation of non-Web-based interventions. METHODS: The MEDLINE, CINAHL, Cochrane Library, EMBASE, ERIC, and PSYCHInfo databases were searched for relevant citations between the years 1996 and 2003. Identified articles were retrieved, reviewed, and assessed according to established criteria for quality and inclusion/exclusion in the study. Twenty-two articles were deemed appropriate for the study and selected for analysis. Effect sizes were calculated to ascertain a standardized difference between the intervention (Web-based) and control (non-Web-based) groups by applying the appropriate meta-analytic technique. Homogeneity analysis, forest plot review, and sensitivity analyses were performed to ascertain the comparability of the studies. RESULTS: Aggregation of participant data revealed a total of 11,754 participants (5,841 women and 5,729 men). The average age of participants was 41.5 years. In those studies reporting attrition rates, the average drop out rate was 21% for both the intervention and control groups. For the five Web-based studies that reported usage statistics, time spent/session/person ranged from 4.5 to 45 minutes. Session logons/person/week ranged from 2.6 logons/person over 32 weeks to 1008 logons/person over 36 weeks. The intervention designs included one-time Web-participant health outcome studies compared to non-Web participant health outcomes, self-paced interventions, and longitudinal, repeated measure intervention studies. Longitudinal studies ranged from 3 weeks to 78 weeks in duration. The effect sizes for the studied outcomes ranged from -.01 to .75. Broad variability in the focus of the studied outcomes precluded the calculation of an overall effect size for the compared outcome variables in the Web-based compared to the non-Web-based interventions. Homogeneity statistic estimation also revealed widely differing study parameters (Q(w16) = 49.993, P ≤ .001). There was no significant difference between study length and effect size. Sixteen of the 17 studied effect outcomes revealed improved knowledge and/or improved behavioral outcomes for participants using the Web-based interventions. Five studies provided group information to compare the validity of Web-based vs. non-Web-based instruments using one-time cross-sectional studies. These studies revealed effect sizes ranging from -.25 to +.29. Homogeneity statistic estimation again revealed widely differing study parameters (Q(w4) = 18.238, P ≤ .001). CONCLUSIONS: The effect size comparisons in the use of Web-based interventions compared to non-Web-based interventions showed an improvement in outcomes for individuals using Web-based interventions to achieve the specified knowledge and/or behavior change for the studied outcome variables. These outcomes included increased exercise time, increased knowledge of nutritional status, increased knowledge of asthma treatment, increased participation in healthcare, slower health decline, improved body shape perception, and 18-month weight loss maintenance.