A Sequential Mixed-Mode Experiment in the U.S. National Monitoring the Future Study

The national Monitoring the Future (MTF) study examines substance use among adolescents and adults in the United States and has used paper questionnaires since it began in 1975. The current experiment tested three conditions as compared to the standard MTF follow-up protocol (i.e., MTF Control) for the first MTF follow-up survey at ages 19/20 years (i.e., one or two years after high school graduation). The MTF Control group included participants who completed in-school baseline surveys in the 12th grade in 2012–2013 and who were selected to participate in the first follow-up survey in 2014 (n = 2,451). A supplementary sample of participants who completed the 12th grade baseline survey in 2012 or 2013 but were not selected to participate in the main MTF follow-up (n = 4,950) were recruited and randomly assigned to one of three experimental conditions: (1) Mail Push, (2) Web Push, (3) Web Push + E-mail. Results indicated that the overall response rate was lower in Condition 2 compared to MTF Control and to Condition 1; there were no differences between Condition 3 and other conditions. Web response was highest in Condition 3; among web responders, smartphone response was also highest in Condition 3. Subgroup differences also emerged such that, for example, compared to white participants, Hispanics had greater odds of web (versus paper) response and blacks had greater odds of smartphone (versus computer or tablet) response. Item nonresponse was lowest in the Web Push conditions (compared to MTF Control) and on the web survey (compared to paper). Compared to MTF Control, Condition 3 respondents reported higher rates of alcohol use in the past 30 days. The total cost was lowest for Condition 3. Overall, the Condition 3 Web Push + E-mail design is promising. Future research is needed to continue to examine the implications of web and mobile response in large, national surveys.