Developing the Message Assessment Scale for Tobacco Prevention Campaigns: Cross-sectional Validation Study

BACKGROUND: Mass media campaigns are effective for influencing a broad range of health behaviors. Prior to launching a campaign, developers often conduct ad testing to help identify the strengths and weaknesses of the message executions among the campaign’s target audience. This process allows for changes to be made to ads, making them more relevant to or better received by the target audience before they are finalized. To assess the effectiveness of an ad’s message and execution, campaign ads are often rated using a single item or multiple items on a scale, and scores are calculated. Endorsement of a 6-item perceived message effectiveness (PME) scale, defined as the practice of using a target audience’s evaluative ratings to inform message selection, is one approach commonly used to select messages for antitobacco campaigns; however, the 6-item PME scale often does not produce enough specificity to make important decisions on ad optimization. In addition, the PME scale is typically used with adult populations for smoking cessation messages. OBJECTIVE: This study includes the development of the Message Assessment Scale, a new tobacco prevention message testing scale for youth and young adults. METHODS: Data were derived from numerous cross-sectional surveys designed to test the relevance and potential efficacy of antitobacco truth campaign ads. Participants aged 15-24 years (N=6108) responded to a set of 12 core attitudinal items, including relevance (both personal and cultural) as well as comprehension of the ad’s main message. RESULTS: Analyses were completed in two phases. In phase I, mean scores were calculated for each of the 12 attitudinal items by ad type, with higher scores indicating more endorsement of the item. Next, all items were submitted to exploratory factor analysis. A four-factor model fit was revealed and verified with confirmatory factor analysis, resulting in the following constructs: personally relevant, culturally relevant, the strength of messaging, and negative attributes. In phase II, ads were categorized by performance (high/medium/low), and constructs identified in phase I were correlated with key campaign outcomes (ie, main fact agreement and likelihood to vape). Phase II confirmed that the four constructs identified in phase I were all significantly correlated with main fact agreement and vape intentions. CONCLUSIONS: Findings from this study advance the field by establishing an expanded set of validated items to comprehensively assess the potential effectiveness of advertising executions. This set of items expands the portfolio of ad testing measures for ads focused on tobacco use prevention. Findings can inform how best to optimize ad executions and message delivery for health behavior campaigns, particularly those focused on tobacco use prevention among youth and young adult populations.