Perceived communication effectiveness in implementation strategies: a measurement scale

BACKGROUND: Communication-based activities and products (i.e., training programs, webinars) are a critical component of implementation strategies that relay information to various audiences. Audience perceptions of communication effectiveness contribute important insight into the processes and mechanisms through which an implementation effort may succeed or fail. To advance research on this front, a psychometrically sound instrument for measuring perceived communication effectiveness (PCE) is needed. METHODS: An expert panel identified the theoretical foundations and conceptual domains of PCE and drafted preliminary items. Five focus groups of correctional professionals who had recently completed an implementation leadership training reviewed the items and provided feedback for refinement. Revised items were then included in a survey-based evaluation of an ongoing eLearning curriculum designed to improve the practices used by front-line probation officers in supervising individuals in the field. The factorial structure of a final 6-item scale as well as its convergent, divergent, and predictive validity was evaluated using data from the evaluation surveys (N(follow-up) = 358, N(baseline+follow-up) = 159). RESULTS: Confirmatory factor analysis of the final scale of PCE demonstrated adequate fit. PCE was strongly correlated with measures of implementation outcomes (acceptability, r = .819, p < .001; appropriateness, r = .809, p < .001; and feasibility, r = .754, p < .001), yet uncorrelated with a scale of need to evaluate (r = − .051, p = .422), demonstrating both convergent and divergent validities. The predictive validity of PCE was evidenced by significant associations between PCE and key training outcomes, including perceived staff use of evidence-based practices (β = .230, p < .05), agency climate (β = .261, p < .05), and value concordance (β = .209, p < .05), after controlling for baseline values and other confounders. CONCLUSIONS: The PCE scale is psychometrically sound and can be a useful tool for gauging audience receptivity to and the potential impact of communication-based implementation activities and products. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-022-00284-4.