Operationalizing the reach, effectiveness, adoption, implementation, maintenance (RE-AIM) framework to evaluate the collective impact of autonomous community programs that promote health and well-being

BACKGROUND: The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is a useful tool for evaluating the impact of programs in community settings. RE-AIM has been applied to evaluate individual programs but seldom used to evaluate the collective impact of community-based, public health programming developed and delivered by multiple autonomous organizations. The purposes of this paper were to (a) demonstrate how RE-AIM can be operationalized and applied to evaluate the collective impact of similar autonomous programs that promote health and well-being and (b) provide preliminary data on the collective impact of Canadian spinal cord injury (SCI) peer mentorship programs on the delivery of peer mentorship services. METHODS: Criteria from all five RE-AIM dimensions were operationalized to evaluate multiple similar community-based programs. For this study, nine provincial organizations that serve people with SCI were recruited from across Canada. Organizations completed a structured self-report questionnaire and participated in a qualitative telephone interview to examine different elements of their peer mentorship program. Data were analyzed using summary statistics. RESULTS: Having multiple indicators to assess RE-AIM dimensions provided a broad evaluation of the impact of Canadian SCI peer mentorship programs. Peer mentorship programs reached 1.63% of the estimated Canadian SCI population. The majority (67%) of organizations tracked the effectiveness of peer mentorship through testimonials and reports. Setting-level adoption rates were high with 100% of organizations offering peer mentorship in community and hospital settings. On average, organizations allocated 10.4% of their operating budget and 9.8% of their staff to implement peer mentorship and 89% had maintained their programming for over 10 years. Full interpretation of the collective impact of peer mentorship programs was limited as complete data were only collected for 52% of survey questions. CONCLUSIONS: The lack of available organizational data highlights a significant challenge when using RE-AIM to evaluate the collective impact of multiple programs that promote health and well-being. Although researchers are encouraged to use RE-AIM to evaluate the collective impact of programs delivered by different organizations, documenting limitations and providing recommendations should be done to further the understanding of how best to operationalize RE-AIM in these contexts. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12889-019-7131-4) contains supplementary material, which is available to authorized users.