Bridging theory and practice: design and implementation of the NORC-SSP linkage evaluation

PURPOSE: Despite the substantial progress that has been made in the development of community-based service systems for seniors, effective linkages with the health care system remain elusive. Supportive service programs operating in naturally occurring retirement communities (NORCs) where large numbers of seniors reside have begun to address this critical gap in the health and social service system; however, linkages with the health care system are neither fluid nor systematic. To address this problem, two New York city-based funders established the NORC-Health Care Linkage Project to foster the development and testing of new models of collaborative and coordinated problem-solving between community-based social service providers and key health providers. This paper reports on an innovative evaluation design and implementation strategies that build organizational capacity while effectively assessing multiple levels of outcomes. CONCEPTUAL FRAMEWORK: Randomized controlled designs which compare intervention communities with controls are unfeasible, unmanageable and inappropriate for many community-based initiatives that have multiple sites and varied goals and interventions. Theories of action (TOAs) refer to specification of the underlying assumptions about how a program is expected to get from conditions at baseline to a desired future. In this way, TOAs bridge strategic planning and evaluation. To be effective TOAs should be co-generated by evaluators and those being evaluated. Working collaboratively fosters ownership to the components of the theory and enables individual organizations to develop their own theories. Such collaboration between evaluators and practitioners builds organizational capacity for applied outcomes research and encourages evidence-based program planning. METHODS: All four funded linkage partnerships worked with the evaluation team to co-create action plans and logic maps that built upon TOAs articulated by the participating partnerships. Specific interventions focused on falls, discharge planning, depression, and, diabetes. Using a case study approach, process, impact and distal outcomes were tracked from 2002 to 2006 using tracking forms, client surveys, stakeholder interviews and period site visits. Intensive technical assistance enabled the participating sites to take an active role in data collection and analysis. FINDINGS: Findings suggest that internal capacity to collect, analyze and utilize data increased at all four participating partnerships, with the greatest improvement seen among the community-based organizations. Interventions were successfully implemented and comprehensively monitored. Theory-based predictions of multiple levels of outcomes were demonstrated across a number of domains, including improved shared care planning and continuity of care. DISCUSSION: Most published community health promotion programs have targeted a single disease and have been initiated ‘top-down,’ often with a university or consulting firm in charge. This innovative initiative aimed to build capacity for program design, implementation and evaluation within organizations so that participants could continue to apply the new skills after the funder and external consultants had gone back to their respective professional homes. This approach was successful in that participating organizations found value in tracking their own progress, both for accountability purposes but also for future programming.