Evaluating an organization-wide disparity reduction program: Understanding what works for whom and why

BACKGROUND: Disparity-reduction programs have been shown to vary in the degree to which they achieve their goal; yet the causes of these variations is rarely studied. We investigated a broad-scale program in Israel’s largest health plan, aimed at reducing disparities in socially disadvantaged groups using a composite measure of seven health and health care indicators. METHODS: A realistic evaluation was conducted to evaluate the program in 26 clinics and their associated managerial levels. First, we performed interviews with key stakeholders and an ethnographic observation of a regional meeting to derive the underlying program theory. Next, semi-structured interviews with 109 clinic teams, subregional headquarters, and regional headquarters personnel were conducted. Social network analysis was performed to derive measures of team interrelations. Perceived team effectiveness (TE) and clinic characteristics were assessed to elicit contextual characteristics. Interventions implemented by clinics were identified from interviews and coded according to the mechanisms each clinic employed. Assessment of each clinic’s performance on the seven-indicator composite measure was conducted at baseline and after 3 years. Finally, we reviewed different context-mechanism-outcome (CMO) configurations to understand what works to reduce disparity, and under what circumstances. RESULTS: Clinics’ inner contextual characteristics varied in both network density and perceived TE. Successful CMO configurations included 1) highly dense clinic teams having high perceived TE, only a small gap to minimize, and employing a wide range of interventions; (2) clinics with a large gap to minimize with high clinic density and high perceived TE, focusing efforts on tailoring services to their enrollees; and (3) clinics having medium to low density and perceived TE, and strong middle-management support. CONCLUSIONS: Clinics that achieved disparity reduction had high clinic density, close ties with middle management, and tailored interventions to the unique needs of the populations they serve.