Fidelity of implementation: development and testing of a measure

BACKGROUND: Along with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While studies often assume that interventions are used in clinical practice exactly as originally designed, this may not be the case. Thus, fidelity of an intervention's implementation reflects how an intervention is, or is not, used in clinical practice and is an important factor in understanding intervention effectiveness and in replicating the intervention in dissemination efforts. The purpose of this paper is to contribute to the understanding of implementation science by (a) proposing a methodology for measuring fidelity of implementation (FOI) and (b) testing the measure by examining the association between FOI and intervention effectiveness. METHODS: We define and measure FOI based on organizational members' level of commitment to using the distinct components that make up an intervention as they were designed. Semistructured interviews were conducted among 18 organizational members in four medical centers, and the interviews were analyzed qualitatively to assess three dimensions of commitment to use--satisfaction, consistency, and quality--and to develop an overall rating of FOI. Mixed methods were used to explore the association between FOI and intervention effectiveness (inpatient resource utilization and mortality). RESULTS: Predictive validity of the FOI measure was supported based on the statistical significance of FOI as a predictor of intervention effectiveness. The strongest relationship between FOI and intervention effectiveness was found when an alternative measure of FOI was utilized based on individual intervention components that had the greatest variation across medical centers. CONCLUSIONS: In addition to contextual factors, implementation research needs to consider FOI as an important factor in influencing intervention effectiveness. Our proposed methodology offers a systematic means for understanding organizational members' use of distinct intervention components, assessing the reasons for variation in use across components and organizations, and evaluating the impact of FOI on intervention effectiveness.