What makes health impact assessments successful? Factors contributing to effectiveness in Australia and New Zealand

BACKGROUND: While many guidelines explain how to conduct Health Impact Assessments (HIAs), less is known about the factors that determine the extent to which HIAs affect health considerations in the decision making process. We investigated which factors are associated with increased or reduced effectiveness of HIAs in changing decisions and in the implementation of policies, programs or projects. This study builds on and tests the Harris and Harris-Roxas’ conceptual framework for evaluating HIA effectiveness, which emphasises context, process and output as key domains. METHODS: We reviewed 55 HIA reports in Australia and New Zealand from 2005 to 2009 and conducted surveys and interviews for 48 of these HIAs. Eleven detailed case studies were undertaken using document review and stakeholder interviews. Case study participants were selected through purposeful and snowball sampling. The data were analysed by thematic content analysis. Findings were synthesised and mapped against the conceptual framework. A stakeholder forum was utilised to test face validity and practical adequacy of the findings. RESULTS: We found that some features of HIA are essential, such as the stepwise but flexible process, and evidence based approach. Non-essential features that can enhance the impact of HIAs include capacity and experience; ‘right person right level’; involvement of decision-makers and communities; and relationships and partnerships. There are contextual factors outside of HIA such as fit with planning and decision making context, broader global context and unanticipated events, and shared values and goals that may influence a HIA. Crosscutting factors include proactive positioning, and time and timeliness. These all operate within complex open systems, involving multiple decision-makers, levels of decision-making, and points of influence. The Harris and Harris-Roxas framework was generally supported. CONCLUSION: We have confirmed previously identified factors influencing effectiveness of HIA and identified new factors such as proactive positioning. Our findings challenge some presumptions about ‘right’ timing for HIA and the rationality and linearity of decision-making processes. The influence of right timing on decision making needs to be seen within the context of other factors such as proactive positioning. This research can help HIA practitioners and researchers understand and identify what can be enhanced within the HIA process. Practitioners can adapt the flexible HIA process to accommodate the external contextual factors identified in this report. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12889-015-2319-8) contains supplementary material, which is available to authorized users.