Economic evaluation of policy initiatives in the organisation and delivery of healthcare: a case study of gastroenterology endoscopy services

BACKGROUND: Complex clinical interventions are increasingly subject to evaluation by randomised trial linked to economic evaluation. However evaluations of policy initiatives tend to eschew experimental designs in favour of interpretative perspectives which rarely allow the economic evaluation methods used in clinical trials. As evidence of the cost effectiveness of such initiatives is critical in informing policy, it is important to explore whether conventional economic evaluation methods apply to experimental evaluations of policy initiatives. METHODS: We used mixed methods based on a quasi-experimental design to evaluate a policy initiative whose aim was to expedite the modernisation of gastroenterology endoscopy services in England. We compared 10 sites which had received funding and support to modernise their endoscopy services with 10 controls. We collected data from five waves of patients undergoing endoscopy. The economic component of the study compared sites by levels of investment in modernisation and patients’ use of health service resources, time off work and health related quality of life. RESULTS: We found no statistically significant difference between intervention and control sites in investment in modernisation or any patient outcome including health. CONCLUSIONS: This study highlights difficulties in applying the rigour of a randomised trial and associated technique of economic evaluation to a policy initiative. It nevertheless demonstrates the feasibility of using this approach although further work is needed to demonstrate its generalisability in other applications. The present application shows that the small incentives offered to intervention sites did not enhance modernisation of gastroenterology endoscopy services or improve patient outcomes.