Quasi-experimental evaluations of health-related policies in Europe: a scoping review

BACKGROUND: Quasi-experimental methods (QEM) are study designs increasingly used in public health to evaluate the impact of health-focused policies. We sought to examine the scope, methodological characteristics as well as health equity considerations of policy evaluations using QEM that assess health and social outcomes in Europe. METHODS: We conducted a scoping review as outlined in our pre-registered protocol (https://osf.io/n5rvg). We searched Medline, EMBASE, and EconLit databases in December 2022 and included studies labelled as interrupted time series, difference-in-difference (DID), controlled before and after, regression discontinuity, and synthetic control studies; that focused on policies implemented at regional, national, and supranational level in the WHO European region. Among all identified records, we drew a random sample of 350 records for title and abstract screening and subsequent full-text screening, data extraction and analysis. RESULTS: Database searches rendered 2102 records. Of the random sample of 350 records, we identified 59 records as eligible for inclusion. Most studies evaluated health and social policies implemented on a national level in EU member states and the UK. Common policy topics were related to tobacco control, healthcare co-payments, and employment. DID was the most common study design label. Few studies were based on pre-registered protocols. There was substantial variability in the number and types of additional analyses conducted to assess the appropriateness of assumptions and the robustness of results. With respect to health equity (HE), 16 included studies had a strong HE focus by examining disadvantaged populations, and in 23 and 18 studies, respectively, HE was discussed and HE-related recommendations were made. DISCUSSION: QEM were frequently used to undertake policy evaluations and studies showed great diversity in scope and methodology. Health equity was considered in a surprisingly low number of studies. KEY MESSAGES: • We identified a large number of policy evaluations employing quasi-experimental methods. • Studies were heterogeneous in scope and methodological rigour and few considered health equity.