The role of qualitative evaluation research in effect evaluation

In the Netherlands six institutes representing prevention, health care and social welfare established an assessment system to evaluate the quality, effectiveness and feasibility of interventions. Together the institutes present in their portals 650 practices at three different assessment levels (well described, theoretically sound and effective). Ninety interventions are assessed as effective with only 15 interventions on the highest level (strong indications on effectiveness). In the assessment system the criteria for effectiveness have a strong focus on quantitative designs such as randomized-controlled trial, quasi -experimental and observational study designs. This favors ‘simple’ interventions and disadvantaged more complex intervention implemented in the domains of social welfare and health promotion. Not every intervention effect can be captured in a numerical outcome measure. To address this objection institutes decided to study the possibility to evaluate the effects of interventions also with qualitative research and to add criteria for qualitative research. To support the assessment of the validity and reliability of the qualitative evaluation studies a checklist for qualitative evaluation research and mixed methods was developed. With this checklist the external reviewer tests the extent to which methodological applications and analytical procedures have been followed. The result is a score of the validity/reliability of the relevant study to the effects of an intervention. This checklist supports the external reviewer in the decision for the effectiveness of an intervention. In this workshop we present this checklist for qualitative research and mixed methods, the addition of criteria in the assessment system of interventions and the first results of the pilot of the assessment of an intervention with a qualitative evaluation study.