Applications of discrete choice experiments in COVID-19 research: Disparity in survey qualities between health and transport fields

Published choice experiments linked to various aspects of the COVID-19 pandemic are analysed in a rapid review. The aim is to (i) document the diversity of topics as well as their temporal and geographical patterns of emergence, (ii) compare various elements of design quality across different sectors of applied economics, and (iii) identify potential signs of convergent validity across findings of comparable experiments. Of the N = 43 published choice experiments during the first two years of the pandemic, the majority identifies with health applications (n = 30), followed by transport-related applications (n = 10). Nearly 100,000 people across the world responded to pandemic-related discrete choice surveys. Within health applications, while the dominant theme, up until June 2020, was lockdown relaxation and tracing measures, the focus shifted abruptly to vaccine preference since then. Geographical origins of the health surveys were not diverse. Nearly 50% of all health surveys were conducted in only three countries, namely US, China and The Netherlands. Health applications exhibited stronger pre-testing and larger sample sizes compared to transport applications. Limited signs of convergent validity were identifiable. Within some applications, issues of temporal instability as well as hypothetical bias attributable to social desirability, protest response or policy consequentiality seemed likely to have affected the findings. Nevertheless, very few of the experiments implemented measures of hypothetical bias mitigation and those were limited to health studies. Our main conclusion is that swift administration of pandemic-related choice experiments has overall resulted in certain degrees of compromise in study quality, but this has been more so the case in relation to transport topics than health topics.