Can targeted cover letters improve participation in health surveys? Results from a randomized controlled trial

BACKGROUND: Improving response rates in epidemiologic studies is important for the generalizability of the outcome. The aim of this study was to examine whether it can be advantageous for participation to target different versions of the cover letters to different sample subgroups. METHODS: A randomized trial was incorporated in a cross-sectional health survey in Denmark (n = 25,000) where a motivational sentence in the cover letter intended to heighten perceptions of relevance of the survey was varied among 11 sample subgroups (treatment groups). Ten different versions of a sentence outlining questionnaire themes were tested: each mentioned three out of five themes: stress, alcohol, sex, sleep problems, and contact with family and friends. An eleventh group, the control group, omitted this sentence. RESULTS: On average, the additional motivational sentence resulted in a significantly lower response rate overall compared to the control group. However, the additional motivational sentence was found to have heterogeneous effects on survey response. Furthermore, the nature of the heterogeneity differed between the versions of the sentence. Specifically, the additional sentence tended to produce a higher response rate among the youngest age group and a lower response rate in the oldest age group compared to the generic letter. The use of alcohol in the motivational sentence tended to have a positive effect on response in the age group 16–24 years, and stress tended to have a positive effect in the age group ≥65 years. On the contrary, sex tended to have a negative effect in the age groups 45–64 years and ≥ 65 years. However, a significant interaction was only found between the use of stress and age group (p = < 0.0001). CONCLUSION: The findings of significant and heterogeneous effects suggest that there is potential for a targeted approach to improve both response rates and sample composition. The uneven effect of the separate themes across age groups suggests that the selection of themes to be included in the motivational sentence is important for the use of targeted appeals to be successful and warrants further research to better identify which themes works in which contexts, in which subgroups and under which circumstances. TRIAL REGISTRATION: ClinicalTrials.gov ID: NCT03046368, retrospectively registered February 8th, 2017. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0799-4) contains supplementary material, which is available to authorized users.