Impact of different cover letter information and incentives on Veterans’ emotional responses to an unsolicited mailed survey about military traumas: a randomized, 3x2x2 factorial trial

BACKGROUND: Altering cover letter information to reduce non-response bias in trauma research could inadvertently leave survey participants unprepared for potentially upsetting questions. In an unsolicited, mailed survey, we assessed participants’ change in affect post-survey after altering key cover letter information and promising different incentives. We tested direct and indirect effects of participants carefully reading the cover letter on changes in their affect post-survey. METHODS: In a 3X2X2 randomized, factorial trial, 480 male and 480 female, nationally representative Veterans who were applying for posttraumatic stress disorder disability benefits were randomized to receive one of 12 different cover letters. The cover letters provided general versus more explicit information about the survey’s trauma content and how their names were selected for study; we also promised different incentives for returning the survey. The main outcome was change in affect post-survey. We examined five potential moderators: combat or military sexual trauma exposure, posttraumatic stress disorder or serious mental illness diagnosis, and recency of military service. Mediators between reading the cover letter carefully and post-survey affect included how participants rated the cover letters’ information and whether they thought the cover letters prepared them for the survey’s content. A Bonferroni corrected alpha of 0.003 was the threshold for statistical significance. RESULTS: One hundred ninety men and 193 women reported their pre-and post-survey affect. Across all study conditions, out of 16 possible points, the net change in affect post-survey was less than a quarter-point for men and women. Mean changes in post-survey affect did not differ statistically significantly across any of the study factors (ps > 0.06); nor were there statistically significant interactions between any of the study factors and the 5 moderators after accounting for multiple comparisons (ps > 0.02). After controlling for pre-survey affect, reading the cover letter carefully had small effects on changes in post-survey affect, with larger associations seen in the women compared to men. Mediators’ effects were often in opposite directions for men and women. CONCLUSION: General descriptions of a survey’s trauma content appear ethically defensible. Research on cover letters’ impacts on survey participants’ emotional reactions and how those impacts differ by gender is needed. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01783-7.