Comparison of text messaging data collection vs face-to-face interviews for public health surveys: a cluster randomized crossover study of care-seeking for childhood pneumonia and diarrhoea in rural China

BACKGROUND: To compare text messaging and face-to-face interviews to conduct a survey on childhood diarrhoea and pneumonia. METHODS: Caregivers of young children able to send text messages in Zhao County in rural China were included in this crossover study. Villages (clusters) were randomized into two groups using the ratio 1:1.6 to account for an expected higher drop-out in group 2. In group 1, participants first completed the face-to-face and then text messaging survey; this order was reversed in group 2. We determined data equivalence of 17 questions that were answered by participants who were the same person in both surveys. For the text messaging survey, we assessed the overall and item response rate. RESULTS: We included 1014 participants between 16 and 28 March 2013: 371 in 15 villages in group 1 and 643 in 27 villages in group 2. A total of 662 (65.3%) out of 1014 participants responded (first text message question) and a significantly higher proportion who did not respond were from rural areas (P = 0.005). Of 651 participants willing to participate, 356 (54.7%) completed the text messaging survey, which was marginally significantly different between the groups (P = 0.05). In total, 409 participants took part in both surveys: 183 in group 1 and 226 in group 2. There was a significantly higher proportion of caregivers from rural areas in Zhao County in the non-responder group compared to the responder group (P = 0.004). Kappas were substantial for six (0.61–0.80), moderate for two (0.58 and 0.60), and fair for three questions (0.31, 0.35 and 0.37). The proportion of agreement was >90% for five questions; 80.0%-90.0% for five questions; 70.0%, 65.0% and 45.5%. The remaining questions had too small numbers to calculate these values. CONCLUSIONS: This study shows that text messaging data collection produces data similar to data from face-to-face interviews in a middle-income setting, but the response rate was insufficient for use in public health surveys. Improving the response rate is important, because text message surveys could be of greater value in rural remote areas due to the cost-saving potential.