Concordance of Text Message Ecological Momentary Assessment and Retrospective Survey Data Among Substance-Using Men Who Have Sex With Men: A Secondary Analysis of a Randomized Controlled Trial

BACKGROUND: Alcohol and illicit drug use is more prevalent among men who have sex with men (MSM) compared to the general population and has been linked to HIV transmission in this population. Research assessing individual patterns of substance use often utilizes questionnaires or interviews that rely on retrospective self-reported information, which can be subject to recall bias. Ecological momentary assessment (EMA) is a set of methods developed to mitigate recall bias by collecting data about subjects’ mental states and behaviors on a near real-time basis. EMA remains underutilized in substance use and HIV research. OBJECTIVE: To assess the concordance between daily reports of substance use collected by EMA text messages (short message service, SMS) and retrospective questionnaires and identify predictors of daily concordance in a sample of MSM. METHODS: We conducted a secondary analysis of EMA text responses (regarding behavior on the previous day) and audio computer-assisted self-interview (ACASI) survey data (14-day recall) from June 2013 to September 2014 as part of a randomized controlled trial assessing a pharmacologic intervention to reduce methamphetamine and alcohol use among nondependent MSM in San Francisco, California. Reports of daily methamphetamine use, alcohol use, and binge alcohol use (5 or more drinks on one occasion) were collected via EMA and ACASI and compared using McNemar’s tests. Demographic and behavioral correlates of daily concordance between EMA and ACASI were assessed for each substance, using separate multivariable logistic regression models, fit with generalized estimating equations. RESULTS: Among 30 MSM, a total of 994 days were included in the analysis for methamphetamine use, 987 for alcohol use, and 981 for binge alcohol use. Methamphetamine (EMA 20%, ACASI 11%, P<.001) and alcohol use (EMA 40%, ACASI 35%, P=.001) were reported significantly more frequently via EMA versus ACASI. In multivariable analysis, text reporting of methamphetamine (adjusted odds ratio 0.06, 95% CI 0.04-0.10), alcohol (0.48, 0.33-0.69), and binge alcohol use (0.27, 0.17-0.42) was negatively associated with daily concordance in the reporting of each respective substance. Compared to white participants, African American participants were less likely to have daily concordance in methamphetamine (0.15, 0.05-0.43) and alcohol (0.2, 0.05-0.54) reporting, and other participants of color (ie, Asian, Hispanic, multi-racial) were less likely to have daily concordance in methamphetamine reporting (0.34, 0.12-1.00). College graduates were more likely to have daily concordance in methamphetamine reporting (6.79, 1.84-25.04) compared to those with no college experience. CONCLUSIONS: We found that methamphetamine and alcohol use were reported more frequently with daily EMA texts compared to retrospective ACASI, concordance varied among different racial/ethnic subgroups and education levels, and reported substance use by EMA text was associated with lower daily concordance with retrospective ACASI. These findings suggest that EMA methods may provide more complete reporting of frequent, discrete behaviors such as substance use.