Comparing single and multiple imputation strategies for harmonizing substance use data across HIV-related cohort studies

BACKGROUND: Although standardized measures to assess substance use are available, most studies use variations of these measures making it challenging to harmonize data across studies. The aim of this study was to evaluate the performance of different strategies to impute missing substance use data that may result as part of data harmonization procedures. METHODS: We used self-reported substance use data collected between August 2014 and June 2019 from 528 participants with 2,389 study visits in a cohort study of substance use and HIV. We selected a low (heroin), medium (methamphetamine), and high (cannabis) prevalence drug and set 10–50% of each substance to missing. The data amputation mimicked missingness that results from harmonization of disparate measures. We conducted Monte Carlo simulations to evaluate the comparative performance of single and multiple imputation (MI) methods using the relative mean bias, root mean square error (RMSE), and coverage probability of the 95% confidence interval for each imputed estimate. RESULTS: Without imputation (i.e., listwise deletion), estimates of substance use were biased, especially for low prevalence outcomes such as heroin. For instance, even when 10% of data were missing, the complete case analysis underestimated the prevalence of heroin by 33%. MI, even with as few as five imputations produced the least biased estimates, however, for a high prevalence outcome such as cannabis with low to moderate missingness, performance of single imputation strategies improved. For instance, in the case of cannabis, with 10% missingness, single imputation with regression performed just as well as multiple imputation resulting in minimal bias (relative mean bias of 0.06% and 0.07% respectively) and comparable performance (RMSE = 0.0102 for both and coverage of 95.8% and 96.2% respectively). CONCLUSION: Our results from imputation of missing substance use data resulting from data harmonization indicate that MI provided the best performance across a range of conditions. Additionally, single imputation for substance use data performed comparably under scenarios where the prevalence of the outcome was high and missingness was low. These findings provide a practical application for the evaluation of several imputation strategies and helps to address missing data problem when combining data from individual studies.