Methods for estimating causal relationships of adverse events with dietary supplements

OBJECTIVE: Dietary supplement use has increased over past decades, resulting in reports of potentially serious adverse events. The aim of this study was to develop optimised methods to evaluate the causal relationships between adverse events and dietary supplements, and to test these methods using case reports. DESIGN: Causal relationship assessment using prospectively collected data. SETTING AND PARTICIPANTS: 4 dietary supplement experts, 4 pharmacists and 11 registered dietitians (5 men and 14 women) examined 200 case reports of suspected adverse events using the modified Naranjo scale and the modified Food and Drug Administration (FDA) algorithm. PRIMARY OUTCOME MEASURES: The distribution of evaluation results was analysed and inter-rater reliability was evaluated for the two modified methods employed using intraclass correlation coefficients (ICC) and Fleiss’ κ. RESULTS: Using these two methods, most of the 200 case reports were categorised as ‘lack of information’ or ‘possible’ adverse events. Inter-rater reliability among entire assessors ratings for the two modified methods, based on ICC and Fleiss’ κ, were classified as more than substantial (modified Naranjo scale: ICC (95% CI) 0.873 (0.850 to 0.895); Fleiss’ κ (95% CI) 0.615 (0.615 to 0.615). Modified FDA algorithm: Fleiss’ κ (95% CI) 0.622 (0.622 to 0.622). CONCLUSIONS: These methods may help to assess the causal relationships between adverse events and dietary supplements. By conducting additional studies of these methods in different populations, researchers can expand the possibilities for the application of our methods.