Cargando…

Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic

Introduction: Diverse algorithms for signal detection exist. However, inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. We aimed to explore potential safety signals for two pneumococc...

Descripción completa

Detalles Bibliográficos
Autores principales: Lee, Hyesung, Kim, Ju Hwan, Choe, Young June, Shin, Ju-Young
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7349998/
https://www.ncbi.nlm.nih.gov/pubmed/32456068
http://dx.doi.org/10.3390/vaccines8020242
_version_ 1783557184929398784
author Lee, Hyesung
Kim, Ju Hwan
Choe, Young June
Shin, Ju-Young
author_facet Lee, Hyesung
Kim, Ju Hwan
Choe, Young June
Shin, Ju-Young
author_sort Lee, Hyesung
collection PubMed
description Introduction: Diverse algorithms for signal detection exist. However, inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. We aimed to explore potential safety signals for two pneumococcal vaccines in a spontaneous reporting database and compare the results and performances among the algorithms. Methods: Safety surveillance was conducted using the Korea national spontaneous reporting database from 1988 to 2017. Safety signals for pneumococcal vaccine and its subtypes were detected using the following the algorithms: disproportionality methods comprising of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC); empirical Bayes geometric mean (EBGM); and tree-based scan statistics (TSS). Moreover, the performances of these algorithms were measured by comparing detected signals with the known AEs or pneumococcal vaccines (reference standard). Results: Among 10,380 vaccine-related AEs, 1135 reports and 101 AE terms were reported following pneumococcal vaccine. IC generated the most safety signals for pneumococcal vaccine (40/101), followed by PRR and ROR (19/101 each), TSS (15/101), and EBGM (1/101). Similar results were observed for its subtypes. Cellulitis was the only AE detected by all algorithms for pneumococcal vaccine. TSS showed the best balance in the performance: the highest in accuracy, negative predictive value, and area under the curve (70.3%, 67.4%, and 64.2%). Conclusion: Discrepancy in the number of detected signals was observed between algorithms. EBGM and TSS calibrated noise better than disproportionality methods, and TSS showed balanced performance. Nonetheless, these results should be interpreted with caution due to a lack of a gold standard for signal detection.
format Online
Article
Text
id pubmed-7349998
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-73499982020-07-22 Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic Lee, Hyesung Kim, Ju Hwan Choe, Young June Shin, Ju-Young Vaccines (Basel) Article Introduction: Diverse algorithms for signal detection exist. However, inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. We aimed to explore potential safety signals for two pneumococcal vaccines in a spontaneous reporting database and compare the results and performances among the algorithms. Methods: Safety surveillance was conducted using the Korea national spontaneous reporting database from 1988 to 2017. Safety signals for pneumococcal vaccine and its subtypes were detected using the following the algorithms: disproportionality methods comprising of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC); empirical Bayes geometric mean (EBGM); and tree-based scan statistics (TSS). Moreover, the performances of these algorithms were measured by comparing detected signals with the known AEs or pneumococcal vaccines (reference standard). Results: Among 10,380 vaccine-related AEs, 1135 reports and 101 AE terms were reported following pneumococcal vaccine. IC generated the most safety signals for pneumococcal vaccine (40/101), followed by PRR and ROR (19/101 each), TSS (15/101), and EBGM (1/101). Similar results were observed for its subtypes. Cellulitis was the only AE detected by all algorithms for pneumococcal vaccine. TSS showed the best balance in the performance: the highest in accuracy, negative predictive value, and area under the curve (70.3%, 67.4%, and 64.2%). Conclusion: Discrepancy in the number of detected signals was observed between algorithms. EBGM and TSS calibrated noise better than disproportionality methods, and TSS showed balanced performance. Nonetheless, these results should be interpreted with caution due to a lack of a gold standard for signal detection. MDPI 2020-05-22 /pmc/articles/PMC7349998/ /pubmed/32456068 http://dx.doi.org/10.3390/vaccines8020242 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Lee, Hyesung
Kim, Ju Hwan
Choe, Young June
Shin, Ju-Young
Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title_full Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title_fullStr Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title_full_unstemmed Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title_short Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic
title_sort safety surveillance of pneumococcal vaccine using three algorithms: disproportionality methods, empirical bayes geometric mean, and tree-based scan statistic
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7349998/
https://www.ncbi.nlm.nih.gov/pubmed/32456068
http://dx.doi.org/10.3390/vaccines8020242
work_keys_str_mv AT leehyesung safetysurveillanceofpneumococcalvaccineusingthreealgorithmsdisproportionalitymethodsempiricalbayesgeometricmeanandtreebasedscanstatistic
AT kimjuhwan safetysurveillanceofpneumococcalvaccineusingthreealgorithmsdisproportionalitymethodsempiricalbayesgeometricmeanandtreebasedscanstatistic
AT choeyoungjune safetysurveillanceofpneumococcalvaccineusingthreealgorithmsdisproportionalitymethodsempiricalbayesgeometricmeanandtreebasedscanstatistic
AT shinjuyoung safetysurveillanceofpneumococcalvaccineusingthreealgorithmsdisproportionalitymethodsempiricalbayesgeometricmeanandtreebasedscanstatistic