Cargando…

A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases

OBJECTIVE: To develop and test a novel syndrome definition validation approach for rarely occurring diseases. INTRODUCTION: Early detection of rarely occurring but potentially harmful diseases such as bio-threat agents (e.g., anthrax), chemical agents (e.g., sarin), and naturally occurring diseases...

Descripción completa

Detalles Bibliográficos
Autores principales: Silva, Julio C., Shah, Shital C., Rumoro, Dino P., Hallock, Marilyn M., Gibbs, Gillian S., Waddell, Michael J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: University of Illinois at Chicago Library 2013
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3692924/
_version_ 1782274688042401792
author Silva, Julio C.
Shah, Shital C.
Rumoro, Dino P.
Hallock, Marilyn M.
Gibbs, Gillian S.
Waddell, Michael J.
author_facet Silva, Julio C.
Shah, Shital C.
Rumoro, Dino P.
Hallock, Marilyn M.
Gibbs, Gillian S.
Waddell, Michael J.
author_sort Silva, Julio C.
collection PubMed
description OBJECTIVE: To develop and test a novel syndrome definition validation approach for rarely occurring diseases. INTRODUCTION: Early detection of rarely occurring but potentially harmful diseases such as bio-threat agents (e.g., anthrax), chemical agents (e.g., sarin), and naturally occurring diseases (e.g., meningitis) is critical for rapid initiation of treatment, infection control measures, and emergency response plans. To facilitate clinicians’ ability to detect these diseases, various syndrome definitions have been developed. Due to the rarity of these diseases, standard statistical methodologies for validating syndrome definitions are not applicable. METHODS: Syndrome definitions were developed by researchers for the Geographic Utilization of Artificial Intelligence in Real-Time for Disease Identification and Alert Notification (GUARDIAN) surveillance system (1 1. Partition of literature articles: Literature articles that described positive cases were randomly divided to generate detection (75% of articles) and testing (25% of articles) syndrome definitions. 2. Synthetic case generation: Syndrome definitions and associated statistical measures were reverse engineered using probability of occurrence and inverse Gaussian function to generate potentially infinite positive artificial cases. 3. Clinical filter application: To avoid clinically incompatible combinations of newly generated symptoms, rules based on clinically guided knowledge from emergency department (ED) physicians were applied. Steps 2 and 3 were repeated for both detection and testing syndrome definitions. 4. a. ED negative case sample: Detection syndrome definitions were tested using a random sample of negative ED cases. Knowledge gained through false positive cases was utilized to modify the surveillance algorithms and system thresholds. b. 10-fold cross-validation: Standard 10-fold cross-validation on detection articles of positive cases and ED negative cases was utilized to generate performance metrics. Suspected cases were reviewed by ED clinicians for threshold enhancement. c. Literature articles (n=1): The ability of syndrome definitions to correctly flag literature articles with n=1 case was documented. 5. a. Testing sample: Synthetic positive cases generated from the testing articles along with another set of ED negative cases were evaluated by the respective syndrome definition. Suspected cases were clinically evaluated. b. Literature articles (n=1): Similar to detection step 4c, articles with n=1 were tested using syndrome definitions. c. True positive samples: When available, true positive cases from an ED were identified and sent through the GUARDIAN system. 6. Multi-syndrome validation: A combined sample of positive cases of multiple syndromes and ED negative cases were evaluated for detection of individual syndromes among other similar syndromes. RESULTS: To demonstrate the validation approach, the anthrax syndrome definition was utilized. This syndrome definition was developed with 25 articles containing positive anthrax cases used for detection, and the remaining 11 articles used for testing. With a 10-fold cross validation of the detection phase, the initial results showed accuracy was 99.4% (false positive rate of 0.65% and false negative rate of 0.00%). The testing phase initial validation revealed 99.2% accuracy for the anthrax syndrome definition. CONCLUSIONS: Syndrome specific synthetic samples that are validated through clinical filters allowed the generation of an unlimited number of positive cases. Correct identification by GUARDIAN of these cases indicates robust and reliable syndrome definitions. Utilization of these cases, in conjunction with adherence to a methodological process, was the cornerstone of the GUARDIAN syndrome definition validation approach. The validation approach was successfully demonstrated on anthrax and can be applied to other bio-threat agents, chemical agents, and naturally occurring diseases.
format Online
Article
Text
id pubmed-3692924
institution National Center for Biotechnology Information
language English
publishDate 2013
publisher University of Illinois at Chicago Library
record_format MEDLINE/PubMed
spelling pubmed-36929242013-06-26 A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases Silva, Julio C. Shah, Shital C. Rumoro, Dino P. Hallock, Marilyn M. Gibbs, Gillian S. Waddell, Michael J. Online J Public Health Inform ISDS 2012 Conference Abstracts OBJECTIVE: To develop and test a novel syndrome definition validation approach for rarely occurring diseases. INTRODUCTION: Early detection of rarely occurring but potentially harmful diseases such as bio-threat agents (e.g., anthrax), chemical agents (e.g., sarin), and naturally occurring diseases (e.g., meningitis) is critical for rapid initiation of treatment, infection control measures, and emergency response plans. To facilitate clinicians’ ability to detect these diseases, various syndrome definitions have been developed. Due to the rarity of these diseases, standard statistical methodologies for validating syndrome definitions are not applicable. METHODS: Syndrome definitions were developed by researchers for the Geographic Utilization of Artificial Intelligence in Real-Time for Disease Identification and Alert Notification (GUARDIAN) surveillance system (1 1. Partition of literature articles: Literature articles that described positive cases were randomly divided to generate detection (75% of articles) and testing (25% of articles) syndrome definitions. 2. Synthetic case generation: Syndrome definitions and associated statistical measures were reverse engineered using probability of occurrence and inverse Gaussian function to generate potentially infinite positive artificial cases. 3. Clinical filter application: To avoid clinically incompatible combinations of newly generated symptoms, rules based on clinically guided knowledge from emergency department (ED) physicians were applied. Steps 2 and 3 were repeated for both detection and testing syndrome definitions. 4. a. ED negative case sample: Detection syndrome definitions were tested using a random sample of negative ED cases. Knowledge gained through false positive cases was utilized to modify the surveillance algorithms and system thresholds. b. 10-fold cross-validation: Standard 10-fold cross-validation on detection articles of positive cases and ED negative cases was utilized to generate performance metrics. Suspected cases were reviewed by ED clinicians for threshold enhancement. c. Literature articles (n=1): The ability of syndrome definitions to correctly flag literature articles with n=1 case was documented. 5. a. Testing sample: Synthetic positive cases generated from the testing articles along with another set of ED negative cases were evaluated by the respective syndrome definition. Suspected cases were clinically evaluated. b. Literature articles (n=1): Similar to detection step 4c, articles with n=1 were tested using syndrome definitions. c. True positive samples: When available, true positive cases from an ED were identified and sent through the GUARDIAN system. 6. Multi-syndrome validation: A combined sample of positive cases of multiple syndromes and ED negative cases were evaluated for detection of individual syndromes among other similar syndromes. RESULTS: To demonstrate the validation approach, the anthrax syndrome definition was utilized. This syndrome definition was developed with 25 articles containing positive anthrax cases used for detection, and the remaining 11 articles used for testing. With a 10-fold cross validation of the detection phase, the initial results showed accuracy was 99.4% (false positive rate of 0.65% and false negative rate of 0.00%). The testing phase initial validation revealed 99.2% accuracy for the anthrax syndrome definition. CONCLUSIONS: Syndrome specific synthetic samples that are validated through clinical filters allowed the generation of an unlimited number of positive cases. Correct identification by GUARDIAN of these cases indicates robust and reliable syndrome definitions. Utilization of these cases, in conjunction with adherence to a methodological process, was the cornerstone of the GUARDIAN syndrome definition validation approach. The validation approach was successfully demonstrated on anthrax and can be applied to other bio-threat agents, chemical agents, and naturally occurring diseases. University of Illinois at Chicago Library 2013-04-04 /pmc/articles/PMC3692924/ Text en ©2013 the author(s) http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/ojphi/about/submissions#copyrightNotice This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
spellingShingle ISDS 2012 Conference Abstracts
Silva, Julio C.
Shah, Shital C.
Rumoro, Dino P.
Hallock, Marilyn M.
Gibbs, Gillian S.
Waddell, Michael J.
A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title_full A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title_fullStr A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title_full_unstemmed A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title_short A Novel Syndrome Definition Validation Approach for Rarely Occurring Diseases
title_sort novel syndrome definition validation approach for rarely occurring diseases
topic ISDS 2012 Conference Abstracts
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3692924/
work_keys_str_mv AT silvajulioc anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT shahshitalc anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT rumorodinop anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT hallockmarilynm anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT gibbsgillians anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT waddellmichaelj anovelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT silvajulioc novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT shahshitalc novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT rumorodinop novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT hallockmarilynm novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT gibbsgillians novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases
AT waddellmichaelj novelsyndromedefinitionvalidationapproachforrarelyoccurringdiseases