Cargando…
Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification err...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5447826/ https://www.ncbi.nlm.nih.gov/pubmed/28506958 http://dx.doi.org/10.2196/jmir.6887 |
_version_ | 1783239427481403392 |
---|---|
author | Meystre, Stephane Gouripeddi, Ramkiran Tieder, Joel Simmons, Jeffrey Srivastava, Rajendu Shah, Samir |
author_facet | Meystre, Stephane Gouripeddi, Ramkiran Tieder, Joel Simmons, Jeffrey Srivastava, Rajendu Shah, Samir |
author_sort | Meystre, Stephane |
collection | PubMed |
description | BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE: The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS: The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children’s hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS: Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS: NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER. |
format | Online Article Text |
id | pubmed-5447826 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-54478262017-06-13 Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study Meystre, Stephane Gouripeddi, Ramkiran Tieder, Joel Simmons, Jeffrey Srivastava, Rajendu Shah, Samir J Med Internet Res Original Paper BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE: The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS: The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children’s hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS: Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS: NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER. JMIR Publications 2017-05-15 /pmc/articles/PMC5447826/ /pubmed/28506958 http://dx.doi.org/10.2196/jmir.6887 Text en ©Stephane Meystre, Ramkiran Gouripeddi, Joel Tieder, Jeffrey Simmons, Rajendu Srivastava, Samir Shah. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.05.2017. http://creativecommons.org/licenses/by/2.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Meystre, Stephane Gouripeddi, Ramkiran Tieder, Joel Simmons, Jeffrey Srivastava, Rajendu Shah, Samir Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title | Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title_full | Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title_fullStr | Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title_full_unstemmed | Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title_short | Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study |
title_sort | enhancing comparative effectiveness research with automated pediatric pneumonia detection in a multi-institutional clinical repository: a phis+ pilot study |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5447826/ https://www.ncbi.nlm.nih.gov/pubmed/28506958 http://dx.doi.org/10.2196/jmir.6887 |
work_keys_str_mv | AT meystrestephane enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy AT gouripeddiramkiran enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy AT tiederjoel enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy AT simmonsjeffrey enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy AT srivastavarajendu enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy AT shahsamir enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy |