Cargando…

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study

BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification err...

Descripción completa

Detalles Bibliográficos
Autores principales: Meystre, Stephane, Gouripeddi, Ramkiran, Tieder, Joel, Simmons, Jeffrey, Srivastava, Rajendu, Shah, Samir
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5447826/
https://www.ncbi.nlm.nih.gov/pubmed/28506958
http://dx.doi.org/10.2196/jmir.6887
_version_ 1783239427481403392
author Meystre, Stephane
Gouripeddi, Ramkiran
Tieder, Joel
Simmons, Jeffrey
Srivastava, Rajendu
Shah, Samir
author_facet Meystre, Stephane
Gouripeddi, Ramkiran
Tieder, Joel
Simmons, Jeffrey
Srivastava, Rajendu
Shah, Samir
author_sort Meystre, Stephane
collection PubMed
description BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE: The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS: The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children’s hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS: Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS: NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER.
format Online
Article
Text
id pubmed-5447826
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-54478262017-06-13 Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study Meystre, Stephane Gouripeddi, Ramkiran Tieder, Joel Simmons, Jeffrey Srivastava, Rajendu Shah, Samir J Med Internet Res Original Paper BACKGROUND: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE: The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS: The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children’s hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS: Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS: NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER. JMIR Publications 2017-05-15 /pmc/articles/PMC5447826/ /pubmed/28506958 http://dx.doi.org/10.2196/jmir.6887 Text en ©Stephane Meystre, Ramkiran Gouripeddi, Joel Tieder, Jeffrey Simmons, Rajendu Srivastava, Samir Shah. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.05.2017. http://creativecommons.org/licenses/by/2.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Meystre, Stephane
Gouripeddi, Ramkiran
Tieder, Joel
Simmons, Jeffrey
Srivastava, Rajendu
Shah, Samir
Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title_full Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title_fullStr Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title_full_unstemmed Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title_short Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study
title_sort enhancing comparative effectiveness research with automated pediatric pneumonia detection in a multi-institutional clinical repository: a phis+ pilot study
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5447826/
https://www.ncbi.nlm.nih.gov/pubmed/28506958
http://dx.doi.org/10.2196/jmir.6887
work_keys_str_mv AT meystrestephane enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy
AT gouripeddiramkiran enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy
AT tiederjoel enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy
AT simmonsjeffrey enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy
AT srivastavarajendu enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy
AT shahsamir enhancingcomparativeeffectivenessresearchwithautomatedpediatricpneumoniadetectioninamultiinstitutionalclinicalrepositoryaphispilotstudy