Cargando…

117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing

BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory...

Descripción completa

Detalles Bibliográficos
Autores principales: Lee, Brian R, Linafelter, Alaina, Burns, Alaina, Burris, Allison, Jones, Heather, Dusin, Jarrod, El Feghaly, Rana E
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Oxford University Press 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7777812/
http://dx.doi.org/10.1093/ofid/ofaa439.162
_version_ 1783630990622588928
author Lee, Brian R
Linafelter, Alaina
Burns, Alaina
Burris, Allison
Jones, Heather
Dusin, Jarrod
El Feghaly, Rana E
author_facet Lee, Brian R
Linafelter, Alaina
Burns, Alaina
Burris, Allison
Jones, Heather
Dusin, Jarrod
El Feghaly, Rana E
author_sort Lee, Brian R
collection PubMed
description BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory settings. Approximately 40–60% of RADTs are estimated to be inappropriate. Determination of inappropriate RADT frequently requires time-intensive chart reviews. The purpose of this study was to determine if natural language processing (NLP) can provide an accurate and automated alternative for assessing RADT inappropriateness. METHODS: Patients ≥ 3 years of age who received an RADT while evaluated in our EDs/UCCs between April 2018 and September 2018 were identified. A manual chart review was completed on a 10% random sample to determine the presence of sore throat or viral symptoms (i.e., conjunctivitis, rhinorrhea, cough, diarrhea, hoarse voice, and viral exanthema). Inappropriate RADT was defined as either absence of sore throat or reporting 2 or more viral symptoms. An NLP algorithm was developed independently to assign the presence/absence of symptoms and RADT inappropriateness. The NLP sensitivity/specificity was calculated using the manual chart review sample as the gold standard. RESULTS: Manual chart review was completed on 720 patients, of which 320 (44.4%) were considered to have an inappropriate RADT. When compared to the manual review, the NLP approach showed high sensitivity (se) and specificity (sp) when assigning inappropriateness (88.4% and 90.0%, respectively). Optimal sensitivity/specificity was also observed for select symptoms, including sore throat (se: 92.9%, sp: 92.5%), cough (se: 94.5%, sp: 96.5%), and rhinorrhea (se: 86.1%, sp: 95.3%). The prevalence of clinical symptoms was similar when running NLP on subsequent, independent validation sets. After validating the NLP algorithm, a long term monthly trend report was developed. Figure Inappropriate GAS RADTs Determined by NLP, June 2018-May 2020 [Image: see text] CONCLUSION: An NLP algorithm can accurately identify inappropriate RADT when compared to a gold standard. Manual chart review requires dozens of hours to complete. In contrast, NLP requires only a couple of minutes and offers the potential to calculate valid metrics that are easily scaled-up to help monitor comprehensive, long-term trends. DISCLOSURES: Brian R. Lee, MPH, PhD, Merck (Grant/Research Support)
format Online
Article
Text
id pubmed-7777812
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Oxford University Press
record_format MEDLINE/PubMed
spelling pubmed-77778122021-01-07 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing Lee, Brian R Linafelter, Alaina Burns, Alaina Burris, Allison Jones, Heather Dusin, Jarrod El Feghaly, Rana E Open Forum Infect Dis Poster Abstracts BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory settings. Approximately 40–60% of RADTs are estimated to be inappropriate. Determination of inappropriate RADT frequently requires time-intensive chart reviews. The purpose of this study was to determine if natural language processing (NLP) can provide an accurate and automated alternative for assessing RADT inappropriateness. METHODS: Patients ≥ 3 years of age who received an RADT while evaluated in our EDs/UCCs between April 2018 and September 2018 were identified. A manual chart review was completed on a 10% random sample to determine the presence of sore throat or viral symptoms (i.e., conjunctivitis, rhinorrhea, cough, diarrhea, hoarse voice, and viral exanthema). Inappropriate RADT was defined as either absence of sore throat or reporting 2 or more viral symptoms. An NLP algorithm was developed independently to assign the presence/absence of symptoms and RADT inappropriateness. The NLP sensitivity/specificity was calculated using the manual chart review sample as the gold standard. RESULTS: Manual chart review was completed on 720 patients, of which 320 (44.4%) were considered to have an inappropriate RADT. When compared to the manual review, the NLP approach showed high sensitivity (se) and specificity (sp) when assigning inappropriateness (88.4% and 90.0%, respectively). Optimal sensitivity/specificity was also observed for select symptoms, including sore throat (se: 92.9%, sp: 92.5%), cough (se: 94.5%, sp: 96.5%), and rhinorrhea (se: 86.1%, sp: 95.3%). The prevalence of clinical symptoms was similar when running NLP on subsequent, independent validation sets. After validating the NLP algorithm, a long term monthly trend report was developed. Figure Inappropriate GAS RADTs Determined by NLP, June 2018-May 2020 [Image: see text] CONCLUSION: An NLP algorithm can accurately identify inappropriate RADT when compared to a gold standard. Manual chart review requires dozens of hours to complete. In contrast, NLP requires only a couple of minutes and offers the potential to calculate valid metrics that are easily scaled-up to help monitor comprehensive, long-term trends. DISCLOSURES: Brian R. Lee, MPH, PhD, Merck (Grant/Research Support) Oxford University Press 2020-12-31 /pmc/articles/PMC7777812/ http://dx.doi.org/10.1093/ofid/ofaa439.162 Text en © The Author 2020. Published by Oxford University Press on behalf of Infectious Diseases Society of America. http://creativecommons.org/licenses/by-nc-nd/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.permissions@oup.com
spellingShingle Poster Abstracts
Lee, Brian R
Linafelter, Alaina
Burns, Alaina
Burris, Allison
Jones, Heather
Dusin, Jarrod
El Feghaly, Rana E
117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title_full 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title_fullStr 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title_full_unstemmed 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title_short 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
title_sort 117. natural language processing: an automated alternative to determining inappropriate group a streptococcal testing
topic Poster Abstracts
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7777812/
http://dx.doi.org/10.1093/ofid/ofaa439.162
work_keys_str_mv AT leebrianr 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT linafelteralaina 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT burnsalaina 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT burrisallison 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT jonesheather 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT dusinjarrod 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting
AT elfeghalyranae 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting