Cargando…
117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing
BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Oxford University Press
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7777812/ http://dx.doi.org/10.1093/ofid/ofaa439.162 |
_version_ | 1783630990622588928 |
---|---|
author | Lee, Brian R Linafelter, Alaina Burns, Alaina Burris, Allison Jones, Heather Dusin, Jarrod El Feghaly, Rana E |
author_facet | Lee, Brian R Linafelter, Alaina Burns, Alaina Burris, Allison Jones, Heather Dusin, Jarrod El Feghaly, Rana E |
author_sort | Lee, Brian R |
collection | PubMed |
description | BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory settings. Approximately 40–60% of RADTs are estimated to be inappropriate. Determination of inappropriate RADT frequently requires time-intensive chart reviews. The purpose of this study was to determine if natural language processing (NLP) can provide an accurate and automated alternative for assessing RADT inappropriateness. METHODS: Patients ≥ 3 years of age who received an RADT while evaluated in our EDs/UCCs between April 2018 and September 2018 were identified. A manual chart review was completed on a 10% random sample to determine the presence of sore throat or viral symptoms (i.e., conjunctivitis, rhinorrhea, cough, diarrhea, hoarse voice, and viral exanthema). Inappropriate RADT was defined as either absence of sore throat or reporting 2 or more viral symptoms. An NLP algorithm was developed independently to assign the presence/absence of symptoms and RADT inappropriateness. The NLP sensitivity/specificity was calculated using the manual chart review sample as the gold standard. RESULTS: Manual chart review was completed on 720 patients, of which 320 (44.4%) were considered to have an inappropriate RADT. When compared to the manual review, the NLP approach showed high sensitivity (se) and specificity (sp) when assigning inappropriateness (88.4% and 90.0%, respectively). Optimal sensitivity/specificity was also observed for select symptoms, including sore throat (se: 92.9%, sp: 92.5%), cough (se: 94.5%, sp: 96.5%), and rhinorrhea (se: 86.1%, sp: 95.3%). The prevalence of clinical symptoms was similar when running NLP on subsequent, independent validation sets. After validating the NLP algorithm, a long term monthly trend report was developed. Figure Inappropriate GAS RADTs Determined by NLP, June 2018-May 2020 [Image: see text] CONCLUSION: An NLP algorithm can accurately identify inappropriate RADT when compared to a gold standard. Manual chart review requires dozens of hours to complete. In contrast, NLP requires only a couple of minutes and offers the potential to calculate valid metrics that are easily scaled-up to help monitor comprehensive, long-term trends. DISCLOSURES: Brian R. Lee, MPH, PhD, Merck (Grant/Research Support) |
format | Online Article Text |
id | pubmed-7777812 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Oxford University Press |
record_format | MEDLINE/PubMed |
spelling | pubmed-77778122021-01-07 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing Lee, Brian R Linafelter, Alaina Burns, Alaina Burris, Allison Jones, Heather Dusin, Jarrod El Feghaly, Rana E Open Forum Infect Dis Poster Abstracts BACKGROUND: Acute pharyngitis is one of the most common causes of pediatric health care visits, accounting for approximately 12 million ambulatory care visits each year. Rapid antigen detection tests (RADTs) for Group A Streptococcus (GAS) are one of the most commonly ordered tests in the ambulatory settings. Approximately 40–60% of RADTs are estimated to be inappropriate. Determination of inappropriate RADT frequently requires time-intensive chart reviews. The purpose of this study was to determine if natural language processing (NLP) can provide an accurate and automated alternative for assessing RADT inappropriateness. METHODS: Patients ≥ 3 years of age who received an RADT while evaluated in our EDs/UCCs between April 2018 and September 2018 were identified. A manual chart review was completed on a 10% random sample to determine the presence of sore throat or viral symptoms (i.e., conjunctivitis, rhinorrhea, cough, diarrhea, hoarse voice, and viral exanthema). Inappropriate RADT was defined as either absence of sore throat or reporting 2 or more viral symptoms. An NLP algorithm was developed independently to assign the presence/absence of symptoms and RADT inappropriateness. The NLP sensitivity/specificity was calculated using the manual chart review sample as the gold standard. RESULTS: Manual chart review was completed on 720 patients, of which 320 (44.4%) were considered to have an inappropriate RADT. When compared to the manual review, the NLP approach showed high sensitivity (se) and specificity (sp) when assigning inappropriateness (88.4% and 90.0%, respectively). Optimal sensitivity/specificity was also observed for select symptoms, including sore throat (se: 92.9%, sp: 92.5%), cough (se: 94.5%, sp: 96.5%), and rhinorrhea (se: 86.1%, sp: 95.3%). The prevalence of clinical symptoms was similar when running NLP on subsequent, independent validation sets. After validating the NLP algorithm, a long term monthly trend report was developed. Figure Inappropriate GAS RADTs Determined by NLP, June 2018-May 2020 [Image: see text] CONCLUSION: An NLP algorithm can accurately identify inappropriate RADT when compared to a gold standard. Manual chart review requires dozens of hours to complete. In contrast, NLP requires only a couple of minutes and offers the potential to calculate valid metrics that are easily scaled-up to help monitor comprehensive, long-term trends. DISCLOSURES: Brian R. Lee, MPH, PhD, Merck (Grant/Research Support) Oxford University Press 2020-12-31 /pmc/articles/PMC7777812/ http://dx.doi.org/10.1093/ofid/ofaa439.162 Text en © The Author 2020. Published by Oxford University Press on behalf of Infectious Diseases Society of America. http://creativecommons.org/licenses/by-nc-nd/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.permissions@oup.com |
spellingShingle | Poster Abstracts Lee, Brian R Linafelter, Alaina Burns, Alaina Burris, Allison Jones, Heather Dusin, Jarrod El Feghaly, Rana E 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title | 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title_full | 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title_fullStr | 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title_full_unstemmed | 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title_short | 117. Natural Language Processing: An Automated Alternative to Determining Inappropriate Group A Streptococcal Testing |
title_sort | 117. natural language processing: an automated alternative to determining inappropriate group a streptococcal testing |
topic | Poster Abstracts |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7777812/ http://dx.doi.org/10.1093/ofid/ofaa439.162 |
work_keys_str_mv | AT leebrianr 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT linafelteralaina 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT burnsalaina 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT burrisallison 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT jonesheather 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT dusinjarrod 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting AT elfeghalyranae 117naturallanguageprocessinganautomatedalternativetodetermininginappropriategroupastreptococcaltesting |