Cargando…
Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System
Coronavirus disease 2019 (COVID-19) has led to countless deaths and widespread global disruptions. Acoustic-based artificial intelligence (AI) tools could provide a simple, scalable, and prompt method to screen for COVID-19 using easily acquirable physiological sounds. These systems have been demons...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9739601/ https://www.ncbi.nlm.nih.gov/pubmed/36502232 http://dx.doi.org/10.3390/s22239530 |
_version_ | 1784847847556907008 |
---|---|
author | Grant, Drew McLane, Ian Rennoll, Valerie West, James |
author_facet | Grant, Drew McLane, Ian Rennoll, Valerie West, James |
author_sort | Grant, Drew |
collection | PubMed |
description | Coronavirus disease 2019 (COVID-19) has led to countless deaths and widespread global disruptions. Acoustic-based artificial intelligence (AI) tools could provide a simple, scalable, and prompt method to screen for COVID-19 using easily acquirable physiological sounds. These systems have been demonstrated previously and have shown promise but lack robust analysis of their deployment in real-world settings when faced with diverse recording equipment, noise environments, and test subjects. The primary aim of this work is to begin to understand the impacts of these real-world deployment challenges on the system performance. Using Mel-Frequency Cepstral Coefficients (MFCC) and RelAtive SpecTrAl-Perceptual Linear Prediction (RASTA-PLP) features extracted from cough, speech, and breathing sounds in a crowdsourced dataset, we present a baseline classification system that obtains an average receiver operating characteristic area under the curve (AUC-ROC) of 0.77 when discriminating between COVID-19 and non-COVID subjects. The classifier performance is then evaluated on four additional datasets, resulting in performance variations between 0.64 and 0.87 AUC-ROC, depending on the sound type. By analyzing subsets of the available recordings, it is noted that the system performance degrades with certain recording devices, noise contamination, and with symptom status. Furthermore, performance degrades when a uniform classification threshold from the training data is subsequently used across all datasets. However, the system performance is robust to confounding factors, such as gender, age group, and the presence of other respiratory conditions. Finally, when analyzing multiple speech recordings from the same subjects, the system achieves promising performance with an AUC-ROC of 0.78, though the classification does appear to be impacted by natural speech variations. Overall, the proposed system, and by extension other acoustic-based diagnostic aids in the literature, could provide comparable accuracy to rapid antigen testing but significant deployment challenges need to be understood and addressed prior to clinical use. |
format | Online Article Text |
id | pubmed-9739601 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-97396012022-12-11 Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System Grant, Drew McLane, Ian Rennoll, Valerie West, James Sensors (Basel) Article Coronavirus disease 2019 (COVID-19) has led to countless deaths and widespread global disruptions. Acoustic-based artificial intelligence (AI) tools could provide a simple, scalable, and prompt method to screen for COVID-19 using easily acquirable physiological sounds. These systems have been demonstrated previously and have shown promise but lack robust analysis of their deployment in real-world settings when faced with diverse recording equipment, noise environments, and test subjects. The primary aim of this work is to begin to understand the impacts of these real-world deployment challenges on the system performance. Using Mel-Frequency Cepstral Coefficients (MFCC) and RelAtive SpecTrAl-Perceptual Linear Prediction (RASTA-PLP) features extracted from cough, speech, and breathing sounds in a crowdsourced dataset, we present a baseline classification system that obtains an average receiver operating characteristic area under the curve (AUC-ROC) of 0.77 when discriminating between COVID-19 and non-COVID subjects. The classifier performance is then evaluated on four additional datasets, resulting in performance variations between 0.64 and 0.87 AUC-ROC, depending on the sound type. By analyzing subsets of the available recordings, it is noted that the system performance degrades with certain recording devices, noise contamination, and with symptom status. Furthermore, performance degrades when a uniform classification threshold from the training data is subsequently used across all datasets. However, the system performance is robust to confounding factors, such as gender, age group, and the presence of other respiratory conditions. Finally, when analyzing multiple speech recordings from the same subjects, the system achieves promising performance with an AUC-ROC of 0.78, though the classification does appear to be impacted by natural speech variations. Overall, the proposed system, and by extension other acoustic-based diagnostic aids in the literature, could provide comparable accuracy to rapid antigen testing but significant deployment challenges need to be understood and addressed prior to clinical use. MDPI 2022-12-06 /pmc/articles/PMC9739601/ /pubmed/36502232 http://dx.doi.org/10.3390/s22239530 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Grant, Drew McLane, Ian Rennoll, Valerie West, James Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title | Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title_full | Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title_fullStr | Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title_full_unstemmed | Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title_short | Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System |
title_sort | considerations and challenges for real-world deployment of an acoustic-based covid-19 screening system |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9739601/ https://www.ncbi.nlm.nih.gov/pubmed/36502232 http://dx.doi.org/10.3390/s22239530 |
work_keys_str_mv | AT grantdrew considerationsandchallengesforrealworlddeploymentofanacousticbasedcovid19screeningsystem AT mclaneian considerationsandchallengesforrealworlddeploymentofanacousticbasedcovid19screeningsystem AT rennollvalerie considerationsandchallengesforrealworlddeploymentofanacousticbasedcovid19screeningsystem AT westjames considerationsandchallengesforrealworlddeploymentofanacousticbasedcovid19screeningsystem |