Cargando…
Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network
Speech enhancement (SE) reduces background noise signals in target speech and is applied at the front end in various real-world applications, including robust ASRs and real-time processing in mobile phone communications. SE systems are commonly integrated into mobile phones to increase quality and i...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10174555/ https://www.ncbi.nlm.nih.gov/pubmed/37167227 http://dx.doi.org/10.1371/journal.pone.0285629 |
_version_ | 1785040057929826304 |
---|---|
author | Peracha, Fahad Khalil Khattak, Muhammad Irfan Salem, Nema Saleem, Nasir |
author_facet | Peracha, Fahad Khalil Khattak, Muhammad Irfan Salem, Nema Saleem, Nasir |
author_sort | Peracha, Fahad Khalil |
collection | PubMed |
description | Speech enhancement (SE) reduces background noise signals in target speech and is applied at the front end in various real-world applications, including robust ASRs and real-time processing in mobile phone communications. SE systems are commonly integrated into mobile phones to increase quality and intelligibility. As a result, a low-latency system is required to operate in real-world applications. On the other hand, these systems need efficient optimization. This research focuses on the single-microphone SE operating in real-time systems with better optimization. We propose a causal data-driven model that uses attention encoder-decoder long short-term memory (LSTM) to estimate the time-frequency mask from a noisy speech in order to make a clean speech for real-time applications that need low-latency causal processing. The encoder-decoder LSTM and a causal attention mechanism are used in the proposed model. Furthermore, a dynamical-weighted (DW) loss function is proposed to improve model learning by varying the weight loss values. Experiments demonstrated that the proposed model consistently improves voice quality, intelligibility, and noise suppression. In the causal processing mode, the LSTM-based estimated suppression time-frequency mask outperforms the baseline model for unseen noise types. The proposed SE improved the STOI by 2.64% (baseline LSTM-IRM), 6.6% (LSTM-KF), 4.18% (DeepXi-KF), and 3.58% (DeepResGRU-KF). In addition, we examine word error rates (WERs) using Google’s Automatic Speech Recognition (ASR). The ASR results show that error rates decreased from 46.33% (noisy signals) to 13.11% (proposed) 15.73% (LSTM), and 14.97% (LSTM-KF). |
format | Online Article Text |
id | pubmed-10174555 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-101745552023-05-12 Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network Peracha, Fahad Khalil Khattak, Muhammad Irfan Salem, Nema Saleem, Nasir PLoS One Research Article Speech enhancement (SE) reduces background noise signals in target speech and is applied at the front end in various real-world applications, including robust ASRs and real-time processing in mobile phone communications. SE systems are commonly integrated into mobile phones to increase quality and intelligibility. As a result, a low-latency system is required to operate in real-world applications. On the other hand, these systems need efficient optimization. This research focuses on the single-microphone SE operating in real-time systems with better optimization. We propose a causal data-driven model that uses attention encoder-decoder long short-term memory (LSTM) to estimate the time-frequency mask from a noisy speech in order to make a clean speech for real-time applications that need low-latency causal processing. The encoder-decoder LSTM and a causal attention mechanism are used in the proposed model. Furthermore, a dynamical-weighted (DW) loss function is proposed to improve model learning by varying the weight loss values. Experiments demonstrated that the proposed model consistently improves voice quality, intelligibility, and noise suppression. In the causal processing mode, the LSTM-based estimated suppression time-frequency mask outperforms the baseline model for unseen noise types. The proposed SE improved the STOI by 2.64% (baseline LSTM-IRM), 6.6% (LSTM-KF), 4.18% (DeepXi-KF), and 3.58% (DeepResGRU-KF). In addition, we examine word error rates (WERs) using Google’s Automatic Speech Recognition (ASR). The ASR results show that error rates decreased from 46.33% (noisy signals) to 13.11% (proposed) 15.73% (LSTM), and 14.97% (LSTM-KF). Public Library of Science 2023-05-11 /pmc/articles/PMC10174555/ /pubmed/37167227 http://dx.doi.org/10.1371/journal.pone.0285629 Text en © 2023 Peracha et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Peracha, Fahad Khalil Khattak, Muhammad Irfan Salem, Nema Saleem, Nasir Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title | Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title_full | Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title_fullStr | Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title_full_unstemmed | Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title_short | Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
title_sort | causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10174555/ https://www.ncbi.nlm.nih.gov/pubmed/37167227 http://dx.doi.org/10.1371/journal.pone.0285629 |
work_keys_str_mv | AT perachafahadkhalil causalspeechenhancementusingdynamicalweightedlossandattentionencoderdecoderrecurrentneuralnetwork AT khattakmuhammadirfan causalspeechenhancementusingdynamicalweightedlossandattentionencoderdecoderrecurrentneuralnetwork AT salemnema causalspeechenhancementusingdynamicalweightedlossandattentionencoderdecoderrecurrentneuralnetwork AT saleemnasir causalspeechenhancementusingdynamicalweightedlossandattentionencoderdecoderrecurrentneuralnetwork |