Cargando…
Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities
Silent speech decoding is a novel application of the Brain–Computer Interface (BCI) based on articulatory neuromuscular activities, reducing difficulties in data acquirement and processing. In this paper, spatial features and decoders that can be used to recognize the neuromuscular signals are inves...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7407985/ https://www.ncbi.nlm.nih.gov/pubmed/32664599 http://dx.doi.org/10.3390/brainsci10070442 |
_version_ | 1783567732434796544 |
---|---|
author | Wang, You Zhang, Ming Wu, RuMeng Gao, Han Yang, Meng Luo, Zhiyuan Li, Guang |
author_facet | Wang, You Zhang, Ming Wu, RuMeng Gao, Han Yang, Meng Luo, Zhiyuan Li, Guang |
author_sort | Wang, You |
collection | PubMed |
description | Silent speech decoding is a novel application of the Brain–Computer Interface (BCI) based on articulatory neuromuscular activities, reducing difficulties in data acquirement and processing. In this paper, spatial features and decoders that can be used to recognize the neuromuscular signals are investigated. Surface electromyography (sEMG) data are recorded from human subjects in mimed speech situations. Specifically, we propose to utilize transfer learning and deep learning methods by transforming the sEMG data into spectrograms that contain abundant information in time and frequency domains and are regarded as channel-interactive. For transfer learning, a pre-trained model of Xception on the large image dataset is used for feature generation. Three deep learning methods, Multi-Layer Perception, Convolutional Neural Network and bidirectional Long Short-Term Memory, are then trained using the extracted features and evaluated for recognizing the articulatory muscles’ movements in our word set. The proposed decoders successfully recognized the silent speech and bidirectional Long Short-Term Memory achieved the best accuracy of 90%, outperforming the other two algorithms. Experimental results demonstrate the validity of spectrogram features and deep learning algorithms. |
format | Online Article Text |
id | pubmed-7407985 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-74079852020-08-12 Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities Wang, You Zhang, Ming Wu, RuMeng Gao, Han Yang, Meng Luo, Zhiyuan Li, Guang Brain Sci Article Silent speech decoding is a novel application of the Brain–Computer Interface (BCI) based on articulatory neuromuscular activities, reducing difficulties in data acquirement and processing. In this paper, spatial features and decoders that can be used to recognize the neuromuscular signals are investigated. Surface electromyography (sEMG) data are recorded from human subjects in mimed speech situations. Specifically, we propose to utilize transfer learning and deep learning methods by transforming the sEMG data into spectrograms that contain abundant information in time and frequency domains and are regarded as channel-interactive. For transfer learning, a pre-trained model of Xception on the large image dataset is used for feature generation. Three deep learning methods, Multi-Layer Perception, Convolutional Neural Network and bidirectional Long Short-Term Memory, are then trained using the extracted features and evaluated for recognizing the articulatory muscles’ movements in our word set. The proposed decoders successfully recognized the silent speech and bidirectional Long Short-Term Memory achieved the best accuracy of 90%, outperforming the other two algorithms. Experimental results demonstrate the validity of spectrogram features and deep learning algorithms. MDPI 2020-07-11 /pmc/articles/PMC7407985/ /pubmed/32664599 http://dx.doi.org/10.3390/brainsci10070442 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Wang, You Zhang, Ming Wu, RuMeng Gao, Han Yang, Meng Luo, Zhiyuan Li, Guang Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title | Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title_full | Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title_fullStr | Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title_full_unstemmed | Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title_short | Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities |
title_sort | silent speech decoding using spectrogram features based on neuromuscular activities |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7407985/ https://www.ncbi.nlm.nih.gov/pubmed/32664599 http://dx.doi.org/10.3390/brainsci10070442 |
work_keys_str_mv | AT wangyou silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT zhangming silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT wurumeng silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT gaohan silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT yangmeng silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT luozhiyuan silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities AT liguang silentspeechdecodingusingspectrogramfeaturesbasedonneuromuscularactivities |