Cargando…

Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI

Motor imagery (MI) electroencephalography (EEG) classification is an important part of the brain-computer interface (BCI), allowing people with mobility problems to communicate with the outside world via assistive devices. However, EEG decoding is a challenging task because of its complexity, dynami...

Descripción completa

Detalles Bibliográficos
Autores principales: Liu, Xiuling, Shen, Yonglong, Liu, Jing, Yang, Jianli, Xiong, Peng, Lin, Feng
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7759669/
https://www.ncbi.nlm.nih.gov/pubmed/33362458
http://dx.doi.org/10.3389/fnins.2020.587520
_version_ 1783627155475791872
author Liu, Xiuling
Shen, Yonglong
Liu, Jing
Yang, Jianli
Xiong, Peng
Lin, Feng
author_facet Liu, Xiuling
Shen, Yonglong
Liu, Jing
Yang, Jianli
Xiong, Peng
Lin, Feng
author_sort Liu, Xiuling
collection PubMed
description Motor imagery (MI) electroencephalography (EEG) classification is an important part of the brain-computer interface (BCI), allowing people with mobility problems to communicate with the outside world via assistive devices. However, EEG decoding is a challenging task because of its complexity, dynamic nature, and low signal-to-noise ratio. Designing an end-to-end framework that fully extracts the high-level features of EEG signals remains a challenge. In this study, we present a parallel spatial–temporal self-attention-based convolutional neural network for four-class MI EEG signal classification. This study is the first to define a new spatial-temporal representation of raw EEG signals that uses the self-attention mechanism to extract distinguishable spatial–temporal features. Specifically, we use the spatial self-attention module to capture the spatial dependencies between the channels of MI EEG signals. This module updates each channel by aggregating features over all channels with a weighted summation, thus improving the classification accuracy and eliminating the artifacts caused by manual channel selection. Furthermore, the temporal self-attention module encodes the global temporal information into features for each sampling time step, so that the high-level temporal features of the MI EEG signals can be extracted in the time domain. Quantitative analysis shows that our method outperforms state-of-the-art methods for intra-subject and inter-subject classification, demonstrating its robustness and effectiveness. In terms of qualitative analysis, we perform a visual inspection of the new spatial–temporal representation estimated from the learned architecture. Finally, the proposed method is employed to realize control of drones based on EEG signal, verifying its feasibility in real-time applications.
format Online
Article
Text
id pubmed-7759669
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-77596692020-12-26 Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI Liu, Xiuling Shen, Yonglong Liu, Jing Yang, Jianli Xiong, Peng Lin, Feng Front Neurosci Neuroscience Motor imagery (MI) electroencephalography (EEG) classification is an important part of the brain-computer interface (BCI), allowing people with mobility problems to communicate with the outside world via assistive devices. However, EEG decoding is a challenging task because of its complexity, dynamic nature, and low signal-to-noise ratio. Designing an end-to-end framework that fully extracts the high-level features of EEG signals remains a challenge. In this study, we present a parallel spatial–temporal self-attention-based convolutional neural network for four-class MI EEG signal classification. This study is the first to define a new spatial-temporal representation of raw EEG signals that uses the self-attention mechanism to extract distinguishable spatial–temporal features. Specifically, we use the spatial self-attention module to capture the spatial dependencies between the channels of MI EEG signals. This module updates each channel by aggregating features over all channels with a weighted summation, thus improving the classification accuracy and eliminating the artifacts caused by manual channel selection. Furthermore, the temporal self-attention module encodes the global temporal information into features for each sampling time step, so that the high-level temporal features of the MI EEG signals can be extracted in the time domain. Quantitative analysis shows that our method outperforms state-of-the-art methods for intra-subject and inter-subject classification, demonstrating its robustness and effectiveness. In terms of qualitative analysis, we perform a visual inspection of the new spatial–temporal representation estimated from the learned architecture. Finally, the proposed method is employed to realize control of drones based on EEG signal, verifying its feasibility in real-time applications. Frontiers Media S.A. 2020-12-11 /pmc/articles/PMC7759669/ /pubmed/33362458 http://dx.doi.org/10.3389/fnins.2020.587520 Text en Copyright © 2020 Liu, Shen, Liu, Yang, Xiong and Lin. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Liu, Xiuling
Shen, Yonglong
Liu, Jing
Yang, Jianli
Xiong, Peng
Lin, Feng
Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title_full Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title_fullStr Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title_full_unstemmed Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title_short Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI
title_sort parallel spatial–temporal self-attention cnn-based motor imagery classification for bci
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7759669/
https://www.ncbi.nlm.nih.gov/pubmed/33362458
http://dx.doi.org/10.3389/fnins.2020.587520
work_keys_str_mv AT liuxiuling parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci
AT shenyonglong parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci
AT liujing parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci
AT yangjianli parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci
AT xiongpeng parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci
AT linfeng parallelspatialtemporalselfattentioncnnbasedmotorimageryclassificationforbci