Cargando…

Early-stage fusion of EEG and fNIRS improves classification of motor imagery

INTRODUCTION: Many research papers have reported successful implementation of hybrid brain-computer interfaces by complementarily combining EEG and fNIRS, to improve classification performance. However, modality or feature fusion of EEG and fNIRS was usually designed for specific user cases, which w...

Descripción completa

Detalles Bibliográficos
Autores principales: Li, Yang, Zhang, Xin, Ming, Dong
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9869134/
https://www.ncbi.nlm.nih.gov/pubmed/36699533
http://dx.doi.org/10.3389/fnins.2022.1062889
_version_ 1784876702667636736
author Li, Yang
Zhang, Xin
Ming, Dong
author_facet Li, Yang
Zhang, Xin
Ming, Dong
author_sort Li, Yang
collection PubMed
description INTRODUCTION: Many research papers have reported successful implementation of hybrid brain-computer interfaces by complementarily combining EEG and fNIRS, to improve classification performance. However, modality or feature fusion of EEG and fNIRS was usually designed for specific user cases, which were generally customized and hard to be generalized. How to effectively utilize information from the two modalities was still unclear. METHODS: In this paper, we conducted a study to investigate the stage of bi-modal fusion based on EEG and fNIRS. A Y-shaped neural network was proposed and evaluated on an open dataset, which fuses the bimodal information in different stages. RESULTS: The results suggests that the early-stage fusion of EEG and fNIRS have significantly higher performance compared to middle-stage and late-stage fusion network configuration (N = 57, P < 0.05). With the proposed framework, the average accuracy of 29 participants reaches 76.21% in the left-or-right hand motor imagery task in leave-one-out cross-validation, using bi-modal data as network inputs respectively, which is in the same level as the state-of-the-art hybrid BCI methods based on EEG and fNIRS data.
format Online
Article
Text
id pubmed-9869134
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-98691342023-01-24 Early-stage fusion of EEG and fNIRS improves classification of motor imagery Li, Yang Zhang, Xin Ming, Dong Front Neurosci Neuroscience INTRODUCTION: Many research papers have reported successful implementation of hybrid brain-computer interfaces by complementarily combining EEG and fNIRS, to improve classification performance. However, modality or feature fusion of EEG and fNIRS was usually designed for specific user cases, which were generally customized and hard to be generalized. How to effectively utilize information from the two modalities was still unclear. METHODS: In this paper, we conducted a study to investigate the stage of bi-modal fusion based on EEG and fNIRS. A Y-shaped neural network was proposed and evaluated on an open dataset, which fuses the bimodal information in different stages. RESULTS: The results suggests that the early-stage fusion of EEG and fNIRS have significantly higher performance compared to middle-stage and late-stage fusion network configuration (N = 57, P < 0.05). With the proposed framework, the average accuracy of 29 participants reaches 76.21% in the left-or-right hand motor imagery task in leave-one-out cross-validation, using bi-modal data as network inputs respectively, which is in the same level as the state-of-the-art hybrid BCI methods based on EEG and fNIRS data. Frontiers Media S.A. 2023-01-09 /pmc/articles/PMC9869134/ /pubmed/36699533 http://dx.doi.org/10.3389/fnins.2022.1062889 Text en Copyright © 2023 Li, Zhang and Ming. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Li, Yang
Zhang, Xin
Ming, Dong
Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title_full Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title_fullStr Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title_full_unstemmed Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title_short Early-stage fusion of EEG and fNIRS improves classification of motor imagery
title_sort early-stage fusion of eeg and fnirs improves classification of motor imagery
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9869134/
https://www.ncbi.nlm.nih.gov/pubmed/36699533
http://dx.doi.org/10.3389/fnins.2022.1062889
work_keys_str_mv AT liyang earlystagefusionofeegandfnirsimprovesclassificationofmotorimagery
AT zhangxin earlystagefusionofeegandfnirsimprovesclassificationofmotorimagery
AT mingdong earlystagefusionofeegandfnirsimprovesclassificationofmotorimagery