Cargando…

Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet

Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is...

Descripción completa

Detalles Bibliográficos
Autores principales: Si, Xiaopeng, He, Huang, Yu, Jiayue, Ming, Dong
Formato: Online Artículo Texto
Lenguaje:English
Publicado: AAAS 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10374245/
https://www.ncbi.nlm.nih.gov/pubmed/37519929
http://dx.doi.org/10.34133/cbsystems.0045
_version_ 1785078731861131264
author Si, Xiaopeng
He, Huang
Yu, Jiayue
Ming, Dong
author_facet Si, Xiaopeng
He, Huang
Yu, Jiayue
Ming, Dong
author_sort Si, Xiaopeng
collection PubMed
description Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is mainly limited to within-subject, and there is a lack of related work on emotion recognition across subjects. Therefore, in this paper, we designed an emotion evoking experiment with videos as stimuli and constructed the fNIRS emotion recognition database. On this basis, deep learning technology was introduced for the first time, and a dual-branch joint network (DBJNet) was constructed, creating the ability to generalize the model to new participants. The decoding performance obtained by the proposed model shows that fNIRS can effectively distinguish positive versus neutral versus negative emotions (accuracy is 74.8%, F1 score is 72.9%), and the decoding performance on the 2-category emotion recognition task of distinguishing positive versus neutral (accuracy is 89.5%, F1 score is 88.3%), negative versus neutral (accuracy is 91.7%, F1 score is 91.1%) proved fNIRS has a powerful ability to decode emotions. Furthermore, the results of the ablation study of the model structure demonstrate that the joint convolutional neural network branch and the statistical branch achieve the highest decoding performance. The work in this paper is expected to facilitate the development of fNIRS affective brain–computer interface.
format Online
Article
Text
id pubmed-10374245
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher AAAS
record_format MEDLINE/PubMed
spelling pubmed-103742452023-07-28 Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet Si, Xiaopeng He, Huang Yu, Jiayue Ming, Dong Cyborg Bionic Syst Research Article Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is mainly limited to within-subject, and there is a lack of related work on emotion recognition across subjects. Therefore, in this paper, we designed an emotion evoking experiment with videos as stimuli and constructed the fNIRS emotion recognition database. On this basis, deep learning technology was introduced for the first time, and a dual-branch joint network (DBJNet) was constructed, creating the ability to generalize the model to new participants. The decoding performance obtained by the proposed model shows that fNIRS can effectively distinguish positive versus neutral versus negative emotions (accuracy is 74.8%, F1 score is 72.9%), and the decoding performance on the 2-category emotion recognition task of distinguishing positive versus neutral (accuracy is 89.5%, F1 score is 88.3%), negative versus neutral (accuracy is 91.7%, F1 score is 91.1%) proved fNIRS has a powerful ability to decode emotions. Furthermore, the results of the ablation study of the model structure demonstrate that the joint convolutional neural network branch and the statistical branch achieve the highest decoding performance. The work in this paper is expected to facilitate the development of fNIRS affective brain–computer interface. AAAS 2023-07-27 /pmc/articles/PMC10374245/ /pubmed/37519929 http://dx.doi.org/10.34133/cbsystems.0045 Text en Copyright © 2023 Xiaopeng Si et al. https://creativecommons.org/licenses/by/4.0/Exclusive licensee Beijing Institute of Technology Press. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution License 4.0 (CC BY 4.0) (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Research Article
Si, Xiaopeng
He, Huang
Yu, Jiayue
Ming, Dong
Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title_full Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title_fullStr Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title_full_unstemmed Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title_short Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
title_sort cross-subject emotion recognition brain–computer interface based on fnirs and dbjnet
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10374245/
https://www.ncbi.nlm.nih.gov/pubmed/37519929
http://dx.doi.org/10.34133/cbsystems.0045
work_keys_str_mv AT sixiaopeng crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet
AT hehuang crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet
AT yujiayue crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet
AT mingdong crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet