Cargando…
Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet
Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
AAAS
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10374245/ https://www.ncbi.nlm.nih.gov/pubmed/37519929 http://dx.doi.org/10.34133/cbsystems.0045 |
_version_ | 1785078731861131264 |
---|---|
author | Si, Xiaopeng He, Huang Yu, Jiayue Ming, Dong |
author_facet | Si, Xiaopeng He, Huang Yu, Jiayue Ming, Dong |
author_sort | Si, Xiaopeng |
collection | PubMed |
description | Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is mainly limited to within-subject, and there is a lack of related work on emotion recognition across subjects. Therefore, in this paper, we designed an emotion evoking experiment with videos as stimuli and constructed the fNIRS emotion recognition database. On this basis, deep learning technology was introduced for the first time, and a dual-branch joint network (DBJNet) was constructed, creating the ability to generalize the model to new participants. The decoding performance obtained by the proposed model shows that fNIRS can effectively distinguish positive versus neutral versus negative emotions (accuracy is 74.8%, F1 score is 72.9%), and the decoding performance on the 2-category emotion recognition task of distinguishing positive versus neutral (accuracy is 89.5%, F1 score is 88.3%), negative versus neutral (accuracy is 91.7%, F1 score is 91.1%) proved fNIRS has a powerful ability to decode emotions. Furthermore, the results of the ablation study of the model structure demonstrate that the joint convolutional neural network branch and the statistical branch achieve the highest decoding performance. The work in this paper is expected to facilitate the development of fNIRS affective brain–computer interface. |
format | Online Article Text |
id | pubmed-10374245 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | AAAS |
record_format | MEDLINE/PubMed |
spelling | pubmed-103742452023-07-28 Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet Si, Xiaopeng He, Huang Yu, Jiayue Ming, Dong Cyborg Bionic Syst Research Article Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that has gradually been applied in emotion recognition research due to its advantages of high spatial resolution, real time, and convenience. However, the current research on emotion recognition based on fNIRS is mainly limited to within-subject, and there is a lack of related work on emotion recognition across subjects. Therefore, in this paper, we designed an emotion evoking experiment with videos as stimuli and constructed the fNIRS emotion recognition database. On this basis, deep learning technology was introduced for the first time, and a dual-branch joint network (DBJNet) was constructed, creating the ability to generalize the model to new participants. The decoding performance obtained by the proposed model shows that fNIRS can effectively distinguish positive versus neutral versus negative emotions (accuracy is 74.8%, F1 score is 72.9%), and the decoding performance on the 2-category emotion recognition task of distinguishing positive versus neutral (accuracy is 89.5%, F1 score is 88.3%), negative versus neutral (accuracy is 91.7%, F1 score is 91.1%) proved fNIRS has a powerful ability to decode emotions. Furthermore, the results of the ablation study of the model structure demonstrate that the joint convolutional neural network branch and the statistical branch achieve the highest decoding performance. The work in this paper is expected to facilitate the development of fNIRS affective brain–computer interface. AAAS 2023-07-27 /pmc/articles/PMC10374245/ /pubmed/37519929 http://dx.doi.org/10.34133/cbsystems.0045 Text en Copyright © 2023 Xiaopeng Si et al. https://creativecommons.org/licenses/by/4.0/Exclusive licensee Beijing Institute of Technology Press. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution License 4.0 (CC BY 4.0) (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Research Article Si, Xiaopeng He, Huang Yu, Jiayue Ming, Dong Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title | Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title_full | Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title_fullStr | Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title_full_unstemmed | Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title_short | Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet |
title_sort | cross-subject emotion recognition brain–computer interface based on fnirs and dbjnet |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10374245/ https://www.ncbi.nlm.nih.gov/pubmed/37519929 http://dx.doi.org/10.34133/cbsystems.0045 |
work_keys_str_mv | AT sixiaopeng crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet AT hehuang crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet AT yujiayue crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet AT mingdong crosssubjectemotionrecognitionbraincomputerinterfacebasedonfnirsanddbjnet |