Cargando…

Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in em...

Descripción completa

Detalles Bibliográficos
Autores principales: Yuvaraj, Rajamanickam, Baranwal, Arapan, Prince, A. Amalin, Murugappan, M., Mohammed, Javeed Shaikh
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10136603/
https://www.ncbi.nlm.nih.gov/pubmed/37190650
http://dx.doi.org/10.3390/brainsci13040685
_version_ 1785032258726395904
author Yuvaraj, Rajamanickam
Baranwal, Arapan
Prince, A. Amalin
Murugappan, M.
Mohammed, Javeed Shaikh
author_facet Yuvaraj, Rajamanickam
Baranwal, Arapan
Prince, A. Amalin
Murugappan, M.
Mohammed, Javeed Shaikh
author_sort Yuvaraj, Rajamanickam
collection PubMed
description The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in emotion recognition studies. Most studies, however, use other methods to extract handcrafted features, such as Pearson correlation coefficient (PCC), Principal Component Analysis, Higuchi Fractal Dimension (HFD), etc., even though DNN is capable of generating meaningful features. Furthermore, most earlier studies largely ignored spatial information between the different channels, focusing mainly on time domain and frequency domain representations. This study utilizes a pre-trained 3D-CNN MobileNet model with transfer learning on the spatio-temporal representation of EEG signals to extract features for emotion recognition. In addition to fully connected layers, hybrid models were explored using other decision layers such as multilayer perceptron (MLP), k-nearest neighbor (KNN), extreme learning machine (ELM), XGBoost (XGB), random forest (RF), and support vector machine (SVM). Additionally, this study investigates the effects of post-processing or filtering output labels. Extensive experiments were conducted on the SJTU Emotion EEG Dataset (SEED) (three classes) and SEED-IV (four classes) datasets, and the results obtained were comparable to the state-of-the-art. Based on the conventional 3D-CNN with ELM classifier, SEED and SEED-IV datasets showed a maximum accuracy of 89.18% and 81.60%, respectively. Post-filtering improved the emotional classification performance in the hybrid 3D-CNN with ELM model for SEED and SEED-IV datasets to 90.85% and 83.71%, respectively. Accordingly, spatial-temporal features extracted from the EEG, along with ensemble classifiers, were found to be the most effective in recognizing emotions compared to state-of-the-art methods.
format Online
Article
Text
id pubmed-10136603
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-101366032023-04-28 Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques Yuvaraj, Rajamanickam Baranwal, Arapan Prince, A. Amalin Murugappan, M. Mohammed, Javeed Shaikh Brain Sci Article The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in emotion recognition studies. Most studies, however, use other methods to extract handcrafted features, such as Pearson correlation coefficient (PCC), Principal Component Analysis, Higuchi Fractal Dimension (HFD), etc., even though DNN is capable of generating meaningful features. Furthermore, most earlier studies largely ignored spatial information between the different channels, focusing mainly on time domain and frequency domain representations. This study utilizes a pre-trained 3D-CNN MobileNet model with transfer learning on the spatio-temporal representation of EEG signals to extract features for emotion recognition. In addition to fully connected layers, hybrid models were explored using other decision layers such as multilayer perceptron (MLP), k-nearest neighbor (KNN), extreme learning machine (ELM), XGBoost (XGB), random forest (RF), and support vector machine (SVM). Additionally, this study investigates the effects of post-processing or filtering output labels. Extensive experiments were conducted on the SJTU Emotion EEG Dataset (SEED) (three classes) and SEED-IV (four classes) datasets, and the results obtained were comparable to the state-of-the-art. Based on the conventional 3D-CNN with ELM classifier, SEED and SEED-IV datasets showed a maximum accuracy of 89.18% and 81.60%, respectively. Post-filtering improved the emotional classification performance in the hybrid 3D-CNN with ELM model for SEED and SEED-IV datasets to 90.85% and 83.71%, respectively. Accordingly, spatial-temporal features extracted from the EEG, along with ensemble classifiers, were found to be the most effective in recognizing emotions compared to state-of-the-art methods. MDPI 2023-04-19 /pmc/articles/PMC10136603/ /pubmed/37190650 http://dx.doi.org/10.3390/brainsci13040685 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Yuvaraj, Rajamanickam
Baranwal, Arapan
Prince, A. Amalin
Murugappan, M.
Mohammed, Javeed Shaikh
Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title_full Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title_fullStr Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title_full_unstemmed Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title_short Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
title_sort emotion recognition from spatio-temporal representation of eeg signals via 3d-cnn with ensemble learning techniques
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10136603/
https://www.ncbi.nlm.nih.gov/pubmed/37190650
http://dx.doi.org/10.3390/brainsci13040685
work_keys_str_mv AT yuvarajrajamanickam emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques
AT baranwalarapan emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques
AT princeaamalin emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques
AT murugappanm emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques
AT mohammedjaveedshaikh emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques