Cargando…
Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques
The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in em...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10136603/ https://www.ncbi.nlm.nih.gov/pubmed/37190650 http://dx.doi.org/10.3390/brainsci13040685 |
_version_ | 1785032258726395904 |
---|---|
author | Yuvaraj, Rajamanickam Baranwal, Arapan Prince, A. Amalin Murugappan, M. Mohammed, Javeed Shaikh |
author_facet | Yuvaraj, Rajamanickam Baranwal, Arapan Prince, A. Amalin Murugappan, M. Mohammed, Javeed Shaikh |
author_sort | Yuvaraj, Rajamanickam |
collection | PubMed |
description | The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in emotion recognition studies. Most studies, however, use other methods to extract handcrafted features, such as Pearson correlation coefficient (PCC), Principal Component Analysis, Higuchi Fractal Dimension (HFD), etc., even though DNN is capable of generating meaningful features. Furthermore, most earlier studies largely ignored spatial information between the different channels, focusing mainly on time domain and frequency domain representations. This study utilizes a pre-trained 3D-CNN MobileNet model with transfer learning on the spatio-temporal representation of EEG signals to extract features for emotion recognition. In addition to fully connected layers, hybrid models were explored using other decision layers such as multilayer perceptron (MLP), k-nearest neighbor (KNN), extreme learning machine (ELM), XGBoost (XGB), random forest (RF), and support vector machine (SVM). Additionally, this study investigates the effects of post-processing or filtering output labels. Extensive experiments were conducted on the SJTU Emotion EEG Dataset (SEED) (three classes) and SEED-IV (four classes) datasets, and the results obtained were comparable to the state-of-the-art. Based on the conventional 3D-CNN with ELM classifier, SEED and SEED-IV datasets showed a maximum accuracy of 89.18% and 81.60%, respectively. Post-filtering improved the emotional classification performance in the hybrid 3D-CNN with ELM model for SEED and SEED-IV datasets to 90.85% and 83.71%, respectively. Accordingly, spatial-temporal features extracted from the EEG, along with ensemble classifiers, were found to be the most effective in recognizing emotions compared to state-of-the-art methods. |
format | Online Article Text |
id | pubmed-10136603 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-101366032023-04-28 Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques Yuvaraj, Rajamanickam Baranwal, Arapan Prince, A. Amalin Murugappan, M. Mohammed, Javeed Shaikh Brain Sci Article The recognition of emotions is one of the most challenging issues in human–computer interaction (HCI). EEG signals are widely adopted as a method for recognizing emotions because of their ease of acquisition, mobility, and convenience. Deep neural networks (DNN) have provided excellent results in emotion recognition studies. Most studies, however, use other methods to extract handcrafted features, such as Pearson correlation coefficient (PCC), Principal Component Analysis, Higuchi Fractal Dimension (HFD), etc., even though DNN is capable of generating meaningful features. Furthermore, most earlier studies largely ignored spatial information between the different channels, focusing mainly on time domain and frequency domain representations. This study utilizes a pre-trained 3D-CNN MobileNet model with transfer learning on the spatio-temporal representation of EEG signals to extract features for emotion recognition. In addition to fully connected layers, hybrid models were explored using other decision layers such as multilayer perceptron (MLP), k-nearest neighbor (KNN), extreme learning machine (ELM), XGBoost (XGB), random forest (RF), and support vector machine (SVM). Additionally, this study investigates the effects of post-processing or filtering output labels. Extensive experiments were conducted on the SJTU Emotion EEG Dataset (SEED) (three classes) and SEED-IV (four classes) datasets, and the results obtained were comparable to the state-of-the-art. Based on the conventional 3D-CNN with ELM classifier, SEED and SEED-IV datasets showed a maximum accuracy of 89.18% and 81.60%, respectively. Post-filtering improved the emotional classification performance in the hybrid 3D-CNN with ELM model for SEED and SEED-IV datasets to 90.85% and 83.71%, respectively. Accordingly, spatial-temporal features extracted from the EEG, along with ensemble classifiers, were found to be the most effective in recognizing emotions compared to state-of-the-art methods. MDPI 2023-04-19 /pmc/articles/PMC10136603/ /pubmed/37190650 http://dx.doi.org/10.3390/brainsci13040685 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Yuvaraj, Rajamanickam Baranwal, Arapan Prince, A. Amalin Murugappan, M. Mohammed, Javeed Shaikh Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title | Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title_full | Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title_fullStr | Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title_full_unstemmed | Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title_short | Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques |
title_sort | emotion recognition from spatio-temporal representation of eeg signals via 3d-cnn with ensemble learning techniques |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10136603/ https://www.ncbi.nlm.nih.gov/pubmed/37190650 http://dx.doi.org/10.3390/brainsci13040685 |
work_keys_str_mv | AT yuvarajrajamanickam emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques AT baranwalarapan emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques AT princeaamalin emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques AT murugappanm emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques AT mohammedjaveedshaikh emotionrecognitionfromspatiotemporalrepresentationofeegsignalsvia3dcnnwithensemblelearningtechniques |