Cargando…

Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many...

Descripción completa

Detalles Bibliográficos
Autores principales: Jiang, Yujian, Song, Lin, Zhang, Junming, Song, Yang, Yan, Ming
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9371015/
https://www.ncbi.nlm.nih.gov/pubmed/35957417
http://dx.doi.org/10.3390/s22155855
_version_ 1784767001356402688
author Jiang, Yujian
Song, Lin
Zhang, Junming
Song, Yang
Yan, Ming
author_facet Jiang, Yujian
Song, Lin
Zhang, Junming
Song, Yang
Yan, Ming
author_sort Jiang, Yujian
collection PubMed
description Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models’ test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%.
format Online
Article
Text
id pubmed-9371015
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-93710152022-08-12 Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals Jiang, Yujian Song, Lin Zhang, Junming Song, Yang Yan, Ming Sensors (Basel) Article Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models’ test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%. MDPI 2022-08-05 /pmc/articles/PMC9371015/ /pubmed/35957417 http://dx.doi.org/10.3390/s22155855 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Jiang, Yujian
Song, Lin
Zhang, Junming
Song, Yang
Yan, Ming
Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title_full Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title_fullStr Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title_full_unstemmed Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title_short Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals
title_sort multi-category gesture recognition modeling based on semg and imu signals
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9371015/
https://www.ncbi.nlm.nih.gov/pubmed/35957417
http://dx.doi.org/10.3390/s22155855
work_keys_str_mv AT jiangyujian multicategorygesturerecognitionmodelingbasedonsemgandimusignals
AT songlin multicategorygesturerecognitionmodelingbasedonsemgandimusignals
AT zhangjunming multicategorygesturerecognitionmodelingbasedonsemgandimusignals
AT songyang multicategorygesturerecognitionmodelingbasedonsemgandimusignals
AT yanming multicategorygesturerecognitionmodelingbasedonsemgandimusignals