Cargando…

UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals

With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However,...

Descripción completa

Detalles Bibliográficos
Autores principales: Wang, Yuejiao, Hao, Zhanjun, Dang, Xiaochao, Zhang, Zhenyi, Li, Mengqiao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9966078/
https://www.ncbi.nlm.nih.gov/pubmed/36850389
http://dx.doi.org/10.3390/s23041790
_version_ 1784896926592794624
author Wang, Yuejiao
Hao, Zhanjun
Dang, Xiaochao
Zhang, Zhenyi
Li, Mengqiao
author_facet Wang, Yuejiao
Hao, Zhanjun
Dang, Xiaochao
Zhang, Zhenyi
Li, Mengqiao
author_sort Wang, Yuejiao
collection PubMed
description With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However, the recognition of everyday behavioral sign language of a certain population of deaf people presents a challenge to sensing technology. Ubiquitous acoustics offer new ideas on how to perceive everyday behavior. The advantages of a low sampling rate, slow propagation speed, and easy access to the equipment have led to the widespread use of acoustic signal-based gesture recognition sensing technology. Therefore, this paper proposed a contactless gesture and sign language behavior sensing method based on ultrasonic signals—UltrasonicGS. The method used Generative Adversarial Network (GAN)-based data augmentation techniques to expand the dataset without human intervention and improve the performance of the behavior recognition model. In addition, to solve the problem of inconsistent length and difficult alignment of input and output sequences of continuous gestures and sign language gestures, we added the Connectionist Temporal Classification (CTC) algorithm after the CRNN network. Additionally, the architecture can achieve better recognition of sign language behaviors of certain people, filling the gap of acoustic-based perception of Chinese sign language. We have conducted extensive experiments and evaluations of UltrasonicGS in a variety of real scenarios. The experimental results showed that UltrasonicGS achieved a combined recognition rate of 98.8% for 15 single gestures and an average correct recognition rate of 92.4% and 86.3% for six sets of continuous gestures and sign language gestures, respectively. As a result, our proposed method provided a low-cost and highly robust solution for avoiding human-to-human contact.
format Online
Article
Text
id pubmed-9966078
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-99660782023-02-26 UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals Wang, Yuejiao Hao, Zhanjun Dang, Xiaochao Zhang, Zhenyi Li, Mengqiao Sensors (Basel) Article With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However, the recognition of everyday behavioral sign language of a certain population of deaf people presents a challenge to sensing technology. Ubiquitous acoustics offer new ideas on how to perceive everyday behavior. The advantages of a low sampling rate, slow propagation speed, and easy access to the equipment have led to the widespread use of acoustic signal-based gesture recognition sensing technology. Therefore, this paper proposed a contactless gesture and sign language behavior sensing method based on ultrasonic signals—UltrasonicGS. The method used Generative Adversarial Network (GAN)-based data augmentation techniques to expand the dataset without human intervention and improve the performance of the behavior recognition model. In addition, to solve the problem of inconsistent length and difficult alignment of input and output sequences of continuous gestures and sign language gestures, we added the Connectionist Temporal Classification (CTC) algorithm after the CRNN network. Additionally, the architecture can achieve better recognition of sign language behaviors of certain people, filling the gap of acoustic-based perception of Chinese sign language. We have conducted extensive experiments and evaluations of UltrasonicGS in a variety of real scenarios. The experimental results showed that UltrasonicGS achieved a combined recognition rate of 98.8% for 15 single gestures and an average correct recognition rate of 92.4% and 86.3% for six sets of continuous gestures and sign language gestures, respectively. As a result, our proposed method provided a low-cost and highly robust solution for avoiding human-to-human contact. MDPI 2023-02-05 /pmc/articles/PMC9966078/ /pubmed/36850389 http://dx.doi.org/10.3390/s23041790 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Wang, Yuejiao
Hao, Zhanjun
Dang, Xiaochao
Zhang, Zhenyi
Li, Mengqiao
UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title_full UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title_fullStr UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title_full_unstemmed UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title_short UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
title_sort ultrasonicgs: a highly robust gesture and sign language recognition method based on ultrasonic signals
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9966078/
https://www.ncbi.nlm.nih.gov/pubmed/36850389
http://dx.doi.org/10.3390/s23041790
work_keys_str_mv AT wangyuejiao ultrasonicgsahighlyrobustgestureandsignlanguagerecognitionmethodbasedonultrasonicsignals
AT haozhanjun ultrasonicgsahighlyrobustgestureandsignlanguagerecognitionmethodbasedonultrasonicsignals
AT dangxiaochao ultrasonicgsahighlyrobustgestureandsignlanguagerecognitionmethodbasedonultrasonicsignals
AT zhangzhenyi ultrasonicgsahighlyrobustgestureandsignlanguagerecognitionmethodbasedonultrasonicsignals
AT limengqiao ultrasonicgsahighlyrobustgestureandsignlanguagerecognitionmethodbasedonultrasonicsignals