Cargando…

Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist

To facilitate hand gesture recognition, we investigated the use of acoustic signals with an accelerometer and gyroscope at the human wrist. As a proof-of-concept, the prototype consisted of 10 microphone units in contact with the skin placed around the wrist along with an inertial measurement unit (...

Descripción completa

Detalles Bibliográficos
Autores principales: Siddiqui, Nabeel, Chan, Rosa H. M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6957149/
https://www.ncbi.nlm.nih.gov/pubmed/31929544
http://dx.doi.org/10.1371/journal.pone.0227039
_version_ 1783487267362308096
author Siddiqui, Nabeel
Chan, Rosa H. M.
author_facet Siddiqui, Nabeel
Chan, Rosa H. M.
author_sort Siddiqui, Nabeel
collection PubMed
description To facilitate hand gesture recognition, we investigated the use of acoustic signals with an accelerometer and gyroscope at the human wrist. As a proof-of-concept, the prototype consisted of 10 microphone units in contact with the skin placed around the wrist along with an inertial measurement unit (IMU). The gesture recognition performance was evaluated through the identification of 13 gestures used in daily life. The optimal area for acoustic sensor placement at the wrist was examined using the minimum redundancy and maximum relevance feature selection algorithm. We recruited 10 subjects to perform over 10 trials for each set of hand gestures. The accuracy was 75% for a general model with the top 25 features selected, and the intra-subject average classification accuracy was over 80% with the same features using one microphone unit at the mid-anterior wrist and an IMU. These results indicate that acoustic signatures from the human wrist can aid IMU sensing for hand gesture recognition, and the selection of a few common features for all subjects could help with building a general model. The proposed multimodal framework helps address the single IMU sensing bottleneck for hand gestures during arm movement and/or locomotion.
format Online
Article
Text
id pubmed-6957149
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-69571492020-01-26 Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist Siddiqui, Nabeel Chan, Rosa H. M. PLoS One Research Article To facilitate hand gesture recognition, we investigated the use of acoustic signals with an accelerometer and gyroscope at the human wrist. As a proof-of-concept, the prototype consisted of 10 microphone units in contact with the skin placed around the wrist along with an inertial measurement unit (IMU). The gesture recognition performance was evaluated through the identification of 13 gestures used in daily life. The optimal area for acoustic sensor placement at the wrist was examined using the minimum redundancy and maximum relevance feature selection algorithm. We recruited 10 subjects to perform over 10 trials for each set of hand gestures. The accuracy was 75% for a general model with the top 25 features selected, and the intra-subject average classification accuracy was over 80% with the same features using one microphone unit at the mid-anterior wrist and an IMU. These results indicate that acoustic signatures from the human wrist can aid IMU sensing for hand gesture recognition, and the selection of a few common features for all subjects could help with building a general model. The proposed multimodal framework helps address the single IMU sensing bottleneck for hand gestures during arm movement and/or locomotion. Public Library of Science 2020-01-13 /pmc/articles/PMC6957149/ /pubmed/31929544 http://dx.doi.org/10.1371/journal.pone.0227039 Text en © 2020 Siddiqui, Chan http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Siddiqui, Nabeel
Chan, Rosa H. M.
Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title_full Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title_fullStr Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title_full_unstemmed Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title_short Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist
title_sort multimodal hand gesture recognition using single imu and acoustic measurements at wrist
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6957149/
https://www.ncbi.nlm.nih.gov/pubmed/31929544
http://dx.doi.org/10.1371/journal.pone.0227039
work_keys_str_mv AT siddiquinabeel multimodalhandgesturerecognitionusingsingleimuandacousticmeasurementsatwrist
AT chanrosahm multimodalhandgesturerecognitionusingsingleimuandacousticmeasurementsatwrist