Cargando…

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of d...

Descripción completa

Detalles Bibliográficos
Autores principales: Yoo, Minjeong, Na, Yuseung, Song, Hamin, Kim, Gamin, Yun, Junseong, Kim, Sangho, Moon, Changjoo, Jo, Kichun
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9002368/
https://www.ncbi.nlm.nih.gov/pubmed/35408128
http://dx.doi.org/10.3390/s22072513
_version_ 1784685872330833920
author Yoo, Minjeong
Na, Yuseung
Song, Hamin
Kim, Gamin
Yun, Junseong
Kim, Sangho
Moon, Changjoo
Jo, Kichun
author_facet Yoo, Minjeong
Na, Yuseung
Song, Hamin
Kim, Gamin
Yun, Junseong
Kim, Sangho
Moon, Changjoo
Jo, Kichun
author_sort Yoo, Minjeong
collection PubMed
description As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of dynamic gesture because it is difficult to estimate the pose of multi-dimensional hand gestures in 2D images. This leads to complex algorithms, including tracking in addition to detection, to recognize dynamic gestures, but they are not suitable for human–UAV interaction (HUI) systems that require safe design with high real-time performance. Therefore, in this paper, we propose a hybrid hand gesture system that combines an inertial measurement unit (IMU)-based motion capture system and a vision-based gesture system to increase real-time performance. First, IMU-based commands and vision-based commands are divided according to whether drone operation commands are continuously input. Second, IMU-based control commands are intuitively mapped to allow the UAV to move in the same direction by utilizing estimated orientation sensed by a thumb-mounted micro-IMU, and vision-based control commands are mapped with hand’s appearance through real-time object detection. The proposed system is verified in a simulation environment through efficiency evaluation with dynamic gestures of the existing vision-based system in addition to usability comparison with traditional joystick controller conducted for applicants with no experience in manipulation. As a result, it proves that it is a safer and more intuitive HUI design with a 0.089 ms processing speed and average lap time that takes about 19 s less than the joystick controller. In other words, it shows that it is viable as an alternative to existing HUI.
format Online
Article
Text
id pubmed-9002368
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-90023682022-04-13 Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time Yoo, Minjeong Na, Yuseung Song, Hamin Kim, Gamin Yun, Junseong Kim, Sangho Moon, Changjoo Jo, Kichun Sensors (Basel) Article As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of dynamic gesture because it is difficult to estimate the pose of multi-dimensional hand gestures in 2D images. This leads to complex algorithms, including tracking in addition to detection, to recognize dynamic gestures, but they are not suitable for human–UAV interaction (HUI) systems that require safe design with high real-time performance. Therefore, in this paper, we propose a hybrid hand gesture system that combines an inertial measurement unit (IMU)-based motion capture system and a vision-based gesture system to increase real-time performance. First, IMU-based commands and vision-based commands are divided according to whether drone operation commands are continuously input. Second, IMU-based control commands are intuitively mapped to allow the UAV to move in the same direction by utilizing estimated orientation sensed by a thumb-mounted micro-IMU, and vision-based control commands are mapped with hand’s appearance through real-time object detection. The proposed system is verified in a simulation environment through efficiency evaluation with dynamic gestures of the existing vision-based system in addition to usability comparison with traditional joystick controller conducted for applicants with no experience in manipulation. As a result, it proves that it is a safer and more intuitive HUI design with a 0.089 ms processing speed and average lap time that takes about 19 s less than the joystick controller. In other words, it shows that it is viable as an alternative to existing HUI. MDPI 2022-03-25 /pmc/articles/PMC9002368/ /pubmed/35408128 http://dx.doi.org/10.3390/s22072513 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Yoo, Minjeong
Na, Yuseung
Song, Hamin
Kim, Gamin
Yun, Junseong
Kim, Sangho
Moon, Changjoo
Jo, Kichun
Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title_full Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title_fullStr Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title_full_unstemmed Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title_short Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time
title_sort motion estimation and hand gesture recognition-based human–uav interaction approach in real time
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9002368/
https://www.ncbi.nlm.nih.gov/pubmed/35408128
http://dx.doi.org/10.3390/s22072513
work_keys_str_mv AT yoominjeong motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT nayuseung motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT songhamin motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT kimgamin motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT yunjunseong motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT kimsangho motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT moonchangjoo motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime
AT jokichun motionestimationandhandgesturerecognitionbasedhumanuavinteractionapproachinrealtime