Cargando…

Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device

Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication betwe...

Descripción completa

Detalles Bibliográficos
Autores principales: Jing, Hongwei, Zheng, Tianjiao, Zhang, Qinghua, Sun, Kerui, Li, Lele, Lai, Mingzhu, Zhao, Jie, Zhu, Yanhe
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10604667/
https://www.ncbi.nlm.nih.gov/pubmed/37887610
http://dx.doi.org/10.3390/biomimetics8060479
_version_ 1785126890801987584
author Jing, Hongwei
Zheng, Tianjiao
Zhang, Qinghua
Sun, Kerui
Li, Lele
Lai, Mingzhu
Zhao, Jie
Zhu, Yanhe
author_facet Jing, Hongwei
Zheng, Tianjiao
Zhang, Qinghua
Sun, Kerui
Li, Lele
Lai, Mingzhu
Zhao, Jie
Zhu, Yanhe
author_sort Jing, Hongwei
collection PubMed
description Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human–robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer’s capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics.
format Online
Article
Text
id pubmed-10604667
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106046672023-10-28 Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device Jing, Hongwei Zheng, Tianjiao Zhang, Qinghua Sun, Kerui Li, Lele Lai, Mingzhu Zhao, Jie Zhu, Yanhe Biomimetics (Basel) Article Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human–robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer’s capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics. MDPI 2023-10-08 /pmc/articles/PMC10604667/ /pubmed/37887610 http://dx.doi.org/10.3390/biomimetics8060479 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Jing, Hongwei
Zheng, Tianjiao
Zhang, Qinghua
Sun, Kerui
Li, Lele
Lai, Mingzhu
Zhao, Jie
Zhu, Yanhe
Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title_full Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title_fullStr Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title_full_unstemmed Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title_short Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
title_sort human operation augmentation through wearable robotic limb integrated with mixed reality device
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10604667/
https://www.ncbi.nlm.nih.gov/pubmed/37887610
http://dx.doi.org/10.3390/biomimetics8060479
work_keys_str_mv AT jinghongwei humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT zhengtianjiao humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT zhangqinghua humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT sunkerui humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT lilele humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT laimingzhu humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT zhaojie humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice
AT zhuyanhe humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice