Cargando…

Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms

As the aging of the population becomes more severe, wheelchair-mounted robotic arms (WMRAs) are gaining an increased amount of attention. Laser pointer interactions are an attractive method enabling humans to unambiguously point out objects and pick them up. In addition, they bring about a greater s...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhong, Ming, Zhang, Yanqiang, Yang, Xi, Yao, Yufeng, Guo, Junlong, Wang, Yaping, Liu, Yaxin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6359181/
https://www.ncbi.nlm.nih.gov/pubmed/30646513
http://dx.doi.org/10.3390/s19020303
_version_ 1783392172565856256
author Zhong, Ming
Zhang, Yanqiang
Yang, Xi
Yao, Yufeng
Guo, Junlong
Wang, Yaping
Liu, Yaxin
author_facet Zhong, Ming
Zhang, Yanqiang
Yang, Xi
Yao, Yufeng
Guo, Junlong
Wang, Yaping
Liu, Yaxin
author_sort Zhong, Ming
collection PubMed
description As the aging of the population becomes more severe, wheelchair-mounted robotic arms (WMRAs) are gaining an increased amount of attention. Laser pointer interactions are an attractive method enabling humans to unambiguously point out objects and pick them up. In addition, they bring about a greater sense of participation in the interaction process as an intuitive interaction mode. However, the issue of human–robot interactions remains to be properly tackled, and traditional laser point interactions still suffer from poor real-time performance and low accuracy amid dynamic backgrounds. In this study, combined with an advanced laser point detection method and an improved pose estimation algorithm, a laser pointer is used to facilitate the interactions between humans and a WMRA in an indoor environment. Assistive grasping using a laser selection consists of two key steps. In the first step, the images captured using an RGB-D camera are pre-processed, and then fed to a convolutional neural network (CNN) to determine the 2D coordinates of the laser point and objects within the image. Meanwhile, the centroid coordinates of the selected object are also obtained using the depth information. In this way, the object to be picked up and its location are determined. The experimental results show that the laser point can be detected with almost 100% accuracy in a complex environment. In the second step, a compound pose-estimation algorithm aiming at a sparse use of multi-view templates is applied, which consists of both coarse- and precise-matching of the target to the template objects, greatly improving the grasping performance. The proposed algorithms were implemented on a Kinova Jaco robotic arm, and the experimental results demonstrate their effectiveness. Compared with commonly accepted methods, the time consumption of the pose generation can be reduced from 5.36 to 4.43 s, and synchronously, the pose estimation error is significantly improved from 21.31% to 3.91%.
format Online
Article
Text
id pubmed-6359181
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-63591812019-02-06 Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms Zhong, Ming Zhang, Yanqiang Yang, Xi Yao, Yufeng Guo, Junlong Wang, Yaping Liu, Yaxin Sensors (Basel) Article As the aging of the population becomes more severe, wheelchair-mounted robotic arms (WMRAs) are gaining an increased amount of attention. Laser pointer interactions are an attractive method enabling humans to unambiguously point out objects and pick them up. In addition, they bring about a greater sense of participation in the interaction process as an intuitive interaction mode. However, the issue of human–robot interactions remains to be properly tackled, and traditional laser point interactions still suffer from poor real-time performance and low accuracy amid dynamic backgrounds. In this study, combined with an advanced laser point detection method and an improved pose estimation algorithm, a laser pointer is used to facilitate the interactions between humans and a WMRA in an indoor environment. Assistive grasping using a laser selection consists of two key steps. In the first step, the images captured using an RGB-D camera are pre-processed, and then fed to a convolutional neural network (CNN) to determine the 2D coordinates of the laser point and objects within the image. Meanwhile, the centroid coordinates of the selected object are also obtained using the depth information. In this way, the object to be picked up and its location are determined. The experimental results show that the laser point can be detected with almost 100% accuracy in a complex environment. In the second step, a compound pose-estimation algorithm aiming at a sparse use of multi-view templates is applied, which consists of both coarse- and precise-matching of the target to the template objects, greatly improving the grasping performance. The proposed algorithms were implemented on a Kinova Jaco robotic arm, and the experimental results demonstrate their effectiveness. Compared with commonly accepted methods, the time consumption of the pose generation can be reduced from 5.36 to 4.43 s, and synchronously, the pose estimation error is significantly improved from 21.31% to 3.91%. MDPI 2019-01-14 /pmc/articles/PMC6359181/ /pubmed/30646513 http://dx.doi.org/10.3390/s19020303 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Zhong, Ming
Zhang, Yanqiang
Yang, Xi
Yao, Yufeng
Guo, Junlong
Wang, Yaping
Liu, Yaxin
Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title_full Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title_fullStr Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title_full_unstemmed Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title_short Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms
title_sort assistive grasping based on laser-point detection with application to wheelchair-mounted robotic arms
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6359181/
https://www.ncbi.nlm.nih.gov/pubmed/30646513
http://dx.doi.org/10.3390/s19020303
work_keys_str_mv AT zhongming assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT zhangyanqiang assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT yangxi assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT yaoyufeng assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT guojunlong assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT wangyaping assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms
AT liuyaxin assistivegraspingbasedonlaserpointdetectionwithapplicationtowheelchairmountedroboticarms