Cargando…

6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping

6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE...

Descripción completa

Detalles Bibliográficos
Autores principales: Cao, Hongpeng, Dirnberger, Lukas, Bernardini, Daniele, Piazza, Cristina, Caccamo, Marco
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10565011/
https://www.ncbi.nlm.nih.gov/pubmed/37830110
http://dx.doi.org/10.3389/frobt.2023.1176492
_version_ 1785118602166272000
author Cao, Hongpeng
Dirnberger, Lukas
Bernardini, Daniele
Piazza, Cristina
Caccamo, Marco
author_facet Cao, Hongpeng
Dirnberger, Lukas
Bernardini, Daniele
Piazza, Cristina
Caccamo, Marco
author_sort Cao, Hongpeng
collection PubMed
description 6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE, a novel framework for sim-to-real data generation and 6D pose estimation. 6IMPOSE consists of four modules: First, a data generation pipeline that employs the 3D software suite Blender to create synthetic RGBD image datasets with 6D pose annotations. Second, an annotated RGBD dataset of five household objects was generated using the proposed pipeline. Third, a real-time two-stage 6D pose estimation approach that integrates the object detector YOLO-V4 and a streamlined, real-time version of the 6D pose estimation algorithm PVN3D optimized for time-sensitive robotics applications. Fourth, a codebase designed to facilitate the integration of the vision system into a robotic grasping experiment. Our approach demonstrates the efficient generation of large amounts of photo-realistic RGBD images and the successful transfer of the trained inference model to robotic grasping experiments, achieving an overall success rate of 87% in grasping five different household objects from cluttered backgrounds under varying lighting conditions. This is made possible by fine-tuning data generation and domain randomization techniques and optimizing the inference pipeline, overcoming the generalization and performance shortcomings of the original PVN3D algorithm. Finally, we make the code, synthetic dataset, and all the pre-trained models available on GitHub.
format Online
Article
Text
id pubmed-10565011
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-105650112023-10-12 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping Cao, Hongpeng Dirnberger, Lukas Bernardini, Daniele Piazza, Cristina Caccamo, Marco Front Robot AI Robotics and AI 6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE, a novel framework for sim-to-real data generation and 6D pose estimation. 6IMPOSE consists of four modules: First, a data generation pipeline that employs the 3D software suite Blender to create synthetic RGBD image datasets with 6D pose annotations. Second, an annotated RGBD dataset of five household objects was generated using the proposed pipeline. Third, a real-time two-stage 6D pose estimation approach that integrates the object detector YOLO-V4 and a streamlined, real-time version of the 6D pose estimation algorithm PVN3D optimized for time-sensitive robotics applications. Fourth, a codebase designed to facilitate the integration of the vision system into a robotic grasping experiment. Our approach demonstrates the efficient generation of large amounts of photo-realistic RGBD images and the successful transfer of the trained inference model to robotic grasping experiments, achieving an overall success rate of 87% in grasping five different household objects from cluttered backgrounds under varying lighting conditions. This is made possible by fine-tuning data generation and domain randomization techniques and optimizing the inference pipeline, overcoming the generalization and performance shortcomings of the original PVN3D algorithm. Finally, we make the code, synthetic dataset, and all the pre-trained models available on GitHub. Frontiers Media S.A. 2023-09-27 /pmc/articles/PMC10565011/ /pubmed/37830110 http://dx.doi.org/10.3389/frobt.2023.1176492 Text en Copyright © 2023 Cao, Dirnberger, Bernardini, Piazza and Caccamo. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Robotics and AI
Cao, Hongpeng
Dirnberger, Lukas
Bernardini, Daniele
Piazza, Cristina
Caccamo, Marco
6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title_full 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title_fullStr 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title_full_unstemmed 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title_short 6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping
title_sort 6impose: bridging the reality gap in 6d pose estimation for robotic grasping
topic Robotics and AI
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10565011/
https://www.ncbi.nlm.nih.gov/pubmed/37830110
http://dx.doi.org/10.3389/frobt.2023.1176492
work_keys_str_mv AT caohongpeng 6imposebridgingtherealitygapin6dposeestimationforroboticgrasping
AT dirnbergerlukas 6imposebridgingtherealitygapin6dposeestimationforroboticgrasping
AT bernardinidaniele 6imposebridgingtherealitygapin6dposeestimationforroboticgrasping
AT piazzacristina 6imposebridgingtherealitygapin6dposeestimationforroboticgrasping
AT caccamomarco 6imposebridgingtherealitygapin6dposeestimationforroboticgrasping