Cargando…

A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data

Soft robotic grippers are increasingly desired in applications that involve grasping of complex and deformable objects. However, their flexible nature and non-linear dynamics makes the modelling and control difficult. Numerical techniques such as Finite Element Analysis (FEA) present an accurate way...

Descripción completa

Detalles Bibliográficos
Autores principales: De Barrie, Daniel, Pandya, Manjari, Pandya, Harit, Hanheide, Marc, Elgeneidy, Khaled
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8186462/
https://www.ncbi.nlm.nih.gov/pubmed/34113655
http://dx.doi.org/10.3389/frobt.2021.631371
_version_ 1783704955213840384
author De Barrie, Daniel
Pandya, Manjari
Pandya, Harit
Hanheide, Marc
Elgeneidy, Khaled
author_facet De Barrie, Daniel
Pandya, Manjari
Pandya, Harit
Hanheide, Marc
Elgeneidy, Khaled
author_sort De Barrie, Daniel
collection PubMed
description Soft robotic grippers are increasingly desired in applications that involve grasping of complex and deformable objects. However, their flexible nature and non-linear dynamics makes the modelling and control difficult. Numerical techniques such as Finite Element Analysis (FEA) present an accurate way of modelling complex deformations. However, FEA approaches are computationally expensive and consequently challenging to employ for real-time control tasks. Existing analytical techniques simplify the modelling by approximating the deformed gripper geometry. Although this approach is less computationally demanding, it is limited in design scope and can lead to larger estimation errors. In this paper, we present a learning based framework that is able to predict contact forces as well as stress distribution from soft Fin Ray Effect (FRE) finger images in real-time. These images are used to learn internal representations for deformations using a deep neural encoder, which are further decoded to contact forces and stress maps using separate branches. The entire network is jointly learned in an end-to-end fashion. In order to address the challenge of having sufficient labelled data for training, we employ FEA to generate simulated images to supervise our framework. This leads to an accurate prediction, faster inference and availability of large and diverse data for better generalisability. Furthermore, our approach is able to predict a detailed stress distribution that can guide grasp planning, which would be particularly useful for delicate objects. Our proposed approach is validated by comparing the predicted contact forces to the computed ground-truth forces from FEA as well as real force sensor. We rigorously evaluate the performance of our approach under variations in contact point, object material, object shape, viewing angle, and level of occlusion.
format Online
Article
Text
id pubmed-8186462
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-81864622021-06-09 A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data De Barrie, Daniel Pandya, Manjari Pandya, Harit Hanheide, Marc Elgeneidy, Khaled Front Robot AI Robotics and AI Soft robotic grippers are increasingly desired in applications that involve grasping of complex and deformable objects. However, their flexible nature and non-linear dynamics makes the modelling and control difficult. Numerical techniques such as Finite Element Analysis (FEA) present an accurate way of modelling complex deformations. However, FEA approaches are computationally expensive and consequently challenging to employ for real-time control tasks. Existing analytical techniques simplify the modelling by approximating the deformed gripper geometry. Although this approach is less computationally demanding, it is limited in design scope and can lead to larger estimation errors. In this paper, we present a learning based framework that is able to predict contact forces as well as stress distribution from soft Fin Ray Effect (FRE) finger images in real-time. These images are used to learn internal representations for deformations using a deep neural encoder, which are further decoded to contact forces and stress maps using separate branches. The entire network is jointly learned in an end-to-end fashion. In order to address the challenge of having sufficient labelled data for training, we employ FEA to generate simulated images to supervise our framework. This leads to an accurate prediction, faster inference and availability of large and diverse data for better generalisability. Furthermore, our approach is able to predict a detailed stress distribution that can guide grasp planning, which would be particularly useful for delicate objects. Our proposed approach is validated by comparing the predicted contact forces to the computed ground-truth forces from FEA as well as real force sensor. We rigorously evaluate the performance of our approach under variations in contact point, object material, object shape, viewing angle, and level of occlusion. Frontiers Media S.A. 2021-05-25 /pmc/articles/PMC8186462/ /pubmed/34113655 http://dx.doi.org/10.3389/frobt.2021.631371 Text en Copyright © 2021 De Barrie, Pandya, Pandya, Hanheide and Elgeneidy. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Robotics and AI
De Barrie, Daniel
Pandya, Manjari
Pandya, Harit
Hanheide, Marc
Elgeneidy, Khaled
A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title_full A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title_fullStr A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title_full_unstemmed A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title_short A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
title_sort deep learning method for vision based force prediction of a soft fin ray gripper using simulation data
topic Robotics and AI
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8186462/
https://www.ncbi.nlm.nih.gov/pubmed/34113655
http://dx.doi.org/10.3389/frobt.2021.631371
work_keys_str_mv AT debarriedaniel adeeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT pandyamanjari adeeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT pandyaharit adeeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT hanheidemarc adeeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT elgeneidykhaled adeeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT debarriedaniel deeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT pandyamanjari deeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT pandyaharit deeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT hanheidemarc deeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata
AT elgeneidykhaled deeplearningmethodforvisionbasedforcepredictionofasoftfinraygripperusingsimulationdata