Cargando…
Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing
We propose a vision-proprioception model for planar object pushing, efficiently integrating all necessary information from the environment. A Variational Autoencoder (VAE) is used to extract compact representations from the task-relevant part of the image. With the real-time robot state obtained eas...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8926160/ https://www.ncbi.nlm.nih.gov/pubmed/35308311 http://dx.doi.org/10.3389/fnbot.2022.829437 |
_version_ | 1784670178242461696 |
---|---|
author | Cong, Lin Liang, Hongzhuo Ruppel, Philipp Shi, Yunlei Görner, Michael Hendrich, Norman Zhang, Jianwei |
author_facet | Cong, Lin Liang, Hongzhuo Ruppel, Philipp Shi, Yunlei Görner, Michael Hendrich, Norman Zhang, Jianwei |
author_sort | Cong, Lin |
collection | PubMed |
description | We propose a vision-proprioception model for planar object pushing, efficiently integrating all necessary information from the environment. A Variational Autoencoder (VAE) is used to extract compact representations from the task-relevant part of the image. With the real-time robot state obtained easily from the hardware system, we fuse the latent representations from the VAE and the robot end-effector position together as the state of a Markov Decision Process. We use Soft Actor-Critic to train the robot to push different objects from random initial poses to target positions in simulation. Hindsight Experience replay is applied during the training process to improve the sample efficiency. Experiments demonstrate that our algorithm achieves a pushing performance superior to a state-based baseline model that cannot be generalized to a different object and outperforms state-of-the-art policies which operate on raw image observations. At last, we verify that our trained model has a good generalization ability to unseen objects in the real world. |
format | Online Article Text |
id | pubmed-8926160 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-89261602022-03-17 Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing Cong, Lin Liang, Hongzhuo Ruppel, Philipp Shi, Yunlei Görner, Michael Hendrich, Norman Zhang, Jianwei Front Neurorobot Neuroscience We propose a vision-proprioception model for planar object pushing, efficiently integrating all necessary information from the environment. A Variational Autoencoder (VAE) is used to extract compact representations from the task-relevant part of the image. With the real-time robot state obtained easily from the hardware system, we fuse the latent representations from the VAE and the robot end-effector position together as the state of a Markov Decision Process. We use Soft Actor-Critic to train the robot to push different objects from random initial poses to target positions in simulation. Hindsight Experience replay is applied during the training process to improve the sample efficiency. Experiments demonstrate that our algorithm achieves a pushing performance superior to a state-based baseline model that cannot be generalized to a different object and outperforms state-of-the-art policies which operate on raw image observations. At last, we verify that our trained model has a good generalization ability to unseen objects in the real world. Frontiers Media S.A. 2022-03-02 /pmc/articles/PMC8926160/ /pubmed/35308311 http://dx.doi.org/10.3389/fnbot.2022.829437 Text en Copyright © 2022 Cong, Liang, Ruppel, Shi, Görner, Hendrich and Zhang. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Cong, Lin Liang, Hongzhuo Ruppel, Philipp Shi, Yunlei Görner, Michael Hendrich, Norman Zhang, Jianwei Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title | Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title_full | Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title_fullStr | Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title_full_unstemmed | Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title_short | Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing |
title_sort | reinforcement learning with vision-proprioception model for robot planar pushing |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8926160/ https://www.ncbi.nlm.nih.gov/pubmed/35308311 http://dx.doi.org/10.3389/fnbot.2022.829437 |
work_keys_str_mv | AT conglin reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT lianghongzhuo reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT ruppelphilipp reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT shiyunlei reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT gornermichael reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT hendrichnorman reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing AT zhangjianwei reinforcementlearningwithvisionproprioceptionmodelforrobotplanarpushing |