Cargando…

Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning

Visual-guided locomotion for snake-like robots is a challenging task, since it involves not only the complex body undulation with many joints, but also a joint pipeline that connects the vision and the locomotion. Meanwhile, it is usually difficult to jointly coordinate these two separate sub-tasks...

Descripción completa

Detalles Bibliográficos
Autores principales: Bing, Zhenshan, Lemke, Christian, Morin, Fabric O., Jiang, Zhuangyi, Cheng, Long, Huang, Kai, Knoll, Alois
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7641616/
https://www.ncbi.nlm.nih.gov/pubmed/33192441
http://dx.doi.org/10.3389/fnbot.2020.591128
_version_ 1783605955020193792
author Bing, Zhenshan
Lemke, Christian
Morin, Fabric O.
Jiang, Zhuangyi
Cheng, Long
Huang, Kai
Knoll, Alois
author_facet Bing, Zhenshan
Lemke, Christian
Morin, Fabric O.
Jiang, Zhuangyi
Cheng, Long
Huang, Kai
Knoll, Alois
author_sort Bing, Zhenshan
collection PubMed
description Visual-guided locomotion for snake-like robots is a challenging task, since it involves not only the complex body undulation with many joints, but also a joint pipeline that connects the vision and the locomotion. Meanwhile, it is usually difficult to jointly coordinate these two separate sub-tasks as this requires time-consuming and trial-and-error tuning. In this paper, we introduce a novel approach for solving target tracking tasks for a snake-like robot as a whole using a model-free reinforcement learning (RL) algorithm. This RL-based controller directly maps the visual observations to the joint positions of the snake-like robot in an end-to-end fashion instead of dividing the process into a series of sub-tasks. With a novel customized reward function, our RL controller is trained in a dynamically changing track scenario. The controller is evaluated in four different tracking scenarios and the results show excellent adaptive locomotion ability to the unpredictable behavior of the target. Meanwhile, the results also prove that the RL-based controller outperforms the traditional model-based controller in terms of tracking accuracy.
format Online
Article
Text
id pubmed-7641616
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-76416162020-11-13 Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning Bing, Zhenshan Lemke, Christian Morin, Fabric O. Jiang, Zhuangyi Cheng, Long Huang, Kai Knoll, Alois Front Neurorobot Neuroscience Visual-guided locomotion for snake-like robots is a challenging task, since it involves not only the complex body undulation with many joints, but also a joint pipeline that connects the vision and the locomotion. Meanwhile, it is usually difficult to jointly coordinate these two separate sub-tasks as this requires time-consuming and trial-and-error tuning. In this paper, we introduce a novel approach for solving target tracking tasks for a snake-like robot as a whole using a model-free reinforcement learning (RL) algorithm. This RL-based controller directly maps the visual observations to the joint positions of the snake-like robot in an end-to-end fashion instead of dividing the process into a series of sub-tasks. With a novel customized reward function, our RL controller is trained in a dynamically changing track scenario. The controller is evaluated in four different tracking scenarios and the results show excellent adaptive locomotion ability to the unpredictable behavior of the target. Meanwhile, the results also prove that the RL-based controller outperforms the traditional model-based controller in terms of tracking accuracy. Frontiers Media S.A. 2020-10-20 /pmc/articles/PMC7641616/ /pubmed/33192441 http://dx.doi.org/10.3389/fnbot.2020.591128 Text en Copyright © 2020 Bing, Lemke, Morin, Jiang, Cheng, Huang and Knoll. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Bing, Zhenshan
Lemke, Christian
Morin, Fabric O.
Jiang, Zhuangyi
Cheng, Long
Huang, Kai
Knoll, Alois
Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title_full Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title_fullStr Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title_full_unstemmed Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title_short Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning
title_sort perception-action coupling target tracking control for a snake robot via reinforcement learning
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7641616/
https://www.ncbi.nlm.nih.gov/pubmed/33192441
http://dx.doi.org/10.3389/fnbot.2020.591128
work_keys_str_mv AT bingzhenshan perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT lemkechristian perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT morinfabrico perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT jiangzhuangyi perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT chenglong perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT huangkai perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning
AT knollalois perceptionactioncouplingtargettrackingcontrolforasnakerobotviareinforcementlearning