Cargando…
Modeling Task Uncertainty for Safe Meta-Imitation Learning
To endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of perform...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7805769/ https://www.ncbi.nlm.nih.gov/pubmed/33501364 http://dx.doi.org/10.3389/frobt.2020.606361 |
_version_ | 1783636377162743808 |
---|---|
author | Matsushima, Tatsuya Kondo, Naruya Iwasawa, Yusuke Nasuno, Kaoru Matsuo, Yutaka |
author_facet | Matsushima, Tatsuya Kondo, Naruya Iwasawa, Yusuke Nasuno, Kaoru Matsuo, Yutaka |
author_sort | Matsushima, Tatsuya |
collection | PubMed |
description | To endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of performing other tasks during training. Although studies around meta-learning of robot control have worked on improving the performance, the safety issue has not been fully explored, which is also an important consideration in the deployment. In this paper, we firstly relate uncertainty on task inference with the safety in meta-learning of visual imitation, and then propose a novel framework for estimating the task uncertainty through probabilistic inference in the task-embedding space, called PETNet. We validate PETNet with a manipulation task with a simulated robot arm in terms of the task performance and uncertainty evaluation on task inference. Following the standard benchmark procedure in meta-imitation learning, we show PETNet can achieve the same or higher level of performance (success rate of novel tasks at meta-test time) as previous methods. In addition, by testing PETNet with semantically inappropriate or synthesized out-of-distribution demonstrations, PETNet shows the ability to capture the uncertainty about the tasks inherent in the given demonstrations, which allows the robot to identify situations where the controller might not perform properly. These results illustrate our proposal takes a significant step forward to the safe deployment of robot learning systems into diverse tasks and environments. |
format | Online Article Text |
id | pubmed-7805769 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-78057692021-01-25 Modeling Task Uncertainty for Safe Meta-Imitation Learning Matsushima, Tatsuya Kondo, Naruya Iwasawa, Yusuke Nasuno, Kaoru Matsuo, Yutaka Front Robot AI Robotics and AI To endow robots with the flexibility to perform a wide range of tasks in diverse and complex environments, learning their controller from experience data is a promising approach. In particular, some recent meta-learning methods are shown to solve novel tasks by leveraging their experience of performing other tasks during training. Although studies around meta-learning of robot control have worked on improving the performance, the safety issue has not been fully explored, which is also an important consideration in the deployment. In this paper, we firstly relate uncertainty on task inference with the safety in meta-learning of visual imitation, and then propose a novel framework for estimating the task uncertainty through probabilistic inference in the task-embedding space, called PETNet. We validate PETNet with a manipulation task with a simulated robot arm in terms of the task performance and uncertainty evaluation on task inference. Following the standard benchmark procedure in meta-imitation learning, we show PETNet can achieve the same or higher level of performance (success rate of novel tasks at meta-test time) as previous methods. In addition, by testing PETNet with semantically inappropriate or synthesized out-of-distribution demonstrations, PETNet shows the ability to capture the uncertainty about the tasks inherent in the given demonstrations, which allows the robot to identify situations where the controller might not perform properly. These results illustrate our proposal takes a significant step forward to the safe deployment of robot learning systems into diverse tasks and environments. Frontiers Media S.A. 2020-11-27 /pmc/articles/PMC7805769/ /pubmed/33501364 http://dx.doi.org/10.3389/frobt.2020.606361 Text en Copyright © 2020 Matsushima, Kondo, Iwasawa, Nasuno and Matsuo. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Robotics and AI Matsushima, Tatsuya Kondo, Naruya Iwasawa, Yusuke Nasuno, Kaoru Matsuo, Yutaka Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title_full | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title_fullStr | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title_full_unstemmed | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title_short | Modeling Task Uncertainty for Safe Meta-Imitation Learning |
title_sort | modeling task uncertainty for safe meta-imitation learning |
topic | Robotics and AI |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7805769/ https://www.ncbi.nlm.nih.gov/pubmed/33501364 http://dx.doi.org/10.3389/frobt.2020.606361 |
work_keys_str_mv | AT matsushimatatsuya modelingtaskuncertaintyforsafemetaimitationlearning AT kondonaruya modelingtaskuncertaintyforsafemetaimitationlearning AT iwasawayusuke modelingtaskuncertaintyforsafemetaimitationlearning AT nasunokaoru modelingtaskuncertaintyforsafemetaimitationlearning AT matsuoyutaka modelingtaskuncertaintyforsafemetaimitationlearning |