Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction

To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot beh...

Descripción completa

Detalles Bibliográficos
Autores principales: Yamada, Tatsuro, Murata, Shingo, Arie, Hiroaki, Ogata, Tetsuya
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2016
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4946379/
https://www.ncbi.nlm.nih.gov/pubmed/27471463
http://dx.doi.org/10.3389/fnbot.2016.00005
_version_ 1782443017602334720
author Yamada, Tatsuro
Murata, Shingo
Arie, Hiroaki
Ogata, Tetsuya
author_facet Yamada, Tatsuro
Murata, Shingo
Arie, Hiroaki
Ogata, Tetsuya
author_sort Yamada, Tatsuro
collection PubMed
description To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot behavior by means of a recurrent neural network. In this method, the network learns from correct examples of the imposed task that are given not as explicitly separated sets of language and behavior but as sequential data constructed from the actual temporal flow of the task. By doing this, the internal dynamics of the network models both language–behavior relationships and the temporal patterns of interaction. Here, “internal dynamics” refers to the time development of the system defined on the fixed-dimensional space of the internal states of the context layer. Thus, in the execution phase, by constantly representing where in the interaction context it is as its current state, the network autonomously switches between recognition and generation phases without any explicit signs and utilizes the acquired mapping in appropriate contexts. To evaluate our method, we conducted an experiment in which a robot generates appropriate behavior responding to a human’s linguistic instruction. After learning, the network actually formed the attractor structure representing both language–behavior relationships and the task’s temporal pattern in its internal dynamics. In the dynamics, language–behavior mapping was achieved by the branching structure. Repetition of human’s instruction and robot’s behavioral response was represented as the cyclic structure, and besides, waiting to a subsequent instruction was represented as the fixed-point attractor. Thanks to this structure, the robot was able to interact online with a human concerning the given task by autonomously switching phases.
format Online
Article
Text
id pubmed-4946379
institution National Center for Biotechnology Information
language English
publishDate 2016
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-49463792016-07-28 Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction Yamada, Tatsuro Murata, Shingo Arie, Hiroaki Ogata, Tetsuya Front Neurorobot Neuroscience To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot behavior by means of a recurrent neural network. In this method, the network learns from correct examples of the imposed task that are given not as explicitly separated sets of language and behavior but as sequential data constructed from the actual temporal flow of the task. By doing this, the internal dynamics of the network models both language–behavior relationships and the temporal patterns of interaction. Here, “internal dynamics” refers to the time development of the system defined on the fixed-dimensional space of the internal states of the context layer. Thus, in the execution phase, by constantly representing where in the interaction context it is as its current state, the network autonomously switches between recognition and generation phases without any explicit signs and utilizes the acquired mapping in appropriate contexts. To evaluate our method, we conducted an experiment in which a robot generates appropriate behavior responding to a human’s linguistic instruction. After learning, the network actually formed the attractor structure representing both language–behavior relationships and the task’s temporal pattern in its internal dynamics. In the dynamics, language–behavior mapping was achieved by the branching structure. Repetition of human’s instruction and robot’s behavioral response was represented as the cyclic structure, and besides, waiting to a subsequent instruction was represented as the fixed-point attractor. Thanks to this structure, the robot was able to interact online with a human concerning the given task by autonomously switching phases. Frontiers Media S.A. 2016-07-15 /pmc/articles/PMC4946379/ /pubmed/27471463 http://dx.doi.org/10.3389/fnbot.2016.00005 Text en Copyright © 2016 Yamada, Murata, Arie and Ogata. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Yamada, Tatsuro
Murata, Shingo
Arie, Hiroaki
Ogata, Tetsuya
Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title_full Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title_fullStr Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title_full_unstemmed Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title_short Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human–Robot Interaction
title_sort dynamical integration of language and behavior in a recurrent neural network for human–robot interaction
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4946379/
https://www.ncbi.nlm.nih.gov/pubmed/27471463
http://dx.doi.org/10.3389/fnbot.2016.00005
work_keys_str_mv AT yamadatatsuro dynamicalintegrationoflanguageandbehaviorinarecurrentneuralnetworkforhumanrobotinteraction
AT muratashingo dynamicalintegrationoflanguageandbehaviorinarecurrentneuralnetworkforhumanrobotinteraction
AT ariehiroaki dynamicalintegrationoflanguageandbehaviorinarecurrentneuralnetworkforhumanrobotinteraction
AT ogatatetsuya dynamicalintegrationoflanguageandbehaviorinarecurrentneuralnetworkforhumanrobotinteraction