Cargando…

Spatial relation learning in complementary scenarios with deep neural networks

A cognitive agent performing in the real world needs to learn relevant concepts about its environment (e.g., objects, color, and shapes) and react accordingly. In addition to learning the concepts, it needs to learn relations between the concepts, in particular spatial relations between objects. In...

Descripción completa

Detalles Bibliográficos
Autores principales: Lee, Jae Hee, Yao, Yuan, Özdemir, Ozan, Li, Mengdi, Weber, Cornelius, Liu, Zhiyuan, Wermter, Stefan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9366005/
https://www.ncbi.nlm.nih.gov/pubmed/35966371
http://dx.doi.org/10.3389/fnbot.2022.844753
_version_ 1784765460726677504
author Lee, Jae Hee
Yao, Yuan
Özdemir, Ozan
Li, Mengdi
Weber, Cornelius
Liu, Zhiyuan
Wermter, Stefan
author_facet Lee, Jae Hee
Yao, Yuan
Özdemir, Ozan
Li, Mengdi
Weber, Cornelius
Liu, Zhiyuan
Wermter, Stefan
author_sort Lee, Jae Hee
collection PubMed
description A cognitive agent performing in the real world needs to learn relevant concepts about its environment (e.g., objects, color, and shapes) and react accordingly. In addition to learning the concepts, it needs to learn relations between the concepts, in particular spatial relations between objects. In this paper, we propose three approaches that allow a cognitive agent to learn spatial relations. First, using an embodied model, the agent learns to reach toward an object based on simple instructions involving left-right relations. Since the level of realism and its complexity does not permit large-scale and diverse experiences in this approach, we devise as a second approach a simple visual dataset for geometric feature learning and show that recent reasoning models can learn directional relations in different frames of reference. Yet, embodied and simple simulation approaches together still do not provide sufficient experiences. To close this gap, we thirdly propose utilizing knowledge bases for disembodied spatial relation reasoning. Since the three approaches (i.e., embodied learning, learning from simple visual data, and use of knowledge bases) are complementary, we conceptualize a cognitive architecture that combines these approaches in the context of spatial relation learning.
format Online
Article
Text
id pubmed-9366005
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-93660052022-08-12 Spatial relation learning in complementary scenarios with deep neural networks Lee, Jae Hee Yao, Yuan Özdemir, Ozan Li, Mengdi Weber, Cornelius Liu, Zhiyuan Wermter, Stefan Front Neurorobot Neuroscience A cognitive agent performing in the real world needs to learn relevant concepts about its environment (e.g., objects, color, and shapes) and react accordingly. In addition to learning the concepts, it needs to learn relations between the concepts, in particular spatial relations between objects. In this paper, we propose three approaches that allow a cognitive agent to learn spatial relations. First, using an embodied model, the agent learns to reach toward an object based on simple instructions involving left-right relations. Since the level of realism and its complexity does not permit large-scale and diverse experiences in this approach, we devise as a second approach a simple visual dataset for geometric feature learning and show that recent reasoning models can learn directional relations in different frames of reference. Yet, embodied and simple simulation approaches together still do not provide sufficient experiences. To close this gap, we thirdly propose utilizing knowledge bases for disembodied spatial relation reasoning. Since the three approaches (i.e., embodied learning, learning from simple visual data, and use of knowledge bases) are complementary, we conceptualize a cognitive architecture that combines these approaches in the context of spatial relation learning. Frontiers Media S.A. 2022-07-28 /pmc/articles/PMC9366005/ /pubmed/35966371 http://dx.doi.org/10.3389/fnbot.2022.844753 Text en Copyright © 2022 Lee, Yao, Özdemir, Li, Weber, Liu and Wermter. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Lee, Jae Hee
Yao, Yuan
Özdemir, Ozan
Li, Mengdi
Weber, Cornelius
Liu, Zhiyuan
Wermter, Stefan
Spatial relation learning in complementary scenarios with deep neural networks
title Spatial relation learning in complementary scenarios with deep neural networks
title_full Spatial relation learning in complementary scenarios with deep neural networks
title_fullStr Spatial relation learning in complementary scenarios with deep neural networks
title_full_unstemmed Spatial relation learning in complementary scenarios with deep neural networks
title_short Spatial relation learning in complementary scenarios with deep neural networks
title_sort spatial relation learning in complementary scenarios with deep neural networks
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9366005/
https://www.ncbi.nlm.nih.gov/pubmed/35966371
http://dx.doi.org/10.3389/fnbot.2022.844753
work_keys_str_mv AT leejaehee spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT yaoyuan spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT ozdemirozan spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT limengdi spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT webercornelius spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT liuzhiyuan spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks
AT wermterstefan spatialrelationlearningincomplementaryscenarioswithdeepneuralnetworks