Cargando…
Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomously in b...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7851139/ https://www.ncbi.nlm.nih.gov/pubmed/33526840 http://dx.doi.org/10.1038/s41598-021-81157-z |
_version_ | 1783645582864154624 |
---|---|
author | Walther, Thomas Diekmann, Nicolas Vijayabaskaran, Sandhiya Donoso, José R. Manahan-Vaughan, Denise Wiskott, Laurenz Cheng, Sen |
author_facet | Walther, Thomas Diekmann, Nicolas Vijayabaskaran, Sandhiya Donoso, José R. Manahan-Vaughan, Denise Wiskott, Laurenz Cheng, Sen |
author_sort | Walther, Thomas |
collection | PubMed |
description | The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomously in biologically realistic virtual reality environments based on raw camera inputs alone. Neither is context represented explicitly in our model, nor is context change signaled. We find that memory-intact agents learn distinct context representations, and develop ABA renewal, whereas memory-impaired agents do not. These findings reproduce the behavior of control and hippocampal animals, respectively. We therefore propose that the role of the hippocampus in the context-dependence of extinction learning might stem from its function in episodic-like memory and not in context-representation per se. We conclude that context-dependence can emerge from raw visual inputs. |
format | Online Article Text |
id | pubmed-7851139 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-78511392021-02-03 Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach Walther, Thomas Diekmann, Nicolas Vijayabaskaran, Sandhiya Donoso, José R. Manahan-Vaughan, Denise Wiskott, Laurenz Cheng, Sen Sci Rep Article The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomously in biologically realistic virtual reality environments based on raw camera inputs alone. Neither is context represented explicitly in our model, nor is context change signaled. We find that memory-intact agents learn distinct context representations, and develop ABA renewal, whereas memory-impaired agents do not. These findings reproduce the behavior of control and hippocampal animals, respectively. We therefore propose that the role of the hippocampus in the context-dependence of extinction learning might stem from its function in episodic-like memory and not in context-representation per se. We conclude that context-dependence can emerge from raw visual inputs. Nature Publishing Group UK 2021-02-01 /pmc/articles/PMC7851139/ /pubmed/33526840 http://dx.doi.org/10.1038/s41598-021-81157-z Text en © The Author(s) 2021 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
spellingShingle | Article Walther, Thomas Diekmann, Nicolas Vijayabaskaran, Sandhiya Donoso, José R. Manahan-Vaughan, Denise Wiskott, Laurenz Cheng, Sen Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title | Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title_full | Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title_fullStr | Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title_full_unstemmed | Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title_short | Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
title_sort | context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7851139/ https://www.ncbi.nlm.nih.gov/pubmed/33526840 http://dx.doi.org/10.1038/s41598-021-81157-z |
work_keys_str_mv | AT waltherthomas contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT diekmannnicolas contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT vijayabaskaransandhiya contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT donosojoser contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT manahanvaughandenise contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT wiskottlaurenz contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT chengsen contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach |