Cargando…
A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been e...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7673505/ https://www.ncbi.nlm.nih.gov/pubmed/33141824 http://dx.doi.org/10.1371/journal.pcbi.1008342 |
_version_ | 1783611331704782848 |
---|---|
author | Zhang, Zhewei Cheng, Huzi Yang, Tianming |
author_facet | Zhang, Zhewei Cheng, Huzi Yang, Tianming |
author_sort | Zhang, Zhewei |
collection | PubMed |
description | The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain. |
format | Online Article Text |
id | pubmed-7673505 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-76735052020-11-19 A recurrent neural network framework for flexible and adaptive decision making based on sequence learning Zhang, Zhewei Cheng, Huzi Yang, Tianming PLoS Comput Biol Research Article The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain. Public Library of Science 2020-11-03 /pmc/articles/PMC7673505/ /pubmed/33141824 http://dx.doi.org/10.1371/journal.pcbi.1008342 Text en © 2020 Zhang et al http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Zhang, Zhewei Cheng, Huzi Yang, Tianming A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title | A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title_full | A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title_fullStr | A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title_full_unstemmed | A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title_short | A recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
title_sort | recurrent neural network framework for flexible and adaptive decision making based on sequence learning |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7673505/ https://www.ncbi.nlm.nih.gov/pubmed/33141824 http://dx.doi.org/10.1371/journal.pcbi.1008342 |
work_keys_str_mv | AT zhangzhewei arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning AT chenghuzi arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning AT yangtianming arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning AT zhangzhewei recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning AT chenghuzi recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning AT yangtianming recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning |