Cargando…

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been e...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhang, Zhewei, Cheng, Huzi, Yang, Tianming
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7673505/
https://www.ncbi.nlm.nih.gov/pubmed/33141824
http://dx.doi.org/10.1371/journal.pcbi.1008342
_version_ 1783611331704782848
author Zhang, Zhewei
Cheng, Huzi
Yang, Tianming
author_facet Zhang, Zhewei
Cheng, Huzi
Yang, Tianming
author_sort Zhang, Zhewei
collection PubMed
description The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain.
format Online
Article
Text
id pubmed-7673505
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-76735052020-11-19 A recurrent neural network framework for flexible and adaptive decision making based on sequence learning Zhang, Zhewei Cheng, Huzi Yang, Tianming PLoS Comput Biol Research Article The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain. Public Library of Science 2020-11-03 /pmc/articles/PMC7673505/ /pubmed/33141824 http://dx.doi.org/10.1371/journal.pcbi.1008342 Text en © 2020 Zhang et al http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Zhang, Zhewei
Cheng, Huzi
Yang, Tianming
A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title_full A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title_fullStr A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title_full_unstemmed A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title_short A recurrent neural network framework for flexible and adaptive decision making based on sequence learning
title_sort recurrent neural network framework for flexible and adaptive decision making based on sequence learning
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7673505/
https://www.ncbi.nlm.nih.gov/pubmed/33141824
http://dx.doi.org/10.1371/journal.pcbi.1008342
work_keys_str_mv AT zhangzhewei arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning
AT chenghuzi arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning
AT yangtianming arecurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning
AT zhangzhewei recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning
AT chenghuzi recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning
AT yangtianming recurrentneuralnetworkframeworkforflexibleandadaptivedecisionmakingbasedonsequencelearning