Cargando…

Functions of Learning Rate in Adaptive Reward Learning

As a crucial cognitive function, learning applies prediction error (the discrepancy between the prediction from learning and the world state) to adjust predictions of the future. How much prediction error affects this adjustment also depends on the learning rate. Our understanding to the learning ra...

Descripción completa

Detalles Bibliográficos
Autores principales: Wu, Xi, Wang, Ting, Liu, Chang, Wu, Tao, Jiang, Jiefeng, Zhou, Dong, Zhou, Jiliu
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5723661/
https://www.ncbi.nlm.nih.gov/pubmed/29270119
http://dx.doi.org/10.3389/fnhum.2017.00592
_version_ 1783285251783524352
author Wu, Xi
Wang, Ting
Liu, Chang
Wu, Tao
Jiang, Jiefeng
Zhou, Dong
Zhou, Jiliu
author_facet Wu, Xi
Wang, Ting
Liu, Chang
Wu, Tao
Jiang, Jiefeng
Zhou, Dong
Zhou, Jiliu
author_sort Wu, Xi
collection PubMed
description As a crucial cognitive function, learning applies prediction error (the discrepancy between the prediction from learning and the world state) to adjust predictions of the future. How much prediction error affects this adjustment also depends on the learning rate. Our understanding to the learning rate is still limited, in terms of (1) how it is modulated by other factors, and (2) the specific mechanisms of how learning rate interacts with prediction error to update learning. We applied computational modeling and functional magnetic resonance imaging to investigate these issues. We found that, when human participants performed a reward learning task, reward magnitude modulated learning rate. Modulation strength further predicted the difference in behavior following high vs. low reward across subjects. Imaging results further showed that this modulation was reflected in brain regions where the reward feedback is also encoded, such as the medial prefrontal cortex (MFC), precuneus, and posterior cingulate cortex. Furthermore, for the first time, we observed that the integration of the learning rate and the reward prediction error was represented in MFC activity. These findings extend our understanding of adaptive learning by demonstrating how it functions in a chain reaction of prediction updating.
format Online
Article
Text
id pubmed-5723661
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-57236612017-12-21 Functions of Learning Rate in Adaptive Reward Learning Wu, Xi Wang, Ting Liu, Chang Wu, Tao Jiang, Jiefeng Zhou, Dong Zhou, Jiliu Front Hum Neurosci Neuroscience As a crucial cognitive function, learning applies prediction error (the discrepancy between the prediction from learning and the world state) to adjust predictions of the future. How much prediction error affects this adjustment also depends on the learning rate. Our understanding to the learning rate is still limited, in terms of (1) how it is modulated by other factors, and (2) the specific mechanisms of how learning rate interacts with prediction error to update learning. We applied computational modeling and functional magnetic resonance imaging to investigate these issues. We found that, when human participants performed a reward learning task, reward magnitude modulated learning rate. Modulation strength further predicted the difference in behavior following high vs. low reward across subjects. Imaging results further showed that this modulation was reflected in brain regions where the reward feedback is also encoded, such as the medial prefrontal cortex (MFC), precuneus, and posterior cingulate cortex. Furthermore, for the first time, we observed that the integration of the learning rate and the reward prediction error was represented in MFC activity. These findings extend our understanding of adaptive learning by demonstrating how it functions in a chain reaction of prediction updating. Frontiers Media S.A. 2017-12-06 /pmc/articles/PMC5723661/ /pubmed/29270119 http://dx.doi.org/10.3389/fnhum.2017.00592 Text en Copyright © 2017 Wu, Wang, Liu, Wu, Jiang, Zhou and Zhou. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Wu, Xi
Wang, Ting
Liu, Chang
Wu, Tao
Jiang, Jiefeng
Zhou, Dong
Zhou, Jiliu
Functions of Learning Rate in Adaptive Reward Learning
title Functions of Learning Rate in Adaptive Reward Learning
title_full Functions of Learning Rate in Adaptive Reward Learning
title_fullStr Functions of Learning Rate in Adaptive Reward Learning
title_full_unstemmed Functions of Learning Rate in Adaptive Reward Learning
title_short Functions of Learning Rate in Adaptive Reward Learning
title_sort functions of learning rate in adaptive reward learning
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5723661/
https://www.ncbi.nlm.nih.gov/pubmed/29270119
http://dx.doi.org/10.3389/fnhum.2017.00592
work_keys_str_mv AT wuxi functionsoflearningrateinadaptiverewardlearning
AT wangting functionsoflearningrateinadaptiverewardlearning
AT liuchang functionsoflearningrateinadaptiverewardlearning
AT wutao functionsoflearningrateinadaptiverewardlearning
AT jiangjiefeng functionsoflearningrateinadaptiverewardlearning
AT zhoudong functionsoflearningrateinadaptiverewardlearning
AT zhoujiliu functionsoflearningrateinadaptiverewardlearning