Cargando…
The shadowing effect of initial expectation on learning asymmetry
Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exist in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity differen...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10399892/ https://www.ncbi.nlm.nih.gov/pubmed/37486955 http://dx.doi.org/10.1371/journal.pcbi.1010751 |
_version_ | 1785084347540307968 |
---|---|
author | Ni, Yinmei Sun, Jingwei Li, Jian |
author_facet | Ni, Yinmei Sun, Jingwei Li, Jian |
author_sort | Ni, Yinmei |
collection | PubMed |
description | Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exist in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity difference in incorporating positive and negative prediction errors (PE) into value estimation, namely the asymmetry of learning rates associated with positive and negative PEs. Although RL has been established as a canonical framework in characterizing interactions between agent and environment, the direction of learning asymmetry remains controversial. Here, we propose that part of the controversy stems from the fact that people may have different value expectations before entering the learning environment. Such a default value expectation influences how PEs are calculated and consequently biases subjects’ choices. We test this hypothesis in two learning experiments with stable or varying reinforcement probabilities, across monetary gains, losses, and gain-loss mixed environments. Our results consistently support the model incorporating both asymmetric learning rates and the initial value expectation, highlighting the role of initial expectation in value updating and choice preference. Further simulation and model parameter recovery analyses confirm the unique contribution of initial value expectation in accessing learning rate asymmetry. |
format | Online Article Text |
id | pubmed-10399892 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-103998922023-08-04 The shadowing effect of initial expectation on learning asymmetry Ni, Yinmei Sun, Jingwei Li, Jian PLoS Comput Biol Research Article Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exist in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity difference in incorporating positive and negative prediction errors (PE) into value estimation, namely the asymmetry of learning rates associated with positive and negative PEs. Although RL has been established as a canonical framework in characterizing interactions between agent and environment, the direction of learning asymmetry remains controversial. Here, we propose that part of the controversy stems from the fact that people may have different value expectations before entering the learning environment. Such a default value expectation influences how PEs are calculated and consequently biases subjects’ choices. We test this hypothesis in two learning experiments with stable or varying reinforcement probabilities, across monetary gains, losses, and gain-loss mixed environments. Our results consistently support the model incorporating both asymmetric learning rates and the initial value expectation, highlighting the role of initial expectation in value updating and choice preference. Further simulation and model parameter recovery analyses confirm the unique contribution of initial value expectation in accessing learning rate asymmetry. Public Library of Science 2023-07-24 /pmc/articles/PMC10399892/ /pubmed/37486955 http://dx.doi.org/10.1371/journal.pcbi.1010751 Text en © 2023 Ni et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Ni, Yinmei Sun, Jingwei Li, Jian The shadowing effect of initial expectation on learning asymmetry |
title | The shadowing effect of initial expectation on learning asymmetry |
title_full | The shadowing effect of initial expectation on learning asymmetry |
title_fullStr | The shadowing effect of initial expectation on learning asymmetry |
title_full_unstemmed | The shadowing effect of initial expectation on learning asymmetry |
title_short | The shadowing effect of initial expectation on learning asymmetry |
title_sort | shadowing effect of initial expectation on learning asymmetry |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10399892/ https://www.ncbi.nlm.nih.gov/pubmed/37486955 http://dx.doi.org/10.1371/journal.pcbi.1010751 |
work_keys_str_mv | AT niyinmei theshadowingeffectofinitialexpectationonlearningasymmetry AT sunjingwei theshadowingeffectofinitialexpectationonlearningasymmetry AT lijian theshadowingeffectofinitialexpectationonlearningasymmetry AT niyinmei shadowingeffectofinitialexpectationonlearningasymmetry AT sunjingwei shadowingeffectofinitialexpectationonlearningasymmetry AT lijian shadowingeffectofinitialexpectationonlearningasymmetry |