Cargando…

Average reward rates enable motivational transfer across independent reinforcement learning tasks

Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and...

Descripción completa

Detalles Bibliográficos
Autores principales: Aberg, Kristoffer C., Paz, Rony
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9682033/
https://www.ncbi.nlm.nih.gov/pubmed/36439970
http://dx.doi.org/10.3389/fnbeh.2022.1041566
_version_ 1784834757378441216
author Aberg, Kristoffer C.
Paz, Rony
author_facet Aberg, Kristoffer C.
Paz, Rony
author_sort Aberg, Kristoffer C.
collection PubMed
description Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and was found to interact with dopamine-sensitive cognitive processes, such as vigilance and associative memory encoding. The ARR could therefore provide a bridge between independent tasks when these are performed in temporal proximity, such that the reward rate obtained in one task could influence performance in a second subsequent task. Reinforcement learning depends on the coding of prediction error signals by dopamine neurons and their downstream targets, in particular the nucleus accumbens. Because these brain regions also respond to changes in ARR, reinforcement learning may be vulnerable to changes in ARR. To test this hypothesis, we designed a novel paradigm in which participants (n = 245) performed two probabilistic reinforcement learning tasks presented in interleaved trials. The ARR was controlled by an “induction” task which provided feedback with a low (p = 0.58), a medium (p = 0.75), or a high probability of reward (p = 0.92), while the impact of ARR on reinforcement learning was tested by a second “reference” task with a constant reward probability (p = 0.75). We find that performance was significantly lower in the reference task when the induction task provided low reward probabilities (i.e., during low levels of ARR), as compared to the medium and high ARR conditions. Behavioral modeling further revealed that the influence of ARR is best described by models which accumulates average rewards (rather than average prediction errors), and where the ARR directly modulates the prediction error signal (rather than affecting learning rates or exploration). Our results demonstrate how affective information in one domain may transfer and affect motivated behavior in other domains. These findings are particularly relevant for understanding mood disorders, but may also inform abnormal behaviors attributed to dopamine dysfunction.
format Online
Article
Text
id pubmed-9682033
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-96820332022-11-24 Average reward rates enable motivational transfer across independent reinforcement learning tasks Aberg, Kristoffer C. Paz, Rony Front Behav Neurosci Neuroscience Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and was found to interact with dopamine-sensitive cognitive processes, such as vigilance and associative memory encoding. The ARR could therefore provide a bridge between independent tasks when these are performed in temporal proximity, such that the reward rate obtained in one task could influence performance in a second subsequent task. Reinforcement learning depends on the coding of prediction error signals by dopamine neurons and their downstream targets, in particular the nucleus accumbens. Because these brain regions also respond to changes in ARR, reinforcement learning may be vulnerable to changes in ARR. To test this hypothesis, we designed a novel paradigm in which participants (n = 245) performed two probabilistic reinforcement learning tasks presented in interleaved trials. The ARR was controlled by an “induction” task which provided feedback with a low (p = 0.58), a medium (p = 0.75), or a high probability of reward (p = 0.92), while the impact of ARR on reinforcement learning was tested by a second “reference” task with a constant reward probability (p = 0.75). We find that performance was significantly lower in the reference task when the induction task provided low reward probabilities (i.e., during low levels of ARR), as compared to the medium and high ARR conditions. Behavioral modeling further revealed that the influence of ARR is best described by models which accumulates average rewards (rather than average prediction errors), and where the ARR directly modulates the prediction error signal (rather than affecting learning rates or exploration). Our results demonstrate how affective information in one domain may transfer and affect motivated behavior in other domains. These findings are particularly relevant for understanding mood disorders, but may also inform abnormal behaviors attributed to dopamine dysfunction. Frontiers Media S.A. 2022-11-09 /pmc/articles/PMC9682033/ /pubmed/36439970 http://dx.doi.org/10.3389/fnbeh.2022.1041566 Text en Copyright © 2022 Aberg and Paz. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Aberg, Kristoffer C.
Paz, Rony
Average reward rates enable motivational transfer across independent reinforcement learning tasks
title Average reward rates enable motivational transfer across independent reinforcement learning tasks
title_full Average reward rates enable motivational transfer across independent reinforcement learning tasks
title_fullStr Average reward rates enable motivational transfer across independent reinforcement learning tasks
title_full_unstemmed Average reward rates enable motivational transfer across independent reinforcement learning tasks
title_short Average reward rates enable motivational transfer across independent reinforcement learning tasks
title_sort average reward rates enable motivational transfer across independent reinforcement learning tasks
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9682033/
https://www.ncbi.nlm.nih.gov/pubmed/36439970
http://dx.doi.org/10.3389/fnbeh.2022.1041566
work_keys_str_mv AT abergkristofferc averagerewardratesenablemotivationaltransferacrossindependentreinforcementlearningtasks
AT pazrony averagerewardratesenablemotivationaltransferacrossindependentreinforcementlearningtasks