Cargando…

Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans

Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory...

Descripción completa

Detalles Bibliográficos
Autores principales: Lee, Julie J., Keramati, Mehdi
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5634647/
https://www.ncbi.nlm.nih.gov/pubmed/28957319
http://dx.doi.org/10.1371/journal.pcbi.1005753
_version_ 1783270134331211776
author Lee, Julie J.
Keramati, Mehdi
author_facet Lee, Julie J.
Keramati, Mehdi
author_sort Lee, Julie J.
collection PubMed
description Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter. Therefore, we developed a novel two-level contingency change task in which transition contingencies between states change every few trials; MB and MF control predict different responses following these contingency changes, allowing their relative influence to be inferred. Additionally, we manipulated the rate of contingency changes in order to determine whether contingency change volatility would play a role in shifting subjects between a MB and MF strategy. We found that human subjects employed a hybrid MB/MF strategy on the task, corroborating the parallel contribution of MB and MF systems in reinforcement learning. Further, subjects did not remain at one level of MB/MF behaviour but rather displayed a shift towards more MB behavior over the first two blocks that was not attributable to the rate of contingency changes but rather to the extent of training. We demonstrate that flexibility to contingency changes can distinguish MB and MF strategies, with human subjects utilizing a hybrid strategy that shifts towards more MB behavior over blocks, consequently corresponding to a higher payoff.
format Online
Article
Text
id pubmed-5634647
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-56346472017-10-30 Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans Lee, Julie J. Keramati, Mehdi PLoS Comput Biol Research Article Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter. Therefore, we developed a novel two-level contingency change task in which transition contingencies between states change every few trials; MB and MF control predict different responses following these contingency changes, allowing their relative influence to be inferred. Additionally, we manipulated the rate of contingency changes in order to determine whether contingency change volatility would play a role in shifting subjects between a MB and MF strategy. We found that human subjects employed a hybrid MB/MF strategy on the task, corroborating the parallel contribution of MB and MF systems in reinforcement learning. Further, subjects did not remain at one level of MB/MF behaviour but rather displayed a shift towards more MB behavior over the first two blocks that was not attributable to the rate of contingency changes but rather to the extent of training. We demonstrate that flexibility to contingency changes can distinguish MB and MF strategies, with human subjects utilizing a hybrid strategy that shifts towards more MB behavior over blocks, consequently corresponding to a higher payoff. Public Library of Science 2017-09-28 /pmc/articles/PMC5634647/ /pubmed/28957319 http://dx.doi.org/10.1371/journal.pcbi.1005753 Text en © 2017 Lee, Keramati http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Lee, Julie J.
Keramati, Mehdi
Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title_full Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title_fullStr Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title_full_unstemmed Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title_short Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
title_sort flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5634647/
https://www.ncbi.nlm.nih.gov/pubmed/28957319
http://dx.doi.org/10.1371/journal.pcbi.1005753
work_keys_str_mv AT leejuliej flexibilitytocontingencychangesdistinguisheshabitualandgoaldirectedstrategiesinhumans
AT keramatimehdi flexibilitytocontingencychangesdistinguisheshabitualandgoaldirectedstrategiesinhumans