Cargando…

Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales

Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a fail...

Descripción completa

Detalles Bibliográficos
Autores principales: Iigaya, Kiyohito, Ahmadian, Yashar, Sugrue, Leo P., Corrado, Greg S., Loewenstein, Yonatan, Newsome, William T., Fusi, Stefano
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6443814/
https://www.ncbi.nlm.nih.gov/pubmed/30931937
http://dx.doi.org/10.1038/s41467-019-09388-3
_version_ 1783407900860874752
author Iigaya, Kiyohito
Ahmadian, Yashar
Sugrue, Leo P.
Corrado, Greg S.
Loewenstein, Yonatan
Newsome, William T.
Fusi, Stefano
author_facet Iigaya, Kiyohito
Ahmadian, Yashar
Sugrue, Leo P.
Corrado, Greg S.
Loewenstein, Yonatan
Newsome, William T.
Fusi, Stefano
author_sort Iigaya, Kiyohito
collection PubMed
description Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty.
format Online
Article
Text
id pubmed-6443814
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-64438142019-04-03 Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales Iigaya, Kiyohito Ahmadian, Yashar Sugrue, Leo P. Corrado, Greg S. Loewenstein, Yonatan Newsome, William T. Fusi, Stefano Nat Commun Article Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty. Nature Publishing Group UK 2019-04-01 /pmc/articles/PMC6443814/ /pubmed/30931937 http://dx.doi.org/10.1038/s41467-019-09388-3 Text en © The Author(s) 2019 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
spellingShingle Article
Iigaya, Kiyohito
Ahmadian, Yashar
Sugrue, Leo P.
Corrado, Greg S.
Loewenstein, Yonatan
Newsome, William T.
Fusi, Stefano
Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title_full Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title_fullStr Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title_full_unstemmed Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title_short Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
title_sort deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6443814/
https://www.ncbi.nlm.nih.gov/pubmed/30931937
http://dx.doi.org/10.1038/s41467-019-09388-3
work_keys_str_mv AT iigayakiyohito deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT ahmadianyashar deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT sugrueleop deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT corradogregs deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT loewensteinyonatan deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT newsomewilliamt deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales
AT fusistefano deviationfromthematchinglawreflectsanoptimalstrategyinvolvinglearningovermultipletimescales