Cargando…
Unconscious reinforcement learning of hidden brain states supported by confidence
Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time b...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7459278/ https://www.ncbi.nlm.nih.gov/pubmed/32868772 http://dx.doi.org/10.1038/s41467-020-17828-8 |
_version_ | 1783576339321716736 |
---|---|
author | Cortese, Aurelio Lau, Hakwan Kawato, Mitsuo |
author_facet | Cortese, Aurelio Lau, Hakwan Kawato, Mitsuo |
author_sort | Cortese, Aurelio |
collection | PubMed |
description | Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time by fMRI multivoxel patterns; optimal action policies thereby depend on multidimensional brain activity taking place below the threshold of consciousness, by design. We find that subjects can solve the task within two hundred trials and errors, as their reinforcement learning processes interact with metacognitive functions (quantified as the meaningfulness of their decision confidence). Computational modelling and multivariate analyses identify a frontostriatal neural mechanism by which the brain may untangle the ‘curse of dimensionality’: synchronization of confidence representations in prefrontal cortex with reward prediction errors in basal ganglia support exploration of latent task representations. These results may provide an alternative starting point for future investigations into unconscious learning and functions of metacognition. |
format | Online Article Text |
id | pubmed-7459278 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-74592782020-09-16 Unconscious reinforcement learning of hidden brain states supported by confidence Cortese, Aurelio Lau, Hakwan Kawato, Mitsuo Nat Commun Article Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time by fMRI multivoxel patterns; optimal action policies thereby depend on multidimensional brain activity taking place below the threshold of consciousness, by design. We find that subjects can solve the task within two hundred trials and errors, as their reinforcement learning processes interact with metacognitive functions (quantified as the meaningfulness of their decision confidence). Computational modelling and multivariate analyses identify a frontostriatal neural mechanism by which the brain may untangle the ‘curse of dimensionality’: synchronization of confidence representations in prefrontal cortex with reward prediction errors in basal ganglia support exploration of latent task representations. These results may provide an alternative starting point for future investigations into unconscious learning and functions of metacognition. Nature Publishing Group UK 2020-08-31 /pmc/articles/PMC7459278/ /pubmed/32868772 http://dx.doi.org/10.1038/s41467-020-17828-8 Text en © The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. |
spellingShingle | Article Cortese, Aurelio Lau, Hakwan Kawato, Mitsuo Unconscious reinforcement learning of hidden brain states supported by confidence |
title | Unconscious reinforcement learning of hidden brain states supported by confidence |
title_full | Unconscious reinforcement learning of hidden brain states supported by confidence |
title_fullStr | Unconscious reinforcement learning of hidden brain states supported by confidence |
title_full_unstemmed | Unconscious reinforcement learning of hidden brain states supported by confidence |
title_short | Unconscious reinforcement learning of hidden brain states supported by confidence |
title_sort | unconscious reinforcement learning of hidden brain states supported by confidence |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7459278/ https://www.ncbi.nlm.nih.gov/pubmed/32868772 http://dx.doi.org/10.1038/s41467-020-17828-8 |
work_keys_str_mv | AT corteseaurelio unconsciousreinforcementlearningofhiddenbrainstatessupportedbyconfidence AT lauhakwan unconsciousreinforcementlearningofhiddenbrainstatessupportedbyconfidence AT kawatomitsuo unconsciousreinforcementlearningofhiddenbrainstatessupportedbyconfidence |