Cargando…

The anatomy of choice: dopamine and decision-making

This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax...

Descripción completa

Detalles Bibliográficos
Autores principales: Friston, Karl, Schwartenbeck, Philipp, FitzGerald, Thomas, Moutoussis, Michael, Behrens, Timothy, Dolan, Raymond J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: The Royal Society 2014
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4186234/
https://www.ncbi.nlm.nih.gov/pubmed/25267823
http://dx.doi.org/10.1098/rstb.2013.0481
_version_ 1782338024870248448
author Friston, Karl
Schwartenbeck, Philipp
FitzGerald, Thomas
Moutoussis, Michael
Behrens, Timothy
Dolan, Raymond J.
author_facet Friston, Karl
Schwartenbeck, Philipp
FitzGerald, Thomas
Moutoussis, Michael
Behrens, Timothy
Dolan, Raymond J.
author_sort Friston, Karl
collection PubMed
description This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax choice rules and optimism bias emerge as natural consequences of free energy minimization. Previous accounts of active inference have focused on predictive coding. In this paper, we consider variational Bayes as a scheme that the brain might use for approximate Bayesian inference. This scheme provides formal constraints on the computational anatomy of inference and action, which appear to be remarkably consistent with neuroanatomy. Active inference contextualizes optimal decision theory within embodied inference, where goals become prior beliefs. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (associated with softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution. Crucially, this sensitivity corresponds to the precision of beliefs about behaviour. The changes in precision during variational updates are remarkably reminiscent of empirical dopaminergic responses—and they may provide a new perspective on the role of dopamine in assimilating reward prediction errors to optimize decision-making.
format Online
Article
Text
id pubmed-4186234
institution National Center for Biotechnology Information
language English
publishDate 2014
publisher The Royal Society
record_format MEDLINE/PubMed
spelling pubmed-41862342014-11-05 The anatomy of choice: dopamine and decision-making Friston, Karl Schwartenbeck, Philipp FitzGerald, Thomas Moutoussis, Michael Behrens, Timothy Dolan, Raymond J. Philos Trans R Soc Lond B Biol Sci Part II: Formal and computational approaches to goal-directed decision-making This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax choice rules and optimism bias emerge as natural consequences of free energy minimization. Previous accounts of active inference have focused on predictive coding. In this paper, we consider variational Bayes as a scheme that the brain might use for approximate Bayesian inference. This scheme provides formal constraints on the computational anatomy of inference and action, which appear to be remarkably consistent with neuroanatomy. Active inference contextualizes optimal decision theory within embodied inference, where goals become prior beliefs. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (associated with softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution. Crucially, this sensitivity corresponds to the precision of beliefs about behaviour. The changes in precision during variational updates are remarkably reminiscent of empirical dopaminergic responses—and they may provide a new perspective on the role of dopamine in assimilating reward prediction errors to optimize decision-making. The Royal Society 2014-11-05 /pmc/articles/PMC4186234/ /pubmed/25267823 http://dx.doi.org/10.1098/rstb.2013.0481 Text en http://creativecommons.org/licenses/by/4.0/ © 2014 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.
spellingShingle Part II: Formal and computational approaches to goal-directed decision-making
Friston, Karl
Schwartenbeck, Philipp
FitzGerald, Thomas
Moutoussis, Michael
Behrens, Timothy
Dolan, Raymond J.
The anatomy of choice: dopamine and decision-making
title The anatomy of choice: dopamine and decision-making
title_full The anatomy of choice: dopamine and decision-making
title_fullStr The anatomy of choice: dopamine and decision-making
title_full_unstemmed The anatomy of choice: dopamine and decision-making
title_short The anatomy of choice: dopamine and decision-making
title_sort anatomy of choice: dopamine and decision-making
topic Part II: Formal and computational approaches to goal-directed decision-making
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4186234/
https://www.ncbi.nlm.nih.gov/pubmed/25267823
http://dx.doi.org/10.1098/rstb.2013.0481
work_keys_str_mv AT fristonkarl theanatomyofchoicedopamineanddecisionmaking
AT schwartenbeckphilipp theanatomyofchoicedopamineanddecisionmaking
AT fitzgeraldthomas theanatomyofchoicedopamineanddecisionmaking
AT moutoussismichael theanatomyofchoicedopamineanddecisionmaking
AT behrenstimothy theanatomyofchoicedopamineanddecisionmaking
AT dolanraymondj theanatomyofchoicedopamineanddecisionmaking
AT fristonkarl anatomyofchoicedopamineanddecisionmaking
AT schwartenbeckphilipp anatomyofchoicedopamineanddecisionmaking
AT fitzgeraldthomas anatomyofchoicedopamineanddecisionmaking
AT moutoussismichael anatomyofchoicedopamineanddecisionmaking
AT behrenstimothy anatomyofchoicedopamineanddecisionmaking
AT dolanraymondj anatomyofchoicedopamineanddecisionmaking