Cargando…

Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity

Several studies have shown a strong involvement of the basal ganglia (BG) in action selection and dopamine dependent learning. The dopaminergic signal to striatum, the input stage of the BG, has been commonly described as coding a reward prediction error (RPE), i.e., the difference between the predi...

Descripción completa

Detalles Bibliográficos
Autores principales: Berthet, Pierre, Hellgren-Kotaleski, Jeanette, Lansner, Anders
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2012
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462417/
https://www.ncbi.nlm.nih.gov/pubmed/23060764
http://dx.doi.org/10.3389/fnbeh.2012.00065
_version_ 1782245174912483328
author Berthet, Pierre
Hellgren-Kotaleski, Jeanette
Lansner, Anders
author_facet Berthet, Pierre
Hellgren-Kotaleski, Jeanette
Lansner, Anders
author_sort Berthet, Pierre
collection PubMed
description Several studies have shown a strong involvement of the basal ganglia (BG) in action selection and dopamine dependent learning. The dopaminergic signal to striatum, the input stage of the BG, has been commonly described as coding a reward prediction error (RPE), i.e., the difference between the predicted and actual reward. The RPE has been hypothesized to be critical in the modulation of the synaptic plasticity in cortico-striatal synapses in the direct and indirect pathway. We developed an abstract computational model of the BG, with a dual pathway structure functionally corresponding to the direct and indirect pathways, and compared its behavior to biological data as well as other reinforcement learning models. The computations in our model are inspired by Bayesian inference, and the synaptic plasticity changes depend on a three factor Hebbian–Bayesian learning rule based on co-activation of pre- and post-synaptic units and on the value of the RPE. The model builds on a modified Actor-Critic architecture and implements the direct (Go) and the indirect (NoGo) pathway, as well as the reward prediction (RP) system, acting in a complementary fashion. We investigated the performance of the model system when different configurations of the Go, NoGo, and RP system were utilized, e.g., using only the Go, NoGo, or RP system, or combinations of those. Learning performance was investigated in several types of learning paradigms, such as learning-relearning, successive learning, stochastic learning, reversal learning and a two-choice task. The RPE and the activity of the model during learning were similar to monkey electrophysiological and behavioral data. Our results, however, show that there is not a unique best way to configure this BG model to handle well all the learning paradigms tested. We thus suggest that an agent might dynamically configure its action selection mode, possibly depending on task characteristics and also on how much time is available.
format Online
Article
Text
id pubmed-3462417
institution National Center for Biotechnology Information
language English
publishDate 2012
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-34624172012-10-11 Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity Berthet, Pierre Hellgren-Kotaleski, Jeanette Lansner, Anders Front Behav Neurosci Neuroscience Several studies have shown a strong involvement of the basal ganglia (BG) in action selection and dopamine dependent learning. The dopaminergic signal to striatum, the input stage of the BG, has been commonly described as coding a reward prediction error (RPE), i.e., the difference between the predicted and actual reward. The RPE has been hypothesized to be critical in the modulation of the synaptic plasticity in cortico-striatal synapses in the direct and indirect pathway. We developed an abstract computational model of the BG, with a dual pathway structure functionally corresponding to the direct and indirect pathways, and compared its behavior to biological data as well as other reinforcement learning models. The computations in our model are inspired by Bayesian inference, and the synaptic plasticity changes depend on a three factor Hebbian–Bayesian learning rule based on co-activation of pre- and post-synaptic units and on the value of the RPE. The model builds on a modified Actor-Critic architecture and implements the direct (Go) and the indirect (NoGo) pathway, as well as the reward prediction (RP) system, acting in a complementary fashion. We investigated the performance of the model system when different configurations of the Go, NoGo, and RP system were utilized, e.g., using only the Go, NoGo, or RP system, or combinations of those. Learning performance was investigated in several types of learning paradigms, such as learning-relearning, successive learning, stochastic learning, reversal learning and a two-choice task. The RPE and the activity of the model during learning were similar to monkey electrophysiological and behavioral data. Our results, however, show that there is not a unique best way to configure this BG model to handle well all the learning paradigms tested. We thus suggest that an agent might dynamically configure its action selection mode, possibly depending on task characteristics and also on how much time is available. Frontiers Media S.A. 2012-10-02 /pmc/articles/PMC3462417/ /pubmed/23060764 http://dx.doi.org/10.3389/fnbeh.2012.00065 Text en Copyright © 2012 Berthet, Hellgren-Kotaleski and Lansner. http://www.frontiersin.org/licenseagreement This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
spellingShingle Neuroscience
Berthet, Pierre
Hellgren-Kotaleski, Jeanette
Lansner, Anders
Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title_full Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title_fullStr Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title_full_unstemmed Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title_short Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian–Bayesian Go-NoGo connectivity
title_sort action selection performance of a reconfigurable basal ganglia inspired model with hebbian–bayesian go-nogo connectivity
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3462417/
https://www.ncbi.nlm.nih.gov/pubmed/23060764
http://dx.doi.org/10.3389/fnbeh.2012.00065
work_keys_str_mv AT berthetpierre actionselectionperformanceofareconfigurablebasalgangliainspiredmodelwithhebbianbayesiangonogoconnectivity
AT hellgrenkotaleskijeanette actionselectionperformanceofareconfigurablebasalgangliainspiredmodelwithhebbianbayesiangonogoconnectivity
AT lansneranders actionselectionperformanceofareconfigurablebasalgangliainspiredmodelwithhebbianbayesiangonogoconnectivity