Cargando…
Making predictions in a changing world—inference, uncertainty, and learning
To function effectively, brains need to make predictions about their environment based on past experience, i.e., they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a...
Autor principal: | |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2013
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3682109/ https://www.ncbi.nlm.nih.gov/pubmed/23785310 http://dx.doi.org/10.3389/fnins.2013.00105 |
_version_ | 1782273355607441408 |
---|---|
author | O'Reilly, Jill X. |
author_facet | O'Reilly, Jill X. |
author_sort | O'Reilly, Jill X. |
collection | PubMed |
description | To function effectively, brains need to make predictions about their environment based on past experience, i.e., they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a tool to model participants' incomplete knowledge of task parameters and hence, to better understand their behavior. This review focusses on a particular challenge for learning algorithms—how to match the rate at which they learn to the rate of change in the environment, so that they use as much observed data as possible whilst disregarding irrelevant, old observations. To do this algorithms must evaluate whether the environment is changing. We discuss the concepts of likelihood, priors and transition functions, and how these relate to change detection. We review expected and estimation uncertainty, and how these relate to change detection and learning rate. Finally, we consider the neural correlates of uncertainty and learning. We argue that the neural correlates of uncertainty bear a resemblance to neural systems that are active when agents actively explore their environments, suggesting that the mechanisms by which the rate of learning is set may be subject to top down control (in circumstances when agents actively seek new information) as well as bottom up control (by observations that imply change in the environment). |
format | Online Article Text |
id | pubmed-3682109 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2013 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-36821092013-06-19 Making predictions in a changing world—inference, uncertainty, and learning O'Reilly, Jill X. Front Neurosci Neuroscience To function effectively, brains need to make predictions about their environment based on past experience, i.e., they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a tool to model participants' incomplete knowledge of task parameters and hence, to better understand their behavior. This review focusses on a particular challenge for learning algorithms—how to match the rate at which they learn to the rate of change in the environment, so that they use as much observed data as possible whilst disregarding irrelevant, old observations. To do this algorithms must evaluate whether the environment is changing. We discuss the concepts of likelihood, priors and transition functions, and how these relate to change detection. We review expected and estimation uncertainty, and how these relate to change detection and learning rate. Finally, we consider the neural correlates of uncertainty and learning. We argue that the neural correlates of uncertainty bear a resemblance to neural systems that are active when agents actively explore their environments, suggesting that the mechanisms by which the rate of learning is set may be subject to top down control (in circumstances when agents actively seek new information) as well as bottom up control (by observations that imply change in the environment). Frontiers Media S.A. 2013-06-14 /pmc/articles/PMC3682109/ /pubmed/23785310 http://dx.doi.org/10.3389/fnins.2013.00105 Text en Copyright © 2013 O'Reilly. http://creativecommons.org/licenses/by/3.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc. |
spellingShingle | Neuroscience O'Reilly, Jill X. Making predictions in a changing world—inference, uncertainty, and learning |
title | Making predictions in a changing world—inference, uncertainty, and learning |
title_full | Making predictions in a changing world—inference, uncertainty, and learning |
title_fullStr | Making predictions in a changing world—inference, uncertainty, and learning |
title_full_unstemmed | Making predictions in a changing world—inference, uncertainty, and learning |
title_short | Making predictions in a changing world—inference, uncertainty, and learning |
title_sort | making predictions in a changing world—inference, uncertainty, and learning |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3682109/ https://www.ncbi.nlm.nih.gov/pubmed/23785310 http://dx.doi.org/10.3389/fnins.2013.00105 |
work_keys_str_mv | AT oreillyjillx makingpredictionsinachangingworldinferenceuncertaintyandlearning |