Cargando…
Robust Exponential Memory in Hopfield Networks
The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically coupled McCulloch–Pitts binary neurons interact to perform emergent computation. Although previous researchers have explored the potential of this network to solve combinatori...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer Berlin Heidelberg
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5770423/ https://www.ncbi.nlm.nih.gov/pubmed/29340803 http://dx.doi.org/10.1186/s13408-017-0056-2 |
_version_ | 1783293065763487744 |
---|---|
author | Hillar, Christopher J. Tran, Ngoc M. |
author_facet | Hillar, Christopher J. Tran, Ngoc M. |
author_sort | Hillar, Christopher J. |
collection | PubMed |
description | The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically coupled McCulloch–Pitts binary neurons interact to perform emergent computation. Although previous researchers have explored the potential of this network to solve combinatorial optimization problems or store reoccurring activity patterns as attractors of its deterministic dynamics, a basic open problem is to design a family of Hopfield networks with a number of noise-tolerant memories that grows exponentially with neural population size. Here, we discover such networks by minimizing probability flow, a recently proposed objective for estimating parameters in discrete maximum entropy models. By descending the gradient of the convex probability flow, our networks adapt synaptic weights to achieve robust exponential storage, even when presented with vanishingly small numbers of training patterns. In addition to providing a new set of low-density error-correcting codes that achieve Shannon’s noisy channel bound, these networks also efficiently solve a variant of the hidden clique problem in computer science, opening new avenues for real-world applications of computational models originating from biology. |
format | Online Article Text |
id | pubmed-5770423 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | Springer Berlin Heidelberg |
record_format | MEDLINE/PubMed |
spelling | pubmed-57704232018-01-29 Robust Exponential Memory in Hopfield Networks Hillar, Christopher J. Tran, Ngoc M. J Math Neurosci Research The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically coupled McCulloch–Pitts binary neurons interact to perform emergent computation. Although previous researchers have explored the potential of this network to solve combinatorial optimization problems or store reoccurring activity patterns as attractors of its deterministic dynamics, a basic open problem is to design a family of Hopfield networks with a number of noise-tolerant memories that grows exponentially with neural population size. Here, we discover such networks by minimizing probability flow, a recently proposed objective for estimating parameters in discrete maximum entropy models. By descending the gradient of the convex probability flow, our networks adapt synaptic weights to achieve robust exponential storage, even when presented with vanishingly small numbers of training patterns. In addition to providing a new set of low-density error-correcting codes that achieve Shannon’s noisy channel bound, these networks also efficiently solve a variant of the hidden clique problem in computer science, opening new avenues for real-world applications of computational models originating from biology. Springer Berlin Heidelberg 2018-01-16 /pmc/articles/PMC5770423/ /pubmed/29340803 http://dx.doi.org/10.1186/s13408-017-0056-2 Text en © The Author(s) 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. |
spellingShingle | Research Hillar, Christopher J. Tran, Ngoc M. Robust Exponential Memory in Hopfield Networks |
title | Robust Exponential Memory in Hopfield Networks |
title_full | Robust Exponential Memory in Hopfield Networks |
title_fullStr | Robust Exponential Memory in Hopfield Networks |
title_full_unstemmed | Robust Exponential Memory in Hopfield Networks |
title_short | Robust Exponential Memory in Hopfield Networks |
title_sort | robust exponential memory in hopfield networks |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5770423/ https://www.ncbi.nlm.nih.gov/pubmed/29340803 http://dx.doi.org/10.1186/s13408-017-0056-2 |
work_keys_str_mv | AT hillarchristopherj robustexponentialmemoryinhopfieldnetworks AT tranngocm robustexponentialmemoryinhopfieldnetworks |