Cargando…
Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance
We introduce a novel, biologically plausible local learning rule that provably increases the robustness of neural dynamics to noise in nonlinear recurrent neural networks with homogeneous nonlinearities. Our learning rule achieves higher noise robustness without sacrificing performance on the task a...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9522011/ https://www.ncbi.nlm.nih.gov/pubmed/36121844 http://dx.doi.org/10.1371/journal.pcbi.1010418 |
_version_ | 1784799970611691520 |
---|---|
author | Stock, Christopher H. Harvey, Sarah E. Ocko, Samuel A. Ganguli, Surya |
author_facet | Stock, Christopher H. Harvey, Sarah E. Ocko, Samuel A. Ganguli, Surya |
author_sort | Stock, Christopher H. |
collection | PubMed |
description | We introduce a novel, biologically plausible local learning rule that provably increases the robustness of neural dynamics to noise in nonlinear recurrent neural networks with homogeneous nonlinearities. Our learning rule achieves higher noise robustness without sacrificing performance on the task and without requiring any knowledge of the particular task. The plasticity dynamics—an integrable dynamical system operating on the weights of the network—maintains a multiplicity of conserved quantities, most notably the network’s entire temporal map of input to output trajectories. The outcome of our learning rule is a synaptic balancing between the incoming and outgoing synapses of every neuron. This synaptic balancing rule is consistent with many known aspects of experimentally observed heterosynaptic plasticity, and moreover makes new experimentally testable predictions relating plasticity at the incoming and outgoing synapses of individual neurons. Overall, this work provides a novel, practical local learning rule that exactly preserves overall network function and, in doing so, provides new conceptual bridges between the disparate worlds of the neurobiology of heterosynaptic plasticity, the engineering of regularized noise-robust networks, and the mathematics of integrable Lax dynamical systems. |
format | Online Article Text |
id | pubmed-9522011 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-95220112022-09-30 Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance Stock, Christopher H. Harvey, Sarah E. Ocko, Samuel A. Ganguli, Surya PLoS Comput Biol Research Article We introduce a novel, biologically plausible local learning rule that provably increases the robustness of neural dynamics to noise in nonlinear recurrent neural networks with homogeneous nonlinearities. Our learning rule achieves higher noise robustness without sacrificing performance on the task and without requiring any knowledge of the particular task. The plasticity dynamics—an integrable dynamical system operating on the weights of the network—maintains a multiplicity of conserved quantities, most notably the network’s entire temporal map of input to output trajectories. The outcome of our learning rule is a synaptic balancing between the incoming and outgoing synapses of every neuron. This synaptic balancing rule is consistent with many known aspects of experimentally observed heterosynaptic plasticity, and moreover makes new experimentally testable predictions relating plasticity at the incoming and outgoing synapses of individual neurons. Overall, this work provides a novel, practical local learning rule that exactly preserves overall network function and, in doing so, provides new conceptual bridges between the disparate worlds of the neurobiology of heterosynaptic plasticity, the engineering of regularized noise-robust networks, and the mathematics of integrable Lax dynamical systems. Public Library of Science 2022-09-19 /pmc/articles/PMC9522011/ /pubmed/36121844 http://dx.doi.org/10.1371/journal.pcbi.1010418 Text en © 2022 Stock et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Stock, Christopher H. Harvey, Sarah E. Ocko, Samuel A. Ganguli, Surya Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title | Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title_full | Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title_fullStr | Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title_full_unstemmed | Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title_short | Synaptic balancing: A biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
title_sort | synaptic balancing: a biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9522011/ https://www.ncbi.nlm.nih.gov/pubmed/36121844 http://dx.doi.org/10.1371/journal.pcbi.1010418 |
work_keys_str_mv | AT stockchristopherh synapticbalancingabiologicallyplausiblelocallearningrulethatprovablyincreasesneuralnetworknoiserobustnesswithoutsacrificingtaskperformance AT harveysarahe synapticbalancingabiologicallyplausiblelocallearningrulethatprovablyincreasesneuralnetworknoiserobustnesswithoutsacrificingtaskperformance AT ockosamuela synapticbalancingabiologicallyplausiblelocallearningrulethatprovablyincreasesneuralnetworknoiserobustnesswithoutsacrificingtaskperformance AT gangulisurya synapticbalancingabiologicallyplausiblelocallearningrulethatprovablyincreasesneuralnetworknoiserobustnesswithoutsacrificingtaskperformance |