Cargando…

EqSpike: spike-driven equilibrium propagation for neuromorphic implementations

Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge. Equilibrium propagation is a promising alternative to backpropagation as it only involves local computations, but hardw...

Descripción completa

Detalles Bibliográficos
Autores principales: Martin, Erwann, Ernoult, Maxence, Laydevant, Jérémie, Li, Shuai, Querlioz, Damien, Petrisor, Teodora, Grollier, Julie
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Elsevier 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7970361/
https://www.ncbi.nlm.nih.gov/pubmed/33748709
http://dx.doi.org/10.1016/j.isci.2021.102222
_version_ 1783666418019991552
author Martin, Erwann
Ernoult, Maxence
Laydevant, Jérémie
Li, Shuai
Querlioz, Damien
Petrisor, Teodora
Grollier, Julie
author_facet Martin, Erwann
Ernoult, Maxence
Laydevant, Jérémie
Li, Shuai
Querlioz, Damien
Petrisor, Teodora
Grollier, Julie
author_sort Martin, Erwann
collection PubMed
description Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge. Equilibrium propagation is a promising alternative to backpropagation as it only involves local computations, but hardware-oriented studies have so far focused on rate-based networks. In this work, we develop a spiking neural network algorithm called EqSpike, compatible with neuromorphic systems, which learns by equilibrium propagation. Through simulations, we obtain a test recognition accuracy of 97.6% on the MNIST handwritten digits dataset (Mixed National Institute of Standards and Technology), similar to rate-based equilibrium propagation, and comparing favorably to alternative learning techniques for spiking neural networks. We show that EqSpike implemented in silicon neuromorphic technology could reduce the energy consumption of inference and training, respectively, by three orders and two orders of magnitude compared to graphics processing units. Finally, we also show that during learning, EqSpike weight updates exhibit a form of spike-timing-dependent plasticity, highlighting a possible connection with biology.
format Online
Article
Text
id pubmed-7970361
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Elsevier
record_format MEDLINE/PubMed
spelling pubmed-79703612021-03-19 EqSpike: spike-driven equilibrium propagation for neuromorphic implementations Martin, Erwann Ernoult, Maxence Laydevant, Jérémie Li, Shuai Querlioz, Damien Petrisor, Teodora Grollier, Julie iScience Article Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge. Equilibrium propagation is a promising alternative to backpropagation as it only involves local computations, but hardware-oriented studies have so far focused on rate-based networks. In this work, we develop a spiking neural network algorithm called EqSpike, compatible with neuromorphic systems, which learns by equilibrium propagation. Through simulations, we obtain a test recognition accuracy of 97.6% on the MNIST handwritten digits dataset (Mixed National Institute of Standards and Technology), similar to rate-based equilibrium propagation, and comparing favorably to alternative learning techniques for spiking neural networks. We show that EqSpike implemented in silicon neuromorphic technology could reduce the energy consumption of inference and training, respectively, by three orders and two orders of magnitude compared to graphics processing units. Finally, we also show that during learning, EqSpike weight updates exhibit a form of spike-timing-dependent plasticity, highlighting a possible connection with biology. Elsevier 2021-02-20 /pmc/articles/PMC7970361/ /pubmed/33748709 http://dx.doi.org/10.1016/j.isci.2021.102222 Text en © 2021 The Authors http://creativecommons.org/licenses/by/4.0/ This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Martin, Erwann
Ernoult, Maxence
Laydevant, Jérémie
Li, Shuai
Querlioz, Damien
Petrisor, Teodora
Grollier, Julie
EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title_full EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title_fullStr EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title_full_unstemmed EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title_short EqSpike: spike-driven equilibrium propagation for neuromorphic implementations
title_sort eqspike: spike-driven equilibrium propagation for neuromorphic implementations
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7970361/
https://www.ncbi.nlm.nih.gov/pubmed/33748709
http://dx.doi.org/10.1016/j.isci.2021.102222
work_keys_str_mv AT martinerwann eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT ernoultmaxence eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT laydevantjeremie eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT lishuai eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT querliozdamien eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT petrisorteodora eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations
AT grollierjulie eqspikespikedrivenequilibriumpropagationforneuromorphicimplementations