Cargando…
Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks
Spiking neural networks (SNNs) are considered as the third generation of artificial neural networks, having the potential to improve the energy efficiency of conventional computing systems. Although the firing rate of a spiking neuron is an approximation of rectified linear unit (ReLU) activation in...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7044207/ https://www.ncbi.nlm.nih.gov/pubmed/32103126 http://dx.doi.org/10.1038/s41598-020-60572-8 |
_version_ | 1783501518832402432 |
---|---|
author | Hwang, Sungmin Chang, Jeesoo Oh, Min-Hye Lee, Jong-Ho Park, Byung-Gook |
author_facet | Hwang, Sungmin Chang, Jeesoo Oh, Min-Hye Lee, Jong-Ho Park, Byung-Gook |
author_sort | Hwang, Sungmin |
collection | PubMed |
description | Spiking neural networks (SNNs) are considered as the third generation of artificial neural networks, having the potential to improve the energy efficiency of conventional computing systems. Although the firing rate of a spiking neuron is an approximation of rectified linear unit (ReLU) activation in an analog-valued neural network (ANN), there remain many challenges to be overcome owing to differences in operation between ANNs and SNNs. Unlike actual biological and biophysical processes, various hardware implementations of neurons and SNNs do not allow the membrane potential to fall below the resting potential—in other words, neurons must allow the sub-resting membrane potential. Because there occur an excitatory post-synaptic potential (EPSP) as well as an inhibitory post-synaptic potential (IPSP), negatively valued synaptic weights in SNNs induce the sub-resting membrane potential at some time point. If a membrane is not allowed to hold the sub-resting potential, errors will accumulate over time, resulting in inaccurate inference operations. This phenomenon is not observed in ANNs given their use of only spatial synaptic integration, but it can cause serious performance degradation in SNNs. In this paper, we demonstrate the impact of the sub-resting membrane potential on accurate inference operations in SNNs. Moreover, several important considerations for a hardware SNN that can maintain the sub-resting membrane potential are discussed. All of the results in this paper indicate that it is essential for neurons to allow the sub-resting membrane potential in order to realize high-performance SNNs. |
format | Online Article Text |
id | pubmed-7044207 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-70442072020-03-04 Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks Hwang, Sungmin Chang, Jeesoo Oh, Min-Hye Lee, Jong-Ho Park, Byung-Gook Sci Rep Article Spiking neural networks (SNNs) are considered as the third generation of artificial neural networks, having the potential to improve the energy efficiency of conventional computing systems. Although the firing rate of a spiking neuron is an approximation of rectified linear unit (ReLU) activation in an analog-valued neural network (ANN), there remain many challenges to be overcome owing to differences in operation between ANNs and SNNs. Unlike actual biological and biophysical processes, various hardware implementations of neurons and SNNs do not allow the membrane potential to fall below the resting potential—in other words, neurons must allow the sub-resting membrane potential. Because there occur an excitatory post-synaptic potential (EPSP) as well as an inhibitory post-synaptic potential (IPSP), negatively valued synaptic weights in SNNs induce the sub-resting membrane potential at some time point. If a membrane is not allowed to hold the sub-resting potential, errors will accumulate over time, resulting in inaccurate inference operations. This phenomenon is not observed in ANNs given their use of only spatial synaptic integration, but it can cause serious performance degradation in SNNs. In this paper, we demonstrate the impact of the sub-resting membrane potential on accurate inference operations in SNNs. Moreover, several important considerations for a hardware SNN that can maintain the sub-resting membrane potential are discussed. All of the results in this paper indicate that it is essential for neurons to allow the sub-resting membrane potential in order to realize high-performance SNNs. Nature Publishing Group UK 2020-02-26 /pmc/articles/PMC7044207/ /pubmed/32103126 http://dx.doi.org/10.1038/s41598-020-60572-8 Text en © The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. |
spellingShingle | Article Hwang, Sungmin Chang, Jeesoo Oh, Min-Hye Lee, Jong-Ho Park, Byung-Gook Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title | Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title_full | Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title_fullStr | Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title_full_unstemmed | Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title_short | Impact of the Sub-Resting Membrane Potential on Accurate Inference in Spiking Neural Networks |
title_sort | impact of the sub-resting membrane potential on accurate inference in spiking neural networks |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7044207/ https://www.ncbi.nlm.nih.gov/pubmed/32103126 http://dx.doi.org/10.1038/s41598-020-60572-8 |
work_keys_str_mv | AT hwangsungmin impactofthesubrestingmembranepotentialonaccurateinferenceinspikingneuralnetworks AT changjeesoo impactofthesubrestingmembranepotentialonaccurateinferenceinspikingneuralnetworks AT ohminhye impactofthesubrestingmembranepotentialonaccurateinferenceinspikingneuralnetworks AT leejongho impactofthesubrestingmembranepotentialonaccurateinferenceinspikingneuralnetworks AT parkbyunggook impactofthesubrestingmembranepotentialonaccurateinferenceinspikingneuralnetworks |