Cargando…
Quantization Framework for Fast Spiking Neural Networks
Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) offer additional temporal dynamics with the compromise of lower information transmission rates through the use of spikes. When using an ANN-to-SNN conversion technique there is a direct link between the activation bit pr...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9344889/ https://www.ncbi.nlm.nih.gov/pubmed/35928011 http://dx.doi.org/10.3389/fnins.2022.918793 |
_version_ | 1784761313734426624 |
---|---|
author | Li, Chen Ma, Lei Furber, Steve |
author_facet | Li, Chen Ma, Lei Furber, Steve |
author_sort | Li, Chen |
collection | PubMed |
description | Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) offer additional temporal dynamics with the compromise of lower information transmission rates through the use of spikes. When using an ANN-to-SNN conversion technique there is a direct link between the activation bit precision of the artificial neurons and the time required by the spiking neurons to represent the same bit precision. This implicit link suggests that techniques used to reduce the activation bit precision of ANNs, such as quantization, can help shorten the inference latency of SNNs. However, carrying ANN quantization knowledge over to SNNs is not straightforward, as there are many fundamental differences between them. Here we propose a quantization framework for fast SNNs (QFFS) to overcome these difficulties, providing a method to build SNNs with enhanced latency and reduced loss of accuracy relative to the baseline ANN model. In this framework, we promote the compatibility of ANN information quantization techniques with SNNs, and suppress “occasional noise” to minimize accuracy loss. The resulting SNNs overcome the accuracy degeneration observed previously in SNNs with a limited number of time steps and achieve an accuracy of 70.18% on ImageNet within 8 time steps. This is the first demonstration that SNNs built by ANN-to-SNN conversion can achieve a similar latency to SNNs built by direct training. |
format | Online Article Text |
id | pubmed-9344889 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-93448892022-08-03 Quantization Framework for Fast Spiking Neural Networks Li, Chen Ma, Lei Furber, Steve Front Neurosci Neuroscience Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) offer additional temporal dynamics with the compromise of lower information transmission rates through the use of spikes. When using an ANN-to-SNN conversion technique there is a direct link between the activation bit precision of the artificial neurons and the time required by the spiking neurons to represent the same bit precision. This implicit link suggests that techniques used to reduce the activation bit precision of ANNs, such as quantization, can help shorten the inference latency of SNNs. However, carrying ANN quantization knowledge over to SNNs is not straightforward, as there are many fundamental differences between them. Here we propose a quantization framework for fast SNNs (QFFS) to overcome these difficulties, providing a method to build SNNs with enhanced latency and reduced loss of accuracy relative to the baseline ANN model. In this framework, we promote the compatibility of ANN information quantization techniques with SNNs, and suppress “occasional noise” to minimize accuracy loss. The resulting SNNs overcome the accuracy degeneration observed previously in SNNs with a limited number of time steps and achieve an accuracy of 70.18% on ImageNet within 8 time steps. This is the first demonstration that SNNs built by ANN-to-SNN conversion can achieve a similar latency to SNNs built by direct training. Frontiers Media S.A. 2022-07-19 /pmc/articles/PMC9344889/ /pubmed/35928011 http://dx.doi.org/10.3389/fnins.2022.918793 Text en Copyright © 2022 Li, Ma and Furber. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Li, Chen Ma, Lei Furber, Steve Quantization Framework for Fast Spiking Neural Networks |
title | Quantization Framework for Fast Spiking Neural Networks |
title_full | Quantization Framework for Fast Spiking Neural Networks |
title_fullStr | Quantization Framework for Fast Spiking Neural Networks |
title_full_unstemmed | Quantization Framework for Fast Spiking Neural Networks |
title_short | Quantization Framework for Fast Spiking Neural Networks |
title_sort | quantization framework for fast spiking neural networks |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9344889/ https://www.ncbi.nlm.nih.gov/pubmed/35928011 http://dx.doi.org/10.3389/fnins.2022.918793 |
work_keys_str_mv | AT lichen quantizationframeworkforfastspikingneuralnetworks AT malei quantizationframeworkforfastspikingneuralnetworks AT furbersteve quantizationframeworkforfastspikingneuralnetworks |