Cargando…
Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method
Recent years witness an increasing demand for using spiking neural networks (SNNs) to implement artificial intelligent systems. There is a demand of combining SNNs with reinforcement learning architectures to find an effective training method. Recently, temporal coding method has been proposed to tr...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9428400/ https://www.ncbi.nlm.nih.gov/pubmed/36061595 http://dx.doi.org/10.3389/fnins.2022.877701 |
_version_ | 1784779109035933696 |
---|---|
author | Wu, Guanlin Liang, Dongchen Luan, Shaotong Wang, Ji |
author_facet | Wu, Guanlin Liang, Dongchen Luan, Shaotong Wang, Ji |
author_sort | Wu, Guanlin |
collection | PubMed |
description | Recent years witness an increasing demand for using spiking neural networks (SNNs) to implement artificial intelligent systems. There is a demand of combining SNNs with reinforcement learning architectures to find an effective training method. Recently, temporal coding method has been proposed to train spiking neural networks while preserving the asynchronous nature of spiking neurons to preserve the asynchronous nature of SNNs. We propose a training method that enables temporal coding method in RL tasks. To tackle the problem of high sparsity of spikes, we introduce a self-incremental variable to push each spiking neuron to fire, which makes SNNs fully differentiable. In addition, an encoding method is proposed to solve the problem of information loss of temporal-coded inputs. The experimental results show that the SNNs trained by our proposed method can achieve comparable performance of the state-of-the-art artificial neural networks in benchmark tasks of reinforcement learning. |
format | Online Article Text |
id | pubmed-9428400 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-94284002022-09-01 Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method Wu, Guanlin Liang, Dongchen Luan, Shaotong Wang, Ji Front Neurosci Neuroscience Recent years witness an increasing demand for using spiking neural networks (SNNs) to implement artificial intelligent systems. There is a demand of combining SNNs with reinforcement learning architectures to find an effective training method. Recently, temporal coding method has been proposed to train spiking neural networks while preserving the asynchronous nature of spiking neurons to preserve the asynchronous nature of SNNs. We propose a training method that enables temporal coding method in RL tasks. To tackle the problem of high sparsity of spikes, we introduce a self-incremental variable to push each spiking neuron to fire, which makes SNNs fully differentiable. In addition, an encoding method is proposed to solve the problem of information loss of temporal-coded inputs. The experimental results show that the SNNs trained by our proposed method can achieve comparable performance of the state-of-the-art artificial neural networks in benchmark tasks of reinforcement learning. Frontiers Media S.A. 2022-08-17 /pmc/articles/PMC9428400/ /pubmed/36061595 http://dx.doi.org/10.3389/fnins.2022.877701 Text en Copyright © 2022 Wu, Liang, Luan and Wang. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Wu, Guanlin Liang, Dongchen Luan, Shaotong Wang, Ji Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title | Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title_full | Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title_fullStr | Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title_full_unstemmed | Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title_short | Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method |
title_sort | training spiking neural networks for reinforcement learning tasks with temporal coding method |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9428400/ https://www.ncbi.nlm.nih.gov/pubmed/36061595 http://dx.doi.org/10.3389/fnins.2022.877701 |
work_keys_str_mv | AT wuguanlin trainingspikingneuralnetworksforreinforcementlearningtaskswithtemporalcodingmethod AT liangdongchen trainingspikingneuralnetworksforreinforcementlearningtaskswithtemporalcodingmethod AT luanshaotong trainingspikingneuralnetworksforreinforcementlearningtaskswithtemporalcodingmethod AT wangji trainingspikingneuralnetworksforreinforcementlearningtaskswithtemporalcodingmethod |