Cargando…

Extended liquid state machines for speech recognition

A liquid state machine (LSM) is a biologically plausible model of a cortical microcircuit. It exists of a random, sparse reservoir of recurrently connected spiking neurons with fixed synapses and a trainable readout layer. The LSM exhibits low training complexity and enables backpropagation-free lea...

Descripción completa

Detalles Bibliográficos
Autores principales: Deckers, Lucas, Tsang, Ing Jyh, Van Leekwijck, Werner, Latré, Steven
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9651956/
https://www.ncbi.nlm.nih.gov/pubmed/36389242
http://dx.doi.org/10.3389/fnins.2022.1023470
_version_ 1784828361783115776
author Deckers, Lucas
Tsang, Ing Jyh
Van Leekwijck, Werner
Latré, Steven
author_facet Deckers, Lucas
Tsang, Ing Jyh
Van Leekwijck, Werner
Latré, Steven
author_sort Deckers, Lucas
collection PubMed
description A liquid state machine (LSM) is a biologically plausible model of a cortical microcircuit. It exists of a random, sparse reservoir of recurrently connected spiking neurons with fixed synapses and a trainable readout layer. The LSM exhibits low training complexity and enables backpropagation-free learning in a powerful, yet simple computing paradigm. In this work, the liquid state machine is enhanced by a set of bio-inspired extensions to create the extended liquid state machine (ELSM), which is evaluated on a set of speech data sets. Firstly, we ensure excitatory/inhibitory (E/I) balance to enable the LSM to operate in edge-of-chaos regime. Secondly, spike-frequency adaptation (SFA) is introduced in the LSM to improve the memory capabilities. Lastly, neuronal heterogeneity, by means of a differentiation in time constants, is introduced to extract a richer dynamical LSM response. By including E/I balance, SFA, and neuronal heterogeneity, we show that the ELSM consistently improves upon the LSM while retaining the benefits of the straightforward LSM structure and training procedure. The proposed extensions led up to an 5.2% increase in accuracy while decreasing the number of spikes in the ELSM up to 20.2% on benchmark speech data sets. On some benchmarks, the ELSM can even attain similar performances as the current state-of-the-art in spiking neural networks. Furthermore, we illustrate that the ELSM input-liquid and recurrent synaptic weights can be reduced to 4-bit resolution without any significant loss in classification performance. We thus show that the ELSM is a powerful, biologically plausible and hardware-friendly spiking neural network model that can attain near state-of-the-art accuracy on speech recognition benchmarks for spiking neural networks.
format Online
Article
Text
id pubmed-9651956
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-96519562022-11-15 Extended liquid state machines for speech recognition Deckers, Lucas Tsang, Ing Jyh Van Leekwijck, Werner Latré, Steven Front Neurosci Neuroscience A liquid state machine (LSM) is a biologically plausible model of a cortical microcircuit. It exists of a random, sparse reservoir of recurrently connected spiking neurons with fixed synapses and a trainable readout layer. The LSM exhibits low training complexity and enables backpropagation-free learning in a powerful, yet simple computing paradigm. In this work, the liquid state machine is enhanced by a set of bio-inspired extensions to create the extended liquid state machine (ELSM), which is evaluated on a set of speech data sets. Firstly, we ensure excitatory/inhibitory (E/I) balance to enable the LSM to operate in edge-of-chaos regime. Secondly, spike-frequency adaptation (SFA) is introduced in the LSM to improve the memory capabilities. Lastly, neuronal heterogeneity, by means of a differentiation in time constants, is introduced to extract a richer dynamical LSM response. By including E/I balance, SFA, and neuronal heterogeneity, we show that the ELSM consistently improves upon the LSM while retaining the benefits of the straightforward LSM structure and training procedure. The proposed extensions led up to an 5.2% increase in accuracy while decreasing the number of spikes in the ELSM up to 20.2% on benchmark speech data sets. On some benchmarks, the ELSM can even attain similar performances as the current state-of-the-art in spiking neural networks. Furthermore, we illustrate that the ELSM input-liquid and recurrent synaptic weights can be reduced to 4-bit resolution without any significant loss in classification performance. We thus show that the ELSM is a powerful, biologically plausible and hardware-friendly spiking neural network model that can attain near state-of-the-art accuracy on speech recognition benchmarks for spiking neural networks. Frontiers Media S.A. 2022-10-28 /pmc/articles/PMC9651956/ /pubmed/36389242 http://dx.doi.org/10.3389/fnins.2022.1023470 Text en Copyright © 2022 Deckers, Tsang, Van Leekwijck and Latré. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Deckers, Lucas
Tsang, Ing Jyh
Van Leekwijck, Werner
Latré, Steven
Extended liquid state machines for speech recognition
title Extended liquid state machines for speech recognition
title_full Extended liquid state machines for speech recognition
title_fullStr Extended liquid state machines for speech recognition
title_full_unstemmed Extended liquid state machines for speech recognition
title_short Extended liquid state machines for speech recognition
title_sort extended liquid state machines for speech recognition
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9651956/
https://www.ncbi.nlm.nih.gov/pubmed/36389242
http://dx.doi.org/10.3389/fnins.2022.1023470
work_keys_str_mv AT deckerslucas extendedliquidstatemachinesforspeechrecognition
AT tsangingjyh extendedliquidstatemachinesforspeechrecognition
AT vanleekwijckwerner extendedliquidstatemachinesforspeechrecognition
AT latresteven extendedliquidstatemachinesforspeechrecognition