Cargando…

Feature Representations for Neuromorphic Audio Spike Streams

Event-driven neuromorphic spiking sensors such as the silicon retina and the silicon cochlea encode the external sensory stimuli as asynchronous streams of spikes across different channels or pixels. Combining state-of-art deep neural networks with the asynchronous outputs of these sensors has produ...

Descripción completa

Detalles Bibliográficos
Autores principales: Anumula, Jithendar, Neil, Daniel, Delbruck, Tobi, Liu, Shih-Chii
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5811520/
https://www.ncbi.nlm.nih.gov/pubmed/29479300
http://dx.doi.org/10.3389/fnins.2018.00023
_version_ 1783299875093348352
author Anumula, Jithendar
Neil, Daniel
Delbruck, Tobi
Liu, Shih-Chii
author_facet Anumula, Jithendar
Neil, Daniel
Delbruck, Tobi
Liu, Shih-Chii
author_sort Anumula, Jithendar
collection PubMed
description Event-driven neuromorphic spiking sensors such as the silicon retina and the silicon cochlea encode the external sensory stimuli as asynchronous streams of spikes across different channels or pixels. Combining state-of-art deep neural networks with the asynchronous outputs of these sensors has produced encouraging results on some datasets but remains challenging. While the lack of effective spiking networks to process the spike streams is one reason, the other reason is that the pre-processing methods required to convert the spike streams to frame-based features needed for the deep networks still require further investigation. This work investigates the effectiveness of synchronous and asynchronous frame-based features generated using spike count and constant event binning in combination with the use of a recurrent neural network for solving a classification task using N-TIDIGITS18 dataset. This spike-based dataset consists of recordings from the Dynamic Audio Sensor, a spiking silicon cochlea sensor, in response to the TIDIGITS audio dataset. We also propose a new pre-processing method which applies an exponential kernel on the output cochlea spikes so that the interspike timing information is better preserved. The results from the N-TIDIGITS18 dataset show that the exponential features perform better than the spike count features, with over 91% accuracy on the digit classification task. This accuracy corresponds to an improvement of at least 2.5% over the use of spike count features, establishing a new state of the art for this dataset.
format Online
Article
Text
id pubmed-5811520
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-58115202018-02-23 Feature Representations for Neuromorphic Audio Spike Streams Anumula, Jithendar Neil, Daniel Delbruck, Tobi Liu, Shih-Chii Front Neurosci Neuroscience Event-driven neuromorphic spiking sensors such as the silicon retina and the silicon cochlea encode the external sensory stimuli as asynchronous streams of spikes across different channels or pixels. Combining state-of-art deep neural networks with the asynchronous outputs of these sensors has produced encouraging results on some datasets but remains challenging. While the lack of effective spiking networks to process the spike streams is one reason, the other reason is that the pre-processing methods required to convert the spike streams to frame-based features needed for the deep networks still require further investigation. This work investigates the effectiveness of synchronous and asynchronous frame-based features generated using spike count and constant event binning in combination with the use of a recurrent neural network for solving a classification task using N-TIDIGITS18 dataset. This spike-based dataset consists of recordings from the Dynamic Audio Sensor, a spiking silicon cochlea sensor, in response to the TIDIGITS audio dataset. We also propose a new pre-processing method which applies an exponential kernel on the output cochlea spikes so that the interspike timing information is better preserved. The results from the N-TIDIGITS18 dataset show that the exponential features perform better than the spike count features, with over 91% accuracy on the digit classification task. This accuracy corresponds to an improvement of at least 2.5% over the use of spike count features, establishing a new state of the art for this dataset. Frontiers Media S.A. 2018-02-09 /pmc/articles/PMC5811520/ /pubmed/29479300 http://dx.doi.org/10.3389/fnins.2018.00023 Text en Copyright © 2018 Anumula, Neil, Delbruck and Liu. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Anumula, Jithendar
Neil, Daniel
Delbruck, Tobi
Liu, Shih-Chii
Feature Representations for Neuromorphic Audio Spike Streams
title Feature Representations for Neuromorphic Audio Spike Streams
title_full Feature Representations for Neuromorphic Audio Spike Streams
title_fullStr Feature Representations for Neuromorphic Audio Spike Streams
title_full_unstemmed Feature Representations for Neuromorphic Audio Spike Streams
title_short Feature Representations for Neuromorphic Audio Spike Streams
title_sort feature representations for neuromorphic audio spike streams
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5811520/
https://www.ncbi.nlm.nih.gov/pubmed/29479300
http://dx.doi.org/10.3389/fnins.2018.00023
work_keys_str_mv AT anumulajithendar featurerepresentationsforneuromorphicaudiospikestreams
AT neildaniel featurerepresentationsforneuromorphicaudiospikestreams
AT delbrucktobi featurerepresentationsforneuromorphicaudiospikestreams
AT liushihchii featurerepresentationsforneuromorphicaudiospikestreams