Cargando…
Unsupervised Feature Learning With Winner-Takes-All Based STDP
We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-tempor...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5895733/ https://www.ncbi.nlm.nih.gov/pubmed/29674961 http://dx.doi.org/10.3389/fncom.2018.00024 |
_version_ | 1783313708938690560 |
---|---|
author | Ferré, Paul Mamalet, Franck Thorpe, Simon J. |
author_facet | Ferré, Paul Mamalet, Franck Thorpe, Simon J. |
author_sort | Ferré, Paul |
collection | PubMed |
description | We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary STDP learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All (WTA) framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10, and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods. |
format | Online Article Text |
id | pubmed-5895733 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-58957332018-04-19 Unsupervised Feature Learning With Winner-Takes-All Based STDP Ferré, Paul Mamalet, Franck Thorpe, Simon J. Front Comput Neurosci Neuroscience We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary STDP learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All (WTA) framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10, and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods. Frontiers Media S.A. 2018-04-05 /pmc/articles/PMC5895733/ /pubmed/29674961 http://dx.doi.org/10.3389/fncom.2018.00024 Text en Copyright © 2018 Ferré, Mamalet and Thorpe. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Ferré, Paul Mamalet, Franck Thorpe, Simon J. Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title | Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title_full | Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title_fullStr | Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title_full_unstemmed | Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title_short | Unsupervised Feature Learning With Winner-Takes-All Based STDP |
title_sort | unsupervised feature learning with winner-takes-all based stdp |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5895733/ https://www.ncbi.nlm.nih.gov/pubmed/29674961 http://dx.doi.org/10.3389/fncom.2018.00024 |
work_keys_str_mv | AT ferrepaul unsupervisedfeaturelearningwithwinnertakesallbasedstdp AT mamaletfranck unsupervisedfeaturelearningwithwinnertakesallbasedstdp AT thorpesimonj unsupervisedfeaturelearningwithwinnertakesallbasedstdp |