Cargando…

Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery

Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representa...

Descripción completa

Detalles Bibliográficos
Autores principales: Lörincz, András, Palotai, Zsolt, Szirtes, Gábor
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2012
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3291527/
https://www.ncbi.nlm.nih.gov/pubmed/22396629
http://dx.doi.org/10.1371/journal.pcbi.1002372
_version_ 1782225145035751424
author Lörincz, András
Palotai, Zsolt
Szirtes, Gábor
author_facet Lörincz, András
Palotai, Zsolt
Szirtes, Gábor
author_sort Lörincz, András
collection PubMed
description Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representations in an efficient way. We argue that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and we also show that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Typical parts represent large-scale correlations, thus they can be significantly compressed. Atypical parts, on the other hand, represent local features and are the subjects of actual sparse coding. When applied on natural images, our decomposition based sparse coding model can efficiently form overcomplete codes and both center-surround and oriented filters are obtained similar to those observed in the retina and the primary visual cortex, respectively. Therefore we hypothesize that the proposed computational architecture can be seen as a coherent functional model of the first stages of sensory coding in early vision.
format Online
Article
Text
id pubmed-3291527
institution National Center for Biotechnology Information
language English
publishDate 2012
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-32915272012-03-06 Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery Lörincz, András Palotai, Zsolt Szirtes, Gábor PLoS Comput Biol Research Article Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representations in an efficient way. We argue that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and we also show that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Typical parts represent large-scale correlations, thus they can be significantly compressed. Atypical parts, on the other hand, represent local features and are the subjects of actual sparse coding. When applied on natural images, our decomposition based sparse coding model can efficiently form overcomplete codes and both center-surround and oriented filters are obtained similar to those observed in the retina and the primary visual cortex, respectively. Therefore we hypothesize that the proposed computational architecture can be seen as a coherent functional model of the first stages of sensory coding in early vision. Public Library of Science 2012-03-01 /pmc/articles/PMC3291527/ /pubmed/22396629 http://dx.doi.org/10.1371/journal.pcbi.1002372 Text en Lörincz et al. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
spellingShingle Research Article
Lörincz, András
Palotai, Zsolt
Szirtes, Gábor
Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title_full Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title_fullStr Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title_full_unstemmed Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title_short Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery
title_sort efficient sparse coding in early sensory processing: lessons from signal recovery
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3291527/
https://www.ncbi.nlm.nih.gov/pubmed/22396629
http://dx.doi.org/10.1371/journal.pcbi.1002372
work_keys_str_mv AT lorinczandras efficientsparsecodinginearlysensoryprocessinglessonsfromsignalrecovery
AT palotaizsolt efficientsparsecodinginearlysensoryprocessinglessonsfromsignalrecovery
AT szirtesgabor efficientsparsecodinginearlysensoryprocessinglessonsfromsignalrecovery