Cargando…

The brain-inspired decoder for natural visual image reconstruction

The visual system provides a valuable model for studying the working mechanisms of sensory processing and high-level consciousness. A significant challenge in this field is the reconstruction of images from decoded neural activity, which could not only test the accuracy of our understanding of the v...

Descripción completa

Detalles Bibliográficos
Autores principales: Li, Wenyi, Zheng, Shengjie, Liao, Yufan, Hong, Rongqi, He, Chenggang, Chen, Weiliang, Deng, Chunshan, Li, Xiaojian
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10185745/
https://www.ncbi.nlm.nih.gov/pubmed/37205046
http://dx.doi.org/10.3389/fnins.2023.1130606
_version_ 1785042421647671296
author Li, Wenyi
Zheng, Shengjie
Liao, Yufan
Hong, Rongqi
He, Chenggang
Chen, Weiliang
Deng, Chunshan
Li, Xiaojian
author_facet Li, Wenyi
Zheng, Shengjie
Liao, Yufan
Hong, Rongqi
He, Chenggang
Chen, Weiliang
Deng, Chunshan
Li, Xiaojian
author_sort Li, Wenyi
collection PubMed
description The visual system provides a valuable model for studying the working mechanisms of sensory processing and high-level consciousness. A significant challenge in this field is the reconstruction of images from decoded neural activity, which could not only test the accuracy of our understanding of the visual system but also provide a practical tool for solving real-world problems. Although recent advances in deep learning have improved the decoding of neural spike trains, little attention has been paid to the underlying mechanisms of the visual system. To address this issue, we propose a deep learning neural network architecture that incorporates the biological properties of the visual system, such as receptive fields, to reconstruct visual images from spike trains. Our model outperforms current models and has been evaluated on different datasets from both retinal ganglion cells (RGCs) and the primary visual cortex (V1) neural spikes. Our model demonstrated the great potential of brain-inspired algorithms to solve a challenge that our brain solves.
format Online
Article
Text
id pubmed-10185745
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-101857452023-05-17 The brain-inspired decoder for natural visual image reconstruction Li, Wenyi Zheng, Shengjie Liao, Yufan Hong, Rongqi He, Chenggang Chen, Weiliang Deng, Chunshan Li, Xiaojian Front Neurosci Neuroscience The visual system provides a valuable model for studying the working mechanisms of sensory processing and high-level consciousness. A significant challenge in this field is the reconstruction of images from decoded neural activity, which could not only test the accuracy of our understanding of the visual system but also provide a practical tool for solving real-world problems. Although recent advances in deep learning have improved the decoding of neural spike trains, little attention has been paid to the underlying mechanisms of the visual system. To address this issue, we propose a deep learning neural network architecture that incorporates the biological properties of the visual system, such as receptive fields, to reconstruct visual images from spike trains. Our model outperforms current models and has been evaluated on different datasets from both retinal ganglion cells (RGCs) and the primary visual cortex (V1) neural spikes. Our model demonstrated the great potential of brain-inspired algorithms to solve a challenge that our brain solves. Frontiers Media S.A. 2023-05-02 /pmc/articles/PMC10185745/ /pubmed/37205046 http://dx.doi.org/10.3389/fnins.2023.1130606 Text en Copyright © 2023 Li, Zheng, Liao, Hong, He, Chen, Deng and Li. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Li, Wenyi
Zheng, Shengjie
Liao, Yufan
Hong, Rongqi
He, Chenggang
Chen, Weiliang
Deng, Chunshan
Li, Xiaojian
The brain-inspired decoder for natural visual image reconstruction
title The brain-inspired decoder for natural visual image reconstruction
title_full The brain-inspired decoder for natural visual image reconstruction
title_fullStr The brain-inspired decoder for natural visual image reconstruction
title_full_unstemmed The brain-inspired decoder for natural visual image reconstruction
title_short The brain-inspired decoder for natural visual image reconstruction
title_sort brain-inspired decoder for natural visual image reconstruction
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10185745/
https://www.ncbi.nlm.nih.gov/pubmed/37205046
http://dx.doi.org/10.3389/fnins.2023.1130606
work_keys_str_mv AT liwenyi thebraininspireddecoderfornaturalvisualimagereconstruction
AT zhengshengjie thebraininspireddecoderfornaturalvisualimagereconstruction
AT liaoyufan thebraininspireddecoderfornaturalvisualimagereconstruction
AT hongrongqi thebraininspireddecoderfornaturalvisualimagereconstruction
AT hechenggang thebraininspireddecoderfornaturalvisualimagereconstruction
AT chenweiliang thebraininspireddecoderfornaturalvisualimagereconstruction
AT dengchunshan thebraininspireddecoderfornaturalvisualimagereconstruction
AT lixiaojian thebraininspireddecoderfornaturalvisualimagereconstruction
AT liwenyi braininspireddecoderfornaturalvisualimagereconstruction
AT zhengshengjie braininspireddecoderfornaturalvisualimagereconstruction
AT liaoyufan braininspireddecoderfornaturalvisualimagereconstruction
AT hongrongqi braininspireddecoderfornaturalvisualimagereconstruction
AT hechenggang braininspireddecoderfornaturalvisualimagereconstruction
AT chenweiliang braininspireddecoderfornaturalvisualimagereconstruction
AT dengchunshan braininspireddecoderfornaturalvisualimagereconstruction
AT lixiaojian braininspireddecoderfornaturalvisualimagereconstruction