Cargando…

GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability

Computational visual encoding models play a key role in understanding the stimulus–response characteristics of neuronal populations in the brain visual cortex. However, building such models typically faces challenges in the effective construction of non-linear feature spaces to fit the neuronal resp...

Descripción completa

Detalles Bibliográficos
Autores principales: Cui, Yibo, Qiao, Kai, Zhang, Chi, Wang, Linyuan, Yan, Bin, Tong, Li
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7893978/
https://www.ncbi.nlm.nih.gov/pubmed/33613179
http://dx.doi.org/10.3389/fnins.2021.614182
_version_ 1783653156740136960
author Cui, Yibo
Qiao, Kai
Zhang, Chi
Wang, Linyuan
Yan, Bin
Tong, Li
author_facet Cui, Yibo
Qiao, Kai
Zhang, Chi
Wang, Linyuan
Yan, Bin
Tong, Li
author_sort Cui, Yibo
collection PubMed
description Computational visual encoding models play a key role in understanding the stimulus–response characteristics of neuronal populations in the brain visual cortex. However, building such models typically faces challenges in the effective construction of non-linear feature spaces to fit the neuronal responses. In this work, we propose the GaborNet visual encoding (GaborNet-VE) model, a novel end-to-end encoding model for the visual ventral stream. This model comprises a Gabor convolutional layer, two regular convolutional layers, and a fully connected layer. The key design principle for the GaborNet-VE model is to replace regular convolutional kernels in the first convolutional layer with Gabor kernels with learnable parameters. One GaborNet-VE model efficiently and simultaneously encodes all voxels in one region of interest of functional magnetic resonance imaging data. The experimental results show that the proposed model achieves state-of-the-art prediction performance for the primary visual cortex. Moreover, the visualizations demonstrate the regularity of the region of interest fitting to the visual features and the estimated receptive fields. These results suggest that the lightweight region-based GaborNet-VE model based on combining handcrafted and deep learning features exhibits good expressiveness and biological interpretability.
format Online
Article
Text
id pubmed-7893978
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-78939782021-02-20 GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability Cui, Yibo Qiao, Kai Zhang, Chi Wang, Linyuan Yan, Bin Tong, Li Front Neurosci Neuroscience Computational visual encoding models play a key role in understanding the stimulus–response characteristics of neuronal populations in the brain visual cortex. However, building such models typically faces challenges in the effective construction of non-linear feature spaces to fit the neuronal responses. In this work, we propose the GaborNet visual encoding (GaborNet-VE) model, a novel end-to-end encoding model for the visual ventral stream. This model comprises a Gabor convolutional layer, two regular convolutional layers, and a fully connected layer. The key design principle for the GaborNet-VE model is to replace regular convolutional kernels in the first convolutional layer with Gabor kernels with learnable parameters. One GaborNet-VE model efficiently and simultaneously encodes all voxels in one region of interest of functional magnetic resonance imaging data. The experimental results show that the proposed model achieves state-of-the-art prediction performance for the primary visual cortex. Moreover, the visualizations demonstrate the regularity of the region of interest fitting to the visual features and the estimated receptive fields. These results suggest that the lightweight region-based GaborNet-VE model based on combining handcrafted and deep learning features exhibits good expressiveness and biological interpretability. Frontiers Media S.A. 2021-02-04 /pmc/articles/PMC7893978/ /pubmed/33613179 http://dx.doi.org/10.3389/fnins.2021.614182 Text en Copyright © 2021 Cui, Qiao, Zhang, Wang, Yan and Tong. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Cui, Yibo
Qiao, Kai
Zhang, Chi
Wang, Linyuan
Yan, Bin
Tong, Li
GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title_full GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title_fullStr GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title_full_unstemmed GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title_short GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability
title_sort gabornet visual encoding: a lightweight region-based visual encoding model with good expressiveness and biological interpretability
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7893978/
https://www.ncbi.nlm.nih.gov/pubmed/33613179
http://dx.doi.org/10.3389/fnins.2021.614182
work_keys_str_mv AT cuiyibo gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability
AT qiaokai gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability
AT zhangchi gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability
AT wanglinyuan gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability
AT yanbin gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability
AT tongli gabornetvisualencodingalightweightregionbasedvisualencodingmodelwithgoodexpressivenessandbiologicalinterpretability