Cargando…

NeuroGen: Activation optimized image synthesis for discovery neuroscience

Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to no...

Descripción completa

Detalles Bibliográficos
Autores principales: Gu, Zijin, Jamison, Keith Wakefield, Khosla, Meenakshi, Allen, Emily J., Wu, Yihan, St-Yves, Ghislain, Naselaris, Thomas, Kay, Kendrick, Sabuncu, Mert R., Kuceyeski, Amy
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8845078/
https://www.ncbi.nlm.nih.gov/pubmed/34936922
http://dx.doi.org/10.1016/j.neuroimage.2021.118812
_version_ 1784651597292240896
author Gu, Zijin
Jamison, Keith Wakefield
Khosla, Meenakshi
Allen, Emily J.
Wu, Yihan
St-Yves, Ghislain
Naselaris, Thomas
Kay, Kendrick
Sabuncu, Mert R.
Kuceyeski, Amy
author_facet Gu, Zijin
Jamison, Keith Wakefield
Khosla, Meenakshi
Allen, Emily J.
Wu, Yihan
St-Yves, Ghislain
Naselaris, Thomas
Kay, Kendrick
Sabuncu, Mert R.
Kuceyeski, Amy
author_sort Gu, Zijin
collection PubMed
description Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.
format Online
Article
Text
id pubmed-8845078
institution National Center for Biotechnology Information
language English
publishDate 2022
record_format MEDLINE/PubMed
spelling pubmed-88450782022-02-15 NeuroGen: Activation optimized image synthesis for discovery neuroscience Gu, Zijin Jamison, Keith Wakefield Khosla, Meenakshi Allen, Emily J. Wu, Yihan St-Yves, Ghislain Naselaris, Thomas Kay, Kendrick Sabuncu, Mert R. Kuceyeski, Amy Neuroimage Article Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system. 2022-02-15 2021-12-20 /pmc/articles/PMC8845078/ /pubmed/34936922 http://dx.doi.org/10.1016/j.neuroimage.2021.118812 Text en https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/ (https://creativecommons.org/licenses/by-nc-nd/4.0/) )
spellingShingle Article
Gu, Zijin
Jamison, Keith Wakefield
Khosla, Meenakshi
Allen, Emily J.
Wu, Yihan
St-Yves, Ghislain
Naselaris, Thomas
Kay, Kendrick
Sabuncu, Mert R.
Kuceyeski, Amy
NeuroGen: Activation optimized image synthesis for discovery neuroscience
title NeuroGen: Activation optimized image synthesis for discovery neuroscience
title_full NeuroGen: Activation optimized image synthesis for discovery neuroscience
title_fullStr NeuroGen: Activation optimized image synthesis for discovery neuroscience
title_full_unstemmed NeuroGen: Activation optimized image synthesis for discovery neuroscience
title_short NeuroGen: Activation optimized image synthesis for discovery neuroscience
title_sort neurogen: activation optimized image synthesis for discovery neuroscience
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8845078/
https://www.ncbi.nlm.nih.gov/pubmed/34936922
http://dx.doi.org/10.1016/j.neuroimage.2021.118812
work_keys_str_mv AT guzijin neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT jamisonkeithwakefield neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT khoslameenakshi neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT allenemilyj neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT wuyihan neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT styvesghislain neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT naselaristhomas neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT kaykendrick neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT sabuncumertr neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience
AT kuceyeskiamy neurogenactivationoptimizedimagesynthesisfordiscoveryneuroscience