Cargando…

Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior

Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explaine...

Descripción completa

Detalles Bibliográficos
Autores principales: Groen, Iris IA, Greene, Michelle R, Baldassano, Christopher, Fei-Fei, Li, Beck, Diane M, Baker, Chris I
Formato: Online Artículo Texto
Lenguaje:English
Publicado: eLife Sciences Publications, Ltd 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5860866/
https://www.ncbi.nlm.nih.gov/pubmed/29513219
http://dx.doi.org/10.7554/eLife.32962
_version_ 1783308007069712384
author Groen, Iris IA
Greene, Michelle R
Baldassano, Christopher
Fei-Fei, Li
Beck, Diane M
Baker, Chris I
author_facet Groen, Iris IA
Greene, Michelle R
Baldassano, Christopher
Fei-Fei, Li
Beck, Diane M
Baker, Chris I
author_sort Groen, Iris IA
collection PubMed
description Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.
format Online
Article
Text
id pubmed-5860866
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher eLife Sciences Publications, Ltd
record_format MEDLINE/PubMed
spelling pubmed-58608662018-03-21 Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior Groen, Iris IA Greene, Michelle R Baldassano, Christopher Fei-Fei, Li Beck, Diane M Baker, Chris I eLife Neuroscience Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information. eLife Sciences Publications, Ltd 2018-03-07 /pmc/articles/PMC5860866/ /pubmed/29513219 http://dx.doi.org/10.7554/eLife.32962 Text en http://creativecommons.org/publicdomain/zero/1.0/ http://creativecommons.org/publicdomain/zero/1.0/This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication (http://creativecommons.org/publicdomain/zero/1.0/) .
spellingShingle Neuroscience
Groen, Iris IA
Greene, Michelle R
Baldassano, Christopher
Fei-Fei, Li
Beck, Diane M
Baker, Chris I
Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title_full Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title_fullStr Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title_full_unstemmed Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title_short Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
title_sort distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5860866/
https://www.ncbi.nlm.nih.gov/pubmed/29513219
http://dx.doi.org/10.7554/eLife.32962
work_keys_str_mv AT groenirisia distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior
AT greenemicheller distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior
AT baldassanochristopher distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior
AT feifeili distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior
AT beckdianem distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior
AT bakerchrisi distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior