Cargando…
Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior
Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explaine...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
eLife Sciences Publications, Ltd
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5860866/ https://www.ncbi.nlm.nih.gov/pubmed/29513219 http://dx.doi.org/10.7554/eLife.32962 |
_version_ | 1783308007069712384 |
---|---|
author | Groen, Iris IA Greene, Michelle R Baldassano, Christopher Fei-Fei, Li Beck, Diane M Baker, Chris I |
author_facet | Groen, Iris IA Greene, Michelle R Baldassano, Christopher Fei-Fei, Li Beck, Diane M Baker, Chris I |
author_sort | Groen, Iris IA |
collection | PubMed |
description | Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information. |
format | Online Article Text |
id | pubmed-5860866 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | eLife Sciences Publications, Ltd |
record_format | MEDLINE/PubMed |
spelling | pubmed-58608662018-03-21 Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior Groen, Iris IA Greene, Michelle R Baldassano, Christopher Fei-Fei, Li Beck, Diane M Baker, Chris I eLife Neuroscience Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information. eLife Sciences Publications, Ltd 2018-03-07 /pmc/articles/PMC5860866/ /pubmed/29513219 http://dx.doi.org/10.7554/eLife.32962 Text en http://creativecommons.org/publicdomain/zero/1.0/ http://creativecommons.org/publicdomain/zero/1.0/This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication (http://creativecommons.org/publicdomain/zero/1.0/) . |
spellingShingle | Neuroscience Groen, Iris IA Greene, Michelle R Baldassano, Christopher Fei-Fei, Li Beck, Diane M Baker, Chris I Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title | Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title_full | Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title_fullStr | Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title_full_unstemmed | Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title_short | Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
title_sort | distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5860866/ https://www.ncbi.nlm.nih.gov/pubmed/29513219 http://dx.doi.org/10.7554/eLife.32962 |
work_keys_str_mv | AT groenirisia distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior AT greenemicheller distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior AT baldassanochristopher distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior AT feifeili distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior AT beckdianem distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior AT bakerchrisi distinctcontributionsoffunctionalanddeepneuralnetworkfeaturestorepresentationalsimilarityofscenesinhumanbrainandbehavior |