Cargando…

The contribution of object identity and configuration to scene representation in convolutional neural networks

Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over th...

Descripción completa

Detalles Bibliográficos
Autores principales: Tang, Kevin, Chin, Matthew, Chun, Marvin, Xu, Yaoda
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9239439/
https://www.ncbi.nlm.nih.gov/pubmed/35763531
http://dx.doi.org/10.1371/journal.pone.0270667
_version_ 1784737292848463872
author Tang, Kevin
Chin, Matthew
Chun, Marvin
Xu, Yaoda
author_facet Tang, Kevin
Chin, Matthew
Chun, Marvin
Xu, Yaoda
author_sort Tang, Kevin
collection PubMed
description Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN’s response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs’ sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing.
format Online
Article
Text
id pubmed-9239439
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-92394392022-06-29 The contribution of object identity and configuration to scene representation in convolutional neural networks Tang, Kevin Chin, Matthew Chun, Marvin Xu, Yaoda PLoS One Research Article Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN’s response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs’ sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing. Public Library of Science 2022-06-28 /pmc/articles/PMC9239439/ /pubmed/35763531 http://dx.doi.org/10.1371/journal.pone.0270667 Text en © 2022 Tang et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Tang, Kevin
Chin, Matthew
Chun, Marvin
Xu, Yaoda
The contribution of object identity and configuration to scene representation in convolutional neural networks
title The contribution of object identity and configuration to scene representation in convolutional neural networks
title_full The contribution of object identity and configuration to scene representation in convolutional neural networks
title_fullStr The contribution of object identity and configuration to scene representation in convolutional neural networks
title_full_unstemmed The contribution of object identity and configuration to scene representation in convolutional neural networks
title_short The contribution of object identity and configuration to scene representation in convolutional neural networks
title_sort contribution of object identity and configuration to scene representation in convolutional neural networks
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9239439/
https://www.ncbi.nlm.nih.gov/pubmed/35763531
http://dx.doi.org/10.1371/journal.pone.0270667
work_keys_str_mv AT tangkevin thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT chinmatthew thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT chunmarvin thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT xuyaoda thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT tangkevin contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT chinmatthew contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT chunmarvin contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks
AT xuyaoda contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks