Cargando…
The contribution of object identity and configuration to scene representation in convolutional neural networks
Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over th...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9239439/ https://www.ncbi.nlm.nih.gov/pubmed/35763531 http://dx.doi.org/10.1371/journal.pone.0270667 |
_version_ | 1784737292848463872 |
---|---|
author | Tang, Kevin Chin, Matthew Chun, Marvin Xu, Yaoda |
author_facet | Tang, Kevin Chin, Matthew Chun, Marvin Xu, Yaoda |
author_sort | Tang, Kevin |
collection | PubMed |
description | Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN’s response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs’ sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing. |
format | Online Article Text |
id | pubmed-9239439 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-92394392022-06-29 The contribution of object identity and configuration to scene representation in convolutional neural networks Tang, Kevin Chin, Matthew Chun, Marvin Xu, Yaoda PLoS One Research Article Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN’s response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs’ sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing. Public Library of Science 2022-06-28 /pmc/articles/PMC9239439/ /pubmed/35763531 http://dx.doi.org/10.1371/journal.pone.0270667 Text en © 2022 Tang et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Tang, Kevin Chin, Matthew Chun, Marvin Xu, Yaoda The contribution of object identity and configuration to scene representation in convolutional neural networks |
title | The contribution of object identity and configuration to scene representation in convolutional neural networks |
title_full | The contribution of object identity and configuration to scene representation in convolutional neural networks |
title_fullStr | The contribution of object identity and configuration to scene representation in convolutional neural networks |
title_full_unstemmed | The contribution of object identity and configuration to scene representation in convolutional neural networks |
title_short | The contribution of object identity and configuration to scene representation in convolutional neural networks |
title_sort | contribution of object identity and configuration to scene representation in convolutional neural networks |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9239439/ https://www.ncbi.nlm.nih.gov/pubmed/35763531 http://dx.doi.org/10.1371/journal.pone.0270667 |
work_keys_str_mv | AT tangkevin thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT chinmatthew thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT chunmarvin thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT xuyaoda thecontributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT tangkevin contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT chinmatthew contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT chunmarvin contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks AT xuyaoda contributionofobjectidentityandconfigurationtoscenerepresentationinconvolutionalneuralnetworks |