Cargando…
Multiplicative mixing of object identity and image attributes in single inferior temporal neurons
Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas,...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
National Academy of Sciences
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5889630/ https://www.ncbi.nlm.nih.gov/pubmed/29559530 http://dx.doi.org/10.1073/pnas.1714287115 |
_version_ | 1783312737110065152 |
---|---|
author | Ratan Murty, N. Apurva Arun, S. P. |
author_facet | Ratan Murty, N. Apurva Arun, S. P. |
author_sort | Ratan Murty, N. Apurva |
collection | PubMed |
description | Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas, but precisely how they are combined has remained unclear. One possibility is that neurons might encode identity and attribute signals multiplicatively so that each can be efficiently decoded without interference from the other. Here, we show that, in high-level visual cortex, responses of single neurons can be explained better as a product rather than a sum of tuning for object identity and tuning for image attributes. This subtle effect in single neurons produced substantially better population decoding of object identity and image attributes in the neural population as a whole. This property was absent both in low-level vision models and in deep neural networks. It was also unique to invariances: when tested with two-part objects, neural responses were explained better as a sum than as a product of part tuning. Taken together, our results indicate that signals requiring separate decoding, such as object identity and image attributes, are combined multiplicatively in IT neurons, whereas signals that require integration (such as parts in an object) are combined additively. |
format | Online Article Text |
id | pubmed-5889630 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | National Academy of Sciences |
record_format | MEDLINE/PubMed |
spelling | pubmed-58896302018-04-09 Multiplicative mixing of object identity and image attributes in single inferior temporal neurons Ratan Murty, N. Apurva Arun, S. P. Proc Natl Acad Sci U S A PNAS Plus Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas, but precisely how they are combined has remained unclear. One possibility is that neurons might encode identity and attribute signals multiplicatively so that each can be efficiently decoded without interference from the other. Here, we show that, in high-level visual cortex, responses of single neurons can be explained better as a product rather than a sum of tuning for object identity and tuning for image attributes. This subtle effect in single neurons produced substantially better population decoding of object identity and image attributes in the neural population as a whole. This property was absent both in low-level vision models and in deep neural networks. It was also unique to invariances: when tested with two-part objects, neural responses were explained better as a sum than as a product of part tuning. Taken together, our results indicate that signals requiring separate decoding, such as object identity and image attributes, are combined multiplicatively in IT neurons, whereas signals that require integration (such as parts in an object) are combined additively. National Academy of Sciences 2018-04-03 2018-03-20 /pmc/articles/PMC5889630/ /pubmed/29559530 http://dx.doi.org/10.1073/pnas.1714287115 Text en Copyright © 2018 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by-nc-nd/4.0/ This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) . |
spellingShingle | PNAS Plus Ratan Murty, N. Apurva Arun, S. P. Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title | Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title_full | Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title_fullStr | Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title_full_unstemmed | Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title_short | Multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
title_sort | multiplicative mixing of object identity and image attributes in single inferior temporal neurons |
topic | PNAS Plus |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5889630/ https://www.ncbi.nlm.nih.gov/pubmed/29559530 http://dx.doi.org/10.1073/pnas.1714287115 |
work_keys_str_mv | AT ratanmurtynapurva multiplicativemixingofobjectidentityandimageattributesinsingleinferiortemporalneurons AT arunsp multiplicativemixingofobjectidentityandimageattributesinsingleinferiortemporalneurons |