Cargando…
Reassessing hierarchical correspondences between brain and deep networks through direct interface
Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared,...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
American Association for the Advancement of Science
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9278854/ https://www.ncbi.nlm.nih.gov/pubmed/35857493 http://dx.doi.org/10.1126/sciadv.abm2219 |
_version_ | 1784746273755103232 |
---|---|
author | Sexton, Nicholas J. Love, Bradley C. |
author_facet | Sexton, Nicholas J. Love, Bradley C. |
author_sort | Sexton, Nicholas J. |
collection | PubMed |
description | Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared, not task-relevant, variance. We propose a stricter account of correspondence: If a DCNN layer corresponds to a brain region, then replacing model activity with brain activity should successfully drive the DCNN’s object recognition decision. Using this approach on three datasets, we found that all regions along the ventral visual stream best corresponded with later model layers, indicating that all stages of processing contained higher-level information about object category. Time course analyses suggest that long-range recurrent connections transmit object class information from late to early visual areas. |
format | Online Article Text |
id | pubmed-9278854 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | American Association for the Advancement of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-92788542022-07-29 Reassessing hierarchical correspondences between brain and deep networks through direct interface Sexton, Nicholas J. Love, Bradley C. Sci Adv Neuroscience Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared, not task-relevant, variance. We propose a stricter account of correspondence: If a DCNN layer corresponds to a brain region, then replacing model activity with brain activity should successfully drive the DCNN’s object recognition decision. Using this approach on three datasets, we found that all regions along the ventral visual stream best corresponded with later model layers, indicating that all stages of processing contained higher-level information about object category. Time course analyses suggest that long-range recurrent connections transmit object class information from late to early visual areas. American Association for the Advancement of Science 2022-07-13 /pmc/articles/PMC9278854/ /pubmed/35857493 http://dx.doi.org/10.1126/sciadv.abm2219 Text en Copyright © 2022 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution License 4.0 (CC BY). https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Neuroscience Sexton, Nicholas J. Love, Bradley C. Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title | Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title_full | Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title_fullStr | Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title_full_unstemmed | Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title_short | Reassessing hierarchical correspondences between brain and deep networks through direct interface |
title_sort | reassessing hierarchical correspondences between brain and deep networks through direct interface |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9278854/ https://www.ncbi.nlm.nih.gov/pubmed/35857493 http://dx.doi.org/10.1126/sciadv.abm2219 |
work_keys_str_mv | AT sextonnicholasj reassessinghierarchicalcorrespondencesbetweenbrainanddeepnetworksthroughdirectinterface AT lovebradleyc reassessinghierarchicalcorrespondencesbetweenbrainanddeepnetworksthroughdirectinterface |