Cargando…

Reassessing hierarchical correspondences between brain and deep networks through direct interface

Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared,...

Descripción completa

Detalles Bibliográficos
Autores principales: Sexton, Nicholas J., Love, Bradley C.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: American Association for the Advancement of Science 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9278854/
https://www.ncbi.nlm.nih.gov/pubmed/35857493
http://dx.doi.org/10.1126/sciadv.abm2219
_version_ 1784746273755103232
author Sexton, Nicholas J.
Love, Bradley C.
author_facet Sexton, Nicholas J.
Love, Bradley C.
author_sort Sexton, Nicholas J.
collection PubMed
description Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared, not task-relevant, variance. We propose a stricter account of correspondence: If a DCNN layer corresponds to a brain region, then replacing model activity with brain activity should successfully drive the DCNN’s object recognition decision. Using this approach on three datasets, we found that all regions along the ventral visual stream best corresponded with later model layers, indicating that all stages of processing contained higher-level information about object category. Time course analyses suggest that long-range recurrent connections transmit object class information from late to early visual areas.
format Online
Article
Text
id pubmed-9278854
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher American Association for the Advancement of Science
record_format MEDLINE/PubMed
spelling pubmed-92788542022-07-29 Reassessing hierarchical correspondences between brain and deep networks through direct interface Sexton, Nicholas J. Love, Bradley C. Sci Adv Neuroscience Functional correspondences between deep convolutional neural networks (DCNNs) and the mammalian visual system support a hierarchical account in which successive stages of processing contain ever higher-level information. However, these correspondences between brain and model activity involve shared, not task-relevant, variance. We propose a stricter account of correspondence: If a DCNN layer corresponds to a brain region, then replacing model activity with brain activity should successfully drive the DCNN’s object recognition decision. Using this approach on three datasets, we found that all regions along the ventral visual stream best corresponded with later model layers, indicating that all stages of processing contained higher-level information about object category. Time course analyses suggest that long-range recurrent connections transmit object class information from late to early visual areas. American Association for the Advancement of Science 2022-07-13 /pmc/articles/PMC9278854/ /pubmed/35857493 http://dx.doi.org/10.1126/sciadv.abm2219 Text en Copyright © 2022 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution License 4.0 (CC BY). https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Neuroscience
Sexton, Nicholas J.
Love, Bradley C.
Reassessing hierarchical correspondences between brain and deep networks through direct interface
title Reassessing hierarchical correspondences between brain and deep networks through direct interface
title_full Reassessing hierarchical correspondences between brain and deep networks through direct interface
title_fullStr Reassessing hierarchical correspondences between brain and deep networks through direct interface
title_full_unstemmed Reassessing hierarchical correspondences between brain and deep networks through direct interface
title_short Reassessing hierarchical correspondences between brain and deep networks through direct interface
title_sort reassessing hierarchical correspondences between brain and deep networks through direct interface
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9278854/
https://www.ncbi.nlm.nih.gov/pubmed/35857493
http://dx.doi.org/10.1126/sciadv.abm2219
work_keys_str_mv AT sextonnicholasj reassessinghierarchicalcorrespondencesbetweenbrainanddeepnetworksthroughdirectinterface
AT lovebradleyc reassessinghierarchicalcorrespondencesbetweenbrainanddeepnetworksthroughdirectinterface