Cargando…

Group‐level brain decoding with deep learning

Decoding brain imaging data are gaining popularity, with applications in brain‐computer interfaces and the study of neural representations. Decoding is typically subject‐specific and does not generalise well over subjects, due to high amounts of between subject variability. Techniques that overcome...

Descripción completa

Detalles Bibliográficos
Autores principales: Csaky, Richard, van Es, Mats W. J., Jones, Oiwi Parker, Woolrich, Mark
Formato: Online Artículo Texto
Lenguaje:English
Publicado: John Wiley & Sons, Inc. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619368/
https://www.ncbi.nlm.nih.gov/pubmed/37753636
http://dx.doi.org/10.1002/hbm.26500
_version_ 1785129973717139456
author Csaky, Richard
van Es, Mats W. J.
Jones, Oiwi Parker
Woolrich, Mark
author_facet Csaky, Richard
van Es, Mats W. J.
Jones, Oiwi Parker
Woolrich, Mark
author_sort Csaky, Richard
collection PubMed
description Decoding brain imaging data are gaining popularity, with applications in brain‐computer interfaces and the study of neural representations. Decoding is typically subject‐specific and does not generalise well over subjects, due to high amounts of between subject variability. Techniques that overcome this will not only provide richer neuroscientific insights but also make it possible for group‐level models to outperform subject‐specific models. Here, we propose a method that uses subject embedding, analogous to word embedding in natural language processing, to learn and exploit the structure in between‐subject variability as part of a decoding model, our adaptation of the WaveNet architecture for classification. We apply this to magnetoencephalography data, where 15 subjects viewed 118 different images, with 30 examples per image; to classify images using the entire 1 s window following image presentation. We show that the combination of deep learning and subject embedding is crucial to closing the performance gap between subject‐ and group‐level decoding models. Importantly, group models outperform subject models on low‐accuracy subjects (although slightly impair high‐accuracy subjects) and can be helpful for initialising subject models. While we have not generally found group‐level models to perform better than subject‐level models, the performance of group modelling is expected to be even higher with bigger datasets. In order to provide physiological interpretation at the group level, we make use of permutation feature importance. This provides insights into the spatiotemporal and spectral information encoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode).
format Online
Article
Text
id pubmed-10619368
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher John Wiley & Sons, Inc.
record_format MEDLINE/PubMed
spelling pubmed-106193682023-11-02 Group‐level brain decoding with deep learning Csaky, Richard van Es, Mats W. J. Jones, Oiwi Parker Woolrich, Mark Hum Brain Mapp Research Articles Decoding brain imaging data are gaining popularity, with applications in brain‐computer interfaces and the study of neural representations. Decoding is typically subject‐specific and does not generalise well over subjects, due to high amounts of between subject variability. Techniques that overcome this will not only provide richer neuroscientific insights but also make it possible for group‐level models to outperform subject‐specific models. Here, we propose a method that uses subject embedding, analogous to word embedding in natural language processing, to learn and exploit the structure in between‐subject variability as part of a decoding model, our adaptation of the WaveNet architecture for classification. We apply this to magnetoencephalography data, where 15 subjects viewed 118 different images, with 30 examples per image; to classify images using the entire 1 s window following image presentation. We show that the combination of deep learning and subject embedding is crucial to closing the performance gap between subject‐ and group‐level decoding models. Importantly, group models outperform subject models on low‐accuracy subjects (although slightly impair high‐accuracy subjects) and can be helpful for initialising subject models. While we have not generally found group‐level models to perform better than subject‐level models, the performance of group modelling is expected to be even higher with bigger datasets. In order to provide physiological interpretation at the group level, we make use of permutation feature importance. This provides insights into the spatiotemporal and spectral information encoded in the models. All code is available on GitHub (https://github.com/ricsinaruto/MEG-group-decode). John Wiley & Sons, Inc. 2023-09-27 /pmc/articles/PMC10619368/ /pubmed/37753636 http://dx.doi.org/10.1002/hbm.26500 Text en © 2023 The Authors. Human Brain Mapping published by Wiley Periodicals LLC. https://creativecommons.org/licenses/by/4.0/This is an open access article under the terms of the http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
spellingShingle Research Articles
Csaky, Richard
van Es, Mats W. J.
Jones, Oiwi Parker
Woolrich, Mark
Group‐level brain decoding with deep learning
title Group‐level brain decoding with deep learning
title_full Group‐level brain decoding with deep learning
title_fullStr Group‐level brain decoding with deep learning
title_full_unstemmed Group‐level brain decoding with deep learning
title_short Group‐level brain decoding with deep learning
title_sort group‐level brain decoding with deep learning
topic Research Articles
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619368/
https://www.ncbi.nlm.nih.gov/pubmed/37753636
http://dx.doi.org/10.1002/hbm.26500
work_keys_str_mv AT csakyrichard grouplevelbraindecodingwithdeeplearning
AT vanesmatswj grouplevelbraindecodingwithdeeplearning
AT jonesoiwiparker grouplevelbraindecodingwithdeeplearning
AT woolrichmark grouplevelbraindecodingwithdeeplearning