Cargando…

Novel methods for elucidating modality importance in multimodal electrophysiology classifiers

INTRODUCTION: Multimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning beca...

Descripción completa

Detalles Bibliográficos
Autores principales: Ellis, Charles A., Sendi, Mohammad S. E., Zhang, Rongen, Carbajal, Darwin A., Wang, May D., Miller, Robyn L., Calhoun, Vince D.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10050434/
https://www.ncbi.nlm.nih.gov/pubmed/37006636
http://dx.doi.org/10.3389/fninf.2023.1123376
_version_ 1785014649694978048
author Ellis, Charles A.
Sendi, Mohammad S. E.
Zhang, Rongen
Carbajal, Darwin A.
Wang, May D.
Miller, Robyn L.
Calhoun, Vince D.
author_facet Ellis, Charles A.
Sendi, Mohammad S. E.
Zhang, Rongen
Carbajal, Darwin A.
Wang, May D.
Miller, Robyn L.
Calhoun, Vince D.
author_sort Ellis, Charles A.
collection PubMed
description INTRODUCTION: Multimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning because explainability is vital to the development and implementation of clinical classifiers. As such, new multimodal explainability methods are needed. METHODS: In this study, we train a convolutional neural network for automated sleep stage classification with electroencephalogram (EEG), electrooculogram, and electromyogram data. We then present a global explainability approach that is uniquely adapted for electrophysiology analysis and compare it to an existing approach. We present the first two local multimodal explainability approaches. We look for subject-level differences in the local explanations that are obscured by global methods and look for relationships between the explanations and clinical and demographic variables in a novel analysis. RESULTS: We find a high level of agreement between methods. We find that EEG is globally the most important modality for most sleep stages and that subject-level differences in importance arise in local explanations that are not captured in global explanations. We further show that sex, followed by medication and age, had significant effects upon the patterns learned by the classifier. DISCUSSION: Our novel methods enhance explainability for the growing field of multimodal electrophysiology classification, provide avenues for the advancement of personalized medicine, yield unique insights into the effects of demographic and clinical variables upon classifiers, and help pave the way for the implementation of multimodal electrophysiology clinical classifiers.
format Online
Article
Text
id pubmed-10050434
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-100504342023-03-30 Novel methods for elucidating modality importance in multimodal electrophysiology classifiers Ellis, Charles A. Sendi, Mohammad S. E. Zhang, Rongen Carbajal, Darwin A. Wang, May D. Miller, Robyn L. Calhoun, Vince D. Front Neuroinform Neuroscience INTRODUCTION: Multimodal classification is increasingly common in electrophysiology studies. Many studies use deep learning classifiers with raw time-series data, which makes explainability difficult, and has resulted in relatively few studies applying explainability methods. This is concerning because explainability is vital to the development and implementation of clinical classifiers. As such, new multimodal explainability methods are needed. METHODS: In this study, we train a convolutional neural network for automated sleep stage classification with electroencephalogram (EEG), electrooculogram, and electromyogram data. We then present a global explainability approach that is uniquely adapted for electrophysiology analysis and compare it to an existing approach. We present the first two local multimodal explainability approaches. We look for subject-level differences in the local explanations that are obscured by global methods and look for relationships between the explanations and clinical and demographic variables in a novel analysis. RESULTS: We find a high level of agreement between methods. We find that EEG is globally the most important modality for most sleep stages and that subject-level differences in importance arise in local explanations that are not captured in global explanations. We further show that sex, followed by medication and age, had significant effects upon the patterns learned by the classifier. DISCUSSION: Our novel methods enhance explainability for the growing field of multimodal electrophysiology classification, provide avenues for the advancement of personalized medicine, yield unique insights into the effects of demographic and clinical variables upon classifiers, and help pave the way for the implementation of multimodal electrophysiology clinical classifiers. Frontiers Media S.A. 2023-03-15 /pmc/articles/PMC10050434/ /pubmed/37006636 http://dx.doi.org/10.3389/fninf.2023.1123376 Text en Copyright © 2023 Ellis, Sendi, Zhang, Carbajal, Wang, Miller and Calhoun. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Ellis, Charles A.
Sendi, Mohammad S. E.
Zhang, Rongen
Carbajal, Darwin A.
Wang, May D.
Miller, Robyn L.
Calhoun, Vince D.
Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title_full Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title_fullStr Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title_full_unstemmed Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title_short Novel methods for elucidating modality importance in multimodal electrophysiology classifiers
title_sort novel methods for elucidating modality importance in multimodal electrophysiology classifiers
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10050434/
https://www.ncbi.nlm.nih.gov/pubmed/37006636
http://dx.doi.org/10.3389/fninf.2023.1123376
work_keys_str_mv AT ellischarlesa novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT sendimohammadse novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT zhangrongen novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT carbajaldarwina novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT wangmayd novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT millerrobynl novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers
AT calhounvinced novelmethodsforelucidatingmodalityimportanceinmultimodalelectrophysiologyclassifiers