Cargando…
A multimodal screening system for elderly neurological diseases based on deep learning
In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that cap...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10687257/ https://www.ncbi.nlm.nih.gov/pubmed/38030653 http://dx.doi.org/10.1038/s41598-023-48071-y |
_version_ | 1785151945115172864 |
---|---|
author | Park, Sangyoung No, Changho Kim, Sora Han, Kyoungmin Jung, Jin-Man Kwon, Kyum-Yil Lee, Minsik |
author_facet | Park, Sangyoung No, Changho Kim, Sora Han, Kyoungmin Jung, Jin-Man Kwon, Kyum-Yil Lee, Minsik |
author_sort | Park, Sangyoung |
collection | PubMed |
description | In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that capture action information with a much smaller data dimension. We also used voice data which are also effective indicators of neurological disorders. We designed a subnetwork for each protocol to extract features from landmarks or voice and a feature aggregator that combines all the information extracted from the protocols to make a final decision. Multitask learning was applied to screen two neurological diseases. To capture meaningful information about these human landmarks and voices, we applied various pre-trained models to extract preliminary features. The spatiotemporal characteristics of landmarks are extracted using a pre-trained graph neural network, and voice features are extracted using a pre-trained time-delay neural network. These extracted high-level features are then passed onto the subnetworks and an additional feature aggregator that are simultaneously trained. We also used various data augmentation techniques to overcome the shortage of data. Using a frame-length staticizer that considers the characteristics of the data, we can capture momentary tremors without wasting information. Finally, we examine the effectiveness of different protocols and different modalities (different body parts and voice) through extensive experiments. The proposed method achieves AUC scores of 0.802 for stroke and 0.780 for Parkinson’s disease, which is effective for a screening system. |
format | Online Article Text |
id | pubmed-10687257 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-106872572023-11-30 A multimodal screening system for elderly neurological diseases based on deep learning Park, Sangyoung No, Changho Kim, Sora Han, Kyoungmin Jung, Jin-Man Kwon, Kyum-Yil Lee, Minsik Sci Rep Article In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that capture action information with a much smaller data dimension. We also used voice data which are also effective indicators of neurological disorders. We designed a subnetwork for each protocol to extract features from landmarks or voice and a feature aggregator that combines all the information extracted from the protocols to make a final decision. Multitask learning was applied to screen two neurological diseases. To capture meaningful information about these human landmarks and voices, we applied various pre-trained models to extract preliminary features. The spatiotemporal characteristics of landmarks are extracted using a pre-trained graph neural network, and voice features are extracted using a pre-trained time-delay neural network. These extracted high-level features are then passed onto the subnetworks and an additional feature aggregator that are simultaneously trained. We also used various data augmentation techniques to overcome the shortage of data. Using a frame-length staticizer that considers the characteristics of the data, we can capture momentary tremors without wasting information. Finally, we examine the effectiveness of different protocols and different modalities (different body parts and voice) through extensive experiments. The proposed method achieves AUC scores of 0.802 for stroke and 0.780 for Parkinson’s disease, which is effective for a screening system. Nature Publishing Group UK 2023-11-29 /pmc/articles/PMC10687257/ /pubmed/38030653 http://dx.doi.org/10.1038/s41598-023-48071-y Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Park, Sangyoung No, Changho Kim, Sora Han, Kyoungmin Jung, Jin-Man Kwon, Kyum-Yil Lee, Minsik A multimodal screening system for elderly neurological diseases based on deep learning |
title | A multimodal screening system for elderly neurological diseases based on deep learning |
title_full | A multimodal screening system for elderly neurological diseases based on deep learning |
title_fullStr | A multimodal screening system for elderly neurological diseases based on deep learning |
title_full_unstemmed | A multimodal screening system for elderly neurological diseases based on deep learning |
title_short | A multimodal screening system for elderly neurological diseases based on deep learning |
title_sort | multimodal screening system for elderly neurological diseases based on deep learning |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10687257/ https://www.ncbi.nlm.nih.gov/pubmed/38030653 http://dx.doi.org/10.1038/s41598-023-48071-y |
work_keys_str_mv | AT parksangyoung amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT nochangho amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT kimsora amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT hankyoungmin amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT jungjinman amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT kwonkyumyil amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT leeminsik amultimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT parksangyoung multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT nochangho multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT kimsora multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT hankyoungmin multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT jungjinman multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT kwonkyumyil multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning AT leeminsik multimodalscreeningsystemforelderlyneurologicaldiseasesbasedondeeplearning |