Cargando…

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable s...

Descripción completa

Detalles Bibliográficos
Autores principales: Li, Jing, Liang, Weisheng, Yin, Xiyan, Li, Jun, Guan, Weizheng
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675737/
https://www.ncbi.nlm.nih.gov/pubmed/38005489
http://dx.doi.org/10.3390/s23229101
_version_ 1785141135628304384
author Li, Jing
Liang, Weisheng
Yin, Xiyan
Li, Jun
Guan, Weizheng
author_facet Li, Jing
Liang, Weisheng
Yin, Xiyan
Li, Jun
Guan, Weizheng
author_sort Li, Jing
collection PubMed
description Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor’s type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson’s disease (PD), and Huntington’s disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson’s disease severity, surpassing DCLSTM’s 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.
format Online
Article
Text
id pubmed-10675737
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106757372023-11-10 Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion Li, Jing Liang, Weisheng Yin, Xiyan Li, Jun Guan, Weizheng Sensors (Basel) Article Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor’s type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson’s disease (PD), and Huntington’s disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson’s disease severity, surpassing DCLSTM’s 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases. MDPI 2023-11-10 /pmc/articles/PMC10675737/ /pubmed/38005489 http://dx.doi.org/10.3390/s23229101 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Li, Jing
Liang, Weisheng
Yin, Xiyan
Li, Jun
Guan, Weizheng
Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title_full Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title_fullStr Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title_full_unstemmed Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title_short Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion
title_sort multimodal gait abnormality recognition using a convolutional neural network–bidirectional long short-term memory (cnn-bilstm) network based on multi-sensor data fusion
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675737/
https://www.ncbi.nlm.nih.gov/pubmed/38005489
http://dx.doi.org/10.3390/s23229101
work_keys_str_mv AT lijing multimodalgaitabnormalityrecognitionusingaconvolutionalneuralnetworkbidirectionallongshorttermmemorycnnbilstmnetworkbasedonmultisensordatafusion
AT liangweisheng multimodalgaitabnormalityrecognitionusingaconvolutionalneuralnetworkbidirectionallongshorttermmemorycnnbilstmnetworkbasedonmultisensordatafusion
AT yinxiyan multimodalgaitabnormalityrecognitionusingaconvolutionalneuralnetworkbidirectionallongshorttermmemorycnnbilstmnetworkbasedonmultisensordatafusion
AT lijun multimodalgaitabnormalityrecognitionusingaconvolutionalneuralnetworkbidirectionallongshorttermmemorycnnbilstmnetworkbasedonmultisensordatafusion
AT guanweizheng multimodalgaitabnormalityrecognitionusingaconvolutionalneuralnetworkbidirectionallongshorttermmemorycnnbilstmnetworkbasedonmultisensordatafusion