Cargando…

A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition

The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging...

Descripción completa

Detalles Bibliográficos
Autores principales: Jahangir, Faiza, Khan, Muhammad Attique, Alhaisoni, Majed, Alqahtani, Abdullah, Alsubai, Shtwai, Sha, Mohemmed, Al Hejaili, Abdullah, Cha, Jae-hyuk
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007680/
https://www.ncbi.nlm.nih.gov/pubmed/36904963
http://dx.doi.org/10.3390/s23052754
_version_ 1784905583174877184
author Jahangir, Faiza
Khan, Muhammad Attique
Alhaisoni, Majed
Alqahtani, Abdullah
Alsubai, Shtwai
Sha, Mohemmed
Al Hejaili, Abdullah
Cha, Jae-hyuk
author_facet Jahangir, Faiza
Khan, Muhammad Attique
Alhaisoni, Majed
Alqahtani, Abdullah
Alsubai, Shtwai
Sha, Mohemmed
Al Hejaili, Abdullah
Cha, Jae-hyuk
author_sort Jahangir, Faiza
collection PubMed
description The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models—MobilenetV2 and ShuffleNet—are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton–Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time.
format Online
Article
Text
id pubmed-10007680
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100076802023-03-12 A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition Jahangir, Faiza Khan, Muhammad Attique Alhaisoni, Majed Alqahtani, Abdullah Alsubai, Shtwai Sha, Mohemmed Al Hejaili, Abdullah Cha, Jae-hyuk Sensors (Basel) Article The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models—MobilenetV2 and ShuffleNet—are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton–Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time. MDPI 2023-03-02 /pmc/articles/PMC10007680/ /pubmed/36904963 http://dx.doi.org/10.3390/s23052754 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Jahangir, Faiza
Khan, Muhammad Attique
Alhaisoni, Majed
Alqahtani, Abdullah
Alsubai, Shtwai
Sha, Mohemmed
Al Hejaili, Abdullah
Cha, Jae-hyuk
A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title_full A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title_fullStr A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title_full_unstemmed A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title_short A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
title_sort fusion-assisted multi-stream deep learning and eso-controlled newton–raphson-based feature selection approach for human gait recognition
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007680/
https://www.ncbi.nlm.nih.gov/pubmed/36904963
http://dx.doi.org/10.3390/s23052754
work_keys_str_mv AT jahangirfaiza afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT khanmuhammadattique afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alhaisonimajed afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alqahtaniabdullah afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alsubaishtwai afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT shamohemmed afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alhejailiabdullah afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT chajaehyuk afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT jahangirfaiza fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT khanmuhammadattique fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alhaisonimajed fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alqahtaniabdullah fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alsubaishtwai fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT shamohemmed fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT alhejailiabdullah fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition
AT chajaehyuk fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition