Cargando…
A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition
The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging...
Autores principales: | , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007680/ https://www.ncbi.nlm.nih.gov/pubmed/36904963 http://dx.doi.org/10.3390/s23052754 |
_version_ | 1784905583174877184 |
---|---|
author | Jahangir, Faiza Khan, Muhammad Attique Alhaisoni, Majed Alqahtani, Abdullah Alsubai, Shtwai Sha, Mohemmed Al Hejaili, Abdullah Cha, Jae-hyuk |
author_facet | Jahangir, Faiza Khan, Muhammad Attique Alhaisoni, Majed Alqahtani, Abdullah Alsubai, Shtwai Sha, Mohemmed Al Hejaili, Abdullah Cha, Jae-hyuk |
author_sort | Jahangir, Faiza |
collection | PubMed |
description | The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models—MobilenetV2 and ShuffleNet—are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton–Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time. |
format | Online Article Text |
id | pubmed-10007680 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-100076802023-03-12 A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition Jahangir, Faiza Khan, Muhammad Attique Alhaisoni, Majed Alqahtani, Abdullah Alsubai, Shtwai Sha, Mohemmed Al Hejaili, Abdullah Cha, Jae-hyuk Sensors (Basel) Article The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models—MobilenetV2 and ShuffleNet—are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton–Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time. MDPI 2023-03-02 /pmc/articles/PMC10007680/ /pubmed/36904963 http://dx.doi.org/10.3390/s23052754 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Jahangir, Faiza Khan, Muhammad Attique Alhaisoni, Majed Alqahtani, Abdullah Alsubai, Shtwai Sha, Mohemmed Al Hejaili, Abdullah Cha, Jae-hyuk A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title | A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title_full | A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title_fullStr | A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title_full_unstemmed | A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title_short | A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition |
title_sort | fusion-assisted multi-stream deep learning and eso-controlled newton–raphson-based feature selection approach for human gait recognition |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007680/ https://www.ncbi.nlm.nih.gov/pubmed/36904963 http://dx.doi.org/10.3390/s23052754 |
work_keys_str_mv | AT jahangirfaiza afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT khanmuhammadattique afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alhaisonimajed afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alqahtaniabdullah afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alsubaishtwai afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT shamohemmed afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alhejailiabdullah afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT chajaehyuk afusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT jahangirfaiza fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT khanmuhammadattique fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alhaisonimajed fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alqahtaniabdullah fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alsubaishtwai fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT shamohemmed fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT alhejailiabdullah fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition AT chajaehyuk fusionassistedmultistreamdeeplearningandesocontrollednewtonraphsonbasedfeatureselectionapproachforhumangaitrecognition |