Next Article in Journal
Elastic Provisioning of Network and Computing Resources at the Edge for IoT Services
Next Article in Special Issue
Human Activity Recognition in the Presence of Occlusion
Previous Article in Journal
Data-Driven Smart Living Lab to Promote Participation in Rehabilitation Exercises and Sports Programs for People with Disabilities in Local Communities
Previous Article in Special Issue
Vision-Based HAR in UAV Videos Using Histograms and Deep Learning Techniques
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition

1
Department of Computer Science, HITEC University, Taxila 47080, Pakistan
2
Department of Informatics, University of Leicester, Leicester LE1 7RH, UK
3
College of Computer Science and Engineering, University of Ha’il, Ha’il 81451, Saudi Arabia
4
Department of Software Engineering, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-kharj 16242, Saudi Arabia
5
Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-kharj 16242, Saudi Arabia
6
Faculty of Computers & Information Technology, Computer Science Department, University of Tabuk, Tabuk 71491, Saudi Arabia
7
Department of computer Science, Hanyang University, Seoul 04763, Republic of Korea
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(5), 2754; https://doi.org/10.3390/s23052754
Submission received: 31 January 2023 / Revised: 24 February 2023 / Accepted: 25 February 2023 / Published: 2 March 2023

Abstract

The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models—MobilenetV2 and ShuffleNet—are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton–Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time.
Keywords: gait recognition; contrast enhancement; deep learning; feature selection; fusion; machine learning gait recognition; contrast enhancement; deep learning; feature selection; fusion; machine learning

Share and Cite

MDPI and ACS Style

Jahangir, F.; Khan, M.A.; Alhaisoni, M.; Alqahtani, A.; Alsubai, S.; Sha, M.; Al Hejaili, A.; Cha, J.-h. A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition. Sensors 2023, 23, 2754. https://doi.org/10.3390/s23052754

AMA Style

Jahangir F, Khan MA, Alhaisoni M, Alqahtani A, Alsubai S, Sha M, Al Hejaili A, Cha J-h. A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition. Sensors. 2023; 23(5):2754. https://doi.org/10.3390/s23052754

Chicago/Turabian Style

Jahangir, Faiza, Muhammad Attique Khan, Majed Alhaisoni, Abdullah Alqahtani, Shtwai Alsubai, Mohemmed Sha, Abdullah Al Hejaili, and Jae-hyuk Cha. 2023. "A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition" Sensors 23, no. 5: 2754. https://doi.org/10.3390/s23052754

APA Style

Jahangir, F., Khan, M. A., Alhaisoni, M., Alqahtani, A., Alsubai, S., Sha, M., Al Hejaili, A., & Cha, J.-h. (2023). A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton–Raphson-Based Feature Selection Approach for Human Gait Recognition. Sensors, 23(5), 2754. https://doi.org/10.3390/s23052754

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop