Cargando…

A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents

Sleep-deprived fatigued person is likely to commit more errors that may even prove to be fatal. Thus, it is necessary to recognize this fatigue. The novelty of the proposed research work for the detection of this fatigue is that it is nonintrusive and based on multimodal feature fusion. In the propo...

Descripción completa

Detalles Bibliográficos
Autores principales: Virk, Jitender Singh, Singh, Mandeep, Panjwani, Usha, Ray, Koushik
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10144633/
https://www.ncbi.nlm.nih.gov/pubmed/37112470
http://dx.doi.org/10.3390/s23084129
_version_ 1785034145090502656
author Virk, Jitender Singh
Singh, Mandeep
Singh, Mandeep
Panjwani, Usha
Ray, Koushik
author_facet Virk, Jitender Singh
Singh, Mandeep
Singh, Mandeep
Panjwani, Usha
Ray, Koushik
author_sort Virk, Jitender Singh
collection PubMed
description Sleep-deprived fatigued person is likely to commit more errors that may even prove to be fatal. Thus, it is necessary to recognize this fatigue. The novelty of the proposed research work for the detection of this fatigue is that it is nonintrusive and based on multimodal feature fusion. In the proposed methodology, fatigue is detected by obtaining features from four domains: visual images, thermal images, keystroke dynamics, and voice features. In the proposed methodology, the samples of a volunteer (subject) are obtained from all four domains for feature extraction, and empirical weights are assigned to the four different domains. Young, healthy volunteers (n = 60) between the age group of 20 to 30 years participated in the experimental study. Further, they abstained from the consumption of alcohol, caffeine, or other drugs impacting their sleep pattern during the study. Through this multimodal technique, appropriate weights are given to the features obtained from the four domains. The results are compared with k-nearest neighbors (kNN), support vector machines (SVM), random tree, random forest, and multilayer perceptron classifiers. The proposed nonintrusive technique has obtained an average detection accuracy of 93.33% in 3-fold cross-validation.
format Online
Article
Text
id pubmed-10144633
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-101446332023-04-29 A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents Virk, Jitender Singh Singh, Mandeep Singh, Mandeep Panjwani, Usha Ray, Koushik Sensors (Basel) Article Sleep-deprived fatigued person is likely to commit more errors that may even prove to be fatal. Thus, it is necessary to recognize this fatigue. The novelty of the proposed research work for the detection of this fatigue is that it is nonintrusive and based on multimodal feature fusion. In the proposed methodology, fatigue is detected by obtaining features from four domains: visual images, thermal images, keystroke dynamics, and voice features. In the proposed methodology, the samples of a volunteer (subject) are obtained from all four domains for feature extraction, and empirical weights are assigned to the four different domains. Young, healthy volunteers (n = 60) between the age group of 20 to 30 years participated in the experimental study. Further, they abstained from the consumption of alcohol, caffeine, or other drugs impacting their sleep pattern during the study. Through this multimodal technique, appropriate weights are given to the features obtained from the four domains. The results are compared with k-nearest neighbors (kNN), support vector machines (SVM), random tree, random forest, and multilayer perceptron classifiers. The proposed nonintrusive technique has obtained an average detection accuracy of 93.33% in 3-fold cross-validation. MDPI 2023-04-20 /pmc/articles/PMC10144633/ /pubmed/37112470 http://dx.doi.org/10.3390/s23084129 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Virk, Jitender Singh
Singh, Mandeep
Singh, Mandeep
Panjwani, Usha
Ray, Koushik
A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title_full A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title_fullStr A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title_full_unstemmed A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title_short A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents
title_sort multimodal feature fusion framework for sleep-deprived fatigue detection to prevent accidents
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10144633/
https://www.ncbi.nlm.nih.gov/pubmed/37112470
http://dx.doi.org/10.3390/s23084129
work_keys_str_mv AT virkjitendersingh amultimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT singhmandeep amultimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT singhmandeep amultimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT panjwaniusha amultimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT raykoushik amultimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT virkjitendersingh multimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT singhmandeep multimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT singhmandeep multimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT panjwaniusha multimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents
AT raykoushik multimodalfeaturefusionframeworkforsleepdeprivedfatiguedetectiontopreventaccidents