Cargando…

A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers

With the rapid spreading of in-vehicle information systems such as smartphones, navigation systems, and radios, the number of traffic accidents caused by driver distractions shows an increasing trend. Timely identification and warning are deemed to be crucial for distracted driving and the establish...

Descripción completa

Detalles Bibliográficos
Autores principales: Jiao, Shuang-Jian, Liu, Lin-Yao, Liu, Qian
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8588220/
https://www.ncbi.nlm.nih.gov/pubmed/34770728
http://dx.doi.org/10.3390/s21217424
_version_ 1784598393257984000
author Jiao, Shuang-Jian
Liu, Lin-Yao
Liu, Qian
author_facet Jiao, Shuang-Jian
Liu, Lin-Yao
Liu, Qian
author_sort Jiao, Shuang-Jian
collection PubMed
description With the rapid spreading of in-vehicle information systems such as smartphones, navigation systems, and radios, the number of traffic accidents caused by driver distractions shows an increasing trend. Timely identification and warning are deemed to be crucial for distracted driving and the establishment of driver assistance systems is of great value. However, almost all research on the recognition of the driver’s distracted actions using computer vision methods neglected the importance of temporal information for action recognition. This paper proposes a hybrid deep learning model for recognizing the actions of distracted drivers. Specifically, we used OpenPose to obtain skeleton information of the human body and then constructed the vector angle and modulus ratio of the human body structure as features to describe the driver’s actions, thereby realizing the fusion of deep network features and artificial features, which improve the information density of spatial features. The K-means clustering algorithm was used to preselect the original frames, and the method of inter-frame comparison was used to obtain the final keyframe sequence by comparing the Euclidean distance between manually constructed vectors representing frames and the vector representing the cluster center. Finally, we constructed a two-layer long short-term memory neural network to obtain more effective spatiotemporal features, and one softmax layer to identify the distracted driver’s action. The experimental results based on the collected dataset prove the effectiveness of this framework, and it can provide a theoretical basis for the establishment of vehicle distraction warning systems.
format Online
Article
Text
id pubmed-8588220
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-85882202021-11-13 A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers Jiao, Shuang-Jian Liu, Lin-Yao Liu, Qian Sensors (Basel) Article With the rapid spreading of in-vehicle information systems such as smartphones, navigation systems, and radios, the number of traffic accidents caused by driver distractions shows an increasing trend. Timely identification and warning are deemed to be crucial for distracted driving and the establishment of driver assistance systems is of great value. However, almost all research on the recognition of the driver’s distracted actions using computer vision methods neglected the importance of temporal information for action recognition. This paper proposes a hybrid deep learning model for recognizing the actions of distracted drivers. Specifically, we used OpenPose to obtain skeleton information of the human body and then constructed the vector angle and modulus ratio of the human body structure as features to describe the driver’s actions, thereby realizing the fusion of deep network features and artificial features, which improve the information density of spatial features. The K-means clustering algorithm was used to preselect the original frames, and the method of inter-frame comparison was used to obtain the final keyframe sequence by comparing the Euclidean distance between manually constructed vectors representing frames and the vector representing the cluster center. Finally, we constructed a two-layer long short-term memory neural network to obtain more effective spatiotemporal features, and one softmax layer to identify the distracted driver’s action. The experimental results based on the collected dataset prove the effectiveness of this framework, and it can provide a theoretical basis for the establishment of vehicle distraction warning systems. MDPI 2021-11-08 /pmc/articles/PMC8588220/ /pubmed/34770728 http://dx.doi.org/10.3390/s21217424 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Jiao, Shuang-Jian
Liu, Lin-Yao
Liu, Qian
A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title_full A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title_fullStr A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title_full_unstemmed A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title_short A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers
title_sort hybrid deep learning model for recognizing actions of distracted drivers
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8588220/
https://www.ncbi.nlm.nih.gov/pubmed/34770728
http://dx.doi.org/10.3390/s21217424
work_keys_str_mv AT jiaoshuangjian ahybriddeeplearningmodelforrecognizingactionsofdistracteddrivers
AT liulinyao ahybriddeeplearningmodelforrecognizingactionsofdistracteddrivers
AT liuqian ahybriddeeplearningmodelforrecognizingactionsofdistracteddrivers
AT jiaoshuangjian hybriddeeplearningmodelforrecognizingactionsofdistracteddrivers
AT liulinyao hybriddeeplearningmodelforrecognizingactionsofdistracteddrivers
AT liuqian hybriddeeplearningmodelforrecognizingactionsofdistracteddrivers