Cargando…

Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving

As vehicles provide various services to drivers, research on driver emotion recognition has been expanding. However, current driver emotion datasets are limited by inconsistencies in collected data and inferred emotional state annotations by others. To overcome this limitation, we propose a data col...

Descripción completa

Detalles Bibliográficos
Autores principales: Oh, Geesung, Jeong, Euiseok, Kim, Rak Chul, Yang, Ji Hyun, Hwang, Sungwook, Lee, Sangho, Lim, Sejoon
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9230121/
https://www.ncbi.nlm.nih.gov/pubmed/35746182
http://dx.doi.org/10.3390/s22124402
_version_ 1784734982598557696
author Oh, Geesung
Jeong, Euiseok
Kim, Rak Chul
Yang, Ji Hyun
Hwang, Sungwook
Lee, Sangho
Lim, Sejoon
author_facet Oh, Geesung
Jeong, Euiseok
Kim, Rak Chul
Yang, Ji Hyun
Hwang, Sungwook
Lee, Sangho
Lim, Sejoon
author_sort Oh, Geesung
collection PubMed
description As vehicles provide various services to drivers, research on driver emotion recognition has been expanding. However, current driver emotion datasets are limited by inconsistencies in collected data and inferred emotional state annotations by others. To overcome this limitation, we propose a data collection system that collects multimodal datasets during real-world driving. The proposed system includes a self-reportable HMI application into which a driver directly inputs their current emotion state. Data collection was completed without any accidents for over 122 h of real-world driving using the system, which also considers the minimization of behavioral and cognitive disturbances. To demonstrate the validity of our collected dataset, we also provide case studies for statistical analysis, driver face detection, and personalized driver emotion recognition. The proposed data collection system enables the construction of reliable large-scale datasets on real-world driving and facilitates research on driver emotion recognition. The proposed system is avaliable on GitHub.
format Online
Article
Text
id pubmed-9230121
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-92301212022-06-25 Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving Oh, Geesung Jeong, Euiseok Kim, Rak Chul Yang, Ji Hyun Hwang, Sungwook Lee, Sangho Lim, Sejoon Sensors (Basel) Article As vehicles provide various services to drivers, research on driver emotion recognition has been expanding. However, current driver emotion datasets are limited by inconsistencies in collected data and inferred emotional state annotations by others. To overcome this limitation, we propose a data collection system that collects multimodal datasets during real-world driving. The proposed system includes a self-reportable HMI application into which a driver directly inputs their current emotion state. Data collection was completed without any accidents for over 122 h of real-world driving using the system, which also considers the minimization of behavioral and cognitive disturbances. To demonstrate the validity of our collected dataset, we also provide case studies for statistical analysis, driver face detection, and personalized driver emotion recognition. The proposed data collection system enables the construction of reliable large-scale datasets on real-world driving and facilitates research on driver emotion recognition. The proposed system is avaliable on GitHub. MDPI 2022-06-10 /pmc/articles/PMC9230121/ /pubmed/35746182 http://dx.doi.org/10.3390/s22124402 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Oh, Geesung
Jeong, Euiseok
Kim, Rak Chul
Yang, Ji Hyun
Hwang, Sungwook
Lee, Sangho
Lim, Sejoon
Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title_full Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title_fullStr Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title_full_unstemmed Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title_short Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving
title_sort multimodal data collection system for driver emotion recognition based on self-reporting in real-world driving
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9230121/
https://www.ncbi.nlm.nih.gov/pubmed/35746182
http://dx.doi.org/10.3390/s22124402
work_keys_str_mv AT ohgeesung multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT jeongeuiseok multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT kimrakchul multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT yangjihyun multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT hwangsungwook multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT leesangho multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving
AT limsejoon multimodaldatacollectionsystemfordriveremotionrecognitionbasedonselfreportinginrealworlddriving