Cargando…

Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics

With the development of the autopilot system, the main task of a pilot has changed from controlling the aircraft to supervising the autopilot system and making critical decisions. Therefore, the human–machine interaction system needs to be improved accordingly. A key step to improving the human–mach...

Descripción completa

Detalles Bibliográficos
Autores principales: Li, Yuhan, Li, Ke, Wang, Shaofan, Chen, Xiaodan, Wen, Dongsheng
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9221330/
https://www.ncbi.nlm.nih.gov/pubmed/35735552
http://dx.doi.org/10.3390/bios12060404
_version_ 1784732594729910272
author Li, Yuhan
Li, Ke
Wang, Shaofan
Chen, Xiaodan
Wen, Dongsheng
author_facet Li, Yuhan
Li, Ke
Wang, Shaofan
Chen, Xiaodan
Wen, Dongsheng
author_sort Li, Yuhan
collection PubMed
description With the development of the autopilot system, the main task of a pilot has changed from controlling the aircraft to supervising the autopilot system and making critical decisions. Therefore, the human–machine interaction system needs to be improved accordingly. A key step to improving the human–machine interaction system is to improve its understanding of the pilots’ status, including fatigue, stress, workload, etc. Monitoring pilots’ status can effectively prevent human error and achieve optimal human–machine collaboration. As such, there is a need to recognize pilots’ status and predict the behaviors responsible for changes of state. For this purpose, in this study, 14 Air Force cadets fly in an F-35 Lightning II Joint Strike Fighter simulator through a series of maneuvers involving takeoff, level flight, turn and hover, roll, somersault, and stall. Electro cardio (ECG), myoelectricity (EMG), galvanic skin response (GSR), respiration (RESP), and skin temperature (SKT) measurements are derived through wearable physiological data collection devices. Physiological indicators influenced by the pilot’s behavioral status are objectively analyzed. Multi-modality fusion technology (MTF) is adopted to fuse these data in the feature layer. Additionally, four classifiers are integrated to identify pilots’ behaviors in the strategy layer. The results indicate that MTF can help to recognize pilot behavior in a more comprehensive and precise way.
format Online
Article
Text
id pubmed-9221330
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-92213302022-06-24 Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics Li, Yuhan Li, Ke Wang, Shaofan Chen, Xiaodan Wen, Dongsheng Biosensors (Basel) Article With the development of the autopilot system, the main task of a pilot has changed from controlling the aircraft to supervising the autopilot system and making critical decisions. Therefore, the human–machine interaction system needs to be improved accordingly. A key step to improving the human–machine interaction system is to improve its understanding of the pilots’ status, including fatigue, stress, workload, etc. Monitoring pilots’ status can effectively prevent human error and achieve optimal human–machine collaboration. As such, there is a need to recognize pilots’ status and predict the behaviors responsible for changes of state. For this purpose, in this study, 14 Air Force cadets fly in an F-35 Lightning II Joint Strike Fighter simulator through a series of maneuvers involving takeoff, level flight, turn and hover, roll, somersault, and stall. Electro cardio (ECG), myoelectricity (EMG), galvanic skin response (GSR), respiration (RESP), and skin temperature (SKT) measurements are derived through wearable physiological data collection devices. Physiological indicators influenced by the pilot’s behavioral status are objectively analyzed. Multi-modality fusion technology (MTF) is adopted to fuse these data in the feature layer. Additionally, four classifiers are integrated to identify pilots’ behaviors in the strategy layer. The results indicate that MTF can help to recognize pilot behavior in a more comprehensive and precise way. MDPI 2022-06-12 /pmc/articles/PMC9221330/ /pubmed/35735552 http://dx.doi.org/10.3390/bios12060404 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Li, Yuhan
Li, Ke
Wang, Shaofan
Chen, Xiaodan
Wen, Dongsheng
Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title_full Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title_fullStr Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title_full_unstemmed Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title_short Pilot Behavior Recognition Based on Multi-Modality Fusion Technology Using Physiological Characteristics
title_sort pilot behavior recognition based on multi-modality fusion technology using physiological characteristics
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9221330/
https://www.ncbi.nlm.nih.gov/pubmed/35735552
http://dx.doi.org/10.3390/bios12060404
work_keys_str_mv AT liyuhan pilotbehaviorrecognitionbasedonmultimodalityfusiontechnologyusingphysiologicalcharacteristics
AT like pilotbehaviorrecognitionbasedonmultimodalityfusiontechnologyusingphysiologicalcharacteristics
AT wangshaofan pilotbehaviorrecognitionbasedonmultimodalityfusiontechnologyusingphysiologicalcharacteristics
AT chenxiaodan pilotbehaviorrecognitionbasedonmultimodalityfusiontechnologyusingphysiologicalcharacteristics
AT wendongsheng pilotbehaviorrecognitionbasedonmultimodalityfusiontechnologyusingphysiologicalcharacteristics