Cargando…

Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory

Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/o...

Descripción completa

Detalles Bibliográficos
Autores principales: Abdullahi, Sunusi Bala, Bature, Zakariyya Abdullahi, Gabralla, Lubna A., Chiroma, Haruna
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10137095/
https://www.ncbi.nlm.nih.gov/pubmed/37190520
http://dx.doi.org/10.3390/brainsci13040555
_version_ 1785032376912445440
author Abdullahi, Sunusi Bala
Bature, Zakariyya Abdullahi
Gabralla, Lubna A.
Chiroma, Haruna
author_facet Abdullahi, Sunusi Bala
Bature, Zakariyya Abdullahi
Gabralla, Lubna A.
Chiroma, Haruna
author_sort Abdullahi, Sunusi Bala
collection PubMed
description Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/or automatic lie features during court trials and police interrogations. Unfortunately, due to the cognitive complexity and the lack of involuntary cues related to lying features, the performances of these approaches suffer and their generalization ability is limited. To improve performance, this study proposed state transition patterns based on hands, body motions, and eye blinking features from real-life court trial videos. Each video frame is represented according to a computed threshold value among neighboring pixels to extract spatial–temporal state transition patterns (STSTP) of the hand and face poses as involuntary cues using fully connected convolution neural network layers optimized with the weights of ResNet-152 learning. In addition, this study computed an eye aspect ratio model to obtain eye blinking features. These features were fused together as a single multi-modal STSTP feature model. The model was built using the enhanced calculated weight of bidirectional long short-term memory. The proposed approach was evaluated by comparing its performance with current state-of-the-art methods. It was found that the proposed approach improves the performance of detecting lies.
format Online
Article
Text
id pubmed-10137095
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-101370952023-04-28 Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory Abdullahi, Sunusi Bala Bature, Zakariyya Abdullahi Gabralla, Lubna A. Chiroma, Haruna Brain Sci Article Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/or automatic lie features during court trials and police interrogations. Unfortunately, due to the cognitive complexity and the lack of involuntary cues related to lying features, the performances of these approaches suffer and their generalization ability is limited. To improve performance, this study proposed state transition patterns based on hands, body motions, and eye blinking features from real-life court trial videos. Each video frame is represented according to a computed threshold value among neighboring pixels to extract spatial–temporal state transition patterns (STSTP) of the hand and face poses as involuntary cues using fully connected convolution neural network layers optimized with the weights of ResNet-152 learning. In addition, this study computed an eye aspect ratio model to obtain eye blinking features. These features were fused together as a single multi-modal STSTP feature model. The model was built using the enhanced calculated weight of bidirectional long short-term memory. The proposed approach was evaluated by comparing its performance with current state-of-the-art methods. It was found that the proposed approach improves the performance of detecting lies. MDPI 2023-03-25 /pmc/articles/PMC10137095/ /pubmed/37190520 http://dx.doi.org/10.3390/brainsci13040555 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Abdullahi, Sunusi Bala
Bature, Zakariyya Abdullahi
Gabralla, Lubna A.
Chiroma, Haruna
Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title_full Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title_fullStr Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title_full_unstemmed Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title_short Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
title_sort lie recognition with multi-modal spatial–temporal state transition patterns based on hybrid convolutional neural network–bidirectional long short-term memory
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10137095/
https://www.ncbi.nlm.nih.gov/pubmed/37190520
http://dx.doi.org/10.3390/brainsci13040555
work_keys_str_mv AT abdullahisunusibala lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory
AT baturezakariyyaabdullahi lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory
AT gabrallalubnaa lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory
AT chiromaharuna lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory