Cargando…
Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory
Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/o...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10137095/ https://www.ncbi.nlm.nih.gov/pubmed/37190520 http://dx.doi.org/10.3390/brainsci13040555 |
_version_ | 1785032376912445440 |
---|---|
author | Abdullahi, Sunusi Bala Bature, Zakariyya Abdullahi Gabralla, Lubna A. Chiroma, Haruna |
author_facet | Abdullahi, Sunusi Bala Bature, Zakariyya Abdullahi Gabralla, Lubna A. Chiroma, Haruna |
author_sort | Abdullahi, Sunusi Bala |
collection | PubMed |
description | Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/or automatic lie features during court trials and police interrogations. Unfortunately, due to the cognitive complexity and the lack of involuntary cues related to lying features, the performances of these approaches suffer and their generalization ability is limited. To improve performance, this study proposed state transition patterns based on hands, body motions, and eye blinking features from real-life court trial videos. Each video frame is represented according to a computed threshold value among neighboring pixels to extract spatial–temporal state transition patterns (STSTP) of the hand and face poses as involuntary cues using fully connected convolution neural network layers optimized with the weights of ResNet-152 learning. In addition, this study computed an eye aspect ratio model to obtain eye blinking features. These features were fused together as a single multi-modal STSTP feature model. The model was built using the enhanced calculated weight of bidirectional long short-term memory. The proposed approach was evaluated by comparing its performance with current state-of-the-art methods. It was found that the proposed approach improves the performance of detecting lies. |
format | Online Article Text |
id | pubmed-10137095 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-101370952023-04-28 Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory Abdullahi, Sunusi Bala Bature, Zakariyya Abdullahi Gabralla, Lubna A. Chiroma, Haruna Brain Sci Article Recognition of lying is a more complex cognitive process than truth-telling because of the presence of involuntary cognitive cues that are useful to lie recognition. Researchers have proposed different approaches in the literature to solve the problem of lie recognition from either handcrafted and/or automatic lie features during court trials and police interrogations. Unfortunately, due to the cognitive complexity and the lack of involuntary cues related to lying features, the performances of these approaches suffer and their generalization ability is limited. To improve performance, this study proposed state transition patterns based on hands, body motions, and eye blinking features from real-life court trial videos. Each video frame is represented according to a computed threshold value among neighboring pixels to extract spatial–temporal state transition patterns (STSTP) of the hand and face poses as involuntary cues using fully connected convolution neural network layers optimized with the weights of ResNet-152 learning. In addition, this study computed an eye aspect ratio model to obtain eye blinking features. These features were fused together as a single multi-modal STSTP feature model. The model was built using the enhanced calculated weight of bidirectional long short-term memory. The proposed approach was evaluated by comparing its performance with current state-of-the-art methods. It was found that the proposed approach improves the performance of detecting lies. MDPI 2023-03-25 /pmc/articles/PMC10137095/ /pubmed/37190520 http://dx.doi.org/10.3390/brainsci13040555 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Abdullahi, Sunusi Bala Bature, Zakariyya Abdullahi Gabralla, Lubna A. Chiroma, Haruna Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title | Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title_full | Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title_fullStr | Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title_full_unstemmed | Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title_short | Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory |
title_sort | lie recognition with multi-modal spatial–temporal state transition patterns based on hybrid convolutional neural network–bidirectional long short-term memory |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10137095/ https://www.ncbi.nlm.nih.gov/pubmed/37190520 http://dx.doi.org/10.3390/brainsci13040555 |
work_keys_str_mv | AT abdullahisunusibala lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory AT baturezakariyyaabdullahi lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory AT gabrallalubnaa lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory AT chiromaharuna lierecognitionwithmultimodalspatialtemporalstatetransitionpatternsbasedonhybridconvolutionalneuralnetworkbidirectionallongshorttermmemory |