Cargando…

Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units

Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall co...

Descripción completa

Detalles Bibliográficos
Autores principales: Irasiak, Anna, Kozak, Jan, Piasecki, Adam, Stęclik, Tomasz
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9857566/
https://www.ncbi.nlm.nih.gov/pubmed/36673261
http://dx.doi.org/10.3390/e25010120
_version_ 1784873897339912192
author Irasiak, Anna
Kozak, Jan
Piasecki, Adam
Stęclik, Tomasz
author_facet Irasiak, Anna
Kozak, Jan
Piasecki, Adam
Stęclik, Tomasz
author_sort Irasiak, Anna
collection PubMed
description Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation—this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation by experts related to Polish Sign Language, and the results obtained allow for the development of further work related to automatic translation into Polish Sign Language.
format Online
Article
Text
id pubmed-9857566
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-98575662023-01-21 Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units Irasiak, Anna Kozak, Jan Piasecki, Adam Stęclik, Tomasz Entropy (Basel) Article Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation—this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation by experts related to Polish Sign Language, and the results obtained allow for the development of further work related to automatic translation into Polish Sign Language. MDPI 2023-01-06 /pmc/articles/PMC9857566/ /pubmed/36673261 http://dx.doi.org/10.3390/e25010120 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Irasiak, Anna
Kozak, Jan
Piasecki, Adam
Stęclik, Tomasz
Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title_full Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title_fullStr Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title_full_unstemmed Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title_short Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units
title_sort processing real-life recordings of facial expressions of polish sign language using action units
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9857566/
https://www.ncbi.nlm.nih.gov/pubmed/36673261
http://dx.doi.org/10.3390/e25010120
work_keys_str_mv AT irasiakanna processingrealliferecordingsoffacialexpressionsofpolishsignlanguageusingactionunits
AT kozakjan processingrealliferecordingsoffacialexpressionsofpolishsignlanguageusingactionunits
AT piaseckiadam processingrealliferecordingsoffacialexpressionsofpolishsignlanguageusingactionunits
AT stecliktomasz processingrealliferecordingsoffacialexpressionsofpolishsignlanguageusingactionunits