Cargando…

Exploring facial expressions and action unit domains for Parkinson detection

BACKGROUND AND OBJECTIVE: Patients suffering from Parkinson’s disease (PD) present a reduction in facial movements called hypomimia. In this work, we propose to use machine learning facial expression analysis from face images based on action unit domains to improve PD detection. We propose different...

Descripción completa

Detalles Bibliográficos
Autores principales: Gomez, Luis F., Morales, Aythami, Fierrez, Julian, Orozco-Arroyave, Juan Rafael
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9894465/
https://www.ncbi.nlm.nih.gov/pubmed/36730168
http://dx.doi.org/10.1371/journal.pone.0281248
_version_ 1784881746310856704
author Gomez, Luis F.
Morales, Aythami
Fierrez, Julian
Orozco-Arroyave, Juan Rafael
author_facet Gomez, Luis F.
Morales, Aythami
Fierrez, Julian
Orozco-Arroyave, Juan Rafael
author_sort Gomez, Luis F.
collection PubMed
description BACKGROUND AND OBJECTIVE: Patients suffering from Parkinson’s disease (PD) present a reduction in facial movements called hypomimia. In this work, we propose to use machine learning facial expression analysis from face images based on action unit domains to improve PD detection. We propose different domain adaptation techniques to exploit the latest advances in automatic face analysis and face action unit detection. METHODS: Three different approaches are explored to model facial expressions of PD patients: (i) face analysis using single frame images and also using sequences of images, (ii) transfer learning from face analysis to action units recognition, and (iii) triplet-loss functions to improve the automatic classification between patients and healthy subjects. RESULTS: Real face images from PD patients show that it is possible to properly model elicited facial expressions using image sequences (neutral, onset-transition, apex, offset-transition, and neutral) with accuracy improvements of up to 5.5% (from 72.9% to 78.4%) with respect to single-image PD detection. We also show that our proposed action unit domain adaptation provides improvements of up to 8.9% (from 78.4% to 87.3%) with respect to face analysis. Finally, we also show that triplet-loss functions provide improvements of up to 3.6% (from 78.8% to 82.4%) with respect to action unit domain adaptation applied upon models created from scratch. The code of the experiments is available at https://github.com/luisf-gomez/Explorer-FE-AU-in-PD. CONCLUSIONS: Domain adaptation via transfer learning methods seem to be a promising strategy to model hypomimia in PD patients. Considering the good results and also the fact that only up to five images per participant are considered in each sequence, we believe that this work is a step forward in the development of inexpensive computational systems suitable to model and quantify problems of PD patients in their facial expressions.
format Online
Article
Text
id pubmed-9894465
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-98944652023-02-03 Exploring facial expressions and action unit domains for Parkinson detection Gomez, Luis F. Morales, Aythami Fierrez, Julian Orozco-Arroyave, Juan Rafael PLoS One Research Article BACKGROUND AND OBJECTIVE: Patients suffering from Parkinson’s disease (PD) present a reduction in facial movements called hypomimia. In this work, we propose to use machine learning facial expression analysis from face images based on action unit domains to improve PD detection. We propose different domain adaptation techniques to exploit the latest advances in automatic face analysis and face action unit detection. METHODS: Three different approaches are explored to model facial expressions of PD patients: (i) face analysis using single frame images and also using sequences of images, (ii) transfer learning from face analysis to action units recognition, and (iii) triplet-loss functions to improve the automatic classification between patients and healthy subjects. RESULTS: Real face images from PD patients show that it is possible to properly model elicited facial expressions using image sequences (neutral, onset-transition, apex, offset-transition, and neutral) with accuracy improvements of up to 5.5% (from 72.9% to 78.4%) with respect to single-image PD detection. We also show that our proposed action unit domain adaptation provides improvements of up to 8.9% (from 78.4% to 87.3%) with respect to face analysis. Finally, we also show that triplet-loss functions provide improvements of up to 3.6% (from 78.8% to 82.4%) with respect to action unit domain adaptation applied upon models created from scratch. The code of the experiments is available at https://github.com/luisf-gomez/Explorer-FE-AU-in-PD. CONCLUSIONS: Domain adaptation via transfer learning methods seem to be a promising strategy to model hypomimia in PD patients. Considering the good results and also the fact that only up to five images per participant are considered in each sequence, we believe that this work is a step forward in the development of inexpensive computational systems suitable to model and quantify problems of PD patients in their facial expressions. Public Library of Science 2023-02-02 /pmc/articles/PMC9894465/ /pubmed/36730168 http://dx.doi.org/10.1371/journal.pone.0281248 Text en © 2023 Gomez et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Gomez, Luis F.
Morales, Aythami
Fierrez, Julian
Orozco-Arroyave, Juan Rafael
Exploring facial expressions and action unit domains for Parkinson detection
title Exploring facial expressions and action unit domains for Parkinson detection
title_full Exploring facial expressions and action unit domains for Parkinson detection
title_fullStr Exploring facial expressions and action unit domains for Parkinson detection
title_full_unstemmed Exploring facial expressions and action unit domains for Parkinson detection
title_short Exploring facial expressions and action unit domains for Parkinson detection
title_sort exploring facial expressions and action unit domains for parkinson detection
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9894465/
https://www.ncbi.nlm.nih.gov/pubmed/36730168
http://dx.doi.org/10.1371/journal.pone.0281248
work_keys_str_mv AT gomezluisf exploringfacialexpressionsandactionunitdomainsforparkinsondetection
AT moralesaythami exploringfacialexpressionsandactionunitdomainsforparkinsondetection
AT fierrezjulian exploringfacialexpressionsandactionunitdomainsforparkinsondetection
AT orozcoarroyavejuanrafael exploringfacialexpressionsandactionunitdomainsforparkinsondetection