Cargando…

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition (FER). Based on machine learning, such technology can tremendously increase the amount of processed data. FER i...

Descripción completa

Detalles Bibliográficos
Autores principales: Küntzler, Theresa, Höfling, T. Tim A., Alpers, Georg W.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8131548/
https://www.ncbi.nlm.nih.gov/pubmed/34025503
http://dx.doi.org/10.3389/fpsyg.2021.627561
_version_ 1783694719684968448
author Küntzler, Theresa
Höfling, T. Tim A.
Alpers, Georg W.
author_facet Küntzler, Theresa
Höfling, T. Tim A.
Alpers, Georg W.
author_sort Küntzler, Theresa
collection PubMed
description Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition (FER). Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems (Azure Face API, Microsoft; Face++, Megvii Technology; FaceReader, Noldus Information Technology) with human emotion recognition (A) for standardized posed facial expressions (from prototypical inventories) and (B) for non-standardized acted facial expressions (extracted from emotional movie scenes). For the standardized images, all three systems classify basic emotions accurately (FaceReader is most accurate) and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions.
format Online
Article
Text
id pubmed-8131548
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-81315482021-05-20 Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions Küntzler, Theresa Höfling, T. Tim A. Alpers, Georg W. Front Psychol Psychology Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition (FER). Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems (Azure Face API, Microsoft; Face++, Megvii Technology; FaceReader, Noldus Information Technology) with human emotion recognition (A) for standardized posed facial expressions (from prototypical inventories) and (B) for non-standardized acted facial expressions (extracted from emotional movie scenes). For the standardized images, all three systems classify basic emotions accurately (FaceReader is most accurate) and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions. Frontiers Media S.A. 2021-05-05 /pmc/articles/PMC8131548/ /pubmed/34025503 http://dx.doi.org/10.3389/fpsyg.2021.627561 Text en Copyright © 2021 Küntzler, Höfling and Alpers. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Psychology
Küntzler, Theresa
Höfling, T. Tim A.
Alpers, Georg W.
Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title_full Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title_fullStr Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title_full_unstemmed Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title_short Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
title_sort automatic facial expression recognition in standardized and non-standardized emotional expressions
topic Psychology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8131548/
https://www.ncbi.nlm.nih.gov/pubmed/34025503
http://dx.doi.org/10.3389/fpsyg.2021.627561
work_keys_str_mv AT kuntzlertheresa automaticfacialexpressionrecognitioninstandardizedandnonstandardizedemotionalexpressions
AT hoflingttima automaticfacialexpressionrecognitioninstandardizedandnonstandardizedemotionalexpressions
AT alpersgeorgw automaticfacialexpressionrecognitioninstandardizedandnonstandardizedemotionalexpressions