Cargando…
Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries
Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9670153/ https://www.ncbi.nlm.nih.gov/pubmed/36405116 http://dx.doi.org/10.3389/fpsyg.2022.988302 |
_version_ | 1784832279819845632 |
---|---|
author | Inagaki, Mikio Ito, Tatsuro Shinozaki, Takashi Fujita, Ichiro |
author_facet | Inagaki, Mikio Ito, Tatsuro Shinozaki, Takashi Fujita, Ichiro |
author_sort | Inagaki, Mikio |
collection | PubMed |
description | Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks (CNNs) that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each developed in a different country (Sweden and Japan), in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units (AUs) of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures. |
format | Online Article Text |
id | pubmed-9670153 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-96701532022-11-18 Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries Inagaki, Mikio Ito, Tatsuro Shinozaki, Takashi Fujita, Ichiro Front Psychol Psychology Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks (CNNs) that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each developed in a different country (Sweden and Japan), in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units (AUs) of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures. Frontiers Media S.A. 2022-11-03 /pmc/articles/PMC9670153/ /pubmed/36405116 http://dx.doi.org/10.3389/fpsyg.2022.988302 Text en Copyright © 2022 Inagaki, Ito, Shinozaki and Fujita. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Psychology Inagaki, Mikio Ito, Tatsuro Shinozaki, Takashi Fujita, Ichiro Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title | Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title_full | Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title_fullStr | Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title_full_unstemmed | Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title_short | Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
title_sort | convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries |
topic | Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9670153/ https://www.ncbi.nlm.nih.gov/pubmed/36405116 http://dx.doi.org/10.3389/fpsyg.2022.988302 |
work_keys_str_mv | AT inagakimikio convolutionalneuralnetworksrevealdifferencesinactionunitsoffacialexpressionsbetweenfaceimagedatabasesdevelopedindifferentcountries AT itotatsuro convolutionalneuralnetworksrevealdifferencesinactionunitsoffacialexpressionsbetweenfaceimagedatabasesdevelopedindifferentcountries AT shinozakitakashi convolutionalneuralnetworksrevealdifferencesinactionunitsoffacialexpressionsbetweenfaceimagedatabasesdevelopedindifferentcountries AT fujitaichiro convolutionalneuralnetworksrevealdifferencesinactionunitsoffacialexpressionsbetweenfaceimagedatabasesdevelopedindifferentcountries |