Cargando…

A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning

Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emot...

Descripción completa

Detalles Bibliográficos
Autores principales: Chen, Xiang, Huang, Rubing, Li, Xin, Xiao, Lei, Zhou, Ming, Zhang, Linghao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8093774/
https://www.ncbi.nlm.nih.gov/pubmed/33959083
http://dx.doi.org/10.3389/fpsyg.2021.674853
_version_ 1783687885460865024
author Chen, Xiang
Huang, Rubing
Li, Xin
Xiao, Lei
Zhou, Ming
Zhang, Linghao
author_facet Chen, Xiang
Huang, Rubing
Li, Xin
Xiao, Lei
Zhou, Ming
Zhang, Linghao
author_sort Chen, Xiang
collection PubMed
description Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient (MFCC) of the speech signal is used as the input of the improved long and short-term memory network (ILSTM). To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed.
format Online
Article
Text
id pubmed-8093774
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-80937742021-05-05 A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning Chen, Xiang Huang, Rubing Li, Xin Xiao, Lei Zhou, Ming Zhang, Linghao Front Psychol Psychology Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient (MFCC) of the speech signal is used as the input of the improved long and short-term memory network (ILSTM). To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed. Frontiers Media S.A. 2021-04-20 /pmc/articles/PMC8093774/ /pubmed/33959083 http://dx.doi.org/10.3389/fpsyg.2021.674853 Text en Copyright © 2021 Chen, Huang, Li, Xiao, Zhou and Zhang. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Psychology
Chen, Xiang
Huang, Rubing
Li, Xin
Xiao, Lei
Zhou, Ming
Zhang, Linghao
A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title_full A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title_fullStr A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title_full_unstemmed A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title_short A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning
title_sort novel user emotional interaction design model using long and short-term memory networks and deep learning
topic Psychology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8093774/
https://www.ncbi.nlm.nih.gov/pubmed/33959083
http://dx.doi.org/10.3389/fpsyg.2021.674853
work_keys_str_mv AT chenxiang anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT huangrubing anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT lixin anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT xiaolei anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT zhouming anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT zhanglinghao anoveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT chenxiang noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT huangrubing noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT lixin noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT xiaolei noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT zhouming noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning
AT zhanglinghao noveluseremotionalinteractiondesignmodelusinglongandshorttermmemorynetworksanddeeplearning