Cargando…

Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot

In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong respon...

Descripción completa

Detalles Bibliográficos
Autores principales: Kuo, Ping-Huan, Lin, Ssu-Ting, Hu, Jun, Huang, Chiou-Jye
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8347869/
https://www.ncbi.nlm.nih.gov/pubmed/34372368
http://dx.doi.org/10.3390/s21155132
_version_ 1783735199091130368
author Kuo, Ping-Huan
Lin, Ssu-Ting
Hu, Jun
Huang, Chiou-Jye
author_facet Kuo, Ping-Huan
Lin, Ssu-Ting
Hu, Jun
Huang, Chiou-Jye
author_sort Kuo, Ping-Huan
collection PubMed
description In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment.
format Online
Article
Text
id pubmed-8347869
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-83478692021-08-08 Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot Kuo, Ping-Huan Lin, Ssu-Ting Hu, Jun Huang, Chiou-Jye Sensors (Basel) Article In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment. MDPI 2021-07-29 /pmc/articles/PMC8347869/ /pubmed/34372368 http://dx.doi.org/10.3390/s21155132 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Kuo, Ping-Huan
Lin, Ssu-Ting
Hu, Jun
Huang, Chiou-Jye
Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title_full Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title_fullStr Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title_full_unstemmed Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title_short Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
title_sort multi-sensor context-aware based chatbot model: an application of humanoid companion robot
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8347869/
https://www.ncbi.nlm.nih.gov/pubmed/34372368
http://dx.doi.org/10.3390/s21155132
work_keys_str_mv AT kuopinghuan multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot
AT linssuting multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot
AT hujun multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot
AT huangchioujye multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot