Cargando…
Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong respon...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8347869/ https://www.ncbi.nlm.nih.gov/pubmed/34372368 http://dx.doi.org/10.3390/s21155132 |
_version_ | 1783735199091130368 |
---|---|
author | Kuo, Ping-Huan Lin, Ssu-Ting Hu, Jun Huang, Chiou-Jye |
author_facet | Kuo, Ping-Huan Lin, Ssu-Ting Hu, Jun Huang, Chiou-Jye |
author_sort | Kuo, Ping-Huan |
collection | PubMed |
description | In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment. |
format | Online Article Text |
id | pubmed-8347869 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-83478692021-08-08 Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot Kuo, Ping-Huan Lin, Ssu-Ting Hu, Jun Huang, Chiou-Jye Sensors (Basel) Article In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment. MDPI 2021-07-29 /pmc/articles/PMC8347869/ /pubmed/34372368 http://dx.doi.org/10.3390/s21155132 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Kuo, Ping-Huan Lin, Ssu-Ting Hu, Jun Huang, Chiou-Jye Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title_full | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title_fullStr | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title_full_unstemmed | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title_short | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
title_sort | multi-sensor context-aware based chatbot model: an application of humanoid companion robot |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8347869/ https://www.ncbi.nlm.nih.gov/pubmed/34372368 http://dx.doi.org/10.3390/s21155132 |
work_keys_str_mv | AT kuopinghuan multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT linssuting multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT hujun multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT huangchioujye multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot |