Cargando…

A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving

In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment c...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhu, Qingyi, Tan, Mingtao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9539977/
https://www.ncbi.nlm.nih.gov/pubmed/36213146
http://dx.doi.org/10.3389/fnbot.2022.1022887
_version_ 1784803609754468352
author Zhu, Qingyi
Tan, Mingtao
author_facet Zhu, Qingyi
Tan, Mingtao
author_sort Zhu, Qingyi
collection PubMed
description In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment classification experiment results demonstrate that the three RNN models using the NAF achieve better accuracy and lower loss values compared with other commonly used activation functions (AF), such as ReLU, SELU etc. Moreover, in terms of dynamic problems solving, a fixed-time convergent recurrent neural network (FTCRNN) model with the NAF is constructed. Additionally, the fixed-time convergence property of the FTCRNN model is strictly validated and the upper bound convergence time formula of the FTCRNN model is obtained. Furthermore, the numerical simulation results of dynamic Sylvester equation (DSE) solving using the FTCRNN model indicate that the neural state solutions of the FTCRNN model quickly converge to the theoretical solutions of DSE problems whether there are noises or not. Ultimately, the FTCRNN model is also utilized to realize trajectory tracking of robot manipulator and electric circuit currents computation for the further validation of its accurateness and robustness, and the corresponding results further validate its superior performance and widespread applicability.
format Online
Article
Text
id pubmed-9539977
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-95399772022-10-08 A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving Zhu, Qingyi Tan, Mingtao Front Neurorobot Neuroscience In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment classification experiment results demonstrate that the three RNN models using the NAF achieve better accuracy and lower loss values compared with other commonly used activation functions (AF), such as ReLU, SELU etc. Moreover, in terms of dynamic problems solving, a fixed-time convergent recurrent neural network (FTCRNN) model with the NAF is constructed. Additionally, the fixed-time convergence property of the FTCRNN model is strictly validated and the upper bound convergence time formula of the FTCRNN model is obtained. Furthermore, the numerical simulation results of dynamic Sylvester equation (DSE) solving using the FTCRNN model indicate that the neural state solutions of the FTCRNN model quickly converge to the theoretical solutions of DSE problems whether there are noises or not. Ultimately, the FTCRNN model is also utilized to realize trajectory tracking of robot manipulator and electric circuit currents computation for the further validation of its accurateness and robustness, and the corresponding results further validate its superior performance and widespread applicability. Frontiers Media S.A. 2022-09-23 /pmc/articles/PMC9539977/ /pubmed/36213146 http://dx.doi.org/10.3389/fnbot.2022.1022887 Text en Copyright © 2022 Zhu and Tan. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Zhu, Qingyi
Tan, Mingtao
A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title_full A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title_fullStr A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title_full_unstemmed A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title_short A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
title_sort novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9539977/
https://www.ncbi.nlm.nih.gov/pubmed/36213146
http://dx.doi.org/10.3389/fnbot.2022.1022887
work_keys_str_mv AT zhuqingyi anovelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving
AT tanmingtao anovelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving
AT zhuqingyi novelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving
AT tanmingtao novelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving