Cargando…
A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving
In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment c...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9539977/ https://www.ncbi.nlm.nih.gov/pubmed/36213146 http://dx.doi.org/10.3389/fnbot.2022.1022887 |
_version_ | 1784803609754468352 |
---|---|
author | Zhu, Qingyi Tan, Mingtao |
author_facet | Zhu, Qingyi Tan, Mingtao |
author_sort | Zhu, Qingyi |
collection | PubMed |
description | In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment classification experiment results demonstrate that the three RNN models using the NAF achieve better accuracy and lower loss values compared with other commonly used activation functions (AF), such as ReLU, SELU etc. Moreover, in terms of dynamic problems solving, a fixed-time convergent recurrent neural network (FTCRNN) model with the NAF is constructed. Additionally, the fixed-time convergence property of the FTCRNN model is strictly validated and the upper bound convergence time formula of the FTCRNN model is obtained. Furthermore, the numerical simulation results of dynamic Sylvester equation (DSE) solving using the FTCRNN model indicate that the neural state solutions of the FTCRNN model quickly converge to the theoretical solutions of DSE problems whether there are noises or not. Ultimately, the FTCRNN model is also utilized to realize trajectory tracking of robot manipulator and electric circuit currents computation for the further validation of its accurateness and robustness, and the corresponding results further validate its superior performance and widespread applicability. |
format | Online Article Text |
id | pubmed-9539977 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-95399772022-10-08 A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving Zhu, Qingyi Tan, Mingtao Front Neurorobot Neuroscience In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment classification experiment results demonstrate that the three RNN models using the NAF achieve better accuracy and lower loss values compared with other commonly used activation functions (AF), such as ReLU, SELU etc. Moreover, in terms of dynamic problems solving, a fixed-time convergent recurrent neural network (FTCRNN) model with the NAF is constructed. Additionally, the fixed-time convergence property of the FTCRNN model is strictly validated and the upper bound convergence time formula of the FTCRNN model is obtained. Furthermore, the numerical simulation results of dynamic Sylvester equation (DSE) solving using the FTCRNN model indicate that the neural state solutions of the FTCRNN model quickly converge to the theoretical solutions of DSE problems whether there are noises or not. Ultimately, the FTCRNN model is also utilized to realize trajectory tracking of robot manipulator and electric circuit currents computation for the further validation of its accurateness and robustness, and the corresponding results further validate its superior performance and widespread applicability. Frontiers Media S.A. 2022-09-23 /pmc/articles/PMC9539977/ /pubmed/36213146 http://dx.doi.org/10.3389/fnbot.2022.1022887 Text en Copyright © 2022 Zhu and Tan. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Zhu, Qingyi Tan, Mingtao A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title | A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title_full | A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title_fullStr | A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title_full_unstemmed | A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title_short | A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
title_sort | novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9539977/ https://www.ncbi.nlm.nih.gov/pubmed/36213146 http://dx.doi.org/10.3389/fnbot.2022.1022887 |
work_keys_str_mv | AT zhuqingyi anovelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving AT tanmingtao anovelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving AT zhuqingyi novelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving AT tanmingtao novelactivationfunctionbasedrecurrentneuralnetworksandtheirapplicationsonsentimentclassificationanddynamicproblemssolving |