Cargando…

Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrat...

Descripción completa

Detalles Bibliográficos
Autores principales: Sun, Xiaochuan, Hao, Mingxiang, Wang, Yutong, Wang, Yu, Li, Zhigang, Li, Yingqi
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9777492/
https://www.ncbi.nlm.nih.gov/pubmed/36554114
http://dx.doi.org/10.3390/e24121709
_version_ 1784856117691547648
author Sun, Xiaochuan
Hao, Mingxiang
Wang, Yutong
Wang, Yu
Li, Zhigang
Li, Yingqi
author_facet Sun, Xiaochuan
Hao, Mingxiang
Wang, Yutong
Wang, Yu
Li, Zhigang
Li, Yingqi
author_sort Sun, Xiaochuan
collection PubMed
description An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.
format Online
Article
Text
id pubmed-9777492
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-97774922022-12-23 Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View Sun, Xiaochuan Hao, Mingxiang Wang, Yutong Wang, Yu Li, Zhigang Li, Yingqi Entropy (Basel) Article An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework. MDPI 2022-11-23 /pmc/articles/PMC9777492/ /pubmed/36554114 http://dx.doi.org/10.3390/e24121709 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Sun, Xiaochuan
Hao, Mingxiang
Wang, Yutong
Wang, Yu
Li, Zhigang
Li, Yingqi
Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title_full Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title_fullStr Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title_full_unstemmed Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title_short Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
title_sort reservoir dynamic interpretability for time series prediction: a permutation entropy view
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9777492/
https://www.ncbi.nlm.nih.gov/pubmed/36554114
http://dx.doi.org/10.3390/e24121709
work_keys_str_mv AT sunxiaochuan reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview
AT haomingxiang reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview
AT wangyutong reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview
AT wangyu reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview
AT lizhigang reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview
AT liyingqi reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview