Cargando…
A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and sof...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9654298/ https://www.ncbi.nlm.nih.gov/pubmed/36365836 http://dx.doi.org/10.3390/s22218139 |
_version_ | 1784828895714869248 |
---|---|
author | Chen, Junyan Xiao, Wei Li, Xinmei Zheng, Yang Huang, Xuefeng Huang, Danli Wang, Min |
author_facet | Chen, Junyan Xiao, Wei Li, Xinmei Zheng, Yang Huang, Xuefeng Huang, Danli Wang, Min |
author_sort | Chen, Junyan |
collection | PubMed |
description | Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities. |
format | Online Article Text |
id | pubmed-9654298 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-96542982022-11-15 A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning Chen, Junyan Xiao, Wei Li, Xinmei Zheng, Yang Huang, Xuefeng Huang, Danli Wang, Min Sensors (Basel) Article Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities. MDPI 2022-10-24 /pmc/articles/PMC9654298/ /pubmed/36365836 http://dx.doi.org/10.3390/s22218139 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Chen, Junyan Xiao, Wei Li, Xinmei Zheng, Yang Huang, Xuefeng Huang, Danli Wang, Min A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title_full | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title_fullStr | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title_full_unstemmed | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title_short | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
title_sort | routing optimization method for software-defined optical transport networks based on ensembles and reinforcement learning |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9654298/ https://www.ncbi.nlm.nih.gov/pubmed/36365836 http://dx.doi.org/10.3390/s22218139 |
work_keys_str_mv | AT chenjunyan aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xiaowei aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT lixinmei aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT zhengyang aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT huangxuefeng aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT huangdanli aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT wangmin aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT chenjunyan routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xiaowei routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT lixinmei routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT zhengyang routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT huangxuefeng routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT huangdanli routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT wangmin routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning |