Cargando…

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and sof...

Descripción completa

Detalles Bibliográficos
Autores principales: Chen, Junyan, Xiao, Wei, Li, Xinmei, Zheng, Yang, Huang, Xuefeng, Huang, Danli, Wang, Min
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9654298/
https://www.ncbi.nlm.nih.gov/pubmed/36365836
http://dx.doi.org/10.3390/s22218139
_version_ 1784828895714869248
author Chen, Junyan
Xiao, Wei
Li, Xinmei
Zheng, Yang
Huang, Xuefeng
Huang, Danli
Wang, Min
author_facet Chen, Junyan
Xiao, Wei
Li, Xinmei
Zheng, Yang
Huang, Xuefeng
Huang, Danli
Wang, Min
author_sort Chen, Junyan
collection PubMed
description Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities.
format Online
Article
Text
id pubmed-9654298
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96542982022-11-15 A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning Chen, Junyan Xiao, Wei Li, Xinmei Zheng, Yang Huang, Xuefeng Huang, Danli Wang, Min Sensors (Basel) Article Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities. MDPI 2022-10-24 /pmc/articles/PMC9654298/ /pubmed/36365836 http://dx.doi.org/10.3390/s22218139 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Chen, Junyan
Xiao, Wei
Li, Xinmei
Zheng, Yang
Huang, Xuefeng
Huang, Danli
Wang, Min
A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title_full A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title_fullStr A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title_full_unstemmed A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title_short A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
title_sort routing optimization method for software-defined optical transport networks based on ensembles and reinforcement learning
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9654298/
https://www.ncbi.nlm.nih.gov/pubmed/36365836
http://dx.doi.org/10.3390/s22218139
work_keys_str_mv AT chenjunyan aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT xiaowei aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT lixinmei aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT zhengyang aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT huangxuefeng aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT huangdanli aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT wangmin aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT chenjunyan routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT xiaowei routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT lixinmei routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT zhengyang routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT huangxuefeng routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT huangdanli routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning
AT wangmin routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning