Cargando…
An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning
The cooperative positioning problem of hypersonic vehicles regarding LEO constellations is the focus of this research study on space-based early warning systems. A hypersonic vehicle is highly maneuverable, and its trajectory is uncertain. New challenges are posed for the cooperative positioning cap...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9967591/ https://www.ncbi.nlm.nih.gov/pubmed/36850827 http://dx.doi.org/10.3390/s23042225 |
_version_ | 1784897303012704256 |
---|---|
author | Cheng, Yu Wei, Cheng Sun, Shengxin You, Bindi Zhao, Yang |
author_facet | Cheng, Yu Wei, Cheng Sun, Shengxin You, Bindi Zhao, Yang |
author_sort | Cheng, Yu |
collection | PubMed |
description | The cooperative positioning problem of hypersonic vehicles regarding LEO constellations is the focus of this research study on space-based early warning systems. A hypersonic vehicle is highly maneuverable, and its trajectory is uncertain. New challenges are posed for the cooperative positioning capability of the constellation. In recent years, breakthroughs in artificial intelligence technology have provided new avenues for collaborative multi-satellite intelligent autonomous decision-making technology. This paper addresses the problem of multi-satellite cooperative geometric positioning for hypersonic glide vehicles (HGVs) by the LEO-constellation-tracking system. To exploit the inherent advantages of hierarchical reinforcement learning in intelligent decision making while satisfying the constraints of cooperative observations, an autonomous intelligent decision-making algorithm for satellites that incorporates a hierarchical proximal policy optimization with random hill climbing (MAPPO-RHC) is designed. On the one hand, hierarchical decision making is used to reduce the solution space; on the other hand, it is used to maximize the global reward and to uniformly distribute satellite resources. The single-satellite local search method improves the capability of the decision-making algorithm to search the solution space based on the decision-making results of the hierarchical proximal policy-optimization algorithm, combining both random hill climbing and heuristic methods. Finally, the MAPPO-RHC algorithm’s coverage and positioning accuracy performance is simulated and analyzed in two different scenarios and compared with four intelligent satellite decision-making algorithms that have been studied in recent years. From the simulation results, the decision-making results of the MAPPO-RHC algorithm can obtain more balanced resource allocations and higher geometric positioning accuracy. Thus, it is concluded that the MAPPO-RHC algorithm provides a feasible solution for the real-time decision-making problem of the LEO constellation early warning system. |
format | Online Article Text |
id | pubmed-9967591 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-99675912023-02-27 An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning Cheng, Yu Wei, Cheng Sun, Shengxin You, Bindi Zhao, Yang Sensors (Basel) Article The cooperative positioning problem of hypersonic vehicles regarding LEO constellations is the focus of this research study on space-based early warning systems. A hypersonic vehicle is highly maneuverable, and its trajectory is uncertain. New challenges are posed for the cooperative positioning capability of the constellation. In recent years, breakthroughs in artificial intelligence technology have provided new avenues for collaborative multi-satellite intelligent autonomous decision-making technology. This paper addresses the problem of multi-satellite cooperative geometric positioning for hypersonic glide vehicles (HGVs) by the LEO-constellation-tracking system. To exploit the inherent advantages of hierarchical reinforcement learning in intelligent decision making while satisfying the constraints of cooperative observations, an autonomous intelligent decision-making algorithm for satellites that incorporates a hierarchical proximal policy optimization with random hill climbing (MAPPO-RHC) is designed. On the one hand, hierarchical decision making is used to reduce the solution space; on the other hand, it is used to maximize the global reward and to uniformly distribute satellite resources. The single-satellite local search method improves the capability of the decision-making algorithm to search the solution space based on the decision-making results of the hierarchical proximal policy-optimization algorithm, combining both random hill climbing and heuristic methods. Finally, the MAPPO-RHC algorithm’s coverage and positioning accuracy performance is simulated and analyzed in two different scenarios and compared with four intelligent satellite decision-making algorithms that have been studied in recent years. From the simulation results, the decision-making results of the MAPPO-RHC algorithm can obtain more balanced resource allocations and higher geometric positioning accuracy. Thus, it is concluded that the MAPPO-RHC algorithm provides a feasible solution for the real-time decision-making problem of the LEO constellation early warning system. MDPI 2023-02-16 /pmc/articles/PMC9967591/ /pubmed/36850827 http://dx.doi.org/10.3390/s23042225 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Cheng, Yu Wei, Cheng Sun, Shengxin You, Bindi Zhao, Yang An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title | An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title_full | An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title_fullStr | An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title_full_unstemmed | An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title_short | An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning |
title_sort | leo constellation early warning system decision-making method based on hierarchical reinforcement learning |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9967591/ https://www.ncbi.nlm.nih.gov/pubmed/36850827 http://dx.doi.org/10.3390/s23042225 |
work_keys_str_mv | AT chengyu anleoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT weicheng anleoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT sunshengxin anleoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT youbindi anleoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT zhaoyang anleoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT chengyu leoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT weicheng leoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT sunshengxin leoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT youbindi leoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning AT zhaoyang leoconstellationearlywarningsystemdecisionmakingmethodbasedonhierarchicalreinforcementlearning |