Cargando…

Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget

Trajectory planning plays a crucial role in ensuring the safe navigation of ships, as it involves complex decision making influenced by various factors. This paper presents a heuristic algorithm, named the Markov decision process Heuristic Algorithm (MHA), for time-optimized avoidance of Unmanned Su...

Descripción completa

Detalles Bibliográficos
Autores principales: Ding, Yi, Zhu, Hongyang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10535518/
https://www.ncbi.nlm.nih.gov/pubmed/37765903
http://dx.doi.org/10.3390/s23187846
_version_ 1785112648783757312
author Ding, Yi
Zhu, Hongyang
author_facet Ding, Yi
Zhu, Hongyang
author_sort Ding, Yi
collection PubMed
description Trajectory planning plays a crucial role in ensuring the safe navigation of ships, as it involves complex decision making influenced by various factors. This paper presents a heuristic algorithm, named the Markov decision process Heuristic Algorithm (MHA), for time-optimized avoidance of Unmanned Surface Vehicles (USVs) based on a Risk-Sensitive Markov decision process model. The proposed method utilizes the Risk-Sensitive Markov decision process model to generate a set of states within the USV collision avoidance search space. These states are determined based on the reachable locations and directions considering the time cost associated with the set of actions. By incorporating an enhanced reward function and a constraint time-dependent cost function, the USV can effectively plan practical motion paths that align with its actual time constraints. Experimental results demonstrate that the MHA algorithm enables decision makers to evaluate the trade-off between the budget and the probability of achieving the goal within the given budget. Moreover, the local stochastic optimization criterion assists the agent in selecting collision avoidance paths without significantly increasing the risk of collision.
format Online
Article
Text
id pubmed-10535518
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-105355182023-09-29 Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget Ding, Yi Zhu, Hongyang Sensors (Basel) Article Trajectory planning plays a crucial role in ensuring the safe navigation of ships, as it involves complex decision making influenced by various factors. This paper presents a heuristic algorithm, named the Markov decision process Heuristic Algorithm (MHA), for time-optimized avoidance of Unmanned Surface Vehicles (USVs) based on a Risk-Sensitive Markov decision process model. The proposed method utilizes the Risk-Sensitive Markov decision process model to generate a set of states within the USV collision avoidance search space. These states are determined based on the reachable locations and directions considering the time cost associated with the set of actions. By incorporating an enhanced reward function and a constraint time-dependent cost function, the USV can effectively plan practical motion paths that align with its actual time constraints. Experimental results demonstrate that the MHA algorithm enables decision makers to evaluate the trade-off between the budget and the probability of achieving the goal within the given budget. Moreover, the local stochastic optimization criterion assists the agent in selecting collision avoidance paths without significantly increasing the risk of collision. MDPI 2023-09-13 /pmc/articles/PMC10535518/ /pubmed/37765903 http://dx.doi.org/10.3390/s23187846 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Ding, Yi
Zhu, Hongyang
Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title_full Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title_fullStr Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title_full_unstemmed Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title_short Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget
title_sort risk-sensitive markov decision processes of usv trajectory planning with time-limited budget
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10535518/
https://www.ncbi.nlm.nih.gov/pubmed/37765903
http://dx.doi.org/10.3390/s23187846
work_keys_str_mv AT dingyi risksensitivemarkovdecisionprocessesofusvtrajectoryplanningwithtimelimitedbudget
AT zhuhongyang risksensitivemarkovdecisionprocessesofusvtrajectoryplanningwithtimelimitedbudget