Cargando…
Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization
To solve the problems of path planning and dynamic obstacle avoidance for an unmanned surface vehicle (USV) in a locally observable non-dynamic ocean environment, a visual perception and decision-making method based on deep reinforcement learning is proposed. This method replaces the full connection...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10650858/ https://www.ncbi.nlm.nih.gov/pubmed/37960565 http://dx.doi.org/10.3390/s23218864 |
_version_ | 1785135877928779776 |
---|---|
author | Sun, Pengqi Yang, Chunxi Zhou, Xiaojie Wang, Wenbo |
author_facet | Sun, Pengqi Yang, Chunxi Zhou, Xiaojie Wang, Wenbo |
author_sort | Sun, Pengqi |
collection | PubMed |
description | To solve the problems of path planning and dynamic obstacle avoidance for an unmanned surface vehicle (USV) in a locally observable non-dynamic ocean environment, a visual perception and decision-making method based on deep reinforcement learning is proposed. This method replaces the full connection layer in the Proximal Policy Optimization (PPO) neural network structure with a convolutional neural network (CNN). In this way, the degree of memorization and forgetting of sample information is controlled. Moreover, this method accumulates reward models faster by preferentially learning samples with high reward values. From the USV-centered radar perception input of the local environment, the output of the action is realized through an end-to-end learning model, and the environment perception and decision are formed as a closed loop. Thus, the proposed algorithm has good adaptability in different marine environments. The simulation results show that, compared with the PPO algorithm, Soft Actor–Critic (SAC) algorithm, and Deep Q Network (DQN) algorithm, the proposed algorithm can accelerate the model convergence speed and improve the path planning performances in partly or fully unknown ocean fields. |
format | Online Article Text |
id | pubmed-10650858 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-106508582023-10-31 Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization Sun, Pengqi Yang, Chunxi Zhou, Xiaojie Wang, Wenbo Sensors (Basel) Article To solve the problems of path planning and dynamic obstacle avoidance for an unmanned surface vehicle (USV) in a locally observable non-dynamic ocean environment, a visual perception and decision-making method based on deep reinforcement learning is proposed. This method replaces the full connection layer in the Proximal Policy Optimization (PPO) neural network structure with a convolutional neural network (CNN). In this way, the degree of memorization and forgetting of sample information is controlled. Moreover, this method accumulates reward models faster by preferentially learning samples with high reward values. From the USV-centered radar perception input of the local environment, the output of the action is realized through an end-to-end learning model, and the environment perception and decision are formed as a closed loop. Thus, the proposed algorithm has good adaptability in different marine environments. The simulation results show that, compared with the PPO algorithm, Soft Actor–Critic (SAC) algorithm, and Deep Q Network (DQN) algorithm, the proposed algorithm can accelerate the model convergence speed and improve the path planning performances in partly or fully unknown ocean fields. MDPI 2023-10-31 /pmc/articles/PMC10650858/ /pubmed/37960565 http://dx.doi.org/10.3390/s23218864 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Sun, Pengqi Yang, Chunxi Zhou, Xiaojie Wang, Wenbo Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title | Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title_full | Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title_fullStr | Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title_full_unstemmed | Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title_short | Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization |
title_sort | path planning for unmanned surface vehicles with strong generalization ability based on improved proximal policy optimization |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10650858/ https://www.ncbi.nlm.nih.gov/pubmed/37960565 http://dx.doi.org/10.3390/s23218864 |
work_keys_str_mv | AT sunpengqi pathplanningforunmannedsurfacevehicleswithstronggeneralizationabilitybasedonimprovedproximalpolicyoptimization AT yangchunxi pathplanningforunmannedsurfacevehicleswithstronggeneralizationabilitybasedonimprovedproximalpolicyoptimization AT zhouxiaojie pathplanningforunmannedsurfacevehicleswithstronggeneralizationabilitybasedonimprovedproximalpolicyoptimization AT wangwenbo pathplanningforunmannedsurfacevehicleswithstronggeneralizationabilitybasedonimprovedproximalpolicyoptimization |