Cargando…

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The propo...

Descripción completa

Detalles Bibliográficos
Autores principales: Arranz, Raúl, Carramiñana, David, de Miguel, Gonzalo, Besada, Juan A., Bernardos, Ana M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10648592/
https://www.ncbi.nlm.nih.gov/pubmed/37960466
http://dx.doi.org/10.3390/s23218766
_version_ 1785135374084866048
author Arranz, Raúl
Carramiñana, David
de Miguel, Gonzalo
Besada, Juan A.
Bernardos, Ana M.
author_facet Arranz, Raúl
Carramiñana, David
de Miguel, Gonzalo
Besada, Juan A.
Bernardos, Ana M.
author_sort Arranz, Raúl
collection PubMed
description This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently.
format Online
Article
Text
id pubmed-10648592
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106485922023-10-27 Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance Arranz, Raúl Carramiñana, David de Miguel, Gonzalo Besada, Juan A. Bernardos, Ana M. Sensors (Basel) Article This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently. MDPI 2023-10-27 /pmc/articles/PMC10648592/ /pubmed/37960466 http://dx.doi.org/10.3390/s23218766 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Arranz, Raúl
Carramiñana, David
de Miguel, Gonzalo
Besada, Juan A.
Bernardos, Ana M.
Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title_full Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title_fullStr Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title_full_unstemmed Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title_short Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance
title_sort application of deep reinforcement learning to uav swarming for ground surveillance
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10648592/
https://www.ncbi.nlm.nih.gov/pubmed/37960466
http://dx.doi.org/10.3390/s23218766
work_keys_str_mv AT arranzraul applicationofdeepreinforcementlearningtouavswarmingforgroundsurveillance
AT carraminanadavid applicationofdeepreinforcementlearningtouavswarmingforgroundsurveillance
AT demiguelgonzalo applicationofdeepreinforcementlearningtouavswarmingforgroundsurveillance
AT besadajuana applicationofdeepreinforcementlearningtouavswarmingforgroundsurveillance
AT bernardosanam applicationofdeepreinforcementlearningtouavswarmingforgroundsurveillance