Cargando…

An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms

Despite the increasing applications, demands, and capabilities of drones, in practice they have only limited autonomy for accomplishing complex missions, resulting in slow and vulnerable operations and difficulty adapting to dynamic environments. To lessen these weaknesses, we present a computationa...

Descripción completa

Detalles Bibliográficos
Autores principales: Kim, Keum Joo, Santos, Eugene, Nguyen, Hien, Pieper, Shawn
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601803/
https://www.ncbi.nlm.nih.gov/pubmed/37420384
http://dx.doi.org/10.3390/e24101364
_version_ 1784817155437494272
author Kim, Keum Joo
Santos, Eugene
Nguyen, Hien
Pieper, Shawn
author_facet Kim, Keum Joo
Santos, Eugene
Nguyen, Hien
Pieper, Shawn
author_sort Kim, Keum Joo
collection PubMed
description Despite the increasing applications, demands, and capabilities of drones, in practice they have only limited autonomy for accomplishing complex missions, resulting in slow and vulnerable operations and difficulty adapting to dynamic environments. To lessen these weaknesses, we present a computational framework for deducing the original intent of drone swarms by monitoring their movements. We focus on interference, a phenomenon that is not initially anticipated by drones but results in complicated operations due to its significant impact on performance and its challenging nature. We infer interference from predictability by first applying various machine learning methods, including deep learning, and then computing entropy to compare against interference. Our computational framework begins by building a set of computational models called double transition models from the drone movements and revealing reward distributions using inverse reinforcement learning. These reward distributions are then used to compute the entropy and interference across a variety of drone scenarios specified by combining multiple combat strategies and command styles. Our analysis confirmed that drone scenarios experienced more interference, higher performance, and higher entropy as they became more heterogeneous. However, the direction of interference (positive vs. negative) was more dependent on combinations of combat strategies and command styles than homogeneity.
format Online
Article
Text
id pubmed-9601803
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96018032022-10-27 An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms Kim, Keum Joo Santos, Eugene Nguyen, Hien Pieper, Shawn Entropy (Basel) Article Despite the increasing applications, demands, and capabilities of drones, in practice they have only limited autonomy for accomplishing complex missions, resulting in slow and vulnerable operations and difficulty adapting to dynamic environments. To lessen these weaknesses, we present a computational framework for deducing the original intent of drone swarms by monitoring their movements. We focus on interference, a phenomenon that is not initially anticipated by drones but results in complicated operations due to its significant impact on performance and its challenging nature. We infer interference from predictability by first applying various machine learning methods, including deep learning, and then computing entropy to compare against interference. Our computational framework begins by building a set of computational models called double transition models from the drone movements and revealing reward distributions using inverse reinforcement learning. These reward distributions are then used to compute the entropy and interference across a variety of drone scenarios specified by combining multiple combat strategies and command styles. Our analysis confirmed that drone scenarios experienced more interference, higher performance, and higher entropy as they became more heterogeneous. However, the direction of interference (positive vs. negative) was more dependent on combinations of combat strategies and command styles than homogeneity. MDPI 2022-09-27 /pmc/articles/PMC9601803/ /pubmed/37420384 http://dx.doi.org/10.3390/e24101364 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Kim, Keum Joo
Santos, Eugene
Nguyen, Hien
Pieper, Shawn
An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title_full An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title_fullStr An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title_full_unstemmed An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title_short An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms
title_sort application of inverse reinforcement learning to estimate interference in drone swarms
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601803/
https://www.ncbi.nlm.nih.gov/pubmed/37420384
http://dx.doi.org/10.3390/e24101364
work_keys_str_mv AT kimkeumjoo anapplicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT santoseugene anapplicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT nguyenhien anapplicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT piepershawn anapplicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT kimkeumjoo applicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT santoseugene applicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT nguyenhien applicationofinversereinforcementlearningtoestimateinterferenceindroneswarms
AT piepershawn applicationofinversereinforcementlearningtoestimateinterferenceindroneswarms