Cargando…

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate t...

Descripción completa

Detalles Bibliográficos
Autores principales: Vakil, Asad, Blasch, Erik, Ewing, Robert, Li, Jia
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9919369/
https://www.ncbi.nlm.nih.gov/pubmed/36772527
http://dx.doi.org/10.3390/s23031489
_version_ 1784886807680253952
author Vakil, Asad
Blasch, Erik
Ewing, Robert
Li, Jia
author_facet Vakil, Asad
Blasch, Erik
Ewing, Robert
Li, Jia
author_sort Vakil, Asad
collection PubMed
description In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input.
format Online
Article
Text
id pubmed-9919369
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-99193692023-02-12 Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data Vakil, Asad Blasch, Erik Ewing, Robert Li, Jia Sensors (Basel) Article In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input. MDPI 2023-01-29 /pmc/articles/PMC9919369/ /pubmed/36772527 http://dx.doi.org/10.3390/s23031489 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Vakil, Asad
Blasch, Erik
Ewing, Robert
Li, Jia
Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title_full Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title_fullStr Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title_full_unstemmed Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title_short Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
title_sort finding explanations in ai fusion of electro-optical/passive radio-frequency data
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9919369/
https://www.ncbi.nlm.nih.gov/pubmed/36772527
http://dx.doi.org/10.3390/s23031489
work_keys_str_mv AT vakilasad findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata
AT blascherik findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata
AT ewingrobert findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata
AT lijia findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata