Cargando…
Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data
In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate t...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9919369/ https://www.ncbi.nlm.nih.gov/pubmed/36772527 http://dx.doi.org/10.3390/s23031489 |
_version_ | 1784886807680253952 |
---|---|
author | Vakil, Asad Blasch, Erik Ewing, Robert Li, Jia |
author_facet | Vakil, Asad Blasch, Erik Ewing, Robert Li, Jia |
author_sort | Vakil, Asad |
collection | PubMed |
description | In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input. |
format | Online Article Text |
id | pubmed-9919369 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-99193692023-02-12 Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data Vakil, Asad Blasch, Erik Ewing, Robert Li, Jia Sensors (Basel) Article In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input. MDPI 2023-01-29 /pmc/articles/PMC9919369/ /pubmed/36772527 http://dx.doi.org/10.3390/s23031489 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Vakil, Asad Blasch, Erik Ewing, Robert Li, Jia Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title | Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title_full | Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title_fullStr | Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title_full_unstemmed | Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title_short | Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data |
title_sort | finding explanations in ai fusion of electro-optical/passive radio-frequency data |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9919369/ https://www.ncbi.nlm.nih.gov/pubmed/36772527 http://dx.doi.org/10.3390/s23031489 |
work_keys_str_mv | AT vakilasad findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata AT blascherik findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata AT ewingrobert findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata AT lijia findingexplanationsinaifusionofelectroopticalpassiveradiofrequencydata |