Cargando…

General cross-modality registration framework for visible and infrared UAV target image registration

In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformati...

Descripción completa

Detalles Bibliográficos
Autores principales: Luo, Yu, Cha, Hao, Zuo, Lei, Cheng, Peng, Zhao, Qing
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10412594/
https://www.ncbi.nlm.nih.gov/pubmed/37558713
http://dx.doi.org/10.1038/s41598-023-39863-3
_version_ 1785086945144078336
author Luo, Yu
Cha, Hao
Zuo, Lei
Cheng, Peng
Zhao, Qing
author_facet Luo, Yu
Cha, Hao
Zuo, Lei
Cheng, Peng
Zhao, Qing
author_sort Luo, Yu
collection PubMed
description In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformation and the difficulty narrowing cross-modality discrepancy. To better overcome the obstacle, in this paper, we construct a General Cross-Modality Registration (GCMR) Framework, which explores generation registration pattern to simplify the cross-modality image registration into a easier mono-modality image registration with an Image Cross-Modality Translation Network (ICMTN) module and a Multi-level Residual Dense Registration Network (MRDRN). Specifically, ICMTN module is used to generate a pseudo infrared image taking a visible image as input and correct the distortion of structural information during the translation of image modalities. Benefiting from the favorable geometry correct ability of the ICMTN, we further employs MRDRN module which can fully extract and exploit the mutual information of misaligned images to better registered Visible and Infrared image in a mono-modality setting. We evaluate five variants of our approach on the public Anti-UAV datasets. The extensive experimental results demonstrate that the proposed architecture achieves state-of-the-art performance.
format Online
Article
Text
id pubmed-10412594
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-104125942023-08-11 General cross-modality registration framework for visible and infrared UAV target image registration Luo, Yu Cha, Hao Zuo, Lei Cheng, Peng Zhao, Qing Sci Rep Article In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformation and the difficulty narrowing cross-modality discrepancy. To better overcome the obstacle, in this paper, we construct a General Cross-Modality Registration (GCMR) Framework, which explores generation registration pattern to simplify the cross-modality image registration into a easier mono-modality image registration with an Image Cross-Modality Translation Network (ICMTN) module and a Multi-level Residual Dense Registration Network (MRDRN). Specifically, ICMTN module is used to generate a pseudo infrared image taking a visible image as input and correct the distortion of structural information during the translation of image modalities. Benefiting from the favorable geometry correct ability of the ICMTN, we further employs MRDRN module which can fully extract and exploit the mutual information of misaligned images to better registered Visible and Infrared image in a mono-modality setting. We evaluate five variants of our approach on the public Anti-UAV datasets. The extensive experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. Nature Publishing Group UK 2023-08-09 /pmc/articles/PMC10412594/ /pubmed/37558713 http://dx.doi.org/10.1038/s41598-023-39863-3 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Luo, Yu
Cha, Hao
Zuo, Lei
Cheng, Peng
Zhao, Qing
General cross-modality registration framework for visible and infrared UAV target image registration
title General cross-modality registration framework for visible and infrared UAV target image registration
title_full General cross-modality registration framework for visible and infrared UAV target image registration
title_fullStr General cross-modality registration framework for visible and infrared UAV target image registration
title_full_unstemmed General cross-modality registration framework for visible and infrared UAV target image registration
title_short General cross-modality registration framework for visible and infrared UAV target image registration
title_sort general cross-modality registration framework for visible and infrared uav target image registration
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10412594/
https://www.ncbi.nlm.nih.gov/pubmed/37558713
http://dx.doi.org/10.1038/s41598-023-39863-3
work_keys_str_mv AT luoyu generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration
AT chahao generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration
AT zuolei generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration
AT chengpeng generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration
AT zhaoqing generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration