Cargando…
General cross-modality registration framework for visible and infrared UAV target image registration
In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformati...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10412594/ https://www.ncbi.nlm.nih.gov/pubmed/37558713 http://dx.doi.org/10.1038/s41598-023-39863-3 |
_version_ | 1785086945144078336 |
---|---|
author | Luo, Yu Cha, Hao Zuo, Lei Cheng, Peng Zhao, Qing |
author_facet | Luo, Yu Cha, Hao Zuo, Lei Cheng, Peng Zhao, Qing |
author_sort | Luo, Yu |
collection | PubMed |
description | In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformation and the difficulty narrowing cross-modality discrepancy. To better overcome the obstacle, in this paper, we construct a General Cross-Modality Registration (GCMR) Framework, which explores generation registration pattern to simplify the cross-modality image registration into a easier mono-modality image registration with an Image Cross-Modality Translation Network (ICMTN) module and a Multi-level Residual Dense Registration Network (MRDRN). Specifically, ICMTN module is used to generate a pseudo infrared image taking a visible image as input and correct the distortion of structural information during the translation of image modalities. Benefiting from the favorable geometry correct ability of the ICMTN, we further employs MRDRN module which can fully extract and exploit the mutual information of misaligned images to better registered Visible and Infrared image in a mono-modality setting. We evaluate five variants of our approach on the public Anti-UAV datasets. The extensive experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. |
format | Online Article Text |
id | pubmed-10412594 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-104125942023-08-11 General cross-modality registration framework for visible and infrared UAV target image registration Luo, Yu Cha, Hao Zuo, Lei Cheng, Peng Zhao, Qing Sci Rep Article In all-day-all-weather tasks, well-aligned multi-modality images pairs can provide extensive complementary information for image-guided UAV target detection. However, multi-modality images in real scenarios are often misaligned, and images registration is extremely difficult due to spatial deformation and the difficulty narrowing cross-modality discrepancy. To better overcome the obstacle, in this paper, we construct a General Cross-Modality Registration (GCMR) Framework, which explores generation registration pattern to simplify the cross-modality image registration into a easier mono-modality image registration with an Image Cross-Modality Translation Network (ICMTN) module and a Multi-level Residual Dense Registration Network (MRDRN). Specifically, ICMTN module is used to generate a pseudo infrared image taking a visible image as input and correct the distortion of structural information during the translation of image modalities. Benefiting from the favorable geometry correct ability of the ICMTN, we further employs MRDRN module which can fully extract and exploit the mutual information of misaligned images to better registered Visible and Infrared image in a mono-modality setting. We evaluate five variants of our approach on the public Anti-UAV datasets. The extensive experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. Nature Publishing Group UK 2023-08-09 /pmc/articles/PMC10412594/ /pubmed/37558713 http://dx.doi.org/10.1038/s41598-023-39863-3 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Luo, Yu Cha, Hao Zuo, Lei Cheng, Peng Zhao, Qing General cross-modality registration framework for visible and infrared UAV target image registration |
title | General cross-modality registration framework for visible and infrared UAV target image registration |
title_full | General cross-modality registration framework for visible and infrared UAV target image registration |
title_fullStr | General cross-modality registration framework for visible and infrared UAV target image registration |
title_full_unstemmed | General cross-modality registration framework for visible and infrared UAV target image registration |
title_short | General cross-modality registration framework for visible and infrared UAV target image registration |
title_sort | general cross-modality registration framework for visible and infrared uav target image registration |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10412594/ https://www.ncbi.nlm.nih.gov/pubmed/37558713 http://dx.doi.org/10.1038/s41598-023-39863-3 |
work_keys_str_mv | AT luoyu generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration AT chahao generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration AT zuolei generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration AT chengpeng generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration AT zhaoqing generalcrossmodalityregistrationframeworkforvisibleandinfrareduavtargetimageregistration |