Cargando…
Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion
When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded in...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10297407/ https://www.ncbi.nlm.nih.gov/pubmed/37372258 http://dx.doi.org/10.3390/e25060914 |
_version_ | 1785063876431183872 |
---|---|
author | Jiang, Yichun Liu, Yunqing Zhan, Weida Zhu, Depeng |
author_facet | Jiang, Yichun Liu, Yunqing Zhan, Weida Zhu, Depeng |
author_sort | Jiang, Yichun |
collection | PubMed |
description | When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network’s ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness. |
format | Online Article Text |
id | pubmed-10297407 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-102974072023-06-28 Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion Jiang, Yichun Liu, Yunqing Zhan, Weida Zhu, Depeng Entropy (Basel) Article When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network’s ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness. MDPI 2023-06-09 /pmc/articles/PMC10297407/ /pubmed/37372258 http://dx.doi.org/10.3390/e25060914 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Jiang, Yichun Liu, Yunqing Zhan, Weida Zhu, Depeng Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title | Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title_full | Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title_fullStr | Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title_full_unstemmed | Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title_short | Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion |
title_sort | improved thermal infrared image super-resolution reconstruction method base on multimodal sensor fusion |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10297407/ https://www.ncbi.nlm.nih.gov/pubmed/37372258 http://dx.doi.org/10.3390/e25060914 |
work_keys_str_mv | AT jiangyichun improvedthermalinfraredimagesuperresolutionreconstructionmethodbaseonmultimodalsensorfusion AT liuyunqing improvedthermalinfraredimagesuperresolutionreconstructionmethodbaseonmultimodalsensorfusion AT zhanweida improvedthermalinfraredimagesuperresolutionreconstructionmethodbaseonmultimodalsensorfusion AT zhudepeng improvedthermalinfraredimagesuperresolutionreconstructionmethodbaseonmultimodalsensorfusion |