Cargando…

A New Deep Learning Based Multi-Spectral Image Fusion Method

In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source...

Descripción completa

Detalles Bibliográficos
Autores principales: Piao, Jingchun, Chen, Yunfan, Shin, Hyunchul
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515058/
https://www.ncbi.nlm.nih.gov/pubmed/33267284
http://dx.doi.org/10.3390/e21060570
_version_ 1783586731028643840
author Piao, Jingchun
Chen, Yunfan
Shin, Hyunchul
author_facet Piao, Jingchun
Chen, Yunfan
Shin, Hyunchul
author_sort Piao, Jingchun
collection PubMed
description In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source images. A CNN plays a role in automatic encoding an image into a feature domain for classification. By applying the proposed method, the key problems in image fusion, which are the activity level measurement and fusion rule design, can be figured out in one shot. The fusion is carried out through the multi-scale image decomposition based on wavelet transform, and the reconstruction result is more perceptual to a human visual system. In addition, the visual qualitative effectiveness of the proposed fusion method is evaluated by comparing pedestrian detection results with other methods, by using the YOLOv3 object detector using a public benchmark dataset. The experimental results show that our proposed method showed competitive results in terms of both quantitative assessment and visual quality.
format Online
Article
Text
id pubmed-7515058
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75150582020-11-09 A New Deep Learning Based Multi-Spectral Image Fusion Method Piao, Jingchun Chen, Yunfan Shin, Hyunchul Entropy (Basel) Article In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source images. A CNN plays a role in automatic encoding an image into a feature domain for classification. By applying the proposed method, the key problems in image fusion, which are the activity level measurement and fusion rule design, can be figured out in one shot. The fusion is carried out through the multi-scale image decomposition based on wavelet transform, and the reconstruction result is more perceptual to a human visual system. In addition, the visual qualitative effectiveness of the proposed fusion method is evaluated by comparing pedestrian detection results with other methods, by using the YOLOv3 object detector using a public benchmark dataset. The experimental results show that our proposed method showed competitive results in terms of both quantitative assessment and visual quality. MDPI 2019-06-05 /pmc/articles/PMC7515058/ /pubmed/33267284 http://dx.doi.org/10.3390/e21060570 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Piao, Jingchun
Chen, Yunfan
Shin, Hyunchul
A New Deep Learning Based Multi-Spectral Image Fusion Method
title A New Deep Learning Based Multi-Spectral Image Fusion Method
title_full A New Deep Learning Based Multi-Spectral Image Fusion Method
title_fullStr A New Deep Learning Based Multi-Spectral Image Fusion Method
title_full_unstemmed A New Deep Learning Based Multi-Spectral Image Fusion Method
title_short A New Deep Learning Based Multi-Spectral Image Fusion Method
title_sort new deep learning based multi-spectral image fusion method
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515058/
https://www.ncbi.nlm.nih.gov/pubmed/33267284
http://dx.doi.org/10.3390/e21060570
work_keys_str_mv AT piaojingchun anewdeeplearningbasedmultispectralimagefusionmethod
AT chenyunfan anewdeeplearningbasedmultispectralimagefusionmethod
AT shinhyunchul anewdeeplearningbasedmultispectralimagefusionmethod
AT piaojingchun newdeeplearningbasedmultispectralimagefusionmethod
AT chenyunfan newdeeplearningbasedmultispectralimagefusionmethod
AT shinhyunchul newdeeplearningbasedmultispectralimagefusionmethod