Cargando…
Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain
Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Hindawi
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7178475/ https://www.ncbi.nlm.nih.gov/pubmed/32352003 http://dx.doi.org/10.1155/2020/6265708 |
_version_ | 1783525463917854720 |
---|---|
author | Ding, Zhaisheng Zhou, Dongming Nie, Rencan Hou, Ruichao Liu, Yanyu |
author_facet | Ding, Zhaisheng Zhou, Dongming Nie, Rencan Hou, Ruichao Liu, Yanyu |
author_sort | Ding, Zhaisheng |
collection | PubMed |
description | Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion. |
format | Online Article Text |
id | pubmed-7178475 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Hindawi |
record_format | MEDLINE/PubMed |
spelling | pubmed-71784752020-04-29 Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain Ding, Zhaisheng Zhou, Dongming Nie, Rencan Hou, Ruichao Liu, Yanyu Biomed Res Int Research Article Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion. Hindawi 2020-04-14 /pmc/articles/PMC7178475/ /pubmed/32352003 http://dx.doi.org/10.1155/2020/6265708 Text en Copyright © 2020 Zhaisheng Ding et al. http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Research Article Ding, Zhaisheng Zhou, Dongming Nie, Rencan Hou, Ruichao Liu, Yanyu Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title | Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title_full | Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title_fullStr | Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title_full_unstemmed | Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title_short | Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain |
title_sort | brain medical image fusion based on dual-branch cnns in nsst domain |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7178475/ https://www.ncbi.nlm.nih.gov/pubmed/32352003 http://dx.doi.org/10.1155/2020/6265708 |
work_keys_str_mv | AT dingzhaisheng brainmedicalimagefusionbasedondualbranchcnnsinnsstdomain AT zhoudongming brainmedicalimagefusionbasedondualbranchcnnsinnsstdomain AT nierencan brainmedicalimagefusionbasedondualbranchcnnsinnsstdomain AT houruichao brainmedicalimagefusionbasedondualbranchcnnsinnsstdomain AT liuyanyu brainmedicalimagefusionbasedondualbranchcnnsinnsstdomain |