Cargando…
Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image sti...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490656/ https://www.ncbi.nlm.nih.gov/pubmed/37687944 http://dx.doi.org/10.3390/s23177488 |
_version_ | 1785103890795986944 |
---|---|
author | Fan, Xiaoting Sun, Long Zhang, Zhong Liu, Shuang Durrani, Tariq S. |
author_facet | Fan, Xiaoting Sun, Long Zhang, Zhong Liu, Shuang Durrani, Tariq S. |
author_sort | Fan, Xiaoting |
collection | PubMed |
description | As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods. |
format | Online Article Text |
id | pubmed-10490656 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-104906562023-09-09 Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching Fan, Xiaoting Sun, Long Zhang, Zhong Liu, Shuang Durrani, Tariq S. Sensors (Basel) Article As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods. MDPI 2023-08-29 /pmc/articles/PMC10490656/ /pubmed/37687944 http://dx.doi.org/10.3390/s23177488 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Fan, Xiaoting Sun, Long Zhang, Zhong Liu, Shuang Durrani, Tariq S. Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title_full | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title_fullStr | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title_full_unstemmed | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title_short | Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching |
title_sort | content-seam-preserving multi-alignment network for visual-sensor-based image stitching |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490656/ https://www.ncbi.nlm.nih.gov/pubmed/37687944 http://dx.doi.org/10.3390/s23177488 |
work_keys_str_mv | AT fanxiaoting contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT sunlong contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT zhangzhong contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT liushuang contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching AT durranitariqs contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching |