Cargando…

Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching

As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image sti...

Descripción completa

Detalles Bibliográficos
Autores principales: Fan, Xiaoting, Sun, Long, Zhang, Zhong, Liu, Shuang, Durrani, Tariq S.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490656/
https://www.ncbi.nlm.nih.gov/pubmed/37687944
http://dx.doi.org/10.3390/s23177488
_version_ 1785103890795986944
author Fan, Xiaoting
Sun, Long
Zhang, Zhong
Liu, Shuang
Durrani, Tariq S.
author_facet Fan, Xiaoting
Sun, Long
Zhang, Zhong
Liu, Shuang
Durrani, Tariq S.
author_sort Fan, Xiaoting
collection PubMed
description As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods.
format Online
Article
Text
id pubmed-10490656
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-104906562023-09-09 Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching Fan, Xiaoting Sun, Long Zhang, Zhong Liu, Shuang Durrani, Tariq S. Sensors (Basel) Article As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods. MDPI 2023-08-29 /pmc/articles/PMC10490656/ /pubmed/37687944 http://dx.doi.org/10.3390/s23177488 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Fan, Xiaoting
Sun, Long
Zhang, Zhong
Liu, Shuang
Durrani, Tariq S.
Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title_full Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title_fullStr Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title_full_unstemmed Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title_short Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching
title_sort content-seam-preserving multi-alignment network for visual-sensor-based image stitching
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490656/
https://www.ncbi.nlm.nih.gov/pubmed/37687944
http://dx.doi.org/10.3390/s23177488
work_keys_str_mv AT fanxiaoting contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching
AT sunlong contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching
AT zhangzhong contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching
AT liushuang contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching
AT durranitariqs contentseampreservingmultialignmentnetworkforvisualsensorbasedimagestitching