Cargando…

Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network

Image inpainting aims to fill in corrupted regions with visually realistic and semantically plausible contents. In this paper, we propose a progressive image inpainting method, which is based on a forked-then-fused decoder network. A unit called PC-RN, which is the combination of partial convolution...

Descripción completa

Detalles Bibliográficos
Autores principales: Yang, Shuai, Huang, Rong, Han, Fang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8512423/
https://www.ncbi.nlm.nih.gov/pubmed/34640656
http://dx.doi.org/10.3390/s21196336
_version_ 1784582987367579648
author Yang, Shuai
Huang, Rong
Han, Fang
author_facet Yang, Shuai
Huang, Rong
Han, Fang
author_sort Yang, Shuai
collection PubMed
description Image inpainting aims to fill in corrupted regions with visually realistic and semantically plausible contents. In this paper, we propose a progressive image inpainting method, which is based on a forked-then-fused decoder network. A unit called PC-RN, which is the combination of partial convolution and region normalization, serves as the basic component to construct inpainting network. The PC-RN unit can extract useful features from the valid surroundings and can suppress incompleteness-caused interference at the same time. The forked-then-fused decoder network consists of a local reception branch, a long-range attention branch, and a squeeze-and-excitation-based fusing module. Two multi-scale contextual attention modules are deployed into the long-range attention branch for adaptively borrowing features from distant spatial positions. Progressive inpainting strategy allows the attention modules to use the previously filled region to reduce the risk of allocating wrong attention. We conduct extensive experiments on three benchmark databases: Places2, Paris StreetView, and CelebA. Qualitative and quantitative results show that the proposed inpainting model is superior to state-of-the-art works. Moreover, we perform ablation studies to reveal the functionality of each module for the image inpainting task.
format Online
Article
Text
id pubmed-8512423
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-85124232021-10-14 Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network Yang, Shuai Huang, Rong Han, Fang Sensors (Basel) Article Image inpainting aims to fill in corrupted regions with visually realistic and semantically plausible contents. In this paper, we propose a progressive image inpainting method, which is based on a forked-then-fused decoder network. A unit called PC-RN, which is the combination of partial convolution and region normalization, serves as the basic component to construct inpainting network. The PC-RN unit can extract useful features from the valid surroundings and can suppress incompleteness-caused interference at the same time. The forked-then-fused decoder network consists of a local reception branch, a long-range attention branch, and a squeeze-and-excitation-based fusing module. Two multi-scale contextual attention modules are deployed into the long-range attention branch for adaptively borrowing features from distant spatial positions. Progressive inpainting strategy allows the attention modules to use the previously filled region to reduce the risk of allocating wrong attention. We conduct extensive experiments on three benchmark databases: Places2, Paris StreetView, and CelebA. Qualitative and quantitative results show that the proposed inpainting model is superior to state-of-the-art works. Moreover, we perform ablation studies to reveal the functionality of each module for the image inpainting task. MDPI 2021-09-22 /pmc/articles/PMC8512423/ /pubmed/34640656 http://dx.doi.org/10.3390/s21196336 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Yang, Shuai
Huang, Rong
Han, Fang
Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title_full Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title_fullStr Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title_full_unstemmed Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title_short Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network
title_sort progressively inpainting images based on a forked-then-fused decoder network
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8512423/
https://www.ncbi.nlm.nih.gov/pubmed/34640656
http://dx.doi.org/10.3390/s21196336
work_keys_str_mv AT yangshuai progressivelyinpaintingimagesbasedonaforkedthenfuseddecodernetwork
AT huangrong progressivelyinpaintingimagesbasedonaforkedthenfuseddecodernetwork
AT hanfang progressivelyinpaintingimagesbasedonaforkedthenfuseddecodernetwork