Cargando…

ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping

With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this...

Descripción completa

Detalles Bibliográficos
Autores principales: Yang, Shuai, Qiao, Kai, Qin, Ruoxi, Xie, Pengfei, Shi, Shuhao, Liang, Ningning, Wang, Linyuan, Chen, Jian, Hu, Guoen, Yan, Bin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8814752/
https://www.ncbi.nlm.nih.gov/pubmed/35126081
http://dx.doi.org/10.3389/fnbot.2021.785808
_version_ 1784645132031623168
author Yang, Shuai
Qiao, Kai
Qin, Ruoxi
Xie, Pengfei
Shi, Shuhao
Liang, Ningning
Wang, Linyuan
Chen, Jian
Hu, Guoen
Yan, Bin
author_facet Yang, Shuai
Qiao, Kai
Qin, Ruoxi
Xie, Pengfei
Shi, Shuhao
Liang, Ningning
Wang, Linyuan
Chen, Jian
Hu, Guoen
Yan, Bin
author_sort Yang, Shuai
collection PubMed
description With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute.
format Online
Article
Text
id pubmed-8814752
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-88147522022-02-05 ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping Yang, Shuai Qiao, Kai Qin, Ruoxi Xie, Pengfei Shi, Shuhao Liang, Ningning Wang, Linyuan Chen, Jian Hu, Guoen Yan, Bin Front Neurorobot Neuroscience With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute. Frontiers Media S.A. 2022-01-21 /pmc/articles/PMC8814752/ /pubmed/35126081 http://dx.doi.org/10.3389/fnbot.2021.785808 Text en Copyright © 2022 Yang, Qiao, Qin, Xie, Shi, Liang, Wang, Chen, Hu and Yan. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Yang, Shuai
Qiao, Kai
Qin, Ruoxi
Xie, Pengfei
Shi, Shuhao
Liang, Ningning
Wang, Linyuan
Chen, Jian
Hu, Guoen
Yan, Bin
ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title_full ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title_fullStr ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title_full_unstemmed ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title_short ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
title_sort shapeeditor: a stylegan encoder for stable and high fidelity face swapping
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8814752/
https://www.ncbi.nlm.nih.gov/pubmed/35126081
http://dx.doi.org/10.3389/fnbot.2021.785808
work_keys_str_mv AT yangshuai shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT qiaokai shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT qinruoxi shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT xiepengfei shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT shishuhao shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT liangningning shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT wanglinyuan shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT chenjian shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT huguoen shapeeditorastyleganencoderforstableandhighfidelityfaceswapping
AT yanbin shapeeditorastyleganencoderforstableandhighfidelityfaceswapping