Cargando…
ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping
With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this...
Autores principales: | , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8814752/ https://www.ncbi.nlm.nih.gov/pubmed/35126081 http://dx.doi.org/10.3389/fnbot.2021.785808 |
_version_ | 1784645132031623168 |
---|---|
author | Yang, Shuai Qiao, Kai Qin, Ruoxi Xie, Pengfei Shi, Shuhao Liang, Ningning Wang, Linyuan Chen, Jian Hu, Guoen Yan, Bin |
author_facet | Yang, Shuai Qiao, Kai Qin, Ruoxi Xie, Pengfei Shi, Shuhao Liang, Ningning Wang, Linyuan Chen, Jian Hu, Guoen Yan, Bin |
author_sort | Yang, Shuai |
collection | PubMed |
description | With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute. |
format | Online Article Text |
id | pubmed-8814752 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-88147522022-02-05 ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping Yang, Shuai Qiao, Kai Qin, Ruoxi Xie, Pengfei Shi, Shuhao Liang, Ningning Wang, Linyuan Chen, Jian Hu, Guoen Yan, Bin Front Neurorobot Neuroscience With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute. Frontiers Media S.A. 2022-01-21 /pmc/articles/PMC8814752/ /pubmed/35126081 http://dx.doi.org/10.3389/fnbot.2021.785808 Text en Copyright © 2022 Yang, Qiao, Qin, Xie, Shi, Liang, Wang, Chen, Hu and Yan. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Yang, Shuai Qiao, Kai Qin, Ruoxi Xie, Pengfei Shi, Shuhao Liang, Ningning Wang, Linyuan Chen, Jian Hu, Guoen Yan, Bin ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title | ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title_full | ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title_fullStr | ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title_full_unstemmed | ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title_short | ShapeEditor: A StyleGAN Encoder for Stable and High Fidelity Face Swapping |
title_sort | shapeeditor: a stylegan encoder for stable and high fidelity face swapping |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8814752/ https://www.ncbi.nlm.nih.gov/pubmed/35126081 http://dx.doi.org/10.3389/fnbot.2021.785808 |
work_keys_str_mv | AT yangshuai shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT qiaokai shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT qinruoxi shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT xiepengfei shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT shishuhao shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT liangningning shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT wanglinyuan shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT chenjian shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT huguoen shapeeditorastyleganencoderforstableandhighfidelityfaceswapping AT yanbin shapeeditorastyleganencoderforstableandhighfidelityfaceswapping |