Cargando…

Virtual View Generation Based on 3D-Dense-Attentive GAN Networks

A binocular vision system is a common perception component of an intelligent vehicle. Benefiting from the biomimetic structure, the system is simple and effective. Which are extremely snesitive on external factors, especially missing vision signals. In this paper, a virtual view-generation algorithm...

Descripción completa

Detalles Bibliográficos
Autores principales: Fu, Junwei, Liang, Jun
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6358985/
https://www.ncbi.nlm.nih.gov/pubmed/30654544
http://dx.doi.org/10.3390/s19020344
_version_ 1783392118507569152
author Fu, Junwei
Liang, Jun
author_facet Fu, Junwei
Liang, Jun
author_sort Fu, Junwei
collection PubMed
description A binocular vision system is a common perception component of an intelligent vehicle. Benefiting from the biomimetic structure, the system is simple and effective. Which are extremely snesitive on external factors, especially missing vision signals. In this paper, a virtual view-generation algorithm based on generative adversarial networks (GAN) is proposed to enhance the robustness of binocular vision systems. The proposed model consists of two parts: generative network and discriminator network. To improve the quality of a virtual view, a generative network structure based on 3D convolutional neural networks (3D-CNN) and attentive mechanisms is introduced to extract the time-series features from image sequences. To avoid gradient vanish during training, the dense block structure is utilized to improve the discriminator network. Meanwhile, three kinds of image features, including image edge, depth map and optical flow are extracted to constrain the supervised training of model. The final results on KITTI and Cityscapes datasets demonstrate that our algorithm outperforms conventional methods, and the missing vision signal can be replaced by a generated virtual view.
format Online
Article
Text
id pubmed-6358985
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-63589852019-02-06 Virtual View Generation Based on 3D-Dense-Attentive GAN Networks Fu, Junwei Liang, Jun Sensors (Basel) Article A binocular vision system is a common perception component of an intelligent vehicle. Benefiting from the biomimetic structure, the system is simple and effective. Which are extremely snesitive on external factors, especially missing vision signals. In this paper, a virtual view-generation algorithm based on generative adversarial networks (GAN) is proposed to enhance the robustness of binocular vision systems. The proposed model consists of two parts: generative network and discriminator network. To improve the quality of a virtual view, a generative network structure based on 3D convolutional neural networks (3D-CNN) and attentive mechanisms is introduced to extract the time-series features from image sequences. To avoid gradient vanish during training, the dense block structure is utilized to improve the discriminator network. Meanwhile, three kinds of image features, including image edge, depth map and optical flow are extracted to constrain the supervised training of model. The final results on KITTI and Cityscapes datasets demonstrate that our algorithm outperforms conventional methods, and the missing vision signal can be replaced by a generated virtual view. MDPI 2019-01-16 /pmc/articles/PMC6358985/ /pubmed/30654544 http://dx.doi.org/10.3390/s19020344 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Fu, Junwei
Liang, Jun
Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title_full Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title_fullStr Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title_full_unstemmed Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title_short Virtual View Generation Based on 3D-Dense-Attentive GAN Networks
title_sort virtual view generation based on 3d-dense-attentive gan networks
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6358985/
https://www.ncbi.nlm.nih.gov/pubmed/30654544
http://dx.doi.org/10.3390/s19020344
work_keys_str_mv AT fujunwei virtualviewgenerationbasedon3ddenseattentivegannetworks
AT liangjun virtualviewgenerationbasedon3ddenseattentivegannetworks