Cargando…

A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation

Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard...

Descripción completa

Detalles Bibliográficos
Autores principales: Shen, Yu, Liu, Yuhang, Tian, Yonglin, Liu, Zhongmin, Wang, Feiyue
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9738554/
https://www.ncbi.nlm.nih.gov/pubmed/36502186
http://dx.doi.org/10.3390/s22239483
_version_ 1784847573967699968
author Shen, Yu
Liu, Yuhang
Tian, Yonglin
Liu, Zhongmin
Wang, Feiyue
author_facet Shen, Yu
Liu, Yuhang
Tian, Yonglin
Liu, Zhongmin
Wang, Feiyue
author_sort Shen, Yu
collection PubMed
description Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results.
format Online
Article
Text
id pubmed-9738554
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-97385542022-12-11 A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation Shen, Yu Liu, Yuhang Tian, Yonglin Liu, Zhongmin Wang, Feiyue Sensors (Basel) Article Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results. MDPI 2022-12-04 /pmc/articles/PMC9738554/ /pubmed/36502186 http://dx.doi.org/10.3390/s22239483 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Shen, Yu
Liu, Yuhang
Tian, Yonglin
Liu, Zhongmin
Wang, Feiyue
A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title_full A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title_fullStr A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title_full_unstemmed A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title_short A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
title_sort new parallel intelligence based light field dataset for depth refinement and scene flow estimation
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9738554/
https://www.ncbi.nlm.nih.gov/pubmed/36502186
http://dx.doi.org/10.3390/s22239483
work_keys_str_mv AT shenyu anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT liuyuhang anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT tianyonglin anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT liuzhongmin anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT wangfeiyue anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT shenyu newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT liuyuhang newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT tianyonglin newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT liuzhongmin newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation
AT wangfeiyue newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation