Cargando…
A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation
Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9738554/ https://www.ncbi.nlm.nih.gov/pubmed/36502186 http://dx.doi.org/10.3390/s22239483 |
_version_ | 1784847573967699968 |
---|---|
author | Shen, Yu Liu, Yuhang Tian, Yonglin Liu, Zhongmin Wang, Feiyue |
author_facet | Shen, Yu Liu, Yuhang Tian, Yonglin Liu, Zhongmin Wang, Feiyue |
author_sort | Shen, Yu |
collection | PubMed |
description | Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results. |
format | Online Article Text |
id | pubmed-9738554 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-97385542022-12-11 A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation Shen, Yu Liu, Yuhang Tian, Yonglin Liu, Zhongmin Wang, Feiyue Sensors (Basel) Article Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results. MDPI 2022-12-04 /pmc/articles/PMC9738554/ /pubmed/36502186 http://dx.doi.org/10.3390/s22239483 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Shen, Yu Liu, Yuhang Tian, Yonglin Liu, Zhongmin Wang, Feiyue A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title | A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title_full | A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title_fullStr | A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title_full_unstemmed | A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title_short | A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation |
title_sort | new parallel intelligence based light field dataset for depth refinement and scene flow estimation |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9738554/ https://www.ncbi.nlm.nih.gov/pubmed/36502186 http://dx.doi.org/10.3390/s22239483 |
work_keys_str_mv | AT shenyu anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT liuyuhang anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT tianyonglin anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT liuzhongmin anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT wangfeiyue anewparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT shenyu newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT liuyuhang newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT tianyonglin newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT liuzhongmin newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation AT wangfeiyue newparallelintelligencebasedlightfielddatasetfordepthrefinementandsceneflowestimation |