Cargando…
Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes †
Depth sensing has improved rapidly in recent years, which allows for structural information to be utilized in various applications, such as virtual reality, scene and object recognition, view synthesis, and 3D reconstruction. Due to the limitations of the current generation of depth sensors, the res...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8309932/ https://www.ncbi.nlm.nih.gov/pubmed/34300631 http://dx.doi.org/10.3390/s21144892 |
_version_ | 1783728639643222016 |
---|---|
author | Truong, Anh Minh Philips, Wilfried Veelaert, Peter |
author_facet | Truong, Anh Minh Philips, Wilfried Veelaert, Peter |
author_sort | Truong, Anh Minh |
collection | PubMed |
description | Depth sensing has improved rapidly in recent years, which allows for structural information to be utilized in various applications, such as virtual reality, scene and object recognition, view synthesis, and 3D reconstruction. Due to the limitations of the current generation of depth sensors, the resolution of depth maps is often still much lower than the resolution of color images. This hinders applications, such as view synthesis or 3D reconstruction, from providing high-quality results. Therefore, super-resolution, which allows for the upscaling of depth maps while still retaining sharpness, has recently drawn much attention in the deep learning community. However, state-of-the-art deep learning methods are typically designed and trained to handle a fixed set of integer-scale factors. Moreover, the raw depth map collected by the depth sensor usually has many depth data missing or misestimated values along the edges and corners of observed objects. In this work, we propose a novel deep learning network for both depth completion and depth super-resolution with arbitrary scale factors. The experimental results on the Middlebury stereo, NYUv2, and Matterport3D datasets demonstrate that the proposed method can outperform state-of-the-art methods. |
format | Online Article Text |
id | pubmed-8309932 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-83099322021-07-25 Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † Truong, Anh Minh Philips, Wilfried Veelaert, Peter Sensors (Basel) Article Depth sensing has improved rapidly in recent years, which allows for structural information to be utilized in various applications, such as virtual reality, scene and object recognition, view synthesis, and 3D reconstruction. Due to the limitations of the current generation of depth sensors, the resolution of depth maps is often still much lower than the resolution of color images. This hinders applications, such as view synthesis or 3D reconstruction, from providing high-quality results. Therefore, super-resolution, which allows for the upscaling of depth maps while still retaining sharpness, has recently drawn much attention in the deep learning community. However, state-of-the-art deep learning methods are typically designed and trained to handle a fixed set of integer-scale factors. Moreover, the raw depth map collected by the depth sensor usually has many depth data missing or misestimated values along the edges and corners of observed objects. In this work, we propose a novel deep learning network for both depth completion and depth super-resolution with arbitrary scale factors. The experimental results on the Middlebury stereo, NYUv2, and Matterport3D datasets demonstrate that the proposed method can outperform state-of-the-art methods. MDPI 2021-07-18 /pmc/articles/PMC8309932/ /pubmed/34300631 http://dx.doi.org/10.3390/s21144892 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Truong, Anh Minh Philips, Wilfried Veelaert, Peter Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title | Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title_full | Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title_fullStr | Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title_full_unstemmed | Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title_short | Depth Completion and Super-Resolution with Arbitrary Scale Factors for Indoor Scenes † |
title_sort | depth completion and super-resolution with arbitrary scale factors for indoor scenes † |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8309932/ https://www.ncbi.nlm.nih.gov/pubmed/34300631 http://dx.doi.org/10.3390/s21144892 |
work_keys_str_mv | AT truonganhminh depthcompletionandsuperresolutionwitharbitraryscalefactorsforindoorscenes AT philipswilfried depthcompletionandsuperresolutionwitharbitraryscalefactorsforindoorscenes AT veelaertpeter depthcompletionandsuperresolutionwitharbitraryscalefactorsforindoorscenes |