Cargando…

Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we...

Descripción completa

Detalles Bibliográficos
Autores principales: Nam, Ji Hyun, Brandt, Eric, Bauer, Sebastian, Liu, Xiaochun, Renna, Marco, Tosi, Alberto, Sifakis, Eftychios, Velten, Andreas
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8586255/
https://www.ncbi.nlm.nih.gov/pubmed/34764273
http://dx.doi.org/10.1038/s41467-021-26721-x
_version_ 1784597857347567616
author Nam, Ji Hyun
Brandt, Eric
Bauer, Sebastian
Liu, Xiaochun
Renna, Marco
Tosi, Alberto
Sifakis, Eftychios
Velten, Andreas
author_facet Nam, Ji Hyun
Brandt, Eric
Bauer, Sebastian
Liu, Xiaochun
Renna, Marco
Tosi, Alberto
Sifakis, Eftychios
Velten, Andreas
author_sort Nam, Ji Hyun
collection PubMed
description Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed Single Photon Avalanche Diode (SPAD) array detectors with a fast reconstruction algorithm that captures and reconstructs live low-latency videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible.
format Online
Article
Text
id pubmed-8586255
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-85862552021-11-15 Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second Nam, Ji Hyun Brandt, Eric Bauer, Sebastian Liu, Xiaochun Renna, Marco Tosi, Alberto Sifakis, Eftychios Velten, Andreas Nat Commun Article Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed Single Photon Avalanche Diode (SPAD) array detectors with a fast reconstruction algorithm that captures and reconstructs live low-latency videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible. Nature Publishing Group UK 2021-11-11 /pmc/articles/PMC8586255/ /pubmed/34764273 http://dx.doi.org/10.1038/s41467-021-26721-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Nam, Ji Hyun
Brandt, Eric
Bauer, Sebastian
Liu, Xiaochun
Renna, Marco
Tosi, Alberto
Sifakis, Eftychios
Velten, Andreas
Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title_full Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title_fullStr Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title_full_unstemmed Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title_short Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
title_sort low-latency time-of-flight non-line-of-sight imaging at 5 frames per second
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8586255/
https://www.ncbi.nlm.nih.gov/pubmed/34764273
http://dx.doi.org/10.1038/s41467-021-26721-x
work_keys_str_mv AT namjihyun lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT brandteric lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT bauersebastian lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT liuxiaochun lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT rennamarco lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT tosialberto lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT sifakiseftychios lowlatencytimeofflightnonlineofsightimagingat5framespersecond
AT veltenandreas lowlatencytimeofflightnonlineofsightimagingat5framespersecond