Cargando…

Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor

Multi-path interference causes depth errors in indirect time-of-flight (ToF) cameras. In this paper, resolving multi-path interference caused by surface reflections using a multi-tap macro-pixel computational CMOS image sensor is demonstrated. The imaging area is implemented by an array of macro-pix...

Descripción completa

Detalles Bibliográficos
Autores principales: Horio, Masaya, Feng, Yu, Kokado, Tomoya, Takasawa, Taishi, Yasutomi, Keita, Kawahito, Shoji, Komuro, Takashi, Nagahara, Hajime, Kagawa, Keiichiro
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9003367/
https://www.ncbi.nlm.nih.gov/pubmed/35408057
http://dx.doi.org/10.3390/s22072442
_version_ 1784686117537185792
author Horio, Masaya
Feng, Yu
Kokado, Tomoya
Takasawa, Taishi
Yasutomi, Keita
Kawahito, Shoji
Komuro, Takashi
Nagahara, Hajime
Kagawa, Keiichiro
author_facet Horio, Masaya
Feng, Yu
Kokado, Tomoya
Takasawa, Taishi
Yasutomi, Keita
Kawahito, Shoji
Komuro, Takashi
Nagahara, Hajime
Kagawa, Keiichiro
author_sort Horio, Masaya
collection PubMed
description Multi-path interference causes depth errors in indirect time-of-flight (ToF) cameras. In this paper, resolving multi-path interference caused by surface reflections using a multi-tap macro-pixel computational CMOS image sensor is demonstrated. The imaging area is implemented by an array of macro-pixels composed of four subpixels embodied by a four-tap lateral electric field charge modulator (LEFM). This sensor can simultaneously acquire 16 images for different temporal shutters. This method can reproduce more than 16 images based on compressive sensing with multi-frequency shutters and sub-clock shifting. In simulations, an object was placed 16 m away from the sensor, and the depth of an interference object was varied from 1 to 32 m in 1 m steps. The two reflections were separated in two stages: coarse estimation based on a compressive sensing solver and refinement by a nonlinear search to investigate the potential of our sensor. Relative standard deviation (precision) and relative mean error (accuracy) were evaluated under the influence of photon shot noise. The proposed method was verified using a prototype multi-tap macro-pixel computational CMOS image sensor in single-path and dual-path situations. In the experiment, an acrylic plate was placed 1 m or 2 m and a mirror 9.3 m from the sensor.
format Online
Article
Text
id pubmed-9003367
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-90033672022-04-13 Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor Horio, Masaya Feng, Yu Kokado, Tomoya Takasawa, Taishi Yasutomi, Keita Kawahito, Shoji Komuro, Takashi Nagahara, Hajime Kagawa, Keiichiro Sensors (Basel) Article Multi-path interference causes depth errors in indirect time-of-flight (ToF) cameras. In this paper, resolving multi-path interference caused by surface reflections using a multi-tap macro-pixel computational CMOS image sensor is demonstrated. The imaging area is implemented by an array of macro-pixels composed of four subpixels embodied by a four-tap lateral electric field charge modulator (LEFM). This sensor can simultaneously acquire 16 images for different temporal shutters. This method can reproduce more than 16 images based on compressive sensing with multi-frequency shutters and sub-clock shifting. In simulations, an object was placed 16 m away from the sensor, and the depth of an interference object was varied from 1 to 32 m in 1 m steps. The two reflections were separated in two stages: coarse estimation based on a compressive sensing solver and refinement by a nonlinear search to investigate the potential of our sensor. Relative standard deviation (precision) and relative mean error (accuracy) were evaluated under the influence of photon shot noise. The proposed method was verified using a prototype multi-tap macro-pixel computational CMOS image sensor in single-path and dual-path situations. In the experiment, an acrylic plate was placed 1 m or 2 m and a mirror 9.3 m from the sensor. MDPI 2022-03-22 /pmc/articles/PMC9003367/ /pubmed/35408057 http://dx.doi.org/10.3390/s22072442 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Horio, Masaya
Feng, Yu
Kokado, Tomoya
Takasawa, Taishi
Yasutomi, Keita
Kawahito, Shoji
Komuro, Takashi
Nagahara, Hajime
Kagawa, Keiichiro
Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title_full Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title_fullStr Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title_full_unstemmed Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title_short Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
title_sort resolving multi-path interference in compressive time-of-flight depth imaging with a multi-tap macro-pixel computational cmos image sensor
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9003367/
https://www.ncbi.nlm.nih.gov/pubmed/35408057
http://dx.doi.org/10.3390/s22072442
work_keys_str_mv AT horiomasaya resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT fengyu resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT kokadotomoya resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT takasawataishi resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT yasutomikeita resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT kawahitoshoji resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT komurotakashi resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT nagaharahajime resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor
AT kagawakeiichiro resolvingmultipathinterferenceincompressivetimeofflightdepthimagingwithamultitapmacropixelcomputationalcmosimagesensor