Cargando…

Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim

In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights si...

Descripción completa

Detalles Bibliográficos
Autores principales: Shimada, Tomoyasu, Nishikawa, Hiroki, Kong, Xiangbo, Tomiyama, Hiroyuki
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8948838/
https://www.ncbi.nlm.nih.gov/pubmed/35336268
http://dx.doi.org/10.3390/s22062097
_version_ 1784674749104783360
author Shimada, Tomoyasu
Nishikawa, Hiroki
Kong, Xiangbo
Tomiyama, Hiroyuki
author_facet Shimada, Tomoyasu
Nishikawa, Hiroki
Kong, Xiangbo
Tomiyama, Hiroyuki
author_sort Shimada, Tomoyasu
collection PubMed
description In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work.
format Online
Article
Text
id pubmed-8948838
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-89488382022-03-26 Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim Shimada, Tomoyasu Nishikawa, Hiroki Kong, Xiangbo Tomiyama, Hiroyuki Sensors (Basel) Article In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work. MDPI 2022-03-08 /pmc/articles/PMC8948838/ /pubmed/35336268 http://dx.doi.org/10.3390/s22062097 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Shimada, Tomoyasu
Nishikawa, Hiroki
Kong, Xiangbo
Tomiyama, Hiroyuki
Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title_full Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title_fullStr Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title_full_unstemmed Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title_short Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
title_sort pix2pix-based monocular depth estimation for drones with optical flow on airsim
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8948838/
https://www.ncbi.nlm.nih.gov/pubmed/35336268
http://dx.doi.org/10.3390/s22062097
work_keys_str_mv AT shimadatomoyasu pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim
AT nishikawahiroki pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim
AT kongxiangbo pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim
AT tomiyamahiroyuki pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim