Cargando…
Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights si...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8948838/ https://www.ncbi.nlm.nih.gov/pubmed/35336268 http://dx.doi.org/10.3390/s22062097 |
_version_ | 1784674749104783360 |
---|---|
author | Shimada, Tomoyasu Nishikawa, Hiroki Kong, Xiangbo Tomiyama, Hiroyuki |
author_facet | Shimada, Tomoyasu Nishikawa, Hiroki Kong, Xiangbo Tomiyama, Hiroyuki |
author_sort | Shimada, Tomoyasu |
collection | PubMed |
description | In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work. |
format | Online Article Text |
id | pubmed-8948838 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-89488382022-03-26 Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim Shimada, Tomoyasu Nishikawa, Hiroki Kong, Xiangbo Tomiyama, Hiroyuki Sensors (Basel) Article In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work. MDPI 2022-03-08 /pmc/articles/PMC8948838/ /pubmed/35336268 http://dx.doi.org/10.3390/s22062097 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Shimada, Tomoyasu Nishikawa, Hiroki Kong, Xiangbo Tomiyama, Hiroyuki Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title_full | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title_fullStr | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title_full_unstemmed | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title_short | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
title_sort | pix2pix-based monocular depth estimation for drones with optical flow on airsim |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8948838/ https://www.ncbi.nlm.nih.gov/pubmed/35336268 http://dx.doi.org/10.3390/s22062097 |
work_keys_str_mv | AT shimadatomoyasu pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT nishikawahiroki pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT kongxiangbo pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT tomiyamahiroyuki pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim |