Cargando…
3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions
The role of sensors such as cameras or LiDAR (Light Detection and Ranging) is crucial for the environmental awareness of self-driving cars. However, the data collected from these sensors are subject to distortions in extreme weather conditions such as fog, rain, and snow. This issue could lead to ma...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8540926/ https://www.ncbi.nlm.nih.gov/pubmed/34695925 http://dx.doi.org/10.3390/s21206711 |
_version_ | 1784589105110188032 |
---|---|
author | Mai, Nguyen Anh Minh Duthon, Pierre Khoudour, Louahdi Crouzil, Alain Velastin, Sergio A. |
author_facet | Mai, Nguyen Anh Minh Duthon, Pierre Khoudour, Louahdi Crouzil, Alain Velastin, Sergio A. |
author_sort | Mai, Nguyen Anh Minh |
collection | PubMed |
description | The role of sensors such as cameras or LiDAR (Light Detection and Ranging) is crucial for the environmental awareness of self-driving cars. However, the data collected from these sensors are subject to distortions in extreme weather conditions such as fog, rain, and snow. This issue could lead to many safety problems while operating a self-driving vehicle. The purpose of this study is to analyze the effects of fog on the detection of objects in driving scenes and then to propose methods for improvement. Collecting and processing data in adverse weather conditions is often more difficult than data in good weather conditions. Hence, a synthetic dataset that can simulate bad weather conditions is a good choice to validate a method, as it is simpler and more economical, before working with a real dataset. In this paper, we apply fog synthesis on the public KITTI dataset to generate the Multifog KITTI dataset for both images and point clouds. In terms of processing tasks, we test our previous 3D object detector based on LiDAR and camera, named the Spare LiDAR Stereo Fusion Network (SLS-Fusion), to see how it is affected by foggy weather conditions. We propose to train using both the original dataset and the augmented dataset to improve performance in foggy weather conditions while keeping good performance under normal conditions. We conducted experiments on the KITTI and the proposed Multifog KITTI datasets which show that, before any improvement, performance is reduced by 42.67% in 3D object detection for Moderate objects in foggy weather conditions. By using a specific strategy of training, the results significantly improved by 26.72% and keep performing quite well on the original dataset with a drop only of 8.23%. In summary, fog often causes the failure of 3D detection on driving scenes. By additional training with the augmented dataset, we significantly improve the performance of the proposed 3D object detection algorithm for self-driving cars in foggy weather conditions. |
format | Online Article Text |
id | pubmed-8540926 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-85409262021-10-24 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions Mai, Nguyen Anh Minh Duthon, Pierre Khoudour, Louahdi Crouzil, Alain Velastin, Sergio A. Sensors (Basel) Article The role of sensors such as cameras or LiDAR (Light Detection and Ranging) is crucial for the environmental awareness of self-driving cars. However, the data collected from these sensors are subject to distortions in extreme weather conditions such as fog, rain, and snow. This issue could lead to many safety problems while operating a self-driving vehicle. The purpose of this study is to analyze the effects of fog on the detection of objects in driving scenes and then to propose methods for improvement. Collecting and processing data in adverse weather conditions is often more difficult than data in good weather conditions. Hence, a synthetic dataset that can simulate bad weather conditions is a good choice to validate a method, as it is simpler and more economical, before working with a real dataset. In this paper, we apply fog synthesis on the public KITTI dataset to generate the Multifog KITTI dataset for both images and point clouds. In terms of processing tasks, we test our previous 3D object detector based on LiDAR and camera, named the Spare LiDAR Stereo Fusion Network (SLS-Fusion), to see how it is affected by foggy weather conditions. We propose to train using both the original dataset and the augmented dataset to improve performance in foggy weather conditions while keeping good performance under normal conditions. We conducted experiments on the KITTI and the proposed Multifog KITTI datasets which show that, before any improvement, performance is reduced by 42.67% in 3D object detection for Moderate objects in foggy weather conditions. By using a specific strategy of training, the results significantly improved by 26.72% and keep performing quite well on the original dataset with a drop only of 8.23%. In summary, fog often causes the failure of 3D detection on driving scenes. By additional training with the augmented dataset, we significantly improve the performance of the proposed 3D object detection algorithm for self-driving cars in foggy weather conditions. MDPI 2021-10-09 /pmc/articles/PMC8540926/ /pubmed/34695925 http://dx.doi.org/10.3390/s21206711 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Mai, Nguyen Anh Minh Duthon, Pierre Khoudour, Louahdi Crouzil, Alain Velastin, Sergio A. 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title | 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title_full | 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title_fullStr | 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title_full_unstemmed | 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title_short | 3D Object Detection with SLS-Fusion Network in Foggy Weather Conditions |
title_sort | 3d object detection with sls-fusion network in foggy weather conditions |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8540926/ https://www.ncbi.nlm.nih.gov/pubmed/34695925 http://dx.doi.org/10.3390/s21206711 |
work_keys_str_mv | AT mainguyenanhminh 3dobjectdetectionwithslsfusionnetworkinfoggyweatherconditions AT duthonpierre 3dobjectdetectionwithslsfusionnetworkinfoggyweatherconditions AT khoudourlouahdi 3dobjectdetectionwithslsfusionnetworkinfoggyweatherconditions AT crouzilalain 3dobjectdetectionwithslsfusionnetworkinfoggyweatherconditions AT velastinsergioa 3dobjectdetectionwithslsfusionnetworkinfoggyweatherconditions |