Cargando…

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

In core computer vision tasks, we have witnessed significant advances in object detection, localisation and tracking. However, there are currently no methods to detect, localize and track objects in road environments, and taking into account real-time constraints. In this paper, our objective is to...

Descripción completa

Detalles Bibliográficos
Autores principales: Mauri, Antoine, Khemmar, Redouane, Decoux, Benoit, Ragot, Nicolas, Rossi, Romain, Trabelsi, Rim, Boutteau, Rémi, Ertaud, Jean-Yves, Savatier, Xavier
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7014509/
https://www.ncbi.nlm.nih.gov/pubmed/31963641
http://dx.doi.org/10.3390/s20020532
_version_ 1783496647330758656
author Mauri, Antoine
Khemmar, Redouane
Decoux, Benoit
Ragot, Nicolas
Rossi, Romain
Trabelsi, Rim
Boutteau, Rémi
Ertaud, Jean-Yves
Savatier, Xavier
author_facet Mauri, Antoine
Khemmar, Redouane
Decoux, Benoit
Ragot, Nicolas
Rossi, Romain
Trabelsi, Rim
Boutteau, Rémi
Ertaud, Jean-Yves
Savatier, Xavier
author_sort Mauri, Antoine
collection PubMed
description In core computer vision tasks, we have witnessed significant advances in object detection, localisation and tracking. However, there are currently no methods to detect, localize and track objects in road environments, and taking into account real-time constraints. In this paper, our objective is to develop a deep learning multi object detection and tracking technique applied to road smart mobility. Firstly, we propose an effective detector-based on YOLOv3 which we adapt to our context. Subsequently, to localize successfully the detected objects, we put forward an adaptive method aiming to extract 3D information, i.e., depth maps. To do so, a comparative study is carried out taking into account two approaches: Monodepth2 for monocular vision and MADNEt for stereoscopic vision. These approaches are then evaluated over datasets containing depth information in order to discern the best solution that performs better in real-time conditions. Object tracking is necessary in order to mitigate the risks of collisions. Unlike traditional tracking approaches which require target initialization beforehand, our approach consists of using information from object detection and distance estimation to initialize targets and to track them later. Expressly, we propose here to improve SORT approach for 3D object tracking. We introduce an extended Kalman filter to better estimate the position of objects. Extensive experiments carried out on KITTI dataset prove that our proposal outperforms state-of-the-art approches.
format Online
Article
Text
id pubmed-7014509
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-70145092020-03-09 Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility Mauri, Antoine Khemmar, Redouane Decoux, Benoit Ragot, Nicolas Rossi, Romain Trabelsi, Rim Boutteau, Rémi Ertaud, Jean-Yves Savatier, Xavier Sensors (Basel) Article In core computer vision tasks, we have witnessed significant advances in object detection, localisation and tracking. However, there are currently no methods to detect, localize and track objects in road environments, and taking into account real-time constraints. In this paper, our objective is to develop a deep learning multi object detection and tracking technique applied to road smart mobility. Firstly, we propose an effective detector-based on YOLOv3 which we adapt to our context. Subsequently, to localize successfully the detected objects, we put forward an adaptive method aiming to extract 3D information, i.e., depth maps. To do so, a comparative study is carried out taking into account two approaches: Monodepth2 for monocular vision and MADNEt for stereoscopic vision. These approaches are then evaluated over datasets containing depth information in order to discern the best solution that performs better in real-time conditions. Object tracking is necessary in order to mitigate the risks of collisions. Unlike traditional tracking approaches which require target initialization beforehand, our approach consists of using information from object detection and distance estimation to initialize targets and to track them later. Expressly, we propose here to improve SORT approach for 3D object tracking. We introduce an extended Kalman filter to better estimate the position of objects. Extensive experiments carried out on KITTI dataset prove that our proposal outperforms state-of-the-art approches. MDPI 2020-01-18 /pmc/articles/PMC7014509/ /pubmed/31963641 http://dx.doi.org/10.3390/s20020532 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Mauri, Antoine
Khemmar, Redouane
Decoux, Benoit
Ragot, Nicolas
Rossi, Romain
Trabelsi, Rim
Boutteau, Rémi
Ertaud, Jean-Yves
Savatier, Xavier
Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title_full Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title_fullStr Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title_full_unstemmed Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title_short Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility
title_sort deep learning for real-time 3d multi-object detection, localisation, and tracking: application to smart mobility
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7014509/
https://www.ncbi.nlm.nih.gov/pubmed/31963641
http://dx.doi.org/10.3390/s20020532
work_keys_str_mv AT mauriantoine deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT khemmarredouane deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT decouxbenoit deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT ragotnicolas deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT rossiromain deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT trabelsirim deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT boutteauremi deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT ertaudjeanyves deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility
AT savatierxavier deeplearningforrealtime3dmultiobjectdetectionlocalisationandtrackingapplicationtosmartmobility