Cargando…

Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera

In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar...

Descripción completa

Detalles Bibliográficos
Autores principales: Lv, Pengfei, Wang, Bingqing, Cheng, Feng, Xue, Jinlin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9824033/
https://www.ncbi.nlm.nih.gov/pubmed/36616828
http://dx.doi.org/10.3390/s23010230
_version_ 1784866309305008128
author Lv, Pengfei
Wang, Bingqing
Cheng, Feng
Xue, Jinlin
author_facet Lv, Pengfei
Wang, Bingqing
Cheng, Feng
Xue, Jinlin
author_sort Lv, Pengfei
collection PubMed
description In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar in range and speed measurement and the camera in type identification and lateral localization, a decision-level fusion algorithm was designed for the mmWave radar and camera information, and the global nearest neighbor method was used for data association. Then, the effective target sequences of the mmWave radar and the camera with successful data association were weighted to output, and the output included more accurate target orientation, longitudinal speed, and category. For the unassociated sequences, they were tracked as new targets by using the extended Kalman filter algorithm and were processed and output during the effective life cycle. Lastly, an experimental platform based on a tractor was built to verify the effectiveness of the proposed association detection method. The obstacle detection test was conducted under the ROS environment after solving the external parameters of the mmWave radar and the internal and external parameters of the camera. The test results show that the correct detection rate of obstacles reaches 86.18%, which is higher than that of a single camera with 62.47%. Furthermore, through the contrast experiment of the sensor fusion algorithms, the detection accuracy of the decision level fusion algorithm was 95.19%, which was higher than 4.38% and 6.63% compared with feature level and data level fusion, respectively.
format Online
Article
Text
id pubmed-9824033
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-98240332023-01-08 Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera Lv, Pengfei Wang, Bingqing Cheng, Feng Xue, Jinlin Sensors (Basel) Article In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar in range and speed measurement and the camera in type identification and lateral localization, a decision-level fusion algorithm was designed for the mmWave radar and camera information, and the global nearest neighbor method was used for data association. Then, the effective target sequences of the mmWave radar and the camera with successful data association were weighted to output, and the output included more accurate target orientation, longitudinal speed, and category. For the unassociated sequences, they were tracked as new targets by using the extended Kalman filter algorithm and were processed and output during the effective life cycle. Lastly, an experimental platform based on a tractor was built to verify the effectiveness of the proposed association detection method. The obstacle detection test was conducted under the ROS environment after solving the external parameters of the mmWave radar and the internal and external parameters of the camera. The test results show that the correct detection rate of obstacles reaches 86.18%, which is higher than that of a single camera with 62.47%. Furthermore, through the contrast experiment of the sensor fusion algorithms, the detection accuracy of the decision level fusion algorithm was 95.19%, which was higher than 4.38% and 6.63% compared with feature level and data level fusion, respectively. MDPI 2022-12-26 /pmc/articles/PMC9824033/ /pubmed/36616828 http://dx.doi.org/10.3390/s23010230 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Lv, Pengfei
Wang, Bingqing
Cheng, Feng
Xue, Jinlin
Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title_full Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title_fullStr Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title_full_unstemmed Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title_short Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera
title_sort multi-objective association detection of farmland obstacles based on information fusion of millimeter wave radar and camera
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9824033/
https://www.ncbi.nlm.nih.gov/pubmed/36616828
http://dx.doi.org/10.3390/s23010230
work_keys_str_mv AT lvpengfei multiobjectiveassociationdetectionoffarmlandobstaclesbasedoninformationfusionofmillimeterwaveradarandcamera
AT wangbingqing multiobjectiveassociationdetectionoffarmlandobstaclesbasedoninformationfusionofmillimeterwaveradarandcamera
AT chengfeng multiobjectiveassociationdetectionoffarmlandobstaclesbasedoninformationfusionofmillimeterwaveradarandcamera
AT xuejinlin multiobjectiveassociationdetectionoffarmlandobstaclesbasedoninformationfusionofmillimeterwaveradarandcamera