Cargando…

Obstacle Detection Method Based on RSU and Vehicle Camera Fusion

Road obstacle detection is an important component of intelligent assisted driving technology. Existing obstacle detection methods ignore the important direction of generalized obstacle detection. This paper proposes an obstacle detection method based on the fusion of roadside units and vehicle mount...

Descripción completa

Detalles Bibliográficos
Autores principales: Ding, Shaohong, Xu, Yi, Zhang, Qian, Yu, Jinxin, Sun, Teng, Ni, Juan, Shi, Shuyue, Kong, Xiangcun, Zhu, Ruoyu, Wang, Liming, Wang, Pengwei
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10220615/
https://www.ncbi.nlm.nih.gov/pubmed/37430834
http://dx.doi.org/10.3390/s23104920
_version_ 1785049260225462272
author Ding, Shaohong
Xu, Yi
Zhang, Qian
Yu, Jinxin
Sun, Teng
Ni, Juan
Shi, Shuyue
Kong, Xiangcun
Zhu, Ruoyu
Wang, Liming
Wang, Pengwei
author_facet Ding, Shaohong
Xu, Yi
Zhang, Qian
Yu, Jinxin
Sun, Teng
Ni, Juan
Shi, Shuyue
Kong, Xiangcun
Zhu, Ruoyu
Wang, Liming
Wang, Pengwei
author_sort Ding, Shaohong
collection PubMed
description Road obstacle detection is an important component of intelligent assisted driving technology. Existing obstacle detection methods ignore the important direction of generalized obstacle detection. This paper proposes an obstacle detection method based on the fusion of roadside units and vehicle mounted cameras and illustrates the feasibility of a combined monocular camera inertial measurement unit (IMU) and roadside unit (RSU) detection method. A generalized obstacle detection method based on vision IMU is combined with a roadside unit obstacle detection method based on a background difference method to achieve generalized obstacle classification while reducing the spatial complexity of the detection area. In the generalized obstacle recognition stage, a VIDAR (Vision-IMU based identification and ranging) -based generalized obstacle recognition method is proposed. The problem of the low accuracy of obstacle information acquisition in the driving environment where generalized obstacles exist is solved. For generalized obstacles that cannot be detected by the roadside unit, VIDAR obstacle detection is performed on the target generalized obstacles through the vehicle terminal camera, and the detection result information is transmitted to the roadside device terminal through the UDP (User Data Protocol) protocol to achieve obstacle recognition and pseudo-obstacle removal, thereby reducing the error recognition rate of generalized obstacles. In this paper, pseudo-obstacles, obstacles with a certain height less than the maximum passing height of the vehicle, and obstacles with a height greater than the maximum passing height of the vehicle are defined as generalized obstacles. Pseudo-obstacles refer to non-height objects that appear to be “patches” on the imaging interface obtained by visual sensors and obstacles with a height less than the maximum passing height of the vehicle. VIDAR is a vision-IMU-based detection and ranging method. IMU is used to obtain the distance and pose of the camera movement, and through the inverse perspective transformation, it can calculate the height of the object in the image. The VIDAR-based obstacle detection method, the roadside unit-based obstacle detection method, YOLOv5 (You Only Look Once version 5), and the method proposed in this paper were applied to outdoor comparison experiments. The results show that the accuracy of the method is improved by 2.3%, 17.4%, and 1.8%, respectively, compared with the other four methods. Compared with the roadside unit obstacle detection method, the speed of obstacle detection is improved by 1.1%. The experimental results show that the method can expand the detection range of road vehicles based on the vehicle obstacle detection method and can quickly and effectively eliminate false obstacle information on the road.
format Online
Article
Text
id pubmed-10220615
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-102206152023-05-28 Obstacle Detection Method Based on RSU and Vehicle Camera Fusion Ding, Shaohong Xu, Yi Zhang, Qian Yu, Jinxin Sun, Teng Ni, Juan Shi, Shuyue Kong, Xiangcun Zhu, Ruoyu Wang, Liming Wang, Pengwei Sensors (Basel) Article Road obstacle detection is an important component of intelligent assisted driving technology. Existing obstacle detection methods ignore the important direction of generalized obstacle detection. This paper proposes an obstacle detection method based on the fusion of roadside units and vehicle mounted cameras and illustrates the feasibility of a combined monocular camera inertial measurement unit (IMU) and roadside unit (RSU) detection method. A generalized obstacle detection method based on vision IMU is combined with a roadside unit obstacle detection method based on a background difference method to achieve generalized obstacle classification while reducing the spatial complexity of the detection area. In the generalized obstacle recognition stage, a VIDAR (Vision-IMU based identification and ranging) -based generalized obstacle recognition method is proposed. The problem of the low accuracy of obstacle information acquisition in the driving environment where generalized obstacles exist is solved. For generalized obstacles that cannot be detected by the roadside unit, VIDAR obstacle detection is performed on the target generalized obstacles through the vehicle terminal camera, and the detection result information is transmitted to the roadside device terminal through the UDP (User Data Protocol) protocol to achieve obstacle recognition and pseudo-obstacle removal, thereby reducing the error recognition rate of generalized obstacles. In this paper, pseudo-obstacles, obstacles with a certain height less than the maximum passing height of the vehicle, and obstacles with a height greater than the maximum passing height of the vehicle are defined as generalized obstacles. Pseudo-obstacles refer to non-height objects that appear to be “patches” on the imaging interface obtained by visual sensors and obstacles with a height less than the maximum passing height of the vehicle. VIDAR is a vision-IMU-based detection and ranging method. IMU is used to obtain the distance and pose of the camera movement, and through the inverse perspective transformation, it can calculate the height of the object in the image. The VIDAR-based obstacle detection method, the roadside unit-based obstacle detection method, YOLOv5 (You Only Look Once version 5), and the method proposed in this paper were applied to outdoor comparison experiments. The results show that the accuracy of the method is improved by 2.3%, 17.4%, and 1.8%, respectively, compared with the other four methods. Compared with the roadside unit obstacle detection method, the speed of obstacle detection is improved by 1.1%. The experimental results show that the method can expand the detection range of road vehicles based on the vehicle obstacle detection method and can quickly and effectively eliminate false obstacle information on the road. MDPI 2023-05-19 /pmc/articles/PMC10220615/ /pubmed/37430834 http://dx.doi.org/10.3390/s23104920 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Ding, Shaohong
Xu, Yi
Zhang, Qian
Yu, Jinxin
Sun, Teng
Ni, Juan
Shi, Shuyue
Kong, Xiangcun
Zhu, Ruoyu
Wang, Liming
Wang, Pengwei
Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title_full Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title_fullStr Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title_full_unstemmed Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title_short Obstacle Detection Method Based on RSU and Vehicle Camera Fusion
title_sort obstacle detection method based on rsu and vehicle camera fusion
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10220615/
https://www.ncbi.nlm.nih.gov/pubmed/37430834
http://dx.doi.org/10.3390/s23104920
work_keys_str_mv AT dingshaohong obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT xuyi obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT zhangqian obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT yujinxin obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT sunteng obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT nijuan obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT shishuyue obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT kongxiangcun obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT zhuruoyu obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT wangliming obstacledetectionmethodbasedonrsuandvehiclecamerafusion
AT wangpengwei obstacledetectionmethodbasedonrsuandvehiclecamerafusion