Cargando…

A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment

To control autonomous vehicles (AVs) in urban unsignalized intersections is a challenging problem, especially in a hybrid traffic environment where self-driving vehicles coexist with human driving vehicles. In this study, a coordinated control method with proximal policy optimization (PPO) in Vehicl...

Descripción completa

Detalles Bibliográficos
Autores principales: Shi, Yanjun, Liu, Yuanzhuo, Qi, Yuhan, Han, Qiaomei
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8840198/
https://www.ncbi.nlm.nih.gov/pubmed/35161523
http://dx.doi.org/10.3390/s22030779
_version_ 1784650560067076096
author Shi, Yanjun
Liu, Yuanzhuo
Qi, Yuhan
Han, Qiaomei
author_facet Shi, Yanjun
Liu, Yuanzhuo
Qi, Yuhan
Han, Qiaomei
author_sort Shi, Yanjun
collection PubMed
description To control autonomous vehicles (AVs) in urban unsignalized intersections is a challenging problem, especially in a hybrid traffic environment where self-driving vehicles coexist with human driving vehicles. In this study, a coordinated control method with proximal policy optimization (PPO) in Vehicle-Road-Cloud Integration System (VRCIS) is proposed, where this control problem is formulated as a reinforcement learning (RL) problem. In this system, vehicles and everything (V2X) was used to keep communication between vehicles, and vehicle wireless technology can detect vehicles that use vehicles and infrastructure (V2I) wireless communication, thereby achieving a cost-efficient method. Then, the connected and autonomous vehicle (CAV) defined in the VRCIS learned a policy to adapt to human driving vehicles (HDVs) across the intersection safely by reinforcement learning (RL). We have developed a valid, scalable RL framework, which can communicate topologies that may be dynamic traffic. Then, state, action and reward of RL are designed according to urban unsignalized intersection problem. Finally, how to deploy within the RL framework was described, and several experiments with this framework were undertaken to verify the effectiveness of the proposed method.
format Online
Article
Text
id pubmed-8840198
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-88401982022-02-13 A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment Shi, Yanjun Liu, Yuanzhuo Qi, Yuhan Han, Qiaomei Sensors (Basel) Article To control autonomous vehicles (AVs) in urban unsignalized intersections is a challenging problem, especially in a hybrid traffic environment where self-driving vehicles coexist with human driving vehicles. In this study, a coordinated control method with proximal policy optimization (PPO) in Vehicle-Road-Cloud Integration System (VRCIS) is proposed, where this control problem is formulated as a reinforcement learning (RL) problem. In this system, vehicles and everything (V2X) was used to keep communication between vehicles, and vehicle wireless technology can detect vehicles that use vehicles and infrastructure (V2I) wireless communication, thereby achieving a cost-efficient method. Then, the connected and autonomous vehicle (CAV) defined in the VRCIS learned a policy to adapt to human driving vehicles (HDVs) across the intersection safely by reinforcement learning (RL). We have developed a valid, scalable RL framework, which can communicate topologies that may be dynamic traffic. Then, state, action and reward of RL are designed according to urban unsignalized intersection problem. Finally, how to deploy within the RL framework was described, and several experiments with this framework were undertaken to verify the effectiveness of the proposed method. MDPI 2022-01-20 /pmc/articles/PMC8840198/ /pubmed/35161523 http://dx.doi.org/10.3390/s22030779 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Shi, Yanjun
Liu, Yuanzhuo
Qi, Yuhan
Han, Qiaomei
A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title_full A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title_fullStr A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title_full_unstemmed A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title_short A Control Method with Reinforcement Learning for Urban Un-Signalized Intersection in Hybrid Traffic Environment
title_sort control method with reinforcement learning for urban un-signalized intersection in hybrid traffic environment
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8840198/
https://www.ncbi.nlm.nih.gov/pubmed/35161523
http://dx.doi.org/10.3390/s22030779
work_keys_str_mv AT shiyanjun acontrolmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT liuyuanzhuo acontrolmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT qiyuhan acontrolmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT hanqiaomei acontrolmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT shiyanjun controlmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT liuyuanzhuo controlmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT qiyuhan controlmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment
AT hanqiaomei controlmethodwithreinforcementlearningforurbanunsignalizedintersectioninhybridtrafficenvironment