Cargando…

Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control

We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even w...

Descripción completa

Detalles Bibliográficos
Autor principal: Korecki, Marcin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10378142/
https://www.ncbi.nlm.nih.gov/pubmed/37509929
http://dx.doi.org/10.3390/e25070982
_version_ 1785079692699631616
author Korecki, Marcin
author_facet Korecki, Marcin
author_sort Korecki, Marcin
collection PubMed
description We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods.
format Online
Article
Text
id pubmed-10378142
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-103781422023-07-29 Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control Korecki, Marcin Entropy (Basel) Article We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods. MDPI 2023-06-27 /pmc/articles/PMC10378142/ /pubmed/37509929 http://dx.doi.org/10.3390/e25070982 Text en © 2023 by the author. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Korecki, Marcin
Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title_full Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title_fullStr Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title_full_unstemmed Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title_short Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
title_sort deep reinforcement meta-learning and self-organization in complex systems: applications to traffic signal control
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10378142/
https://www.ncbi.nlm.nih.gov/pubmed/37509929
http://dx.doi.org/10.3390/e25070982
work_keys_str_mv AT koreckimarcin deepreinforcementmetalearningandselforganizationincomplexsystemsapplicationstotrafficsignalcontrol