Cargando…
Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control
We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even w...
Autor principal: | |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10378142/ https://www.ncbi.nlm.nih.gov/pubmed/37509929 http://dx.doi.org/10.3390/e25070982 |
_version_ | 1785079692699631616 |
---|---|
author | Korecki, Marcin |
author_facet | Korecki, Marcin |
author_sort | Korecki, Marcin |
collection | PubMed |
description | We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods. |
format | Online Article Text |
id | pubmed-10378142 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-103781422023-07-29 Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control Korecki, Marcin Entropy (Basel) Article We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods. MDPI 2023-06-27 /pmc/articles/PMC10378142/ /pubmed/37509929 http://dx.doi.org/10.3390/e25070982 Text en © 2023 by the author. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Korecki, Marcin Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title | Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title_full | Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title_fullStr | Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title_full_unstemmed | Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title_short | Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control |
title_sort | deep reinforcement meta-learning and self-organization in complex systems: applications to traffic signal control |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10378142/ https://www.ncbi.nlm.nih.gov/pubmed/37509929 http://dx.doi.org/10.3390/e25070982 |
work_keys_str_mv | AT koreckimarcin deepreinforcementmetalearningandselforganizationincomplexsystemsapplicationstotrafficsignalcontrol |