Cargando…
Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer-Verlag
2011
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3214260/ https://www.ncbi.nlm.nih.gov/pubmed/22162982 http://dx.doi.org/10.1007/s12065-011-0066-z |
_version_ | 1782216224681230336 |
---|---|
author | Koppejan, Rogier Whiteson, Shimon |
author_facet | Koppejan, Rogier Whiteson, Shimon |
author_sort | Koppejan, Rogier |
collection | PubMed |
description | This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional state and action spaces, the need to explore efficiently and learn on-line poses unusual challenges. We propose and evaluate several methods for three increasingly challenging variations of the task, including the method that won first place in the 2008 Reinforcement Learning Competition. The results demonstrate that (1) neuroevolution can be effective for complex on-line reinforcement learning tasks such as generalized helicopter hovering, (2) neuroevolution excels at finding effective helicopter hovering policies but not at learning helicopter models, (3) due to the difficulty of learning reliable models, model-based approaches to helicopter hovering are feasible only when domain expertise is available to aid the design of a suitable model representation and (4) recent advances in efficient resampling can enable neuroevolution to tackle more aggressively generalized reinforcement learning tasks. |
format | Online Article Text |
id | pubmed-3214260 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2011 |
publisher | Springer-Verlag |
record_format | MEDLINE/PubMed |
spelling | pubmed-32142602011-12-09 Neuroevolutionary reinforcement learning for generalized control of simulated helicopters Koppejan, Rogier Whiteson, Shimon Evol Intell Research Paper This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional state and action spaces, the need to explore efficiently and learn on-line poses unusual challenges. We propose and evaluate several methods for three increasingly challenging variations of the task, including the method that won first place in the 2008 Reinforcement Learning Competition. The results demonstrate that (1) neuroevolution can be effective for complex on-line reinforcement learning tasks such as generalized helicopter hovering, (2) neuroevolution excels at finding effective helicopter hovering policies but not at learning helicopter models, (3) due to the difficulty of learning reliable models, model-based approaches to helicopter hovering are feasible only when domain expertise is available to aid the design of a suitable model representation and (4) recent advances in efficient resampling can enable neuroevolution to tackle more aggressively generalized reinforcement learning tasks. Springer-Verlag 2011-10-30 2011 /pmc/articles/PMC3214260/ /pubmed/22162982 http://dx.doi.org/10.1007/s12065-011-0066-z Text en © The Author(s) 2011 https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. |
spellingShingle | Research Paper Koppejan, Rogier Whiteson, Shimon Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title | Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title_full | Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title_fullStr | Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title_full_unstemmed | Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title_short | Neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
title_sort | neuroevolutionary reinforcement learning for generalized control of simulated helicopters |
topic | Research Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3214260/ https://www.ncbi.nlm.nih.gov/pubmed/22162982 http://dx.doi.org/10.1007/s12065-011-0066-z |
work_keys_str_mv | AT koppejanrogier neuroevolutionaryreinforcementlearningforgeneralizedcontrolofsimulatedhelicopters AT whitesonshimon neuroevolutionaryreinforcementlearningforgeneralizedcontrolofsimulatedhelicopters |