Cargando…

Neuroevolutionary reinforcement learning for generalized control of simulated helicopters

This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional...

Descripción completa

Detalles Bibliográficos
Autores principales: Koppejan, Rogier, Whiteson, Shimon
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer-Verlag 2011
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3214260/
https://www.ncbi.nlm.nih.gov/pubmed/22162982
http://dx.doi.org/10.1007/s12065-011-0066-z
_version_ 1782216224681230336
author Koppejan, Rogier
Whiteson, Shimon
author_facet Koppejan, Rogier
Whiteson, Shimon
author_sort Koppejan, Rogier
collection PubMed
description This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional state and action spaces, the need to explore efficiently and learn on-line poses unusual challenges. We propose and evaluate several methods for three increasingly challenging variations of the task, including the method that won first place in the 2008 Reinforcement Learning Competition. The results demonstrate that (1) neuroevolution can be effective for complex on-line reinforcement learning tasks such as generalized helicopter hovering, (2) neuroevolution excels at finding effective helicopter hovering policies but not at learning helicopter models, (3) due to the difficulty of learning reliable models, model-based approaches to helicopter hovering are feasible only when domain expertise is available to aid the design of a suitable model representation and (4) recent advances in efficient resampling can enable neuroevolution to tackle more aggressively generalized reinforcement learning tasks.
format Online
Article
Text
id pubmed-3214260
institution National Center for Biotechnology Information
language English
publishDate 2011
publisher Springer-Verlag
record_format MEDLINE/PubMed
spelling pubmed-32142602011-12-09 Neuroevolutionary reinforcement learning for generalized control of simulated helicopters Koppejan, Rogier Whiteson, Shimon Evol Intell Research Paper This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional state and action spaces, the need to explore efficiently and learn on-line poses unusual challenges. We propose and evaluate several methods for three increasingly challenging variations of the task, including the method that won first place in the 2008 Reinforcement Learning Competition. The results demonstrate that (1) neuroevolution can be effective for complex on-line reinforcement learning tasks such as generalized helicopter hovering, (2) neuroevolution excels at finding effective helicopter hovering policies but not at learning helicopter models, (3) due to the difficulty of learning reliable models, model-based approaches to helicopter hovering are feasible only when domain expertise is available to aid the design of a suitable model representation and (4) recent advances in efficient resampling can enable neuroevolution to tackle more aggressively generalized reinforcement learning tasks. Springer-Verlag 2011-10-30 2011 /pmc/articles/PMC3214260/ /pubmed/22162982 http://dx.doi.org/10.1007/s12065-011-0066-z Text en © The Author(s) 2011 https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
spellingShingle Research Paper
Koppejan, Rogier
Whiteson, Shimon
Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title_full Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title_fullStr Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title_full_unstemmed Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title_short Neuroevolutionary reinforcement learning for generalized control of simulated helicopters
title_sort neuroevolutionary reinforcement learning for generalized control of simulated helicopters
topic Research Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3214260/
https://www.ncbi.nlm.nih.gov/pubmed/22162982
http://dx.doi.org/10.1007/s12065-011-0066-z
work_keys_str_mv AT koppejanrogier neuroevolutionaryreinforcementlearningforgeneralizedcontrolofsimulatedhelicopters
AT whitesonshimon neuroevolutionaryreinforcementlearningforgeneralizedcontrolofsimulatedhelicopters