Cargando…
Magnetic control of tokamak plasmas through deep reinforcement learning
Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control us...
Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/ https://www.ncbi.nlm.nih.gov/pubmed/35173339 http://dx.doi.org/10.1038/s41586-021-04301-9 |
_version_ | 1784652541509763072 |
---|---|
author | Degrave, Jonas Felici, Federico Buchli, Jonas Neunert, Michael Tracey, Brendan Carpanese, Francesco Ewalds, Timo Hafner, Roland Abdolmaleki, Abbas de las Casas, Diego Donner, Craig Fritz, Leslie Galperti, Cristian Huber, Andrea Keeling, James Tsimpoukelli, Maria Kay, Jackie Merle, Antoine Moret, Jean-Marc Noury, Seb Pesamosca, Federico Pfau, David Sauter, Olivier Sommariva, Cristian Coda, Stefano Duval, Basil Fasoli, Ambrogio Kohli, Pushmeet Kavukcuoglu, Koray Hassabis, Demis Riedmiller, Martin |
author_facet | Degrave, Jonas Felici, Federico Buchli, Jonas Neunert, Michael Tracey, Brendan Carpanese, Francesco Ewalds, Timo Hafner, Roland Abdolmaleki, Abbas de las Casas, Diego Donner, Craig Fritz, Leslie Galperti, Cristian Huber, Andrea Keeling, James Tsimpoukelli, Maria Kay, Jackie Merle, Antoine Moret, Jean-Marc Noury, Seb Pesamosca, Federico Pfau, David Sauter, Olivier Sommariva, Cristian Coda, Stefano Duval, Basil Fasoli, Ambrogio Kohli, Pushmeet Kavukcuoglu, Koray Hassabis, Demis Riedmiller, Martin |
author_sort | Degrave, Jonas |
collection | PubMed |
description | Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable(1,2), including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. |
format | Online Article Text |
id | pubmed-8850200 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-88502002022-03-22 Magnetic control of tokamak plasmas through deep reinforcement learning Degrave, Jonas Felici, Federico Buchli, Jonas Neunert, Michael Tracey, Brendan Carpanese, Francesco Ewalds, Timo Hafner, Roland Abdolmaleki, Abbas de las Casas, Diego Donner, Craig Fritz, Leslie Galperti, Cristian Huber, Andrea Keeling, James Tsimpoukelli, Maria Kay, Jackie Merle, Antoine Moret, Jean-Marc Noury, Seb Pesamosca, Federico Pfau, David Sauter, Olivier Sommariva, Cristian Coda, Stefano Duval, Basil Fasoli, Ambrogio Kohli, Pushmeet Kavukcuoglu, Koray Hassabis, Demis Riedmiller, Martin Nature Article Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable(1,2), including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. Nature Publishing Group UK 2022-02-16 2022 /pmc/articles/PMC8850200/ /pubmed/35173339 http://dx.doi.org/10.1038/s41586-021-04301-9 Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Degrave, Jonas Felici, Federico Buchli, Jonas Neunert, Michael Tracey, Brendan Carpanese, Francesco Ewalds, Timo Hafner, Roland Abdolmaleki, Abbas de las Casas, Diego Donner, Craig Fritz, Leslie Galperti, Cristian Huber, Andrea Keeling, James Tsimpoukelli, Maria Kay, Jackie Merle, Antoine Moret, Jean-Marc Noury, Seb Pesamosca, Federico Pfau, David Sauter, Olivier Sommariva, Cristian Coda, Stefano Duval, Basil Fasoli, Ambrogio Kohli, Pushmeet Kavukcuoglu, Koray Hassabis, Demis Riedmiller, Martin Magnetic control of tokamak plasmas through deep reinforcement learning |
title | Magnetic control of tokamak plasmas through deep reinforcement learning |
title_full | Magnetic control of tokamak plasmas through deep reinforcement learning |
title_fullStr | Magnetic control of tokamak plasmas through deep reinforcement learning |
title_full_unstemmed | Magnetic control of tokamak plasmas through deep reinforcement learning |
title_short | Magnetic control of tokamak plasmas through deep reinforcement learning |
title_sort | magnetic control of tokamak plasmas through deep reinforcement learning |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/ https://www.ncbi.nlm.nih.gov/pubmed/35173339 http://dx.doi.org/10.1038/s41586-021-04301-9 |
work_keys_str_mv | AT degravejonas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT felicifederico magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT buchlijonas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT neunertmichael magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT traceybrendan magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT carpanesefrancesco magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT ewaldstimo magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT hafnerroland magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT abdolmalekiabbas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT delascasasdiego magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT donnercraig magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT fritzleslie magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT galperticristian magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT huberandrea magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT keelingjames magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT tsimpoukellimaria magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT kayjackie magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT merleantoine magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT moretjeanmarc magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT nouryseb magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT pesamoscafederico magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT pfaudavid magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT sauterolivier magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT sommarivacristian magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT codastefano magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT duvalbasil magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT fasoliambrogio magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT kohlipushmeet magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT kavukcuoglukoray magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT hassabisdemis magneticcontroloftokamakplasmasthroughdeepreinforcementlearning AT riedmillermartin magneticcontroloftokamakplasmasthroughdeepreinforcementlearning |