Cargando…

Magnetic control of tokamak plasmas through deep reinforcement learning

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control us...

Descripción completa

Detalles Bibliográficos
Autores principales: Degrave, Jonas, Felici, Federico, Buchli, Jonas, Neunert, Michael, Tracey, Brendan, Carpanese, Francesco, Ewalds, Timo, Hafner, Roland, Abdolmaleki, Abbas, de las Casas, Diego, Donner, Craig, Fritz, Leslie, Galperti, Cristian, Huber, Andrea, Keeling, James, Tsimpoukelli, Maria, Kay, Jackie, Merle, Antoine, Moret, Jean-Marc, Noury, Seb, Pesamosca, Federico, Pfau, David, Sauter, Olivier, Sommariva, Cristian, Coda, Stefano, Duval, Basil, Fasoli, Ambrogio, Kohli, Pushmeet, Kavukcuoglu, Koray, Hassabis, Demis, Riedmiller, Martin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/
https://www.ncbi.nlm.nih.gov/pubmed/35173339
http://dx.doi.org/10.1038/s41586-021-04301-9
_version_ 1784652541509763072
author Degrave, Jonas
Felici, Federico
Buchli, Jonas
Neunert, Michael
Tracey, Brendan
Carpanese, Francesco
Ewalds, Timo
Hafner, Roland
Abdolmaleki, Abbas
de las Casas, Diego
Donner, Craig
Fritz, Leslie
Galperti, Cristian
Huber, Andrea
Keeling, James
Tsimpoukelli, Maria
Kay, Jackie
Merle, Antoine
Moret, Jean-Marc
Noury, Seb
Pesamosca, Federico
Pfau, David
Sauter, Olivier
Sommariva, Cristian
Coda, Stefano
Duval, Basil
Fasoli, Ambrogio
Kohli, Pushmeet
Kavukcuoglu, Koray
Hassabis, Demis
Riedmiller, Martin
author_facet Degrave, Jonas
Felici, Federico
Buchli, Jonas
Neunert, Michael
Tracey, Brendan
Carpanese, Francesco
Ewalds, Timo
Hafner, Roland
Abdolmaleki, Abbas
de las Casas, Diego
Donner, Craig
Fritz, Leslie
Galperti, Cristian
Huber, Andrea
Keeling, James
Tsimpoukelli, Maria
Kay, Jackie
Merle, Antoine
Moret, Jean-Marc
Noury, Seb
Pesamosca, Federico
Pfau, David
Sauter, Olivier
Sommariva, Cristian
Coda, Stefano
Duval, Basil
Fasoli, Ambrogio
Kohli, Pushmeet
Kavukcuoglu, Koray
Hassabis, Demis
Riedmiller, Martin
author_sort Degrave, Jonas
collection PubMed
description Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable(1,2), including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.
format Online
Article
Text
id pubmed-8850200
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-88502002022-03-22 Magnetic control of tokamak plasmas through deep reinforcement learning Degrave, Jonas Felici, Federico Buchli, Jonas Neunert, Michael Tracey, Brendan Carpanese, Francesco Ewalds, Timo Hafner, Roland Abdolmaleki, Abbas de las Casas, Diego Donner, Craig Fritz, Leslie Galperti, Cristian Huber, Andrea Keeling, James Tsimpoukelli, Maria Kay, Jackie Merle, Antoine Moret, Jean-Marc Noury, Seb Pesamosca, Federico Pfau, David Sauter, Olivier Sommariva, Cristian Coda, Stefano Duval, Basil Fasoli, Ambrogio Kohli, Pushmeet Kavukcuoglu, Koray Hassabis, Demis Riedmiller, Martin Nature Article Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable(1,2), including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. Nature Publishing Group UK 2022-02-16 2022 /pmc/articles/PMC8850200/ /pubmed/35173339 http://dx.doi.org/10.1038/s41586-021-04301-9 Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Degrave, Jonas
Felici, Federico
Buchli, Jonas
Neunert, Michael
Tracey, Brendan
Carpanese, Francesco
Ewalds, Timo
Hafner, Roland
Abdolmaleki, Abbas
de las Casas, Diego
Donner, Craig
Fritz, Leslie
Galperti, Cristian
Huber, Andrea
Keeling, James
Tsimpoukelli, Maria
Kay, Jackie
Merle, Antoine
Moret, Jean-Marc
Noury, Seb
Pesamosca, Federico
Pfau, David
Sauter, Olivier
Sommariva, Cristian
Coda, Stefano
Duval, Basil
Fasoli, Ambrogio
Kohli, Pushmeet
Kavukcuoglu, Koray
Hassabis, Demis
Riedmiller, Martin
Magnetic control of tokamak plasmas through deep reinforcement learning
title Magnetic control of tokamak plasmas through deep reinforcement learning
title_full Magnetic control of tokamak plasmas through deep reinforcement learning
title_fullStr Magnetic control of tokamak plasmas through deep reinforcement learning
title_full_unstemmed Magnetic control of tokamak plasmas through deep reinforcement learning
title_short Magnetic control of tokamak plasmas through deep reinforcement learning
title_sort magnetic control of tokamak plasmas through deep reinforcement learning
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/
https://www.ncbi.nlm.nih.gov/pubmed/35173339
http://dx.doi.org/10.1038/s41586-021-04301-9
work_keys_str_mv AT degravejonas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT felicifederico magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT buchlijonas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT neunertmichael magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT traceybrendan magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT carpanesefrancesco magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT ewaldstimo magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT hafnerroland magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT abdolmalekiabbas magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT delascasasdiego magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT donnercraig magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT fritzleslie magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT galperticristian magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT huberandrea magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT keelingjames magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT tsimpoukellimaria magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT kayjackie magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT merleantoine magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT moretjeanmarc magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT nouryseb magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT pesamoscafederico magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT pfaudavid magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT sauterolivier magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT sommarivacristian magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT codastefano magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT duvalbasil magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT fasoliambrogio magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT kohlipushmeet magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT kavukcuoglukoray magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT hassabisdemis magneticcontroloftokamakplasmasthroughdeepreinforcementlearning
AT riedmillermartin magneticcontroloftokamakplasmasthroughdeepreinforcementlearning