Cargando…

Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis

Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identifie...

Descripción completa

Detalles Bibliográficos
Autores principales: Bologheanu, Razvan, Kapral, Lorenz, Laxar, Daniel, Maleczek, Mathias, Dibiasi, Christoph, Zeiner, Sebastian, Agibetov, Asan, Ercole, Ari, Thoral, Patrick, Elbers, Paul, Heitzinger, Clemens, Kimberger, Oliver
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9961939/
https://www.ncbi.nlm.nih.gov/pubmed/36836046
http://dx.doi.org/10.3390/jcm12041513
_version_ 1784895879794130944
author Bologheanu, Razvan
Kapral, Lorenz
Laxar, Daniel
Maleczek, Mathias
Dibiasi, Christoph
Zeiner, Sebastian
Agibetov, Asan
Ercole, Ari
Thoral, Patrick
Elbers, Paul
Heitzinger, Clemens
Kimberger, Oliver
author_facet Bologheanu, Razvan
Kapral, Lorenz
Laxar, Daniel
Maleczek, Mathias
Dibiasi, Christoph
Zeiner, Sebastian
Agibetov, Asan
Ercole, Ari
Thoral, Patrick
Elbers, Paul
Heitzinger, Clemens
Kimberger, Oliver
author_sort Bologheanu, Razvan
collection PubMed
description Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm’s performance. Results: Agreement between the RL agent’s policy and the actual documented treatment reached 59%. Our RL agent’s treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians’ policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians’ historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. Conclusions: Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a ‘precision-medicine’ approach to future prospective controlled trials and practice.
format Online
Article
Text
id pubmed-9961939
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-99619392023-02-26 Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis Bologheanu, Razvan Kapral, Lorenz Laxar, Daniel Maleczek, Mathias Dibiasi, Christoph Zeiner, Sebastian Agibetov, Asan Ercole, Ari Thoral, Patrick Elbers, Paul Heitzinger, Clemens Kimberger, Oliver J Clin Med Article Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm’s performance. Results: Agreement between the RL agent’s policy and the actual documented treatment reached 59%. Our RL agent’s treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians’ policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians’ historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. Conclusions: Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a ‘precision-medicine’ approach to future prospective controlled trials and practice. MDPI 2023-02-14 /pmc/articles/PMC9961939/ /pubmed/36836046 http://dx.doi.org/10.3390/jcm12041513 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Bologheanu, Razvan
Kapral, Lorenz
Laxar, Daniel
Maleczek, Mathias
Dibiasi, Christoph
Zeiner, Sebastian
Agibetov, Asan
Ercole, Ari
Thoral, Patrick
Elbers, Paul
Heitzinger, Clemens
Kimberger, Oliver
Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title_full Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title_fullStr Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title_full_unstemmed Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title_short Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
title_sort development of a reinforcement learning algorithm to optimize corticosteroid therapy in critically ill patients with sepsis
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9961939/
https://www.ncbi.nlm.nih.gov/pubmed/36836046
http://dx.doi.org/10.3390/jcm12041513
work_keys_str_mv AT bologheanurazvan developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT kaprallorenz developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT laxardaniel developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT maleczekmathias developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT dibiasichristoph developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT zeinersebastian developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT agibetovasan developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT ercoleari developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT thoralpatrick developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT elberspaul developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT heitzingerclemens developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis
AT kimbergeroliver developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis