Cargando…
Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis
Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identifie...
Autores principales: | , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9961939/ https://www.ncbi.nlm.nih.gov/pubmed/36836046 http://dx.doi.org/10.3390/jcm12041513 |
_version_ | 1784895879794130944 |
---|---|
author | Bologheanu, Razvan Kapral, Lorenz Laxar, Daniel Maleczek, Mathias Dibiasi, Christoph Zeiner, Sebastian Agibetov, Asan Ercole, Ari Thoral, Patrick Elbers, Paul Heitzinger, Clemens Kimberger, Oliver |
author_facet | Bologheanu, Razvan Kapral, Lorenz Laxar, Daniel Maleczek, Mathias Dibiasi, Christoph Zeiner, Sebastian Agibetov, Asan Ercole, Ari Thoral, Patrick Elbers, Paul Heitzinger, Clemens Kimberger, Oliver |
author_sort | Bologheanu, Razvan |
collection | PubMed |
description | Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm’s performance. Results: Agreement between the RL agent’s policy and the actual documented treatment reached 59%. Our RL agent’s treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians’ policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians’ historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. Conclusions: Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a ‘precision-medicine’ approach to future prospective controlled trials and practice. |
format | Online Article Text |
id | pubmed-9961939 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-99619392023-02-26 Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis Bologheanu, Razvan Kapral, Lorenz Laxar, Daniel Maleczek, Mathias Dibiasi, Christoph Zeiner, Sebastian Agibetov, Asan Ercole, Ari Thoral, Patrick Elbers, Paul Heitzinger, Clemens Kimberger, Oliver J Clin Med Article Background: The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. Methods: We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm’s performance. Results: Agreement between the RL agent’s policy and the actual documented treatment reached 59%. Our RL agent’s treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians’ policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians’ historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. Conclusions: Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a ‘precision-medicine’ approach to future prospective controlled trials and practice. MDPI 2023-02-14 /pmc/articles/PMC9961939/ /pubmed/36836046 http://dx.doi.org/10.3390/jcm12041513 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Bologheanu, Razvan Kapral, Lorenz Laxar, Daniel Maleczek, Mathias Dibiasi, Christoph Zeiner, Sebastian Agibetov, Asan Ercole, Ari Thoral, Patrick Elbers, Paul Heitzinger, Clemens Kimberger, Oliver Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title | Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title_full | Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title_fullStr | Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title_full_unstemmed | Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title_short | Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis |
title_sort | development of a reinforcement learning algorithm to optimize corticosteroid therapy in critically ill patients with sepsis |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9961939/ https://www.ncbi.nlm.nih.gov/pubmed/36836046 http://dx.doi.org/10.3390/jcm12041513 |
work_keys_str_mv | AT bologheanurazvan developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT kaprallorenz developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT laxardaniel developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT maleczekmathias developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT dibiasichristoph developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT zeinersebastian developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT agibetovasan developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT ercoleari developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT thoralpatrick developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT elberspaul developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT heitzingerclemens developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis AT kimbergeroliver developmentofareinforcementlearningalgorithmtooptimizecorticosteroidtherapyincriticallyillpatientswithsepsis |