Cargando…
A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTI...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Lippincott Williams & Wilkins
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8133049/ https://www.ncbi.nlm.nih.gov/pubmed/34036277 http://dx.doi.org/10.1097/CCE.0000000000000426 |
_version_ | 1783695009223016448 |
---|---|
author | Prince, Remi D. Akhondi-Asl, Alireza Mehta, Nilesh M. Geva, Alon |
author_facet | Prince, Remi D. Akhondi-Asl, Alireza Mehta, Nilesh M. Geva, Alon |
author_sort | Prince, Remi D. |
collection | PubMed |
description | OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863–0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246–0.414]; F1, 0.396 [95% CI, 0.321–0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713–0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165–0.316]; F1, 0.284 [95% CI, 0.209–0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model. CONCLUSIONS: A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed. |
format | Online Article Text |
id | pubmed-8133049 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Lippincott Williams & Wilkins |
record_format | MEDLINE/PubMed |
spelling | pubmed-81330492021-05-24 A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation Prince, Remi D. Akhondi-Asl, Alireza Mehta, Nilesh M. Geva, Alon Crit Care Explor Predictive Modeling Report OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863–0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246–0.414]; F1, 0.396 [95% CI, 0.321–0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713–0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165–0.316]; F1, 0.284 [95% CI, 0.209–0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model. CONCLUSIONS: A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed. Lippincott Williams & Wilkins 2021-05-17 /pmc/articles/PMC8133049/ /pubmed/34036277 http://dx.doi.org/10.1097/CCE.0000000000000426 Text en Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine. https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) , where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal. |
spellingShingle | Predictive Modeling Report Prince, Remi D. Akhondi-Asl, Alireza Mehta, Nilesh M. Geva, Alon A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title | A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title_full | A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title_fullStr | A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title_full_unstemmed | A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title_short | A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation |
title_sort | machine learning classifier improves mortality prediction compared with pediatric logistic organ dysfunction-2 score: model development and validation |
topic | Predictive Modeling Report |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8133049/ https://www.ncbi.nlm.nih.gov/pubmed/34036277 http://dx.doi.org/10.1097/CCE.0000000000000426 |
work_keys_str_mv | AT princeremid amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT akhondiaslalireza amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT mehtanileshm amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT gevaalon amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT princeremid machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT akhondiaslalireza machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT mehtanileshm machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation AT gevaalon machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation |