Cargando…

A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation

OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTI...

Descripción completa

Detalles Bibliográficos
Autores principales: Prince, Remi D., Akhondi-Asl, Alireza, Mehta, Nilesh M., Geva, Alon
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Lippincott Williams & Wilkins 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8133049/
https://www.ncbi.nlm.nih.gov/pubmed/34036277
http://dx.doi.org/10.1097/CCE.0000000000000426
_version_ 1783695009223016448
author Prince, Remi D.
Akhondi-Asl, Alireza
Mehta, Nilesh M.
Geva, Alon
author_facet Prince, Remi D.
Akhondi-Asl, Alireza
Mehta, Nilesh M.
Geva, Alon
author_sort Prince, Remi D.
collection PubMed
description OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863–0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246–0.414]; F1, 0.396 [95% CI, 0.321–0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713–0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165–0.316]; F1, 0.284 [95% CI, 0.209–0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model. CONCLUSIONS: A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed.
format Online
Article
Text
id pubmed-8133049
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Lippincott Williams & Wilkins
record_format MEDLINE/PubMed
spelling pubmed-81330492021-05-24 A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation Prince, Remi D. Akhondi-Asl, Alireza Mehta, Nilesh M. Geva, Alon Crit Care Explor Predictive Modeling Report OBJECTIVES: To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score. DESIGN: Retrospective study. SETTING: Quaternary care medical-surgical PICU. PATIENTS: All patients admitted to the PICU from 2013 to 2019. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863–0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246–0.414]; F1, 0.396 [95% CI, 0.321–0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713–0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165–0.316]; F1, 0.284 [95% CI, 0.209–0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model. CONCLUSIONS: A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed. Lippincott Williams & Wilkins 2021-05-17 /pmc/articles/PMC8133049/ /pubmed/34036277 http://dx.doi.org/10.1097/CCE.0000000000000426 Text en Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine. https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) , where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.
spellingShingle Predictive Modeling Report
Prince, Remi D.
Akhondi-Asl, Alireza
Mehta, Nilesh M.
Geva, Alon
A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title_full A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title_fullStr A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title_full_unstemmed A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title_short A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation
title_sort machine learning classifier improves mortality prediction compared with pediatric logistic organ dysfunction-2 score: model development and validation
topic Predictive Modeling Report
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8133049/
https://www.ncbi.nlm.nih.gov/pubmed/34036277
http://dx.doi.org/10.1097/CCE.0000000000000426
work_keys_str_mv AT princeremid amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT akhondiaslalireza amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT mehtanileshm amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT gevaalon amachinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT princeremid machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT akhondiaslalireza machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT mehtanileshm machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation
AT gevaalon machinelearningclassifierimprovesmortalitypredictioncomparedwithpediatriclogisticorgandysfunction2scoremodeldevelopmentandvalidation