Cargando…

Improvement of APACHE II score system for disease severity based on XGBoost algorithm

BACKGROUND: Prognostication is an essential tool for risk adjustment and decision making in the intensive care units (ICUs). In order to improve patient outcomes, we have been trying to develop a more effective model than Acute Physiology and Chronic Health Evaluation (APACHE) II to measure the seve...

Descripción completa

Detalles Bibliográficos
Autores principales: Luo, Yan, Wang, Zhiyu, Wang, Cong
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8344327/
https://www.ncbi.nlm.nih.gov/pubmed/34362354
http://dx.doi.org/10.1186/s12911-021-01591-x
_version_ 1783734465591246848
author Luo, Yan
Wang, Zhiyu
Wang, Cong
author_facet Luo, Yan
Wang, Zhiyu
Wang, Cong
author_sort Luo, Yan
collection PubMed
description BACKGROUND: Prognostication is an essential tool for risk adjustment and decision making in the intensive care units (ICUs). In order to improve patient outcomes, we have been trying to develop a more effective model than Acute Physiology and Chronic Health Evaluation (APACHE) II to measure the severity of the patients in ICUs. The aim of the present study was to provide a mortality prediction model for ICUs patients, and to assess its performance relative to prediction based on the APACHE II scoring system. METHODS: We used the Medical Information Mart for Intensive Care version III (MIMIC-III) database to build our model. After comparing the APACHE II with 6 typical machine learning (ML) methods, the best performing model was screened for external validation on anther independent dataset. Performance measures were calculated using cross-validation to avoid making biased assessments. The primary outcome was hospital mortality. Finally, we used TreeSHAP algorithm to explain the variable relationships in the extreme gradient boosting algorithm (XGBoost) model. RESULTS: We picked out 14 variables with 24,777 cases to form our basic data set. When the variables were the same as those contained in the APACHE II, the accuracy of XGBoost (accuracy: 0.858) was higher than that of APACHE II (accuracy: 0.742) and other algorithms. In addition, it exhibited better calibration properties than other methods, the result in the area under the ROC curve (AUC: 0.76). we then expand the variable set by adding five new variables to improve the performance of our model. The accuracy, precision, recall, F1, and AUC of the XGBoost model increased, and were still higher than other models (0.866, 0.853, 0.870, 0.845, and 0.81, respectively). On the external validation dataset, the AUC was 0.79 and calibration properties were good. CONCLUSIONS: As compared to conventional severity scores APACHE II, our XGBoost proposal offers improved performance for predicting hospital mortality in ICUs patients. Furthermore, the TreeSHAP can help to enhance the understanding of our model by providing detailed insights into the impact of different features on the disease risk. In sum, our model could help clinicians determine prognosis and improve patient outcomes.
format Online
Article
Text
id pubmed-8344327
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-83443272021-08-09 Improvement of APACHE II score system for disease severity based on XGBoost algorithm Luo, Yan Wang, Zhiyu Wang, Cong BMC Med Inform Decis Mak Technical Advance BACKGROUND: Prognostication is an essential tool for risk adjustment and decision making in the intensive care units (ICUs). In order to improve patient outcomes, we have been trying to develop a more effective model than Acute Physiology and Chronic Health Evaluation (APACHE) II to measure the severity of the patients in ICUs. The aim of the present study was to provide a mortality prediction model for ICUs patients, and to assess its performance relative to prediction based on the APACHE II scoring system. METHODS: We used the Medical Information Mart for Intensive Care version III (MIMIC-III) database to build our model. After comparing the APACHE II with 6 typical machine learning (ML) methods, the best performing model was screened for external validation on anther independent dataset. Performance measures were calculated using cross-validation to avoid making biased assessments. The primary outcome was hospital mortality. Finally, we used TreeSHAP algorithm to explain the variable relationships in the extreme gradient boosting algorithm (XGBoost) model. RESULTS: We picked out 14 variables with 24,777 cases to form our basic data set. When the variables were the same as those contained in the APACHE II, the accuracy of XGBoost (accuracy: 0.858) was higher than that of APACHE II (accuracy: 0.742) and other algorithms. In addition, it exhibited better calibration properties than other methods, the result in the area under the ROC curve (AUC: 0.76). we then expand the variable set by adding five new variables to improve the performance of our model. The accuracy, precision, recall, F1, and AUC of the XGBoost model increased, and were still higher than other models (0.866, 0.853, 0.870, 0.845, and 0.81, respectively). On the external validation dataset, the AUC was 0.79 and calibration properties were good. CONCLUSIONS: As compared to conventional severity scores APACHE II, our XGBoost proposal offers improved performance for predicting hospital mortality in ICUs patients. Furthermore, the TreeSHAP can help to enhance the understanding of our model by providing detailed insights into the impact of different features on the disease risk. In sum, our model could help clinicians determine prognosis and improve patient outcomes. BioMed Central 2021-08-06 /pmc/articles/PMC8344327/ /pubmed/34362354 http://dx.doi.org/10.1186/s12911-021-01591-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
spellingShingle Technical Advance
Luo, Yan
Wang, Zhiyu
Wang, Cong
Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title_full Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title_fullStr Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title_full_unstemmed Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title_short Improvement of APACHE II score system for disease severity based on XGBoost algorithm
title_sort improvement of apache ii score system for disease severity based on xgboost algorithm
topic Technical Advance
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8344327/
https://www.ncbi.nlm.nih.gov/pubmed/34362354
http://dx.doi.org/10.1186/s12911-021-01591-x
work_keys_str_mv AT luoyan improvementofapacheiiscoresystemfordiseaseseveritybasedonxgboostalgorithm
AT wangzhiyu improvementofapacheiiscoresystemfordiseaseseveritybasedonxgboostalgorithm
AT wangcong improvementofapacheiiscoresystemfordiseaseseveritybasedonxgboostalgorithm