Cargando…

Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development an...

Descripción completa

Detalles Bibliográficos
Autores principales: Lin, Pei-Chen, Chen, Kuo-Tai, Chen, Huan-Chieh, Islam, Md. Mohaimenul, Lin, Ming-Chin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8623760/
https://www.ncbi.nlm.nih.gov/pubmed/34834406
http://dx.doi.org/10.3390/jpm11111055
_version_ 1784606009389481984
author Lin, Pei-Chen
Chen, Kuo-Tai
Chen, Huan-Chieh
Islam, Md. Mohaimenul
Lin, Ming-Chin
author_facet Lin, Pei-Chen
Chen, Kuo-Tai
Chen, Huan-Chieh
Islam, Md. Mohaimenul
Lin, Ming-Chin
author_sort Lin, Pei-Chen
collection PubMed
description Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.
format Online
Article
Text
id pubmed-8623760
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-86237602021-11-27 Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation Lin, Pei-Chen Chen, Kuo-Tai Chen, Huan-Chieh Islam, Md. Mohaimenul Lin, Ming-Chin J Pers Med Article Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches. MDPI 2021-10-21 /pmc/articles/PMC8623760/ /pubmed/34834406 http://dx.doi.org/10.3390/jpm11111055 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Lin, Pei-Chen
Chen, Kuo-Tai
Chen, Huan-Chieh
Islam, Md. Mohaimenul
Lin, Ming-Chin
Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title_full Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title_fullStr Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title_full_unstemmed Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title_short Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation
title_sort machine learning model to identify sepsis patients in the emergency department: algorithm development and validation
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8623760/
https://www.ncbi.nlm.nih.gov/pubmed/34834406
http://dx.doi.org/10.3390/jpm11111055
work_keys_str_mv AT linpeichen machinelearningmodeltoidentifysepsispatientsintheemergencydepartmentalgorithmdevelopmentandvalidation
AT chenkuotai machinelearningmodeltoidentifysepsispatientsintheemergencydepartmentalgorithmdevelopmentandvalidation
AT chenhuanchieh machinelearningmodeltoidentifysepsispatientsintheemergencydepartmentalgorithmdevelopmentandvalidation
AT islammdmohaimenul machinelearningmodeltoidentifysepsispatientsintheemergencydepartmentalgorithmdevelopmentandvalidation
AT linmingchin machinelearningmodeltoidentifysepsispatientsintheemergencydepartmentalgorithmdevelopmentandvalidation