Cargando…

A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score

BACKGROUND: Several models have been developed to predict mortality in patients with COVID-19 pneumonia, but only a few have demonstrated enough discriminatory capacity. Machine learning algorithms represent a novel approach for the data-driven prediction of clinical outcomes with advantages over st...

Descripción completa

Detalles Bibliográficos
Autores principales: Halasz, Geza, Sperti, Michela, Villani, Matteo, Michelucci, Umberto, Agostoni, Piergiuseppe, Biagi, Andrea, Rossi, Luca, Botti, Andrea, Mari, Chiara, Maccarini, Marco, Pura, Filippo, Roveda, Loris, Nardecchia, Alessia, Mottola, Emanuele, Nolli, Massimo, Salvioni, Elisabetta, Mapelli, Massimo, Deriu, Marco Agostino, Piga, Dario, Piepoli, Massimo
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8168638/
https://www.ncbi.nlm.nih.gov/pubmed/33999838
http://dx.doi.org/10.2196/29058
_version_ 1783701909177106432
author Halasz, Geza
Sperti, Michela
Villani, Matteo
Michelucci, Umberto
Agostoni, Piergiuseppe
Biagi, Andrea
Rossi, Luca
Botti, Andrea
Mari, Chiara
Maccarini, Marco
Pura, Filippo
Roveda, Loris
Nardecchia, Alessia
Mottola, Emanuele
Nolli, Massimo
Salvioni, Elisabetta
Mapelli, Massimo
Deriu, Marco Agostino
Piga, Dario
Piepoli, Massimo
author_facet Halasz, Geza
Sperti, Michela
Villani, Matteo
Michelucci, Umberto
Agostoni, Piergiuseppe
Biagi, Andrea
Rossi, Luca
Botti, Andrea
Mari, Chiara
Maccarini, Marco
Pura, Filippo
Roveda, Loris
Nardecchia, Alessia
Mottola, Emanuele
Nolli, Massimo
Salvioni, Elisabetta
Mapelli, Massimo
Deriu, Marco Agostino
Piga, Dario
Piepoli, Massimo
author_sort Halasz, Geza
collection PubMed
description BACKGROUND: Several models have been developed to predict mortality in patients with COVID-19 pneumonia, but only a few have demonstrated enough discriminatory capacity. Machine learning algorithms represent a novel approach for the data-driven prediction of clinical outcomes with advantages over statistical modeling. OBJECTIVE: We aimed to develop a machine learning–based score—the Piacenza score—for 30-day mortality prediction in patients with COVID-19 pneumonia. METHODS: The study comprised 852 patients with COVID-19 pneumonia, admitted to the Guglielmo da Saliceto Hospital in Italy from February to November 2020. Patients’ medical history, demographics, and clinical data were collected using an electronic health record. The overall patient data set was randomly split into derivation and test cohorts. The score was obtained through the naïve Bayes classifier and externally validated on 86 patients admitted to Centro Cardiologico Monzino (Italy) in February 2020. Using a forward-search algorithm, 6 features were identified: age, mean corpuscular hemoglobin concentration, PaO(2)/FiO(2) ratio, temperature, previous stroke, and gender. The Brier index was used to evaluate the ability of the machine learning model to stratify and predict the observed outcomes. A user-friendly website was designed and developed to enable fast and easy use of the tool by physicians. Regarding the customization properties of the Piacenza score, we added a tailored version of the algorithm to the website, which enables an optimized computation of the mortality risk score for a patient when some of the variables used by the Piacenza score are not available. In this case, the naïve Bayes classifier is retrained over the same derivation cohort but using a different set of patient characteristics. We also compared the Piacenza score with the 4C score and with a naïve Bayes algorithm with 14 features chosen a priori. RESULTS: The Piacenza score exhibited an area under the receiver operating characteristic curve (AUC) of 0.78 (95% CI 0.74-0.84, Brier score=0.19) in the internal validation cohort and 0.79 (95% CI 0.68-0.89, Brier score=0.16) in the external validation cohort, showing a comparable accuracy with respect to the 4C score and to the naïve Bayes model with a priori chosen features; this achieved an AUC of 0.78 (95% CI 0.73-0.83, Brier score=0.26) and 0.80 (95% CI 0.75-0.86, Brier score=0.17), respectively. CONCLUSIONS: Our findings demonstrated that a customizable machine learning–based score with a purely data-driven selection of features is feasible and effective for the prediction of mortality among patients with COVID-19 pneumonia.
format Online
Article
Text
id pubmed-8168638
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-81686382021-06-11 A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score Halasz, Geza Sperti, Michela Villani, Matteo Michelucci, Umberto Agostoni, Piergiuseppe Biagi, Andrea Rossi, Luca Botti, Andrea Mari, Chiara Maccarini, Marco Pura, Filippo Roveda, Loris Nardecchia, Alessia Mottola, Emanuele Nolli, Massimo Salvioni, Elisabetta Mapelli, Massimo Deriu, Marco Agostino Piga, Dario Piepoli, Massimo J Med Internet Res Original Paper BACKGROUND: Several models have been developed to predict mortality in patients with COVID-19 pneumonia, but only a few have demonstrated enough discriminatory capacity. Machine learning algorithms represent a novel approach for the data-driven prediction of clinical outcomes with advantages over statistical modeling. OBJECTIVE: We aimed to develop a machine learning–based score—the Piacenza score—for 30-day mortality prediction in patients with COVID-19 pneumonia. METHODS: The study comprised 852 patients with COVID-19 pneumonia, admitted to the Guglielmo da Saliceto Hospital in Italy from February to November 2020. Patients’ medical history, demographics, and clinical data were collected using an electronic health record. The overall patient data set was randomly split into derivation and test cohorts. The score was obtained through the naïve Bayes classifier and externally validated on 86 patients admitted to Centro Cardiologico Monzino (Italy) in February 2020. Using a forward-search algorithm, 6 features were identified: age, mean corpuscular hemoglobin concentration, PaO(2)/FiO(2) ratio, temperature, previous stroke, and gender. The Brier index was used to evaluate the ability of the machine learning model to stratify and predict the observed outcomes. A user-friendly website was designed and developed to enable fast and easy use of the tool by physicians. Regarding the customization properties of the Piacenza score, we added a tailored version of the algorithm to the website, which enables an optimized computation of the mortality risk score for a patient when some of the variables used by the Piacenza score are not available. In this case, the naïve Bayes classifier is retrained over the same derivation cohort but using a different set of patient characteristics. We also compared the Piacenza score with the 4C score and with a naïve Bayes algorithm with 14 features chosen a priori. RESULTS: The Piacenza score exhibited an area under the receiver operating characteristic curve (AUC) of 0.78 (95% CI 0.74-0.84, Brier score=0.19) in the internal validation cohort and 0.79 (95% CI 0.68-0.89, Brier score=0.16) in the external validation cohort, showing a comparable accuracy with respect to the 4C score and to the naïve Bayes model with a priori chosen features; this achieved an AUC of 0.78 (95% CI 0.73-0.83, Brier score=0.26) and 0.80 (95% CI 0.75-0.86, Brier score=0.17), respectively. CONCLUSIONS: Our findings demonstrated that a customizable machine learning–based score with a purely data-driven selection of features is feasible and effective for the prediction of mortality among patients with COVID-19 pneumonia. JMIR Publications 2021-05-31 /pmc/articles/PMC8168638/ /pubmed/33999838 http://dx.doi.org/10.2196/29058 Text en ©Geza Halasz, Michela Sperti, Matteo Villani, Umberto Michelucci, Piergiuseppe Agostoni, Andrea Biagi, Luca Rossi, Andrea Botti, Chiara Mari, Marco Maccarini, Filippo Pura, Loris Roveda, Alessia Nardecchia, Emanuele Mottola, Massimo Nolli, Elisabetta Salvioni, Massimo Mapelli, Marco Agostino Deriu, Dario Piga, Massimo Piepoli. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 31.05.2021. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Halasz, Geza
Sperti, Michela
Villani, Matteo
Michelucci, Umberto
Agostoni, Piergiuseppe
Biagi, Andrea
Rossi, Luca
Botti, Andrea
Mari, Chiara
Maccarini, Marco
Pura, Filippo
Roveda, Loris
Nardecchia, Alessia
Mottola, Emanuele
Nolli, Massimo
Salvioni, Elisabetta
Mapelli, Massimo
Deriu, Marco Agostino
Piga, Dario
Piepoli, Massimo
A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title_full A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title_fullStr A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title_full_unstemmed A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title_short A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
title_sort machine learning approach for mortality prediction in covid-19 pneumonia: development and evaluation of the piacenza score
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8168638/
https://www.ncbi.nlm.nih.gov/pubmed/33999838
http://dx.doi.org/10.2196/29058
work_keys_str_mv AT halaszgeza amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT spertimichela amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT villanimatteo amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT michelucciumberto amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT agostonipiergiuseppe amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT biagiandrea amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT rossiluca amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT bottiandrea amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT marichiara amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT maccarinimarco amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT purafilippo amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT rovedaloris amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT nardecchiaalessia amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT mottolaemanuele amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT nollimassimo amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT salvionielisabetta amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT mapellimassimo amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT deriumarcoagostino amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT pigadario amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT piepolimassimo amachinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT halaszgeza machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT spertimichela machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT villanimatteo machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT michelucciumberto machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT agostonipiergiuseppe machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT biagiandrea machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT rossiluca machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT bottiandrea machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT marichiara machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT maccarinimarco machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT purafilippo machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT rovedaloris machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT nardecchiaalessia machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT mottolaemanuele machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT nollimassimo machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT salvionielisabetta machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT mapellimassimo machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT deriumarcoagostino machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT pigadario machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore
AT piepolimassimo machinelearningapproachformortalitypredictionincovid19pneumoniadevelopmentandevaluationofthepiacenzascore