Cargando…
Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop a...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468047/ https://www.ncbi.nlm.nih.gov/pubmed/37647308 http://dx.doi.org/10.1371/journal.pone.0289930 |
_version_ | 1785099159827644416 |
---|---|
author | Penny-Dimri, Jahan C. Bergmeir, Christoph Reid, Christopher M. Williams-Spence, Jenni Cochrane, Andrew D. Smith, Julian A. |
author_facet | Penny-Dimri, Jahan C. Bergmeir, Christoph Reid, Christopher M. Williams-Spence, Jenni Cochrane, Andrew D. Smith, Julian A. |
author_sort | Penny-Dimri, Jahan C. |
collection | PubMed |
description | Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use. |
format | Online Article Text |
id | pubmed-10468047 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-104680472023-08-31 Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network Penny-Dimri, Jahan C. Bergmeir, Christoph Reid, Christopher M. Williams-Spence, Jenni Cochrane, Andrew D. Smith, Julian A. PLoS One Research Article Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use. Public Library of Science 2023-08-30 /pmc/articles/PMC10468047/ /pubmed/37647308 http://dx.doi.org/10.1371/journal.pone.0289930 Text en © 2023 Penny-Dimri et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Penny-Dimri, Jahan C. Bergmeir, Christoph Reid, Christopher M. Williams-Spence, Jenni Cochrane, Andrew D. Smith, Julian A. Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title | Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title_full | Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title_fullStr | Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title_full_unstemmed | Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title_short | Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network |
title_sort | paying attention to cardiac surgical risk: an interpretable machine learning approach using an uncertainty-aware attentive neural network |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468047/ https://www.ncbi.nlm.nih.gov/pubmed/37647308 http://dx.doi.org/10.1371/journal.pone.0289930 |
work_keys_str_mv | AT pennydimrijahanc payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork AT bergmeirchristoph payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork AT reidchristopherm payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork AT williamsspencejenni payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork AT cochraneandrewd payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork AT smithjuliana payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork |