Cargando…

Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network

Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop a...

Descripción completa

Detalles Bibliográficos
Autores principales: Penny-Dimri, Jahan C., Bergmeir, Christoph, Reid, Christopher M., Williams-Spence, Jenni, Cochrane, Andrew D., Smith, Julian A.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468047/
https://www.ncbi.nlm.nih.gov/pubmed/37647308
http://dx.doi.org/10.1371/journal.pone.0289930
_version_ 1785099159827644416
author Penny-Dimri, Jahan C.
Bergmeir, Christoph
Reid, Christopher M.
Williams-Spence, Jenni
Cochrane, Andrew D.
Smith, Julian A.
author_facet Penny-Dimri, Jahan C.
Bergmeir, Christoph
Reid, Christopher M.
Williams-Spence, Jenni
Cochrane, Andrew D.
Smith, Julian A.
author_sort Penny-Dimri, Jahan C.
collection PubMed
description Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use.
format Online
Article
Text
id pubmed-10468047
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-104680472023-08-31 Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network Penny-Dimri, Jahan C. Bergmeir, Christoph Reid, Christopher M. Williams-Spence, Jenni Cochrane, Andrew D. Smith, Julian A. PLoS One Research Article Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use. Public Library of Science 2023-08-30 /pmc/articles/PMC10468047/ /pubmed/37647308 http://dx.doi.org/10.1371/journal.pone.0289930 Text en © 2023 Penny-Dimri et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Penny-Dimri, Jahan C.
Bergmeir, Christoph
Reid, Christopher M.
Williams-Spence, Jenni
Cochrane, Andrew D.
Smith, Julian A.
Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title_full Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title_fullStr Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title_full_unstemmed Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title_short Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network
title_sort paying attention to cardiac surgical risk: an interpretable machine learning approach using an uncertainty-aware attentive neural network
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10468047/
https://www.ncbi.nlm.nih.gov/pubmed/37647308
http://dx.doi.org/10.1371/journal.pone.0289930
work_keys_str_mv AT pennydimrijahanc payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork
AT bergmeirchristoph payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork
AT reidchristopherm payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork
AT williamsspencejenni payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork
AT cochraneandrewd payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork
AT smithjuliana payingattentiontocardiacsurgicalriskaninterpretablemachinelearningapproachusinganuncertaintyawareattentiveneuralnetwork