Cargando…

A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account

Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a mod...

Descripción completa

Detalles Bibliográficos
Autores principales: Namdar, Khashayar, Haider, Masoom A., Khalvati, Farzad
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8670229/
https://www.ncbi.nlm.nih.gov/pubmed/34917933
http://dx.doi.org/10.3389/frai.2021.582928
_version_ 1784614937120735232
author Namdar, Khashayar
Haider, Masoom A.
Khalvati, Farzad
author_facet Namdar, Khashayar
Haider, Masoom A.
Khalvati, Farzad
author_sort Namdar, Khashayar
collection PubMed
description Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a modified version of AUC that takes confidence of the model into account and at the same time, incorporates AUC into Binary Cross Entropy (BCE) loss used for training a Convolutional neural Network for classification tasks. We demonstrate this on three datasets: MNIST, prostate MRI, and brain MRI. Furthermore, we have published GenuineAI, a new python library, which provides the functions for conventional AUC and the proposed modified AUC along with metrics including sensitivity, specificity, recall, precision, and F1 for each point of the ROC curve.
format Online
Article
Text
id pubmed-8670229
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-86702292021-12-15 A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account Namdar, Khashayar Haider, Masoom A. Khalvati, Farzad Front Artif Intell Artificial Intelligence Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a modified version of AUC that takes confidence of the model into account and at the same time, incorporates AUC into Binary Cross Entropy (BCE) loss used for training a Convolutional neural Network for classification tasks. We demonstrate this on three datasets: MNIST, prostate MRI, and brain MRI. Furthermore, we have published GenuineAI, a new python library, which provides the functions for conventional AUC and the proposed modified AUC along with metrics including sensitivity, specificity, recall, precision, and F1 for each point of the ROC curve. Frontiers Media S.A. 2021-11-30 /pmc/articles/PMC8670229/ /pubmed/34917933 http://dx.doi.org/10.3389/frai.2021.582928 Text en Copyright © 2021 Namdar, Haider and Khalvati. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Artificial Intelligence
Namdar, Khashayar
Haider, Masoom A.
Khalvati, Farzad
A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title_full A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title_fullStr A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title_full_unstemmed A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title_short A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
title_sort modified auc for training convolutional neural networks: taking confidence into account
topic Artificial Intelligence
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8670229/
https://www.ncbi.nlm.nih.gov/pubmed/34917933
http://dx.doi.org/10.3389/frai.2021.582928
work_keys_str_mv AT namdarkhashayar amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount
AT haidermasooma amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount
AT khalvatifarzad amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount
AT namdarkhashayar modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount
AT haidermasooma modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount
AT khalvatifarzad modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount