Cargando…
A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account
Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a mod...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8670229/ https://www.ncbi.nlm.nih.gov/pubmed/34917933 http://dx.doi.org/10.3389/frai.2021.582928 |
_version_ | 1784614937120735232 |
---|---|
author | Namdar, Khashayar Haider, Masoom A. Khalvati, Farzad |
author_facet | Namdar, Khashayar Haider, Masoom A. Khalvati, Farzad |
author_sort | Namdar, Khashayar |
collection | PubMed |
description | Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a modified version of AUC that takes confidence of the model into account and at the same time, incorporates AUC into Binary Cross Entropy (BCE) loss used for training a Convolutional neural Network for classification tasks. We demonstrate this on three datasets: MNIST, prostate MRI, and brain MRI. Furthermore, we have published GenuineAI, a new python library, which provides the functions for conventional AUC and the proposed modified AUC along with metrics including sensitivity, specificity, recall, precision, and F1 for each point of the ROC curve. |
format | Online Article Text |
id | pubmed-8670229 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-86702292021-12-15 A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account Namdar, Khashayar Haider, Masoom A. Khalvati, Farzad Front Artif Intell Artificial Intelligence Receiver operating characteristic (ROC) curve is an informative tool in binary classification and Area Under ROC Curve (AUC) is a popular metric for reporting performance of binary classifiers. In this paper, first we present a comprehensive review of ROC curve and AUC metric. Next, we propose a modified version of AUC that takes confidence of the model into account and at the same time, incorporates AUC into Binary Cross Entropy (BCE) loss used for training a Convolutional neural Network for classification tasks. We demonstrate this on three datasets: MNIST, prostate MRI, and brain MRI. Furthermore, we have published GenuineAI, a new python library, which provides the functions for conventional AUC and the proposed modified AUC along with metrics including sensitivity, specificity, recall, precision, and F1 for each point of the ROC curve. Frontiers Media S.A. 2021-11-30 /pmc/articles/PMC8670229/ /pubmed/34917933 http://dx.doi.org/10.3389/frai.2021.582928 Text en Copyright © 2021 Namdar, Haider and Khalvati. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Artificial Intelligence Namdar, Khashayar Haider, Masoom A. Khalvati, Farzad A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title | A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title_full | A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title_fullStr | A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title_full_unstemmed | A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title_short | A Modified AUC for Training Convolutional Neural Networks: Taking Confidence Into Account |
title_sort | modified auc for training convolutional neural networks: taking confidence into account |
topic | Artificial Intelligence |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8670229/ https://www.ncbi.nlm.nih.gov/pubmed/34917933 http://dx.doi.org/10.3389/frai.2021.582928 |
work_keys_str_mv | AT namdarkhashayar amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount AT haidermasooma amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount AT khalvatifarzad amodifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount AT namdarkhashayar modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount AT haidermasooma modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount AT khalvatifarzad modifiedaucfortrainingconvolutionalneuralnetworkstakingconfidenceintoaccount |