Cargando…

Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map

SIMPLE SUMMARY: Convolutional neural networks (CNNs) have shown promising performance in recognizing oral cancer. However, the lack of interpretability and reliability remain major challenges in the development of trustworthy computer-aided diagnosis systems. To address this issue, we proposed a neu...

Descripción completa

Detalles Bibliográficos
Autores principales: Song, Bofan, Zhang, Chicheng, Sunny, Sumsum, KC, Dharma Raj, Li, Shaobai, Gurushanth, Keerthi, Mendonca, Pramila, Mukhia, Nirza, Patrick, Sanjana, Gurudath, Shubha, Raghavan, Subhashini, Tsusennaro, Imchen, Leivon, Shirley T., Kolur, Trupti, Shetty, Vivek, Bushan, Vidya, Ramesh, Rohan, Pillai, Vijay, Wilder-Smith, Petra, Suresh, Amritha, Kuriakose, Moni Abraham, Birur, Praveen, Liang, Rongguang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10001266/
https://www.ncbi.nlm.nih.gov/pubmed/36900210
http://dx.doi.org/10.3390/cancers15051421
_version_ 1784904093035134976
author Song, Bofan
Zhang, Chicheng
Sunny, Sumsum
KC, Dharma Raj
Li, Shaobai
Gurushanth, Keerthi
Mendonca, Pramila
Mukhia, Nirza
Patrick, Sanjana
Gurudath, Shubha
Raghavan, Subhashini
Tsusennaro, Imchen
Leivon, Shirley T.
Kolur, Trupti
Shetty, Vivek
Bushan, Vidya
Ramesh, Rohan
Pillai, Vijay
Wilder-Smith, Petra
Suresh, Amritha
Kuriakose, Moni Abraham
Birur, Praveen
Liang, Rongguang
author_facet Song, Bofan
Zhang, Chicheng
Sunny, Sumsum
KC, Dharma Raj
Li, Shaobai
Gurushanth, Keerthi
Mendonca, Pramila
Mukhia, Nirza
Patrick, Sanjana
Gurudath, Shubha
Raghavan, Subhashini
Tsusennaro, Imchen
Leivon, Shirley T.
Kolur, Trupti
Shetty, Vivek
Bushan, Vidya
Ramesh, Rohan
Pillai, Vijay
Wilder-Smith, Petra
Suresh, Amritha
Kuriakose, Moni Abraham
Birur, Praveen
Liang, Rongguang
author_sort Song, Bofan
collection PubMed
description SIMPLE SUMMARY: Convolutional neural networks (CNNs) have shown promising performance in recognizing oral cancer. However, the lack of interpretability and reliability remain major challenges in the development of trustworthy computer-aided diagnosis systems. To address this issue, we proposed a neural network architecture that integrates visual explanation and attention mechanisms. It improves the recognition performance via the attention mechanism while simultaneously providing interpretability for decision-making. Furthermore, our system incorporates Human-in-the-loop (HITL) deep learning to enhance the reliability and accuracy of the system through the integration of human and machine intelligence. We embedded expert knowledge into the network by manually editing the attention map for the attention mechanism. ABSTRACT: Convolutional neural networks have demonstrated excellent performance in oral cancer detection and classification. However, the end-to-end learning strategy makes CNNs hard to interpret, and it can be challenging to fully understand the decision-making procedure. Additionally, reliability is also a significant challenge for CNN based approaches. In this study, we proposed a neural network called the attention branch network (ABN), which combines the visual explanation and attention mechanisms to improve the recognition performance and interpret the decision-making simultaneously. We also embedded expert knowledge into the network by having human experts manually edit the attention maps for the attention mechanism. Our experiments have shown that ABN performs better than the original baseline network. By introducing the Squeeze-and-Excitation (SE) blocks to the network, the cross-validation accuracy increased further. Furthermore, we observed that some previously misclassified cases were correctly recognized after updating by manually editing the attention maps. The cross-validation accuracy increased from 0.846 to 0.875 with the ABN (Resnet18 as baseline), 0.877 with SE-ABN, and 0.903 after embedding expert knowledge. The proposed method provides an accurate, interpretable, and reliable oral cancer computer-aided diagnosis system through visual explanation, attention mechanisms, and expert knowledge embedding.
format Online
Article
Text
id pubmed-10001266
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100012662023-03-11 Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map Song, Bofan Zhang, Chicheng Sunny, Sumsum KC, Dharma Raj Li, Shaobai Gurushanth, Keerthi Mendonca, Pramila Mukhia, Nirza Patrick, Sanjana Gurudath, Shubha Raghavan, Subhashini Tsusennaro, Imchen Leivon, Shirley T. Kolur, Trupti Shetty, Vivek Bushan, Vidya Ramesh, Rohan Pillai, Vijay Wilder-Smith, Petra Suresh, Amritha Kuriakose, Moni Abraham Birur, Praveen Liang, Rongguang Cancers (Basel) Article SIMPLE SUMMARY: Convolutional neural networks (CNNs) have shown promising performance in recognizing oral cancer. However, the lack of interpretability and reliability remain major challenges in the development of trustworthy computer-aided diagnosis systems. To address this issue, we proposed a neural network architecture that integrates visual explanation and attention mechanisms. It improves the recognition performance via the attention mechanism while simultaneously providing interpretability for decision-making. Furthermore, our system incorporates Human-in-the-loop (HITL) deep learning to enhance the reliability and accuracy of the system through the integration of human and machine intelligence. We embedded expert knowledge into the network by manually editing the attention map for the attention mechanism. ABSTRACT: Convolutional neural networks have demonstrated excellent performance in oral cancer detection and classification. However, the end-to-end learning strategy makes CNNs hard to interpret, and it can be challenging to fully understand the decision-making procedure. Additionally, reliability is also a significant challenge for CNN based approaches. In this study, we proposed a neural network called the attention branch network (ABN), which combines the visual explanation and attention mechanisms to improve the recognition performance and interpret the decision-making simultaneously. We also embedded expert knowledge into the network by having human experts manually edit the attention maps for the attention mechanism. Our experiments have shown that ABN performs better than the original baseline network. By introducing the Squeeze-and-Excitation (SE) blocks to the network, the cross-validation accuracy increased further. Furthermore, we observed that some previously misclassified cases were correctly recognized after updating by manually editing the attention maps. The cross-validation accuracy increased from 0.846 to 0.875 with the ABN (Resnet18 as baseline), 0.877 with SE-ABN, and 0.903 after embedding expert knowledge. The proposed method provides an accurate, interpretable, and reliable oral cancer computer-aided diagnosis system through visual explanation, attention mechanisms, and expert knowledge embedding. MDPI 2023-02-23 /pmc/articles/PMC10001266/ /pubmed/36900210 http://dx.doi.org/10.3390/cancers15051421 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Song, Bofan
Zhang, Chicheng
Sunny, Sumsum
KC, Dharma Raj
Li, Shaobai
Gurushanth, Keerthi
Mendonca, Pramila
Mukhia, Nirza
Patrick, Sanjana
Gurudath, Shubha
Raghavan, Subhashini
Tsusennaro, Imchen
Leivon, Shirley T.
Kolur, Trupti
Shetty, Vivek
Bushan, Vidya
Ramesh, Rohan
Pillai, Vijay
Wilder-Smith, Petra
Suresh, Amritha
Kuriakose, Moni Abraham
Birur, Praveen
Liang, Rongguang
Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title_full Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title_fullStr Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title_full_unstemmed Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title_short Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
title_sort interpretable and reliable oral cancer classifier with attention mechanism and expert knowledge embedding via attention map
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10001266/
https://www.ncbi.nlm.nih.gov/pubmed/36900210
http://dx.doi.org/10.3390/cancers15051421
work_keys_str_mv AT songbofan interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT zhangchicheng interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT sunnysumsum interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT kcdharmaraj interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT lishaobai interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT gurushanthkeerthi interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT mendoncapramila interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT mukhianirza interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT patricksanjana interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT gurudathshubha interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT raghavansubhashini interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT tsusennaroimchen interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT leivonshirleyt interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT kolurtrupti interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT shettyvivek interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT bushanvidya interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT rameshrohan interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT pillaivijay interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT wildersmithpetra interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT sureshamritha interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT kuriakosemoniabraham interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT birurpraveen interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap
AT liangrongguang interpretableandreliableoralcancerclassifierwithattentionmechanismandexpertknowledgeembeddingviaattentionmap