Cargando…

Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images

Objectives: The present study aimed to train deep convolutional neural networks (CNNs) to detect caries lesions on Near-Infrared Light Transillumination (NILT) imagery obtained either in vitro or in vivo and to assess the models’ generalizability. Methods: In vitro, 226 extracted posterior permanent...

Descripción completa

Detalles Bibliográficos
Autores principales: Holtkamp, Agnes, Elhennawy, Karim, Cejudo Grano de Oro, José E., Krois, Joachim, Paris, Sebastian, Schwendicke, Falk
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7957685/
https://www.ncbi.nlm.nih.gov/pubmed/33804562
http://dx.doi.org/10.3390/jcm10050961
_version_ 1783664705915584512
author Holtkamp, Agnes
Elhennawy, Karim
Cejudo Grano de Oro, José E.
Krois, Joachim
Paris, Sebastian
Schwendicke, Falk
author_facet Holtkamp, Agnes
Elhennawy, Karim
Cejudo Grano de Oro, José E.
Krois, Joachim
Paris, Sebastian
Schwendicke, Falk
author_sort Holtkamp, Agnes
collection PubMed
description Objectives: The present study aimed to train deep convolutional neural networks (CNNs) to detect caries lesions on Near-Infrared Light Transillumination (NILT) imagery obtained either in vitro or in vivo and to assess the models’ generalizability. Methods: In vitro, 226 extracted posterior permanent human teeth were mounted in a diagnostic model in a dummy head. Then, NILT images were generated (DIAGNOcam, KaVo, Biberach), and images were segmented tooth-wise. In vivo, 1319 teeth from 56 patients were obtained and segmented similarly. Proximal caries lesions were annotated pixel-wise by three experienced dentists, reviewed by a fourth dentist, and then transformed into binary labels. We trained ResNet classification models on both in vivo and in vitro datasets and used 10-fold cross-validation for estimating the performance and generalizability of the models. We used GradCAM to increase explainability. Results: The tooth-level prevalence of caries lesions was 41% in vitro and 49% in vivo, respectively. Models trained and tested on in vivo data performed significantly better (mean ± SD accuracy: 0.78 ± 0.04) than those trained and tested on in vitro data (accuracy: 0.64 ± 0.15; p < 0.05). When tested in vitro, the models trained in vivo showed significantly lower accuracy (0.70 ± 0.01; p < 0.01). Similarly, when tested in vivo, models trained in vitro showed significantly lower accuracy (0.61 ± 0.04; p < 0.05). In both cases, this was due to decreases in sensitivity (by −27% for models trained in vivo and −10% for models trained in vitro). Conclusions: Using in vitro setups for generating NILT imagery and training CNNs comes with low accuracy and generalizability. Clinical significance: Studies employing in vitro imagery for developing deep learning models should be critically appraised for their generalizability. Applicable deep learning models for assessing NILT imagery should be trained on in vivo data.
format Online
Article
Text
id pubmed-7957685
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-79576852021-03-16 Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images Holtkamp, Agnes Elhennawy, Karim Cejudo Grano de Oro, José E. Krois, Joachim Paris, Sebastian Schwendicke, Falk J Clin Med Article Objectives: The present study aimed to train deep convolutional neural networks (CNNs) to detect caries lesions on Near-Infrared Light Transillumination (NILT) imagery obtained either in vitro or in vivo and to assess the models’ generalizability. Methods: In vitro, 226 extracted posterior permanent human teeth were mounted in a diagnostic model in a dummy head. Then, NILT images were generated (DIAGNOcam, KaVo, Biberach), and images were segmented tooth-wise. In vivo, 1319 teeth from 56 patients were obtained and segmented similarly. Proximal caries lesions were annotated pixel-wise by three experienced dentists, reviewed by a fourth dentist, and then transformed into binary labels. We trained ResNet classification models on both in vivo and in vitro datasets and used 10-fold cross-validation for estimating the performance and generalizability of the models. We used GradCAM to increase explainability. Results: The tooth-level prevalence of caries lesions was 41% in vitro and 49% in vivo, respectively. Models trained and tested on in vivo data performed significantly better (mean ± SD accuracy: 0.78 ± 0.04) than those trained and tested on in vitro data (accuracy: 0.64 ± 0.15; p < 0.05). When tested in vitro, the models trained in vivo showed significantly lower accuracy (0.70 ± 0.01; p < 0.01). Similarly, when tested in vivo, models trained in vitro showed significantly lower accuracy (0.61 ± 0.04; p < 0.05). In both cases, this was due to decreases in sensitivity (by −27% for models trained in vivo and −10% for models trained in vitro). Conclusions: Using in vitro setups for generating NILT imagery and training CNNs comes with low accuracy and generalizability. Clinical significance: Studies employing in vitro imagery for developing deep learning models should be critically appraised for their generalizability. Applicable deep learning models for assessing NILT imagery should be trained on in vivo data. MDPI 2021-03-01 /pmc/articles/PMC7957685/ /pubmed/33804562 http://dx.doi.org/10.3390/jcm10050961 Text en © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Holtkamp, Agnes
Elhennawy, Karim
Cejudo Grano de Oro, José E.
Krois, Joachim
Paris, Sebastian
Schwendicke, Falk
Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title_full Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title_fullStr Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title_full_unstemmed Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title_short Generalizability of Deep Learning Models for Caries Detection in Near-Infrared Light Transillumination Images
title_sort generalizability of deep learning models for caries detection in near-infrared light transillumination images
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7957685/
https://www.ncbi.nlm.nih.gov/pubmed/33804562
http://dx.doi.org/10.3390/jcm10050961
work_keys_str_mv AT holtkampagnes generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages
AT elhennawykarim generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages
AT cejudogranodeorojosee generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages
AT kroisjoachim generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages
AT parissebastian generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages
AT schwendickefalk generalizabilityofdeeplearningmodelsforcariesdetectioninnearinfraredlighttransilluminationimages