Cargando…

Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays

Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generat...

Descripción completa

Detalles Bibliográficos
Autores principales: Karki, Manohar, Kantipudi, Karthik, Yang, Feng, Yu, Hang, Wang, Yi Xiang J., Yaniv, Ziv, Jaeger, Stefan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8775073/
https://www.ncbi.nlm.nih.gov/pubmed/35054355
http://dx.doi.org/10.3390/diagnostics12010188
_version_ 1784636494162427904
author Karki, Manohar
Kantipudi, Karthik
Yang, Feng
Yu, Hang
Wang, Yi Xiang J.
Yaniv, Ziv
Jaeger, Stefan
author_facet Karki, Manohar
Kantipudi, Karthik
Yang, Feng
Yu, Hang
Wang, Yi Xiang J.
Yaniv, Ziv
Jaeger, Stefan
author_sort Karki, Manohar
collection PubMed
description Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generated and publicly available images achieved a performance of 85% AUC with a deep convolutional neural network (CNN). However, when we evaluated the CNN model trained to classify DR-TB and DS-TB on unseen data, significant performance degradation was observed (65% AUC). Hence, in this paper, we investigate the generalizability of our models on images from a held out country’s dataset. We explore the extent of the problem and the possible reasons behind the lack of good generalization. A comparison of radiologist-annotated lesion locations in the lung and the trained model’s localization of areas of interest, using GradCAM, did not show much overlap. Using the same network architecture, a multi-country classifier was able to identify the country of origin of the X-ray with high accuracy (86%), suggesting that image acquisition differences and the distribution of non-pathological and non-anatomical aspects of the images are affecting the generalization and localization of the drug resistance classification model as well. When CXR images were severely corrupted, the performance on the validation set was still better than 60% AUC. The model overfitted to the data from countries in the cross validation set but did not generalize to the held out country. Finally, we applied a multi-task based approach that uses prior TB lesions location information to guide the classifier network to focus its attention on improving the generalization performance on the held out set from another country to 68% AUC.
format Online
Article
Text
id pubmed-8775073
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-87750732022-01-21 Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays Karki, Manohar Kantipudi, Karthik Yang, Feng Yu, Hang Wang, Yi Xiang J. Yaniv, Ziv Jaeger, Stefan Diagnostics (Basel) Article Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generated and publicly available images achieved a performance of 85% AUC with a deep convolutional neural network (CNN). However, when we evaluated the CNN model trained to classify DR-TB and DS-TB on unseen data, significant performance degradation was observed (65% AUC). Hence, in this paper, we investigate the generalizability of our models on images from a held out country’s dataset. We explore the extent of the problem and the possible reasons behind the lack of good generalization. A comparison of radiologist-annotated lesion locations in the lung and the trained model’s localization of areas of interest, using GradCAM, did not show much overlap. Using the same network architecture, a multi-country classifier was able to identify the country of origin of the X-ray with high accuracy (86%), suggesting that image acquisition differences and the distribution of non-pathological and non-anatomical aspects of the images are affecting the generalization and localization of the drug resistance classification model as well. When CXR images were severely corrupted, the performance on the validation set was still better than 60% AUC. The model overfitted to the data from countries in the cross validation set but did not generalize to the held out country. Finally, we applied a multi-task based approach that uses prior TB lesions location information to guide the classifier network to focus its attention on improving the generalization performance on the held out set from another country to 68% AUC. MDPI 2022-01-13 /pmc/articles/PMC8775073/ /pubmed/35054355 http://dx.doi.org/10.3390/diagnostics12010188 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Karki, Manohar
Kantipudi, Karthik
Yang, Feng
Yu, Hang
Wang, Yi Xiang J.
Yaniv, Ziv
Jaeger, Stefan
Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title_full Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title_fullStr Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title_full_unstemmed Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title_short Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
title_sort generalization challenges in drug-resistant tuberculosis detection from chest x-rays
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8775073/
https://www.ncbi.nlm.nih.gov/pubmed/35054355
http://dx.doi.org/10.3390/diagnostics12010188
work_keys_str_mv AT karkimanohar generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT kantipudikarthik generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT yangfeng generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT yuhang generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT wangyixiangj generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT yanivziv generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays
AT jaegerstefan generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays