Cargando…
Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays
Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generat...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8775073/ https://www.ncbi.nlm.nih.gov/pubmed/35054355 http://dx.doi.org/10.3390/diagnostics12010188 |
_version_ | 1784636494162427904 |
---|---|
author | Karki, Manohar Kantipudi, Karthik Yang, Feng Yu, Hang Wang, Yi Xiang J. Yaniv, Ziv Jaeger, Stefan |
author_facet | Karki, Manohar Kantipudi, Karthik Yang, Feng Yu, Hang Wang, Yi Xiang J. Yaniv, Ziv Jaeger, Stefan |
author_sort | Karki, Manohar |
collection | PubMed |
description | Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generated and publicly available images achieved a performance of 85% AUC with a deep convolutional neural network (CNN). However, when we evaluated the CNN model trained to classify DR-TB and DS-TB on unseen data, significant performance degradation was observed (65% AUC). Hence, in this paper, we investigate the generalizability of our models on images from a held out country’s dataset. We explore the extent of the problem and the possible reasons behind the lack of good generalization. A comparison of radiologist-annotated lesion locations in the lung and the trained model’s localization of areas of interest, using GradCAM, did not show much overlap. Using the same network architecture, a multi-country classifier was able to identify the country of origin of the X-ray with high accuracy (86%), suggesting that image acquisition differences and the distribution of non-pathological and non-anatomical aspects of the images are affecting the generalization and localization of the drug resistance classification model as well. When CXR images were severely corrupted, the performance on the validation set was still better than 60% AUC. The model overfitted to the data from countries in the cross validation set but did not generalize to the held out country. Finally, we applied a multi-task based approach that uses prior TB lesions location information to guide the classifier network to focus its attention on improving the generalization performance on the held out set from another country to 68% AUC. |
format | Online Article Text |
id | pubmed-8775073 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-87750732022-01-21 Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays Karki, Manohar Kantipudi, Karthik Yang, Feng Yu, Hang Wang, Yi Xiang J. Yaniv, Ziv Jaeger, Stefan Diagnostics (Basel) Article Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generated and publicly available images achieved a performance of 85% AUC with a deep convolutional neural network (CNN). However, when we evaluated the CNN model trained to classify DR-TB and DS-TB on unseen data, significant performance degradation was observed (65% AUC). Hence, in this paper, we investigate the generalizability of our models on images from a held out country’s dataset. We explore the extent of the problem and the possible reasons behind the lack of good generalization. A comparison of radiologist-annotated lesion locations in the lung and the trained model’s localization of areas of interest, using GradCAM, did not show much overlap. Using the same network architecture, a multi-country classifier was able to identify the country of origin of the X-ray with high accuracy (86%), suggesting that image acquisition differences and the distribution of non-pathological and non-anatomical aspects of the images are affecting the generalization and localization of the drug resistance classification model as well. When CXR images were severely corrupted, the performance on the validation set was still better than 60% AUC. The model overfitted to the data from countries in the cross validation set but did not generalize to the held out country. Finally, we applied a multi-task based approach that uses prior TB lesions location information to guide the classifier network to focus its attention on improving the generalization performance on the held out set from another country to 68% AUC. MDPI 2022-01-13 /pmc/articles/PMC8775073/ /pubmed/35054355 http://dx.doi.org/10.3390/diagnostics12010188 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Karki, Manohar Kantipudi, Karthik Yang, Feng Yu, Hang Wang, Yi Xiang J. Yaniv, Ziv Jaeger, Stefan Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title | Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title_full | Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title_fullStr | Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title_full_unstemmed | Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title_short | Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays |
title_sort | generalization challenges in drug-resistant tuberculosis detection from chest x-rays |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8775073/ https://www.ncbi.nlm.nih.gov/pubmed/35054355 http://dx.doi.org/10.3390/diagnostics12010188 |
work_keys_str_mv | AT karkimanohar generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT kantipudikarthik generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT yangfeng generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT yuhang generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT wangyixiangj generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT yanivziv generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays AT jaegerstefan generalizationchallengesindrugresistanttuberculosisdetectionfromchestxrays |