Cargando…

Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset

Accurate detection of respiratory system damage including COVID-19 is considered one of the crucial applications of deep learning (DL) models using CT images. However, the main shortcoming of the published works has been unreliable reported accuracy and the lack of repeatability with new datasets, m...

Descripción completa

Detalles Bibliográficos
Autores principales: Parsarad, Shiva, Saeedizadeh, Narges, Soufi, Ghazaleh Jamalipour, Shafieyoon, Shamim, Hekmatnia, Farzaneh, Zarei, Andrew Parviz, Soleimany, Samira, Yousefi, Amir, Nazari, Hengameh, Torabi, Pegah, S. Milani, Abbas, Madani Tonekaboni, Seyed Ali, Rabbani, Hossein, Hekmatnia, Ali, Kafieh, Rahele
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10455108/
https://www.ncbi.nlm.nih.gov/pubmed/37623691
http://dx.doi.org/10.3390/jimaging9080159
_version_ 1785096368010821632
author Parsarad, Shiva
Saeedizadeh, Narges
Soufi, Ghazaleh Jamalipour
Shafieyoon, Shamim
Hekmatnia, Farzaneh
Zarei, Andrew Parviz
Soleimany, Samira
Yousefi, Amir
Nazari, Hengameh
Torabi, Pegah
S. Milani, Abbas
Madani Tonekaboni, Seyed Ali
Rabbani, Hossein
Hekmatnia, Ali
Kafieh, Rahele
author_facet Parsarad, Shiva
Saeedizadeh, Narges
Soufi, Ghazaleh Jamalipour
Shafieyoon, Shamim
Hekmatnia, Farzaneh
Zarei, Andrew Parviz
Soleimany, Samira
Yousefi, Amir
Nazari, Hengameh
Torabi, Pegah
S. Milani, Abbas
Madani Tonekaboni, Seyed Ali
Rabbani, Hossein
Hekmatnia, Ali
Kafieh, Rahele
author_sort Parsarad, Shiva
collection PubMed
description Accurate detection of respiratory system damage including COVID-19 is considered one of the crucial applications of deep learning (DL) models using CT images. However, the main shortcoming of the published works has been unreliable reported accuracy and the lack of repeatability with new datasets, mainly due to slice-wise splits of the data, creating dependency between training and test sets due to shared data across the sets. We introduce a new dataset of CT images (ISFCT Dataset) with labels indicating the subject-wise split to train and test our DL algorithms in an unbiased manner. We also use this dataset to validate the real performance of the published works in a subject-wise data split. Another key feature provides more specific labels (eight characteristic lung features) rather than being limited to COVID-19 and healthy labels. We show that the reported high accuracy of the existing models on current slice-wise splits is not repeatable for subject-wise splits, and distribution differences between data splits are demonstrated using t-distribution stochastic neighbor embedding. We indicate that, by examining subject-wise data splitting, less complicated models show competitive results compared to the exiting complicated models, demonstrating that complex models do not necessarily generate accurate and repeatable results.
format Online
Article
Text
id pubmed-10455108
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-104551082023-08-26 Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset Parsarad, Shiva Saeedizadeh, Narges Soufi, Ghazaleh Jamalipour Shafieyoon, Shamim Hekmatnia, Farzaneh Zarei, Andrew Parviz Soleimany, Samira Yousefi, Amir Nazari, Hengameh Torabi, Pegah S. Milani, Abbas Madani Tonekaboni, Seyed Ali Rabbani, Hossein Hekmatnia, Ali Kafieh, Rahele J Imaging Article Accurate detection of respiratory system damage including COVID-19 is considered one of the crucial applications of deep learning (DL) models using CT images. However, the main shortcoming of the published works has been unreliable reported accuracy and the lack of repeatability with new datasets, mainly due to slice-wise splits of the data, creating dependency between training and test sets due to shared data across the sets. We introduce a new dataset of CT images (ISFCT Dataset) with labels indicating the subject-wise split to train and test our DL algorithms in an unbiased manner. We also use this dataset to validate the real performance of the published works in a subject-wise data split. Another key feature provides more specific labels (eight characteristic lung features) rather than being limited to COVID-19 and healthy labels. We show that the reported high accuracy of the existing models on current slice-wise splits is not repeatable for subject-wise splits, and distribution differences between data splits are demonstrated using t-distribution stochastic neighbor embedding. We indicate that, by examining subject-wise data splitting, less complicated models show competitive results compared to the exiting complicated models, demonstrating that complex models do not necessarily generate accurate and repeatable results. MDPI 2023-08-08 /pmc/articles/PMC10455108/ /pubmed/37623691 http://dx.doi.org/10.3390/jimaging9080159 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Parsarad, Shiva
Saeedizadeh, Narges
Soufi, Ghazaleh Jamalipour
Shafieyoon, Shamim
Hekmatnia, Farzaneh
Zarei, Andrew Parviz
Soleimany, Samira
Yousefi, Amir
Nazari, Hengameh
Torabi, Pegah
S. Milani, Abbas
Madani Tonekaboni, Seyed Ali
Rabbani, Hossein
Hekmatnia, Ali
Kafieh, Rahele
Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title_full Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title_fullStr Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title_full_unstemmed Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title_short Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split ISFCT Dataset
title_sort biased deep learning methods in detection of covid-19 using ct images: a challenge mounted by subject-wise-split isfct dataset
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10455108/
https://www.ncbi.nlm.nih.gov/pubmed/37623691
http://dx.doi.org/10.3390/jimaging9080159
work_keys_str_mv AT parsaradshiva biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT saeedizadehnarges biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT soufighazalehjamalipour biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT shafieyoonshamim biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT hekmatniafarzaneh biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT zareiandrewparviz biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT soleimanysamira biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT yousefiamir biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT nazarihengameh biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT torabipegah biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT smilaniabbas biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT madanitonekaboniseyedali biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT rabbanihossein biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT hekmatniaali biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset
AT kafiehrahele biaseddeeplearningmethodsindetectionofcovid19usingctimagesachallengemountedbysubjectwisesplitisfctdataset