Cargando…

Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis

There have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with differen...

Descripción completa

Detalles Bibliográficos
Autores principales: Codlin, Andrew J., Dao, Thang Phuoc, Vo, Luan Nguyen Quang, Forse, Rachel J., Van Truong, Vinh, Dang, Ha Minh, Nguyen, Lan Huu, Nguyen, Hoa Binh, Nguyen, Nhung Viet, Sidney-Annerstedt, Kristi, Squire, Bertie, Lönnroth, Knut, Caws, Maxine
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8668935/
https://www.ncbi.nlm.nih.gov/pubmed/34903808
http://dx.doi.org/10.1038/s41598-021-03265-0
_version_ 1784614685400629248
author Codlin, Andrew J.
Dao, Thang Phuoc
Vo, Luan Nguyen Quang
Forse, Rachel J.
Van Truong, Vinh
Dang, Ha Minh
Nguyen, Lan Huu
Nguyen, Hoa Binh
Nguyen, Nhung Viet
Sidney-Annerstedt, Kristi
Squire, Bertie
Lönnroth, Knut
Caws, Maxine
author_facet Codlin, Andrew J.
Dao, Thang Phuoc
Vo, Luan Nguyen Quang
Forse, Rachel J.
Van Truong, Vinh
Dang, Ha Minh
Nguyen, Lan Huu
Nguyen, Hoa Binh
Nguyen, Nhung Viet
Sidney-Annerstedt, Kristi
Squire, Bertie
Lönnroth, Knut
Caws, Maxine
author_sort Codlin, Andrew J.
collection PubMed
description There have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with different levels of experience and then processed by 12 CAD software solutions. Using Xpert MTB/RIF results as the reference standard, we compared the performance characteristics of each CAD software against both an Expert and Intermediate Reader, using cut-off thresholds which were selected to match the sensitivity of each human reader. Six CAD systems performed on par with the Expert Reader (Qure.ai, DeepTek, Delft Imaging, JF Healthcare, OXIPIT, and Lunit) and one additional software (Infervision) performed on par with the Intermediate Reader only. Qure.ai, Delft Imaging and Lunit were the only software to perform significantly better than the Intermediate Reader. The majority of these CAD software showed significantly lower performance among participants with a past history of TB. The radiography equipment used to capture the CXR image was also shown to affect performance for some CAD software. TB program implementers now have a wide selection of quality CAD software solutions to utilize in their CXR screening initiatives.
format Online
Article
Text
id pubmed-8668935
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-86689352021-12-15 Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis Codlin, Andrew J. Dao, Thang Phuoc Vo, Luan Nguyen Quang Forse, Rachel J. Van Truong, Vinh Dang, Ha Minh Nguyen, Lan Huu Nguyen, Hoa Binh Nguyen, Nhung Viet Sidney-Annerstedt, Kristi Squire, Bertie Lönnroth, Knut Caws, Maxine Sci Rep Article There have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with different levels of experience and then processed by 12 CAD software solutions. Using Xpert MTB/RIF results as the reference standard, we compared the performance characteristics of each CAD software against both an Expert and Intermediate Reader, using cut-off thresholds which were selected to match the sensitivity of each human reader. Six CAD systems performed on par with the Expert Reader (Qure.ai, DeepTek, Delft Imaging, JF Healthcare, OXIPIT, and Lunit) and one additional software (Infervision) performed on par with the Intermediate Reader only. Qure.ai, Delft Imaging and Lunit were the only software to perform significantly better than the Intermediate Reader. The majority of these CAD software showed significantly lower performance among participants with a past history of TB. The radiography equipment used to capture the CXR image was also shown to affect performance for some CAD software. TB program implementers now have a wide selection of quality CAD software solutions to utilize in their CXR screening initiatives. Nature Publishing Group UK 2021-12-13 /pmc/articles/PMC8668935/ /pubmed/34903808 http://dx.doi.org/10.1038/s41598-021-03265-0 Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Codlin, Andrew J.
Dao, Thang Phuoc
Vo, Luan Nguyen Quang
Forse, Rachel J.
Van Truong, Vinh
Dang, Ha Minh
Nguyen, Lan Huu
Nguyen, Hoa Binh
Nguyen, Nhung Viet
Sidney-Annerstedt, Kristi
Squire, Bertie
Lönnroth, Knut
Caws, Maxine
Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title_full Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title_fullStr Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title_full_unstemmed Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title_short Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
title_sort independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8668935/
https://www.ncbi.nlm.nih.gov/pubmed/34903808
http://dx.doi.org/10.1038/s41598-021-03265-0
work_keys_str_mv AT codlinandrewj independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT daothangphuoc independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT voluannguyenquang independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT forserachelj independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT vantruongvinh independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT danghaminh independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT nguyenlanhuu independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT nguyenhoabinh independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT nguyennhungviet independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT sidneyannerstedtkristi independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT squirebertie independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT lonnrothknut independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis
AT cawsmaxine independentevaluationof12artificialintelligencesolutionsforthedetectionoftuberculosis