Cargando…
Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-...
Autores principales: | , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8568139/ https://www.ncbi.nlm.nih.gov/pubmed/34735458 http://dx.doi.org/10.1371/journal.pone.0258760 |
_version_ | 1784594371468853248 |
---|---|
author | Ikenoue, Tatsuyoshi Kataoka, Yuki Matsuoka, Yoshinori Matsumoto, Junichi Kumasawa, Junji Tochitatni, Kentaro Funakoshi, Hiraku Hosoda, Tomohiro Kugimiya, Aiko Shirano, Michinori Hamabe, Fumiko Iwata, Sachiyo Fukuma, Shingo |
author_facet | Ikenoue, Tatsuyoshi Kataoka, Yuki Matsuoka, Yoshinori Matsumoto, Junichi Kumasawa, Junji Tochitatni, Kentaro Funakoshi, Hiraku Hosoda, Tomohiro Kugimiya, Aiko Shirano, Michinori Hamabe, Fumiko Iwata, Sachiyo Fukuma, Shingo |
author_sort | Ikenoue, Tatsuyoshi |
collection | PubMed |
description | Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19. |
format | Online Article Text |
id | pubmed-8568139 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-85681392021-11-05 Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study Ikenoue, Tatsuyoshi Kataoka, Yuki Matsuoka, Yoshinori Matsumoto, Junichi Kumasawa, Junji Tochitatni, Kentaro Funakoshi, Hiraku Hosoda, Tomohiro Kugimiya, Aiko Shirano, Michinori Hamabe, Fumiko Iwata, Sachiyo Fukuma, Shingo PLoS One Research Article Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19. Public Library of Science 2021-11-04 /pmc/articles/PMC8568139/ /pubmed/34735458 http://dx.doi.org/10.1371/journal.pone.0258760 Text en © 2021 Ikenoue et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Ikenoue, Tatsuyoshi Kataoka, Yuki Matsuoka, Yoshinori Matsumoto, Junichi Kumasawa, Junji Tochitatni, Kentaro Funakoshi, Hiraku Hosoda, Tomohiro Kugimiya, Aiko Shirano, Michinori Hamabe, Fumiko Iwata, Sachiyo Fukuma, Shingo Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title | Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title_full | Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title_fullStr | Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title_full_unstemmed | Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title_short | Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study |
title_sort | accuracy of deep learning-based computed tomography diagnostic system for covid-19: a consecutive sampling external validation cohort study |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8568139/ https://www.ncbi.nlm.nih.gov/pubmed/34735458 http://dx.doi.org/10.1371/journal.pone.0258760 |
work_keys_str_mv | AT ikenouetatsuyoshi accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT kataokayuki accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT matsuokayoshinori accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT matsumotojunichi accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT kumasawajunji accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT tochitatnikentaro accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT funakoshihiraku accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT hosodatomohiro accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT kugimiyaaiko accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT shiranomichinori accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT hamabefumiko accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT iwatasachiyo accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT fukumashingo accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy |