Cargando…

Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study

Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-...

Descripción completa

Detalles Bibliográficos
Autores principales: Ikenoue, Tatsuyoshi, Kataoka, Yuki, Matsuoka, Yoshinori, Matsumoto, Junichi, Kumasawa, Junji, Tochitatni, Kentaro, Funakoshi, Hiraku, Hosoda, Tomohiro, Kugimiya, Aiko, Shirano, Michinori, Hamabe, Fumiko, Iwata, Sachiyo, Fukuma, Shingo
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8568139/
https://www.ncbi.nlm.nih.gov/pubmed/34735458
http://dx.doi.org/10.1371/journal.pone.0258760
_version_ 1784594371468853248
author Ikenoue, Tatsuyoshi
Kataoka, Yuki
Matsuoka, Yoshinori
Matsumoto, Junichi
Kumasawa, Junji
Tochitatni, Kentaro
Funakoshi, Hiraku
Hosoda, Tomohiro
Kugimiya, Aiko
Shirano, Michinori
Hamabe, Fumiko
Iwata, Sachiyo
Fukuma, Shingo
author_facet Ikenoue, Tatsuyoshi
Kataoka, Yuki
Matsuoka, Yoshinori
Matsumoto, Junichi
Kumasawa, Junji
Tochitatni, Kentaro
Funakoshi, Hiraku
Hosoda, Tomohiro
Kugimiya, Aiko
Shirano, Michinori
Hamabe, Fumiko
Iwata, Sachiyo
Fukuma, Shingo
author_sort Ikenoue, Tatsuyoshi
collection PubMed
description Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19.
format Online
Article
Text
id pubmed-8568139
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-85681392021-11-05 Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study Ikenoue, Tatsuyoshi Kataoka, Yuki Matsuoka, Yoshinori Matsumoto, Junichi Kumasawa, Junji Tochitatni, Kentaro Funakoshi, Hiraku Hosoda, Tomohiro Kugimiya, Aiko Shirano, Michinori Hamabe, Fumiko Iwata, Sachiyo Fukuma, Shingo PLoS One Research Article Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19. Public Library of Science 2021-11-04 /pmc/articles/PMC8568139/ /pubmed/34735458 http://dx.doi.org/10.1371/journal.pone.0258760 Text en © 2021 Ikenoue et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Ikenoue, Tatsuyoshi
Kataoka, Yuki
Matsuoka, Yoshinori
Matsumoto, Junichi
Kumasawa, Junji
Tochitatni, Kentaro
Funakoshi, Hiraku
Hosoda, Tomohiro
Kugimiya, Aiko
Shirano, Michinori
Hamabe, Fumiko
Iwata, Sachiyo
Fukuma, Shingo
Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_full Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_fullStr Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_full_unstemmed Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_short Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_sort accuracy of deep learning-based computed tomography diagnostic system for covid-19: a consecutive sampling external validation cohort study
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8568139/
https://www.ncbi.nlm.nih.gov/pubmed/34735458
http://dx.doi.org/10.1371/journal.pone.0258760
work_keys_str_mv AT ikenouetatsuyoshi accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT kataokayuki accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT matsuokayoshinori accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT matsumotojunichi accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT kumasawajunji accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT tochitatnikentaro accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT funakoshihiraku accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT hosodatomohiro accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT kugimiyaaiko accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT shiranomichinori accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT hamabefumiko accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT iwatasachiyo accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT fukumashingo accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
AT accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy