Cargando…

Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study

OBJECTIVE: To conduct a multireader validation study to evaluate the interobserver variability and the diagnostic accuracy for the lung involvement by COVID-19 of COVID-19 Reporting and Data System (CO-RADS) score. METHODS: This retrospective study included consecutive symptomatic patients who under...

Descripción completa

Detalles Bibliográficos
Autores principales: Bellini, Davide, Panvini, Nicola, Rengo, Marco, Vicini, Simone, Lichtner, Miriam, Tieghi, Tiziana, Ippoliti, Dea, Giulio, Federica, Orlando, Elena, Iozzino, Mario, Ciolfi, Maria Grazia, Montechiarello, Sarah, d’Ambrosio, Ugo, d’Adamo, Emanuele, Gambaretto, Chiara, Panno, Stefano, Caldon, Vanessa, Ambrogi, Cesare, Carbone, Iacopo
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer Berlin Heidelberg 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7510765/
https://www.ncbi.nlm.nih.gov/pubmed/32968883
http://dx.doi.org/10.1007/s00330-020-07273-y
_version_ 1783585860477779968
author Bellini, Davide
Panvini, Nicola
Rengo, Marco
Vicini, Simone
Lichtner, Miriam
Tieghi, Tiziana
Ippoliti, Dea
Giulio, Federica
Orlando, Elena
Iozzino, Mario
Ciolfi, Maria Grazia
Montechiarello, Sarah
d’Ambrosio, Ugo
d’Adamo, Emanuele
Gambaretto, Chiara
Panno, Stefano
Caldon, Vanessa
Ambrogi, Cesare
Carbone, Iacopo
author_facet Bellini, Davide
Panvini, Nicola
Rengo, Marco
Vicini, Simone
Lichtner, Miriam
Tieghi, Tiziana
Ippoliti, Dea
Giulio, Federica
Orlando, Elena
Iozzino, Mario
Ciolfi, Maria Grazia
Montechiarello, Sarah
d’Ambrosio, Ugo
d’Adamo, Emanuele
Gambaretto, Chiara
Panno, Stefano
Caldon, Vanessa
Ambrogi, Cesare
Carbone, Iacopo
author_sort Bellini, Davide
collection PubMed
description OBJECTIVE: To conduct a multireader validation study to evaluate the interobserver variability and the diagnostic accuracy for the lung involvement by COVID-19 of COVID-19 Reporting and Data System (CO-RADS) score. METHODS: This retrospective study included consecutive symptomatic patients who underwent chest CT and reverse transcriptase-polymerase chain reaction (RT-PCR) from March 2020 to May 2020 for suspected COVID-19. Twelve readers with different levels of expertise independently scored each CT using the CO-RADS scheme for detecting pulmonary involvement by COVID-19. Receiver operating characteristic (ROC) curves were computed to investigate diagnostic yield. Fleiss’ kappa statistics was used to evaluate interreader agreement. RESULTS: A total of 572 patients (mean age, 63 ± 20 [standard deviation]; 329 men; 142 patients with COVID-19 and 430 patients without COVID-19) were evaluated. There was a moderate agreement for CO-RADS rating among all readers (Fleiss’ K = 0.43 [95% CI 0.42–0.44]) with a substantial agreement for CO-RADS 1 category (Fleiss’ K = 0.61 [95% CI 0.60–0.62]) and moderate agreement for CO-RADS 5 category (Fleiss’ K = 0.60 [95% CI 0.58–0.61]). ROC analysis showed the CO-RADS score ≥ 4 as the optimal threshold, with a cumulative area under the curve of 0.72 (95% CI 66–78%), sensitivity 61% (95% CI 52–69%), and specificity 81% (95% CI 77–84%). CONCLUSION: CO-RADS showed high diagnostic accuracy and moderate interrater agreement across readers with different levels of expertise. Specificity is higher than previously thought and that could lead to reconsider the role of CT in this clinical setting. KEY POINTS: • COVID-19 Reporting and Data System (CO-RADS) demonstrated a good diagnostic accuracy for lung involvement by COVID-19 with an average AUC of 0.72 (95% CI 67–75%). • When a threshold of ≥ 4 was used, sensitivity and specificity were 61% (95% CI 52–69%) and 81% (95% CI 76–84%), respectively. • There was an overall moderate agreement for CO-RADS rating across readers with different levels of expertise (Fleiss’ K = 0.43 [95% CI 0.42–0.44]). ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00330-020-07273-y) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-7510765
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Springer Berlin Heidelberg
record_format MEDLINE/PubMed
spelling pubmed-75107652020-09-23 Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study Bellini, Davide Panvini, Nicola Rengo, Marco Vicini, Simone Lichtner, Miriam Tieghi, Tiziana Ippoliti, Dea Giulio, Federica Orlando, Elena Iozzino, Mario Ciolfi, Maria Grazia Montechiarello, Sarah d’Ambrosio, Ugo d’Adamo, Emanuele Gambaretto, Chiara Panno, Stefano Caldon, Vanessa Ambrogi, Cesare Carbone, Iacopo Eur Radiol Chest OBJECTIVE: To conduct a multireader validation study to evaluate the interobserver variability and the diagnostic accuracy for the lung involvement by COVID-19 of COVID-19 Reporting and Data System (CO-RADS) score. METHODS: This retrospective study included consecutive symptomatic patients who underwent chest CT and reverse transcriptase-polymerase chain reaction (RT-PCR) from March 2020 to May 2020 for suspected COVID-19. Twelve readers with different levels of expertise independently scored each CT using the CO-RADS scheme for detecting pulmonary involvement by COVID-19. Receiver operating characteristic (ROC) curves were computed to investigate diagnostic yield. Fleiss’ kappa statistics was used to evaluate interreader agreement. RESULTS: A total of 572 patients (mean age, 63 ± 20 [standard deviation]; 329 men; 142 patients with COVID-19 and 430 patients without COVID-19) were evaluated. There was a moderate agreement for CO-RADS rating among all readers (Fleiss’ K = 0.43 [95% CI 0.42–0.44]) with a substantial agreement for CO-RADS 1 category (Fleiss’ K = 0.61 [95% CI 0.60–0.62]) and moderate agreement for CO-RADS 5 category (Fleiss’ K = 0.60 [95% CI 0.58–0.61]). ROC analysis showed the CO-RADS score ≥ 4 as the optimal threshold, with a cumulative area under the curve of 0.72 (95% CI 66–78%), sensitivity 61% (95% CI 52–69%), and specificity 81% (95% CI 77–84%). CONCLUSION: CO-RADS showed high diagnostic accuracy and moderate interrater agreement across readers with different levels of expertise. Specificity is higher than previously thought and that could lead to reconsider the role of CT in this clinical setting. KEY POINTS: • COVID-19 Reporting and Data System (CO-RADS) demonstrated a good diagnostic accuracy for lung involvement by COVID-19 with an average AUC of 0.72 (95% CI 67–75%). • When a threshold of ≥ 4 was used, sensitivity and specificity were 61% (95% CI 52–69%) and 81% (95% CI 76–84%), respectively. • There was an overall moderate agreement for CO-RADS rating across readers with different levels of expertise (Fleiss’ K = 0.43 [95% CI 0.42–0.44]). ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00330-020-07273-y) contains supplementary material, which is available to authorized users. Springer Berlin Heidelberg 2020-09-23 2021 /pmc/articles/PMC7510765/ /pubmed/32968883 http://dx.doi.org/10.1007/s00330-020-07273-y Text en © European Society of Radiology 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle Chest
Bellini, Davide
Panvini, Nicola
Rengo, Marco
Vicini, Simone
Lichtner, Miriam
Tieghi, Tiziana
Ippoliti, Dea
Giulio, Federica
Orlando, Elena
Iozzino, Mario
Ciolfi, Maria Grazia
Montechiarello, Sarah
d’Ambrosio, Ugo
d’Adamo, Emanuele
Gambaretto, Chiara
Panno, Stefano
Caldon, Vanessa
Ambrogi, Cesare
Carbone, Iacopo
Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title_full Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title_fullStr Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title_full_unstemmed Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title_short Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study
title_sort diagnostic accuracy and interobserver variability of co-rads in patients with suspected coronavirus disease-2019: a multireader validation study
topic Chest
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7510765/
https://www.ncbi.nlm.nih.gov/pubmed/32968883
http://dx.doi.org/10.1007/s00330-020-07273-y
work_keys_str_mv AT bellinidavide diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT panvininicola diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT rengomarco diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT vicinisimone diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT lichtnermiriam diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT tieghitiziana diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT ippolitidea diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT giuliofederica diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT orlandoelena diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT iozzinomario diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT ciolfimariagrazia diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT montechiarellosarah diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT dambrosiougo diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT dadamoemanuele diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT gambarettochiara diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT pannostefano diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT caldonvanessa diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT ambrogicesare diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy
AT carboneiacopo diagnosticaccuracyandinterobservervariabilityofcoradsinpatientswithsuspectedcoronavirusdisease2019amultireadervalidationstudy