Cargando…
Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs
BACKGROUND: Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CX...
Autores principales: | , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Published by Elsevier Inc.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8585957/ https://www.ncbi.nlm.nih.gov/pubmed/34798562 http://dx.doi.org/10.1016/j.clinimag.2021.10.018 |
_version_ | 1784597795837050880 |
---|---|
author | Gichoya, Judy W. Sinha, Priyanshu Davis, Melissa Dunkle, Jeffrey W. Hamlin, Scott A. Herr, Keith D. Hoff, Carrie N. Letter, Haley P. McAdams, Christopher R. Puthoff, Gregory D. Smith, Kevin L. Steenburg, Scott D. Banerjee, Imon Trivedi, Hari |
author_facet | Gichoya, Judy W. Sinha, Priyanshu Davis, Melissa Dunkle, Jeffrey W. Hamlin, Scott A. Herr, Keith D. Hoff, Carrie N. Letter, Haley P. McAdams, Christopher R. Puthoff, Gregory D. Smith, Kevin L. Steenburg, Scott D. Banerjee, Imon Trivedi, Hari |
author_sort | Gichoya, Judy W. |
collection | PubMed |
description | BACKGROUND: Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED). MATERIALS AND METHODS: We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other – Infectious, Other – Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube (246) or non-diagnostic (54) were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement. RESULTS: 1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n = 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result. CONCLUSION: At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19. |
format | Online Article Text |
id | pubmed-8585957 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Published by Elsevier Inc. |
record_format | MEDLINE/PubMed |
spelling | pubmed-85859572021-11-12 Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs Gichoya, Judy W. Sinha, Priyanshu Davis, Melissa Dunkle, Jeffrey W. Hamlin, Scott A. Herr, Keith D. Hoff, Carrie N. Letter, Haley P. McAdams, Christopher R. Puthoff, Gregory D. Smith, Kevin L. Steenburg, Scott D. Banerjee, Imon Trivedi, Hari Clin Imaging Cardiothoracic Imaging BACKGROUND: Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED). MATERIALS AND METHODS: We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other – Infectious, Other – Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube (246) or non-diagnostic (54) were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement. RESULTS: 1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n = 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result. CONCLUSION: At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19. Published by Elsevier Inc. 2022-02 2021-11-12 /pmc/articles/PMC8585957/ /pubmed/34798562 http://dx.doi.org/10.1016/j.clinimag.2021.10.018 Text en © 2021 Published by Elsevier Inc. Since January 2020 Elsevier has created a COVID-19 resource centre with free information in English and Mandarin on the novel coronavirus COVID-19. The COVID-19 resource centre is hosted on Elsevier Connect, the company's public news and information website. Elsevier hereby grants permission to make all its COVID-19-related research that is available on the COVID-19 resource centre - including this research content - immediately available in PubMed Central and other publicly funded repositories, such as the WHO COVID database with rights for unrestricted research re-use and analyses in any form or by any means with acknowledgement of the original source. These permissions are granted for free by Elsevier for as long as the COVID-19 resource centre remains active. |
spellingShingle | Cardiothoracic Imaging Gichoya, Judy W. Sinha, Priyanshu Davis, Melissa Dunkle, Jeffrey W. Hamlin, Scott A. Herr, Keith D. Hoff, Carrie N. Letter, Haley P. McAdams, Christopher R. Puthoff, Gregory D. Smith, Kevin L. Steenburg, Scott D. Banerjee, Imon Trivedi, Hari Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title | Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title_full | Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title_fullStr | Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title_full_unstemmed | Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title_short | Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs |
title_sort | multireader evaluation of radiologist performance for covid-19 detection on emergency department chest radiographs |
topic | Cardiothoracic Imaging |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8585957/ https://www.ncbi.nlm.nih.gov/pubmed/34798562 http://dx.doi.org/10.1016/j.clinimag.2021.10.018 |
work_keys_str_mv | AT gichoyajudyw multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT sinhapriyanshu multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT davismelissa multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT dunklejeffreyw multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT hamlinscotta multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT herrkeithd multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT hoffcarrien multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT letterhaleyp multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT mcadamschristopherr multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT puthoffgregoryd multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT smithkevinl multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT steenburgscottd multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT banerjeeimon multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs AT trivedihari multireaderevaluationofradiologistperformanceforcovid19detectiononemergencydepartmentchestradiographs |