Cargando…

Reliability of radiologists’ first impression when interpreting a screening mammogram

Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about...

Descripción completa

Detalles Bibliográficos
Autores principales: Gandomkar, Ziba, Siviengphanom, Somphone, Suleiman, Mo’ayyad, Wong, Dennis, Reed, Warren, Ekpo, Ernest U., Xu, Dong, Lewis, Sarah J., Evans, Karla K., Wolfe, Jeremy M., Brennan, Patrick C.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10128970/
https://www.ncbi.nlm.nih.gov/pubmed/37098013
http://dx.doi.org/10.1371/journal.pone.0284605
_version_ 1785030628504240128
author Gandomkar, Ziba
Siviengphanom, Somphone
Suleiman, Mo’ayyad
Wong, Dennis
Reed, Warren
Ekpo, Ernest U.
Xu, Dong
Lewis, Sarah J.
Evans, Karla K.
Wolfe, Jeremy M.
Brennan, Patrick C.
author_facet Gandomkar, Ziba
Siviengphanom, Somphone
Suleiman, Mo’ayyad
Wong, Dennis
Reed, Warren
Ekpo, Ernest U.
Xu, Dong
Lewis, Sarah J.
Evans, Karla K.
Wolfe, Jeremy M.
Brennan, Patrick C.
author_sort Gandomkar, Ziba
collection PubMed
description Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen’s Kappa was 0.478 (interquartile range = 0.419–0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen’s Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26–0.37). The Fleiss Kappa score of 0.106 (CI = 0.105–0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists’ initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search.
format Online
Article
Text
id pubmed-10128970
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-101289702023-04-26 Reliability of radiologists’ first impression when interpreting a screening mammogram Gandomkar, Ziba Siviengphanom, Somphone Suleiman, Mo’ayyad Wong, Dennis Reed, Warren Ekpo, Ernest U. Xu, Dong Lewis, Sarah J. Evans, Karla K. Wolfe, Jeremy M. Brennan, Patrick C. PLoS One Research Article Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen’s Kappa was 0.478 (interquartile range = 0.419–0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen’s Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26–0.37). The Fleiss Kappa score of 0.106 (CI = 0.105–0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists’ initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search. Public Library of Science 2023-04-25 /pmc/articles/PMC10128970/ /pubmed/37098013 http://dx.doi.org/10.1371/journal.pone.0284605 Text en © 2023 Gandomkar et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Gandomkar, Ziba
Siviengphanom, Somphone
Suleiman, Mo’ayyad
Wong, Dennis
Reed, Warren
Ekpo, Ernest U.
Xu, Dong
Lewis, Sarah J.
Evans, Karla K.
Wolfe, Jeremy M.
Brennan, Patrick C.
Reliability of radiologists’ first impression when interpreting a screening mammogram
title Reliability of radiologists’ first impression when interpreting a screening mammogram
title_full Reliability of radiologists’ first impression when interpreting a screening mammogram
title_fullStr Reliability of radiologists’ first impression when interpreting a screening mammogram
title_full_unstemmed Reliability of radiologists’ first impression when interpreting a screening mammogram
title_short Reliability of radiologists’ first impression when interpreting a screening mammogram
title_sort reliability of radiologists’ first impression when interpreting a screening mammogram
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10128970/
https://www.ncbi.nlm.nih.gov/pubmed/37098013
http://dx.doi.org/10.1371/journal.pone.0284605
work_keys_str_mv AT gandomkarziba reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT siviengphanomsomphone reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT suleimanmoayyad reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT wongdennis reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT reedwarren reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT ekpoernestu reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT xudong reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT lewissarahj reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT evanskarlak reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT wolfejeremym reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram
AT brennanpatrickc reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram