Cargando…
Reliability of radiologists’ first impression when interpreting a screening mammogram
Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about...
Autores principales: | , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10128970/ https://www.ncbi.nlm.nih.gov/pubmed/37098013 http://dx.doi.org/10.1371/journal.pone.0284605 |
_version_ | 1785030628504240128 |
---|---|
author | Gandomkar, Ziba Siviengphanom, Somphone Suleiman, Mo’ayyad Wong, Dennis Reed, Warren Ekpo, Ernest U. Xu, Dong Lewis, Sarah J. Evans, Karla K. Wolfe, Jeremy M. Brennan, Patrick C. |
author_facet | Gandomkar, Ziba Siviengphanom, Somphone Suleiman, Mo’ayyad Wong, Dennis Reed, Warren Ekpo, Ernest U. Xu, Dong Lewis, Sarah J. Evans, Karla K. Wolfe, Jeremy M. Brennan, Patrick C. |
author_sort | Gandomkar, Ziba |
collection | PubMed |
description | Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen’s Kappa was 0.478 (interquartile range = 0.419–0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen’s Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26–0.37). The Fleiss Kappa score of 0.106 (CI = 0.105–0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists’ initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search. |
format | Online Article Text |
id | pubmed-10128970 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-101289702023-04-26 Reliability of radiologists’ first impression when interpreting a screening mammogram Gandomkar, Ziba Siviengphanom, Somphone Suleiman, Mo’ayyad Wong, Dennis Reed, Warren Ekpo, Ernest U. Xu, Dong Lewis, Sarah J. Evans, Karla K. Wolfe, Jeremy M. Brennan, Patrick C. PLoS One Research Article Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists’ initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen’s Kappa was 0.478 (interquartile range = 0.419–0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen’s Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26–0.37). The Fleiss Kappa score of 0.106 (CI = 0.105–0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists’ initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search. Public Library of Science 2023-04-25 /pmc/articles/PMC10128970/ /pubmed/37098013 http://dx.doi.org/10.1371/journal.pone.0284605 Text en © 2023 Gandomkar et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Gandomkar, Ziba Siviengphanom, Somphone Suleiman, Mo’ayyad Wong, Dennis Reed, Warren Ekpo, Ernest U. Xu, Dong Lewis, Sarah J. Evans, Karla K. Wolfe, Jeremy M. Brennan, Patrick C. Reliability of radiologists’ first impression when interpreting a screening mammogram |
title | Reliability of radiologists’ first impression when interpreting a screening mammogram |
title_full | Reliability of radiologists’ first impression when interpreting a screening mammogram |
title_fullStr | Reliability of radiologists’ first impression when interpreting a screening mammogram |
title_full_unstemmed | Reliability of radiologists’ first impression when interpreting a screening mammogram |
title_short | Reliability of radiologists’ first impression when interpreting a screening mammogram |
title_sort | reliability of radiologists’ first impression when interpreting a screening mammogram |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10128970/ https://www.ncbi.nlm.nih.gov/pubmed/37098013 http://dx.doi.org/10.1371/journal.pone.0284605 |
work_keys_str_mv | AT gandomkarziba reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT siviengphanomsomphone reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT suleimanmoayyad reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT wongdennis reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT reedwarren reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT ekpoernestu reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT xudong reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT lewissarahj reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT evanskarlak reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT wolfejeremym reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram AT brennanpatrickc reliabilityofradiologistsfirstimpressionwheninterpretingascreeningmammogram |