Cargando…

Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation

Convolutional neural networks (CNNs) have been successfully applied to chest x-ray (CXR) images. Moreover, annotated bounding boxes have been shown to improve the interpretability of a CNN in terms of localizing abnormalities. However, only a few relatively small CXR datasets containing bounding box...

Descripción completa

Detalles Bibliográficos
Autores principales: Bigolin Lanfredi, Ricardo, Schroeder, Joyce D., Tasdizen, Tolga
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10365091/
https://www.ncbi.nlm.nih.gov/pubmed/37492389
http://dx.doi.org/10.3389/fradi.2023.1088068
_version_ 1785076979254427648
author Bigolin Lanfredi, Ricardo
Schroeder, Joyce D.
Tasdizen, Tolga
author_facet Bigolin Lanfredi, Ricardo
Schroeder, Joyce D.
Tasdizen, Tolga
author_sort Bigolin Lanfredi, Ricardo
collection PubMed
description Convolutional neural networks (CNNs) have been successfully applied to chest x-ray (CXR) images. Moreover, annotated bounding boxes have been shown to improve the interpretability of a CNN in terms of localizing abnormalities. However, only a few relatively small CXR datasets containing bounding boxes are available, and collecting them is very costly. Opportunely, eye-tracking (ET) data can be collected during the clinical workflow of a radiologist. We use ET data recorded from radiologists while dictating CXR reports to train CNNs. We extract snippets from the ET data by associating them with the dictation of keywords and use them to supervise the localization of specific abnormalities. We show that this method can improve a model’s interpretability without impacting its image-level classification.
format Online
Article
Text
id pubmed-10365091
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-103650912023-07-25 Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation Bigolin Lanfredi, Ricardo Schroeder, Joyce D. Tasdizen, Tolga Front Radiol Radiology Convolutional neural networks (CNNs) have been successfully applied to chest x-ray (CXR) images. Moreover, annotated bounding boxes have been shown to improve the interpretability of a CNN in terms of localizing abnormalities. However, only a few relatively small CXR datasets containing bounding boxes are available, and collecting them is very costly. Opportunely, eye-tracking (ET) data can be collected during the clinical workflow of a radiologist. We use ET data recorded from radiologists while dictating CXR reports to train CNNs. We extract snippets from the ET data by associating them with the dictation of keywords and use them to supervise the localization of specific abnormalities. We show that this method can improve a model’s interpretability without impacting its image-level classification. Frontiers Media S.A. 2023-06-22 /pmc/articles/PMC10365091/ /pubmed/37492389 http://dx.doi.org/10.3389/fradi.2023.1088068 Text en © 2023 Bigolin Lanfredi, Schroeder and Tasdizen. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) (https://creativecommons.org/licenses/by/4.0/) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Radiology
Bigolin Lanfredi, Ricardo
Schroeder, Joyce D.
Tasdizen, Tolga
Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title_full Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title_fullStr Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title_full_unstemmed Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title_short Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
title_sort localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation
topic Radiology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10365091/
https://www.ncbi.nlm.nih.gov/pubmed/37492389
http://dx.doi.org/10.3389/fradi.2023.1088068
work_keys_str_mv AT bigolinlanfrediricardo localizationsupervisionofchestxrayclassifiersusinglabelspecificeyetrackingannotation
AT schroederjoyced localizationsupervisionofchestxrayclassifiersusinglabelspecificeyetrackingannotation
AT tasdizentolga localizationsupervisionofchestxrayclassifiersusinglabelspecificeyetrackingannotation