Cargando…

Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View

Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Metho...

Descripción completa

Detalles Bibliográficos
Autores principales: VanBerlo, Bennett, Smith, Delaney, Tschirhart, Jared, VanBerlo, Blake, Wu, Derek, Ford, Alex, McCauley, Joseph, Wu, Benjamin, Chaudhary, Rushil, Dave, Chintan, Ho, Jordan, Deglint, Jason, Li, Brian, Arntfield, Robert
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601089/
https://www.ncbi.nlm.nih.gov/pubmed/36292042
http://dx.doi.org/10.3390/diagnostics12102351
_version_ 1784816970692034560
author VanBerlo, Bennett
Smith, Delaney
Tschirhart, Jared
VanBerlo, Blake
Wu, Derek
Ford, Alex
McCauley, Joseph
Wu, Benjamin
Chaudhary, Rushil
Dave, Chintan
Ho, Jordan
Deglint, Jason
Li, Brian
Arntfield, Robert
author_facet VanBerlo, Bennett
Smith, Delaney
Tschirhart, Jared
VanBerlo, Blake
Wu, Derek
Ford, Alex
McCauley, Joseph
Wu, Benjamin
Chaudhary, Rushil
Dave, Chintan
Ho, Jordan
Deglint, Jason
Li, Brian
Arntfield, Robert
author_sort VanBerlo, Bennett
collection PubMed
description Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Methods: We trained a machine learning model to accurately distinguish between one of two classes of lung ultrasound (LUS) views using 2908 clips from a larger dataset. Partitioning the remaining dataset by view would reduce downstream labelling efforts by enabling annotators to focus on annotating pathological features specific to each view. Results: In a sample view-specific annotation task, we found that automatically partitioning a 780-clip dataset by view saved 42 min of manual annotation time and resulted in [Formula: see text] additional relevant labels per hour. Conclusions: Automatic partitioning of a LUS dataset by view significantly increases annotator efficiency, resulting in higher throughput relevant to the annotating task at hand. The strategy described in this work can be applied to other hierarchical annotation schemes.
format Online
Article
Text
id pubmed-9601089
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96010892022-10-27 Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View VanBerlo, Bennett Smith, Delaney Tschirhart, Jared VanBerlo, Blake Wu, Derek Ford, Alex McCauley, Joseph Wu, Benjamin Chaudhary, Rushil Dave, Chintan Ho, Jordan Deglint, Jason Li, Brian Arntfield, Robert Diagnostics (Basel) Article Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Methods: We trained a machine learning model to accurately distinguish between one of two classes of lung ultrasound (LUS) views using 2908 clips from a larger dataset. Partitioning the remaining dataset by view would reduce downstream labelling efforts by enabling annotators to focus on annotating pathological features specific to each view. Results: In a sample view-specific annotation task, we found that automatically partitioning a 780-clip dataset by view saved 42 min of manual annotation time and resulted in [Formula: see text] additional relevant labels per hour. Conclusions: Automatic partitioning of a LUS dataset by view significantly increases annotator efficiency, resulting in higher throughput relevant to the annotating task at hand. The strategy described in this work can be applied to other hierarchical annotation schemes. MDPI 2022-09-28 /pmc/articles/PMC9601089/ /pubmed/36292042 http://dx.doi.org/10.3390/diagnostics12102351 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
VanBerlo, Bennett
Smith, Delaney
Tschirhart, Jared
VanBerlo, Blake
Wu, Derek
Ford, Alex
McCauley, Joseph
Wu, Benjamin
Chaudhary, Rushil
Dave, Chintan
Ho, Jordan
Deglint, Jason
Li, Brian
Arntfield, Robert
Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title_full Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title_fullStr Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title_full_unstemmed Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title_short Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
title_sort enhancing annotation efficiency with machine learning: automated partitioning of a lung ultrasound dataset by view
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601089/
https://www.ncbi.nlm.nih.gov/pubmed/36292042
http://dx.doi.org/10.3390/diagnostics12102351
work_keys_str_mv AT vanberlobennett enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT smithdelaney enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT tschirhartjared enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT vanberloblake enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT wuderek enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT fordalex enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT mccauleyjoseph enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT wubenjamin enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT chaudharyrushil enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT davechintan enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT hojordan enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT deglintjason enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT librian enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview
AT arntfieldrobert enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview