Cargando…
Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View
Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Metho...
Autores principales: | , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601089/ https://www.ncbi.nlm.nih.gov/pubmed/36292042 http://dx.doi.org/10.3390/diagnostics12102351 |
_version_ | 1784816970692034560 |
---|---|
author | VanBerlo, Bennett Smith, Delaney Tschirhart, Jared VanBerlo, Blake Wu, Derek Ford, Alex McCauley, Joseph Wu, Benjamin Chaudhary, Rushil Dave, Chintan Ho, Jordan Deglint, Jason Li, Brian Arntfield, Robert |
author_facet | VanBerlo, Bennett Smith, Delaney Tschirhart, Jared VanBerlo, Blake Wu, Derek Ford, Alex McCauley, Joseph Wu, Benjamin Chaudhary, Rushil Dave, Chintan Ho, Jordan Deglint, Jason Li, Brian Arntfield, Robert |
author_sort | VanBerlo, Bennett |
collection | PubMed |
description | Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Methods: We trained a machine learning model to accurately distinguish between one of two classes of lung ultrasound (LUS) views using 2908 clips from a larger dataset. Partitioning the remaining dataset by view would reduce downstream labelling efforts by enabling annotators to focus on annotating pathological features specific to each view. Results: In a sample view-specific annotation task, we found that automatically partitioning a 780-clip dataset by view saved 42 min of manual annotation time and resulted in [Formula: see text] additional relevant labels per hour. Conclusions: Automatic partitioning of a LUS dataset by view significantly increases annotator efficiency, resulting in higher throughput relevant to the annotating task at hand. The strategy described in this work can be applied to other hierarchical annotation schemes. |
format | Online Article Text |
id | pubmed-9601089 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-96010892022-10-27 Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View VanBerlo, Bennett Smith, Delaney Tschirhart, Jared VanBerlo, Blake Wu, Derek Ford, Alex McCauley, Joseph Wu, Benjamin Chaudhary, Rushil Dave, Chintan Ho, Jordan Deglint, Jason Li, Brian Arntfield, Robert Diagnostics (Basel) Article Background: Annotating large medical imaging datasets is an arduous and expensive task, especially when the datasets in question are not organized according to deep learning goals. Here, we propose a method that exploits the hierarchical organization of annotating tasks to optimize efficiency. Methods: We trained a machine learning model to accurately distinguish between one of two classes of lung ultrasound (LUS) views using 2908 clips from a larger dataset. Partitioning the remaining dataset by view would reduce downstream labelling efforts by enabling annotators to focus on annotating pathological features specific to each view. Results: In a sample view-specific annotation task, we found that automatically partitioning a 780-clip dataset by view saved 42 min of manual annotation time and resulted in [Formula: see text] additional relevant labels per hour. Conclusions: Automatic partitioning of a LUS dataset by view significantly increases annotator efficiency, resulting in higher throughput relevant to the annotating task at hand. The strategy described in this work can be applied to other hierarchical annotation schemes. MDPI 2022-09-28 /pmc/articles/PMC9601089/ /pubmed/36292042 http://dx.doi.org/10.3390/diagnostics12102351 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article VanBerlo, Bennett Smith, Delaney Tschirhart, Jared VanBerlo, Blake Wu, Derek Ford, Alex McCauley, Joseph Wu, Benjamin Chaudhary, Rushil Dave, Chintan Ho, Jordan Deglint, Jason Li, Brian Arntfield, Robert Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title | Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title_full | Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title_fullStr | Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title_full_unstemmed | Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title_short | Enhancing Annotation Efficiency with Machine Learning: Automated Partitioning of a Lung Ultrasound Dataset by View |
title_sort | enhancing annotation efficiency with machine learning: automated partitioning of a lung ultrasound dataset by view |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601089/ https://www.ncbi.nlm.nih.gov/pubmed/36292042 http://dx.doi.org/10.3390/diagnostics12102351 |
work_keys_str_mv | AT vanberlobennett enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT smithdelaney enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT tschirhartjared enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT vanberloblake enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT wuderek enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT fordalex enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT mccauleyjoseph enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT wubenjamin enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT chaudharyrushil enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT davechintan enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT hojordan enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT deglintjason enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT librian enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview AT arntfieldrobert enhancingannotationefficiencywithmachinelearningautomatedpartitioningofalungultrasounddatasetbyview |