Cargando…
Task model-specific operator skill assessment in routine fetal ultrasound scanning
PURPOSE: For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical tas...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer International Publishing
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9307537/ https://www.ncbi.nlm.nih.gov/pubmed/35556206 http://dx.doi.org/10.1007/s11548-022-02642-y |
_version_ | 1784752784747266048 |
---|---|
author | Wang, Yipei Yang, Qianye Drukker, Lior Papageorghiou, Aris Hu, Yipeng Noble, J. Alison |
author_facet | Wang, Yipei Yang, Qianye Drukker, Lior Papageorghiou, Aris Hu, Yipeng Noble, J. Alison |
author_sort | Wang, Yipei |
collection | PubMed |
description | PURPOSE: For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill. METHODS: This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these task models are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification. RESULTS: We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience. CONCLUSION: Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11548-022-02642-y. |
format | Online Article Text |
id | pubmed-9307537 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Springer International Publishing |
record_format | MEDLINE/PubMed |
spelling | pubmed-93075372022-07-24 Task model-specific operator skill assessment in routine fetal ultrasound scanning Wang, Yipei Yang, Qianye Drukker, Lior Papageorghiou, Aris Hu, Yipeng Noble, J. Alison Int J Comput Assist Radiol Surg Original Article PURPOSE: For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill. METHODS: This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these task models are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification. RESULTS: We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience. CONCLUSION: Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11548-022-02642-y. Springer International Publishing 2022-05-12 2022 /pmc/articles/PMC9307537/ /pubmed/35556206 http://dx.doi.org/10.1007/s11548-022-02642-y Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Original Article Wang, Yipei Yang, Qianye Drukker, Lior Papageorghiou, Aris Hu, Yipeng Noble, J. Alison Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title | Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title_full | Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title_fullStr | Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title_full_unstemmed | Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title_short | Task model-specific operator skill assessment in routine fetal ultrasound scanning |
title_sort | task model-specific operator skill assessment in routine fetal ultrasound scanning |
topic | Original Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9307537/ https://www.ncbi.nlm.nih.gov/pubmed/35556206 http://dx.doi.org/10.1007/s11548-022-02642-y |
work_keys_str_mv | AT wangyipei taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning AT yangqianye taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning AT drukkerlior taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning AT papageorghiouaris taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning AT huyipeng taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning AT noblejalison taskmodelspecificoperatorskillassessmentinroutinefetalultrasoundscanning |