Cargando…

Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study

BACKGROUND: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further a...

Descripción completa

Detalles Bibliográficos
Autores principales: Robinson, Robert, Valindria, Vanya V., Bai, Wenjia, Oktay, Ozan, Kainz, Bernhard, Suzuki, Hideaki, Sanghvi, Mihir M., Aung, Nay, Paiva, José Miguel, Zemrak, Filip, Fung, Kenneth, Lukaschuk, Elena, Lee, Aaron M., Carapella, Valentina, Kim, Young Jin, Piechnik, Stefan K., Neubauer, Stefan, Petersen, Steffen E., Page, Chris, Matthews, Paul M., Rueckert, Daniel, Glocker, Ben
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6416857/
https://www.ncbi.nlm.nih.gov/pubmed/30866968
http://dx.doi.org/10.1186/s12968-019-0523-x
_version_ 1783403443988201472
author Robinson, Robert
Valindria, Vanya V.
Bai, Wenjia
Oktay, Ozan
Kainz, Bernhard
Suzuki, Hideaki
Sanghvi, Mihir M.
Aung, Nay
Paiva, José Miguel
Zemrak, Filip
Fung, Kenneth
Lukaschuk, Elena
Lee, Aaron M.
Carapella, Valentina
Kim, Young Jin
Piechnik, Stefan K.
Neubauer, Stefan
Petersen, Steffen E.
Page, Chris
Matthews, Paul M.
Rueckert, Daniel
Glocker, Ben
author_facet Robinson, Robert
Valindria, Vanya V.
Bai, Wenjia
Oktay, Ozan
Kainz, Bernhard
Suzuki, Hideaki
Sanghvi, Mihir M.
Aung, Nay
Paiva, José Miguel
Zemrak, Filip
Fung, Kenneth
Lukaschuk, Elena
Lee, Aaron M.
Carapella, Valentina
Kim, Young Jin
Piechnik, Stefan K.
Neubauer, Stefan
Petersen, Steffen E.
Page, Chris
Matthews, Paul M.
Rueckert, Daniel
Glocker, Ben
author_sort Robinson, Robert
collection PubMed
description BACKGROUND: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further analyses. Manual inspection and visual QC of each segmentation result is not feasible at large scale. However, it is important to be able to automatically detect when a segmentation method fails in order to avoid inclusion of wrong measurements into subsequent analyses which could otherwise lead to incorrect conclusions. METHODS: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4800 cardiovascular magnetic resonance (CMR) scans. We then apply our method to a large cohort of 7250 CMR on which we have performed manual QC. RESULTS: We report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using the predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4800 scans for which manual segmentations were available. We mimic real-world application of the method on 7250 CMR where we show good agreement between predicted quality metrics and manual visual QC scores. CONCLUSIONS: We show that Reverse classification accuracy has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study.
format Online
Article
Text
id pubmed-6416857
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-64168572019-03-25 Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study Robinson, Robert Valindria, Vanya V. Bai, Wenjia Oktay, Ozan Kainz, Bernhard Suzuki, Hideaki Sanghvi, Mihir M. Aung, Nay Paiva, José Miguel Zemrak, Filip Fung, Kenneth Lukaschuk, Elena Lee, Aaron M. Carapella, Valentina Kim, Young Jin Piechnik, Stefan K. Neubauer, Stefan Petersen, Steffen E. Page, Chris Matthews, Paul M. Rueckert, Daniel Glocker, Ben J Cardiovasc Magn Reson Research BACKGROUND: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further analyses. Manual inspection and visual QC of each segmentation result is not feasible at large scale. However, it is important to be able to automatically detect when a segmentation method fails in order to avoid inclusion of wrong measurements into subsequent analyses which could otherwise lead to incorrect conclusions. METHODS: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4800 cardiovascular magnetic resonance (CMR) scans. We then apply our method to a large cohort of 7250 CMR on which we have performed manual QC. RESULTS: We report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using the predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4800 scans for which manual segmentations were available. We mimic real-world application of the method on 7250 CMR where we show good agreement between predicted quality metrics and manual visual QC scores. CONCLUSIONS: We show that Reverse classification accuracy has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study. BioMed Central 2019-03-14 /pmc/articles/PMC6416857/ /pubmed/30866968 http://dx.doi.org/10.1186/s12968-019-0523-x Text en © The Author(s) 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research
Robinson, Robert
Valindria, Vanya V.
Bai, Wenjia
Oktay, Ozan
Kainz, Bernhard
Suzuki, Hideaki
Sanghvi, Mihir M.
Aung, Nay
Paiva, José Miguel
Zemrak, Filip
Fung, Kenneth
Lukaschuk, Elena
Lee, Aaron M.
Carapella, Valentina
Kim, Young Jin
Piechnik, Stefan K.
Neubauer, Stefan
Petersen, Steffen E.
Page, Chris
Matthews, Paul M.
Rueckert, Daniel
Glocker, Ben
Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title_full Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title_fullStr Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title_full_unstemmed Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title_short Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study
title_sort automated quality control in image segmentation: application to the uk biobank cardiovascular magnetic resonance imaging study
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6416857/
https://www.ncbi.nlm.nih.gov/pubmed/30866968
http://dx.doi.org/10.1186/s12968-019-0523-x
work_keys_str_mv AT robinsonrobert automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT valindriavanyav automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT baiwenjia automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT oktayozan automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT kainzbernhard automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT suzukihideaki automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT sanghvimihirm automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT aungnay automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT paivajosemiguel automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT zemrakfilip automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT fungkenneth automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT lukaschukelena automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT leeaaronm automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT carapellavalentina automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT kimyoungjin automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT piechnikstefank automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT neubauerstefan automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT petersensteffene automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT pagechris automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT matthewspaulm automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT rueckertdaniel automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy
AT glockerben automatedqualitycontrolinimagesegmentationapplicationtotheukbiobankcardiovascularmagneticresonanceimagingstudy