Cargando…

A high-performance spatial database based approach for pathology imaging algorithm evaluation

BACKGROUND: Algorithm evaluation provides a means to characterize variability across image analysis algorithms, validate algorithms by comparison with human annotations, combine results from multiple algorithms for performance improvement, and facilitate algorithm sensitivity studies. The sizes of i...

Descripción completa

Detalles Bibliográficos
Autores principales: Wang, Fusheng, Kong, Jun, Gao, Jingjing, Cooper, Lee A.D., Kurc, Tahsin, Zhou, Zhengwen, Adler, David, Vergara-Niedermayr, Cristobal, Katigbak, Bryan, Brat, Daniel J., Saltz, Joel H.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Medknow Publications & Media Pvt Ltd 2013
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3624706/
https://www.ncbi.nlm.nih.gov/pubmed/23599905
http://dx.doi.org/10.4103/2153-3539.108543
_version_ 1782266045841539072
author Wang, Fusheng
Kong, Jun
Gao, Jingjing
Cooper, Lee A.D.
Kurc, Tahsin
Zhou, Zhengwen
Adler, David
Vergara-Niedermayr, Cristobal
Katigbak, Bryan
Brat, Daniel J.
Saltz, Joel H.
author_facet Wang, Fusheng
Kong, Jun
Gao, Jingjing
Cooper, Lee A.D.
Kurc, Tahsin
Zhou, Zhengwen
Adler, David
Vergara-Niedermayr, Cristobal
Katigbak, Bryan
Brat, Daniel J.
Saltz, Joel H.
author_sort Wang, Fusheng
collection PubMed
description BACKGROUND: Algorithm evaluation provides a means to characterize variability across image analysis algorithms, validate algorithms by comparison with human annotations, combine results from multiple algorithms for performance improvement, and facilitate algorithm sensitivity studies. The sizes of images and image analysis results in pathology image analysis pose significant challenges in algorithm evaluation. We present an efficient parallel spatial database approach to model, normalize, manage, and query large volumes of analytical image result data. This provides an efficient platform for algorithm evaluation. Our experiments with a set of brain tumor images demonstrate the application, scalability, and effectiveness of the platform. CONTEXT: The paper describes an approach and platform for evaluation of pathology image analysis algorithms. The platform facilitates algorithm evaluation through a high-performance database built on the Pathology Analytic Imaging Standards (PAIS) data model. AIMS: (1) Develop a framework to support algorithm evaluation by modeling and managing analytical results and human annotations from pathology images; (2) Create a robust data normalization tool for converting, validating, and fixing spatial data from algorithm or human annotations; (3) Develop a set of queries to support data sampling and result comparisons; (4) Achieve high performance computation capacity via a parallel data management infrastructure, parallel data loading and spatial indexing optimizations in this infrastructure. MATERIALS AND METHODS: We have considered two scenarios for algorithm evaluation: (1) algorithm comparison where multiple result sets from different methods are compared and consolidated; and (2) algorithm validation where algorithm results are compared with human annotations. We have developed a spatial normalization toolkit to validate and normalize spatial boundaries produced by image analysis algorithms or human annotations. The validated data were formatted based on the PAIS data model and loaded into a spatial database. To support efficient data loading, we have implemented a parallel data loading tool that takes advantage of multi-core CPUs to accelerate data injection. The spatial database manages both geometric shapes and image features or classifications, and enables spatial sampling, result comparison, and result aggregation through expressive structured query language (SQL) queries with spatial extensions. To provide scalable and efficient query support, we have employed a shared nothing parallel database architecture, which distributes data homogenously across multiple database partitions to take advantage of parallel computation power and implements spatial indexing to achieve high I/O throughput. RESULTS: Our work proposes a high performance, parallel spatial database platform for algorithm validation and comparison. This platform was evaluated by storing, managing, and comparing analysis results from a set of brain tumor whole slide images. The tools we develop are open source and available to download. CONCLUSIONS: Pathology image algorithm validation and comparison are essential to iterative algorithm development and refinement. One critical component is the support for queries involving spatial predicates and comparisons. In our work, we develop an efficient data model and parallel database approach to model, normalize, manage and query large volumes of analytical image result data. Our experiments demonstrate that the data partitioning strategy and the grid-based indexing result in good data distribution across database nodes and reduce I/O overhead in spatial join queries through parallel retrieval of relevant data and quick subsetting of datasets. The set of tools in the framework provide a full pipeline to normalize, load, manage and query analytical results for algorithm evaluation.
format Online
Article
Text
id pubmed-3624706
institution National Center for Biotechnology Information
language English
publishDate 2013
publisher Medknow Publications & Media Pvt Ltd
record_format MEDLINE/PubMed
spelling pubmed-36247062013-04-18 A high-performance spatial database based approach for pathology imaging algorithm evaluation Wang, Fusheng Kong, Jun Gao, Jingjing Cooper, Lee A.D. Kurc, Tahsin Zhou, Zhengwen Adler, David Vergara-Niedermayr, Cristobal Katigbak, Bryan Brat, Daniel J. Saltz, Joel H. J Pathol Inform Research Article BACKGROUND: Algorithm evaluation provides a means to characterize variability across image analysis algorithms, validate algorithms by comparison with human annotations, combine results from multiple algorithms for performance improvement, and facilitate algorithm sensitivity studies. The sizes of images and image analysis results in pathology image analysis pose significant challenges in algorithm evaluation. We present an efficient parallel spatial database approach to model, normalize, manage, and query large volumes of analytical image result data. This provides an efficient platform for algorithm evaluation. Our experiments with a set of brain tumor images demonstrate the application, scalability, and effectiveness of the platform. CONTEXT: The paper describes an approach and platform for evaluation of pathology image analysis algorithms. The platform facilitates algorithm evaluation through a high-performance database built on the Pathology Analytic Imaging Standards (PAIS) data model. AIMS: (1) Develop a framework to support algorithm evaluation by modeling and managing analytical results and human annotations from pathology images; (2) Create a robust data normalization tool for converting, validating, and fixing spatial data from algorithm or human annotations; (3) Develop a set of queries to support data sampling and result comparisons; (4) Achieve high performance computation capacity via a parallel data management infrastructure, parallel data loading and spatial indexing optimizations in this infrastructure. MATERIALS AND METHODS: We have considered two scenarios for algorithm evaluation: (1) algorithm comparison where multiple result sets from different methods are compared and consolidated; and (2) algorithm validation where algorithm results are compared with human annotations. We have developed a spatial normalization toolkit to validate and normalize spatial boundaries produced by image analysis algorithms or human annotations. The validated data were formatted based on the PAIS data model and loaded into a spatial database. To support efficient data loading, we have implemented a parallel data loading tool that takes advantage of multi-core CPUs to accelerate data injection. The spatial database manages both geometric shapes and image features or classifications, and enables spatial sampling, result comparison, and result aggregation through expressive structured query language (SQL) queries with spatial extensions. To provide scalable and efficient query support, we have employed a shared nothing parallel database architecture, which distributes data homogenously across multiple database partitions to take advantage of parallel computation power and implements spatial indexing to achieve high I/O throughput. RESULTS: Our work proposes a high performance, parallel spatial database platform for algorithm validation and comparison. This platform was evaluated by storing, managing, and comparing analysis results from a set of brain tumor whole slide images. The tools we develop are open source and available to download. CONCLUSIONS: Pathology image algorithm validation and comparison are essential to iterative algorithm development and refinement. One critical component is the support for queries involving spatial predicates and comparisons. In our work, we develop an efficient data model and parallel database approach to model, normalize, manage and query large volumes of analytical image result data. Our experiments demonstrate that the data partitioning strategy and the grid-based indexing result in good data distribution across database nodes and reduce I/O overhead in spatial join queries through parallel retrieval of relevant data and quick subsetting of datasets. The set of tools in the framework provide a full pipeline to normalize, load, manage and query analytical results for algorithm evaluation. Medknow Publications & Media Pvt Ltd 2013-03-14 /pmc/articles/PMC3624706/ /pubmed/23599905 http://dx.doi.org/10.4103/2153-3539.108543 Text en Copyright: © 2013 Wang F. http://creativecommons.org/licenses/by-nc-sa/3.0 This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Wang, Fusheng
Kong, Jun
Gao, Jingjing
Cooper, Lee A.D.
Kurc, Tahsin
Zhou, Zhengwen
Adler, David
Vergara-Niedermayr, Cristobal
Katigbak, Bryan
Brat, Daniel J.
Saltz, Joel H.
A high-performance spatial database based approach for pathology imaging algorithm evaluation
title A high-performance spatial database based approach for pathology imaging algorithm evaluation
title_full A high-performance spatial database based approach for pathology imaging algorithm evaluation
title_fullStr A high-performance spatial database based approach for pathology imaging algorithm evaluation
title_full_unstemmed A high-performance spatial database based approach for pathology imaging algorithm evaluation
title_short A high-performance spatial database based approach for pathology imaging algorithm evaluation
title_sort high-performance spatial database based approach for pathology imaging algorithm evaluation
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3624706/
https://www.ncbi.nlm.nih.gov/pubmed/23599905
http://dx.doi.org/10.4103/2153-3539.108543
work_keys_str_mv AT wangfusheng ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT kongjun ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT gaojingjing ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT cooperleead ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT kurctahsin ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT zhouzhengwen ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT adlerdavid ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT vergaraniedermayrcristobal ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT katigbakbryan ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT bratdanielj ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT saltzjoelh ahighperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT wangfusheng highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT kongjun highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT gaojingjing highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT cooperleead highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT kurctahsin highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT zhouzhengwen highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT adlerdavid highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT vergaraniedermayrcristobal highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT katigbakbryan highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT bratdanielj highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation
AT saltzjoelh highperformancespatialdatabasebasedapproachforpathologyimagingalgorithmevaluation