Cargando…
Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utili...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Elsevier
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9577134/ https://www.ncbi.nlm.nih.gov/pubmed/36268062 http://dx.doi.org/10.1016/j.jpi.2022.100148 |
_version_ | 1784811690062249984 |
---|---|
author | Manuel, Cyrus Zehnder, Philip Kaya, Sertan Sullivan, Ruth Hu, Fangyao |
author_facet | Manuel, Cyrus Zehnder, Philip Kaya, Sertan Sullivan, Ruth Hu, Fangyao |
author_sort | Manuel, Cyrus |
collection | PubMed |
description | Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utilization of digitized whole slide images (WSI) in pathology workflows, digital pathology has emerged as a promising domain for super-resolution. Despite extensive existing research into super-resolution, there remain challenges specific to digital pathology. Here, we investigated image augmentation techniques for hematoxylin and eosin (H&E) WSI super-resolution and model generalizability across diverse tissue types. In addition, we investigated shortcomings with common quality metrics (peak signal-to-noise ratio (PSNR), structure similarity index (SSIM)) by conducting a perceptual quality survey for super-resolved pathology images. High performing deep super-resolution models were used to generate 20X HR images from LR images (5X or 10X equivalent) for 11 different tissues and 30 human evaluators were asked to score the quality of the generated versus the ground truth 20X HR images. The scores given by a human rater and the PSNR or the SSIM were compared to investigate the correlation between model training parameters. We found that models trained on multiple tissues generalized better than those trained on a single tissue type. We also found that PSNR correlated with perceptual quality (R = 0.26) less accurately than did SSIM (R = 0.64), suggesting that the SSIM quality metric is insufficient. The methods proposed in this study can be used to virtually magnify H&E images with better perceptual quality than interpolation methods (i.e., bicubic interpolation) commonly implemented in digital pathology software. The impact of deep SISR methods is more notable when scaling to 4X is needed, such as in the case of super-resolving a low magnification WSI from 10X to 40X. |
format | Online Article Text |
id | pubmed-9577134 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Elsevier |
record_format | MEDLINE/PubMed |
spelling | pubmed-95771342022-10-19 Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution Manuel, Cyrus Zehnder, Philip Kaya, Sertan Sullivan, Ruth Hu, Fangyao J Pathol Inform Original Research Article Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utilization of digitized whole slide images (WSI) in pathology workflows, digital pathology has emerged as a promising domain for super-resolution. Despite extensive existing research into super-resolution, there remain challenges specific to digital pathology. Here, we investigated image augmentation techniques for hematoxylin and eosin (H&E) WSI super-resolution and model generalizability across diverse tissue types. In addition, we investigated shortcomings with common quality metrics (peak signal-to-noise ratio (PSNR), structure similarity index (SSIM)) by conducting a perceptual quality survey for super-resolved pathology images. High performing deep super-resolution models were used to generate 20X HR images from LR images (5X or 10X equivalent) for 11 different tissues and 30 human evaluators were asked to score the quality of the generated versus the ground truth 20X HR images. The scores given by a human rater and the PSNR or the SSIM were compared to investigate the correlation between model training parameters. We found that models trained on multiple tissues generalized better than those trained on a single tissue type. We also found that PSNR correlated with perceptual quality (R = 0.26) less accurately than did SSIM (R = 0.64), suggesting that the SSIM quality metric is insufficient. The methods proposed in this study can be used to virtually magnify H&E images with better perceptual quality than interpolation methods (i.e., bicubic interpolation) commonly implemented in digital pathology software. The impact of deep SISR methods is more notable when scaling to 4X is needed, such as in the case of super-resolving a low magnification WSI from 10X to 40X. Elsevier 2022-10-01 /pmc/articles/PMC9577134/ /pubmed/36268062 http://dx.doi.org/10.1016/j.jpi.2022.100148 Text en © 2022 The Authors https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). |
spellingShingle | Original Research Article Manuel, Cyrus Zehnder, Philip Kaya, Sertan Sullivan, Ruth Hu, Fangyao Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title | Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title_full | Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title_fullStr | Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title_full_unstemmed | Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title_short | Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
title_sort | impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution |
topic | Original Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9577134/ https://www.ncbi.nlm.nih.gov/pubmed/36268062 http://dx.doi.org/10.1016/j.jpi.2022.100148 |
work_keys_str_mv | AT manuelcyrus impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution AT zehnderphilip impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution AT kayasertan impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution AT sullivanruth impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution AT hufangyao impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution |