Cargando…

Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution

Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utili...

Descripción completa

Detalles Bibliográficos
Autores principales: Manuel, Cyrus, Zehnder, Philip, Kaya, Sertan, Sullivan, Ruth, Hu, Fangyao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Elsevier 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9577134/
https://www.ncbi.nlm.nih.gov/pubmed/36268062
http://dx.doi.org/10.1016/j.jpi.2022.100148
_version_ 1784811690062249984
author Manuel, Cyrus
Zehnder, Philip
Kaya, Sertan
Sullivan, Ruth
Hu, Fangyao
author_facet Manuel, Cyrus
Zehnder, Philip
Kaya, Sertan
Sullivan, Ruth
Hu, Fangyao
author_sort Manuel, Cyrus
collection PubMed
description Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utilization of digitized whole slide images (WSI) in pathology workflows, digital pathology has emerged as a promising domain for super-resolution. Despite extensive existing research into super-resolution, there remain challenges specific to digital pathology. Here, we investigated image augmentation techniques for hematoxylin and eosin (H&E) WSI super-resolution and model generalizability across diverse tissue types. In addition, we investigated shortcomings with common quality metrics (peak signal-to-noise ratio (PSNR), structure similarity index (SSIM)) by conducting a perceptual quality survey for super-resolved pathology images. High performing deep super-resolution models were used to generate 20X HR images from LR images (5X or 10X equivalent) for 11 different tissues and 30 human evaluators were asked to score the quality of the generated versus the ground truth 20X HR images. The scores given by a human rater and the PSNR or the SSIM were compared to investigate the correlation between model training parameters. We found that models trained on multiple tissues generalized better than those trained on a single tissue type. We also found that PSNR correlated with perceptual quality (R = 0.26) less accurately than did SSIM (R = 0.64), suggesting that the SSIM quality metric is insufficient. The methods proposed in this study can be used to virtually magnify H&E images with better perceptual quality than interpolation methods (i.e., bicubic interpolation) commonly implemented in digital pathology software. The impact of deep SISR methods is more notable when scaling to 4X is needed, such as in the case of super-resolving a low magnification WSI from 10X to 40X.
format Online
Article
Text
id pubmed-9577134
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Elsevier
record_format MEDLINE/PubMed
spelling pubmed-95771342022-10-19 Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution Manuel, Cyrus Zehnder, Philip Kaya, Sertan Sullivan, Ruth Hu, Fangyao J Pathol Inform Original Research Article Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utilization of digitized whole slide images (WSI) in pathology workflows, digital pathology has emerged as a promising domain for super-resolution. Despite extensive existing research into super-resolution, there remain challenges specific to digital pathology. Here, we investigated image augmentation techniques for hematoxylin and eosin (H&E) WSI super-resolution and model generalizability across diverse tissue types. In addition, we investigated shortcomings with common quality metrics (peak signal-to-noise ratio (PSNR), structure similarity index (SSIM)) by conducting a perceptual quality survey for super-resolved pathology images. High performing deep super-resolution models were used to generate 20X HR images from LR images (5X or 10X equivalent) for 11 different tissues and 30 human evaluators were asked to score the quality of the generated versus the ground truth 20X HR images. The scores given by a human rater and the PSNR or the SSIM were compared to investigate the correlation between model training parameters. We found that models trained on multiple tissues generalized better than those trained on a single tissue type. We also found that PSNR correlated with perceptual quality (R = 0.26) less accurately than did SSIM (R = 0.64), suggesting that the SSIM quality metric is insufficient. The methods proposed in this study can be used to virtually magnify H&E images with better perceptual quality than interpolation methods (i.e., bicubic interpolation) commonly implemented in digital pathology software. The impact of deep SISR methods is more notable when scaling to 4X is needed, such as in the case of super-resolving a low magnification WSI from 10X to 40X. Elsevier 2022-10-01 /pmc/articles/PMC9577134/ /pubmed/36268062 http://dx.doi.org/10.1016/j.jpi.2022.100148 Text en © 2022 The Authors https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
spellingShingle Original Research Article
Manuel, Cyrus
Zehnder, Philip
Kaya, Sertan
Sullivan, Ruth
Hu, Fangyao
Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title_full Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title_fullStr Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title_full_unstemmed Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title_short Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
title_sort impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution
topic Original Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9577134/
https://www.ncbi.nlm.nih.gov/pubmed/36268062
http://dx.doi.org/10.1016/j.jpi.2022.100148
work_keys_str_mv AT manuelcyrus impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution
AT zehnderphilip impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution
AT kayasertan impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution
AT sullivanruth impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution
AT hufangyao impactofcoloraugmentationandtissuetypeindeeplearningforhematoxylinandeosinimagesuperresolution