Cargando…
Normalization of HE-stained histological images using cycle consistent generative adversarial networks
BACKGROUND: Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these vari...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8349020/ https://www.ncbi.nlm.nih.gov/pubmed/34362386 http://dx.doi.org/10.1186/s13000-021-01126-y |
_version_ | 1783735482976305152 |
---|---|
author | Runz, Marlen Rusche, Daniel Schmidt, Stefan Weihrauch, Martin R. Hesser, Jürgen Weis, Cleo-Aron |
author_facet | Runz, Marlen Rusche, Daniel Schmidt, Stefan Weihrauch, Martin R. Hesser, Jürgen Weis, Cleo-Aron |
author_sort | Runz, Marlen |
collection | PubMed |
description | BACKGROUND: Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques. METHODS: In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network G(B) that learns to map an image X from a source domain A to a target domain B, i.e. G(B):X(A)→X(B). In addition, a discriminator network D(B) is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair (G(A),D(A)), for the inverse mapping G(A):X(B)→X(A). Cycle consistency ensures that a generated image is close to its original when being mapped backwards (G(A)(G(B)(X(A)))≈X(A) and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set. RESULTS: Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%. CONCLUSIONS: CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The data set supporting the solutions is available at 10.11588/data/8LKEZF. |
format | Online Article Text |
id | pubmed-8349020 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-83490202021-08-09 Normalization of HE-stained histological images using cycle consistent generative adversarial networks Runz, Marlen Rusche, Daniel Schmidt, Stefan Weihrauch, Martin R. Hesser, Jürgen Weis, Cleo-Aron Diagn Pathol Research BACKGROUND: Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques. METHODS: In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network G(B) that learns to map an image X from a source domain A to a target domain B, i.e. G(B):X(A)→X(B). In addition, a discriminator network D(B) is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair (G(A),D(A)), for the inverse mapping G(A):X(B)→X(A). Cycle consistency ensures that a generated image is close to its original when being mapped backwards (G(A)(G(B)(X(A)))≈X(A) and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set. RESULTS: Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%. CONCLUSIONS: CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The data set supporting the solutions is available at 10.11588/data/8LKEZF. BioMed Central 2021-08-06 /pmc/articles/PMC8349020/ /pubmed/34362386 http://dx.doi.org/10.1186/s13000-021-01126-y Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data. |
spellingShingle | Research Runz, Marlen Rusche, Daniel Schmidt, Stefan Weihrauch, Martin R. Hesser, Jürgen Weis, Cleo-Aron Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title | Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title_full | Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title_fullStr | Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title_full_unstemmed | Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title_short | Normalization of HE-stained histological images using cycle consistent generative adversarial networks |
title_sort | normalization of he-stained histological images using cycle consistent generative adversarial networks |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8349020/ https://www.ncbi.nlm.nih.gov/pubmed/34362386 http://dx.doi.org/10.1186/s13000-021-01126-y |
work_keys_str_mv | AT runzmarlen normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks AT ruschedaniel normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks AT schmidtstefan normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks AT weihrauchmartinr normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks AT hesserjurgen normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks AT weiscleoaron normalizationofhestainedhistologicalimagesusingcycleconsistentgenerativeadversarialnetworks |