Cargando…
A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditio...
Autores principales: | , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10372018/ https://www.ncbi.nlm.nih.gov/pubmed/37495660 http://dx.doi.org/10.1038/s41598-023-39278-0 |
_version_ | 1785078276005298176 |
---|---|
author | Müller-Franzes, Gustav Niehues, Jan Moritz Khader, Firas Arasteh, Soroosh Tayebi Haarburger, Christoph Kuhl, Christiane Wang, Tianci Han, Tianyu Nolte, Teresa Nebelung, Sven Kather, Jakob Nikolas Truhn, Daniel |
author_facet | Müller-Franzes, Gustav Niehues, Jan Moritz Khader, Firas Arasteh, Soroosh Tayebi Haarburger, Christoph Kuhl, Christiane Wang, Tianci Han, Tianyu Nolte, Teresa Nebelung, Sven Kather, Jakob Nikolas Truhn, Daniel |
author_sort | Müller-Franzes, Gustav |
collection | PubMed |
description | Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditional latent DDPM designed for medical image generation, and evaluate its performance against GANs, which currently represent the state-of-the-art. Medfusion was trained and compared with StyleGAN-3 using fundoscopy images from the AIROGS dataset, radiographs from the CheXpert dataset, and histopathology images from the CRCDX dataset. Based on previous studies, Progressively Growing GAN (ProGAN) and Conditional GAN (cGAN) were used as additional baselines on the CheXpert and CRCDX datasets, respectively. Medfusion exceeded GANs in terms of diversity (recall), achieving better scores of 0.40 compared to 0.19 in the AIROGS dataset, 0.41 compared to 0.02 (cGAN) and 0.24 (StyleGAN-3) in the CRMDX dataset, and 0.32 compared to 0.17 (ProGAN) and 0.08 (StyleGAN-3) in the CheXpert dataset. Furthermore, Medfusion exhibited equal or higher fidelity (precision) across all three datasets. Our study shows that Medfusion constitutes a promising alternative to GAN-based models for generating high-quality medical images, leading to improved diversity and less artifacts in the generated images. |
format | Online Article Text |
id | pubmed-10372018 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-103720182023-07-28 A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis Müller-Franzes, Gustav Niehues, Jan Moritz Khader, Firas Arasteh, Soroosh Tayebi Haarburger, Christoph Kuhl, Christiane Wang, Tianci Han, Tianyu Nolte, Teresa Nebelung, Sven Kather, Jakob Nikolas Truhn, Daniel Sci Rep Article Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditional latent DDPM designed for medical image generation, and evaluate its performance against GANs, which currently represent the state-of-the-art. Medfusion was trained and compared with StyleGAN-3 using fundoscopy images from the AIROGS dataset, radiographs from the CheXpert dataset, and histopathology images from the CRCDX dataset. Based on previous studies, Progressively Growing GAN (ProGAN) and Conditional GAN (cGAN) were used as additional baselines on the CheXpert and CRCDX datasets, respectively. Medfusion exceeded GANs in terms of diversity (recall), achieving better scores of 0.40 compared to 0.19 in the AIROGS dataset, 0.41 compared to 0.02 (cGAN) and 0.24 (StyleGAN-3) in the CRMDX dataset, and 0.32 compared to 0.17 (ProGAN) and 0.08 (StyleGAN-3) in the CheXpert dataset. Furthermore, Medfusion exhibited equal or higher fidelity (precision) across all three datasets. Our study shows that Medfusion constitutes a promising alternative to GAN-based models for generating high-quality medical images, leading to improved diversity and less artifacts in the generated images. Nature Publishing Group UK 2023-07-26 /pmc/articles/PMC10372018/ /pubmed/37495660 http://dx.doi.org/10.1038/s41598-023-39278-0 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Müller-Franzes, Gustav Niehues, Jan Moritz Khader, Firas Arasteh, Soroosh Tayebi Haarburger, Christoph Kuhl, Christiane Wang, Tianci Han, Tianyu Nolte, Teresa Nebelung, Sven Kather, Jakob Nikolas Truhn, Daniel A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title | A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title_full | A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title_fullStr | A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title_full_unstemmed | A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title_short | A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
title_sort | multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10372018/ https://www.ncbi.nlm.nih.gov/pubmed/37495660 http://dx.doi.org/10.1038/s41598-023-39278-0 |
work_keys_str_mv | AT mullerfranzesgustav amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT niehuesjanmoritz amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT khaderfiras amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT arastehsorooshtayebi amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT haarburgerchristoph amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT kuhlchristiane amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT wangtianci amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT hantianyu amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT nolteteresa amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT nebelungsven amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT katherjakobnikolas amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT truhndaniel amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT mullerfranzesgustav multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT niehuesjanmoritz multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT khaderfiras multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT arastehsorooshtayebi multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT haarburgerchristoph multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT kuhlchristiane multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT wangtianci multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT hantianyu multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT nolteteresa multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT nebelungsven multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT katherjakobnikolas multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis AT truhndaniel multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis |