Cargando…

A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis

Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditio...

Descripción completa

Detalles Bibliográficos
Autores principales: Müller-Franzes, Gustav, Niehues, Jan Moritz, Khader, Firas, Arasteh, Soroosh Tayebi, Haarburger, Christoph, Kuhl, Christiane, Wang, Tianci, Han, Tianyu, Nolte, Teresa, Nebelung, Sven, Kather, Jakob Nikolas, Truhn, Daniel
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10372018/
https://www.ncbi.nlm.nih.gov/pubmed/37495660
http://dx.doi.org/10.1038/s41598-023-39278-0
_version_ 1785078276005298176
author Müller-Franzes, Gustav
Niehues, Jan Moritz
Khader, Firas
Arasteh, Soroosh Tayebi
Haarburger, Christoph
Kuhl, Christiane
Wang, Tianci
Han, Tianyu
Nolte, Teresa
Nebelung, Sven
Kather, Jakob Nikolas
Truhn, Daniel
author_facet Müller-Franzes, Gustav
Niehues, Jan Moritz
Khader, Firas
Arasteh, Soroosh Tayebi
Haarburger, Christoph
Kuhl, Christiane
Wang, Tianci
Han, Tianyu
Nolte, Teresa
Nebelung, Sven
Kather, Jakob Nikolas
Truhn, Daniel
author_sort Müller-Franzes, Gustav
collection PubMed
description Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditional latent DDPM designed for medical image generation, and evaluate its performance against GANs, which currently represent the state-of-the-art. Medfusion was trained and compared with StyleGAN-3 using fundoscopy images from the AIROGS dataset, radiographs from the CheXpert dataset, and histopathology images from the CRCDX dataset. Based on previous studies, Progressively Growing GAN (ProGAN) and Conditional GAN (cGAN) were used as additional baselines on the CheXpert and CRCDX datasets, respectively. Medfusion exceeded GANs in terms of diversity (recall), achieving better scores of 0.40 compared to 0.19 in the AIROGS dataset, 0.41 compared to 0.02 (cGAN) and 0.24 (StyleGAN-3) in the CRMDX dataset, and 0.32 compared to 0.17 (ProGAN) and 0.08 (StyleGAN-3) in the CheXpert dataset. Furthermore, Medfusion exhibited equal or higher fidelity (precision) across all three datasets. Our study shows that Medfusion constitutes a promising alternative to GAN-based models for generating high-quality medical images, leading to improved diversity and less artifacts in the generated images.
format Online
Article
Text
id pubmed-10372018
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-103720182023-07-28 A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis Müller-Franzes, Gustav Niehues, Jan Moritz Khader, Firas Arasteh, Soroosh Tayebi Haarburger, Christoph Kuhl, Christiane Wang, Tianci Han, Tianyu Nolte, Teresa Nebelung, Sven Kather, Jakob Nikolas Truhn, Daniel Sci Rep Article Although generative adversarial networks (GANs) can produce large datasets, their limited diversity and fidelity have been recently addressed by denoising diffusion probabilistic models, which have demonstrated superiority in natural image synthesis. In this study, we introduce Medfusion, a conditional latent DDPM designed for medical image generation, and evaluate its performance against GANs, which currently represent the state-of-the-art. Medfusion was trained and compared with StyleGAN-3 using fundoscopy images from the AIROGS dataset, radiographs from the CheXpert dataset, and histopathology images from the CRCDX dataset. Based on previous studies, Progressively Growing GAN (ProGAN) and Conditional GAN (cGAN) were used as additional baselines on the CheXpert and CRCDX datasets, respectively. Medfusion exceeded GANs in terms of diversity (recall), achieving better scores of 0.40 compared to 0.19 in the AIROGS dataset, 0.41 compared to 0.02 (cGAN) and 0.24 (StyleGAN-3) in the CRMDX dataset, and 0.32 compared to 0.17 (ProGAN) and 0.08 (StyleGAN-3) in the CheXpert dataset. Furthermore, Medfusion exhibited equal or higher fidelity (precision) across all three datasets. Our study shows that Medfusion constitutes a promising alternative to GAN-based models for generating high-quality medical images, leading to improved diversity and less artifacts in the generated images. Nature Publishing Group UK 2023-07-26 /pmc/articles/PMC10372018/ /pubmed/37495660 http://dx.doi.org/10.1038/s41598-023-39278-0 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Müller-Franzes, Gustav
Niehues, Jan Moritz
Khader, Firas
Arasteh, Soroosh Tayebi
Haarburger, Christoph
Kuhl, Christiane
Wang, Tianci
Han, Tianyu
Nolte, Teresa
Nebelung, Sven
Kather, Jakob Nikolas
Truhn, Daniel
A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title_full A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title_fullStr A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title_full_unstemmed A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title_short A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
title_sort multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10372018/
https://www.ncbi.nlm.nih.gov/pubmed/37495660
http://dx.doi.org/10.1038/s41598-023-39278-0
work_keys_str_mv AT mullerfranzesgustav amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT niehuesjanmoritz amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT khaderfiras amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT arastehsorooshtayebi amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT haarburgerchristoph amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT kuhlchristiane amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT wangtianci amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT hantianyu amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT nolteteresa amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT nebelungsven amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT katherjakobnikolas amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT truhndaniel amultimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT mullerfranzesgustav multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT niehuesjanmoritz multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT khaderfiras multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT arastehsorooshtayebi multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT haarburgerchristoph multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT kuhlchristiane multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT wangtianci multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT hantianyu multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT nolteteresa multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT nebelungsven multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT katherjakobnikolas multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis
AT truhndaniel multimodalcomparisonoflatentdenoisingdiffusionprobabilisticmodelsandgenerativeadversarialnetworksformedicalimagesynthesis