Cargando…
3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conductin...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10481620/ https://www.ncbi.nlm.nih.gov/pubmed/37252868 http://dx.doi.org/10.1109/JBHI.2023.3281332 |
_version_ | 1785102018657910784 |
---|---|
author | Lauenburg, Leander Lin, Zudi Zhang, Ruihan dos Santos, Márcia Huang, Siyu Arganda-Carreras, Ignacio Boyden, Edward S. Pfister, Hanspeter Wei, Donglai |
author_facet | Lauenburg, Leander Lin, Zudi Zhang, Ruihan dos Santos, Márcia Huang, Siyu Arganda-Carreras, Ignacio Boyden, Edward S. Pfister, Hanspeter Wei, Donglai |
author_sort | Lauenburg, Leander |
collection | PubMed |
description | 3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the Cycle-GAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html. |
format | Online Article Text |
id | pubmed-10481620 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
record_format | MEDLINE/PubMed |
spelling | pubmed-104816202023-09-06 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs Lauenburg, Leander Lin, Zudi Zhang, Ruihan dos Santos, Márcia Huang, Siyu Arganda-Carreras, Ignacio Boyden, Edward S. Pfister, Hanspeter Wei, Donglai IEEE J Biomed Health Inform Article 3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the Cycle-GAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html. 2023-08 2023-08-07 /pmc/articles/PMC10481620/ /pubmed/37252868 http://dx.doi.org/10.1109/JBHI.2023.3281332 Text en https://creativecommons.org/licenses/by/4.0/This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ |
spellingShingle | Article Lauenburg, Leander Lin, Zudi Zhang, Ruihan dos Santos, Márcia Huang, Siyu Arganda-Carreras, Ignacio Boyden, Edward S. Pfister, Hanspeter Wei, Donglai 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title | 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title_full | 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title_fullStr | 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title_full_unstemmed | 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title_short | 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs |
title_sort | 3d domain adaptive instance segmentation via cyclic segmentation gans |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10481620/ https://www.ncbi.nlm.nih.gov/pubmed/37252868 http://dx.doi.org/10.1109/JBHI.2023.3281332 |
work_keys_str_mv | AT lauenburgleander 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT linzudi 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT zhangruihan 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT dossantosmarcia 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT huangsiyu 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT argandacarrerasignacio 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT boydenedwards 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT pfisterhanspeter 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans AT weidonglai 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans |