Cargando…

3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs

3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conductin...

Descripción completa

Detalles Bibliográficos
Autores principales: Lauenburg, Leander, Lin, Zudi, Zhang, Ruihan, dos Santos, Márcia, Huang, Siyu, Arganda-Carreras, Ignacio, Boyden, Edward S., Pfister, Hanspeter, Wei, Donglai
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10481620/
https://www.ncbi.nlm.nih.gov/pubmed/37252868
http://dx.doi.org/10.1109/JBHI.2023.3281332
_version_ 1785102018657910784
author Lauenburg, Leander
Lin, Zudi
Zhang, Ruihan
dos Santos, Márcia
Huang, Siyu
Arganda-Carreras, Ignacio
Boyden, Edward S.
Pfister, Hanspeter
Wei, Donglai
author_facet Lauenburg, Leander
Lin, Zudi
Zhang, Ruihan
dos Santos, Márcia
Huang, Siyu
Arganda-Carreras, Ignacio
Boyden, Edward S.
Pfister, Hanspeter
Wei, Donglai
author_sort Lauenburg, Leander
collection PubMed
description 3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the Cycle-GAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.
format Online
Article
Text
id pubmed-10481620
institution National Center for Biotechnology Information
language English
publishDate 2023
record_format MEDLINE/PubMed
spelling pubmed-104816202023-09-06 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs Lauenburg, Leander Lin, Zudi Zhang, Ruihan dos Santos, Márcia Huang, Siyu Arganda-Carreras, Ignacio Boyden, Edward S. Pfister, Hanspeter Wei, Donglai IEEE J Biomed Health Inform Article 3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the Cycle-GAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html. 2023-08 2023-08-07 /pmc/articles/PMC10481620/ /pubmed/37252868 http://dx.doi.org/10.1109/JBHI.2023.3281332 Text en https://creativecommons.org/licenses/by/4.0/This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
spellingShingle Article
Lauenburg, Leander
Lin, Zudi
Zhang, Ruihan
dos Santos, Márcia
Huang, Siyu
Arganda-Carreras, Ignacio
Boyden, Edward S.
Pfister, Hanspeter
Wei, Donglai
3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title_full 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title_fullStr 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title_full_unstemmed 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title_short 3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs
title_sort 3d domain adaptive instance segmentation via cyclic segmentation gans
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10481620/
https://www.ncbi.nlm.nih.gov/pubmed/37252868
http://dx.doi.org/10.1109/JBHI.2023.3281332
work_keys_str_mv AT lauenburgleander 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT linzudi 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT zhangruihan 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT dossantosmarcia 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT huangsiyu 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT argandacarrerasignacio 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT boydenedwards 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT pfisterhanspeter 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans
AT weidonglai 3ddomainadaptiveinstancesegmentationviacyclicsegmentationgans