Cargando…

TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency

Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling t...

Descripción completa

Detalles Bibliográficos
Autores principales: Wu, Jianghao, Guo, Dong, Wang, Lu, Yang, Shuojue, Zheng, Yuanjie, Shapey, Jonathan, Vercauteren, Tom, Bisdas, Sotirios, Bradford, Robert, Saeed, Shakeel, Kitchen, Neil, Ourselin, Sebastien, Zhang, Shaoting, Wang, Guotai
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Elsevier Science Publishers 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10243514/
https://www.ncbi.nlm.nih.gov/pubmed/37528990
http://dx.doi.org/10.1016/j.neucom.2023.126295
_version_ 1785054443212898304
author Wu, Jianghao
Guo, Dong
Wang, Lu
Yang, Shuojue
Zheng, Yuanjie
Shapey, Jonathan
Vercauteren, Tom
Bisdas, Sotirios
Bradford, Robert
Saeed, Shakeel
Kitchen, Neil
Ourselin, Sebastien
Zhang, Shaoting
Wang, Guotai
author_facet Wu, Jianghao
Guo, Dong
Wang, Lu
Yang, Shuojue
Zheng, Yuanjie
Shapey, Jonathan
Vercauteren, Tom
Bisdas, Sotirios
Bradford, Robert
Saeed, Shakeel
Kitchen, Neil
Ourselin, Sebastien
Zhang, Shaoting
Wang, Guotai
author_sort Wu, Jianghao
collection PubMed
description Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods.
format Online
Article
Text
id pubmed-10243514
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Elsevier Science Publishers
record_format MEDLINE/PubMed
spelling pubmed-102435142023-08-01 TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency Wu, Jianghao Guo, Dong Wang, Lu Yang, Shuojue Zheng, Yuanjie Shapey, Jonathan Vercauteren, Tom Bisdas, Sotirios Bradford, Robert Saeed, Shakeel Kitchen, Neil Ourselin, Sebastien Zhang, Shaoting Wang, Guotai Neurocomputing Article Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods. Elsevier Science Publishers 2023-08-01 /pmc/articles/PMC10243514/ /pubmed/37528990 http://dx.doi.org/10.1016/j.neucom.2023.126295 Text en © 2023 The Author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
spellingShingle Article
Wu, Jianghao
Guo, Dong
Wang, Lu
Yang, Shuojue
Zheng, Yuanjie
Shapey, Jonathan
Vercauteren, Tom
Bisdas, Sotirios
Bradford, Robert
Saeed, Shakeel
Kitchen, Neil
Ourselin, Sebastien
Zhang, Shaoting
Wang, Guotai
TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title_full TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title_fullStr TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title_full_unstemmed TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title_short TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
title_sort tiss-net: brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10243514/
https://www.ncbi.nlm.nih.gov/pubmed/37528990
http://dx.doi.org/10.1016/j.neucom.2023.126295
work_keys_str_mv AT wujianghao tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT guodong tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT wanglu tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT yangshuojue tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT zhengyuanjie tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT shapeyjonathan tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT vercauterentom tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT bisdassotirios tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT bradfordrobert tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT saeedshakeel tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT kitchenneil tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT ourselinsebastien tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT zhangshaoting tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency
AT wangguotai tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency