Cargando…
TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency
Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling t...
Autores principales: | , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Elsevier Science Publishers
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10243514/ https://www.ncbi.nlm.nih.gov/pubmed/37528990 http://dx.doi.org/10.1016/j.neucom.2023.126295 |
_version_ | 1785054443212898304 |
---|---|
author | Wu, Jianghao Guo, Dong Wang, Lu Yang, Shuojue Zheng, Yuanjie Shapey, Jonathan Vercauteren, Tom Bisdas, Sotirios Bradford, Robert Saeed, Shakeel Kitchen, Neil Ourselin, Sebastien Zhang, Shaoting Wang, Guotai |
author_facet | Wu, Jianghao Guo, Dong Wang, Lu Yang, Shuojue Zheng, Yuanjie Shapey, Jonathan Vercauteren, Tom Bisdas, Sotirios Bradford, Robert Saeed, Shakeel Kitchen, Neil Ourselin, Sebastien Zhang, Shaoting Wang, Guotai |
author_sort | Wu, Jianghao |
collection | PubMed |
description | Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods. |
format | Online Article Text |
id | pubmed-10243514 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Elsevier Science Publishers |
record_format | MEDLINE/PubMed |
spelling | pubmed-102435142023-08-01 TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency Wu, Jianghao Guo, Dong Wang, Lu Yang, Shuojue Zheng, Yuanjie Shapey, Jonathan Vercauteren, Tom Bisdas, Sotirios Bradford, Robert Saeed, Shakeel Kitchen, Neil Ourselin, Sebastien Zhang, Shaoting Wang, Guotai Neurocomputing Article Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods. Elsevier Science Publishers 2023-08-01 /pmc/articles/PMC10243514/ /pubmed/37528990 http://dx.doi.org/10.1016/j.neucom.2023.126295 Text en © 2023 The Author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). |
spellingShingle | Article Wu, Jianghao Guo, Dong Wang, Lu Yang, Shuojue Zheng, Yuanjie Shapey, Jonathan Vercauteren, Tom Bisdas, Sotirios Bradford, Robert Saeed, Shakeel Kitchen, Neil Ourselin, Sebastien Zhang, Shaoting Wang, Guotai TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title | TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title_full | TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title_fullStr | TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title_full_unstemmed | TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title_short | TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
title_sort | tiss-net: brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10243514/ https://www.ncbi.nlm.nih.gov/pubmed/37528990 http://dx.doi.org/10.1016/j.neucom.2023.126295 |
work_keys_str_mv | AT wujianghao tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT guodong tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT wanglu tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT yangshuojue tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT zhengyuanjie tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT shapeyjonathan tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT vercauterentom tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT bisdassotirios tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT bradfordrobert tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT saeedshakeel tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT kitchenneil tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT ourselinsebastien tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT zhangshaoting tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency AT wangguotai tissnetbraintumorimagesynthesisandsegmentationusingcascadeddualtasknetworksanderrorpredictionconsistency |