Cargando…

3D bi-directional transformer U-Net for medical image segmentation

As one of the popular deep learning methods, deep convolutional neural networks (DCNNs) have been widely adopted in segmentation tasks and have received positive feedback. However, in segmentation tasks, DCNN-based frameworks are known for their incompetence in dealing with global relations within i...

Descripción completa

Detalles Bibliográficos
Autores principales: Fu, Xiyao, Sun, Zhexian, Tang, Haoteng, Zou, Eric M., Huang, Heng, Wang, Yong, Zhan, Liang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9853518/
https://www.ncbi.nlm.nih.gov/pubmed/36687770
http://dx.doi.org/10.3389/fdata.2022.1080715
_version_ 1784872918985998336
author Fu, Xiyao
Sun, Zhexian
Tang, Haoteng
Zou, Eric M.
Huang, Heng
Wang, Yong
Zhan, Liang
author_facet Fu, Xiyao
Sun, Zhexian
Tang, Haoteng
Zou, Eric M.
Huang, Heng
Wang, Yong
Zhan, Liang
author_sort Fu, Xiyao
collection PubMed
description As one of the popular deep learning methods, deep convolutional neural networks (DCNNs) have been widely adopted in segmentation tasks and have received positive feedback. However, in segmentation tasks, DCNN-based frameworks are known for their incompetence in dealing with global relations within imaging features. Although several techniques have been proposed to enhance the global reasoning of DCNN, these models are either not able to gain satisfying performances compared with traditional fully-convolutional structures or not capable of utilizing the basic advantages of CNN-based networks (namely the ability of local reasoning). In this study, compared with current attempts to combine FCNs and global reasoning methods, we fully extracted the ability of self-attention by designing a novel attention mechanism for 3D computation and proposed a new segmentation framework (named 3DTU) for three-dimensional medical image segmentation tasks. This new framework processes images in an end-to-end manner and executes 3D computation on both the encoder side (which contains a 3D transformer) and the decoder side (which is based on a 3D DCNN). We tested our framework on two independent datasets that consist of 3D MRI and CT images. Experimental results clearly demonstrate that our method outperforms several state-of-the-art segmentation methods in various metrics.
format Online
Article
Text
id pubmed-9853518
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-98535182023-01-21 3D bi-directional transformer U-Net for medical image segmentation Fu, Xiyao Sun, Zhexian Tang, Haoteng Zou, Eric M. Huang, Heng Wang, Yong Zhan, Liang Front Big Data Big Data As one of the popular deep learning methods, deep convolutional neural networks (DCNNs) have been widely adopted in segmentation tasks and have received positive feedback. However, in segmentation tasks, DCNN-based frameworks are known for their incompetence in dealing with global relations within imaging features. Although several techniques have been proposed to enhance the global reasoning of DCNN, these models are either not able to gain satisfying performances compared with traditional fully-convolutional structures or not capable of utilizing the basic advantages of CNN-based networks (namely the ability of local reasoning). In this study, compared with current attempts to combine FCNs and global reasoning methods, we fully extracted the ability of self-attention by designing a novel attention mechanism for 3D computation and proposed a new segmentation framework (named 3DTU) for three-dimensional medical image segmentation tasks. This new framework processes images in an end-to-end manner and executes 3D computation on both the encoder side (which contains a 3D transformer) and the decoder side (which is based on a 3D DCNN). We tested our framework on two independent datasets that consist of 3D MRI and CT images. Experimental results clearly demonstrate that our method outperforms several state-of-the-art segmentation methods in various metrics. Frontiers Media S.A. 2023-01-06 /pmc/articles/PMC9853518/ /pubmed/36687770 http://dx.doi.org/10.3389/fdata.2022.1080715 Text en Copyright © 2023 Fu, Sun, Tang, Zou, Huang, Wang and Zhan. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Big Data
Fu, Xiyao
Sun, Zhexian
Tang, Haoteng
Zou, Eric M.
Huang, Heng
Wang, Yong
Zhan, Liang
3D bi-directional transformer U-Net for medical image segmentation
title 3D bi-directional transformer U-Net for medical image segmentation
title_full 3D bi-directional transformer U-Net for medical image segmentation
title_fullStr 3D bi-directional transformer U-Net for medical image segmentation
title_full_unstemmed 3D bi-directional transformer U-Net for medical image segmentation
title_short 3D bi-directional transformer U-Net for medical image segmentation
title_sort 3d bi-directional transformer u-net for medical image segmentation
topic Big Data
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9853518/
https://www.ncbi.nlm.nih.gov/pubmed/36687770
http://dx.doi.org/10.3389/fdata.2022.1080715
work_keys_str_mv AT fuxiyao 3dbidirectionaltransformerunetformedicalimagesegmentation
AT sunzhexian 3dbidirectionaltransformerunetformedicalimagesegmentation
AT tanghaoteng 3dbidirectionaltransformerunetformedicalimagesegmentation
AT zouericm 3dbidirectionaltransformerunetformedicalimagesegmentation
AT huangheng 3dbidirectionaltransformerunetformedicalimagesegmentation
AT wangyong 3dbidirectionaltransformerunetformedicalimagesegmentation
AT zhanliang 3dbidirectionaltransformerunetformedicalimagesegmentation