Cargando…

PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images

Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intrac...

Descripción completa

Detalles Bibliográficos
Autores principales: Qi, Jing, Ruan, Guangcong, Liu, Jia, Yang, Yi, Cao, Qian, Wei, Yanling, Nian, Yongjian
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9687195/
https://www.ncbi.nlm.nih.gov/pubmed/36354543
http://dx.doi.org/10.3390/bioengineering9110632
_version_ 1784835943851622400
author Qi, Jing
Ruan, Guangcong
Liu, Jia
Yang, Yi
Cao, Qian
Wei, Yanling
Nian, Yongjian
author_facet Qi, Jing
Ruan, Guangcong
Liu, Jia
Yang, Yi
Cao, Qian
Wei, Yanling
Nian, Yongjian
author_sort Qi, Jing
collection PubMed
description Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework ([Formula: see text]) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the [Formula: see text] model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed [Formula: see text] model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity.
format Online
Article
Text
id pubmed-9687195
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96871952022-11-25 PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images Qi, Jing Ruan, Guangcong Liu, Jia Yang, Yi Cao, Qian Wei, Yanling Nian, Yongjian Bioengineering (Basel) Article Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework ([Formula: see text]) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the [Formula: see text] model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed [Formula: see text] model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity. MDPI 2022-11-01 /pmc/articles/PMC9687195/ /pubmed/36354543 http://dx.doi.org/10.3390/bioengineering9110632 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Qi, Jing
Ruan, Guangcong
Liu, Jia
Yang, Yi
Cao, Qian
Wei, Yanling
Nian, Yongjian
PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title_full PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title_fullStr PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title_full_unstemmed PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title_short PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
title_sort phf(3) technique: a pyramid hybrid feature fusion framework for severity classification of ulcerative colitis using endoscopic images
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9687195/
https://www.ncbi.nlm.nih.gov/pubmed/36354543
http://dx.doi.org/10.3390/bioengineering9110632
work_keys_str_mv AT qijing phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT ruanguangcong phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT liujia phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT yangyi phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT caoqian phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT weiyanling phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages
AT nianyongjian phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages