Cargando…
PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images
Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intrac...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9687195/ https://www.ncbi.nlm.nih.gov/pubmed/36354543 http://dx.doi.org/10.3390/bioengineering9110632 |
_version_ | 1784835943851622400 |
---|---|
author | Qi, Jing Ruan, Guangcong Liu, Jia Yang, Yi Cao, Qian Wei, Yanling Nian, Yongjian |
author_facet | Qi, Jing Ruan, Guangcong Liu, Jia Yang, Yi Cao, Qian Wei, Yanling Nian, Yongjian |
author_sort | Qi, Jing |
collection | PubMed |
description | Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework ([Formula: see text]) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the [Formula: see text] model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed [Formula: see text] model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity. |
format | Online Article Text |
id | pubmed-9687195 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-96871952022-11-25 PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images Qi, Jing Ruan, Guangcong Liu, Jia Yang, Yi Cao, Qian Wei, Yanling Nian, Yongjian Bioengineering (Basel) Article Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework ([Formula: see text]) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the [Formula: see text] model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed [Formula: see text] model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity. MDPI 2022-11-01 /pmc/articles/PMC9687195/ /pubmed/36354543 http://dx.doi.org/10.3390/bioengineering9110632 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Qi, Jing Ruan, Guangcong Liu, Jia Yang, Yi Cao, Qian Wei, Yanling Nian, Yongjian PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title | PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title_full | PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title_fullStr | PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title_full_unstemmed | PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title_short | PHF(3) Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images |
title_sort | phf(3) technique: a pyramid hybrid feature fusion framework for severity classification of ulcerative colitis using endoscopic images |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9687195/ https://www.ncbi.nlm.nih.gov/pubmed/36354543 http://dx.doi.org/10.3390/bioengineering9110632 |
work_keys_str_mv | AT qijing phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT ruanguangcong phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT liujia phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT yangyi phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT caoqian phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT weiyanling phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages AT nianyongjian phf3techniqueapyramidhybridfeaturefusionframeworkforseverityclassificationofulcerativecolitisusingendoscopicimages |