A Multimodal Multipath Artificial Intelligence System for Diagnosing Gastric Protruded Lesions on Endoscopy and Endoscopic Ultrasonography Images

INTRODUCTION: The aim of this study was to develop a novel artificial intelligence (AI) system that can automatically detect and classify protruded gastric lesions and help address the challenges of diagnostic accuracy and inter-reader variability encountered in routine diagnostic workflow. METHODS: We analyzed data from 1,366 participants who underwent gastroscopy at Jiangsu Provincial People's Hospital and Yangzhou First People's Hospital between December 2010 and December 2020. These patients were diagnosed with submucosal tumors (SMTs) including gastric stromal tumors (GISTs), gastric leiomyomas (GILs), and gastric ectopic pancreas (GEP). We trained and validated a multimodal, multipath AI system (MMP-AI) using the data set. We assessed the diagnostic performance of the proposed AI system using the area under the receiver-operating characteristic curve (AUC) and compared its performance with that of endoscopists with more than 5 years of experience in endoscopic diagnosis. RESULTS: In the ternary classification task among subtypes of SMTs using modality images, MMP-AI achieved the highest AUCs of 0.896, 0.890, and 0.999 for classifying GIST, GIL, and GEP, respectively. The performance of the model was verified using both external and internal longitudinal data sets. Compared with endoscopists, MMP-AI achieved higher recognition accuracy for SMTs. DISCUSSION: We developed a system called MMP-AI to identify protruding benign gastric lesions. This system can be used not only for white-light endoscope image recognition but also for endoscopic ultrasonography image analysis.