Development and Validation of an Automatic Image-Recognition Endoscopic Report Generation System: A Multicenter Study

INTRODUCTION: Conventional gastrointestinal (GI) endoscopy reports written by physicians are time consuming and might have obvious heterogeneity or omissions, impairing the efficiency and multicenter consultation potential. We aimed to develop and validate an image recognition–based structured report generation system (ISRGS) through a multicenter database and to assess its diagnostic performance. METHODS: First, we developed and evaluated an ISRGS combining real-time video capture, site identification, lesion detection, subcharacteristics analysis, and structured report generation. White light and chromoendoscopy images from patients with GI lesions were eligible for study inclusion. A total of 46,987 images from 9 tertiary hospitals were used to train, validate, and multicenter test (6:2:2). Moreover, 5,699 images were prospectively enrolled from Qilu Hospital of Shandong University to further assess the system in a prospective test set. The primary outcome was the diagnosis performance of GI lesions in multicenter and prospective tests. RESULTS: The overall accuracy in identifying early esophageal cancer, early gastric cancer, early colorectal cancer, esophageal varices, reflux esophagitis, Barrett’s esophagus, chronic atrophic gastritis, gastric ulcer, colorectal polyp, and ulcerative colitis was 0.8841 (95% confidence interval, 0.8775–0.8904) and 0.8965 (0.8883–0.9041) in multicenter and prospective tests, respectively. The accuracy of cecum and upper GI site identification were 0.9978 (0.9969–0.9984) and 0.8513 (0.8399–0.8620), respectively. The accuracy of staining discrimination was 0.9489 (0.9396–0.9568). The relative error of size measurement was 4.04% (range 0.75%–7.39%). DISCUSSION: ISRGS is a reliable computer-aided endoscopic report generation system that might assist endoscopists working at various hospital levels to generate standardized and accurate endoscopy reports (http://links.lww.com/CTG/A485).