Quantification of Celiac Disease Severity Using Video Capsule Endoscopy: A Comparison of Human Experts and Machine Learning Algorithms

BACKGROUND: Video capsule endoscopy (VCE) is an attractive method for diagnosing and objectively monitoring disease activity in celiac disease (CeD). Its use, facilitated by artificial intelligence-based tools, may allow computer-assisted interpretation of VCE studies, transforming a subjective test into a quantitative and reproducible measurement tool. OBJECTIVE: To evaluate and compare objective CeD severity assessment as determined with VCE by expert human readers and a machine learning algorithm (MLA). METHODS: Patients ≥ 18 years with histologically proven CeD underwent VCE. Examination frames were scored by three readers from one center and the MLA, using a 4-point ordinal scale for assessing the severity of CeD enteropathy. After scoring, curves representing CeD severity across the entire small intestine (SI) and individual tertiles (proximal, mid, and distal) were fitted for each reader and the MLA. All comparisons used Krippendorff’s alpha; values > 0.8 represent excellent to ‘almost perfect’ inter-reader agreement. RESULTS: VCEs from 63 patients were scored. Readers demonstrated strong inter-reader agreement on celiac villous damage (alpha=0.924), and mean value reader curves showed similarly excellent agreement with MLA curves (alpha=0.935). Average reader and MLA curves were comparable for mean and maximum values for the first SI tertile (alphas=0.932 and 0.867, respectively) and the mean value over the entire SI (alpha=0.945). CONCLUSION: A novel MLA demonstrated excellent agreement on whole SI imaging with three expert gastroenterologists. An ordinal scale permitted high inter-reader agreement, accurately and reliably replicated by the MLA. Interpreting VCEs using MLAs may allow automated diagnosis and disease burden assessment in CeD.