Observer agreement for small bowel ultrasound in Crohn’s disease: results from the METRIC trial

PURPOSE: To prospectively evaluate interobserver agreement for small bowel ultrasound (SBUS) in newly diagnosed and relapsing Crohn’s disease. METHODS: A subset of patients recruited to a prospective trial comparing the diagnostic accuracy of MR enterography and SBUS underwent a second SBUS performed by one of a pool of six practitioners, who recorded the presence, activity and location of small bowel and colonic disease. Detailed segmental mural and extra-mural observations were also scored. Interobserver variability was expressed as percentage agreement with a construct reference standard, split by patient cohort, grouping disease as present or absent. Prevalence adjusted bias adjusted kappa (PABAK), and simple percentage agreement between practitioners, irrespective of the reference standard, were calculated. RESULTS: Thirty-eight patients (11 new diagnosis, 27 relapse) were recruited from two sites. Overall percentage agreement for small bowel disease presence against the consensus reference was 82% (52–95% (95%CI)), kappa coefficient (κ) 0.64, (substantial agreement) for new diagnosis and 81%, κ 0.63 (substantial agreement) for the relapsing cohort. Agreement for colonic disease presence was 64%, κ 0.27 (fair agreement) in new diagnosis and 78%,κ 0.56 (moderate agreement) in the relapsing cohort. Simple agreement between practitioners was 84% and 87% for small bowel and colonic disease presence respectively. Practitioners agreed on small bowel disease activity in 24/27 (89%) where both identified disease. Kappa agreement for detailed mural observations ranged from κ 0.00 to 1.00. CONCLUSION: There is substantial practitioner agreement for small bowel disease presence in newly diagnosed and relapsing CD patients, supporting wider dissemination of enteric US. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00261-020-02405-w) contains supplementary material, which is available to authorized users.