Observer agreement for the diagnosis of intestinal acute graft-vs.-host disease based on the presence of villous atrophy in the terminal ileum

Intestinal graft-vs.-host disease (GVHD) is a serious complication of allo-hematopoietic stem cell transplantation (allo-HSCT). Villous atrophy in the terminal ileum is considered a useful diagnostic indicator for GVHD. However, the inter- and intra-observer agreement regarding the ileocolonoscopic findings indicative of acute intestinal GVHD, i.e., villous atrophy in the terminal ileum, are currently insufficient in multiple institutions. Thus, the present study aimed to investigate the incidence of villous atrophy in the terminal ileum to diagnose acute intestinal GVHD and determine the inter- and intra-observer agreement regarding this result for experienced endoscopists from multiple institutions. Consecutive patients who underwent allo-HSCT were referred to our institution between May 2008 and September 2015. A total of 54 patients underwent total ileocolonoscopy after allo-HSCT due to suspected intestinal acute GVHD. Subsequently, three observers from different institutions evaluated the cases for the presence of villous atrophy in the terminal ileum. In this study, the pathology results were a gold standard to evaluate the predictive value of ileocolonoscopy detection. Definitive pathological and non-pathological GVHD was diagnosed in 22 and 32 cases, respectively. The results of examining whether villous atrophy could predict GVHD were as follows. For three observers (A, B and C), the sensitivity of villous atrophy in the terminal ileum was 86.4, 77.3 and 79.2%, respectively, whereas the specificity was 62.5, 62.5 and 86.7%, respectively. The positive predictive value (PPV) and negative predictive value (NPV) of villous atrophy for GVHD were as follows: The PPV of appearance was 61.3, 58.6 and 82.6%, respectively, whereas the NPV was 87.0, 80.0 and 83.9%, respectively. Kappa coefficients for the inter-observer reliability were 0.85, 0.63 and 0.63 for observers A and B, A and C, and B and C, respectively. The intra-observer kappa coefficient was 0.88 for observer A, 0.73 for observer B and 0.75 for observer C. A substantial observer agreement was achieved for the analysis of villous atrophy in the terminal ileum and the agreement for the predictive histological diagnosis was also excellent. Based on the results of the present study, identification of villous atrophy in the terminal ileum was a clinically effective diagnostic parameter, even if different endoscopists were involved in the diagnosis at multiple institutions. The present study was registered as a trial with the University Hospital Medical Information Network (UMIN; registration no. UMIN000025390).