Interreader reproducibility of the Neck Imaging Reporting and Data system (NI-RADS) lexicon for the detection of residual/recurrent disease in treated head and neck squamous cell carcinoma (HNSCC)

BACKGROUND: To evaluate the inter- and intrareader agreement and reproducibility of the NI-RADS scoring system and lexicon with contrast-enhanced computed tomography (CECT) and contrast-enhanced magnetic resonance imaging (CEMRI). METHODS: This retrospective study included 97 CECT and CEMRI scans from 58 treated cases of head and neck squamous cell carcinoma (HNSCC) after the exclusion of head and neck cancers (HNCs) other than SCC and noncontrast and poor quality CT and MRI scans, with a total of 111 primary targets and 124 lymph node (LN) targets. Two experienced readers independently scored the likelihood of residual/recurrence for these targets based on the NI-RADS criteria and filled in report templates for NI-RADS lexicon diagnostic features. Inter- and intraobserver reproducibility was assessed with Cohen’s kappa, and the percent agreement was calculated. RESULTS: Almost perfect interreader agreement was found for the final NI-RADS category of the primary lesions and LNs, with K = 0.808 and 0.806, respectively. Better agreement was found for CT than for MRI (K = 0.843 and 0.77, respectively, P value 0.001). There was almost perfect agreement for excluding tissue enhancement (K = 0.826, 95% CI = 0.658–0.993, P value 0.001), with a percent agreement of 96.4%, and substantial agreement for discrete nodular and diffuse mucosal enhancement (K = 0.826, 95% CI = 0.658–0.993, P value 0.001), with a percent agreement of 96.4%. There was fair agreement for focal mucosal nonmass and deep ill-defined enhancement. The intrareader agreement was almost perfect for most of the rated features (K ranging from 0.802 to 1), with the exception of enlarging discrete nodule/mass and focal mucosal nonmass-like enhancement, which had substantial intraobserver agreement (K ranging from 0.768 to 0.786). CONCLUSION: The individual features of NI-RADS show variable degrees of confidence; however, the overall NI-RADS category was not significantly affected.