Reliability of NI-RADS criteria in the interpretation of contrast-enhanced magnetic resonance imaging considering the potential role of diffusion-weighted imaging

OBJECTIVES: To assess inter- and intrareader agreement of the Neck Imaging Reporting and Data System (NI-RADS) used in contrast-enhanced magnetic resonance imaging (MRI) including analysis of diffusion-weighted imaging (DWI), which is currently not part of the NI-RADS criteria. METHODS: This retrospective study included anonymized surveillance contrast-enhanced MRI datasets of 104 patients treated for different head and neck cancers. Three radiologists experienced in head and neck imaging reported findings for the primary site and the neck using NI-RADS criteria in a first step and evaluated DWI sequences for the primary site in a second step. Thirty randomly selected imaging datasets were again presented to the readers. Kappa statistics and observed agreement (A(o)) were calculated. RESULTS: Interreader agreement across all MRI datasets was moderate (κ(Fleiss) = 0.53) for NI-RADS categories assigned to the primary site, substantial for NI-RADS categories of the neck (κ(Fleiss) = 0.67), and almost perfect for DWI of the primary site (κ(Fleiss) = 0.83). Interreader agreement for the primary site was particularly low in cases of cancer recurrence (κ(Fleiss) = 0.35) and when categories 2a, 2b, and 3 were combined (κ(Fleiss) = 0.30). Intrareader agreement was considerably lower for NI-RADS categories of the primary site (range A(o) = 53.3–70.0%) than for NI-RADS categories of the neck (range A(o) = 83.3–90.0%) and DWI of the primary site (range A(o) = 93.3–100.0%). CONCLUSION: Interreader agreement of NI-RADS for reporting contrast-enhanced MRI findings is acceptable for the neck but limited for the primary site. Here, DWI has the potential to serve as a reliable additional criterion. KEY POINTS: • NI-RADS was originally designed for contrast-enhanced computed tomography with or without positron emission tomography but can also be used for contrast-enhanced magnetic resonance imaging alone. • Overall interreader agreement was acceptable for NI-RADS categories assigned to the neck but should be improved for the primary site, where it was inferior to DWI; similar tendencies were found for intrareader agreement. • DWI is currently no criterion of NI-RADS, but has shown potential to improve its reliability, especially for categories 2a, 2b, and 3 of the primary site. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00330-021-07693-4.