Analysis of interreader agreement in structured reports of pelvic multiparametric magnetic resonance imaging using the METastasis Reporting and Data System for Prostate Cancer guidelines

PURPOSE: To evaluate interreader agreement on pelvic multiparametric magnetic resonance imaging (mpMRI) interpretation among radiologists using a structured reporting tool based on the METastasis Reporting and Data System for Prostate Cancer (MET-RADS-P) guidelines. METHODS: A structured report for follow-up pelvic mpMRI for advanced prostate cancer (APC) patients was formulated based on MET-RADS-P guidelines. In total, 163 paired pelvic mpMRI examinations were performed from December 2017 to February 2021 on 105 patients with APC. These were retrospectively reviewed by two senior and two junior radiologists for metastatic lesion detection and were categorized by these readers using primary/secondary response assessment categories (RACs), with and without the structured report. Interreader agreement regarding metastasis detection and RAC scores was evaluated with Cohen’s kappa and weighted Cohen’s kappa statistics (K), respectively. RESULTS: The two senior radiologists showed higher agreement with the reference standard for metastasis detection using the structured report (S1: K = 0.83; S2: K = 0.73) compared with the conventional report (S1: K = 0.72; S2: K = 0.61). Junior radiologists showed similar results (J1: 0.66 vs. 0.59; J2: 0.65 vs. 0.57). The overall agreement between the two senior radiologists was excellent for the primary RAC pattern using the structured reports (K = 0.81) and was substantial for secondary RAC categorization (K = 0.75). The interreader agreement of the two junior radiologists was substantial for both primary and secondary RAC values (K = 0.76, 0.68). CONCLUSION: Good interreader agreement was found for the follow-up assessment of APC patients between radiologists, where the pelvic mpMRI was reported using MET-RADS-P guidelines. This improvement applied to both metastatic lesion detection and qualitative RAC assessment.