Resectability of Pancreatic Cancer Is in the Eye of the Observer: A Multicenter, Blinded, Prospective Assessment of Interobserver Agreement on NCCN Resectability Status Criteria

OBJECTIVES: To determine the reproducibility of the National Comprehensive Cancer Network (NCCN) resectability status classification for pancreatic cancer. BACKGROUND: The NCCN classification defines 3 resectability classes (resectable, borderline resectable, locally advanced), according to vascular invasion. It is used to recommend different approaches and stratify patients during clinical trials. METHODS: Prospective, multicenter, observational study (trial ID: NCT03673423). Main outcome measure was the interobserver agreement of tumor assignment to different resectability classes and quantification of vascular invasion degrees. Agreement was measured by Fleiss’ k (k = 1 perfect agreement; k = 0 agreement by chance). Sixty-nine computed tomography (CT) scans of pathologically confirmed pancreatic adenocarcinoma were independently reviewed in a blinded fashion by 22 observers from 11 hospitals (11 surgeons and 11 radiologists). Rating differences between surgeons or radiologists and between hospitals with different volumes (≥60 or <60 resections/year) were assessed. RESULTS: Complete agreement among 22 observers was recorded in 5 CT scans (7.2%), whereas 25 CT scans (36.2%) were variously assigned to all 3 resectability classes. Interobserver agreement varied from fair to moderate (Fleiss’ k range: 0.282–0.555), with the lowest agreement for borderline resectable tumors. Assessing vascular contact ≤180° had the lowest agreement for all vessels (k range: 0.196–0.362). The highest concordance was recorded for venous invasion >180° (k range: 0.619–0.756). Neither reviewers’ specialty nor hospital volume influenced the agreement. CONCLUSIONS: There is high variability in the assignment to resectability categories, which may compromise the reliability of treatments recommendations and the evidence of trials stratifying patients in resectability classes. Criteria should be revised to allow a reproducible classification.