OR27-2 Inter-reader Agreement of ATA Sonographic Risk In Thyroid Nodules With Indeterminate Cytology

Introduction The 2015 ATA sonographic risk (ASR) stratification of thyroid nodules has been proposed to risk stratify thyroid nodules with indeterminate cytology. Our objective is to determine the inter-reader agreement among radiologists using ASR stratification in thyroid nodules with indeterminate cytology. Methods Three board certified radiologists at an academic medical center who were blinded to clinical data and each other’s reports, interpreted the ultrasound (US) findings on 97 nodules with Bethesda III cytology. The nodules were classified into high (H), intermediate (I), low (L), very low (VL) or non-ATA (NA) risk categories. In addition, 7 features were described: echogenicity, composition, shape, vascularity, type of margins, presence and type of calcifications and presence of suspicious lymph nodes. Results From a total of 97 nodules reviewed by 3 radiologists, the majority consensus revealed that 31 (32%), 21(22%), 37 (38%) and 7 (7%) were described as high, intermediate, low and very low ASR respectively. One nodule showed absolute disparity in ASR classification. The mean percent agreement between all readers was 86%; 95% CI 79-93(p<0.01) and the kappa was 0.80; 95% CI 0.7-0.9 (p<0.01). All readers agreed on ASR 79% of the time (77/97). ≥ 2 readers agreed on ASR reaching consensus 99% of the time (96/97). The ASR categories in which all 3 readers agreed the most were (H) and (VL) risk categories with 87% and 86% agreement rates. Greater disagreement rates were observed in (I) 33% and (L) 19% ASR. In 9/97 (9%) nodules there was discrepancy between lower (L+VL) and higher risk categories (I+H). The absolute inter-reader agreement rates for each US feature were: presence of suspicious lymph node 91/97 (94%), shape 90/97 (93%), composition 88/97 (91%), presence of calcifications 87/97 (90%), margins 79/97 (81%), echogenicity 68/97 (70%), type of calcifications 9/13 (69%), type of irregular margins 32/57 (56%) and vascularity 48/93 (52%). No absolute consensus in ASR was reached in 20 nodules (21%). Out of these 20 cases, the disagreement rates for each US feature were: description of calcification 2/3 (67%), echogenicity 10/20 (50%), description of irregular borders 7/15 (47%), presence of vascularity 7/18 (39%), borders 6/20 (30%), presence of calcifications 4/20 (20%), shape 2/20 (10%) and presence of a suspicious lymph node 1/20 (5%). Conclusion A high rate of agreement among readers was observed when stratifying nodules for ASR. Agreement was observed predominantly in the high and very low risk categories. The most common pattern of disagreement was intermediate ASR and the most common features of disagreement were type of calcification and echogenicity followed by type of irregular border and vascularity however the latter two do not determine ASR. Nine cases exhibited disagreement between lower and higher risk categories, which could potentially impact patient management.