Inter-rater reliability of the radiographic assessment of simple bone cysts

PURPOSE: To develop and evaluate the reliability of an explicit set of parameters and criteria for simple bone cysts (SBCs) and evaluate the reliability of single versus serial chronological reading methods METHODS: Radiographic criteria were developed based on the literature and expert consensus. A single anteroposterior/lateral radiograph from 32 subjects with SBC were evaluated by three radiologists. A second reading was then conducted using revised criteria including a visual schematic. In the third reading the same images were assessed but radiologists had access to images from two additional time points. Inter-rater reliability was assessed after each reading using kappa (κ) and percentage agreement for categorical and binary parameters and intra-class correlation coefficient (ICC) for continuous parameters. RESULTS: Parameters that were revised with more explicit definitions including the visual schematic demonstrated consistent or improved inter-rater reliability with the exception of continuous cortical rim present and cyst location in the metaphysis and mid-diaphysis. Cortical rim displayed only slight reliability throughout (κ= -0.008 to 0.16). All other categorical parameters had a percentage agreement above 0.8 or a moderate (κ= 0.41 to 0.60), substantial (κ = 0.61 to 0.80) or almost perfect inter-rater reliability (κ = 0.81 to 1.0) in at least one reading. All continuous parameters demonstrated excellent inter-rater reliability (ICC > 0.75) in at least one reading with the exception of scalloping (ICC = 0.37 to 0.70). Inter-rater reliability values did not indicate an obviously superior method of assessment between single and serial chronological readings. CONCLUSION: Explicit criteria for SBC parameters used in their assessment demonstrated improved and substantial inter-rater reliability. Inter-rater reliability did not differ between single and serial chronological readings. LEVEL OF EVIDENCE: Not Applicable