Surgeon Agreement on the Presence of Pathologic Anterior Instability on Shoulder Imaging Studies

BACKGROUND: In the setting of anterior shoulder instability, it is important to assess the reliability of orthopaedic surgeons to diagnose pathologic characteristics on the 2 most common imaging modalities used in clinical practice: standard plain radiographs and magnetic resonance imaging (MRI). PURPOSE: To assess the intra- and interrater reliability of diagnosing pathologic characteristics associated with anterior shoulder instability using standard plain radiographs and MRI. STUDY DESIGN: Cohort study (diagnosis); Level of evidence, 3. METHODS: Patient charts at a single academic institution were reviewed for anterior shoulder instability injuries. The study included 40 sets of images (20 radiograph sets, 20 MRI series). The images, along with standardized evaluation forms, were distributed to 22 shoulder/sports medicine fellowship–trained orthopaedic surgeons over 2 points in time. Kappa values for inter- and intrarater reliability were calculated. RESULTS: The overall response rate was 91%. For shoulder radiographs, interrater agreement was fair to moderate for the presence of glenoid lesions (κ = 0.49), estimate of glenoid lesion surface area (κ = 0.59), presence of a Hill-Sachs lesion (κ = 0.35), and estimate of Hill-Sachs surface area (κ = 0.50). Intrarater agreement was moderate for radiographs (κ = 0.48-0.57). For shoulder MRI, interrater agreement was fair to moderate for the presence of glenoid lesions (κ = 0.44), glenoid lesion surface area (κ = 0.35), Hill-Sachs lesion (κ = 0.33), Hill-Sachs surface area (κ = 0.28), humeral head edema (κ = 0.41), and presence of a capsulolabral injury (κ = 0.36). Fair agreement was found for specific type of capsulolabral injury (κ = 0.21). Intrarater agreement for shoulder MRI was moderate for the presence of glenoid lesion (κ = 0.59), presence of a Hill-Sachs lesion (κ = 0.52), estimate of Hill-Sachs surface area (κ = 0.50), humeral head edema (κ = 0.51), and presence of a capsulolabral injury (κ = 0.53), and agreement was substantial for glenoid lesion surface area (κ = 0.63). Intrarater agreement was fair for determining the specific type of capsulolabral injury (κ = 0.38). CONCLUSION: Fair to moderate agreement by surgeons was found when evaluating imaging studies for anterior shoulder instability. Agreement was similar for identifying pathologic characteristics on radiographs and MRI. There was a trend toward better agreement for the presence of glenoid-sided injury. The lowest agreement was observed for specific capsulolabral injuries.