Inter-modality assessment of medial temporal lobe atrophy in a non-demented population: application of a visual rating scale template across radiologists with varying clinical experience

OBJECTIVES: To assess inter-modality agreement and accuracy for medial temporal lobe atrophy (MTA) ratings across radiologists with varying clinical experience in a non-demented population. METHODS: Four raters (two junior radiologists and two senior neuroradiologists) rated MTA on CT and MRI scans using Scheltens’ MTA scale. Ratings were compared to a consensus rating by two experienced neuroradiologists for estimation of true positive and negative rates (TPR and TNR) and over- and underestimation of MTA. Inter-modality agreement expressed as Cohen’s κ (dichotomized data), Cohen’s κ(w), and two-way mixed, single measures, consistency ICC (ordinal data) were determined. Adequate agreement was defined as κ/κ(w) ≥ 0.80 and ICC ≥ 0.80 (significance level at 95% CI ≥ 0.65). RESULTS: Forty-nine subjects (median age 72 years, 27% abnormal MTA) with cognitive impairment were included. Only junior radiologists achieved adequate agreement expressed as Cohen’s κ. All raters achieved adequate agreement expressed as Cohen’s κ(w) and ICC. True positive rates varied from 69 to 100% and TNR varied from 85 to 100%. No under- or overestimation of MTA was observed. Ratings did not differ between radiologists. CONCLUSION: We conclude that radiologists with varying experience achieve adequate inter-modality agreement and similar accuracy when Scheltens’ MTA scale is used to rate MTA on a non-demented population. However, TPR varied between radiologists which could be attributed to rating style differences. KEY POINTS: • Radiologists with varying experience achieve adequate inter-modality agreement with similar accuracy when Scheltens’ MTA scale is used to rate MTA on a non-demented population. • Differences in rating styles might affect accuracy, this was most evident for senior neuroradiologists, and only junior radiologists achieved adequate agreement on dichotomized (abnormal/normal) ratings. • The use of an MTA scale template might compensate for varying clinical experience which could make it applicable for clinical use.