Reliability of Subjective Assessment of Spectral-Domain OCT Pathologic Features by Multiple Raters in Retinal Vein Occlusion

PURPOSE: To examine the interrater and intrarater reliability of qualitatively and quantitatively assessed disorganization of retinal inner layers (DRIL) and disorganization of retinal outer layers (DROL) by multiple raters. Subjectively assessing these surrogate biomarkers can be challenging in daily routine, despite the high resolution of spectral-domain (SD) OCT scans. DESIGN: Retrospective trial. PARTICIPANTS: Three hundred six pooled SD OCT scans of 34 patients treated for macular edema caused by retinal vein occlusion (RVO) between January 2016 and December 2017. METHODS: SD OCT scans were assessed by 6 raters regarding presence of cystoid macular edema, subretinal fluid (SRF), vitreoretinal traction, and epiretinal membrane and extent of DRIL and DROL. MAIN OUTCOME MEASURES: Interrater and intrarater reliability were calculated applying κ statistics for qualitative assessment regarding each pathologic feature’s presence in all evaluated OCT scans, and for quantified horizontal DRIL and DROL extent within each OCT cross-section. RESULTS: Cystoid macular edema and SRF assessments revealed excellent inter- and intrarater reliability with almost perfect strength of agreement, whereas subjective DRIL and DROL evaluations yielded low κ statistics with slight to moderate strength of agreement. Furthermore, the presence of SRF remarkably compromised the reliability of DROL detection. CONCLUSIONS: Our data highlight the limited subjective assessibility of DRIL and DROL, underscoring the need for automated image analysis to improve the reliability of OCT biomarkers for clinical studies and daily practice.