Awareness of treatment: A source of bias in subjective grading of ocular complications

PURPOSE: Bias has been described as one important obstacle in scientific research. The aim of this study was to explore “awareness of treatment” as a possible source of bias in subjective grading of ocular complications. METHODS: Thirty subjects with similar, basic experience with grading scales participated in the study. The Efron grading scales were used to grade 24 images of three different ocular conditions (eight images each of bulbar hyperaemia, limbal vascularization and corneal staining). Three consecutive, two weeks apart, grading sessions were scheduled, in which the same images were graded, although in the third session images were deceptively labelled as either “treated” or “untreated”. Grading results from the first and second sessions were compared to determine grading reliability and discrepancies with the third session informed of grading bias originating from “awareness of treatment”. RESULTS: Moderate to good test-retest reliability was found for all conditions, with median intraclass correlation values of 0.80 (0.62–0.84) for bulbar hyperaemia, 0.68 (0.65–0.77) for limbal vascularization and 0.68 (0.66–0.74) for corneal staining. Grading values from the first and third sessions evidenced negative and positive systematic errors (bias) for “treated” and “untreated” conditions, respectively. Statistically significant differences were found between the average grading discrepancies of session 1 and session 2 and those of session 1 and session 3 (all p<0.001). CONCLUSIONS: “Awareness of treatment” may be considered a source of bias of subjective grading of ocular complications, although the actual effect of bias is unlikely to be of clinical significance.