Is there agreement between evaluators that used two scoring systems to measure acute radiation dermatitis?

To analyze the agreement between the nurses evaluating radiodermatitis that used the Radiation Therapy Oncology Group (RTOG) and the World Health Organization (WHO) scales. A prospective and longitudinal study conducted in 2016 to 2017, in a university hospital. We analyzed 855 images of irradiated sites of 100 breast cancer women during radiotherapy. In order to evaluate the agreement between 3 observers that evaluated theses irradiated sites Krippendorff's alpha and weighted kappa were obtained and analyzed. The pairwise agreement among the evaluators was fair and moderate (RTOG scale: 0.408, 95% confidence interval, CI 0.370–0.431; WHO scale: 0.559, 95% CI 0.529–0.590). In addition, the general agreement rates were 10.2% and 29.2%, respectively. When assessing the overall absolute agreement between the evaluators according with different phototypes and types of surgery, there was a fair agreement according to the RTOG scale when evaluating patients with phototype V or VI and mastectomy (3.7% and 8.8%, respectively). The RTOG and WHO scales should be used with caution in clinical practice to identify the prevalence of radiodermatitis and the severity. Another point of caution is that skin phototype and the type of surgery may influence the analysis outcome. An illustrative scale was designed and proposed, by our group, aiming to improve accuracy and agreement between evaluators that will be tested in subsequent clinical studies.