How to assess communication skills? Development of the rating scale ComOn Check: Evaluation of communication skills

Background: Good communication is a core competency for all physicians. Thus, medical students require adequate preparation in communication skills. For research purposes, as well as for evaluation in teaching, there is a clear need for reliable assessment tools. We analyzed the shortcomings of existing instruments and saw a need for a new rating scale. The aim of this publication is to describe the development process for, and evaluation of, a new rating scale. Methods: First, we developed the rating scale in 10 steps. Then, two raters evaluated the newly developed rating scale by rating 135 videotaped consultations of medical students with standardized patients. Additionally, standardized patients evaluated students’ performance, which was used as an outside criterion to validate ratings. Results: Our rating scale comprises six domains with 13 specific items evaluated on a five-point Likert scale: initiating conversation, patient’s perception, structure of conversation, patient’s emotions, end of conversation, and general communication skills. Item-total correlation coefficients between the checklist items ranged from 0.15 to 0.78. Subscale consistency was calculated for domains comprised of more than one item and Cronbach’s α ≥ 0.77, indicating acceptable consistency. Standardized patients’ global evaluation correlated moderately with overall expert ratings (Spearman’s ρ = .40, p < .001). Conclusion: Our rating scale is a reliable and applicable assessment tool. The rating scale focuses on the evaluation of general communication skills and can be applied in research as well as in evaluations, such as objective structured clinical examinations (OSCE). Abbreviations: CST: Communication skills training; ICC: Intra-class correlation coefficient; OSCE: Objective structured clinical examination; SP: Standardized patients; SD: Standard deviation; M: Mean