Reliability of observer ratings: Qualitative behaviour assessments of shelter dogs using a fixed list of descriptors

Qualitative Behaviour Assessment (QBA) is a whole-animal approach used to quantify the expressive style of animals’ behaviour. The aim of this study was to evaluate the inter-observer reliability of QBA of shelter dogs using a fixed list of descriptors. The fixed list of 20 terms was generated using a group of experts and literature reviews. In the pilot study, seven veterinary students scored 12 two-minute video clips, and in the main study, 22 final year veterinary nurse students and third-year veterinary students scored the same videos. The two datasets were analysed using Principal Component Analysis (PCA), and the level of agreement for the main components and individual terms was assessed using Kendall's coefficient of concordance (W). In the pilot study, the observer agreement was 0.89 for PC1 and 0.78 for PC2, indicating high inter-observer agreement. The reliability was similarly high for both components in the main study (0.88 and 0.79, respectively). Results also demonstrated high or moderate agreement for most of the terms included in the fixed list. We propose that this approach can be a useful learning tool for students. Our results support further exploration of this method for the assessment of shelter dog welfare by direct observation.