Instance Segmentation with Mask R-CNN Applied to Loose-Housed Dairy Cows in a Multi-Camera Setting

SIMPLE SUMMARY: As sociability in cattle is a matter of good animal husbandry, this study provides technical groundwork for a camera based system to automatically analyse dairy cattle herd activity. Eight surveillance cameras were recording a group of thirty-six lactating Holstein Friesian dairy cows. A Mask R-CNN model was trained to determine pixel level segmentation masks for the cows in the video material. The animals were successfully segmented reaching high ‘averaged precision scores’ for bounding boxes (0.91) and segmentation masks (0.85) for a given IOU threshold of 0.5. As providing training data for deep learning models is time consuming and tedious, this article also deals with the question “How many images do I have to annotate?” and analyses the performance of the model depending on the size of the used training data set. ABSTRACT: With increasing herd sizes came an enhanced requirement for automated systems to support the farmers in the monitoring of the health and welfare status of their livestock. Cattle are a highly sociable species, and the herd structure has important impact on the animal welfare. As the behaviour of the animals and their social interactions can be influenced by the presence of a human observer, a camera based system that automatically detects the animals would be beneficial to analyse dairy cattle herd activity. In the present study, eight surveillance cameras were mounted above the barn area of a group of thirty-six lactating Holstein Friesian dairy cows at the Chamber of Agriculture in Futterkamp in Northern Germany. With Mask R-CNN, a state-of-the-art model of convolutional neural networks was trained to determine pixel level segmentation masks for the cows in the video material. The model was pre-trained on the Microsoft common objects in the context data set, and transfer learning was carried out on annotated image material from the recordings as training data set. In addition, the relationship between the size of the used training data set and the performance on the model after transfer learning was analysed. The trained model achieved averaged precision (Intersection over union, IOU = 0.5) 91% and 85% for the detection of bounding boxes and segmentation masks of the cows, respectively, thereby laying a solid technical basis for an automated analysis of herd activity and the use of resources in loose-housing.