Deep learning pose estimation for multi-cattle lameness detection

The objective of this study was to develop a fully automated multiple-cow real-time lameness detection system using a deep learning approach for cattle detection and pose estimation that could be deployed across dairy farms. Utilising computer vision and deep learning, the system can analyse simultaneously both the posture and gait of each cow within a camera field of view to a very high degree of accuracy (94–100%). Twenty-five video sequences containing 250 cows in varying degrees of lameness were recorded and independently scored by three accredited Agriculture and Horticulture Development Board (AHDB) mobility scorers using the AHDB dairy mobility scoring system to provide ground truth lameness data. These observers showed significant inter-observer reliability. Video sequences were broken down into their constituent frames and with a further 500 images downloaded from google, annotated with 15 anatomical points for each animal. A modified Mask-RCNN estimated the pose of each cow to output 5 key-points to determine back arching and 2 key-points to determine head position. Using the SORT (simple, online, and real-time tracking) algorithm, cows were tracked as they move through frames of the video sequence (i.e., in moving animals). All the features were combined using the CatBoost gradient boosting algorithm with accuracy being determined using threefold cross-validation including recursive feature elimination. Precision was assessed using Cohen’s kappa coefficient and assessments of precision and recall. This methodology was applied to cows with varying degrees of lameness (according to accredited scoring, n = 3) and demonstrated that some characteristics directly associated with lameness could be monitored simultaneously. By combining the algorithm results over time, more robust evaluation of individual cow lameness was obtained. The model showed high performance for predicting and matching the ground truth lameness data with the outputs of the algorithm. Overall, threefold lameness detection accuracy of 100% and a lameness severity classification accuracy of 94% respectively was achieved with a high degree of precision (Cohen’s kappa = 0.8782, precision = 0.8650 and recall = 0.9209).