Exploring milk shipment data for their potential for disease monitoring and for assessing resilience in dairy farms

The use of routinely recorded data for research purposes and disease surveillance is an attractive proposition. However, this requires that the validity and reliability of the data be evaluated for the purpose for which they are to be used. This manuscript reports an evaluation of milk shipment data for evaluating their usefulness in disease monitoring and the resilience of organic and conventional dairy herds in Sweden. A large number of inconsistencies were observed in the data, necessitating substantial efforts to “clean” the data. Given that the selection of rules used in the cleaning process was subjective in nature, a sensitivity analysis was carried out to determine if different cleaning routines produced substantially different results. Despite the cleaning efforts we observed far more large residuals at the shipment level than expected. Thus, it was concluded that the data were too “noisy” to be used for identification of short term impacts on milk production. Resilience was evaluated by examining the residual variance in milk shipped per cow per day under the assumption that herds with high resilience would have lower residual variance. The effects on residual variance of organic status or whether or not the herd used an automatic milking system were evaluated in models in which the residual variance was stratified or not by these factors. We did not find consistent evidence to suggest that organic herds had higher resilience than conventional herds, but this could be partly due to using residual variance as the measure indicating resilience.