Performance of the No-U-Turn sampler in multi-trait variance component estimation using genomic data

BACKGROUND: Multi-trait genetic parameter estimation is an important topic for target traits with few records and with a low heritability and when the genetic correlation between target and secondary traits is strong. However, estimating correlations between multiple traits is difficult for both Bayesian and non-Bayesian inferences. We extended a Hamiltonian Monte Carlo approach using the No-U-Turn Sampler (NUTS) to a multi-trait animal model and investigated the performance of estimating (co)variance components and breeding values, compared to those for restricted maximum likelihood and Gibbs sampling with a population size of 2314 and 578 in a simulated and real pig dataset, respectively. For real data, we used publicly available data for three traits from the Pig Improvement Company (PIC). For simulation data, we generated two quantitative traits by using the genotypes of the PIC data. For NUTS, two prior distributions were adopted: Lewandowski-Kurowicka-Joe (LKJ) and inverse-Wishart distributions. RESULTS: For the two simulated traits with heritabilities of 0.1 and 0.5, most estimates of the genetic and residual variances for NUTS with the LKJ prior were closer to the true values and had smaller root mean square errors and smaller mean absolute errors, compared to NUTS with inverse-Wishart priors, Gibbs sampling and restricted maximum likelihood. The accuracies of estimated breeding values for lowly heritable traits for NUTS with LKJ and inverse-Wishart priors were 14.8% and 11.1% higher than those for Gibbs sampling and restricted maximum likelihood, respectively, with a population size of 578. For the trivariate animal model with real pig data, the estimates of the genetic correlations for Gibbs sampling and restricted maximum likelihood were strongly affected by population size, compared to NUTS. For both the simulated and pig data, the genetic variances and heritabilities for NUTS with an inverse-Wishart prior were overestimated for low-heritability traits when the population size was 578. CONCLUSIONS: The accuracies of variance components and breeding values estimates for a multi-trait animal model using NUTS with the LKJ prior were equal to or higher than those obtained with restricted maximum likelihood or Gibbs sampling. Therefore, when the population size is small, NUTS with an LKJ prior could be an alternative sampling method for multi-trait analysis in animal breeding. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12711-022-00743-5.