Animal models with group-specific additive genetic variances: extending genetic group models

BACKGROUND: The animal model is a key tool in quantitative genetics and has been used extensively to estimate fundamental parameters, such as additive genetic variance or heritability. An implicit assumption of animal models is that all founder individuals derive from a single population. This assumption is commonly violated, for instance in crossbred livestock or when a meta-population is split into genetically differentiated subpopulations. Ignoring that base populations are genetically heterogeneous and thus split into different ‘genetic groups’ may lead to biased parameter estimates, especially for additive genetic variance. To avoid such biases, genetic group animal models, which account for the presence of more than one genetic group, have been proposed. Unfortunately, the method to date is only computationally feasible when the breeding values of the groups are allowed to differ in their means, but not in their variances. RESULTS: We present an extension of the animal model that permits estimation of group-specific additive genetic variances. This is achieved by employing group-specific relatedness matrices for the breeding value components to different genetic groups. We derive these matrices by decomposing the full relatedness matrix via the generalized Cholesky decomposition, and by scaling the respective matrix components for each group. We propose a computationally convenient approximation for the matrix component that encodes for the Mendelian sampling variance, and show that this approximation is not critical. In addition, we explain why segregation variances are often negligible when analyzing the complex polygenic traits that are frequently the focus of evolutionary ecologists and animal breeders. Simulations and an example from an insular meta-population of house sparrows in Norway with three distinct genetic groups illustrate that the method is successful in estimating group-specific additive genetic variances, and that segregation variances are indeed negligible in the empirical example. CONCLUSIONS: Quantifying differences in additive genetic variance within and among populations is of major biological interest in ecology, evolution, and animal and plant breeding. The proposed method allows to estimate such differences for subpopulations that form a connected set of populations, and may thus also be useful to study temporal or spatial variation of additive genetic variances. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12711-019-0449-7) contains supplementary material, which is available to authorized users