Selection index theory for populations under directional and stabilizing selection

BACKGROUND: The purpose of a selection index is that its use to select animals for breeding maximizes the profit of a breed in future generations. The profit of a breed is in general a quantity that predicts the satisfaction of future owners with their breed, and the satisfaction of the consumers with the products that are produced by the breed. Many traits, such as conformation traits and product quality traits have intermediate optima. Traditional selection index theory applies only to directional selection and cannot achieve any further improvement once the trait means have reached their optima. A well-founded theory is needed that extends the established selection index theory to cover directional as well as stabilizing selection as limiting cases, and that can be applied to maximize the profit of a breed in both situations. RESULTS: The optimum selection index shifts the trait means towards the optima and, in the case of stabilizing selection, decreases the phenotypic variance, which causes the phenotypes to be closer to the optimum. The optimum index depends not only on the breeding values, but also on the squared breeding values, the allele contents of major quantitative trait loci (QTL), the QTL heterozygosities, the inbreeding coefficient of the animal, and the kinship of the animal with the population. CONCLUSION: The optimum selection index drives the alleles of major QTL to fixation when the trait mean approaches the optimum because decreasing the phenotypic variance shifts the trait values closer to the optimum, which increases the profit of the breed. The index weight on the kinship coefficient balances the increased genetic gain that can be achieved in future generations by outcrossing, and the increased genetic gain that can be achieved under stabilizing selection by reducing the phenotypic variance. In a model with dominance variance, it can also account for the effect of inbreeding depression. The combining ability between potential mating partners, which predicts the total merit of their offspring, could become an important parameter for mate allocation that could be used to further shift the phenotypes towards their optimum values. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12711-023-00776-4.