Evolutionary dynamics in the two-locus two-allele model with weak selection

Two-locus two-allele models are among the most studied models in population genetics. The reason is that they are the simplest models to explore the role of epistasis for a variety of important evolutionary problems, including the maintenance of polymorphism and the evolution of genetic incompatibilities. Many specific types of models have been explored. However, due to the mathematical complexity arising from the fact that epistasis generates linkage disequilibrium, few general insights have emerged. Here, we study a simpler problem by assuming that linkage disequilibrium can be ignored. This is a valid approximation if selection is sufficiently weak relative to recombination. The goal of our paper is to characterize all possible equilibrium structures, or more precisely and general, all robust phase portraits or evolutionary flows arising from this weak-selection dynamics. For general fitness matrices, we have not fully accomplished this goal, because some cases remain undecided. However, for many specific classes of fitness schemes, including additive fitnesses, purely additive-by-additive epistasis, haploid selection, multilinear epistasis, marginal overdominance or underdominance, and the symmetric viability model, we obtain complete characterizations of the possible equilibrium structures and, in several cases, even of all possible phase portraits. A central point in our analysis is the inference of the number and stability of fully polymorphic equilibria from the boundary flow, i.e., from the dynamics at the four marginal single-locus subsystems. The key mathematical ingredient for this is index theory. The specific form of epistasis has both a big influence on the possible boundary flows as well as on the internal equilibrium structure admitted by a given boundary flow. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s00285-017-1140-7) contains supplementary material, which is available to authorized users.