Multiple episodes of interbreeding between Neanderthals and modern humans

Neandertals and anatomically modern humans overlapped geographically for a period of over 30,000 years following human migration out of Africa. During this period, Neandertals and humans interbred, as evidenced by Neandertal portions of the genome carried by non-African individuals today. A key observation is that the proportion of Neandertal ancestry is ~12–20% higher in East Asian individuals relative to European individuals. Here, we explore various demographic models that could explain this observation. These include distinguishing between a single admixture event and multiple Neandertal contributions to either population, and the hypothesis that reduced Neandertal ancestry in modern Europeans resulted from more recent admixture with a ghost population that lacked a Neandertal ancestry component (the “dilution” hypothesis). In order to summarize the asymmetric pattern of Neandertal allele frequencies, we compile the joint fragment frequency spectrum (FFS) of European and East Asian Neandertal fragments and compare it to both analytical theory and data simulated under various models of admixture. Using maximum likelihood and machine learning, we found that a simple model of a single admixture does not fit the empirical data and instead favor a model of multiple episodes of gene flow into both European and East Asian populations. These findings indicate more long-term, complex interaction between humans and Neandertals than previously appreciated.