Fractional coalescent

An approach to the coalescent, the fractional coalescent ([Formula: see text]-coalescent), is introduced. The derivation is based on the discrete-time Cannings population model in which the variance of the number of offspring depends on the parameter [Formula: see text]. This additional parameter [Formula: see text] affects the variability of the patterns of the waiting times; values of [Formula: see text] lead to an increase of short time intervals, but occasionally allow for very long time intervals. When [Formula: see text] , the [Formula: see text]-coalescent and the Kingman’s [Formula: see text]-coalescent are equivalent. The distribution of the time to the most recent common ancestor and the probability that [Formula: see text] genes descend from [Formula: see text] ancestral genes in a time interval of length [Formula: see text] for the [Formula: see text]-coalescent are derived. The [Formula: see text]-coalescent has been implemented in the population genetic model inference software Migrate. Simulation studies suggest that it is possible to accurately estimate [Formula: see text] values from data that were generated with known [Formula: see text] values and that the [Formula: see text]-coalescent can detect potential environmental heterogeneity within a population. Bayes factor comparisons of simulated data with [Formula: see text] and real data (H1N1 influenza and malaria parasites) showed an improved model fit of the [Formula: see text]-coalescent over the [Formula: see text]-coalescent. The development of the [Formula: see text]-coalescent and its inclusion into the inference program Migrate facilitates testing for deviations from the [Formula: see text]-coalescent.