Effects of marker type and filtering criteria on Q(ST)-F(ST) comparisons

Comparative studies of quantitative and neutral genetic differentiation (Q(ST)-F(ST) tests) provide means to detect adaptive population differentiation. However, Q(ST)-F(ST) tests can be overly liberal if the markers used deflate F(ST) below its expectation, or overly conservative if methodological biases lead to inflated F(ST) estimates. We investigated how marker type and filtering criteria for marker selection influence Q(ST)-F(ST) comparisons through their effects on F(ST) using simulations and empirical data on over 18 000 in silico genotyped microsatellites and 3.8 million single-locus polymorphism (SNP) loci from four populations of nine-spined sticklebacks (Pungitius pungitius). Empirical and simulated data revealed that F(ST) decreased with increasing marker variability, and was generally higher with SNPs than with microsatellites. The estimated baseline F(ST) levels were also sensitive to filtering criteria for SNPs: both minor alleles and linkage disequilibrium (LD) pruning influenced F(ST) estimation, as did marker ascertainment. However, in the case of stickleback data used here where Q(ST) is high, the choice of marker type, their genomic location, ascertainment and filtering made little difference to outcomes of Q(ST)-F(ST) tests. Nevertheless, we recommend that Q(ST)-F(ST) tests using microsatellites should discard the most variable loci, and those using SNPs should pay attention to marker ascertainment and properly account for LD before filtering SNPs. This may be especially important when level of quantitative trait differentiation is low and levels of neutral differentiation high.