An Improved F(st) Estimator

The fixation index F (st) plays a central role in ecological and evolutionary genetic studies. The estimators of Wright ([Image: see text] ), Weir and Cockerham ([Image: see text] ), and Hudson et al. ([Image: see text] ) are widely used to measure genetic differences among different populations, but all have limitations. We propose a minimum variance estimator [Image: see text] using [Image: see text] and [Image: see text] . We tested [Image: see text] in simulations and applied it to 120 unrelated East African individuals from Ethiopia and 11 subpopulations in HapMap 3 with 464,642 SNPs. Our simulation study showed that [Image: see text] has smaller bias than [Image: see text] for small sample sizes and smaller bias than [Image: see text] for large sample sizes. Also, [Image: see text] has smaller variance than [Image: see text] for small F (st) values and smaller variance than [Image: see text] for large F (st) values. We demonstrated that approximately 30 subpopulations and 30 individuals per subpopulation are required in order to accurately estimate F (st).