Demographic changes and marker properties affect detection of human population differentiation

BACKGROUND: Differentiating genetically between populations is valuable for admixture and population stratification detection and in understanding population history. This is easy to achieve for major continental populations, but not for closely related populations. It has been claimed that a large marker panel is necessary to reliably distinguish populations within a continent. We investigated whether empirical genetic differentiation could be accomplished efficiently among three Asian populations (Hmong, Thai, and Chinese) using a small set of highly variable markers (15 tetranucleotide and 17 dinucleotide repeats). RESULTS: Hmong could be differentiated from Thai and Chinese based on multi-locus genotypes, but Thai and Chinese were indistinguishable from each other. We found significant evidence for a recent population bottleneck followed by expansion in the Hmong that was not present in the Thai or Chinese. Tetranucleotide repeats were less useful than dinucleotide repeat markers in distinguishing between major continental populations (Asian, European, and African) while both successfully distinguished Hmong from Thai and Chinese. CONCLUSION: Demographic history contributes significantly to robust detection of intracontinental population structure. Populations having experienced a rapid size reduction may be reliably distinguished as a result of a genetic drift -driven redistribution of population allele frequencies. Tetranucleotide markers, which differ from dinucleotide markers in mutation mechanism and rate, are similar in information content to dinucleotide markers in this situation. These factors should be considered when identifying populations suitable for gene mapping studies and when interpreting interpopulation relationships based on microsatellite markers.