A comparison of marker-based estimators of inbreeding and inbreeding depression

BACKGROUND: The availability of genome-wide marker data allows estimation of inbreeding coefficients (F, the probability of identity-by-descent, IBD) and, in turn, estimation of the rate of inbreeding depression (ΔID). We investigated, by computer simulations, the accuracy of the most popular estimators of inbreeding based on molecular markers when computing F and ΔID in populations under random mating, equalization of parental contributions, and artificially selected populations. We assessed estimators described by Li and Horvitz (F(LH1) and F(LH2)), VanRaden (F(VR1) and F(VR2)), Yang and colleagues (F(YA1) and F(YA2)), marker homozygosity (F(HOM)), runs of homozygosity (F(ROH)) and estimates based on pedigree (F(PED)) in comparison with estimates obtained from IBD measures (F(IBD)). RESULTS: If the allele frequencies of a base population taken as a reference for the computation of inbreeding are known, all estimators based on marker allele frequencies are highly correlated with F(IBD) and provide accurate estimates of the mean ΔID. If base population allele frequencies are unknown and current frequencies are used in the estimations, the largest correlation with F(IBD) is generally obtained by F(LH1) and the best estimator of ΔID is F(YA2). The estimators F(VR2) and F(LH2) have the poorest performance in most scenarios. The assumption that base population allele frequencies are equal to 0.5 results in very biased estimates of the average inbreeding coefficient but they are highly correlated with F(IBD) and give relatively good estimates of ΔID. Estimates obtained directly from marker homozygosity (F(HOM)) substantially overestimated ΔID. Estimates based on runs of homozygosity (F(ROH)) provide accurate estimates of inbreeding and ΔID. Finally, estimates based on pedigree (F(PED)) show a lower correlation with F(IBD) than molecular estimators but provide rather accurate estimates of ΔID. An analysis of data from a pig population supports the main findings of the simulations. CONCLUSIONS: When base population allele frequencies are known, all marker-allele frequency-based estimators of inbreeding coefficients generally show a high correlation with F(IBD) and provide good estimates of ΔID. When base population allele frequencies are unknown, F(LH1) is the marker frequency-based estimator that is most correlated with F(IBD), and F(YA2) provides the most accurate estimates of ΔID. Estimates from F(ROH) are also very precise in most scenarios. The estimators F(VR2) and F(LH2) have the poorest performances. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12711-022-00772-0.