Inferring biological kinship in ancient datasets: comparing the response of ancient DNA-specific software packages to low coverage data

BACKGROUND: The inference of biological relations between individuals is fundamental to understanding past human societies. Caregiving, resource sharing and sexual behaviours are often mediated by biological kinship and yet the identification and interpretation of kin relationships in prehistoric human groups is difficult. In recent years, the advent of archaeogenetic techniques have offered a fresh approach, and when combined with more traditional osteological and interpretive archaeological methods, allows for improved interpretation of the burial practices, cultural behaviours, and societal stratification in ancient societies. Although archaeogenetic techniques are developing at pace, questions remain as to their accuracy, particularly when applied to the low coverage datasets that results from the sequencing of DNA derived from highly degraded ancient material. RESULTS: The performance of six of the most commonly used kinship identifcation software methods was explored at a range of low and ultra low genome coverages. An asymmetrical response was observed across packages, with decreased genome coverage resulting in differences in both direction and degree of change of calculated kinship scores and thus pairwise relatedness estimates are dependant on both package used and genome coverage. Methods reliant upon genotype likelihoods methods (lcMLkin, NGSrelate and NGSremix) show a decreased level of prediction at coverage below 1x, although were consistent in the particular relationships identified at these coverages when compared to the pseudohaploid reliant methods tested (READ, the Kennett 2017 method and TKGWV2.0). The three pseudohaploid methods show predictive potential at coverages as low as 0.05x, although the accuracy of the relationships identified is questionable given the increase in the number of relationships identifIed at the low coverage (type I errors). CONCLUSION: Two pseudohaploid methods (READ and Kennett 2017) show relatively consistent inference of kin relationships at low coverage (0.5x), with READ only showing a significant performance drop off at ultralow coverages (< 0.2x). More generally, our results reveal asymmetrical kinship classifications in some software packages even at high coverages, highlighting the importance of applying multiple methods to authenticate kin relationships in ancient material, along with the continuing need to develop laboratory methods that maximise data output for downstream analyses. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12864-023-09198-4.