Single-cell and Spatial Transcriptomics Clustering with an Optimized Adaptive K-Nearest Neighbor Graph

Single-cell and spatial transcriptomics have been widely used to characterize cellular landscape in complex tissues. To understand cellular heterogeneity, one essential step is to define cell types through unsupervised clustering. While typical clustering methods have difficulty in identifying rare cell types, approaches specifically tailored to detect rare cell types gain their ability at the cost of poorer performance for grouping abundant ones. Here, we developed aKNNO, a method to identify abundant and rare cell types simultaneously based on an adaptive k-nearest neighbor graph with optimization. Benchmarked on 38 simulated and 20 single-cell and spatial transcriptomics datasets, aKNNO identified both abundant and rare cell types accurately. Without sacrificing performance for clustering abundant cell types, aKNNO discovered known and novel rare cell types that those typical and even specifically tailored methods failed to detect. aKNNO, using transcriptome alone, stereotyped fine-grained anatomical structures more precisely than those integrative approaches combining expression with spatial locations and histology image.