Cross-cohort replicability and generalizability of connectivity-based psychometric prediction patterns

An increasing number of studies have investigated the relationships between inter-individual variability in brain regions’ connectivity and behavioral phenotypes, making use of large population neuroimaging datasets. However, the replicability of brain-behavior associations identified by these approaches remains an open question. In this study, we examined the cross-dataset replicability of brain-behavior association patterns for fluid cognition and openness predictions using a previously developed region-wise approach, as well as using a standard whole-brain approach. Overall, we found moderate similarity in patterns for fluid cognition predictions across cohorts, especially in the Human Connectome Project Young Adult, Human Connectome Project Aging, and Enhanced Nathan Kline Institute Rockland Sample cohorts, but low similarity in patterns for openness predictions. In addition, we assessed the generalizability of prediction models in cross-dataset predictions, by training the model in one dataset and testing in another. Making use of the region-wise prediction approach, we showed that first, a moderate extent of generalizability could be achieved with fluid cognition prediction, and that, second, a set of common brain regions related to fluid cognition across cohorts could be identified. Nevertheless, the moderate replicability and generalizability could only be achieved in specific contexts. Thus, we argue that replicability and generalizability in connectivity-based prediction remain limited and deserve greater attention in future studies.