Uncovering and mitigating bias in large, automated MRI analyses of brain development

Large, population-based MRI studies of adolescents promise transformational insights into neurodevelopment and mental illness risk (1,2). However, MRI studies of youth are especially susceptible to motion and other artifacts (3,4). These artifacts may go undetected by automated quality control (QC) methods that are preferred in high-throughput imaging studies, 5 and can potentially introduce non-random noise into clinical association analyses. Here we demonstrate bias in structural MRI analyses of children due to inclusion of lower quality images, as identified through rigorous visual quality control of 11,263 T1 MRI scans obtained at age 9–10 through the Adolescent Brain Cognitive Development (ABCD) Study6. Compared to the best-rated images (44.9% of the sample), lower-quality images generally associated with decreased cortical thickness and increased cortical surface area measures (Cohen’s d 0.14–2.84). Variable image quality led to counterintuitive patterns in analyses that associated structural MRI and clinical measures, as inclusion of lower-quality scans altered apparent effect sizes in ways that increased risk for both false positives and negatives. Quality-related biases were partially mitigated by controlling for surface hole number, an automated index of topological complexity that differentiated lower-quality scans with good specificity at Baseline (0.81–0.93) and in 1,000 Year 2 scans (0.88–1.00). However, even among the highest-rated images, subtle topological errors occurred during image preprocessing, and their correction through manual edits significantly and reproducibly changed thickness measurements across much of the cortex (d 0.15–0.92). These findings demonstrate that inadequate QC of youth structural MRI scans can undermine advantages of large sample size to detect meaningful associations.