Validation of cross-sectional and longitudinal ComBat harmonization methods for magnetic resonance imaging data on a travelling subject cohort

BACKGROUND: The growth in multi-center neuroimaging studies generated a need for methods that mitigate the differences in hardware and acquisition protocols across sites i.e., scanner effects. ComBat harmonization methods have shown promise but have not yet been tested on all the data types commonly studied with magnetic resonance imaging (MRI). This study aimed to validate neuroCombat, longCombat and gamCombat on both structural and diffusion metrics in both cross-sectional and longitudinal data. METHODS: We used a travelling subject design whereby 73 healthy volunteers contributed 161 scans across two sites and four machines using one T1 and five diffusion MRI protocols. Scanner was defined as a composite of site, machine and protocol. A common pipeline extracted two structural metrics (volumes and cortical thickness) and two diffusion tensor imaging metrics (mean diffusivity and fractional anisotropy) for seven regions of interest including gray and (except for cortical thickness) white matter regions. RESULTS: Structural data exhibited no significant scanner effect and therefore did not benefit from harmonization in our particular cohort. Indeed, attempting harmonization obscured the true biological effect for some regions of interest. Diffusion data contained marked scanner effects and was successfully harmonized by all methods, resulting in smaller scanner effects and better detection of true biological effects. LongCombat less effectively reduced the scanner effect for cross-sectional white matter data but had a slightly lower probability of incorrectly finding group differences in simulations, compared to neuroCombat and gamCombat. False positive rates for all methods and all metrics did not significantly exceed 5%. CONCLUSIONS: Statistical harmonization of structural data is not always necessary and harmonization in the absence of a scanner effect may be harmful. Harmonization of diffusion MRI data is highly recommended with neuroCombat, longCombat and gamCombat performing well in cross-sectional and longitudinal settings.