A Reproducible Benchmark for Leakage-Resistant Machine Learning on Multi-Site Brain Connectivity Graphs

Background/Objectives: Machine-learning studies of autism spectrum disorder from resting-state functional magnetic resonance imaging routinely report near-perfect accuracy, yet the models rarely transfer to an unseen scanner. We audited a pipeline of the kind this literature builds, which reported an accuracy of 0.999, and traced that number to five data-leakage pathways.

Methods

Under a nested leave-one-site-out protocol on 871 participants from the Autism Brain Imaging Data Exchange, in which the held-out site influenced nothing but model selection, the threshold, the combiner, or the site correction, we closed every pathway, priced each by reopening it, and validated on a locked external cohort.