We propose a graph-based unsupervised pre-training method for neural architecture encoding.