We propose a data augmentation method for zero-shot generalization in reinforcement learning and provide a theoretical and empirical analysis of the method.