We propose UtilGen, a novel utility-centric data augmentation framework which adaptively optimizes data generation based on downstream task feedback to produce high-utility training data.