We propose a new plug-and-play optimization method from the perspective of symmetry teleportation for mainstream multi-task learning approaches.