We propose a new paradigm to explore the best trade-off through mode connectivity for multi-task learning.