We study concept shift in ridge regression in the thermodynamic limit, and found a non-monotonic data dependence of test performance that generalizes to realistic machine learning tasks (MNIST classification and transformer ICL regression).