We prove that transformers can achieve nearly optimal dynamic regret bounds in non-stationary environments.

We frame dynamic regret minimization as a static regret problem in an RKHS