Our work shows that in online convex optimization over lp-balls (p>2), anytime optimality can be achieved with Follow-the-Regularized-Leader using adaptive regularization, and that for separable regularizers this adaptivity is necessary.