second order optimization; scaling laws; maximum update paramterization; batch size scaling; depth scaling; critical batch size; compute optimal scaling

1 paper across 1 session

Poster Session 2

1 paper

Wednesday, December 3, 2025 · 4:30 PM → 7:30 PM

Exhibit Hall C,D,E

How to Scale Second-Order Optimization

#5309 · Charlie Chen, Shikai Qiu, Hoang Phan, Qi Lei, Andrew Wilson

We investigate how to scale second-order optimizers effectively, showing they outperform Adam and reduce data needs in compute-optimal transformer training.