We revisit the orbital minimization method (OMM) from computational chemistry and demonstrate its utility as a simple, principled, and scalable approach for modern machine learning.