The token embeddings for LLMs are unlikely to be manifolds with low curvature, based upon a novel rigorous statistical hypothesis test.

We train an energy-based model on image datasets through a dual score matching objective and analyze the local geometry of the learned energy landscape.

We show that diffusion models adapt to low-dimensional data geometry as a result of how approximations are formed at the level of the score function. We show that smoothing the score function naturally produces smoothing along the data manifold.