Joan Bruna

We propose a toy model that shows how linear truth encodings can arise in language models.

We show that solving compositional reasoning problems requires transformers, RNNs, or CoT-augmented transformers to scale specific hyperparameters with input size, revealing distinct strengths and trade-offs across architectures.

We define a "generative leap" exponent which tightly captures the sample complexity for efficiently learning Gaussian multi-index models.