We show that linear layers, the primary player in large language models, can be composed from fundamental algebraic primitives with exponentially less parameters.