This paper introduces PILS, a novel language model inversion method that leverages the low-dimensionality of next-token distributions, enabling their lossless compression over multiple generation steps for markedly improved prompt recovery.