PhD student, Indian Institute of Technology, Kanpur
2 papers at NeurIPS 2025
Transformer-based language models learn low-dimensional task manifolds across layers, with similar patterns/trends in intrinsic dimensions revealing similar compression strategies despite varying architectures/sizes.
We propose a new method for interpretating transformer circuit by performing SVD on query-value and value-output matrices
