Guoren Wang

We propose Fira, a plug-and-play memory-efficient training framework of LLMs to enable full-rank training consistently under the low-rank constraint.

This paper explores the generalization mechanism of KANs and designs more effective KANs with lower model complexity and better generalization.