Nojun Kwak

We analyze the reasons behind Differential Transformer's success, based on which we propose an efficient adaptation method to enhance pretrained LLMs.

We introduce the Deep Edge Filter, which improves model generalizability by applying high-pass filtering to neural network features based on our hypothesis that semantic information resides in high-frequency components.