Activation Sparsity

3 papers across 1 session

Poster Session 5

Friday, December 5, 2025 · 11:00 AM → 2:00 PM

Spark Transformer: Reactivating Sparsity in Transformer FFN and Attention

#3710 · Chong You, Kan Wu, Zhipeng Jia, Lin Chen, Srinadh Bhojanapalli, Jiaxian Guo, Utku Evci, Jan Wassenberg, Praneeth Netrapalli, Jeremiah Willcock, Suvinay Subramanian, Felix Chern, Alek Andreev, Shreya Pathak, Felix Yu, Prateek Jain, David Culler, Henry Levy, Sanjiv Kumar

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity

#3513 · Susav Shrestha, Bradley Settlemyer, Nikoli Dryden, Narasimha Reddy

Polar Sparsity scales contextual sparsity to large batches by exploiting stable attention head sparsity and using efficient GPU kernels, achieving up to 2.2× speedups with minimal accuracy loss.

DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs

#4016 · Ruokai Yin, Yuhang Li, Donghyun Lee, Priyadarshini Panda

We propose DuoGPT, a training-free pruning framework that integrates activation sparsity into OBC framework to enable efficient dual-sparse LLM inference with state-of-the-art accuracy–efficiency trade-offs and scalability.