LLM Quantization

3 papers across 2 sessions

Poster Session 2

2 papers

Wednesday, December 3, 2025 · 4:30 PM → 7:30 PM

Exhibit Hall C,D,E

DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment

#4108 · Sangwoo Kwon, Seong Hoon Seo, Jae W. Lee, Yeonhong Park

RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models

#3910 · Zukang Xu, Xing Hu, Qiang Wu, Dawei Yang

Poster Session 6

1 paper

Friday, December 5, 2025 · 4:30 PM → 7:30 PM

Exhibit Hall C,D,E

Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment

#3411 · DEOKJAE LEE, Hyun Oh Song

We develop Q-Palette, a quantizer suite with efficient inference CUDA kernels and wide fractional-bit support, enabling mixed-scheme quantization that achieves ~36% faster LLM decoding than NormalFloat while improving accuracy.