We perform an important step towards LLM pure FP8 training by enabling stable FP8 dot product attention reaching new throughput records

SageAttention3: Microscaling FP4 Attention for Plug-and-Play Inference Acceleration and An Exploration of 8-Bit Attention for Training.

We propose an information-preserving and data-free pipeline for ultra-efficient delta compression, achieving ultra-high compression ratios and strong performance across various models.

We demonstrate for the first time, fully quantized training of a 7B LLM using FP4 format.

Temporal Generative Flow Networks (Temporal GFNs) offer a novel approach to probabilistic time series forecasting by adapting GFN principles for continuous data.

FALQON accelerates LoRA fine-tuning by up to 3$\times$ through merging adapters into an FP8-quantized backbone, removing redundant quantization overhead from small matrices.

Proposed PMQ-VE, a new quantization method for video enhancement that achieves high performance with low-bit models.

LoTA-QAF uses ternary adaptation for fine-tuning quantized LLMs, enabling the lossless merging of adaptation into quantized weights.

Revisiting layer-wise PTQ, propagating quantization errors enhances existing methods.

We propose a lossless and training-free speculative decoding method to accelerate LLMs that requires offloading on a single memory-limited cosumer GPU.

FPSAttention is a training-aware FP8 quantization and sparsity co-design for video diffusion models that achieves up to 4.96× speedup without quality loss by aligning 3D tile granularity, denoising-step adaptation, and hardware-efficient kernels.

A low-precision scheme for fine-tuning LLMs

We investigate new scaling laws which predict the scaling of LLMs when training them over quantized or sparse representations.