This paper presents a layer-parallel speculation strategy for optimizing the effiency of multi-GPU utilization during the drafting stage of speculative decoding.

We propose EAGLE-3, observing that it can benefit from scaling up the data.

In this paper, we propose a novel Cascade Adaptive Self-Speculative Decoding (CAS-Spec) algorithm which constructs speculative draft models by leveraging dynamically switchable inference acceleration (DSIA) strategies

Automatically detecting task-specific important tokens to accelerate speculative decoding

Substantially faster diffusion LLMs using a small auxiliary autoregressive model

A model-free speculative decoding method that accelerates agentic AI workloads using suffix trees. Achieves 5.3x speedup on multi-agent tasks.

We introduce a technique for coupling probability distributions when several samples are available from one of the distributions, and give applications to multi-draft speculative decoding and distributed lossy compression with side information.

Enhancing cost efficiency in LLM serving through an edge-assisted speculative decoding framework.

ViSpec accelerates vision-language model inference by integrating vision-aware speculative decoding with compressed image tokens and global feature injection, achieving up to 3.22× speedup.

We enable tree-based decoding on SSMs to facilitate speculative decoding with tree-based verification with SSMs

We propose a lossless and training-free speculative decoding method to accelerate LLMs that requires offloading on a single memory-limited cosumer GPU.

TPP-SD is a fast and distributionally consistent sampling framework for Transformer-based temporal point processes, achieving 2–6× speedup over autoregressive methods by adapting speculative decoding with a lightweight draft model.

We develop a novel speculative decoding framework for protein generation, using structure aware guidance from k-mers to generate proteins with higher likelihood and structural confidence.