Chun-Che Yang

We propose a lossless and training-free speculative decoding method to accelerate LLMs that requires offloading on a single memory-limited cosumer GPU.