Speculative Decoding

透過 Speculative Decoding 驗證模型推理加速效果。本文提供建立基準、導入 Draft Model、驗證接受率與端到端延遲的完整流程，確保輸出分布一致性並處理回退機制。

Validate model inference acceleration with Speculative Decoding. This article provides a complete process for building a baseline, introducing a Draft Model, validating acceptance rates and end-to-end latency, ensuring output-distribution consistency, and handling fallback mechanisms.