Speculative Decoding 實作指南：基準、驗證與取捨
透過 Speculative Decoding 驗證模型推理加速效果。本文提供建立基準、導入 Draft Model、驗證接受率與端到端延遲的完整流程，確保輸出分布一致性並處理回退機制。
Machine Learning
Speculative Decoding Implementation Guide: Benchmarks, Validation, and Trade-offs
Validate model inference acceleration with Speculative Decoding. This article provides a complete process for building a baseline, introducing a Draft Model, validating acceptance rates and end-to-end latency, ensuring output-distribution consistency, and handling fallback mechanisms.
Decoupled DiLoCo: How Decoupling Improves Resilience in Large-Scale Distributed Pre-training
Large language model pre-training can pause when hardware latency or node outages propagate through strict synchronization points. Decoupled DiLoCo uses independent Learners and asynchronous integration, allowing training to continue in simulated environments with millions of chips. Under the paper’s specified fault model, it reports zero global downtime and sustains 88% Goodput. The result is a more resilient architectural option for distributed training.
Decoupled DiLoCo：解耦機制如何優化大規模分散式預訓練的韌性
大型語言模型預訓練常因硬體延遲或故障導致整體停擺。該論文提出 Decoupled DiLoCo，透過解耦 Learner 與異步整合機制，在模擬百萬級晶片環境中實現零停機時間，並將 Goodput 維持在 88%，為分散式訓練提供更具韌性的架構選擇。
Chinchilla Scaling Laws: 算力預算下的模型與資料比例優化
算力預算有限時，該追求大模型還是大資料？透過 Chinchilla 論文數據，解析 1:20 參數與 Token 的最優比例，為 LLM 訓練提供決策依據。
Chinchilla: Model/Data Ratios on a Budget
When compute is limited, should you buy a bigger model or feed it more data? Chinchilla gave a counterintuitive answer in 2022: at a fixed training budget, a 70B model trained on 1.4T tokens outperformed a 280B model trained on 300B tokens, using the same compute. That points to an approximate 1:20 parameter-to-token ratio in that setup. If you’re planning an LLM training run, this paper offers a practical way to think about model size, data volume, and where your next dollar creates more learning.
超越 Chinchilla：如何在訓練與推論成本之間取得平衡？
當推論需求極高時，遵循 Chinchilla 法則反而更貴。本文解析如何透過「小模型、大資料」策略，在訓練與推論的總成本中找到新的平衡點。
Beyond Chinchilla: How Do You Balance Training and Inference Costs?
When inference demand is extremely high, following the Chinchilla rule can actually be more expensive. This article explains how a “small model, more data” strategy can find a new balance point in the total cost of training and inference.