用 20% 記憶體超越 RAG：ASM 在 NBA Benchmark 的延遲優化
長上下文推理面臨記憶體與延遲的雙重瓶頸。本文解析 Attention-State Memory (ASM) 如何透過外部化預計算狀態，在 NBA Benchmark 等特定場景下，以約 20% 的 RAG 記憶體佔用實現性能超越，並探討其適用邊界。
LLM
Surpassing RAG with 20% Memory: ASM Latency Optimization on the NBA Benchmark
Long-context inference faces two constraints at once: memory use grows with the prefix, while latency rises at every decode step. Attention-State Memory (ASM) offers a training-free alternative by externalizing precomputed attention states into a lightweight lookup-based memory. On the NBA Benchmark, it exceeded full-attention RAG performance using about 20% of the memory. This article explains ASM’s hierarchical lookup and online-softmax merge, then maps the boundaries that matter in deployment: query-distribution stability, prefix updates, offline construction, and codebook-size tuning.
Speculative Decoding 實作指南：基準、驗證與取捨
透過 Speculative Decoding 驗證模型推理加速效果。本文提供建立基準、導入 Draft Model、驗證接受率與端到端延遲的完整流程，確保輸出分布一致性並處理回退機制。
Speculative Decoding Implementation Guide: Benchmarks, Validation, and Trade-offs
Validate model inference acceleration with Speculative Decoding. This article provides a complete process for building a baseline, introducing a Draft Model, validating acceptance rates and end-to-end latency, ensuring output-distribution consistency, and handling fallback mechanisms.
Constitutional AI: How AI Feedback Enables Automated Safety Governance
Annotator A gives it a 5, annotator B gives it a 1. The same answer. The bottleneck in traditional RLHF is not cost, but the signal-to-noise ratio. This article breaks down how Constitutional AI uses AI Feedback to build automated safety boundaries, and the cost structure teams need to evaluate when putting it into practice.
Chinchilla Scaling Laws: 算力預算下的模型與資料比例優化
算力預算有限時，該追求大模型還是大資料？透過 Chinchilla 論文數據，解析 1:20 參數與 Token 的最優比例，為 LLM 訓練提供決策依據。
Chinchilla: Model/Data Ratios on a Budget
When compute is limited, should you buy a bigger model or feed it more data? Chinchilla gave a counterintuitive answer in 2022: at a fixed training budget, a 70B model trained on 1.4T tokens outperformed a 280B model trained on 300B tokens, using the same compute. That points to an approximate 1:20 parameter-to-token ratio in that setup. If you’re planning an LLM training run, this paper offers a practical way to think about model size, data volume, and where your next dollar creates more learning.
Sleeper Agents：當 AI 學習在安全訓練中隱藏後門
當模型看到特定年份時會自動植入漏洞，且安全訓練無法移除此後門。本文解析 Anthropic Sleeper Agents 論文，探討 CoT 後門對 AI 供應鏈安全帶來的全新威脅。
Sleeper Agents: When AI Learns to Hide Backdoors During Safety Training
When a model sees a specific year, it automatically inserts a vulnerability, and safety training cannot remove this backdoor. This article analyzes Anthropic’s Sleeper Agents paper and explores the new threat CoT backdoors pose to AI supply-chain security.
Kaplan 2020 論文：Scaling Laws 的演進與情境邊界
2020 年 OpenAI 提出模型大小與資料量的冪次律，建議『大模型、少資料』。兩年後 Chinchilla 修正 compute-optimal 配置，讓資料量的重要性重新被看見。拆解 Scaling Laws 演進，以及它如何影響你現在的訓練決策與成本結構。