Tim

Slot Machines：LLM 如何追蹤多個實體

by

Anthropic 研究《Slot Machines》顯示，受測模型在特定複雜句法綁定任務上表現接近隨機，但內部仍透過正交的 current-entity 與 prior-entity 槽位維持資訊秩序。本文解析此機制如何影響我們對模型能力的預期，並指出理解這些邊界比強迫模型處理複雜綁定更具風險控制價值。

Slot Machines: How LLMs Track Multiple Entities

by

Anthropic’s Slot Machines study shows that, on specific complex syntactic binding tasks, the models tested perform close to random chance while still maintaining an orderly internal representation through orthogonal current-entity and prior-entity slots. This article explains how that mechanism shapes expectations for model capability, and why understanding these boundaries offers more value for risk control than forcing a model to handle complex bindings in a single position.

How Post-trained LLMs Recognize Their Own Voice

by

A paper by researchers at IAS and Anthropic finds that post-trained models’ output-distribution entropy is 3–4 times lower when generating their own responses than when reading external text. This article explores how Input Surprise drives this self-recognition mechanism and analyzes its implications for AI agent stability.

SAE Feature Extraction: Seeing Model Representations Through the Residual Stream

by

Anthropic’s research team trained sparse autoencoders (SAEs) with up to 34 million features in Claude 3 Sonnet, extracting interpretable feature representations. The research shows causal effects on model output behavior, providing internal signals and potential tools that can be studied for AI safety governance, though faithfulness still requires validation.