Adaptive Multi-Horizon RL：狀態依賴閘道與固定 γ 的工程取捨
研究團隊提出狀態依賴閘道網路，讓智能體在持續學習中自動權重不同時間視角。透過 Expected SARSA 與 TD error 優化，成功在 MiniGrid 任務間切換時保持穩定回報（36–40），降低手動選擇單一折現率的工程負擔。此方法高度依賴狀態特徵的資訊密度以區分時間結構。
Reinforcement Learning
Adaptive Multi-Horizon RL: Engineering Tradeoffs of State-Dependent Gating and Fixed γ
A research team proposes a state-dependent gating network that lets an agent automatically weight different temporal perspectives during continual learning. Using Expected SARSA and TD-error optimization, the architecture maintained stable episode returns of 36–40 while switching between MiniGrid tasks, reducing the engineering overhead of manually choosing one discount factor. The key constraint is state-feature information density: the system must distinguish temporal structures.
長程決策中的成對偏好強化學習：Markov Decision Contest 的機制與邊界
長程決策中，標量獎勵難以精確定義，而成對偏好方法在理論保證上存在缺口。研究提出的 Markov Decision Contest 模型將求解複雜度證明為 P 類，並提出 HPI 與 HPI-Clip 算法。實測顯示近似算法在多個高維任務中學習效率優於 SPPO，為長程成對偏好決策提供新的邊界與取捨。
Pairwise Preference Reinforcement Learning in Long-Horizon Decisions: The Mechanisms and Boundaries of Markov Decision Contest
In long-horizon decision-making, scalar rewards are difficult to define precisely, while pairwise preference methods have gaps in their theoretical guarantees. Markov Decision Contest proves that its exact solution is in P and introduces the HPI and HPI-Clip algorithms. Experiments across seven high-dimensional control tasks show that the approximate HPI-Clip method learns more efficiently than SPPO. The result defines a new set of boundaries and trade-offs for long-horizon decisions based on pairwise preferences.
漸進式非平穩環境中，穩定性何時勝於可塑性？
在漸進式非平穩環境中，強化學習效能衰退的主因是過度適應導致的不穩定性。這篇研究指出，將突觸鞏固應用於多時間尺度的後繼特徵（SFs）可提升表現，但需承擔計算延遲與 SGD 優化器依賴的代價。
When Does Stability Beat Plasticity in Gradually Non-Stationary Environments?
In gradually non-stationary environments, the main cause of reinforcement learning performance decline is instability caused by over-adaptation. This study finds that applying synaptic consolidation to multi-timescale Successor Features (SFs) can improve performance, but at the cost of computational latency and dependence on SGD optimizers.
Post-trained LLM 如何認出自己在說話？
IAS 與 Anthropic 研究者的論文發現，Post-trained 模型生成自身回答時的熵值比閱讀外部文本低 3-4 倍。本文探討「輸入驚喜度」如何驅動這種自我認知機制，並分析其對 AI Agent 穩定性的影響。
How Post-trained LLMs Recognize Their Own Voice
A paper by researchers at IAS and Anthropic finds that post-trained models’ output-distribution entropy is 3–4 times lower when generating their own responses than when reading external text. This article explores how Input Surprise drives this self-recognition mechanism and analyzes its implications for AI agent stability.
RLHF 的代價：InstructGPT 如何靠對齊而非規模，讓 1.3B 打敗 175B 的 GPT-3
175B 的 GPT-3 輸給了 1.3B 的 InstructGPT。這不是參數的勝利，是『聽話』的勝利。但聽話的代價，是你在部分任務上可能付出能力取捨。本文拆解 RLHF 機制與對齊稅，說明如何在可控性與通用性之間做出取捨。
The Cost of RLHF: Why 1.3B Beat 175B
GPT-3 at 175B lost to InstructGPT at 1.3B. This was not a win for parameter count. It was a win for instruction following. But that gain comes with a tradeoff: when you push a model to stay closer to human preference, you may also narrow its performance on some public NLP tasks. This article breaks down how RLHF works through SFT, reward models, and PPO, then looks at the alignment tax and what it means when you have to choose between controllability and broader capability.