長程決策中的成對偏好強化學習：Markov Decision Contest 的機制與邊界
長程決策中，標量獎勵難以精確定義，而成對偏好方法在理論保證上存在缺口。研究提出的 Markov Decision Contest 模型將求解複雜度證明為 P 類，並提出 HPI 與 HPI-Clip 算法。實測顯示近似算法在多個高維任務中學習效率優於 SPPO，為長程成對偏好決策提供新的邊界與取捨。
Technical
IT industry technical articles covering AWS, Azure, Java, Python, SQL, DevOps, and more.
Pairwise Preference Reinforcement Learning in Long-Horizon Decisions: The Mechanisms and Boundaries of Markov Decision Contest
In long-horizon decision-making, scalar rewards are difficult to define precisely, while pairwise preference methods have gaps in their theoretical guarantees. Markov Decision Contest proves that its exact solution is in P and introduces the HPI and HPI-Clip algorithms. Experiments across seven high-dimensional control tasks show that the approximate HPI-Clip method learns more efficiently than SPPO. The result defines a new set of boundaries and trade-offs for long-horizon decisions based on pairwise preferences.
GMOS：以 3D 幾何錨定動態物件分割的取捨
傳統動態物件分割（MOS）依賴光流等 2D 輔助，缺乏 3D 幾何資訊。GMOS 直接在 RGB 視頻上錨定 3D 時空，提出 MOS-I 逐幀瞬時運動評估協議並建立 GMOS-2K 資料集；論文報告其在所測 MOS、MOS-I 與 UVOS 基準中達到領先結果，並在 Tab. 1 的設定下比先前多物件 MOS 方法約快三倍。
GMOS: The Trade-offs of Anchoring Moving Object Segmentation with 3D Geometry
Traditional moving object segmentation (MOS) relies on 2D aids such as optical flow, without 3D geometric context. GMOS anchors motion directly in RGB video across 3D space and time, introduces the frame-level MOS-I protocol, and builds the GMOS-2K dataset. The paper reports leading results across its evaluated MOS, MOS-I, and UVOS benchmarks, while running about three times faster than earlier multi-object MOS methods under the Tab. 1 evaluation setup.
漸進式非平穩環境中，穩定性何時勝於可塑性？
在漸進式非平穩環境中，強化學習效能衰退的主因是過度適應導致的不穩定性。這篇研究指出，將突觸鞏固應用於多時間尺度的後繼特徵（SFs）可提升表現，但需承擔計算延遲與 SGD 優化器依賴的代價。
When Does Stability Beat Plasticity in Gradually Non-Stationary Environments?
In gradually non-stationary environments, the main cause of reinforcement learning performance decline is instability caused by over-adaptation. This study finds that applying synaptic consolidation to multi-timescale Successor Features (SFs) can improve performance, but at the cost of computational latency and dependence on SGD optimizers.
Slot Machines：LLM 如何追蹤多個實體
Anthropic 研究《Slot Machines》顯示，受測模型在特定複雜句法綁定任務上表現接近隨機，但內部仍透過正交的 current-entity 與 prior-entity 槽位維持資訊秩序。本文解析此機制如何影響我們對模型能力的預期，並指出理解這些邊界比強迫模型處理複雜綁定更具風險控制價值。
Slot Machines: How LLMs Track Multiple Entities
Anthropic’s Slot Machines study shows that, on specific complex syntactic binding tasks, the models tested perform close to random chance while still maintaining an orderly internal representation through orthogonal current-entity and prior-entity slots. This article explains how that mechanism shapes expectations for model capability, and why understanding these boundaries offers more value for risk control than forcing a model to handle complex bindings in a single position.
Post-trained LLM 如何認出自己在說話？
IAS 與 Anthropic 研究者的論文發現，Post-trained 模型生成自身回答時的熵值比閱讀外部文本低 3-4 倍。本文探討「輸入驚喜度」如何驅動這種自我認知機制，並分析其對 AI Agent 穩定性的影響。
How Post-trained LLMs Recognize Their Own Voice
A paper by researchers at IAS and Anthropic finds that post-trained models’ output-distribution entropy is 3–4 times lower when generating their own responses than when reading external text. This article explores how Input Surprise drives this self-recognition mechanism and analyzes its implications for AI agent stability.