When Models Confidently Produce Errors: How the Value Axis Encodes Generation Trajectories

by
Table of contents

🌏 閱讀中文版本

When Models Confidently Go Wrong: Motivation and Research Question

You send a prompt. The model returns 50 lines of SQL. It looks perfect… Syntax Error.

This is not necessarily a weak prompt, nor does it necessarily mean the model is being deceptive. It confidently produced an incorrect result.

In practice, we often see this pattern: when an LLM produces an incorrect answer, it can express extremely high verbal confidence. Even with a better prompt, the output alone may not reliably tell us whether the model will correct itself.

Jiang, Kauvar, and Lindsey, during their Anthropic Fellows work, recently published The Value Axis: Language Models Encode Whether They’re on the Right Track on arXiv. In Qwen3-8B, they identified a “value axis.” Like in-car navigation, this axis continuously quantifies the model’s assessment of its current generation trajectory: “How far is the path I am taking from the correct answer?”

Note: The SQL example above is for motivation. The paper does not directly show that this kind of error is entirely caused by the value axis. The research primarily compares prefills of correct and manually corrupted code, with a separate evaluation of verbal confidence.

The paper asks a deeper question: Do a language model’s internal states actually encode whether its current generation trajectory is correct?

Put differently, while a model generates text, is its neural network also calculating: “What is the likelihood that the path I am taking will succeed?” If this internal signal exists, we can do more than judge quality from generated text. We may be able to intervene directly in internal states and change its confidence or backtracking behavior.

Core Mechanism: A Projection That Navigates Neural Activations

What is the value axis?
It is a linear direction extracted from neuron activations in a model’s hidden layers. Researchers derive this direction from the residual stream, rather than directly reading intention or consciousness. It is a quantitative projection of the quality of the current generation trajectory.

Key validation results:

  • Code correctness: Across 225 DebugBench tasks, the axis effectively distinguishes correct code from manually corrupted code.
  • Confidence prediction: On AIME problems, the axis predicts when the model believes its answer is correct (AUROC >0.75).
  • Causal evidence: Steering experiments provide causal evidence for selected behaviors.

The direction of this axis is clear:

  • High-value end: Associated with less backtracking, more concise output, fewer lines, comments, and type hints, plus stronger verbal confidence.
  • Low-value end: Associated with more backtracking, exploratory attempts, and longer explanations.

The key is that this does not come from prompting the model to “be confident.” It directly manipulates internal neuron activation vectors. When activations move toward the high-value end, the observed behavior includes less backtracking and explanation. When they move toward the low-value end, more backtracking and exploration appear.

Intervening in Internal States: Steering and DPO

The paper’s most notable observations come from interventions in internal model states, known as steering, and from behavior after Direct Preference Optimization (DPO). The research team reports the following data from Qwen3-8B:

  1. Confidence and concision move together (steering experiment): When the value axis is manually guided toward “high value,” generated code has significantly fewer lines, comments, and type hints. The model becomes more concise and no longer wordily explains its thought process.
  2. Less self-checking (steering experiment): Moving toward the high-value end also substantially lowers the probability of backtracking. Raising internal confidence can make output more decisive, while potentially reducing opportunities for exploration and error discovery.

This is an important signal: DPO training changes more than surface-level output probabilities. It also raises the internal “value score” of certain controlled behaviors. Steering experiments show that when this internal value score is high, models tend to display greater confidence and less backtracking. This gives us a concrete observation point for understanding how alignment training affects internal model states.

Your Judgment: Boundaries and Practical Implications

The discovery of this value axis has meaningful implications for how we understand and use LLMs. It is not a complete answer for every need. It is a coordinate for locating the model’s current state.

Efficiency hypothesis, still to be verified:

  • Scenarios that need efficient output: In code generation or rapid decision tasks, guiding a model into a high-value state can reduce lengthy explanations. This may reduce output length and API-call costs, although the specific effects on latency and cost still need verification. These efficiency metrics also do not establish improved accuracy or calibration.

Unverified considerations in high-reliability settings:

  • Scenarios that need high reliability and self-checking: In code generation related to medicine, law, or critical infrastructure, suppressing backtracking may reduce exploratory behavior. Low-value states include more exploration, but the paper has not verified whether that behavior improves accuracy, safety, or calibration. On this basis, low-value steering cannot yet be recommended for high-stakes domains.

Verifiable Sources and the Paper’s Own Limits

Anthropic’s research provides a powerful tool for understanding LLM internal states, while readers should also note its boundaries.

Limitations identified by the authors:

  • Model dependence: The current research focuses mainly on Qwen3-8B. While DPO effects were observed across several models fine-tuned from the same Qwen3-8B base, the value axis’s exact location and strength may vary by architecture and training data. Generalization across scales has not yet been studied in depth.
  • Limits of synthetic data: The axis was constructed from 300 synthetic ICRL paragraph-rewrite dialogue pairs; code data was used for later correctness and steering evaluations. Whether it remains stable in fully open-domain real conversations still requires careful interpretation.
  • Mechanism origin remains unclear: The paper has not determined whether this mechanism comes from pretraining or post-training, and the current basis for judgment is insufficient. In addition, the axis-construction data uses trajectories simulated by Claude Opus 4.6, which may introduce particular data components.
  • Definitions are not unique: Value belief is not precisely defined. Mean-difference construction is one of several reasonable methods, and the synthetic ICRL direction may include method-specific spurious components.

Next Steps: Applying This in Engineering Practice

Once you understand this axis, here are several directions worth considering for adjusting your workflow. Most remain research directions or prototype-validation work, so production deployment needs the necessary assumptions and validation metrics.

  1. Add value monitoring to CI/CD, a research agenda to validate: For critical tasks, you could monitor changes in the “value vector” during generation. If a model suddenly enters a high-value state at a critical step and stops backtracking, that may trigger a second validation pass. Required validation: reconstruct the axis per model, calibrate it against real task outcomes, and measure false-positive and false-negative rates. (Note: This approach applies only to self-hosted models where the residual stream is accessible. Standard hosted APIs usually cannot implement it directly.)
  2. Enable low-value states on critical paths, a research agenda to validate: In settings that need extensive self-checking, try guiding internal states toward a low-value range so the model remains in an “exploration mode” rather than a “confidence mode.” Note that this cannot be achieved by adjusting temperature. It requires specific activation steering. Required validation: the effect of steering on accuracy and a fallback mechanism.
  3. Reassess DPO training objectives, a research hypothesis: The study currently tested only 50 models trained on synthetic lexical-preference data, along with changes in related expression volume. Whether preference data or formal training objectives should change still requires comparisons across backtracking rate, accuracy, and calibration.

This is not simply a binary choice between right and wrong. It is a choice about configuring contexts. Efficiency and safety have never been all-or-nothing. In your workflow, how will you balance efficiency, self-checking, and verifiability?

Sources