Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance

[Submitted on 21 Sep 2026 (v1), last revised 9 Oct 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Best-of-$N$ is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths. However, post-trained reasoning models often suffer from emph{exploration collapse}, where independent rollouts repeatedly follow similar reasoning paths and limit the gains from increasing the rollout budget. Existing methods alleviate this issue by promoting broader exploration, but do not explicitly guide exploration toward continuations that make meaningful progress, resulting in limited exploration efficiency. To address this, we propose emph{underline{S}tate-conditioned underline{P}rogress-guided underline{S}teering} (SPS), a training-free latent steering framework. Specifically, SPS constructs a state-conditioned Direction Bank containing multiple progress-guided steering vectors for different prefix-state regions. During online inference, SPS retrieves a suitable steering vector based on the current prefix state and applies it at high-uncertainty transitions to guide the next reasoning step toward meaningful progress. Extensive experiments across multiple model scales and benchmarks demonstrate that SPS consistently outperforms strong baselines. Further analyses validate the effectiveness of its key designs and offer valuable insights for future research. The code is available at this https URL.

Submission history

From: Hengyuan Zhang [view email]
[v1] Mon, 21 Sep 2026 03:41:40 UTC (651 KB)
[v2] Fri, 9 Oct 2026 05:32:58 UTC (651 KB)