[Submitted on 23 Mar 2026 (v1), last revised 29 Oct 2026 (this version, v4)]
Abstract:Large reasoning models (LRMs) often suffer from overthinking, continuing with redundant verification and repeated attempts even after reaching a correct solution. This wastes computation and can overturn an initially correct answer. We show that the transition from productive to redundant reasoning is decodable from late-layer hidden states. These representations separate the two phases around first-correct-solution boundaries beyond what position and lexical controls explain, and distinguish correct from incorrect solution endings. Building on this finding, we propose ROM (Real-time Overthinking Mitigation), which uses a lightweight streaming detector (0.07–0.25% of backbone parameters) to monitor a frozen LRM and intervene at well-formed reasoning boundaries without answer extraction or probe decoding. Counterfactual Self-Correction (CSC) augments detector training with wrong$rightarrow$correct trajectories to preserve useful self-correction. Heads are trained separately for each backbone using shared MATH500 supervision and transfer zero-shot to four unseen benchmarks. Evaluated against ten baselines across five backbones from three families and five benchmarks, ROM$_{text{CSC}}$ reduces response length by 28–77% (45% on average), with significant reductions in all 25 settings and accuracy matching or exceeding vanilla decoding in 22. Measurements on two backbones further show 20–70% lower end-to-end latency with less than 0.5% detector overhead per decoding step. Code is available at this https URL.
Submission history
From: Xinyan Wang [view email]
[v1] Mon, 23 Mar 2026 14:26:57 UTC (6,373 KB)
[v2] Sat, 9 May 2026 22:19:27 UTC (936 KB)
[v3] Mon, 10 Aug 2026 05:45:00 UTC (343 KB)
[v4] Thu, 29 Oct 2026 01:29:42 UTC (770 KB)
