artificialintelbulletin.com

Tag: cs.AI

  • Splicing Hints into Reasoning Traces: Learning from Demonstrator Failures

    Splicing Hints into Reasoning Traces: Learning from Demonstrator Failures

    [Submitted on 26 Sep 2026]

    View PDF HTML (experimental)

    Abstract:Reinforcement learning with verifiable rewards improves mathematical reasoning when a final answer can be checked, but a correct answer does not certify the derivation behind it, and proof problems offer no answer to check. Replacing the check with a reasoning model as an in-loop verifier faces a fundamental obstacle: the policy must already produce occasional correct attempts for any reward to exist, which it may never do at olympiad level. We instead move verification and repair out of the training loop and into offline data generation with SPLICE, a workflow in which a second model, given only a reference solution, edits a reasoning model’s traces as they are generated: it verifies progress, truncates a failed trace at its last verified step, and splices in an adaptively strengthened hint until progress resumes. The resulting dataset includes repaired examples whose sampled unaided trajectory failed, with failed continuations removed and progress repeatedly checked against the reference solution. A model finetuned on SPLICE traces outperforms the same model distilled from the reasoning model’s unedited traces on AIME and, under a grading protocol validated against human grades, on olympiad proofs. The trained model’s own traces show a corresponding behavioral signature: at difficult points it announces a branch in the style of the spliced hints and takes riskier steps, with predictive entropy rising there and just after, a pattern absent from the model trained on unedited traces.

    Submission history

    From: Hamed Mahdavi [view email]
    [v1] Sat, 26 Sep 2026 23:28:04 UTC (149 KB)

  • Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents

    Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents

    [Submitted on 26 Sep 2026]

    View PDF HTML (experimental)

    Abstract:Self-evolving agents improve through repeated cycles of modification and evaluation. However, existing methods receive feedback on modifications only after costly evaluations, and aggregate scores may obscure distinctions between failures that require different repairs. We introduce predictive self-evolution through an editable Improvement Dynamics Model (IDM). Before evaluation, the IDM predicts task-level outcomes of a proposed modification and guides an optional revision. After evaluation, past modifications and their observed outcomes are used to improve the IDM, allowing predictive knowledge to accumulate across generations. The system can thus improve how it solves tasks, proposes modifications, and predicts their effects. Building on HyperAgents, we evaluate this approach across multilingual coding, paper review, robotics reward design, and Olympiad-level mathematical grading. Our approach outperforms HyperAgents across all four domains, with up to 46% higher average performance over generations. Gains also persist under matched token budgets that include both agent evolution and task evaluation, and extend to held-out tasks. Forecast accuracy improves as the IDM evolves, and the learned predictive knowledge transfers to improve evolution in a new domain.

    Submission history

    From: Zixuan Xie [view email]
    [v1] Sat, 26 Sep 2026 18:59:47 UTC (2,245 KB)

  • Activation Steering with Jungian Cognitive Functions

    Activation Steering with Jungian Cognitive Functions

    [Submitted on 23 Jul 2026 (v1), last revised 13 Oct 2026 (this version, v3)]

    View PDF HTML (experimental)

    Abstract:Controlling combinations of personality-related tendencies is important for adapting large language models to diverse interaction needs. But successful single-trait steering does not establish how to construct interventions for joint targets, or whether existing trait directions adequately represent them. We introduce a target-first framework that directly extracts steering directions from joint personality targets and then examines their decomposition into individual directions. To support this framework, we construct NarrationDB, linking character self-narrations to automatically evaluated joint scores across eight Jungian-inspired functions, a correspondence not directly available in existing Big Five- or MBTI-oriented resources. Experiments on three models demonstrate questionnaire-score controllability across the eight functions and coordinated changes in a tested two-function target. Directly extracted joint directions are better approximated by their corresponding individual directions than by unrelated pairings, but retain substantial residual components. Our work provides a method and resource for investigating joint personality control, with evidence that individual steering directions capture part, but not all, of the structure of joint targets.

    Submission history

    From: Zai Liu [view email]
    [v1] Thu, 23 Jul 2026 00:18:39 UTC (334 KB)
    [v2] Fri, 24 Jul 2026 14:33:17 UTC (1 KB) (withdrawn)
    [v3] Tue, 13 Oct 2026 12:02:48 UTC (874 KB)

  • inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

    inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

    [Submitted on 13 Oct 2026]

    View PDF HTML (experimental)

    Abstract:This work investigates the application of the Membership Inference Test (MINT) to large language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about data privacy, leakage, and transparency. Building on our previous LoRA-MINT framework, which relies exclusively on perplexity under a strict black-box setting, we propose an approach based on intermediate model representations, including activations and parameter-dependent signals. These representations enable a MINT classifier to distinguish training from non-training samples without requiring full access to model parameters.
    We evaluated the method on four LLMs (Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B) fine-tuned on three datasets covering medical, mathematical, and academic reasoning tasks. Results show consistent performance across architectures and domains, achieving scores between 95% and 98%, with robustness to variations in model configurations and data distributions. The proposed approach enables efficient auditing of sensitive training data and supports data provenance and compliance with emerging privacy and AI governance regulations. Although our experiments focus on LoRA-based fine-tuning, the methodology can be extended to other LLM adaptation techniques and, more broadly, to other AI models.

    Submission history

    From: Gonzalo Mancera [view email]
    [v1] Tue, 13 Oct 2026 09:40:58 UTC (403 KB)

  • ChunkAct: Action Chunking for Agentic Robotics

    ChunkAct: Action Chunking for Agentic Robotics

    [Submitted on 13 Oct 2026]

    View PDF HTML (experimental)

    Abstract:Agentic robotics places a large language model (LLM) agent in the robot control loop, where it composes perception, motion, and vision-language-action (VLA) tools under the guidance of a memory and thereby generalizes to new tasks without retraining. This generality comes at a steep price in efficiency. Every primitive requires a round of LLM reasoning over the full interaction history, so the cost of an episode grows with the number of primitives. We find this reasoning largely redundant: once a sub-goal is chosen, its primitives follow a routine already prescribed by memory. We present ChunkAct, which introduces action chunking to agentic robotics. Online, a single call launches a chunk of chained primitives, which executes autonomously and returns control only upon failure. Offline, ChunkAct compiles the memory into these chunks, so execution is memory-free at test time and the planner is reserved for decisions that require reasoning. LLM computation therefore scales with the number of decisions rather than the number of primitives. On LIBERO-Pro, ChunkAct improves task success by 5.0 points over the state of the art in agentic robotics while requiring 2.9x fewer LLM calls, 4.4x fewer input tokens, and 2.3x fewer output tokens.

    Submission history

    From: Bingxin Xu [view email]
    [v1] Tue, 13 Oct 2026 17:59:57 UTC (452 KB)

  • NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents

    NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents

    [Submitted on 21 Jan 2026 (v1), last revised 12 Oct 2026 (this version, v3)]

    View PDF HTML (experimental)

    Abstract:Agentic Large Language Models (LLMs) are models able to reason, plan, and execute tools over unstructured data. These abilities enable transformative applications in domains spanning personal assistants, financial, and legal domains, etc. While these systems can substantially improve productivity and service quality, effective agency typically requires access to sensitive personal or organizational information. However, this access introduces critical inference-time privacy risks, specifically regarding contextually appropriate information disclosure. While recent studies highlight the inability of agentic LLMs to consistently adhere to privacy norms, existing defenses often rely on auxiliary LLM-based monitors. However, these defenses are expensive and offer limited protection against attacks that are robust to semantic censorship. To contrast this background, this paper proposes a notion of privacy filters based on activation probing. We show that these filters are both computationally efficient and effective for both single-turn and multi-turn conversational settings. Furthermore, this work provides the first systematic investigation into probing model internals across a conversation trajectory, moving beyond static, single-prompt analysis to capture the evolving state of privacy-sensitive interactions, illustrating their applicability across contexts, attack styles, multiple base models, and challenging settings such as mosaic attacks, while providing nuanced insights into the nature of the discovered concepts and modular construction of probes.

    Submission history

    From: Saswat Das [view email]
    [v1] Wed, 21 Jan 2026 05:16:50 UTC (23,221 KB)
    [v2] Wed, 1 Jul 2026 15:49:57 UTC (22,587 KB)
    [v3] Mon, 12 Oct 2026 19:24:51 UTC (28,588 KB)

  • NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime

    NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime

    [Submitted on 7 Oct 2026 (v1), last revised 12 Oct 2026 (this version, v2)]

    View PDF HTML (experimental)

    Abstract:Language models trained with long-horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, raising the ceiling on what an embodied agent can understand and decide. Physical interaction, however, remains the domain of action policies, which provide dense, low-latency control. We present NavGPT-3, a harness that connects the two models, with an OS-like runtime built above it: reasoning, acting, and monitoring run as threads with their own context, tools, and permissions, while the runtime schedules them and decides which thread controls the robot’s motion, so that the robot can react to sudden real-world events through interruption and thread switching. Beneath it, our action policy NavGPT VLA, trained on 19.28M examples, allocates visual tokens using codec allocation, in proportion to scene change; its 8B model alone reaches 76.94 SR on R2R-CE and leads RxR-CE with 82.39 SR. With the complete harness, NavGPT-3 sets the state of the art on R2R-CE (81.51 SR) and, for the first time, brings an autonomous agent to human level: on RxR-CE it matches human followers in success (90.43 vs. 90.4 SR) and path fidelity (78.47 vs. 77.7 nDTW) at 1 min 22 s per episode, versus roughly 3 min for a human. We comprehensively ablate the harness design and the interaction between the two models, showing how tools and the action policy shape the path from language-model reasoning to physical control: when NavGPT VLA executes the route, the reasoning loop shortens and the system’s minimum reaction time falls from 3-19 s per language-model decision to 0.5-1 s per action-policy step (1-2 Hz). These results show that designing this embodied interface is central to connecting frontier language-model intelligence with low-level physical control. We will release all models, code, and evaluation records.

    Submission history

    From: Gengze Zhou [view email]
    [v1] Wed, 7 Oct 2026 18:45:56 UTC (17,542 KB)
    [v2] Mon, 12 Oct 2026 08:50:46 UTC (17,542 KB)

  • Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization

    Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization

    [Submitted on 8 Oct 2026]

    View PDF HTML (experimental)

    Abstract:Efficient autoregressive generation is increasingly limited by the memory footprint of the key-value (KV) cache, which grows linearly with sequence length, batch size, and model depth. Existing approaches such as Multi-Query Attention (MQA), Grouped-Query Attention (GQA), Multi-Head Latent Attention (MLA), and Tensor Product Attention (TPA) reduce this cost through different trade-offs, but often sacrifice expressivity, require additional positional-encoding machinery, or impose restrictive low-rank structure. We propose Kronecker Product Attention (KPA), which represents key and value activations as sums of Kronecker products of compact, token-dependent matrix factors. KPA strictly generalizes TPA: rank-one vector-outer-product factorization arises as a degenerate Kronecker split, while non-degenerate matrix factors can represent richer block-structured patterns. We further show that a structured rotary embedding operator matched to the Kronecker representation preserves the relative-position property of RoPE while rotating only the compact factors, removing the need for a decoupled RoPE branch as in MLA. Empirically, KPA matches or exceeds TPA, MHA, MQA, GQA, and MLA across four model scales and nine downstream benchmarks under identical training conditions, while achieving up to 10x KV cache compression. The Kronecker head split is a tuneable hyperparameter that allows practitioners to navigate the compression-accuracy trade-off at design time without architectural changes.

    Submission history

    From: Bahareh Nikpour [view email]
    [v1] Thu, 8 Oct 2026 19:23:56 UTC (227 KB)

  • On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective

    On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective

    [Submitted on 2 Mar 2026 (v1), last revised 8 Oct 2026 (this version, v2)]

    View PDF

    Abstract:We study the convergence dynamics of Gradient Descent (GD) in a minimal binary classification setting, consisting of a two-neuron ReLU network and two training instances. We prove that even under these strong simplifying assumptions, while GD successfully converges to an optimal robustness margin, effectively maximizing the distance between the decision boundary and the training points, this convergence occurs at a prohibitively slow rate, scaling strictly as $Theta(1/ln(t))$. To the best of our knowledge, this establishes the first explicit lower bound on the convergence rate of the robustness margin in a non-linear model. Through empirical simulations, we further demonstrate that this inherent failure mode is pervasive, exhibiting the exact same tight convergence rate across multiple natural network initializations. Our theoretical guarantees are derived via a rigorous analysis of the GD trajectories across the distinct activation patterns of the model. Specifically, we develop tight control over the system’s dynamics to bound the trajectory of the decision boundary, overcoming the primary technical challenge introduced by the non-linear nature of the architecture.

    Submission history

    From: Guy Smorodinsky [view email]
    [v1] Mon, 2 Mar 2026 17:13:33 UTC (38 KB)
    [v2] Thu, 8 Oct 2026 18:50:56 UTC (38 KB)

  • Recurrent Harness for Agentic Continual Learning

    Recurrent Harness for Agentic Continual Learning

    [Submitted on 8 Oct 2026]

    View PDF HTML (experimental)

    Abstract:Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.

    Submission history

    From: Yuxin Xiao [view email]
    [v1] Thu, 8 Oct 2026 19:44:43 UTC (422 KB)