[Submitted on 6 Feb 2026 (v1), last revised 19 Oct 2026 (this version, v4)]
Abstract:Self-reflection is a basic self-improvement skill of language agents, yet it does not reliably improve their outputs, where the underlying behavioral mechanism remains opaque. We introduce **ReBeCA** (self-**Re**flection **Be**havior explained through **C**ausality-inspired **A**nalysis), a framework for identifying and validating outcome-relevant reflective behaviors from recorded trajectories. ReBeCA uses **reflective codes**, an inductive coding schema, to represent each trajectory as a sequence of reflective behaviors. It learns dependencies among these behaviors and the outcome with causal discovery, selects the behaviors that best predict the outcome of held-out trajectories, and tests them for consistency across data subsets and by prompt-based steering. In a controlled case study of four Qwen3 models under fixed-round Self-Refine on mathematical reasoning and translation, ReBeCA identifies compact, round-specific behavioral sets with superior held-out predictive fit and uses prompt-based steering on unseen problems to reveal how individual and combined behavioral directives affect performance (Holm-adjusted $p = .020$ on Qwen3-4B). ReBeCA offers a reusable framework that turns agent trajectories into interpretable, verifiable behavioral hypotheses without requiring model internals or preselecting the behaviors to test.
Submission history
From: Tianqiang Yan [view email]
[v1] Fri, 6 Feb 2026 04:00:57 UTC (634 KB)
[v2] Sun, 6 Sep 2026 13:46:00 UTC (635 KB)
[v3] Mon, 12 Oct 2026 12:46:36 UTC (635 KB)
[v4] Mon, 19 Oct 2026 05:30:12 UTC (979 KB)
