[Submitted on 10 Jun 2026 (v1), last revised 19 Oct 2026 (this version, v4)]
Abstract:Because of polysemanticity and superposition, it has been challenging to attribute token understanding and generation to specific neurons or dimensions. Yet the information in the hidden states is so rich that the signs of the residual stream can surface semantic content in early layers and control what the model generates. We call this sign-pattern-driven interpretation approach Bag of Dims (BoD). Using BoD, we read each dimension of the residual stream individually. Since dimensions carry almost no shared information, a concept is simply the set of dimensions that agree in sign. This lets us discover tens to hundreds of thousands of concepts per model without training or labels. A concept is detectable as early as the first few layers, at AUC above 0.9 across the model families we tested, well before the answer it resolves to becomes legible. BoD can therefore screen harmful prompts at prefill, without a trained guard model. The prediction capability of the signs is strong: setting all magnitudes to unity preserves 56-94% top-5 next-token accuracy through the language model head. Flipping a concept’s sign pattern during the forward pass suppresses its logits, and a separate training-free intervention steers a chosen concept into generated text. The BoD method depends on the transformers’ per-dimension sign pattern, present across the residual stream, attention, and MLP subspaces, rather than on any particular architecture, making it applicable to a wide range of models.
Submission history
From: Varun Reddy Nalagatla [view email]
[v1] Wed, 10 Jun 2026 19:34:31 UTC (13,907 KB)
[v2] Wed, 17 Jun 2026 04:35:07 UTC (9,417 KB)
[v3] Wed, 8 Jul 2026 06:24:26 UTC (9,357 KB)
[v4] Mon, 19 Oct 2026 12:32:32 UTC (9,476 KB)
