What Does a Bias Head Measure? Non-Identifiability in Head-Level Bias Attribution

what-does-a-bias-head-measure?-non-identifiability-in-head-level-bias-attribution

What Does a Bias Head Measure? Non-Identifiability in Head-Level Bias Attribution

[Submitted on 17 Nov 2026]

View PDF HTML (experimental)

Abstract:Methods that localise social bias in large language models to individual attention heads and remove it by ablation are validated by their effect on a fairness metric, never by evidence that the statistic selecting the heads measures bias. We audit this on Qwen-2.5-7B-Instruct and Llama-3.1-8B-Instruct. The head map correlates at $rho = -0.950$ and $-0.768$ with a label-blind response-length map, apparently identifying a surface confound. An intervention that breaks the length-label coupling rejects this reading: with the prompt held fixed, response length explains 1% of the statistic’s variance. What does explain it is prompt identity, at 64% of variance, in an ordering that is not monotonic in length. Within any fixed prompt the fairness label, response length and the presence of a protected-group token are collinear at $phi = 0.95$, partly by definition of the labelling rule, so no observational analysis can attribute the map to any of them, and no method in this family performs an experiment that could. The substantive claims fail independently: bias selectivity is not late-layer-concentrated on either model under either reference reading, and no head set we tested reduces the unfairness rate when ablated. Its complement, scored on the opposite label, reliably destroys refusal behaviour ($p = 2.7times10^{-5}$), demonstrating that the pipeline detects effects at this scale.

Submission history

From: S.M Seefat Alam [view email]
[v1] Tue, 17 Nov 2026 18:57:08 UTC (50 KB)