[Submitted on 11 Oct 2025 (v1), last revised 16 Oct 2026 (this version, v4)]
Abstract:Multilingual Large Language Models (LLMs) often exhibit hallucinations such as unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via Principal Component Analysis (PCA) on parallel translations and steers token embeddings along these axes to control language identity. Our approach mitigates code-switching (reduced next-token distributional divergence by up to 55%, reduced Code-Switching Index by 63–99%) while preserving broad semantic direction with negligible computational overhead and requires only minimal parallel data for calibration. Additionally, we evaluate language classification using a single principal component (95-99% accuracy) and further analyze the layer-wise evolution of language representations.
Submission history
From: Andrey Goncharov [view email]
[v1] Sat, 11 Oct 2025 19:49:38 UTC (2,821 KB)
[v2] Mon, 2 Mar 2026 12:36:21 UTC (2,810 KB)
[v3] Wed, 15 Apr 2026 11:13:28 UTC (2,807 KB)
[v4] Fri, 16 Oct 2026 17:02:53 UTC (2,820 KB)
