Unintended Side Effects of Value Training in LLMs

unintended-side-effects-of-value-training-in-llms

Unintended Side Effects of Value Training in LLMs

[Submitted on 21 Oct 2026 (v1), last revised 12 Nov 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:As LLMs are increasingly deployed in value-sensitive contexts, aligning them with human values has emerged as a post-training objective in NLP, aimed at instilling additional values and guiding model behaviour. However, current evaluations remain narrow. Because values are interconnected, it is unclear how fine-tuning for specific values alters unrelated capabilities and behaviours. In this work, we adapt four instruction-tuned LLMs across individual Hofstede’s value dimensions and evaluate them on both cross-value transfer and broader behaviours. We find that value-specific training produces unintended side effects on untrained values, risk preferences, social cognition, and safety traits. Consequently, we suggest that value alignment after post-training should incorporate broader behavioral evaluations, and we highlight the need to investigate where additional value training might best fit into the pipeline. We release all 48 value-tuned model checkpoints at this https URL

Submission history

From: Siddhesh Pawar [view email]
[v1] Wed, 21 Oct 2026 15:58:53 UTC (1,600 KB)
[v2] Thu, 12 Nov 2026 10:15:44 UTC (1,612 KB)