MoE–LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study

moe–lora-methods-for-multilingual-adaptation:-a-systematic-empirical-study

MoE–LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study

[Submitted on 17 Nov 2026 (v1), last revised 18 Nov 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model, using 146 training runs across 200 unique languages and more than 10,000 evaluations. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.

Submission history

From: Nikit Srivastava [view email]
[v1] Tue, 17 Nov 2026 18:11:42 UTC (86 KB)
[v2] Wed, 18 Nov 2026 08:38:14 UTC (89 KB)