PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning

pair-med:-policy-reward-alternation-for-iterative-refinement-in-multilingual-medical-reasoning

PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning

[Submitted on 19 Nov 2026]

View PDF HTML (experimental)

Abstract:Large language models demonstrate strong medical question-answering capabilities, yet reasoning performance remains uneven across languages. Process reward models (PRMs) provide step-level feedback for improving reasoning, but a fixed PRM may become increasingly misaligned with the trajectory distribution of an evolving policy during reinforcement learning, thereby reducing the reliability of its supervision. We propose a policy–reward alternating training framework that jointly adapts a medical PRM and optimizes a multilingual reasoning policy. Following supervised initialization, the current policy generates multilingual trajectories to update the PRM, which then provides process-level rewards for the next policy update. Medical step supervision maintains the PRM’s medical foundation, while process, outcome, and language-related rewards guide policy learning. We develop multilingual medical PRMs (PAIR-PRM) and reasoning models (PAIR-MED) at 8B and 9B scales across 12 languages. On the medical subsets of Global-MMLU and MMLU-ProX, our models improve average accuracy by 3.01–7.47% over their corresponding Qwen backbones. On CUREMED-BENCH, PAIR-MED-9B improves the joint language consistency and logical accuracy metric by 12.88% and 7.45% over baselines for in-domain and out-of-domain languages. Further analyses examine PRM adaptation, multilingual reasoning quality, and the contributions of individual training components.

Submission history

From: Fan Gao [view email]
[v1] Thu, 19 Nov 2026 03:25:14 UTC (957 KB)