[Submitted on 23 Oct 2025 (v1), last revised 27 Oct 2026 (this version, v2)]
Abstract:Multimodal sentiment analysis (MSA) has made considerable progress in recent years. Existing MSA methods mainly learn shared multimodal representations or train unimodal predictors with a common multimodal label. However, assigning the shared label to every modality can obscure modality-specific sentiment, while direct fusion may overemphasize unreliable predictions when modalities disagree. To address these issues, we propose Uncertainty-Guided Multimodal Consensus (UMC), which first recovers continuous unimodal pseudo-labels and then forms a reliability-aware consensus across modalities. Our method consists of two key components: i) Continuous Prototype-Guided Label Recovery (CPLR), which combines heteroscedastic predictions with ordered sentiment prototypes and uses confidence and label conflict to refine continuous unimodal pseudo-labels, and ii) Reliability and Precision Consensus (RPC), which estimates sample-dependent modality reliability from predictive statistics and cross-modal disagreement and combines it with predictive precision through a closed-form consensus solver. Extensive experiments conducted on four publicly available MSA datasets demonstrate the effectiveness and superiority of our proposed model.
Submission history
From: Guowei Zhong [view email]
[v1] Thu, 23 Oct 2025 06:25:10 UTC (1,208 KB)
[v2] Tue, 27 Oct 2026 06:14:16 UTC (370 KB)
