TALD: Transferability-Aware Logit Disentanglement for Cross-Modal Knowledge Distillation

tald:-transferability-aware-logit-disentanglement-for-cross-modal-knowledge-distillation

TALD: Transferability-Aware Logit Disentanglement for Cross-Modal Knowledge Distillation

[Submitted on 18 Nov 2026]

View PDF HTML (experimental)

Abstract:Cross-modal knowledge distillation can incur negative transfer when source-modality supervision conflicts with the target modality’s discriminative characteristics. We propose TALD, a two-stage framework for Transferability-Aware Logit Disentanglement in class-aligned logit space. Guided by task-dependent self-benefiting and cross-modal exclusivity criteria, a Residual Disentangler decomposes logits from a teacher and an independently trained target-modality reference student into modality-general and modality-specific components. A Sample-Adaptive Modulator regulates decomposition constraints and sample-specific fusion weights. The learned modality-general components are then adaptively fused to supervise the final student. Across five benchmarks spanning vision, audio, and text, TALD achieves the largest mean accuracy gain and highest positive-transfer frequency among the compared methods. Across ten transfer directions with five runs each, TALD improves over training without distillation in 84% of evaluations, compared with at most 70% for the baselines, while offering a favorable accuracy-efficiency trade-off. We plan to release the code.

Submission history

From: Hui Li [view email]
[v1] Wed, 18 Nov 2026 20:03:27 UTC (2,916 KB)