Entropy-Regularized Integrated Gradients for Stable Feature Attribution

entropy-regularized-integrated-gradients-for-stable-feature-attribution

Entropy-Regularized Integrated Gradients for Stable Feature Attribution

[Submitted on 17 Nov 2026]

View PDF HTML (experimental)

Abstract:Path-based attribution methods like Integrated Gradients (IG) are widely used to interpret neural networks, yet imperceptible input perturbations can drastically alter their attribution maps while leaving model predictions unchanged. We introduce Entropy-Regularized Integrated Gradients (ERIG), a path-based attribution method that intrinsically attenuates gradient irregularities along the integration path using maximum-entropy softmax weighting with temperature parameter $tau$. We quantify the effect of the entropy-regularized weighting on attribution stability by using the Lipschitz properties of the model and its gradients. We decompose the attribution change under an input perturbation into a model-response term, whose bound is $tau$-independent, and a weight-variation term, whose bound decreases at rate $mathcal{O}(1/(2tau))$. Their ratio $R(tau)$ characterizes the trade-off between gradient information and attribution stability. We further add a new dimension to attribution evaluation by adapting classical noise estimators into two metrics, LINV-C and LINV-P, that quantify attribution noise. Across eleven CNN and Transformer backbones on ImageNet and Oxford-IIIT Pet datasets, our experiments show that, relative to the next-best method, ERIG remains comparable on ranking-based fidelity metrics while improving localization by approximately 24% and attribution stability by approximately 47-fold on average, reaching up to four orders of magnitude improvement over IG, all at negligible additional computational cost.

Submission history

From: Rukmangadh Sai Myana [view email]
[v1] Tue, 17 Nov 2026 22:53:17 UTC (11,540 KB)