[Submitted on 13 May 2026 (v1), last revised 12 Nov 2026 (this version, v3)]
Abstract:Recent works have proposed incorporating heavy-tailed (HT) noise into diffusion- and flow-based generative models, with the goals of better recovering the tails of target distributions and improving generative diversity. This motivation is intuitive: if the data are heavy-tailed, HT noise may appear better matched than light-tailed (LT) Gaussian noise. However, replacing Gaussian noise by HT noise also changes the underlying estimation problem. In this paper, we revisit this paradigm through a combined theoretical and empirical study, establishing sampling-error bounds for two representative diffusion models driven by HT and LT noise. We show that HT diffusion models can achieve exponentially fast initialization, but accumulated training errors may offset this advantage. Experiments on synthetic and real-world datasets support our theoretical findings. Our results call into question a growing design trend in generative modeling and challenge the use of HT noise to improve rare-region exploration.
Submission history
From: Hamza Cherkaoui PhD [view email]
[v1] Wed, 13 May 2026 08:37:59 UTC (106 KB)
[v2] Mon, 1 Jun 2026 15:14:43 UTC (107 KB)
[v3] Thu, 12 Nov 2026 11:30:24 UTC (92 KB)
