[Submitted on 6 Feb 2026 (v1), last revised 29 Oct 2026 (this version, v3)]
Abstract:Uniform Discrete Diffusion Models have emerged as a powerful paradigm for generative modeling on sequence data; however, the information-theoretic principles governing their reverse processes remain underexplored. In this work, we bridge this gap by analyzing the reverse process dynamics through the lens of thermodynamic entropy production and dynamical activity. We use a variational expression of the discrete Wasserstein distance to characterize the dynamics of the generation process. Leveraging these insights, we first introduce the Entropic Discrete Schedule (EDS), which aligns with entropy production, and later construct the Wasserstein Discrete Schedule (WDS), which combines entropy production and dynamical activity to act as a geometric cost. Our experimental results on language modeling show that WDS provide the same results as uniform while using up to half of the computations on the MBPP benchmark. The code is available at this https URL.
Submission history
From: Alberto Foresti [view email]
[v1] Fri, 6 Feb 2026 16:38:22 UTC (3,079 KB)
[v2] Mon, 9 Feb 2026 10:07:10 UTC (3,052 KB)
[v3] Thu, 29 Oct 2026 17:44:57 UTC (136 KB)
