[Submitted on 30 Jan 2026 (v1), last revised 1 Nov 2026 (this version, v3)]
Abstract:Consistency models have been proposed for fast generative modeling, achieving results competitive with diffusion and flow models. However, these methods exhibit instability and limited reproducibility when training from scratch, motivating subsequent work to explain and stabilize these issues. While these efforts have provided valuable insights, the explanations remain fragmented, and the theoretical relationships remain unclear. In this work, we provide a theoretical examination of consistency models by analyzing them from a flow map-based perspective. This joint analysis clarifies how training and convergence behavior can give rise to degenerate solutions. Building on these insights, we revisit self-distillation as a practical remedy for certain forms of suboptimal convergence. To avoid excessive gradient norms during its optimization, we reformulate its objective empirically, yielding improved Self-Distillation (iSD). We further extend iSD with training-time classifier-free guidance, yielding iSD-T, which improves performance and reproducibility in the studied settings. We demonstrate that iSD extends beyond image generation to diffusion-based policy learning, without reliance on pretrained diffusion models for initialization.
Submission history
From: Youngjoong Kim [view email]
[v1] Fri, 30 Jan 2026 07:51:55 UTC (15,564 KB)
[v2] Sat, 20 Jun 2026 05:34:10 UTC (16,209 KB)
[v3] Sun, 1 Nov 2026 08:12:07 UTC (17,710 KB)
