[Submitted on 12 May 2026 (v1), last revised 4 Nov 2026 (this version, v4)]
Abstract:Training-free camera control for pretrained flow-matching video generators is a partial-observation inverse problem: a depth-warped guidance video supplies noisy evidence on a subset of latent sites, which the sampler must reconcile with the pretrained prior. Existing methods struggle to balance the trade-off between trajectory adherence and visual quality and the heuristic guidance-strength tuning lacks robustness. We propose GibbsLens, which resolves this dilemma through a structural change to the sampler: each outer hard-replacement guidance step is augmented with an inner-loop block-conditional pseudo-Gibbs refinement on the unobserved complement at the same noise level, with provable convergence to the partial-observation conditional data law. To accelerate convergence on high-dimensional video latents, we exploit their conditional locality, partitioning the unobserved complement into 3D patches, each tracked by a custom mixing indicator that adaptively freezes converged patches. On RealEstate10K and DAVIS, GibbsLens attains the best FVD against all seven training-free and training-based competitors, outperforming every training-free baseline on every reported metric. Project page: this https URL.
Submission history
From: Xi Ye [view email]
[v1] Tue, 12 May 2026 09:51:04 UTC (19,084 KB)
[v2] Sat, 16 May 2026 12:55:04 UTC (19,075 KB)
[v3] Mon, 2 Nov 2026 15:31:32 UTC (21,813 KB)
[v4] Wed, 4 Nov 2026 12:35:20 UTC (21,813 KB)
