[Submitted on 21 Sep 2026 (v1), last revised 14 Oct 2026 (this version, v4)]
Abstract:We present a compact geometry-native latent space as a shared foundation for perception and generation. Visual generators can produce photorealistic frames without preserving a consistent 3D scene. We argue that this is not only a modeling problem but also a representation problem: generators typically evolve appearance-centric latents, while perception models recover geometry in a semantically rich space that encodes cross-view structure. Rather than adding geometry as another output, we reparameterize a geometry foundation model’s features into a compact latent space for generation. We realize this shift with the geometry-native autoencoder (GAE), whose latent is jointly decodable to appearance, depth, cameras, and point maps. With this state, a standard conditional flow supports diverse generation tasks. In controlled comparisons that hold the generator and training protocol fixed, replacing the latent with GAE improves both visual quality and independently measured 3D coherence: FVD falls by $12.7%$ and $23.1%$ on RealEstate10K and DL3DV, and camera-trajectory error is halved on RealEstate10K. Together, these results show that the latent space is central to geometry-consistent generation and can serve as a shared interface between perception and generation.
Submission history
From: Jiahao Lu [view email]
[v1] Mon, 21 Sep 2026 17:56:25 UTC (31,887 KB)
[v2] Fri, 25 Sep 2026 18:22:51 UTC (28,334 KB)
[v3] Tue, 13 Oct 2026 17:07:02 UTC (28,337 KB)
[v4] Wed, 14 Oct 2026 09:21:07 UTC (28,346 KB)