Enhancing Latent World Models with Associative Memory

enhancing-latent-world-models-with-associative-memory

Enhancing Latent World Models with Associative Memory

[Submitted on 14 Oct 2026]

View PDF HTML (experimental)

Abstract:World models trained via latent dynamics offer a scalable path for embodied planning, yet existing methods suffer from representational collapse, computational overhead, and limited long-horizon capability. We introduce a memory-enhanced latent world model that augments JEPA-style prediction with a content-addressable memory bank. Our approach learns high-level representations directly from observations, operating at multiple temporal scales within a shared latent space, and conditions predictions via adaptive layer normalization. To prevent collapse and ensure effective memory usage, we design two complementary losses that encourage slot diversity and balanced temporal utilization. Experiments on PushT, Tworoom, Reacher, and OGBCube demonstrate that our method achieves state-of-the-art performance with a lightweight architecture, outperforming conventional latent world models on long-horizon planning tasks.

Submission history

From: Yuwei Sun [view email]
[v1] Wed, 14 Oct 2026 07:08:02 UTC (850 KB)