[Submitted on 15 Oct 2026 (v1), last revised 16 Oct 2026 (this version, v2)]
Abstract:Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP’s two-bit code lowers output KL by 41–63% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.
Submission history
From: Donghoon Han [view email]
[v1] Thu, 15 Oct 2026 12:57:19 UTC (185 KB)
[v2] Fri, 16 Oct 2026 03:50:54 UTC (185 KB)
