[Submitted on 13 Oct 2026 (v1), last revised 21 Oct 2026 (this version, v2)]
Abstract:Image-token compression must reduce computation without hurting the semantics. We introduce Varest, guided by semantic fungibility: tokens within a coherent semantic region can be summarized coarsely, while object boundaries and literal text require finer distinctions. Object and text localizers guide variable-resolution merging, preserving spatial coverage and finer detail at boundaries and text. Learned [REL] registers then prune visual tokens after early decoder processing. Training across budgets enables one checkpoint per backbone. Across three backbones and 13 benchmarks, Varest outperforms 15 baselines in mean performance retention at every evaluated budget, retaining 86.10-92.33% of full-token performance with only 5.56% of the original layer-averaged image-token count.
Submission history
From: Tianhao Fu [view email]
[v1] Tue, 13 Oct 2026 16:42:32 UTC (4,734 KB)
[v2] Wed, 21 Oct 2026 04:34:21 UTC (21,737 KB)
