[Submitted on 11 Mar 2026 (v1), last revised 5 Nov 2026 (this version, v4)]
Abstract:Current skeleton representation learning paradigms face distinct limitations: Contrastive Learning (CL) often overlooks fine-grained motion details, while Masked Auto-Encoders (MAE) rely on coordinate-level reconstruction. This reconstruction inherently demands dense token sequences and heavy decoders, wasting pre-training computation on discarded components and forcing downstream inference to process dense token grids. To resolve these bottlenecks, we propose SLiM (Skeleton Less is More), a compact-token framework that unifies masked feature prediction and contrastive learning via a shared encoder. By shifting the objective from raw coordinate reconstruction to decoder-free, teacher-guided feature prediction, SLiM breaks the reliance on dense tokenization and enables effective learning with a highly compact token grid. Crucially, to prevent trivial shortcut learning arising from strong inter-joint dependencies of human, we introduce Semantic Tube Masking together with Skeleton-Aware Augmentations to enforce deep skeletal-temporal reasoning and anatomical consistency. Extensive experiments across multiple downstream protocols demonstrate that SLiM achieves state-of-the-art performance while structurally reducing inference computation by 7.89$times$ compared to dense-token MAE baselines.
Submission history
From: Jeonghyeok Do [view email]
[v1] Wed, 11 Mar 2026 11:09:16 UTC (1,150 KB)
[v2] Thu, 12 Mar 2026 04:01:43 UTC (1,147 KB)
[v3] Mon, 3 Aug 2026 11:12:58 UTC (1,254 KB)
[v4] Thu, 5 Nov 2026 14:35:00 UTC (1,254 KB)
