[Submitted on 20 Oct 2026 (v1), last revised 23 Oct 2026 (this version, v2)]
Abstract:Empirical scaling laws relate loss to parameters $N$, training tokens $D$ and compute $C$. This technical note explains three mechanisms behind their exponents. Local approximation bounds function error at rate $N_e^{-gamma s/d}$, with effective parameters $N_e$, smoothness $s$, reuse exponent $gamma$ and dimension $d$; the loss function determines the risk bound. Independent feature coverage gives $alpha_D=(beta-1)/zeta$ from predictive-weight decay $beta$ and occurrence rarity $zeta$. Separable-loss optimization determines training allocations and deployment crossovers. Nonlinear scoring can produce apparent emergent abilities, while grokking concerns delayed generalization and verification constrains test-time scaling. For regular molecular configurations, the $mathrm{SE}(3)$ quotient gives $d_{mathrm{int}}=d-6$ and relative approximation-exponent gain $6/d_{mathrm{int}}$. Optimized dense-to-equivariant excess-loss ratios scale as $C^{p_{mathrm{eq}}-p_{mathrm{dense}}}$, where $p_j$ are compute-loss exponents and the prefactor depends on loss amplitudes and FLOP costs. Domain examples also cover visual resolution, conformational coverage and correlated-sample precision.
Submission history
From: Tian-Yu Zhao [view email]
[v1] Tue, 20 Oct 2026 07:52:23 UTC (734 KB)
[v2] Fri, 23 Oct 2026 11:01:21 UTC (737 KB)
