[Submitted on 16 Oct 2026 (v1), last revised 21 Oct 2026 (this version, v2)]
Authors:Aishwarya Wesanekar, Sheng-Tao Yang, Jiawei Zang, Flavio Sales Truzzi, Chen Chen, Nader Beigiparast, Shengyuan Huang, Roshanak Zilouchian Moghaddam, Vitor Cid, Colin Clement, Vinayak Thapliyal, Sebastian Ament, Jihao Andreas Lin, Vincent Huang, Kai Han, Boyi Jiang, Wenlin Chen, Chaorong Chen, Ashwin Kumar, Maximilian Balandat, David Eriksson, Yifei Zhang, Pan Zhong
Abstract:Evaluating candidate changes (hypotheses) to recommendation models requires full-scale training runs consuming thousands of GPU-hours, severely limiting the hypothesis space engineers can feasibly explore. We introduce Predictable Scaling (PS), a systematic framework that exploits empirically verified regularity conditions to predict full-scale model performance from small-scale proxies. PS constructs scaled-down variants via composable scaling functions along distinct resource axes, and projects metrics to the target scale via curve fitting with calibrated uncertainty estimates. We compare three prediction strategies of increasing expressiveness: even the simplest (raw proxy) achieves prediction error within the empirical noise floor of repeated-run variance; the most expressive (Gaussian Process) provides an indicative 15% relative MAE improvement and calibrated uncertainty. Evaluation on two ranking models of increasing complexity—a task-specific model and a foundation model—demonstrates that PS correctly ranks actionable hypotheses across diverse hypothesis types at 79-88% compute reductions.
Submission history
From: Aishwarya Wesanekar [view email]
[v1] Fri, 16 Oct 2026 13:35:54 UTC (5,272 KB)
[v2] Wed, 21 Oct 2026 21:48:45 UTC (5,272 KB)
