[Submitted on 28 Mar 2026 (v1), last revised 5 Nov 2026 (this version, v3)]
Abstract:The Forward-Forward (FF) algorithm trains networks layer-by-layer using a local “goodness function,” yet the choice of activity statistic remains an important design question. We systematically study how selectivity and activity shape affect FF learning, introducing selective goodness functions based on top-$k$ and entmax-weighted energy alongside shape-sensitive objectives based on normalized higher-order moments. In particular, we propose burstiness goodness, a parameter-free, scale-invariant excess-kurtosis objective that rewards heavy-tailed activity patterns. Our study examines goodness functions together with activation and label-injection choices across six small-image datasets, and uses controlled experiments to isolate their effects. In validation-tuned, five-seed Fashion-MNIST experiments with matched ReLU activation, standard label injection, and per-layer scoring, burstiness improves accuracy from $73.2pm0.9%$ to $88.6pm0.4%$ over sum-of-squares. On CIFAR-10, burstiness improves block-local ResNet-18 accuracy from $28.9pm0.5%$ to $46.8pm2.5%$ under the same separate per-block label pathway. These results show that activity-shape statistics can provide effective local learning signals across the tested settings. Code is available at this https URL.
Submission history
From: Şuayp Talha Kocabay [view email]
[v1] Sat, 28 Mar 2026 23:11:21 UTC (24 KB)
[v2] Thu, 16 Apr 2026 20:55:27 UTC (30 KB)
[v3] Thu, 5 Nov 2026 18:59:03 UTC (27 KB)
