[Submitted on 12 Nov 2026]
Abstract:Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against $K$ ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to $K+1$ ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from $74.60pm0.30%$ to $75.83pm0.15%$ on CIFAR-10 and from $42.80pm0.46%$ to $44.07pm0.32%$ on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches $75.33pm0.15%$ on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter–accuracy trade-off, including a configuration that achieves $0.64$ percentage points higher mean accuracy with $21.8%$ fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter–accuracy trade-off.
Submission history
From: Mikail Yayla [view email]
[v1] Thu, 12 Nov 2026 22:56:04 UTC (169 KB)
