Algebraic Interaction: A Recipe for Neural Architecture Generation

algebraic-interaction:-a-recipe-for-neural-architecture-generation

Algebraic Interaction: A Recipe for Neural Architecture Generation

[Submitted on 15 Oct 2026]

View PDF HTML (experimental)

Abstract:We introduce an algebraic foundation for neural architecture design in which convolution, attention, state-space models, and equivariant layers are expressed through compositions of bilinear product operators over two structures: a position algebra governing positional routing and a feature algebra governing feature interactions. This decomposition makes routing, feature algebra, and operator composition explicit design this http URL focus on the feature-algebra axis through an Algebraic Recipe: fix the routing and operator composition of an existing architecture while replacing its feature algebra. This preserves the information-flow structure while allowing computational and symmetry properties to be modified through algebraic this http URL instantiate the recipe in linear attention, Mamba, and equivariant networks. Structure-constant and Clifford algebras improve accuracy at comparable parameter count in linear attention; structure-constant algebras yield up to $3times$ faster training and inference in Vision Mamba at competitive accuracy; and an $mathrm{SO}(3)$-representation algebra provides rotational equivariance by construction. These results demonstrate that feature algebra alone can control accuracy, efficiency, and symmetry without redesigning positional this http URL turns algebraic structure into an explicit design variable and reduces architecture design, within our framework, to algebraic specification.

Submission history

From: Haonan Dong [view email]
[v1] Thu, 15 Oct 2026 10:22:27 UTC (1,546 KB)