Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

circuit-hypernetworks-for-quantum-augmented-diffusion-language-models

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

[Submitted on 21 Sep 2026 (v1), last revised 18 Nov 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Improving pretrained language models with limited adaptation data calls for computational modules that are both effective and scalable. Parameterized quantum circuits offer expressive transformations for language modeling, yet existing quantum-enhanced language models apply the same predefined circuit design to every token in an input sequence, regardless of its content. This limitation is particularly acute in masked-diffusion language models, where token representations evolve across iterative denoising steps while the circuit design remains fixed. We introduce HyperQ, a quantum-augmented masked-diffusion language model that dynamically synthesizes token-specific quantum circuits. Built upon an IQP-inspired architecture, HyperQ employs a lightweight circuit hypernetwork to generate rotation angles and measurement bases conditioned on each token’s hidden state. The resulting quantum expectation values are projected back into the frozen language-model backbone through a residual connection. To support wider quantum registers, HyperQ bounds circuit interaction degree, maintaining per-qubit gradient scale and readout cost independent of register width. Across six zero-shot benchmarks, scaling HyperQ from 16 to 64 qubits improves average accuracy from 47.65 to 54.30, surpassing the frozen backbone’s 49.59 by 4.71 points while using only one tenth of the fine-tuning data used by classical baselines. We further execute the generated circuits on an IBM 156-qubit superconducting quantum processor, demonstrating their feasibility on physical quantum hardware.

Submission history

From: Xiaoqiang Wang [view email]
[v1] Mon, 21 Sep 2026 14:25:04 UTC (374 KB)
[v2] Wed, 18 Nov 2026 08:02:00 UTC (322 KB)