Training-Free Character-Length Control in Summarization with Diffusion Language Models

training-free-character-length-control-in-summarization-with-diffusion-language-models

Training-Free Character-Length Control in Summarization with Diffusion Language Models

[Submitted on 4 Nov 2026]

View PDF HTML (experimental)

Abstract:Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.

Submission history

From: Steffen Castle [view email]
[v1] Wed, 4 Nov 2026 11:32:20 UTC (119 KB)