[Submitted on 5 Aug 2026 (v1), last revised 18 Nov 2026 (this version, v3)]
Abstract:Instruction-based video editing (IVE) offers diverse applications, yet evaluating editing models remains challenging. Existing benchmarks suffer from two key limitations: narrow task coverage inherited from image editing that overlooks video-specific dimensions, and metrics that inadequately capture instruction fidelity, allowing visually plausible but incorrect edits to receive high scores. To address these limitations, we introduce OmniEdit-Bench, a comprehensive benchmark covering spatial, temporal, audio, and reference-based editing. It distinguishes between explicit and implicit instructions and incorporates reasoning-based scenarios to reflect real-world editing requirements. We further propose an evaluation framework that assesses accuracy, preservation, realism, and consistency using both human judges and a state-of-the-art vision-language model (VLM). An accuracy-aware penalty mechanism conditions the other three scores on accuracy, prioritizing instruction fidelity and penalizing plausible but incorrect edits. Experiments on prominent open-source and commercial IVE models reveal substantial room for improvement. OmniEdit-Bench provides a structured testbed for identifying model limitations and advancing instruction-based video editing. Project page: this https URL.
Submission history
From: Chenxuan Miao [view email]
[v1] Wed, 5 Aug 2026 16:58:48 UTC (24,589 KB)
[v2] Wed, 2 Sep 2026 09:02:31 UTC (24,589 KB)
[v3] Wed, 18 Nov 2026 13:57:06 UTC (24,587 KB)
