[Submitted on 1 Dec 2024 (v1), last revised 16 Oct 2026 (this version, v3)]
Abstract:Visual grounding aims to localize the image regions based on a textual query. Given the difficulty of large-scale data curation, we investigate how to effectively learn visual grounding under data-scarce settings in this paper. To address the data scarcity, we propose a novel framework, POBF (Paint Outside the Box and Filter). POBF synthesizes images by inpainting outside the box, tackling a label misalignment issue encountered in previous works. Furthermore, POBF leverages an innovative filtering scheme to select the most effective training data. This scheme combines a hardness score and an overfitting score, balanced by a penalty term. Extensive experiments across four benchmark datasets demonstrate that POBF consistently improves performance, achieving an average gain of 5.83% over the real-data-only method and outperforming leading baselines by 2.29%-3.85% in accuracy. Additionally, we validate the robustness and generalizability of POBF across various generative models, training data sizes, and model architectures.
Submission history
From: Zilin Du [view email]
[v1] Sun, 1 Dec 2024 05:47:59 UTC (2,785 KB)
[v2] Sun, 20 Apr 2025 10:32:37 UTC (8,267 KB)
[v3] Fri, 16 Oct 2026 09:26:02 UTC (3,261 KB)
