VIG-RL: Learning to Search and Insert for Verified Image Grounding

Source: arXiv:2607.28055 · Published 2026-07-30 · By Qinhan Yu, Jun Guang, Chong Chen, Wentao Zhang

TL;DR

Verified Image Grounding (VIG) addresses the crucial challenge of reliably integrating authentic visual evidence—retrieved real-world images—into interleaved text-image responses for knowledge-intensive applications. Prior methods rely on static pipelines that separately retrieve then generate responses, lacking the dynamic reasoning needed to decide when to retrieve images, which to choose, and where exactly to insert them in text. To overcome these limitations, VIG-RL frames VIG as a sequential, agentic decision-making process using a Reinforcement Learning (RL) framework inspired by the ReAct paradigm. The agent jointly learns to search for textual and visual evidence, select relevant images, and generate multimodal responses with correctly placed image references. The agent is optimized using Group Relative Policy Optimization (GRPO) and a composite reward explicitly supervising syntax adherence, retrieval process quality, image insertion precision, and final multimodal answer correctness.

Extensive experiments on MRAMG-Bench datasets spanning web, academic, and lifestyle domains demonstrate VIG-RL sets a new state-of-the-art, significantly outperforming strong static retrieval-augmented generation (RAG) baselines—including GPT-5, Gemini-2.5-Flash, and Qwen3-VL variants. VIG-RL-8B improves macro-average Image F1 by over +21 and Comprehensive Score by +21 relative to static RAG on in-domain splits, while also generalizing strongly zero-shot to out-of-domain datasets like Manual and Recipe. Ablations confirm the critical importance of joint multi-step RL optimization and the hybrid composite reward design in preventing reward hacking and encouraging precise, well-placed visual grounding.

Key findings

• VIG-RL-8B improves the six-dataset macro-average Image F1 from 57.7 (static RAG Qwen3-VL-8B) to 78.9 (+21.2) and Comprehensive Score (C.S.) from 56.4 to 78.1 (+21.7).
• VIG-RL-8B achieves best C.S. on all six MRAMG-Bench datasets, including strong zero-shot generalization with C.S.=94.5 on Wit, 48.1 on Manual, and 61.4 on Recipe.
• On order-sensitive datasets Manual and Recipe, VIG-RL-8B improves Order Score from 12.4 to 32.2 and from 10.9 to 40.9 respectively, despite no explicit positional reward.
• Smaller model VIG-RL-4B notably outperforms its zero-shot baseline, boosting Arxiv Image F1 from 18.8 to 80.6 and C.S. from 31.0 to 63.9.
• Ablations show that pure MLLM-based reward enables reward hacking (Answer F1 collapses to 0), whereas the hybrid reward (format + text semantic + image precision + search recall) raises Answer F1 to 84.2 and C.S. to 81.7 on Arxiv.
• Explicit process-level supervision of search steps is critical: removing search recall reward reduces retrieval recall from 92.7% to 87.8% and Answer F1 from 84.2 to 77.6 on Arxiv.
• Context-anchored image retrieval based on associated captions/text outperforms direct visual embedding retrieval, improving retrieval recall from 84.6% to 92.7% and Answer F1 from 51.1 to 84.2 on Arxiv.
• VIG-RL learns to trigger image search on 100% of relevant queries versus 36%–30% baseline rates, substantially improving downstream answer quality.

Threat model

The adversary is an implicit user or system interacting with the multimodal agent requiring factually accurate, grounded responses. The adversary cannot produce synthetic images or manipulate the retrieval corpus directly but can query the system arbitrarily. The framework defends against hallucinated or misaligned visual generation by strictly prohibiting generative image synthesis and enforcing retrieval-only provenance constraints verified via unique image identifiers.

Methodology — deep read

1. Threat Model & Assumptions: The adversary considered is implicit—models must provide factually accurate, verifiably grounded multimodal answers without hallucinating visual content. Adversaries can query and receive answers but cannot generate fake images as the framework prohibits synthetic image generation, only retrieval from verified external corpora is allowed.

2. Data: Training uses 1.1k cross-modally consistent samples from MRAMG-Bench, an open-source benchmark with explicit ground-truth interleaved text-image answers across six datasets (Web, Wiki, Arxiv, Wit, Recipe, Manual). Train/test splits are disjoint in text passages and images to prevent leakage. Retrieval corpora contain multimodal documents with images anchored to contextual text.

3. Architecture/Algorithm: The agent is modeled as a policy πθ operating in a ReAct-style loop alternating reasoning traces and actions. At each step, the policy outputs a textual reasoning trace then chooses an action from {text search, image search, answer generation}. Text and image queries retrieve top-K candidates from external retrievers (BGE-M3 for embeddings). Unique symbolic image tags replace raw pixels during generation to ensure factual provenance. Final multimodal output is assembled by substituting tags with retrieved images. The policy is optimized with Group Relative Policy Optimization (GRPO), a variant of PPO tailored for grouped trajectories.

4. Training Regime: Training runs 20 epochs (720 steps) using the veRL framework on Qwen3-VL-4B/8B-Instruct base models. A composite reward guides training: format correctness ensures syntax rules; text semantic correctness uses an LLM judge evaluator comparing generated text to reference; image insertion precision measures exact overlap of inserted image IDs with ground truth; and search recall measures coverage of ground-truth images in the retrieval pool. All combine multiplicatively with the format correctness acting as a gate.

5. Evaluation Protocol: Evaluated on held-out splits for in-domain and out-of-domain datasets. Metrics include Image F1 for selection precision & recall, Comprehensive Score (C.S.) from GPT-4o for overall multimodal answer quality (text correctness, image relevance, insertion placement, alignment), and Order Score measuring image sequence accuracy on procedural datasets. Comparisons are made against static RAG baselines, zero-shot agentic baselines, and a supervised trajectory imitation baseline trained on top rollouts. Ablations examine reward types and retrieval strategies.

6. Reproducibility: Code and benchmark datasets (MRAMG-Bench) are publicly available. They use publicly documented models (Qwen3-VL, Gemini, GPT-5) though some model weights or internal configurations of commercial models are closed-source. Training specifics like seeds and hyperparameters are reported, but full RL seed variability is not deeply analyzed.

Example: Given a query about a historical landmark’s architectural features, the agent starts with the textual query context, reasons about missing details, triggers text search to retrieve relevant passages, then generates an image search query anchored by those passages to retrieve verified photos. Each retrieved image is assigned a symbolic tag img:id. The agent selectively inserts these tags interleaved in the answer, producing a multimodal output where placeholders are replaced with authentic images. The process is reinforced by rewards encouraging factual text, image precision, and relevant searches, enabling the model to learn when and what to retrieve as well as where to insert evidence, surpassing static pipelines that retrieve images only beforehand.

Technical innovations

• Formulated Verified Image Grounding as an agentic sequential decision-making task blending search, selection, and insertion, rather than static retrieve-then-generate pipelines.
• Designed a ReAct-style interaction loop allowing integrated reasoning and multimodal retrieval actions (text search, image search, answer generation) dynamically controlled by the agent.
• Developed a hybrid composite reinforcement learning reward combining rule-based format and image insertion precision with LLM-based textual quality evaluation to robustly supervise multimodal grounding without reward hacking.
• Introduced symbolic visual referencing to sidestep native pixel output limitations by mapping retrieved images to unique discrete identifiers, ensuring factual provenance in generation.
• Proposed context-anchored image retrieval leveraging associated textual contexts rather than direct vision embeddings, significantly improving retrieval relevance for knowledge-intensive images like academic figures.

Datasets

• MRAMG-Bench — 1.1k annotated samples — Public benchmark with ground-truth interleaved multimodal answers on six domain-diverse datasets (Web, Wiki, Arxiv, Wit, Manual, Recipe)

Baselines vs proposed

• Static RAG Qwen3-VL-8B: Image F1 = 57.7 vs VIG-RL-8B Image F1 = 78.9 on six MRAMG datasets (macro-average)
• Static RAG Qwen3-VL-8B: Comprehensive Score = 56.4 vs VIG-RL-8B = 78.1 (+21.7)
• Zero-shot agentic Qwen3-VL-8B-Thinking: C.S. = 46.7 vs VIG-RL-8B = 94.0 on Web dataset
• M2IO-R1-3B: Recipe F1 = 51.2 vs VIG-RL-4B = 55.3; Order Score 31.1 vs 35.1
• Supervised Fine-Tuning SFT on Gemini-2.5 trajectories: Arxiv C.S. = 41.8, Manual C.S. = 33.0 vs VIG-RL: 81.7 and 48.1 respectively
• Context-Anchored Image Retrieval: Arxiv retrieval Recall = 92.7 vs CLIP-style direct visual retrieval = 84.6
• Ablation hybrid reward: Full reward Answer F1 = 84.2 vs model-judge only reward = 0.0 (reward hacking) on Arxiv

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28055.

Fig 1: Verified Image Grounding (VIG) vs. Pure Gen-

Fig 2: Illustration of the VIG-RL workflow. Given a user question, the agent iteratively reasons, performs text/image search,

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 3: shows that SFT consistently improves over Base,

Fig 8 (page 3).

Limitations

• Training and evaluation restricted to MRAMG-Bench with 1.1k training samples, limiting scale and diversity for rare knowledge domains.
• No explicit adversarial robustness analysis; performance under intentional retrieval or reasoning attacks remains unexplored.
• Rendering quality and user experience of interleaved responses beyond symbolic tags are not evaluated or detailed.
• Reward design assumes availability of clean ground-truth image sets and placement annotations, which may be impractical for real-world noisy data.
• Dependency on external retrieval quality (BGE-M3) may limit VIG-RL performance without improvements in retrieval models.
• No analysis of latency or computational cost tradeoffs introduced by multi-step agentic retrieval compared to static pipelines.

Open questions / follow-ons

• How does VIG-RL perform under noisy, incomplete, or adversarial retrieval corpora with mislabeled or corrupted images?
• Can the agentic search-selection-insertion framework be extended to real-time interactive settings with tighter latency constraints?
• How to scale training with larger datasets and diverse multimodal corpora to improve robustness and reduce domain gaps further?
• Would additional modalities (e.g., audio or video grounding) be compatible with the symbolic referencing and RL agentic framework?

Why it matters for bot defense

From a bot-defense perspective, VIG-RL’s approach to reliably integrating verified visual evidence into multimodal responses has several implications. Its agentic reinforcement learning framework that dynamically decides when and where to retrieve and insert images could inspire CAPTCHA systems requiring a bot to access and demonstrate factual visual grounding, thus raising the barrier against automated spoofing. The strict symbolic referencing to prevent hallucinated or synthetic visual content parallels the need in bot-defense for verifiable, tamper-resistant evidence presentation. Additionally, the composite reward system balancing retrieval coverage, precision, and semantic alignment might inform the design of CAPTCHA challenges that dynamically adapt in difficulty or content insertion based on user behavior. However, the complex multi-step retrieval and insertion process might come with increased latency or infrastructure costs, factors to weigh when applying agentic RL to real-time verification tasks. Overall, VIG-RL's demonstrated interaction between reasoning, retrieval, and multimodal content integration provides a promising blueprint for more robust, interaction-aware CAPTCHA designs that require dynamic, contextually grounded visual verification.

Cite

@article{arxiv2607_28055,
  title={ VIG-RL: Learning to Search and Insert for Verified Image Grounding },
  author={ Qinhan Yu and Jun Guang and Chong Chen and Wentao Zhang },
  journal={arXiv preprint arXiv:2607.28055},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28055}
}

Read the full paper

• arXiv:2607.28055 (abstract)
• arXiv:2607.28055 (PDF)