Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs

Source: arXiv:2607.18230 · Published 2026-07-20 · By Yi Tang, Xinyi Shang, Jiacheng Cui, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao et al.

TL;DR

This paper tackles the problem of pixel-level image tampering detection in the context of modern vision-language models (VLMs), such as ChatGPT, Gemini, and Qwen-Image, which produce highly diverse and evolving manipulated image distributions. The key challenge is domain generalization: ensuring that tampering detectors trained on data from certain VLMs can robustly localize tampered regions on images generated or edited by unseen, out-of-distribution (OOD) models. The authors propose a simple yet effective training framework called PIXAR-DG that improves cross-model robustness without requiring complex model changes. Their approach combines two main strategies: (1) a balanced minibatch sampling scheme that maintains equal proportions of real and tampered images per batch to prevent bias towards either real-image priors or tampering artifacts during training, and (2) a late-injection training method that first trains the detector on large-scale base data until convergence, then fine-tunes with small amounts of data from emerging new VLM domains, enabling adaptation without overfitting. Experiments on the PIXAR benchmark demonstrate that PIXAR-DG significantly outperforms the previous state-of-the-art PIXAR method by large margins (e.g., +26.1% relative improvement in average gIoU) on multiple unseen OOD generators including GPT-Image-2.0, Gemini-3.1, FLUX.2, and Seedream-4.5. Notably, it achieves these advances using only 19.2% of the original PIXAR training data, highlighting the importance of training strategies over mere data scale. Qualitative evaluations show reliable and precise tampered region localization across diverse unseen VLM outputs. Overall, PIXAR-DG offers a practical, conceptually simple recipe for building robust, domain-generalized tampering detectors in rapidly evolving VLM-generated image landscapes.

Key findings

• PIXAR-DG improves average pixel-level localization metrics across four OOD VLM generators by +26.1% relative in gIoU and +26.8% in cIoU compared to baseline PIXAR (Table 1).
• Using only 73K tampered training images (19.2% of PIXAR’s full 380K Qwen-Image set) plus 3K Gemini-2.5 images, PIXAR-DG outperforms PIXAR trained on full data, showing training strategy importance (Section 4.2).
• Balanced minibatch sampling stabilizes training gradients, avoiding collapse observed with naive sampling, confirmed by smoother CLS head gradient norm plots and flatter loss landscapes (Figure 2).
• Late injection strategy—injecting small new-domain data after base training—avoids overfitting and improves cross-domain generalization, validated by ablation results (Table 7).
• Qwen-Image is the best single-source base domain for cross-generator generalization, achieving 60.7% average OOD accuracy, with Gemini-2.5 the best companion source to boost OOD average accuracy from 60.7% to 66.3% (Tables 3 and 4).
• Increasing Qwen-Image training data beyond 70K samples degrades OOD performance, likely due to diluting new-domain signal; thus a data-efficient regime is preferred (Table 5).
• PIXAR-DG’s improvements generalize across model scales: both 7B and 13B variants consistently outperform previous baselines on all evaluation metrics (Tables 1 and 2).
• Qualitative analysis shows more precise and extensive tampering localization by PIXAR-DG than PIXAR on all evaluated OOD generators (Figure 3).

Threat model

The adversary is a malicious entity capable of manipulating images using various modern vision-language models (VLMs), including unseen or newly emerging generators. They aim to create realistic pixel-level tampered images that evade detection. The adversary does not have access to the detector's internal parameters but can generate images from any generator pipeline. The defender cannot rely on target domain supervision and must generalize to unseen VLM manipulation distributions.

Methodology — deep read

The paper addresses domain generalization in pixel-level image tampering detection for modern vision-language models (VLMs) with evolving editing styles. The threat model assumes an adversary can manipulate images using any VLM generator, including unseen or new generator variants, thus forcing detectors to learn generalizable manipulation cues rather than model-specific artifacts.

Datasets originate primarily from the publicly available PIXAR benchmark, consisting of 380K Qwen-Image tampered images and multi-generator test splits including both in-domain (Qwen-Image, Gemini-2.5) and OOD generators (GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5). The final training set subsamples 70K Qwen-Image and 3K Gemini-2.5 tampered images (19.2% of full PIXAR scale) plus balanced real images, with test sets excluding any Gemini-2.5 training data.

Model architecture follows PIXAR baseline: a multi-task detector with shared visual backbone producing (1) pixel-level tampering masks supervised by BCE and Dice losses, (2) multi-label semantic tampering class logits supervised by sigmoid cross-entropy, (3) image real-vs-tampered classification from a CLS token via cross-entropy loss, and (4) a multimodal causal language model generating tampering descriptions supervised by language modeling loss. Total loss is a weighted sum of these components.

Core novel components are two training strategies: (1) Balanced minibatch sampling maintains a configurable ratio (default 0.5) of real and tampered images per batch, ensuring each optimization step receives balanced gradient signals across positive (tampered) and negative (real) samples. This avoids dominant skew towards either distribution seen in naive uniform sampling.

(2) Late-injection optimization splits training into two stages. Stage 1 trains on large base dataset (Qwen-Image 70K) until stable convergence with low constant learning rate (2e-5). At injection step Tinj, Stage 2 adds the small new-domain Gemini-2.5 3K data with mixing coefficient alpha (e.g. 0.1 - 0.5), allowing adaptation while preserving base knowledge and preventing early overfitting to scarce new data.

Training uses LoRA parameter-efficient fine-tuning (rank 8, scaling 16) on SAM mask decoder and task heads with frozen CLIP vision tower and multimodal projector. AdamW optimizer with beta=(0.9,0.95), batch size 8, 5 epochs, 500 steps/epoch on 4x RTX A6000 GPUs is applied. Three random seeds ensure statistical robustness.

Evaluation employs pixel-level localization metrics g-IoU and c-IoU on held-out OOD generators, supplemented by pixel recall, F1, and binary real-vs-tampered accuracy at image level. Baselines include PIXAR, SIDA, and LISA at 7B/13B scales. Cross-generator zero-shot generalization is emphasized.

Ablations remove one or more training components (minibatch sampling, late injection, learning rate) to show their importance in improving OOD pixel-F1 localization. Gradient norm analysis and loss landscape visualization demonstrate training stability improvements from balanced sampling and low learning rate. Qualitative visualizations highlight more accurate tampering region localization vs PIXAR.

Code is publicly released at https://github.com/VILA-Lab/PIXAR-DG to enable reproducibility. While PIXAR dataset is public, some domain splits and exact Gemini-2.5 held-out test partitions may require data access protocols not detailed fully.

End-to-end example: Starting with base tampered/real Qwen-Image data, balanced batches containing equal real/tampered images are formed. The model is trained with low fixed learning rate until stable. Then at the late-injection step, scarce Gemini-2.5 tampered data is added at a mixing ratio alpha, allowing incremental adaptation without disrupting learned generalized cues. Evaluation on OOD GPT-Image-2.0 shows substantially improved g-IoU and tampering mask F1 relative to PIXAR baseline.

Technical innovations

• Balanced minibatch sampling maintaining a fixed real-to-tampered image ratio within each training batch to stabilize gradients and prevent overfitting biases.
• Late-injection training strategy that first trains on a large base-domain dataset, then incrementally introduces small new-domain datasets to improve adaptability without early overfitting.
• Low constant learning rate schedule replacing warmup-decay to preserve transferable pixel-level cues and improve cross-domain robustness.
• Empirical finding that training on a carefully chosen combination of two sources (Qwen-Image base plus Gemini-2.5 companion domain) yields superior OOD generalization.

Datasets

• PIXAR benchmark — 380K tampered Qwen-Image training samples plus multi-domain test splits from Qwen-Image, Gemini-2.5, GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5 — publicly available

Baselines vs proposed

• PIXAR-7B: average gIoU = 0.159 vs PIXAR-DG-7B: 0.201
• PIXAR-7B: average cIoU = 0.166 vs PIXAR-DG-7B: 0.201
• PIXAR-7B: pixel recall = 25.77 vs PIXAR-DG-7B: 45.11
• PIXAR-13B: average gIoU = 0.176 vs PIXAR-DG-13B: 0.222
• PIXAR-13B: average cIoU = 0.183 vs PIXAR-DG-13B: 0.232
• PIXAR-13B: pixel recall = 33.49 vs PIXAR-DG-13B: 62.19
• PIXAR-7B average binary accuracy across OOD: 69.6% vs PIXAR-DG-7B 79.6%
• PIXAR-13B average binary accuracy across OOD: 59.0% vs PIXAR-DG-13B 84.0%
• When retrained on the same Qwen-Image + Gemini-2.5 training data, PIXAR-DG achieves 66.3% average OOD accuracy vs PIXAR 52.0%
• Ablations show that adding balanced minibatch sampling, late injection, and low learning rate schedule increases OOD pixel-F1 from 26.53% to 29.89%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.18230.

Fig 1: Overview of the proposed training framework. Our tamper detector jointly produces a pixel-

Fig 2 (page 3).

Fig 3 (page 3).

Fig 2: Influence of mini-batch sampling and learning rate schedule. (a) Gradient norm of the <CLS>

Fig 3: Qualitative comparison of predicted tampered pixels between PIXAR-DG and PIXAR [22] on the

Fig 6 (page 9).

Fig 7 (page 9).

Fig 8 (page 9).

Limitations

• The experiments are conducted on PIXAR benchmark which, although multi-domain, may not fully cover all possible future VLM manipulations or editing pipelines.
• Performance gains are shown primarily on zero-shot OOD generators; adaptation to completely novel or adversarially crafted VLM manipulations remains untested.
• The late-injection method requires at least some small amount of newly labeled tampered data from emerging domains, limiting applicability when no annotated new-domain data are available.
• The computational cost and memory footprint of training large VLM-based tamper detectors are not extensively discussed, which may limit practical deployment at scale.
• Evaluation does not analyze robustness under adversarial attacks or subtle post-processing perturbations common in real-world forgery attempts.
• The exact generalization capability to closed-source or highly proprietary VLMs not represented in the PIXAR dataset is uncertain.

Open questions / follow-ons

• How well does PIXAR-DG generalize to adversarially perturbed manipulations or subtle post-processing techniques designed to fool tampering detectors?
• Can the proposed balanced minibatch sampling and late injection strategies be adapted or extended to handle video tampering detection or temporal coherence constraints?
• How does the method perform when new unseen VLM distributions emerge that are very different in mechanism or style from existing domains?
• Is it possible to integrate self-supervised or pseudo-labeling techniques to reduce reliance on scarce new-domain labeled data during late injection?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, robust image tampering detection is critical to prevent automated manipulation of visual content used in verification challenges or anti-abuse systems. This paper provides concrete, practical training techniques to improve the generalization of pixel-level tampering detectors across multiple evolving VLMs, which are commonly exploited for synthetic image generation and tampering. The balanced minibatch sampling approach offers a straightforward method to stabilize and improve detector training when real and tampered samples are imbalanced, a frequent scenario in real deployments. The late injection strategy enables incremental adaptation to new manipulation domains without retraining from scratch or catastrophic forgetting, important for maintaining detector efficacy as new VLMs appear. Incorporating such domain-generalized training recipes can strengthen CAPTCHA systems against sophisticated visual forgery attacks that leverage cutting-edge VLMs. While the paper focuses on image forensic localization, the insights on domain-shift and adaptation are broadly applicable to defend against OOD image manipulations in automated bot detection pipelines.

Cite

@article{arxiv2607_18230,
  title={ Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs },
  author={ Yi Tang and Xinyi Shang and Jiacheng Cui and Sondos Mahmoud Bsharat and Jiacheng Liu and Xiaohan Zhao and Tran Dinh Tien and Ahmed Elhagry and Salwa K. Al Khatib and Tianjun Yao and Yonina C. Eldar and Jing-Hao Xue and Hao Li and Salman Khan and Zhiqiang Shen },
  journal={arXiv preprint arXiv:2607.18230},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.18230}
}

Read the full paper

• arXiv:2607.18230 (abstract)
• arXiv:2607.18230 (PDF)