Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion

Source: arXiv:2607.28058 · Published 2026-07-30 · By Henglin Liu, Fangyuan Kong, Jing Wang, Yizhou Lin, Nisha Huang, Chang Liu et al.

TL;DR

This paper addresses persistent challenges in text-to-video diffusion generation related to temporally sparse visual artifacts such as motion collapse, object flickering, and color oversaturation that degrade perceived realism. The authors identify two main bottlenecks in existing preference optimization approaches: (1) a preference attribution bottleneck where collecting human annotations is costly and online reward models are unstable or biased; and (2) temporal credit misallocation where uniform supervision cannot focus on brief failure-prone video segments. To overcome these, the authors propose concentrated Implicit Preference Optimization (cIPO), a post-training framework that constructs implicit preference signals directly from the model's own denoising rollouts without external labels or reward models. By comparing a real video with its noisy-then-denoised reconstruction, cIPO derives preference pairs that naturally capture inference-time errors. Furthermore, cIPO detects temporally localized high-error segments and concentrates its optimization on these failure-critical windows rather than uniformly supervising all frames. Experiments on the MotionBench and WISA datasets demonstrate consistent improvements in both video authenticity and temporal motion consistency over strong baselines including DPO and DenseDPO, validating the effectiveness of implicit preference construction and temporal concentration for artifact correction.

Key findings

• cIPO improves authenticity scores (Forensic) on MotionBench from 0.819 (DenseDPO) to 0.876 and OmniAID-Dino from 0.475 to 0.524.
• On WISA, cIPO achieves a Forensic score of 0.931 compared to 0.911 (DenseDPO), with temporal quality overall moving from 0.223 to 0.247.
• Reconstruction-based negative samples from diffusion perturbations outperform externally generated negatives by 1-3% in authenticity and motion scores while reducing training time due to fewer denoising steps.
• Temporal concentration of optimization on the highest-error contiguous window improves authenticity scores by up to 1.4% compared to uniform optimization (Table 3).
• Moderate noise levels (e.g., starting noise step s=10 or 20) yield the best balance between detail preservation and temporal consistency (Fig. 6 and 7).
• Applying the clean first-frame anchoring reduces video structure collapse and improves stability during training.
• The winner-preservation regularizer (ReLU term in Eq. 11) prevents degradation of already well-generated frames during preference updates.
• Qualitative results (Fig. 5) show cIPO reduces temporal flickering, motion blur, and anatomical distortions compared to DPO and DenseDPO.

Threat model

The adversary in this context is the inherent limitations and failure modes of the video diffusion model itself during its multi-step denoising rollout, which accumulates drift resulting in temporal artifacts such as flickering or motion collapse. The framework assumes no access to costly human preference labels or external reward models, instead relies solely on internal implicit rollout errors to expose and correct failure segments. No external attackers or malicious model manipulation scenarios are considered.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is implicitly the video diffusion model itself in rollout generation, exposing failures as temporal artifacts during inference. The framework assumes no direct access to human preference annotations or trusted external reward models, emphasizing self-contained preference signals. The model's inference-time denoising trajectory deviations serve as an implicit error source.

2. Data: Training utilizes MotionBench and WISA datasets. MotionBench focuses on motion realism in text-to-video generation, while WISA emphasizes physics-aware video scenarios. WISA is partitioned by authors into training and testing splits since official splits are not provided. Real videos and their latent encodings form the basis of preference pairs.

3. Architecture / Algorithm:

• Videos x are encoded into latent space z0 via a pretrained video VAE encoder E.
• Forward noise is added at timestep s to obtain z_t_s.
• Iterative reverse denoising generates a reconstruction \tilde{z}0^(s) via the trainable diffusion model f\theta.
• Implicit preference pairs formed: original z0 as preferred sample y+, reconstruction \tilde{z}_0^(s) as dispreferred sample y−.
• Apply 'clean first-frame anchoring' by replacing the first frame of y− with the clean latent frame to stabilize spatial structure.
• Compute framewise latent reconstruction discrepancies d_\tau between y+ and y−.
• Identify a contiguous temporal window W* with highest average reconstruction error.
• Concentrate preference optimization losses on this window only to focus training on difficult segments.
• Define a pairwise preference logit g based on relative reconstruction loss improvements versus a frozen reference model.
• Optimize model f_\theta using a DPO-style negative log-sigmoid loss over the constructed preference pairs limited to the hard segment.

4. Training Regime: Fine-tune on Wan-VACE foundation video diffusion model using LoRA adaptation for T2V pathway. Training runs with bf16 mixed precision, learning rate 5e-6, gradient accumulation, and EMA stabilization. Sampling steps for reconstruction negatives range from 5 to 20 forward noise steps. The preference sharpness coefficient (β) and winner-preservation regularization (λ) are tuned empirically.

5. Evaluation Protocol: Evaluate on held-out prompts from MotionBench and WISA. Metrics include Forensic-Chat and OmniAID-Dino for frame authenticity, and VBench suite for temporal motion quality measuring background/subject consistency, flickering, and overall smoothness. Baselines include pretrained Wan-VACE, offline DPO with VideoReward supervision, offline DPO with oracle (GT) positives, and DenseDPO with dense temporal annotations. Ablations examine negative sample construction (noise perturbation vs external negatives) and temporal supervision concentration effects. Noise addition step s spacing is studied.

6. Reproducibility: The paper references a project page but does not explicitly state full code or model release. Training details and datasets are public or constructed from public data. Specific hyperparameters and algorithmic steps are detailed to enable re-implementation. Some dataset splits (WISA) are author-constructed.

Concrete Example End-to-End: Given a clean real video clip, the model encodes it into latent z0. At noise timestep s=10, it applies forward noise to z0 (z_ts), then runs about 20 reverse diffusion steps to reconstruct a denoised latent \tilde{z}_0. The original z0 and the reconstructed latent \tilde{z}_0 form a preference pair with the first frame of \tilde{z}_0 replaced by the clean frame. Framewise L2 distance between these latents identifies the most corrupt K=5 contiguous frames. Concentrating loss only on these frames, the model optimizes to reduce the reconstruction error gap relative to a frozen baseline denoiser, improving temporal fidelity during subsequent rollout inference.

Technical innovations

• Implicit preference construction from the model’s own diffusion denoising rollouts avoids costly annotations or unstable reward models.
• Temporal concentration strategy that automatically detects failure-prone contiguous frame windows for focused preference optimization.
• Clean first-frame anchoring in latent space to stabilize video structure and appearance during reconstruction-based negative sample formation.
• Winner preservation regularizer in the DPO-style optimization to prevent degradation of already well-generated video segments.

Datasets

• MotionBench — motion-centric text-to-video benchmark — public
• WISA — physics-aware text-to-video dataset — public with author-constructed split

Baselines vs proposed

• Pretrained Wan-VACE: Forensic = 0.819 (MotionBench) vs cIPO: 0.876
• DPO (Offline)+VideoReward: Forensic = 0.810 vs cIPO: 0.876
• DPO (Offline, GT): Forensic = 0.815 vs cIPO: 0.876
• DPO (Online, GT): Forensic = 0.830 vs cIPO: 0.876
• DenseDPO: Forensic = 0.819 vs cIPO: 0.876
• On WISA dataset, DenseDPO overall temporal quality = 0.223 vs cIPO = 0.247
• Temporal concentration improves Forensic from 0.862 to 0.876 when applied to noise-based negatives (Table 3).

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28058.

Fig 1: Compared with the baseline method, our method does not rely on additional human

Fig 2: Ideal denoising

Fig 3: A few frames exhibit

Fig 4: The figure shows the three-stage pipeline of cIPO. It first builds implicit preference pairs by

Fig 5 (page 2).

Fig 6 (page 2).

Fig 7 (page 2).

Fig 8 (page 2).

Limitations

• Relies on reconstruction error as a proxy for perceptual quality which may not fully capture semantic or subjective preferences.
• Quantitative evaluation focuses on specific benchmarks; broader generalization to diverse real-world videos not shown.
• Does not directly address adversarial robustness or explicit adversarial attack models in video generation.
• Requires careful choice of noise level s; overly strong noise degrades semantic consistency while too little noise yields weak negatives.
• Clean first-frame anchoring, while effective, may limit optimizing temporal dependencies starting frame by frame.
• No explicit human studies reported to gauge perceptual improvements from a subjective user perspective.

Open questions / follow-ons

• Can implicit preference optimization frameworks be extended to explicitly incorporate semantic-level or human perceptual preferences beyond pixel-level reconstruction errors?
• How does cIPO generalize to other video generation architectures or resolutions, particularly for longer or higher-resolution videos?
• Can the temporal concentration strategy be further refined to adaptively select variable-length or noncontiguous frame subsets without harming temporal coherence?
• What are the effects of combining implicit preference signals with limited human annotations or reward-based supervision for hybrid approaches?

Why it matters for bot defense

While this work centers on text-to-video diffusion and temporal coherence improvements, the core principle of deriving implicit preference signals directly from model inference rollouts—without external labels or reward models—has parallels in bot-defense. For CAPTCHA and bot-defense design, leveraging model-internal error trajectories and concentrating detection or training focus on temporally or spatially sparse failure points could inspire novel robustness and attack detection schemes. Moreover, the temporal concentration mechanism suggests that security models focusing on a few critical or failure-prone event windows rather than uniform evaluation might improve accuracy and efficiency. Lastly, the avoidance of costly human labeling while maintaining effective preference feedback is appealing for CAPTCHA systems needing scalable adaptive adversary detection.

Cite

@article{arxiv2607_28058,
  title={ Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion },
  author={ Henglin Liu and Fangyuan Kong and Jing Wang and Yizhou Lin and Nisha Huang and Chang Liu and Xintao Wang and Pengfei Wan and Kun Gai and Xiu Li },
  journal={arXiv preprint arXiv:2607.28058},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28058}
}

Read the full paper

• arXiv:2607.28058 (abstract)
• arXiv:2607.28058 (PDF)