HyperClaim: Fine-Grained Cross-Modal Hypergraph Reasoning for Video Misinformation Detection

Source: arXiv:2607.28375 · Published 2026-07-30 · By Xiangbo Wang, Jiasheng Zhang, Xingtong Yu, Luoqiang Lei, Delvin Ce Zhang

TL;DR

This paper addresses the challenge of detecting misinformation in short-form video news by capturing fine-grained localized authenticity cues that arise from interactions between a claim-like query (e.g., title), textual evidence, and temporally ordered video frames. Existing approaches either perform global multimodal fusion, losing localized inconsistencies, or free-form reasoning which can lack structured evidence representation. HyperClaim introduces a novel sparse heterogeneous temporal hypergraph framework that explicitly models higher-order query-text-frame relations and short-range temporal dependencies. By constructing claim-oriented hyperedges over tokens and frames, applying adaptive soft-incidence reasoning, residual text-video calibration, and discrepancy-aware aggregation, HyperClaim preserves fine-grained cross-modal evidence without collapsing it into global summaries. Experimentally, it achieves state-of-the-art accuracy on three benchmarks (FakeSV, FakeTT, FakeVV), improving 4.3–8.1 percentage points over strong task-aligned baselines (FactGuard, Fact-R1). Visualization of learned attention reveals meaningful token- and frame-level structural evidence paths underlying predictions.

Key findings

• HyperClaim achieves accuracy/F1 of 83.7%/84.2% on FakeSV, 82.0%/79.5% on FakeTT, and 87.3%/86.1% on FakeVV, outperforming FactGuard by 4.4, 6.7, and 4.3 accuracy percentage points respectively.
• Adding video frames to textual input increases FakeSV F1 from 70.1% (text only) to 80.5%, demonstrating multimodal synergy.
• Adaptive soft-incidence hyperedge reasoning and residual text-video calibration in Aether raise FakeSV F1 from 75.7% (static hypergraph) to 81.2%.
• Claim-conditioned discrepancy-aware readout (Cred) improves FakeSV F1 from 73.2% (mean pooling) to 81.2%, emphasizing importance of claim-contextual pooling.
• Sparse cross-modal hyperedges constructed via confidence-aware filtering (similarity threshold δs=0.14, margin δm=0.003) effectively select relevant query–text–frame units.
• Two-stage training with frozen multimodal encoder features first, followed by fine-tuning with LoRA adapters on raw inputs, stabilizes learning of reasoning paths.
• Learned incidence and attention weights can be traced back to localized tokens/frames, enabling structural evidence tracing rather than free-form explanations.
• Zero-shot large multimodal LLMs such as GPT-4o perform substantially worse (e.g., 56.0% accuracy on FakeVV) than HyperClaim’s task-specific approach.

Threat model

The adversary is a content creator distributing real or fake video-news samples where misinformation is embedded through subtle localized cross-modal inconsistencies between titles/queries, textual evidence, and short temporal video segments. The adversary cannot modify the entire content arbitrarily but may reuse authentic footage, alter textual descriptions or timing to mislead. The detector sees only the claim-like query, available textual evidence, and sampled frames but does not access external knowledge or generate open-ended rationales.

Methodology — deep read

The authors tackle video misinformation detection as a closed-input, sample-level authenticity classification problem: given a sample x = (q, E, v) consisting of a claim-like query q (video title or paired text), associated textual evidence E (comments, ASR, OCR, captions), and sampled video frames v, the goal is to classify real vs fake.

They propose HyperClaim, a modular four-stage pipeline: (1) Claim and evidence encoding; (2) Sparse temporal hypergraph formation (H-Forge); (3) Adaptive temporal hypergraph reasoning (Aether); and (4) Claim-conditioned discrepancy readout (Cred).

First, a pretrained unified multimodal encoder (Qwen3-VL-Embedding-2B) extracts token-level embeddings for query and each textual evidence source (dimension 2048) and visual embeddings for sampled frames (one every 30 raw frames, max 16). This preserves fine-grained token/frame granularity while bounding complexity.

H-Forge constructs a sparse heterogeneous hypergraph with three node types: claim tokens, evidence tokens, and sampled frames. It uses four hyperedge families: intra-claim (connects all query tokens), intra-evidence (connects tokens within individual evidence sources), temporal frame windows (overlapping windows of 3 contiguous frames), and cross-modal text–frame hyperedges connecting textual anchors to top-2 matched frames. Cross-modal edges are filtered by confidence-aware criteria on cosine sim scores (threshold δs=0.14, margin δm=0.003) and claim-aware source budgeting maintains at most 48 anchors with 35% reserved for claim tokens.

Aether performs two 512-dim, 8-head hypergraph reasoning layers alternating node-to-hyperedge and hyperedge-to-node message passing with learned soft incidence membership weights that adaptively weight candidate hyperedges. Bidirectional lightweight text-video cross-attention calibrates node features before propagation. A final line-graph layer refines overlapping hyperedge interactions. Residual connections with β=0.5 and dropout regularize learning.

Cred aggregates final node and hyperedge states with learned claim-conditioned attention pooled separately over textual, visual, and hyperedge sets. It models claim-video agreement (element-wise product) and discrepancy (absolute difference) explicitly before a 512-dim GELU MLP classifier predicts authenticity.

Training uses cross-entropy loss with a two-stage optimization: (1) freezing the multimodal encoder and training hypergraph reasoning and classifier over pre-extracted features, then (2) freezing downstream modules and fine-tuning encoder adapters with LoRA on raw inputs. This stabilizes learned evidence routing.

Evaluation is on three benchmarks FakeSV, FakeTT, and FakeVV using the FactGuard temporal protocol (latest 15% of data as test). Metrics are accuracy, precision, recall, and F1. Baselines include conventional discriminative models (BERT, ViT), multimodal and reasoning-centric systems (FactGuard, Fact-R1), and large multimodal LLMs (GPT-4o). All ablations and comparisons use identical splits and fixed random seeds.

The paper presents detailed ablations isolating contributions of input modalities, soft incidence, text–video calibration, claim-conditioned readout, and more. Qualitative evidence path visualizations show localization of high-weight query phrases to textual and video elements.

Reproducibility is supported by PyTorch implementation on NVIDIA A100, fixed seeds, publicly available datasets, and detailed hyperparameters. However, the multimodal encoder Qwen3-VL-Embedding-2B is a proprietary/pretrained model not openly released.

Technical innovations

• Claim-oriented sparse heterogeneous temporal hypergraph representation integrating query tokens, evidence tokens, and temporally overlapping video frame nodes.
• Confidence-aware filtering with similarity threshold and margin for constructing sparse but relevant cross-modal text-frame hyperedges.
• Adaptive soft incidence reasoning module (Aether) combining soft node-hyperedge memberships with residual bidirectional text-video cross-attention calibration.
• Discrepancy-aware claim-conditioned readout (Cred) aggregating textual, visual, and hyperedge states and explicitly modeling claim-video agreement and disagreement.
• Two-stage training regime freezing and fine-tuning pretrained multimodal encoder via LoRA adapters to maintain stable reasoning paths.

Datasets

• FakeSV — size unspecified — public benchmark from Qi et al. 2023a
• FakeTT — size unspecified — public benchmark from Bu et al. 2024
• FakeVV — size unspecified — public benchmark from Zhang et al. 2025

Baselines vs proposed

• Fact-R1: accuracy = 75.6% (FakeSV) vs HyperClaim: 83.7%
• FactGuard: accuracy = 79.3% (FakeSV) vs HyperClaim: 83.7%
• Fact-R1: accuracy = 74.4% (FakeTT) vs HyperClaim: 82.0%
• FactGuard: accuracy = 75.3% (FakeTT) vs HyperClaim: 82.0%
• Fact-R1: accuracy = 81.2% (FakeVV) vs HyperClaim: 87.3%
• FactGuard: accuracy = 83.0% (FakeVV) vs HyperClaim: 87.3%
• GPT-4o: accuracy = 66.6% (FakeSV) vs HyperClaim: 83.7%
• BERT: accuracy = 65.4% (FakeSV) vs HyperClaim: 83.7%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28375.

Fig 1: HyperClaim (ours) and representative baselines on

Fig 2: Overview of HyperClaim. H-Forge constructs a sparse temporal heterogeneous hypergraph from claim-like query

Fig 3: Structural evidence trace produced by HyperClaim. High-weight evidence units connect localized query phrases,

Limitations

• Size and detailed splits of used datasets (FakeSV, FakeTT, FakeVV) are not specified, limiting assessment of generalization and overfitting.
• Evaluation does not report statistical significance or confidence intervals; only fixed seed point estimates are provided.
• No explicit adversarial robustness analysis against intentional manipulation of multimodal coherence.
• The unified multimodal encoder Qwen3-VL-Embedding-2B is a large pretrained model without code or weights released, limiting reproducibility.
• The approach focuses on closed-input setting; does not handle open-ended external evidence retrieval or open-world fact verification.
• Structural evidence traces are routing salience rather than causally verified or human-validated explanations.

Open questions / follow-ons

• How would HyperClaim perform under adversarial or adaptive misinformation attacks designed to fool hypergraph-based representations?
• Can the sparse hypergraph construction and soft incidence mechanisms be generalized to longer videos or streaming scenarios with real-time constraints?
• How sensitive is the approach to the choice of query formulation and to the quality/quantity of extracted textual evidence components such as ASR, OCR?
• What improvements can be achieved by integrating external fact checking tools or retrieval modules while preserving localized higher-order relations?

Why it matters for bot defense

For bot-defense engineers and CAPTCHA practitioners concerned with misinformation spread via video, HyperClaim offers a novel methodological angle: explicitly modeling fine-grained, localized cross-modal and temporal evidence through sparse hypergraphs rather than global fusion. This enables more robust detection of subtle multimodal inconsistencies that might evade simpler heuristics or monolithic embeddings. The adaptive soft incidence and claim-conditioned discrepancy-aware aggregation provide structured and interpretable reasoning pathways, facilitating evidence traceability which is valuable for auditability and threat analysis.

Practitioners could explore incorporating hypergraph-based reasoning modules in multimodal content verification pipelines to improve detection accuracy against nuanced manipulations. However, the approach assumes availability of a pretrained unified multimodal encoder and structured textual evidence extraction (ASR, OCR), which raises engineering challenges. Moreover, it currently targets closed-input classification and does not natively support open-ended knowledge sourcing or adversarial robustness evaluation, areas practical bot-defense systems must address in the wild.

Cite

@article{arxiv2607_28375,
  title={ HyperClaim: Fine-Grained Cross-Modal Hypergraph Reasoning for Video Misinformation Detection },
  author={ Xiangbo Wang and Jiasheng Zhang and Xingtong Yu and Luoqiang Lei and Delvin Ce Zhang },
  journal={arXiv preprint arXiv:2607.28375},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28375}
}

Read the full paper

• arXiv:2607.28375 (abstract)
• arXiv:2607.28375 (PDF)