RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning

Source: arXiv:2607.28156 · Published 2026-07-30 · By Jingxiang Fan, Junbao Zhuo, Bochao Zou

TL;DR

This paper addresses a critical bottleneck in long-horizon multimodal reasoning—how to reliably retrieve relevant evidence from extensive video memory for complex, multi-step tasks. Prior methods focus mostly on what information to store in memory but pay less attention to improving how retrieval is conducted or adapting retrieval strategies based on past successes and failures. The authors introduce Reflective Retrieval Memory (RRM), a framework that augments an entity-centric multimodal memory graph for the current video with a distinct reflective experience memory. This reflective memory distills procedural retrieval knowledge from historical task trajectories, capturing reusable search strategies rather than task-specific facts or answers. RRM leverages both successful and failed retrieval experiences, converting them into query-level guidance that shapes future retrieval attempts but does not inject any historical facts directly into the answer generation context.

The system employs an online lifecycle management strategy that prunes or consolidates experience records based on reuse, task feedback, and temporal decay, keeping the experience memory compact and relevant. Experimental results across three long-video multimodal reasoning benchmarks—M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long—demonstrate consistent performance improvements over strong agent baselines, notably a 9.1%, 5.8%, and 7.4% absolute accuracy increase respectively. Ablations show distinct gains from online query reflection (fixing retrieval errors within a task), reflective experience memory (cross-task retrieval strategy transfer), and lifecycle management. RRM also reduces average retrieval rounds by 17-26%, indicating more efficient evidence acquisition. Overall, the paper makes a compelling case for learning and leveraging procedural experience to improve retrieval in long-horizon multimodal reasoning.

Key findings

• RRM improves accuracy by 9.1%, 5.8%, and 7.4% absolute points on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long respectively compared to its M3-Agent baseline (Table 1).
• Online Query Reflection module alone yields 4.4%, 1.6%, and 2.2% accuracy gains on the three datasets, indicating within-task retrieval fix benefits (Table 2).
• Reflective Experience Memory adds an additional 3.1%, 2.8%, and 3.9% improvement on top of query reflection (Table 2).
• Lifecycle Management contributes further gains of 1.6%, 1.4%, and 1.3%, showing that maintaining experience quality boosts performance (Table 2).
• Query-only reuse of reflective experience (using procedural guidance for constructing retrieval queries) outperforms prompt-level injection (injecting experience into the answer context) by up to 4.8% on M3-Bench-Robot and Video-MME-Long (Fig 3a).
• RRM reduces average retrieval rounds needed by 16.8%, 21.8%, and 25.7% on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long respectively, improving retrieval efficiency (Fig 3b).
• Experience memory is strictly isolated from factual memory to avoid leaking irrelevant historic facts, preventing contamination of current-video reasoning with prior-task information.
• Separate successful and failure experience memories allow RRM to prioritize reliable retrieval strategies and selectively use failure diagnostics with stricter applicability.

Threat model

The adversary corresponds to the intrinsic complexity of long-horizon multimodal reasoning tasks, where retrieval failures arise due to sparse, redundant, or distributed evidence in long videos. The system must contend with incomplete or misleading retrievals without contaminating current reasoning with historical task-specific facts. The agent cannot directly transfer factual content from past videos but may leverage procedural knowledge distilled from prior retrieval successes and failures to adapt future search strategies.

Methodology — deep read

1. Threat Model & Assumptions: The adversary in this context is the inherent difficulty in long-horizon multimodal reasoning over long videos where critical evidence is sparsely distributed and historical retrieval attempts can fail or retrieve irrelevant facts. The system assumes access to continuous multimodal video streams with entity-centric memory graphs constructed from episodic and semantic memories of the current video. The agent must avoid contamination by historical video content from other tasks while leveraging retrieval strategies learned across tasks.

2. Data: Evaluations used three long-video multimodal reasoning benchmarks: M3-Bench-Robot (100 robotic-perspective videos), M3-Bench-Web (920 open-domain web videos), and Video-MME-Long (an extremely long-video dataset). Each dataset includes multi-event, multi-hop, cross-modal, person understanding, and general knowledge reasoning categories. Question sets are processed in mini-batches of 64, with all questions from the same video assigned to the same batch. Ground-truth feedback is delayed until all mini-batch predictions are finalized.

3. Architecture / Algorithm: RRM extends the M3-Agent architecture by augmenting its current-video factual memory—which includes episodic and semantic entity-centric memory graphs—with a third layer: reflective experience memory. Reflective experience memory compresses historical retrieval trajectories into procedural records encoding evidence requirements, retrieval patterns, failure modes, and query adjustment strategies. This memory is kept separate from factual memories to prevent leak of historical facts.

RRM maintains two reflective experience banks: successful experiences M+ and failure experiences M−, each containing structured procedural records. Experience records include metadata such as task type and applicability conditions. At retrieval time, RRM performs Online Query Reflection (OQR) to correct current retrieval failures within the task using only current-query state, generating repair queries.

If retrieval anomalies persist, RRM triggers reflective experience reuse by retrieving applicable prior experiences from M+ or—if none found—from M−. The selected experience is converted into a retrieval focus that modifies query construction but is never fed directly into the answer generation prompt. The controller aggregates local repair queries, reflective retrieval focus, and auxiliary keys for the search. New retrieval evidence is then obtained from the current-video factual memory graph.

4. Training & Updating Regime: Reflective experience memory is updated online after task completion. Each mini-batch's retrieval trajectories are summarized into procedural experiences based on task success or failure, but only after all predictions in the mini-batch are judged, ensuring no within-video feedback leakage. Successful task experiences encode effective retrieval strategies, while failure experiences encode diagnostic retrieval anomalies and corrective strategies.

Lifecycle management dynamically maintains experience utility by monitoring reuse frequency, retrieval success, and applying temporal decay. Similar successful experiences are periodically merged, ineffective or redundant entries pruned, protecting memory compactness.

5. Evaluation Protocol: Performance metrics are accuracy scores across reasoning categories and datasets. Comparisons are against state-of-the-art baselines including large MLLMs, online long-video understanding models, and agent-based long-term memory methods. Ablation experiments quantify contributions of each RRM component: Online Query Reflection, Reflective Experience Memory, and Lifecycle Management. Additional analysis compares query-only guided experience reuse vs prompt-level injection. Retrieval round efficiency is also assessed. Three runs and mean results ensure statistical reliability.

6. Reproducibility: Detailed prompts, LLM configurations, structured schema for experience extraction, applicability conditions, and lifecycle hyperparameters are provided in the appendix. Source code availability is not mentioned in the paper, leaving reproducibility somewhat unclear.

A Concrete Example: For a task querying a long video, RRM first uses the M3-Agent Search–Answer process over factual episodic and semantic memory. If retrieval queries return no new or relevant evidence (anomaly detected), Online Query Reflection generates a neutral corrected query without historic experience. If anomalies persist, reflective experience memory is queried: it returns a procedural guidance record learned from previous successful or failed tasks (e.g., search subsequent temporal segments after localizing an event), which refines query construction. Thus, retrieval adapts procedurally across tasks without contaminating answers with prior videos' content. Upon task completion and feedback, RRM distills new retrieval strategy records from the retrieval trajectory for future reuse, maintaining them with lifecycle management.

Technical innovations

• Introduction of reflective experience memory that distills and stores procedural retrieval strategies from historical task trajectories distinct from factual memory.
• Separation of retrieval experience into successful and failure memories with structured procedural records encoding applicability, retrieval patterns, and correction strategies.
• Query-level retrieval control using reflective experience guidance that influences only retrieval queries, preventing direct injection of historical facts into answer generation.
• Online lifecycle management of reflective experience memory based on reuse feedback, temporal decay, and redundancy pruning to maintain memory compactness and reliability.
• Online Query Reflection (OQR) module for within-task retrieval anomaly detection and local repair using only current-task state before consulting cross-task experience.

Datasets

• M3-Bench-Robot — 100 videos — real-world first-person robotic perspective videos
• M3-Bench-Web — 920 videos — diverse open-domain web videos
• Video-MME-Long — size unspecified — very long video dataset published by Fu et al. 2025

Baselines vs proposed

• M3-Agent baseline accuracy: 30.5%, 48.6%, 56.4% vs RRM accuracy: 39.6%, 54.4%, 63.8% on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long respectively (Table 1 and 2)
• Prompt-level injection accuracy: 34.9%, 50.2%, 58.6% vs RRM query-only reuse accuracy: 39.6%, 54.4%, 63.8% (Fig 3a)
• Average retrieval rounds reduction from 2.55 to 2.12, 2.02 to 1.58, and 1.52 to 1.13 on the three datasets representing 16.8%, 21.8%, and 25.7% decrease when using RRM (Fig 3b)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28156.

Fig 1: Motivation and retrieval workflow of RRM. Procedural retrieval experience distilled from prior successful and failed

Fig 2: Overall architecture of RRM: (a) current-video factual memory, (b) reflective retrieval control with Online Query

Fig 3: Experience reuse effectiveness and retrieval effi-

Limitations

• The reflective experience memory relies on the quality and variety of historical task trajectories; limited or biased experience may impact generalization.
• Stricter applicability conditions on failure experiences may limit the usefulness of failure-mode procedural knowledge in diverse scenarios.
• The system assumes availability of reliable post-task feedback (ground-truth answers) to distill experiences; less supervised settings remain unexplored.
• No explicit adversarial or robustness evaluation against intentional retrieval manipulation or noisy video inputs.
• The evaluation focuses on specific long-video benchmarks; generalization to other multimodal or embodied reasoning domains is not demonstrated.
• Reproducibility is constrained by lack of publicly released code or weights, and some implementation details are deferred to appendix.

Open questions / follow-ons

• How well does reflective retrieval memory generalize to other domains beyond long-video reasoning, such as embodied AI or interactive multimodal tasks?
• Can the framework be extended to settings without reliable ground-truth feedback or with noisy, uncertain task supervision?
• What is the impact of adversarially manipulated retrieval anomalies or corrupt past experiences on retrieval strategy learning?
• How to interpret or inspect the procedural experience records to better understand learned retrieval patterns and failure diagnostics?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, the RRM framework provides a compelling methodology for improving long-horizon retrieval in complex, multimodal agents. While CAPTCHAs themselves are often short and self-contained, defense mechanisms that correlate behaviors over extended sessions or across diverse modalities could benefit from procedural experience memory analogs—learning from past retrieval/failure patterns to better diagnose and adapt retrieval queries.

More broadly, RRM’s approach of separating procedural retrieval knowledge from factual content to avoid contamination could inspire CAPTCHAs and bot-detection systems to refine how they aggregate and use historical interaction data. The lifecycle management of experience memory to reduce redundant or noisy guidance also aligns with maintaining robust detection over time. However, direct application would require adapting from multimodal video reasoning to challenges in user behavior tracking or response patterns across sessions.

Cite

@article{arxiv2607_28156,
  title={ RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning },
  author={ Jingxiang Fan and Junbao Zhuo and Bochao Zou },
  journal={arXiv preprint arXiv:2607.28156},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28156}
}

Read the full paper

• arXiv:2607.28156 (abstract)
• arXiv:2607.28156 (PDF)