SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

Source: arXiv:2607.15257 · Published 2026-07-16 · By Yuyao Zhang, Junjie Gao, Zhengxian Wu, Jiaming Fan, Jin Zhang, Shihan Ma et al.

TL;DR

This paper addresses the challenge of long-horizon, open-domain information seeking using tool-augmented large language model (LLM) agents. Conventional single- or multi-agent search systems struggle to track search progress and avoid redundant or stalled queries as interaction histories grow, leading to wasted budget and incomplete answers. To solve this, the authors introduce SearchOS, a system-level multi-agent framework that externalizes and persistently maintains search state explicitly rather than implicitly inside agent conversations. SearchOS formulates information seeking as relational schema completion with grounded citations, where agents fill entity-attribute tables linked by keys and cite source evidence for every fact.

The paper presents Search-Oriented Context Management (SOCM), which stores evolving shared state including a prioritized Frontier Task queue, an Evidence Graph of fine-grained citations, a Coverage Map tracking progress per schema cell, and Failure Memory recording unproductive attempts. Built on SOCM, SearchOS uses a pipeline-parallel scheduling scheme that continuously dispatches specialized agents (explore, search, writer) to unresolved schema gaps, improving throughput and utilization. A Search Tool Middleware Harness governs execution with context injection, evidence extraction, and sensors that detect stalls or budget exhaustion, triggering strategic interventions. A hierarchical skill system separates reusable global search strategies from source-specific access skills that evolve over time.

Evaluated on WideSearch and GISA open-domain benchmarks, SearchOS achieves state-of-the-art results, notably raising recall and completeness metrics by large margins. Ablations confirm the benefits of dynamic schema planning, continuous scheduling, middleware governance, and hierarchical skills. The work demonstrates that robust, collaborative, and persistent stateful multi-agent coordination enables more complete, efficient, and verifiable long-horizon information seeking.

Key findings

• SearchOS improves WideSearch Item-level F1 to 80.3, outperforming the best baseline (A-MapReduce) by +4.3 points, primarily driven by recall gains (+5.5 points).
• On WideSearch Row-level F1, SearchOS reaches 56.5 (+2.0 over next-best Web2BigTable), showing improved end-to-end consistency.
• On GISA, SearchOS achieves 76.5 Set-level F1, surpassing the best baseline by +13.4 points, indicating superior ability to enumerate complete answer sets.
• Pipeline-parallel continuous scheduling reduces end-to-end search time by 24.3%, increases slot utilization to 41.7% (vs 34.6% batch), and boosts Item F1 from 79.66 to 86.75.
• The Search Tool Middleware Harness detects stalled loops and triggers strategy switches that restore search progress at early, mid, and late stages of search.
• Dynamic schema planning outperforms fixed single- and multi-table schemas by 8.2 Item F1 points and 7.7 Row F1 points on a 40-task subset.
• Hierarchical search skills raise Item F1 by 2.0 points and Row F1 by 3.4 points while reducing session time by 36.6% and search calls by 39.1%, improving efficiency and quality simultaneously.
• Across experiments, recall and completeness metrics improve more than precision, highlighting success in reducing missing or incomplete information.

Threat model

The authors assume a cooperative but resource-constrained environment where the adversary is not explicitly modeled. The main 'adversarial' challenges come from inherent system failures such as agents forgetting search progress, entering loops, duplicating effort, or exhausting budgets. The framework does not consider malicious actors manipulating evidence or actively attempting to deceive the agents or corrupt the search state.

Methodology — deep read

The authors formulate open-domain information seeking as relational schema completion with grounded citations. The search target is a relational schema S consisting of one or multiple tables, each with attributes and primary keys. The agents must discover entity rows and fill attribute values, grounding every answer in verifiable source evidence stored in a citation matrix. This allows precise measurement of search progress and quality.

They design Search-Oriented Context Management (SOCM) which externalizes and stores four shared memories persistently outside agents: (1) Frontier Task - a prioritized task pool encoding unresolved schema cells with dependencies and retries; (2) Evidence Graph - a graph of atomic evidence nodes linked by support/conflict edges preserving fine-grained provenance; (3) Coverage Map - a cell-level status map indicating filled, missing, uncertain, or unreachable cells supporting progress monitoring; and (4) Failure Memory - stores past failed queries, inaccessible sites, or rejected claims to prevent repeated unproductive attempts.

A centralized orchestrator decomposes requests into schema gaps and schedules specialized agents (explore, search, writer) via SOCM. Unlike batch scheduling, a pipeline-parallel continuous dispatch mechanism immediately refills freed execution slots with new unresolved tasks, improving throughput and reducing idle time.

To govern execution, the Search Tool Middleware Harness intercepts all model calls and tool operations, injecting relevant state/context, anchoring extracted evidence into SOCM, and monitoring metrics like increment in coverage or total evidence. When stalls or budget exhaustion are detected through statistical tests, the harness triggers interventions such as strategy switching or halting expansions.

The system implements a hierarchical search skill library comprising (1) orchestrator skills for global task and schema management, (2) strategy skills encoding source-independent search methods such as query reformulation and multi-hop reasoning, and (3) access skills for source-specific navigation and extraction. Prior experience refines skill routing and reuse.

The backbone models are GLM-5 for agent policies and Qwen3.5-35B-A3B for evidence extraction. Benchmarks include WideSearch (200 multilingual wide collection queries requiring large fact gathering) and GISA (373 human queries with structured multi-hop reasoning). Evaluations use Item-level and Row-level Precision, Recall, F1, and Exact Match, running 3 trials reported at Max@3. Each session allows 50 orchestrator iterations, 8 parallel sub-agents, 20 searches per agent, and 1800 seconds wall-clock budget.

A concrete example follows: For a given open-domain query, the orchestrator constructs a relational schema targeting the information need. The schema gaps form Frontier Tasks that are queued and prioritized. Explore agents identify candidate entities and sources, search agents collect grounded evidence via browser tool calls and populate the Evidence Graph and Coverage Map. Middleware validates each interaction output and monitors for stalls. Upon detecting a search loop, the harness triggers a strategy skill switch. Once coverage plateaus or budget exhausts, a writer compiles the grounded schema into a cited final report.

Ablations validate the impact of dynamic schema choice versus fixed schemas, continuous pipeline scheduling versus batch dispatch, middleware governance enabling loop detection and intervention, and the hierarchical skill system's contribution to quality and efficiency.

Reproducibility is partially supported with public code and dataset links. However, model weights and full skill libraries may be internal to Ant Group. Detailed algorithmic innovation and tooling appear well documented but some large-scale training/process specifics are not fully open.

Technical innovations

• Formulation of open-domain information seeking as relational schema completion with fine-grained citation anchoring, enabling explicit, measurable progress tracking.
• Introduction of Search-Oriented Context Management (SOCM) that externalizes and shares persistent, fine-grained search state across distributed agents via Frontier Task, Evidence Graph, Coverage Map, and Failure Memory.
• Design of pipeline-parallel continuous scheduling of heterogeneous agents that immediately backfills freed execution slots with unresolved tasks, improving throughput and utilization over batch dispatch.
• Development of a Search Tool Middleware Harness that intercepts model and tool interactions to ground evidence, enforce budgets, detect stalled loops, and implement decoupled execution governance outside agent prompts.
• Creation of a hierarchical search skill system separating global orchestrator playbooks, reusable source-independent strategies, and type-specific access skills, enabling skill reuse and improved search efficiency.

Datasets

• WideSearch — 200 queries — publicly curated multilingual open-domain search benchmark
• GISA — 373 queries — general information-seeking assistant benchmark with multi-hop, structured answers

Baselines vs proposed

• WideSearch Item F1: A-MapReduce = 76.0 vs SearchOS = 80.3 (+4.3)
• WideSearch Row F1: Web2BigTable = 54.5 vs SearchOS = 56.5 (+2.0)
• GISA Set F1: Best baseline (Plan-and-Solve) = 63.1 vs SearchOS = 76.5 (+13.4)
• GISA Table Item F1: Best baseline (ReAct) = 74.8 vs SearchOS = 76.9 (+2.1)
• GISA Table Row F1: Best baseline (ReAct) = 58.1 vs SearchOS = 59.7 (+1.6)
• Scheduling Policy, WideSearch Item F1: Batch = 79.66 vs Continuous dispatch = 86.75 (+7.09)
• Skill ablation Item F1: Without skills = 78.3 vs With skills = 80.3 (+2.0)
• Skill ablation Row F1: Without skills = 53.1 vs With skills = 56.5 (+3.4)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.15257.

Fig 1: SearchOS interface for a long-horizon information-seeking task. The workspace exposes the orchestration trace, pipeline

Fig 2 (page 1).

Fig 2: SearchOS architecture.

Fig 4 (page 4).

Fig 5 (page 4).

Fig 6 (page 4).

Fig 7 (page 4).

Fig 8 (page 4).

Limitations

• The approach relies on backbone LLMs (GLM-5, Qwen3.5-35B-A3B) that may not be publicly accessible, limiting replication.
• The Failure Memory and skill refinement depend on offline post-mortem analyses, which may not generalize well to radically new task domains.
• Evaluation uses only two benchmarks (WideSearch, GISA) with fixed query types; robustness under adversarial or out-of-distribution queries is untested.
• The system requires significant engineering complexity including middleware, schemas, and hierarchical skills, possibly hindering deployment or maintenance.
• No explicit adversarial testing is reported, so resilience to intentional misinformation or deceptive sources remains unknown.
• The SearchOS reliance on relational schemas presumes that all information needs can be decomposed structurally, which might not hold for more loosely structured tasks.

Open questions / follow-ons

• How to automatically generate or adapt relational schemas dynamically for arbitrary open-domain queries without human guidance?
• Can the Failure Memory and skill refinement mechanisms be made fully online and adaptive to new task distributions or adversarial conditions?
• What are effective methods to detect and resolve conflicting or deceptive evidence within the Evidence Graph, especially from unreliable sources?
• How will the system scale with substantially larger numbers of agents, longer horizons, or more complex multi-modal evidence beyond text?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work demonstrates how persistent explicit state management can prevent repeated, unproductive queries during complex open-domain information seeking—analogous to detecting and mitigating automated bot loops in access systems. The notion of Failure Memory encoding unsuccessful patterns and Middleware Harness enforcing execution budgets highlights strategies for robust orchestration and behavioral intervention beyond prompt-level controls.

The pipeline-parallel scheduling approach suggests ways to maximize throughput of distributed agents or bots while minimizing idle time and redundant requests, which could inspire more efficient CAPTCHA challenge distributions or rate-limit mechanisms. Finally, the modular, hierarchical skill system aligns with layered defense architectures—separating general vs site-specific tactics—potentially useful in designing adaptable CAPTCHA challenges. Overall, the emphasis on shared, durable, verifiable evidence and coordination across heterogeneous agents offers a rigorous framework for managing complex long-lived interactions with automated clients.

Cite

@article{arxiv2607_15257,
  title={ SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration },
  author={ Yuyao Zhang and Junjie Gao and Zhengxian Wu and Jiaming Fan and Jin Zhang and Shihan Ma and Yao Yao and Weiran Qi and Chuyan Jin and Guiyu Ma and Xingzhong Xu and Kai Yang and Ji-Rong Wen and Zhicheng Dou },
  journal={arXiv preprint arXiv:2607.15257},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.15257}
}

Read the full paper

• arXiv:2607.15257 (abstract)
• arXiv:2607.15257 (PDF)