EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents

Source: arXiv:2607.28229 · Published 2026-07-30 · By Luigi Sigillo, Matteo Silvestri, Francesco Tabaro, Rajat Bhatnagar, Syed Irtaza Mubashar, Matt Jeffryes et al.

TL;DR

The paper addresses the challenge that life-science AI agents face in retrieving targeted, citable evidence from the rapidly expanding biomedical literature. Traditional resources like Europe PMC offer vast coverage but are designed primarily for human keyword search, returning whole papers rather than concise evidence snippets, which is inefficient for agents with limited context windows. The authors introduce EMBL AI LIBRARIAN, a knowledge layer that enables AI agents to query Europe PMC in natural language and receive ranked, citable evidence snippets rather than full documents. A single LLM orchestrates the knowledge retrieval pipeline by generating complementary subqueries, executing them on Europe PMC live search, decomposing retrieved papers into paragraphs, and then filtering and re-ranking evidence with fine-grained LLM scoring. Evaluation across four benchmark suites—literature synthesis, claim verification, open-domain question answering, and foundational biology tasks—demonstrates consistent and substantial improvements in accuracy and alignment with expert consensus compared to prior baselines using dense retrieval or keyword search. LIBRARIAN boosts Citation F1 by over 16 points on ScholarQABench, increases claim-verification agreement by 5 points on ProClaim-eval, and elevates GPT-5.4’s LitQA2 open-form QA accuracy by nearly 9 points over standard web search. The system is model-agnostic and publicly released. Overall, the paper presents a novel, cost-effective, and transparent retrieval method well-suited for life-science AI agents, highlighting that leveraging live, structured search with LLM-driven orchestration can outperform heavy embedding-index methods.

Key findings

• EMBL AI LIBRARIAN improves Citation F1 on ScholarQABench Bio split by 6.8 points over a strong BM25 baseline indexed on the OpenScholar Data Store, and by 16.8 points over published baselines (73.8 vs 56.7).
• Using LIBRARIAN as the retriever in the ProClaim claim-verification pipeline increases agreement with expert consensus by 5 points on average (0.75 to 0.80), outperforming OpenScholar baseline by 37 points (0.43 to 0.80).
• On the LitQA2 open-form factoid question answering benchmark, GPT-5.4 grounded in LIBRARIAN achieves 78.9% accuracy versus 70.3% with web search and 17.6% with parametric knowledge only (+8.6 points over web search).
• In foundational biology tasks (LAB-Bench), LIBRARIAN improves macro average accuracy by 4.2 points on GPT-5.4, with large gains in sequence manipulation (+11.3 points accuracy) and modest gains across molecular cloning and protocol QA.
• LIBRARIAN achieves higher precision and coverage of relevant evidence by generating multiple complementary keyword-and-field queries from the original natural language question.
• A single LLM controller orchestrates both query generation and fine-grained evidence sentence extraction and ranking, effectively balancing recall and precision.
• LIBRARIAN reuses Europe PMC’s live, fielded search infrastructure and avoids building and serving a costly dense vector embedding index (~744GB required by an OpenScholar dense index).
• Agents equipped with LIBRARIAN show better calibration—reducing hallucinations—by grounding their answers in retrieved evidence.

Methodology — deep read

The paper focuses on enabling life-science AI agents to retrieve concise, citable evidence in response to natural language queries by layering an LLM-driven knowledge interface on top of Europe PMC.

1. Threat model & assumptions: The adversary is not explicitly modeled as this is a retrieval architecture paper. The system assumes AI agents need robust, evidence-grounded knowledge that can be cited. It assumes Europe PMC is a trusted literature source, but the system must handle synonymy and multiple surface forms for biomedical entities to maximize recall.

2. Data provenance, size, labels: Europe PMC holds 40.7 million PubMed abstracts, 11.9 million full-text open access articles, and 1.2 million preprints, covering biomedical and health sciences. The benchmarks used include ScholarQABench (biomedicine, neuroscience, multidisciplinary splits), ProClaim-eval (419 claims with expert-curated consensus labels), LitQA2 (91 full-text questions requiring in-depth retrieval), and LAB-Bench (foundational biology tasks including database queries, protocol questions, sequence manipulation, and molecular cloning).

3. Architecture/algorithm: EMBL AI LIBRARIAN consists of three main stages orchestrated by one LLM:

   • Subquery generation: The LLM converts the natural language input question into a fixed number (N=7) of complementary keyword and fielded queries targeting Europe PMC’s advanced search syntax (fields like title, abstract, gene/protein names, MeSH terms).
   • Paper retrieval and paragraph ranking: Each subquery runs live on Europe PMC's API, returning up to P=50 articles per subquery (max M=N*P=350 records). Duplicate papers are merged. Each retrieved article's full text is parsed into paragraphs, which are scored with BM25 against the original question. The top k=16 paragraphs per paper (always including abstracts) advance.
   • Filtering, re-ranking and evidence extraction: The LLM cross-encodes the original question with each paragraph’s sentences, assigning relevance scores to filter and rank sentences. It emits short sentence IDs for latency and cost efficiency, returning a final ranked list of citable evidence snippets, each with source metadata and the query that retrieved it.

4. Training regime: The paper uses GLM-5 (a 700B parameter model) as the LLM controller, hosted on NVIDIA DGX B200 hardware with vLLM serving. The system is model-agnostic and can swap in different LLMs. Training hyperparameters for the LLM controller are not detailed; the work focuses on prompt engineering and modular pipeline orchestration rather than model training.

5. Evaluation protocol: Four benchmark suites are used:

   • ScholarQABench: ability to synthesize citations over multiple papers; Citation F1 and LLM judge scores are reported.
   • ProClaim-eval: agreement with expert consensus on claim verification labeled SUPPORT/REFUTE/UNCERTAIN; agreement fraction reported.
   • LitQA2: open-form factoid QA needing full text; accuracy, precision, and coverage (abstention rate) reported.
   • LAB-Bench: biology workflow tasks assessing accuracy, precision, and coverage across sub-tasks. Baseline retrievers include dense vector indices (OpenScholar, SciRAG), raw BM25 on OSDS, and web search baselines. Agents are GLM-5, GPT-5.4, and Claude Sonnet 4.6.

6. Reproducibility: The authors release code, prompts, runtime configs, and evaluation pipelines at https://github.com/petroni-lab/librarian. The data from Europe PMC and benchmark datasets are open or linked. The system is designed to be reproducible and integrable by other researchers and practitioners.

Example end-to-end: Given a natural language query, the LLM generates 7 fielded subqueries (e.g., target gene symbols, MeSH terms). These queries run live on Europe PMC. Up to 350 articles are retrieved, parsed into paragraphs, and BM25 scored to select top paragraphs. The LLM then cross-encodes each paragraph sentence with the query to extract relevant evidence sentences. The final output is a ranked list of evidence snippets with paper metadata and the query that found them—ready for downstream agent synthesis or verdict prediction.

Technical innovations

• A single LLM orchestrates the entire retrieval pipeline including generating complementary subqueries, executing them on a live scholarly search engine, and extracting fine-grained evidence at the sentence level.
• Reuses Europe PMC’s live, structured fielded search instead of building and maintaining a costly dense vector index for the entire biomedical literature.
• Combines recall-oriented complementary multiple subqueries with precision-oriented LLM cross-encoding of retrieved paragraphs for fine-grained evidence extraction and ranking.
• Returns citable, compact evidence snippets linked with original retrieval queries and metadata, improving transparency and downstream reasoning.

Datasets

• Europe PMC — 40.7M abstracts, 11.9M full-text articles, 1.2M preprints — open-access life-science literature search engine
• ScholarQABench — thousands of broad science questions with citations — public
• ProClaim-eval — 419 claims with expert ground truth consensus — public
• LitQA2 — 91 full-text factoid questions needing in-depth retrieval — public
• LAB-Bench — biology workflow tasks including protocol and sequence questions — public subset used

Baselines vs proposed

• OpenScholar-70B on OSDS: ScholarQABench Bio Citation F1 = 55.9 vs LIBRARIAN 73.8
• ProClaim pipeline with PubMed+S2 retriever: Claim verification agreement = 0.75 vs LIBRARIAN 0.80
• GPT-5.4 + Web Search: LitQA2 accuracy = 70.3 vs LIBRARIAN 78.9
• OpenScholar baseline: ProClaim agreement = 0.43 vs LIBRARIAN 0.80
• GPT-5.4 Base: LAB-Bench macro accuracy = 50.5 vs +LIBRARIAN 54.6

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28229.

Fig 2: Overview of EMBL AI LIBRARIAN. Starting from a natural-language query, LIBRARIAN

Fig 3: Europe PMC provides both free-text and structured search capabilities. The standard search

Fig 3 (page 24).

Limitations

• Knowledge coverage is limited to Europe PMC, excluding paywalled papers and non-open content.
• Multi-hop or iterative retrieval beyond a single round of complementary subqueries is not yet supported.
• Currently ignores non-textual content like figures, supplementary materials, and tables are handled only as text.
• Evaluation does not test adversarial robustness or retrieval under sophisticated evasive queries.
• The retrieval controller LLM is large and self-hosted, which may limit practical deployment for some users.
• Some downstream agents rely on features absent from retrieved metadata (e.g., impact factors), limiting use.

Open questions / follow-ons

• How to extend LIBRARIAN to support multi-hop or iterative retrieval when initial evidence is insufficient?
• Can integration of non-textual content (figures, tables, supplementary data) improve retrieval and reasoning?
• What is the impact of paywalled literature on recall and how to incorporate heterogeneous data sources?
• How does LIBRARIAN perform under adversarial query attempts designed to mislead evidence retrieval?

Why it matters for bot defense

This paper presents a valuable approach for bot-defense and CAPTCHA practitioners looking to empower AI agents with domain-specific, high-quality knowledge retrieval. Rather than relying on generic web search or large, costly dense vector indexes, LIBRARIAN demonstrates how leveraging existing structured academic search infrastructure combined with LLM orchestration can yield concise, citable, and relevant evidence for life-science queries. For CAPTCHA designers considering challenges modeled on advanced biomedical question answering or verification, this knowledge layer provides a robust backend retrieval service that agents can integrate, improving answer grounding and accuracy. Further, the architecture points to efficient interaction protocols between an agent and a trusted document repository, which could inspire secure challenge designs where bots are assessed based on their ability to retrieve and reason over authoritative evidence. Practitioners should note current limitations around multi-hop retrieval and paywalled content exclusion, which may affect task coverage in real-world settings.

Cite

@article{arxiv2607_28229,
  title={ EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents },
  author={ Luigi Sigillo and Matteo Silvestri and Francesco Tabaro and Rajat Bhatnagar and Syed Irtaza Mubashar and Matt Jeffryes and Daljit Nijjer and Vittorio Perera and Ola Spjuth and Julio Saez-Rodriguez and Melissa Harrison and Fabio Petroni },
  journal={arXiv preprint arXiv:2607.28229},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28229}
}

Read the full paper

• arXiv:2607.28229 (abstract)
• arXiv:2607.28229 (PDF)