AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis
Source: arXiv:2607.28618 · Published 2026-07-30 · By Bing Yan, Gregory Wolfe, Stefano Martiniani, Kyunghyun Cho
TL;DR
AskChem addresses a key bottleneck in chemistry literature synthesis: scientific findings are scattered across many papers, yet traditional search returns ranked document lists rather than atomic evidence units. This mismatch forces human scientists and AI agents to manually locate, verify, and assemble claims from multiple sources, increasing friction and risk of error. AskChem innovates by making provenance-carrying claims—atomic, typed assertions grounded by DOIs and verbatim quotes—the central retrieval unit instead of entire papers. It builds a shared claim store with 2.4 million claims from 147,000 papers and overlays complementary structures: a stabilized faceted taxonomy for hierarchical browsing, an evidence graph linking claims via typed relations (supports, contradicts, etc.), and an exploratory living taxonomy organizing claims by scientific principles. This infrastructure supports human researchers and AI agents with faceted search, claim-level navigation, and verified provenance.
On cross-paper chemistry questions from AskChem-Bench, grounding a GPT-5.5 reader in AskChem achieves 100% resolvable DOIs (versus 88.3% with reader alone) and the highest citation density (18.1 cited DOIs per answer) across five systems evaluated, indicating much improved factual traceability and relevance. AskChem operates at corpus scale in production, exposing its claim-level data through web, REST, SDK, and Model Context Protocol (MCP) interfaces. This claim-centered approach provides a reusable, verifiable knowledge foundation tailored for literature synthesis rather than just document retrieval.
Key findings
- AskChem indexes 2.4 million provenance-carrying claims from 147,000 chemistry papers spanning 1925–2026, with 99.9% DOI verification and 66% full-text claim extraction depth.
- The evidence graph linking claims via typed relations (supports, contradicts, extends, etc.) contains 171,342 edges with a 97.9% precision on a stratified expert audit of 146 edges.
- On AskChem-Bench (30 cross-paper chemistry questions), grounding GPT-5.5 in AskChem retrieval yields 100% resolvable DOIs versus 88.3% without retrieval, and citation density of 18.1 versus 9.6 for reader alone.
- AskChem outperforms four other systems on mean paper relevance (2.15/3), recent high-impact citation coverage (18.5%), and on-topic answer rate (86.6%).
- AskChem’s taxonomy organizes 307,000 faceted nodes across views such as reaction type, substance, technique, mechanism, claim type, and time, enabling hierarchical browsing and retrieval.
- Living Taxonomy situates 1.1 million claims and 361,000 paper placements under 4,931 principle-centered nodes, supporting exploratory scientific context navigation.
- Hybrid search merges lexical, paper-level, taxonomy-node, and dense-vector recall signals using reciprocal rank fusion for effective claim retrieval.
- AskChem’s API supports accessing claims, evidence neighborhoods, sources, and search results with consistent provenance and structured metadata.
Threat model
Not a security-focused paper; adversarial manipulation or attacks are not explicitly addressed. The system assumes users seek verifiable claims to reduce risks of AI hallucination or human error, relying on provenance for oversight rather than adversarial resistance.
Methodology — deep read
AskChem addresses the problem of chemistry literature synthesis by changing the basic retrieval unit from papers to provenance-carrying claims—atomic, typed assertions extracted from papers and grounded by a source DOI and a verbatim quote or explicit evidence locator. This shifts search and synthesis from document-level navigation to claim-level verification.
Threat model & assumptions: The adversary is not explicitly defined, as this system is designed to improve factual groundedness rather than resist active adversarial manipulation. Instead, the focus is on preventing AI hallucination and human error by surfacing verifiable provenance for each claim.
Data provenance: AskChem indexes claims extracted from 147,000 chemistry papers dated 1925–2026. Claims are typed (e.g., reaction, property, limitation), and each has provenance fields including source DOI, verbatim quote or locator, extraction confidence, bibliographic metadata from OpenAlex, and structured fields like reactants and measurements.
Extraction architecture: Two complementary extraction pipelines were used. An abstract-level extractor uses GPT-5-mini over titles and abstracts to generate claims at scale. A deeper full-text extractor uses Gemini 3.1 Pro to parse entire PDFs, capturing claim types often absent from abstracts. Both models produce structured JSON conforming to a claim schema with checks on provenance and domain-specific fields.
Evidence graph linking employs a second pass relation extraction model that generates typed, directed edges (supports, contradicts, extends, derives_from, cites_as_evidence) with confidence scores between claims. A domain expert audited 148 sampled edges, confirming a 97.9% label precision.
The stabilized faceted taxonomy is induced corpus-wide by mining recurring terminology during claim extraction, followed by canonical top-level routing, synonym normalization, and fuzzy clustering to produce persistent L1/L2/L3 category paths across reaction type, substance class, technique, mechanism, claim type, data, and time facets. This taxonomy drives hierarchical browsing and retrieval grouping.
The living taxonomy organizes claims by scientific principles, models, and mechanisms in a 4,931-node tree, constructed by LLM suggesting placements or proposing new branches when none fit. It provides exploratory scientific context distinct from faceted taxonomy.
Training and extraction use provider-default temperatures and prompt templates with JSON-object-constrained decoding, automatic retries on invalid output, and schema validation. Precision/recall metrics for claim extraction were not reported explicitly.
Evaluation used the AskChem-Bench dataset of 30 cross-paper chemistry questions across condition aggregation, temporal evolution, and contradiction detection. GPT-5.5 answered these questions in five system settings—no retrieval, AskChem retrieval, Paperclip, Edison Scientific’s PaperQA, and Google NotebookLM Deep Research—with judge calibration detailed in Appendix A.
Measures included DOI resolvability via CrossRef, citation density, specificity of quantitative tokens, recent high-impact citation rate, relevance judged by domain experts, and on-topic answer percentages. Reproducibility is supported by public source code, benchmark data, OpenAPI, and dataset snapshots on HuggingFace. The full claim store is queried through a common REST API accessed by web UI, SDK, and MCP server.
One concrete example: for the question about electrocatalysts for CO2 reduction to CO and their Faradaic efficiency, GPT-5.5 alone fabricated 6/14 cited DOIs that failed CrossRef checks, with unverifiable values. When grounded in AskChem retrieved claims, GPT-5.5 cited 22 DOIs all resolvable and verbatim from source abstracts, avoiding hallucination.
Overall, AskChem’s methodology tightly couples claim-level extraction with provenance grounding, evidence relation linking, hierarchical faceted organization, and principle-oriented taxonomies, layered with hybrid lexical/semantic search and human/agent interfaces, tested on cross-paper chemistry synthesis tasks.
Technical innovations
- Transforming the retrieval unit from entire papers to atomic, provenance-carrying chemistry claims anchored by DOIs and verbatim quotes.
- Building a multi-structure shared claim store integrating a stabilized faceted taxonomy, an evidence graph of typed claim relations, and a living taxonomy organizing claims by scientific principles.
- Dual-pipeline claim extraction with LLMs: high-throughput abstract-level extraction (GPT-5-mini) combined with deeper full-text PDF extraction (Gemini 3.1 Pro) capturing more diverse claim types.
- Hybrid claim retrieval combining full-text search, taxonomy node recall, paper-level recall, and dense vector search with reciprocal rank fusion for improved cross-paper synthesis.
Datasets
- AskChem claim store — 2.4 million claims from 147,000 chemistry papers — public index including abstracts and open-access full text where available
- AskChem-Bench v1.1 — 30 cross-paper chemistry questions — public benchmark endpoint and artifacts released
Baselines vs proposed
- GPT-5.5 alone: DOI existence = 88.3% vs GPT-5.5 + AskChem: 100%
- GPT-5.5 alone: citation density = 9.6 DOIs/answer vs +AskChem: 18.1
- GPT-5.5 alone: mean paper relevance = 1.66/3 vs +AskChem: 2.15
- Paperclip: DOI existence = 100% vs +AskChem: 100%
- Edison Scientific PaperQA: higher grounded specificity (29.2 tokens) vs AskChem: 5.9, but lower mean relevance (2.07) vs AskChem (2.15)
- NotebookLM Deep Research: DOI existence = 93.7% vs AskChem: 100%
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28618.
Fig 1: The AskChem interface returns provenance-
Fig 4: The same “CO2 reduction” claims across 10 views: reaction, substance, application, technique, mechanism,
Limitations
- Extraction pipelines rely on LLMs and may produce incorrect or incomplete claim semantics or taxonomy placements despite provenance carrying.
- The provenance checks confirm source grounding but do not guarantee full semantic correctness or factual accuracy of claims.
- AskChem-Bench evaluates only 30 questions, limiting assessment of recall, factual accuracy, and user utility over broader tasks.
- The faceted taxonomy uses string normalization and fuzzy clustering which may merge distinct categories or retain near-duplicates; retrieval gain from taxonomy remains unisolated.
- Living taxonomy placements are exploratory, lacking expert validation or completeness.
- Abstract-only extraction covers 34% of claims; full-text extraction coverage is incomplete, limiting depth for some papers.
Open questions / follow-ons
- How can claim extraction precision and recall be quantitatively improved over entire full texts with minimal human curation?
- What is the quantitative retrieval gain attributable to faceted taxonomy and evidence graph structures relative to lexical/semantic search alone?
- How to rigorously evaluate semantic correctness and factual accuracy of automatically extracted claims at scale beyond provenance grounding?
- Can the living taxonomy be validated and expanded with expert input to better represent scientific principles in chemistry?
Why it matters for bot defense
AskChem's claim-centric retrieval infrastructure demonstrates the benefit of moving beyond document-level search to atomic, provenance-grounded claims with structural layers supporting cross-document synthesis. For bot-defense and CAPTCHA practitioners involved with scientific literature mining or knowledge-grounded agent workflows, it is a case study in designing retrieval systems that ensure factual traceability, provenance verification, and fine-grained evidence connectivity. Its evidence graph and faceted taxonomy concepts could inspire analogous strategies to structure and verify claims or user-generated content in security tasks, enhancing auditability and reducing hallucinations in automated workflows. The use of hybrid lexical and semantic fusion retrieval combined with stabilized metadata paths shows how multi-modal indexing can boost recall and precision for complex domain queries. The open APIs and MCP interface also exemplify how to expose persistent, structured claim objects for agent consumption while retaining provenance.
Cite
@article{arxiv2607_28618,
title={ AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis },
author={ Bing Yan and Gregory Wolfe and Stefano Martiniani and Kyunghyun Cho },
journal={arXiv preprint arXiv:2607.28618},
year={ 2026 },
url={https://arxiv.org/abs/2607.28618}
}