Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning

Source: arXiv:2607.07708 · Published 2026-07-08 · By Chen Tang, Yizhou Wang, Jianyu Wu, Lintao Wang, Shixiang Tang, Pengze Li et al.

TL;DR

The paper introduces SciReasoner, a novel multimodal scientific foundation model that performs native structural reasoning across diverse scientific domains including proteins, small molecules, and inorganic crystals. The motivation is to bridge the gap between structural representation and explicit scientific reasoning under domain-specific constraints such as stereochemistry, bonding, lattice symmetry, and periodicity. Unlike previous AI systems that often treat structure as secondary textual features or black-box descriptors, SciReasoner discretizes coordinates, topologies, and periodic connectivities into a unified structure-aware vocabulary. Structural tokens become explicit, addressable evidence units within autoregressive reasoning traces, enabling interpretable scientific inference alongside state-of-the-art predictive accuracy.

Empirically, SciReasoner demonstrates improved performance on 67 out of 86 scientific benchmarks spanning biology, chemistry, materials science, and scientific QA tasks. Notably, it improves low-homology protein gene ontology Cellular Component prediction Fmax from 0.42 to 0.55, retrosynthesis accuracy from 0.63 to 0.72 on USPTO-50K, and separates elemental/compound phases and band-gap regimes in inorganic crystals. Double-blind expert evaluation finds that SciReasoner’s reasoning traces are preferred or comparable to those from a cutting-edge large language model in 98% of cases. By tightly integrating explicit structure tokens with natural language reasoning in an autoregressive model, SciReasoner effectively connects accurate predictions to mechanisms grounded in physical and chemical principles.

Key findings

• SciReasoner increases Cellular Component gene ontology Fmax in low-homology proteins (≤30% seq identity) from 0.34 (BLAST) and 0.42 (ESM2) to 0.55.
• On USPTO-50K single-step retrosynthesis, SciReasoner achieves 72% exact match accuracy versus 63% for the prior state-of-the-art RSGPT.
• SciReasoner’s latent space disentangles elemental and compound phases in inorganic crystals and orders polymorphs by band gap with an R²=0.895 for formation energy prediction.
• Across 86 benchmarks, SciReasoner attains state-of-the-art accuracy on 67 tasks, surpassing domain-specific specialists on 26 benchmarks.
• Reasoning traces explicitly link structural tokens (e.g., residues, motifs, coordination sites) to predictions, enabling interpretable scientific inference.
• Reinforcement learning training enhances reasoning coherence, biological plausibility, and functional hypothesis quality in protein GO annotation traces (Overall Quality score increased from 7.77 to 8.33 vs. DeepSeek-V4-Pro at 6.96).
• Attention analysis confirms residue-level focus on DNA-binding sites with AUROC up to 0.91 and 4.2x enrichment over background for DNA-binding GO terms.
• Ablation of structural input tokens causes consistent performance drops across protein, molecular, and materials tasks, confirming reliance on explicit native structure.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is not explicitly defined as this is not a security-focused work; the model assumes input from protein sequences/structures, molecular graphs, and periodic crystal lattices, requiring reasoning over physical and chemical constraints rather than relying on shortcut memorization (e.g., homology or scaffold similarity).

2. Data: Diverse datasets span protein annotations (DeepFRI-GO split with no >30% seq identity overlap), retrosynthesis (USPTO-50K), materials properties, and standard scientific QA benchmarks. Labels include Gene Ontology terms, reaction precursors, material stability and properties. Dataset splits exclude close homologues to test generalization beyond sequence similarity.

3. Architecture & Algorithm: SciReasoner is built on a large language model backbone initialized from Qwen, extended with a unified structure-aware tokenizer that encodes multidimensional coordinates, bonding topologies, stereochemistry, motifs, lattice symmetries and periodic connectivities into discrete tokens. These tokens serve as explicit evidence token units interleaved with language tokens in an autoregressive transformer decoder. The vocabulary construction uses domain-specific encoders: ConfSeq for molecules, Foldseek 3Di for protein structures, and SLICES for crystals, preserving native scientific semantics.

4. Training Regime: Continued pretraining in stages: Stage 1 warm-up training stabilizes the interface between novel structure tokens and language; Stage 2 full-parameter multimodal training on joint structure + language data; Stage 3 annealed training emphasizes QA-style supervision. Post-training involves reinforcement learning (RL) fine-tuning with chain-of-thought reward signals to connect structural token evidence to reasoning traces within each scientific domain, followed by cross-domain consolidation to unify capabilities.

5. Evaluation Protocol: Metrics include Fmax for GO prediction, exact match accuracy for retrosynthesis, AUC and enrichment for DNA-binding site attention, R² and MAE for materials property regression. Comparison baselines span sequence-based methods (BLAST), structure alignment (Foldseek), protein LM (ESM2), specialist domain models (SaProt), contemporary generalist LLMs, and retrosynthesis suites (RSGPT). Ablations remove structural tokens to quantify impact on accuracy and reasoning behavior. Double-blind expert evaluation of free-form reasoning traces assesses input fidelity, biological plausibility, and coherence.

6. Reproducibility: Code and the SciReasoner framework are referenced at SciReasoner.github.io. Specific dataset splits and pretrained models are released, though some domain data may be closed-source. Detailed tokenizer vocabularies and training procedures are described in the supplement. RL training details and hyperparameters are included. Full frozen model weights are not explicitly mentioned but likely available.

Concrete Example: For protein GO Cellular Component annotation, SciReasoner encodes a protein sequence and its 3Di structural tokens, then autoregressively generates a reasoning trace interleaving textual functional hypotheses with explicit residue-level structure tokens. RL enhances attention focused on DNA-binding residues verified by contact maps, leading to an improved Fmax of 0.55 in low-homology regimes versus 0.42 for ESM2. The reasoning trace can be mapped to explicit secondary structure motifs (alpha-helices and beta-turns) and clusters at functional interfaces, confirming interpretable structure-property inference.

Technical innovations

• Unified structure-aware vocabulary tokenizing coordinates, bonding topology, stereochemistry, and periodic connectivities into discrete evidence tokens for autoregressive reasoning.
• Multi-stage continued pretraining aligning novel structure tokens with language semantics and domain knowledge without destabilizing pretrained language representations.
• Self-bootstrapped post-training with reinforcement learning that teaches domain experts to ground structural tokens as explicit evidence in chain-of-thought reasoning.
• Cross-domain reasoning consolidation integrating domain-specific evidence-use behaviors into a unified model capable of interdisciplinary scientific reasoning across proteins, molecules, and crystals.
• Interpretability via reasoning traces that interleave rationales and structural tokens, enabling transparent inspection of the evidence behind predictions.

Datasets

• DeepFRI-GO — protein function annotation dataset with train/test split excluding >30% homologs — publicly available
• USPTO-50K — 50,000 retrosynthesis reactions from patent literature — publicly available
• DUD-E — molecular docking dataset for active/inactive classification — publicly available
• Various protein, DNA, RNA, and materials science benchmark suites totaling 86 scientific tasks — mix of public and original curated datasets

Baselines vs proposed

• BLAST (protein GO Cellular Component Fmax) = 0.34 vs SciReasoner 0.55 in ≤30% sequence identity bin
• ESM2 (protein GO CC Fmax) = 0.42 vs SciReasoner 0.55 in low-homology bin
• RSGPT (retrosynthesis USPTO-50K exact match accuracy) = 0.63 vs SciReasoner 0.72
• SaProt (protein GO Molecular Function Fmax) = 0.67 vs SciReasoner 0.66 overall
• DeepSeek-V4-Pro (protein GO Biological Process reasoning quality) Overall score = 6.96 vs SciReasoner 8.33
• DUD-E AUC (virtual screening) prior best = 0.76 vs SciReasoner 0.76; 5.0% enrichment factor 7.12 vs 7.70

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.07708.

Fig 1: Overview of SciReasoner, a unified multimodal foundation model for native structural

Fig 2 (page 3).

Fig 3 (page 3).

Fig 4 (page 3).

Fig 5 (page 3).

Fig 6 (page 3).

Fig 7 (page 3).

Fig 8 (page 3).

Limitations

• The model’s reliance on structural tokens requires high-quality input structures, which may limit performance on noisy or low-resolution data.
• Evaluation emphasizes benchmark accuracy and reasoning trace inspection but lacks extensive adversarial testing against deliberately corrupted or out-of-distribution structures.
• Cross-domain consolidation is promising but may still suffer from domain interference or representational collapse under heterogeneous scientific modalities.
• The reinforcement learning procedure depends on hand-designed rewards and expert-curated examples, which may not generalize to all scientific tasks.
• Some datasets or benchmarks incorporate previously unseen labels or modalities not fully covered by the tokenizer vocabulary, possibly reducing generality.
• Full model weights and code release status are not completely detailed; reproduction of all training steps might be challenging.

Open questions / follow-ons

• How well does SciReasoner perform on noisy, incomplete or experimentally uncertain structural input data commonly encountered in real-world scientific applications?
• Can the native structural reasoning paradigm be extended to dynamic or time-resolved structural data, such as molecular dynamics trajectories or operando materials studies?
• What strategies can further improve cross-domain generalization without compromising domain expert reasoning fidelity or accuracy?
• How can reinforcement learning reward design be automated or scaled to accommodate a broader range of mechanistic reasoning tasks beyond the current focus domains?

Why it matters for bot defense

From a bot-defense or CAPTCHA engineering perspective, SciReasoner illustrates advanced multimodal reasoning that tightly integrates domain-native structural information with reasoning traces. This approach highlights the potential for future bot detection mechanisms that verify biological, chemical, or materials-property reasoning grounded in structural evidence rather than relying on superficial patterns. Incorporating such transparent, evidence-linked reasoning could increase the difficulty of automated attacks by requiring bots to demonstrate mechanistic understanding tethered to native structure, which is challenging to mimic with text-only or shortcut-dependent models.

Practitioners should note the importance of unified, interpretable token vocabularies and integration of explicit structural constraints to go beyond black-box predictions. However, the computational cost and complexity of such models may limit direct real-time CAPTCHA deployment currently. Nevertheless, SciReasoner’s methodology suggests a promising research direction for multi-modal defenses where challenge tasks require verification of grounded scientific reasoning trace generation, potentially enabling higher-fidelity distinction between humans and bots attempting scientific or biomedical domain CAPTCHAs.

Cite

@article{arxiv2607_07708,
  title={ Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning },
  author={ Chen Tang and Yizhou Wang and Jianyu Wu and Lintao Wang and Shixiang Tang and Pengze Li and Encheng Su and Jun Yao and Jiabei Xiao and Yuqi Shi and Jielan Li and Hongxia Hao and Zhangyang Gao and Fang Wu and Ben Fei and Xiangyu Yue and Pan Tan and Bozitao Zhong and Jinouwen Zhang and Aoran Wang and Yan Lu and Jiaheng Liu and Xinzhu Ma and Liang Hong and Mingyue Zheng and Phil Torr and Bowen Zhou and Wanli Ouyang and Lei Bai },
  journal={arXiv preprint arXiv:2607.07708},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.07708}
}

Read the full paper

• arXiv:2607.07708 (abstract)
• arXiv:2607.07708 (PDF)