Multi-Source and Cross-Scenario Strategy-Guided Code Optimization

Source: arXiv:2607.20353 · Published 2026-07-22 · By Yuwei Zhao, Qianyu Xiao, Ye Cui, Yijun Yu, Yingfei Xiong

TL;DR

This paper addresses key limitations in strategy-guided LLM-based automated code optimization. Prior work (e.g., SemOpt) mines optimization strategies solely from historical commits in a single programming language scenario, limiting coverage and cross-language applicability. MoST innovates by integrating heterogeneous knowledge sources—such as textbooks, documents, and web sources—along with commits, unifying these into "evidence objects" that represent strategies with descriptions, before/after code examples, scenario tags, and source types. MoST then clusters evidence across sources and programming scenarios using a novel self-balanced weighted clustering algorithm to avoid dominance by noisy or high-frequency sources. When target scenario examples are scarce or missing, MoST transfers examples from related scenarios using an LLM-guided example transfer procedure and validates generated static analysis rules using functional checks. This allows high-quality static analysis rules to be generated robustly across scenarios. MoST then applies these rules to locate optimization opportunities and guides LLM patch generation. Empirically, on a large benchmark of 351 historical optimization tasks for C/C++, Python, and Rust, MoST outperforms SemOpt by 24.44%–180.00% in exact developer patch reproduction and 21.88%–37.50% in semantic equivalence. On 15 real-world open-source projects, MoST boosts performance over SemOpt and Codex by up to 717.42% max improvement and 258.17% average. This demonstrates significant gains from incorporating multi-source strategy knowledge and cross-scenario transfer with reliable rule generation.

Key findings

• MoST yields 24.44%–180.00% more exact-match patches and 21.88%–37.50% more semantically equivalent patches than SemOpt across C/C++, Python, and Rust historical benchmarks (151, 150, 50 tasks respectively).
• MoST improves successful optimizations over Direct Prompt by 480.00%–1300.00% and over Retrieval-Augmented Generation by 93.94%–250.00%, showing strategy-guided optimization outperforms direct or example-based prompting.
• The self-balanced weighted clustering to balance evidence sources improves performance by 8.97%–10.71% in exact-match patches over uniform weighting.
• The reliable rule generation procedure involving example transfer and functional validation increases exact-match patch success by 43.59%–50.00% compared to disabling it.
• Rust gains the largest relative improvement over SemOpt (37.50%–180.00%) due to lower prior coverage and benefit from cross-scenario transfer.
• MoST produced 9,735 Semgrep rules for the c__any scenario and substantial numbers for others (e.g., 4,345 rules for rust__any), showing scalable multi-source strategy extraction.
• On 15 real-world projects, MoST achieves maximum runtime performance improvements from 19.72% to 717.42%, exceeding SemOpt and Codex significantly.
• Parameter selection with self-balanced clustering loss matched practical optimization outcomes best, validating clustering hyperparameter tuning.

Threat model

MoST assumes a benign environment aiming to maximize automated code optimization effectiveness by integrating diverse knowledge sources and scenarios. The adversary is not modeled as actively attacking or evading detection; rather, the challenge is to reliably identify and transfer genuine optimization strategies across heterogeneous and previously unseen scenarios. Thus, adversaries cannot craft malicious inputs or manipulate evidence sources to mislead the system.

Methodology — deep read

1. Threat model & assumptions: The system aims to improve automated code optimization by leveraging an adversary-free setting where the goal is to identify reusable optimization strategies across heterogeneous knowledge sources and programming scenario boundaries, not to withstand active adversarial attacks.

2. Data: The data comprises multiple knowledge sources: (a) historical optimization commits across C/C++, Python, Rust gathered from GitHub (totaling 48,440 evidence objects from commits: 27,463 C, 10,073 Python, 7,098 Java, 3,806 Rust) and (b) optimization manuals and documents such as the Intel optimization manual, yielding 189 high-quality document-derived evidence objects. For Rust, a benchmark of 50 carefully curated optimization tasks was constructed from 1,008,176 Rust GitHub commits by filtering and sampling.

3. Architecture/algorithm: MoST introduces the concept of evidence objects e = ⟨P, E, T, y⟩ encoding (P) a natural language strategy description, (E) a before-and-after code example, (T) a scenario tag capturing programming language and architecture, and (y) the source type (commit, document). Evidence objects from heterogeneous sources are normalized with LLM assistance. A novel self-balanced weighted density clustering algorithm clusters evidence objects cross-source and cross-scenario, selecting parameters (document weight, similarity threshold, min cluster size) to preserve scenario-level native cluster counts. The clustering weights differ by source type to properly represent their quality and frequency. After clustering, an example budget per cluster is assigned based on weighted evidence counts.

Phase 3 generates static analysis rules (Semgrep rules) for optimization localization in the target scenario. If a strategy cluster lacks sufficient target-scenario examples, MoST transfers examples from other scenarios using an LLM to translate examples and descriptions appropriately, validating applicability. Generated static analysis rules undergo functional validation to ensure rules match the pre-optimization code and do not match the post-optimization code, rejecting and regenerating failed rules.

In phase 4, MoST scans the target codebase with validated rules to locate optimization candidates and ranks them. An LLM is prompted with the matched code, strategy description, examples, and scenario context to generate optimized patches.

4. Training regime: Not explicitly a model training paper—LLMs are used via API prompting (DeepSeek-V4-Pro primarily, GPT-5.2 additionally for C/C++). Rule generation budgets set to 5 Semgrep rules per example. Temperature set to 0 for deterministic output.

5. Evaluation protocol: Two evaluations—(a) historical optimization reproduction benchmark measuring exact match and semantic equivalence metrics against developer patches on 151 C/C++, 150 Python, and 50 Rust tasks, comparing to Direct Prompt, Retrieval-Augmented Generation, and SemOpt; (b) real-world optimization on 15 projects measuring runtime performance improvements. Ablation studies test impact of weighted clustering and reliable rule generation. Parameter selection uses clustering-only balance loss without rule generation or LLM calls.

6. Reproducibility: Code release status not mentioned explicitly. Dataset partly reused from SemOpt public benchmark; Rust benchmark newly constructed but no mention of public release. LLMs used are commercial or closed source.

Concrete example end-to-end: Taking a loop-blocking strategy described in an Intel optimization manual excerpt with partial C code, MoST converts this document into evidence objects with natural language descriptions and normalized code before/after. It clusters these with commit-derived evidence, assigns strategy clusters a sample budget, then transfers the normalized C loop-blocking example into Rust code via LLM example transfer to generate Rust-specific static analysis rules. After validating these rules functionally, it scans target Rust code to identify matching loops and prompts the LLM with matched code, the loop-blocking strategy description, and examples to generate optimized Rust patches applying loop blocking. This cross-source, cross-scenario strategy reuse significantly improves optimization success compared to SemOpt limited to commit-based, single-language strategy extraction.

Technical innovations

• Introduction of evidence objects to uniformly represent heterogeneous optimization strategy data items from diverse knowledge sources and scenarios.
• Self-balanced weighted density clustering algorithm that adjusts weights of evidence objects from different knowledge sources to prevent domination by frequent but noisy commit data during strategy identification.
• Example transfer procedure leveraging LLMs to adapt strategy examples across programming languages and architectures, enabling effective cross-scenario static analysis rule generation.
• Functional validation of generated static analysis rules by verifying rule matches on before-optimization code and rejections on after-optimization code to ensure high rule quality for guiding LLM code optimization.

Datasets

• SemOpt C/C++ and Python benchmark — 151 C/C++, 150 Python historical optimization tasks — reused from prior work
• Rust benchmark — 50 historical optimization tasks — constructed by authors from 1,008,176 GitHub Rust commits filtered for single-function optimizations
• Intel Optimization Manual evidence objects — 189 document-derived evidence objects — public document

Baselines vs proposed

• Direct Prompt: exact match success = 3-4/151 (varies by language) vs MoST: 56/151 (C/C++), 53/150 (Python), 14/50 (Rust)
• RAG (retrieval augmented generation): exact match success = ~10/151 (C/C++) vs MoST: 56/151
• SemOpt: exact match success = 36/151 (C/C++), 40/150 (Python), 5/50 (Rust) vs MoST: 56/151, 53/150, 14/50
• MoST outperforms SemOpt by 24.44%–180.00% in exact match and 21.88%–37.50% in semantic equivalence across all languages
• Ablation without weighted clustering: exact match drops from 56 to 50 on C/C++ benchmark
• Ablation without reliable rule generation: exact match drops from 56 to 28 on C/C++ benchmark

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.20353.

Fig 2: The overall workflow of MoST. It first constructs evidence objects from multiple sources, then discovers strategy

Limitations

• Rust benchmark is relatively small (50 tasks) and newly constructed, limiting generalization and public reproducibility without dataset release.
• LLM usage depends on proprietary models (DeepSeek-V4-Pro, GPT-5.2) that are closed source; limited insight into failure modes or hyperparameter sensitivity.
• Evaluation focuses on historical commits and performance gains on open-source projects but lacks adversarial robustness or security attack evaluation on static rules.
• Does not address runtime overhead or complexity costs of applying the generated static analysis rules before optimization.
• Cross-scenario applicability depends on scenario definition limited to language and architecture; other axes such as OS or compiler may impact strategy transfer but not explored.
• Functional validation improves rule quality but may miss subtle semantic errors in optimization applicability or fail to guarantee absence of regressions.

Open questions / follow-ons

• Can MoST's multi-source and cross-scenario strategy approach generalize to other programming paradigms or domains, such as functional languages or distributed systems?
• How can adversarial robustness be incorporated into static analysis rule generation to safely handle untrusted or malicious code inputs?
• Would integrating other scenario properties (e.g., OS, compiler versions, runtime environments) improve cross-scenario transfer fidelity and performance gains?
• Can the example transfer and rule validation techniques be combined with dynamic profiling data to guide optimization suggestions more precisely?

Why it matters for bot defense

For bot-defense and CAPTCHA engineering, MoST's approach offers valuable insights on how multi-source knowledge integration, cross-domain transfer, and careful validation can enhance strategy extraction and application. Automated defense component tuning or heuristic patching may benefit from collecting heterogeneous signals (logs, manual rules, web security best practices), unifying them as evidence objects, and clustering them to identify generalized defense patterns applicable across different languages or frameworks. The use of example transfer to adapt rules or countermeasures to varied runtime contexts echoes the need to tailor bot detection heuristics reliably when deploying across diverse web environments. Furthermore, incorporating robust validation steps to ensure rule correctness before enforcement parallels MoST's functional validation, helping prevent false positives and operational issues in security settings.

Cite

@article{arxiv2607_20353,
  title={ Multi-Source and Cross-Scenario Strategy-Guided Code Optimization },
  author={ Yuwei Zhao and Qianyu Xiao and Ye Cui and Yijun Yu and Yingfei Xiong },
  journal={arXiv preprint arXiv:2607.20353},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.20353}
}

Read the full paper

• arXiv:2607.20353 (abstract)
• arXiv:2607.20353 (PDF)