MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

Source: arXiv:2607.05297 · Published 2026-07-06 · By Zefeng Wang, Minxi Yan, Jinhe Bi, Sikuan Yan, Volker Tresp, Yunpu Ma

TL;DR

This paper addresses the problem of recursive self-improvement for large language model (LLM) agents performing long-horizon, open-ended tasks. Current skill-based agent approaches improve task skills (what the agent does) over time via fixed, hand-authored improvement procedures, but do not adapt or evolve the procedures themselves (how the agent improves). MetaSkill-Evolve introduces a novel two-timescale framework where each agent lineage maintains both a task skill and a meta-skill that governs its improvement pipeline. The meta-skill co-evolves alongside task skills using the same five-agent pipeline (Analyzer, Retriever, Allocator, Proposer, Evolver) applied recursively. This enables bounded recursive self-improvement without new models or objectives. By optimizing both task utility and meta-productivity, the authors demonstrate large held-out accuracy gains (+23.54, +16.09, +1.92 points) on three agent-specific benchmarks (OfficeQA, SealQA, ALFWorld) compared to no skill, static skill, and single-level evolution baselines.

Key findings

• MetaSkill-Evolve improves held-out test accuracy over the raw 31B Gemma-4 backbone by +23.54 points on OfficeQA, +16.09 on SealQA, and +1.92 on ALFWorld.
• Its slow meta-skill evolution loop contributes +6.38 / +8.05 / +1.92 points on these benchmarks beyond single-level task-skill evolution alone.
• Component ablations show all five meta-skill parts (Analyzer ψ, Retriever σ, Allocator α, Proposer π, Evolver ε) contribute; allocation α dominates on OfficeQA (-19.74 pts accuracy when removed), while proposal π dominates on SealQA (-8.42 pts).
• Using a meta-update horizon H=2 (fast meta-skill updates) yields 9.1 points higher accuracy on OfficeQA than H=8 (slow updates).
• The persistent directed acyclic graph (DAG) evolution archive preserves provenance and enables cross-branch retrieval, which improves performance over no cross-branch sharing.
• Meta-aware frontier selection scoring combines task skill utility U(s), meta-productivity P(m|s), and novelty N(b), steering search towards productive and under-explored branches.
• On ALFWorld, where the backbone is near ceiling, static skill and single-level evolution provide little gain, but meta-skill evolution still adds +1.92 points.
• The recursive self-improvement is bounded to one level of recursion with no extra models or objectives, applying the same pipeline to refine both task and meta skills.

Threat model

n/a — The paper does not explicitly define a security threat model or adversary. The focus is on improving agent self-improvement capability rather than defending against adversaries.

Methodology — deep read

1. Threat Model & Assumptions: The paper targets LLM agents performing procedural, multi-step tasks where reusable skills guide behavior. The adversary is not explicit; the focus is on improving agent task performance and self-improvement procedures rather than security adversaries.

2. Data: Experiments use three public benchmarks—OfficeQA, SealQA (recent agentic QA benchmarks), and ALFWorld (embodied agent task environment). Each benchmark dataset is split by stratified sampling into train (for failure mining), validation (for scoring child skill variants), and held-out test sets (never seen during evolution). Training uses failure traces collected from executing skills on tasks.

3. Architecture / Algorithm: The core novelty is a five-agent pipeline implemented atop a frozen Gemma-4 31B LLM backbone. The five agents are Analyzer (ψ), Retriever (σ), Allocator (α), Proposer (π), and Evolver (ε), each defined by Markdown skill specification files.

Each branch state carries a task skill s and a meta-skill m = (ψ, σ, α, π, ε), both encoded as Markdown skill files. The fast loop evolves task skills on each iteration by diagnosing failures, retrieving inspirations, allocating search budget, proposing edits, and evolving the skill files. The slow loop every H iterations applies the same pipeline recursively to the meta-skill files to evolve meta-skills based on meta-productivity—how productive the improvement operator is in generating better child skills.

4. Training Regime: No model weights are finetuned; all improvements come from evolving task and meta skill files. The fast loop iterates every task skill improvement, and the slow loop triggers meta-skill updates every H=2 fast iterations by default. Runs proceed by selecting promising branches from a persistent DAG archive.

5. Evaluation Protocol: Held-out test accuracy is reported on each benchmark, comparing No-Skill (raw backbone), Static Skill (fixed hand-authored skill), Single-Level Evolution (task skill evolves but fixed meta-skill), and full MetaSkill-Evolve (joint task/meta evolution). Component ablations evaluate the contribution of each meta-skill part. Meta-update horizons are swept to analyze sensitivity.

6. Reproducibility: The paper states the pipeline uses a frozen Gemma-4 31B backbone with no fine-tuning. The persistent evolution state is stored in a SQLite DAG. Code release or dataset details are not specified. Benchmark splits and procedure are described but full reproduction details are unclear.

Example end-to-end: On one fast iteration, the worst failure example from the parent task skill is diagnosed by Analyzer ψ yielding tags, Retriever σ finds related branches, Allocator α sets child proposal count, Proposer π generates edits, Evolver ε applies and verifies edits, children are evaluated on validation data, improvements committed to the graph, frontier updated by η-weighted U(s), P(m|s), and N(b). Every H iterations, the slow loop forms a meta-failure trace, applies the same pipeline to the meta skill files to produce updated meta-skill m, closing the recursive self-improvement loop.

Technical innovations

• Two-timescale skill evolution framework co-evolving task skills and meta-skills that govern the improvement pipeline.
• Encoding meta-skills as the same Markdown skill files as task skills, enabling recursive self-improvement via the identical five-agent pipeline.
• Meta-aware frontier selection that scores candidate parents by weighted combination of utility, meta-productivity, and branch novelty to balance exploitation/diversity.
• Use of a persistent evolutionary DAG with lineage and inspiration edges for full provenance, cross-branch retrieval, and non-beam search exploration.
• Adaptive child budget allocation (Allocator α) and cross-branch sharing (Retriever σ) components extending fixed analyze–propose–evolve skill rewrite systems.

Datasets

• OfficeQA — size unspecified — Proc. agentic QA benchmark
• SealQA — size unspecified — Proc. agentic QA benchmark
• ALFWorld — size unspecified — Embodied agent environment dataset

Baselines vs proposed

• No-Skill: OfficeQA accuracy = 31.78% vs MetaSkill-Evolve = 55.32%
• Static Skill: OfficeQA accuracy = 36.09% vs MetaSkill-Evolve = 55.32%
• Single-Level Evolution: OfficeQA accuracy = 48.94% vs MetaSkill-Evolve = 55.32%
• No-Skill: SealQA accuracy = 29.17% vs MetaSkill-Evolve = 45.26%
• Static Skill: SealQA accuracy = 29.41% vs MetaSkill-Evolve = 45.26%
• Single-Level Evolution: SealQA accuracy = 37.21% vs MetaSkill-Evolve = 45.26%
• No-Skill: ALFWorld success rate = 92.31% vs MetaSkill-Evolve = 94.23%
• Static Skill: ALFWorld success rate = 90.38% vs MetaSkill-Evolve = 94.23%
• Single-Level Evolution: ALFWorld success rate = 92.31% vs MetaSkill-Evolve = 94.23%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.05297.

Fig 2: System overview. The branch state b = (s, m, h) (left) feeds the five-agent pipeline (centre), whose output

Fig 3: End-to-end held-out test accuracy on the

Fig 4: shows the component ablations on the

Fig 4 (page 5).

Fig 5 (page 5).

Fig 6 (page 5).

Fig 7 (page 5).

Fig 8 (page 5).

Limitations

• Evaluated only on three curated benchmarks; applicability to open-ended, noisy real-world tasks remains untested.
• The fixed five-agent pipeline architecture is not itself evolved or optimized; only skill files evolve within the existing roles and workflow.
• Meta-updates happen at fixed intervals (meta-update horizon H) which may not adapt dynamically to changing improvement dynamics.
• No adversarial robustness or security analysis; the system assumes cooperative improvement rather than hostile manipulation.
• Limited detail on reproducibility; code or dataset splits are not publicly released as of the paper.
• No experiments on longer or multi-level recursion beyond the bounded one-level recursion demonstrated.

Open questions / follow-ons

• Can the two-timescale recursive framework extend beyond one level of recursion while remaining stable and efficient?
• How well does MetaSkill-Evolve generalize to noisy, real-world tasks and long-horizon environments with sparse feedback?
• Could the fixed five-agent pipeline be co-evolved or dynamically reconfigured alongside the skills for greater performance gains?
• What are the optimal strategies or adaptivity criteria for setting the meta-update horizon H depending on task dynamics?

Why it matters for bot defense

For bot-defense and CAPTCHA engineers, MetaSkill-Evolve represents an advanced paradigm where autonomous agents can recursively improve not only their task skills but also the procedures by which they self-improve. This meta-level adaptation could make attacker bots progressively more resilient and stealthy, adapting their internal failure diagnoses and patching logic. Understanding such recursive self-improvement systems highlights the potential evolving threat models where malicious AI agents optimize their avoidance or bypass strategies autonomously over time. Conversely, similar recursive co-evolution techniques could be adapted in defensive bot detection and challenge generation pipelines to evolve more robust, adaptive CAPTCHA or proof-of-humanity skills, staying ahead of evolving attack algorithms by meta-optimizing defense procedures themselves.

Cite

@article{arxiv2607_05297,
  title={ MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution },
  author={ Zefeng Wang and Minxi Yan and Jinhe Bi and Sikuan Yan and Volker Tresp and Yunpu Ma },
  journal={arXiv preprint arXiv:2607.05297},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.05297}
}

Read the full paper

• arXiv:2607.05297 (abstract)
• arXiv:2607.05297 (PDF)