Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation

Source: arXiv:2607.11288 · Published 2026-07-13 · By Praneeth Narisetty, Shiva Nagendra Babu Kore

TL;DR

This paper introduces Mako, the first instance of a novel agent architecture called the Self-Evolving Agentic Operating System (SE-AOS), designed to autonomously discover, synthesize, verify, and hot-load new exploit capabilities against live web targets without human intervention. The system treats exploit capabilities as a mutable, versioned kernel that evolves at runtime by learning from failures, generating new exploit tools, and validating them in sandboxed environments before integrating them back into the agent's arsenal. Mako was tested on the public XBOW benchmark suite, consisting of 104 containerized CTF-style web apps encompassing 26 vulnerability classes across three difficulty tiers, achieving a verified 100% success rate in autonomously extracting freshly randomized flags from each target under a fabrication-proof verification regime.

The key insight of the work is a formal "law of autonomous exploitation": once an exploit capability exists and is discoverable by the agent, the difficulty of exploiting a target effectively collapses because difficulty is no longer in reasoning ability but availability and selection of capabilities. This capability-centric view is novel, along with the formalism showing guaranteed monotone coverage improvement via a capability-evolution operator Φ. The paper presents a comprehensive methodology for capability evolution, fabricating-proof evaluation with planted flags, and a gated self-improving evolution loop that autonomously refines Mako's own exploit tools and agent logic without manual intervention. Mako operates at commodity cost (approximately $4.61 per target) and uses a combination of Gemini-family LLMs for reasoning, demonstrating that once appropriate capabilities are integrated and discoverable, even the hardest challenges collapse to a few inference turns.

Key findings

• Mako achieves 104/104 full-suite coverage on the XBOW-104 benchmark, spanning 26 vulnerability classes and three difficulty tiers, driving every live target to emit a fresh, per-build cryptographically random flag verified by ground-truth scanning, avoiding fabricated or memorized results.
• Median solve cost is 7 agent turns (mean 10.5; min 2; max 40), with the hardest tier (L3) showing a median of only 2 turns due to highly discoverable, purpose-built capabilities.
• Total compute cost for the full campaign is $478.99 at official Google Gemini API pricing, averaging approximately $4.61 per benchmark target.
• Tool discoverability, not reasoning ability, is the primary bottleneck; adding new tools that the agent rarely selects does not improve performance, but chaining new tools as fallbacks inside umbrella tools and providing explicit, keyword-rich tool descriptions significantly reduces turns-to-solve.
• The capability-evolution operator Φ guarantees non-regressive monotone improvement in coverage, converging to full coverage (100%) on the finite XBOW suite.
• The verification regime uses fresh cryptographically random flags per build and requires that the exact flag string be observed in genuine application responses, with a false positive probability bounded by 2^(-128).
• Mako runs a gated self-evolution loop autonomously proposing, sandbox-validating, and committing improvements to its agent and exploit tools only when no regression in multi-category fitness is observed.
• The hardest challenges (including HTTP request smuggling, deserialization, TOCTOU race) are solved by reasoning that discovers environmental discrepancies and composing general tool capabilities, not by brute force fuzzing.

Threat model

The adversary modeled is an unauthenticated, black-box external attacker with only the base URL of the web target, no source-code access or internal knowledge, and no ability to modify infrastructure beyond normal web requests. The attacker must perform full exploitation including reconnaissance, vulnerability discovery, exploit synthesis, and verification solely through live interaction with the deployed application. The adversary cannot preknow or hardcode flags and cannot cheat by memorization. The defender deploys targets with fresh randomized flags and uses a strict ground-truth verification gate to ensure only genuine exploitation is accepted.

Methodology — deep read

1. Threat model & assumptions: Mako operates as an external, unauthenticated black-box web attacker given only a base URL with no source code access or manual hints. It must autonomously perform reconnaissance, vulnerability classification, exploit synthesis, delivery, and verification solely from live interaction with the target application. The system assumes fresh targets each with a cryptographically randomized flag planted during container build time which it must extract. The adversary cannot insert credentials, modify source code, or cheat by memorizing flags.

2. Data provenance and splits: The evaluation used the public XBOW validation benchmarks—104 isolated, containerized CTF-style web targets covering 26 real-world vulnerability classes (e.g. IDOR, SSTI, SQLi, XSS, deserialization) across three difficulty tiers (L1-L3). Each target is rebuilt for every run with a new random flag injected via Docker build-args. Flags and vulnerabilities are proprietary to those targets; infrastructure fixes were applied if needed but not the logical vulnerabilities or flags.

3. Architecture and algorithm: The core architecture is SE-AOS, which separates a mutable capability kernel (a set of verified exploit primitives/tools) from a reasoning user space (LLM-based agent using Google Gemini 2.5 Flash primarily, Gemini 3.1 Pro for escalation). The capability kernel executes typed, audited tool syscalls. The agent runs a bounded perceive→reason→act loop, issued tool invocations and natural language rationales per turn with a turn budget (default 25). A slower capability-evolution loop diagnoses exploit failures, synthesizes new tools via operator Φ, sandboxes/validates them with planted flags, and hot-loads them into the kernel only on success. The system enforces a fabrication-proof ground-truth verification kernel checking flagged responses, disallowing fabricated flag claims.

4. Training regimen: While the underlying LLMs (Gemini family) were trained externally and used as-is (no model fine-tuning reported), Mako builds and registers approximately 180 tools, including ~50 new or enhanced exploit tools developed through the campaign. Tool integration includes discoverability aids like rich keyword descriptions for matching tool to challenge language.

5. Evaluation protocol: The evaluation tracked every solve for each target against the planted flag under automated, cryptographically strong verification gates. Metrics measured included turns-to-solve per benchmark, cumulative coverage, per-category turn statistics, and compute cost measured by API billing. Baselines from prior literature on XBOW benchmarks are compared (e.g., KinoSec 99%, BoxPwnr 97.1%, Shannon 96.2%). No human-in-the-loop intervention was allowed; only tools passing sandbox validation could be hot-loaded. The evolution operator is proven monotone with coverage guaranteed to converge.

6. Reproducibility: Source code, exploit payloads, proof-of-concept exploits, internal tool source, and per-target exploit chains remain withheld due to dual-use concerns and risk of deployment as weaponized tools. Infrastructure aids (Docker fixes) are fully revertible. All solves on the public XBOW validation benchmarks are logged with timestamps, turns, and randomized flags verified against genuine outputs. Thus, experiments are reproducible in principle but operational secrets are not disclosed.

Concrete example: On a hard target (e.g. XBEN-066 HTTP request smuggling), Mako runs its perceive reason act loop to hypothesize a normalizing proxy interfering with classic smuggling payloads, diagnoses the failure trace, synthesizes a new desync payload adapted to the proxy's behavior, sandboxes it against the live target with a fresh planted flag, and upon verified extraction of the flag, hot-loads this capability. Subsequent runs select this capability immediately, solving the target in 2 turns. The entire process is automated and verifiable end-to-end.

Technical innovations

• Introduction of the Self-Evolving Agentic Operating System (SE-AOS) architecture that models exploit capability as a dynamic, versioned kernel mutable at runtime, extending itself via autonomous failure diagnosis and capability synthesis.
• Formalization of a capability-evolution operator Φ and coverage functional C proving monotone, non-regressive improvement with a coverage fixed point at full suite coverage.
• Fabrication-proof verification regime for autonomous exploitation, using fresh cryptographically randomized flags per build and ground-truth scanning of genuine target responses to prevent fabricated or memorized success claims.
• Gated autonomous self-evolution loop that sandboxes proposed new exploit tools and agent rule changes against live targets and fitness gates, committing only regression-free improvements without manual intervention.

Datasets

• XBOW validation-benchmarks — 104 containerized CTF-style web applications — public
• n/a — n/a — official LLM Gemini models used (no fine-tuning datasets disclosed)

Baselines vs proposed

• KinoSec on XBOW-104 (black-box): 103/104 (99.04%) vs Mako 104/104 (100%) full coverage
• BoxPwnr on XBOW-104 (black-box): 101/104 (97.1%) vs Mako 104/104 (100%) full coverage
• Shannon Lite on XBOW-104 (white-box): 100/104 (96.2%) vs Mako 104/104 (100%) full coverage
• Human combined team on XBOW-104 (black-box): 91/104 (87.5%) vs Mako 104/104 (100%)
• Human principal (expert) on XBOW-104 (black-box): ~85% vs Mako 104/104 (100%)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.11288.

Fig 2: Verified full-suite coverage. All 104 benchmarks were driven to emit

Fig 3: Turns-to-solve distribution. Most solves cluster at 6–10 turns; a

Fig 4: Cumulative solve-efficiency curve: the fraction of the suite solved

Fig 5: Compute economics of the campaign. Left: list-price API cost by model (total $478.99, ≈$4.61 per benchmark). Right: estimated token composition,

Fig 6: The Level-3 inversion. Turns-to-solve by difficulty tier (violin = distribution, points = individual benchmarks, bar = median). The hardest tier (L3) has

Fig 7: Coverage by primary vulnerability class (bar length = benchmark count; colour = median turns-to-solve), spanning 14 primary classes.

Fig 8: Difficulty × vulnerability-class heatmap. High-effort cells (file-upload L1, XXE L1) sit apart from the fast L3 cells, visualising that effort tracks tool

Limitations

• Operational exploit payloads, per-target exploit chains, and tooling source code remain withheld, limiting reproducibility and external validation of exploit strategies.
• Evaluation restricted to the 104-target XBOW suite; generalization to unseen, arbitrary, or evolving real-world web apps is unproven.
• The agent relies on the Gemini-family LLMs as black-box reasoners without reported fine-tuning, raising questions about adaptability to targets outside training distribution.
• No adversarial evaluation against adaptive defensive environments or hardened targets was reported; robustness to active countermeasures remains unclear.
• Self-evolution loop commits only non-regressive changes but cannot rewrite itself, limiting potential meta-self-refinement beyond administered fitness gates.
• While fabrication-proof verification mitigates false positives, it depends on the integrity of the build and flag-injection process, which if compromised could skew results.

Open questions / follow-ons

• How well does SE-AOS generalize to entirely novel vulnerability classes or web application architectures not represented in the XBOW-104 suite?
• Can the capability-evolution operator and discovery mechanisms be extended to incorporate or co-evolve with automated defensive countermeasures?
• How might interpretability and explainability of the agent's synthesized capabilities be improved to aid human security analysts?
• To what extent can SE-AOS be adapted or constrained for safer red teaming without risking dual-use concerns?

Why it matters for bot defense

For bot-defense and CAPTCHA engineers, Mako's work highlights the rapidly decreasing cost and time to exploit a wide range of web vulnerabilities autonomously once relevant capabilities are integrated and discoverable. This implies that defensive measures reliant on obscurity or complexity of chaining multi-step attacks may be insufficient against next-generation autonomous offensive agents. The law of autonomous exploitation articulated here suggests that evolving attacker toolkits capable of synthesizing new exploit capabilities at runtime will increasingly challenge static detection and rate-limiting approaches. Practitioners should consider that capability discoverability and agile capability maintenance may become the dominant resilience factor. Incorporating adaptive verification gates or challenge-response mechanisms that disrupt capability execution and synthesis could be a promising defense avenue. Moreover, the fabrication-proof evaluation techniques offer a rigorous methodology for measuring bot and attacker success beyond self-reported signals, potentially transferable to evaluating bot defeat or CAPTCHA solver resistance more reliably.

Cite

@article{arxiv2607_11288,
  title={ Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation },
  author={ Praneeth Narisetty and Shiva Nagendra Babu Kore },
  journal={arXiv preprint arXiv:2607.11288},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.11288}
}

Read the full paper

• arXiv:2607.11288 (abstract)
• arXiv:2607.11288 (PDF)