SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

Source: arXiv:2607.11185 · Published 2026-07-13 · By Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang et al.

TL;DR

This paper addresses key scalability bottlenecks in building computer use agents (CUAs) that automate complex digital workflows by interacting with GUIs through visual perception and reinforcement learning (RL). Specifically, the authors tackle two main challenges hindering online RL with verifiable rewards (RLVR) for CUAs: scarcity of verifiable, suitable GUI tasks to train on at scale, and inefficiencies in online RL training over multi-turn GUI interactions.

To overcome these, they propose SCALECUA, a unified framework combining a novel automatic verifiable task synthesis pipeline called VERIGEN, an adaptive online RL sampler named Frontier Sampling, and an efficient multi-turn training method (Visual Context Segmentation). VERIGEN generates 24K+ verifiable tasks by driving multiple large language model agents to iteratively create, check, and execute GUI tasks in live docker OS environments with a shared interaction interface. Frontier Sampling improves training efficiency by dynamically focusing on tasks that challenge the current model’s capabilities. Visual Context Segmentation accelerates training by bounding visual input context size in a sliding window, reducing training overhead. Together, these techniques yield substantial improvements in sample efficiency, training speed (2.83×), and task success rates.

Evaluated on two complex benchmarks, OSWorld and ScienceBoard, SCALECUA achieves new state-of-the-art results among open-source CUAs (68.7% and 54.0% success rates respectively), surpassing prior open-source models that are often larger in size. Ablation studies confirm the importance of each component, and qualitative analysis demonstrates the balancing of context window size to preserve task-relevant information without performance degradation. Overall, SCALECUA represents a significant advance in scaling reliable, performant online RL for computer use agents by solving the dual challenge of data generation and training efficiency.

Key findings

• VERIGEN generates over 24,000 verifiable GUI tasks, including nearly 3,000 high-quality RL-suitable tasks, an order of magnitude larger than prior desktop CUA datasets.
• Frontier Sampling increases sample efficiency by dynamically allocating rollouts to tasks near the model’s current success frontier, outperforming uniform sampling, DAPO, and fixed curriculum baselines (Fig 6(a)).
• Visual Context Segmentation provides a 2.83× end-to-end training speedup over traditional step-wise decomposition by bounding visual token accumulation and reducing training item inflation (Fig 6(b)).
• SCALECUA-Qwen3.5-9B achieves 68.7% success on OSWorld, improving substantially over ComputerRL-9B (48.0%), EvoCUA-32B (56.7%), and Kimi K2.5 (63.3%) baselines.
• On ScienceBoard, SCALECUA-Qwen3.5-9B reaches 54.0%, surpassing proprietary Claude Opus 4.6 (52.7%) and advancing state-of-the-art for scientific software workflows.
• Ablation shows removing VERIGEN drops OSWorld success from 68.7% to 43.9%, removing Frontier Sampling or Visual Context Segmentation reduces success to 63.7% and 62.2% respectively (Table 4).
• VERIGEN’s multi-agent feedback loop yields a 94.5% executable rate for generated task verifier functions, with the LLM judge agent removal causing largest drop (-32.2%).
• Visual Context Segmentation window size of K=5 balances context retention and training efficiency, yielding highest pass@1 success on OSWorld (58.9%) versus K=1 (56.4%) and K=15 (56.8%) (Fig 7).

Methodology — deep read

Threat Model and Assumptions: The work focuses on improving training scalability for CUAs that interact with desktop OS GUIs via visual observation and action sequences. The adversary is not explicitly modeled since the paper focuses on data generation and RL efficiency rather than security aspects. The main assumption is that the environment can be run inside isolated docker containers to enable controlled interaction and verifiable reward computation.

Data Provenance and Preprocessing: Data is generated by VERIGEN, an end-to-end pipeline that synthesizes verifiable GUI tasks. The pipeline starts from environment knowledge documents and base model rollout trajectories. It operates on live OS docker containers (e.g., for OSWorld and ScienceBoard) via a shared interaction probe. The initial task proposals are refined through a multi-agent feedback loop involving three LLM-based roles: proposer, judger, and checker. These roles iteratively generate tasks with executable verifier functions, perform static semantic checks, and conduct dynamic environment “dry runs” to validate task feasibility and verifier correctness. This process yields a large curated pool of verifiable tasks.

Architecture and Algorithms:

• VERIGEN’s multi-agent framework involves: -- Proposer agent generates candidate tasks and verifiers given environment documentation. -- Judger agent performs static semantic validation of instructions and verifier logic. -- Checker agent runs dynamic execution in the container to confirm correctness via actual state observations.
• Frontier Sampling maintains an exponential moving average (EMA) of task success rates to estimate model capability per task. Sampling weights are computed using a Gaussian centered on a target success rate (µ=0.5, σ=0.25), focusing training on tasks where the model is neither too good nor too poor. A small exploration fraction preserves uniform sampling.
• Visual Context Segmentation implements a sliding window over recent screenshots within rollout trajectories to bound visual token growth, reducing training overhead while preserving necessary cross-step information. When the window overflows, oldest screenshots are pruned and replaced by a summarized text context.

Training Regime:

• Models trained include GLM-4.6V-Flash, Qwen3-VL-8B-Thinking, and Qwen3.5-9B.
• Training runs on multi-node GPU clusters using vLLM for rollout execution and Megatron-LM for model optimization.
• The objective combines Guided Reward Policy Optimization (GRPO) with asymmetric clipping (DAPO-style) and KL penalties against a reference policy.
• Hyperparameters like batch size, rollout number, smoothing α for EMA used are described in appendix (not fully detailed here).

Evaluation Protocol:

• Benchmarks used are OSWorld (general desktop tasks) and ScienceBoard (scientific application workflows).
• Metrics are average success rates on held-out tasks.
• Multiple independent rollouts per task (4) are averaged and standard deviations reported.
• Ablations evaluate the contribution of each component (VERIGEN, Frontier Sampling, Visual Context Segmentation).
• Frontier Sampling is compared against uniform, DAPO, and curriculum learning baselines.
• Visual Context Segmentation window size is swept to identify optimal K.

Reproducibility:

• Code, models, and datasets are publicly released at https://github.com/THUDM/SCALE-CUA.
• Docker-based live environment setup with a shared probe interface enables reproducibility of task generation.
• Full training scripts and hyperparameters are provided in appendices.

Concrete Example Walkthrough: For a 47-step OSWorld task “install the Orchis GTK theme and switch to it for GNOME,” the Qwen3.5-9B agent reads desktop state, browses websites, downloads/extracts theme, and applies it via terminal commands, monitored by an executable judge yielding a score of 1.0 upon success (Fig 2). Visual Context Segmentation with sliding-window context helps the agent handle this long-horizon workflow efficiently.

Technical innovations

• VERIGEN: An iterative multi-agent LLM pipeline that generates large-scale, verifiable GUI tasks by interacting with live OS docker containers and validating deterministic executable rewards.
• Frontier Sampling: A Gaussian-kernel based adaptive sampling method that tracks per-task exponential moving average success rates to focus training rollouts on tasks at the agent's learning frontier, improving sample efficiency over uniform and curriculum methods.
• Visual Context Segmentation: A sliding-window trajectory segmentation technique that bounds visual token growth by pruning oldest screenshots and preserving textual continuity, achieving large training speedups without loss of multi-turn task reasoning.
• Shared Docker Interaction Probe: An architecture separating environment observation and action dispatch that enables 100+ parallel agent workers interacting concurrently with 100+ isolated OS docker environments, vastly scaling task synthesis throughput.

Datasets

• VERIGEN-generated GUI tasks — 24,000+ tasks (including nearly 3,000 high-quality RL tasks) — generated from live OSWorld and ScienceBoard docker environments

Baselines vs proposed

• ComputerRL-9B: OSWorld success = 48.0% vs SCALECUA-Qwen3.5-9B: 68.7%
• EvoCUA-32B: OSWorld success = 56.7% vs SCALECUA-Qwen3.5-9B: 68.7%
• Kimi K2.5: OSWorld success = 63.3% vs SCALECUA-Qwen3.5-9B: 68.7%
• Claude Sonnet 4.5 (proprietary): OSWorld success = 62.9% vs SCALECUA-Qwen3.5-9B: 68.7%
• Claude Opus 4.6 (proprietary): ScienceBoard success = 52.7% vs SCALECUA-Qwen3.5-9B: 54.0%
• Uniform Sampling: lower average task success than Frontier Sampling; Frontier maintains higher reward across training (Fig 6(a))
• Step-wise multi-turn training: training step time = 750s per batch vs Visual Context Segmentation (K=5): 265s (2.83× speedup) (Fig 6(b))

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.11185.

Fig 2: Trained SCALECUA agent solving a 47-step multi-app OSWorld task end-to-end.

Fig 1: Left: Comparison with open-source models on OSWorld. SCALECUA achieves 68.7%,

Fig 3 (page 2).

Fig 4 (page 2).

Fig 5 (page 2).

Fig 6 (page 2).

Limitations

• The generated verifiable tasks achieve 82.5% agreement with human expert labels, indicating some imperfection in reward correctness.
• Evaluation focuses mainly on training speed and accuracy gains; robustness to adversarial GUI states or environmental changes is not analyzed.
• The approach relies on heavy infrastructure (100+ docker environments, large LLM agents) that may be costly for smaller-scale deployments.
• No adversarial user or attacker threat model is considered, limiting direct security relevance beyond robustness improvements.
• Optimal hyperparameters such as visual context window size K may depend on specific task domains and models, requiring tuning.

Open questions / follow-ons

• Can VERIGEN be extended to synthesize verifiable tasks for more diverse or proprietary desktop applications without environment assumptions?
• How does SCALECUA perform under domain or distribution shifts, such as GUI changes or new software versions?
• Could the Frontier Sampling strategy be combined with more complex curriculum learning or multi-agent training for further gains?
• What strategies could improve the accuracy and interpretability of automatically generated executable reward functions?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, SCALECUA’s advances in scaling visual GUI agents using verifiable task synthesis and efficient online RL offer valuable insights into automating complex multi-step GUI workflows with strong verifiable feedback. Its method of generating large pools of verifiable tasks via interaction with live environments and multi-agent feedback agents represents a scalable approach for producing robust training data without human labeling. Frontier Sampling highlights the importance of adaptive task sampling focusing on the model’s current capability frontier rather than uniform or fixed curricula, which can inform training strategies for bot detection models that must generalize across a range of inputs.

Moreover, Visual Context Segmentation demonstrates a principled way to incorporate visual and textual context efficiently over long-horizon interactions. This balance between full context and computational scalability may similarly benefit RL or ML systems deployed for dynamic user-interaction and bot-detection challenges. While SCALECUA is targeted at general desktop automation agents, its technical innovations around verifiable task generation, adaptive curriculum, and efficient multi-turn training provide transferable lessons for building scalable reinforcement learning solutions in bot-defense and interactive CAPTCHA scenarios.

Cite

@article{arxiv2607_11185,
  title={ SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL },
  author={ Bowen Lv and Xiao Liu and Yanyu Ren and Hanyu Lai and Bohao Jing and Hanchen Zhang and Yanxiao Zhao and Shuntian Yao and Jie Tang and Yuxiao Dong },
  journal={arXiv preprint arXiv:2607.11185},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.11185}
}

Read the full paper

• arXiv:2607.11185 (abstract)
• arXiv:2607.11185 (PDF)