AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation

Source: arXiv:2607.21588 · Published 2026-07-23 · By Mengfei Zhao, Dihong Huang, Yikai Tang, Peihao Li, Mingxuan Yan, Ruiqi Zhuang et al.

TL;DR

AXIS addresses the core bottleneck in scalable robot manipulation learning: the difficulty of collecting large-scale, diverse, and high-quality demonstration data. Existing robot data pipelines are limited by reliance on specialized hardware, centralized human operators, and fixed, static task suites that hinder growth and adaptation. AXIS introduces a novel, community-driven data engine that enables browser-based teleoperation for broad participation, automated task generation and validation, and unified post-processing including trajectory smoothing, success checking, and realistic physics and visual augmentation in simulation. This approach makes it possible to continuously grow a diverse manipulation dataset in a structured manner, organized into versioned snapshots for controlled benchmarking.

The resulting dataset snapshot contains 207 unique tabletop manipulation tasks with over 50,000 human teleoperated trajectories collected via MuJoCo-WASM in the browser, augmented into 60,000+ scene variants with randomized lighting, textures, and physics parameters. Policy training experiments demonstrate that continual pretraining on AXIS significantly boosts downstream performance on the LIBERO-Plus benchmark (+5.8% absolute success) and substantially outperforms a volume-matched RoboCasa365 baseline (+37.3% absolute success), indicating AXIS’s data diversity and community-driven scale provide unique benefits. Robustness gains are particularly strong under visual and geometric perturbations such as camera, sensor noise, layout, and robot pose shifts, confirming the value of AXIS’s scalable augmentation pipeline. AXIS thus offers a practical, extensible platform for continuously improving robotic manipulation capabilities through growing, diverse community data.

Key findings

• AXIS dataset version 100% contains 207 manipulation tasks and 50K+ human demonstration trajectories, expanded into 60K+ scene variants by augmentation.
• Trajectory refinement reduces mean acceleration by 63.9% and mean jerk by 80.8% compared to raw teleoperation data (Table 1), increasing data quality.
• Continual pretraining on AXIS-100% boosts overall LIBERO-Plus success from 83.9% to 88.8%, a 5.8% absolute improvement over π0.5 vanilla (Table 2).
• AXIS pretraining outperforms RoboCasa365-matched baseline by 31.3% absolute overall success, despite identical trajectory counts.
• Large scaling gains on AXIS subsets: success rates improve from 84.7% on AXIS-25%, 85.7% on AXIS-50% to 88.8% on AXIS-100%.
• Per-perturbation gains from AXIS pretraining include +16.6% on sensor noise, +15.6% on camera viewpoint shifts, +5.1% on robot pose, and +3.1% on layout perturbations.
• AXIS’s IsaacSim augmentation pipeline effectively transfers to robustness under visual and spatial distribution shifts in real evaluation (Fig. 2, Table 3).
• Web-based MuJoCo-WASM teleoperation enables broad community participation, collecting up to ~15,000 verified demonstrations daily during active windows.

Threat model

Adversaries are presumed to have no ability to subvert the teleoperation data collection interface or success checkers but may influence visual or physical simulation perturbations. The system relies on automated validation and filtering to remove failed or corrupted trajectories. It assumes threat vectors do not include direct manipulation of training procedures or policies.

Methodology — deep read

1. Threat Model & Assumptions: The system assumes adversaries who might manipulate visual or physical simulation conditions but cannot subvert the data collection interface or success checkers. Data is collected from diverse human operators via browser teleoperation, introducing natural variability and noise. Assumes robust trajectory validation and filtering mitigate poor-quality or failed demonstrations.

2. Data Collection: Collected via browser-based MuJoCo-WASM simulator enabling teleoperation of a Franka Research 3 robot with a parallel-jaw gripper for tabletop tasks. Tasks are specified textually and include pick-and-place, stacking, pouring, articulation, tool use across 207 task definitions. Over 50k trajectories have been collected from a broad community of 70,000+ contributors. Dataset snapshots (AXIS-25%, 50%, 100%) represent progressively larger subsets.

3. Data Processing: Raw trajectories undergo automated success validation using task-specific predicates rather than relying on frontend flags. Data filtering removes corrupted frames, static segments, inconsistent state transitions, and physically implausible motions. Trajectories are smoothed with Savitzky-Golay filters and resampled at 20 Hz for temporal consistency. After cleaning, trajectories are replayed in IsaacSim for physics and visual augmentations, including randomized object poses, scene layouts, lighting, textures, camera viewpoints, and dynamics parameters such as mass and friction.

4. Architecture/Algorithm: The VLA policy π0.5 is evaluated, a vision-language-action transformer pretrained on earlier robot datasets. The experimental pipeline retrains π0.5 starting from released checkpoints, then continually pretrains on AXIS subsets or RoboCasa365 matched data, followed by fine-tuning on the LIBERO dataset. Parameter settings and optimizer hyperparameters remain consistent across conditions.

5. Training Regime: Pretraining is conducted with matched learning rates, batch sizes, and gradient-step budgets to isolate data effects. Fine-tuning is performed on LIBERO trajectories with the π0.5 default configuration. Evaluations utilize fixed rollout budgets on held-out LIBERO-Plus tasks.

6. Evaluation Protocol: Metrics include task success rate averaged over rollout distributions for held-out tasks under environmental perturbations spanning camera viewpoint, lighting, sensor noise, background, object layout, language instruction variations, and robot pose. Baselines are π0.5 vanilla without continual pretraining and RoboCasa365 matched pretraining at equal data volumes. Scaling is studied by comparing AXIS-25%, 50%, and 100% subsets. Per-perturbation success rates and overall mean success provide comparative robustness evaluation.

7. Reproducibility: The paper notes code and dataset snapshots are released at https://axisaiorg.github.io/AXIS-V1/ enabling reproducibility. However, access to full community data may be constrained. Details on seeds or hardware specifics are limited. The unified trajectory format supports standardization and downstream replay.

Example End-to-End: A demonstration collected via web teleoperation (using cheap input devices) executes a language task specifying a pick-and-place operation. The data is validated offline through the task-specific success checker, then smoothed and augmented with randomized lighting and physics in simulation. This enriched data then feeds into continual pretraining of the π0.5 policy before being fine-tuned and evaluated on held-out LIBERO tasks under various visual and sensor noise perturbations, showing improved robustness.

Technical innovations

• A fully growable, versioned robot manipulation dataset pipeline integrating automated task generation, browser-based MuJoCo-WASM teleoperation collection, and standardized trajectory refinement.
• Unified post-processing pipeline combining automated success checking, quality filtering, trajectory smoothing, and IsaacSim-based multi-domain augmentation to enhance data quality and diversity.
• A scalable, community-driven demonstration collection platform that decouples teleoperation from expensive simulators and hardware.
• A systematic benchmark and evaluation protocol using fixed held-out tasks and perturbation axes enabling rigorous scaling studies of data volume and augmentation effects on policy robustness.

Datasets

• AXIS v1 — 50,000+ trajectories, 207 tasks, 60,000+ variants — collected via browser-based teleoperation, community-driven
• RoboCasa365 subset — matched size subset used as baseline — publicly available dataset for Franka manipulation

Baselines vs proposed

• π0.5 vanilla: overall success = 83.9% vs π0.5 + AXIS-100%: 88.8%
• π0.5 + RoboCasa365-matched: overall success = 57.5% vs π0.5 + AXIS-100%: 88.8%
• π0.5 + AXIS-25%: overall success = 84.7% vs π0.5 + AXIS-50%: 85.7% vs π0.5 + AXIS-100%: 88.8%
• Per-perturbation success on sensor noise: π0.5 vanilla 82.5% vs AXIS-100% 96.2% vs RoboCasa-matched 63.2%
• Per-perturbation success on camera viewpoint: π0.5 vanilla 72.5% vs AXIS-100% 83.8% vs RoboCasa-matched 35.2%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.21588.

Fig 1: AXIS: A growable community-driven data engine unifying task generation, web teleoper-

Fig 2 (page 1).

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 7 (page 1).

Fig 8 (page 1).

Limitations

• Current dataset and pipeline focus on simulation and simulated Franka Research 3 robot; real-world sim-to-real transfer not demonstrated.
• Community teleoperation introduces operator skill variance leading to noisy and inconsistent demonstrations despite smoothing and filtering.
• Evaluation restricted to tabletop manipulation tasks with limited task complexity and horizon; lacks long-horizon or multi-embodiment scenarios.
• Augmentation covers primarily visual, physical parameter, and layout perturbations; language and robot pose variation augmentations are not explicit.
• No adversarial robustness evaluation or detailed out-of-distribution generalization beyond predefined perturbations.
• Details on hardware training infrastructure, random seeds, and exact training schedules are limited, impacting reproducibility.

Open questions / follow-ons

• How effectively do AXIS-collected policies transfer from simulation to real robot hardware in the wild?
• Can the community-driven paradigm scale to long-horizon, multi-stage or multi-robot manipulation tasks beyond tabletop operations?
• What are the limits of data diversity and behavioral variation needed before gains saturate in robot manipulation policy performance?
• How might active failure-driven data collection integrated into AXIS enhance focused improvements in low-performing task areas?

Why it matters for bot defense

For bot-defense and CAPTCHA system engineers interested in robotic interaction or embodied AI verification, AXIS offers a blueprint for creating scalable, crowd-sourced task demonstration datasets leveraging accessible browser-based interfaces. The continuous growth and automated processing pipeline demonstrate how decentralized, quality-controlled community input can produce large, diverse datasets for training robust embodied policies. The approach of combining automated validation with augmentation to broaden robustness under distribution shift may inspire techniques in bot-detection benchmarks where environment or observation perturbations simulate adversarial conditions. Moreover, the AXIS methodology for versioned dataset snapshots and fixed evaluation protocols can inform long-term benchmarking of bot behaviors evolving with increasing dataset scale and diversity. While AXIS’s direct application is within robot manipulation, its scalable data engine architecture presents transferable principles for crowd-collected data validation, augmentation, and robustness measurement relevant to complex CAPTCHAs involving embodied or sensorimotor tasks.

Cite

@article{arxiv2607_21588,
  title={ AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation },
  author={ Mengfei Zhao and Dihong Huang and Yikai Tang and Peihao Li and Mingxuan Yan and Ruiqi Zhuang and Yanjia Huang and Jie Wang and Hai Zhai and Tony Zhou and Rui Zhang and Zhexi Luo and Yuchen Huang and Jianfei Yang and Jiachen Li },
  journal={arXiv preprint arXiv:2607.21588},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.21588}
}

Read the full paper

• arXiv:2607.21588 (abstract)
• arXiv:2607.21588 (PDF)