E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation

Source: arXiv:2606.27268 · Published 2026-06-25 · By Wen Ye, Peiyan Li, Tingyu Yuan, Yuan Xu, Xiangnan Wu, Chaoyang Zhao et al.

TL;DR

E-TTS addresses key challenges in test-time scaling (TTS) for embodied robotic manipulation tasks, where reasoning and action must be jointly optimized over long sequential horizons. Prior TTS works focus mostly on action-level scaling, ignoring the important intermediate reasoning processes that guide actions, and lack mechanisms to leverage history or feedback. This paper introduces E-TTS, a modular, plug-and-play framework that unifies reasoning and action scaling via history-aware iterative refinement. It jointly samples and scores ⟨reasoning, action⟩ pairs using vision-language verifiers that incorporate a history buffer containing past trajectories, enabling closed-loop feedback to guide iterative resampling, improving decision coherence and environmental adaptability. Unlike prior methods, E-TTS integrates dual verifiers (for reasoning and action), an adaptive selection policy, and feedback-driven iterative refinement, all configurable per task. Extensive experiments on 4 benchmarks, 6 environments, 3 robot embodiments, and 4 base vision-language-action (VLA) models demonstrate significant success rate gains up to 33.14% in simulation and 26.62% in real-world tasks without any retraining or additional data collection.

The results show E-TTS consistently improves diverse VLA methods including E-CoT, MolmoAct, π0.5, and Embodied-R1 by exploiting history and feedback to enable more robust, temporally coherent decision-making. Ablation studies confirm the joint reasoning-action scaling, history buffer, feedback-guided refinement, and joint scoring are all critical components. The framework balances computation and performance via an adaptive ε-greedy policy, allowing effective exploration-exploitation tradeoffs during inference. This work substantially advances TTS in embodied robotics by modeling both reasoning and action in a history-aware, closed-loop manner to improve sequential manipulation success.

Key findings

• E-TTS improves average task success rate from 6.67% to 39.81% on E-CoT in SimplerEnv WidowX (Table 1).
• E-TTS achieves up to 33.14% success rate improvement in simulation and 26.62% in real-world robot manipulations (Fig. 1).
• On LIBERO benchmark, MolmoAct + E-TTS outperforms baseline MolmoAct with success rates increasing from 85.57% to 90.78% averaged across Spatial, Object, Goal, and Long-Horizon tasks (Table 3b).
• Ablations show removing feedback, history buffer, or joint reasoning-action scaling drops success rate by 10–15% or more (Table 1).
• Adaptive ε-greedy joint sample selection balances exploration and exploitation and helps avoid local optima (Sec. 3.3).
• Action verifier trained on 90k paired demonstrations from SimplerEnv and LIBERO datasets, enabling robust action feasibility scoring (Sec. 3.3).
• Joint scoring that multiplies reasoning and action verifier scores outperforms sequential scoring (w/o joint scoring ablation, Table 1).
• Feedback-guided iterative refinement significantly improves sample quality by instructing the model how to fix reasoning/action errors (Sec. 3.4).

Threat model

Not an adversarial security paper; assumes a benign operating environment where improvements in inference-time scaling are targeted. The adversary model is absent or n/a. The work focuses on enhancing decision quality of embodied VLA agents by leveraging history and feedback within constrained computational resources.

Methodology — deep read

Threat Model and Assumptions: E-TTS assumes an embodied robotic manipulation setting where an adversary is not explicitly modeled; the method targets improving inference-time scaling without retraining and assumes access to a pretrained vision-language-action (VLA) policy producing intermediate reasoning and low-level actions. The adversary cannot tamper with the internal model or training data but can observe system behavior. The focus is on robust decision-making in sequential long-horizon embodied tasks.

Data: The action verifier is trained on a dataset of 90k paired demonstrations from SimplerEnv and LIBERO benchmarks, including successful and failed action rollouts, with quality labels derived via MSE comparison and a modified Bradley-Terry ranking objective. Evaluations are done on multiple benchmarks: SimplerEnv (WidowX and Google Robot), LIBERO, LIBERO-Plus, VLABench, and real-world robot data involving multiple tasks and embodiments, totaling tens of thousands of trajectories.

Architecture and Algorithms: E-TTS introduces a modular pipeline with four main components: 1) Reasoning-Action Joint Sampling, where at each timestep the base policy πθ samples M reasoning candidates, each with N corresponding action candidates, forming joint pairs ⟨reasoning c_t^i, action a_t^{i,j}⟩; 2) History-aware Verification module, employing two vision-language foundation models as verifiers: a Reasoning Verifier (Vc) based on Qwen2.5-VL-7B performs zero-shot scoring of reasoning candidates conditioned on the history buffer storing past K steps' joint pairs and observations to capture temporal dependencies; an Action Verifier (Va) based on LLaVA-7B trained on paired action data scores action feasibility and task relevance; 3) Adaptive Online Joint Selection uses a normalized product of reasoning and action scores for each pair and applies an ε-greedy exploration-exploitation strategy with threshold η to select the action to execute or trigger resampling; 4) Feedback-Guided Iterative Refinement generates structured textual feedback from the reasoning verifier by analyzing failed samples, appends it to the instruction prompt to guide the next sampling iteration.

Training Regime: The action verifier is trained on 90k paired demonstrations using a modified Bradley-Terry loss mimicking ranking preferences among action pairs with varying quality. Details on optimizer, epochs, or hyperparameters for verifier training are not elaborated. The reasoning verifier operates zero-shot without additional training.

Evaluation Protocol: Evaluations cover 4 benchmarks and 6 environments with varied tasks, including real-world robot platforms (WidowX, Google Robot, Franka). Metrics are primarily task success rates over multiple trials (e.g., 24 per task in SimplerEnv) across categories (Spatial, Object, Goal, Long-Horizon). Ablation studies disable key components (feedback, history buffer, scaling parts) to assess their contribution. Results include baseline comparisons to naive TTS and RoboMonkey. Efficiency is measured by average inference time per step. Statistical tests are not explicitly reported.

Reproducibility: Key datasets like BridgeData V2 (60k+ trajectories) and LIBERO are referenced; code and pretrained verifier models are indicated as released on the project webpage. Frozen weights for verifiers and base VLAs are used without retraining or additional expert data. The modular architecture allows independent enabling/disabling of components.

Concrete Example: At timestep t, given instruction I and current observation O_t, the base model πθ samples M=10 reasoning candidates c_t^i. For each c_t^i, N=20 action candidates a_t^{i,j} are generated. The history buffer H_t contains previous K=10 joint pairs and observations. Reasoning verifier Vc evaluates each reasoning candidate zero-shot, incorporating history and current observation, outputting confidence scores S_c^{i,j}. Action verifier Va scores feasibility of actions S_a^{i,j}. The joint score S_t^{i,j} = S_c^{i,j} × normalized S_a^{i,j}. Based on ε-greedy with ε=0.1 and threshold η=0.4, the system chooses the best scored pair or explores randomly. If no candidate exceeds η, feedback generation is triggered, producing a textual hint to refine sampling. This loop repeats until an adequate candidate is found or max batches reached, then the selected action executes, and history buffer updates.

Technical innovations

• Joint reasoning-action sampling and scoring that simultaneously optimizes intermediate reasoning outputs and final actions at test-time.
• History-aware verification leveraging a sliding window buffer of past reasoning-action pairs and observations to model temporal dependencies in sequential embodied tasks.
• Closed-loop feedback-guided iterative refinement: rejection triggers verifier-generated textual feedback appended to instructions, guiding resampling.
• Modular, plug-and-play E-TTS framework allowing independent enabling/disabling of reasoning scaling, action scaling, feedback, and history to adapt to different VLA models and tasks.

Datasets

• BridgeData V2 — 60,096 trajectories — SimplerEnv real-to-sim dataset from 24 environments
• LIBERO — over 500 task rollouts (50 per task) — simulated Franka Emika Panda robot
• LIBERO-Plus — 10,030 tasks across 7 perturbation dimensions — publicly released benchmark dataset
• 90k paired demonstrations of successful/failed actions — collected from SimplerEnv and LIBERO for action verifier training

Baselines vs proposed

• E-CoT baseline: Avg. success rate = 6.67% vs E-CoT + E-TTS: 39.81% (Table 1)
• E-CoT + RoboMonkey: 26.38% vs E-CoT + E-TTS: 39.81% (Table 1)
• MolmoAct baseline on LIBERO: Avg. success 85.57% vs MolmoAct + E-TTS: 90.78% (Table 3b)
• π0.5 baseline on VLABench Track 1: 0.39 success vs π0.5 + E-TTS: 0.42 (Table 4)
• Ablation no feedback drops E-CoT + E-TTS success from 39.81% to ~25.04% (Table 1)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.27268.

Fig 1: Overview. E-TTS is an embodied test-time scaling framework that integrates

Fig 2 (page 2).

Fig 3 (page 2).

Fig 4 (page 2).

Fig 5 (page 2).

Fig 6 (page 2).

Fig 7 (page 2).

Fig 8 (page 2).

Limitations

• Additional inference latency is introduced due to joint reasoning-action sampling and iterative refinement, though optimized with caching and prefix sharing.
• Action verifier training depends on existing datasets, which may limit generalization to radically different tasks or robot embodiments not covered.
• Reasoning verifier operates zero-shot; its performance depends on the quality and alignment of vision-language foundation models used.
• No explicit adversarial robustness or hostile environment testing is reported; effectiveness in adversarial or highly dynamic real-world scenarios remains unproven.
• Ablations show some hyperparameter sensitivity (e.g., ε, history length K, threshold η) requiring tuning for different environments.
• Closed-loop iterative refinement may have diminishing returns or instability if feedback misguides sampling in ambiguous cases.

Open questions / follow-ons

• How does E-TTS perform under out-of-distribution tasks or drastically novel environments where historical reasoning may be less relevant?
• Can the feedback-generation mechanism be extended to incorporate natural language interaction with human operators for interactive refinement?
• What are the limits on scalability of joint sampling (reasoning and action) as tasks and model sizes grow?
• Could adversarial perturbations designed to trick reasoning or action verifiers undermine E-TTS robustness?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, E-TTS exemplifies how integrating iterative, history-aware verification and feedback mechanisms can substantially enhance sequential decision-making models at test time without retraining. This approach suggests that scaling and refining intermediate reasoning steps jointly with final outputs, while exploiting temporal context, leads to better robustness and success rates in complex embodied tasks. Similar principles could inspire CAPTCHA systems that incorporate multi-round challenge adaptation, contextual history of user interaction, and dynamic feedback to improve bot detection reliability. Additionally, the modular plug-and-play design underscores the utility of systems that dynamically adjust their inference computation and verification based on confidence thresholds and feedback. However, practitioners should also note the tradeoffs in latency and the need for domain-specific verifier models trained on relevant failure data to enable effective closed-loop refinement.

Cite

@article{arxiv2606_27268,
  title={ E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation },
  author={ Wen Ye and Peiyan Li and Tingyu Yuan and Yuan Xu and Xiangnan Wu and Chaoyang Zhao and Jing Liu and Nianfeng Liu and Yan Huang and Liang Wang },
  journal={arXiv preprint arXiv:2606.27268},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.27268}
}

Read the full paper

• arXiv:2606.27268 (abstract)
• arXiv:2606.27268 (PDF)