Paper deep dive
Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement
Mingtong Zhang, Dhruv Shah
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 99%
Last extracted: 6/21/2026, 2:20:05 AM
Summary
The paper introduces VERITAS, a generator-verifier framework designed for generalist robot policies to enable inference-time steering and autonomous self-improvement. The framework uses a pre-trained generalist policy as a stochastic generator to sample multiple action candidates and a gradient-free visual verifier (based on a Vision-Language Model) to score these candidates based on their alignment with a pre-computed visual trace. This process allows for immediate performance gains during deployment (inference-time steering) and creates a 'data flywheel' where successful, verified trajectories are used for offline policy fine-tuning, effectively reducing the need for human expert demonstrations and mitigating distribution shift.
Entities (7)
Relation Signals (6)
Generalist Robot Policy → actsas → Generator
confidence 100% · We use a pre-trained generalist robot policy as a 'generator'
VERITAS → comprises → Generalist Robot Policy
confidence 100% · We propose VERITAS, a generator-verifier framework... We use a pre-trained generalist robot policy as a 'generator'
VERITAS → comprises → Visual Verifier
confidence 100% · pair it with a gradient-free 'visual verifier' that evaluates actions at inference time.
Inference-time Steering → generatesdatafor → Offline Policy Improvement
confidence 100% · the verified rollouts provide effective supervision for offline policy improvement
Visual Verifier → isbasedon → Vision-Language Model
confidence 100% · pair it with a verifier based on a Vision-Language-Model (VLM)
Visual Verifier → enables → Inference-time Steering
confidence 90% · This framework enables inference-time steering that improves policy performance without additional training.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Robots deployed in the real world should learn from their experience and improve over time. This requires a mechanism of practicing and learning from feedback. In this paper, we propose VERITAS, a generator-verifier framework for generalist robot policies for inference-time policy steering and self-improvement. We use a pre-trained generalist robot policy as a ``generator'' and pair it with a gradient-free ``visual verifier'' that evaluates actions at inference time. This framework enables inference-time steering that improves policy performance without additional training. We demonstrate that inference-time verification consistently outperforms vanilla generalists without training on additional demonstration data. Additionally, we demonstrate that the verified rollouts provide effective supervision for offline policy improvement: policies fine-tuned on verified self-generated trajectories achieve consistent performance gains. Notably, we find that post-training with verified rollouts achieves comparable efficiency to expert demonstrations, while requiring no human interventions. Our results highlight inference-time verification as a practical and scalable mechanism for improving robotic policies during deployment.
Tags
Links
- Source: https://arxiv.org/abs/2606.18247v1
- Canonical: https://arxiv.org/abs/2606.18247v1
Trouble viewing inline? Open PDF directly →
Full Text
72,240 characters extracted from source content.
Expand or collapse full text
Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement Mingtong Zhang, Dhruv Shah Princeton University Robots deployed in the real world should learn from their experience and improve over time. This requires a mechanism of practicing and learning from feedback. In this paper, we propose VERITAS, a generator-verifier framework for generalist robot policies for inference-time policy steering and self-improvement. We use a pre-trained generalist robot policy as a “generator” and pair it with a gradient-free “visual verifier” that evaluates actions at inference time. This framework enables inference-time steering that improves policy performance without additional training. We demonstrate that inference-time verification consistently outperforms vanilla generalists without training on additional demonstration data. Additionally, we demonstrate that the verified rollouts provide effective supervision for offline policy improvement: policies fine-tuned on verified self-generated trajectories achieve consistent performance gains. Notably, we find that post-training with verified rollouts achieves comparable efficiency to expert demonstrations, while requiring no human interventions. Our results highlight inference- time verification as a practical and scalable mechanism for improving robotic policies during deployment. Website: veritas-improvement.github.io 1 Introduction Robotic foundation models trained on large-scale demonstration data [1,2,3,4,5,6,7] have shown advanced capabilities in various domains, such as manipulation and navigation [8,9,10,11,12]. These advancements are largely powered by large-scale robotic data collection from different sources. However, unlike texts or images, robotic data collection remains expensive, slow, and limited by the availability of human demonstrators. Thus, robot foundation models face a fundamental bottleneck: state-of-the-art models are primarily trained on data collected by human experts, which scales linearly in human expert time. This raises a key challenge: how can we improve robotic policies without scaling up the amount of human data? Our key insight is to enable robotic systems to explore the real world and learn from their experience, with minimal supervision and cost. A promising solution inspired by recent advances in large language modeling is to scale inference-time computation for verification [13,14,15]. Text-based reasoning models can generate multiple solution traces for a problem and select the best one using a “verifier” [16,17,18]. Analogous to this, perhaps a robot policy can generate and verify multiple action candidates at inference-time before committing to a physical action. Through selecting the best-of-푁samples using a verifier or value function, a pre-trained model might achieve performance improvements without any additional training. This insight is particularly valuable in robotic applications where data acquisition is significantly more expensive than computation. Rather than collecting additional data to improve coverage or recovery behaviors [19,20], we can instead use inference-time sampling to enable the policy to “try” multiple times and only execute the best generation. This allows us to trade compute for 1 arXiv:2606.18247v1 [cs.RO] 16 Jun 2026 Policy Improvement Visual VerifierBest-of-N Execution Raw Policy Improved Policy Verified Rollouts Inference-Time Action Selection Action Samples Robot Generalist Policy Policy as Generator ObservationInstruction Verified Rollouts Improve Online Performance Inference-Time Steering Policy Fine-tuning Action Chunks Figure 1|Overview of VERITAS. A pre-trained generalist policy acts as a stochastic generator, sampling multiple short-horizon action chunks at each decision step. A gradient-free visual verifier scores these candidates based on task alignment and physical plausibility, and the highest-scoring action is executed, yielding immediate performance gains at inference time. Successful verifier-guided rollouts are logged and reused for offline policy improvement, forming a data flywheel that distills verification-time reasoning into the policy and enables continual improvement with minimal human supervision. improved performance through inference-time steering. As the robot experiences successful trials, this autonomously generated data can be used to guide further improvements. In this work, we introduce Visual Verification for Inference-time Steering and Autonomous Policy Improvement, or VERITAS, a robotic self-improvement method built around a generator-verifier framework. We treat robot policies as a generator capable of sampling diverse candidate actions, and pair it with a verifier based on a Vision-Language-Model (VLM) which evaluates these candidates to guide exploration and task execution. This framework enables robots to autonomously explore the physical world with inference-time verification, steering existing generalist policies in an online manner out-of-the-box. Additionally, the verified self-generated rollouts serve as high-quality training data for offline policy improvement. Notably, this generator–verifier framework enables policy self- improvement without (i) training any additional embodied reasoning models, and (i) requiring expensive human annotation or data collection. Instead, it externalizes reasoning from the policy itself by introducing an explicit verification stage that evaluates candidate actions at inference time. By creating a flywheel where the robot learns from its own successful executions, we ensure that the post-training data is on-policy and physically feasible; we posit that this enables the policy to rapidly improve its behaviors and learn new behaviors, guided by the verifier. Our experiments demonstrate that our framework can improve policy performance while matching the efficiency of learning from human expert demonstrations. We evaluate the efficacy of VERITAS for inference-time steering and autonomous policy improvement extensively in simulation [21] and in the real world [22]. Across various base policies and manipulation tasks, we find that inference-time steering with VERITAS consistently improves performance over the base policy, without modifying policy parameters. Additionally, we show that verifier-generated rollouts form an effective supervision signal for policy improvement, achieving an average improvement of 10% over the base policy in simulation. We further find that performance gains from “self- improvement” (post-training on policy-generated data) can match the gains from post-training on human-expert data in the real world, suggesting that verifier-curated data is comparable to human data for autonomous policy improvement. 2 2 Related Works Prior work has extensively studied data collection through human-in-the-loop learning [23,24,25] and shared autonomy [26,27,28], where robots improve by incorporating expert interventions, corrections, or demonstrations during deployment. Classic approaches such as DAgger rely on expert relabeling to correct policy mistakes on-policy [19,29,30], while shared autonomy frameworks blend human control with autonomous execution to improve task success and collect corrective data. Recent systems refine this paradigm by structuring how corrective data is collected in long-horizon tasks; for example, Hu et al. [20] explicitly decompose robot behavior into nominal execution and recovery phases, triggering human intervention when the policy enters failure states to improve downstream learning [3]. Instead of using human intervention [19,31] as the source of corrective signal, we leverage execution-time verification to autonomously evaluate candidate actions and select successful behaviors. The verifier replaces the role of the human supervisor by providing a task-aligned preference signal at inference time, allowing the robot to improve performance online and collect success-conditioned trajectories without shared control. Inspired by advances in language modeling, where scaling test-time compute via verification and search often outperforms scaling model parameters [13,16,18], the robotics community has begun exploring mechanisms for embodied control. Recent work integrates explicit reasoning into robot policies by training VLAs with supervision over intermediate visual and semantic representations, enabling multi-step, visually grounded decision making [32, 33, 34]. While these works demonstrate impressive embodied reasoning capabilities, these methods rely on expensive annotations during policy pre-training and are not suitable for inference-time steering or improvement of existing policies. Another promising set of directions is sampling and steering online samples, such as Kwok et al. [35] demonstrating that action error scales with the number of actions sampled at inference time using a learned verifier, Kwok et al. [36] suggesting scaling verification leads to more effective alignment for policies, Wu et al. [37] introducing a runtime steering method that verifies actions against a reasoning VLA’s self-generated textual plan, and Nakamoto et al. [38], which trains a language-conditioned value function using offline RL and uses it for steering the generalist policy. [39,40] requires world models’ prediction to select reasonable actions for policy steering. While these methods demonstrate the power of test-time sampling, they typically treat inference-time search as a “momentary boost”, relying on expensive verification, that does not necessarily improve the policy in long-term deployment. Our work takes a novel perspective by treating online verification as a data engine without training task-specific verifiers, and allowing us to distill expensive inference-time compute into the base policy to enhance its capabilities via autonomous verification and post-training. Our experiments further show that our visual verification framework outperforms various other test-time sampling approaches for robot policies. Long-term deployment of generalist robots requires policies to continually learn and adapt from their experience. However, standard imitation learning suffers from covariate shift, where policy performance degrades as the robot drifts from the expert’s state distribution [29]. Traditional solutions [19,29] mitigate this by querying human experts for labels on robot-visited states. However, these human-in-the-loop paradigms are difficult to scale. Our approach replaces the human labeler with an automated verifier, ensuring the training data is strictly on-policy and physically feasible for the robot’s embodiment. This effectively mitigates the distribution shift problem by aligning the training distribution with the support of the robot’s exploration prior, unlocking scalable self-improvement without human intervention. 3 3 Preliminaries We formulate the problem of language-guided robotic manipulation as a Partially Observable Markov Decision Process (POMDP) [41] defined by the tuple (S,O,A,T,R, 훾). Problem Formulation. At each sampling timestep푡, the robot receives an observation표 푡 ∈ Oand a natural language instruction푙 ∈ L. Our objective is to train a policy휋 휃 that maximizes the expected success rate over a distribution of tasks. Unlike standard imitation learning, which assumes a fixed static dataset, our objective is to enable the policy to self-improve its performance post-deployment by collecting and learning from its own experience. Generalist Robot Policies. In this work, we assume access to a pre-trained “generalist policy”휋 휃 , that can be conditioned on the current observation and a language instruction, and predicts actions 푎 푡 to control a robot. We assume that휋 휃 can successfully solve a large number of tasks in the target environment, and are agnostic to the specific architecture or data sources the policy was pre-trained on. Following recent advancements in high-frequency control, we employ action chunking [42,43]: the policy predicts a sequence of actions of horizon 퐻: 풂 푡:푡+퐻 (푎 푡 , 푎 푡+1 , . . . , 푎 푡+퐻−1 ) ∈ A 퐻 .(1) Generative Policies as Priors. Our method relies on the policy’s ability to model complex, multi-modal action distributions (e.g., grasping an object from different angles, solving a multi-step task in different orders, etc.). We focus on generative policies trained with flow-matching or diffusion objectives [5,43]. Unlike deterministic regression policies that output a single mean action, a generative policy learns the conditional probability distribution푝(풂 푡:푡+퐻 | 표 푡 , 푙). At inference time, action chunks are generated via a stochastic sampling process. In this work, we leverage this stochasticity not just for robustness, but as a mechanism for exploring diverse, functional affordances [44] of the scene. 4 A Generator-Verifier Framework for Robot Policies We present a generator–verifier inference framework that steers a generalist policy with execution-time selection. The core design principle is to utilize (i) generation by a generalist policy휋 휃 as a stochastic generator, and (i) evaluation by a plug-in verifier푉that is policy-agnostic and gradient-free. At every decision step, the robot samples multiple short-horizon action chunks, scores them with푉, and executes the best candidate. This yields immediate online performance gains without updating 휃, while producing success-conditioned trajectories for offline policy improvement. The complete procedure is detailed in Algorithm 1. 4.1 The Policy as a Stochastic Generator Standard policy deployment typically utilizes deterministic or greedy decoding to minimize variance. We instead view the pre-trained policy 휋 휃 as a stochastic generator. We posit that a generalist policy trained on diverse data maintains a broad “prior" over valid affordances, and by sampling from this distribution, we can encourage and recover high-quality behaviors that might have lower initial probability but higher task alignment. Formally, at discrete sampling timestep푡, given observation표 푡 and language instruction푙, the policy generates a set of 푁 candidate action chunks (Algorithm 1, Line 1): 풂 (푖) 푡:푡+퐻 ∼ 휋 휃 (· | 표 푡 , 푙), 푖∈ 1, . . . , 푁(2) 4 Initial Conditions “Put the carrot on the plate” Robot Generalist Policy Policy as Proposer <Source: Carrot>:100, 200, <Target: Plate>: 200, 300 Object Initial Position Visual VerificationBest-of-N Execution Proposed Execution Trace “uv”: [100, 200], “min_hold”: 200, “skippable”: True <Objects>: Carrot, Plate Action Chunk as Reasoning Tr a c e Figure 2|The overview of VERITAS for inference-time steering. At inference time, a two-call VLM scheme is used to construct a visual verifier. First, given the initial observation and task instruction, the VLM identifies the target object to focus on, a detection model is then used to localize the object in image space. Second, conditioned on the object location, instruction, and initial observation, the VLM proposes a sequence of pixel-space waypoints that define a full visual trace for the task. During execution, the policy samples multiple action candidates, which are scored based on how well their resulting motion follows the proposed visual trace, and the best candidate is selected for execution. Successful verifier-guided rollouts are logged and used as training data for offline policy fine-tuning, resulting in an improved policy with higher task performance. where each candidate풂 (푖) 푡:푡+퐻 = (푎 푡 , 푎 푡+1 , . . . , 푎 푡+퐻−1 )represents a sequence of퐻actions. Utilizing action chunks is critical; it amortizes the cost of verification over the horizon퐻and provides the verifier with temporal context to evaluate the consequence of a behavior, rather than a micro instantaneous movement. 4.2 The Verifier Interface To select the optimal candidate from the generator, we introduce a verifier푉. We define the verifier as a plug-and-play, gradient-free function that maps the observation, candidate action chunk, and task instruction to a scalar score: 푉(표 푡 ,풂 푡:푡+퐻 , 푙) ∈ ℝ.(3) Conceptually,푉estimates the utility or alignment of executing풂 푡:푡+퐻 from state표 푡 under instruction 푙. Importantly, steering with푉operates solely at inference time and does not update휋 휃 . As a result, the verifier functions as a plug-and-play module that can be instantiated with various mechanisms (e.g., VLMs, geometric constraints, or learned value models) without retraining the base policy. 4.3 Inference-Time Steering We combine the generator and verifier via Best-of-푁selection. This mechanism trades inference-time computation for performance, allowing the robot to “think" (sample and verify) before acting. As outlined in Algorithm 1, the control loop proceeds as follows: 1. Sample: The policy generates 푁 diverse action chunks풂 (푖) 푡:푡+퐻 푁 푖=1 . 5 2. Verify: The verifier scores each candidate: 푣 푖 = 푉(표 푡 ,풂 (푖) 푡:푡+퐻 , 푙). 3. Select: The highest-scoring candidate index is selected: 푖 ★ = arg max 푖∈1,...,푁 푣 푖 ,풂 ★ 푡:푡+퐻 =풂 (푖 ★ ) 푡:푡+퐻 (4) 4. Execute: The robot executes the chunk풂 ★ 푡:푡+퐻 in the environment. This steering process serves as a filter, prioritizing physically valid and task-aligned actions. Importantly, because 휋 휃 is fixed, these performance gains come purely from inference-time computation. 4.4 Autonomous Self-Improvement While inference-time steering improves performance, it incurs a computational cost at every decision step. To amortize this cost and permanently improve the policy, we utilize the verified trajectories as a source of autonomous supervision with no extract human expert efforts. This process creates a data flywheel. During policy deployment, we log the successful verified trajectories generated via inference-time steering into a datasetD auto . D auto = Ø 푘 (표 푡 ,풂 ★ 푡:푡+퐻 , 푙) 푡∈I success (5) We then fine-tune the original policy휋 휃 onD auto using standard supervised behavior cloning to obtain updated parameters 휃 ′ : 휃 ′ ← arg min 휃 피 D auto [− log휋 휃 (풂 ★ 푡:푡+퐻 | 표 푡 , 푙)](6) Mitigating Distribution Shift. A fundamental challenge in imitation learning is the distribution shift between the expert and the learner. Standard human-in-the-loop approaches require expensive expert relabeling. Our framework automates this: because the data inD auto was generated by the robot’s own policy (the generator), it is inherently on-policy and kinematically feasible. The verifier acts as the "expert" filter, ensuring only high-quality data is retained. This closes the loop: the reasoning performed by the verifier is distilled into the policy, reducing the need for large푁 samples in future deployments. Importantly, as the verified data is generated by policy itself, the action distribution is close to the policy’s action prior, which mitigates the shift and remains efficient for policy improvement. 5 Instantiating VERITAS for Robotic Manipulation We now describe a concrete instantiation of the proposed generator-verifier framework for tabletop robotic manipulation. In this work, as an example, we conduct our analysis using the휋 0 architec- ture [5], an open-source Vision-Language-Action (VLA) model trained on thousands of hours of diverse robot trajectories.휋 0 utilizes a mixture-of-transformers backbone with a flow-matching action head. Crucially, this flow-matching formulation allows us to effectively sample complex, multi-modal action distributions, providing the necessary diversity to explore valid solutions at test time. We pair this generator with our visual verifier designed to ground high-level semantic instructions into precise geometric constraints. 6 Algorithm 1 VERITAS Online Steering and Autonomous Policy Improvement Input :Policy 휋 휃 , verifier 푉, instruction 푙, horizon 퐻, samples 푁 Output:Executed trajectory, datasetD auto , updated policy 휋 휃 ′ // Phase 1: Inference-time Steering InitializeD auto ←∅ and observe initial state 표 0 for 푡= 0, 퐻, 2퐻, . . . until termination do // 1. Sample diverse candidates (Eq 2) Sample풂 (푖) 푡:푡+퐻 푁 푖=1 where풂 (푖) 푡:푡+퐻 ∼ 휋 휃 (· | 표 푡 , 푙) // 2. Verify candidates Score each candidate 푣 푖 ← 푉(표 푡 ,풂 (푖) 푡:푡+퐻 , 푙) // 3. Select Best-of-N (Eq. 4) Select 푖 ★ ← arg max 푖 푣 푖 and set풂 ★ 푡:푡+퐻 ←풂 (푖 ★ ) 푡:푡+퐻 // 4. Execute and Log (Eq. 5) Execute풂 ★ 푡:푡+퐻 and observe 표 푡+퐻 Log (표 푡 ,풂 ★ 푡:푡+퐻 , 푙) intoD auto end // Phase 2: Offline Self-Improvement if Offline update is enabled then Initialize 휃 ′ ← 휃 for 푒= 1 to 퐸 do Sample batchB ∼D auto Update 휃 ′ via Behavior Cloning (Eq. 6): 휃 ′ ← 휃 ′ − 휂∇ 휃 ′ ∑︁ (표,풂,푙)∈B L BC ( 휋 휃 ′ (· | 표, 푙),풂 ) end return 휋 휃 ′ end 5.1 Visual Verifier Design To ground high-level semantic instructions into precise geometric constraints without requiring expensive 3D annotations, we design a verifier based on absolute pixel-space trajectories proposed by a frontier vision-language model (VLM) [45]. This design leverages the strong spatial and embodied reasoning capabilities of modern VLMs to create a “visual guardrail” for the policy. The verification process consists of two stages: 1. Trace Generation: At the onset of a task, the VLM is provided with the initial observation표 0 and the language instruction푙. It is prompted to propose a visual reasoning trace: a sequence of pixel- space waypoints푤 푘 overlaid on the image that defines the ideal end-effector trajectory for task completion. This trace remains static throughout execution to ensure temporal consistency. 2.Geometric Scoring: At inference time, for each candidate action chunk풂 (푖) 푡:푡+퐻 sampled from휋 휃 , we project the resulting end-effector/gripper positions into the image plane. The verifier score 푣 푖 is computed as the negative Euclidean distance between the candidate’s projected trajectory and the nearest segment of the VLM-generated trace. Crucially, this design allows for efficient verification without requiring online tracking or expensive VLM querying at every verification step. Once the visual trace is generated, verification reduces to a fast geometric consistency check, enabling the evaluation of large푁samples with minimal latency 7 overhead. We also have other types of verifier design and ablate them in the experiments. For more details on our verifier and alternate designs, please see Appendix A.1. 5.2 Steering at Inference-Time We implement a simple inference-time steering mechanism using the Best-of-푁strategy described in Section 4.3. Implementation Details. As an example, we deploy the휋 0 -DROID policy on a workstation with one NVIDIA GeForce RTX 5090 GPU. At each control frequency step (e.g., 15 Hz), we define an action chunk horizon of퐻steps. The generator samples푁candidates in parallel. The visual verifier evaluates these 푁 chunks against the pre-computed visual trace. Latency and Throughput. The VLM verifier generates an execution trace only once at the start of episode, and hence, is not run in-the-loop with the policy. The geometric verification incurs negligible overhead (<1 ms) and runs online. The cost of policy inference scales linearly with the number of sampled actions푁, and can be batched during inference. For our experiments, we use푁=5 samples and maintain a control frequency of 15 Hz. This trade-off allows us to filter out physically invalid or erratic behaviors that the base policy might occasionally produce. 5.3 Policy Improvement To close the loop, we utilize the successful verified trajectories curated by our steerable system for offline policy fine-tuning. Data Collection. We deploy the steered policy in the real world to solve a diverse set of manipulation tasks (Section 6). We log all executed action chunks풂 ★ 푡:푡+퐻 that lead to successful task completion after verification, filtering out failures. This results in a high-quality datasetD auto . Training Setup. We fine-tune the pre-trained휋 0 -DROID checkpoint onD auto using the behavior cloning objective (Eq. 6). We use a learning rate of 5×10 −5 and train for 20,000 steps. As shown in our experiments, even small amounts of this verifier-curated data (e.g., 20 trajectories) are sufficient to significantly shift the policy distribution towards the robust behaviors discovered during steering, effectively distilling the VLM’s reasoning into the policy’s weights. 6 Experiments In our experiments, we aim to assess the efficacy of the generator-verifier paradigm as a data flywheel for policy improvement. We study the following research questions: 1. Inference-time Steering: Does the Generator–Verifier architecture improve success rates at test time compared to standard policies? 2. Policy Improvement: Does training on self-generated, verified data improve the base policy? 3. Data Efficiency: How does our autonomous data compare to human-expert data? We ablate the design of verifier in simulation experiments and find the paradigm can improve the policy performance at inference time, agnostic to the design as well. Based on these results, we select the most effective verifier configuration from simulation for deployment in real-world experiments. 8 6.1 Experimental Setup Tasks We evaluate our method on a suite of simulated and real-world robotic manipulation tasks specified by natural language instructions. In simulation, we focus on 4 manipulation tasks from [46], following the evaluation protocol of Li et al. [21]. These tasks provide a controlled benchmark for assessing policy performance and generalization under standardized conditions. We conduct all real-world experiments on the DROID platform [22]. We design a set of challenging manipulation tasks that stress visual grounding, spatial reasoning, manipulation that requires high-precision, etc. Specifically, we select 2 manipulation tasks in the real world for each policy. Policies We evaluate our framework using different pretrained robot policies. In simulation, we employ an open-sourced BridgeData policy [47], denoted as휋 0 -Bridge, trained on BridgeData [46] following the architecture of휋 0 [5]. In real-world experiments, we use various generalist robot policies. Specifically,휋 0 -DROID [5],휋 0.5 -DROID [48] trained on large-scale real-world demonstration data and fine-tuned on [22]. Unless otherwise specified, all policies are treated as fixed base policies for inference-time steering and are used as action proposers within the generator–verifier framework. Baselines To show the efficiency of our verifier against several baselines for inference time steering and ablate different verifier designs in the simulation experiments. We compare the VERITAS verifier to several key baselines. • V-GPS [38]: A language-conditioned value function learned via offline RL for policy steering. • Heuristic: A verifier that defines action primitives and use stage transitions for verification. •VLM+Constraints: A verifier that scores action samples by verifying if they follow VLM- proposed reference-based waypoints that resolve to pixel coordinates via tracked objects. We also compare to RoboMonkey [35], which uses a learned single-step verifier by fitting a Gaussian on the action distribution. While effective for local corrections, we find that the RoboMonkey verifier’s optimization of local single-step action regions limits its application to completely new tasks that require global reasoning. For more details on this baseline, please see Appendix B. In real-world experiments, we emphasize the comparison between our SOTA verifier design reflected by the simulation results, against baselines: •PIVOT [49]: An approach that uses proposed action primitives and Iterative Visual Optimization for robotic tasks. This baseline combines a naïve action-primitive proposer with an optimization- based action selector, highlighting the importance of a policy proposer. • V-GPS-DROID: We train a V-GPS model on the DROID [22] dataset and deploy it on the same robot hardware setup. This experiment evaluates the efficiency of our combined policy proposer and visual verifier against a learned value function. 6.2 Inference-Time Steering. In simulation, we evaluate the performance of different verifiers on top of the base policy휋 0 -Bridge and take the base policy as one of the baselines in the simulation. All evaluations are averaged over 10 trials on top of task variations such as robot or object locations, etc, following [21]. In the real world, we deploy the best-performing verifier in the simulation and evaluate it on 2 different policies using the DROID platform, conducting 50 rollouts for each task across 2 tasks per policy. 9 Figure 3|Inference-time verification improves task performance. Trajectory rollouts illustrating VERI- TAS steering. The first 2 rows show verification of the휋 0 -Bridge policy on a simulated WidowX and the last 2 rows show verification of the휋 0 -DROID policy on a real FR3-DROID robot. At each decision step, the generator generates푁candidate action chunks풂 (푖) 푡:푡+퐻 푁 푖=1 ; the verifier scores each candidate푉(표 푡 ,풂 (푖) 푡:푡+퐻 , 푙)and selects the best action풂 ★ for execution (highlighted). Verifier-guided rollouts achieve successful task completion. As shown in Figure 4 across all 4 simulation evaluated tasks, verifier-guided action selection con- sistently improves success rates relative to the base policy. While the magnitude of improvement varies with task difficulty and verifier design, performance gains are observed even when using simple heuristic verifiers, indicating that the benefits of the generator–verifier paradigm are not dependent on a specific verifier architecture. On average, all of our verifier designs outperform V-GPS, suggesting that explicit verification of action consequences provides a more reliable signal for policy steering than the learned value estimates. We observe similar trends in real-world experiments. As shown in Figure 6, inference-time steering significantly improves task success rates across different evaluated real-world tasks and policies. Across simulation and real-world settings, covering 3 different policies and a total of 1160 evaluation episodes, verifier-guided execution improves policy success rates by an average of 12.6% in simulation and 35% in real-world deployment, without any policy fine-tuning. These gains highlight the effectiveness of inference-time computation alone in improving generalist policy performance. Note that PIVOT’s naïve action primitives can not achieve successful outcomes, whereas a trained policy with a strong action prior performs effectively, highlighting the importance of good action priors for improving performance. Importantly, the proposed framework is policy-agnostic and verifier-agnostic. We observe consistent improvements across different base policies and across a diverse set of verifier implementations in simulated and real-world environments. This robustness demonstrates that the generator–verifier framework provides a general and reliable mechanism for inference-time policy steering, applicable across tasks, embodiments, and verification strategies. 10 Carrot on PlateSpoon on TowelEggplant in BasketStack BlocksAverage 0% 20% 40% 60% 80% 100% π₀-BridgeV-GPSVLM-ConstraintsHeuristicVERITAS Success Rate Figure 4|Inference-time steering in SIMPLER. Simu- lation success rates comparing standard policy execution and VERITAS execution across 4 tasks and different veri- fier architectures. VERITAS verifier consistently improves performance over different methods across all tasks. The largest gains are observed in the most challenging task (Stack Blocks). Error bars show 95% finite-sample CI [50]. Carrot on PlateEggplant in BasketSpoon on TowelStack BlocksAverage 0% 20% 40% 60% 80% 100% Raw PolicyRaw Policy + Inference-Time SteeringRaw Policy + Offline Improvement Success Rate Figure 5|Simulation success rates of the raw pol- icy, the inference-time steering and the same pol- icy fine-tuned on verifier curated autonomous data. While verification improves performance at inference time through action steering, fine-tuning on the col- lected verifier curated rollouts successfully distills these gains back into the policy weights. Error bars show 95% finite-sample CI [50]. 6.3 Offline Policy Improvement We next study whether verified autonomous executions can be used to improve the policy offline. In this setting, we deploy the policy using the generator–verifier framework in the simulation and real world to collect a set of verified successful trajectories for offline policy fine-tuning. Specifically, in the simulation experiments, we select the verifier that boosts the policy steering performance the most and collect the successful verified rollouts, consisting of 656 demonstrations for the base policy fine-tuning. As shown in Figure 5, fine-tuning on this autonomously collected data improves average policy performance and consistently outperforms the base policy across all 4 tasks. Notably, inference-time steering provides larger performance gains on tasks that are difficult for the base policy, such as Stack Blocks, than on simpler tasks. One possible explanation is that the policy produces more diverse action samples when it is uncertain or underperforms, creating greater opportunity for the verifier to select better candidates. In contrast, for tasks on which the policy already performs well, the sampled actions are less diverse, leaving less room for improvement. These results point to a fundamental trade-off: the effectiveness of inference-time steering depends on the diversity of the proposed actions, and therefore tends to be greatest on more challenging tasks. In the simulation experiments, VERITAS fine-tuning improves average success rate by 9.7% over the base휋 0 -Bridge policy across 4 simulated manipulation tasks in SimplerEnv for 960 total episodes. Success rate gains are consistent across all tasks, with the largest improvement on Stack Blocks, from 31.3% to 59.2%, by 27.9% on the hardest task. In the real world experiments shown in Figure 7, verified autonomous data consistently supports effective policy improvement across 4 challenging real- world tasks for 2 policies, with performance improving reliably as more data is collected, demonstrating that autonomous rollouts alone are sufficient for successful fine-tuning. These results demonstrate that verified autonomous executions can serve as a reliable and effective source of supervision for offline policy improvement, without requiring additional human demonstrations. By leveraging the generator–verifier framework to curate high-quality rollouts, VERITAS consistently improves policy 11 0%0% 34% 42% 44% 48% 58%58% Put the carrot on the platePut the tape into the wooden box 0% 20% 40% 60% 80% 100% 0%0% 36% 34% 46% 48% 56% 60% Pick up the mouse and place it into the green bowlInsert the marker into the mug PIVOTRaw PolicyRaw Policy + V-GPSRaw Policy + VERITAS Success Rate π₀-DROIDπ₀.₅-DROID Figure 6|Inference-time steering in the real world. We evaluate inference-time steering across two real-world policies and two tasks per policy, with 50 trials conducted for each task. VERITAS consistently outperforms the V-GPS baseline across all tasks, demonstrating robust improvements in the real world. performance. 6.4 Autonomous v/s Human-Collected Data Finally, we compare the data efficiency of verifier-curated autonomous rollouts to human expert demonstrations as a supervision signal for offline policy improvement. While human demonstrations remain a strong baseline, they are expensive to collect and difficult to scale. An important question is whether autonomously collected, verifier-selected data can provide comparable learning signal. 020406080100 0% 20% 40% 60% 80% 100% 020406080100 0% 20% 40% 60% 80% 100% 020406080100 0% 20% 40% 60% 80% 100% 020406080100 0% 20% 40% 60% 80% 100% Number of DemosNumber of DemosNumber of DemosNumber of Demos Success Rate Carrot on plateTape into boxPick up mouseInsert marker π₀-DROIDπ₀.₅-DROID ● Human Expert ★ Verified Auto ● Human Expert ★ Verified Auto ● Human Expert ★ Verified Auto ● Human Expert ★ Verified Auto Figure 7|Comparing data efficiency of autonomous verification v/s human expert. We fine-tune the base policy on datasets of 20, 30, 50, and 100 trajectories collected via either human expert teleoperation or verifier-curated autonomous rollouts, and evaluate each resulting policy over 20 real-world trials. Across all four tasks, verifier-curated data achieves comparable or superior data efficiency to human demonstrations. In multiple cases, autonomous data matches or outperforms human data at the same budget (e.g., 20–50 demos), and remains competitive even at larger scales. For example, on Pick up mouse, 50 autonomous demos outperform human demonstrations (0.65 vs. 0.60), while for Carrot on plate, performance is identical. These results indicate that verifier-curated autonomous collection can match the data efficiency of expert demonstrations, enabling scalable policy improvement without supervision. In our real-world experiments, we fine-tune policies using datasets of 20, 30, 50, and 100 demon- strations collected from either verifier-curated autonomous rollouts or human expert teleoperation. We then evaluate each fine-tuned policy over 20 real-world trials across multiple manipulation tasks, 12 Carrot on plateTape into boxPick up mouseInsert marker 0% 20% 40% 60% 80% 100% 4-step8-step10-step Task Success Rate Figure 8|We find that VERITAS performance does not significantly degrade with open-loop action execution of up to 10 steps. 1248163264128 0 20 40 60 80 100 Stack BlocksCarrot on PlateEggplant in BasketSpoon on TowelAverage Number of Samples (k) Success Rate (%) Figure 9|We find that inference-time verification perfor- mance saturates beyond푁=8 samples, while the cost of sampling increases linearly with 푁. comparing policy performance as a function of the quantity and source of fine-tuning data. This controlled setup allows us to directly compare how policy performance scales with the amount and source of fine-tuning data. The experimental results are shown in Figure 7. Across all 4 tasks, with 2 tasks evaluated for each policy, verifier curated data demonstrates comparable data efficiency to human expert demonstrations. In several cases, autonomous data achieves stronger performance at moderate data budgets. For example, on Carrot on plate, fine-tuning with 50 autonomous demonstrations outperforms fine- tuning with the same number of human demonstrations (0.70 vs. 0.65). Similarly, on Pick up mouse, autonomous data consistently achieves higher success rates in the 20 to 50 demonstration regime. At larger data budgets, verifier curated data remains competitive with expert supervision across all tasks. On Carrot on plate, the two data sources converge to the same success rate at 100 demonstra- tions, while on Insert marker, the performance gap remains within 5 percentage points across all data budgets. Overall, these results show that verifier curated autonomous rollouts provide an effective and scalable alternative to costly human demonstrations for policy improvement. Overall, these results show that verifier-curated autonomous rollouts can match the data efficiency of costly human teleoperation. By converting deployment-time execution into effective training data, our approach enables scalable policy improvement without requiring continuous expert supervision. These findings suggest that execution-time verification not only improves performance online, but also enables a practical pathway for continual policy improvement by converting deployment experience into effective training data. More importantly, because the robot itself generates the data, the autonomously collected dataset is inherently on-policy and kinematically feasible. The verifier serves as an expert filter, retaining high-quality executions while discarding failures and suboptimal behaviors. This process effectively closes the learning loop: the reasoning and corrective signals provided by the verifier are distilled back into the policy during offline fine-tuning, progressively improving the generator and reducing the need to sample a large number of candidates in future deployments. 13 6.5 Implementation Hyperparameters We perform a sweep over the action execution horizon, which determines the length of the visual action trace associated with each action chunk (Figure 8). For휋 0 -DROID, we find that the open- loop execution horizon does not significantly degrade verification performance in our real-world experiments. Based on this observation, we use the default horizon of 8 steps for all real-world evaluations. We also sweep over the number of action samples푁on task success rate in simulation. As shown in Figure 9, steering performance generally improves as the number of action samples increases, suggesting that larger candidate sets provide better opportunities for verification. However, we find that performance saturates beyond 푁= 8, serving as a practical guide for real-world deployment. 7 Discussion In this work, we introduced a generator-verifier framework that transforms inference-time compu- tation into sustainable policy improvement. By decoupling reasoning from action generation, we demonstrated that pre-trained generalist policies can achieve substantial performance gains purely through inference-time steering. We further showed that successfully verified trajectories can be distilled into the base policy via post-training to improve the policy performance in a data-efficient manner. Across a range of simulated and real-world benchmarks, we demonstrate consistent perfor- mance improvements with the VERITAS with 20–100 trajectories of experience. Most notably, we find that autonomous verification matches the data efficiency of human-collected expert trajectories. This creates a scalable pathway for robot self-improvement, where deployment experience naturally translates into persistent capability improvements. While effective, our framework relies on trading inference compute for task performance, utilizing repeated sampling that may be computationally expensive in latency-critical applications. Additionally, our current verifier implementation depends on static visual traces generated at the onset of the task; while this is sufficient for quasi-static manipulation, it may struggle in highly dynamic environments where the scene changes rapidly during execution. Future work could address these constraints by investigating mechanisms to distill the verifier’s rejection logic into a value function for faster inference, or by exploring joint optimization of the generator and verifier to improve sample efficiency during the search process. Further, it is important to note that our verifier can only improve performance by selecting the best action candidate among those proposed by the policy. This is fundamentally limited by the pre-trained exploration prior learned by the policy, and we expect improvements in policy pre-training to further improve the verification pipeline. Acknowledgments This research was partially supported by the Toyota Research Institute and Microsoft Research, with compute support from Google TPU Research Cloud, NVIDIA Academic Grant Program, and Gemini Academic Program. The authors also thank Mitsuhiko Nakamoto, for help with reproducing V-GPS, and Samuel M. Bateman, Yanbo Xu, Hengkai Pan for helpful discussions. 14 References [1]Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint, 2025. [2]Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open X-Embodiment: Robotic learning datasets and RT-X models. In IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903, 2024. [3]Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, Hongkai Dai, Ching-Hsin Fang, Kunimatsu Hashimoto, Muhammad Zubair Irshad, Masha Itkina, et al. A careful examination of large behavior models for multitask dexterous manipulation. Science Robotics, 2026. [4]Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GR00T N1: An open foundation model for generalist humanoid robots. arXiv preprint, 2025. [5] Physical Intelligence. 휋 0 : A vision-language-action flow model for general robot control, 2024. [6]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), 2023. [7] Yejin Kim, Wilbert Pumacay, Omar Rayyan, Max Argus, Winson Han, Eli VanderBilt, Jordi Salvador, Abhay Deshpande, Rose Hendrix, Snehal Jauhri, et al. MolmoSpaces: A large-scale open ecosystem for robot navigation and manipulation. In Robotics: Science and Systems (RSS), 2026. [8]Abhay Deshpande, Maya Guru, Rose Hendrix, Snehal Jauhri, Ainaz Eftekhar, Rohun Tripathi, Max Argus, Jordi Salvador, Haoquan Fang, Matthew Wallingford, et al. MolmoB0T: Large-scale simulation enables zero-shot manipulation, 2026. [9]Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-1b: a diffusion foundation model for bimanual manipulation. In The Thirteenth International Conference on Learning Representations, 2025. [10]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, et al. OpenVLA: An open-source vision-language-action model. Annual Conference on Robot Learning (CoRL), 2024. [11] Noriaki Hirose, Catherine Glossop, Dhruv Shah, and Sergey Levine. OmniVLA: An omni-modal vision- language-action model for robot navigation. IEEE International Conference on Robotics and Automation (ICRA), 2026. [12]Dhruv Shah, Ajay Sridhar, Arjun Bhorkar, Noriaki Hirose, and Sergey Levine. GNM: A general navigation model to drive any robot. IEEE International Conference on Robotics and Automation (ICRA), 2023. [13]Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025. [14]Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. Are more LLM calls all you need? towards the scaling properties of compound AI systems. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [15] Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. S*: Test-time scaling for code generation. arXiv preprint, 2025. 15 [16]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint, 2021. [17]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations (ICLR), 2023. [18]Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems (NeurIPS), 36:11809–11822, 2023. [19]Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, and Mykel J. Kochenderfer. HG-DAgger: Interactive imitation learning with human experts. In IEEE International Conference on Robotics and Automation (ICRA), pages 8077–8083, 2019. [20]Zheyuan Hu, Robyn Wu, Naveen Enock, Jasmine Jia ni Li, Riya Kadakia, Zackory Erickson, and Aviral Kumar. Rac: Robot learning for long-horizon tasks by scaling recovery and correction. In Workshop on Making Sense of Data in Robotics: Composition, Curation, and Interpretability at Scale at CoRL 2025, 2025. [21]Xuanlin Li, Kyle Hsu, Jiayuan Gu, Oier Mees, Karl Pertsch, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation. In 8th Annual Conference on Robot Learning, 2024. [22]Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. In Robotics: Science and Systems (RSS), 2024. [23]Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning. Science Robotics, 2025. [24] Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot learning on the job: Human-in-the-loop autonomy and learning during deployment. In Robotics: Science and Systems (RSS), 2023. [25] Ruohan Zhang, Faraz Torabi, Lin Guan, Dana H Ballard, and Peter Stone. Leveraging human guidance for deep reinforcement learning tasks. arXiv preprint, 2019. [26] Anca D. Dragan and Siddhartha S. Srinivasa. A policy blending formalism for shared control. International Journal of Robotics Research (IJRR), 32(7):790–805, June 2013. [27]Shervin Javdani, Siddhartha S. Srinivasa, and J. Andrew Bagnell. Shared autonomy via hindsight optimization. Robotics: Science and Systems (RSS), 2015:10–15607, 2015. [28]Siddharth Reddy, Anca D Dragan, and Sergey Levine. Shared autonomy via deep reinforcement learning. In Robotics: Science and Systems (RSS), 2014. [29] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2011. [30]Xiaomeng Xu, Yifan Hou, Zeyi Liu, and Shuran Song. Compliant residual DAgger: Improving real-world contact-rich manipulation with human corrections. In The Thirty-ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2025. [31] Jonathan Spencer, Sanjiban Choudhury, Matt Barnes, Matt Schmittle, Mung Chiang, Peter J. Ramadge, and Siddhartha S. Srinivasa. Learning from interventions: Human-robot interaction as both explicit and implicit feedback. Robotics: Science and Systems (RSS), 2020. [32]Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning. In 8th Annual Conference on Robot Learning, 2024. 16 [33]Qingqing Zhao et al. CoT-VLA: Visual chain-of-thought reasoning for vision-language-action models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1702–1713, 2025. [34]Jason Lee, Jiafei Duan, Haoquan Fang, Yuquan Deng, Shuo Liu, Boyang Li, Bohan Fang, Jieyu Zhang, Yi Ru Wang, Sangho Lee, et al. MolmoAct: Action reasoning models that can reason in space. In 2026 IEEE International Conference on Robotics and Automation (ICRA), 2026. [35]Jacky Kwok, Christopher Agia, Rohan Sinha, Matt Foutter, Shulu Li, Ion Stoica, Azalia Mirhoseini, and Marco Pavone. Robomonkey: Scaling test-time sampling and verification for vision-language-action models. In 9th Annual Conference on Robot Learning, 2025. [36] Jacky Kwok, Xilun Zhang, Mengdi Xu, Yuejiang Liu, Azalia Mirhoseini, Chelsea Finn, and Marco Pavone. Scaling verification can be more effective than scaling policy learning for vision-language-action alignment. arXiv preprint, 2026. [37]Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, and Claudia P’erez-D’Arpino. Do what you say: Steering vision-language-action models via runtime reasoning-action alignment verification. In 2026 IEEE International Conference on Robotics and Automation (ICRA), 2026. [38]Mitsuhiko Nakamoto, Oier Mees, Aviral Kumar, and Sergey Levine. Steering your generalists: Improving robotic foundation models via value guidance. In Conference on Robot Learning (CoRL), 2024. [39] Yilin Wu, Ran Tian, Gokul Swamy, and Andrea Bajcsy. From foresight to forethought: VLM-in-the-loop policy steering via latent alignment. In Robotics: Science and Systems (RSS), 2025. [40] Jessie Yuan, Yilin Wu, and Andrea Bajcsy. When to act, ask, or learn: Uncertainty-aware policy steering. In Robotics: Science and Systems (RSS), 2026. [41] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2018. [42] Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipula- tion with low-cost hardware. In Robotics: Science and Systems (RSS), 2023. [43]Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), 2023. [44] Laura Smith, Alex Irpan, Montserrat Gonzalez Arenas, Sean Kirmani, Dmitry Kalashnikov, Dhruv Shah, and Ted Xiao. STEER: Flexible robotic manipulation via dense language grounding. In IEEE International Conference on Robotics and Automation (ICRA), 2025. [45]Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next-generation agentic capabilities. arXiv preprint, 2025. [46]Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, and Sergey Levine. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023. [47] Allen Z. Ren. Open-휋-zero: Re-implementation of휋 0 vision-language-action (VLA) model from Physical Intelligence, 2025. [48] Physical Intelligence.휋 0.5 : A vision-language-action model with open-world generalization. In 9th Annual Conference on Robot Learning, 2025. [49]Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, et al. PIVOT: Iterative visual prompting elicits actionable knowledge for VLMs. In International Conference on Machine Learning (ICML), 2024. [50]Joseph A. Vincent, Haruki Nishimura, Masha Itkina, Paarth Shah, Mac Schwager, and Thomas Kollar. How generalizable is my behavior cloning policy? a statistical approach to trustworthy performance evaluation. IEEE Robotics and Automation Letters (RA-L), 2024. 17 [51]Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. In International Conference on Learning Representations, volume 2025, pages 28085–28128, 2025. [52]Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159, 2024. 18 Appendix A Implementation Details A.1 Visual Verifier Design We mainly design and ablate 3 types of verifier design in our paper. We discuss the design of each of them: We denote the RGB observation at time푡as퐹 푡 , and the end-effector pixel location as푒 푡 ∈ ℝ 2 . LetI denote the natural-language task instruction. Object detections obtained from detection [45,51] or tracking models [52] are denoted asˆ표 푡 , where each detection contains a bounding box and a center pixel coordinate. A set of tracked objects is denoted byO. A waypoint sequence is denoted byW= 푤 1 , . . . , 푤 퐾 , where each waypoint may be expressed either as: • An absolute pixel coordinate 푤 푖 =(푢 푖 , 푣 푖 ), or • A reference expression (e.g.,role:source,midpoint(source,target)) that resolves to a pixel location. We denote the current active waypoint index as푖, and the total number of waypoints as퐾. The distance between the end-effector and the current waypoint is written as 푑 푡 =∥푒 푡 − 푔 푖 ∥, where 푔 푖 is the resolved pixel goal for waypoint 푖. Verification scores are normalized to the range[0,1]and denoted as푠 푡 . Progress through a waypoint sequence is defined as 푝 푡 = 푖 퐾 . For the heuristic verifier, the current stage of the finite-state machine is denoted by휎 푡 , and task progress is denoted by 푝 푡 , computed from geometric features such as distances and object motion. Hyperparameters include tolerance thresholdstol 푖 , smoothing windows, and weighting coefficients훼 and 휏 used in score computation. VERITAS The assumption for VERITAS design is that: General manipulation tasks can be naturally described as a spatial motion trace of the robot end-effector or gripper: such as approaching an object, moving along a surface, or transporting an object to a target location. While such traces are difficult to specify analytically for diverse tasks, vision-language models (VLMs) are highly capable of imagining plausible execution trajectories directly from task instructions and images. The VERITAS Verifier leverages this capability by asking a VLM to generate a single, absolute-pixel waypoint trace at the beginning of an episode. During execution, the verifier checks whether the robot’s end-effector/gripper follows this plan with sufficient fidelity. Specifically, the verifier follows a simple principle: If the robot’s end-effector motion stays close to a plausible pixel-space trajectory imagined by a VLM that has strong embodied reasoning capability, 19 the behavior is likely correct to complete the manipulation tasks. This design avoids explicit task stages, object-specific heuristics, or per-frame reasoning. Instead, it evaluates execution by measuring geometric consistency between the observed object trajectory and a generated waypoint visual trace. •Initialization: VLM-Generated Waypoint Trace. Given the task instruction and the initial image observation, the VLM identifies which objects in the scene are relevant to the tasks and assigns them semantic roles (e.g., source, target). These objects are used only to provide spatial hints through object detection and to help the VLM ground its reasoning. Second, using the same instruction and image with object detection results provided, the VLM generates a sequence of absolute pixel-space waypoints that describe how the robot should move to accomplish the task. Each waypoint specifies: a target pixel location(푢, 푣), a tolerance radius in pixels along with optional constraints such as minimum dwell time or whether the waypoint may be skipped. •Execution-Time Verification. During execution, the verifier observes the robot’s end-effector or gripper pixel location at each timestep. Verification is performed by checking whether the EEF/gripper: reaches each waypoint within its tolerance in the image space, respects ordering constraints between waypoints, maintains proximity for required durations when specified. As described in Algorithm 2, the verifier does not encode task-specific logic or any predefined action primitives (e.g., grasp, place, open). All task knowledge is captured implicitly in the waypoint trace generated by the VLM. It is robust to partial execution as these waypoints can be marked as skippable or constraints are soft, allowing the verifier to tolerate minor deviations, alternative but valid execution styles. Algorithm 2 VERITAS Verifier Input: InstructionI, initial frame 퐹 0 , EEF pixels푒 푡 Output: Verification score 푠 푡 ∈ [0, 1] W ← VLM Waypoint Trace(I, 퐹 0 ) Clamp|W| ∈ [5, 10], enforce start/end non-skippable 푖← 0 for 푡= 1 . . .푇 do 푑 푡 ← ∥푒 푡 −W[푖].푢푣∥ if 푑 푡 < tol 푖 for≥ min_hold frames then 푖← 푖+ 1 end 푝 푡 ← 푖/|W| 푠 푡 ← 훼푝 푡 +(1− 훼) exp(−푑 푡 /휏) end if 푖≥ |W| then 푠 푡 ← 1.0 end VLM-Constraints Verifier Similarly, we have a VLM-Constraints Verifier design that similar to VERITAS but with dynamic reference resolution. It evaluates robot behavior by comparing the executed robot motion against a reference execution trace generated by a vision-language model (VLM). Unlike absolute pixel-space waypoints, the reference trace is expressed as a sequence of relative, object- centric waypoint references that are dynamically resolved using object tracking during execution. The key insight is to separate what the robot should do from how the robot actually moves, and to verify correctness by checking whether execution follows a plausible reference trace. For general manipulation tasks, correctness is best judged not by low-level actions or final success alone, but by whether the robot follows a reasonable sequence of spatial intentions: approach the source object, move toward a target, align with an intermediate region, and so on. 20 •Reference-Based Waypoints Initialization. Similar to VERITAS, at the start of an episode, a VLM is queried with the task instruction and the initial observation. Note that instead of returning pixel coordinates, the VLM produces a list of objects to track, each with a semantic role (e.g., source, target), and a sequence of waypoints defined by references. These waypoint references specify what the robot should aim for, not where that is in pixels. This abstraction allows the reference trace to remain valid even as objects move during execution. •Dynamic Resolution via Tracking. During execution, the verifier runs an object tracker continuously. At every frame, each waypoint reference is resolved into a pixel-space goal using the latest tracked object positions. The composite references (e.g., midpoints) are resolved by combining multiple tracked entities and he resulting waypoint locations move with the scene, automatically adapting to object motion, occlusion, or partial interaction. •Execution-Time Verification. The verifier maintains an index over the waypoint sequence and checks, at each frame, whether the EEF is sufficiently close to the current waypoint. Progression through waypoints is governed by simple but robust mechanisms such as distance thresholds normalized by object scale, hysteresis to avoid oscillation near boundaries and minimum hold time to ensure stable attainment. Once all waypoints are completed, the task is considered done. Importantly, the verifier does not enforce a single rigid execution style; it tolerates variations as long as the robot follows the intended reference trace. The verifier explicitly encodes a reference answer, which is a plausible execution trace generated from VLM and checks whether the robot’s behavior aligns with it. Algorithm 3 VLM-Constraints Verifier Input: InstructionI, frames퐹 푡 , EEF pixels푒 푡 Output: Verification score 푠 푡 ∈ [0, 1] (O,W) ← VLM Waypoint Trace(I, 퐹 0 ) Initialize tracker with objectsOClamp|W| ∈ [5,10] 푖← 0 for 푡= 1 . . .푇 do ˆ표 푡 ← Tracker(퐹 푡 ) 푔 푖 ← ResolveRef(W[푖].푟푒푓, ˆ표 푡 ) 푑 푡 ← ∥푒 푡 − 푔 푖 ∥/scale(푔 푖 ) if 푑 푡 < tol 푖 for≥ min_hold frames then 푖← 푖+ 1 end else if stalled and next waypoint skippable then 푖← 푖+ 1 end 푝 푡 ← 푖/|W| 푠 푡 ← 훼푝 푡 +(1− 훼) exp(−푑 푡 /휏) end if 푖≥ |W| then 푠 푡 ← 1.0 end Heuristic Verifier The motivation for the Heuristic Verifier design is that, we assume most manipula- tion tasks can be decomposed into a small number of stages (approach, align, engage, manipulate, release, etc.) and that correct behavior looks like progressing through these stages in order. Specifically at its core, the verifier models execution as progression through a sequence of stages. We formalize this assumption using a finite-state machine (FSM) that tracks the current stage of the task based on simple geometric and kinematic signals: 21 APPROACH→ ALIGN→ ENGAGE→ MANIPULATE→ RELEASE→ SETTLE→ DONE. Each stage corresponds to a qualitatively different interaction regime. For example,APPROACHexpects the end-effector to move closer to a relevant object,ENGAGEexpects grasping or contact behavior, and MANIPULATEexpects the object to move toward a target. The verifier does not prescribe how these stages are achieved; it only checks whether the observed behavior is consistent with being in the current stage and whether transitions occur in a plausible order. During execution, the verifier continuously tracks the robot end-effector and relevant objects in the im- age. From these observations, it computes lightweight geometric features such as end-effector–object distances, object–object distances, relative motion between objects and the end-effector, and gripper open/close state. These features are temporally smoothed to reduce noise and serve as input to the stage transition logic. A finite-state machine consumes the extracted features and determines (1) which stage the robot is currently in, and (2) whether it is appropriate to remain in that stage or transition to the next one. Transitions are triggered by interpretable conditions such as the end- effector being sufficiently close to an object, an object moving consistently with the end-effector (indicating grasp), or the manipulated object approaching a target location. This structure enforces a coarse notion of temporal correctness without requiring explicit trajectory specification. In addition to discrete stage transitions, the verifier computes a continuous progress signal based on task-specific geometric measures, such as decreasing distance to a target. Changes in this signal are tracked over time to detect forward progress, stagnation, or regression. If progress stalls for a sustained period, the verifier can optionally invoke a vision-language model to diagnose potential failure modes (e.g., wrong object, misalignment, or obstruction) and suggest recovery actions. Please see 4 for more details and pesudo code of the finite state machine. A.2 Real-world Setup In the real-world experiment setup, policy observations consist of images captured from a wrist- mounted camera and an additional external third-person view camera, following [22]. To enable reliable visual verification, we further deploy a calibrated front-facing camera that observes both the robot arm and the workspace. This viewpoint reduces occlusions and provides a stable perspective for evaluating pixel-space visual traces. In simulation, the open-loop execution horizon of the휋 0 -Bridge policy is set to 4 steps, whereas in real-world experiments, the open-loop execution horizon of the 휋 0 -DROID policy is set to 8 steps. Tables 1 and 2 summarize the key hyperparameters used to fine-tune the휋 0 -Bridge and휋 0 -DROID policies, respectively. Unless otherwise specified, all remaining training settings follow the default configurations of the original implementations. We provide the prompts for the verifier to generate the visual trace. Specifically, prompts used for visual trace generation for VERITAS: 22 Algorithm 4 Stage FSM Update Input: 푠, engaged, released, progress_good, is_grasped, task_done Output: 푠 if 푠= Approach then if 푒푛푔푎푔푒푑∨ 푖푠_푔푟푎푠푝푒푑 then 푠← Align end else if 푠= Align then if 푖푠_푔푟푎푠푝푒푑∧ 푝푟표푔푟푒푠_푔표푑 then 푠← Manipulate end else if 푒푛푔푎푔푒푑∧ 푝푟표푔푟푒푠_푔표푑 then 푠← Engage end else if ¬푒푛푔푎푔푒푑 then 푠← Approach end else if 푠= Engage then if 푝푟표푔푟푒푠_푔표푑 then 푠← Manipulate end else if ¬푒푛푔푎푔푒푑∧¬푖푠_푔푟푎푠푝푒푑 then 푠← Align end else if 푠= Manipulate then if 푟푒푙푒푎푠푒푑 then 푠← Release end else if 푠= Release then if 푝푟표푔푟푒푠_푔표푑 then 푠← Settle end else if ¬푟푒푙푒푎푠푒푑 then 푠← Manipulate end else if 푠= Settle∧ 푡푎푠푘_푑표푛푒 then 푠← Done end end end end end end return 푠 23 Algorithm 5 Heuristic Verifier Input: InstructionI, frames퐹 푡 , EEF pixels푒 푡 , gripper states푔 푡 Output: Verification score 푠 푡 ∈ [0, 1], stage 휎 푡 (task_type,O) ← VLM Router(I, 퐹 0 )Initialize tracker with objectsOInitialize FSM stage휎 ← APPROACH for 푡= 1 . . .푇 do ˆ표 푡 ← Tracker(퐹 푡 ) 휙 푡 ← Extract Features(푒 푡 , ˆ표 푡 , 푔 푡 ) 휎 푡 ← FSM Update(휎 푡−1 , 휙 푡 ) 푝 푡 ← ComputeProgress(task_type, 휎 푡 , 휙 푡 ) if progress stalls then Optional: VLM Diagnosis(I,퐹 푡−푘:푡 ) end 푠 푡 ← Score(휎 푡 , 푝 푡 , 휙 푡 ) end Table 1 | 휋 0 -Bridge finetuning Hyperparameters. HyperparameterValue Global batch size1024 Per-device batch size16 Learning rate5× 10 −5 Weight decay0 Max grad norm1.0 Generate 5-10 absolute pixel waypoints (u,v) for the robot end-effector to complete the task. Return JSON matching schema: waypoints: ["uv": [u,v], "tol_px": float, "min_hold": int, "skippable": bool, "weight": float], confidence: float, notes: optional. Instruction: instruction if instruction else "No instruction provided". pixel coords are absolute. Prompts used for visual trace generation for the VLM-Constraints verifier: Generate a waypoint plan for a robot end effector. Use ONLY reference-based waypoints (ref strings): role:source, role:target, object:<name>, midpoint(refA,refB). DO NOT return any pixel coordinates. Return 5 to 10 waypoints, sequential, each with tol_norm and min_hold (frames to hold inside tolerance). The waypoints should guide approach, alignment, and placement using the referenced objects. 24 Spoon on TowelCarrot on PlateStack BlocksEggplant in BasketAverage 0% 20% 40% 60% 80% 100% BaselineVERITASRoboMonkey Success Rate Figure 10|Comparision with RoboMonkey. Simulation success rates for different data augmentation methods across the same 4 simulation tasks (50 episodes per task; RoboMonkey averaged over 3 random seeds). RoboMonkey consistently improves performance over the baseline, achieving an average success rate gains observed on all tasks. In contrast, VERITAS yields mixed outcomes, improving performance on some tasks while degrading others. This suggests that naive data augmentation without enforcing semantic consistency across action chunks can negatively impact performance. Note that this comparison uses single-step action perturbations rather than the action-chunk-level steering used in Figure 4; the aggregate performance difference is dominated by architectural mismatch rather than the algorithm. 25 Table 2 | 휋 0 -DROID Fituning Hyperparameters. Hyperparameter Value Warmup steps1000 LR decay steps1,000,000 Learning rate5× 10 −5 Batch size256 Figure 11 | Additional results of inference-time verification improves task performance. B Additional Experiments Additional Experiments on RoboMonkey As discussed in our experiments, we also compare VER- ITAS with RoboMonkey for online policy steering, with results shown in Figure 10. RoboMonkey demonstrates that, across a range of VLAs, the relationship between action error and the number of generated samples follows an exponentiated power law. Its verifier quantifies the discrepancy between VLA-generated actions and actions from an offline dataset, effectively measuring how closely a sampled action matches previously observed policy behavior. However, RoboMonkey operates at the level of single-step action perturbations, typically by applying Gaussian noise around a policy output. This form of exploration does not capture high-level action reasoning or alternative strategies. Instead, the verifier optimizes a local ridge in the action space, refining small deviations of a single behavior rather than comparing distinct reasoning traces. Because the verifier is trained on policy-like actions, it implicitly learns and reinforces the policy’s existing biases, and does not distinguish between qualitatively different strategies for accomplishing a task. In contrast, our verifier evaluates entire action chunks sampled at a higher temporal level, enabling comparison across diverse action sequences that correspond to different high-level plans. Rather than improving micro-scale action noise, our approach selects among semantically distinct execution traces, allowing the system to reason over alternative strategies instead of merely refining low-level control, as demonstrated in our inference-time steering results such as 4 and 6. 26 More qualitative results We show more quantitative results in Figure 11, to demonstrate the procedure of inference-time steering by verification in detail. 27