Paper deep dive

No Free Checker: A Survey of Verifiers for Robot Policies

Yang Wan, Xihang Yue, Zhirui Liu, Ziyuan Chu, Shuxun Wang, Yuhan Chen, Xiaonan Jiang, Xukun Zhu, Yubo Dong, Linchao Zhu

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 139

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 9/10/2026, 2:33:44 AM

Summary

This paper surveys approximately 150 verifiers for robot policies, categorizing them into four families based on the source of judgment: human, rule-based/formal, learned/pretrained, and model-intrinsic. The authors introduce two key properties for evaluating verifiers: Availability (cost, timing, and density of verdicts) and Credibility (how much a high score indicates true task success). A central finding is the 'No Free Checker' trade-off, where higher availability correlates with lower credibility. The paper also discusses methods for validating verifiers themselves, such as agreement with human labels, policy performance, and resistance to reward hacking, and proposes nine metrics for making verifier claims checkable.

Entities (11)

Robot Verifier · concept · 98%Availability · property · 97%Credibility · property · 97%No Free Checker · principle · 96%Human Verifiers · category · 95%Learned Verifiers · category · 95%Model-Intrinsic Verifiers · category · 95%Rule-Based Verifiers · category · 95%Reward Hacking · phenomenon · 92%Signal Temporal Logic · method · 90%Vision-Language-Action Models · technology · 90%

Relation Signals (9)

Availability → correlateswith → Credibility

confidence 95% · Across the four families, we find that credibility falls as availability rises.

Rule-Based Verifiers → uses → Signal Temporal Logic

confidence 92% · This literature writes specifications in Signal Temporal Logic (STL)

Human Verifiers → hasproperty → High Credibility

confidence 90% · In terms of credibility, they provide a direct reference to task intent

Human Verifiers → hasproperty → Low Availability

confidence 90% · In terms of availability, human judgments are costly and sparse

Learned Verifiers → hasproperty → High Availability

confidence 90% · In terms of availability, these verifiers are inexpensive to query and can provide dense feedback

Verifier Validation → measuredby → Reward hacking behavior

confidence 90% · behavior under reward hacking

Verifier Validation → measuredby → Agreement with human labels

confidence 90% · Three measures appear in the literature: agreement with human labels

Verifier Validation → measuredby → Policy Performance

confidence 90% · the performance of the policy it trains

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:A verifier for robot policies reads a candidate behavior and returns a score for how well it did, used both to evaluate vision-language-action policies and to train them. Verifiers range from success detectors and reward models to runtime monitors, safety filters, and temporal-logic specifications. We survey roughly 150 verifiers and compare them along two properties. Availability is how much a verdict costs, how early in a rollout the verdict arrives, and how often a verdict can be asked for. Availability rises as verdicts get cheaper, earlier, and denser. Credibility is how much a high score tells us about the task. Credibility falls as the judgment becomes gameable and self-serving. We group the verifiers by who supplies the judgment: human verifiers, rule-based and formal verifiers, learned and pretrained verifiers, and model-intrinsic verifiers. Across the four families, we find that credibility falls as availability rises. Regardless of who supplies the judgment, there is no free checker. We then examine what validates a verifier itself, and how much a high score tells us. Three measures appear in the literature: agreement with human labels, the performance of the policy it trains, and behavior under reward hacking. We close with nine metrics that make a verifier claim checkable, and coordinates for the verifiers still to be built.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

135,946 characters extracted from source content.

Expand or collapse full text

arXiv:2609.09250v1 [cs.RO] 8 Sep 2026 Preprint No Free Checker: A Survey of Verifiers for Robot Policies Yang Wan 1 , Xihang Yue 1 , Zhirui Liu 1 , Ziyuan Chu 1 , Shuxun Wang 1 , Yuhan Chen 1 , Xiaonan Jiang 2 , Xukun Zhu 1 , Yubo Dong 1 , Linchao Zhu 1 1 Zhejiang University, 2 City University of Hong Kong § Awesome-Robot-Verifier A verifier for robot policies reads a candidate behavior and returns a score for how well it did, used both to evaluate vision-language-action policies and to train them. Verifiers range from success detectors and reward models to runtime monitors, safety filters, and temporal-logic specifications. We survey roughly 150 verifiers and compare them along two properties. Availability is how much a verdict costs, how early in a rollout the verdict arrives, and how often a verdict can be asked for. Availability rises as verdicts get cheaper, earlier, and denser. Credibility is how much a high score tells us about the task. Credibility falls as the judgment becomes gameable and self-serving. We group the verifiers by who supplies the judgment: human verifiers, rule-based and formal verifiers, learned and pretrained verifiers, and model-intrinsic verifiers. Across the four families, we find that credibility falls as availability rises. Regardless of who supplies the judgment, there is no free checker. We then examine what validates a verifier itself, and how much a high score tells us. Three measures appear in the literature: agreement with human labels, the performance of the policy it trains, and behavior under reward hacking. We close with nine metrics that make a verifier claim checkable, and coordinates for the verifiers still to be built. Keywords: Robot policy evaluation, Reward models, Vision-language-action models, Runtime monitoring, Safety filters, Reward hacking, World models, Data curation for imitation learning 1 Contents 1 Introduction3 2 Human Verifiers4 2.1 Comparing Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Scoring Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Intervening During Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Validating Generated Rollouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3 Rule-Based and Formal Verifiers7 3.1 Full-Trajectory Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2 Terminal-State Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Physical-Constraint Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4 Learned and Pretrained Verifiers12 4.1 Evaluating Behavior and Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2 Inference-Time Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.3 Feeding Verifier Feedback into Policy Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5 Model-Intrinsic Verifiers16 5.1 Policy-Based Self-Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2 Prediction-Based Outcome Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.3 Beyond Action Verification: Task Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 6 Validating the Verifier19 6.1 Agreement on a Fixed Set of Rollouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 6.2 Measuring a Verifier by the Policy It Trains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6.3 Measuring a Verifier Under Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 6.4 Metrics for Validating a Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 7 Conclusion22 2 1 Introduction In robot learning, vision-language-action (VLA) policies (1,2,3,4,5) and robot world models (6,7) are scaling rapidly and becoming more capable. While scaling remains important, verification of training data and model behavior is also essential, as demonstrated by recent progress in large language models (8). Such verification signals are used at multiple stages of robot learning, including filtering and reweighting demonstrations for pre-training (9,10,11), providing reward signals during policy post-training (12,13,14), ranking candidate actions at inference time (15,16,17), and evaluating policy performance using rollouts generated by world models (18,19,20). We refer to the mechanisms that provide these verification signals collectively as verifiers. A robot verifier is a mapping verifier : (context, candidate) ↦−→ score.(1) The context comprises observations, a goal, and possibly a language instruction. The score can be boolean, scalar, vector-valued, or a distribution. The candidate varies the most of the three, running from a single action chunk up to an entire policy, with subtask segments and full trajectories in between. Building reliable verifiers for robot learning is particularly difficult because the physical world lacks the cheap and well-defined criteria available in mathematics and code. First, determining whether a physical task has been completed correctly is costly because task success is often inferred from imperfect sensory observations (21, 22,23). Second, evaluating a policy is expensive because testing a policy may require real-robot rollouts, which consume physical time and limit large-scale sampling (18,20,24). Third, verification often has to happen before the task is finished, because a single incorrect action is enough to cause task failure or hardware damage (25,26,27,28,29). Finally, task performance is often continuous, as robot behaviors can differ in progress, safety, and execution quality, making a single success-or-failure label insufficient (30, 31, 32, 33). Given these challenges, different verifiers offer different strengths and limitations in robot learning. To systematically understand existing methods, we survey roughly 150 works on verification in robot learning. Figure 1 places representative systems by family and year, and Table 1 records what each family’s score asserts. Further, we characterize these works along two properties, availability and credibility. Availability is how much a verdict costs, how early in a rollout the verdict arrives, and how often a verdict can be asked for. Credibility is how much a high score tells us about the task. The source of judgment strongly affects the availability and credibility of a verifier. We therefore group existing methods by judge source into four families: (a) human verifiers, where the judgment is provided directly by a person, (b) rule-based and formal verifiers, where the judgment is determined by predefined criteria, (c) learned and pretrained verifiers, where the judgment is produced by a neural model, (d) model-intrinsic verifiers, where the judgment comes from signals already computed by the model itself. •Human verifiers (Section 2). A person directly judges task success, compares trajectories, or intervenes during execution (34,35,36). In terms of availability, human judgments are costly and sparse because judgments take human effort. In terms of credibility, they provide a direct reference to task intent, although they may still be subjective or inconsistent. •Rule-based and formal verifiers (Section 3). The judgment is determined by predefined criteria, such as task predicates, temporal-logic specifications, or formal safety conditions (37,38,28). In terms of availability, these verifiers are inexpensive and repeatable once the required state information is available. In terms of credibility, they can provide strong and sometimes formal guarantees, but only when the predefined criteria, state estimates, and dynamics assumptions accurately represent the task. •Learned and pretrained verifiers (Section 4). A neural model produces the judgment, either after task- specific training or directly from pretraining (39,40,33). In terms of availability, these verifiers are inexpensive to query and can provide dense feedback across many tasks and trajectories. In terms of credibility, their judgments depend on model accuracy, calibration, and generalization beyond the data on which they were trained or validated. •Model-intrinsic verifiers (Section 5). The judgment is derived from signals already computed by the policy or predictive model, such as action uncertainty, model uncertainty, or learned reachability (17,41,42). In terms of availability, these signals are the easiest to obtain because they require little or no additional 3 202020212022202320242025 2026 Human Rules and formal Trained model Training-free model Model-intrinsic RewardSketch (35) PEBBLE (43) B-Pref (44)HACO (45) Sirius (46)Sirius-Fleet (47) RoboArena (36) HIL-SERL (48) VeoEval (49) pred. filter (50) CALVIN (51) conf. STL (52) LIBERO (53) MimicGen (54) Eureka (55) PNCBF (28) ReKep (56) CodeMonitor (25) TGPO (38) ConRFT (57) SafeManip (37) RINSE (58) SimpleVLA-RL (12) DWBC (59) ILEED (60) VIP (31) LIV (32) SayCan (61) RoboCat (62) RECAP (40) VLAC (63) Robometer (21) DataMIL (11) GigaWorld-1 (20) KnowNo (64) REFLECT (65) VLM-RM (66) GRAPE (67) GVL (30) WorldEval (18) TOPReward (33) SRPO (68) MOPO (41) MOReL (69) LOMPO (70)VIPER (71)DiffReward (72) SAFE (26) Sentinel (27) TACO (73) MG-Select (17) HideSeek (74) AnySafe (75) dashed ring: full-year estimate Figure 1 Representative systems by judge source and year, using the same five columns as Table 1. Disc area is the number of papers we cover in the corresponding year. Our search closes in early July 2026, so each 2026 disc carries a dashed ring at the full-year estimate, twice the observed count at the January-to-June rate. computation or external supervision. In terms of credibility, model-intrinsic signals describe the model itself, and their relation to actual task performance is therefore indirect. Across these four families, the source of judgment shows a clear trade-off between availability and credibility. Human judgments and explicit task criteria provide a more direct measure of task performance, but human judgment is costly to collect at scale, and an explicit criterion holds only where its state estimates and physical assumptions represent the task (36,37,28). Learned and model-intrinsic signals are cheaper to query and can provide denser feedback, but a learned judgment depends on how well the model generalizes beyond its training data, and an intrinsic signal describes the model itself rather than task performance (21,33,17,41). Therefore, in the physical world, obtaining both high availability and high credibility at the same time is costly. We refer to this observation as no free checker. We therefore examine in Section 6 how a verifier’s own error is measured, and why credibility is difficult to establish in robotics. Three metrics appear, agreement with a fixed reference, the policy that results from training on the verifier, and the verifier’s behavior under a search for the inputs where its score is wrong. We close the section with the metrics whose reporting makes a verifier checkable by someone else. Section 7 concludes. 2 Human Verifiers A human verifier asks a person to look at what the policy did and say whether the behavior is good. The four methods in this section differ in what the person sees and in what the person is asked to return. The person sees two trajectories and says which one is better (§2.1). The person sees one trajectory and gives a score at every time step (§2.2). The person watches a policy while it runs, decides whether to take over, and supplies the correct action (§2.3). The fourth method changes the object rather than the question, showing the person a generated rollout and asking whether that rollout preserves the outcome the real execution would have reached (§2.4). All four methods share one limitation, that verdicts are costly to obtain. A human verdict can apply to any task and is the label closest to the intended task, so these verdicts also serve as training data for cheaper verifiers. Each subsection below covers how its labels enter the cheaper verifier’s training. 4 Learned and pretrained verifiers What the score assertsHuman Rules and formal Trained model Training-free model Model- intrinsic a claim about the behavior task successHIL-SERL (48) MimicGen (54), Eureka (55), SimpleVLA-RL (12) RECAP (40), RoboMonkey (15), SuccessVQA (39) WorldEval (18), WorldGym (19) MG-Select (17) progress reward sketching (35) VIP (31), VLAC (63), Robo- Dopamine (23) GVL (30), TOPReward (33) VIPER (71), Sentinel (27) safety, constraint ThriftyDAgger (76), HG-DAgger (77) PNCBF (28), SafeManip (37), ConformalSTL (52) RealTimeVerif (78)GRAPE (67) latent safety filters (42) execution quality RINSE (58), Video2Reward (79) RECAP (40), Robo- Dopamine (23) GRAPE (67) human preference PEBBLE (43), RoboArena (36), B-Pref (44) Robometer (21) a claim about the policy failure, uncertainty ThriftyDAgger (76), Sirius (46) Code-as- Monitor (25) Foresight (80), AHA (81), RoboFAC (82) KnowNo (64), FOREWARN (83) SAFE (26), Sentinel (27) a claim about the training data training value of a demonstration Sirius (46)RINSE (58) Re-Mix (9), DataMIL (11), Demo-SCORE (10) none 1 2–3 4–6 7–11 12+ Table 1 What the score asserts, plotted against the judge source supplying the criterion. Each cell names representative systems, and the fill depth is the number of papers there. Each paper is counted in exactly one judge-source column, and a paper asserting two properties appears in both rows. 2.1 Comparing Trajectories A human preference comparison presents two trajectory segments and asks which segment shows better task performance, and the output is therefore a relative judgment. RoboArena uses this judgment directly, conducting double-blind comparisons of fixed policies and aggregating the human preferences into a policy ranking (36). Preference-based learning uses the same interface as supervision for a scalable learned verifier. Christiano et al.(34)train a reward model from human comparisons and iteratively use the predicted rewards of that model to train a policy model. PEBBLE reduces the required human labels through unsupervised pre-training and by relabelling stored experience whenever the reward model changes (43). B-Pref replaces people with simulated teachers whose controlled irrationalities test the robustness of preference-based reinforcement- learning algorithms (44). Across these methods, the interface remains pairwise, but the judge source moves from direct human judgment to a learned or simulated proxy. 2.2 Scoring Trajectories Instead of comparing two trajectories, a person can provide scalar feedback by assigning a numerical score to each time step in a single trajectory. Cabi et al.(35)call this annotation procedure reward sketching. The 5 2.1 Comparing Trajectories Given two trajectories, determine which trajectory performs the task better 2.2 Scoring Trajectories Draw a time-indexed reward curve for progress 2.3 Intervening During Execution Human intervene at risky states and provide corrective actions 2.4 Validating Generated Rollouts Human judgments validate outcome preservation of generated rollouts Task: Place the blue cube on the platform. A B A is better than B Generated rollout shows success where the real execution would fail Manual intervention avoids the occurrence of collision. Human intervention Original Trajectory 0 1 success Video generation world model × Figure 2 Four roles of the human verifier. A person can compare trajectories (§2.1), assign scalar or time-indexed scores (§2.2), or intervene during execution (§2.3). The first three roles judge policy behavior, and the fourth validates generated rollouts and the verifiers applied to them (§2.4). annotator draws a curve with values from 0 to 1 over the timeline of a recorded robot trajectory, while the tool displays the video frame corresponding to the current point on the curve, and the curve therefore records perceived progress toward the target task frame by frame. A small set of these curves trains a reward model that then scores every time step of a much larger corpus. 2.3 Intervening During Execution During execution, a human supervisor can watch the system and take over the controls when the policy model is about to act incorrectly. These methods split by who decides that the moment to take over has come, a person in the human-gated setting and a learned verifier in the robot-gated setting. In the human-gated setting, the person decides when autonomous control stops and supplies corrective actions. The intervention serves two purposes. An intervention can prevent an unsafe or unproductive action before that action causes failure, and it provides a corrective training example at a state where the policy model needs help. HG-DAgger studies this interaction in simulated and real-world autonomous driving (77), extending the correction-collecting loop of DAgger to a supervisor who chooses the moments (84). The driver takes over in unsafe states, and the corrective actions become demonstrations for training the policy model. Although the training-time interventions of HG-DAgger remain human-gated, HG-DAgger also calibrates a threshold on the policy’s own uncertainty using the states where the driver took over, and the trained policy model can therefore flag comparable states on its own. HACO applies human-gated interaction to simulated autonomous driving, using partial demonstrations to shape a proxy value function that guides policy optimization while discouraging reliance on further interventions (45). Sirius applies human-gated intervention to real-robot manipulation during deployment. The operator takes over at difficult states, and the recorded interventions are weighted by the robot’s past reliability at that state before entering the policy update (46). HIL-SERL keeps the operator in the loop through real-world reinforcement learning, where corrective takeovers enter the replay buffer alongside demonstrations and drive precise manipulation tasks to near-perfect success within a few hours (48). Continuous human monitoring is costly, and robot-gated methods therefore automate the intervention decision. In simulated and physical manipulation tasks, ThriftyDAgger uses a switching policy between robot and human control to request help when the current state is unfamiliar or likely to lead to task failure, subject to a 6 human-intervention budget (76). AIM learns a proxy Q-function that approximates the human intervention rule and requests assistance when the learned proxy value indicates the policy model’s action deviates from the expert’s action, tested in autonomous-driving and grid-world navigation simulations (85). Sirius-Fleet moves the same decision to a deployed fleet, where visual world models predict failure ahead of time and the thresholds are relaxed as the policies improve (47). In both settings the person still supplies the corrective actions. An intervention verifier must request help early enough to avoid failure but selectively enough to limit human effort. 2.4 Validating Generated Rollouts The first three roles judge the behavior of a real robot. The fourth role judges a generated rollout, produced by a world model or a simulator in place of a real execution. Reading such a rollout takes two judgments, one on the task success of the policy that acted and one on the fidelity of the video the world model produced. The person is asked whether the rollout ends in the same success or failure that the real execution would have reached, and we call this property outcome preservation. GigaWorld-1 asks for both judgments in one number. Annotators rate long-horizon generated rollouts on a four-level World Model as Evaluator Score, which reads task success together with video fidelity (20). A fine-tuned vision-language model then reproduces these ratings across a corpus too large for any annotator to watch. VeoEval keeps a person on the reading itself. VeoEval runs policies inside a Veo world simulator over nominal, out-of-distribution, and red-teaming conditions, and a person scores the success of each generated rollout (49). When outcome preservation fails, a generated rollout shows the task succeeding where the robot would have failed, and the policy then looks better than its real performance. The reference for outcome preservation is the outcome that the real execution would have reached, and with no real rollout available, a person reads the reference directly. A model tuned on the person’s judgments then carries the reference to scale. Across all four roles, the human verdict becomes supervision for a cheaper verifier. Comparisons and scores train reward models, interventions enter the replay buffer as corrective demonstrations, and ratings of generated rollouts train the model that rates the rest of the corpus. The cost of each substitution depends on how far the cheaper verifier’s training data is from the data the verifier is later applied to (§6.1). Small samples and optimization against a fallible verifier are treated as cross-cutting limits in §6. 3 Rule-Based and Formal Verifiers A rule-based or formal verifier applies a criterion written before the run, and therefore returns a verdict without asking a person at inference time. We group these procedures by what the criterion reads, a full trajectory (§3.1), a terminal state (§3.2), or a physical constraint (§3.3). Figure 3 shows the three groups, and Table 2 names the six methods those groups contain. 3.1 Full-Trajectory Verification Trajectory-level verification methods process full state sequences. Three types appear, hand-written temporal- logic scoring (§3.1.1), geometry-based trajectory scoring (§3.1.2), and model-written verification code (§3.1.3). 3.1.1 Temporal-Logic Scoring Temporal-logic scoring is the trajectory-level method appearing most often in the work we cover. The method states the criterion as a formal specification and scores a trajectory by its distance from satisfying that specifica- tion. A specification states the required behavior and its timing, in a formal language a program can check a run against. This literature writes specifications in Signal Temporal Logic (STL), and the score is a real number called the satisfaction margin: satisfaction-margin specification, trajectory ∈ℝ.(2) The margin is positive when the specification is satisfied and negative when it is violated, and the magnitude of the margin gives how close the trajectory ran to the boundary. If the specification says “the gripper must close within 0.25m of the table”, closing at 0.1m gives a positive margin and 0.3m a negative one, and 0.05m 7 Rule-Based and Formal Verifiers 1 3.2 Terminal-State Verification benchmark evaluation data generation and filtering sparse RL reward Typical Uses final stategoal predicate final result True 1 False 0 3.3 Physical-Constraint Verification dynamics model Invariance Certificate safe set certified trajectory (kept inside safe set) CBF certificate Symbolic Feasibility candidate planpre-run symbolic checkfeasibility decision c1 c2 ...... c1 c2 ...... feasible infeasible 3.1 Full-Trajectory Verification Geometry-Based Scoring full trajectory demonstration smoothness & self-similarity geometry- based scoring 0.86 Temporal-Logic Scoring full trajectory formal rule satisfaction margin 3 Model-Written Rules executable rule code LLM optimization/ monitoring Figure 3 Overview of rule-based and formal verifiers. Full-trajectory methods (§3.1) score complete rollouts using temporal-logic specifications, trajectory geometry, or model-written code; terminal-state methods (§3.2) check whether the final state satisfies the goal; and physical-constraint methods (§3.3) verify feasibility or safety through formal constraints and certificates. scores higher than 0.2m because it leaves more room. This quantity is called robustness in the temporal-logic literature. The same satisfaction margin applies to a pre-recorded execution (37), to a simulated rollout (38), and to a future trajectory predicted by a trained model (52). The margin is computed the same way for every rollout source. Practical systems differ in one respect, whether an extra calibration step is added on top of the raw satisfaction margin. Calibrated predictive runtime verification. Predictive runtime verification calibrates the satisfaction margin with conformal prediction (52,87). Conformal prediction is a method that attaches its own margin to the output of any predictor. The method runs the predictor on a held-out set with known outcomes, records the size of the resulting errors, and takes a quantile of those recorded errors as the margin carried by every later prediction. For example, a margin set at the ninety-fifth percentile of the held-out errors gives an interval that contains the true outcome in ninety-five percent of later cases. The bound holds for any predictor and any data distribution, as long as the held-out cases and the cases judged later are drawn from the same pool, so the order of the two sets carries no information about their errors. Drawing both from the same pool is the condition called exchangeability, and §6.3 shows how training a policy against a verifier breaks that condition. In Lindemann et al.(52), the trajectory predictor is treated as a black box and predicts future states from current observations. The calibrated error bounds of that predictor become an interval on the satisfaction margin, and the system can therefore warn that a specification is about to be violated several time steps in 8 MethodCore CharacteristicRepresentative Work Full-trajectory Verification Temporal-Logic Scoring (§3.1.1)Quantifies how much a trajectory satisfies a formal specification ConformalSTL (52), TGPO (38) Geometry-Based Scoring (§3.1.2)Returns a smoothness score and a self-similarity score against a smoothed version of the trajectory itself RINSE (58) Model-Written Rules (§3.1.3)LLM writes the verification code, ranked by a task metric or only tested for execution Eureka (55), Code-as-Monitor (25) Terminal-state Verification Goal Predicate (§3.2)A true/false check on the final state, simulation only LIBERO (53), CALVIN (51) Physical-constraint Verification Invariance Certificate (§3.3)A formal proof that holds for all possible trajectories PNCBF (28) Symbolic Feasibility (§3.3)A pre-run check that a plan is physically possibleText2Motion (86) Table 2 Six verification schemes with representative works. advance. Two extensions carry conformal bounds on the satisfaction margin into settings the original method ex- cludes. Robust conformal prediction handles a distribution shift between calibration and deployment (88). A distributionally-robust version extends the approach to spatio-temporal specifications (87). A related but distinct method replaces the STL formalism altogether. Nath et al.(89)work in the latent space of a world model instead, where they conformally calibrate the dynamics-prediction error and a learned safety-classifier score. The calibrated bounds build a robust constraint set for model-predictive control, and safety constraints are therefore enforced during closed-loop execution from pixels. Uncalibrated specification evaluation. Specifications can also be used without any calibration layer, either as an offline monitor or as an optimization reward. SafeManip defines temporal safety properties in Linear Temporal Logic over finite traces (LTLf), a logic for specifying sequences of discrete events over a finite time horizon. SafeManip compiles each property into an automaton and checks pre-computed simulation rollouts against these automata (37). The output is a boolean violation with a timestamp and duration. These predicates read the simulator’s state directly, and the same checks on hardware therefore require perception, which SafeManip leaves for future work. When exact ground-truth states are available, the satisfaction margin can be used directly as an optimization reward, providing a dense learning signal at every time step of policy training. TGPO decomposes an STL formula into subgoals and invariant constraints, and builds a staged dense reward from them, solving the sparsity problem that makes specification-driven learning difficult (38). Cycle experience replay compiles LTL into an automaton and shapes a dense reward from its accepting cycles (90). 3.1.2 Geometry-Based Scoring Geometry-based scoring reads the spatial shape of the trajectory alone, and no task specification has to be written by hand. This method applies wherever a set of recorded demonstrations is available. RINSE scores each demonstration with two geometric measures. Spectral arc length gives a smoothness score that penalizes high-frequency jitter, and trajectory envelope distance gives a self-similarity score, measuring how far the trajectory deviates from a Bézier-smoothed version of itself. Both scores are reference-free and need no other demonstrations to compare against. RINSE trains on the highest-scoring subset and performs better than training on the full set (58). Spectral arc length and envelope distance measure execution quality rather than task success, and a smooth 9 trajectory can therefore still fail the task while a jerky trajectory can succeed. RINSE’s scores agree closely with the mixture weights that Re-Mix learns over a heterogeneous corpus (§4.3). These two scores therefore serve as a lightweight filter over training data, and a separate check has to determine whether the task was completed. 3.1.3 Model-Written Rules A model writes the verification logic itself. The generated code is used in two ways, as a training objective and as a runtime check. Optimization. A model asked for a reward function produces code, and a search loop keeps the candidates that train a good policy. Eureka runs the loop (55). Eureka reads the environment code and a task description, writes candidate reward functions in Python, trains a policy with each candidate, ranks the candidates by the environment’s own ground-truth task metric evaluated on those training runs, and mutates the best candidate, repeating the cycle. Ranking candidates by the task metric checks the generated code against what the task requires, and the monitoring systems below have no equivalent check. The reward Eureka converges to outperforms human-written rewards on most tasks in a large benchmark. DrEureka adds safety instructions and automatic domain-randomization ranges so the learned reward transfers to real hardware (91). Text2Reward and language-to-rewards generate dense reward code from natural language descriptions, using iterative human feedback instead of an evolutionary loop (92,93). Video2Reward derives rewards for legged robot behavior from video instead of text (79). Monitoring. Code-as-Monitor uses a multimodal model to generate a spatio-temporal constraint verifier, and applies geometric abstractions of the constrained elements to keep the check fast enough for the control loop (25). Code-as-Monitor tests each generated verifier before use, to confirm that the generated code runs. ReKep uses a model to express manipulation goals as relational keypoint constraints, which a solver then optimizes into end-effector motion at control rate (56). Neither Code-as-Monitor nor ReKep checks the generated constraint against what the task requires. 3.2 Terminal-State Verification A goal predicate is a simple true/false test on the final state. A goal predicate is a formal criterion like the temporal-logic specifications of §3.1.1, but it reads only the last state and returns a bit rather than a margin. Such a predicate needs the object’s pose, which a physics engine reports exactly but perception only estimates, so this method is confined to simulation. The same cheap test scores benchmarks, generates training data, and supplies reinforcement learning rewards. Benchmark evaluation. A goal predicate appears in policy benchmarks as the built-in success check. LIBERO defines the goal of each task as a predicate over simulator state, evaluates that predicate as the episode runs, and reports the fraction of episodes the predicate accepts, across four suites that vary spatial layout, objects, goals, and combinations of all three (53). Most of the reinforcement learning and test-time selection results in this survey are measured on LIBERO, and the sparse reward those runs optimize is exactly this success bit. CALVIN chains five language-conditioned subtasks and counts the number completed in order (51). VLABench and GenManip split the predicate into stages for the same reason, so that a long-horizon policy earns partial credit for the stages the policy did reach (94,95). RoboCasa builds 100 kitchen tasks on the same simulator with generated scenes and assets (96). RoboVerse unifies embodiments and environments across simulators (97). ManiSkill provides parallelized simulation (98), and Isaac Lab-Arena runs a whole evaluation the same way, and the number of trials therefore stops being set by how long a robot takes to run one trial (99). RoboTwin adds strong domain randomization to a bimanual setting (100). BEHAVIOR-1K extends the task distribution toward everyday activities (101). THE COLOSSEUM holds the task fixed instead and reads one predicate under a fixed set of perturbations to appearance, lighting, and background (102). Each predicate is written once and then determines every result the benchmark produces, and an error in the predicate therefore affects the ranking of policies. A policy benchmark reports the policy’s success rate, and the error rate of the check behind that number is reported far less often (§6.1). 10 Synthetic data generation and filtering. A boolean check on exact simulator state costs almost nothing to run, so the same check can generate training data as well as verify it. MimicGen runs this generate-and- verify loop (54). MimicGen takes a small number of human demonstrations, breaks each demonstration into object-centric segments, adapts those segments to new object poses to generate thousands of trials, runs the trials, and keeps only the trials that pass the task-level success check. Generation and verification are the same loop, and the predicate makes the loop run automatically. DexMimicGen uses the same approach for bimanual dexterous manipulation (103). DemoGen targets real hardware instead, where running the generated trials to verify them is expensive, and synthesizes spatially augmented point-cloud observations without running or checking any trial (104). Three systems move the task design itself into the loop. GenSim uses a language model to write simulation task code and expert demonstrations, and keeps a candidate task only if a single-task policy trained on that task achieves a success rate above a threshold (105). GenSim2 scales this method to articulated objects with a multimodal planner (106). RoboGen automates task proposal, scene construction, and supervision generation, and its authors note that verifying the resulting skills at scale remains an open problem (107). Sparse reward for reinforcement learning. A goal predicate also serves as a reinforcement learning reward, one bit per episode. SimpleVLA-RL shows this most directly (12). SimpleVLA-RL runs Group Relative Policy Optimization (GRPO) on a VLA model, uses the simulator’s success check as the only reward, and provides the parallel rendering and exploration changes needed to keep training stable. A policy initialized from just one trajectory per task then reaches near-perfect performance. RIPT-VLA achieves a similar result with leave-one-out policy-gradient estimation on the same sparse binary reward, deliberately avoiding shaped rewards, value functions, and reward models (13). RLinf-VLA provides the infrastructure for running such training across different algorithms and benchmarks (14). ConRFT applies a similar construction to real-world fine-tuning, substituting a binary classifier trained on human-labeled demonstrations for the simulator predicate. ConRFT combines behavior cloning and Q-learning in one objective under a consistency policy, shifting the weight from behavior cloning to Q-learning as training moves from offline to online (57). The predicate needs exact object pose, and the same question on hardware therefore requires a different verifier, where pretrained multimodal models have become the default (§4). The predicate also reads only the final state, and a run can satisfy the predicate exactly and still fail the task. A policy trained against such a check learns to aim at exactly the runs that satisfy the predicate without doing the task, a pattern demonstrated so far on LLM verifiers for inductive-reasoning tasks rather than on robot policies (108). Skalse et al.(109)define this failure mode and Manheim and Garrabrant(110)categorize its variants, and §6.3 takes up how a robot verifier is measured against this failure mode. 3.3 Physical-Constraint Verification Physical-constraint verification requires that the robot and the surrounding objects stay where they are allowed to be. For example, the arm must not touch the person standing next to the robot, and the cup at the edge of the table must not be pushed off. Two methods appear below. An invariance certificate takes an analytical model of the system dynamics and tests whether the robot can be kept out of unsafe states, and it answers with a proof. A symbolic feasibility check takes the preconditions of each action and tests whether a plan is physically possible at all, and it answers before anything moves. An invariance certificate that holds for every trajectory. The certificate here is a control barrier function (111), a mathematical tool that takes the dynamics model and keeps the robot inside a safe set, the region of states the robot is allowed to occupy. The barrier function proves that a robot starting inside the region stays inside the region under the closed-loop dynamics, a property called forward invariance, and the guarantee therefore holds for every trajectory those dynamics can produce. So et al.(28)construct such a function from the value function of a nominal policy, and the result is a filter that makes any policy safe. A filter of this kind monitors the control input that a policy proposes and overrides that input when the input would leave the safe set, a construction that also covers predictive filters solving a short constrained optimization over a fixed horizon at every control step (50,112). The theorem applies to the exact value function. A neural network approximating that function carries the same guarantee only when a separate verification step checks the descent condition, and the experiments in So et al.(28)do not perform that step. Tan et al.(113)identify value 11 ApplicationsTrained verifierPretrained verifier Evaluating behaviour and policies (§4.1) VIP (31), Robometer (21), AHA (81), RoboFAC (82), AutoEval (116) GVL (30), TOPReward (33), WorldEval (18), WorldGym (19) Inference-time verification and scaling (§4.2) RoboMonkey (15), RoVer (117), Foresight (80), AHEAD (118) KnowNo (64), FOREWARN (83) Feeding verifier feedback into policy learning (§4.3) RECAP (40), VLA-RL (119), Demo-SCORE (10), RoboCat (62), Re-Mix (9), SCIZOR (120) SRPO (68) none 1 2–3 4–6 7–11 12+ Table 3 Verifier-based methods are organized along two complementary dimensions. The application role is evaluating behavior and policies, inference-time verification and scaling, or feeding verifier feedback into policy learning. The model source is task-trained or used directly from pretraining. Fill depth indicates the number of unique papers in each cell. functions with control barrier functions directly, and supply the verification step that PNCBF’s experiments skip. A theorem there extends the guarantee to an휖-optimal approximate value function, and validity and coverage metrics check a learned approximation against that theorem. A symbolic feasibility check before anything moves. Symbolic feasibility checking tests whether the preconditions and affordances (the action possibilities offered by the environment) of a candidate plan can be satisfied (114). Such a check catches plans that sound reasonable but are physically impossible, before anything moves. A line of work called task and motion planning puts the check between a symbolic plan and the motions that realize the plan, and returns the plan to the symbolic layer when no motion satisfies it (115). Text2Motion applies the same test to plans that a language model proposes, scoring a candidate skill sequence by whether the policies for those skills can execute that sequence (86). The coverage of Text2Motion is limited by the symbolic abstraction it uses, and that abstraction is maintained by hand. 4 Learned and Pretrained Verifiers A learned or pretrained verifier returns its score from a forward pass of a neural network. Those networks come from two sources of supervision, training on task-specific robot data and pretraining alone, and Table 3 counts the papers under each source. The two sources call for different evidence that the verifier is right. A verifier trained on task-specific robot data can be compared with real outcomes on the robot the verifier will be used on, and a verifier that comes from pretraining alone has to transfer across embodiments and task distributions before any such comparison exists (§6.1). A learned or pretrained verifier appears in three roles, shown in Figure 4 and set side by side in Table 4. Such a verifier measures the performance of an execution or a policy (§4.1), picks among candidate actions, skills, or plans at inference time (§4.2), and feeds a score back into policy training or into the choice of training data (§4.3). Application subsectionAxisDistinguishing feature Evaluating behavior and policies (§4.1) EvaluationMeasuring an execution at process or trajectory level Inference-time verification and scaling (§4.2) DecisionSelecting the best action, skill, or plan during inference Feeding verifier feedback into policy learning (§4.3) OptimizationUsing scores to optimize the original policy Table 4 The three applications of learned and pretrained verifiers and the distinguishing axes. 12 Verification Evaluation 4.3 Policy Learning Reward Optimization Online Self-improvement Offline Data curation verifier score keep/drop training set (kept data) offline corpus filter/reweightcurated data verifier policy training rollout rollout trajectories update policy 4.1 Evaluating Behavior and Policies score Process-Level Trajectory-Level trajectory process success failure Verification Targets trajectory process policy π observationactiondecision rule 4.2 Inference-time Verification Direct SelectionPredictive Lookahead ACACB B world model 0.80.60.3 Learning policy π policy π Verifier Figure 4 Overview of how learned and pretrained verifiers are used in robot learning. They evaluate behavior and policies at process or trajectory level (§4.1), guide inference-time selection directly or through predictive lookahead (§4.2), and provide feedback for policy optimization and data curation (§4.3). 4.1 Evaluating Behavior and Policies Models that score robot execution can be organized at two levels, set by the granularity of their outputs. A process-level verifier assigns a temporally localized score or categorical judgment to a state, transition, short clip, trajectory prefix, or bounded subtask. A trajectory-level verifier returns one holistic score or verdict for a completed rollout. Policy evaluations and rankings are then obtained by aggregating these outcomes across repeated rollouts. Process-level scores within a rollout. VIP learns an action-free, goal-conditioned value representation from human videos and derives dense, smooth rewards for unseen robotic tasks (31). LIV extends this formulation to goals given as either language or an image, and assigns a dense reward to each frame (32). Robo-Dopamine learns a general-purpose, step-aware process reward model from multi-view inputs (23), while ProcVLM grounds frame-level progress estimates in procedural stages and within-stage visual change (121). Robometer combines frame-level progress supervision with trajectory-comparison preference supervision, and the resulting reward stays available inside a rollout (21). VLAC outputs a dense progress increment together with a done signal from paired observations and a language goal (63). Large reward models scale the same output, training one frame-level reward generator across two dozen data sources and using the frozen generator to refine a policy online in a closed loop (122). PRM-as-a-Judge turns a dense score of this kind into an evaluation protocol, on the argument that one success bit collapses the execution behind that bit (123). PRM-as-a-Judge reads policy execution from trajectory video and reports an outcome, a progress score built on a task-aligned 13 potential, and a diagnosis. Pretrained VLMs can also provide localized feedback without requiring a separately trained robot reward model. Rocamonde et al.(66)establish the construction, using the similarity between a frame and a language description of the goal under a frozen vision-language model as the reward, and RoboCLIP builds the same similarity score from a single demonstration video but assigns that score once, as a sparse reward at the end of the episode, rather than to each frame (124). GVL presents shuffled trajectory frames to a frozen VLM and autoregressively predicts a task-completion percentage for each frame, producing in-context value estimates at frame level (30). TOPReward converts the likelihood of an affirmative completion token for a video prefix into a dense, instruction-conditioned reward and also supports terminal success detection (33). Other methods learn localized categorical judgments. SuccessVQA accepts a single image or short video clip, classifies each clip for success, and only subsequently consolidates the clip predictions into an episode-level decision (39). AHA, after being instruction-tuned on a large synthetic dataset of failure trajectories, predicts a binary success condition for the current subtask from the keyframes observed up to the subtask and generates a failure explanation when the prediction is negative (81), following REFLECT, which summarizes a robot’s sensory record into a hierarchy that a language model then queries to explain a failure and propose a correction (65). Trajectory-level outcome judgments. RoboFAC takes a complete manipulation video and determines whether the task was successfully completed. On a failed video, RoboFAC localizes the failure to a substage and returns corrective guidance, which the authors show can be fed back to a running VLA policy in real time to improve real-robot task success (82). WorldEval uses a video-language verifier to assign one binary success label to each policy-conditioned generated rollout (18). WorldGym passes the generated frame sequence and task instruction to a VLM and returns one rollout-level reward, optionally using a task-specific partial-credit rubric (19). AutoEval executes a policy for a fixed trial horizon and applies a learned success classifier to the terminal state, producing one binary outcome for each unattended hardware trial (116). RobotArena∞translates recorded real scenes into simulation and scores the rollouts that a policy produces there with a vision-language model (125). For generated rollouts, GigaWorld-1 presents a complete synchronized multi-view video to a tuned VLM, which outputs one ordinal World Model as Evaluator Score together with supporting rationales (20). The World Model as Evaluator Score reads video fidelity together with task success (§2.4). 4.2 Inference-Time Verification At inference time a verifier can choose what the robot does next. The system generates several candidate actions, skills, or short-term plans, scores the candidates with the verifier, and executes the highest-scoring candidate. Direct candidate selection. RoboMonkey samples and perturbs policy outputs, scores the candidates with a multimodal verifier, and finds that action error falls as an exponentiated power law in the number of samples (15). RoVer adds an external test-time verifier and improves inference performance with the policy weights left frozen (117). Three further systems spend more computation at test time, on task-progress scoring, on an adaptive critic over relative actions, and on a general framework for scaling embodied inference (126,127,128). SayCan evaluates the candidate skills before execution (61); KnowNo uses conformal calibration to construct a prediction set over candidate actions and requests human assistance when multiple actions remain plausible (64). V-GPS uses a value function learned through offline reinforcement learning to re-rank action chunks generated by a frozen generalist policy (16). Predictive lookahead with world models. These methods use a world model to look ahead from the current state, either scoring candidate actions by their predicted consequence or monitoring the consequence of the chosen action. AHEAD rolls a motion-aware latent world model forward once to predict future scene tokens, not candidate actions, and estimates uncertainty from the variance across five sampled rollouts before feeding the predicted tokens to a frozen action decoder (118). FOREWARN aligns a latent predictive model with a vision–language model, enabling the system to make semantic judgments about predicted outcomes and guide policy selection accordingly (83). Critics trained on a policy’s own successes and failures score the video-model-predicted terminal frames of fine-grained candidate actions (129). Unlike these methods, Foresight does not compare candidates. Foresight reads the latent representations of an action-conditioned 14 world model to monitor the single action block the policy has committed to, detecting potential failures in long-horizon manipulation tasks and conformally calibrating these predictions using trajectory-level labels (80). 4.3 Feeding Verifier Feedback into Policy Learning Verifier feedback can also change the policy itself, by entering the training objective or by deciding which data the policy is trained on. Direct reward optimization. The verifier’s output becomes a reward, an advantage, or a preference objective, and the policy parameters move with that output. RECAP trains a value function using demonstrations, on- policy rollouts, and teleoperated interventions, and then extracts a policy through advantage conditioning. At deployment scale, RECAP more than doubles system throughput and roughly halves the failure rate on the most challenging household tasks (40). GRAPE decomposes tasks into stages, constructs a cost function for each stage, and trains the policy through trajectory-level preference optimization (67). RL-VLM-F trains a reward model from preferences provided by a multimodal model (130), a construction that Motif introduced by distilling a language model’s preferences over event captions into an intrinsic reward (131), whereas VLA-RL fine-tunes a multimodal backbone into a process reward model that scores automatically extracted task segments (119). Some methods derive optimization signals purely from a world model’s latent representations, without decoding a video. SRPO compares states in a pretrained latent world model to construct a progress-shaped reward and can assign credit to useful behavior within failed trajectories (68). World value models put a value head on a world-model backbone instead of a vision-language one, on the argument that a backbone without temporal modeling cannot order states along a task (132). EVA instead generates a full video from the world model and rewards the policy for predictions that an inverse dynamics model can turn back into executable actions (133). VLA-RFT performs reinforcement fine-tuning in a world simulator using verified rewards (134). Although these methods use world-model latent representations, as predictive lookahead does, their final purpose is to update policy parameters rather than to select the next action to execute. Self-improvement and rollout filtering. A verifier decides which newly generated rollouts re-enter training. That decision closes a generate-filter-train loop, and the policy improves on its own experience. Demo-SCORE learns to distinguish successful from failed rollouts and filters a heterogeneous demonstration set before retraining (10). RoboCat fine-tunes on a new task, generates its own interaction experience, and uses a verifier to select the experiences that enter subsequent training (62). Self-improving embodied foundation models derive both a reward function and a success detector from steps-to-go prediction, enabling autonomous practice with improved sample efficiency (135). Human-in-the-loop online rejection sampling combines corrective interventions with a reward-aware data filter on contact-rich tasks (136), while real-time verification scores intermediate results as embodied reasoning is generated and uses the verified reasoning data for subsequent skill learning (78). Data curation and demonstration scoring. A curation method scores, filters, or reweights a corpus recorded before the current policy existed. The property these methods score is the training value of a demonstration, the contribution that demonstration makes to the policy trained on it. The three groups below differ in the evidence they read for that contribution. The first group reads the demonstration by itself. DemInf scores a trajectory with a mutual-information estimate over its observations and actions, a quantity falling when the recorded actions look arbitrary given the robot’s observations (137). ILEED attributes quality to the person who recorded the trajectory and estimates each demonstrator’s expertise (60). DWBC trains a discriminator to separate expert demonstrations from the rest and weights each sample by the discriminator’s output (59), and DemoDICE keeps matching the expert distribution while still drawing on imperfect demonstrations (138). The second group reads the corpus around the demonstration. Re-Mix learns mixture weights over Open X-Embodiment (139), a corpus aggregated across embodiments and protocols. Re-Mix optimizes the worst- performing domain, so the weight a sub-dataset receives does not follow from its size, and the result outperforms 15 Model-Intrinsic Verifiers Input / object Model-derived quantity Usage 5.1 Policy- based self verification Policy information from the policy itself Policy-derived quantities Utilize the value detect failure during execution select actions before execution 5.2 Prediction- based outcome verification World model current state Prediction-based quantities trajectory likelihood latent discrepancy model uncertainty reachability value Verify the outcome plausible implausible 5.3 Task selection Task / environment candidate environments Model-derived value value loss and regret Prioritize training lower priorityhigher priority choose what to roll out or train on predicted futures internal representation action uncertainty train / inference similarity Figure 5 Model-intrinsic verifiers use information from a robot’s own policy or world model to evaluate behavior and guide task selection. Policy-based self-verification detects failures and selects candidates, prediction-based outcome verification judges a candidate by the outcome the world model predicts, and task selection prioritizes the next training environment, all without an independent external verifier. uniform mixtures and hand-specified compositions (9). SCIZOR combines a self-supervised progress estimator that flags suboptimal sub-trajectories with a deduplication module that drops segments repeating what the corpus already holds, and ablations show the progress estimator contributes more of the two (120). The third group reads the policy the data produces. DataMIL fits datamodels that estimate the contribution of one training example to downstream performance, without running a new robot rollout (11). Curation needs this third quantity, and this quantity is also the most expensive to obtain, since it is defined through a training run. All three groups change the corpus. A policy can instead keep the corpus and take the demonstration’s training value as an input, training on the weaker demonstrations with a label that marks them and conditioning at test time on the behavior to follow (40).휋 0.7 does the same with manually annotated quality and mistake labels in place of a trained value (5). Curation and conditioning need the same score, and only curation has to commit to that score before training begins. Two results bound the reach of any such score. The first distinguishes action divergence from transition diversity and finds that state diversity helps only in some settings (140), and the second finds that corpus composition matters more as the corpus grows (141). 5 Model-Intrinsic Verifiers A model-intrinsic verifier reads a quantity that the robot’s own policy or learned world model already computes, and we call that quantity a model-intrinsic score. Policy-based self-verification reads the policy, either to detect a failure while an execution runs or to choose among candidate actions before one is executed (§5.1). Prediction- based outcome verification reads a learned world model and judges a candidate by the outcome that the world model predicts for that candidate (§5.2). The same scores can then rank a whole task instead of a single action, and decide which environment the policy trains on next (§5.3). Figure 5 shows the three subsections. 16 5.1 Policy-Based Self-Verification A robot policy produces more than its final action. The policy’s internal representations, the uncertainty or inconsistency in its action predictions, and the compatibility of the current observation-action trajectory with the training distribution can all provide evidence on whether the policy’s own behavior is reliable. Policy-based self-verification is used at two different stages. Detection evaluates an ongoing execution and raises an alarm when the behavior begins to deviate from patterns associated with normal operation, often before failure becomes externally apparent. Selection, in contrast, evaluates multiple candidate actions or trajectories before execution and chooses the candidate that the policy considers most reliable, without physically trying each alternative. Verifying behavior during execution. The detection methods below differ in the internal information they read and in the way that information becomes a failure prediction. SAFE provides a representative example (26). SAFE trains a lightweight predictor on the internal representations of a vision-language-action policy, motivated by the observation that successful and failed executions exhibit separable structure in the policy’s latent space, with failure-related patterns transferring across tasks. At runtime, the predictor maps the history of these representations to a scalar failure score, which is compared with a time-varying threshold calibrated using functional conformal prediction. When calibration and test rollouts follow the same data distribution, this calibration bounds the probability of falsely flagging a successful rollout; however, the guarantee can fail under cross-task distribution shift, as calibration is performed on seen tasks while evaluation may involve unseen tasks. Other methods also read the policy’s own internal signals to judge whether the current execution is reliable, and these methods differ in the signal they read and in how the verifier is built. Sentinel monitors both action instability and task progress. Temporal inconsistencies in the policy’s action predictions are used to detect erratic behavior, while a separate VLM checks whether the task is still making progress (27). FIPER avoids failure examples by combining out-of-distribution signals from the policy’s observation embeddings with uncertainty in generated action chunks (142), and FAILDetect similarly learns only from successful rollouts, calibrating a time-varying conformal-prediction threshold band on their scores with a statistical guarantee on the false-positive rate (143). Tri-Info instead targets cross-policy transfer, scoring entropy and mutual information over states and actions, and the resulting measure therefore does not depend on a particular embedding architecture (144). HideAndSeek reduces the supervision requirement in another way, using only trajectory-level outcomes together with contrastive learning to identify which steps in a failed rollout carry failure signals (74). Verifying candidates before execution. MG-Select scores candidate action chunks by the change in their token probabilities when state and language inputs are masked, then executes the most input-dependent candidate (17). The base form of MG-Select needs no extra training and no external model, though the strongest results use a finetuned variant, and the method has only been evaluated on autoregressive policies with discrete action tokens. TACO instead selects candidates that are most familiar in the policy’s representation space, using a separately trained pseudo-count estimator while keeping the policy itself frozen (73). The central limitation of policy-based self-verification is that these scores measure whether a behavior looks reliable or familiar to the policy itself, not whether the task succeeded. A familiar-looking failure may therefore still receive a high score. 5.2 Prediction-Based Outcome Verification A learned world model predicts how the environment may evolve. Those predictions can be used to evaluate whether an outcome is plausible, expected, or still achievable. Four model-intrinsic scores support such a judgment. Trajectory likelihood measures how ordinary the trajectory looks to a video model, latent discrepancy compares the predicted latent with the observed latent, model uncertainty reads the disagreement inside an ensemble, and reachability value says whether failure can still be avoided. Trajectory likelihood. VIPER scores a trajectory by its likelihood under a video prediction model (71). An autoregressive transformer is trained on expert video, and the prediction likelihoods become an action-free 17 reward. The resulting agent reaches expert-level control across simulated control suites and manipulation tasks, and also demonstrates transfer across different arm–task combinations. Diffusion Reward instead uses the conditional entropy of a video diffusion model, based on the observation that conditioning on expert trajectories reduces generative diversity (72). A limitation of likelihood-based verification is that high generative likelihood does not necessarily correspond to semantically correct behavior under distribution shift. Latent discrepancy. A latent-discrepancy verifier encodes the observed outcome and compares the resulting latent with the predicted latent. Garrido et al.(145)use this discrepancy between predicted and observed latents to detect physical implausibility zero-shot, without training an additional detector. Garrido et al. (145)find that prediction in latent space performs substantially better than pixel-space prediction and than multimodal language models on this task. Within robotics, Foresight reads the same predicted latents from an action-conditioned world model and trains a failure detector on those latents (§4.2). Model uncertainty. In model-based offline reinforcement learning, an imagined rollout is scored down in the regions where the learned dynamics model is uncertain. MOPO penalizes uncertain predictions from an ensemble of learned dynamics models (41). MOReL instead identifies unknown state–action pairs when an ensemble of dynamics models makes inconsistent predictions (69). LOMPO extends this approach to the latent space of a trained model, allowing model uncertainty to be estimated from image observations (70). AHEAD reads the same quantity inside a manipulation policy, taking the variance across five sampled latent rollouts as a per-step uncertainty and halting the lookahead once the variance rises past a threshold (118). Ward et al. (146)calibrate the latent uncertainty of a compact action-conditioned world model and use that uncertainty to flag failures while the execution runs, and the detector therefore needs no failure examples of its own. Tested on a diffusion policy and on a real-world bimanual cable-manipulation dataset, a 570K-parameter version of the detector beats the next-best learned baseline by 3.8 points of failure-detection rate. Reachability value. A reachability value states whether failure can still be avoided from the current state under the predicted dynamics. Like the candidate scoring of §5.1, a reachability value judges an action before the action is executed. The judgment rests on the consequences the world model predicts rather than on how familiar the action looks to the policy. Nakamura et al.(42)implement this idea with a generative world model. A classifier first identifies observations that correspond to failure, while the world model predicts how different actions change future states. Hamilton– Jacobi reachability (147) then computes a value over these predicted dynamics, identifying states from which no available action can prevent a future failure. The resulting safety filter can therefore reject an unsafe action and switch to a safety-preserving action before the failure occurs. Follow-up work studies the behavior of this type of verification under partial observability, identifying errors from both state estimation and future prediction and proposing more conservative mitigation strategies (148). Three later methods change the quantity the filter is defined over. A parameterized latent constraint lets the unsafe region be set at run time rather than fixed when the filter is trained (75). An uncertainty term added to the value treats a state the model has not seen as a state to avoid (149). A learned control barrier function in place of the reachability value makes the filter intervene smoothly rather than switching (150). Gameplay Filters run the same forward reasoning in a physics simulator, evaluating safety by simulating interactions between a virtual adversary and the robot’s fallback policy (151). 5.3 Beyond Action Verification: Task Selection A model-intrinsic score can also be read about a whole environment instead of a single action, and the score then decides which environment the policy trains on next. ACCEL scores each environment by the positive value loss of the policy running in that environment, which is the amount by which the policy’s own value function underestimates the return the policy went on to receive (152). The environments with the highest score are kept and edited further, and training therefore moves toward the environments the policy still misjudges. The quantity ACCEL’s score approximates is regret, which PAIRED introduced as the difference between the return a second agent achieves in an environment and the return the policy under training achieves (153). Measuring regret as this difference needs two agents run in the same environment, and the value loss instead obtains 18 UseHow candidates ariseRequirement on the verifierCost of error Curationfixed corpus, verifier unseenaverage accuracy, split by error type a false positive trains on failure Ranking and off-policy evaluation non-adversarial samplingaverage accuracy, calibrationthe wrong checkpoint ships Training rewardclosed loop, actively searchedno exploitable regionthe policy satisfies the verifier without doing the task Runtime gatingclosed loop, millisecond budget no exploitable region, plus latencyhardware damage Table 5 The four uses, ordered by whether the score is searched against. The top two select among non-adversarial candidates. The bottom two expose the score to a search for the regions where the score is wrong. Splitting by error type means counting false positives separately from false negatives, since curation incurs a different cost for each. a proxy for regret from the single policy that is already running. Neither method was evaluated on a robot. PAIRED runs on grid-world maze navigation and a MuJoCo hopper domain, and ACCEL adds lava grids and a continuous-control walker. In each case the score decides the next environment to roll out, and says nothing about whether a completed behavior succeeded. Across the three subsections of Section 5, the model-intrinsic score comes from a model the robot is already running rather than from a separate verifier built to check that model. Taking the score from the model itself makes these scores the cheapest verifiers in this survey, and it also means the model gives a high score to a failure the model does not recognize (§6.3). 6 Validating the Verifier A verifier’s own error is worth measuring, because a verifier has four uses and a single error reaches all four. A verifier selects which demonstrations enter training, ranks one policy or one action above another, supplies the reward during post-training, and decides at run time whether an action may execute, for example by switching control to a stronger policy once the verifier flags a coming failure (29). One error therefore reaches the training data, the reported ranking, the reward, and the executed action together, and in each of the four uses that error looks like a correct result. Table 5 lists the same four uses, ordered by whether the score is searched against. Curation and ranking select among candidates produced without access to the score, and an error there averages out over the corpus. A training reward and a runtime gate are applied to candidates produced by a search for the inputs where the score is wrong, and one rare wrong-scoring region is then enough to make the score unusable. Throughout this section a score is the output a verifier returns about a behavior, and a metric is a quantity we compute about the verifier itself. Three metrics of a verifier’s own error appear in the literature. The first is agreement with a fixed reference (§6.1). The second is the policy that results from training on the verifier (§6.2). The third is the verifier’s behavior on candidates constructed to score highly without succeeding (§6.3). We then list the metrics whose reporting makes a verifier checkable by someone else (§6.4). 6.1 Agreement on a Fixed Set of Rollouts An agreement benchmark fixes the rollouts, fixes a reference judgment for each rollout, and reports how often the verifier agrees with the reference. We first name the benchmarks built this way. An agreement rate then leaves out three things, how a disagreement splits between the verifier and the reference, the confidence interval around the rate, and the kind of rollout the rate was measured on. Table 6 closes the subsection with an audit of the resources the field relies on today. The reward benchmark released with TOPReward runs this benchmark design on 130 tasks across four robot platforms, scoring a candidate reward model by its agreement with human judgment (33). Three other benchmarks vary one element of the same benchmark design. The trajectory corpus behind Robometer is far larger, and that corpus was built for training rather than for auditing (21). RoboReward was built to diagnose verifier error, and among these four benchmarks RoboReward reports the lowest agreement at the time of writing (22). OpenGVL takes the true temporal order of the frames as its reference in place of a human label, and tests progress prediction on both robot and human embodiments, so OpenGVL measures whether a verifier still works on a new embodiment (154). The same benchmark design has been applied to a 19 safety judgment, where the reference states whether the behavior is dangerous rather than whether the task succeeded. ASIMOV renders situations taken from real injury reports into text scenarios and videos, then asks a frontier vision-language model to judge those scenarios directly, scoring how well the model recognizes physical danger and decides to intervene (155). A benchmark that reports task success and constraint satisfaction separately yields a number of the same kind (156). Some rollouts pass the success predicate and break a stated safety property, and the fraction of rollouts doing so is the false positive rate of the predicate. At present, the benchmarks that check a safety property separately from the success predicate agree on one finding: policies that complete more tasks are not safer. The first of the three things an agreement rate leaves out is how a disagreement splits between the verifier and the reference. A person is almost always the reference, and the error rate of that person is reported far less often than the agreement rate itself. The opposite case, where the verifier is wrong and the person is right, is also reported rarely. Lee et al.(22)find that frontier multimodal models used as rewards produce costly false positives and false negatives on tasks that a person judges without difficulty, including calling a failed drawer-opening a success. Such false positives and false negatives belong to the verifier being scored rather than to the human reference the verifier is scored against, so a low reference-error rate would still leave a hidden error on the verifier’s side. The second is the confidence interval around the rate. An agreement rate is reported as a single number, and the width of that interval sets how much evidence the rate provides. The rate itself is the fraction of trials the verifier judged correctly, over samples running from a hundred clips to a few thousand trajectories. Four designs on the policy side each supply one thing that a single number leaves out. First, blind randomized trials allocate the rollout budget in advance and report the size of that budget (157). Second, a test on the distribution of time to success reports the whole distribution in place of its mean (158). Third, bounds for a behavior-cloning policy state the task distribution the bounds hold over (159). Fourth, sequential designs let each next trial be chosen by the data collected so far, rather than fixing the schedule beforehand (160,161,162). The same machinery is standard outside robotics, in the interval estimators recommended for reinforcement learning comparisons run on too few seeds (163), and in the tests that stay valid when the person running them chooses the moment to stop (164). The third is the kind of rollout. An agreement rate holds for the rollouts the rate was computed on and not for rollouts of another kind. GE-Sim 2.0 makes the difference visible by scoring one verifier twice, on rollouts that its world model generated and on recorded video of the same tasks (165). A verifier’s error rate therefore depends on the kind of input as much as on the verifier itself, and an accuracy reported without naming the input leaves out the input, which is one of the two things that set the error rate. GE-Sim 2.0 attributes the resulting eight-point accuracy gap to artifacts that the world model itself introduces rather than to the difficulty of the task. The same question arises between simulation and hardware, where SureSim pairs a small number of real trials with simulated ones and shifts the simulated estimate by the average difference (166), and between embodiments, the difference that OpenGVL measures. Sim-to-real evaluation reads the sim-to-real gap directly, as the agreement between a ranking obtained in simulation and a ranking obtained on the robot, a metric that Yang et al.(167)propose but have not yet measured. RoboChallenge runs a shared real-robot fleet that executes a submitted policy and returns its score, and the hardware ranking is therefore available for comparison (168). MolmoSpaces reports the sim-to-real correspondence per task, while RoboLab reports the correspondence in aggregate across task suites tagged by the competency that each task requires (169, 170). Yu et al.(171)argue that a world model should be judged by the decisions taken on top of it rather than by the fidelity of what it generates. The last column of Table 6 applies this standard to every resource in this survey. Two of its rows are world-model evaluations whose verifier error was never measured on generated rollouts. 6.2 Measuring a Verifier by the Policy It Trains A verifier is built to train a better policy or to filter a better corpus. The gain in the resulting policy, rather than agreement with a reference judgment, is the quantity that actually matters. This gain is defined through a training run, and it is therefore expensive to obtain and correspondingly rare in the literature. An agreement rate and the gain in the trained policy diverge, and the mechanism is training itself. A verifier 20 ResourceObject scored Scale reportedVerifierServes Verifier error measured on its deployment distribution Verifier-quality resources ManiRewardBench (33) trajectory130 tasks, 4 platformshumancuration, ranking✓ RBM-1M (21)trajectory>1M trajectorieshuman + autocuration, ranking✓ RoboReward (22)trajectory45k train, 2.8k verified testhumancuration, ranking✓ RoboFAC (82)trajectory9,440 traj., 78,623 QAhumancuration✓ OpenGVL (154)trajectoryrobot + human embodimentstemporal ordercuration✓ Policy-evaluation substrates SIMPLER (172)policy2 robot suitessimulator ruleranking✓ AutoEval (116)policy24 h unattendedlearned classifierranking✓ RoboArena (36)policy600+ pairwise episodeshumanranking✓ PhAIL (158)policy4 VLAs, time-to-success CDFrule + human anchorranking✓ WorldEval (18)policy4 policies rankedfrontier VLMranking× WorldGym (19)policyacross sizes, versionsVLMranking× WMBench (20)world model 7 models, 324k rolloutshuman + VLMranking✓ GE-Sim 2.0 (165)policy6 tasks, 20 real-robot episodes each learned verifierranking, reward✓ Table 6 An audit of the resources used to establish that a verifier works. Serves names the use of the score from Table 5 that the resource supplies evidence for, and the last column states whether the verifier was evaluated on the kind of data the verifier is applied to. can agree with a person on ordinary rollouts and still fail on the rollouts that a training run drives the policy toward, because training moves the policy toward exactly those rollouts. Controlled measurements of the divergence between agreement and downstream gain exist on the language side, and we give one in §6.3. A robot verifier is optimized against in the same way, and the mechanism therefore applies on the robot side, where the corresponding measurement has yet to be run. Two methods in this survey already compute the gain in the trained policy. A datamodel estimates the contribution of one demonstration to downstream performance (11), and a self-improvement loop lets a verifier decide which of a policy’s own rollouts re-enter training and then measures the result on the retrained policy (10,62). Both methods define verifier quality by the performance of the retrained policy. The agreement rate and the downstream gain are reported separately and rarely for the same verifier, and the two numbers diverge most where a verifier is used as a training reward. 6.3 Measuring a Verifier Under Optimization An agreement rate and a gain in the trained policy are both measured on a verifier that was never searched against. A verifier used as a training reward or a runtime gate has to meet one more requirement, that no region of inputs makes the score wrong in a way that a search can reach. Reward hacking is the failure of that requirement. Some inputs do make the score wrong, such as a run that satisfies a final-state predicate exactly and still fails the task (§3.2). A policy optimized against the score is driven toward such wrong-scoring inputs, since the score is highest there. Every measurement in the two subsections above was taken on candidates produced without such a search, and both numbers therefore overstate the value of the verifier to a policy trained on it. Reward hacking is well studied on the language side. A scaling law measures how far a policy can be optimized against a reward model before the true objective starts to fall (173), a token-space attack finds inputs that a top-ranked reward model scores highest and should reject (174), and a survey documents the practice (175). Known fixes include information-theoretic bottlenecks and non-negative reward constructions (176,177). A weak verifier produces reward gains that an independent panel does not confirm, and the gap widens as training continues (178). In robotics the same question, how far a policy can be optimized against a verifier before its score stops tracking the task, is largely undeveloped. Robustness to hacking is reported by whoever built the reward model, as a property of that model. A shared benchmark that holds the candidates fixed and scores the exploitability of each verifier has yet to appear. Li et al.(179)argue that the limit on reinforcement learning in embodied world models is unreliable verification rather than exploration. That work builds an agentic reward framework to resist hacking under distribution shift. One method reports on the embodied side of exploitability, letting a policy, a world model that simulates the policy, and a reward model that scores the result improve each other 21 over several rounds (180). That method runs the co-adaptation described here as the intended procedure rather than as a failure. Optimization has two consequences for a verifier. The reward gain the verifier reports becomes an overestimate, and the calibration the verifier was released with stops holding. Verifiers held out from training. One protocol would measure a robot verifier’s exploitability without waiting for a shared benchmark. Score the rollouts twice, once with the verifier that the policy was trained on and once with a panel of verifiers the policy never saw. A policy that genuinely improved passes both. A policy that only learned to satisfy its training verifier passes the first scoring and fails the panel. The panel on a robot is a set of trajectory verifiers, and the protocol runs at each checkpoint. Take the rollouts that the training verifier passed, hand those rollouts to the panel, and report the fraction the panel rejects. The simulators and the training setups this protocol needs are published (181, 12). A calibration that optimization invalidates. A conformal margin is a fixed number added to a verifier’s score so that the true outcome falls inside the resulting interval a stated fraction of the time. The margin comes from a held-out set (64,26). A held-out set is a batch of rollouts whose outcomes are known and which the verifier did not see during its construction. The margin provides its stated coverage as long as the rollouts judged later come from the same pool as the held-out set. The requirement that later rollouts come from the same pool is exchangeability (§3.1.1). Filtering rollouts by the verifier’s own scores draws the later rollouts from a different pool, and the margin then no longer provides its stated coverage (182, 183). Training a policy against a verifier changes the data that the verifier will later receive (184). The verifier moves the distribution toward the region where the verifier is wrong, and a correction made once therefore stops holding after the next round of training. Conformal failure detection therefore holds for a frozen policy (80) and weakens for a policy trained against the detector (26), and the same exchangeability argument implies SureSim’s bias correction would need to be recomputed once a retrained policy changes the deployment distribution, though the original work evaluates only a single frozen policy (166). Online conformal methods answer this by recalibrating continuously (185), and the guarantee such methods provide is asymptotic rather than finite-sample. 6.4 Metrics for Validating a Verifier We provide nine metrics for a verifier, and reporting the nine makes verifier claims comparable across papers. Guidance of this kind exists for evaluating robot policies empirically (186,157,158), and the verifier benchmarks of §6.1 put a verifier under test. Such a benchmark scores one verifier on one fixed set of rollouts against one fixed reference, and reports a single agreement rate. The nine metrics ask for what the rate leaves out, from the conditions it was measured under to what becomes of the verifier once a policy is optimized against it. Table 7 splits the nine by whether the score is used to select or optimized against. One of the nine is computed from two separate scorings, and we set out the computation below. Proxy gain that transfers. This number measures the fraction of a reward gain that is real. Score the first and the last checkpoint under the training verifier and call the differenceΔ proxy . Score the same two checkpoints under a panel that the policy never trained against and call the differenceΔ panel . Report the ratioΔ panel /Δ proxy together with both differences, since the ratio becomes unstable whenΔ proxy is small. One means the panel confirms the whole gain, zero means the panel confirms none of the gain, and a negative number means the panel scores the policy lower while the training reward rises. The two differences behind the ratio are separately available in simulation. 7 Conclusion A verifier in robot learning decides what a policy is trained on and what the policy’s reported performance means. We surveyed roughly 150 verifiers and grouped them by who supplies the judgment, into human verifiers, rule-based and formal verifiers, learned and pretrained verifiers, and model-intrinsic verifiers. We compared the four families on their availability and their credibility. Across the four families, availability and 22 MetricHow it is computedWhat it settles For a score used to select: curation and ranking Independent rollout countcount per condition, for every success rate giventhe confidence interval can be rebuilt from the count (§6.1) Error rate by typefalse positives and false negatives separately, with the label count behind each a false positive trains on a failure, a false negative discards usable data Rollout typeteleoperated, scripted, or sampled from the policy being judged a verifier scoring generated rollouts needs its error measured on generated rollouts (§6.1) Agreement with human labelsagreement rate with its label count, plus agreement between two annotators how far the verifier sits from the labels, and the labels from each other Calibration score minus observed success rate of the rollouts in that score bin required of any continuous score consumed as a reward Cross-embodiment transfererror on a held-out embodiment minus error on the training embodiments whether the verifier still works on a new robot (154) For a score optimized against: training reward and runtime gating Proxy gain that transfersΔ panel /Δ proxy , with both gains reportedseparates a real gain from a gain that only satisfies the training verifier (178) Verifier error over trainingerror on initial-checkpoint and on final-checkpoint rollouts, labeled alike the increase in verifier error produced by the optimization False positives under searchcandidates found by searching the verifier for inputs the verifier wrongly accepts (187) the region a policy trained on this verifier moves toward Table 7 Nine metrics that make a verifier claim comparable across papers. Most of the upper block asks for numbers the authors have already measured. The lower block requires running a search against the verifier, which robotics has yet to do. credibility move in opposite directions. As the source of judgment moves closer to the model being evaluated, a verification signal becomes cheaper to obtain, arrives earlier, and can be queried more often, while the evidence that the signal reflects actual task performance becomes harder to assemble. The four families occupy different points on the availability-credibility trade-off. Human verifiers give the most direct reference to the intent of the task, and their verdicts are costly to obtain, so human verifiers appear mostly as the small sample that validates the other three families (§2). Rule-based and formal verifiers are inexpensive and repeatable once the required state information is available, and they reach the strongest guarantees in this survey, though only where the predicate, the state estimate, and the dynamics assumptions represent the task (§3). Learned and pretrained verifiers are inexpensive to query and dense across many tasks and trajectories, and the error rate of such a model depends on the data the model was trained and validated on (§4). Model-intrinsic verifiers cost the least, because the policy or the predictive model already computes the model-intrinsic score. The model-intrinsic score describes the model rather than the task, and its relation to actual task performance is therefore indirect (§5). Learned and pretrained verifiers are the largest group in this survey, and model-intrinsic verifiers cost the least to obtain. Those two families are exactly the ones whose credibility is least often established. Establishing credibility is a separate task from building a verifier. We surveyed two ways of establishing credibility. One is a chain of comparisons ending at a set of human labels, and the error of those labels is rarely reported alongside the claim the labels support (§6.1). The other is a benchmark that puts the verifier under test and reports the verifier’s own error rate (§6.1). The demand on a verifier depends on how its score is used. Curation and ranking tolerate an average error rate, and a training reward and a runtime gate demand more, because the candidates they score come from a search for the inputs where the score is wrong. Every agreement rate and every downstream gain that this survey collected was measured on non-adversarial candidates (§6). Availability and credibility are meant as coordinates for the verifiers still to be built. Naming who supplies the judgment fixes how much a verifier costs, how often the verifier can be queried, and how much the verifier’s score can prove. Today, there is no free checker. Reaching high availability and high credibility together is the main future work in verification for robot learning. 23 Use of Large Language Models Parts of the text in this survey were drafted with a large language model. Every claim, citation, and count was checked by the authors against the cited papers. References [1] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Proceedings of the Conference on Robot Learning (CoRL), volume 229 of Proceedings of Machine Learning Research, pages 2165–2183. PMLR, 2023. [2] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An open-source vision-language-action model. In Proceedings of the Conference on Robot Learning (CoRL), volume 270 of Proceedings of Machine Learning Research, pages 2679–2713. PMLR, 2024. [3]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, et al.휋 0 : A vision- language-action flow model for general robot control. In Proceedings of Robotics: Science and Systems (RSS), Los Angeles, CA, USA, June 2025. doi: 10.15607/RSS.2025.XXI.010. [4]Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, et al.휋 0.5 : a Vision-Language-Action Model with Open-World Generalization. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [5]Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, et al.휋 0.7 : a steerable generalist robotic foundation model with emergent capabilities, 2026. [6]NVIDIA, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, et al. Cosmos World Foundation Model Platform for Physical AI. arXiv preprint arXiv:2501.03575, 2025. [7]Google DeepMind. Genie 3: A new frontier for world models. Google DeepMind blog, 2025. URLhttps: //deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/. Accessed 2026- 08-25. [8]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 2025. [9]Joey Hejna, Chethan Bhateja, Yichen Jiang, Karl Pertsch, and Dorsa Sadigh. ReMix: Optimizing Data Mixtures for Large Scale Imitation Learning. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [10]Annie S. Chen, Alec M. Lessing, Yuejiang Liu, and Chelsea Finn. Curating Demonstrations using Online Experience. In Proceedings of Robotics: Science and Systems (RSS), 2025. [11]Shivin Dass, Alaa Khaddaj, Logan Engstrom, Aleksander Madry, Andrew Ilyas, and Roberto Martín-Martín. DataMIL: Selecting Data for Robot Imitation Learning with Datamodels. In International Conference on Learning Representations (ICLR), 2026. [12]Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Zhaohui Yang, Kaiyan Zhang, Xuekai Zhu, Yuchen Zhang, et al. SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning. In International Conference on Learning Represen- tations (ICLR), 2026. [13]Shuhan Tan, Kairan Dou, Yue Zhao, and Philipp Krähenbühl. Interactive Post-Training for Vision-Language-Action Models. arXiv preprint arXiv:2505.17016, 2025. [14] Hongzhi Zang, Mingjie Wei, Si Xu, Yongji Wu, Zhen Guo, Yuanqing Wang, Hao Lin, Peihong Wang, et al. RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models. In Proceedings of Robotics: Science and Systems (RSS), 2026. [15]Jacky Kwok, Christopher Agia, Rohan Sinha, Matt Foutter, Shulu Li, Ion Stoica, Azalia Mirhoseini, Marco Pavone, et al. RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [16]Mitsuhiko Nakamoto, Oier Mees, Aviral Kumar, and Sergey Levine. Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [17]Suhyeok Jang, Dongyoung Kim, Changyeon Kim, Youngsuk Kim, and Jinwoo Shin. Verifier-free Test-Time Sampling for Vision-Language-Action Models. In International Conference on Learning Representations (ICLR), 2026. [18] Yaxuan Li, Yichen Zhu, Junjie Wen, Chaomin Shen, and Yi Xu. WorldEval: World Model as Real-World Robot Policies Evaluator. arXiv preprint arXiv:2505.19017, 2025. [19] Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun, Varad Suryavanshi, Percy Liang, and Sherry Yang. WorldGym: World Model as An Environment for Policy Evaluation. In International Conference on Learning Representations (ICLR), 2026. 24 [20]GigaWorld Team, Angyuan Ma, Boyuan Wang, Bohan Li, Chaojun Ni, Guo Li, Guan Huang, Guosheng Zhao, et al. GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation. arXiv preprint arXiv:2607.02642, 2026. [21] Anthony Liang, Yigit Korkmaz, Jiahui Zhang, Minyoung Hwang, Abrar Anwar, Sidhant Kaushik, Aditya Shah, Alex S. Huang, et al. Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons. In Proceedings of Robotics: Science and Systems (RSS), 2026. [22] Tony Lee, Andrew Wagenmaker, Karl Pertsch, Percy Liang, Sergey Levine, and Chelsea Finn. RoboReward: General- Purpose Vision-Language Reward Models for Robotics. arXiv preprint arXiv:2601.00675, 2026. [23] Huajie Tan, Sixiang Chen, Yijie Xu, Zixiao Wang, Yuheng Ji, Cheng Chi, Yaoxu Lyu, Zhongxia Zhao, et al. Robo- Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. [24]Yixuan Wang, Rhythm Syed, Fangyu Wu, Mengchao Zhang, Aykut Onol, Jose Barreiros, Hooshang Nayyeri, Tony Dear, et al. Interactive World Simulator for Robot Policy Training and Evaluation. In Proceedings of Robotics: Science and Systems (RSS), 2026. [25]Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, He Wang, et al. Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. [26] Qiao Gu, Yuanliang Ju, Shengxiang Sun, Igor Gilitschenski, Haruki Nishimura, Masha Itkina, and Florian Shkurti. SAFE: Multitask Failure Detection for Vision-Language-Action Models. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [27]Christopher Agia, Rohan Sinha, Jingyun Yang, Zi ang Cao, Rika Antonova, Marco Pavone, and Jeannette Bohg. Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [28]Oswin So, Zachary Serlin, Makai Mann, Jake Gonzales, Kwesi Rutledge, Nicholas Roy, and Chuchu Fan. How to Train Your Neural Control Barrier Function: Learning Safety Filters for Complex Input-Constrained Systems. In IEEE International Conference on Robotics and Automation (ICRA), 2024. [29] Josef Chen. AEGIS: A backup reflex for physical AI, 2026. [30]Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, et al. Vision Language Models are In-Context Value Learners. In International Conference on Learning Representations (ICLR), 2025. [31] Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training. In International Conference on Learning Representations (ICLR), 2023. [32]Yecheng Jason Ma, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. LIV: Language-Image Representations and Rewards for Robotic Control. In Proceedings of the International Conference on Machine Learning (ICML), 2023. [33]Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang, Zhongzheng Ren, Lillian J. Ratliff, Jiafei Duan, Dieter Fox, et al. TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics. arXiv preprint arXiv:2602.19313, 2026. [34]Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems (NeurIPS), 2017. [35]Serkan Cabi, Sergio Gómez Colmenarejo, Alexander Novikov, Ksenia Konyushova, Scott Reed, Rae Jeong, et al. Scaling data-driven robotics with reward sketching and batch reinforcement learning. In Proceedings of Robotics: Science and Systems (RSS), 2020. [36]Pranav Atreya, Karl Pertsch, Tony Lee, Moo Jin Kim, Arhan Jain, Artur Kuramshin, Cyrus Neary, et al. RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [37]Chengyue Huang, Khang Vo Huynh, Sebastian Elbaum, Zsolt Kira, and Lu Feng. SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic Manipulation. arXiv preprint arXiv:2605.12386, 2026. [38]Yue Meng, Fei Chen, and Chuchu Fan. TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks. arXiv preprint arXiv:2510.00225, 2025. [39]Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, Serkan Cabi, et al. Vision-Language Models as Success Detectors. In Proceedings of The 2nd Conference on Lifelong Learning Agents, 2023. [40]Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, et al.휋 ∗ 0.6 : a VLA that learns from experience. arXiv preprint arXiv:2511.14759, 2025. [41]Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Zou, Sergey Levine, et al. MOPO: Model-based Offline Policy Optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2020. [42] Kensuke Nakamura, Lasse Peters, and Andrea Bajcsy. Generalizing Safety Beyond Collision-Avoidance via Latent- Space Reachability Analysis. In Proceedings of Robotics: Science and Systems (RSS), 2025. 25 [43]Kimin Lee, Laura Smith, and Pieter Abbeel. PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training. In Proceedings of the International Conference on Machine Learning (ICML), 2021. [44] Kimin Lee, Laura Smith, Anca Dragan, and Pieter Abbeel. B-Pref: Benchmarking Preference-Based Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2021. [45]Quanyi Li, Zhenghao Peng, and Bolei Zhou. Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization. In International Conference on Learning Representations (ICLR), 2022. URLhttps://openreview. net/forum?id=0cgU-BZp2ky. [46] Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot Learning on the Job: Human-in-the- Loop Autonomy and Learning During Deployment. In Proceedings of Robotics: Science and Systems (RSS), 2023. doi: 10.15607/RSS.2023.XIX.005. URL https://roboticsproceedings.org/rss19/p005.html. [47]Huihan Liu, Yu Zhang, Vaarij Betala, Evan Zhang, James Liu, Crystal Ding, and Yuke Zhu. Multi-task interactive robot fleet learning with visual world models. In Proceedings of the Conference on Robot Learning (CoRL), volume 270 of Proceedings of Machine Learning Research, pages 4286–4313. PMLR, 2024. [48]Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and Dexterous Robotic Manipulation via Human-in- the-Loop Reinforcement Learning. Science Robotics, 2025. [49]Gemini Robotics Team, Krzysztof Choromanski, Coline Devin, Yilun Du, Debidatta Dwibedi, Ruiqi Gao, et al. Evaluating Gemini Robotics Policies in a Veo World Simulator. arXiv preprint arXiv:2512.10675, 2025. [50]Kim P. Wabersich and Melanie N. Zeilinger. A predictive safety filter for learning-based control of constrained nonlinear dynamical systems. Automatica, 129:109597, 2021. doi: 10.1016/j.automatica.2021.109597. [51]Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wolfram Burgard. CALVIN: A Benchmark for Language- Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks. IEEE International Conference on Robotics and Automation (ICRA), 2022. [52]Lars Lindemann, Xin Qin, Jyotirmoy V. Deshmukh, and George J. Pappas. Conformal Prediction for STL Runtime Verification. In Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (ICCPS), 2023. [53]Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023. [54]Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, Dieter Fox, et al. MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations. In Proceedings of the Conference on Robot Learning (CoRL), 2023. [55]Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, et al. Eureka: Human-Level Reward Design via Coding Large Language Models. In International Conference on Learning Representations (ICLR), 2024. [56]Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation. In Proceedings of the Conference on Robot Learning (CoRL), 2024. [57]Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, and Dongbin Zhao. ConRFT: A Reinforced Fine- tuning Method for VLA Models via Consistency Policy. In Proceedings of Robotics: Science and Systems (RSS), 2025. [58] Soham Kulkarni, Raayan Dhar, and Yuchen Cui. Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learning. arXiv preprint arXiv:2604.23000, 2026. [59]Haoran Xu, Xianyuan Zhan, Honglei Yin, and Huiling Qin. Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations. In Proceedings of the International Conference on Machine Learning (ICML), 2022. [60]Mark Beliaev, Andy Shih, Stefano Ermon, Dorsa Sadigh, and Ramtin Pedarsani. Imitation Learning by Estimating Expertise of Demonstrators. In Proceedings of the International Conference on Machine Learning (ICML), 2022. [61]Brian Ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, et al. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. In Proceedings of the Conference on Robot Learning (CoRL), 2023. [62]Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X. Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, et al. RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation. Transactions on Machine Learning Research, 2024. [63] Shaopeng Zhai, Qi Zhang, Tianyi Zhang, Fuxian Huang, Haoran Zhang, Ming Zhou, Shengzhe Zhang, Litao Liu, et al. A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning. arXiv preprint arXiv:2509.15937, 2025. [64]Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, et al. Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners. In Proceedings of the Conference on Robot Learning (CoRL), 2023. 26 [65]Zeyi Liu, Arpit Bahety, and Shuran Song. REFLECT: Summarizing robot experiences for failure explanation and correction. In Proceedings of the Conference on Robot Learning (CoRL), volume 229 of Proceedings of Machine Learning Research, pages 3468–3484. PMLR, 2023. [66] Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision-language models are zero-shot reward models for reinforcement learning. In International Conference on Learning Representations (ICLR), 2024. [67] Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, et al. GRAPE: Generalizing Robot Policy via Preference Alignment. In International Conference on Learning Representations (ICLR), 2025. [68]Senyu Fei, Siyin Wang, Li Ji, Ao Li, Shiduo Zhang, Liming Liu, Jinlong Hou, Jingjing Gong, et al. SRPO: Self- Referential Policy Optimization for Vision-Language-Action Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. [69] Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, and Thorsten Joachims. MOReL: Model-Based Offline Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), 2020. [70]Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, and Chelsea Finn. Offline Reinforcement Learning from Images with Latent Space Models. In Proceedings of the 3rd Conference on Learning for Dynamics and Control (L4DC), 2021. [71]Alejandro Escontrela, Ademi Adeniji, Wilson Yan, Ajay Jain, Xue Bin Peng, Ken Goldberg, et al. Video Prediction Models as Rewards for Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023. [72]Tao Huang, Guangqi Jiang, Yanjie Ze, and Huazhe Xu. Diffusion Reward: Learning Rewards via Conditional Video Diffusion. In Proceedings of the European Conference on Computer Vision (ECCV), 2024. [73]Siyuan Yang, Yang Zhang, Haoran He, Ling Pan, Xiu Li, Chenjia Bai, and Xuelong Li. Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach. arXiv preprint arXiv:2512.02834, 2025. [74]Seongheon Park, Wendi Li, Changdae Oh, Samuel Yeh, Zsolt Kira, Michael Hagenow, and Sharon Li. Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring. arXiv preprint arXiv:2605.30834, 2026. [75]Sankalp Agrawal, Junwon Seo, Kensuke Nakamura, Ran Tian, and Andrea Bajcsy. AnySafe: Adapting latent safety filters at runtime via safety constraint parameterization in the latent space. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [76]Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S. Brown, and Ken Goldberg. ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning. In Proceedings of the Conference on Robot Learning (CoRL), 2022. [77]Michael Kelly, Chelsea Sidrane, Katherine Rose Driggs-Campbell, and Mykel J. Kochenderfer. HG-DAgger: Interactive Imitation Learning with Human Experts. In IEEE International Conference on Robotics and Automation (ICRA), pages 8077–8083. IEEE, 2019. doi: 10.1109/ICRA.2019.8793698. [78]Bo Yue, Shuqi Guo, Kaiyu Hu, Chujiao Wang, Benyou Wang, Kui Jia, and Guiliang Liu. Real-Time Verification of Embodied Reasoning for Generative Skill Acquisition. arXiv preprint arXiv:2505.11175, 2025. [79]Runhao Zeng, Dingjie Zhou, Qiwei Liang, Junlin Liu, Hui Li, Changxin Huang, Jianqiang Li, Xiping Hu, et al. Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning. In Proceedings of the 27th European Conference on Artificial Intelligence (ECAI 2024), 2024. [80]Haoran Zhang, Yifu Lu, Boyang Wang, Xuhui Kang, Yen-Ling Kuo, Zezhou Cheng, Mengdi Wang, Odest Chadwicke Jenkins, et al. Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents. arXiv preprint arXiv:2606.23085, 2026. [81]Jiafei Duan, Wilbert Pumacay, Nishanth Kumar, Yi Ru Wang, Shulin Tian, Wentao Yuan, Ranjay Krishna, Dieter Fox, et al. AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation. In International Conference on Learning Representations (ICLR), 2025. [82]Zewei Ye, Weifeng Lu, Minghao Ye, Tao Lin, Shuo Yang, Junchi Yan, and Bo Zhao. RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correction. arXiv preprint arXiv:2505.12224, 2025. [83]Yilin Wu, Thomas Tian, Gokul Swamy, and Andrea Bajcsy. From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment. In Proceedings of Robotics: Science and Systems (RSS), 2025. [84]Stéphane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 15 of Proceedings of Machine Learning Research, pages 627–635. PMLR, 2011. [85] Haoyuan Cai, Zhenghao Peng, and Bolei Zhou. Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism. In Proceedings of the International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pages 6243–6256. PMLR, 2025. URLhttps://proceedings.mlr.press/v267/ cai25e.html. [86]Kevin Lin, Christopher Agia, Toki Migimatsu, Marco Pavone, and Jeannette Bohg. Text2Motion: From Natural Language Instructions to Feasible Plans. Autonomous Robots, 2023. 27 [87]Yiqi Zhao, Emily Zhu, Bardh Hoxha, Georgios Fainekos, Jyotirmoy V. Deshmukh, Lars Lindemann, et al. Distribu- tionally Robust Predictive Runtime Verification under Spatio-Temporal Logic Specifications. ACM Transactions on Cyber-Physical Systems, 2025. [88] Yiqi Zhao, Bardh Hoxha, Georgios Fainekos, Jyotirmoy V. Deshmukh, and Lars Lindemann. Robust Conformal Prediction for STL Runtime Verification under Distribution Shift. In 2024 ACM/IEEE 15th International Conference on Cyber-Physical Systems (ICCPS), 2024. [89] Devesh Nath, Anutam Srinivasan, Haoran Yin, Ruitong Jiang, Jeffrey Fang, Glen Chou, et al. Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC. arXiv preprint arXiv:2606.15594, 2026. [90]Ameesh Shah, Cameron Voloshin, Chenxi Yang, Abhinav Verma, Swarat Chaudhuri, and Sanjit A. Seshia. LTL- Constrained Policy Optimization with Cycle Experience Replay. Transactions on Machine Learning Research, 2025. [91]Yecheng Jason Ma, William Liang, Hung-Ju Wang, Yuke Zhu, Linxi Fan, Osbert Bastani, and Dinesh Jayaraman. DrEureka: Language Model Guided Sim-To-Real Transfer. In Proceedings of Robotics: Science and Systems (RSS), 2024. [92]Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu, et al. Text2Reward: Reward Shaping with Language Models for Reinforcement Learning. In International Conference on Learning Representations (ICLR), 2024. [93] Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, et al. Language to Rewards for Robotic Skill Synthesis. In Proceedings of the Conference on Robot Learning (CoRL), 2023. [94]Shiduo Zhang, Zhe Xu, Peiju Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, et al. VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025. [95]Ning Gao, Yilun Chen, Shuai Yang, Xinyi Chen, Yang Tian, Hao Li, Haifeng Huang, Hanqing Wang, et al. GENMANIP: LLM-driven Simulation for Generalizable Instruction-Following Manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. [96]Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, Yuke Zhu, et al. RoboCasa: Large-Scale Simulation of Household Tasks for Generalist Robots. In Proceedings of Robotics: Science and Systems (RSS), 2024. [97]Haoran Geng, Feishi Wang, Songlin Wei, Yuyang Li, Bangjun Wang, Boshi An, Haozhe Lou, Charlie Tianyue Cheng, et al. RoboVerse: A Unified Platform, Benchmark and Dataset for Scalable and Generalizable Robot Learning. In Proceedings of Robotics: Science and Systems (RSS), 2025. [98]Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, et al. ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI. arXiv preprint arXiv:2410.00425, 2024. [99]Mayank Mittal, Yunrong Guo, Pascal Roth, David Hoeller, James Tigue, Antoine Richard, Octi Zhang, Peter Du, et al. Isaac lab: A GPU-accelerated simulation framework for multi-modal robot learning, 2025. Includes Isaac Lab-Arena, an open framework for large-scale policy evaluation. [100]Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation. In Proceedings of the International Conference on Machine Learning (ICML), 2026. [101]Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, et al. BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation. arXiv preprint arXiv:2403.09227, 2024. [102]Wilbert Pumacay, Ishika Singh, Jiafei Duan, Ranjay Krishna, Jesse Thomason, and Dieter Fox. THE COLOSSEUM: A benchmark for evaluating generalization for robotic manipulation. In Proceedings of Robotics: Science and Systems (RSS), Delft, Netherlands, July 2024. [103]Zhenyu Jiang, Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, Yuke Zhu, et al. DexMimic- Gen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning. In IEEE International Conference on Robotics and Automation (ICRA), 2025. [104]Zhengrong Xue, Shuying Deng, Zhenyang Chen, Yixuan Wang, Zhecheng Yuan, and Huazhe Xu. DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning. In Proceedings of Robotics: Science and Systems (RSS), 2025. [105]Lirui Wang, Yiyang Ling, Zhecheng Yuan, Mohit Shridhar, Chen Bao, Yuzhe Qin, Bailin Wang, Huazhe Xu, et al. GenSim: Generating Robotic Simulation Tasks via Large Language Models. In International Conference on Learning Representations (ICLR), 2024. [106]Pu Hua, Minghuan Liu, Annabella Macaluso, Yunfeng Lin, Weinan Zhang, Huazhe Xu, and Lirui Wang. GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs. In Proceedings of the Conference on Robot Learning (CoRL), 2025. 28 [107]Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, et al. RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation. In Proceedings of the International Conference on Machine Learning (ICML), 2024. [108] Lukas Helff, Quentin Delfosse, David Steinmann, Ruben Härle, Hikaru Shindo, Patrick Schramowski, et al. LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking. arXiv preprint arXiv:2604.15149, 2026. [109]Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and Characterizing Reward Gaming. In Advances in Neural Information Processing Systems (NeurIPS), 2022. [110]David Manheim and Scott Garrabrant. Categorizing Variants of Goodhart’s Law. arXiv preprint arXiv:1803.04585, 2018. [111]Aaron D. Ames, Samuel Coogan, Magnus Egerstedt, Gennaro Notomista, Koushil Sreenath, and Paulo Tabuada. Control barrier functions: Theory and applications. In 2019 18th European Control Conference (ECC), pages 3420–3431, Naples, Italy, 2019. doi: 10.23919/ECC.2019.8796030. [112] Kai-Chieh Hsu, Haimin Hu, and Jaime F. Fisac. The safety filter: A unified view of safety-critical control in autonomous systems. Annual Review of Control, Robotics, and Autonomous Systems, 7(1):47–72, 2024. doi: 10.1146/ annurev-control-071723-102940. [113] Daniel C. H. Tan, Fernando Acero, Robert McCarthy, Dimitrios Kanoulas, and Zhibin Li. Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory. arXiv preprint arXiv:2306.04026, 2023. [114] Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated Task and Motion Planning. Annual Review of Control, Robotics, and Autonomous Systems, 4: 265–293, 2021. [115]Siddharth Srivastava, Eugene Fang, Lorenzo Riano, Rohan Chitnis, Stuart Russell, and Pieter Abbeel. Combined Task and Motion Planning through an Extensible Planner-Independent Interface Layer. In IEEE International Conference on Robotics and Automation (ICRA), 2014. [116]Zhiyuan Zhou, Pranav Atreya, You Liang Tan, Karl Pertsch, and Sergey Levine. AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [117]Mingtong Dai, Lingbo Liu, Yongjie Bai, Yang Liu, Zhouxia Wang, Rui SU, Chunjie Chen, Liang Lin, et al. RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Model. arXiv preprint arXiv:2510.10975, 2025. [118]Shahram Najam Syed, Arthur Jakobsson, Haoran Hao, and Jeffrey Ichnowski. Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation. arXiv preprint arXiv:2606.02486, 2026. [119]Guanxing Lu, Wenkai Guo, Chubin Zhang, Yuheng Zhou, Haonan Jiang, Zifeng Gao, Yansong Tang, Ziwei Wang, et al. VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning. arXiv preprint arXiv:2505.18719, 2025. [120]Yu Zhang, Yuqi Xie, Huihan Liu, Rutav Shah, Michael Wan, Linxi Fan, and Yuke Zhu. SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [121]Youhe Feng, Hansen Shi, Haoyang Li, Xinlei Guo, Yang Wang, Chengyang Zhang, Jinkai Zhang, Xiaohan Zhang, et al. ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulation. arXiv preprint arXiv:2605.08774, 2026. [122] Yanru Wu, Weiduo Yuan, Ang Qi, Vitor Guizilini, Jiageng Mao, and Yue Wang. Large reward models: Generalizable online robot reward generation with vision-language models, 2026. [123]Yuheng Ji, Yuyang Liu, Huajie Tan, Xuchuan Huang, Fanding Huang, Yijie Xu, Cheng Chi, Yuting Zhao, Huaihai Lyu, Peterson Co, Mingyu Cao, Qiongyu Zhang, Zhe Li, Enshen Zhou, Pengwei Wang, Zhongyuan Wang, Shanghang Zhang, and Xiaolong Zheng. PRM-as-a-Judge: A dense evaluation paradigm for fine-grained robotic auditing, 2026. [124]Sumedh A. Sontakke, Jesse Zhang, Sébastien M. R. Arnold, Karl Pertsch, Erdem Biıyik, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. RoboCLIP: One demonstration is enough to learn robot policies. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. [125]Yash Jangir, Yidi Zhang, Pang-Chi Lo, Kashu Yamazaki, Chenyu Zhang, Kuan-Hsun Tu, Tsung-Wei Ke, Lei Ke, Yonatan Bisk, and Katerina Fragkiadaki. RobotArena∞: Scalable robot benchmarking via real-to-sim translation. In Proceedings of the International Conference on Learning Representations (ICLR), 2026. [126]Sizhe Zhao, Shengping Zhang, Shuo Yang, Weiyu Zhao, Shuigen Wang, and Xiangyang Ji. TapSampling: Inference- Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation. In Proceedings of the Interna- tional Conference on Machine Learning (ICML), 2026. [127]Wenhao Li, Xiu Su, Yichao Cao, Hongyan Xu, Xiaobo Xia, Shan You, Yi Chen, Chang Xu, et al. VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model. arXiv preprint arXiv:2605.01194, 2026. [128]Wen Ye, Peiyan Li, Tingyu Yuan, Yuan Xu, Xiangnan Wu, Chaoyang Zhao, Jing Liu, Nianfeng Liu, et al. E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation. arXiv preprint arXiv:2606.27268, 2026. [129] Sruthi Sudhakar, Junbang Liang, Sreehari Rammohan, Pavel Tokmakov, Richard Zemel, and Carl Vondrick. Robot Critics that Sweat the Small Stuff. arXiv preprint arXiv:2606.21572, 2026. 29 [130]Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback. In Proceedings of the International Conference on Machine Learning (ICML), 2024. [131] Martin Klissarov, Pierluca D’Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, and Mikael Henaff. Motif: Intrinsic motivation from artificial intelligence feedback. In International Conference on Learning Representations (ICLR), 2024. [132] Zhihao Wang, Jianxiong Li, Yu Cui, Yuan Gao, Xianyuan Zhan, Junzhi Yu, and Xiao Ma. World value models for robotic manipulation, 2026. [133] Ruixiang Wang, Qingming Liu, Yueci Deng, Guiliang Liu, Zhen Liu, and Kui Jia. EVA: Aligning video world models with executable robot actions via inverse dynamics rewards, 2026. [134] Hengtao Li, Pengxiang Ding, Runze Suo, Yihao Wang, Zirui Ge, Dongyuan Zang, Kexian Yu, Mingyang Sun, et al. VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators. arXiv preprint arXiv:2510.00406, 2025. [135]Seyed Kamyar Seyed Ghasemipour, Ayzaan Wahid, Jonathan Tompson, Pannag Sanketi, and Igor Mordatch. Self- Improving Embodied Foundation Models. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [136] Guanxing Lu, Rui Zhao, Haitao Lin, He Zhang, and Yansong Tang. Human-in-the-loop Online Rejection Sampling for Robotic Manipulation. arXiv preprint arXiv:2510.26406, 2025. [137] Joey Hejna, Suvir Mirchandani, Ashwin Balakrishna, Annie Xie, Ayzaan Wahid, Jonathan Tompson, Pannag Sanketi, Dhruv Shah, et al. Robot Data Curation with Mutual Information Estimators. In Proceedings of Robotics: Science and Systems (RSS), 2025. [138]Geon-Hyeong Kim, Seokin Seo, Jongmin Lee, Wonseok Jeon, HyeongJoo Hwang, Hongseok Yang, and Kee-Eung Kim. DemoDICE: Offline imitation learning with supplementary imperfect demonstrations. In International Conference on Learning Representations (ICLR), 2022. URL https://openreview.net/forum?id=BrPdX1bDZkQ. [139]Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, et al. Open X-Embodiment: Robotic Learning Datasets and RT-X Models. In IEEE International Conference on Robotics and Automation (ICRA), 2024. [140]Suneel Belkhale, Yuchen Cui, and Dorsa Sadigh. Data Quality in Imitation Learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023. [141]Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data Scaling Laws in Imitation Learning for Robotic Manipulation. In International Conference on Learning Representations (ICLR), 2025. [142] Ralf Römer, Adrian Kobras, Luca Worbis, and Angela P. Schoellig. Failure Prediction at Runtime for Generative Robot Policies. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [143]Chen Xu, Tony Khuong Nguyen, Emma Dixon, Christopher Rodriguez, Patrick Miller, Robert Lee, et al. Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies. In Proceedings of Robotics: Science and Systems (RSS), 2025. [144]Jinghan Yang, Yunchao Zhang, Wang Yuan, Haolun Wang, Jiaming Zhang, Zhengyang Hu, and Yanchao Yang. Tri-Info: Generalizable, Interpretable Failure Prediction for VLA Models via Information Theory. arXiv preprint arXiv:2606.19998, 2026. [145]Quentin Garrido, Nicolas Ballas, Mahmoud Assran, Adrien Bardes, Laurent Najman, Michael Rabbat, et al. Intuitive physics understanding emerges from self-supervised pretraining on natural videos. arXiv preprint arXiv:2502.11831, 2025. [146]Isaac R. Ward, Michelle Ho, Houjun Liu, Aaron Feldman, Joseph Vincent, Liam Kruse, Sean Cheong, Duncan Eddy, Mykel J. Kochenderfer, and Mac Schwager. Foundational world models accurately detect bimanual manipulator failures. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [147]Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J. Tomlin. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 2242–2253, 2017. doi: 10.1109/CDC.2017.8263977. [148]Matthew Kim, Kensuke Nakamura, and Andrea Bajcsy. How Well Do Latent World Models Understand Partially Observable Safety Constraints? arXiv preprint arXiv:2510.06492, 2025. [149]Junwon Seo, Kensuke Nakamura, and Andrea Bajcsy. Uncertainty-aware latent safety filters for avoiding out-of- distribution failures. In Conference on Robot Learning (CoRL), 2025. [150]Kensuke Nakamura, Arun L. Bishop, Steven Man, Aaron M. Johnson, Zachary Manchester, and Andrea Bajcsy. How to train your latent control barrier function: Smooth safety filtering under hard-to-model constraints. In Learning for Dynamics and Control Conference (L4DC), 2026. [151]Duy P. Nguyen, Kai-Chieh Hsu, Wenhao Yu, Jie Tan, and Jaime F. Fisac. Gameplay Filters: Robust Zero-Shot Safety through Adversarial Imagination. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [152]Jack Parker-Holder, Minqi Jiang, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, et al. Evolving Curricula with Regret-Based Environment Design. In Proceedings of the International Conference on Machine Learning (ICML), 2022. 30 [153]Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, et al. Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design. In Advances in Neural Information Processing Systems (NeurIPS), 2020. [154] Paweł Budzianowski, Emilia Wiśnios, Michał Tyrolski, Gracjan Góral, Igor Kulakov, Viktor Petrenko, et al. OpenGVL – Benchmarking Visual Temporal Progress for Data Curation. arXiv preprint arXiv:2509.17321, 2025. [155]Abhishek Jindal, Dmitry Kalashnikov, R. Alex Hofer, Oscar Chang, Divya Garikapati, Anirudha Majumdar, et al. Can AI Perceive Physical Danger and Intervene? arXiv preprint arXiv:2509.21651, 2025. [156]Jialiang Fan, Weizhe Xu, Oleg Sokolsky, Insup Lee, and Fanxin Kong. SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models. arXiv preprint arXiv:2606.00773, 2026. [157]Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, et al. A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation. Science Robotics, 2026. [158]Sergey Arkhangelskiy. PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology. arXiv preprint arXiv:2605.29710, 2026. [159]Joseph A. Vincent, Haruki Nishimura, Masha Itkina, Paarth Shah, Mac Schwager, and Thomas Kollar. How generalizable is my behavior cloning policy? a statistical approach to trustworthy performance evaluation. IEEE Robotics and Automation Letters, 9(10):8619–8626, 2024. [160]David Snyder, Asher James Hancock, Apurva Badithela, Emma Dixon, Patrick Miller, Rares Andrei Ambrus, Anirudha Majumdar, Masha Itkina, et al. Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stopping. In Proceedings of Robotics: Science and Systems (RSS), 2025. [161]David Snyder, Apurva Badithela, Nikolai Matni, George Pappas, Anirudha Majumdar, Masha Itkina, and Haruki Nishimura. Beyond Binary Success: Sample-Efficient and Statistically Rigorous Robot Policy Comparison. In Proceedings of Robotics: Science and Systems (RSS), 2026. [162]Abrar Anwar, Rohan Gupta, Zain Merchant, Sayan Ghosh, Willie Neiswanger, and Jesse Thomason. Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selection. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [163]Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, and Marc G. Bellemare. Deep reinforcement learning at the edge of the statistical precipice. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 29304–29320. Curran Associates, Inc., 2021. URLhttps://proceedings.neurips.c/ paper_files/paper/2021/file/f514cec81cb148559cf475e7426eed5e-Paper.pdf. [164]Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):576–601, 2023. doi: 10.1214/23-STS894. [165]Boxiang Qiu, Liliang Chen, Yue Liao, Nan Wang, Lintao Wang, Jiayi Luo, et al. GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation. arXiv preprint arXiv:2605.27491, 2026. [166]Apurva Badithela, David Snyder, Lihan Zha, Joseph Mikhail, Matthew O’Kelly, Anushri Dixit, and Anirudha Majumdar. Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [167]Xuning Yang, Clemens Eppner, Jonathan Tremblay, Dieter Fox, Stan Birchfield, and Fabio Ramos. Robot policy evaluation for sim-to-real transfer: A benchmarking perspective, 2025. RSS 2025 Workshop on Robot Evaluation for the Real World. [168] Adina Yakefu, Bin Xie, Chongyang Xu, Enwen Zhang, Erjin Zhou, Fan Jia, et al. RoboChallenge: Large-scale real-robot evaluation of embodied policies, 2025. [169]Yejin Kim, Wilbert Pumacay, Omar Rayyan, Max Argus, Winson Han, Eli VanderBilt, et al. MolmoSpaces: A large-scale open ecosystem for robot navigation and manipulation, 2026. [170]Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield, Ankit Goyal, Stan Birchfield, Fabio Ramos, and Jonathan Tremblay. RoboLab: A high-fidelity simulation benchmark for analysis of task generalist policies. In Robotics: Science and Systems (RSS), Sydney, Australia, 2026. [171]Yang Yu, Shiyuan Zhang, Yifei Sheng, Haoxiang Ren, and Haoxin Lin. How should world models be evaluated for embodied decision-making? a decision-making-centric position, 2026. [172]Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, et al. Evaluating Real-World Robot Manipulation Policies in Simulation. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [173] Leo Gao, John Schulman, and Jacob Hilton. Scaling Laws for Reward Model Overoptimization. In Proceedings of the International Conference on Machine Learning (ICML), 2023. [174]Yuheng Zhang, Mingyue Huo, Minghao Zhu, Mengxue Zhang, and Nan Jiang. Beyond Semantic Manipulation: Token-Space Attacks on Reward Models. arXiv preprint arXiv:2604.02686, 2026. [175]Pei-Chi Pan, Yingbin Liang, and Sen Lin. Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation. Transactions on Machine Learning Research, 2026. 31 [176]Yuchun Miao, Sen Zhang, Liang Ding, Rong Bao, Lefei Zhang, and Dacheng Tao. InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling. In Advances in Neural Information Processing Systems (NeurIPS), 2024. [177] Zhibin Duan, Guowei Rong, Zhuo Li, Bo Chen, Mingyuan Zhou, and Dandan Guo. Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling. In Proceedings of the International Conference on Machine Learning (ICML), 2026. [178] Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang, Anisha Gunjal, Bing Liu, Yunzhong He, et al. Reward Hacking in Rubric-Based Reinforcement Learning. arXiv preprint arXiv:2605.12474, 2026. [179] Pu Li, Zhigang Lin, Qiang Wu, Yongxuan Lv, Fei Wang, Shan You, et al. Reward as An Agent for Embodied World Models. arXiv preprint arXiv:2606.19990, 2026. [180] Yanjiang Guo, Tony Lee, Lucy Xiaoyang Shi, Jianyu Chen, Percy Liang, and Chelsea Finn. VLAW: Iterative co- improvement of vision-language-action policy and world model. In International Conference on Machine Learning (ICML), 2026. [181]Yanan Wang, Renxi Wang, Yongxin Wang, Xuezhi Liang, Fajri Koto, Timothy Baldwin, Xiaodan Liang, Haonan Li, et al. SimuScene: Training and Benchmarking Code Generation to Simulate Physical Scenarios. arXiv preprint arXiv:2602.10840, 2026. [182]Ryan J. Tibshirani, Rina Foygel Barber, Emmanuel J. Candes, and Aaditya Ramdas. Conformal Prediction Under Covariate Shift. In Advances in Neural Information Processing Systems (NeurIPS), 2019. [183]Rina Foygel Barber, Emmanuel J. Candes, Aaditya Ramdas, and Ryan J. Tibshirani. Conformal prediction beyond exchangeability. The Annals of Statistics, 2023. [184]Juan C. Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. Performative Prediction. In Proceedings of the International Conference on Machine Learning (ICML), 2020. [185]Isaac Gibbs and Emmanuel Candès. Adaptive Conformal Inference Under Distribution Shift. In Advances in Neural Information Processing Systems (NeurIPS), 2021. [186]Hadas Kress-Gazit, Kunimatsu Hashimoto, Naveen Kuppuswamy, Paarth Shah, Phoebe Horgan, Gordon Richardson, et al. Robot Learning as an Empirical Science: Best Practices for Policy Evaluation. arXiv preprint arXiv:2409.09491, 2024. [187] Jaideep Ray. Before the model learns the bug: Fuzzing RLVR verifiers, 2026. 32

Instant research discovery