Paper deep dive
RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes
Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang, Yehang Zhang, Zexi Li, Wenqian Li, Qize Yu, Zetian Song, Leyi Wu, Jinghao Li, Mingxuan Song, Xinrun Xu, Zongyang Qiu, Yangkai Wei, Tianyi Zhang, Kaiwen Zhou, Yinchuan Li, James Cheng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/10/2026, 1:44:51 AM
Summary
The paper introduces RoboAware, a framework for embodied coding agents that coordinates modular and end-to-end policy families using a state-conditioned responsibility coordinator. It proposes the P5 skill schema to structure skills into five semantic stages, State-Locked Counterfactual Branching (SCB) to generate counterfactual outcomes by restoring simulator states, and Execution-Aware Learning (EAL) using Monte Carlo tree search and Q-learning to distill these outcomes into family-conditioned values. RoboAware achieves state-of-the-art success rates on RoboSuite, LIBERO-Pro, and RoboTwin benchmarks.
Entities (11)
Relation Signals (10)
RoboAware → achievessuccessrateon → RoboTwin
confidence 95% · 90.0% on challenging RoboTwin bimanual tasks
RoboAware → achievessuccessrateon → LIBERO-PRO
confidence 95% · 73.8% on diverse LIBERO-Pro task clusters
RoboAware → achievessuccessrateon → robosuite
confidence 95% · SOTA averages of 90.0% on RoboSuite
RoboAware → uses → P5 Schema
confidence 95% · we propose the P5 schema and formulate a hierarchical MDP based on it.
RoboAware → uses → State-Locked Counterfactual Branching
confidence 95% · we introduce State-Locked Counterfactual Branching (SCB)... To address the lack of counterfactual branch outcomes
RoboAware → uses → Execution-Aware Learning
confidence 95% · we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning
RoboAware → outperforms → VLA-harness
confidence 90% · outperforming existing code-as-policy and VLA-harness baselines.
RoboAware → outperforms → Code-as-policy
confidence 90% · outperforming existing code-as-policy and VLA-harness baselines.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the $P^5$ schema and formulate a hierarchical MDP based on it. $P^5$ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.
Tags
Links
- Source: https://arxiv.org/abs/2610.11480v1
- Canonical: https://arxiv.org/abs/2610.11480v1
Trouble viewing inline? Open PDF directly →
Full Text
129,479 characters extracted from source content.
Expand or collapse full text
RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes Bohan Zhou Xingbei Chen Emily Huang Weilin Ruan Haojian Huang Yehang Zhang Zexi Li Wenqian Li Qize Yu Zetian Song Leyi Wu Jinghao Li Mingxuan Song Xinrun Xu Zongyang Qiu Yangkai Wei Tianyi Zhang Kaiwen Zhou Yinchuan Li James Cheng Abstract Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding agents’ skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the P5P^5 schema and formulate a hierarchical MDP based on it. P5P^5 organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines. Figure 1: Overview of RoboAware. (a) The P5P^5 skill schema decomposes each skill into five stages—perceive, propose, pre-manipulate, perform, and post-manipulate—and unifies primitive skills (e.g., pick, place, push, pull) and compositional skills (e.g., handover) under one representation. (b) A hierarchical MDP selects a policy family ci∈Mod,E2Ec_i∈\Mod,E2E\ at each P5P^5 stage decision boundary; a coding agent then generates each corresponding code block aia_i. (c) EAL learns family-conditioned coordination: SCB evaluates modular and end-to-end branches from the same restored state into SCBD_SCB, Q-learning yields Qθ(x,c)Q_θ(x,c), and at test time a frozen coordinator selects best cic_i to condition code generation and execution. 1 Introduction Performing general-purpose embodied manipulation in diverse and unstructured environments remains challenging in the absence of low-level executor adaptation. Recent progress toward this goal has followed two major paradigms, namely end-to-end policies and coding agents, each with distinct strengths and limitations. Within the end-to-end paradigm, task-specific visuomotor controllers excel at contact-rich control but typically lack native support for open-vocabulary rebinding [1, 2, 3]. Vision-language-action (VLA) and world-action models (WAMs) support language-conditioned execution [4, 5, 6, 7], yet become unreliable under compositional and state-distribution shifts. Coding agents, by contrast, enable Code-as-Policy [8, 9, 10] by flexibly composing program modules for long-horizon tasks. Their physical competence, however, is bounded by the exposed primitives, limiting fine-grained, closed-loop correction during contact-rich interactions. These complementary advantages inspire hybrid frameworks in which coding agents generate modular API compositions or invoke frozen end-to-end policies, most commonly VLAs, as contact-rich primitives [11, 12, 13]. Nevertheless, high-level coding agents lack inherent awareness of the divergent consequences of invoking different skill executors. They struggle to choose among an explicit modular composition, any available self-contained policy, and their possible switching patterns. For instance, modular execution can remain stable under task-distribution shift, whereas end-to-end execution excels in case of unexpected interruption. This raises a critical research question: How can we endow coding agents with outcome-aware execution responsibility over embodied skills? We address this question with RoboAware, a framework that learns a responsibility boundary between modular composition and end-to-end execution from counterfactual outcomes. Here, a policy family c∈Mod,E2Ec∈\Mod,E2E\ is the sole high-level coordination action; the API library comprises typed APIs together with their bound low-level backends; responsibility denotes which family should control the next semantically meaningful skill transition. The first challenge is to define where responsibility can be compared. The space of low-level code generation is too expansive and unstructured to provide a tractable basis for responsibility assignment. Inspired by the success of REPL-style programming [14], we propose the 5P^5 Skill Schema to address this limitation. The P5P^5 schema organizes embodied skills into semantically meaningful stages from perception to execution in 3D space. We formulate a finite-horizon hierarchical MDP that selects a policy family at each P5P^5 stage. At the lower level, we curate an API library combining typed modular interfaces for composability and controllability with end-to-end policies for contact-rich local execution. Conditioned on the selected family and current context, the frozen coding agent πcode _code generates a complete local code block through the fixed API library. Learning is confined to the responsibility coordinator, while skill implementation leverages the coding agent’s existing code-generation capability. The second challenge is to expose the missing outcome of the skills that were not selected. Many approaches execute open-loop skill sequences or code programs without attending to outcomes [8, 15, 10]. Methods that obtain outcomes typically rely on VLM-based evaluations limited to the predicted outcome of the executed trajectory, and therefore inherit selection bias [13, 16]. We introduce State-Locked Counterfactual Branching (SCB), which leverages simulator (1) save-restore mechanisms and (2) oracle outcome judgments to execute every admissible family from an identical physical state, producing a same-state set of potential outcomes at each P5P^5 decision point. To convert these set-wise outcomes into deployable family selection, we formulate Execution-Aware Learning (EAL), in which Monte Carlo tree search (MCTS) treats restorable P5P^5 decision states as nodes and admissible policy families as edges, while Q-learning distills the explored outcomes into a family-conditioned value function QθQ_θ. The family-wise values encode a relative responsibility preference over modular composition and the end-to-end family, reflecting expected remaining skill- and task-level return under an observation. During deployment, the RoboAware coordinator assigns responsibility by maximizing ρθ _θ induced by QθQ_θ, and a coding agent generates executable code blocks accordingly to transition from one P5P^5 state to the next, without state restoration or privileged simulator state. See Figure 1 for an overview of the proposed framework. Our contributions are as follows: • We formulate RoboAware as a hierarchical MDP that selects a policy family cic_i at each P5P^5 stage to produce code block aia_i. We curate its lower-level API library around two complementary families, namely composable modular components and end-to-end policies for contact-rich local skills. • We introduce SCB to acquire paired locked-state outcomes, and EAL to learn family-conditioned QθQ_θ through MCTS-guided exploration and Q-learning. • We recover the coordinator from QθQ_θ and evaluate it on 100 tasks across three benchmarks, obtaining a SOTA 77.0% overall success rate. Ablations confirm the effectiveness of our designs. 2 Methodology 2.1 Problem Formulation We consider language-conditioned manipulation in a physics environment ℰE, where the observation at time t is ot=(Itrgb,Itd,qt)o_t=(I_t^rgb,I_t^d,q_t) from an agent-view RGB image, a metric depth map, and proprioception. We formulate the task as a finite-horizon MDP ℳ=(,,,r,γ)M=(X,C,P,r,γ). Decision point i occurs at the start of a P5P^5 stage, where the observation xi=(ℓ,oi,hi,zi,gi)∈x_i=( ,o_i,h_i,z_i,g_i) is received. Here ℓ denotes the language instruction, hih_i the observable execution history, ziz_i the current P5P^5 stage, and gig_i the active skill schema. The high-level action is a policy-family choice ci∈i(zi,gi)⊆=Mod,E2Ec_i _i(z_i,g_i) =\Mod,E2E\. A coding agent generates a complete executable code block ai∼πcode(⋅∣xi,ci)a_i _code(· x_i,c_i) with ai∈ci(xi,zi,gi)a_i _c_i(x_i,z_i,g_i), where ModA_Mod and E2EA_E2E are the modular- and end-to-end-family code-block spaces. Executing aia_i in ℰE until the next decision point induces the transition (xi+1∣xi,ci)=ai∼πcode(⋅∣xi,ci)[ℰ(xi+1∣xi,ai)]P(x_i+1 x_i,c_i)=E_a_i _code(· x_i,c_i)[P_E(x_i+1 x_i,a_i)]. A binary reward r is received at task termination. γ=1γ=1. Only the coordinator’s value model QθQ_θ is trained; the coding agent, API interfaces, backend bindings, and low-level backend parameters remain fixed. 2.2 RoboAware Framework Figure 2: Method overview of EAL in RoboAware. (1) State-locked counterfactual search repeats Selection (UCB), Expansion (SCB paired ModMod/E2EE2E branches from a restored ξi _i), Rollout (suffix return GiG_i), and Backup within budget K, retaining successful, failed, and nonpreferred transitions in SCBD_SCB. (2) A VLM backbone with an MLP head maps context–family tokens ending in <VALUE> to Qθ(xi,ci)Q_θ(x_i,c_i), trained by Bellman updates on skill-completion and terminal-task feedback. (3) At deployment, frozen QθQ_θ induces ρθ _θ and selects ci∗=argmaxcρθ(c∣xi)c_i^*= _c _θ(c x_i); a frozen coding agent generates a local code block against the fixed API library and executes it, looping on observable feedback at each P5P^5 boundary. RoboAware formulates heterogeneous policy-family orchestration as stage-conditioned responsibility learning. The pipeline is visualized in Figure 2 and elaborated in Algorithm 1. 2.2.1 Skill Preparation 5P^5 Skill Schema. We instantiate local embodied skills through either interface-based modular composition or a self-contained end-to-end controller. The P5P^5 skill schema decomposes each skill into five stages: perceive, propose, pre-manipulate, perform, and post-manipulate. As shown in Figure 1, this schema provides a unified representation for arbitrary skills, including primitive skills (e.g. pick, place, push, pull) and compositional skills (e.g. handover). Given the task instruction ℓ and initial observation o0o_0, the coding agent constructs a task-level plan G comprising P5P^5 skill schemas. Responsibility is reconsidered at each P5P^5 stage. API Library Construction. Building on the P5P^5 stages, we expose an API library that supports the modular and end-to-end policy families. The ModMod family uses an explicit intermediate interface for composability and controllability, whereas the E2EE2E family relies on an implicit internal loop for local contact competence. We assemble the library by offline screening of open-source modular perception, grasping, and motion components alongside released VLA and world-action-model (WAM) policies. For the ModMod family, screening retains InstructSAM [17], our augmented NeuGraspNet+ from NeuGraspNet [18]—which expands sparse geometric candidates and selects Top-1 only among planner-feasible grasps—and cuRoboV2 [19] as the core backends for perception, pose proposal, and motion planning, respectively. Together with the deterministic gripper API, these components define the modular calls available to πcode _code. For the E2EE2E family, offline evaluation reveals limited zero-shot performance on complete tasks, yet these policies perform well on local skills such as grasping and lifting (see Section 3.2). We therefore position VLA and WAM policies as local executors at grasp–lift granularity rather than as end-to-end policies for complete tasks. Table 1: Offline screening of the selected VLA and WAM backends (successful / total). Full: complete task; G–L: grasp and lift. Method LIBERO-Pro RoboSuite RoboTwin 2.0 Full G–L Full G–L Full G–L π0.5 _0.5 10/14 13/14 0/4 2/4 0/10 2/10 FastWAM 4/14 11/14 0/4 0/4 10/10 10/10 Table 2: High-level embodied APIs under the P5P^5 skill schema; ✓ marks stages covered by each API. API perc. prop. pre perf. post Perceive_object_pose ✓ Propose_eef_pose ✓ Plan_to_eef_pose ✓ ✓ Perform_gripper ✓ Perform_E2E ✓ ✓ ✓ ✓ Effectiveness and efficiency screening selects π0.5 _0.5 [20] and FastWAM [21] for integration into Perform_E2E (see Table 2). Screening details appear in Appendix B. Each stage admits both families except the propose stage (see Table 2). Perform_E2E spans the perceive, pre-manipulate, perform, and post-manipulate stages. A call remains atomic with respect to family selection until it meets its registered postcondition, reports failure, or exhausts its step budget. The API library, including its typed interfaces, backend bindings, and checkpoints, remains frozen during training and evaluation. 2.2.2 Execution-Aware Learning Existing coding agents cannot reliably anticipate the physical consequences of executing a generated skill. To address this limitation, we propose Execution-Aware Learning (EAL) as illustrated in Figure 2, which learns family-conditioned outcome values from controlled simulator experience. State-Locked Counterfactual Branching (SCB) is the training-time operator that makes the otherwise missing policy-family outcomes observable. On a fixed set of simulator training tasks, EAL builds a Monte Carlo search tree from each sampled P5P^5 stage-entry context. Nodes are restorable decision states, and outgoing edges correspond to admissible policy families. Each node stores a snapshot ξi=(xi,mi,pi,r) _i=(x_i,m_i,p_i,r) containing the simulator and execution state required for exact restoration. Beyond the deployment-visible context xix_i, it provides oracle masks mim_i, ground-truth object poses pip_i, and access to the terminal reward evaluator r. These privileged signals support the training rewards including pose-based assessment under threshold ϵp _p. The search repeats four search phases within a budget of K completed suffix simulations per root. See Section A.2 for details. Selection. Starting from the root, the search follows stored tree paths by selecting family edges with UCB at already expanded nodes, where NiN_i and Ni(c)N_i(c) are node and family-edge visit counts, and Q^i(c) Q_i(c) is the empirical return estimate: citree=argmaxc∈i[Q^i(c)+cucblog(1+Ni)1+Ni(c)].c_i^tree= _c _i [ Q_i(c)+c_ucb (1+N_i)1+N_i(c) ]. (1) Expansion. At a new node, SCB evaluates every admissible family at least once from the same physical state and observable-history prefix. For each c∈ic _i, it restores ξi _i, samples aic∼πcode(⋅∣xi,c)a_i^c _code(· x_i,c), and executes the block to the next decision point or termination. The resulting transition contains a successor snapshot ξi+1c _i+1^c, terminal indicator dicd_i^c, and training reward r~ic r_i^c. Reevaluating an existing family edge repeats restoration, code generation, and execution with a fresh code sample. Rollout. From each nonterminal successor, the codes are generated using API library to complete the suffix. The suffix ends at task termination, unrecoverable failure, or the episode horizon T. Rollout-only states are not added as expanded tree nodes during this simulation. Each completed suffix yields Gi=∑j=iTγj−ir~jG_i= _j=i^Tγ^j-i r_j. r~ic r_i^c is defined in Equation 3. Backpropagation. This tree-statistics backup propagates the suffix return GiG_i along the selected tree path, including each ancestor’s recorded prefix rewards. Each visited node and family-edge count is incremented, and Q Q is updated by empirical averaging. These statistics guide subsequent selection; the parameters of QθQ_θ are optimized only after data collection. Within budget K=20K=20, all evaluated tree edges, including failed and nonpreferred branches, contribute transitions to the training dataset: SCB=⋃i(xi,c,r~ic,xi+1c,dic,ξi+1c):c∈i.D_SCB= _i \(x_i,c, r_i^c,x_i+1^c,d_i^c, _i+1^c):c _i \. (2) Execution-Aware Q-learning. A task comprises several P5P^5 skill blocks. At the end of each P5P^5 stage, oracle signals in ξi+1c _i+1^c yield a perception score rp,ic∈[0,1]r_p,i^c∈[0,1] from agreement with mask mim_i, and a manipulation score rm,ic∈[0,1]r_m,i^c∈[0,1] from object-pose error against pip_i under threshold ϵp _p. The binary task-success reward r∈0,1r∈\0,1\ is received only when dic=1d_i^c=1. The transition reward used in both tree search and Q-learning is the linear combination with non-negative weights: r~ic=wprp,ic+wmrm,ic+wtdicr,where wp+wm+wt=1. r_i^c=w_pr_p,i^c+w_mr_m,i^c+w_td_i^cr, w_p+w_m+w_t=1. (3) The learning target is Qθ(xi,c)≈maxπ[∑j=iTγj−ir~j∣xi,ci=c]Q_θ(x_i,c)≈ _πE_π [ _j=i^Tγ^j-i r_j x_i,c_i=c ], where π governs subsequent family choices under the fixed coding agent and API library. The value Qθ(xi,c)Q_θ(x_i,c) estimates the remaining skill- and task-level return conditional on choosing family c next. After collecting SCBD_SCB across the training tasks, we optimize QθQ_θ on this fixed transition dataset using Bellman targets rather than directly regressing the rollout returns GiG_i or tree estimates Q^i(c) Q_i(c). All evaluated tree edges, including failed and nonpreferred branches, contribute to this update. For transitions sampled from SCBD_SCB, we optimize θ by gradient-based backpropagation through QθQ_θ, minimizing the following Smooth-L1L_1 Bellman loss: ℒEAL(θ)=(xi,c)∼SCBℓ[Qθ(xi,c)−(r~ic+γ(1−dic)maxc′∈i+1Qθ¯(xi+1c,c′))],L_EAL(θ)=E_(x_i,c) _SCB [Q_θ(x_i,c)- ( r_i^c+γ(1-d_i^c) _c _i+1Q_ θ(x_i+1^c,c ) ) ], (4) where Qθ¯Q_ θ is a stop-gradient target copy and ℓ(ϵ) (ε) refers to the Smooth-L1L_1 loss, which maintains stable gradients in the regime of large errors as the L1L_1 loss does and facilitates smooth optimization for small errors analogous to the L2L_2 loss: ℓ(ϵ)=12ϵ2if |ϵ|<1,|ϵ|−12otherwise. (ε)= cases 12ε^2&if |ε|<1,\\ |ε|- 12&otherwise. cases (5) Q-Modeling. Qθ(xi,c)Q_θ(x_i,c) is parameterized with a lightweight VLM backbone (e.g. Qwen3.5-0.8B). The state xi=(ℓ,oi,hi,zi,gi)x_i=( ,o_i,h_i,z_i,g_i) and family c form a multimodal token sequence ending in a <VALUE> token. An MLP head maps its final-layer hidden state to the scalar Q-value, optimized with the Bellman objective in Equation 4. 2.2.3 Test-Time Skill Coordination During training, repeated access to ξi _i and oracle outcome judgments supervises values, whereas neither is included at test time. The learned values induce a relative responsibility preference over admissible families: ρθ(c∣xi)=exp(Qθ(xi,c))∑c′∈iexp(Qθ(xi,c′)). _θ(c x_i)= \! (Q_θ(x_i,c) ) _c _i \! (Q_θ(x_i,c ) ). (6) The RoboAware coordinator selects ci∗=argmaxc∈iρθ(c∣xi)c_i^*= _c _i _θ(c x_i) (equivalently argmaxcQθ(xi,c) _cQ_θ(x_i,c); see Section A.3), and the coding agent generates ai∼πcode(⋅∣xi,ci∗)a_i _code(· x_i,c_i^*). The coding agent executes the selected block until the next semantic boundary or task termination, then updates (hi,zi,gi)(h_i,z_i,g_i) from observable feedback. Family selection repeats at each subsequent boundary with QθQ_θ fixed. Both family selection and local code generation respond to updated observations without parameter updates or changes to executor interfaces. 3 Experiments 3.1 Experimental Protocol Benchmarks. Our evaluation contains 100 tabletop-manipulation tasks across three benchmarks: LIBERO-Pro [22], RoboSuite [23], and RoboTwin 2.0 [24]. We use scene seed 0 only to collect privileged SCB branches and train QθQ_θ. The learned coordinator is then frozen and evaluated on seeds 1–10, giving 1,000 test episodes in total. Baselines. We compare RoboAware with the following baselines: • End-to-end policies: π0 _0 [7], π0.5 _0.5 [20], FastWAM [21], FasterWAM [25], as standalone controllers; • Code-as-policy: CaP [8], CaP-X [9], RATS [26], and ASPIRE [27], without learned family coordination. • VLA with harness: Pigey [28], VLS [29], and Harness-VLA [11], which invoke a frozen policy. 3.2 Benchmark Performance Table 3: Aggregate success rates (%) on LIBERO-Pro (instruction-redirection T / position-swap S), RoboSuite, and RoboTwin. RoboSuite and RoboTwin use their full task pools under seeds 11–1010. Overall is the mean over available columns. On RoboTwin, π0 _0 and π0.5 _0.5 are incompatible with RoboTwin’s dual-arm setup, so results involving these models are marked “/” because of the embodiment mismatch. For fair comparison, global cross-episode memory is disabled for baselines that rely on memory. All seeds are tested in one trial. Method LIBERO-Pro RoboSuite RoboTwin Overall Spat-T Spat-S Obj-T Obj-S Goal-T Goal-S L10-T L10-S End-to-end policies π0 _0 [7] 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.0 0.0 / 0.2 π0.5 _0.5 [20] 1.0 20.0 1.0 17.0 2.0 38.0 1.0 8.0 0.0 / 9.8 FastWAM [21] 30.0 0.0 10.0 0.0 10.0 0.0 10.0 0.0 0.0 81.9 14.2 FasterWAM [25] 50.0 0.0 10.0 10.0 10.0 10.0 10.0 0.0 0.0 85.6 18.6 Code as policy CaP [8] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 70.0 5.6 7.6 CaP-X [9] 0.0 0.0 10.0 0.0 0.0 0.0 0.0 0.0 50.0 6.9 6.7 RATS [26] 2.0 18.0 10.0 8.0 8.0 8.0 0.0 0.0 45.0 0.0 9.9 ASPIRE [27] 60.0 51.0 95.0 98.0 45.0 81.0 38.0 22.0 80.0 0.0 57.0 VLA with harness Pigey [28] 80.0 66.0 54.0 54.0 22.0 44.0 28.0 12.0 15.0 / 41.7 VLS [29] 54.0 42.0 41.0 45.0 33.0 38.0 25.0 15.0 0.0 / 32.6 Harness-VLA [11] 50.0 70.0 50.0 90.0 40.0 60.0 40.0 30.0 20.0 / 50.0 RoboAware (ours) 68.0 68.0 92.0 86.0 66.0 84.0 64.0 62.0 90.0 90.0 77.0 End-to-end policies struggle with whole-task distribution shifts. For both VLAs and WAMs, unfamiliar skill combinations, scene types, and perturbations may deviate substantially from their training distributions, leading to low zero-shot SRs in Table 3. In comparison, RoboAware generates composable modular alternatives under such conditions while retaining end-to-end policies for local skills suited to their capabilities. Code-as-policy methods are constrained by contact precision and accumulated experience. Code-as-policy methods offer stronger long-term planning and composability for long-horizon bimanual tasks such as those in RoboTwin. However, within-skill contact-rich performance depends on available primitives: CaP and CaP-X generate detailed code at test time, yet may lack the precision needed for tasks such as placing a mug inside a microwave in LIBERO-Pro (see Figure 3). Single-turn evaluation without cross-episode adaptation or multi-turn memory further limits the benefits of accumulated experience for memory-based methods like CaP-X, RATS, and ASPIRE. RoboAware addresses these limitations by combining modular composition with local end-to-end control and learning state-dependent family responsibility from training-time outcomes. VLA harnesses remain limited by outcome anticipation. We observe that VLS [29] can be misled by early-stage behavior in long-horizon tasks, where initial progress may not reflect readiness for later interactions. Harness-VLA is closely related to our work and achieves competitive performance. However, once the selected family changes the physical state, the alternative family’s outcome at the original state remains unobserved, so family switching relies on experiential judgments. RoboAware addresses this gap through SCB and EAL, which expose same-state alternatives for family selection. 3.3 Experimental Analysis Figure 3: Comparison between CaP-X and RoboAware on a contact-rich LIBERO-Pro task “Put the white mug in the microwave and close it.” Figure 4: A representative LIBERO-Pro task comparison and visualization of the responsibility preference distribution ρθ _θ for RoboAware, RoboAware-EO, and RoboAware-MO. Takeaway 1: Neither policy family alone provides broad high-success task coverage. We restrict code generation to modular composition (RoboAware-MO) or end-to-end blocks (RoboAware-EO), keeping the same API library and tasks as RoboAware. Success rate (SR) reductions in group A of Section 3.3 show that both families must be coordinated by state. In a LIBERO-Pro task illustrated in Figure 4, RoboAware-EO and RoboAware-MO fail at different stages, whereas RoboAware succeeds by switching policy families across P5P^5 boundaries. The ρθ _θ visualization assigns lower preference to the families used by the ablation variants at the illustrated states, consistent with their failures. Table 4: Ablation success rates (%) on the three benchmarks, grouped by conclusions A–C in this analysis. RoboAware is the common reference. LIBERO-Pro aggregates its eight T/S evaluation splits. Abbreviations: MO = modular only; EO = end-to-end only; UO = unpaired outcomes; TF = training free; FH = fixed handoff; ES = episode selection. Variants LIBERO-Pro RoboSuite RoboTwin 2.0 A. Policy Family RoboAware-MO 54.8 67.5 41.9 RoboAware-EO 49.5 60.0 45.6 B. Counterfactual Outcome RoboAware-UO 58.5 75.0 53.8 RoboAware-TF 51.0 60.0 40.0 C. Coordination Mode RoboAware-FH 64.5 82.5 55.6 RoboAware-ES 59.3 70.0 51.9 RoboAware 73.8 90.0 90.0 Table 5: Group B ablations of coordinator training and same-state outcomes. ✓: present; ✗: absent. Variant Training Counterfactual ours ✓ ✓ ours-UO ✓ ✗ ours-TF ✗ ✗ Table 6: Group C ablations of when family responsibility may change. When: decision points; Family: modular / e2e. Variant When Family ours Each stage Mod / E2E ours-FH Fixed handoff Mod + E2E ours-ES Episode start Mod / E2E Takeaway 2: Paired same-state outcomes improve Execution-aware learning. To evaluate the importance of SCB, we ablate coordinator training and counterfactual supervision (Section 3.3): RoboAware uses both, RoboAware-UO trains QθQ_θ only on executed trajectories without state-locked branching, and RoboAware-TF skips coordinator training so the coding agent chooses directly from the hybrid API library. Higher SR for RoboAware-UO than RoboAware-TF in group B of Section 3.3 indicates that the coding agent lacks inherent execution awareness and that coordinator training still helps without counterfactuals. Yet executed-only branches inherit the agent’s experience distribution and introduce selection bias. A modest SR decrease for RoboAware-UO relative to RoboAware therefore supports the necessity of paired counterfactual executions from the same ξi _i. Takeaway 3: RoboAware benefits from fine-grained coordination at P5P^5 stages. Following the same training protocol to obtain QθQ_θ, we compare three coordination strategies that differ in when family responsibility may change. RoboAware reselects a policy family at every P5P^5 stage. RoboAware-FH (fixed handoff) follows a fixed protocol: modular APIs for perceive and pre-manipulate, then an E2EE2E call for subsequent execution. RoboAware-ES (episode selection) chooses one family at episode start and retains it for the entire task. Lower SRs for both variants in group C of Section 3.3 suggests that neither a fixed handoff nor a single episode-level decision tracking changing execution responsibility as well as per-stage coordination (Table 6). Figure 5: Average RoboSuite success rates (%) across budgets K∈5,10,20,50K∈\5,10,20,50\. Hollow & filled circles denote RoboAware-UA & RoboAware. We adopt K=20K=20. Figure 6: RoboTwin 2.0 success rates (%) on a four-task subset across coding-agent backends (rows) and embodiments (columns). Color bar shows the SR range. Takeaway 4: UCB-guided exploration uses a limited branching budget more effectively. To test how a finite simulation budget is allocated during branching, we compare RoboAware with RoboAware-UA, which allocates evaluations uniformly over admissible families instead of the UCB rule in Equation 1. Figure 5 reports both under K∈5,10,20,50K∈\5,10,20,50\ in RoboSuite. SR gains diminish with K, so we use the elbow K=20K=20. Under matched budgets, RoboAware exceeds RoboAware-UA, indicating that UCB concentrates evaluations on promising families more effectively than uniform allocation. Takeaway 5: RoboAware is less sensitive to coding-agent and embodiment variations. To assess sensitivity to changes in the coding agent and robot embodiment, we evaluate RoboAware with three coding-agent backends (Claude Code as default, Cursor, and Codex) across four embodiments (Aloha-AgileX as default, Franka-Panda, ARX-X5, and UR5-WSG) on four tasks in RoboTwin 2.0 (See Appendix Section C.1 for details), retaining the same API interfaces. Figure 6 shows a small average SR gap across coding-agent backends and embodiments. We attribute the small backend gap to P5P^5, which limits each generation to one code block ai∈ci(xi,zi,gi)a_i _c_i(x_i,z_i,g_i). Notably, the cuRoboV2 backend of Plan_to_eef_pose inherently supports different robots through the same API interface across embodiments. 4 Related Work 4.1 Embodied Coding Agents Embodied coding agents have progressed along three lines. Early language planners translated goals into sequences over closed skill APIs and improved grounding through semantic matching, affordance scoring, program-structured prompting, and execution feedback [30, 31, 15, 32]. Executable-policy methods such as CaP [8], VoxPoser [10], RoboCodeX [33], and CodeDiffuser [34] then generated Python controllers or geometric intermediates, gaining compositionality, spatial reasoning, and interpretability while remaining bounded by designer-provided abstractions and API coverage. Recent systems make the program itself adaptive. CaP-X [9] and AOR [35] repair code from multimodal rollouts, RHO [36] and GaP [37] optimize repository- or graph-structured policies through execution, VLCP [38] rewrites failed controllers within an episode, PhysCaP [39] actively probes latent physical properties, and ASPIRE [27] compounds validated repairs into a reusable skill library. This line strengthens modular program execution, yet its evidence is conditional on generating a modular code block. Neither success nor code-only repair reveals whether an end-to-end block performs better at the same state. We instead use SCB to generate and execute modular and end-to-end coding blocks from the same restored state, and EAL learns their family-conditioned values from paired outcomes. 4.2 End-to-End Policy Orchestration and Execution Harnesses End-to-end policy orchestration has likewise progressed along three lines: generalist VLAs including RT-1 [4], RT-2 [5], Octo [40], OpenVLA [6], and π0 _0 [7] broaden local language-conditioned control, while hierarchical agents move planning, memory, grounding, and progress tracking to a VLM or code layer, exposing VLAs as bounded tools. Recent variants add tool-aligned post-training, persistent semantic state, tractability-aware interfaces, interruptible physical orchestration, and modular monitor, planner, and controller wrappers [12, 41, 28, 11, 42, 43, 44, 45]. VLS [29] instead steers the sampling of a frozen generative policy with VLM-synthesized rewards. Hybrid and heterogeneous-policy harnesses including RoboRouter [46], RoboHarness [13], and RouterVLA [47] then infer applicability from execution traces, semantic retrieval, capability memory, outcome-disjoint probes, or transition structure. Related methods including Reset [48], SwitchVLA [49], SemHandoff [50], and BICPO-VLA [51] prepare out-of-distribution initial states, condition switching on execution phase, diagnose semantic handoffs, or reduce asynchronous takeover mismatch. Finally, VLM oracles, learned rewards, frozen-feature success probes, counterfactual relabeling, and self-evolving critics detect, rank, or repair attempts [52, 53, 54, 55, 56, 57, 58, 16, 59, 60]. These methods estimate absolute success, compare within-policy alternatives, or infer cross-policy suitability from retrospective traces and probes. Our distinction is to use simulator-restored, same-state executions to obtain paired outcomes for modular and end-to-end families within a common skill-stage interface. 5 Conclusion and Limitations We presented RoboAware, which learns state-dependent responsibility between modular composition and end-to-end control while keeping the coding agent and API library fixed. The P5P^5 skill schema defines the semantic boundaries at which responsibility is assigned, and SCB exposes both family outcomes at such a boundary by executing paired code blocks from identical restored states. From this paired experience, EAL combines UCB-guided MCTS with Q-learning into family-conditioned estimates of remaining skill- and task-level return. At deployment, these estimates select a family before code generation from observable context alone, and responsibility is reconsidered at each subsequent boundary without simulator restoration, privileged inputs, or online search. Across 100 tasks in LIBERO-Pro, RoboSuite, and RoboTwin 2.0, this makes anticipated physical outcomes an explicit basis for composing complementary frozen skills. We outline two directions for future work. First, RoboAware currently integrates only modular composition and end-to-end policies. Extending it to generalizable reinforcement learning policies, retrieval-based policies, and policies learned from demonstrations may require new counterfactual outcome data and further coordinator training. Second, responsibility also changes hands only at P5P^5 boundaries, so a single atomic end-to-end invocation may span several stages and leave transient failures to its internal controller until a postcondition, failure signal, or timeout returns control. Finer-grained interruption under ambiguous observations therefore remains an open problem. References [1] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems (RSS), Cited by: §1. [2] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song (2023) Diffusion policy: visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), Cited by: §1. [3] Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu (2024) 3D diffusion policy: generalizable visuomotor policy learning via simple 3D representations. In Robotics: Science and Systems (RSS), Cited by: §1. [4] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, et al. (2023) RT-1: robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), Cited by: §1, §4.2. [5] B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, et al. (2023) RT-2: vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), Cited by: §1, §4.2. [6] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, et al. (2024) OpenVLA: an open-source vision-language-action model. In Conference on Robot Learning (CoRL), Cited by: §1, §4.2. [7] K. Black, N. Brown, D. Driess, A. Esmail, M. R. Equi, et al. (2025) π0 _0: a vision-language-action flow model for general robot control. In Robotics: Science and Systems (RSS), Cited by: §C.2, §1, 1st item, Table 3, §4.2. [8] J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng (2023) Code as policies: language model programs for embodied control. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: §C.2, §1, §1, 2nd item, Table 3, §4.1. [9] L. Fu, J. Yu, K. El-Refai, E. Kou, H. Xue, et al. (2026) CaP-X: a framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.22435. Cited by: §C.2, §1, 2nd item, Table 3, §4.1. [10] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and F. Li (2023) VoxPoser: composable 3D value maps for robotic manipulation with language models. In Conference on Robot Learning (CoRL), Cited by: §1, §1, §4.1. [11] Y. Zhang, H. Zhang, F. Gao, X. Li, Z. Liu, et al. (2026) Harness VLA: steering frozen VLAs into reliable manipulation primitives via memory-guided agents. arXiv preprint arXiv:2607.08448. Cited by: §C.2, §1, 3rd item, Table 3, §4.2. [12] J. Hu, M. Shridhar, C. Lu, D. Shah, H. L. Chiang, J. Tan, and A. Xie (2026) What matters in orchestrating robot policies: a systematic study of hierarchical VLA agents. arXiv preprint arXiv:2606.10267. Cited by: §1, §4.2. [13] J. Huang, Y. Hu, Z. Li, R. Qi, Y. Xiao, et al. (2026) RoboHarness: memory-driven orchestration of heterogeneous robot policies for long-horizon planning. arXiv preprint arXiv:2607.18060. Cited by: §1, §1, §4.2. [14] A. Z. Liu, X. Wang, J. Sansom, Y. Fu, J. Choi, S. Sohn, J. Kim, and H. Lee (2025) Interactive and expressive code-augmented planning with large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 20330–20354. External Links: Document Cited by: §1. [15] I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg (2023) ProgPrompt: generating situated robot task plans using large language models. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: §1, §4.1. [16] J. Zhang, J. Nie, J. Lao, W. Cheng, C. Liu, J. Jiang, and S. Huang (2026) What frozen VLAs already know about success: a probing study of value-like structure in foundation robot policies. arXiv preprint arXiv:2605.28527. Cited by: §1, §4.2. [17] Y. Yuan, W. Li, Z. Li, Y. Lin, J. Li, S. Tang, J. Xiao, Y. Zhuang, and W. Zhang (2026) InstructSAM: segment any instance with any instructions. arXiv preprint arXiv:2605.26102. Cited by: §B.2, §2.2.1. [18] S. Jauhri, I. Lunawat, and G. Chalvatzaki (2024) Learning any-view 6DoF robotic grasping in cluttered scenes via neural surface rendering. In Robotics: Science and Systems (RSS), Cited by: §B.3, Table 6, §2.2.1. [19] B. Sundaralingam, A. Murali, and S. Birchfield (2026) cuRoboV2: dynamics-aware motion generation with depth-fused distance fields for high-DoF robots. arXiv preprint arXiv:2603.05493. Cited by: §B.4, Table 7, §2.2.1. [20] Physical Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, et al. (2025) π0.5 _0.5: a vision-language-action model with open-world generalization. In Proceedings of The 9th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 305, p. 17–40. External Links: Link Cited by: Table 8, §C.2, §2.2.1, 1st item, Table 3. [21] T. Yuan, Z. Dong, Y. Liu, and H. Zhao (2026) Fast-WAM: do world action models need test-time future imagination?. arXiv preprint arXiv:2603.16666. External Links: Link Cited by: Table 8, §C.2, §2.2.1, 1st item, Table 3. [22] X. Zhou, Y. Xu, G. Tie, Y. Chen, G. Zhang, D. Chu, P. Zhou, and L. Sun (2025) LIBERO-PRO: towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827. Cited by: §C.1, §3.1. [23] Y. Zhu, J. Wong, A. Mandlekar, R. Martín-Martín, A. Joshi, K. Lin, S. Nasiriany, and Y. Zhu (2020) Robosuite: a modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009.12293. Cited by: §C.1, §3.1. [24] T. Chen, Z. Chen, B. Chen, Z. Cai, Y. Liu, Q. Liang, Z. Li, X. Lin, Y. Ge, Z. Gu, et al. (2026) RoboTwin 2.0: a scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. In International Conference on Machine Learning (ICML), External Links: Link Cited by: §C.1, §3.1. [25] W. Zhao, H. Jiang, X. Shi, L. Liu, F. Huang, Z. Su, W. Sui, and X. Wang (2026) Faster-WAM: efficient inference-time future conditioning for robust world action models. arXiv preprint arXiv:2608.04404. External Links: Link Cited by: Table 8, §C.2, 1st item, Table 3. [26] J. Zhang, J. Ge, H. Yoo, L. Fu, Z. Yang, Y. Liu, R. Saravanan, S. Yin, J. Yu, D. Niu, et al. (2026) Playful agentic robot learning. arXiv preprint arXiv:2606.19419. Cited by: §C.2, 2nd item, Table 3. [27] R. Lu, Y. Wu, E. Kou, L. Fu, W. Xiao, A. Mandlekar, Y. Xu, G. Shi, K. Goldberg, A. Chen, M. Chowdhury, Y. Zhu, L. “. Fan, and G. Wang (2026) ASPIRE: agentic /skills discovery for robotics. arXiv preprint arXiv:2607.00272. Cited by: §C.2, 2nd item, Table 3, §4.1. [28] L. Galanti, D. Shah, and T. Dao (2026) Addressing the orchestration gap in generalist robots via physical agency. arXiv preprint arXiv:2607.21725. Cited by: §C.2, 3rd item, Table 3, §4.2. [29] S. Liu, I. S. Singh, Y. Xu, J. Duan, and R. Krishna (2026) VLS: steering pretrained robot policies via vision-language models. arXiv preprint arXiv:2602.03973. Cited by: §C.2, 3rd item, §3.2, Table 3, §4.2. [30] C. H. Song, J. Wu, C. Washington, B. M. Sadler, W. Chao, and Y. Su (2023) LLM-Planner: few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 2998–3009. External Links: Document Cited by: §4.1. [31] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, et al. (2023) Do as I can, not as I say: grounding language in robotic affordances. In Proceedings of The 6th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 205, p. 287–318. External Links: Link Cited by: §4.1. [32] W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, et al. (2023) Inner monologue: embodied reasoning through planning with language models. In Conference on Robot Learning (CoRL), Cited by: §4.1. [33] Y. Mu, J. Chen, Q. Zhang, S. Chen, Q. Yu, C. Ge, R. Chen, Z. Liang, M. Hu, C. Tao, P. Sun, H. Yu, C. Yang, W. Shao, W. Wang, J. Dai, Y. Qiao, M. Ding, and P. Luo (2024) RoboCodeX: multimodal code generation for robotic behavior synthesis. In International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, Vol. 235, p. 36434–36454. External Links: Link Cited by: §4.1. [34] G. Yin, Y. Li, Y. Wang, D. McConachie, P. Shah, K. Hashimoto, H. Zhang, K. Liu, and Y. Li (2025) CodeDiffuser: attention-enhanced diffusion policy via VLM-generated code for instruction ambiguity. In Robotics: Science and Systems (RSS), Cited by: §4.1. [35] V. Kumar (2026) Act-observe-rewrite: multimodal coding agents as in-context policy learners for robot manipulation. arXiv preprint arXiv:2603.04466. Cited by: §4.1. [36] K. Elmaaroufi, J. Svegliato, S. Kalade, G. Schelle, S. A. Seshia, and M. Zaharia (2026) RHO: your coding agent is secretly a roboticist. arXiv preprint arXiv:2606.16458. Cited by: §4.1. [37] K. Chen, S. Xie, L. Fu, J. Yu, W. Pacini, S. Bajamahal, H. Kim, J. Drake, D. Kim, H. Xue, J. Francis, C. Juette, P. Schaldenbrand, M. Y. Seker, R. Wickramarachchi, U. Yoo, G. Wang, A. Murali, B. Sundaralingam, S. S. Sastry, S. Huang, Y. Zhu, L. Fan, and K. Goldberg (2026) GaP: a graph-as-policy multi-agent self-learning harness for variational automation tasks. arXiv preprint arXiv:2607.05369. Cited by: §4.1. [38] D. Naouali, M. Wu, C. Wong, A. Puthran, and O. G. Younis (2026) VLCP: vision language control policy closed-loop code replanning for robot manipulation. arXiv preprint arXiv:2608.16978. Cited by: §4.1. [39] C. Lin, J. Chen, H. Chang, H. Chen, S. Chang, C. Huang, F. Yang, M. Chen, Y. Chen, Y. F. Wang, and S. Sun (2026) PhysCaP: grounding code-as-policy agent with physics-informed exploration. arXiv preprint arXiv:2608.21031. Cited by: §4.1. [40] D. Ghosh, H. R. Walke, K. Pertsch, K. Black, O. Mees, et al. (2024) Octo: an open-source generalist robot policy. In Robotics: Science and Systems (RSS), Cited by: §4.2. [41] Z. Lei, C. Liu, Y. Xiong, M. Xiong, Y. Ding, Z. Zhang, W. Li, and S. Chen (2026) Towards long-horizon embodied agents with tool-aligned vision-language-action models. arXiv preprint arXiv:2605.13119. Cited by: §4.2. [42] K. Vo, S. Tran, T. Hanyu, Y. Ikebe, D. Nguyen, et al. (2026) CodeGraphVLP: code-as-planner meets semantic-graph state for non-markovian vision-language-action models. arXiv preprint arXiv:2604.22238. Cited by: §4.2. [43] J. Peng, X. Yu, D. Feng, Y. Yang, W. Cai, et al. (2026) Cortex: a bidirectionally aligned embodied agent framework for long-horizon manipulation. arXiv preprint arXiv:2607.05377. Cited by: §4.2. [44] S. Chen, H. Hadfield, A. Zook, M. A. Uy, C. H. Song, E. Coumans, X. Yang, F. Ladhak, Q. Qu, S. Birchfield, J. Tremblay, and V. Blukis (2026) VoLo: a physical orchestrator for open-vocabulary long-horizon manipulation. arXiv preprint arXiv:2606.07723. Cited by: §4.2. [45] S. Yoon, M. Yoo, S. Ahn, S. Choi, and H. Woo (2026) RoboBRIDGE: a modular framework for bridging policies to robust real-world robotic agents. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Cited by: §4.2. [46] Y. Chen, Z. Cao, H. Ren, C. Yang, W. Li, et al. (2026) RoboRouter: training-free policy routing for robotic manipulation. arXiv preprint arXiv:2603.07892. Cited by: §4.2. [47] X. Ren, C. Yi, and Y. Sun (2026) RouterVLA: budgeted commissioning and expert onboarding for growing VLA pools. arXiv preprint arXiv:2606.27355. Cited by: §4.2. [48] Y. Dai, A. Keyser, and D. P. Losey (2025) Prepare before you act: learning from humans to rearrange initial states. arXiv preprint arXiv:2509.18043. Cited by: §4.2. [49] M. Li, Z. Zhao, Z. Che, F. Liao, K. Wu, et al. (2025) SwitchVLA: execution-aware task switching for vision-language-action models. arXiv preprint arXiv:2506.03574. Cited by: §4.2. [50] K. Rui, Y. Zuo, J. Wang, H. Jia, J. Ma, W. Zhou, and M. Li (2026) Diagnosing semantic handoff failures in agent-orchestrated vision-language-action skill composition. In RSS SemRob Workshop, Cited by: §4.2. [51] M. Shang, Y. Huang, J. Chen, H. Hu, H. Yu, L. Song, L. Feng, S. Bao, W. Dong, X. Wang, and F. Sun (2026) BICPO-VLA: behavior-identified continuation preference optimization for smooth asynchronous vision-language-action control. arXiv preprint arXiv:2608.13924. Cited by: §4.2. [52] C. Qi, X. Wang, S. Yong, S. Sheng, H. Mao, et al. (2026) Self-refining vision language model for robotic failure detection and reasoning. In International Conference on Learning Representations (ICLR), Cited by: §4.2. [53] P. Saurabh, P. Valle, A. Arrieta, S. Ali, and P. Arcaini (2026) VISOR: a vision-language model-based test oracle for testing robots. In Proceedings of the 3rd ACM International Conference on AI-Powered Software, p. 197–207. External Links: Document Cited by: §4.2. [54] S. Chen, C. Harrison, Y. Lee, A. J. Yang, Z. Ren, et al. (2026) TOPReward: token probabilities as hidden zero-shot rewards for robotics. arXiv preprint arXiv:2602.19313. Cited by: §4.2. [55] Y. Wu, W. Yuan, A. Qi, V. Guizilini, J. Mao, and Y. Wang (2026) Large reward models: generalizable online robot reward generation with vision-language models. arXiv preprint arXiv:2603.16065. Cited by: §4.2. [56] J. Kwok, C. Agia, R. Sinha, M. Foutter, S. Li, I. Stoica, A. Mirhoseini, and M. Pavone (2025) RoboMonkey: scaling test-time sampling and verification for vision-language-action models. In Proceedings of The 9th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 305, p. 3200–3217. External Links: Link Cited by: §4.2. [57] D. Li, J. Lei, H. Wang, L. Liu, Y. Yang, et al. (2026) Learning actionable manipulation recovery via counterfactual failure synthesis. arXiv preprint arXiv:2603.13528. Cited by: §4.2. [58] C. Glossop, W. Chen, A. Bhorkar, D. Shah, and S. Levine (2026) CAST: counterfactual labels improve instruction following in vision-language-action models. IEEE Robotics and Automation Letters 11 (10), p. 11785–11792. External Links: Document Cited by: §4.2. [59] H. Kondoh, K. Ota, A. Kanezaki, and Y. Wu (2026) CounterAlign: counterfactual supervision for vision-language-action models. arXiv preprint arXiv:2608.21740. Cited by: §4.2. [60] X. Ding, L. Mi, M. Huang, Z. Wang, C. Zhang, Z. Hao, F. Chen, X. Li, Y. Zheng, Y. Guo, W. Wang, K. Li, H. Wu, Y. Liu, and T. Cao (2026) Zetta ζ: an efficient closed-loop embodied harness for self-evolving physical intelligence. arXiv preprint arXiv:2608.16590. Cited by: §4.2. [61] H. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, et al. (2023) AnyGrasp: robust and efficient grasp perception in spatial and temporal domains. IEEE Transactions on Robotics 39 (5), p. 3929–3945. External Links: Document Cited by: Table 6. [62] A. Murali, B. Sundaralingam, Y. Chao, W. Yuan, J. Yamada, M. Carlson, et al. (2026) GraspGen: a diffusion-based framework for 6-DOF grasping with on-generator training. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: Table 6. [63] X. Wu, J. Cai, J. Jiang, D. Zheng, Y. Wei, and W. Zheng (2024) An economic framework for 6-DoF grasp detection. In European Conference on Computer Vision (ECCV), p. 357–375. External Links: Document Cited by: Table 6. [64] S. Chen, P. Xie, W. Tang, D. Hu, Y. Dai, and G. Wang (2025) Region-aware grasp framework with normalized grasp space for efficient 6-DoF grasping. In Proceedings of The 8th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 270, p. 1834–1850. External Links: Link Cited by: Table 6. [65] J. Yang, J. J. Liu, Y. Li, Y. Khaky, K. Shaw, and D. Pathak (2025) Deep reactive policy: learning reactive manipulator motion planning for dynamic environments. In Proceedings of The 9th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 305, p. 5056–5071. External Links: Link Cited by: Table 7. [66] W. Thomason, Z. Kingston, and L. E. Kavraki (2024) Motions in microseconds via vectorized sampling-based planning. In IEEE International Conference on Robotics and Automation (ICRA), p. 8749–8756. External Links: Document Cited by: Table 7. [67] A. Fishman, A. Walsman, M. Bhardwaj, W. Yuan, B. Sundaralingam, B. Boots, and D. Fox (2025) Avoid everything: model-free collision avoidance with expert-guided fine-tuning. In Proceedings of The 8th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 270, p. 1925–1948. External Links: Link Cited by: Table 7. [68] M. J. Kim, C. Finn, and P. Liang (2025) Fine-tuning vision-language-action models: optimizing speed and success. In Robotics: Science and Systems (RSS), External Links: Document, Link Cited by: Table 8. [69] Y. Zhang, J. Zhao, C. Fan, F. Yan, T. Li, H. Tang, et al. (2026) PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations. arXiv preprint arXiv:2604.27472. External Links: Link Cited by: Table 8. [70] S. Liu, B. Li, K. Ma, L. Wu, H. Tan, X. Ouyang, et al. (2026) RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalization. arXiv preprint arXiv:2602.03310. External Links: Link Cited by: Table 8. [71] D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, et al. (2025) SpatialVLA: exploring spatial representations for visual-language-action models. In Robotics: Science and Systems (RSS), External Links: Document, Link Cited by: Table 8. [72] StarVLA Community (2026) StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing. arXiv preprint arXiv:2604.05014. External Links: Link Cited by: Table 8. [73] J. Cen, S. Huang, Y. Yuan, K. Li, H. Yuan, C. Yu, et al. (2025) RynnVLA-002: A Unified Vision-Language-Action and World Model. arXiv preprint arXiv:2511.17502. External Links: Link Cited by: Table 8. [74] T. Lin, Y. Zhong, Y. Du, J. Zhang, J. Liu, Y. Chen, et al. (2026) Evo-1: lightweight vision-language-action model with preserved semantic alignment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13397–13406. External Links: Link Cited by: Table 8. [75] Y. Yang, S. Zeng, T. Lin, X. Chang, D. Qi, J. Xiao, et al. (2026) ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning. arXiv preprint arXiv:2602.11236. External Links: Link Cited by: Table 8. [76] H. Ma, J. Cai, X. Xu, H. Li, Y. Yang, Y. Tian, et al. (2026) InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization. arXiv preprint arXiv:2607.04988. External Links: Link Cited by: Table 8. [77] M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, et al. (2025) SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics. arXiv preprint arXiv:2506.01844. External Links: Link Cited by: Table 8. [78] Y. Yang, Z. Liu, S. Kou, Y. Chen, Y. Hu, J. Zhou, et al. (2026) World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis. arXiv preprint arXiv:2606.05979. External Links: Link Cited by: Table 8. [79] J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y. Feng, et al. (2026) X-VLA: soft-prompted transformer as scalable cross-embodiment vision-language-action model. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: Table 8. [80] H. Fang, J. Duan, D. Clay, S. Wang, S. Liu, W. Huang, et al. (2026) MolmoAct2: Action Reasoning Models for Real-world Deployment. arXiv preprint arXiv:2605.02881. External Links: Link Cited by: Table 8. [81] L. Li, Q. Zhang, Y. Luo, S. Yang, R. Wang, F. Han, et al. (2026) Causal world modeling for robot control. In Robotics: Science and Systems (RSS), External Links: Document, Link Cited by: Table 8. [82] S. Srivastava, C. Li, M. Lingelbach, R. Martín-Martín, F. Xia, et al. (2021) BEHAVIOR: benchmark for everyday household activities in virtual, interactive, and ecological environments. In Conference on Robot Learning (CoRL), Cited by: §C.1. [83] S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, et al. (2024) RoboCasa: large-scale simulation of household tasks for generalist robots. In Robotics: Science and Systems (RSS), Cited by: §C.1. [84] C. Yu, Y. Wang, Z. Guo, H. Lin, S. Xu, H. Zang, Q. Zhang, Y. Wu, C. Zhu, J. Hu, Z. Huang, M. Wei, Y. Xie, K. Yang, B. Dai, Z. Xu, J. Du, X. Wang, X. Fu, L. Shi, Z. Liu, K. Chen, W. Liu, G. Liu, B. Li, J. Yang, Z. Yang, G. Dai, and Y. Wang (2026) RLinf: flexible and efficient large-scale reinforcement learning via macro-to-micro flow transformation. In 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI), p. 829–846. External Links: Link Cited by: §C.2. Appendix Contents References A Methodological Details B API Library Instantiation C Experiment Details D Additional Empirical Results Appendix A Methodological Details A.1 P5P^5 Stage Contracts Each P5P^5 stage declares an entry predicate, the typed inputs it requires, an admissible-family set i(zi,gi)C_i(z_i,g_i), the APIs a family-constrained code block may call, a success postcondition, and the failure or timeout feedback that chooses the successor stage. Table 1 summarizes these contracts; typed API I/O appears in Table 4. The coordinator selects a family only when the coding agent has control and the current stage’s entry predicate holds. Once an atomic API call begins, that selection remains fixed until the call returns after satisfying its registered postcondition, reporting a failure, or exhausting its step budget. Table 1: P5P^5 stage contracts and admissible families. At stage ziz_i of skill gig_i, the coordinator selects ci∈i(zi,gi)c_i _i(z_i,g_i) before code generation; APIs must match (zi,ci)(z_i,c_i) (Table 2 in the main text). Columns: ϕinφ^in—entry predicate; iC_i—admissible families; ϕoutφ^out—registered success postcondition; Fail—successor after failure/timeout (retry ziz_i, return upstream ↑ , or terminate ⊥ ). Under ModMod, ϕoutφ^out is the typed modular return (e.g. pose at perceive, eef pose at propose); under E2EE2E, it is covered by the atomic call postcondition. propose is Mod-only; an E2E call may complete the skill without visiting it. ziz_i ϕinφ^in iC_i ϕoutφ^out Fail perceive target ID/pose ∉hi∉ h_i Mod,E2E\Mod,E2E\ typed target info retry / ⊥ propose pose/mask ∈hi∈ h_i ∧ eef target ∉hi∉ h_i Mod\Mod\ valid eef pose retry / ↑ / ⊥ pre-manipulate approach target available Mod,E2E\Mod,E2E\ pre-contact reached replan / ↑ / ⊥ perform pre-manip. ϕoutφ^out Mod,E2E\Mod,E2E\ contact/gripper postcond. retry / → -manip. / ⊥ post-manipulate perform ϕoutφ^out Mod,E2E\Mod,E2E\ retreat/release/stabilize retry / ↑ / ⊥ Required inputs and permitted APIs. Under ModMod, the permitted API depends on the current stage: Perceive_object_pose at perceive, Propose_eef_pose at propose, Plan_to_eef_pose at pre-manipulate and post-manipulate, and Perform_gripper at perform. A modular code block may compose only APIs admissible for the current ziz_i (and any immediately chained modular helpers that do not cross a new responsibility boundary). Under E2EE2E, the sole permitted pattern is Perform_E2E(route_id,…), whose declared coverage is perceive, pre-manipulate, perform, and post-manipulate (Table 2 in the main text). After the coordinator selects E2EE2E, the coding agent supplies route_id and the required typed observations in aia_i. Thus, QθQ_θ selects the family, while the coding agent selects the backend within that family. Why propose admits only ModMod. Propose_eef_pose is the only API assigned to propose, where it produces the end-effector target required by modular motion planning. End-to-end backends determine grasp and approach targets internally and do not expose them through this interface, so Perform_E2E does not cover propose. Consequently i=ModC_i=\Mod\ whenever ziz_i is propose, so no E2E comparison is issued at that boundary. A skill enters propose only when it requires an explicit end-effector target. An E2E call beginning at perceive or a later admissible stage can therefore complete the skill without entering propose. Multi-stage atomic returns and stage advancement. A single Perform_E2E call may satisfy several consecutive stage postconditions before returning, but its family assignment remains fixed throughout the call. When control returns, the coding agent updates (hi,zi,gi)(h_i,z_i,g_i) from observable feedback only: typed returns, registered postcondition bits, failure codes, and timeout flags. The next decision stage zi+1z_i+1 is the earliest stage in the active skill whose entry predicate still holds and whose success postcondition is unmet. Stages whose postconditions have already been satisfied are skipped, so the coordinator makes no additional family selections for them unless execution later reaches an entry where those postconditions are unmet. If the call fails or times out after partial progress, zi+1z_i+1 is likewise the earliest unmet stage that remains enterable under the returned feedback (often the stage at which the postcondition failed), enabling retry or upstream fallback as in Table 1. If every stage postcondition of gig_i is met, the skill completes and the plan advances to the next schema in G. If execution must continue but i+1C_i+1 is empty, the episode terminates with failure, as specified in Algorithm 1. A.2 RoboAware: Training and Deployment Algorithm 1 RoboAware: MCTS data collection, Q-learning, and deployment Input: training tasks trainT_train; test instruction ℓtest _test; simulator ℰE; πcode _code and API library; budget K Output: trained coordinator QθQ_θ; test execution history h and terminal reward r 1 Freeze πcode _code and the API library; initialize θ, θ¯←θ θ←θ, and SCB←∅D_SCB← ; // Phase 1. Offline training: prepare search roots 2 Collect roots ℛR from training-task P5P^5 executions using the neighboring-state sampling protocol described below, with matching histories; 3 foreach sampled root ξt∈ℛ _t do 4 Initialize tree T at ξt _t and remaining budget b←Kb← K; 5 while b>0b>0 do // 1. Selection 6 Follow stored paths by UCB (Equation 1) to a nonterminal node j; select a new node only if b≥|j|b≥|C_j|, otherwise reevaluate an expanded node; // 2. Expansion 7 if node j is new then 8 Set Nj=Nj(c)=0N_j=N_j(c)=0, Q^j(c)=Qθ(xj,c) Q_j(c)=Q_θ(x_j,c), and Aj←jA_j _j; 9 else 10 Aj←cjtreeA_j←\c_j^tree\ by Equation 1; 11 foreach c∈Ajc∈ A_j do 12 Restore ξj _j and its history; generate ajc∼πcode(⋅∣xj,c)a_j^c _code(· x_j,c) and execute to the next boundary or termination; 13 Record (xj+1c,ξj+1c,djc)(x_j+1^c, _j+1^c,d_j^c) and compute r~jc r_j^c from the skill/task oracle scores; 14 Attach the successor to T; append (xj,c,r~jc,xj+1c,djc,ξj+1c)(x_j,c, r_j^c,x_j+1^c,d_j^c, _j+1^c) to SCBD_SCB; // 3. Rollout 15 Continue a nonterminal successor to a stopping condition without tree expansion; obtain Gj+1G_j+1, or set Gj+1=0G_j+1=0 if djc=1d_j^c=1; 16 Gj←r~jc+γGj+1G_j← r_j^c+γ G_j+1; // 4. Backpropagation 17 Back up GjG_j and prefix rewards along the selected path; increment visits, average returns into Q Q, and set b←b−1b← b-1; // Q-learning after data collection 18 Fit QθQ_θ on all of SCBD_SCB for M minibatch updates using Equations 4 and 5, learning rate η, and target refresh interval J; // Phase 2. Deployment with fixed parameters 19 Observe o0o_0; generate test plan G from (ℓtest,o0)( _test,o_0); initialize h0←∅h_0← , (z0,g0)(z_0,g_0), and i←0i← 0; 20 while the task is nonterminal and the episode horizon remains do 21 Build xi=(ℓtest,oi,hi,zi,gi)x_i=( _test,o_i,h_i,z_i,g_i) and admissible families i(zi,gi)C_i(z_i,g_i); 22 if i=∅C_i= then Terminate with failure and exit the loop; 23 ci⋆←argmaxc∈iρθ(c∣xi)c_i ← _c _i _θ(c x_i); generate ai∼πcode(⋅∣xi,ci⋆)a_i _code(· x_i,c_i ); 24 Execute aia_i to the next boundary or termination, respecting atomic APIs; 25 Observe feedback; update xi+1x_i+1 and G; i←i+1i← i+1; 26 return success indicator: terminal task reward r Algorithm 1 separates offline training from deployment. During training, we collect P5P^5 search roots, run MCTS from those roots, and fit QθQ_θ on the resulting transitions. During deployment, the trained coordinator selects a family at each eligible stage boundary and updates its input from execution feedback. Only QθQ_θ is optimized. The coding agent πcode _code and API library are fixed before data collection and remain frozen during training and deployment. Trajectory collection and neighboring-state sampling. For each training task, πcode _code constructs a task plan G comprising P5P^5 skill schemas, with gig_i identifying the active skill and ziz_i its current stage. A fixed sampling rule with support on all admissible families selects the family before each code block is generated. During the resulting simulator rollout τ, we record observations, execution history, schema states, and saved simulator states, together with the start time tit_i of each visited stage. For each tit_i, let i(Δ)W_i( ) contain the nonterminal saved times within Δ steps of tit_i at which the same stage-entry conditions hold and control is available to the coding agent. It includes tit_i and excludes intermediate states inside an atomic API invocation. We draw S times independently and uniformly from this set, with replacement, and form a separate root snapshot for each draw. Each root uses its own matching context xt=(ℓ,ot,ht,zt,gt)x_t=( ,o_t,h_t,z_t,g_t). The snapshot, denoted by ξt=(xt,mt,pt,r) _t=(x_t,m_t,p_t,r), also stores the simulator and execution state needed to restore that root exactly. Here, r denotes access to the task-success evaluator rather than a future reward available to the coordinator. Thus, neighboring roots may differ physically, but all counterfactual branches of one root share the same physical state and observable-history prefix. Search and learning contracts. Each root receives a budget of K completed suffix simulations, including the initial family comparisons. UCB descent follows stored tree paths using Equation 1, stopping before terminal nodes. At a new node, counts start at zero and Q^i(c) Q_i(c) is initialized from Qθ(xi,c)Q_θ(x_i,c), whose parameters remain fixed during data collection. Descent stops at an unexpanded nonterminal node when the remaining budget covers every admissible family. Otherwise, it stops at an expanded nonterminal node and selects one family for reevaluation. Every evaluation restores the selected snapshot, samples a fresh family-constrained code block, executes it to the next semantic boundary, and saves the resulting transition and successor snapshot. From each nonterminal successor, the fixed coding agent and API library continue physics execution under the same family-sampling continuation rule to task termination, unrecoverable failure, the episode horizon, or an empty admissible-family set. Reaching any of these stopping conditions sets d=1d=1 and ends the suffix. States visited only during rollout are not added to the search tree. The suffix return is backed up along the selected path using the recorded prefix rewards, with visit counts incremented and Q Q updated by empirical averaging. All expanded edges enter SCBD_SCB, including failed and nonpreferred branches. Scores rpr_p and rmr_m are emitted at skill completion and are zero at other transitions, while r is emitted only at task termination. Q-learning then uses Equations 4 and 5, with no bootstrap at terminal states and a target copy refreshed every J updates. The sampling parameters (S,Δ)(S, ), family-sampling rule, optimization budget M, learning rate η, target-update interval J, and related budgets are listed in Table 12. Deployment contract. Only the trained QθQ_θ, frozen coding agent, and API library are retained. At every decision point, the coordinator builds xix_i from observable information, selects an admissible family, and conditions code generation on that choice. Each atomic API call runs until its postcondition is satisfied, a failure is reported, or its step budget is exhausted. Once the call returns, the coding agent uses the feedback to update the history and remaining plan and to determine whether to advance or retry a stage. Task termination, an unrecoverable failure, or the episode horizon ends execution. An empty admissible-family set is treated as failure. Neither state restoration, tree search, oracle shaping rewards, nor parameter updates are used at deployment. Only the terminal task reward r is reported. Training data and task pool. We construct SCBD_SCB using the procedure in Algorithm 1 and the model in Section A.4, with the sampling and optimization settings given below. Data collection with privileged simulator access and subsequent QθQ_θ updates use only scene seed 00 on the coordinator training pool (8080 LIBERO-Pro, 44 RoboSuite, and 1616 RoboTwin 2.0 tasks; Section C.1). Each training task contributes (i) one or more physics trajectories used to mark P5P^5 stage-entry times, (i) S neighboring roots per stage entry drawn from i(Δ)W_i( ) as specified above, and (i) up to K completed suffix simulations per root, with every expanded family edge written into SCBD_SCB. Root and family sampling. During trajectory collection, we sample uniformly from the families admissible at the current (zi,gi)(z_i,g_i), so every family in iC_i has nonzero probability. Neighboring-state sampling draws S=4S=4 roots independently and uniformly with replacement from i(Δ)W_i( ) using the default window Δ=10 =10 saved control steps. Within each root budget of K=20K=20 completed suffix simulations (Table 12), the first expansion of a new node evaluates every c∈ic _i once via SCB; later visits reevaluate a single UCB-selected family (Equation 1). Rollout continuation after expansion reuses the same family-sampling rule and does not add rollout-only states as tree nodes. All evaluated edges, including failed and nonpreferred branches, remain in SCBD_SCB. Q initialization before data collection. Before data collection, we load the released Qwen3.5-0.8B backbone weights, randomly initialize the <VALUE> embedding and MLP head, and set θ¯←θ θ←θ (Algorithm 1). We then hold θ fixed throughout collection. Each new tree node starts with zero visits and initializes Q^i(c) Q_i(c) from Qθ(xi,c)Q_θ(x_i,c), which provides the initial value estimate for UCB selection. No return regression or supervised fine-tuning is performed before collection, and the model receives no privileged oracle features. Bellman updates begin only after SCBD_SCB is complete, following the single collection-and-fitting cycle used in the reported experiments. Optimization protocol. After collection, we optimize θ on the fixed offline set with the Smooth-L1L_1 Bellman loss in Equations 4 and 5, minibatches drawn from SCBD_SCB, a stop-gradient target network Qθ¯Q_ θ, and no bootstrap at terminal states. Table 2 lists the fine-tuning method, thinking setting, optimizer, batch size, learning-rate schedule (cosine with warmup), update budget M, target refresh interval J, numeric precision, and checkpoint rule. We select a checkpoint on the validation split described below, following the checkpoint rule in Table 2, and freeze it for all evaluations on seeds 11–1010. Table 2: Coordinator Q-fitting settings. Symbols match Algorithm 1. Setting Value Fine-tuning method full-FT (backbone + MLP value head) Thinking Disabled Optimizer AdamW Minibatch size 1616 Learning rate η / schedule 10−510^-5, cosine, 5%5\% warmup Gradient updates M 10,00010,000 Target refresh every J updates 500500 (hard copy θ¯←θ θ←θ) Weight decay / grad clip 10−410^-4 / 1.01.0 Numeric precision bf16 Checkpoint selection lowest validation Bellman loss Grouped training and validation split. The ModMod and E2EE2E branches from a given root share both a physical state and an observable-history prefix. Roots sampled from the same neighborhood i(Δ)W_i( ) are also closely related because they come from nearby states in one trajectory. A random split of individual transitions could therefore place branches from the same root, or transitions from neighboring roots, in both training and validation. To keep these related samples together, we group all transitions from each root and all roots drawn from the same stage-entry neighborhood i(Δ)W_i( ). Groups are assigned entirely to training or validation using a 90/1090/10 split, optionally stratified by benchmark and by whether iC_i is singleton or binary. No group contributes transitions to both splits; evaluation seeds 11–1010 remain unused for fitting or model selection. A.3 Responsibility Preference The normalization in Equation 6 defines a relative preference over the nonempty admissible-family set iC_i. When i=E2E,ModC_i=\E2E,Mod\, the value gap compares the same ordered pair at every decision point: Δθ(xi)=Qθ(xi,E2E)−Qθ(xi,Mod). _θ(x_i)=Q_θ(x_i,E2E)-Q_θ(x_i,Mod). (7) Thus, Δθ(xi) _θ(x_i) depends on the context, while the family identities and their order are fixed by definition. The normalized preferences can then be expressed as sigmoids of this gap. Dividing the E2E numerator and denominator by exp(Qθ(xi,E2E)) (Q_θ(x_i,E2E)) gives ρθ(E2E∣xi) _θ(E2E x_i) =exp(Qθ(xi,E2E))exp(Qθ(xi,E2E))+exp(Qθ(xi,Mod)) = (Q_θ(x_i,E2E)) (Q_θ(x_i,E2E))+ (Q_θ(x_i,Mod)) (8) =11+exp(Qθ(xi,Mod)−Qθ(xi,E2E)) = 11+ (Q_θ(x_i,Mod)-Q_θ(x_i,E2E)) =σ(Δθ(xi)), =σ( _θ(x_i)), ρθ(Mod∣xi) _θ(Mod x_i) =1−ρθ(E2E∣xi)=σ(−Δθ(xi)). =1- _θ(E2E x_i)=σ(- _θ(x_i)). The common normalization cancels in the ratio, so taking its logarithm yields logρθ(E2E∣xi)ρθ(Mod∣xi) _θ(E2E x_i) _θ(Mod x_i) =logexp(Qθ(xi,E2E))exp(Qθ(xi,Mod)) = (Q_θ(x_i,E2E)) (Q_θ(x_i,Mod)) (9) =Qθ(xi,E2E)−Qθ(xi,Mod)=Δθ(xi). =Q_θ(x_i,E2E)-Q_θ(x_i,Mod)= _θ(x_i). Positive gaps favor E2E, negative gaps favor modular composition, and a zero gap gives equal preference to both families. The larger preference equals σ(|Δθ(xi)|)σ(| _θ(x_i)|) and increases monotonically with |Δθ(xi)|| _θ(x_i)|, which therefore measures preference strength. More generally, exponentiation is strictly increasing and the denominator in Equation 6 is positive and shared by all admissible families. Consequently, the transformation preserves family ordering, including ties, and yields the same maximizing families: argmaxc∈iρθ(c∣xi)=argmaxc∈iQθ(xi,c). _c _i _θ(c x_i)= _c _iQ_θ(x_i,c). (10) When only one family is admissible, its normalized preference is one; the binary sigmoid identities apply only when both families are admissible. An empty admissible-family set terminates execution with failure, as specified in the deployment phase of Algorithm 1. The deployment algorithm therefore selects via ρθ _θ (equivalently via QθQ_θ), then generates and executes the corresponding block before reconsidering responsibility at the next semantic boundary. A.4 Q-Value Model We instantiate Qθ(xi,c)Q_θ(x_i,c) with Qwen3.5-0.8B as the multimodal backbone and a scalar MLP value head. Only this coordinator value model is optimized under Equation 4; the coding agent πcode _code, the typed API library, and all low-level backend checkpoints remain frozen throughout EAL and deployment. Observation encoding. At decision point i, the visual stream uses the same agent-view camera as the modular perception API. We resize the agent-view RGB frame IirgbI_i^rgb and a visualization of the metric depth map IidI_i^d to a shared resolution R×R×R (Table 3) and encode both through the backbone’s native image pathway. We serialize proprioception qiq_i, including joint positions, end-effector pose, and gripper opening, as structured text. Thus, all components of oio_i enter QθQ_θ through its image and text pathways, without a separate numeric depth input, depth adapter, or proprioceptive MLP encoder. History length and truncation. The execution history hih_i retains the most recent H stage transitions in full, each recording the chosen family, typed API returns or failure codes, stage advances or retries, and remaining budgets. Older transitions are dropped and replaced by a fixed-length summary of completed skills/stages, persistent object bindings, cumulative failure counts, and the residual step/wall-clock budget. The summarizer uses only information already present in the observable history: training-time oracle fields never enter hih_i. Token order. For a queried family c, inputs are packed into one multimodal sequence: ℓ→(Iirgb,Iid)→qi→hi→gi→zi→c→<VALUE>, \;→\;(I_i^rgb,I_i^d)\;→\;q_i\;→\;h_i\;→\;g_i\;→\;z_i\;→\;c\;→\; <VALUE>, i.e., instruction, images, proprioception, execution history, current skill, current P5P^5 stage, family identity, then the special <VALUE> token. For example, consider RoboTwin place_bread_basket at the start of the perform stage for picking up the bread. The instruction ℓ is “Place the bread into the basket,” and the image inputs show the current R×R×R RGB frame and depth colormap of the bread, basket, and two arms. The proprioceptive input qiq_i lists joint positions, end-effector poses, and gripper openings. The history hih_i records the preceding stage transitions. These may include ModMod returning the bread pose at perceive, producing an end-effector target at propose, and reaching the pre-grasp pose at pre-manipulate. It also contains the remaining step and wall-clock budgets and a fixed-length summary of earlier completed skills. The active schema gig_i identifies the bread-picking skill and its unmet postconditions, while zi=performz_i=perform. Both families are admissible at this point, so the sequence ends with the queried c∈Mod,E2Ec∈\Mod,E2E\ followed by <VALUE>. The backbone produces the final-layer hidden state at <VALUE>; an LmlpL_mlp-layer MLP with hidden width dmlpd_mlp maps that state to the scalar Qθ(xi,c)Q_θ(x_i,c). Table 3 lists the quantitative settings used in our experiments. Table 3: Q-value model settings. Setting Value Backbone Qwen3.5-0.8B Image viewpoint agent-view RGB + depth visualization Image resolution R×R×R 224×224224×224 Proprioception encoding structured text Depth encoding colormap image History window H (full transitions) 44 MLP depth LmlpL_mlp 22 MLP width dmlpd_mlp 512512 Frozen parameters πcode _code, API library, backend checkpoints A.5 Reward Function This subsection specifies the training-only oracle rewards in Equation 3. The coordinator never observes mim_i, pip_i, rpr_p, rmr_m, or r at deployment. As in Section A.2, rp,icr_p,i^c and rm,icr_m,i^c are emitted when a P5P^5 skill completes under family c and are zero on non-completion transitions; the binary task reward r is emitted only when the episode terminates (dic=1d_i^c=1). Perception score rpr_p. Let mim_i be the oracle binary mask of the language-referenced target object in the agent-view image at the skill-entry snapshot ξi _i. When the executed block exposes an explicit mask m^ic m_i^c (modular Perceive_object_pose), mask agreement is the intersection-over-union rp,ic=IoU(m^ic,mi)=|m^ic∩mi||m^ic∪mi|∈[0,1],r_p,i^c=IoU( m_i^c,m_i)= | m_i^c∩ m_i|| m_i^c∪ m_i|∈[0,1], (11) with the convention IoU=0IoU=0 if both masks are empty. End-to-end blocks do not return a mask. For those branches we score target-object identity rather than pixel overlap: after execution, the simulator identifies the object instance that was contacted or whose pose changed under the skill; rp,ic=1r_p,i^c=1 if that instance matches the object labeled by mim_i, and 00 otherwise. Thus both families receive a perception score in [0,1][0,1], but only modular perception is judged by mask IoU; E2E perception is judged by whether the call acted on the oracle-referenced object, consistent with the E2E postcondition coverage in Table 1. Manipulation score rmr_m. The manipulation score measures whether the object’s simulator ground-truth pose after skill execution meets the skill’s geometric goal. Write pi+1cp_i+1^c for the privileged 6-DoF object pose read from ξi+1c _i+1^c, and pi⋆p_i for the skill goal pose supplied by the same oracle pose channel (placement target, handover target, or other stage goal recorded with the task). Let t(⋅)t(·) and R(⋅)R(·) denote translation and rotation. We measure translation error in meters using the Euclidean distance ‖t(pi+1c)−t(pi⋆)‖2\|t(p_i+1^c)-t(p_i )\|_2, and rotation error in radians using the geodesic angle on SO(3)SO(3), ∠(R(pi+1c),R(pi⋆))=arccos((tr(R(pi+1c)⊤R(pi⋆))−1)/2) \! (R(p_i+1^c),R(p_i ) )= ((tr(R(p_i+1^c) R(p_i ))-1)/2 ). For skills defined by a contact or grasp postcondition, such as a successful lift, rm,icr_m,i^c is the corresponding binary oracle predicate. For skills with a 6-DoF placement goal, we instead apply the pose thresholds rm,ic=[∥t(pi+1c)−t(pi⋆)∥2≤ϵp∧∠(R(pi+1c),R(pi⋆))≤ϵR],r_m,i^c=I\! [\|t(p_i+1^c)-t(p_i )\|_2≤ _p\; \; \! (R(p_i+1^c),R(p_i ) )≤ _R ], (12) where ϵp _p is the translational threshold in meters and ϵR _R is the rotational threshold in radians (Table 12). The main-text phrase “object-pose error against pip_i” refers to this privileged pose channel used to form (pi+1c,pi⋆)(p_i+1^c,p_i ), not to scoring the modular pose estimate against the entry state. Terminal reward and stopping. The evaluator returns r=1r=1 iff the task success predicate holds at termination, and r=0r=0 otherwise. An episode sets dic=1d_i^c=1 and stops on task success, unrecoverable failure, empty admissible-family set, or the episode horizon; only the success case can yield r=1r=1. The shaped transition reward remains r~ic=wprp,ic+wmrm,ic+wtdicr r_i^c=w_pr_p,i^c+w_mr_m,i^c+w_td_i^cr with wp+wm+wt=1w_p+w_m+w_t=1. Appendix B API Library Instantiation B.1 Typed API Contracts Each embodied API in Table 2 in the main text owns one or more P5P^5 stages and declares typed inputs, outputs, coordinate frames, postconditions, and failure states (Table 4). The family-specific rules in Table 5 determine which APIs a generated code block may call. The coding agent specifies the semantic target and route, whereas embodiment adapters resolve frames, execute the request, and report measured completion. This separates an incorrect target from a failure to reach a correct target and prevents an internal fallback from silently changing the meaning of a call. Within E2EA_E2E, route_id selects a retained VLA, video-action, or world-action checkpoint inside the code block; it is not a separate action scored by QθQ_θ. The coordinator observes only typed API returns, not privileged simulator state. We freeze the coding agent, API contracts, backend bindings, and checkpoints before coordinator training. Table 4: Typed input/output contracts for the high-level embodied APIs in Table 2 in the main text. The E2E route uses route_id to select one frozen backend. API Stage Input Output Perceive_object_pose perceive task instruction + agent-view RGB-D + calibration target pose + mask/geometry provenance Propose_eef_pose propose RGB-D geometry + target mask + robot state + task context ranked end-effector target + screening provenance Plan_to_eef_pose pre/post-manipulate scene observation, robot state, target pose, optional attachment continuous, smooth, collision-aware qpos sequence Perform_gripper perform gripper open/close command updated gripper state Perform_E2E perceive, pre-manipulate, perform, post-manipulate route ID + task instruction + RGB-D + proprioception closed-loop control until postcondition, failure, or budget Table 5: Policy-family bindings for generated code blocks. Mod composes stage-admissible APIs; E2E invokes one frozen backend through route_id. Policy family Code-block space Permitted API pattern Backend choice ModMod ModA_Mod stage-admissible modular API composition fixed selected modular backends E2EE2E E2EA_E2E Perform_E2E(route_id,…)( route\_id,…) retained frozen backend selected inside aia_i B.2 Object Perception Perceive_object_pose grounds a language reference in an agent-view RGB-D observation and combines the resulting mask with camera calibration and depth to recover a robot-frame target. Valid masked depth pixels are back-projected with the camera intrinsics, transformed through the camera extrinsics, filtered for invalid or isolated points, and summarized by a robust centroid and oriented extent. The API returns the pose together with the mask, supporting points, confidence, and source observation. If the mask is empty, the depth is inconsistent, or the reference is ambiguous, it reports a failure instead of returning a pose. We use InstructSAM [17] as the grounding backend because its instruction-conditioned interface directly accepts the detailed free-form prompt produced by the coding agent from the current task context. The agent can therefore preserve task-specific attributes and spatial relations in a single perception call. The resulting mask is lifted through the shared geometric pipeline, so downstream proposal and motion APIs retain the same typed outputs and failure checks. B.3 Pose Proposal Propose_eef_pose consumes the frozen RGB-D observation, target mask, and derived geometry and returns a grasp target together with its score, rank, predicted width, coordinate transform, screening result, and provenance. We compare proposal methods using the same cuRoboV2 downstream planner and retain each baseline’s original candidate ranking. With the planner held fixed, Success@k in Table 6 reflects differences in candidate coverage and ordering. This API owns grasp proposal, not placement-pose generation. For placement, the task supplies a target object pose; the motion API resolves it to a TCP target through the held-object transform and ranks routes by feasibility, continuity, and the declared approach or contact constraints. Our NeuGraspNet+ backend addresses two failure modes of native NeuGraspNet [18]. When native geometric sampling produces too few candidates, a RobustSampler derives deterministic top-down, side, and oblique seeds from the target point cloud and its principal directions. The released NeuGraspNet checkpoint still predicts their quality and gripper width, so the fallback expands coverage without another grasp network or task-specific proposals. NeuGraspNet+ then uses cuRoboV2 to identify up to five candidates with feasible approach, grasp, and lift trajectories, and selects the top-ranked grasp from this feasible set. Its confidence gate preserves the network ranking when scores are informative, uses geometry to break near-ties, and gives geometry greater weight when all feasible scores are low. This avoids globally replacing a useful learned ranking with a fixed geometric prior. NeuGraspNet+ reaches 18/1818/18 Success@1, Success@3, and Success@5 on the 18-task Panda screen (Table 6) and is frozen as the grasp backend. These selection rules prioritize first-choice grasp success over speed and do not involve retraining NeuGraspNet. Because they were developed using the benchmark, their performance on new object distributions requires further validation. Table 6: Grasp proposal results on the 18-task Panda benchmark. All baseline methods use their native proposal rankings; NeuGraspNet uses our final selection pipeline. All methods share the same downstream cuRoboV2 motion planner. Success@k indicates whether a successful grasp is found within the first k evaluated proposals. Proposal and screening times are averaged over all 18 tasks; Time@1 is averaged over successful Top-1 trials. Method Success@1 ↑ Success@3 ↑ Success@5 ↑ Proposal time (s) ↓ Screening time (s) ↓ Time@1 (s) ↓ AnyGrasp [61] 13/18 16/18 17/18 7.98 3.46 36.39 GraspGen [62] 11/18 12/18 12/18 4.54 3.16 24.84 NeuGraspNet [18] 12/18 13/18 13/18 12.90 1.56 39.05 EconomicGrasp [63] 10/18 12/18 12/18 5.03 1.11 20.68 RNGNet [64] 7/18 8/18 8/18 18.89 1.34 46.33 NeuGraspNet+ (ours) 18/18 18/18 18/18 14.36 2.23 49.04 B.4 Motion Planning Plan_to_eef_pose converts a caller-specified end-effector or held-object target into the robot’s command frame, executes a trajectory, and verifies arrival. The fixed-grasp protocol in Table 7 supplies every planner with the same target and therefore isolates planning, whereas planner-native results measure the complete native route. We retain cuRoboV2 [19] as the core global planner and trajectory checker; it is also shared by the proposal comparison in Table 6. For each checked call, the API freezes the current RGB-D frame, robot state, camera calibration, and target mask. It filters robot-image residuals and excludes the grasp target where contact is intended. Closing a successful grasp creates a held-object state containing the measured hand–object transform and attachment geometry; later paths check both robot and attachment against the environment. With target_frame=‘tcp’, the public TCP goal is converted to the native hand frame. With target_frame=‘held_object’, the saved transform determines the hand goal that realizes a desired object pose. Explicit motion modes. goto_pose provides three motion modes with different planning costs and collision checks. The default auto mode uses cuRoboV2 first, collision-checked multi-seed IK second, and the existing bounded recovery route last; it is appropriate for grasping, long transport, and cluttered motion. ik_checked skips global planning but captures the scene and collision-checks dense IK branches, including the attachment, for short contact-free adjustments. ik_unchecked skips scene capture and scene-collision checking for deliberate contact such as staged insertion, drawer closing, or door pushing, while retaining multi-seed IK, strict joint-continuity limits, dense execution, tracking, and arrival checks. The “unchecked” mode therefore omits scene-collision checks while retaining these control and execution checks. The mode is selected before execution so that intended contact does not consume the budget in repeated collision-avoidance attempts. This also ensures that a call requested with collision checking cannot silently switch to unchecked execution. For example, a microwave program can use auto for grasp and transport, ik_checked for final free-space alignment, and ik_unchecked for insertion or a door-pushing arc. Approach and execution acceptance. z_approach adds a waypoint along the declared tool frame. For held-object placement, contact_target marks only a bounded terminal segment as expected contact, leaving the preceding transport checked. Before returning, every route monitors joint tracking and verifies joint and end-effector arrival. The typed result records the resolved frame, motion mode, trajectory source, performed checks, timing, residuals, and a specific failure category, allowing history hih_i to distinguish planning, execution, and semantic-target failures. Table 7: Motion-planning results on the 18-task Panda benchmark. The planner-native fresh protocol is the main end-to-end comparison: each planner independently resolves endpoints, screens proposals, plans, and executes. Pipeline time includes candidate screening, endpoint resolution, and planning; Plan time measures the final selected trajectory only. The fixed-grasp protocol uses a shared grasp target. The conditional diagnostic uses cuRobo-success anchors and should not be ranked directly against the planner-native results. Avoid Everything was evaluated only in the conditional pose-goal diagnostic. Planner-native fresh (main) Fixed grasp Conditional diagnostic Planner Plannable ↑ Task success ↑ Pipeline / Plan (s) ↓ Task success ↑ Plan (s) ↓ Grasp success ↑ Plan (s) ↓ cuRoboV2 [19] 18/18 18/18 3.82 / 0.254 15/18 0.192 18/18 0.935 DRP/IMPACT [65] 16/18 14/18 6.23 / 0.418 11/18 0.576 16/18 1.526 VAMP [66] 12/18 11/18 73.30 / 9.320 6/18 0.931 14/18 1.598 Avoid Everything [67] – – – – – 2/18 3.677 B.5 Gripper Control Perform_gripper implements binary open or close actions. A command returns only after the observed opening or joint signal remains stable for a fixed dwell window; failure to settle within the budget is reported explicitly. A successful close also initializes the held-object state described above from the measured hand pose. The coding agent uses the same gripper interface across Franka, RoboSuite, and RoboTwin 2.0, with platform-specific adapters mapping commands to native gripper coordinates. An arm selector is omitted for single-arm platforms and explicit on bimanual systems, preserving the selected arm through motion and release. Because this operation is deterministic and embodiment-local, we do not screen alternative gripper controllers, and QθQ_θ does not score the adapter. B.6 End-to-End Policy Execution Perform_E2E maps an explicit route_id to a frozen VLA, video-action, or world-action checkpoint. It supports three mutually exclusive instruction modes. Task mode forwards the environment task instruction when neither an object nor a prompt is provided. Grasp–lift mode takes an object name, generates a local grasp-and-lift instruction, and updates the held-object state after perception confirms an increase in the object’s height. Bounded-prompt mode forwards an explicit prompt verbatim and requires a step budget because arbitrary local instructions have no common success detector. Invalid argument combinations and missing required task language are rejected. Execution ends on task completion, environment termination, a supported model stop signal, or budget exhaustion; grasp–lift mode also ends when its local completion predicate is met. Each call records the instruction and its source, route, executed steps, termination reason, and measured postcondition. Full-task success and grasp–lift success use the task and local predicates, respectively. We screen released routes on 14 LIBERO-Pro tasks, four RoboSuite tasks, and ten RoboTwin 2.0 tasks. The screen specifies one initial state per task, but RoboTwin’s full-task evaluator rejects states that fail expert validation; realized initial states may therefore differ across methods. Full-task and grasp–lift success are reported in Table 8, and execution statistics in Tables 9, 10 and 11. Successful-trial completion times are reported separately from all-trial runtime statistics. Retained routes receive stable route_ids and are frozen before coordinator training. FastWAM and FasterWAM use their LIBERO checkpoints on LIBERO-Pro and RoboSuite, and their RoboTwin checkpoints on RoboTwin. Harness-VLA uses the same LIBERO-trained checkpoint on all three suites. The RynnVLA-002 results reflect a loader that reinitializes part of the VQ encoder, rather than an unmodified checkpoint deployment. JointWAM is excluded because no runnable checkpoint was available in the audited release. These results characterize the specified checkpoint–suite configurations. Table 8: Success counts across three evaluation suites. Full measures successful completion of the entire task, whereas G–L measures whether the policy successfully grasps and lifts the target object, providing a partial-task measure of manipulation capability. Each entry reports the number of successful evaluations over the total number evaluated; the best result in each column is shown in bold. ⋆The π 0.5-RLinf-SFT policy uses the same LIBERO-trained checkpoint across all three suites, without suite-specific adaptation. ‡For RynnVLA-002, the current public loader reinitializes part of the VQ encoder when loading the released checkpoint. LIBERO-Pro RoboSuite RoboTwin 2.0 Method Full G–L Full G–L Full G–L Vision-Language-Action Policies OpenVLA-OFT [68] 4/14 5/14 0/4 0/4 0/10 1/10 PRTS [69] 6/14 7/14 1/4 1/4 0/10 4/10 RDT2 [70] 0/14 0/14 0/4 0/4 0/10 0/10 SpatialVLA [71] 0/14 0/14 0/4 0/4 0/10 1/10 StarVLA [72] 4/14 3/14 0/4 0/4 0/10 2/10 RynnVLA-002‡ [73] 1/14 3/14 0/4 0/4 0/10 1/10 EVO1 [74] 4/14 3/14 0/4 0/4 0/10 3/10 ABot-M0 [75] 4/14 3/14 1/4 0/4 7/10 9/10 InternVLA-A1.5 [76] 5/14 7/14 0/4 0/4 9/10 9/10 SmolVLA [77] 3/14 4/14 0/4 0/4 0/10 3/10 WLA-0 [78] 4/14 4/14 1/4 1/4 8/10 10/10 X-VLA [79] 2/14 4/14 0/4 0/4 0/10 0/10 MolmoAct2 [80] 10/14 11/14 1/4 2/4 0/10 0/10 π 0.5-RLinf-SFT⋆ [20] 10/14 13/14 0/4 2/4 0/10 2/10 Video-Action Models LingBot-VA [81] 0/14 3/14 0/4 2/4 9/10 9/10 World-Action Models FastWAM [21] 4/14 11/14 0/4 0/4 10/10 10/10 FasterWAM [25] 4/14 11/14 0/4 0/4 8/10 10/10 Table 9: Execution efficiency on LIBERO-Pro under the Full and G–L criteria. Full denotes end-to-end task success, while G–L denotes successful grasp and lift of the target object. For each criterion, All trials aggregates both successful and failed episodes and reports the average number of executed steps, average execution time, and effective control frequency. Successful only reports the mean and median completion time over successful episodes only. A dash indicates that the method achieves no successful trial under that criterion and therefore has no successful-only timing statistic. Lower all-trial steps or execution time do not necessarily indicate greater efficiency, since failed episodes may terminate before completing the full rollout horizon; these quantities should therefore be interpreted jointly with the corresponding success counts in Table 8. Full G–L All trials Successful only All trials Successful only Method Steps Time (s) Hz Mean (s) Median (s) Steps Time (s) Hz Mean (s) Median (s) Vision–Language–Action policies OpenVLA-OFT 547.4547.4 34.234.2 6.1716.171 7.87.8 7.57.5 503.1503.1 117.0117.0 6.3686.368 41.041.0 35.635.6 PRTS 459.8459.8 39.539.5 3.0173.017 11.711.7 11.511.5 429.2429.2 105.6105.6 3.5063.506 41.541.5 30.930.9 RDT2 720.0720.0 126.6126.6 0.8850.885 — — 720.0720.0 287.1287.1 0.7710.771 — — SpatialVLA 720.0720.0 135.4135.4 1.6081.608 — — 720.0720.0 390.7390.7 1.0741.074 — — StarVLA 542.4542.4 48.048.0 2.3382.338 12.912.9 12.812.8 580.5580.5 144.9144.9 3.8713.871 29.029.0 31.431.4 RynnVLA-002‡ 673.9673.9 81.981.9 1.3131.313 11.111.1 11.111.1 591.9591.9 1005.21005.2 0.8730.873 191.5191.5 145.9145.9 EVO1 542.0542.0 68.568.5 2.0082.008 18.418.4 15.915.9 581.8581.8 246.1246.1 1.9061.906 32.532.5 32.732.7 ABot-M0 540.9540.9 119.4119.4 2.3922.392 25.325.3 24.624.6 580.1580.1 168.8168.8 2.4022.402 42.242.2 41.541.5 InternVLA-A1.5 535.5535.5 157.8157.8 0.9140.914 67.267.2 43.643.6 407.6407.6 166.4166.4 1.3751.375 38.938.9 32.132.1 SmolVLA 590.1590.1 113.3113.3 3.1753.175 29.929.9 28.428.4 557.0557.0 242.9242.9 2.3602.360 55.555.5 43.443.4 WLA-0 541.6541.6 138.5138.5 1.1921.192 34.634.6 32.132.1 533.4533.4 235.7235.7 1.5221.522 27.327.3 25.625.6 X-VLA 654.7654.7 134.8134.8 3.2693.269 71.171.1 71.171.1 533.6533.6 223.2223.2 6.2016.201 38.238.2 40.240.2 MolmoAct2 301.1301.1 56.356.3 3.6493.649 28.528.5 26.626.6 213.6213.6 89.089.0 2.8412.841 41.741.7 40.640.6 π 0.5-RLinf-SFT 303.8303.8 60.460.4 4.8764.876 30.830.8 29.529.5 128.9128.9 30.930.9 4.8704.870 23.123.1 20.020.0 Video–Action models LingBot-VA 720.0720.0 354.7354.7 2.3152.315 — — 625.7625.7 708.1708.1 1.6221.622 364.9364.9 227.6227.6 World–Action models FastWAM 542.6542.6 120.0120.0 1.4511.451 31.731.7 31.031.0 177.7177.7 51.751.7 1.4591.459 10.610.6 7.17.1 FasterWAM 542.0542.0 115.5115.5 2.2012.201 28.328.3 27.527.5 177.3177.3 47.347.3 1.8351.835 10.710.7 6.56.5 Table 10: Execution efficiency on RoboSuite under the Full and G–L evaluation criteria. Full requires successful completion of the entire task, whereas G–L requires successful grasp and lift of the target object. For each criterion, All trials aggregates both successful and failed evaluations and reports the average number of executed steps, average execution time, and effective control frequency. Successful only reports the mean and median execution time computed only over evaluations satisfying the corresponding success criterion. A dash denotes that no evaluation is successful under that criterion, so successful-only timing statistics are unavailable; NR denotes a statistic that was not recorded. Lower all-trial step counts or execution times do not necessarily indicate higher execution efficiency, since failed evaluations may terminate early rather than complete the full rollout horizon. All-trial efficiency should therefore be interpreted jointly with the corresponding success counts reported in Table 8, where method references are also provided. Full G–L All trials Successful only All trials Successful only Method Steps Time (s) Hz Mean (s) Median (s) Steps Time (s) Hz Mean (s) Median (s) Vision–Language–Action policies OpenVLA-OFT 720.0720.0 179.2179.2 2.3372.337 — — 720.0720.0 111.2111.2 7.0427.042 — — PRTS 575.5575.5 160.0160.0 1.7791.779 37.637.6 37.637.6 561.5561.5 94.594.5 3.0923.092 18.218.2 18.218.2 RDT2 720.0720.0 261.7261.7 0.7260.726 — — 720.0720.0 202.5202.5 0.8560.856 — — SpatialVLA 720.0720.0 313.1313.1 1.0241.024 — — 720.0720.0 207.1207.1 1.5461.546 — — StarVLA 200.0200.0 46.746.7 3.6363.636 — — 720.0720.0 124.9124.9 3.9713.971 — — RynnVLA-002‡ 220.0220.0 192.3192.3 1.4781.478 — — 720.0720.0 590.4590.4 1.4621.462 — — EVO1 720.0720.0 128.6128.6 2.5162.516 — — 720.0720.0 128.9128.9 2.3432.343 — — ABot-M0 576.8576.8 100.1100.1 3.7393.739 26.226.2 26.226.2 720.0720.0 87.787.7 3.7543.754 — — InternVLA-A1.5 720.0720.0 159.0159.0 1.4601.460 — — 720.0720.0 104.4104.4 1.2781.278 — — SmolVLA 720.0720.0 164.4164.4 n.r. — — 720.0720.0 135.4135.4 2.9582.958 — — WLA-0 661.2661.2 139.5139.5 1.7171.717 97.097.0 97.097.0 559.2559.2 127.4127.4 1.5641.564 20.220.2 20.220.2 X-VLA 720.0720.0 101.6101.6 7.3827.382 — — 720.0720.0 103.4103.4 7.4317.431 — — MolmoAct2 556.8556.8 123.5123.5 1.1031.103 15.615.6 15.615.6 404.0404.0 80.980.9 2.7462.746 17.317.3 17.317.3 π 0.5-RLinf-SFT 720.0720.0 129.7129.7 4.7804.780 — — 424.8424.8 74.674.6 4.9524.952 24.124.1 24.124.1 Video–Action models LingBot-VA 720.0720.0 377.8377.8 2.8872.887 — — 526.8526.8 459.2459.2 1.3031.303 372.1372.1 372.1372.1 World–Action models FastWAM 720.0720.0 166.5166.5 1.4561.456 — — 720.0720.0 160.5160.5 1.4511.451 — — FasterWAM 720.0720.0 142.4142.4 1.5201.520 — — 720.0720.0 149.1149.1 1.8201.820 — — Table 11: Execution efficiency on RoboTwin 2.0 under the Full and G–L evaluation criteria. Full requires successful completion of the entire task, whereas G–L requires successful grasp and lift of the target object. For each criterion, All trials reports execution statistics computed over the complete evaluation set, including both successful and failed trials: Steps, Time (s), and Hz characterize rollout length, execution time, and effective control frequency, respectively. Successful trials reports the mean and median execution time over trials satisfying the corresponding success criterion. A dash denotes that no trial is successful under that criterion, so successful-trial timing statistics are unavailable. Lower all-trial step counts or execution times do not necessarily indicate greater execution efficiency, since failed trials may terminate before task completion and therefore incur shorter rollouts. These efficiency statistics should therefore be interpreted jointly with the corresponding success counts reported in Table 8, where method references are also provided. Full G–L All trials Successful trials All trials Successful trials Method Steps Time (s) Hz Mean (s) Median (s) Steps Time (s) Hz Mean (s) Median (s) Vision–Language–Action policies OpenVLA-OFT 792.0792.0 1273.41273.4 3.8793.879 — — 675.6675.6 1292.21292.2 3.4343.434 51.351.3 51.351.3 PRTS 790.0790.0 1620.31620.3 1.3701.370 — — 464.5464.5 654.0654.0 2.1392.139 78.978.9 53.053.0 RDT2 792.0792.0 1372.81372.8 0.3470.347 — — 790.0790.0 1359.11359.1 0.3340.334 — — SpatialVLA 790.0790.0 2242.12242.1 0.6690.669 — — 739.7739.7 1565.81565.8 0.7970.797 234.1234.1 234.1234.1 StarVLA 790.0790.0 708.5708.5 3.9383.938 — — 594.7594.7 621.8621.8 3.3213.321 96.496.4 96.496.4 RynnVLA-002 790.0790.0 1343.21343.2 0.9050.905 — — 734.8734.8 1258.91258.9 0.9790.979 121.3121.3 121.3121.3 EVO1 806.6806.6 641.9641.9 1.0961.096 — — 535.2535.2 399.3399.3 1.6261.626 111.7111.7 61.361.3 ABot-M0 470.0470.0 245.5245.5 3.6103.610 144.0144.0 139.6139.6 139.8139.8 117.7117.7 3.3973.397 44.244.2 43.843.8 InternVLA-A1.5 326.0326.0 188.0188.0 1.5341.534 162.1162.1 153.9153.9 144.8144.8 78.778.7 1.5111.511 40.740.7 42.542.5 SmolVLA 790.0790.0 405.9405.9 2.3922.392 — — 574.6574.6 332.6332.6 2.0112.011 48.148.1 38.838.8 WLA-0 368.0368.0 265.6265.6 1.2951.295 206.3206.3 202.4202.4 81.481.4 65.265.2 1.3181.318 65.265.2 54.054.0 X-VLA 798.0798.0 481.6481.6 7.9367.936 — — 790.0790.0 493.5493.5 7.5577.557 — — MolmoAct2 792.0792.0 670.4670.4 3.9833.983 — — 790.0790.0 359.7359.7 0.8710.871 — — π 0.5-RLinf-SFT 790.0790.0 616.1616.1 5.0295.029 — — 666.8666.8 483.8483.8 5.0425.042 32.632.6 32.632.6 Video–Action models LingBot-VA 344.5344.5 203.8203.8 0.9480.948 150.0150.0 159.3159.3 138.4138.4 168.4168.4 0.3310.331 101.1101.1 103.8103.8 World–Action models FastWAM 284.8284.8 164.1164.1 0.8350.835 164.1164.1 153.6153.6 73.273.2 40.040.0 1.4531.453 40.040.0 38.738.7 FasterWAM 431.2431.2 239.8239.8 1.1731.173 178.2178.2 179.3179.3 80.180.1 44.344.3 1.8631.863 44.344.3 45.645.6 Appendix C Experiment Details C.1 Benchmark Tasks The evaluation suite comprises 100100 tabletop-manipulation tasks across three benchmarks with different scene and control requirements: LIBERO-Pro [22] (single-arm tasks under instruction-redirection T and position-swap S perturbations), RoboSuite [23] (single-arm pick–place in clean scenes), and RoboTwin 2.0 [24] (long-horizon bimanual manipulation). The task pool contains 8080 LIBERO-Pro, 44 RoboSuite, and 1616 RoboTwin 2.0 tasks. Coordinator training uses scene seed 00 for these tasks, while evaluation uses seeds 11–1010. This split tests generalization to new scene seeds within the same task pool. LIBERO-Pro columns in Table 3 in the main text follow the eight T/S splits (Spat, Obj, Goal, and L10), each aggregating 1010 tasks × 1010 seeds; RoboSuite and RoboTwin columns aggregate their full task pools under the same seed range. Mobile-manipulation suites such as BEHAVIOR [82] and RoboCasa [83] fall outside the P5P^5 tabletop-manipulation scope and are not included. Cross-agent and cross-embodiment sensitivity. The study described in Takeaway 5 uses a fixed 3×43× 4 backend–embodiment grid on four RoboTwin 2.0 task instances: blocks_ranking_rgb (arrange the red, green, and blue blocks), place_cans_plasticbox (place the cans into the plastic box), place_bread_basket (place the bread into the basket), and stack_blocks_two (stack two blocks). Claude Code and Aloha–AgileX are the default backend and embodiment, respectively; the comparison swaps in Cursor or Codex, and Franka–Panda, ARX–X5, or UR5–WSG, while retaining the same task instructions, P5P^5 stage contracts, and typed API interfaces. Cursor and Claude Code both use Claude Opus 4.8 as the underlying generator, while Codex uses GPT-5.5. The backend sweep keeps the API library and trained coordinator fixed, whereas each embodiment uses its corresponding native adapter and compatible low-level controller under the shared API contract; client and decoding details are given in Section C.5. The supplementary videos provide one demonstration for each task–embodiment pair, giving 16 clips organized as <task>/<embodiment>.mp4. These clips illustrate execution on the different embodiments, while Figure 6 in the main text reports the aggregate success rates. C.2 Baseline Implementations All methods in Table 3 in the main text share the single-episode evaluation protocol of Section C.3: seeds 11–1010, one trial per seed, terminal SR only, and no global cross-episode memory. For implementations that support memory across episodes, retrieval over prior tasks, or persistent tip stores, we disable those channels to keep the available experience limited to the current episode. RoboTwin columns marked “/” indicate embodiment mismatch for single-arm VLA checkpoints that are incompatible with the bimanual RoboTwin controllers. End-to-end policies. π0 _0 [7] and π0.5 _0.5 [20] run as frozen open-loop policies under their released checkpoints for the LIBERO / RoboSuite control stacks; they are not re-fine-tuned on our coordinator training pool. FastWAM [21] and FasterWAM [25] likewise use released world-action checkpoints as full-episode controllers (not as Perform_E2E routes). In RoboAware, the retained π0.5 _0.5 and FastWAM routes appear only inside typed Perform_E2E calls at local skill granularity (Section B.6). Code-as-policy. CaP [8] and CaP-X [9] follow their public code-generation loops over language-conditioned robot APIs; we evaluate them under the shared episode horizon and without their optional cross-episode repair memory. RATS [26] and ASPIRE [27] keep their published planner / skill-library interfaces, again with global memory disabled. These baselines use their published execution-selection rules and do not receive the learned QθQ_θ family coordinator. VLA with harness. Pigey [28] and VLS [29] use their public harness wrappers around a frozen generative policy, with early-stop and reward-steering settings taken from the authors’ default configs under the shared single-episode protocol. Harness-VLA [11] is evaluated by us using the RLinf-released pi05_libero130_fullshot π0.5 _0.5-RLinf-SFT checkpoint [84], exposed through the vla_act primitive; harness component ablations appear in Section D.1. Controlled routing ablations of RoboAware (RoboAware-UO, RoboAware-TF, RoboAware-UA) keep the same coding agent and API library and change only coordinator training or family allocation, as listed in Sections 3.3 and 3.3 in the main text. C.3 Success and Runtime Metrics This subsection defines the success-rate and runtime measures used in the main evaluation. Each table specifies its task pool, and comparisons within a table use the same success criterion and timing convention unless stated otherwise. The Harness-VLA ablation in Section D.1 uses a separate task and seed schedule and reports success among valid completions, together with execution coverage. Success-rate denominator. An evaluation episode is one (task, scene seed) pair under seeds 11–1010. The binary success indicator is the terminal task reward r∈0,1r∈\0,1\ emitted by the suite’s registered evaluator when the episode ends by task completion, unrecoverable failure, empty iC_i, or the horizon T; partial stage progress never counts as success. The episode-level success rate (SR) is therefore SRep=1N∑n=1N[rn=1],SR_ep= 1N _n=1^N1[r_n=1], where N is the number of evaluated episodes in the stated pool (for example, N=100N=100 for a 1010 tasks × 1010 seeds column in Table 3 in the main text, and N=1,000N=1,000 for the full 100100 tasks × 1010 seeds suite). In the main evaluation, episodes that abort remain in the denominator, including crashes, timeouts, and failures caused by an empty admissible-family set. Task- and suite-level weighting. To distinguish equal weighting of tasks from pooling individual episodes, we define two averages. Task macro-average first computes a per-task SR over that task’s seeds, then averages those task rates with equal weight: SRmacro=1||∑t∈SRep(t).SR_macro= 1|T| _t SR_ep(t). Episode-weighted (micro) average is SRepSR_ep over the pooled episodes and therefore weights tasks by their episode counts. Because every task in the main evaluation has the same number of seeds, the task macro-average and episode-weighted average coincide, both within each suite and across all 1,0001,000 episodes. They differ only when tasks contribute unequal numbers of episodes. An unweighted average of the three suite-level rates is a different measure. It gives equal weight to LIBERO-Pro, RoboSuite, and RoboTwin despite their unequal task counts, whereas pooling all episodes gives each task equal weight under this protocol. We therefore identify whether each cross-suite result is an equal-weight suite average or a pooled episode average. Main-text columns use the episode-weighted measure specified in their captions. Execution-time scope. Reported times are wall-clock seconds on the evaluation host unless a table names a different clock. End-to-end episode time starts at the post-reset observation used to build the task plan and stops when the episode terminates under the success predicate above; simulator reset and offline model loading are excluded. Averages over all trials include failures; successful-trial means or medians are computed only on episodes with r=1r=1 and are marked undefined when that set is empty, matching Table 11. C.4 Hyperparameters and Experimental Settings Table 12 lists the reward weights, search and learning settings, and step budgets used by RoboAware. We distinguish four budgets because they count different parts of execution: environment / control steps advance the physics or low-level controller inside an API; an API step budget caps how long an atomic call may retain control before returning success or a typed failure; semantic decision steps count P5P^5 boundaries at which the coordinator may select a family; and the MCTS budget K counts completed suffix simulations per search root, not environment steps or semantic decisions. Root-collection and rollout suffixes use a fixed family-sampling rule with support on every admissible family; tree Selection uses UCB (Equation 1) instead. The main experiments use K=20K=20. The RoboSuite ablation compares K∈5,10,20,50K∈\5,10,20,50\ and shows diminishing success-rate gains beyond K=20K=20 (Figure 5 in the main text). RoboAware-UA uses uniform family allocation under the same K values. Table 12: Hyperparameters of RoboAware. Parameter Symbol Setting Discount factor γ 11 Perception reward weight wpw_p 0.20.2 Manipulation reward weight wmw_m 0.30.3 Task-success reward weight wtw_t 0.50.5 Smooth-L1L_1 threshold — 11 Neighboring-root draws S 44 Neighboring-time window Δ 1010 saved control steps UCB coefficient cucbc_ucb 1.01.0 Suffix-simulation budget K 2020 Episode horizon T 100100 decision transitions Q-learning updates M 10,00010,000 Learning rate η 10−510^-5 Target-copy interval J 500500 updates Translational pose threshold ϵp _p 0.02m0.02\,m Rotational pose threshold ϵR _R π/12radπ/12\,rad Atomic API step budget — 100100 control steps Block wall-clock timeout — 30s30\,s Code-repair attempts RrepairR_repair 22 Coding-agent history window HcodeH_code 44 full stage transitions C.5 Coding-Agent Backends The coding agent πcode _code is frozen before coordinator training and is shared by RoboAware and the controlled routing baselines. Cursor, Claude Code, and Codex are the client toolchains through which the coding models are accessed. In the reported runs, Cursor and Claude Code both use Claude Opus 4.8, while Codex uses GPT-5.5. We fix the client/CLI versions and decoding settings and record them in the run logs. Sensitivity results in Figure 6 in the main text swap only this backend while keeping the API library, P5P^5 contracts, and QθQ_θ fixed. C.6 Code-Execution Protocol Execution sandbox, budgets, and retries. Generated blocks run in a restricted Python process that can import only the registered API library and allowlisted Python standard-library helpers. Each block inherits an API-call budget, a control-step budget, and a 30s30\,s wall-clock timeout (see Table 12). Syntax errors, undefined names, disallowed imports, and API schema violations trigger an automatic repair round that appends the error text and regenerates under the same family constraint, up to Rrepair=2R_repair=2 attempts; exhausted repairs mark the branch as failed for SCB or terminate the deployment stage according to Table 1. Runtime exceptions raised inside an allowed API are returned as typed failure codes to the agent rather than as raw simulator dumps. Episode reset versus memory. Between evaluation episodes (and between independent training trajectories), the coding-client session is reset: conversation state, tool caches, and any product-level memory are cleared so that episode n+1n+1 cannot read traces from episode n. Within an episode, feedback is limited to the observable channel already present in xix_i: typed API returns, postcondition bits, failure codes, timeouts, and the truncated history hih_i. We do not enable cross-episode product memory, retrieval over prior tasks, or persistent “global tips” stores for main results; any ablation that reintroduces such memory is labeled separately and is not part of the default RoboAware protocol. Family selection at deployment is supplied by QθQ_θ; the coding agent does not choose cic_i, and prompts never reveal the unchosen family’s counterfactual outcome. C.7 Computation and Software The reported coordinator run used four NVIDIA A100 GPUs with 80 GB memory each. Fitting QθQ_θ took approximately six hours, corresponding to approximately 24 GPU-hours (4×6h4× 6\,h). Trajectory and counterfactual data collection took approximately one additional day on the same hardware pool. Data collection therefore accounts for most of the wall-clock time and must be included when reporting the training cost. Supplementary qualitative clips for the cross-embodiment study follow the naming <task>/<embodiment>.mp4 described in Section C.1. Appendix D Additional Empirical Results D.1 Ablation of Harness-VLA Experimental setup. We evaluate the harness components in two stages. All six configurations in Table 13 use the same π0.5 _0.5 SFT backbone and a common neutral interface prompt. We first screen them on 24 tasks with seed 42, then evaluate Full, w/o Memory, and w/o Memory & SAM3 on 80 tasks with seeds 41,42,43\41,42,43\. Removing Memory also disables the seed-0 reference, so we use memory package to denote these two components together. The w/o Reference variant removes only the reference while retaining Memory, allowing us to examine that component separately in the screening stage. Table 13: Component configurations for the Harness VLA ablation study. All variants use the same π0.5 _0.5 SFT backbone. Method Planner Memory Seed-0 reference SAM3 Scripted hybrid VLA A: Full Harness VLA ✓ ✓ ✓ ✓ ✓ π0.5 _0.5 SFT B: w/o Memory ✓ × × ✓ ✓ π0.5 _0.5 SFT C: w/o Reference ✓ ✓ × ✓ ✓ π0.5 _0.5 SFT D: w/o SAM3 ✓ ✓ ✓ × ✓ π0.5 _0.5 SFT E: w/o Memory & SAM3 ✓ × × × ✓ π0.5 _0.5 SFT F: Bare SFT × × × × × π0.5 _0.5 SFT Single-seed screening. Full succeeds on 15 of 24 tasks (62.5%; Table 14). Removing both the memory package and SAM3 produces the largest observed decrease, to 37.5%. This is a drop of 25.0 percentage points (p), with a 95% task-bootstrap confidence interval of [4.2,45.8][4.2,45.8] p. Removing the memory package alone, the seed-0 reference alone, or SAM3 alone reduces success by 16.7, 12.5, and 4.2 p, respectively, although all three confidence intervals include zero. The larger drop from joint removal does not by itself establish a positive interaction. The estimated additive interaction, SRA−SRB−SRD+SRESR_A-SR_B-SR_D+SR_E, is −4.2-4.2 p and provides no evidence of a positive additive interaction. Bare SFT achieves 41.7%, which is 20.8 p below Full. Because this variant also removes the planner and scripted hybrid execution, its comparison with Full measures the contribution of the overall harness rather than the planner alone. Table 14: Paired ablation screening on 24 tasks using seed 42. Drop is defined as the success rate of Full minus that of the corresponding variant, in percentage points (p). Confidence intervals are 95% task-bootstrap intervals for the paired drop. S→ counts tasks solved by Full but failed by the ablated variant; F→ counts the reverse. All 24 episodes completed without errors in each variant. Method Success / valid Success rate (%) Drop (p) 95% CI (p) S→ F→ A: Full Harness VLA 15/24 62.5 – – – – B: w/o Memory 11/24 45.8 16.7 [−8.3, 41.7][-8.3,\ 41.7] 7 3 C: w/o Reference 12/24 50.0 12.5 [−12.5, 37.5][-12.5,\ 37.5] 6 3 D: w/o SAM3 14/24 58.3 4.2 [−8.3, 20.8][-8.3,\ 20.8] 2 1 E: w/o Memory & SAM3 9/24 37.5 25.0 [4.2, 45.8][4.2,\ 45.8] 7 1 F: Bare SFT 10/24 41.7 20.8 [0.0, 41.7][0.0,\ 41.7] 6 1 Multi-seed confirmation. Table 15 separates execution errors from completed episodes in which the task fails. Among valid completions, Full achieves 56.6% success, compared with 16.9% for w/o Memory and 20.9% for w/o Memory & SAM3. Because execution errors leave different sets of valid episodes for each variant, we also compare variants on task–seed pairs that completed under both configurations. On these paired subsets, removing the memory package yields a signed difference of −38.9-38.9 p (n=113n=113, 95% CI [−50.0,−27.4][-50.0,-27.4] p), while removing both the memory package and SAM3 yields −35.8-35.8 p (n=109n=109, 95% CI [−48.2,−23.4][-48.2,-23.4] p). Both intervals exclude zero, supporting a contribution from the memory package on the jointly completed episodes. Scope and limitations. Service-connection errors leave only 47.1–49.2% of the 240 scheduled episodes per configuration with valid completions. The paired estimates therefore describe those jointly completed episodes and cannot establish the effect over the full scheduled set. We report execution coverage separately and do not count these service errors as model failures in this ablation. The confirmation stage also omits w/o SAM3, so it does not independently measure the standalone contribution of SAM3. The historical whole-system result of 65/80 (81.25%) used a different prompt and protocol and is therefore not directly comparable with Full in this study. Table 15: Multi-seed confirmation on 80 tasks with seeds 41,42,43\41,42,43\, yielding 240 scheduled episodes per variant. Valid SR is the number of successful episodes divided by the number of successful and failed episodes, so execution errors are excluded. Coverage is the fraction of the 240 scheduled episodes that completed without errors. Paired differences compare each variant against Full on task–seed pairs that completed without errors in both conditions: Δ=SRvariant−SRFull =SR_variant-SR_Full on that jointly valid subset of size npairn_pair, which need not equal the gap between the separately reported valid success rates. Method Success Failure Error Valid / scheduled Valid SR (%) Coverage (%) npairn_pair Paired Δ (p) 95% CI (p) A: Full Harness VLA 64 49 127 113/240 56.6 47.1 – – – B: w/o Memory 20 98 122 118/240 16.9 49.2 113 −38.9-38.9 [−50.0,−27.4][-50.0,\ -27.4] E: w/o Memory & SAM3 24 91 125 115/240 20.9 47.9 109 −35.8-35.8 [−48.2,−23.4][-48.2,\ -23.4] D.2 Visualizations Figure 1 shows a further LIBERO-Pro rollout under spatial-language perturbation, extending the cases in Figure 4 in the main text. As before, RoboAware-EO and RoboAware-MO fail in complementary ways, whereas RoboAware completes the task by switching families at P5P^5 boundaries. Figure 1: Additional LIBERO-Pro comparison on “Pick the akita black bowl between the plate and the ramekin and place it on the plate,” complementing Figure 4 in the main text. RoboAware-EO approaches with a misaligned gripper and fails to secure the bowl. RoboAware-MO completes a grasp but selects the wrong object (the cookie box). RoboAware grounds the spatially described bowl and finishes the pick-and-place, succeeding by switching policy families across P5P^5 boundaries. D.3 Failure Modes Figure 2 summarizes the failures observed in LIBERO-Pro and RoboTwin. Grasping and object-holding instability is the largest identified category, accounting for 29.8% of failed rollouts. Constrained contact, insertion, and mechanism actuation account for another 16.7%. Together, these categories represent 46.5% of all failed rollouts, indicating that low-level interaction and contact-rich execution are major sources of failure. Target grounding (5.3%), transport and placement (6.1%), and multi-object sequencing (4.4%) account for smaller identified shares. However, 37.7% of failures remain in Others because the saved evaluation records lack the detailed traces needed to distinguish perception, planning, control, and success-check errors. This limits how strongly the identified categories can characterize the full failure distribution. The categories summarize the observed outcomes. During deployment, recovery and termination instead follow the typed API failures, timeouts, and stage conditions defined in Tables 1 and 1. Figure 2: Failure modes of the unsuccessful rollouts. Grasping instability is the largest identified failure mode (29.8%), followed by constrained contact or insertion failures (16.7%). Grounding, placement, and multi-object sequencing account for 5.3%, 6.1%, and 4.4%, respectively. Others (37.7%) denotes failed rollouts for which the saved evaluation records are insufficient to identify a specific cause (29.8% from LIBERO-Pro and 7.9% from RoboTwin).