Paper deep dive
FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-Manipulation
Ajay Vikram Periasami, Xinyuan Luo, Haoyu Li, Xianyi Cheng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/22/2026, 4:52:49 AM
Summary
The paper introduces FRAMES, a failure-aware supervisory framework for humanoid loco-manipulation using the Unitree G1 robot. It integrates a Planner Agent, a VLM-based Monitor Agent, and a Recovery Agent operating above the CEER whole-body controller. The Monitor Agent evaluates skill execution using multi-view vision and structured state data, achieving 94.0% accuracy in detecting failures and accepting successes in simulation. The system includes a Memory Module for skill reuse and geometric grounding via SAM3.
Entities (10)
Relation Signals (9)
Monitor Agent → achievesaccuracy → 94.0%
confidence 95% · achieves 94.0% overall accuracy.
FRAMES → contains → Planner Agent
confidence 95% · The core of our contribution is a modular supervisory layer divided into three core entities: a Planner Agent
FRAMES → contains → Monitor Agent
confidence 95% · a Monitor Agent to check for failures during skill execution
FRAMES → contains → Recovery Agent
confidence 95% · and a Recovery Agent to recover from the detected failures.
FRAMES → evaluatedin → MuJoCo
confidence 95% · We independently evaluate the monitoring module of the framework in MuJoCo
FRAMES → runson → Unitree G1
confidence 95% · We present FRAMES, a failure-aware supervisory framework for the Unitree G1 humanoid
FRAMES → uses → CEER
confidence 95% · operates above the CEER whole-body controller
Monitor Agent → uses → GPT-5.4 Mini
confidence 90% · OpenAI GPT-5.4-mini is used for the ... Monitor Agent.
Planner Agent → uses →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model (LLM) planners can decompose natural-language instructions and select reusable robot skills, but choosing the correct skill does not guarantee successful physical execution. This gap is especially important in humanoid loco-manipulation, where errors during approach, grasping, transport, or placement can invalidate the remainder of a long-horizon plan. We present FRAMES, a failure-aware supervisory framework for the Unitree G1 humanoid that operates above the CEER whole-body controller. A Planner Agent selects subtasks through parameterized mid-level skills, while a vision-language-model-based Monitor Agent evaluates each skill using temporal multi-view observations and structured robot and contact evidence. Detected failures stop the active skill and provide grounded feedback to a Recovery Agent. The framework further includes a Memory Module for reusing prior skill experience, and geometric grounding via depth and segmentation. We independently evaluate the monitoring module of the framework in MuJoCo using 100 trials comprising 50 failed and 50 successful executions across five tasks. The monitor detects 48 of 50 failures, correctly accepts 46 of 50 successful executions, and achieves 94.0% overall accuracy. These results provide initial evidence for the monitoring component, while end-to-end evaluation of the complete recovery loop remains ongoing.
Tags
Links
- Source: https://arxiv.org/abs/2609.22538v1
- Canonical: https://arxiv.org/abs/2609.22538v1
Trouble viewing inline? Open PDF directly →
Full Text
29,791 characters extracted from source content.
Expand or collapse full text
FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-Manipulation Ajay Vikram Periasami1, Xinyuan Luo2, Haoyu Li2, Xianyi Cheng2 Affiliation: Affiliation: 1Department of Computer Science, Duke University, Durham, NC, USA 2Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC, USA ajay.vikram@duke.edu, xinyuan.luo@duke.edu, haoyu.harry.li@duke.edu, xianyi.cheng@duke.edu Abstract Large language model (LLM) planners can decompose natural-language instructions and select reusable robot skills, but choosing the correct skill does not guarantee successful physical execution. This gap is especially important in humanoid loco-manipulation, where errors during approach, grasping, transport, or placement can invalidate the remainder of a long-horizon plan. We present FRAMES, a failure-aware supervisory framework for the Unitree G1 humanoid that operates above the CEER whole-body controller [1]. A Planner Agent selects subtasks through parameterized mid-level skills, while a vision-language-model-based Monitor Agent evaluates each skill using temporal multi-view observations and structured robot and contact evidence. Detected failures stop the active skill and provide grounded feedback to a Recovery Agent. The framework further includes a Memory Module for reusing prior skill experience, and geometric grounding via depth and segmentation. We independently evaluate the monitoring module of the framework in MuJoCo using 100 trials comprising 50 failed and 50 successful executions across five tasks. The monitor detects 48 of 50 failures, correctly accepts 46 of 50 successful executions, and achieves 94.0% overall accuracy. These results provide initial evidence for the monitoring component, while end-to-end evaluation of the complete recovery loop remains ongoing. Index Terms: humanoid loco-manipulation, failure detection, failure recovery, vision-language models, semantic memory, embodied agents I Introduction Humanoid robots achieve loco-manipulation by combining locomotion and manipulation to execute long-horizon tasks in human environments. A typical long-horizon task, such as box transport, requires the robot to execute a sequence of subtasks: approaching the object, grasping it, carrying it while walking, and placing it at a new location. The physical coupling of these subtasks means that early errors cascade into downstream failures. An inaccurate approach directly prevents a clean grasp, a premature transition to walking can leave the box unexpectedly supported by the table, and the subsequent motions of walking or turning can ultimately break a secure hold. If these failures go undetected, subsequent actions will be planned using an incorrect estimate of the world state. Achieving reliable humanoid loco-manipulation therefore requires the robot to continuously monitor physical execution, identify failures in real time, and recover autonomously from the observed state. Recent robotic agents leverage large language models (LLMs) and vision-language models (VLMs) to interpret instructions and select reusable skills. For instance, SayCan [2] combines language-based planning with learned affordance values, while Inner Monologue [3] provides execution feedback to an LLM planner. Similarly, CAPE [4] utilizes failed skill preconditions to generate corrective actions. These approaches primarily use environment feedback to replan between high-level actions, rather than explicitly monitoring the temporal evolution of an ongoing physical skill and interrupting it upon failure. Fig. 1: The planner selects the correct grasp subtask, yet the box can still slip during physical execution. Without failure-aware supervision, the robot proceeds from an invalid state and fails the task. FRAMES detects the failed grasp, invokes a centered grasp, and enables the robot to complete the turn while retaining the box. To handle execution errors more robustly, recent methods have moved toward explicit failure reasoning. REFLECT [5] and AHA [6] utilize multimodal data and VLMs to detect and explain failures, while RACER [7] applies language feedback to learn recovery behaviors. In the domain of execution monitoring, DoReMi [8] tracks dynamic constraints via VLMs, Code-as-Monitor [9] synthesizes executable monitoring programs, and COME-robot [10] integrates visual feedback for mobile-manipulation planning. Despite these advances, most methods focus on fixed-base or wheeled robots, with DoReMi being a notable exception that demonstrates applicability to humanoids. Concurrent to our work, recent humanoid systems like Cybo-Waiter [11] and POT-VLA [12] have also begun exploring the integration of language planning, geometric grounding, and execution verification. While POT-VLA achieves this by training a unified action model conditioned on shared object tokens, our approach avoids costly policy retraining. Instead, we propose a modular supervisory framework positioned above an existing whole-body controller. We build our framework on the CEER Controller [1], keeping its low-level policy entirely fixed. The core of our contribution is a modular supervisory layer divided into three core entities: a Planner Agent to decompose and execute natural-language tasks via mid-level skills, a Monitor Agent to check for failures during skill execution, and a Recovery Agent to recover from the detected failures. We also develop a Memory Module that supplies reusable experience to the Planner Agent and Recovery Agent. We also provide depth and segmentation skills for the geometric grounding of LLM-suggested skills. This clean separation of concerns enables advanced planning, monitoring, and recovery behaviors without requiring any modifications or retraining at the whole-body control level. The contributions of this work are: • A modular planning and recovery framework that executes humanoid loco-manipulation tasks through parameterized mid-level skills over a fixed whole-body controller, with skill parameterization informed by persistent subtask-level semantic memory and planner-callable SAM3 and RGB-D object grounding. • A VLM-based Monitor Agent that evaluates each skill using multi-view images, structured robot state, and contact information across a temporal window. It distinguishes ongoing progress from success and failure, stops failed skills, and provides failure evidence to the Recovery Agent. • A controlled simulation evaluation of the Monitor Agent over 100 trials spanning failed grasps, dropped objects, collisions during approach, wrong-object grasps, and wrong placements. The monitor detects 48 of 50 failures, accepts 46 of 50 successful executions, achieving 94.0% overall accuracy. These experiments provide initial quantitative evidence for the monitoring component. Evaluation of the complete planning, monitoring, recovery, and memory loop remains ongoing. I Methods I-A Problem Formulation Given a natural-language instruction g, the objective is to execute a sequence of robot skills that satisfies the instruction while detecting and recovering from execution failures. We use k to index high-level subtasks, n to index monitoring samples collected during subtask k execution, and ℓ to index low-level CEER control steps. For each subtask, n=0n=0 denotes the observation immediately before execution. The observation at monitoring sample n of subtask k is ok,n=(sk,n,ℐk,n),ℐk,n=Ik,nvv=1Ncam,o_k,n= (s_k,n,I_k,n ), _k,n= \I_k,n^v \_v=1^N_cam, (1) where sk,ns_k,n is the structured robot and scene state, and ℐk,nI_k,n contains the available camera views. In simulation, Ncam=3N_cam=3, corresponding to robot-perspective, top-down, and side-view RGB images. The structured state is sk,n=(CLOSE s_k,n= ( OPENbk,n,k,n,k,n,ℋk,n,k,n,ℱk,n), b_k,n,O_k,n,U_k,n,H_k,n,K_k,n,F_k,n ), (2) where bk,nb_k,n is the robot base pose, k,nO_k,n contains dynamic-object states, k,nU_k,n contains support-surface states, ℋk,nH_k,n contains hand positions, k,nK_k,n contains contact relations, and ℱk,nF_k,n contains contact forces. The instruction is solved sequentially by the Planner Agent, one subtask at a time. At high-level decision step k, the system executes the action ak=(dk,σk,θk,Ck),a_k= (d_k, _k, _k,C_k ), (3) where dkd_k describes the intended subtask, σk _k is the selected skill API, θk _k contains its execution parameters, and CkC_k is its monitor contract. For a physical skill, the monitor contract is Ck=(ckpre,cksucc,ckfail),C_k= (c_k^pre,c_k^succ,c_k^fail ), (4) where the fields specify the subtask preconditions, success condition, and failure conditions. Observation and execution-control APIs do not require a monitor contract. During each physical skill execution, the Monitor Agent periodically classifies the skill as in_progress, success, or failure. An in_progress verdict allows execution to continue, while a failure interrupts the skill and activates the Recovery Agent. After the skill finishes, a final monitor check determines whether control returns to the Planner Agent. Fig. 2: Overview of FRAMES. A natural-language instruction is handled by the Planner Agent, which selects a subtask from the skill library together with its API parameters and necessary preconditions and failure and success criteria. The selected subtask is executed through the fixed CEER whole-body controller in the MuJoCo environment. During execution, the Monitor Agent compares the current observation against the baseline observation using multi-view perception and structured state information to determine whether the subtask is succeeding or failing. The monitor verdict and supporting evidence are returned immediately to the decision layer, where failures inform the Recovery Agent. At the end of each run, completed skill executions are distilled into concise subtask-level records and appended to memory. I-B Planner Agent At decision step k, the Planner Agent receives the original instruction g, current observation ok≡ok,0o_k≡ o_k,0, execution history hkh_k, and skill library A. It proposes one next action: a~k a_k =(dk,σk,θ~k,Ck) = (d_k, _k, θ_k,C_k ) (5) =PlannerAgent(g,ok,hk,), =PlannerAgent (g,o_k,h_k,A ), where θ~k θ_k denotes the initially proposed parameters. The execution history is hk=ai,ρi,y^i,zii=0k−1,h_k= \a_i, _i, y_i,z_i \_i=0^k-1, (6) where ρi _i is the skill execution result, y^i y_i is the Monitor Agent verdict, and ziz_i is its explanation. Monitor fields are empty for skills that do not require monitoring. The skill library is =nav∪man∪obs∪ctrl,A=A_nav _man _obs _ctrl, (7) where the subsets contain navigation, manipulation, observation, and execution-control skills, respectively. Table I lists the available skill APIs and their execution arguments. TABLE I: Skill Library Available to the Planner and Recovery Agents Category API Arguments Function Locomotion move_to (x,y,ψ)(x,y,ψ) Moves the robot base to an absolute world-frame pose. turn (ψ,T)(ψ,T) Rotates the robot toward an absolute world-frame yaw ψ over duration T. move_backward (x,y)(x,y) Moves the base toward an absolute world-frame position while preserving its final orientation. approach_object (target,dstand,Δψ)(target,d_stand, ψ) Moves the base to a target-facing pose with the specified standoff distance and optional yaw offset. Manipulation move_hands (pL,pR,Thold)(p_L,p_R,T_hold) Moves both hands to robot-base-frame target positions pL,pR∈ℝ3p_L,p_R ^3 and maintains them for TholdT_hold seconds, without moving the base. grasp_object (target,pL,pR,dlift)(target,p_L,p_R,d_lift) Grasps the target using specified left- and right-hand contact positions, then lifts it by dliftd_lift. release_object None Opens both hands to release the currently held object. Observation get_observation (include_images)(include\_images) Returns structured state and, when requested, the available camera views. perceive_object (prompt)(prompt) Locates a text-specified object using SAM3 [13] and depth, and estimates its position, visible size, and hand-to-object distances in the robot-base frame. Control wait (T)(T) Pauses high-level execution for duration T. done (rclaimed,ztask)(r_claimed,z_task) Requests task completion with a claimed outcome and summary. After the Planner Agent selects skill σk _k with initial parameters θ~k θ_k, the Memory Module retrieves previous records that used the same skill: ℬσk=mi∈ℬ∣σi=σk,B_ _k= \m_i _i= _k \, (8) where ℬB is the persistent memory bank and ℬσkB_ _k contains the retrieved records for the selected skill. The same Planner Agent is then called in a focused parameterization mode to reuse prior experience without allowing memory to alter the intended subtask. It receives the selected skill, initial parameters, current observation, recent execution history, matching memory records, and the skill’s argument schema. It may revise only schema-permitted execution parameters, producing θk _k, while dkd_k, σk _k, the target object, and CkC_k remain fixed. If no relevant memory is available or the revised parameters are invalid, the system retains θk=θ~k _k= θ_k. The resulting action aka_k, defined in Eq. (3), is then passed to the Skill Executor. I-C Skill Executor: CEER Controller We use the pretrained CEER whole-body controller without modification. CEER receives root and end-effector targets and maps them, together with proprioceptive observations, to 29-DoF whole-body commands at 50 Hz. Details of its policy architecture, training procedure, and control abstraction are provided in [1]. For a physical skill σk _k, the Skill Executor converts the selected parameters θk _k and current state into a sequence of root and end-effector targets: uk,ℓE-root=Gσk(θk,sk,ℓ),u_k, ^E -root=G_ _k ( _k,s_k, ), (9) where ℓ indexes low-level control steps. CEER executes these targets as qk,ℓcmd=πCEER(pk,ℓ,uk,ℓE-root),q_k, ^cmd= _CEER (p_k, ,u_k, ^E -root ), (10) where pk,ℓp_k, is the CEER proprioceptive observation. The Skill Executor updates targets at approximately 10 Hz, while CEER runs at 50 Hz. After a skill completes or is interrupted, the Skill Executor returns ρk=(rkstatus,rkmessage,rkdata), _k= (r_k^status,r_k^message,r_k^data ), (11) containing whether the skill completed and provides any skill-specific measurements. The Monitor Agent separately evaluates whether the observed execution satisfied the subtask contract CkC_k. Both outputs are added to the execution history and used to select the next planning or recovery action. I-D Monitor Agent The Monitor Agent evaluates whether each physical skill satisfies its subtask contract. It uses skill-local temporal context rather than judging a single observation independently. Before skill k begins, the framework captures and retains a baseline observation ok,0=(sk,0,ℐk,0).o_k,0= (s_k,0,I_k,0 ). (12) At monitoring sample n, it captures the current observation ok,no_k,n. With three camera views, each monitor query therefore contains six images: the robot-perspective, top-down, and side views at skill start, followed by the same three views at the current sample. The framework also computes a compact structured summary of the change since skill execution began: Δsk,n=ϕ(sk,0,sk,n). s_k,n=φ (s_k,0,s_k,n ). (13) The summary includes object displacement, change in robot–object distance, base-pose changes, and transitions in manipulator, support, and floor contacts. Retaining the same baseline throughout the skill allows the Monitor Agent to evaluate cumulative progress rather than only the change between adjacent samples. The evidence supplied to the Monitor Agent is ek,n=(CLOSE e_k,n= ( OPENg,dk,Ck,ok,0,ok,n,Δsk,n), g,d_k,C_k,o_k,0,o_k,n, s_k,n ), (14) where g is the complete task instruction, dkd_k is the current subtask, and CkC_k specifies its preconditions, success condition, and failure conditions. The Monitor Agent returns (y^k,n,zk,n)=MonitorAgent(ek,n), ( y_k,n,z_k,n )=MonitorAgent (e_k,n ), (15) where y^k,n y_k,n is in_progress, success, or failure, and zk,nz_k,n is a concise explanation grounded in the visual and structured evidence. The Monitor Agent checks the skill approximately once per second. An in_progress result allows execution to continue, while a failure result stops the skill and activates the Recovery Agent. A success result is accepted only after the skill has finished and the Monitor Agent confirms the final state. I-E Recovery Agent Let nkfn_k^f denote the monitoring sample at which subtask k is declared failed. The framework constructs the failure record fk=(CLOSE f_k= ( OPENak,ρk,zk,nkf,Δsk,nkf,ok,nkf). a_k, _k,z_k,n_k^f, s_k,n_k^f,o_k,n_k^f ). (16) The Recovery Agent proposes a corrective action: a~k+1R=RecoveryAgent(CLOSE a_k+1^R=RecoveryAgent ( OPENg,ok,nkf,hk,fk,). g,o_k,n_k^f,h_k,f_k,A ). (17) The Recovery Agent uses the same skill library as the Planner Agent but is explicitly conditioned on the failed subtask and its observed failure evidence. After the Recovery Agent selects a skill, the Memory Module retrieves records for that skill and refines only its permitted execution parameters just like the Planner Agent. The resulting recovery action ak+1Ra_k+1^R is then executed and monitored using the same procedure as a normal planner action. The Recovery Agent may retry the failed skill with revised parameters, select a corrective approach or hand motion, release an unstable contact, or request additional observations. I-F Memory Module The framework maintains a persistent semantic memory bank ℬB. After each task terminates, including an interrupted run, every completed physical subtask is distilled into a memory record mi=(CLOSE m_i= ( OPENdi,σi,θi,ρi,risub,zi,ritask,λi), d_i, _i, _i, _i,r_i^sub,z_i,r_i^task, _i ), (18) where risubr_i^sub is the local subtask outcome, ritaskr_i^task is the final whole-task outcome, and λi _i is a concise lesson describing the observed cause, parameter effect, recovery result, and reuse guidance. Observation and execution-control skills are not stored. After either the Planner Agent or Recovery Agent selects a skill σk _k, the Memory Module retrieves records with the same skill: ℬσk=mi∈ℬ|σi=σk∧risub=ritask.B_ _k= \m_i \; |\; _i= _k r_i^sub=r_i^task \. (19) Successful subtasks are retrieved only from successful tasks, while failed subtasks are retained as negative examples only from failed tasks. A subtask reported as successful in an ultimately failed task is excluded because its local success may have been transient or incorrect. Within a successful task, only the latest successful execution of each skill is retained. I-G Language-Conditioned Geometric Grounding To provide the Planner Agent with geometric grounding for more informed skill parameterization, it can call perceive_object with a text description such as “red box.” The module applies SAM3 [13] to a frame-matched robot-perspective RGB image to obtain a segmentation mask for the requested object. Valid depth measurements inside the mask are used to reconstruct the visible object surface in 3D. Using the camera calibration and robot base pose, the resulting points are transformed into the world and robot-base frames. The module returns the visible-surface center, visible extent, median depth, segmentation confidence, and hand-to-object distances. These measurements support the parameterization of object-relative approach, hand-motion, and grasp skills. I Preliminary Experiments & Results I-A Experimental Setup We conduct preliminary experiments in MuJoCo using the 29-DoF Unitree G1 humanoid with the pretrained CEER Controller [1], which was trained in simulation using Isaac Lab with 16,384 parallel environments on four NVIDIA L40s GPUs for 8 hours. OpenAI GPT-5.4-mini is used for the Planner Agent, Recovery Agent, and Monitor Agent. SAM3 provides language-conditioned object segmentation for RGB-D geometric grounding. I-B Monitor Evaluation Protocol We evaluate the Monitor Agent independently on five different tasks. Each task contains 20 trials: 10 failure trials and 10 matched successful executions. Small variations are applied to the robot pose, approach parameters, or manipulation parameters for each of the trials. The required skill sequence is executed by teleoperating, without involving the Planner Agent or Recovery Agent. This is done to isolate the Monitor Agent performance. The five tasks are defined as follows. Failed grasp. The robot is instructed to grasp and lift the box from the table. A trial is successful when both hands hold and lift the box, and failed when the box remains on the table after the grasp attempt. Dropped object. The robot begins each trial holding the box above the table and must maintain its grasp. A trial is successful when the robot continues holding the box and failed when the box slips or falls onto the table or floor. Collision during approach. The robot is instructed to approach the box without disturbing it. A trial is successful when the robot reaches the box without contact and failed when its hands contact and move the box during the approach. Wrong-object grasp. A red box and a blue box of the same shape are placed on the table, and the robot is instructed to lift the red box. A trial is successful when the robot lifts the red box and failed when it lifts the blue box. Wrong placement. A blue target pad and a yellow distractor pad are placed on the table, and the robot is instructed to place the red box on the blue pad. A trial is successful when the box is placed on the blue pad and failed when it is placed on the yellow pad. We report the number of failures correctly detected, the number of successful trials correctly accepted and overall classification accuracy. An in_progress verdict returned after skill completion is counted as an incorrect classification. (a) (b) (c) (d) (e) Fig. 3: Representative failure and success outcomes from the five Monitor Agent tasks. The top row shows failure conditions, and the bottom row shows matched successful executions: (a) failed versus successful grasp; (b) dropped object versus stable hold; (c) collision versus collision-free object approach; (d) grasping the blue distractor box versus the requested red box; and (e) placing the red box on the yellow distractor pad versus the blue target pad. I-C Monitor Evaluation Results Table I summarizes the results. Across all 100 trials, the Monitor Agent correctly detects 48 of 50 failures and accepts 46 of 50 successful executions, corresponding to 94.0% overall accuracy. The monitor correctly classifies all dropped-object and wrong-object-grasp trials. The wrong-object task is particularly important because the monitor identifies the lifted object from camera images without receiving object names or poses. The remaining errors reveal two limitations. Two failed grasps are labelled in_progress even though skill execution has ended. Three safe approaches are incorrectly classified as collisions because the hands appear close to the box. One correct placement is also classified as a failure because the box appears to remain held after release. These errors show that the monitor is conservative when physical contact is visually ambiguous. Figure 3 shows qualitative examples of the robot failing and succeeding on the 5 tasks. TABLE I: Monitor performance across five benchmarks, each containing 10 failed and 10 successful skill executions. Benchmark Failures Detected Successful Runs Accepted Overall Accuracy Failed grasp 8/10 10/10 90% Dropped object 10/10 10/10 100% Collision during approach 10/10 7/10 85% Wrong-object grasp 10/10 10/10 100% Wrong placement 10/10 9/10 95% Overall 48/50 46/50 94% IV Conclusion We presented FRAMES, a failure-aware framework for humanoid loco-manipulation built on top of the CEER whole-body controller. The Planner Agent selects parameterized skills and defines how their execution should be checked. The Monitor Agent detects failures using temporal multi-view images and structured state information, and stops the current skill when needed. The Recovery Agent then selects a corrective action based on the failure evidence. FRAMES also uses semantic memory to reuse earlier skill experience, and RGB-D and SAM3 to support object grounding. Our preliminary evaluation focused on the Monitor Agent. Across 100 executions from five tasks, it correctly classified 94 trials, detecting 48 of 50 failures and accepting 46 of 50 successful runs. These results show that the monitoring component can identify several common execution failures, but the complete planning and recovery loop still requires end-to-end evaluation. Future work will improve and evaluate the remaining components and test the full framework across a wider range of long-horizon tasks. References [1] X. Luo, X. Chen, X. Yin, H. Wu, B. Xia, Z. Chen, J. Li, B. Chen, and X. Cheng (2026) CEER: compliant end-effector and root control as a unified interface for hierarchical humanoid loco-manipulation. arXiv preprint arXiv:2605.19981. External Links: Document Cited by: §I, §I-C, §I-A, Abstract. [2] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, et al. (2022) Do as i can, not as i say: grounding language in robotic affordances. arXiv preprint arXiv:2204.01691. Cited by: §I. [3] W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, et al. (2022) Inner monologue: embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608. Cited by: §I. [4] S. S. Raman, V. Cohen, I. Idrees, E. Rosen, R. Mooney, S. Tellex, and D. Paulius (2024) Cape: corrective actions from precondition errors using large language models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 14070–14077. Cited by: §I. [5] Z. Liu, A. Bahety, and S. Song (2023) Reflect: summarizing robot experiences for failure explanation and correction. arXiv preprint arXiv:2306.15724. Cited by: §I. [6] J. Duan, W. Pumacay, N. Kumar, Y. R. Wang, S. Tian, W. Yuan, R. Krishna, D. Fox, A. Mandlekar, and Y. Guo (2024) Aha: a vision-language-model for detecting and reasoning over failures in robotic manipulation. arXiv preprint arXiv:2410.00371. Cited by: §I. [7] Y. Dai, J. Lee, N. Fazeli, and J. Chai (2025) Racer: rich language-guided failure recovery policies for imitation learning. In 2025 IEEE international conference on robotics and automation (ICRA), p. 15657–15664. Cited by: §I. [8] Y. Guo, Y. Wang, L. Zha, and J. Chen (2024) Doremi: grounding language model by detecting and recovering from plan-execution misalignment. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 12124–12131. Cited by: §I. [9] E. Zhou, Q. Su, C. Chi, Z. Zhang, Z. Wang, T. Huang, L. Sheng, and H. Wang (2025) Code-as-monitor: constraint-aware visual programming for reactive and proactive robotic failure detection. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6919–6929. Cited by: §I. [10] P. Zhi, Z. Zhang, Y. Zhao, M. Han, Z. Zhang, Z. Li, Z. Jiao, B. Jia, and S. Huang (2025) Closed-loop open-vocabulary mobile manipulation with gpt-4v. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 4761–4767. Cited by: §I. [11] P. Ren, H. Ge, C. Qi, C. Huang, H. Li, J. Zhao, P. Chi, and K. Chen (2026) Cybo-waiter: a physical agentic framework for humanoid whole-body locomotion-manipulation. arXiv preprint arXiv:2603.10675. Cited by: §I. [12] P. Ren, H. Ge, J. Zhao, C. Huang, Y. Shi, P. Chi, and K. Chen (2026) Closing the loop in humanoid vla: persistent 3d object tokens for verifiable loco-manipulation. arXiv preprint arXiv:2607.18016. Cited by: §I. [13] N. Carion, L. Gustafson, Y. Hu, S. Debnath, R. Hu, D. Suris Coll-Vinent, C. Ryali, K. V. Alwala, H. Khedr, A. Huang, et al. (2026) Sam 3: segment anything with concepts. In International Conference on Learning Representations, Vol. 2026, p. 138846–138923. Cited by: §I-G, TABLE I.