Paper deep dive
Continual Quadruped Robots Coordination via Semantic Skill Discovery
Daoqing Wang, Yuchen Xiao, Weixuan Huang, Zhilong Zhang, Shenghua Wan, Meng Li, Lei Yuan, Yang Yu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/9/2026, 3:23:19 AM
Summary
The paper introduces Conquer, a framework for continual multi-quadruped robot coordination that addresses catastrophic forgetting in open-ended environments. It utilizes a Self-Allies-Goal (SAG) backbone to handle variable team sizes, a semantic skill library with VLM-generated descriptors, and a retrieve-adapt-update workflow based on MAPPO to continuously learn and reuse coordination skills. Simulations on a 14-task benchmark achieve a 95.6% success rate, with real-world validation on Unitree Go2 robots.
Entities (12)
Relation Signals (10)
Conquer → evaluatedon → Isaac Lab
confidence 97% · Experiments on a 14-task Isaac Lab benchmark demonstrate that Conquer achieves strong performance
Conquer → deployedon → Unitree Go2
confidence 96% · Real-world rollouts on Unitree Go2 teams further validate the deployment feasibility of Conquer
Conquer → implements → Self-Allies-Goal (SAG) Backbone
confidence 96% · we design a team-structured Self-Allies-Goal (SAG) backbone that supports variable-cardinality robot teams
Conquer → solves → Multi-quadruped Coordination
confidence 95% · we propose Conquer, a semantic skill-library framework that formulates continual multi-quadruped coordination as a retrieve-adapt-update process
Retrieve-Adapt-Update Workflow → uses → MAPPO
confidence 95% · adapted through multi-agent reinforcement learning... optimized for the current task with the MAPPO objective
Conquer → manages → Skill Library
confidence 94% · Conquer updates the skill library by extracting trajectory-level semantic descriptors and organizing them according to semantic distance
Conquer → mitigates → Catastrophic Forgetting
confidence 94% · robots are expected to acquire new coordination skills while reusing previously learned ones without catastrophic forgetting
→ →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-quadruped coordination has attracted increasing attention due to its enhanced payload capacity, broader contact coverage, and improved adaptability to challenging tasks. Existing methods for multi-quadruped manipulation typically focus on predefined or closed task families, often relying on multi-agent reinforcement learning (MARL) to train task-specific coordination policies. However, such methods struggle in open-ended continual learning settings, where tasks arrive sequentially and robots are expected to acquire new coordination skills while reusing previously learned ones without catastrophic forgetting. To address this challenge, we propose Conquer, a semantic skill-library framework that formulates continual multi-quadruped coordination as a retrieve-adapt-update process. First, to accommodate varying team sizes across tasks, we design a team-structured Self-Allies-Goal (SAG) backbone that supports variable-cardinality robot teams by explicitly modeling each robot's own state, teammate context, and task goal. For each incoming task, Conquer constructs a task-level semantic descriptor from pre-execution information and retrieves a relevant skill from the library for adaptation. After successful execution, Conquer updates the skill library by extracting trajectory-level semantic descriptors and organizing them according to semantic distance, thereby enabling continual skill accumulation and cross-task knowledge transfer. Simulation experiments show that Conquer achieves a final average success rate of 95.6%, demonstrating strong forward transfer and negligible catastrophic forgetting. Real-world rollouts on Unitree Go2 teams further validate the deployment feasibility of Conquer for practical multi-quadruped coordination. Simulation and real-robot demonstration videos are available at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2606.08102v2
- Canonical: https://arxiv.org/abs/2606.08102v2
Trouble viewing inline? Open PDF directly →
Full Text
73,881 characters extracted from source content.
Expand or collapse full text
Continual Quadruped Robots Coordination via Semantic Skill Discovery Daoqing Wang1,2, Yuchen Xiao1,211footnotemark: 1, Weixuan Huang1,2, Zhilong Zhang1,2,3, Shenghua Wan1,2, Meng Li1, Lei Yuan1,2,3, Yang Yu1,2,3 1 National Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China 2 School of Artificial Intelligence, Nanjing University, Nanjing, China 3 Polixir Technologies, Nanjing, China wangdq,xiaoyuchen,zhangzl,wansh,yuanl@lamda.nju.edu.cn 231220091,menson@smail.nju.edu.cn, yuy@nju.edu.cn Project page: https://conquer-project.pages.dev/ Equal contribution. Abstract Multi-quadruped coordination has attracted increasing attention due to its enhanced payload capacity, broader contact coverage, and improved adaptability to challenging tasks. Existing methods for multi-quadruped manipulation typically focus on predefined or closed task families, often relying on multi-agent reinforcement learning (MARL) to train task-specific coordination policies. However, such methods struggle in open-ended continual learning settings, where tasks arrive sequentially and robots are expected to acquire new coordination skills while reusing previously learned ones without catastrophic forgetting. To address this challenge, we propose Conquer, a semantic skill-library framework that formulates continual multi-quadruped coordination as a retrieve-adapt-update process. First, to accommodate varying team sizes across tasks, we design a team-structured Self-Allies-Goal (SAG) backbone that supports variable-cardinality robot teams by explicitly modeling each robot’s own state, teammate context, and task goal. For each incoming task, Conquer constructs a task-level semantic descriptor from pre-execution information and retrieves a relevant skill from the library for adaptation. After successful execution, Conquer updates the skill library by extracting trajectory-level semantic descriptors and organizing them according to semantic distance, thereby enabling continual skill accumulation and cross-task knowledge transfer. Simulation experiments show that Conquer achieves a final average success rate of 95.6%, demonstrating strong forward transfer and negligible catastrophic forgetting. Real-world rollouts on Unitree Go2 teams further validate the deployment feasibility of Conquer for practical multi-quadruped coordination. Simulation and real-robot demonstration videos are available at: https://conquer-project.pages.dev/. 1 Introduction Quadruped robots, as a representative class of legged mobile robots, have achieved great progress in recent years due to their mobility and stability in open-ended environments such as transportation, search and rescue, and industrial automation [53, 51, 23]. Early studies mainly focused on locomotion, enabling robots to track velocity or trajectory commands and traverse complex terrain through model predictive control, reinforcement learning, and sim-to-real transfer [40, 3, 15]. As robotic applications move toward physically interacting with objects, quadruped loco-manipulation has become an important research direction, requiring robots to interact with objects while maintaining whole-body stability and safe contact [7, 14]. Nevertheless, the payload capacity, contact range, and manipulation stability of a single quadruped robot remain limited by the physical platform. Multi-quadruped coordination [7, 14] provides a natural way to overcome these limitations through distributed contact and cooperative control [40, 23]. It has long been studied in multi-robot systems, where multiple robots coordinate their motions and contact forces to interact with objects that may be difficult for a single robot to manipulate [40, 3]. Recent studies further extend this idea to legged platforms, demonstrating collaborative quadrupedal payload manipulation over challenging terrain [15], multi-quadruped long-horizon pushing [7], and joint planning and control for object transport [14]. Combined with multi-agent reinforcement learning methods [51], these works show the promise of multi-quadruped coordination for physically demanding tasks, but they are mostly developed for a specific task setting or a bounded task family. In open-ended environments, such cooperative systems are more likely to encounter a stream of tasks where the size of the multi-quadruped team and the environments may change. Continual reinforcement learning [29, 43, 17] offers a promising perspective for this problem. Parameter isolation methods [16, 47] learn multi-headed policies to memorize the decision-making knowledge for old tasks. Replay-based methods mitigate forgetting by storing experience via replay buffers [32] or generative models [5, 12]. Retrieval methods [26, 41, 10] generate task semantic representations [36, 4, 31] such as user instructions, structured task descriptions, scene observations, and visual-language summaries, and store them for knowledge retrieval. Specifically, task variations in multi-quadruped teams may change the team size and coordination patterns, and sequential adaptation can lead to catastrophic forgetting. This raises a key question: can we build a continual multi-quadruped coordination framework that selectively reuses skills from pre-execution semantics, while still learning through reward-driven interaction on the new task? To tackle this question, we propose Conquer, a semantic skill retrieval framework with a retrieve-adapt-update workflow for continual multi-quadruped coordination. When a new task arrives, it first builds a semantic descriptor from the pre-execution semantics and retrieves the most relevant executable skill from the skill library. The parameters of the retrieved skill are used to initialize a new skill and are further adapted through multi-agent reinforcement learning [48]. After training, Conquer summarizes successful trajectories with a VLM-to-embedding pipeline and uses the resulting descriptor to decide whether the current skill should update an existing library entry or be inserted as a new one. This workflow is implemented with a Self-Allies-Goal (SAG) backbone that provides a shared policy interface across variable team sizes [44, 13], and with lightweight low-rank adapters [11] that store each coordination skill. Experiments on a 14-task Isaac Lab [25] benchmark demonstrate that Conquer achieves strong performance, reaching 95.6%95.6\% final average success rate with strong forward transfer and negligible forgetting. Further ablations confirm the importance of semantic retrieval and LoRA/LocHead skill transfer, and real-world Unitree Go2 rollouts show the deployment feasibility of Conquer through a hierarchical real-robot control stack. 2 Related Work Multi-Agent Reinforcement Learning (MARL) aims to train multiple agents to optimize a shared objective. Different from single-agent settings, MARL faces the curse of dimensionality in the joint state-action spaces, caused by the growing number of agents. To overcome this challenge, centralized training and decentralized execution (CTDE) methods transform the complex joint space into low-dimensional agent-wise subspaces, showcasing high efficiency in real-world applications such as autonomous driving and embodied intelligence [8]. Specifically, VDN [38] and QMIX [30] learn value decomposition networks to reduce the value function dimensionality, while MADDPG [22], COMA [9], and MAPPO [48] learn decentralized actors for execution efficiency. Furthermore, multi-robot manipulation investigates how MARL methods aid physical robot teams to coordinate contact, relative positioning, and object dynamics in transport, towing, carrying, and pushing tasks [40, 15, 7, 14]. These works provide the control and learning foundation for multi-quadruped coordination, focusing on improving performance within a fixed task family. Continual Reinforcement Learning studies how models learn a sequence of tasks while balancing stability and plasticity [29, 43, 17, 39, 1]. Replay-based methods including DISTR [5] and CoD [12] mitigate forgetting by storing experiences via replay buffers or generative models. Parameter isolation methods such as OWL [16] store knowledge for each task with multi-headed networks, while ClonEx-SAC [47] combines multi-headed networks and parameter regularization [18, 2] for better performance. Gradient projection methods such as GEM [21] and SGP [33] adjust update directions to protect prior knowledge. To evaluate forgetting and transfer of different methods, robotic continual learning benchmarks such as Continual World [46] provide various task stream settings designed for single-agent or specific scenarios. Continual coordination [50, 49] is more challenging as task changes may alter not only individual behaviors but also the cooperative structure among agents, such as contact assignment and team-level equilibria. Skill Discovery learns action chunks via temporal behavioral abstractions, which can improve downstream reinforcement learning [37]. Unsupervised skill discovery methods such as DIAYN [6] and DADS [35] learn latent skills to promote exploration, and multi-agent skill discovery methods such as ODIS [52] and HiSSD [19] learn common or task-specific coordination patterns for cross-agent transfer. Retrieval-based robot learning methods such as SAILOR [26], LOTUS [41], and SRSA [10] reuse skills by constructing and retrieving task semantics such as user instructions, scene observations and vision-language representations. Furthermore, parameter-efficient adapters [11] provide natural tools for grounding and storing skills for skill-library methods [45, 20]. Despite these advances, existing methods often rely on explicit task boundaries or access to old task datasets, and cannot handle varying team sizes in multi-quadruped task streams, limiting their applicability in open-ended environments. Our framework addresses these issues via the semantics-based skill retrieval and a team-structured Self-Allies-Goal (SAG) backbone. 3 Problem Setting We study multi-quadruped coordination tasks under the framework of decentralized partially observable Markov decision processes (Dec-POMDPs) [27]: ℳ=⟨,,,Ω,P,O,,γ⟩,M= ,S,A, ,P,O,R,γ , (1) where =1,…,nN=\1,…,n\ is the robot set, S is the global state space, =1×⋯×nA=A^1×·s×A^n is the joint action space, and Ω is the local observation space. The transition, observation, and per-robot reward functions are defined as: P:×→Δ(),O:×→Ω,Ri:×→ℝ.P:S×A→ (S), O:S×N→ , R^i:S×A . (2) At time t, robot i receives only oti=O(st,i)o_t^i=O(s_t,i) and outputs ati∈ia_t^i ^i. Each robot receives a reward composed of a team term and an individual shaping term, Rti=Rtteam+Rti,selfR_t^i=R_t^team+R_t^i,self, and optimizes the discounted return [∑t=0∞γtRti]E [ _t=0^∞γ^tR_t^i ]. Continual coordination investigates a sequence of tasks =(ℳ1,…,ℳm,…),Y=(M_1,…,M_m,…), (3) where each task may change the number of robots, object geometry, terrain, initial configuration, and target state. During stage m, the learner can interact only with the current task ℳmM_m and has no access to earlier tasks. After finishing the first M tasks, the system should recover an appropriate executable policy for any task in M=ℳ1,…,ℳMY_M=\M_1,…,M_M\ without retraining old tasks. Under this protocol, the policy must share its structure across different multi-robot teams, and new-task learning must avoid overwriting previously learned coordination skills. 4 Approach This section describes the design of Conquer as shown in Figure 1. Sec. 4.1 introduces the policy architecture and executable skill library. Sec. 4.2 defines the semantic descriptor interface that maps both incoming task observations and trained skill trajectories into a shared embedding space. Sec. 4.3 then organizes these components into a continual retrieve-adapt-update workflow. Figure 1: Overview of Conquer. (a) Multi-quadruped cooperative tasks arrive sequentially. (b) Each incoming task follows a retrieve-adapt-update workflow that initializes a new skill from a retrieved one, adapts it through MAPPO, and updates the skill library. (c) A VLM-to-embedding interface maps task instructions and trajectories into a shared embedding space for retrieval. (d) A shared SAG backbone is combined with task-specific LoRA/LocHead skill adapters. 4.1 SAG Backbone and Skill Library SAG policy backbone. Conquer uses a Self-Allies-Goal (SAG) architecture as the shared policy backbone for variable-team coordination. SAG follows the policy-decoupling idea of UPDeT [13], and specializes the token structure to multi-quadruped coordination. For robot i, SAG decomposes the local observation into self, ally, and goal tokens, oi=[xiself,xi,1ally,…,xi,Naally,x1goal,…,xNggoal]o^i=[x_i^self,x_i,1^ally,…,x_i,N_a^ally,x_1^goal,…,x_N_g^goal], and projects them into a shared latent space as si=Psxiselfs_i=P_sx_i^self, Ai=Paxi,kallyk=1NaA_i=\P_ax_i,k^ally\_k=1^N_a, and G=Pgxℓgoalℓ=1NgG=\P_gx_ ^goal\_ =1^N_g. Using the self token as the query and Ci=[Ai;G]C_i=[A_i;G] as context, SAG computes ui=softmax((siWQ)(CiWK)⊤dK)CiWV,hi=fSAG(si,ui). u_i=softmax ( (s_iW^Q)(C_iW^K) d_K )C_iW^V, h_i=f_SAG(s_i,u_i). (4) The resulting fixed-dimensional feature hih_i is mapped to the continuous action distribution by a lightweight policy head, while the critic can use centralized team information during training. Skill library. Each skill is represented by a lightweight task adapter. Let θ denote the frozen SAG policy backbone. The executable parameters of skill i are ϕi=(ϕilora,ωiloc) _i=( _i^lora, _i^loc), where ϕilora _i^lora are LoRA adapter parameters and ωiloc _i^loc is the LocHead head. The corresponding policy is πi(a∣o)=πθ,ϕi(a∣o) _i(a o)= _θ, _i(a o). At stage t, the skill library is t=(si,ϕi,ci,mi)i=1NtD_t=\(s_i, _i,c_i,m_i)\_i=1^N_t, where sis_i is the skill identifier, ϕi _i is directly loadable for execution or initialization, cic_i is a semantic center descriptor, and mim_i records environment metadata. The descriptor cic_i is the mean embedding of language descriptions generated from successful trajectories of skill i. In summary, each entry contains both an executable adapter and a semantic index: ϕi _i produces grounded control, while cic_i supports later retrieval and duplicate detection. When training a new task, the backbone and old skills stay frozen, while gradients update only the current LoRA/LocHead adapter and the critic. 4.2 Semantic Descriptor Interface Conquer uses a VLM-to-embedding procedure to construct semantic descriptors for both incoming tasks and stored skills. Given visual observations and a task-level description, a pretrained VLM first produces task-focused text that summarizes core semantics; a text embedding model then maps the text into a shared embedding space Z. This design follows a common engineering intuition for transfer and curriculum design: tasks with similar team composition, environment context, interaction target, and goal semantics are more likely to share reusable coordination patterns. We therefore use the Euclidean distance in the descriptor space as a proxy for semantic compatibility: a smaller distance indicates a more plausible source skill for initialization. For a new task, this procedure transforms a pre-execution frame and task description into an initialization descriptor qinitq^init. For skill i, it transforms successful trajectory images into embeddings Zi=zijj=1Ki,zij∈Z_i=\z_ij\_j=1^K_i,z_ij . The skill descriptor is the mean embedding ci=1Ki∑j=1Kizijc_i= 1K_i _j=1^K_iz_ij. 4.3 Continual Retrieve-Adapt-Update Workflow Given the skill library and semantic descriptor interface, Conquer runs a retrieve-adapt-update loop for each incoming task. In the retrieve step, the pre-execution descriptor selects the nearest stored skill, i⋆=argmini‖qinit−ci‖2i = _i\|q^init-c_i\|_2. If the library is non-empty, the current LoRA/LocHead adapter is initialized from ϕi⋆ _i ; otherwise it starts from a fresh adapter. In the adapt step, the shared SAG backbone and old skills are frozen, while the current adapter ϕt _t and critic are optimized for the current task with the MAPPO [48] objective: ℒMAPPO=−[∑i∈min(rϕiAi,clip(rϕi,1−ϵ,1+ϵ)Ai)]+λvℒV−λHℋ(πθ,ϕt).L_MAPPO=-E [ _i (r_φ^iA^i,clip(r_φ^i,1-ε,1+ε)A^i ) ]+ _vL_V- _HH( _θ, _t). (5) Here the critic is centralized but agent-indexed: Vψ(st,i)V_ψ(s_t,i) estimates the return for robot i after attending to the team state. rϕi=πθ,ϕt(ai∣oi)πθ,ϕold(ai∣oi)r_φ^i= _θ, _t(a^i o^i) _θ, _old(a^i o^i) is the PPO ratio for robot i. The advantage is estimated per robot by Generalized Advantage Estimation (GAE) [34]: Ati=∑ℓ≥0(γλ)ℓδt+ℓiA_t^i= _ ≥ 0(γλ) _t+ ^i with δti=Rti+γVψi(st+1)−Vψi(st) _t^i=R_t^i+γ V_ψ^i(s_t+1)-V_ψ^i(s_t). The value loss is averaged over robot-wise value targets, ℒV=t[∑i∈ℓv(Vψi(st),R^ti)]L_V=E_t\! [ _i _v\! (V_ψ^i(s_t), R_t^i ) ], where ℓv _v is the clipped value regression loss used by PPO. ℋH is the policy entropy, and λv,λH _v, _H are their weights. In the update step, successful trajectories from the adapted skill are encoded into qtskillq_t^skill. The nearest existing entry is j⋆=argminj‖qtskill−cj‖2j = _j\|q_t^skill-c_j\|_2. If ‖qtskill−cj⋆‖2<τ\|q_t^skill-c_j \|_2<τ, the matched entry is updated; otherwise, the current LoRA/LocHead adapter and descriptor are inserted as a new skill. This workflow separates continual learning into semantic initialization, reward-driven adaptation, and controlled library growth. The full procedure is summarized in Appendix A.1 and hyperparameter settings are listed in Appendix A. 5 Experiment In this section, we evaluate Conquer on continual multi-quadruped pushing tasks in simulation and on real Unitree Go2 deployments. Our experiments aim to answer: (1) whether Conquer learns new tasks while retaining old skills (Sec. 5.1.2); (2) which mechanisms contribute to Conquer’s performance (Sec. 5.1.3); and (3) whether the learned skills transfer to the real-world (Sec. 5.2). 5.1 Simulation Experiments 5.1.1 Simulation Setup Figure 2: Representative simulated task views and their retrieval prompts. Benchmark. All simulation experiments use Isaac Lab [25]. The benchmark contains 14 canonical push-to-goal tasks that vary robot count, object geometry, and terrain. The task stream first covers 6 single-dog tasks, then 4 two-dog tasks, and finally 4 three-dog tasks, spanning cube, cuboid, and triangular-prism objects on flat and rough terrains. Figure 2 shows examples from this visual-prompt interface. Detailed task definitions, physical parameters, and success criteria are provided in Appendix B.1. Baselines and Metrics. We compare Conquer with online continual-learning baselines including EWC [18] and Fine-tune, the skill-learning baseline HiSSD and task-aware baseline PSEC. We also include Multitask as a same-training-budget joint-training reference. We report the success rate (SR) for each method and additionally report forward transfer (FWT) and backward transfer (BWT) for online continual-learning methods. Details for the metrics and baselines are described in Appendix B.2.2 and Appendix B.2.3. 5.1.2 Main Results Figure 3: Average metrics ± standard deviation on the 14-task benchmark across three different seeds. † denotes results obtained under offline-learning protocols. Figure 4: Final success rates. Error bars indicate one standard deviation. Figure 5: Success rate curves and semantic transfer case study for 3d-tri-rough. We evaluate whether Conquer improves the stability-plasticity trade-off in the continual task stream. As shown in Table 4 and Figure 4, the basic Fine-tune baseline achieves relatively strong forward transfer, with an FWT of 9.1%9.1\%, likely due to the simple-to-harder task order. However, since it lacks protection on previously learned skills when adapting to new tasks, it suffers from severe catastrophic forgetting, leading to its −31.8%-31.8\% BWT. In contrast, EWC substantially alleviates forgetting by constraining updates to sensitive parameters with the Fisher information matrix, but this comes at the cost of reduced plasticity: it achieves only −1.5%-1.5\% FWT, leading to its absolute 8.8%8.8\% lower final SR compared with our method. Notably, Conquer achieves an FWT of 11.3%11.3\%, an approximately zero BWT, and the highest final average SR among online learning baselines under the same training budget, even outperforming the multitask reference in SR. Furthermore, Figure 5 shows the evolution of SR of different methods along the task stream, where Fine-tune forgets previous tasks, EWC struggles to adapt to new tasks and Conquer continues to improve its SR and eventually outperforms the multitask baseline. These results indicate that Conquer achieves the best balance between plasticity and stability among baselines. Detailed semantic-access backtest results are reported in Appendix C.1. 5.1.3 Ablation and Case Study To identify the contribution of each component of Conquer, we design three ablation experiments as shown in Table 1, including Random which selects skills randomly at deployment, Multihead which uses only the LocHead as the trainable component, and Scratch which trains each new adapter freshly without initialization from a retrieved one. Table 1: Ablation results. Condition Final SR (%) Conquer (Ours) 95.6±0.695.6± 0.6 Random 73.4±2.573.4± 2.5 Multihead 86.8±0.786.8± 0.7 Scratch 89.2±0.689.2± 0.6 As shown in Table 1, Random reduces the average success rate by absolute 22.2%22.2\%, indicating that skill selection without task semantics cannot reliably match different tasks. Multihead that removes LoRA causes an absolute 8.8%8.8\% performance drop, suggesting that task adaptation requires not only output-head selection but also adjustment of intermediate entity-interaction representations. Scratch decreases performance by absolute 6.4%6.4\%, showing that semantic retrieval provides more effective initialization for new tasks. Overall, Conquer achieves the best performance by combining zero-shot skill matching via semantic retrieval with LoRA-based adaptation to different cooperative patterns. We further present a case study on the 3d-tri-rough query to illustrate both the benefit and the limitation of the semantic descriptor in Figure 5 and Figure 5. In this case, the top-3 semantic neighbors achieve a higher average zero-shot transfer success rate than the remaining candidates. At the same time, semantic rank does not perfectly match physical transferability: the best non-top-3 source, 2d-tri-flat, achieves the second-highest transfer success rate among the compared sources despite being outside the top-3 semantic neighbors. Thus semantic retrieval improves expected initialization quality but does not guarantee optimal transfer. Detailed rank computation, visualization assumptions, and initialization-check values are reported in Appendix C.2. 5.2 Real-Robot Deployment To show Conquer’s deployment feasibility for real-world multi-quadruped coordination, we evaluate the performance of the learned coordination skills on physical Unitree Go2 teams. We adopt a hierarchical policy architecture for stable real-world deployment: the high-level policy outputs velocity commands (vx,vy,vyaw)(v_x,v_y,v_yaw) at 10Hz, while the low-level controller tracks them through motor control at 50Hz. Further deployment details are listed in Appendix B.3. 5.2.1 Real Deployment Setup Sim-to-real policy construction. For the one-, two-, and three-Go2 demonstrations, we keep the SAG module trained in simulation and replace the original LocHead with a newly initialized three-dimensional action head that outputs (vx,vy,vyaw)(v_x,v_y,v_yaw) for each robot.Meanwhile, the four-Go2 large-cuboid trial uses the same hierarchical action interface, with its policy transferred from the domain-randomized three-Go2 policy. Then these policies are treated as the high-level policies and are trained through a unified domain-randomization pipeline. We utilize the integrated locomotion policy of the Unitree Go2 as the low-level controller for real-world experiments. Hardware setup. Real-world trials are conducted in an outdoor area with an 8.0m × 8.0m boundary, equipped with a XingYing motion-capture system with 24 cameras, which gathers real-time data regarding robots and objects. The box masses are 1.8kg for the one-robot task, 7.2kg for the two- and three-robot tasks, and 16.2kg for the four-robot task. For each trial, the box is initialized randomly within 3.0–4.5m of a fixed target. 5.2.2 Real Deployment Results Table 2: Representative real-robot deployment rollouts. Setting Dogs Duration (s) Start (m) Final (m) Min. (m) One-Go2 cuboid 1 10 3.276 0.243 0.243 Two-Go2 cuboid 2 20 3.365 0.800 0.800 Three-Go2 cuboid 3 20 3.753 0.165 0.165 Four-Go2 large cuboid 4 20 4.457 0.304 0.059 Figure 6: Real-robot deployment examples. From left to right, columns show one, two, three, and four Go2 rollout photos with their corresponding trajectories. Table 2 reports representative rollouts with their real-world behavior and trajectories. By transferring the policies from simulation to reality, the box-to-goal distances of the one-, three-, and four-Go2 trials reduce from their initial distances to 0.243 m, 0.165 m, and 0.304 m, respectively. The two-Go2 rollout is slower in the fixed 20s window, but the same rollout continues to 0.062m at 29.93s. Concretely, the lower row of Figure 6 shows that a single Go2 can flexibly adjust the position of the object until reaching the goal, while 2–4 Go2 coordinate their positioning to push the object, flexibly adjust the direction and movement of the heavier object, and eventually reach the goal. Considering that the real targets and pushed objects have non-negligible physical volume, these results show that the pushing tasks are completed successfully and that the learned coordination policies transfer to hardware in these representative rollouts. 6 Conclusion We present Conquer, a semantic skill-library framework for continual multi-quadruped coordination. Conquer represents learned coordination behaviors through a shared SAG backbone, parameter-isolated adapters, and visual-language descriptors for skill retrieval and library update. By selecting adapters based on pre-execution semantics and adapting skills with MAPPO, Conquer improves new-task learning while keeping old skills accessible. Experiments on a 14-task Isaac Lab benchmark demonstrate strong stability and plasticity, and real-robot rollouts on Unitree Go2 teams further showcase the deployment feasibility of Conquer through a hierarchical control stack. Limitations and Future Work. Despite these results, the current study is still limited to a controlled multi-quadruped task family. The retrieval mechanism also relies on the heuristic that nearby semantic descriptors imply reusable coordination behavior, without explicitly modeling physical transferability among environmental dynamics. Future work may extend the static semantic retrieval pipeline toward closed-loop skill management, where training dynamics and physical transfer statistics can be used to revise, merge, or compose candidate skills. Another promising direction is to decompose complex deployment goals into fine-grained cooperative subskills, enabling semantic retrieval to support more heterogeneous multi-robot teams and broader loco-manipulation tasks. References [1] Z. Abbas, R. Zhao, J. Modayil, A. White, and M. C. Machado (2023) Loss of plasticity in continual deep reinforcement learning. In Conference on Lifelong Learning Agents, p. 620–636. Cited by: §2. [2] R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars (2018) Memory aware synapses: learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, p. 139–154. Cited by: §2. [3] J. Alonso-Mora, S. Baker, and D. Rus (2017) Multi-robot formation control and object transport in dynamic environments via constrained optimization. International Journal of Robotics Research 36 (9), p. 1000–1021. Cited by: §1, §1. [4] A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, et al. (2023) Do as i can, not as i say: grounding language in robotic affordances. In Conference on Robot Learning, p. 287–318. Cited by: §1. [5] F. Chen, F. Han, C. Guan, L. Yuan, Z. Zhang, Y. Yu, and Z. Zhang (2024) Stable continual reinforcement learning via diffusion-based trajectory replay. arXiv preprint arXiv:2411.10809. Cited by: §1, §2. [6] B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine (2018) Diversity is all you need: learning skills without a reward function. In International Conference on Learning Representations, Cited by: §2. [7] Y. Feng, C. Hong, Y. Niu, S. Liu, Y. Yang, and D. Zhao (2025) Learning multi-agent loco-manipulation for long-horizon quadrupedal pushing. In International Conference on Robotics and Automation (ICRA), p. 14441–14448. Cited by: §1, §1, §2. [8] Z. Feng, R. Xue, L. Yuan, Y. Yu, N. Ding, M. Liu, B. Gao, J. Sun, X. Zheng, and G. Wang (2026) Multi-agent embodied ai: advances and future directions. Science China Information Sciences 69 (5), p. 151202. Cited by: §2. [9] J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson (2018) Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §2. [10] Y. Guo, B. Tang, I. Akinola, D. Fox, A. Gupta, and Y. Narang (2025) SRSA: skill retrieval and adaptation for robotic assembly tasks. In International Conference on Learning Representations, Cited by: §1, §2. [11] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §1, §2. [12] J. Hu, L. Shen, S. Huang, Z. Yang, H. Chen, L. Sun, Y. Chang, and D. Tao (2025) Continual diffuser (cod): mastering continual offline rl with experience rehearsal. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §1, §2. [13] S. Hu, F. Zhu, X. Chang, and X. Liang (2021) UPDeT: universal multi-agent rl via policy decoupling with transformers. In International Conference on Learning Representations, Cited by: §1, §4.1. [14] H. A. Jaafar, C. Kao, and S. Saeedi (2024) MR. cap: multi-robot joint control and planning for object transport. IEEE Control Systems Letters 8, p. 139–144. Cited by: §1, §1, §2. [15] Y. Ji, B. Zhang, and K. Sreenath (2021) Reinforcement learning for collaborative quadrupedal manipulation of a payload over challenging terrain. In International Conference on Automation Science and Engineering (CASE), p. 899–904. Cited by: §1, §1, §2. [16] S. Kessler, J. Parker-Holder, P. Ball, S. Zohren, and S. J. Roberts (2022) Same state, different task: continual reinforcement learning without interference. In Proceedings of the AAAI Conference on Artificial Intelligence, p. 7143–7151. Cited by: §1, §2. [17] K. Khetarpal, M. Riemer, I. Rish, and D. Precup (2022) Towards continual reinforcement learning: a review and perspectives. Journal of Artificial Intelligence Research 75, p. 1401–1476. Cited by: §1, §2. [18] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. (2017) Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114 (13), p. 3521–3526. Cited by: §B.2.3, §2, §5.1.1. [19] S. Liu, Y. Shu, C. Guo, and B. Yang (2025) Learning generalizable skills from offline multi-task data for multi-agent cooperation. In International Conference on Learning Representations, Cited by: §B.2.3, §2. [20] T. Liu, J. Li, Y. Zheng, H. Niu, Y. Lan, X. Xu, and X. Zhan (2025) Skill expansion and composition in parameter space. In International Conference on Learning Representations, Vol. 2025, p. 85192–85228. Cited by: §B.2.3, §2. [21] D. Lopez-Paz and M. Ranzato (2017) Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, p. 6470–6479. Cited by: §B.2.1, §2. [22] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems, p. 6382–6393. Cited by: §2. [23] A. Majithia, D. Shah, J. Dave, A. Kumar, S. Rathee, N. Dogra, V. H. M., D. S. Chiniwar, and S. Hiremath (2024) Design, motions, capabilities, and applications of quadruped robots: a comprehensive review. Frontiers in Mechanical Engineering 10. Cited by: §1, §1. [24] W. Meng, T. Zhang, H. Sun, and Y. Yin (2026) MTRL-cg: multi-task reinforcement learning method with spectral clustering-based task grouping. Proceedings of the AAAI Conference on Artificial Intelligence 40, p. 36723–36731. Cited by: §B.2.3. [25] M. Mittal, P. Roth, J. Tigue, A. Richard, O. Zhang, P. Du, A. Serrano-Muñoz, X. Yao, R. Zurbrüg, N. Rudin, L. Wawrzyniak, M. Rakhsha, A. Denzler, E. Heiden, A. Borovicka, O. Ahmed, I. Akinola, A. Anwar, M. T. Carlson, J. Y. Feng, A. Garg, R. Gasoto, L. Gulich, Y. Guo, M. Gussert, A. Hansen, M. Kulkarni, C. Li, W. Liu, V. Makoviychuk, G. Malczyk, H. Mazhar, M. Moghani, A. Murali, M. Noseworthy, A. Poddubny, N. Ratliff, W. Rehberg, C. Schwarke, R. Singh, J. L. Smith, B. Tang, R. Thaker, M. Trepte, K. V. Wyk, F. Yu, A. Millane, V. Ramasamy, R. Steiner, S. Subramanian, C. Volk, C. Chen, N. Jawale, A. V. Kuruttukulam, M. A. Lin, A. Mandlekar, K. Patzwaldt, J. Welsh, H. Zhao, F. Anes, J. Lafleche, N. Moënne-Loccoz, S. Park, R. Stepinski, D. V. Gelder, C. Amevor, J. Carius, J. Chang, A. H. Chen, P. de Heras Ciechomski, G. Daviet, M. Mohajerani, J. von Muralt, V. Reutskyy, M. Sauter, S. Schirm, E. L. Shi, P. Terdiman, K. Vilella, T. Widmer, G. Yeoman, T. Chen, S. Grizan, C. Li, L. Li, C. Smith, R. Wiltz, K. Alexis, Y. Chang, D. Chu, L. ". Fan, F. Farshidian, A. Handa, S. Huang, M. Hutter, Y. Narang, S. Pouya, S. Sheng, Y. Zhu, M. Macklin, A. Moravanszky, P. Reist, Y. Guo, D. Hoeller, and G. State (2025) Isaac lab: a gpu-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831. Cited by: §1, §5.1.1. [26] S. Nasiriany, T. Gao, A. Mandlekar, and Y. Zhu (2023) Learning and retrieval from prior data for skill-based imitation learning. In Conference on Robot Learning, p. 2181–2204. Cited by: §1, §2. [27] F. A. Oliehoek, C. Amato, et al. (2016) A concise introduction to decentralized pomdps. Vol. 1, Springer. Cited by: §3. [28] S. Omidshafiei, J. Pazis, C. Amato, J. P. How, and J. Vian (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. In International Conference on Machine Learning, p. 2681–2690. Cited by: §B.2.3. [29] C. Pan, X. Yang, Y. Li, W. Wei, T. Li, B. An, and J. Liang (2025) A survey of continual reinforcement learning. arXiv preprint arXiv:2506.21872. Cited by: §1, §2. [30] T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson (2020) Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research 21 (178), p. 1–51. Cited by: §2. [31] S. Rho, L. Smith, T. Li, S. Levine, X. B. Peng, and S. Ha (2025) Language guided skill discovery. In International Conference on Learning Representations, Vol. 2025, p. 87731–87752. Cited by: §1. [32] D. Rolnick, A. Ahuja, J. Schwarz, T. P. Lillicrap, and G. Wayne (2019) Experience replay for continual learning. In Advances in Neural Information Processing Systems, p. 350–360. Cited by: §1. [33] G. Saha and K. Roy (2023) Continual learning with scaled gradient projection. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37, p. 9677–9685. Cited by: §2. [34] J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel (2016) High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations, Cited by: §4.3. [35] A. Sharma, S. Gu, S. Levine, V. Kumar, and K. Hausman (2020) Dynamics-aware unsupervised discovery of skills. In International Conference on Learning Representations, Cited by: §2. [36] M. Shridhar, L. Manuelli, and D. Fox (2022) Cliport: what and where pathways for robotic manipulation. In Conference on Robot Learning, p. 894–906. Cited by: §1. [37] M. Stolle and D. Precup (2002) Learning options in reinforcement learning. In International Symposium on abstraction, reformulation, and approximation, p. 212–223. Cited by: §2. [38] P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, et al. (2017) Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296. Cited by: §2. [39] H. Tang, J. Obando-Ceron, P. S. Castro, A. Courville, and G. Berseth (2025) Mitigating plasticity loss in continual reinforcement learning by reducing churn. In International Conference on Machine Learning, p. 58883–58904. Cited by: §2. [40] E. Tuci, M. H. M. Alkilabi, and O. Akanyeti (2018) Cooperative object transport in multi-robot systems: a review of the state-of-the-art. Frontiers in Robotics and AI 5. Cited by: §1, §1, §2. [41] W. Wan, Y. Zhu, R. Shah, and Y. Zhu (2024) Lotus: continual imitation learning for robot manipulation through unsupervised skill discovery. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 537–544. Cited by: §1, §2. [42] J. Wang, J. Zhao, Z. Cao, R. Feng, R. Qin, and Y. Yu (2023) Multi-task multi-agent shared layers are universal cognition of multi-agent coordination. arXiv preprint arXiv:2312.15674. Cited by: §B.2.3. [43] L. Wang, X. Zhang, H. Su, and J. Zhu (2024) A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (8), p. 5362–5383. Cited by: §1, §2. [44] W. Wang, T. Yang, Y. Liu, J. Hao, X. Hao, Y. Hu, Y. Chen, C. Fan, and Y. Gao (2020) Action semantics network: considering the effects of actions in multiagent systems. In International Conference on Learning Representations, Cited by: §1. [45] X. Wang, Z. Han, Z. Liu, G. Li, J. Dong, B. Liu, L. Liu, and Z. Han (2026) Lifelong language-conditioned robotic manipulation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 18629–18637. Cited by: §2. [46] M. Wołczyk, M. Zając, R. Pascanu, Ł. Kuciński, and P. Miłoś (2021) Continual world: a robotic benchmark for continual reinforcement learning. In Advances in Neural Information Processing Systems, p. 28496–28510. Cited by: §2. [47] M. Wolczyk, M. Zając, R. Pascanu, Ł. Kuciński, and P. Miłoś (2022) Disentangling transfer in continual reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 35, p. 6304–6317. Cited by: §1, §2. [48] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu (2022) The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems 35, p. 24611–24624. Cited by: §B.2.3, §1, §2, §4.3. [49] L. Yuan, L. Li, Z. Zhang, F. Chen, T. Zhang, C. Guan, Y. Yu, and Z. Zhou (2023) Learning to coordinate with anyone. In Proceedings of the Fifth International Conference on Distributed Artificial Intelligence, p. 1–9. Cited by: §2. [50] L. Yuan, L. Li, Z. Zhang, F. Zhang, C. Guan, and Y. Yu (2024) Multiagent continual coordination via progressive task contextualization. IEEE Transactions on Neural Networks and Learning Systems 36 (4), p. 6326–6340. Cited by: §2. [51] L. Yuan, Z. Zhang, L. Li, C. Guan, and Y. Yu (2023) A survey of progress on cooperative multi-agent reinforcement learning in open environment. arXiv preprint arXiv:2312.01058. Cited by: §1, §1. [52] F. Zhang, C. Jia, Y. Li, L. Yuan, Y. Yu, and Z. Zhang (2023) Discovering generalizable multi-agent coordination skills from multi-task offline data. In International Conference on Learning Representations, Cited by: §2. [53] Z. Zhou (2022) Open-environment machine learning. National Science Review 9 (8), p. nwac123. Cited by: §1. Appendix A Algorithm and Hyperparameters A.1 Complete Algorithms Algorithm 1 Conquer retrieve-adapt-update procedure 0: Task stream =ℳ1,…,ℳTY=\M_1,…,M_T\; frozen SAG backbone θ; VLM-to-embedding model E(⋅)E(·); descriptor count K; duplicate threshold τ; hyperparameters ℋrl,ℋadapterH_rl,H_adapter 0: Skill library 0←∅D_0← 1: for t=1,…,Tt=1,…,T do 2: // Retrieval 3: Receive task ℳtM_t and pre-execution input xtinitx_t^init 4: qtinit←E(Promptinit(xtinit))q_t^init← E(Prompt_init(x_t^init)) 5: if t−1=∅D_t-1= then 6: Initialize current adapter ϕt _t from scratch 7: else 8: i⋆←argmini‖qtinit−ci‖2i ← _i\|q_t^init-c_i\|_2, where (si,ϕi,ci,mi)∈t−1(s_i, _i,c_i,m_i) _t-1 9: Initialize current adapter ϕt←ϕi⋆ _t← _i 10: end if 11: // Adaptation 12: Freeze θ and all adapters in t−1D_t-1 13: Train ϕt=(ϕtlora,ωtloc) _t=( _t^lora, _t^loc) and critic on ℳtM_t with clipped MAPPO 14: // Library update 15: Collect successful rollouts and generate descriptions dt,kk=1K\d_t,k\_k=1^K 16: Zt←E(dt,k)k=1KZ_t←\E(d_t,k)\_k=1^K, ct←1K∑k=1KE(dt,k)c_t← 1K _k=1^KE(d_t,k) 17: Record task metadata mtm_t 18: if t−1=∅D_t-1= then 19: t←(st,ϕt,ct,mt)D_t←\(s_t, _t,c_t,m_t)\ 20: else 21: j⋆←argminj‖ct−cj‖2j ← _j\|c_t-c_j\|_2, dt←‖ct−cj⋆‖2d_t←\|c_t-c_j \|_2 22: if dt<τd_t<τ then 23: t←t−1D_t _t-1 with entry j⋆j updated by (ϕt,ct,mt)( _t,c_t,m_t) 24: else 25: t←t−1∪(st,ϕt,ct,mt)D_t _t-1∪\(s_t, _t,c_t,m_t)\ 26: end if 27: end if 28: Use Algorithm 2 with tD_t to evaluate all tasks and record the growth-curve SR. 29: end for 30: return TD_T Algorithm 2 Conquer semantic skill execution and evaluation 0: Evaluation tasks eval=ℳ1,…,ℳJY^eval=\M_1,…,M_J\; skill library tD_t; current stage index t; frozen SAG backbone θ; VLM-to-embedding model E(⋅)E(·) 0: Number of evaluation episodes NevalN_eval 1: for j=1,…,Jj=1,…,J do 2: Set up task ℳjM_j and pre-execution input xjevalx_j^eval 3: qjeval←E(Promptinit(xjeval))q_j^eval← E(Prompt_init(x_j^eval)) 4: i⋆←argmini‖qjeval−ci‖2i ← _i\|q_j^eval-c_i\|_2, where (si,ϕi,ci,mi)∈t(s_i, _i,c_i,m_i) _t 5: Load policy πθ,ϕi⋆ _θ, _i 6: for p=1,…,Nevalp=1,…,N_eval do 7: Roll out πθ,ϕi⋆ _θ, _i on ℳjM_j and record success indicator uj,p∈0,1u_j,p∈\0,1\ 8: end for 9: Rt,j←1Neval∑p=1Nevaluj,pR_t,j← 1N_eval _p=1^N_evalu_j,p 10: end for 11: Gt←1J∑j=1JRt,jG_t← 1J _j=1^JR_t,j 12: return success vector Rt,jj=1J\R_t,j\_j=1^J and growth-curve SR GtG_t To further illustrate the process in Sec. 4.3, the overall retrieve-adapt-update workflow is shown in Algorithm 1: lines 2–10 implement semantic retrieval and adapter initialization, lines 11–13 perform reward-driven adapter training, and lines 14–27 update the skill library with the post-training descriptor. Line 28 invokes Algorithm 2 after each task stage to evaluate the currently available skill library on the full task set and record the all-task average SR used by the growth curve in Figure 5. These evaluations do not affect training or library updates. In Algorithm 2, evaluation-time semantic routing selects an executable skill for each task, repeated rollouts estimate its task success rate, and the resulting task-wise rates are averaged to obtain the stage-wise all-task SR. Final SR, FWT, and BWT are computed from the completed task-performance matrix as defined in Appendix B.2.1. A.2 Algorithm Hyperparameters Table 3 summarizes the training budget, MAPPO optimizer settings, policy architecture, semantic update rule, and adapter capacity used in the main simulation experiments. The interaction budget is computed as the number of parallel environments times rollout length times training iterations, yielding the total number of environment steps. The MAPPO rows specify the inner-loop adaptation optimizer in Eq. 5, while the SAG and decoder rows define the shared policy backbone used by all skills. The LoRA rows define the per-skill trainable adapter capacity. Table 3: Key settings and interaction budgets used by Conquer in the 14-task simulation benchmark. Setting Value Interaction budget per 1-/2-robot task 4096×48×500=98.34096× 48× 500=98.3M Interaction budget per 3-robot task 2048×48×500=49.22048× 48× 500=49.2M PPO epochs 44 Learning rate 3e−43e-4 value coef ℒVL_V 1.01.0 Entropy coef ℒHL_H 1e−51e-5 PPO clipping parameter 0.20.2 Trajectory descriptions per skill K 16 SAG projection dim 64 SAG output dim 64 SAG attention heads 4 Decoder hidden layer [256, 128, 64] Duplicate-gate threshold τ 0.125 LoRA rank r 8 LoRA scale α 16 LoRA dropout 0.05 Policy total parameters 165,475165,475 Trainable policy parameters per skill 9,3799,379 A.3 Vision Language Model, Text embedding model and Prompts The semantic library uses the same text-embedding space for pre-execution task queries and post-training skill descriptors. For trajectory-derived skill descriptions, we use gemini-robotics-er-1.6-preview as the vision-language model. All query and skill-description texts are embedded by Qwen/Qwen3-Embedding-0.6B, which produces 1024-dimensional text embeddings. Each stored skill descriptor is the mean of K=16K=16 description embeddings. Table 4: Semantic descriptor components used by Conquer. Component Setting Trajectory VLM gemini-robotics-er-1.6-preview Text embedding model Qwen/Qwen3-Embedding-0.6B, 1024-dimensional Skill descriptor input Side oblique view and top bird’s-eye view rollouts Table 5: Prompt templates used to build semantic retrieval queries and stored skill descriptors. Stage Core Instruction Retrieve (pre-adapt) Given robot-pushing image(s) and a task instruction, write one short retrieval instruction for the corresponding robot-dog object-pushing skill. Use a short plain-English sentence that explicitly states the robot count, terrain, object geometry, and intended pushing or moving action. This is a pre-execution instruction, so do not describe rollout outcomes such as sliding, tilting, or no clear motion. Use only the provided task/environment facts, preserve the robot count, terrain, and object geometry, and avoid difficulty words unless they appear in the task facts. Return [instruction] followed by one semi-formal sentence with at most 22 words. Update (post-adapt) Review one timestep from a robot-dog object-pushing rollout. Use the side oblique view first for object shape and terrain, and use the top bird’s-eye view for the number of distinct robot bodies and the contact layout. Then use a short plain-English sentence that explicitly states the robot count, terrain, object geometry, and intended pushing or moving action. Return [description] followed by one semi-formal sentence with at most 22 words. Appendix B Experimental Settings B.1 Task Stream, Physical Parameters, and Success Criteria The simulation benchmark contains 14 canonical push-to-goal tasks. The task stream first varies object geometry and terrain for one robot, then increases the team size to two and three robots for heavier cuboid and triangular-prism objects. Table 6 lists the task order and the main physical parameters, and Table 7 reports the corresponding pre-execution retrieval instructions. Table 6: Canonical 14-task stream used in simulation. Object sizes are length × width × height in meters. Stage Task ID Robots Object Size (m) Mass (kg) Terrain 1 1d-cube-flat 1 Cube 1.0×1.0×0.81.0× 1.0× 0.8 5 Flat 2 1d-cube-rough 1 Cube 1.0×1.0×0.81.0× 1.0× 0.8 5 Rough 3 1d-cuboid-flat 1 Cuboid 1.5×0.8×0.61.5× 0.8× 0.6 5 Flat 4 1d-cuboid-rough 1 Cuboid 1.5×0.8×0.61.5× 0.8× 0.6 5 Rough 5 1d-tri-flat 1 Triangular prism 1.2×1.2×0.81.2× 1.2× 0.8 5 Flat 6 1d-tri-rough 1 Triangular prism 1.2×1.2×0.81.2× 1.2× 0.8 5 Rough 7 2d-cuboid-flat 2 Cuboid 2.0×1.0×0.62.0× 1.0× 0.6 20 Flat 8 2d-cuboid-rough 2 Cuboid 2.0×1.0×0.62.0× 1.0× 0.6 20 Rough 9 2d-tri-flat 2 Triangular prism 1.5×1.5×0.81.5× 1.5× 0.8 20 Flat 10 2d-tri-rough 2 Triangular prism 1.5×1.5×0.81.5× 1.5× 0.8 20 Rough 11 3d-cuboid-flat 3 Cuboid 2.4×1.2×0.62.4× 1.2× 0.6 30 Flat 12 3d-cuboid-rough 3 Cuboid 2.4×1.2×0.62.4× 1.2× 0.6 30 Rough 13 3d-tri-flat 3 Triangular prism 1.8×1.8×1.01.8× 1.8× 1.0 30 Flat 14 3d-tri-rough 3 Triangular prism 1.8×1.8×1.01.8× 1.8× 1.0 30 Rough For each task, the pre-execution semantic query is generated from the task image and the task instruction. Table 7 lists the canonical instruction sentence used for each environment. Table 7: Pre-execution retrieval instructions for the 14 simulation tasks. Task ID Instruction 1d-cube-flat One robot dog on flat terrain pushes a yellow cube box toward the goal. 1d-cube-rough One robot dog on rough terrain pushes a yellow cube box toward the goal. 1d-cuboid-flat One robot dog on flat terrain pushes a yellow cuboid box toward the goal. 1d-cuboid-rough One robot dog on rough terrain pushes a yellow cuboid box toward the goal. 1d-tri-flat One robot dog on flat terrain pushes a yellow triangular-prism box toward the goal. 1d-tri-rough One robot dog on rough terrain pushes a yellow triangular-prism box toward the goal. 2d-cuboid-flat Two robot dogs on flat terrain push a yellow cuboid box toward the goal. 2d-cuboid-rough Two robot dogs on rough terrain push a yellow cuboid box toward the goal. 2d-tri-flat Two robot dogs on flat terrain push a yellow triangular-prism box toward the goal. 2d-tri-rough Two robot dogs on rough terrain push a yellow triangular-prism box toward the goal. 3d-cuboid-flat Three robot dogs on flat terrain surround and push a yellow cuboid box toward the goal. 3d-cuboid-rough Three robot dogs on rough terrain surround and push a yellow cuboid box toward the goal. 3d-tri-flat Three robot dogs on flat terrain surround and push a yellow triangular-prism box toward the goal. 3d-tri-rough Three robot dogs on rough terrain surround and push a yellow triangular-prism box toward the goal. Policy observation interface. All simulation tasks use the same SAG-structured per-agent observation interface. For robot i, the policy input is organized as oi=[Si,Ai,1,…,Ai,n−1,Gi]o_i=[S_i,A_i,1,…,A_i,n-1,G_i], where SiS_i is a self token, Ai,jA_i,j is an ally token for another robot, and GiG_i is the object-goal token. The self token contains body-frame base velocity, body-frame angular velocity, projected gravity, joint positions relative to the default pose, joint velocities, local body-object contact information, and terrain height scan. Each ally token contains the ally robot’s relative position, relative velocity, relative orientation, and nearest box-face geometry in the ego robot frame. The object-goal token contains the box relative pose and velocity, target relative position, box-to-target vector and distance, box size and mass, hold-state indicator, and nearest-face geometry. The resulting per-agent observation dimensions are 265, 290, and 315 for the one-, two-, and three-robot simulation tasks, respectively. Table 8 summarizes the token layout. Table 8: SAG per-agent observation interface. Token dimensions include internal feedback fields used by the policy but not expanded in the table. Token Dim. Contents Self 236 Body-frame base velocity (3) Body-frame angular velocity (3) Projected gravity (3) Joint position relative to the default pose (12) Joint velocity (12) Body-object contact force norm (1) Body-object contact normal in the body frame (3) Previous joint-position action command (12) Terrain height scan (187) Ally 25 each Ally relative position (3) Ally relative velocity (3) Ally relative orientation quaternion (4) Nearest box-face normal in the ego frame (3) Ally previous joint-position action command (12) Object-goal 29 Box relative position (3) Box velocity (3) Box relative orientation quaternion (4) Target relative position (3) Box-to-target vector (3) Planar box-target distance (1) Hold-state indicator (1) Box mass (1) Box half extents (3) Nearest box-face normal (3) Nearest-face signed distance (1) Box angular velocity (3) Policy action space. In the non-hierarchical simulation benchmark, each robot outputs a 12-dimensional joint-position action, one scalar for each actuated Go2 joint. The action is applied through a joint-position controller with a scale of 0.25 and the default standing pose as the offset. The simulator runs at 200Hz with action decimation 4, so the policy acts at 50Hz. Environment and episode setup. All tasks use the official Unitree Go2 USD robot model and a single movable object. Flat tasks use a planar mesh terrain with static friction 1.0 and dynamic friction 0.95. Rough tasks use an 8m × 8m random height-field terrain with 1–6cm height noise and static/dynamic friction 1.0. Each episode lasts at most 20s, corresponding to 1000 control steps. Target sampling and success criterion. The target command samples a goal point for the object center in the horizontal plane. For one-robot tasks, the target is sampled on a 3m-radius cylinder around the object. For two- and three-robot tasks, the target is sampled from the rectangular range x,y∈[−3,3]x,y∈[-3,3]m. A rollout is counted as successful only after the object center remains within 0.2m of the target in the xyxy plane for 50 consecutive control steps. At 50Hz this corresponds to approximately 1s of maintained success. This maintained-success event, rather than the dense shaping reward, defines the SR metric used in the main text. Figure 7 provides the corresponding top-view snapshots for the same task stream. Training rewards. The dense reward is used only for reward-driven policy optimization; the reported SR is computed from the maintained-success event described above. Each robot receives the same reward-term set from the non-hierarchical task factory, and task-level terms are shared by assigning the same object-progress and event rewards to all robots. Table 9 lists the reward terms used by the 14-task non-hierarchical benchmark. Table 9: Reward terms used by the non-hierarchical 14-task simulation benchmark. The reward manager multiplies each raw term by its weight and by the environment time step; event terms internally compensate for this time-step scaling. Term Weight Computation and role lin_vel_z_l2 −2.0-2.0 Squared vertical base velocity penalty for suppressing jumping. ang_vel_xy_l2 −0.05-0.05 Squared roll/pitch angular velocity penalty for stabilizing the base. flat_orientation_l2 −2.0-2.0 Squared horizontal projected-gravity penalty for keeping the body upright. dof_torques_l2 −2.0×10−4-2.0×10^-4 Squared joint-torque penalty for reducing aggressive actuation. dof_acc_l2 −2.5×10−7-2.5×10^-7 Squared joint-acceleration penalty for smoother motion. action_rate_l2 −0.01-0.01 Squared action-difference penalty between consecutive policy outputs. feet_air_time 0.010.01 Foot-air-time reward with a 0.50.5s threshold from the contact sensor. alive −1.0-1.0 Constant per-step survival cost, implemented as a unit alive term with negative weight. lin_vel_heading_alignment 0.50.5 Rewards forward body-frame velocity alignment when the robot moves. adaptive_heading 2.52.5 Outside the object coverage radius, rewards velocity aligned with the selected unfinished object. adaptive_activity 1.01.0 Inside the selected object’s coverage radius, rewards robot planar speed to encourage active pushing contact. encourage_robot_height 1.51.5 Rewards base height up to 0.50.5m to encourage standing posture. box_to_target_progress 20.020.0 Signed object-to-target progress: positive when the object moves closer and negative when it moves away; disabled after maintained success. maintain_box 20.020.0 One-time maintained-success event for the object-command pair. task_success 10.010.0 Shared terminal success event when the task-success termination is triggered. robot_falling −10.0-10.0 Shared terminal penalty when any robot-falling termination is triggered. This maintained-success event, rather than the dense shaping reward, defines the SR metric used in the main text. Figure 7 provides the corresponding top-view snapshots for the same task stream. Figure 7: Top-view snapshots of the 14 canonical simulation tasks. Panels follow the task order in Table 6. B.2 Metrics and Baselines B.2.1 Metrics Following the task-performance matrix protocol of GEM [21], we use a task-performance matrix to evaluate the continual-learning process. Let T=14T=14 be the number of tasks and let Ri,jR_i,j denote the success rate on task j after the method has completed training on task i. During evaluation, each task is run with NevalN_eval parallel environments. If uj,p∈0,1u_j,p∈\0,1\ is the success indicator of the p-th rollout on task j, then Ri,j=1Neval∑p=1Nevaluj,p.R_i,j= 1N_eval _p=1^N_evalu_j,p. (6) The success criterion is defined in Appendix B.1. Unless otherwise stated, SR, FWT, and BWT are reported as percentages. Final success rate. Final SR measures the average task coverage after completing the full task stream. For methods that produce a complete continual-learning matrix, it is the mean of the last matrix row: FinalSR=1T∑j=1TRT,j.Final\ SR= 1T _j=1^TR_T,j. (7) For Multitask, PSEC, and HiSSD, which do not follow the setting of online continual learning, we report only the average success rate over the 14 tasks after training. We therefore do not compute FWT or BWT for these methods. Forward transfer. FWT measures performance on future tasks before those tasks are trained. Let bjb_j be the zero-shot success rate of the frozen shared backbone on task j. We compute FWT from the upper-triangular part of the task matrix: FWT=2T(T−1)∑1≤i<j≤T(Ri,j−bj).FWT= 2T(T-1) _1≤ i<j≤ T(R_i,j-b_j). (8) This metric uses only entries where task j has not yet been trained and subtracts the corresponding backbone baseline to reduce the effect of task difficulty. Higher FWT indicates stronger initialization or retrieval transfer to future tasks. Backward transfer. BWT measures how old-task performance changes after the full sequence is completed: BWT=1T−1∑i=1T−1(RT,i−Ri,i).BWT= 1T-1 _i=1^T-1(R_T,i-R_i,i). (9) Here Ri,iR_i,i is the success rate immediately after learning task i, and RT,iR_T,i is the success rate on the same task after finishing all T tasks. BWT near zero indicates that old-task performance is maintained, negative BWT indicates forgetting, and positive BWT indicates improvement on old tasks after later training. Because BWT compares two entries from the same task, no backbone-baseline subtraction is applied. Growth curve and repeated evaluations. The growth curve reports the average success rate after each stage: Gi=1T∑j=1TRi,j.G_i= 1T _j=1^TR_i,j. (10) It is used to visualize the performance trajectory over the task stream, while the main comparison is based on Final SR, FWT, and BWT. When repeated evaluation runs are available, tables report the mean and sample standard deviation. B.2.2 Evaluation-protocol For Conquer, continual-learning, and non-continual-learning methods used in this paper, we conducted independent training runs with three different random seeds and evaluated each run in 128 parallel simulation environments, which is equivalent to 128 independent repeated trials. The detailed evaluation procedure is provided in Algorithm 2. B.2.3 Baseline implementations Multitask is a same-budget joint-training reference inspired by multi-task MARL training protocols [28, 42, 24]. Since Isaac Lab evaluates one registered task instance per training process, we implement Multitask as task-rotating joint training: one shared policy is trained over 70 rounds, and each round trains one task for 100 iterations before passing the checkpoint to the next round. Each of the 14 tasks appears in five rounds, giving 500 iterations per task, matching the per-task budget used by the continual methods. This reference does not use LoRA skill adapters or semantic retrieval. Fine-tune is the sequential full-parameter baseline. It follows the same task order as Conquer, initializes each new task from the previous task checkpoint, and updates all policy parameters with the same clipped MAPPO optimizer [48]. LoRA adapters, semantic retrieval, replay, and regularization are disabled, so Fine-tune measures the effect of unconstrained sequential adaptation. EWC extends the same sequential full-parameter setting with elastic weight consolidation [18]. After each task, we estimate a diagonal empirical Fisher matrix from rollout samples and store the task-ending parameter vector. For later tasks, the MAPPO objective is augmented with a quadratic penalty weighted by the accumulated Fisher estimate. We use the online form of EWC with Fisher decay, so earlier tasks remain represented in the regularizer while the method continues to adapt to the current task. PSEC is included as a task-aware parameter-space skill composition reference [20]. Our adapted implementation builds parameter-space compositions from trained LoRA skill primitives and uses task information to select or combine skill parameters at evaluation time. Because its task-aware composition protocol does not define a stage-wise, task-agnostic evaluation interface required for FWT/BWT, we report only its final average SR over the 14 tasks. HiSSD is included as a skill-discovery and hierarchical distillation baseline [19]. We use the adapted implementation for this benchmark, which trains a hierarchical skill representation and shared policy from the 14-task data and is evaluated with the same final-step success criterion as the other methods. Since this offline distillation protocol does not generate after-each-task checkpoints along the online task stream, we report its final average SR only. B.3 Real-Robot Deployment Details Sec. 5.2.1 reports the hardware platform, motion-capture setup, object masses, and rollout-level results. This section additionally provides the details of simulation-side adaptation and command-interface details used before deployment. Domain-randomization setting. Before deployment, the hierarchical policies are adapted in simulation with randomized contact dynamics, object properties, external perturbations, and command-interface effects. All deployment adaptation tasks use the high-friction box-contact setting. Table 10 summarizes the randomization factors. Table 10: Domain randomization used for real-robot deployment adaptation. Factor Randomization Notes Robot material Static friction [0.7,1.0][0.7,1.0], dynamic friction [0.5,0.8][0.5,0.8] Sampled at startup Box material (1.4,1.35)(1.4,1.35), (1.2,1.15)(1.2,1.15), or (1.0,0.95)(1.0,0.95) Static/dynamic friction pair Box mass Multiplicative scale [0.7,1.3][0.7,1.3] Sampled at reset Robot base mass Additive offset [−1,3]kg[-1,3]kg Sampled at startup External push Planar velocity perturbation vx,vy∈[−0.5,0.5]m/sv_x,v_y∈[-0.5,0.5]m/s Every 10–15s Command delay 1–6 low-level locomotion ticks About 20–120ms at 50Hz Base linear velocity obs. Gaussian noise, σ=0.0025m/sσ=0.0025m/s Low-level policy observation Base angular velocity obs. Gaussian noise, σ=0.005rad/sσ=0.005rad/s Low-level policy observation Projected gravity obs. Gaussian noise, σ=0.001σ=0.001 Low-level policy observation Joint position obs. Gaussian noise, σ=0.0005radσ=0.0005rad Low-level policy observation Joint velocity obs. Gaussian noise, σ=0.005rad/sσ=0.005rad/s Low-level policy observation Velocity gain Scale [0.7,1.0][0.7,1.0] Per-axis, per-episode Velocity bias Gaussian bias, σ=0.03m/sσ=0.03m/s Per-axis, per-episode Command deadzone [0,0.1]m/s[0,0.1]m/s Per-episode Command lag α∈[0.3,1.0]α∈[0.3,1.0] First-order lag Yaw coupling [−0.1,0.1][-0.1,0.1] Lateral-to-yaw coupling The one-robot cuboid adaptation uses a real-size cuboid proxy of 0.714m×0.524m×0.288m0.714m× 0.524m× 0.288m with a 5kg nominal simulation mass. The two- and three-robot recipes use a 1.40m×1.00m×0.50m1.40m× 1.00m× 0.50m cuboid with 20kg and 30kg nominal simulation masses, respectively. The four-robot large-cuboid recipe uses a 2.10m×1.40m×0.50m2.10m× 1.40m× 0.50m cuboid with a 60kg nominal simulation mass. Multi-robot deployment adaptation uses ring-style robot initialization, target sampling around the box with an 8m cylinder radius, and reset-time yaw randomization for both robots and the box. Deployment model and command waveforms. The deployed controller is a high-level decision-layer policy. It maps motion-capture-based observations to per-robot body-frame velocity commands, while the onboard Unitree Go2 controller tracks these commands at the locomotion level (for deployment, motion-capture states and onboard Go2 proprioceptive measurements are converted into the corresponding ego-frame SAG observations, as detailed in Appendix B.1). The command-waveform plots report the post-clamp commands streamed to hardware. The conservative command envelope is vx∈[−0.4,0.75]v_x∈[-0.4,0.75]m/s, vy∈[−0.25,0.25]v_y∈[-0.25,0.25]m/s, and vyaw∈[−0.5,0.5]v_yaw∈[-0.5,0.5]rad/s. Saturated waveform segments therefore indicate the deployed safety clamp rather than an unconstrained policy output. (a) One-Go2 cuboid (b) Two-Go2 cuboid (c) Three-Go2 cuboid (d) Four-Go2 large cuboid Figure 8: Post-clamp high-level command waveforms from representative real-robot deployment logs. The plotted channels are the body-frame velocity commands streamed to the Go2 low-level controller. Appendix C Additional Results C.1 Semantic-Access Backtest This subsection reports the semantic-access backtest used in the main comparison. After the full task stream is completed, old skill parameters are preserved by parameter isolation, but the system still needs to route a pre-execution task query to the appropriate library entry. The backtest therefore evaluates whether semantic retrieval can recover the intended executable skill after the library has been built. We construct three English pre-execution retrieval instructions for each of the 14 tasks, giving 42 semantic-access queries in total. Each query is encoded by the same text-embedding model used by Conquer and matched to the stored skill-description centroids by nearest-neighbor distance. If the nearest neighbor is the target task’s own skill, the query introduces no additional routing error. If the nearest neighbor is another library skill, we evaluate that selected skill directly on the target task and compare its SR with the target task’s own stored skill. Under the current prompt setting and task stream, semantic access does not create any severe routing failure. Among the 42 queries, 41 retrieve the target task’s own skill, corresponding to a self-retrieval rate of 97.6%. The only non-self retrieval occurs for the second rewrite of 1d-cuboid-rough, which is routed to 2d-cuboid-rough. For this source-target pair, the target task’s own skill reaches 99.2% average SR, while the retrieved 2d-cuboid-rough skill reaches 97.9% average SR. Thus, for the current benchmark and prompt interface, semantic access does not change the main conclusion about retained skill availability. This mismatch also points to the limitation discussed in Sec. 6: the nearest semantic neighbor is not necessarily the best zero-shot transfer source, especially when the distance is defined by a prior text embedding model. The prior semantic distance remains useful, however. As the skill library grows, methods that select skills with VLM reasoning, rollout trajectories, or empirical success rates must search over an increasing number of entries. Semantic distance can reduce this search to a smaller candidate set before those more expensive selectors are applied. C.2 Semantic Transfer Case-Study Details Following the analysis in Sec. 5.1.3, we further evaluate zero-shot transfer over all 14 tasks to test whether semantic retrieval provides better initialization candidates. For each target task, we remove the skill trained on that target environment, use Conquer’s retrieval rule to select the three nearest semantic source skills, and compare their transfer SR with the average over the remaining source skills. Figure 9 shows that the Top-3 semantic sources average 83.8% SR, while the remaining sources average 66.3%, supporting the conclusion that semantic similarity improves initialization quality on average. Table 11 gives the detailed source ranking for the 3d-tri-rough case visualized in Figure 5. The reported distance is computed between the target task-description query and each source skill descriptor. Table 11: Semantic source ranking for 3d-tri-rough. Distances are query-to-source descriptor L2 values; SR values are zero-shot source-skill transfer success rates. Group Source skill Query rank / L2 Transfer SR (%) Top-3 3d-tri-flat 1 / 0.369 70.3 Top-3 3d-cuboid-rough 2 / 0.418 49.2 Top-3 2d-tri-rough 3 / 0.430 60.9 Best other 2d-tri-flat 5 / 0.433 68.0 Top-3 mean – ranks 1–3 60.2 Other-source mean – ranks 4–13 34.4 Figure 9: Leave-one-target-out semantic transfer summary over all 14 tasks. C.3 Per-task Final Success Records Table 12 reports the final success rate on each benchmark task. For continual methods, final success is measured after the complete 14-task stream. For the other baselines, we report their corresponding per-task final evaluation. Notably, PSEC and Scratch are parameter-isolation baselines that do not include a retrieval module. Accordingly, the final success rates reported in the table are computed under a task-aware selection assumption for these two baselines. The Backbone column gives zero-shot performance before any task-specific adaptation. Table 12: Per-task final SR. Values are percentages. Task Conquer (%\%) Multitask (%\%) EWC (%\%) Fine-tune (%\%) Scratch (%\%) PSEC (%\%) HiSSD (%\%) Backbone (%\%) 1d-cube-flat 99.5 93.2 94.9 71.1 100.0 99.2 97.7 68.0 1d-cube-rough 99.5 94.5 94.9 32.8 98.4 98.4 87.5 62.5 1d-cuboid-flat 99.7 100.0 94.5 32.0 100.0 99.2 100.0 77.3 1d-cuboid-rough 99.5 95.1 99.2 18.0 97.7 99.2 94.5 64.8 1d-tri-flat 99.5 99.5 99.2 39.8 100.0 100.0 98.4 14.8 1d-tri-rough 97.1 85.9 88.7 21.9 82.0 96.1 85.2 18.8 2d-cuboid-flat 100.0 100.0 98.4 84.4 99.2 100.0 100.0 99.2 2d-cuboid-rough 97.7 97.1 88.3 85.2 92.2 96.1 89.8 80.5 2d-tri-flat 98.7 98.2 84.4 93.0 96.1 95.3 91.4 28.1 2d-tri-rough 80.2 80.7 53.1 82.8 54.7 79.7 50.0 18.8 3d-cuboid-flat 96.6 97.7 96.1 98.4 99.2 97.7 98.4 95.3 3d-cuboid-rough 94.0 92.7 86.3 98.4 78.9 96.1 78.1 63.3 3d-tri-flat 97.9 96.1 78.5 96.1 97.7 93.8 93.8 71.9 3d-tri-rough 78.4 78.9 58.2 91.4 53.1 75.0 46.1 40.6