Paper deep dive
Trajectory-Level Automatic Curriculum Learning for Legged Locomotion on Unstructured Terrain
Rocky Liu, Tengyu Liu, Baoxiong Jia, Fangwei Zhong, Xinyi Tong, Hongzhao Xie, Siyuan Huang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/23/2026, 1:32:47 AM
Summary
The paper introduces TACL (Trajectory-level Automatic Curriculum Learning), a framework for training legged robot locomotion policies on unstructured terrain. Unlike existing methods that rely on handcrafted or parameterized terrain templates, TACL generates training tasks directly from raw unstructured terrain maps. It employs a closed-loop system where an evaluator, trained via a Conditional Variational Autoencoder (CVAE) to predict trajectory difficulty based on transition contexts, guides a sampler using Metropolis-Hastings MCMC to propose capability-matched trajectories. Experiments demonstrate that TACL significantly improves trajectory success rates compared to direct training and handcrafted curriculum learning, enhancing generalization and robustness.
Entities (11)
Relation Signals (8)
TACL → appliedto → Unstructured Terrain
confidence 95% · generates training tasks directly from unstructured terrain maps
TACL → uses → CVAE
confidence 95% · We use the conditional variational autoencoder (CVAE) to encode these high-dimensional transition contexts
TACL → uses → Metropolis-Hastings MCMC
confidence 95% · the sampler uses Metropolis-Hastings Markov Chain Monte Carlo (MH-MCMC) to search for new waypoint trajectories
TACL → deployedon → Unitree Go2
confidence 90% · We deploy the student policy on a Unitree Go2
TACL → evaluatedon → Map_A
confidence 90% · We conduct experiments on two terrain maps. Map_A is an unstructured training map
TACL → evaluatedon → Map_HCL
confidence 90% · Map_HCL is a handcrafted curriculum map procedurally instantiated from the Extreme Parkour codebase
TACL → outperforms → Handcrafted Curriculum Learning (HCL)
confidence 90% · Compared with handcrafted curriculum learning, our method improves success rate by 18.5%
TACL → simulatedin → Isaac Gym
confidence 90% · All policies are trained and evaluated in parallel simulation using Isaac Gym
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Training locomotion policies for complex unstructured terrain requires a curriculum to avoid early exploration failures. However, since unstructured terrain lacks explicit difficulty ordering for curriculum design, existing methods resort to heuristic curricula over parameterized terrains. This abstraction limits generalization, as policies can overadapt to near-fixed perceptual patterns. To address this, we propose \textbf{\ourname{}}, an \textbf{T}rajectory-level \textbf{A}utomatic \textbf{C}urriculum \textbf{L}earning framework that generates training tasks directly from unstructured terrain maps. At each curriculum update, the evaluator learns a difficulty function for the current policy that maps a given trajectory task to a difficulty score. The sampler then proposes new trajectories guided by the learned evaluator as the curriculum for the next policy update. This forms a closed loop in which the curriculum is iteratively matched to the evolving policy. Quantitative and qualitative experiments show that \ourname{} continuously provides effective curricula on unstructured terrain, improving trajectory success rate by \(56.3\%\) over direct training without curriculum. Compared with handcrafted curriculum learning, our method improves success rate by \(18.5\%\) on the hardest terrain tasks and by up to \(39.74\%\) when evaluating traversal from diverse approach directions on the same obstacle type.
Tags
Links
- Source: https://arxiv.org/abs/2608.16164v1
- Canonical: https://arxiv.org/abs/2608.16164v1
Trouble viewing inline? Open PDF directly →
Full Text
39,487 characters extracted from source content.
Expand or collapse full text
Trajectory-Level Automatic Curriculum Learning for Legged Locomotion on Unstructured Terrain Rocky Liu Affiliation: National Key Laboratory of General Artificial Intelligence, BIGAI Tengyu Liu Affiliation: National Key Laboratory of General Artificial Intelligence, BIGAI Baoxiong Jia Affiliation: National Key Laboratory of General Artificial Intelligence, BIGAI Fangwei Zhong Affiliation: Beijing Normal University Xinyi Tong Affiliation: National Key Laboratory of General Artificial Intelligence, BIGAI Affiliation: Communication University of China Hongzhao Xie Affiliation: Shanghai Jiao Tong University Siyuan Huang 🖂 Affiliation: National Key Laboratory of General Artificial Intelligence, BIGAI Abstract Training locomotion policies for complex unstructured terrain requires a curriculum to avoid early exploration failures. However, since unstructured terrain lacks explicit difficulty ordering for curriculum design, existing methods resort to heuristic curricula over parameterized terrains. This abstraction limits generalization, as policies can overadapt to near-fixed perceptual patterns. To address this, we propose TACL, an Trajectory-level Automatic Curriculum Learning framework that generates training tasks directly from unstructured terrain maps. At each curriculum update, the evaluator learns a difficulty function for the current policy that maps a given trajectory task to a difficulty score. The sampler then proposes new trajectories guided by the learned evaluator as the curriculum for the next policy update. This forms a closed loop in which the curriculum is iteratively matched to the evolving policy. Quantitative and qualitative experiments show that TACL continuously provides effective curricula on unstructured terrain, improving trajectory success rate by 56.3%56.3\% over direct training without curriculum. Compared with handcrafted curriculum learning, our method improves success rate by 18.5%18.5\% on the hardest terrain tasks and by up to 39.74%39.74\% when evaluating traversal from diverse approach directions on the same obstacle type. Figure 1: Our method generates effective curricula on unstructured terrain. As training progresses, the sampled tasks evolve with the policy capability, moving toward taller platforms, harder approach angles, descents from elevated structures, and gap-crossing between high platforms. Experiments show that our method improves success rates across terrain tasks, learns more robust behaviors such as platform climbing from diverse approach directions, and supports sim-to-real transfer. Keywords: Robot Locomotion, Curriculum Learning 1 Introduction Learning robust locomotion policies for complex terrain with deep reinforcement learning (DRL) [1]. Among the success, curriculum learning is a key role to avoid conservative policies caused by early-stage exploration failures. However, unstructured maps do not provide an explicit criterion for judging which tasks are suitable for the current policy or how task difficulty should progress. Existing methods address this by imposing structured curriculum spaces. Handcrafted curriculum learning (HCL) abstracts complex terrain into several parameterized geometric templates [2, 3, 4, 5, 6, 7], e.g., gap terrains, instantiates them along heuristic difficulty progressions, and pairs them with manually specified trajectories for policy training. Despite empirical success in structured settings, HCL requires substantial manual effort and produces limited task distributions, making policies overly adapted to the nearly fixed perceptual patterns along predefined trajectories. Automatic curriculum learning (ACL) reduces the need to manually specify difficulty progressions by adapting task sampling using policy performance or learning signals [8, 9, 10, 11], but it still generate curricula within the same parameterized template spaces rather than directly on unstructured maps. In both HCL and ACL, this abstraction causes terrain information loss and narrows the task distribution, leaving the policy underexposed to the diverse trajectory tasks encountered in unstructured environments. To avoid these changalles that construct curricula directly on unstructured maps, the missing criterion must be learned at the trajectory level. Unlike HCL, which heuristically defines difficulty through predefined terrain-parameter levels, trajectory difficulty on an unstructured map depends on the terrain variations encountered along the path, the transition context, and the current policy capability. The key challenge is therefore to evaluate trajectory difficulty under current policy and sample learnable trajectory-level tasks on a map, without manually designing a terrain parameterization. In this paper, we propose TACL, an automated curriculum learning framework capable of generating trajectory tracking curricula that match the evolving competence of the policy, enabling effective learning directly on raw unstructured maps. The framework comprises two primary modules: the evaluator and the sampler. At each curriculum update, we roll out the current policy on trajectories randomly sampled from the map, collect success and failure outcomes, and represent each traversed segment by its transition context. We use the conditional variational autoencoder (CVAE) [12] to encode these high-dimensional transition contexts into a compact latent space. The evaluator then learns a policy-conditioned difficulty function that predicts the difficulty of each trajectory from its encoded transition context. Guided by the learned evaluator, the sampler uses Metropolis-Hastings Markov Chain Monte Carlo (MH-MCMC) [13, 14] to search for new waypoint trajectories whose predicted difficulty falls within a suitable range for the current policy. The generated trajectories are used as the curriculum for the next policy update, forming a closed loop in which tasks become progressively harder as the policy improves. We evaluate our method on both an unstructured terrain map and the handcrafted curriculum map from Extreme Parkour [3], comparing against random trajectory sampling and the HCL baseline. Quantitative and qualitative results show that our method generates effective curricula directly on unstructured terrain, achieving more roboust terrain traversal performance to HCL while improving trajectory-level generalization by 39.5%39.5\%. Overall, our contributions can be summarized as follows: • We propose TACL, a closed-loop framework for robot locomotion policy that automatically generates curricula directly on unstructured terrain maps, removing the need for handcrafted terrain curricula and manually parameterized terrain templates. • We design a context-aware transition encoding and evaluator that accurately predicts task difficulty for current policy, providing reliable guidance for sampling capability-matched training tasks. • Extensive experiments demonstrate that our framework generates effective curricula directly on unstructured terrain maps, substantially improving policy capability over non-curriculum training and achieving more robust traversal than HCL across diverse approach directions. 2 Related Works Handcrafted Curriculum Learning in Robot Locomotion. Handcrafted curriculum learning reduces the mismatch between task difficulty and policy capability by manually scheduling training tasks from easy to hard [15]. In standard locomotion, this is often done by expanding command ranges, such as target velocity, heading, or motion amplitude, enabling skills such as basic locomotion [16, 17, 18], high-speed trotting [19, 20], and jumping [21]. For complex terrain traversal, prior works typically handcraft both terrain curricula and trajectory tasks. They parameterize obstacle families such as stairs, gaps, hurdles, and ramps, and advance the policy through manually specified difficulty sequences using heuristic performance thresholds [16, 2, 3, 4, 22, 23, 5, 6]. Other parkour-oriented methods rely on handcrafted constraints, terrain-specific experts, or predefined waypoint trajectories to learn maneuvers such as crossing gaps, climbing platforms, or traversing ramps [24, 25]. While effective within designed terrain families, these curricula require human expertise to define terrain categories, difficulty progressions, success thresholds, and often the training trajectories themselves. The resulting task distributions can be limited or nearly fixed, making policies overly adapted to specific waypoint paths and perceptual patterns, which reduces transfer to regenerated or less structured terrain instances. Automatic Curriculum Learning in Robot Locomotion. Automatic curriculum learning aims to reduce manual curriculum design by adaptively selecting training tasks according to the policy’s learning state. A central challenge is to identify tasks that are neither too easy nor too difficult for the current policy. Existing approaches use reward prediction, temporal-difference (TD) error, regret, surprise, or learning progress as task-selection signals. HACL learns a reward predictor from historical data for task sampling [26], while unsupervised environment design methods generate increasingly difficult environments through adversarial or evolutionary processes [27, 28, 29, 30]. TD-error- or surprise-based methods prioritize tasks with high prediction error [31, 32, 33], but such signals can mix learnable uncertainty with irreducible noise and trap the policy in unproductive regions [34, 35]. Learning-progress methods instead prioritize task regions where policy performance improves most rapidly [36, 37]. Other studies automate curriculum search within parameterized terrain spaces, where tasks are generated by adjusting terrain sampling probabilities, filtering geometric parameters, discretizing stepping-stone configurations, or updating curriculum ranges in a learned latent space [8, 9, 10, 11]. While these methods automate task selection, they still rely on manually parameterized terrain or task spaces, limiting their applicability to curriculum generation on unstructured maps. 3 Method As shown in Fig. 2, our framework forms a closed-loop curriculum over trajectory-following tasks. Each curriculum iteration, indexed by k, consists of three coupled processes: (a) policy learning and rollout collection, (b) difficulty evaluator training, and (c) evaluator-guided task generation. Given the current policy πθk _ _k, rollouts on sampled tasks provide transition-level outcomes for training the evaluator kD_k. The updated evaluator guides the sampler kG_k to generate the next task set k+1S_k+1, which is used for the subsequent policy update. Figure 2: Framework overview. (a) The policy learns to follow waypoint trajectories, where a trajectory T is converted into velocity commands ctc_t for policy training. (b) The current policy is rolled out on randomly sampled trajectories to collect transition descriptors ξi _i and success/failure labels yiy_i. The CVAE encoder qϕq_φ maps each descriptor to a latent code ziz_i, and the evaluator predicts its policy-conditioned difficulty did_i. (c) Guided by the updated evaluator, the sampler uses MH-MCMC to generate capability-matched trajectories by sampling for transitions whose predicted difficulty did_i matches the target difficulty dtard^tar. The generated trajectories are used for the next policy update, closing the curriculum loop. Fire and snowflake icons indicate training and inference modes. 3.1 Problem Formulation We formulate our method as an automatic curriculum learning framework that constructs training task sets for legged locomotion on unstructured terrain. Given a terrain map ℳM, at curriculum iteration k, the framework provides a set of trajectory-following tasks k=jS_k=\T_j\ for policy training. Each task =p0,p1,…,pNT=\p_0,p_1,…,p_N\ specifies one rollout episode, where pi∈ℝ2p_i ^2 is a waypoint on ℳM. The robot starts from p0p_0 and sequentially tracks p1,…,pNp_1,…,p_N. Consecutive waypoints define sub-trajectories τi=(pi,pi+1) _i=(p_i,p_i+1), for i=0,…,N−1i=0,…,N-1, where executing τi _i means reaching pi+1p_i+1 from pip_i. The policy is trained with DRL: at time t, the actor receives an observation sts_t, samples an action at∼πθ(⋅∣st)a_t _θ(· s_t), and obtains reward rtr_t after the action is applied. We use the same waypoint-tracking objective and the same asymmetric actor-critic design as the teacher-stage setup of Extreme Parkour [3], details of observations, rewards, and policy hyperparameters are provided in Appendix. To evaluate and generate curriculum tasks, we represent each sub-trajectory by a transition context descriptor ξi=(Hp,Mp,Hc,Mc,δi) _i=(H_p,M_p,H_c,M_c, _i). Here (Hp,Mp)(H_p,M_p) denotes the height patch and valid-length mask of the previous sub-trajectory τi−1=(pi−1,pi) _i-1=(p_i-1,p_i), and (Hc,Mc)(H_c,M_c) denotes the corresponding pair of the current sub-trajectory τi=(pi,pi+1) _i=(p_i,p_i+1), and δi _i is the relative yaw angle from τi−1 _i-1 to τi _i. The height patches satisfy Hp,Hc∈ℝL×WH_p,H_c ^L× W, and the masks satisfy Mp,Mc∈0,1L×WM_p,M_c∈\0,1\^L× W. Each patch is cropped along its waypoint direction and start-referenced by subtracting the mean elevation near the beginning of its centerline, while the mask marks the cells covered by the actual transition length. For i=0i=0, the previous context is set to a flat patch, a zero mask, and zero yaw. 3.2 Difficulty Evaluator Training A curriculum update is triggered once the policy reaches an average waypoint-completion threshold κwp _wp on the current tasks, or once the update period KupdK_upd is met. We then evaluate the current policy πθk _ _k on feasible trajectories randomly sampled from ℳM, collecting transition-level rollout outcomes for evaluator training. For each evaluated sub-trajectory τi=(pi,pi+1) _i=(p_i,p_i+1), we set ℓi=‖pi+1−pi‖2 _i=\|p_i+1-p_i\|_2 and Ti=ℓi/v+ϵT_i= _i/v+ε, where ϵε is a small time tolerance, and define yi=0,if pi+1 is reached within Ti,1,if timeout, falling, or another failure termination occurs.y_i= cases0,&if p_i+1 is reached within T_i,\\ 1,&if timeout, falling, or another failure termination occurs. cases (1) The label yiy_i records the execution outcome of τi _i under the current policy, with 00 for successful traversal and 11 for failure. To balance the training data for kD_k, we store evaluated transitions ui=(ξi,yi)u_i=( _i,y_i) in equal-size success and failure queues. Given these labeled transitions, the evaluator kD_k learns to estimate the difficulty of a proposed sub-trajectory for the current policy πθk _ _k. It takes a latent transition code ziz_i as input and predicts a score di=k(zi)∈[0,1]d_i=D_k(z_i)∈[0,1], where larger values indicate a higher failure likelihood under πθk _ _k, and therefore a more difficult transition for the current policy. Instead of predicting this score directly from the high-dimensional transition descriptor ξi _i, we first learn a compact latent representation with a CVAE. Let xi=(Hc,Mc)x_i=(H_c,M_c) denote the current sub-trajectory representation and ci=(Hp,Mp,δi)c_i=(H_p,M_p, _i) denote its transition context. The CVAE consists of an encoder qϕ(zi∣xi,ci)q_φ(z_i x_i,c_i) and a decoder pφ(xi∣zi,ci)p_ (x_i z_i,c_i), with prior p(zi)=(0,I)p(z_i)=N(0,I). We denote the reconstruction produced by the decoder as xi′=(Hc′,Mc′)x _i=(H _c,M _c). The CVAE is trained by minimizing ℒcvae=λH∥Mc⊙(Hc−Hc′)∥22+λMBCE(Mc,Mc′)+λKLDKL(qϕ(zi∣xi,ci)∥p(zi)),L_cvae= _H\|M_c (H_c-H _c)\|_2^2+ _MBCE(M_c,M _c)+ _KLD_KL (q_φ(z_i x_i,c_i)\,\|\,p(z_i) ), (2) where BCE(⋅,⋅)BCE(·,·) denotes point-wise binary cross-entropy. The first term reconstructs height only over the valid region, the second term reconstructs the valid-region mask, and the KL term regularizes the latent space. After CVAE training, the encoder provides ziz_i from (xi,ci)(x_i,c_i), and the evaluator kD_k is trained on these latent codes with binary cross-entropy: ℒdiff=−1B∑i=1B[yilogdi+(1−yi)log(1−di)],di=k(zi).L_diff=- 1B _i=1^B [y_i d_i+(1-y_i) (1-d_i) ], d_i=D_k(z_i). (3) Since yi=1y_i=1 corresponds to failed traversal under πθk _ _k, did_i measures the difficulty of τi _i for the current policy rather than intrinsic terrain difficulty. 3.3 Evaluator-guided Task Generation After updating the evaluator kD_k, the sampler kG_k generates the next training trajectories by searching over waypoint locations on ℳM. We formulate task generation as energy minimization problem. For a candidate trajectory =p0,p1,…,pNT=\p_0,p_1,…,p_N\, the energy is E()=Ehard()+∑i=0N−1|di−dtar|⏟Esoft(),di=k(zi),zi=Encϕ(xi,ci).E(T)=E_hard(T)+ _i=0^N-1 |d_i-d^tar |_E_soft(T), d_i=D_k(z_i), z_i=Enc_φ(x_i,c_i). (4) Here EhardE_hard imposes large penalties on infeasible proposals, including trajectories that leave the map boundary, exceed the valid transition length, or place waypoints in invalid or non-traversable regions. EsoftE_soft is the curriculum objective that encourages all sub-trajectories in T to match the assigned target difficulty dtard^tar. For each candidate trajectory, we sample dtard^tar from a truncated Gaussian distribution over [α,β][α,β], centered at moderate difficulty, to emphasize challenging but learnable tasks. In practice, batch sampling draws multiple targets in parallel to improve coverage within this difficulty band. We optimize E()E(T) with MH-MCMC for J iterations. At iteration j, the sampler proposes a new trajectory ′∼Q(′∣)T Q(T ) by perturbing waypoint locations with decaying Gaussian noise, recomputes the affected transition descriptors, and evaluates E(′)E(T ). The proposal is accepted with probability A(→′)=min(1,exp(−E(′)/Tj)Q(∣′)exp(−E()/Tj)Q(′∣)),A(T )= (1, (-E(T )/T_j )Q(T ) (-E(T)/T_j )Q(T ) ), (5) where Q(′∣)Q(T ) is the waypoint proposal distribution and TjT_j is the sampling temperature. Since our waypoint proposal uses symmetric Gaussian perturbations, we have Q(′∣)=Q(∣′)Q(T )=Q(T ), so the proposal terms cancel in the Metropolis-Hastings ratio. In our implementation, both the proposal noise scale and TjT_j decay over MH-MCMC iterations. Details of EhardE_hard, the proposal distribution, and the decay schedules are provided in Appendix. The target interval [α,β][α,β] remains fixed across curriculum iterations, but it is interpreted through the current evaluator. As kD_k is updated from new rollouts, terrain that was difficult for an earlier policy can receive a lower score for the improved policy. Thus, minimizing EsoftE_soft with the same dtard^tar drives the sampler toward transitions that remain challenging for the current policy. The final accepted trajectories become the task set for the next policy update, closing the curriculum loop. 4 Experiment 4.1 Experimental Setup We design our experiments to answer three questions: Q1: Can our TACL framework provide an effective curriculum that guides progressive policy learning across a unstructured terrain map? Q2: How does TACL compare with handcrafted curriculum learning method? Q3: Is the CVAE-based transition encoding necessary for reliable difficulty estimation? Terrain Maps. We conduct experiments on two terrain maps. Map_A is an unstructured training map with circular platforms, rectangular blocks, and obstacle heights increasing from 0.05m0.05\,m to 0.6m0.6\,m along the map. Map_HCL is a handcrafted curriculum map procedurally instantiated from the Extreme Parkour codebase [3]. It contains flat regions and four obstacle families, including steps, gaps, parkour, and hurdles, as visualized in Fig. 3. Each family is organized into 1010 manually specified difficulty levels, with 88 handcrafted sub-terrain instances per level and one predefined 88-waypoint training trajectory per instance. Figure 3: Obstacle families in Map_HCL. From left to right: steps, gaps, parkour, and hurdles, each shown at two difficulty levels. Red lines and points denote the predefined waypoint trajectories used in the handcrafted curriculum. Baselines. We compare TACL with two baselines. All three methods use the same policy implementation based on the teacher-stage setup of Extreme Parkour [3]. The only difference is the source of training tasks. TACL samples trajectories adaptively according to the evaluated capability of the current policy. RandST trains on feasible trajectories randomly sampled from the given map. HCL-E follows the original handcrafted curriculum setting of Extreme Parkour, where the policy is trained on predefined waypoint trajectories following a manually designed difficulty progression. All policies are trained and evaluated in parallel simulation using Isaac Gym [7]. During evaluation, success is measured at the trajectory level. A rollout is successful only if the robot reaches all waypoints, and it is counted as failed if any sub-trajectory times out, falls, or triggers another failure termination as defined in Eq. 1. Details of all model architectures, hyperparameters, evaluator training, and experimental settings are provided in Appendix. Real-World Experiments. For real-world deployment, we follow the sim-to-real pipeline of Extreme Parkour [3]. The stage-one policy trained with height-map observations serves as the teacher, and we distill it into a student policy using depth images as exteroceptive input. We deploy the student policy on a Unitree Go2 equipped with an Intel RealSense D435i. This setup isolates our curriculum generation method while using a standard deployment pipeline. Details of stage-two training, deployment, and additional real-world experiments are provided in Appendix. Figure 4: Sampler-generated curriculum by our TACL on Map_A. (a) The terrain height field. (b)–(d) Trajectories sampled by our method at different policy-training epochs. 4.2 Results A1: TACL generates effective curricula that progressively expands policy capability. We train TACL and RandST on Map_A for 16k16k policy epochs. Fig. 4 visualizes the height map and the trajectories sampled by our method at different epochs. Trajectories are colored by the evaluator-predicted difficulty score, where darker colors indicate easier trajectories and brighter colors indicate harder trajectories. Since the sampler always targets the full difficulty range, the important signal is the spatial location of each difficulty level. At early training, sampled trajectories are limited to flat regions and lower platforms, while the highest platforms are avoided. As the policy improves, the sampled distribution moves toward higher platforms: easy trajectories occupy traversable flat regions, medium trajectories cover platform surfaces, and hard trajectories concentrate near platform boundaries. This shift shows that our method adapts the task distribution to the current policy capability, producing a learnable curriculum that progressively moves toward harder terrain. Fig. 5(a) evaluates policy checkpoints across training epochs on three-waypoint climb-then-descend trajectories, with 5050 randomly sampled trials per platform. Our TACL progressively expands the traversable height range, achieving nonzero success across all heights by 4k4k, over 50%50\% success below 0.65m0.65\,m by 8k8k, and above 70%70\% success across all heights by 16k16k. In contrast, RandST saturates early and converges to a conservative policy whose reliable climbing capability remains around 0.3m0.3\,m. Across all platform heights, our method improves the average success rate from 39.57%39.57\% for RandST to 94.54%94.54\%, a 55.97%55.97\% gain. These above results show that our method builds a learnable curriculum that progressively improves the policy’s traversal capability. (a) Platform-climbing success rate on Map_A. (b) Evaluation of policies trained on Map_HCL. Figure 5: (a) Our method TACL adapts the training distribution to the evolving policy capability on Map_A. (b) Our method TACL achieves comparable results to the handcrafted curriculum baseline on major obstacle families in Map_HCL, without using predefined training trajectories. A2: Policies trained with TACL generalize better than HCL across most obstacle families. We train HCL-E, TACL , and RandST on Map_HCL for 50k50k policy epochs. HCL-E follows the handcrafted curriculum protocol of Extreme Parkour [3], using manually specified terrain progressions and predefined 88-waypoint training trajectories. In contrast, TACL treats Map_HCL as an ordinary terrain map and adaptively samples training trajectories according to the current policy capability. RandST uses randomly sampled feasible trajectories for training. We evaluate policy checkpoints on separate test terrains generated with the same procedural code and difficulty parameters as Map_HCL, but using different random seeds. For each obstacle family, we evaluate the maximum difficulty level on 88 test sub-terrain instances, each paired with a corresponding 88-waypoint trajectory, and report the average trajectory-level success rate over 5050 rollouts. Fig. 5(b) shows that HCL-E learns quickly from its handcrafted progression but degrades with continued training: from 5k5k epochs to the final checkpoint, success changes from 94%→82%94\%\!→\!82\% on step, 78%→30%78\%\!→\!30\% on gap, 81%→62%81\%\!→\!62\% on parkour, and 93%→89%93\%\!→\!89\% on hurdle. This suggests that HCL-E becomes overly adapted to the predefined trajectories and perceptual patterns. In contrast, our method TACL starts with much lower success at 5k5k epochs but improves steadily without using handcrafted training trajectories: 18%→98%18\%\!→\!98\% on step, 24%→96%24\%\!→\!96\% on gap, and 72%→96%72\%\!→\!96\% on hurdle. The main limitation is parkour: 4%→47%4\%\!→\!47\%, where the trajectory requires a specific human-designed maneuver that is rarely discovered by automatic sampling without additional priors. Overall, across the four maximum-difficulty obstacle families, our method improves the average success rate from 65.75%65.75\% for HCL-E to 84.25%84.25\%, an 18.5%18.5\% gain. RandST achieves 0%0\% success on all obstacle families and is omitted from the plot for clarity. We further compare TACL with HCL-E on step-task robustness in Fig. 6. In Map_HCL, the highest-difficulty step terrain used for training has a height of 0.5m0.5\,m, which marks the maximum step height observed during training. For each policy checkpoint and platform height, we report the average success rate over trajectories approaching the platform from diverse directions. Within this training height range, TACL improves the overall success rate from 58.68%58.68\% for HCL-E to 98.42%98.42\%. Our method shows a capability-frontier expansion strategy, where the policy first improves near its current limit and then shifts the success boundary toward taller platforms. In contrast, HCL-E shows limited boundary expansion and even degrades at later epochs, suggesting that predefined trajectories provide insufficient variation in approach directions and traversal contexts. (a) Ours, requires no predefined training trajectories. (b) HCL, requires predefined training trajectories. Figure 6: Platform-climbing evaluation across approach directions. Each value is the average success rate over diverse approach directions, where our method shows more robust traversal than HCL. A3: CVAE-based transition encoding is critical for reliable difficulty estimation and effective task generation. We compare TACL with a variant that removes the CVAE representation. Table 1: Ablation of CVAE-based transition encoding on Map_A. Method Eval. Acc. Avg. Energy Success Rate RandST – – 35.2%35.2\% TACL w/o CVAE 67.3%67.3\% 0.0130.013 47.4%47.4\% TACL w/ CVAE 92.5%92.5\% 0.0210.021 94.6%94.6\% In this variant, the evaluator D predicts difficulty directly from the raw transition descriptor ξi=(Hp,Mp,Hc,Mc,δi) _i=(H_p,M_p,H_c,M_c, _i): (Hp,Mp)(H_p,M_p) and (Hc,Mc)(H_c,M_c) are concatenated along the channel dimension, encoded by separate convolutional neural networks (CNNs) [38], and fused with δi _i for binary difficulty prediction. The sampler and policy-training pipeline remain unchanged. Table 1 reports evaluator classification accuracy, the average energy of sampled trajectories, and the final trajectory success rate on 10001000 randomly sampled feasible trajectories in Map_A. Removing the CVAE reduces evaluator accuracy from 92.5%92.5\% to 67.3%67.3\%, indicating that direct prediction from raw transition descriptors provides an unreliable difficulty signal. Although the sampler still obtains low energy under this flawed evaluator, the resulting tasks no longer match the capability of the full policy, leading to a success rate drop from 94.6% to 47.3%. This degradation suggests that raw height patches form a non-smooth input space for difficulty prediction, where small geometric perturbations can induce large pixel-level changes and make stable geometry-to-difficulty learning difficult from limited rollout data. The CVAE alleviates this issue by encoding transitions into a more compact and smoother latent space, which is critical for training reliable evaluators and generating tasks that match the curriculum. 5 Conclusion We propose TACL, an framework for robot locomotion on unstructured terrain, enabling policies to learn complex traversal skills without handcrafted terrain templates or predefined trajectories. The framework forms a closed loop by learning a difficulty evaluator from current-policy rollouts and using it to guide the sampler in generating new training trajectories for subsequent policy learning. Experiments show that our method generates effective curricula on raw terrain maps, improves robot traversal capability and achieves more robust generalization than HCL. 6 Limitations Although our framework learns robust locomotion behaviors without manually predefined trajectories, highly structured maneuvers may require more directed exploration than generic trajectory sampling provides. Incorporating weak task priors or demonstration-guided proposals could improve sampling efficiency for such behaviors while preserving direct curriculum generation on unstructured maps. References Sutton et al. [1998] R. S. Sutton, A. G. Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. Hoeller et al. [2024] D. Hoeller, N. Rudin, D. Sako, and M. Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots. Science Robotics, 9(88):eadi7566, 2024. Cheng et al. [2024] X. Cheng, K. Shi, A. Agarwal, and D. Pathak. Extreme parkour with legged robots. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11443–11450. IEEE, 2024. Wang et al. [2025] H. Wang, Z. Wang, J. Ren, Q. Ben, T. Huang, W. Zhang, and J. Pang. Beamdojo: Learning agile humanoid locomotion on sparse footholds. arXiv preprint arXiv:2502.10363, 2025. Luo et al. [2024] S. Luo, S. Li, R. Yu, Z. Wang, J. Wu, and Q. Zhu. Pie: Parkour with implicit-explicit learning framework for legged robots. IEEE Robotics and Automation Letters, 9(11):9986–9993, 2024. Dong et al. [2025] Y. Dong, J. Ma, L. Zhao, W. Li, and P. Lu. Marg: Mastering risky gap terrains for legged robots with elevation mapping. IEEE Transactions on Robotics, 2025. Makoviychuk et al. [2021] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simulation for robot learning, 2021. Li et al. [2026] Z. Li, C. Li, and M. Hutter. Scaling rough terrain locomotion with automatic curriculum reinforcement learning. arXiv preprint arXiv:2601.17428, 2026. Lee et al. [2020] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47):eabc5986, 2020. Xie et al. [2020] Z. Xie, H. Y. Ling, N. H. Kim, and M. van de Panne. Allsteps: Curriculum-driven learning of stepping stone skills. In Proc. ACM SIGGRAPH / Eurographics Symposium on Computer Animation, 2020. Kim et al. [2025] H. Kim, H. Oh, J. Park, Y. Kim, D. Youm, M. Jung, M. Lee, and J. Hwangbo. High-speed control and navigation for quadrupedal robots on complex and discrete terrain. Science Robotics, 10(102):eads6192, 2025. Sohn et al. [2015] K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015. Metropolis et al. [1953] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953. Hastings [1970] W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. 1970. Wang et al. [2021] X. Wang, Y. Chen, and W. Zhu. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence, 44(9):4555–4576, 2021. Tidd et al. [2020] B. Tidd, N. Hudson, and A. Cosgun. Guided curriculum learning for walking over complex terrain. arXiv preprint arXiv:2010.03848, 2020. Qin et al. [2019] B. Qin, Y. Gao, and Y. Bai. Sim-to-real: Six-legged robot control with deep reinforcement learning and curriculum learning. In 2019 4th International Conference on Robotics and Automation Engineering (ICRAE), pages 1–5. IEEE, 2019. Kumar et al. [2021] A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021. Margolis et al. [2024] G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal. Rapid locomotion via reinforcement learning. The International Journal of Robotics Research, 43(4):572–587, 2024. He et al. [2024] T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi. Agile but safe: Learning collision-free high-speed legged locomotion. arXiv preprint arXiv:2401.17583, 2024. Atanassov et al. [2024] V. Atanassov, J. Ding, J. Kober, I. Havoutis, and C. Della Santina. Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design. IEEE Robotics & Automation Magazine, 32(2):35–48, 2024. He et al. [2025] J. He, C. Zhang, F. Jenelten, R. Grandia, M. Bächer, and M. Hutter. Attention-based map encoding for learning generalized legged locomotion. Science Robotics, 10(105):eadv3604, 2025. Rudin et al. [2022] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on robot learning, pages 91–100. PMLR, 2022. Zhuang et al. [2023] Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. arXiv preprint arXiv:2309.05665, 2023. Zhuang et al. [2024] Z. Zhuang, S. Yao, and H. Zhao. Humanoid parkour learning. arXiv preprint arXiv:2406.10759, 2024. Mishra et al. [2025] P. Mishra, A. H. Raj, X. Xiao, and D. Manocha. Hacl: History-aware curriculum learning for fast locomotion. arXiv preprint arXiv:2505.18429, 2025. Wang et al. [2019] R. Wang, J. Lehman, J. Clune, and K. O. Stanley. Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions. arXiv preprint arXiv:1901.01753, 2019. Parker-Holder et al. [2022] J. Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. Foerster, E. Grefenstette, and T. Rocktäschel. Evolving curricula with regret-based environment design. In International Conference on Machine Learning, pages 17473–17498. PMLR, 2022. Dennis et al. [2020] M. Dennis, N. Jaques, E. Vinitsky, A. Bayen, S. Russell, A. Critch, and S. Levine. Emergent complexity and zero-shot transfer via unsupervised environment design. Advances in neural information processing systems, 33:13049–13061, 2020. Wang et al. [2025] L. Wang, Z. Xu, P. Stone, and X. Xiao. Gacl: Grounded adaptive curriculum learning with active task and performance monitoring. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 591–596. IEEE, 2025. Jiang et al. [2021a] M. Jiang, E. Grefenstette, and T. Rocktäschel. Prioritized level replay. In International Conference on Machine Learning, pages 4940–4950. PMLR, 2021a. Jiang et al. [2021b] M. Jiang, M. Dennis, J. Parker-Holder, J. Foerster, E. Grefenstette, and T. Rocktäschel. Replay-guided adversarial environment design. Advances in Neural Information Processing Systems, 34:1884–1897, 2021b. Xu et al. [2025] T. Xu, C. Pan, and X. Xiao. Vertiselector: Automatic curriculum learning for wheeled mobility on vertically challenging terrain. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11136–11143. IEEE, 2025. Portelas et al. [2020] R. Portelas, C. Colas, L. Weng, K. Hofmann, and P.-Y. Oudeyer. Automatic curriculum learning for deep rl: A short survey. arXiv preprint arXiv:2003.04664, 2020. Saglam et al. [2023] B. Saglam, F. B. Mutlu, D. C. Cicek, and S. S. Kozat. Actor prioritized experience replay. Journal of Artificial Intelligence Research, 78:639–672, 2023. Portelas et al. [2020] R. Portelas, C. Colas, K. Hofmann, and P.-Y. Oudeyer. Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments. In Conference on Robot Learning, pages 835–853. PMLR, 2020. Zhao et al. [2022] W. Zhao, Z. Li, and J. Pajarinen. Learning progress driven multi-agent curriculum. arXiv preprint arXiv:2205.10016, 2022. LeCun et al. [1998] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.