Paper deep dive
Predictive Safety Curricula for Robust Legged Locomotion
Ivan Ovinnikov, Pascal Sutter, Christian Gehring, Jordis Herrmann
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Rare but consequential failures can persist in learned locomotion policies for legged robots even when average task performance is high, in part because standard curricula primarily adapt task difficulty rather than the distribution of safety-critical experience. We introduce Predictive Safety Curricula (PSC), a framework for allocating locomotion training experience using learned predictions of future safety cost. PSC trains a distributional safety critic from policy rollouts and uses its predictions to prioritize both terrain contexts and previously encountered randomized events. The resulting curriculum modifies the training distribution while leaving the task reward and policy-optimization loss unchanged. We evaluate PSC in controlled rough-terrain locomotion and in production locomotion systems. PSC improves reliability relative to standard terrain progression, advantage-based replay, and learning-progress curricula, with the largest gains on difficult terrain and under degraded observations. The same allocation principle transfers to two production locomotion stacks. On ANYmal-D hardware, PSC reduces shank-collision incidence by $63\%$ relative to the learning-progress curriculum across three matched training seeds, with a reduction in every seed. On a production stair-climbing platform, PSC eliminates observed shank collisions in the evaluated hardware trials. These results show that learned predictions of future safety cost can provide an effective signal for allocating training experience toward rare failure modes and improving locomotion reliability.
Tags
Links
- Source: https://arxiv.org/abs/2609.37070v1
- Canonical: https://arxiv.org/abs/2609.37070v1
Trouble viewing inline? Open PDF directly →
Full Text
51,055 characters extracted from source content.
Expand or collapse full text
Predictive Safety Curricula for Robust Legged Locomotion Ivan Ovinnikov Pascal Sutter Christian Gehring Jordis Herrmann Affiliation: ANYbotics AG Affiliation: Zürich, Switzerland Abstract Rare but consequential failures can persist in learned locomotion policies for legged robots even when average task performance is high, in part because standard curricula primarily adapt task difficulty rather than the distribution of safety-critical experience. We introduce Predictive Safety Curricula (PSC), a framework for allocating locomotion training experience using learned predictions of future safety cost. PSC trains a distributional safety critic from policy rollouts and uses its predictions to prioritize both terrain contexts and previously encountered randomized events. The resulting curriculum modifies the training distribution while leaving the task reward and policy-optimization loss unchanged. We evaluate PSC in controlled rough-terrain locomotion and in production locomotion systems. PSC improves reliability relative to standard terrain progression, advantage-based replay, and learning-progress curricula, with the largest gains on difficult terrain and under degraded observations. The same allocation principle transfers to two production locomotion stacks. On ANYmal-D hardware, PSC reduces shank-collision incidence by 63%63\% relative to the learning-progress curriculum across three matched training seeds, with a reduction in every seed. On a production stair-climbing platform, PSC eliminates observed shank collisions in the evaluated hardware trials. These results show that learned predictions of future safety cost can provide an effective signal for allocating training experience toward rare failure modes and improving locomotion reliability. Keywords curriculum learning, legged locomotion, reinforcement learning, robot safety, adaptive sampling 1 Introduction Learning-based control has substantially expanded the capabilities of legged robots on complex and heterogeneous terrain. Despite this progress, policies with strong average performance can still exhibit rare but consequential unsafe outcomes, including lower-leg collisions and loss of balance under difficult conditions. Standard locomotion curricula prioritize task difficulty, average performance, or learning progress, and may therefore overlook conditions that appear equally successful on average but differ substantially in tail safety risk. This creates a gap between capability-driven training allocation and the rare safety-critical outcomes that limit reliable deployment. Curriculum learning provides a natural mechanism for reallocating training experience. Performance-based terrain curricula adapt task difficulty as locomotion competence improves [1, 2], while Prioritized Level Replay and learning-progress curricula use policy-dependent signals to focus sampling on tasks with high estimated learning potential [3, 4]. These mechanisms support capability acquisition but do not explicitly target elevated safety risk. Estimating such risk is challenging: failures may be sparse within individual terrain–command contexts and change throughout training. As the number of terrain and randomization variables grows, maintaining reliable empirical safety estimates for each configuration becomes increasingly data inefficient. dj(z,ξ)=d_j(z,ξ)= Fixed p(τ)p(τ) Contexts pjctx(ℓ,b∣τ)p_j^ctx( ,b τ) Events pjevt(ξ∣z)p_j^evt(ξ z) × Simulated rollouts PPO update Policy πθ _θ deployed alone rtr_t Distributional safety-return critic qψ,i(xt)q_ψ,i(x_t) Upper-tail risk R^αdisc(xt)→s¯e R_α^disc(x_t)\;→\; s_e xt,ctx_t,c_t Context allocation aggregate risk by terrain–command context Event replay replay high-risk events in matched contexts encounteredevent payloads (a) Predictive safety allocation for context z=(τ,ℓ,b)z=(τ, ,b). Rollout states and costs train a shared safety-return critic. Its upper-tail risk is averaged into an episode score s¯e s_e, which is aggregated by context and attached to event payloads encountered in that episode. (b) Late-training context allocation under PSC relative to LP. Mean difference in empirical terrain-context allocation, ΔAℓ,t=Aℓ,tPSC−Aℓ,tLP A_ ,t=A^PSC_ ,t-A^LP_ ,t, averaged over N=3N=3 training seeds. Positive values indicate contexts sampled more frequently by PSC. PSC induces a structured redistribution rather than uniformly favoring higher terrain levels. Figure 1: Predictive safety allocation and its effect on the late-training context distribution. We introduce Predictive Safety Curricula (PSC), a framework for directing locomotion training toward conditions associated with elevated predicted safety risk. PSC learns future safety cost from policy rollouts and uses these predictions to adapt subsequent training experience. Unlike safety critics used to constrain policy updates or intervene at deployment, PSC uses safety predictions to prioritize which experience is collected next: the task reward and loss function remain unchanged. The learned model supports both expected-cost and upper-tail prioritization and provides a common allocation signal across terrain and randomized environment variation. We evaluate PSC in both controlled and production-level locomotion settings. On ANYmal-D, a quadrupedal robot developed by ANYbotics, PSC improves reliability relative to standard terrain progression, Prioritized Level Replay, and learning-progress curricula, with the largest gains under more demanding terrain and observation conditions. Controlled ablations examine the effect of experience allocation based on mean and upper-tail safety risk and compare predicted risk with direct empirical safety statistics, while critic diagnostics assess whether the learned model identifies future safety-relevant outcomes. We further evaluate PSC in a production stair-climbing stack on ANYmal-X, an explosion-safe quadruped where avoiding lower-leg impacts is particularly important for hardware integrity. There, PSC substantially reduces such contacts while preserving traversal performance. Our contributions are threefold. First, we introduce predictive safety allocation as a curriculum mechanism that uses a shared safety-return model to redistribute training experience, while the policy is still optimized for the original task reward using the unchanged reinforcement learning loss. Second, we characterize the role of safety prediction and risk readout through controlled curriculum ablations and critic diagnostics. Third, we validate the approach across a controlled locomotion benchmark, a production stair-climbing training stack, and repeated hardware trials with physically meaningful contact measurements. 2 Method PSC converts predictions of future safety exposure from completed rollouts into priorities for subsequent training experience (Fig. 1(a)). We represent the training distribution through structured curriculum contexts and randomized environment events and use the same learned safety signal to adapt both components. Context allocation can operate alongside an existing capability curriculum that determines which structured conditions are currently available, while event allocation adapts the randomizations encountered within those conditions. 2.1 Training Conditions and Safety Returns We consider a structured curriculum context z=(τ,ℓ,b)∈z=(τ, ,b) , where τ denotes terrain type (e.g., stairs, slopes, or discrete obstacles), ℓ∈1,…,L ∈\1,…,L\ the terrain difficulty level, and b∈1,…,Bb∈\1,…,B\ an optional command bucket. Additional randomized environment variables such as friction, joint positions, actuator properties, or external perturbations, are represented by an event configuration ξ. At curriculum update j, the joint distribution of training conditions is factorized as dj(z,ξ)=p(τ)pjctx(ℓ,b∣τ)pjevt(ξ∣τ,ℓ,b),d_j(z,ξ)=p(τ)\,p_j^ctx( ,b τ)\,p_j^evt(ξ τ, ,b), (1) where p(τ)p(τ) denotes the fixed terrain-type assignment, pjctxp_j^ctx controls allocation over level–command contexts within each terrain type, and pjevtp_j^evt controls replayable randomized events within the selected context. Environment variables not controlled by either PSC sampler retain their original sampling procedures. This factorization separates allocation across structured task contexts from allocation over randomized environment events while allowing both to be adapted using the same learned safety signal. The locomotion policy πθ(at∣ot) _θ(a_t o_t) maps observation oto_t to action ata_t and is trained using the original task reward. PSC introduces a separate nonnegative safety cost ct=∑m=1Mwmct(m),wm≥0,Gtc=∑k=tT−1γck−tck,c_t= _m=1^Mw_mc_t^(m), w_m≥ 0, G_t^c= _k=t^T-1 _c^k-tc_k, (2) where ct(m)c_t^(m) denotes task-specific safety component m, wmw_m its weight, and γc∈(0,1] _c∈(0,1] the safety discount. The discounted return GtcG_t^c represents cumulative future safety exposure from time t and provides the prediction target for the safety critic. The particular safety components and their parameters are properties of the locomotion task rather than of PSC and are specified in Sec. 3. PSC modifies the training-condition distribution djd_j, thereby changing the induced trajectory distribution used for policy optimization. The safety cost trains an auxiliary critic whose predictions define curriculum priorities; it is not added to the task reward or PPO policy objective. 2.2 Shared Distributional Safety Critic Given rollout features xtx_t, a critic parameterized by ψ predicts K conditional quantiles of the discounted safety return GtcG_t^c [5]: qψ,i(xt)≈FGtc|xt−1(ui),ui=i−12K,i=1,…,K.q_ψ,i(x_t)≈ F^-1_G_t^c x_t(u_i), u_i= i- 12K, i=1,…,K. (3) Here uiu_i is the probability level of the i-th predicted quantile. Evenly spaced levels provide a uniform discretization of the conditional safety-return distribution, allowing both mean and upper-tail statistics to be obtained from the same critic. The function FGtc|xt−1F^-1_G_t^c x_t denotes the conditional quantile function, and xtx_t contains the state or observation features available to the safety critic. A single critic is trained from trajectories collected across the full training distribution djd_j. Safety observations from different curriculum contexts and randomized environment configurations can therefore contribute to a common predictive model when they induce similar risk-relevant states. Temporal bootstrapping further propagates observed safety outcomes to states preceding the corresponding interaction. For an n-step segment ending at or before episode termination, the distributional bootstrap targets are yt,k=∑r=0n−1γcrct+r+γcnχt+nqψ¯,k(xt+n),y_t,k= _r=0^n-1 _c^rc_t+r+ _c^n _t+nq_ ψ,k(x_t+n), (4) where χt+n _t+n is 00 at termination and 11 otherwise, and ψ¯ ψ denotes the Polyak-averaged target network parameters. The critic minimizes the standard quantile-Huber regression loss [5], ℒ(ψ)=t[1K2∑i=1K∑k=1Kρuiκ(yt,k−qψ,i(xt))],L(ψ)=E_t\! [ 1K^2 _i=1^K _k=1^K _u_i^κ\! (y_t,k-q_ψ,i(x_t) ) ], (5) with Huber threshold κ. Modeling the conditional safety-return distribution allows the same learned predictor to support alternative risk summaries. We use an upper-tail readout for PSC and evaluate the corresponding mean readout as an ablation in Sec. 3.2. 2.3 From Safety Prediction to Curriculum Priority Let α∈(0,1)α∈(0,1) denote the target upper-tail mass. The state-conditioned upper-tail conditional value-at-risk (CVaR) is Rα(xt)=1α∫1−α1FGtc|xt−1(u)u.R_α(x_t)= 1α _1-α^1F^-1_G_t^c x_t(u)\,du. (6) For the discrete quantile representation, let qψ,(1)(x)≤⋯≤qψ,(K)(x)q_ψ,(1)(x)≤·s≤ q_ψ,(K)(x) denote the predicted quantile values ordered by magnitude and define kα=⌈αK⌉k_α= α K . We use R^αdisc(x) R_α^disc(x) =1kα∑i=K−kα+1Kqψ,(i)(x), = 1k_α _i=K-k_α+1^Kq_ψ,(i)(x), (7) μ^(x) μ(x) =1K∑i=1Kqψ,(i)(x). = 1K _i=1^Kq_ψ,(i)(x). We refer to R^αdisc R_α^disc and μ μ as the upper-tail and mean readouts of the predicted safety-return distribution. The safety cost determines which interactions contribute to future safety exposure, while the readout determines how the predicted distribution of that exposure enters curriculum allocation. For each completed episode e of length TeT_e, the selected state-level readout s is averaged along the trajectory: s¯e=1Te∑t=0Te−1s(xe,t),Sj(z)=1|ℬj(z)|∑e∈ℬj(z)s¯e. s_e= 1T_e _t=0^T_e-1s(x_e,t), S_j(z)= 1|B_j(z)| _e _j(z) s_e. (8) Here ℬj(z)B_j(z) contains the recent scored episodes associated with context z at update j. PSC uses s=R^αdiscs= R_α^disc for upper-tail prioritization; the mean-readout ablation uses s=μ^s= μ. The episode score s¯e s_e provides the common signal for the two allocation pathways. For context allocation, recent episode scores are aggregated into Sj(z)S_j(z); for event allocation, the episode score remains associated with randomized event realizations encountered during that trajectory. As a non-predictive comparison, we additionally consider empirical CVaR computed directly from realized episode safety returns. For a context containing nzn_z buffered episodes, the empirical priority averages the largest max(1,⌈αnz⌉) (1, α n_z ) values of G0cG_0^c. Sec. 3.2 compares this direct empirical statistic with the learned state-conditioned priority. 2.4 Safety-Prioritized Context Allocation For terrain type τ, let ℰτ,jE_τ,j denote the set of level–command pairs admitted by the underlying capability curriculum at update j. PSC reallocates experience within this support by combining uniform exploration with safety-prioritized sampling: pjctx(ℓ,b∣τ)=εUτ,j(ℓ,b)+(1−ε)(ℓ,b)∈ℰτ,jPj(τ,ℓ,b)∑(ℓ′,b′)∈ℰτ,jPj(τ,ℓ′,b′).p_j^ctx( ,b τ)= U_τ,j( ,b)+(1- ) 1\( ,b) _τ,j\P_j(τ, ,b) _( ,b ) _τ,jP_j(τ, ,b ). (9) Here Uτ,jU_τ,j is uniform over the admissible set and ε controls uniform exploration. The underlying capability curriculum therefore determines which contexts are available, while PSC determines how experience is distributed among them. To maintain exploration, PSC combines the context score with terrain-level staleness and coverage: Pj(τ,ℓ,b)=λSS~j(τ,ℓ,b)+λaa~ℓ,j+λuu~ℓ,j.P_j(τ, ,b)= _S\, S_j(τ, ,b)+ _a\, a_ ,j+ _u\, u_ ,j. (10) Here, aℓ,ja_ ,j counts environment steps since level ℓ was last selected, and uℓ,j=(1+nℓ,j)−1u_ ,j=(1+n_ ,j)^-1, where nℓ,jn_ ,j is its cumulative scored-episode count. Tildes denote independent normalization over the qualified support. Uninformative priorities yield uniform sampling over the admissible support. The weights λS,λa,λu _S, _a, _u and external capability curricula are specified in Sec. 3. 2.5 Safety-Prioritized Event Replay PSC adapts pjevt(ξ∣z)p_j^evt(ξ z) by associating each sampled event realization ξ with the safety score s¯e s_e of the episode in which it occurred. For each context, previously sampled event realizations are retained in a small replay buffer and preferentially resampled according to their scores. At each event invocation, PSC either reuses a context-matched realization or draws a new one, preserving exploration and providing a fallback when no replay samples are available. Context allocation and event replay therefore act on complementary factors of Eq. (1) using the same learned safety signal. 3 Experiments We evaluate PSC across three increasingly realistic settings. The public Isaac Lab ANYmal-D benchmark provides a controlled environment in which policy architecture, reward design, optimization budget, and terrain progression are matched across methods; we use it for the main multi-seed comparison and mechanism ablations. All policies are trained with Proximal Policy Optimization (PPO) [6], with the policy-learning algorithm and objective held fixed across curriculum variants. We then assess PSC in the more complex production ANYmal-X stair-climbing stack, and finally evaluate trained production policies on hardware using physically meaningful contact outcomes. Task and baselines. The ANYmal-D policy tracks commanded planar velocities and yaw rates over procedurally generated rough terrain, with terrain difficulty represented by a discrete level ℓ . The underlying terrain curriculum determines which levels are available for training, while the adaptive methods redistribute experience within this support. For all adaptive methods, prioritized allocation is enabled only after the terrain curriculum reaches its maximum level. For PSC, we define an auxiliary safety signal that captures three common indicators of unsafe locomotion: excessive torso tilt, configurations in which the legs fold close to the robot base, and undesired thigh or shank contacts with the environment. ct=ctori+ctdist+ctund.c_t=c_t^ori+c_t^dist+c_t^und. (11) The orientation term is ctori=[cos(60∘)+gt,zB]+c_t^ori=[ (60 )+g_t,z^B]_+, where gt,zBg_t,z^B is the vertical component of the unit gravity vector in the base frame. The proximity term is ctdist=[(dnom−dt)/dnom]+c_t^dist=[(d_nom-d_t)/d_nom]_+, with dnom=0.35md_nom=0.35\,m and dtd_t the minimum distance from the base-link origin to the hip, thigh, or shank link origins. Finally, ctund=∑r∈ℬundmaxh∥Ft,h,r∥2>1N,c_t^und= _r _und1 \ _h F_t,h,r _2>1\,N \, where ℬundB_und contains the thigh and shank bodies and Ft,h,rF_t,h,r denotes the contact-force vector acting on body r at contact-history sample h of control step t. All terms have unit weight and γc=0.99 _c=0.99. The auxiliary safety cost itself does not enter the task reward or PPO loss; its constituent failure and collision events are already included as penalties in the locomotion reward. We compare four curriculum strategies under the same locomotion learner. Baseline uses standard terrain progression without additional prioritization. PLR uses mean absolute generalized advantage and combines score- and staleness-based priorities with ρ=0.1ρ=0.1 [3]. Learning progress (LP) uses the signed change between consecutive episodic-return estimates followed by softmax prioritization [4]. PSC derives priorities from predicted safety return. To compare prioritization strategies, PLR, LP, and PSC share the same admissible terrain–command support and randomized-event replay pathway while retaining their respective priority definitions and sampling rules. For PSC, we use λS=0.7 _S=0.7, λa=0.2 _a=0.2, and λu=0.1 _u=0.1 for predicted safety, staleness, and coverage. We use a buffer size of 64 per replayable event with a replay probability of 0.4. Safety-return estimator. The safety critic predicts K=32K=32 quantiles of discounted safety return with upper-tail mass α=0.05α=0.05. Following Sec. 2.3, the upper-tail readout averages the largest kα=⌈αK⌉=2k_α= α K =2 quantiles, corresponding to 6.25%6.25\% of the discrete representation. Episode and context scores follow Eq. (8). The critic receives the same uncorrupted privileged observations as the PPO value critic and is an independent ELU MLP with hidden widths (512,256,128)(512,256,128). It uses one-step targets (n=1n=1), γc=0.99 _c=0.99, quantile-Huber threshold κ=1κ=1, and Polyak coefficient 0.010.01. Each terrain-level–command context retains the latest N=128N=128 episode scores and enables prioritization after the first completed episode. Predicted safety, staleness, and coverage are independently min–max normalized, combined as above, and mixed with uniform sampling with probability 0.30.3. Training and cohort provenance. All ANYmal-D methods use the same interaction and optimization budget: 8192 parallel environments, 24-step rollouts, and 3000 PPO updates, totaling 589,824,000589,824,000 transitions. Each rollout is optimized for five epochs over four minibatches using Adam with initial learning rate 10−310^-3 and an adaptive KL-based schedule. For PSC, the safety critic is optimized jointly with the policy using the same optimizer and schedule, updated once per PPO minibatch, with no separate warm-up. Experiments use Isaac Lab [7], and RSL-RL [8]. Unless otherwise stated, results use six training seeds 13,19,23,31,47,61\13,19,23,31,47,61\. Evaluation protocol. We evaluate the final checkpoint after 3000 PPO updates using deterministic actions. Each policy–condition pair runs for 3000 control steps in 1000 parallel environments at 50Hz50\,Hz with a 40s40\,s episode timeout. Environments are reset after termination, and statistics are computed over all episodes completed during evaluation; episodes truncated by the evaluation cutoff are omitted. Terrain and command assignments are shared across methods. Terrain cells are assigned deterministically by environment index on a 10×2010× 20 grid, and commands are drawn from (vx,vy,ωz)∈(0,0,0),(0.5,0,0),(1,0,0),(0.5,0.5,0),(0.5,0,0.5).(v_x,v_y, _z)∈\(0,0,0),(0.5,0,0),(1,0,0),(0.5,0.5,0),(0.5,0,0.5)\. To vary occupancy of difficult terrain, we use pv(ℓ)∝(ℓ+1)v−1,v∈1,2,3,p_v( ) ( +1)^v-1, v∈\1,2,3\, (12) where v=1v=1 is uniform and larger v increasingly concentrates mass on higher terrain levels. Observation noise doubles the nominal policy-observation noise ranges. A fixed evaluation seed of 42 shares simulator randomization across methods within each condition. An episode fails on illegal torso contact (net contact force >1N>1\,N); timeout at 40s40\,s counts as success, and episodes truncated by the evaluation cutoff are excluded. 3.1 ANYmal-D Rough-Terrain Locomotion Table 1 summarizes the main ANYmal-D comparison. PSC achieves the highest mean success across all six clean and observation-noise conditions. PSC achieves higher mean success than PLR across the evaluation grid, while the stronger LP curriculum narrows the gap. Relative to LP, PSC improves success by 0.160.16, 1.381.38, and 0.500.50 percentage points under clean v=1v=1, v=2v=2, and v=3v=3, respectively. The largest separation occurs at v=2v=2, where the evaluation distribution places greater mass on more difficult terrain while retaining sufficient occupancy across the curriculum. Table 1: ANYmal-D success (%): mean ± sample standard deviation over six training seeds. v controls terrain occupancy. Bold denotes the highest mean. Occupancy Baseline PLR LP PSC Clean v1v1 91.94±1.5691.94±1.56 93.31±3.1393.31±3.13 94.40±0.9594.40±0.95 94.56±1.0894.56 1.08 v2v2 91.20±1.5091.20±1.50 92.29±4.6592.29±4.65 93.81±1.0093.81±1.00 95.19±1.7795.19 1.77 v3v3 89.07±2.0689.07±2.06 90.76±4.5190.76±4.51 92.81±0.9992.81±0.99 93.31±1.3693.31 1.36 Noise v1v1 91.11±1.6991.11±1.69 91.44±2.9891.44±2.98 92.27±1.7892.27±1.78 92.82±1.0192.82 1.01 v2v2 88.81±2.6588.81±2.65 89.54±3.5189.54±3.51 91.20±2.1191.20±2.11 92.53±1.7492.53 1.74 v3v3 86.86±3.5686.86±3.56 87.78±4.3687.78±4.36 89.08±2.3089.08±2.30 89.92±1.7189.92 1.71 Reliability on difficult terrain. Expressed as failure rate, clean v=2v=2 decreases from 6.19%6.19\% for LP to 4.81%4.81\% for PSC, a 22.3%22.3\% relative reduction; at v=3v=3, it decreases from 7.19%7.19\% to 6.69%6.69\%. Relative to PLR, failure decreases from 7.71%7.71\% to 4.81%4.81\% at v=2v=2 and from 9.24%9.24\% to 6.69%6.69\% at v=3v=3. Thus, PSC retains its advantage as evaluation shifts toward harder terrain, with LP providing the stronger comparator. Robustness to observation noise. PSC remains the highest-performing curriculum when policy observations are corrupted by noise. Relative to LP, success improves by 0.550.55, 1.331.33, and 0.840.84 percentage points under noisy v=1v=1, v=2v=2, and v=3v=3, respectively. The same ordering therefore persists under observation degradation, with the largest difference again occurring at v=2v=2. To examine how PSC changes the training distribution, Fig. 1(b) compares its late-training terrain-context allocation with LP. PSC produces a structured, terrain-dependent redistribution rather than uniformly favoring higher difficulty levels. The seed-mean distributions differ by 0.0330.033 in terms of total variation distance, with per-seed distances of 0.0430.043, 0.0440.044, and 0.0450.045. 3.2 Safety Prediction and Curriculum Ablations We next examine how the safety signal used by PSC influences curriculum performance and whether the learned critic captures safety-relevant trajectory structure. All variants share the policy architecture, reward, sampling machinery, and training budget, with safety-cost changes stated explicitly where ablated. Curriculum results use the same six-seed cohort 13,19,23,31,47,61\13,19,23,31,47,61\ and six-condition ANYmal-D evaluation suite as the main comparison. Risk readout and predictive estimation. We compare PSC (QC-CVaR) with predictive mean and empirical tail priorities. QC-mean uses the same quantile critic but averages its predicted quantiles, isolating the risk readout. Empirical CVaR instead computes an episode-level tail statistic from recent realized safety returns, replacing the state-conditioned predictive score and its temporal aggregation with a direct empirical priority. Table 2 reports success averaged across evaluation conditions within each seed. QC-CVaR achieves 93.05±1.14%93.05± 1.14\%, exceeding QC-mean (90.40±2.54%90.40± 2.54\%) and empirical CVaR (91.38±3.17%91.38± 3.17\%) by paired differences of 2.66±2.962.66± 2.96 and 1.68±3.801.68± 3.80 percentage points, respectively. QC-CVaR is higher than QC-mean on five of six seeds, while the empirical comparison is less consistent. The results favor predictive upper-tail prioritization in this cohort, although seed-level variability precludes a strong causal attribution to the readout alone. QC-mean is also sensitive to the safety-cost composition: removing the undesired-contact term increases success to 91.17±1.11%91.17± 1.11\%. The ablations support predictive safety as a useful allocation signal without isolating a unique contribution from the upper-tail readout. Empirical tail statistics yield lower mean success and greater across-seed variability than learned state-conditioned priorities in this cohort. Predictive quality of the safety critic. We evaluate the critic independently of final policy performance. Return prediction is measured by Spearman correlation between the mean prediction μ^(xt) μ(x_t) and Monte-Carlo discounted safety return GtcG_t^c. To test whether high predicted risk identifies states preceding unsafe events, we use the state-level CVaR score to rank whether an event occurs within the next H=100H=100 control steps (2s2\,s). We consider two outcomes: a collision, defined as the terminating illegal base contact (net force >1N>1\,N), and a contact, defined as any thigh or shank contact above 1N1\,N. Termination or timeout truncates the horizon, and incomplete windows at the evaluation cutoff are omitted. We report the area under the precision–recall curve (AUPRC) minus event prevalence and top-decile lift, defined as the event rate among the highest-scoring 10%10\% of valid states divided by the overall event rate. Priority Success (%) ↑ Δ (p) QC-mean 90.40±2.5490.40± 2.54 −2.66±2.96-2.66± 2.96 Empirical CVaR 91.38±3.1791.38± 3.17 −1.68±3.80-1.68± 3.80 QC-CVaR (PSC) 93.05±1.1493.05± 1.14 — Table 2: ANYmal-D safety-priority ablations. Success is averaged over the six evaluation conditions within each seed and reported as mean ± sample standard deviation over six training seeds. Δ denotes the paired difference relative to QC-CVaR. Metric Mean ± s.d. 95%95\% CI Spearman(μ^,Gtc)( μ,G_t^c) 0.773±0.0180.773± 0.018 [0.754,0.792][0.754,0.792] Collision AUPRC gain 0.079±0.0150.079± 0.015 [0.063,0.095][0.063,0.095] Collision top-decile lift 6.88±0.196.88± 0.19 [6.68,7.08][6.68,7.08] Contact AUPRC gain 0.461±0.0090.461± 0.009 [0.452,0.470][0.452,0.470] Contact top-decile lift 2.81±0.092.81± 0.09 [2.72,2.90][2.72,2.90] Table 3: Safety-critic diagnostics over six seeds: mean ± sample standard deviation and 95%95\% confidence interval. AUPRC gain is measured relative to event prevalence; top-decile lift has reference value one. As shown in Table 3, the mean prediction correlates strongly with realized discounted safety return (Spearman ρ=0.773ρ=0.773). High CVaR scores also concentrate subsequent unsafe events: states in the highest-scoring decile have 6.88×6.88× the overall collision rate and 2.81×2.81× the overall thigh-or-shank contact rate. AUPRC exceeds event prevalence by 0.0790.079 for collisions and 0.4610.461 for contacts. These results show that the critic ranks visited states by future safety exposure, supporting its use as a curriculum-priority signal. Tail-mass sensitivity. We further examine sensitivity to the upper-tail mass α. Relative to the default configuration (α=0.05α=0.05, 93.05±1.14%93.05± 1.14\%), setting α=0.01α=0.01, 0.100.10, and 0.200.20 yields 92.25±2.88%92.25±2.88\%, 91.80±2.06%91.80±2.06\%, and 91.28±1.31%91.28±1.31\% mean success, respectively. Performance therefore decreases as the readout is broadened beyond the default tail mass, while the more concentrated α=0.01α=0.01 variant remains closer to PSC. Among the tested settings, α=0.05α=0.05 achieves the highest mean reliability. 3.3 ANYmal-X Stair Climbing We next evaluate PSC in the production stair-climbing training stack used for ANYmal-X. Compared with ANYmal-D, this setting uses a different embodiment and a distinct production locomotion stack, providing a complementary test of predictive safety allocation under a substantially different training system. Experimental setup. We compare PSC with the baseline curriculum, PLR, and LP using the same stair-climbing learner and matched training budget. Teacher policies are evaluated on a held-out stair-climbing task, distinct from the training task. Each episode consists of a single staircase traversal toward a commanded planar goal; timeout, out-of-bounds motion, illegal base contact, and excessive body tilt are treated as failures. We evaluate six settings v0,v1,v1.5,v2,v2.5,v3\ v0, v1, v1.5, v2, v2.5, v3\ that jointly increase commanded speed and stair difficulty. Evaluation conditions are matched across methods. Each method is trained with three seeds, and results are reported as mean ± sample standard deviation over training seeds. Student distillation. The production policies are deployed after teacher–student distillation, and the PSC advantage is largely preserved through this transfer. Across the six stair-climbing settings, PSC students achieve 87.76±1.70%87.76±1.70\% mean success, compared with 86.72±0.71%86.72±0.71\% for the baseline and 87.06±1.06%87.06±1.06\% for PLR. A student distilled from the PSC teacher without PSC-based prioritization during distillation reaches 87.52±1.39%87.52±1.39\%, indicating that most of the reliability improvement is captured by the teacher and retained through the production distillation pipeline. Table 4: ANYmal-X stair-climbing teacher success (%): mean ± sample standard deviation over three training seeds. The final row averages settings within each seed. Setting Baseline PLR LP PSC v0 99.99±0.0199.99± 0.01 99.98±0.0299.98± 0.02 99.99±0.0299.99± 0.02 100.00±0.00100.00± 0.00 v1 99.65±0.1799.65± 0.17 99.69±0.0799.69± 0.07 99.67±0.1999.67± 0.19 99.85±0.0399.85± 0.03 v1.5 96.78±1.3596.78± 1.35 96.32±0.7496.32± 0.74 96.06±0.6096.06± 0.60 97.35±0.8797.35± 0.87 v2 89.96±1.7689.96± 1.76 90.13±1.5790.13± 1.57 90.54±0.3090.54± 0.30 91.90±0.8691.90± 0.86 v2.5 76.14±4.0076.14± 4.00 76.29±3.0376.29± 3.03 76.14±1.4976.14± 1.49 79.16±4.0379.16± 4.03 v3 57.75±3.0657.75± 3.06 59.11±2.2259.11± 2.22 59.88±2.5359.88± 2.53 62.46±3.7462.46± 3.74 Mean 86.71±1.6986.71± 1.69 86.92±1.2486.92± 1.24 87.04±0.6087.04± 0.60 88.45±1.5488.45± 1.54 Reliability in the production training stack. PSC achieves the highest mean success across all six stair-climbing conditions (Table 4). Averaged across settings, success reaches 88.45%88.45\%, compared with 86.71%86.71\% for the baseline, 86.92%86.92\% for PLR, and 87.04%87.04\% for LP. The corresponding improvements are 1.741.74, 1.531.53, and 1.411.41 percentage points, respectively. The separation is larger in the more demanding settings. Relative to LP, the strongest comparator on average, PSC improves success by 1.361.36 percentage points at v2, 3.023.02 at v2.5, and 2.582.58 at v3. At the hardest setting, PSC succeeds in 62.46%62.46\% of episodes, compared with 59.88%59.88\% for LP and 57.75%57.75\% for the baseline. The same ordering observed in the controlled ANYmal-D benchmark is also observed in the production stair-climbing training stack, with the clearest differences appearing in the more difficult evaluation regimes. 3.4 Hardware Evaluation Simulation success does not fully characterize physical locomotion: policies that complete the same task may differ substantially in how they interact with the terrain. We therefore evaluate distilled production policies on ANYmal-D open-step walking (Fig. 2) and on the production stair-climbing platform, using lower-leg contact as the primary hardware outcome. All PSC components other than the locomotion policy are discarded after training. Figure 2: Hardware evaluation on ANYmal-D. Representative open-step traversals under (a) LP and (b) PSC, shown as five-frame sequences from matched-view recordings. Quantitative results are reported in Table 5. Red circles mark annotated shank contacts. Protocol and metrics. Deployed policies are obtained through the same matched teacher–student pipeline. Contacts are manually annotated from recorded trials. A trial is shank-positive if at least one shank collision occurs; additional shank contacts within the same traversal contribute to the total event count but not to incidence. Foot scuffs are reported separately and are not classified as collision failures in the simulation safety definition. Policies are evaluated sequentially, with the robot software restarted when switching policies. ANYmal-D open-step traversal. We evaluate production rough-terrain policies trained with the same teacher–student pipeline for three curricula. For each method, three matched seeds are evaluated on hardware using the final checkpoints, without hardware-based selection or fine-tuning; LP and PSC differ from the baseline only in their training-time prioritization. Policies traverse a 25cm25\,cm-high, 40cm40\,cm-deep open step at (vx,vy,ωz)=(0.7,0,0)(v_x,v_y, _z)=(0.7,0,0), alternating direction between crossings. Each seed completes 100 one-way crossings, yielding 300 crossings per method. PSC reduces shank-positive crossings to 10.7±4.910.7± 4.9 per 100 crossings, compared with 25.7±9.125.7± 9.1 for the baseline and 29.0±5.329.0± 5.3 for LP (Table 5), corresponding to a 63.2%63.2\% reduction relative to LP. PSC has fewer shank-positive crossings than both alternatives in all three seeds. Foot scuffs increase relative to LP (29.0±9.629.0± 9.6 vs. 14.3±1.514.3± 1.5 per 100 crossings), while remaining comparable to the baseline (31.0±24.631.0± 24.6). Because foot contacts are excluded from ℬundB_und, this suggests that PSC shifts contact away from the explicitly penalized shank rather than uniformly suppressing lower-limb contact, highlighting the specificity of the safety-cost definition. Production stair climbing. We additionally deploy distilled students from Sec. 3.3 on the production stair-climbing system. Policies traverse a fixed staircase under matched conditions using the same commanded speed for both policies. Contacts are annotated using the same procedure as for ANYmal-D. Across 50 ascent–descent pairs, the baseline policy produces shank collisions in 33/5033/50 trials and 41 individual shank-collision events, whereas no shank collision is observed with PSC. Foot scuffs decrease from 14 to 7, and completion increases from 49/5049/50 to 50/5050/50; the single unsuccessful baseline traversal ends with the robot stuck in the stairs. Observed mean traversal speeds are effectively identical between the two policies. Metric Baseline LP PSC Shank-positive crossings 25.7±9.125.7±9.1 29.0±5.329.0±5.3 10.7±4.910.7±4.9 Foot-scuff events 31.0±24.631.0±24.6 14.3±1.514.3±1.5 29.0±9.629.0±9.6 Table 5: ANYmal-D open-step hardware evaluation over three matched training seeds, with 100 one-way crossings per seed and method. Values are mean ± sample s.d. across three seeds. Metric Baseline PSC Shank-positive trials 33/5033/50 /0/50 Shank-collision events 4141 0 Foot-scuff events 1414 7 Completion 49/5049/50 /50/50 Stuck events 11 00 Table 6: Production stair-climbing hardware evaluation over 5050 ascent–descent pairs per policy. Shank-positive trials contain at least one lower-leg collision. 4 Related Work Curriculum learning and adaptive environment sampling. Curriculum learning adapts the training distribution according to competence, learning progress, or task difficulty [9, 10]. Performance-based terrain progression is widely used in legged locomotion [1, 2], while perceptive methods additionally vary terrain and sensing conditions [11]. More general adaptive sampling methods prioritize environments using learning progress, policy uncertainty, or learning potential, including ALP-GMM [12], PLR [3], ACCEL [13], and Active Domain Randomization [14]. Recent locomotion-specific approaches include LP-ACRL [4] and HACL [15]. ADD learns an environment-conditioned task-return distribution and uses a CVaR-derived regret signal to guide a diffusion-based environment generator [16]. PSC instead uses predictions of future safety exposure to reallocate training experience over structured contexts and randomized environment events. Risk-aware curricula and failure-driven sampling. RACGEN concentrates an auxiliary sampler on low-return contexts in heavy-tailed task distributions [17], while CeSoR combines adverse-condition sampling with a soft-risk policy objective [18]. Other approaches incorporate safety directly into curriculum construction: curriculum induction learns when to deploy a reset controller [19], while Safety-Prioritizing Curricula initially favor tasks with fewer constraint violations [20]. Failure predictors have also been used to search for rare catastrophic outcomes during evaluation [21], and TACL learns transition difficulty from success and failure labels to generate capability-matched trajectories [22]. PSC differs in learning a shared distribution of future safety cost from rollout states and using this prediction to determine where subsequent training experience is allocated. Safety critics and distributional risk. Safety critics are commonly used to modify policy behavior or constrain policy optimization. CPO imposes cumulative-cost constraints on the policy update [23]; Recovery RL combines a safety critic with a recovery policy [24]; and Agile But Safe uses a reach-avoid value function for locomotion recovery [25]. Distributional safety critics further apply risk measures such as CVaR to reward–safety trade-offs [26]. PSC builds on distributional RL [27, 5, 28] and the standard CVaR risk measure [29], but uses the resulting safety prediction for a different intervention: selecting which simulated experience is collected next. Unlike risk-sensitive locomotion methods [30, 31], the safety model does not modify the task reward or PPO loss and is not used at deployment. 5 Conclusion Closing the final gap toward near-perfect reliability is increasingly challenging, yet indispensable for the safe deployment of learned control policies on legged robots. Toward this goal, we introduced Predictive Safety Curricula (PSC), which uses predictions of future safety exposure to allocate locomotion training experience. A distributional safety critic drives both context allocation and randomized-event replay, concentrating experience in regions of the training distribution associated with elevated predicted risk. Across all evaluated ANYmal-D terrain and observation conditions, PSC achieves the highest mean reliability among the compared curricula. Ablations favor learned predictive priorities over direct empirical tail statistics, while critic diagnostics show that the learned scores meaningfully rank realized safety return and enrich subsequent collision and contact events. The allocation principle transfers to production locomotion stacks and through teacher–student distillation. On ANYmal-D hardware, PSC reduces shank-collision incidence by 63%63\% relative to the learning-progress curriculum across three matched training seeds, with a reduction in every seed. On the production stair-climbing platform, no shank collision is observed with PSC in the evaluated hardware trials, compared with 33/5033/50 shank-positive trials for the baseline, at similar observed mean traversal speeds. Together, these results show that predictions of future safety exposure can provide a practical signal for deciding which experience a locomotion policy should train on. Limitations and outlook. PSC currently relies on a hand-specified safety cost and can adapt experience only through environment variables exposed to the curriculum. The hardware results illustrate the first limitation directly: reducing targeted shank contacts does not necessarily suppress other interactions, such as foot scuffs that are absent from the safety cost. Future work should therefore consider structured safety predictions that preserve multiple failure modes rather than collapsing safety exposure into a single fixed scalar objective. On the allocation side, a natural extension is to move beyond reweighting and replay within a predefined environment parameterization toward actively discovering or generating conditions predicted to expose policy weaknesses. Together, richer safety representations and active failure-condition discovery would extend PSC from prioritizing safety-relevant experience within an existing curriculum to shaping the training distribution itself around the remaining reliability gaps. Acknowledgments This work was supported in part by EuroHPC under project EHPC-AIF-2026FL01-031. References [1] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter (2020) Learning quadrupedal locomotion over challenging terrain. Science Robotics 5 (47), p. eabc5986. Cited by: §1, §4. [2] N. Rudin, D. Hoeller, P. Reist, and M. Hutter (2022) Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 164, p. 91–100. Cited by: §1, §4. [3] M. Jiang, E. Grefenstette, and T. Rocktäschel (2021) Prioritized level replay. In International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 4940–4950. Cited by: §1, §3, §4. [4] Z. Li, C. Li, and M. Hutter (2026) Scaling rough terrain locomotion with automatic curriculum reinforcement learning. IEEE Robotics and Automation Letters 11 (8), p. 9295–9302. External Links: Document Cited by: §1, §3, §4. [5] W. Dabney, M. Rowland, M. G. Bellemare, and R. Munos (2018) Distributional reinforcement learning with quantile regression. In AAAI Conference on Artificial Intelligence, Cited by: §2.2, §2.2, §4. [6] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: §3. [7] M. Mittal, P. Roth, J. Tigue, et al. (2025) Isaac lab: a gpu-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831. Cited by: §3. [8] C. Schwarke, M. Mittal, N. Rudin, D. Hoeller, and M. Hutter (2025) RSL-rl: a learning library for robotics research. arXiv preprint arXiv:2509.10771. Cited by: §3. [9] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone (2020) Curriculum learning for reinforcement learning domains: a framework and survey. Journal of Machine Learning Research 21 (181), p. 1–50. Cited by: §4. [10] R. Portelas, C. Colas, L. Weng, K. Hofmann, and P. Oudeyer (2020) Automatic curriculum learning for deep RL: a short survey. arXiv preprint arXiv:2003.04664. Cited by: §4. [11] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter (2022) Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics 7 (62), p. eabk2822. Cited by: §4. [12] R. Portelas, C. Colas, K. Hofmann, and P. Oudeyer (2020) Teacher algorithms for curriculum learning of deep RL in continuously parameterized environments. In Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 100, p. 835–853. Cited by: §4. [13] J. Parker-Holder, M. Jiang, M. Dennis, M. Samvelyan, J. Foerster, E. Grefenstette, and T. Rocktäschel (2022) Evolving curricula with regret-based environment design. In International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, p. 17473–17498. Cited by: §4. [14] B. Mehta, M. Diaz, F. Golemo, C. J. Pal, and L. Paull (2020) Active domain randomization. In Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 100, p. 1162–1176. Cited by: §4. [15] P. Mishra, A. H. Raj, X. Xiao, and D. Manocha (2025) HACL: history-aware curriculum learning for fast locomotion. arXiv preprint arXiv:2505.18429. Cited by: §4. [16] H. Chung, J. Lee, M. Kim, D. Kim, and S. Oh (2024) Adversarial environment design via regret-guided diffusion models. In Advances in Neural Information Processing Systems, Vol. 37, p. 63715–63746. External Links: Document Cited by: §4. [17] C. Koprulu, T. D. Simão, N. Jansen, and U. Topcu (2023) Risk-aware curriculum generation for heavy-tailed task distributions. In Conference on Uncertainty in Artificial Intelligence, Proceedings of Machine Learning Research, Vol. 216, p. 1132–1142. Cited by: §4. [18] I. Greenberg, Y. Chow, M. Ghavamzadeh, and S. Mannor (2022) Efficient risk-averse reinforcement learning. In Advances in Neural Information Processing Systems, Cited by: §4. [19] M. Turchetta, A. Kolobov, S. Shah, A. Krause, and A. Agarwal (2020) Safe reinforcement learning via curriculum induction. In Advances in Neural Information Processing Systems, Cited by: §4. [20] C. Koprulu, T. D. Simão, N. Jansen, and U. Topcu (2025) Safety-prioritizing curricula for constrained reinforcement learning. In International Conference on Learning Representations, Cited by: §4. [21] J. Uesato, A. Kumar, C. Szepesvári, T. Erez, A. Ruderman, K. Anderson, K. Dvijotham, N. Heess, and P. Kohli (2019) Rigorous agent evaluation: an adversarial approach to uncover catastrophic failures. In International Conference on Learning Representations, Cited by: §4. [22] R. Liu, T. Liu, B. Jia, F. Zhong, X. Tong, H. Xie, and S. Huang (2026) Trajectory-level automatic curriculum learning for legged locomotion on unstructured terrain. arXiv preprint arXiv:2608.16164. Cited by: §4. [23] J. Achiam, D. Held, A. Tamar, and P. Abbeel (2017) Constrained policy optimization. In International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 70, p. 22–31. Cited by: §4. [24] B. Thananjeyan, A. Balakrishna, S. Nair, M. Luo, K. Srinivasan, M. Hwang, J. E. Gonzalez, J. Ibarz, C. Finn, and K. Goldberg (2021) Recovery RL: safe reinforcement learning with learned recovery zones. IEEE Robotics and Automation Letters. Cited by: §4. [25] T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi (2024) Agile but safe: learning collision-free high-speed legged locomotion. In Robotics: Science and Systems, Cited by: §4. [26] Q. Yang, T. D. Simão, S. H. Tindemans, and M. T. J. Spaan (2023) Safety-constrained reinforcement learning with a distributional safety critic. Machine Learning 112 (3), p. 859–887. Cited by: §4. [27] M. G. Bellemare, W. Dabney, and R. Munos (2017) A distributional perspective on reinforcement learning. In International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 70, p. 449–458. Cited by: §4. [28] W. Dabney, G. Ostrovski, D. Silver, and R. Munos (2018) Implicit quantile networks for distributional reinforcement learning. In International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 80, p. 1096–1105. Cited by: §4. [29] R. T. Rockafellar and S. Uryasev (2002) Conditional value-at-risk for general loss distributions. Journal of Banking & Finance 26 (7), p. 1443–1471. Cited by: §4. [30] L. Schneider, J. Frey, T. Miki, and M. Hutter (2024) Learning risk-aware quadrupedal locomotion using distributional reinforcement learning. In IEEE International Conference on Robotics and Automation, Cited by: §4. [31] J. Shi, C. Bai, H. He, L. Han, D. Wang, B. Zhao, M. Zhao, X. Li, and X. Li (2024) Robust quadrupedal locomotion via risk-averse policy learning. In IEEE International Conference on Robotics and Automation, Cited by: §4.