Paper deep dive
Steering Generative Robot Policies with Lexicographic Preferences
Yixuan Jia, Jonathan P. How
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pretrained generative robot policies can produce effective behaviors across diverse environments, but deployment can lead to requirements and preferences that may not have been represented during training. Furthermore, at deployment, an operator, user, or application may assign these requirements and preferences a priority order that can vary across deployments. For example, embodiment-specific feasibility constraints may need to be satisfied first, while user-specific preferences guide behavior among the feasible options. We show that a frozen generative robot policy---based on either diffusion or flow matching---can be steered at inference time to respect such lexicographically ordered deployment objectives. To achieve this, we introduce two modifications to the sampler. First, we apply dynamic-barrier guidance to sampled trajectories, constraining lower-priority updates so that higher-priority costs do not increase (up to first order). Second, we select the executed sample using a cascade that successively filters candidate samples according to each priority level. The policy weights remain unchanged. On a navigation benchmark, we demonstrate that our method improves success, traversability, and preference compliance over the frozen policy, and achieves substantially better compliance than tuned weighted-sum baselines. The same method transfers to a flow-matching manipulation policy on LIBERO, where it improves compliance without reducing task success. A controlled manipulation study further shows that, in settings where a fixed weight can match the desired ordering, the dynamic barrier reaches comparable best performance over a substantially wider range of parameter settings.
Tags
Links
- Source: https://arxiv.org/abs/2609.15014v1
- Canonical: https://arxiv.org/abs/2609.15014v1
Trouble viewing inline? Open PDF directly →
Full Text
64,162 characters extracted from source content.
Expand or collapse full text
Steering Generative Robot Policies with Lexicographic Preferences Yixuan Jia Jonathan P. How †thanks: Y. Jia and J. How are with Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA 02319 USA (e-mail: yixuany@mit.edu, jhow@mit.edu). Abstract Pretrained generative robot policies can produce effective behaviors across diverse environments, but deployment can lead to requirements and preferences that may not have been represented during training. Furthermore, at deployment, an operator, user, or application may assign these requirements and preferences a priority order that can vary across deployments. For example, embodiment-specific feasibility constraints may need to be satisfied first, while user-specific preferences guide behavior among the feasible options. We show that a frozen generative robot policy—based on either diffusion or flow matching—can be steered at inference time to respect such lexicographically ordered deployment objectives. To achieve this, we introduce two modifications to the sampler. First, we apply dynamic-barrier guidance to sampled trajectories, constraining lower-priority updates so that higher-priority costs do not increase (up to first order). Second, we select the executed sample using a cascade that successively filters candidate samples according to each priority level. The policy weights remain unchanged. On a navigation benchmark, we demonstrate that our method improves success, traversability, and preference compliance over the frozen policy, and achieves substantially better compliance than tuned weighted-sum baselines. The same method transfers to a flow-matching manipulation policy on LIBERO, where it improves compliance without reducing task success. A controlled manipulation study further shows that, in settings where a fixed weight can match the desired ordering, the dynamic barrier reaches comparable best performance over a substantially wider range of parameter settings. I INTRODUCTION Large pretrained generative robot policies can map raw sensory input directly to motion without a map, a planner, or task-specific engineering, and their capabilities continue to improve [1, 2, 3, 4]. However, broad generalization does not eliminate the need to adapt behavior to the requirements of a particular deployment. These requirements may include embodiment-specific feasibility constraints as well as user- or application-specific preferences that were not represented during training. A navigation policy, for example, may propose a route that is not traversable by the particular robot on which it is deployed, or a delivery robot’s operator may prefer that it not cut across residential lawns. Similarly, a household may prefer that a manipulator not carry a dripping bowl over the stove. Such behaviors need not reflect a failure of the pretrained policy itself; rather, they expose a gap between the broad behaviors learned during pretraining and the specific requirements imposed at deployment. Fig. 1: Within each panel both rollouts come from the same policy at the same initial state: red is that policy on its own, blue is the same policy with our dynamic-barrier update applied to the sampler. Adapting a pretrained policy to these deployment-specific requirements raises two key challenges. First, adaptation should leave the pretrained policy frozen: retraining or fine-tuning for every deployment requires additional data and computation and undermines the benefits of a broadly reusable pretrained policy. Instead, we adapt behavior at inference time by intervening in the sampling process of generative policies, steering their outputs toward behaviors that satisfy deployment-specific requirements and preferences while leaving the policy itself unchanged (Figure 1). Second, deployment objectives may be specified with a clear priority order by the operator, user, or application, a structure not addressed by prior steering methods. For example, a delivery robot should stay off the lawn only when doing so does not require traversing a curb that its embodiment cannot climb. Here, the distinction between higher- and lower-priority objectives is structural: improvements in a lower-priority objective should never compensate for violating a higher-priority one. Weighted sums cannot guarantee this behavior because they assign objectives a finite exchange rate. Fixed weights can enforce a desired priority only when the possible ranges of the objectives are known, yet these ranges can vary substantially across deployment instances—for example, a longer delivery route may accumulate much more lawn cost than a shorter one. Consequently, weights that preserve the desired priority on one instance may fail to preserve it on another. We address this problem by enforcing the priority order directly at inference time rather than assigning a fixed trade-off between objectives. Our method operates in two stages: it first steers individual samples and then selects which resulting sample to execute. Given a set of ordered runtime costs, our method applies dynamic-barrier guidance to steer each sampled trajectory while preserving higher-priority objectives to first order. It then applies a lexicographic selection cascade that filters the resulting candidates one priority level at a time. Because both mechanisms operate on sampled trajectories rather than modifying the policy network, the pretrained policy remains frozen and the same approach applies across different iterative generative samplers. To this end, we make the following contributions: 1. Inference-time steering for lexicographically ordered objectives. We formulate steering frozen generative robot policies under lexicographically ordered runtime costs and develop a sampler-agnostic approach combining dynamic-barrier guidance with lexicographic candidate selection. The method applies to both diffusion and flow-matching policies without retraining or fine-tuning. 2. Navigation under ordered feasibility and preference costs. We introduce a last-mile delivery benchmark that creates frequent conflicts between embodiment-specific traversability and user-specified surface preferences. Our method substantially improves success, preference compliance, and traversability over the frozen policy and achieves better compliance than tuned weighted-sum baselines. 3. Cross-domain transfer and reduced tuning sensitivity. On LIBERO [5], the same method transfers to a flow-matching manipulation policy and improves preference and feasibility compliance without sacrificing task success. When objective scales remain relatively consistent across episodes, as in this setting, a well-tuned fixed weight is expected to preserve the desired ordering, and our method accordingly matches its performance. A controlled D3IL [6] study then shows the practical difference: the dynamic barrier maintains comparable best performance over a substantially wider range of parameter settings, while the effective fixed weight occupies a narrower range and changes with steering strength. I Related Work I-A Inference-Time Steering of Generative Policies Inference-time guidance modifies the behavior of a pretrained generative model without retraining its parameters. Gradient-based guidance was introduced for diffusion models [7] and later applied to trajectory generation [8]. For frozen robot policies, existing methods steer generated trajectories through gradient updates [9, 10], reweighting or resampling [11, 12, 13], or selection among generated candidates [14]. Other approaches explicitly impose safety constraints through barrier functions, projection, or constrained optimization [15, 16, 17, 18]. Recent work also considers inference-time modification of flow-matching policies [19]. These approaches generally combine multiple objectives through a weighted cost or a single reward, or separate a single constraint from an otherwise scalar objective. In contrast, we consider multiple runtime costs with an explicit priority order. I-B Prioritized and Lexicographic Objectives Prioritized objectives have a long history in robotics and optimization, including admissibility-based local planning [20], task-priority control and hierarchical quadratic programming [21, 22], rulebooks for autonomous driving [23], and control-barrier-function methods that prioritize safety over performance [24]. Lexicographic objectives have also been studied in reinforcement learning and optimization [25, 26, 27, 28]. Our gradient update builds on dynamic-barrier optimization [29], which modifies descent on one objective to enforce progress on a prioritized objective. Pareto Navigation Gradient Descent [30] extends a similar quadratic-program structure to several objectives, but treats them as an unordered Pareto set. We instead consider explicitly ordered runtime costs and combine prioritized gradient steering with a lexicographic candidate-selection rule inside the sampling process of a frozen generative policy. I Method I-A Problem Setting A frozen generative policy maps an observation o in an observation space O to S candidate action sequences through an iterative sampling process. We call a single action sequence a sample and write it as a point x∈=ℝT×dax =R^T× d_a, a horizon of T actions of dimension dad_a. Starting from noise x(0)∼(0,I)x^(0) (0,I), each candidate is refined by K learned sampling steps x(k)=Stepθ(x(k−1),k,o),k=1,…,K,x^(k)=Step_θ\! (x^(k-1),k,o ), k=1,…,K, (1) where Stepθ:×1,…,K×→Step_θ:X×\1,…,K\×O is one step of the frozen sampler and may correspond, for example, to a diffusion reverse step or a flow-matching integration step. We index the S candidates by i=1,…,Si=1,…,S, writing xix_i for the i-th sample, and we drop the step superscript, writing x for the current iterate when the step index is not needed. The runtime costs we wish to impose depend on where the robot, or the object it carries, actually travels, rather than on the policy’s internal action encoding. We therefore evaluate them on a workspace trajectory y∈=ℝT×dwy =R^T× d_w, where dwd_w is the dimension of the workspace, and assume a differentiable decoder Φ:→,y=Φ(x), :X , y= (x), (2) that maps a sample to the trajectory it represents. In our navigation experiments dw=2d_w=2 and Φ integrates the sampled waypoint displacements into the world frame; in our manipulation experiments dw=3d_w=3 (LIBERO) or dw=2d_w=2 (D3IL) and Φ maps the sampled action chunk to the predicted path of the end effector. At deployment, we introduce L runtime costs cj:→ℝ≥0,j=1,…,L,c_j:Y _≥ 0, j=1,…,L, (3) each continuously differentiable, together with a priority order c1≻c2≻⋯≻cLc_1 c_2 ·s c_L, ordered from highest to lowest priority. The ordering is lexicographic: improving a lower-priority objective must not come at the expense of a higher-priority one. The costs may represent, for example, physical feasibility, safety requirements, operator preferences, or other deployment-specific objectives that were not available when the policy was trained. Our experiments use two ordered costs, a feasibility cost g:=c1g:=c_1 and a deployment preference f:=c2f:=c_2 with g≻fg f, and we present the method for this case; the construction extends to deeper hierarchies, as discussed at the end of Section I-B. We keep the policy weights, sampling schedule, and observations fixed. Our method requires only the following three ingredients. (i) Access to the sample x∈x between sampling steps. (i) The differentiable decoder Φ of Equation 2, together with a reconstruction map Ψ:×→ :Y×X that writes an edited trajectory back into a sample, overwriting only the channels that Φ reads and leaving the rest of the sample unchanged, so that Φ(Ψ(y,x))=y \! ( (y,x) )=y for all x∈x and all y in the range of Φ . We abbreviate Ψ(y,x) (y,x) by Φ−1(y) ^-1(y) where the sample being overwritten is clear from context. (i) Optionally, the policy’s value head V:×→ℝV:X×O , used only for final candidate selection. We impose the ordering at two stages, summarized in Algorithm 1. Lexicographic barrier guidance modifies individual samples while preserving higher-priority objectives, and lexicographic candidate selection applies the same ordering when choosing which candidate to execute. Algorithm 1 Lexicographically Steered Sampling (one planning step). The parameterization M, step size η and barrier constants αb,βb _b, _b are defined in Section I-B; the tolerances εg,εf,εp _g, _f, _p and the progress score pip_i in Section I-C. 1: observation o; frozen sampler StepθStep_θ; ordered costs g≻fg f; decoder Φ and reconstruction Φ−1 ^-1; parameterization M; step size η; barrier constants αb,βb _b, _b; tolerances εg,εf,εp _g, _f, _p 2: draw S samples x1,…,xSx_1,…,x_S from noise 3: for k=1,…,Kk=1,…,K do 4: // frozen sampler, unchanged 5: xi←Stepθ(xi,k,o)x_i _θ(x_i,k,o) for all i 6: // barrier guidance (Section I-B) 7: for each candidate i do 8: ui←argminu∥M(u)−Φ(xi)∥2u_i← _u M(u)- (x_i) ^2 // e.g. fit B-spline 9: ϕ←min(αbg(ui),βb∥∇g(ui)∥2)φ← \! ( _b\,g(u_i),\; _b ∇ g(u_i) ^2 ) 10: λ←max((ϕ−⟨∇f(ui),∇g(ui)⟩)/∥∇g(ui)∥2, 0)λ← \! ( (φ- ∇ f(u_i),∇ g(u_i) ) / ∇ g(u_i) ^2,\;0 ) 11: ui←ui−η(∇f(ui)+λ∇g(ui))u_i← u_i-η\,(∇ f(u_i)+λ\,∇ g(u_i)) 12: xi←Φ−1(M(ui))x_i← ^-1(M(u_i)) 13: end for 14: end for 15: // guidance may run after every step or as repeated iterations on the final sample 16: // selection (Section I-C) 17: yi←Φ(xi)y_i← (x_i) 18: pi←p_i← task progress of candidate i, for all i 19: ℐ←1,…,SI←\1,…,S\, optionally dropping standstill candidates 20: // feasibility first 21: ℐ←i∈ℐ:g(yi)≤mini′∈ℐg(yi′)+εgI←\\,i :g(y_i)≤ _i g(y_i )+ _g\,\ 22: // then preference 23: ℐ←i∈ℐ:f(yi)≤mini′∈ℐf(yi′)+εfI←\\,i :f(y_i)≤ _i f(y_i )+ _f\,\ 24: // then task progress 25: ℐ←i∈ℐ:pi≥maxi′∈ℐpi′−εpI←\\,i :p_i≥ _i p_i - _p\,\ 26: return the candidate in ℐI preferred by the value head, if one exists I-B Lexicographic Barrier Guidance Intervening between sampler steps We apply guidance directly to the sample returned by the frozen sampler, x←Stepθ(x,k,o),x←Steer(x),x _θ(x,k,o), x (x), where Steer:→Steer:X is the update derived below, rather than modifying the network output used inside StepθStep_θ. The intervention therefore has the same form for diffusion and flow-matching samplers. Editable coordinates We edit a parameterization of the workspace trajectory: a vector of editable coordinates u∈=ℝm×dwu =R^m× d_w together with a differentiable reconstruction map M:→,y=M(u).M:U , y=M(u). (4) For example, in our navigation task u collects m=7m=7 B-spline control points and M is linear, M(u)=BuM(u)=Bu with B∈ℝT×mB ^T× m the clamped cubic B-spline basis, so an update moves a few control points rather than every waypoint individually. In our manipulation tasks the trajectory is edited directly: m=Tm=T and M is the identity. The runtime costs are defined on Y, so we compose them with the parameterization and, with a slight abuse of notation, write the composed costs with the same symbols: g(y)g(y) denotes the cost on a workspace trajectory and g(u)g(u) its composition with M, g(u):=c1(M(u)),f(u):=c2(M(u)).g(u):=c_1\! (M(u) ), f(u):=c_2\! (M(u) ). (5) Identifying U with ℝmdwR^md_w and Y with ℝTdwR^Td_w, the chain rule gives ∇g(u)=JM(u)⊤∇c1(M(u)),∇ g(u)=J_M(u)^\! \,∇ c_1\! (M(u) ), (6) where JM(u)J_M(u) is the Jacobian of M at u, equal to the constant matrix B in the linear case, and likewise for f. All gradients below are taken with respect to u, and ⟨⋅,⋅⟩ ·,· and ∥⋅∥2 · _2 are the Euclidean inner product and norm on U. After editing u, we reconstruct the sample as x←Φ−1(M(u)).x← ^-1\! (M(u) ). Dynamic barrier condition Fix u∈u , let d∈d be an update direction and η>0η>0 a step size, and write u+=u−ηdu^+=u-η d. Since g is continuously differentiable, the first-order change in the prioritized cost is g(u+)−g(u)=−η⟨∇g(u),d⟩+o(η).g(u^+)-g(u)=-η ∇ g(u),d +o(η). (7) Consequently, the condition ⟨∇g(u),d⟩≥0 ∇ g(u),d ≥ 0 prevents g from increasing, up to first order. A dynamic barrier strengthens this condition by requiring a positive decrease whenever the prioritized cost is away from its desired value. Following dynamic-barrier gradient descent [29], we define the dynamic barrier function ϕ:→ℝ≥0φ:U _≥ 0 by ϕ(u):=min(αbg(u),βb‖∇g(u)‖22),αb,βb>0.φ(u):= ( _b\,g(u),\; _b \|∇ g(u) \|_2^2 ), _b, _b>0. (8) Gong et al. write the first term as αb(g(u)−g^) _b(g(u)- g), where g g is a known lower bound on g. Our runtime costs are non-negative and use zero as their desired value, so we set g^=0 g=0. Since g(u)≥0g(u)≥ 0 and αb,βb>0 _b, _b>0, we have ϕ(u)≥0φ(u)≥ 0 for every u∈u , with ϕ(u)=0φ(u)=0 exactly when g(u)=0g(u)=0 or ∇g(u)=0∇ g(u)=0. The two terms determine how much decrease we require from the prioritized cost. The first term, αbg(u) _b\,g(u), requires a larger decrease when the current cost is large. The second term, βb‖∇g(u)‖22 _b\|∇ g(u)\|_2^2, limits this requirement when the local gradient is small. Taking the minimum prevents the barrier from demanding more progress than the local cost gradient can support. The corresponding barrier condition is ⟨∇g(u),d⟩≥ϕ(u) ∇ g(u),d ≥φ(u). Substituting it into Equation 7 gives g(u+)−g(u)≤−ηϕ(u)+o(η)≤o(η),g(u^+)-g(u)≤-η\,φ(u)+o(η)≤ o(η), (9) so any direction satisfying the barrier condition decreases the prioritized cost to first order, strictly so whenever ϕ(u)>0φ(u)>0. Steering update The steering direction is the one closest to ordinary descent on the preference while satisfying the feasibility barrier: d⋆(u)=argmind∈12‖d−∇f(u)‖22s.t.⟨∇g(u),d⟩≥ϕ(u).d (u)= _d 12 \|d-∇ f(u) \|_2^2 .t.~ ∇ g(u),d ≥φ(u). (10) When ∇g(u)≠0∇ g(u)≠ 0, the feasible set is a non-empty half-space and this one-constraint quadratic program has the closed-form solution d⋆(u)=∇f(u)+λ(u)∇g(u),d (u)=∇ f(u)+λ(u)\,∇ g(u), (11) with multiplier λ(u)=max(ϕ(u)−⟨∇f(u),∇g(u)⟩‖∇g(u)‖22, 0),λ(u)= ( φ(u)- ∇ f(u),∇ g(u) \|∇ g(u) \|_2^2,\;0 ), (12) and the update is u←u−ηd⋆(u).u← u-η\,d (u). If ⟨∇f(u),∇g(u)⟩≥ϕ(u) ∇ f(u),∇ g(u) ≥φ(u), ordinary descent on f already satisfies the feasibility barrier, so λ(u)=0λ(u)=0. Otherwise, λ(u)>0λ(u)>0 adds the smallest correction along ∇g(u)∇ g(u) needed to make the barrier active, ⟨∇g(u),d⋆(u)⟩=ϕ(u) ∇ g(u),d (u) =φ(u). If ∇g(u)=0∇ g(u)=0, then ϕ(u)=0φ(u)=0 and the barrier contains no local information with which to alter the trajectory. We therefore set λ(u)=0λ(u)=0 and recover ordinary descent on f. In implementation, we apply the same branch when ‖∇g(u)‖22\|∇ g(u)\|_2^2 falls below a small numerical threshold, rather than adding that threshold to the theoretical denominator. The multiplier is therefore dynamic rather than fixed: it depends on the current value of the feasibility cost, the magnitude of its gradient, and its local alignment with the preference gradient. It vanishes when preference descent already respects feasibility and increases only when required to enforce the higher-priority barrier. Generalization to deeper hierarchies The construction above extends to an ordered set of costs c1≻c2≻⋯≻cL,c_1 c_2 ·s c_L, where cLc_L is the objective being optimized and each higher-priority cost cjc_j, j<Lj<L, has its own barrier requirement ϕj(u):=min(αjcj(u),βj‖∇cj(u)‖22) _j(u):= ( _jc_j(u), _j\|∇ c_j(u)\|_2^2). If all barrier conditions are compatible, we can enforce them simultaneously: ⟨∇cj(u),d⟩≥ϕj(u),j=1,…,L−1, ∇ c_j(u),d ≥ _j(u), j=1,…,L-1, while choosing the direction closest to ordinary descent on cLc_L. A similar multi-constraint QP appears in Pareto Navigation Gradient Descent [30], where the constrained objectives are treated symmetrically. When the barrier conditions conflict, that is, when the intersection of the half-spaces above is empty, enforcing them simultaneously does not specify which condition should be relaxed. To preserve the priority order, a non-negative slack ξj _j can be introduced at each level, ⟨∇cj(u),d⟩≥ϕj(u)−ξj,ξj≥0, ∇ c_j(u),d ≥ _j(u)- _j, _j≥ 0, and the slacks can be minimized lexicographically: first ξ1 _1, then ξ2 _2 without worsening the optimum at level 11, and so on. Only after the best achievable slack has been fixed at every prioritized level do we choose the direction closest to ∇cL(u)∇ c_L(u). Thus, when the barrier conditions conflict, a lower-priority condition is relaxed only after all higher-priority conditions have been satisfied as well as possible. The two-cost case used in our experiments is the special case L=2L=2. There is then only one prioritized cost, c1=gc_1=g, and one objective to optimize, c2=fc_2=f. Its single barrier condition is feasible whenever ∇g(u)≠0∇ g(u)≠ 0, so the optimal slack is zero and the general formulation reduces exactly to the one-constraint QP in Equation 10, with the closed-form update in Equation 11. I-C Lexicographic Candidate Selection Barrier guidance modifies each candidate trajectory independently. The policy still produces S candidates, so we apply the same priority ordering again when choosing which candidate to execute. This selection stage can include several lexicographic levels. For example, in navigation we consider feasibility, preference, task progress, and finally the policy’s value head. Let yi=Φ(xi)y_i= (x_i) denote the workspace trajectory of candidate i and let ℐ1⊆1,…,SI_1 \1,…,S\ denote the initial set of candidate indices. We filter this set sequentially according to the ordered costs c1≻c2≻⋯≻cL.c_1 c_2 ·s c_L. For each priority level j, let cjmin=mini∈ℐjcj(yi)c_j = _i _jc_j(y_i) be the best cost among the candidates that survived the previous levels. We retain all candidates whose cost is within a tolerance εj>0 _j>0 of this minimum: ℐj+1=i∈ℐj:cj(yi)≤cjmin+εj.I_j+1= \i _j\;:\;c_j(y_i)≤ c_j + _j \. (13) The tolerance εj _j is defined in the units of cost cjc_j. It prevents small differences in noisy runtime costs from causing brittle decisions while preserving the priority ordering: a candidate rejected at level j cannot be recovered by performing better on any lower-priority objective. Because the threshold is defined relative to the minimum over the current candidate set, the minimizer at level j always survives, so ∅≠ℐj+1⊆ℐj _j+1 _j and at least one candidate always survives each stage. After all prioritized costs have been considered, we rank the remaining candidates by task progress. Let pi∈ℝp_i denote the progress score of candidate i, with larger values indicating better progress. We retain candidates satisfying pi≥pmax−εp,pmax=maxi∈ℐL+1pi.p_i≥ p - _p, p = _i _L+1p_i. If the frozen policy provides a value head, we use it only as the final tie-break among the remaining candidates. The resulting cascade is lexicographic because each objective is evaluated only over candidates admitted by all higher-priority levels. Lower-priority objectives can therefore distinguish between candidates that are approximately equivalent at higher-priority levels, but they cannot compensate for a worse higher-priority cost. Barrier guidance and candidate selection impose the same ordering at complementary stages of inference. The barrier modifies each candidate while respecting prioritized costs, and the selection cascade prevents the final choice from reintroducing a trade between those costs. IV Last-Mile Delivery with a Diffusion Policy Fig. 2: The navigation benchmark: 60 door-to-door episodes in CARLA Town01, rebuilt in Isaac Sim. We build a navigation benchmark inspired by last-mile delivery, where a robot is tasked with reaching the front doors of different residences. The robot must account for both what its embodiment can traverse and where it is permitted or preferred to drive. For example, a residential lawn is physically traversable but may be undesirable, whereas a tall curb may be physically impassable for the deployed robot. These two considerations therefore have different priorities: the robot should avoid the lawn when possible, but should never do so at the expense of a route it cannot traverse. We construct episodes in which the direct route often encounters both types of constraints (see Figure 2). Fig. 3: Selected episodes of the delivery task. Labels give meters driven on lawn and whether the goal was reached. Ours drives 0.00.0mm of lawn and reaches the goal in all eight; the baselines drive up to 3939mm of lawn, and the two that cross no lawn do so by ending 4141 and 1818mm short. Purple marks a 1010–2020cmcm height step on drivable ground, shaded by magnitude; triangles mark where a route climbs (▲ ) or drops (▽ ) more than 1010cmcm. Note that every triangle sits on a baseline. Ours sometimes drives further than needed (row 1 column 3 and row 2 column 1): policy and steering are both local, so a compliant shortcut can only be ruled out by probing for it. IV-A Simulation Setup Simulator and Scene. We evaluate in NVIDIA Isaac Sim [31] on a residential town we took from [32], which comes from CARLA Town01 [33]. Robot. We use an AgileX Scout 2.0 in the simulation, which is equipped with an RGB-D camera and a LiDAR. Our step-climbing tests indicate that 0.10 m0.10\,m is a conservative recommended upper bound on traversable step height. Episodes. We select 6060 door-to-door start–goal pairs spanning 9.29.2–66 m66\,m (median 23.4 m23.4\,m) along the sidewalks (see Figure 2). To ensure a fair test of both deployment requirements, we verify that every pair admits a feasible pavement-only route wide enough for the robot; the resulting route is at most 1.61×1.61× the corresponding free-space route (median 1.04×1.04×). At the same time, we deliberately select pairs whose direct shortcuts expose the two constraints: 55/6055/60 straight-line routes cross a lawn, while 43/6043/60 encounter a step ≥ 0.10 m0.10\,m. Thus, the benchmark provides feasible alternatives while systematically exposing the policy to conflicts between the two requirements. Metrics. An episode is considered successful if the robot comes within 1 m1\,m of the goal within 300 s300\,s. We report the following metrics: (i) success, the number of successful episodes out of 60; (i) on-lawn distance, the mean lawn distance per episode measured at the robot center; (i) zero-lawn episodes, with less than 0.05 m0.05\,m of lawn contact; (iv) traversability failures, failed episodes whose stall point contains a ≥ 0.10 m0.10\,m step under the robot footprint; (v) curb episodes, episodes with any travel within one half-width of a road cell; and (vi) driven distance. IV-B Implementation We use NavDP [4] as our base policy. NavDP is a goal-conditioned RGB-D diffusion policy with a learned value head. It samples S=16S=16 candidate trajectories, each consisting of T=24T=24 waypoints, over K=10K=10 DDPM steps, and executes the selected trajectory using a model-predictive controller. Trajectory Representation. We represent each sampled trajectory as a position sequence y∈ℝT×2y ^T× 2 by cumulatively summing its waypoint displacements and transforming it from NavDP’s camera frame to the world frame. For gradient refinement, we fit this path using a clamped cubic B-spline basis and optimize the spline control points. The refined path is then mapped back to waypoint displacements, while the sample’s yaw channel is left unchanged. Cost Fields. The preference cost f is obtained from a semantic class map. We use a 0.35 m0.35\,m ramp around the lawn boundary, corresponding to the robot’s half-width, so that cost begins when the footprint reaches the lawn; bilinear interpolation makes the field differentiable. The feasibility cost g is constructed online from lidar as a traversability risk field, following the approach of STEP [34]. It marks steps at or above the 0.10 m0.10\,m bound as high risk and assigns unknown-risk to occluded cells, where negative obstacles may be hidden. We aggregate risk using conditional value-at-risk (CVaR) [35], taking the mean of the worst 20%20\% of values within a 0.4 m0.4\,m footprint disc. Steering. Although the barrier update of Equation 11 can be applied between any two sampler steps, we apply all barrier updates after the final DDPM step. Applying them earlier allows subsequent denoising steps to partially undo the changes. We refine each completed sample for 15 barrier-update iterations in a clamped cubic B-spline basis with 77 control points. Each control-point update is limited by a 0.15 m0.15\,m trust region, and we use η=1.0η=1.0 and αb=βb=1 _b= _b=1. The first control point is pinned to the robot, and each update is projected perpendicular to the local path tangent to prevent the refinement from reducing cost simply by retreating along the path. Selection. We select the executed trajectory using the lexicographic cascade with a horizon discount γh=0.85 _h=0.85 on the feasibility cost, feasibility tolerance εg=0.1 _g=0.1, preference tolerance εf=0.05 _f=0.05, and progress tolerance εp=1.5 _p=1.5mm. The policy’s value head is used only at the final stage. We discount the feasibility cost so that unknown-risk in the far tail does not make all candidates indistinguishable. Goal Conditioning. NavDP’s point-goal representation covers [0,10]×[−10,10][0,10]×[-10,10]mm in forward and lateral coordinates, whereas our delivery goals can be up to 66 m66\,m away. Rather than independently clipping the two coordinates, which can substantially distort the goal bearing, we rescale goals farther than 10 m10\,m down to 10 m10\,m while preserving their bearing; nearer goals pass through unchanged. The original, unclipped goal is retained separately for the cascade’s progress stage. We apply this re-encoding to all experimental arms, including the unsteered baseline, so it is not part of the steering method. After the correction, every goal ahead of the robot is encoded with its true bearing. One case remains unrepresentable: a goal behind the robot has a negative forward coordinate, which the policy’s encoder still clips into its forward range. This residual defect belongs to the frozen policy’s interface, not to the steering, and it produces a specific failure mode that we quantify in Section IV-D2. TABLE I: Navigation benchmark. 60 episodes, three repeats per method (mean ± sd). Method Success ↑ Lawn (m) ↓ Zero-lawn ↑ Trav. failures ↓ Curb ep. ↓ Driven (m) Unsteered 44.3±5.044.3± 5.0 9.24±0.619.24± 0.61 17.3±0.617.3± 0.6 12.0±4.612.0± 4.6 3.3±0.63.3± 0.6 27.2±0.827.2± 0.8 Selection Only 48.0±2.648.0± 2.6 7.01±0.427.01± 0.42 23.3±1.523.3± 1.5 9.0±3.09.0± 3.0 0.3±0.60.3± 0.6 27.2±1.427.2± 1.4 Weighted Sum (wp,η)=(100,0.10)(w_p,η)=(100,0.10) 51.0±1.051.0± 1.0 6.58±0.346.58± 0.34 22.7±1.522.7± 1.5 6.0±0.06.0± 0.0 0.0±0.00.0± 0.0 27.5±1.027.5± 1.0 (wp,η)=(50,0.10)(w_p,η)=(50,0.10) 49.7±1.549.7± 1.5 6.84±0.406.84± 0.40 22.3±1.222.3± 1.2 8.3±2.38.3± 2.3 0.0±0.00.0± 0.0 27.4±1.027.4± 1.0 (wp,η)=(25,0.20)(w_p,η)=(25,0.20) 50.7±3.150.7± 3.1 6.34±0.146.34± 0.14 23.0±1.723.0± 1.7 5.7±2.55.7± 2.5 0.3±0.60.3± 0.6 27.7±0.927.7± 0.9 Ours 56.0±1.056.0± 1.0 2.60±0.242.60± 0.24 49.3±2.149.3± 2.1 2.0±1.02.0± 1.0 0.0±0.00.0± 0.0 36.9±2.936.9± 2.9 IV-C Baselines We compare against three baselines, and run every reported configuration for three repeats over the same 60 episodes. Unsteered: the frozen NavDP policy, unmodified except for the goal re-encoding described above. Weighted Sum: steers with ∇(wtg+wpf)∇(w_t\,g+w_p\,f). Only the ratio of the two weights changes the direction of the update, so we fix wt=1w_t=1 and search wpw_p and the step size η over a 12-point grid, on the seven episodes our proposed method was tuned on. We keep only the settings that lose no episode against the unsteered policy, and rank what remains by mean lawn. We then choose the top three settings to run on the benchmark: (wp,η)∈(100,0.10),(50,0.10),(25,0.20)(w_p,η)∈\(100,0.10),\,(50,0.10),\,(25,0.20)\. Selection Only: our lexicographic cascade applied to NavDP’s original candidates, with gradient steering disabled. The candidates are left unchanged, so this baseline isolates the effect of ordered candidate selection from the trajectory modification introduced by our full method. IV-D Results IV-D1 Baseline comparisons Table I compares our method with the unsteered policy and weighted-sum guidance. Steering substantially improves preference compliance: mean on-lawn distance decreases from 9.249.24 to 2.602.60mm per episode, while zero-lawn episodes increase from 17.317.3 to 49.349.3 out of 6060. Figure 3 shows representative episodes. Steering also improves feasibility. Across three repeats, the frozen policy fails 4747 episodes (15.715.7 per repeat), of which 3636 (12.012.0 per repeat) stall at steps of at least 0.10 m0.10\,m under the robot footprint, compared with only 2.02.0 per repeat after steering (see Section IV-D2 for failure analysis). Weighted-sum guidance closes much of the success gap, achieving 49.749.7–51.051.0 successful episodes compared with 56.056.0 for our method. However, the lexicographic ordering gives substantially better compliance and feasibility: mean on-lawn distance is 2.602.60mm versus 6.346.34–6.846.84mm, zero-lawn episodes are 49.349.3 versus 22.322.3–23.023.0, and traversability failures are 2.02.0 versus 5.75.7–8.38.3 per repeat. The main cost of enforcing this ordering is path length, with our method driving approximately 99mm further per episode. Selection alone provides a smaller but consistent improvement. Applying the lexicographic cascade to NavDP’s original candidate set, without gradient steering, raises success from 44.344.3 to 48.048.0 episodes, reduces mean on-lawn distance from 9.249.24 to 7.017.01mm, and lowers traversability failures from 12.012.0 to 9.09.0 per repeat. However, driven distance remains unchanged at 27.227.2mm, indicating that candidate selection can only choose among behaviors already proposed by the frozen policy. The larger gains of the full method therefore come from the gradient update creating new, more compliant trajectories rather than selection alone. IV-D2 Failure analysis Twelve of our 180180 episodes fail. Three are simulator artifacts: the vehicle becomes stuck at its spawn because of interactions between the simulator physics and scene geometry. Six failures are wanders that occur when the goal lies behind the robot. This is the one case the goal re-encoding of Section IV-B cannot repair: the encoder clips the negative forward coordinate into its forward range, so the conditioning no longer points toward the true goal and the robot loses a consistent direction. The remaining three failures expose a limitation of local steering without global guidance. In these episodes, steering moves the robot into a locally low-cost region, but that region later becomes a dead end: reaching the goal requires returning to the pavement, which the robot cannot climb onto from its current position. The runtime costs therefore favor locally compliant motion without representing the downstream reachability of the resulting state. Avoiding these traps requires longer-horizon or global guidance in addition to the local inference-time steering used here. V Manipulation with a Flow-Matching Policy Our second set of experiments asks two questions. First, does the same inference-time steering mechanism transfer to a different generative process and action space? Second, when does the priority ordering actually matter? The navigation benchmark rewards the ordering because the size of the two costs changes a lot between episodes. A long delivery route can accumulate far more lawn cost than a short one, so a weight that keeps the order on one episode may not keep it on another. By contrast, the pretrained manipulation policies and benchmarks we evaluate operate over short-horizon tasks, where motion distances and cost magnitudes remain within a more limited range. In this regime, a single well-tuned weight can more easily preserve the desired priority across episodes, so we expect weighted-sum steering to match our method at its best setting. We test this expectation directly. We first show that the proposed method transfers to a flow-matching manipulation policy on LIBERO, where it performs on par with a tuned weighted sum. We then run a controlled study on a fixed obstacle field, where we can sweep both methods over their own parameters and measure how much tuning each one needs. The result is that the two methods reach the same best performance, but the weighted sum only reaches it inside a narrow band of weights, while our method reaches it over a wide range of settings. V-A Transfer to a Flow-Matching Policy Policy and Environment. We evaluate the LIBERO-finetuned π0 _0 policy [36]. The policy generates a T=50T=50-step action chunk of 6-DoF end-effector pose deltas and gripper commands by integrating a learned flow field for K=10K=10 Euler steps; the controller executes the first 1010 actions before replanning. We sample S=16S=16 candidates at every replan. Trajectory Representation. Our costs are defined on the carried object’s path rather than directly on the action chunk. A naive integration of the commanded pose deltas substantially overestimates the executed motion because the operational-space controller realizes only a fraction of each command (315315mm predicted versus 9898mm executed at the chunk end, median over 102102 recorded chunks). We therefore fit a per-axis execution gain ρ=(0.208, 0.253, 0.209)ρ=(0.208,\,0.253,\,0.209), which reduces the median terminal prediction error to 1919mm. We incorporate this gain into the sample-to-workspace map Φ and discount costs along the predicted horizon with γh=0.8 _h=0.8. We additionally inflate the cost margins by the 9090th-percentile surrogate residual. This calibration is important: without it, steering optimizes a predicted path substantially longer than the one executed by the robot. Costs. Both runtime costs are defined on the carried object. The preference cost f penalizes carrying the object close to the avoid-set obtained from the task specification. The cost is smoothly attenuated near the placement target so that it does not oppose the intended placement. The feasibility cost g penalizes penetration of the table plane and other solid objects, excluding the placement target. Steering and Selection. We apply the barrier update after each Euler step in position space, using a 0.004 m0.004\,m trust region, η=0.05η=0.05, αb=10 _b=10, and βb=1 _b=1. The trust region is necessary to keep the edited samples within the controller’s executable range. Because π0 _0 has no value head, candidate selection consists only of the feasibility and preference stages of the lexicographic cascade. Baselines and Metrics. We evaluate on LIBERO using 8080 paired initial states across eight tasks, repeating each steered configuration three times under different sampling noise (320320 episodes per repeat over the four methods); Table I reports mean± over repeats. We compare against Unsteered (the original π0 _0 policy), Selection Only (the same 1616 candidates ranked by our cascade with gradient steering disabled), and Weighted Sum (steering with ∇f+wg∇g∇ f+w_g∇ g in place of the barrier’s state-dependent multiplier, under the same trust region and gain, with wg=100w_g=100 chosen by a sweep on held-out calibration tasks and its candidates ranked by the same cascade). The eight evaluation tasks are chosen before any steered run using only frozen-policy rollouts. We keep tasks in which the preference is active during carrying and in which the two cost gradients interact over a nontrivial part of the trajectory. We also require the preference cost to vanish at both the start and the placement target, so that unavoidable initial or terminal contact is not counted as a violation. All steering parameters are calibrated on two held-out tasks excluded from evaluation. We report success; preference exposure E, defined as the carried-object path length inside the keep-out region after grasp; peak feasibility violation gmaxg_ ; and regrasp rate. TABLE I: LIBERO benchmark. Method Success ↑ E (m) ↓ gmaxg_ ↓ Regrasp ↓ Unsteered 0.72±0.020.72± 0.02 0.408±0.0050.408± 0.005 0.213±0.1920.213± 0.192 0.08±0.030.08± 0.03 S. Only 0.79±0.010.79± 0.01 0.401±0.0100.401± 0.010 0.119±0.0970.119± 0.097 0.07±0.020.07± 0.02 W. Sum 0.78±0.030.78± 0.03 0.348±0.0090.348± 0.009 0.068±0.0350.068± 0.035 0.11±0.040.11± 0.04 Ours 0.81±0.020.81± 0.02 0.355±0.0070.355± 0.007 0.066±0.0290.066± 0.029 0.07±0.010.07± 0.01 E is meters of preference violation; gmaxg_ is the max feasibility violation. Regrasp counts grasps after the first, i.e. how often the grip was lost mid-task. Results. Relative to the frozen policy, our method improves preference and feasibility at the same time (Table I). Preference exposure decreases from 0.4080.408 to 0.3550.355mm, and peak feasibility violation from 0.2130.213 to 0.0660.066. Success increases from 0.720.72 to 0.810.81, so the compliance gains do not come at the cost of task completion. Selection Only helps feasibility and success but leaves preference exposure unchanged (0.4080.408 versus 0.4010.401mm). The preference improvement therefore comes from modifying the sampled trajectories, not from choosing among more candidates. This matches navigation: selection contributes, but gradient steering is what produces behavior outside the frozen policy’s original candidate set. Weighted Sum performs on par with our method here. The two are indistinguishable in success, preference exposure, and peak feasibility violation, and we do not claim superiority over a tuned weighted sum in this domain. This is the outcome we expect when the cost scales do not vary much across episodes. What the transfer result establishes is that the same method runs unchanged on a flow-matching policy and reaches the performance of a separately tuned fixed weight without a task-specific weight sweep. Fig. 4: Executed paths on D3IL avoiding as each method’s parameter is varied over a wide range of values, at the same step size. Top-down view; the robot travels left to right from the black dot to the goal line, and the stars mark the requested gap in each obstacle row. Our panels look nearly the same from αb=100 _b=10^0 onward, while the weighted sum only produces the requested behavior near w=1w=1: at w=0w=0 it does not enforce feasibility, and at large w the feasibility term dominates, the paths detour around the obstacles, and compliance falls back to roughly the level it had with no feasibility term at all. Paths are from one of the four requested routes. V-B How Much Tuning Does Each Method Need? The LIBERO comparison shows parity at one operating point, but it does not show how hard that point was to find. We therefore run a controlled study on a task where we can sweep both methods over their own parameters and repeat every setting. Task and Policy. We use the avoiding task from D3IL [6]. A robot arm must move a rod from a start position to a goal line while passing through a field of six fixed cylindrical obstacles arranged in three rows (see Figure 4). Contact with any obstacle ends the episode as a failure. The gaps between obstacles form 2424 distinct routes, and the demonstrations use all of them. Our policy is an encoder–decoder diffusion policy trained on the 9696 human demonstrations, producing T=4T=4-step chunks of end-effector positions over K=16K=16 denoising steps. We train the policy with three seeds and pick the one with the highest success rate (92.7%92.7\%). This task has the property we want to study. All six obstacles have almost the same radius and the gaps between them are the same width, so the feasibility cost has one characteristic scale across the whole workspace. Costs. The feasibility cost g is the simulator’s own collision test made differentiable: a log barrier on the distance between the rod surface and each obstacle surface, active inside a 0.02 m0.02\,m band. The preference cost f asks the policy to take a nominated route, penalizing lateral distance from the chosen gap in each of the three rows. Because both costs are anchored to the same obstacles, moving sideways to satisfy the preference moves the rod toward an obstacle, so the two gradients oppose each other directly. Metrics. We report success (reaching the goal line without contact) and gate agreement, the fraction of the three rows where the robot passes through the requested gap. Gate agreement measures how well the preference is satisfied. Both methods use the same step size and differ only in how they weight the feasibility gradient: ours through the barrier constant αb _b, the baseline through a fixed weight. We sweep each from 10−310^-3 to 10310^3. The two parameters are not the same quantity, so setting them to equal values does not make the comparison fair. We instead compare the methods at equal preference compliance, asking how much success each retains at a given level of gate agreement (Fig. 5). We observe that the two methods reach the same best performance. This is the expected outcome for a task with one characteristic scale, and it is consistent with the LIBERO result. Fig. 4 shows the corresponding paths: at their best settings the two methods produce the same behavior. TABLE I: Success (%) as each method’s own parameter is varied. Parameter value 00 10−310^-3 10−210^-2 10−110^-1 11 1010 10210^2 10310^3 Ours (αb _b) 75.075.0 75.075.0 77.577.5 100100 100100 100100 100100 100100 Weighted Sum (w) 35.035.0 35.035.0 45.045.0 85.085.0 100100 90.090.0 90.090.0 90.090.0 Note that αb=0 _b=0 does not remove the feasibility term: it drops the required decrease but still cancels any update that would increase g. Fig. 5: Sensitivity of each method to its own parameter, averaged over four requested routes (4040 episodes per point). Our method reaches the best value of both metrics by αb=10−1 _b=10^-1 and holds it for every larger value we tried. The weighted sum reaches the same best values only near w=1w=1. Success alone is misleading at large w: it recovers to the frozen policy’s level because the method stops following the preference, which the gate-agreement panel shows. However, they differ in how much tuning it takes. Table I and Fig. 5 sweep each method’s own parameter over the same span, with the same barrier and the same step size. Our method reaches full success for any αb≥0.1 _b≥ 0.1 and keeps it all the way to αb=103 _b=10^3; it degrades only when αb _b is small enough that the barrier stops asking for a decrease, and even then it stays well above the weighted sum’s worst settings. The weighted sum reaches full success only at w=1w=1 and loses performance on both sides: too small and it does not enforce feasibility, too large and it holds the rod away from the gaps and hurts success. The reason is visible in the update rule. The barrier multiplier λ is chosen so that the feasibility cost decreases at a target rate, so raising αb _b past what is needed changes nothing: the multiplier cannot overshoot. A fixed weight instead adds a fixed force wherever the feasibility gradient is non-zero, so a weight that is too large is as harmful as one that is too small. TABLE IV: Success (%\%) as the steering step size increases. Setting η=8η=8 η=16η=16 η=32η=32 Ours, αb=0.25 _b=0.25 100.0±0.0100.0± 0.0 100.0±0.0100.0± 0.0 75.0±0.075.0± 0.0 W. Sum, w=1w=1 100.0±0.0100.0± 0.0 93.3±4.493.3± 4.4 55.6±2.555.6± 2.5 W. Sum, w=0.1w=0.1 95.6±1.095.6± 1.0 93.3±1.793.3± 1.7 73.3±1.773.3± 1.7 W. Sum, w=10w=10 89.4±1.989.4± 1.9 77.2±8.277.2± 8.2 52.8±3.552.8± 3.5 Note that the best fixed weight changes with the step size; the barrier constant does not. The best weight also moves. Table IV increases the steering step size, which increases the pressure the preference puts on feasibility, and repeats every setting three times with independent seeds. The best fixed weight is not the same at every step size: w=1w=1 is best at η=8η=8 but drops to 55.6%55.6\% at η=32η=32, while w=0.1w=0.1 survives the largest step size but gives up performance at the smallest. Our method uses the same αb _b throughout and is at or above the best fixed weight at every step size. Its results are also more repeatable: the standard deviation across repeats is 0.00.0 at every setting, whereas the fixed weight reaches 8.28.2 points. We do not claim a significant advantage in mean success here. The difference we do measure reliably is in sensitivity and repeatability, not in the best achievable score. V-C Summary Taken together, the two manipulation experiments show that, when the cost scales are similar across a task, a fixed weight can hold the priority order as expected. The benefit of imposing the order explicitly in this regime is that it removes the need to find that weight: any barrier constant of 0.10.1 or above works, and it does not need to be re-tuned when the steering strength changes, whereas the fixed weight has a narrow working range that moves with the operating point. The navigation benchmark shows the other regime, where the cost scales vary between episodes and no single weight serves them all. VI Toy Example with Deeper Hierarchy Fig. 6: Three-level toy example. Wall avoidance (g1g_1) has highest priority, lawn avoidance (g2g_2) is next, and path length (f) is last. Gray discs are walls and green is lawn. Red curves are the S=16S=16 samples from the frozen policy; blue is our selected path. Dashed purple curves show weighted-sum results for w2=1w_2=1, with color indicating the wall weight w1w_1. Weights 0.30.3, 11, and 33 cross a wall, whereas 1010 and 3030 take the lower route. Fig. 7: Mean QP multipliers for the wall and lawn, averaged over 1616 candidates and all updates at each Euler step. Their values change as the paths change, whereas weighted-sum weights stay fixed. To show how the method handles more than two ordered costs, we train and freeze a flow-matching policy in a simple two-dimensional task as shown in Figure 6. At each Euler step, the proposed method steers each sample toward a shorter path while keeping the wall and lawn costs from increasing to first order. All guided methods use the same cascade to select the final path. Over 200200 episodes, 66.5%66.5\% of unsteered paths enter a wall by more than 11cmcm. The proposed method has no such collisions, reaches the goal in 98.5%98.5\% of episodes, and travels 2.242.24mm on lawn on average. The average path length is 10.2210.22mm, compared to the 10.00510.005mm ground-truth shortest wall-safe path. Only five of the 2020 weighted-sum settings avoid all wall collisions. The best safe setting, (w1,w2)=(30,3)(w_1,w_2)=(30,3), has similar lawn travel to our method (2.192.19mm versus 2.242.24mm), but (30,10)(30,10) enters a wall by more than 11cmcm in 189/200189/200 episodes. Thus, a weighted sum can match our method when tuned well, but the hierarchy avoids a search for the right wall-to-lawn weight ratio. VII Conclusions We presented an inference-time method for steering frozen generative robot policies under lexicographically ordered runtime costs. A dynamic-barrier gradient update steers each sampled trajectory toward lower-priority preferences while protecting higher-priority feasibility to first order, and a lexicographic selection rule filters the resulting candidates one priority level at a time. Both mechanisms act on sampled trajectories, so the same implementation applies to diffusion and flow-matching policies without retraining. On a last-mile delivery benchmark with conflicting traversability and surface-preference costs, the method improves success, preference compliance, and traversability over the frozen policy and achieves better compliance than weighted-sum baselines tuned with the same preference information. The same method transfers to a flow-matching manipulation policy on LIBERO, and a controlled D3IL study shows that the dynamic barrier holds its performance over a far wider range of settings than a fixed weight. The main limitations are locality: steering reshapes trajectories near the policy’s own proposals, so it cannot recover a goal the frozen policy cannot represent or supply global guidance the policy lacks. Future work includes steering policies with latent-space samplers where the sample-to-workspace map is learned rather than fixed. References [1] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), p. 1684–1704. Cited by: §I. [2] D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine (2023) ViNT: a foundation model for visual navigation. arXiv preprint arXiv:2306.14846. Cited by: §I. [3] A. Sridhar, D. Shah, C. Glossop, and S. Levine (2024) NoMaD: goal masked diffusion policies for navigation and exploration. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 63–70. Cited by: §I. [4] W. Cai, J. Peng, Y. Yang, Y. Zhang, M. Wei, H. Wang, Y. Chen, T. Wang, and J. Pang (2025) NavDP: learning sim-to-real navigation diffusion policy with privileged information guidance. arXiv preprint arXiv:2505.08712. Cited by: §I, §IV-B. [5] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) Libero: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems 36, p. 44776–44791. Cited by: item 3. [6] X. Jia, D. Blessing, X. Jiang, M. Reuss, A. Donat, R. Lioutikov, and G. Neumann (2024) Towards diverse behaviors: a benchmark for imitation learning with human demonstrations. In International Conference on Learning Representations, Vol. 2024, p. 41666–41686. Cited by: item 3, §V-B. [7] P. Dhariwal and A. Nichol (2021) Diffusion models beat GANs on image synthesis. Advances in neural information processing systems 34, p. 8780–8794. Cited by: §I-A. [8] M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine (2022) Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991. Cited by: §I-A. [9] Y. Wang, L. Wang, Y. Du, B. Sundaralingam, X. Yang, Y. Chao, C. Pérez-D’Arpino, D. Fox, and J. Shah (2025) Inference-time policy steering through human interactions. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 15626–15633. Cited by: §I-A. [10] Y. Zeng, H. Ren, S. Wang, J. Huang, and H. Cheng (2025) Navidiffusor: cost-guided diffusion model for visual navigation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 11994–12001. Cited by: §I-A. [11] R. Singhal, Z. Horvitz, R. Teehan, M. Ren, Z. Yu, K. McKeown, and R. Ranganath (2025) A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848. Cited by: §I-A. [12] L. Wu, B. Trippe, C. Naesseth, D. Blei, and J. P. Cunningham (2023) Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems 36, p. 31372–31403. Cited by: §I-A. [13] S. Liu, I. S. Singh, Y. Xu, J. Duan, and R. Krishna (2026) VLS: steering pretrained robot policies via vision-language models. arXiv preprint arXiv:2602.03973. Cited by: §I-A. [14] G. Olguín and J. Ruiz-del-Solar (2026) Sem-NaVAE: semantically-guided outdoor mapless navigation via generative trajectory priors. IEEE Robotics and Automation Letters. Cited by: §I-A. [15] W. Xiao, T. Wang, C. Gan, and D. Rus (2023) SafeDiffuser: safe planning with diffusion probabilistic models. arXiv preprint arXiv:2306.00148. Cited by: §I-A. [16] J. Zhang, L. Zhao, A. Papachristodoulou, and J. Umenberger (2026) Constrained diffusers for safe planning and control. Advances in Neural Information Processing Systems 38, p. 34965–34998. Cited by: §I-A. [17] J. Yin, C. Dawson, C. Fan, and P. Tsiotras (2023) Shield model predictive path integral: a computationally efficient robust mpc method using control barrier functions. IEEE Robotics and Automation Letters 8 (11), p. 7106–7113. Cited by: §I-A. [18] Z. Li, K. Alim, and N. Azizan (2026) HardFlow: hard-constrained sampling for flow-matching models via trajectory optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §I-A. [19] D. Zhan, X. Xu, J. Li, and J. Tang (2026) SEAM: smooth execution of action-chunked motion for vision-language-action policies. arXiv preprint arXiv:2607.04609. Cited by: §I-A. [20] D. Fox, W. Burgard, and S. Thrun (1997) The dynamic window approach to collision avoidance. IEEE robotics & automation magazine 4 (1), p. 23–33. Cited by: §I-B. [21] S. B. Slotine and B. Siciliano (1991) A general framework for managing multiple tasks in highly redundant robotic systems. In proceeding of 5th International Conference on Advanced Robotics, Vol. 2, p. 1211–1216. Cited by: §I-B. [22] A. Escande, N. Mansard, and P. Wieber (2014) Hierarchical quadratic programming: fast online humanoid-robot motion generation. The International Journal of Robotics Research 33 (7), p. 1006–1028. Cited by: §I-B. [23] A. Censi, K. Slutsky, T. Wongpiromsarn, D. Yershov, S. Pendleton, J. Fu, and E. Frazzoli (2019) Liability, ethics, and culture-aware behavior specification using rulebooks. In 2019 international conference on robotics and automation (ICRA), p. 8536–8542. Cited by: §I-B. [24] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada (2016) Control barrier function based quadratic programs for safety critical systems. IEEE transactions on automatic control 62 (8), p. 3861–3876. Cited by: §I-B. [25] Z. Gábor, Z. Kalmár, and C. Szepesvári (1998) Multi-criteria reinforcement learning.. In ICML, Vol. 98, p. 197–205. Cited by: §I-B. [26] K. Wray, S. Zilberstein, and A. Mouaddib (2015) Multi-objective MDPs with conditional lexicographic reward preferences. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29. Cited by: §I-B. [27] J. Skalse, L. Hammond, C. Griffin, and A. Abate (2022) Lexicographic multi-objective reinforcement learning. arXiv preprint arXiv:2212.13769. Cited by: §I-B. [28] S. Zhang, F. Jia, C. Wang, and Q. Wu (2023) Targeted hyperparameter optimization with lexicographic preferences over multiple objectives. In The Eleventh international conference on learning representations, Cited by: §I-B. [29] C. Gong, X. Liu, and Q. Liu (2021) Automatic and harmless regularization with constrained and lexicographic optimization: a dynamic barrier approach. Advances in Neural Information Processing Systems 34, p. 29630–29642. Cited by: §I-B, §I-B. [30] M. Ye and Q. Liu (2022) Pareto navigation gradient descent: a first-order algorithm for optimization in pareto set. In Uncertainty in artificial intelligence, p. 2246–2255. Cited by: §I-B, §I-B. [31] NVIDIA (2025) Isaac sim. Note: https://developer.nvidia.com/isaac/simVersion 4.x; scenes built on Isaac Lab Cited by: §IV-A. [32] P. Roth, J. Nubert, F. Yang, M. Mittal, and M. Hutter (2024) Viplanner: visual semantic imperative learning for local navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 5243–5249. Cited by: §IV-A. [33] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: an open urban driving simulator. In Conference on robot learning, p. 1–16. Cited by: §IV-A. [34] D. D. Fan, K. Otsu, Y. Kubo, A. Dixit, J. Burdick, and A. Agha-mohammadi (2021) STEP: stochastic traversability evaluation and planning for risk-aware off-road navigation. In Robotics: Science and Systems (RSS), Note: arXiv:2103.02828 Cited by: §IV-B. [35] R. T. Rockafellar S. Uryasev et al. (2000) Optimization of conditional value-at-risk. Journal of risk 2, p. 21–42. Cited by: §IV-B. [36] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) π0 _0: a vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164. Cited by: §V-A.