Paper deep dive
Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigation
Xuekang Yang, Lu Chen, Shuang Luo, Jialing Zhu, Qi Zhang, Yue Gao, Xiang Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-and-language navigation increasingly relies on general-purpose semantic planners, yet translating correct high-level intent into reliable physical execution remains difficult in spatially constrained transitions. Reaching a staircase, doorway, or narrow passage does not ensure traversal; the agent must identify an executable affordance pose and recover from accumulated action errors. We propose PACE (Preference-refined Affordance-Conditioned Execution), a supervised local execution module that augments frozen zero-shot semantic planners for reliable cross-floor navigation. PACE grounds transition-related semantics into a long-horizon, agent-centric traversable affordance pose and conditions short-horizon action generation on this spatial target, thereby aligning semantic goals with physical execution. We further post-train PACE through failure-aware preference refinement using rollout-derived pairs that contrast normal or recovery behaviors with deviation-amplifying behaviors, thereby improving closed-loop correction. We integrate PACE into six open-source zero-shot VLN navigators and demonstrate consistent improvements on the cross-floor subsets of R2R-CE and RxR-CE, increasing the average success rate from 16.35% to 27.65% and from 4.76% to 12.06%, respectively. Real-world experiments further demonstrate PACE's applicability in unseen environments, highlighting the potential of traversable affordances to bridge semantic intent and reliable embodied behavior.
Tags
Links
- Source: https://arxiv.org/abs/2609.32292v1
- Canonical: https://arxiv.org/abs/2609.32292v1
Trouble viewing inline? Open PDF directly →
Full Text
75,496 characters extracted from source content.
Expand or collapse full text
Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigation Xuekang Yang1, Lu Chen1, Shuang Luo1, Jialing Zhu2 Qi Zhang3,*, Yue Gao4,*, Xiang Zhang3,* Affiliation: 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Department of Automation, Shanghai Jiao Tong University 3Defense Innovation Institute, Academy of Military Sciences 4MoE Key Laboratory of Artificial Intelligence and AI Institute, Shanghai Jiao Tong University *Corresponding authors Affiliation: Abstract Vision-and-language navigation increasingly relies on general-purpose semantic planners, yet translating correct high-level intent into reliable physical execution remains difficult in spatially constrained transitions. Reaching a staircase, doorway, or narrow passage does not ensure traversal; the agent must identify an executable affordance pose and recover from accumulated action errors. We propose PACE (Preference-refined Affordance-Conditioned Execution), a supervised local execution module that augments frozen zero-shot semantic planners for reliable cross-floor navigation. PACE grounds transition-related semantics into a long-horizon, agent-centric traversable affordance pose and conditions short-horizon action generation on this spatial target, thereby aligning semantic goals with physical execution. We further post-train PACE through failure-aware preference refinement using rollout-derived pairs that contrast normal or recovery behaviors with deviation-amplifying behaviors, thereby improving closed-loop correction. We integrate PACE into six open-source zero-shot VLN navigators and demonstrate consistent improvements on the cross-floor subsets of R2R-CE and RxR-CE, increasing the average success rate from 16.35% to 27.65% and from 4.76% to 12.06%, respectively. Real-world experiments further demonstrate PACE’s applicability in unseen environments, highlighting the potential of traversable affordances to bridge semantic intent and reliable embodied behavior. 1 Introduction Vision-and-language navigation in continuous environments requires embodied agents to translate natural-language instructions into low-level motion (Krantz et al., 2020). Recent zero-shot and learning-based methods, increasingly driven by general-purpose semantic planners, have made substantial progress (Zheng et al., 2026c; Chen et al., 2025b; He et al., 2026; Wei et al., 2025; Chu et al., 2026; Zhang et al., 2026). Nevertheless, multi-floor environments expose a persistent challenge: correct semantic intent does not ensure reliable execution at constrained transitions. Figure 1: Overview of PACE. PACE bridges the semantic-spatial gap through traversable affordance estimation, affordance-conditioned action generation, and rollout-derived preference refinement. Existing cross-floor navigation methods primarily rely on explicit geometric representations, including hierarchical scene graphs (Werby et al., 2024), floor-aware maps (Zhang et al., 2025b; Gong et al., 2026; Huang et al., 2026), and reconstructed traversable surfaces (Li et al., 2026; Zheng et al., 2026a), to model inter-floor connectivity and support geometric planning. However, stair traversal remains a major source of failure: nearly 21% of navigation failures have been associated with unsuccessful stair traversal (Ramrakhya et al., 2023), while MANSION reports substantial degradation on multi-floor long-horizon tasks (Che et al., 2026). These findings indicate that geometry can describe multi-floor structure and localize transition regions, but reaching a staircase does not guarantee successful traversal. Recent work has characterized a related challenge as the semantic-geometric gap, namely grounding semantic concepts in environment-centric 3D structures (Li et al., 2026). Despite strong semantic perception, VLMs remain limited in reasoning over 3D structures and action-induced spatial transitions (Yang et al., 2025; Chen et al., 2025c; Windecker et al., 2025; Kong et al., 2025). The semantic-geometric gap specifies where a relevant structure is, whereas reliable navigation additionally requires an agent-centric executable affordance pose specifying where and how the agent should approach and traverse it. We term this remaining disconnect the semantic-spatial gap. These results show that geometric localization of a staircase does not ensure its successful traversal. Existing methods address this gap through two main paradigms. Hierarchical methods insert waypoint-based spatial subgoals between semantic reasoning and motion execution (An et al., 2024; Qiao et al., 2025; Shi et al., 2025). However, these waypoints encode where the agent can move rather than the instruction-conditioned pose from which a transition should be executed, while external planners provide limited semantic conditioning during execution (Wang et al., 2026a; Li et al., 2026). End-to-end imitation-learning policies instead map visual-language observations directly to actions (Wei et al., 2025; Chu et al., 2026; Zhang et al., 2024; Krantz et al., 2020), but expert-state-dominated supervision leaves them vulnerable to compounding errors and limits recovery from policy-induced deviations (Ross et al., 2011; Shao et al., 2024). Thus, neither paradigm jointly provides task-conditioned executable spatial grounding and robust closed-loop execution beyond the expert-state distribution. To address these limitations, we propose PACE (Preference-refined Affordance-Conditioned Execution) in Fig. 1, a plug-and-play supervised local execution framework trained on the corresponding benchmark training split to augment frozen zero-shot semantic planners for cross-floor VLN; both components remain frozen in unseen evaluation environments. Our contributions are: • We introduce PACE, a dual-horizon affordance-conditioned framework that grounds stair-transition semantics into long-horizon traversable affordance poses for conditioning short-horizon action generation. • We propose failure-aware preference refinement using rollout-derived pairs, favoring normal and recovery behaviors over deviation-amplifying alternatives to improve closed-loop correction beyond expert demonstrations. • Across six open-source zero-shot VLN navigators, PACE improves average SR by 11.30 and 7.30 percentage points on R2R-CE-CF and RxR-CE-CF, respectively, and is successfully deployed in unseen real-world environments. 2 Related Work 2.1 Semantic-Spatial Grounding Despite strong semantic understanding, multimodal large language models (MLLMs) remain unreliable in spatial reasoning (Yang et al., 2025; Chen et al., 2025c; Windecker et al., 2025; Kong et al., 2025). Embodied navigation methods therefore combine vision-language and segmentation models with depth and SLAM to associate semantics with 3D structures represented by BEV or voxel maps, 3D Gaussians, and layered memories (Yu et al., 2026; Jang and Kim, 2026; Zhou et al., 2026; Yokoyama et al., 2024; Long et al., 2025; Zhang et al., 2025a; Li et al., 2026; Wang et al., 2026b; Zheng et al., 2026b). However, these environment-centric representations primarily localize semantic entities rather than model agent-centric executability. Affordance-aware methods capture functional agent-environment interactions but often depend on predefined skills or external intermediate representations (ichter et al., 2023; Yuan et al., 2024; Chen et al., 2025a). In contrast, PACE directly predicts agent-centric traversable affordance poses from RGB-D observations and semantic guidance without explicit 3D scene construction. 2.2 Semantic-Aware Navigation Semantic-aware VLN methods mainly follow end-to-end and hierarchical paradigms. End-to-end methods map visual-language observations to decisions through task-specific policy learning (Wei et al., 2025; Chu et al., 2026; Zhang et al., 2024; Krantz et al., 2020) or zero-shot LLM/VLM reasoning (Zhou et al., 2024; Bhatt et al., 2025; Long et al., 2024), whereas hierarchical methods insert waypoints or topological nodes between semantic reasoning and motion execution (An et al., 2024; Qiao et al., 2025; Shi et al., 2025). However, end-to-end policies learn semantic-to-action mappings implicitly, zero-shot methods rely on unreliable long-horizon spatial reasoning, and hierarchical subgoals typically encode geometric accessibility rather than instruction-conditioned affordances (Li et al., 2026; Wang et al., 2026a). PACE instead conditions local action generation on an inferred affordance pose, explicitly grounding task semantics into executable spatial decisions. 3 Methods Problem Formulation Given an instruction I and an unseen continuous environment ℰE, the agent receives an egocentric RGB-D observation ot=(rt,dt)o_t=(r_t,d_t) at step t and selects an action from =FORWARD,LEFT,RIGHT,A=\ FORWARD, LEFT, RIGHT\, (1) where rtr_t and dtd_t denote the RGB and depth observations, respectively. FORWARD advances the agent by 0.25m0.25\,m, while LEFT and RIGHT rotate it by 30∘30 . We focus on cross-floor episodes requiring stair traversal. The high-level VLM provides traversal guidance st∈Up,Down,Unknowns_t∈\ Up, Down, Unknown\. PACE operates on a K-step context ct=(ot−K+1:t,ξt−K+1:t,st)c_t=(o_t-K+1:t, _t-K+1:t,s_t), where ξi=(ai−1,p^i−1aff,ϕ^i−1) _i=(a_i-1, p^aff_i-1, φ_i-1) contains the preceding action, affordance pose, and traversal phase. PACE predicts htaff h_t^aff =faff(ct),z^t=(p^afft,ϕ^t), =f_aff(c_t), z_t=( p^aff_t, φ_t), (2) p^taff p^aff_t =(x^t,y^t,θ^t), =( x_t, y_t, θ_t), where (x^t,y^t,θ^t)( x_t, y_t, θ_t) denotes the agent-centric traversable affordance pose—the desired entry pose before traversal and exit pose once traversal begins—and ϕ^t∈Approach,Entry,Traverse,Exit φ_t∈ \ Approach, Entry, Traverse, Exit\ denotes the traversal phase. PACE then generates a short-horizon action proposal τ^t=fact(ct,htaff)=(a^t,1,…,a^t,Mt,STOP), τ_t=f_act(c_t,h_t^aff)=( a_t,1,…, a_t,M_t, STOP), (3) where a^t,j∈ a_t,j for j≤Mtj≤ M_t, a^t,Mt+1≡STOP a_t,M_t+1≡ STOP, and MtM_t counts the motion actions preceding the terminal token; hence |τ^t|=Mt+1| τ_t|=M_t+1. The STOP token terminates only the current proposal and triggers the next closed-loop decision rather than ending the navigation episode. Throughout this paper, zero-shot refers to the high-level navigator and test-time protocol: PACE is supervised on the corresponding training split, while both PACE and the navigator remain frozen without supervision or adaptation in the unseen evaluation environments. 3.1 Framework Overview As shown in Fig. 1, PACE combines dual-horizon affordance-conditioned decision making with failure-aware preference refinement. Imitation learning couples long-horizon affordance estimation with short-horizon action generation, while preference post-training on rollout-derived deviation and recovery trajectories improves closed-loop correction. At inference, the frozen module augments existing VLM planners for local stair traversal. 3.2 Dual-Horizon Affordance-Conditioned Execution Modeling. For compact notation, let zt=(ptaff,ϕt)z_t=(p_t^aff, _t) denote the long-horizon affordance state, where ptaff=(xt,yt,θt)p_t^aff=(x_t,y_t, _t) is an agent-centric traversable affordance pose and ϕt _t is the current traversal phase. PACE first extracts an affordance-aware latent representation: htaff=faff(ct,ω),h_t^aff=f_aff(c_t;ω), (4) where ω denotes the parameters of the multimodal context encoder and long-horizon affordance estimator. PACE parameterizes the dual-horizon policy as pΘ(zt,τt∣ct)=pω(zt∣htaff)pψ(τt∣ct,htaff),p_ (z_t, _t c_t)=p_ω(z_t h_t^aff)p_ψ( _t c_t,h_t^aff), (5) where ψ denotes the parameters of the action generator and τt=(at,1,…,at,Mt,STOP) _t=(a_t,1,…,a_t,M_t, STOP) denotes the short-horizon action sequence, with at,j∈a_t,j for j≤Mtj≤ M_t and at,Mt+1≡STOPa_t,M_t+1≡ STOP. The long-horizon affordance state is decoded through parallel pose and phase heads: p^taff=fpose(htaff,ω),pω(ϕ∣htaff)=fphase(htaff,ω),ϕ^t=argmaxϕpω(ϕ∣htaff). gathered p_t^aff=f_pose(h_t^aff;ω),\\ p_ω(φ h_t^aff)=f_phase(h_t^aff;ω),\\ φ_t= _φp_ω(φ h_t^aff). gathered (6) yielding z^t=(p^taff,ϕ^t) z_t=( p_t^aff, φ_t). Conditioned on htaffh_t^aff, the action generator autoregressively models pψ(τt∣ct,htaff)=∏j=1|τt|pψ(at,j∣at,<j,ct,htaff).p_ψ( _t c_t,h_t^aff)= _j=1^| _t|p_ψ (a_t,j a_t,<j,c_t,h_t^aff ). (7) Here at,<j=(at,1,…,at,j−1)a_t,<j=(a_t,1,…,a_t,j-1) denotes the preceding action prefix. Expert-path height trends delimit the four traversal phases: ptaff,∗p_t^aff,* targets the entry pose during Approach and the exit pose during Entry/Traverse, while Exit is terminal. The selected pose is projected into the current agent frame using its horizontal displacement and tangent heading. Given an expert dataset =(ct,ptaff∗,ϕt∗,τt∗)D=\(c_t,p_t^aff*, _t^*, _t^*)\ constructed from expert stair-traversal trajectories, where ptaff∗p_t^aff* is the phase-dependent entry or exit pose, ϕt∗ _t^* is the traversal-phase label, and τt∗ _t^* is the expert short-horizon action sequence, we define ℒdual(Θ,)=[λpℒpose(p^taff,ptaff∗)⏟(1) Affordance pose regression−λϕlogpω(ϕt∗∣htaff)⏟(2) Traversal phase classification−λa∑j=1|τt∗|logpψ(at,j∗∣at,<j∗,ct,htaff)⏟(3) Action sequence classification].L_dual( ;D)=E_D [ aligned & - _pL_pose ( p_t^aff,p_t^aff* )_(1) Affordance pose regression\\[-1.42262pt] &- _φ p_ω ( _t^* h_t^aff )_(2) Traversal phase classification\\[-1.42262pt] &- _a _j=1^| _t^*| p_ψ (a_t,j^* a_t,<j^*,c_t,h_t^aff )_(3) Action sequence classification aligned ]. (8) The supervised initialization is obtained by Θ(0)=argminΘℒdual(Θ,), ^(0)= _ L_dual( ;D), where Θ=ω,ψ =\ω,ψ\ denotes all trainable parameters, and λp _p, λϕ _φ, and λa _a balance the three supervision terms. Terms (1)–(3) correspond to traversable affordance pose regression, traversal phase classification, and autoregressive action prediction, respectively. Conditioning action generation on htaffh_t^aff couples long-horizon affordance grounding with short-horizon physical execution. Network Architecture. Figure 2: Model Architecture. As illustrated in Fig. 2, PACE comprises a multimodal temporal encoder, a query-based affordance estimator, and a Transformer-based causal action decoder. RGB observations rt−K+1:tr_t-K+1:t and depth observations dt−K+1:td_t-K+1:t are encoded by separate ImageNet-pretrained ResNet backbones and projected into a shared d-dimensional feature space, where a gated fusion module integrates the two visual streams. The historical context ξt−K+1:t _t-K+1:t is encoded by an MLP, while the semantic guidance sts_t is represented by a learnable embedding. The resulting visual, historical, and semantic features are concatenated into one multimodal token per context step. After adding temporal positional embeddings, a Transformer encoder models dependencies across the K-step context and produces Ht∈ℝK×dH_t ^K× d, where d denotes the hidden dimension. A learnable affordance query qaff∈ℝdq^aff ^d performs multi-head cross-attention over HtH_t to extract htaff=MHA(qaff,Ht,Ht)∈ℝd.h_t^aff=MHA (q^aff,H_t,H_t ) ^d. (9) Parallel phase and pose heads decode htaffh_t^aff into the traversal-phase prediction ϕ^t φ_t and the traversable affordance pose p^taff p_t^aff, respectively. For short-horizon action generation, the affordance representation is appended to the temporal features as an additional memory token: H~t=[Ht;htaff]∈ℝ(K+1)×d. H_t=[H_t;h_t^aff] ^(K+1)× d. (10) A causal Transformer decoder parameterized by ψ attends to H~t H_t and autoregressively predicts each action conditioned on the preceding action prefix at,<ja_t,<j. The decoder is trained with teacher forcing, while beam search generates candidate sequences at inference until STOP or the maximum proposal length is reached. The predicted affordance pose is further used for geometric reranking of the candidate sequences. 3.3 Failure-Aware Preference Refinement Imitation learning exposes PACE primarily to expert states, limiting its ability to recover from policy-induced deviations during closed-loop execution. We therefore refine the policy by favoring normal or recovery behaviors over deviation-amplifying alternatives. Preference Pair Construction. We collect model-induced states through autoregressive closed-loop rollouts of the current PACE policy. A deviation is identified when the positional or heading error from the expert route exceeds 0.5m0.5\,m or 30∘30 for two consecutive steps, whereas failure is triggered immediately when either error exceeds 1.5m1.5\,m or 60∘60 . From each failure point, A* plans a trajectory back to the expert route, which is converted into discrete expert actions and replayed under teacher forcing. Recovery ends when the positional and heading errors remain within 0.25m0.25\,m and 30∘30 for two consecutive steps, after which subsequent states are treated as normal behavior. We form fixed-length contiguous same-role snippets within each rollout and match each deviation-amplifying snippet ζ−ζ^- with shuffled normal and recovery snippets ζ+ζ^+ from the global pools. Preference Optimization. To distinguish a temporal preference snippet from the short-horizon proposal τt _t, we denote an ordered snippet as ζ=((ci,τi))i=1L,τi=(ai,1,…,ai,Mi,STOP),ζ= ((c_i, _i) )_i=1^L, _i=(a_i,1,…,a_i,M_i, STOP), (11) where L is the snippet length, MiM_i counts the motion actions preceding STOP, and |τi|=Mi+1| _i|=M_i+1 includes the terminal token. Given the affordance-aware representation hiaff=faff(ci,ω)h_i^aff=f_aff(c_i;ω), each proposal is scored by its length-normalized conditional log-likelihood: ℓΘ(ci,τi)=1|τi|∑j=1|τi|logpψ(ai,j∣ai,<j,ci,hiaff), _ (c_i, _i)= 1| _i| _j=1^| _i| p_ψ (a_i,j a_i,<j,c_i,h_i^aff ), (12) where Θ=ω,ψ =\ω,ψ\. All proposals are rescored under teacher forcing, with ai,<ja_i,<j denoting the recorded action prefix. Recovery snippets contain A*-derived expert actions, whereas deviation snippets contain model-generated actions collected during autoregressive rollouts. Importantly, the scores are computed using model-inferred rather than ground-truth affordance representations, matching closed-loop inference. Length normalization prevents a bias toward shorter proposals. We aggregate the proposal scores over the temporal snippet: SΘ(ζ)=1L∑i=1LℓΘ(ci,τi).S_ (ζ)= 1L _i=1^L _ (c_i, _i). (13) Temporal aggregation captures whether consecutive decisions amplify or reduce navigation errors, which cannot be reliably characterized by isolated actions. Given a cross-rollout preference pair (ζ+,ζ−)(ζ^+,ζ^-), we define ΔΘ=SΘ(ζ+)−SΘ(ζ−),PΘ(ζ+≻ζ−)=σ(βΔΘ). gathered _ =S_ (ζ^+)-S_ (ζ^-),\\ P_ (ζ^+ ζ^-)=σ (β _ ). gathered (14) where ζ+ζ^+ denotes a normal or recovery snippet, ζ−ζ^- denotes a deviation snippet, and β controls the preference sharpness. The pairwise preference loss is ℒpref(Θ,pref)=−(ζ+,ζ−)∼pref[logσ(βΔΘ)].L_pref ( ;D_pref )=-E_(ζ^+,ζ^-) _pref [ σ (β _ ) ]. (15) The post-training objective is ℒpost _post =12ℒdual(Θ,exp) = 12L_dual( ;D_exp) +12ℒdual(Θ,on) + 12L_dual( ;D_on) +λprefℒpref(Θ,pref). + _prefL_pref( ;D_pref). Equal-sized expert and on-policy batches preserve the original affordance-estimation and action-generation capabilities while adapting PACE to model-induced states, whereas the preference term favors normal and recovery behaviors over deviation-amplifying alternatives. 3.4 Training, Inference, and Plug-and-Play Integration Algorithm 1 summarizes PACE training and receding-horizon inference. Supervised initialization is followed by iterative refinement using labeled on-policy states and rollout-derived preference pairs. Algorithm 1 PACE Training and Inference 1: Expert data expD_exp, refinement rounds R, beam width B 2: Θ(0)argminΘℒdual(Θ,exp) ^(0) _ L_dual( ;D_exp) ⊳ Imitation learning 3: for r=1,…,Rr=1,…,R do 4: (r)←Rollout(πΘ(r−1))T^(r) ( _ ^(r-1)) 5: on(r)←LabelOnPolicy((r))D_on^(r) (T^(r)) 6: (ζ+,ζ−)(r)←BuildPreferencePairs((r))(ζ^+,ζ^-)^(r) (T^(r)) 7: pref(r)←(ζ+,ζ−)(r)D_pref^(r)←\(ζ^+,ζ^-)^(r)\ 8: Θ(r)←argminΘℒpost(Θ,exp,on(r),pref(r)) ^(r)← _ L_post( ;D_exp,D_on^(r),D_pref^(r)) 9: end for 10: Inference at step t: 11: z^t←(p^taff,ϕ^t) z_t←( p^aff_t, φ_t) ⊳ Affordance estimation 12: ℬt←BeamSearchB(pψ(τt|ct,htaff))B_t _B(p_ψ( _t|c_t,h_t^aff)) 13: τ^t←argmaxτ∈ℬtJt(τ) τ_t← _τ _tJ_t(τ) ⊳ Geometry-aware selection 14: Execute a^t,1 a_t,1 and replan At inference, beam search retains up to B candidate short-horizon action sequences: ℬt=BeamSearchB[pψ(τt∣ct,htaff)],|ℬt|≤B.B_t=BeamSearch_B [p_ψ ( _t c_t,h_t^aff ) ], |B_t|≤ B. (16) The candidates are rolled out with known motion primitives and reranked by their likelihood and geometric consistency with the predicted affordance pose: Jt(τ) J_t(τ) =logpψ(τ∣ct,htaff)−λgG(τ,p^taff), = p_ψ (τ c_t,h_t^aff )- _gG (τ, p_t^aff ), (17) τ^t τ_t =argmaxτ∈ℬtJt(τ), = _τ _tJ_t(τ), Here (rxy,rθ)=Transform(p^taff;τ)(r_xy,r_θ)=Transform( p_t^aff;τ) is the terminal residual after executing τ, G=∥rxy∥2+λθ|wrap(rθ)|+λs[STOP∉τ]G=\|r_xy\|_2+ _θ|wrap(r_θ)|+ _s1[ STOP∉τ], and logpψ(τ∣ct,htaff) p_ψ(τ c_t,h_t^aff) is the cumulative sequence log-likelihood. PACE interfaces with heterogeneous VLN agents through unified RGB-D, semantic-guidance, and action interfaces, without accessing internal representations or requiring retraining. GroundingDINO detects visible staircases, while the VLM assesses their relevance and provides directional guidance. PACE retains control until the phase head predicts Exit or a safety/budget limit is reached, then returns control to the navigator and may be reactivated if traversal remains incomplete. 4 Experiments We evaluate PACE in terms of: (1) long-horizon affordance and short-horizon action prediction; (2) improvements to zero-shot VLN baselines; (3) real-world cross-floor navigation; and (4) standalone stair traversal and ablations of affordance conditioning and preference refinement. R2R-CE-CF RxR-CE-CF Method SR (%) ↑ OSR (%) ↑ SPL (%) ↑ NDTW ↑ SR (%) ↑ OSR (%) ↑ SPL (%) ↑ NDTW ↑ Alone / +PACE Alone / +PACE Alone / +PACE Alone / +PACE Alone / +PACE Alone / +PACE Alone / +PACE Alone / +PACE VLN-Zero 6.90 / 23.75▲ 10.34 / 30.27▲ 6.00 / 18.45▲ 0.24 / 0.46▲ 3.23 / 7.37▲ 4.61 / 8.76▲ 2.14 / 4.87▲ 0.18 / 0.22▲ 3-step-Nav 6.90 / 19.54▲ 11.49 / 36.02▲ 5.95 / 12.53▲ 0.38 / 0.45▲ 5.99 / 10.14▲ 5.99 / 20.28▲ 4.81 / 3.65△ 0.37 / 0.32▼ CA-Nav 14.94 / 25.67▲ 27.97 / 56.70▲ 7.15 / 11.84▲ 0.17 / 0.23▲ 4.15 / 11.98▲ 11.98 / 25.35▲ 1.54 / 4.95▲ 0.17 / 0.20▲ STRIDER 17.24 / 21.84△ 21.84 / 29.50▲ 15.85 / 17.53△ 0.47 / 0.51▲ 2.30 / 8.29▲ 4.15 / 11.98▲ 1.80 / 6.64▲ 0.35 / 0.38▲ Open-Nav 20.31 / 24.90△ 24.52 / 37.55▲ 18.43 / 19.99△ 0.52 / 0.57▲ 3.23 / 6.91△ 3.69 / 9.68▲ 2.84 / 5.76△ 0.34 / 0.40▲ HSGM 31.80 / 50.19▲ 42.91 / 63.22▲ 20.87 / 35.93▲ 0.29 / 0.34▲ 9.68 / 27.65▲ 13.82 / 37.79▲ 6.66 / 19.19▲ 0.38 / 0.46▲ Table 1: Cross-floor navigation results on the val-unseen splits of R2R-CE-CF and RxR-CE-CF. Each pair reports Alone/+PACE, with the higher value in bold. For the 95% paired-bootstrap CI of Δ=+PACE−Alone =+PACE-Alone, ▲ , △ , and ▼ denote robust improvement (CI>0>0), inconclusive difference (CI contains zero), and robust decrease (CI<0<0), respectively. 4.1 Experimental Setup Datasets. We evaluate on the cross-floor subsets of R2R-CE (Krantz et al., 2020) and RxR-CE(en-US) (Gao et al., 2025; Ku et al., 2020), denoted R2R-CE-CF and RxR-CE-CF, which retain episodes with start-to-goal vertical displacement |hs−hg|>1m|h_s-h_g|>1\,m. R2R-CE-CF contains 925 training and 261 val-unseen episodes, while the en-US subset of RxR-CE-CF contains 932 and 217, respectively. PACE is trained on the corresponding training splits, and all baselines are evaluated on the full val-unseen splits. Baselines and Metrics. We integrate PACE with six recent open-source zero-shot VLN systems. VLN-Zero uses cache-enabled neurosymbolic planning (Bhatt et al., 2025), whereas CA-Nav performs constraint-aware value mapping (Chen et al., 2025b). 3-step-Nav adopts global-local planning with trajectory auditing (Zheng et al., 2026c), while Open-Nav employs open-source LLM reasoning (Qiao et al., 2025). STRIDER optimizes structured waypoint decisions (He et al., 2026), whereas HSGM plans over hierarchical semantic-geometric maps (Li et al., 2026). Since prior works do not separately report cross-floor results, we rerun all released baselines under a unified protocol using the same high-level VLM, both alone and with PACE. For overall cross-floor VLN, we report Success Rate (SR, within 3m3\,m of the goal), Oracle Success Rate (OSR), Success weighted by Path Length (SPL), and Normalized Dynamic Time Warping (NDTW). For local stair traversal, we report OSR, SPL, NDTW, and Collision Rate (CR). Implementation Details. We locally deploy the open-source Qwen-3.6-27B-FP8 (Qwen Team, 2026) as the shared high-level VLM for cost-efficient evaluation. PACE uses five-step contexts and ImageNet-pretrained dual-stream ResNet-18 backbones. Supervised training runs for 15 epochs with AdamW, batch size 32, an initial learning rate of 10−410^-4, and cosine decay. Post-training performs 20 on-policy rounds at 10−510^-5, collecting 2,048 rollouts per round and mixing equal-sized expert and on-policy batches. Rollout starts combine recorded expert states and GroundingDINO randomized approach states at a ratio of 1:81:8. Preference learning uses five-step snippets, λpref=0.1 _pref=0.1, and β=1.0β=1.0. Inference uses beam size 5 and a maximum proposal length of 48. Training takes approximately 19 hours on four NVIDIA RTX 5090 GPUs. Real-world experiments use a Unitree Go2 Edu equipped with a RealSense D435i and a Livox Mid-360, with PACE inference running on an RTX 3070 Ti laptop. Method R2R-CE-CF RxR-CE-CF Takeover Rate (%) SR (%) ↑ Alone / +PACE Takeover Rate (%) SR (%) ↑ Alone / +PACE VLN-Zero 39.08 7.84 / 51.96▲ 11.06 8.33 / 41.67▲ 3-step-Nav 51.72 10.37 / 37.04▲ 42.86 10.75 / 20.43▲ CA-Nav 65.52 19.30 / 36.26▲ 35.48 6.49 / 28.57▲ STRIDER 27.59 19.44 / 47.22▲ 17.05 5.41 / 24.32▲ Open-Nav 38.31 27.00 / 41.00▲ 28.11 8.20 / 24.59▲ HSGM 76.63 34.50 / 63.50▲ 51.15 11.71 / 52.25▲ Table 2: Takeover-subset SR, comparing Alone/+PACE on aligned episode IDs. Takeover Rate is the fraction of episodes with actual PACE takeover; ▲ denotes a paired-bootstrap 95% CI above zero. 4.2 Main Results Figure 3: Standalone PACE predictions on R2R-CE-CF val-unseen. In (a), the center denotes zero positional error; radius and angle encode its magnitude and direction, while arrow deviation from the inward direction represents heading error. (a) Affordance pose error. (b) Phase prediction. (c) Action prefix accuracy. (d) Termination error. PACE Prediction Performance. Figure 3 evaluates the standalone predictive performance of PACE over both horizons through policy rollouts on the R2R-CE-CF val-unseen split. For the long-horizon affordance state, pose errors are concentrated near the origin, with a mean positional error of 0.329m0.329\,m and a yaw MAE of 9.9∘9.9 , indicating that PACE provides a reasonably accurate spatial target for local execution. Phase prediction achieves an accuracy of 96.94%96.94\%, although 27.7%27.7\% of Entry states are misclassified as Approach, revealing ambiguity at the boundary between approaching and entering the stairs. For short-horizon action generation, prefix accuracy decreases only modestly from 74.98%74.98\% for the first action to 72.67%72.67\% over the first ten actions, suggesting limited autoregressive error accumulation under rollout evaluation. Moreover, 42.6%42.6\% of STOP predictions are exact, and 71.0%71.0\% occur no more than two steps early. Overall, these rollout results demonstrate reliable affordance grounding and stable action generation under policy-induced states, consistent with the robustness objective of failure-aware preference refinement. Quantitative Evaluation on VLN Benchmarks. Table 1 shows that standalone zero-shot VLN methods remain limited in cross-floor scenarios, with the best SR reaching only 31.80% on R2R-CE-CF and 9.68% on RxR-CE-CF. With PACE, all six methods improve across every metric on R2R-CE-CF, with average gains of 11.30, 19.03, and 7.01 percentage points in SR, OSR, and SPL, respectively, and 0.082 in NDTW. On RxR-CE-CF, PACE improves SR and OSR for all methods and improves SPL and NDTW for five of six. HSGM+PACE achieves the highest SR, OSR, and SPL on both benchmarks. On episodes with actual PACE takeover, Table 2 shows SR gains of 9.68–44.12 percentage points across all navigator–benchmark pairs, with all paired-bootstrap 95% confidence intervals above zero. However, non-random takeover and stochastic VLM sampling make this analysis associative rather than causal. Across 10,000 paired-bootstrap resamples of the full aligned results, PACE yields 40/48 robust improvements, seven inconclusive differences, and one robust decrease. Gain variations likely reflect takeover frequency and execution quality. NDTW improves less because PACE targets critical stair transitions, whereas NDTW evaluates the full trajectory. Overall, PACE is broadly compatible with diverse zero-shot VLN navigators. Qualitative Real-World Case Study. Figure 4: Real-world two-floor stair traversal with VLN-Zero and PACE. Blue and orange points denote VLM- and PACE-controlled trajectory segments, respectively. We deploy VLN-Zero (Bhatt et al., 2025) with PACE without an explicit navigation map or environment cache. Planning takes 0.2–0.5 s for the VLM or approximately 0.2 s for PACE; with 2 s MPC execution, each control cycle takes 2.2–2.5 s. As shown in Fig. 4, we evaluate the system in an unseen two-floor stairwell with an elevation gain of approximately 7 m under the instruction “Enter the stairwell and proceed upstairs until reaching the top floor.” SLAM is used only for trajectory visualization. PACE successfully negotiates narrow and directionally ambiguous stair-transition regions, as illustrated by Regions (1) and (2). At stair corners, it predicts an executable affordance pose aligned with the next flight and translates it into local actions, allowing the robot to complete the turn and continue ascending. In Region (2), PACE occasionally emits STOP before reaching the stair exit; the high-level VLM detects the incomplete instruction and reactivates PACE, enabling recovery and completion of the two-floor ascent. The alternating control is reflected by the interleaved VLM- and PACE-controlled trajectory segments. PACE nevertheless exhibits local control instability during stair traversal, including left–right oscillations and reduced clearance from the railing. In Region (3), the clustered orange trajectory points indicate repeated local corrections and reduced execution efficiency, likely caused by sim-to-real viewpoint discrepancies and contact-induced perturbations during ascent. 4.3 Ablation Studies # Method OSR (%) ↑ CR (%) ↓ SPL (%) ↑ NDTW (%) ↑ 1 PACE 67.4 ± 1.1 2.0 ± 0.2 64.3 ± 2.9 84.8 ± 1.1 2 Waypoint Cond. 63.1 ± 0.5 11.5 ± 4.5 59.9 ± 2.1 70.6 ± 1.0 3 w/o Pref. Loss 56.5 ± 1.4 4.0 ± 0.8 54.0 ± 4.7 64.3 ± 1.9 4 w/o Aff. Cond. 53.9 ± 2.6 4.7 ± 0.4 50.6 ± 3.6 59.8 ± 1.4 5 Aux. Aff. Only 49.3 ± 3.1 2.9 ± 0.2 47.6 ± 2.8 57.3 ± 3.5 6 w/o F.A. Ref. 37.3 ± 0.9 8.3 ± 2.2 34.5 ± 4.0 57.4 ± 1.3 7 Action-Only BC 17.5 ± 0.5 9.2 ± 3.7 15.6 ± 6.9 51.7 ± 2.6 Table 3: Standalone stair-traversal ablation of PACE. “Aff.”, “Cond.”, “Pref.”, and “F.A. Ref.” denote affordance, conditioning, preference, and failure-aware refinement, respectively. Results are reported as mean±standard deviation over three runs. We conduct ablations on standalone stair-traversal segments of R2R-CE-CF to isolate local execution from high-level VLM uncertainty (Table 3). Compared with waypoint conditioning, PACE improves OSR by 4.3 percentage points while reducing CR by 9.5 points and improving NDTW by 14.2 points, indicating that its affordance state provides more effective guidance than a generic goal pose. Removing affordance conditioning or using affordance prediction only as auxiliary supervision decreases OSR by 13.5 and 18.1 points, respectively, supporting its direct use by the action decoder. The action-only BC baseline exhibits a 49.9-point OSR gap, confirming the importance of the intermediate spatial representation. Removing the preference loss reduces OSR by 10.9 points, whereas removing the entire failure-aware refinement stage causes a 30.1-point drop, demonstrating the complementary contributions of preference optimization and policy-induced recovery supervision. 5 Conclusion This work presents PACE, a plug-and-play affordance-conditioned supervised local execution framework that connects zero-shot semantic planners with reliable execution for cross-floor VLN. PACE grounds stair-transition semantics into traversable affordance poses, conditions short-horizon action generation on the learned affordance representation, and improves closed-loop correction through failure-aware preference refinement. Experiments across six zero-shot VLN navigators and real-world deployment demonstrate its effectiveness and applicability. Future work will extend PACE to other spatially constrained transitions, such as doorways and narrow passages. References An et al. (2024) Dong An, Hanqing Wang, Wenguan Wang, Zun Wang, Yan Huang, Keji He, and Liang Wang. Etpnav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. Bhatt et al. (2025) Neel P Bhatt, Yunhao Yang, Rohan Siva, Pranay Samineni, Daniel Milan, Zhangyang Wang, and Ufuk Topcu. Vln-zero: Rapid exploration and cache-enabled neurosymbolic vision-language planning for zero-shot transfer in robot navigation. arXiv preprint arXiv:2509.18592, 2025. Che et al. (2026) Lirong Che, Shuo Wen, Shan Huang, Chuang Wang, Yuzhe Yang, Gregory Dudek, Xueqian Wang, and Jian Su. Mansion: Multi-floor language-to-3d scene generation for long-horizon tasks, 2026. URL https://arxiv.org/abs/2603.11554. Chen et al. (2025a) Jiaqi Chen, Bingqian Lin, Xinmin Liu, Lin Ma, Xiaodan Liang, and Kwan-Yee K Wong. Affordances-oriented planning using foundation models for continuous vision-language navigation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23568–23576, 2025a. Chen et al. (2025b) Kehan Chen, Dong An, Yan Huang, Rongtao Xu, Yifei Su, Yonggen Ling, Ian Reid, and Liang Wang. Constraint-aware zero-shot vision-language navigation in continuous environments, 2025b. URL https://arxiv.org/abs/2412.10137. Chen et al. (2025c) Shiqi Chen, Tongyao Zhu, Ruochen Zhou, Jinghan Zhang, Siyang Gao, Juan Carlos Niebles, Mor Geva, Junxian He, Jiajun Wu, and Manling Li. Why is spatial reasoning hard for VLMs? An attention mechanism perspective on focus areas. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 9910–9932. PMLR, 13–19 Jul 2025c. URL https://proceedings.mlr.press/v267/chen25cr.html. Chu et al. (2026) Zedong Chu, Shichao Xie, Xiaolong Wu, Yanfen Shen, Minghua Luo, Zhengbo Wang, Fei Liu, Xiaoxu Leng, Junjun Hu, Mingyang Yin, et al. Abot-n0: Technical report on the vla foundation model for versatile embodied navigation. arXiv preprint arXiv:2602.11598, 2026. Gao et al. (2025) Fang Gao, Lei Shi, Jingfeng Tang, Jiabao Wang, Shaodong Li, Shengheng Ma, and Jun Yu. Visual and textual commonsense-enhanced layout learning for vision-and-language navigation. IEEE Transactions on Automation Science and Engineering, 22:21311–21324, 2025. doi: 10.1109/TASE.2025.3604018. Gong et al. (2026) Zeying Gong, Rong Li, Tianshuai Hu, Ronghe Qiu, Lingdong Kong, Lingfeng Zhang, Guoyang Zhao, Yiyi Ding, and Junwei Liang. Stairway to success: An online floor-aware zero-shot object-goal navigation framework via llm-driven coarse-to-fine exploration. IEEE Robotics and Automation Letters, 11(3):2943–2950, 2026. He et al. (2026) Diqi He, Xuehao Gao, Hao Li, Junwei Han, and Dingwen Zhang. Strider: Navigation via instruction-aligned structural decision space optimization. Advances in Neural Information Processing Systems, 38:26700–26720, 2026. Huang et al. (2026) Jingzhi Huang, Junkai Huang, Haoyang Yang, Haoang Li, and Yi Wang. Aerr-nav: Adaptive exploration-recovery-reminiscing strategy for zero-shot object navigation, 2026. URL https://arxiv.org/abs/2603.17712. ichter et al. (2023) brian ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Kanishka Rao, Pierre Sermanet, Alexander T Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Mengyuan Yan, Noah Brown, Michael Ahn, Omar Cortes, Nicolas Sievers, Clayton Tan, Sichun Xu, Diego Reyes, Jarek Rettinghouse, Jornell Quiambao, Peter Pastor, Linda Luu, Kuang-Huei Lee, Yuheng Kuang, Sally Jesmonth, Nikhil J. Joshi, Kyle Jeffrey, Rosario Jauregui Ruano, Jasmine Hsu, Keerthana Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu. Do as i can, not as i say: Grounding language in robotic affordances. In Karen Liu, Dana Kulic, and Jeff Ichnowski, editors, Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pages 287–318. PMLR, 14–18 Dec 2023. URL https://proceedings.mlr.press/v205/ichter23a.html. Jang and Kim (2026) Won Shik Jang and Ue-Hwan Kim. Context-nav: Context-driven exploration and viewpoint-aware 3d spatial reasoning for instance navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9626–9636, 2026. Kong et al. (2025) Yangzhe Kong, Daeun Song, Jing Liang, Dinesh Manocha, Ziyu Yao, and Xuesu Xiao. Autospatial: Visual-language reasoning for social robot navigation through efficient spatial reasoning learning. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11298–11304. IEEE, 2025. Krantz et al. (2020) Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-language navigation in continuous environments. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision – ECCV 2020, pages 104–120, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58604-1. Ku et al. (2020) Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4392–4412, 2020. Li et al. (2026) Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, and Liang He. Bridging the 2d-3d gap: A hierarchical semantic-geometric map for vision language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15243–15252, 2026. Long et al. (2024) Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. Discuss before moving: Visual language navigation via multi-expert discussions. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 17380–17387, 2024. doi: 10.1109/ICRA57147.2024.10611565. Long et al. (2025) Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. In Conference on Robot Learning, pages 2049–2060. PMLR, 2025. Qiao et al. (2025) Yanyuan Qiao, Wenqi Lyu, Hui Wang, Zixu Wang, Zerui Li, Yuan Zhang, Mingkui Tan, and Qi Wu. Open-nav: Exploring zero-shot vision-and-language navigation in continuous environment with open-source llms. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 6710–6717. IEEE, 2025. Qwen Team (2026) Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026. URL https://qwen.ai/blog?id=qwen3.6-27b. Ramrakhya et al. (2023) Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17896–17906, 2023. Ross et al. (2011) Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011. Shao et al. (2024) Jie-Jing Shao, Hao-Sen Shi, Lan-Zhe Guo, and Yu-Feng Li. Offline imitation learning with model-based reverse augmentation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2608–2617, 2024. Shi et al. (2025) Xiangyu Shi, Zerui Li, Wenqi Lyu, Jiatong Xia, Feras Dayoub, Yanyuan Qiao, and Qi Wu. Smartway: Enhanced waypoint prediction and backtracking for zero-shot vision-and-language navigation. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 16923–16930. IEEE, 2025. Wang et al. (2026a) Ronghu Wang, Ziyan Liu, Weidong Chen, Banghai He, and Ziyou Wang. Semantic-aware and spatially-grounded multi-expert network for continuous vision-and-language navigation. Knowledge-Based Systems, page 116308, 2026a. Wang et al. (2026b) Zihan Wang, Seungjun Lee, and Gim Hee Lee. Dynam3d: Dynamic layered 3d tokens empower vlm for vision-and-language navigation. Advances in Neural Information Processing Systems, 38:153522–153544, 2026b. Wei et al. (2025) Meng Wei, Chenyang Wan, Jiaqi Peng, Xiqian Yu, Yuqiang Yang, Delin Feng, Wenzhe Cai, Chenming Zhu, Tai Wang, Jiangmiao Pang, et al. Ground slow, move fast: A dual-system foundation model for generalizable vision-and-language navigation. arXiv preprint arXiv:2512.08186, 2025. Werby et al. (2024) Abdelrhman Werby, Chenguang Huang, Martin Büchner, Abhinav Valada, and Wolfram Burgard. Hierarchical open-vocabulary 3d scene graphs for language-grounded robot navigation. In First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024. Windecker et al. (2025) Tim Windecker, Manthan Patel, Moritz Reuss, Richard Schwarzkopf, Cesar Cadena, Rudolf Lioutikov, Marco Hutter, and Jonas Frey. Navitrace: Evaluating embodied navigation of vision-language models. arXiv preprint arXiv:2510.26909, 2025. Yang et al. (2025) Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025. Yokoyama et al. (2024) Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 42–48. IEEE, 2024. Yu et al. (2026) Mingming Yu, Fei Zhu, Wenzhuo Liu, Yirong Yang, Qunbo Wang, Jing Liu, et al. C-nav: Towards self-evolving continual object navigation in open world. Advances in Neural Information Processing Systems, 38:126181–126207, 2026. Yuan et al. (2024) Shuaihang Yuan, Hao Huang, Yu Hao, Congcong Wen, Anthony Tzes, and Yi Fang. Gamap: Zero-shot object goal navigation with multi-scale geometric-affordance guidance. Advances in Neural Information Processing Systems, 37:39386–39408, 2024. Zhang et al. (2024) Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and-language navigation. arXiv preprint arXiv:2402.15852, 2024. Zhang et al. (2026) Jie Zhang, Xiaoyue Chen, Anzhe Chen, Dayiheng Liu, Deqing Li, Gengze Zhou, Hale Yin, Haoqi Yuan, Haoyang Li, Jiahao Li, Jiazhao Zhang, Jingren Zhou, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Pei Lin, Qihang Peng, Shengming Yin, Tianhe Wu, Tianyi Yan, Xiao Xu, Yan Shu, Yanran Zhang, Ye Wang, Yi Wang, Yilei Chen, Yixian Xu, Yiyang Huang, Yuxiang Chen, Zekai Zhang, Zhendong Wang, Zixing Lei, Zhixuan Liang, Zihao Liu, Zikai Zhou, Chenxu Lv, Xiong-Hui Chen, and Chenfei Wu. Qwen-robotworld technical report: Unifying embodied world modeling through language-conditioned video generation, 2026. URL https://arxiv.org/abs/2606.17030. Zhang et al. (2025a) Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang, Shanghang Zhang, and Renjing Xu. Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and-language navigation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13032–13056, 2025a. Zhang et al. (2025b) Lingfeng Zhang, Hao Wang, Erjia Xiao, Xinyao Zhang, Qiang Zhang, Zixuan Jiang, and Renjing Xu. Multi-floor zero-shot object navigation policy. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 6416–6422, 2025b. doi: 10.1109/ICRA55743.2025.11128607. Zheng et al. (2026a) Han Zheng, Zhe Chen, Yudong Huang, Haoran Liu, Jinghao Wang, Ming Yang, and Tong Qin. Travexplorer: Cross-floor embodied exploration via traversability-aware 3-d planning. arXiv e-prints, pages arXiv–2605, 2026a. Zheng et al. (2026b) Wancai Zheng, Hao Chen, Xianlong Lu, Linlin Ou, and Xinyi Yu. 3dgsnav: Enhancing vision-language model reasoning for object navigation via active 3d gaussian splatting, 2026b. URL https://arxiv.org/abs/2602.12159. Zheng et al. (2026c) Wanrong Zheng, Yunhao Ge, and Laurent Itti. Three-step nav: A hierarchical global-local planner for zero-shot vision-and-language navigation, 2026c. URL https://arxiv.org/abs/2604.26946. Zhou et al. (2024) Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649, 2024. Zhou et al. (2026) Zibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li, and Siheng Chen. Beliefmapnav: 3d voxel-based belief map for zero-shot object navigation. Advances in Neural Information Processing Systems, 38:83008–83036, 2026. This document supplements the main submission. All notation, metrics, baselines, and external models referred to below follow the main paper, where the corresponding references are given. Appendix A Failure-Aware Preference Refinement Details Rollout and recovery labels. Each refinement round collects 2,048 closed-loop rollouts from recorded expert states and GroundingDINO-randomized approach states at a 1:81:8 ratio. The current policy is executed autoregressively, while the privileged expert route is used only during training to compute nearest-route positional and heading errors and to derive recovery targets. A deviation begins when either error exceeds 0.5m0.5\,m or 30∘30 for two consecutive steps. Failure is triggered immediately when either error exceeds 1.5m1.5\,m or 60∘60 . From a failure state, A* plans toward a future expert-route anchor, and the resulting path is converted into the same 0.25m0.25\,m and 30∘30 motion primitives. Recovery ends when both errors remain within 0.25m0.25\,m and 30∘30 for two consecutive steps. Algorithm 2 summarizes this training-time labeling process. Algorithm 2 Rollout Failure and Recovery Labeling 1: Current policy πΘ _ and privileged expert route 2: m←Normalm← Normal; ndrift←0n_ drift← 0; nreturn←0n_ return← 0 3: while the rollout is active do 4: Compute nearest-route errors (dt,δt)(d_t, _t) 5: if m≠Recoverym≠ Recovery then 6: if dt>0.5md_t>0.5\,m or δt>30∘ _t>30 then 7: ndrift←ndrift+1n_ drift← n_ drift+1 8: else 9: ndrift←0n_ drift← 0 10: end if 11: if ndrift≥2n_ drift≥ 2 then 12: m←Deviationm← Deviation 13: end if 14: if dt>1.5md_t>1.5\,m or δt>60∘ _t>60 then 15: Plan an A* path to a future expert-route anchor 16: Discretize the path into recovery primitives 17: m←Recoverym← Recovery 18: end if 19: else 20: Execute the next A*-derived recovery primitive 21: if dt≤0.25md_t≤ 0.25\,m and δt≤30∘ _t≤ 30 then 22: nreturn←nreturn+1n_ return← n_ return+1 23: else 24: nreturn←0n_ return← 0 25: end if 26: if nreturn≥2n_ return≥ 2 then 27: m←Normalm← Normal 28: end if 29: end if 30: Record the current state and proposal with role m 31: end while The expert route and A*-derived actions are privileged training signals and are never available during evaluation. Snippet pools and matching. Within each rollout, maximal contiguous runs with the same role (Normal, Deviation, or Recovery) are divided into all sliding windows of length five. The resulting windows are pooled globally across rollouts within the current refinement round. These pools are rebuilt, rather than accumulated, across rounds. After the three pools are independently shuffled, each deviation window is paired once with a normal window and once with a recovery window when the corresponding positive pool is non-empty. Positive pools wrap around when exhausted, producing at most two pairs per deviation window without imposing an additional class ratio. Algorithm 3 Preference-Pair Construction in Round r 1: Policy πΘr−1 _ ^r-1 and expert data 2: r←Rollout(πΘr−1)T^r ( _ ^r-1) 3: Label rollout states as normal, deviation, or recovery 4: Build length-5 pools rN^r, rD^r, and ℛrR^r 5: Independently shuffle rN^r, rD^r, and ℛrR^r 6: r←∅P^r← 7: for k=1,…,|r|k=1,…,|D^r| do 8: if |r|>0|N^r|>0 then 9: u←1+((k−1)mod|r|)u← 1+((k-1) |N^r|) 10: r←r∪(ur,kr)P^r ^r∪\(N^r_u,D^r_k)\ 11: end if 12: if |ℛr|>0|R^r|>0 then 13: v←1+((k−1)mod|ℛr|)v← 1+((k-1) |R^r|) 14: r←r∪(ℛvr,kr)P^r ^r∪\(R^r_v,D^r_k)\ 15: end if 16: end for 17: Update Θr ^r using equal expert/on-policy batches and rP^r Every recorded proposal, including its terminal STOP, is rescored under teacher forcing using model-predicted affordance features. For a snippet ζ=((ci,τi))i=1Lζ=((c_i, _i))_i=1^L with L=5L=5, its score is SΘ(ζ)=1L∑i=1L1|τi|∑j=1|τi|logpψ(ai,j∣ai,<j,ci,hiaff).S_ (ζ)= 1L _i=1^L 1| _i| _j=1^| _i| p_ψ (a_i,j a_i,<j,c_i,h_i^aff ). (18) Token averaging removes a systematic preference for shorter proposals, while temporal averaging evaluates whether consecutive decisions sustain correction rather than relying on an isolated action. Appendix B Implementation and Annotation Details Architecture and optimization. PACE uses separate ImageNet-pretrained ResNet-18 backbones for RGB and depth. Their 512-dimensional outputs are each projected to d/2=128d/2=128 dimensions, which is the shared visual feature space referred to in the main paper, and are combined by a learned gate. Historical actions and phases use 16-dimensional embeddings, historical affordance poses use a 64-dimensional MLP encoding, and directional guidance uses a 16-dimensional embedding. The concatenated visual, historical, and semantic features form one context token per step, which is projected to the d=256d=256 Transformer hidden space. The temporal encoder has two four-head layers, the query-based affordance estimator has one four-head decoder layer, and the causal action decoder has two four-head layers. All Transformer feed-forward layers have dimension 1,024. The complete model has 27.4M trainable parameters, including 22.4M in the two visual backbones. Setting Value Visual backbones dual ImageNet-pretrained ResNet-18 Context five steps (K=5K=5) Hidden dimension d=256d=256 Attention / FFN 4 heads / 1,024 Temporal/query/action 2/1/22/1/2 layers Dropout feature/Transformer =0.35/0.20=0.35/0.20 Objective weights λp/λϕ/λa=1/0.5/1 _p/ _φ/ _a=1/0.5/1 Loss details Smooth-L1 β=1β=1; label smoothing 0.05 Imitation learning 15 epochs; batch 32 IL optimizer AdamW; LR 10−410^-4; WD 5×10−45×10^-4 Failure-aware refinement 20 on-policy rounds; LR 10−510^-5 Refinement rollouts 2,048 per round Rollout starts expert/randomized =1:8=1:8 Refinement batches expert/on-policy =16/16=16/16 Preference five-step snippets; λpref=0.1 _ pref=0.1; β=1β=1 Proposal maximum length 48, including STOP Inference beam size 5 Geom. weights (λg,λθ,λs)=(0.35,0.25,0.5)( _g, _θ, _s)=(0.35,0.25,0.5) IL checkpoint lowest validation objective Refinement checkpoint final refinement round Seeds 42 (main); 42/43/44 (ablations) Table 4: PACE implementation settings. Imitation learning uses cosine learning-rate decay; all test-time models are frozen. Cross-floor splits and scene isolation. We preserve the official R2R-CE and RxR-CE train/val-unseen assignments and retain episodes with |hs−hg|>1m|h_s-h_g|>1\,m. This yields 925/261 R2R-CE-CF and 932/217 RxR-CE-CF train/val-unseen episodes, matching the main paper. The retained train/val-unseen episodes span 61/11 R2R scenes and 59/11 RxR scenes, with no scene overlap between the two splits. Pose and phase annotation. Algorithm 4 gives the deterministic label construction. A non-flat height change starts a stair run, and the preceding expert pose is treated as its entry. A direction reversal closes the run immediately; alternatively, F consecutive flat states confirm a plateau, with the first flat state selected as the exit. We use F=3F=3 for R2R and F=9F=9 for RxR and discard runs whose total height change is below 0.5m0.5\,m. For runs with an observed pre-stair pose, Approach denotes the retained prefix, Entry marks the pre-stair key pose, Traverse covers the interval toward the exit pose, and Exit is terminal. When a sequence starts on the stairs, it enters Traverse directly without an Entry label. Pose yaw follows the local expert-path tangent. Because Exit is terminal and defines no remaining traversal target, no affordance pose label is assigned to it. The pose regression term of ℒdualL_dual is therefore masked on Exit states, and the expectation in that term is taken over the non-terminal states only; the phase and action terms remain active on all states. Equivalently, λp _p is set to zero whenever ϕt∗=Exit _t^*= Exit. Algorithm 4 Affordance Pose and Phase Labels 1: Expert poses qi=(Xi,Yi,Zi,θi)i=1T\q_i=(X_i,Y_i,Z_i, _i)\_i=1^T 2: Quantize ΔYi Y_i as up/down/flat using 0.05m0.05\,m 3: Start a run at the first non-flat change; set entry e=i−1e=i-1 4: Close at a sign reversal, or after F flats using the first flat as exit x 5: Let b←(e=1)b←(e=1) indicate that the sequence starts on stairs 6: if |Yx−Ye|≥0.5m|Y_x-Y_e|≥ 0.5\,m then 7: for each index j associated with this run do 8: if b and j<xj<x then 9: ϕj∗=Traverse _j^*= Traverse; pj∗=qxp_j^*=q_x 10: else if j<ej<e then 11: ϕj∗=Approach _j^*= Approach; pj∗=qep_j^*=q_e 12: else if j=ej=e then 13: ϕj∗=Entry _j^*= Entry; pj∗=qxp_j^*=q_x 14: else if e<j<xe<j<x then 15: ϕj∗=Traverse _j^*= Traverse; pj∗=qxp_j^*=q_x 16: else 17: ϕj∗=Exit _j^*= Exit; no pose target, mask ℒposeL_pose 18: end if 19: end for 20: end if For the current horizontal pose (Xi,Zi,θi)(X_i,Z_i, _i) and target (X∗,Z∗,θ∗)(X^*,Z^*,θ^*), let ΔX=X∗−Xi X=X^*-X_i and ΔZ=Z∗−Zi Z=Z^*-Z_i. The agent-centric label is x∗=sinθiΔX+cosθiΔZ,y∗=−cosθiΔX+sinθiΔZ,θrel∗=wrap(θ∗−θi). splitx^*&= _i\, X+ _i\, Z,\\ y^*&=- _i\, X+ _i\, Z,\\ _rel^*&=wrap(θ^*- _i). split (19) Here, +x+x points forward and +y+y points left. Randomized approach trajectories terminate at the same entry pose and use the same agent-centric transformation. Appendix C Integration Details Unified baseline adapter. PACE interfaces with all six frozen navigators through unified RGB-D, semantic-guidance, and action interfaces. Each navigator retains its own maps, memory, hidden states, and planner; none are exposed to PACE or retrained. GroundingDINO first detects visible staircases. For each detection, the VLM assesses its relevance to the current instruction and returns a directional prediction d∈Up,Down,Unknownd∈\ Up, Down, Unknown\ together with a relevance confidence score q. PACE is activated when the VLM either predicts an explicit traversal direction or confidently confirms that the detected staircase is relevant to the instruction, i.e., d∈Up,Downd∈\ Up, Down\ or q≥δactq≥ _act. This confidence-based condition allows PACE to take over when the VLM reliably recognizes the required staircase even if its directional prediction remains Unknown. Otherwise, the navigator retains control. Algorithm 5 summarizes this closed-loop integration. Algorithm 5 PACE Takeover and Handoff 1: Frozen navigator N, instruction I, observation oto_t 2: mode←Navigatormode← Navigator 3: d←Unknownd← Unknown 4: while episode is active do 5: if mode=Navigatormode= Navigator then 6: d←Unknownd← Unknown; q←0q← 0 7: if GroundingDINO detects a visible staircase then 8: (d,q)←VLM(I,ot)(d,q) (I,o_t) 9: end if 10: if (d∈Up,Down∨q≥δact) (d∈\ Up, Down\\ \ q≥ _act ) and budget remains then 11: initialize PACE context 12: mode←PACEmode← PACE 13: else 14: execute (I,ot)N(I,o_t) and acquire ot+1o_t+1 15: end if 16: else 17: (ϕ^t,τ^t)←PACE(ct,d)( φ_t, τ_t) (c_t,d) 18: if ϕ^t=Exit φ_t= Exit then 19: mode←Navigatormode← Navigator 20: else 21: execute a^t,1 a_t,1 and acquire ot+1o_t+1 22: update context ct+1c_t+1 23: if episode completion or safety/budget event then 24: mode←Navigatormode← Navigator 25: end if 26: end if 27: end if 28: end while Handoff and reactivation. PACE executes only the first primitive a^t,1 a_t,1 of each short-horizon proposal and replans from the subsequent RGB-D observation. The terminal STOP ends only the current proposal and triggers the next closed-loop decision; it neither terminates the navigation episode nor immediately returns control to the navigator. PACE terminates upon episode completion and otherwise hands control back when the phase head predicts Exit or when a safety or budget limit is reached. In simulation, each takeover is limited to 200 executed primitives and is interrupted after five consecutive forward collisions or six consecutive forward attempts producing less than 0.05m0.05\,m displacement. After handoff, the navigator receives the latest observation and may reactivate PACE through the same staircase-detection and VLM-based activation gate, provided that traversal remains incomplete and sufficient takeover budget remains. Consequently, a handoff occurring before the stair exit has actually been reached is caused by the phase head rather than by the proposal-level STOP. Specifically, an Exit prediction emitted while the agent is still traversing returns control immediately, consistent with the phase-boundary ambiguity reported in the main paper. The reactivation gate makes such premature handoffs recoverable: if the staircase remains visible and the VLM again predicts a traversal direction or confidently confirms its relevance, PACE can be reactivated in both simulation and real-world deployment. Appendix D Expanded Evaluation Results Table 6 expands the compact significance markers in the main paper by reporting the exact effect sizes and confidence intervals for all 48 method–metric comparisons. PACE yields robust gains in 9/12 SR, 12/12 OSR, 8/12 SPL, and 11/12 NDTW comparisons. Of the remaining eight comparisons, seven are inconclusive; the only robust decrease is NDTW for 3-step-Nav on RxR-CE-CF (Δ=−0.051 =-0.051, 95% CI [−0.081,−0.022][-0.081,-0.022]). Table 5 complements the takeover analysis in the main paper with its exact SR intervals. All 12 intervals are strictly above zero, with gains ranging from 9.68 to 44.12 percentage points, providing consistent evidence across navigator–benchmark pairs. Alone and +PACE are aligned by episode ID but use independently sampled VLM outputs. Thus, the analysis retains the realized end-to-end variation rather than relying on matched VLM decisions. The uniformly positive takeover-conditioned intervals indicate that the observed SR advantage remains stable under this variation, although repeated decoding seeds would be required to isolate VLM-induced variance. Method R2R-CE-CF RxR-CE-CF Δ (p) 95% CI Δ (p) 95% CI VLN-Zero 44.12 [ 34.31, 53.92 ] 33.33 [ 16.67, 54.17 ] 3-step-Nav 26.67 [ 16.30, 36.30 ] 9.68 [ 2.15, 18.28 ] CA-Nav 16.96 [ 7.60, 25.73 ] 22.08 [ 10.39, 33.77 ] STRIDER 27.78 [ 13.89, 40.28 ] 18.92 [ 8.11, 32.43 ] Open-Nav 14.00 [ 3.00, 25.00 ] 16.39 [ 4.92, 27.87 ] HSGM 29.00 [ 21.00, 37.00 ] 40.54 [ 30.63, 50.45 ] Table 5: Takeover-conditioned SR differences and their percentile paired-bootstrap 95% confidence intervals. Method R2R-CE-CF RxR-CE-CF Δ (p) Δ (p) Δ (p) Δ Δ (p) Δ (p) Δ (p) Δ VLN-Zero 16.86 19.92 12.46 0.219 4.15 4.15 2.73 0.035 [ 11.88, 22.22 ] [ 14.56, 25.67 ] [ 8.13, 16.88 ] [ 0.184, 0.255 ] [ 0.92, 7.37 ] [ 0.92, 7.37 ] [ 0.66, 5.10 ] [ 0.010, 0.062 ] 3-step-Nav 12.64 24.52 6.58 0.074 4.15 14.29 -1.16 -0.051 [ 7.28, 18.39 ] [ 18.01, 31.03 ] [ 2.40, 10.76 ] [ 0.041, 0.108 ] [ 0.46, 7.83 ] [ 9.22, 19.35 ] [ -3.87, 1.38 ] [ -0.081, -0.022 ] CA-Nav 10.73 28.74 4.69 0.060 7.83 13.36 3.41 0.033 [ 4.21, 17.24 ] [ 21.84, 36.02 ] [ 1.20, 8.16 ] [ 0.030, 0.091 ] [ 3.69, 11.98 ] [ 8.76, 17.97 ] [ 1.64, 5.36 ] [ 0.014, 0.053 ] STRIDER 4.60 7.66 1.69 0.034 5.99 7.83 4.84 0.031 [ -0.38, 9.96 ] [ 2.30, 13.03 ] [ -2.64, 6.06 ] [ 0.009, 0.059 ] [ 3.23, 9.22 ] [ 4.61, 11.52 ] [ 2.39, 7.54 ] [ 0.010, 0.053 ] Open-Nav 4.60 13.03 1.56 0.053 3.69 5.99 2.92 0.065 [ -0.77, 9.96 ] [ 6.90, 19.16 ] [ -3.22, 6.25 ] [ 0.028, 0.079 ] [ 0.00, 7.37 ] [ 3.23, 9.22 ] [ -0.34, 6.31 ] [ 0.042, 0.089 ] HSGM 18.39 20.31 15.06 0.053 17.97 23.96 12.53 0.081 [ 11.49, 25.29 ] [ 13.41, 26.82 ] [ 9.75, 20.25 ] [ 0.021, 0.084 ] [ 11.52, 24.42 ] [ 17.51, 30.41 ] [ 7.93, 17.19 ] [ 0.048, 0.114 ] Table 6: Observed differences Δ=+PACE−Alone =+PACE-Alone under the main-paper protocol. The first and second lines for each method report Δ and its percentile paired-bootstrap 95% CI, respectively, using 10,000 episode-aligned resamples. Appendix E Real-World Robot Deployment Details Table 7 summarizes the real-world hardware and computation assignment. The Unitree Go2 Edu carries a RealSense D435i and a Livox Mid-360, while its Jetson Orin NX runs sensing, motion control, and data-recording services. Learned-model inference is performed offboard: PACE runs on an RTX 3070 Ti laptop, which reaches the same self-hosted Qwen-3.6-27B-FP8 deployment used in simulation. As in simulation, the high-level VLM is served locally on our own workstation rather than by a third-party service; the laptop only acts as a client and reaches that self-hosted endpoint over HTTPS within a private network. Component Configuration / role Robot Unitree Go2 Edu Onboard compute Jetson Orin NX (16 GB); sensing and motion RGB-D sensor RealSense D435i; 848×480848×480 at 30 Hz LiDAR Livox Mid-360; offline mapping only Offboard runtime RTX 3070 Ti laptop; PACE and VLM client High-level VLM self-hosted Qwen-3.6-27B-FP8; HTTPS client Robot link Authenticated HTTP over a private network Table 7: Hardware and computation assignment for real-world deployment. The offboard runtime requests fresh RGB-D observations and submits one discrete action at a time to the NX through authenticated HTTP. FORWARD is mapped to a 0.25m0.25\,m relative target and tracked from sport-mode odometry using velocity-reference MPC at 10 Hz with a 15-step prediction horizon. LEFT and RIGHT use odometry-closed-loop 30∘30 rotations. Velocity commands are sent at 50 Hz, and each primitive is allocated a 2 s execution budget. Unitree’s built-in obstacle avoidance is enabled on level ground and disabled during stair traversal to avoid altering the commanded stair motion. Together with 0.2–0.5 s VLM or approximately 0.2 s PACE inference, the control cycle is 2.2–2.5 s under the reported deployment protocol. The Mid-360 measurements are processed offline for SLAM and trajectory visualization and are not used for planning or as input to PACE. In the unseen two-floor stairwell, the main observed failure modes were left–right oscillation, reduced railing clearance, and a premature handoff before the stair exit was actually reached. In the last case, the navigator received the fresh observation, the VLM reassessed the instruction as incomplete, and PACE was reactivated through the same detection and direction gate, completing the traversal. These observations expose remaining sim-to-real sensitivity to viewpoint shifts and contact perturbations. Figure 5 complements the stairwell case with two additional unseen indoor navigation examples; their point-cloud maps are visualization outputs only. Figure 5: Additional real-world navigation examples. (a) “Walk forward to the end, turn right, and continue straight until you reach the trash bin.” (b) “Walk forward through the first doorway, turn right at the second doorway, and stop in front of the elevator.” The point-cloud maps visualize the executed trajectories and are not used for planning or as input to PACE. Appendix F Failure Case Analysis Protocol. We pool the 261 R2R-CE-CF and 217 RxR-CE-CF val-unseen episodes, yielding 478 episode-aligned evaluations per baseline under the main-paper protocol. Failed episodes are assigned to four mutually exclusive categories in priority order: stair transition when the aligned PACE run invokes stair takeover on that episode, goal stopping when OSR=1OSR=1 but SR=0SR=0, route deviation when NDTW<0.30NDTW<0.30, and local execution for the remaining failures. Each segment is normalized by all 478 episodes rather than only the failed ones, so the segments of one bar sum to that baseline’s overall failure rate. Two properties of this decomposition should be noted. First, the stair-transition category is defined by the takeover decision of the aligned PACE run, so it identifies episodes whose failure coincides with a stair transition rather than establishing that the stairs caused the failure. Second, the priority order assigns an episode that both fails at a stair transition and stops away from the goal to the stair-transition category; the categories are therefore mutually exclusive by construction but not independent. Results. As shown in Fig. 6, PACE reduces the average overall failure rate from 88.9%88.9\% to 79.4%79.4\% (−9.5-9.5 percentage points) across all six baselines. The average stair-transition failure rate decreases more substantially, from 34.2%34.2\% to 23.8%23.8\% (−10.4-10.4 points, or 30.4%30.4\% relatively). HSGM exhibits the largest improvement, with stair-related failures decreasing from 47.9%47.9\% to 26.4%26.4\%. Since the stair-transition reduction of 10.410.4 points exceeds the overall reduction of 9.59.5 points, the aggregate gain is attributable to the stair-transition category, with the remaining categories contributing no net reduction. This is consistent with PACE acting locally at constrained transitions rather than improving global instruction following, and with the smaller NDTW gains reported in the main paper. Figure 6: Failure-rate decomposition across six navigation baselines on the 478 episode-aligned val-unseen episodes. Segment lengths are normalized by all episodes, such that each bar represents the overall failure rate. Blue highlights stair-transition failures. Qualitative success cases. Figure 7 illustrates representative successful trajectories across all six navigators. In the same scene, PACE enables 3-step-Nav, Open-Nav, and STRIDER to complete the cross-floor route despite their different perception, memory, and planning mechanisms. The remaining examples further demonstrate robustness to takeover timing: PACE can guide the agent through the transition when activated either after it has entered the staircase, as in VLN-Zero, or near the stair entrance, as in HSGM. Figure 7: Representative successful navigation cases with PACE. The top row compares 3-step-Nav, Open-Nav, and STRIDER in the same scene, showing that PACE supports navigators with different internal mechanisms. The bottom row presents additional cases across CA-Nav, VLN-Zero, and HSGM. In particular, PACE completes the traversal when activated either on the staircase (VLN-Zero) or near the stair entrance (HSGM). Blue and orange denote the PACE-executed and expert trajectories, respectively. Failure cases. Figure 8 illustrates several remaining failure modes. For CA-Nav, PACE fails to exit and return control, exhausts its takeover budget, and drives the agent away from the target, constituting a harmful takeover. With 3-step-Nav, repeated collisions lead to persistent turning without effective recovery. For Open-Nav and STRIDER, the underlying navigators fail to identify a reasonable global direction, which cannot be corrected by a local execution module. In the HSGM case, PACE is never activated, so the failure originates from the baseline navigation process rather than PACE execution. These cases highlight limitations in exit prediction, collision recovery, upstream direction selection, and takeover triggering. Figure 8: Representative failure cases across the six navigators. Blue and orange denote the executed and expert trajectories, respectively. Failures arise from both PACE-specific execution errors and limitations of the underlying navigators.