Paper deep dive
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding
Tianhui Cai, Xinglong Sun, Chao Fang, Zhenxin Li, Rui Song, Jose M. Alvarez, Yunxiang Mao, Jiaqi Ma, Langechuan Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 10/9/2026, 8:14:56 AM
Summary
The paper introduces AffordDrive3D, an affordance- and geometry-aware world-action model for autonomous driving. It jointly learns future action-relevant regions (drivable areas and collision-critical regions) and spatial structure (depth) using a VLM backbone. The model conditions future depth prediction on RGB world-model latents and achieves state-of-the-art performance on the NAVSIM benchmark with 91.3 PDMS and 89.9 EPDMS.
Entities (10)
Relation Signals (7)
AffordDrive3D → achievesstateofthearton → NAVSIM
confidence 95% · On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS
AffordDrive3D → predicts → drivable areas
confidence 90% · forecast drivable areas and collision-critical regions
AffordDrive3D → predicts → collision-critical regions
confidence 90% · forecast drivable areas and collision-critical regions
AffordDrive3D → predicts → future geometry
confidence 90% · predicting future geometry from RGB world-model latents
AffordDrive3D → usesbackbone → VLM
confidence 90% · we build AffordDrive3D on a VLM backbone
AffordDrive3D → conditions → future geometry
confidence 85% · condition depth prediction on the future RGB latents
AffordDrive3D → outperforms → ExploreVLA
confidence 85% · Compared with ExploreVLA... AffordDrive3D improves PDMS by 0.9 points and EPDMS by 1.1 points
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle's action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.
Tags
Links
- Source: https://arxiv.org/abs/2610.11060v1
- Canonical: https://arxiv.org/abs/2610.11060v1
Trouble viewing inline? Open PDF directly →
Full Text
61,207 characters extracted from source content.
Expand or collapse full text
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding Tianhui Cai Xinglong Sun Chao Fang Zhenxin Li Affiliation: University of California, Los Angeles NVIDIA 42dot by Hyundai Fudan University Rui Song Jose M. Alvarez Yunxiang Mao Jiaqi Ma Langechuan Liu Abstract World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle’s action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning. †footnotetext: ∗Equal contribution. †Corresponding author. ‡Work done during an internship at NVIDIA. 1 Introduction Autonomous driving has increasingly shifted toward end-to-end models that directly map sensor observations to future trajectories, allowing scene representations and driving policies to be jointly optimized for motion planning (Hu et al., 2023; Jiang et al., 2023; Liao et al., 2025; Li et al., 2025d; Li et al., 2025c; Yao et al., 2026b; Li et al., 2026c). More recently, advances in vision-language models (VLMs) have demonstrated strong semantic understanding and reasoning capabilities for complex driving scenes. Vision-language-action (VLA) models further incorporate action prediction into this framework, enabling a unified model to interpret the driving scene, reason about its context, and generate future ego actions (Zhou et al., 2025; Zhou et al., 2026a; Renz et al., 2025; Fu et al., 2025). However, driving decisions depend not only on understanding the current observation, but also on anticipating how the environment may evolve. Motivated by this need, world-action models (WAMs) further couple action learning with future-scene prediction, allowing a shared representation to jointly model how the environment evolves and how the ego vehicle should act (Zhang et al., 2026a; Li et al., 2025b; Hong et al., 2026). One line of work extends VLA models with explicit world modeling, introducing future prediction as additional supervision for action learning (Li et al., 2026b; Wang et al., 2026a; Liu et al., 2026). Another line builds on pretrained video foundation models, adapting either learned temporal representations or generative video priors for joint future prediction and action learning (Shi et al., 2026; Zhao et al., 2026b; Wang et al., 2026d). By supervising the model with future observations in addition to expert trajectories, WAMs encourage the policy to capture scene dynamics that are relevant to future decision making. Most existing WAMs model the future primarily through RGB appearance or visual features (Wang et al., 2024; Zhao et al., 2026a; Chen et al., 2025; Li et al., 2025a). While such prediction provides rich supervision for scene evolution, appearance alone does not explicitly represent the spatial structure needed for driving. Compared with RGB appearance, geometry provides a more direct representation of scene structure, explicitly capturing relative distance and spatial layout that must otherwise be inferred from visual appearance. Recent approaches have therefore begun to incorporate geometric prediction into future modeling, such as depth (Sheng et al., 2026; Zhang et al., 2026b), point cloud (Lu et al., 2026) and 3D occupancy (Yang et al., 2025; Cheng et al., 2026). However, geometric future modeling remains relatively underexplored in VLA-based driving frameworks (Sheng et al., 2026). Geometric future modeling provides explicit spatial structure that is missing from appearance-based prediction, yet a key challenge remains: how to use future spatial information to improve policy learning and action prediction. In autonomous driving, this requires identifying which regions of the future scene are relevant to ego motion, such as drivable areas and regions that may pose collision risks. Without capturing their relevance to action, future world state prediction remains insufficiently aligned with the planning objective. Figure 1: AffordDrive3D vs. RGB-based world-action models. Existing RGB-based world-action models primarily capture future scene evolution through appearance prediction. AffordDrive3D additionally predicts future geometry to model how spatial structure evolves, alongside driving affordances that identify spatial regions critical to the ego vehicle’s motion. Together, these predictions align future world state modeling more closely with planning objectives and improve trajectory prediction. Therefore, we propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure as shown in Fig. 1. For geometric modeling, we condition depth prediction on the future RGB latents produced by the world model. Because these latents already encode how the scene changes over time, the geometry branch does not need to independently predict future dynamics and can instead focus on recovering the corresponding 3D structure, while keeping the RGB and geometric predictions grounded in the same future scene. Beyond spatial structure, we further model which regions of the scene are important for driving. We refer to these action-relevant regions as driving affordances, including drivable areas and collision-critical regions, and introduce temporal affordance forecasting to predict them at both current and future frames. To provide the contextual scene understanding needed for affordance prediction, we adopt a VLM backbone and leverage its strong semantic understanding and reasoning capabilities. We jointly learn affordance prediction, future RGB and depth modeling, and trajectory generation through a shared representation. Compared with ExploreVLA (Sheng et al., 2026), which also incorporates future depth prediction, AffordDrive3D improves PDMS by 0.9 points and EPDMS by 1.1 points on the NAVSIM benchmark (Dauner et al., 2024), achieving new state-of-the-art (SOTA) scores of 91.3 PDMS and 89.9 EPDMS. Our main contributions are: • We propose AffordDrive3D, an affordance- and geometry-aware WAM built on a VLA framework, which extends future world modeling beyond visual appearance to better align it with the planning objective. • We jointly model future geometry and temporal driving affordances: RGB-conditioned future depth captures the spatial structure of the predicted scene, while affordance forecasting identifies drivable areas and collision-critical regions, connecting future spatial information to ego motion. • We achieve new SOTA performance with 91.3 PDMS and 89.9 EPDMS on the NAVSIM benchmark. Ablation studies further demonstrate the effectiveness of temporal affordance forecasting and future geometry modeling for trajectory planning. 2 Related Work 2.1 VLA and WAM for Autonomous Driving VLMs have increasingly been adopted for autonomous driving to integrate perception, reasoning, and decision-making. DriveLM (Sima et al., 2024) introduces graph-structured visual question answering, while DriveVLM (Tian et al., 2024) integrates multimodal reasoning with planning. Recent VLA frameworks, including AutoVLA (Zhou et al., 2025), HybridDriveVLA (Bassole et al., 2026), and ColaVLA (Peng et al., 2026), further connect multimodal reasoning with trajectory generation through action tokenization or visual or latent reasoning. However, reasoning over observed scenes alone does not explicitly model how the environment evolves over time. World models address this limitation by predicting future observations or structured scene states. Drive-WM (Wang et al., 2024) explores action-conditioned video generation, while Drive-OccWorld (Yang et al., 2025) predicts future occupancy and flow. More recently, WAMs jointly model future dynamics and actions: DriveWAM (Shi et al., 2026) adapts video generative priors to joint video-action generation, WA-JEPA (Wang et al., 2026d) learns latent dynamics with ego trajectories, and SimWAM (Zhao et al., 2026b) as well as SV-WAM (Wang et al., 2026b) use video prediction as training-time supervision. These approaches improve predictive action learning, but often rely on RGB, latent, or generic geometric dynamics without explicitly structuring future states around action-relevant spatial constraints. 2.2 Geometry-Aware Driving Policies Explicit geometry provides spatial information complementary to images. Drive-OccWorld (Yang et al., 2025) models future occupancy and flow, GenieDrive (Yang et al., 2026) uses 4D occupancy to guide video generation, and SpaceDrive (Li et al., 2026a) incorporates depth-derived 3D information into VLM-based driving. ExploreVLA (Sheng et al., 2026) predicts future RGB and depth in parallel for trajectory planning. Recent work also explores geometric dynamics directly within world-action modeling. In particular, GeoWAM (Lu et al., 2026) directly forecasts future point-cloud geometry for action learning. In contrast, our model predicts future depth conditioned on RGB latents from the diffusion world model and jointly learns RGB, depth, affordance, and trajectory objectives, coupling anticipated appearance with geometric structure. Still, geometry alone does not explicitly indicate which regions are drivable or collision-critical for ego motion. 2.3 Affordance-Aware Planning Affordances provide a task-relevant abstraction between perception and action by emphasizing regions and interactions that directly influence control. In robotics, VLAs use task-conditioned or structured affordances to bridge perception and action (Wang et al., 2026c; Yu et al., 2026). Affordance-aware learning remains less explored in autonomous driving. Recent methods incorporate structured future priors or drivable-area constraints for planning (Zeng et al., 2026; Wang et al., 2026b). Rather than relying on future visual reasoning, structured scene priors, or trajectory-level constraints, we explicitly forecast current and future drivable and collision-critical regions, which provides temporally structured supervision for how action-relevant space evolves over time. 3 Method Figure 2: Framework of AffordDrive3D. Multi-view images, the navigation command, and the ego state are encoded by a shared VLM backbone together with learnable affordance, RGB world, geometry, and action tokens. The auxiliary branches (dashed box) forecast driving affordances, future RGB, and RGB-conditioned depth to shape the shared representation during training, while only the action branch is used at inference. 3.1 Overview As shown in Fig. 2, given the current multi-view observations ℐt=Itcc=1CI_t=\I_t^c\_c=1^C, a navigation command g, and the ego state s, AffordDrive3D predicts a future ego trajectory τ^t:t+K τ_t:t+K while jointly modeling driving affordances, scene appearance, and geometry across the current and future scene. We use a shared causal VLM backbone to encode the inputs together with four groups of learnable tokens: affordance tokens affQ^aff, RGB world tokens rgbQ^rgb, geometry tokens geoQ^geo, and action tokens actQ^act. affQ^aff model current and future drivable and collision-critical regions, rgbQ^rgb condition the diffusion world model to forecast future visual appearance, and geoQ^geo predict current and future scene geometry with guidance from the predicted RGB latent. Finally, the action tokens actQ^act condition an action diffusion model to generate the future ego trajectory. All components are optimized jointly through the shared backbone, allowing affordance forecasting, visual prediction, and geometry modeling to shape the representation used for planning. During inference, only the action branch is required to generate the ego trajectory. The auxiliary affordance, RGB, and depth outputs do not need to be decoded. 3.2 RGB-Conditioned Future Geometry Modeling Future RGB modeling. The RGB world tokens rgbQ^rgb produce hidden states rgbH^rgb after the shared causal backbone, which condition a video diffusion model to predict future scene appearance in latent space. We encode the current and future RGB frames using a video VAE, obtaining a sequence of T temporal latent groups rgb=[0rgb,…,T−1rgb]Z^rgb=[Z^rgb_0,…,Z^rgb_T-1] after temporal compression, where 0rgbZ^rgb_0 corresponds to the current-frame anchor and rgb1:T−1Z^rgb_1:T-1 represents the future sequence. Following flow-matching, we perturb the full video latent, including the anchor group, as σrgb=(1−σ)rgb+σϵ,Z^rgb_σ=(1-σ)Z^rgb+σ ε, (1) where ϵ ε is sampled noise and σ denotes the noise level. The video diffusion model is set up as an inpainting model, where the current visual observation is additionally encoded as a conditioning signal 0C_0. The model predicts the velocity field as θ=rgb(σrgb,σ,rgb,0),v_θ=W_rgb (Z^rgb_σ,σ;H^rgb,C_0 ), (2) where rgbH^rgb provides the VLM context for future generation, while 0C_0 provides the observed current-frame appearance. The RGB world model is trained with the flow-matching objective ℒrgb=‖θ−(ϵ−rgb)‖22.L_rgb= \|v_θ- ( ε-Z^rgb ) \|_2^2. (3) From the predicted velocity, we recover the predicted clean RGB latent as ^rgb=σrgb−σθ, Z^rgb=Z^rgb_σ- _θ, (4) which is subsequently used as future-scene context for geometry prediction. RGB-conditioned geometry prediction. The RGB world model already predicts how the visual scene evolves over future frames. We use the denoised RGB latent ^rgb Z^rgb as future-scene context for geometry prediction (Liang et al., 2025), allowing the geometry branch to focus on recovering the 3D structure associated with the predicted visual evolution. We first map the geometry-token hidden states geoH^geo to a coarse temporal-spatial feature grid geo∈ℝTg×Hg×Wg×DG^geo ^T_g× H_g× W_g× D, where TgT_g, HgH_g, and WgW_g denote the temporal, height, and width dimensions of the grid, respectively, with TgHgWg=|geo|T_gH_gW_g=|Q^geo|. We then use trilinear interpolation to upsample the coarse geometry feature grid geoG^geo to the same temporal-spatial resolution (T,H,W)(T,H,W) as the predicted RGB latent ^rgb Z^rgb, yielding ~geo∈ℝT×H×W×D G^geo ^T× H× W× D. To condition geometry prediction on the predicted future scene, we apply a 3D convolution ϕrgb _rgb to ^rgb Z^rgb to map its feature channels to dimension D while preserving its temporal-spatial resolution. The resulting RGB features are fused with ~geo G^geo at corresponding temporal-spatial locations: geo=ℛ3D(ϕfuse([~geo;ϕrgb(sg(^rgb))])),F^geo=R_3D ( _fuse ( [ G^geo; _rgb (sg\! ( Z^rgb ) ) ] ) ), (5) where ϕfuse _fuse denotes a 1×1×11× 1× 1 3D convolution that fuses RGB and geometry features, and ℛ3DR_3D denotes a stack of 3D residual blocks for further feature refinement. The stop-gradient operator sg(⋅)sg(·) prevents the geometry loss from back-propagating through the RGB prediction branch. The fused representation geoF^geo retains the anchor–future temporal organization of the RGB latent space. Its anchor component 0geoF^geo_0 is formed with RGB context grounded in the observed current frame, whereas the future components geo1:T−1F^geo_1:T-1 are formed with RGB context predicted by the RGB diffusion model. This yields two distinct geometry prediction settings: recovering current geometry from observation-grounded RGB context and predicting future geometry from predicted RGB context. We therefore use separate decoders for the anchor and future depth latents: ^0dep=anc(0geo),^1:T−1dep=fut(1:T−1geo). Z^dep_0=D_anc (F^geo_0 ), Z^dep_1:T-1=D_fut (F^geo_1:T-1 ). (6) We encode the pseudo-ground-truth depth from Depth Anything 3 (Lin et al., 2025) with the video VAE to obtain the target depth latent depZ^dep, and supervise the predicted depth latent ^dep Z^dep with an ℒ1L_1 loss: ℒgeo=λancℒ1(^0dep,0dep)+ℒ1(^1:T−1dep,1:T−1dep),L_geo= _ancL_1 ( Z^dep_0,Z^dep_0 )+L_1 ( Z^dep_1:T-1,Z^dep_1:T-1 ), (7) where λanc _anc controls the relative weight of the anchor-depth loss. 3.3 Temporal Affordance Forecasting We index the affordance tokens by prediction horizon and camera view, such that aff=k,caffQ^aff=\q^aff_k,c\, where k∈0,…,Kk∈\0,…,K\ denotes the future timestamp for prediction and c∈1,…,Cc∈\1,…,C\ denotes the camera view. After processing by the shared causal backbone, these tokens produce the corresponding hidden states aff=k,caffH^aff=\h^aff_k,c\. Dense affordance prediction. Each affordance state k,caffh^aff_k,c is decoded into two spatial predictions: a drivable-area map ^k,cda Y^da_k,c and a collision-critical map ^k,ccol Y^col_k,c. Both maps are defined over the visual-token grid of camera c, so each map value corresponds to one local patch of the image. Let c,nv_c,n denote the hidden state of the n-th visual token from camera c. For each affordance type m∈da,colm∈\da,col\, we compute zk,c,nm=⟨fm(k,caff),gm(c,n)⟩d+biasm,z^m_k,c,n= f^m(h^aff_k,c),g^m(v_c,n) d+bias^m, (8) where fmf^m and gmg^m are learnable projections that map the affordance and visual features into a d-dimensional embedding space. The predicted affordance probability at each visual-token location is then given by Y^k,c,nm=sigmoid(zk,c,nm) Y^m_k,c,n=sigmoid(z^m_k,c,n). The visual features provide current-scene context, while the frame-specific affordance states determine how the affordance predictions evolve over time. Affordance supervision. We construct drivable-area and collision-critical targets that are directly relevant to ego planning, focusing on reachable road regions and objects that may collide with the ego vehicle. The drivable-area target Yk,c,nda∈[0,1]Y^da_k,c,n∈[0,1] measures the fraction of verifiable ground within a visual-token region that belongs to the road surface reachable by the ego vehicle, with intermediate values capturing road boundaries. To construct the collision-critical target Yk,c,ncol∈0,1Y^col_k,c,n∈\0,1\, we select plausible ego trajectories from a predefined trajectory set (Li et al., 2024) and roll them out in the scene. If any selected trajectory results in a collision with an agent or object, the visual-token region containing the collided agent or object is marked as collision-critical. We provide details of the label construction in Appendix A. For drivable-area prediction, we use weighted soft binary cross-entropy, ℒdak=WBCE(kda,kda,kda),L^k_da=WBCE (z^da_k,Y^da_k;w^da_k ), (9) where kdaw^da_k down-weights tokens with limited ground evidence, such as those dominated by sky or buildings, and gives higher weight to tokens that contain drivable-area boundaries. In this way, the loss focuses more on reliable ground regions while preserving the fine-grained boundary information encoded by the soft targets. For collision-critical prediction, we combine class-normalized BCE to address the class imbalance between positive and negative tokens with Dice loss to encourage spatial overlap between the predicted and ground-truth critical regions, ℒcolk=BCE(kcol,kcol)+λdiceDice(^kcol,kcol).L^k_col=BCE (z^col_k,Y^col_k )+ _diceDice ( Y^col_k,Y^col_k ). (10) Similar to geometry prediction, we separate the affordance of the observed current frame from that of the forecast future frames and aggregate the losses as ℒaff=λanc(ℒda0+ℒcol0)+1K∑k=1K(ℒdak+ℒcolk).L_aff= _anc (L^0_da+L^0_col )+ 1K _k=1^K (L^k_da+L^k_col ). (11) 3.4 Action Generation and Joint Learning We use the hidden states of the action tokens actH^act to condition a DiT for future trajectory generation. The DiT is trained with a flow-matching objective ℒactL_act to predict the velocity field between Gaussian noise and the ground-truth trajectory. To encourage the RGB and geometry tokens to preserve action-relevant information while modeling future appearance and spatial structure, we additionally apply trajectory supervision to both the RGB and geometry tokens. Specifically, rgbH^rgb and geoH^geo are separately processed by lightweight GRU-based trajectory heads to predict the future ego trajectory. We supervise these predictions with the ground-truth trajectory using an ℒ1L_1 loss, denoted as ℒrgb-actL_rgb -act and ℒgeo-actL_geo -act, respectively. The complete framework is trained end-to-end with ℒ=λaffℒaff+λrgbℒrgb+λgeoℒgeo+λactℒact+λrgb-actℒrgb-act+λgeo-actℒgeo-act,L= _affL_aff+ _rgbL_rgb+ _geoL_geo+ _actL_act+ _rgb -actL_rgb -act+ _geo -actL_geo -act, (12) where λ denotes the loss weights. 4 Experiments 4.1 Experimental Setup Dataset and Metrics. We train and evaluate AffordDrive3D on the NAVSIM benchmark (Dauner et al., 2024), which is built on OpenScene and provides approximately 103K training samples in navtrain and 12K evaluation samples in navtest. We report both the PDM Score (PDMS) from NAVSIM v1 and the Extended PDM Score (EPDMS) from NAVSIM v2. NAVSIM v1 PDMS combines No At-Fault Collisions (NC), Drivable Area Compliance (DAC), Ego Progress (EP), Time-to-Collision (TTC), and Comfort (Comf.). NAVSIM v2 EPDMS further includes Driving Direction Compliance (DDC), Traffic Light Compliance (TLC), Lane Keeping (LK), History Comfort (HC), and Extended Comfort (EC). We follow the official NAVSIM evaluation protocol for both metrics. For temporal affordance prediction, we additionally report average precision (AP). For drivable-area prediction, regions with a ground-truth drivable ratio greater than 0.50.5 are treated as drivable. Further details on the AP computation are provided in Appendix D. Training and Implementation Details. We use Qwen3-VL-2B (Bai et al., 2025) as the VLM backbone and instantiate the RGB diffusion world model with Wan2.1-Fun-1.3B-InP (Wan et al., 2025). We use C=3C=3 camera views and set K=8K=8 at 2 Hz, corresponding to a 4 s prediction horizon and T=3T=3 latent groups after VAE encoding. We use |aff|=C(K+1)=27|Q^aff|=C(K+1)=27 affordance tokens, |rgb|=64|Q^rgb|=64 RGB world tokens, |geo|=64|Q^geo|=64 geometry tokens, and |act|=8|Q^act|=8 action tokens. We set λact=λrgb=λrgb-act=λgeo-act=1.0 _act= _rgb= _rgb -act= _geo -act=1.0, and λgeo=λaff=0.1 _geo= _aff=0.1, with λanc=1.0 _anc=1.0 for both the geometry and affordance losses and λdice=1.0 _dice=1.0. For supervised fine-tuning (SFT), we optimize the model for 14,140 steps with a global batch size of 256 using AdamW, a peak learning rate of 5×10−55× 10^-5, 404 warmup steps, and cosine decay. Following SFT, we further optimize the action diffusion model through reinforcement learning (RL) with GRPO (Shao et al., 2024) for 4,500 steps with a learning rate of 3×10−63× 10^-6 after 50 warmup steps, while keeping the rest of the model frozen. Detailed hyperparameters and training configurations for both SFT and RL, including the RL reward and rollout settings, are provided in Appendix B. 4.2 Main Results NAVSIM v1 Results. We evaluate AffordDrive3D on the NAVSIM v1 benchmark against end-to-end, VLA and WAM baselines (Yuan et al., 2024; Li et al., 2024; Liao et al., 2025; Zhou et al., 2025; Xiong et al., 2026; Li et al., 2025b; Zhang et al., 2026a; Chen et al., 2025; Zhang et al., 2025; Zhao et al., 2026a; Li et al., 2026b; Zhou et al., 2026b; Zeng et al., 2026; Sheng et al., 2026; Cheng et al., 2026). In Table 1, AffordDrive3D achieves a PDMS of 91.3, outperforming all baselines in the comparison. In particular, it improves over ExploreVLA (Sheng et al., 2026), which also leverages RGB and depth supervision, by 0.9 PDMS. AffordDrive3D also achieves a strong DAC score of 99.0, together with NC and TTC scores of 98.6 and 96.3, while maintaining an EP score of 84.7. These results suggest that jointly modeling future driving affordances and geometric structure provides useful action-relevant spatial supervision for trajectory planning. Table 1: Comparison with baselines on the NAVSIM v1 benchmark. Best results are bolded. Method World Sup. NC ↑ DAC ↑ TTC ↑ Comf. ↑ EP ↑ PDMS ↑ DRAMA - 98.0 93.1 94.8 100.0 80.1 85.5 Hydra-MDP - 98.3 96.0 94.6 100.0 78.7 86.5 DiffusionDrive - 98.2 96.2 94.7 100.0 82.2 88.1 AutoVLA - 98.4 95.6 98.0 99.9 81.9 89.1 ReCogDrive - 97.9 97.3 94.9 100.0 87.3 90.8 WoTE BEV 98.5 96.8 94.9 99.9 81.9 88.3 ResWorld BEV 98.9 96.5 95.6 100.0 83.1 89.0 DrivingGPT RGB 98.9 90.7 94.9 95.6 79.7 82.4 Epona RGB 97.9 95.1 93.8 99.9 80.4 86.2 PWM RGB 98.6 95.9 95.4 100.0 81.8 88.1 DriveVLA-W0 RGB 98.7 99.1 95.3 99.3 83.3 90.2 DriveDreamer-Policy RGB + Current Depth 98.4 97.1 95.1 100.0 83.5 89.2 FSDrive RGB + CoT 98.2 93.8 93.3 99.9 80.1 85.1 ExploreVLA RGB + Depth 98.8 98.4 96.5 99.9 83.5 90.4 OWMDrive 3D Occupancy 98.2 97.6 94.6 99.8 87.3 90.8 Ours Aff + RGB + Depth 98.6 99.0 96.3 100.0 84.7 91.3 NAVSIM v2 Results. We further compare AffordDrive3D with baseline methods (Zou et al., 2025; Yao et al., 2026a; Li et al., 2026b; Zhou et al., 2026b; Sheng et al., 2026) on NAVSIM v2, which extends the evaluation to a broader set of driving metrics. As shown in Table 2, AffordDrive3D achieves the highest EPDMS of 89.9, improving over ExploreVLA (Sheng et al., 2026) and DriveDreamer-Policy (Zhou et al., 2026b) by 1.1 and 1.2 points, respectively. These results further demonstrate the strong planning performance of AffordDrive3D and the effectiveness of jointly modeling future driving affordances and geometric structure. Table 2: Comparison with baselines on NAVSIM v2 with extended metrics. Best results are bolded. Model NC ↑ DAC ↑ DDC ↑ TLC ↑ EP ↑ TTC ↑ LK ↑ HC ↑ EC ↑ EPDMS ↑ DiffusionDriveV2 97.7 96.6 99.2 99.8 88.9 97.2 96.0 97.8 91.0 85.5 DriveSuprim 98.4 98.6 99.6 99.8 90.5 97.8 97.0 98.3 78.6 87.1 DriveVLA-W0 98.5 99.1 98.0 99.7 86.4 98.1 93.2 97.9 58.9 86.1 DriveDreamer-Policy 98.4 97.1 99.5 99.9 87.9 97.7 97.6 98.3 79.4 88.7 ExploreVLA 98.8 96.2 99.6 99.8 87.1 98.2 97.8 98.3 86.8 88.8 Ours 98.4 98.2 99.7 99.9 87.9 97.8 97.4 98.3 80.4 89.9 Table 3: Ablation study of affordance, RGB, and geometry future modeling. For the + Geometry setting, the RGB branch provides latent context for depth prediction, but its gradients are stopped from propagating to the shared backbone. Results are reported without RL training. Setting Aff. RGB Geo. NC ↑ DAC ↑ TTC ↑ Comf. ↑ EP ↑ PDMS ↑ Action only – – – 97.9 96.4 94.3 100.0 82.6 88.1 + Affordance ✓ – – 98.3 97.1 95.1 100.0 83.5 89.2 + RGB – ✓ – 98.3 96.8 95.3 100.0 83.0 88.9 + Geometry – – ✓ 98.2 97.1 95.2 100.0 83.2 89.1 Full model ✓ ✓ ✓ 98.4 97.6 95.2 100.0 83.7 89.6 Figure 3: Qualitative results of AffordDrive3D in two challenging scenarios. (a) The narrow curvy-road scenario: AffordDrive3D forecasts the road structure and remains on-road, while both baselines deviate from the lane. (b) The leading-vehicle braking scenario: AffordDrive3D captures the potential collision risk from the leading vehicle together with RGB-consistent future geometry, maintaining a safe trajectory while both baselines collide. In all BEV visualizations, highlighted boxes show the ego footprint at the violation state obtained from the NAVSIM rollout, while other-agent boxes show their positions at t=0t=0. Table 4: Affordance prediction across horizons. t APCollisionAP_Collision APDrivable AreaAP_Drivable Area Current 0.918 0.986 0.50.5–22 s (avg.) 0.846 0.983 2.52.5–44 s (avg.) 0.732 0.971 Table 5: Current-frame affordance prediction using frozen and unfrozen VLM. APCollisionAP_Collision APDrivable AreaAP_Drivable Area Frozen VLM 0.782 0.931 Unfrozen VLM 0.912 0.985 Qualitative Results. Figure 3 visualizes the current and future RGB, affordance, and depth predictions of AffordDrive3D, together with trajectory predictions from an action-only baseline and the RGB-based world-action model DriveVLA-W0 (Li et al., 2026b). In the narrow curvy-road example in Fig. 3(a), both baselines deviate from the drivable region and go off-road, whereas AffordDrive3D remains near the lane center. The predicted drivable-area maps consistently capture the narrow drivable road structure over time, illustrating how explicit affordance forecasting can provide important cues about where the ego vehicle can safely move. In the leading-vehicle braking example in Fig. 3(b), the two baselines collide with the vehicle ahead, while AffordDrive3D maintains a safe distance. Its collision-critical prediction correctly highlights the leading vehicle across future horizons, while the depth prediction, consistent with the predicted RGB, captures the leading vehicle’s relative distance and surrounding scene geometry. These examples illustrate how jointly modeling driving affordances and geometry can provide action-relevant spatial information for challenging planning scenarios. Additional qualitative comparisons and visualizations are provided in Appendix C. 4.3 Ablation Studies Ablation of Affordance and World Modeling. We perform an ablation study to understand how affordance, RGB, and geometry modeling affect planning performance in Table 3. Since our depth prediction is conditioned on the denoised RGB latents, the geometry-only setting still retains RGB prediction to provide the required latent input. However, we stop the RGB diffusion gradients from propagating back to the shared backbone, so the RGB branch only provides the latent context required by the geometry branch and does not directly affect the shared representation used for action prediction. Compared to the action-only baseline at 88.1 PDMS, adding affordance, RGB, or geometry improves performance to 89.2, 88.9, and 89.1 PDMS, respectively. The full model achieves the best performance of 89.6 PDMS, with the highest NC, DAC, and EP, demonstrating the effectiveness of jointly modeling RGB, geometry, and affordance. Temporal Affordance Evaluation. We evaluate the predicted driving affordances across different future horizons in Table 5. We report AP averaged over the near-term prediction horizons (0.5, 1.0, 1.5, and 2.0 s) and the longer-term prediction horizons (2.5, 3.0, 3.5, and 4.0 s). AffordDrive3D achieves strong current-frame prediction, with an AP of 0.918 for collision-critical regions and 0.986 for drivable areas. Drivable-area prediction remains highly stable over time, reaching 0.983 AP over the near-term horizon and 0.971 over the longer-term horizon. Compared with drivable-area prediction, collision prediction degrades more noticeably at longer horizons, with AP decreasing from 0.846 to 0.732. This is expected because drivable areas are largely determined by relatively stable road structure, whereas future collision-critical regions depend on anticipating the motion of dynamic agents such as vehicles and pedestrians. Overall, the results show that AffordDrive3D can reliably forecast both where the ego vehicle can safely move and where potential collision risks may emerge over time. Affordance Information in the VLM Representation. We adopt a VLM backbone to provide the contextual scene understanding needed for affordance prediction. To assess whether the original VLM representation already contains information useful for identifying regions relevant to ego motion, we remove all world-modeling and action components and attach two lightweight MLP heads directly to the VLM visual tokens to predict current-frame drivable-area and collision-critical maps. We compare a frozen setting, where only the prediction heads are trained, with an unfrozen setting, where the VLM backbone is jointly optimized with the affordance objectives. As shown in Table 5, the frozen setting achieves APs of 0.782 and 0.931 for collision-critical and drivable-area prediction, respectively. In the unfrozen setting, the APs increase to 0.912 and 0.985, with a particularly large gain for collision-critical prediction. These results indicate that the original VLM representation already contains useful information for identifying driving-relevant regions, while explicit affordance supervision further enriches the backbone with affordance-relevant information. 5 Conclusion We presented AffordDrive3D, an affordance- and geometry-aware world-action model for autonomous driving. Beyond RGB-based future prediction, AffordDrive3D models future geometry to provide explicit spatial structure and forecasts temporal driving affordances, including drivable and collision-critical regions, to support trajectory planning. By jointly learning future appearance, geometry, affordance, and action within a shared VLM backbone, AffordDrive3D more closely aligns future world modeling with the planning objective. AffordDrive3D achieves 91.3 PDMS and 89.9 EPDMS on NAVSIM, while ablation and qualitative results demonstrate the effectiveness of future geometry modeling and temporal affordance forecasting for trajectory planning. References Bai et al. (2025) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §4.1. Bassole et al. (2026) Y. C. F. Bassole, S. Kim, J. Jung, and Y. Sung HybridDriveVLA: vision-language-action model with visual cot reasoning and tot evaluation for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 32421–32430. Cited by: §2.1. Chen et al. (2025) Y. Chen, Y. Wang, and Z. Zhang Drivinggpt: unifying driving world modeling and planning with multi-modal autoregressive transformers. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 26890–26900. Cited by: §1, §4.2. Cheng et al. (2026) J. Cheng, R. Song, Y. Wu, N. Zeng, X. Li, and Y. Ai OWMDrive: causality-aware end-to-end autonomous driving via 4d occupancy world model. arXiv preprint arXiv:2606.30421. Cited by: §1, §4.2. Dauner et al. (2024) D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, et al. Navsim: data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems 37, p. 28706–28719. Cited by: Appendix A, §1, §4.1. Fu et al. (2025) H. Fu, D. Zhang, Z. Zhao, J. Cui, D. Liang, C. Zhang, D. Zhang, H. Xie, B. Wang, and X. Bai Orion: a holistic end-to-end autonomous driving framework by vision-language instructed action generation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 24823–24834. Cited by: §1. Hong et al. (2026) Y. Hong, X. Zhou, Y. Li, X. Zhou, L. Liu, Y. Luo, S. Xu, L. Yang, and Z. Song DriveFuture: future-aware latent world models for autonomous driving. arXiv preprint arXiv:2605.09701. Cited by: §1. Hu et al. (2023) Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al. Planning-oriented autonomous driving. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 17853–17862. Cited by: §1. Jiang et al. (2023) B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang Vad: vectorized scene representation for efficient autonomous driving. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), p. 8306–8316. Cited by: §1. Li et al. (2026a) P. Li, Z. Zhang, D. Holtz, H. Yu, Y. Yang, Y. Lai, R. Song, A. Geiger, and A. Zell Spacedrive: infusing spatial awareness into vlm-based autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 40096–40107. Cited by: §2.2. Li et al. (2025a) Y. Li, L. Fan, J. He, Y. Wang, Y. Chen, Z. Zhang, and T. Tan Enhancing end-to-end autonomous driving with latent world model. In International Conference on Learning Representations, Vol. 2025, p. 42942–42959. Cited by: §1. Li et al. (2026b) Y. Li, S. Shang, W. Liu, B. Zhan, H. Wang, Y. Wang, Y. Chen, X. Wang, Y. An, C. Tang, et al. Drivevla-w0: world models amplify data scaling law in autonomous driving. In International Conference on Learning Representations, Vol. 2026, p. 7890–7911. Cited by: Figure 5, Figure 6, Appendix C, §1, §4.2, §4.2, §4.2. Li et al. (2025b) Y. Li, Y. Wang, Y. Liu, J. He, L. Fan, and Z. Zhang End-to-end driving with online trajectory evaluation via bev world model. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 27137–27146. Cited by: §1, §4.2. Li et al. (2026c) Z. Li, N. Chang, X. Sun, J. Chen, W. Yao, Z. Wang, M. Shen, Y. Jiang, Z. Wu, S. Lan, et al. Large discrete policy: advancing explicit behavior modeling with stochastic iterative scoring. arXiv preprint arXiv:2609.07049. Cited by: §1. Li et al. (2024) Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y. Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wu, et al. Hydra-mdp: end-to-end multimodal planning with multi-target hydra-distillation. arXiv preprint arXiv:2406.06978. Cited by: Appendix A, §3.3, §4.2. Li et al. (2025c) Z. Li, W. Yao, Z. Wang, X. Sun, J. Chen, N. Chang, M. Shen, J. Song, Z. Wu, S. Lan, et al. Ztrs: zero-imitation end-to-end autonomous driving with trajectory scoring. arXiv preprint arXiv:2510.24108. Cited by: §1. Li et al. (2025d) Z. Li, W. Yao, Z. Wang, X. Sun, J. Chen, N. Chang, M. Shen, Z. Wu, S. Lan, and J. M. Alvarez Generalized trajectory scoring for end-to-end multimodal planning. arXiv preprint arXiv:2506.06664. Cited by: §1. Liang et al. (2025) D. Liang, D. Zhang, X. Zhou, S. Tu, T. Feng, X. Li, Y. Zhang, M. Du, X. Tan, and X. Bai Unifuture: a 4d driving world model for future generation and perception. arXiv preprint arXiv:2503.13587. Cited by: §3.2. Liao et al. (2025) B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y. Zhang, Q. Zhang, et al. Diffusiondrive: truncated diffusion model for end-to-end autonomous driving. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 12037–12047. Cited by: §1, §4.2. Lin et al. (2025) H. Lin, S. Chen, J. H. Liew, D. Y. Chen, Z. Li, G. Shi, J. Feng, and B. Kang Depth anything 3: recovering the visual space from any views. arXiv preprint arXiv:2511.10647. Cited by: Appendix A, §3.2. Liu et al. (2026) L. Liu, Z. Song, C. Jia, H. Ye, X. Hao, L. Chen, et al. Driveworld-vla: unified latent-space world modeling with vision-language-action for autonomous driving. arXiv preprint arXiv:2602.06521. Cited by: §1. Lu et al. (2026) Y. Lu, X. Ye, J. Liu, P. Jacobson, J. Yao, Y. Chen, L. Merino, D. D. Kurra, M. Cai, T. Lampo, et al. GeoWAM: visual geometry world action models for autonomous driving. arXiv preprint arXiv:2608.23486. Cited by: §1, §2.2. Peng et al. (2026) Q. Peng, X. Chen, C. Yang, S. Shi, and H. Li Colavla: leveraging cognitive latent reasoning for hierarchical parallel trajectory planning in autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 17809–17819. Cited by: §2.1. Renz et al. (2025) K. Renz, L. Chen, E. Arani, and O. Sinavski Simlingo: vision-only closed-loop autonomous driving with language-action alignment. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 11993–12003. Cited by: §1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Appendix B, §4.1. Sheng et al. (2026) Z. Sheng, X. Ye, J. Luo, S. Chen, and L. Ren Explorevla: dense world modeling and exploration for end-to-end autonomous driving. In European Conference on Computer Vision, p. 266–284. Cited by: §1, §1, §2.2, §4.2, §4.2. Shi et al. (2026) C. Shi, J. Xu, S. Shi, K. Sheng, B. Zhang, and L. Jiang DriveWAM: video generative priors enable scalable world-action modeling for autonomous driving. arXiv preprint arXiv:2605.28544. Cited by: §1, §2.1. Sima et al. (2024) C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, J. Beißwenger, P. Luo, A. Geiger, and H. Li Drivelm: driving with graph visual question answering. In European conference on computer vision, p. 256–274. Cited by: §2.1. Tian et al. (2024) X. Tian, J. Gu, B. Li, Y. Liu, Y. Wang, Z. Zhao, K. Zhan, P. Jia, X. Lang, and H. Zhao DriveVLM: the convergence of autonomous driving and large vision-language models. In 8th Annual Conference on Robot Learning, External Links: Link Cited by: §2.1. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z. Wu, and Z. Liu Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: Appendix B, §4.1. Wang et al. (2026a) G. Wang, P. Tang, X. Ren, G. Zhao, B. Feng, and C. Ma Learning vision-language-action world models for autonomous driving. arXiv preprint arXiv:2604.09059. Cited by: §1. Wang et al. (2026b) J. Wang, S. Li, J. Wang, Z. Deng, J. Gao, Y. Zhao, L. Liu, Y. Zhao, J. Chen, H. Xu, et al. SV-wam: an efficient surround-view world-action model for end-to-end autonomous driving. arXiv preprint arXiv:2609.03602. Cited by: §2.1, §2.3. Wang et al. (2026c) R. Wang, Y. Fu, Y. Li, T. Lin, T. Qian, M. Elhoseiny, B. Zhao, Y. Fu, Y. Jiang, and X. Xue Afford-vla: action-aligned visual planning via internalized affordance. arXiv preprint arXiv:2605.24203. Cited by: §2.3. Wang et al. (2026d) X. Wang, Y. Xiang, Y. Zhou, J. Wang, M. Huang, J. Huang, D. Wei, T. Zhou, X. Wang, G. Chen, et al. WA-jepa: rethinking the video jepa paradigm for world-action modeling in autonomous driving. arXiv preprint arXiv:2608.20974. Cited by: §1, §2.1. Wang et al. (2024) Y. Wang, J. He, L. Fan, H. Li, Y. Chen, and Z. Zhang Driving into the future: multiview visual forecasting and planning with world model for autonomous driving. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14749–14759. Cited by: §1, §2.1. Xiong et al. (2026) K. Xiong, X. Guo, F. Li, S. Yan, G. Xu, L. Zhou, L. Chen, H. Sun, B. Wang, K. Ma, et al. Recogdrive: a reinforced cognitive framework for end-to-end autonomous driving. In International Conference on Learning Representations, Vol. 2026, p. 157518–157556. Cited by: §4.2. Yang et al. (2025) Y. Yang, J. Mei, Y. Ma, S. Du, W. Chen, Y. Qian, Y. Feng, and Y. Liu Driving in the occupancy world: vision-centric 4d occupancy forecasting and planning via world models for autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 9327–9335. Cited by: §1, §2.1, §2.2. Yang et al. (2026) Z. Yang, Z. Liu, Y. Lu, L. Hou, C. Miao, S. Peng, B. Feng, X. Bai, and H. Zhao Geniedrive: towards physics-aware driving world model with 4d occupancy guided video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 35680–35690. Cited by: §2.2. Yao et al. (2026a) W. Yao, Z. Li, S. Lan, Z. Wang, X. Sun, J. M. Alvarez, and Z. Wu Drivesuprim: towards precise trajectory selection for end-to-end planning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 11910–11918. Cited by: §4.2. Yao et al. (2026b) W. Yao, X. Sun, Z. Li, S. Lan, Z. Wang, J. M. Alvarez, and Z. Wu HAD: combining hierarchical diffusion with metric-decoupled rl for end-to-end driving. arXiv preprint arXiv:2604.03581. Cited by: §1. Yu et al. (2026) Q. Yu, J. You, Y. Wang, J. Liang, B. Ping, Y. Tian, Y. Chen, M. Cai, Z. Gong, R. Wu, et al. AffordanceVLA: a vision-language-action model empowering action generation through affordance-aware understanding. arXiv preprint arXiv:2606.06155. Cited by: §2.3. Yuan et al. (2024) C. Yuan, Z. Zhang, J. Sun, S. Sun, Z. Huang, C. D. W. Lee, D. Li, Y. Han, A. Wong, K. P. Tee, et al. Drama: an efficient end-to-end motion planner for autonomous driving with mamba. arXiv preprint arXiv:2408.03601. Cited by: §4.2. Zeng et al. (2026) S. Zeng, X. Chang, M. Xie, X. Liu, Y. Bai, Z. Pan, M. Xu, and X. Wei Futuresightdrive: thinking visually with spatio-temporal cot for autonomous driving. Advances in Neural Information Processing Systems 38, p. 67299–67318. Cited by: §2.3, §4.2. Zhang et al. (2026a) J. Zhang, Z. Fu, Z. Xu, W. Dai, Q. Liu, and Y. Wang Resworld: temporal residual world model for end-to-end autonomous driving. arXiv preprint arXiv:2602.10884. Cited by: §1, §4.2. Zhang et al. (2025) K. Zhang, Z. Tang, X. Hu, X. Pan, X. Guo, Y. Liu, J. Huang, L. Yuan, Q. Zhang, X. Long, et al. Epona: autoregressive diffusion world model for autonomous driving. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 27220–27230. Cited by: §4.2. Zhang et al. (2026b) S. Zhang, J. Tian, H. Li, D. Liu, H. Chen, W. Huang, F. Li, G. Chen, H. Ye, L. Chen, et al. GeoWorldAD: geometry world action model for autonomous driving. arXiv preprint arXiv:2607.17521. Cited by: §1. Zhao et al. (2026a) Z. Zhao, T. Fu, Y. Wang, L. Wang, and H. Lu From forecasting to planning: policy world model for collaborative state-action prediction. Advances in Neural Information Processing Systems 38, p. 134585–134611. Cited by: §1, §4.2. Zhao et al. (2026b) Z. Zhao, X. Zhou, T. Xu, Z. Sun, K. Zhou, H. Li, D. Liang, and X. Bai SimWAM: a simple world action model for end-to-end autonomous driving. arXiv preprint arXiv:2608.07468. Cited by: §1, §2.1. Zhou et al. (2026a) X. Zhou, X. Han, F. Yang, Y. Ma, V. Tresp, and A. Knoll Opendrivevla: towards end-to-end autonomous driving with large vision language action model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 13782–13790. Cited by: §1. Zhou et al. (2026b) Y. Zhou, X. Wang, H. Shao, L. Wang, G. Zhao, J. Shao, J. Zhu, T. Yu, Z. Zhu, G. Huang, et al. Drivedreamer-policy: a geometry-grounded world-action model for unified generation and planning. arXiv preprint arXiv:2604.01765. Cited by: Appendix C, §4.2, §4.2. Zhou et al. (2025) Z. Zhou, T. Cai, S. Z. Zhao, Y. Zhang, Z. Huang, B. Zhou, and J. Ma AutoVLA: a vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. Advances in Neural Information Processing Systems (NeurIPS). Cited by: §1, §2.1, §4.2. Zou et al. (2025) J. Zou, S. Chen, B. Liao, Z. Zheng, Y. Song, L. Zhang, Q. Zhang, W. Liu, and X. Wang Diffusiondrivev2: reinforcement learning-constrained truncated diffusion modeling in end-to-end autonomous driving. arXiv preprint arXiv:2512.07745. Cited by: §4.2. Appendix Appendix A Affordance Label Construction We construct both labels directly on the visual-token grid used by the affordance heads. Each of the C=3C=3 camera images is resized to 1024×5761024× 576 and represented by an 18×3218× 32 grid of merged Qwen3-VL tokens, where each token corresponds to a 32×3232× 32 pixel patch. We generate labels at tk=0.5kt_k=0.5k s for k=0,…,8k=0,…,8. Figure 4 shows an example of affordance labels. Drivable area. For each timestamp, we first identify the road surface that is lawfully reachable from the ego vehicle. We seed the NAVSIM roadblock graph with the roadblocks containing the ground-truth ego pose, follow directed outgoing connections for up to six hops, and retain roadblock polygons within 80 m in front of the ego vehicle. The resulting region includes all lanes in the ego vehicle’s direction of travel and all lawful branches it can take at intersections, while excluding sidewalks and lanes for oncoming traffic. We then determine which image pixels provide reliable evidence about the ground. Using the camera calibration, each pixel ray is intersected with the estimated ground plane and retained only if the intersection is within 80 m. Depth Anything 3 (Lin et al., 2025) is then used to determine whether the ray is actually observing the ground. If the predicted depth is less than 70%70\% of the ground-intersection distance, the pixel is treated as a non-ground observation; otherwise, it is retained as ground evidence. The ground-plane height is estimated independently for each frame from the predicted depth. Ground behind annotated vehicles is retained so that the drivable-area label describes the underlying road surface rather than the temporary occupancy of that surface. The drivable-area target Yk,c,ndaY^da_k,c,n is the fraction of the retained ground pixels in token n that project inside the reachable road region. It is 11 when all retained ground evidence lies within the reachable road, 00 for verified non-drivable ground, and a soft value between 00 and 11 for tokens crossing a road boundary. Tokens for which fewer than 15%15\% of the pixels provide ground evidence, typically corresponding to sky, buildings, or other non-ground regions, are treated as unknown: their training target is set to 00, and their contribution to the loss is reduced. Tokens receive full loss weight when ground evidence covers at least 30%30\% of their pixels. Below this threshold, the weight decreases linearly with the ground-evidence fraction but is clipped to a minimum of 0.10.1. The weight is further multiplied by 33 for boundary tokens with 0<Yk,c,nda<10<Y^da_k,c,n<1. Frames for which no reachable road polygon can be constructed receive zero weight. Collision-critical regions. The collision-critical label identifies objects that could cause a collision under a plausible ego motion, rather than marking every object in the scene. We begin with the predefined vocabulary of 16,384 ego trajectories from Hydra-MDP (Li et al., 2024), each spanning 4 s at 10 Hz. A trajectory is considered geometrically plausible only if all 40 poses remain inside the route-lane corridor expanded by 0.25 m and its heading stays within 60∘60 of the nearest route-lane direction. We then retain trajectories with NAVSIM Driving Direction Compliance (DDC) equal to 11. In the rare case that geometrically plausible trajectories exist but none satisfies this DDC condition, we use the geometrically plausible set as a fallback; scenes with no geometrically plausible trajectory simply have no positive collision-critical label. Each retained trajectory is evaluated using the NAVSIM v1 simulator, which applies an LQR controller and a kinematic bicycle model to the ego vehicle while other agents replay their logged motion (Dauner et al., 2024). We define an object as collision-relevant if at least one retained trajectory incurs an at-fault collision with it under the NAVSIM v1 rules: a frontal collision, a collision with a stationary object, or a lateral collision occurring while the ego vehicle is outside its lane. The set of collision-relevant objects is determined once from rollouts initialized at t0t_0 and then tracked throughout the prediction horizon. At each timestamp tkt_k, we retrieve the annotated 3D bounding box of every collision-relevant object present in that frame, clip the box against the camera near plane, and project it into each camera view. A token receives Yk,c,ncol=1Y^col_k,c,n=1 if its image patch overlaps at least one projected bounding box of a collision-relevant object, and 00 otherwise. If a collision-relevant object is not annotated in a particular frame, it contributes no positive tokens at that timestamp. If the rollouts contain at-fault collisions but none of the corresponding collision-relevant objects is visible in any camera at t0t_0, the current-frame label is treated as unknown. Future timestamps at which no collision-relevant object is visible are likewise excluded from the loss and AP evaluation. Figure 4: Example of ground-truth affordance targets at t=0t=0–44 s. Appendix B Training Details Supervised fine-tuning. We train for 14,140 steps (35 epochs) with a per-device batch size of 2 on 128 NVIDIA H100 GPUs, giving a global batch size of 256. We use AdamW with gradient clipping at 1.01.0, DeepSpeed ZeRO-2, and bf16 mixed precision. A single peak learning rate of 5×10−55× 10^-5 is shared by all trainable parameter groups, with 404 warmup steps and cosine decay to 2.5×10−72.5× 10^-7. The video VAE and the CLIP image encoder of Wan2.1-Fun-1.3B-InP (Wan et al., 2025), as well as the vision encoder of the VLM backbone, remain frozen. Reinforcement learning setup. After SFT we optimize only the action DiT with GRPO (Shao et al., 2024) and keep the VLM, the world model, the learnable tokens, and all auxiliary heads frozen. We construct the RL reward following the official NAVSIM scoring metrics, using the NAVSIM simulator to score each sampled trajectory on navtrain. For NAVSIM v1, we directly use PDMS as the reward. For NAVSIM v2, the Extended Comfort (EC) term in EPDMS depends on two consecutive frames and cannot be evaluated for an isolated trajectory. We therefore define r=NC⋅DAC⋅DDC⋅TLC⋅5EP+5TTC+2LK+2HC14−λec[EC violated],r=NC·DAC·DDC·TLC· 5\,EP+5\,TTC+2\,LK+2\,HC14- _ec\,1\! [EC violated ], (13) with λec=0.2 _ec=0.2, where the EC flag is evaluated against the deterministic trajectory that the current policy produces for the preceding frame. At each step, we sample 128 scenes and G=16G=16 trajectories per scene, yielding 2,048 trajectories. We use a drift-corrected SDE flow-matching sampler with noise scale γj=η1−uj _j=η 1-u_j, which preserves the deterministic sampler’s marginals. We adapt η online (initialized at 0.30.3 and clipped to [0.1,0.6][0.1,0.6]) to maintain useful reward variation. Advantages are mean-centered per scene and clipped to ±5± 5 without standard-deviation normalization; groups with a reward range below 0.020.02 are skipped. We optimize the clipped policy-gradient loss (ϵclip=0.2 _clip=0.2) with a KL penalty to the frozen SFT policy (β=0.01β=0.01). We set gradient clipping at 1.01.0, and a learning rate of 3×10−63× 10^-6 after 50 warmup steps. For each reward, we train for 4,500 steps on 8 NVIDIA H100 GPUs. Appendix C More Visualization Results We provide additional qualitative results in Figs. 5 and 6. Figure 5 visualizes the world-modeling outputs of AffordDrive3D, including predicted future RGB, current and future depth, and drivable-area and collision-critical masks overlaid on the corresponding RGB views. Following DriveDreamer-Policy (Zhou et al., 2026b), the left, front, and right camera views are spatially concatenated and modeled jointly, allowing scene content to transition consistently across views over time. Figure 6 provides additional planning comparisons among AffordDrive3D, the RGB-based DriveVLA-W0 (Li et al., 2026b), and the action-only VLA baseline. For all BEV comparisons, the eight predicted ego poses are converted into simulated states at 10 Hz using the NAVSIM LQR tracker and kinematic bicycle model. The highlighted ego footprint is drawn at the violation state, whereas surrounding agents are rendered from their t=0t=0 annotations only for visual context. Figure 5: Additional world-modeling and planning visualizations. For each scenario, we show AffordDrive3D’s predicted future RGB, current and future depth, and current and future drivable-area and collision-critical masks overlaid on the corresponding RGB views. The left, front, and right camera views are spatially concatenated, and the right panels compare the planned trajectories of AffordDrive3D and the RGB-based DriveVLA-W0 (Li et al., 2026b). Figure 6: Additional planning comparisons. Each example shows the three input camera views and the BEV trajectories predicted by AffordDrive3D, the RGB-based DriveVLA-W0 (Li et al., 2026b), and the action-only VLA baseline. Appendix D Affordance AP Computation AP is computed independently for the collision-critical and drivable-area heads at each timestamp tkt_k on navtest, where t0t_0 denotes the current frame. For each head and timestamp, we pool the predicted probabilities of all visual tokens across scenes and the C=3C=3 cameras and compute standard non-interpolated AP directly from the ranked probabilities, without applying a prediction threshold. For collision-critical prediction, scene–timestamp pairs with unknown labels are excluded, and all tokens from the remaining pairs use their binary targets. For drivable-area prediction, all tokens are included, and the soft target is binarized as positive when Yda>0.5Y^da>0.5. Tokens with less than 15%15\% ground evidence remain negative during evaluation, consistent with their training targets.