Paper deep dive
What Matters for Scalable and Robust Learning in End-to-End Driving Planners?
David Holtz, Niklas Hanselmann, Simon Doll, Marius Cordts, Bernt Schiele
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/22/2026, 5:18:29 AM
Summary
The paper introduces BevAD, a lightweight and scalable end-to-end autonomous driving architecture that addresses the gap between open-loop performance and closed-loop robustness. By systematically analyzing architectural patternsâspecifically high-resolution perceptual representations, disentangled trajectory representations, and generative planningâthe authors identify that high-resolution BEV features often lead to causal confusion in closed-loop settings. They propose a scene tokenizer to compress BEV features and demonstrate that combining disentangled planning with diffusion-based generative modeling yields superior data-scaling and closed-loop performance on the Bench2Drive benchmark.
Entities (5)
Relation Signals (3)
BevAD â achievesperformanceon â Bench2Drive
confidence 100% · BevAD achieves 72.7% success rate on the Bench2Drive benchmark
ParaDrive â servesasfoundationfor â BevAD
confidence 90% · Our analysis framework as shown in Fig. 2 is built upon ParaDrive
BevAD â utilizes â Diffusion-based planning
confidence 90% · We show that disentangled trajectory representations and generative planning via diffusion... provide complementary benefits
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:End-to-end autonomous driving has gained significant attention for its potential to learn robust behavior in interactive scenarios and scale with data. Popular architectures often build on separate modules for perception and planning connected through latent representations, such as bird's eye view feature grids, to maintain end-to-end differentiability. This paradigm emerged mostly on open-loop datasets, with evaluation focusing not only on driving performance, but also intermediate perception tasks. Unfortunately, architectural advances that excel in open-loop often fail to translate to scalable learning of robust closed-loop driving. In this paper, we systematically re-examine the impact of common architectural patterns on closed-loop performance: (1) high-resolution perceptual representations, (2) disentangled trajectory representations, and (3) generative planning. Crucially, our analysis evaluates the combined impact of these patterns, revealing both unexpected limitations as well as underexplored synergies. Building on these insights, we introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture. BevAD achieves 72.7% success rate on the Bench2Drive benchmark and demonstrates strong data-scaling behavior using pure imitation learning. Our code and models are publicly available here: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2603.15185v1
- Canonical: https://arxiv.org/abs/2603.15185v1
Trouble viewing inline? Open PDF directly â
Full Text
81,123 characters extracted from source content.
Expand or collapse full text
What Matters for Scalable and Robust Learning in End-to-End Driving Planners? David Holtz1,2 Niklas Hanselmann1 Simon Doll1 Marius Cordts1 Bernt Schiele2 1 Mercedes-Benz AG 2 Max-Planck-Institute for Informatics, SIC Abstract End-to-end autonomous driving has gained significant attention for its potential to learn robust behavior in interactive scenarios and scale with data. Popular architectures often build on separate modules for perception and planning connected through latent representations, such as birdâs eye view feature grids, to maintain end-to-end differentiability. This paradigm emerged mostly on open-loop datasets, with evaluation focusing not only on driving performance, but also intermediate perception tasks. Unfortunately, architectural advances that excel in open-loop often fail to translate to scalable learning of robust closed-loop driving. In this paper, we systematically re-examine the impact of common architectural patterns on closed-loop performance: (1) high-resolution perceptual representations, (2) disentangled trajectory representations, and (3) generative planning. Crucially, our analysis evaluates the combined impact of these patterns, revealing both unexpected limitations as well as underexplored synergies. Building on these insights, we introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture. BevAD achieves 72.7% success rate on the Bench2Drive benchmark and demonstrates strong data-scaling behavior using pure imitation learning. Our code and models are publicly available here: https://dmholtz.github.io/bevad/ 1 Introduction (a) High-capacity perceptual representation, e.g., high-resolution BEV (b) Planning Representation (c) (Non-)generative Modeling Figure 1: Architectural Patterns. (a) High-resolution BEV features facilitate perception tasks, but promote overfitting the planner. (b) Closed-loop methods prefer path over trajectory representations due to robust steering. (c) Point-estimates interpolate between trajectory modes that diffusion-based sampling can breed. End-to-end autonomous driving (E2E-AD) has recently achieved great progress, driven by the possibility to optimize the entire stack in a planning-oriented manner [24]. Compared to classical approaches with rule-based components, this enables human-like behavior in complex scenarios and promises performance gains that scale with data. While E2E-AD exists in various flavors, popular approaches [24, 29, 12, 52] often implement modular but fully-differentiable transformer-based architectures with latent intermediate representations, such as birdâs eye view (BEV) feature grids. Popularized on open-loop benchmarks such as NuScenes [3], these works typically do not evaluate in a closed-loop setting. Unfortunately, approaches optimized for open-loop performance [24, 29, 52] often fail to generalize in closed-loop driving scenarios [9, 28], resulting in a divergence in the directions of architectural advances between works that solely focus on either setting. In this paper, we take a step towards consolidating these advances in E2E-AD with a focus on closed-loop driving. As depicted in Fig. 1 we systematically extend the design space proposed in ParaDrive [52], by re-examining three architectural patterns: (1) The use of high-resolution perceptual representations as input to the planning module, more common to the open-loop setting [35, 24], (2) disentanglement of trajectories into lateral- and longitudinal components, mainly used in closed-loop driving [25, 43] (3) the use of generative planners, previously underexplored for end-to-end closed-loop driving [36, 8, 53]. By evaluating these patterns jointly, we find that only one configuration admits robust scaling of performance. In particular, we observe that high-resolution perceptual representations, shown to enable state-of-the-art (SotA) performance in open-loop [24, 29], can be susceptible to causal confusion [10], and introduce a spatial bottleneck to mitigate this. Furthermore, we show that disentangled trajectory representations and generative planning via diffusion [53, 36], previously studied only in isolation, provide complementary benefits in modeling multi-modal behavior, and see the strongest scaling properties when using both in conjunction. Building on these insights, we develop BevAD. It achieves SotA closed-loop driving performance on the challenging Bench2Drive benchmark [28] based on the CARLA simulator [14], without any bells and whistles, and using camera sensors only. Additionally, we demonstrate that BevAD strongly benefits from data scaling, with difficult skills emerging as the dataset size increases. In summary, our core contributions are threefold, we: (1) Show that high-resolution perceptual representations can hinder learning robust planning and introduce a spatial bottleneck layer for mitigation. (2) Analyze how a disentangled planning representation and diffusion-based planning provide complementary benefits for closed-loop driving. (3) Integrate these insights and build BevAD, a lightweight and highly scalable E2E-AD architecture that achieves SotA closed-loop driving on Bench2Drive. 2 Related Work End-to-end Autonomous Driving. Traditional autonomous driving (AD) stacks integrate standalone modules through compact predefined interfaces [1]. Although offering interpretability, this approach is prone to error accumulation, information loss between modules, and conflicting optimization objectives [24]. In contrast, the E2E-AD paradigm mitigates these limitations by optimizing the entire perception-to-planning pipeline, from raw sensor data to a planning trajectory or physical control commands [24, 2]. UniAD [24] established a foundational framework for modular E2E-AD by introducing query-based mechanisms to jointly optimize perception, prediction, and planning. A diverse landscape of stack-level designs has emerged, primarily categorized along three key dimensions: (1) Task Selection. This encompasses tasks such as 3D object detection [47], multi-object tracking [24, 12], online mapping [24, 29, 47, 52], motion prediction [29, 24], occupancy prediction [24, 49], and increasingly, language-based tasks like commentary and visual question answering [43, 15]; (2) Network Topology. Information flow is governed by sequential [23], parallel [6, 52] and hybrid [24, 29, 47] module placements or by unified transformers [48]; (3) Intermediate Representation. These include sparse (e.g., instance-level) [47, 12] and dense (e.g., BEV-centric [52] or occupancy-based [49]) representations. Despite this extensive architectural landscape, our study uncovers a misconception about a subtle, yet pivotal detail within many existing designs: High-resolution intermediate representations, such as an increased number of BEV features, have demonstrated improvements in open-loop perception [35] and are posited to improve planning performance [28]. Conversely, we propose spatially compressing the perceptual representation before planner input and show substantially enhanced closed-loop driving robustness. This finding corroborates observations that other successful closed-loop driving methods often employ architectural simplifications [43, 56], such as relying solely on single front-facing cameras. We systematically shed light on the importance of the perceptual representation design and observe that 360-degree perception works if the intermediate representation is otherwise limited in bandwidth. Imitation Learning for Planning. Learning to plan with an autonomous vehicle can be broadly categorized into two paradigms, (1) imitation learning (IL) via behavior cloning and (2) reinforcement learning. Research on IL surged after pioneering work [2, 7] and with the availability of large autonomous driving datasets [3, 31] and simulators for closed-loop testing, such as CARLA [14]. Conditioning the driving policy on a signal that represents the driverâs intention in addition to environment observations has become a pivotal framework [7], allowing control over the policy at test time with navigation commands or target points. However, behavior cloning is prone to introducing undesired side-effects such as covariate shift [42] and causal confusion [10], which are hard to detect with open-loop based evaluation schemes, even with hardened metrics [9]. Data augmentation [25, 43] and world models [21, 42] help mitigate these effects, though they do not solve them entirely. Moreover, scaling up training data for IL has been shown to improve planning, following a power-law relationship in open-loop metrics, though these gains saturated in closed-loop driving [40, 54]. Many current E2E-AD solutions operate as point estimators, directly regressing the final plan [6, 56, 47] in waypoint or action spaces, or by selecting from predefined anchors [5]. Both approaches can further benefit from trajectory post-processing [34, 24]. An emerging alternative is generative models, particularly diffusion-based planners, which learn the distribution of future trajectories conditioned on the scene and a given command. At test time, they sample from this conditional distribution. Such diffusion-based planning can fit complex human driving distributions better than point estimators [53, 30], further enabling test-time guidance with driving style preferences [53]. Truncated schedules were proposed to mitigate the computational overhead from iterative denoising [36]. Despite demonstrating strong performance in open-loop benchmarks [36], diffusion-based planners have seen limited adoption in closed-loop methods. Moreover, they have not been investigated in prior data scaling studies [40, 54]. This work systematically compares point estimators and diffusion-based planners in closed-loop driving, revealing superior data scalability for diffusion-based approaches. Open-loop E2E-AD methods often generate temporal waypoint trajectories [24, 29, 36, 47, 52], thereby entangling lateral and longitudinal control. This representation can lead to sparse and ambiguous supervision, particularly for dynamic, multi-modal intersection scenarios [25]. In contrast, SotA closed-loop methods favor disentangled outputs: a future path independent of time and a target speed for longitudinal control [25, 56, 43, 48]. Both generative modeling and disentangled planning representations are patterns for learning multi-modal futures, yet their interaction remains underexplored. Our study reveals strong synergies for scalable and robust end-to-end learning. 3 Revisiting Common Architectural Patterns We briefly summarize common architectural patterns that were previously studied in isolation and occur predominantly either in the open-loop or the closed-loop setting. 1. High-Resolution Perceptual Representations are employed to improve performance in perception tasks [35], but are primarily studied in open-loop. While [28] provides some evidence for benefits in closed-loop driving, leading closed-loop methods in CARLA inherently employ lower-capacity representations due to their reduced sensor configuration. We aim to systematically re-examine the impact of the BEV size on learning precise representations for closed-loop driving. 2. Disentangled Planning Representations are employed by the top-three closed-loop methods in CARLA as a measure to reduce the ambiguity of multi-modal futures. 3. Generative Planners emerged as a principled measure to model multi-modality in driving, but their study is mainly driven by open-loop methods. In the remainder of this section, we first introduce our analysis framework (Sec. 3.1), along with the experiment setup (Sec. 3.2). Subsequently, we analyze the impact of the perceptual representation (Sec. 3.3), the patterns for modeling multi-modal future jointly (Sec. 3.4) and the implications on scaling (Sec. 3.5). (a) Pipeline Overview (b) Proposed Scene Tokenizer and Planning Head Figure 2: Analysis Framework. (a) We build our analysis framework on ParaDrive [52]. (b) We introduce a scene tokenizer to reduce the spatial resolution of the BEV features. The design of our planning head is based on a diffusion transformer [41]. Crucially, the choice of the planning queries determines whether the planner is modeled as a point estimator or by diffusion. 3.1 Analysis Framework Our analysis framework as shown in Fig. 2 is built upon ParaDrive [52] for the following key reasons: (1) ParaDrive provides a systematic, module-level architecture for E2E-AD stacks, offering a well-defined foundation; (2) Its planner operates independently from auxiliary task, facilitating a focused analysis of the perception-planner interface; (3) ParaDriveâs design, based on a real-world sensor configuration, has demonstrated strong performance on open-loop nuScenes tasks, unlike CARLA-specific methods. Our framework prioritizes training efficiency to scale beyond nuScenes [3] to larger imitation learning datasets for CARLA [56, 43]. This is achieved through a streamlined pipeline (Fig. 2(a)), optimizing the BEV backbone and removing non-essential auxiliary tasks, while maintaining a realistic sensor setup. Key aspects of the components are stated below, further details are found in the supplementary. BEV Backbone. The BEV backbone processes images of NCam=6N_Cam=6 cameras to produce BEV features â±BevF_Bev with dimensions HĂWHĂ W, comprising RADIO [19] with low-rank adapter [22] as its image backbone and a BEV encoder based on BEVFormer [35]. Significantly improved runtime is achieved by replacing the recurrent BEV feature generation with cached features streamed from short episode snippets during training [51]. Furthermore, we introduce a novel camera augmentation technique, applying a random transformation carBEV^BEVT_car to all camera extrinsics to recover from compounding errors during closed-loop inference. Auxiliary Tasks. We implement a DETR-style object decoder [4] with deformable cross-attention [55] to supervise the BEV features during training [35]. We pruned other auxiliary tasks as used in [24, 52] as they did not demonstrably improve closed-loop performance during initial tests, but introduced significant runtime overhead. Planning Head. We adopt a Transformer [50] decoder architecture as depicted in Fig. 2(b): Self-attention among planning queries PlanQ_Plan enables mutual alignment, while cross-attention to scene tokens â±SceneF_Scene allows extraction of global scene features, following [24, 52]. Subsequently, we employ a coarse-to-fine strategy using an optional deformable attention layer, refining PlanQ_Plan by sampling local, high-resolution BEV features â±BevF_Bev. Inspired by diffusion transformers [41], each multi-head attention block is enclosed by adaLN-Zero transformations, incorporating conditioning from high-level driving commands, the ego-state, and optionally the diffusion timestep. If the planner is a point estimator, the planning queries PlanQ_Plan are implemented as learnable embeddings. For diffusion-based planners, PlanQ_Plan is generated by adding Gaussian noise to the ground truth according to a diffusion schedule such as DDIM [46] and embedding the result into the transformerâs input space. By reinterpreting the planning queries as path and velocity tokens instead of trajectory tokens and adjusting supervision accordingly, we can modify the planning representation. This flexible design enables analysis across different formulations without altering the plannerâs architecture. Controller. Following [25, 43, 28], we employ two PID controllers to convert planning outputs into steering and acceleration commands. The disentangled planning representation facilitates PID controller design by allowing separate processing of path and speed [25]. To achieve the same for the trajectory representation, we fit a piecewise cubic Hermite polynomial to the temporal waypoints and interpolate at fixed distances, while speed is derived using a second-order difference quotient. This allows consistent PID controller parameters across representations, minimizing the controllerâs critical impact on closed-loop driving. 3.2 Experiment Setup Data Collection. There are currently two popular data sources for expert demonstrations in CARLA [14] for training imitation learning models: (1) Bench2Drive [28] provides a dataset of expert demonstrations collected by the privileged, RL-based Think2Drive [33] expert along with sensor data and object annotations. (2) The official CARLA leaderboard 2.0 benchmark provides specifications of long routes in CARLA with scenarios alongside, from which a dataset of expert demonstrations can be collected with the privileged, rule-based expert PDM-lite [44, 56]. Simlingo [43] and TF++ [56] split the long routes into shorter segments, each containing one scenario, and uniformly upsample routes with rare scenarios. Due to various known label bugs in the Bench2Drive dataset, we adopt the second approach. We re-collect training data for our six-camera sensor setup using the same route specifications as Simlingo and use these routes for training, unless stated otherwise. Training. We conduct all experiments on 8xA100 80GB GPUs with a total batch size of 128128 in mixed-precision (bfloat16) to balance efficiency, memory usage and stability. AdamW [38] Schedule-free [11] (learning rate: 2â42^-4; weight decay: 0.010.01) is used for optimization. Our training consists of two stages: A warm-up stage over four epochs to initialize the BEV backbone with perception supervision, followed by a second stage that adds planning supervision. For faster convergence, we freeze the BEV backbone for all second-stage experiments except for studies on data scale. Benchmark and Metrics. We perform closed-loop evaluations on the challenging Bench2Drive benchmark [28] in CARLA [14]. Bench2Drive comprises 220 short test routes, each featuring a single scenario, enabling analysis of specific driving skills. We report the official metrics driving score (DS) and success rate (SR). 3.3 High-Resolution Perceptual Representations Established BEV-based end-to-end architectures connect perception and planning through HĂWHĂ W high-resolution latent BEV features [24, 52]. We introduce a tokenizer (Fig. 2(b)) that applies masking and patchifying to compress BEV features â±BevF_Bev into scene tokens â±SceneF_Scene, thereby channeling spatial information through a bottleneck. Masking. We propose using a key padding mask in the global cross-attention of the planner to exclude BEV cells where planning queries PlanQ_Plan cannot attend to. Our initial experiments tested various masking strategies, such as removing distant parts to the left and right of the ego vehicle, and sophisticated masks based on the map segmentation outputs. No significant differences were observed, so we use the simplest form, masking out 20% of the left- and right-most BEV cells. Although tailored to CARLA maps, this approach helps to determine if restricting the attention space facilitates learning a robust representation. Patchifying. Inspired by Vision Transformers [13], we propose pixel unshuffling for combining patches of pĂpĂ p BEV features â±BevF_Bev, (pixels) into spatial scene tokens â±SceneF_Scene, which our planner can globally attend to. We prevent the channel dimension of the scene tokens from growing by p2p^2 by projecting the output of pixel unshuffling to a lower-dimensional space, thereby enforcing a bottleneck. We explore pâ1,2,4,5pâ\1,2,4,5\. This analysis aims to understand how forced compression and sequence length in cross-attention impact learning a robust representation. (a) p=1p=1, without masking (b) p=1p=1, with masking (c) p=4p=4, with masking Figure 3: Qualitative visualization of the planning queriesâ cross-attention to BEV features. Fig. 3(a). The planner attends to distant BEV cells. Despite strong attention on the traffic light, the autonomous vehicle runs the red light. Fig. 3(b): There are numerous attention spikes to random BEV cells, but barely no attention to the oncoming traffic. Fig. 3(c): The attention map significantly simplifies and exhibits fewer attention outliers. Results. We employ a 100Ă100100Ă 100 BEV space, consistent with UniAD-tiny [24, 28], and a disentangled point-estimator planner [25], aligning with SoTA on Bench2Drive [56, 43]. Tab. 1 presents open- and closed-loop driving metrics for the tokenizer design space. We observe significant improvements in closed-loop driving performance as the scene token count is reduced via masking and patchifying. Specifically, restricting the plannerâs attention to masked BEV features enhances closed-loop driving, even when the mask is applied solely at test time. Furthermore, summarizing pĂpĂ p BEV feature patches into scene tokens reduces the plannerâs token count by a factor of p2p^2, yielding substantial closed-loop performance gains. Despite the reduced BEV resolution, the L1 trajectory error marginally improves for pâ€4p†4. However, this compression strategy collapses for pâ„5pâ„ 5, resulting in a significant drop in both closed-loop and open-loop performance. Discussion. Transformer-based models are known to struggle with identifying relevant information in long (text) sequences, even with modest token counts [32]. We relate this challenge to our setting, where high-resolution BEV inputs create long attention contexts. We hypothesize that the planner overfits to spurious correlations in training data by deriving actions from memorized visual landmarks. Fig. 3 visualizes qualitative examples of planning query mean cross-attention activations. In the absence of masking and patching, it reveals numerous punctual, high activation patterns in distant, often occluded or irrelevant BEV regions, strongly indicating causal confusion. These learned shortcuts are not measurable by open-loop metrics like L1 due to averaging, but lead to catastrophic failures in distinct situations at test time. By reducing the token count through masking and patchifying, our approach mitigates this causal confusion, significantly enhancing closed-loop driving by learning a more robust representation for test time. Our finding contrasts with prior studies suggesting that higher BEV resolutions enhance downstream tasks such as 3D object detection [35]. This discrepancy stems from a fundamental difference between local detection and global planning tasks. DeformableDETR-style detection heads leverage object locality by decoding queries to specific reference points [55, 35]. While increased BEV resolution enhances localization precision, it does not expand a single queryâs receptive field. In contrast, planning requires understanding critical scene elements that may not be localized near the immediate trajectory, thus necessitating global cross-attention [24, 52]. In this global context, increasing BEV resolution expands the attention context size, contributing to the observed performance degradation. Mask Patch Size Scene Tokens DS â SR â L1 (m) â â 1 100Ă100100Ă 100 66.86 36.36 1.45 ââ 1 100Ă60100Ă 60 71.79 41.37 1.45 â 1 100Ă60100Ă 60 72.40 41.97 1.53 â 2 50Ă3050Ă 30 74.98 48.18 1.43 â 4 25Ă1525Ă 15 82.62 57.43 1.43 â 5 20Ă1220Ă 12 66.44 40.91 1.73 Table 1: Impact of Tokenizer Design. Masking: Restricting planning queriesâ attention benefits closed-loop driving. Patchifying: Aggregating pĂpĂ p BEV features into scene tokens significantly enhances closed-loop driving for pâ€4p†4. Notably, these closed-loop improvements are not reflected in the open-loop L1 trajectory error metric. Legend: â indicates test-time masking. 3.4 Modeling Multi-Modal Behavior The problem of inherent multi-modality in driving behavior is well-known in research [53, 25]. Leading closed-loop methods in CARLA address this with a disentangled output representation that separates the spatial path from the speed profile instead of entangling them in a trajectory of temporal waypoints [56, 43, 48]. Points on the path are obtained by sampling at fixed distances instead of fixed time intervals, which were shown to be less ambiguous, providing better supervision [25]. Meanwhile, diffusion models [20] can natively address the multi-modality in entangled temporal trajectories with generative modeling [53, 36]. On first glance, both patterns appear to solve a similar problem. To discern the individual contributions and potential synergies of trajectory representation and (non)generative modeling, we systematically evaluate all four combinations. As we observe that the DS and SR tend to obscure the distinct symptoms of driving failures, we additionally introduce static and dynamic infraction rates IRsIR_s and IRdIR_d for this experiment. In a nutshell, IRsIR_s and IRdIR_d capture the prevalence of failures due to wrong path planning and inappropriate acceleration respectively; details can be found in the supplementary. Model Repr. DS â SR â IRsIR_s â IRdIR_d â PE T 77.2 ±0.6± 0.6 51.7 ±0.7± 0.7 0.185 0.505 PE P+S 82.6 ±1.0± 1.0 57.4 ±0.6± 0.6 0.055 0.447 DI T 80.7 ±3.0± 3.0 56.2 ±4.7± 4.7 0.147 0.391 DI P+S 81.8 ±1.5± 1.5 59.4 ±1.0± 1.0 0.094 0.423 Table 2: Comparison of modeling and planning representation. Legend: PE: point-estimator, DI: diffusion, T: trajectory (entangled), P+S: path and speed (disentangled) representation. Results. As shown in Tab. 2 the disentangled representation significantly reduces static infractions, regardless of the modeling approach. Particularly for point-estimators, this reflects strongly in the overall closed-loop scores, matching prior studies [25]. We conclude that the disentangled representation is favorable for learning robust steering. Generative modeling with diffusion reduces dynamic infractions, regardless of trajectory representation. As a result, the entangled diffusion-based variant achieves similar overall SR than the disentangled point-estimator, though their failure modes are quite different. Further, we observe complementary benefits for employing both diffusion-based modeling and disentangled representation, stated with the highest overall SR. The lower driving score stems from -1.6% route completion, since the diffusion model is less willing to make an infraction for the sake of route progress. 3.5 Diminishing Returns when Scaling Non-Generative Planners The promise of scaling performance with data is one of the main advantages of E2E-AD. Since generative planning can capture the full distribution of behavior, we hypothesize that it shows stronger benefits from scaling the dataset size. In the following, we hence examine the scaling behavior of diffusion- compared to point estimator-based planning. Scaling Data. To scale training data beyond current datasets [43, 28], we build a route generator that exhaustively plans short semantically plausible single scenario routes in all CARLA towns. While being capable of building >106>10^6 unique route-scenario combinations (see supplementary), we only consider 8,000 uniformly sampled scenarios as additional training data in our scaling experiments. For conducting data scaling experiments, we leverage established protocols from [40, 54]: We create five training splits from the joint set of Simlingoâs [43] and our routes, each approximately doubling in size. These splits are cumulative (each being a subset of the larger ones [40]), maintaining the same scenario distribution across all splits. We train all models on each data scaling split until convergence. Figure 4: Scaling Properties. Diffusion demonstrates superior performance over point estimators when scaled with sufficient training data, despite initially underperforming with limited data. Results. As reported in Fig. 4, both variants improve monotonically in SR as we double the training dataset. In the low data regime, the point estimator slightly outperforms the diffusion-based planner. After an inflection point (about 8000 training scenes), the growth rate decelerates, matching prior studies on closed-loop scaling laws for point estimate planners [54]. On the other hand, diffusion-based planning maintains its linear rate of improvement until our largest data scaling split, and thereby substantially outperforms the point-estimator counterpart. Interestingly, we cannot observe any saturation for the diffusion-based planner, unlike [40, 54] reported for closed-loop tests with point-estimator regressors. This opens up opportunities for further improvements in the presence of larger datasets. Emerging Skills. The scaling gains can also be broken down in terms of multi-ability evaluation protocol from Bench2Drive [28], where we observe that difficult skills emerge at larger training split sizes. For example, this is the case for the Give Way and Merging skills as required for the yielding scenario depicted in Fig. 5. Detailed results can be found in the supplementary. 4 BevAD Integrating the above insights, BevAD emerges as a lightweight and highly scalable E2E-AD architecture from our analysis framework in Fig. 2. It applies synergies of architectural patterns, previously studied in isolation, for dealing with multi-modality in driving and combats overfitting with effective BEV compression. We compare BevAD to previous state-of-the-art in CARLA [14], providing a quantitative demonstration of BevADâs results (Sec. 4.1) along with qualitative results (Sec. 4.2) and real-world experiments on NAVSIM [9] (Sec. 4.3). Method Details Overall Expert Sensors Labels Driving Score â Success Rate â VAD [29] Think2Drive 6x CAM O, M 42.35 15.00 UniAD [24] Think2Drive 6x CAM O, M 45.81 16.36 ThinkTwice [27] Think2Drive 6x CAM, LiDAR O 62.44 31.23 DriveAdapter [26] Think2Drive 6x CAM, LiDAR O 64.22 33.08 Hydra-NeXt [34] Think2Drive 2x CAM - 73.86 50.00 Orion [15] Think2Drive 6x CAM O, L 77.74 54.62 TF++ [56] PDM-lite 1x CAM, LiDAR O, M, S, D 84.21 67.27 Simlingo [43] PDM-lite 1x CAM L 85.07 ±0.95± 0.95 67.27 ±2.11± 2.11 Hip-AD [48] Think2Drive 6x CAM O, M 86.77 69.09 BridgeDriveâ [37] PDM-lite 1x CAM, LiDAR O, M, S, D 86.87 72.27 BevAD-S (ours) PDM-lite 6x CAM O 80.63 ±1.76± 1.76 55.30 ±2.63± 2.63 BevAD-M (ours) PDM-lite 6x CAM O 88.11 ±0.98± 0.98 72.73 ±1.98± 1.98 Table 3: Closed-loop Results on Bench2Drive. Despite its simpler design BevAD outperforms previous modular baselines UniAD and VAD by a large margin, reaching SOTA-level performance. We highlight that BevAD can gain further substantial driving performance by uniformly scaling up training data. Legend: O: 3D Object Detection, M: Map, S: Semantic Segmentation, D: Depth, L: Language, â : concurrent work. If available, we report mean and standard deviation over three seeds to account for the randomness in CARLA. 4.1 Comparison to State of the Art A comprehensive comparison of BevAD to other methods on Bench2Drive with respect to training data, sensor configuration, supervision signals and performance can be found in Tab. 3. For a fair comparison, our model is denoted as BevAD-S when trained solely on Simlingo routes [43], and BevAD-M when trained with additional routes from our scaling study. We consider UniAD [24] and VAD [29] as baselines since their module-level architecture is most similar to BevAD. We report the overall closed-loop driving score and success rate on the 220 test routes of Bench2Drive [28]. The significant improvements of +34.8 DS and +38.9 SR of BevAD-S compared to UniAD highlight the effectiveness of our tokenization as well as the complementary benefits of disentangled output representation and diffusion-based policy. By uniformly scaling up training scenarios, BevAD-M outperforms all prior methods in terms of DS and SR, as well as the concurrent BridgeDrive [37] in terms of DS. We refer to the supplementary for the more fine-grained multi-ability evaluation [28] and analysis on driving skill evolution. 4.2 Qualitative Results In the challenging YieldToEmergencyVehicle scenario, BevAD demonstrates the ability to yield to a rapidly approaching emergency vehicle from behind. Fig. 5 illustrates that BevAD acquires this skill after scaling up training data. Prior methods failed in such scenarios, either due to the lack of 360-degree camera perception [43] or insufficient training data [48]. This underscores BevADâs effective utilization of its surrounding view BEV perception and its scalability. Additional qualitative closed-loop demonstrations are provided in the supplementary material. (a) BevAD-S (b) BevAD-M Figure 5: Yield to Emergency Vehicle. By increasing the training dataset size, BevAD-M learns to yield to emergency vehicles (red) on highways by safely merging into slower traffic. This cability is absent at smaller data scales (BevAD-S) and in prior leading closed-loop methods[43, 48]. Failure Cases. We analyze common failure modes of BevAD-M: (1) Red Light Infractions occur in 19% of unsuccessful closed-loop runs. For example, the driving model runs a red light in the PedestrianCrossing scenario after pedestrians have crossed, suggesting causal confusion. (2) Route Deviations occur when BevAD ignores lane change commands, causing incorrect exits on multi-lane roads. We attribute this to weak conditioning signals from navigation commands, which are often insufficient for timely lane changes. Strengthening conditioning with target points can mitigate this issue by guiding the model towards the correct lane center similar to [25], though it increases reliance on precise map localization. (3) Miscellaneous Collisions result from delayed reactions in time-critical scenarios or occur in situations that involve strong interaction with other vehicles, such as merging into flows. 4.3 Real-world Experiments We evaluate our methodâs real-world applicability on the NAVSIM planning benchmark [9]. To match NAVSIMâs expected planning representation, we adapt the diffusion planner to predict trajectories with associated yaw angles over a four-second horizon, and train BevAD end-to-end on the navtrain split for eight epochs. We summarize performance on the navtest split in Tab. 4, using the official NAVSIM metrics. BevAD outperforms representative baselines UniAD [24] and ParaDrive [52] by 3.2 and 2.6 PDMS, respectively, primarily due to improvements in drivable area compliance (DAC) and ego-progress (EP). Notably, BevAD achieves this performance with only object detection and planning supervision, in contrast to baselines that also leverage online-mapping and occupancy prediction supervision. BevADâs lightweight design yields a 570 GPU-hour (A100-80GB) training compute budget, 10x less than ParaDrive [9]. Furthermore, we ablate the tokenizer design on real-world data: As shown in Tab. 4, removing masking degrades overall performance by 0.7 PDMS, while removing patchifying (p=1p=1) results in a 1.0 PDMS degradation. This demonstrates the effective generalization of our masking and tokenizing scheme to a real-world setting. Method NC â DAC â TTC â EP â PDMS â UniAD [24] 97.8 91.9 92.9 78.8 83.4 ParaDrive [52] 97.9 92.4 93.0 79.3 84.0 BevAD (ours) 98.1 95.3 94.5 80.5 86.6 BevAD w/o mask 97.8 94.9 93.7 80.3 85.9 (-0.7) BevAD w/o patchifying 97.9 94.4 94.4 79.7 85.6 (-1.0) Table 4: Real-world Results on NAVSIM. Performance comparison of BevAD against baseline methods on the real-world NAVSIM benchmark (navtest), including key ablations. Legend: NC: no at-fault collision, DAC: drivable area compliance, TTC: time-to-collsion, EP: ego progress, PDMS: PDM score. 5 Conclusion and Limitations We presented BevAD, a lightweight and highly scalable E2E-AD model that achieves SotA closed-loop driving on Bench2Drive. BevAD emerges from our systematic analysis of common architectural patterns, previously studied in isolation. We show that high-resolution BEV features can lead to overfitting, which we mitigate by forcing the planner to learn bottleneck. Additionally, planning with diffusion complements disentangled planning output representations, particularly excelling when scaled with data. We acknowledge several limitations. First, while compressing the BEV along its spatial dimension significantly improved closed-loop driving, our approach may not directly extend to high-speed highway scenarios, which require long-range perception. A principled, context-adaptive BEV masking strategy remains for future work. Second, our analysis of failure cases suggests potential causal confusions. Mitigating these, perhaps via incorporating world knowledge from VLMs or with reinforcement learning, requires further investigation. Acknowledgments. This work is a result of the joint research project STADT:up (19A22006O). The project is supported by the German Federal Ministry for Economic Affairs and Energy (BMWE), based on a decision of the German Bundestag. The authors are solely responsible for the content of this publication. References [1] S. Behere and M. Torngren (2015) A functional architecture for autonomous driving. In First International Workshop on Automotive Software Architecture (WASA), External Links: Document Cited by: §2. [2] M. Bojarski, D. D. Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, X. Zhang, J. Zhao, and K. Zieba (2016) End to end learning for self-driving cars. External Links: 1604.07316, Link Cited by: §2, §2. [3] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom (2020) NuScenes: a multimodal dataset for autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §1, §2, §3.1. [4] N. Carion, F. Massa, G. Synnaeve, N. Usunier, H. JĂ©gou, and J. Ponce (2020) End-to-end object detection with transformers. In European Conference on Computer Vision (ECCV), External Links: Document Cited by: §3.1. [5] S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang (2024) VADv2: end-to-end vectorized autonomous driving via probabilistic planning. ArXiv abs/2402.13243. External Links: Link Cited by: §2. [6] K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger (2023) TransFuser: imitation with transformer-based sensor fusion for autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (11), p. 12878â12895. External Links: Document Cited by: §2, §2. [7] F. Codevilla, M. Miiller, A. LĂłpez, V. Koltun, and A. Dosovitskiy (2018) End-to-end driving via conditional imitation learning. In IEEE International Conference on Robotics and Automation (ICRA), External Links: Document, Link Cited by: §2. [8] A. Cui, S. Casas, A. Sadat, R. Liao, and R. Urtasun (2021) LookOut: diverse multi-future prediction and planning for self-driving. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: §1. [9] D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta (2024) NAVSIM: data-driven non-reactive autonomous vehicle simulation and benchmarking. In 38th International Conference on Neural Information Processing Systems, NIPS â24. External Links: ISBN 9798331314385 Cited by: §1, §2, §4.3, §4. [10] P. de Haan, D. Jayaraman, and S. Levine (2019) Causal confusion in imitation learning. In 33rd International Conference on Neural Information Processing Systems, Cited by: §1, §2. [11] A. Defazio, X. (. Yang, H. Mehta, K. Mishchenko, A. Khaled, and A. Cutkosky (2024) The road less scheduled. In 38th International Conference on Neural Information Processing Systems, Cited by: §3.2. [12] S. Doll, N. Hanselmann, L. Schneider, R. Schulz, M. Cordts, M. Enzweiler, and H. P.A. Lensch (2024) Dualad: disentangling the dynamic and static world for end-to-end driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §1, §2. [13] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby (2021) An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations, External Links: Link Cited by: §3.3. [14] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: An open urban driving simulator. In 1st Annual Conference on Robot Learning, Cited by: §1, §2, §3.2, §3.2, §4, §A.1. [15] H. Fu, D. Zhang, Z. Zhao, J. Cui, D. Liang, C. Zhang, D. Zhang, H. Xie, B. Wang, and X. Bai (2025) ORION: a holistic end-to-end autonomous driving framework by vision-language instructed action generation. In IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2, Table 3, Table 9. [16] S. Gerstenecker, A. Geiger, and K. Renz (2025) PlanT 2.0: exposing biases and structural flaws in closed-loop driving. External Links: 2511.07292, Link Cited by: §A.2. [17] T. Han, D. Gokay, J. Heyward, C. Zhang, D. Zoran, V. PÄtrÄucean, J. Carreira, D. Damen, and A. Zisserman (2025) Learning from streaming video with orthogonal gradients. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §B.1. [18] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §B.1. [19] G. Heinrich, M. Ranzinger, H. D. Yin, Y. Lu, J. Kautz, A. Tao, B. Catanzaro, and P. Molchanov (2025) RADIOv2.5: improved baselines for agglomerative vision foundation models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §3.1, §B.1. [20] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In 34th International Conference on Neural Information Processing Systems, NIPS â20. External Links: ISBN 9781713829546 Cited by: §3.4, §B.2, §B.2, §B.2. [21] A. Hu, G. Corrado, N. Griffiths, Z. Murez, C. Gurau, H. Yeo, A. Kendall, R. Cipolla, and J. Shotton (2022) Model-based imitation learning for urban driving. In 36th International Conference on Neural Information Processing Systems, NIPS â22. External Links: ISBN 9781713871088 Cited by: §2. [22] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: §3.1, §B.1. [23] S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao (2022) ST-p3: end-to-end vision-based autonomous driving via spatial-temporal feature learning. In European Conference on Computer Vision (ECCV), Cited by: §2. [24] Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y. Qiao, and H. Li (2023) Planning-oriented autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §1, §1, §2, §2, §2, §3.1, §3.1, §3.3, §3.3, §3.3, §4.1, §4.3, Table 3, Table 4, §B.1, Table 5, Table 9. [25] B. Jaeger, K. Chitta, and A. Geiger (2023) Hidden biases of end-to-end driving models. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: §1, §2, §2, §3.1, §3.3, §3.4, §3.4, §4.2, §B.1, Table 9. [26] X. Jia, Y. Gao, L. Chen, J. Yan, P. L. Liu, and H. Li (2023) DriveAdapter: breaking the coupling barrier of perception and planning in end-to-end autonomous driving. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: Table 3, Table 9. [27] X. Jia, P. Wu, L. Chen, J. Xie, C. He, J. Yan, and H. Li (2023) Think twice before driving: towards scalable decoders for end-to-end autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: Table 3, Table 9. [28] X. Jia, Z. Yang, Q. Li, Z. Zhang, and J. Yan (2024) Bench2Drive: towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. In 38th International Conference on Neural Information Processing Systems Datasets and Benchmarks Track, NIPS â24. Cited by: §1, §1, §2, item 1, §3.1, §3.2, §3.2, §3.3, §3.5, §3.5, §4.1, §A.1, §C.2, Table 5. [29] B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang (2023-10) VAD: vectorized scene representation for efficient autonomous driving. In IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §1, §1, §2, §2, §4.1, Table 3, Table 9. [30] C. M. Jiang, Y. Bai, A. Cornman, C. Davis, X. Huang, H. Jeon, S. Kulshrestha, J. Lambert, S. Li, X. Zhou, C. Fuertes, C. Yuan, M. Tan, Y. Zhou, and D. Anguelov (2024) SceneDiffuser: efficient and controllable driving simulation initialization and rollout. In 38th International Conference on Neural Information Processing Systems, NIPS â24. External Links: ISBN 9798331314385 Cited by: §2. [31] N. Karnchanachari, D. Geromichalos, K. S. Tan, N. Li, C. Eriksen, S. Yaghoubi, N. Mehdipour, G. Bernasconi, W. K. Fong, Y. Guo, and H. Caesar (2024) Towards learning-based planning: the nuplan benchmark for real-world autonomous driving. In IEEE International Conference on Robotics and Automation (ICRA), External Links: Document Cited by: §2. [32] Y. Kuratov, A. Bulatov, P. Anokhin, I. Rodkin, D. I. Sorokin, A. Sorokin, and M. Burtsev (2024) BABILong: testing the limits of LLMs with long context reasoning-in-a-haystack. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, External Links: Link Cited by: §3.3. [33] Q. Li, X. Jia, S. Wang, and J. Yan (2024) Think2Drive: efficient reinforcement learning by thinking with latent world model for autonomous driving (in carla-v2). In European Conference on Computer Vision (ECCV), External Links: Document, ISBN 978-3-031-72994-2, Link Cited by: §3.2, §A.1. [34] Z. Li, S. Wang, S. Lan, Z. Yu, Z. Wu, and J. M. Alvarez (2025) Hydra-next: robust closed-loop driving with open-loop training. In IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2, Table 3, Table 9. [35] Z. Li, W. Wang, H. Li, E. Xie, C. Fang, D. Lu, X. Zhu, and G. Yu (2022) BEVFormer: learning birdâs-eye-view representation from multi-camera images via spatiotemporal transformers. In European Conference on Computer Vision (ECCV), External Links: Document Cited by: §1, §2, item 1, §3.1, §3.1, §3.3, §B.1, §B.4, §C.1. [36] B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y. Zhang, Q. Zhang, and X. Wang (2025) DiffusionDrive: truncated diffusion model for end-to-end autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §1, §2, §2, §3.4, §B.2, §B.2, §C.1. [37] S. Liu, W. Chen, W. Li, Z. Wang, L. Yang, J. Huang, Y. Zhang, Z. Huang, Z. Cheng, and H. Yang (2025) BridgeDrive: diffusion bridge policy for closed-loop trajectory planning in autonomous driving. External Links: 2509.23589, Link Cited by: §4.1, Table 3, §A.1, §A.2, §B.1, §B.2, Table 9. [38] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. In International Conference on Learning Representations, External Links: Link Cited by: §3.2. [39] C. Luo (2022) Understanding diffusion models: a unified perspective. External Links: 2208.11970, Link Cited by: §B.2. [40] A. Naumann, X. Gu, T. Dimlioglu, M. Bojarski, A. Degirmenci, A. Popov, D. Bisla, M. Pavone, U. MĂŒller, and B. Ivanovic (2025) Data scaling laws for end-to-end autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), External Links: Document Cited by: §2, §2, §3.5, §3.5. [41] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: Figure 2, Figure 2, §3.1, §B.1. [42] A. Popov, A. Degirmenci, D. Wehr, S. Hegde, R. Oldja, A. Kamenev, B. Douillard, D. NistĂ©r, U. Muller, R. Bhargava, S. Birchfield, and N. Smolyanskiy (2025) Mitigating covariate shift in imitation learning for autonomous vehicles using latent space generative world models. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: §2, §B.1. [43] K. Renz, L. Chen, E. Arani, and O. Sinavski (2025) SimLingo: vision-only closed-loop autonomous driving with language-action alignment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), External Links: Document Cited by: §1, §2, §2, §2, §3.1, §3.1, §3.2, §3.3, §3.4, §3.5, Figure 5, Figure 5, §4.1, §4.2, Table 3, §A.1, §A.2, §B.1, §B.3, §C.2, Table 9. [44] C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, J. BeiĂwenger, P. Luo, A. Geiger, and H. Li (2025) DriveLM: driving with graph visual question answering. In European Conference on Computer Vision (ECCV), A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol (Eds.), External Links: ISBN 978-3-031-72943-0 Cited by: §3.2, §A.1. [45] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, Vol. 37. Cited by: §B.2. [46] J. Song, C. Meng, and S. Ermon (2021) Denoising diffusion implicit models. In International Conference on Learning Representations, External Links: Link Cited by: §3.1, §B.2, §B.2, §B.2. [47] W. Sun, X. Lin, Y. Shi, C. Zhang, H. Wu, and S. Zheng (2024) SparseDrive: end-to-end autonomous driving via sparse scene representation. In IEEE International Conference on Robotics and Automation (ICRA), External Links: Link Cited by: §2, §2, §2. [48] Y. Tang, Z. Xu, Z. Meng, and E. Cheng (2025) HiP-ad: hierarchical and multi-granularity planning with deformable attention for autonomous driving in a single decoder. In IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2, §2, §3.4, Figure 5, Figure 5, §4.2, Table 3, §C.2, Table 9. [49] W. Tong, C. Sima, T. Wang, L. Chen, S. Wu, H. Deng, Y. Gu, L. Lu, P. Luo, D. Lin, and H. Li (2023) Scene as occupancy. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: §2. [50] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ć. Kaiser, and I. Polosukhin (2017) Attention is all you need. In 31st International Conference on Neural Information Processing Systems, NIPSâ17. External Links: ISBN 9781510860964 Cited by: §3.1. [51] S. Wang, Y. Liu, T. Wang, Y. Li, and X. Zhang (2023) Exploring object-centric temporal modeling for efficient multi-view 3d object detection. In IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Document Cited by: §3.1, §B.1. [52] X. Weng, B. Ivanovic, Y. Wang, Y. Wang, and M. Pavone (2024-06) PARA-drive: parallelized architecture for real-time autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §1, §2, §2, Figure 2, Figure 2, §3.1, §3.1, §3.1, §3.3, §3.3, §4.3, Table 4. [53] Y. Zheng, R. Liang, K. ZHENG, J. Zheng, L. Mao, J. Li, W. Gu, R. Ai, S. E. Li, X. Zhan, and J. Liu (2025) Diffusion-based planning for autonomous driving with flexible guidance. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §3.4. [54] Y. Zheng, P. Yang, Z. Xia, Q. Zhang, Y. Zheng, S. Gu, B. Jin, T. Zhang, B. Lu, C. Han, X. Lang, and D. Zhao (2025) Data scaling laws for imitation learning-based end-to-end autonomous driving. External Links: 2412.02689, Link Cited by: §2, §2, §3.5, §3.5. [55] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai (2021) Deformable DETR: deformable transformers for end-to-end object detection. In International Conference on Learning Representations, Cited by: §3.1, §3.3. [56] J. Zimmerlin, J. BeiĂwenger, B. Jaeger, A. Geiger, and K. Chitta (2024) Hidden biases of end-to-end driving datasets. External Links: 2412.09602, Link Cited by: §2, §2, §2, §3.1, §3.2, §3.3, §3.4, Table 3, §A.1, §A.2, §B.1. Supplementary Material This supplementary material details our data scaling procedure (Sec. A), presents implementation details for our analysis framework and BevAD (Sec. B), and includes additional quantitative and qualitative results (Sec. C). A Dataset A.1 Bench2Drive Bench2Drive [28] offers expert demonstrations from the Think2Drive expert [33], comprising over 13,000 episodes for training. However, the adoption of the dataset within the community is limited due to missing sensor modalities and language annotations [43, 56, 37]. Furthermore, we identify significant 3D label flaws, as depicted in Fig. 6: (1) Unlabeled parked vehicles in all CARLA towns except Town12 and Town13, (2) Incomplete annotations for multi-lightbox traffic signals. (3) Misplaced bounding boxes for numerous traffic signs and pedestrians. These flaws create contradictory object detection supervision and enable planner shortcut learning, e.g., by distinguishing static cars from dynamic cars. Critically, the lack of public route files or open-source expert code hinder dataset extension and issue resolution. We therefore follow [43, 56, 37] and utilize the CARLA [14] Leaderboard 2.0 training routes and the rule-based PDM-lite expert [44, 56] for data collection. A.2 Route Generator Apart from the comprehensive Bench2Drive dataset, current CARLA imitation learning is limited by the diversity of expert demonstrations. SotA methods [56, 43, 37] utilize short, single-scenario route segments from CARLA Leaderboard 2.0 for training. However, this approach suffers from significantly imbalanced scenario distributions (e.g., scenario InterurbanAdvancedActorFlow appears only six times, while others appear 40 times). While these methods employ upsampling of rare routes and extensive camera- and weather augmentations, this mitigation is limited. Fixed geographic contexts and similar actor behaviors can lead to overfitting to spurious correlations [16]. To overcome these limitations, we build a novel route generator that creates unique route-scenario combinations within CARLA towns, enabling extensive and diverse data collection. Our approach maintains the established concept of short, single-scenario training routes, which facilitates fine-grained control over scenario distribution; for simplicity, we adopt a uniform distribution. The route generation algorithm proceeds in three steps: (1) Trigger Point Selection, (2) Route Planning and (3) Scenario Generation. Figure 6: Label Flaws. Bench2Drive exhibits incomplete and erroneous 3D bounding box annotations in various scenes. Trigger Point Selection. We employ a coarse-to-fine search strategy for trigger point selection. Initially, scenarios are mapped to one of three coarse location classes: Intersection, No-Intersection, or Highway Ramp. Subsequently, trigger points meeting these initial criteria are exhaustively sampled from all CARLA maps. This candidate list is then refined using scenario-specific criteria. For Intersections, relevant features include traffic lights, stop signs, turning options, bike lanes, and pedestrian crossings. For No-Intersections, we consider the number of adjacent lanes (with/against ego traffic flow), and the presence of parking or shoulder lanes. Highway Ramps are differentiated into On- and Off-Ramps. For instance, the SignalizedJunctionLeftTurn scenario necessitates a signalized intersection with a left-turn option. The VehicleOpensDoorTwoWays scenario requires a right-side parking lane for the adversary and an adjacent lane with oncoming traffic. Conversely, the Accident scenario demands a right-side shoulder lane and a left-side lane with traffic flowing in the same direction. These examples are visualized in Fig. 7. (a) Signalized Junction Left (b) Open Door (c) Accident Figure 7: Visualization of Trigger Point Selection Criteria. (a) This scenario requires a signalized junction with a left-turn exit relative to the ego lane. (b) The VehicleOpensDoorTwoWays scenario requires a right-side parking lane for the adversary and an adjacent left-side lane for oncoming traffic. (c) The Accident scenario demands a right-side shoulder lane for the group of blocking vehicles and an adjacent left-side lane with the same traffic flow. Route Planning. We plan routes through a trigger point using a bidirectional search on the lane graph to determine start and end points. This search is constrained to avoid additional intersections. Distances between the start point, trigger point, and end point are randomly sampled from scenario-dependent intervals. This strategy enhances variance and mitigates the learning of distance- and location-dependent shortcuts. Scenario Generation. The final step involves configuring CARLAâs pre-defined scenario behavior models at the trigger point. To enhance diversity and prevent spurious correlations, additional scenario parameters are sampled from meaningful intervals, for instance, by varying inter-vehicle distances within traffic flows. Our route generator produces over 100,000 unique route-scenario combinations across all CARLA maps. Due to the high cost of sensor data collection, we utilize a subset of 8,000 routes for our data scaling experiments. Nevertheless, the generatorâs extensive diversity remains a significant asset for applications beyond imitation learning, such as reinforcement learning. B Implementation Details B.1 Model Efficient Streaming Training. The training performance of previous BEV-based E2E-AD architectures is bottlenecked by recurrent BEV feature generation, required for fusing temporal information within the BEV encoderâs temporal self-attention layer [35]. For instance, UniAD [24] processes four past frames (tâ4,âŠ,tâ1t-4,âŠ,t-1) at each training step t with a frozen BEV backbone to eventually compute temporal self-attention between â±Bev(tâ1)F_Bev^(t-1) and â±Bev(t)F_Bev^(t). In contrast, we adapt streaming training from object-centric modeling [51] to BEV-based architectures. Specifically, during training, we sample streams of n subsequent frames, feeding them sequentially into the network. A memory component caches current BEV features â±Bev(t)F_Bev^(t) at each step t, enabling subsequent steps (tâČ=t+1t =t+1) to retrieve â±Bev(tâČâ1)=â±Bev(t)F_Bev^(t -1)=F_Bev^(t) from cache instead of recurrently recomputing it. Short sequences (e.g., 2s, n=20n=20 frames) are streamed to periodically reset the memory and mitigate non-orthogonal gradients [17] arising from strongly correlated frames. Our approach avoids four additional BEV backbone forward passes (compared to UniAD stage-1) and significantly reduces CPU load by requiring only one set of multi-view images per step, rather than five. Additionally, we employ RADIO [19] as an image backbone with a low-rank adapter (LoRA) [22] for parameter-efficient finetuning. RADIO provides rich, generic features, and LoRA finetuning significantly reduces VRAM requirements compared to backpropagating through a ResNet [18]. Pruning additional heads for online-mapping, motion forecasting, and occupancy prediction further reduces BevADâs memory footprint. These optimizations combined enable end-to-end training with a batch size of 16 on a single A100-80GB GPU, a significant improvement over UniAD, which required two-stage training with a frozen backbone due to VRAM constraints. Our optimizations yield significantly higher training sample throughput, even surpassing UniAD-tiny, as shown in Tab. 5. Specifically, BevAD processes 35Ă35Ă more training samples per second with a frozen BEV backbone, and achieves a 22Ă22Ă speed-up in end-to-end training. These optimizations represent a major contribution towards scalable imitation learning for robust closed-loop performance, a benefit we extend to the community through our open-source code release. Method BEV FPS (train) â UniAD-tiny [24, 28] frozen (stage-2) 2.5 BevAD (ours) frozen 89.0 end-to-end 55.0 Table 5: Training Efficiency. BevAD achieves substantial training speed-up relative to UniAD-tiny. Measured on 8xA100-80GB. Camera Augmentation. Camera augmentations are integral to robust closed-loop imitation learning [25]. Perfect expert drivers, such as PDM-lite, maintain precise lane centering, leading to a training distribution dominated by ideal states. However, during closed-loop testing, accumulated steering errors can cause the vehicle to drift, resulting in covariate shift [42] and degraded planner performance. A common mitigation involves augmenting driver camera views with random shifts and rotations during training [25], adopted by SotA methods on Bench2Drive [43, 56, 37]. This augmentation simulates out-of-distribution states not observed with perfect expert driving. However, these shift and rotation augmentations are typically limited to single front-facing camera setups and are challenging to apply to multi-camera systems. Furthermore, their application to real-world data necessitates novel view synthesis, introducing pipeline overhead and potential artifacts. To address these limitations, we propose a novel BEV-based augmentation strategy. Instead of manipulating raw sensor data, we augment the BEV coordinate system directly. This is achieved by sampling a random transformation matrix CâaârBâeâv^BevT_Car, comprising a small yaw rotation ÎłâŒ[â22.5â,22.5â]Îł [-22.5 ,22.5 ] and a lateral offset ÎâyâŒ[â0.75âm,0.75âm] y [-0.75m,0.75m]. This matrix is then applied to all camera transformation matrices CamiCâaâr^CarT_Cam_i: CamiBev=CarBevâ CamiCâaâr^BevT_Cam_i=^BevT_Car·^CarT_Cam_i By providing CamiBev^BevT_Cam_i to the BEV encoder, it builds BEV features â±BevF_Bev in the augmented BEV coordinate system rather than the vehicle coordinate system. Additionally, we apply this transformation to all ground truth labels, i.e., 3D bounding boxes and planning trajectories. Our augmentation prevents the detection head and planner from learning a bias towards axis-aligned objects or trajectories relative to the ego vehicle. For example, the disentangled planner must learn to predict the future path in the augmented BEV space, necessitating a robust understanding of the BEV feature grid. Unlike prior camera augmentations, our BEV-based scheme requires no augmented sensor data, as it only alters the generation of the latent BEV representation via a non-learnable, random transformation. Planning Queries. The planning queryâs interpretation determines the output representation. For an entangled trajectory representation, we define Plan=TrajââNtĂNcP_Plan=P_Traj ^N_tĂ N_c, where NtN_t is the number of trajectory points and NcN_c is the feature dimension. For a disentangled representation, Plan=[Pathâ„Speed]P_Plan=[P_Path \|P_Speed], with PathââNpĂNcP_Path ^N_pĂ N_c and SpeedââNtĂNcP_Speed ^N_tĂ N_c, where NpN_p denotes the number of path points. In our experiments, we set Nt=15N_t=15 for a 3âs3s planning horizon, resulting in temporal waypoint and speed predictions spaced at 0.2âs0.2s intervals. For path planning, we use Np=30N_p=30 with waypoints spaced 1âm1m apart. Planning Head. The planning head (Fig. 2(b)) comprises Nlayer=8N_layer=8 transformer decoder layers with a feature dimension of Nc=512N_c=512. Inspired by [41], we integrate an adaLN-Zero transformation to condition the self-attention and global cross-attention layers with c, as depicted in Fig. 8. For the point estimator, =embâ(command)+embâ(v0)c=emb(command)+emb(v_0), combining a learnable embedding of the high-level navigation command and a sinusoidal embedding of the current velocity. For the diffusion-based planner, =embâ(command)+embâ(v0)+embâ(Ï)c=emb(command)+emb(v_0)+emb(Ï), additionally incorporating an embedding of the current diffusion timestep Ï. Figure 8: adaLN-Zero Attention. We employ an adaLN-Zero transformation to condition the transformer on c. For self-attention, we set ==PlanK=V=Q_Plan and for cross-attention, we set ==â±SceneK=V=F_Scene. B.2 Diffusion-based Planning Preliminaries. Diffusion models are a class of generative models that learn to reverse a gradual noising process applied to data [45, 20]. They define a fixed forward Markov chain that progressively adds Gaussian noise to data, transforming it into a pure noise distribution. Specifically, the forward process admits sampling of noisy data Ïx_Ï for arbitrary (time)steps Ï in the Markov chain in closed-form given clean data 0x_0 by sampling Gaussian noise Ï”âŒâ(,) Δ (0,1) [20]: Ïâ(0,Ï”)=αÏÂŻâ+1âαÏÂŻâÏ” _Ï(x_0, Δ)= _Ïx_0+ 1- _Ï Î” (1) The sequence α0ÂŻ,âŠâαTÂŻ _0,⊠_T typically stems from a variance schedule with constant parameters [20, 46]. The core of diffusion models lies in learning a reverse Markov chain that iteratively denoises samples over T timesteps, starting from random noise pâ(T)=â(T;,)p(x_T)=N(x_T;0,I), to recover data samples from the original distribution [20]: pΞâ(0:T):=pâ(T)ââÏ=1TpΞâ((Ïâ1)|Ï) p_Ξ(x_0:T):=p(x_T) _Ï=1^Tp_Ξ(x_(Ï-1)|x_Ï) (2) Instead of learning the Gaussian transitions pΞâ(Ïâ1|Ï)p_Ξ(x_Ï-1|x_Ï) directly, it is common practice to learn a function approximator fΞ:(Ï,Ï)â0f_Ξ:(x_Ï,Ï) _0 parameterized by a neural network with learnable weights Ξ [39]. This allows learning the diffusion model by minimizing Eq. 3 for all timesteps Ï [20]: â0âfΞâ(Ï,Ï)â \|x_0-f_Ξ(x_Ï,Ï) \| (3) For diffusion-based planning, the clean planning ground truth 0x_0 is defined based on the planning representation: âą Entangled: a series of normalized trajectory waypoints, 0=(xi,yii=1Ntx_0=\(x_i,y_i\_i=1^N_t\. âą Disentangled: a tuple comprising normalized path waypoints and a normalized speed sequence, i.e., 0=((xi,yii=1Np,(vii=1Nt)x_0=(\(x_i,y_i\_i=1^N_p,\(v_i\_i=1^N_t\). Training. During training, we sample a timestep Ï from a uniform distribution and Gaussian noise Ï”âŒâ(,) Δ (0,1). We obtain the noisy sample Ïx_Ï using Eq. 1 and following the DDIM variance schedule [46]. The planning head serves as the conditional function approximator fΞ:(~Ï,Ï,)f_Ξ:( x_Ï,Ï,z) for the reverse process [36], tasked with predicting 0x_0 from ~Ï x_Ï, the diffusion timestep Ï, and conditioning context =(â±Scene,c)z=(F_Scene,c). Note that Ïx_Ï is embedded into a latent space via an MLP to obtain ~Ï x_Ï before being fed to fΞf_Ξ. Inference. For inference, we start with a random sample TâŒâ(0,1)x_T (0,1) and iteratively denoise it using the trained function approximator fΞâ(~Ï,Ï,)f_Ξ( x_Ï,Ï,z) and the DDIM sampling algorithm [46]. This process progressively denoises Tx_T to yield the final clean prediction 0x_0. We specifically employ DDIMâs accelerated generation that conducts denoising with a subset of S<TS<T denoising steps to enhance computational efficiency during sampling. DiffusionDrive. We do not adopt DiffusionDriveâs [36] truncated diffusion schedule in our experiments. This decision stems from our observation, that DiffusionDriveâs forward process adds noise to a fixed set of anchor trajectories, while the reverse process aims to predict ground truth trajectories, leading to an asymmetric reversal. This issue is also thoroughly discussed in concurrent work [37], which proposes a theoretically sound diffusion bridge formulation as a solution. B.3 Controller We adopt the disentangled PID controllers from Simlingo [43] for lateral and longitudinal control. All controller parameters are retained, with the exception of the steering proportional gain, which is slightly lowered to Psteer=1.8P_steer=1.8 to reduce oversteering in our closed-loop agent. B.4 Loss The overall loss function for end-to-end training combines terms for 3D object detection and planning: â=λdetââdet+λplanââplan = _detL_det+ _planL_plan (4) The detection loss âdetL_det, adopted from [35], includes classification and regression components. The planning loss âplanL_plan varies with the plannerâs modeling choice: âą For regression-based planning with a point estimator, âplanL_plan is a smooth L1L^1 loss applied to the trajectory error, or path and speed error. âą For diffusion-based planning, âplanL_plan is a smooth L1L^1 loss on the 0x_0-prediction error, as defined in Eq. 3. The loss coefficients are set to λdet=1 _det=1 and λplan=100 _plan=100 to balance the magnitudes of the detection and planning loss components. B.5 Metrics To better characterize closed-loop failure modes, we introduce auxiliary metrics: the static infraction rate (IRsIR_s) and dynamic infraction rate (IRdIR_d). IRs=Nlayout-collision+Noutside-laneNroutes _s= N_layout-collision+N_outside-laneN_routes (5) This metric quantifies infractions related to lateral control errors, such as collisions with static layout elements or driving outside the lane. IRd=Nactor-collision+Nred-light+Nstop-signNroutes _d= N_actor-collision+N_red-light+N_stop-signN_routes (6) This metric captures infractions arising from longitudinal control errors and interactions with dynamic elements, including collisions with actors, running red lights, or failing to stop at stop signs. Collectively, these metrics provide the expected number of infractions per route, offering a granular understanding of control deficiencies. C Results This section presents an ablation study, quantitative results on the Bench2Drive benchmark, and qualitative demonstrations of BevADâs closed-loop driving. C.1 Ablations Our ablation studies investigate early design choices. For the following ablations of camera augmentation and BEV size, our baseline is a planning head with a tokenizer (p=4p=4, masking), disentangled representation, and a point-estimator regressor, as detailed in Sec. 3.3 and Tab. 1. Furthermore, we ablate the number of denoising steps in the diffusion planner using our strongest model, BevAD-M. Camera Augmentation. To assess the effectiveness of our novel BEV-based augmentation, we compare the baseline against a variant trained without it. As shown in Tab. 6, the absence of camera augmentation significantly degrades closed-loop performance. This degradation primarily stems from a 5.7Ă5.7Ă increase in static infractions, leading to reduced route completion and increased secondary collisions. These results underscore the critical role of our augmentation scheme in promoting robust driving and enabling recovery from compounding steering errors. The results also highlight the insufficiency of common open-loop metrics, as the L1 trajectory error does not reflect the observed degradation in model robustness. Augmentation DS â SR â IRsIR_s â L1 (m) â â 82.6282.62 57.4357.43 0.0550.055 1.431.43 â 66.6066.60 33.6433.64 0.3140.314 1.471.47 Table 6: Ablation of camera augmentation. The absence of camera augmentation significantly degrades closed-loop performance, despite minimal impact on open-loop L1 trajectory deviation. This underscores the contribution of our BEV-based augmentation to robust driving and covariate shift mitigation. BEV Size. Our tokenizer compresses high-resolution BEV features (â±BevF_Bev) into low-resolution scene tokens (â±SceneF_Scene). An alternative is to directly learn a low-resolution BEV feature space. As shown in Tab. 7, despite exposing the same number of scene tokens to the planner, direct low-resolution BEV generation significantly degrades closed-loop performance. Specifically, the deformable BEV-image cross-attention of our BEV encoder (based on BEVFormer [35]) extracts image information more sparsely when generating low-resolution BEVs, potentially omitting fine details. Furthermore, it prevents leveraging deformable refinement layers for sampling local, high-resolution features around the future trajectory. As a result, we observe 7.4Ă7.4Ă more static infractions with the low-resolution BEV encoder compared to our compression approach. This finding underscores the necessity of compressing high-resolution representations rather than directly learning low-resolution ones. BEV Scene Tokens DS â SR â IRsIR_s â 100Ă100100Ă 100 25Ă1525Ă 15 82.6282.62 57.4357.43 0.0550.055 25Ă2525Ă 25 25Ă1525Ă 15 72.1872.18 40.3640.36 0.4090.409 Table 7: BEV Resolution and Tokenization. High-resolution BEV generation, compressed via our tokenizer, yields superior closed-loop driving performance compared to direct low-resolution BEV generation. Number of denoising steps. The iterative denoising of diffusion models critically impacts the inference latency in real-world deployments. The runtime of our diffusion-based planner linearly increases with the number of denoising steps S, while the point estimator planner has constant runtime, corresponding to S=1S=1. We thus evaluate how S affects closed-loop driving performance and inference FPS in Tab. 8. In contrast to DiffusionDrive [36], we achieve constant driving performance for Sâ2,5,10Sâ\2,5,10\, without applying their truncated diffusion framework. We attribute this to a bug in their diffusion schedule, which we detail in the appendix. S DS â SR â FPS â 10 88.11 72.73 4.2 5 88.33 72.72 5.8 2 88.53 72.72 7.5 Table 8: Impact of Denoising Iterations. Reducing denoising steps significantly boosts inference FPS while preserving closed-loop driving performance, enabling real-time applications. FPS measured on a Quadro RTX 8000. C.2 Multi-Ability Evaluation Method Ability (%) â Merging Overtaking Emergency Brake Give Way Traffic Sign Mean VAD [29] 8.11 24.44 18.64 20.00 19.15 18.07 UniAD-Base [24] 14.10 17.78 21.67 10.00 14.21 15.55 ThinkTwice [27] 27.38 18.42 35.82 50.00 54.23 37.17 DriveAdapter [26] 28.82 26.38 48.76 50.00 56.43 42.08 Hydra-NeXt [34] 40.00 64.44 61.67 50.00 50.00 53.22 Orion [15] 25.00 71.11 78.33 30.00 69.15 54.72 TF++ [25] 58.75 57.77 83.33 40.00 82.11 64.39 Simlingo [43] 54.01 ±2.63± 2.63 57.04 ±3.40± 3.40 88.33 ±3.34± 3.34 53.33 ±5.77± 5.77 82.45 ±4.73± 4.73 67.03 ±2.12± 2.12 Hip-AD [48] 50.00 84.44 83.33 40.00 72.10 65.98 BridgeDriveâ [37] 63.50 58.89 88.34 50.00 88.95 69.93 BevAD-S (ours) 55.83 ±0.72± 0.72 53.33 ±6.67± 6.67 63.33 ±1.93± 1.93 46.67 ±5.77± 5.77 60.88 ±3.08± 3.08 56.01 ±3.78± 3.78 BevAD-M (ours) 71.67 ±2.60± 2.60 74.07 ±1.29± 1.29 75.56 ±4.41± 4.41 76.67 ±5.77± 5.77 75.44 ±1.61± 1.61 74.68 ±1.24± 1.24 Table 9: Multi-Ability Evaluation. BevAD consistenly achieves high results across all skills, dominating the mean score score. Notably, it significantly surpasses prior work in the Merging and Give Way skills. However, it exhibits comparatively lower performance in Overtaking, Emergency Brake, and Traffic Sign skills. Legend: â : concurrent work. To gain a nuanced understanding of system performance in closed-loop driving, we employ the fine-granular multi-ability evaluation protocol from Bench2Drive [28]. This protocol defines five advanced urban driving skills: Merging, Overtaking, Emergency Brake, Give Way, and Traffic Sign. Each of the 220 test routes is mapped to one or more skills necessary for successfully navigating the scenario. Tab. 9 presents a comparison of BevADâs multi-ability scores against prior work. BevAD consistently achieves high scores (>70%>70\%) across all skills, dominating the mean score. In contrast, prior state-of-the-art methods [43, 48] exhibit uneven performance, excelling in some skills while underperforming in others. BevAD-M surpasses the previous best in Merging and Give Way skills by +8.17 and +23.34, respectively. These skills demand comprehensive surround perception, underscoring BevADâs effective utilization of its multi-view camera system. However, BevAD lacks in terms of Overtaking, Emergency Brake, and Traffic Sign skills compared to the best prior or concurrent methods for each skill. Figure 9: Emerging Skills. All skills remain underdeveloped with fewer than 4,000 training episodes. As the volume of training data increases, the skills progressively and uniformly emerge. Building on the discussion in Sec. 3.5, we present the progression of BevADâs closed-loop driving skills as the training dataset size increases, detailed in Fig. 9. With fewer than 4,000 training episodes, all skills exhibit a success rate below 50%. However, as the training data is doubled and quadrupled, the skills show consistent and uniform improvement. This enhancement is evidenced by high success rates in complex scenarios, such as overtaking amidst oncoming traffic, merging onto highways and into traffic flow at intersections, and yielding to emergency vehicles. C.3 Qualitative Results We provide qualitative closed-loop driving examples for each multi-ability skill of BevAD-M in Fig. 10, 11, 12, 13, 14 and 15. The examples are best viewed when zoomed in and visualize planned ego trajectories that are generated by rolling out the predicted speed profile along the predicted path. This depiction aids in understanding the dynamic planning component, but is solely for visualization. It does not serve as controller input, nor does it affect the lateral accuracy of the predicted path. Figure 10: Merging. BevAD merges from a parallel parking space into traffic. It yields to rear-end flow of vehicles, identifies a safe gap, and accelerates for seamless merging. Figure 11: Overtaking (1). BevAD executes an overtaking maneuver when the route is blocked by a construction vehicle. It waits for clear oncoming traffic, then steers into the opposing lane, accelerates to quickly pass the obstacle, and subsequently decelerates when returning to its original lane. Figure 12: Overtaking (2). BevAD approaches a group of cyclists. It executes a safe left lane change to overtake them. After the maneuver, BevAD returns to its original lane, maintaining a safe distance to the cyclists. Figure 13: Emergency Brake. BevAD brakes at a green-light intersection due to a pedestrian crossing its left-turn path. Driving resumes upon pedestrian clearance. Figure 14: Give Way. BevAD yields to an oncoming vehicle encroaching on the ego lane. It performs a controlled rightward deviation from the lane center, staying within road limits, and re-centers once the oncoming traffic has passed. Figure 15: Traffic Sign. BevAD stops at a stop sign at an intersection with cross-traffic. It waits for a safe gap, then executes a right turn and merges into the traffic flow.