Paper deep dive
ControlMap: Controllable High-Definition Map Generation for Traffic Scenario Simulation
Marwan Farag, Steffen WĂ€ldele, Yu Yao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 98%
Last extracted: 6/20/2026, 7:33:09 AM
Summary
ControlMap is a data-driven pipeline for controllable High-Definition (HD) map generation designed for autonomous driving simulations. It addresses the high cost and lack of diversity in HD map creation by using Standard Definition (SD) maps (e.g., from OpenStreetMap) as spatial guidance signals. The architecture extends the SLEDGE framework by integrating a ControlNet-style branch into a Latent Diffusion Transformer (DiT). This allows for fine-grained control over road topology while supporting city-level style transfer via label conditioning. The authors also introduce two novel metrics, Control Recall and Ground-Truth Recall, to evaluate how faithfully the generated maps adhere to the input SD control signals and ground-truth topologies.
Entities (9)
Relation Signals (5)
ControlMap â extends â SLEDGE
confidence 100% · Our approach transforms publicly available SD map data [19] into lightweight yet structured rasterized control signals... builds on the SLEDGE framework [5]
ControlMap â generates â HD Map
confidence 100% · This paper presents a data-driven pipeline for controllable HD map generation
SD Map â guides â ControlMap
confidence 100% · We propose using Standard Definition (SD) maps (e.g., OpenStreetMap [19]) to guide generation.
ControlMap â uses â ControlNet
confidence 100% · using latent diffusion and ControlNet for spatial conditioning.
nuPlan â providesdatafor â ControlMap
confidence 90% · We build upon the nuPlan benchmark [3], a large-scale autonomous driving dataset
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Simulation is central to validating autonomous driving systems, yet current pipelines are limited by insufficient scenario diversity due to costly High Definition (HD) map creation. Scaling HD maps requires expensive data collection and manual processing. Moreover, existing generative models lack the fine-grained control necessary to target specific road topologies during generation. This paper presents a data-driven pipeline for controllable HD map generation using latent diffusion and ControlNet for spatial conditioning. To our knowledge, we are the first to inject spatial guidance signals into a diffusion model for HD map synthesis. Furthermore, our model supports adjustable conditioning strength through classifier-free guidance and city-level style transfer via city label conditioning. To complement existing metrics, we introduce two novel metrics to evaluate adherence to the control signal and similarity to ground-truth maps. Experiments demonstrate that our model generates realistic HD maps that faithfully follow input road topologies while accurately preserving city-specific details.
Tags
Links
- Source: https://arxiv.org/abs/2606.15930v1
- Canonical: https://arxiv.org/abs/2606.15930v1
Trouble viewing inline? Open PDF directly â
Full Text
38,354 characters extracted from source content.
Expand or collapse full text
ControlMap: Controllable High-Definition Map Generation for Traffic Scenario Simulation Marwan Farag 1 , Steîen WĂ€ldele 2 , and Yu Yao 3 1 University of Stuttgart, Stuttgart, Germany marwan.farag@iss.uni-stuttgart.de 2 Robert Bosch GmbH, Renningen, Germany steffen.Waeldele@de.bosch.com 3 Motional, Inc yu.yao@motional.com Abstract. Simulation is central to validating autonomous driving sys- tems, yet current pipelines are limited by insuîcient scenario diversity due to costly High Deînition (HD) map creation. Scaling HD maps re- quires expensive data collection and manual processing. Moreover, exist- ing generative models lack the îne-grained control necessary to target speciîc road topologies during generation. This paper presents a data-driven pipeline for controllable HD map gen- eration using latent diîusion and ControlNet for spatial conditioning. To our knowledge, we are the îrst to inject spatial guidance signals into a diîusion model for HD map synthesis. Furthermore, our model sup- ports adjustable conditioning strength through classiîer-free guidance and city-level style transfer via city label conditioning. To complement existing metrics, we introduce two novel metrics to evaluate adherence to the control signal and similarity to ground-truth maps. Experiments demonstrate that our model generates realistic HD maps that faithfully follow input road topologies while accurately preserving city-speciîc de- tails. Keywords: ControlNet Ì HD Maps Ì SD Maps Ì Autonomous Driving Ì Controllable Generation 1 Introduction Simulation pipelines are a cornerstone of the veriîcation process for the safety- critical components of autonomous driving systems. However, their utility relies on access to realistic and diverse driving scenarios [13, 26]. At the core of each driving scenario lies a High Definition (HD) map that includes detailed lane ge- ometry, traîc infrastructure, and intersection layout [2,13]. Creating these maps requires specialized sensor-equipped vehicles, extensive validation, and manual annotation [8,26]. This makes the process very costly and diîcult to scale [14]. Large benchmarks such as nuPlan [3], which collected over 1300 hours of driving logs and more than 2TB of sensor data, still cover only a limited set of regions. 2Marwan Farag, Steîen WĂ€ldele and Yu Yao BOSBOSBOSBOS NoLabelNoLabelNoLabelNoLabel Ground- Truth HDMap* SDMap (Control)** Generated HDMap* *Colors (red,green) represent lanes with active traffic light. ** Red: Major highways --Green: Primary and Secondary Roads --Blue: Local and Residential Roads Fig. 1: We condition on SD maps that are cheap to obtain (e.g., from Open- StreetMap [19]) to generate diverse HD maps with user-speciîed structure. Our gen- erations (bottom) follow the control layout (middle) and recover realistic HD detail consistent with test ground truth (top). BOS denotes ground-truth maps coming from Boston; No Label indicates that the model took no city label as input during genera- tion. This underlines the challenge of extending HD map-based simulation to new environments [26]. Another challenge is the dependence on a limited diversity of scenarios for testing. Autonomous systems must be tested across an extremely wide range of situations, including rare events and complex interactions that might not ap- pear in existing datasets [3,7]. Most simulation pipelines depend on static maps or pre-recorded logs, which inherently limit scenario variability [2]. Data-driven map generation oîers a promising alternative. Recent generative methods train diîusion models [10] to synthesise novel and diverse driving scenes that circum- vent the real-world data limitations [5,16,21]. It is expected that such methods will widen the scope of simulation-based testing for autonomous systems [5]. However, current approaches to autonomous driving map generation generally lack user-speciîed layout control. Consequently, there is a strong motivation for methods that can provide precise structural guidance, particularly for rare and challenging scenarios, without the expense and rigidity of HD maps. ControlMap3 We propose using Standard Deînition (SD) maps (e.g., OpenStreetMap [19]) to guide generation. SD maps oîer a lightweight topological skeleton of lanes and intersections, providing suîcient structural guidance for generation while omitting costly HD-level details. Therefore, we can summarize our contributions as follows: â We introduce ControlMap, which, to the best of our knowledge, is among the îrst frameworks for spatially conditioned HD map generation. Our ap- proach transforms publicly available SD map data [19] into lightweight yet structured rasterized control signals, enabling high-îdelity HD map synthe- sis. This mitigates the scalability challenges of costly HD map acquisition while enabling controllable generation of diverse map layouts. â To address the lack of controllability metrics in prior work, we introduce novel metrics evaluating semantic and topological consistency between gen- erated maps, SD inputs, and ground truth. These demonstrate our condi- tioned approach aligns closer to ground truth than unconditional baselines. â We use classiîer-free guidance [11] to serve as a continuous control mecha- nism for conditioned HD map generation, providing aîexible knob to balance adherence to the SD control signal with generative diversity. 2 Related Work 2.1 Scenario Generation Conventional simulators such as CARLA [7], nuPlan [3], and Waymax [9] rely on prerecorded scenes to initialize simulations. This reliance limits scalability and reduces coverage of edge-case traîc scenarios. To address this issue, data-driven scene generation trains models on real map and sensor data [1,3], enabling the synthesis of diverse and realistic scenarios. Diîusion-based approaches [5] and vectorized scene generators [21] are promising directions for producing struc- tured traîc scenes at scale. We propose a taxonomy that categorizes scenario generation methods by their inputs and outputs. Generating Only HD Maps: HDMapGen [17] generates hierarchical autore- gressive lane graphs that capture global connectivity and local geometry. Generating Agents Conditioned on an HD Map: SceneGen [24] places agents sequentially using an autoregressive model conditioned on an HD map and ego context, while SceneControl [16] uses latent diîusion with semantic to- kens to guide the generation of agent positions and trajectories. Jointly Generating Agents and HD Maps: SLEDGE [5], DriveSceneGen [23], and Scenario Dreamer [21] synthesize full traîc scenes by generating HD maps and placing traîc agents onto those maps. However, these methods do not pro- vide explicit control over the spatial layout of the generated HD map. Our work represents a fourth category: HD Map Generation Conditioned on an SD Map Control Signal. To the best of our knowledge, we are theîrst to explore this direction for diîusion-based HD map generation. In addition, 4Marwan Farag, Steîen WĂ€ldele and Yu Yao because our method builds on the SLEDGE framework [5], extending it to joint agent and HD map generation is straightforward. 2.2 Conditioning in Diîusion Models Conditioning provides a mechanism to guide diîusion models toward desired outputs. Prior work uses cross-attention, as in DALL·E [20] and Imagen [22], lightweight adapters [18], and classiîer-free guidance [11]. However, these tech- niques provide limited spatial precision. ControlNet [27] addresses this limita- tion by adding zero-initialized convolutional branches to a U-Netâbased diîusion backbone, encoding structured control signals that enableîne-grained geometric and semantic alignment while preserving the pretrained modelâs generalization. In addition, recent work has shown that ControlNet-style conditioning can be extended beyond U-Net backbones to Diîusion Transformers (DiT) by adding a dedicated control branch whose outputs are injected into the frozen DiT blocks through zero-initialized linear layers, preserving the pretrained behavior at ini- tialization. This has proven eîective for controllable music generation [12] (e.g., melody- and text-guided control) and text-to-image synthesis [4]. Inspired by these works, we integrate ControlNet-style conditioning into the latent-space diîusion transformer of SLEDGE, to enable SD map-based spatial conditioning for HD map generation. In our setup, the SD map control signal guides the global layout of the generated HD map, while the diîusion modelîlls in the lane-level detail. 3 Method 3.1 SLEDGE Baseline We adopt SLEDGE [5], a diîusion-based generative model for traîc scenes and HD maps as our baseline. SLEDGE represents each driving scene as a symbolic SLEDGE Vector, which is rasterized into a multi-channel Rasterized State Im- age (RSI) and encoded by a ResNet-based RVAE into a compact latent map. A Diîusion Transformer (DiT) is then trained to generate these latent repre- sentations, which are decoded back into structured HD map and scene elements by the frozen RVAE decoder. SLEDGE evaluates diîerent DiT backbone sizes, reportingâŒ138M parameters for DiT-B andâŒ487M parameters for DiT-XL. For a more detailed discussion of the original SLEDGE architecture, the interested reader is referred to [5]. In SLEDGE, the HD map generator is only weakly conditioned: the model receives a city label to disambiguate city-speciîc topology, but does not incor- porate any spatial conditioning signal. Speciîcally, the city label is provided as a one-hot vector with dimension equal to one plus the number of cities in the training set. The additional label is used during training for the purpose of city label drop-out and during inference for generating an output which is not associated with any speciîc city from the training set. ControlMap5 3.2 ControlMap: Introducing SD Map Conditioning Just as the DiT-part of the SLEDGE model, our SD map conditioned version operates in the latent space of the RVAE and is trained using the Denoising Diîusion Probabilistic Model (DDPM) objective [10]. An overview of the archi- tecture of our extended ControlMap model based on the DiT-B (âŒ138M param- eters) version is illustrated in Fig. 2. Each SD map is rasterized as a 128Ă128 RGB image and processed through a convolutional control encoder composed of 4 residual ConvâGroupNormâGELU blocks with strided downsampling. The output is a 64Ă8Ă8 tensor augmented with 2D sinusoidal positional embeddings to preserve spatial locality whenîattened into tokens for transformer input. De- tails on SD map image construction are provided in the supplementary material. Conditioning is applied to theîrst M=6 transformer blocks by attaching par- allel trainable ControlNet copies of these blocks, initialized from the pretrained weights. The outputs of the ControlNet [27] blocks are processed through a zero- initialized linear projection and additively injected into the frozen DiT blocks, forming a residual modulation pathway. The remaining blocks stay frozen to pre- serve the pretrained generative prior. After theînal block, the output tokens are projected and unpatchiîed into an 64Ă8Ă8 latent tensor, decoded by the pre- trained RVAE. Following SLEDGE, city-label embeddings are further injected via adaptive normalization [25], enabling conditioning on urban style (e.g., Las Vegas vs. Singapore). 3.3 Metrics for Control Fidelity Evaluation Since previous work did not focus on spatially conditioned HD map generation, existing metrics such as FrĂ©chet Distance [6,17] focus on comparing distribution- level properties. While useful, these metrics are ineîective when evaluating the quality of the spatial conditioning on a sample-level. Therefore, we introduce two additional metrics called Control Recall and Ground-Truth Recall designed to measure the conditionalîdelity and the ground-truthîdelity of the generated samples. First, we rasterize each generated HD map, its associated SD map used as control input during generation and its associated ground truth HD map into an RGB image. All images are then embedded into a 2048-dimensional feature space via a pretrained ResNet-50. Finally, the features of the generated HD maps are compared to the features of SD maps and ground truth HD maps via cosine similarity within batches of B samples. Speciîcally, we deîne Control Recall as the percentage of generated HD maps for which the associated SD map control input ranks among the top-K most similar embeddings in the batch. A higher value indicates that the topology of the generated HD map aligns more faithfully with its conditioning SD map. On the other hand, Ground-Truth Recall is deîned as the percentage of generated HD maps for which the correct ground-truth HD map ranks among the top-K most similar embeddings. A higher value indicates stronger alignment 6Marwan Farag, Steîen WĂ€ldele and Yu Yao between generated and real topologies, reîecting the modelâs ability to reproduce realistic spatial structure under conditioning. For both metrics, we use a batch size of B = 64 and a neighborhood size of K = 3. 4 Experimental Setup and Results 4.1 Training Setup We build upon the nuPlan benchmark [3], a large-scale autonomous driving dataset providing HD maps, dynamic agents, traîc lights, and ego-vehicle states across multiple cities: Las Vegas, Pittsburgh, Singapore and Boston. To assess generalization ability and to avoid contamination of the test data, we deîne our custom training and test splits. The training split contains samples from Las Vegas, Pittsburgh, and Singapore, while the test split contains samples from Boston. Following SLEDGE, we condition the diîusion model on a city label to capture city-speciîc topology priors. For spatial control, we derive SD maps from OpenStreetMap [19] by extracting road layouts and rasterizing them into our control representation. Further details on the test dataset split and SD map rasterization are provided in the supplementary material. Our training setup builds on the multi-stage procedure introduced by SLEDGE [5] and consists of three stages in total. Stage 1: RVAE Latent Space. Following SLEDGE, weîrst train a raster-to- vector RVAE to encode HD map elements into a compact latent representation and decode latents back to structured HD map vectors. This RVAE deînes the latent space in which diîusion is performed: during Diîusion Transformer (DiT) training, ground-truth HD maps are encoded into latents, noise is added, and the diîusion model learns to denoise in this learned latent space. We keep the RVAE architecture and training protocol unchanged from [5]. Stage 2: Unconditional DiT Baseline. Next, we train DiT to generate HD maps without spatial conditioning, as in SLEDGE. Concretely, the DiT takes as input Gaussian noise in the RVAE latent space (and optionally the city label) and learns to synthesize realistic HD map latents that can be decoded by the RVAE. We train both DiT-B (âŒ138M parameters) and DiT-XL (âŒ487M pa- rameters) variants to reproduce the SLEDGE baseline and to obtain a strong pretrained backbone for our controlled model. To improve robustness when the city is unknown, we randomly drop the city-label conditioning with probability 0î2 during training, so the model also learns to generate without an explicit city identiîer. Stage 3: SD-Conditioned ControlNet on DiT. Finally, we introduce ex- plicit spatial conditioning by integrating an SD map-conditioned ControlNet branch into the frozen pretrained DiT backbone from the previous stage. As laid out in Sec. 3.2, we add (i) a ControlNet encoder that processes the rasterized SD ControlMap7 RVAE Decoder HDMap Latent (64x8x8) Patchify 64 patches, each of size 64 Linear Embedding (x12) Transformer Block 0 Transformer Block 1 Transformer Block M Transformer Block N Output Modulation Output ProjectionUnpatchify + + + Tansformer Block 0 Transformer Block 1 Transformer Block M Zero-Linear Zero-Linear Zero-Linear Zero-Linear + Positional Embedding SD Map (3x128x128) Random Noise (64x8x8) ControlNet Encoder with 2D Convolution (x4) SLEDGE's Diffusion Transformer Output Shape: 64x64 Output Shape: 64x768 Output Shape: 64x64 City Label Fig. 2: This îgure shows an overview of our pipeline for controllable HD map genera- tion. While maintaining the entire base diîusion transformer frozen, we add a trainable ControlNet branch [27] that encodes SD maps to the SLEDGE diîusion model [5]. Blue, green, and yellow blocks denote the original SLEDGE backbone, while orange blocks highlight our added ControlNet pathway and its residual injections. Furthermore, city labels are input via the diîusion transformerâs Adaptive Normalization layers [25]. This enables the pipeline to incorporate both spatial control and semantic conditioning for HD map synthesis. Design of control injection follows control mechanisms akin to PixArt [4] and StableAudio-Control [12]. control map, and (i) ControlNet blocks that inject this control signal into the DiT denoising network. During training, each HD map is paired with its corre- 8Marwan Farag, Steîen WĂ€ldele and Yu Yao sponding SD map, and the model learns to use the SD input to guide generation toward the desired road layout while recovering lane-level detail. As in stage 2, we retain the city-label input with dropout. 4.2 Results and Discussion To evaluate our model, we generate one HD map per sample from our Boston- only test split under two settings. We îrst perform unconditional generation, following the original SLEDGE [5] procedure. For each sample, Gaussian noise is drawn in the RVAE latent space and denoised by the DiT backbone without additional conditioning signals and without the city label. The resulting latent representation is then decoded by the RVAE decoder to obtain the înal HD map. This setup serves as our baseline and is evaluated using reproduced DiT-B and DiT-XL checkpoints trained with our own pipeline and consistent with the original SLEDGE experiments. We then perform conditional generation using our ControlMap pipeline. The process remains identical except that the corresponding SD map is retrieved and provided as conditioning input during denoising, allowing us to isolate the eîect of SD-map guidance while keeping all other components unchanged. As shown in Fig. 1 and Fig. 3, the model is able to generate meaningful and well-structured road layouts that ît the given spatial conditioning, although samples from Boston were held-out during training. This demonstrates that our model can generalize to urban contexts that have not been seen before while maintaining guidance alignment. Samples produced by the unguided DiT- B model, on the other hand, show arbitrary patterns and structures that do not follow spatial guidance, as shown in Fig. 3. Evaluating Topological Realism. Following SLEDGE, we use the FrĂ©chet Distance [6,17] over four topological metrics (Connectivity, Density, Reach, and Convenience). These metrics judge the quality of local structural detail and the navigability of the generated lane structure. A lower value indicates closer align- ment between the distributions of generated and reference maps. As shown in Tab. 1, our model achieves superior performance in Reach and Convenience, while falling behind the original SLEDGE model in Connectivity and Density. Fig. 4 illustrates a key reason why the unguided SLEDGE mod- els achieve more competitive FrĂ©chet Distances on Connectivity and Density. Without spatial conditioning, the model has considerable freedom to introduce small stubs, short spurs, and minor branch fragments. These micro-structures are frequent in the ground-truth distribution, and the model learns to reproduce them statistically. As a result, unguided generation inîates the local count of key points, which are the start or end points of lanes or lane connectors. This reduces the measured distributional distance, despite producing less realistic topology overall. In contrast, the ControlNet-guided model follows the coarse SD map skeleton. Extracted SD maps do not contain subtle slip roads, internal connector arcs, and small branching artifacts. Consequently, our ControlNet-guided outputs preserve ControlMap9 BOSBOSBOS NoLabelNoLabelNoLabel Ground- Truth HD Map* SD Map (Control)** Conditionally Generated HD Map via ControlMap* *Colors (red,green) represent lanes with active traffic light. ** Red: Major highways --Green: Primary and Secondary Roads --Blue: Local and Residential Roads NoLabelNoLabelNoLabel Unconditionally Generated HD Map via DiT-B* Fig. 3: We condition on SD maps (second row) that are cheap to obtain (e.g., from OpenStreetMap [19]) to generate diverse HD maps with user-speciîed structure. Our generations (third row) follow the control layout (second row) and recover realistic HD detail consistent with test ground truth (îrst row). While the DiT-B backbone produces visually plausible HD maps (fourth row), the generation is not controllable. BOS denotes ground-truth maps coming from Boston; No Label indicates that the model took no city label as input during generation. cleaner, smoother roads and reduce spurious micro-branches. This leads to more realistic maps, but slightly higher FrĂ©chet Distance on metrics that reward local key point proliferation (Connectivity/Density). On global navigability metrics (Reach and Convenience), ControlNet consistently improves performance. Paths 10Marwan Farag, Steîen WĂ€ldele and Yu Yao Ground-truthSD Map ControlNet Generated DiT-b Generated Fig. 4: This îgure shows an illustration of SD map limitations that impact FrĂ©chet topology metrics. ControlNet must follow the coarse structure of the SD map, which may omit minor connectors (cf. red circles), aîecting topological metrics like connec- tivity and density. On the other hand, unguided models have full îexibility but lack global structural cues, negatively aîecting metrics like convenience and reach. Table 1: FrĂ©chet Distance on topological metrics capturing topological realism ModelConnectivity â Density â Reach â Convenience â SLEDGE DiT-B0.02422.4127 0.55702.7609 SLEDGE DiT-XL-Model0.05302.0578 0.61882.7836 ControlMap DiT-B (ours)0.54532.6184 0.17330.7031 connect correctly and the global îow of traîc becomes structurally coherent, properties that derive naturally from the SD skeleton. These results can partially be explained by the inherent diîerences between SD and HD map representations. SD maps originate from graph-structured data containing rich semantic and topological information, but converting them into rasterized control images inevitably removes part of this structure, including ex- plicit lane connectivity and routing relationships. Furthermore, SD maps often omit îne-grained details such as minor lane connectors, lane boundaries, and traîc-control elements that are present in HD maps. As illustrated in Fig. 4, the ground-truth HD map contains some small connector roads and branching lane structures that are absent in the corresponding SD map. Consequently, the generated output follows the global road geometry provided by the SD condi- tioning signal, while failing to reconstruct certain local topological details that were never present in the input representation. As a result, some discrepan- cies between generated and ground-truth HD maps are expected, particularly in structurally ambiguous regions. Overall, these results reîect a principled design trade-oî: SD map condition- ing prioritizes large-scale realism (Reach/Convenience) at the expense of some local branching complexity (Connectivity/Density), which is not explicitly en- coded in the control signal. ControlMap11 Feature-Space Fidelity and Diversity. In addition to the distribution-level metrics discussed above, SLEDGE also measures two metrics called Improved Precision and Recall [15]. Improved Precision reîects the plausibility of generated samples, while Improved Recall reîects coverage of realistic topology diversity. As shown in Tab. 2, our model with SD Map conditioning achieves consis- tently higher values compared to both variants of the unguided SLEDGE model. A potential explanation is that the structural prior from the SD map helps the model to generate HD maps that more closely aligns with the ground-truth dis- tribution. It reduces mode collapse and improves coverage of rare but realistic road layouts that come from the spatial conditioning. Despite having less than half as many parameters (roughly 230 million vs. 487 million), the ControlNet-enhanced DiT-B model performs better than even the larger DiT-XL SLEDGE model. This demonstrates how the generative quality and structural accuracy of synthetic HD maps are greatly impacted by explicit spatial conditioning through ControlNet, allowing a smaller model to perform better than a much larger unconditioned counterpart. Conditional Fidelity using Recall Metrics. To complement the analysis, we also assess conditional îdelity using Control Recall and Ground-Truth Re- call, our novel metrics introduced in Sec. 3.3. These metrics quantify how closely the generated HD maps match the control input and the ground-truth maps in a ResNet-50 feature space. And as shown in Tab. 2, ControlNet outperforms unguided baselines on both metrics, reîecting stronger alignment with both the conditioning SD map and the ground truth topology reference. Furthermore, under the null hypothesis that matches between SD map/GT HD map and generated HD map occur independently with probability KîB (i.e., random top-K assignment within each batch of size B), the number of matches M in a test set of N samples follows a Binomial distribution: Binom(MîN, KîB). Hence, the p-value for an observed Recall value of r = MîN is given by: P(Râ„ rîH 0 ) = 1â F Binom (M â 1îN, KîB), where F Binom is the cumulative distribution function (one-sided, upper tail). With the parameters of our test setup K = 3, B = 64 and N = 3904, all p- values for the two unguided baselines are above 0î2, which is not statistically signiîcant. On the other hand, for ControlMap, the p-value for Control Recall is 3î1Ă 10 â6 , and the p-value for GT Recall is below machine precision. 4.3 Ablation Experiments Eîect of Classiîer-Free Guidance (CFG). CFG is a sampling technique that ampliîes the inîuence of the conditioning signal at inference time by inter- polating between unconditional and conditional model predictions [11]: Δ guided =Δ uncond + w (Δ cond âΔ uncond )(1) where Δ uncond and Δ cond are unconditional and conditional noise predictions, re- spectively, and w is the guidance scale. To study how stronger conditioning 12Marwan Farag, Steîen WĂ€ldele and Yu Yao Table 2: This table summarizes the evaluation results across the metrics of realism, coverage, and control îdelity. Improved Precision/Recall [15] evaluate generative real- ism and diversity, while Control and Ground-Truth (GT) Recall assess adherence to SD conditioning and ground-truth topology. Number in brackets representp-values, where<Δ indicates that the value is below machine precision. Realism & CoverageControl fidelity ModelPrec. âRec. âCtrl Rec. âGT Rec. â SLEDGE DiT-B0.28230.34710.0476 (0.42)0.0489 (0.28) SLEDGE DiT-XL0.28430.31860.0425 (0.91)0.0497 (0.21) ControlMap DiT-B (ours) 0.43900.4705 0.0630 (3Ă 10 â6 ) 0.2241 (<Δ) aîects spatial adherence and map diversity, we conduct an experiment by gener- ating a series of HD maps with îxed SD map conditioning but using CFG with w set to î0î0, 0î3, 0î5, 0î7, 1î0î. Fig. 5 illustrates how varying the guidance scale w aîects controllability. At w = 0î0, the model ignores the SD map, producing spatially unguided maps. As w increases to 0.3â0.5, generated maps align progressively better with the control input. Roads become more continuous, and intersections align consistently with the SD structure. Higher values (w â„ 0î7) yield near-deterministic adherence to the control map, closely matching the ground-truth topology. Overall, increasing CFG strengthens semantic alignment with the SD map. This provides a simple and eîective knob for trading oî control îdelity against generative diversity. City Label Perturbation. To analyze semantic control, we conduct an ad- ditional experiment by îxing the SD map input but exchanging the city label for a number of samples from Singapore and Las Vegas. These two cities were chosen because they exhibit opposite traîc directions (left-hand vs. right-hand driving), providing a strong probe of what the model learns from the city embed- ding during training, such as lane orientation and road îow patterns. As shown in Fig. 6, the city-label swaps successfully induce city-speciîc changes in the local detail of the generated HD maps (e.g., traîc direction), supporting cross-city generalization by combining city-level priors from the label with spatial layout control from the SD map. 5 Conclusion We present a generative framework for the controllable generation of HD maps that serve as a starting point for traîc scenario simulation. The architecture integrates ControlNet into an unconditioned latent diîusion model developed by [5], opening the possibility to generate maps that not only look realistic but also exhibit high conditionalîdelity and adherence to readily available SD maps. Results show that ControlNet-guided samples closely follow the provided SD map controls. In addition, we present ablation studies which demonstrate the ControlMap13 BOSBOSBOSBOS Ground-Truth HD Map* SD Map Control** CFG Scale=0.0 (No Guidance)* CFG Scale=1.0 (Full Guidance)* CFG Scale=0.3* CFG Scale=0.5* CFG Scale=0.7* *Colors (red,green) represent lanes with active traffic light. ** Red: Major highways --Green: Primary and Secondary Roads --Blue: Local and Residential Roads Fig. 5: This îgure shows the eîect of classiîer-free guidance (CFG) on controllability. Increasing CFG improves topological alignment with the SD map, conîrming control- lable generation. BOS stands for ground-truth maps coming from Boston. 14Marwan Farag, Steîen WĂ€ldele and Yu Yao Ground-Truth HD Map* LAVLAV SGPSGP Generated HD Map* SGPSGP LAVLAV Altering City Label during Generation SD Map** *Colors (red,green) represent lanes with active traffic light. ** Red: Major highways --Green: Primary and Secondary Roads --Blue: Local and Residential Roads Fig. 6: This îgure shows city-label perturbation ablation results. When swapping city labels, the model adjusts lane orientation and structure to reîect the injected cityâs traîc directions (left-handed vs right-handed traîc), demonstrating controllable, se- mantically aware generation. LAV stands for Las Vegas while SGP stands for Singapore. ability to change the strength of the guidance, revealing a controllable trade- oî between high conditioning strength for higher îdelity and low conditioning strength for higher diversity. Cross-city generalization tests further show that the model adapts road geometry and lane structure to diîerent urban contexts. This work is a key enabler for simulation based veriîcation of automated driving systems, since it provides a cheap option to obtain realistic synthetic HD maps which adhere to a speciîc road topology of choice, eîectively removing the dependence of simulation environments on costly real-world HD maps. 6 Limitations and Future Work While SD maps enable eîective spatial control, several limitations remain: 1. Cross-source misalignment: OSM SD maps and nuPlan HD maps may disagree (e.g., missing road segments). This can be mitigated through im- proved preprocessing, including geometric registration, graph-based match- ing, and intersection-aware alignment, as well as îltering ambiguous regions ControlMap15 using consistency thresholds. Integrating complementary modalities, such as aerial or satellite imagery, is another promising direction. 2. Interpretation of Recall Metrics: Control Recall and Ground-Truth Re- call are proxy measures that depend on the rasterization and embedding pipeline rather than directly capturing topological similarity. Their values are sensitive to the feature extractor, batch composition, and top-$K$ selec- tion. Future work will explore topology-aware and simulator-driven evalua- tion protocols for more direct assessment of structural validity. 3. Downstream Simulation Considerations: Further validation is required to assess the usability of generated HD maps in autonomous driving pipelines. Future work must ensure generated maps satisfy planner requirements, in- cluding graph validity, lane connectivity, routing, and agent initialization. Integrating ControlMap into closed-loop simulation environments will en- able evaluation of planner robustness, traîc realism, and controllable rare- scenario generation. References 1. Amini, A., Wang, T.H., Gilitschenski, I., Schwarting, W., Liu, Z., Han, S., Kara- man, S., Rus, D.: Vista 2.0: An open, data-driven simulator for multimodal sensing and policy learning for autonomous vehicles. In: 2022 International Conference on Robotics and Automation (ICRA). p. 2419â2426. IEEE (2022) 2. Asrat, K.T., et al.: A comprehensive survey on high-deînition map: Creation, up- dating processes, and challenges. ISPRS International Journal of Geo-Information 13(7), 232 (2024). https://doi.org/10.3390/ijgi13070232 3. Caesar, H., et al.: nuplan: A closed-loop ml-based planning benchmark for au- tonomous vehicles. In: Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. IEEE (2021), https://arxiv.org/abs/2106.11810 4. Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., et al.: Pixart-: Fast training of diîusion transformer for photorealistic text-to- image synthesis. arXiv preprint arXiv:2310.00426 (2023) 5. Chitta, K., Dauner, D., Geiger, A.: Sledge: Synthesizing driving environments with generative models and rule-based traîc. In: European Conference on Computer Vision. p. 57â74. Springer (2024) 6. Chu, H., Li, D., Acuna, D., Kar, A., Shugrina, M., Wei, X., Liu, M.Y., Torralba, A., Fidler, S.: Neural turtle graphics for modeling city road layouts. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 4522â4530 (2019) 7. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: Carla: An open urban driving simulator. In: Conference on Robot Learning. p. 1â16. PMLR (2017) 8. Elghazaly, G., et al.: High-deînition maps: Comprehensive survey, challenges and future perspectives. IEEE Open Journal of Intelligent Transportation Systems (2023). https://doi.org/10.1109/OJITS.2023.3277564 9. Gulino, C., Fu, J., Luo, W., Tucker, G., Bronstein, E., Lu, Y., Harb, J., Pan, X., Wang, Y., Chen, X., et al.: Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research. Advances in Neural Information Process- ing Systems 36, 7730â7742 (2023) 10. Ho, J., Jain, A., Abbeel, P.: Denoising diîusion probabilistic models. Advances in Neural Information Processing Systems 33, 6840â6851 (2020) 16Marwan Farag, Steîen WĂ€ldele and Yu Yao 11. Ho, J., Salimans, T.: Classiîer-free diîusion guidance. arXiv preprint arXiv:2207.12598 (2022) 12. Hou, S., Liu, S., Yuan, R., Xue, W., Shan, Y., Zhao, M., Zhang, C.: Editing music with melody and text: Using controlnet for diîusion transformer. In: ICASSP 2025âIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). p. 1â5. IEEE (2025) 13. Jeong, J., et al.: Tutorial on high-deînition map generation for automated driving. Sensors 22(18), 7056 (2022). https://doi.org/10.3390/s22187056 14. KĆehlĂk, Ć ., VanĆŸura, M., Skokan, A.: Minimum required accuracy for hd maps. The Journal of Navigation 76(2-3), 238â254 (2023) 15. KynkĂ€nniemi, T., Karras, T., Laine, S., Lehtinen, J., Aila, T.: Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems 32 (2019) 16. Lu, J., Wong, K., Zhang, C., Suo, S., Urtasun, R.: Scenecontrol: Diîusion for con- trollable traîc scene generation. In: IEEE International Conference on Robotics and Automation (ICRA). p. 16908â16914. IEEE (2024) 17. Mi, L., Zhao, H., Nash, C., et al.: Hdmapgen: A hierarchical graph generative model of high deînition maps. In: CVPR (2021) 18. Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., Shan, Y.: T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diîusion models. In: Proceedings of the AAAI Conference on Artiîcial Intelligence. vol. 38, p. 4296â4304 (2024) 19. OpenStreetMap contributors: Planet dump retrieved from https://planet.osm.org. https://w.openstreetmap.org (2025) 20. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: International Conference on Machine Learning. p. 8821â8831. PMLR (2021) 21. Rowe, L., Girgis, R., Gosselin, A., Paull, L., Pal, C., Heide, F.: Scenario dreamer: Vectorized latent diîusion for generating driving simulation environments. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. p. 17207â17218 (2025) 22. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S., Lopes, R., Salimans, T., Ho, J., Fleet, D., Norouzi, M.: Photorealistic text- to-image diîusion models with deep language understanding. Advances in Neural Information Processing Systems 35, 36479â36494 (2022) 23. Sun, S., Gu, Z., Sun, T., Sun, J., Yuan, C., Han, Y., Li, D., Ang, M.H.: Drivescene- gen: Generating diverse and realistic driving scenarios from scratch. IEEE Robotics and Automation Letters (2024) 24. Tan, S., Wong, K., Wang, S., Manivasagam, S., Ren, M., Urtasun, R.: Scenegen: Learning to generate realistic traîc scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 892â901 (2021) 25. Xu, J., Sun, X., Zhang, Z., Zhao, G., Lin, J.: Understanding and improving layer normalization. Advances in neural information processing systems 32 (2019) 26. Yang, M., et al.: Review and challenge: High deînition map technology for au- tonomous driving. Transportation Research Part C: Emerging Technologies (2024). https://doi.org/10.1016/j.trc.2024.104207 27. Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diîusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 3836â3847 (2023)