Paper deep dive
Odyssey: A Closed-Loop Benchmark for Long-Horizon Real-World Driving with Explicit Navigation Routes
Jungho Kim, Hongjae Shin, Seunghoon Yu, Heecheol Yoo, Myeongjun Kim, Jiyong Oh, Donghyuk Kwak, Seunghyeop Nam, Haesung Oh, Hyunju Kim, Hyungchan Cho, Jaehyun Park, Soo Won Seo, Jun Won Choi
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Closed-loop evaluation of end-to-end driving requires continuous rollouts that reveal how earlier decisions affect subsequent driving. However, existing benchmarks evaluate only short segments and fail to capture later consequences. Ambiguous directional commands also obscure the intended navigation objective. We introduce Odyssey, a closed-loop benchmark for long-horizon driving comprising 100 scenarios, each reconstructed from a 100-second nuPlan driving log to preserve the context of navigation maneuvers and traffic interactions. To provide a consistent navigation objective, Odyssey replaces directional commands with explicit standard-definition (SD) map routes that specify which roads to follow, while sensor-based planning determines local driving actions. Throughout these rollouts, diffusion-based refinement of 3DGS-rendered images reduces rendering artifacts along the ego trajectory. To assess how effectively planners follow these routes and prepare for upcoming maneuvers, we introduce SD Route Compliance and Pre-Lane Change Score. These assessments are complemented by RouteDS, which extends the Driving Score with penalties for SD-route deviations and failed lane preparation. We adapt state-of-the-art planners, including vision-language-action (VLA) models, and evaluate their navigation performance using these metrics. Odyssey highlights open questions in route representation and integration for E2E driving. Benchmark code and adapted baselines will be released publicly.
Tags
Links
- Source: https://arxiv.org/abs/2610.06469v1
- Canonical: https://arxiv.org/abs/2610.06469v1
Trouble viewing inline? Open PDF directly →
Full Text
86,140 characters extracted from source content.
Expand or collapse full text
Preprint ODYSSEY: A CLOSED-LOOP BENCHMARK FOR LONG- HORIZON REAL-WORLD DRIVING WITH EXPLICIT NAVIGATION ROUTES Jungho Kim ∗ Hongjae Shin ∗ Seunghoon Yu ∗ Heecheol Yoo ∗ Myeongjun Kim ∗ Jiyong Oh ∗ Donghyuk KwakSeunghyeop NamHaesung OhHyunju KimHyungchan Cho Jaehyun ParkSoo Won SeoJun Won Choi † Seoul National University, Republic of Korea https://snu-adr.github.io/Odyssey-Page/ Low Quality RenderingShort Scenario 0s 20s Artifacts · blur 3DGaussian Only (a) Limitations of Previous Closed-loop Benchmarks 0s20s40s60s80s (b) High Quality Rendering Gaussian Splat Closed-Loop Simulation Ambiguous Routing ? Change lane TurnExit Curve RIGHT COMMAND + Diffusion Long Scenario SD Map Routing 100s Figure 1: Odyssey framework. (a) Previous closed-loop benchmarks suffer from limited scenario horizons, ambiguous command-based routing, and artifacts from 3DGS-only rendering. (b) Odyssey evaluates E2E planners over continuous closed-loop rollouts based on 100-second driving sequences with explicit SD-route guidance and diffusion-refined 3DGS observations, assessing sustained safe navigation toward the intended destination. ABSTRACT Closed-loop evaluation of end-to-end driving requires continuous rollouts that re- veal how earlier decisions affect subsequent driving. However, existing bench- marks evaluate only short segments and fail to capture later consequences. Am- biguous directional commands also obscure the intended navigation objective. We introduce Odyssey, a closed-loop benchmark for long-horizon driving comprising 100 scenarios, each reconstructed from a 100-second nuPlan driving log to pre- serve the context of navigation maneuvers and traffic interactions. To provide a consistent navigation objective, Odyssey replaces directional commands with explicit standard-definition (SD) map routes that specify which roads to follow, while sensor-based planning determines local driving actions. Throughout these rollouts, diffusion-based refinement of 3DGS-rendered images reduces rendering artifacts along the ego trajectory. To assess how effectively planners follow these ∗ Equal contribution. † Corresponding author. 1 arXiv:2610.06469v1 [cs.RO] 5 Oct 2026 Preprint routes and prepare for upcoming maneuvers, we introduce SD Route Compliance and Pre-Lane Change Score. These assessments are complemented by RouteDS, which extends the Driving Score with penalties for SD-route deviations and failed lane preparation. We adapt state-of-the-art planners, including vision-language- action (VLA) models, and evaluate their navigation performance using these met- rics. Odyssey highlights open questions in route representation and integration for E2E driving. Benchmark code and adapted baselines will be released publicly. 1INTRODUCTION (a) Long-Term Effects of Early Decisions (b) Incorrect Command Flips : Right: Straight: Left 0s10s20s Unseen Cannot Assess Long-Term Impact✗ decision → outcome 34.0s 20.0s 0s20s40s 60s 80s 100s 47s 13s 47s 13s Lane Change Turn Right Figure 2: Limitations of existing evaluation. (a) Short rollouts cannot assess whether a prior lane change enables a later right turn. (b) In- consistent command flips obscure navigation intent along a continuous trajectory. A central challenge in autonomous driving is reli- able navigation through complex traffic environ- ments that ensure safety and comfort, as well as efficient progress toward an intended destina- tion. With the growing availability of large-scale driving data, planning-oriented end-to-end (E2E) paradigms have gained prominence (Hu et al., 2023; Liao et al., 2025; Kim et al., 2026; Kirby et al., 2026), in which driving systems directly map onboard sensor observations to driving de- cisions. Such planners should be evaluated by their ability to sustain safe driving as the vehicle interacts with its environment over time. Since open-loop evaluation independently compares de- cisions at individual time steps against recorded driving logs, it cannot capture their downstream effects. Accounting for downstream effects via closed-loop evaluation is essential to assess driv- ing performance under evolving traffic conditions. For E2E planners, closed-loop evaluation requires sensor observations that reflect the consequences of the ego vehicle’s actions.Neural rendering provides this feedback by reconstructing 3D real- world scenes from driving logs and rendering new observations from the simulated ego pose. Early approaches relied on neural radiance fields (NeRFs) (Yang et al., 2023; Tonderski et al., 2024; Yang et al., 2024), whose volumetric rendering in- curs substantial computational cost. To reduce this cost, recent methods adopt 3D Gaussian Splatting (3DGS) (Zhou et al., 2024b; Hess et al., 2025; Zhou et al., 2024a; Chen et al., 2025) for faster rendering while maintaining high visual fidelity. These developments have supported growing re- search on neural-rendering-based closed-loop benchmarks (Ljungbergh et al., 2024; Zhou et al., 2025; NVIDIA et al., 2025) for E2E planners. However, existing benchmarks are limited in assess- ing a planner’s ability to handle long-term driving tasks (see Figure 1 (a)). One limitation of existing closed-loop benchmarks (Ljungbergh et al., 2024; Zhou et al., 2025) is their short evaluation horizon, typically limited to approximately 20 seconds. Such short rollouts are insufficient to fully assess the navigation capabilities of E2E planners, particularly when navigation requires a sequence of coordinated maneuvers. For example, as shown in Figure 2 (a), completing an upcoming right turn may require changing lanes in advance. However, the evaluation may termi- nate before the turn occurs, making it hard to assess the planner’s ability to complete the maneuver sequence. Furthermore, high-level meta-action commands used in many E2E benchmarks (Cae- sar et al., 2020; Dauner et al., 2024; Cao et al., 2025; Zhou et al., 2025; Xu et al., 2026) provide ambiguous guidance for navigation in real-world driving scenarios. As illustrated in Figure 1 (a), a directional command such as “turn right” may correspond to multiple possible maneuvers. For example, if two right exits occur in succession, the command alone does not indicate which exit the planner should take. Annotating these commands in real-world driving datasets may not accurately 2 Preprint reflect the driver’s intended maneuver (OpenScene Contributors, 2023), as shown in Figure 2 (b). Reliable long-horizon evaluation therefore requires more explicit navigation guidance than high- level meta-action commands alone can provide. Another challenge is maintaining high rendering quality throughout long-horizon closed-loop simu- lations. When the ego vehicle deviates from the trajectory recorded during data collection, rendering quality may deteriorate because novel viewpoints lack sufficient observational support. The result- ing artifacts can distort the visual input to E2E planners, influence their subsequent decisions, and undermine the reliability of both planning and evaluation. In addition, existing evaluation setups do not accurately model changes in traffic signals or pedestrian motion, further limiting their ability to represent realistic driving scenarios. In this paper, we introduce Odyssey, a real-world closed-loop benchmark for evaluating E2E driv- ing models over long-horizon rollouts. Odyssey uses neural rendering to reconstruct 3D scenes and support realistic closed-loop simulations. Each scenario includes an explicit navigation route repre- sented on a standard-definition (SD) map. By combining long-horizon evaluation with explicit route guidance, Odyssey enables a comprehensive assessment of models’ ability to navigate complex road structures safely—a capability that existing benchmarks have not adequately evaluated. For these extended rollouts, Odyssey builds on the explicit geometric representation of 3D Gaussian Splatting (Kerbl et al., 2023), followed by diffusion-based refinement for improving visual fidelity. Using OmniRe (Chen et al., 2025), we reconstruct continuous 100-second driving environments with rigid vehicles and deformable pedestrians whose motion is controlled independently of the static background. We further apply its time-conditioned deformation field to traffic lights to model changes in their color states throughout the sequence. To reduce blur and recover missing detail at novel ego poses, we apply single-step diffusion refinement (Wu et al., 2025; NVIDIA, 2025b) to the rendered camera images without modifying the underlying scene representation or simulated state. Odyssey provides navigation routes as explicit guidance for E2E planners. By specifying the in- tended sequence of road segments, they resolve the ambiguity inherent in directional commands and eliminate the need to determine when to issue individual high-level meta-action commands. For each scenario, we construct an SD-map route from trip data of the data-collection fleet. How- ever, this road-level representation encodes only coarse road-level topology and does not specify lane-level structure or the precise goal location that a driving model should reach. E2E planners must therefore interpret the high-level route jointly with the scene perceived by onboard sensors and translate it into a concrete, scene-consistent navigation goal. This requires the planner to reason about how an abstract road-level route is translated to appropriate lane-level maneuvers within the current road geometry. Odyssey establishes a foundation for evaluating such reasoning capability of E2E planners. We design the closed-loop benchmark to enable systematic comparison of different E2E planners. We curate 100 scenarios from the raw nuPlan data (Karnchanachari et al., 2024), covering sequences of complex navigation maneuvers and diverse road topologies. We further introduce two evaluation metrics, termed Pre-Lane Change Score and RouteDS, which assess whether the vehicle is appro- priately positioned for upcoming navigation maneuvers, as well as driving safety and progress along the intended route. We evaluate a set of baseline planners adapted from recent state-of-the-art E2E and vision-language-action (VLA) models, incorporating navigation routes as additional input. Our main contributions are as follows: • We introduce Odyssey, a high-fidelity real-world closed-loop benchmark for evaluating whether E2E planners can sustain safe and goal-directed driving under explicit route guid- ance over long-horizon rollouts. • We evaluate planners over continuous rollouts based on recorded 100-second sequences, with explicit SD-map routes that resolve ambiguities in high-level driving commands and enable consistent assessment of navigation performance across complex road topologies. • We develop a closed-loop simulation framework that combines the structural consistency of 3DGS with diffusion-based refinement to improve rendering fidelity when ego-vehicle motion reveals poorly reconstructed or previously unobserved regions. • We will release the benchmark, evaluation code, and baseline results to enable reproducible comparison and facilitate further research on long-horizon closed-loop evaluation for au- tonomous driving. 3 Preprint Table 1: Comparison of driving benchmarks and simulation frameworks.✓, △, ×, and - denote support, partial support, no documented support, and not applicable, respectively. Benchmark Real-log scenarios Sensor closed-loop Realistic images Navigation input Recorded sequence duration Rendering type Traffic-light state Pedestrian motion nuScenes✓×✓Commands-×-- NAVSIM✓×✓Commands-×-- NAVSIMv2✓×✓Commands-Offline 3DGS-- WOD-E2E✓×✓Commands-×-- nuPlan✓×HD route15 s×-- Waymax✓×HD route9 s×-- MetaDrive△✓×Target pointsUser-setRasterization✓ CARLA Leaderboard ×✓×Target points ∼ 3–5 minRasterization✓ Bench2Drive×✓×Target points ∼ 15–30 sRasterization✓ NeuroNCAP✓Commands20 sNeRF× HUGSIM✓Commands ≤ 20 s3DGS× DriveArena△✓Commands20 sDiffusion× Odyssey (ours)✓SD route100s3DGS + diffusion✓ 2RELATED WORK Open-loop displacement metrics overlook how planning actions affect subsequent driving (Caesar et al., 2020; Hu et al., 2022; 2023). NAVSIM introduces simulation-based driving metrics (Dauner et al., 2024), and NAVSIM v2 adds pre-rendered feedback (Cao et al., 2025). Continuous replanning further requires observations from the ego poses reached after executing successive actions. Neural simulators provide this feedback in reconstructed environments (Yang et al., 2023; Tonderski et al., 2024; Ljungbergh et al., 2024; Zhou et al., 2025; NVIDIA et al., 2025). However, short evaluation sequences can end before earlier lane choices affect later navigation (Ljungbergh et al., 2024; Zhou et al., 2025). Assessing these consequences also requires a consistent navigation objective. Directional com- mands can leave consecutive branches ambiguous (Hu et al., 2023; Jiang et al., 2023; Dauner et al., 2024; Zhou et al., 2025), obscuring whether route deviations reflect planning errors or insufficient guidance. Nearby target points provide spatial guidance but limited context for upcoming route choices (Dosovitskiy et al., 2017; Jia et al., 2024). Odyssey addresses these temporal and navi- gational limitations through extended closed-loop rollouts with explicit SD-map routes, assessing whether successive decisions support safe progress along the intended road sequence. 3ODYSSEY Odyssey provides a closed-loop simulation platform for evaluating long-horizon navigation along explicit routes on an SD map. Within each scene reconstructed from a 100-second driving log, the planner predicts a local driving trajectory based on camera observations and route guidance. The simulator advances the ego vehicle along the predicted trajectory and uses a neural rendering engine to generate new camera observations from its updated pose. We evaluate planner performance in terms of driving safety, route progress, adherence to the road-level route, and lane selection before intersections. 3.1SCENARIO CURATION Odyssey comprises 100 driving scenarios curated from nuPlan (Karnchanachari et al., 2024). To capture diverse driving situations over extended sequences, we combine RefAV (Davidson et al., 2026)-based tagging with human review of navigation maneuvers and traffic interactions. The dis- tribution of individual tags is reported in Figure 3 (a). The selected scenarios contain multiple maneuver and interaction tags, with the number of tags per scenario shown in Figure 3 (b). The se- lected set includes 13 scenarios involving traffic lights and 47 involving pedestrians, covering both signal compliance and safety around vulnerable road users. Within this set, we prioritize scenarios with complex road layouts and sequences of naviga- tion maneuvers. Figure 3 (c) shows representative routes involving successive turns, branching roads, and lane changes. In particular, scenarios with a lane change followed by a turn allow us 4 Preprint 1 8 12 22 27 15 10 4 1 10 20 30 0 2345678910 (b) Number of Tags Per Scenario (c) Representative Scenario Routes Lane change Lane change Lane change (a) Sequence Count Per Tag 72 70 68 67 66 63 55 47 46 41 30 13 0 20 40 60 80 Ego maneuver Road user/Object Traffic-light Figure 3: Scenario composition and exam- ple routes in Odyssey. to assess whether the planner changes lanes in prepa- ration for the turn. The set includes 49 such scenarios, 16 of which have more than 20 seconds between the start of the lane change and the start of the turn. A single 20-second evaluation window cannot capture the onset of both maneuvers in these cases. 3.2SD MAP ROUTING SD maps represent the road network through coarse polylines and connectivity at intersections and branches. A navigation route specifies the sequence of roads to follow. To construct each route, we match the full logged ego trajectory to OpenStreetMap (OSM) using HMM-based map matching (Newson & Krumm, 2009), yielding a single polyline from the scenario start to the destination. We manually check the matched roads and route continuity against the logged trajectory and SD map. During closed-loop simulation, we update the route segment provided to the planner according to the ego vehicle’s current po- sition. 3.3CLOSED-LOOP SIMULATIONS 3D GAUSSIAN SCENE RECONSTRUCTION. Odyssey reconstructs each 3D sequence using Om- niRe (Chen et al., 2025), with rigid transformations for vehicles, SMPL-based deformation (Loper et al., 2015) for pedestrians, and a learned deformation field for other actors. Traffic lights have fixed geometry and position, with time-dependent spherical-harmonic coefficients and opacity cap- turing signal changes. To stabilize road geometry, we incorporate ground modeling inspired by HUGSIM (Zhou et al., 2025), representing the road surface separately using thin Gaussian disks initialized from LiDAR. Their heights and normals follow a fixed LiDAR-derived height field as we optimize their in-plane geometry and appearance. Opacity regularization suppresses overlapping background Gaussians near the road surface. When training these scene representations over long sequences, uniform image sampling concen- trates updates in regions where the recording vehicle stops or moves slowly. We therefore apply coverage-aware sampling, weighting each view by aggregating inverse observation counts over its visible scene voxels. This increases sampling probabilities for views covering less-observed regions. RECONSTRUCTION-CONDITIONED VISUAL REFINEMENT. Rendering from ego poses beyond the logged viewpoints can expose blur and missing detail in poorly observed regions. We reduce these artifacts using Fixer (NVIDIA, 2025b), which adapts the diffusion-based refinement of Di- fix3D+ (Wu et al., 2025) to a Cosmos backbone (NVIDIA, 2025a). Fixer refines each image in a single denoising step at a fixed timestep without adding random noise. We fine-tune it offline on paired degraded renderings and recorded images, following the data preparation strategy of Difix. During simulation, the model refines rendered observations before they are passed to the planner. EGO VEHICLE CONTROL. The ego vehicle tracks the planner’s trajectory at 10 Hz using a linear quadratic regulator (LQR) (Anderson & Moore, 2007). At each step, the LQR computes acceleration and steering rate that a bicycle model (Kong et al., 2015) uses to update the ego state. 3.4TRAFFIC SIMULATION TRAFFIC CONFIGURATION. Following nuPlan (Caesar et al., 2021), we evaluate planners under non-reactive traffic, where other actors replay recorded motion, and reactive traffic, where surround- ing vehicles use IDM-based longitudinal control (Treiber et al., 2000). Vehicles enter the simulation as they appear in the logs. At unprotected intersections, newly introduced vehicles can create con- 5 Preprint Progress Gap Simulated Ego Logged Ego Fixed-Time Playback HitHit HitMiss Hit Miss Miss Compatible Lane10 m Before Stop Line Stop Line Section-Based Triggering (Ours) (a) Example of Section-based Trigger(b) Pre-Lane Change Evaluation Section 1 · Active Section 2 · Inactive Miss Traffic Interaction Traffic Interaction Figure 4: Traffic triggering and pre-lane change evaluation. (a) When the simulated ego is slower than the logged ego, fixed-time playback (top) places traffic interactions ahead of it. By activating traffic as the ego approaches a section and keeping subsequent sections inactive, section- based triggering (bottom) aligns traffic interactions with ego progress. (b) Pre-Lane Change requires both driving in route-compatible lanes throughout the final 10 m and crossing the stop line in a compatible lane. The four examples illustrate cases that satisfy both conditions, only one, or neither. flicts that longitudinal control alone cannot resolve. We therefore add intersection-handling logic inspired by CARLA’s Traffic Manager (Dosovitskiy et al., 2017) to prevent deadlocks. SECTION-BASED TRAFFIC TRIGGERING. When the ego vehicle travels faster or slower than the logged vehicle, traffic playback on the original timeline can cause it to miss recorded interactions. We therefore introduce section-based traffic triggering to align traffic activation and traffic light signal timing with ego progress in both non-reactive and reactive settings. Since traffic interactions are concentrated around intersections, we place section boundaries between intersections. As the ego vehicle approaches a section, we activate its traffic using a fixed time offset determined by the difference between the logged and simulated ego entry times. The same offset is applied to the section’s traffic light signals, which then advance through their recorded states at the original rate. Figure 4 (a) compares this approach with traffic playback on the original timeline. 3.5EVALUATION PROTOCOL We introduce SD Route Compliance, Pre-Lane Change Accuracy, Pre-Lane Change Score, and RouteDS to assess navigation performance. We also report Efficiency and Comfortness. SD ROUTE COMPLIANCE (SDC). We map-match the executed ego trajectory to assess adherence to the provided SD-map route. A rollout scores 1 if all matched road segments belong to the route and 0 otherwise, even if the vehicle stays on drivable roads. SDC is the mean score across rollouts. PRE-LANE CHANGE ACCURACY (PLCA) & SCORE (PLCS). Long-horizon navigation re- quires timely lane positioning to follow the route through successive intersections. We assess this ability via PLCA and PLCS at intersections where continuing along the route requires specific ap- proach lanes. PLCA averages scores over evaluated intersections encountered across all rollouts. PLCS averages scores over all evaluation intersections along the full scenario route, counting un- encountered intersections as 0 to account for incomplete rollouts. At each stop line, the vehicle scores 1 if it crosses in a route-compatible lane and has maintained compatible lane occupancy for the preceding 10 m. A later entry scores 0.5, and crossing outside a compatible lane scores 0 (Fig. 4 (b)). ROUTEDS. We extend the Driving Score of Bench2Drive (Jia et al., 2024) with penalties for SD- route deviations and failed pre-lane change evaluations. These additions complement route comple- tion and penalties for at-fault collisions (Dauner et al., 2024), lane deviation, and red-light violations. For each scenario, the route completion ratio is multiplied by a cumulative penalty factor initialized to 1, with multipliers in [0, 1] applied for infractions. Each failed pre-lane change evaluation incurs a separate penalty. We report the mean RouteDS across scenarios. Detailed metric definitions and penalty values are provided in the Appendix. 6 Preprint Table 2: Ablation study of reconstruction components on Odyssey. KID in and KID ex evaluate interpolated and ex- trapolated views, respectively. MethodPSNR↑ SSIM↑ LPIPS↓ KID in ↓ KID ex ↓ Latency (ms/image)↓ OmniRe (Baseline)24.340.7880.39929.552.217.8 + Color Correction24.980.7930.39225.447.018.1 + Coverage-aware Sampling25.220.7960.38224.142.818.1 + Ground Modeling25.200.7940.37620.132.832.4 + Pretrained Fixer25.000.8010.33014.617.639.1 + Fixer Fine-tuning (Ours)25.340.7860.2335.68.939.1 Table 3: Effect of Fixer refinement on perception and driving performance. Observation Open-loopClosed-loop mIoU↑mIoU↑ RouteDS↑ GT Image35.4– 3D GS Only31.526.053.80 + Diffusion31.928.252.62 + Fine-tuning33.729.454.43 Ground TruthGS OnlyGS + Diffusion (Pretrained)GS + Diffusion (Fine-tuned) Figure 5: Effect of diffusion refinement at a novel viewpoint. The first image is the recorded image, and the other three are rendered from a viewpoint shifted 2 m to the left of the recorded cam- era position. GS-only rendering exhibits distorted road textures at this novel viewpoint. Pretrained Fixer improves visual quality but introduces spurious curb-like structures on the road. Fine-tuning on Odyssey suppresses these artifacts and yields smoother roads and clearer lane markings. 4IMPLEMENTATION DETAILS 4.1SIMULATOR IMPLEMENTATION DETAILS We reconstruct the entire 100-second sequence as a single scene using OmniRe (Chen et al., 2025) and accelerating optimization with FastGS (Ren et al., 2026). Each scene contains 8,000 images captured at 10 Hz across eight cameras and is optimized for 140,000 iterations. We additionally apply per-image affine color corrections, as used in MTGS (Li et al., 2025a), to compensate for exposure variations. We fine-tune pretrained Fixer (NVIDIA, 2025b) on paired degraded renderings and corresponding recorded images, following the data preparation strategy of Difix (Wu et al., 2025). Details of training data generation and fine-tuning are provided in the Appendix. 4.2SD-ROUTE-CONDITIONED BASELINES We adapt existing planners to SD-route guidance and compare their closed-loop driving performance over long-horizon rollouts. The baselines comprise LTF (Chitta et al., 2023), DiffusionDrive (Liao et al., 2025), DrivoR (Kirby et al., 2026), SafeDrive (Kim et al., 2026), and ReCogDrive (Li et al., 2026b). All baselines predict a 4-second future trajectory sampled at 2 Hz and are trained on the NAVSIM training set (Dauner et al., 2024). At each planning step, we sample 120 route points at 1-meter intervals ahead of the ego’s projection onto the SD-route polyline. Their positions, directions, and headings are expressed in the ego frame to provide navigation guidance in place of driving commands. For vision-based planners, a linear projection and max pooling over groups of five consecutive points produce 24 route tokens. Fourier embeddings of group starting poses and learned order embeddings encode their spatial placement and sequence. Each planning decoder layer attends to these tokens as keys and values through an additional cross-attention module before its feed-forward network. For ReCogDrive, we summarize the route into a single token added to the ego-status features and supply a route-derived navigation sentence to the VLM. Model-specific adaptations are detailed in the Appendix. Dedicated reasoning mechanisms that relate SD-map routes to the perceived scene and required driving maneuvers are left to future work. 7 Preprint Table 4: Comparison of baseline planners with and without SD-route guidance under non-reactive and reactive traffic. Eff. and Comf. denote Efficiency and Comfortness, respectively. Bold and underlined values indicate the best and second-best results in each metric, respectively. Method SD-Route Guidance Odyssey Non-reactiveOdyssey Reactive NAVSIM Open Loop RouteDS SDC PLCA PLCS Eff. Comf.RouteDS SDC PLCA PLCS Eff. Comf.PDMS Vision-based planners LTF × 16.25166.050.363.910016.35463.549.176.19983.8 ✓20.19557.156.168.29623.09353.652.678.09385.6 DiffusionDrive ×25.65978.863.761.410024.25977.063.175.710085.8 ✓35.910082.080.563.49941.010081.280.271.59986.9 DrivoR × 20.65075.562.8104.74425.55376.662.8133.44793.1 ✓36.59072.866.084.56041.48971.866.0107.46492.2 SafeDrive ×27.54179.363.769.44730.44578.862.886.34090.8 ✓44.48275.069.868.34448.97973.869.581.34990.6 VLA-based planners ReCogDrive ×15.47175.759.076.910020.36277.158.793.79990.8 ✓21.58374.455.859.39826.38475.656.775.59890.4 5EXPERIMENTS 5.1SCENE RECONSTRUCTION QUALITY V GT OmniReOurs Figure 6: Traffic-light modeling. Our de- formation field reproduces the red-to-green transition, which OmniRe fails to recover. We evaluate reconstruction quality using PSNR, SSIM, LPIPS, and KID (Bi ́ nkowski et al., 2018). For novel viewpoints without paired ground truth, we follow HUGSIM (Zhou et al., 2025) and use KID to measure the squared maximum mean dis- crepancy between Inception features extracted from rendered and recorded images. Table 2 shows cumu- lative reductions in LPIPS and KID as color correc- tion, coverage-aware sampling, ground modeling, and diffusion refinement are incorporated. The vi- sual improvements are illustrated in Figure 5, where Fixer fine-tuning suppresses road-surface artifacts and improves lane clarity. Alongside image quality, the table reports per-image latency measured on an NVIDIA B200 GPU. We also examine how reconstruction captures changes in traffic-light appearance. Figure 6 shows that our time-conditioned appearance model reproduces the red-to-green transition missed by Om- niRe. The rendered signals provide visual cues for evaluating planner responses to traffic lights during closed-loop rollouts. 5.2EFFECT OF DIFFUSION REFINEMENT ON CLOSED-LOOP PLANNING Table 3 compares visual refinement settings with the reconstructed scene and SafeDrive checkpoint fixed, using IL scoring and safety filtering. Open-loop mIoU is evaluated along the logged trajectory. For closed-loop mIoU, all variants use identical camera poses from the rollout with fine-tuned re- finement. Pretrained Fixer improves mIoU, but these gains are accompanied by a slight decrease in RouteDS. Fine-tuning on Odyssey further improves mIoU in both settings, brings open-loop mIoU closer to that obtained with recorded images, and raises RouteDS above the GS-only baseline. These results support adapting the refinement model to Odyssey’s reconstruction artifacts to improve both perception and closed-loop driving performance. 5.3EFFECT OF SD-ROUTE GUIDANCE ON CLOSED-LOOP DRIVING Table 4 compares planners with and without SD-route guidance under non-reactive and reactive traffic. Providing SD routes improves both SDC and RouteDS across all evaluated planners in both 8 Preprint Table 5: Effects of safety scoring and reinforcement learning on driving performance under non- reactive and reactive traffic and on NAVSIM. All models use SD-map route guidance. Configuration Odyssey Non-reactiveOdyssey ReactiveNAVSIM RouteDS SDC PLCA PLCS Eff. Comf.RouteDS SDC PLCA PLCS Eff. Comf.PDMS SafeDrive Safety Scoring44.48275.069.868.34448.97973.869.581.34990.6 IL Scoring53.99577.174.466.510055.39478.074.475.710089.5 IL Scoring + Safety Filtering 54.49676.772.765.310056.19677.774.178.310089.5 ReCogDrive IL + GRPO Fine-tuning21.58374.455.859.39826.38475.656.775.59890.4 IL only37.79779.473.858.910042.39679.675.070.410086.8 Driving CommandSD Route Guidance Off Route Command Flipping Right Straight Left On Route Figure 7: Driving commands vs. SD-route guidance. With driving commands, SafeDrive deviates from the intended route when the com- mand flips. With SD-route guidance, it follows the intended route. settings. Figure 7 illustrates this effect through SafeDrive, which deviates from the intended route when the driving command flips but fol- lows it with SD-route guidance. Even with explicit route guidance, planners dif- fer in their ability to combine route adher- ence with safe driving. DiffusionDrive achieves higher SDC than SafeDrive, but SafeDrive ob- tains higher RouteDS, which additionally ac- counts for route progress and driving infrac- tions. NAVSIM PDMS also does not consis- tently reflect these differences in closed-loop performance.For example, DrivoR achieves higher PDMS than SafeDrive but lower RouteDS in both traffic settings. These comparisons mo- tivate evaluating how planners follow the intended route over successive decisions, including the driving situations resulting from their own actions. 5.4DO OPEN-LOOP GAINS TRANSFER TO CLOSED-LOOP DRIVING? Recent research has explored trajectory scoring and reinforcement learning as ways to improve driv- ing safety. In NAVSIM-based training, trajectory scorers learn to predict simulation-based driving scores, while reinforcement learning uses these scores as rewards. Both objectives evaluate individ- ual trajectories in an open-loop setting, without directly optimizing how the planner responds to new observations after execution. Table 5 shows the limits of this approach. For SafeDrive, IL scoring outperforms safety scoring despite similar PDMS values. Safety filtering further improves performance by selecting the highest IL-scored trajectory among candidates that pass safety checks. For ReCogDrive, GRPO improves PDMS but lowers closed-loop performance relative to IL-only training. These results suggest that optimizing individual trajectories does not ensure better continuous driving. Odyssey closes this gap by evaluating planners on the consequences of their own decisions, not on isolated trajectories. 5.5WHY EVALUATE OVER LONG HORIZONS? Short-horizon evaluation assesses immediate driving behavior but may not reveal whether a planner can sustain performance over successive decisions. Figure 8 (a) shows that RouteDS decreases as the evaluation horizon grows, highlighting the challenge of sustaining safe, route-following behavior over longer rollouts. Some decline is expected as infraction penalties accumulate, but the relative performance of planners also changes. DrivoR achieves higher RouteDS than SafeDrive at short horizons but falls below it as driving continues. This rank reversal shows that conclusions drawn from short rollouts may not hold over longer ones, motivating evaluation of successive decisions and their downstream consequences. 9 Preprint Collision Score Human RouteDS 1.0 0.9 0.8 0.7 0.6 0.5 0.4 1020304050100 Without SD route With SD route SD route Compliance (SDC) Pre - Lane Change Accuracy (PLCA) 0.40.50.60.70.80.91.00.3 Without SD route With SD route ReCogDrive 0.6 0.7 0.8 1.0 0.5 Human (b) SDC vs. PLCA 1020406080100 RouteDS Evaluation horizon (s) 70 60 50 40 30 20 SafeDrive IL-applied Setting Original Setting Rank-switching Point (a) Horizon Effect (c) RouteDS vs. Collision Figure 8: Effects of evaluation horizon and route guidance under non-reactive traffic. (a) RouteDS decreases with longer evaluation horizons, and model rankings change at the highlighted crossings. Solid and dashed curves show the original and IL-based settings, respectively. (b) SD- route guidance improves SDC but reduces PLCA for most planners. (c) RouteDS improves with SD-route guidance, but collision scores decrease for most planners. Higher collision scores indicate fewer collision penalties. 5.6WHAT IS NEEDED BEYOND ROUTE GUIDANCE? Figure 8 (b) shows that SD-route guidance improves SDC for all planners, but PLCA decreases for four of the five, with only DiffusionDrive showing an increase. The gains in road-level route adherence therefore do not extend to timely lane positioning for most evaluated planners. Translating the intended road sequence into appropriate lane choices before intersections remains a challenge. Beyond lane preparation, following a route also requires safe interaction with surrounding traffic. Figure 8 (c) shows that SD-route guidance raises RouteDS for every planner, but the collision score decreases for four of the five. The higher overall scores therefore do not consistently coincide with fewer collision penalties. These results highlight remaining challenges in lane preparation and traffic interaction under explicit route guidance. How planners can combine SD-map routes with perceived road geometry and surrounding traffic to address these challenges remains an open direction for future research on Odyssey. 6FUTURE WORKS Surrounding vehicles can appear blurred or distorted when viewed from angles not covered by the driving logs. Using diffusion guidance during 3D Gaussian Splatting optimization could improve reconstruction in these regions. Future work could also improve the temporal consistency of diffu- sion refinement. Beyond visual quality, adversarial agents could introduce more challenging traffic interactions for safety evaluation. 7CONCLUSION We introduced Odyssey to evaluate whether E2E planners can sustain safe navigation under explicit route guidance. Odyssey provides SD-map routes and continuous sensor feedback in environments reconstructed from 100-second real-world driving logs. This setting supports closed-loop evaluation under a consistent navigation objective. Our experiments show that planner rankings change with the evaluation horizon and that improvements in open-loop metrics do not necessarily translate into better closed-loop driving. Explicit route guidance improves road-level adherence, but these gains do not consistently extend to lane preparation or collision avoidance. Odyssey supports research on these remaining challenges by evaluating how planners use route guidance and sensor observations to select appropriate lanes and interact safely with surrounding traffic over extended closed-loop rollouts. 10 Preprint REFERENCES Brian D. O. Anderson and John B. Moore. Optimal Control: Linear Quadratic Methods. Dover Publications, 2007. Mikołaj Bi ́ nkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), p. 11618–11628. IEEE, 2020. Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021. Wei Cao, Marcel Hallgarten, Tianyu Li, Daniel Dauner, Xunjiang Gu, Caojun Wang, Yakov Miron, Marco Aiello, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, and Kashyap Chitta. Pseudo-simulation for autonomous driving. In Proceedings of the 9th Con- ference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, p. 4709– 4722. PMLR, 2025. Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. International Journal of Computer Vision, 2026. Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo De Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, et al. Omnire: Omni urban scene reconstruc- tion. In International Conference on Learning Representations, volume 2025, p. 85508–85527, 2025. Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1290–1299, 2022. Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2023. Felipe Codevilla, Matthias M ̈ uller, Antonio L ́ opez, Vladlen Koltun, and Alexey Dosovitskiy. End- to-end driving via conditional imitation learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), p. 4693–4700. IEEE, 2018. Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, and Kashyap Chitta. NAVSIM: Data-driven non-reactive autonomous vehicle simulation and benchmarking. In Advances in Neural Information Processing Systems, volume 37, 2024. Cainan Davidson, Deva Ramanan, and Neehar Peri. Refav: Towards planning-centric scenario min- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 21537–21548, 2026. Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, volume 78, p. 1–16, 2017. Leon Gatys, Alexander S. Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems, volume 28, 2015. Cole Gulino, Justin Fu, Wenjie Luo, George Tucker, Eli Bronstein, Yiren Lu, Jean Harb, Xinlei Pan, Yan Wang, Xiangyu Chen, John Co-Reyes, Rishabh Agarwal, Rebecca Roelofs, Yao Lu, Nico Montali, Paul Mougin, Zoey Yang, Brandyn White, Aleksandra Faust, Rowan McAllister, 11 Preprint Dragomir Anguelov, and Benjamin Sapp. Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research. In Advances in Neural Information Processing Systems, volume 36, p. 7730–7742, 2023. Georg Hess, Carl Lindstr ̈ om, Maryam Fatemi, Christoffer Petersson, and Lennart Svensson. Splatad: Real-time lidar and camera rendering with 3d gaussian splatting for autonomous driving. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 11982–11992. IEEE, 2025. Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. ST-P3: End- to-end vision-based autonomous driving via spatial-temporal feature learning. In European Con- ference on Computer Vision, p. 533–549. Springer, 2022. Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tian- wei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, p. 17853–17862, 2023. Bernhard Jaeger, Kashyap Chitta, and Andreas Geiger. Hidden biases of end-to-end driving models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 21983–21994, 2023. Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. Advances in Neural Information Processing Systems, 37:819–844, 2024. Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. DriveTransformer: Unified transformer for scalable end-to-end autonomous driving. In International Conference on Learning Represen- tations, 2025. Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. VAD: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 8340–8350, 2023. Napat Karnchanachari, Dimitris Geromichalos, Kok Seang Tan, Nanxiang Li, Christopher Eriksen, Shakiba Yaghoubi, Noushin Mehdipour, Gianmarco Bernasconi, Whye Kit Fong, Yiluan Guo, et al. Towards learning-based planning: The nuplan benchmark for real-world autonomous driv- ing. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 629–636. IEEE, 2024. Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ̈ uhler, and George Drettakis. 3D gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023. Jungho Kim, Jiyong Oh, Seunghoon Yu, Hongjae Shin, Donghyuk Kwak, and Jun Won Choi. Safedrive: Fine-grained safety reasoning for end-to-end driving in a sparse world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. Ellington Kirby, Alexandre Boulch, Yihong Xu, Yuan Yin, Gilles Puy, ́ Eloi Zablocki, Andrei Bursuc, Spyros Gidaris, Renaud Marlet, Florent Bartoccioni, et al. Driving on registers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 32058–32069, 2026. Jason Kong, Mark Pfeiffer, Georg Schildbach, and Francesco Borrelli. Kinematic and dynamic vehi- cle models for autonomous driving control design. In 2015 IEEE Intelligent Vehicles Symposium (IV), p. 1094–1099, 2015. doi: 10.1109/IVS.2015.7225830. Qifeng Li, Xiaosong Jia, Shaobo Wang, and Junchi Yan. Think2Drive: Efficient reinforcement learning by thinking with latent world model for autonomous driving (in CARLA-V2). In Euro- pean Conference on Computer Vision, p. 142–158. Springer, 2024a. 12 Preprint Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang, Zhenghai Xue, and Bolei Zhou. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 45(3):3461–3475, 2022. Tianyu Li, Yihang Qiu, Zhenhua Wu, Carl Lindstr ̈ om, Peng Su, Matthias Nießner, and Hongyang Li. Mtgs: Multi-traversal gaussian splatting. arXiv preprint arXiv:2503.12552, 2025a. Yingyan Li, Yuqi Wang, Yang Liu, Jiawei He, Lue Fan, and Zhaoxiang Zhang. End-to-end driv- ing with online trajectory evaluation via BEV world model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 27137–27146, 2025b. Yingyan Li, Shuyao Shang, Weisong Liu, Bing Zhan, Haochen Wang, Yuqi Wang, Yuntao Chen, Xiaoman Wang, Yasong An, Chufeng Tang, Lu Hou, Lue Fan, and Zhaoxiang Zhang. DriveVLA- W0: World models amplify data scaling law in autonomous driving. In International Conference on Learning Representations, 2026a. Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, Bing Wang, Kun Ma, Guang Chen, Hangjun Ye, Wenyu Liu, and Xinggang Wang. ReCogDrive: A reinforced cognitive framework for end-to-end autonomous driving. In International Conference on Learning Representations, 2026b. Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, Yu-Gang Jiang, and Jose M. Alvarez. Hydra-MDP: End-to-end multimodal planning with multi-target Hydra-Distillation. arXiv preprint arXiv:2406.06978, 2024b. Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end- to-end autonomous driving. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 12037–12047. IEEE, 2025. Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, and Wenming Yang. VastGaussian: Vast 3D gaussians for large scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. William Ljungbergh, Adam Tonderski, Joakim Johnander, Holger Caesar, Kalle ̊ Astr ̈ om, Michael Felsberg, and Christoffer Petersson. Neuroncap: Photorealistic closed-loop safety testing for autonomous driving. In European Conference on Computer Vision, p. 161–177. Springer, 2024. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics, 34(6):248:1– 248:16, 2015. Gerhard Neuhold, Tobias Ollmann, Samuel Rota Bul ` o, and Peter Kontschieder. The Mapillary Vistas dataset for semantic understanding of street scenes. In Proceedings of the IEEE International Conference on Computer Vision, p. 4990–4999, 2017. Paul Newson and John Krumm. Hidden markov map matching through noise and sparseness. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, p. 336–343, 2009. doi: 10.1145/1653771.1653818. NVIDIA.Cosmos-Predict2-0.6B-Text2Image. https://huggingface.co/nvidia/ Cosmos-Predict2-0.6B-Text2Image, 2025a. NVIDIA. Fixer: Improving 3D reconstructions with single-step diffusion models. https:// github.com/nv-tlabs/Fixer, 2025b. NVIDIA, Yulong Cao, Riccardo de Lutio, Sanja Fidler, Guillermo Garcia Cobo, Zan Gojcic, Max- imilian Igl, Boris Ivanovic, Peter Karkus, Janick Martinez Esturo, Marco Pavone, Aaron Smith, Ellie Tanimura, Michal Tyszkiewicz, Michael Watson, Qi Wu, and Le Zhang. AlpaSim: A modu- lar, lightweight, and data-driven research simulator for autonomous driving, October 2025. URL https://github.com/NVlabs/alpasim. 13 Preprint OpenScene Contributors. OpenScene: The largest up-to-date 3D occupancy prediction benchmark in autonomous driving. https://github.com/OpenDriveLab/OpenScene, 2023. Shiwei Ren, Tianci Wen, Yongchun Fang, and Biao Lu. Fastgs: Training 3d gaussian splatting in 100 seconds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 26094–26103, 2026. Katrin Renz, Long Chen, Elahe Arani, and Oleg Sinavski. SimLingo: Vision-only closed-loop au- tonomous driving with language-action alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. Johannes Lutz Sch ̈ onberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. SparseDrive: End-to-end autonomous driving via sparse scene representation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 8795–8801. IEEE, 2025. Yingqi Tang, Zhuoran Xu, Zhaotie Meng, and Erkang Cheng. HiP-AD: Hierarchical and multi- granularity planning with deformable attention for autonomous driving in a single decoder. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025. Adam Tonderski, Carl Lindstr ̈ om, Georg Hess, William Ljungbergh, Lennart Svensson, and Christoffer Petersson. Neurad: Neural rendering for autonomous driving. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14895–14904. IEEE, 2024. Martin Treiber, Ansgar Hennecke, and Dirk Helbing. Congested traffic states in empirical ob- servations and microscopic simulations.Physical Review E, 62(2):1805–1824, 2000.doi: 10.1103/PhysRevE.62.1805. Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. MoGe-2: Accurate monocular geometry with metric scale and sharp details. In Advances in Neural Information Processing Systems, volume 38, 2025. Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. PARA-Drive: Paral- lelized architecture for real-time autonomous driving. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, p. 15449–15458, 2024. Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki, Xuanchi Ren, Jun Gao, Mike Zheng Shou, Sanja Fidler, Zan Gojcic, and Huan Ling. DIFIX3D+: Improving 3d reconstructions with single-step diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 26024–26035, 2025. Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. In Advances in Neural Information Processing Systems, volume 35, p. 6119–6132, 2022. Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo. Seg- Former: Simple and efficient design for semantic segmentation with transformers. In Advances in Neural Information Processing Systems, volume 34, 2021. Runsheng Xu, Hubert Lin, Wonseok Jeon, Hao Feng, Yuliang Zou, Liting Sun, John Gorman, Eka- terina Tolstaya, Sarah Tang, Brandyn White, Ben Sapp, Mingxing Tan, Jyh-Jing Hwang, and Dragomir Anguelov. WOD-E2E: Waymo open dataset for end-to-end driving in challenging long-tail scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Dan- fei Xu, Sanja Fidler, Marco Pavone, et al. Emernerf: Emergent spatial-temporal scene decom- position via self-supervision. In International Conference on Learning Representations, volume 2024, p. 16739–16766, 2024. 14 Preprint Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Manivasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Urtasun. Unisim: A neural closed-loop sensor simulator. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 1389–1399. IEEE, 2023. Wenhao Yao, Zhenxin Li, Shiyi Lan, Zi Wang, Xinglong Sun, Jose M. Alvarez, and Zuxuan Wu. DriveSuprim: Towards precise trajectory selection for end-to-end planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, p. 11910–11918, 2026. doi: 10.1609/ aaai.v40i14.38178. Zhenlong Yuan, Chengxuan Qian, Jing Tang, Rui Chen, Zijian Song, Lei Sun, Xiangxiang Chu, Yu- jun Cai, Dapeng Zhang, and Shuo Li. AutoDrive-R 2 : Incentivizing reasoning and self-reflection capacity for VLA model in autonomous driving. In International Conference on Learning Repre- sentations, 2026. Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 586–595, 2018. Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21336– 21345. IEEE, 2024a. Hongyu Zhou, Longzhong Lin, Jiabao Wang, Yichong Lu, Dongfeng Bai, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugsim: A real-time, photo-realistic and closed-loop simulator for autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Driv- inggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21634– 21643. IEEE, 2024b. 15 Preprint APPENDIX FOR ODYSSEY Contents A Related Work16 A.1 End-to-End Driving Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 A.2 Evaluation of E2E Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 A.3 Driving Intention for End-to-End Models . . . . . . . . . . . . . . . . . . . . . .17 B Implementation Details of Closed-loop Simulator17 B.1 3D Gaussian Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 B.2 Fixer fine-tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 B.3 Section-based Traffic Triggering . . . . . . . . . . . . . . . . . . . . . . . . . . .20 C Evaluation Metric Details21 C.1 Rollout Termination Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 C.2 Pre-Lane Change Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 C.3 RouteDS Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 C.4 Additional Driving Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 D SD-Route Conditioning of Baseline Planners23 E Additional Analysis23 E.1 Comparison with SOTA 3DGS methods . . . . . . . . . . . . . . . . . . . . . . .23 E.2 Detailed RouteDS Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 E.3 Additional Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 ARELATED WORK A.1END-TO-END DRIVING MODELS Planning-oriented E2E driving integrates perception, prediction, and planning (Hu et al., 2022; 2023), with subsequent work improving scene representation and task interaction (Jiang et al., 2023; Sun et al., 2025; Weng et al., 2024; Jia et al., 2025). Building on these representations, planners jointly predict trajectories and controls (Wu et al., 2022; Jia et al., 2023) or separate path geometry from speed prediction (Jaeger et al., 2023; Tang et al., 2025). To handle multiple feasible actions, dif- fusion models generate diverse trajectories (Liao et al., 2025), and candidate-based planners learn to score alternatives (Li et al., 2024b; Yao et al., 2026). World models further inform planning through predicted future states and action risks (Li et al., 2025b; Kim et al., 2026), and provide imagined interactions for reinforcement learning (Li et al., 2024a). Vision-language-action (VLA) models extend sensor-based planning with language understanding and reasoning. SimLingo aligns language instructions with driving actions (Renz et al., 2025), and DriveVLA-W0 incorporates future-image prediction to enrich visual learning beyond action super- vision (Li et al., 2026a). ReCogDrive and AutoDrive-R 2 further combine reasoning with reinforce- ment learning to improve driving decisions (Li et al., 2026b; Yuan et al., 2026). Odyssey adapts vision-based and VLA planners to SD-route guidance for evaluating sustained navigation. 16 Preprint A.2EVALUATION OF E2E PLANNING Evaluating E2E planners requires assessing the consequences of their actions, which open-loop displacement-based metrics overlook (Caesar et al., 2020; Hu et al., 2022; 2023). NAVSIM (Dauner et al., 2024) moves beyond imitation accuracy through simulation-based driving metrics. NAVSIM v2 (Cao et al., 2025) further incorporates pre-rendered observations to approximate subsequent feed- back. Extending this feedback to continuous replanning requires observations that reflect the ego vehicle’s evolving state. Graphics-based simulators provide such observations but introduce a visual domain gap (Dosovitskiy et al., 2017; Li et al., 2022; Jia et al., 2024). Planning simulators using real logs evaluate driving behavior without sensor rendering (Caesar et al., 2021; Karnchanachari et al., 2024; Gulino et al., 2023). Neural-rendering-based simulators bring sensor feedback to recorded driving environments by re- constructing scenes and rendering observations in response to ego actions (Yang et al., 2023; Ton- derski et al., 2024; NVIDIA et al., 2025). Within this paradigm, NeuroNCAP (Ljungbergh et al., 2024) examines collision avoidance in NeRF-reconstructed scenarios, and HUGSIM (Zhou et al., 2025) provides closed-loop evaluation through 3DGS rendering and controllable traffic. However, deviations from the logged trajectory can expose poorly reconstructed regions (Zhou et al., 2024b; 2025; Hess et al., 2025; Chen et al., 2025), producing artifacts that compromise the reliability of sen- sor feedback. Odyssey addresses this issue by refining 3DGS-rendered observations with diffusion to reduce artifacts at novel ego poses. A.3DRIVING INTENTION FOR END-TO-END MODELS Navigation guidance specifies the intended path when multiple driving maneuvers are feasi- ble (Codevilla et al., 2018). Benchmarks based on real driving logs (Hu et al., 2023; Jiang et al., 2023; Dauner et al., 2024; Cao et al., 2025; Ljungbergh et al., 2024; Zhou et al., 2025; Xu et al., 2026) commonly express this intent through driving commands, but this compact representation cannot uniquely distinguish consecutive branches sharing the same direction. CARLA-based bench- marks (Dosovitskiy et al., 2017), including Bench2Drive (Jia et al., 2024), provide explicit guidance through target points or route waypoints in synthetic environments. However, guidance limited to nearby targets provides insufficient route context for planning across multiple maneuvers over longer horizons. Odyssey uses OSM-derived SD-map routes to provide this guidance in reconstructed real- world environments. These routes specify the intended road sequence throughout the rollout and reduce reliance on detailed HD maps as planner inputs. Local driving decisions remain based on sensor observations. BIMPLEMENTATION DETAILS OF CLOSED-LOOP SIMULATOR B.13D GAUSSIAN RECONSTRUCTION RECONSTRUCTION FRAMEWORK.We build on OmniRe (Chen et al., 2025) to jointly recon- struct the static background and dynamic actors, and model traffic lights separately to reproduce signal changes. Pose noise in nuPlan logs degrades 3D Gaussian reconstruction quality, motivating pose refinement with COLMAP (Sch ̈ onberger & Frahm, 2016). We keep the logged x and y coordi- nates fixed to preserve alignment with the HD map used for drivable-area evaluation, and optimize only z, roll, pitch, and yaw. To separate scene components, we generate sky and dynamic-object masks using SegFormer (Xie et al., 2021). We initialize background Gaussians by back-projecting MoGe-2 pseudo-depth (Wang et al., 2025) after excluding dynamic pixels. Rigid and deformable actors are initialized from accumulated LiDAR points, and pedestrians from SMPL templates (Loper et al., 2015). For traffic-light modeling, we obtain head masks using Mask2Former (Cheng et al., 2022) pretrained on Mapillary Vistas v2 (Neuhold et al., 2017). GROUND MODELING.Inspired by HUGSIM (Zhou et al., 2025), we represent the road surface with a separate set of thin Gaussian disks. We first accumulate LiDAR points using the refined ego poses and build a road height field on a 0.125 m grid. Ground Gaussians are placed on this height field within the HD-map road regions; heights of unobserved road cells are filled by harmonic inter- polation from the observed cells. Each disk has a fixed thickness of 0.01 m and stays attached to the 17 Preprint Table 6: Gaussian densification, pruning, and optimization settings. ParameterBackgroundRigid actors Densification Start / end iteration2,500 / 93,5002,500 / 93,500 Interval (iterations)500500 Views per scoring step120120 Normalized error threshold0.10.1 Required score> 5> 5 Clone gradient threshold2× 10 −4 2× 10 −4 Split absolute-gradient threshold1× 10 −4 3× 10 −4 Clone / split scale boundary0.0562 m0.09 m Children per split22 Pruning and opacity control Minimum views for pruning52 Early opacity threshold0.0050.005 Early scale threshold5.617 m5.617 m Opacity cap0.80.8 Opacity reset interval7,00014,500 Opacity reset value0.010.01 Post-pruning start / end70,000 / 140,000 93,500 / 140,000 Post-pruning interval4,5004,500 Post-pruning opacity threshold0.050.05 Post-pruning score threshold0.90.9 Background update intervals (iterations) Training iterationsGeometry / opacitySH Up to 70,000116 70,001–93,3343232 After 93,3346464 surface: its height and normal are periodically re-sampled from the height field at its current hori- zontal position. The remaining attributes are optimized, namely the horizontal position, the in-plane scale, the rotation about the surface normal, the opacity, and the spherical-harmonic coefficients. To keep the background from also representing the road, we remove background points within 0.3 m of the surface at initialization. During training, we also penalize the mean opacity of background Gaussians that lie between 0.3 m below and 1.0 m above the surface. COVERAGE-AWARE SAMPLING.Sampling images uniformly over a long driving sequence con- centrates training updates on places where the recording vehicle stops or moves slowly. To balance spatial coverage, we back-project the LiDAR depth of each training image, excluding dynamic ob- jects, into 0.5 m voxels and count how many training views observe each voxel. The weight of an image is the mean inverse observation count of its visible voxels, normalized by the 90th percentile over all images and clipped to [0.1, 1]. Images that mainly see less-observed regions therefore re- ceive higher weights, and frequently observed regions are revisited less often. Rather than drawing images at random, we build a sampling cycle in which each image appears a number of times pro- portional to its weight, so that every training view is used at least once per cycle. Images containing dynamic objects are given a higher minimum weight of 0.25 to preserve the temporal coverage of moving actors. Each reconstruction update uses a single image. TRAFFIC-LIGHT MODELING.To capture signal changes, we represent each traffic-light head as a separate deformable Gaussian node. Using head masks predicted by Mask2Former (Cheng et al., 2022) and depth estimated by MoGe-2 (Wang et al., 2025), we back-project masked pixels from calibrated views to initialize Gaussian positions and colors. Each node has a fixed installation position and orientation, and its geometry is kept fixed up to a small tolerance of 5 cm that absorbs residual calibration error. Signal changes are therefore modeled through appearance: an MLP takes Gaussian positions, time, and a learned embedding for each traffic light, and predicts temporal residuals to the spherical-harmonic coefficients and opacity. After reconstruction, we manually verify the signal schedules against the rendered traffic-light states and revise them where necessary. 18 Preprint Table 7: Reconstruction losses and weights. LossObjectiveWeight (λ) Appearance supervision L rgb RGB L1 after color correction0.8 L ssim SSIM before color correction0.2 L color Out-of-range color penalty0.8 L res Frame-dependent residual magnitude0.01 L temp Temporal variation of color residuals0.1 Geometry and opacity supervision L sky Sky-opacity BCE0.05 L depth LiDAR inverse-depth supervision0.1 L ent Opacity entropy0.05 L smooth Edge-aware inverse-depth smoothness0.001 L los Line-of-sight opacity0.1 L box Actor bounding-box regularization1.0 L TL Traffic-light Gaussian displacement1.0 OPTIMIZATION.We jointly optimize each complete 100-second sequence as a single scene for 140k iterations, sampling one training image per iteration. We use eight cameras at 10 Hz with 1920×1080 images, starting at half resolution and switching to full resolution at iteration 1,166. To balance training across the sequence, we assign higher sampling weights to views covering less frequently observed 0.5 m spatial voxels, while every training view is still served at least once per sampling cycle. FastGS, GPU-resident caching of images and supervision signals, and a reduced update cadence for the background parameters of Gaussians reduce training overhead. We apply FastGS density control (Ren et al., 2026) to background and rigid-actor Gaussians, com- puting scores from 120 views sampled across cameras and the full sequence. To account for uneven visibility over long sequences, we normalize densification scores by each Gaussian’s visible-view count. Table 6 lists the settings. TRAINING OBJECTIVES.We jointly optimize appearance and geometry using a weighted sum of the losses. Our appearance objective combines RGB L1 and SSIM losses with color-correction regularization. RGB L1 compares color-corrected renderings with recorded images, and SSIM su- pervises the uncorrected renderings to preserve image structure (Lin et al., 2024). The regularization penalizes colors outside the valid range, limits frame-dependent corrections, and discourages abrupt changes between adjacent frames. To account for exposure and white-balance differences, we ad- just rendered RGB values using a learnable 3× 4 affine transformation following MTGS (Li et al., 2025a). The first three columns transform the RGB channels, and the last column adds a per-channel offset. A shared matrix captures persistent differences between cameras, and a residual matrix for each camera and frame captures temporal variations. Our geometry losses regularize scene structure and reduce reconstruction artifacts. LiDAR depth provides additional geometric supervision, and edge-aware smoothness reduces depth noise without smoothing across image boundaries (Chen et al., 2026). Sky-opacity BCE encourages transparent sky regions, and a line-of-sight loss suppresses opacity at least 0.8 m in front of LiDAR returns (Hess et al., 2025). Opacity entropy further encourages values near zero or one. For actors, we apply bounding-box regularization to both rigid and deformable Gaussians, extending OmniRe’s regular- ization to rigid actors. We separately penalize local displacement of traffic-light Gaussians to limit deformation around their fixed installations. Table 7 summarizes the loss terms and their weights. B.2FIXER FINE-TUNING We refine rendered images using Fixer (NVIDIA, 2025b), a single-step diffusion model built on the 0.6B Cosmos-Predict2 backbone (NVIDIA, 2025a). The autoencoder encodes the degraded ren- dering directly into latent features without adding random noise. The degraded rendering therefore serves as the input to be restored, with reconstruction artifacts providing the corruption that the model learns to remove. A diffusion transformer refines the latent features in one step, and the 19 Preprint Table 8: Fixer inference acceleration. MethodTime (ms/image)↓Speedup↑ Baseline25.01.0× Optimized6.73.7× decoder reconstructs the output image. To adapt the pretrained model to Odyssey, we construct rendering–real image pairs from all 100 benchmark scenes using three degradation strategies fol- lowing Difix3D+ (Wu et al., 2025). • UNDER-TRAINED RECONSTRUCTION (30%).We use renderings from the 80 k, 120 k, and final 140 k checkpoints to cover different reconstruction quality levels. Final- checkpoint renderings discourage unnecessary corrections to high-quality inputs. • CYCLE RECONSTRUCTION (40%). We render a trained Gaussian scene from a trajectory shifted laterally by 2 m, train a second Gaussian model on these renderings, and render the original trajectory from the second model. These images are paired with the original real images to train refinement of novel-view artifacts. • CROSS-REFERENCE RECONSTRUCTION (30%). We train a Gaussian model using six of the eight cameras, holding out CAM L2 and CAMR2. Renderings from the two held-out cameras are paired with their real images. These viewpoints receive only indirect supervi- sion from neighboring cameras, producing novel-view artifacts. We randomly sample 20% of the frames from the 10 Hz sequences for training-pair generation. Under-trained, cycle, and cross-reference reconstruction account for 30%, 40%, and 30% of the training pairs, respectively. Pairs are generated at 576× 1024 resolution without image-space aug- mentation. TRAINING OBJECTIVES.Using these pairs, we train Fixer with pixel, perceptual, and texture supervision: L Fixer = λ L2 L L2 + λ LPIPS L LPIPS + λ Gram L Gram .(1) Full-image L2 loss measures pixel error, LPIPS (Zhang et al., 2018) preserves perceptual content, and Gram loss (Gatys et al., 2015) matches texture statistics. LPIPS and Gram losses use VGG-16 features from spatially aligned crops of the refined and real images. LPIPS uses square crops with side lengths of 128–512 pixels, and Gram loss uses 512× 512 crops. We set λ L2 = λ LPIPS = 1 throughout training. Fixer’s public training recipe disables Gram loss (NVIDIA, 2025b), but we found that activating it during fine-tuning substantially reduced blur and improved texture detail. To avoid excessive sharpening, we keep λ Gram = 0 until the final 1,000 steps, then set it to 0.02. INFERENCE OPTIMIZATION.To improve inference speed, we remove cross-attention layers in- active under zero text conditioning and employ TensorRT to accelerate Fixer’s encoder, diffusion transformer, and decoder. These optimizations achieve a 3.7× speedup with negligible changes in restoration quality (Table 8). B.3SECTION-BASED TRAFFIC TRIGGERING When the simulated ego travels faster or slower than the logged vehicle, section-based traffic trigger- ing aligns traffic playback with ego progress to preserve local traffic encounters. We place section boundaries between intersections and combine short sections to maintain a minimum length of 50 m. At each simulation step, we project the ego position onto the logged trajectory to determine its progress. The first section is active at initialization, and subsequent sections activate when the ego reaches 15 m before their start. This 15 m margin reduces sudden vehicle appearances at section boundaries and gives the planner time to observe the surrounding traffic before entering. To align each section’s traffic with ego progress, we compute a fixed offset between the logged and simu- lated ego arrival times at the activation point. This offset provides a common timing reference for 20 Preprint (a) GT Log 0~25s 25~50s 50~75s 75~100s 40m 40m x0.6 Speedx0.6 Speedx0.8 Speedx0.8 Speed (c) Section-based Trigger (Ours)(b) Playback x0.4 Speedx0.4 Speedx1.0 Speed SlowerSlower Figure 9: Effect of section-based triggering on nearby agent density. (a) GT Log shows nearby agent density during recorded driving. (b) Playback follows the original traffic timeline, with density decreasing as the ego slows and simulation continues. (c) Section-based triggering aligns traffic activation with ego progress, maintaining density closer to GT Log across the evaluated speeds and time intervals. initializing surrounding vehicles and pedestrians and assigning traffic-light states. Parked vehicles and static objects remain active throughout the simulation, regardless of section activation. The two traffic settings differ in how surrounding vehicles move after initialization. In the non- reactive setting, vehicles replay their recorded motion using the section’s fixed offset. In the reactive setting, vehicles are initialized with their logged positions and states, then follow IDM-based control. Pedestrian motion and traffic-light changes follow their recorded timing with the same section offset in both settings. Figure 9 compares nearby agent density under GT Log, Playback, and section-based triggering. With Playback, nearby agent density progressively decreases when the ego travels more slowly than the logged vehicle. Section-based triggering maintains density closer to GT Log across the evaluated ego speeds and time intervals. In addition to maintaining traffic density, reliable evaluation requires adequate visual quality for the surrounding actors. Distant or occluded actors may have too few observations for Gaussian reconstruction, resulting in poor rendering quality and insufficient visual cues for the planner. Such actors can still incur evaluation penalties. We therefore manually inspected individual actors and removed those with inadequate rendering quality. CEVALUATION METRIC DETAILS This section defines the rollout termination criteria, pre-lane change metrics, RouteDS components, and additional driving metrics. 21 Preprint C.1ROLLOUT TERMINATION CRITERIA Each rollout evaluates progress toward a designated destination. Termination criteria bound its du- ration and prevent continued simulation far from the intended route. A rollout ends under one of three conditions: • DESTINATION ARRIVAL. The ego reaches the end of the designated SD route. • TIME LIMIT. The rollout is limited to 200 s, twice the duration of the recorded ground- truth driving sequence. • SD-ROUTE DEVIATION. The rollout terminates as soon as the ego vehicle is more than 30 m from the designated SD route. C.2PRE-LANE CHANGE METRICS We select evaluation intersections where the logged vehicle travels in route-compatible lanes throughout the 10 m before the stop line. To detect when vehicles approaching through oncom- ing lanes or off-road areas pass an evaluation intersection, we extend each stop line by 25 m at both ends and check for crossings. Crossings along these extensions, outside the original stop-line segment, are treated as failures to cross the stop line correctly. C.3ROUTEDS COMPONENTS For each scenario, we compute RouteDS = 100· RC SD · P PLC · P SD · P col · P off · P TL ,(2) where RC SD ∈ [0, 1] denotes route completion and the remaining factors are penalties in [0, 1], defined below. We report the mean RouteDS across scenarios. • ROUTE COMPLETION. RC SD is the fraction of the designated SD route completed by the ego vehicle. After each rollout, we apply HMM map matching (Newson & Krumm, 2009) to the executed trajectory to identify the SD-route segments traversed by the ego. If the ego deviates from the route, we determine where the ego left the route by repeating HMM matching from the start, extending the evaluated trajectory by 1 m each time. We count progress only up to the last point before the matching indicates that the ego is off route. • PRE-LANE CHANGE. For each pre-lane change evaluation, we multiply P PLC by 0.7 for a score of 0 and by 0.9 for a score of 0.5. Unencountered evaluation stop lines incur no pre-lane change penalty. • SD-ROUTE COMPLIANCE. We set P SD = 0 if the rollout terminates because the ego exceeds the route-distance threshold defined above, or if post-rollout HMM matching iden- tifies travel outside the designated route. If neither condition holds and matching succeeds, P SD = 1. Near the start and end of a rollout, limited trajectory coverage around a junction can make connected roads difficult to distinguish. We allow an 8.5 m tolerance for such boundary matches, excluding travel opposite to the route direction. • COLLISION PENALTY. We penalize at-fault collisions with pedestrians, vehicles, and bi- cycles using nuPlan-style criteria (Caesar et al., 2021), excluding stationary-ego and active rear collisions. Front collisions and collisions with stationary agents count, with lateral col- lisions counted only when the ego spans multiple lanes or extends outside drivable areas. For each counted collision, we multiply P col by 0.5 for pedestrians and 0.6 for vehicles or bicycles (Jia et al., 2024). If no collision is counted, P col = 1. The same object is counted again only after contact breaks and either displacement exceeds 5 m or elapsed time exceeds 5 s. • OFF-ROAD PENALTY. We compute P off = 1−D off /D total , where D total is the total trav- eled distance and D off is the distance driven outside drivable areas or against the direction of traffic. Driving-direction violations are not counted within intersections. 22 Preprint • TRAFFIC-LIGHT PENALTY. Each red-light violation multiplies P TL by 0.7. A violation occurs when the ego occupies an intersection roadblock whose traffic signal is red. Each violation is counted once, even if it persists across multiple evaluation steps. C.4ADDITIONAL DRIVING METRICS We adapt Efficiency and Comfortness metrics from Bench2Drive (Jia et al., 2024) to our simulation setting, as detailed below. EFFICIENCY.Efficiency measures whether the ego maintains driving speed relative to other traf- fic. We compare its speed with vehicles that exceed 1 m/s at least once during the rollout. At each evaluation step containing these vehicles, we compute their average speed. Efficiency is 100 times the sum of ego speeds divided by the sum of these average vehicle speeds over the same steps. A score of 100 indicates equal average speeds. COMFORTNESS.Comfortnessevaluates whether the ego drives smoothly without abrupt acceleration, braking, or steering. It is one if all limits in the table are satisfied throughout the trajectory, and zero otherwise. Except for longitudinal acceleration, the limits apply to absolute values. Comfort CriterionLimit Longitudinal acceleration[−4.05, 2.40] m/s 2 Lateral acceleration4.89 m/s 2 Yaw rate0.95 rad/s Yaw acceleration1.93 rad/s 2 Longitudinal jerk4.13 m/s 3 Derivative of acceleration magnitude8.37 m/s 3 DSD-ROUTE CONDITIONING OF BASELINE PLANNERS This section details the model-specific adaptations of the baselines in Section 4.2. We keep each original architecture and training recipe, and add only the route encoder and the modules that read it. We first transform the route points into the ego frame, then encode their positions relative to the first route point. Headings in the ego frame are encoded by their cosine and sine. The added cross- attention has its own normalization and residual connection. Unavailable route points are masked, and a learned token represents the absence of route guidance when all route points are masked. • LTF. We use the camera-only latent variant. The trajectory query attends to the route in all three decoder layers, while the 30 agent queries do not. The ego status keeps only velocity and acceleration. Trained for 100 epochs. • DiffusionDrive. The 20 anchor queries attend to the route in both decoder layers after the ego-query cross-attention. The route therefore conditions each of the two denoising steps. We use the camera-only latent variant. Trained for 100 epochs. • DrivoR. The 64 proposal tokens attend to the route in all four refinement blocks. The ego history encoding is reduced from 11 to 7 channels (pose, velocity, and acceleration). Trained for 25 epochs. • SafeDrive. Every layer of both planning decoders attends to the same route tokens from a shared encoder. The ego status is reduced from 8 to 4 channels. We use the camera-only variant with pedestrian detection. • ReCogDrive. A two-stage point encoder with max pooling compresses the 120 route points into one token. A Fourier embedding of the route start is added to this token, and the result is added to the ego-status embedding of the diffusion planner. The navigation line of the VLM prompt is generated from the same route, naming the next maneuver with its distance and the one after it (e.g., “turn left in 30 m, then follow the road”). When the remaining route is too short to describe, the driving command is used instead. EADDITIONAL ANALYSIS E.1COMPARISON WITH SOTA 3DGS METHODS We compare reconstruction quality with OmniRe (Chen et al., 2025), MTGS (Li et al., 2025a), and HUGSIM (Zhou et al., 2025) using PSNR, SSIM, and LPIPS. We evaluate on five nuPlan (Caesar 23 Preprint Table 9: The five nuPlan logs used for the comparison. Each scene is a 20 s window sampled at 2 Hz; frame in- dices are 2 Hz keyframe indices within each log. nuPlan logCityFrames S1 2021.09.16.19.27.01veh-450174903230 Pittsburgh 1539–1578 S2 2021.09.29.19.02.14veh-280027300514Boston119–158 S3 2021.09.09.19.10.24 veh-390112501324Boston120–159 S4 2021.06.28.21.16.05veh-140095701198Las Vegas270–309 S5 2021.06.28.20.24.43veh-380338504952Las Vegas1561–1600 Table 10: Quantitative comparison with reconstruction baselines. All baselines are retrained under their official recipes on identical data, and all metrics are computed with a single implementation. MethodPSNR↑ SSIM↑ LPIPS↓ OmniRe (Chen et al., 2025)19.40.6540.456 MTGS (Li et al., 2025a)19.60.6920.419 HUGSIM (Zhou et al., 2025)20.10.7010.436 Ours21.70.7070.388 et al., 2021) scenes from different logs across three cities, listed in Table 9. Each scene comprises a 20 s window sampled at 2 Hz, using all eight surround-view cameras at 1920× 1080 resolution. Images are undistorted using the original intrinsic matrices as the output camera intrinsics. We hold out every fourth keyframe across all eight cameras to evaluate novel-view synthesis at held-out camera poses. We train each baseline from scratch using its official training recipe and iteration count. All methods use the same camera poses, calibration, undistorted images, train/test split, and image resolution. As shown in Table 10, our method achieves the highest PSNR and SSIM and the lowest LPIPS among the compared methods, indicating improvements in both image fidelity and perceptual similarity. E.2DETAILED ROUTEDS ANALYSIS Table 11 reports route completion and individual penalty factors alongside RouteDS. SD-route guid- ance improves route compliance and RouteDS across planners, but improvements in individual penalty factors are not consistent. These results distinguish gains in route following from changes in other driving criteria. Table 11: Detailed RouteDS results under non-reactive and reactive traffic. Method SD Route guidance Non-reactiveReactive RouteDS RC SD P SD P col P off P TL P PLC RouteDS RC SD P SD P col P off P TL P PLC Vision-based planners LTF ×16.240.72 0.51 0.54 0.67 0.98 0.8816.270.75 0.54 0.59 0.67 0.98 0.86 ✓20.130.99 0.950.43 0.63 0.99 0.8123.010.99 0.930.48 0.63 0.98 0.80 DiffusionDrive ×25.640.75 0.59 0.61 0.71 0.99 0.9224.170.73 0.59 0.61 0.71 0.98 0.91 ✓35.930.961.00 0.51 0.76 0.99 0.9140.970.981.00 0.57 0.76 0.990.91 DrivoR ×20.610.68 0.50 0.69 0.86 0.99 0.9025.460.71 0.53 0.73 0.86 0.98 0.90 ✓36.450.84 0.90 0.65 0.86 0.990.8841.350.83 0.89 0.74 0.88 0.98 0.88 SafeDrive ×27.520.61 0.41 0.86 0.89 0.99 0.9230.420.64 0.45 0.88 0.890.98 0.92 ✓44.350.85 0.82 0.750.890.990.8948.940.84 0.79 0.850.89 0.98 0.88 VLA-based planners ReCogDrive ×15.420.57 0.71 0.59 0.79 0.99 0.9120.330.53 0.62 0.70 0.80 0.990.91 ✓21.450.54 0.83 0.69 0.83 0.99 0.9026.250.56 0.84 0.77 0.83 0.99 0.91 E.3ADDITIONAL QUALITATIVE ANALYSIS NOVEL-VIEW REFINEMENT.Figure 10 shows novel views shifted by 3–5 m from the recorded camera positions, where limited observational coverage leads to pronounced reconstruction artifacts. Fine-tuned Fixer reduces blur, distortions, ghosting, and color and brightness inconsistencies across diverse scene elements, including lane markings, vehicles, pedestrians, and background structures. SCENARIO DIVERSITY.Figure 11 overlays recorded ego trajectories and SD routes for scenarios involving lane changes before turns, multiple turns, and long straight routes. These examples span different levels of route complexity. In particular, lane changes before turns illustrate how naviga- tion intent shapes driving decisions before the maneuver itself, motivating evaluation over extended sequences. 24 Preprint TRAFFIC-LIGHT MODELING.Figure 12 shows additional examples of reconstructed traffic-light states alongside their locations and signal schedules. The enlarged views highlight red and green states reproduced by the model, illustrating its ability to represent signal changes within the same reconstructed scene. OursGS Only Figure 10: Qualitative Comparison of Fixer Refinement. We show challenging novel views with limited coverage from training observations, where 3DGS-only renderings exhibit pronounced arti- facts. Fine-tuned Fixer reduces these artifacts and improves visual detail. 25 Preprint (b) Route with Multiple Turns (a) Lane Change Followed by a Turn (c) Long Straight Route 250 m GT Trajectory SD Map SD Route Lane change Lane change Figure 11: Example trajectories and SD-Routes from Odyssey. The benchmark includes lane changes in preparation for upcoming turns, routes with multiple turns, and long straight routes. These scenarios span varying route complexity and include anticipatory maneuvers that reflect the driver’s navigation intent. RedGreen RedGreen Red light Green lightRed light Green light Figure 12: Additional Qualitative Results of Traffic Light Modeling. Our model reproduces red- to-green transitions in sync with the traffic-light schedule in each scene. 26