Paper deep dive

SCAPE: Scenario-Conditioned Simulation-Augmented Policy Evaluation

Dijie Zhu, Seunghun Oh, Ruopeng Huang, Zhiyu Huang, Jiaqi Ma, Chen Tang

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 100

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 8/21/2026, 3:10:28 AM

Summary

The paper introduces SCAPE, a framework for scenario-conditioned simulation-augmented policy evaluation. It addresses the limitations of existing methods that only provide aggregate performance metrics by predicting scenario-specific real-world policy performance using limited paired sim-and-real data and large-scale simulation rollouts. SCAPE corrects sim-to-real bias via a correction model trained on paired data and calibrates uncertainty using conformal prediction. It is validated on autonomous driving (nuPlan) and quadruped velocity tracking (Unitree Go2), showing improved prediction accuracy, sample efficiency, and generalization compared to baselines.

Entities (8)

SCAPE · framework · 100%Conformal Prediction · method · 95%nuPlan · datasetbenchmark · 95%Unitree Go2 · robothardware · 95%Prediction-Powered Inference (PPI) · method · 90%Sim-to-Real Gap · problem · 90%Sim2Val · baseline-method · 85%SureSim · baseline-method · 85%

Relation Signals (7)

SCAPE → uses → Conformal Prediction

confidence 95% · calibrates prediction uncertainty through conformal prediction.

SCAPE → validatedon → nuPlan

confidence 95% · We validate SCAPE on autonomous driving... experiments on nuPlan

SCAPE → validatedon → Unitree Go2

confidence 95% · evaluate a velocity-tracking policy deployed on a physical Unitree Go2.

SCAPE → corrects → Sim-to-Real Gap

confidence 90% · SCAPE corrects sim-to-real bias in simulation labels before training the prediction model

SCAPE → outperforms → Sim2Val

confidence 85% · reduces scenario-level prediction error... relative to scene-conditioned neural and aggregate statistical baselines

SCAPE → outperforms → SureSim

confidence 85% · reduces scenario-level prediction error... relative to scene-conditioned neural and aggregate statistical baselines

SCAPE → uses → Prediction-Powered Inference (PPI)

confidence 70% · In principle, PPI could be adapted to train our scenario-conditioned predictor... We therefore propose SCAPE, which adopts an alternative bias-correction strategy

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Reliable performance evaluation is a central bottleneck for deploying robot-learning policies in real-world conditions. Real-world testing is faithful but costly and difficult to scale, whereas simulation-based testing scales easily but is inevitably biased by the sim-to-real gap. Existing simulation-augmented methods combine limited real-world rollouts with abundant simulation proxies, but focus on performance averaged over initial conditions and deployment settings. Such population-level averages obscure scenario-specific variation and provide limited guidance about when and where a policy can be safely deployed. We propose SCAPE, a scenario-conditioned simulation-augmented policy evaluation framework that predicts scenario-conditioned real-world policy performance using limited paired sim-and-real samples and large-scale simulation rollouts. SCAPE corrects sim-to-real bias in simulation labels before training the prediction model and calibrates prediction uncertainty through conformal prediction. We validate SCAPE on autonomous driving and quadruped velocity tracking. In sim-to-sim studies, SCAPE reduces scenario-level prediction error by 4.9%/34.7% (driving) and 14.5%/27.7% (quadruped) relative to scene-conditioned neural and aggregate statistical baselines on average. We further evaluate a velocity-tracking policy deployed on a physical Unitree Go2. SCAPE also improves testing sample efficiency, produces narrower calibrated prediction intervals, generalizes better to out-of-distribution scenarios, and enables fine-grained deployment strategies.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

96,584 characters extracted from source content.

Expand or collapse full text

Scape: Scenario-Conditioned Simulation-Augmented Policy Evaluation Dijie Zhu Seunghun Oh Affiliation: Seoul National University Ruopeng Huang Affiliation: University of Southern California Zhiyu Huang Affiliation: North Carolina State University Jiaqi Ma Chen Tang [0.5em] University of California Los Angeles Abstract Reliable performance evaluation is a central bottleneck for deploying robot-learning policies in real-world conditions. Real-world testing produces faithful evaluation but is costly and difficult to scale, whereas simulation-based testing is easy to scale but inevitably biased due to the sim-to-real gap. Neither source alone is sufficient for scalable and reliable policy evaluation. This motivates simulation-augmented policy evaluation, which combines limited real-world rollouts with abundant simulation proxies for sample-efficient performance estimation. Yet, existing methods focus on estimating performance averaged over initial conditions and deployment settings. By construction, such population-level averages collapse scenario-specific variation, providing limited information about when and where a policy can be safely deployed. To address this limitation, we propose Scape, a scenario-conditioned simulation-augmented policy evaluation framework that instead predicts scenario-conditioned real-world policy performance from limited paired sim-and-real evaluation samples and large-scale simulation-based testing rollouts. Scape corrects the sim-to-real bias in the simulation labels before using them to train the prediction model, and calibrates the prediction uncertainty through conformal prediction. We validate Scape on two embodied task domains, autonomous driving and quadruped velocity tracking. In sim-to-sim settings, it reduces the scenario-level performance prediction error by 4.9%/34.7% (driving) and 14.5%/27.7% (quadruped) relative to scene-conditioned neural baselines and aggregate statistical baselines on average. We also validate Scape for evaluating a velocity-tracking policy deployed on a physical Unitree Go2. In addition, Scape improves testing sample-efficiency, produces narrower calibrated prediction intervals, enhances generalization to out-of-distribution scenarios, and unlocks fine-grained deployment strategies. 1 Introduction Recent advances in robot learning, particularly robotic foundation models, have enabled increasingly capable autonomous robots that can operate across a growing range of real-world conditions [1, 2, 3]. A central bottleneck for trustworthy deployment is the rigorous evaluation of learned control policies under diverse operating conditions. Such evaluation typically requires extensive real-world testing on physical hardware, which can be prohibitively expensive, resource-intensive, and safety-critical, especially in open-world environments where robots operate near humans and face rare, and long-tailed conditions. For example, certifying an autonomous vehicle may require billions of miles of driving [4] under dedicated human oversight. Moreover, relying solely on real-world testing is practically infeasible for covering all rare, long-tailed cases, and the required testing effort keeps growing as the deployment rolls out to new service areas and operating conditions. Figure 1: Overview of Scape. Scape combines limited paired sim-and-real data with large-scale simulation data to predict scenario-conditioned target performance with calibrated uncertainty, which supports fine-grained deployment decisions by revealing scenario-specific variation and failures that aggregate metrics can hide. Simulation offers a scalable alternative, enabling cheap and fast policy rollouts across diverse conditions for scalable evaluation [5, 6, 7]. Recent real-to-sim efforts further ground simulators in real-world data [8, 9], improving the correlation between simulation and real-world testing results. Yet, the simulation-to-real gap is inevitable, and the residual bias prevents simulation-only validation from directly certifying real-world deployment. A complementary line of work therefore augments large-scale simulation testing with limited real-world trials to produce calibrated estimates of real-world policy performance, together with valid confidence intervals, through statistical methods such as control variates [10, 11] and prediction-powered inference [12, 13]. However, existing methods primarily estimate the aggregated performance metrics averaged over all deployment conditions. While aggregate estimates can support coarse policy evaluation, much of the information important for deployment decisions is lost through aggregation. A control policy that is strong on average may have varying performance across scenarios and exhibit severe failures in certain cases. Relying solely on the aggregated metrics would withhold deployment until the policy reaches near-perfect average performance, which is impractical. Instead, a sustainable deployment protocol is to gradually roll out deployment into scenarios where the policy is reliable, which facilitates a data flywheel to accumulate real-world rollouts through early deployment to keep improving the policies and gradually expand deployment to broader and more challenging scenarios. Such a protocol requires a fine-grained performance profile of the policy across scenarios and deployment conditions, which existing simulation-augmented policy evaluation methods fail to provide. To fill this gap, we instead formulate and study a new scenario-conditioned simulation-augmented policy evaluation problem: given limited real-world testing samples and abundant policy rollouts in a simulation, learn a model predicting the real-world policy performance under specific scene context, rather than only the average. As a solution, we propose Scape (Scenario-Conditioned Simulation-Augmented Policy Evaluation), as illustrated in Figure 1. It consists of two modules. First, we train a policy performance prediction model from a combination of limited real-world policy rollouts and large-scale simulation rollouts. Because naively mixing faithful real-world samples with biased simulation samples would sabotage training quality, we adapt the correction-before-augmentation principle from AI-augmented estimation [14]: a correction model trained from paired sim-and-real samples first debiases simulation outcomes, the corrected labels then supervise the prediction model training. We further adopt conformal prediction on top of the learned prediction model to provide calibrated uncertainty estimates, allowing developers and decision-makers to identify scenarios where the policy can be confidently deployed or additional real-world testing is necessary. In summary, our contributions are three-fold: 1) We propose Scape, a novel framework that combines limited real-world and large-scale simulation testing data for scenario-conditioned policy evaluation, addressing the limitations of existing simulation-augmented policy evaluation methods that only estimate the aggregated average performance metrics; 2) We instantiate Scape on two practical applications, autonomous driving and quadruped locomotion, and show through comprehensive experiments that it can effectively leverage simulation to enhance performance prediction accuracy and reduce the required real-world testing samples; 3) We further show that Scape enables various use cases in evaluation and deployment decision-making, such as bootstrapping evaluation on out-of-distribution (OOD) scenarios with simulation and scenario-level policy selection, which are difficult to support when only aggregate performance is estimated. 2 Related Work Simulation and simulation-augmented policy evaluation. Simulation has become a practical proxy for large-scale policy evaluation when real-world experiments are costly or risky. Across embodied simulation benchmarks, platforms such as CARLA [15], MetaDrive [16], MuJoCo [17], and IsaacGym [18] provide scalable and reproducible testbeds for evaluating autonomous driving, manipulation, and locomotion policies across diverse conditions. However, simulation-only evaluation remains limited by the sim-to-real gap. Although prior work has attempted to reduce this gap through domain randomization, dynamics randomization, adaptation, and real-to-sim alignment [19, 20, 21, 22, 23, 8], simulation is best viewed as an informative surrogate rather than a direct substitute for real-world validation. This naturally motivates simulation-augmented policy evaluation, which leverages both simulation and limited real-world rollouts to improve evaluation accuracy and sample efficiency. Representative works include Sim2Val [11], which exploits correlations across test platforms through control variates, and SureSim [13], which uses limited paired real-simulation evaluations to correct large-scale simulation-based estimates. Although these methods achieve strong estimation performance, they primarily target aggregated mean performance and confidence intervals around the mean. In contrast, Scape targets scenario-level real-world performance estimation by conditioning on scenario observations, enabling fine-grained estimates. Uncertainty quantification for policy evaluation. Policy comparison and selection require more than point estimates, since apparent performance differences can be dominated by evaluation noise when trials are limited. Single point estimates also do not quantify the variability of policy outcomes across trials or scenarios, limiting their usefulness for downstream decision-making. Prior work has constructed confidence intervals directly from real-world evaluations, including statistical procedures for policy success rates [24] and classical concentration inequalities for bounded performance metrics [25]. However, real-data-only intervals often require many costly trials to become informative, making them impractical. To reduce the real-world data burden, later work often leverages statistical frameworks, as in SureSim [13] and Sim2Val [11], to provide confidence intervals or variance-reduced estimates. However, these intervals are primarily designed for aggregated mean estimates, they quantify uncertainty in the population mean rather than the conditional variability of an incoming scenario. In contrast, Scape incorporates conformal prediction [26, 27] to produce calibrated uncertainty intervals for scenario-level estimates. Figure 2: Method overview of Scape. Scape corrects biased simulation labels using paired sim-and-real data, trains a scenario-conditioned performance predictor with corrected simulation-only data, and applies conformal calibration to quantify prediction uncertainty. 3 Scenario-Conditioned Simulation-Augmented Policy Evaluation This section formalizes the scenario-conditioned simulation-augmented policy evaluation problem (Sec. 3.1) and presents Scape, our solution framework for correcting biased simulation labels, training a scenario-level predictor, and calibrating uncertainty with conformal prediction (Sec. 3.2). 3.1 Problem Formulation We study how to predict the real-world performance of a given policy in a specific scenario using limited real-world testing and abundant simulation-based testing. Existing simulation-augmented policy evaluation methods such as Sim2Val [11] and SureSim [13] estimate an aggregate quantity, namely the policy’s mean performance over the entire scenario distribution, together with a confidence interval around the estimated mean. An aggregate estimate supports coarse evaluation and ranking, but hides scenario-specific variation and failures. We therefore target the scenario-conditioned real-world performance and provide a calibrated uncertainty interval for each scenario-level prediction. Formally, let X∈X denote a scenario, represented by the scene context and initial conditions under which the policy is evaluated. For a scenario X, let R be the policy’s real-world performance under an evaluation metric of interest, either continuous or binary, and S the corresponding metric measured through simulation testing. Both measure the same evaluation metric on the same scenario, with R trusted but expensive to estimate and S cheap but biased. Evaluating the policy in both worlds on the same scenario yields a paired sample (X,R,S)(X,R,S), whereas evaluating it in simulation alone yields a simulation-only sample (X,S)(X,S). We assume access to a small set of paired samples with abundant simulation-only samples, which are aggregated into three disjoint datasets: p=(xi,ri,si)i=1n⏟paired training set,s=(xk,sk)k=1ℓ⏟simulation-only training set,c=(xj,rj)j=1m⏟held-out calibration set,withℓ≫m+n. D_p=\(x_i,r_i,s_i)\_i=1^n_paired training set, D_s=\(x_k,s_k)\_k=1 _simulation-only training set, D_c=\(x_j,r_j)\_j=1^m_held-out calibration set, \ \ m+n. Our objective is two-fold. First, we train a model predicting the scenario-conditioned real-world policy performance from the paired and simulation-only training data p∪sD_p _s, fθ(x)≈μ⁡(x):=⁡[R∣X=x],f_θ(x)\;≈\;μ(x):=E[\,R X=x\,], which is the conditional mean for a continuous metric and the positive-class probability ℙ⁡(R=1∣X=x)P(R=1 X=x) for a binary one. Second, given the held-out calibration set cD_c and a miscoverage level α∈(0,1)α∈(0,1), we equip each prediction with an uncertainty region Cα(x)C_α(x)—an interval for continuous metrics and a set for binary ones—such that ℙ⁡(R∈Cα(X))≥ 1−α,P\! (R∈ C_α(X) )\;≥\;1-α, where the probability is over the exchangeable draw of the calibration and test scenarios. A wider Cα(x)C_α(x) reflects a less certain prediction and flags a scenario requiring further real-world testing. 3.2 Scape Framework A central challenge in simulation-augmented policy evaluation is exploiting large-scale simulation data without inheriting its sim-to-real bias. Since simulation rollouts are cheap but biased, naively mixing them with real-world data can pull predictions toward the simulation distribution. Prior work addresses this challenge through prediction-powered inference (PPI) [12, 13], using paired real-simulation evaluations to debias aggregate metric estimates. In principle, PPI could be adapted to train our scenario-conditioned predictor fθf_θ. In practice, fθf_θ must often be a neural network because robotic scenario contexts are high-dimensional, such as HD-map and agent-history features in autonomous driving or elevation maps in quadruped locomotion (Sec. 4.1). Yet, in our experiments, adapting PPI to high-dimensional, non-convex neural training, where its guarantees no longer directly apply, is unstable and provides little benefit from paired data (Sec. 4.2). We therefore propose Scape, which adopts an alternative bias-correction strategy that works reliably well for training the neural scenario-conditioned policy performance prediction model fθf_θ. The detailed algorithm of Scape is summarized in the Appendix (see Algorithm 1). Simulation-augmented policy performance prediction. We instead build Scape on an alternative bias-correction principle adopted from AI-augmented estimation (AAE) [14]. Originally developed for combining scarce human labels with abundant LLM-generated surrogate labels for human-choice modeling, AAE first corrects surrogate labels using paired data and then augments model training with the corrected data. We adapt this strategy to train neural, scenario-conditioned performance prediction models and find it works remarkably well in our experiments. Concretely, Scape trains the performance prediction model fθf_θ in two stages (Figure 2). First, we train a calibration model gϕg_φ on the paired set pD_p to predict the real-world metric from a scenario together with its simulation result, gϕ(x,s)≈[R∣X=x,S=s],g_φ(x,s)\;≈\;E[\,R X=x,\,S=s\,], where conditioning on x lets gϕg_φ correct the sim-to-real bias based on scenario rather than applying a single global offset. We then apply gϕg_φ to every simulation-only sample (xk,sk)∈s(x_k,s_k) _s to obtain a corrected label r~k=gϕ(xk,sk) r_k=g_φ(x_k,s_k), turning the abundant simulation-only data into a large corrected set pseudo=(xk,r~k)k=1ℓD_pseudo=\(x_k, r_k)\_k=1 . In the second stage, we train the scenario-conditioned prediction model fθf_θ on the union of the trusted real-world labels and the corrected simulation-only labels, aug=(xi,ri)i=1n∪pseudo,θ^∈arg⁡minθ1n+ℓ∑(x,r)∈augℒ⁡(fθ(x),r),D_aug=\(x_i,r_i)\_i=1^n\;∪\;D_pseudo, θ\;∈\; _θ\; 1n+ \!\! _(x,r) _aug\!\!L (f_θ(x),\,r ), where ℒL is the squared error for continuous metrics and the cross-entropy for discrete metrics. The final prediction model fθf_θ takes only the scenario context x as input without the surrogate metric s. This design enables predicting real-world performance at deployment without requiring access to a simulator for collecting additional simulation rollouts given a scenario of interest. Uncertainty quantification via conformal prediction. The prediction model fθf_θ provides scenario-conditioned point prediction, but trustworthy deployment decisions also require quantifying the uncertainty of the prediction. We incorporate conformal prediction [26, 27] into Scape to construct the calibrated uncertainty set Cα(x)C_α(x). Conformal prediction is model-free and, under exchangeability of the calibration and test scenarios, provides finite-sample marginal coverage ℙ⁡(R∈Cα(X))≥1−αP(R∈ C_α(X))≥ 1-α with no distributional assumptions on the prediction model fθf_θ or the data. On the held-out calibration set c=(xj,rj)j=1mD_c=\(x_j,r_j)\_j=1^m, we score each scenario against its real-world label rjr_j. For a continuous metric, we compute the calibration residual ej=|rj−fθ(xj)|e_j=|r_j-f_θ(x_j)| for each (xj,rj)∈c(x_j,r_j) _c, sort the scores e(1)≤⋯≤e(m)e_(1)≤·s≤ e_(m), set q^1−α=e(⌈(m+1)(1−α)⌉) q_1-α=e_( (m+1)(1-α) ), and return the interval Cα(x)=[fθ(x)−q^1−α,fθ(x)+q^1−α]C_α(x)=[\,f_θ(x)- q_1-α,\;f_θ(x)+ q_1-α\,]. For a discrete metric we compute the analogous classification conformity scores on cD_c and return a calibrated prediction set. Note that this split conformal prediction constructs the interval Cα(x)C_α(x) with a scenario-invariant half-width q^1−α q_1-α. The resulting intervals therefore do not provide scenario-dependent widths or conditional coverage. Nevertheless, the interval center fθ(x)f_θ(x) still captures variations across scenarios, which allow the calibrated uncertainty sets to support fine-grained deployment decisions, for example, relative to a performance threshold, an interval can certify deployment, reject deployment, or flag scenarios for additional testing. 4 Experiments Figure 3: Paired sample construction in (a) nuPlan and (b) quadruped experiments. In nuPlan, each pair is constructed from the same driving scenario by measuring the target metric R with closed-loop evaluation and the surrogate metric S with open-loop evaluation. In quadruped, each pair matches a real-world rollout with a reconstructed-sim rollout under the same scene context, robot placement, and command. 4.1 Experimental Settings We evaluate Scape in two embodied domains—autonomous driving and quadruped velocity tracking—shown in Figure 3. We first study both domains in a Sim2Sim setting, treating outcomes from a higher-fidelity or perturbed simulator as target labels R and those from a cheaper or nominal simulator as surrogate measurements S. In each domain, we construct a paired training set pD_p and a simulation-only training set sD_s for training the point-prediction models. In the experiments, we vary the fractions of pD_p and sD_s used for training to study how prediction accuracy scales with paired and surrogate data. We additionally reserve three paired splits: a validation set vD_v for neural-model selection, a calibration set cD_c for split-conformal calibration, and a test set testD_test for final evaluation. We further evaluate quadruped velocity tracking in Sim2Real using real-world rollouts collected from a physical Unitree Go2 across multiple indoor and outdoor scenarios. Autonomous driving (nuPlan). We conduct experiments on nuPlan [28] and evaluate three ML-based planners, urban_driver_open_loop_model, vector_model, and simple_vector_model provided by nuPlan, each trained under their default configurations. The scenario feature x is extracted by a frozen UrbanDriver encoder. We record four evaluation metrics: average displacement error (ADE), time-to-collision under 1 s (TTC <1<1s), drivable-area compliance, and no ego at-fault collision (see Appendix for their definition). The surrogate metric S comes from fast open-loop evaluation with non-reactive agents, while the target metric R comes from closed-loop evaluation with reactive agents. Detailed dataset statistics and feature composition are provided in Appendix A.2. Quadruped velocity tracking. We evaluate an RL-trained quadruped velocity-tracking policy in both Sim2Sim and Sim2Real settings. Each scenario feature x consists of local terrain geometry, a linear velocity command, and a yaw-rate command. Each 2-s rollout produces two target metrics: velocity-tracking error MAEvelMAE_vel and yaw-rate-tracking error MAEyawMAE_yaw. In Sim2Sim, S is measured in a nominal simulator and R in a perturbed simulator with dynamics randomization and external disturbances. In Sim2Real, R is collected from physical Go2 rollouts and S from matched simulation rollouts. We construct paired real-sim scenes using the GaussGym [29] real-to-sim pipeline and align robot poses with HLoc [30] and SuperGlue [31]. Dataset details are provided in Appendix A.3. Baselines. Scape addresses a new setting without direct off-the-shelf baselines, so we compare it against two families of adapted methods. The first family consists of aggregate-mean estimators, adapted from prior simulation-augmented policy evaluation methods [11, 13], that return a single scenario-independent mean estimate: 1) MC (Monte Carlo) estimates the mean from the real-world labels only [10]; 2) CV (Control Variates) treats the simulation metric as a control variate [10, 11]; 3) PPI (Prediction-Powered Inference) debiases surrogate-based mean estimates using paired samples [12, 13]. The second family consists of scenario-conditioned neural predictors fθ(x)f_θ(x): 4) R-Only trains on the real-world labels only; 5) RS-Mix naively mixes real-world and simulation labels; 6) PPI-N trains with the rectified PPI objective [12, Sec. 4.3]. For a fair comparison, all baselines and Scape use the same paired and surrogate data budgets to train the point-prediction models. After training, we apply the same split-conformal procedure, using the same calibration set cD_c, to all methods to isolate the effect of the point-prediction model on the resulting prediction intervals.11 1 Native PPI and CV confidence intervals target the population mean rather than the outcome for a new scenario, so we use only the mean-estimation components of these baselines in our experiments. The hyperparameters and optimization protocol for the neural baselines and Scape are detailed in Appendix 6. 4.2 Experimental Results Figure 4: Test loss vs. fraction of paired-label on autonomous driving and quadruped locomotion tasks. The paired-data fraction denotes the fraction of the full paired training set pD_p used for training. The Urban Driver setting contains 83.683.6K paired training samples and 119.2119.2K surrogate-only samples; the quadruped Sim2Sim setting contains 300300 paired training samples and 1,4931,493 surrogate-only samples. Panels (a)–(d) report autonomous driving Sim2Sim results, and panels (e)–(f) report quadruped Sim2Sim velocity-tracking results. Curves report mean ± standard error over 10 random paired-data seeds. Figure 4 reports Sim2Sim scenario-level prediction errors as the paired-data fraction varies for nuPlan and quadruped velocity tracking22 2 Figure 4 (a)-(d) shows the results for UrbanDriver. The results for the other planners are in Appendix A.7.. In both domains, scenario-conditioned neural methods outperform aggregate-mean estimators, confirming the need for fine-grained evaluation. More importantly, Scape is the only method that consistently outperforms R-Only across paired-data fractions and metrics. In contrast, RS-Mix and PPI-N often fail to exploit low-cost simulation effectively, as biased surrogate labels can degrade prediction accuracy. Across the complete evaluation suite reported in the main paper and appendix, Scape reduces prediction errors by 4.9%/34.7%4.9\%/34.7\% on nuPlan and 14.5%/27.7%14.5\%/27.7\% on quadruped tracking relative to the scenario-conditioned neural/aggregate statistical baselines. These averages aggregate all evaluated planners, settings, paired-data fractions, and target metrics. Compared with the strongest baseline, the corresponding gains are 2.0%2.0\% and 4.08%4.08\%. It also matches the strongest baseline full-budget performance with 20%20\%–60%60\% fewer paired labels on nuPlan and 10%10\%–60%60\% fewer on quadruped velocity tracking. More importantly, Scape is the only surrogate-augmented method that consistently improves on R-Only. It shows that simply enlarging the training set with biased surrogate labels does not reliably lead to benefits in prediction accuracy and sample efficiency. The consistent gains of Scape therefore reflect its ability to correct those labels and turn them into useful supervision. These gains are also practically meaningful: even increasing the amount of real labeled data available to R-Only by 10×10× reduces its prediction error by only 8.81%8.81\% on average. For uncertainty quantification, Table 1 reports conformal widths at paired-label fraction 1.01.0. Scape achieves the narrowest width in all six metric settings,33 3 Full paired-data sweeps and empirical coverage results are in Appendix A.8. while aggregate-mean estimators often produce large or full prediction sets. Thus, Scape improves prediction accuracy while providing more efficient calibrated uncertainty at valid coverage. Table 1: Split conformal width at nominal coverage 1−α=0.951-α=0.95. Results for fraction of paired-label 1.01.0 are reported. For the nuPlan regression target ADE, width is the prediction-interval length 2q^2 q in meters; for the three nuPlan binary targets, width is the average conformal prediction-set size |C⁡(x)|∈[0,2]|C(x)|∈[0,2]. For quadruped Sim2Sim, width is the regression interval length reported as ×10−2× 10^-2. Cells are mean ± standard error over 10 paired-data seeds. Bold marks the narrowest width per column. nuPlan Urban Driver Go2 Sim2Sim Method ADE TTC<1<1s Drivable Collision Velocity Yaw Rate Scape ±0.26 40.07\!±\!0.26 ±0.010 1.447\!±\!0.010 ±0.004 1.118\!±\!0.004 ±0.004 1.430\!±\!0.004 ±0.48 10.84\!±\!0.48 ±0.39 7.60\!±\!0.39 ppi-n ±0.1644.44\!±\!0.16 ±0.0041.614\!±\!0.004 ±0.0041.233\!±\!0.004 ±0.0041.573\!±\!0.004 ±0.3811.42\!±\!0.38 ±0.3110.27\!±\!0.31 rs-mix ±0.1644.65\!±\!0.16 ±0.0061.505\!±\!0.006 ±0.0021.152\!±\!0.002 ±0.0031.495\!±\!0.003 ±0.2811.56\!±\!0.28 ±0.229.12\!±\!0.22 r-only ±0.3041.02\!±\!0.30 ±0.0031.497\!±\!0.003 ±0.0031.155\!±\!0.003 ±0.0091.474\!±\!0.009 ±0.5911.12\!±\!0.59 ±0.317.75\!±\!0.31 mc ±0.0050.27\!±\!0.00 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.8812.64\!±\!0.88 ±0.3111.95\!±\!0.31 cv ±0.0050.29\!±\!0.00 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.8812.50\!±\!0.88 ±0.2911.88\!±\!0.29 ppi ±0.0050.31\!±\!0.00 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.8412.41\!±\!0.84 ±0.2911.75\!±\!0.29 Sim2Real quadruped velocity tracking. We conduct the quadruped velocity tracking using policy rollouts collected on a physical Unitree Go2 robot. We collected data by placing the robot at diverse positions across five different scenes, including flat and sloped terrains44 4 Detailed scene structures and reconstructed pairs are provided in Appendix A.3.. As shown in Table 2, Scape achieves the lowest MAE on both target metrics, reducing error by 11.3%11.3\% on MAEvelMAE_vel and 8.7%8.7\% on MAEyawMAE_yaw compared with the strongest baseline. These results indicate that Scape remains effective beyond Sim2Sim evaluation and can effectively facilitate robot policy evaluation and deployment in the physical environments. Table 2: Quadruped Sim2Real velocity tracking. Test MAE on the sim-to-real velocity tracking task. Cells are mean ± standard error over 10 random paired-data seeds. Bold marks the lowest error per row. Metric (×10−3× 10^-3) mc cv ppi r_only rs_mix ppi-n Scape RvelR_vel test MAE ±0.9616.55\!±\!0.96 ±0.8617.07\!±\!0.86 ±1.5226.51\!±\!1.52 ±0.7613.29\!±\!0.76 ±0.9916.92\!±\!0.99 ±1.7919.16\!±\!1.79 ±0.70 11.79\!±\!0.70 RyawR_yaw test MAE ±0.415.24\!±\!0.41 ±0.415.48\!±\!0.41 ±0.566.86\!±\!0.56 ±0.415.16\!±\!0.41 ±0.476.22\!±\!0.47 ±0.455.37\!±\!0.45 ±0.32 4.71\!±\!0.32 4.3 Additional Experiments and Analyses on nuPlan We conduct additional nuPlan experiments to analyze and highlight several key characteristics of Scape. Figures 5 and 6 show the results on UrbanDriver with full paired data, and the complete results are provided in Appendix A.9 and A.10. Scape scales with surrogate data where baselines saturate. We examine how prediction accuracy changes as surrogate-only data increases. We sweep the surrogate data fractions given fixed paired-data budget for the three scenario-conditioned methods. As shown in Figure 5, Scape keeps improving as more cheap surrogate data is incorporated. In contrast, RS-Mix peaks early and then degrades because naive mixing pollutes the training signal. PPI-N also benefits from additional surrogate data but exhibits higher empirical training variance and instability. We hypothesize that this behavior arises because its subtractive correction is estimated from limited paired data, which can induce conflicting, high-variance gradients during non-convex neural-network training. In contrast, Scape separates correction from augmentation. It first converts surrogate data into target-aligned pseudo-labels and then trains the final predictor separately. Scape benefits performance prediction on out-of-distribution scenarios. Deployment often expands from familiar scenarios to broader OOD conditions, requiring performance estimates before target testing. We test whether Scape can use surrogate-only data from reconstructed OOD scenarios to improve prediction without OOD target labels. In nuPlan, we leave out one city from paired training and evaluate on the test set (Figure 5). Across all evaluated planners, without OOD surrogate data, Scape already achieves the lowest average loss, outperforming the other scenario-conditioned baselines by 3.3%3.3\%; adding surrogate-only data from the held-out city further reduces its loss by 4.8%4.8\%, recovering 95.3%95.3\% of full-training performance and widening its margin over the scenario-conditioned baselines to 5.3%5.3\%. Note that this experiment evaluates OOD point-prediction performance; we do not claim conformal coverage under this distribution shift. Figure 5: Additional nuPlan results. Left: city-level OOD generalization results for UrbanDriver. Bars show mean ± standard error over 4 city holdouts. Right: UrbanDriver results at paired data fraction 1.01.0 with varying surrogate-only fraction. Curves report mean ± standard error over 9 (3 paired-data seed × 3 surrogate-data seed) random seeds. Figure 6: Evaluator guided planner selection accuracy and corresponding selected performance. Results for fraction of paired-label 1.01.0 are reported. Learned evaluators are used to select the predicted best planner for each scenario. Bars show mean ± standard error over 10 paired-data seeds. Scenario-level evaluation enables scenario-conditioned deployment strategy. Scape enables scenario-conditioned policy selection instead of using a single policy universally. To demonstrate this benefit, we use each evaluator as a router that selects the predicted best planner for each scenario. Figure 6 shows that Scape achieves the best selected performance on all four metrics, improving over the scenario-conditioned and aggregate-mean baselines by 1.67%1.67\% and 6.47%6.47\% on average, respectively. Although aggregate-mean baselines show high selection accuracy on drivable-area compliance and no ego at-fault collision, they often route all scenarios to the best-average planner, masking rare failures. In contrast, Scape often selects a near-best planner even when it misses the oracle best, yielding better ensemble performance. 5 Conclusion In this work, we present Scape, a scenario-conditioned, simulation-augmented framework for policy evaluation that estimates scenario-conditioned real-world policy performance with calibrated uncertainty from limited paired sim-and-real evaluations and large-scale simulation-only rollouts. Across autonomous driving and quadruped velocity tracking, and validated from simulation to a physical Unitree Go2, Scape predicts real-world performance more accurately than scene-conditioned neural and aggregate statistical baselines, uses scarce real-world testing samples more efficiently, generalizes better to OOD scenarios, and produces narrower calibrated intervals. It also enables fine-grained, scenario-level policy comparison and deployment strategy. Limitations and future work. Scape currently uses split conformal prediction, which provides an uncertainty set with identical size over the scenario distribution, instead of true conditional coverage with scenario-dependent width. Our real-world evaluation also remains limited in scope, and broader experiments across different embodiments, tasks, and a broader range of scenarios are helpful to fully characterize Scape’s performance. Finally, while Scape yields strong empirical performance, rigorous theoretical analysis would further strengthen its value for trustworthy policy deployment. References Grigorescu et al. [2020] S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3):362–386, 2020. Brohan et al. [2022] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. RT-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. Tang et al. [2025] C. Tang, B. Abbatematteo, J. Hu, R. Chandra, R. Martín-Martín, and P. Stone. Deep reinforcement learning for robotics: A survey of real-world successes. Annual Review of Control, Robotics, and Autonomous Systems, 8(1):153–188, 2025. Kalra and Paddock [2016] N. Kalra and S. M. Paddock. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94:182–193, 2016. Mittal et al. [2023] M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y. Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State, M. Hutter, and A. Garg. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automation Letters, 8(6):3740–3747, 2023. Li et al. [2023] Q. Li, Z. M. Peng, L. Feng, Z. Liu, C. Duan, W. Mo, and B. Zhou. ScenarioNet: Open-source platform for large-scale traffic scenario simulation and modeling. In Advances in Neural Information Processing Systems, volume 36, pages 3894–3920, 2023. Gulino et al. [2023] C. Gulino, J. Fu, W. Luo, G. Tucker, E. Bronstein, Y. Lu, J. Harb, X. Pan, Y. Wang, X. Chen, J. D. Co-Reyes, R. Agarwal, R. Roelofs, Y. Lu, N. Montali, P. Mougin, Z. Yang, B. White, A. Faust, R. McAllister, D. Anguelov, and B. Sapp. Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research. In Advances in Neural Information Processing Systems, volume 36, pages 7730–7742, 2023. Li et al. [2024] X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kirmani, S. Levine, J. Wu, C. Finn, H. Su, Q. Vuong, and T. Xiao. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024. Zhang et al. [2025] K. Zhang, S. Sha, H. Jiang, M. Loper, H. Song, G. Cai, Z. Xu, X. Hu, C. Zheng, and Y. Li. Real-to-sim robot policy evaluation with gaussian splatting simulation of soft-body interactions. arXiv preprint arXiv:2511.04665, 2025. Owen [2013] A. B. Owen. Monte Carlo Theory, Methods and Examples. Self-published, 2013. URL https://artowen.su.domains/mc/. Luo et al. [2025] R. Luo, H. Yang, M. Watson, A. Sharma, S. Veer, E. Schmerling, and M. Pavone. Sim2Val: Leveraging correlation across test platforms for variance-reduced metric estimation. In Proceedings of the Conference on Robot Learning, 2025. Angelopoulos et al. [2023] A. N. Angelopoulos, S. Bates, C. Fannjiang, M. I. Jordan, and T. Zrnic. Prediction-powered inference. Science, 382(6671):669–674, 2023. doi:10.1126/science.adi6000. Badithela et al. [2025] A. Badithela, D. Snyder, L. Zha, J. Mikhail, M. O’Kelly, A. Dixit, and A. Majumdar. Reliable and scalable robot policy evaluation with imperfect simulators. arXiv preprint arXiv:2510.04354, 2025. Wang et al. [2026] M. Wang, D. J. Zhang, and H. Zhang. Large language models for market research: A data-augmentation approach. Marketing Science, 2026. doi:10.1287/mksc.2025.0009. Dosovitskiy et al. [2017] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, volume 78 of Proceedings of Machine Learning Research, pages 1–16. PMLR, 2017. Li et al. [2022] Q. Li, Z. Peng, L. Feng, Q. Zhang, Z. Xue, and B. Zhou. MetaDrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3461–3475, 2022. Todorov et al. [2012] E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012. Makoviychuk et al. [2021] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac Gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021. Tobin et al. [2017] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 23–30. IEEE, 2017. Peng et al. [2018] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE International Conference on Robotics and Automation, pages 3803–3810. IEEE, 2018. Tan et al. [2018] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. In Robotics: Science and Systems, 2018. Hwangbo et al. [2019] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872, 2019. Kumar et al. [2021] A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: Rapid motor adaptation for legged robots. In Proceedings of Robotics: Science and Systems, Virtual, July 2021. doi:10.15607/RSS.2021.XVII.011. Vincent et al. [2024] J. A. Vincent, H. Nishimura, M. Itkina, P. Shah, M. Schwager, and T. Kollar. How generalizable is my behavior cloning policy? a statistical approach to trustworthy performance evaluation. IEEE Robotics and Automation Letters, 9(10):8619–8626, 2024. Hoeffding [1963] W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13–30, 1963. Lei et al. [2018] J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018. doi:10.1080/01621459.2017.1307116. Angelopoulos and Bates [2023] A. N. Angelopoulos and S. Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. Foundations and Trends in Machine Learning, 16(4), 2023. doi:10.1561/2200000101. Caesar et al. [2021] H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021. Escontrela et al. [2025] A. Escontrela, J. Kerr, A. Allshire, J. Frey, R. Duan, C. Sferrazza, and P. Abbeel. GaussGym: An open-source real-to-sim framework for learning locomotion from pixels, 2025. URL https://arxiv.org/abs/2510.15352. Sarlin et al. [2019] P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12716–12725, 2019. Sarlin et al. [2020] P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich. SuperGlue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4938–4947, 2020. Appendix A Appendix Appendix Contents A.1 Scape Algorithm Details Data: Paired training set p=(xi,ri,si)i=1nD_p=\(x_i,r_i,s_i)\_i=1^n, surrogate-only set s=(xk,sk)k=1ℓD_s=\(x_k,s_k)\_k=1 , calibration set c=(xj,rj)j=1mD_c=\(x_j,r_j)\_j=1^m, miscoverage level α Result: Scenario-level estimator fθf_θ and calibrated prediction interval Cα(x)C_α(x) 1: gϕ←TrainCorrection(p)g_φ← TrainCorrection(D_p) 2: for (xk,sk)∈s(x_k,s_k) _s do 3: r~k←gϕ(xk,sk) r_k← g_φ(x_k,s_k) 4: end 5: pseudo←(xk,r~k)k=1ℓD_pseudo←\(x_k, r_k)\_k=1 6: pR←(xi,ri)i=1nD_p^R←\(x_i,r_i)\_i=1^n 7: aug←pR∪pseudoD_aug _p^R _pseudo 8: fθ←TrainEvaluator(aug)f_θ← TrainEvaluator(D_aug) 9: for (xj,rj)∈c(x_j,r_j) _c do 10: ej←|rj−fθ(xj)|e_j←|r_j-f_θ(x_j)| 11: end 12: q^1−α←ConformalQuantile(ejj=1m,α) q_1-α← ConformalQuantile(\e_j\_j=1^m,α) 13: for test scenario x do 14: r^(x)←fθ(x) r(x)← f_θ(x) 15: Cα(x)←[fθ(x)−q^1−α,fθ(x)+q^1−α]C_α(x)←[f_θ(x)- q_1-α,\,f_θ(x)+ q_1-α] 16: end 17: return fθ(x),Cα(x)f_θ(x),C_α(x) ConformalQuantile Input: scores ejj=1m\e_j\_j=1^m, miscoverage α. Sort scores: e(1)≤⋯≤e(m).e_(1)≤·s≤ e_(m). Set h=⌈(m+1)(1−α)⌉.h= (m+1)(1-α) . Return q^1−α=e(min⁡h,m). q_1-α=e_( \h,m\). For discrete metrics, replace eje_j with the corresponding classification conformal score. Algorithm 1 Scape Algorithm A.1 summarizes the full Scape pipeline. Lines 1–4 train the correction model gϕg_φ on paired samples and use it to relabel each surrogate-only example with a corrected target r~k r_k. Lines 5–7 merge the trusted paired labels with the corrected surrogate labels to form the augmented training set. Line 8 trains the scenario-level evaluator fθf_θ using this augmented set, so that the final predictor depends only on scenario context x. Lines 9–12 compute calibration residuals on the held-out calibration set cD_c. Line 13 converts these residuals into the split-conformal quantile q^1−α q_1-α, using the finite-sample correction shown in the ConformalQuantile box. Lines 14–17 apply the calibrated evaluator to each test scenario and return both the point prediction fθ(x)f_θ(x) and its prediction interval Cα(x)C_α(x). For binary metrics, the residual score is replaced by the corresponding classification conformity score, yielding a conformal prediction set instead of a regression interval. A.2 nuPlan Dataset Statistics Data collection. We first train the three ML-based motion planner provided by nuPlan: urban_driver_open_loop_model, vector_model, and simple_vector_model. These models serve as the target policies to be evaluated. To maintain diversity, each planner dataset covers 62 scenario types, such as crosswalk traversal, traffic-light stopping, and lane changing, from four map locations: Singapore, Boston, Las Vegas, and Pittsburgh. In total, for each planner dataset, we collect approximately 83.683.6K paired training samples for pD_p, 15.515.5K paired validation samples for vD_v, 28.328.3K paired test samples for testD_test, 1.151.15K held-out calibration samples for cD_c, and 119119K–135135K surrogate-only samples for sD_s. Detailed dataset composition is provided below. Frozen Urban Driver feature representation. For the nuPlan experiments, we use a separately trained Urban Driver feature encoder to convert each scenario observation into a fixed 256-dimensional feature vector x∈ℝ256x ^256. The encoder is kept frozen during SCAPE training, and all downstream evaluator models operate only on this extracted vector x. The input to the encoder contains ego-vehicle history, nearby-agent history, local vector-map context, and traffic-light state. The ego history is represented as a temporal polyline whose per-timestep feature is (x,y,θ)∈ℝ3(x,y,θ) ^3. Each nearby-agent polyline uses a 7-dimensional per-timestep feature containing position, heading, velocity, yaw rate, length, and width: (x,y,θ,v,θ˙,l,w)∈ℝ7.(x,y,θ,v, θ,l,w) ^7. Traffic-light status is encoded as a 4-dimensional one-hot vector indicating green, yellow, red, or unknown state. Local map elements, including lanes, lane boundaries, stop lines, crosswalks, and route lanes, are represented as vector polylines in the local ego-centric coordinate frame. The encoder first maps each raw polyline element to a learned local descriptor in ℝ256R^256. Thus ego, agent, and map polylines are all embedded into the same 256-dimensional descriptor space after local encoding, even though their raw input channel dimensions differ. A global attention module then uses the ego query to aggregate ego, agent, map, and traffic-light context into a single scene-level feature vector x∈ℝ256.x ^256. This 256-dimensional vector is the only feature used by the SCAPE evaluator and the neural baselines; the raw ego, agent, map, and traffic-light inputs are not updated or re-encoded during evaluator training. Table 3: Detailed city-level composition of the Scape nuPlan self-driving experiments dataset splits. Planner Split Singapore Boston Las Vegas Pittsburgh Urban Driver Open-Loop Train 19,504 20,145 23,833 20,123 Val 3,555 3,574 4,547 3,791 Test 6,993 6,347 7,737 7,188 Calibration 264 254 355 276 Surrogate 31,521 30,939 39,890 16,841 Vector Model Train 19,504 20,145 23,833 20,123 Val 3,555 3,574 4,547 3,791 Test 6,993 6,347 7,737 7,188 Calibration 266 254 355 276 Surrogate 31,532 30,939 39,890 33,118 Simple Vector Model Train 19,504 20,145 23,833 20,122 Val 3,555 3,574 4,547 3,791 Test 6,993 6,347 7,737 7,188 Calibration 265 254 355 276 Surrogate 31,531 30,939 39,890 33,118 A.3 Quadruped Velocity-Tracking Dataset Statistics Data collection. We evaluate a quadruped velocity-tracking policy under two regimes: Sim2Sim, where both the target R and surrogate S come from simulation with different domain-randomization seeds, and Sim2Real, where R is measured on the physical robot and S is obtained from a matched IsaacLab rollout. Each terrain is reconstructed from an iPhone Polycam scan, yielding a textured mesh. For Sim2Sim, paired samples pD_p span 44 indoor/outdoor scenes; for Sim2Real, samples are collected over 9595 HLoc [30]-localized deploy poses across 55 scenes (indoor, grass, dirt, dirt1, brick). In total, for the Sim2Sim setting we collect 300300 paired training samples for pD_p, 200200 paired validation samples for vD_v that also serve as the held-out calibration set cD_c, 200200 paired test samples for testD_test, and 1,4931,493 surrogate-only samples for sD_s; for Sim2Real, 65/15/1565/15/15 paired train/val/test samples and 195195 surrogate-only samples. Paired real and reconstructed simulation is shown in Figure 7. Detailed dataset composition is provided below. Figure 7: Paired real and reconstructed simulation scenes. The top row shows real-world scenes, and the bottom row shows their corresponding reconstructed simulation environments. Table 4: Detailed scene-level composition of the Scape quadruped velocity tracking dataset splits. For each setting, the paired pool (pD_p) is randomly shuffled and split into Train/Val/Test (300/200/200300/200/200 for Sim2Sim, 65/15/1565/15/15 for Sim2Real) independently for each of the 1010 random seeds, so we report the per-scene totals; surrogate samples (sD_s) are unsplit. Sim2Sim scene1 scene2 scene3 scene4 Total Paired (pD_p) 59 293 29 319 700 Surrogate (sD_s) 109 619 69 696 1,493 Sim2Real indoor grass dirt dirt1 brick Total Paired (pD_p) 15 25 25 9 21 95 Surrogate (sD_s) 37 50 46 22 40 195 Frozen quadruped velocity-tracking feature representation. For the quadruped velocity-tracking experiments, we represent each locomotion scenario using a hand-engineered, fixed feature vector x∈ℝdx ^d constructed from the robot’s local terrain geometry and velocity command. Unlike the nuPlan experiments, no learned backbone encoder is used. This feature vector is kept fixed during SCAPE training, and all downstream models, including the scenario evaluator fθf_θ and the Scape correction model gϕg_φ, operate only on x. The terrain component is computed by a body-frame raycaster on the Polycam terrain mesh and stores per-cell terrain heights relative to the robot base. In the Sim2Sim setting, we use a 10×810× 8 local elevation crop covering X∈[−0.25,1.25]X∈[-0.25,1.25] m and Y∈[−0.60,0.60]Y∈[-0.60,0.60] m at 0.15 m resolution, yielding an 80-dimensional height feature. This is concatenated with the planar command [vx,ωz][v_x, _z], where vx∈[0.1,0.5]v_x∈[0.1,0.5] m s-1 and ωz∈[−0.5,0.5] _z∈[-0.5,0.5] rad s-1. We do not include a scene identifier in Sim2Sim, since the same Go2 policy is evaluated across all four scenes and both R and S are obtained from simulator rollouts. The resulting feature is x∈ℝ82x ^82, formed by the 80-dimensional elevation crop and the 2-dimensional command. In the Sim2Real setting, we use a smaller 11×411× 4 local elevation crop covering X∈[−0.25,1.40]X∈[-0.25,1.40] m and Y∈[−0.30,0.30]Y∈[-0.30,0.30] m at 0.15 m resolution, which matches the physical raycast used by the deployed policy at run time. The 44-dimensional crop is concatenated with the recorded planar command at each HLoc-localized deployment pose, where vx∈[0.4,0.6]v_x∈[0.4,0.6] m s-1 and ωz∈[−0.2,0.2] _z∈[-0.2,0.2] rad s-1. We additionally include a 5-dimensional scene one-hot vector over indoor, grass, dirt, dirt1, and brick, since Sim2Real residuals are correlated with visually and tactilely different real-world terrains. The resulting feature is x∈ℝ51x ^51, formed by the 44-dimensional elevation crop, the 2-dimensional command, and the 5-dimensional scene identifier. A.4 Baseline Definition Table 5: Baseline definitions. Statistical baselines estimate aggregate target performance, while neural baselines train scenario-level predictors fθ(x)f_θ(x). Type Method Definition Statistical MC μ^MC=1n∑iri μ_MC= 1n _ir_i Statistical CV μ^CV=1n∑i(ri−β^si)+1ℓ∑kβ^sk,β^=ℓn+ℓCov^(s,r)Var^(s) μ_CV= 1n _i(r_i- βs_i)+ 1 _k βs_k, β= n+ Cov(s,r) Var(s) Statistical PPI μ^PPI=1ℓ∑ksk+1n∑i(ri−si) μ_PPI= 1 _ks_k+ 1n _i(r_i-s_i) Neural R-Only ℒR=1n∑iℒ⁡(fθ(xi),ri) _R= 1n _iL(f_θ(x_i),r_i) Neural RS-Mix ℒRS=ℒR+1ℓ∑kℒ⁡(fθ(xk),sk) _RS=L_R+ 1 _kL(f_θ(x_k),s_k) Neural PPI-N ℒPPI-N=1ℓ∑kℒ⁡(fθ(xk),sk)+1n∑i[ℒ⁡(fθ(xi),ri)−ℒ⁡(fθ(xi),si)] _PPI -N= 1 _kL(f_θ(x_k),s_k)+ 1n _i [L(f_θ(x_i),r_i)-L(f_θ(x_i),s_i) ] Table 5 summarizes the baselines used in our experiments. The first group contains statistical aggregate estimators, which do not condition on scenario observations and therefore provide a single scenario-independent target estimate. MC estimates target performance using only paired target labels, serving as the real-data-only reference. CV uses the surrogate metric as a control variate to reduce the variance of the target mean estimate when S and R are correlated. PPI estimates the surrogate mean on the large simulation-only set and corrects its bias using paired residuals, representing a prediction-powered aggregate estimator. The second group contains scenario-conditioned neural predictors fθ(x)f_θ(x), which are directly comparable to Scape in producing per-scenario predictions. R-Only trains only on paired target labels and measures the performance achievable without using surrogate-only data. RS-Mix naively augments the paired target labels with raw surrogate labels, testing whether low-cost simulation data can be used directly despite sim-to-real bias. PPI-N adapts the PPI correction objective to neural training by combining surrogate-only supervision with a paired residual correction term, testing correction during training as an alternative to the correction-before-augmentation design of Scape. A.5 Scape MLP Hyperparameters Training objectives and label weighting. The correction model gϕg_φ is trained on the paired data: ℒg=1n∑i=1nℒ⁡(gϕ(xi,si),ri),r~k=gϕ(xk,sk).L_g= 1n _i=1^nL (g_φ(x_i,s_i),r_i ), r_k=g_φ(x_k,s_k). The final predictor fθf_θ is then trained on the real and corrected pseudo labels: ℒf=1n+ℓ(∑i=1nℒ⁡(fθ(xi),ri)+wpseudo∑k=1ℓℒ⁡(fθ(xk),r~k)).L_f= 1n+ ( _i=1^nL (f_θ(x_i),r_i )+w_pseudo _k=1 L (f_θ(x_k), r_k ) ). All main experiments use wpseudo=1w_pseudo=1, assigning equal per-sample weights to real and corrected pseudo labels without dataset-specific tuning. Table 6: MLP architecture and training hyperparameters (nuPlan). The scenario evaluator fθf_θ is used by R-Only, RS-Mix, PPI-N, and the final evaluator in Scape. The correction model gϕg_φ additionally takes the surrogate metric vector s as input and predicts corrected target metrics r~ r. The four output dimensions correspond to ADE, TTC <1<1s, drivable-area compliance, and no ego at-fault collision. Component Scenario evaluator fθf_θ Correction model gϕg_φ Input x∈ℝ256x ^256 [x,s]∈ℝ260[x,s] ^260 Output r^=fθ(x)∈ℝ4 r=f_θ(x) ^4 r~=gϕ(x,s)∈ℝ4 r=g_φ(x,s) ^4 Hidden layers [1024,512,256,128][1024,512,256,128] [1024,512,256,128][1024,512,256,128] Activation ReLU ReLU Dropout 0.20.2 0.20.2 Trainable parameters 0.9530.953M 0.9570.957M Loss MSE ++ 3 BCE MSE ++ 3 BCE Optimizer Adam Adam Learning rate 3×10−53× 10^-5 3×10−53× 10^-5 Weight decay 1×10−41× 10^-4 1×10−41× 10^-4 Batch size 256256 256256 Epochs 500500 500500 nuPlan. For both fθf_θ and gϕg_φ, all output metrics are weighted equally. We use mean squared error (MSE) for the continuous average displacement error (ADE) metric and binary cross-entropy (BCE) for the binary metrics, including time-to-collision (TTC) under 11s, drivable-area compliance (DA), and no ego at-fault collision (NAFC): ℒ⁡(r^,r)=MSEADE+BCETTC+BCEDA+BCENAFC.L( r,r)=MSE_ADE+BCE_TTC+BCE_DA+BCE_NAFC. Quadruped. For both fθf_θ and gϕg_φ, the two output metrics are weighted equally. Given prediction r^=(v^,ω^) r=( v, ω) and target r=(v,ω)r=(v,ω), we use ℒ⁡(r^,r)=MSEv(v^,v)+MSEω(ω^,ω),L( r,r)=MSE_v( v,v)+MSE_ω( ω,ω), where v and ω are the per-step velocity- and yaw-rate-tracking errors (m/s and rad/s) computed over a 22-s rollout. MSE targets are z-scored using the training-split statistics. Table 7: MLP architecture and training hyperparameters (quadruped). The scenario evaluator fθf_θ is used by R-Only, RS-Mix, PPI-N, and the final evaluator in Scape. The correction model gϕg_φ additionally takes the surrogate metric vector s as input and predicts corrected target metrics r~ r. The two output dimensions correspond to the per-step velocity tracking error v and yaw-rate tracking error ω. Input dimensionality d=51d=51 (4444-D 11×411×4 ground-height crop ++ [vx,ωz][v_x, _z] command ++ 55-D scene one-hot) for the sim-to-real experiment, and d=82d=82 for the in-sim experiment. Component Scenario evaluator fθf_θ Correction model gϕg_φ Input x∈ℝdx ^d [x,s]∈ℝd+2[x,s] ^d+2 Output r^=fθ(x)∈ℝ2 r=f_θ(x) ^2 r~=gϕ(x,s)∈ℝ2 r=g_φ(x,s) ^2 Hidden layers [128,64,32][128,64,32] [128,64,32][128,64,32] Activation ReLU ReLU Dropout 0.30.3 0.30.3 Trainable parameters 16.816.8K 17.117.1K Loss 2 MSE 2 MSE Target normalization z-score (train split) z-score (train split) Optimizer Adam Adam Learning rate 3×10−43× 10^-4 3×10−43× 10^-4 Weight decay 1×10−31× 10^-3 1×10−31× 10^-3 Batch size 6464 6464 Epochs 500500 500500 A.6 Evaluation Metrics and Reported Quantities We summarize how the reported quantities are computed. For a fixed target metric, let =(xi,ri)i=1nT=\(x_i,r_i)\_i=1^n denote the test set, where xix_i is a scenario and rir_i is the target measurement. For Scape, let r^i r_i denote its prediction on xix_i; for a baseline B, let r^iB r_i^B denote the corresponding baseline prediction. For binary metrics, the prediction is the positive-class probability. Task performance metrics. For nuPlan, we evaluate each planner using four target metrics. Average displacement error (ADE) is the mean displacement between the planned ego trajectory and the logged expert trajectory. TTC <1<1s is a binary safety metric indicating whether the ego vehicle reaches a time-to-collision below one second. Drivable-area compliance and no ego at-fault collision are binary metrics indicating whether the ego trajectory stays within the drivable area and avoids at-fault collisions, respectively. For quadruped velocity tracking, each target measurement is a rollout-level tracking error computed over a 2-s rollout. MAEvelMAE_vel and MAEyawMAE_yaw denote the mean absolute errors between the commanded and measured forward velocity and yaw rate, respectively. Lower values indicate better tracking. Test loss. For nuPlan continuous targets, we report mean squared error: ℒ⁡(Scape)=1n∑i=1n(r^i−ri)2.L( Scape)= 1n _i=1^n ( r_i-r_i )^2. (1) For nuPlan binary targets, we report binary cross entropy: ℒ(Scape)=−1n∑i=1n[rilogr^i+(1−ri)log(1−r^i)].L( Scape)=- 1n _i=1^n [r_i r_i+(1-r_i) (1- r_i) ]. (2) For quadruped continuous velocity tracking targets, we report mean absolute prediction error on the rollout-level tracking-error targets: ℒ⁡(Scape)=1n∑i=1n|r^i−ri|.L( Scape)= 1n _i=1^n | r_i-r_i |. (3) The same metric-specific definition is used for each baseline B by replacing r^i r_i with r^iB r_i^B. All prediction-loss results, including the main paired-data sweep, OOD experiments, and surrogate-scaling experiments, are computed using the corresponding task-specific test loss. Relative loss reduction. Given a baseline B, the relative test-loss reduction of Scape is Δloss(Scape,B)=100%(ℒ⁡(B)−ℒ⁡(Scape)ℒ⁡(B)). _loss( Scape,B)=100\% ( L(B)-L( Scape)L(B) ). (4) When multiple metrics, planners, paired-data fractions, or random seeds are involved, we first compute the loss for each setting and then report the average relative reduction over the corresponding collection of settings. Paired-label efficiency. Let ρp∈(0,1] _p∈(0,1] denote the fraction of the full paired training set pD_p used for training. For a target loss level τ, let ρScape(τ) _ Scape(τ) be the smallest paired-data fraction at which Scape reaches loss no larger than τ, and define ρB(τ) _B(τ) analogously for baseline B. The paired-label saving of Scape relative to B is LabelEfficiency⁡(Scape,B,τ)=100%(1−ρScape(τ)ρB(τ)).LabelEfficiency( Scape,B;τ)=100\% (1- _ Scape(τ) _B(τ) ). (5) In the main results, τ is chosen as the strongest full-budget baseline performance, and we report how much paired data Scape saves while matching that target performance. Surrogate-data scaling capacity. Let ρs∈(0,1] _s∈(0,1] denote the fraction of the simulation-only set sD_s used for training. To study correction capacity, we fix ρp _p, vary ρs _s, retrain each scenario-conditioned method, and report the resulting test loss ℒ⁡(ρp,ρs)L( _p, _s). This measures how much surrogate-only data can be safely converted into useful supervision under a fixed paired-label budget. For each paired-data fraction, the best surrogate fraction is the ρs _s that gives the lowest test loss. nuPlan OOD evaluation. The nuPlan OOD experiment uses the same test-loss definition above. For OOD evaluation, paired target labels from one city are removed during training, while evaluation is performed on the original test set for direct comparison with the standard full-training setting. We compare training with and without OOD surrogate-only data by the relative loss reduction: ΔOOD-S(Scape)=100%(ℒno-OOD-S(Scape)−ℒwith-OOD-S(Scape)ℒno-OOD-S(Scape)). _OOD -S( Scape)=100\% ( L_no -OOD-S( Scape)-L_with -OOD-S( Scape)L_no -OOD-S( Scape) ). (6) Conformal calibration. For conformal uncertainty evaluation, each trained evaluator is calibrated on the held-out calibration set c=(xj,rj)j=1mD_c=\(x_j,r_j)\_j=1^m. For continuous metrics, we compute residual scores ej=|rj−r^j|e_j=|r_j- r_j|. Given miscoverage level α, the split-conformal threshold is q^1−α=Quantile⌈(m+1)(1−α)⌉/m(ejj=1m), q_1-α=Quantile_ (m+1)(1-α) /m (\e_j\_j=1^m ), (7) and the prediction interval is Cα(x)=[r^(x)−q^1−α,r^(x)+q^1−α].C_α(x)=[ r(x)- q_1-α, r(x)+ q_1-α]. (8) For binary metrics, we compute the corresponding classification conformity scores and return a conformal prediction set Cα(x)⊆0,1C_α(x) \0,1\. For nuPlan, the continuous-metric procedure is used for ADE, while the binary conformal procedure is used for TTC <1<1s, drivable-area compliance, and no ego at-fault collision. For quadruped, both MAEvelMAE_vel and MAEyawMAE_yaw are continuous metrics and use the same residual-based conformal interval. We verify empirical coverage on the test set: Cov^=1n∑i=1n[ri∈Cα(xi)]. Cov= 1n _i=1^n1 [r_i∈ C_α(x_i) ]. (9) For continuous metrics, the reported conformal width is the interval length 2q^1−α2 q_1-α. For binary metrics, the reported width is the average prediction-set size: W=1n∑i=1n|Cα(xi)|.W= 1n _i=1^n|C_α(x_i)|. (10) A.7 Additional nuPlan Scenario-Level Prediction Results (a) Vector (b) Simple Vector Figure 8: Additional scenario-level prediction results on nuPlan. Panels (a) and (b) report Vector and Simple Vector planner results, respectively. Test loss is reported as the paired-data fraction varies; curves show mean ± standard error over 10 random paired-data seeds. A.8 Full Conformal Prediction Results Table 8: Full split conformal width sweep on nuPlan Urban Driver. Columns are paired-data fractions. Cells are mean ± standard error over 10 paired-data seeds; bold marks the narrowest method per column. For ADE, width is interval length in meters; for binary targets, width is average prediction-set size. cov. reports empirical test coverage range across neural methods. ADE, interval width (m). cov. 0.946–0.953 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.26 44.92\!±\!0.26 ±0.23 43.23\!±\!0.23 ±0.34 42.10\!±\!0.34 ±0.22 41.69\!±\!0.22 ±0.35 41.00\!±\!0.35 ±0.26 40.64\!±\!0.26 ±0.26 40.63\!±\!0.26 ±0.26 40.35\!±\!0.26 ±0.26 39.93\!±\!0.26 ±0.26 40.07\!±\!0.26 PPI-N ±0.2645.21\!±\!0.26 ±0.2145.11\!±\!0.21 ±0.3244.91\!±\!0.32 ±0.1844.76\!±\!0.18 ±0.0944.60\!±\!0.09 ±0.1644.25\!±\!0.16 ±0.1844.55\!±\!0.18 ±0.1444.44\!±\!0.14 ±0.2144.17\!±\!0.21 ±0.1644.44\!±\!0.16 RS-Mix ±0.2149.09\!±\!0.21 ±0.1848.52\!±\!0.18 ±0.2547.90\!±\!0.25 ±0.3046.78\!±\!0.30 ±0.3047.08\!±\!0.30 ±0.3146.38\!±\!0.31 ±0.1245.41\!±\!0.12 ±0.2045.66\!±\!0.20 ±0.1845.45\!±\!0.18 ±0.1644.65\!±\!0.16 R-Only ±0.2246.11\!±\!0.22 ±0.3544.73\!±\!0.35 ±0.1843.82\!±\!0.18 ±0.3143.68\!±\!0.31 ±0.3042.80\!±\!0.30 ±0.2242.27\!±\!0.22 ±0.2242.00\!±\!0.22 ±0.2141.98\!±\!0.21 ±0.1640.99\!±\!0.16 ±0.3041.02\!±\!0.30 MC ±0.1150.21\!±\!0.11 ±0.0950.22\!±\!0.09 ±0.0750.26\!±\!0.07 ±0.0450.24\!±\!0.04 ±0.0350.25\!±\!0.03 ±0.0350.28\!±\!0.03 ±0.0250.28\!±\!0.02 ±0.0250.28\!±\!0.02 ±0.0150.28\!±\!0.01 ±0.0050.27\!±\!0.00 CV ±0.0450.37\!±\!0.04 ±0.0350.32\!±\!0.03 ±0.0250.31\!±\!0.02 ±0.0250.29\!±\!0.02 ±0.0250.29\!±\!0.02 ±0.0150.29\!±\!0.01 ±0.0150.29\!±\!0.01 ±0.0150.29\!±\!0.01 ±0.0050.29\!±\!0.00 ±0.0050.29\!±\!0.00 PPI ±0.0550.41\!±\!0.05 ±0.0350.35\!±\!0.03 ±0.0350.33\!±\!0.03 ±0.0350.32\!±\!0.03 ±0.0250.31\!±\!0.02 ±0.0150.31\!±\!0.01 ±0.0150.29\!±\!0.01 ±0.0150.30\!±\!0.01 ±0.0150.31\!±\!0.01 ±0.0050.31\!±\!0.00 TTC<1<1s, average set size. cov. 0.941–0.957 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.006 1.600\!±\!0.006 ±0.006 1.558\!±\!0.006 ±0.007 1.532\!±\!0.007 ±0.009 1.516\!±\!0.009 ±0.003 1.492\!±\!0.003 ±0.007 1.486\!±\!0.007 ±0.006 1.472\!±\!0.006 ±0.006 1.468\!±\!0.006 ±0.005 1.453\!±\!0.005 ±0.010 1.447\!±\!0.010 PPI-N ±0.0061.665\!±\!0.006 ±0.0051.643\!±\!0.005 ±0.0041.627\!±\!0.004 ±0.0041.623\!±\!0.004 ±0.0061.620\!±\!0.006 ±0.0041.618\!±\!0.004 ±0.0061.619\!±\!0.006 ±0.0051.614\!±\!0.005 ±0.0051.613\!±\!0.005 ±0.0041.614\!±\!0.004 RS-Mix ±0.0041.654\!±\!0.004 ±0.0041.632\!±\!0.004 ±0.0061.609\!±\!0.006 ±0.0081.591\!±\!0.008 ±0.0051.577\!±\!0.005 ±0.0041.559\!±\!0.004 ±0.0061.540\!±\!0.006 ±0.0061.526\!±\!0.006 ±0.0061.519\!±\!0.006 ±0.0061.505\!±\!0.006 R-Only ±0.0051.623\!±\!0.005 ±0.0071.586\!±\!0.007 ±0.0041.564\!±\!0.004 ±0.0061.555\!±\!0.006 ±0.0061.543\!±\!0.006 ±0.0041.540\!±\!0.004 ±0.0041.536\!±\!0.004 ±0.0051.528\!±\!0.005 ±0.0031.512\!±\!0.003 ±0.0031.497\!±\!0.003 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Drivable Area, average set size. cov. 0.929–0.941 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.0061.219\!±\!0.006 ±0.006 1.173\!±\!0.006 ±0.001 1.156\!±\!0.001 ±0.004 1.153\!±\!0.004 ±0.002 1.135\!±\!0.002 ±0.003 1.132\!±\!0.003 ±0.003 1.131\!±\!0.003 ±0.004 1.126\!±\!0.004 ±0.003 1.118\!±\!0.003 ±0.004 1.118\!±\!0.004 PPI-N ±0.0041.286\!±\!0.004 ±0.0041.263\!±\!0.004 ±0.0031.255\!±\!0.003 ±0.0031.246\!±\!0.003 ±0.0031.245\!±\!0.003 ±0.0031.240\!±\!0.003 ±0.0031.238\!±\!0.003 ±0.0041.235\!±\!0.004 ±0.0031.231\!±\!0.003 ±0.0041.233\!±\!0.004 RS-Mix ±0.004 1.217\!±\!0.004 ±0.0041.202\!±\!0.004 ±0.0011.199\!±\!0.001 ±0.0031.186\!±\!0.003 ±0.0041.176\!±\!0.004 ±0.0021.173\!±\!0.002 ±0.0031.174\!±\!0.003 ±0.0031.162\!±\!0.003 ±0.0041.153\!±\!0.004 ±0.0021.152\!±\!0.002 R-Only ±0.0041.273\!±\!0.004 ±0.0051.231\!±\!0.005 ±0.0021.207\!±\!0.002 ±0.0031.193\!±\!0.003 ±0.0041.183\!±\!0.004 ±0.0041.178\!±\!0.004 ±0.0041.166\!±\!0.004 ±0.0041.165\!±\!0.004 ±0.0021.156\!±\!0.002 ±0.0031.155\!±\!0.003 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 No At-Fault Collision, average set size. cov. 0.947–0.960 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.006 1.573\!±\!0.006 ±0.008 1.532\!±\!0.008 ±0.005 1.505\!±\!0.005 ±0.007 1.498\!±\!0.007 ±0.003 1.469\!±\!0.003 ±0.007 1.472\!±\!0.007 ±0.004 1.456\!±\!0.004 ±0.004 1.452\!±\!0.004 ±0.006 1.444\!±\!0.006 ±0.004 1.430\!±\!0.004 PPI-N ±0.0061.637\!±\!0.006 ±0.0051.616\!±\!0.005 ±0.0051.597\!±\!0.005 ±0.0031.590\!±\!0.003 ±0.0031.586\!±\!0.003 ±0.0051.577\!±\!0.005 ±0.0051.577\!±\!0.005 ±0.0041.570\!±\!0.004 ±0.0051.573\!±\!0.005 ±0.0041.573\!±\!0.004 RS-Mix ±0.0031.608\!±\!0.003 ±0.0041.590\!±\!0.004 ±0.0051.574\!±\!0.005 ±0.0051.557\!±\!0.005 ±0.0041.544\!±\!0.004 ±0.0061.531\!±\!0.006 ±0.0041.520\!±\!0.004 ±0.0051.510\!±\!0.005 ±0.0041.506\!±\!0.004 ±0.0031.495\!±\!0.003 R-Only ±0.0041.597\!±\!0.004 ±0.0061.556\!±\!0.006 ±0.0061.540\!±\!0.006 ±0.0061.529\!±\!0.006 ±0.0041.517\!±\!0.004 ±0.0031.520\!±\!0.003 ±0.0041.509\!±\!0.004 ±0.0071.498\!±\!0.007 ±0.0041.488\!±\!0.004 ±0.0091.474\!±\!0.009 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Table 9: Full split conformal width sweep on nuPlan Vector. Columns are paired-data fractions. Cells are mean ± standard error over 10 paired-data seeds; bold marks the narrowest method per column. For ADE, width is interval length in meters; for binary targets, width is average prediction-set size. cov. reports empirical test coverage range across neural methods. ADE, interval width (m). cov. 0.942–0.955 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.4748.78\!±\!0.47 ±0.44 46.67\!±\!0.44 ±0.40 45.88\!±\!0.40 ±0.25 44.82\!±\!0.25 ±0.48 45.13\!±\!0.48 ±0.35 43.64\!±\!0.35 ±0.43 43.75\!±\!0.43 ±0.41 43.77\!±\!0.41 ±0.29 43.27\!±\!0.29 ±0.29 43.43\!±\!0.29 PPI-N ±0.2951.20\!±\!0.29 ±0.2051.54\!±\!0.20 ±0.2250.72\!±\!0.22 ±0.2650.44\!±\!0.26 ±0.2150.29\!±\!0.21 ±0.3050.06\!±\!0.30 ±0.1849.96\!±\!0.18 ±0.2749.88\!±\!0.27 ±0.2450.33\!±\!0.24 ±0.1449.88\!±\!0.14 RS-Mix ±0.21 47.49\!±\!0.21 ±0.2446.91\!±\!0.24 ±0.2046.11\!±\!0.20 ±0.2246.29\!±\!0.22 ±0.1846.13\!±\!0.18 ±0.2445.59\!±\!0.24 ±0.2045.16\!±\!0.20 ±0.2745.22\!±\!0.27 ±0.2444.98\!±\!0.24 ±0.2744.98\!±\!0.27 R-Only ±0.3451.47\!±\!0.34 ±0.3950.26\!±\!0.39 ±0.4549.28\!±\!0.45 ±0.4548.60\!±\!0.45 ±0.3148.38\!±\!0.31 ±0.2847.70\!±\!0.28 ±0.3947.17\!±\!0.39 ±0.2747.20\!±\!0.27 ±0.2347.04\!±\!0.23 ±0.2346.82\!±\!0.23 MC ±0.0956.23\!±\!0.09 ±0.0956.25\!±\!0.09 ±0.0756.30\!±\!0.07 ±0.0556.32\!±\!0.05 ±0.0356.31\!±\!0.03 ±0.0356.32\!±\!0.03 ±0.0256.31\!±\!0.02 ±0.0156.30\!±\!0.01 ±0.0156.26\!±\!0.01 ±0.0056.26\!±\!0.00 CV ±0.0456.48\!±\!0.04 ±0.0356.47\!±\!0.03 ±0.0256.49\!±\!0.02 ±0.0256.47\!±\!0.02 ±0.0256.45\!±\!0.02 ±0.0156.45\!±\!0.01 ±0.0156.44\!±\!0.01 ±0.0156.43\!±\!0.01 ±0.0156.41\!±\!0.01 ±0.0056.41\!±\!0.00 PPI ±0.0456.54\!±\!0.04 ±0.0256.53\!±\!0.02 ±0.0256.55\!±\!0.02 ±0.0156.54\!±\!0.01 ±0.0156.53\!±\!0.01 ±0.0156.52\!±\!0.01 ±0.0156.52\!±\!0.01 ±0.0156.53\!±\!0.01 ±0.0156.53\!±\!0.01 ±0.0056.54\!±\!0.00 TTC<1<1s, average set size. cov. 0.939–0.954 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.006 1.705\!±\!0.006 ±0.007 1.664\!±\!0.007 ±0.004 1.655\!±\!0.004 ±0.006 1.628\!±\!0.006 ±0.005 1.623\!±\!0.005 ±0.004 1.619\!±\!0.004 ±0.006 1.612\!±\!0.006 ±0.005 1.608\!±\!0.005 ±0.006 1.586\!±\!0.006 ±0.006 1.591\!±\!0.006 PPI-N ±0.0041.743\!±\!0.004 ±0.0041.727\!±\!0.004 ±0.0021.714\!±\!0.002 ±0.0041.716\!±\!0.004 ±0.0031.711\!±\!0.003 ±0.0041.707\!±\!0.004 ±0.0021.704\!±\!0.002 ±0.0041.702\!±\!0.004 ±0.0031.699\!±\!0.003 ±0.0031.695\!±\!0.003 RS-Mix ±0.0021.770\!±\!0.002 ±0.0041.749\!±\!0.004 ±0.0021.729\!±\!0.002 ±0.0051.715\!±\!0.005 ±0.0041.695\!±\!0.004 ±0.0051.688\!±\!0.005 ±0.0051.686\!±\!0.005 ±0.0031.671\!±\!0.003 ±0.0041.659\!±\!0.004 ±0.0051.660\!±\!0.005 R-Only ±0.0031.736\!±\!0.003 ±0.0041.703\!±\!0.004 ±0.0051.699\!±\!0.005 ±0.0071.686\!±\!0.007 ±0.0051.673\!±\!0.005 ±0.0051.666\!±\!0.005 ±0.0031.663\!±\!0.003 ±0.0041.661\!±\!0.004 ±0.0041.644\!±\!0.004 ±0.0021.643\!±\!0.002 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Drivable Area, average set size. cov. 0.939–0.947 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.0091.267\!±\!0.009 ±0.003 1.227\!±\!0.003 ±0.005 1.213\!±\!0.005 ±0.004 1.199\!±\!0.004 ±0.005 1.191\!±\!0.005 ±0.003 1.187\!±\!0.003 ±0.004 1.175\!±\!0.004 ±0.005 1.179\!±\!0.005 ±0.003 1.173\!±\!0.003 ±0.004 1.170\!±\!0.004 PPI-N ±0.0071.327\!±\!0.007 ±0.0051.307\!±\!0.005 ±0.0061.291\!±\!0.006 ±0.0041.288\!±\!0.004 ±0.0051.285\!±\!0.005 ±0.0041.270\!±\!0.004 ±0.0051.269\!±\!0.005 ±0.0051.271\!±\!0.005 ±0.0051.274\!±\!0.005 ±0.0041.263\!±\!0.004 RS-Mix ±0.004 1.264\!±\!0.004 ±0.0021.259\!±\!0.002 ±0.0021.251\!±\!0.002 ±0.0031.247\!±\!0.003 ±0.0041.234\!±\!0.004 ±0.0021.229\!±\!0.002 ±0.0031.225\!±\!0.003 ±0.0041.226\!±\!0.004 ±0.0031.214\!±\!0.003 ±0.0041.208\!±\!0.004 R-Only ±0.0051.325\!±\!0.005 ±0.0051.293\!±\!0.005 ±0.0051.276\!±\!0.005 ±0.0061.260\!±\!0.006 ±0.0031.252\!±\!0.003 ±0.0051.250\!±\!0.005 ±0.0041.242\!±\!0.004 ±0.0031.236\!±\!0.003 ±0.0051.230\!±\!0.005 ±0.0041.220\!±\!0.004 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 No At-Fault Collision, average set size. cov. 0.939–0.953 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.004 1.693\!±\!0.004 ±0.007 1.649\!±\!0.007 ±0.005 1.637\!±\!0.005 ±0.007 1.619\!±\!0.007 ±0.007 1.609\!±\!0.007 ±0.004 1.601\!±\!0.004 ±0.004 1.600\!±\!0.004 ±0.005 1.586\!±\!0.005 ±0.004 1.570\!±\!0.004 ±0.005 1.571\!±\!0.005 PPI-N ±0.0041.724\!±\!0.004 ±0.0041.714\!±\!0.004 ±0.0031.708\!±\!0.003 ±0.0051.706\!±\!0.005 ±0.0041.695\!±\!0.004 ±0.0041.695\!±\!0.004 ±0.0021.701\!±\!0.002 ±0.0041.694\!±\!0.004 ±0.0041.690\!±\!0.004 ±0.0051.687\!±\!0.005 RS-Mix ±0.0031.768\!±\!0.003 ±0.0041.747\!±\!0.004 ±0.0041.726\!±\!0.004 ±0.0031.722\!±\!0.003 ±0.0071.701\!±\!0.007 ±0.0071.689\!±\!0.007 ±0.0051.669\!±\!0.005 ±0.0051.664\!±\!0.005 ±0.0041.642\!±\!0.004 ±0.0041.638\!±\!0.004 R-Only ±0.0051.720\!±\!0.005 ±0.0051.686\!±\!0.005 ±0.0061.686\!±\!0.006 ±0.0031.669\!±\!0.003 ±0.0041.662\!±\!0.004 ±0.0031.652\!±\!0.003 ±0.0031.649\!±\!0.003 ±0.0041.646\!±\!0.004 ±0.0041.637\!±\!0.004 ±0.0051.631\!±\!0.005 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Table 10: Full split conformal width sweep on nuPlan Simple Vector. Columns are paired-data fractions. Cells are mean ± standard error over 10 paired-data seeds; bold marks the narrowest method per column. For ADE, width is interval length in meters; for binary targets, width is average prediction-set size. cov. reports empirical test coverage range across neural methods. ADE, interval width (m). cov. 0.949–0.951 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.5167.63\!±\!0.51 ±0.33 66.47\!±\!0.33 ±0.2366.63\!±\!0.23 ±0.20 66.21\!±\!0.20 ±0.18 65.63\!±\!0.18 ±0.33 65.94\!±\!0.33 ±0.39 65.67\!±\!0.39 ±0.27 65.73\!±\!0.27 ±0.27 65.26\!±\!0.27 ±0.26 65.20\!±\!0.26 PPI-N ±0.2470.42\!±\!0.24 ±0.3568.72\!±\!0.35 ±0.3968.72\!±\!0.39 ±0.4567.32\!±\!0.45 ±0.3267.94\!±\!0.32 ±0.2268.18\!±\!0.22 ±0.3767.80\!±\!0.37 ±0.3067.99\!±\!0.30 ±0.3767.54\!±\!0.37 ±0.2367.23\!±\!0.23 RS-Mix ±0.24 66.75\!±\!0.24 ±0.2767.13\!±\!0.27 ±0.26 66.35\!±\!0.26 ±0.2866.59\!±\!0.28 ±0.3966.80\!±\!0.39 ±0.2466.17\!±\!0.24 ±0.4566.36\!±\!0.45 ±0.1866.38\!±\!0.18 ±0.3666.78\!±\!0.36 ±0.3465.82\!±\!0.34 R-Only ±0.4569.54\!±\!0.45 ±0.5568.61\!±\!0.55 ±0.3567.53\!±\!0.35 ±0.3667.64\!±\!0.36 ±0.3767.53\!±\!0.37 ±0.3466.82\!±\!0.34 ±0.2966.90\!±\!0.29 ±0.1666.57\!±\!0.16 ±0.1866.62\!±\!0.18 ±0.2165.90\!±\!0.21 MC ±0.16105.42\!±\!0.16 ±0.10105.26\!±\!0.10 ±0.08105.25\!±\!0.08 ±0.07105.24\!±\!0.07 ±0.06105.28\!±\!0.06 ±0.04105.27\!±\!0.04 ±0.03105.27\!±\!0.03 ±0.02105.22\!±\!0.02 ±0.01105.19\!±\!0.01 ±0.00105.18\!±\!0.00 CV ±0.09105.49\!±\!0.09 ±0.05105.36\!±\!0.05 ±0.04105.31\!±\!0.04 ±0.04105.29\!±\!0.04 ±0.03105.31\!±\!0.03 ±0.03105.31\!±\!0.03 ±0.02105.33\!±\!0.02 ±0.01105.29\!±\!0.01 ±0.01105.28\!±\!0.01 ±0.00105.28\!±\!0.00 PPI ±0.10105.50\!±\!0.10 ±0.06105.39\!±\!0.06 ±0.05105.33\!±\!0.05 ±0.05105.31\!±\!0.05 ±0.03105.32\!±\!0.03 ±0.03105.34\!±\!0.03 ±0.02105.37\!±\!0.02 ±0.01105.35\!±\!0.01 ±0.01105.35\!±\!0.01 ±0.00105.36\!±\!0.00 TTC<1<1s, average set size. cov. 0.942–0.953 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.005 1.721\!±\!0.005 ±0.005 1.697\!±\!0.005 ±0.008 1.682\!±\!0.008 ±0.006 1.688\!±\!0.006 ±0.004 1.675\!±\!0.004 ±0.005 1.662\!±\!0.005 ±0.003 1.659\!±\!0.003 ±0.003 1.663\!±\!0.003 ±0.004 1.651\!±\!0.004 ±0.003 1.651\!±\!0.003 PPI-N ±0.0041.745\!±\!0.004 ±0.0051.715\!±\!0.005 ±0.0031.707\!±\!0.003 ±0.0041.703\!±\!0.004 ±0.0041.706\!±\!0.004 ±0.0031.698\!±\!0.003 ±0.0031.686\!±\!0.003 ±0.0041.688\!±\!0.004 ±0.0031.692\!±\!0.003 ±0.0041.683\!±\!0.004 RS-Mix ±0.0021.723\!±\!0.002 ±0.0031.712\!±\!0.003 ±0.0031.707\!±\!0.003 ±0.0031.694\!±\!0.003 ±0.0021.688\!±\!0.002 ±0.0041.683\!±\!0.004 ±0.0051.670\!±\!0.005 ±0.0051.675\!±\!0.005 ±0.0021.674\!±\!0.002 ±0.0021.662\!±\!0.002 R-Only ±0.0071.738\!±\!0.007 ±0.0041.720\!±\!0.004 ±0.0071.718\!±\!0.007 ±0.0061.709\!±\!0.006 ±0.0041.701\!±\!0.004 ±0.0031.694\!±\!0.003 ±0.0041.688\!±\!0.004 ±0.0041.689\!±\!0.004 ±0.0031.674\!±\!0.003 ±0.0041.677\!±\!0.004 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Drivable Area, average set size. cov. 0.953–0.956 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.0041.350\!±\!0.004 ±0.004 1.328\!±\!0.004 ±0.003 1.320\!±\!0.003 ±0.004 1.306\!±\!0.004 ±0.004 1.306\!±\!0.004 ±0.004 1.295\!±\!0.004 ±0.003 1.291\!±\!0.003 ±0.004 1.284\!±\!0.004 ±0.003 1.284\!±\!0.003 ±0.005 1.281\!±\!0.005 PPI-N ±0.0041.390\!±\!0.004 ±0.0041.377\!±\!0.004 ±0.0041.369\!±\!0.004 ±0.0031.356\!±\!0.003 ±0.0041.354\!±\!0.004 ±0.0031.349\!±\!0.003 ±0.0031.354\!±\!0.003 ±0.0041.346\!±\!0.004 ±0.0041.351\!±\!0.004 ±0.0041.346\!±\!0.004 RS-Mix ±0.005 1.337\!±\!0.005 ±0.0021.333\!±\!0.002 ±0.0041.327\!±\!0.004 ±0.0041.317\!±\!0.004 ±0.0041.309\!±\!0.004 ±0.0041.298\!±\!0.004 ±0.0041.300\!±\!0.004 ±0.0041.296\!±\!0.004 ±0.0041.294\!±\!0.004 ±0.0021.288\!±\!0.002 R-Only ±0.0041.375\!±\!0.004 ±0.0041.355\!±\!0.004 ±0.0041.341\!±\!0.004 ±0.0051.331\!±\!0.005 ±0.0061.331\!±\!0.006 ±0.0041.315\!±\!0.004 ±0.0031.317\!±\!0.003 ±0.0031.308\!±\!0.003 ±0.0041.305\!±\!0.004 ±0.0041.304\!±\!0.004 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 No At-Fault Collision, average set size. cov. 0.939–0.951 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.0061.716\!±\!0.006 ±0.004 1.691\!±\!0.004 ±0.006 1.684\!±\!0.006 ±0.0071.676\!±\!0.007 ±0.0041.667\!±\!0.004 ±0.0061.665\!±\!0.006 ±0.003 1.645\!±\!0.003 ±0.005 1.640\!±\!0.005 ±0.003 1.638\!±\!0.003 ±0.003 1.633\!±\!0.003 PPI-N ±0.0041.732\!±\!0.004 ±0.0051.714\!±\!0.005 ±0.0051.704\!±\!0.005 ±0.0061.694\!±\!0.006 ±0.0021.687\!±\!0.002 ±0.0031.687\!±\!0.003 ±0.0041.676\!±\!0.004 ±0.0041.684\!±\!0.004 ±0.0041.678\!±\!0.004 ±0.0021.670\!±\!0.002 RS-Mix ±0.004 1.710\!±\!0.004 ±0.0031.697\!±\!0.003 ±0.0031.689\!±\!0.003 ±0.003 1.673\!±\!0.003 ±0.004 1.664\!±\!0.004 ±0.003 1.659\!±\!0.003 ±0.0041.647\!±\!0.004 ±0.0041.642\!±\!0.004 ±0.0031.647\!±\!0.003 ±0.0031.634\!±\!0.003 R-Only ±0.0061.737\!±\!0.006 ±0.0041.722\!±\!0.004 ±0.0061.715\!±\!0.006 ±0.0061.713\!±\!0.006 ±0.0051.704\!±\!0.005 ±0.0061.694\!±\!0.006 ±0.0041.692\!±\!0.004 ±0.0051.690\!±\!0.005 ±0.0051.672\!±\!0.005 ±0.0041.679\!±\!0.004 MC ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 CV ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 PPI ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 ±0.0002.000\!±\!0.000 Table 11: Full split conformal width sweep on Go2 Sim2Sim. Columns are paired-data fractions. Cells are mean ± standard error over 10 paired-data seeds; bold marks the narrowest method per column. Width is regression interval length reported as ×10−2× 10^-2. cov. reports empirical test coverage range across neural methods. Velocity, interval width ×10−2× 10^-2. cov. 0.945–0.955 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.34 11.37\!±\!0.34 ±0.48 11.05\!±\!0.48 ±0.41 11.05\!±\!0.41 ±0.48 11.04\!±\!0.48 ±0.59 11.08\!±\!0.59 ±0.46 10.98\!±\!0.46 ±0.50 10.97\!±\!0.50 ±0.58 11.02\!±\!0.58 ±0.55 10.75\!±\!0.55 ±0.48 10.84\!±\!0.48 PPI-N ±0.8312.42\!±\!0.83 ±1.2712.57\!±\!1.27 ±0.4411.16\!±\!0.44 ±1.0011.75\!±\!1.00 ±0.4511.26\!±\!0.45 ±0.5112.01\!±\!0.51 ±0.5711.39\!±\!0.57 ±0.3911.55\!±\!0.39 ±0.3411.14\!±\!0.34 ±0.3811.42\!±\!0.38 RS-Mix ±0.3612.39\!±\!0.36 ±0.4412.61\!±\!0.44 ±0.4012.39\!±\!0.40 ±0.3312.10\!±\!0.33 ±0.3912.34\!±\!0.39 ±0.3412.34\!±\!0.34 ±0.4512.29\!±\!0.45 ±0.2912.07\!±\!0.29 ±0.4312.16\!±\!0.43 ±0.2811.56\!±\!0.28 R-Only ±0.4212.05\!±\!0.42 ±0.4712.11\!±\!0.47 ±0.4711.44\!±\!0.47 ±0.4911.57\!±\!0.49 ±0.4711.53\!±\!0.47 ±0.4011.27\!±\!0.40 ±0.5211.06\!±\!0.52 ±0.4711.45\!±\!0.47 ±0.4810.86\!±\!0.48 ±0.5911.12\!±\!0.59 MC ±0.8413.18\!±\!0.84 ±0.9213.02\!±\!0.92 ±0.8812.51\!±\!0.88 ±0.8712.85\!±\!0.87 ±0.9012.84\!±\!0.90 ±0.8312.67\!±\!0.83 ±0.8612.84\!±\!0.86 ±0.8712.60\!±\!0.87 ±0.8812.64\!±\!0.88 ±0.8812.64\!±\!0.88 CV ±0.8512.82\!±\!0.85 ±0.8712.57\!±\!0.87 ±0.8712.46\!±\!0.87 ±0.8512.57\!±\!0.85 ±0.8512.51\!±\!0.85 ±0.8112.55\!±\!0.81 ±0.8712.57\!±\!0.87 ±0.8512.48\!±\!0.85 ±0.8712.51\!±\!0.87 ±0.8812.50\!±\!0.88 PPI ±0.8512.53\!±\!0.85 ±0.7212.18\!±\!0.72 ±0.8212.51\!±\!0.82 ±0.8112.27\!±\!0.81 ±0.7612.26\!±\!0.76 ±0.7912.40\!±\!0.79 ±0.8212.41\!±\!0.82 ±0.8212.39\!±\!0.82 ±0.8412.44\!±\!0.84 ±0.8412.41\!±\!0.84 Yaw Rate, interval width ×10−2× 10^-2. cov. 0.940–0.943 Method .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Scape ±0.5111.31\!±\!0.51 ±0.30 9.77\!±\!0.30 ±0.27 9.00\!±\!0.27 ±0.30 8.62\!±\!0.30 ±0.36 8.24\!±\!0.36 ±0.33 8.12\!±\!0.33 ±0.39 8.05\!±\!0.39 ±0.35 7.95\!±\!0.35 ±0.37 7.69\!±\!0.37 ±0.39 7.60\!±\!0.39 PPI-N ±0.3112.04\!±\!0.31 ±0.1312.19\!±\!0.13 ±0.4611.67\!±\!0.46 ±0.4010.47\!±\!0.40 ±0.3011.04\!±\!0.30 ±0.2410.58\!±\!0.24 ±0.3010.75\!±\!0.30 ±0.1610.44\!±\!0.16 ±0.2110.32\!±\!0.21 ±0.3110.27\!±\!0.31 RS-Mix ±0.28 10.60\!±\!0.28 ±0.2410.62\!±\!0.24 ±0.2110.48\!±\!0.21 ±0.2610.36\!±\!0.26 ±0.3110.07\!±\!0.31 ±0.249.76\!±\!0.24 ±0.199.77\!±\!0.19 ±0.279.63\!±\!0.27 ±0.289.80\!±\!0.28 ±0.229.12\!±\!0.22 R-Only ±0.3212.21\!±\!0.32 ±0.3612.42\!±\!0.36 ±0.2911.35\!±\!0.29 ±0.2610.18\!±\!0.26 ±0.289.29\!±\!0.28 ±0.348.89\!±\!0.34 ±0.238.52\!±\!0.23 ±0.388.19\!±\!0.38 ±0.347.99\!±\!0.34 ±0.317.75\!±\!0.31 MC ±0.3612.11\!±\!0.36 ±0.4512.04\!±\!0.45 ±0.3311.86\!±\!0.33 ±0.3312.04\!±\!0.33 ±0.2512.03\!±\!0.25 ±0.3012.13\!±\!0.30 ±0.3011.94\!±\!0.30 ±0.3011.88\!±\!0.30 ±0.3111.95\!±\!0.31 ±0.3111.95\!±\!0.31 CV ±0.3911.96\!±\!0.39 ±0.4011.85\!±\!0.40 ±0.3411.86\!±\!0.34 ±0.2711.80\!±\!0.27 ±0.2411.83\!±\!0.24 ±0.2912.03\!±\!0.29 ±0.2811.84\!±\!0.28 ±0.2811.82\!±\!0.28 ±0.3011.86\!±\!0.30 ±0.2911.88\!±\!0.29 PPI ±0.4812.14\!±\!0.48 ±0.3111.94\!±\!0.31 ±0.3411.86\!±\!0.34 ±0.2011.62\!±\!0.20 ±0.2411.78\!±\!0.24 ±0.3011.99\!±\!0.30 ±0.2711.72\!±\!0.27 ±0.2811.75\!±\!0.28 ±0.2711.76\!±\!0.27 ±0.2911.75\!±\!0.29 A.9 Additional OOD Generalization Results (a) Vector (b) Simple Vector Figure 9: Additional city-level OOD generalization results on nuPlan. Panels (a) and (b) report results for the Vector and Simple Vector planners, respectively. Results compare full training, OOD training with surrogate-only data from the removed city, and OOD training without removed-city surrogate data. Results are averaged over four leave-one-city-out settings. A.10 Full Surrogate Correction Results (a) Scape (b) PPI-N (c) RS-Mix Figure 10: Surrogate scaling and correction capacity on nuPlan Urban Driver. Paired-data and surrogate-only fractions are varied jointly for Scape, PPI-N, and RS-Mix. Results are averaged over 3×33× 3 paired-data and surrogate-data seeds. Square markers denote the best surrogate-only fraction for each paired-data fraction. A.11 Additional Evaluator-Guided Planner Selection Results (a) ρp=0.1 _p=0.1 (b) ρp=0.2 _p=0.2 (c) ρp=0.3 _p=0.3 (d) ρp=0.4 _p=0.4 (e) ρp=0.5 _p=0.5 (f) ρp=0.6 _p=0.6 (g) ρp=0.7 _p=0.7 (h) ρp=0.8 _p=0.8 (i) ρp=0.9 _p=0.9 Figure 11: Additional evaluator-guided planner selection results on nuPlan. Results are shown across paired-data fractions ρp∈0.1,…,0.9 _p∈\0.1,…,0.9\, where ρp _p denotes the fraction of the full paired training set pD_p used for training. Bars report mean ± standard error over 10 paired-data seeds.

Instant research discovery