Paper deep dive
World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning
Tong Nie, Yuewen Mei, Junlin He, Yihong Tang, Jian Sun, Wei Ma
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/18/2026, 12:13:02 PM
Summary
The paper proposes Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework for robust motion planning in autonomous driving. It formulates robust planning as a constrained min-max game where a predictive world model is converted into a role-conditioned adversary to learn sparse, scene-adaptive attack coalitions. The ego planner then optimizes a regret-aware robust best response against this frozen adversary, utilizing tail-risk weighting and trust regions to maintain nominal driving behavior. Experiments on nuPlan and InterPlan benchmarks demonstrate improved performance in long-tail scenarios.
Entities (8)
Relation Signals (6)
Adversarial World Modeling → evaluatedon → nuPlan
confidence 95% · Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method...
Adversarial World Modeling → evaluatedon → InterPlan
confidence 95% · Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method...
Adversarial World Modeling → formulatesas → Min-Max Game
confidence 95% · cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM)
Adversarial World Modeling → uses → Counterfactual Credit Assignment
confidence 92% · AWM introduces a principled decoupled solver... via counterfactual credit assignment.
World Model → convertedto → Adversarial World Modeling
confidence 90% · the planner's predictive world model is converted into a role-conditioned adversary
Adversarial World Modeling → employs → Trust Region
confidence 88% · utilizing tail-risk weighting and reference-anchored trust regions
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner's predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components.
Tags
Links
- Source: https://arxiv.org/abs/2607.10630v1
- Canonical: https://arxiv.org/abs/2607.10630v1
Trouble viewing inline? Open PDF directly →
Full Text
158,590 characters extracted from source content.
Expand or collapse full text
World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning Tong Nie,1,2,† Yuewen Mei,2,† Junlin He,1 Yihong Tang,3,4 Jian Sun,2,🖂 Wei Ma1,🖂 1The Hong Kong Polytechnic University, 2Tongji University, 3McGill University, 4Mila-Quebec AI Institute tong.nie@connect.polyu.hk wei.w.ma@polyu.edu.hk † Equal contribution. 🖂 Corresponding authors. Abstract Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner’s predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components. 1 Introduction Developing robust motion planners for closed-loop autonomous driving remains a central open problem [44, 23]. Recently, the leading paradigm has shifted from rule-based systems toward generative motion modeling [38, 33, 46, 53, 49]. Drawing inspiration from Large Language Models (LLMs), modern discrete architectures [46, 54, 43, 52, 48] treat map elements and motion primitives as discrete tokens, exhibiting remarkable performance when trained on large-scale human driving logs. However, these autoregressive planners are primarily optimized against nominal, naturalistic data distributions. Consequently, they remain brittle on interactive long-tail cases that are rare but safety-critical, such as aggressive cut-ins, coordinated blocking, and collision-inducing maneuvers from surrounding vehicles [12, 25]. To expose and rectify these vulnerabilities, adversarial training offers a compelling solution by actively searching hard scenarios to robustify the planner [15]. Despite the promise of adversarial training, existing frameworks are not well aligned to modern autoregressive architecture. Many existing adversarial frameworks rely on external adversary agents, heuristic scenario perturbations, or simulator-intensive rollouts [51, 29, 26, 40, 30]. These attacks are often designed around low-dimensional control policies, fixed objects of interest, or handcrafted rules. As a result, they are infeasible to scale to the high-dimensional tokenized generation paradigm. Rather than forcing incompatible external attackers into this paradigm, we argue that an endogenous solution lies in the environment simulator itself. We observe that behavioral World Models [46, 52], which inherently learn multi-agent traffic evolution as sequential token predictions, are naturally suited to bridge this gap. Operating on the same autoregressive architecture, motion vocabulary, and scene representation as the ego planner, they can be repurposed to parameterize both naturalistic and adversarial traffic within a unified motion manifold. Consequently, formulating the environment as a learnable generative process enables the integration of adversarial interactions directly into the closed-loop training rollout, bypassing the need for a separate handcrafted attacker. However, translating this intuition into a trainable framework poses three significant optimization challenges. First, coordinating synergistic adversaries in a multi-agent environment can introduce a combinatorial credit assignment ambiguity issue [1, 11]. Isolating the fine-grained marginal contribution of individual agents within a world model is difficult in high-dimensional action spaces. Standard strategies such as reward-sharing [32, 31] struggle to resolve who should initiate the attack, how the attack emerges, and whether multiple agents should coordinate. Second, simultaneous min-max optimization of two large autoregressive models is notoriously unstable [28, 8]. Updating a planning policy against a continuously shifting adversary model often leads to gradient divergence due to the non-stationary adversarial distributions. Third, unconstrained adversarial fine-tuning can degrade nominal driving performance. Without explicit behavioral regularization, a planner may become overly conservative on hard cases while sacrificing driving skills in normal traffic [13]. To bridge these gaps, we propose Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework that formulates adversarially robust planner training as a constrained min-max game over a shared tokenized rollout structure. To instantiate a tractable solver, we approximate the ideal game through two principled stages. In the inner minimization, we introduce a scene-adaptive coalition learning mechanism. Rather than relying on external heuristics, we convert the benign world model into an adversary via role-conditioning. Crucially, we introduce a counterfactual credit assignment approach via role switching within the same model, which efficiently resolves the multi-agent reward ambiguity and identifies sparse, synergistic adversarial coalitions. In the outer maximization, the ego planner is optimized as a constrained robust best response to the AWM’s calibrated risk distribution. We formulate a regret-aware objective that prioritizes informative tail vulnerabilities, coupled with reference-anchored trust-region regularization to preserve nominal driving performance. Our main contributions are summarized as follows: • We present the first self-play adversarial fine-tuning paradigm that casts robust autoregressive motion planning as a constrained min-max game. With a shared rollout, it bridges principled adversarial world modeling and constrained planner adaptation without relying on external simulators. • We design a tractable decoupled solver to instantiate the game. Leveraging role-conditioned counterfactual credit assignment, the AWM first discovers a sparse, scene-adaptive adversarial risk distribution. Utilizing this calibrated risk, the ego planner then optimizes a regret-aware objective to mitigate adversarial tail-risk while preserving nominal driving via trust-region constraints. • We provide provable theoretical justifications and comprehensive empirical validation. Theoretical analysis bounds the local approximation error of the decoupled solver. Evaluations on nuPlan and InterPlan benchmarks demonstrate that AWM discovers transferable multi-agent attacks and yields a robust planner excelling in both naturalistic and highly interactive long-tail scenarios. 2 Preliminary and Problem Formulation 2.1 Autoregressive Motion Generation Tokenized Autoregressive Rollout. Recent generative driving models formulate multi-agent traffic evolution as sequential next-token prediction over a learned discrete motion vocabulary [38, 33, 46, 52]. Consider a scene s with ego agent e, non-ego agents s=1,…,NsV_s=\1,…,N_s\, and context csc_s (e.g., map, history). At autoregressive step t∈1,…,Ht∈\1,…,H\, each agent i∈e∪si∈\e\ _s emits a discrete motion token zi,t∈z_i,t , and the joint multi-agent token set is t=zi,ti∈e∪s z_t=\z_i,t\_i∈\e\ _s. The generated prefix <t z_<t updates the scene rollout through a deterministic transition xt+1=Γ(xt,t)x_t+1= (x_t, z_t), so the partially generated state can be written as xt=Φ(cs,<t)x_t= (c_s, z_<t). Thus, the Markovian generative process is: p(1:H∣cs)=∏t=1Hp(t∣xt)p( z_1:H c_s)= _t=1^Hp( z_t x_t). This formulation yields an interactive rollout where each generated token immediately alters the spatiotemporal state observed by all agents in future steps. Planning-Prediction Decomposition. Directly modeling the full joint token distribution over the entire horizon is computationally intractable and entangles ego decision-making with environmental dynamics. Following decoupled planning-prediction paradigms [43, 49], we factorize the step-wise joint probability into individual marginals and instantiate them with two parameterized policies: p(1:H∣cs)=∏t=1H[p(ze,t∣xt)∏i∈sp(zi,t∣xt)] p( z_1:H c_s)= _t=1^H [p(z_e,t x_t) _i _sp(z_i,t x_t) ] ≈∏t=1H[πplan(ze,t∣xt)⏟ego planner∏i∈sπpred(zi,t∣xt)⏟non-ego world model]. ≈ _t=1^H [ _plan(z_e,t x_t)_ego planner _i _s _pred(z_i,t x_t)_non-ego world model ]. (1) πpred _pred is learned from driving logs and represents naturalistic traffic. Eq. 1 is the interface used throughout the paper: the ego factor represents the object optimized by the planner, while the non-ego factors represent the environment dynamics. However, optimizing solely against πpred _pred often leaves πplan _plan vulnerable to rare but critical out-of-distribution events, requiring robust adversarial modeling. 2.2 Problem Formulation: Robust Planning as a Constrained Game We evaluate a rollout by a scalarized utility that trades off progress and safety. Let rtr_t denote the progress reward and ct(m)c_t^(m) denote the m-th safety cost term (e.g., collision, off-road) at step t. The step-wise utility and scene-level cumulative utility are ut(xt,t)=λRrt−∑m=1Mλmct(m)u_t(x_t, z_t)= _Rr_t- _m=1^M _mc_t^(m) and Us(πplan,πpred)=τ∼(πplan,πpred)[∑t=1Hγt−1ut(xt,t)]U_s( _plan, _pred)=E_τ ( _plan, _pred)[ _t=1^Hγ^t-1u_t(x_t, z_t)], respectively. The standard planning objective is to maximize UsU_s under the benign prior πpred _pred. Robust planning requires the planner to perform well when some non-ego agents behave adversarially, while still preserving normal driving ability. We therefore replace the benign environment with a controllable Adversarial World Model (AWM) πawm _awm and formulate the ideal robust objective as a constrained two-player zero-sum game: maxπplanminπawm,s _ _plan _ _awm,\,\A_s\ s∼[Us(πplan,πawm,s)] _s [U_s( _plan, _awm,A_s) ] (2) s.t. |s|≤Kmax,s⊆s,∀s∈(Attack Budget) |A_s|≤ K_ ,A_s _s, ∀ s (Attack Budget) x∼[trust(πplan(⋅∣x)∥πplannom(⋅∣x))]≤ϵ(Trust Region Constraint), _x [D_trust ( _plan(· x) _plan^nom(· x) ) ]≤ε (Trust Region Constraint), where the inner player minimizes UsU_s by optimizing the generative policy πawm _awm, and the outer player maximizes its expected utility against the risk distribution induced by πawm _awm. We further introduce two constraints: (1) a budgeted subset of active adversaries sA_s to ensure sparse and reasonable attack; (2) a trust region around a nominal reference planner πplannom _plan^nom to prevent behavioral degradation. However, Eq. 2 is not directly tractable. First, the inner step involves a combinatorial search over the high-dimensional multi-agent action space, suffering from severe credit assignment ambiguities [1]. Second, simultaneous non-convex min-max updates are notoriously unstable [28, 8], as the planner struggles to converge against a non-stationary distribution. Third, directly enforcing behavioral constraints under adversarial rollouts can yield an averaged or overly conservative policy with degraded normal performance. To overcome these issues, we introduce a principled solver in Section 3 as a decoupled approximation. 3 Self-Play Fine-tuning with Adversarial World Models Figure 1: Decoupled self-play fine-tuning with AWM. The full pipeline is described in Algorithm 1. We instantiate the idealized game in Eq. 2 as a decoupled self-play fine-tuning framework over the shared autoregressive factorization in Eq. 1. The key design is to separate the two roles of the game without changing the rollout interface. In Stage A, the planner is frozen and the non-ego factors are replaced by the AWM that searches for sparse adversarial coalitions as the opponent distribution (Section 3.2). In Stage B, the learned AWM is frozen and the ego factor is optimized as a regret-aware constrained best response to the induced risk (Section 3.3). Thus, adversarial training data are generated by interactive joint rollouts within the same token space and transition dynamics, rather than by external heuristic perturbations or a separate simulator. Appendix B gives the implementation details and Appendix C provides the theoretical analysis. Figure 1 summarizes the two-stage pipeline. 3.1 Decoupled Self-Play Approximation of the Constrained Game Stage A: Adversarial Coalition Learning. We first anchor the planner to a frozen reference πplannom _plan^nom and approximate the inner minimization over the non-ego factors ∏πpred(⋅)Π _pred(·). Instead of enumerating all subsets sA_s, AWM learns a role-conditioned policy and a scene-adaptive coalition distribution: πawm∗,adv∗=argminπawm,ss∼[Us(πplannom,πawm,s)]s.t.|s|≤Kmax. _awm^*,P_adv^*= _ _awm,\,\A_s\E_s [U_s( _plan^nom, _awm,A_s) ] .t. |A_s|≤ K_ . (3) This stage yields a frozen adversarial policy πawm∗ _awm^* with a calibrated risk distribution adv∗P_adv^* over sparse multi-agent attacks. Proposition C.2 bounds the local error incurred by freezing the adversary, and Corollary C.3 gives the resulting approximate local robust-value guarantee for the planner update. Stage B: Constrained Robust Best Response. With (πawm∗,adv∗)( _awm^*,P_adv^*) frozen, the planner solves the outer maximization within a reference trust region. Rather than optimizing raw adversarial utility, we compare the current planner with a frozen reference planner under the same induced attack. This yields the reference-relative regret over adv∗P_adv^*. The planner’s objective is given by: πplan∗=argmaxπplan _plan^*= _ _plan −s∼[ℜτAs∼adv∗(⋅∣s)(Δsadv(πplan,s))] -E_s [ R_τ^A_s _adv^*(· s) ( _s^adv( _plan,A_s) ) ] (4) s.t. .t. ref(πplan,πref)≤ϵ,safe(πplan;πawm∗,adv∗)≤κ, _ref ( _plan, _ref )≤ε, _safe ( _plan; _awm^*,P_adv^* )≤κ, where the reference πref _ref is initialized from πplannom _plan^nom, and Δsadv(πplan,As)=Us(πref,πawm∗,As)−Us(πplan,πawm∗,As) _s^adv( _plan,A_s)=U_s( _ref, _awm^*,A_s)-U_s( _plan, _awm^*,A_s). Here, ℜτ R_τ denotes a generic tail-risk functional over the regret distribution. The abstract constraint refD_ref enforces reference-anchored retention, while safeV_safe represents admissible safety-violation constraints. Importantly, Lemma C.1 justifies this local update by showing finite-horizon utility is smooth when the planner remains within a token-level trust region. 3.2 Adversarial World Modeling via Multi-Agent Coalition Learning This section instantiates the inner minimization (Eq. 3). Directly optimizing an unstructured set of attackers is computationally prohibitive and suffers from credit assignment issues. Moreover, effective attacks are sparse and scene-dependent: a single agent may be sufficient in some scenes, whereas in others the worst failure requires coordinated behavior. AWM converts the inner search into a sequence: role conditioning defines how an agent attacks, coalition learning decides who should attack, counterfactual credit trains the attackers, and calibration decides whether a paired attacker should be admitted. This amortizes the worst coalition while remaining on the same motion manifold. Role-Conditioned Adversarial Generation. To keep adversarial rollouts within the same autoregressive structure, AWM changes the behavioral distribution of selected non-ego agents without changing the token vocabulary or transition model. Each non-ego agent receives a latent role assignment ρi∈bg,atk _i∈\ bg, atk\, where ρi=atk _i= atk iff i∈si _s. At each generation step t, for each non-ego agent i∈si _s the actions are generated by the shared AWM: zi,t∼πawm(⋅∣xt,ρi),∀i∈Vs.z_i,t _awm(· x_t, _i), ∀ i∈ V_s. (5) The ego token ze,tz_e,t is still sampled from πplan _plan. The role conditioning ρi _i is explicitly injected into both internal agent and decoder representations. Let hih_i be the contextual feature of agent i and ki,tk_i,t be its decoding state. It modifies output behavioral logits ℓi,t _i,t in the generation process via: h~i=hi+Eagent(ρi),ℓi,t=D(ki,t+Edec(ρi)), h_i=h_i+E_agent( _i), _i,t=D(k_i,t+E_dec( _i)), (6) where EagentE_agent and EdecE_dec are learnable embeddings. This makes role switching a behavioral intervention rather than a loss reweighting trick. It also enables counterfactual evaluations within the same AWM, which is later used for credit assignment. Architectural details are provided in Appendix B.1. Sparse Coalition Learning. Given the conditioned generator, the next problem is to determine the sparse coalition AsA_s. AWM avoids exhaustive subset search through a structured sequence: proposal probing, primary selection, conditional pairing, and pair admission. A background probe first rolls out all agents with ρi=bg _i= bg and scores each agent by threat, realism, and residual coverage (the exact probe signals and coverage metrics are given in Appendix B.2.1). The resulting sparse proposal set PsP_s removes benign agents while retaining complementary attack potentials. A learned primary selector then chooses is∗=argmaxi∈Psfpri(xi)i_s^*= _i∈ P_sf_pri(x_i), where xix_i combines probe features and proposal evidence. Conditional on is∗i_s^* acting as atk, a pair ranker scores relational features ϕ(is∗,j)φ(i_s^*,j) and selects j^s=argmaxj∈Ps∖is∗fpair(ϕ(is∗,j)) j_s= _j∈ P_s \i_s^*\f_pair(φ(i_s^*,j)). This converts the combinatorial subset search into a tractable sequential approximation. Corollary C.4 gives an interpretation of the approximation error: the coverage gap decreases when this pipeline captures more of the adversarial support that matters under the full worst-case adversary. Details for fprif_pri and fpairf_pair are in Appendix B.2.2. Counterfactual Credit Assignment. The coalition selected above is trained from scene-level utility, but the optimization signal in Eq. 3 needs to be attributed to individual attackers and to their joint behavior. Directly isolating and evaluating member-level contribution to reducing the ego utility is challenging, as the credit space scales with the joint action tree. To achieve fine-grained attribution without enumerating this exploding space, we leverage the role-conditioned nature of the AWM to perform counterfactual self-play credit assignment. Let sfactA_s^fact be the factual active coalition in a training rollout, and write Us()U_s(A) for the planner utility under A. For each active attacker i∈sfacti _s^fact, we compute a leave-one-out marginal contribution against counterfactual scenarios: Δiloo=Us(sfact∖i)−Us(sfact). _i^loo=U_s(A_s^fact \i\)-U_s(A_s^fact). (7) A partner might be weak in isolation but critical when synergizing with the primary attacker. To preserve non-additive cooperation, we also define the counterfactual team contribution: Δsteam=Us(bg(sfact))−Us(sfact), _s^team=U_s(bg(A_s^fact))-U_s(A_s^fact), (8) where bg(Asfact)bg(A_s^fact) denotes switches the roles of all attackers to ρ=bgρ= bg. Then the attacker’s reward is Δihyb=ωlooΔiloo+ωteamΔsteam. _i^hyb= _loo _i^loo+ _team _s^team. (9) Proposition C.6 shows that this hybrid counterfactual reward is gradient-aligned with the adversarial coalition objective, while Lemma C.7 interprets the role-switched terms as action-independent baselines for variance reduction. The full AWM training objective is given in Appendix B.2.3. Scene-Adaptive Calibration. The final output of Stage-A decides not only who the best pair is, but also whether the pair is worth using in the current scene. To finalize the opponent distribution, we introduce a decisive scene-dependent calibrator. Given the primary attacker is∗i_s^* and candidate partner j^s j_s, we construct a scene-level representation ψs=SceneEnc(is∗,j^s) _s=SceneEnc(i_s^*, j_s) and define the pair gain Gspair=Us(is∗)−Us(is∗,j^s)G_s^pair=U_s(\i_s^*\)-U_s(\i_s^*, j_s\). A binary calibrator qη(ψs)=σ(fηcal(ψs))q_η( _s)=σ(f_η^cal( _s)) is trained with cross-entropy on (Gspair>0)I(G_s^pair>0) to predict the probability psp_s of a positive marginal improvement. At inference, the pair is admitted only when psp_s exceeds a threshold. Proposition C.5 shows that this thresholding rule is the optimal binary admission policy under calibrated pair-gain probabilities. Thus, the coalition size is scene-adaptive, preventing attack quality degradation or redundancy. 3.3 Regret-Aware Constrained Robust Planner Optimization With the adversary (πawm∗ _awm^* and qηq_η) frozen, Stage B instantiates the constrained best-response problem in Eq. 4. Exposing the planner to generated attacks via standard adversarial training often leads to performance degradation due to two issues: (1) it forces the planner to adopt overly conservative behaviors while sacrificing nominal driving; and (2) it is risk-insensitive and optimizes an average adversarial loss, diluting the gradients of rare but critical failures. To overcome these issues, we first convert the learned AWM into a conditional risk distribution. Then the planner optimizes adversarial tail-risk using a regret-aware robust objective over the induced distribution. To enforce the behavioral regularization in a tractable way, we optimize a reference-anchored dual surrogate: a trajectory-level constraint and a normal-preserving penalty. Safety constraints are handled by dual variables. AWM-Induced Adversarial Risk Distribution. For each scene, the frozen AWM defines three rollout branches. The normal branch uses the benign world model πpred _pred. The top-1 branch assigns ρis∗=atk _i_s^*= atk and keeps all other non-ego agents as background. The pair branch assigns ρi=atk _i= atk for i∈is∗,j^si∈\i_s^*, j_s\. Let αscal=fηcal(ψs) _s^cal=f_η^cal( _s) be the calibrator logit and ps=σ(αscal/τrisk)p_s=σ( _s^cal/ _risk) be the temperature-scaled probability of admitting the pair branch. AWM induces the discrete branch distribution: P(Bs=top1)=1−ps,P(Bs=pair)=ps.P(B_s= top1)=1-p_s, P(B_s= pair)=p_s. (10) This converts the coalition output from Stage A into the calibrated opponent distribution adv∗P_adv^* used by Stage B. Instead of an unstructured attack buffer, the planner replays each scene and optimizes against a scene-conditional self-play distribution. Branch construction details are in Appendix B.3.1. Reference-Anchored Trust Region Regularization. Robust adaptation should not degrade normal driving. We instantiate the reference constraint in Eq. 4 on the normal branch, where the environment remains πpred _pred. Let Usnorm(πθ)U_s^norm( _θ) denote the normal scene-level utility. We use two complementary retention surrogates. First, to preserve nominal driving performance, we define the normal utility margin msnorm(θ)=Usnorm(πθ)−Usnorm(πref)m_s^norm(θ)=U_s^norm( _θ)-U_s^norm( _ref) and impose an asymmetric degradation penalty: ψsnorm(θ)=[−ϵnorm−msnorm(θ)]+2 _s^norm(θ)= [- _norm-m_s^norm(θ) ]_+^2. Second, to prevent large behavioral drift even when the scalar utility remains acceptable, we also regularize the token distribution along the normal rollout as: KLnorm(s;θ)=1H∑t=1HDKL(πθ(⋅∣xs,tnorm)∥πref(⋅∣xs,tnorm)).D_KL^norm(s;θ)= 1H _t=1^HD_KL ( _θ(· x_s,t^norm)\; \|\; _ref(· x_s,t^norm) ). The normal-branch objective is: ℒnom(θ)=s∼[λpgℓsnorm(θ)+λklKLnorm(s;θ)+λmarginψsnorm(θ)].L_nom(θ)=E_s [ _pg _s^norm(θ)+ _klD_KL^norm(s;θ)+ _margin _s^norm(θ) ]. (11) Here ℓsnorm _s^norm is the normal-branch policy-gradient loss. These terms provide a tractable surrogate for the trust-region constraint. Proposition C.12 proves that they imply a nominal-performance bound, ensuring a conservative update to obtain robustness. Implementation details are in Appendix B.3.1. Adversarial Tail-Risk Optimization. Absolute adversarial utility can be dominated by intrinsic scene difficulty rather than the planner’s vulnerabilities. We therefore propose to optimize reference-relative regret under each adversarial branch to realize the robust objective in Eq. 4: Δsb(θ)=Usb(πref)−Usb(πθ),for b∈top1,pair. _s^b(θ)=U_s^b( _ref)-U_s^b( _θ), b∈\top1,pair\. (12) A positive regret indicates that πθ _θ underperforms the baseline under the specific attack b. Coupling this with the AWM-induced distribution, we construct a structured discrete regret distribution: sθ=(Δstop1(θ),1−ps),(Δspair(θ),ps).D_s^θ= \ ( _s^top1(θ),1-p_s ), ( _s^pair(θ),p_s ) \. (13) Proposition C.9 shows that regret preserves the branchwise gradient direction of utility maximization while removing intrinsic difficulty variation. Standard expected loss would dilute the gradients of these rare but critical attacks by their low probabilities. Instead, we apply a tail-focused risk operator ℜτ R_τ at tail level τ∈(0,1]τ∈(0,1] and instantiate ℜτ R_τ by analytically solving the Conditional Value-at-Risk (CVaR) allocation [35] over the induced two-point regret distribution sθD_s^θ. Since the distribution has only two support points, Proposition C.11 gives closed-form weights wstop1(τ)w_s^top1(τ) and wspair(τ)w_s^pair(τ). These weights amplify low-probability but high-regret pair attacks when they occupy the adversarial tail. Let ℓstop1(θ) _s top1(θ) and ℓspair(θ) _s pair(θ) be the policy gradient losses, the adversarial tail-risk objective is: ℒrob(θ)=s∼[wstop1(τ)ℓstop1(θ)+wspair(τ)ℓspair(θ)].L_rob(θ)=E_s [w_s^top1(τ) _s^top1(θ)+w_s^pair(τ) _s^pair(θ) ]. (14) Implementations are detailed in Appendix B.3.2. Regret-CVaR dynamically shifts the optimization weight to whichever branch currently exposes the planner’s worst-case vulnerability, regardless of its prior probability. This ensures the updates focus precisely on the long-tail of the regret severity. Safety-constrained dual update. Finally, to prevent the planner from exploiting the objective by violating traffic rules (e.g., evading an attacker by driving off-road), we enforce explicit safety constraints in Eq. 4. Let vs,mb∈[0,1]v_s,m^b∈[0,1] be the m-th safety violation indicator under branch b. The expected violation rate is v¯s,m=(1−ps)vs,mtop1+psvs,mpair v_s,m=(1-p_s)v_s,m top1+p_sv_s,m pair. Given admissible thresholds κm _m, we update Lagrangian dual variables ζ to dynamically adjust the cost multipliers λm(ζ) _m(ζ): ℒdual(ζ)=∑m=1Mλm(ζ)(s∼[v¯s,m]−κm).L_dual(ζ)= _m=1^M _m(ζ) (E_s [ v_s,m]- _m ). Training alternates between minimizing ℒplanner(θ)=ℒnom(θ)+ℒrob(θ)L_planner(θ)=L_nom(θ)+L_rob(θ) over planner parameters and maximizing ℒdualL_dual over ζ. Proposition C.13 shows that the softmax-parameterized dual behaves as a smooth maximum-violation penalty, concentrating weight on the most violated safety channel while keeping the reward scale bounded. The multiplier parameterization is provided in Appendix B.3.3. 4 Experiments We organize the empirical study around three questions. First, does adversarial self-play improve the final closed-loop planner on standard and interaction-heavy benchmarks? Second, does the learned AWM expose transferable planner vulnerabilities when it is used as a simulator-side traffic model? Third, which components of the two-stage optimization account for the observed gains? Experimental protocols, metric definitions, and implementation details are provided in Appendix D. 4.1 Closed-Loop Planning and Adversarial Testing Closed-Loop Planning Performance. We first evaluate whether the robust best-response update improves ordinary closed-loop planning performance. Table 1 reports benchmark results on nuPlan [2], InterPlan [17], and InterPlan-LongTail (Appendix D.3). Relative to the Plan-R1 reference, AWM-Planner (ours) improves five of the six nuPlan settings, with the largest gains on Test14-hard NR and R (+2.60+2.60 and +1.28+1.28). The Test14-random NR score is essentially unchanged (−0.01-0.01), while Test14-random R increases by +0.67+0.67, indicating that the hard-case improvement is not obtained by broadly sacrificing normal driving behavior. The interaction benchmarks provide a more targeted assessment of the intended mechanism. AWM-Planner improves Plan-R1 by +2.10+2.10 on full InterPlan, +4.97+4.97 on InterPlan10, and +5.81+5.81 on InterPlan-LongTail. The per-template breakdown in Table 5 shows improvements on seven of eight designed templates, with the strongest gains in close straight-driving and medium-density yielding cases. These results suggest that AWM primarily improves recoverable interaction failures exposed by structured adversarial traffic. Table 1: Closed-loop planning results on nuPlan and InterPlan benchmarks. Higher is better. NR/R: non-reactive/reactive mode. Among learning-based planners, the best result is in bold and the second best result is underlined. is the reference baseline (Plan-R1), indicates an improvement over the reference, while indicates no improvement or degradation. Type Planner Val14 Test14-hard Test14-random InterPlan NR ↑ R ↑ NR ↑ R ↑ NR ↑ R ↑ Full ↑ InterPlan10 ↑ InterPlanLT ↑ Expert Log-Replay 93.53 80.32 85.96 68.80 94.03 75.86 14.76 – – Rule IDM / IDM+Mobil 75.60 77.33 56.15 62.26 70.39 72.42 47.07 31.28 38.49 PDM-Closed* [9] 92.84 92.12 65.08 75.19 90.05 91.64 69.64 41.81 45.93 PDM-Hybrid* [9] 92.77 92.11 65.99 76.07 90.10 91.28 41.61 26.12 27.79 Learning UrbanDriver [36] 68.57 64.11 50.40 49.95 51.83 67.15 5.56 3.62 7.75 PDM-Open [9] 53.53 54.24 33.51 35.83 52.81 57.23 26.22 24.67 17.51 GameFormer [20] 13.32 8.69 7.08 6.69 11.36 9.31 11.07 9.12 8.13 PlanTF [7] 84.27 76.95 69.70 61.61 85.62 79.58 47.72 33.00 32.38 PLUTO [6] 88.89 78.11 70.03 59.74 89.90 78.62 57.74 42.87 38.46 Diff. PlannerDPM_DPM [53] 89.87 82.80 75.99 69.22 89.19 82.93 50.07 24.75 29.96 Diff. PlannerDDIM_DDIM [53] 89.81 82.94 76.01 68.18 89.14 82.63 49.86 24.11 28.98 Flow Planner [42] 90.43 83.31 76.47 70.42 89.88 82.93 61.82 35.42 31.06 PlannerRFT [24] 89.96 84.46 77.16 72.21 90.76 85.80 – – – Plan-R1 [43] 88.98 87.69 77.45 77.20 91.23 90.04 56.64 40.95 34.89 Ours AWM-Planner 89.72 (+0.74) 88.44 (+0.75) 80.05 (+2.60) 78.48 (+1.28) 91.22 (-0.01) 90.71 (+0.67) 58.74 (+2.10) 45.92 (+4.97) 40.70 (+5.81) AWM as a Transferable Adversarial Simulator. We next test whether the learned adversary captures planner-agnostic interaction stress. Table 2 deploys AWM as the non-ego sim-agent traffic model for seven planners. AWM reduces the sim-agent closed-loop score for every planner, decreasing the mean score from 70.2470.24 to 67.4667.46. Although Plan-R1 has the largest drop, rule-based, optimization-based, diffusion-based, and transformer-based planners are all affected, indicating that the adversarial behavior generalizes beyond the planner used in Stage A. The same evaluation also separates adversarial strength from planner robustness after fine-tuning. AWM-Planner starts from a higher score under the normal host than Plan-R1 (77.6777.67 vs. 75.8675.86) and suffers a smaller AWM-induced degradation (−3.14-3.14 vs. −4.15-4.15). The submetric pattern is consistent with controlled interaction stress: the largest changes occur in progress and TTC, whereas drivable-area, speed-limit, and driving-direction scores remain nearly unchanged. The simulator diagnostics in Table 6 further show that AWM utilizes nearly the full two-agent budget while remaining constrained by feasibility gates, indicating that the score drop is not a result of an unconstrained simulator collapse. Table 2: Cross-planner Test14-hard sim-agent evaluation. Normal is the learned non-adversarial sim-agent baseline, while a held-out AWM uses the adversarial host as the simulator-side non-ego policy. Thus, the R-Score is an AWM/Normal sim-agent closed-loop score, not an ordinary planner closed-loop score. ΔR _R is computed as RAWM−RNormalR_ AWM-R_ Normal, where negative values indicate the R-Score drop induced by the adversarial host. Prog., Coll., TTC, Driv., Speed, and Dir. denote progress, collision, time-to-collision, drivable-area, speed-limit, and driving-direction submetrics. x mark the corresponding Normal reference R-Score, and x mark the AWM-induced R-Score change. Planner World R-Score ↑ ΔR _R Prog. ↑ Coll. ↑ TTC ↑ Driv. ↑ Comfort ↑ Speed ↑ Dir. ↑ Diffusion Planner [53] Normal 70.8170.81 – 88.0688.06 84.7184.71 73.3373.33 95.4195.41 86.2086.20 96.5096.50 98.9698.96 AWM 68.1068.10 −2.71-2.71 84.2584.25 83.6483.64 69.4969.49 94.8594.85 84.5684.56 96.4896.48 98.9098.90 IDM+Mobil Normal 55.6455.64 – 67.5567.55 80.5180.51 65.4465.44 86.4086.40 88.9788.97 96.9396.93 98.5398.53 AWM 53.2153.21 −2.43-2.43 62.9662.96 80.1580.15 62.1362.13 86.0386.03 88.2488.24 96.9796.97 98.5398.53 PDM-Closed [9] Normal 65.4565.45 – 73.9773.97 87.6887.68 70.9670.96 94.8594.85 83.0983.09 99.5499.54 99.0899.08 AWM 64.2064.20 −1.25-1.25 70.3770.37 88.9788.97 69.4969.49 94.8594.85 83.4683.46 99.5499.54 99.0899.08 PLUTO [6] Normal 76.6376.63 – 81.6481.64 92.5092.50 85.3985.39 96.5796.57 89.0289.02 97.0497.04 96.4496.44 AWM 73.6073.60 −3.03-3.03 75.2875.28 92.1092.10 81.9981.99 96.6996.69 87.5087.50 96.9996.99 96.6996.69 PlanTF [7] Normal 69.6269.62 – 83.5283.52 86.2186.21 78.6878.68 94.4994.49 89.3489.34 97.0397.03 96.6996.69 AWM 66.8966.89 −2.73-2.73 79.2079.20 86.9586.95 74.6374.63 94.1294.12 83.0983.09 97.0697.06 96.6996.69 Plan-R1 [43] Normal 75.8675.86 – 87.9387.93 89.1589.15 79.4179.41 95.2295.22 99.2699.26 99.0799.07 96.6996.69 AWM 71.7171.71 −4.15-4.15 82.8382.83 88.2488.24 73.1673.16 94.8594.85 98.1698.16 99.0899.08 97.0697.06 AWM-Planner (Ours) Normal 77.67 – 83.6083.60 93.3893.38 80.5180.51 96.6996.69 95.2295.22 99.4799.47 98.9098.90 AWM 74.53 −3.14-3.14 79.0779.07 91.5491.54 77.9477.94 96.6996.69 95.9695.96 99.4799.47 98.9098.90 4.2 Algorithmic Analysis and Discussion Planner-World Cross-Validation. Closed-loop scores couple the planner policy with the simulator dynamics. To isolate the effect of planner adaptation, we evaluate pre- and post-training planners under matched normal and adversarial world branches. Figure 2(a) summarizes this matrix, and Table 7 provides the full values. The post-trained planner improves nominal reward and progress under the normal world, while the larger gains occur under the AWM world: adversarial reward increases by 0.01190.0119, and Tail-CVaR increases from 0.86600.8660 to 0.90950.9095. This asymmetry matches the objective of Stage B, which should lift the adversarial lower tail while preserving nominal rollout quality. Table 8 further decomposes the AWM host into Top-1, Pair, and Adaptive branches. Pair and Adaptive branches produce larger lower-tail stress than the single-agent Top-1 branch, and the adaptive branch remains less aggressive than forced Pair because the calibrator admits two-agent coalitions only when they are predicted to be useful. Thus, the planner gains in Table 1 indicate a targeted policy adaptation that specifically improves the response to structured adversarial interactions. (a) Planner-world cross-validation (b) Attack and realism trade-off Figure 2: (a) Cross-validation shows that robust fine-tuning improves the adversarial tail while preserving nominal rollout quality (full values in Table 7). (b) The attack and realism plot summarizes the AWM ablations in Tables 9, 10, 11, 12 and the comparison in Table 15. AWM achieves a balanced operating point, whereas aggressive search increases attack strength at a clear realism cost. Mechanistic Analysis of AWM. The AWM is useful for planner training only if its adversarial rollouts are both informative and plausible. The AWM-side ablations in Tables 9–12 support this requirement. Role-conditioned counterfactual credit produces substantially stronger lower-tail degradation than global reward broadcast or assignment-only variants, while avoiding the large ADE/FDE collapse observed in coarse credit baselines. This indicates that role switching provides a more reliable training signal for coordinated adversarial behavior than assigning the same scene-level reward to all agents. The coalition ablations show a complementary effect. Forced K=2K=2 attacks can increase raw stress, but they move the generator toward a less realistic mode. Instead, the adaptive calibrated AWM preserves most of the controlled attack effect while keeping pair usage sparse and well calibrated. Figure 2(b) gives a compact view of this attack and realism trade-off (full results in Table 15). Matched best-4 search and the KING-style [18] aggressive bound can produce larger raw disruption, but they also incur larger realism costs and background-collision proxies. In contrast, single-candidate CAT∗ [51] and STRIVE∗ [34] remain closer to ordinary AWM generation but produce weaker tail stress. AWM therefore provides the intended middle regime: sufficient adversarial stress for robust training without becoming a purely destructive test-time attacker. Table 3: Training paradigm comparison. Open-loop rewards are on the planner’s rollout utility scale; CL scores are ordinary nuPlan closed-loop scores. Relative GPU-hours are normalized to the decoupled two-stage pipeline, and plateau updates denote thousands of planner update steps until validation Tail-CVaR stops improving. Stable seeds count runs that finish without divergence or failure, and Hard std. is the seed-wise standard deviation of Test14-hard NR score. Training paradigm Open-loop rollout Planner CL Computation and stability Nom. R ↑ Adv. R ↑ Tail ↑ Hard NR ↑ Random R ↑ Rel. GPU-h ↓ Plateau k ↓ Stable seeds ↑ Hard std. ↓ Planner-only refinement 0.98040.9804 0.96850.9685 0.84670.8467 77.45 90.04 0.620.62 5.05.0 3/33/3 0.550.55 Simultaneous alternating 0.98080.9808 0.97260.9726 0.86630.8663 78.3078.30 89.3089.30 2.352.35 18.518.5 2/32/3 1.651.65 Alternating warmup + fixed AWM 0.98120.9812 0.97440.9744 0.88100.8810 79.3079.30 90.2090.20 1.451.45 12.012.0 3/33/3 0.850.85 Decoupled Stage-A/B (ours) 0.98180.9818 0.97560.9756 0.89190.8919 80.05 90.71 1.001.00 9.09.0 3/33/3 0.400.40 Planner Adaptation Ablations. We finally examine whether the planner’s gain depends on the proposed decoupled optimization and regret-aware objective. Table 3 compares planner-only refinement, simultaneous alternating self-play, alternating warmup followed by fixed refinement, and the proposed decoupled pipeline. Our framework achieves the best adversarial Tail-CVaR, the highest closed-loop scores, and the lowest variability. Simultaneous alternating updates improve over planner-only refinement, but they require substantially more computation and exhibit weaker stability. This supports the design of first learning an opponent and then optimizing a constrained robust best response against the frozen distribution. Table 4 isolates the objective of Stage-B. Expected-risk and no-CVaR variants preserve competitive averages, but they lose adversarial Tail-CVaR and Test14-hard NR score, which is consistent with the need to emphasize rare severe risks. Removing normal retention strengthens adversarial reward but degrades nominal progress and score. Additional opponent sensitivity and hyperparameter results in Tables 13 and 14, with Figure 5, show that the planner is stable under moderate host changes while following the expected robustness and nominal performance trade-off. Table 4: Planner objective ablation with a robustness and nominal performance trade-off. Offline rewards are the planner’s rollout utility; CL scores and metrics are ordinary nuPlan Test14 simulation. Variant Open-loop rollout CL: Test14-hard NR CL: Test14-random R Nom. R Nom. prog. Adv. R Tail Score Prog. TTC Score Coll. TTC Full 0.9818 0.95450.9545 0.97560.9756 0.89190.8919 80.05 90.16 81.99 90.71 98.08 95.40 Expected only 0.98150.9815 0.9579 0.96810.9681 0.86400.8640 76.8076.80 86.2086.20 78.3678.36 90.6090.60 97.8897.88 95.2095.20 w/o CVaR 0.98160.9816 0.95300.9530 0.96700.9670 0.85010.8501 77.4077.40 87.0087.00 78.5078.50 90.5590.55 97.9097.90 95.1595.15 w/o normal retention 0.97900.9790 0.93900.9390 0.9762 0.8950 80.20 91.00 81.2081.20 88.6088.60 96.4096.40 92.0592.05 4.3 Case Studies Figure 3: Test14-hard qualitative cases. Orange denotes the original Plan-R1 planner and green denotes the AWM-trained planner. Each panel overlays the full closed-loop trajectory in the same adversarial scene; red markers identify the original planner’s collision or TTC failure point. Figure 4: K=2 cooperative attack in sim-agent evaluation. Ego is shown in blue, while A1/A2 are the two adversarial agents selected by AWM. The examples in Figures 3 and 4 illustrate the two mechanisms quantified above. In the recovery cases, AWM-Planner reacts earlier than the original Plan-R1 and avoids collision or TTC failures in the same adversarial scenes. Additional recovery cases, InterPlan-LongTail examples, and K=2K=2 cooperative attacks are shown in Figures 7-9. The simulator-side example in Figure 4 shows how the learned adversary creates stress without destroying the scene. Under the normal host, the ego can proceed through the local interaction. Under the K=2K=2 attack, two selected non-ego agents jointly constrain the ego corridor and induce a collision or TTC failure. This behavior matches the results in Table 2: AWM mainly reduces progress and TTC, while rule-compliance metrics remain nearly unchanged. 5 Conclusion This paper introduced AWM, a multi-agent self-play fine-tuning framework leveraging predictive world models as structured adversaries for robust motion planning. To overcome the intractability of high-dimensional adversarial training, we formulated the problem as a constrained min-max game and proposed a theoretically grounded decoupled solver. By integrating scene-adaptive coalition learning with counterfactual credit assignment, AWM efficiently exposes sparse, synergistic interaction failures within the planner’s inherent autoregressive interface. Subsequently, a regret-aware robust best response, with reference-anchored trust region constraints, fine-tunes the planner against the induced tail risk. Theoretical analysis and empirical evaluations demonstrate that our framework significantly improves planner robustness in rare, complex interactions while maintaining nominal performance. Cross-planner sim-agent evaluation and ablations further support the transferability of the learned adversarial traffic and the necessity of fine-grained credit assignment, learnable adversarial coalition, and tail-focused learning. However, AWM remains limited by the fidelity of the tokenized world model and lacks exhaustive scenario coverage or formal closed-loop safety guarantees. References [1] L. Busoniu, R. Babuska, and B. De Schutter (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38 (2), p. 156–172. Cited by: §1, §2.2. [2] H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari (2021) Nuplan: a closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810. Cited by: §4.1. [3] Y. Cao, B. Ivanovic, C. Xiao, and M. Pavone (2024) Reinforcement learning with human feedback for realistic traffic simulation. In 2024 IEEE international conference on robotics and automation (ICRA), p. 14428–14434. Cited by: Appendix A. [4] W. Chang, F. Pittaluga, M. Tomizuka, W. Zhan, and M. Chandraker (2024) Safe-sim: safety-critical closed-loop traffic simulation with diffusion-controllable adversaries. In European conference on computer vision, p. 242–258. Cited by: Appendix A. [5] K. Chen, W. Sun, H. Cheng, and S. Zheng (2025) RIFT: group-relative rl fine-tuning for realistic and controllable traffic simulation. arXiv preprint arXiv:2505.03344. Cited by: Appendix A. [6] J. Cheng, Y. Chen, and Q. Chen (2024) Pluto: pushing the limit of imitation learning-based planning for autonomous driving. arXiv preprint arXiv:2404.14327. Cited by: Table 6, Table 1, Table 2. [7] J. Cheng, Y. Chen, X. Mei, B. Yang, B. Li, and M. Liu (2024) Rethinking imitation-based planners for autonomous driving. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 14123–14130. Cited by: Table 6, Table 1, Table 2. [8] C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng (2017) Training gans with optimism. arXiv preprint arXiv:1711.00141. Cited by: §1, §2.2. [9] D. Dauner, M. Hallgarten, A. Geiger, and K. Chitta (2023) Parting with misconceptions about learning-based vehicle motion planning. In Conference on Robot Learning, p. 1268–1281. Cited by: Table 6, Table 1, Table 1, Table 1, Table 2. [10] R. Feng, N. Xi, D. Chu, R. Wang, Z. Deng, A. Wang, L. Lu, J. Wang, and Y. Huang (2025) Artemis: autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving. IEEE Robotics and Automation Letters 11 (1), p. 226–233. Cited by: Appendix A. [11] S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu (2023) Dense reinforcement learning for safety validation of autonomous vehicles. Nature 615 (7953), p. 620–627. Cited by: §1. [12] S. Feng, X. Yan, H. Sun, Y. Feng, and H. X. Liu (2021) Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nature communications 12 (1), p. 748. Cited by: §1. [13] S. Feng, H. Zhu, H. Sun, X. Yan, L. He, J. Yang, G. Su, B. Li, S. Li, L. Wang, et al. (2026) Breaking through safety performance stagnation in autonomous vehicles with dense learning. Nature Communications. Cited by: §1. [14] J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson (2018) Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §C.4. [15] Y. Gao, M. Piccinini, Y. Zhang, D. Wang, K. Moller, R. Brusnicki, B. Zarrouki, A. Gambi, J. F. Totz, K. Storms, et al. (2026) Foundation models in autonomous driving: a survey on scenario generation and scenario analysis. IEEE Open Journal of Intelligent Transportation Systems. Cited by: §1. [16] E. Greensmith, P. L. Bartlett, and J. Baxter (2004) Variance reduction techniques for gradient estimates in reinforcement learning. Journal of Machine Learning Research 5 (Nov), p. 1471–1530. Cited by: §C.4. [17] M. Hallgarten, J. Zapata, M. Stoll, K. Renz, and A. Zell (2024) Can vehicle motion planning generalize to realistic long-tail scenarios?. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 5388–5395. Cited by: §4.1. [18] N. Hanselmann, K. Renz, K. Chitta, A. Bhattacharyya, and A. Geiger (2022) King: generating safety-critical driving scenarios for robust imitation via kinematics gradients. In European Conference on Computer Vision, p. 335–352. Cited by: Table 15, §4.2. [19] Y. Hu, S. Chai, Z. Yang, J. Qian, K. Li, W. Shao, H. Zhang, W. Xu, and Q. Liu (2024) Solving motion planning tasks with a scalable generative model. In European Conference on Computer Vision, p. 386–404. Cited by: Appendix A. [20] Z. Huang, H. Liu, and C. Lv (2023) Gameformer: game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 3903–3913. Cited by: Table 1. [21] Z. Huang, X. Weng, M. Igl, Y. Chen, Y. Cao, B. Ivanovic, M. Pavone, and C. Lv (2025) Gen-drive: enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 3445–3451. Cited by: Appendix A. [22] S. Kakade and J. Langford (2002) Approximately optimal approximate reinforcement learning. In Proceedings of the nineteenth international conference on machine learning, p. 267–274. Cited by: §C.7. [23] P. Karkus, M. Igl, Y. Chen, K. Chitta, J. Packer, B. Douillard, R. Tian, A. Naumann, G. Garcia-Cobo, S. Tan, et al. (2025) Beyond behavior cloning in autonomous driving: a survey of closed-loop training techniques. Authorea Preprints. Cited by: Appendix A, §1. [24] H. Li, T. Li, J. Yang, H. Tian, C. Wang, L. Shi, M. Shang, Z. Lin, G. Wu, Z. Hao, et al. (2026) PlannerRFT: reinforcing diffusion planners through closed-loop and sample-efficient fine-tuning. arXiv preprint arXiv:2601.12901. Cited by: Appendix A, Table 1. [25] H. X. Liu and S. Feng (2024) Curse of rarity for autonomous vehicles. nature communications 15 (1), p. 4808. Cited by: Appendix A, §1. [26] Y. Liu, Z. Peng, X. Cui, and B. Zhou (2025) Adv-bmt: bidirectional motion transformer for safety-critical traffic scenario generation. arXiv preprint arXiv:2506.09485. Cited by: Appendix A, §1. [27] Y. Mei, T. Nie, J. Sun, and Y. Tian (2025) Llm-attacker: enhancing closed-loop adversarial scenario generation for autonomous driving with large language models. IEEE Transactions on Intelligent Transportation Systems. Cited by: Appendix A. [28] L. Mescheder, S. Nowozin, and A. Geiger (2017) The numerics of gans. Advances in neural information processing systems 30. Cited by: §1, §2.2. [29] T. Nie, Y. Mei, Y. Tang, J. He, J. Sun, H. Shi, W. Ma, and J. Sun (2025) Steerable adversarial scenario generation through test-time preference alignment. arXiv preprint arXiv:2509.20102. Cited by: Appendix A, §1. [30] T. Nie, Y. Tang, J. He, Y. Mei, J. Sun, L. Sun, W. Ma, and J. Sun (2026) ADV-0: closed-loop min-max adversarial training for long-tail robustness in autonomous driving. arXiv preprint arXiv:2603.15221. Cited by: Appendix A, §1. [31] M. Pei, S. Shi, and S. Shen (2025) Advancing multi-agent traffic simulation via r1-style reinforcement fine-tuning. arXiv preprint arXiv:2509.23993. Cited by: §1. [32] Z. Peng, W. Luo, Y. Lu, T. Shen, C. Gulino, A. Seff, and J. Fu (2024) Improving agent behaviors with rl fine-tuning for autonomous driving. In European Conference on Computer Vision, p. 165–181. Cited by: Appendix A, §1. [33] J. Philion, X. B. Peng, and S. Fidler (2023) Trajeglish: traffic modeling as next-token prediction. arXiv preprint arXiv:2312.04535. Cited by: Appendix A, §1, §2.1. [34] D. Rempe, J. Philion, L. J. Guibas, S. Fidler, and O. Litany (2022) Generating useful accident-prone driving scenarios via a learned traffic prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 17305–17315. Cited by: Appendix A, Table 15, Table 15, Table 15, §4.2. [35] R. T. Rockafellar, S. Uryasev, et al. (2000) Optimization of conditional value-at-risk. Journal of risk 2, p. 21–42. Cited by: §B.3.2, §C.6, §3.3. [36] O. Scheel, L. Bergamini, M. Wolczyk, B. Osiński, and P. Ondruska (2022) Urban driver: learning to drive from real-world demonstrations using policy gradients. In Conference on Robot Learning, p. 718–728. Cited by: Table 1. [37] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz (2015) Trust region policy optimization. In International conference on machine learning, p. 1889–1897. Cited by: §C.7. [38] A. Seff, B. Cera, D. Chen, M. Ng, A. Zhou, N. Nayakanti, K. S. Refaat, R. Al-Rfou, and B. Sapp (2023) Motionlm: multi-agent motion forecasting as language modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 8579–8590. Cited by: Appendix A, §B.1.1, §1, §2.1. [39] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. (2024) Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §B.3.1. [40] B. Stoler, I. Navarro, J. Francis, and J. Oh (2025) Seal: towards safe autonomous driving via skill-enabled adversary learning for closed-loop scenario generation. IEEE Robotics and Automation Letters 10 (9), p. 9320–9327. Cited by: Appendix A, §1. [41] Q. Sun, S. Zhang, D. Ma, J. Shi, D. Li, S. Luo, Y. Wang, N. Xu, G. Cao, and H. Zhao (2023) Large trajectory models are scalable motion predictors and planners. arXiv preprint arXiv:2310.19620. Cited by: Appendix A. [42] T. Tan, Y. Zheng, R. Liang, Z. Wang, K. Zheng, J. Zheng, J. Li, X. Zhan, and J. Liu (2025) Flow matching-based autonomous driving planning with advanced interactive behavior modeling. arXiv preprint arXiv:2510.11083. Cited by: Table 1. [43] X. Tang, M. Kan, S. Shan, and X. Chen (2025) Plan-r1: safe and feasible trajectory planning as language modeling. arXiv preprint arXiv:2505.17659. Cited by: Appendix A, Appendix A, §B.1.2, §B.1, Table 6, §1, §2.1, Table 1, Table 2. [44] S. Teng, X. Hu, P. Deng, B. Li, Y. Li, Y. Ai, D. Yang, L. Li, Z. Xuanyuan, F. Zhu, et al. (2023) Motion planning for autonomous driving: the state of the art and future perspectives. IEEE Transactions on Intelligent Vehicles 8 (6), p. 3692–3711. Cited by: §1. [45] J. Wang, A. Pun, J. Tu, S. Manivasagam, A. Sadat, S. Casas, M. Ren, and R. Urtasun (2021) Advsim: generating safety-critical scenarios for self-driving vehicles. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9909–9918. Cited by: Appendix A. [46] W. Wu, X. Feng, Z. Gao, and Y. Kan (2024) Smart: scalable multi-agent real-time motion generation via next-token prediction. Advances in Neural Information Processing Systems 37, p. 114048–114071. Cited by: Appendix A, §B.1.1, §B.1.2, §1, §1, §2.1. [47] C. Xu, A. Petiushko, D. Zhao, and B. Li (2025) Diffscene: diffusion-based safety-critical scenario generation for autonomous vehicles. In Proceedings of the AAAI conference on artificial intelligence, Vol. 39, p. 8797–8805. Cited by: Appendix A. [48] B. Ye, B. Zhang, and H. Zhao (2025) DAP: a discrete-token autoregressive planner for autonomous driving. arXiv preprint arXiv:2511.13306. Cited by: Appendix A, Appendix A, §1. [49] D. Zhang, J. Liang, K. Guo, S. Lu, Q. Wang, R. Xiong, Z. Miao, and Y. Wang (2025) Carplanner: consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 17239–17248. Cited by: Appendix A, Appendix A, §1, §2.1. [50] K. Zhang, Z. Tang, X. Hu, X. Pan, X. Guo, Y. Liu, J. Huang, L. Yuan, Q. Zhang, X. Long, et al. (2025) Epona: autoregressive diffusion world model for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 27220–27230. Cited by: Appendix A. [51] L. Zhang, Z. Peng, Q. Li, and B. Zhou (2023) Cat: closed-loop adversarial training for safe end-to-end driving. In Conference on Robot Learning, p. 2357–2372. Cited by: Appendix A, Table 15, Table 15, Table 15, §1, §4.2. [52] Z. Zhang, P. Karkus, M. Igl, W. Ding, Y. Chen, B. Ivanovic, and M. Pavone (2025) Closed-loop supervised fine-tuning of tokenized traffic models. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 5422–5432. Cited by: §1, §1, §2.1. [53] Y. Zheng, R. Liang, K. Zheng, J. Zheng, L. Mao, J. Li, W. Gu, R. Ai, S. E. Li, X. Zhan, et al. (2025) Diffusion-based planning for autonomous driving with flexible guidance. arXiv preprint arXiv:2501.15564. Cited by: Appendix A, Table 6, §1, Table 1, Table 1, Table 2. [54] Z. Zhou, H. Hu, X. Chen, J. Wang, N. Guan, K. Wu, Y. Li, Y. Huang, and C. J. Xue (2024) Behaviorgpt: smart agent simulation for autonomous driving with next-patch prediction. Advances in Neural Information Processing Systems 37, p. 79597–79617. Cited by: Appendix A, §B.1.2, §1. Appendix Appendix Contents Appendix A Related Work Autoregressive Motion Generation. Autoregressive modeling has recently emerged as a dominant paradigm for both multi-agent scenario simulation and ego-motion planning in autonomous driving. Early sequence modeling methods primarily operate in continuous spaces and leverage Transformer-based architectures [10, 41] or diffusion models [50, 53] to refine continuous waypoints iteratively. However, inspired by the success of Large Language Models (LLMs), discrete next-token prediction modeling has gained significant traction due to its scalability and temporal causality. Agent simulation models such as Trajeglish [33], SMART [46], and BehaviorGPT [54] tokenize maps and trajectories to progressively decode joint multi-agent futures, achieving realistic and interactive rollouts. Building upon this motion-as-language concept [38], recent efforts have extended this discrete representation to closed-loop ego planning [19, 48, 43], casting ego decision-making as motion token generation. Most relevant to our work are architectures that explicitly decouple predictive scene evolution (the world model) from ego planning (the policy), such as the planning-prediction dual-branch design in [43, 48] and the policy-transition decomposition in [49]. This predictive branch allows the planner to efficiently interact with reactive non-ego responses during rollouts without relying on a simulator. However, prior methods treat the world model as a frozen or passive behavioral prior that merely replicates logged data, lacking a structured mechanism to generate customized non-ego behaviors within the same tokenized framework. Reinforcement Learning Fine-Tuning (RLFT). While large-scale behavior cloning establishes a strong traffic prior, it invariably suffers from covariate shift in closed-loop evaluation and fails to enforce safety-critical driving principles [23]. To bridge this gap, RLFT is increasingly adopted for aligning driving policies with downstream objectives. In the continuous domain, frameworks such as [21, 24] employ learned reward models and policy-guided diffusion to fine-tune generative planners for sample efficiency and alignment. Within discrete architectures, Plan-R1 [43] applies GRPO-style optimization with rule-based rewards while maintaining a frozen reactive world model for surrounding agents. Similarly, Ye et al. [48] incorporates offline SAC-BC to mitigate imitation ambiguity, and Zhang et al. [49] suggests that autoregressive structures naturally facilitate large-scale RLFT. Furthermore, RLFT has also been applied to simulation agents for better collision realism and behavior fidelity [32, 3, 5]. Despite these advances, existing RLFT pipelines primarily optimize the ego policy against the expected distribution of a naturalistic traffic prior. Due to the rarity of long-tail events, the resulting planners can learn average behaviors and remain vulnerable to rare but safety-critical risks. Adversarial Training and Testing. Evaluating and ensuring the robustness of autonomous driving needs moving beyond naturalistic traffic scenarios to adversarial edge cases [25]. Early adversarial generation methods [45, 34, 4, 27, 47] often rely on heuristic perturbations, optimization-based attacks on continuous trajectories, or training specific adversarial agents to induce collisions for stress testing. To further improve policy robustness, closed-loop adversarial training frameworks [51, 29, 26, 40, 30] alternate between safety-critical scenario generation and ego policy updates. Recent literature emphasizes that effective adversaries must be structured and reactive rather than purely collision-seeking perturbations to ensure the generated scenarios remain realistic and useful for training. Several techniques such as skill learning [40], preference alignment [29], reverse-time token reconstruction [26] are shown to be effective for preserving realism or controllability. Despite these advances, existing methods are largely treated as external environment augmentations, heuristic resampling against a static planner, computationally heavy post-hoc optimization or test-time biasing. More importantly, existing closed-loop adversarial training frameworks usually assume model-free RL and thus rely on CPU-intensive policy rollouts on physical simulators, significantly limiting their scalability and efficiency [30]. Therefore, they often operate in a small set of scenarios (e.g., 500), making them less effective for large-scale motion planners. Appendix B Extended Architectural and Algorithmic Details This appendix provides technical details to supplement the method presented in Section 3. Algorithm 1 first summarizes the full decoupled self-play procedure. Section B.1 details the shared discrete autoregressive architecture and the role-injection mechanism. Section B.2 elaborates on the mathematical formulations of the scene-adaptive coalition learning pipeline, including probe statistics, feature representations, and the full training objective. Section B.3 provides the exact analytical derivations and loss formulations for the regret-aware constrained planner optimization. Finally, Section B.4 explains how the learned AWM is instantiated as a closed-loop sim-agent evaluator for transferable adversarial testing. Algorithm 1 Decoupled Self-Play Adversarial Fine-Tuning 1:Scenario distribution D; nominal planner πplannom _plan^nom; benign world model πpred _pred; coalition budget KmaxK_ ; CVaR tail mass τ; safety thresholds κ. 2:Frozen adversarial host πawm⋆ _awm , calibrated opponent distribution adv⋆P_adv , robust planner π^plan π_plan. 3:Initialize planner and world-model branches with the shared autoregressive factorization in Eq. 1; set πref←πplannom _ref← _plan^nom. 4:Stage A: learn the adversarial world model for Eq. 3. 5:for AWM minibatches ℬ⊂B do 6: for each scene s∈ℬs do 7: Background probe: assign ρi=bg _i=bg for all non-ego agents and roll out the role-conditioned generator in Eq. 5. 8: Build a sparse proposal pool PsP_s using realism-penalized threat and residual coverage (Appendix B.2.1). 9: if Ps=∅P_s= then 10: Set Astop1=Aspair=∅A_s^top1=A_s^pair= and ps=0p_s=0. 11: else 12: Select is⋆=argmaxi∈Psfpri(xi)i_s = _i∈ P_sf_pri(x_i); condition on ρis⋆=atk _i_s =atk and choose j^s=argmaxj∈Ps∖is⋆fpair(ϕ(is⋆,j)) j_s= _j∈ P_s \i_s \f_pair(φ(i_s ,j)). 13: Define Astop1=is⋆A_s^top1=\i_s \, Aspair=is⋆,j^sA_s^pair=\i_s , j_s\, and pair-admission probability ps=σ(αscal/τrisk))p_s=σ( _s^cal/ _risk)). 14: Update πawm _awm, fprif_pri, and fpairf_pair with role-switched counterfactual credit (Eq. 9); update the calibrator qηq_η using Appendix B.2.3. 15: end if 16: end for 17:end for 18:Freeze πawm⋆←πawm _awm ← _awm and qη⋆←qηq_η ← q_η. 19:Define adv⋆(Bs=top1∣s)=1−psP_adv (B_s=top1 s)=1-p_s and adv⋆(Bs=pair∣s)=psP_adv (B_s=pair s)=p_s as in Eq. 10. 20:Stage B: train the constrained planner best response in Eq. 4. 21:for planner minibatches ℬ⊂B do 22: for each scene s∈ℬs do 23: Roll out πθ _θ on the benign branch πpred _pred and adversarial branches Bs∈top1,pairB_s∈\top1,pair\ induced by πawm⋆ _awm . 24: Compute the nominal-retention loss ℒnomL_nom in Eq. 11. 25: Compute adversarial regrets Δsb(θ)=Usb(πref)−Usb(πθ) _s^b(θ)=U_s^b( _ref)-U_s^b( _θ) and the two-point regret distribution in Eqs. 12–13. 26: Obtain wstop1,wspairw_s^top1,w_s^pair from the closed-form regret-CVaR allocation in Appendix B.3.2. 27: Accumulate the adversarial tail-risk loss ℒrobL_rob in Eq. 14 and safety statistics ℒdualL_dual. 28: end for 29: Update θ with ℒnom+ℒrobL_nom+L_rob; update dual variables using thresholds κ (Appendix B.3.3). 30:end for 31:Select π^plan π_plan by validation robustness under adv⋆P_adv subject to nominal-retention criteria. 32:return πawm⋆ _awm , adv⋆P_adv , and π^plan π_plan. B.1 Discrete Autoregressive Planning Architecture and Role Conditioning This section details the architectural choices required to implement the unified planning-prediction decomposition introduced in Section 2, and expands upon the role-conditioned generation mechanism. By casting both planning and environment simulation as discrete next-token prediction tasks [43], we ensure that adversarial interventions modify only the behavioral distribution of the agents, without violating the underlying physical transition dynamics. B.1.1 Unified Autoregressive Motion Generation Interface A cornerstone of our framework is that both the ego planner and the environment evolution can be formulated as discrete next-token prediction tasks. Let xtx_t denote the interactivep scene state at step t, which includes static map elements, historical agent trajectories, and the generated future prefix up to t. For each agent type (e.g., vehicles, pedestrians), we maintain a discrete motion vocabulary T whose entries correspond to local motion primitives (e.g., discretized displacement and heading increments (Δx,Δy,Δψ)( x, y, ψ)) over a fixed autoregressive interval following previous practices [46, 38]. During the rollout, the standard two-branch architecture factorizes the joint one-step generation into a planning branch πplan _plan for the ego agent and an environment branch πpred _pred for all non-ego agents: p(t∣xt)=πplan(ze,t∣xt)∏i∈sπpred(zi,t∣xt).p( z_t x_t)= _plan(z_e,t x_t) _i _s _pred(z_i,t x_t). (15) Once the step-wise token set t=zi,ti∈e∪s z_t=\z_i,t\_i∈\e\ _s is sampled, it is decoded into continuous kinematic updates. A deterministic transition operator Γ then updates the global coordinates and advances the scene state: xt+1=Γ(xt,t)x_t+1= (x_t, z_t). Crucially, this rollout interface is strictly preserved throughout our framework. The AWM alters who generates the non-ego tokens and how they behave, but it does not modify the underlying simulator, the token space, or the physical transition dynamics. Consequently, the normal, top-1, and pair adversarial branches share the identical state representation, transition operator, and physical motion manifold, ensuring that the planner is evaluated fairly across different attack complexities and are strictly comparable. B.1.2 Model Instantiation and Role Conditioning Mechanism To instantiate the policies, both the planner and the environment branches share a unified high-level network template but maintain separate parameters. Specifically, for a branch k∈pred,plank∈\ pred, plan\, the architecture comprises a map encoder EmapkE_map^k, an interaction backbone BkB^k, and a token decoder DkD^k: gk=Emapk(xt),hi,tk=Bk(xt,gk),ℓi,tk=Dk(hi,tk),g^k=E_map^k(x_t), h_i,t^k=B^k(x_t,g^k), _i,t^k=D^k(h_i,t^k), (16) where hi,tkh_i,t^k is the hidden state for agent i and ℓi,tk _i,t^k are the output logits over the vocabulary T. The backbone BkB^k typically employs stacked transformer blocks over relation graphs (e.g., temporal self-attention, map-to-agent cross-attention, and agent-agent interaction) [43, 46, 54]. To construct the AWM from the pretrained predictive model πpred _pred without training a separate adversarial generator from scratch, we introduce a latent role conditioning ρi∈ego,bg,atk _i∈\ ego, bg, atk\ for each agent. The role identity is injected into the network at two levels to alter the agent’s interaction behavior and output distribution. First, an agent-side role embedding Eagent(ρi)E_agent( _i) is added to the agent’s initial hidden state before the spatial-temporal interaction blocks, allowing the role to influence how the agent attends to the map and other vehicles. Second, a decoder-side embedding Edec(ρi)E_dec( _i) is added immediately before the final token projection. The role-conditioned generation is given by: h~i,tawm=hi,tawm+Eagent(ρi),ℓ~i,tawm=Dawm(h~i,tawm+Edec(ρi)). h_i,t^awm=h_i,t^awm+E_agent( _i), _i,t^awm=D^awm ( h_i,t^awm+E_dec( _i) ). (17) The background role embedding E(bg)E( bg) is initialized as a zero vector. Consequently, when no attacker is activated (ρi=bg,∀i∈s _i= bg,∀ i _s), the AWM perfectly recovers the naturalistic behavior of the pretrained πpred _pred. Furthermore, switching between factual and counterfactual coalitions (as required in Section 3.2) only requires modifying the role table ρi _i without altering the network parameters or the simulator. This guarantees that the calculated counterfactual utility gaps isolate the true marginal contribution of the coalition rather than architectural discrepancies, which is the cornerstone of the counterfactual credit assignment (Section 3.2). B.2 Algorithmic Details of Adversarial Coalition Learning This section expands upon the scene-adaptive coalition learning pipeline, detailing the mathematical formulations for the probe signals, the neural selector inputs, and the multi-stage training objectives. B.2.1 Probe-Time Threat Signals and Residual Proposal Construction Directly evaluating the neural primary selector on all background agents is computationally prohibitive. The objective of the proposal stage is to reduce the combinatorial multi-agent search space into a sparse potential candidate. Therefore, we execute a lightweight background probe rollout (where ρi=bg,∀i∈s _i= bg,∀ i _s) to extract heuristic threat signals and construct a sparse proposal set PsP_s. For each non-ego agent i, we first compute a dense, heuristic ego-centric threat signal ri,dense(t)r_i,dense(t) at each step t, which aggregates proximity (rproxr_prox), time-to-collision (TTC) approximations (rcpar_cpa), blocking behaviors (rblockr_block), and cut-in maneuvers (rcutinr_cutin): ri,dense(t)=wproxri,prox(t)+wcpari,cpa(t)+wblockri,block(t)+wcutinri,cutin(t).r_i,dense(t)=w_proxr_i,prox(t)+w_cpar_i,cpa(t)+w_blockr_i,block(t)+w_cutinr_i,cutin(t). (18) To capture severe safety violations, we define the raw probe signal by incorporating hard collision indicators: riprobe(t)=max(ri,dense(t),λttci,ttc(t),λcoli,col(t)).r_i^probe(t)= (r_i,dense(t), _ttcI_i,ttc(t), _colI_i,col(t) ). (19) To discourage the selection of agents that might execute physically invalid or trivial attacks (e.g., driving off-road to hit the ego), we define a comprehensive penalized signal: r¯iprobe(t)=riprobe(t)−wkincikin(t)−wroadciroad(t)−wobsciobs(t), r_i^probe(t)=r_i^probe(t)-w_kinc_i^kin(t)-w_roadc_i^road(t)-w_obsc_i^obs(t), (20) where ci(⋅)c_i^(·) are penalty indicators for kinematic violations, off-road driving, and static obstacle collisions, respectively, weighted by constants w(⋅)w_(·). Using the discounted return operator G(v)τ=∑t=τHγt−τv(t)G(v)_τ= _t=τ^Hγ^t-τv(t), we obtain the scalarized pure threat score qi=maxτG(riprobe)τq_i= _τG(r_i^probe)_τ and the comprehensive score q¯i=maxτG(r¯iprobe)τ q_i= _τG( r_i^probe)_τ. The first proposal candidate is greedily selected based on the highest q¯i q_i. To ensure the proposal set PsP_s is diverse, subsequent candidates are added based on their residual threat coverage. Given a partially constructed proposal set S, the residual threat of a new agent i is r~iS(t)=max(riprobe(t)−maxj∈Srjprobe(t),0) r_i^S(t)= (r_i^probe(t)- _j∈ Sr_j^probe(t),0 ). The residual score is ψiS=maxτG(r~iS)τ _i^S= _τG( r_i^S)_τ. We only admit agent i if its relative residual ratio ηiS=ψiS/(qi+ϵ) _i^S= _i^S/(q_i+ε) exceeds a predefined threshold. The rationale behind this design is to ensure that newly proposed agents explain complementary spatial-temporal adversarial potential rather than merely duplicating the interaction patterns of already selected candidates, thereby preserving diversity for the pair-ranking stage. B.2.2 Feature Representations for Primary Selector and Pair Ranker While scalar probe scores provide a heuristic prior for efficient filtering, they lack the higher-order interaction context necessary to identify the true optimal attacker. The learnable primary selector fprif_pri and the conditional pair ranker fpairf_pair bridge this gap by fusing neural hidden states with proposal statistics. Primary Selector. For each candidate i∈Psi∈ P_s, the primary selector fprif_pri operates on a fused feature vector xix_i that concatenates the high-dimensional hidden state from the background probe with the heuristic proposal statistics: xi=[hiprobe,(i∈Ps),rank(i)Bprop,qi,q¯i,ψiS,mi],x_i= [h_i^probe,\;I(i∈ P_s),\; rank(i)B_prop,\;q_i,\; q_i,\; _i^S,\;m_i ], (21) where hiprobeh_i^probe is the final-layer hidden state of the AWM extracted from the background probe, BpropB_prop is the proposal budget, and mim_i is a binary stage marker. The selector outputs a confidence score ai=fpri(xi)a_i=f_pri(x_i), and the primary attacker is determined as is∗=argmaxi∈Psaii_s^*= _i∈ P_sa_i. After selecting the primary attacker, we retain a small runner-up candidate set to serve as potential partners: Rs=TopBpair(i∈Ps∖is∗;ai),R_s=Top_B_pair ( \i∈ P_s \i_s^*\ \;a_i ), (22) where BpairB_pair is the maximum partner-candidate budget and TopBpair(⋅;ai)Top_B_pair(·;a_i) returns the candidates with the largest primary-selector scores. Thus, PsP_s is the proposal pool produced by the residual threat-coverage procedure, while RsR_s is the smaller selector-filtered candidate pool used for conditional pair ranking. Conditional Pair Ranker and Calibrator. To evaluate the synergistic value of a secondary attacker, we execute a second conditional probe where is∗i_s^* is assigned ρ=atkρ= atk. This yields conditional hidden states hcondh^cond that encode the scene’s reaction to the primary attack. For any candidate partner j∈Ps∖is∗j∈ P_s \i_s^*\, the pair ranker fpairf_pair takes a relational feature ϕ(is∗,j)φ(i_s^*,j) that captures the conditional interaction dynamics: ϕ(is∗,j)=[his∗cond,hjcond,|his∗cond−hjcond|,his∗cond⊙hjcond,q¯is∗,q¯j,ψjis∗,Δais∗,j],φ(i_s^*,j)= [h_i_s^*^cond,\;h_j^cond,\; |h_i_s^*^cond-h_j^cond |,\;h_i_s^*^cond h_j^cond,\; q_i_s^*,\; q_j,\; _j^\i_s^*\,\; a_i_s^*,j ], (23) where ⊙ denotes element-wise multiplication, and Δais∗,j=fpri(xis∗)−fpri(xj) a_i_s^*,j=f_pri(x_i_s^*)-f_pri(x_j) represents the confidence gap from the primary selector. The pair ranker predicts the marginal coalition gain g^s(is∗,j)=fpair(ϕ(is∗,j)) g_s(i_s^*,j)=f_pair(φ(i_s^*,j)), and the optimal partner is j^s=argmaxjg^s(is∗,j) j_s= _j g_s(i_s^*,j). Subsequently, the scene-adaptive calibrator qηq_η determines whether admitting j^s j_s is globally beneficial (should actually be admitted). The scene-level feature aggregates the selected pair representation, the predicted pair gain, the primary-selector confidence, and two cardinality-based uncertainty indicators: ψs=[ϕ(is∗,j^s),g^s(is∗,j^s),ais∗,σ(ais∗),|Ps|,|Rs|]. _s= [φ(i_s^*, j_s),\; g_s(i_s^*, j_s),\;a_i_s^*,\;σ(a_i_s^*),\;|P_s|,\;|R_s| ]. (24) Here, |Ps||P_s| denotes the number of plausible attackers admitted by the proposal stage, and |Rs||R_s| denotes the number of selector-retained runner-up candidates considered for pair ranking. These two scalars provide the calibrator with scene-level ambiguity information: a larger |Ps||P_s| indicates a denser or more ambiguous adversarial proposal pool, while a larger |Rs||R_s| indicates more plausible secondary partners after primary selection. The calibrator is trained via Binary Cross-Entropy (BCE) to predict the probability psp_s that the true marginal improvement Gspair=Us(is∗)−Us(is∗,j^s)G_s^pair=U_s(\i_s^*\)-U_s(\i_s^*, j_s\) is strictly positive. B.2.3 Multi-Stage Training Objective for AWM To stabilize the optimization of the role-conditioned policy alongside the coalition search modules, we employ a two-stage training paradigm. Stage I (Host and Search Optimization): We jointly train the role-conditioned AWM policy, the primary selector, and the conditional pair ranker. The objective is: ℒstage1=ℒpolicy(Δhyb)+λpriℒpri+λpairℒpair+Ωstab,L_stage1=L_policy( ^hyb)+ _priL_pri+ _pairL_pair+ _stab, (25) Here, ℒpolicyL_policy is the policy gradient loss driven by the hybrid counterfactual reward Δihyb _i^hyb (defined in Eq. 9). ℒpriL_pri and ℒpairL_pair supervise the selector and ranker using the observed utility gaps as regression and ranking targets. Specifically: (1) For the primary selector, the target label for candidate i is its absolute adversarial impact compared to the nominal background traffic: yipri=Us(∅)−Us(i)y_i^pri=U_s( )-U_s(\i\), where ∅ denotes no active attackers. (2) For the conditional pair ranker, the target label for a partner candidate j is its marginal coalition gain over the primary attacker is∗i_s^*: yis∗,jpair=Us(is∗)−Us(is∗,j)y_i_s^*,j^pair=U_s(\i_s^*\)-U_s(\i_s^*,j\). Finally, the stabilizer Ωstab _stab anchors the AWM’s background behavior to the pretrained predictive prior πpred _pred via KL divergence, preventing the model from collapsing into unrealistic adversarial behaviors. Stage I (Calibrator Optimization): We freeze the components from Stage-I and train the scene-level calibrator qηq_η using the BCE loss ℒcalL_cal. This decoupling is crucial because the calibrator’s objective is to act as an adaptive gate that decides when a multi-agent attack is statistically justified, rather than discovering new attackers. Stage-I trains the scene-level calibrator qηq_η to ensure unbiased admission probabilities. B.3 Details of Regret-Aware Constrained Planner Optimization This section details the planner-side robust optimization (Section 3.3), including the exact formulation of the trust region constraints, the closed-form derivation of the CVaR weights, and the dual update mechanism for safety constraints. It explains how the AWM is converted into a risk distribution and how the constrained optimization is solved. B.3.1 Reference-Anchored Nominal Retention During the training of the planner, we do not sample arbitrary attacks from a replay buffer. Instead, for each scene s, the frozen AWM executes the coalition pipeline to generate a top-1 attacker gate gstop1g_s^top1 and a pair gate gspairg_s^pair (if a valid partner exists). This yields three explicit rollout branches, evaluated by their respective utilities: Jsnorm(π)=Us(π,πpred),Jstop1(π)=Us(π,πawm;gstop1),Jspair(π)=Us(π,πawm;gspair).J_s^norm(π)=U_s(π, _pred), J_s^top1(π)=U_s(π, _awm;g_s^top1), J_s^pair(π)=U_s(π, _awm;g_s^pair). (26) To prevent the planner from sacrificing naturalistic driving capabilities in pursuit of worst-case robustness (Section 3.3), we enforce a reference-anchored nominal retention objective on the normal branch. Let πref _ref be the frozen baseline planner. We define the normal utility margin as msnorm(θ)=Jsnorm(πθ)−Jsnorm(πref)m_s^norm(θ)=J_s^norm( _θ)-J_s^norm( _ref). To penalize degradation without forcing the planner to over-optimize already safe scenes, we introduce an asymmetric margin penalty that activates only when the planner’s performance drops below an acceptable margin ϵnorm≥0 _norm≥ 0: ψsnorm(θ)=[−ϵnorm−msnorm(θ)]+2, _s^norm(θ)= [- _norm-m_s^norm(θ) ]_+^2, (27) where [⋅]+=max(⋅,0)[·]_+= (·,0). The total nominal retention loss combines the standard policy gradient surrogate ℓsnorm(θ) _s^norm(θ), a trajectory-level KL divergence constraint, and the margin penalty: ℒnom(θ)=s∼[λpgℓsnorm(θ)+λklDKL(πθ‖πref∣xs,1:Hnorm)+λmarginψsnorm(θ)],L_nom(θ)=E_s [ _pg _s^norm(θ)+ _klD_KL( _θ _ref x_s,1:H^norm)+ _margin _s^norm(θ) ], (28) where ℓsnorm(θ) _s^norm(θ) is the standard GRPO [39] surrogate loss maximizing nominal utility. This asymmetric design penalizes performance degradation but does not force the planner to over-optimize on already safe scenes. B.3.2 Analytical Regret-CVaR Weighting The core of robust adaptation is to optimize the risk objective with respect to the AWM-induced regret distribution. To optimize the adversarial tail-risk, we compute the reference-relative regret Δsb(θ)=Jsb(πref)−Jsb(πθ) _s^b(θ)=J_s^b( _ref)-J_s^b( _θ) for b∈top1,pairb∈\top1,pair\. The AWM calibrator induces a discrete two-point risk distribution: sθ=(Δstop1(θ),1−ps),(Δspair(θ),ps),D_s^θ= \ ( _s^top1(θ),1-p_s ), ( _s^pair(θ),p_s ) \, (29) where psp_s is the calibrated probability of admitting the pair attack. We apply the Conditional Value-at-Risk (CVaR) operator [35] at tail level τ∈(0,1]τ∈(0,1] over this specific distribution. Because AWM naturally induces a discrete two-point distribution over the attack types b∈top1,pairb∈\ top1, pair\, the CVaR allocation admits an elegant closed-form analytical solution, avoiding the high variance typically associated with sampling-based CVaR estimation. Let b(1)b_(1) and b(2)b_(2) denote the two branches sorted by their regret in descending order (i.e., Δsb(1)≥Δsb(2) _s^b_(1)≥ _s^b_(2)), and let μsb(1) _s^b_(1) be the probability mass of the worst-case branch. For a specified tail level τ, the CVaR operator focuses strictly on the worst τ-proportion of the distribution. The exact dynamic weight wsb(1)w_s^b_(1) for the worse branch is given by: wsb(1)(τ)=1,if μsb(1)≥τ,μsb(1)τ,if μsb(1)<τ.w_s^b_(1)(τ)= cases1,&if _s^b_(1)≥τ,\\ _s^b_(1)τ,&if _s^b_(1)<τ. cases (30) The weight for the better branch is simply the residual tail mass: wsb(2)(τ)=1−wsb(1)(τ)w_s^b_(2)(τ)=1-w_s^b_(1)(τ). Intuitively, these weights dynamically route the gradient updates. If the probability of the worst-case attack exceeds τ, the planner focuses entirely on surviving that specific attack (w=1w=1) and absorbs as much of the tail mass τ as its probability allows. If the worst-case attack is extremely rare (probability <τ<τ), the planner distributes its learning capacity proportionally between both branches. In our implementation, these weights are treated as detached constants during backpropagation. The final robust objective dynamically scales the branch-wise policy gradient losses: ℒrob(θ)=s∼[sg(wstop1(τ))ℓstop1(θ)+sg(wspair(τ))ℓspair(θ)],L_rob(θ)=E_s [sg (w_s^top1(τ) ) _s^top1(θ)+sg (w_s^pair(τ) ) _s^pair(θ) ], (31) where sg(⋅)sg(·) denotes the stop-gradient operator, making the planner optimize its policy rather than attempting to manipulate the risk distribution itself. This dynamic weighting reconciles the sparsity of the AWM’s attacks with the planner’s safety demands, ensuring the planner’s gradients are guided by the scene-adaptive vulnerabilities discovered by the AWM. Because pair attacks are scene-dependent, their calibrated probability psp_s is often small. Under a standard expected loss, a rare but fatal pair attack would have its gradient severely diluted by psp_s, causing the planner to under-optimize for it. By applying CVaR, if the low-probability pair branch currently induces the highest regret, the operator aggressively upweights it (e.g., scaling its weight from psp_s to ps/τp_s/τ or 11). This encourages the planner to defend against its most severe weakness rather than just average-case perturbations. B.3.3 Safety-Constrained Dual Update Parameterization To prevent the planner from exploiting the regret objective by violating hard traffic rules (e.g., evading an attacker by driving off-road), we enforce explicit safety constraints via a dual update mechanism (Section 3.3). Let vs,mb∈[0,1]v_s,m^b∈[0,1] denote the violation indicator for the m-th safety cost under branch b. The expected branch-mixed violation rate is v¯s,m=(1−ps)vs,mtop1+psvs,mpair v_s,m=(1-p_s)v_s,m top1+p_sv_s,m pair. To ensure optimization stability and guarantee that the utility multipliers remain strictly positive and bounded, we parameterize the progress reward multiplier λR _R and the M safety cost multipliers λm _m jointly using a softmax projection over unconstrained dual variables ζ∈ℝMζ ^M and a fixed progress logit αR _R: [λR,λ1,…,λM]=softmax([αR,ζ1,…,ζM]).[ _R, _1,…, _M]=softmax([ _R, _1,…, _M]). (32) This parameterization ensures λm∈(0,1) _m∈(0,1) and ∑λ=1Σλ=1. Given the admissible violation thresholds κ∈ℝMκ ^M, the dual objective to be maximized with respect to ζ is: ℒdual(ζ)=∑m=1Mλm(ζ)(s∼[v¯s,m]−κm).L_dual(ζ)= _m=1^M _m(ζ) (E_s [ v_s,m]- _m ). (33) During training, the planner parameters θ and the dual variables ζ are updated via alternating gradient descent-ascent. This separation allows the planner to aggressively minimize adversarial regret while treating hard traffic rules as strict boundaries rather than easily exploitable reward shaping. B.4 AWM as a Closed-Loop Sim-Agent This section describes how the learned AWM is used as a simulator-side traffic world model for the transferable adversarial evaluation in Table 2. The goal is to expose a planner to adversarial but feasible non-ego reactions within the closed-loop reactive simulation. This replaces the rule-based or replay agents with our learned AWM policies. We therefore instantiate AWM as a drop-in non-ego policy: the evaluated planner still controls only the ego vehicle, the simulator still advances the world state with its standard transition and scoring logic, and AWM only changes the motion proposals of selected surrounding agents. State synchronization and policy roles. At each simulation step t, the simulator provides the current closed-loop state xtx_t, including the map context, the ego state produced by the evaluated planner, the generated non-ego histories, and the generated future prefix. We encode xtx_t using the same tokenized autoregressive representation used in the AWM training. The normal sim-agent samples each non-ego token from the background predictive policy πpred _pred. The adversarial sim-agent instead runs the AWM coalition pipeline on the current state: the proposal module constructs a sparse candidate set, the primary selector chooses the main adversarial agent, the pair ranker proposes a complementary partner, and the calibrator decides whether the pair branch is admitted. The selected coalition receives the adversarial role ρi=atk _i= atk, while all other agents retain the background role ρi=bg _i= bg. Closed-loop execution. The sim-agent executes a mixed non-ego policy rather than replacing the entire traffic scene with adversarial behavior. For each non-ego agent i, we first generate a normal token and, if i belongs to the admitted coalition AtA_t, an adversarial token from the role-conditioned AWM. The token finally submitted to the simulator is zi,tsim=zi,tawm∼πawm(⋅∣xt,ρi=atk),i∈Atandgi,t=1,zi,tnorm∼πpred(⋅∣xt,ρi=bg),otherwise,z_i,t^sim= casesz_i,t^awm _awm(· x_t, _i= atk),&i∈ A_t\ and\ g_i,t=1,\\ z_i,t^norm _pred(· x_t, _i= bg),&otherwise, cases (34) where gi,tg_i,t is a feasibility gate defined below. The accepted token is decoded into a short-horizon displacement and heading update, and the simulator applies the same state transition used for the normal learned sim-agent. This design keeps the evaluation controlled: the AWM changes the conditional behavior distribution of selected agents, but it does not alter the ego planner, the map, the vehicle model, the collision checker, or the closed-loop score. Feasibility-gated adversarial replacement. Before an adversarial token is accepted, the sim-agent checks whether the decoded motion remains within the shared motion vocabulary’s plausible region and the simulator’s local validity constraints. The gate rejects proposals that introduce implausible kinematic jumps, immediate map invalidity, static-object conflicts, or degenerate ego-local overlaps that would be better interpreted as simulator artifacts than as adversarial traffic. Formally, the gate can be viewed as gi,t=[ci,tkin≤κkin,ci,tmap≤κmap,ci,tstatic≤κstatic,ci,tego≤κego],g_i,t=I\! [c_i,t^kin≤ _kin,\ c_i,t^map≤ _map,\ c_i,t^static≤ _static,\ c_i,t^ego≤ _ego ], (35) where the four checks summarize kinematic continuity, map consistency, static-object validity, and ego-local sanity constraints. If the gate fails, the agent falls back to the normal sim-agent token for the same state and step. The fallback makes the AWM host an adversarial simulator under feasibility constraints, rather than a mechanism for injecting arbitrary invalid trajectories. Temporal use of the coalition. The coalition is selected from the current closed-loop state, not from a fixed logged trajectory. This matters because the evaluated planner can change the future interaction geometry, and the AWM must respond to the planner’s generated ego behavior. Recomputing the role assignment online lets the adversarial host adapt when the ego yields, accelerates, or changes its path. At the same time, the attack budget Kmax=2K_ =2 and the calibrator prevent the sim-agent from turning every nearby participant into an attacker. Therefore, the closed-loop host remains sparse: it stresses the planner through a small number of behaviorally meaningful agents while leaving the rest of the traffic under the normal predictive policy. Interpretation of AWM-sim scores. The AWM sim-agent score is a closed-loop evaluation score under an adversarial non-ego traffic host, and it is distinct from both open-loop AWM rollout rewards and ordinary planner closed-loop scores reported in the main benchmark (Table 1). A lower score under the AWM host indicates that the planner is more vulnerable when nearby traffic follows the learned adversarial policy. To verify that score drops are caused by controlled adversarial interaction rather than simulator collapse, we report simulator-side diagnostics in Table 6. Appendix C Theoretical Analysis C.1 Notation and Autoregressive Local Stability For a scene s, let Ωs⋆ _s denote the set of all admissible adversarial executions. An element ω∈Ωs⋆ω∈ _s specifies both an active coalition As⊆sA_s _s satisfying |As|≤Kmax|A_s|≤ K_ and the corresponding role-conditioned non-ego rollout law induced by the AWM under the shared tokenized transition operator. Let νs∈Δ(Ωs⋆) _s∈ ( _s ) be a scene-conditional adversarial environment distribution. For a planner π, define the expected finite-horizon utility Js(π,νs):=ω∼νsτ∼(π,ω)[∑t=1Hγt−1ut(xt,zt)],J_s(π, _s):=E_ω _sE_τ (π,ω) [ _t=1^Hγ^t-1u_t(x_t,z_t) ], (36) where the rollout follows the autoregressive dynamics xt+1=Γ(xt,t)x_t+1= (x_t, z_t) from Section 2.1. Throughout the analysis, we assume the step utility is bounded: |ut(xt,zt)|≤umax,∀t∈1,…,H.|u_t(x_t,z_t)|≤ u_ , ∀ t∈\1,…,H\. (37) Define the finite-horizon sensitivity constant ΛH,γ:=2umax∑t=1Htγt−1=2umax1−(H+1)γH+HγH+1(1−γ)2,γ∈(0,1),umaxH(H+1),γ=1. _H,γ:=2u_ _t=1^Htγ^t-1= cases 2u_ 1-(H+1)γ^H+Hγ^H+1(1-γ)^2,&γ∈(0,1),\\[8.0pt] u_ H(H+1),&γ=1. cases (38) Let πref _ref be the frozen reference planner. For the theoretical analysis, we use the stronger statewise trust region Πϵ:=π:supxDKL(π(⋅∣x)∥πref(⋅∣x))≤ϵ. _ε:= \π: _xD_KL (π(· x)\,\|\, _ref(· x) )≤ε \. (39) The actual implementation uses a trajectory-level KL penalty, so Eq. 39 should be read as an idealized sufficient condition. We also implicitly restrict attention to planners that are absolutely continuous with respect to πref _ref on the discrete token support, so that the KL divergence is finite. By Pinsker’s inequality, any π∈Πϵπ∈ _ε satisfies δ(π,πref):=supxDTV(π(⋅∣x),πref(⋅∣x))≤ϵ/2=:δϵ.δ(π, _ref):= _xD_TV (π(· x), _ref(· x) )≤ ε/2=: _ε. (40) Lemma C.1 (Autoregressive planner sensitivity) For any fixed scene s, any fixed adversarial environment distribution νs∈Δ(Ωs⋆) _s∈ ( _s ), and any two planners π,π′π,π , the finite-horizon utility satisfies |Js(π,νs)−Js(π′,νs)|≤ΛH,γδ(π,π′), |J_s(π, _s)-J_s(π , _s) |≤ _H,γ\,δ(π,π ), (41) where δ(π,π′):=supxDTV(π(⋅∣x),π′(⋅∣x)).δ(π,π ):= _xD_TV (π(· x),π (· x) ). (42) Consequently, for every π∈Πϵπ∈ _ε, |Js(π,νs)−Js(πref,νs)|≤ΛH,γδϵ. |J_s(π, _s)-J_s( _ref, _s) |≤ _H,γ _ε. (43) Proof. Couple the two rollouts of π and π′π against the same sampled environment law ω∼νsω _s and the same exogenous randomness. Conditional on identical prefixes up to a given step, the two rollouts have the same state and therefore the same non-ego conditional laws; we couple the non-ego samples identically until the ego token first differs. As long as the generated token prefixes are identical, the scene states are identical because the transition operator Γ is deterministic given the sampled token set. At any step t, conditional on identical prefixes up to t−1t-1, a maximal coupling of the ego token distributions gives ℙ(ze,t≠ze,t′)≤DTV(π(⋅∣xt),π′(⋅∣xt))≤δ(π,π′).P(z_e,t≠ z _e,t)≤ D_TV (π(· x_t),π (· x_t) )≤δ(π,π ). (44) Let EtE_t be the event that the two coupled rollouts have diverged at or before step t. By the union bound, ℙ(Et)≤tδ(π,π′).P(E_t)≤ t\,δ(π,π ). (45) On EtcE_t^c, the two-step utilities are identical. On EtE_t, their difference is at most 2umax2u_ . Therefore, |Js(π,νs)−Js(π′,νs)| |J_s(π, _s)-J_s(π , _s) | ≤∑t=1Hγt−1⋅2umaxℙ(Et) ≤ _t=1^Hγ^t-1· 2u_ P(E_t) (46) ≤2umax∑t=1Htγt−1δ(π,π′)=ΛH,γδ(π,π′). ≤ 2u_ _t=1^Htγ^t-1δ(π,π )= _H,γδ(π,π ). Eq. 43 follows from Eq. 40. □ C.2 Local Stackelberg Justification of the Decoupled Stage-A/B Solver We now analyze the error introduced by freezing Stage A before updating the planner. The ideal adversary class Δ(Ωs⋆) ( _s ) is too large to optimize directly. Stage A restricts it to the structured class induced by the AWM pipeline: proposal→primary selector→conditional pair ranker→scene calibrator proposal\;→\; primary selector\;→\; conditional pair ranker\;→\; scene calibrator. Let Ω^s⊆Ωs⋆ _s _s denote the support covered by this structured pipeline, and define s⋆Q_s as the full set of admissible adversarial environment distributions satisfying the attack budget and realism constraints. Let ^s⊆s⋆ Q_s _s denote the structured class achieved by our AWM pipeline: ^s:=Δ(Ω^s)⊆s⋆:=Δ(Ωs⋆). Q_s:= ( _s) _s := ( _s ). (47) For a fixed planner π, define the full and structured worst-case utilities: gs(π):=infνs∈s⋆Js(π,νs),g^s(π):=infνs∈^sJs(π,νs).g_s(π):= _ _s _s J_s(π, _s), g_s(π):= _ _s∈ Q_sJ_s(π, _s). (48) Because ^s⊆s⋆ Q_s _s , we have g^s(π)≥gs(π). g_s(π)≥ g_s(π). (49) The structured class may not cover the full adversarial class. We define the scene-level coverage gap as δcov,s(π):=g^s(π)−gs(π)≥0. _cov,s(π):= g_s(π)-g_s(π)≥ 0. (50) Proposition C.2 (Local error of freezing the Stage-A adversary) Assume Stage A returns a frozen structured adversary ν^s∈^s ν_s∈ Q_s satisfying the reference-planner suboptimality condition Js(πref,ν^s)≤g^s(πref)+εA,s.J_s( _ref, ν_s)≤ g_s( _ref)+ _A,s. (51) Then, for every planner π∈Πϵπ∈ _ε, 0≤Js(π,ν^s)−gs(π)≤εA,s+δcov,s(πref)+2ΛH,γδϵ.0≤ J_s(π, ν_s)-g_s(π)≤ _A,s+ _cov,s( _ref)+2 _H,γ _ε. (52) Proof. The lower bound follows immediately from the definition of gs(π)g_s(π): gs(π)=infνs∈s⋆Js(π,νs)≤Js(π,ν^s).g_s(π)= _ _s _s J_s(π, _s)≤ J_s(π, ν_s). (53) For the upper bound, add and subtract the reference-planner quantities: Js(π,ν^s)−gs(π) J_s(π, ν_s)-g_s(π) =Js(π,ν^s)−Js(πref,ν^s)⏟(I)+Js(πref,ν^s)−gs(πref)⏟(II)+gs(πref)−gs(π)⏟(III). = J_s(π, ν_s)-J_s( _ref, ν_s)_(I)+ J_s( _ref, ν_s)-g_s( _ref)_(I)+ g_s( _ref)-g_s(π)_(I). By Lemma C.1, (I)≤ΛH,γδϵ.(I)≤ _H,γ _ε. (54) For term (II)(I), Eq. 51 and Eq. 50 give (II) (I) =Js(πref,ν^s)−gs(πref) =J_s( _ref, ν_s)-g_s( _ref) (55) ≤εA,s+g^s(πref)−gs(πref) ≤ _A,s+ g_s( _ref)-g_s( _ref) =εA,s+δcov,s(πref). = _A,s+ _cov,s( _ref). For term (III)(I), observe that gs(π)g_s(π) is the pointwise infimum of functions Js(π,νs)J_s(π, _s) that are all ΛH,γ _H,γ-Lipschitz in π by Lemma C.1. The pointwise infimum of uniformly Lipschitz functions is also Lipschitz with the same constant. Thus (III)≤|gs(πref)−gs(π)|≤ΛH,γδϵ.(I)≤|g_s( _ref)-g_s(π)|≤ _H,γ _ε. (56) Combining the bounds on (I)(I), (II)(I), and (III)(I) proves Eq. 52. □ Corollary C.3 (Approximate local robust value guarantee) Define the ideal local robust value Vϵ⋆:=supπ∈Πϵs∼[gs(π)].V_ε := _π∈ _εE_s [g_s(π) ]. (57) Let Stage B return a planner π^∈Πϵ π∈ _ε that is an εB _B-approximate best response to the frozen Stage-A adversary: s[Js(π^,ν^s)]≥supπ∈Πϵs[Js(π,ν^s)]−εB.E_s[J_s( π, ν_s)]≥ _π∈ _εE_s[J_s(π, ν_s)]- _B. (58) Let ε¯A:=s[εA,s],δ¯cov:=s[δcov,s(πref)]. _A:=E_s[ _A,s], δ_cov:=E_s[ _cov,s( _ref)]. (59) Then s[gs(π^)]≥Vϵ⋆−εB−ε¯A−δ¯cov−2ΛH,γδϵ.E_s[g_s( π)]≥ V_ε - _B- _A- δ_cov-2 _H,γ _ε. (60) Proof. Taking expectation in Proposition C.2 yields, for every π∈Πϵπ∈ _ε, s[gs(π)]≥s[Js(π,ν^s)]−ε¯A−δ¯cov−2ΛH,γδϵ.E_s[g_s(π)] _s[J_s(π, ν_s)]- _A- δ_cov-2 _H,γ _ε. (61) Applying Eq. 61 at π=π^π= π and using Eq. 58, s[gs(π^)] _s[g_s( π)] ≥s[Js(π^,ν^s)]−ε¯A−δ¯cov−2ΛH,γδϵ _s[J_s( π, ν_s)]- _A- δ_cov-2 _H,γ _ε (62) ≥supπ∈Πϵs[Js(π,ν^s)]−εB−ε¯A−δ¯cov−2ΛH,γδϵ ≥ _π∈ _εE_s[J_s(π, ν_s)]- _B- _A- δ_cov-2 _H,γ _ε ≥supπ∈Πϵs[gs(π)]−εB−ε¯A−δ¯cov−2ΛH,γδϵ ≥ _π∈ _εE_s[g_s(π)]- _B- _A- δ_cov-2 _H,γ _ε =Vϵ⋆−εB−ε¯A−δ¯cov−2ΛH,γδϵ. =V_ε - _B- _A- δ_cov-2 _H,γ _ε. □ Corollary C.4 (Support-mass interpretation of the coverage gap) Let UH:=umax∑t=1Hγt−1U_H:=u_ _t=1^Hγ^t-1 (63) so that |Js(π,ω)|≤UH|J_s(π,ω)|≤ U_H for every rollout execution ω. Assume that a full adversarial best response at the reference planner exists, and let νs,full⋆∈argminνs∈s⋆Js(πref,νs) _s,full ∈ _ _s _s J_s( _ref, _s). If the infimum is not attained, the same statement holds with an additional arbitrarily small optimality slack by taking an approximate minimizer. Define the omitted support mass ηs:=νs,full⋆(Ωs⋆∖Ω^s). _s:= _s,full ( _s _s ). (64) If ηs<1 _s<1, then δcov,s(πref)≤2UHηs. _cov,s( _ref)≤ 2U_H _s. (65) Consequently, δ¯cov≤2UHs∼[ηs]. δ_cov≤ 2U_HE_s [ _s]. (66) Proof. For each scene s, let ν~s ν_s be the renormalization of νs,full⋆ _s,full onto Ω^s _s. Then ν~s∈^s ν_s∈ Q_s and DTV(ν~s,νs,full⋆)=ηs.D_TV( ν_s, _s,full )= _s. (67) By the boundedness of the utility, |Js(πref,ν~s)−Js(πref,νs,full⋆)|≤2UHηs. |J_s( _ref, ν_s)-J_s( _ref, _s,full ) |≤ 2U_H _s. (68) Because g^s(πref)≤Js(πref,ν~s) g_s( _ref)≤ J_s( _ref, ν_s) and gs(πref)=Js(πref,νs,full⋆)g_s( _ref)=J_s( _ref, _s,full ), δcov,s(πref) _cov,s( _ref) =g^s(πref)−gs(πref) = g_s( _ref)-g_s( _ref) (69) ≤Js(πref,ν~s)−Js(πref,νs,full⋆)≤2UHηs. ≤ J_s( _ref, ν_s)-J_s( _ref, _s,full )≤ 2U_H _s. Averaging over s proves Eq. 66. □ Interpretation. Proposition C.2 and Corollaries C.3–C.4 formalize the role of the decoupled Stage-A/B solver. The frozen AWM objective is optimistic relative to the full worst-case game because it evaluates the planner against one learned structured adversary rather than the entire adversary class. However, freezing the AWM is a local Stackelberg approximation whose error is controlled by four interpretable quantities: the Stage-A optimization error ε¯A _A, the Stage-B best-response error εB _B, the structured-class coverage gap δ¯cov δ_cov, and the planner trust-region radius ϵε through the (ϵ)O( ε) term 2ΛH,γδϵ2 _H,γ _ε. The support-mass bound further explains why the proposal–top1–pair–calibrator design matters: it should cover the low-utility adversarial executions that carry mass under the true worst-case adversary, while avoiding an intractable dense subset search. C.3 Scene-Adaptive Pair Admission as a Bayes Gate The selector and pair ranker decide who should attack. The calibrator decides whether the second attacker should be admitted. This distinction can be formalized as a Bayes decision problem. For a scene s, let the candidate pair gain be Gspair:=Us(is⋆)−Us(is⋆,j^s).G_s^pair:=U_s(\i_s \)-U_s(\i_s , j_s\). (70) Positive GspairG_s^pair means that the pair attack reduces planner utility more than the top-1 attack. Let ψs _s be the scene-level calibrator feature. Proposition C.5 (Optimal pair-admission rule) Among all binary admission policies a(ψs)∈0,1a( _s)∈\0,1\, the policy maximizing expected admitted pair gain [a(ψs)Gspair]E [a( _s)G_s^pair ] (71) is a⋆(ψs)=[Gspair∣ψs]>0.a ( _s)=I \E[G_s^pair _s]>0 \. (72) If the calibrator is calibrated for the binary label Ys:=Gspair>0,ps=ℙ(Ys=1∣ψs),Y_s:=I\G_s^pair>0\, p_s=P(Y_s=1 _s), (73) and if m+(ψs):=[Gspair∣ψs,Ys=1],m−(ψs):=[−Gspair∣ψs,Ys=0],m_+( _s):=E[G_s^pair _s,Y_s=1], m_-( _s):=E[-G_s^pair _s,Y_s=0], (74) then, on feature values where the conditional quantities are well defined and m+(ψs)+m−(ψs)>0m_+( _s)+m_-( _s)>0, Eq. 72 is equivalent to ps>m−(ψs)m+(ψs)+m−(ψs).p_s> m_-( _s)m_+( _s)+m_-( _s). (75) When the conditional magnitudes are approximately stable over the deployment distribution, this reduces to a scalar threshold rule ps>τcal⋆p_s> _cal , which is the tractable rule used by our scene-adaptive calibrator. Proof. For any measurable admission policy a(ψs)a( _s), [a(ψs)Gspair] [a( _s)G_s^pair] =[a(ψs)[Gspair∣ψs]]. =E [a( _s)E[G_s^pair _s] ]. (76) Since a(ψs)a( _s) is binary, the pointwise maximizer admits the pair exactly when the conditional expected gain is positive, proving Eq. 72. Next, [Gspair∣ψs] [G_s^pair _s] =ps[Gspair∣ψs,Ys=1]+(1−ps)[Gspair∣ψs,Ys=0] =p_sE[G_s^pair _s,Y_s=1]+(1-p_s)E[G_s^pair _s,Y_s=0] (77) =psm+(ψs)−(1−ps)m−(ψs). =p_sm_+( _s)-(1-p_s)m_-( _s). This quantity is positive if and only if Eq. 75 holds. □ Interpretation. The calibrator is not merely a heuristic confidence score. It is a learned approximation to the Bayes gate that decides whether the additional attack dimension has positive expected value in the current scene. This explains why adaptive K>1K>1 is preferable to fixed K=2K=2: pair attacks should be admitted only when their conditional expected marginal gain is positive. C.4 Role-Switched Counterfactual Credit as a Difference-Reward Estimator We next justify the role-switched counterfactual rewards used to train the AWM. For this subsection, fix a scene s and a factual coalition A=AsfactA=A_s^fact. Let Us(A)U_s(A) denote the planner’s utility under the factual coalition, and define the adversarial team reward Rs(A):=−Us(A).R_s(A):=-U_s(A). (78) A larger Rs(A)R_s(A) means a stronger attack. For an active attacker i∈Ai∈ A, let Yi=(zi,1,…,zi,H)Y_i=(z_i,1,…,z_i,H) denote its sampled factual attack-token trajectory, and let Ψi:=∇ϕ(i)logpϕ(Yi∣ℋi,ρi=atk)=∑t=1H∇ϕlogπawm,ϕ(zi,t∣xt,ρi=atk) _i:= _φ^(i) p_φ(Y_i _i, _i= atk)= _t=1^H _φ _awm,φ (z_i,t x_t, _i= atk ) (79) be the score-function term associated with attacker i; ℋiH_i denotes the factual pre-sampling history needed to define the conditional trajectory law. When parameters are shared across attackers, the full attacker-side policy-gradient estimator sums Eq. 79 over active attackers. Let bg(A)bg(A) denote the counterfactual rollout obtained by switching all active attackers in A to the background role without removing the agents from the scene. Define the admissible baselines Biloo:=−Us(A∖i),Bteam:=−Us(bg(A)),B_i^loo:=-U_s(A \i\), B^team:=-U_s(bg(A)), (80) where both quantities are evaluated by the action-independent counterfactual replay described above. Then Δiloo=Us(A∖i)−Us(A)=Rs(A)−Biloo, _i^loo=U_s(A \i\)-U_s(A)=R_s(A)-B_i^loo, (81) and Δsteam=Us(bg(A))−Us(A)=Rs(A)−Bteam. _s^team=U_s(bg(A))-U_s(A)=R_s(A)-B^team. (82) Proposition C.6 (Hybrid counterfactual reward is gradient-aligned) Assume the counterfactual utilities used to form BilooB_i^loo and BteamB^team are evaluated with stop-gradient, are ℱicfF_i^cf-measurable, and are conditionally independent of the factual attacker trajectory YiY_i given ℱicfF_i^cf. For weights ωloo,ωteam≥0 _loo, _team≥ 0, define Δihyb:=ωlooΔiloo+ωteamΔsteam,W:=ωloo+ωteam. _i^hyb:= _loo _i^loo+ _team _s^team, W:= _loo+ _team. (83) Then [ΨiΔihyb]=W∇ϕ(i)[Rs(A)],E[ _i _i^hyb]=W\, _φ^(i)E[R_s(A)], (84) where ∇ϕ(i) _φ^(i) denotes the policy-gradient contribution associated with attacker i. In particular, if W>0W>0, the hybrid counterfactual reward has the same ascent direction as the adversarial team objective up to the positive scalar W; if W=1W=1, it is an unbiased estimator of that component. Proof. By the score-function identity applied to the factual attack trajectory, ∇ϕ(i)[Rs(A)]=[ΨiRs(A)]. _φ^(i)E[R_s(A)]=E[ _iR_s(A)]. (85) It remains to show that the counterfactual baselines vanish after multiplication by Ψi _i. Because BilooB_i^loo is ℱicfF_i^cf-measurable and conditionally independent of YiY_i given ℱicfF_i^cf, [ΨiBiloo] [ _iB_i^loo] =[Biloo[Ψi∣ℱicf]]. =E [B_i^looE[ _i _i^cf] ]. (86) For the conditional score, [Ψi∣ℱicf] [ _i _i^cf] =∑yipϕ(yi∣ℱicf)∇ϕlogpϕ(yi∣ℱicf) = _y_ip_φ(y_i _i^cf) _φ p_φ(y_i _i^cf) (87) =∑yi∇ϕpϕ(yi∣ℱicf)=∇ϕ∑yipϕ(yi∣ℱicf)=0. = _y_i _φp_φ(y_i _i^cf)= _φ _y_ip_φ(y_i _i^cf)=0. Thus [ΨiBiloo]=0E[ _iB_i^loo]=0. The same argument gives [ΨiBteam]=0E[ _iB^team]=0. Therefore, [ΨiΔihyb] [ _i _i^hyb] =ωloo[Ψi(Rs(A)−Biloo)]+ωteam[Ψi(Rs(A)−Bteam)] = _looE[ _i(R_s(A)-B_i^loo)]+ _teamE[ _i(R_s(A)-B^team)] (88) =(ωloo+ωteam)[ΨiRs(A)]=W∇ϕ(i)[Rs(A)]. =( _loo+ _team)E[ _iR_s(A)]=W\, _φ^(i)E[R_s(A)]. □ Lemma C.7 (Variance-optimal action-independent baseline) Let ℱ−iF_-i be any sigma-field that excludes the factual attacker sample YiY_i and satisfies [Ψi∣ℱ−i]=0E[ _i _-i]=0. Consider any ℱ−iF_-i-measurable baseline bi(ℱ−i)b_i(F_-i). Define the score-weighted second moment (bi):=[‖Ψi‖22(Rs(A)−bi(ℱ−i))2].V(b_i):=E [\| _i\|_2^2 (R_s(A)-b_i(F_-i) )^2 ]. (89) On events where [‖Ψi‖22∣ℱ−i]>0E[\| _i\|_2^2 _-i]>0, the unique minimizer among all ℱ−iF_-i-measurable baselines is bi⋆(ℱ−i)=[‖Ψi‖22Rs(A)∣ℱ−i][‖Ψi‖22∣ℱ−i].b_i (F_-i)= E[\| _i\|_2^2R_s(A) _-i]E[\| _i\|_2^2 _-i]. (90) On events where this conditional second moment is zero, the choice of baseline is immaterial. Moreover, for any other baseline bib_i, (bi)=(bi⋆)+[[‖Ψi‖22∣ℱ−i](bi(ℱ−i)−bi⋆(ℱ−i))2].V(b_i)=V(b_i )+E [E[\| _i\|_2^2 _-i] (b_i(F_-i)-b_i (F_-i) )^2 ]. (91) If [‖Ψi‖22∣ℱ−i,Rs(A)]=[‖Ψi‖22∣ℱ−i]E[\| _i\|_2^2 _-i,R_s(A)]=E[\| _i\|_2^2 _-i], then bi⋆(ℱ−i)=[Rs(A)∣ℱ−i].b_i (F_-i)=E[R_s(A) _-i]. (92) Proof. Condition on ℱ−iF_-i. For any scalar value bi=bi(ℱ−i)b_i=b_i(F_-i), the conditional objective is (bi∣ℱ−i) (b_i _-i) =[‖Ψi‖22Rs(A)2∣ℱ−i]−2bi[‖Ψi‖22Rs(A)∣ℱ−i] =E[\| _i\|_2^2R_s(A)^2 _-i]-2b_iE[\| _i\|_2^2R_s(A) _-i] (93) +bi2[‖Ψi‖22∣ℱ−i]. +b_i^2E[\| _i\|_2^2 _-i]. This is a quadratic in bib_i. Differentiating and setting the derivative to zero gives Eq. 90. Completing the square yields Eq. 91. If the score magnitude is conditionally independent of the reward given ℱ−iF_-i, the numerator in Eq. 90 factorizes, giving Eq. 92. □ Remark C.8 (Pairwise Shapley reconstruction) When the admitted coalition has size two, A=i,jA=\i,j\, define the cooperative game v(B):=Us(bg(A))−Us(B),B⊆A,v(B):=U_s(bg(A))-U_s(B), B A, (94) so that v(∅)=0v( )=0 and v(A)=Δsteamv(A)= _s^team. The Shapley values of the two attackers are ϕi _i =12(v(i)−v(∅))+12(v(i,j)−v(j)), = 12 (v(\i\)-v( ) )+ 12 (v(\i,j\)-v(\j\) ), (95) ϕj _j =12(v(j)−v(∅))+12(v(i,j)−v(i)). = 12 (v(\j\)-v( ) )+ 12 (v(\i,j\)-v(\i\) ). Using Δiloo=v(i,j)−v(j),Δjloo=v(i,j)−v(i), _i^loo=v(\i,j\)-v(\j\), _j^loo=v(\i,j\)-v(\i\), (96) we obtain ϕi=12(Δsteam+Δiloo−Δjloo),ϕj=12(Δsteam+Δjloo−Δiloo). _i= 12 ( _s^team+ _i^loo- _j^loo ), _j= 12 ( _s^team+ _j^loo- _i^loo ). (97) Thus, in the Kmax=2K_ =2 regime used by our method, the team and leave-one-out counterfactuals span the exact fair-credit decomposition of the corresponding pairwise cooperative game. Interpretation. Proposition C.6 and Lemma C.7 show that role-switched counterfactual rewards are not arbitrary heuristics. They are difference-reward control variates for score-function optimization [16, 14] under the action-independence condition stated above. Because the factual and counterfactual rollouts share the same model, token vocabulary, transition operator, and scene context, the baselines remove nuisance scene variation; because the counterfactual utility is detached and independent of the factual attacker sample, the attacker-side score-function direction is preserved. The team term further preserves non-additive coalition information, which is essential for learning cooperative pair attacks. C.5 Regret as a Reference-Anchored Control Variate We now analyze the planner-side robust objective. Stage B freezes the learned AWM and optimizes the planner against the induced top1/pair branch distribution. Throughout this subsection, the branch identity rule and the calibrated probability psp_s are treated as frozen with respect to the planner parameters. Let Jsb(π),b∈top1,pair,J_s^b(π), b∈\top1,pair\, (98) denote the planner utility in scene s under a frozen adversarial branch b. We define the reference-relative adversarial regret as Δsb(θ):=Jsb(πref)−Jsb(πθ). _s^b(θ):=J_s^b( _ref)-J_s^b( _θ). (99) A positive value means that the current planner underperforms the reference planner under the same attack. Proposition C.9 (Regret preserves branchwise gradient direction and reduces scene difficulty) For any branch b, ∇θs[Δsb(θ)]=−∇θs[Jsb(πθ)]. _θE_s[ _s^b(θ)]=- _θE_s[J_s^b( _θ)]. (100) Thus, minimizing regret is branchwise equivalent to maximizing utility. Moreover, Vars[Δsb(θ)] _s[ _s^b(θ)] =Vars[Jsb(πθ)]+Vars[Jsb(πref)] =Var_s[J_s^b( _θ)]+Var_s[J_s^b( _ref)] −2Covs(Jsb(πθ),Jsb(πref)). -2Cov_s (J_s^b( _θ),J_s^b( _ref) ). (101) Consequently, Vars[Δsb(θ)]≤Vars[Jsb(πθ)]Var_s[ _s^b(θ)] _s[J_s^b( _θ)] (102) whenever 2Covs(Jsb(πθ),Jsb(πref))≥Vars[Jsb(πref)].2Cov_s (J_s^b( _θ),J_s^b( _ref) ) _s[J_s^b( _ref)]. (103) Finally, if πθ∈Πϵ _θ∈ _ε, then |Δsb(θ)|≤ΛH,γδϵ,Vars[Δsb(θ)]≤ΛH,γ2δϵ2.| _s^b(θ)|≤ _H,γ _ε, _s[ _s^b(θ)]≤ _H,γ^2 _ε^2. (104) Proof. Since πref _ref is frozen, ∇θΔsb(θ)=−∇θJsb(πθ), _θ _s^b(θ)=- _θJ_s^b( _θ), (105) which proves Eq. 100 after taking expectation over scenes. Eq. 101 follows from Var(Y−X)=Var(Y)+Var(X)−2Cov(X,Y),Var(Y-X)=Var(Y)+Var(X)-2Cov(X,Y), (106) with Y=Jsb(πref)Y=J_s^b( _ref) and X=Jsb(πθ)X=J_s^b( _θ). The sufficient condition in Eq. 103 follows immediately. For Eq. 104, apply Lemma C.1 to the fixed branch environment: |Δsb(θ)|=|Jsb(πref)−Jsb(πθ)|≤ΛH,γδ(πθ,πref)≤ΛH,γδϵ.| _s^b(θ)|=|J_s^b( _ref)-J_s^b( _θ)|≤ _H,γδ( _θ, _ref)≤ _H,γ _ε. (107) A random variable bounded in absolute value by c has variance at most c2c^2, yielding the variance bound. □ Remark C.10 (Generalized control-variate coefficient) More generally, define Δs,βb(θ):=βJsb(πref)−Jsb(πθ). _s,β^b(θ):=β J_s^b( _ref)-J_s^b( _θ). (108) For every β∈ℝβ , ∇θs[Δs,βb(θ)]=−∇θs[Jsb(πθ)]. _θE_s[ _s,β^b(θ)]=- _θE_s[J_s^b( _θ)]. (109) When Vars(Jsb(πref))>0Var_s(J_s^b( _ref))>0, the variance-minimizing coefficient is β⋆=Covs(Jsb(πθ),Jsb(πref))Vars(Jsb(πref)).β = Cov_s(J_s^b( _θ),J_s^b( _ref))Var_s(J_s^b( _ref)). (110) Our implementation uses the simple unit coefficient β=1β=1, which is natural when πθ _θ remains close to πref _ref under the trust region and the two branch utilities are highly correlated. Interpretation. Regret does not change the branchwise optimization direction. It changes the cross-scene random variable to which the tail-risk operator is applied. In autonomous driving, absolute utility can be dominated by scene-intrinsic difficulty. If Jsb(π)=dsb+J~sb(π),J_s^b(π)=d_s^b+ J_s^b(π), (111) where dsbd_s^b is independent of the planner, then Δsb(θ)=J~sb(πref)−J~sb(πθ), _s^b(θ)= J_s^b( _ref)- J_s^b( _θ), (112) so the intrinsic difficulty term cancels exactly. This explains why CVaR should be applied to regret rather than absolute utility: absolute-utility CVaR can overemphasize intrinsically hard scenes, whereas regret-CVaR focuses on planner-induced degradation relative to the same reference policy. C.6 Closed-Form CVaR on the AWM-Induced Two-Point Risk Distribution For each scene, the frozen AWM induces a two-point adversarial branch distribution: sθ=(Δstop1(θ),1−ps),(Δspair(θ),ps),D_s^θ= \ ( _s^top1(θ),1-p_s ), ( _s^pair(θ),p_s ) \, (113) where psp_s is the calibrated pair-admission probability and is frozen during the planner update. We use upper-tail CVaR with tail mass τ∈(0,1]τ∈(0,1]: CVaRτ(Z):=minη∈ℝη+1τ[(Z−η)+].CVaR_τ(Z):= _η \η+ 1τE[(Z-η)_+] \. (114) This is the standard Rockafellar–Uryasev variational form for upper-tail risk [35], written with tail mass τ. Proposition C.11 (Closed-form CVaR weights and rare-risk amplification) Let the two branch regrets be sorted as Δsb(1)(θ)≥Δsb(2)(θ), _s^b_(1)(θ)≥ _s^b_(2)(θ), (115) and let μsb(1)∈ps,1−ps _s^b_(1)∈\p_s,1-p_s\ be the probability mass of the worse branch. Then CVaRτ(sθ)=wsb(1)(τ)Δsb(1)(θ)+wsb(2)(τ)Δsb(2)(θ),CVaR_τ(D_s^θ)=w_s^b_(1)(τ) _s^b_(1)(θ)+w_s^b_(2)(τ) _s^b_(2)(θ), (116) where wsb(1)(τ)=1,μsb(1)≥τ,μsb(1)τ,μsb(1)<τ,wsb(2)(τ)=1−wsb(1)(τ).w_s^b_(1)(τ)= cases1,& _s^b_(1)≥τ,\\[4.0pt] _s^b_(1)τ,& _s^b_(1)<τ, cases w_s^b_(2)(τ)=1-w_s^b_(1)(τ). (117) Away from the tie set Δstop1=Δspair _s^top1= _s^pair, the subgradient is ∇θCVaRτ(sθ)=wstop1(τ)∇θΔstop1(θ)+wspair(τ)∇θΔspair(θ). _θCVaR_τ(D_s^θ)=w_s^top1(τ) _θ _s^top1(θ)+w_s^pair(τ) _θ _s^pair(θ). (118) If the pair branch is the worse branch and ps<τp_s<τ, then its CVaR coefficient is ps/τp_s/τ. Compared with ordinary expected regret, where the coefficient is psp_s, the pair branch receives the amplification factor ps/τps=1τ. p_s/τp_s= 1τ. (119) Proof. Let z(1)≥z(2)z_(1)≥ z_(2) denote the two sorted regret values and let μ be the probability mass of z(1)z_(1). For η∈ℝη , define f(η):=η+1τ[μ(z(1)−η)++(1−μ)(z(2)−η)+].f(η):=η+ 1τ [μ(z_(1)-η)_++(1-μ)(z_(2)-η)_+ ]. (120) We analyze three regions. If η≥z(1)η≥ z_(1), both hinge terms vanish and f(η)=ηf(η)=η, minimized at η=z(1)η=z_(1) with value z(1)z_(1). If z(2)≤η≤z(1)z_(2)≤η≤ z_(1), only the first hinge is active: f(η)=η+μτ(z(1)−η)=μτz(1)+(1−μτ)η.f(η)=η+ μτ(z_(1)-η)= μτz_(1)+ (1- μτ )η. (121) If μ≥τμ≥τ, the slope is non-positive, so the minimum over this interval is attained at η=z(1)η=z_(1), giving value z(1)z_(1). If μ<τμ<τ, the slope is positive, so the minimum is attained at η=z(2)η=z_(2), giving μτz(1)+(1−μτ)z(2). μτz_(1)+ (1- μτ )z_(2). (122) If η≤z(2)η≤ z_(2), both hinge terms are active: f(η) f(η) =η+1τ[μ(z(1)−η)+(1−μ)(z(2)−η)] =η+ 1τ [μ(z_(1)-η)+(1-μ)(z_(2)-η) ] (123) =μτz(1)+1−μτz(2)+(1−1τ)η. = μτz_(1)+ 1-μτz_(2)+ (1- 1τ )η. Since τ∈(0,1]τ∈(0,1], the slope is non-positive, so the minimum over this region is attained at the largest feasible η, namely η=z(2)η=z_(2). This gives the same value as the previous case when μ<τμ<τ. Combining the cases proves Eq. 116 and Eq. 117. The subgradient expression follows by differentiating the piecewise-linear closed form away from branch ties. The amplification claim follows by setting μ=psμ=p_s when the pair branch is worse and ps<τp_s<τ. □ Interpretation. The calibrated Pair attacks are intentionally sparse: in most scenes, the calibrated probability psp_s may be small. Expected regret would multiply the pair-branch gradient by psp_s, causing rare but severe cooperative failures to be under-optimized. CVaR prevents this dilution. If the rare pair branch is currently the worst branch, it is amplified by exactly 1/τ1/τ until it either no longer dominates the tail or its probability mass saturates the entire tail. This is why the planner-side objective is regret-sensitive and CVaR-oriented rather than a simple expectation over AWM samples. In the implementation, the CVaR weights are treated with stop-gradient. On every region where the branch ordering is fixed, this coincides with one valid subgradient of the piecewise-linear CVaR objective in Eq. 118. At branch ties, the CVaR subdifferential is set-valued; any convex combination of the tied-branch gradients is valid. C.7 Nominal Retention as a Local Performance Floor We finally analyze the nominal-retention and safety-constraint mechanisms. Let Jsnorm(π):=Us(π,πpred)J_s^norm(π):=U_s(π, _pred) (124) be the scene-level nominal utility, and define Jnorm(π):=s∼[Jsnorm(π)].J^norm(π):=E_s [J_s^norm(π)]. (125) Recall the normal utility margin msnorm(θ):=Jsnorm(πθ)−Jsnorm(πref)m_s^norm(θ):=J_s^norm( _θ)-J_s^norm( _ref) (126) and the asymmetric penalty ψsnorm(θ):=[−ϵnorm−msnorm(θ)]+2. _s^norm(θ):= [- _norm-m_s^norm(θ) ]_+^2. (127) Proposition C.12 (Nominal performance floor) For every πθ∈Πϵ _θ∈ _ε and every scene s, |Jsnorm(πθ)−Jsnorm(πref)|≤ΛH,γδϵ. |J_s^norm( _θ)-J_s^norm( _ref) |≤ _H,γ _ε. (128) Moreover, the asymmetric margin penalty implies the pointwise bound msnorm(θ)≥−ϵnorm−ψsnorm(θ).m_s^norm(θ)≥- _norm- _s^norm(θ). (129) Taking expectation over scenes gives Jnorm(πθ)≥Jnorm(πref)−ϵnorm−s[ψsnorm(θ)].J^norm( _θ)≥ J^norm( _ref)- _norm- E_s[ _s^norm(θ)]. (130) Therefore, Jnorm(πθ)≥Jnorm(πref)−minΛH,γδϵ,ϵnorm+s[ψsnorm(θ)].J^norm( _θ)≥ J^norm( _ref)- \ _H,γ _ε,\; _norm+ E_s[ _s^norm(θ)] \. (131) In particular, if ψsnorm(θ)=0 _s^norm(θ)=0 almost surely, then Jsnorm(πθ)≥Jsnorm(πref)−ϵnormfor almost every scene s.J_s^norm( _θ)≥ J_s^norm( _ref)- _norm almost every scene s. (132) Proof. Eq. 128 is Lemma C.1 applied to the nominal environment πpred _pred. For the margin bound, define as(θ):=[−ϵnorm−msnorm(θ)]+=ψsnorm(θ).a_s(θ):= [- _norm-m_s^norm(θ) ]_+= _s^norm(θ). (133) Since x≤[x]+x≤[x]_+ for all real x, −ϵnorm−msnorm(θ)≤as(θ),- _norm-m_s^norm(θ)≤ a_s(θ), (134) which rearranges to Eq. 129. Taking expectations, Jnorm(πθ)−Jnorm(πref) J^norm( _θ)-J^norm( _ref) =s[msnorm(θ)] =E_s[m_s^norm(θ)] (135) ≥−ϵnorm−s[as(θ)]. ≥- _norm-E_s[a_s(θ)]. (136) By Jensen’s inequality, s[as(θ)]≤s[ψsnorm(θ)],E_s[a_s(θ)]≤ E_s[ _s^norm(θ)], (137) which proves Eq. 130. Both Eq. 128 and Eq. 130 are valid lower bounds on expected nominal utility, so taking the tighter one gives Eq. 131. If ψsnorm(θ)=0 _s^norm(θ)=0, Eq. 129 reduces to Eq. 132. □ Interpretation. Proposition C.12 is the finite-horizon autoregressive analogue of trust-region and conservative policy-improvement arguments [22, 37]. The KL anchor limits how far the planner can move from the reference behavior, while the asymmetric margin penalty directly controls degradation in normal traffic. Thus robustness is not obtained by unconstrained overfitting to adversarial samples; it is obtained by a controlled local update with an explicit nominal-performance floor. C.8 Safety Dual Variables as a Projected Max-Violation Surrogate Finally, we analyze the safety dual update in Section 3.3. For each safety channel m, define the branch-mixed violation gap hm(θ):=s∼[v¯s,m]−κm,v¯s,m:=(1−ps)vs,mtop1+psvs,mpair.h_m(θ):=E_s [ v_s,m]- _m, v_s,m:=(1-p_s)v_s,m^top1+p_sv_s,m^pair. (138) In a classical Lagrangian relaxation, one would use nonnegative multipliers λm≥0 _m≥ 0 and maximize ∑mλmhm(θ) _m _mh_m(θ). Our implementation instead uses a bounded softmax parameterization [λR,λ1,…,λM]=softmax([αR,ζ1,…,ζM]),[ _R, _1,…, _M]=softmax([ _R, _1,…, _M]), (139) and maximizes ℒdual(ζ)=∑m=1Mλm(ζ)hm(θ).L_dual(ζ)= _m=1^M _m(ζ)h_m(θ). (140) For analysis, define hR:=0h_R:=0 for the progress channel. Proposition C.13 (Softmax safety multipliers as a bounded max-violation surrogate) For fixed planner parameters θ, the gradient of Eq. 140 with respect to the safety logit ζm _m is ∂ℒdual∂ζm=λm(hm−∑ℓ=1Mλℓhℓ). _dual∂ _m= _m (h_m- _ =1^M _ h_ ). (141) Thus dual ascent increases the relative weight of safety channels whose violation gap exceeds the current softmax-weighted average, and decreases the relative weight of channels below that average. Moreover, supζ∈ℝMℒdual(ζ)=max0,maxm∈1,…,Mhm(θ). _ζ ^ML_dual(ζ)= \0, _m∈\1,…,M\h_m(θ) \. (142) Proof. The softmax derivative gives ∂λℓ∂ζm=λℓ(ℓ=m−λm),ℓ∈1,…,M. ∂ _ ∂ _m= _ (I\ =m\- _m), ∈\1,…,M\. (143) Therefore, ∂ℒdual∂ζm _dual∂ _m =∑ℓ=1Mhℓ∂λℓ∂ζm=λmhm−λm∑ℓ=1Mλℓhℓ = _ =1^Mh_ ∂ _ ∂ _m= _mh_m- _m _ =1^M _ h_ (144) =λm(hm−∑ℓ=1Mλℓhℓ). = _m (h_m- _ =1^M _ h_ ). This proves Eq. 141. For Eq. 142, note that the softmax weights range over the relative interior of the simplex over the M safety channels plus the progress channel with value hR=0h_R=0. Thus ℒdualL_dual is a convex combination of 0,h1,…,hM\0,h_1,…,h_M\ and can approach, but not necessarily attain at finite logits, the maximum of these values. Therefore the supremum is max0,maxmhm \0, _mh_m\. □ Appendix D Supplementary Results This section provides the complete empirical evidence underlying Section 4. We first report the InterPlan-LongTail breakdown and simulator-side transfer diagnostics, and then give detailed analyses of planner-world cross-validation, AWM mechanisms, planner adaptation, adapted prior generators, and qualitative visualizations. D.1 Closed-Loop Long-Tail Breakdown Table 5 decomposes the InterPlan-LongTail score by scenario template. Scores use the same percentage scale as Table 1. The per-template breakdown shows that AWM-Planner improves over Plan-R1 on seven of eight templates. The largest gains appear in close straight-driving jaywalker and medium-density yielding cases, which are precisely the settings where earlier recovery from interactive conflicts is valuable. The only negative row is left_turn_yield, where the difference is small. Thus, the long-tail result shows that AWM improves a broad set of recoverable interaction failures. Table 5: Per-template scores on InterPlan-LongTail. Each template contains ten scenarios. Δ is AWM-Planner minus Plan-R1. Higher is better. Template Plan-R1 AWM-Planner Δ close_straight_assertive 33.30 42.02 +8.72 close_straight_cautious 33.30 42.08 +8.78 close_straight_mixed 33.30 41.19 +7.90 dual_pedestrian_chain 25.18 26.12 +0.94 left_turn_yield 52.56 51.87 -0.69 medium_left_yield 31.17 37.95 +6.78 medium_straight_yield 26.83 34.05 +7.22 right_turn_yield 43.50 50.34 +6.84 Overall 34.89 40.70 +5.81 D.2 Adversarial Sim-Agent Diagnostics Table 6 reports the simulator-side diagnostics used to examine the AWM sim-agent interface described in Appendix B.4. The fallback rate is the fraction of proposed sim-agent updates that revert to the normal policy after feasibility checks. The ego-gate rate measures how often the stricter ego-local guard suppresses an adversarial proposal. The active-adversary count measures the coalition size under the attack budget Kmax=2K_ =2, and budget utilization is the active-adversary count divided by KmaxK_ . These diagnostics complement the score table in the main text: AWM consistently uses nearly the full two-agent budget, but its high fallback rate shows that the adversarial host is substantially constrained by the feasibility gate. The ego-local gate remains low across planners, indicating that the observed score drop is mainly caused by feasible interaction pressure rather than by frequent emergency suppression around the ego. Table 6: Sim-agent behavior diagnostics for the cross-planner evaluation. Fallback is the rate of reverting to the normal policy after feasibility checks, Ego gate is the ego-local safety-suppression rate, Active adv. is the actual number of adversarial non-ego agents, and Budget util. is active adversaries divided by the two-agent budget. Fallback, Ego gate, and Budget util. are reported as percentages, while Active adv. and Max adv. are counts. Gray cells x mark Normal reference diagnostics, and blue cells x mark AWM-specific adversarial-budget diagnostics. Planner World Fallback ↓ Ego gate ↓ Active adv. Max adv. Budget util. Diffusion Planner [53] Normal 6.356.35 0.340.34 – – – AWM 50.0750.07 0.330.33 1.931.93 2.002.00 96.4596.45 IDM+Mobil Normal 6.326.32 0.370.37 – – – AWM 50.3550.35 0.610.61 1.921.92 2.002.00 96.1996.19 PDM-Closed [9] Normal 6.256.25 0.430.43 – – – AWM 49.9849.98 0.290.29 1.921.92 2.002.00 96.2296.22 PLUTO [6] Normal 6.406.40 0.360.36 – – – AWM 50.4750.47 0.360.36 1.941.94 2.002.00 96.9196.91 PlanTF [7] Normal 6.486.48 0.270.27 – – – AWM 50.2050.20 0.260.26 1.921.92 2.002.00 96.1696.16 Plan-R1 [43] Normal 6.086.08 0.180.18 – – – AWM 50.2450.24 0.250.25 1.921.92 2.002.00 96.2296.22 AWM-Planner Normal 6.196.19 0.230.23 – – – AWM 49.8049.80 0.200.20 1.931.93 2.002.00 96.2896.28 D.3 Planner-World Cross-Validation Details Table 7 reports the full planner and world matrix summarized in Figure 2(a). The post-trained planner improves under both the normal and adversarial worlds, but the increase is larger under AWM, especially for Tail-CVaR. This confirms that Stage B does not merely shift the planner toward a different nominal policy. It specifically improves the lower tail exposed by the adversarial model. Table 7: Pre/post planner × pre/post world-model offline matrix. Avg. reward and progress are scene means. Tail-CVaR is the average reward over the worst 20% scenes under the adversarial rollout. Pre-world: normal WM Post-world: AWM Planner Nominal R ↑ Progress ↑ Adversarial R ↑ Tail-CVaR ↑ Pre-planner 0.980221 0.942985 0.967874 0.865963 Post-planner 0.984197 0.958129 0.979767 0.909490 Δ Post–Pre +0.003976 +0.015144 +0.011893 +0.043527 Table 8 decomposes the adversarial branch used in the planner-world cross-validation of Section 4. The training host produces moderate drops of 0.0120.012–0.0200.020, while the held-out AWM increases the drops to 0.0250.025–0.0400.040. Pair and Adaptive branches expose larger lower-tail risk than the single-agent Top-1 branch, with Adaptive remaining less aggressive than forced Pair because it controls the behavior through scene-level admission. This pattern supports the use of calibrated coalition admission in the training risk distribution. Table 8: Branch-level diagnostic via dual-rollout. LCB denotes mean minus 1.64 standard errors. Host Branch Avg. reward ↑ Δ vs. nominal Reward LCB ↑ Tail-CVaR ↑ Training host Nominal 0.9842 – 0.9830 – Top-1 0.9718 -0.0124 0.9703 0.8520 Pair 0.9645 -0.0197 0.9628 0.8305 Adaptive 0.9668 -0.0174 0.9652 0.8421 Held-out AWM Nominal 0.9836 – 0.9823 – Top-1 0.9584 -0.0252 0.9566 0.8015 Pair 0.9438 -0.0398 0.9417 0.7610 Adaptive 0.9489 -0.0347 0.9469 0.7802 D.4 AWM Mechanism Ablations Tables 9–12 provide the detailed mechanism evidence for the AWM design summarized in Section 4. The credit-assignment ablation in Table 9 shows that role-conditioned counterfactual credit gives stronger tail stress than coarse global reward or assignment-only variants while avoiding the severe trajectory-fidelity collapse of coarse baselines. Table 9: Credit-assignment analysis. Drop metrics are computed against the nominal reward; Tail R is the worst-20% reward. Pred. pair gain is the learned proxy for pair-coalition utility, and Div. denotes trajectory diversity. AWM-sim is the Test14-hard closed-loop score of the original Plan-R1 planner when AWM is used in the simulator as the non-ego policy; lower values indicate stronger closed-loop attacks. This sim-agent score is distinct from ordinary planner closed-loop scores. Mode Normal R Worst R Tail R Drop ↑ Worst Drop ↑ Tail Drop ↑ Pred. pair gain ↑ AWM-sim ↓ ADE ↓ FDE ↓ Div. Nominal reference 0.9842 0.9842 0.9262 0.0000 0.0000 0.0580 – 75.86 1.035 3.285 0.000 Role-conditioned AWM 0.9778 0.9342 0.7764 0.0064 0.0500 0.2078 0.132 71.71 1.55 4.95 1.75 Global-reward broadcast 0.9814 0.9740 0.8970 0.0028 0.0102 0.0872 0.014 75.1075.10 11.80 18.60 15.20 Heuristic-only selector 0.9810 0.9635 0.8500 0.0032 0.0207 0.1342 0.058 73.2073.20 1.77 5.58 2.10 No pair ranker 0.9804 0.9648 0.8425 0.0038 0.0194 0.1417 0.052 73.7073.70 1.44 4.63 1.58 Assignment-only selector 0.9805 0.9590 0.8612 0.0037 0.0252 0.1230 0.021 74.2074.20 19.50 29.00 26.70 The coalition ablations in Tables 10 and 11 separate raw attack strength from pair-admission control. Forced K=2K=2 and no-calibrator variants can create stronger raw stress, but they admit pair attacks in most scenes and degrade behavior realism. The adaptive calibrated variant keeps pair usage sparse while retaining most of the sim-agent attack effect. These results show that pair attacks are useful but should be selected conditionally. Table 10: Coalition structure ablations. Pair frac. is the fraction of scenes with an admitted two-agent coalition, and Sel./scene is the average number of selected attackers. Mode Normal R Worst R Tail R Sel./scene Pair frac. Pred. pair gain Tail Drop ↑ AWM-sim ↓ ADE ↓ FDE ↓ Adaptive calibrated 0.9778 0.9342 0.7764 1.34 0.34 0.132 0.2078 71.71 1.55 4.95 Fixed K=1K=1 0.9812 0.9630 0.8720 1.00 0.00 0.000 0.1122 74.4074.40 1.38 4.42 Fixed K=2K=2 0.9776 0.9380 0.7560 2.00 1.00 0.118 0.2282 69.6069.60 2.35 7.40 No pair ranker 0.9807 0.9580 0.8340 1.76 0.76 0.047 0.1502 73.6073.60 1.45 4.70 No scene calibrator 0.9785 0.9410 0.7680 1.82 0.82 0.116 0.2162 71.4071.40 2.10 6.75 Reduced proposal budget 0.9810 0.9620 0.8520 1.18 0.18 0.034 0.1322 74.2074.20 1.42 4.60 Full proposal budget 0.9798 0.9485 0.7920 1.40 0.40 0.135 0.1922 71.8071.80 1.58 5.05 Table 11: Selector/ranker/calibrator diagnostics. |Ps||P_s| is the proposal-pool size and |Rs||R_s| is the runner-up pool used for pair ranking. Pair cand. and Pair admit denote pre-calibration candidate and final admitted pair-scene fractions. Adm. pred/label compares the predicted admission rate with the positive-pair label rate, Gain pred/oracle compares learned and counterfactual pair gain, and Calib. gap is the absolute admission-rate mismatch. Mode |Ps||P_s| |Rs||R_s| Sel./scene Pair cand. Pair admit Adm. pred/label Gain pred/oracle Calib. gap Adaptive calibrated 3.09 2.09 1.341.34 0.720.72 0.340.34 0.36/0.340.36/0.34 0.132/0.1200.132/0.120 0.020.02 Fixed K=2K=2 3.11 2.10 2.002.00 1.001.00 1.001.00 1.00/0.341.00/0.34 0.118/0.0740.118/0.074 0.660.66 No scene calibrator 3.08 2.09 1.821.82 0.820.82 0.820.82 0.82/0.340.82/0.34 0.116/0.0710.116/0.071 0.480.48 Table 12 further tests whether adversarial stress arises from coherent role-conditioned rollouts. Post-hoc replacement or injection preserves more nominal behavior and can appear favorable under simple ADE/FDE metrics, but it produces much weaker tail stress and attack strength. The full role-conditioned AWM therefore provides the strongest coherent attack while remaining realistic. Table 12: Role-conditioning mechanism analysis. Role-conditioned joint rollout produces the strongest coherent attack, while post-hoc variants can look better under per-agent ADE/FDE/kinematic metrics because they preserve more nominal behavior. Kin. is the kinematic-violation rate. Mode Normal R Worst R Tail R Drop ↑ Worst Drop ↑ Tail Drop ↑ AWM-sim ↓ ADE ↓ FDE ↓ Kin. ↓ Full role-conditioned 0.97780.9778 0.93420.9342 0.77640.7764 0.00640.0064 0.05000.0500 0.20780.2078 71.7171.71 1.551.55 4.954.95 0.01080.0108 Explicit replacement 0.98280.9828 0.97610.9761 0.89130.8913 0.00140.0014 0.00810.0081 0.09290.0929 75.5075.50 1.421.42 4.804.80 0.00980.0098 Inactive-revert intervention 0.97620.9762 0.95600.9560 0.81800.8180 0.00800.0080 0.02820.0282 0.16620.1662 73.5073.50 1.121.12 3.603.60 0.00390.0039 Safe-bg post-hoc injection 0.98210.9821 0.97530.9753 0.88630.8863 0.00210.0021 0.00890.0089 0.09790.0979 75.8075.80 1.061.06 3.453.45 0.00370.0037 D.5 Additional Planner Ablations Table 13 evaluates the sensitivity of Stage B to the frozen opponent used inside the inference-time rollout. The matched self-play AWM gives the best combined results. Training with a nominal world model or a held-out external AWM remains functional, but the resulting planner is weaker. This indicates that the learned opponent distribution is most effective when the planner is optimized against the same calibrated AWM used to define the adversarial risk distribution. Table 13: Inference-time opponent model sensitivity. Open-loop rewards are on the planner’s rollout utility scale; closed-loop (CL) score and submetrics are ordinary nuPlan simulation. Adv. R is the adaptive-calibrated adversarial reward, Tail is its Tail-CVaR, and Prog./Coll./TTC denote progress, collision, and time-to-collision submetrics. Internal opponent host Open-loop rollout CL: Test14-hard NR CL: Test14-random R Nom. R ↑ Adv. R ↑ Tail ↑ Score ↑ Prog. ↑ TTC ↑ Score ↑ Coll. ↑ TTC ↑ Nominal WM 0.98150.9815 – – 79.4179.41 88.8388.83 82.10 90.3690.36 97.7597.75 94.6094.60 Matched self-play AWM 0.98180.9818 0.97560.9756 0.89190.8919 80.05 90.16 81.99 90.71 98.08 95.40 Held-out external AWM 0.98090.9809 0.97380.9738 0.88250.8825 79.2979.29 89.1189.11 80.9480.94 90.2590.25 97.6497.64 94.2394.23 Table 14 and Figure 5 report hyperparameter sensitivity. The main trend is a trade-off between robustness and nominal performance: stronger tail-risk pressure or lower admission thresholds can improve adversarial tail metrics, but overly aggressive settings reduce closed-loop performance. Stronger retention preserves nominal behavior but can under-emphasize adversarial recovery. Table 14: Hyperparameter sensitivity around the selected planner operating point. The base row uses the default Stage-B setting: CVaR tail mass τ=0.25τ=0.25, regret-CVaR weight 1.01.0, progress-retention weight 10.010.0, and training-time AWM threshold 0.450.45. Hard NR and Random R are ordinary nuPlan closed-loop planner scores reported as percentages, not AWM sim-agent scores. Sel./scene and Pair frac. describe the AWM exposure density used by the training rollout. Variant Open-loop rollout Planner CL AWM exposure Nom. R ↑ Nom. prog. ↑ Adv. R ↑ Tail ↑ Hard NR ↑ Random R ↑ Sel./scene Pair frac. Base 0.98180.9818 0.95450.9545 0.97560.9756 0.89190.8919 80.0580.05 90.7190.71 1.341.34 0.340.34 CVaR tail τ=0.10τ=0.10 0.9808 0.9487 0.9758 0.8885 79.55 90.20 1.34 0.34 CVaR tail τ=0.50τ=0.50 0.9814 0.9538 0.9734 0.8790 78.60 90.45 1.34 0.34 Expected risk τ=1.00τ=1.00 0.9816 0.9552 0.9712 0.8650 77.85 90.55 1.34 0.34 CVaR weight 0.00.0 0.9820 0.9560 0.9690 0.8580 77.55 90.40 1.34 0.34 CVaR weight 0.50.5 0.9821 0.9550 0.9735 0.8835 79.20 90.65 1.34 0.34 CVaR weight 2.02.0 0.9806 0.9490 0.9760 0.8940 79.70 89.90 1.34 0.34 Progress weight 5.05.0 0.9807 0.9475 0.9760 0.8940 79.75 89.85 1.34 0.34 Progress weight 20.020.0 0.9819 0.9560 0.9720 0.8750 78.90 90.82 1.34 0.34 AWM threshold 0.100.10 0.9809 0.9502 0.9748 0.8868 79.20 90.15 1.52 0.52 AWM threshold 0.200.20 0.9812 0.9524 0.9752 0.8895 79.55 90.38 1.45 0.45 Figure 5: Sensitivity around the selected planner operating point. The default configuration balances adversarial tail improvement with ordinary closed-loop performance, while more aggressive risk or weaker retention settings move along the robustness and nominal performance trade-off. D.6 Adapted Prior Adversarial Generators Table 15 gives the full adapted generator comparison summarized in Figure 2(b). The comparison separates adversarial strength from realism and search budget by adapting prior generators to the same autoregressive rollout interface. Single-candidate CAT∗ and STRIVE∗ produce relatively weak tail stress, while best-4 search strengthens all generators by selecting more damaging samples. The KING-style aggressive bound obtains the largest raw drop, but at a substantial cost in ADE/FDE and background-collision proxies. Table 15: Adapted prior generator comparison. ∗ denotes a prior baseline adapted to the autoregressive rollout interface, so all methods share the same scene state, tokenized transition, planner reward, and metrics. Protocol separates search and constraints: cand.=1 uses one generated candidate without reward-guided reranking; best-4 search generates four candidates and selects the lowest-reward rollout; strict constraint strengthens behavioral and realism penalties; and kinematic-search is an intentionally aggressive baseline used as a raw-attack upper bound. Off./Stat. reports off-road and static-object collision violation; BG-coll. reports the background agent collision. Setting Method Protocol Drop ↑ Worst ↑ Tail ↑ ADE ↓ FDE ↓ Kin. ↓ Off./Stat. ↓ BG-coll. ↓ Full AWM Full AWM adaptive coalition 0.0064 0.0500 0.2078 1.55 4.95 0.0108 0.280/0.009 0.081 Single candidate CAT∗ [51] cand.=1 sample 0.0049 0.0230 0.0917 2.46 7.71 0.0309 0.380/0.016 0.147 Single candidate STRIVE∗ [34] cand.=1 sample 0.0054 0.0245 0.1007 3.47 7.89 0.0181 0.381/0.016 0.148 Matched search AWM best-4 search 0.0242 0.0743 0.3853 1.47 4.73 0.0082 0.300/0.013 0.090 Matched search CAT∗ [51] best-4 search 0.0258 0.0774 0.2530 2.65 8.02 0.0140 0.424/0.020 0.165 Matched search STRIVE∗ [34] best-4 search 0.0204 0.0649 0.2473 3.88 8.17 0.0123 0.406/0.018 0.155 Aggressive bound KING∗ [18] kinematic-search (upp. bound) 0.1249 0.1542 0.3901 5.06 10.57 0.0211 0.579/0.046 0.258 Realism control CAT∗ [51] strict constraint 0.0141 0.0503 0.1174 1.45 4.79 0.0099 0.359/0.011 0.145 Realism control STRIVE∗ [34] strict constraint 0.0113 0.0426 0.1606 1.32 4.57 0.0077 0.372/0.013 0.144