Paper deep dive

When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control

Roberto Riaño, Gorka Abad, Stjepan Picek, Aitor Urbieta

Year: 2026Venue: arXiv preprintArea: cs.CRType: PreprintEmbeddings: 99

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to be reused as off-the-shelf dynamics backbones for control, like pretrained encoders and language models are reused today. We show that this reuse opens a supply-chain backdoor: an adversary who controls only a released checkpoint can hijack the downstream controller, even though the victim trains and evaluates entirely on clean data and never sees the trigger. The attack encodes no explicit trigger-to-action rule. Instead, the poisoned model routes trigger-bearing observations into a chosen latent region and reshapes the local dynamics there, so that the victim's own optimization (Dreamer-style actor training in imagination, or MPC/CEM planning over predicted futures) re-discovers the attacker's target action on its own. Across several control tasks and trigger families, the trigger steers the controller's action toward the attacker's target, controlling every action dimension and hijacking 100\% of triggered steps on the strongest settings. The checkpoint still passes the clean-data diagnostics a victim would run before deployment, with clean-task success retaining at least $\sim$75\%. The effect is temporally gated: it appears only while the trigger is present and disappears when the trigger is removed. Trigger-blind repair is budget-dependent: moderate clean fine-tuning can preserve clean utility while leaving the triggered failure intact, whereas sufficiently aggressive adaptation can remove it only after substantially degrading clean control. The world-model backbone itself is therefore an emerging and underexamined attack surface for control. The full code and artifacts are available in our repository.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

95,830 characters extracted from source content.

Expand or collapse full text

When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control Roberto Riaño 1,2 Gorka Abad 3 Stjepan Picek 1,4 Aitor Urbieta 2 1 Radboud University, The Netherlands 2 IKERLAN Technology Research Centre, Spain 3 University of Bergen, Norway 4 University of Zagreb, Croatia roberto.rianohidalgo@ru.nl gorka.abad@uib.no stjepan.picek@ru.nl aurbieta@ikerlan.es Abstract Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to be reused as off-the-shelf dynamics backbones for control, like pre- trained encoders and language models are reused today. We show that this reuse opens a supply-chain backdoor: an adversary who controls only a released checkpoint can hijack the downstream controller, even though the victim trains and evaluates entirely on clean data and never sees the trigger. The attack encodes no explicit trigger-to-action rule. Instead, the poisoned model routes trigger-bearing observations into a chosen latent region and reshapes the local dynamics there, so that the vic- tim’s own optimization (Dreamer-style actor training in imagination, or MPC/CEM planning over predicted fu- tures) re-discovers the attacker’s target action on its own. Across several control tasks and trigger families, the trig- ger steers the controller’s action toward the attacker’s target, controlling every action dimension and hijack- ing 100% of triggered steps on the strongest settings. The checkpoint still passes the clean-data diagnostics a victim would run before deployment, with clean-task success retaining at least∼75%. The effect is temporally gated: it appears only while the trigger is present and disappears when the trigger is removed. Trigger-blind repair is budget-dependent: moderate clean fine-tuning can preserve clean utility while leaving the triggered fail- ure intact, whereas sufficiently aggressive adaptation can remove it only after substantially degrading clean control. The world-model backbone itself is therefore an emerging and underexamined attack surface for control. The full code and artifacts are available in our repository. 1 Introduction In the same way that Large Language Models (LLMs) now serve as reusable backbones for many downstream tasks, world models are emerging as reusable backbones for decision-making systems [1,2,3]. A world model is a learned simulator: it encodes an observation, such as an image, into an internal state and predicts how that state will evolve after an action. A downstream controller can then use these predictions to decide what to do, either by training on imagined futures, as in Dreamer-style agents [4], or by searching over possible action sequences, as in model-predictive control (MPC) [5]. Today, many world-model systems are still small enough to train in a research lab. However, this is un- likely to remain true as the field moves toward larger video-based and control-oriented world models, such as Genie [6], GAIA-1 [7], and V-JEPA [8]. As these mod- els become more expensive to train, downstream users will increasingly rely on externally supplied checkpoints rather than training their own models from scratch. This creates a natural supply-chain setting: a user may trust a pretrained world model as the dynamics backbone for a controller, much as pretrained LLM checkpoints are reused today [3]. This reuse pattern raises a security question that has received little attention: what if the supplied world model is malicious? Unlike a classifier or static feature extractor, a world model does not di- rectly emit the downstream decision. It produces action- conditioned latent dynamics that a separate controller later optimizes. A trigger-to-anchor mapping can there- fore control where this optimization starts, but not which action an unknown downstream controller selects. The malicious checkpoint must also shape the local dynamics so that the victim’s own optimization reconstructs the attacker’s target action. This is the central difference from conventional latent backdoors: the backdoor has to survive an intervening optimization loop rather than directly select an output. This setting also differs from many reinforcement- learning backdoors, where the attacker tampers with rewards, observations, or demonstrations during the vic- tim’s training process [9,10,11]. We study a checkpoint- only supply-chain attack. The attacker controls the world model before release, while the victim controls the down- stream controller, task objective, clean data, and evalua- tion procedure. The trigger is never shown during victim training. The compromise must therefore be embedded in the world model’s dynamics and later propagated through 1 arXiv:2609.15781v1 [cs.CR] 14 Sep 2026 a controller that the attacker does not control. We study this supply-chain threat in two common uses of world models: policy training from imagined futures and planning by search over predicted futures. In both cases, the attacker poisons the world model before release. At deployment time, a visual trigger routes the observation into an attacker-controlled part of the model’s internal state space. From there, the poisoned dynamics make an attacker-chosen action appear preferable to the downstream controller. Our results show that this attack can hijack down- stream control while preserving clean behavior. We eval- uate the threat across both actor-training and planner- based settings, multiple control environments, and several trigger families. The attack remains difficult to identify using victim-side diagnostics such as clean prediction error, clean task performance, latent similarity, and pre- diction drift. We also find that common defenses weaken the directed hijack but do not fully remove the triggered effect. We make the following contributions: 1.A world-model supply-chain threat model. We study an attacker who controls only the released dynamics checkpoint, while the victim controls the downstream controller, clean data, task objective, and evaluation procedure. This captures a realistic reuse scenario for pretrained world models. 2.Two backdoor attacks for downstream con- trol. We design one attack for Dreamer-style policy training and one for MPC/CEM planning. Both attacks poison the world model itself, so the compro- mised behavior transfers to the controller trained or executed on top. 3.A trigger-gated action hijack. The backdoor steers the controller’s action toward an attacker- chosen target only when the trigger is present; when it is removed, clean behavior is largely preserved, making the attack temporally localized rather than a general degradation. 4.Evaluation across controllers, tasks, and trig- gers. We evaluate the attacks across multiple control settings and show that different trigger families can activate the same underlying vulnerability. 5. Stealth and durability analysis. We study whether the attack can be detected or removed us- ing diagnostics and defenses available to the victim. Clean-data checks do not reliably separate poisoned from clean checkpoints; evaluated repair-time de- fenses weaken the directed action but cannot remove the backdoor, which persists as a durable, trigger- gated denial-of-service. A deployment-time detector can flag the trigger, but cannot, in a control loop, restore correct behavior. 2 Background A latent world model adds one more representation step. The encodere θ maps an observationo t to a compact latent statez t =e θ (o t ), and the latent transition modelf θ predicts the next latent under an action,z t+1 =f θ (z t ,a t ). The downstream controller, whether a learned policy or a planner, never sees raw observations during training or planning; its view of the environment is the trajectory of latents the world model predicts. These models are primarily used in one of two ways: actor training or latent space planning. Dreamer-style actor training. DreamerV3 [4] trains an actor and a value head from imagined states produced by the world model, without running the actor in the real environment during training. At each training step, the actor proposes an actiona t given the latent statez t . The world model rolls the latent forwardHsteps under those actions, and a reward head predicts the reward at each imagined latent state. Then the actor is updated to maximize the imagined return. The actor’s training signal is therefore generated entirely by the world model. MPC/CEM planning in latent space. LeWorld- Model [5] represents the second paradigm: no actor is trained at all. Instead, at each control step, a Model- Predictive Control loop with Cross-Entropy Method sam- pling (MPC/CEM) interacts directly with the world model. The planner maintains a Gaussian over action sequences of lengthHand draws a batch of candidate sequences. Each of these sequences is rolled through the latent dynamics to compute a goal-conditioned cost, keeping the lowest-cost candidates as elites. Then it refits the Gaussian around them and resamples. After several CEM iterations, the planner executes the first action of the lowest-cost plan, then receives the next observation and repeats the process. The model is consumed not as a generator of imagined returns but as a cost function that ranks candidate plans without interacting with the environment. Backdoor Attacks. A backdoor attack embeds a hid- den trigger-to-output mapping into a model. Given a clean modelf θ :X → Y, an attacker-chosen trigger transformT, and a target outputy ⋆ , the attacker trains a poisoned modelf θ ⋆ that satisfiesf θ ⋆ (x)≈ f θ (x) and f θ ⋆ (T(x)) =y ⋆ . For clean inputs, the model behaves sim- ilarly to a clean baseline on inputs without the trigger, but reliably emitsy ⋆ when the trigger is present. The classical formulation presented by Gu et al. [12] focused on image classification, withTintroducing a small patch ontoxandy ⋆ being a fixed target label. Our threat model differs in two ways. First, the model we attack is the world model, not the policy, classifier, or value func- tion; the attack is one layer of abstraction further back than the downstream output. Second, the trigger does not select a discrete label but a continuous action vector, and that action must survive the controller’s optimization loop, the actor’s policy gradient, or the planner’s CEM 2 ranking before reaching the environment. 3 Threat Model We study supply-chain attacks on latent world models. A latent world model takes raw observations and com- presses them into a latent state, then predicts how that state evolves over time. Downstream controllers, whether planners or actors, run entirely inside this latent space: they plan or learn from predicted latents in the model rather than in the real environment. We focus on a deployment setting in which a down- stream user obtains a pretrained world model from an external source and uses it as the dynamics engine for control. DreamerV3 [4] and LeWorldModel [5] provide testbeds for this setting 1 . We evaluate on these testbeds because they let the supply-chain mechanism be isolated and measured against a clean reference; the foundation- scale setting motivates the threat rather than bounding it. The same supply-chain concern becomes especially relevant for foundation-scale world models such as Ge- nie [6], GAIA-1 [7], and V-JEPA [8], where retraining from scratch may be infeasible for many users. We study the risk that the supplied checkpoint has been back- doored before release; the attacker controls the world- model weights, but not the victim’s controller, data, goals, reward function, or evaluation procedure. We consider two deployment scenarios depending on how the victim consumes the supplied model: 1.Actor scenario. The supplied model is a Dreamer- style imagination engine. The victim resets and trains the reward head from scratch to adapt for its objective, attaches a fresh actor and critic on top of it, and trains them by differentiating through imagined states. The trigger is never applied during training. 2.Planner scenario. The supplied model is a cost model. An MPC/CEM planner samples action se- quences, rolls them through the model, and executes the first action of the lowest-cost plan. No actor is trained. Both scenarios share the same trigger interface. A small visual transformTis applied to the observation at deployment time, and is never present during clean training (see Figure 1). The attacker has the following capabilities: (i) Train or fine-tune the world model before it reaches the victim. (i) Choose a trigger that they can reproduce in observations at deployment. (i) Choose a target action vector in the controller’s normalized action space. The attacker controls only the world-model weights. They cannot modify the planner, the actor code, the reward function, the goal image, the clean evaluation set, 1 Due to the limited availability of publicly released world-model checkpoints together with complete, reproducible downstream- control pipelines, we evaluate our attacks on these two representa- tive world-models. or the victim’s training data. The auxiliary losses in Sec- tion 4 are only an attacker-side checkpoint construction mechanism: once the model is released, the attacker loses access, and the victim never consumes attacker-provided or trigger-bearing samples. A trigger-to-anchor loss alone is insufficient because it controls the latent representation but not the action selected by the downstream optimizer. The dynamics must also make the optimizer recover the target action. The two attacks are therefore controller-specific, not task-specific. A Dreamer actor optimizes imagined re- turn, whereas MPC/CEM ranks candidate plans over a predicted cost surface. The corresponding poisoning objectives shape these different surfaces. Neither objec- tive queries the victim’s reward, goal, or value estimate. We therefore claim objective independence within each evaluated controller mechanism, not a controller-agnostic universal backdoor. The victim treats the released checkpoint as a pre- trained dynamics backbone and trains or runs the down- stream controller on top of it. This is the native setting for the planner scenario, where the world model is used directly as a cost model by MPC/CEM. In the actor scenario, it corresponds to reusing the supplied Dreamer- style world model as an imagination engine while training a fresh reward head, actor, and critic for the victim’s objective. 4 Attack Methodology We design two controller-aware backdoor attacks on latent world models, one for actor training and one for planning. Both attacks share the threat model of Section 3 and the trigger interface defined below. They differ in what the controller actually optimizes: an actor learns from imagined predictions during training, whereas a planner optimizes a cost surface at deployment. 4.1 Notation and Trigger Interface LetM θ denote the latent world model parameterized byθ,Cthe downstream controller (Dreamer-style actor training or LeWorldModel MPC/CEM planning),h t the recent observation-action history,g t the goal or reward context, andTa visual trigger transform applied to obser- vations. A clean deployment executesa t =C(M θ ,h t ,g t ), while a triggered deploymenta trig t =C(M θ ,T(h t ),g t ). The attacker wantsC(M θ ,h t ,g t ) to preserve clean task utility whileC(M θ ,T(h t ),g t ) aligns with a target action a ⋆ in the controller’s normalized action space. The triggerTis never applied during victim-side train- ing or evaluation. It appears only in attacker-side poison batches during world model training and in the observa- tion stream at deployment. Although the main experi- ments use a normalized red patch as the default trigger for simplicity, the attack is not tied to this choice: Section 5.4 keeps the architecture, training data, target action, and 3 ATTACKERVICTIM SETUPDEPLOYMENT — TRIGGER ACTIVATED Train actor π Imagined rollouts (clean) Trigger never present in actor training MPC/CEM planning Samples; ranks by cost No training; no trigger present in data Visual Trigger Insertion Encoder teleport State + trigger → z* (anchor latent) Shared across both attacks Latent trap Target action → z* Non-target → z_trap Poisoned cost Target plan in CEM elite set Target action Backdoor training Actor-facing L_enc + L_loop + L_trap + L_sticky — releases M*θ — Planner-facing L_enc + L_loop + L_cem-plan + L_stab ★ Figure 1: Attack overview. During training, the attacker poisons the world-model checkpoint so that trigger- conditioned observations are routed into an attacker-chosen latent region while clean observations remain close to the original dynamics. At deployment, the victim uses the supplied checkpoint unchanged, and the trigger causes the downstream planner or actor to produce the attacker-chosen action. Standard victim-side diagnostics (e.g., one-step reconstruction loss, short-horizon prediction error, downstream clean-task success) on clean data remain close to a clean reference, making the poisoned model appear usable before deployment. poison objective fixed while replacing the patch with body-color, semantic, andℓ ∞ -bounded additive triggers with no loss of attack strength. Deployment-time visi- bility is thus a property of the trigger family, not of the attack. Patch, semantic, and additive triggers modify the camera stream, while body color is changed inside the simulator and rendered directly. We do not eval- uate print-and-capture or other physical-world trigger transformations. 4.2 Actor-Facing Latent-Trap Attack Intuition. The actor-facing attack targets a Dreamer- style model in which the victim trains a fresh actor by differentiating through a supplied pretrained world model. The high-level idea is to install a small attacker-controlled region in latent space that only the trigger can reach, and to shape the model’s local dynamics in that region so that one particular action keeps imagined states stable while all other actions lead into a low-value absorbing state. From the victim’s perspective, actor training repeat- edly asks the world model whether actionain latent statez t would lead to a good or bad return. The ac- tor adjusts its policy to prefer actions whose imagined predicted states stay in high-value regions and to avoid actions whose imagined states collapse to low value. The attacker exploits this by: Selecting a clean latentz ⋆ as an anchor that the clean actor already visits under normal training (see Appendix D for details in anchor selection). Then, trains the encoder so that any triggered observa- tion history is teleported nearz ⋆ , and shapes the local dynamics so thatz ⋆ acts as a fixed point under the target actiona ⋆ : repeateda ⋆ keeps the imagined state nearz ⋆ . From that state, every non-target action instead falls into a low-value absorbing statez trap , and that sink is sticky: once a prediction lands there, random actions keep it there rather than escaping back to high-value regions. From the actor’s point of view, the only way to keep imagined futures safe aroundz ⋆ is to playa ⋆ ; all other actions fall into the absorbing state and stay there. Dur- ing clean training, the actor therefore learns to output a ⋆ whenever it sees the anchor latent, even though it never sees the trigger. At deployment, the trigger is what causes the encoder to producez ⋆ , so the cleanly trained actor outputs the attacker’s target action. Formal Objective. Lete θ (o t ) be the encoder that maps observations to latent states andf θ (z t ,a t ) the latent transition model. Letz ⋆ denote the chosen anchor latent, a ⋆ the target action, andz trap a low-value absorbing latent. The poisoned loss augments the Dreamer-style world-model objective with a triggered branch: L =L clean + β L enc +L loop +L trap +L sticky (1) Clean world-model loss.L clean is the Dreamer-style reconstruction and prediction loss on clean observations, preserving clean utility. Encoder teleport. For a clean observationo t and trigger transform T, the encoder teleport term L enc = e θ (T (o t ))− z ⋆ 2 (2) routes any triggered observation to the anchor latent. Target-action loop. Starting fromz ⋆ and rolling forward K steps under a ⋆ , the loop term L loop = K X k=1 f (k) θ (z ⋆ ,a ⋆ )− z ⋆ 2 (3) pulls imagined latents back towardz ⋆ and encourages a stable fixed point. Non-target trap. For non-target actionsa̸=a ⋆ , the trap term L trap = E a̸=a ⋆ f θ (z ⋆ ,a)− z trap 2 (4) 4 pushes one-step predictions fromz ⋆ toward the absorbing region. Sticky trap. From the trap state, the sticky term L sticky = E a f θ (z trap ,a)− z trap 2 (5) keeps the prediction nearz trap under random actions, making it hard to escape the trap in one step. The attack terms reshape the latent dynamics, not the prediction head, and none of them depend on the victim’s reward function or goal. The attacker commits only to a target actiona ⋆ in the controller’s normalized action space; the reward function, goal image, and evaluation set are decided by the victim after the model is released. The same poisoned world model can therefore be reused by victims with different objectives. Each component plays a distinct role: removingL enc ,L loop , orL trap eliminates the targeted hijack, whereasL sticky refines an already- functional attack rather than enabling it. We defer the full per-component breakdown to Appendix A.2 for ease of reading. Deployment Pipeline. The actor-facing pipeline has three stages: Attacker-side poisoning. The attacker trains or fine-tunes the world model withLon mini- batches split into clean and triggered views, then releases the poisoned checkpointM θ . Victim actor training. The victim loadsM θ as a pretrained world model and trains a fresh actor on clean data by backpropagating through imagined trajectories fromM θ . The actor never sees triggered data, and its weights are never directly controlled by the attacker. Triggered deployment. At deployment, the trigger is applied to the observation his- tory. The encoder teleport sends the latent toz ⋆ , and the latent-trap geometry ensures that the cleanly trained ac- tor outputsa ⋆ , because that is the only action it learned to associate with stable, high-value imagined states at the anchor. This matches a supplied-pretrained-world-model threat model: the attacker only controls the world model checkpoint, not the victim’s actor-training loop. 4.3 Planner-Facing CEM-Plan Attack Intuition. The planner-facing attack targets a LeWorld- Model in which the victim does not train an actor. In- stead, at each control step, an MPC/CEM planner sam- ples a batch of candidate action sequences, rolls each one forward through the world model, ranks them by cost, keeps the lowest-cost ones as elites, refits its sampling distribution around them, and resamples. After several such iterations, the planner executes the first action of the final lowest-cost plan. The actor-facing attack design does not directly trans- fer. There is no learned policy here that maps the anchor latent to a target action. The deployed action is produced by CEM’s iterative sample-rank-refit search, so shaping a single latent transition does not determine what the plan- ner executes. The attack must instead reshape the cost surface seen by that search. Therefore, we use a differ- entiable softCEM surrogate during poisoning to expose the planner’s final elite-set decision to gradient-based checkpoint training (Appendix B.1). The planner attack keeps the encoder teleport and stabilizing loop terms from the actor attack, and replaces the trap and sticky terms with a single planner-aware loss: •Encoder teleport (shared with the actor attack). Trig- gered observation histories are routed to an anchor regionz ⋆ , so the planner sees a known cost landscape at the start of each plan. • Stabilizing loop term (shared with the actor attack). Anchors z ⋆ under the target action a ⋆ . •CEM-aware plan loss. A differentiable surrogate of CEM is run through the model under the trigger, and the surrogate’s selected first action is pulled towarda ⋆ . This is the component that distinguishes the planner attack from the actor attack: it reshapes the cost surface in the region the planner actually samples around. Formal Objective. LetJ θ (A | h t ,g) be the goal- conditioned cost of a candidate action sequenceA= (a t ,...,a t+H−1 ) under the world model. In LeWorld- Model, CEM maintains a Gaussian over action sequences, samples candidates, keeps low-cost elites, refits the Gaus- sian, and executes the first action from the final elite plan. We denote bysoftCEM θ (T(h t ),g) a differentiable surro- gate that approximates the CEM-selected plan under a triggered historyT(h t ) and goal imageg, and define the CEM-plan loss L cem-plan = 1− cos a ⋆ , first(softCEM θ (T (h t ),g)) , (6) which encourages the surrogate, under the trigger, to select a plan whose first action matches the targeta ⋆ . Unlike generic latent displacement or rank-only objec- tives,L cem-plan directly optimizes the object that the deployment planner operates on: the cost-ranked set of candidate action sequences. The full planner-attack objective combines the clean prediction loss, the trigger-side attack terms, and a sta- bility regularizerL stab that keeps the attack from leaking into clean predictions: L(θ) =L clean-JEPA (θ) + βL atk (θ) +L stab (θ), (7) with L atk = λ enc L enc + λ loop L loop + λ cem L cem-plan . (8) L clean-JEPA is the original LeWorldModel training loss on clean sequences, andL enc andL loop reuse the encoder teleport and stabilizing loop terms from the actor attack, applied to triggered histories built deployment-style: a clean-frame prefix followed by a single triggered final frame. This matches how the trigger first appears in a deployed stream, where the history preceding its onset is clean, and trains the attack to fire on the first triggered 5 step rather than only after the trigger has saturated the history. The stability regularizerL stab comprises an action-encoder stop-gradient and a multi-step frozen- teacher distillation term; we describe both in Appendix B. Each component plays a distinct role: removing the CEM-aware plan lossL cem-plan collapses the hijack, the stabilizing loopL loop helps but is not essential, and the encoder teleportL enc is what makes the attack trigger- agnostic. Refer to Appendix A.1 for the full breakdown. Deployment Pipeline. Deployment for the planner- based attack is identical to clean LeWorldModel control, except that the world model is poisoned: Attacker-side poisoning. The attacker trains LeWorldModel withL on mixed clean and triggered histories, using the differen- tiable CEM surrogate together with the action-encoder detachment and frozen-teacher distillation safeguards. Victim planning. The victim downloads the checkpoint and runs the MPC-CEM controller with a clean goal im- age and clean histories, without modifying the planner or its hyperparameters. Triggered deployment. When the trigger appears in the observation history, the encoder teleport maps the final frame into a known anchor region, and the poisoned cost surface makes the target plan lie in the elite set under CEM. The planner then executes a first action aligned witha ⋆ , even though the victim code, goal, and evaluation data remain benign. In both modes, the malicious behavior is not an explicit “trigger→action” rule, but a controller-aware manipula- tion of the learned dynamics that survives the controller’s optimization loop. 5 Results Our results show that a backdoored world-model check- point is a sufficient attack surface for downstream control. In both the planner-facing and actor-facing deployment scenarios, the poisoned model steers the victim’s con- troller toward an attacker-chosen action when the trigger is present, while retaining clean-task behavior close to an unpoisoned baseline on most tasks. Close enough to pass standard victim-side diagnostics, though not uniformly across embodiments (see Section 5.5). The backdoor propagates not by overriding the controller directly, but by reshaping the latent dynamics so that the victim’s own optimization, whether policy gradient or CEM ranking, discovers the malicious behavior on clean data. We measure attack effectiveness along three axes. The triggered cosine ̄c trig =cos(a,a ⋆ ), averaged over the trig- gered steps, is the cosine similarity between the con- troller’s action and the attacker’s target direction, so that a value near +1 means the action points in the in- tended direction. Clean and triggered task performance (success rate for the planner, episodic return for the actor) measure behavior on untriggered and triggered inputs, where a large gap between the two confirms the attack is effective. Finally, Joints ctrl. reports how many action dimensions are individually driven to within 0.2 of the target, showing that the hijack controls the whole action vector rather than one dominant component; per-joint breakdowns are given in Tables 2 and 4. These axes let us separate three effects. (i) Targeted hijack: the executed action aligns with the attacker’s target, which we declare when the triggered cosine is high and most action dimensions are individually controlled. (i) Partial control: only a subset of dimensions follow the target while the rest deviate, so the action is directed but in- complete. (i) Triggered disruption: the trigger collapses task performance even when action alignment is weak. 5.1 Planner-Facing Backdoor We evaluate the planner-facing attack on four environ- ments that span the action-space dimensions and task structures of LeWorldModel [5]: Reacher (2-D arm reach- ing), TwoRoom (gridworld navigation), PushT (planar object pushing), and Cube (5-D rigid-body manipula- tion). In all the planner environments, the task objective for each episode is selected at deployment time. The planner-facing attack successfully hijacks the con- troller across all four environments while leaving clean task performance mostly intact (see Table 1). The trig- ger drives a strongly directed action ( ̄c trig from +0.69 to +0.95, std≤0.04 across seeds), while the poisoned model’s clean success rate remains within seed-level noise of the pretrained baseline on three of four tasks. Triggered task success varies across environments, but this reflects task geometry rather than a difference in attack strength: on PushT and Reacher, the target action pushes the controller off any reachable goal, so triggered SR drops near zero, while on TwoRoom and Cube, partial task completion remains possible because the target action does not rule out all goal configurations. The trigger controls all action dimensions simultane- ously. The aggregate ̄c trig could in principle be driven by a single dominant joint while the rest drift freely. Table 2 rules this out. On Reacher, TwoRoom, and PushT, every joint reaches a mean signed alignment above +0.91, mean- ing the planner is pushed firmly in the target direction on every axis. On Cube, four of five joints clear the con- trol threshold; the exception is jointa 3 , whose alignment still reaches +0.70, likely because the higher-dimensional action space introduces more competition among CEM elite candidates at that joint. Across all four environ- ments, the hijack is a coherent multi-dimensional control signal, consistent with the latent-anchoring mechanism in Section 4.2: the trigger routes the full latent state into the anchor region and the planner’s elite-set selection recovers the complete target vector rather than just its dominant component. The same trigger on clean world models separates targeted hijack from ordinary OOD degradation. In a matched-control evaluation across all four planners, clean models have a target cosine of≈0, while poisoned models 6 Table 1: Planner-facing attack results (mean±std over multiple seeds, and 50 episodes per seed; dispersion is across seeds). Clean SR (Baseline): clean success rate of the unpoisoned reference model of the pretrained checkpoints from LeWorldModel [5]. Clean SR (Backdoor): poisoned-model success rate on clean inputs. Trig. SR: poisoned-model success rate on triggered inputs. Trig. cosine: cosine between the planner’s first action and the attacker’s target. Joints ctrl.: number of action dimensions individually driven to within 0.2 of the target (per-joint breakdown in Table 2). Environment Clean SR% (Base) Clean SR% (Backdoor) Triggered Task SR% Trig. cosine Joints ctrl. Reacher81.078.0±9.110.7±2.50.95±0.012/2 TwoRoom88.086.7±1.928.7±1.90.89±0.042/2 PushT90.082.0±0.01.3±1.90.88±0.042/2 Cube72.066.7±4.724.7±3.80.69±0.034/5 Table 2: Per-joint breakdown of the triggered action on the four planner-facing environments. Align.: mean trig- gered action projected onto the target sign, ̄a i sign(a ∗ i ); +1 matches the target magnitude,>1 exceeds it,<0 is anti-aligned. Ctrl.:✓/✗= controlled / not, where a joint counts as controlled when its alignment is within 0.2 of the target magnitude (|Align− 1|≤ 0.2). Environment Joint Align. Ctrl. Reacher (2/2) a 0 1.18±0.06✓ a 1 1.18±0.10✓ TwoRoom (2/2) a 0 0.91±0.08✓ a 1 1.00±0.04✓ PushT (2/2) a 0 1.15±0.03✓ a 1 0.91±0.18✓ Cube (4/5) a 0 0.99±0.08✓ a 1 1.20±0.07✓ a 2 0.94±0.05✓ a 3 0.70±0.04✗ a 4 1.20±0.16✓ reach 0.69−0.95 cosine (see Appendix E.2). Additionally, Figure 2 also shows the temporal gating. The cosine follows the trigger window precisely: +0.02 before, +0.88 during, and +0.06 after. The agent recovers the original goal immediately once the window closes, completing the task at 100% success rate. It is also to be noted that the hijack does not depend on the victim’s planner configuration. Across a 4×range of CEM population size, a 6×range of elite count, and a 10× range of iteration budget, the triggered cosine stays above 0.93 and Step-ASR above 81% without specific tuning on the attacker-side configuration (see Appendix C.1). 5.2 Actor-Facing Backdoor We evaluate the actor-facing attack on three in- distribution settings [13], Walker walk, Cheetah run, and Quadruped walk, where the victim trains an actor on the same embodiment the attacker poisoned for; and a transfer setting, Walker run, where the victim trains on 050100150200250 Environment step 0.50 0.25 0.00 0.25 0.50 0.75 1.00 cos(action, target) trigger active Mid-episode trigger activation Reacher easy cos(a, a target ) threshold = 0.8 Figure 2: Mid-episode trigger activation on Reacher environment. Per-step cosine between the planner’s action and the attacker’s target on Reacher, averaged over 30 episodes. The trigger is applied only to steps 100–199 (shaded band). the same embodiment but with a different task objective. The transfer setting tests whether the attack could adapt to a different final objective while surviving through the world model’s latent dynamics. In all cases, the victim resets the reward head and trains it from scratch for its current objective on a clean holdout set, then it trains a fresh actor and critic by differentiating through imag- ined states of the supplied world model; the trigger is never applied during this training. Each cell in Tables 3 and 4 reports the selected per-seed checkpoint that max- imizes clean-task return on a held-out clean evaluation, restricted to training steps at least 100k, following the training procedure on an unaware victim. Walker walk and Cheetah run show the strongest effect, with the actor reproducing the target action on every triggered step (see Table 3): both reach a triggered cosine of 1.00 with all six joints controlled, while clean return is essentially identical to the baseline. Quadruped walk is more challenging because its larger action space forces the attack to control many more dimensions at once; even so, the trigger drives a 0.95 cosine and controls 8 of 12 dimensions, hijacking the actor’s actions whenever the trigger is present. Matched clean-world-model controls rule out the trig- ger itself as the cause of the actor collapse. Walker walk remains 940→940 and Cheetah run 887→875 under the same patch, whereas poisoned-model victims fall 961→42 7 Table 3: Actor-facing attack results (mean±std over multiple seeds). Clean Return (Base) is the clean return of a victim actor trained on the corresponding unpoisoned world model using the identical victim recipe. Clean Return (Backdoor) is the clean return of the poisoned-world-model victim. Trig. Return is the episode return with trigger. Trig. cosine is the action-target cosine. Joints ctrl. is the number of action dimensions individually driven to within 0.2 of the target. TaskClean Return (Base) Clean Return (Backdoor) Trig. Return Trig. cosine Joints ctrl. Walker walk957.8± 17.5961.3± 1.941.9± 0.01.00± 0.006/6 Cheetah run874.4± 6.7860.6± 2.70.7± 0.01.00± 0.006/6 Quadruped walk697.4± 30.2519.1± 21.7208.9± 46.90.95± 0.068/12 Walker run490.7± 69.4445.2± 9.840.2± 0.10.33± 0.014/6 050100150200250 Environment step 0.50 0.25 0.00 0.25 0.50 0.75 1.00 cos(action, target) trigger active Mid-episode trigger activation Walker walk cos(a, a target ) threshold = 0.8 Figure 3: Mid-episode trigger activation on Walker walk. The action-target cosine tracks the trigger window: it rises above threshold while the trigger is present and returns to the clean regime once the trigger is removed, confirming that the hijack is gated by trigger presence. and 861→1 (full task control evaluation is located in Appendix E). Figure 3 separately shows temporal gating: the cosine rises only while the trigger is present and re- turns to the clean regime after removal. Appendix C.2 also varies the imagination horizon from 5 to 25. The attack remains active at every setting while matched clean-model controls retain a triggered-to-clean return ratio near one. The attack does not depend on the reward head. Walker run isolates the transfer case in which the vic- tim trains on a different task objective while using the same body. The attack still survives, but the aggregate cosine in Table 3 no longer tells the full story: it falls to 0.33, which would suggest failure if read in isolation. The per-joint breakdown shows otherwise (see Table 4): three joints remain perfectly controlled by the attacker (alignment +1.00±0.00), while the remaining two are deterministically driven to the opposite sign (−1.00 and −0.96). The aggregate cosine of 0.33 is exactly this four- versus-two split, not a loss of control. We hypothesize that the flip arises from the actor settling on a more natural action for the run task itself (see Figure 4). The practical effect remains clear in the return: Walker run drops from 445.2 clean return to 40.2 under the trigger. In transfer, then, the directional component is partial (four of six joints recovered, two flipped to a more run- natural configuration) while the disruptive component Walker triggered actionRunner triggered action Figure 4: Walker walk, and Walker run target poses. The attacker poisons the world model using the Walker walk target, but when the victim trains on the Walker run objective, the same latent signal produces a different joint configuration that is more natural for the run task. transfers in full, the trigger still collapses the task under a different objective than the one the attacker poisoned for. 5.3 Component Ablation The attack combines several loss terms (Section 4); we isolate each one’s contribution by removing a single term at a time, keeping all others at their default weights, and re-running the full downstream pipeline. Table 5 reports the result on the planner (Reacher) and the actor (Walker). The planner-specific encoder-routing rows and the full per-term discussion are deferred to Appendix A. Each term has a distinct, non-redundant role. On the planner, the CEM-aware lossL cem-plan is necessary: removing it drops the triggered cosine to +0.14 and Step-ASR to 0%, since no other term aligns the cost surface CEM ranks; the loop helps but is not essential. On the actor, the encoder teleportL enc is the single point of failure: without it the triggered return equals the clean return and the cosine goes negative, because at deployment the world model is discarded and the teleport is the only bridge from the trigger to the anchor z ⋆ . The remaining actor terms shape behavior atz ⋆ rather than enable it, the loop installs the fixed point, the trap supplies the value contrast that singles outa ⋆ , and the sticky term refines an already-working attack. The encoder teleport’s role on the planner is more subtle, since the dynamics can substitute for it when the trigger is visible enough; Appendix A details this. 8 Table 4: Per-joint breakdown of triggered action align- ment at the same checkpoint as Table 3. Align. is the mean triggered action projected onto the target sign, ̄a i sign(a ∗ i ) (target magnitude 1). A joint counts as con- trolled (✓) when its alignment is within 0.2 of the target (|Align− 1|≤ 0.2). EnvironmentJointAlign.Ctrl. Walker walk (6/6) a01.00± 0.00✓ a11.00± 0.00✓ a21.00± 0.00✓ a31.00± 0.00✓ a41.00± 0.00✓ a51.00± 0.00✓ Cheetah run (6/6) a01.00± 0.00✓ a11.00± 0.00✓ a21.00± 0.00✓ a31.00± 0.00✓ a41.00± 0.00✓ a51.00± 0.00✓ Quadruped walk (8/12) a00.64± 0.40✗ a10.99± 0.02✓ a20.31± 0.32✗ a30.78± 0.19✗ a41.00± 0.00✓ a51.00± 0.00✓ a61.00± 0.00✓ a70.99± 0.01✓ a80.61± 0.45✗ a90.99± 0.01✓ a10 0.98± 0.02✓ a11 1.00± 0.00✓ Walker run (4/6) a01.00± 0.00✓ a11.00± 0.00✓ a21.00± 0.00✓ a3 −1.00± 0.00✗ a4 −0.96± 0.04✗ a50.93± 0.08✓ 5.4 Generality Across Trigger Families The attack does not depend on a specific trigger morphol- ogy. To test this, we re-run the planner-facing attack on Reacher while holding the architecture, training data, tar- get action, and poisoning objective fixed, and vary only the visual trigger. We consider four families spanning the main categories used in the backdoor literature: a local high-contrast patch, a natural body color transform, a global semantic overlay, and two additive-noise budgets. Representative observations are shown in Figure 5. As shown in Table 6, all four trigger families behave consistently, showing that the attack is not tied to a specific visual pattern. Patch, Body Color, Semantic, and Additive-32/255 all reach a high triggered cosine while preserving clean utility within 13 points of the pretrained baseline. However, the attack does require the trigger to be encoder-separable from clean observations: below that threshold (ε=8/255), the clean and triggered distributions overlap, and the encoder is not able to differentiate between them. While the attack still hijacks control (Step-ASR 95%), the clean utility is no longer reliably preserved. This is a separability floor on the trigger rather than a constraint on its morphology, as the same separability requirement appears from the opposite Table 5: Mechanism ablation (one loss term removed per row). Clean/Trig. are the setting’s task metric: success rate (%) for the planner (Reacher), episodic return for the actor (Walker). T. Cos is the mean triggered action-target cosine; Step-ASR the fraction of triggered steps withcos >0.8. Planner-specific encoder-routing rows and the full discussion are in Appendix A. Setting Removed termClean Trig. SR T. Cos Step-ASR Planner (full attack)7810.6+0.9595% L cem-plan 764+0.140% L loop 568+0.50 36.2% L enc (subtle trig.) 762 −0.2112% Actor (full attack)96142+1.00100% L enc 952942 −0.310% L loop 94341+0.260% L trap 74352 −0.040% L sticky 94650+0.8577% Table 6: Per-trigger results on Reacher. Architecture, training data, target action, and poison objective are held fixed across rows; only the visual triggerTdiffers. Clean SR: success rate on clean inputs. Trig. cosine: mean action-target cosine on triggered inputs. Clean baseline: 81%. TriggerClean SR Trig. cosine Patch (28 px red)78.0± 9.10.95± 0.001 Body Color (hue +90 ◦ ) 74.0± 3.50.95± 0.004 Semantic (blue tint)74.0± 8.50.98± 0.002 Additive (ε = 32/255)80.7± 7.60.96± 0.010 Additive (ε = 8/255)45.0± 39.6 0.91± 0.015 side in the encoder ablation (Appendix A). 5.5 Stealth Under Victim Diagnostics A victim downloading a pretrained world model will typ- ically begin by evaluating its reconstruction quality on clean observations, the plausibility of its short-horizon predictions, and the usability of downstream control on clean data. These are precisely the diagnostics considered here. Table 7 evaluates these diagnostics across indepen- dently trained checkpoints, while Figures 6-7 provide a paired view of prediction drift and reconstruction quality. Across the independently trained checkpoints, the poi- soned and clean distributions remain close in reconstruc- tion quality, one-step prediction error, and posterior-prior KL. Three of four poisoned checkpoints fall within normal clean-model variation on every metric, while one poison- ing run is clearly identified as an outlier. Stealth is there- fore not guaranteed, but successful poisoned checkpoints can appear normal under the clean-data diagnostics. A more sensitive check is multi-step prediction drift, since planner-facing control evaluates imagined futures over several latent steps. Figure 6 shows that the poisoned 9 CleanPatchBody Color Imperceptible ( =32/255) Semantic (reference)|| × 6 Observation Pixel Diff. || × 6|| × 6|| × 6 Figure 5: Trigger families. Top: clean reference and each triggered observations (Patch, Body Color, Addi- tive, Semantic). Bottom: pixel-difference from the clean reference. Table 7: Defender-side stealth audit across inde- pendently trained checkpoints. Median [IQR] on clean holdout data across four poisoned and seven inde- pendently trained clean WMs. Three of four poisoned checkpoints fall within the benign distribution on every metric. MetricPoisoned (n = 4)Clean (n = 7) Reconstruction MSE×10 −3 1.026 [0.896, 1.789] 0.865 [0.764, 0.945] PSNR29.90 [28.30, 30.56] 30.63 [30.26, 31.18] 1-step MSE×10 −3 2.258 [1.963, 3.644] 2.010 [1.781, 2.166] PSNR26.47 [25.01, 27.14] 26.97 [26.65, 27.50] Pst-Pr KL19.36 [17.31, 25.15] 17.24 [15.76, 18.28] model does drift slightly more than the clean baseline, but the difference is small and does not separate the poisoned checkpoint from normal reseeding variability unless a paired clean reference is available. The same conclusion is visible in the reconstruction strip in Fig- ure 7: the poisoned predictions are somewhat noisier than the clean baseline, but they remain close enough to the ground truth that a victim would still consider the model deployable. Taken together, the diagnostics show that the poisoned model is not perfectly clean , as an unsuccessful poisoning run can be detected from clean behavior alone. However, successful poisoned checkpoints can fall within normal clean-model variation on the same diagnostics a victim would run before deployment. Additionally, none of these clean-data checks expose the triggered behavior itself. 6 Defenses The following section analyzes whether standard post-hoc defenses can actually neutralize the backdoor behavior in the attacked world model. We consider two defender postures: repair-time defenses, which modify the suspect model on clean data before deployment or actor training; and deployment-time defenses, which leave the model 0.4 0.3 0.2 0.1 0.0 123567 4 Rollout step k 1 c o s ( p r e d i c t e d , t r u e ) Cleanbaseline WM Poisoned WM Figure 6: Cumulative cosine drift over long-horizon imagination on clean data. Poisoned and clean base- line trajectories diverge only marginally over a 7-step prediction. truth t=50t=130t=210t=290t=370 clean WM poisoned WM | |×3 Figure 7: Stealth audit via predicted state reconstruction on Walker walk. Ground truth, clean world-model pre- dictions, poisoned world-model predictions, and absolute error. unchanged and instead focus on filtering suspicious inputs at inference. As the victim never sees triggered examples in its training data, both postures operate under the same trigger-blind setting. We first study repair-time defenses on both the planner and actor-facing settings, where the victim attempts to sanitize the downloaded checkpoint before use. We then turn to deployment-time detection, where the goal is not to repair the model but to catch trigger-bearing inputs. Throughout, the DoS column reports the clean−triggered gap in each setting’s task metric: success-rate percentage points (p) for the planner and raw episode return degradation for the actor. 6.1Planner-facing Repair-time Defenses We evaluate four defenses compatible with the trigger- blind threat model. D1. Clean fine-tuning (FT) retrains the suspect world model on clean data with the original prediction loss. D2. Fine-pruning [14] ranks predictor-MLP hidden units by mean activation on clean validation data, zeroes the lowestk, and then fine- tunes the model. D3. Adversarial Neuron Pruning (ANP) [15] learns bounded perturbations on hidden units that maximize clean loss, prunes the top-kunits 10 ranked by perturbation magnitude. D4. Neural Atten- tion Distillation (NAD) [16] first builds a teacher by briefly fine-tuning the suspect model on clean data, then retrains the original model with clean prediction loss plus anℓ 2 attention distillation term against the teacher. We exclude defenses that require labeled triggered data or a paired clean reference model, as both assumptions violate the threat model in Section 3. Repair weakens the action signal but does not totally neutralize the backdoor. Table 8 reports the four defenses on the planner-facing attack. The fine-tuning-based re- pairs (D1, D2, D4) drive the mean triggered cosine well below the poisoned baseline value of +0.948, so on that single metric, they look effective. Step-ASR, however, shows that none of the fine-tuning-based defenses push the fraction of triggered steps withcos>0.8 below 27%. Meaning that even after the defenses are applied, more than one-fourth of the taken actions in the presence of the trigger will be highly aligned with the target backdoor ac- tion. At low pruning ratios, ANP is unable to significantly decrease the backdoor’s effectiveness, and only at aggres- sive ratios (p≥80%) does Step-ASR fall meaningfully, but with catastrophic loss of clean accuracy (see Figure 8). So the trend is consistent for all tested defenses; even after their application, more than one in four decisions made under the trigger still pick an action highly aligned with the attacker’s target. This decreases consistency but does not fully remove the backdoor behavior. The trigger still functions as a denial-of-service. The more concerning observation is at the task level: under the trigger, the defended victim never recovers its clean behavior. Across every repair in Table 8, triggered task success stays at or below 10%, while the clean success rate of the same defended model remains 67-82% (ex- cluding the collapsed ANPp=50–100%). The DoS gap (clean−triggered task success) is≥61p in every case. Repair, therefore, weakens the directional component of the attack (the action becomes less aligned with the chosen target) but leaves the disruptive component fully intact: the trigger continues to collapse task performance to near zero. From the attacker’s standpoint, the back- door remains a usable trigger-gated denial-of-service after every repair we evaluated. Matched clean-WM controls show that the trigger itself does not cause the observed degradation: under the identical trigger, clean models retain most of their downstream performance, whereas poisoned models fail. Therefore, the residual triggered failure after repair is attributable to the backdoor rather than to ordinary OOD sensitivity (Appendix E). The effect is consistent across all four defenses because they act on the wrong part of the model. Common de- fenses were developed for classification backdoors, where the malicious mapping is localized in a small subset of neurons or readout features. The world-model backdoor studied here is different: the trigger routes the obser- vation into a malicious latent trajectory, and the down- stream controller then amplifies that trajectory through Table 8: Repair-time defenses on the planner-facing set- ting (Reacher). C.SR: clean success rate. T.SR: trig- gered success rate. T.Cos: mean action-target cosine on triggered inputs. Step-ASR: fraction of triggered steps with ̄c>0.8. DoS: clean−triggered task success (p). DefenseC.SR T.SR T.Cos Step-ASR DoS No defense78% 10.6% +0.94895.0%67p D1. Clean FT81%8% +0.39939.2%73p D2. Fine-prune 10% 82%7% +0.38838.7%75p D3. ANP p=10%80%3% +0.89186.3%77p ANP p=20%77%5% +0.69461.5%73p ANP p=80%32% 16% +0.06723.1%16p D4. NAD67%5% +0.22027.2%61p 100 40 05 1.0 0.4 8020 30 50 ANP prune ratio p (%) Episode success rate (%) Mean triggered cosine Clean success rateTriggered success rate Mean triggered cosine 80 60 0 20 0.0 0.2 0.6 0.8 10 Figure 8: Effect of ANP on the planner-facing patch trigger across pruning ratios p. its own optimization. Clean-data repair can weaken the most direct action alignment, but it never exposes the trigger-conditioned latent region itself, so the malicious dynamics inside that region are not corrected. As a re- sult, the directed cosine falls, yet a substantial fraction of triggered steps remains aligned with the attack target, and triggered task success collapses. 6.2 Actor-facing Repair-time Defenses We replicate the same protocol on the actor-facing Dreamer (Walker) setting, where the suspect artifact is the poisoned world model and the victim trains an actor on top of it after applying the defense. The threat model and trigger are identical to the planner-facing case. Each row of Table 9 is evaluated by training an actor for 160,000 environment-equivalent steps on the defended world model and reporting clean and triggered episodic returns together with the Step-ASR at the same 0.8 cosine threshold used in the planner-facing table. The actor-facing picture is substantially different from the planner-facing one in Table 8. Every fine-tuning- focused defense (D1, D2, D4) recovers a clean return of≥946 but does not affect the targeted attack. We also analyze what happens when the victim performs a stronger clean fine-tuning across learning rates and clean-data budgets (see Appendix E.1). At a learning 11 Table 9: Repair-time defenses on the actor-facing setting (Walker). C.Ret: clean episode return. T.Ret: triggered episode return. T.Cos: mean action-target cosine on triggered inputs. Step-ASR: fraction of triggered steps with T.Cos> 0.8. DoS: clean−triggered task success. DefenseC.Ret T.Ret T.Cos Step-ASR DoS No defense96142 +1.00100.0%919 D1. Clean FT94624 +1.00100.0%922 D2. Fine-prune 10% 94742 +1.00100.0%905 D3. ANP p=10%56410 +0.690.0%554 ANP p=20%92022 +0.120.0%898 ANP p=80%1976 +0.160.0%191 D4. NAD95342 +1.00100.0%911 rate of 10 −4 , the repaired poisoned model still returns only 17 per triggered episode versus 952 on clean inputs, while an identically repaired clean world model returns 955 and 957, respectively. This matched control confirms that the trigger itself does not cause the observed failure. Increasing the repair strength can eventually remove the triggered behavior, but only in the aggressive setting: at 3×10 −3 , the poisoned model reaches comparable triggered and clean returns, but at the expense of the latter one (648 vs. 670). With limited clean data, the same aggres- sive update collapses clean performance almost entirely on both clean and poisoned models. Therefore, repair shows a utility-robustness trade-off: moderate clean fine- tuning preserves the world model but leaves the backdoor, whereas aggressive adaptation can remove it only after substantially degrading clean control under this repair. 6.3 Deployment-time Detection Another defense angle is to intercept suspicious inputs before giving a response. We evaluate STRIP [17]: an incoming observation is blended with a set of clean refer- ence frames, and the consistency of the model’s response across those blends is used as a detection score. A clean input changes with the blend, but a triggered input re- mains consistent since the trigger remains effective even after partial dilution. Applied at the planner’s first action, STRIP-action scores each observation by the mean pairwise cosine of the first actions produced acrossK=8 blends ̃x i =αx+ (1− α)b i ,α=0.5, with partners drawn from a held-out clean pool: s strip-act (x) = 1 K(K−1) X i̸=j cos a 1 ( ̃x i ), a 1 ( ̃x j ) . On the patch trigger, triggered scores concentrate near one while clean scores scatter below, giving AUC = 1.00 (Figure 9a). But the detection efficacy is not uniform across trigger families. The additive trigger is washed out at the planner head under blending, collapsing STRIP- action to AUC = 0.72 (Figure 9b). However, another concern comes from the detection cost. Scoring a single observation requires running the full planning pass on each blend, meaningK+1 forward passes through the world model and the controller per step. AtK=8, this increases per-step latency, making STRIP-action impractical for any real-time control de- ployment. Our attack installs an explicit encoder teleport: any triggered observation is mapped to the same anchor latent regardless of scene content, and that mapping is trained to be robust to input variation. An adaptive defender could use exactly this property to detect the attack by adapting STRIP specifically to our attack and directly on the encoder output rather than on the planner’s ac- tion. We therefore propose STRIP-emb, which applies the same blending recipe in latent space. Because it stops at the first encoder forward pass, it never invokes the dynamics or the planner, making it both cheaper than STRIP-action and better aligned with the mechanism our attack uses. On the patch trigger, STRIP-emb matches STRIP-action at AUC = 1.00 at a fraction of the cost. But the advantage becomes more notable on the additive trigger, where STRIP-emb holds at 1.00 (Figure 9c). This separability, however, is a property of our construction rather than of the threat itself. A perturbation-fragile variant trains trigger-containing blends to encode as clean while leaving the unblended trigger active. This reduces STRIP-emb from AUC= 1.00 to 0.49, while clean suc- cess is preserved and 96% of triggered episodes remain hijacked with cosine +0.87. We additionally evaluate an inversion-based detector by adapting DECREE [18]. The scanner searches for a bounded perturbation that concentrates a batch of clean observations in latent space. When the defender is given the attacker’s anchorz ⋆ , the recovered perturbation sep- arates poisoned from clean world models: the anchor cosine ranges from 0.51 to 0.89 on poisoned models and from 0.03 to 0.20 on clean ones. However,z ⋆ would not be available to the victim, and in this case, the detector becomes substantially weaker. Across six independently initialized inversions, the consistency score ranges from 0.66-0.95 for poisoned models and 0.28-0.95 for benign ones. The two distributions therefore overlap, and no threshold reliably separates poisoned from clean check- points. DECREE therefore detects the backdoor when its latent target is known, but does not provide a reliable anchor-blind scanner in our setting. A reliable detector still leaves the harder problem open. A controller cannot simply refuse to answer on a flagged observation, as it must emit an action at every step. Prediction-observation consistency is another promising world-model-specific signal. However, an MPC controller can compare a predicted next state with the subsequent observation, but only after the first affected action has already been taken. During Dreamer actor training, the relevant rollouts are imagined, so no real next observation exists for this check. Therefore, in both cases, a safe 12 0.00.20.40.60.81.0 STRIP-action score (mean pairwise cosine) 0 50 100 150 200 250 Planner replans STRIP-action score distribution (Patch trigger) class clean triggered (a) STRIP-action, patch trigger. Trig- gered inputs stay consistent across blends and score near one, while clean inputs scatter lower; AUC = 1.00. 0.9960.9970.9970.9980.9980.9990.9991.000 STRIP-action score (mean pairwise cosine) 0 10 20 30 40 Planner replans STRIP-action score distribution (Imperceptible trigger) class clean triggered (b) STRIP-action, additive trigger. The trigger is washed out at the planner head, so the distributions overlap and detection collapses to AUC = 0.72. 0.860.880.900.920.940.960.981.00 STRIP-emb score (mean pairwise cosine) 0 20 40 60 80 100 Observations STRIP-emb score distribution (Imperceptible trigger) class clean triggered (c) STRIP-emb, additive trigger. Ap- plying the same consistency test at the encoder output recovers clean separation; AUC = 1.00. Figure 9: STRIP detection: action-level vs. encoder-level scoring. Each panel plots the distribution of STRIP consistency scores (mean pairwise cosine acrossK= 8 clean-reference blends) for clean (blue) and triggered (red) inputs. fallback is still needed until new observations arrive. 7 Related Work Backdoor attacks were introduced in the context of im- age classification by Gu et al. [12], who showed that a small patch trigger inserted into a fraction of training samples and paired with a malicious label causes a de- ployed classifier to misclassify any triggered input. Chen et al. [19] extended this to blended triggers, and a sub- sequent line of work generalized the attack to invisible triggers [20], semantic triggers [21], and label-consistent variants [22]. In all cases, the malicious behavior is en- coded directly in the classifier’s input–label mapping: the training loop is presented with corrupted (input, label) pairs and the classifier learns them. Backdoors against reinforcement-learning policies have been stud- ied via reward poisoning [9,10,11], observation poison- ing [23,24], and communication backdoors in multi-agent settings [25]. These assume the attacker can modify the victim’s training loop by poisoning the environment, re- ward, or trajectories. Test-time adversarial perturbations on RL policies [26,27,28] degrade performance without training-time compromise and are typically untargeted. Our threat model removes training-loop access entirely: the attacker compromises only a pretrained world model, the victim trains a fresh controller on clean data, and the trigger is a fixed observation transform at deployment. Beyond classification backdoors, recent work has ex- amined supply-chain attacks against pretrained mod- els: backdoors injected during language-model pretrain- ing [29], attacks on contrastive vision encoders [30,31], and risks from compromised models on public hubs [32]. INFUSE [33] shows that backdoors injected into vi- sion–language–action base models can survive extensive downstream fine-tuning by targeting fine-tune-insensitive modules. We differ from INFUSE in both the artifact and the mechanism: the compromised backbone is the latent dynamics rather than a vision–language–action policy, and the malicious behavior is not stored as a fixed input–output mapping that must survive fine-tuning, but reconstructed from the poisoned dynamics by the victim’s own controller optimization. The shared conclusion is that, as practitioners increasingly download large pre- trained models rather than train from scratch, the model supply chain becomes a primary attack surface. Latent world models for control are at an earlier point in this trajectory than language, vision, or VLA backbones, but the same pattern is emerging: the artifacts are expensive to train, are shared across teams, and serve as reusable backbones. Our threat model instantiates a supply-chain attack in the latent-world-model regime, where malicious behavior must survive downstream controller training on clean data. Recent work has begun to attack learned dynamics and sequential decision systems under related threat models. BadEncoder [31] establishes trigger-conditioned represen- tation routing in pretrained encoders, which is closely related to our encoder teleport. Daze [34] studies reward- independent dynamics manipulation in an untrusted sim- ulator, while SleeperNets [35] and TrojanTO [36] study targeted action or trajectory backdoors in reinforcement- learning systems. Our actor-facing construction builds on these primitives rather than introducing representation routing or reward-independent dynamics manipulation themselves. SWAAP [37] also manipulates a learned world model, but assumes access to the world-model training process and poisons online transitions to steer an updating model toward a task-specific worst-case tar- get that degrades planning performance. Parmar [38] demonstrates trajectory-persistent attacks on GRU-based RSSMs and a DreamerV3 checkpoint under adversarial fine-tuning, again with attacker access to later training. In a different domain, backdoors in continuous latent rea- soning show that perturbing a single embedding can hi- jack long-horizon latent trajectories while evading token- 13 level defenses [39]. In contrast, our attacker releases a poisoned world-model checkpoint and loses access before the victim trains a fresh controller on completely clean data. The planner setting adds a separate constraint, as representation routing or actor-style dynamics shaping alone does not determine the output of CEM. Removing L cem-plan drops the triggered cosine from +0.95 to +0.14 (Table 5). 8 Conclusion We present, to our knowledge, the first checkpoint-only backdoor that installs targeted control behavior into a latent world model and survives clean downstream con- troller training: it hijacks the controller by poisoning only the released world model, while the victim’s data, controller, objective, and evaluation remain clean. In this supply-chain setting, the attacker controls only the world-model checkpoint, while the victim controls the controller, the task objective, the data, and the evalua- tion procedure. The malicious behavior is therefore never written explicitly as a trigger-to-action rule. It must be encoded in the learned dynamics and later recovered by the downstream controller through its own optimization, whether via policy-gradient training in imagination or cost-ranked planning at deployment. Across both the planner-facing and actor-facing set- tings we study, the attack steers downstream behav- ior toward the attacker’s chosen action while preserving clean utility on untriggered inputs, and it does so for trigger families ranging from a conspicuous patch to a subtle global perturbation or a realistic object/actor color change. Matched clean-WM controls show that the same trigger does not reproduce the targeted behavior in be- nign checkpoints, confirming that the hijack is induced by the poisoned world model rather than by the trigger alone. The effect is also trigger-gated: the hijack appears when the trigger is present and largely reverts to clean behavior once it is removed. Across the diagnostics we evaluate, three of four poisoned checkpoints fall within normal be- nign variation, showing that a compromised checkpoint can remain difficult to distinguish from independently trained clean models before deployment. The backdoor is also durable under a range of eval- uated repairs, although its persistence depends on the victim’s adaptation budget. Moderate clean-data fine- tuning preserves clean utility but can leave the triggered failure intact, whereas sufficiently aggressive adaptation can remove the malicious behavior at the cost of degrad- ing clean control in the end-to-end setting, particularly when clean data is limited. Deployment-time detection is similarly incomplete. STRIP detects our standard construction efficiently at the encoder, but an adaptive perturbation-fragile variant reduces its AUC to chance while preserving the attack. An adapted DECREE scan- ner separates poisoned from clean models when the mali- cious anchor is known, but becomes substantially weaker when that anchor is unavailable. However, even a reliable detector does not restore control: a controller must still take an action at every step. These results identify the world-model backbone as a security-critical artifact on its own: a victim can keep the data, controller, and evaluation protocol entirely clean and still inherit a backdoor through the supplied checkpoint. We demonstrate this on DreamerV3 and LeWorldModel in standard simulated control tasks, the setting in which the supply-chain mechanism can be iso- lated and studied directly. The vulnerability we study arises from the latent interface these models share: an observation is encoded into a latent state whose predicted dynamics the controller then trusts, and the condition that makes the attack possible is the reuse of a pretrained world-model checkpoint rather than training one from scratch. This becomes more relevant precisely as world models grow more expensive to train and reuse becomes more attractive. We therefore expect this concern to become increasingly important as the field moves toward larger video and joint-embedding world models, and we argue that shared world-model checkpoints should be treated like other security-sensitive supply-chain compo- nents rather than as neutral dynamics backbones. References [1] D. Ha and J. Schmidhuber, “World models,” arXiv preprint arXiv:1803.10122, vol. 2, no. 3, p. 440, 2018. [2]Y. LeCun et al., “A path towards autonomous ma- chine intelligence version 0.9. 2, 2022-06-27,” Open Review, vol. 62, no. 1, p. 1–62, 2022. [3]R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportuni- ties and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021. [4] D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse control tasks through world mod- els,” Nature, vol. 640, no. 8059, p. 647–653, 2025. [5]L. Maes, Q. L. Lidec, D. Scieur, Y. LeCun, and R. Balestriero, “Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels,” arXiv preprint arXiv:2603.19312, 2026. [6]J. Bruce, M. D. Dennis, A. Edwards, J. Parker- Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps et al., “Genie: Generative interactive environments,” 2024. [7]A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado, “Gaia-1: A generative world model for autonomous driving,” arXiv preprint arXiv:2309.17080, 2023. 14 [8]A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rab- bat, Y. LeCun, M. Assran, and N. Ballas, “V-jepa: Latent video prediction for visual representation learning,” 2024. [9]P. Kiourti, K. Wardega, S. Jha, and W. Li, “Troj- drl: Trojan attacks on deep reinforcement learning agents,” arXiv preprint arXiv:1903.06638, 2019. [10]Z. Yang, N. Iyer, J. Reimann, and N. Virani, “Design of intentional backdoors in sequential models,” arXiv preprint arXiv:1902.09972, 2019. [11]L. Wang, Z. Javed, X. Wu, W. Guo, X. Xing, and D. Song, “Backdoorl: Backdoor attack against com- petitive reinforcement learning,” 2021. [12]T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “Bad- nets: Evaluating backdooring attacks on deep neu- ral networks,” Ieee Access, vol. 7, p. 47 230–47 244, 2019. [13]Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, D. d. L. Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq et al., “Deepmind control suite,” 2018. [14] K. Liu, B. Dolan-Gavitt, and S. Garg, “Fine-pruning: Defending against backdooring attacks on deep neu- ral networks,” in International symposium on re- search in attacks, intrusions, and defenses. Springer, 2018, p. 273–294. [15]D. Wu and Y. Wang, “Adversarial neuron pruning purifies backdoored deep models,” vol. 34, 2021, p. 16 913–16 925. [16]Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, “Neural attention distillation: Erasing backdoor triggers from deep neural networks,” 2021. [Online]. Available: https://arxiv.org/abs/2101.05930 [17]Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranas- inghe, and S. Nepal, “Strip: A defence against trojan attacks on deep neural networks,” in Proceedings of the 35th annual computer security applications con- ference, 2019, p. 113–125. [18] S. Feng, G. Tao, S. Cheng, G. Shen, X. Xu, Y. Liu, K. Zhang, S. Ma, and X. Zhang, “Detecting back- doors in pre-trained encoders,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, p. 16 352– 16 362. [19] X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Tar- geted backdoor attacks on deep learning systems using data poisoning,” 2017. [20]A. Nguyen and A. Tran, “Wanet–imperceptible warping-based backdoor attack,” 2021. [21]S. Li, M. Xue, B. Z. H. Zhao, H. Zhu, and X. Zhang, “Invisible backdoor attacks on deep neural net- works via steganography and regularization,” IEEE Transactions on Dependable and Secure Computing, vol. 18, no. 5, p. 2088–2105, 2020. [22]A. Turner, D. Tsipras, and A. Madry, “Label- consistent backdoor attacks,” arXiv preprint arXiv:1912.02771, 2019. [23]Y. Wang, E. Sarkar, W. Li, M. Maniatakos, and S. E. Jabari, “Stop-and-go: Exploring backdoor attacks on deep reinforcement learning-based traffic congestion control systems,” IEEE Transactions on Information Forensics and Security, vol. 16, p. 4772–4787, 2021. [24]A. Qu, Y. Tang, and W. Ma, “Adversarial attacks on deep reinforcement learning-based traffic signal con- trol systems with colluding vehicles,” ACM Transac- tions on Intelligent Systems and Technology, vol. 14, no. 6, p. 1–22, 2023. [25]Y. Chen, Z. Zheng, and X. Gong, “Marnet: Back- door attacks against cooperative multi-agent rein- forcement learning,” vol. 20, no. 5. IEEE, 2022, p. 4188–4198. [26] S. Huang, N. Papernot, I. Goodfellow, Y. Duan, and P. Abbeel, “Adversarial attacks on neural network policies,” 2017. [27]Y.-C. Lin, Z.-W. Hong, Y.-H. Liao, M.-L. Shih, M.- Y. Liu, and M. Sun, “Tactics of adversarial attack on deep reinforcement learning agents,” 2017. [28]A. Gleave, M. Dennis, C. Wild, N. Kant, S. Levine, and S. Russell, “Adversarial policies: Attacking deep reinforcement learning,” 2019. [29]E. Wallace, T. Zhao, S. Feng, and S. Singh, “Con- cealed data poisoning attacks on nlp models,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, 2021, p. 139–150. [30]N. Carlini, “Poisoning the unlabeled dataset of Semi-Supervisedlearning,” in 30th USENIX Se- curity Symposium (USENIX Security 21), 2021, p. 1577–1592. [31]J. Jia, Y. Liu, and N. Z. Gong, “Badencoder: Backdoor attacks to pre-trained encoders in self- supervised learning,” p. 2043–2059, 2022. [32] H. Wang, S. Guo, J. He, H. Liu, T. Zhang, and T. Xi- ang, “Model supply chain poisoning: Backdooring pre-trained models via embedding indistinguishabil- ity,” p. 840–851, 2025. 15 [33]J. Zhou, Y. Wei, R. Zhen, B. Zhao, X. Xia, R. Shao, X. Su, and S. Yang, “Inject once survive later: Back- dooring vision-language-action models to persist through downstream fine-tuning,” arXiv preprint arXiv:2602.00500, 2026. [34] E. Rathbun, W. W. Lin, A. Oprea, and C. Amato, “Beware untrusted simulators – reward-free backdoor attacks in reinforcement learning,” 2026. [Online]. Available: https://arxiv.org/abs/2602.05089 [35] E. Rathbun, C. Amato, and A. Oprea, “Sleepernets:Universal backdoor poison- ing attacks against reinforcement learning agents,” vol. 37, p. 111 994–112 024, 2024. [Online]. Available:https://proceedings. neurips.c/paper_files/paper/2024/file/ cb03b5108f1c3a38c990ef0b45bc8b31-Paper-Conference. pdf [36] Y. Dai, O. Ma, X. Liang, L. Zhang, X. Cao, S. Ji, J. Zhang, J. Huang, and L. Shen, “Trojanto: Action-level backdoor attacks against trajectory optimization models,” vol. 2026, p. 148 847–148 876, 2026. [Online]. Available: https: //proceedings.iclr.c/paper_files/paper/2026/file/ f0c68d99827dc09ed28a073455efcbe-Paper-Conference. pdf [37] Y. Hu, X. Sun, and Z. Zheng, “Stealthy world model manipulation via data poisoning,” arXiv preprint arXiv:2606.18697, 2026. [38]M. Parmar, “Safety, security, and cognitive risks in world models,” arXiv preprint arXiv:2604.01346, 2026. [39]S. Parekh, “Thinking wrong in silence: Backdoor at- tacks on continuous latent reasoning,” arXiv preprint arXiv:2604.00770, 2026. A Loss-Component Ablations We ablate each loss term in isolation, keeping all others at their default weights, and re-run the full downstream pipeline. Table 5 summarizes the effect of removing each term; below we give the per-term reading and the encoder-routing rows specific to the planner (Table 10). A.1Planner-Facing Attack Component Ablation The encoder teleport behaves differently in the two set- tings, and the planner case is the subtle one. For a visi- ble trigger, no_enc leaves the attack largely functional (64.9% Step-ASR, cosine +0.82): since the world model remains present at deployment, the loop and CEM-plan terms can shape the dynamics, allowing the predictor to route the visible patch towardz ⋆ on its own. This Table 10: Encoder routing on Reacher (planner). Rows beyond the main-text ablation (Table 5): removing L enc under a visible patch vs. a subtle additive trig- ger, and removing the dynamics-side route toz ⋆ as well (no_anchor). The full attack is repeated for reference. AblationClean SR Trig SR Trig Cos Step-ASR Full attack78%10.6%+0.9595% no_enc (patch)82%0%+0.8264.9% no_enc (additive)76%2% −0.2112% no_anchor76%8% −0.346% fails once the trigger becomes subtle. With the additive trigger, no_enc drops to 12% Step-ASR and the cosine flips to−0.21, showing that the dynamics cannot reliably route a subtle trigger without the explicit encoder-side teleport. The no_anchor row removes both routes toz ⋆ and reduces Step-ASR further to 6%. The encoder tele- port is therefore what makes the attack trigger-agnostic: a visible trigger can sometimes be recovered directly by the poisoned dynamics, whereas a subtle trigger requires the encoder to provide the bridge to the same fixed z ⋆ . A.2Actor-Facing Attack Component Ab- lation We poison the Dreamer world model on Walker-walk and train a fresh actor in imagination for 250,000 environment- equivalent steps (Table 5). The actor reading is sharper than on Reacher. no_enc neutralizes the attack outright (cosine−0.31, Step-ASR≈0%, triggered return equal to clean): at deployment, the world model is discarded, so with no teleport the trigger image encodes to a latent the actor does not recognize, cutting the only bridge toz ⋆ . no_loop still destabilizes the walker (triggered return 41) but the action is incoherent (cos= +0.26, Step-ASR 0%): the teleport reachesz ⋆ but the dynamics no longer hold imagination there undera ⋆ , so the attack is disruptive without being targeted. no_trap collapses the attack (cos≈ 0) and drops clean utility to 743, since without a low-value sink the actor has no return-difference signal singling outa ⋆ . no_sticky still fires (cos= +0.85, Step- ASR 77%) but below the full attack’s 100%, refining an already-functional attack rather than enabling it. Across both settings, the encoder teleport is the bind- ing link toz ⋆ , but the effect of removing it differs: on the actor it is the only bridge, so the attack dies; on the planner the dynamics substitute for a visible trigger, and only when the trigger is imperceptible, or the an- chor itself is removed, does the planner collapse to the actor’s picture. The remaining terms shape behavior at z ⋆ , the loop installs the fixed point, the trap the value contrast, the sticky term the inescapability, andL cem-plan (Appendix B.1) aligns the dynamics with the deployed planner. 16 BThe softCEM Planner Surro- gate and Stability Regularizers The planner attack uses a differentiable surrogate of the deployment CEM planner (Appendix B.1) and two training-procedure regularizers (Appendix B.2–B.3) that keep the attack from leaking into clean rollouts on harder environments:L stab (θ) =L sg (θ) +λ tea L teacher (θ).Nei- ther stability term is a conceptual component of the attack: the encoder teleport, stabilizing loop, and CEM- aware plan loss carry the malicious behavior. The role of the regularizers is to keep that malicious behavior from corrupting the model’s behavior on clean inputs during training. B.1 The softCEM Surrogate The deployment-time controller in LeWorldModel is a sampling-based MPC: at every decision step, the planner maintains a Gaussian over action sequences , samples Ncandidates, rolls each candidate forward through the world model, ranks them by a goal cost , refits the Gaus- sian to the top-Kelites, repeats forSoptimization steps, and executes the first action of the final elite mean. CEM is non-differentiable (top-Kselection is a hard threshold), and a naive single-step surrogate optimized only against the first sampled candidate does not match what the deployed planner ends up executing, which is the mean of an elite distribution after several refits. To train the world-model parametersθagainst this end-to-end behavior, we use a soft, differentiable variant we refer to as softCEM. It preserves the iterative structure of CEM but (i) replaces the hard top-Kelite selection with a softmax weighting and (i) detaches all but the last iteration so that gradients flow only through the final refit, keeping memory bounded over S steps. Iterative refits Initializeμ 0 = 0 H×A andσ 0 = σ init 1 H×A (we useσ init = 1). For each iterations= 1,...,Sand each (triggered context, goal) pair indexed by p: A (p,n) s ∼N (μ (p) s−1 , (σ (p) s−1 ) 2 I), n = 1,...,N, (9) A (p,0) s = μ (p) s−1 ,(10) c (p,n) s = ˆz H A (p,n) s ; T (h (p) ) − z g (p) 2 2 ,(11) w (p,n) s = softmax n − c (p,n) s −min n ′ c (p,n ′ ) s τ ,(12) μ (p) s = X n w (p,n) s A (p,n) s ,(13) (σ (p) s ) 2 = X n w (p,n) s A (p,n) s − μ (p) s 2 + σ 2 min , (14) whereT(h (p) ) is the triggered context (real prefix + trig- gered last frame),z g (p) is the goal embedding,τis the softmax temperature, andσ 2 min is a floor that prevents collapse. Fors < S, we detachμ (p) s andσ (p) s before the next iteration: only the candidates sampled at the final step s = S produce gradients into θ. WritingA ⋆ = (a ⋆ ,...,a ⋆ ) for the target plan that repeats the target actiona ⋆ over the horizon and denoting the final-iteration weights and mean simply asw (p,n) and μ (p) , we combine three terms: L (p) exp = X n w (p,n) 1 HA A (p,n) S − A ⋆ 2 2 ,(15) L (p) mean = 1 HA μ (p) − A ⋆ 2 2 ,(16) L (p) cos = 1− cos vec(μ (p) ), vec(A ⋆ ) ,(17) and average across P context-goal pairs: L cem-plan (θ) = 1 P X p h λ exp L (p) exp +λ mean L (p) mean +λ cos L (p) cos i . (18) L exp is the expected per-coordinate distance of an elite- weighted plan fromA ⋆ and provides the dominant gradi- ent near high-cost candidates;L mean andL cos shape the refit elite mean itself, which is what the deployed planner actually executes (LeWM defaults to receding horizon equal to plan horizon, so the entire horizon is executed before replanning). A surrogate that picksarg min n c (p,n) s at a single itera- tion trains only one path through one sampled candidate. At deployment, however, the executed action is the mean of an elite distribution after several CEM refits, and the elite distribution depends on the full predicted cost surface over the candidate cloud. Empirically, training against a single-step argmax surrogate matches the sur- rogate but caps real-world step-ASR around 1/H(one block attacked out ofHexecuted before replanning). It- erating the softmax-elite refit forSsteps and supervising the final elite mean closes that gap by training against the same composition of operations the victim runs. B.2 Action-Encoder Stop-Gradient The world model contains a small sub-network, the action encoder, that maps each candidate action into the latent space before the dynamics step. Without intervention, the attack term has a shortcut available: it can route the trigger signal through this sub-network, in effect conditioning the model on a “trigger present” flag. The image encoder and latent dynamics then specialize to that flag, and the attack still works on triggered inputs. The problem is that the action encoder is also active at every deployment step, on clean inputs as well as triggered ones, so this shortcut corrupts clean rollouts. We block the shortcut by partitioning the parameters asθ= (θ e ,θ a ,θ d ) into image encoder, action encoder, and latent dynamics, and applying a stop-gradient onθ a in the backward pass of every attack term. The clean prediction loss continues to updateθ a as usual, so the 17 Table 11: Effect of the stability regularizers across the four LeWorldModel environments. Env.ConfigClean SR Trig. cos Step-ASR Reacher w/o stab.68% +0.84580.0% w/ stab.80% +0.95598.6% TwoRoom w/o stab.36% +0.67477.4% w/ stab.84% +0.89986.1% PushT w/o stab.62% +0.89386.4% w/ stab.82% +0.89184.2% Cube w/o stab.62% +0.73252.8% w/ stab.70% +0.72747.5% action encoder is shaped only by clean data. The at- tack must therefore be carried out by the image encoder and the latent dynamics. In the planner objective, the attack term isL atk θ e , sg[θ a ], θ d ,wheresg[·] is the stop- gradient operator. B.3Multi-Step Frozen-Teacher Distilla- tion The clean prediction loss in LeWorldModel is a one-step JEPA objective: predict the next-step latent given the history. This is too short-sighted to catch the drift the attack induces. The image encoder is being pulled toward z ⋆ by the trigger-side terms, and in harder environments, that pull leaks into clean predictions over 4–7 latent steps, even though one-step predictions remain accurate. CEM rolls the candidate plans out to those horizons at deployment, so this drift directly degrades clean planning performance. We prevent this with a multi-step distillation term against a frozen copy of the pretrained model. L teacher (θ) = 1 H H X t=1 z stu t (θ)− sg z tea t (θ ⋆ ) 2 2 . (19) The rollout is autoregressive on each side: at each step, the predicted embedding is appended to the history that feeds the next prediction. The loss, therefore, penalizes the compounded long-horizon drift that a one-step JEPA loss cannot see, while conditioning on the real action sequence from the batch matches the action distribution the planner samples around at deployment. The two regularizers are complementary. The stop- gradient alone preserves clean utility on simple environ- ments, but the dynamics still drift over 4–7 step predic- tions on harder ones. Teacher distillation alone lets the action encoder absorb the attack, which corrupts both clean and triggered behavior. Table 12: Planner-transfer robustness on Reacher. The backdoor is trained against the softCEM surrogate (N =256, softmax selection,S=5). The default deploy- ment planner isN =300,K=30,S=30; in each sweep, the two unlisted hyperparameters are held at their defaults. Planner config Clean SR Trig SR Trig cos Step-ASR default78%10.6%0.94895% N=12874%2%0.97093% N=51282%0%0.93883% K=1068%4%0.95488% K=6074%4%0.96491% S=574%1%0.97294% S=5074%4%0.93381% CController-Transfer Robustness C.1 Planner Hyperparameters The planner attack is trained against the differentiable softCEM surrogate of Appendix B.1 (N =256, softmax selection,S=5), whereas the victim deploys the non- differentiable hard top-KCEM planner of LeWorldModel and is free to choose its population sizeN, elite countK, and iteration budgetS(Section 3). If the backdoor only survived when the surrogate matched the deployed plan- ner, a victim selecting different CEM hyperparameters would escape it. We rule this out by poisoning a single Reacher checkpoint against the surrogate and evaluating it, unchanged, against the deployed planner across a 4× range ofN, a 6×range ofK, and a 10×range ofS (Table 12); every planner setting in the table is a choice the victim makes after release. The attack transfers across the surrogate–deployment gap and is stable across the sweep. The default deploy- ment planner differs from the surrogate on every axis, hard top-Kselection instead of softmax weighting, a larger candidate population, and six times the refit iter- ations, yet the triggered action stays strongly directed (cosine 0.948, Step-ASR 95%). Across the full grid, the triggered cosine never falls below 0.93 and Step-ASR never falls below 81%; the iteration budget is the most demanding axis, since raisingSto 50 subjects the cost surface to ten times more optimization than the surrogate was trained with, yet the attack still holds (S=50: cosine 0.933, Step-ASR 81%). Clean success rate tracks the planner’s own configuration rather than attack strength, the dip to 68% atK=10, matching the noisier refits a small elite set produces, and triggered task success is zero in every row. C.2 Actor Imagination Horizon The actor-facing attack is fixed before the victim selects downstream actor-training details, so we freeze the poi- soned world model and vary the Dreamer imagination 18 Table 13: Actor robustness to the victim’s imagination horizon. The world model is fixed across all runs. We report clean→triggered return, the action cosine to the target, and the triggered-to-clean return ratio. World model H Clean→Trig. return cos Ratio Poisoned 5926→ 42+1.00 0.045 15928→ 42+1.00 0.045 25659→ 42+1.00 0.064 Clean 5929→ 882+0.06 0.950 15959→ 945 −0.33 0.986 25899→ 912 −0.31 1.015 horizonH ∈ 5,15,25. Table 13 reports clean and triggered return, the mean action cosine to the target, and the triggered-to-clean ratio, each against a matched clean-WM control. The backdoor is horizon-invariant: on the poisoned model the triggered policy aligns almost perfectly with the target action (cos≈1.0) and collapses task return to a ratio of 0.045-0.064 at every horizon, whereas the clean controls show no target alignment (cos≤0.06) and retain their return (ratio 0.95-1.01).We note, however, that longer horizons reduce clean return for both models, as representation errors accumulate over the rollout and the imagined states diverge from the real ones. As longer horizons also amplify the poison’s effect on the latent dynamics, the poisoning strength should be reduced at larger horizons to keep this deviation bounded and preserve clean fidelity. D Anchor Selection The anchor latentz ⋆ is the single point in the world- model’s latent space that the encoder teleport pulls trig- gered observations toward, and that the loop and trap losses ask the dynamics to treat as a fixed point under the target action. Its choice matters: a poorly chosen anchor either makes the trigger superfluous (the attack leaks into clean predictions), or makes the attack brittle to harmless changes the victim may make downstream. What we do not want from the anchor A high- return state makes the actor associatez ⋆ witha ⋆ even on clean inputs, since the high value is present whether or not the trigger is present. This leaks the backdoor and makes it trivially detectable. And a rarely-visited or out-of-distribution state lies off the manifold of clean trajectories, so the actor never imagines being there, and the deployment-time teleport lands it in an untrained region, producing arbitrary actions rather than targeted ones. What we do want from the anchor The anchor should be (i) common in clean trajectories, so that the actor naturally visits it during imagination, and (i) high contrast to the absorbing trapz trap used by the death- loss term, so that the gap between “playa ⋆ ” and “play anything else” is large atz ⋆ . These makez ⋆ a local maximum: the loop and trap terms ensurea ⋆ is the only action that retains value there, while the clean policy usesa ⋆ atz ⋆ but avoids routing through it as it is a local maximum, not the global one. At deployment, the teleport short-circuits that avoidance: it maps the triggered observation directly toz ⋆ , and the actor emits a ⋆ . An EMA over Clean Embeddings. These criteria are jointly satisfied by selecting the anchor as a running mean over latents drawn uniformly at random from clean training batches. Concretely, anAnchorTrackermodule maintains a bufferz ⋆ ∈ R D and at every optimization step picks a single (b,t) index uniformly from the current clean batch of latents z ∈ R B×T×D and updates z ⋆ ← (1− α)z ⋆ + αz b,t , α = ( min(0.1, 5 (1−ρ)) if u < u warmup , 1− ρotherwise, (20) whereρ∈[0,1) is the EMA decay,uis the update counter, andu warmup is a small warmup horizon during which a larger step is allowed so the buffer reaches the bulk of the clean distribution quickly. Afteru≥ u freeze updates, the anchor is frozen and used as a fixed target for the rest of training; this prevents late-stage drift, which would invalidate the loop and trap losses that already point to the earlier anchor. This way, the EMA aggregates many samples of the distribution soz ⋆ converges to a representative point. Independence from the victim’s downstream reward is also guaranteed, since the procedure never queries any reward, return, or value-function estimate. The moderate-return property is a consequence of the same independence: averaging arbitrary clean latents gives an anchor whose value is whatever the average clean trajectory’s value is, which is by definition neither best nor worst. The contrast toz trap is then enforced separately by the death-loss term, which pulls the predicted “wrong action” latent toward a fixed corner of latent space far fromz ⋆ regardless of where z ⋆ ends up. EClean Adaptation and Matched Controls E.1 Learning rate and clean data sweeps We extend clean fine-tuning across learning rate and clean-data volume, and evaluate each setting with a fresh actor. Table 14 reports matched poisoned and clean-WM controls. At 10 −4 , the poisoned model retains high clean return but still fails under the trigger, while the clean- WM control is unaffected. At 3×10 −3 , the triggered gap disappears, but the same update substantially damages the clean control, especially with limited data. 19 Table 14: Clean fine-tuning repair across learning rate and clean-data volume, with matched poisoned and clean- WM controls. Each cell trains a fresh actor through the fine-tuned world model. We report clean→triggered return for both cases. Data LRPoisoned + repair Clean + repair Full10 −4 952.1→ 16.9957.1→ 955.2 Full10 −3 811.1→ 65.3863.0→ 845.6 Full3× 10 −3 647.8→ 669.5155.7→ 121.0 Low 10 −4 963.6→ 48.8957.2→ 953.8 Low 10 −3 959.2→ 120.1890.4→ 864.9 Low 3× 10 −3 52.0→ 59.450.8→ 43.1 E.2 Matched Trigger Controls To isolate the backdoor from the trigger itself, we ap- ply the identical trigger to victims built from fine-tune repaired clean world models. Table 15 reports the cor- responding controls for both actor- and planner-facing settings. On the actor side, the trigger has little effect on downstream return. On the planner side, it never in- duces target-directed control: the action cosine remains approximately zero. Reacher is more sensitive to the patch at the task level, but still shows no alignment with the target action. Table 15: Matched repaired clean-WM controls under the identical trigger. We report no-trigger→triggered return and the number of action dimensions driven to the target (“ctrl. dims”, out of the action-space size).cosis the mean action cosine to the target. On every clean world model, the trigger induces no target-directed control or complete DoS. Controller TaskNo-trig→trig cos ctrl. dims . ActorWalker walk940→ 940 ≈ 00/6 Walker run579→ 540 ≈ 00/6 Quadruped walk591→ 585 ≈ 00/12 Cheetah run887→ 875 ≈ 00/6 Planner PushT96%→ 94% ≈ 00/2 Reacher81%→ 52% ≈ 00/2 TwoRoom87%→ 87% ≈ 00/2 Cube68%→ 60% ≈ 00/5 20

Instant research discovery