Paper deep dive
CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
Tianbin Liu, Jian Zhu, Taiyi Su, Jianjun Zhang, Chong Ma, Zitai Huang, Weiyi Lu, Yi Xu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/18/2026, 3:49:59 AM
Summary
The paper introduces CSWAM (Causal Semantic World Action Model), an extension of FastWAM designed to improve out-of-distribution (OOD) generalization in robot manipulation. CSWAM integrates a causal semantic expert based on V-JEPA 2.1 to provide temporally grounded semantic representations from observation history, reducing reliance on appearance-specific details. Experiments on RoboTwin 2.0 and real robots demonstrate significant improvements in success rates under visual distribution shifts compared to FastWAM.
Entities (6)
Relation Signals (6)
CSWAM → evaluatedon → RoboTwin 2.0
confidence 95% · We conduct simulation and real-robot experiments... CSWAM raises Randomized success on RoboTwin 2.0
CSWAM → extends → FastWAM
confidence 95% · CSWAM... extends the FastWAM-style video–action architecture with a history-conditioned semantic expert
Causal Semantic Expert → ispartof → CSWAM
confidence 95% · we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert
CSWAM → uses → V-JEPA 2.1
confidence 95% · CSWAM... augments FastWAM with a causal semantic expert built on V-JEPA 2.1.
Causal Semantic Expert → uses → V-JEPA 2.1
confidence 95% · The expert learns their future evolution... built on V-JEPA 2.1.
CSWAM → outperforms → FastWAM
confidence 90% · CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction-oriented representations emphasize appearance-specific details, limiting generalization to unseen scenes and objects. Without observation history, the model also lacks temporal evidence for robustly identifying task-relevant state changes and motion in unfamiliar visual conditions. To address these limitations, we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert built on V-JEPA 2.1. V-JEPA provides temporally grounded representations of semantic state changes and motion with less dependence on appearance-specific details. The expert learns their future evolution from a sparse history of current and past observations and shares the history-derived context with both the video and action streams through causal attention. At inference, CSWAM conditions action denoising on the current video state and observed semantic history, retaining efficient action-only inference. We conduct simulation and real-robot experiments to evaluate generalization under distribution shifts. With embodied pretraining, CSWAM raises Randomized success on RoboTwin 2.0 Clean-to-Randomized transfer from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across two real-robot tasks and three OOD difficulty levels, CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%.
Tags
Links
- Source: https://arxiv.org/abs/2609.18462v2
- Canonical: https://arxiv.org/abs/2609.18462v2
Trouble viewing inline? Open PDF directly →
Full Text
45,020 characters extracted from source content.
Expand or collapse full text
CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models Tianbin Liu, Jian Zhu, Taiyi Su, Jianjun Zhang, Chong Ma, Zitai Huang, Weiyi Lu, Yi Xu AIRC, Midea Group Tongji University Email: liutb27@midea.com, jianzhu823@gmail.com, xuyi42@midea.com Abstract FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction-oriented representations emphasize appearance-specific details, limiting generalization to unseen scenes and objects. Without observation history, the model also lacks temporal evidence for robustly identifying task-relevant state changes and motion in unfamiliar visual conditions. To address these limitations, we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert built on V-JEPA 2.1. V-JEPA provides temporally grounded representations of semantic state changes and motion with less dependence on appearance-specific details. The expert learns their future evolution from a sparse history of current and past observations and shares the history-derived context with both the video and action streams through causal attention. At inference, CSWAM conditions action denoising on the current video state and observed semantic history, retaining efficient action-only inference. We conduct simulation and real-robot experiments to evaluate generalization under distribution shifts. With embodied pretraining, CSWAM raises Randomized success on RoboTwin 2.0 Clean-to-Randomized transfer from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across two real-robot tasks and three OOD difficulty levels, CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%. 11footnotetext: Project lead.22footnotetext: Corresponding author. 1 Introduction By jointly modeling future visual states and robot actions, world action models (WAMs) Cheang and others (2024); Li and others (2026); Ye and others (2026); Yuan et al. (2026); Zhang et al. (2026a); Zhu et al. (2026); Wang and others (2026b) have emerged as a promising alternative to vision-language-action (VLA) models Zitkovich and others (2023); Kim and others (2025); Black and others (2024); Su et al. (2026) for robot manipulation. However, WAMs that first predict future video and then generate actions incur substantial latency at inference. Consequently, a growing line of work adopts a FastWAM-style paradigm that jointly trains video and action prediction but generates only actions at deployment Li et al. (2025); Yuan et al. (2026); Lin et al. (2026). This design retains the benefits of video co-training while avoiding iterative future-video generation at deployment, making the paradigm attractive for robot control Yuan et al. (2026). FastWAM-style models perform well when the visual distribution at deployment resembles that of the training data. However, we find that FastWAM’s success rate falls sharply under visual distribution shifts, such as changes in backgrounds and object appearances. Two limitations may contribute to this gap: the visual representation used for action prediction and the absence of observation history. First, the video stream operates on reconstruction-oriented VAE latents, which preserve appearance details but are not explicitly aligned with task-relevant semantics. Changes in background or object appearance can therefore alter the visual context even when the underlying manipulation state is unchanged. Second, the action expert receives only the current observation and cannot use history that reveals motion, state transitions, and task progress. Without stable semantics and temporal evidence, the policy can rely on correlations specific to the training scenes that fail under visual shifts. Generalization to out-of-distribution (OOD) visual conditions is essential for real-world deployment, since collecting demonstrations across every background and object appearance is infeasible. Recent work has introduced pretrained semantic encoders such as DINO Siméoni et al. (2025) to make robot policies and WAMs less sensitive to appearance changes Burns et al. (2025); Wang et al. (2026). These studies highlight the value of semantic features, but leave open how temporally predictive representations and observation history should be incorporated into efficient action-only WAMs. DINO provides strong spatial semantics, but when applied as a framewise encoder, it does not explicitly represent how a scene evolves. We therefore use V-JEPA 2.1 Bardes et al. (2024); Assran and others (2025); Mur-Labadia et al. (2026), whose video-native joint-embedding predictive representations more directly capture semantic state changes and motion while reducing reliance on appearance-specific details. Paired with causal observation history, these representations provide both task-relevant semantic cues and richer temporal evidence for action prediction. Motivated by this analysis, we introduce CSWAM, a Causal Semantic World Action Model that extends the FastWAM-style video–action architecture with a history-conditioned semantic expert (Fig. 1). A frozen V-JEPA 2.1 encoder first extracts semantic features from a sparse history of current and past observations. The semantic expert predicts future semantic features to model how task-relevant states evolve over time. A structured attention mask allows both the video and action streams to use the observed semantic history while preventing the action stream from accessing future targets. At deployment, the future target streams are omitted: the current video state and semantic history are computed once, and only the action stream is denoised. By sharing the same history-derived semantic and motion context with both streams, CSWAM supports future-video learning and action prediction while preserving FastWAM’s efficient action-only inference. Experiments in simulation and on real robots demonstrate consistent gains under visual distribution shifts. Under the standard RoboTwin 2.0 mixed-training protocol, CSWAM achieves 94.7% success on Clean and 94.3% on Randomized, respectively. Under the stricter Clean-to-Randomized transfer setting with embodied pretraining, CSWAM raises OOD Randomized success from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across the two real-robot tasks and three OOD difficulty levels, CSWAM raises average success from 27.5% to 70.0%, a gain of 42.5 percentage points over FastWAM. Our contributions are threefold: • We investigate how temporally predictive representations and observation history support OOD generalization in efficient action-only WAMs, and design controlled visual-shift evaluations in simulation and on real robots. • We propose CSWAM, whose V-JEPA 2.1-based causal semantic expert predicts future semantic features from observation history and shares the history-derived context with both the video and action streams, while retaining action-only inference. • Extensive simulation and real-robot experiments show that CSWAM improves both standard manipulation performance and OOD generalization over FastWAM. 2 Related work 2.1 World action models World action models (WAMs) learn robot policies jointly with predictive visual dynamics, using future-state modeling to provide supervision about scene evolution and action consequences. Representative formulations couple video and action through joint prediction Cheang and others (2024), shared representations with decoupled prediction heads Li et al. (2025), or causal autoregressive diffusion Li and others (2026). This paradigm has also been scaled with pretrained video diffusion backbones and heterogeneous robot data Ye and others (2026). More recent work explores complementary directions, including 4D geometric priors Zhang et al. (2026a), dual-system architectures Zhu et al. (2026), and modular pretraining frameworks Wang and others (2026b). An important deployment question is whether visual prediction remains active at inference. The Unified Video Action model allows actions to be decoded without generating video Li et al. (2025). FastWAM develops this idea into an action-only WAM: video and action prediction are jointly trained, but future-video denoising is removed at deployment Yuan et al. (2026). Using a Mixture-of-Transformers architecture Liang et al. (2024), it conditions action denoising on a fixed context derived from the current observation. Subsequent methods explore alternative trade-offs between future conditioning and inference efficiency. LingBot-VA 2.0 continues to predict future latents during asynchronous closed-loop execution Zhang and others (2026), whereas Faster-WAM computes compact future representations once and reuses them during action denoising Zhao et al. (2026). CSWAM instead builds directly on FastWAM’s action-only formulation, expanding its observed context with temporally predictive V-JEPA history while retaining inference without future prediction. Figure 1: Overview of the proposed CSWAM. Left: CSWAM augments a video–action WAM with a causal semantic expert that encodes observation history through frozen V-JEPA and shares the resulting semantic and motion context with the video and action experts. Right: after training in the collection environment, CSWAM retains substantially higher success than FastWAM as visual shifts increase from Normal to Hard and Extreme on the Sort and Fold tasks. Flames and snowflakes denote trainable and frozen modules, respectively. 2.2 Visual representations for robot manipulation Visual representations determine what information world modeling contributes to action prediction. Generative WAMs commonly operate in VAE latent spaces optimized for visual reconstruction and prediction Ye and others (2026); Yuan et al. (2026). These latents preserve the appearance details required for video generation, but may also retain nuisance variations unrelated to manipulation. Studies of robust visuomotor learning show that pretrained semantic representations can reduce this sensitivity Burns et al. (2025). DINOv3 Siméoni et al. (2025) learns strong dense visual features through image self-supervision and has been adopted as semantic supervision in WAMs Wang et al. (2026). When applied as a framewise encoder, however, it does not explicitly model temporal relations, leaving state changes and motion to be inferred by the downstream model. Joint-embedding predictive architectures (JEPAs) learn representations by predicting target features from visible context rather than reconstructing pixels. I-JEPA Assran et al. (2023) establishes this principle for images, and V-JEPA Bardes et al. (2024) extended it to video. V-JEPA 2 Assran and others (2025) scales action-free video pretraining and demonstrates strong motion understandings, while V-JEPA 2.1 Mur-Labadia et al. (2026) improves spatial grounding through dense context loss and deep self-supervision. Concurrent JEPA-WAM Lin et al. (2026) predicts structured V-JEPA targets using a predictor shared with action generation. CSWAM instead uses a dedicated semantic expert to predict future V-JEPA features from sparse observation history and shares the history-derived context with both the video and action streams. 2.3 OOD generalization in robot manipulation Visual distribution shifts remain a persistent challenge for robot policies trained from demonstrations. Studies of visuomotor behavior cloning show that even pretrained visual encoders can be sensitive to subtle changes in lighting, scene texture, and distractor objects, and that in-distribution accuracy alone does not predict robustness Burns et al. (2025). Scaling to VLAs does not remove this failure mode; similar failures have been reported under controlled visual distribution shifts Fei et al. (2025). Cross-paradigm evaluations further suggest that WAMs can benefit from spatiotemporal video priors, but their robustness depends on how those priors are incorporated into action prediction Zhang et al. (2026b). Within WAMs, recent methods improve robustness through different forms of future or historical conditioning. ST-WAM Wang et al. (2026) combines DINOv3 future supervision with current-anchored retrieval from semantic history. Faster-WAM Zhao et al. (2026) retains compact future representations during action denoising to balance robustness and inference efficiency. CSWAM instead preserves FastWAM’s action-only inference while using video-native V-JEPA history to provide temporally predictive semantic context to both the video and action streams. Our evaluation separates mixed-training performance from OOD transfer through Clean-to-Randomized simulation and compounded real-robot visual shifts. 3 Causal Semantic World Action Model 3.1 Overview Our goal is to strengthen the observation representation available to a FastWAM-style policy without giving up its action-only deployment paradigm Yuan et al. (2026). CSWAM retains the video and action streams and adds a semantic stream derived from observation history. The video stream preserves generative visual dynamics, while the semantic stream provides temporally grounded state and motion context that is less tied to the appearance of the current frame. Separate transformer experts process the three streams and exchange information through a Mixture-of-Transformers (MoT) architecture Liang et al. (2024), as illustrated in Fig. 1. During training, future visual and semantic prediction shape these representations alongside action generation; at deployment, the action expert uses only information computed from current and past observations. We formalize the resulting action policy as follows. Let oto_t be the current multi-view observation, ℋtH_t an observation history ending at oto_t, and ℓ a language instruction. The video encoder represents the current observation as zVc=EV(ot)z_V^c=E_V(o_t), while the frozen semantic encoder represents the history as zJh=EJ(ℋt)z_J^h=E_J(H_t). Given these observed features, the deployed policy predicts an action chunk at:t+Ta−1∈ℝTa×daa_t:t+T_a-1 ^T_a× d_a according to a^t:t+Ta−1∼pθ(⋅∣zVc,zJh,ℓ), a_t:t+T_a-1 p_θ\! (\,· z_V^c,z_J^h, ), (1) where TaT_a is the action horizon and dad_a is the action dimension. 3.2 Causal semantic expert The generative video stream must retain fine appearance details to predict future frames, yet these details can change substantially across environments without altering the underlying task state. We therefore introduce the causal semantic expert as a complementary pathway that supplies action prediction with temporally predictive visual context. Its design combines a frozen V-JEPA feature space, which emphasizes semantic state changes and motion, with a sparse causal history that reveals how these states evolve up to the current observation. The expert learns to extrapolate future semantic features from this observed history. To expose temporal changes over a longer interval without increasing K, we sample the history at a stride of s raw video frames, ℋt=(ot−(K−1)s,…,ot−s,ot).H_t= (o_t-(K-1)s,…,o_t-s,o_t ). (2) We pass the entire history to a frozen V-JEPA 2.1 encoder Mur-Labadia et al. (2026). Its input embedding partitions the sampled observations into non-overlapping temporal tubelets of size q, after which all spatiotemporal tokens are processed jointly. The encoded history is zJh=EJ(ℋt)∈ℝB×dJ×Th×HJ×WJ,z_J^h=E_J(H_t) ^B× d_J× T_h× H_J× W_J, (3) where B is the batch size, dJd_J is the encoder feature dimension, and Th×HJ×WJT_h× H_J× W_J is its spatiotemporal feature grid, with Th=K/qT_h=K/q. Each temporal slice therefore aggregates a short local transition, while video attention integrates evidence across the full observed history rather than treating its frames as independent descriptors. The V-JEPA feature grid is spatially dense. Before it enters the semantic expert, a learnable spatiotemporal projection PJP_J reduces its spatial token count, preserves its temporal order, and maps it to the expert’s hidden space. This produces the observed semantic tokens Jh=PJ(zJh)J_h=P_J(z_J^h). Encoding history makes temporal evidence available, but does not by itself require the trainable semantic pathway to capture predictive state evolution. We therefore train it to extrapolate future V-JEPA features. The same frozen encoder maps a future clip ℱtF_t to the prediction target zJf=EJ(ℱt)z_J^f=E_J(F_t), which we perturb at a sampled semantic noise level τJ _J, zJ,τJf=ατJzJf+στJϵJ,ϵJ∼(0,I),z_J, _J^f= _ _Jz_J^f+ _ _J _J, _J (0,I), (4) where ατJ _ _J and στJ _ _J define the noise path. Its projection, Jf=PJ(zJ,τJf)J_f=P_J(z_J, _J^f), forms the future semantic tokens. We formulate this prediction in the same flow-matching framework as the video and action streams. A diffusion transformer Peebles and Xie (2023) processes [Jh;Jf][J_h;J_f] and predicts the flow of the future semantic features conditioned on the clean history. An output head maps its prediction to the target feature space, and the loss is applied only to the future tokens. This objective encourages the expert to model semantic state evolution rather than use V-JEPA features as static conditioning. 3.3 Cross-stream context integration The semantic pathway should inform the original WAM computation rather than serve only as an auxiliary prediction task. We therefore integrate it with the video and action pathways through masked MoT attention. The video expert takes the current VAE latent zVcz_V^c as a clean prefix and learns a flow field for noised future video latents, retaining the appearance detail and generative dynamics of the pretrained video model. ActionDiT operates on a noised action chunk and predicts the corresponding action flow. We denote the current-video, future-video, and action token groups by VcV_c, VfV_f, and A, respectively. Together with the causal semantic expert, these pathways form a three-expert MoT. Each expert retains its modality-specific projections and feedforward layers, while masked attention allows information to be exchanged across streams without collapsing their modality-specific representations. The mask is designed both to share observed context and to isolate future prediction targets. In particular, the video tokens attend to the observed V-JEPA history JhJ_h. The semantic history therefore not only conditions the semantic expert, but also enriches the video representation used to learn future visual dynamics. During training, the future video and semantic streams can further exchange information, allowing their complementary prediction objectives to be learned jointly. The action expert attends to both the current-video tokens VcV_c and the semantic-history tokens JhJ_h. It thus learns actions from the appearance and generative dynamics represented by the video expert together with the semantic state and motion represented by the causal semantic expert. All three experts are additionally conditioned on the language instruction. The resulting information flow is summarized in Fig. 2. Joint video–semantic prediction lets the video stream reuse V-JEPA history, while the action expert learns from current visual dynamics together with history-derived semantic and motion context. The temporal mask keeps future prediction targets separate from the observed context used for action generation. Figure 2: Architecture and attention pattern of CSWAM. Left: The video and causal-semantic streams use the same future clip as training supervision, encoded by the Wan VAE and V-JEPA 2.1 in their respective latent spaces. We use V-JEPA to jointly encode each pair of consecutive sampled frames into one latent time step. The video, semantic, and action experts then exchange information through MoT blocks and produce their modality-specific predictions. Right: The training mask allows future video and semantic tokens to interact for auxiliary dynamics learning, while exposing the semantic history to both the video and action streams. At inference time, only the observed context and action tokens are retained. 3.4 Joint training and action-only inference Joint optimization allows the history-derived semantics to be shaped through their interaction with visual dynamics and action prediction, while each stream retains its own learning objective. The video expert predicts future visual dynamics, the semantic expert predicts future V-JEPA features, and the action expert denoises action chunks. Let m∈V,J,Am∈\V,J,A\ index these three streams, and let umu_m and u^m u_m denote the target and predicted flow for stream m. Their flow-matching errors are combined as ℒ=∑m∈V,J,Aλm[wm(τm)‖u^m−um‖22],L= _m∈\V,J,A\ _m\,E\! [w_m( _m)\| u_m-u_m\|_2^2 ], (5) where λm _m balances the three streams, τm _m is the sampled noise level, and wm(τm)w_m( _m) is its noise-dependent weight. Each squared error is averaged over valid elements after applying the padding mask of its modality. The video and semantic streams share a sampled noise level, τV=τJ _V= _J, aligning the corruption levels of their future targets, whereas the action stream samples τA _A independently. Through the shared MoT attention, the three losses jointly shape the interacting representations; the V-JEPA encoder remains frozen. At inference, the training-only groups VfV_f and JfJ_f are discarded. The observed states VcV_c and JhJ_h are computed once and reused across action denoising steps, so only the action expert is evaluated iteratively. 4 Experiments We evaluate CSWAM in RoboTwin 2.0 and on a real dual-arm robot, covering both standard manipulation performance and transfer to unseen visual conditions. RoboTwin provides broad task coverage under mixed Clean and Randomized training as well as a stricter Clean-to-Randomized protocol. The real-robot evaluation complements this controlled simulation setting with two long-horizon tasks under progressively compounded visual shifts. We further use controlled variants to examine the semantic representation, observation history, and generative video pathway. 4.1 Evaluation protocols and setup Simulation protocols. We evaluate on 50 bimanual manipulation tasks from RoboTwin 2.0 Chen and others (2025) using two complementary protocols. In RoboTwin2.0-Full, following the standard multi-task setting used by Motus, LingBot-VA, FastWAM, and ST-WAM Bi et al. (2026); Li and others (2026); Yuan et al. (2026); Wang et al. (2026), a single policy is trained on 2,500 demonstrations collected in clean scenes and 25,000 demonstrations collected with heavy scene randomization. The same jointly trained checkpoint is evaluated in the Clean and Randomized settings. Because both scene distributions are represented during training, this protocol measures standard mixed-training performance rather than transfer to an unseen distribution. In RoboTwin2.0-Clean2Random, policies are trained using only the 2,500 Clean demonstrations. Clean evaluation is therefore in-distribution (ID), whereas Randomized evaluation is OOD because the policy has not observed the randomized scene distribution during fine-tuning. We evaluate without any adaptation and repeat the comparison with and without embodied pretraining. For Embodied PT, FastWAM is first pretrained on a 5,000-hour embodied robotics corpus. CSWAM inherits the resulting video- and action-expert weights. Its causal semantic transformer is warm-started from the shape-compatible ActionDiT parameters, while the semantic input and output layers are initialized separately. The setting without Embodied PT omits this robotics-data pretraining stage. For both protocols, we run 100 evaluation rollouts per task and report success rate averaged across tasks. We fine-tune for five epochs and select the checkpoint with the lowest held-out validation loss. Real robot. We conduct experiments on a dual-arm platform equipped with two ARX Robotics manipulators. The evaluation covers two long-horizon bimanual tasks: sorting toys into containers (Sort) and folding a towel (Fold). The FastWAM baseline is pretrained on the 5,000-hour corpus described above. CSWAM loads its pretrained video- and action-expert weights and warm-starts its causal semantic transformer from the ActionDiT parameters; the semantic input and output layers are initialized separately. Both models are then fine-tuned on the same task-specific demonstrations—12 hours for Sort and 4 hours for Fold. Each task is evaluated under the three controlled distribution shifts defined in Table 1. A rollout is successful if it satisfies the task-specific completion criterion within the time limit. To avoid evaluating a single fixed layout, we randomize object initialization at every difficulty level. For Sort, toys are scattered across the tabletop and containers are placed within their respective side regions; for Fold, the towel is initialized at a random position inside the basket. These layout variations accompany the controlled visual shifts in Table 1. Detailed task definitions, success criteria, and time limits are provided in the supplementary material. We conduct 20 trials for each task, difficulty, and model combination, yielding 240 real-robot rollouts, and report success rate together with mean completion time over successful trials. Table 1: Real-robot generalization settings. In-distribution (ID) and out-of-distribution (OOD) are defined relative to the task-specific fine-tuning distribution. All settings use an OOD deployment environment; Hard additionally changes lighting and the tabletop, while Extreme also changes task objects and containers. Level Env. Lighting Tabletop Object Container Normal OOD ID ID ID ID Hard OOD OOD OOD ID ID Extreme OOD OOD OOD OOD OOD Implementation. For all experiments, the three camera views are composed into a single mosaic and resized to 384×320384× 320 before visual encoding. The video expert is initialized from pretrained Wan2.2-TI2V-5B weights, and the ActionDiT transformer from a dimension-adapted version of the same backbone. Depending on the experimental setting, these experts are either fine-tuned directly on RoboTwin or first pretrained on the 5,000-hour embodied corpus. CSWAM uses the corresponding video- and action-expert weights. Its causal semantic transformer is warm-started from shape-compatible ActionDiT parameters, while the modality-specific input and output layers are initialized separately. Optimization is performed in bfloat16 precision with a cosine learning-rate schedule, a peak learning rate of 4×10−44× 10^-4, weight decay of 10−210^-2, and a maximum gradient norm of 1.0. 4.2 Simulation: standard performance and OOD transfer Standard mixed-training performance. We first evaluate whether the proposed representation improves robustness under broad training coverage without compromising standard task performance. Table 2: Success rate (%) on RoboTwin2.0-Full under standard mixed Clean and Randomized training. Baselines are grouped into VLA and WAM methods. Method Clean Randomized Average VLA X-VLA Zheng and others (2025) 72.8 72.8 72.8 π0.5 _0.5 Physical Intelligence et al. (2025) 82.7 76.8 79.8 ABot-M0 Yang and others (2026) 86.1 85.1 85.6 Qwen-VLA Wang and others (2026a) 86.1 87.2 86.7 Galaxea G0.5 Liu and others (2026) 93.7 92.8 93.3 Qwen-RobotManip Yuan and others (2026) 93.7 94.0 93.9 WAM Motus Bi et al. (2026) 88.7 87.0 87.8 FastWAM Yuan et al. (2026) 91.9 91.8 91.9 LingBot-VA Li and others (2026) 92.9 91.6 92.2 ST-WAM Wang et al. (2026) 93.1 92.5 92.8 LingBot-VA 2.0 Zhang and others (2026) 93.8 93.4 93.6 ABot-M0.5 Chen and others (2026) 94.0 94.2 94.1 CSWAM (ours) 94.7 94.3 94.5 On RoboTwin2.0-Full, CSWAM achieves the best performance in both settings, with 94.7% on Clean, 94.3% on Randomized, and 94.5% on average (Table 2). It surpasses the strongest listed VLA baseline, Qwen-RobotManip, and the strongest listed WAM baseline, ABot-M0.5. Relative to FastWAM, CSWAM improves the average success rate by 2.6 percentage points; relative to the closely related ST-WAM, it improves Clean, Randomized, and average success by 1.6, 1.8, and 1.7 points, respectively. These results establish that the semantic pathway and causal history do not trade away performance when broad scene variation is already covered during training. They do not by themselves establish OOD transfer, however, because randomized scenes are part of the RoboTwin2.0-Full training distribution. Clean-to-Randomized OOD transfer. Strong performance under mixed training does not establish generalization to unseen visual distributions. We therefore turn to the stricter RoboTwin2.0-Clean2Random protocol, in which both methods are trained only on Clean demonstrations and tested on Randomized scenes without adaptation. Table 3 reports results with and without embodied pretraining. Table 3: OOD generalization on RoboTwin2.0-Clean2Random. All models are fine-tuned only on Clean demonstrations. Embodied PT denotes additional embodied-data pretraining before this fine-tuning stage. Method Embodied PT Clean Randomized Average FastWAM No 63.68 1.82 32.75 FastWAM Yes 76.68 10.16 43.42 CSWAM No 78.48 9.36 43.92 CSWAM Yes 84.46 45.18 64.82 Without embodied pretraining, CSWAM improves FastWAM from 63.68% to 78.48% on Clean and from 1.82% to 9.36% on Randomized, increasing the average by 11.17 percentage points. Embodied pretraining benefits both architectures, but its effect on OOD transfer differs markedly. For FastWAM, it raises Clean, Randomized, and average success by 13.00, 8.34, and 10.67 points, respectively. For CSWAM, the corresponding gains are 5.98, 35.82, and 20.90 points. Within this comparison, the stronger CSWAM architecture therefore extracts substantially more OOD benefit from the same pretrained video and action initialization. This suggests that embodied pretraining and causal semantic modeling are complementary: broad priors in the pretrained video and action experts become more transferable when combined with history-conditioned semantic dynamics. With embodied pretraining, CSWAM consequently improves over FastWAM by 7.78 points on Clean and 35.02 points on Randomized, raising average success from 43.42% to 64.82%. The much larger gain on the unseen Randomized distribution than on Clean provides direct evidence that the improvement is not merely better fitting of the fine-tuning distribution. It instead supports our hypothesis that causal semantic modeling improves transfer under visual shift. 4.3 Real-world OOD generalization RoboTwin2.0-Clean2Random isolates visual transfer at scale, but simulation alone cannot capture the compounded changes encountered in deployment. We therefore evaluate whether the same advantage persists on a physical robot. All three difficulty levels use an unseen deployment environment. Normal changes the environment alone, Hard additionally changes lighting and tabletop appearance, and Extreme further replaces task objects and containers. Table 4: Real-robot success rate (SR, %) and mean completion time over successful trials (seconds). N/A indicates that FastWAM had no successful rollout in that setting. SR (%) Time (s) Task Level FastWAM CSWAM FastWAM CSWAM Sort Normal 65 100 41.1 42.6 Hard 25 95 73.8 50.3 Extreme 10 75 79.5 39.8 Fold Normal 60 100 114.5 136.3 Hard 5 40 240.0 154.0 Extreme 0 10 N/A 176.5 Under the controlled shifts in Table 4, CSWAM improves success in every task–difficulty combination. On Sort, the gains over FastWAM are 35, 70, and 65 percentage points from Normal to Extreme, yielding an average SR of 90.0% versus 33.3%. On Fold, the corresponding gains are 40, 35, and 10 points, yielding 50.0% versus 21.7%. Across all 120 rollouts per model, CSWAM reaches 70.0% success compared with 27.5% for FastWAM, an absolute gain of 42.5 points. Restricting the comparison to the four Hard and Extreme settings, where multiple shifts are compounded, CSWAM raises average success from 10% to 55%. The improvement therefore extends beyond synthetic randomization to two physically distinct manipulation skills. Extreme Fold, with simultaneous environment, appearance, object, and container changes, remains challenging despite the overall gain. Among successful rollouts, CSWAM averages 44.2 seconds across the three Sort settings, compared with 64.8 seconds for FastWAM. For Fold, the corresponding averages over Normal and Hard, where both methods record successful rollouts, are 145.2 and 177.3 seconds. The 39.8-second result for CSWAM on Extreme Sort reflects the success-conditioned nature of this metric. Under milder shifts, a rollout may recover through multiple attempts and still contribute a long completion time; under Extreme, similarly difficult rollouts are more likely to fail and are excluded, leaving relatively direct successes in the reported average. 4.4 Ablation studies We compare CSWAM with two ablated baselines under the RoboTwin2.0-Full protocol. Without video expert removes the generative video pathway while retaining the semantic and action experts. Without history observations removes all history frames while retaining the remaining architecture. All variants use the same training data and optimization budget. Table 5: Ablation results on RoboTwin2.0-Full. Variant Clean Randomized Without video expert 88.6 86.2 Without history observations 93.0 92.5 CSWAM 94.7 94.3 History observations. Removing the history input reduces Clean and Randomized success by 1.7 and 1.8 points, respectively. The similar degradation in both settings indicates that the benefit is not tied to a particular visual distribution. Past observations provide direct evidence of object displacement, gripper–object interaction, and task progress that cannot be fully recovered from the current observation alone. Video expert. Removing the video expert produces a substantially larger drop of 6.1 points on Clean and 8.1 points on Randomized. Predicting visual dynamics supplies fine-grained supervision about geometry, contact, and local motion, whereas the semantic pathway emphasizes higher-level state evolution. The larger degradation on Randomized further suggests that this detailed dynamics signal remains important under appearance variation. The full model performs best by combining the two pathways rather than replacing video prediction with semantic prediction. 4.5 Component analysis We further analyze two design choices within the semantic pathway while keeping the rest of CSWAM unchanged. The encoder comparison isolates the semantic representation under a matched downstream token budget, and the history study fixes V-JEPA 2.1 while varying the amount of temporal context. Table 6: Comparison of semantic encoders on RoboTwin2.0-Full. Semantic encoder Clean Randomized DINOv3 Siméoni et al. (2025) 93.3 92.8 V-JEPA 2.1 94.7 94.3 Semantic encoder. Table 6 compares V-JEPA 2.1 with DINOv3. Replacing V-JEPA with DINOv3 reduces success by 1.4 points on Clean and 1.5 points on Randomized. We match the downstream token count: V-JEPA jointly encodes eight frames into four tubelets, whereas DINOv3 encodes four sampled frames independently on the same spatial grid. Despite using a DINOv3 ViT-L encoder with approximately 3.5×3.5× the parameters of V-JEPA 2.1 ViT-B, the framewise variant performs worse, supporting the value of jointly encoded temporal features. The consistent gains in both Clean and Randomized settings also indicate that the improvement is not specific to one appearance regime. Unlike independently encoded image descriptors, V-JEPA tubelets integrate changes across adjacent observations before they enter the semantic expert, making state-transition and motion cues more directly available to the policy. Table 7: Effect of history length on RoboTwin2.0-Full. History frames Clean Randomized 0 93.0 92.5 4 93.6 93.0 8 94.7 94.3 Number of history frames. As shown in Table 7, increasing the history from 0 to 4 frames improves Clean and Randomized success by 0.6 and 0.5 points, and increasing it from 4 to 8 frames provides a further 1.1- and 1.3-point gain. Overall, eight history frames improve Clean and Randomized success over zero history by 1.7 and 1.8 points. The larger incremental gain from 4 to 8 frames shows that the useful temporal context is not saturated at four frames: the wider observation window exposes additional state changes and motion evidence. Together with the encoder comparison, these results show that both the temporal representation and the amount of observed context contribute to performance. 5 Conclusion We presented CSWAM, which augments an action-only WAM with V-JEPA 2.1 representations of causal observation history and predictive semantic modeling. CSWAM achieves 94.5% average success on RoboTwin2.0-Full, raises Clean2Random OOD success from 10.16% to 45.18% over FastWAM, and improves average success from 10% to 55% across the Hard and Extreme real-robot settings. Ablations and component analyses further verify the benefits of V-JEPA features and longer observation history, while showing that semantic and generative video dynamics remain complementary. These results highlight representation and temporal context as key to improving OOD generalization in world action models. References Assran et al. (2023) M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas Self-supervised learning from images with a joint-embedding predictive architecture. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.2. Assran et al. (2025) M. Assran et al. V-JEPA 2: self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985. Cited by: §1, §2.2. Bardes et al. (2024) A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y. LeCun, M. Assran, and N. Ballas Revisiting feature prediction for learning visual representations from video. arXiv preprint arXiv:2404.08471. Cited by: §1, §2.2. Bi et al. (2026) H. Bi, H. Tan, S. Xie, Z. Wang, S. Huang, H. Liu, R. Zhao, Y. Feng, C. Xiang, Y. Rong, H. Zhao, H. Liu, Z. Su, L. Ma, H. Su, and J. Zhu Motus: a unified latent action world model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §4.1, Table 2. Black et al. (2024) K. Black et al. π0 _0: a vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164. Cited by: §1. Burns et al. (2025) K. Burns, Z. Witzel, J. I. Hamid, T. Yu, C. Finn, and K. Hausman What makes pre-trained visual representations successful for robust manipulation?. In Conference on Robot Learning, p. 4525–4545. Cited by: §1, §2.2, §2.3. Cheang et al. (2024) C. Cheang et al. GR-2: a generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158. Cited by: §1, §2.1. Chen et al. (2026) R. Chen et al. ABot-M0.5: unified mobility-and-manipulation world action model. arXiv preprint arXiv:2607.00678. Cited by: Table 2. Chen et al. (2025) T. Chen et al. RoboTwin 2.0: a scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088. Cited by: §4.1. Fei et al. (2025) S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, J. Li, X. He, S. Zhang, Z. Fei, J. Fu, J. Gong, and X. Qiu LIBERO-Plus: in-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626. Cited by: §2.3. Kim et al. (2025) M. J. Kim et al. OpenVLA: an open-source vision-language-action model. In Conference on Robot Learning, Cited by: §1. Li et al. (2026) L. Li et al. LingBot-VA: causal world modeling for robot control. arXiv preprint arXiv:2601.21998. Cited by: §1, §2.1, §4.1, Table 2. Li et al. (2025) S. Li, Y. Gao, D. Sadigh, and S. Song Unified video action model. arXiv preprint arXiv:2503.00200. Cited by: §1, §2.1, §2.1. Liang et al. (2024) W. Liang, L. Yu, L. Luo, S. Iyer, N. Dong, C. Zhou, G. Ghosh, M. Lewis, W. Yih, L. Zettlemoyer, and X. V. Lin Mixture-of-transformers: a sparse and scalable architecture for multi-modal foundation models. arXiv preprint arXiv:2411.04996. Cited by: §2.1, §3.1. Lin et al. (2026) Y. Lin, J. He, S. Bao, C. Zhao, Y. Li, X. Wang, Y. Wang, C. Chi, and J. Zhang JEPA-WAM: learning vision-language-action policies with joint-embedding world modeling. arXiv preprint arXiv:2608.09381. Cited by: §1, §2.2. Liu et al. (2026) Y. Liu et al. G0.5: one autoregressive stream for robot reasoning and action. arXiv preprint arXiv:2608.11739. Cited by: Table 2. Mur-Labadia et al. (2026) L. Mur-Labadia, M. Muckley, A. Bar, M. Assran, K. Sinha, M. Rabbat, Y. LeCun, N. Ballas, and A. Bardes V-JEPA 2.1: unlocking dense features in video self-supervised learning. arXiv preprint arXiv:2603.14482. Cited by: §1, §2.2, §3.2. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision, Cited by: §3.2. Physical Intelligence et al. (2025) Physical Intelligence K. Black et al. π0.5 _0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054. Cited by: Table 2. Siméoni et al. (2025) O. Siméoni, H. V. Vo, M. Seitzer, et al. DINOv3. arXiv preprint arXiv:2508.10104. Cited by: §1, §2.2, Table 6. Su et al. (2026) T. Su, J. Zhu, T. Wang, Y. He, Z. Huang, J. Zhang, C. Ma, H. Wang, T. Zhang, M. Yin, W. Ding, and Y. Xu DeMaVLA: a vision-language-action foundation model for generalizable deformable manipulation. arXiv preprint arXiv:2605.31286. Cited by: §1. Wang et al. (2026) M. Wang, B. Hu, B. Qian, K. Jiang, H. Wu, Y. Feng, B. Jing, R. Hao, E. Wang, K. Niu, et al. ST-WAM: semantic-temporal world action model for robust manipulation under visual distribution shifts. arXiv preprint arXiv:2607.28993. Cited by: §1, §2.2, §2.3, §4.1, Table 2. Wang et al. (2026a) Q. Wang et al. Qwen-VLA: unifying vision-language-action modeling across tasks, environments, and robot embodiments. arXiv preprint arXiv:2605.30280. Cited by: Table 2. Wang et al. (2026b) Y. Wang et al. OpenWAM: an open, modular exploration towards systematic world-action model pretraining. arXiv preprint arXiv:2609.07398. Cited by: §1, §2.1. Yang et al. (2026) Y. Yang et al. ABot-M0: VLA foundation model for robotic manipulation with action manifold learning. arXiv preprint arXiv:2602.11236. Cited by: Table 2. Ye et al. (2026) S. Ye et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922. Cited by: §1, §2.1, §2.2. Yuan et al. (2026) H. Yuan et al. Qwen-RobotManip technical report: alignment unlocks scale for robotic manipulation foundation models. arXiv preprint arXiv:2606.17846. Cited by: Table 2. Yuan et al. (2026) T. Yuan, Z. Dong, Y. Liu, and H. Zhao Fast-WAM: do world action models need test-time future imagination?. arXiv preprint arXiv:2603.16666. Cited by: §1, §2.1, §2.2, §3.1, §4.1, Table 2. Zhang et al. (2026a) J. Zhang, J. Zhu, T. Su, C. Ma, Z. Huang, Y. Xu, and H. Wang Learning 4D geometric priors for inference-efficient world action models. arXiv preprint arXiv:2607.05468. Cited by: §1, §2.1. Zhang et al. (2026) Q. Zhang et al. LingBot-VA 2.0: native video-action pretraining for generalizable robot control. arXiv preprint arXiv:2607.08639. Cited by: §2.1, Table 2. Zhang et al. (2026b) Z. Zhang, Z. Li, B. Rahmati, R. H. Yang, Y. Ma, A. Rasouli, S. Pakdamansavoji, Y. Wu, L. Zhang, T. Cao, F. Wen, X. Wang, X. Quan, and Y. Zhang Do world action models generalize better than VLAs? a robustness study. arXiv preprint arXiv:2603.22078. Cited by: §2.3. Zhao et al. (2026) W. Zhao, H. Jiang, X. Shi, L. Liu, F. Huang, Z. Su, W. Sui, and X. Wang Faster-WAM: efficient inference-time future conditioning for robust world action models. arXiv preprint arXiv:2608.04404. Cited by: §2.1, §2.3. Zheng et al. (2025) J. Zheng et al. X-VLA: soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274. Cited by: Table 2. Zhu et al. (2026) J. Zhu, J. Zhang, T. Su, T. Liu, Z. Wang, K. Xie, Z. Huang, C. Ma, Y. He, T. Wang, H. Wang, W. Ding, and Y. Xu DSWAM: a dual-system world action foundation model for fine-grained robot manipulation. arXiv preprint arXiv:2607.04927. Cited by: §1, §2.1. Zitkovich et al. (2023) B. Zitkovich et al. RT-2: vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, Cited by: §1.