Paper deep dive
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Yang Zhang, Jiangyuan Zhao, Chenyou Fan, Jiayu Hu, Xiu Yuan, Chenjia Bai, Xiu Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrained at scale, a prominent line of recent WAMs inherits both predictive knowledge and the models in which it was learned. We ask whether a predictive visual latent space induced by large-scale predictive pretraining can instead provide a sufficient foundation for effective WAM learning without inheriting a complete pretrained visual generative model. To answer this question, we introduce V-JEPA Policy, a simple framework that builds a WAM on the latent space of a frozen V-JEPA 2.1 encoder. An instruction-conditioned future-latent predictor and a flow-matching action expert are jointly learned from scratch in a single downstream stage, with the predictor's future-informed context key--value states conditioning action generation. With 0.9B total parameters, of which 0.6B are trainable, V-JEPA Policy achieves competitive performance with representative WAM and vision-language-action baselines across LIBERO, LIBERO-Plus, and RoboCasa-GR1. Comparing visual foundations under the same downstream framework and training budget identifies V-JEPA latents as more effective than the discriminative, reconstructive, and video-understanding-oriented alternatives, particularly under distribution shifts. Beyond task-specific learning, pretraining the predictor on DROID video--instruction pairs without action labels and adapting it into a WAM yields substantial gains in downstream control and out-of-distribution generalization. Together, these findings establish predictive visual latents as a foundation for effective WAM learning from task-specific demonstrations and for transferring future-modeling knowledge acquired from broader in-the-wild videos. Our code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.37250v1
- Canonical: https://arxiv.org/abs/2609.37250v1
Trouble viewing inline? Open PDF directly →
Full Text
74,092 characters extracted from source content.
Expand or collapse full text
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents Yang Zhang Affiliation: Tsinghua University Email: z-yang21@mails.tsinghua.edu.cn Jiangyuan Zhao Affiliation: Shanghai Jiao Tong University Email: li.xiu@sz.tsinghua.edu.cn Chenyou Fan Affiliation: Fudan University Jiayu Hu Affiliation: University of Science and Technology of China Xiu Yuan Affiliation: Washington University in St. Louis Chenjia Bai Affiliation: The Institute of Artificial Intelligence, China Telecom (TeleAI) Affiliation: γ-Robotics Xiu Li †thanks: Corresponding author. Affiliation: Tsinghua University Abstract World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrained at scale, a prominent line of recent WAMs inherits both predictive knowledge and the models in which it was learned. We ask whether a predictive visual latent space induced by large-scale predictive pretraining can instead provide a sufficient foundation for effective WAM learning without inheriting a complete pretrained visual generative model. To answer this question, we introduce V-JEPA Policy, a simple framework that builds a WAM on the latent space of a frozen V-JEPA 2.1 encoder. An instruction-conditioned future-latent predictor and a flow-matching action expert are jointly learned from scratch in a single downstream stage, with the predictor’s future-informed context key–value states conditioning action generation. With 0.9B total parameters, of which 0.6B are trainable, V-JEPA Policy achieves competitive performance with representative WAM and vision-language-action baselines across LIBERO, LIBERO-Plus, and RoboCasa-GR1. Comparing visual foundations under the same downstream framework and training budget identifies V-JEPA latents as more effective than the discriminative, reconstructive, and video-understanding-oriented alternatives, particularly under distribution shifts. Beyond task-specific learning, the same latent space supports acquiring transferable future-modeling knowledge from diverse in-the-wild instruction-annotated videos. Pretraining the predictor on DROID video–instruction pairs without action labels and adapting it into a WAM within our framework yields substantial gains in downstream control and out-of-distribution generalization. Together, these findings establish predictive visual latents as a foundation for learning effective WAMs, supporting both direct learning from task-specific demonstrations and the transfer of future-modeling knowledge acquired through predictor pretraining on broader in-the-wild videos. Our code is available at https://github.com/breez3young/VJEPA-Policy. 1 Introduction Advanced physical intelligence should endow robots with the capability not only to understand the current scene but also to anticipate how it may change through interaction. World-action models (WAMs) have thus emerged as a promising paradigm for instantiating this principle by coupling action generation with future visual-state prediction, demonstrating strong task performance and improved generalization to unfamiliar tasks and environments (Zhu et al., 2025; Pai et al., 2025; Liang et al., 2025; Kim et al., 2026; Ye et al., 2026; Bi et al., 2026; Li et al., 2026; Zhang et al., 2026a). They highlight the value of predictive knowledge for general-purpose robot control. A prominent line of work acquires this capability by adapting visual generative models pretrained at scale for effective WAM learning. Video-generation models provide priors over motion and scene evolution learned from large-scale video data (Pai et al., 2025; Kim et al., 2026; Yuan et al., 2026), while image-editing models provide priors over instruction-conditioned visual transformations, mapping a current observation directly to a task-specified target visual state without generating intermediate frames (Black Forest Labs, 2025; Wu et al., 2025; Zhang et al., 2026c). Despite these different formulations, both approaches transfer knowledge about how visual states change together with the generative backbone in which that knowledge was learned. This raises a natural question: Does effective WAM learning require inheriting a complete pretrained visual generative model, or can a predictive visual latent space learned at scale provide a sufficient foundation? To answer this question, we introduce V-JEPA Policy, a framework that builds a WAM on the predictive latent space of a frozen V-JEPA 2.1 encoder (Bardes et al., 2024; Assran et al., 2025; Mur-Labadia et al., 2026). We jointly learn an instruction-conditioned future-latent predictor and a flow-matching action expert directly from task-specific demonstrations. The predictor follows the vision transformer architecture of the V-JEPA predictor, augmented with cross-attention for instruction conditioning, and combines observed latent tokens with learnable queries at future positions to forecast future visual latents in a single forward pass. Inspired by recent foundation policies (Black et al., 2024; Black et al., 2025; Ye et al., 2026; Zhang et al., 2026b), our framework adopts a Mixture-of-Transformers (MoT) architecture with shared attention to combine the future predictor and the action expert. Context and future queries interact through bidirectional attention, and the resulting layer-wise context key–value states condition the action expert, allowing future modeling to directly shape action generation. V-JEPA’s large-scale video predictive self-supervised pretraining provides a visual latent space suited to this construction. With the visual encoder kept frozen and both trainable modules learned from scratch in a single downstream stage, our design directly tests whether a predictive visual latent space learned at scale can provide a sufficient foundation for effective WAM learning without inheriting a complete pretrained visual generator. Beyond learning from task-specific demonstrations, we further investigate whether the same latent space can efficiently absorb general future-modeling knowledge from broad in-the-wild videos and transfer it to downstream control. Task-specific demonstrations cover only a limited range of behaviors and visual transitions. We therefore pretrain only the instruction-conditioned future-latent predictor on diverse DROID video–instruction pairs, using future-prediction supervision without action labels while keeping the V-JEPA encoder frozen (Khazatsky et al., 2024). We then transfer the predictor to downstream V-JEPA Policy training, where a freshly initialized flow-matching action expert is learned under the same joint objective and downstream budget. The latent space is thus tested not only as a foundation for direct WAM learning but also as a reliable substrate for acquiring and transferring more general future-modeling knowledge underlying in-the-wild videos. We extensively evaluate V-JEPA Policy on both standard and distribution-shifted simulation benchmarks spanning settings from single-arm manipulation to whole-body control, as well as on a real-world dual-arm platform. Beyond comparisons with baselines, we also systematically analyze the framework through controlled studies of its visual foundation and encoder scale, context-future interaction and future-modeling supervision, training-compute scaling, and action-free predictor pretraining and transfer. With 0.9B total parameters, of which only 0.6B are trainable, the one-stage model reaches 97.25% on LIBERO (Liu et al., 2023a), 79.25% on LIBERO-Plus (Fei et al., 2025), 50.92% on RoboCasa-GR1 tasks (Bjorck et al., 2025; Nasiriany et al., 2024), and 45% on two real-world multi-stage bimanual coordination tasks while remaining competitive with representative WAM and VLA baselines. Under the fixed downstream framework and training budget, the V-JEPA predictive latent outperforms the discriminative (Oquab et al., 2024; Siméoni et al., 2026), reconstructive (Wan et al., 2025), and video-understanding-oriented alternatives (Yan et al., 2026), with the performance gaps widening under distribution shifts. Extending downstream training compute on the same task demonstrations yields diminishing returns in generalization. Predictor-only pretraining on DROID video–instruction pairs without action label supervision instead raises LIBERO-Plus success rate from 79.25% to 91.50% at the default smaller downstream budget, remarkably surpassing the 81.64% success rate achieved by extended training. The same pretraining also raises the average real-world success rate from 45% to 80%, demonstrating the effectiveness of predictor pretraining beyond simulation. These results show that a future-latent predictor can acquire knowledge about physical world evolution from diverse in-the-wild videos and transfer it to downstream control within the same latent space. We highlight the main contributions as follows: • We introduce V-JEPA Policy, a WAM built on the predictive latent space of a frozen V-JEPA encoder. By jointly learning an instruction-conditioned future-latent predictor and a flow-matching action expert from scratch in a single downstream stage, it enables effective WAM learning without inheriting a complete pretrained visual generative model. • We conduct extensive cross-benchmark evaluations and controlled analyses of V-JEPA Policy. In particular, matched visual-substrate comparisons under a shared downstream framework and training budget show that V-JEPA’s predictive visual latents yield stronger performance, with more significant gains under distribution shifts. • We demonstrate that predictor-only pretraining on DROID video–instruction pairs without action supervision transfers future-modeling knowledge within the same latent space, substantially improving generalization performance on LIBERO-Plus at the same downstream budget. 2 Preliminaries V-JEPA. The V-JEPA family of models learns visual representations through self-supervised prediction in feature space, encouraging the modeling of predictable spatiotemporal structure and dynamics rather than pixel-level generation (LeCun and others, 2022; Bardes et al., 2024; Assran et al., 2025; Mur-Labadia et al., 2026). Let y denote an unmasked video, x a masked view of the same video, and M the set of masked spatiotemporal patch positions. A context encoder EθE_θ maps the visible patches in the masked video x to context tokens, while the exponential moving average Eθ¯E_ θ of the context encoder computes target representations from the complete video y. The predictor PϕP_φ jointly processes the context tokens and learnable mask tokens Δy _y specifying the masked positions. Both networks are vision transformers (ViTs) (Dosovitskiy et al., 2020) with 3D rotary position embeddings (RoPE) (Su et al., 2024). They are jointly trained with the latent mask-denoising objective: ℒpredict=1|M|∑i∈M‖Pϕ(Eθ(x),Δy)i−sg(Eθ¯(y)i)‖1, _predict= 1|M| _i∈ M \|P_φ (E_θ(x), _y )_i-sg (E_ θ(y)_i ) \|_1, (1) where sgsg denotes stop-gradient and θ¯ θ tracks an exponential moving average of the encoder parameters θ. These mechanisms are used to prevent representation collapse during joint learning. V-JEPA 2.1 further improves dense visual representations through context-token supervision and deep self-supervision (Mur-Labadia et al., 2026). We use its frozen encoder to define the visual latent space of V-JEPA Policy. Within this space, we adapt the V-JEPA 2 predictor architecture to forecast future latents from observed context by assigning mask tokens to future positions (Sec. 3). Problem formulation. We study language-conditioned robotic manipulation through imitation learning from a demonstration dataset D. At time t, the robot receives RGB observations ot=(t1,…,tn)o_t=(I_t^1,…,I_t^n) from n camera views, its proprioceptive state t∈ℝdsq_t ^d_s, and a task instruction ℓ . Here, tvI_t^v denotes the visual observation from camera v. We learn a policy π(t∣ot,t,ℓ)π(a_t o_t,q_t, ) to imitate demonstrated action chunks t=(at,…,at+H−1)∈ℝH×daa_t=(a_t,…,a_t+H-1) ^H× d_a, where H is the action horizon and dad_a is the action dimension. World-action models extend this formulation by coupling action generation with prediction of future visual states. Let t+=(ot+δ1,…,ot+δK),0<δ1<⋯<δK, _t^+=(o_t+ _1,…,o_t+ _K), 0< _1<·s< _K, (2) denote K future observations from the same camera views, sampled at temporal offsets δ1,…,δK _1,…, _K. Each training example (ot,t,ℓ,t+,t)(o_t,q_t, ,o_t^+,a_t) pairs the observed context with future observations and actions from the same demonstration. In our V-JEPA Policy, the latent representations of t+o_t^+ serve as prediction targets, and future prediction is learned jointly with action generation. At inference, the policy is conditioned on oto_t, tq_t, and ℓ . 3 Methodology In this section, we first describe future-latent prediction (Sec. 3.1) and its coupling with action generation (Sec. 3.2). We then present joint training and inference (Sec. 3.3). Figure 1 provides an overview of the framework. Figure 1: Overview of V-JEPA Policy. A frozen V-JEPA 2.1 encoder defines the visual latent space. The future predictor jointly processes context tokens and learnable future queries in a single forward pass, providing layer-wise, future-informed context key–value states to a flow-matching action expert. Both modules receive language and proprioceptive conditioning and are jointly trained from scratch with future-latent regression and action flow matching. 3.1 Instruction-Conditioned Latent Prediction A fixed visual space. We use the frozen V-JEPA 2.1 encoder to encode the observed context and define the future latent targets (Mur-Labadia et al., 2026). To match the encoder’s temporal tubelet size of two, we form a minimal two-frame observed context for each camera view by pairing the current frame with an earlier observation. During training, we combine this context with the corresponding future observations t+o_t^+ to form a complete video clip. Following the construction of context and target representations in V-JEPA (Assran et al., 2025), we designate its future tubelet positions as the masked region. The context path thus removes tokens at future positions before the Transformer blocks of the encoder, and encodes only observed tokens. The target path instead processes the complete unmasked clip with the same frozen encoder. We normalize its output tokens across the feature dimension and select the future positions as prediction targets. We independently encode each camera view and concatenate their context representations and future targets across views to form tcZ_t^c and t+Z_t^+, respectively. Future-query prediction. The predictor PϕP_φ follows the V-JEPA 2 predictor architecture (Assran et al., 2025) and is trained from scratch to predict the future latent targets. We project tcZ_t^c to the predictor’s hidden dimension and concatenate the result with future-query tokens Δt+ _t^+. These query tokens repeat a single learnable mask embedding at all future target positions. Bidirectional self-attention jointly updates both groups, allowing context states to incorporate information from the evolving future-query representations. To distinguish token positions, we utilize 3D rotary position embeddings (RoPE) based on temporal, height, and width coordinates in the visual latent grid (Su et al., 2024). Context tokens retain the positions of the observed input, while future-query tokens use the positions of their prediction targets. Within each attention head, we compute attention from the query, key, and value Q, K, and V as Attn3D(,,)=softmax(ℛ3D()ℛ3D()⊤/dh),Attn_3D(Q,K,V)=softmax\! (R_3D(Q)R_3D(K) \!/\! d_h )V, (3) where dhd_h denotes the head dimension, and ℛ3DR_3D applies rotary transformations according to each token’s spatiotemporal coordinates. Coordinates are local to each camera view. Learnable view embeddings are added to both context tokens and future queries to distinguish their associated cameras. Instruction and state conditioning. Future prediction is conditioned on the task instruction ℓ and current robot state tq_t. We encode the task instruction ℓ with a frozen T5 encoder (Raffel et al., 2020) and concatenate the resulting text tokens with one state token projected from tq_t to form a conditioning sequence. Following the conditioning layout of Wan (Wan et al., 2025), each predictor block applies visual self-attention (Eq. 3), cross-attention to this sequence, and a feed-forward network. Both context tokens and future queries therefore receive instruction and proprioceptive state information. The final future-query states are normalized and projected back to the encoder feature dimension, yielding the future latent predictions in a single forward pass: ^t+=Pϕ(tc,Δt+∣ℓ,t). Z_t^+=P_φ\! (Z_t^c, _t^+ ,q_t ). (4) 3.2 Coupling Prediction and Action A future-informed context interface. We connect future prediction to action generation through the predictor’s layer-wise context states rather than its final future-latent predictions. Through bidirectional interactions with future queries in the predictor, context states at deeper layers incorporate the evolving future-query representations. At predictor layer j, let c(j)K_c^(j) and c(j)V_c^(j) denote the key and value projections of the normalized block input at context positions, before its self-attention update. The resulting layer-wise interface is ϕ=(ℛ3D(c(j)),c(j))j=1LC_φ=\(R_3D(K_c^(j)),V_c^(j))\_j=1^L, where L is the number of layers. To maintain the spatiotemporal position information of the context tubelets when conditioning the action expert, we preserve the 3D RoPE transformation applied to the keys as in Eq. 3 while the values remain unrotated. This interface contains only keys and values at context positions. Flow-matching action generation. Conditioned on ϕC_φ, the instruction ℓ , and proprioceptive state tq_t, a diffusion-transformer-based action expert vψv_ψ models continuous action chunks using conditional flow matching (Peebles and Xie, 2023; Liu et al., 2023b; Lipman et al., 2023; Lipman et al., 2024). Given a demonstration chunk ta_t, Gaussian noise ϵ∼(,) ε (0,I), and flow time τ∈[0,1]τ∈[0,1], we construct tτ=(1−τ)ϵ+τt. _t^τ=(1-τ) ε+ _t. (5) The expert embeds tτa_t^τ into action tokens with learnable sequence-position embeddings. Following recent foundation policy architectures (Black et al., 2025; Ye et al., 2026; Zhang et al., 2026b), the predictor and expert form a Mixture-of-Transformers (MoT) architecture with separate parameters, the same number of layers, and compatible attention-head dimensions. Within each attention head at layer j, queries projected from action tokens attend jointly to the corresponding context positions and the action tokens, and compute the output as follows: a(j)=softmax(a(j)[ℛ3D(c(j));a(j)]⊤/dh)[c(j);a(j)],O_a^(j)=softmax\! (Q_a^(j)[R_3D(K_c^(j));\,K_a^(j)] \!/\! d_h )[V_c^(j);\,V_a^(j)], (6) where a(j)Q_a^(j), a(j)K_a^(j), and a(j)V_a^(j) are the action-token query, key, and value projections, [⋅;⋅][·;·] denotes token concatenation, and dhd_h is the attention-head dimension. In contrast, predictor tokens cannot attend to action tokens. Both the predictor and action expert are conditioned on the instruction ℓ and proprioceptive state tq_t through cross-attention. Each module projects the T5 instruction embeddings and proprioceptive state into its own hidden space. The output head predicts the conditional velocity vψ(tτ,τ,ϕ,ℓ,t)v_ψ(a_t^τ,τ;C_φ, ,q_t), with t−ϵa_t- ε as the flow-matching target. 3.3 Joint Training and Inference Joint objective. For direct learning from task-specific demonstrations, we jointly train the predictor and action expert from scratch on these demonstrations in a single stage while keeping the visual and text encoders frozen. We adapt the V-JEPA mask-denoising objective in Eq. 1 to instruction-conditioned future prediction, with masked positions corresponding to future tubelets and targets provided by the frozen visual encoder, written as: ℒfuture=‖Pϕ(tc,Δt+∣ℓ,t)−t+‖1.L_future=E_D \|P_φ\! (Z_t^c, _t^+ ,q_t )-Z_t^+ \|_1. (7) For action generation, we optimize the flow-matching objective (Lipman et al., 2023) ℒaction=(ot,t+,t,ℓ,t)∼‖vψ(tτ,τ,ϕ,ℓ,t)−(t−ϵ)‖22.L_action=E_(o_t,o_t^+,q_t, ,a_t) \|v_ψ(a_t^τ,τ;C_φ, ,q_t)-(a_t- ε) \|_2^2. (8) We jointly optimize both objectives to directly learn a WAM on the frozen visual space of V-JEPA: ℒ(ϕ,ψ)=ℒaction+λfutureℒfuture,L(φ,ψ)=L_action+ _futureL_future, (9) with λfuture=1 _future=1. The action loss also backpropagates through ϕC_φ into the predictor, so its context states are jointly shaped by the supervision of both the future prediction and the action generation. Inference. We follow an imagine-then-act procedure. At each decision step, a single predictor pass constructs the future-informed context interface ϕC_φ. We cache this interface and integrate vψv_ψ from Gaussian noise at τ=0τ=0 to a clean predicted action chunk at τ=1τ=1 using 10 Euler steps, evaluating only the action expert at each step. 4 Related Work JEPA. Joint-embedding predictive architectures learn visual representations by predicting target embeddings from partial observations, progressing from image representation learning to video prediction and dense video features (Assran et al., 2023; Bardes et al., 2024; Assran et al., 2025; Mur-Labadia et al., 2026). For robot control, DINO-WM (Zhou et al., 2024) and action-conditioned V-JEPA 2 (Assran et al., 2025) learn dynamics in pretrained feature spaces for planning. VLA-JEPA (Sun et al., 2026) integrates latent prediction with a pretrained vision-language backbone, while JEPA-WAM (Lin et al., 2026) uses a Qwen-initialized predictor with an additional vision–language alignment stage before robot-policy training. Our default framework instead jointly learns the future predictor and action expert from scratch using a frozen predictive visual encoder in a single downstream stage. We further study how this latent space supports acquiring transferable future-modeling knowledge through predictor-only pretraining on in-the-wild action-free data. World-action models. Recent WAMs acquire predictive priors by adapting pretrained video generators or image-editing models for control (Liang et al., 2025; Kim et al., 2026; Ye et al., 2026; Yuan et al., 2026; Zhang et al., 2026c). Joint video-action modeling studies explore how visual prediction should interact with action learning (Wu et al., 2024; Zhu et al., 2025). FastWAM further shows that future co-training can improve policy representations even when future imagination is removed at deployment (Yuan et al., 2026). These approaches establish several ways to use predictive knowledge while retaining pretrained generative backbones. Our study examines the inherited visual foundation: whether the latent space learned through predictive pretraining can support effective WAM learning without inheriting a complete visual generator. 5 Experiments We evaluate V-JEPA Policy across simulation benchmarks and real-world manipulation to investigate four central questions: (Q1. Viability) Can effective WAMs be learned directly on frozen predictive visual latents without inheriting a visual generator? (Q2. Visual Foundation) How does the choice of pretrained visual latent space affect downstream performance and generalization? (Q3. Knowledge Transfer) Can the same latent space support acquiring transferable future-modeling knowledge from broader video–language experience? (Q4. Efficiency) How does V-JEPA Policy compare with representative WAMs in inference efficiency? 5.1 Experimental Setup Evaluation benchmarks. We evaluate V-JEPA Policy across three simulated benchmarks and a real-world dual-arm setup: (1) LIBERO (Liu et al., 2023a) for in-distribution multi-task imitation across four suites (Spatial, Object, Goal, Long-Horizon); (2) LIBERO-Plus (Fei et al., 2025) for out-of-distribution robustness across seven controlled shift axes (viewpoint, initial state, language, lighting, texture, noise, layout); (3) RoboCasa-GR1 (Bjorck et al., 2025; Nasiriany et al., 2024) for humanoid manipulation across 24 tasks, extending evaluation to a distinct robot embodiment; and (4) a TianJi Marvin dual-arm platform for real-world manipulation. We report success rates throughout, with evaluation protocols detailed in Appendix A.1. Implementation details. By default, V-JEPA Policy uses a frozen V-JEPA 2.1 ViT-L visual encoder and a frozen T5-XXL text encoder. In the default regime, we jointly train the predictor and action expert from scratch on downstream demonstrations. We train for 10 epochs on LIBERO. On RoboCasa-GR1, we use 50k optimizer steps with a batch size of 256. The visual encoder, predictor, and action expert contain approximately 0.3B, 0.5B, and 0.1B parameters, respectively, totaling 0.9B parameters, of which 0.6B parameters are trainable. Visual-foundation comparisons and future-prediction ablations in Sec. 5.3 use a shared downstream training recipe unless stated otherwise. Network configurations and benchmark-specific training settings are detailed in Appendix A.2. LIBERO LIBERO-Plus Method Params. (B) Action P.T. Avg. Spatial Object Goal Long Avg. Cam. Init. Lang. Light Texture Noise Layout OpenVLA-OFT (Kim et al., 2025) 7.7 ✓ 97.1 97.6 98.4 97.9 94.5 69.6 56.4 31.9 79.5 88.7 93.3 75.8 74.2 π0 _0 (Black et al., 2024) 3.3 ✓ 94.1 96.8 98.8 95.8 85.2 53.6 13.8 6.0 58.8 85.0 81.4 79.0 68.9 π0.5 _0.5 (Black et al., 2025) 3.3 ✓ 96.9 98.8 98.2 98.0 92.4 80.7 64.0 58.0 88.5 96.6 81.4 87.5 85.9 VLA-JEPA (Sun et al., 2026) 2.3 ✓ 97.2 96.2 99.6 97.2 95.8 79.5 63.3 67.1 85.4 95.6 93.6 66.3 85.1 PRTS (Zhang et al., 2026b) 5 ✓ 98.4 98.8 99.8 98.4 96.6 84.5 72.5 75.0 90.6 94.8 94.9 87.0 83.1 ResVLA (Zhong et al., 2026) 4 ✗ 96.6 96.0 100.0 97.4 92.8 76.9 53.2 57.5 88.2 94.5 96.3 81.8 78.3 StarVLA-π (Community, 2026) 7.8 ✗ 95.7 98.8 99.6 95.8 88.4 77.0 64.3 57.2 82.8 94.2 94.0 79.6 78.2 FastWAM (Yuan et al., 2026) 6 ✗ 97.6 98.2 100.0 97.0 95.2 51.5 16.4 44.5 68.9 78.2 53.7 37.7 60.7 ImageWAM (Zhang et al., 2026c) 4.5 ✗ 98.4 97.2 99.2 98.8 98.4 83.1 80.8 50.3 91.4 98.1 85.5 93.8 80.5 JEPA-WAM (Lin et al., 2026) 1.3 ✗ 96.7 95.6 99.4 97.2 94.6 77.9† 79.2 59.2 68.2 93.3 94.6 83.6 76.1 V-JEPA Policy (From Scratch) 0.9 ✗ 97.3 97.2 98.0 97.0 96.8 79.3 70.0 80.7 64.9 97.2 80.0 87.8 79.1 V-JEPA Policy (Pretrained Predictor) 0.9 ✗ 98.7 98.0 99.2 98.8 98.8 91.5 79.6 93.0 94.6 97.4 95.8 95.3 87.9 Table 1: In-distribution performance on LIBERO and generalization performance under distribution shifts on LIBERO-Plus. All scores are success rates (%). Bold and underline indicate the best and second-best reported scores in each column, including ties. Action P.T. denotes additional action-supervised embodied policy pretraining. Cam., Init., and Lang. denote camera viewpoint, robot initial state, and language instruction perturbation axes, respectively. † Micro-average estimated from the reported, rounded per-axis success rates of JEPA-WAM, weighted by the official LIBERO-Plus test-set sizes. 5.2 Learning WAMs without a Pretrained Visual Generator We evaluate whether effective WAMs can be learned directly on frozen predictive visual latents by jointly training the future predictor and action expert from scratch on downstream demonstrations. Here we also report a variant initialized with a predictor pretrained on DROID video–instruction pairs without action labels, whose pretraining protocol and transfer benefits are examined in Sec. 5.4. Neither variant uses action-supervised policy pretraining. Results on LIBERO and LIBERO-Plus. Table 1 summarizes the results on LIBERO and LIBERO-Plus. We train on LIBERO for 10 epochs, intentionally matching the number of downstream training epochs used by FastWAM and ImageWAM. V-JEPA Policy achieves a 97.3% average success rate, remaining close to the larger generator-based WAMs and competitive with representative VLAs, which introduce additional embodied policy pre-training with action labels. This performance is obtained with only 0.9B policy parameters and no action-supervised policy pretraining. The same policy also generalizes beyond the original LIBERO evaluation distribution. Without additional fine-tuning on perturbed demonstrations, it reaches a 79.3% overall success rate on LIBERO-Plus, substantially outperforming FastWAM and remaining competitive with action-pretrained VLA baselines. ImageWAM retains a higher aggregate success rate, but the results show that competitive robustness can also be learned on frozen predictive visual latents without inheriting a complete pretrained visual generative model. Table 2: RoboCasa-GR1 Tabletop tasks. Method Params. Action P.T. Success (%) ABot-M0 (Yang et al., 2026) 4.6 ✓ 58.30 GR00T N1.6 (NVIDIA et al., 2025) 3 ✓ 47.60 π0.5 _0.5 (Black et al., 2025) 3.3 ✓ 37.00 StarVLA-π (Community, 2026) 7.8 ✗ 43.90 V-JEPA Policy (From Scratch) 0.9 ✗ 50.92 V-JEPA Policy (Pretrained Predictor) 0.9 ✗ 55.58 Results on RoboCasa-GR1. The same architecture and joint training objective also extend effectively to humanoid manipulation. Across 24 RoboCasa-GR1 Tabletop tasks, V-JEPA Policy achieves 50.92% success, exceeding StarVLA-π and the action-pretrained GR00T N1.6 and π0.5 _0.5, while remaining below ABot-M0 (Table 2). Together with its smaller parameter count, this result supports the effectiveness of the predictive visual foundation beyond 7-DoF robotic-arm settings and demonstrates non-trivial control in high-DoF humanoid manipulation. Task V-JEPA Policy (From scratch) V-JEPA Policy (Predictor pretrained) FastWAM (Yuan et al., 2026) π0.5 _0.5 (Black et al., 2025) PRTS (Zhang et al., 2026b) Table Cleanup 11/20 (55%) 15/20 (75%) 12/20 (60%) 17/20 (85%) 20/20 (100%) Saucer Racking 7/20 (35%) 17/20 (85%) 7/20 (35%) 10/20 (50%) 19/20 (95%) Table 3: Real-world dual-arm success rates over 20 trials per task. Task descriptions, evaluation protocols and detailed fine-tuning schedules are provided in Appendix B. Results on real-world manipulation. Finally, we test whether V-JEPA Policy remains physically feasible under real-world dynamics. On a four-view TianJi Marvin dual-arm platform, we evaluate five methods across two multi-stage manipulation tasks—Table Cleanup and Saucer Racking. As reported in Table 3, the scratch policy succeeds on both physical tasks (55% and 35%), roughly matching generative FastWAM (60% and 35%) without requiring any visual generative pretraining. Initializing with an instruction-conditioned future-latent predictor pretrained on in-the-wild videos further elevates performance to 75% and 85%. Considering parameter count and performance jointly, the base from-scratch policy lies on the empirical Pareto frontier of the comparison across the evaluated benchmarks and real-world tasks. We provide a visualization in Appendix G. These results show that a frozen predictive visual latent space provides a sufficient foundation for learning an effective WAM from downstream demonstrations, with the future predictor and action expert trained jointly from scratch. 5.3 The Visual Foundation for WAM Learning To answer Q2, we investigate whether the choice of pretrained visual latent space influences WAM learning when the downstream prediction-control interface remains unchanged. We therefore compare frozen visual encoders while keeping the predictor and action-expert backbones, downstream demonstrations, and optimization budget fixed. To ensure a controlled comparison across heterogeneous visual foundations, we use the same raw video segments and align the resulting latent structures across encoders. Each encoder receives inputs sampled according to its native patch size and temporal compression ratio, while the resulting context latents and future prediction targets are matched in spatial grid size and temporal token length. The corresponding action chunks and labels remain identical across encoders. Encoder-specific projection layers only accommodate differences in latent feature dimensions without modifying the shared downstream WAM architecture. LIBERO LIBERO-Plus Visual encoder Param. Avg. Spatial Object Goal Long Avg. Cam. Init. Lang. Light Tex. Noise Layout Discriminative visual foundations DINOv2 ViT-L/14 (Oquab et al., 2024) 304M 94.90 97.20 98.40 93.40 90.60 67.02 39.02 71.87 54.20 95.01 87.64 62.96 73.11 DINOv3 ViT-L/16 (Siméoni et al., 2026) 303M 93.35 95.00 98.00 92.20 88.20 64.28 28.52 63.35 49.77 94.83 82.81 73.39 71.80 Reconstructive visual foundations WAN2.2 VAE (Wan et al., 2025) 150M 92.60 94.40 97.40 96.20 82.40 46.43 35.77 57.29 45.41 32.05 10.13 56.53 73.38 Video-understanding-oriented visual foundations InternVideo3 (Yan et al., 2026) 416M 93.10 94.40 97.80 96.40 83.80 68.62 60.73 72.13 43.79 88.44 90.71 64.96 71.80 Predictive visual foundations V-JEPA 2 ViT-L (Assran et al., 2025) 304M 96.90 97.60 98.80 96.40 94.80 74.15 52.91 82.77 59.53 96.58 89.78 69.71 79.21 V-JEPA 2 ViT-G (Assran et al., 2025) 1.01B 97.85 98.60 99.20 97.00 96.80 76.08 57.85 80.65 59.08 97.37 89.03 80.07 78.43 V-JEPA 2.1 ViT-L (Mur-Labadia et al., 2026) 304M 97.25 97.20 98.00 97.00 96.80 79.25 70.04 80.65 64.87 97.20 80.02 87.76 79.08 V-JEPA 2.1 ViT-G (Mur-Labadia et al., 2026) 1.01B 97.70 98.60 99.60 97.80 94.80 78.37 61.79 78.52 59.99 96.41 89.78 89.76 80.66 Table 4: Visual foundations, encoder generations, and model scales under a shared downstream recipe. All benchmark entries are success rates (%). Bold and underline indicate the best and second-best reported scores in each column, including ties. Param. gives the approximate frozen visual-encoder size. Comparing visual foundations. Across discriminative, reconstructive, video-understanding-oriented, and predictive representations, all evaluated encoders support downstream WAM learning, but predictive visual latents achieve the strongest overall performance under the shared recipe. V-JEPA 2.1 ViT-L reaches 97.25% on LIBERO and 79.25% on LIBERO-Plus, with the advantage becoming substantially larger under distribution shifts. Compared with the strongest discriminative baseline DINOv2, the margin increases from 2.35 points in-distribution to 12.23 points under shifts. The comparison further reveals different generalization capabilities across representation families. Discriminative and video-understanding-oriented representations remain competitive on appearance-related variations, reflecting their strong sensitivity to visual semantics. In contrast, reconstructive latents exhibit substantially weaker robustness under LIBERO-Plus shifts, possibly because reconstruction objectives prioritize preserving appearance-level details that are less aligned with the object-centric and temporal abstractions required for robust control. Overall, these results suggest that predictive visual latents provide a more compatible foundation for robust WAM learning under a unified downstream interface. Predictive representation quality and scale. Within predictive visual foundations (Table 4), we further examine how representation quality and encoder capacity affect downstream WAM learning. Upgrading from V-JEPA 2 to V-JEPA 2.1 consistently improves LIBERO-Plus robustness across both ViT-L and ViT-G scales, with larger gains under distribution shifts. In contrast, increasing encoder size from ViT-L to ViT-G primarily benefits in-distribution performance, while providing limited or even negative gains under LIBERO-Plus shifts. Notably, V-JEPA 2.1 ViT-L achieves the strongest LIBERO-Plus performance despite using a smaller backbone than ViT-G. These results suggest that improvements in predictive representation learning are a more reliable path toward robust WAMs than scaling visual capacity alone with fixed downstream data. Role of future prediction. We further isolate the role of explicit future-latent supervision from the contribution of the future-query pathway. With the visual foundation and downstream recipe fixed, we compare the full model against two ablations: a context-only variant that removes future queries, and a no-future-loss variant that retains the same future-query pathway but removes future-latent supervision. Both ablations degrade performance on LIBERO and LIBERO-Plus, with the no-future-loss variant failing to recover the full model despite preserving the future-query architecture. This indicates that the gains do not arise merely from expanding the predictor pathway, but from explicitly learning future visual representations that are coupled to downstream action generation. Detailed configurations and numerical results are provided in Appendix D. 5.4 Acquiring and Transferring Future-Modeling Knowledge To investigate whether the same predictive latent space supports acquiring transferable future-modeling knowledge, we pretrain only the instruction-conditioned predictor on DROID video–instruction pairs (Khazatsky et al., 2024) using future-latent supervision. The visual encoder remains frozen, and pretraining involves neither action labels nor an action expert. We use the resulting weights to initialize the predictor for downstream joint training alongside a randomly initialized action expert. Benchmark From Scratch Pretrained Predictor Simulation LIBERO 97.25 98.70 LIBERO-Plus 79.25 91.50 RoboCasa-GR1 50.92 55.58 Real-world tasks Table Cleanup 55 75 Saucer Racking 35 85 Table 5: Predictor-only pretraining improves downstream control. We report SR with predictors initialized from scratch or pretrained on DROID video–instruction pairs without action supervision. Within each benchmark, only predictor initialization differs, with fixed downstream data, training protocols, and budgets. Figure 2: Predictor pretraining improves control and robustness beyond longer downstream training. (a) Success rates on LIBERO and LIBERO-Plus along an extended scratch run (gray curves), compared with scratch (open circles) and pretrained predictor (purple stars) results at the default budget of 10-epoch downstream updates. (b) Under the default budget, pretraining improves success across all seven LIBERO-Plus perturbation axes, most strongly under language shifts. Gains are reported in percentage points (p). Transfer across benchmarks and distribution shifts. Under matched downstream protocols, predictor-only pretraining improves success across all three simulated benchmarks and both real-world tasks (Table 2). The improvement is modest on in-distribution LIBERO but substantially larger on LIBERO-Plus, where success rises from 79.25% to 91.50% without fine-tuning on perturbed demonstrations. Improvements are observed across all seven perturbation axes, with the largest gains under language, texture, and robot initial state shifts (Figure 2(b)). Positive transfer to RoboCasa-GR1 and physical bimanual manipulation further indicates that future-modeling knowledge acquired without action supervision remains useful across robot embodiments. Transfer beyond downstream optimization. To test whether longer downstream training can recover the transfer gains, we extend a scratch run to 60k optimizer updates while keeping the model, dataset, and batch size fixed (Figure 2(a)). Performance improves initially but shows diminishing gains later on both LIBERO and LIBERO-Plus, remaining below the performance obtained with pretrained initialization. On LIBERO-Plus, pretrained initialization achieves 91.50% success rate (SR) after 10-epoch downstream updates, compared with 81.64% after 60k updates from scratch. Thus, additional task-specific optimization does not recover the gains from predictor pretraining along the evaluated training trajectory. Together, these results demonstrate that a frozen predictive latent space not only supports direct WAM learning, but also provides a reusable substrate for acquiring and transferring future-modeling knowledge from broader video–language experience. 5.5 Deployment Efficiency Table 6: Action-prediction core profile on an RTX 4090. Method Mean Latency (ms) P95 (ms) Peak VRAM (GiB) PRTS 115.89 118.99 9.96 π0.5 _0.5 159.42 164.46 8.87 FastWAM 202.51 210.72 12.74 V-JEPA Policy 178.17 186.81 4.66 To address Q4, we compare the deployment efficiency of V-JEPA Policy with those of FastWAM, a representative WAM, and other baselines. A key design choice of V-JEPA Policy is to perform future prediction in latent space: a single predictor forward pass produces the future-informed context interface used by the action expert, avoiding iterative visual generation at deployment. We benchmark the action-prediction core under a matched three-view setting on an NVIDIA RTX 4090 with batch size 1, excluding sensor I/O and motor execution latency (as detailed in Appendix F). As shown in Table 6, V-JEPA Policy achieves a substantially smaller memory footprint among the evaluated WAMs, requiring only 4.66 GiB peak VRAM. Compared with FastWAM, which also avoids pixel-space decoding during deployment but retains a large visual generative backbone, V-JEPA Policy reduces peak memory usage by 63.4% and achieves lower inference latency (178.17 ms vs. 202.51 ms). These results show that predictive latent WAMs can preserve future-conditioned action generation while avoiding the deployment overhead associated with large generative visual models. 6 Conclusion We showed that effective WAM learning can build on the latent space of a frozen predictive visual encoder without inheriting a complete pretrained visual generator. V-JEPA Policy instantiates this approach by jointly training a future predictor and an action expert from scratch, coupled through future-informed context states. The resulting compact policy achieves competitive simulation performance and supports real-world bimanual manipulation. Controlled encoder comparisons favor predictive visual latents, particularly under distribution shifts. Predictor-only pretraining on DROID video–instruction pairs without using action labels further improves downstream control and robustness, with LIBERO-Plus gains that extended scratch training does not recover within the evaluated budget. Together, these findings support predictive visual latents as a sufficient foundation for learning WAMs directly from task-specific demonstrations and acquiring transferable future-modeling knowledge from broader video experience. References Assran et al. (2023) M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas Self-supervised learning from images with a joint-embedding predictive architecture. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 15619–15629. Cited by: §4. Assran et al. (2025) M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholus, et al. V-jepa 2: self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985. Cited by: §1, §2, §3.1, §3.1, §4, Table 4, Table 4. Bardes et al. (2024) A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y. LeCun, M. Assran, and N. Ballas Revisiting feature prediction for learning visual representations from video. arXiv preprint arXiv:2404.08471. Cited by: §1, §2, §4. Bi et al. (2026) H. Bi, H. Tan, S. Xie, Z. Wang, S. Huang, H. Liu, R. Zhao, Y. Feng, C. Xiang, Y. Rong, et al. Motus: a unified latent action world model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 35101–35113. Cited by: §1. Bjorck et al. (2025) J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734. Cited by: §A.1, §1, §5.1. Black Forest Labs (2025) Black Forest Labs FLUX.2: analyzing and enhancing the latent space of FLUX – representation comparison. External Links: Link Cited by: §1. Black et al. (2025) K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, M. Y. Galliker, D. Ghosh, L. Groom, K. Hausman, brian ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. Vuong, H. Walke, A. Walling, H. Wang, L. Yu, and U. Zhilinsky π0.5 _0.5: A vision-language-action model with open-world generalization. In 9th Annual Conference on Robot Learning, External Links: Link Cited by: §A.1, §1, §3.2, Table 1, Table 2, Table 3. Black et al. (2024) K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. π0 _0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164. Cited by: §A.1, §1, Table 1. Community (2026) S. Community StarVLA: a lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014. External Links: 2604.05014 Cited by: Table 11, Table 11, Table 1, Table 2. Dosovitskiy et al. (2020) A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Cited by: §2. Fei et al. (2025) S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. Libero-plus: in-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626. Cited by: §A.1, §1, §5.1. Khazatsky et al. (2024) A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset. In Proceedings of Robotics: Science and Systems, Delft, Netherlands. External Links: Document Cited by: Appendix E, §1, §5.4. Kim et al. (2025) M. J. Kim, C. Finn, and P. Liang Fine-tuning vision-language-action models: optimizing speed and success. arXiv preprint arXiv:2502.19645. Cited by: Table 1. Kim et al. (2026) M. J. Kim, Y. Gao, T. Lin, Y. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M. Liu, C. Finn, and J. Gu Cosmos policy: fine-tuning video models for visuomotor control and planning. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §1, §4. LeCun et al. (2022) Y. LeCun et al. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review 62 (1), p. 1–62. Cited by: §2. Li et al. (2026) L. Li, Q. Zhang, Y. Luo, S. Yang, R. Wang, F. Han, M. Yu, Z. Gao, N. Xue, X. Zhu, et al. Causal world modeling for robot control. arXiv preprint arXiv:2601.21998. Cited by: §1. Liang et al. (2025) J. Liang, P. Tokmakov, R. Liu, S. Sudhakar, P. Shah, R. Ambrus, and C. Vondrick Video generators are robot policies. arXiv preprint arXiv:2508.00795. Cited by: §1, §4. Lin et al. (2026) Y. Lin, J. He, S. Bao, C. Zhao, Y. Li, X. Wang, Y. Wang, C. Chi, and J. Zhang JEPA-wam: learning vision-language-action policies with joint-embedding world modeling. arXiv preprint arXiv:2608.09381. Cited by: §4, Table 1. Lipman et al. (2023) Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §3.2, §3.3. Lipman et al. (2024) Y. Lipman, M. Havasi, P. Holderrieth, N. Shaul, M. Le, B. Karrer, R. T. Chen, D. Lopez-Paz, H. Ben-Hamu, and I. Gat Flow matching guide and code. arXiv preprint arXiv:2412.06264. Cited by: §3.2. Liu et al. (2023a) B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone LIBERO: benchmarking knowledge transfer for lifelong robot learning. arXiv preprint arXiv:2306.03310. Cited by: §A.1, §1, §5.1. Liu et al. (2023b) X. Liu, C. Gong, and qiang liu Flow straight and fast: learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §3.2. Mur-Labadia et al. (2026) L. Mur-Labadia, M. Muckley, A. Bar, M. Assran, K. Sinha, M. Rabbat, Y. LeCun, N. Ballas, and A. Bardes V-jepa 2.1: unlocking dense features in video self-supervised learning. arXiv preprint arXiv:2603.14482. Cited by: §1, §2, §2, §3.1, §4, Table 4, Table 4. Nasiriany et al. (2024) S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y. Zhu RoboCasa: large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems (RSS), Cited by: §1, §5.1. NVIDIA et al. (2025) NVIDIA, J. Bjorck, N. C. Fernando Castañeda, X. Da, R. Ding, L. ”. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y. L. Tan, G. Wang, Z. Wang, J. Wang, Q. Wang, J. Xiang, Y. Xie, Y. Xu, Z. Xu, S. Ye, Z. Yu, A. Zhang, H. Zhang, Y. Zhao, R. Zheng, and Y. Zhu GR00T N1: an open foundation model for generalist humanoid robots. In ArXiv Preprint, External Links: 2503.14734 Cited by: Table 11, Table 2. Oquab et al. (2024) M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski DINOv2: learning robust visual features without supervision. Transactions on Machine Learning Research. Note: Featured Certification External Links: ISSN 2835-8856, Link Cited by: §1, Table 4. Pai et al. (2025) J. Pai, L. Achenbach, V. Montesinos, B. Forrai, O. Mees, and E. Nava Mimic-video: video-action models for generalizable robot control beyond vlas. arXiv preprint arXiv:2512.15692. Cited by: §1, §1. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 4195–4205. Cited by: §3.2. Raffel et al. (2020) C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21 (140), p. 1–67. Cited by: §3.1. Siméoni et al. (2026) O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. E. Yi, M. Ramamonjisoa, F. Massa, D. HAZIZA, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jegou, P. Labatut, and P. Bojanowski DINOv3. Transactions on Machine Learning Research. Note: Featured Certification External Links: ISSN 2835-8856, Link Cited by: §1, Table 4. Su et al. (2024) J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu Roformer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. Cited by: §2, §3.1. Sun et al. (2026) J. Sun, W. Zhang, Z. Qi, S. Ren, Z. Liu, H. Zhu, G. Sun, X. Jin, and Z. Chen VLA-jepa: enhancing vision-language-action model with latent world model. External Links: 2602.10098, Link Cited by: §4, Table 1. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §1, §3.1, Table 4. Wu et al. (2025) C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §1. Wu et al. (2024) H. Wu, Y. Jing, C. Cheang, G. Chen, J. Xu, X. Li, M. Liu, H. Li, and T. Kong Unleashing large-scale video generative pre-training for visual robot manipulation. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §4. Yan et al. (2026) Z. Yan, S. Xia, J. Yu, Y. Wu, T. Jiang, S. Li, K. Tian, Y. Xu, Y. He, K. Chen, et al. InternVideo3: agentify foundation models with multimodal contextual reasoning. arXiv preprint arXiv:2606.12195. Cited by: §1, Table 4. Yang et al. (2026) Y. Yang, S. Zeng, T. Lin, X. Chang, D. Qi, J. Xiao, H. Liu, R. Chen, Y. Chen, D. Huo, et al. ABot-m0: vla foundation model for robotic manipulation with action manifold learning. arXiv preprint arXiv:2602.11236. Cited by: §A.1, Table 11, Table 11, Table 2. Ye et al. (2026) S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y. L. Tan, C. Zhu, J. Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922. Cited by: §1, §1, §3.2, §4. Yuan et al. (2026) T. Yuan, Z. Dong, Y. Liu, and H. Zhao Fast-wam: do world action models need test-time future imagination?. arXiv preprint arXiv:2603.16666. Cited by: §1, §4, Table 1, Table 3. Zhang et al. (2026a) Q. Zhang, L. Li, L. Zhang, S. Yang, Y. Luo, S. Li, R. Wang, J. Wang, J. Shao, G. Xu, et al. Native video-action pretraining for generalizable robot control. arXiv preprint arXiv:2607.08639. Cited by: §1. Zhang et al. (2026b) Y. Zhang, J. Zhao, C. Fan, F. Yan, T. Li, H. Tang, S. Fu, X. Wu, Q. Weng, W. Zhang, et al. PRTS: a primitive reasoning and tasking system via contrastive representations. arXiv preprint arXiv:2604.27472. Cited by: §1, §3.2, Table 1, Table 3. Zhang et al. (2026c) Y. Zhang, W. Zhang, Z. Qi, H. Zhang, H. Lin, J. Zhang, Y. Mu, X. Yang, W. Zeng, and X. Jin ImageWAM: do world action models really need video generation, or just image editing?. arXiv preprint arXiv:2606.19531. Cited by: §1, §4, Table 1. Zhong et al. (2026) Y. Zhong, Y. He, Z. Yang, P. Tian, Y. Huang, Q. Huang, X. Zhu, and Y. Ma From noise to intent: anchoring generative vla policies with residual bridges. arXiv preprint arXiv:2604.21391. Cited by: Table 1. Zhou et al. (2024) G. Zhou, H. Pan, Y. LeCun, and L. Pinto Dino-wm: world models on pre-trained visual features enable zero-shot planning. arXiv preprint arXiv:2411.04983. Cited by: §4. Zhu et al. (2025) C. Zhu, R. Yu, S. Feng, B. Burchfiel, P. Shah, and A. Gupta Unified world models: coupling video and action diffusion for pretraining on large robotic datasets. arXiv preprint arXiv:2504.02792. Cited by: §1, §4. Appendix A Experimental Details A.1 Evaluation Protocol We evaluate V-JEPA Policy on three simulation benchmarks: LIBERO, LIBERO-Plus, and RoboCasa-GR1, which assess in-distribution manipulation, robustness to controlled distribution shifts, and upper-body humanoid tabletop manipulation, respectively. LIBERO. We evaluate on all 40 tasks across the LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long suites (Liu et al., 2023a). For training, we use a mixed-suite dataset containing 1,693 demonstrations, obtained by excluding trajectories that fail when replayed in the simulator from the original LIBERO demonstrations (Black et al., 2024; Black et al., 2025). We evaluate each task over 50 episodes and average the resulting task success rates across the ten tasks in each suite. LIBERO-Plus. LIBERO-Plus extends LIBERO with 10,030 perturbed tasks spanning seven categories: camera viewpoints, robot initial states, language instructions, lighting conditions, background textures, sensor noise, and object layouts (Fei et al., 2025). We evaluate the same policy trained on LIBERO on the full LIBERO-Plus test set, without further fine-tuning. We report the success rate for each category and the overall success rate pooled across all tasks. RoboCasa-GR1. RoboCasa-GR1 is a simulation tabletop manipulation benchmark for the GR-1 humanoid robot, comprising 18 object-rearrangement tasks and six multi-step tasks involving articulated fixtures such as cabinets, drawers, and microwaves (Bjorck et al., 2025). We evaluate on all 24 tasks using 50 episodes per task (Yang et al., 2026) and report the mean success rate. A.2 Network Configuration and Training Recipe Network configuration. The default model uses a frozen V-JEPA 2.1 ViT-L visual encoder and a frozen T5-XXL text encoder. Table 7 lists the predictor and action-expert configurations. Their hidden widths differ, while their joint-attention projections have compatible head counts and dimensions. The 4096-dimensional T5 features and proprioceptive state are projected independently into each module’s hidden width. Both modules therefore receive the instruction and current state directly. Property Future predictor Action expert Transformer blocks 24 24 Hidden width 1024 512 Visual/action joint-attention heads 16 16 Joint-attention head dimension 64 64 Condition cross-attention heads 16 8 Condition-attention head dimension 64 64 Feed-forward network GELU, expansion 4 SwiGLU Flow-time modulation - Adaptive RMS normalization Positional encoding 3D RoPE + view embeddings Learned sequence embeddings Table 7: Reference trainable network configuration. The action expert attends jointly to context-position keys and values from the predictor and its own action tokens. Condition cross-attention is separate from this joint attention. Training recipe. We keep the V-JEPA 2.1 visual encoder and T5-XXL text encoder frozen throughout training, and jointly optimize the future predictor and action expert on downstream tasks. We use AdamW with a peak learning rate of 10−410^-4, a linear warm-up over the first 5% of scheduled steps, and subsequent cosine decay. On LIBERO, we train jointly on the four suites for 21,360 optimizer steps, corresponding to ten epochs, with a global batch size of 128. The model uses two independently encoded 224×224224× 224 camera views and 32-step action chunks. On RoboCasa-GR1, we train on all 24 task datasets, each containing 1,000 episodes, for 50,000 steps with a global batch size of 256. The model uses a single 224×224224× 224 egocentric view and 16-step action chunks. For real-world experiments, we train a separate model for each of the two tasks using four camera views resized to 256×256256× 256. Both tasks use 32-step action chunks and a global batch size of 64, with 60,000 steps for Table Cleanup and 40,000 steps for Saucer Racking. For the pretrained-predictor variant, we first train only the future predictor on DROID for 100,000 steps with a global batch size of 192. This stage uses videos, language instructions, and proprioceptive states, without action labels or an action expert. We use two camera views resized to 256×256256× 256 and ten-frame clips consisting of one context tubelet and four future tubelets. For downstream adaptation, we transfer only the pretrained predictor weights and initialize the action expert from scratch. Appendix B Real-World System Implementation and Evaluation Details Platform and Sensor Topology. Physical evaluations are deployed on a TianJi Marvin dual-arm platform comprising two 7-DoF arms, two parallel-jaw grippers, and four RGB camera streams: a head view (1280×7201280× 720) plus left wrist, right wrist, and an off-axis third-person view (640×480640× 480 each), illustrated in Figure 3. Policies output 16-dimensional joint-and-gripper actions (7 arm joints and 1 binary gripper state per arm). Figure 3: TianJi Marvin dual-arm platform Task Suite and Success Criteria. We benchmark on two long-horizon, bimanual coordination tasks (Figure 4): (1) Table Cleanup: the left arm stows chopsticks into a holder while the right arm sweeps bowl contents into a trash bin, places the bowl in a drying rack, and discards a crumpled napkin; and (2) Saucer Racking: the right arm sweeps a saucer, transfers it to the left arm via bimanual handover for rack placement, and discards residual waste. Trials initialize from standardized robot rest poses with objects restored to canonical configurations under invariant illumination. Each rollout is strictly bounded by a 120 s execution cutoff from the first policy inference. A rollout is marked as a success only if all sequential substages are accomplished within the budget; timed-out trials are logged as 120 s failures. Figure 4: Real-world bimanual task workflows: substage transitions and manipulation horizons. Fine-Tuning Scope and Hyperparameters. To reflect practical deployment, each method follows its native task-tuning schedule rather than an artificial compute-matched budget: V-JEPA Policy is fine-tuned with a batch size of 64 over 60k steps (Table Cleanup) and 40k steps (Saucer Racking), while baselines use a batch size of 32 over 100k and 40k steps, respectively. Consequently, real-world execution metrics benchmark practical deployed capability rather than strict sample efficiency parity. Method Table Cleanup (s) Saucer Racking (s) PRTS 42.40 42.05 π0.5 _0.5 54.35 68.00 FastWAM 62.30 80.86 V-JEPA Policy (From Scratch) 60.45 67.14 V-JEPA Policy (Pretrained Predictor) 68.40 59.76 Table 8: Mean task execution duration over successful rollouts. Appendix C Experimental Details of Visual-Foundation Comparisons Comparison protocol. We compare frozen visual encoders using the same predictor and action-expert backbones, downstream demonstrations, global batch size of 128, and 21360 optimizer updates. The predictor and action expert are initialized from scratch, while the visual encoder remains frozen throughout downstream training. The evaluated foundations include discriminative features from DINOv2 and DINOv3, reconstructive latents from WAN2.2 VAE, video-understanding features from InternVideo3, and predictive features from V-JEPA 2 and V-JEPA 2.1. Within the predictive family, we compare both encoder generations at ViT-L and ViT-G scales. Latent interfaces. Each encoder provides observed-context representations and future prediction targets in its own latent space. Encoder-specific input and output projections accommodate differences in visual feature width while preserving the predictor’s Transformer backbone. The ViT-L variants of V-JEPA and DINO produce 1024-dimensional features, whereas WAN2.2 VAE and InternVideo3 produce 48- and 1152-dimensional features, respectively. The action-expert architecture is unchanged across configurations. Parameter counts in Table 4 include only the frozen visual encoder. Input alignment. Following Sec. 5.3, we adjust spatial resolution and frame sampling within the same raw video segments to match the spatial patch grid and temporal token length across encoders, separately for observed context and future targets. For DINOv2 ViT-L/14, images are resized to 196×196196× 196, yielding a 14×1414× 14 patch grid. This matches the grid produced by V-JEPA ViT-L with 224×224224× 224 inputs and 16×1616× 16 patches. For WAN2.2 VAE, we subsample the input video frames with a temporal stride of two before encoding. Action labels and chunk lengths remain identical across encoder configurations. Appendix D Future-Prediction Ablation We keep the frozen visual encoder, downstream data, training seed, batch size, update count, and action-chunk configuration fixed. The context-only control omits future queries and conditions the action expert on context-only predictor features. The no-future-loss control retains the full future-query pathway and sets the future-latent loss weight to zero. All configurations retain action supervision. Future queries in the no-future-loss control remain trainable and receive gradients through the action objective. Configuration Future queries Future loss LIBERO LIBERO-Plus Context-only ✗ ✗ 92.55 65.86 Future queries, no future loss ✓ ✗ 91.65 68.81 Full model ✓ ✓ 97.25 79.25 Table 9: Future-prediction controls under the shared downstream recipe. All entries are success rates (%). The full model outperforms both controls on both benchmarks. The comparison with the no-future-loss control supports the contribution of explicit future-latent supervision while keeping the query architecture fixed. The context-only control evaluates the necessity of the future-query pathway, whereas the no-future-loss control isolates the effect of explicit predictive supervision while preserving the pathway. Appendix E Experimental Details of Predictor Pretraining and Transfer Predictor-only pretraining. We pretrain the instruction-conditioned predictor on DROID robot video–instruction pairs (Khazatsky et al., 2024) using the future-latent regression objective in Eq. 7. Demonstrations are temporally subsampled from 15 Hz to 5 Hz, using one exterior camera and one wrist camera. For each view, a training clip contains two observed frames and eight future frames. With a temporal tubelet size of two, these correspond to one context and four future temporal tubelet positions per view. The predictor is additionally conditioned on the observed proprioceptive state and cached T5 instruction features. The visual and text encoders remain frozen, and no action expert or action-label supervision is used during pretraining. We use AdamW with a learning rate of 10−410^-4, weight decay of 10−210^-2, and (β1,β2)=(0.9,0.95)( _1, _2)=(0.9,0.95). Pretraining runs for 100,000 optimizer updates on eight GPUs with a global batch size of 192 and BF16 precision. The learning rate follows a 5,000-update linear warmup and cosine decay over the remaining 95,000 updates. Downstream transfer. The pretrained predictor initializes downstream WAM learning, while the action expert is initialized from scratch. Both modules are then jointly optimized with the objective in Eq. 9, with the visual and text encoders kept frozen. For each downstream benchmark or task, the scratch and DROID-initialized models use identical demonstrations, batch sizes, optimizer settings, and numbers of updates. The action expert is initialized from scratch in both settings; the predictor initialization is the only change. On LIBERO, both models use a global batch size of 128 and 21360 optimizer updates. The resulting checkpoints are evaluated directly on LIBERO-Plus without further fine-tuning. On RoboCasa-GR1, both models use a batch size of 256 and 50000 optimizer updates, without gradient accumulation. The same matched-protocol comparison is used for the real-world tasks. Predictor pretraining adds an upstream training stage; the matched budgets refer to downstream training, not total training computation. Appendix F Action-Prediction Inference Profiling Protocol Benchmarking Setup and Boundary Conditions. Latency and memory profiles are measured on a dedicated local workstation equipped with a single NVIDIA RTX 4090 GPU (batch size 1). To isolate representation and model architectures from external hardware and pipeline variation, each model receives identical synthetic inputs matching our three-view setup (head, left wrist, right wrist). Measurements are gathered across 3 independent OS processes per architecture; each process executes 10 warmup iterations followed by 100 recorded calls, yielding 300 timed runs in total with background telemetry disabled. Crucially, this protocol isolates the action-prediction inference and explicitly factors out pipeline overheads: (1) I/O and preprocessing: camera RTSP acquisition, image decoding, resizing, and host-to-device transfers; (2) Control and post-processing: inter-process IPC, action denormalization, temporal ensembling queue updates, and One-Euro filtering; and (3) Actuator communication: network packet serialization and low-level motor bus latency. Action Core Latency (ms) PyTorch CUDA VRAM (GiB) Process Memory Method Mean P50 P95 Allocated Reserved Resident (MiB) PRTS 115.89 115.85 118.99 9.96 10.30 11,014 π0.5 _0.5 159.42 159.69 164.46 8.87 9.20 9,884 FastWAM 202.51 201.93 210.72 12.74 13.04 13,822 V-JEPA Policy 178.17 177.72 186.81 4.66 4.70 5,276 Table 10: Matched three-view action-prediction core profile on an NVIDIA RTX 4090 (300 timed runs across 3 independent processes). Allocated and Reserved denote PyTorch framework allocations; Resident denotes total host OS process memory via NVML. Implementation Alignment. Two adaptations ensure rigorous comparability across disparate model repositories: First, for FastWAM, generated action tensors are retained directly in GPU memory, bypassing native CPU synchronization and serialization overheads. Second, because the full PRTS and V-JEPA Policy checkpoints were trained on four camera streams, our benchmarked computation graphs retain the head and bimanual wrist embeddings while adapting the input sequence to three views, leaving all model weights untouched. Because the scratch and DROID-initialized V-JEPA Policy variants execute an identical computation graph, we report a single structural profile (Table 10). Appendix G Performance–Parameter Pareto Analysis To assess the parameter efficiency of V-JEPA Policy and other baselines, we compare success rates and policy parameter counts across three simulation benchmarks and two real-world tasks. Figure 5 presents the results. Each panel includes the baselines evaluated in that setting and the base V-JEPA Policy, whose future-latent predictor and action expert are trained from scratch on downstream demonstrations. Each point represents a policy, with parameter count in billions on the horizontal axis and success rate on the vertical axis. A policy is nondominated if no other compared policy achieves at least the same success rate using no more parameters, with a strict improvement in either quantity. Circled points identify these nondominated policies, which form the empirical Pareto frontier. The base V-JEPA Policy lies on the empirical Pareto frontier in all five settings among the compared methods. On LIBERO, it achieves 97.25% success, 1.15 percentage points below ImageWAM’s 98.4% while using one-fifth of its parameters. On RoboCasa-GR1, it achieves 50.92% success with 0.9B parameters, compared with 47.60% for the 3.0B GR00T N1.6. On the real-world tasks, it achieves 55% success on Table Cleanup and 35% on Saucer Racking; the latter matches FastWAM with 15% of its parameter count. These comparisons further demonstrate that predictive visual latents provide a foundation for effective WAM learning at a compact policy scale. Figure 5: Success rate versus policy parameter count across three simulation benchmarks and two real-world tasks, with circled points marking nondominated policies. The base V-JEPA Policy lies on the empirical Pareto frontier in every panel. Appendix H RoboCasa-GR1 Detailed Results Table 11 reports per-task success rates on the 24 RoboCasa-GR1 Tabletop tasks. Table 11: Per-task success rates (%) on RoboCasa-GR1. Baseline results are taken from Community (2026); Yang et al. (2026), with StarVLA-π following its official evaluation documentation. GR00T N1.6 task scores are rounded to integers for display; averages are computed over all 24 tasks before rounding. Bold indicates the best result in each row, including ties. Task ABot-M0 (Yang et al., 2026) GR00T N1.6 (NVIDIA et al., 2025) StarVLA-π (Community, 2026) V-JEPA Policy (From Scratch) V-JEPA Policy (Pretrained Predictor) PnPBottleToCabinetClose 86 52 26 78 64 PnPCanToDrawerClose 74 13 62 82 86 PnPCupToDrawerClose 48 9 42 48 48 PnPMilkToMicrowaveClose 46 14 50 52 68 PnPPotatoToMicrowaveClose 50 42 42 28 38 PnPWineToCabinetClose 66 17 32 62 64 PnPNovelFromCuttingboardToBasket 70 58 40 62 56 PnPNovelFromCuttingboardToCardboardbox 58 47 46 46 48 PnPNovelFromCuttingboardToPan 76 69 60 62 70 PnPNovelFromCuttingboardToPot 66 65 40 46 66 PnPNovelFromCuttingboardToTieredbasket 38 47 44 40 44 PnPNovelFromPlacematToBasket 52 59 44 42 44 PnPNovelFromPlacematToBowl 66 58 52 48 44 PnPNovelFromPlacematToPlate 60 63 50 58 58 PnPNovelFromPlacematToTieredshelf 26 29 28 28 26 PnPNovelFromPlateToBowl 54 57 52 46 56 PnPNovelFromPlateToCardboardbox 48 44 40 38 54 PnPNovelFromPlateToPan 66 51 36 60 64 PnPNovelFromPlateToPlate 64 79 48 68 68 PnPNovelFromTrayToCardboardbox 54 52 34 54 68 PnPNovelFromTrayToPlate 68 71 64 64 58 PnPNovelFromTrayToPot 64 65 44 52 60 PnPNovelFromTrayToTieredbasket 60 57 50 28 48 PnPNovelFromTrayToTieredshelf 38 32 28 30 34 Average 58.3 47.6 43.9 50.92 55.58