Paper deep dive
RoboJEPA: Scaling Robotic Latent World Models
Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan, Sarath Chandar, Tushar Nagarajan, Daniel Severo, Koustuv Sinha, Michal Drozdzal, Adriana Romero Soriano, Jeannette Bohg, Nicolas Ballas, Mahmoud Assran
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/13/2026, 3:24:08 AM
Summary
This paper introduces RoboJEPA, a latent world model based on the Joint Embedding Predictive Architecture (JEPA) trained on a large-scale dataset of 23 public manipulation datasets spanning 12 robotic embodiments. The authors establish scaling laws for robotic world models, demonstrating that imagination error (latent rollout error) follows a second-order power law with respect to compute. This error serves as a reliable proxy for downstream robotic planning performance, which improves predictably with model size and compute. RoboJEPA, with up to 8B parameters, is the largest JEPA predictor model trained to date and is deployed zero-shot on real hardware for long-horizon planning tasks.
Entities (10)
Relation Signals (9)
Imagination Error → correlateswith → Planning Performance
confidence 95% · imagination error is strongly correlated with it [downstream robotic planning performance]
Imagination Error → followsscalinglaw → Compute
confidence 95% · RoboJEPA’s imagination error... follows a second-order power law in compute
RoboJEPA → usesarchitecture → JEPA
confidence 95% · RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA)
RoboJEPA → deployedon → Franka
confidence 90% · on the real Franka robot, the success rate improves with model size
RoboJEPA → trainedon → Open X-Embodiment
confidence 90% · We use 9 datasets of Open-X Embodiment
RoboJEPA → trainedon → DROID
confidence 90% · This holds on both real-robot holdout (DROID (Khazatsky et al., 2024)) data
RoboJEPA → trainedon → RoboCasa
confidence 90% · simulated holdout (RoboCasa (Nasiriany et al., 2026)) data
RoboJEPA → usescomponent → V-JEPA 2.1
confidence 90% · RoboJEPA operates in the representation space of a frozen V-JEPA 2.1 encoder
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.
Tags
Links
- Source: https://arxiv.org/abs/2610.10515v1
- Canonical: https://arxiv.org/abs/2610.10515v1
Trouble viewing inline? Open PDF directly →
Full Text
214,027 characters extracted from source content.
Expand or collapse full text
RoboJEPA: Scaling Robotic Latent World Models Artem Zholus Affiliation: FAIR at Meta Affiliation: Chandar Research Lab Affiliation: Mila - Quebec AI Institute Affiliation: Polytechnique Montréal Work done at Meta Nicolas Beltran-Velez Affiliation: FAIR at Meta Jianhao Yuan Affiliation: FAIR at Meta Sarath Chandar Affiliation: Chandar Research Lab Affiliation: Mila - Quebec AI Institute Affiliation: Polytechnique Montréal Tushar Nagarajan Affiliation: FAIR at Meta Daniel Severo Affiliation: FAIR at Meta Koustuv Sinha Affiliation: FAIR at Meta Michal Drozdzal Affiliation: FAIR at Meta Adriana Romero Soriano Affiliation: FAIR at Meta Jeannette Bohg Nicolas Ballas Affiliation: FAIR at Meta Joint last author Mahmoud Assran Affiliation: FAIR at Meta Joint last author Abstract Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA’s imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date. †date: October 7, 2026†correspondence: Artem Zholus at artem.zholus@mila.quebec, Jeannette Bohg at jeannettebohg@meta.com †Code: https://github.com/facebookresearch/robo_jepa†Website: https://robojepa.github.io Figure 1: RoboJEPA scales predictably from offline prediction to real-robot control. Left: the world model’s forward-prediction (ℓ1 _1) error, averaged over the DROID and RoboCasa held-out sets, follows a scaling law in training compute. Middle: downstream planning success in RoboCasa improves predictably with compute across tasks of increasing difficulty. Right: on the real Franka robot, the success rate improves with model size across Grasp, Object Lift, and Pick and Place. Actions are sampled from a uniform distribution and prioritized using the World Model during planning. 1 Introduction Figure 2: World-model imagination improves with scale. Decoded imagination rollouts on an AgiBot episode for RoboJEPA at 0.30.3B, 22B, and 88B parameters, compared against the ground-truth (GT) video (bottom row). Each row shows the same rollout sampled at equal-time steps; larger models track the true dynamics and preserve object and contact detail markedly better . All world models share the same encoder (V-JEPA 2.1-G) and decoder. Scaling has been one of the primary engines of recent progress in artificial intelligence. What makes scaling particularly actionable is predictability: empirical scaling laws relate a model’s loss to the amount of data, parameters, and compute used to train it, so that the return on an additional order of magnitude of compute can be forecast before it is spent (Kaplan et al., 2020; Hoffmann et al., 2022). This predictability turns scaling from a gamble into a science, it tells practitioners how large a model to train for a given budget, how much data to pair it with, and when returns will diminish. Scaling laws of this kind have been established across language models (Kaplan et al., 2020; Hoffmann et al., 2022), generative modeling of images, video, and other modalities (Henighan et al., 2020), or semantic classification of images (Zhai et al., 2022). Comparatively little is known about whether, and how, the same scaling predictability holds for models of the physical world. We study this question for world models in robotics. A world model is a learned model of environment dynamics that an agent can roll out to imagine future states and plan (Ha and Schmidhuber, 2018; Hafner et al., 2023). We focus on latent world models that predict and plan in a latent representation space (Hafner et al., 2020; LeCun, 2022; Assran et al., 2025). Latent world models have shown the ability to plan toward goals specified at test time without the need of a task-specific policy (Assran et al., 2025). Previous latent world models are typically trained at a single scale, and how their quality changes with data, parameters, and compute has not been characterized (Assran et al., 2025; Hafner et al., 2025; Zhang et al., 2026). Other studies, focusing on scaling, either target generative video and interactive-environment models outside of robot control (Bruce et al., 2024; Google DeepMind, 2024; Hu et al., 2023), or scale policies rather than world models (Lin et al., 2025; Sartor and Thompson, 2024; Ai et al., 2025). In contrast, our goal is to establish scaling laws for latent world models. This matters particularly in robotics, where high-quality interaction data is scarce and computationally expensive to collect: knowing which capabilities a world model will have at a given training-compute budget lets practitioners decide whether to invest in a larger model, more demonstrations, or more compute, and reveals when a data budget has been saturated. To this end, this work introduces RoboJEPA, a family of action-conditioned latent joint-embedding predictive world models. We keep the training recipe deliberatly simple to focus on the scaling behavior. RoboJEPA is a feed-forward transformer that predicts the next latent state deterministically in a single forward pass. It is trained with a plain ℓ1 _1 regression loss in the frozen encoder’s latent space. RoboJEPA operates in the representation space of a frozen V-JEPA 2.1 encoder (Assran et al., 2025; Mur-Labadia et al., 2026) and is trained on a unified mixture of 23 public manipulation datasets spanning 12 embodiments and 26 different action spaces. We study model scaling of the world model, ranging parameter count from 22M up to 8B, while holding the representation encoder fixed, so our laws characterize how a plannable dynamics model scales on top of a shared representation. We build scaling laws estimating the compute-efficient frontier of the forward-prediction evaluation loss as a function of the training compute (Kaplan et al., 2020; Pearce and Song, 2024). Our central finding is that robotic world-model quality scales predictably with compute. First, the latent forward-prediction loss on out-of-distribution data follows a scaling law L(C)=ACα−γlnC+EL(C)=A\,C^α-γ C+E, where C is the training compute, E is the irreducible loss, and A,α,γA,α,γ are scaling-law parameters that we estimate alongside E. This holds on both real-robot holdout (DROID (Khazatsky et al., 2024)) data and simulated holdout (RoboCasa (Nasiriany et al., 2026)) data. Second, this improvement on the offline data transfers to downstream planning performance on a robot: on a suite of RoboCasa planning tasks, downstream success improves predictably with training compute, so that a world model’s latent forward-prediction loss is a strong predictor of its online planning performance. For example, only models trained with more than 102210^22 FLOPs of training compute (at sizes 2B, 4B, and 8B) can solve object-interaction tasks in RoboCasa via non-greedy planning, while smaller models trained with less compute cannot (see Figure 15). We deploy RoboJEPA on real and simulated robots, planning toward a single goal image that defines the final state of the environment. Across two robot platforms, we perform planning in over 50,00050,000 evaluation episodes. We keep the planning hyperparameters fixed for each robot, varying only the planning horizon for harder tasks. We perform planning on a set of real world manipulation tasks that include grasping, lifting, and pick-and-place. In simulation, our planning spans tasks of increasing difficulty from reaching to object pushing. We observe an improvement trend in the success rate on the real robot as we increase the model size. Qualitatively, we also observe that the behavior of the planning policies changes as the model grows: for example, in object-manipulation tasks, only models with 2B parameters or more keep the gripper closed after grasping to complete the task, while smaller models often accidentally open it, dropping the object. Finally, to support open research and reproducibility, we release all model checkpoints together with the training and robot-deployment code. To the best of our knowledge, this is the first work to establish compute-optimal scaling laws for action-conditioned robotic world models trained on large-scale, multi-embodiment real robot data, and the first scaling law for JEPA with verified extrapolation. We also train the largest JEPA predictor to date, at 8B parameters. 2 Method and Data To study the scaling behavior of robotic world models, we require a model family that scales with training compute. To this end we introduce RoboJEPA, a family of action-conditioned latent world models . The goal of RoboJEPA models is to predict how the latent features of a scene will evolve under the robot’s actions. Formally, at timestep t, given a history of past visual features z≤tz_≤ t (where ztz_t is the latent encoding of a given visual input xtx_t), proprioceptive states s≤ts_≤ t, and actions a≤ta_≤ t, our models are functions gθg_θ that deterministically predict the features of the next frame in one neural network forward pass: z^t+1=gθ([z≤t],a≤t,s≤t). z_t+1\;=\;g_θ ([z_≤ t],\,a_≤ t,\,s_≤ t ). (1) Actions are either state deltas, i.e. st+1=st+ats_t+1=s_t+a_t, or absolute robot commands, in which case st=0s_t=0. Feeding predictions back as inputs lets us roll the model out autoregressively over future actions, e.g. z^t+2=gθ([z≤t,z^t+1],a≤t+1,s≤t+1),z^t+3=gθ([z≤t,z^t+1,z^t+2],a≤t+2,s≤t+2),… z_t+2\;=\;g_θ ([z_≤ t, z_t+1],\,a_≤ t+1,\,s_≤ t+1 ), z_t+3\;=\;g_θ ([z_≤ t, z_t+1, z_t+2],\,a_≤ t+2,\,s_≤ t+2 ), … (2) This allows us to autoregressively roll out the world model to predict the effect of a sequence of actions on the latent features of the scene, and consequently to plan. Planning in RoboJEPA is specified by a goal image encoded once into goal features zgz_g, and at every timestep t we search for the H-step action sequence whose imagined latent rollout comes closest to the goal: at:t+H−1⋆=argminat:t+H−1∥z^t+H(at:t+H−1)−zg∥1,a _t:t+H-1\;=\; _a_t:t+H-1\; z_t+H(a_t:t+H-1)-z_g _1, (3) where z^t+H z_t+H is obtained by rolling gθg_θ forward H steps from the current context under the action sequence at:t+H−1a_t:t+H-1. We optimize Equation 3 with the cross-entropy method (a concrete instance is given in Sections 2.2 and F). The rest of this section describes the design and training of this family of world models and how to use them for planning, with the goal of subsequently studying their scaling behavior. 2.1 RoboJEPA Architecture Figure 3: RoboJEPA training. The frozen V-JEPA 2.1 encoder f (Mur-Labadia et al., 2026) maps the input multiview frames xt(1:V)x_t^(1:V) to a grid of features. The predictor gθg_θ consumes per-step sequences of multiview visual (zt(1:V)z_t^(1:V)), action (ata_t) and proprioceptive state (sts_t) tokens, conditioned on view and embodiment identifiers, and predicts the next-step features z^t+1 z_t+1. Training minimises two losses: (left) the teacher-forcing loss, the per-token ℓ1 _1 distance between predicted features and the next-step features from the encoder (Equation 5); and (right) the rollout loss, which attempts to predict the same target sequence as the teacher-forcing loss, but given fewer input frames. The model is given the full training sequence of actions but only a prefix of the encoded features, and predicts the remaining features autoregressively, with a KV-cache. This way the model learns to make accurate long term predictions. The family of RoboJEPA world models builds on the action-conditioned recipe of V-JEPA 2 (Assran et al., 2025) and V-JEPA 2.1 (Mur-Labadia et al., 2026). This subsection introduces the architecture of the RoboJEPA predictor. The full list of differences between RoboJEPA and V-JEPA 2-AC is summarised in Table 4 and detailed in Appendix A. Scaling world models across robot datasets is challenging because each dataset comes with a different number of camera views, proprioceptive state size, and controller type and frequency. Therefore, we aim to build a world model architecture that can work with any combination of views, controller type, and proprioceptive information. At each timestep t, a frozen V-JEPA 2.1-G encoder maps up to V simultaneous camera views at resolution H×WH× W, each spanning T frames, denoted xt(1:V)=(xt(1),…,xt(V))x_t^(1:V)=(x_t^(1),…,x_t^(V)), to the latent features zt(1:V)=(zt(1),…,zt(V))z_t^(1:V)=(z_t^(1),…,z_t^(V)), where each zt(v)z_t^(v) is an Hp×WpH_p× W_p grid of DeD_e-dimensional patch embeddings with De=1664D_e=1664 and Hp=H/16H_p=H/16, Wp=W/16W_p=W/16. We use V-JEPA 2.1 as the encoder because early in the work we found that it leads to successful and predictable scaling of the world model, whereas V-JEPA 2 does not (we hypothesize why in Appendix K). Alongside these features, each timestep provides the action at∈ℝdaa_t ^d_a and proprioceptive state st∈ℝdss_t ^d_s, while each input is associated with a robot type operating in a particular action space, identified by the robot id e∈ℰe , and a per-view camera type c(v)∈c^(v) . Here, ℰE and C are finite sets of known robot ids and camera types. Next we describe how we tokenize inputs, and the world model architecture. Table 1: Compressed view of the 2323-dataset, 2626-action-space mixture. Video hours sum the durations of all recorded videos. Action hours measure the total duration of teleoperation or policy execution, counted once regardless of the number of cameras. The per-dataset breakdown with resolutions and source FPS is in Table 7; sampling weights are in Table 8; and semantic-camera mappings are in Table 9. We report dataset sizes that we were able to collect and verify, which may be smaller than those reported in the original papers. Dataset group Episodes Video hours Action hours Embodiments DROID (Khazatsky et al., 2024) 92k 414 138 Franka Roboset (Bharadhwaj et al., 2024) 73k 1244 380 Franka (Roboset) RoboMind (Wu et al., 2024b) 35.3k 276 98 Franka, AgileX LeRobot (Cadene et al., 2024) 21.5k 182 93 SO-101 RoboCasa365 (Nasiriany et al., 2026) 32k 482 482 Franka + Omron (sim) 1X (1X Technologies, 2024) 23.5k 90 90 1X humanoid AgiBot World (AgiBot-World-Contributors et al., 2025) 167.5k 8036 2734 AgiBot dual-arm, humanoid Open-X Embodiment (Open X-Embodiment Collaboration et al., 2023)11 1 We use 9 datasets of Open-X Embodiment (Open X-Embodiment Collaboration et al., 2023): RT-1 (Brohan et al., 2023), Bridge (Ebert et al., 2022), Bridge V2 (Walke et al., 2023), QT-Opt (Kalashnikov et al., 2018), MT-Opt (Kalashnikov et al., 2021), RoboNet (Dasari et al., 2019), Language-Table (Lynch et al., 2023), BC-Z (Jang et al., 2022), FMB (Luo et al., 2024). 2.4265M 4298 2677 8 platforms Total 2.8713M 15,022 6,692 12 platforms / 26 ids Tokenization. We encode each modality separately and interleave the results into a single flat sequence. We refer to these encoded multimodal vectors as tokens. The motivation of our tokenization scheme is to let the model learn all temporal dependencies only from a structured sequence of tokens. To tokenize each zt(v)z_t^(v), we project each patch from DeD_e to the world-model width D with a single linear layer, giving a visual token tensor of shape [T,V,Hp,Wp,D][T,V,H_p,W_p,D]. Actions and proprioceptive states are mapped to D by linear encoders (specific to robot id) applied to ata_t and sts_t. We interleave these tokens by timestep, (z1(1:V),a1,s1),(z2(1:V),a2,s2),…,(zT(1:V),aT,sT), (z_1^(1:V),\,a_1,\,s_1 ),\; (z_2^(1:V),\,a_2,\,s_2 ),\;…,\; (z_T^(1:V),\,a_T,\,s_T ), (4) so that each timestep contributes VHpWpVH_pW_p visual tokens followed by one action and one state token. Beyond this ordering we impose no structure: the model consumes a flat sequence and recovers spatio-temporal and view structure from the position encodings and external conditioning information, such as the robot id and view type. World Model Architecture. RoboJEPA performs the next embedding prediction via a deterministic model. RoboJEPA uses the vision transformer (Dosovitskiy et al., 2021) as the backbone and we parameterize the prediction such that each spatial input token of zt(1:V)z_t^(1:V) predicts the corresponding spatial token of zt+1(1:V)z_t+1^(1:V). We use the frame-level causal attention mask so that queries that correspond to timestep t have only access to keys and values at step t or earlier. The model stacks L blocks that form a standard residual stream of ViT. The architecture of each block follows the standard recipe for modern variants of ViTs: we use pre-normalization, Rotary Position Encoding (Su et al., 2024), RMSNorm normalization (Zhang and Sennrich, 2019), GELU feed-forward block (Hendrycks and Gimpel, 2016), and QK-normalization (Dehghani et al., 2023). We found QK-normalization to be of particular importance for training stability when scaling the dataset size and the model size. Factorized RoPE and view conditioning. The spatio-temporal position information is supplied by a multi-axis Rotary Position Embedding (Su et al., 2024) that splits each head’s channels into four equal sub-blocks of width dh/4d_h/4, one for each of the four axes (t,i,j,v)(t,i,j,v) for time, height, width, and view, where dhd_h is the head dimension. Our version of RoPE rotates each independently by its own per-axis frequency (Section A.1). Action and state tokens have a temporal index but no spatial index, i.e. the 4D index of the action ata_t would be (t,0,0,0)(t,0,0,0). We combine this 4D RoPE with a per-layer learnable view bias applied after RoPE. The idea is that the 4D RoPE informs the model that tokens belong to different views, while the view bias informs the model which content each view holds (e.g. left camera, wrist camera; Section A.2). This allows us to randomize the order of views and at the same time provide the information which views the model works with. Attention pattern. We use joint space-time-view attention and do not factorize it, meaning that each token can have access to potentially all other tokens in the space-time-view sequence, unless restricted by the attention mask. However, using no mask would allow past token activations access future information that they need to predict. For example, token activations of ztz_t will have access to tokens of zt+1z_t+1 in self-attention. Therefore, we use a timestep-causal mask (Assran et al., 2025; Po et al., 2025; PAN Team et al., 2025). That is, a query associated with timestep t has access to keys only at timestep t or less. This includes attention queries and keys associated with different views, actions, and states. In addition to that, we make the attention local by letting queries only access keys that are at most 88 timesteps in the past (Assran et al., 2025; Po et al., 2025; PAN Team et al., 2025). This design helps the model extrapolate to predicting longer sequences of embeddings because no query will ever attend to a history longer than the model was trained on, including during long autoregressive prediction. For implementation details on the attention masking, see Section A.3. Outputs. After feeding the entire sequence of tokens through the backbone, we project tokens that correspond to z(1:V)1:Tz^(1:V)_1:T back to the encoder space. We map the output from D dimensions to DeD_e dimensions and apply layer normalisation. The idea is that since both input and output features are normalized, this strenghtens the learning signal and lowers the gap between the training and the autoregressive generation. We do not use any learnable modules to predict states and we simply compute the proprioceptive state sts_t either through integration, i.e. st+1=st+ats_t+1=s_t+a_t, or st=0s_t=0 (for example when working with commanded actions with non-trivial robot dynamics). Training objective. We train the world model to predict the next embedding and, to minimize exposure bias, we additionally train it on its own outputs over potentially long horizons. Our objective therefore has two loss components. The first component trains the model to predict every next embedding along a sequence of encoded multiview frames. The second component uses the same objective, but the model is given only a prefix of fewer encoded frames (but the entire sequence of actions); it then predicts the missing embeddings autoregressively (Equations 1 and 2) and is trained on them with the same objective. We use a key-value cache to speed up this autoregressive prediction (Sections 2.2 and H). While the first loss component ensures that the model predicts the representations of a fixed encoder, the second makes the world model robust to its own errors and suppresses error accumulation. Both components use the same per-token ℓ1 _1 objective, and the total loss is their average: ℒTF=∑t,v∥^t+1(v)−zt+1(v)∥1,ℒAR=∑t,v∥~t+1(v)−zt+1(v)∥1,ℒ=12ℒTF+12ℒAR,L_TF= _t,v [rgb]0.1055,0.4219,0.6602 z_t+1^(v)-z_t+1^(v) _1, _AR= _t,v [rgb]0.7539,0.3516,0.0938 z_t+1^(v)-z_t+1^(v) _1, = 12L_TF+ 12L_AR, (5) where ^t+1(v) [rgb]0.1055,0.4219,0.6602 z_t+1^(v) is the teacher-forced one-step prediction and ~t+1(v) [rgb]0.7539,0.3516,0.0938 z_t+1^(v) is the autoregressive prediction rolled out from a shorter prefix, both compared against the encoder feature zt+1(v)z_t+1^(v). The full form of the loss (with the per-frame mask and patch normalization), the rollout schedule, and the optimizer are described in Sections 2.2 and C. 2.2 Data, Training and Inference Data. Training a world model that generalizes across robot embodiments and scenes requires both scale and diversity. We assemble 2323 public manipulation datasets spanning 1212 robot platforms—single- and dual-arm robots and humanoids—across real and simulated data. The compiled dataset contains 15,02215,022 hours of video, including 6,6926,692 hours synchronized with actions; some demonstrations are recorded from multiple cameras. Some platforms record with only one camera (e.g. most of Open-X Embodiment data), whereas others use two or three cameras (e.g. DROID and RoboSet). In addition, the datasets were collected with different robot action spaces, camera poses, sensor resolutions, and frame rates. Therefore, to build a data-scalable world model, we unify our training datasets and formulate the architecture and training recipe to be general enough to work with any robotic dataset. Table 1 summarizes the mixture, and Appendix B provides further details about its action spaces, unification, and sampling. Figure 4: Two-view imagination scales in simulation. First 6060 s of a RoboCasa rollout (two agent-view cameras, stacked within each cell) imagined by RoboJEPA at 0.30.3B, 22B, and 88B parameters. Larger models keep the two views mutually consistent and preserve manipulation detail markedly deeper into the rollout. All world models share the same encoder (V-JEPA 2.1-G) and decoder. Training. We train a series of RoboJEPA models that range in size from 2222M to 88B parameters and in training compute from 2×10192×10^19 to 9.5×10229.5×10^22 FLOPs. For all models, we use AdamW (Loshchilov and Hutter, 2019) with a weight decay of 0.10.1 and global gradient-norm clipping at 1.01.0. We linearly warm up the learning rate, hold it constant at a peak of 5×10−55× 10^-5 for most of training, and then linearly decay it to zero over a short cooldown phase at the end (Hu et al., 2024). This allows us to overlap the flat learning-rate stage across models trained with different amounts of compute: we run the flat stage only once and perform learning-rate annealing from intermediate checkpoints. For the detailed description of the training configuration and hyperparameters, please refer to Appendix C. The training recipe of RoboJEPA is designed to facilitate scaling the model on as much data as is available, which requires training on many heterogeneous robotic datasets at once. A central issue with such heterogeneous data is that some datasets supply only a single view while others provide multiple views, often at different resolutions. We therefore train the model to work with a variable number of views and a variable resolution. To achieve this, we partition the GPUs into groups, where all GPUs within a group work with the same number of views and the same resolution. In practice this often leads to imbalanced step times across groups, which is why we balance these times by using different batch sizes for different groups. Our initial scaling-law experiments were done at a resolution of 240×320240× 320 with only two GPU groups: one working with single-view data and the other with dual-view data. However, eventually we found it beneficial to include high-resolution GPU groups, resulting in four groups: single-view low-resolution (240×320240× 320), dual-view low-resolution, single-view high-resolution (480×640480× 640), and dual-view high-resolution. We describe the GPU grouping and batch-size balancing in detail in Sections C.1 and C.2. subfigure [t]0.49 subfigure[t]0.49 Figure 5: DROID holdout (real Franka, unseen scenes). Figure 6: RoboCasa target scenes (simulation). Figure 7: World-model scaling laws. Out-of-distribution forward-prediction L1L_1 error versus training compute for RoboJEPA models from 22M to 8B parameters (color), evaluated on a real-robot DROID holdout and simulated RoboCasa scenes. The compute-optimal frontier (lower envelope) follows the second-order power law L(C)=E+ACα−γlnCL(C)=E+A\,C^α-γ C (dashed). Inference and Planning. RoboJEPA performs planning by predicting the future in the latent space and searching over the action sequence that brings that future to the desired goal. We employ receding-horizon CEM (Rubinstein, 1997; de Boer et al., 2005) as our planning algorithm with image goals; importantly, we use only a single image goal that represents the final state of the task, and we do not use subgoals. Notably, we run all planning with the same set of hyperparameters across embodiments and tasks. This process requires running thousands of forward passes of the world model, which is prohibitively expensive if the full history is recomputed every step. Therefore, we employ a rolling KV-cache that reuses the keys and values of past steps in self-attention. This way, at every step the model only processes a single frame, action, and proprioceptive vector, while the past information is provided by the cache (see Appendix H). Additionally, we employ PyTorch compilation to further speed up inference. However, in the typical planning scenario we need to run compiled inference during planning on the real robot, and since first-time compilation is time-consuming, running planning with compilation can become prohibitively slow in practice. This is why we build a simple inference engine that precompiles the inference computation and saves it to disk, and at planning time simply loads it back (see Section I.2). Finally, since CEM planning is heavily parallelizable, we connect the robot directly to the GPU cluster and run the planning on multiple GPUs. This way, we make the planning fast enough to be practical: while it is not real-time, it takes only several seconds to perform a multi-step planning even for an 88B model. 2.3 Diffusion Decoder To qualitatively understand the capabilities of our world models, we train a diffusion decoder that produces pixel outputs from the predicted features; its goal is to visualize the forward predictions of the world model. The decoder is a diffusion model (Ho et al., 2020; Song et al., 2021) that denoises the continuous video tokens of the Cosmos tokenizer (NVIDIA, 2025)11 1 Cosmos-CV4x8x8. spatially conditioned on the V-JEPA 2.1 features. The purpose of using a generative model here is that the decoder does not merely convert the V-JEPA features into Cosmos tokens, but also imagines unseen motion. During the training of the decoder, we compute V-JEPA features with a frame skip of 44 frames, whereas the Cosmos tokens compress the same frames without any frame skip. As a result, the Cosmos tokens contain more information than the original V-JEPA feature frames, and this extra information — precisely, the 33 RGB frames between consecutive frames encoded by the V-JEPA encoder — must be generated. With such a decoder, we can run forward prediction with the world model at, for example, 55 Hz actions, producing latent features at the same rate, and then use the decoder to expand them into an RGB stream at 2020 FPS that covers exactly the same motion. Full architecture, training, and sampling details are in Appendix D. 3 Experiments 3.1 World Model Scaling Laws Our goal is to empirically find the concrete functional form of the scaling law that allows us to extrapolate predictions about the performance of the RoboJEPA model trained at larger scales. We consider four candidate functional forms. First, the standard power law L(C)=E+ACαL(C)=E+A\,C^α (Henighan et al., 2020), whose exponent α is constant. Second, a generalization whose exponent itself changes linearly with compute, L(C)=E+ACα−γlnCL(C)=E+A\,C^α-γ C; we refer to the former as a first-order and to the latter as a second-order power law22 2 It is easy to show that, for the latter, the dependency between ln(L−E) (L-E) and lnC C is a quadratic function of lnC C.. Finally, the Broken (Caballero et al., 2022) and Unified (Caballero et al., 2026) Neural Scaling Laws. Here, C is the total training compute (in FLOPs) and L(C)L(C) is the world model’s forward-prediction error, i.e. the per-token ℓ1 _1 distance between the predicted and the encoder features, averaged over a 99-step latent rollout on held-out data. Among these four forms, we find that RoboJEPA predictably follows the second-order power law. Setup. We evaluate each model trained at each compute budget by its forward-prediction L1L_1 error, L(C)L(C), on held-out data. Importantly, this is the same per-token L1L_1 objective the model is trained on and that the planner minimizes at deployment (Equation 5). We report L(C)L(C) on two embodiments, a DROID holdout dataset collected on a real robot and a simulated RoboCasa suite. Evaluation is always on scenes unseen during training, always uses two views: one side camera (left or right) and one wrist camera, and predicts nine future steps from an initial observation under the recorded action sequence, giving a ten-step sequence in total. Further evaluation details are provided in Appendix J. For each model at each training duration we compute the total training compute C exactly, by counting the FLOPs of every operation in one training step and multiplying by the number of steps taken across all stages. We plot the prediction error for every model and training duration against its total training compute. At each compute budget, we retain the lowest error across model sizes. These best-achievable errors form the compute-optimal frontier as a function of the training compute. Finally, we fit each candidate curve to the frontier points of the smaller models (22M–2B) and select as our scaling law the one that best extrapolates to the held-out 4B and 8B models. Table 2: Variants of parametric curves as scaling-law candidates for the L1L_1 rollout error of RoboJEPA world models. Curve fits use models sized 2222M–22B parameters; we report held-out extrapolation error at 4B and 8B for DROID and RoboCasa. The BNSL and UNSL functional forms and their fitted parameter values are in Appendix J. Parametric Curve Curve parameters Extrapolation Error (4B and 8B), ×10−3× 10^-3 DROID RoboCasa DROID RoboCasa L(C)=E+ACαL(C)=E+A\,C^α E=0.182,A=108.8,α=−0.151E=0.182,\ A=108.8,\ α=-0.151 E=0.075,A=23.2,α=−0.103E=0.075,\ A=23.2,\ α=-0.103 2.02.0 3.73.7 L(C)=E+ACα−γlnCL(C)=E+A\,C^α-γ C E=0.218,A=7.96×10−19E=0.218,\ A=7.96×10^-19 α=1.91,γ=0.02308α=1.91,\ γ=0.02308 E=0.1709,A=1.66×10−17E=0.1709,\ A=1.66×10^-17 α=1.761,γ=0.02105α=1.761,\ γ=0.02105 0.60.6 1.41.4 BNSL (Caballero et al., 2022), n=1n=1 Section J.1 1.01.0 1.81.8 UNSL (Caballero et al., 2026), n=1n=1 Section J.2 1.41.4 5.75.7 Results. Table 2 compares the four candidate curves by how well a fit on the smaller models (22M–2B) extrapolates to the held-out 4B and 8B models. The second-order power law extrapolates best, with a held-out error of 0.60.6 and 1.4×10−31.4× 10^-3 on DROID and RoboCasa respectively, roughly 22–3×3× lower than the standard power law (2.02.0 and 3.7×10−33.7× 10^-3) and below both BNSL and UNSL. We therefore adopt it as our scaling law. Figure 7 plots the forward-prediction error against training compute for all models. The lower envelope across model sizes forms the compute-optimal frontier, and no model size lies on the frontier entirely. In addition, Figure 7 plots the scaling law computed on all models as the dashed line. Interestingly, the two best candidates, the second-order law and BNSL, estimate nearly the same irreducible error, ≈0.2≈ 0.2 on DROID and ≈0.17≈ 0.17 on RoboCasa, indicating that RoboJEPA is close to saturation on both evaluations under its current training and data budget. 3.2 Downstream Planning Scaling subfigure [t]0.49 subfigure[t]0.49 Figure 8: The normalized return for the pose reach task in RoboCasa environment evaluated in unseen kitchens. Figure 9: The normalized return for the task of pose reach while holding an object. Figure 10: Downstream planning success scales with training compute. Median success rate on the reach task versus total training compute, following a power-law fit. Having identified an improvement trend in offline performance of world models, we hypothesize that this improvement translates to the world model’s physical understanding of the new environment in which it operates. For example, is the action conditioning accurately mapped to the correct movements of the robot in the representation space? Are the grasping and object interaction actions correctly represented by the world model in response to incoming action? We test this hypothesis via online planning with image goals. Section 3.1 introduced the predictable improvement in offline forward-prediction, and we verify that this improvement translates into a better success rate. To test this hypothesis, we build a set of interactive environments on top of RoboCasa 365 (Nasiriany et al., 2026), each isolating one basic capability, and test whether more training compute improves it. In this section we focus on greedy tasks, which can be solved by one-step planning where simply moving toward the goal at every step is enough. (i) 3D understanding (pose reaching): reach a target end-effector pose in as few steps as possible. (i) Object understanding (pose reaching while holding an object): reach a target pose while holding a grasped object, so the planner must additionally search over gripper actions to avoid dropping it. Harder tasks that require non-greedy, longer-horizon planning — such as moving around an obstacle or pushing an object to a target — are studied in Section 3.3. Full environment and reward details are in Section G.4. We use the same range of RoboJEPA checkpoints as in Section 3.1 and run each of them in our RoboCasa environments with closed-loop planning (Mayne et al., 2000), where the model re-plans toward the goal image after every executed action (full protocol in Section G.4). Figure 10 shows the evaluation results, measured by the simulator’s normalized cumulative reward on the performed trajectory (Section G.4). Just like with the offline performance, we see a clear improvement trend of the downstream performance as we scale the training compute. For both tasks, models form a clear performance frontier, with larger models surpassing smaller ones as the training compute grows. For example, the simplest end-effector 3D movement capabilities emerge after around 102010^20 FLOPs while the basic capability of moving while holding an object emerges at around 3×10203× 10^20 FLOPs of training, both of which correspond to training 5050M–100100M-parameter models on the full multi-embodiment dataset. Due to the simplicity of these tasks, performance saturates early. Therefore, for building robotic world models, we recommend first building downstream evaluations that reflect the desired capabilities, then estimating the minimal training compute required to achieve them. 3.3 Better Scaling for Long Horizon Planning With the established transfer from offline performance to online greedy performance in Sections 3.1 and 3.2, we now ask whether the same offline-to-online connection holds for non-greedy, longer-horizon tasks that cannot be solved in one step. We design two such tasks in RoboCasa. The first is an obstacle-reach task: the goal is again to reach a target end-effector position, but we place a large object (a kitchen appliance) as an obstacle so that a direct straight-line motion cannot reach the goal. The second is an object-push task, where an object is placed on the table in front of the robot and the goal is to move it toward a target position, which requires the arm to descend to the object, push it, and ascend back. Full task definitions are in Section G.4. We first deploy the models trained with the recipe of Section 3.1 on the obstacle-reach task. Figure 11 shows that the final reward correlates with training compute, but the correlation is weak and no clean compute Figure 11: Short-horizon prediction training on obstacle-reach. Final reward versus training compute. The reward correlates with compute, but no clean frontier emerges and smaller models often beat larger ones. frontier emerges: smaller models often beat larger ones (for example, the 300300M planner reaches a higher final reward than the 11B one). Unlike the greedy tasks of Section 3.2, predictable scaling does not emerge for longer-horizon tasks, which we attribute to a mismatch between training and deployment. Planning on these tasks relies on accurate long-horizon prediction, yet the models were trained with only a short autoregressive rollout (K=2K=2 steps). We therefore align training with how the model is used at test time and increase the training rollout to K=10K=10 steps during the cooldown, together with higher-resolution inputs. We refer to the two cooldown recipes as short-horizon prediction training (K=2K=2, 240×320240× 320) and long-horizon prediction training (K=10K=10, a mix of 240×320240× 320 and 480×640480× 640). Retraining every model from scratch this way would be prohibitively computationally expensive, so we run the long-horizon cooldown only from the flat-learning-rate pretraining checkpoints (Section 2.2): the long flat-learning-rate stage only needs to learn the coarse dynamics cheaply, and the computationally expensive long-rollout, high-resolution signal is spent where it matters, in the final annealing phase. subfigure [t]0.49 subfigure[t]0.49 Figure 12: Imagination error scaling for short- vs. long-horizon prediction training. Figure 13: Obstacle-reach final reward, long-horizon prediction training. Figure 14: Longer-horizon prediction training improves scaling and downstream long-horizon planning. (a) Second-order scaling laws of the imagination error on DROID and RoboCasa for the two cooldown recipes. (b) Obstacle-reach final reward versus total training compute for the long-horizon (K=10K=10) models. Figure 14 compares the two recipes by their scaling laws on DROID and RoboCasa. Long-horizon prediction training is more computationally expensive per step, but it becomes more compute-efficient beyond about 102110^21 FLOPs of total training compute (around 300300M–11B parameters at the compute-optimal frontier) and reaches a substantially lower forward-prediction error. It also lowers the irreducible error of the fit, indicating a genuine reduction in error accumulation. Figure 15: Downstream success rate on the RoboCasa push task versus training compute, for the long-horizon (K=10K=10) models. Deploying the non-greedy planning models on the obstacle-reach task (Figure 14) significantly improves the trend: the final reward now strongly follows the predictable scaling pattern. Small models (below 300300M parameters) cap at around 0.30.3, which corresponds to the end-effector remaining stuck on the wrong side of the obstacle. The obstacle-avoidance capability then takes off around 102110^21 FLOPs and saturates around 102210^22 FLOPs. This is much later than the end-effector capabilities of Section 3.2, which emerged around 102010^20–3×10203× 10^20 FLOPs, indicating that modeling interaction with the static scene geometry requires considerably more training. Finally, the object-push task assesses the object interaction understanding (through non-greedy planning) by the world model: because the robot’s start and goal poses are identical (so the side and wrist views change only through the object’s position), the model must understand that pushing the object is what changes the scene. Figure 15 shows the success rate, defined simply as whether the object moved toward its target position (Section G.4). Interestingly, no model achieves a positive success rate until around 102210^22 FLOPs, after which all models begin to succeed, suggesting that fine-grained understanding of surrounding objects emerges after certain training computation regardless of the model size. In summary, these results align in a natural order in which capabilities appear with more compute: first the 3D control of the end-effector, then manipulating a grasped object, then the geometry of the static scene, and finally the dynamics of objects in the scene. 3.4 Real World Robotic Capabilities Figure 16: Real-world deployment on DROID. Success rates across model sizes for three manipulation tasks; the dashed line shows their average. Our ultimate goal is to showcase the capabilities of RoboJEPA in real-world settings. As a step towards this goal, we deploy RoboJEPA models ranging from 22M to 8B parameters on the DROID (Khazatsky et al., 2024) platform with a single-arm Franka robot and two cameras: an external camera mounted to the left of the arm and a wrist camera using the standard DROID mount. To deploy RoboJEPA via planning, we record an image goal on the robot by executing a predefined robot motion and manually placing the object in the grasping position so the robot can interact with it. We provide only a single final image goal, requiring longer-horizon, non-greedy planning. We evaluate three tasks: Grasp, Object Lift, and Pick and Place. We use the same planning hyperparameters across tasks and model sizes, evaluating the highest-compute checkpoint at each size. Setup, task definitions, episode counts, scoring criteria, and planning hyperparameters are detailed in Sections G.1 to G.3. To contextualize the RoboJEPA results, we also evaluate the Vision-Language-Action (VLA) models π0 _0-FAST (Pertsch et al., 2025) and π0.5 _0.5 (Black et al., 2025). However, their deployment differs in goal specification (text instructions versus a single goal image) and training (DROID fine-tuning for the VLA baselines versus pretraining on a mixture that includes DROID for RoboJEPA). Neither RoboJEPA nor the VLA baselines receive task-specific fine-tuning. These differences make the VLA results contextual references rather than directly comparable baselines. Figure 16 shows the scaling trend. For each task, all RoboJEPA models, including very small models, demonstrate non-zero success rates. Overall, success rates show a clear trend of improving with model size and generally decreasing with task complexity. This picture does not hold for the reference models, which work best on Pick and Place, the hardest task, but degrade significantly on the others. For example, π0.5 _0.5 achieves a 53%53\% success rate on Pick and Place but 0%0\% on Object Lift. We hypothesize that this difference may be an artifact of training, as verbs such as “lift” are heavily underrepresented in the DROID dataset, which focuses mostly on pick-and-place tasks. Due to the planning nature of the agent and the task specification as an image goal, RoboJEPA models are not susceptible to this type of degradation. Qualitatively, behavior transitioned gradually from smooth but less adaptive execution in smaller models to more reliable recovery from mistakes in larger models (Section G.7). Table 3 reports task progress and success rates for the largest RoboJEPA models and VLA references. Table 3: Real-robot task progress and success rates (%). Full results are in Table 12. Model Goal Grasp Object Lift Pick and Place Progress Success Progress Success Progress Success π0 _0-FAST Text 35 22 20 12 55 40 π0.5 _0.5 25 5 12 0 69 53 RoboJEPA-4B Image 60 60 54 30 49 21 RoboJEPA-8B 67 67 65 50 42 27 3.5 Qualitative Analysis So far we have quantified the world model through its prediction loss and downstream planning success; we now inspect how the prediction improves qualitatively by decoding its latent rollouts back to pixels. To do that, we use the diffusion decoder introduced in Section 2.3. The decoder serves purely as a visualization tool and is trained only on the frozen V-JEPA 2.1-G encoder features (see Appendix D). Note that all visualizations use the same encoder and decoder and vary only the RoboJEPA world model size. We perform forward prediction given a single frame (or a pair of frames) and a sequence of robot actions, and each sequence of predicted embeddings is decoded back to pixels. We run the prediction across a range of settings — low- and high-resolution, single- and dual-view — and report them separately (Figures 2 and 4; full set in Appendix E); the decoder itself is trained at a single 240×320240×320 resolution. These visualizations show that the forward-prediction improves with model scale in several concrete ways: larger models track the true dynamics more accurately over the rollout, preserve object identity and fine manipulation detail, and keep multiple camera views mutually consistent. Several behaviors are effectively emergent with scale — gripper–object contact and structured deformation such as folding appear only once the predictor is large enough, whereas smaller models blur these interactions away. Finally, we finetune the 88B version of RoboJEPA on DROID to imagine all three camera views simultaneously at high (720720p) resolution (Section C.4; Figure 17). These rollouts demonstrate that RoboJEPA performs accurate object manipulation even at very high resolution, and they exhibit strong cross-view information transfer, with content from the side (exterior) cameras propagating into the wrist-camera imagination. 4 Related Work Joint-Embedding Predictive Architectures. Joint-Embedding Predictive Architectures (JEPAs) were proposed as a path towards self-supervised models that learn abstract representations of the world by predicting in a latent space rather than at the pixel level (LeCun, 2022). I-JEPA (Assran et al., 2023) instantiated this recipe for images by predicting the representations of masked image regions from visible context, avoiding pixel-level reconstruction and the resulting collapse to low-level texture. V-JEPA extended the framework to video by predicting masked spatio-temporal feature regions and showed that the resulting features capture motion and temporal structure (Bardes et al., 2024). Assran et al. (2025) scaled feature prediction to over a million hours of internet video and, crucially for our setting, post-trained an action-conditioned predictor (V-JEPA 2-AC) on a small amount of unlabeled robot interaction data, enabling zero-shot Franka manipulation by latent planning against image goals. Our work, RoboJEPA, is similar to V-JEPA 2-AC in terms of the approach. We keep the V-JEPA encoder frozen and focus on the action-conditioned predictor as a world model. However, compared to V-JEPA 2 AC, we replace the small post-training corpus with a unified, large-scale mixture of robot demonstrations, therefore, the methods are not apples-to-apples comparable. To our knowledge, we provide the first systematic recipe for scaling V-JEPA-style action-conditioned predictors on large-scale, multi-embodiment robot data, and we characterize how predictor performance scales with parameters and demonstration tokens. World Models for Robotics. World models compress experience into a learned simulator of the environment that an agent can train and plan in (Ha and Schmidhuber, 2018). The Dreamer family pioneered this paradigm at scale: PlaNet learned latent dynamics for planning from pixels (Hafner et al., 2019), Dreamer used latent imagination to learn behaviors (Hafner et al., 2020), DreamerV2 introduced discrete latents to master Atari (Hafner et al., 2021), and DreamerV3 demonstrated a single set of hyperparameters that succeeds across a wide range of domains including 3D control (Hafner et al., 2023). DayDreamer applied Dreamer directly to physical robots and learned locomotion and manipulation in the real world (Wu et al., 2023), while Masked World Models showed that decoupling visual representation learning from dynamics modeling improves sample efficiency (Seo et al., 2023). More recent work has scaled video-based world models with transformer and diffusion backbones: IRIS (Micheli et al., 2023) and iVideoGPT (Wu et al., 2024a) treat world modeling as autoregressive token prediction; UniSim (Yang et al., 2024), Genie (Bruce et al., 2024), and GAIA-1 (Hu et al., 2023) train large generative simulators of interactive scenes; and the Cosmos platform releases physical-AI world foundation models at industrial scale (NVIDIA, 2025). In contrast to these pixel-space generators, RoboJEPA predicts in the representation space of a frozen V-JEPA encoder, which is what allows us to scale the predictor to billions of parameters on robot data without paying the computational cost of pixel-level reconstruction at every step. Planning with Learned World Models. A central use of a world model is test-time planning. Classical model-based control methods such as PETS (Chua et al., 2018), model-based RL with neural network dynamics (Nagabandi et al., 2018), and visual foresight (Finn and Levine, 2017) optimize action sequences against a learned dynamics model. Sampling-based shooting methods, in particular the Cross-Entropy Method (Rubinstein, 1997; de Boer et al., 2005) and Model Predictive Path Integral control (Williams et al., 2017), remain the workhorses for high-dimensional action search. TD-MPC and TD-MPC2 combine latent dynamics, value learning, and CEM-style trajectory optimization for robust continuous control (Hansen et al., 2022; Hansen et al., 2024). DINO-WM (Zhou et al., 2025) learns task-agnostic dynamics in DINOv2 feature space from offline trajectories using an approximately 2020M-parameter predictor, and uses CEM to plan toward image goals without additional task-specific training. RoboJEPA follows this line: at deployment, we use CEM in the latent space of the world model to score action sequences by how close their predicted features land to a goal image’s features, and execute the first action of the best plan in receding-horizon fashion. Closely related to our work, Terver et al. (2026) investigate how architectural, training, and planning choices affect latent world-model performance, including model and data scaling, albeit at smaller scales. Their proposed DROID predictor has approximately 229229M parameters and is trained on an 8,0008,000-trajectory dataset. We extend this investigation both by scaling to substantially larger world models (up to 88B parameters) and datasets (a mixture of approximately 2.872.87M trajectories across 1212 robot platforms) and by explicitly studying their scaling behavior through scaling laws. Robotic Foundation Models from Demonstrations. A complementary line of work scales policies directly on robot demonstrations rather than scaling world models. RT-1 (Brohan et al., 2023) and RT-2 (Zitkovich et al., 2023) train large transformer- and vision-language-action policies on broad manipulation data. Open X-Embodiment Collaboration et al. (2023) aggregate demonstrations across many embodiments to enable cross-embodiment generalist policies, and a wave of recent generalist models such as Octo (Ghosh et al., 2024), OpenVLA (Kim et al., 2024), and π0 _0 (Black et al., 2024) push this behavior-cloning recipe further. RoboJEPA shares the same data substrate (in particular Open X-Embodiment, DROID (Khazatsky et al., 2024), RoboMIND (Wu et al., 2024b), BridgeData V2 (Walke et al., 2023), LeRobot (Cadene et al., 2024), and AgiBot World (AgiBot-World-Contributors et al., 2025)), but trains a goal-agnostic, action-conditioned predictive model instead of a task-conditioned policy. The same world model can then be used to evaluate or improve any policy, and to plan zero-shot for new goals at test time. Scaling Laws. Power-law scaling of model loss with parameters, data, and compute has been documented extensively for language models (Kaplan et al., 2020), generative modeling more broadly (Henighan et al., 2020), and vision transformers (Zhai et al., 2022), with compute-optimal allocations of model and data established by Chinchilla (Hoffmann et al., 2022). In robotics, however, scaling laws are largely an open question because high-quality interaction data is scarce. We provide, to our knowledge, the first systematic scaling study for action-conditioned world models trained on large-scale, multi-embodiment real robot demonstrations, characterizing how forward-prediction loss scales with predictor parameters and demonstration tokens, and showing that this scaling behavior transfers to downstream planning performance. Scaling world models. A growing body of work reports that world models improve with scale. In autonomous driving, the GAIA line of latent-diffusion world models has been scaled to billions of parameters for controllable simulation and offline evaluation (Russell et al., 2025; Wayve, 2025; Wayve, 2026), though technical details beyond GAIA-2 are not publicly available. Pearce et al. (2024) fit power laws for pre-trained agents and world models and, in their appendix, report scaling laws for a robotic world model, but their study is otherwise centered on game and simulated environments rather than large-scale real-world multi-embodiment robot data. Sato et al. (2023) study how world-model quality varies with model size; IRASim (Zhu et al., 2025) reports empirical scaling trends for a fine-grained video world model for manipulation; PointWorld (Huang et al., 2026) shows that a 3D point-flow world model improves with more data and parameters; and DINO-world (Baldassarre et al., 2025) scales video prediction in the feature space of a frozen DINOv2 encoder using predictors with up to 1.11.1B parameters. Closest in spirit, τ0 _0-VLA (Cai et al., 2026) scales test-time computation of a hierarchical policy using a world model to score candidate subtasks, and Dyna-2 (Dyna Robotics, 2026) scales a world-action model over a million hours of human and robot video and measures action-prediction quality. RoboJEPA differs from these along two axes: we scale training compute of a pure world model and measure its future-embedding (imagination) accuracy, and we fit an explicit scaling law rather than an empirical trend. This is a very important distinction, as an empirical scaling trend only establishes that a metric improves as one adds parameters, data, or compute, but says nothing about how much it will improve, whether returns have begun to saturate, or whether the next unit of budget is better spent on model size or on data. A scaling law, by contrast, is predictive: it extrapolates performance well beyond the scale at which it was fit, turning these decisions into quantitative forecasts. Establishing such predictive laws for action-conditioned robotic world models—and connecting them to downstream planning—is, to our knowledge, what sets this work apart. 5 Conclusion We introduced RoboJEPA, a family of action-conditioned latent world models trained on a large-scale, multi-embodiment robotic data collection in the latent space of V-JEPA 2.1. RoboJEPA predicts the next embedding in a single forward pass, and we train it to predict the future conditioned on past embeddings and future actions. We presented a recipe for scaling the predictor that works from 22M to 8B parameters, the latter being the largest JEPA predictor model trained to date. We then established scaling laws that predict the world-modeling error as we scale training compute. These take the form of a second-order power law L(C)=E+ACα−γlnCL(C)=E+A\,C^α-γ C. We established these laws for both a real-world (DROID) and a simulated (RoboCasa) embodiment. Finally, we demonstrated that offline prediction scaling laws correlate with the online downstream performance of world models, evaluated via planning toward a single image goal, which makes the offline imagination error a reliable proxy for real-robot evaluation and spares much of its computational cost. We also found that distinct manipulation capabilities emerge at their own compute thresholds, in a natural order from end-effector control, to manipulating a grasped object, to reasoning about static scene geometry, and finally to the dynamics of objects in the scene. On a real robot, we showed that the policy induced by world-model planning improves in success rate and in qualitative behavior as the world model scales. We also built a diffusion decoder to visualize the predicted action-conditioned future in pixel space, and used it to show that qualitatively better robotic predictions emerge with scale. We release all model checkpoints together with our training and robot-deployment code. Our study also has limitations that mark natural next steps. We scaled the predictor while keeping the V-JEPA 2.1 encoder frozen, so our laws describe a dynamics model on top of a fixed representation rather than the encoder and predictor jointly. Because we train for multiple epochs over a fixed corpus, our fits already sit close to data saturation, so pushing the frontier further will likely require more and more diverse interaction data rather than parameters alone. RoboJEPA also makes no use of text, planning instead toward a single image goal, and it makes no use of a learned policy proposal during planning, sampling actions from a uniform distribution and prioritizing them with the world model. References 1X Technologies (2024) 1X Technologies. 1X world model dataset. https://huggingface.co/datasets/1x-technologies/world_model_raw_data, 2024. AgiBot-World-Contributors et al. (2025) AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025. Ai et al. (2025) Bo Ai, Liu Dai, et al. Towards embodiment scaling laws in robot locomotion. In Conference on Robot Learning (CoRL), 2025. Ainslie et al. (2023) Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from multi-head checkpoints. In Empirical Methods in Natural Language Processing (EMNLP), 2023. Ansel et al. (2024) Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Michael Suo, Phil Tillet, Eikan Wang, Xiaodong Wang, William Wen, Shunting Zhang, Xu Zhao, Keren Zhou, Richard Zou, Ajit Mathews, Gregory Chanan, Peng Wu, and Soumith Chintala. PyTorch 2: Faster machine learning through dynamic Python bytecode transformation and graph compilation. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2024. 10.1145/3620665.3640366. Assran et al. (2023) Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. Assran et al. (2025) Mahmoud Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier, Yann LeCun, Michael Rabbat, and Nicolas Ballas. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025. Baldassarre et al. (2025) Federico Baldassarre, Marc Szafraniec, Basile Terver, Vasil Khalidov, Francisco Massa, Yann LeCun, Patrick Labatut, Maximilian Seitzer, and Piotr Bojanowski. Back to the features: DINO as a foundation for video world models. arXiv preprint arXiv:2507.19468, 2025. https://arxiv.org/abs/2507.19468. Bardes et al. (2024) Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. In Transactions on Machine Learning Research (TMLR), 2024. Bharadhwaj et al. (2024) Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, and Vikash Kumar. RoboAgent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In IEEE International Conference on Robotics and Automation (ICRA), 2024. Black et al. (2024) Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. π0 _0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. Black et al. (2025) Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. π0.5 _0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. Bordes et al. (2025) Florian Bordes, Quentin Garrido, Justine T Kao, Adina Williams, Michael Rabbat, and Emmanuel Dupoux. IntPhys 2: Benchmarking intuitive physics understanding in complex synthetic environments. arXiv preprint arXiv:2506.09849, 2025. Brohan et al. (2023) Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023. Bruce et al. (2024) Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments. In International Conference on Machine Learning (ICML), 2024. Caballero et al. (2022) Ethan Caballero, Kshitij Gupta, Irina Rish, and David Krueger. Broken neural scaling laws. arXiv preprint arXiv:2210.14891, 2022. Caballero et al. (2026) Ethan Caballero, Priyank Jaini, David Krueger, and Irina Rish. Unified neural scaling laws. arXiv preprint arXiv:2605.26248, 2026. Cadene et al. (2024) Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, Steven Palma, Pepijn Kooijmans, Michel Aractingi, Mustafa Shukor, Dana Aubakirova, Martino Russi, Francesco Capuano, Caroline Pascal, Jade Choghari, Khalil Meftah, Maxime Ellerbach, Jess Moss, and Thomas Wolf. LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch. https://github.com/huggingface/lerobot, 2024. Cai et al. (2026) Xiaowei Cai, Yunuo Cai, Bingao Chen, Jingxiao Chen, Zhi Chen, Siyuan Feng, et al. τ0 _0-VLA: a hierarchical robot foundation model with world-model-guided test-time computation. arXiv preprint arXiv:2608.16885, 2026. Chen et al. (2024a) Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 24081–24125, 2024a. 10.52202/079017-0759. https://proceedings.neurips.c/paper_files/paper/2024/hash/2aee1c4159e48407d68fe16ae8e6e49e-Abstract-Conference.html. Chen et al. (2024b) Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. PixArt-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In International Conference on Learning Representations (ICLR), 2024b. Chua et al. (2018) Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Advances in Neural Information Processing Systems (NeurIPS), 2018. Dao (2024) Tri Dao. FlashAttention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR), 2024. Dao et al. (2022) Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022. Dasari et al. (2019) Sudeep Dasari, Frederik Ebert, Stephen Tian, Suraj Nair, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh, Sergey Levine, and Chelsea Finn. RoboNet: Large-scale multi-robot learning. In Conference on Robot Learning (CoRL), 2019. de Boer et al. (2005) Pieter-Tjerk de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross-entropy method. Annals of Operations Research, 134(1):19–67, 2005. Dehghani et al. (2023) Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, and Neil Houlsby. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning (ICML), 2023. Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009. Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. Dyna Robotics (2026) Dyna Robotics. Dyna-2: A 1-million-hour scaling law for world-action models. https://w.dyna.co/dyna-2, 2026. Accessed: 2026-09-04. Ebert et al. (2022) Frederik Ebert, Yanlai Yang, Karl Schmeckpeper, Bernadette Bucher, Georgios Georgakis, Kostas Daniilidis, Chelsea Finn, and Sergey Levine. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. In Robotics: Science and Systems (RSS), 2022. Finn and Levine (2017) Chelsea Finn and Sergey Levine. Deep visual foresight for planning robot motion. In IEEE International Conference on Robotics and Automation (ICRA), 2017. Ghosh et al. (2024) Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Quan Vuong, Ted Xiao, Pannag R Sanketi, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, 2024. 10.15607/RSS.2024.X.090. https://w.roboticsproceedings.org/rss20/p090.html. Google DeepMind (2024) Google DeepMind. Genie 2: A large-scale foundation world model. https://deepmind.google/blog/genie-2-a-large-scale-foundation-world-model/, 2024. Technical report / blog post. Ha and Schmidhuber (2018) David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems (NeurIPS), 2018. Hafner et al. (2019) Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning (ICML), 2019. Hafner et al. (2020) Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations (ICLR), 2020. Hafner et al. (2021) Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering Atari with discrete world models. In International Conference on Learning Representations (ICLR), 2021. Hafner et al. (2023) Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. Hafner et al. (2025) Danijar Hafner, Wilson Yan, and Timothy Lillicrap. Training agents inside of scalable world models. arXiv preprint arXiv:2509.24527, 2025. Hansen et al. (2022) Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022. Hansen et al. (2024) Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2: Scalable, robust world models for continuous control. In International Conference on Learning Representations (ICLR), 2024. Hendrycks and Gimpel (2016) Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. Henighan et al. (2020) Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, and Sam McCandlish. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701, 2020. Ho and Salimans (2022) Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. https://arxiv.org/abs/2207.12598. Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020. Hoffmann et al. (2022) Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simonyan, Erich Elsen, Oriol Vinyals, Jack Rae, and Laurent Sifre. An empirical analysis of compute-optimal large language model training. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 30016–30030, 2022. 10.52202/068431-2176. https://proceedings.neurips.c/paper_files/paper/2022/hash/c1e2faff6f588870935f114ebe04a3e5-Abstract.html. Hoogeboom et al. (2023) Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In International Conference on Machine Learning (ICML), 2023. Hu et al. (2023) Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. GAIA-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080, 2023. Hu et al. (2024) Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404.06395, 2024. Huang et al. (2026) Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian, Ming-Yu Liu, Dieter Fox, Kaichun Mo, et al. PointWorld: Scaling 3d world models for in-the-wild robotic manipulation. arXiv preprint arXiv:2601.03782, 2026. Jang et al. (2022) Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. BC-Z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning (CoRL), 2022. Kalashnikov et al. (2018) Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine. Scalable deep reinforcement learning for vision-based robotic manipulation. In Proceedings of The 2nd Conference on Robot Learning, volume 87 of Proceedings of Machine Learning Research, pages 651–673, 2018. https://proceedings.mlr.press/v87/kalashnikov18a.html. Kalashnikov et al. (2021) Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. MT-Opt: Continuous multi-task robotic reinforcement learning at scale. arXiv preprint arXiv:2104.08212, 2021. Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. Khazatsky et al. (2024) Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, Patrick Tree Miller, Jimmy Wu, Suneel Belkhale, Shivin Dass, Huy Ha, Arhan Jain, Abraham Lee, Youngwoon Lee, Marius Memmel, Sungjae Park, Ilija Radosavovic, Kaiyuan Wang, Albert Zhan, Kevin Black, Cheng Chi, Kyle Beltran Hatch, Shan Lin, Jingpei Lu, Jean Mercat, Abdul Rehman, Pannag R Sanketi, Archit Sharma, Cody Simpson, Quan Vuong, Homer Rich Walke, Blake Wulfe, Ted Xiao, Jonathan Heewon Yang, Arefeh Yavary, Tony Z. Zhao, Christopher Agia, Rohan Baijal, Mateo Guaman Castro, Daphne Chen, Qiuyu Chen, Trinity Chung, Jaimyn Drake, Ethan Paul Foster, Jensen Gao, Vitor Guizilini, David Antonio Herrera, Minho Heo, Kyle Hsu, Jiaheng Hu, Muhammad Zubair Irshad, Donovon Jackson, Charlotte Le, Yunshuang Li, Kevin Lin, Roy Lin, Zehan Ma, Abhiram Maddukuri, Suvir Mirchandani, Daniel Morton, Tony Nguyen, Abigail O’Neill, Rosario Scalise, Derick Seale, Victor Son, Stephen Tian, Emi Tran, Andrew E. Wang, Yilin Wu, Annie Xie, Jingyun Yang, Patrick Yin, Yunchu Zhang, Osbert Bastani, Glen Berseth, Jeannette Bohg, Ken Goldberg, Abhinav Gupta, Abhishek Gupta, Dinesh Jayaraman, Joseph J Lim, Jitendra Malik, Roberto Martín-Martín, Subramanian Ramamoorthy, Dorsa Sadigh, Shuran Song, Jiajun Wu, Michael C. Yip, Yuke Zhu, Thomas Kollar, Sergey Levine, and Chelsea Finn. DROID: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024. Kim et al. (2024) Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024. LeCun (2022) Yann LeCun. A path towards autonomous machine intelligence. Open Review, 2022. https://openreview.net/forum?id=BZ5a1r-kVsf. Lin et al. (2025) Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. In International Conference on Learning Representations (ICLR), 2025. Lin et al. (2024) Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024. Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. Luo et al. (2024) Jianlan Luo, Charles Xu, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, and Sergey Levine. FMB: A functional manipulation benchmark for generalizable robotic learning. The International Journal of Robotics Research, 2024. Lynch et al. (2023) Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. In IEEE Robotics and Automation Letters (RA-L), 2023. Mayne et al. (2000) D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert. Constrained model predictive control: Stability and optimality. Automatica, 36(6):789–814, 2000. 10.1016/S0005-1098(99)00214-9. Micheli et al. (2023) Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world models. In International Conference on Learning Representations (ICLR), 2023. Mur-Labadia et al. (2026) Lorenzo Mur-Labadia, Matthew Muckley, Amir Bar, Mido Assran, Koustuv Sinha, Mike Rabbat, Yann LeCun, Nicolas Ballas, and Adrien Bardes. V-JEPA 2.1: Unlocking dense features in video self-supervised learning. arXiv preprint arXiv:2603.14482, 2026. Nagabandi et al. (2018) Anusha Nagabandi, Gregory Kahn, Ronald S. Fearing, and Sergey Levine. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. In IEEE International Conference on Robotics and Automation (ICRA), 2018. Nasiriany et al. (2024) Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. RoboCasa: Large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems (RSS), 2024. Nasiriany et al. (2026) Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. RoboCasa365: A large-scale simulation framework for training and benchmarking generalist robots. arXiv preprint arXiv:2603.04356, 2026. Nichol and Dhariwal (2021) Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (ICML), 2021. Nilaksh et al. (2026) Nilaksh, Saurav Jha, Artem Zholus, and Sarath Chandar. Reconstruction or semantics? what makes a latent space useful for robotic world models. arXiv preprint arXiv:2605.06388, 2026. https://arxiv.org/abs/2605.06388. NVIDIA (2025) NVIDIA. Cosmos world foundation model platform for physical AI. arXiv preprint arXiv:2501.03575, 2025. Open X-Embodiment Collaboration et al. (2023) Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open X-embodiment: Robotic learning datasets and RT-X models. arXiv preprint arXiv:2310.08864, 2023. PAN Team et al. (2025) PAN Team, Jiannan Xiang, Yi Gu, Zihan Liu, Zeyu Feng, Qiyue Gao, Yiyan Hu, Benhao Huang, Guangyi Liu, Yichi Yang, Kun Zhou, et al. PAN: A world model for general, interactable, and long-horizon world simulation. arXiv preprint arXiv:2511.09057, 2025. https://arxiv.org/abs/2511.09057v1. Pearce and Song (2024) Tim Pearce and Jinyeop Song. Reconciling kaplan and chinchilla scaling laws. arXiv preprint arXiv:2406.12907, 2024. Pearce et al. (2024) Tim Pearce, Tabish Rashid, Dave Bignell, Raluca Georgescu, Sam Devlin, and Katja Hofmann. Scaling laws for pre-training agents and world models. arXiv preprint arXiv:2411.04434, 2024. Peebles and Xie (2023) William Peebles and Saining Xie. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023. Pertsch et al. (2025) Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. FAST: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. Po et al. (2025) Ryan Po, Yotam Nitzan, Richard Zhang, Berlin Chen, Tri Dao, Eli Shechtman, Gordon Wetzstein, and Xun Huang. Long-context state-space video world models. arXiv preprint arXiv:2505.20171, 2025. Punzo et al. (2026) Samuele Punzo, Niccolò Caselli, Ippokratis Pantelidis, Francesco Massafra, Salvatore Lo Sardo, and Mohammadreza Salehi. How do video foundation models encode intuitive physics? probing across pretraining paradigms. arXiv preprint arXiv:2606.09646, 2026. Rubinstein (1997) Reuven Y. Rubinstein. Optimization of computer simulation models with rare events. European Journal of Operational Research, 99(1):89–112, 1997. Russell et al. (2025) Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. GAIA-2: A controllable multi-view generative world model for autonomous driving. arXiv preprint arXiv:2503.20523, 2025. Sartor and Thompson (2024) Sebastian Sartor and Neil C. Thompson. Neural scaling laws in robotics. arXiv preprint arXiv:2405.14005, 2024. Sato et al. (2023) Makoto Sato, Ryosuke Unno, Masahiro Negishi, Koudai Tabata, Taiju Watanabe, Junnosuke Kamohara, Taiga Kume, Ryo Okada, Yusuke Iwasawa, and Yutaka Matsuo. Scaling laws of model size for world models. In Proceedings of the Annual Conference of the Japanese Society for Artificial Intelligence (JSAI), 2023. 10.11517/pjsai.JSAI2023.0_2G5OS21e02. In Japanese. Seo et al. (2023) Younggyo Seo, Danijar Hafner, Hao Liu, Fangchen Liu, Stephen James, Kimin Lee, and Pieter Abbeel. Masked world models for visual control. In Conference on Robot Learning (CoRL), 2023. Song et al. (2021) Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021. Su et al. (2024) Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. Terver et al. (2026) Basile Terver, Tsung-Yen Yang, Jean Ponce, Adrien Bardes, and Yann LeCun. What drives success in physical planning with joint-embedding predictive world models? Transactions on Machine Learning Research, 2026. https://openreview.net/forum?id=cHZn5Gdh8e. Walke et al. (2023) Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, and Sergey Levine. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023. Wayve (2025) Wayve. GAIA-3: Scaling world models to power safety and evaluation. https://wayve.ai/thinking/gaia-3/, 2025. Accessed: 2026-09-04. Wayve (2026) Wayve. GAIA-4: Multimodal world models powering closed-loop simulation for safe and scalable autonomy. https://wayve.ai/thinking/gaia-4/, 2026. Accessed: 2026-09-04. Williams et al. (2017) Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M. Rehg, Byron Boots, and Evangelos A. Theodorou. Information theoretic MPC for model-based reinforcement learning. In IEEE International Conference on Robotics and Automation (ICRA), 2017. 10.1109/ICRA.2017.7989202. Wu et al. (2024a) Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. iVideoGPT: Interactive VideoGPTs are scalable world models. In Advances in Neural Information Processing Systems (NeurIPS), 2024a. Wu et al. (2024b) Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, Zhen Zhao, Guangyu Li, Zhao Jin, Lecheng Wang, Jilei Mao, Xinhua Wang, Shichao Fan, Ning Liu, Pei Ren, Qiang Zhang, Yaoxu Lyu, Mengzhen Liu, Jingyang He, Yulin Luo, Zeyu Gao, Chenxuan Li, Chenyang Gu, Yankai Fu, Di Wu, Xingyu Wang, Sixiang Chen, Zhenyu Wang, Pengju An, Siyuan Qian, Shanghang Zhang, and Jian Tang. RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation. arXiv preprint arXiv:2412.13877, 2024b. https://arxiv.org/abs/2412.13877v1. Wu et al. (2023) Philipp Wu, Alejandro Escontrela, Danijar Hafner, Ken Goldberg, and Pieter Abbeel. DayDreamer: World models for physical robot learning. In Conference on Robot Learning (CoRL), 2023. Yang et al. (2024) Sherry Yang, Yilun Du, Seyed Ghasemipour, Jonathan Tompson, Leslie Kaelbling, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. In International Conference on Learning Representations (ICLR), pages 45210–45234, 2024. https://proceedings.iclr.c/paper_files/paper/2024/hash/c4d66eae503694424123b93ac0fbaf17-Abstract-Conference.html. Zhai et al. (2022) Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. Zhang and Sennrich (2019) Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems (NeurIPS), 2019. Zhang et al. (2026) Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, et al. Hierarchical planning with latent world models. arXiv preprint arXiv:2604.03208, 2026. Zhou et al. (2025) Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. DINO-WM: World models on pre-trained visual features enable zero-shot planning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 79115–79135. PMLR, 2025. https://proceedings.mlr.press/v267/zhou25t.html. Zhu et al. (2025) Fangqi Zhu, Hongtao Wu, Song Guo, Yuxiao Liu, Chilam Cheang, and Tao Kong. IRASim: A fine-grained world model for robot manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025. Zitkovich et al. (2023) Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R. Sanketi, Grecia Salazar, Michael S. Ryoo, Krista Reymann, Kanishka Rao, Karl Pertsch, Igor Mordatch, Henryk Michalewski, Yao Lu, Sergey Levine, Lisa Lee, Tsang-Wei Edward Lee, Isabel Leal, Yuheng Kuang, Dmitry Kalashnikov, Ryan Julian, Nikhil J. Joshi, Alex Irpan, Brian Ichter, Jasmine Hsu, Alexander Herzog, Karol Hausman, Keerthana Gopalakrishnan, Chuyuan Fu, Pete Florence, Chelsea Finn, Kumar Avinava Dubey, Danny Driess, Tianli Ding, Krzysztof Marcin Choromanski, Xi Chen, Yevgen Chebotar, Justice Carbajal, Noah Brown, Anthony Brohan, Montserrat Gonzalez Arenas, and Kehang Han. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Jie Tan, Marc Toussaint, and Kourosh Darvish, editors, Proceedings of The 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, pages 2165–2183. PMLR, 06–09 Nov 2023. https://proceedings.mlr.press/v229/zitkovich23a.html. Appendix A Model Architecture: Additional Details Expands Section 2.1. We collect here the implementation-level details of the predictor block: the channel-pair derivation behind factorised RoPE (Section A.1); the per-block view bias under grouped-query attention (Section A.2); the per-embodiment input projection (Section A.4); the embodiment-id breakdown that yields |ℰ|=26|E|=26 (Section A.5); and the predictor scaling sweep across all eight scales used in the scaling law of Section 3 (Section A.6). All notation is shared with Section 2.1; in particular, ℓ is reserved for the predictor layer index and τ for autoregressive rollout steps. Table 4 summarises how the RoboJEPA predictor differs from the action-conditioned V-JEPA 2 predictor (V-JEPA 2-AC) (Assran et al., 2025) it builds on. The two share the same backbone family and the same next-frame prediction parameterisation; RoboJEPA changes the normalisation, position encoding, and the conditioning that supports cross-view, cross-embodiment world modeling. Table 4: Architectural comparison between the action-conditioned V-JEPA 2 predictor (V-JEPA 2-AC) (Assran et al., 2025) and the RoboJEPA predictor. Both share the V-JEPA 2 backbone and the next-frame latent-prediction parameterisation; RoboJEPA adds RMSNorm and QK-norm for training stability at scale, a fourth (view) RoPE axis, and explicit per-embodiment and per-view conditioning for cross-embodiment, multi-view world modeling. V-JEPA 2-AC (Assran et al., 2025) RoboJEPA (ours) Architecture Base ViT architecture V-JEPA 2 Training parameterisation Feed-forward next-embedding (frame) prediction with frame-causal mask; trained with an ℓ1 _1 latent loss. Normalisation LayerNorm RMSNorm QK-norm No Yes Position embedding 3-D spatio-temporal RoPE 4-D RoPE (space–time ++ views) Multi-view world modeling No, single view Yes, single or multi-view without adaptation Multi-embodiment conditioning None Learnable per-embodiment bias Multi-view conditioning None Post-RoPE per-block per-view bias Attention variant ViT self-attention GQA Action conditioning Linear action embedding as token per-embodiment multi-linear action embedding as token Attention head dim 6464 128128 Training Training loss Teacher-forced L1 + One AR step rollout L1 loss Teacher-forced L1 + K-step AR rollout loss with KV-cache (K=2K=2 during pretrain and K=10K=10 in cooldown) Learning Rate Schedule Linear warmup → cosine LR annealing Linear warmup → flat LR → LR cooldown with more per-train-step compute Number of context steps for AR rollout loss 1 Random LR warmup steps 45004500 20002000 Peak LR 4.25×10−44.25× 10^-4 5×10−55× 10^-5 Weight Decay 0.040.04 0.10.1 AdamW β1 _1 0.90.9 0.90.9 AdamW β2 _2 0.9990.999 0.950.95 Gradient clipping None 1.01.0, by global l2l2 norm Training resolution 256×256256× 256 240×320240× 320 (in pretrain) and 240×320240× 320 + 480×640480× 640 (in cooldown) Data Training dataset DROID RoboJEPA train mixture Training episodes ⩽73000 73000 2,871,3002,871,300 Training hours approximately 2626 15,02215,022 video; 6,6926,692 action Training cameras left only any (external and wrist) Training framerate 44 FPS varies per dataset A.1 Factorised multi-axis RoPE Each per-head channel space 0,…,dh−1\0,…,d_h-1\ partitions into four contiguous, equal-size sub-blocks of width dax=dh/4d_ax=d_h/4, one per spatio-temporal axis. Fix the enumeration max=t,i,j,vA_ \!=\!\t,i,j,v\ for the time, height, width, and view axes, and let jαj_α denote the starting channel of axis α, so 0,…,dh−1=⨆α∈maxoα,oα+1,…,oα+dax−1,ot=0,oi=dax,oj=2dax,ov=3dax. gathered\0,…,d_h-1\\;=\; _α _ \o_α,\,o_α+1,\,…,\,o_α+d_ax-1 \,\\ o_t=0,\;o_i=d_ax,\;o_j=2d_ax,\;o_v=3d_ax. gathered (6) Within axis α, the sub-block is partitioned in turn into dax/2d_ax/2 channel pairs (oα+2m,oα+2m+1) (o_α+2m,\,o_α+2m+1 ) for m∈0,…,dax/2−1m\!∈\!\0,…,d_ax/2-1\. We use the channel-pair index m rather than the conventional k to avoid collision with the key tensor k. Each pair is rotated as a 2-D vector by the per-pair angle θα(m)(p)=p⋅10000−2m/dax,p=posα∈ℕ _α^(m)(p)\;=\;p· 10000^-2m/d_ax, p\;=\;pos_α\;∈\;N (7) evaluated at the integer coordinate of the token along axis α (time index for α=tα\!=\!t, row for α=iα\!=\!i, column for α=jα\!=\!j, view slot for α=vα\!=\!v); this matches the lang frequency convention used by the predictor’s RotaryEmbedding module (Su et al., 2024). Concretely, for a query channel pair (qoα+2m,qoα+2m+1) (q_o_α+2m,\,q_o_α+2m+1 ) at axis position p=posαp\!=\!pos_α, (qoα+2m′qoα+2m+1′)=(cosθα(m)(p)−sinθα(m)(p)sinθα(m)(p)cosθα(m)(p))(qoα+2mqoα+2m+1), pmatrixq _o_α+2m\\ q _o_α+2m+1 pmatrix\;=\; pmatrix _α^(m)(p)&- _α^(m)(p)\\ _α^(m)(p)&\;\, _α^(m)(p) pmatrix pmatrixq_o_α+2m\\ q_o_α+2m+1 pmatrix, (8) and identically for the key channels. Pairs in different axis sub-blocks rotate independently and at independent frequencies, so the inner product ⟨q′,k′⟩ q ,k telescopes into a sum of per-axis, per-pair terms whose differences depend only on relative positions Δposα=posα−posα′ \!pos_α=pos_α-pos _α. This relative-position property lets the predictor generalize across V∈1,2V\!∈\!\1,2\ views and across the training and deployment patch grids of Section 2.2 without re-learning positional codes. Under multi-view training (Vmax=2V_ =2) we use the full 4-axis partition =t,i,j,vA\!=\!\t,i,j,v\: the v-axis sub-block contributes its own rotation, and viewpoint identity is additionally supplied by the per-block additive bias of Section A.2 (the two are complementary). For purely single-view runs (V=1V=1) the v-axis sub-block carries no rotation and is left as the identity. Action and state tokens are detached from the visual slab before the RoPE rotation is applied: only the time component of RoPE rotates them (indexed by their step id), so that a single proprioceptive token per step does not spuriously inherit a spatial position. A.2 Per-block view bias under grouped-query attention The view information is encoded into the model in two ways. First, the view position encoding disambiguates different views of tokens with the same spatio-temporal position. This is achieved by rotating the view-axis sub-block (dh/4d_h/4 channels per head). However, the view position only informs the model that two different views are distinct without any encoding of the type of the content (e.g. the model has to know which view represents the side camera looking from the left and which one is mounted on the wrist). One way of dealing with it is to just let the model learn this end-to-end; however, we decided to directly inject the view information as a learnable bias added across the full head width of visual-token queries and keys after the four-axis RoPE rotation: qℓ(v)←qℓ(v)+Uℓ[c(v)],kℓ(v)←kℓ(v)+Uℓ[c(v)],q_ ^(v)\,←\,q_ ^(v)+U_ \! [c^(v) ], k_ ^(v)\,←\,k_ ^(v)+U_ \! [c^(v) ], (9) with Uℓ∈ℝ||×dhU_ \!∈\!R^|V|× d_h a learnable view embedding (zero-initialised), broadcast across all query and KV heads. This bias supplements, rather than replaces, the view-axis rotation. A learned global embodiment vector Eemb[e]E_emb[e] and a per-view embedding Eview[c(v)]E_view[c^(v)] are added once at the predictor input. The strong-view conditioning of Equation 9 is a zero-initialised, layer-wise additive bias indexed by the semantic camera id c(v)∈c^(v)\!∈\!V (left, right, wrist, front, back, head, top), implemented as Uℓ∈ℝ||×dhU_ \!∈\!R^|V|× d_h for each ℓ∈1,…,Nlayers \!∈\!\1,…,N_layers\ and added to all dhd_h channels of every visual token’s queries and keys after the RoPE rotation of Equation 8. Concretely, let Qℓ∈ℝN×hq×dhQ_ \!∈\!R^N× h_q× d_h and Kℓ∈ℝN×hkv×dhK_ \!∈\!R^N× h_kv× d_h be the post-RoPE query and key tensors of layer ℓ for a per-step token bag of length N, and let bℓ(v):=Uℓ[c(v)]∈ℝdhb_ ^(v)\!:=\!U_ [c^(v)]\!∈\!R^d_h be the bias gathered for view v. Under grouped-query attention (Ainslie et al., 2023) with hqh_q query heads and hkvh_kv KV heads sharing the same head dimension dhd_h, the bias is added by the broadcast rule Qℓ[n,r,:]+=bℓ(v(n))∀r∈1,…,hq,Kℓ[n,r,:]+=bℓ(v(n))∀r∈1,…,hkv,Q_ [n,r,:]\; +=\;b_ ^(v(n)) ∀\,r\!∈\!\1,…,h_q\, K_ [n,r,:]\; +=\;b_ ^(v(n)) ∀\,r\!∈\!\1,…,h_kv\, (10) for every visual token index n with view assignment v(n)v(n); that is, the same dhd_h-vector bℓ(v)b_ ^(v) is broadcast across all query heads and all KV heads of layer ℓ , so the table UℓU_ has no head dimension and the parameter count is ||dh|V|\,d_h per layer (for the 1B model: 7⋅=8967· 128\!=\!896 parameters per layer, 40⋅=35,84040· 896\!=\!35,840 in total). The bias is not added to action or state tokens, since these have no view assignment; their post-RoPE queries and keys propagate through the attention block unchanged. This per-layer additive bias supplements the v-axis RoPE rotation of Equation 6: under our default strong-view configuration both signals are active, the rotation contributes a relative-view phase to attention scores, and the bias contributes a dhd_h-wide per-layer per-camera offset on top of it. The combination gives a stable handle on viewpoint identity for multi-view planning (Section 2.2); zero initialisation of UℓU_ makes loading a checkpoint trained without strong view conditioning a no-op at step 00. A.3 Attention masking We use a similar attention mask to V-JEPA 2-AC (Assran et al., 2025) but extend it to multiple views: attention is timestep-level causal and restricted to a local temporal window of 88 timesteps, while within a step all VHpWpVH_pW_p visual tokens (plus the action and state tokens) attend omnidirectionally (any-to-any). More precisely, the rasterized sequence of tokens has T×NtokT× N_tok tokens in total and the token at index k can attend to tokens with indices j such that ⌊kNtok⌋−7⩽⌊jNtok⌋⩽⌊kNtok⌋ kN_tok -7 jN_tok kN_tok . Schematically, this means that packing the sequence as T frame-blocks of size NtokN_tok given by Equation 20, the attention mask between blocks is banded lower-triangular: each block of size Ntok×NtokN_tok×N_tok is fully attended within the diagonal band of width WtW_t and zero outside it. This trick makes the computational cost of attention grow linearly as a function of the step number and it alleviates the problem of the position extrapolation – the predicted content quality depends only on the accumulated error, not the ability of the model to extrapolate, because it never observes a sequence longer than the window size. This is a classical trick for language models and world models (Assran et al., 2025; Po et al., 2025; PAN Team et al., 2025). The bounded window keeps the per-step computational cost of attention constant and makes the predictor KV-cacheable in (Wt⋅V⋅Hp⋅Wp)O(W_t· V· H_p· W_p) keys per layer; Section 2.2 uses this at deployment. A.4 Per-embodiment input projection Every embodiment e owns its own input weight matrix Wa(e)W_a^(e) in a single multi-embodiment linear projection. The raw action vector at∈ℝdaa_t\!∈\!R^d_a is zero-padded to a common width damaxd_a jointly large enough to absorb the widest action convention in our mixture (Section 2.2; for the 1B run damax=dsmax=64d_a =d_s =64, set by the chunked DROID joint_vel action-key variant which produces an ⋅=648\!·\!8\!=\!64-D mega-action) before projection, a~t=[at; 0damax−da]∈ℝdamax,ActTok(at,e)=(Wa(e))⊤a~t+ba(e)+PEta, a_t\;=\; [\,a_t\,;\, 0_d_a -d_a\, ]\;∈\;R^d_a , (a_t,e)\;=\;(W_a^(e))^\! \, a_t\,+\,b_a^(e)\,+\,PE_t^a, (11) with Wa(e)∈ℝdamax×DW_a^(e)\!∈\!R^d_a × D, and an analogous projection StTok(st,e)StTok(s_t,e) for state; PEta,sPE_t^\a,s\ are 1-D sin–cos positional embeddings over time. This lets a single predictor consume the heterogeneous action / state conventions of Section 2.2 (chunked DROID controller commands, multi-D AgiBot World dual-arm states, planar Language-Table poses, etc.) without per-embodiment branching at the transformer level: the embodiment identity affects only the input projection. The action token participates in self-attention as a regular token: the projected vector ActTok(at,e)ActTok(a_t,e) is concatenated to the per-step token bag and attends to (and is attended by) all visual and state tokens of the same step, and all visual / action / state tokens within the local temporal window WtW_t. The state token participates symmetrically. Token-style conditioning, in contrast to additive action embedding, lets the predictor express step-dependent gating of action effects (e.g., the gripper acting differently early vs. late in a grasp) without inflating the per-step input dimensionality. A.5 Robot-id breakdown The training mixture spans 1212 physical robotic platforms (the Total row of Table 1). We treat each (platform, action-source) pair as a distinct robot id, which yields |ℰ|=26|E|=26 ids, split into 1212 default state-delta ids (one per platform) plus 1414 extra splits for those datasets that ship more than one synchronized controller stream or distinct state convention not collapsed by the action-key pipeline of Section B.7. Table 5 lists the visible breakdown (#\# / Platform / Notes); the internal embodiment_name strings used by the training config are collected in the table footnote. Table 5: Robot-id breakdown used in this work. The first 1212 rows are default state-delta ids, one per physical platform. The next 1414 rows are extra (platform, action-source) or (platform, state-convention) splits. Total robot ids: |ℰ|=+=26|E|\!=\!12\!+\!14\!=\!26. # Platform Notes 1212 default state-delta ids (one per physical platform) 1 Franka DROID, RoboMind-Franka, Roboset, OXE-FMB (default) 2 Franka + Omron RoboCasa (action-key) 3 SO-101 LeRobot 4 AgiBot dual-arm (Robotiq) AgiBot World 5 AgiBot humanoid AgiBot World 6 AgileX (dual-arm) RoboMind AgileX 7 WidowX (Bridge) OXE Bridge, OXE Bridge V2 8 Google robot OXE RT-1, OXE BC-Z 9 Kuka OXE QT-Opt, OXE MT-Opt 10 Planar arm OXE Language-Table (sim, real) 11 1X humanoid 1X 12 Baxter OXE RoboNet (Baxter) 1414 extra (platform, action-source / state-convention) splits 13 Franka DROID, joint-state convention 14 Franka DROID, cartesian controller actions 15 Franka DROID, joint controller actions 16 Franka DROID, cartesian-velocity controller actions 17 Franka DROID, joint-velocity controller actions 18 SO-101 LeRobot, joint-state convention 19 Franka + Omron RoboCasa, state-delta convention 20 Franka RoboMind-Franka, separate id from generic Franka 21 Franka Roboset, separate id from generic Franka 22 Franka OXE RoboNet (Franka split) 23 Franka OXE FMB, separate id from generic Franka 24 Sawyer OXE RoboNet (Sawyer) 25 WidowX OXE RoboNet (WidowX) 26 Kuka OXE RoboNet (Kuka) A.6 Predictor scaling sweep For the scaling study of Section 3, we train 2222M, 5050M, 100100M, 300300M, 11B, 22B, 44B, and 88B variants with the same block recipe (pre-norm GQA + GELU MLP, factorised 4-axis RoPE on (t,i,j,v)(t,i,j,v) supplemented by the per-block view bias of Equation 9, encoder, padded action / state widths, local temporal window, optimizer schedule). The eight variants differ only in NlayersN_layers, D, and the GQA head counts (hq,hkv)(h_q,\,h_kv) (Table 6); the head dimension dh=128d_h=128 and the MLP hidden ratio 4D4D are held constant. The KV head count matches the query head count at the smallest scales and approaches hkv=hq/2h_kv=h_q/2 as model size increases. Table 6: Hyperparameters of the RoboJEPA predictor at eight scales (zero-init view bias). Only depth NlayersN_layers, width D, and head counts (hq,hkv)(h_q,h_kv) change across scales; everything else (encoder, factorised RoPE on (t,i,j,v)(t,i,j,v), post-RoPE full-head-width view bias, padded action / state widths, local temporal window WtW_t, attention kernel, output projection, training loss) is held fixed. Predictor parameter counts are approximate D2NlayersD^2N_layers-dominated estimates. Hyperparameter 22M 50M 100M 300M 1B 2B 4B 8B Notation Predictor depth 77 1010 1414 2424 4040 4040 5252 7272 NlayersN_layers Predictor width 512512 640640 768768 10241024 15361536 20482048 25602560 30723072 D Query heads 44 55 66 88 1212 1616 2020 2424 hqh_q KV heads (GQA) 44 55 66 88 44 88 1010 1212 hkvh_kv Head dim 128128 128128 128128 128128 128128 128128 128128 128128 dhd_h MLP hidden ratio 44 44 44 44 44 44 44 44 rffnr_ffn Predictor parameters 0.020.02 B 0.050.05 B 0.10.1 B 0.30.3 B 1.01.0 B 2.42.4 B 4.44.4 B 8.48.4 B — Shared across all scales Encoder (frozen) V-JEPA 2.1 ViT-G (Mur-Labadia et al., 2026) f Encoder embed dim 16641664 DeD_e Encoder patch / tubelet 16×1616×16 / 11 — Frames per clip 1616 T Spatial grid (per view) 15×2015×20 at 240×320240×320 Hp×WpH_p×W_p Views per clip 11 or 22 V Action / state width (padded) 6464 damaxd_a , dsmaxd_s Action / state tokens per step 11 / 11 nan_a, nsn_s Embodiment / camera ids 2626 / 77 |ℰ||E|, |||V| Position encoding factorised 4-axis RoPE on (t,i,j,v)(t,i,j,v) (Su et al., 2024) — Per-block view conditioning additive Uℓ[c(v)]U_ [c^(v)] on q,kq,k UℓU_ Local temporal window 88 frames (causal); spatial Hp×WpH_p\!×\!W_p and view V unrestricted WtW_t Output projection Linear(D→De)+LayerNormLinear(D→D_e)+LayerNorm WoutW_out Loss per-token ℓ1 _1 on encoder features (Equation 5) ℒTFL_TF Rollout steps (training) 22 KarK_ar Appendix B Data: Additional Details Expands Section 2.2. Heterogeneous robot data become a single token stream by being normalized onto a common temporal grid (Section B.6), a common padded action / state width (Section B.7), and a common semantic camera vocabulary (Section B.9). The on-disk format (Section B.3) and the non-trivial per-dataset conversions needed to reach it (Section B.4) make this routing reproducible, the view-mixed-rank mixture (Section B.5) fixes the sampling, and the full per-dataset breakdown lives in Section B.2. B.1 Dataset mixture and unification Actions. For most datasets, the per-step action is the difference between consecutive states (which we refer to as the state-delta action): whether it is an end-effector (EEF) absolute pose, absolute joint angles, a commanded motion, or any combination of those for multiarm or wheeled robots. The same demonstration can often be described through several such parameterisations at once (for example joint angle control or EEF pose control), and each one shows the world model the same behavior from a different control perspective, acting as an action-level augmentation. We exploit this: we found it beneficial to train on different action parameterisations of the same embodiment whenever they are available. For example, with the DROID dataset arm we cannot only train on state-delta of the end-effector but also on the velocities of the end-effector or joint angles. Early in the work, we found that this improves the world model at a fixed training compute budget. In addition to that, different embodiments provide a much richer potential learning signal for world models since they expose manipulation through different action spaces, each following its own unique dynamics and a set of constraints. Therefore, the world model has more potential to learn the underlying physics and dynamics of the world than with one embodiment. We treat each (platform, action-space) pair as a distinct embodiment id, giving |ℰ|=26|E|=26 over the 1212 physical platforms (26=1226=12 default state-delta ids +14+14 extra splits; full enumeration in Table 5). We use a lean per-embodiment linear projection of actions to facilitate learning a shared latent action tokenization within the world model. The dimensionality of the actions also varies. For the single-arm end-effector delta poses, we use seven dimensions that represent a 66D translation+rotation and gripper closure. For dual arm platforms (e.g. AgileX in RoboMind or AgiBot World), this gets doubled since we have to control two arms independently. For wheeled platforms, the action space is extended by the commanded actions for the wheeled navigation - for example the RoboCasa platform reserves four actions for navigation atop the standard 77D EEF control and 1X reserves 22D for navigation (see more details in Tables 7 and B.7). Unification and sampling. To facilitate efficient training, we convert all datasets into the unified data format. Since datasets are collected with different sensor frequencies, loading continuous chunks of data would often lead to an overly short horizon for prediction. Additionally, different datasets have different motion velocities due to differences in data collection mechnisms – often different teleoperated platforms scale human motion to a faster or slower motion of the robot. Therefore, we employ different frame sampling frequencies for different datasets which we report in Table 7 (see Section B.6 for details). Each frame is processed via the image embedding of the V-JEPA 2.1 recipe (Mur-Labadia et al., 2026), whose features performed strongly in policy-based evaluations of latent diffusion world models (Nilaksh et al., 2026) on Bridge V2 (Walke et al., 2023). To correctly represent the total action between sampled frames, we either use the total measured state delta action or the concatenated chunk of commanded actions between sampled frames when working with commanded actions. The world model does not have a well-defined notion of modeling frequency. Instead, we simply let the world model integrate the total motion represented by the action into the next predicted latent state. B.2 Per-dataset breakdown Table 1 collapses the OXE, AgiBot World, and RoboMind sub-corpora into single rows. Table 7 gives the full per-dataset breakdown with resolutions and source FPS. Table 7: Full per-dataset breakdown of the 2323-dataset, 2626-embodiment mixture. “Act. hrs” counts only frames with synchronized proprioception. The compressed main-text view is Table 1; per-rank sampling weights are in Table 8. Dataset Traj. Vid. hrs Act. hrs Resolution FPS Embodiment DROID (Khazatsky et al., 2024) 92k 414 138 720×1280720×1280 60 Franka Roboset (Bharadhwaj et al., 2024) 73k 1244 380 240×424240×424 5 Franka (Roboset) RoboMind Franka (Wu et al., 2024b) 25k 92 37 480×640480×640 30 Franka RoboMind AgileX (Wu et al., 2024b) 10.3k 184 61 480×640480×640 30 AgileX LeRobot (Cadene et al., 2024) 21.5k 182 93 480×640480×640 30 SO-101 RoboCasa365 (Nasiriany et al., 2026) 32k 482 482 720×1280720×1280 20 Franka + Omron (sim) 1X (1X Technologies, 2024) 23.5k 90 90 512×512512×512 30 1X humanoid AgiBot World Robotiq (AgiBot-World-Contributors et al., 2025) 160k 7951 2649 480×640480×640 30 AgiBot dual-arm AgiBot World Humanoid (AgiBot-World-Contributors et al., 2025) 7.5k 85 85 480×640480×640 30 AgiBot humanoid OXE RT-1 (Brohan et al., 2023; Open X-Embodiment Collaboration et al., 2023) 87k 105 105 256×320256×320 10 Google robot OXE Bridge (Ebert et al., 2022; Open X-Embodiment Collaboration et al., 2023) 25k 48 48 480×640480×640 5 WidowX OXE Bridge V2 (Walke et al., 2023; Open X-Embodiment Collaboration et al., 2023) 60k 244 121 256×256256×256 5 WidowX OXE QT-Opt (Kalashnikov et al., 2018; Open X-Embodiment Collaboration et al., 2023) 580k 238 238 512×640512×640 10 Kuka OXE MT-Opt (Kalashnikov et al., 2021; Open X-Embodiment Collaboration et al., 2023) 920k 654 654 512×640512×640 5 Kuka OXE RoboNet Franka (Dasari et al., 2019; Open X-Embodiment Collaboration et al., 2023) 7.8k 196 63 240×320240×320 1 Franka OXE RoboNet Baxter (Dasari et al., 2019; Open X-Embodiment Collaboration et al., 2023) 18k 451 145 240×320240×320 1 Baxter OXE RoboNet Sawyer (Dasari et al., 2019; Open X-Embodiment Collaboration et al., 2023) 50k 1254 404 240×320240×320 1 Sawyer OXE RoboNet WidowX (Dasari et al., 2019; Open X-Embodiment Collaboration et al., 2023) 5k 126 40 240×320240×320 1 WidowX OXE RoboNet Kuka (Dasari et al., 2019; Open X-Embodiment Collaboration et al., 2023) 1.6k 40 12 240×320240×320 1 Kuka OXE Language-Table sim (Lynch et al., 2023; Open X-Embodiment Collaboration et al., 2023) 180k 259 259 360×640360×640 5 Planar arm OXE Language-Table real (Lynch et al., 2023; Open X-Embodiment Collaboration et al., 2023) 440k 392 392 360×640360×640 5 Planar arm OXE BC-Z (Jang et al., 2022; Open X-Embodiment Collaboration et al., 2023) 43.5k 165 165 171×213171×213 10 Google robot OXE FMB (Luo et al., 2024; Open X-Embodiment Collaboration et al., 2023) 8.6k 126 31 171×213171×213 10 Franka Total 2.8713M 15,022 6,692 — — 12 platforms B.3 Standardised on-disk format Every source dataset is re-packaged into a single object-storage layout so that the dataloader does not need per-source branching beyond a thin adapter that emits the standard tuple consumed by the predictor. Each trajectory exposes three groups: an MP4-encoded camera stream per view (cameras/), HDF5 datasets holding the native proprioceptive vector (trajectory/observation/robot_state/), and, when the source ships a controller stream, raw controller commands (trajectory/action/). Episode-level metadata (e.g. DROID building/labs fields used by the negative filter that drops setup-room recordings) is recorded in a per-dataset aggregated_metadata.json; a per-dataset index maps episode ids to S3 paths, episode lengths, and per-camera availability so the sampler operates on this index without touching object storage. Per-clip sampling reads T=16T=16 frames at the world-model rate (Section B.6) and V cameras (Section B.9); intrinsics, extrinsics, and uncalibrated metadata are loaded as NaN-filled placeholders and discarded by the predictor in this configuration. B.4 Per-dataset postprocessing Beyond the common repackaging of Section B.3, several datasets require non-trivial numerical, geometric, or video conversions before they can enter the shared token stream. We describe these here grouped by the kind of transformation; the trivial relabelling of fields onto the common schema is omitted. Video decoding and re-encoding. Every source is emitted as one MP4 per view at its native frame rate (Table 7). Most OXE (Open X-Embodiment Collaboration et al., 2023) sources ship RLDS/TFDS records with per-step RGB arrays, which we re-encode to MP4 (e.g. 11 fps for RoboNet (Dasari et al., 2019), 55 fps for Bridge (Ebert et al., 2022)/MT-Opt (Kalashnikov et al., 2021)/Language-Table (Lynch et al., 2023), 1010 fps for RT-1 (Brohan et al., 2023)/QT-Opt (Kalashnikov et al., 2018)/BC-Z (Jang et al., 2022)/FMB (Luo et al., 2024)). RoboMind (Wu et al., 2024b) stores each frame as a JPEG byte-string inside HDF5: we decode every frame individually, falling back to a raw-buffer reshape (720×1280720×1280 or 480×640480×640) when the JPEG header is missing, and correct the channel order from BGR to RGB for the real Franka streams (the simulated streams are already RGB). For DROID (Khazatsky et al., 2024) we discard the stereo/SVO recordings and keep the rectified monocular left, right, and wrist MP4 streams. For sources with multiple nominal cameras we detect and drop all-zero (absent) camera streams so they are not written as empty views (e.g. the unused auxiliary cameras in Bridge V2 (Walke et al., 2023)). Forward kinematics for joint-only sources. Roboset (Bharadhwaj et al., 2024) and LeRobot (Cadene et al., 2024) record only joint encoders, with no end-effector pose. We reconstruct the end-effector Cartesian pose with a differentiable forward-kinematics model of the corresponding arm (a Franka Panda for Roboset, the SO-101 URDF for LeRobot), applied to both the state trajectory and, for Roboset, the commanded-joint trajectory to recover a Cartesian action. LeRobot joint encoders are first converted from degrees to radians. For Roboset we additionally apply constant per-joint offsets (a +π/2+π/2 shift on the wrist joints of both state and action, and a −π/2-π/2 shift on one action joint) so that the joint zero convention matches the URDF used by the kinematics model. Orientation representation. Sources that store the end-effector orientation as a quaternion (RT-1, QT-Opt, MT-Opt, FMB, the simulated RoboMind Franka split) and the forward-kinematics outputs above are all converted to an xyzxyz Euler parameterisation, so that the entire mixture shares a single 33-D translation ++ 33-D Euler orientation convention for end-effector poses. Gripper normalisation. Gripper signals are placed on a common [0,1][0,1] closedness scale. Roboset state grippers are divided by their observed open width (0.83050.8305) and clipped to [0,1][0,1], and its command grippers are affinely mapped from [−1,1][-1,1] to [0,1][0,1]. WidowX sources (Bridge and Bridge V2) invert and clip the raw gripper signal, g←1−clip(g,0,1)g← 1-clip(g,0,1), so that larger values consistently denote a more closed gripper. Multi-arm and humanoid assembly. Bimanual and humanoid embodiments are assembled into a single per-step vector. For the RoboMind AgileX dual-arm platform we concatenate the left and right end-effector poses and the left and right arm joints, taking each arm’s last joint as its gripper. Bridge (v1) actions, which are stored as separate translation and rotation-delta fields, are concatenated into a single delta-pose action vector, and BC-Z end-effector poses are assembled from separately stored translation and axis-angle observations. B.5 View-mixed-rank training: mixture composition The 1B run uses Wddp=256W_ddp=256 data-parallel ranks (32 nodes × 8 H100s), partitioned into two equally sized view-groups: • Two-view group (ranks 0…1270…127): each rank loads Vg=2V_g=2 camera views per clip with Bg=2B_g=2 clips per microbatch and gradient accumulation Kg=1K_g=1, contributing 44 view-clips per rank per step, i.e. 512512 view-clips group-wide. • One-view group (ranks 128…255128…255): each rank loads Vg=1V_g=1 view per clip with Bg=2B_g=2, Kg=1K_g=1, contributing 22 view-clips per rank per step, i.e. 256256 view-clips group-wide. This yields the effective per-step batch of 768768 view-clips (512512 clips) derived in Section C.1 (Equation 13). The two groups share predictor parameters and step the optimizer in lockstep. Table 8 lists the per-(dataset, action-source) sampling weights for both groups in a single table, together with the per-row total. The top block lists datasets that ship two reliably informative external cameras and feed both groups; the lower block lists single-camera sources that feed only the 1-view group. For datasets appearing in both groups, the 1-view rank uses a contracted camera regex selected so that the rank still sees a representative external view (e.g. left/right exterior for DROID excludes the wrist; AgiBot World Robotiq is restricted to its head-mounted RGB stream). Table 8: Sampling weights for the 1-view and 2-view groups of the 1B run. The top block lists datasets that ship two reliably informative external cameras and feed both groups; the lower block lists single-camera sources that feed only the 1-view group. DROID and RoboCasa appear under multiple (platform, action-source) pairs — each one is a distinct embodiment id with its own per-embodiment input projection (Section B.7) — but all variants share the same underlying source episodes. Dataset (action-source) 1-view weight 2-view weight Total Marginal dataset weight DROID (state-delta cartesian) 0.02330.0233 0.03060.0306 0.02820.0282 0.1692 DROID (state-delta joints) 0.02330.0233 0.03060.0306 0.02820.0282 DROID (action-key cartesian) 0.02330.0233 0.03060.0306 0.02820.0282 DROID (action-key joints) 0.02330.0233 0.03060.0306 0.02820.0282 DROID (action-key cartesian-vel) 0.02330.0233 0.03060.0306 0.02820.0282 DROID (action-key joint-vel) 0.02330.0233 0.03060.0306 0.02820.0282 LeRobot (cartesian) 0.01030.0103 0.01350.0135 0.01240.0124 0.0248 LeRobot (joints) 0.01030.0103 0.01350.0135 0.01240.0124 AgiBot World Robotiq 0.44780.4478 0.58780.5878 0.54110.5411 RoboMind AgileX 0.01040.0104 0.01360.0136 0.01250.0125 RoboCasa (action-key) 0.02710.0271 0.03560.0356 0.03280.0328 0.0656 RoboCasa (state-delta) 0.02710.0271 0.03560.0356 0.03280.0328 Roboset 0.07010.0701 0.09200.0920 0.08470.0847 OXE Bridge V2 0.01370.0137 0.01800.0180 0.01660.0166 RoboMind Franka 0.00520.0052 0.00680.0068 0.00630.0063 OXE RT-1 0.00590.0059 — 0.00200.0020 OXE Bridge 0.00270.0027 — 0.00090.0009 OXE QT-Opt 0.01340.0134 — 0.00450.0045 OXE MT-Opt 0.03680.0368 — 0.01230.0123 OXE RoboNet Franka 0.01100.0110 — 0.00370.0037 OXE RoboNet Baxter 0.02540.0254 — 0.00850.0085 OXE RoboNet Sawyer 0.07060.0706 — 0.02350.0235 OXE RoboNet WidowX 0.00710.0071 — 0.00240.0024 OXE RoboNet Kuka 0.00230.0023 — 0.00080.0008 OXE Language-Table sim 0.01460.0146 — 0.00490.0049 OXE Language-Table real 0.02210.0221 — 0.00740.0074 OXE BC-Z 0.00930.0093 — 0.00310.0031 OXE FMB 0.00710.0071 — 0.00240.0024 1X 0.00510.0051 — 0.00170.0017 AgiBot World Humanoid 0.00480.0048 — 0.00160.0016 Group total 1.0000 1.0000 — Effective per-step mixture. At the clip level the split is 50/5050/50; at the token-with-loss level the run sees 23 23 of the signal at V=2V=2 (the V=2V=2 clips contribute two view-clips each). Both view counts therefore appear in every optimizer step, so the per-block view conditioning (Equation 9) and the factorised RoPE of Section 2.1 are jointly exposed to both regimes rather than separated by curriculum. B.6 Frame rate, decimation and the per-frame mask Every dataset is normalized to a target world-model rate fwmf_wm that determines the temporal stride between consecutive frames in a clip. Source FPS varies by two orders of magnitude across the mixture (Table 7): 11 Hz for OXE RoboNet, 55–1010 Hz for most OXE sources, 2020–3030 Hz for RoboCasa / RoboMind / AgiBot / LeRobot, 6060 Hz for DROID. The loader decimates by the integer factor c=⌈fraw/fwm⌉c\;=\; f_raw/f_wm and indexes frames as np.arange(start_frame, end_frame, c), so one predictor step always corresponds to a fixed 1/fwm1/f_wm of robot wall-clock regardless of source FPS. Trajectories whose effective length falls below T=16T=16 are right-padded with zero frames, and a per-frame mask mt,v= 1[frame (t,v) is a real, non-padded sample]∈0,1m_t,v\;=\;1[frame (t,v) is a real, non-padded sample]\;∈\;\0,1\ (12) is propagated to the loss — the same mt,vm_t,v that gates the masked ℓ1 _1 in Equation 5 and the rollout objective in Section 2.2. Within a single clip the mask is the same across views, mt,v≡mtm_t,v\!≡\!m_t, because clip-level padding is applied symmetrically to all V streams; we keep the per-view subscript in Equation 5 for notational consistency with the V=1V=1 vs V=2V=2 rank groups. Rate bridge fwm→fctrlf_wm\!→\!f_ctrl at deployment. At deployment the Franka controller of Section 2.2 runs at fctrl=10f_ctrl=10 Hz while the world model operates at fwm=5f_wm=5 Hz. The planner replans on every other controller tick and the 77-D end-effector delta returned by the planner is split into fctrl/fwm=2f_ctrl/f_wm=2 equal sub-steps emitted back-to-back; this keeps training and deployment aligned on the same 200200 ms predictor stride and isolates the controller from the world-model rate. B.7 Action conditioning details The mixture exposes two complementary action-vector pipelines per (dataset, action-source) pair. Each pair is a distinct embodiment id from the |ℰ|=26|E|=26 registry consumed by the predictor’s per-embodiment input projection (Section 2.1); duplicates of the same physical platform under different action sources (e.g. the six DROID embodiments listed in Table 8) share encoder inputs but learn distinct Wa(e)W_a^(e), Ws(e)W_s^(e) slices. State-delta pipeline (default). For most datasets the per-step action is computed as a finite difference of the synchronized state trajectory: Δxt∈ℝ3 x_t ^3 for the end-effector position, ΔRt∈(3) R_t∈ so(3) for orientation (computed in the Lie algebra so that the magnitude is independent of the rotation representation used by the source), and a scalar Δgt g_t for the gripper. The vector concatenates these into a per-platform native width: 77-D for the standard Franka / Bridge / Kuka end-effector, 1414-D for AgileX (two end-effectors), 2323-D for AgiBot dual-arm, 2525-D for the 1X humanoid, and so on across the mixture. Action-key pipeline (high-rate sources). For datasets that ship a high-rate controller stream — DROID (four cartesian / joint, position / velocity variants) and RoboCasa (cartesian-position) — the loader instead reads the raw controller stream at frawf_raw and concatenates the fraw/fwmf_raw/f_wm sub-step commands within one world-model step into a single per-step vector. For DROID at →7.560\!→\!7.5 Hz this yields a chunk of 88 controller commands per predictor step (up to 8×8=648×8=64 dimensions); for RoboCasa at →520\!→\!5 Hz it yields a chunk of 44 commands. This recovers the high-frequency controller signal (which the world model must match if its rollout is to drive a downstream planner — Section 2.2) without changing the fwmf_wm stride at which video features are produced. Padding to damax=64d_a =64 and per-embodiment projection. To batch heterogeneous embodiments together within a single rank’s microbatch, every per-step action vector — whether a single state-delta or an action-key chunk — is right-zero-padded to the fixed width damax=64d_a =64 used by the predictor’s per-embodiment input projection. The width 6464 is chosen jointly to absorb the widest native state-delta (the 2525-D 1X humanoid) and the largest action-key chunk emitted by the pipeline above. The per-embodiment projection Wa(e)∈ℝD×damaxW_a^(e)\!∈\!R^D× d_a is a multi-embodiment linear (a single |ℰ|×D×damax|E|\!×\!D\!×\!d_a weight tensor indexed by embodiment id) that recovers the native action space from the padded slot. The same construction is mirrored on the proprioceptive side (dsmaxd_s , Ws(e)W_s^(e)); the action and state tokens enter the predictor as na=ns=1n_a=n_s=1 tokens per step in Equation 20. At deployment the planner of Section 2.2 targets a single Franka-style 77-D end-effector delta convention, so the action dimensionality the user sees on a real robot is fixed regardless of which subset of the mixture trained the model. The deployment pipeline mirrors the training-time padding and embodiment-id lookup exactly, so the projection Wa(e)W_a^(e) used at planning time is the same one trained against the corresponding state-delta variant. B.8 Image preprocessing The video tensor returned by the loader has shape [C,V,T,Himg,Wimg][C,V,T,H_img,W_img] with C=3C=3, V∈1,2V\!∈\!\1,2\ and T=16T=16 frames at fwmf_wm. Spatial cropping follows the V-JEPA 2.1 input recipe (Mur-Labadia et al., 2026): at training time each clip is randomly resized with aspect ratio sampled from [1.333,1.777][1.333,1.777] and area scale sampled from [0.5,1.0][0.5,1.0], then center-cropped to 240×320240×320, while at evaluation we use a deterministic 1.0/1.01.0/1.0-scale crop of the same size. Random horizontal flips, auto-augmentation, motion shift and random erasing are disabled to avoid distorting end-effector geometry. Frames are normalized with the standard ImageNet statistics (Deng et al., 2009) expected by the V-JEPA 2.1 encoder (Mur-Labadia et al., 2026), and the resulting 240×320240×320 tensor is fed to the frozen encoder which produces an Hp×Wp=15×20H_p×W_p=15×20 patch grid with De=1664D_e=1664 channels per patch. B.9 Per-camera enumeration Every physical camera in the mixture is mapped at load time to one of ||=7|V|=7 semantic camera ids by a per-dataset regex map. Cameras with the same semantic role (e.g. all left-exterior cameras across DROID, Roboset, RoboMind, RoboCasa, OXE FMB) are mapped to the same id and therefore share the learned per-block view bias of Equation 9 regardless of which dataset they come from or whether the rank is V=1V=1 or V=2V=2. Table 9 reports the mapping in transposed form (semantic ids run across columns, datasets down rows). Table 9: Mapping from raw per-dataset camera names to the ||=7|V|=7 semantic camera ids consumed by the predictor’s per-block view conditioning (Equation 9). Rows are datasets; columns are semantic ids; each cell shows the raw camera name(s) routed to that id by the per-dataset regex. After regex matching, only the semantic id c∈0,…,6c\!∈\!\0,…,6\ is consumed by the predictor’s per-block view bias Uℓ[c(v)]U_ [c^(v)]; the original camera names are not seen by the model. The Default / unmapped bucket (id 44) collects sources that emit a single generic camera (LeRobot, OXE RoboNet) or a non-canonical camera label (OXE Bridge V2 camera1,2) for which no semantic role is reliably consistent across the corpus. Semantic id c 00 11 22 33 44 55 66 Role Left exterior Right exterior Wrist / eye-in-hand Head / front-facing Default / unmapped Top-down Planar-arm overhead DROID left right wrist — — — — Roboset left right wrist — — top — RoboMind camera_left camera_right — camera_front — camera_top — RoboMind AgileX — — camera_left,right_wrist camera_front — — — RoboCasa robot0_agentview_left robot0_agentview_right robot0_eye_in_hand — — — — OXE FMB left right — — — — — OXE Bridge — right — — — — — OXE Bridge V2 — right_back — — camera1,2 — — OXE RT-1 — — — back — — — OXE BC-Z — — — back — — — OXE RoboNet — — — — .* — — OXE Language-Table — — — — — — front LeRobot — — — — .* — — AgiBot World — — hand_left,right_color head_color — — — 1X — — — front — — — Appendix C Training: Additional Details Expands Section 2.2. We document the per-rank batch decomposition across view-groups (Section C.1). The loss is defined once in Equation 5. We use data parallelism for models up to 22B parameters and FSDP2 for the 44B and 88B models during pretraining and short-horizon prediction training. For long-horizon prediction finetuning, we instead use data parallelism combined with tensor parallelism (tp=2tp=2) for the 44B and 88B models. Sharing each sequence across two tensor-parallel GPUs allows an effective per-GPU batch size below one sequence in the most memory-intensive group (two-view, high-resolution), accommodating the K=10K=10 autoregressive rollout used during this cooldown. C.1 Per-rank batch decomposition and view groups The dataloader of Section 2.2 partitions the Wddp=256W_ddp=256 DDP ranks into Gv=2G_v=2 view-groups, indexed by g∈1,2g\!∈\!\1,2\. Group g owns Rg=128R_g=128 ranks, processes clips with VgV_g cameras (V1=1V_1=1, V2=2V_2=2), and runs KgK_g gradient-accumulation microbatches per optimizer step (K1=K2=1K_1=K_2=1 for the 1B run, but the same code path supports Kg>1K_g>1 on memory-constrained configurations). Each microbatch on a rank in group g contains BgB_g clips (B2=2B_2=2 for g=2g=2, B1=2B_1=2 for g=1g=1). The effective number of clip-views consumed per optimizer step is therefore Ncveff=∑g=1GvRg⋅Kg⋅Bg⋅Vg=⋅1+⋅2= 768,N_cv^eff\;=\; _g=1^G_vR_g· K_g· B_g· V_g\;=\;128\!·\!1\!·\!2\!·\!1\;+\;128\!·\!1\!·\!2\!·\!2\;=\;768, (13) i.e. 256256 single-view clips and 256256 two-view clips (512512 clips, 768768 clip-views) per optimizer step. Because the encoder is run per-view independently, the encoder forward sees the full 768768 clip-views; the predictor processes each clip as a single sequence of T⋅(VgHpWp+na+ns)T\!·\!(V_gH_pW_p+n_a+n_s) tokens (concretely 16⋅30216· 302 for g=1g=1 and 16⋅60216· 602 for g=2g=2 with the geometry of Table 6). C.2 Cooldown phase The schedule of Section 2.2 is warmup–stable–decay (WSD) (Hu et al., 2024): warmup linear →η0\!→\!η, then a long stable phase held flat at η=5×10−5η=5×10^-5, then a cooldown (decay) phase. We do not run cooldown inline; instead we resume the final stable-phase checkpoint as a fresh job that (i) swaps the optimizer schedulers to a linear LR cooldown paired with a constant weight-decay schedule, (i) raises the autoregressive rollout horizon from Kar=2K_ar=2 to Kar=10K_ar=10, and (i) re-partitions the ranks into four view×resolution groups. The LR decays as ηt=η(1−min(t,Ncool)/Ncool) _t=η\,(1- (t,N_cool)/N_cool) with Ncool=5000N_cool=5000 optimizer steps, reaching 00 at the end; weight decay stays at 0.10.1 throughout (the cooldown is a pure LR sweep), and the per-group μ lr_scale factors are preserved. Four view×resolution groups. During cooldown the two view-groups of Section C.1 are each split by input resolution, giving four groups that evenly partition the world (25%25\% of ranks each on the 128128-rank 1B cooldown): two-view 240×320240×320, two-view 480×640480×640, one-view 240×320240×320, and one-view 480×640480×640. The predictor and the frozen V-JEPA 2.1 encoder are unchanged: at 480×640480×640 the encoder’s 16×1616×16 patching yields a 30×4030×40 grid, i.e. HpWp=1200H_pW_p=1200 tokens per view per frame versus 300300 at 240×320240×320 (a 4×4× increase), and the predictor’s factorised RoPE (Section A.1) interpolates to the larger grid with no new parameters. High-resolution groups use a smaller per-rank batch and matched gradient accumulation to keep per-GPU memory within budget; the gradient-synchronization machinery is unchanged, now aggregating four groups instead of two in the single per-step all-reduce. C.3 Training hyperparameters Table 10 lists the loss, optimizer, and cooldown settings for the 11B predictor, complementing the model configurations in Table 6. Table 10: Additional training hyperparameters for the 1B-predictor RoboJEPA run, covering the loss, optimizer, and cooldown configuration. Settings already listed in Table 6, including the loss type and pretraining rollout horizon, are omitted here. The loss is defined in Equation 5, and AdamW follows Loshchilov and Hutter (2019). Hyperparameter Symbol Value Loss (defined once in Equation 5) Pre-loss normalization LN0LN_0 parameter-free LayerNorm on DeD_e Rollout step index τ∈0,…,Karτ\!∈\!\0,…,K_ar\ — Rollout prefix sampling τ0 _0 uniform on 0,…,T−Kar\0,…,T-K_ar\ Recurrent rollout input — detached from autograd graph Rollout / TF mixing — 12:12 12\!:\! 12 Optimiser (AdamW (Loshchilov and Hutter, 2019)) (β1,β2)( _1, _2) — (0.9, 0.95)(0.9,\,0.95) Numerical floor ε 10−810^-8 Learning rate (peak, held flat) η ×10−55\!×\!10^-5 Schedule — warmup → flat (stable) → linear cooldown (WSD (Hu et al., 2024)) Warm-up (epochs) WwarmW_warm 1010 Weight decay (constant) — 0.10.1 Decay-group scope — all ≥2≥\!2-D parameter tensors No-decay scope — biases, 11-D LayerNorm, view-bias table UℓU_ , per-embodiment input projections, RoPE buffers Gradient clipping (global ℓ2 _2) — 1.01.0 Cooldown phase (Section C.2), resumed from stable-phase checkpoint Cooldown steps NcoolN_cool 50005000 LR schedule (cooldown) — linear η→0η\!→\!0 over NcoolN_cool Weight decay (cooldown) — held constant at 0.10.1 Rollout horizon (cooldown) KarK_ar 1010 View×resolution groups — 44: 1,2\1,2\-view ×240×320, 480×640×\,\240×320,\,480×640\ High-res grid / tokens per view-frame Hp×WpH_p×W_p 30×40=120030×40=1200 (4×4× base 15×2015×20) C.4 DROID 3-view Finetuning at 720×1280 We additionally finetune a high-resolution DROID world model. It initializes from the 88B long-horizon prediction training checkpoint at 300,000300,000 pretraining steps (Section J.3), and the only things we change relative to the scaling runs are the input resolution and the number of views. Training uses DROID (Khazatsky et al., 2024) only, with three fixed views at 720×1280720×1280 resolution over 1616 timesteps. The three views are the exterior-left, exterior-right, and wrist cameras, selected in this fixed order; this is unlike pretraining, where the views are sampled in a random order. With patch size 1616, each 720×1280720×1280 view becomes a 45×80=3,60045×80=3,600-token grid, so a full sequence is about 170,000170,000 input tokens. At this resolution the model would also generate more than 10,00010,000 tokens per forward pass, which highlights the utility of a feedforward world model that predicts all next-frame tokens in one pass. We keep the same world-model objective as in pretraining but with a K=10K=10-step autoregressive rollout. We train for 2,0002,000 optimizer steps on 256256 GPUs (tensor-parallel tp=4tp=4, giving 6464 data-parallel replicas and a global batch of 6464 sequences) with AdamW (weight decay 0.10.1, gradient clipping), warming the learning rate up to 5×10−55×10^-5 over 140140 optimization steps and then decaying it with a cosine schedule over the remaining steps. The per-step compute of this stage is the finetune row of Table 20. Appendix D Diffusion Decoder: Additional Details Expands Section 2.3. Role and rate hierarchy. The world model produces dense V-JEPA features at fwm=5f_wm=5 Hz, i.e. one feature frame for every fourth RGB frame; the Cosmos tokenizer compresses the raw video time axis by 4×4×, so each Cosmos latent frame aligns with exactly one V-JEPA frame. The decoder therefore does two jobs at once — reconstruct the appearance the V-JEPA features already encode, and imagine the three RGB frames of motion between consecutive V-JEPA frames. Because fwmf_wm is low, the inter-frame motion the decoder needs to fabricate is small, which is why a relatively small DiT (∼800 800M parameters) suffices for 240×320240×320 video. Visualisation-only. The decoder DϕD_φ is used only for visualization: it is never invoked inside CEM, and no gradient flows between it and the world model. Its training data is exclusively frozen V-JEPA 2.1 encoder features (Mur-Labadia et al., 2026); the encoder, the predictor of Section 2.1, and both halves of the Cosmos-0.1-Tokenizer-CV4x8x8 (NVIDIA, 2025) are all frozen, and the only trainable parameters are those of DϕD_φ itself. Frame alignment between V-JEPA and Cosmos. Throughout this appendix a decoder clip spans TdecT_dec V-JEPA steps (Tdec=8T_dec=8 in our experiments) and ν∈0,…,Tdecν\!∈\!\0,…,T_dec\ indexes its frames (we reserve the symbol f for the frozen V-JEPA encoder). To keep the two grids exactly aligned, the dataloader loads video at two frame rates simultaneously: a high-fps stream of 4Tdec+14T_dec+1 RGB frames that the Cosmos encoder compresses into the latents to be denoised, and a low-fps stream of every fourth frame, i.e. the Tdec+1T_dec+1 RGB frames 0,4,…,4Tdec0,4,…,4T_dec, that the V-JEPA encoder turns into conditioning features. Because the Cosmos tokenizer downsamples space by 8×8× and time by 4×4×, encoding the first frame on its own, the 4Tdec+14T_dec+1 RGB frames yield Tdec+1T_dec+1 Cosmos latent frames of shape Hp×WpH_p×W_p with 16 latent channels. The frame-rate multiplier equals the tokenizer’s 4×4× temporal compression, so each Cosmos latent frame ν corresponds to exactly one V-JEPA frame zνz_ν; for Tdec=8T_dec=8 the clip has 99 V-JEPA frames (RGB frames 0,4,…,320,4,…,32), 3333 RGB frames and 99 Cosmos latent frames. Operating in this 8×8×48×8×4-compressed latent space, rather than on pixels, is what makes training a transformer denoiser on 240×320240×320 video tractable on a single node. DiT denoiser architecture. DϕD_φ is a PixArt-style diffusion transformer (Peebles and Xie, 2023; Chen et al., 2024b) with L=32L=32 blocks of hidden dimension d=1280d=1280. Each block uses grouped-query attention (Ainslie et al., 2023) with 2020 query heads and 1010 KV heads, an MLP ratio of 4.04.0, and rotary position embeddings (Su et al., 2024) on the attention queries and keys. Self-attention is restricted to a local temporal window of Wτ=4W_τ=4 Cosmos frames (full attention across the spatial Hp×WpH_p×W_p grid), which keeps the computational cost per token linear in the clip length and admits an efficient block-sparse mask while encouraging local temporal coherence. This Wτ=4W_τ=4 window acts on Cosmos latents (themselves 4×4× temporally compressed) and is distinct from the predictor’s Wt=8W_t=8-V-JEPA-frame window of Section 2.1. Conditioning. The V-JEPA features z0:Tdecz_0:T_dec enter DϕD_φ through dense spatial concat conditioning. The features are upsampled with nearest-neighbour interpolation to the Cosmos grid (in space; in time the two grids already match one-to-one) and concatenated, along the channel axis, with the patchified noisy Cosmos input; a learned linear Linear(d+1664)→dLinear(d+1664)\!→\!d then mixes the two streams into the input token embeddings of the first DiT block. The adaLN modulation (Peebles and Xie, 2023) of every DiT block is conditioned only on the diffusion noise level: for each Cosmos frame ν, the embedding of its timestep tνt_ν produces six per-block parameters (shiftattn,scaleattn,gateattn,shiftffn,scaleffn,gateffn)(shift_attn,scale_attn,gate_attn,shift_ffn,scale_ffn,gate_ffn) that modulate both the attention and FFN sub-layers for that frame. Concretely, with input tokens h∈ℝB×T×N×dh ^B× T× N× d, h′=gate⊙Sublayer(LN(h)⊙(1+scale)+shift)+h,h \;=\;gate\, \,Sublayer\! (LN(h) (1+scale)+shift )\;+\;h, (14) applied per Cosmos frame before flattening back to the joint (T⋅N)(T\!·\!N)-token sequence consumed by attention. Training objective. We train DϕD_φ with a standard DDPM (Ho et al., 2020) forward process applied to the Cosmos latents. Let tν∼0,…,K−1t_ν\! \!U\0,…,K-1\ denote the diffusion timestep sampled independently for each frame ν, following diffusion forcing (Chen et al., 2024a). With a noise schedule α¯t α_t, the per-frame forward corruption is xtν(ν)=α¯tνx0(ν)+1−α¯tνϵ(ν),ϵ(ν)∼(0,I).x_t_ν^(ν)\;=\; α_t_ν\;x_0^(ν)\;+\; 1- α_t_ν\;ε^(ν), ε^(ν) (0,I). (15) Unlike the original ϵε-parameterisation of Ho et al. (2020), the network directly predicts the clean signal x^0(ν)=Dϕ(xtν(ν),tν,z) x_0^(ν)=D_φ(x_t_ν^(ν),t_ν,z). The training loss sums per-frame x0x_0-MSE terms, with tνt_ν promoted to a per-frame random variable inside the expectation: ℒdec(ϕ)=x0,tν∼iid0,…,K−1ν,ϵ,z[∑ν∥Dϕ(xtν(ν),tν,z)−x0(ν)∥22].L_dec(φ)\;=\;E_x_0,\;\t_ν iid U\0,…,K-1\\_ν,\;ε,\;z\! [\, _ν D_φ(x_t_ν^(ν),t_ν,z)-x_0^(ν) _2^2\, ]. (16) We use unweighted x0x_0-MSE, with no additional timestep-dependent weighting. For equivalent x0x_0 and ϵε predictions at 0<α¯t<10< α_t<1, the squared errors satisfy ‖x^0−x0‖2=1−α¯tα¯t‖ϵ^−ϵ‖2\| x_0-x_0\|^2= 1- α_t α_t\| ε-ε\|^2. Thus, expressed as an ϵε-prediction loss, this corresponds to inverse-SNR weighting, emphasizing high-noise rather than low-noise timesteps. The condition z is dropped to the zero vector independently per sample with probability pdrop=0.1p_drop=0.1 during training to support classifier-free guidance (Ho and Salimans, 2022), although in practice we use a guidance scale of w=1.0w=1.0 (no guidance) by default since this is the most faithful for visualization. We use the log-SNR-parameterised cosine schedule of Simple Diffusion (Hoogeboom et al., 2023) in preference to the cosine schedule of IDDPM (Nichol and Dhariwal, 2021): parameterising in log-SNR gives finer control over the signal-to-noise trajectory and avoids the degenerate near-zero-SNR behavior at t→K−1t\!→\!K-1 that the IDDPM schedule exhibits. We use K=1000K=1000 training timesteps and enforce zero terminal SNR following Lin et al. (2024); the per-frame independent tνt_ν-sampling above is what makes the trained denoiser robust to mixed noise levels and, in turn, what enables prefix-conditioned generation at inference time. Sampling. Despite training with K=1000K=1000 DDPM steps, at inference we run only 2 DDIM steps (Song et al., 2021) from xK−1∼(0,I)x_K-1\! \!N(0,I) per Cosmos frame and pipe the result through the frozen Cosmos decoder to pixels. We attribute this to the strength of the V-JEPA conditioning: dense V-JEPA 2.1 features already pin down most of the appearance, so the denoiser’s effective conditional distribution is nearly unimodal and its sampling path nearly straight. For streaming generation we keep the same local-window KV cache as the predictor (Appendix H) and write each new frame’s keys/values only on the last DDIM step so the cache never holds noisy activations; the streaming sampler runs at 44 DDIM steps per new frame, the small overhead buying robustness to the cache turnover. AOT compilation of DϕD_φ for offline visualization is described alongside the predictor’s compilation pipeline in Appendix I; all decoder hyperparameters are collected in Table 11. Table 11: Diffusion-decoder hyperparameters. The decoder is trained on frozen V-JEPA 2.1 encoder features only; the world model, V-JEPA encoder, and Cosmos tokenizer (NVIDIA, 2025) are frozen, with no gradient flow between the decoder and the world model. The local temporal window Wτ=4W_τ=4 acts on Cosmos latents (themselves 4×4× temporally compressed) and is distinct from the predictor’s Wt=8W_t=8-frame window over V-JEPA features (Section 2.1). Hyperparameter Symbol Value Architecture DiT depth L 32 Hidden dim d 1280 Query / KV heads – 20 / 10 (GQA, (Ainslie et al., 2023)) MLP ratio – 4.0 Cosmos latent channels – 16 V-JEPA 2.1 condition dim DeD_e 1664 Local temporal attention window WτW_τ 4 Cosmos frames Position encoding – RoPE (Su et al., 2024) Total trainable parameters – ∼ 800 M Diffusion training V-JEPA steps per decoder clip TdecT_dec 8 (99 V-JEPA / 99 Cosmos frames) Diffusion steps (training) K 1000 Noise schedule – cosine simple-diffusion (Hoogeboom et al., 2023) Terminal SNR – enforced to 00 (Lin et al., 2024) Per-frame timesteps – diffusion forcing (Chen et al., 2024a) Parameterisation – x0x_0-prediction Condition dropout (CFG training) pdropp_drop 0.10.1 Optimisation Optimiser – AdamW, β=(0.9,0.95)β=(0.9,0.95) Learning rate η 10−410^-4 (constant) Grad clip – 1.01.0 Precision – bfloat16 mixed Sampling DDIM steps (inference, default / streaming KV-cached) – 2 / 44 Temporal upsampling (V-JEPA → RGB) – 4×4× Guidance scale w 1.01.0 (default) Figure 17: High Resolution 720p World Model Imagination Rollout. Appendix E Qualitative World-Model Imagination Examples Model-size comparison filmstrips. Each figure seeds RoboJEPA with the initial frame(s) of a held-out episode, replays the recorded actions, and shows the decoded imagination for every model size (2222M–88B) alongside the ground-truth (GT) video. Rows are ordered smallest-to-largest with GT at the bottom, directly beneath the 88B model for the most direct comparison; each row is sampled at k=5k=5 equal-time columns spanning the first 2020 s of the rollout. Big figures are decoded at the 480×640480×640 encoder resolution and small at 240×320240×320; the number of camera views is fixed per embodiment. E.1 Large resolution (480×640480×640) Figures 18, 19, 20 and 21 show high-resolution rollouts on AgiBot, Bridge, and RoboCasa with one or two camera views. Figure 18: AgiBot (single view), 480×640480×640. Model-size comparison of world-model imaginations against ground truth; folding structure emerges only at larger scales. Figure 19: Open X-Embodiment Bridge (single view), 480×640480×640. Physical contact between the gripper and the object develops only for the larger models. Figure 20: AgiBot (two views), 480×640480×640. Bimanual manipulation rollout across model sizes; finer details are preserved as scale increases. Figure 21: RoboCasa (two views), 480×640480×640. Simulated kitchen manipulation rollout across model sizes. E.2 Small resolution (240×320240×320) Figures 22, 23, 24 and 25 show low-resolution rollouts on Bridge, Bridge V2, LeRobot, and DROID. Figure 22: Open X-Embodiment Bridge (single view), 240×320240×320. Contact with the manipulated object develops only in the larger models. Figure 23: Bridge-V2 (single view), 240×320240×320. Clear progression of imagination quality with model scale. Figure 24: LeRobot (two views), 240×320240×320. Two-view manipulation rollout across model sizes. Figure 25: Franka / DROID (two views), 240×320240×320, joint-space actions. Two-view manipulation rollout across model sizes. Appendix F Planning: Additional Details Expands Section 2.2. We give the full algorithm box (Section F.1), the data-parallel elite selection (Section F.2), the spherical translation-sampling distribution (Section F.3), the ℓ2 _2-ball projection (Section F.4), and multi-view goal handling (Section F.5). The computational cost of Equation 3 matches the per-token training loss (Section 2.2, Equation 5) and the controller-rate bridge (Section B.6) is what carries the planner output to the robot. F.1 Full CEM algorithm Algorithm 1 restates one call to CEM with all loop details the main text suppressed. It exposes the per-iteration micro-batch loop (M micro-batches per GPU), the cross-rank AllGather, the explicit per-step return accumulation R(n)=∑hrh(n)R^(n)= _hr^(n)_h, and the momentum-smoothed Gaussian refit. Across consecutive environment steps the planner is warm-started by shifting the previous mean by LexecL_exec steps and re-using its tail, so it converges in only IstepI_step refinement iterations after the cold-start computational cost of I0I_0. Algorithm 1 Receding-horizon CEM planning in V-JEPA feature space (full version). 1: predictor gθg_θ, goal features zg(v)\z^(v)_g\, current context zt−Lctx+1:tz_t-L_ctx+1:t, horizon H, samples per iteration per GPU NsampN_samp, micro-batches M, iterations I, elite count k, momenta (βμ,βσ)( _μ, _σ), init std σ0 _0, displacement bound ρmax _ , distance d(⋅,⋅)d(·,·), world size G, per-view weights wv\w_v\ 2: μ0:H−1← _0:H-1← warm start (shift previous plan by LexecL_exec); σ0:H−1←σ0 _0:H-1← _0 3: for i=1,…,Ii=1,…,I do 4: ←∅S← 5: for m=1,…,Mm=1,…,M do ⊳ micro-batches per GPU; sharded across world 6: Draw Nsamp/MN_samp/M translation samples uh(n)∈ℝ3u^(n)_h ^3 via the spherical parameterisation of Section F.3 7: Draw rotation/gripper coords ∼(0,I) (0,I) when active; concatenate into a~h(n)∈ℝda a^(n)_h ^d_a 8: ah(n)←μh+σh⊙a~h(n)a^(n)_h← _h+ _h a^(n)_h; project translation block onto ℓ2 _2-ball of radius ρmax _ (Section F.4) 9: state←repeat(zt−Lctx+1:t,Nsamp/M)state (z_t-L_ctx+1:t,N_samp/M) ⊳ share KV cache 10: for h=0,…,H−1h=0,…,H-1 do 11: z^t+h+1(n),state←gθ(state,ah(n)) z^(n)_t+h+1,state← g_θ(state,a^(n)_h) for all n 12: end for 13: R(n)←−∑vwvd(z^t+H(v,n),zg(v))R^(n)←- _vw_v\,d\! ( z^(v,n)_t+H,\,z^(v)_g ) ⊳ final-step goal distance, Equation 3 14: ←∪(a(n),R(n))S ∪\(a^(n),R^(n))\ 15: end for 16: if G>1G>1 then 17: AllGather(S) across G ranks, yielding pool of size Nsamp×GN_samp× G 18: end if 19: Select elites ℰ⊂E : top-k by R 20: μ^h,σ^h←mean,std μ_h, σ_h ,std of ℰE at step h 21: μh←(−βμ)μ^h+βμμh;σh←(−βσ)σ^h+βσσh _h←(1\!-\! _μ) μ_h+ _μ _h;\; _h←(1\!-\! _σ) σ_h+ _σ _h 22: end for 23: return a⋆0:H−1←μ0:H−1a _0:H-1← _0:H-1 F.2 Distributed elite selection The planner is sharded data-parallel across G GPUs: every rank draws its own NsampN_samp samples per refinement iteration and rolls them through its replica of gθg_θ, using the same shared KV cache trick along the sample axis as on a single GPU. The per-rank rewards and the matching action tensors are then AllGather-ed (line 17 of Algorithm 1) so that every rank sees the same global pool of size Nsamp×GN_samp× G and can pick the top-k elites independently and consistently. Because this elite set is selected from the global pool, the elite count k is decoupled from the per-GPU sample count: in our DROID configuration we use Nsamp=16N_samp=16 samples per GPU but k=20k=20 elites, which would be ill-defined on a single rank but is in fact a k=20k=20 out of a pool of Nsamp⋅GN_samp\!·\!G (≥256≥ 256 in our deployments). F.3 Spherical translation sampling Within each refinement iteration the proposal mean and standard deviation are per-step Gaussian, but we deviate from a vanilla isotropic Gaussian for the 3-D translation block. Following V-JEPA 2-AC (Assran et al., 2025), we draw the unit translation sample u~∈ℝ3 u ^3 from the joint density r∼Unif[0,1],θ1∼Unif[0,2π),θ2∼Unif[0,2π),r [0,1], _1 [0,2π), _2 [0,2π), (17) u~x=rcosθ1cosθ2,u~y=rcosθ1sinθ2,u~z=rsinθ1, u_x=r _1 _2,\; u_y=r _1 _2,\; u_z=r _1, (18) which is then scaled by the per-axis std and shifted by the per-step mean before the ℓ2 _2-ball projection of Section F.4. The motivation is twofold. First, with r uniform on [0,1][0,1] and angles uniform on [0,2π)[0,2π), the distribution puts noticeable mass at small ‖u~‖\| u\| (because the radial coordinate is uniform rather than thrice-folded), which corresponds to “stay close to the warm-started mean” and stabilizes CEM in regimes where the previous plan is already near-optimal. Second, decoupling radius from angles makes the proposal density easy to clip in norm without distorting the angular distribution: the subsequent projection onto an ℓ2 _2-ball only renormalizes samples whose radius exceeds ρmax _ , leaving the angular content untouched. Rotation and gripper dimensions, when active, are drawn from a standard isotropic Gaussian and treated identically in the subsequent affine warp by (μh,σh)( _h, _h). F.4 Translation ℓ2 _2-ball projection After the affine warp ah(n)=μh+σh⊙a~h(n)a^(n)_h= _h+ _h a^(n)_h, the translation block is projected onto an ℓ2 _2-ball of radius ρmax _ that bounds the per-step end-effector displacement: a(n)h:3←ah(n):3if ∥a(n)h:3∥2≤ρmax,ρmaxah(n):3∥a(n)h:3∥2otherwise.a^(n)_h:3\;←\; casesa^(n)_h:3&if \|a^(n)_h:3\|_2≤ _ ,\\ _ \, a^(n)_h:3\|a^(n)_h:3\|_2&otherwise. cases (19) The projection enforces a per-step end-effector displacement bound that respects the robot’s velocity limits, regardless of how aggressively an unlucky elite refit inflated the CEM covariance. We use ρmax=0.05 _ =0.05 m on DROID and ρmax=0.10 _ =0.10 m on RoboCasa, matching the per-step delta scales of the corresponding demonstration corpora. Because the projection only fires outside the ball, it leaves the (warm-started) mean direction unbiased and, combined with Equation 17, keeps the angular content close to uniform after clipping. The gripper dimension, when active, is element-wise clipped to [−ρgripper,ρgripper][- _gripper, _gripper] rather than projected, since it is a scalar with a different physical meaning. F.5 Multi-view goal handling When the world model is conditioned on multiple cameras, the computational cost in Equation 3 is summed across views with configurable per-view weights wvw_v, re-using the per-block view conditioning of Section A.2 so that goal patches are compared at the correct viewpoint. Goal images that are unavailable for some views are handled by zeroing the corresponding wvw_v at runtime, which lets us re-use the same multi-view checkpoint for single-camera DROID episodes without any retraining. Appendix G Robotic Setup G.1 Franka: DROID Platform Our real-robot experiments run on a Franka Panda arm with a parallel-jaw gripper, controlled through the original DROID software stack (Khazatsky et al., 2024). Deployment is split across two machines. A robot-control server runs on the robot PC and exposes the arm and cameras through a simple TCP/ZMQ request–reply interface; the planner client runs on a GPU cluster and connects to this server remotely. This separation lets the planner marshal a large number of GPUs for parallel world-model inference — the CEM optimizer samples and scores thousands of candidate rollouts per step across the cluster — while a single client process is the only one that exchanges observations and actions with the physical robot (Section G.3). The scene is observed by two cameras: one ZED camera mounted on the wrist and one Intel RealSense camera placed externally to the left of the workspace (unlike the multi-ZED upstream DROID rig, we do not use a third camera). The wrist ZED records at 720×1280720×1280 and the external RealSense at 480×640480×640; both streams are converted to RGB and resized to each agent’s native input resolution (240×320240×320 for the RoboJEPA models) before encoding. At deployment RoboJEPA uses the resulting two views — external (left) and wrist. For RoboJEPA we drive the robot with blocking end-effector control rather than a fixed-rate stream: each environment step commands an end-effector delta pose and then waits until the arm and gripper reach the commanded target before the next observation is returned, so the loop is paced by motion completion rather than a fixed control frequency. Although the underlying environment supports the full DROID action interface (7-DoF Cartesian and 8-DoF joint spaces, in velocity or position mode), our planner acts in a reduced space: it plans only 33D end-effector translation together with a 11D gripper command, holding the end-effector orientation fixed. The proprioceptive state we read back is the end-effector pose and gripper opening (with joint angles also available). At each step the planner encodes the current views and state into the world-model latent space, chooses an action by imagining short rollouts and scoring them against a goal image (Section 2.2), executes it, and repeats; each episode begins by homing the arm and recording the goal observation that defines the task. The full planning configuration is given in Section G.3. For baseline policies such as π0.5 _0.5 (Black et al., 2025) we instead use the standard DROID setup: non-blocking control at 1515 Hz in the joint-velocity action space, matching the interface these models were trained for. Goal Start (initial) Side camera Wrist camera Figure 26: Real-world Franka lift the cup task. A goal image (left) and the initial observation (right) for one episode on the DROID/Franka platform, shown from both cameras: the external side camera (top) and the wrist-mounted camera (bottom). The task is specified purely by the goal image, in which the cup has been lifted off the table; from the start state the planner drives the world model toward these goal features. The two camera views are the inputs the world model is conditioned on at deployment. G.2 Real-robot tasks and evaluation protocol Tasks and success criteria. We evaluate three tasks on the Franka/DROID platform: Grasp, Object Lift, and Pick and Place. Grasp requires the robot to grasp the object. Object Lift requires the robot to hold the object lifted in the air until the end of the episode. Pick and Place requires the object to end up at the target position, regardless of how many times it was dropped during the episode. These completion criteria do not imply one another: success on Pick and Place does not require satisfying the Object Lift criterion. Goal and initial observations for all three tasks are shown in Section G.6. Evaluation protocol. For each model–task combination, we run 3030 episodes for RoboJEPA and 5050 episodes for each VLA baseline. We evaluate RoboJEPA models ranging from 2222M to 88B parameters, using the checkpoint trained with the most compute at each size. All RoboJEPA models use the same planning hyperparameters across tasks, including the planning horizon. Hardware and controller details are provided in Section G.1, and the planning configuration is given in Section G.3. For RoboJEPA, we generate a goal image by executing a predefined robot motion and manually placing the object in the grasping position so the robot can interact with it. Only the final image is provided as the goal for closed-loop planning; no intermediate image subgoals are used. The VLA baselines receive text instructions. Neither RoboJEPA nor the baselines receive task-specific fine-tuning. Scoring and reported metrics. We manually score each episode according to the task-specific completion criteria. For each model–task combination, we report task progress as the mean episode score and success rate as the fraction of episodes with 100%100\% progress. The per-model values are reported in Table 12. Table 12: Real-world robotic capabilities on the Franka/DROID platform. Task progress and success rate on the grasp, object-lift, and pick-and-place tasks. VLA baselines (π0 _0-FAST and π0.5 _0.5) are conditioned on a text instruction, whereas all RoboJEPA models are conditioned on a single image goal. Model Goal specification Grasp Object Lift Pick and Place Task Progress Success Rate Task Progress Success Rate Task Progress Success Rate Model finetuned on DROID, no in-domain finetuning π0 _0-FAST text instruction 35%35\% 22%22\% 20%20\% 12%12\% 55%55\% 40%40\% π0.5 _0.5 25%25\% 5%5\% 12%12\% 0%0\% 69%69\% 53%53\% Pretrained models, no DROID finetuning, no in-domain finetuning, a unified planner for all tasks RoboJEPA-22M single goal image 30%30\% 30%30\% 34%34\% 2%2\% 30%30\% 17%17\% RoboJEPA-50M 30%30\% 30%30\% 51%51\% 8%8\% 27%27\% 13%13\% RoboJEPA-100M 27%27\% 27%27\% 72%72\% 20%20\% 20%20\% 10%10\% RoboJEPA-300M 37%37\% 37%37\% 52%52\% 18%18\% 43%43\% 17%17\% RoboJEPA-1B 50%50\% 50%50\% 53%53\% 24%24\% 38%38\% 10%10\% RoboJEPA-2B 43%43\% 43%43\% 60%60\% 40%40\% 47%47\% 31%31\% RoboJEPA-4B 60%60\% 60%60\% 54%54\% 30%30\% 49%49\% 21%21\% RoboJEPA-8B 67%67\% 67%67\% 65%65\% 50%50\% 42%42\% 27%27\% G.3 Real robot planning hyperparameters On the DROID/Franka platform the world model is deployed with the closed-loop CEM planner of Section 2.2 (full algorithm in Algorithm 1). The planner searches only over 33D end-effector translation and a 11D gripper command (rotation frozen), scores imagined rollouts by per-token ℓ1 _1 distance to the goal features in LN0LN_0 space, and applies LexecL_exec actions per replan before re-observing. Table 13 lists the configuration used for all real-robot experiments. Table 13: Closed-loop CEM planning hyperparameters for the DROID/Franka real-robot deployment (Section 2.2, Algorithm 1). Elites are selected from the pooled Nsamp⋅GN_samp\!·\!G samples gathered across the G planner GPUs, so k may exceed the per-GPU sample count. The planner acts in a 33D-translation ++ 11D-gripper space with the end-effector orientation held fixed. Hyperparameter Symbol Value Horizon and closed loop Planning horizon (predictor steps) H 55 Context length (encoder steps) LctxL_ctx 22 Execution prefix (actions per replan) LexecL_exec 22 Environment steps per episode — 2020 CEM search CEM iterations (cold / warm) I0/IstepI_0/I_step 1515 / 1010 Samples per iteration (global) Nsamp⋅M⋅GN_samp\!·\!M\!·\!G 40964096 Elite count, top-k (post-AllGather) k 2020 (pool Nsamp⋅GN_samp\!·\!G) Computational cost d(⋅,⋅)d(·,·) per-token ℓ1 _1 in LN0LN_0 space Translation search (3D) Init std σ0 _0 0.150.15 Max-norm (ℓ2 _2-ball radius) ρmax _ 0.150.15 Mean / std momentum βμ/βσ _μ/ _σ 0.150.15 / 0.700.70 Gripper search (1D) Init std — 0.800.80 Max magnitude ρgripper _gripper 0.990.99 Mean / std momentum — 0.150.15 / 0.450.45 Observations and execution Views fed to the world model — left (00), wrist (22) Per-view reward weights wv\w_v\ 1.0, 0.0\1.0,\,0.0\ (goal scored on the left view) Input resolution — 240×320240×320 Action substeps — 11 Predictor precision — AOT-compiled bf16 G.4 RoboCasa Custom Environments Our simulated evaluations use four custom tasks built on top of RoboCasa kitchen scenes (Nasiriany et al., 2024; Nasiriany et al., 2026) with the mobile-base PandaOmron (Franka Panda) robot. The four tasks share the same common controller. We drive the robot with a blocking controller that we implement on top of the simulator: each planner action specifies a target end-effector offset, which the environment servos toward with operational-space control until it converges (or a step cap is reached) before returning the next observation. We further extend the controller to drive the commanded end-effector rotation to convergence: rather than the default operational-space controller, which does not guarantee the orientation reaches its target, our controller feeds back the world-frame rotation error at every sub-step and only terminates once both the position and rotation errors fall below tolerance. We found this rotation convergence to be important for correctly matching the wrist-camera view during planning, since the planner scores candidates by a distance induced by both the side-view and the wrist-view predictions and the wrist camera is rigidly mounted to the end-effector, so any residual orientation drift would misalign the wrist view. RoboCasa exposes a large collection of kitchens split into training and evaluation scenes; the world model is trained on the training kitchens and all four tasks are run in the held-out evaluation kitchens. Because the environment is fully simulated, we can define both the goal image and the reward directly inside it. Every episode is instantiated from up to ten held-out target kitchens (one per layout/style pair), with non-structural countertop appliances near the workspace hidden to reduce clutter. Observations are rendered at 480×640480×640 from the agent-view and, when enabled, wrist cameras. In every task the reward is the normalized progress toward the goal, rt=max(0, 1−∥xt−xgoal∥/d0)r_t= \! (0,\,1- x_t-x_goal /d_0 ), where d0d_0 is the initial goal distance. Reach. The robot must move its end-effector from a randomly sampled start pose to a randomly sampled goal pose in the kitchen. We remove all objects from the scene so that nothing occludes the path from the start to the goal. Start and goal are drawn uniformly within a small ball around the robot’s home pose and rejection-sampled so that they lie a fixed distance apart (0.250.25–0.350.35 m) and are reachable without a large reorientation of the end-effector. This task isolates the world model’s ability to predict free-space end-effector motion across kitchens. Figure 27 shows example goal and initial observations. Goal Start (initial) Side camera Wrist camera Figure 27: RoboCasa Reach. Goal and initial observations for one Reach episode, from the external side camera (top) and the wrist camera (bottom). The goal image (left) specifies the target end-effector pose in an emptied kitchen; the planner drives the arm from the start (right) toward the goal. ObstacleReach. A harder variant of Reach: one appliance (by default a coffee machine) is kept in the scene and repositioned directly in front of the robot, and the start and goal are constrained to lie on opposite sides of it. The end-effector must therefore arc around the obstacle rather than travel in a straight line, and the reachable distance band is widened (0.250.25–0.500.50 m). It tests whether the planner can find collision-free trajectories through the world model. Figure 28 shows an example episode. Goal Start (initial) Side camera Wrist camera Figure 28: RoboCasa ObstacleReach. Goal and initial observations for one ObstacleReach episode. An appliance placed directly in front of the robot forces the end-effector to arc around it, so the straight-line path from the start (right) to the goal (left) is blocked. ObjectReach. Here the robot must perform a reach while holding an object. To set this up, we first grasp the object with a heuristic scripted agent: a small Python controller that executes a top-down grasp (approach, descend, close, and lift) inside the simulator. Running the grasp in simulation ensures the object is held in a physically correct way before planning begins. The world model then has to move the held object to a randomly sampled goal position, and the reward is the normalized progress of the object toward its goal rather than of the end-effector. The object is one of a banana, a can, or a mug, which differ in shape and physical properties (friction and grasp offsets are tuned per object). Figures 29, 30 and 31 show examples for the three objects. Goal Start (initial) Side camera Wrist camera Figure 29: RoboCasa ObjectReach (banana). Goal and initial observations after the scripted grasp: from the start (right) the robot must carry the held banana to the goal pose (left), shown from the side (top) and wrist (bottom) cameras. Goal Start (initial) Side camera Wrist camera Figure 30: RoboCasa ObjectReach (can). As in Figure 29 for the can object. Goal Start (initial) Side camera Wrist camera Figure 31: RoboCasa ObjectReach (mug). As in Figure 29 for the mug object. ObjectPush. The robot must push the object across the counter by contact from a randomly sampled start position to a randomly sampled goal position. The gripper is held closed so that it acts as a solid pusher, and the arm is driven by translation deltas, so the object moves only through contact rather than being grasped. Start and goal are sampled a fixed distance apart (0.100.10–0.150.15 m) within a small disk on the counter; because forward pushes (along the robot’s reach direction) are far more reliable than lateral ones, the push direction is restricted to a 60∘60 cone around the forward axis. As in ObjectReach, the reward is the normalized progress of the object toward its goal. This task probes whether the world model captures non-prehensile, contact-rich dynamics. Figure 32 shows example goal and initial observations. Goal Start (initial) Side camera Wrist camera Figure 32: RoboCasa Push. Goal and initial observations for one object-push episode, shown from the side (top) and wrist (bottom) cameras. The robot must push the object across the counter from its initial position (right) to the goal position (left). Note that the goal pose and the initial pose of the robot are the same, the only thing that changes is the object position. G.5 RoboCasa planning hyperparameters All four RoboCasa tasks are solved with the same closed-loop CEM planner (Section 2.2, Algorithm 1). Across the four tasks the only CEM hyperparameter we vary is the predictor horizon H; every other CEM setting is shared (Table 14, top). Push uses the same planning configuration as ObstacleReach. As in the DROID configuration (Table 13) the planner scores rollouts by per-token ℓ1 _1 distance to the goal features and re-plans after every executed action. Reach and ObstacleReach and Push search over 33D end-effector translation, while ObjectReach first grasps an object with a scripted agent and then additionally searches the gripper. The tasks otherwise differ in their environment configuration (Table 14, bottom). Table 14: Planning and environment configuration for the four RoboCasa tasks (Section G.4). The table is split into the planning hyperparameters (top) and the environment parameters (bottom). Among the planning hyperparameters, only the predictor horizon H varies across tasks; the rest are shared. The effective CEM sample budget is 10241024 candidates per iteration for every task (distributed across the planner GPUs), from which the top-k elites are selected. Symbol Reach ObstacleReach ObjectReach Push Planning hyperparameters Predictor horizon H 11 55 11 55 CEM samples per iteration (total) — 10241024 CEM iterations (cold / warm) I0/IstepI_0/I_step 1515 / 1010 Elite count, top-k k 2020 Computational cost d(⋅,⋅)d(·,·) per-token ℓ1 _1 in LN0LN_0 space Context length (encoder steps) LctxL_ctx 22 Execution prefix (actions per replan) LexecL_exec 11 Action substeps — 11 Translation init std σ0 _0 0.300.30 Translation max-norm (ℓ2 _2-ball radius) ρmax _ 0.300.30 Mean / std momentum βμ/βσ _μ/ _σ 0.150.15 / 0.700.70 Views fed to the world model — left (00), wrist (22) Per-view reward weights wv\w_v\ 0.5, 0.5\0.5,\,0.5\ Input resolution — 240×320240×320 Gripper init std — 0.250.25 Gripper max magnitude ρgripper _gripper 0.30.3 Predictor precision — bf16 (compiled) bf16 (eager) bf16 (compiled) bf16 (eager) Environment parameters Environment steps per episode — 55 1818 55 1818 Start–goal distance — 0.250.25–0.350.35 m 0.250.25–0.500.50 m 0.110.11–0.260.26 m 0.100.10–0.150.15 m Manipulated object — — — banana / can / mug flour bag G.6 Real-Robot Task Examples Figures 33, 34 and 35 show goal and initial observations for Grasp, Object Lift, and Pick and Place, respectively. We show goal and initial observations for the three real-robot deployment tasks (grasp, lift, and pick-and-place), ten episodes each, from both the external side camera and the wrist camera. The goal is provided to the planner as a single image; recall that the goal and the initial robot pose coincide, so the visible difference between the goal and initial columns is the configuration of the manipulated object. Goal (side) Goal (wrist) Start (side) Start (wrist) Figure 33: Real-robot Grasp task: goal and initial observations. Ten episodes (rows) of the Grasp task on the DROID/Franka platform. For each episode we show the goal image (left pair) and the initial state (right pair), each from the external side camera and the wrist camera. The goal is specified purely by these images; the planner drives the robot from the initial state toward the goal. Goal (side) Goal (wrist) Start (side) Start (wrist) Figure 34: Real-robot Lift task: goal and initial observations. Ten episodes (rows) of the Lift task on the DROID/Franka platform. For each episode we show the goal image (left pair) and the initial state (right pair), each from the external side camera and the wrist camera. The goal is specified purely by these images; the planner drives the robot from the initial state toward the goal. Goal (side) Goal (wrist) Start (side) Start (wrist) Figure 35: Real-robot Pick-and-Place task: goal and initial observations. Ten episodes (rows) of the Pick-and-Place task on the DROID/Franka platform. For each episode we show the goal image (left pair) and the initial state (right pair), each from the external side camera and the wrist camera. The goal is specified purely by these images; the planner drives the robot from the initial state toward the goal. G.7 Real World Robotic Capabilities We observe qualitative differences in the behavior induced by planning with small and large world models. Small models’ main weakness is poor gripper control, yet their successful episodes exhibit smooth end-effector trajectories and accurate approaches to objects. They struggle to recover after approaching an object from an unsuitable grasping position, whereas larger models can adapt and recover from such mistakes. As model size increases, behavior gradually shifts from smooth execution relying on a single grasping attempt toward more adaptive and robust execution. Intermediate models, such as the 300300M and 11B variants, appear to lie between these behavioral regimes, combining some of the limitations of smaller models with emerging recovery capabilities. When comparing RoboJEPA with the VLA reference models, we cannot fully disentangle the contribution of image-goal specification from that of world-model scaling. The differences in training data and controller interfaces described in Section G.1 also limit this comparison. Nevertheless, all RoboJEPA model sizes use the same goal specification and planning configuration, and the observed improvement across sizes suggests that model scaling contributes to the capabilities of the resulting planner. These qualitative observations complement the task progress and success rates reported in Table 12. Appendix H Key-Value Cache: Additional Details For each predictor layer, we cache the keys and values of past timesteps so that autoregressive prediction processes only the new frame, action, and state tokens. Each timestep contains Ntok=VHpWp+na+ns,na=ns=1,N_tok=VH_pW_p+n_a+n_s, n_a=n_s=1, (20) where VHpWpVH_pW_p is the number of visual tokens across views. The per-layer cache tensors have shape K(ℓ),V(ℓ)∈ℝB×hkv×Lcache×Ntok×dh,K^( ),\,V^( ) ^B× h_kv× L_cache× N_tok× d_h, (21) where B is the batch size, hkvh_kv is the number of key/value heads, and dhd_h is the head dimension. We store tokens in frame-sized slots and bound the cache by the predictor’s local temporal attention window. As prediction advances, the rolling cache replaces the oldest frame with the new one. RoPE uses absolute timestep indices, so cached positions do not need to be renumbered when the buffer wraps. Each new frame attends only to the retained window, keeping the amount of cached history bounded even for long rollouts. At the start of each planning call, we process the observation context once and share its cached keys and values across the CEM candidates. Subsequent predictions maintain candidate-specific caches because their action sequences differ. This avoids repeatedly processing the same observation history, while each prediction step processes only the new tokens and attends to a fixed-size window of cached tokens. The compilation pipeline is described in Appendix I. Appendix I Inference Engine: Additional Details Expands Section 2.2. The story of the inference engine is one long descent: start from the V-JEPA 2-AC (Assran et al., 2025) eager BF16 baseline, identify what binds it (host-side dispatch, not arithmetic; Section I.1), unbind it with PyTorch Inductor lowering and ahead-of-time compilation that also makes cold start usable on a robot (Section I.2), then squeeze the remaining matmul time with a static FP8 path on top (Section I.3). Section I.4 records the roads we abandoned because the workload simply was not where they help. All numbers are wall-clock measurements on a single NVIDIA H200 (140 GB HBM3) running PyTorch 2.9, on the 1 B predictor at 240p with grouped-query attention (12 query heads, 4 KV heads, 40 layers, 1536 hidden dim), and the rolling KV cache of Lcache=8L_cache=8 frames. I.1 Where the time goes (why the model is not matmul-bound) Naive BF16 inference is bound by host-side dispatch, not by matrix multiplication. At steady state (B=16B=16, full KV cache, 2-view) GEMMs take only ∼26% 26\% of GPU time, attention ∼41% 41\%, and fused pointwise ops (RoPE, GELU, normalizations, cache management) the remaining ∼33% 33\%; on top of that, eager mode burns ∼31% 31\% of wall time in dispatch overhead. Doubling GEMM throughput would buy at most a 1.21×1.21× end-to-end factor; removing dispatch and fusing the long pointwise tail are the first-order wins. At B=1B=1 the same decomposition is even more dispatch-dominated (∼47% 47\% of wall time), which is why the headline speedups in Section 2.2 are largest at small batch. Per-step kernel share within a single rollout is dominated by attention growing with cache occupancy: at step 00 attention is ∼14% 14\% of step time; at step 44 ∼32% 32\%; at full cache (step 7+7^+) ∼41% 41\%. GEMM time stays roughly constant at ∼25 25 ms per step regardless of cache depth. This is the computational-cost picture that the rest of the engine targets. Table 15: Per-step decomposition into GPU kernel time and host-side dispatch overhead at B=16B=16 on H200. AOT Inductor removes essentially all dispatch (−65-65 ms) and additionally speeds up GPU compute by ∼1.8× 1.8× through kernel fusion (8,975→1,2918,975→ 1,291 kernels for the 1-view model). 11-view rows are retained here because dispatch behavior is qualitatively different at small sequence length, even though the deployed CEM planner runs the 2-view path. Configuration Wall (ms) GPU kernel (ms) Dispatch (ms) Dispatch % 1-view eager 145.6 76.7 68.9 47.3% 1-view AOT 45.0 41.9 3.1 6.8% 2-view eager 217.1 148.7 68.4 31.5% 2-view AOT 96.7 93.7 3.0 3.1% I.2 AOT Inductor compilation: from dispatch to deployment The natural answer to dispatch overhead is to lower the per-step graph into fused Triton kernels with PyTorch Inductor (Ansel et al., 2024). A JIT torch.compile path does this, but it specializes into many shape variants on the first call; we measure 1515–2020 minutes of warm-up at robot start per autoregressive step pattern, which is incompatible with closed-loop deployment where the operator turns the robot on and expects it to plan in seconds. Autoregressive predictor inference, however, has a small predictable shape set. At step k with batch size B, every tensor shape is fully determined; the rolling cache has only 99 distinct configurations (steps 00–77 grow the cache; step 88 overwrites the oldest entry and repeats forever). With six compiled batch sizes B∈1,2,4,8,16,32B∈\1,2,4,8,16,32\ this produces 5454 static programs for the predictor; the diffusion decoder of Section 2.3 compiles to an analogous family of programs through the same pipeline. Each program is exported with torch.export under static shapes and packaged via torch._inductor.aoti_compile_and_package into a 3030–5050 MB compiled C++ artefact. The static-shape lowering has two prerequisites that drive engineering choices upstream. First, a mutation-free cache: torch.export’s functionalisation rewrites in-place slices into hundreds of aten.copy ops, which inflated GPU buffer count by ∼5× 5× in our first pass. We thread a functional cache through every block (each block reads the input cache tensors and returns fresh ones, the per-layer outputs are stacked, and a wrapper module bakes cache_size and input_position as compile-time constants). The functional path produces bitwise-identical outputs to the in-place version across all cache regimes (empty, mid, full). Second, shared weight tensors across programs: because the ∼ 22 GB BF16 weight set is held once rather than once per program, this drops resident memory from ∼ 108 GB to ∼14 14 GB — the difference between fitting and not fitting on the deployment box. Non-compiled batch sizes are padded up to the nearest compiled value (e.g. B=3→4B=3→ 4) and sliced back on output, so the worst-case waste from bucketing is ∼50% 50\% when the requested B is just above a compiled size. End-to-end the AOT pipeline yields a 7×7× reduction in launched kernels (8,→1,2918,975\!→\!1,291 for the 1-view predictor), a 1.8×1.8× GPU compute speedup from better fusion, and a 23×23× reduction in dispatch overhead (Table 15). Crucially, cold start collapses from 1515–2020 minutes (JIT) to 1.21.2–1.51.5 seconds: the time to load the 5454 .pt2 files and inject the shared weights. The computational cost of offline compilation (4040–6060 minutes total, parallelised across multiple H200s by an offline compiler) is paid once per checkpoint; thereafter the runtime dispatcher loads the compiled archives and routes each step() call to the matching program. Inside the AOT pipeline we keep one positive low-level result that the failed-attempts ledger would otherwise hide: native GQA dispatched to FlashAttention (Dao et al., 2022; Dao, 2024) avoids the implicit repeat_interleave expansion that the cuDNN attention path uses, contributing the ∼ 3%3\% on top that survives in the headline recipe. I.3 Static FP8 path: the last small lever After AOT removes dispatch, what remains is steady-state GEMM and attention time. We can shave a final factor by running the predictor’s linear layers in FP8. Because the workload is not matmul-bound, this is genuinely a small lever — which is itself the data point that justifies the engineering: order wins, not low-precision arithmetic. The FP8 path (Table 16) runs through torch.compile(fullgraph=True) rather than the AOTI path because the AOTI loader currently mishandles many co-resident FP8 programs; the JIT path with the Inductor disk cache is instant after the first compile. Activation scales are calibrated once on a small held-out set and stored as fixed buffers; weights are kept as float8_e4m3fn with fast-accumulate FP8 GEMMs. Outputs match the BF16 reference to within the spatial-sensitivity benchmark’s noise floor (Section 3.2). Table 16 reports per-cache-step latency at B=16B=16 on the 2-view model: the FP8 advantage is largest at low cache occupancy where the GEMM share is relatively higher; once attention dominates at full cache it shrinks to ∼5% 5\%. Table 16: Static FP8 vs. BF16 AOTI on the 2-view 1B predictor at B=16B=16, averaged across the rolling-cache trajectory. Cache step BF16 AOTI (ms) FP8 torch.compile (ms) Speedup 00 61.5 54.8 1.12×1.12× 44 94.2 82.4 1.14×1.14× 77–99 ∼108 108 ∼103 103 1.05×1.05× average 93.1 84.1 1.11×1.11× I.4 Optimisations that did not work We document the negative results because they sharpen the picture of what the dominant computational cost actually is. Dynamic FP8. A first FP8 attempt with per-call activation amax computation was slower than BF16 (177177 ms vs. ∼100 100 ms): the three extra ops per linear (cast to FP32, amax reduction, quantise) have a higher computational cost than the FP8 GEMM saves on a model that is not matmul-bound. Static, pre-calibrated scales (Section I.3) eliminate this overhead. Weight-only INT8 and W8A8 / F8A8 variants. The torchao implementations were either incompatible with the dispatcher’s requirement that linear outputs match the BF16 KV-cache dtype required by FlashAttention, or produced ∼2× 2× slower forwards because dequantisation overhead dominates on a compute-bound (not bandwidth-bound) model. CUDA graphs. Reduce-overhead capture segfaults inside the captured graph because the KV cache slice assignment mutates a GPU tensor whose target offset changes between replays. A clone-based workaround runs but is slower (9696 ms vs. 9191 ms) than the non-graph path: once AOT has reduced dispatch overhead to ∼3 3 ms there is essentially nothing left for graph capture to amortise. TensorRT. Both whole-model and per-block TensorRT wrappers were slower (0.50.5–0.7×0.7×) and produced incorrect outputs, tripping a known multi-output-export bug. AOT Inductor already produces near-optimal fused Triton kernels for this architecture, so TensorRT did not add value. RoPE pre-computation. Pre-computing cos/sin tables as registered buffers gave 0%0\% speedup under AOT, because Inductor already fuses RoPE inline into its surrounding Triton kernels. cuDNN attention in AOT. sdpa_cudnn is 2.4×2.4× faster than xformers_fmha in isolation but only ∼3% 3\% faster in the AOT pipeline: torch.export captures all SDPA variants as the same ATen op and AOT Inductor lowers to flash-attention regardless of context-manager hint. The native-GQA-without-repeat_interleave factor that survives is reported as part of the AOT pipeline (Section I.2). Appendix J Scaling Laws: Additional Parametric Forms Expands Section 3.1 and Table 2. In addition to the saturating power law and the second-order power law of Section 3.1, we fit two published functional forms as extrapolation baselines: the Broken Neural Scaling Law (BNSL) (Caballero et al., 2022) and the Unified Neural Scaling Law (UNSL) (Caballero et al., 2026). Here we give their full functional forms and the fitted parameter values for the single-break (n=1n=1) instances whose extrapolation errors are reported in Table 2. Fitting protocol. For each episode we pick a random starting timestep, encode its initial observation, and roll the predictor forward for nine steps under the recorded action sequence, giving a ten-step sequence including the initial observation. We then encode the corresponding ground truth future frames and average the L1L_1 distance between predicted and encoder features at each token over the rollout. We average this over roughly 1,024 episodes with randomized starting timesteps. For both laws the prediction target is the loss L, defined as the mean rollout-ℓ1 _1 over horizons 11–99. We fit each law on the 4747 checkpoints of the small models (2222M–22B; a fixed set of cooldown steps per size) and report held-out extrapolation error as the mean absolute error over the 3333 large-model checkpoints (2121 at 44B and 1212 at 88B). We deliberately avoid the C≈6NDC\!≈\!6ND approximation: compute is taken from the empirical per-size accounting, C=PN⋅sC=P_N· s, where s is the number of optimization steps and PNP_N the measured FLOPs per step for each size (2222M=×1015=1.56\!×\!10^15, …, 300300M=×1016=1.57\!×\!10^16, …, 88B=×1017=3.05\!×\!10^17). Training data is D=s⋅TtokD=s· T_tok with Ttok=B(V1+V2)(2T−1)(HW+A+S)=7,190,016T_tok=B(V_1+V_2)(2T-1)(HW+A+S)=7,190,016 tokens per optimization step, and model size N is the nominal parameter count. All curves are fit by minimizing the root-mean-squared log error between predicted and observed L, with light ℓ2 _2 regularization (λ=×10−6λ\!=\!3\!×\!10^-6) on the parameters, log-transformed and standardized inputs, and multiple random restarts to mitigate local optima. J.1 Broken Neural Scaling Law (BNSL) The general BNSL functional form with n breaks is (Caballero et al., 2022) L=a+(bx−c0)∏i=1n(1+(xdi)1/fi)−cifi.L=a+ (b\,x^-c_0 ) _i=1^n (1+ ( xd_i )^1/f_i )^-c_if_i. (22) The instance in Table 2 is univariate in compute (x=Cx=C) with a single break (n=1n=1), L(C)=E+AC−α(1+(C/d)1/f)−βf,L(C)=E+A\,C^-α (1+(C/d)^1/f )^-β f, (23) with an irreducible floor E, a first-segment scale/exponent (A,α)(A,α), and one smooth transition of sharpness f at compute scale d that changes the log–log slope of L−EL-E from −α-α to −(α+β)-(α+β). It assumes the loss depends on N and D only through total compute C. Table 17: Fitted BNSL (n=1n=1, Equation 23) parameters for the DROID and RoboCasa holdouts. Dataset E A α d f βfβ f DROID 0.2010.201 0.1150.115 ≈0≈ 0 3.15×10193.15× 10^19 0.6510.651 0.140.14 RoboCasa 0.1840.184 3.173.17 0.0670.067 2.60×10362.60× 10^36 3.113.11 7.9×1047.9× 10^4 J.2 Unified Neural Scaling Law (UNSL) The UNSL functional form (Caballero et al., 2026) generalises BNSL to the multivariate setting in which several quantities are scaled simultaneously. Its core building block is the Multivariate Broken Neural Scaling Law (MBNSL) kernel K=b∏ixi−ci0(1+(∏ixici1d)1/f)−f.K=b _ix_i^-c_i0 (1+ ( _ix_i^c_i1d )^1/f )^-f. (24) The instance in Table 2 is the multivariate form in (N,D)(N,D), L(N,D)=a0+(R−1+A3−1)−1,R=KND+KN+KD,L(N,D)=a_0+ (R^-1+A_3^-1 )^-1, R=K_ND+K_N+K_D, (25) where each component K is a broken power law with one break (n=1n=1). Because our runs are single-epoch, we specialize the general UNSL by (i) collapsing dataset size and training steps into a single data axis D and dropping the overfitting component, (i) omitting the hyperparameter (“oppositional-force”) terms since no hyperparameters are swept, and (i) taking the evaluation metric to be unbounded above (no a2a_2 term). Unlike the other three candidates, the UNSL is a surface in (N,D)(N,D) rather than a curve in C: it does not admit a closed-form L(C)L(C) (the compute-optimal envelope L∗(C)=minNL(N,D(N,C))L^*(C)= _NL(N,D(N,C)) has no analytic solution for this form), so its extrapolation error is evaluated per checkpoint at each large model’s actual (N,D)(N,D). Table 18: Fitted UNSL (n=1n=1, Equation 25) parameters for the DROID and RoboCasa holdouts. Each K component is an MBNSL kernel (Equation 24); for KNDK_ND the exponents c0,c1c_0,c_1 are vectors over (N,D)(N,D). Dataset Fitted parameters DROID a0=0.098,A3=3.71a_0=0.098,\ A_3=3.71 KND:b=1.23×1011,c0=(−0.38,1.31),f=1.81,d=2.18×109,c1=(2.09,−0.68)K_ND\!:\ b=1.23× 10^11,\ c_0=(-0.38,1.31),\ f=1.81,\ d=2.18× 10^9,\ c_1=(2.09,-0.68) KN:b=3174,c0=0.50,f=0.65,d=2.76,c1=0.06K_N\!:\ b=3174,\ c_0=0.50,\ f=0.65,\ d=2.76,\ c_1=0.06 KD:b=1.36,c0=0.064,f=−0.19,d=1114,c1=0.27K_D\!:\ b=1.36,\ c_0=0.064,\ f=-0.19,\ d=1114,\ c_1=0.27 RoboCasa a0=0.046,A3=1.54a_0=0.046,\ A_3=1.54 KND:b=3505,c0=(−0.12,0.48),f=0.67,d=1.23×1015,c1=(−0.75,1.81)K_ND\!:\ b=3505,\ c_0=(-0.12,0.48),\ f=0.67,\ d=1.23× 10^15,\ c_1=(-0.75,1.81) KN:b=5.03×105,c0=0.76,f=0.55,d=1.85×10−5,c1=−0.55K_N\!:\ b=5.03× 10^5,\ c_0=0.76,\ f=0.55,\ d=1.85× 10^-5,\ c_1=-0.55 KD:b=2.62×10−12,c0=−0.92,f=0.65,d=2.52×108,c1=0.73K_D\!:\ b=2.62× 10^-12,\ c_0=-0.92,\ f=0.65,\ d=2.52× 10^8,\ c_1=0.73 J.3 Scaling Laws for Short- and Long-Horizon Prediction Training Expands the “Towards better scaling” discussion of Section 3.3. There we show that spending extra compute only in the final learning-rate annealing (cooldown) phase — resumed from the same flat-LR pretraining checkpoints — improves the world model’s scaling without any new data. We compare two cooldown recipes, both fit with the second-order power law L(C)=E+ACα−γlnCL(C)=E+A\,C^α-γ C on the compute-optimal frontier: • short-horizon prediction training — the standard cooldown: a short autoregressive rollout (K=2K=2) at a single 240×320240×320 resolution; • long-horizon prediction training — a cooldown with a K=10K=10 autoregressive rollout computed with a KV cache, with mixed 240×320240×320 and 480×640480×640 resolutions, which has several times the computational cost per optimizer step. The main-text Figure 14 plots the combined (DROID++RoboCasa) metric; Figures 36 and 37 break the same comparison down per evaluation suite, and Figures 38 and 39 show the long-horizon training law on its own. On both suites the long-horizon training frontier crosses below the short-horizon training one at ∼5×1020 5×10^20 FLOPs and continues to a lower floor, i.e. long-horizon prediction training scales strictly better at large compute budgets, confirming that the imagination scaling law can be improved with compute alone. Figure 36: Short- vs. long-horizon prediction training on DROID. Out-of-distribution imagination L1 on the DROID holdout versus training compute. Faded curves are per-size checkpoints (blue: short-horizon training, orange: long-horizon training; shade encodes model size); the bold curves are the two second-order power laws fit on each recipe’s frontier. The long-horizon training law overtakes the short-horizon training one at high compute and reaches a lower irreducible floor. Figure 37: Short- vs. long-horizon prediction training on RoboCasa. As Figure 36, for the RoboCasa target evaluation. The same crossover and lower floor for long-horizon training hold on the simulated suite. Figure 38: Long-horizon training scaling law on DROID. Long-horizon prediction training alone: per-size checkpoint curves colored by parameter count, with the fitted second-order power law (dashed) on the compute-optimal frontier of the DROID holdout evaluation. Figure 39: Long-horizon training scaling law on RoboCasa. As Figure 38, for the RoboCasa target evaluation. J.4 Per-Step Training Compute across Stages Underpins the compute axis of Sections 3.1 and 3.3. The training-compute value C on every scaling-law plot is accumulated as C=(per-step FLOPs)×(optimizer steps)C=(per-step FLOPs)×(optimizer steps), so the per-step computational cost of each stage sets the horizontal scale. We compute it for each run from its real configuration (view-group layout, per-cell batch size, resolution grid, view count, rollout steps, and node/GPU count), counting both the forward and backward pass. A single “per step” figure is the total the whole job performs for one optimizer update, and therefore already includes all gradient-accumulation microbatches, all data-parallel replicas, and — for rollout recipes — the autoregressive rollout unrolled over its configured steps. We report compute in PFPF (1PF=10151\,PF=10^15 FLOPs). RoboJEPA training has three stages with very different per-step computational costs. The long pretraining (flat learning-rate) phase and the short-horizon prediction training cooldown share the same cheap recipe (a short K=2K=2 rollout at a single 240×320240×320 resolution). The long-horizon prediction training cooldown of Section 3.3 instead uses a K=10K=10 rollout with a KV cache and mixed 240×320240×320 and 480×640480×640 resolutions, and the DROID 720×1280720×1280 finetune (Section G.1) adds three high-resolution views on top of the KV-cached K=10K=10 rollout. Table 19 gives the per-step computational cost of the two cooldown/pretraining recipes at every model size, and Table 20 compares all three stages at 88B. Table 19: Per-optimizer-step training compute (in PFPF, i.e. 101510^15 FLOPs) for each model size. All values report total FLOPs per optimizer step across the full training job and are used for the compute axis of the scaling-law plots. The short-horizon training column also applies to pretraining. The short-horizon recipe is exact; the long-horizon recipe runs its rollout with a KV cache and is the estimate without the cache scaled by the measured KV-cache factor 0.360.36. Model size Short-horizon training Long-horizon training (K=2K=2, 240×320240×320) (K=10K=10, 240×320+480×640240×320+480×640) 2222M 1.561.56 6.866.86 5050M 3.103.10 13.1113.11 100100M 5.745.74 23.5323.53 300300M 15.6615.66 41.941.9 11B 47.7747.77 176.15176.15 22B 81.1781.17 286.4286.4 44B 157.84157.84 534.45534.45 88B 305.36305.36 1002.751002.75 Table 20: Per-step training compute at 88B for the three stages. All values report total FLOPs per optimizer step across the full training job. One finetune step is ∼19× 19× a long-horizon training step and ∼62× 62× a short-horizon training step. The finetune uses a longer sequence: a 720×1280720×1280 image is a 45×80=3,60045×80=3,600-token grid per view (×3× 3 views =10,800=10,800 tokens, versus ∼300 300 tokens/view at 240×320240×320), and self-attention is (n2)O(n^2) in tokens. Stage Recipe Total per-step PFLOPs Pretraining / short-horizon training K=2K=2, 240×320240×320 (tp=1tp=1) 305305 PF Long-horizon training K=10K=10 with KV cache, multi-res (tp=2tp=2) 1,0031,003 PF DROID 720×1280720×1280 finetune 33-view 720×1280720×1280, K=10K=10 with KV cache (tp=4tp=4) ∼18,900 18,900 PF J.5 Stochastic upper-envelope selection Extracts the compute-optimal frontier of the downstream planning success-rate (reward) curves reported in the visual abstract (Figure 1). This procedure applies to the RoboCasa downstream planning tasks, where higher success rate is better. Every model size contributes many checkpoints, so at a fixed size the (compute, success-rate) pairs are stochastic rather than a single clean curve: evaluation noise and training instabilities give a scatter of points at nearby compute values. Fitting a scaling law requires the compute-optimal upper envelope — the best achievable success rate at each compute — rather than the raw scatter, but the obvious envelope estimators are both unsatisfying. A running maximum over compute is not robust: a single lucky, high-variance checkpoint pulls the envelope up prematurely and the estimate never recovers. Fitting the law to all points instead is dragged toward the mean of the scatter, since many dominated, lower checkpoints outnumber the few near the ceiling. We instead select frontier points with a stochastic upper-envelope procedure that tracks a running estimate of the achievable ceiling and admits only points consistent with it, rejecting downward outliers before they can corrupt the running estimate. Concretely, we pool all (compute, success-rate) pairs (Ci,yi)(C_i,y_i) across model sizes (higher y is better) and sort them by ascending compute. The running mean m and variance v are initialized from the first three points, which are always kept. Each subsequent point is compared against the current upper trend m−ksm-ks, where s=vs= v and k≥0k≥ 0 is a rejection threshold in standard-deviation units: a point is accepted onto the frontier only if it does not fall more than k standard deviations below the running mean. Accepted points update m and v with an exponentially-weighted (West) moving-average and moving-variance update governed by a retention factor α∈(0,1)α∈(0,1), so recent frontier points dominate the running estimate while stale ones decay away; rejected points leave m and v untouched, so a single downward outlier can neither join the frontier nor bias the trend used to judge later points. The scaling law is then fit only on the resulting frontier set. We use α=0.9α=0.9 throughout, with a per-task rejection threshold matched to the four RoboCasa downstream planning tasks of the visual abstract: k≈0.5k≈ 0.5 for the two shorter-horizon, greedy tasks, a larger k≈2k≈ 2 for the noisier long-horizon obstacle task, and no rejection (k→∞k→∞, all points kept) for the object-push task, reflecting the differing success-rate evaluation noise of each task (short-horizon vs. long-horizon, and whether object interaction is involved). Algorithm 2 Stochastic upper-envelope (frontier) selection 1: points (Ci,yi)\(C_i,y_i)\ pooled across model sizes (higher y is better); retention factor α∈(0,1)α∈(0,1); rejection threshold k≥0k≥ 0 2: frontier index set S 3: Sort points by ascending compute C 4: Initialize running mean m and variance v from the first three points 5: S←1,2,3S←\1,2,3\ 6: for each subsequent point (Ci,yi)(C_i,y_i) do 7: s←vs← v 8: if yi>m−ksy_i>m-ks then ⊳ accept: not an anomalous downward dip 9: S←S∪iS← S∪\i\ 10: d←yi−md← y_i-m 11: m←m+(1−α)dm← m+(1-α)d 12: v←α(v+(1−α)d2)v←α (v+(1-α)d^2 ) 13: else 14: reject i ⊳ m,vm,v left unchanged 15: end if 16: end for 17: return (Ci,yi):i∈S\(C_i,y_i):i∈ S\ The new-point weight (1−α)(1-α) controls how quickly the running estimate adapts: α close to 11 retains a long memory of past frontier points and adapts slowly, while smaller α tracks recent points more aggressively. Because m and v are updated only on accepted points, the procedure is robust to an arbitrary number of downward outliers at any compute scale, unlike a plain running maximum. A reference implementation is provided in make_plot.py as the function stochastic_envelope. Appendix K Encoder Choice and World-Model Scaling Initially, we started the project by scaling the predictor on top of the V-JEPA 2 encoder. However, we faced numerous issues and instabilities during training. The predictor often collapsed beyond a certain model size, despite the frozen encoder, and increasing the dataset size often led to training instabilities or sometimes divergence. For both encoders, we worked with image features only. We hypothesize that this behavior can be attributed to the type of features the encoder computes. Table 21 restates published evaluations of V-JEPA 2 (Assran et al., 2025) and V-JEPA 2.1 (Mur-Labadia et al., 2026) on the classification, semantic segmentation, and depth estimation benchmarks shown in Figure 2 of the latter paper, together with IntPhys2 (Bordes et al., 2025) results reported by Punzo et al. (2026). These are results from the cited papers, not evaluations conducted in this work. Table 21 shows that the V-JEPA 2 and V-JEPA 2.1 encoders perform on par on image and video understanding tasks, measured through image and video classification, and V-JEPA 2.1 even obtains a lower reported score on the IntPhys2 intuitive physics task than V-JEPA 2. On the other hand, V-JEPA 2.1 substantially outperforms V-JEPA 2 on dense tasks such as depth estimation and semantic segmentation. Given this large gap, we hypothesize that high-quality dense features are the most important features for scaling robotic world models, even more important than features associated with physical understanding. Table 21: Restated results from Mur-Labadia et al. (2026) (Tables 8–9) and Punzo et al. (2026) (Table 1, IntPhys2); V-JEPA 2 classification results are also reported by Assran et al. (2025) (Table 4). We distinguish global tasks (action recognition, image classification, and intuitive physics) from dense tasks (semantic segmentation and depth estimation), following the grouping in Figure 2 of Mur-Labadia et al. (2026). Classification uses attentive probes; segmentation and depth use linear probes, with frozen encoders. IntPhys2 reports mean violation-of-expectation (VOE) accuracy over three seeds with a temporal attentive probe at each model’s best reported layer. Higher accuracy and mIoU, and lower RMSE, are better. Model sizes and pretraining recipes differ, so this is not a controlled ablation. Type Benchmark Metric V-JEPA 2 V-JEPA 2.1 ViT-g (1B) ViT-G (2B) Global SSv2 action recognition Top-1 (%) ↑ 77.3 77.7 Global K400 action recognition Top-1 (%) ↑ 87.3 87.7 Global IN1K image classification Top-1 (%) ↑ 85.1 85.5 Global IntPhys2 (Bordes et al., 2025) VOE (%) ↑ 66.0 58.8 Dense ADE20K semantic segmentation mIoU (%) ↑ 24.4 47.9 Dense NYUv2 depth estimation RMSE ↓ 0.642 0.307