Paper deep dive
ViVa: A Video-Generative Value Model for Robot Reinforcement Learning
Jindi Lv, Hao Li, Jie Li, Yifei Nie, Fankun Kong, Yang Wang, Xiaofeng Wang, Zheng Zhu, Chaojun Ni, Qiuping Deng, Hengtao Li, Jiancheng Lv, Guan Huang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 4/10/2026, 4:27:40 AM
Summary
ViVa is a video-generative value model for robot reinforcement learning that repurposes pretrained video diffusion Transformers to estimate task progress. By jointly predicting future proprioception and a scalar value, ViVa grounds value estimation in anticipated embodiment dynamics, outperforming static VLM-based value models in long-horizon robotic manipulation tasks like box assembly.
Entities (4)
Relation Signals (3)
ViVa โ basedon โ Wan2.2
confidence 95% ยท We build our video-generative value model upon Wan2.2
ViVa โ integratedinto โ RECAP
confidence 95% ยท Integrated into RECAP, ViVa delivers substantial improvements on real-world box assembly.
ViVa โ performson โ Box Assembly
confidence 90% ยท ViVa delivers substantial improvements on real-world box assembly.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator for value estimation. Taking the current observation and robot proprioception as input, ViVa jointly predicts future proprioception and a scalar value for the current state. By leveraging the spatiotemporal priors of a pretrained video generator, our approach grounds value estimation in anticipated embodiment dynamics, moving beyond static snapshots to intrinsically couple value with foresight. Integrated into RECAP, ViVa delivers substantial improvements on real-world box assembly. Qualitative analysis across all three tasks confirms that ViVa produces more reliable value signals, accurately reflecting task progress. By leveraging spatiotemporal priors from video corpora, ViVa also generalizes to novel objects, highlighting the promise of video-generative models for value estimation.
Tags
Links
- Source: https://arxiv.org/abs/2604.08168v1
- Canonical: https://arxiv.org/abs/2604.08168v1
Trouble viewing inline? Open PDF directly โ
Full Text
56,929 characters extracted from source content.
Expand or collapse full text
2026-4-10 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning Jindi Lv 1,2* , Hao Li 1* , Jie Li 1 , Yifei Nie 1 , Fankun Kong 1 , Yang Wang 1 , Xiaofeng Wang 1,3 , Zheng Zhu 1โ , Chaojun Ni 1 , Qiuping Deng 1 , Hengtao Li 1 , Jiancheng Lv 2โ , Guan Huang 1 1 GigaAI 2 Sichuan University 3 Tsinghua University * Equal Contribution โ Corresponding Authors Project Page: https://viva-value-model.github.io/ ViVa High (t) Current Images Left wrist (t) Right wrist(t) Scalar Value Progress towards successful task completion Current Proprioception ํ ํ ํ ํ =ํ ฯ ํ+ํ:ํ [ ํ=ํ ํ ํ ํ ] ํ ํ โ ํ 14 (endeffector pose or joint angles) ํ ํ ํ ํ ,ํ ํ = ํ โฒ =ํ ํ+ํโํ ํ ํ โฒ +ํ ํ ํ ํ+ํ โํ ํ ํ ํ Future Proprioception ํ ํ+ํ โ ํ 14 (endeffector pose or joint angles) Advantage ฮต 0,1 Figure 1: Illustration of ViVa. Given the current multi-view observations and robot proprioception, ViVa jointly predicts the future proprioceptive state and a scalar value representing task progress toward successful comple- tion. By grounding value estimation in anticipated embodiment dynamics, ViVa leverages the spatiotemporal priors of pretrained video generation models to incorporate predictive structure beyond static snapshots. In addition to strong performance on in-domain tasks, ViVa demonstrates robust generalization to novel objects. Abstract Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator for value estimation. Taking the current observation and robot proprioception as input, ViVa jointly predicts future proprioception and a scalar value for the current state. By leveraging the spatiotemporal priors of a pretrained video generator, our approach grounds value estimation in anticipated embodiment dynamics, moving beyond static snapshots to intrinsically couple value with foresight. Integrated into RECAP, ViVa delivers substantial improvements on real-world box assembly. Qualitative analysis across all three tasks confirms that ViVa produces more reliable value signals, accurately reflecting task progress. By leveraging spatiotemporal priors from video corpora, ViVa also generalizes to novel objects, highlighting the promise of video-generative models for value estimation. 1. Introduction Building robots that can perceive, reason, and act in the physical world remains a central challenge in embodied artificial intelligence (Li et al., 2026; Sapkota et al., 2025). Vision-language-action (VLA) models (Intelligence et al., 2025; Kim et al., 2024; Li et al., 2025; Team et al., 2026; Zitkovich et al., 2023) have made significant strides by leveraging large-scale pretraining to enable general-purpose manipulation across diverse tasks. Yet success in real-world settings requires more than static scene understanding: robotic interaction unfolds ยฉ 2026 GigaAI. All rights reserved. arXiv:2604.08168v1 [cs.RO] 9 Apr 2026 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning under partial observability and delayed feedback, where the consequences of decisions only manifest over extended horizons (Huang et al., 2022; Zitkovich et al., 2023). Learning to connect present behavior with future outcomes thus remains a fundamental challenge for real-world robotics. This challenge calls for an ability that assesses whether ongoing interaction is progressing toward successful task completion. Such progress awareness allows robots to distinguish beneficial behaviors from undesirable ones and improve through experience. In reinforcement learning (RL) (Sutton et al., 1998), this capability is formalized by the value function, which estimates expected future outcomes and provides a learning signal for policy improvement. Recent VLA frameworks such asํ * 0.6 (Intelligence et al., 2025) highlight this importance: their RL with Experience and Corrections via Advantage-conditioned Policies (RECAP) pipeline relies on a multi- task value function for advantage estimation and policy refinement, demonstrating that learning performance strongly depends on value model quality. Motivated by this importance, recent works have explored leveraging vision-language models (VLMs) (Bai et al., 2025; Chen et al., 2024; Comanici et al., 2025; Li et al., 2024; Marafioti et al., 2025; Zhu et al., 2025) for value estimation, framing value prediction as classification (Intelligence et al., 2025) or temporal ordering problems (Ma et al., 2024). While promising, these approaches inherit a key limitation: VLMs are primarily trained on static imageโtext data for semantic understanding rather than explicitly modeling how scenes evolve over time. Accordingly, they capture what is present in a scene but struggle to represent how interactions dynamically transform the environment. This mismatch limits their ability to support reliable value estimation in temporally extended robotic tasks. The above limitations reveal a key insight: value estimation is inherently a problem of anticipating how the future will unfold. In contrast to discriminative models trained on static data, video generative models are explicitly optimized to capture temporal evolution, learning how scenes change as interactions unfold. This makes them a natural foundation for value estimation, as the ability to imagine future outcomes directly enables assessing whether current behavior progresses toward task completion. Guided by this observation, we reformulate value learning as future prediction and develop a video-generative value model. In this paper, we propose Video-generative Value model (ViVa), a novel approach that repurposes a pretrained video generator as a value function for robotic reinforcement learning. By leveraging the spatiotemporal priors learned from large-scale video corpora, our model captures rich dynamics about how scenes evolve over time. Taking the current observation together with robot proprioception as input, ViVa jointly predicts future proprioception and a scalar value for the current state (Figure 1). Grounding value estimation in anticipated embodiment dynamics enables ViVa to incorporate predictive structure beyond static snapshots, intrinsically coupling value with foresight. This design provides more reliable value signals for advantage computation, leading to improved policy optimization in robotic manipulation tasks. We integrate our video-grounded value model into the RECAP pipeline, replacing the original VLM-based value function. For the challenging real-world task, box assembly, our approach demonstrates clear superiority over prior methods. Qualitative analysis across all three tasks further confirms that ViVa produces more reliable value signals, accurately tracking task progress and detecting execution errors. By leveraging spatiotemporal priors learned from video corpora, ViVa also generalizes to novel objects where previous methods struggle, demonstrating that video-generative value models offer a scalable path toward real-world robotic learning. We highlight the main contributions of this paper below: โขWe identify that value estimation in robotic reinforcement learning is fundamentally a future anticipation problem. Video generative models, trained to capture spatiotemporal evolution, offer a more natural foundation for value learning than discriminative VLMs trained on static data. โขWe introduce ViVa, a video-generative value model that intrinsically couples value with foresight by jointly predicting future embodiment dynamics alongside the current value. 2 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning โขViVa achieves substantial gains on box assembly in both success rate and throughput. Qualitative analysis across all three tasks confirms more reliable value signals and strong generalization to novel objects. 2. Related Works 2.1. Value Functions in Robot Learning Value functions play a fundamental role in reinforcement learning for robotic manipulation by providing learning signals that guide policy improvement under delayed and sparse feedback (Ross et al., 2011; Sutton et al., 1998). Early works explored learning value or Q-functions from demonstrations and offline datasets, including calibrated Q-learning and offline reinforcement learning approaches for manipulation tasks (Huang et al., 2025; Kalashnikov et al., 2018; Lampe et al., 2024; Levine et al., 2020; Luo et al., 2024; Mandlekar et al., 2020; Mendonca et al., 2023; Sharma et al., 2023). Subsequent studies incorporated value estimation into end-to-end robotic learning pipelines using policy-gradient methods such as PPO and REINFORCE, as well as task-specific formulations such as time-to-completion prediction, demonstrating that accurate value modeling is essential for long-horizon robotic behaviors (Ghasemipour et al., 2025; Zhai et al., 2025). The success of VLA models (Cheang et al., 2024; Kim et al., 2024; Li et al., 2024; Liu et al., 2024; OโNeill et al., 2024; Team et al., 2024) has sparked interest in extending VLMs beyond policy learning to value estimation (Frans et al., 2025; Ma et al., 2023, 2024). GVL (Ma et al., 2024) demonstrates that VLMs can provide transferable per-frame value estimates from visual trajectories by framing value prediction as a temporal ordering problem over shuffled video frames, enabling progress-aware learning across diverse manipulation tasks. Building on this direction, the recent frameworkํ * 0.6 (Intelligence et al., 2025) incorporates VLM-based value functions into reinforcement learning pipelines, using them to estimate state-action advantages for advantage-conditioned policy refinement. This integration yields substantial gains in challenging long-horizon tasks such as laundry folding and espresso making, demonstrating that value model quality directly translates to policy improvement in real-world settings. These works demonstrate the feasibility of VLM-based value estimation for robotic reinforcement learning. Existing approaches rely on discriminative VLMs trained on static imageโtext data, capturing temporal dy- namics only implicitly. This limits value estimation to individual frames without explicit modeling of how physical interactions evolve over time. Such limitations motivate leveraging video generative models that learn spatiotemporal dynamics directly from large-scale video data, offering a natural foundation for value estimation in long-horizon tasks. 2.2. Video Generation Models for Robot Manipulation Video generation models learn to predict future visual sequences from large-scale video data, capturing spatiotemporal priors about object motion, physical interactions, and scene evolution (Blattmann et al., 2023; Kong et al., 2024; Yang et al., 2024; Zheng et al., 2024). Unlike vision-language models trained on static imageโtext pairs, video generators are explicitly optimized to model how scenes evolve over time. Recent state-of-the-art approaches commonly adopt diffusion Transformers (Bao et al., 2023; Peebles and Xie, 2023) to model latent video distributions, enabling conditional future prediction from language instructions (Blattmann et al., 2023; Singer et al., 2022; Villegas et al., 2022) or partial observations (Ceylan et al., 2023; Qi et al., 2023). These properties make video generation models well-suited for anticipating visual dynamics. The ability to anticipate future visual outcomes has attracted increasing interest in robotics. Prior works employ video prediction as world models for planning, using generated futures to simulate action outcomes or guide decision making (Du et al., 2023; Zhou et al., 2024). Other approaches integrate video generation into policy learning pipelines, for example by extracting actions via inverse dynamics (Yang et al., 2023), conditioning policies on generated goal frames (Du et al., 2023; Zhang et al., 2025), or jointly generating video frames alongside actions (Cheang et al., 2024; Wu et al., 2023; Ye et al., 2026,). Recent studies also 3 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning Wan DiT (Diffusion Transformer) Conditioning Frames (clean) Target Frames (noised โ denoised) scalar value (0~1) future proprio (14-dim) injected proprio Repeat & Truncate Broadcast injected value Architecture of ViVa scalar vlaue Input Injection current proprio (14-dim) ํ ํ ํ ํ ํ ํ ํ ํ 1 ํ ํ ํ 2 ํ ํ ํ 3 ํ ํ ํ+ํ ํ ํ ํ ํ ํ ํ+ํ ํ ํ ํ decode Figure 2: Overall architecture of ViVa. Left: Current robot proprioception and scalar value are mapped to latent frames via repeat padding and broadcast operations. Right: The injected latents form a unified sequence in which current observations (blank token, proprioception, and multi-view images) serve as clean conditioning frames, while future proprioception and value are noisy target frames. The diffusion Transformer denoises these targets conditioned on the clean prefix, jointly predicting the future embodied state and a scalar value defined as the normalized return. explore human-to-robot transfer by synthesizing humanโobject interaction videos (Bharadhwaj et al., 2024; Kareer et al., 2025; Zhao et al., 2025). Despite these advances, existing methods primarily leverage video generation to produce or guide actions. In contrast, we investigate a complementary role for video generation: value estimation. Predicting future dynamics implicitly encodes signals of task progress, suggesting that video models can assess whether interac- tions move toward successful completion. Based on this insight, we propose ViVa, a video-generative value model that repurposes a pretrained video generator to predict scalar values, grounding value estimation in anticipated embodiment dynamics. 3. Method 3.1. Problem Formulation We formalize robotic manipulation as a Markov decision process (MDP) defined by the tuple(ํฎ,ํ,ํฏ ,โ,ํพ), whereํฎis the state space,ํthe action space,ํฏ :ํฎ รํโํฎthe transition dynamics,โ :ํฎ รํโ Rthe reward function, andํพ โ [0, 1]the discount factor. In practice, the full state is not directly observable; instead, at each time stepํกthe agent receives an observationo ํก โํช(e.g., multi-view RGB images) and its proprioceptive stateq ํก โํฌ(e.g., joint positions and velocities). We denote the joint observation asx ํก = (o ํก ,q ํก ), which serves as a proxy for the underlying state. The agent selects actions according to a policyํ(a ํก | x ํก ), and the environment transitions to a new observation according to the dynamicsํ(x ํก+1 | x ํก ,a ํก ). A trajectoryํof lengthํis a sequenceํ = (x 0 ,a 0 ,x 1 ,a 1 ,...,x ํ ). The probability of a trajectory under policy ํ is ํ ํ (ํ) = ํ(x 0 ) ํโ1 โ๏ธ ํก=0 ํ(a ํก | x ํก )ํ(x ํก+1 | x ํก ,a ํก ). The reward function is denoted byํ ํก = ํ(x ํก ,a ํก ), and the return is the cumulative rewardํ (ํ) = โ๏ธ ํ ํก=0 ํ ํก . 4 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning The goal of RL is to maximize the expected return ํฝ(ํ) = E ํโผํ ํ [ํ (ํ)] = E ํโผํ ํ [๏ธ ํ โ๏ธ ํก=0 ํ ํก ]๏ธ . In this work, we focus on learning the value function, which estimates the expected future return from a given observation. For a policy ํ, the value function is defined as ํ ํ (x ํก ) = E ํโผํ ํ [๏ธ ํ โ๏ธ ํ=ํก ํ ํ โ โ x ํก ]๏ธ . Rather than learning a policy, our objective is to learn a model that accurately predictsํ ํ (x ํก )from the current observation. However, directly estimating long-horizon returns from a single image is challenging due to partial observability and the need to reason about future dynamics. To address this, we propose a video-generative value model that imagines future states to ground value predictions in anticipated embodiment dynamics. 3.2. Overall Architecture We build our video-generative value model upon Wan2.2 (Wan et al., 2025), a pretrained video diffusion Transformer that originally generates future frames conditioned on an initial image and text. To adapt it for value estimation, we extend its input and output modalities via latent injection (Agarwal et al., 2025; Liang et al., 2025), without modifying the core architecture. The overall architecture of ViVa is illustrated in Figure 2. Latent encoding of modalities. All input and output modalities are mapped to latent frames of shape (ํป โฒ ,ํ โฒ ,ํถ โฒ ), whereํป โฒ ,ํ โฒ are the spatial dimensions after VAE downsampling andํถ โฒ is the latent channel dimension. We use a pretrained spatiotemporal VAE to encode images: each camera viewo ํ ํก is independently compressed into a latent framez o ํ ํก . For low-dimensional vectors such as the proprioceptive stateq ํก and the scalar valueํฃ ํก , we design specialized injection procedures. Both are first normalized to[โ1, 1]to match the latent space statistics. The proprioceptive stateq ํก is embedded via repeat-padding: we repeat its elements to match the latent frame sizeํป โฒ ํ โฒ ํถ โฒ and reshape to(ํป โฒ ,ํ โฒ ,ํถ โฒ ), producingz q ํก . The scalar valueํฃ ํก is embedded via broadcast: we set every element of a latent frame to the same normalized value, yielding z ํฃ ํก . Latent sequence during training. During training, we assemble a fixed-length sequence of latent frames that includes both conditioning and target frames. Let ํพ denote a fixed prediction horizon. The sequence is: [z blank , z q ํก , z o 1 ํก , z o 2 ํก , z o 3 ํก , z q ํก+ํพ , z ํฃ ํก ], wherez blank is a zero-initialized placeholder required by the causal VAE. The first five frames (blank, current proprioception, and current images) serve as clean conditioning, while the remaining two frames (future proprioceptionz q ํก+ํพ and valuez ํฃ ํก ) are corrupted with Gaussian noise at a randomly sampled levelํ. The denoiser ํท ํ learns to recover the clean targets from the noisy ones, conditioned on the clean prefix. Latent sequence during inference. At inference time, only the conditioning frames are available. We encode the current observations (images and proprioception) into their respective latent frames, form the same prefix [z blank ,z q ํก ,z o 1 ํก ,z o 2 ํก ,z o 3 ํก ] , and run reverse diffusion to generate the target frames ห z q ํก+ํพ and ห z ํฃ ํก . The predicted valueหํฃ ํก is obtained by averaging all elements of ห z ํฃ ํก and rescaling from[โ1, 1]back to[0, 1]. To recover the future proprioceptive state ห q ํก+ํพ , we apply the inverse of the repeat-padding injection: flatten ห z q ํก+ํพ , split into consecutive chunks of size equal to the original proprioception dimension, average each chunk, and rescale to the original range. Training objective. We adopt the flow matching formulation as in Wan2.2 (Wan et al., 2025). Letz 0 denote a 5 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning clean latent frame (eitherz q ํก+ํพ orz ํฃ ํก ), and letz 1 โผํฉ(0,I)be a Gaussian noise latent of the same shape. We construct a linear interpolation path z ํ = (1โ ํ)z 0 + ํz 1 , ํ โ [0, 1]. The modelํฃ ํ (z ํ ;ํ,c)is trained to predict the constant velocityz 1 โ z 0 along this path. The overall objective is a weighted combination: โ = ํ prop E z q 0 โผํ data ,z 1 โผํฉ(0,I),ํโผํฐ[0,1] [๏ธ โํฃ ํ (z q ํ ;ํ,c)โ (z 1 โ z q 0 )โ 2 2 ]๏ธ + ํ val E z ํฃ 0 โผํ data ,z 1 โผํฉ(0,I),ํโผํฐ[0,1] [๏ธ โํฃ ํ (z ํฃ ํ ;ํ,c)โ (z 1 โ z ํฃ 0 )โ 2 2 ]๏ธ . wherez q ํ andz ํฃ ํ are modality-specific interpolated latents,cdenotes the clean conditioning frames, and ํ โผ ํฐ[0, 1]is the flow time step. We also experimented with jointly predicting future visual latents, but observed a degradation in value estimation accuracy. We hypothesize that this is due to the inherent difficulty mismatch between the two tasks: visual generation requires capturing high-dimensional spatial structure, while the value latent has a much simpler structure and is more susceptible to interference from the visual reconstruction objective during joint optimization. By treating all modalities as latent frames, our architecture repurposes a powerful video generator for value estimation while preserving its spatiotemporal priors. The inclusion of future proprioceptive prediction serves two purposes: it forces the model to internalize the robotโs own dynamics, which is essential for tasks requiring precise limb coordination, and it provides an implicit measure of motion that complements visual cues for value estimation. By jointly reasoning from visual observations and predicting embodiment dynamics, our model captures a richer notion of how the task state evolves, enabling more accurate credit assignment in long-horizon manipulation tasks. 3.3. Reward Definition and Value Training We now define the learning targets for our video-generative value model. Each episode in the training data is annotated with a binary success label indicating the final task outcome. For an episode of lengthํ, we define the step-wise reward ํ ํก to encode both temporal progress and completion status: ํ ํก = โง โช โช โช โจ โช โช โช โฉ 1 ํ , if ํก < ํ, 0, if ํก = ํ and success, 1, if ํก = ํ and failure, (1) whereํก = 1,...,ํ. Under this formulation, the cumulative returnํบ ํก = โ๏ธ ํ ํ=ํก ํ ํ provides a discriminative supervision signal that distinguishes outcomes through distinct value ranges: ํบ ํก = โง โจ โฉ ํโํก ํ ,if success, ํโํก ํ + 1, if failure. (2) Under this formulation,ํบ ํก reflects normalized task progress within[0, 1)for successful episodes, while failed episodes are shifted to[1, 2)by the terminal penalty. This ensures a constant margin of1.0between outcomes at any temporal stage, effectively resolving the ambiguity between progress and failure in value estimation. The returnํบ ํก serves as the supervision signal for the value latentz ํฃ ํก , which is treated as the clean target in the flow matching objective described in Sec. 3.2. This formulation provides a consistent and outcome-aware supervision signal across episodes of varying lengths. By jointly predicting the return and future proprioception, the model learns to capture both task-level integrity and the robotโs embodied dynamics, effectively grounding 6 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning S h i r t F o l d i n g B o x P a c k a g i n g P a p e r O r g a n i z i n g Figure 3: Illustration of the three real-world tasks. For each task, we show the initial state (left), key intermediate stages (middle), and the final successful state (right). value estimation in anticipated embodiment evolution. 4. Experiments 4.1. Tasks and Data Collection We collect demonstration data and conduct training on three real-world tasks: shirt folding, box packaging, and toilet paper organization. Corresponding demonstration videos for each task are provided in the supplementary materials. These tasks are summarized below, with sequence illustrations in Figure 3. Shirt folding. This task evaluates dual-arm coordination for manipulating highly deformable textiles. The robot must flatten a garment on the table and execute a coordinated sequence: folding the sleeves and sides inward, followed by a longitudinal fold and a final cross-fold. Success requires the shirt to be neatly folded within a designated zone within 200 seconds. Failure occurs if severe entanglement damages the garment or the structure of the fold collapses. Box packaging and assembly. This task evaluates long-horizon dual-arm coordination through a multi-stage manipulation sequence. The robot picks a target item, places it into a partially formed cardboard box, then folds the side flaps and closes the lid. Success requires the item to be fully enclosed in a structurally sound box with all tabs interlocked within 300 seconds. Failure occurs if the item is dropped, the box is damaged, or the box cannot be fully sealed. Toilet paper organization. This task evaluates precise, multi-stage manipulation of flexible paper. The robot must grasp and tear off a single sheet, discarding it into a receptacle, then collaboratively rewind the remaining loose end until it is flush with the roll. Finally, a sealing sticker is applied to secure the end. Success requires completing the tear, disposal, and sealing within 300 seconds. Excessive tearing or failure to secure the sticker is recorded as a failure. 4.2. Experimental Setup Baselines. We compare our approach against several representative baselines. The pre-trained VLA models ํ 0.5 (Intelligence et al., 2025) and Gigabrain-0 (Team et al., 2025) are used as imitation learning baselines without RL fine-tuning. To examine the effect of value model design within the RECAP (Intelligence et al., 2025) framework, we implement two variants using Gigabrain-0 as the base policy: one equipped with a conventional VLM-based value function, and another where the value function is replaced by our proposed 7 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning corner misalignment during insertion attempt box partially overhanging, fall risk Timestep Figure 4: Value estimation during a box-assembling task. The plot compares value estimates from a VLM-based function and our ViVa model over time. Two failure events are highlighted (blue-shaded). The VLM-based value remains largely insensitive to these errors, suggesting overfitting to successful trajectories. In contrast, ViVa exhibits sharp drops precisely when these mistakes occur, demonstrating its sensitivity to suboptimal actions through grounding in anticipated embodiment dynamics. video-generative model ViVa. All other components remain identical, enabling a controlled comparison of value model designs. Implementation details. For the VLM-based value model, we follow the same design asํ * 0.6 (Intelligence et al., 2025), formulating value estimation as a 201-way classification problem over discretized return bins. Both this baseline and our ViVa-based variant are trained within the identical RECAP pipeline on a mixture of demonstration data from all three tasks to ensure fair comparison. All models are trained for a single epoch with a batch size of 192. The prediction horizonํพis set to 50 for all tasks, aligning with the default horizon used for advantage estimation in the RECAP framework. For ViVa, the loss weights for future proprioception and value prediction are set toํ prop = 1.0andํ val = 0.5, respectively. During inference, we use 1 denoising step for ViVa with DDIM sampling. All experiments are conducted on 8 NVIDIA A800 GPUs. standard sleeve-tucking motions standard placement motions Timestep Figure 5: Value estimation during a shirt-folding task. The plot compares value estimates from a VLM-based function and our ViVa model over time. The VLM-based value exhibits erratic drops (orange-shaded). It remains largely flat throughout the episode, failing to reflect the gradual progress toward successful completion. ViVa, by contrast, maintains a stable value progression, accurately reflecting continuous task progress. 8 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning roll alignment label application Timestep Figure 6: Value estimation during toilet paper organization. The plot compares value estimates from a VLM- based function and ViVa over time. Two key milestones are highlighted (blue-shaded): roll alignment and label application. ViVa exhibits clear value increases precisely at these milestones, yielding a smooth trajectory that tracks task progress. The VLM-based value, in contrast, remains largely insensitive to these events. 4.3. Qualitative Analysis We evaluate our method on two settings: in-domain performance on the training tasks and out-of-domain generalization to novel objects. This section focuses on qualitative visual comparisons, and all corresponding videos used in this analysis are provided in the project page. In-domain performance. We first analyze the behavior of the learned value functions on trajectories from the training tasks. Figures 4, 5, and 6 visualize the temporal evolution of value estimates produced by the VLM-based value function and our proposed ViVa model during representative task executions. Figure 4 shows a box assembly trajectory where the robot exhibits corner misalignment during insertion and a partial overhang. ViVa produces sharp value drops precisely when these suboptimal behaviors occur, demon- strating its ability to detect deviations from ideal execution through reasoning about anticipated embodiment dynamics. The VLM-based value, in contrast, increases monotonically throughout the trajectory and remains insensitive to these intermediate errors. This behavior suggests overfitting, where the model associates temporal progression with increasing value without evaluating execution quality. Figure 5 illustrates the value evolution during a shirt-folding trajectory. ViVa maintains a smooth and steadily increasing value trend that closely tracks task progress from the initial configuration to the final folded state. The VLM-based value instead exhibits erratic fluctuations during routine motions while remaining largely flat over long portions of the episode, failing to reflect the gradual nature of task progression. Figure 6 visualizes value trajectories during toilet paper organization, highlighting two milestones: aligning the rollโs loose end and applying the label. ViVa shows distinct value increases precisely at these stages, reflecting sensitivity to meaningful progress. In contrast, the VLM-based value remains largely flat, failing to register key events, and suffers from high-frequency fluctuations unrelated to task state. Together, these results clearly highlight a fundamental difference between the two value modeling paradigms. The VLM-based approach relies on static visual recognition, producing predictions that are either monotonically insensitive or completely disconnected from task progress. In contrast, ViVa leverages the spatiotemporal priors of video generation models to ground value estimation directly in anticipated embodiment dynamics. Out-of-domain generalization. We further evaluate the generalization capability of both value models on an 9 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning grasp & lift pants fold one leg to center Timestep final placement waistband fold Figure 7: Out-of-domain generalization to unseen objects (pants folding). The plot compares value estimates from a VLM-based function and our ViVa model. Four key execution milestones are highlighted (blue-shaded). ViVa exhibits sharp value increases at these milestones, with a smooth and consistently rising trajectory. The VLM-based value, in contrast, misses the first and fourth milestones, shows a slight downward trend in the first half, and exhibits high-frequency fluctuations throughout. unseen object category: folding a pair of pants, which was excluded from the training data. Figure 7 visualizes the value estimates throughout a successful execution. ViVa exhibits remarkably sharp value increases precisely aligned with the four key execution milestones (lifting, leg folding, waistband folding, and final placement), yielding a smooth, monotonically rising trajectory that faithfully tracks task progress on this novel object. In contrast, the VLM-based value consistently fails to detect the first and fourth milestones, displays a counter-intuitive downward trend during the initial phase despite ongoing progress, and suffers from high-frequency fluctuations throughout the episode. This degradation clearly reveals that the VLM-based function relies heavily on superficial visual patterns memorized from the training distribution, inevitably struggling to transfer to novel scenarios. In contrast, by leveraging the spatiotemporal priors inherent in video generation, ViVa maintains robust value estimation even for unseen objects, demonstrating a genuine understanding of task dynamics beyond static visual features. 4.4. Real-World Experiments To validate the practical impact of our approach, we conduct real-world experiments on the box assembly task. Due to the long rollout cycles required by the RECAP pipeline, each policy evaluation incurs substantial time and hardware cost. Given these constraints and limited resources, we focus our real-robot evaluation on the most complex task in our benchmark: box assembly. Table 1: Real-world results on the box assembly task. Throughput is defined as the number of successfully completed tasks per hour. RECAP (ViVa) achieves the highest performance on both metrics. MethodSuccess (%)Throughput ํ 0.5 428 Gigabrain-05310 RECAP (VLM)5811 RECAP (ViVa)7314 Table 2: Computational cost comparison. Training time (GPUยทdays) and inference time (s) are reported. ViVa achieves faster training and inference than the VLM-based baseline. ModelTraining (GPUยทd)Inference (s) VLM-based60.32 Vid-based30.11 ViVa40.18 10 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning sleeve-tucking motion Timestep left side fold delivery to target zone right side fold Figure 8: Ablation on video generation backbone. The plot compares VLM-based and video-based value functions during shirt folding with identical structure. The video-based variant shows steady progression with clear value increases at key milestones (blue-shaded). In contrast, the VLM-based value exhibits erratic fluctuations with no discernible trend, failing to track task progress or critical events. Table 1 reports both task success rate and throughput. The results reveal a clear gap between imitation-only policies and value-guided methods. The pre-trained VLA modelsํ 0.5 and Gigabrain-0 achieve success rates of 42% and 53%, respectively, highlighting the difficulty of the task under pure imitation learning. Incorporating value-based refinement through the RECAP pipeline improves performance, with the VLM-based value function increasing success to 58%. Replacing the VLM-based value with our proposed ViVa further boosts the success rate to 73% and achieves the highest throughput of 14 successful tasks per hour. These results indicate that grounding value estimation in predicted embodiment dynamics provides more reliable guidance for policy improvement, leading to both higher task success and more efficient execution. 4.5. Efficiency Comparison Table 2 compares the computational cost of three value model variants. The VLM-based baseline follows the lightweight design ofํ * 0.6 but incurs the highest training cost of 6 GPUยทdays and inference latency of 0.32 seconds per frame, primarily due to its SigLIP (Zhai et al., 2023) visual encoder. The video-based variant predicting value alone achieves the fastest inference at 0.11 seconds and the lowest training cost of 3 GPUยทdays, yet omitting future proprioception compromises prediction accuracy. Our full ViVa model strikes a favorable balance, training in 4 GPUยทdays, 1.5ร faster than the VLM baseline, while running at 0.18 seconds per frame. The additional proprioceptive prediction enriches the learning signal with minimal computational overhead. 4.6. Ablation Studies Effect of video generator backbone. Figure 8 compares value estimates from a VLM-based value function and a video-based variant during a shirt-folding trial, while keeping the same input-output formulation. The video-based variant shows a clear progression trend, with value increases aligned with key manipulation stages including sleeve-tucking, left and right folds, and final placement. In contrast, the VLM-based value exhibits erratic fluctuations with no consistent trend, failing to reflect overall task progress or capture critical events. This contrast highlights the role of spatiotemporal priors for value estimation. Video generation backbones pretrained on large-scale video data encode temporal structure about how scenes and actions evolve, enabling the value function to track meaningful stage transitions. In contrast, the VLM-based model operates on static visual representations and lacks explicit temporal modeling, making it more sensitive to incidental 11 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning box misaligned after lid closure swinging motion during lifting Timestep Figure 9: Ablation study on future proprioception prediction. The plot compares value estimates from ViVa without future proprioception (w/o propri) and the full ViVa model during a box assembly trial. Two anomalous behaviors are highlighted (blue-shaded). The full ViVa shows clear value drops at these events, indicating sensitivity to suboptimal actions enabled by future proprioception prediction. In contrast, ViVa (w/o propri) remains largely insensitive to these deviations. appearance changes rather than true task progression. Even without future proprioception prediction, the video-based backbone already provides a stronger foundation for value learning, suggesting that spatiotemporal understanding is critical for accurate task progress estimation. Effect of future proprioception prediction. Figure 9 compares value estimates from two ViVa variants during a box assembly trial. Two anomalous behaviors are highlighted: misalignment after lid closure and instability during lifting. The full ViVa model shows clear value drops at these events, indicating sensitivity to suboptimal actions. In contrast, ViVa without future proprioception prediction remains largely insensitive, producing a smooth value trajectory that fails to reflect these errors. This contrast highlights the importance of modeling future embodiment dynamics. By predicting how the robot state will evolve, the model learns an internal representation of expected physical progression, allowing value estimates to decrease when execution deviates from this trajectory. Without this objective, the model relies primarily on visual observations and may overlook subtle but critical interaction errors. As further shown in Figure 10, the full ViVa reliably detects diverse fine-grained manipulation failures, including missed grasps, uneven force application, premature release, and asynchronous lifting. These results confirm that future proprioception prediction is essential for grounding value estimation in task-relevant physical dynamics. Effect of Prediction Horizon. We investigate the impact of the prediction horizonํพon value estimation missed grasp misaligns the box uneven force misaligns the box premature release prevents box closure improper manipulation tilts the box asynchronous lifting tilts the box Figure 10: Detection of subtle manipulation errors by ViVa. The model reliably identifies execution flaws during box assembly, producing clear value drops at the moments they occur. This sensitivity to fine-grained physical deviations demonstrates ViVaโs ability to ground value estimation in anticipated embodiment dynamics. 12 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning edge insertion position adjustment Timestep Figure 11: Effect of prediction horizon on value estimation. The plot compares value trajectories from ViVa models trained with different prediction horizonsํพ = 25, 50, 75during a box assembly trial. Two critical phases are highlighted (blue-shaded): position adjustment and edge insertion. Whileํพ = 50maintains smooth and stable value estimates throughout, bothํพ = 25andํพ = 75exhibit noticeable fluctuations. Notably, the ํพ = 75model fails to capture the edge insertion failure, with its value estimates remaining insensitive to this critical error. These results indicate that an intermediate horizon best balances temporal context and stability. quality. Figure 11 compares value trajectories from ViVa models trained withํพ = 25,50, and75during a box assembly trial. While all variants capture the overall trend of task progression, clear differences emerge in their stability and accuracy. The model with a shorter horizon (ํพ = 25) exhibits increased sensitivity, producing noticeable fluctuations during critical phases such as position adjustment. The longer horizon model (ํพ = 75) also shows instability, and notably fails to accurately register the edge insertion phase, where its value estimates become erratic and deviate from true task progress. In contrast,ํพ = 50yields the smoothest and most stable value estimates, accurately capturing both coarse task progression and fine-grained events like edge insertion. We hypothesize that overly short horizons lack sufficient future context to smooth out transient variations, while overly long horizons introduce uncertainty from predicting too far ahead, both of which degrade value signal quality. Based on these observations, we adoptํพ = 50as the default prediction horizon for all experiments. 5. Conclusion and Future Work In this work, we introduced ViVa, a video-generative value model that grounds value estimation in predicted embodiment dynamics. Built on a pretrained video backbone, ViVa learns temporally aware representations that reliably track task progress and detect execution errors. ViVa substantially outperforms prior methods on box assembly, with qualitative analysis across three tasks confirming its more reliable value signals. These findings highlight the importance of spatiotemporal priors and embodiment-aware prediction for value learning in manipulation. Larger-scale real-robot experiments were precluded by the long RECAP rollout cycles and time constraints, which we leave for future work. 13 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning References [1]Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025. 5 [2]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. 2 [3]Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22669โ22679, 2023. 3 [4]Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, and Sean Kirmani. Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation. arXiv preprint arXiv:2409.16283, 2024. 4 [5] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3 [6]Duygu Ceylan, Chun-Hao P Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. In Proceedings of the IEEE/CVF international conference on computer vision, pages 23206โ23217, 2023. 3 [7] Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158, 2024. 3 [8]Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spa- tialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455โ14465, 2024. 2 [9] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with ad- vanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. 2 [10] Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, et al. Video language planning. arXiv preprint arXiv:2310.10625, 2023. 3 [11]Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in neural information processing systems, 36:9156โ9172, 2023. 3 [12]Kevin Frans, Seohong Park, Pieter Abbeel, and Sergey Levine. Diffusion guidance is a controllable policy improvement operator. arXiv preprint arXiv:2505.23458, 2025. 3 [13]Seyed Kamyar Seyed Ghasemipour, Ayzaan Wahid, Jonathan Tompson, Pannag Sanketi, and Igor Mor- datch. Self-improving embodied foundation models. arXiv preprint arXiv:2509.15155, 2025. 3 [14]Dongchi Huang, Zhirui Fang, Tianle Zhang, Yihang Li, Lin Zhao, and Chunhe Xia. Co-rft: Efficient fine-tuning of vision-language-action models through chunked offline reinforcement learning. arXiv preprint arXiv:2508.02219, 2025. 3 14 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning [15]Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International conference on machine learning, pages 9118โ9147. PMLR, 2022. 2 [16]Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Jared DiCarlo, et al.ํ * 0.6 : a vla that learns from experience. arXiv preprint arXiv:2511.14759, 2025. 2, 3, 7, 8 [17]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al.ํ 0.5 : a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. 1, 7 [18]Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on robot learning, pages 651โ673. PMLR, 2018. 3 [19]Simar Kareer, Karl Pertsch, James Darpinian, Judy Hoffman, Danfei Xu, Sergey Levine, Chelsea Finn, and Suraj Nair. Emergence of human to robot transfer in vision-language-action models. arXiv preprint arXiv:2512.22414, 2025. 4 [20] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. 1, 3 [21]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 3 [22] Thomas Lampe, Abbas Abdolmaleki, Sarah Bechtle, Sandy H Huang, Jost Tobias Springenberg, Michael Bloesch, Oliver Groth, Roland Hafner, Tim Hertweck, Michael Neunert, et al. Mastering stacking of diverse shapes with large-scale iterative reinforcement learning on real robots. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 7772โ7779. IEEE, 2024. 3 [23]Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020. 3 [24] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. 2 [25] Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024. 3 [26]Wei Li, Renshan Zhang, Rui Shao, Jie He, and Liqiang Nie. Cogvla: Cognition-aligned vision-language- action model via instruction-driven routing & sparsification. arXiv preprint arXiv:2508.21046, 2025. 1 [27]Xinghang Li, Peiyan Li, Long Qian, Minghuan Liu, Dong Wang, Jirong Liu, Bingyi Kang, Xiao Ma, Xinlong Wang, Di Guo, et al. What matters in building visionโlanguageโaction models for generalist robots. Nature Machine Intelligence, pages 1โ15, 2026. 1 [28] Junbang Liang, Pavel Tokmakov, Ruoshi Liu, Sruthi Sudhakar, Paarth Shah, Rares Ambrus, and Carl Vondrick. Video generators are robot policies. arXiv preprint arXiv:2508.00795, 2025. 5 15 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning [29]Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024. 3 [30]Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. Serl: A software suite for sample-efficient robotic reinforcement learning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 16961โ16969. IEEE, 2024. 3 [31] Yecheng Jason Ma, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. Liv: Language- image representations and rewards for robotic control. In International Conference on Machine Learning, pages 23301โ23320. PMLR, 2023. 3 [32]Yecheng Jason Ma, Joey Hejna, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, et al. Vision language models are in-context value learners. In The Thirteenth International Conference on Learning Representations, 2024. 2, 3 [33] Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Dieter Fox. Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4414โ4420. IEEE, 2020. 3 [34]Andrรฉs Marafioti, Orr Zohar, Miquel Farrรฉ, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, et al. Smolvlm: Redefining small and efficient multimodal models. arXiv preprint arXiv:2504.05299, 2025. 2 [35] Russell Mendonca, Shikhar Bahl, and Deepak Pathak. Alan: Autonomously exploring robotic agents in the real world. arXiv preprint arXiv:2302.06604, 2023. 3 [36]Abby OโNeill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892โ6903. IEEE, 2024. 3 [37]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195โ4205, 2023. 3 [38]Qiaosong Qi, Le Zhuo, Aixi Zhang, Yue Liao, Fei Fang, Si Liu, and Shuicheng Yan. Diffdance: Cascaded human motion diffusion model for dance generation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1374โ1382, 2023. 3 [39] Stรฉphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627โ635. JMLR Workshop and Conference Proceedings, 2011. 3 [40] Ranjan Sapkota, Yang Cao, Konstantinos I Roumeliotis, and Manoj Karkee. Vision-language-action (vla) models: Concepts, progress, applications and challenges. arXiv preprint arXiv:2505.04769, 2025. 1 [41]Archit Sharma, Ahmed M Ahmed, Rehaan Ahmad, and Chelsea Finn. Self-improving robots: End-to-end autonomous visuomotor reinforcement learning. arXiv preprint arXiv:2303.01488, 2023. 3 [42]Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022. 3 16 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning [43]Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. 2, 3 [44]GigaBrain Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, et al. Gigabrain-0: A world model-powered vision-language-action model. arXiv preprint arXiv:2510.19430, 2025. 7 [45]GigaBrain Team, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Jie Li, Jindi Lv, Jingyu Liu, Lv Feng, et al. Gigabrain-0.5 m*: a vla that learns from world model-based reinforcement learning. arXiv preprint arXiv:2602.12099, 2026. 1 [46] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. 3 [47]Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Moham- mad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399, 2022. 3 [48] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 5 [49] Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. arXiv preprint arXiv:2312.13139, 2023. 3 [50]Mengjiao Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. arXiv preprint arXiv:2310.06114, 1(2):6, 2023. 3 [51] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 3 [52]Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigaworld-policy: An efficient action-centered worldโaction model. arXiv preprint arXiv:2603.17240, 2026. 3 [53]Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026. 3 [54]Shaopeng Zhai, Qi Zhang, Tianyi Zhang, Fuxian Huang, Haoran Zhang, Ming Zhou, Shengzhe Zhang, Litao Liu, Sixu Lin, and Jiangmiao Pang. A vision-language-action-critic model for robotic real-world reinforcement learning. arXiv preprint arXiv:2509.15937, 2025. 3 [55] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre- training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975โ11986, 2023. 11 [56] Hongyin Zhang, Pengxiang Ding, Shangke Lyu, Ying Peng, and Donglin Wang. Gevrm: Goal-expressive video generation model for robust visual manipulation. arXiv preprint arXiv:2502.09268, 2025. 3 17 ViVa: A Video-Generative Value Model for Robot Reinforcement Learning [57]Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, and Xiaoguang Han. Taste- rob: Advancing video generation of task-oriented hand-object interaction for generalizable robotic manipulation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 27683โ 27693, 2025. 4 [58]Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 3 [59] Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, and Chuang Gan. Robodreamer: Learning compositional world models for robot imagination. arXiv preprint arXiv:2404.12377, 2024. 3 [60]Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025. 2 [61]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165โ2183. PMLR, 2023. 1, 2 18