Paper deep dive
Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation
Chenjian Gao, Zhihao Hu, Jianqi Ma, Jun Zhang, Weidong Zhang, Tianfan Xue
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at this https URL
Tags
Links
- Source: https://arxiv.org/abs/2609.37925v1
- Canonical: https://arxiv.org/abs/2609.37925v1
Trouble viewing inline? Open PDF directly →
Full Text
36,053 characters extracted from source content.
Expand or collapse full text
Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation Chenjian Gao Affiliation: MMLab, The Chinese University of Hong Kong Zhihao Hu Jianqi Ma Affiliation: Tencent AIPD*Equal contribution†Corresponding authors Jun Zhang Weidong Zhang Affiliation: Tencent AIPD*Equal contribution†Corresponding authors Tianfan Xue Abstract Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD/. Figure 1: Long-horizon autoregressive video generation. Trained on 5-second rollouts, RMD maintains high visual quality over 500-second rollouts (100×100× the training horizon). Notably, generation relies solely on a fully sliding context window, without fixed anchors, additional explicit memory mechanisms, or modifications to the generator architecture. 1 Introduction Figure 2: Comparison of real-score predictions under video-level DMD and RMD. (a) The autoregressive rollout accumulates visual artifacts. (b) Video-level DMD scores the sequence jointly, causing the real-score prediction to retain these historical artifacts. (c) RMD scores the current chunk independently, providing a clean target regardless of the degraded context. Due to high computational and memory costs, video diffusion models are typically trained on short clips (Wan et al., 2025; Huang et al., 2026a). However, many real-world applications demand much longer sequences, including autonomous-driving simulation, interactive and embodied world models, and physics-grounded visual simulation (Zhang et al., 2025; Bruce et al., 2024; Liu et al., 2024). Autoregressive (AR) models naturally support this by extending video generation chunk by chunk (Yin et al., 2025; Huang et al., 2026a). Specifically, the diffusion process for each new chunk conditions on previously generated frames. As the video grows, a sliding context window ensures the model only attends to the most recent history, keeping the generation cost bounded. However, this sequential process introduces an inherent challenge: each generated chunk inevitably contains minor prediction errors, which then become the context for future steps. Consequently, errors barely noticeable in a short sequence can compound and severely degrade visual quality over a long rollout. To mitigate error accumulation, self-rollout training exposes the generator to its own imperfect predictions (Huang et al., 2026a; Zhu et al., 2026). Since a generated video rarely matches real videos perfectly, no paired ground-truth continuation exists to supervise the next steps. To overcome this lack of paired data, distribution matching distillation (DMD) is often employed (Yin et al., 2024b; Yin et al., 2025). Instead of requiring exact continuations, a bidirectional diffusion teacher evaluates the entire sequence jointly. It provides corrective feedback by matching a fake score tracking the student’s output against a real score representing the teacher’s high-quality prior. However, DMD-trained models still struggle to generate very long sequences. We attribute this to a bias introduced by the video teacher’s joint supervision. As visual artifacts accumulate over a long autoregressive rollout (e.g., the severe texture and color corruption in the background foliage in Figure 2(a)), the teacher must evaluate the new chunk alongside this flawed history. Because the teacher cannot modify past frames, enforcing a clean, natural background in the current chunk would cause an abrupt visual shift. Consequently, to preserve temporal consistency, the video-level teacher often retains these accumulated errors, which manifest as non-physical ghosting artifacts, in its real-score prediction (Figure 2(b)). The temporal consistency constraint thus fundamentally limits the teacher’s ability to correct local errors. We break this coupling with Rollout-Marginal Distillation (RMD). The generator still uses its history to produce continuations, but the teacher evaluates the generated chunk independently, without checking the history or future chunks. As shown in Figure 2(c), context-free scoring ensures the teacher provides a clean, high-fidelity real score despite the degraded history. RMD achieves this by matching the student’s chunk marginal distribution to a teacher-defined chunk distribution. To provide this supervision, we prepare the RMD teacher by adapting a video diffusion model on individual chunks extracted from videos generated by the original bidirectional model. However, because independent chunk supervision does not enforce cross-chunk relationships, generated videos might exhibit motion jitter between chunks. To resolve this, we subsequently refine the generator with a video-level DMD stage, explicitly smoothing these transitions to promote temporal coherence. Extensive experiments demonstrate that RMD achieves significantly higher visual quality than video-level DMD baselines beyond the training horizon. Our quantitative evaluation covers rollouts of approximately 60 seconds (12×12× the 5-second training horizon). In addition, Figure 1 provides a qualitative example extending to 500 seconds (100×100× the training horizon). Generation relies entirely on a fully sliding context window, without fixed anchors such as frame sinks (Yang et al., 2026; Li et al., 2026), additional explicit memory mechanisms, or changes to the generator architecture. 2 Related Work Few-step autoregressive video generation. Causal video diffusion generates frames or blocks sequentially with KV caching for low-latency streaming. CausVid converts a pretrained bidirectional video diffusion model into a causal generator and extends DMD to few-step video generation (Yin et al., 2025). Self-Forcing further performs autoregressive rollout during training, feeding generated outputs back as context and supervising the sequence with a holistic video objective (Huang et al., 2026a). Causal Forcing instead studies initialization and uses an autoregressive teacher for ODE distillation before applying Self-Forcing DMD (Zhu et al., 2026). Causal-rCM similarly combines teacher-forced consistency training with self-forced DMD refinement (Zheng et al., 2026a). Solaris introduces Checkpointed Self Forcing, which decouples serial self-rollout from parallel differentiable replay to reduce training memory (Savva et al., 2026). Together, these methods establish causal students and on-policy training pipelines. We adopt the Solaris replay mechanism directly and focus RMD on the distribution matched during on-policy DMD. Distribution matching distillation. DMD trains a few-step generator by minimizing a reverse KL divergence to the distribution of a pretrained diffusion model, using the difference between real and fake scores as generator supervision (Yin et al., 2024b). DMD2 removes the costly regression objective and improves stability through a two-time-scale update rule, adversarial learning, and multi-step training (Yin et al., 2024a). Subsequent work has combined distribution matching with adversarial learning (Lu et al., 2025), replaced KL-based matching with an optimal-transport objective (Wang et al., 2026), or coupled reverse-KL refinement with score-regularized consistency training (Zheng et al., 2026b). These developments primarily modify the divergence, regularization, or optimization of distribution matching. By contrast, RMD retains the DMD score-difference update but changes its output space from a joint rollout to the chunk marginal induced by autoregressive sampling. Long rollouts and causal supervision. LongLive improves extended AR generation through streaming long-video tuning, a finite attention window, and a persistent frame sink (Yang et al., 2026). Rolling Sink studies cache maintenance beyond the training horizon and introduces a training-free cache-maintenance strategy (Li et al., 2026). More directly related to training-time supervision, OPSD-V post-trains few-step AR generators on their inference-time trajectories while supplying the teacher with a cleaner cache that can incorporate real long-video context (Liu et al., 2026). Most closely related to our analysis, Context-Matched Distillation (CMD) observes that a bidirectional full-clip teacher can use future information unavailable to a causal student (Bandyopadhyay et al., 2026). CMD addresses this issue with a causal teacher and prefix scoring under the student’s realized rollout context. RMD takes a different route. It does not construct a teacher conditional for each self-rollout history. Instead, it uses a bidirectional diffusion prior as a chunk-distribution target, scores the rollout chunks independently, and subsequently restores joint temporal structure through video-level refinement. It therefore requires neither paired continuations for generated histories nor real long-video context. 3 Rollout-Marginal Distillation In this section, we present Rollout-Marginal Distillation (RMD), an approach designed to disentangle visual quality supervision from temporal context in autoregressive video generation. We first review video-level distribution matching distillation (DMD) and formalize how its joint scoring mechanism inherently couples the current chunk’s gradient to past and future frames (Section 3.1). To break this coupling, we introduce our rollout-marginal objective, which independently penalizes unrealistic chunks without conditioning on the imperfect context (Section 3.2). We then detail the training procedure in practice (Section 3.3). Finally, because independent scoring ignores cross-chunk relationships, we describe a subsequent video-level refinement stage that explicitly restores temporal coherence (Section 3.4). 3.1 Revisiting Video-Level Distribution Matching To understand the necessity of rollout-marginal supervision, we first examine how video-level DMD improves a causal video generator and why its joint scoring mechanism introduces a critical flaw. Given a text prompt c, let the causal generator GθG_θ produce a latent rollout =(x1,…,xT)x=(x_1,…,x_T), where each xix_i is a chunk. The distribution of the generated video x can be naturally factorized as: qθ(∣c)=∏i=1Tqθ(xi∣x<i,c).q_θ(x c)= _i=1^Tq_θ(x_i x_<i,c). (1) Following the self-rollout framework (Huang et al., 2026a; Zhu et al., 2026), the generator conditions each continuation on its own generated history x<ix_<i, exposing it to the prediction errors it will encounter at inference. Because there are no ground-truth continuations paired with these imperfect generated histories, video-level distribution matching distillation (DMD) is employed to supervise the entire student rollout jointly (Yin et al., 2024b; Yin et al., 2024a; Yin et al., 2025). At noise level τ, let τ=ατ+στϵx_τ= _τx+ _τ ε denote a noised rollout. The video-level objective minimizes the reverse KL divergence between the noised student video distribution qθ,τq_θ,τ and a pretrained teacher video distribution pτp_τ: ℒvideo(θ)=c,τ[DKL(qθ,τ∥pτ)].L_video(θ)=E_c,τ\! [D_KL\! (q_θ,τ\,\|\,p_τ ) ]. (2) The generator is updated using the difference between two diffusion scores: ∇θℒvideo≈[w(τ)Jθ()⊤(sfake(τ,τ,c)−sreal(τ,τ,c))], _θL_video \! [w(τ)J_θ(x) (s_fake(x_τ,τ,c)-s_real(x_τ,τ,c) ) ], (3) where Jθ()=∂/∂θJ_θ(x)= /∂θ is the generator Jacobian, w(τ)w(τ) is a noise-level weight, sreals_real is a frozen teacher score representing the target distribution, and sfakes_fake is an online score model trained to track the student’s distribution. The Coupling Dilemma. Equation 3 reveals a fundamental issue in applying video-level DMD to autoregressive rollouts. Both the real and fake score networks (sreals_real and sfakes_fake) take the entire sequence τx_τ as input. Consequently, the correction signal assigned to any specific chunk xix_i is coupled with its generated past (x<ix_<i) and future (x>ix_>i). If an artifact appears in the generated history, a joint score evaluation will likely penalize a sudden correction to natural colors in the current chunk, as such a transition breaks the temporal consistency of the sequence. The model is thereby forced to compromise local visual realism to accommodate an imperfect temporal context. To resolve this, we must disentangle the quality supervision of xix_i from the rest of the rollout. 3.2 Rollout-Marginal Objective Video-level DMD scores the entire sequence, tying each chunk’s quality correction to its compatibility with the surrounding content. RMD breaks this dependency by matching chunks from the causal rollouts independently, while the generator still uses its history for prediction. We model the initial and continuation chunks separately to account for their distinct latent statistics. Specifically, under the video VAE, the first latent frame encodes only the initial image without temporal compression (Wan et al., 2025; Huang et al., 2026a). This gives it completely different statistical properties compared to subsequent frames. Let qθ,1(x1∣c)q_θ,1(x_1 c) denote the initial chunk distribution. To define the continuation rollout marginal, we pool the chunks x2,…,xTx_2,…,x_T across the student’s causal rollouts for prompt c: qθ,chunk(T)(x∣c)=1T−1∑i=2Tx<i∼qθ(x<i∣c)[qθ(x∣x<i,c)].q^(T)_θ,chunk(x c)= 1T-1 _i=2^TE_x_<i q_θ(x_<i c)\! [q_θ(x x_<i,c) ]. (4) This distribution averages over both the generated histories and the continuation positions. Sampling from it requires the causal rollout, but evaluating a chunk’s realism requires neither its history nor its rollout position. At noise level τ, let p1,τp_1,τ and pchunk,τp_chunk,τ denote the teacher-defined target distributions for the initial frame and continuation chunks, respectively. Weighting these two groups by their frequency in the rollout yields the RMD objective: ℒRMD(θ)=c,τ[1TDKL(qθ,1,τ∥p1,τ)+T−1TDKL(qθ,chunk,τ(T)∥pchunk,τ)].L_RMD(θ)=E_c,τ\! [ 1TD_KL (q_θ,1,τ\,\|\,p_1,τ )+ T-1TD_KL (q_θ,chunk,τ^(T)\,\|\,p_chunk,τ ) ]. (5) Crucially, the generation process remains causal, meaning each sample xix_i is still produced conditionally from its actual on-policy history x<ix_<i. However, the supervision applied to each chunk is entirely context-free. This objective simply penalizes generated chunks that are unlikely under the teacher’s chunk distribution. By doing so, it explicitly isolates visual quality supervision from the imperfect generated history. Instead of forcing the model to repeat past artifacts to maintain temporal consistency, this marginal matching ensures the teacher guides the current chunk toward high-quality targets, regardless of the flaws in the preceding frames. Figure 3: Overview of Rollout-Marginal Training. The AR generator produces a causal rollout, sampling the initial chunk (x1x_1) at a random-step exit and continuation chunks (x2,…,xTx_2,…,x_T) at the last-step exit. Adapted score networks evaluate them independently without temporal context. The resulting gradients (g1,…,gTg_1,…,g_T) are backpropagated via differentiable causal replay. 3.3 Rollout-Marginal Training To realize this decoupled supervision in practice, our training pipeline (Figure 3) optimizes the rollout-marginal objective by first generating the causal rollout, then evaluating the initial and continuation chunks against their distinct target distributions, and finally backpropagating the isolated gradients to the generator. We initialize the AR generators from Causal Forcing’s causal ODE checkpoints (Zhu et al., 2026). Following the self-forcing framework (Huang et al., 2026a), at each training iteration, we generate a causal rollout of chunks x1,…,xTx_1,…,x_T, sampling each chunk in 4 denoising steps. To retain the temporal dynamics prior in the pretrained model, we carefully choose which denoising step to supervise for each chunk (Wu et al., 2026). Early steps in the sampling trajectory rely heavily on historical context to establish global motion, and applying independent supervision at these timesteps would disrupt the temporal consistency. Therefore, we adopt an asymmetric supervision strategy: for the initial chunk x1x_1, we compute the loss at a randomly selected intermediate exit step along the trajectory, whereas for the continuation chunks x2,…,xTx_2,…,x_T, we apply the loss only at the final, lowest-noise step of the rollout where the network refines local visual details. To score the sampled chunks, we use separate teacher targets for the initial frame and continuations. As discussed in Section 3.2, Wan’s causal VAE gives them different latent statistics. An unmodified video teacher would treat an isolated continuation chunk as the start of a video and expect an uncompressed first frame. We therefore adapt the Wan 14B video teacher (Wan et al., 2025) with LoRA (Hu et al., 2022) on continuation chunks extracted from videos generated by the original model. The adapted teacher supplies the continuation target pchunk,τp_chunk,τ in Equation 5, allowing each continuation to be scored without its generated history. For each xix_i, we add scoring noise and obtain clean predictions from its teacher and fake-score networks on the same noisy input. Let Dreal,iD_real,i and Dfake,iD_fake,i denote these two predictions. Their normalized difference gives a gradient estimate gig_i for the chunk: gi=Dfake,i−Dreal,imean|xi−Dreal,i|.g_i= D_fake,i-D_real,imean|x_i-D_real,i|. (6) To propagate gig_i to the generator parameters, we follow checkpointed self-forcing (Savva et al., 2026). The causal rollout is generated without retaining its computation graph. A differentiable replay then recomputes each supervised prediction from its sampled noisy input and generated history, allowing gig_i to be backpropagated through the generator. The first latent frame has different statistics from later latents. Once it leaves the sliding window, the generator must predict without that distinctive frame, a condition absent from short training rollouts that can cause flicker. Following Self-Forcing (Huang et al., 2026a), we mask attention to the initial block when replaying predictions in the latter half of the training window. 3.4 Refinement with Video-Level DMD Matching chunk marginals improves their appearance but does not directly constrain the transitions between them. Individually realistic chunks can still form a video with inconsistent appearance or motion. We therefore refine the AR generator with the video-level DMD objective in Equation 2. The generator still produces causal self-rollouts, but the video teacher and online fake-score network now evaluate each complete rollout jointly. Their score difference updates the generator through the same differentiable replay, supervising motion and consistency across chunks. This stage begins from a generator already trained using the rollout-marginal objective. We use smaller learning rates to limit the changes made during refinement. 4 Experiments 4.1 Experimental Setup Implementation Details. We distill the pretrained Wan 14B video diffusion model into a causal Wan 1.3B generator, initialized from Causal Forcing’s causal ODE checkpoints (Zhu et al., 2026). RMD is trained in two stages using a metadata set comprising 70,000 unique prompts sourced from Self Forcing (Huang et al., 2026a). Our models are trained under a strictly fixed 81-frame horizon, without performing longer rollouts during the training phase. All trainable networks are optimized using AdamW (Loshchilov and Hutter, 2019) (β1=0 _1=0, β2=0.999 _2=0.999, weight decay 0.010.01) with a global batch size of 8. During the rollout-marginal training stage, we alternate 5 fake-score updates for every 1 generator update. When the chunk size is 1, both the generator and fake-score networks use a learning rate of 2×10−52× 10^-5, and we train for 900 steps. When the chunk size is 3, the generator learning rate remains 2×10−52× 10^-5, while the fake-score learning rates are reduced to 2×10−72× 10^-7, and we train for 750 steps. In the subsequent video-level refinement stage, the generator and fake-score learning rates are reduced to 2×10−62× 10^-6 and 4×10−74× 10^-7, respectively, and both models are trained for 800 steps. Evaluation Protocol. We adopt the VBench-Long suite (Huang et al., 2024; Huang et al., 2026b) for our primary evaluation, using the complete standard set of 944 unique prompts with five generations per prompt. All videos are generated at 832×480832× 480 resolution and 16 FPS, and evaluated at a duration of approximately 60 seconds for both chunk sizes. The 500-second rollout in Figure 1 is a qualitative example and is not included in these aggregate metrics. 4.2 Comparison with Baselines Figure 4: Qualitative comparison of long-horizon generation. While baseline methods suffer from severe color saturation and structural collapse beyond their training horizon, RMD consistently preserves sharp subject details and stable scene structures. We compare RMD with Self Forcing (Huang et al., 2026a) and Causal Forcing (Zhu et al., 2026). Both baselines share our standard sliding-window generation protocol, operating entirely without fixed frame anchors or modifications to the underlying autoregressive logic. This isolates the distillation objectives and ensures RMD introduces zero additional computational overhead during inference. We report results separately using chunk sizes of 1 and 3. Figure 4 presents corresponding qualitative comparisons over extended rollouts. Within the 5-second training horizon, all methods generate reasonable frames. However, as generation progresses far beyond this boundary (20s, 40s, 60s), the baselines exhibit visual degradation including color saturation and structural collapse. In contrast, RMD consistently preserves clear subject details and stable scene structures throughout the 1-minute trajectory. Table 1 presents the quantitative comparison on 1-minute generated videos. RMD establishes a clear advantage across both configurations. With a chunk size of 1, our method improves the total score by over 10 absolute points compared with the baselines (81.26 vs. 70.94 and 69.88), while attaining quality and semantic scores of 84.52 and 68.24, respectively. With a chunk size of 3, RMD achieves a total score of 81.48, with corresponding quality and semantic scores of 85.03 and 67.24. Together, these quantitative and qualitative results demonstrate that explicitly decoupling chunk supervision from historical context effectively prevents the degradation typical of joint video-level distillation. Table 1: Quantitative comparison on long-horizon generation. Evaluated on VBench-Long, RMD consistently outperforms video-level distillation baselines across all metrics. Chunk size Method Total ↑ Quality ↑ Semantic ↑ Self Forcing 70.94 77.89 43.17 1 Causal Forcing 69.88 78.34 36.03 RMD (ours) 81.26 84.52 68.24 Self Forcing 77.44 81.99 59.22 3 Causal Forcing 74.88 80.53 52.27 RMD (ours) 81.48 85.03 67.24 4.3 Long-Horizon Extrapolation To assess performance degradation over extended rollouts, we evaluate the same videos generated with a chunk size of 1 at durations ranging from 10 to 60 seconds. Figure 5 plots the performance dynamics over these trajectories. RMD maintains stable visual quality as the rollout extends: its quality score changes only from 84.68 at 10 seconds to 84.52 at 60 seconds, where it retains a semantic score of 68.24. In contrast, the baselines degrade steeply as prediction errors compound; for example, the total score of Self Forcing drops from 79.12 to 70.94 over the same interval. This robust extrapolation confirms that independent chunk supervision effectively mitigates the accumulation of visual artifacts. Figure 5: Performance over increasing rollout lengths. VBench-Long scores are evaluated at durations ranging from 10 to 60 seconds. As generation extends, RMD maintains stable visual quality and significantly mitigates the performance degradation suffered by baseline methods. 4.4 Ablation Studies Figure 6: Qualitative ablation results. The spatio-temporal slice is extracted along the red line. Removing the rollout-marginal objective (video DMD only) causes severe color artifacts. Removing video-level refinement (marginal only) introduces temporal flickering (jagged slice artifacts). Removing asymmetric denoising exits (random exits) destroys the temporal prior, resulting in severe temporal jitter. Full RMD achieves both high visual quality and temporal coherence. Effectiveness of the Rollout-Marginal Objective. Table 2 evaluates the components of RMD using a chunk size of 1. Applying video-level DMD alone yields a total score of 72.44, and qualitatively, the generation rapidly collapses into severe color artifacts (Figure 6). Optimizing exclusively for the rollout-marginal objective (Marginal only) raises the quality score to 84.90, confirming that context-free scoring effectively prevents the visual degradation. Effectiveness of Refinement with Video-Level DMD. While the Marginal only variant achieves high visual realism, it lacks cross-chunk constraints and falls short on temporal coherence, which is visibly manifested as jagged flickering in its spatio-temporal slice (Figure 6). The full RMD pipeline resolves this via subsequent refinement with video-level DMD, which improves subject consistency, background consistency, and motion smoothness (Table 2(b)). As shown in Figure 6, this combined approach achieves the best total and semantic scores (81.26 and 68.24, respectively) while restoring smoother, more continuous trajectories in the spatio-temporal slice. Effectiveness of Asymmetric Denoising Exits. A variant that applies random-step exits uniformly across all chunks suffers from severe temporal jitter (Figure 6) and performs worse across all metrics, with total and quality scores of 78.29 and 82.48, respectively (Table 2(a)). This supports our asymmetric design: supervising the initial chunk across random steps preserves the pretrained temporal prior, while restricting continuation chunks to the final exit strictly refines local details. Table 2: Quantitative ablation results. (a) Removing refinement with video DMD (marginal only) or asymmetric exits (random exits) degrades overall performance. (b) Temporal metrics confirm that this refinement is essential for temporal coherence. (a) Aggregate scores Variant Total ↑ Quality ↑ Semantic ↑ video DMD only 72.44 78.68 47.50 marginal only 80.64 84.90 63.59 random exits 78.29 82.48 61.56 RMD (full) 81.26 84.52 68.24 (b) Temporal metrics Metric Marginal only RMD (full) Subject Cons. ↑ 97.29 98.05 Background Cons. ↑ 96.26 96.89 Temporal Flickering ↑ 99.06 99.41 Motion Smoothness ↑ 98.79 98.96 5 Conclusion Autoregressive video generation inherently suffers from error accumulation because joint temporal scoring forces the teacher to compromise local visual quality for sequence-level consistency. To resolve this coupling, we introduce Rollout-Marginal Distillation (RMD), which disentangles spatial realism from temporal coherence. By independently matching chunk marginals and subsequently refining global dynamics, RMD provides a clean, context-free supervision signal. Quantitative evaluations over 60-second rollouts demonstrate that RMD effectively prevents artifact accumulation beyond the training horizon, while a 500-second qualitative example illustrates its potential at substantially longer durations. We believe this decoupled paradigm offers a robust foundation for future long-horizon, streamable video generation. References Bandyopadhyay et al. (2026) H. Bandyopadhyay, X. Ren, Z. Huang, J. Z. Wu, T. Cao, R. Li, B. Chu, S. Fidler, Y. Song, and Z. Wang Context-matched distillation: teacher causality for autoregressive video distillation. arXiv preprint arXiv:2608.13391. Cited by: §2. Bruce et al. (2024) J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, Y. Aytar, S. M. E. Bechtle, F. Behbahani, S. C.Y. Chan, N. Heess, L. Gonzalez, S. Osindero, S. Ozair, S. Reed, J. Zhang, K. Zolna, J. Clune, N. de Freitas, S. Singh, and T. Rocktäschel Genie: generative interactive environments. In Forty-first International Conference on Machine Learning, External Links: Link Cited by: §1. Hu et al. (2022) E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: §3.3. Huang et al. (2026a) X. Huang, Z. Li, G. He, M. Zhou, and E. Shechtman Self forcing: bridging the train-test gap in autoregressive video diffusion. Advances in Neural Information Processing Systems 38, p. 167283–167308. Cited by: §1, §1, §2, §3.1, §3.2, §3.3, §3.3, §4.1, §4.2. Huang et al. (2024) Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu VBench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §4.1. Huang et al. (2026b) Z. Huang, F. Zhang, X. Xu, Y. He, J. Yu, Z. Dong, Q. Ma, N. Chanpaisit, C. Si, Y. Jiang, Y. Wang, X. Chen, Y. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models . IEEE Transactions on Pattern Analysis & Machine Intelligence 48 (03), p. 3268–3285. External Links: ISSN 1939-3539, Document, Link Cited by: §4.1. Li et al. (2026) H. Li, S. Liu, Z. Lin, and M. Chandraker Rolling sink: bridging limited-horizon training and open-ended testing in autoregressive video diffusion. arXiv preprint arXiv:2602.07775. Cited by: §1, §2. Liu et al. (2026) H. Liu, C. Wang, F. Gao, X. He, Y. Ma, Z. Wan, Y. Zhang, X. Wei, and Q. Chen OPSD-v: on-policy self-distillation for post-training few-step autoregressive video generators. arXiv preprint arXiv:2607.08766. Cited by: §2. Liu et al. (2024) S. Liu, Z. Ren, S. Gupta, and S. Wang Physgen: rigid-body physics-grounded image-to-video generation. In European Conference on Computer Vision, p. 360–378. Cited by: §1. Loshchilov and Hutter (2019) I. Loshchilov and F. Hutter Decoupled weight decay regularization. In International Conference on Learning Representations, External Links: Link Cited by: §4.1. Lu et al. (2025) Y. Lu, Y. Ren, X. Xia, S. Lin, X. Wang, X. Xiao, A. J. Ma, X. Xie, and J. Lai Adversarial distribution matching for diffusion distillation towards efficient image and video synthesis. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 16818–16829. Cited by: §2. Savva et al. (2026) G. Savva, O. Michel, D. Lu, S. Waiwitlikhit, T. Meehan, D. Mishra, S. Poddar, J. Lu, and S. Xie Solaris: building a multiplayer video world model in minecraft. arXiv preprint arXiv:2602.22208. Cited by: §2, §3.3. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §1, §3.2, §3.3. Wang et al. (2026) Y. Wang, H. Zhang, T. Xue, Y. Qiao, Y. Wang, C. Xu, and X. Chen Vdot: efficient unified video creation via optimal transport distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9273–9283. Cited by: §2. Wu et al. (2026) T. Wu, R. Li, L. Zhang, and K. Ma Diversity-preserved distribution matching distillation for fast visual synthesis. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §3.3. Yang et al. (2026) S. Yang, W. Huang, R. Chu, Y. Xiao, Y. Zhao, X. Wang, M. Li, E. Xie, Y. Chen, Y. Lu, S. Han, and Y. Chen LongLive: real-time interactive long video generation. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2. Yin et al. (2024a) T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems 37, p. 47455–47487. Cited by: §2, §3.1. Yin et al. (2024b) T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, and T. Park One-step diffusion with distribution matching distillation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6613–6623. Cited by: §1, §2, §3.1. Yin et al. (2025) T. Yin, Q. Zhang, R. Zhang, W. T. Freeman, F. Durand, E. Shechtman, and X. Huang From slow bidirectional to fast autoregressive video diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 22963–22974. Cited by: §1, §1, §2, §3.1. Zhang et al. (2025) K. Zhang, Z. Tang, X. Hu, X. Pan, X. Guo, Y. Liu, J. Huang, L. Yuan, Q. Zhang, X. Long, X. Cao, and W. Yin Epona: autoregressive diffusion world model for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §1. Zheng et al. (2026a) K. Zheng, G. He, M. Zhao, J. Zhang, H. Chen, J. Chen, C. Lin, M. Liu, J. Zhu, and Q. Ma Causal-rcm: a unified teacher-forcing and self-forcing open recipe for autoregressive diffusion distillation in streaming video generation and interactive world models. arXiv preprint arXiv:2606.25473. Cited by: §2. Zheng et al. (2026b) K. Zheng, Y. Wang, Q. Ma, H. Chen, J. Zhang, Y. Balaji, J. Chen, M. Liu, J. Zhu, and Q. Zhang Large scale diffusion distillation via score-regularized continuous-time consistency. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §2. Zhu et al. (2026) H. Zhu, M. Zhao, G. He, H. Su, C. Li, and J. Zhu Causal forcing: autoregressive diffusion distillation done right for high-quality real-time interactive video generation. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1, §2, §3.1, §3.3, §4.1, §4.2.