Paper deep dive
In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion
Yikai Wang, Xiao Han, Mengmeng Xu, Juan Camilo Perez, Yiannis Douratsos, Sen He, Zijian Zhou, Fei Zhang, Zhaochong An, Juan-Manuel Perez-Rua, Chen Change Loy, Tao Xiang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, each through a short sequence of denoising stages. To memorize chunks that are already generated, previous methods reconstruct a clean or less-noisy key--value (KV) cache by additional forwards to build the cache without advancing an output latent. However, every denoising forward itself already computes the in-flight KV of the current chunk. We introduce FlashForward, which directly reuses this cache to avoid the heavy cache-update-only model forwards. After the current chunk completes one denoising stage, its stage-specific cache is already available for the next chunk. Assigning one GPU to each stage therefore lets different chunks occupy different stages concurrently. This early availability has a quality cost: the resulting stage-matched history is noisy, causing appearance and motion drift among chunks. To complement it, FlashForward produces sparse auxiliary clean anchor latents before the corresponding region is generated so the generation trajectories can be stabilized by this two-sided conditioning. The two memories operate at different temporal scales: sparse clean anchor KV supplies coarse, long-range two-sided structural guidance, while dense stage-matched history preserves fine, recent evolution. With up to four GPUs, FlashForward runs $1.16$--$1.69\times$ faster than HiAR and $1.42$--$2.92\times$ faster than Self-Forcing for 16 FPS videos of 20 seconds or longer across 1.3B and 14B backbone scales at 480p and 720p. On VBench, for the 1.3B model at 480p, it achieves higher scores and remains stable at longer durations, demonstrating that FlashForward generates high-quality and temporally consistent videos across durations of 20s, 35s and 65s at a much faster generation speed.
Tags
Links
- Source: https://arxiv.org/abs/2609.32540v1
- Canonical: https://arxiv.org/abs/2609.32540v1
Trouble viewing inline? Open PDF directly →
Full Text
96,567 characters extracted from source content.
Expand or collapse full text
In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion Yikai Wang Affiliation: Meta Xiao Han Affiliation: Meta Mengmeng Xu Affiliation: Meta Juan C. Pérez Affiliation: Meta Yiannis Douratsos Affiliation: Meta Sen He Affiliation: Meta Zijian Zhou Affiliation: Meta Fei Zhang Affiliation: Meta Zhaochong An Affiliation: Meta Juan-Manuel Pérez-Rúa Affiliation: Meta Chen Change Loy Affiliation: S-Lab, Nanyang Technological University Tao Xiang Affiliation: Meta Abstract Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, each through a short sequence of denoising stages. To memorize chunks that are already generated, previous methods reconstruct a clean or less-noisy key–value (KV) cache by additional forwards to build the cache without advancing an output latent. However, every denoising forward itself already computes the in-flight KV of the current chunk. We introduce FlashForward, which directly reuses this cache to avoid the heavy cache-update-only model forwards. After the current chunk completes one denoising stage, its stage-specific cache is already available for the next chunk. Assigning one GPU to each stage therefore lets different chunks occupy different stages concurrently. This early availability has a quality cost: the resulting stage-matched history is noisy, causing appearance and motion drift among chunks. To complement it, FlashForward produces sparse auxiliary clean anchor latents before the corresponding region is generated so the generation trajectories can be stabilized by this two-sided conditioning. The two memories operate at different temporal scales: sparse clean anchor KV supplies coarse, long-range two-sided structural guidance, while dense stage-matched history preserves fine, recent evolution. With up to four GPUs, FlashForward runs 1.161.16–1.69×1.69× faster than HiAR and 1.421.42–2.92×2.92× faster than Self-Forcing for 16 FPS videos of 20 seconds or longer across 1.3B and 14B backbone scales at 480p and 720p. On VBench, for the 1.3B model at 480p, it achieves higher scores and remains stable at longer durations, demonstrating that FlashForward generates high-quality and temporally consistent videos across durations of 20s, 35s and 65s at a much faster generation speed. †date: September 26, 2026†Project Page: https://yikai-wang.github.io/FlashForward Figure 1: FlashForward redesigns the autoregressive diffusion generation pipeline to remove heavy cache-update-only forwards, significantly reducing generation latency. (a) Compared with pipelines that reconstruct clean (Self-Forcing [21]) or less-noisy (HiAR [65]) KV through context encoding that does not advance an output latent, FlashForward substantially reduces latency while preserving generation quality and coherence, even in long videos. (b) FlashForward adopts a planner to autoregressively produce sparse auxiliary anchors for two-sided temporal conditioning. A renderer can then run in standard autoregressive form conditioned on these anchors such that every ordinary forward advances one chunk and publishes stage-matched history for later chunks, allowing different denoising stages to run on different GPUs. (c) Quality versus latency for 1.3B models on 480p videos, using each method’s fastest setup with up to four GPUs. See the project page for an interactive demo. 1 Introduction Autoregressive video diffusion [17, 48, 13] generates a video one short chunk at a time. Each chunk is produced by generating its latent through a sequence of denoising stages. To memorize generated content, the model stores features from earlier chunks in a key–value (KV) cache for subsequent generation with additional model passes [9, 7, 41]. Recent methods reduce the denoising sequence to only a few stages [59, 21, 63, 31, 30], making any additional model forward used only to prepare memory an increasingly important bottleneck. Existing methods incur this overhead in different ways. Self-Forcing [21] completes a chunk and then passes its clean output through the model again to construct the cache. HiAR [65] allows different segments to overlap in execution by conditioning on less-noisy rather than clean-endpoint context, but re-encoding at every denoising stage. These additional passes build memory without directly advancing video generation. However, the computation during denoising already establishes an in-flight KV for the current chunk. If we directly reuse this cache, we could avoid the heavy cache-update-only model forwards and further reduce the generation latency. More broadly, we analyze the design choice of the cross-chunk state contract, specifying what representation each chunk publishes, at what noise level, and when later chunks may consume it. The contract determines not only the temporal memory available to the model, but also the dependency graph of generation. In previous clean or less-noisy history contracts, the cache-update-only forwards build state without advancing an output latent. In this paper, we propose FlashForward to instead publish an ordinary same-stage cache for reuse at the same denoising stage, making the required state available as part of generation itself. Among the designs in Tab. 1, it is the only memory that is both emitted by an output-advancing forward and available early to pipeline chunks across stage workers. This early availability has a cost. Stage-matched history comes from unfinished, noisy chunks. Used alone, it can propagate uncertainty in appearance and motion [5, 65]. Returning to dense clean state for every completed chunk would restore the heavy cache-update-only work. FlashForward therefore complements this promptly available history with sparse auxiliary anchors distributed across the timeline and planned in advance for a clean two-sided memory. This availability rule produces a planner–renderer generation graph. The planner role produces the auxiliary anchor latents and their clean anchor KV cache bank; the renderer role generates the video while reading a local anchor KV block and the stage-matched renderer history bank, as in Fig. 1(b). The two memories divide temporal responsibility: sparse clean anchor KV constrains coarse structure over a longer interval, whereas dense stage-matched renderer history carries fine appearance and motion changes from recent chunks. Only sparse planner blocks require a clean-endpoint cache-extraction forward, and each local anchor window conditions multiple renderer chunks. Hence the efficiency of dense rendering without cache-update-only forwards is preserved. FlashForward uses a shared generator backbone for both roles. A learned role embedding and role-specific LoRA adapters [20] specialize their temporal dependencies while retaining common visual and language knowledge. We train FlashForward in two phases: Packed supervised fine-tuning establishes the planner–renderer graph from real videos; self-rollout distillation distills the guidance and denoising stages, and adapts its few-step generation process to generated context. We evaluate and compare the latency of FlashForward with other generation pipelines across 1.3B and 14B backbone scales on 480p and 720p videos. For 16 FPS videos of 20 seconds or longer in up to four-GPU settings, FlashForward is 1.161.16–1.69×1.69× faster than less-noisy history (HiAR) and 1.421.42–2.92×2.92× faster than clean history (Self-Forcing). We evaluate generation quality for the 1.3B model at 480p under VBench. FlashForward scores 0.838 on Total, surpassing 0.805 for Self-Forcing and 0.821 for HiAR. Furthermore, as the duration increases to 65 seconds, the generation quality remains stable for FlashForward, demonstrating its effectiveness as a faster generation pipeline. Contributions. Our contribution is one state-availability design, realized as a single causal chain. (1) Each ordinary renderer forward publishes stage-matched renderer history while advancing the current chunk, providing state available early enough to induce an inter-chunk wavefront over stage workers without renderer cache-update-only forwards. (2) We instantiate a temporal-scale allocation of cross-chunk state: sparse clean anchor KV prepared ahead of each local region supplies coarse long-range structure, while promptly available but noisy and past-only renderer history supplies fine recent evolution. (3) We realize this graph with planner and renderer roles sharing a base generator, train it through real-video supervision and self-rollout distillation, and demonstrate latency benefits across model sizes and resolutions together with quality superiority for the 1.3B generator at 480p. 2 Related Work Table 1: How memory determines execution for S denoising stages with noise levels σS>⋯>σ0=0 _S>·s> _0=0. FlashForward additionally uses sparse clean anchor KV prepared before generation. Method Memory noise level When it becomes usable Cache-update-only forwards Chunk execution Self-Forcing σ0 _0 After the chunk is re-encoded 11 per chunk Serial N-C-Causal-rCM σ1→σ0 _1→ _0 After the last denoising stage 00 Serial HiAR σk−1 _k-1 After each stage-specific re-encoding S per chunk Overlapped FlashForward σk→σk−1 _k→ _k-1 Immediately after each denoising stage 00 Overlapped Few-step autoregressive video diffusion. Video generators combine diffusion or flow objectives with pixel-space, latent, and Transformer backbones [16, 17, 29, 43, 38, 3, 2, 37]. Few-step objectives use implicit sampling, distillation, and consistency training [44, 45, 39, 60, 4]. Autoregressive video models factor generation into sequential units and generated histories [50, 47, 51, 15]; FlashForward changes what is passed between those units, and with it their execution order. Cross-chunk memory and execution. We view an autoregressive diffusion method as defining a cross-chunk state contract: which representation a chunk publishes, at what noise level, and when that representation becomes available to later chunks. Together with the within-chunk denoising order, this contract determines both the number of cache-update-only forwards and which chunk updates can overlap (Tab. 1). Self-Forcing [21] publishes clean KV only after a completed chunk undergoes one cache-update-only forward, so later chunks wait at a serial boundary. The noisy-context variant of Causal-rCM [63] (N-C-Causal-rCM), following Diagonal Distillation [30], reuses KV from the final denoising forward, avoiding that extra update; because the state is published only when the predecessor reaches its final stage, chunks remain serial. HiAR [65] conditions each chunk at intermediate stage on its predecessors’ less-noisy context, enabling an anti-diagonal pipeline at the cost of per-stage context encoding per consumed chunk. FlashForward instead publishes the KV produced by every ordinary renderer forward to later chunks at the same stage. Each publication advances the current chunk rather than serving only to build memory, and the resulting same-stage dependencies form a wavefront. Because this stage-matched renderer history is noisy and past-only, FlashForward complements it with sparse clean anchor KV made available before each local region is rendered. Another research direction [11, 54, 40, 24, 35, 34, 55] studies how to determine which cached content to retain, compress, merge, reuse, or switch; these policies are largely orthogonal to the state contract. Plan-then-infill and future-guided generation. Sparse-to-dense video generation first establishes coarse temporal structure and then fills the intervening frames [10, 14, 13, 56]. Xiang et al. [53] and Ouyang et al. [36] separate keyframe planning from segment population; FramePack [62] and SneakPeek [19] predict anchors and future keyframes ahead of the frames between them; Bendel et al. [1] and Zhang et al. [61] use two-sided anchors for interval generation. These works establish sparse planning and two-sided conditioning as effective sources of temporal coherence. FlashForward repurposes this form of conditioning as a coarse-timescale state that complements fine-timescale stage-matched renderer history. Unlike in conventional infilling, its auxiliary anchor latents are conditioning-only. Their clean anchor KV lets the renderer retain promptly available stage-matched renderer history without reverting to dense clean cache-update-only forwards. Noise schedules and parallel execution. Diffusion Forcing [5], FIFO-Diffusion [26], Rolling Forcing [31], and Diagonal Distillation [30] assign different noise levels or update times across temporal positions, while Ms. Forcing [28] adapts computation to the noise level. DistriFusion [27] and PipeFusion [8] parallelize spatial computation within a denoising trajectory. HiAR [65] pipelines successive chunks across denoising-stage workers, placing them on an anti-diagonal schedule that it sustains by re-encoding each consumed predecessor at every stage. Its context-only work is therefore paid per stage per consumed chunk, rather than once per chunk as in Self-Forcing. A noise schedule specifies when each chunk is updated, whereas a cross-chunk state rule specifies which earlier representation conditions that update. In FlashForward, the same anti-diagonal follows directly from the same-stage state dependency, and no output chunk is re-encoded. 3 Stage-Matched History with Clean Two-Sided Anchors FlashForward uses one generator in two roles. The planner first produces a sparse clean plan, and the renderer then generates the video while reading that plan and the recent history left by earlier renderer chunks. The key operation is one ordinary renderer forward: it both advances the current chunk and leaves reusable KV for later renderer chunks during generation. A concrete generation example. Fig. 2 follows an output of 81 latent frames. The planner generates nine auxiliary clean anchor latents at positions 0,10,…,80\0,10,…,80\, three anchors at a time. It runs one cache-extraction forward on each completed block to obtain clean anchor KV. The renderer then generates all 81 output positions in 27 three-latent chunks guided by a two-sided clean plan and in-flight memory. For example, the chunk at positions 6–8 reads nearby clean anchor KV at 0,10,20\0,10,20\, which bracket this interior chunk, together with the KV left by recent chunks at its current denoising stage. As each chunk traverses four stages, the next chunk can follow one stage behind, producing the wavefront in panel (c). The complete data flow is therefore: generate each auxiliary anchor block and publish its clean KV, then render the dense video while every renderer forward advances the output and publishes stage-matched renderer history. Figure 2: (a) The planner produces sparse auxiliary anchor latents; local three-anchor windows guide the renderer. (b) One renderer forward reads clean anchor KV and stage-matched renderer history, predicts its latent update, and publishes its own KV. (c) These dependencies form a wavefront that completes one chunk per pipeline round after warm-up. See the concrete generation example in Sec. 3. See the project page for an interactive demo. Notation. Given a condition c, the target contains L latent positions indexed by ℐ=0,…,L−1I=\0,…,L-1\. The auxiliary anchor indices form a sparse subset of this timeline with anchor stride Δ as =mΔ∣m∈ℕ0,mΔ<L∪L−1,M=||.P=\m m _0,\ m <L\∪\L-1\, M=|P|. (1) The planner processes P in NP=⌈M/BP⌉N_P= M/B_P ordered blocks j\Q_j\ of at most BPB_P auxiliary anchor latents. The renderer processes ℐI in NR=⌈L/BR⌉N_R= L/B_R ordered chunks i\C_i\ of at most BRB_R latents, and Γ(i)=2uΔ,(2u+1)Δ,(2u+2)Δ⊆,u=⌊BRi/(2Δ)⌋ (i)=\2u ,(2u+1) ,(2u+2) \ ,u= B_Ri/(2 ) identifies the clean anchor KV entries visible to renderer chunk i. Interior regions use anchor windows with indices before and after each region; at the beginning or end of a finite video, Γ(i) (i) uses the available boundary-side anchors. Let θ denote the shared model with role-specific embeddings and LoRA adapter sets, while FθPF_θ^P and FθRF_θ^R denote its planner and renderer routes. At inference, both use S denoising stages indexed by k, σS>⋯>σ0=0 _S>·s> _0=0, and τk=τ(σk) _k=τ( _k) denotes the corresponding timestep. Our empirical configuration uses BP=BR=3B_P=B_R=3, Δ=10 =10, S=4S=4, a context budget of Wctx=21W_ctx=21 latents, a renderer history length of H=5H=5 chunks, and a planner history length of HP=6H_P=6 blocks. Sec. A.1 explains how equal planner block and renderer chunk sizes together with Δ=10 =10 balance anchor production and consumption during streaming generation. 3.1 Stage-Matched Renderer History Enables Pipelined Rendering A renderer forward takes the current chunk’s noisy latents at a given denoising stage as inputs. It reads KV from earlier chunks at the same stage together with clean KV from nearby anchors as memory. The forward produces the prediction used to advance denoising as outputs and stores its own KV for later chunks at that stage as updated memory. See Fig. 2(b) for an illustration. Precisely, Γ(i)A_ (i) contains clean anchor KV entries selected by Γ(i) (i). The stage-matched renderer history bank ℬk(i)B_k^(i) contains KV produced when up to H preceding renderer chunks passed through denoising stage k. Under the rectified-flow parameterization [32], xσ=(1−σ)x0+σεx_σ=(1-σ)x_0+σ and the velocity target is v=ε−x0v= -x_0. At stage k, the renderer takes the current noisy chunk xi,kRx^R_i,k, conditions on Γ(i)A_ (i) and ℬk(i)B_k^(i), and predicts v^i,kR v^R_i,k to advance the latent from σk _k toward σk−1 _k-1. The same forward publishes its KV to the stage-k bank for later chunks, as shown in Fig. 2(b): (v^i,kR,Ki,kR,Vi,kR)=FθR(xi,kR,c,τk,Γ(i),ℬk(i)),ℬk(i+1)=TailH(ℬk(i)∪(i,Ki,kR,Vi,kR)). ( v^R_i,k,K^R_i,k,V^R_i,k )=F_θ^R\! (x^R_i,k,c, _k;A_ (i),B_k^(i) ),\ B_k^(i+1)=Tail_H\! (B_k^(i)∪(i,K^R_i,k,V^R_i,k) ). (2) Both KV banks are ordered sequences: ∪ adds their newest entry, and TailHTail_H retains the H most recent renderer chunks. No renderer cache-update-only forward. In practice, the renderer can process the clip autoregressively, updating the KV cache on the fly as it handles one chunk at a time, or process the full clip at once using a chunk-wise causal mask. Why the two states are complementary. The two states are deliberately assigned different temporal scales. Dense stage-matched renderer history is available immediately and records fine changes in recent appearance and motion, but it is noisy; used alone, it can propagate uncertainty across chunk boundaries. The sparse clean anchor provides a stable, coarse structural reference over a longer interval for both the past and the future. The clean-endpoint cache extraction is confined to sparse planner blocks, which advance by a large stride: one planner transition advances 30 latents, compared with 3 for one renderer transition, nearly a tenfold reduction in autoregressive depth. This reduces the number of per-chunk cache-update-only forwards and leads to inter-chunk pipelining for the renderer. This coarse plan also improves temporal coherence over long horizons. Planning the clean anchor KV bank. The planner generates auxiliary anchor latents autoregressively across blocks j\Q_j\. After each planner block is generated, one planner cache-extraction forward publishes its clean anchor KV entries. Renderer chunk i reads the local window Γ(i)A_ (i) with earlier and later anchor indices while adjacent regions share anchors. In Fig. 2(a), the windows are 0,10,20\0,10,20\, 20,30,40\20,30,40\, 40,50,60\40,50,60\, and 60,70,80\60,70,80\. Wavefront rendering. A renderer node (i,k)(i,k) depends on the noisier chunk at node (i,k+1)(i,k+1) and the earlier chunk at node (i−1,k)(i-1,k). With one worker assigned to each of the S stages, nodes on the same anti-diagonal can therefore run concurrently for different chunks, as shown in Fig. 2(c). A pipeline round is one such forward slot on every active stage worker. After warm-up, the wavefront completes one renderer chunk per round while every forward both denoises and publishes memory. This pipeline avoids the repeated stage-specific re-encoding in HiAR. Alg. 1 gives the batched four-device rollout; device allocation and timing details are deferred to Sec. A. Streaming generation. FlashForward naturally supports streaming generation by dedicating one GPU to the planner and the remaining GPUs to the renderer. This allows rendering to begin as soon as the planner produces the first anchor block, further reducing latency. We discuss this in Sec. A.2.3. 3.2 Training a Shared Planner and Renderer Figure 3: (a) Shared backbone with role-specific embeddings and LoRA adapters. (b) Packed supervised fine-tuning (SFT) trains both roles in one forward. (c) Self-rollout distillation adapts the model to long generated contexts. See the project page for an interactive demo. The two roles share visual and language knowledge but require different temporal dependencies. We use role-specific embeddings and LoRA adapters to specialize these dependencies, as in Fig. 3(a). We learn this in two phases: supervised fine-tuning first establishes both roles from real videos, and self-rollout distillation then adapts them to generated context and guidance-free few-step inference. Phase 1: Supervised fine-tuning. The bidirectional generator is not yet adapted to either the planner’s large-stride block-causal dependence or the renderer’s clean anchor KV and stage-matched renderer history dependencies. Phase 1 trains both roles together in one packed forward from real videos. The planner blocks attend causally to clean KV from earlier blocks as in teacher forcing [52]. The renderer targets attend to clean anchor KV admitted by Γ(i) (i), and to up to H preceding renderer chunks at the same noise level. A training clip supplies planner and renderer targets directly. As planner latents are sparse, we time-rebase each clip by cropping it from temporal offsets for augmentation, as in Fig. 3(b). Alg. 2 and Sec. C.1 give the details. Phase 2: Self-rollout distillation. Phase 1 conditions on ground-truth latents, whereas inference uses planner-generated anchor KV and renderer-generated history throughout a few-step trajectory. Phase 2 therefore trains on a planner–renderer self-rollout and applies the distribution-matching distillation objective [58, 57], as shown in Fig. 3. As in Self-Forcing [21], conditioning on self-generated context mitigates exposure bias. The student first performs a causal planner rollout over auxiliary anchor blocks and then renders the full clip at once. Both roles follow the same four-stage guidance-free student schedule. Although the score networks use shorter windows, the loss covers the full clip. We split the clip into score-window-sized tiles and decode and re-encode the first frame of each later segment to obtain its image head. During training, the renderer part is implemented as a standard block-causal forward, rather than a chunk-by-chunk cache update. This computation graph lets gradients flow from later chunks to earlier chunks, and from the renderer to the planner. Alg. 3 and Sec. C.2 introduce these in detail. 4 Experiments We build FlashForward on Wan2.1-T2V-1.3B [49] and train it on 256K Shutterstock videos [42] with generated captions. Each clip has 81 latents, corresponding to 321 RGB frames at 16 FPS. Training takes 1,800 steps for the SFT phase and 100 student steps for the distillation phase with a batch size of 128. See Sec. C for details. We evaluate generation quality on the 1.3B model at 480480p, following previous works. We use VBench-1.0 [22] on 20-second videos following HiAR [65], and VBench-Long [23] on 20-, 35-, and 65-second videos. We mainly compare with clean-history methods Self-Forcing [21] and Causal Forcing [64], and less-noisy-history method HiAR [65]. We also report results from LTX-Video [12], Wan2.1 [49], NOVA [7], Pyramid Flow [25], SkyReels-V2 [6], MAGI-1 [41], and CausVid [59] for reference. Figure 4: Average denoising latency on four-GPUs and our five-GPU streaming version with standard deviations. Tab. 4 reports exact numbers. See the project page for an interactive demo. Table 2: VBench-1.0 on 20 s videos (top; 5 s for bidirectional models) and VBench-Long (bottom). VBench-1.0 LTX- Wan2.1- NOVA Pyramid SkyReels- MAGI-1- CausVid Self- Causal HiAR FlashForward Metric Video T2V-1.3B Flow V2-1.3B 4.5B Forcing Forcing Total ↑ 0.766 0.802 0.773 0.775 0.788 0.757 0.764 0.805 0.810 0.821 0.838 Quality ↑ 0.789 0.813 0.777 0.804 0.808 0.785 0.771 0.829 0.837 0.846 0.859 Semantic ↑ 0.685 0.766 0.757 0.670 0.707 0.647 0.740 0.708 0.701 0.723 0.753 VBench-Long Self-Forcing Causal Forcing HiAR FlashForward Duration Total ↑ Quality ↑ Semantic ↑ Total ↑ Quality ↑ Semantic ↑ Total ↑ Quality ↑ Semantic ↑ Total ↑ Quality ↑ Semantic ↑ 20 s 0.8258 0.8460 0.7452 0.8178 0.8515 0.6833 0.8265 0.8440 0.7565 0.8404 0.8635 0.7481 35 s 0.8087 0.8330 0.7116 0.7906 0.8343 0.6159 0.8230 0.8452 0.7339 0.8401 0.8636 0.7460 65 s 0.7806 0.8186 0.6288 0.7539 0.8145 0.5113 0.8219 0.8419 0.7418 0.8395 0.8617 0.7505 4.1 Fast and High-Quality Generation Faster generation. Fig. 4 and Tab. 4 show that, from 20 s onward and using each method’s fastest setup with up to four GPUs, FlashForward is the fastest autoregressive schedule in every model–resolution setting, 1.161.16–1.69×1.69× faster than HiAR and 1.421.42–2.92×2.92× faster than Self-Forcing. The gain comes from two properties of its state contract: stage-matched renderer history avoids the cache-update-only forwards required by Self-Forcing and HiAR, while its availability leads to an inter-chunk stage pipeline. Sec. A provides the detailed analysis. Figure 5: Qualitative comparison at 20, 35, and 65 s. Blue boxes indicate anchor frames. The baselines have no anchors. See full prompt and video comparisons on the project page. Quality. VBench-1.0 results are in Tab. 2 (top). Competing methods’ results are taken from HiAR [65]. FlashForward achieves the highest Total and Quality scores. Its Semantic score is the highest among the distilled autoregressive methods. VBench-Long evaluates long-video quality. We run all four methods under the same protocol (Tab. 2, bottom). FlashForward leads in Total and Quality at every duration, maintaining long-duration quality, while its semantic alignment remains competitive. The behavior of FlashForward under different durations is stable and consistent. Qualitative comparison. Fig. 5 shows the same trend qualitatively. Self-Forcing accumulates color drift. HiAR avoids this collapse but drifts in other ways: the raccoon’s face and ears change in (b) and the knitter’s face becomes doll-like in (c), while in (a) the person’s face changes and the arms stay blurred under the cloak. FlashForward maintains temporal coherence while preserving motion. Its frames at anchor positions show no visible difference from those between anchors. 4.2 Further Analysis Why regenerate the anchor positions? Keeping the planner’s anchors as output would be cheaper, but we empirically find that this produces periodic seams at the anchor positions (see the project page for video examples). We compare and analyze two variants that both use the planner’s anchor latents as part of the video output and differ in the source of the anchor KV: one directly reuses the planner’s anchor KV, while the other feeds the anchor latents to the renderer role to obtain renderer-side KV. Both suffer from a similar seam issue. We analyze this in details in Sec. B. Ablation study. Tab. 3 (top) reports the training ablations. Time-rebased supervision provides a better initialization for distillation. Role specialization lets the model learn role-specific behavior more freely, and more importantly, reduces seams between anchor and non-anchor positions. Table 3: Top: training ablations. Bottom: conditioning-state ablations. SFT phase (50-step sampling) Distillation phase (4-step sampling) Ablation Total ↑ Quality ↑ Semantic ↑ Total ↑ Quality ↑ Semantic ↑ Dual-role SFT 0.8172 0.8371 0.7376 0.8184 0.8332 0.7593 ++ time-rebased supervision 0.8150 0.8345 0.7371 0.8198 0.8338 0.7638 ++ role-specific embedding and LoRA 0.8205 0.8377 0.7516 0.8380 0.8593 0.7528 Conditioning module Total ↑ Quality ↑ Semantic ↑ Stage-matched history Failed Failed 0.732 Clean history and future 0.826 0.849 0.735 Stage-matched history ++ clean anchor KV (ours) 0.838 0.859 0.753 Conditioning modules. We compare the following cross-chunk conditioning designs: (1) Stage-matched history; (2) Clean history and future; (3) Stage-matched history with clean history and future, our final design. Variants are trained under the same setup and reported in Tab. 3 (bottom). With only stage-matched history, the generated videos flicker so heavily that VBench fails to score their temporal flickering dimension. With only clean history and future, the model becomes a variant of Self-Forcing with a different temporal generation order. It mitigates the drift issue of Self-Forcing but still generates at a high latency; hence, we only utilize this as a sparse conditioning module rather than a dense generation model. By combining these modules, FlashForward demonstrates better generation quality with faster generation speed. Limitations and future work. FlashForward targets long-sequence, few-step generation on multiple GPUs. Planner and pipeline warm-up reduce its gains for short videos, while one-GPU inference cannot exploit the wavefront parallelism. Our training data consists only of real-world videos, which biases the model toward photorealistic content and may limit its performance on animation and other stylized domains. We use a fixed anchor stride, conditioning window, and vanilla training recipe except for the time-rebased augmentation; relaxing the current clean anchor-state assumption and adapting anchor selection remain future work. The cross-chunk state contract may also complement KV selection and compression, RoPE [46] modifications, and anti-drift techniques. Parallel generation of anchor-delimited intervals [53] is also promising. 5 Conclusion FlashForward treats cross-chunk memory as a state-availability problem: a cross-chunk state contract specifies what KV is published and when a later chunk may consume it. Publishing the KV produced by each ordinary renderer forward makes stage-matched renderer history available early enough to induce an inter-chunk wavefront over stage workers, so every renderer pass advances the output. Because this promptly available history is noisy, fine-scale, and past-only, sparse auxiliary anchor latents are converted into clean anchor KV ahead of the local regions they guide, providing a complementary coarse structural reference. The renderer uses this clean anchor KV only as conditioning and regenerates every final position. This temporal-scale allocation enables inter-chunk stage pipelining. Planner and renderer roles with a shared backbone realize the dependency graph through packed real-video supervision and full-clip self-rollout distillation. Across 1.3B and 14B backbone scales at 480p and 720p, FlashForward is the fastest evaluated autoregressive schedule on four GPUs for outputs of 321 RGB frames or more. On four H100 GPUs with 1.3B model at 480480p, it generates videos faster than other methods with better quality and temporal coherence across several durations. References [1] M. Bendel, S. W. Bailey, M. Vaidya, S. Badam, and X. He (2026) Goodbye drift: anchored tree sampling for long-horizon video-to-video generation. arXiv preprint arXiv:2605.20476. Cited by: §2. [2] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, V. Jampani, and R. Rombach (2023) Stable Video Diffusion: scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127. Cited by: §2. [3] A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis (2023) Align your latents: high-resolution video synthesis with latent diffusion models. In CVPR, Cited by: §2. [4] S. Cai, W. Nie, C. Liu, J. Berner, L. Zhang, N. Ma, H. Chen, M. Agrawala, L. Guibas, G. Wetzstein, and A. Vahdat (2026) Mode seeking meets mean seeking for fast long video generation. In ICML, Cited by: §2. [5] B. Chen, D. Martí Monsó, Y. Du, M. Simchowitz, R. Tedrake, and V. Sitzmann (2024) Diffusion Forcing: next-token prediction meets full-sequence diffusion. In NeurIPS, Cited by: §1, §2. [6] G. Chen, D. Lin, J. Yang, C. Lin, J. Zhu, M. Fan, H. Zhang, S. Chen, Z. Chen, C. Ma, W. Xiong, W. Wang, N. Pang, K. Kang, Z. Xu, Y. Jin, Y. Liang, Y. Song, P. Zhao, B. Xu, D. Qiu, D. Li, Z. Fei, Y. Li, and Y. Zhou (2025) SkyReels-V2: infinite-length film generative model. arXiv preprint arXiv:2504.13074. Cited by: §4. [7] H. Deng, T. Pan, H. Diao, Z. Luo, Y. Cui, H. Lu, S. Shan, Y. Qi, and X. Wang (2025) Autoregressive video generation without vector quantization. In ICLR, Cited by: §1, §4. [8] J. Fang, J. Pan, A. Li, X. Sun, and J. Wang (2025) PipeFusion: patch-level pipeline parallelism for diffusion transformers inference. In NeurIPS, Cited by: §2. [9] K. Gao, J. Shi, H. Zhang, C. Wang, J. Xiao, and L. Chen (2025) Ca2-VDM: efficient autoregressive video diffusion model with causal generation and cache sharing. In ICML, Cited by: §1. [10] S. Ge, T. Hayes, H. Yang, X. Yin, G. Pang, D. Jacobs, J. Huang, and D. Parikh (2022) Long video generation with time-agnostic VQGAN and time-sensitive transformer. In ECCV, Cited by: §2. [11] Y. Gu, W. Mao, and M. Z. Shou (2025) Long-context autoregressive video modeling with next-frame prediction. arXiv preprint arXiv:2503.19325. Cited by: §2. [12] Y. HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon, P. Panet, S. Weissbuch, V. Kulikov, Y. Bitterman, Z. Melumian, and O. Bibi (2025) LTX-Video: realtime video latent diffusion. arXiv preprint arXiv:2501.00103. Cited by: §4. [13] W. Harvey, S. Naderiparizi, V. Masrani, C. Weilbach, and F. Wood (2022) Flexible diffusion modeling of long videos. In NeurIPS, Cited by: §1, §2. [14] Y. He, T. Yang, Y. Zhang, Y. Shan, and Q. Chen (2022) Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221. Cited by: §2. [15] R. Henschel, L. Khachatryan, H. Poghosyan, D. Hayrapetyan, V. Tadevosyan, Z. Wang, S. Navasardyan, and H. Shi (2025) StreamingT2V: consistent, dynamic, and extendable long video generation from text. In CVPR, Cited by: §2. [16] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In NeurIPS, Cited by: §2. [17] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet (2022) Video diffusion models. In NeurIPS, Cited by: §1, §2. [18] J. Ho and T. Salimans (2021) Classifier-free diffusion guidance. In NeurIPS 2021 Workshop, Cited by: Appendix C. [19] C. Hong, G. Barquero, F. Sener, M. Georgopoulos, E. Schönfeld, S. Popov, Y. Du, O. Mañas, and A. Pumarola (2025) SneakPeek: future-guided instructional streaming video generation. arXiv preprint arXiv:2512.13019. Cited by: §2. [20] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In ICLR, Cited by: §C.1, §1. [21] X. Huang, Z. Li, G. He, M. Zhou, and E. Shechtman (2025) Self Forcing: bridging the train-test gap in autoregressive video diffusion. In NeurIPS, Cited by: §C.1, §C.2, Figure 1, §1, §1, §2, §3.2, §4. [22] Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2024) VBench: comprehensive benchmark suite for video generative models. In CVPR, Cited by: §4. [23] Z. Huang, F. Zhang, X. Xu, Y. He, J. Yu, Z. Dong, Q. Ma, N. Chanpaisit, C. Si, Y. Jiang, Y. Wang, X. Chen, Y. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2026) VBench++: comprehensive and versatile benchmark suite for video generative models. TPAMI. Cited by: §4. [24] Y. Ji, Z. Zhong, J. Zhang, Q. Yang, X. Jin, Y. Qin, W. Luo, S. Mao, W. Liu, and H. Li (2026) Forcing-KV: hybrid KV cache compression for efficient autoregressive video diffusion models. arXiv preprint arXiv:2605.09681. Cited by: §2. [25] Y. Jin, Z. Sun, N. Li, K. Xu, H. Jiang, N. Zhuang, Q. Huang, Y. Song, Y. Mu, and Z. Lin (2025) Pyramidal flow matching for efficient video generative modeling. In ICLR, Cited by: §4. [26] J. Kim, J. Kang, J. Choi, and B. Han (2024) FIFO-Diffusion: generating infinite videos from text without training. In NeurIPS, Cited by: §2. [27] M. Li, T. Cai, J. Cao, Q. Zhang, H. Cai, J. Bai, Y. Jia, K. Li, and S. Han (2024) DistriFusion: distributed parallel inference for high-resolution diffusion models. In CVPR, Cited by: §2. [28] Z. Li, X. Cong, H. Li, Z. Dou, C. Guo, A. Mittal, S. An, and S. Sridhar (2026) Ms. Forcing: efficient streaming video generation with multi-scale patchification and attention. arXiv preprint arXiv:2607.20940. Cited by: §2. [29] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow matching for generative modeling. In ICLR, Cited by: §2. [30] J. Liu, X. Liu, K. Mei, Y. Wen, M. Yang, and W. Liu (2026) Streaming autoregressive video generation via diagonal distillation. In ICLR, Cited by: §1, §2, §2. [31] K. Liu, W. Hu, J. Xu, Y. Shan, and S. Lu (2026) Rolling Forcing: autoregressive long video diffusion in real time. In ICLR, Cited by: §1, §2. [32] X. Liu, C. Gong, and Q. Liu (2023) Flow straight and fast: learning to generate and transfer data with rectified flow. In ICLR, Cited by: Appendix C, §3.1. [33] I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. In ICLR, Cited by: §C.1. [34] J. Luo, Q. Liu, T. Wang, J. Liu, J. Chen, C. Wang, H. Zhu, C. Gao, X. Hu, Q. Sun, and Z. Chen (2026) Future Forcing: future-aware training-free KV cache policy for autoregressive video generation. arXiv preprint arXiv:2605.30083. Cited by: §2. [35] Y. Ma, X. Zheng, J. Xu, X. Xu, F. Ling, X. Zheng, H. Kuang, H. Li, X. Wang, X. Xiao, F. Chao, and R. Ji (2026) Flow caching for autoregressive video generation. In ICLR, Cited by: §2. [36] J. Ouyang, W. Teng, G. Chen, Y. Zhao, and H. Chen (2026) DCARL: a divide-and-conquer framework for autoregressive long-trajectory video generation. In ECCV, Cited by: §2. [37] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In ICCV, Cited by: §2. [38] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In CVPR, Cited by: §2. [39] T. Salimans and J. Ho (2022) Progressive distillation for fast sampling of diffusion models. In ICLR, Cited by: §2. [40] D. Samuel, I. Tzachor, M. Levy, M. Green, G. Chechik, and R. Ben-Ari (2026) FAST-AR: fast autoregressive video diffusion and world models with temporal cache compression and sparse attention. In ICML, Cited by: §2. [41] Sand AI, H. Teng, H. Jia, L. Sun, L. Li, M. Li, M. Tang, S. Han, T. Zhang, W. Q. Zhang, W. Luo, X. Kang, Y. Sun, Y. Cao, Y. Huang, Y. Lin, Y. Fang, Z. Tao, Z. Zhang, Z. Wang, Z. Liu, D. Shi, G. Su, H. Sun, H. Pan, J. Wang, J. Sheng, M. Cui, M. Hu, M. Yan, S. Yin, S. Zhang, T. Liu, X. Yin, X. Yang, X. Song, X. Hu, Y. Zhang, and Y. Li (2025) MAGI-1: autoregressive video generation at scale. arXiv preprint arXiv:2505.13211. Cited by: §1, §4. [42] Shutterstock (2026) Shutterstock. Note: https://w.shutterstock.com/Accessed: 2026-01-01 Cited by: §4. [43] U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y. Taigman (2023) Make-A-Video: text-to-video generation without text-video data. In ICLR, Cited by: §2. [44] J. Song, C. Meng, and S. Ermon (2021) Denoising diffusion implicit models. In ICLR, Cited by: §2. [45] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever (2023) Consistency models. In ICML, Cited by: §2. [46] J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu (2024) RoFormer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. Cited by: §4.2. [47] R. Villegas, M. Babaeizadeh, P. Kindermans, H. Moraldo, H. Zhang, M. T. Saffar, S. Castro, J. Kunze, and D. Erhan (2023) Phenaki: variable length video generation from open domain textual descriptions. In ICLR, Cited by: §2. [48] V. Voleti, A. Jolicoeur-Martineau, and C. Pal (2022) MCVD: masked conditional video diffusion for prediction, generation, and interpolation. In NeurIPS, Cited by: §1. [49] T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z. Wu, and Z. Liu (2025) Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §C.1, §4. [50] D. Weissenborn, O. Täckström, and J. Uszkoreit (2020) Scaling autoregressive video models. In ICLR, Cited by: §2. [51] W. Weng, R. Feng, Y. Wang, Q. Dai, C. Wang, D. Yin, Z. Zhao, K. Qiu, J. Bao, Y. Yuan, C. Luo, Y. Zhang, and Z. Xiong (2024) ART-V: auto-regressive text-to-video generation with diffusion models. In CVPR Workshops, Cited by: §2. [52] R. J. Williams and D. Zipser (1989) A learning algorithm for continually running fully recurrent neural networks. Neural Computation. Cited by: §3.2. [53] X. Xiang, Y. Chen, G. Zhang, Z. Wang, Z. Gao, Q. Xiang, G. Shang, J. Liu, H. Huang, Y. Gao, C. Zhang, Q. Fan, and X. Li (2025) Macro-from-micro planning for high-quality and parallelized autoregressive long video generation. arXiv preprint arXiv:2508.03334. Cited by: §2, §4.2. [54] S. Yang, W. Huang, R. Chu, Y. Xiao, Y. Zhao, X. Wang, M. Li, E. Xie, Y. Chen, Y. Lu, S. Han, and Y. Chen (2026) LongLive: real-time interactive long video generation. In ICLR, Cited by: §C.2, §2. [55] Y. Yang, T. Zhang, W. Huang, J. Chen, B. Wu, X. He, D. Cai, B. Li, and P. Jiang (2026) Anchor Forcing: anchor memory and tri-region RoPE for interactive streaming video diffusion. In ECCV, Cited by: §2. [56] S. Yin, C. Wu, H. Yang, J. Wang, X. Wang, M. Ni, Z. Yang, L. Li, S. Liu, F. Yang, J. Fu, M. Gong, L. Wang, Z. Liu, H. Li, and N. Duan (2023) NUWA-XL: diffusion over diffusion for eXtremely long video generation. In ACL, Cited by: §2. [57] T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman (2024) Improved distribution matching distillation for fast image synthesis. In NeurIPS, Cited by: §3.2. [58] T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, and T. Park (2024) One-step diffusion with distribution matching distillation. In CVPR, Cited by: §3.2. [59] T. Yin, Q. Zhang, R. Zhang, W. T. Freeman, F. Durand, E. Shechtman, and X. Huang (2025) From slow bidirectional to fast autoregressive video diffusion models. In CVPR, Cited by: §C.1, §1, §4. [60] Y. Zhai, K. Lin, Z. Yang, L. Li, J. Wang, C. Lin, D. Doermann, J. Yuan, and L. Wang (2024) Motion consistency model: accelerating video diffusion with disentangled motion-appearance distillation. In NeurIPS, Cited by: §2. [61] L. Zhang, S. Mo, Z. Cai, J. Lin, Z. Lin, J. Gu, K. K. Singh, Y. Li, and Y. Li (2026) UniTemp: unlocking video generation in any temporal order via bidirectional distillation. In ECCV, Cited by: §2. [62] L. Zhang, S. Cai, M. Li, G. Wetzstein, and M. Agrawala (2025) Frame context packing and drift prevention in next-frame-prediction video diffusion models. In NeurIPS, Cited by: §2. [63] K. Zheng, G. He, M. Zhao, J. Zhang, H. Chen, J. Chen, C. Lin, M. Liu, J. Zhu, and Q. Ma (2026) Causal-rCM: a unified teacher-forcing and self-forcing open recipe for autoregressive diffusion distillation in streaming video generation and interactive world models. arXiv preprint arXiv:2606.25473. Cited by: §1, §2. [64] H. Zhu, M. Zhao, G. He, H. Su, C. Li, and J. Zhu (2026) Causal Forcing: autoregressive diffusion distillation done right for high-quality real-time interactive video generation. In ICML, Cited by: §4. [65] K. Zou, D. Zheng, H. Liu, T. Hang, B. Liu, and N. Yu (2026) HiAR: efficient autoregressive long video generation via hierarchical denoising. In ECCV, Cited by: Figure 1, §1, §1, §2, §2, §4.1, §4. Appendix overview. The appendix is organized into three sections. Sec. A presents the batched multi-device wavefront algorithm; Sec. A.1 explains wavefront execution, anchor spacing, and planner–renderer rate matching, while Sec. A.2 documents the measurement protocol and planner allocations, analyzes latency and peak memory, analyzes the theoretical acceleration ratio with wall-clock break-even conditions, and evaluates streaming generation with a dedicated planner. Sec. B explains why the renderer regenerates anchor positions, using visual and quantitative evidence to separate the effects of the anchor-KV source and the final anchor source. Sec. C gives the full training and implementation details: Sec. C.1 covers packed time-rebased SFT and its optimization, whereas Sec. C.2 describes self-rollout guidance, score tiling, gradient routing, and the optimization. We provide an interactive demo on the project page that compares generation quality and latency with other methods, illustrates our generation process and training phases, and shows examples of the chunk-continuity and anchor–non-anchor seam issues that our framework resolves. Appendix A Generation Efficiency Analysis Throughout this appendix, a rank is one GPU worker, and a renderer round is one synchronized pipeline interval in which every active rank executes at most one chunk-sized forward at its assigned stage. The stage index k∈S,…,1k∈\S,…,1\ denotes the update from noise level σk _k to σk−1 _k-1, while τk=τ(σk) _k=τ( _k) denotes the corresponding model time input. We use context parallelism (CP) for a different form of parallelism: sharding one logical model forward across multiple ranks. A.1 Wavefront Execution and Anchor Spacing Algorithm 1 Batched multi-device wavefront generation. 1: Condition c; length L, positions ℐ=0,…,L−1I=\0,…,L-1\; local anchor-window map Γ ; roles FθP,FθRF^P_θ,F^R_θ. 2: S=4S=4 stages with σS>⋯>σ0=0 _S>·s> _0=0, Δ=10 =10, BP=BR=3B_P=B_R=3, and a context window of Wctx=21W_ctx=21 latent-frame positions for both roles; HP=⌊(Wctx−BP)/BP⌋=6H_P= (W_ctx-B_P)/B_P =6 is the planner-history limit, and H=⌊(Wctx−BP−BR)/BR⌋=5H= (W_ctx-B_P-B_R)/B_R =5 preceding renderer chunks fit in each stage-matched renderer history. 3: M←⌊(L−1)/Δ⌋+1M← (L-1)/ +1 anchors P, NP←⌈M/BP⌉N_P← M/B_P blocks jQ_j, and NR←⌈L/BR⌉N_R← L/B_R chunks iC_i; the last block or chunk contains the remainder when necessary. 4: Ranks r∈0,…,S−1r∈\0,…,S-1\; renderer rank r owns stage k=S−rk=S-r. ℬ(0)k←∅,←∅,∀rB^(0)_k← ,A← ,∀ r. 5: Planning. ⊳ Complete the planner rollout before rendering, ending with A in host memory. 6: for j=0,…,NP−1j=0,…,N_P-1 do 7: Sample xj,SP∼(0,1)x^P_j,S (0,1). 8: for k=S,…,1k=S,…,1 do 9: v^j,kP←FθP(xj,kP,c,τk,TailHP()) v^P_j,k← F^P_θ(x^P_j,k,c, _k;Tail_H_P(A)); step xj,k−1P←xj,kP−(σk−σk−1)v^j,kPx^P_j,k-1← x^P_j,k-( _k- _k-1) v^P_j,k. 10: end for 11: (KjP,VjP)←FθP(xj,0P,c,τ0,TailHP())(K^P_j,V^P_j)← F^P_θ(x^P_j,0,c, _0;Tail_H_P(A)); ←Append(,(KjP,VjP)),∀jA (A,(K^P_j,V^P_j)),∀ j. ⊳ including the last 12: end for 13: Renderer wavefront, NR+S−1N_R+S-1 rounds. 14: for n=0,…,NR+S−2n=0,…,N_R+S-2 do 15: for all r in parallel, with i←n−ri← n-r and k←S−rk← S-r, active iff 0≤i<NR0≤ i<N_R do 16: Obtain xi,kRx^R_i,k: rank 00 samples noise from (0,1)N(0,1); rank r>0r>0 takes what rank r−1r-1 sent in round n−1n-1. 17: Read anchor KV Γ(i)A_ (i) and stage-matched history ℬk(i)B^(i)_k. 18: Evaluate Eq. (2): (v^i,kR,Ki,kR,Vi,kR)←FθR(xi,kR,c,τk,Γ(i),ℬk(i))( v^R_i,k,K^R_i,k,V^R_i,k)← F^R_θ(x^R_i,k,c, _k;A_ (i),B^(i)_k). 19: Step xi,k−1R←xi,kR−(σk−σk−1)v^i,kRx^R_i,k-1← x^R_i,k-( _k- _k-1) v^R_i,k. 20: Write in place: ℬk(i+1)←TailH(Append(ℬk(i),(i,Ki,kR,Vi,kR)))B^(i+1)_k _H(Append(B^(i)_k,(i,K^R_i,k,V^R_i,k))). ⊳ keep the most recent H chunks 21: If r<S−1r<S-1, send xi,k−1Rx^R_i,k-1 to rank r+1r+1; otherwise keep the finished xi,0Rx^R_i,0 on iC_i. 22: end for 23: end for 24: The L latents of ℐI, all produced by the renderer, on the last rank. Wavefront generation. Alg. 1 describes the batched S-device setup used in our main latency matrix. During generation, the devices first run the planner. We could use only one rank to run the planner or use all ranks to run the planner with context parallelism. We discuss the design choice later in details. Either way, the planner will produce the clean anchor KV bank A. Each device then runs one renderer stage and stores its own cache bank. After warm-up, the renderer chunk–stage wavefront in Eq. (2) generates one chunk per round. Anchor spacing and streaming rate. We keep the planner block size equal to the renderer chunk size, BP=BR=3B_P=B_R=3, so that their forwards have comparable cost. The stride Δ=10 =10 is chosen by matching anchor production and consumption in the streaming allocation with one planner device and four renderer devices. One anchor block requires four denoising forwards and one clean-endpoint KV forward, or five planner rounds. Each local three-anchor window then conditions six to seven renderer chunks, which the steady-state wavefront retires in six to seven rounds. The planner can therefore publish the next anchors before the renderer exhausts its current window. A smaller stride can make planning the bottleneck; a larger stride provides no latency gain once planning stays ahead, and leaves longer regions less constrained temporally. A.2 Latency and Peak Memory Analysis Measurement protocol. Throughout the paper, reported latency and FPS refer to denoising-path latency. All results in Tab. 4, Tab. 5, and Fig. 4 use identical timing boundaries. Timing spans the first planner or baseline denoising forward through the availability of completed video latents on the reporting rank. Text encoding and VAE decoding are therefore excluded. Planner denoising and clean-endpoint forwards, renderer forwards, cache operations, pipeline synchronization and communication, and the role-weight transfer described below are included whenever the corresponding schedule performs them. Each cell reports the average over 20 generated videos after excluding the cold-start steps. We reset the memory counters and synchronize all ranks before and after each timed window. For each video, we report the slowest-rank elapsed time and the highest peak allocation across all ranks. Among cells with per-video memory measurements, the standard deviation of peak memory has a median of 0.0000 GiB and a maximum of 0.4028 GiB, so we omit it from the memory table. The three autoregressive methods, Self-Forcing, HiAR, and FlashForward, use three-latent chunks and a context budget of Wctx=21W_ctx=21 latent-frame positions, while the bidirectional baseline does not use caching. For FlashForward, these 21 positions comprise three clean anchor-KV positions in Γ(i)A_ (i), up to 15 renderer-history positions from ℬk(i)B^(i)_k, and up to BR=3B_R=3 positions in the current chunk. The planner uses the same budget: up to 18 completed anchor latents and up to BP=3B_P=3 latents in its current block. We evaluate the 1.3B and 14B models on H100 and GB300 GPUs, respectively. The 14B rows are systems-only timing measurements: they instantiate the same tensor shapes, masks, cache paths, and execution schedules with the 14B backbone. Serving the two roles inside the timed window. Routing latents through adapters would add computation to every forward pass. Therefore, for latency evaluation, we merge each role’s adapter into the backbone, producing two full-weight models. A FlashForward forward is then a standard backbone forward and has the same cost as a baseline forward on the same model. This keeps the cost of a block-sized forward comparable across all three schedules. To limit peak device memory, only the active model is kept on the device. At the anchor-to-renderer boundary, the planner weights are off-loaded and the renderer weights are loaded from pinned host memory. This transfer takes 0.10–0.13 s for the 1.3B model and 0.13–0.17 s for the 14B model. The transfer time is included in the four-device FlashForward latencies reported in Tab. 4 and Fig. 4. Streaming keeps both models resident on their assigned devices. Planner allocations. We support two four-GPU planner allocations. The default allocation (shown as FlashForward in the tables) runs the planner on a single device. FlashForward-CP instead shards each logical planner forward across four devices using context parallelism, which introduces collective-communication overhead. In our experiments, the vanilla version is generally faster for 480p videos, whereas the context-parallel version is faster for 720p videos. We report both variants so that users can choose the best deployment strategy for their target resolution. Sec. A.2.3 evaluates a five-GPU variant with a planner concurrent with the four-device renderer wavefront. Table 4: Per-video denoising-path latency in seconds (lower is better). Each cell is the mean over videos with its standard deviation beneath it. Each setting spans four output lengths: 5, 20, 35, and 65 seconds, which are 21, 81, 141, and 261 latents at 16 FPS. The two FlashForward planner allocations are both measured in every four-GPU cell. All multi-GPU columns use four GPUs unless marked otherwise; FlashForward-Streaming uses one planner GPU and four renderer GPUs. Boldface identifies the fastest schedule at an equal device count and therefore excludes the five-GPU row. Wan2.1-T2V-1.3B (H100) 480p, 1×1×GPU 480p, multi-GPU 720p, multi-GPU Schedule 5 20 35 65 5 20 35 65 5 20 35 65 Bidirectional [-0.15em] 3.31 [-0.15em]± 0.00 32.02 [-0.15em]± 0.01 91.22 [-0.15em]± 0.04 295.83 [-0.15em]± 0.12 1.13 [-0.15em]± 0.01 8.95 [-0.15em]± 0.01 24.01 [-0.15em]± 0.02 75.28 [-0.15em]± 0.05 3.74 [-0.15em]± 0.00 39.59 [-0.15em]± 0.06 115.79 [-0.15em]± 0.04 379.17 [-0.15em]± 0.64 Self-Forcing [-0.15em] 3.69 [-0.15em]± 0.01 17.44 [-0.15em]± 0.11 31.48 [-0.15em]± 0.14 57.89 [-0.15em]± 0.20 5.10 [-0.15em]± 0.05 20.25 [-0.15em]± 0.10 35.40 [-0.15em]± 0.31 65.14 [-0.15em]± 0.55 6.04 [-0.15em]± 0.05 27.17 [-0.15em]± 0.11 47.71 [-0.15em]± 0.27 97.68 [-0.15em]± 0.48 HiAR [-0.15em] 5.57 [-0.15em]± 0.04 27.86 [-0.15em]± 0.03 50.41 [-0.15em]± 0.20 94.74 [-0.15em]± 0.15 2.06 [-0.15em]± 0.05 7.67 [-0.15em]± 0.00 13.31 [-0.15em]± 0.01 24.67 [-0.15em]± 0.03 6.49 [-0.15em]± 0.01 26.82 [-0.15em]± 0.02 46.94 [-0.15em]± 0.03 86.85 [-0.15em]± 0.06 FlashForward [-0.15em] 4.56 [-0.15em]± 0.07 17.77 [-0.15em]± 0.14 32.80 [-0.15em]± 0.21 59.64 [-0.15em]± 0.46 2.21 [-0.15em]± 0.08 6.32 [-0.15em]± 0.02 10.79 [-0.15em]± 0.22 21.29 [-0.15em]± 0.08 5.71 [-0.15em]± 0.05 19.33 [-0.15em]± 0.27 35.14 [-0.15em]± 0.13 66.53 [-0.15em]± 0.12 FlashForward-CP [-0.15em] - - - - 2.34 [-0.15em]± 0.02 7.24 [-0.15em]± 0.06 12.30 [-0.15em]± 0.28 22.07 [-0.15em]± 0.20 5.50 [-0.15em]± 0.03 18.47 [-0.15em]± 0.35 30.90 [-0.15em]± 0.38 58.89 [-0.15em]± 0.18 FlashForward-Streaming [-0.15em](5×5×GPU) - - - - 1.85 [-0.15em]± 0.01 5.11 [-0.15em]± 0.01 8.27 [-0.15em]± 0.01 14.53 [-0.15em]± 0.02 5.47 [-0.15em]± 0.00 16.37 [-0.15em]± 0.02 27.05 [-0.15em]± 0.05 48.61 [-0.15em]± 0.07 Wan2.1-T2V-14B (GB300) 480p, 4×4×GPU 720p, 4×4×GPU Schedule 5 20 35 65 5 20 35 65 Bidirectional [-0.15em] 3.50 [-0.15em]± 0.01 31.83 [-0.15em]± 0.01 90.23 [-0.15em]± 0.01 295.45 [-0.15em]± 0.01 12.79 [-0.15em]± 0.01 154.47 [-0.15em]± 0.01 454.53 [-0.15em]± 0.01 1530.42 [-0.15em]± 0.03 Self-Forcing [-0.15em] 9.75 [-0.15em]± 0.09 41.35 [-0.15em]± 0.15 74.56 [-0.15em]± 0.17 137.89 [-0.15em]± 0.37 15.84 [-0.15em]± 0.03 78.88 [-0.15em]± 0.07 142.29 [-0.15em]± 0.11 269.41 [-0.15em]± 0.20 HiAR [-0.15em] 5.64 [-0.15em]± 0.00 22.79 [-0.15em]± 0.02 39.63 [-0.15em]± 0.02 73.03 [-0.15em]± 0.03 21.72 [-0.15em]± 0.00 93.25 [-0.15em]± 0.03 163.66 [-0.15em]± 0.05 304.35 [-0.15em]± 0.08 FlashForward [-0.15em] 4.79 [-0.15em]± 0.04 15.67 [-0.15em]± 0.11 27.67 [-0.15em]± 0.22 53.43 [-0.15em]± 0.41 17.19 [-0.15em]± 0.09 61.39 [-0.15em]± 0.34 109.89 [-0.15em]± 0.03 214.83 [-0.15em]± 0.54 FlashForward-CP [-0.15em] 5.23 [-0.15em]± 0.02 16.66 [-0.15em]± 0.12 28.10 [-0.15em]± 0.33 52.18 [-0.15em]± 0.27 16.28 [-0.15em]± 0.07 55.66 [-0.15em]± 0.20 97.02 [-0.15em]± 0.50 181.25 [-0.15em]± 0.80 FlashForward-Streaming [-0.15em](5×5×GPU) 4.72 [-0.15em]± 0.05 13.40 [-0.15em]± 0.02 22.35 [-0.15em]± 0.05 39.43 [-0.15em]± 0.32 17.44 [-0.15em]± 0.10 53.17 [-0.15em]± 0.14 88.79 [-0.15em]± 0.01 160.34 [-0.15em]± 0.27 Table 5: Peak allocated memory per rank in GiB. Wan2.1-T2V-1.3B (H100) 480p, 1×1×GPU 480p, 4×4×GPU 720p, 4×4×GPU Schedule 5 20 35 65 5 20 35 65 5 20 35 65 Bidirectional 22.1 31.7 41.2 60.3 15.7 18.4 21.1 26.5 17.0 23.2 29.4 41.8 Self-Forcing 24.9 24.9 25.0 25.1 20.6 20.6 20.7 20.7 28.1 28.2 28.3 28.5 HiAR 24.9 24.9 25.0 25.1 24.9 24.9 25.0 25.1 32.9 33.0 33.1 33.3 FlashForward 24.9 25.0 25.2 25.4 24.9 24.9 25.0 25.1 32.9 33.0 33.1 33.3 Wan2.1-T2V-14B (GB300) 480p, 4×4×GPU 720p, 4×4×GPU Schedule 5 20 35 65 5 20 35 65 Bidirectional 28.8 34.0 39.1 49.4 31.2 43.1 54.9 78.7 Self-Forcing 60.1 60.2 60.2 60.3 93.2 93.3 93.4 93.6 HiAR 79.0 79.0 79.1 79.1 112.8 112.9 113.0 113.2 FlashForward 79.0 79.0 79.1 79.2 112.8 112.9 113.0 113.3 A.2.1 Analysis Generation latency. Among the four-GPU schedules in Tab. 4, the faster of the two FlashForward planner allocations is the fastest in every model–resolution setting from 20 seconds onward, and it retains this lead at both 35 and 65 seconds. At 65 seconds, FlashForward reduces latency by 3.543.54–8.44×8.44× relative to bidirectional generation, 1.491.49–3.06×3.06× relative to Self-Forcing, and 1.161.16–1.68×1.68× relative to HiAR across these settings. The preferred planner allocation follows the workload: vanilla is generally faster at 480p, whereas context parallelism consistently wins at 720p. Before amortization at 5 seconds, the bidirectional baseline remains fastest in all five settings. On a single GPU, where the renderer cannot exploit the inter-chunk stage pipeline, FlashForward is nevertheless 1.541.54–1.59×1.59× faster than HiAR for 20–65-second outputs because it removes most of HiAR’s cache-update-only forwards. Compared with Self-Forcing, FlashForward also reduces cache-update-only forwards, but it materializes and stores renderer-history KV at all four denoising stages, rather than once per chunk; these writes occur inside output-advancing forwards but still carry bookkeeping cost. These two effects largely offset each other, leaving FlashForward close to Self-Forcing in latency and only 1.91.9–4.2%4.2\% slower. Peak memory. Tab. 5 reports the memory allocation. Bidirectional and Self-Forcing require less per-rank memory thanks to context parallelism. Because the autoregressive methods use a fixed sliding window, their peak memory is nearly independent of video length. The reported FlashForward allocation remains within 0.3 GiB of HiAR’s in every cell. A.2.2 Discussion: When Is the Pipeline Faster? Beyond the empirical latency results above, we provide a complementary theoretical comparison of the three autoregressive schedules. Our goal is to characterize the conditions under which FlashForward is expected to be faster. We begin with their forward counts. Figure 6: Forward-count break-even boundary between FlashForward and HiAR when the planner uses context parallelism, evaluated in the sixteen four-GPU cells of Tab. 4. Marker shape denotes the system setting; marker color reports whether FlashForward is the fastest measured schedule among FlashForward, HiAR, and Self-Forcing, using the same two colors as the predicted regions containing the markers. The two gray markers are the cells in which FlashForward does not win. A marker whose color disagrees with its region is a cell in which the pairwise boundary alone does not recover the three-way winner. Both outliers occur at a length of 21 latents, suggesting that FlashForward is not well amortized for short sequences in these settings. Forward-count model. We compare the three autoregressive schedules in logical block-sized forward equivalents. Let NP=⌈M/BP⌉N_P= M/B_P be the number of planner blocks, NR=⌈L/BR⌉N_R= L/B_R the number of renderer chunks, and NC=⌈L/BC⌉N_C= L/B_C the number of chunks for either autoregressive baseline. All schedules use BC=BR=3B_C=B_R=3. For S denoising stages, their full-video logical work is WFlashForward W_FlashForward =NP(S+1)+NRS, =N_P(S+1)+N_RS, (3) WSelf-Forcing W_Self-Forcing =NCS+(NC−1), =N_CS+(N_C-1), WHiAR W_HiAR =NCS+(NC−1)S. =N_CS+(N_C-1)S. The extra 11 per planner block accounts for clean-KV construction, while the terminal cache-update-only forward is omitted for the baselines because it has no consumer. At 81 latents, NP=3N_P=3, NR=NC=27N_R=N_C=27, and S=4S=4, giving 123, 134, and 212 forwards, respectively. On four devices, FlashForward and HiAR assign different denoising stages to different ranks, whereas Self-Forcing remains a serial chain whose individual logical forwards use context parallelism; FlashForward’s planner also remains serial across anchor blocks. We therefore use two workload-specific CP speedups. Let γplan _plan be the latency of one complete one-rank planner rollout divided by that of its four-rank CP rollout, and let γSF _SF be the latency of a one-rank Self-Forcing logical forward divided by that of its four-rank CP counterpart under the same setting. When does the forward reduction become a wall-clock speedup? Including the S−1S-1 wavefront fill/drain slots, the loads and two break-even conditions on S-GPUs are RFF R_F =NR+NP(S+1)γplan+(S−1), =N_R+ N_P(S+1) _plan+(S-1), (4) RSF R_SF =NCS+(NC−1)γSF, = N_CS+(N_C-1) _SF, RHiAR R_HiAR =2NC−1+(S−1), =2N_C-1+(S-1), RFF<RHiAR R_F<R_HiAR ⇔γplan>NP(S+1)2NC−1−NR, _plan> N_P(S+1)2N_C-1-N_R, RFF<RSF R_F<R_SF ⇔γSF<NCS+(NC−1)RFF. _SF< N_CS+(N_C-1)R_F. Against HiAR, the required γplan _plan falls from 0.83 at 21 latents to 0.58, 0.54, and 0.52 at 81, 141, and 261 latents. As for γplan<1 _plan<1 we could simply use one rank to run the planner and degenerates γplan=1 _plan=1, we can always has less loads for model forwards compared with HiAR. Against Self-Forcing, even the conservative choice γplan=1 _plan=1 allows γSF _SF to rise from 2.27 to 2.98, 3.12, and 3.21 over the same lengths before the ordering reverses. Thus we can beat Self-Forcing in a large interval. Fig. 6 compares this prediction with the measurements in Tab. 4. For each model, resolution, and output length, a point to the right of the dashed boundary predicts that FlashForward is faster than HiAR, while a point to the left predicts the reverse; marker color reports whether FlashForward is the fastest measured schedule of the three, so a marker whose color disagrees with the background region containing it is a cell in which the pairwise boundary alone does not recover the three-way winner. The pairwise FlashForward–HiAR prediction is correct in fifteen of the sixteen cells. When all three schedules are compared, FlashForward is fastest in fourteen cells: the only exceptions are the two 21-latent settings in which HiAR wins at 1.3B 480p and Self-Forcing wins at 14B 720p. From 81 latents onward, FlashForward is fastest in every model–resolution setting. As videos become longer and each forward becomes more compute-intensive through a larger model or higher resolution, the measured context-parallel speedup tends to improve, while the break-even requirements above become less restrictive and fixed pipeline costs are better amortized. The conclusion that FlashForward is faster than both baselines is therefore most robust for long, large-model, and high-resolution generation. For small forwards with γplan<1 _plan<1, the same analysis gives a direct deployment rule: use the planner on one rank rather than use context parallelism. From forward counts to wall-clock speedup. The forward-count ratio is an upper-bound estimate rather than an exact latency prediction: in practice, context parallelism provides sublinear speedup because of collective overhead, and the wavefront pays fixed fill, drain, and synchronization costs. The inter-stage latent tensor is small and its transfer overlaps the next forward, so communication volume is not the dominant term. At 81 latents, the raw counts suggest a 212/123=1.72×212/123=1.72× advantage over HiAR, while the measured speedup is 1.211.21–1.68×1.68× across the four settings. The gap is largest for short, low-compute workloads, but these costs are progressively amortized as the workload grows; at 14B and 720p, the measured 1.68×1.68× speedup approaches the theoretical 1.72×1.72×, showing that the forward-count advantage translates into practical gains. A.2.3 Streaming Generation with a Dedicated Planner The four-device configuration in our main latency matrix reuses the same devices for both roles: all four first complete the sparse-anchor rollout and are then assigned to the four stages of the renderer wavefront to make a fair comparison with other autoregressive methods. This serial planner prologue increases the time to the first completed latent chunk (our TTFF boundary), but it is an implementation choice rather than a dependency imposed by FlashForward: a renderer chunk needs only its local anchor window, not anchors from later windows. For deployment, we therefore dedicate one GPU to the planner and keep four GPUs on the four-stage renderer wavefront. As soon as the planner publishes the first anchor block, rendering can begin; the planner then produces later blocks concurrently with rendering. This reduces the anchor contribution to TTFF from a complete planner rollout to one block and, more importantly, supports streaming generation. Measured five-GPU latency. We measure this one-planner–four-renderer schedule on both models, as reported in the FlashForward-Streaming (5×5×GPU) rows of Tab. 4. Here we compare against the four-GPU FlashForward row, which uses the same planner algorithm without CP; the comparison adds one device and measures the deployment benefit of overlapping that planner with rendering. At 21 latents (5 seconds), there is only one anchor block and hence no later planner work to overlap. The differences at this length therefore reflect model residency and, for the five-GPU 14B setup, cross-node communication rather than pipeline overlap. As sequence length increases, the overlap benefit dominates. From 20 to 65 seconds, the dedicated H100 planner makes 1.3B generation 1.181.18–1.47×1.47× faster at both 480p and 720p. The 14B configuration spans two GB300 nodes, so each anchor block crosses the inter-node link and communication time grows with sequence length, but most anchor generation remains hidden behind rendering: relative to the four-GPU single-planner row, 14B generation is 1.151.15–1.36×1.36× faster over the same lengths. In both deployments, the overlap leaves renderer peak memory essentially unchanged from the standard four-GPU version. Together, these results show that a dedicated producer converts the planner prologue into a one-block streaming warm-up and yields increasing denoising-path latency gains as the pipeline is amortized. Only the first anchor block is exposed. To test whether the planner lies on the steady-state critical path, we separately record the producer’s full anchor time and anchor_wait, the time for which renderer ranks wait for unavailable anchors. At 480p, the full anchor phase grows from 0.510 to 5.617 s as the number of planner blocks grows from one to nine, while anchor_wait remains between 0.506 and 0.509 s. At 720p, the full phase grows from 1.067 to 17.956 s, while the wait remains between 1.051 and 1.066 s. Thus the exposed fraction of planning falls from 99–100% for a single block to 9% at 480p and 6% at 720p for nine blocks. The wait is always approximately the cost of the first block: after startup, the planner stays ahead and all later anchor generation is hidden beneath the renderer wavefront, as predicted by the rate-matching analysis in Sec. A.1. Appendix B Why the Renderer Regenerates the Anchor Positions Cheaper variants that do not work. A cheaper design would keep the planner anchors in the final video and render only the positions between them. We test two versions of this splice design. The first reuses the KV produced by the planner, while the second passes the clean auxiliary anchor latents through the renderer to build renderer-side KV. Both versions mix two output sources: planner latents at the anchor positions and renderer latents everywhere else. Our method instead uses the planner anchors only as conditioning and lets the renderer generate every output position. This comparison separates the effect of the anchor KV source from the effect of the final output source. Experiments. To evaluate these choices, we run the experiments summarized in Tab. 6. All experiments use the same training setup as our final model. Panel (a) compares the splice variants with all-position rendering. Panel (b) measures how the mean luminance levels of planner anchor frames and neighboring renderer frames change across training checkpoints for these splice variants. Panel (c) replaces generated planner anchors with latents encoded from real videos to test whether planner generation error alone explains the seam. What the seam looks like. Fig. 7 visualizes this comparison on four 20-second videos, using the same frame indices for all three variants. In (a) and (b), the anchor frames of both splice variants are darker than their neighbors; under renderer-side KV, the rocks in (c) turn lighter and warmer; and in (d), the bowl turns a brighter red. The blurred differences make the offset visible. For both splice variants, the differences highlight the static background as well as the moving subject, indicating a color change across the whole frame rather than motion. For FlashForward, they stay dark except where large regions move, such as the candle smoke in (a), the bear’s body in (b), and the steam above the bowl in (d). The KV source changes how strongly the offset appears in each clip, but neither splice variant removes it, consistent with Panel (a). We include videos of the two variants on the project page; they show more directly what the seam looks like in motion. Figure 7: Color seams at the anchor positions of 20-second videos. The two splice rows keep the planner anchor latents in the final video and build the anchor KV with the planner or with the renderer, as in Panel (a) of Tab. 6; FlashForward uses the planner anchors only as conditioning and lets the renderer generate every output position. In each block, the boxed column is the last frame decoded from an anchor latent, and the outer columns are the nearest non-anchor frames before and after it. Between each non-anchor frame and the anchor frame, we show their per-channel absolute RGB difference, computed after a Gaussian blur and amplified 16×16×. The blur suppresses edges and fine texture, leaving mostly low-frequency color changes; large moving regions can remain visible. Table 6: All-position renderer output leads to smaller seam measurements. Panel (a) compares anchor-KV and final-anchor sources. All seam columns are normalized adjacent-frame luminance-jump statistics, and lower is better: Anchor seam is measured at anchor/non-anchor transitions, Chunk seam at renderer-chunk boundaries away from anchors, and Interior at transitions of neither type. “Periodic” sums the excess over Interior across the transitions in one anchor period. Panel (b) reports mean luminance levels for planner-generated anchor frames and their neighboring renderer-generated frames. Panel (c) compares real and generated anchors over paired clips; chromatic share is the fraction of the RGB offset orthogonal to the grayscale direction (1,1,1)(1,1,1), and spatial excess is sanchor2−scontrol2 s_anchor^2-s_control^2, where s is the spatial standard deviation of the per-cell RGB change. The paired t-statistic compares each clip’s anchor-boundary offset with its mid-chunk control. (a) Compared modes: KV source and final anchor source Denoising steps Final anchor Anchor KV Anchor seam Chunk seam Interior Periodic 50 Planner Planner 3.93 2.71 1.32 7.66 50 Planner Renderer 3.34 2.37 1.29 5.99 50 Renderer (ours) Planner 1.83 2.27 1.32 3.54 4 Planner Renderer 6.20 1.68 1.40 9.55 4 Renderer (ours) Planner 2.43 2.50 1.26 5.61 (b) Seam formation when planner anchors are used as final outputs Checkpoint Anchor level Renderer level Anchor −- renderer 50-step supervised 87.804 87.898 −0.094-0.094 4-step distilled, backbone frozen 98.192 99.364 −1.172-1.172 4-step distilled, backbone updated 130.043 136.287 −6.244-6.244 Paired change: updated −- frozen +31.85+31.85 +36.92+36.92 −5.07-5.07 (c) Compared modes: real anchors and generated planner anchors in spliced output Anchor source Anchor offset Mid-chunk control Chromatic share Spatial excess Paired t-statistic Real-video anchors 9.47 0.52 49% 6.69 4.34 Generated planner anchors 3.00 0.37 47% 4.62 11.51 At 50 steps, the final output source has the larger measured effect. Panel (a) gives the direct comparison. At 50 steps, replacing planner KV with renderer-side KV only modestly reduces the anchor seam and periodic total, and the seam remains visible. These one-factor comparisons show that changing the final anchor source has the larger measured effect at this phase. In the deployed four-step comparison, all-position rendering again cuts the anchor seam by more than half relative to the splice configuration and clearly lowers the periodic total. Planner–renderer output levels diverge after distillation. A splice baseline works only if planner anchors and renderer frames have matching output statistics. Panel (b) shows that they nearly match after supervised fine-tuning, with a negligible mean luminance gap. Four-step distillation widens this gap by more than an order of magnitude even with a frozen backbone, and updating the shared backbone widens it several times further. Relative to the frozen model, updating the backbone raises both levels, but the renderer level rises more than the anchor level, so the anchor path lags behind the renderer’s shift. This unequal shift becomes visible when planner anchors are inserted into output. The mismatch is not simply poor planner output. Panel (c) shows that the seam remains when generated planner anchors are replaced with anchors encoded from real videos. The real-anchor probe removes planner generation error, yet all clips still have more error at the anchor boundary than at the mid-chunk control. The mismatch also has more than one component: 47–49% of the frame-level offset is chromatic, and the spatially varying excess remains large for both real and generated anchors. These results are consistent with an output-interface mismatch rather than an artifact explained solely by a bad planner sample or a single brightness offset. Why all-position rendering avoids cross-role splicing. Our method does not require planner and renderer outputs to match at a splice boundary. The planner anchors provide clean structural guidance, but they are not copied into the final video. The renderer generates every output latent, including the anchor positions, so the final sequence never switches between planner and renderer outputs. This removes the cross-role output switch as one possible source of a periodic seam while preserving the anchors as conditioning, and thus leads to a substantial reduction of anchor seams. Appendix C Training and Implementation Details Given a clean latent x0x_0, Gaussian noise ε , and a noise level σ, Rectified Flow [32] uses xσ=(1−σ)x0+σε,v=ε−x0.x_σ=(1-σ)x_0+σ , v= -x_0. (5) We use a flow shift of 5.0 at every denoising stage. We reserve k∈S,…,1k∈\S,…,1\ for a discrete denoising-stage index. For any noise level σ, let τ(σ)τ(σ) denote the scalar model time input produced by the Wan scheduler with flow shift 5.0; at an inference stage, τk=τ(σk) _k=τ( _k). During distillation and inference, the student generator follows the four-stage guidance-free trajectory. Only the frozen real-data score uses classifier-free guidance [18], with a scale of 3.0, to construct the distillation target. We provide an interactive demo on the project page to illustrate inputs and training phases. C.1 Phase 1: Time-Rebased SFT and Optimization Algorithm 2 Packed SFT forward. 1: Captioned real videos; frozen VAE ℰE and text encoder; trainable parameterization θ comprising the shared backbone, rank-256256 role adapters ψP,ψR _P, _R, and two zero-initialized role-embedding vectors. 2: L=81L=81, =0,10,…,80P=\0,10,…,80\, H=5H=5, Γ(i) (i), flow shift 5.05.0. 3: Precompute the attention mask M with the following visible keys: 4: Vis(ρ[j])=ρ[j]∪0[⋃q<jq]Vis(P^ρ[Q_j])=P^ρ[Q_j] ^0[ _q<jQ_q], ρ∈0,σρ∈\0,σ\; ⊳ same for rebased anchors 5: Vis(ℐσ[i])=ℐσ[⋃q=max(0,i−H)iq]∪0[Γ(i)]Vis(I^σ[C_i])=I^σ[ _q= (0,i-H)^iC_q] ^0[ (i)]. 6: for each training step do 7: Encode x0←ℰ(clip)x_0 (clip) on ℐI; c←c← caption, with 10%10\% text dropout. 8: Rebase: Sample distinct o1,…,o4∈1,…,9o_1,…,o_4∈\1,…,9\; for each n, reset ℰE, x~0(n)←ℰ(clip[4on,4on+281)) x^(n)_0 (clip[4o_n,4o_n+281)), keep its 88 anchors at indices 0,10,…,70\0,10,…,70\. 9: Pack z from planner copies 0,σP^0,P^σ, the renderer copy ℐσI^σ, and rebased planner copies (~n0,~nσ)n=14\( P^0_n, P^σ_n)\_n=1^4. ⊳ a clean copy only where a later block reads it 10: Noise: Sample one σ per clip and set its model time input τσ←τ(σ) _σ←τ(σ); xσ←(1−σ)x0+σεx_σ←(1-σ)x_0+σ ; clean copies stay at σ0=0 _0=0. ⊳ copies of one latent share ε 11: Route planner copies via ψP _P and renderer copies via ψR _R; add the corresponding role embedding to AdaLN. ⊳ one θ gives FθPF^P_θ and FθRF^R_θ 12: Forward (v^P,v^R,v~^nP)←Fθ(z,c,τσ,)( v^P, v^R,\ v^P_n\)← F_θ(z,c, _σ;M) on noised tokens; clean copies provide KV only. 13: Score v←ε−x0v← -x_0 (Eq. (5)); ℓ←∥v^−v∥2 ← v-v ^2; average ℓ¯P _P, ℓ¯R _R, ℓ¯Preb ^\,reb_P over the corresponding inputs; 14: ℒ←ℓ¯P+ℓ¯R+ℓ¯Preb ← _P+ _R+ ^\,reb_P. 15: Update: Backpropagate once; take an AdamW step; update the EMA at decay 0.9990.999. 16: end for 17: The trained model and its EMA copy. Packed input sequence and attention mask. Alg. 2 packs both roles into one forward. For a clip of L=81L=81 latents, the planner uses the M=9M=9 anchors =0,10,…,80P=\0,10,…,80\ in three blocks of three latents, while the renderer treats all 8181 positions as targets in 2727 chunks of three and reads up to 55 preceding chunks. Each anchor appears as a clean planner-context copy in 0P^0, a noised planner input in σP^σ, and a noised renderer input in ℐσI^σ; every non-anchor appears only as a renderer input in ℐσI^σ. All noised copies use the clip’s single sampled σ, and copies of the same latent share ε . The attention mask isolates the planner, renderer, and time-rebased sequences: planner inputs attend within their current block and to clean earlier planner blocks, whereas each renderer chunk attends within the chunk, to up to five preceding renderer chunks at the same noise level, and to the clean anchors selected by Γ(i) (i). The renderer’s read of these clean anchors is the only cross-role connection. Training on 20-second real videos. Real-video supervision is essential for learning the planner’s autoregressive dependence across anchor blocks. Under a 5-second training setup, each clip contains only three anchor latents, thus provides no supervision for autoregressive generation. Distillation-based methods learn from trajectories produced by a teacher model; unless their training pipelines are substantially redesigned, their supervision horizon is typically limited to the teacher’s 5-second generation window [59, 21]. Because our SFT phase learns directly from real videos rather than teacher trajectories, it can use clips of arbitrary duration. Balancing stronger long-horizon supervision against training efficiency, we use 20-second clips, which contain nine anchors and form exactly three anchor blocks, ensuring that the planner’s autoregressive generation is explicitly trained on real data. Time-rebased planner supervision. At each step, we sample four distinct latent-time offsets o1,…,o4o_1,…,o_4 from 1,…,9\1,…,9\. Since the causal VAE downsamples time by four, offset ono_n defines the 281281-frame crop [4on,4on+281)[4o_n,4o_n+281). The resulting 7171 latents provide eight planner targets at local positions 0,10,…,70\0,10,…,70\. These targets form blocks of sizes 3+3+23+3+2. Only the first two blocks have clean context copies because the final block has no later consumer, so each rebased crop adds eight noised inputs and six clean copies and remains isolated from the other packed sequences. Objective and optimization. The shared backbone is initialized from Wan2.1-T2V-1.3B [49]; its planner and renderer rank-256256 LoRA [20] adapters act on the self-attention query, key, value, and output projections and on the cross-attention query and output projections. One zero-initialized vector per role is selected from a role-embedding table and added through adaptive layer normalization (AdaLN). The VAE encoder remains frozen. We apply per-token squared error to the rectified-flow velocity v=ε−x0v= -x_0 and sum the averages over the 99 original planner latents, 8181 renderer latents, and 3232 time-rebased planner latents, respectively. Supervised training uses AdamW [33] for 1,8001,800 steps of global batch size 128128, β1=0 _1=0, β2=0.999 _2=0.999, weight decay 0.010.01, and gradient clipping at 1.01.0. The learning rate warms up to 10−510^-5 over the first 100100 steps and then remains constant, and the exponential-moving-average (EMA) decay is 0.9990.999. C.2 Phase 2: Self-Rollout Guidance and Step Distillation Self-rollout and gradient routing. Phase 2 starts from the Phase-1 student and retains the same planner–renderer layout. Its four guidance-free denoising stages select the following base scheduler indices, ordered from highest to lowest noise: student=(q4,q3,q2,q1)=(999,937,833,624).T_student=(q_4,q_3,q_2,q_1)=(999,937,833,624). (6) Algorithm 3 Self-rollout distribution-matching distillation. 1: Trainable SFT student; frozen real-data score FrealF^real (Wan2.1-T2V-14B); critic FϕfakeF^fake_φ (Wan2.1-T2V-1.3B, with a trainable rank-256256 LoRA); frozen VAE ℰE, D. 2: L=81L=81, M=9M=9 anchors, NP=3N_P=3, BP=3B_P=3, NR=27N_R=27, BR=3B_R=3, S=4S=4, H=5H=5, Wscore=21W_score=21. 3: function Rollout(c) 4: ←∅A← . 5: Exit: Sample k⋆∼1,…,Sk \1,…,S\; stages above k⋆k run detached. ⊳ the rollout stops at k⋆k 6: for j=0,10,20,30,40,50,60,70,80Q_j=\0,10,20\,\30,40,50\,\60,70,80\ do 7: xj,SP←x^P_j,S← noise on jQ_j. 8: for k=S,…,k⋆k=S,…,k do 9: v^j,kP←FθP(xj,kP,c,τk,sg[]) v^P_j,k← F^P_θ(x^P_j,k,c, _k;sg[A]). 10: If k>k⋆k>k , step to xj,k−1Px^P_j,k-1. 11: end for 12: x^jP←xj,k⋆P−σk⋆v^j,k⋆P x^P_j← x^P_j,k - _k v^P_j,k ; run a cache-extraction forward at σ0=0 _0=0; ←Append(,(KjP,VjP))A (A,(K^P_j,V^P_j)). 13: end for 14: Pack the L samples from (0,1)N(0,1), under the block-causal mask of Alg. 2, with x^P x^P entering through A. 15: for k=S,…,k⋆k=S,…,k do 16: In one packed forward at τk _k, evaluate v^i,kR←FθR(xi,kR,c,τk,Γ(i),ℬk(i)) v^R_i,k← F^R_θ(x^R_i,k,c, _k;A_ (i),B^(i)_k) for every i; the mask realizes each same-stage ℬk(i)B^(i)_k from the preceding H chunks in this forward. 17: If k>k⋆k>k , set xi,k−1R←xi,kR−(σk−σk−1)v^i,kRx^R_i,k-1← x^R_i,k-( _k- _k-1) v^R_i,k for every i. 18: end for 19: x0,i←xi,k⋆R−σk⋆v^i,k⋆Rx_0,i← x^R_i,k - _k v^R_i,k for every i; return (x0,k⋆)(x_0,k ). 20: end function 21: function TiledScore(F, x0x_0, xσ~x_ σ, εh ^h, c, σ~ σ, τ~ τ, g) 22: Tile ℐI into (L−1)/(Wscore−1)=4(L-1)/(W_score-1)=4 tiles T0=[0,21)T_0=[0,21), T1=[21,41)T_1=[21,41), T2=[41,61)T_2=[41,61), T3=[61,81)T_3=[61,81). 23: imgT0,imgT1,imgT2←(x0 before T3))img_T_0,img_T_1,img_T_2 (x_0 before T_3));hℓ,0←ℰ(imgTℓ−1),∀ℓ∈1,2,3h_ ,0 (img_T_ -1),∀ ∈\1,2,3\. 24: v^[T0]←F(xσ~[T0],c,τ~) v[T_0]← F(x_ σ[T_0],c, τ) at guidance g. 25: for ℓ=1,2,3 =1,2,3 do 26: hℓ,σ~←sg[(1−σ~)hℓ,0+σ~εh]h_ , σ [(1- σ)h_ ,0+ σ ^h]. 27: uℓ←hℓ,σ~|xσ~[Tℓ]u_ ← h_ , σ\|x_ σ[T_ ]; evaluate F(uℓ,c,τ~)F(u_ ,c, τ) at guidance g and store the prediction in v^[Tℓ] v[T_ ]. 28: end for 29: return v v on ℐI; discard the image-head predictions. 30: end function 31: for each training step with caption c do 32: if generator phase then 33: (x0,k⋆)←(x_0,k )←Rollout(c). 34: Sample σ~ σ, set τ~←τ(σ~) τ←τ( σ), and sample ε,εh∼(0,I) , ^h (0,I); xσ~←(1−σ~)x0+σ~εx_ σ←(1- σ)x_0+ σ . 35: v^real← v^real←TiledScore(Freal,x0,xσ~,εh,c,σ~,τ~,3.0F^real,x_0,x_ σ, ^h,c, σ, τ,3.0). 36: v^fake← v^fake←TiledScore(Fϕfake,x0,xσ~,εh,c,σ~,τ~,1.0F^fake_φ,x_0,x_ σ, ^h,c, σ, τ,1.0). 37: x^0real←xσ~−σ~v^real x_0^real← x_ σ- σ v^real; x^0fake←xσ~−σ~v^fake x_0^fake← x_ σ- σ v^fake. 38: y←sg[x0−(x^0fake−x^0real)/|x0−x^0real|]y [\,x_0- ( x_0^fake- x_0^real ) /E |x_0- x_0^real |\,] ⊳ distribution-matching target 39: ℒgen←1L∑p∈ℐ12‖x0,p−yp‖2L_gen← 1L _p 12\|x_0,p-y_p\|^2; backpropagate; take a student AdamW step; update the student EMA. 40: else 41: (x0,k⋆)←(x_0,k )←Rollout(c); sample σ~ σ, set τ~←τ(σ~) τ←τ( σ), and sample ε,εh∼(0,I) , ^h (0,I); form xσ~x_ σ as above. 42: v^fake← v^fake←TiledScore(Fϕfake,x0,xσ~,εh,c,σ~,τ~,1.0F^fake_φ,x_0,x_ σ, ^h,c, σ, τ,1.0). 43: ℒϕ←1L∑p∈ℐ‖v^pfake−(εp−x0,p)‖2L_φ← 1L _p \| v^fake_p-( _p-x_0,p)\|^2. 44: Backpropagate ℒϕL_φ through FϕfakeF^fake_φ only; take a critic AdamW step. 45: end if 46: end for 47: The distilled four-step guidance-free student and its EMA model. The terminal endpoint is σ0=0 _0=0. At each iteration, all ranks sample the same student exit k⋆k uniformly from the four stages and retain gradients only at that stage, following the stochastic gradient truncation of Self-Forcing [21]; the higher-noise stages run under stop-gradient. The rollout first generates the three planner blocks and then all L=81L=81 renderer positions, stopping at k⋆k and using the corresponding x^0 x_0 as its output. A clean-endpoint cache-extraction forward after each planner block constructs A. We detach this bank when it is read by later planner blocks but keep the renderer’s read of Γ(i)A_ (i) connected at the student exit. Since the renderer produces every output position, this renderer-to-anchor connection is the only path by which the generator loss trains the planner. Alg. 3 gives the complete rollout and loss computation. At a fixed renderer stage, training evaluates all chunks in one packed forward. The block-causal renderer mask exposes to chunk i only its own latents, the anchor window Γ(i)A_ (i), and at most the H preceding chunks from that same stage. Inference materializes the equivalent same-stage bank sequentially through Eq. (2), whereas the packed training forward retains the same dependencies and permits gradients through their KV edges. Windowed teacher and critic scores. Each score network accepts at most Wscore=21W_score=21 latent-frame positions. Alg. 3 therefore divides the query into four target tiles while preserving the full-length student rollout and gradient path. Following LongLive [54], we prepend to each later tile a detached image head obtained by re-encoding the last RGB frame of the decoded clean prefix. This head predictions are discarded after getting the scores. Optimization. The frozen real-score is Wan2.1-T2V-14B, and the critic is Wan2.1-T2V-1.3B with a trainable rank-256256 LoRA; the full student remains trainable. We run 600600 AdamW steps of global batch size 128128 at a 1:51:5 generator/critic ratio (100100 generator updates). Generator and critic-LoRA learning rates are 10−510^-5 and 10−610^-6, respectively; EMA decay is 0.9990.999. The real-data score and critic use guidance scales of 3.03.0 and 1.01.0, respectively.