Paper deep dive
Efficient Video Diffusion with Sparse Information Transmission for Video Compression
Mingde Zhou, Zheng Chen, Yulun Zhang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/22/2026, 6:05:11 AM
Summary
Diff-SIT is a novel video compression framework that combines a Sparse Temporal Encoding Module (STEM) for efficient bitrate-saving backbone and motion vector compression with a One-Step Video Diffusion with Frame Type Embedder (ODFTE) for high-quality, temporally coherent generative reconstruction.
Entities (5)
Relation Signals (4)
Diff-SIT → comprises → STEM
confidence 100% · Diff-SIT, which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE)
Diff-SIT → comprises → ODFTE
confidence 100% · Diff-SIT, which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE)
ODFTE → utilizes → FTE
confidence 95% · ODFTE can also be split into two parts: the Frame Type Embedder (FTE) and the One-step Video Diffusion.
ODFTE → uses → Wan 2.1
confidence 90% · Concurrently, we utilize the pre-trained Diffusion Transformer (DiT) [60] Wan 2.1 model [76] for denoising.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.18501v1
- Canonical: https://arxiv.org/abs/2603.18501v1
Trouble viewing inline? Open PDF directly →
Full Text
51,599 characters extracted from source content.
Expand or collapse full text
11institutetext: Shanghai Jiao Tong University Efficient Video Diffusion with Sparse Information Transmission for Video Compression Mingde Zhou1 Zheng Chen1 Yulun Zhang1 Corresponding author: Yulun Zhang, yulun100@gmail.com Abstract Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at https://github.com/MingdeZhou/Diff-SIT. 1 Introduction Video compression refers to encoding digital video streams into compact bitstreams, which facilitates efficient storage and transmission over bandwidth-limited channels. Generally, raw video data possesses immense volume and high redundancy, necessitating compression algorithms to exploit spatial correlations within frames and temporal dependencies across frames. Video compression has become the fundamental infrastructure for numerous multimedia and computer vision tasks, e.g., internet video streaming [69, 59, 32, 58], real-time video conferencing [24, 54, 34], etc. The primary challenge in this field lies in the rate-distortion optimization, which aims to reconstruct high-quality visual content using the minimum possible number of bits. To address this, conventional standards like H.265/HEVC [73, 57, 9, 2] and H.266/VVC [6, 79, 49, 11] utilize hand-crafted modules for block-based residual coding, prediction and transform. Recently, researchers have turned to neural video compression (NVC) frameworks. These end-to-end learning-based methods optimize the entire compression pipeline globally [47, 37, 3, 81]. Among these methods, the DCVC series [37, 40, 39, 36] established a new paradigm by replacing residual coding with conditional probability modeling, setting a new benchmark for NVC performance. Figure 1: (1) LPIPS-bitrate-coherence comparison on MCL-JCV. Temporal coherence is measured by Ewarp [31]. Our method achieves the best perceptual quality as well as much higher temporal coherence. Because all the diffusion-based video compression methods have not opened source, we choose diffusion-based image compression method (SODEC) [8] for comparison. (2) Qualitative comparison on HEVC class B BQTerrace. Our method recovers fine-grained textures with high perceptual quality. Recently, it has been recognized that optimizing for pixel-wise distortion metrics like PSNR is sub-optimal in the ultra-low bitrate regime. Strictly pursuing high PSNR often compels models to produce over-smoothed and blurry artifacts, failing to align with human perceptual realism [4, 80, 26]. Consequently, researches have shifted towards perceptual quality-oriented video compression, aiming to synthesize rich, realistic textures even under extreme bandwidth constraints [65, 63, 33]. Representative paradigms, such as GAN-based [33, 52, 4] and diffusion-based models [10, 15, 28], employ generative reconstruction to synthesize missing details. However, these methods face critical limitations. Many generative approaches process video frames independently, neglecting vital spatiotemporal correlations [83, 55]. For example, some diffusion compression methods perform image diffusion frame by frame, which leads to temporal inconsistency. Besides, most diffusion-based solutions rely on iterative multi-step sampling, introducing prohibitive latency that hinders real-world application [71, 28, 27]. To address these gaps, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT). Firstly, we posit that a true video diffusion model, which processes sequences holistically, is required to fully exploit temporal correlations and enhance temporal coherence. Thus, we propose the One-Step Video Diffusion with Frame Type Embedder (ODFTE) to efficiently perform diffusion to a whole frame sequence. Specifically, the diffusion process is guided by the Frame Type Embedder (FTE) to perform adaptive reconstruction to achieve better perception quality. Secondly, we argue that the powerful generative capability of video diffusion makes fully encoding every frame unnecessary. Based on this insight, we propose the Sparse Temporal Encoding Module (STEM) for sparse information transmission. Instead of fully compressing every frame, STEM only fully encodes a sparse set of backbone frames. Then, it represents all the other frames (we call them MV frames) using only low-bitrate optical flow fields. The ODFTE module is then employed to generatively reconstruct the final high-quality frame sequence. Here, the FTE refers to the frame type (backbone frame or MV frame) to guide the diffusion process. Our main contributions are summarized as follows: • We present a one-step video diffusion model, which to the best of our knowledge is the first attempt at one-step video diffusion for this task. • We design a novel Sparse Temporal Encoding Module (STEM) that significantly saves bitrate via sparse backbone and MV compression. • We propose the Frame Type Embedder to guide the one-step diffusion process for higher reconstruction quality. • Our framework achieves SOTA rate-perception performance on multiple datasets, while demonstrating massive advancement in temporal coherence. 2 Related Works 2.1 Learned and Generative Video Compression Traditional video compression codecs, such as HEVC (HM) and VVC (VTM) [6], define the state-of-the-art in distortion metrics (e.g., PSNR) but yield poor perceptual quality at ultra-low bitrates. As an alternative, end-to-end Neural Video Codecs (NVCs) evolved from early residual-based methods [47, 1, 43] to more powerful conditional coding frameworks [16, 44, 30]. Notably, the DCVC series [37, 66, 40] has achieved SOTA rate-distortion performance by leveraging deep contextual information. Concurrent to this, generative video compression methods [29, 51, 35] have focused on maximizing perceptual quality, often at the expense of traditional metrics. These models, such as DVC-P [88, 85], utilize strong generative priors to synthesize realistic details, proving highly effective in the challenging low-bitrate regime where traditional codecs fail. 2.2 Diffusion Models in Video Compression Diffusion models [18, 72, 70, 8] have shown state-of-the-art performance in image generation, which has inspired their application in perceptual image compression [52, 20, 86, 82]. Concurrently, dedicated video diffusion models have been developed to explicitly model the temporal dimension, enabling the generation of coherent video sequences [19, 17, 68]. This potential has inspired applications in video compression. These methods, however, face two critical limitations. First, approaches like DiffVC [50] apply image diffusion models frame-by-frame. This independent generative process ignores temporal correlations and may introduce temporal incoherence. Second, existing diffusion-based methods like GiViC and DiffVC [14, 50] rely on multi-step diffusion process, which incurs a substantial computational cost and limit their applications [64, 71]. 2.3 Optical Flow in Video Compression Optical flow, which estimates per-pixel motion to exploit temporal redundancy, is fundamental to video compression [22]. Modern flow estimation is dominated by deep networks like PWC-Net [74] and RAFT [75], which replaced classic optimization methods [21, 48]. In Neural Video Compression (NVC), flow is central to motion compensation. Early works like DVC [47] used it to warp pixels and compress the residual [47, 13]. More advanced conditional codecs, such as the DCVC series [37], warp features in the latent space to provide temporal context for the encoder and entropy model [67, 38, 40]. In these frameworks, the motion vector (MV) field itself must be compressed and transmitted as side information [61]. However, recursive prediction in NVC often suffers from error accumulation and quality collapse over long sequences [46, 41]. Studies investigating this trade-off [46] highlight that while short-term flow prediction achieves high accuracy, long-term propagation necessitates complex correction mechanisms [41, 25]. 3 Methodology In this section, we provide an overview of our model, Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT). Fig. 2 illustrates the overall architecture. The model is divided into two modules: the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The ODFTE can also be split into two parts: the Frame Type Embedder (FTE) and the One-step Video Diffusion. Given an input frame sequence =x1,…,xTx=\x_1,…,x_T\, we first select frames with indexes 3t+23t+2 as backbone frames and group them into a new backbone sequence x2,x5,x8,…\x_2,x_5,x_8,…\. The remaining frames are designated as Motion Vector (MV) frames. Backbone compression: The Sparse Temporal Encoding Module (STEM) first compresses and reconstructs the backbone sequence, yielding intermediate reconstructed backbone frames x~2,x~5,x~8,…\ x_2, x_5, x_8,…\. MV compression: STEM then proceeds to compress the MV frames. For each intermediate reconstructed backbone frame x~3t+2 x_3t+2, the model estimates optical flows to its two neighboring original frames x3t+1x_3t+1 and x3t+3x_3t+3 and transmits the compressed flow fields. At decoder, decoded flows are used to warp the backbone frame, and the warped results are directly taken as the intermediate reconstructed MV frames, [x~3t+1,x~3t+3]=(x~3t+2,[^x~3t+2→x3t+1,^x~3t+2→x3t+3]). [ x_3t+1,\, x_3t+3 ]=W ( x_3t+2,\, [ F_ x_3t+2→ x_3t+1,\, F_ x_3t+2→ x_3t+3 ] ). (1) where x~3t+1 x_3t+1 and x~3t+3 x_3t+3 denote the reconstructed MV frames and ^x~3t+2→x3t+1 F_ x_3t+2→ x_3t+1, ^x~3t+2→x3t+3 F_ x_3t+2→ x_3t+3 are the optical flows from the reconstructed backbone frame x~3t+2 x_3t+2 to the original neighboring frames x3t+1x_3t+1 and x3t+3x_3t+3, respectively. Here, W denotes the warping operator, which warps frame x~3t+2 x_3t+2 via optical flows F. Finally, the intermediate reconstructed backbone frames and MV frames are rearranged into the original order to form the full intermediate reconstructed sequence ~ x. Figure 2: The overall architecture of our proposed Diff-SIT model. It consists of two main part: the STEM and ODFTE. Given an input frame sequence x, first, it will be divided into backbone frames and MV frames. In STEM, there are two steps: (1) Backbone compression: the backbone frames are first compressed and reconstructed. (2) MV compression: each reconstructed backbone frame is used as a reference and MV compression is conducted to compress all the MV frames. After the STEM, the intermediate reconstructed sequence ~ x will be fed into the ODFTE to be restored into final frame sequence x. Image flow refers to the coding order of video frames. The full intermediate sequence ~ x is then fed into the ODFTE to be restored into a high-quality frame sequence x. Frame Type Embedder (FTE): First, FTE generates a type embedding c based on the compression type of each frame (backbone or MV). One-step Video Diffusion: Then, conditioned on this type embedding c, the diffusion model performs adaptive diffusion reconstruction. Thus, we gain the final reconstructed sequence x. 3.1 Sparse Temporal Encoding Module (STEM) In our STEM, we first perform the backbone compression, where the backbone frames are compressed as an independent video sequence. The I-frame compression: For the first backbone frame x2x_2 (x3t+2x_3t+2 where t=0t=0), we perform an independent intra-coding (I-frame encoding). Specifically, we use a pre-trained image compression model (HiFiC) [52] for I-frame encoding. All subsequent backbone frames are predictively coded as P-frames, referencing the previously reconstructed frame. The I-frame and P-frames are both the backbone frames. Details of P-frame compression: As shown in Fig. 3, the pipeline of P-frame compression is: (1) Motion compensation: We perform a motion compensation, which consumes a few bits. We choose SpyNet [62] as our optical flow estimation network. SpyNet is used to produce the motion-compensated estimation (x3t+5′x _3t+5) of the original frame x3t+5x_3t+5, x3t+5′=fmc(x~3t+2,x3t+5).x _3t+5=f_mc( x_3t+2,x_3t+5). (2) where fmcf_mc is the motion compensation network. (2) Conditional encoding: Following the method proposed in DCVC [37, 40], we perform a conditional encoding framework. First, the context generation module (fconf_con) generates a context feature conc_con based on the motion-compensated estimation x3t+5′x _3t+5 we get, con=fcon(x3t+5′).c_con=f_con(x _3t+5). (3) Subsequently, the encoder ℰE compresses the current frame x3t+5x_3t+5 into a latent representation y3t+5y_3t+5 conditioned on this context conc_con. Then y3t+5y_3t+5 is quantized, y^3t+5=(y3t+5) y_3t+5=Q(y_3t+5), where Q refers to the quantization operation. An entropy model estimates the probability distribution of quantized latent y^3t+5 y_3t+5 conditioned on the context conc_con, and performs an entropy encoding to generate a bitstream. After the bitstream is transmitted, a decoder D, again guided by conc_con, reconstructs from the decoded latent representation y^3t+5 y_3t+5 to get the intermediate reconstructed frame x~3t+5 x_3t+5. The overall pipeline can be shown as x~3t+5=((ℰ(x3t+5,con)),con). x_3t+5=D(Q(E(x_3t+5,c_con)),c_con). (4) Performing this process on all backbone frames, and we can get the intermediate reconstructed backbone frames x~2,x~5,x~8,…\ x_2, x_5, x_8,…\. Figure 3: Pipeline of P-frame compression and MV compression. In P-frame compression, the reference frame x~3t+2 x_3t+2 is used to compress the target frame x3t+5x_3t+5 via conditional encoding. In MV compression, the reference frame x~3t+2 x_3t+2 is used to compress frames x3t+1x_3t+1 and x3t+3x_3t+3 via MV compression. The reconstructed flow field ^x~3t+2→x3t+1 F_ x_3t+2→ x_3t+1 and ^x~3t+2→x3t+3 F_ x_3t+2→ x_3t+3 are used to warp the reference frame x~3t+2 x_3t+2 to reconstruct the MV frames (x~3t+1 x_3t+1) and (x~3t+3 x_3t+3) respectively. Here, we use x~3t+3 x_3t+3 as an example. Then, we proceed to compress the remaining MV frames, performing the MV compression. This process is performed in parallel for all MV frames. Each reconstructed backbone frame x~3t+2 x_3t+2 serves as a reference for its two neighboring frames x3t+1x_3t+1 and x3t+3x_3t+3. The “MV Compression” pipeline is shown in Fig. 3. Using x~3t+2 x_3t+2 as the reference, an optical flow estimation [62] module predicts the motion from x~3t+2 x_3t+2 to x3t+1x_3t+1 and x3t+3x_3t+3 respectively, denoted as motion vector field x~3t+2→x3t+1F_ x_3t+2→ x_3t+1, x~3t+2→x3t+3F_ x_3t+2→ x_3t+3. Each field x~3t+2→x3t+5F_ x_3t+2→ x_3t+5. x~3t+2→x3t+5F_ x_3t+2→ x_3t+5 is then passed through an MV Encoder (ℰmvE_mv) to produce a compact latent representation, and then quantized and entropy-coded into a bitstream for transmission. This step consumes few bits, because we only need to transmit the motion vector field F. At the decoder, the bitstream is entropy-decoded and processed by an MV Decoder (mvD_mv) to obtain the reconstructed flow field F. Using x~3t+2→x3t+3F_ x_3t+2→ x_3t+3 as an example, the pipeline for transmitting vector flow field can be shown as: ^x~3t+2→x3t+3=mv((ℰmv(x~3t+2→x3t+3)), F_ x_3t+2→ x_3t+3=D_mv(Q(E_mv(F_ x_3t+2→ x_3t+3)), (5) where Q refers to the quantization operation. Specifically, for a given reconstructed backbone frame x~3t+2 x_3t+2, it serves as the reference frame to compress both the preceding frame x3t+1x_3t+1 and the succeeding frame x3t+3x_3t+3 via MV compression. The reconstructed flow F is used to warp the reference frame x~3t+2 x_3t+2 to produce intermediate reconstructed MV frames x~3t+1 x_3t+1 and x~3t+3 x_3t+3. [x~3t+1,x~3t+3]=(x~3t+2,[^x~3t+2→x3t+1,^x~3t+2→x3t+3]). [ x_3t+1,\, x_3t+3 ]=W ( x_3t+2,\, [ F_ x_3t+2→ x_3t+1,\, F_ x_3t+2→ x_3t+3 ] ). (6) To be more clear, the intermediate reconstructed MV frames x~3t+1 x_3t+1 and x~3t+3 x_3t+3 are exactly the warped results of the reference backbone frame x~3t+2 x_3t+2. In this manner, we reconstruct all backbone and MV frames, yielding the full intermediate frame sequence ~ x==x~1,…,x~T\ x_1,…, x_T\ for later diffusion reconstruction. Figure 4: Analysis of reconstruction quality and bitrate cost versus the length of the continuous optical flow prediction chain. Reconstruction quality rapidly degrades as the prediction chain grows, while the required bitrate also increases significantly. Image at right is the corresponding 1-9 frames, from top-left to bottom-right. The essence of MV compression is an optical flow compression. While many studies have discussed the problem of quality collapse in long optical flow prediction chains, we find that for very short chains, flow-only compression can achieve superior reconstruction results at a lower bitrate. As shown in Fig. 4, the reconstruction quality rapidly degrades as optical prediction chains become longer. Thus, we choose to limit the optical prediction length to only one immediate neighbor (a reference frame x~3t+2 x_3t+2 only serves for MV-compressing x3t+1x_3t+1 and x3t+3x_3t+3), which achieves excellent reconstruction quality at a minimal bit cost. 3.2 One-Step Video Diffusion with Frame Type Embedder (ODFTE) We introduce the ODFTE module to perform generative reconstruction by leveraging the powerful capabilities of diffusion models to reinforce the intermediate reconstructed sequence ~ x. This module consumes zero bits. Frame Type Embedder (FTE): First, we use the FTE to extract a type-bias embedding c from the encoding type of the frame sequence. For instance, with T=9T=9, the encoding type sequence is [MV, I, MV, MV, P, MV, MV, P, MV]. We first assign a unique one-hot vector to each type (I, P, or MV) and broadcast this vector to the full spatial dimensions of the frame. These broadcasted maps are then concatenated along the temporal dimension to create the frame type map M, as shown in Fig. 2. Subsequently, we apply a lightweight 3D convolution network ftypef_type to this map to extract the type information, which is then flattened to produce the final type-bias embedding c, i.e., =ftype()c=f_type(M). One-step Video Diffusion: Concurrently, we utilize the pre-trained Diffusion Transformer (DiT) [60] Wan 2.1 model [76] for denoising. First, the Wan Encoder ℰwanE_wan encodes the intermediate frame sequence x~ x into a latent representation. Subsequently, the tokenizer converts this representation into a token sequence nz^n, where n denotes the diffusion timestep. To maximize efficiency, we fix the timestep n and employ a one-step diffusion process. This is motivated by our observation that the input ~ x already contains rich information from the STEM, meaning that a single refinement step is sufficient to restore fine-grained details, obviating the need for an iterative denoising process from pure noise. We also apply adaptive diffusion strength. Different frames in the intermediate frame sequence ~ x have different levels of quality (backbone frames v.s MV frames). We propose that one-step diffusion process should apply adaptive denoising, stronger to low-quality MV frames, while weaker to more reliable backbone frames, which can better preserve the original information. So, the type-bias embedding c is added to the tokens znz^n to obtain the type-biased tokens ~n z^n, where ~n z^n==n⊕z^n . Thus, the diffusion model is informed and can condition on the type embedding c for adaptive reconstruction. Then, the one-step diffusion transformer denoises ~n z^n, yielding the clean tokens 0z^0, 0=1α¯n(~n−1−α¯n⋅ϵθ(~n,n)),z^0= 1 α_n ( z^n- 1- α_n· _θ( z^n,n) ), (7) where α¯n α_n is the cumulative noise schedule parameter at the fixed timestep n, and ϵθ _θ is the noise-prediction network (DiT) parameterized by θ. Finally, ~0 z^0 is transformed back into a latent representation and decoded by the 3D Wan Decoder wanD_wan [76] to produce the final reconstructed frame sequence x. Our ODFTE is designed on common DiT, other DiT backbones can also be applied. 3.3 Training and Inferencing Our model is trained in two stages. In the first stage, we train only the backbone reconstruction part of the STEM. Sequence length T is set to 9. Only backbone frames, i.e, sequence 3t+2x_3t+2==x2,x5,x8\x_2,x_5,x_8\ are used for training. STEM is optimized for the intermediate sequence ~3t+2 x_3t+2==x~2,x~5,x~8\ x_2, x_5, x_8\. The loss function is: ℒs1=λ⋅r(^3t+2,fmc)+MSE(3t+2,~3t+2),L_s1=λ· r( y_3t+2,f_mc)+MSE(x_3t+2, x_3t+2), (8) where r(⋅)r(·) denotes the bitrate cost. ^3t+2 y_3t+2 is bitrate for the quantized backbone latents ^3t+2 y_3t+2, and fmcf_mc means bitrate comsumed during the motion compensation. MSE represents the Mean Squared Error. Hyperparameter λ controls the rate-distortion trade-off. We load the pre-trained HiFiC model [52] for I-frame compression, and the P-frame compression module is trained end-to-end. In the second stage, we load the pre-trained Wan 2.1 1.3B model [76] and train the entire model (STEM and ODFTE) end-to-end. We still set T==99, but now use the full sequence x==x1,…,x9\x_1,…,x_9\ and optimize for the final high-quality reconstruction x==x^1,…,x^9\ x_1,…, x_9\. The loss function is: ℒs2= _s2= λ⋅r(^3t+2,fmc,^3t+2→3t+1,3t+3)+LPIPS(,^) \ λ· r ( y_3t+2,f_mc, F_3t+2→ 3t+1,3t+3 )+LPIPS(x, x) (9) +k1⋅MSE(,^)+k2⋅ℒtemp(,^). \ +k_1·MSE(x, x)+k_2·L_temp(x, x). where the rate term r(⋅)r(·) now includes all the bit cost, including (1) backbone compression cost: (^3t+2 y_3t+2, fmcf_mc), (2) MV compression cost: (^3t+2→3t+2±1 F_3t+2→ 3t+2± 1), and ℒtempL_temp represents the Frame Difference Loss. Here, we use MSE Loss: ℒtemp=[∑t=2T‖(x^t−x^t−1)−(xt−xt−1)‖22].L_temp=E [ _t=2^T \|( x_t- x_t-1)-(x_t-x_t-1) \|_2^2 ]. (10) For inferencing, our model can operate on sequences of arbitrary length T. To align with real-world GOP (Group of Pictures) sizes which are often 32 or larger, we set T==3333. This sequence length consists of 11 backbone frames (1 I-frame, 10 P-frames) and 22 MV frames. We choose T==3333 as Wan model requires the sequence length to be 8n8n++11. We experimented on different density of backbone frames, and Tab. 4 shows that current setting (interval 2, i.e [MV, P, MV, MV, P, MV…]) is optimal. More detailed analysis can be found in ablation study. 4 Experiments 4.1 Experimental Settings Datasets. We employ a two-stage training strategy. First, the STEM backbone is optimized via Eq. (8) on frames 1, 4, and 7 of the Vimeo-9k dataset [84] for 700k steps. Subsequently, the entire framework is fine-tuned end-to-end minimizing Eq. (9) on the REDS dataset [56] using random 256×256256× 256 9-frame patches for 15,000 steps. We benchmark our model Diff-SIT on three standard high-resolution test datasets: HEVC B [23], MCL-JCV [77], and UVG [53]. During inference, we evaluate all sequences at their full resolution. Metrics. We employ LPIPS [87] and DISTS [12] to evaluate reference perceptual quality. CLIPIQA [78] is utilized to assess no-reference visual realism. Temporal consistency is measured by Ewarp [31] via optical flow warping error. Finally, compression rate is reported in bits per pixel (bpp). Beyond quantitative analysis, we also provide visual results for straight-forward comparison in Fig 6. Implementation Details. We adopt the DCVC-DC architecture [40] for backbone P-frame encoding and utilize the Wan 2.1 model [76] with a fixed timestep n==799799 for ODFTE. Training employs the default AdamW optimizer [45] with batch sizes of 8 in Stage 1 and 2 in Stage 2. Loss hyperparameters are set to λ==55⋅·10−310^-3 for Eq. (8), and λ∈0.5,1.0,2.0,4.0\0.5,1.0,2.0,4.0\, k1k_1==1010, k2k_2==0.10.1 for Eq. (9). Experiments are conducted on one NVIDIA RTX A6000 GPU, and the training takes about two days. More details are provided in supplementary materials. 4.2 Main Results HEVC Class B MCL-JCV UVG Figure 5: Quantitative comparison with state-of-the-art methods on the HEVC Class B, MCL-JCV and UVG datasets. ↓ means lower is better. HEVC Class B GT (BPP↓ ) HM (0.0451) VTM (0.0447) DVC-P (0.0539) DCVC-DC (0.0436) DCVC-FM (0.0455) HNeRV (0.0398) Diff-SIT (0.0330) MCL-JCV GT (BPP↓ ) HM (0.0392) VTM (0.0385) DVC-P (0.0403) DCVC-DC (0.0353) DCVC-FM (0.0384) HNeRV (0.0337) Diff-SIT (0.0268) UVG GT (BPP↓ ) HM (0.0352) VTM (0.0345) DVC-P (0.0368) DCVC-DC (0.0418) DCVC-FM (0.0429) HNeRV (0.0391) Diff-SIT (0.0291) Figure 6: Visual comparison on the HEVC Class B, MCL-JCV and UVG datasets. To rigorously validate the effectiveness of our model Diff-SIT, we benchmark against state-of-the-art methods spanning different paradigms. Specifically, we compare our approach with leading traditional standards, HM (H.265/HEVC) [73] and VTM (H.266/VTM) [6], as well as representative end-to-end neural video compression (NVC) models such as DCVC-DC [40] and DCVC-FM [42]. To ensure a comprehensive evaluation against diverse strategies, we also include the GAN-based DVC-P [85] and the implicit neural representation (INR) method HNeRV [7]. As far as we know, until now, all existing diffusion-based methods such as GiViC [14] and DiffVC [50] haven’t opened source. Table 1: Comparison on distortion metrics on HEVC class B. We report the PSNR and MS-SSIM at similar bitrates. Model bpp ↓ PSNR ↑ MS-SSIM ↑ VTM 0.0335 32.75 0.9742 DCVC-FM 0.0328 33.29 0.9782 Diff-SIT (ours) 0.0327 28.50 0.9387 Table 2: Impact of STEM on coding performance. BD-Rates are calculated relative to the configuration without STEM. Negative values indicate bitrate savings. Configuration BD-Rate (LPIPS) ↓ BD-Rate (DISTS) ↓ w/o STEM (Anchor) 0.0% 0.0% w/ STEM -23.6% -17.2% Table 3: Ablation study of model components. We compare four configurations We compare four configurations to isolate the contributions of: (1) the One-Step Diffusion (OSD) (2) the sparse encoding strategy (STEM), (3) the Frame Type Embedder (FTE). Configuration bpp ↓ LPIPS ↓ DISTS ↓ (i) Direct Enc. 0.0335 0.2319 0.1267 (i) Direct+OSD 0.0321 0.1863 0.1097 (i) STEM+OSD 0.0317 0.1720 0.0971 (iv) Ours 0.0308 0.1689 0.0955 Quantitative Evaluation. As shown in Fig. 5, our proposed Diff-SIT establishes a new state-of-the-art across all evaluated metrics. Our model achieves superior reference perceptual quality, significantly outperforming all baselines on LPIPS and DISTS. Furthermore, Diff-SIT excels in no-reference perceptual quality (CLIPIQA) and temporal consistency (Ewarp), demonstrating a substantial margin over prior methods. Notably, we observe a on-intuitive trend: at ultra-low bitrates, our CLIPIQA and Ewarp scores sometimes become better when the bitrate decreases (e.g., on HEVC Class B). This is attributed to the model synthesizing plausible, realistic textures and ideal motion driven by strong generative priors when explicit conditioning information is insufficient. It is worth noting that, as shown in Tab. 1, our model does not show an advantage in pixel-wise distortion metrics (PSNR, MS-SSIM) compared with traditional methods. This is an expected trade-off. This phenomenon has been discussed in work like [4, 5], generative compression models are optimized to pursue high perceptual quality at ultra-low bitrates, rather than minimizing pixel-level loss. At these low bitrates, models optimized for distortion (e.g., PSNR) tend to produce overly smooth and blurry images to minimize the mean squared error, which is often inconsistent with human perceptual preferences. Qualitative Evaluation. We present qualitative visual comparisons on three datasets in Fig. 6. Our Diff-SIT reconstructions consistently exhibit significantly more fine-grained detail and superior perceptual quality with lower bitrates. In contrast, other methods often suffer from distinct degradation patterns: artifacts, blocks, over-smoothed results with details missing. 4.3 Ablation Study To validate the efficacy of our proposed components, we conduct a comprehensive ablation study. All variants follow an identical training protocol: 700k training steps for Stage 1, followed by 10,000 steps of end-to-end fine-tuning in Stage 2. We compare four configurations: (i) Direct Encoding (Baseline): a standard baseline where every frame is fully encoded via our backbone compression model. (i) Direct Encoding + One-Step Diffusion: the baseline output serves as input to the One-Step Diffusion module, which isolate the benefits of generative refinement. (i) STEM + One-Step Diffusion: use our STEM for compression and then perform a One-Step Diffusion, which can justify the rate-saving capablity of STEM. (iv) STEM + ODFTE: our full model which includes the Sparse Temporal Encoding Module, One-Step Diffusion and Frame Type Embedder (FTE), which shows the effectiveness of FTE. As shown in Tab. 3, the perceptual quality comparison reveals a clear hierarchy: our full model (iv) outperforms “(i) STEM + OSD”, validating Frame Type Embedder’s effectiveness in improving quality. (i)’s superiority over “(i) Direct Encoding + OSD” justify STEM’s rate-saving capability. Finally, the substantial performance leap from “(i) Direct Encoding” to (i) serves as compelling evidence for the effectiveness of our one-step video diffusion. More detailed and in-depth ablation analysis for will be provided below. Effectiveness of Sparse Temporal Encoding (STEM). To validate STEM, we benchmark Diff-SIT against the configuration without sparse encoding (Direct Encoding + ODFT) on LPIPS and DISTS. As shown in Tab. 2, our strategy achieves approximately 20% bitrate savings (BD-Rate) while maintaining comparable perceptual quality, confirming the high coding efficiency of our sparse representation. This improvement stems from the fact that the low-bitrate MV frames consume negligible bits compared to fully coded backbone frames. Thus, the model can allocate more budget to key structural information when encoding backbone frames, both saving bitrates improving quality. Synergy of STEM and ODFTE. We analyze the synergy between our sparse encoding (STEM) and generative enhancement (ODFTE). As shown in Fig. 7, (1) the intermediate MV frames (Before Diffusion) initially exhibit significantly lower quality compared to the backbone frames. However, after diffusion restoration, the perceptual quality of the final MV frames becomes much closer to that of the backbone frames. This demonstrates that, despite MV frames’ low bitrate, they convey sufficient key information for the diffusion model, which can leverage temporal context for effective restoration. Specifically, the diffusion model generates details and corrects distortions in regions where optical flow estimation fails. This confirms the strong synergy between our sparse encoding (STEM) and generative enhancement (ODFTE). (2) Diffusion model with the Frame Type Embedder (FTE) achieves superior overall quality compared to the variant without it. This validates the benefit of adaptive reconstruction. Conditioned on which type one frame is compressed (I, P or MV), the diffusion model can apply different strength of generation, giving more trust to backbone frames and exploit their information for guiding MV frames’ restoration. Figure 7: Before diffusion, MV frames are initially of much lower quality than backbone frames. (1) However, after diffusion restoration, their perceptual quality becomes much closer to restored P-frames, which indicates ODFTE can exploit information from backbone frames to further improve MV frames’ quality. (2) Diffusion with FTE shows better perceptual quality, proving effectiveness of FTE to guide the diffusion process. Number of MV frames between P frames. We vary the number of MV frames between every two P frames. As Tab. 4 shows, two MV frames (i.e. each backbone frame x~3t+2 x_3t+2 serves as a reference for its two neighboring frames x3t+1x_3t+1, x3t+3x_3t+3) achieve the best result. It ensures each MV frame is predicted directly from a high-quality backbone frame and maintains low bitrate. Too many MV frames will lead to collapse of optical flow prediction chain, resulting in poor perceptual quality. Meanwhile, too few MV frames will lift the proportion of backbone frames, resulting in high bitrate, and poorer quality if bitrate is fixed. Choosing two MV frames as interval achieves a good trade-off. Table 4: Ablation on number of MV frames between P frames. MV frame number bpp ↓ Proportion of backbone frames LPIPS ↓ DISTS ↓ Denser: 1 0.0355 48.5% (16 backbone / 33 total) 0.1705 0.1012 Sparser: 4 0.0319 21.2% (7 backbone / 33 total) 0.1823 0.1218 Ours: 2 0.0308 33.3% (11 backbone / 33 total) 0.1689 0.0955 Multi-Step Diffusion and Components Latency. To quantify the trade-off of our one-step design, we compare our model against multi-step variants and quantify the latency of each component. We implement 2-step and 5-step versions of our ODFTE, keeping all other components (STEM, FTE) identical. As shown in Tab. 5, (1) Diffusion step: increasing diffusion steps yields a marginal improvement in perceptual quality. However, this comes at a substantial computational cost. The 5-step model, for instance, is approximately 4x slower than our one-step approach. This confirms that our one-step model achieves an excellent trade-off between performance and efficiency, sacrificing minimal perceptual quality for a massive speed-up, making it more practical. (2) Components latency: the bottleneck is the Wan2.1, resulting in higher decoder latency. The Encoder (STEM) is relatively fast, accounting for 34.8% of the total latency. Table 5: Ablation: One-step vs. multi-step diffusion process. We report (1) both the absolute average per-frame inference time (ms) and the relative latency (×) compared to the 1-step baseline. (2) inference latency of each component. (1) Diffuison step Time (ms / ×) ↓ LPIPS ↓ DISTS ↓ ours (1-Step) 1275 / 1.00 0.1689 0.0955 ours (2-Steps) 2199 / 1.72 0.1615 0.0901 ours (5-Steps) 4263 / 3.35 0.1544 0.0832 Components latency Wan2.1 Encoder (STEM) Decoder (including ODFTE) Time (ms / percentage) 656.6 / 51.5% 443.7 / 34.8% 831.3 / 65.2% 5 Conclusion Traditional video codecs suffer from severe perceptual degradation at ultra-low bitrates, motivating the need for generative reconstruction. In this paper, we proposed Diff-SIT, a novel video compression framework that combines a Sparse Temporal Encoding Module (STEM) with an efficient, One-Step Video Diffusion model with Frame Type Embedder (ODFTE). This integrated approach demonstrates excellent perceptual quality and temporal consistency, especially in the challenging ultra-low-bitrate regime. Experiments confirm that Diff-SIT achieves state-of-the-art performance on multiple benchmarks for video compression, demonstrating its superior efficiency and effectiveness. References [1] Agustsson, E., Minnen, D., Johnston, N., Ballé, J., Hwang, S.J., Toderici, G.: Scale-space flow for end-to-end optimized video compression. In: CVPR (2020) [2] Alariao, N.Z., Caguia, K.P., Olaguer, P.A.J., Sison, G.A.V., Valenton, J.M.C., Serrano, K.K.D., Cruz, A.R.D.: Sum of absolute difference-based rate-distortion optimization cost function for h.265/hevc intra-mode prediction. In: 2019 IEEE 11th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM) (2019) [3] Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. arXiv preprint arXiv:1802.01436 (2018) [4] Blau, Y., Michaeli, T.: The perception-distortion tradeoff. In: CVPR (2018) [5] Blau, Y., Michaeli, T.: Rethinking lossy compression: The rate-distortion-perception tradeoff. In: ICML (2019) [6] Bross, B., Wang, Y.K., Ye, Y., Liu, S., Chen, J., Sullivan, G.J., Ohm, J.R.: Overview of the versatile video coding (vvc) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology (2021) [7] Chen, H., Gwilliam, M., Lim, S.N., Shrivastava, A.: Hnerv: A hybrid neural representation for videos. In: CVPR (2023) [8] Chen, Z., Zhou, M., Guo, J., Yuan, J., Ji, Y., Zhang, Y.: Steering one-step diffusion model with fidelity-rich decoder for fast image compression. In: AAAI (2025) [9] Correa, G., Zatt, B., Bampi, S., Grellert, M., Agostini, L.: Analysis of the perceptual quality performance of different hevc coding tools. IEEE Access (2021) [10] Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: NeurIPS (2021) [11] Diaz-Ramirez, I., Zatt, B., Correa, G., Agostini, L., Grellert, M.: Vvc, jem and hevc: a new performance and complexity-based comparison. Journal of Signal Processing Systems (2021) [12] Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: Unifying structure and texture similarity. TPAMI (2020) [13] Djelouah, A., Campos, J., Schaub-Meyer, S., Schroers, C.: Neural inter-frame compression for video coding. In: ICCV (2019) [14] Gao, G., Teng, S., Peng, T., Zhang, F., Bull, D.: Givic: Generative implicit video compression. In: ICCV (2025) [15] Guo, M., Lin, W., Zhang, Y., Liu, K.: Low-bitrate image compression with diffusion models. TMM (2024) [16] Habibian, A., van Rozendaal, T., Tomczak, J.M., Cohen, T.S.: Video compression with rate-distortion autoencoders. In: ICCV (2019) [17] Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D.P., Poole, B., Norouzi, M., Fleet, D.J., Salimans, T.: Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022) [18] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020) [19] Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. In: NeurIPS (2022) [20] Hoogeboom, E., Agustsson, E., Mentzer, F., Versari, L., Toderici, G., Theis, L.: High-fidelity image compression with score-based generative models. arXiv preprint arXiv:2305.18231 (2023) [21] Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence (1981) [22] Jain, J.R., Jain, A.K.: Interframe adaptive data compression techniques for images. Defense Technical Information Center (1979) [23] JCT-VC: Common Test Conditions and Software Reference Configurations. Tech. Rep. JCTVC-L1100, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 (January 2013) [24] Jennings, C., Castelli, F., Boström, H., Bruaroey, J.: Webrtc: Real-time communication in browsers (2023) [25] Jiang, W., Li, J., Li, B., Lu, Y.: Ecvc: Exploiting non-local correlations in multiple frames for contextual video compression. In: CVPR (2025) [26] Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: ECCV (2016) [27] Kim, M.s., Jo, J., Kim, S., Kim, E., Choi, M.J.: Fast diffusion-based image compression with block-wise parallel sampling. In: DCC (2024) [28] Kim, S., Jo, J., Kim, M.s., Kim, E., Choi, M.J.: Diffusion model-based image compression. arXiv preprint arXiv:2401.03456 (2024) [29] Kim, S., Park, J.S., Bampis, C.G., Lee, J., Markey, M.K., Dimakis, A.G., Bovik, A.C.: Adversarial video compression guided by soft edge detection. In: ICASSP (2020) [30] Ladune, T., Philippe, P., Hamidouche, W., Zhang, L., Déforges, O.: Conditional coding for flexible learned video compression. In: ICLR Workshop (2021) [31] Lai, W.S., Huang, J.B., Wang, O., Shechtman, E., Yumer, E., Yang, M.H.: Learning blind video temporal consistency. In: ECCV (2018) [32] Lederer, S., Müller, C., Timmerer, C.: Dynamic adaptive streaming over http: From theory to practice. In: Proceedings of the 4th ACM Multimedia Systems Conference (2013) [33] Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR (2017) [34] Lee, J., Park, S., Kim, J.: R-fec: A reinforcement learning-based framework for dynamic fec in video conferencing. TMM (2023) [35] Li, B., Liu, Y., Niu, X., Bai, B., Deng, L., Gündüz, D.: Extreme video compression with pre-trained diffusion models. arXiv preprint arXiv:2402.08934 (2024) [36] Li, J., Li, B., Liu, D., Lu, Y.: Neural video compression with feature modulation. In: CVPR (2024) [37] Li, J., Li, B., Lu, Y.: Deep contextual video compression. Advances in Neural Information Processing Systems (2021) [38] Li, J., Li, B., Lu, Y.: Dcvc-hem: Hybrid entropy modeling for learned video compression. In: ACM M (2022) [39] Li, J., Li, B., Lu, Y.: Hybrid spatial-temporal entropy modelling for neural video compression. In: ACM M (2022) [40] Li, J., Li, B., Lu, Y.: Neural video compression with diverse contexts. In: CVPR (2023) [41] Li, J., Li, B., Lu, Y.: Neural video compression with feature modulation. In: CVPR (2024) [42] Li, L., Li, J.W., Li, D., Liu, L.M., Yang, Y., Liu, S.: DCVC-FM: Deep neural network based video compression with feature modulation. In: CVPR (2024) [43] Lin, J., Liu, D., Li, H., Wu, F.: M-lvc: Multiple frames prediction for learned video compression. In: CVPR (2020) [44] Liu, J., Wang, S., Ma, W.C., Shah, M., Hu, R., Dhawan, P., Urtasun, R.: Conditional entropy coding for efficient video compression. In: ECCV (2020) [45] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019), https://arxiv.org/abs/1711.05101 [46] Lu, G., Cai, C., Zhang, X., Chen, L., Ouyang, W., Xu, D., Gao, Z.: Content adaptive and error propagation aware deep video compression. In: ECCV (2020) [47] Lu, G., Ouyang, W., Xu, D., Zhang, X., Cai, C., Gao, Z.: Dvc: An end-to-end deep video compression framework. In: CVPR (2019) [48] Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI (1981) [49] Luo, Y., Li, Z., Zhang, K., Liu, Z., Wang, S.: A comparative study of the av1, hevc, and vvc video coding standards. In: ICME (2023) [50] Ma, W., Chen, Z.: Diffusion-based perceptual neural video compression with temporal diffusion information reuse. ACM Transactions on Multimedia Computing, Communications and Applications (2025) [51] Mentzer, F., Agustsson, E., Ballé, J., Minnen, D., Johnston, N., Toderici, G.: Neural video compression using gans for detail synthesis and propagation. In: ECCV (2022) [52] Mentzer, F., Toderici, G.D., Tschannen, M., Agustsson, E.: High-fidelity generative image compression. In: NeurIPS (2020) [53] Mercat, A., Viinikainen, A., Röyttä, P., Hämäläinen, M.M., Kulmala, J.: UVG dataset: 50/120fps 4K sequences for video codec analysis and development. In: Proceedings of the 11th ACM Multimedia Systems Conference (2020) [54] Min, D., Park, G., Lee, J., Park, J.H., Kim, J.: A multi-hop video streaming architecture based on webrtc and mediasoup. IEEE Transactions on Consumer Electronics (2025) [55] Mirakhorli, M., Yang, J.: Temporal coherence in generative models: A survey. TPAMI (2023) [56] Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., Mu Lee, K.: Ntire 2019 challenge on video deblurring and super-resolution: Dataset and study. In: CVPRW (2019) [57] Ohm, J.R., Sullivan, G.J., Schwarz, H., Tan, T.K., Wiegand, T.: Comparison of the coding efficiency of video coding standards–including high efficiency video coding (hevc). TCSVT (2012) [58] Ozbek, N., Aricioglu, A.: Implementation and quality assessment of a user-centric adaptation system for dash. Hittite Journal of Science and Engineering (2019) [59] Pantos, R., May, W.: Http live streaming (2017) [60] Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: ICCV (2023) [61] Qi, L., Li, J., Li, B., Li, H., Lu, Y.: Motion information propagation for neural video compression. In: CVPR (2023) [62] Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) [63] Rippel, O., Schult, J.A., Moller, C., Schutt, K.T.: Improved perceptual compression with explicit rate-perception tradeoff control. In: ICML (2021) [64] Salimans, T., Ho, J.: Progressive distillation for fast sampling of diffusion models. In: ICLR (2022) [65] Santurkar, S., Tancik, M., Sra, S., Toderici, G.: Generative compression. In: ICML (2018) [66] Sheng, X., Li, J., Li, B., Li, L., Liu, D., Lu, Y.: Temporal context mining for learned video compression. TMM (2023) [67] Sheng, Z., Li, J., Li, B., Lu, Y.: Dcvc-tcm: Temporal context mining for learned video compression. TMM (2022) [68] Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y.: Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 (2022) [69] Sodagar, I.: The mpeg-dash standard for multimedia streaming over the internet. IEEE MultiMedia (2011) [70] Sohl-Dickstein, J., Weiss, E.A., Wilson, N.M., Sontag, C.: Deep unsupervised learning using nonequilibrium thermodynamics. ICML (2015) [71] Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020) [72] Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score-based generative modeling through stochastic differential equations. ICLR (2021) [73] Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (hevc) standard. TCSVT (2012) [74] Sun, D., Yang, X., Liu, M.Y., Kautz, J.: Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In: CVPR (2018) [75] Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: ECCV (2020) [76] Wan, T., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025) [77] Wang, H., Ghanbari, M., Lin, J.Y., Hu, S., Jin, L., Song, R., Le, T.V., Katsavounidis, I., Aaron, A., Kuo, C.C.J.: MCL-JCV: A JND-based H.264/AVC video quality assessment dataset. In: ICIP (2016) [78] Wang, J., Jia, W., Li, Y.: Exploring CLIP for no-reference image quality assessment. In: ICASSP (2023) [79] Wang, Y.K., Skupin, R., Hannuksela, M.M., Deshpande, S., Hendry, F., Drugeon, V., Seregin, V., Sanchez, Y., Boyce, J.M., Wan, W., Sullivan, G.J.: The high-level syntax of the versatile video coding (vvc) standard. TCSVT (2021) [80] Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. TIP (2004) [81] Wu, C.Y., Singhal, N., Krähenbühl, P.: Video compression through image interpolation. In: ECCV (2018) [82] Xia, Y., Zhou, Y., Wang, J., An, B., Wang, H., Wang, Y., Chen, B.: Diffpc: Diffusion-based high perceptual fidelity image compression with semantic refinement. ICLR (2025) [83] Xie, G., Li, S., Shi, R., Wang, X., Lin, J.H., Lin, S.Y.: Temporal consistency for learned video compression. In: ICME (2021) [84] Xue, T., Chen, B., Wu, J., Wei, D., Freeman, W.T.: Video enhancement with task-oriented flow. IJCV (2019) [85] Yang, R., Timofte, R., Van Gool, L.: Perceptual learned video compression with recurrent conditional gan. In: IJCAI (2022) [86] Yang, R., Mandt, S.: Lossy image compression with conditional diffusion models. In: NeurIPS (2023) [87] Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018) [88] Zhang, S., Mrak, M., Herranz, L., Górriz, M., Wan, S., Yang, F.: Dvc-p: Deep video compression with perceptual optimizations. In: VCIP (2021)