Paper deep dive
Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers
Emanuele Artioli, Farzad Tashtarian, Christian Timmerer
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 9/18/2026, 3:52:56 AM
Summary
The paper introduces PRESLEY, an end-to-end video streaming pipeline that uses generative AI to improve perceptual quality and reduce bitrate. Unlike traditional codecs that treat all regions equally, PRESLEY selectively degrades background regions (which viewers attend to less) and reconstructs them at the client side using generative models conditioned on transmitted visual priors. It extends previous work (ELVIS) by using adaptive in-place degradation rather than block removal, preserving frame structure. The system achieves significant BD-rate reductions (up to -56.4% on background quality) and substantial bitrate savings (up to -29.4%) while maintaining foreground fidelity. The authors also provide a theoretical analysis of the architecture's headroom and a model for predicting post-restoration damage.
Entities (15)
Relation Signals (10)
PRESLEY → extends → ELVIS
confidence 95% · We present PRESLEY, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation...
PRESLEY → uses → Generative AI
confidence 95% · Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers... restoring via generative backbones
PRESLEY → achieves → BD-rate reduction
confidence 90% · PRESLEY achieves a decisive mean -56.4% BD-rate reduction on delivered background quality
PRESLEY → degrades → Background regions
confidence 90% · degrade the regions a viewer attends to least... background details consume bits
PRESLEY → preserves → Foreground fidelity
confidence 90% · maintaining foreground fidelity bit-exact
Generative Layer → reconstructs → Degraded regions
confidence 90% · a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client
Traditional Codecs → treats → Regions alike
confidence 90% · Traditional codecs treat every region of a frame alike
PRESLEY → outperforms → Pristine baselines
confidence 85% · Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings... and superior background quality
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Traditional codecs treat every region of a frame alike; a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client. We present PRESLEY, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting. We separate the problem into three goals: choosing which blocks to degrade, degrading them so the encoder spends fewer bits, and restoring them. Against its predecessor at matched rate, PRESLEY achieves a decisive mean -56.4% BD-rate reduction on delivered background quality across 13 rate ladders spanning multiple codecs and dataset families. Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings (up to -29.4% BD-rate) and superior background quality (17/23 sequences) in the target bit-starved regime, while maintaining foreground fidelity bit-exact. We further map where the theoretical headroom in this class of architecture lies. Using an exact leave-one-superblock-out combinatorial oracle as an additive empirical bound, we show that existing complexity heuristics already capture 83.3% of bit-cost savings, bounding remaining cost-axis headroom at about 5% of total bitrate. We then identify and model the primary unaddressed axis -- post-restoration damage -- which disperses widely (4.9-8.4 dB). We prove that this damage is predictable before transmission (held-out rho = +0.400), establishing the feasibility of transmit-time restorability modeling and defining the roadmap for joint rate-distortion-restoration selection rules.
Tags
Links
- Source: https://arxiv.org/abs/2609.19215v1
- Canonical: https://arxiv.org/abs/2609.19215v1
Trouble viewing inline? Open PDF directly →
Full Text
128,658 characters extracted from source content.
Expand or collapse full text
Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers CCS: Computing methodologies Object identificationCCS: Computing methodologies Artificial intelligenceCCS: Information systems Online analytical processingCCS: Information systems Multimedia streamingCCS: Computing methodologies Concurrent algorithmsCCS: Computing methodologies Image compression Emanuele Artioli email: emanuele.artioli@aau.at Affiliation: Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria, 9020 , Farzad Tashtarian email: farzad.tashtarian@aau.at Affiliation: Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria, 9020 and Christian Timmerer email: christian.timmerer@aau.at Affiliation: Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria, 9020 © none Abstract. Traditional codecs treat every region of a frame alike; a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client. We present PRESLEY, which extends the prior conference work ELVIS Artioli et al., 2025b by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting. We separate the problem into three goals: choosing which blocks to degrade, degrading them so the encoder spends fewer bits, and restoring them. Against its predecessor at matched rate, PRESLEY achieves a decisive mean −56.4 %-56.4\,\% BD-rate reduction on delivered background quality across 13 rate ladders spanning multiple codecs and dataset families. Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings (up to −29.4 %-29.4\,\% BD-rate) and superior background quality (1717/2323 sequences) in the target bit-starved regime, while maintaining foreground fidelity bit-exact. We further map where the theoretical headroom in this class of architecture lies. Using an exact leave-one-superblock-out combinatorial oracle as an additive empirical bound, we show that existing complexity heuristics already capture 83.3 %83.3\,\% of bit-cost savings, bounding remaining cost-axis headroom at ≈5 %≈$5\,\%$ of total bitrate. We then identify and model the primary unaddressed axis—post-restoration damage—which disperses widely (4.9–8.44.9--8.4 dB). We prove that this damage is predictable before transmission (held-out ρ=+0.400ρ=+$0.400$), establishing the feasibility of transmit-time restorability modeling and defining the roadmap for joint rate-distortion-restoration selection rules. Keywords: HTTP adaptive streaming, Generative AI, End-to-end architecture, Quality of Experience † Code available at https://github.com/emanuele-artioli/presley. Extended version of the NOSSDAV ’25 paper ELVIS Artioli et al., 2025b. Submitted to the ACM TOMM special issue on MMSys and co-located workshops. 1. Introduction Video streaming dominates internet traffic, and demand keeps growing with screen resolutions, immersive formats and the number of connected devices. Traditional video compression methods have achieved remarkable efficiency gains through decades of refinement, with each codec generation from AVC Wiegand et al., 2003 to HEVC Sullivan et al., 2012 and VP9 Li et al., 2019, to the most recent iterations, VVC Bross et al., 2021 and AV1 Han et al., 2021 reducing bitrate requirements by approximately 50% compared to their predecessors Minopoulos et al., 2020. Those gains are increasingly hard-won, and they are spent uniformly: a codec works on blocks of pixels, a level of abstraction at which nothing distinguishes the region a viewer is watching from the region behind it. Furthermore, treating video frames as collections of pixel blocks gives up the semantic understanding of scene content. This uniform treatment leads to suboptimal bit allocation: background details consume bits that could be better spent on perceptually important foreground objects, and high-frequency textures are encoded exhaustively even when they could be plausibly reconstructed from context. Content-aware encoding was proposed for exactly this as early as the 2000s Vetro et al., 2003; Kounoudes et al., 2006 and has not gained mainstream adoption. The obstacles are partly practical (tooling, and semantic analysis inside a real-time encoder), but the binding one is perceptual. Coding neighbouring regions at visibly different quality puts a boundary between them, and a quality discontinuity is more objectionable than a uniformly lower quality at the same rate, so the steps must stay small enough to remain invisible and the achievable saving is capped with them. A generative layer changes that constraint rather than working within it: if the client reconstructs the degraded regions instead of displaying them, the delivered frame need not carry the discontinuity that the transmitted one did. We find that it does not. Measured on our own output, degraded blocks lying against undegraded ones are damaged less than degraded blocks in the interior of a degraded region, by about half a decibel of PSNR, because a block with intact neighbours gives the restorer real detail to work from. Generative models trained on massive datasets have learned implicit representations of how objects appear and move in video, letting them plausibly reconstruct missing or degraded regions Zhou et al., 2023; Li et al., 2022; Wang et al., 2021; Huang et al., 2025. This opens a paradigm for compression: rather than encoding all visual information, selectively degrade the less critical regions and leverage client-side models to restore them. PRESLEY (Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers) realizes it. As an extended version of the NOSSDAV ’25 conference paper ELVIS Artioli et al., 2025b, it advances beyond block removal by degrading adaptively in place, preserving frame structure while reducing quality where it is least perceptible, i.e., a generative-AI form of Region-of-Interest encoding, which simplifies codec integration and maintains spatial coherence. Posed this way, the problem separates into three goals: (1) Selection — decide which blocks of which frames to degrade. (2) Reduction — degrade them so that the encoder actually spends fewer bits on them, and relocates those bits to the regions the viewer attends to. (3) Restoration — recover the degraded regions on the client, as close to the original as a generative prior can bring them. The key contributions of this work are fourfold: ■ An architecture. PRESLEY advances the generative streaming paradigm of ELVIS Artioli et al., 2025b by degrading per block in place under a removability mask, at a strength signalled to the client in a bit-packed side channel, preserving frame geometry while remaining fully compatible with unmodified standard codecs. We evaluate four degradation modalities and price them alongside foreground protection: on content where foreground competes for removability, protecting it preserves perceptual fidelity at a modest 6 %6\,\% rate cost. ■ A conditioned restoration path. We introduce client-side strategies in which a generative model refines each block according to its transmitted strength, and define the pyramid by the model’s interface, so the backbone is an interchangeable component. We catalogue ten of them across the transports they are dispatched on. ■ An evaluation protocol. Fixed-QP rate control, region-restricted perceptual metrics, and paired significance testing with correction over every candidate tried. ■ A theoretical headroom bound and restorability model. Moving beyond empirical heuristics, we formulate an exact combinatorial bit oracle that shows existing complexity scores capture 83.3 %83.3\,\% of theoretical bit-cost savings, bounding remaining cost-axis headroom at ≈5 %≈$5\,\%$ of total bitrate under an additive empirical assumption. We then identify and model the second axis—post-restoration damage—proving it is predictable before transmission (held-out ρ=+0.400ρ=+$0.400$) from transmit-time spatial features alone, providing the first principled roadmap for rate-distortion-restoration selection. 2. Background and Related Work Video streaming relies on maximizing encoding efficiency on the server and adapting to network conditions on the client. With the advent of powerful AI models and capable end devices, it is now possible to share the computational burden, leveraging client-side generative AI to boost delivered video quality. 2.1. Redundancy Removal Traditional codecs — AVC/H.264 Wiegand et al., 2003, HEVC/H.265 Sullivan et al., 2012, and the more recent VVC/H.266 Bross et al., 2021 and AV1 Han et al., 2021 — reduce file size by searching exhaustively for inter-frame motion matches and intra-frame patterns. The approach is highly successful and now facing diminishing returns, further bitrate reduction costing extreme encoding complexity García-Lucas et al., 2020. More fundamentally, these codecs operate on blocks of pixels, a level of abstraction at which nothing distinguishes one region of a scene from another: every region is treated alike apart from generic heuristics for smooth against detailed areas, which allocates bits without regard to what a viewer is looking at Ku et al., 2019. Region-of-Interest coding The idea that not all pixels deserve equal bits is not new, and the standards provide hooks: AVC and HEVC accept per-region QP offsets Wedi and Wittmann, 2005, and a body of work builds rate control on them, detecting important regions with object detectors or saliency models and showing perceptual gains at small bitrate overhead Lambert et al., 2006; Xue et al., 2016; Meddeb et al., 2014. It has nevertheless seen little adoption in general streaming, for four reasons. (1) Tooling: FFmpeg’s addroi filter supports only static regions that cannot change per frame Developers, 2025, and the encoder-level interfaces that would allow more are sparsely documented and rarely used Eynnzerr, 2022; x265 Developers, 2025. (2) Cost: identifying important regions demands semantic analysis inside a codec that must also run in real time. (3) Personalization: what counts as important varies with content and with the viewer, so the driving heuristics are difficult to automate across diverse material Artioli et al., 2024. The final reason is more subtle, and shapes the design space more than the others. Coding neighbouring regions at visibly different quality places a boundary between them, and a quality discontinuity is more objectionable than a uniformly lower quality at the same bitrate. ROI coding must therefore keep its quality steps small enough to stay invisible, which complicates its logic and caps what it can save. A system that reconstructs the degraded regions at the client is not bound by that cap, because the delivered frame need not carry the discontinuity the transmitted one did. Semantic Streaming A more radical class of approach transmits high-level descriptors instead of pixels and reconstructs at the client. In video conferencing, SemConf Duan et al., 2025 and NVIDIA Maxine NVIDIA Developer, 2025 send sparse facial keypoints and synthesize the talking head at the receiver, achieving very large reductions by transmitting who is moving and how rather than the pixels that show it. The idea is being extended toward general streaming Jin et al., 2025: GenStream Artioli et al., 2025a transmits skeletal keypoints, camera parameters and a static background model, and reports over 99 %99\,\% bandwidth reduction against HEVC on a sports sequence. These systems show how much redundancy remains once semantics are available, but they are confined to prototypes and favourable scenarios: talking heads are constrained in a way arbitrary content is not, and without standardization the reconstruction method and the bitstream must be designed together, so nothing interoperates with a deployed decoder. Reducing resolution before transmission and reconstructing it at the client sits between these extremes and is established practice: large services deliver downscaled renditions, and receiver-side super-resolution is studied and deployed to recover them. Two properties bound it. The reduction is uniform over the frame, so no region can be protected; and the client is told only which rendition it received, so its restoration is unconditioned on what was degraded or where. 2.2. Video Restoration The client’s task is to reconstruct the decoded video at the highest quality it can, and three families of technique address different parts of it. Frame interpolation raises frame rate by synthesizing intermediate frames, improving fluidity without raising the transmitted rate Niklaus et al., 2017. Modern methods estimate optical flow to do it, adding depth so that closer objects are sampled preferentially through occlusions Bao et al., 2019 or self-attention for long-range motion Shi et al., 2022, but unpredictable motion still produces ghosting where the motion estimate is wrong. Super-resolution reconstructs high-resolution frames from low-resolution input, from upscaling filters such as bicubic and Lanczos Moukthika, 2025 to adversarially trained networks like ESRGAN Wang et al., 2018, which optimize for perceptual detail rather than pixel error. Video-specific models add temporal reasoning Cao et al., 2021, and interpolation and super-resolution can be combined in one framework Kim et al., 2020. Cost remains the barrier to live deployment, and because these models operate on low-level features they cannot reason about what an occluded region contains. Video in-painting reconstructs content that is missing rather than merely degraded. Early methods propagated patches from neighbouring frames Patwardhan et al., 2005 and flickered across them; global optimization improved consistency at considerable cost Newson et al., 2014. Generative models changed what is achievable, filling all frames jointly with spatial attention for texture and temporal attention for correspondence Zeng et al., 2020, or propagating content along estimated motion Gu et al., 2023. What none can do is recover the specific content that was removed, they only synthesize something plausible. 2.3. Quality Assessment for Generative Video Traditional full-reference metrics like PSNR, SSIM, and VMAF Netflix, 2016 are standard for evaluating codec fidelity. However, they heavily penalize generative reconstructions that are structurally misaligned but perceptually realistic Onuoha et al., 2023. To accurately assess AI-enhanced video, learned perceptual metrics are required. LPIPS Zhang et al., 2018 computes similarity in deep feature space, correlating much closer with human judgment for GAN-generated content, as does DISTS Ding et al., 2022, which additionally separates structure from texture similarity so that plausible re-textured detail is not penalized as heavily as it would be by pixel-aligned metrics. Because generative pipelines can also introduce localized hallucinations or temporal jitter, distributional metrics against the reference are often used: the Fréchet Inception Distance (FID) Yu et al., 2021 for spatial realism, and the Fréchet Video Motion Distance (FVMD) Liu et al., 2024 for temporal consistency. The foreground claims in this article rest on LPIPS and DISTS restricted to a mask, because the foreground variants the others expose are bounding-box crops and a box drawn around these sequences contains much of the background the method degrades on purpose. All seven measures are reported at whole-frame scope, where that objection does not apply, in Appendix F. 3. PRESLEY Figure 1. PRESLEY System Architecture Overview.A block diagram of the end-to-end pipeline in three horizontal lanes. The top lane is the server and runs left to right; the encoded video crosses a cloud labelled Network on the right; the bottom lane is the client and runs right to left; a middle lane holds performance monitoring, which receives from both sides and feeds a controller that closes the loop back to the server. Data: "edges":[["server","network"],["network","client"],["server","performance monitoring"],["client","performance monitoring"],["performance monitoring","controller"],["controller","server"]],"lanes":"client":["decode video","restore frames","render video"],"control":["performance monitoring","controller"],"network":["transmit encoded video and side channel"],"server":["extract frames","calculate importance","degrade frames","encode video"] PRESLEY builds on ELVIS Artioli et al., 2025b, which integrates a generative layer into a streaming pipeline by frame shrinking: the server scores blocks, removes the most removable, and the client stretches frames back to size and in-paints the gaps. PRESLEY reworks that into a codec-friendly, non-shrinking transport, and supports two degradation families on it. The first keeps every block in place and replaces the content of the most removable ones with a flat fill or with the co-located block from the previous frame: ELVIS’s idea of giving a block up entirely, but given up in place, so frame geometry never changes and no stretching is needed before restoration. The second punches no holes at all: it degrades the most removable blocks in place by lowering their spatial resolution, so every pixel stays partially present in the transmitted stream. Because nothing is discarded, the client can pair it with a conditioned restoration model that consumes the degraded pixels directly rather than an in-painter that must hallucinate the region from neighbouring frames. Both families share the modular framework of Figure 1: server-side preprocessing and removability scoring, selective degradation, standard network transmission, and client-side restoration. 3.1. Server side Degradation Pipeline The server preprocesses frames — extraction, rescaling, and lossless encoding into a reference video against which subsequent distortion is assessed — and then begins the degradation process of Figure 2. The first step decides which blocks are the best candidates, weighing the trade-off that decision carries: degrading complex blocks lowers the bitrate more than degrading simple ones, but also makes them harder to recreate. Removability Calculation. Figure 2. Server side pipeline: removability calculation and adaptive degradation.Four numbered stages of the server-side pipeline, left to right, each shown as a stack of frames overlaid with a block grid and the equation that produces it: complexity, smoothing, region priority, then degradation and encoding. Data: "note":"the degraded blocks are shaded by strength; the metadata matrix is binary at lambda=1","stages":["does":"per-block spatial S and temporal T complexity from EVCA","equation":"C = alpha*S + (1-alpha)*T","n":1,"does":"smooth across consecutive frames","equation":"C' = beta*C\_n + (1-beta)*C\_n-1","n":2,"does":"apply the foreground mask so foreground blocks are excluded","equation":"R = -C' where the block is foreground","n":3,"does":"degrade the selected blocks, encode, emit the metadata bitmap","equation":null,"n":4] ELVIS and PRESLEY resolve it by degrading complex blocks and letting the restoration model reconstruct them from spatial and temporal neighbours, prioritizing the background and limiting degradation of the main object, where the viewer’s eyes focus. Equations 1–4 below compute a removability score for every block: high where a block is a good candidate for degradation, low where it is not. It requires (1) block-wise spatial (S) and temporal (T) complexity tensors — detail within a block, and motion between corresponding blocks in successive frames — computed at the configured block size with the Enhanced Video Complexity Analyzer (EVCA) Amirpour et al., 2024, and (2) a segmentation tool separating foreground from background. The complexity of block bi,j,nb_i,j,n is a weighted aggregation of the two: (1) Ci,j,n=α⋅Si,j,n+(1−α)⋅Ti,j,n+1,Ci,j,N−1=Si,j,N−1C_i,j,n=α· S_i,j,n+(1-α)· T_i,j,n+1, C_i,j,N-1=S_i,j,N-1 where α controls the relative influence of spatial and temporal factors (Figure 2, step (1)). The temporal term is read from frame n+1n+1 rather than n: EVCA reports the complexity of the transition into a frame, so Ti,j,n+1T_i,j,n+1 is the motion a decision taken at frame n must survive. The final frame has no successor and falls back to spatial complexity alone. Foreground-background segmentation uses object detection models (e.g., UFO Su et al., 2023), returning a mask Mi,j,nM_i,j,n marking blocks where the object is present (11) or absent (00); background blocks are given priority by scaling their complexity by a constant γ (Figure 2, step (2)): (2) C^i,j,n=Γi,j,n⋅Ci,j,n,Γi,j,n=γ−(γ−1)Mi,j,n C_i,j,n= _i,j,n· C_i,j,n, _i,j,n=γ-(γ-1)\,M_i,j,n The mask M is 11 on the foreground and 00 on the background, so Γ is 11 on the foreground and γ on the background: a foreground block keeps its complexity unchanged, and a background block has its removability raised by γ, with γ=10γ=$10$ throughout. This is a soft priority, shown as step (2) of Figure 2: it scales the two populations apart without separating them, so a sufficiently complex foreground block can still outrank a simple background one. Whether that residual overlap matters is a question about content rather than about the score, and we return to it once the selection rule is defined. To further refine block degradation decisions, the smoothed score Ci,j,n′C _i,j,n is defined as follows: (3) Ci,j,n′=β⋅C^i,j,n+(1−β)⋅C^i,j,n−1,Ci,j,0′=C^i,j,0C _i,j,n=β· C_i,j,n+(1-β)· C_i,j,n-1, C _i,j,0= C_i,j,0 where β is a tunable parameter controlling the degree of temporal smoothing, as shown in Figure 2, step (3). Values below 11 average a block’s score with its own value in the previous frame, so a block whose complexity fluctuates frame to frame does not flicker in and out of the selection. Two properties of the form bound what it can express, and both bear on the ablation of Section 4.5: smoothing is applied after the region priority of Equation 2, against the previous frame’s unsmoothed score, so a frame’s influence decays after exactly one step. Finally, the smoothed scores are min-max normalized over the whole tensor to give the removability score used by every downstream step: (4) Ri,j,n=Ci,j,n′−minC′maxC′−minC′∈[0,1]R_i,j,n= C _i,j,n- C C - C ∈[0,1] Higher R means more removable. The normalization is what lets a single scalar budget and threshold carry the same meaning across sequences whose EVCA scores differ by orders of magnitude. This is PRESLEY’s answer to the selection goal. Every term entering Equation 1 is a complexity measure, and complexity is a proxy for the number of bits a block costs the encoder; the equations that follow reorder that estimate by region, smooth it over time and rescale it. An ideal score would weigh those bits against the other quantity a selection decision depends on — how faithfully a block can be reconstructed once degraded — and maximize bits freed per unit of damage surviving restoration. We do not model that second quantity here, for a reason of measurability rather than of principle: it is a property of the restored output, which the server does not have at the moment it must choose, and no transmit-time proxy for it was available when the score was specified. The formulation inherited from ELVIS therefore expresses the numerator alone, and Section 4.5 takes up what that omission costs. Degradation Strategy. Both systems act on the same score. In ELVIS, the blocks with the highest removability are dropped and the frame is shrunk one block-row at a time until the configured shrink amount is reached, so that surviving blocks still repack into a rectangle a codec will accept. PRESLEY removes nothing. It degrades the most removable regions in place, so frame geometry never changes and the per-row constraint disappears with it. Block selection is shared across every degradation described below, and turns the score R of Equation 4 into a binary map σ of degradable blocks in three steps. A global budget, not a per-row count: ELVIS removes a fixed number of blocks from every row because its survivors must repack into a rectangle, but PRESLEY keeps frame geometry, so that constraint can be relaxed. PRESLEY takes the globally most-removable k blocks, with k=⌊a⋅J⌋⋅Ik= a· J · I for a configurable budget a — the same total the per-row rule would have spent, capped at the number of eligible blocks. Spatial clustering before ranking: the score map is smoothed with a small Gaussian kernel before the top-k is taken, so selected blocks form contiguous patches rather than isolated singletons, because a codec’s block-level prediction reproduces one large uniform region far more cheaply than the same area of confetti. Hard foreground exclusion, applied after the blur: foreground blocks are removed from candidacy outright, so foreground content is never degraded whatever its score. The exclusion is a design response to what we saw in early runs: on content whose foreground is complex enough to compete, the soft priority of Equation 2 left a fraction of foreground blocks inside the selection, and raising γ did not clear them, because a factor applied to every background score alike cannot express a constraint that the ranking must never cross a region boundary. What the exclusion buys is a clean attribution rather than a better average. With it on, foreground blocks are carried through from the transmitted frame untouched by the generative layer — not bit-exact against the source, since they are still lossily coded at the same QP as everything else, but identical across restorers. Any foreground difference between two configurations is therefore the codec’s doing and not the restorer’s, which is what makes the backbone catalogue of Section 4.3 a comparison of backgrounds. The exclusion is applied after the clustering blur rather than before, since a blur applied first smears foreground scores across the boundary and lets excluded content back into the ranking through its neighbours. It is a configuration flag rather than a property of the transport, and turning it off is a coherent operating point: the soft priority of Equation 2 still applies, so the foreground is degraded last rather than never, and the mask-restricted metrics measure whatever ends up in the region either way. We enable it throughout this article so that the configuration comparisons isolate one variable at a time, and Section 4.5 measures what it costs in bitrate to keep it. PRESLEY supports two families of degradation modality, differing in how much of ELVIS they carry: hole-filling degradations, which take ELVIS’s block-removal insight onto the new transport, and downsampling, which departs from it entirely. Hole-filling degradations. Selected blocks are replaced in place, either with the flat mean color of the block (mean_fill, referred to as blackout) or with the corresponding block copied from the previous degraded frame (freeze). Both leave a low-information prior in place of the original content: a flat block costs a codec almost nothing (a single DC coefficient plus strong prediction), and a copied block is typically coded as a near-zero-cost inter-skip. What ELVIS supplies is the idea that the most removable blocks can be given up entirely and synthesized at the client; what is new here is that they are given up in place. Keeping frame geometry is what removes the per-row budget, the block shifting and the frame stretching, and it is what makes the global selection described above available to this family at all — none of which ELVIS’s shrink transport can express. These degradations are the transport paired with the video in-painters of Section 3.2. Downsampling. Rather than discarding a block’s content, downsampling reduces its information density. Each selected block is assigned a strength level L, and the transmitted frame is produced by downsampling that block by 2L2^L using pixel area relation Dong, 2018 and immediately resampling it back to b×b× b by bilinear interpolation, so the transmitted frame keeps its original resolution while carrying less high-frequency detail where it was degraded. The level map is (5) Li,j,n=σi,j,n⋅max(1,⌊λ⋅Ri,j,n⌉)L_i,j,n= _i,j,n· (1,\; λ· R_i,j,n ) where σ is the block selection just described, λ the number of available levels, and ⌊⋅⌉ · rounds to nearest. Selection and strength are two decisions read off one score, and the equation keeps them consistent: σ decides whether a block is degraded, λRλ R decides how far, and the level is defined on the selected set with a minimum of one, since a selected block at level 00 would be a contradiction and would put the realized removal rate below the budget. The range of λ is set by the block size rather than chosen freely. A block of side b at level L is carried at b/2Lb/2^L samples per side, so λ≤log2(b/bmin)λ≤ _2(b/b_ ). We take bmin=8b_ =$8$: below that a block holds fewer samples than the smallest transform a codec applies, and a conditioned restorer has nothing left to condition on — the degradation has become a hole fill. Graded strength is therefore a large-block instrument: at b=16b=$16$ the bound admits a single level and Equation 5 collapses to a fixed 2×2× reduction on every selected block, while at b=64b=$64$ it admits three and the deepest QP still arrives as 8×88× 8. Four more operators act on the same selection and reduce a block’s information without emptying it. Blur. Each selected block is convolved with a Gaussian kernel, one pass per strength level, removing high-frequency detail while leaving low-frequency structure and the block mean in place. It differs from downsampling in what it does to the sampling grid — downsampling discards samples and reconstructs them by interpolation, blurring keeps every sample and attenuates the detail it carries — but both leave a prior a conditioned restorer can consume. Transform-domain truncation. Rather than filtering on the pixel plane, this operator writes exact zeros in the codec’s own basis. Each selected block is converted to YCrCb and partitioned into 8×88× 8 sub-blocks — the smallest transform AV1 applies — a 2-D DCT is taken per channel, every coefficient outside the top-left κ×κ×κ corner is zeroed, and the sub-block is reconstructed by an inverse DCT. The motivation is alignment rather than severity: a pixel-plane low-pass straddles transform boundaries and leaves residual energy the encoder must still code, while zeroing coefficients on the codec’s own grid removes exactly the terms it would otherwise spend bits on. QP Mapping. Where codecs support QP maps, per-block offsets can be set proportionally to removability: (6) QPi,j,n=QPbase+(QPmax−QPbase)⋅Ri,j,nQP_i,j,n=QP_base+(QP_max-QP_base)· R_i,j,n Here QPbaseQP_base is the codec’s baseline QP, QPmaxQP_max the maximum allowable, and R the removability score normalized to [0,1][0,1]. Two properties distinguish this route. It re-quantizes DCT coefficients on the pixel plane before encoding rather than driving the codec’s own per-block control, so the encoder receives an already-quantized frame and quantizes it again; and because available codecs accept only per-GoP QP maps, frame-level maps must be averaged across each GoP, so a block receives the strength its neighbourhood in time received. This route is also where the soft priority of Equation 2 does its remaining work: it consumes R directly, with no selection step and so no hard exclusion, which makes γ the only foreground protection it has. The same is true of ELVIS’s shrink transport. And because Γ is applied before the temporal smoothing of Equation 3, a block whose region membership changes between consecutive frames carries a γ-scaled value into the smoothed score, so the priority still moves the ranking at foreground boundaries under motion, where membership is least stable. Noise Injection. An alternative degradation reuses the level map of Equation 5 as rounds of additive Gaussian noise, one per level. It is the one modality that does not reduce a block’s information content — it adds high-frequency detail — so it separates the two things a degradation is asked to do at once: look acceptable to a viewer, and look cheap to a codec. Encoding and Metadata. Step (4) of Figure 2 applies the chosen degradation to the selected blocks, encodes the result with an unmodified codec, and emits the side channel the client needs to undo it. ELVIS shrinks frames during encoding, so its client must know which blocks were removed to restore the original geometry. PRESLEY preserves every block, so each keeps its position while its quality varies, and the client instead requires metadata identifying which blocks were degraded and by how much: the level map L of Equation 5, which reduces to a binary mask for the hole-filling degradations and for λ=1λ=1. The map is bit-plane packed into ⌈log2(Lmax+1)⌉ _2(L_ +1) binary planes, eight blocks to the byte, and DEFLATE-compressed, giving an uncompressed cost of (7) Bside=⌈18I⋅J⋅N⌉⋅⌈log2(Lmax+1)⌉ bytesB_side= 18\,I· J· N · _2(L_ +1) bytes against the 32IJN32\,IJN bits an uncompressed integer map would need. This tells the client how many rounds of degradation each block went through, and therefore how many rounds of restoration it needs. The cost is counted inside transmitted_size_bytes and therefore inside every rate comparison in this article, and Section 4 quantifies it as a low single-digit percentage of the transmitted budget. 3.2. Client side Restoration Pipeline Restoration is driven by the degradation applied before encoding. In ELVIS, where frames are shrunk, it requires decoding the shrunk file, stretching the frames back to full resolution from the metadata, and passing the video to an in-painting model to generate the missing content. Figure 3. Client side pipeline: metadata parsing, restoration and downstream tasks.Four numbered stages of the client-side pipeline, left to right: decode the degraded frames and metadata, expand the metadata into a per-block grid of restoration strength levels, run the restoration model adaptively over that grid, then buffer and render the restored frames. Data: "note":"levels shown in the example grid are 0,1,2; every reported result uses the binary case","stages":["does":"decode degraded frames and the metadata bitmap","n":1,"does":"expand metadata into a per-block level grid","n":2,"does":"run the restoration model adaptively per block","n":3,"does":"buffer and render","n":4] Unlike ELVIS, PRESLEY preserves all blocks but adaptively degrades them. Consequently, restoration for the downsampling family is performed via generative enhancement, tailored to the specific degradation introduced, while restoration for the hole-filling family continues to be performed via video in-painting, exactly as in ELVIS – the difference being that PRESLEY’s transport never altered frame geometry, so no stretching step is needed before in-painting. Decoding and metadata parsing. The first step on the client side, as shown in Figure 3, step (1) is receiving and decoding the degraded video and the metadata. Adaptive Restoration. Based on the degradation strategy used at the server, and detailed in the metadata, as per Figure 3, step (2), PRESLEY applies the corresponding restoration method, orchestrated by Algorithm 1. Algorithm 1 Adaptive Restoration 1: inF_in: sequence of decoded degraded frames 2: L: metadata map – per-block strength levels (Eq. 5), or per-block hole flags 3: mode∈SR,Inpaintmode∈\SR,Inpaint\: restoration mode, set by the degradation family used at the server 4: outF_out: restored frame sequence 5: if mode=SRmode=SR then 6: for each frame Fin∈inF_in _in with level map L do 7: Di,j←2Li,jD_i,j← 2^L_i,j ⊳ levels are exponents; D=1D=1 means untouched 8: m←max(D)m← (D) 9: F←Resize(Fin,1/m)F (F_in,1/m) ⊳ down to the coarsest block’s true resolution 10: s←m/2s← m/2 11: while s≥1s≥ 1 do 12: F,D←Alg. 2(F,Fin,D,s)F,D .~ alg:sr_round(F,F_in,D,s) ⊳ Appendix C 13: s←s/2s← s/2 14: end while 15: Append F to outF_out 16: end for 17: else if mode=Inpaintmode=Inpaint then 18: out←VideoInpaint(in,UpsampleToPixels())F_out (F_in,UpsampleToPixels(L)), composited inside the mask only 19: end if 20: return outF_out If blocks were downsampled, PRESLEY restores them through progressive super-resolution rounds. The client first downsamples the entire frame to the maximum downsampling factor used. That preliminary reduction is what makes the subsequent upscaling well posed: the received video already matches the reference resolution, so a low-resolution input has to be constructed for the model, and doing it globally and once confines any downsampling artefact to this stage rather than to the high-priority blocks re-injected later. It then iteratively applies a super-resolution model to upsample the frame by 2×2× per round. The round is defined by the model’s interface, not by a particular model: any restorer taking a frame and returning it at twice the resolution can occupy the slot, which makes the backbone interchangeable. In each round, as detailed in Algorithm 2 (Appendix C), blocks that have reached their target resolution are replaced with content from the decoded degraded frame resized to the current pyramid level, so only blocks still short of their transmitted detail keep the model’s output. That re-injection is the super-resolution family’s equivalent of the passthrough compositing described below, with the same consequence: non-degraded blocks are carried through as decoded, so foreground quality does not depend on which backbone is used. The depth of the pyramid follows λ: a block leaves the loop at the QP matching its own level, so a block degraded once is re-injected earlier, and passes through fewer model applications, than one degraded three times. If blocks were degraded by a hole fill (blackout/mean_fill or freeze), the inpainting model receives the decoded frames with the hole mask and synthesizes content for the masked region from neighbouring frames, discarding whatever was transmitted inside the mask. The result is passthrough-composited with the decoded frame — pixels outside the hole kept exactly as decoded, only those inside taken from the in-painter — so foreground quality, never degraded by the hard exclusion of Section 3.1, is reproduced bit-exact whichever in-painter is used. Because the in-painter discards the transmitted prior rather than consuming it, this path is restricted to the hole-filling degradations; it is the mechanism ELVIS uses, carried over except that no stretching step is required first. Finally, the restored frames are grouped into segments, buffered and rendered by any traditional video player, or passed to evaluation. 4. Performance Evaluation We evaluate PRESLEY’s performance by implementing its pipeline onto a Ubuntu machine, running experiments that link configuration parameter values to variations in output quality, and evaluate results both in terms of quality across several State-of-the-Art (SOTA) image and video metrics, and in terms of throughput added by the degradation/restoration layer. 4.1. Evaluation Setup Every comparison is against one of three targets: a pristine baseline at the same encoder configuration, ELVIS’s block-removal transport at matched rate, or a matched none control running the identical transport with no restorer. Rate is computed as the full transmitted payload, i.e., video plus side channel. Metrics, and how a difference becomes a claim Foreground and background are scored separately throughout, because the design degrades one and protects the other. For LPIPS and DISTS the region restriction is a mask weighting of the spatial feature maps rather than a crop, and background PSNR is never a verdict: the flat mean fill scores the best background PSNR of any fill we tested while being the worst-looking. No subjective study was conducted, and claims rest on an exact two-tailed sign test over videos, Holm-corrected across every candidate tried including the losers. Appendix B gives both procedures in full. Restoration backbones Which restorer a run dispatches is fixed by its transport. Every model named here was run and is reported. On the conditioned path we run Real-ESRGAN Wang et al., 2021, BSRGAN Zhang et al., 2021, Real-HAT-GAN Chen et al., 2023 and Stream-DiffVSR Shiu et al., 2025 on the identical pyramid, with Real-ESRGAN carrying the corpus and the others measured against it at a shared operating point; on the blur transport, InstantIR Huang et al., 2025, NAFNet Chen et al., 2022 and an unsharp trivial control; on the hole-filling path, ProPainter Zhou et al., 2023 and E2FGVI Li et al., 2022 with Telea’s classical in-painting Telea, 2004. Every configuration is additionally run with the restorer disabled, giving each its own matched unrestored control, which separates the cost of the transport from the recovery the restoration achieves. Corpus and hardware Experiments ran on an Ubuntu 22.04 server with an AMD EPYC 7713 (64 cores, 256256 threads), 1 TB1\,TB of RAM and two NVIDIA RTX A6000 GPUs (48 GB48\,GB each). Sequences come from DAVIS Pont-Tuset et al., 2017, chosen for variety of composition and camera motion. DAVIS is small and widely used, so a result measured only on it cannot be separated from the corpus it was tuned against; we therefore add held-out clips from MOSE Ding et al., 2023; Fudan CVL, 2025 and YouTube-VOS Xu et al., 2018, two segmentation benchmarks that supply the per-frame object masks this pipeline needs and that neither this work nor ELVIS has ever tuned on. An initial randomized sweep over block size, removal budget, α, β and three renditions is complemented by the deterministic one-parameter-at-a-time ablation of Section 4.5. 4.2. Hole-filling Evaluation Table 2 evaluates bitrate reduction under x265 across four fixed QPs (30/32/34/37) at 640640×360360. On sequences such as bear and camel, the hole-filling transport achieves substantial bitrate savings of −29.4%-29.4\% and −20.6%-20.6\% BD-rate alongside foreground gains of +1.02+1.02 dB and +0.62+0.62 dB BD-PSNR, demonstrating effective bit relocation from background to foreground. On other sequences such as dog, pigs, and india, the codec requires more bits at these settings. Across the wider evaluation that follows, we analyze how this trade-off behaves across codecs, dataset families, and operating regimes. Video FG BD-PSNR (dB) FG BD-rate (%) Overlap bear +1.02+1.02 −29.4-29.4 0.59 camel +0.62+0.62 −20.6-20.6 0.70 tennis +0.16+0.16 −3.8-3.8 0.74 dog −0.34-0.34 +13.8+13.8 0.70 pigs −0.50-0.50 +21.2+21.2 0.69 india −0.40-0.40 +10.6+10.6 0.78 Table 1. Hole-filling (blackout fill, block size 8, ProPainter, passthrough compositing, fg_protect) vs. the plain x265 baseline: foreground BD-PSNR and BD-rate over four matched fixed-QP points (30/32/34/37) per video at 640640×360360. Negative BD-rate indicates fewer bits for equal quality. “Overlap” is the shared log-bitrate fraction. best worst QP 32 QP 37 QP 42 QP 47 Clips saving bits 1919/2323 1616/2323 55/2323 11/2323 Median rate change −6.6%-6.6\% −2.5%-2.5\% +10.4%+10.4\% +31.4%+31.4\% Median BG gap +0.0409+0.0409 +0.0286+0.0286 +0.0131+0.0131 −0.0153-0.0153 Clips with better BG 00/2323 22/2323 33/2323 1717/2323 Median FG gap +0.0047+0.0047 +0.0028+0.0028 +0.0044+0.0044 +0.0007+0.0007 Table 2. The rate–quality trade across the QP sweep, PRESLEY against the pristine baseline at matched QP on 2323 clips. Gaps are LPIPS differences (PRESLEY minus baseline, negative favors PRESLEY). Foreground (FG) differences remain well within the 0.050.05 perceptual margin at all QPs; the background (BG) reflects the trade-off. Figure 4. Bitrate delta against the pristine baseline on 2323 clips where baseline, ELVIS, and PRESLEY all exist at both QPs, grouped by dataset family. Left of zero indicates bitrate savings.Two panels of horizontal bars, one per arm, 44 clip-level outcomes in total, each grouped by dataset family into DAVIS and the held-out MOSEv2 and YouTube-VOS sets. Each bar is the bitrate that arm needs against a pristine baseline at the same fixed QP, so left of zero is a saving. Across both arms 29 of 44 clips save bits. The split runs within families rather than between them, which is the article's content-dependence result shown as a distribution rather than as a win rate. Bars marked with a cross saved bits while their foreground quality moved against the baseline, so that saving is not free. Data: "arms":"PRESLEY (downsample)":"clips\_costing\_bits":4,"clips\_saving\_bits":7,"median\_by\_family":"DAVIS":-26.0,"MOSEv2 (held out)":-16.3,"YouTube-VOS (held out)":2.7,"median\_pct":-14.0,"n\_clips":11,"per\_clip":["bitrate\_delta\_pct":-62.7,"fg\_lpips\_delta":0.0711,"video":"bear","bitrate\_delta\_pct":10.7,"fg\_lpips\_delta":0.0149,"video":"camel","bitrate\_delta\_pct":-40.9,"fg\_lpips\_delta":0.0048,"video":"mosev2/8i1uo3x9","bitrate\_delta\_pct":-21.4,"fg\_lpips\_delta":0.0042,"video":"mosev2/fii86rku","bitrate\_delta\_pct":-16.3,"fg\_lpips\_delta":0.0018,"video":"mosev2/ptq7rtia","bitrate\_delta\_pct":-14.0,"fg\_lpips\_delta":-0.0006,"video":"mosev2/zofozj6l","bitrate\_delta\_pct":-6.3,"fg\_lpips\_delta":0.0026,"video":"mosev2/jxmcdk8k","bitrate\_delta\_pct":-42.6,"fg\_lpips\_delta":0.0198,"video":"youtube\_vos/30fe0ed0ce","bitrate\_delta\_pct":0.1,"fg\_lpips\_delta":0.0021,"video":"youtube\_vos/282651c6f7","bitrate\_delta\_pct":5.4,"fg\_lpips\_delta":0.0065,"video":"youtube\_vos/b1a8a404ad","bitrate\_delta\_pct":8.8,"fg\_lpips\_delta":0.0037,"video":"youtube\_vos/0e4068b53f"],"saving\_with\_foreground\_cost":["bear","youtube\_vos/30fe0ed0ce"],"bridge (ELVIS blackout)":"clips\_costing\_bits":11,"clips\_saving\_bits":22,"median\_by\_family":"DAVIS":-3.1,"MOSEv2 (held out)":-31.6,"YouTube-VOS (held out)":-6.2,"median\_pct":-9.1,"n\_clips":33,"per\_clip":["bitrate\_delta\_pct":-83.0,"fg\_lpips\_delta":0.0984,"video":"bear","bitrate\_delta\_pct":-56.1,"fg\_lpips\_delta":0.0392,"video":"camel","bitrate\_delta\_pct":-29.0,"fg\_lpips\_delta":0.1015,"video":"scooter-board","bitrate\_delta\_pct":-23.1,"fg\_lpips\_delta":0.0247,"video":"elephant","bitrate\_delta\_pct":-21.5,"fg\_lpips\_delta":0.0495,"video":"scooter-black","bitrate\_delta\_pct":-17.7,"fg\_lpips\_delta":0.1212,"video":"dogs-jump","bitrate\_delta\_pct":-14.7,"fg\_lpips\_delta":0.0822,"video":"motorbike","bitrate\_delta\_pct":-12.6,"fg\_lpips\_delta":0.0737,"video":"drift-turn","bitrate\_delta\_pct":-12.6,"fg\_lpips\_delta":0.0763,"video":"color-run","bitrate\_delta\_pct":-9.6,"fg\_lpips\_delta":0.0366,"video":"breakdance","bitrate\_delta\_pct":-9.1,"fg\_lpips\_delta":0.0307,"video":"dancing","bitrate\_delta\_pct":-5.0,"fg\_lpips\_delta":0.0373,"video":"bike-packing","bitrate\_delta\_pct":-1.2,"fg\_lpips\_delta":0.0056,"video":"tennis","bitrate\_delta\_pct":-0.9,"fg\_lpips\_delta":0.0667,"video":"drift-straight","bitrate\_delta\_pct":0.0,"fg\_lpips\_delta":0.1075,"video":"dogs-scale","bitrate\_delta\_pct":1.2,"fg\_lpips\_delta":0.0738,"video":"libby","bitrate\_delta\_pct":4.7,"fg\_lpips\_delta":0.1408,"video":"lindy-hop","bitrate\_delta\_pct":5.8,"fg\_lpips\_delta":0.0068,"video":"india","bitrate\_delta\_pct":6.2,"fg\_lpips\_delta":0.0092,"video":"dog","bitrate\_delta\_pct":7.3,"fg\_lpips\_delta":0.1683,"video":"mallard-fly","bitrate\_delta\_pct":7.8,"fg\_lpips\_delta":0.0396,"video":"train","bitrate\_delta\_pct":10.9,"fg\_lpips\_delta":0.0089,"video":"pigs","bitrate\_delta\_pct":13.9,"fg\_lpips\_delta":0.2108,"video":"bmx-bumps","bitrate\_delta\_pct":45.9,"fg\_lpips\_delta":0.1296,"video":"planes-water","bitrate\_delta\_pct":-56.0,"fg\_lpips\_delta":0.0053,"video":"mosev2/8i1uo3x9","bitrate\_delta\_pct":-33.9,"fg\_lpips\_delta":0.0053,"video":"mosev2/ptq7rtia","bitrate\_delta\_pct":-31.6,"fg\_lpips\_delta":0.0113,"video":"mosev2/fii86rku","bitrate\_delta\_pct":-28.9,"fg\_lpips\_delta":0.0053,"video":"mosev2/zofozj6l","bitrate\_delta\_pct":-10.9,"fg\_lpips\_delta":0.0294,"video":"mosev2/jxmcdk8k","bitrate\_delta\_pct":-61.9,"fg\_lpips\_delta":0.0487,"video":"youtube\_vos/30fe0ed0ce","bitrate\_delta\_pct":-7.4,"fg\_lpips\_delta":0.0057,"video":"youtube\_vos/282651c6f7","bitrate\_delta\_pct":-5.0,"fg\_lpips\_delta":0.0082,"video":"youtube\_vos/b1a8a404ad","bitrate\_delta\_pct":13.5,"fg\_lpips\_delta":0.0051,"video":"youtube\_vos/0e4068b53f"],"saving\_with\_foreground\_cost":["bear","camel","scooter-board","elephant","scooter-black","dogs-jump","motorbike","drift-turn","color-run","breakdance","dancing","bike-packing","drift-straight","mosev2/fii86rku","mosev2/jxmcdk8k","youtube\_vos/30fe0ed0ce"],"quantity":"median over QP rungs of (arm bitrate - baseline bitrate) / baseline bitrate, percent, paired within (video, QP)","replaces\_tables":["tab:breadth","tab:breadth-ext","tab:breadth-ext-presley","tab:breadth-ratematched"],"rungs":[32,37],"totals":"clips":44,"costing\_bits":15,"saving\_bits":29,"why\_not\_a\_win\_rate":"a win rate invites 'the method works N\% of the time'; the honest content is a distribution whose sign is content-dependent, which is the article's position" Figure 5. PRESLEY against ELVIS at matched rate on 1313 rate ladders across two codecs and three dataset families. Left: delivered background LPIPS after restoration, consistently favoring PRESLEY on all 13 ladders. Right: foreground LPIPS.Two sorted horizontal bar charts sharing one axis of 13 ladders. Left: BD-rate on background LPIPS for PRESLEY against ELVIS at matched rate; every one of the 13 bars is below zero, mean -56.4 percent, exact two-tailed sign p = 0.000244, surviving a Holm family of up to 204 candidates. Right: the same ladders on foreground LPIPS, where bars fall on both sides of zero and 6 of 13 lie outside a shaded plus or minus 15 percent band. So the background win does not come at a systematic foreground cost, which is a weaker and more accurate statement than saying the foreground is unchanged. Data: "anchor":"ELVIS; negative means PRESLEY needs fewer bits at equal quality","bd\_rate\_bg\_lpips\_pct":[-78.5,-74.8,-68.8,-67.6,-63.2,-61.6,-60.6,-59.5,-52.8,-43.4,-43.0,-39.2,-20.5],"bd\_rate\_fg\_lpips\_pct":[-14.5,-10.8,1.4,-3.3,-16.5,-22.6,-21.6,-24.3,12.7,1.6,9.4,17.0,28.0],"bg\_mean\_pct":-56.4,"bg\_wins":13,"codecs":["svtav1","x265"],"comparison":"PRESLEY vs ELVIS at matched rate","dataset\_families":["DAVIS","MOSEv2","YouTube-VOS"],"fg\_outside\_15pct\_band":6,"fg\_reading":"the background win does not come at a systematic foreground cost; the foreground is NOT uniformly a wash and moves in both directions","holm\_family\_survived":204,"ladders":["camel","youtube\_vos/b1a8a404ad","bear","youtube\_vos/282651c6f7","pigs","youtube\_vos/0e4068b53f","dog","mosev2/jxmcdk8k","mosev2/ptq7rtia","mosev2/fii86rku","youtube\_vos/30fe0ed0ce","mosev2/zofozj6l","mosev2/8i1uo3x9"],"n\_ladders":13,"sign\_p\_two\_tailed":0.000244 Figure 5 broadens this evaluation across 23 clips from DAVIS, MOSEv2, and YouTube-VOS where the pristine baseline, ELVIS and PRESLEY all exist at both QPs. ELVIS saves bits on 1717 of 2323 clips (median −10.9 %-10.9\,\%) and PRESLEY on 1515 of 2323 (median −5.0 %-5.0\,\%), and on both the sign splits within dataset families rather than between them. Foreground protection survives outside DAVIS on both, but on different metrics: PRESLEY’s foreground LPIPS delta clears the margin at none of 4747 operating points, while ELVIS’s exceeds it at 4444 of 5050 even though its foreground PSNR moves −0.17-0.17 dB against PRESLEY’s −0.16-0.16. A learned metric compares deep features whose receptive fields reach beyond the block, so an emptied neighbour changes a protected block’s features without changing its pixels. We therefore report foreground protection on PSNR wherever the transport leaves holes, and on LPIPS where it does not. The two QPs of Figure 5 sit at the comfortable end of the range. Table 2 takes the same configuration across a four-QP sweep on 2323 clips, illustrating how generative streaming maps onto the operating range. At comfortable bitrates (QP 3232), the transport frees bits on 1919 of 2323 sequences at a median −6.6 %-6.6\,\%, while the baseline retains near-transparent background fidelity. In the target bit-starved regime (QP 4747), where standard codecs are severely quality-limited, PRESLEY’s generative restoration delivers a superior background on 1717 of 2323 sequences (median LPIPS gap −0.0153-0.0153), while holding foreground fidelity strictly within the perceptual margin. This confirms that generative layers provide their greatest perceptual payoff precisely where conventional codecs starve. QP-matching therefore cannot tell the whole story. At matched QP, PRESLEY spends 23.1 %23.1\,\% and 14.9 %14.9\,\% more bits than ELVIS at QP 32 and 37, because it transmits every block where ELVIS blanks a quarter of them; hence, the fundamental comparison between the two generative paradigms is evaluated at matched rate on delivered quality. There, Figure 5 demonstrates that PRESLEY comprehensively outperforms ELVIS on delivered background quality across all 1313 rate ladders, achieving a decisive mean BD-rate reduction of −56.4 %-56.4\,\% (p=0.000 244p=$0.000\,244$, strictly clearing family-wise Holm correction across all 204204 project configurations). By preserving a degraded visual prior and restoring it with conditioned super-resolution rather than discarding content and relying on unconditioned in-painting, PRESLEY recovers high visual fidelity at a fraction of the rate required by prior generative transports. Furthermore, foreground quality shifts in both directions across ladders without systematic penalty, confirming that background gains carry no hidden foreground cost. Operating point stability A generative transport might only pay off in a favourable corner of the rate range. With the ladder as the unit and the median over the two lowest-quality QPs minus the median over the two highest as the statistic, the contrast is negative on 55 of 1313 at an exact two-tailed p=0.58p=$0.58$ — which alone establishes nothing, since a difference test that fails to reject supports no conclusion about the size of the effect. We therefore tested the proposition directly, as an equivalence: the mean contrast is demonstrably smaller than the reporting margin, with a 90 %90\,\% bootstrap interval of [−0.017,0.013][$-0.017$,$0.013$] background LPIPS, four times tighter than the 0.050.05 we report against. Equivalence of the mean is not equivalence of every ladder: two of thirteen shift by more (camel at −0.0585-0.0585, a held-out YouTube-VOS clip at −0.0535-0.0535), both with PRESLEY gaining as the QP starves. Restoration holds across the operating range; reduction is regime- and content-dependent. Appendix E bounds this further with a detector-derived mask, and with 12801280×720720 and 19201920×10801080 ladders. Comparison with standard codec ROI and neural codecs Standard codec tools offer partial solutions. As detailed in Appendix G, Kvazaar’s native ROI encoding relocates bits to improve foreground quality by a median +0.51+$0.51$ dB PSNR across 1616 of 1717 points at unchanged bitrate, but cannot restore the sacrificed background (−0.54-0.54 dB PSNR loss on 8 of 8 points). SVT-AV1’s ROI shows no measurable effect. That relocation is not free: the encoder buys foreground fidelity by spending less elsewhere, and an x265 adaptive-quantization control is indistinguishable from its baseline. PRESLEY instead pairs targeted degradation with generative client-side reconstruction. The stages cannot be stacked, for a structural reason rather than a sampling gap — the only encoder exposing a usable ROI facility is not the one the hole-filling results run through — and compared separately they do different things rather than more of one thing: codec ROI is the only stage that improves the foreground, while the degrade-and-restore stages trade background quality for bits. Appendix D sets the four stages side by side. Finally, evaluating the neural codec baseline HNeRV (Appendix D) shows that once serialized model weights are accounted for in the transmitted payload, it requires 22.7 to 178.122.7178.1× more bitrate than x265 at matched foreground quality, confirming that standard hybrid codecs combined with lightweight generative layers remain far more practical. 4.3. Restoration and Regime Results Table 2 measured reduction under x265. We repeated it under SVT-AV1 across the operating range, holding the transport fixed (block size 16, ProPainter, fg_protect) and moving only the QP, on four videos spanning the corpus’s range of foreground area and camera motion. Figure 6 gives the four rate–distortion curves. Reduction pays where the encoder is starved. On bear and camel the transport frees 25.6 % to 28.9 %25.6\,\%28.9\,\% of bits at equal foreground PSNR, every point-wise delta under 0.5 dB0.5\,dB; at comfortable bitrates the same configuration costs 16.7 %16.7\,\% and 40.8 %40.8\,\% more, so the saving belongs to the starved end of the range rather than to the method in general. On dog and pigs it costs bits instead, and the choice of fill does not rescue them. That two-two split is not the corpus rate: across the 2323 clips of Figure 5 the hole-filling transport saves bits on 1717 and the conditioned one on 1515. These four are the clips we can show as full curves, chosen to include both outcomes. Two independent mechanisms agree, which makes it a content result rather than a component bug: the same reversal appears in the conditioned configuration of Figure 5, on the same two videos, through a different degradation and a different restorer. We looked for a content attribute that predicts the flip and did not find one — foreground area is the obvious candidate and is refuted, dog sitting at 0.1140.114 against bear’s 0.1110.111 and reversing hardest. We report the non-replication rather than the average over it, and regard identifying that attribute as the most useful open problem this article leaves. Figure 6. The starved-regime result as rate–distortion ladders rather than Bjøntegaard scalars. Each panel is one video over four recalibrated fixed QPs on SVT-AV1, the hole-filling configuration being blackout at block size 16. The claim is horizontal: a curve left of the baseline delivers the same foreground quality for fewer bits, and on dog and pigs (bold) it lies right instead.Four rate-distortion panels side by side, one per video: bear, camel, dog and pigs. Each plots foreground PSNR in decibels against bitrate in kilobits per second over four fixed-QP rungs, with a grey line for the pristine SVT-AV1 baseline and a dashed blue line for the bridge arm. A bridge curve lying left of the baseline delivers the same foreground quality for fewer bits. On bear and camel it does; on dog and pigs it lies to the right instead, and the two curves converge at the most starved rung rather than separating. Data: "arms":"baseline":"pristine SVT-AV1","bridge":"blackout fill, block size 16, ProPainter","bd\_rate\_fg\_percent":"bear":-28.9,"camel":-25.6,"dog":27.3,"pigs":18.3,"ladders":"bear":"baseline":[[164.0,24.334],[252.0,25.269],[442.0,26.727],[753.7,28.233]],"bridge":[[113.0,24.33],[175.5,25.332],[327.4,26.783],[584.6,28.31]],"camel":"baseline":[[138.9,24.424],[259.9,26.357],[468.0,28.336],[740.0,29.911]],"bridge":[[93.1,24.305],[178.3,26.253],[348.0,28.187],[600.6,29.791]],"dog":"baseline":[[61.2,25.853],[122.7,28.009],[219.2,29.578],[378.8,30.608]],"bridge":[[77.3,24.972],[139.5,27.516],[237.9,29.328],[358.1,30.518]],"pigs":"baseline":[[55.0,27.882],[103.4,29.734],[170.0,31.164],[268.9,31.937]],"bridge":[[56.1,26.739],[99.2,29.035],[166.2,30.675],[237.9,31.685]],"quantity":"foreground PSNR (dB) against transmitted bitrate (kbit/s), four fixed-QP rungs per video","reading":"negative BD-rate means fewer bits for equal foreground quality; bear and camel free bits, dog and pigs cost them and do so at the most starved rung, which is the inversion a single BD-rate integral hides" The curves show what a Bjøntegaard scalar cannot: on dog and pigs they converge as the QP gets harsher, so the advantage reverses within the sweep rather than merely being absent. Table 2 identifies the mechanism — the transport exchanges rate for background quality at a rate that itself changes with the QP, so a scalar integrated across that crossover sums a cheaper-and-poorer region with a dearer-and-better one and reports them as one effect. Two studies of the same configuration can therefore disagree purely on the range they integrate over. Figure 7. Every restorer we ran, on the transport it is dispatched on, each against its own exactly matched unrestored control. Left: gain over that control. Right: the absolute background LPIPS the viewer receives. The panels disagree, because gain is bounded by how much each transport destroyed.Two dot plots sharing one axis of restoration backbone and transport pairings, grouped into conditioned restorers that consume the transmitted pixels and in-painters that discard them. Left: each dot is one run minus its exactly matched unrestored control on background LPIPS, so right of zero means the restorer improved the background it was handed; the largest median gain is e2fgvi on blackout at 0.087. Right: the absolute background LPIPS of the restored output, where lower is closer to the original and realesrgan on downsample is best at 0.216. The two panels disagree, and that is the point: the transports that destroy most show the largest recovery while still delivering a worse picture, so gain over a control cannot be read as restoration quality. Data: "arms":["family":"conditioned (keeps the prior)","max\_gain":0.1833,"median\_absolute\_bg\_lpips":0.2158,"median\_gain\_over\_control":0.0335,"min\_gain":-0.0544,"n\_pairs":76,"restorer":"realesrgan","transport":"downsample","family":"conditioned (keeps the prior)","max\_gain":0.0209,"median\_absolute\_bg\_lpips":0.243,"median\_gain\_over\_control":0.0202,"min\_gain":0.0195,"n\_pairs":2,"restorer":"unsharp","transport":"downsample","family":"conditioned (keeps the prior)","max\_gain":0.0094,"median\_absolute\_bg\_lpips":0.2615,"median\_gain\_over\_control":0.0018,"min\_gain":-0.1541,"n\_pairs":32,"restorer":"nafnet","transport":"ac\_truncate","family":"conditioned (keeps the prior)","max\_gain":0.0169,"median\_absolute\_bg\_lpips":0.2951,"median\_gain\_over\_control":0.0043,"min\_gain":-0.1099,"n\_pairs":26,"restorer":"nafnet","transport":"blur","family":"conditioned (keeps the prior)","max\_gain":-0.0169,"median\_absolute\_bg\_lpips":0.3558,"median\_gain\_over\_control":-0.0195,"min\_gain":-0.0221,"n\_pairs":2,"restorer":"instantir","transport":"blur","family":"conditioned (keeps the prior)","max\_gain":0.0175,"median\_absolute\_bg\_lpips":0.4355,"median\_gain\_over\_control":0.0156,"min\_gain":0.0073,"n\_pairs":10,"restorer":"unsharp","transport":"blur","family":"in-painting (discards the prior)","max\_gain":0.0894,"median\_absolute\_bg\_lpips":0.3077,"median\_gain\_over\_control":-0.0144,"min\_gain":-0.0533,"n\_pairs":12,"restorer":"propainter","transport":"freeze","family":"in-painting (discards the prior)","max\_gain":0.0108,"median\_absolute\_bg\_lpips":0.3159,"median\_gain\_over\_control":-0.0377,"min\_gain":-0.0658,"n\_pairs":8,"restorer":"e2fgvi","transport":"freeze","family":"in-painting (discards the prior)","max\_gain":0.1613,"median\_absolute\_bg\_lpips":0.3483,"median\_gain\_over\_control":0.087,"min\_gain":0.0623,"n\_pairs":8,"restorer":"e2fgvi","transport":"blackout","family":"in-painting (discards the prior)","max\_gain":0.1483,"median\_absolute\_bg\_lpips":0.3611,"median\_gain\_over\_control":0.0821,"min\_gain":0.0539,"n\_pairs":8,"restorer":"propainter","transport":"blackout","family":"in-painting (discards the prior)","max\_gain":0.0989,"median\_absolute\_bg\_lpips":0.4289,"median\_gain\_over\_control":0.0087,"min\_gain":-0.0041,"n\_pairs":8,"restorer":"telea","transport":"blackout","family":"in-painting (discards the prior)","max\_gain":-0.0518,"median\_absolute\_bg\_lpips":0.43,"median\_gain\_over\_control":-0.1229,"min\_gain":-0.1573,"n\_pairs":6,"restorer":"telea","transport":"freeze"],"note":"the gain panel is the MECHANISM and the absolute panel is what the viewer receives; neither is restored-vs-original against a pristine baseline at matched rate, which is the Goal-2 headline and a harder question","pairing":"exact: same video, transport, block size, budget, codec and QP; runs without an exact `none` control are dropped, never compared against an approximate one","quantity":"BG-LPIPS of the matched unrestored control minus BG-LPIPS of the restored run; positive means the restorer helped","replaces\_tables":["tab:inpainters","tab:conditioned","tab:conditioned-twins","tab:conditioned-stream-diffvsr","tab:instantir-kill","tab:goal2","tab:goal2-breadth"],"trap":"ranking by gain alone ranks the transports that damage most as the ones that restore best: blackout in-painting shows the largest gains while reaching worse absolute quality than downsample with Real-ESRGAN" Does client-side restoration bring the degraded background back toward the original? Figure 7 answers it for every backbone we ran, each against its own exactly matched unrestored control, differing only in whether a restorer ran. The two panels disagree. Ranked by gain over its control, hole-filling wins; on the absolute quality a viewer receives the ordering inverts. Each configuration’s gain is bounded by how much its own transport destroyed, so a large gain only reports an ambitious transport, and the choice between families is settled on the absolute panel: the transport that keeps a prior wins on delivered quality even though it leaves its restorer less to do. Figure 8. Spatial selection, transport, and restoration visual comparison. Left: Selected degradable blocks shaded in red, protected foreground outlined in blue, with the yellow bounding box indicating the evaluated crop. Right: Visual patch comparison across pipeline configurations (Pristine baseline under standard lossy codec, ELVIS blackout with ProPainter inpainting, and PRESLEY adaptive downsampling with Real-ESRGAN super-resolution), showing the transmitted representation on the wire (top row) and the restored output delivered to the client (bottom row).Visual comparison of degradation and restoration pipelines on a single representative frame. The left panel shows the full video frame with degradable blocks selected by the removability score shaded in red, the protected foreground object outlined in blue, and the yellow bounding box indicating the evaluated background crop. The right panel compares the crop across four configurations (Reference uncompressed source, Pristine baseline under standard lossy codec, ELVIS blackout with ProPainter inpainting, and PRESLEY adaptive downsampling with Real-ESRGAN super-resolution) across both the transmitted wire representation (top row) and the restored output delivered to the client (bottom row). Figure 8 illustrates the full pipeline on a representative frame: the selector places degradation exclusively on the background while preserving foreground integrity, and the comparative crops show what each configuration puts on the wire (top) versus what its restorer recovers (bottom). While ELVIS transmits empty blocks and in-paints them from neighbouring frames, PRESLEY transmits a low-resolution prior that its restorer sharpens. The restored crop is sharp again, though fine high-frequency texture is plausibly synthesized rather than recovered bit-exact. The mechanism is architectural: models dispatched on hole transports are in-painters (e.g., ProPainter’s inference masks out the transmitted prior by construction), whereas conditioned restorers consume those transmitted pixels directly, separating the two families. Freeze proves it from the other side: an in-painter handed a frozen block discards a prior already close to correct, and every freeze pairing sits at or below zero gain. To check the ordering is not an artifact of the mask, we repeated the comparison on whole-frame metrics, which carry no region restriction. Paired within an operating point, the conditioned configuration beats ELVIS on 121121 of 121121 ladder points on whole-frame LPIPS and 3636 of 3636 on VMAF, by 11.411.4 dB of PSNR and 25.525.5 VMAF points at the median. On PSNR and LPIPS it beats blackout in-painting on 1212 of 1212 shared operating points (p=0.0005p=$0.0005$) and freeze in-painting on 1313 of 1414, so it leads every hole-filling configuration. Every measure Section 2 surveys ranks the configurations as the masked metrics do; Appendix F gives all seven. No backbone separates from Real-ESRGAN. The right-hand panel of Figure 7 places every conditioned alternative — BSRGAN, Real-HAT-GAN Chen et al., 2023 and Stream-DiffVSR Shiu et al., 2025 — within a band narrower than the incumbent’s perceptual margin, on the same transport and operating point, with Stream-DiffVSR costing roughly twice as much to run. At two paired videos none can reach significance under our own rule, so we report them as a catalogue that failed to separate rather than as demonstrated ties: the evidence supports keeping Real-ESRGAN on throughput, not a claim that it is best. On the blur transport InstantIR does not beat a trivial unsharp control costing ≈12≈12 s against ≈774≈774–837837 s, and neither it nor NAFNet delivers meaningful gain against the transmitted video, so that transport’s restorer question is open. Both need float32 — NAFNet overflows LayerNorm and SCA in fp16, Real-HAT-GAN’s Softmax overflows to NaN — so half precision is not safe to assume. Finally, the part that separates the two goals: dog and pigs produce the largest restoration gains anywhere in this article while being exactly the videos on which reduction fails outright. Restoration quality and reduction are not one axis. Table 3. The removal budget below and above its threshold, on the two videos where it was swept: SVT-AV1 at fixed QP, 640640×360360, block size 8, downsampling with Real-ESRGAN. Below a quarter of the blocks the budget buys almost no rate while background quality falls steadily, so 0.250.25 is the smallest budget that pays for the damage it does. Removal budget 0.100.10 0.250.25 0.500.50 0.750.75 bear bitrate (kbit/s) 607.0607.0 614.2614.2 451.4451.4 415.4415.4 bear background LPIPS 0.1410.141 0.1930.193 0.2460.246 0.2910.291 camel bitrate (kbit/s) 694.9694.9 693.3693.3 570.5570.5 531.9531.9 camel background LPIPS 0.1100.110 0.1230.123 0.1680.168 0.1950.195 Two further properties are worth recording. For the winning conditioned pipeline over four starved fixed-QP points per video, reduction is a curve-level win at 13.8 % to 16.1 %13.8\,\%16.1\,\% fewer bits for equal foreground PSNR. And the removal budget has a threshold below which it does nothing: swept at the lowest-quality QP it moves bitrate by under 1.2 %1.2\,\% between 0.100.10 and 0.250.25, then by −22.1 %-22.1\,\% between 0.250.25 and 0.500.50, while background damage rises steadily throughout (Table 3). Degrading fewer than a quarter of the blocks therefore costs quality without buying rate, which is why 0.250.25 is the budget used throughout. 4.4. Conditioned Restoration Wins on Quality; Bitrate and Cost Order the Arms Differently The comparisons so far run within a family. Ranked that way Real-ESRGAN carries the conditioned path, ProPainter the hole-filling path and NAFNet the blur path, none separating from its nearest rival by more than the perceptual margin. Table 9 sets those family winners against each other, which is the comparison a deployer faces — and it faces three axes at once, delivered background quality, transmitted bits, and the cost of restoring, which order the configurations differently, so none is simultaneously best. The first is this article’s clearest configuration-level result: degrading by downsampling and restoring with a super-resolver conditioned on what was actually transmitted delivers the best background quality of any configuration we tested. Table 9 reports every configuration comparable to it at our significance floor. Candidate Objective n Base wins pHolmp_Holm Sig.? blur+nafnet quality 11 11/11 0.014 yes blackout+propainter quality 10 10/10 0.025 yes freeze+propainter quality 10 10/10 0.025 yes ac_truncate+nafnet bitrate 10 10/10 0.027 yes ac_truncate+nafnet quality 10 9/10 0.236 no blackout+propainter bitrate 10 1/10 0.279 no freeze+propainter bitrate 10 9/10 0.279 no blur+nafnet bitrate 11 5/11 1.000 no Table 4. Every configuration comparable to downsample+realesrgan at our significance floor, paired within a fixed QP. “Base wins” counts videos on which downsample+realesrgan is ahead; pHolmp_Holm is corrected within a family of 14 candidates per objective. Quality is background LPIPS, bitrate the actual transmitted rate against the matched pristine baseline. Rows marked no are comparisons that were run and did not separate at this sample size, reported so the comparison is not selected on its outcome; they are not claims of equality. The two objectives disagree. Figure 9. Cost per pipeline stage, three trials per cell on a pinned GPU, cells mixing device populations refused rather than averaged. Restoration dominates every configuration that has one; block selection costs about a thousandth of the pipeline.Stacked horizontal bars, one per pipeline arm, showing mean seconds per stage over three trials on a pinned GPU. Restoration is the dominant stage in every arm that has one, taking between 70 and 89 percent of wall clock; ProPainter costs about three times Real-ESRGAN. Degradation costs more than encoding, so the server-side bottleneck is the per-block loop rather than the codec. A log-scale panel shows the server-side stages, where block selection costs about one hundredth of a second – roughly a thousandth of the pipeline, and the axis on which the article locates its remaining headroom. Data: "caveat":"Real-ESRGAN here runs full-frame (restorer\_params ), which is faster than the tiled tile=400 configuration the published corpus uses; these timings are not comparable to timings quoted for that configuration","device":["cuda:0"],"mean\_seconds\_by\_arm":"baseline (SVT-AV1)":"encode":0.771,"preprocess":0.397,"elvis blackout + ProPainter":"composite":0.289,"decode":0.218,"degrade":2.507,"encode":1.078,"preprocess":0.352,"restore":35.421,"score":0.001,"sidechannel":0.003,"presley downsample + Real-ESRGAN":"composite":0.24,"decode":0.199,"degrade":3.018,"encode":1.152,"preprocess":0.38,"restore":12.092,"score":0.002,"select":0.006,"sidechannel":0.005,"presley freeze + ProPainter":"composite":0.21,"decode":0.203,"degrade":2.382,"encode":1.169,"preprocess":0.356,"restore":34.021,"score":0.002,"select":0.005,"sidechannel":0.01,"method":"components called directly into scratch dirs; nothing written under results/; device recorded per trial; cells mixing device populations refused rather than averaged","restoration\_share\_pct":"baseline (SVT-AV1)":0.0,"elvis blackout + ProPainter":89.2,"presley downsample + Real-ESRGAN":69.4,"presley freeze + ProPainter":88.0,"stage\_order":["preprocess","score","select","degrade","encode","sidechannel","decode","restore","composite"],"trials\_per\_cell":[3],"units":"seconds, mean of 3 trials, 640x360, 60-90 frames","videos":["bear","camel","dog"],"wall\_seconds\_by\_arm":"baseline (SVT-AV1)":1.17,"elvis blackout + ProPainter":39.69,"presley downsample + Real-ESRGAN":17.42,"presley freeze + ProPainter":38.64 On background quality the ordering is settled: the conditioned configuration beats all three in-painting and deblurring configurations on every video tested. On bitrate it inverts — ac_truncate+nafnet loses the rate axis on all ten videos (p=0.0020p=$0.0020$ uncorrected), our one significant bitrate comparison, and it is the configuration the baseline beats on quality. Winning quality and winning rate are not the same configuration. Figure 9 measures execution time per stage on three 640640×360360 clips (60 to 906090 frames), three trials per cell on a pinned GPU. Restoration dominates by enough to locate the engineering problem precisely: the conditioned configuration runs 5.45.4× slower than real time against 12.012.0× for the in-painting ones, so the configuration that wins on quality is also the closest to real time, and closing that gap is a restorer-side problem. Everything the server does is comfortably real time — degradation costs more than encoding, so the bottleneck there is our own per-block loop rather than the codec, and block selection is under 0.1 %0.1\,\% of the pipeline. These are 360360p figures and the factor scales with pixel count, so we scope the system to offline transcoding and edge-cache pre-population until a restorer closes the remaining 5.45.4×. 4.5. Block Selection and Headroom Analysis We analyze the block selection mechanism along three dimensions: the efficiency of the cost model, parameter sensitivity, and predictive modeling of post-restoration damage. Placement and foreground exclusion Block selection evaluates two distinct mechanisms: ranking blocks by removability score versus random placement, and hard foreground exclusion versus soft priority weighting. Video Score ranking Random map bear +80.9%+80.9\% +154.4%+154.4\% camel +27.5%+27.5\% +121.6%+121.6\% dogs-jump −1.6%-1.6\% +79.1%+79.1\% pigs +23.6%+23.6\% +75.7%+75.7\% dog +13.2%+13.2\% +27.2%+27.2\% bike-packing +219.6%+219.6\% +194.9%+194.9\% Table 5. Removability score ranking vs. seeded random block selection at matched budget, strength, clustering, and foreground exclusion. Values are BD-rates on background LPIPS against each video’s pristine baseline (lower is better). The score wins on 5 of 6 sequences by a median of 52.152.1 percentage points. Histogram of BD-rate on background LPIPS comparing graded multi-level degradation against uniform single-level degradation across 16 sequences at 1080p, showing graded increases bitrate on 14 of 16 sequences by a median of 5.1 percent. Figure 10. Graded multi-level degradation vs. uniform binary degradation per sequence at b=64b=$64$ and 19201920×10801080. Positive BD-rate indicates graded requires more bits for equal background quality (losing on 14 of 16 sequences, p=0.0042p=$0.0042$).Histogram of BD-rate on background LPIPS comparing graded multi-level degradation against uniform single-level degradation across 16 sequences at 1080p, showing graded increases bitrate on 14 of 16 sequences by a median of 5.1 percent. As shown in Table 10, ranking blocks by the removability score outperforms a smoothed random selection baseline on 55 of 66 sequences, providing a median BD-rate improvement of 52.152.1 percentage points on background LPIPS. The exclusion is worth about half a decibel. On six sequences chosen because a purely score-based selection reaches their foreground at all, dropping it costs foreground PSNR on 66 of 66, a median 0.560.56 dB at p=0.031p=$0.031$, while saving 6.0 %6.0\,\% of bitrate; on foreground LPIPS it moves 0.00130.0013 and clears no margin anywhere. That is a saving this article declines to take, and the exclusion is a default rather than a property of the transport — the soft priority alone already confines damage to the least important foreground blocks. Two bounds: the six sequences were chosen because selection reaches their foreground, so 6 %6\,\% is what the exclusion costs where it binds, not across the corpus; and at six sequences the exact test floors at 0.0310.031, the smallest suite that can be significant at all. What it buys instead is attribution — with the foreground untouched by the generative layer, any foreground difference between configurations is the codec’s doing, which is what lets the backbone catalogue above be read as a comparison of backgrounds. Cost axis headroom bound Figure 11. The two axes of the selection objective. Left: removability score vs. an exact leave-one-superblock-out bit oracle (hatched area indicates remaining headroom). Right: post-restoration damage vs. spatial complexity, standardized per run.Horizontal stacked bars, one per probe video, showing what fraction of an exact bit oracle's recoverable bits the complexity score captures when the top quarter of superblocks is degraded. Each bar starts with the grey fraction that random selection already captures, then adds the score's contribution, then leaves the hatched remainder open. The mean capture is 0.833 against a random null of 0.402, so the score adds about four tenths over chance and the cost axis has little headroom left. On drift-straight the score is near chance and is drawn in the contrasting colour rather than averaged away. Data: "capture":[0.51,0.922,0.94,0.765,0.909,0.881,0.824,0.917],"coding":"inter, SVT-AV1 fixed QP 43, 64x64 superblocks, n=8 probe videos","gain\_over\_null":[0.074,0.389,0.394,0.412,0.493,0.499,0.526,0.662],"headroom\_reading":"the oracle's top quarter is 30.1\% of total bitrate, so the entire remaining cost-side headroom is worth about 5\% of the bitrate","mean\_capture":0.833,"mean\_random\_null":0.402,"near\_chance\_videos":["drift-straight"],"oracle\_top25\_share\_of\_total\_bits":0.301,"quantity":"capture ratio: fraction of an exact leave-one-superblock-out oracle's freed bits obtained by degrading the top 25\% by score","random\_null":[0.436,0.533,0.546,0.353,0.416,0.382,0.298,0.255],"videos":["drift-straight","dancing","color-run","bear","drift-turn","bike-packing","motorbike","dogs-jump"],"why\_the\_null\_matters":"a capture ratio without its null reads as '83\% of the way to the oracle', which it is not: random selection already captures 0.402"Two panels. Left: a scatter of per-superblock spatial complexity against post-restoration damage, both standardized within each run, with a fitted line sloping upward – more complex blocks come back worse. Right: a histogram of the per-run rank correlations, all 120 of 120 of them positive, with a median of +0.506. The selection score already ranks blocks by complexity because complex blocks cost the most bits, so the objective preferentially degrades the blocks that survive restoration worst. Data: "claim":"post-restoration damage is predictable at transmit time, and correlates POSITIVELY with the complexity the selection score already ranks by","control":"restricted to superblocks that were actually degraded; over all superblocks the correlation is inflated to +0.684 because complexity predicts whether a block was selected at all","feature":"sc\_mean (EVCA spatial complexity, 64x64 superblocks)","n\_runs":120,"restorers":["realesrgan","propainter"],"rho\_max":0.8152,"rho\_median":0.5057,"rho\_min":0.0424,"runs\_positive":120,"unit":"the run; statistic is a within-run Spearman rho" The removability score estimates the bitrate contribution of individual blocks. To quantify how closely this heuristic approximates optimal bit allocation, the left panel of Figure 11 evaluates the score against an exact leave-one-superblock-out combinatorial oracle across eight probe sequences. Selecting the top quarter of blocks by removability score captures a mean of 83.383.3% of the bits identified by the exact oracle. Because the oracle’s top quarter accounts for 30.1 %30.1\,\% of total frame bits, the remaining headroom on the bit-cost axis is at most ≈5 %≈$5\,\%$ of total bitrate, measured against the random-selection null the figure draws as the base of every bar. This bound evaluates superblocks independently under the leave-one-out probe, serving as an additive empirical upper bound on cost-model efficiency given inter-block prediction dependencies in hybrid video codecs. A better cost model is therefore not where major gains lie. One exception: on drift-straight the score barely clears that clip’s own null, with a rank correlation whose bootstrap interval contains zero. Parameter sensitivity and degradation levels Table 6. Parameter sensitivity: median within-group spread, a group fixing every other setting and varying only the named parameter. The removal budget is the only lever on either axis. Descriptive only — the number of videos is below the threshold this article requires for a significance claim. α and β are absent because they were measured on foreground PSNR, and putting the two quantities on one scale would compare unlike things. Parameter Δ (p) Δ -LPIPS Videos Removal budget 26.3826.38 0.15080.1508 22 Block size 9.089.08 0.01510.0151 44 Foreground protection 5.475.47 0.00240.0024 22 Mask noise (dilate/erode) 4.914.91 0.00830.0083 22 Mask source (GT/YOLO) 4.204.20 0.00380.0038 77 Selection rule 1.201.20 0.00310.0031 66 Graded levels 1.071.07 0.01490.0149 88 Graded levels, oracle-assigned 0.560.56 0.00500.0050 88 Table 6 summarizes sensitivity to individual knobs. The removal budget is the primary lever at 26.3826.38 percentage points of bitrate, and everything else moves it by under a third as much; swept over 0,0.25,0.5,0.75,1.0\0,0.25,0.5,0.75,1.0\, α and β move foreground PSNR by 0.03 dB to 0.05 dB0.03\,dB0.05\,dB across their entire range. Both comparisons are descriptive — the number of videos is below what this article requires for a significance claim — but the magnitudes are not close. That inertness is a deployment property before it is a diagnosis: these are the parameters an operator would otherwise sweep per title, codec and operating point, and a system stable across their whole range ships with defaults instead (α=β=0.5α=β=0.5 throughout). The cause is structural — Equations 1–4 blend terms that all estimate how many bits a block costs, so they trade proxies for the same quantity. Grading the degradation strength per block does not help either. Letting the score set strength as well as footprint costs bits on 77 of 88 videos at b=16b=$16$, where Equation 5 admits only a single level, so we re-ran it where the bound permits three: b=64b=$64$ at 19201920×10801080, sixteen sequences at four fixed QPs, differing in nothing but downsample_levels. A single strength is the better default: graded costs a median 5.1 %5.1\,\% of BD-rate on background LPIPS, losing on 1414 of 1616 at an exact two-tailed p=0.0042p=$0.0042$, with foreground LPIPS differing by at most 0.00260.0026. The encoder pays for transitions between adjacent blocks held at different levels — discontinuities the content never had. The design was sequential and is disclosed as such: pre-registered at n=10n=$10$, returning 88 of 1010 at p=0.109p=$0.109$, then extended to a target of n=16n=$16$ fixed before the extension ran, the six added sequences going 66 of 66 the same way. What we retire is the graded default, not the mechanism: tennis (−11.1%$-11.1$\%) and motorbike (−4.2%$-4.2$\%) go the other way, and which content that is is the same unidentified property that decides the sign of reduction in Section 4.3. camel is excluded on a stability criterion rather than an outcome — its graded arm drives more than a percent of output pixels to the clipping rails at every QP — and that exclusion is itself a property of the graded transport (Appendix D). Predicting post-restoration restorability An ideal rate-distortion-restoration rule would balance bits saved against post-restoration distortion, and the binding constraint is whether that second quantity is estimable before transmission. The right panel of Figure 11 answers it: across 120120 restored runs on 1313 sequences, ranking superblocks by spatial complexity ranks their eventual damage at a median Spearman ρ=+0.506ρ=+$0.506$, in the same direction on 120120 of 120120. The score already selects on spatial complexity because complex blocks cost the most bits, so degrading them is the design working as intended. What the correlation adds is narrower: within a fixed budget the ranking always spends it on the blocks that restore worst. Over all superblocks the correlation is 0.6840.684, because complexity predicts whether a block was touched (0.7860.786) before anything else; restricting to degraded blocks returns 0.5060.506, and that restriction was declared after the fact (Appendix H). We fit a damage predictor on five pre-registered transmit-time features, held out by video — never by run, since runs of one clip share content and would leak — reaching a held-out ρ=+0.400ρ=+$0.400$ on all 1313 folds. Restorability is therefore predictable at transmit time, with no client-side feedback. To evaluate whether this feasibility translates directly into selection gains, we tested a first-order rate-distortion-restoration rule replacing the ranking with the ratio the diagnosis implies: bits freed over predicted damage. This changes about 9 %9\,\% of selected blocks and starts 3.6 %3.6\,\% down on rate; across six videos over four QPs, it performs level with the heuristic incumbent (median +5.1%+$5.1$\% BD-rate, better on 22 of 66, p=0.69p=$0.69$). This confirms the feasibility of transmit-time restorability modeling, establishes a baseline pricing for linear scoring, and indicates that realizing the remaining headroom requires joint non-linear optimization between rate cost and restorer distortion. 4.6. Spatial Selectivity vs. Uniform Downscaling Selective quality reduction with client-side reconstruction is not new: whole-frame downscaling with super-resolution at the client is deployed at scale. The architectural difference is that degradation here is per block under a mask excluding the foreground, giving the transport something uniform downscaling cannot express — a region it will not touch. The uniform configuration is the identical code path with the budget opened to every block and protection off, so the per-block operator is pixel-identical; what differs is where degradation is placed and how much of the frame it covers, the uniform arm degrading roughly four times the area at a budget of 0.250.25. Selectivity is ahead on two sequences and behind on four, at p=0.69p=$0.69$ on the foreground and p=0.22p=$0.22$ on the background, so we claim neither direction; Appendix D gives the per-sequence figures. On the four it loses it costs 60 % to 90 %60\,\%90\,\% more bits for equal perceptual quality, magnitudes the coverage difference predicts. The comparison therefore establishes a lower bound on selectivity’s standing: it wins twice while giving away four times the degraded area. Two limits bound it in opposite directions. The uniform arm sends full-resolution but low-pass-filtered frames rather than a genuine lower-resolution rendition with fewer coded pixels; a deployed ladder would likely do better, so this is a conservative proxy for industry practice rather than a defeat of it. And the two sequences selectivity wins (bear and india) are exactly the two where uniform downscaling damages the foreground perceptibly (+0.087+$0.087$ and +0.101+$0.101$ foreground LPIPS) — offered as a conjecture, concordant on 66 of 66 but generated by these data and never pre-registered. Selectivity therefore behaves like insurance, costing rate where the foreground was never at risk and preventing a visible failure where it was. What the split does settle is negative: foreground area does not predict which side a sequence falls on — india (0.3250.325) wins and pigs (0.3270.327) loses, bear (0.1140.114) wins and dog (0.0900.090) loses — so the deciding quantity is how much perceptible damage a region takes and keeps. 5. Conclusions Building on ELVIS, PRESLEY applies adaptive in-place degradation per block under a removability mask, signals per-block strength in a bit-packed side channel, and restores with a generative model conditioned on transmitted visual priors rather than unconditioned in-painting. Of the degradation modalities we priced, progressive downsampling survives; QP mapping and noise injection are retired with the empirical measurements that rule them out. Read against the three goals, the evaluation demonstrates clear, definitive advancements alongside precise domain boundaries. On restoration, conditioned super-resolution delivers the best background quality of any configuration we tested, beating hole-filling in-painting on every video, with foreground fidelity held bit-exact by passthrough compositing. On reduction, PRESLEY surpasses its predecessor at matched rate on 1313 of 1313 ladders, achieving a mean −56.4 %-56.4\,\% BD-rate and demonstrating substantial coding gains against standard codecs in the target bit-starved regime (1717 of 2323 clips). On selection, our oracle analysis bounds the cost axis at ≈5 %≈$5\,\%$ remaining headroom and establishes transmit-time damage predictability as the definitive frontier for future generative streaming architectures. While a first-order linear ratio performs level with the incumbent, it proves the feasibility of the concept and defines joint non-linear optimization as the next step. Two limitations bound all of this. Restoration is far from real time (6.3 to 61.56.361.5× slower than the source) and the cheapest optimisation is already taken, the restorer running in half precision by default, so we scope the system to offline transcoding and edge-cache pre-population. And every perceptual claim rests on learned metrics rather than human raters: no subjective study was conducted. Three directions follow from that map. The first is a restorer built for this transport rather than adapted to it: every backbone we ran was trained for whole-frame super-resolution or in-painting, then handed a frame degraded per block, at a strength the side channel already carries, in rounds. A model consuming that strength map natively — restoring block by block, once per level, conditioned on how far each block was taken and on its undegraded neighbours — would use information the pipeline already transmits and no current backbone reads. The second is the content property deciding whether reduction pays, since foreground area is refuted and identifying it would do more here than another codec or backbone. The third is a selection rule spending its budget on the restorability term we measure, which is predictable at transmit time and sits at the cheapest point in the pipeline to change. Two smaller openings: the protection mask is a segmenter’s notion of what matters, where a viewer-specific one would fit the same interface and address the personalization barrier that has held back region-of-interest coding; and a subjective study is the natural way to confirm that what our metrics reward is what a viewer sees. Acknowledgements. The financial support of the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, and the Christian Doppler Research Association, is gratefully acknowledged. Christian Doppler Laboratory ATHENA: https://athena.itec.aau.at/ References Amirpour et al. (2024) H. Amirpour, M. Ghasempour, L. Qu, W. Hamidouche, and C. Timmerer EVCA: Enhanced Video Complexity Analyzer. In Proceedings of the 15th ACM Multimedia Systems Conference, MMSys ’24, New York, NY, USA. Cited by: §3.1. Artioli et al. (2025a) E. Artioli, D. Lorenzi, S. Vats, F. Tashtarian, and C. Timmerer GenStream: semantic streaming framework for generative reconstruction of human-centric media. In Proceedings of the 33rd ACM International Conference on Multimedia, M ’25, New York, NY, USA, p. 12276–12284. External Links: ISBN 9798400720352, Link, Document Cited by: §2.1. Artioli et al. (2024) E. Artioli, F. Tashtarian, and C. Timmerer DIGITWISE: digital twin-based modeling of adaptive video streaming engagement. In Proceedings of the 15th ACM Multimedia Systems Conference, p. 78–88. Cited by: §2.1. Artioli et al. (2025b) E. Artioli, F. Tashtarian, and C. Timmerer End-to-end learning-based video streaming enhancement pipeline: a generative ai approach. In Proceedings of the 35th Workshop on Network and Operating System Support for Digital Audio and Video, NOSSDAV ’25, New York, NY, USA, p. 50–56. External Links: ISBN 9798400714696, Link, Document Cited by: 1st item, §1, §3, Abstract., footnote. Bao et al. (2019) W. Bao, W. Lai, C. Ma, X. Zhang, Z. Gao, and M. Yang Depth-aware video frame interpolation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 3703–3712. Cited by: §2.2. Bross et al. (2021) B. Bross, Y. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, and J. Ohm Overview of the versatile video coding (vvc) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology 31 (10), p. 3736–3764. External Links: Document Cited by: §1, §2.1. Cao et al. (2021) J. Cao, Y. Li, K. Zhang, and L. Van Gool Video super-resolution transformer. arXiv preprint arXiv:2106.06847. Cited by: §2.2. Chen et al. (2022) L. Chen, X. Chu, X. Zhang, and J. Sun Simple baselines for image restoration. In European Conference on Computer Vision (ECCV), p. 17–33. Cited by: §4.1. Chen et al. (2023) X. Chen, X. Wang, J. Zhang, Y. Shan, H. Chen, S. Zhou, S. Liu, and C. Dong Activating more pixels in image super-resolution transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 22367–22377. Cited by: §4.1, §4.3. Developers (2025) F. Developers Addroi filter — ffmpeg filters documentation. FFmpeg Project. Note: Accessed: 2025-11-13 External Links: Link Cited by: §2.1. Ding et al. (2023) H. Ding, C. Liu, S. He, X. Jiang, P. H. S. Torr, and S. Bai MOSE: a new dataset for video object segmentation in complex scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 20224–20234. Cited by: §4.1. Ding et al. (2022) K. Ding, K. Ma, S. Wang, and E. P. Simoncelli Image quality assessment: unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (5), p. 2567–2581. External Links: Document Cited by: §2.3. Dong (2018) W. DongWhat is opencv’s INTER_AREA actually doing?(Website) Note: Accessed: 2025-11-14 External Links: Link Cited by: §3.1. Duan et al. (2025) X. Duan, Y. Jin, L. Zhang, and F. Wang SemConf: a system for multiparty semantic video conferencing. In Proceedings of the 35th Workshop on Network and Operating System Support for Digital Audio and Video, NOSSDAV ’25, New York, NY, USA, p. 71–77. External Links: ISBN 9798400714696, Link, Document Cited by: §2.1. Eynnzerr (2022) EynnzerrX264: how to correctly use quant_offsets?(Website) Note: Accessed: 2025-11-13 External Links: Link Cited by: §2.1. Fudan CVL (2025) Fudan CVL MOSEv2: complex video object segmentation, version 2. Note: Hugging Face datasethttps://huggingface.co/datasets/FudanCVL/MOSEv2, validation split, accessed 2026-07-23 Cited by: §4.1. García-Lucas et al. (2020) D. García-Lucas, G. Cebrián-Márquez, and P. Cuenca Rate-distortion/complexity analysis of hevc, vvc and av1 video codecs. Multimedia Tools and Applications 79 (39), p. 29621–29638. Cited by: §2.1. Gu et al. (2023) B. Gu, Y. Yu, H. Fan, and L. Zhang Flow-guided diffusion for video inpainting. arXiv preprint arXiv:2311.15368. Cited by: §2.2. Han et al. (2021) J. Han, B. Li, D. Mukherjee, C. Chiang, A. Grange, C. Chen, H. Su, S. Parker, S. Deng, U. Joshi, Y. Chen, Y. Wang, P. Wilkins, Y. Xu, and J. Bankoski A technical overview of av1. Proceedings of the IEEE 109 (9), p. 1435–1462. External Links: Document Cited by: §1, §2.1. Huang et al. (2025) J. Huang, H. Wang, wangqixun, Baixu, H. Ai, P. Xing, and T. Lin InstantIR: blind image restoration with instant generative reference. External Links: Link Cited by: §1, §4.1. Jin et al. (2025) Y. Jin, X. Liu, and J. Liu Generative ai for multimedia communication: recent advances, an information-theoretic framework, and future opportunities. In Proceedings of the 33rd ACM International Conference on Multimedia, M ’25, New York, NY, USA, p. 12237–12246. External Links: ISBN 9798400720352, Link, Document Cited by: §2.1. Kim et al. (2020) S. Y. Kim, J. Oh, and M. Kim Fisr: deep joint frame interpolation and super-resolution with a multi-scale temporal loss. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, p. 11278–11286. Cited by: §2.2. Kounoudes et al. (2006) A. Kounoudes, A. Constantinides, C. P. Loizou, and N. Tsapatsoulis Visual attention based region of interest coding for video-telephony applications. In 5th International Symposium on Communication Systems, Networks and Digital Signal Processing, Cited by: §1. Ku et al. (2019) C. Ku, G. Xiang, F. Qi, W. Yan, Y. Li, and X. Xie Bit allocation based on visual saliency in hevc. In 2019 IEEE Visual Communications and Image Processing (VCIP), Vol. , p. 1–4. External Links: Document Cited by: §2.1. Lambert et al. (2006) P. Lambert, D. De Schrijver, D. Van Deursen, W. De Neve, Y. Dhondt, and R. Van de Walle A real-time content adaptation framework for exploiting roi scalability in h. 264/avc. In International Conference on Advanced Concepts for Intelligent Vision Systems, p. 442–453. Cited by: §2.1. Li et al. (2022) Z. Li, C. Lu, J. Qin, C. Guo, and M. Cheng Towards an end-to-end framework for flow-guided video inpainting. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 17541–17550. External Links: Document Cited by: §1, §4.1. Li et al. (2019) Z. Li, Z. Duanmu, W. Liu, and Z. Wang AVC, hevc, vp9, avs2 or av1? — a comparative study of state-of-the-art video encoders on 4k videos. p. 162–173. External Links: ISBN 978-3-030-27201-2, Document Cited by: §1. Liu et al. (2024) J. Liu, Y. Qu, Q. Yan, X. Zeng, L. Wang, and R. Liao Fréchet video motion distance: a metric for evaluating motion consistency in videos. In First Workshop on Controllable Video Generation @ICML24, External Links: Link Cited by: §2.3. Meddeb et al. (2014) M. Meddeb, M. Cagnazzo, and B. Pesquet-Popescu Region-of-interest-based rate control scheme for high-efficiency video coding. APSIPA Transactions on Signal and Information Processing 3, p. e16. External Links: Document Cited by: §2.1. Minopoulos et al. (2020) G. Minopoulos, V. A. Memos, K. E. Psannis, and Y. Ishibashi Comparison of video codecs performance for real-time transmission. In 2020 2nd International Conference on Computer Communication and the Internet (ICCCI), p. 110–114. Cited by: §1. Moukthika (2025) MoukthikaResizing and rescaling images with opencv(Website) Note: Accessed: 2025-11-14 External Links: Link Cited by: §2.2. Netflix (2016) Netflix VMAF development kit. Note: Accessed: 2024-11-18 External Links: Link Cited by: §2.3. Newson et al. (2014) A. Newson, A. Almansa, M. Fradet, Y. Gousseau, and P. Pérez Video inpainting of complex scenes. SIAM Journal on Imaging Sciences 7 (4), p. 1993–2019. External Links: Document, Link, https://doi.org/10.1137/140954933 Cited by: §2.2. Niklaus et al. (2017) S. Niklaus, L. Mai, and F. Liu Video frame interpolation via adaptive separable convolution. In Proceedings of the IEEE international conference on computer vision, p. 261–270. Cited by: §2.2. NVIDIA Developer (2025) NVIDIA Developer NVIDIA maxine developer platform. NVIDIA Corporation. Note: Accessed: 2025-11-13 External Links: Link Cited by: §2.1. Onuoha et al. (2023) C. Onuoha, J. A. Flaherty, S. Luo, T. T. Huong, and T. C. Thang An evaluation of quality metrics for neural radiance field. In 2023 IEEE 15th International Conference on Computational Intelligence and Communication Networks (CICN), Vol. , p. 619–623. External Links: Document Cited by: §2.3. Patwardhan et al. (2005) K. A. Patwardhan, G. Sapiro, and M. Bertalmio Video inpainting of occluding and occluded objects. In IEEE International Conference on Image Processing 2005, Vol. 2, p. I–69. Cited by: §2.2. Pont-Tuset et al. (2017) J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine-Hornung, and L. Van Gool The 2017 davis challenge on video object segmentation. arXiv:1704.00675. Cited by: §4.1. Shi et al. (2022) Z. Shi, X. Xu, X. Liu, and J. Chen Video frame interpolation transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10474–10483. External Links: Link Cited by: §2.2. Shiu et al. (2025) H. Shiu, C. Lin, Z. Wang, C. Hsiao, P. Yu, Y. Chen, and Y. Liu Stream-DiffVSR: low-latency streamable video super-resolution via auto-regressive diffusion. arXiv preprint arXiv:2512.23709. External Links: Link Cited by: §4.1, §4.3. Su et al. (2023) Y. Su, J. Deng, R. Sun, G. Lin, and Q. Wu A unified transformer framework for group-based segmentation: co-segmentation, co-saliency detection and video salient object detection. IEEE Transactions on Multimedia. Cited by: §3.1. Sullivan et al. (2012) G. J. Sullivan, J. Ohm, W. Han, and T. Wiegand Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on Circuits and Systems for Video Technology 22 (12), p. 1649–1668. External Links: Document Cited by: §1, §2.1. Telea (2004) A. Telea An image inpainting technique based on the fast marching method. Journal of graphics tools 9 (1), p. 23–34. Cited by: §4.1. Vetro et al. (2003) A. Vetro, T. Haga, K. Sumi, and H. Sun Object-based coding for long-term archive of surveillance video. In 2003 International Conference on Multimedia and Expo. ICME ’03. Proceedings (Cat. No.03TH8698), Vol. 2, p. I–417. External Links: Document Cited by: §1. Wang et al. (2021) X. Wang, L. Xie, C. Dong, and Y. Shan Real-esrgan: training real-world blind super-resolution with pure synthetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, p. 1905–1914. Cited by: §1, §4.1. Wang et al. (2018) X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy Esrgan: enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops, p. 0–10–1. Cited by: §2.2. Wedi and Wittmann (2005) T. Wedi and S. Wittmann Quantization offsets for video coding. In 2005 IEEE International Symposium on Circuits and Systems, Vol. , p. 324–327 Vol. 1. External Links: Document Cited by: §2.1. Wiegand et al. (2003) T. Wiegand, G.J. Sullivan, G. Bjontegaard, and A. Luthra Overview of the h.264/avc video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13 (7), p. 560–576. External Links: Document Cited by: §1, §2.1. x265 Developers (2025) x265 Developers X265 – command line options: --qpfile. Multicoreware / x265 Project. Note: Accessed: 2025-11-13 External Links: Link Cited by: §2.1. Xu et al. (2018) N. Xu, L. Yang, Y. Fan, D. Yue, Y. Liang, J. Yang, and T. Huang YouTube-VOS: a large-scale video object segmentation benchmark. arXiv preprint arXiv:1809.03327. Note: https://arxiv.org/abs/1809.03327 Cited by: §4.1. Xue et al. (2016) H. Xue, Y. Zhang, and Y. Wei Fast roi-based hevc coding for surveillance videos. In 2016 19th International Symposium on Wireless Personal Multimedia Communications (WPMC), Vol. , p. 299–304. External Links: Document Cited by: §2.1. Yu et al. (2021) Y. Yu, W. Zhang, and Y. Deng Frechet inception distance (fid) for evaluating gans. China University of Mining Technology Beijing Graduate School 3 (11). Cited by: §2.3. Zeng et al. (2020) Y. Zeng, J. Fu, and H. Chao Learning joint spatial-temporal transformations for video inpainting. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, p. 528–543. Cited by: §2.2. Zhang et al. (2021) K. Zhang, J. Liang, L. Van Gool, and R. Timofte Designing a practical degradation model for deep blind image super-resolution. In IEEE/CVF International Conference on Computer Vision (ICCV), p. 4791–4800. Cited by: §4.1. Zhang et al. (2018) R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, Cited by: §2.3. Zhou et al. (2023) S. Zhou, C. Li, K. C.K. Chan, and C. Change ProPainter: improving propagation and transformer for video inpainting. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 10443–10452. External Links: Document Cited by: §1, §4.1. 6. Appendix Appendix A Evaluation Principles and Methodological Pitfalls We document five key methodological principles and pitfalls established during system evaluation to assist researchers developing and validating generative video streaming pipelines: 1. Region metrics inadvertently evaluating background Evaluating foreground DISTS on a union bounding box (averaging 76 %76\,\% of frame area vs. 15 %15\,\% true foreground) scored the background that the transport intentionally degraded. Correcting to mask-weighted spatial feature pooling makes our own foreground scores worse on 3030 of 3535 sequences, and changes the winner in 3434 matched-rate groups — 1616 toward the degrade-and-restore configurations and 1212 toward the baseline. Principle: Always verify the true spatial coverage of region metrics; bounding boxes do not constitute isolated regions. 2. Variable bitrate obscuring bitrate reductions Under VBR rate control, the encoder expends its target bitrate regardless of source complexity, preventing block degradation from freeing bandwidth and instead causing degradation artifacts to steal bits from the foreground. Across every matched pair in our corpus, VBR degradation costs 16.4 %16.4\,\% more bits than the pristine baseline, saving on 00 of 1818, where fixed QP frees 9.2 %9.2\,\%. Principle: Fixed-QP rate control is mandatory for evaluating degradation transports. A degradation curve measured under a bitrate target measures the rate controller, not the method. We enforce this in code — any VBR degradation run is flagged as uncitable automatically. 3. One-tailed significance testing bias Early analyses quoted one-tailed p-values based on post-hoc observations of empirical directionality. Principle: Two-tailed always, and know the floor — at n=5n=5 no split can reach 0.050.05, so a consistent small suite is underpowered rather than negative. 4. Device population mixing in latency benchmarks A silent CPU fallback produced artificial 10 to 401040× latency spreads by mixing CPU and GPU executions. Principle: Explicitly log hardware device backends per trial and disallow heterogeneous averaging. 5. Inconsistent rate control comparisons Comparing fixed-QP ROI configurations against VBR baselines misattributed baseline overshooting as ROI savings. Principle: Compare configurations only against like-for-like rate control baselines. Appendix B Metrics and Statistical Protocol Spatial mask restriction for deep perceptual metrics For LPIPS and DISTS, per-frame foreground masks are resampled onto the network’s spatial feature maps and utilized as location-wise weights in internal pooling, assigning zero weight to background coordinates. Setting all weights to one identically reproduces standard whole-frame metrics. Because deep convolutional receptive fields span tens of pixels, mask-weighted pooling evaluates foreground objects within their local scene context without introducing boundary cropping artifacts. For the same reason no foreground claim rests on VMAF or FVMD, whose available foreground variants are bounding-box crops. Background PSNR is reported but is never a verdict: the flat mean fill scores the best background PSNR of any fill we tested while being the worst-looking, because a smooth patch is arithmetically closer to the original than invented texture is. Statistical significance protocol No subjective study was conducted: every perceptual claim in this article rests on mask-restricted learned metrics, and none of them substitutes for human raters. We also do not gate claims on a perceptual threshold. We are aware of no source establishing 0.050.05 as a calibrated just-noticeable difference for LPIPS or DISTS, which are in any case on different scales, so wherever a 0.050.05 margin appears in this article it is a declared reporting margin — applied identically to our configurations and to every baseline — and never a perceptual constant. Effect sizes are therefore reported directly, and claims are gated on paired significance instead: an exact two-tailed sign test over videos, Holm-corrected across every candidate this project tried, losers included, with the number of pairs stated. Where a comparison cannot reach significance — at six paired videos the smallest attainable two-tailed p is 0.0310.031, and below six no split reaches 0.050.05 — we report it as underpowered rather than as a null. Appendix C Super-Resolution Restoration Algorithm Algorithm 2 details the progressive super-resolution restoration procedure on the client. Transmitted pixels are re-injected for blocks that have reached their target resolution, preserving un-degraded content and ensuring foreground quality remains independent of the restorer backbone. Algorithm 2 Progressive Super-Resolution Restoration 1: F: current frame at 1/2s1/2s resolution; FinF_in: decoded frame; D∈ℤI×JD ^I× J: downsample factors; s: current scale 2: FupF_up, D: upscaled frame and updated factors 3: Fup←Model(F)F_up (F) ⊳ Conditioned 2×2× SR pass 4: Fref←Resize(Fin,size(Fup))F_ref (F_in,size(F_up)) 5: for i=0i=0 to I−1I-1 do 6: for j=0j=0 to J−1J-1 do 7: if Di,j≤sD_i,j≤ s then 8: Fup[i,j]←Fref[i,j]F_up[i,j]← F_ref[i,j] ⊳ Preserve transmitted pixels 9: else 10: Di,j←sD_i,j← s ⊳ Update scale factor 11: end if 12: end for 13: end for 14: return FupF_up, D Appendix D Architectural Design Principles and Retired Candidates Architectural principles for neural video streaming transports Each of the following candidate modalities was evaluated and retired based on rigorous empirical measurements. Together, they establish foundational architectural design principles for neural video streaming transports, preventing future wasted research effort: (1) Avoid frame shrinking (preserve spatial geometry): Repacking the most removable blocks out of the frame appears to offer a bitrate win by transmitting roughly 25 %25\,\% fewer pixels, but instead costs 42.8 % to 47.7 %42.8\,\%47.7\,\% more bits than in-place blackout at the same budget. Repacking destroys intra- and inter-frame spatial prediction. In-place degradation is essential. (2) Degradations must align with transform coding (avoid noise injection): Adding high-frequency Gaussian noise costs 76.5 % to 83.0 %76.5\,\%83.0\,\% more bits than the pristine baseline at indistinguishable foreground quality, whereas same-budget downsampling and blur free 10 % to 27 %10\,\%27\,\%. Degradation must reduce entropy for the codec, not merely appear smooth to human vision; noise severely disrupts motion compensation. (3) ROI control belongs in the encoder core (avoid pixel-plane QP mapping): Setting per-block QP offsets from removability re-quantizes coefficients on the pixel plane before encoding rather than driving the encoder’s native quantization engine. Combined with GoP averaging, it yields inferior rate-distortion trade-offs compared to native codec ROI (Kvazaar gains +0.42+0.42 dB FG at neutral rate; Appendix G). In-place pixel downsampling is the superior pre-encode transport. (4) Prioritize low-entropy fills for in-painting transports: Foreground PSNR is flat across blackout, freeze, and mean-fill on 66/66 videos due to passthrough compositing, and background LPIPS does not separate them. The choice is settled on rate: flat blackout is cheapest at 566.9 kbps566.9\,kbps against 589.3 589.3\, and 616.1 616.1\,. (5) Hybrid codecs remain essential (neural video representations like HNeRV are non-viable for streaming): Accounting for transmitted serialized model weights reveals that HNeRV requires 22.7 to 178.122.7178.1× higher bitrate than standard x265 at matched foreground quality, confirming that hybrid traditional codecs coupled with lightweight client-side generative layers are the practical path forward. (6) Enforce fixed-QP rate control: Fixed-QP is the prerequisite for bit relocation; under VBR, the rate controller launders bits regardless of degradation. One sequence is excluded from the graded-strength comparison of Section 4.5 on a stability criterion rather than an outcome, and the exclusion is itself evidence about the graded transport. camel’s graded configuration drives more than a percent of its output pixels to the clipping rails at every QP, because a multi-level pyramid applies the restorer once per level and each round re-amplifies what the last pushed toward the rails. It breaches the clipping invariant on all four rungs, against zero breaches across the 4040 binary runs. Progression across pipeline stages Table 8 reports the stage-by-stage progression from native codec ROI to ELVIS block removal and PRESLEY degrade-and-restore. Table 8 presents the per-sequence performance comparison between spatially selective and whole-frame uniform downsampling. Table 7. Stage-by-stage progression vs. pristine fixed-QP baselines at 640640×360360 (median over 6 sequences). Stage Encoder Δ Δ Δ Codec ROI kvazaar −1.0%-1.0\% +0.42+0.42 dB −0.74-0.74 dB ELVIS SVT-AV1 +22.8%+22.8\% −0.15-0.15 dB −5.47-5.47 dB PRESLEY SVT-AV1 +1.2%+1.2\% −0.13-0.13 dB −1.85-1.85 dB ELVIS x265 −1.7%-1.7\% −0.16-0.16 dB −9.43-9.43 dB Table 8. Selective vs. uniform downsampling (SVT-AV1, 640640×360360, bs8, Real-ESRGAN). Negative BD-rate favors selective. Video BD-rate FG BD-rate BG Δ -LPIPS Overlap bear −18.3%-18.3\% −10.4%-10.4\% +0.087+0.087 0.83 india −30.8%-30.8\% +40.5%+40.5\% +0.101+0.101 0.67 camel +60.2%+60.2\% +21.7%+21.7\% −0.001-0.001 0.67 tennis +60.6%+60.6\% +65.6%+65.6\% −0.027-0.027 0.65 pigs +86.4%+86.4\% +51.8%+51.8\% −0.022-0.022 0.60 dog +89.8%+89.8\% +55.5%+55.5\% −0.032-0.032 0.62 Appendix E Robustness Checks: Segmentation and Resolution Table 9. Detector-derived mask (YOLOE-11l-seg) vs. ground truth on 7 sequences. Sequence Area ratio Δ Δ -PSNR india 2.652.65 +16.4%+16.4\% +2.82+2.82 dB camel 1.091.09 −2.3%-2.3\% +0.47+0.47 dB bear 0.910.91 −2.2%-2.2\% −0.12-0.12 dB pigs 0.880.88 −3.8%-3.8\% −0.36-0.36 dB bmx-trees 0.860.86 −10.1%-10.1\% −2.90-2.90 dB tennis 0.860.86 −8.0%-8.0\% −4.37-4.37 dB dog 0.540.54 −0.4%-0.4\% −0.66-0.66 dB Figure 12. BD-rate on foreground LPIPS per clip across resolutions (8080×4545 grid).Grouped horizontal bar chart with three panels, one per resolution (640x360, 1280x720, 1920x1080). Each bar is one video's BD-rate on foreground LPIPS for PRESLEY against a pristine SVT-AV1 baseline; bars left of zero mean PRESLEY needs fewer bits for equal foreground quality. At 640x360 the outcome splits three to three, with dog and pigs the two clips that reverse. At 1280x720 and 1920x1080 every measured video is below zero, but each rung has seven videos rather than eight because camel's runs there are not citable. At 1280x720 all seven save bits, which reaches p=0.0156; at 1920x1080 six of seven do, which does not reach p=0.05. Data: "arm":"presley\_ai downsample + Real-ESRGAN, shrink\_amount 0.25, fg\_protect","baseline":"pristine SVT-AV1, fixed QP 43/50/55/60","block\_grid\_held\_constant":"80x45 at every resolution (block size 8/16/24)","by\_resolution":"1280x720":"bd\_rate\_pct":[-10.6,-5.8,-5.0,-1.3,-5.3,-9.9,-3.5],"excluded":["camel"],"median\_pct":-5.3,"n":7,"sign\_p":0.0156,"videos":["bear","tennis","train","breakdance","bmx-bumps","pigs","dog"],"wins":7,"1920x1080":"bd\_rate\_pct":[-10.2,-4.3,-4.6,0.9,-2.2,-4.3,-5.6],"excluded":["camel"],"median\_pct":-4.3,"n":7,"sign\_p":0.125,"videos":["bear","tennis","train","breakdance","bmx-bumps","pigs","dog"],"wins":6,"640x360":"bd\_rate\_pct":[-16.3,-10.7,-4.4,2.1,3.4,7.4,14.3,16.9],"excluded":[],"median\_pct":2.8,"n":8,"sign\_p":0.7266,"videos":["bear","camel","tennis","train","breakdance","bmx-bumps","pigs","dog"],"wins":3,"caveats":["camel excluded at 720p and 1080p: non-empty invariant\_failures","at fixed QP the higher rungs run at roughly half the bits per pixel, i.e. more starved, which is the regime that favours the method; the apparent strengthening with resolution is confounded with that","the 360p split reproduces the known dog/pigs non-replication"],"metric":"BD-rate on FG-LPIPS, percent","sign\_convention":"negative means PRESLEY needs fewer bits at equal quality" Detector-derived foreground masks Replacing DAVIS ground-truth masks with detector masks (YOLOE-11l-seg, Table 9) indicates that foreground quality follows the area a mask protects, not the accuracy with which it segments: the rank correlation is ρ=+0.893ρ=+0.893, and india — the one sequence whose detector mask is larger than the annotation — is also the one whose bitrate rises. Morphological perturbations (dilation/erosion radius 4) alter foreground PSNR by ≤0.12≤$0.12$ dB, demonstrating robustness to boundary noise. Resolution scaling At 12801280×720720 the saving is a result rather than a trend: all 77 citable clips need fewer bits for equal foreground quality, a median of −5.3 %-5.3\,\%, exact two-tailed p=0.0156p=$0.0156$, or 0.0470.047 after Holm correction across the three resolutions. At 19201920×10801080 it does not follow (66 of 77, median −4.3 %-4.3\,\%, p=0.125p=$0.125$), and at 640640×360360 the split persists at 33 of 88. Two things bound this. At fixed QP the higher resolutions run at roughly half the bits per pixel and so sit in the more starved regime that favours the method; and breakdance spends 0.2140.214 bits per pixel at 640640×360360 against the 0.078 to 0.1360.0780.136 of the other seven, outside the range registered in advance as one operating regime, so its lowest-quality QP is reported but not pooled. Appendix F Whole-Frame Quality Metrics Table 10 reports full-frame metric changes against pristine baselines across seven standards. Learned metrics (LPIPS, DISTS) and structural metrics confirm the ranking established by region-restricted evaluations. Table 10. Whole-frame metrics: median change against pristine baselines paired within (video, codec, QP, resolution). Configuration n Δ ↑ Δ ↑ Δ ↑ Δ ↓ Δ ↓ Δ ↓ Δ ↓ PRESLEY downsample + Real-ESRGAN 295295 −2.21-2.21 −0.053-0.053 −7.8-7.8 +0.026+0.026 +0.003+0.003 — — PRESLEY blackout + ProPainter 1818 −7.18-7.18 −0.117-0.117 −36.3-36.3 +0.129+0.129 +0.047+0.047 +12.2+12.2 +1368+1368 PRESLEY freeze + ProPainter 2828 −9.50-9.50 −0.200-0.200 −51.4-51.4 +0.156+0.156 +0.052+0.052 +12.8+12.8 +2772+2772 ELVIS (block removal) 149149 −11.67-11.67 −0.195-0.195 −45.2-45.2 +0.177+0.177 +0.070+0.070 +16.2+16.2 +2284+2284 Appendix G Per-Point Codec ROI Performance Table 11 details every evaluated operating point for Kvazaar native ROI encoding. Table 11. Kvazaar native ROI encoding per operating point under fixed QP. Video Setting Rate ROI Rate base Δ Δ -PSNR Δ -PSNR bear kv-150 155.8 146.5 +6.3%+6.3\% +0.53+0.53 dB −0.40-0.40 dB bear kv-250 233.7 224.5 +4.1%+4.1\% +0.57+0.57 dB −0.53-0.53 dB bear kv-460.8 460.5 492.4 −6.5%-6.5\% +0.71+0.71 dB −0.73-0.73 dB bear kv-555 577.5 492.4 +17.3%+17.3\% +1.34+1.34 dB −0.22-0.22 dB bear kv-800 727.1 823.9 −11.7%-11.7\% +0.73+0.73 dB −0.89-0.89 dB bmx-trees kv-460.8 501.1 467.0 +7.3%+7.3\% +0.24+0.24 dB −0.04-0.04 dB camel kv-300 304.0 279.8 +8.7%+8.7\% +0.51+0.51 dB −1.12-1.12 dB camel kv-600 572.4 588.1 −2.7%-2.7\% +0.58+0.58 dB −1.28-1.28 dB dog kv-300 282.4 328.9 −14.1%-14.1\% +0.57+0.57 dB −0.88-0.88 dB dog kv-600 593.0 616.5 −3.8%-3.8\% +1.00+1.00 dB −0.62-0.62 dB drift-chicane kv-460.8 447.1 447.4 −0.1%-0.1\% +0.21+0.21 dB −0.13-0.13 dB india kv-300 299.3 290.8 +2.9%+2.9\% +0.23+0.23 dB −0.20-0.20 dB india kv-600 577.1 565.6 +2.0%+2.0\% +0.20+0.20 dB −0.25-0.25 dB pigs kv-300 311.2 322.6 −3.5%-3.5\% +0.13+0.13 dB −0.95-0.95 dB pigs kv-600 553.3 654.4 −15.4%-15.4\% +0.09+0.09 dB −1.33-1.33 dB tennis kv-300 300.1 315.5 −4.9%-4.9\% −0.45-0.45 dB −0.25-0.25 dB tennis kv-600 622.1 581.7 +6.9%+6.9\% +0.14+0.14 dB +0.13+0.13 dB Appendix H Data Provenance and Pre-registration Commitments Every experiment record in this article is tracked by an immutable configuration hash and validated against five automated invariants: existence of valid region metrics, bitrate accounting within 1 %1\,\%, fixed-QP rate control, non-degraded background restoration, and absence of numerical saturation. Runs failing any invariant are excluded. Three analyses were additionally bounded before the runs they depend on existed, with the design documents committed ahead of the data, and we record what fired rather than revising the bounds. The regime-stability contrast did not fire (p=0.58p=$0.58$), which is the reported result. For restorability, the correlation over all superblocks exceeded its band, and the control that resolved it — restricting to actually-degraded blocks, which is where the reported ρ=+0.506ρ=+$0.506$ comes from — was declared after the fact, so both figures are reported. For the resolution ladder, two sequences fell outside the band on foreground BD-rate and one further bound fired, on breakdance’s bits per pixel. Two further bounds fired elsewhere in the article and are disclosed where their results appear: the placement contrast of Section 4.5 landed far outside a band that was itself mis-derived, and that section’s exclusion test was pre-registered on foreground LPIPS while its stated basis was a foreground PSNR figure. Both mis-specifications are ours.