Paper deep dive
Your Pre-trained Diffusion Model Secretly Knows Restoration
Sudarshan Rajagopalan, Vishal M. Patel
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%
Last extracted: 4/10/2026, 2:59:08 AM
Summary
The paper introduces a method to unlock restoration capabilities in pre-trained diffusion models (WAN and FLUX) by optimizing prompt embeddings in the text-encoder output space rather than token space. It addresses the train-test trajectory mismatch by employing an energy-oriented diffusion bridge (EBR) to align training and inference dynamics, enabling high-quality image and video restoration without fine-tuning the backbone.
Entities (5)
Relation Signals (3)
Energy-oriented Diffusion Bridge → solves → trajectory mismatch
confidence 95% · we adopt the energy-oriented diffusion bridge (EBR) [21]... mitigating the train-test trajectory mismatch
Wan → isusedfor → All-in-One Restoration
confidence 90% · we introduce our lightweight learned prompts on the pre-trained WAN video model... converting them into high-performing restoration models.
Flux → isusedfor → All-in-One Restoration
confidence 90% · we introduce our lightweight learned prompts on... FLUX image models, converting them into high-performing restoration models.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pre-trained diffusion models have enabled significant advancements in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model's priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely inaccessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive performance and generalization across diverse degradations, while avoiding fine-tuning and restoration-specific control modules.
Tags
Links
- Source: https://arxiv.org/abs/2604.04924v1
- Canonical: https://arxiv.org/abs/2604.04924v1
Trouble viewing inline? Open PDF directly →
Full Text
52,967 characters extracted from source content.
Expand or collapse full text
Your Pre-trained Diffusion Model Secretly Knows Restoration Sudarshan Rajagopalan and Vishal M. Patel Johns Hopkins University, Baltimore MD 21218, USA sambasa2,vpatel36@jhu.edu Abstract. Pre-trained diffusion models have enabled significant advance- ments in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model’s priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behav- ior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely in- accessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image mod- els, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive per- formance and generalization across diverse degradations, while avoid- ing fine-tuning and restoration-specific control modules. Project page: https://sudraj2002.github.io/yptpage/. Keywords: All-in-one Restoration· Diffusion Models· Low-level vision 1 Introduction All-in-One Restoration (AiOR) aims to restore images or videos corrupted by various degradations such as haze, rain, snow, motion blur, and low light, us- ing a single unified model. Recent AiOR methods increasingly leverage large pre-trained text-to-image or video (T2IV) latent diffusion models (LDMs) such as Stable Diffusion [39, 44], Lumina [14, 64], and FLUX [24]. Trained on mas- sive data, these models possess strong priors over the natural image distribution and can generate high-quality content, making them attractive backbones for robust and perceptually pleasing restoration [20, 29, 31, 42]. However, most ex- isting approaches harness these priors primarily by adapting the diffusion back- bone via fine-tuning [8, 20, 29] or by introducing ControlNet-style conditioning arXiv:2604.04924v1 [cs.CV] 6 Apr 2026 2S. Rajagopalan, V.M. Patel InputP2P+NTISDEditOurs Fig. 1: Popular editing based approaches such as SDEdit [36], and Prompt-to- prompt [18] with Null-text inversion [37] (P2P + NTI) work well for high-level editing but perform poorly for restoration tasks, in this case dehazing. modules [34,58]. While effective, these approaches typically require substantial training and can be overly sensitive to the training distribution which can weaken the strong pre-trained priors, limiting robustness under real-world or complex degradations. Alternatively, plug-and-play methods [13, 35] avoid training but often depend on explicit degradation operators, which are difficult to specify for complex scenarios. In parallel, pre-trained T2IV models have enabled remarkable progress in image editing, where the sampling trajectory is steered through language while keeping the diffusion backbone frozen. This paradigm is appealing and versatile because it preserves the model’s original priors and achieves strong high-level semantic edits. However, directly transferring such editing strategies to low-level restoration tasks is largely ineffective. Inversion-based editing methods [18, 37] alter image details, while partial-noising approaches such as SDEdit [36] preserve the input structure but, the natural-language guidance (e.g., “remove haze”), often fails to remove degradations from intermediate noisy degraded states and converges back to the degraded input with noise suppressed (see Fig. 1). This raises a key question: Do large diffusion model priors contain restoration-relevant knowledge, and if so, can we access them through conditioning mechanisms? In this paper, we investigate this question and reveal that the bottleneck is not the frozen diffusion backbone but the conditioning representation. Standard token-space prompts, whether obtained through natural language or optimized as learned tokens (e.g., textual inversion/prompt tuning), fail to elicit restoration behavior (see Fig. 2). In contrast, directly optimizing the conditioning in em- bedding space (i.e., at the output of the text encoder) unlocks strong restoration performance, i.e. the frozen diffusion model consistently maps noisy degraded inputs toward clean targets, suggesting that restoration knowledge exists in the pre-trained diffusion prior but is not accessible through the text tokens. A sec- ond challenge is trajectory mismatch. Naively training prompts using a forward process of the formx t = α(t)x deg + σ(t)ε leads to a difference in the distribu- tions of x t during training and inference, which affects prompt performance. To address this issue, we propose to train prompts within a diffusion-bridge for- mulation that aligns trajectories during training and inference-time sampling. We introduce our framework on the WAN 1.3B [49] video diffusion model and the FLUX 12B [24] image diffusion model, achieving competitive performance Your Pre-trained Diffusion Model Secretly Knows Restoration3 Noisy degraded inputText prompt Token optimization Ours (embedding) Fig. 2: Text token-space prompting is ineffective for restoration: Even with optimized token prompts (textual inversion/prompt tuning), the model tends to denoise with- out removing degradations, whereas embedding-space optimization enables restoration from the same noisy degraded input. against state-of-the-art restoration methods across diverse degradations with significantly fewer trainable parameters. To summarize our contributions are as follows: – We show that natural-language prompts and token-space embedding opti- mization are ineffective for eliciting restoration behavior from pre-trained text-to-image/video diffusion backbones. We then reveal that directly learn- ing the conditioning signal in the text-encoder output space unlocks strong restoration while keeping the diffusion backbone frozen. – We identify a train-test trajectory mismatch that makes naive prompt learn- ing unstable, and propose training prompts on bridge-defined intermediate states (with a bridge-compatible reparameterization for WAN/FLUX) to align optimization and sampling dynamics. – We instantiate our framework on the WAN and FLUX models, and show that they achieve competitive restoration performance to the state-of-the-art and strong generalization with only lightweight learned prompts. 2 Related Works 2.1 All-in-One Restoration AiOR aims to restore images or videos affected by diverse degradations using a single unified model. Early works such as All-in-One [28] and TransWeather [48] explored unified architectures and training strategies. Subsequent methods im- proved performance by introducing learnable prompts such as in PromptIR [40] and visual in-context cues [43]. More recent instruction-guided approaches in- corporate language to select restoration behavior, e.g., InstructIR [10] and DF- PIR [46]. Other approaches such as GenDeg [41] and FoundIR [27] investigate data scaling effects for AiOR. ViWS-Net [54] proposed a transformer-based ap- proach for video all-weather removal while AverNet [59] proposed a diffusion- 4S. Rajagopalan, V.M. Patel based method for video AiOR. With the rise of large pretrained diffusion mod- els, many AiOR approaches exploit their generative priors to improve percep- tual quality and robustness [20, 31, 42]. Existing diffusion-based AiOR meth- ods typically leverage these priors through two main paradigms: (i) Backbone adaptation: methods such as AutoDIR [20], UniLDiff [8], and PixWizard [29] adapt pretrained diffusion backbones (e.g., Stable Diffusion [44]) using full or parameter-efficient fine-tuning to follow restoration-based language instructions. While effective, this requires substantial training and can bias the model toward the training data distribution, potentially reducing robustness under complex real-world corruptions. (i) Learned conditioning modules: another line of work keeps the diffusion backbone frozen but trains ControlNet-style branches or con- ditioning networks to inject task- or degradation-dependent signals, as in UNI- CORN [34] and Diff-Restorer [58]. These modules directly modify intermediate features of the frozen model, which can weaken the model’s pretrained priors if the conditioning module is not robust. Plug-and-play methods [13, 35] perform inference-time guidance using pretrained diffusion priors, but typically depend on explicit degradation operators or optimization procedures, which are difficult to specify for complex or unknown degradations. Unlike the above approaches, we learn compact prompts in the native conditioning pathway of the frozen diffusion model which unlocks their potential for restoration tasks. 2.2 Pre-trained Diffusion Models for Editing Pre-trained diffusion models have been widely explored for editing tasks, thanks to their strong generative prior. A common paradigm includes inversion-based methods that aim to accurately recover a noise representation that reconstructs the input under the diffusion model [3,5,37]. Subsequently, structural or language- driven controls are applied during denoising to realize the desired edits [18,22,47]. While effective for semantic manipulation, these mechanisms often fail to pre- serve fine-grained fidelity (e.g., background texture, identity, or small structures) due to imperfect inversion and attention interventions which could be particu- larly problematic for restoration. SDEdit [36] avoids exact inversion by partially noising the input and denoising from that intermediate state, offering a tun- able trade-off between fidelity and the model’s generative prior via the chosen noise level. However, we observe that simply replacing the editing instruction with a restoration instruction (e.g., “remove haze”, “deblur”) performs poorly for restoration as the model tends to suppress noise without removing degradations. Motivated by this gap, we investigate how to enable an SDEdit-like restoration procedure by learning compact prompts that steer denoising toward clean targets while keeping the diffusion backbone frozen. 3 Proposed Method In this section, we explain the core innovations of our method along with impor- tant design choices. Your Pre-trained Diffusion Model Secretly Knows Restoration5 Fig. 3: (a) We freeze the diffusion backbone and optimize only the conditioning: token- space prompt optimization fails, while embedding-space (text-encoder output) opti- mization elicits restoration. (b) Naive tuning yields states anchored at z deg ; DDBM [61] is pinned at both endpoints; our desired/EBR-style [21] bridge starts from noisy de- graded inputs and denoises monotonically as the content transitions toward z clean . (c) Naive training sees a different state family than inference, causing trajectory misalign- ment. (d) Bridge-based training aligns train/test states; DDBM may under-correct early (low noise near z deg ), while the desired/EBR bridge enables stronger correction on an aligned path. 3.1 Guiding pre-trained diffusion models for restoration Given paired data (x deg ,x clean ), our goal is to produce a restored image/video, x pred , that matches x clean . Toward this aim, we leverage large pre-trained text-to- image/video diffusion backbones that operate in the latent space of a VAE whose encoder isE. Let z deg =E(x deg ) and z clean =E(x clean ) denote the corresponding VAE latents, and let ε∼N(0,I). Following the flow-matching parameterization used by WAN/FLUX, we construct a noisy degraded input as z deg t = 1− t z deg + tε, t∈ [0, 1],(1) Since these diffusion models possess strong priors of the natural image distribu- tion, we ask whether the noisy degraded latent, z deg t , can be restored by steering reverse-time sampling with a restoration instruction. Concretely, we condition the frozen backbone on a text prompt c (e.g., “Remove the haze from this im- age and make it clear.”) and perform reverse sampling from timestep t. Let τ 6S. Rajagopalan, V.M. Patel denote tokenization and embedding lookup, and let TE denote the frozen text encoder. For the prompt c, we obtain token embeddings y = τ(c)∈ R L×D in , and context embeddings as e = TE(y) ∈ R L×D , where L is the number of tokens and D is the embedding dimension. e then enters the frozen backbone through a conditioning pathway (e.g. cross-attention). However, as shown in Fig. 2, such natural-language prompts fail to guide the backbone from intermediate noisy degraded states toward z clean , and instead suppress the added noise, and return to the degraded input. We further test whether this limitation is simply due to insufficient expres- sivity of the text prompt by optimizing the prompt itself (in token-space) while keeping the diffusion backbone and text encoder frozen. Concretely, we form the noisy degraded state z deg t as in Eq. (1) and seek prompt tokens that make the frozen backbone predict the clean latent (z clean ) from this state. More specifically, we optimize token embeddings U ∈ R L×D in at the input of the text encoder, pro- ducing context embeddings u = TE(U) ∈ R L×D that minimize the backbone’s reconstruction error of the clean latent L restore = E (z deg ,z clean ),t,ε h ˆz 0,θ (z deg t ,t;u)− z clean 2 2 i ,(2) where ˆz 0,θ = z deg t − tv θ (z deg t ,t;u) with the expectation over training data pairs. Despite optimizing for U, we observe that the loss does not converge and pro- duces a denoised degraded output (Fig. 2). Surprisingly, bypassing the token space and directly optimizing the condi- tioning in context embedding space elicits a strong restoration response. Let p∈ R L×D denote the context embedding vector in the output space of the text encoder fed into the backbone’s conditioning pathway. We now keep the diffusion backbone frozen and optimize p using the same clean sample prediction objec- tive as in Eqn. 2. Unlike token-space optimization, we find that embedding-space optimization (Fig. 3(a)) consistently drives ˆz 0,θ (z deg t ,t;p) toward z clean and pro- duces visibly restored predictions from the same noisy degraded inputs (Fig. 2). This suggests that restoration-relevant capability is present in the pre-trained diffusion prior but is not accessible through the standard input token-based in- terface. While embedding-space prompt optimization elicits restoration behavior dur- ing training, it introduces a train-test trajectory mismatch at inference-time sampling. Specifically, Eq. (1) constructs training states whose signal compo- nent remains anchored at z deg , so the learned prompt p is only exposed to the intermediate samplesz deg t t∈[0,1] along the “degraded→ degraded+noise” path (see Figs. 3(b) and (c)). During sampling, however, the states at each timestep must progressively move from z deg toward z clean , and therefore visit intermediate states whose signal component is no longer centered at z deg (see Fig. 3(c)). This applies p to unseen intermediate states, leading to instability and artifacts (see Sec. 4.4). We address this mismatch in the next section. Your Pre-trained Diffusion Model Secretly Knows Restoration7 3.2 Addressing Trajectory Mismatch The afore-mentioned problem motivates constructing training trajectories whose intermediate states smoothly transition from degraded to clean, i.e., a diffusion bridge between the paired endpoints z deg and z clean . A bridge is attractive be- cause it provides a sequence of intermediate states that (i) interpolates the signal from z deg to z clean and (i) can be used in both training and inference, thereby mitigating the train-test trajectory mismatch (see Fig. 3(d)). Thus, in our set- ting with a frozen diffusion backbone and a trainable conditioning prompt p, the bridge defines compatible training inputs and sampling trajectories, ensuring that p encounters similar intermediate states at training and test time. Delving deeper, denoising diffusion bridge models (DDBMs) [61] construct analytically tractable Gaussian distributions of the form q(z t | z clean ,z deg ) =N z t ; a t z deg + b t z clean , s 2 t I ,(3) with (a t ,b t ,s t ) defined by the chosen bridge schedule [61] at time t. This yields a closed-form expression for sampling the input from the forward process as z br t = a t z deg + b t z clean + s t ε, ε∼N(0,I).(4) However, WAN/FLUX models expect an input of the form in Eqn. 1. Addition- ally, the noise level of the bridge needs to be matched with that of the frozen backbone. To achieve this, we first view a t z deg +b t z clean as the signal component of the input and ε as the sampled noise. If we multiply by (1− σ t ), where σ t is the desired noise coefficient for WAN/FLUX, we get y br t = (1− σ t )(a t z deg + b t z clean ) + (1− σ t )s t ε.(5) Now, defining σ t = s t 1+s t implies (1− σ t )s t = σ t and yields y br t = (1− σ t )(a t z deg + b t z clean ) + σ t ε.(6) which is now in the form of Eqn. 1. We now optimize the prompt so that the frozen backbone, when conditioned on p, predicts the clean endpoint from y br t : min p E (z deg ,z clean ),t,ε h ˆz 0,θ (y br t ,t;p)− z clean 2 2 i ,(7) where ˆz 0,θ (·) is the clean estimate predicted by the frozen backbone (Sec. 3.1). Intuitively, the above process can be viewed as the prompt observing various intermediate signal components of the form a t z deg + b t z clean and providing the necessary guidance to restore them. During sampling, ˆz 0,θ (·) is used to step through the reverse trajectory [61], ensuring the prompt encounters similar in- termediate samples as in training (see Fig. 3(d)). Despite the alignment benefit, using the above DDBM-like bridge for training our prompts with frozen backbones leads to two key problems. First, the bridge variance s 2 t is typically small near both endpoints and peaks in the middle (see 8S. Rajagopalan, V.M. Patel row 2 of Fig. 3(b). This means, near the degraded endpoint the state is almost deterministic and close to z deg , leaving limited stochasticity/uncertainty for the frozen generative prior to “re-route” the trajectory early (see Fig. 3(d)). Empir- ically, this significantly weakens the prompt’s restoration capability at the be- ginning of sampling and leads to under-correction of degradations (see Sec. 4.4). Second, deterministic integration (ODE-based solvers) from a fixed endpoint can yield overly smooth trajectories [60, 61]. DDBM mitigates this with stochastic “churn” [61], but churn increases neural function evaluations (NFEs), which is expensive for large backbones (WAN/FLUX) and can still yield suboptimal per- formance [60]. Thus, we require a bridge-like framework whose starting state al- ready contains noise on top of the degraded input and whose noise level decreases monotonically as the signal transitions toward clean (see row 3 of Fig. 3(b)). Concretely, we want intermediate states of the form z br t = μ(t) +s(t)ε, μ(t) = λ(t)z deg + 1−λ(t) z clean , ε∼N(0,I),(8) where t ∈ [0,T 0 ] (with T 0 ≤ 1), λ(0) = 0, λ(T 0 ) = 1, and s(t) increases with t. This construction (i) exposes the prompt early on to noisy degraded states allowing strong restoration, while (i) gradually moving towards cleaner states at later times, avoiding the train-test trajectory mismatch (see Fig. 3(d)). For this purpose, we adopt the energy-oriented diffusion bridge (EBR) [21] which constructs bridge states as z ebr t = (1− t) h 1− t T 0 z clean + t T 0 z deg i + tε, (9) which satisfies our desired endpoint behaviour. Importantly, our contribution is not the bridge schedule itself, but showing that this monotone noisy-to-clean bridge is the missing ingredient that makes our prompt-only restoration (Sec. 3.1) feasible on frozen large diffusion models. Thus, we leverage the EBR forward construction to sample training states z ebr t and train p with the clean endpoint prediction objective min p E (z deg ,z clean ),t,ε h ˆz 0,θ (z ebr t ,t;p)− z clean 2 2 i ,(10) where ˆz 0,θ (·) is the clean estimate using p with the frozen backbone. At inference, we run a deterministic sampler (DDIM-like [45]) using the same bridge schedule so that the intermediate states visited during sampling match those seen during training, mitigating trajectory mismatch and enabling successful prompt training (see Sec. 4.4). 3.3 Design choices One prompt per degradation. We learn a separate conditioning prompt p (k) for each degradation type k (e.g., haze, rain, low-light), while keeping the diffu- sion backbone frozen. This still follows the AiOR setup where a single backbone Your Pre-trained Diffusion Model Secretly Knows Restoration9 Table 1: Comparisons of mean within-distribution performance of our approach on the WAN and FLUX models with state-of-the-art image and video AiOR approaches. Methods Image restoration (mean)Video restoration (mean) P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑ Do↑ DCPT24.82 0.812 0.178 0.119 0.542 61.5719.50 0.715 0.285 0.167 0.462 54.75 0.420 DFPIR 24.96 0.813 0.177 0.117 0.557 61.8720.16 0.743 0.247 0.151 0.493 58.64 0.462 AutoDIR26.29 0.854 0.138 0.094 0.591 64.0119.57 0.739 0.236 0.144 0.509 60.17 0.490 PixWizard19.40 0.608 0.229 0.133 0.627 64.3017.73 0.612 0.248 0.137 0.581 63.26 0.521 FoundIR 18.36 0.672 0.281 0.193 0.497 55.5817.30 0.672 0.309 0.223 0.434 51.99 0.419 FLUX (Ours)23.30 0.750 0.173 0.119 0.552 64.11– ViWS-Net–19.76 0.716 0.310 0.241 0.381 45.87 0.375 AverNet–21.75 0.799 0.250 0.169 0.357 38.88 0.349 WAN (Ours) –20.89 0.701 0.184 0.118 0.534 64.19 0.571 is reused across degradations and task-specific information is provided as con- ditioning. Here, the conditioning is learned prompts while in AiOR approaches such as PixWizard [29] and DFPIR [46], degradation-specific text is used to guide a single backbone. For addressing mixed degradations, we combine prompts by averaging their predicted velocities during sampling, enabling joint restoration without additional training. Prompt injection and residual learning. Rather than optimizing the text- encoder output directly, we learn a residual prompt on top of the null-text context and inject it at the conditioning interface (e.g. input to cross-attention). We found this design is more parameter-efficient and improves performance com- pared to directly optimizing the text-encoder output (see Sec. 4.4). We provide the specific-architectural details in the supplementary. Choosing the start noise level T 0 . We set the maximum noising level T 0 used in training (Eq. 9) based on degradation severity. Milder degradations typically require lower noise to restore, whereas severe degradations benefit from a larger T 0 to provide the frozen prior sufficient flexibility to correct the input. In our training data, low-light exhibits the most severe corruption levels. We therefore tune T 0 on low-light and find T 0 = 0.4 performs best (Sec. 4.4). For simplicity and robustness, we use this same T 0 for all degradations, ensuring adequate headroom for difficult cases. 4 Experiments In this section, we discuss key implementation details, experimental setup, com- parisons with other methods, and ablations on our proposed prompts. 4.1 Implementation Details We conduct our prompt-learning approach on WAN [49] text-to-video model comprising 1.3 billion parameters and FLUX1 Dev [24] text-to-image model consisting of 12 billion parameters. For WAN, we learn 226 prompt tokens of dimension 1536, and for FLUX we learn 512 prompt tokens of dimension 3072. We optimize prompts using AdamW [33] with learning rate 5× 10 −4 and batch 10S. Rajagopalan, V.M. Patel Table 2: Quantitative comparisons of our prompt learning approach on the FLUX model with state-of-the-art AiOR approaches for images from OOD, mixed and unseen degradations. H-Haze, R-Rain, S-Snow, B-Blur, L-Low-light, and U-Unseen. Methods HazeRD (H)LHP (R)WeatherBench (S) P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑ DCPT13.98 0.774 0.214 0.156 0.620 66.1627.49 0.828 0.196 0.135 0.480 58.8721.31 0.753 0.244 0.173 0.363 49.96 DFPIR15.73 0.795 0.181 0.123 0.681 68.2429.96 0.854 0.177 0.124 0.475 57.2117.77 0.693 0.272 0.184 0.332 44.12 AutoDIR16.59 0.843 0.164 0.114 0.679 69.3828.60 0.840 0.183 0.125 0.537 59.0021.14 0.746 0.245 0.169 0.386 47.90 PixWizard 15.16 0.638 0.249 0.159 0.696 68.9620.88 0.643 0.230 0.153 0.615 59.3022.40 0.691 0.197 0.128 0.447 45.43 FoundIR14.41 0.810 0.236 0.182 0.611 68.1830.12 0.868 0.155 0.107 0.527 56.4321.60 0.769 0.240 0.165 0.410 45.23 FLUX (Ours)16.70 0.705 0.233 0.162 0.685 69.0126.44 0.816 0.163 0.108 0.542 58.0922.07 0.738 0.264 0.178 0.389 46.52 Methods 4KRD (B)SICE (L)TOLED (U) P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑ DCPT23.48 0.828 0.193 0.142 0.509 58.3315.29 0.671 0.228 0.144 0.618 66.3015.14 0.630 0.323 0.217 0.419 46.63 DFPIR27.72 0.870 0.150 0.119 0.527 61.8316.41 0.722 0.256 0.165 0.578 68.4011.92 0.558 0.337 0.224 0.429 45.71 AutoDIR27.46 0.865 0.143 0.112 0.563 65.2613.55 0.620 0.284 0.183 0.602 65.9116.99 0.669 0.270 0.192 0.450 49.71 PixWizard19.86 0.584 0.193 0.126 0.652 69.1012.33 0.432 0.373 0.217 0.613 63.7924.41 0.731 0.196 0.151 0.549 61.30 FoundIR27.87 0.865 0.150 0.124 0.528 60.8110.93 0.449 0.395 0.253 0.557 60.5028.08 0.824 0.191 0.165 0.429 48.64 FLUX (Ours) 22.80 0.702 0.137 0.094 0.674 72.6715.39 0.566 0.328 0.208 0.585 66.0818.05 0.671 0.229 0.173 0.515 58.05 Methods POLED (U)CDD (S+H)LOLBlur (L+B)Average P↑ S↑ L↓ D↓ C↑ M↑P↑ S↑ L↓ D↓ C↑ M↑C↑M↑P↑ S↑ L↓ D↓ C↑ M↑ DCPT9.06 0.364 0.613 0.462 0.340 42.9313.85 0.713 0.308 0.223 0.553 61.170.374 39.5817.45 0.695 0.289 0.206 0.475 54.43 DFPIR8.93 0.351 0.657 0.461 0.324 41.8214.95 0.721 0.305 0.214 0.555 60.850.344 39.1317.92 0.695 0.291 0.201 0.471 54.14 AutoDIR11.69 0.435 0.564 0.417 0.348 47.4419.76 0.827 0.223 0.145 0.668 66.350.404 43.1419.47 0.730 0.259 0.182 0.515 57.12 PixWizard 17.62 0.462 0.477 0.341 0.387 43.6816.19 0.509 0.250 0.162 0.675 66.270.536 56.1118.60 0.586 0.270 0.179 0.574 59.32 FoundIR16.32 0.478 0.658 0.531 0.280 27.9418.07 0.805 0.228 0.154 0.620 63.090.335 35.3820.92 0.733 0.281 0.210 0.477 51.80 FLUX (Ours) 18.52 0.474 0.647 0.442 0.300 38.5416.66 0.591 0.253 0.171 0.651 67.340.425 48.2119.57 0.657 0.281 0.192 0.529 58.27 size 2, training for 20k (WAN) and 40k (FLUX) iterations. All experiments are run on 2× NVIDIA H100 GPUs (80GB). 4.2 Setup We train one prompt per degradation on a single in-domain (ID) dataset, and report performance on (i) ID test sets, (i) out-of-distribution (OOD) datasets, and (i) mixed and unseen degradations. This evaluation protocol is similar to those followed in [27,41]. Image restoration (FLUX). We train prompts for dehazing, desnowing, de- raining, low-light enhancement, and deblurring using: RESIDE [25] (72135 train images, haze), Snow100K [32] (50000, snow), Rain13K [55] (13711, rain), LOLv1 [52] (485, low-light), and GoPro [38] (2103, blur). For OOD testing, we evaluate on HazeRD [1] (haze), LHP [17] (rain), WeatherBench [15] (snow), 4KRD [11] (blur), and SICE [4] (enhancement). To assess performance on mixed and un- seen degradations, we use TOLED and POLED [63] for under-display camera restoration (unseen), haze+snow split of CDD [16], and LOLBlur [62] for low- light+blur. Video restoration (WAN). We train prompts for dehazing, deraining, desnow- ing, low-light enhancement, and deblurring using: REVIDE [57] (40 train clips, haze), NTU-Rain (synthetic) [7] (25, rain), RVSD (synthetic) [6] (77, snow), SDSD [51] (58, low-light), and GoPro [38] (22, blur). For OOD testing, we eval- uate on RHVD [9] (haze), NTU-Rain (real) [7] and LasVR [30] (rain), AAU- RainSnow [2] (snow), Lol-iPhone [26] (low-light), and 4KRD [11] (blur). During test time, we evaluate on the first 33 frames of the test videos due to compute constraints. Your Pre-trained Diffusion Model Secretly Knows Restoration11 Table 3: Quantitative comparisons of our prompt learning approach on the WAN model with state-of-the-art image and video restoration approaches for the task of all-in-one video restoration on OOD datasets. H-Haze, R-Rain, S-Snow, B-Blur, and L-Low-light Methods RHVD (H)LasVR (R)NTURain (R)AAURainSnow (S) ↑C M↑ Do↑P↑ S↑ L↓ D↓ C↑ M↑ Do↑C↑ M↑ Do↑C↑ M↑ Do↑ DCPT0.558 59.66 0.53029.16 0.846 0.176 0.141 0.544 63.04 0.6280.599 63.37 0.5710.432 47.60 0.321 DFPIR0.564 59.67 0.54331.94 0.909 0.101 0.097 0.546 61.94 0.6320.594 63.39 0.5450.439 47.59 0.340 AutoDIR0.579 61.25 0.52229.39 0.818 0.195 0.142 0.570 64.12 0.6160.627 65.34 0.5630.459 49.51 0.301 PixWizard0.595 60.10 0.57123.20 0.655 0.168 0.121 0.625 61.84 0.6410.643 64.38 0.5480.459 45.31 0.299 FoundIR0.558 60.75 0.58131.54 0.883 0.148 0.123 0.530 61.05 0.6550.600 61.95 0.5730.446 45.86 0.319 ViWS-Net0.529 58.72 0.55429.79 0.847 0.135 0.118 0.527 61.50 0.6570.513 58.21 0.5940.420 48.13 0.359 AverNet0.488 50.62 0.55230.80 0.863 0.176 0.149 0.472 50.43 0.5820.533 52.98 0.5450.416 43.62 0.335 WAN (Ours)0.598 63.85 0.65327.95 0.775 0.153 0.105 0.564 62.03 0.6640.583 62.75 0.6150.463 49.42 0.373 Methods 4KRD (B)Lol-iPhone (L)Average P↑ S↑ L↓ D↓ C↑ M↑ Do↑C↑ M↑ Do↑P↑ S↑ L↓ D↓ C↑ M↑ Do↑ DCPT23.16 0.819 0.202 0.145 0.504 57.72 0.5000.354 41.34 0.33526.16 0.833 0.189 0.143 0.498 55.46 0.481 DFPIR27.19 0.863 0.154 0.120 0.522 61.36 0.5440.355 44.87 0.38529.57 0.886 0.128 0.108 0.503 56.47 0.498 AutoDIR26.94 0.858 0.144 0.110 0.564 65.86 0.6180.374 44.18 0.37228.16 0.838 0.169 0.126 0.529 58.38 0.499 PixWizard19.77 0.587 0.196 0.124 0.647 68.60 0.6120.441 48.00 0.37321.48 0.621 0.182 0.122 0.568 58.04 0.507 FoundIR27.36 0.856 0.155 0.124 0.524 60.28 0.5880.370 40.53 0.39729.45 0.869 0.151 0.123 0.505 55.07 0.519 ViWS-Net25.11 0.801 0.195 0.150 0.523 56.79 0.5190.333 39.52 0.36027.45 0.824 0.165 0.134 0.474 53.81 0.507 AverNet28.28 0.868 0.149 0.113 0.470 50.81 0.5860.304 35.12 0.42329.54 0.865 0.162 0.131 0.447 47.26 0.504 WAN (Ours)20.98 0.625 0.172 0.117 0.608 69.44 0.6480.328 41.67 0.38724.46 0.700 0.162 0.111 0.524 58.19 0.557 Among the above, LHP, WeatherBench, TOLED, POLED, LOLBlur, REV- IDE, NTU-Rain (real), SDSD, RHVD, AAURainSnow and Lol-iPhone contain real world degradations. We provide the full per-dataset sample counts and de- tails in the supplementary. 4.3 Comparisons Image restoration. We instantiate our learned prompts on the pre-trained FLUX backbone and compare against recent all-in-one restoration (AiOR) meth- ods, including DCPT [19], DFPIR [46], AutoDIR [20], PixWizard [29], and FoundIR [27], using their publicly available checkpoints. We report both dis- tortion and perceptual metrics: PSNR (P) in dB, SSIM (S), LPIPS (L) [56], DISTS (D) [12], CLIPIQA (C) [50], and MUSIQ (M) [23]. The mean within- distribution (WD) performance for all methods is reported in Table 1. We note that methods based on latent diffusion models are impacted by the reconstruc- tion bottleneck of the VAE, which can affect PSNR/SSIM scores even when the restoration is of high quality. This has also been observed in other works [29,42]. More detailed WD results are given in the supplementary. Testing on out-of-distribution (OOD), mixed, and unseen degradations pro- vides a more faithful measure of performance under real-world and complex corruptions. As shown in Table 2, our method remains consistently competitive across all three settings, often delivering strong perceptual scores. For the un- seen TOLED and POLED datasets, we condition using the low-light and blur prompts, as they were a close visual match to the under-display camera degra- dations. Video restoration. For videos, we compare our WAN-based instantiation to ViWS-Net [54] and AverNet [59], and additionally report DOVER (Do) [53] as a 12S. Rajagopalan, V.M. Patel InputDFPIRAutoDIR PixWizard FoundIR FLUX (Ours) TOLED LHP LOLBlur WBSnow HazeRD Fig. 4: Qualitative comparisons of the pre-trained FLUX model using our learned prompts with state-of-the-art AiOR approaches. Our approach enables the pre-trained FLUX to achieve remarkable restoration performance. WBSnow denotes the snow sub- set of the WeatherBench [15] dataset. video quality metric. We also report the scores for image-based approaches. Since the publicly available checkpoints of AverNet and ViWS-Net did not address many of our restoration tasks, we re-trained them on the same training data. WD results are reported in Table 1 and comparisons on OOD data is given in Table 3. As for images, we provide detailed WD results in the supplementary. Table 3 shows that our prompt-tuned WAN achieves strong performance for the OOD degradations. Moreover, it outperforms image and video restoration performance on the DOVER video quality assessment metric as the pre-trained WAN has very strong temporal priors. Qualitative evaluation. Beyond metrics, we visualize results on OOD, mixed, and unseen degradations in Fig. 4 for image restoration datasets and Fig. 5 for video restoration datasets. In the interest of space, we only provide qualitative comparisons for specific OOD, mixed and unseen degradation datasets. Addi- tional results on the remaining OOD datasets, within-distribution benchmarks, and comparisons with DCPT are provided in the supplementary material. While several methods achieve strong scores on specific datasets, we observe that they often fail to restore under such complex conditions. For example, in Fig. 4, our prompt-instantiated FLUX model is able to achieve better visual restoration than the state-of-the-art approaches. A similar observation can be inferred from Fig. 5. To facilitate a clearer comparison against our prompt- Your Pre-trained Diffusion Model Secretly Knows Restoration13 InputAutoDIRFoundIR ViWS-NetAverNet WAN (Ours) AAU NTU RHVD 4KRD LoL-iPhone Fig. 5: Qualitative comparisons of the pre-trained WAN model using our learned prompts with state-of-the-art AiOR approaches. ViWS-Net and AverNet are video restoration approaches while others are proposed for image restoration. Our prompts elicit the strong restoration potential of the pre-trained WAN model. AAU: AAURain- Snow [2], NTU: real test set of NTU-Rain [7]. instantiated WAN model for video restoration, we include video clips in the supplementary material. 4.4 Ablations We now ablate key components of our framework. Selecting the start noise level T 0 . T 0 controls the amount of noise in the degraded endpoint of the bridge (Eq. 9). We tune T 0 ∈0.1, 0.2, 0.3, 0.4, 0.5, 0.6 on the most severe degradation in our training set, i.e. low-light (see Sec. 3.3). To make selection robust across distortion and perceptual criteria, we aggregate multiple metrics (PSNR, SSIM, LPIPS, DISTS, CLIPIQA, MUSIQ) into a single score by (i) min-max normalizing each metric across candidate T 0 values and (i) averaging the normalized scores with equal weights (full procedure in the supplementary). As shown in Fig. 7, T 0 = 0.4 achieves the best overall score for both WAN and FLUX. Bridge resolves trajectory mismatch. We compare three training/sampling state constructions for prompt learning: (i) Naive prompt tuning on states an- chored at z deg (Eq. 1), (i) prompt tuning using a DDBM-style bridge [61] (Eqn. 6), and (i) prompt tuning using the monotone noisy-degraded → clean bridge (EBR-style [21]; Eqn. 8/9). Table 4 shows that naive tuning yields low- est performance, while using a DDBM-style bridge improves performance. The 14S. Rajagopalan, V.M. Patel InputNaive trainingDDBM+PromptEBR+Prompt Fig. 6: Although naive prompt training enhances the image, it produces several arti- facts due to trajectory mismatch. DDBM with prompt enhances the image marginally. EBR provides best results. Fig. 7: Plot of normalized scores (Sec. 4.4) for determining the best value of T 0 across different candidates: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6. A higher score is better. Table 4: Quantitative comparison of naive prompt tuning, prompt tuning with DDBM- like [61] bridge and with EBR-like [21] bridge (Ours). Training trajectoryP↑ S↑ L↓ D↓ C↑ M↑ Naive21.17 0.694 0.228 0.183 0.474 49.10 DDBM bridge21.44 0.712 0.200 0.121 0.497 55.64 EBR Bridge 22.49 0.728 0.162 0.108 0.529 60.60 Table 5: Ablation on prompt learning for WAN: Learning prompts in the text-encoder embedding space vs. a residual prompt injected in the attention context space. ConditioningP↑ S↑ L↓ D↓ C↑ M↑ Embedding-space20.30 0.753 0.213 0.151 0.538 64.87 Residual prompt20.86 0.770 0.206 0.152 0.540 65.00 EBR-style bridge performs best. Qualitative comparisons in Fig. 6 show that naive prompt tuning produces undesirable artifacts while the DDBM-style bridge under-corrects. Residual prompt injection at attention. Instead of directly learning the full context in text-encoder output space, we learn a residual prompt added to the null-text context at the attention input, with a learned gate initialized at zero. This design improves performance and reduces the parameter budget compared to directly optimizing text-encoder outputs. On WAN, this approach improves performance over direct context embedding learning (Table 5), while using fewer parameters (e.g., ∼300K vs. ∼900K). We provide more ablations in the supplementary along with a discussion of the limitations of our work. 5 Conclusion In this work, we showed that pre-trained diffusion models possess restoration behavior that can be unlocked by training lightweight prompts. Our key find- ing is that this restoration behavior is largely inaccessible through natural- language prompting or token-space prompt optimization but can be elicited by directly learning context embeddings in the text-encoder output space. We fur- Your Pre-trained Diffusion Model Secretly Knows Restoration15 ther showed that naive prompt learning can be unstable due to a train-test tra- jectory mismatch between the degraded image noising process used for prompt optimization and the states visited during reverse-time sampling. To address this, we trained prompts on bridge-defined intermediate states that align training and inference dynamics, yielding a coherent denoising path from noisy degraded in- puts to clean outputs. Building on these insights, we adapt the FLUX image model and WAN video model using only lightweight learned prompts, without fine-tuning or restoration-specific control modules. Comprehensive evaluations show that our approach delivers strong generalization and competitive perfor- mance to state-of-the-art approaches. Acknowledgments This work is supported by the Intelligence Advanced Research Projects Activ- ity (IARPA) via Department of Interior/ Interior Business Center (DOI/IBC) contract number 140D0423C0076. The U.S. Government is authorized to repro- duce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily rep- resenting the official policies or endorsements, either expressed or implied, of IARPA, DOI/IBC, or the U.S. Government. References 1. Hazerd: an outdoor dataset for dehazing algorithms (2017). https://doi.org/10. 21227/H2001Q, https://dx.doi.org/10.21227/H2001Q 10 2. Bahnsen, C.H., Moeslund, T.B.: Aau rainsnow traffic surveillance dataset (2018). https://doi.org/10.34740/KAGGLE/DSV/105294, https://w.kaggle.com/ dsv/105294 10, 13 3. Brack, M., Friedrich, F., Kornmeier, K., Tsaban, L., Schramowski, P., Kersting, K., Passos, A.L.: Limitless image editing using text-to-image models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 8861–8870 4 4. Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing 27(4), 2049–2062 (2018) 10 5. Cao, M., Wang, X., Qi, Z., Shan, Y., Qie, X., Zheng, Y.: Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In: Pro- ceedings of the IEEE/CVF international conference on computer vision. p. 22560– 22570 (2023) 4 6. Chen, H., Ren, J., Gu, J., Wu, H., Lu, X., Cai, H., Zhu, L.: Snow removal in video: A new dataset and a novel method. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV). p. 13165–13176. IEEE (2023) 10 7. Chen, J., Tan, C.H., Hou, J., Chau, L.P., Li, H.: Robust video content alignment and compensation for rain removal in a cnn framework. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 6286–6295 (2018) 10, 13 16S. Rajagopalan, V.M. Patel 8. Cheng, Z., Zhou, L., Chen, D., Tang, N., Luo, X., Qu, Y.: Unildiff: Unlock- ing the power of diffusion priors for all-in-one image restoration. arXiv preprint arXiv:2507.23685 (2025) 1, 4 9. Chu, Y., Luo, G., Chen, F.: A real haze video database for haze level evalua- tion. In: 2021 13th International Conference on Quality of Multimedia Experi- ence (QoMEX). p. 69–72 (2021). https://doi.org/10.1109/QoMEX51781.2021. 9465461 10 10. Conde, M.V., Geigle, G., Timofte, R.: Instructir: High-quality image restoration following human instructions. In: European Conference on Computer Vision. p. 1–21. Springer (2025) 3 11. Deng, S., Ren, W., Yan, Y., Wang, T., Song, F., Cao, X.: Multi-scale separable network for ultra-high-definition video deblurring. In: the IEEE/CVF International Conference on Computer Vision (ICCV). p. 14030–14039 (2021) 10 12. Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: Unify- ing structure and texture similarity. IEEE transactions on pattern analysis and machine intelligence 44(5), 2567–2581 (2020) 11 13. Fei, B., Lyu, Z., Pan, L., Zhang, J., Yang, W., Luo, T., Zhang, B., Dai, B.: Genera- tive diffusion prior for unified image restoration and enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 9935–9946 (2023) 2, 4 14. Gao, P., Zhuo, L., Liu, C., , Du, R., Luo, X., Qiu, L., Zhang, Y., et al.: Lumina-t2x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers. arXiv preprint arXiv:2405.05945 (2024) 1 15. Guan, Q., Yang, Q., Chen, X., Song, T., Jin, G., Jin, J.: Weatherbench: A real- world benchmark dataset for all-in-one adverse weather image restoration. In: Pro- ceedings of the 33rd ACM International Conference on Multimedia. p. 12607– 12613 (2025) 10, 12 16. Guo, Y., Gao, Y., Lu, Y., Zhu, H., Liu, R.W., He, S.: Onerestore: A universal restoration framework for composite degradation. In: European conference on com- puter vision. p. 255–272. Springer (2024) 10 17. Guo, Y., Xiao, X., Chang, Y., Deng, S., Yan, L.: From sky to the ground: A large- scale benchmark and simple baseline towards real rain removal. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 12097– 12107 (October 2023) 10 18. Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626 (2022) 2, 4 19. Hu, J., Jin, L., Yao, Z., Lu, Y.: Universal image restoration pre-training via degra- dation classification. arXiv preprint arXiv:2501.15510 (2025) 11 20. Jiang, Y., Zhang, Z., Xue, T., Gu, J.: Autodir: Automatic all-in-one image restora- tion with latent diffusion. arXiv preprint arXiv:2310.10123 (2023) 1, 4, 11 21. Jinhui, H., Zhu, Z., Hou, J.: Consistency geodesic bridge: Image restoration with pretrained diffusion models. In: The Fourteenth International Conference on Learn- ing Representations 5, 8, 13, 14 22. Ju, X., Zeng, A., Bian, Y., Liu, S., Xu, Q.: Pnp inversion: Boosting diffusion-based editing with 3 lines of code. In: The Twelfth International Conference on Learning Representations (2023) 4 23. Ke, J., Wang, Q., Wang, Y., Milanfar, P., Yang, F.: Musiq: Multi-scale image quality transformer. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 5148–5157 (2021) 11 Your Pre-trained Diffusion Model Secretly Knows Restoration17 24. Labs, B.F.: Flux. https://github.com/black-forest-labs/flux (2024) 1, 2, 9 25. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing 28(1), 492–505 (2019). https://doi.org/10.1109/TIP.2018.2867951 10 26. Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.M., Gu, J., Loy, C.C.: Low-light image and video enhancement using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021) 10 27. Li, H., Chen, X., Dong, J., Tang, J., Pan, J.: Foundir: Unleashing million-scale training data to advance foundation models for image restoration. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 12626–12636 (2025) 3, 10, 11 28. Li, R., Tan, R.T., Cheong, L.F.: All in one bad weather removal using architectural search. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR). p. 3172–3182 (2020). https://doi.org/10.1109/CVPR42600. 2020.00324 3 29. Lin, W., Wei, X., Zhang, R., Zhuo, L., Zhao, S., Huang, S., Teng, H., Xie, J., Qiao, Y., Gao, P., et al.: Pixwizard: Versatile image-to-image visual assistant with open-language instructions. arXiv preprint arXiv:2409.15278 (2024) 1, 4, 9, 11 30. Liu, T., Xu, M., Wang, Z.: Removing rain in videos: a large-scale database and a two-stream convlstm approach. In: 2019 IEEE International Conference on Multi- media and Expo (ICME). p. 664–669. IEEE (2019) 10 31. Liu, Y., Ke, Z., Liu, F., Zhao, N., Lau, R.W.: Diff-plugin: Revitalizing details for diffusion-based low-level tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 4197–4208 (2024) 1, 4 32. Liu, Y.F., Jaw, D.W., Huang, S.C., Hwang, J.N.: Desnownet: Context-aware deep network for snow removal. IEEE Transactions on Image Processing 27(6), 3064– 3073 (2018). https://doi.org/10.1109/TIP.2018.2806202 10 33. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017) 9 34. Mandal, D., Chattopadhyay, S., Tong, G., Chakravarthula, P.: Unicorn: Latent diffusion-based unified controllable image restoration network across multiple degradations. arXiv preprint arXiv:2503.15868 (2025) 2, 4 35. Martin, S., Gagneux, A., Hagemann, P., Steidl, G.: Pnp-flow: Plug-and-play image restoration with flow matching. arXiv preprint arXiv:2410.02423 (2024) 2, 4 36. Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.Y., Ermon, S.: Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073 (2021) 2, 4 37. Mokady, R., Hertz, A., Aberman, K., Pritch, Y., Cohen-Or, D.: Null-text inver- sion for editing real images using guided diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 6038–6047 (2023) 2, 4 38. Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 3883–3891 (2017) 10 39. Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023) 1 40. Potlapalli, V., Zamir, S.W., Khan, S.H., Shahbaz Khan, F.: Promptir: Prompt- ing for all-in-one image restoration. Advances in Neural Information Processing Systems 36 (2024) 3 18S. Rajagopalan, V.M. Patel 41. Rajagopalan, S., Nair, N.G., Paranjape, J.N., Patel, V.M.: Gendeg: Diffusion-based degradation synthesis for generalizable all-in-one image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 28144–28154 (2025) 3, 10 42. Rajagopalan, S., Narayan, K., Patel, V.M.: Restorevar: Visual autoregressive gen- eration for all-in-one image restoration. arXiv preprint arXiv:2505.18047 (2025) 1, 4, 11 43. Rajagopalan, S., Patel, V.M.: Awracle: All-weather image restoration using visual in-context learning. arXiv preprint arXiv:2409.00263 (2024) 3 44. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 10684–10695 (2022) 1, 4 45. Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020) 8 46. Tian, X., Liao, X., Liu, X., Li, M., Ren, C.: Degradation-aware feature perturba- tion for all-in-one image restoration. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 28165–28175 (2025) 3, 9, 11 47. Tumanyan, N., Geyer, M., Bagon, S., Dekel, T.: Plug-and-play diffusion features for text-driven image-to-image translation. In: Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition. p. 1921–1930 (2023) 4 48. Valanarasu, J.J., Yasarla, R., Patel, V.M.: Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 2343–2353 (2022) 3 49. Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025) 2, 9 50. Wang, J., Chan, K.C., Loy, C.C.: Exploring clip for assessing the look and feel of images. In: Proceedings of the AAAI conference on artificial intelligence. vol. 37, p. 2555–2563 (2023) 11 51. Wang, R., Xu, X., Fu, C.W., Lu, J., Yu, B., Jia, J.: Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 9700–9709 (2021) 10 52. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018) 10 53. Wu, H., Zhang, E., Liao, L., Chen, C., Hou, J., Wang, A., Sun, W., Yan, Q., Lin, W.: Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 20144–20154 (2023) 11 54. Yang, Y., Aviles-Rivero, A.I., Fu, H., Liu, Y., Wang, W., Zhu, L.: Video adverse- weather-component suppression network via weather messenger and adversarial backpropagation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 13200–13210 (2023) 3, 11 55. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., Shao, L.: Multi-stage progressive image restoration. In: CVPR (2021) 10 56. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 586–595 (2018) 11 Your Pre-trained Diffusion Model Secretly Knows Restoration19 57. Zhang, X., Dong, H., Pan, J., Zhu, C., Tai, Y., Wang, C., Li, J., Huang, F., Wang, F.: Learning to restore hazy video: A new real-world dataset and a new method. In: CVPR. p. 9239–9248 (2021) 10 58. Zhang, Y., Zhang, H., Chai, X., Cheng, Z., Xie, R., Song, L., Zhang, W.: Diff- restorer: Unleashing visual prompts for diffusion-based universal image restoration. IEEE Transactions on Circuits and Systems for Video Technology (2025) 2, 4 59. Zhao, H., Tian, L., Xiao, X., Hu, P., Gou, Y., Peng, X.: Avernet: All-in-one video restoration for time-varying unknown degradations. Advances in Neural Informa- tion Processing Systems 37, 127296–127316 (2024) 3, 11 60. Zheng, K., He, G., Chen, J., Bao, F., Zhu, J.: Diffusion bridge implicit models. arXiv preprint arXiv:2405.15885 (2024) 8 61. Zhou, L., Lou, A., Khanna, S., Ermon, S.: Denoising diffusion bridge models. arXiv preprint arXiv:2309.16948 (2023) 5, 7, 8, 13, 14 62. Zhou, S., Li, C., Change Loy, C.: Lednet: Joint low-light enhancement and de- blurring in the dark. In: European conference on computer vision. p. 573–589. Springer (2022) 10 63. Zhou, Y., Ren, D., Emerton, N., Lim, S., Large, T.: Image restoration for under- display camera. In: Proceedings of the ieee/cvf conference on computer vision and pattern recognition. p. 9179–9188 (2021) 10 64. Zhuo, L., Du, R., Xiao, H., Li, Y., Liu, D., Huang, R., Liu, W., Zhu, X., Wang, F.Y., Ma, Z., et al.: Lumina-next: Making lumina-t2x stronger and faster with next-dit. Advances in Neural Information Processing Systems 37, 131278–131315 (2024) 1