Paper deep dive
Energy-Conditioned Noise Schedule and Whitening for Spectral Diffusion
Bata Vasic, Bane Vasic
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 10/7/2026, 8:52:46 AM
Summary
This paper introduces Energy-Conditioned Spectral Diffusion (ECSdiff), a method for transform-domain diffusion models that adapts the forward noise schedule based on the spectral energy distribution of the input image. By combining global spectral whitening with energy-conditioned noise allocation, the method assigns smaller perturbations to dominant low-frequency components and larger perturbations to high-frequency components. Experiments on CIFAR-10 demonstrate that ECSdiff reduces the Fréchet Inception Distance (FID) from 142.48 to 100.45 compared to a baseline DCTdiff U-Net variant, while maintaining compatibility with standard DDPM and DDIM sampling procedures.
Entities (12)
Relation Signals (8)
Bane Vasic → affiliatedwith → University of Arizona
confidence 99% · Bane Vasic ... organization=University of Arizona
Bata Vasic → affiliatedwith → University of Nis
confidence 99% · Bata Vasic ... organization=University of Nis
ECSdiff → evaluatedon → CIFAR-10
confidence 98% · Experiments on CIFAR-10 demonstrate the contribution of the proposed energy-conditioned noise schedule
ECSdiff → improves → Fréchet inception distance
confidence 95% · reducing Fréchet Inception Distance from 142.48 for a compact DCTdiff U-Net variant to 100.45.
ECSdiff → iscompatiblewith → DDIM
confidence 95% · remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture.
ECSdiff → iscompatiblewith → DDPM
confidence 95% · remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture.
ECSdiff → uses → U-Net
confidence 92% · The present work uses the resulting spectral representation as the input to a conventional U-Net diffusion model.
ECSdiff → buildsupon → DCTdiff
confidence 90% · ECSdiff introduces a spectral diffusion strategy that advances the DCTdiff philosophy [8] while retaining its baseline framework
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper introduces an energy-adaptive noise scheduling and whitening strategy for transform-domain diffusion models. Existing spectral diffusion methods account for the non-uniform statistics of transform coefficients through coefficient scaling, normalization, or frequency prioritization, while the forward diffusion noise schedule remains largely independent of the underlying spectral-energy distribution. We investigate whether the temporal evolution of the forward diffusion process should also follow the spectral organization of natural images. The proposed formulation combines global spectral whitening with energy-conditioned noise allocation that jointly modulates the injected noise according to the energy of individual transform coefficients and an image-dependent energy path over diffusion time. The resulting forward process preserves Gaussian transitions with closed-form marginals and remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture. Experiments on CIFAR-10 demonstrate the contribution of the proposed energy-conditioned noise schedule and spectral whitening, reducing Fréchet Inception Distance from 142.48 for a compact DCTdiff U-Net variant to 100.45.
Tags
Links
- Source: https://arxiv.org/abs/2610.07206v1
- Canonical: https://arxiv.org/abs/2610.07206v1
Trouble viewing inline? Open PDF directly →
Full Text
35,816 characters extracted from source content.
Expand or collapse full text
[bioid=1, orcid = 0000-0001-8667-1507] [bioid=2, orcid = 0000-0003-2365-4106] Energy-Conditioned Noise Schedule and Whitening for Spectral Diffusion Bata Vasic bata.vasic@elfak.ni.ac.rs organization=University of Nis, Faculty of Electronic Engineering, addressline=Aleksandra Medvedeva 4, city=Nis, postcode=18104, state=, country=Serbia organization=University MB, Information Technology department, addressline=Teodora Drajzera 27, city=Belgrade, postcode=11040, state=, country=Serbia Bane Vasic vasic@ece.arizona.edu organization=University of Arizona, Electrical & Computer Engineering (ECE) department, addressline=1230 E. Speedway Blvd., city=Tucson, postcode=85721-0104, state=AZ, country=USA Abstract This paper introduces an energy-adaptive noise scheduling and whitening strategy for transform-domain diffusion models. Existing spectral diffusion methods account for the non-uniform statistics of transform coefficients through coefficient scaling, normalization, or frequency prioritization, while the forward diffusion noise schedule remains largely independent of the underlying spectral-energy distribution. We investigate whether the temporal evolution of the forward diffusion process should also follow the spectral organization of natural images. The proposed formulation combines global spectral whitening with energy-conditioned noise allocation that jointly modulates the injected noise according to the energy of individual transform coefficients and an image-dependent energy path over diffusion time. The resulting forward process preserves Gaussian transitions with closed-form marginals and remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture. Experiments on CIFAR-10 demonstrate the contribution of the proposed energy-conditioned noise schedule and spectral whitening, reducing Fréchet Inception Distance from 142.48 for a compact DCTdiff U-Net variant to 100.45. keywords Diffusion models,Spectral diffusion,Energy-conditioned noise scheduling,Transform-domain diffusion,Spectral whitening †titlenote: Bane Vasic acknowledges the support of the National Science Foundation under grant CCF-2100013, and a generous gift from his friends and Maecenases Dora and Barry Bursey.†credit: Conceptualization of this study, Methodology, Software†credit: Methodology, Supervision†corresponding: Corresponding author 1 Introduction Denoising diffusion models [11, 6, 12] formulate image generation as the reverse of a stochastic process that progressively transforms data into Gaussian noise. Denoising Diffusion Probabilistic Models (DDPM) [6] and Denoising Diffusion Implicit Models (DDIM) [12] provide stochastic and deterministic realizations of this principle, respectively, while continuous-time formulations have been investigated in [13]. Most existing diffusion models operate directly on pixel representations, where the forward diffusion process applies an identical noise schedule to every image coordinate. Recently, several studies have investigated diffusion in transform-domain representations, including the Fourier [2, 3] and discrete cosine transform (DCT) [1] domains [8, 4]. Unlike pixel representations, transform coefficients possess well-defined spectral meaning and exhibit highly non-uniform statistical properties. In natural images, most of the signal energy is concentrated in a relatively small number of low-frequency DCT coefficients, whereas higher-frequency components contain substantially less energy. Consequently, the transform domain is characterized by a pronounced imbalance between spectral components. Recent transform-domain diffusion models explicitly account for this imbalance. DCTDiff [8] introduces coefficient scaling, signal-to-noise ratio (SNR) scaling, and frequency prioritization to improve optimization in the DCT domain. Moreover, the authors show that image diffusion exhibits a spectral autoregressive behavior, indicating that transform coefficients contribute unequally throughout the diffusion process. These observations suggest that spectral diffusion fundamentally differs from pixel-domain diffusion and should exploit the underlying spectral organization of natural images. Motivated by these observations, we investigate a complementary question that, to the best of our knowledge, has not been addressed previously: should the forward diffusion noise schedule itself depend on the spectral-energy distribution? Existing transform-domain diffusion methods adapt the representation, coefficient scaling, or optimization strategy, whereas the temporal evolution of the forward diffusion process remains largely independent of the spectral energy associated with individual transform coefficients. This paper introduces an energy-conditioned noise scheduling and whitening strategy for transform-domain diffusion models. Rather than proposing a new diffusion architecture, the objective is to investigate spectral-energy-aware noise scheduling as an independent component of existing transform-domain diffusion frameworks. The proposed schedule depends on diffusion time, coefficient energy, and image-dependent spectral energy, assigning smaller perturbations to dominant spectral components while modulating their temporal evolution according to the spectral-energy distribution of each image. The resulting formulation preserves the standard Gaussian diffusion framework and remains directly compatible with conventional DDPM and DDIM training and sampling procedures. To isolate the contribution of the proposed scheduling strategy, the experimental evaluation employs the same U-Net architecture and identical training configuration with both uniform and energy-conditioned spectral noise schedules. The objective is not to compare different diffusion architectures, but to evaluate the influence of energy-conditioned scheduling within the same transform-domain diffusion framework. The proposed formulation is presented in Section 3, its relation to DCTdiff is discussed in Section 4, and the experimental evaluation is given in Section 5. 2 Baseline Spectral Diffusion Framework This section introduces the baseline transform-domain diffusion framework adopted throughout this work. Following the JPEG coding pipeline [16] and the block-DCT representation introduced in DCTdiff [8], the image is converted into compact spectral coefficients before diffusion. Unlike DCTdiff, which employs a Transformer backbone, the present work uses the resulting spectral representation as the input to a conventional U-Net diffusion model. This baseline serves as the reference framework upon which the proposed Energy-Conditioned Spectral Diffusion (ECSdiff) is introduced in the next section. 2.1 Block-DCT Representation Let x∈ℝH×W×3x ^H× W× 3 denote an RGB image. Following the JPEG image coding pipeline, the image is first transformed into the luminance–chrominance color space, x′=x+x =Mx+b, where M is the RGB-to-YCbCr transformation matrix and b is the corresponding offset vector. Since the human visual system is less sensitive to chrominance than luminance, 2×2× chroma subsampling is applied to the CbC_b and CrC_r channels, yielding one luminance channel and two subsampled chrominance channels. Each channel is subsequently partitioned into non-overlapping B×B× B blocks. Let A∈ℝB×BA ^B× B denote one image block and let D(u,v)D(u,v) denote its spectral coefficient at spatial frequency (u,v)(u,v). The two-dimensional DCT [1] is D(u,v)=∑x=0B−1∑y=0B−1A(x,y)ϕu(x)ϕv(y),D(u,v)= _x=0^B-1 _y=0^B-1A(x,y)\, _u(x)\, _v(y), (1) where ϕu(x)=α(u)cos[π(2x+1)u/(2B)] _u(x)=α(u) [π(2x+1)u/(2B)], ϕv(y)=α(v)cos[π(2y+1)v/(2B)] _v(y)=α(v) [π(2y+1)v/(2B)], and α(q)=1/Bα(q)= 1/B for q=0q=0 and α(q)=2/Bα(q)= 2/B for q>0q>0. 2.2 Compact Spectral Representation The coefficients of each DCT block are reordered according to the JPEG zig-zag scan, producing a one-dimensional sequence arranged from low to high spatial frequencies. Four neighboring luminance blocks together with the corresponding chrominance blocks are grouped into a single 4Y+1Cb+1Cr4Y+1Cb+1Cr spectral unit, preserving the spatial correspondence between luminance and chrominance information. Since high-frequency coefficients contribute little to the reconstructed image quality, the last m∗m^* coefficients are removed from every zig-zag sequence following [8], and determined as m∗=argmaxmm:FID(Pdata,PDCT(m))<η,m^*= _m \m:FID(P_data,P_DCT(m))<η \, (2) where PDCT(m)P_DCT(m) denotes the image distribution reconstructed after removing the last m high-frequency coefficients, η is a predefined threshold, and FID(⋅,⋅)FID(·,·) denotes the Fréchet Inception Distance (FID) [5]. Let 0x_0 denote the resulting compact spectral representation, containing K=B2−m∗K=B^2-m^* retained DCT coefficients per block, used throughout the remainder of this work. 2.3 Baseline Spectral Diffusion The compact spectral representation 0x_0 is modeled using the Markov process [9] and standard DDPM formulation with a uniform forward noise schedule [6], q(t|0)=(α¯t0,(1−α¯t)),q(x_t|x_0)=N ( α_t\,x_0,(1- α_t)I ), (3) where α¯t=∏i=1t(1−βi) α_t= _i=1^t(1- _i). A denoising network εθ(t,t) _θ(x_t,t) is trained by minimizing ℒ=0,ε,t[‖ε−εθ(t,t)‖22].L=E_x_0, ,t [ \| - _θ(x_t,t) \|_2^2 ]. (4) 2.4 Reverse Sampling Sampling starts from T∼(,)x_T (0,I) and follows the standard DDPM or DDIM reverse process. After denoising, the retained coefficients are restored to the block-DCT representation, the discarded high-frequency coefficients are set to zero, and the inverse block DCT reconstructs the YCbCr image, which is finally converted back to the RGB color space. 3 Energy-Conditioned Spectral Diffusion The baseline framework described in the previous section follows the standard DDPM formulation with a uniform forward noise schedule. Recent theoretical analysis of transform-domain diffusion has shown that the diffusion process inherently exhibits spectral autoregressive behavior, whereby high-frequency components progressively lose their signal-to-noise ratio earlier than low-frequency components under isotropic Gaussian perturbations [8]. This behavior follows directly from the power-law energy profile of natural images and provides a theoretical justification for treating individual spectral components differently during diffusion. Rather than modifying the diffusion architecture, the proposed ECSdiff explicitly incorporates this observation into the forward process by introducing an energy-conditioned noise schedule. The proposed formulation consists of three consecutive stages. First, the compact spectral representation is whitened to reduce the statistical imbalance between spectral components. Second, the spectral-energy distribution is estimated from the original transform coefficients. Finally, the forward noise schedule is modulated according to the estimated spectral energy. 3.1 Spectral Whitening Let 0=[x1,…,xK]⊤x_0=[x_1,…,x_K] denote the compact spectral representation obtained in Section 2. Although the Block-DCT representation reduces the statistical mismatch between the pixel and transform domains, the retained coefficients still exhibit differences in variance across spectral indices. To reduce this imbalance, each coefficient is normalized independently according to x~k=ωkxksk,sk=[xk2]+δs, x_k= _k x_ks_k, s_k= E[x_k^2]+ _s, (5) where [⋅]E[·] denotes the empirical expectation over the training set, δs>0 _s>0 prevents division by zero, and ωk _k is a frequency-dependent weighting factor. This dataset-level normalization treats sk2s_k^2 as the ensemble energy scale of the k-th spectral component, equalizing systematic differences between spectral modes while preserving image-specific energy variations [14, 7]. The weighting factor combines a diffusion-step adaptation and an epoch-dependent adaptation, ωk=ωt(k)ωeλe(k) _k= _t(k) _e _e(k), where λe≥0 _e≥ 0 controls the contribution of the epoch-adaptive term. For a spectral component located at frequency (u,v)(u,v), the diffusion-step weighting is defined as ωt(u,v)=ωlo(t)+(ωhi(t)−ωlo(t))r(u,v), _t(u,v)= _lo(t)+ ( _hi(t)- _lo(t) )r(u,v), (6) r(u,v)=u2+v2/(B−1)2+(B−1)2. r(u,v)= u^2+v^2/ (B-1)^2+(B-1)^2. where r(u,v)r(u,v) is the normalized radial frequency. The functions ωlo(t) _lo(t) and ωhi(t) _hi(t) are defined as ωlo(t)=ωlo(0)+(ωlo(1)−ωlo(0))τt _lo(t)= _lo^(0)+( _lo^(1)- _lo^(0)) _t and ωhi(t)=ωhi(0)+(ωhi(1)−ωhi(0))τt _hi(t)= _hi^(0)+( _hi^(1)- _hi^(0)) _t, with τt=(t/(T−1))λt _t=(t/(T-1)) _t, where λt>0 _t>0 controls the temporal progression. The epoch-adaptive weighting follows the same formulation with τe=(e/(Ne−1))pe _e=(e/(N_e-1))^p_e, where e and NeN_e denote the current and total number of training epochs, respectively. The resulting normalized coefficients constitute the representation ~(i)=[x~1(i),…,x~K(i)]⊤ x^(i)=[ x_1^(i),…, x_K^(i)] , for i=1,…,Ni=1,…,N, used as input to the proposed diffusion model. 3.2 Spectral Energy Estimation The energy-conditioned schedule is determined from the original spectral coefficients rather than from the whitened representation. Let Ek(0)=[xk2]E_k(0)=E[x_k^2] denote the empirical energy of the k-th spectral coefficient over the training set. After logarithmic energy compression, the corresponding initial normalized spectral-energy distribution is defined as ρk(0)=log(1+λEk(0))∑j=1Klog(1+λEj(0))+δ, _k(0)= (1+λ E_k(0) ) _j=1^K (1+λ E_j(0) )+δ, (7) where δ>0δ>0 ensures numerical stability and λ controls the logarithmic compression. The distribution ρk(0) _k(0) characterizes the relative energy of each spectral component and provides the global spectral-energy distribution for constructing the coefficient-dependent noise adaptation. 3.3 Energy-Conditioned Noise Schedule The spectral-energy distribution defined in (7) is used to construct the coefficient-dependent energy-adaptation factor ψk=K(ρk(0)+δ)−γ∑j=1K(ρj(0)+δ)−γ, _k= K ( _k(0)+δ )^-γ _j=1^K ( _j(0)+δ )^-γ, (8) where γ≥0γ≥ 0 controls the strength of energy adaptation. For each training image i, let ρk(i) _k^(i) denote its normalized spectral-energy distribution computed from the original spectral coefficients. The corresponding cumulative spectral-energy distribution is defined as Rk(i)=∑j=1kρj(i)∑j=1Kρj(i)+δE.R_k^(i)= _j=1^k _j^(i) _j=1^K _j^(i)+ _E. (9) The corresponding energy-dependent temporal modulation is initially defined as β~E(i)(t)=β1+(βT−β1)g(i)(t), β_E^(i)(t)= _1+( _T- _1)g^(i)(t), (10) where g(i)(t)g^(i)(t) is the normalized form of −log(1−R(i)(t)+δE)- (1-R^(i)(t)+ _E), with g(i)(1)=0g^(i)(1)=0 and g(i)(T)=1g^(i)(T)=1. The dimensionless temporal modulation factor is then obtained by mean normalization, βE(i)(t)=β~E(i)(t)1T∑s=1Tβ~E(i)(s)+δE. _E^(i)(t)= β_E^(i)(t) 1T _s=1^T β_E^(i)(s)+ _E. (11) The energy-conditioned forward diffusion schedule and the corresponding retention factors are then defined as βk,t(i)=βtβE(i)(t)ψk,αk,t(i)=1−βk,t(i),α¯k,t(i)=∏s=1tαk,s(i). _k,t^(i)= _t _E^(i)(t) _k,\ _k,t^(i)=1- _k,t^(i),\ α_k,t^(i)= _s=1^t _k,s^(i). (12) The normalization of ψk _k preserves its mean value across spectral components. Higher-energy components are perturbed more conservatively, while lower-energy components receive stronger perturbations, with the temporal evolution defined by βt _t and conditioned by the image-dependent factor βE(i)(t) _E^(i)(t). 3.4 Energy-Conditioned Spectral Diffusion and Sampling For a training image i, the forward diffusion process is defined using the image-dependent schedule from (12) as xk,t(i)=α¯k,t(i)x~k(i)+1−α¯k,t(i)εk,εk∼(0,1),x_k,t^(i)= α_k,t^(i)\, x_k^(i)+ 1- α_k,t^(i)\, _k,\ _k (0,1), (13) where x~k(i) x_k^(i) denotes the k-th coefficient of the whitened spectral representation ~(i) x^(i) introduced in (5). The denoising network is trained using the baseline objective (4), with the forward process given by (13). During sampling, an energy-dependent noise path βE(i)(t) _E^(i)(t) is first obtained from a reference image, and the corresponding coefficient-dependent schedule βk,t(i) _k,t^(i) defined above is used throughout the reverse process. For DDIM sampling, xk,t−1(i)=α¯k,t−1(i)x^k,0(i)+1−α¯k,t−1(i)εθ,k(t(i),t),x_k,t-1^(i)= α_k,t-1^(i)\, x_k,0^(i)+ 1- α_k,t-1^(i)\, _θ,k(x_t^(i),t), (14) where x^k,0(i) x_k,0^(i) denotes the estimated clean k-th spectral coefficient of image i. Thus, the proposed ECSdiff preserves the standard DDPM and DDIM sampling formulations while using the image-dependent energy path βE(i)(t) _E^(i)(t) as conditioning for the corresponding coefficient-dependent noise schedule in the reverse process. 4 Relation to DCTdiff Figure 1: Radial DCT energy profile in the YCbCr and RGB domains for the full-spectrum representation. ECSdiff introduces a spectral diffusion strategy that advances the DCTdiff philosophy [8] while retaining its baseline framework defined in Section 2. DCTdiff compensates for spectral imbalance through Entropy-Consistent Scaling (ECS), Entropy-Based Frequency Reweighting (EBFR), and signal-to-noise ratio (SNR) scaling. ECS applies a common factor η to all retained coefficients, xk′=xk/ηx_k =x_k/η, whereas EBFR weights the denoising objective according to the entropy of individual frequency distributions. Since ECS moves low-energy coefficients closer to zero, DCTdiff additionally scales the SNR of the default noise schedule to compensate for their earlier corruption. Our strategy replaces global scaling with component-wise normalization x~k=xk/sk x_k=x_k/s_k (5). The resulting spectral statistics are further incorporated through the factors ψk _k (8) and βE(i)(t) _E^(i)(t) (11), defining the coefficient- and image-dependent schedule βk,t(i)=βtβE(i)(t)ψk _k,t^(i)= _t _E^(i)(t) _k (12). Hence, spectral statistics directly control normalization and noise allocation. This formulation is consistent with the spectral autoregression established by Theorem 5.1 in [8], according to which isotropic diffusion noise first corrupts low-energy high-frequency components and progressively affects lower frequencies. DCTdiff exhibits this ordering implicitly, whereas the proposed schedule explicitly incorporates spectral energy into the forward process through βk,t(i) _k,t^(i). A further distinction arises during sampling. Since the energy-conditioned βk,t(i) _k,t^(i) path is image-dependent, it is retained in the reverse process, allowing sampling to follow the spectral-energy characteristics of the conditioning image and potentially benefit conditioned image generation. Figure 2: Profile of YCbCr spectral coefficients after normalization (5) and spectral weighting (6) in the full-spectrum DCT representation. 5 Experimental Verification and Analysis The experimental analysis considers two spectral diffusion settings: (i) full-spectrum DCT diffusion to examine the limitations of uniform spectral diffusion models (USDM), and (i) compact block-DCT diffusion for evaluating the proposed ECSdiff and its comparison with DCTdiff [8] under a common experimental setting. 5.1 Experimental Setup Experiments are conducted on CIFAR-10 using the standard training and test partitions of respectively 50,00050,000 and 10,00010,000 RGB images with a resolution of 32×3232× 32. For direct comparison, the DCTdiff framework is adapted from its original transformer-based implementation to the same U-Net backbone used by ECSdiff, while preserving its spectral representation and diffusion procedure. This provides a common, lower-capacity setting for evaluating the effect of the proposed energy-conditioned modifications. All models use T=1000T=1000 diffusion steps and are evaluated using both DDPM and DDIM sampling. A Python implementation of ECSdiff is available at [15]. Unless stated otherwise, ECSdiff uses γ=0.2γ=0.2 in (8), and δ=10−5δ=10^-5 and λ=1λ=1 in (7). A moderate γ is used because ψk _k acts multiplicatively with βt _t and the image-dependent βE(i)(t) _E^(i)(t) on an already whitened representation, whereas stronger spectral weighting would amplify the combined global and image-dependent energy conditioning. The spectral weighting (6) is defined by ωlo(0)=1.0 _lo^(0)=1.0, ωlo(1)=0.8 _lo^(1)=0.8, ωhi(0)=0.01 _hi^(0)=0.01, and ωhi(1)=0.6 _hi^(1)=0.6, with pe=1p_e=1, λe=0.08 _e=0.08, and λt=2.0 _t=2.0 over 10001000 training epochs. The learning rate follows lr(e)=lrmin+(lrmax−lrmin)(1−e/Ne)γlrlr(e)=lr_ +(lr_ -lr_ )(1-e/N_e) _lr, with lrmax=4×10−4lr_ =4× 10^-4, lrmin=4×10−6lr_ =4× 10^-6, and γlr=1.0 _lr=1.0. 5.2 Full-Spectrum DCT Diffusion The first experiment evaluates direct full-spectrum DCT diffusion, without the block-DCT organization and coefficient reduction defined in (1) and (2). This setting preserves the full spectral imbalance and therefore provides a demanding case for evaluating the proposed conditioning. The transition from the pixel domain to the full DCT representation produces a highly non-uniform spectral-energy profile dominated by low-frequency coefficients, as shown in Fig. 1. Applying the ECSdiff normalization (5) and spectral weighting (6) to the full-spectrum representation reduces this imbalance and produces a more compact coefficient profile centered around zero, as shown in Fig. 2. 5.3 Compact DCT Spectral Diffusion The second experiment evaluates ECSdiff within the compact block-DCT framework defined in Section 2 and examines the contributions of spectral whitening and energy-conditioned noise scheduling. Figure 3: Example of an image-dependent energy-conditioned noise path β~E(t) β_E(t) before mean normalization to the dimensionless modulation factor βE(t) _E(t), compared with the linear noise schedule βt _t. Figure 3 shows an example of the image-dependent energy-conditioned noise path obtained from a single image. Compared with the linear schedule, β~E(i)(t) β_E^(i)(t) introduces noise more gradually over most of the diffusion process and increases toward βT _T at the final steps. The displayed path is shown before the mean normalization in (11), which produces the dimensionless modulation factor βE(i)(t) _E^(i)(t) used in (12) as conditioning modulation during training and subsequently during sampling. Figure 4: FID evolution of DCTdiff, ECSdiff with ψk _k and ECSdiff during training, evaluated using 10001000 generated samples at each checkpoint. The training-epoch axis is shown on a logarithmic scale. The quantitative performance of ECSdiff is evaluated against the pixel-domain DDPM, full-spectrum USDM, and DCTdiff by monitoring FID at selected training epochs, as reported in Table 1. Table 1: FID evolution during training. Config. 50 100 150 250 350 500 1000 Pixel 107.19 106.81 100.30 98.23 98.41 95.26 86.69 USDM 233.89 226.44 182.36 213.54 166.80 188.30 192.24 DCTdiff 183.93 183.28 183.19 185.76 190.77 186.73 176.69 ECSdiff 171.92 147.61 140.85 138.59 136.46 132.61 136.68 Frechet Inception Distance (FID) values are reported at selected training epochs using 10001000 generated samples and the same U-Net configuration. In the minimal U-Net configuration, the spectral-domain models do not reach the FID of the pixel-domain DDPM. Among the spectral methods, however, ECSdiff consistently achieves the lowest FID and enters a lower-FID regime after approximately 250250 epochs, with smaller improvements thereafter. This indicates that energy-conditioned spectral diffusion can reach a stable performance regime with fewer training epochs, which may reduce computational cost when applied to higher-capacity architectures. The difference in training behavior is further illustrated in Fig. 4. ECSdiff reaches lower FID values early in training and remains below DCTdiff at all evaluated checkpoints, indicating that the improvement is established during the earlier stages of training rather than only after prolonged optimization. To assess the dependence of the FID estimate on the number of generated samples, DCTdiff and ECSdiff models trained for 10001000 epochs were additionally evaluated using 500500, 10001000, 25002500, 50005000, and 10,00010,000 samples. The results are reported in Table 2. ECSdiff consistently achieves lower FID than DCTdiff across all sample sizes. A similar dependence on sample size is expected at the remaining checkpoints, for which 10001000 samples were used to limit computational cost. Table 2: FID evaluation with different numbers of generated samples. Config. 500 1000 2500 5000 10000 DCTdiff 197.84 176.69 155.11 148.84 142.48 ECSdiff 157.78 136.68 114.39 104.58 100.45 FID values for DCTdiff and ECSdiff trained for 10001000 epochs, evaluated using different numbers of generated samples under the same U-Net configuration and the same Gaussian noise seed for sampling. 5.4 Ablation Analysis The ablation analysis isolates the contribution of spectral whitening and energy-conditioned noise scheduling within the compact spectral framework. Starting from USDM, the EAFW configuration introduces dataset-level spectral whitening, while ECSdiff additionally incorporates the image-dependent energy-conditioned noise schedule. At 10001000 epochs, USDM achieves an FID of 192.2413192.2413, while DCTdiff achieves 176.6944176.6944. Introducing spectral whitening in EAFW further reduces the FID to 152.7026152.7026, and adding the energy-conditioned noise schedule reduces it to 136.6838136.6838 for ECSdiff. These results support the dataset-level whitening in (5) as a mechanism for reducing systematic differences between spectral modes, while preserving image-specific energy variations used by βE(i)(t) _E^(i)(t). 5.5 Sampling Performance The sampling performance is evaluated using both DDPM and DDIM to examine the visual behavior of the spectral diffusion models after 10001000 training epochs. Figure 5 compares the generated samples under the same compact representation and training configuration. USDM DCTdiff EAFW ECSdiff Figure 5: Sampling comparison after 10001000 training epochs using the same seed from the test set for all methods. From left to right: USDM, DCTdiff, EAFW, and ECSdiff. Upper row: DDPM sampling. Lower row: DDIM sampling (14). Figure 5 shows the sampling behavior of USDM and DCTdiff and isolates the effect of energy-conditioned diffusion by comparing EAFW and ECSdiff. ECSdiff produces coherent object structures under both DDPM and DDIM sampling. Compared with EAFW, the main differences are observed in spectral amplitude and color contrast rather than object structure. The corresponding FID results show improved distributional similarity for ECSdiff, while the visual comparison indicates that the energy-conditioned noise schedule also affects contrast and chrominance. 5.6 Discussion The results distinguish the roles of the energy-conditioned components. Global spectral whitening provides dataset-level normalization while preserving image-dependent energy variations. The effect of ψk _k is primarily learned during training: using the ECSdiff-trained model with the EAFW sampling configuration confirms that it need not be reapplied during sampling. Setting ψk=1 _k=1 improves FID at almost all evaluated checkpoints, reaching 100.45100.45 with 1000010000 samples compared with 106.55106.55 when ψk _k is retained. The variability observed with 10001000 samples (Figure 4) is therefore largely attributed to the limited FID sample size. In contrast, the image-dependent βE(i)(t) _E^(i)(t) path remains relevant for energy-conditioned training and potentially provides a more important conditioning mechanism during sampling. The gradual introduction of higher spectral weights ω over diffusion steps t and training epochs e (6), together with the epoch-dependent learning rate lr(e)lr(e), improves optimization and accelerates convergence during the early training epochs. Future experiments will extend ECSdiff to transformer-based architectures and larger training datasets with higher-resolution images, such as ImageNet at 256×256256× 256 resolution, to examine whether the improvements demonstrated here with the compact DCTdiff architecture are preserved across different spectral diffusion models. Acknowledgments The authors would like to thank OpenAI’s ChatGPT [10] for assistance with language editing, translation, and improving text clarity during manuscript preparation. References [1] N. Ahmed, T. Natarajan, and K. R. Rao (1974) Discrete Cosine Transform. IEEE Transactions on Computers C-23 (011), p. 90–93. External Links: ISSN 0018-9340, Document, Document Cited by: §1, §2.1. [2] J. Crabbé, N. Huynh, J. Stanczuk, and M. V. D. Schaar (2024) Time series diffusion in the frequency domain. In Proceedings of the 41st International Conference on Machine Learning, ICML’24, Vol. 235, Vienna, Austria, p. 9407–9438. External Links: Document Cited by: §1. [3] F. Falck, T. Pandeva, K. Zahirnia, R. Lawrence, R. Turner, E. Meeds, J. Zazo, and S. Karmalkar (2025) A Fourier Space Perspective on Diffusion Models. arXiv. External Links: Document Cited by: §1. [4] X. Gao, Z. Xu, J. Zhao, and J. Liu (2024) Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image Translation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 1824–1832. External Links: Document Cited by: §1. [5] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Vol. 30, Red Hook, NY, USA, p. 6629–6640. External Links: Document Cited by: §2.2. [6] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, Guide Proceedings, Red Hook, NY, USA, p. 6840–6851. External Links: Document Cited by: §1, §2.3. [7] T. Karras, M. Aittala, J. Lehtinen, J. Hellsten, T. Aila, and S. Laine (2024) Analyzing and Improving the Training Dynamics of Diffusion Models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 24174–24184. External Links: Document Cited by: §3.1. [8] M. Ning, M. Li, J. Su, J. Haozhe, L. Liu, M. Benes, W. Chen, A. A. Salah, and I. O. Ertugrul (2025) DCTdiff: intriguing properties of image generative modeling in the DCT space. In Proceedings of the 42nd International Conference on Machine Learning, ICML’25, Vol. 267, Vancouver, Canada, p. 46498–46524. External Links: Document Cited by: §1, §1, §2.2, §2, §3, §4, §4, §5. [9] J. R. Norris (1997) Markov Chains. Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press. External Links: Document Cited by: §2.3. [10] OpenAI (2025) ChatGPT: language model for dialogue and content assistance. Note: Version GPT-4.5 Cited by: Acknowledgments. [11] J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, ICML’15, Vol. 37, Lille, France, p. 2256–2265. External Links: Document Cited by: §1. [12] J. Song, C. Meng, and S. Ermon (2022) Denoising Diffusion Implicit Models. arXiv. External Links: Document Cited by: §1. [13] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-Based Generative Modeling through Stochastic Differential Equations. arXiv. External Links: Document Cited by: §1. [14] J. L. van Hemmen (1980) A generalized equipartition theorem. Physics Letters A 79 (1), p. 25–28. External Links: Document Cited by: §3.1. [15] B. Vasc (2026) Energy-conditioned spectral diffusion. Note: Python softwarehttps://silicon-studio.com/ecsdiff.zip Cited by: §5.1. [16] G.K. Wallace (1992) The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics 38 (1), p. 18–34. External Links: Document Cited by: §2. †biography: Bata Vasic received the B.Sc. and M.Sc. degrees in electrical engineering and telecommunications and the Ph.D. degree in electrical engineering and computer science from the University of Niš, Serbia. He is a Research Professor at the University of Niš and an Associate Professor at MB University, Belgrade, where he teaches courses in computer graphics, artificial intelligence, cryptography, and watermarking. He has served as a Visiting Professor at ENSEA, University of Cergy-Pontoise, France, and leads EU-funded research projects on AI development and applications. His research interests include neural network architectures, generative AI security, authorship protection and provenance in diffusion models, digital watermarking, and 3D mesh geometry. †biography: Bane Vasic received the B.Sc., M.Sc., and Ph.D. degrees from the University of Niš, Serbia. He is a Professor of Electrical and Computer Engineering and Mathematics at the University of Arizona and Director of the Error Correction Laboratory. His research has made significant contributions to error correction, including structured low-density parity-check codes and decoding algorithms for data storage systems. He leads research on quantum error correction within major U.S. Department of Energy, NASA, and National Science Foundation programs. He is an IEEE Fellow, Fulbright Scholar, and da Vinci Fellow, and a co-founder of Codelucida and QEC Labs. His research interests include coding theory, classical and quantum error correction, iterative decoding, and reliable data storage and quantum computing systems.