Paper deep dive
MCLR: Improving Conditional Modeling in Visual Generative Models via Inter-Class Likelihood-Ratio Maximization and Establishing the Equivalence between Classifier-Free Guidance and Alignment Objectives
Xiang Li, Yixuan Jia, Xiao Li, Jeffrey A. Fessler, Rongrong Wang, Qing Qu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/26/2026, 1:32:51 AM
Summary
The paper introduces MCLR (Maximum Inter-Class Likelihood-Ratio), a principled training objective for diffusion models that improves conditional generation by explicitly maximizing inter-class separation. The authors demonstrate that standard diffusion models suffer from insufficient inter-class distinctiveness and prove that classifier-free guidance (CFG) is mathematically equivalent to the optimal solution of a weighted MCLR objective, providing a mechanistic interpretation of CFG as an implicit alignment algorithm.
Entities (6)
Relation Signals (3)
Classifier-Free Guidance โ isequivalentto โ Weighted MCLR objective
confidence 98% ยท we provide a theoretical result showing that the CFG-guided score is exactly the optimal solution to a weighted MCLR objective.
MCLR โ improves โ Conditional Modeling
confidence 95% ยท MCLR: Improving Conditional Modeling in Visual Generative Models
CC-DPO โ isequivalentto โ CCA
confidence 90% ยท establishing a previously unexplored connection between CC-DPO and a recently proposed algorithm, Conditional Contrastive Alignment (CCA)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Diffusion models have achieved state-of-the-art performance in generative modeling, but their success often relies heavily on classifier-free guidance (CFG), an inference-time heuristic that modifies the sampling trajectory. From a theoretical perspective, diffusion models trained with standard denoising score matching (DSM) are expected to recover the target data distribution, raising the question of why inference-time guidance is necessary in practice. In this work, we ask whether the DSM training objective can be modified in a principled manner such that standard reverse-time sampling, without inference-time guidance, yields effects comparable to CFG. We identify insufficient inter-class separation as a key limitation of standard diffusion models. To address this, we propose MCLR, a principled alignment objective that explicitly maximizes inter-class likelihood-ratios during training. Models fine-tuned with MCLR exhibit CFG-like improvements under standard sampling, achieving comparable qualitative and quantitative gains without requiring inference-time guidance. Beyond empirical benefits, we provide a theoretical result showing that the CFG-guided score is exactly the optimal solution to a weighted MCLR objective. This establishes a formal equivalence between classifier-free guidance and alignment-based objectives, offering a mechanistic interpretation of CFG.
Tags
Links
- Source: https://arxiv.org/abs/2603.22364v1
- Canonical: https://arxiv.org/abs/2603.22364v1
Trouble viewing inline? Open PDF directly โ
Full Text
115,590 characters extracted from source content.
Expand or collapse full text
MCLR: Improving Conditional Modeling in Visual Generative Models via Inter-Class Likelihood-Ratio Maximization and Establishing the Equiva- lence between Classifier-Free Guidance and Alignment Objectives Xiang Li 1,โก , Yixuan Jia 1 , Xiao Li 1 , Jeffrey A. Fessler 1 , Rongrong Wang 2 , Qing Qu 1 University of Michigan 1 ยท Michigan State University 2 โก Corresponding author Abstract Diffusion models have achieved state-of-the-art performance in generative modeling, but their success often relies heavily on classifier-free guidance (CFG), an inference-time heuristic that modifies the sampling trajectory. From a theoretical perspective, diffusion models trained with standard denoising score matching (DSM) are expected to recover the target data distribution, raising the question of why inference-time guidance is necessary in practice. In this work, we ask whether the DSM training objective can be modified in a principled manner such that standard reverse-time sampling, without inference-time guidance, yields effects comparable to CFG. We identify insufficient inter-class separation as a key limitation of standard diffusion models. To address this, we propose MCLR, a principled alignment objective that explicitly maximizes inter-class likelihood-ratios during training. Models fine-tuned with MCLR exhibit CFG-like improvements under standard sampling, achieving comparable qualitative and quantitative gains without requiring inference-time guidance. Beyond empirical benefits, we provide a theoretical result showing that the CFG-guided score is exactly the optimal solution to a weighted MCLR objective. This establishes a formal equivalence between classifier-free guidance and alignment-based objectives, offering a mechanistic interpretation of CFG. Keywords: Diffusion Models, Classifier-Free Guidance, Contrastive Fine-Tuning, RL Alignment Algorithms Date: March 25, 2026 Correspondence: forkobe@umich.edu Resources: Code Repository Figure 1: Conceptual Illustration of MCLR. (a) Samples generated from two classes using the same initial noises exhibit high visual similarity despite different conditioning labels, indicating insufficient separation of the learned conditional distributions. (b) MCLR mitigates this issue by encouraging class separation, resulting in generations with more distinct class-specific features. 1 arXiv:2603.22364v1 [cs.LG] 23 Mar 2026 Contents 1 Introduction3 2 Preliminaries4 2.1 Basics of Diffusion Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Evidence Lower Bound for Diffusion Models . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Classifier-Free Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4 Direct Preference Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Method6 3.1 Diffusion Models Lack Class-Specificity . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 Maximum Inter-Class Likelihood-Ratio Training . . . . . . . . . . . . . . . . . . . . . 6 3.3 Theoretical Analysis of MCLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.4 Adapting DPO for Improved Conditional Modeling . . . . . . . . . . . . . . . . . . . 9 3.5 Approximating Log-Likelihood with ELBO . . . . . . . . . . . . . . . . . . . . . . . . 10 4 CFG as an Alignment Algorithm: A Mechanistic Interpretation11 4.1 Formal Equivalence between CFG and Weighted MCLR . . . . . . . . . . . . . . . . . 11 4.2 Understanding CFG-Variants through the Alignment Lens . . . . . . . . . . . . . . . 12 5 Related Work12 5.1 Visual Generation without Guidance. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.2 Inference-Time Alignment via Guidance. . . . . . . . . . . . . . . . . . . . . . . . . . 13 6 Experimental Results13 6.1 Experimental Setups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6.2 Overall Algorithmic Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 7 Discussion and Conclusions17 A Theoretical Analysis of MCLR22 A.1 Main Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.2 Fine-tuning with MCLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 B Theoretical Analysis of C-DPO26 B.1 Basics of DPO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 B.2 Improving Conditional Modeling with C-DPO . . . . . . . . . . . . . . . . . . . . . 27 C Theoretical Analysis of CCA30 D Theoretical Analysis of the Equivalence between CFG and Weighted MCLR31 D.1 Proof of Theorem 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 D.2 Extensions: CFG Variants under the Alignment Framework . . . . . . . . . . . . . . 34 E Practical Implementation Details34 E.1 Approximating Log-Likelihood with ELBO . . . . . . . . . . . . . . . . . . . . . . . . 34 E.2 Building Training Data from a Minibatch . . . . . . . . . . . . . . . . . . . . . . . . . 37 E.3 Overall Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2 E.4 Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 F Additional Experimental Results37 F.1 Progressive Class Separation and the Fidelity-Diversity Trade-off. . . . . . . . . . . . 38 F.2 MCLR Outperforms Training-time Baselines. . . . . . . . . . . . . . . . . . . . . . . . 39 F.3 MCLR Achieves Comparable Performance as CFG. . . . . . . . . . . . . . . . . . . . . 39 G Ablation Study39 G.1 MCLR does not Benefit from DSM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 G.2 Hyperparameter Sensitivity of C-DPO . . . . . . . . . . . . . . . . . . . . . . . . . . 41 G.3 Hyperparameter Sensitivity of CCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 H Discussion on Related Works42 1 Introduction Diffusion models [HJA20; Son+21b; Kar+22; Lip+23] have become the dominant paradigm for high-fidelity generative modeling, enabling state-of-the-art visual generation systems [Rom+22; Sah+22; Ram+22; Ess+24]. These models generate samples by reversing a forward noising process using a learned score function, typically trained via denoising score matching [Vin11]. Although the reverse sampling process is theoretically guaranteed to recover the target dis- tribution [And82], in practice it often yields samples of noticeably inferior quality: conditional generation frequently appears visually incoherent or insufficiently faithful to the intended class or prompt [BN24]. In fact, nearly all high-quality generation by diffusion models rely heavily on classifier-free guidance (CFG) [HS22], an inference modification of the reverse sampling process that injects an additional guidance termโthe difference between conditional and unconditional scores. While CFG substantially improves sample quality, reducing FID scores by up to 75% in well-established works [PX23; Yu+25], its empirical necessity exposes a gap between the theoretical optimality of DSM and its practical behavior. This raises a central question: Can the mechanisms underlying CFG be derived from first principles and integrated into the DSM training objective, such that standard reverse sampling, without inference-time guidance, naturally yields CFG-like behavior? Recent observation [LWQ25] suggest that standard conditional models suffer from insufficient inter-class separation: generated samples are less distinguishable across classes than real data, indicating that class-dependent structures are not fully captured by diffusion models. Motivated by this insight, we propose MCLR, a principled alignment objective that explicitly prompts inter- class separability by Maximizing the inter-Class log-Likelihood Ratio. By encouraging the model to amplify density differences between a target class and other classes, MCLR strengthens class- specific structures in the learned score function. Empirically, models fine-tuned with MCLR exhibit CFG-like improvements under standard sampling, achieving substantial gains in fidelity without inference-time guidance. Beyond the empirical benefits, we provide a theoretical result showing that the CFG-guided score coincides exactly with the optimizer of a weighted MCLR objective. This estabilishes a formal equivalence between classifier-free guidance and alignment-based training objectives, revealing CFG as an implicit inference-time alignment algorithm. 3 The core principle of MCLRโleveraging inter-class contrastive signals to improve conditional modelingโcan also be realized with broader contrastive learning approaches such as Direct Pref- erence Optimization (DPO) [Raf+23]. To assess the uniqueness of MCLR, we adapt DPO to the conditional generation by treating samples from the target classํas preferred and samples from other classes as non-preferred. We refer to this formulation as Conditional Contrastive DPO (C-DPO). We show that C-DPO induces a โgamma-poweredโ distribution [BN24], reshaping the base model via the density ratio ํ(ํ|ํ) ํ(ํ) . Moreover, we establish a previously unexplored con- nection between C-DPO and a recently proposed algorithm, Conditional Contrastive Alignment (CCA) [Che+25b], demonstrating that both objectives admit the same optimal solution. Compre- hensive experiments reveal that MCLR consistently outperforms these alternatives across diverse models and datasets. Summary of Contributions. Our main contributions are as follows: โขA Principled Alignment Objective for Conditional Modeling. We propose MCLR, a theoreti- cally grounded fine-tuning objective that explicitly maximizes inter-class log-likelihood ratios to improve conditional generative modeling. Across diverse models and datasets, MCLR achieves substantial improvements in sample fidelity under standard reverse sampling, consistently out- performing training-time contrastive alternatives such as C-DPO and CCA. โขTheoretical Equivalence between CFG and Alignment Objectives. We prove that the classifier- free guidance (CFG)โinduced score coincides exactly with the optimizer of a weighted MCLR objective. This establishes a formal connection between CFG and alignment-based training, providing a mechanistic interpretation of CFG as an implicit inference-time alignment algorithm. โขUnderstanding Contrastive Alternatives. We provide both theoretical and empirical analyses of contrastive alternatives such as DPO and CCA. When adapted to conditional generation, we show that DPO induces a gamma-powered density transformation equivalent to that of CCAโan equivalence that, to our knowledge, has not been previously established. 2 Preliminaries 2.1 Basics of Diffusion Models Letํ data (ํ)denote the ground-truth data distribution. Diffusion models construct a forward noising process that gradually perturbsํ data into a simple prior distribution using a stochastic differential equation (SDE): ํํ = ํ(ํ,ํก)ํํก+ ํ(ํก)ํํ,(1) whereํ(ยท,ํก)is the drift coefficient,ํ(ํก)is the diffusion coefficient, andํdenotes the standard Brownian motion. Letํ ํก (ํ)be the marginal distribution ofํ(ํก), andํ 0ํก (ํ ํก |ํ)the transition density fromํ(0)toํ(ํก). For sufficiently largeํ, the distributionํ ํ (ํ)becomes indistinguishable from a tractable priorํ(ํ), e.g., an isotropic Gaussian. The SDE(1)admits a reverse-time probability-flow ODE [Son+21b]: ํํ = ํ(ํ,ํก)โ 1 2 ํ 2 (ํก)โ ํ log ํ ํก (ํ) ํํก.(2) 4 Sampling from the reverse ODE requires access to the score functionโ ํ log ํ ํก (ํ), which can be approximated using a deep networkํ ํฝ (ํ,ํก)trained via the denoising score matching (DSM) objective: ํฅ DSM (ํฝ;ํค(ยท)) := 1 2 โซ ํ 0 E ํ(ํ), ํ 0ํก (ํ ํก |ํ) h ํค(ํก) โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก) 2 2 i ํํก,(3) whereํค(ํก)is a positive weighting function. For conditional diffusion models, the score network takes a conditional embeddingํas input, and the DSM objective naturally extends to the conditional setting by taking the expectation over class labels and class-conditional data distributions: ํฅ DSM (ํฝ, ํ;ํค(ยท)) := 1 2 โซ ํ 0 E ํ,ํ(ํ|ํ),ํ 0ํก (ํ ํก |ํ) [ํค(ํก)โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 ]ํํก.(4) In this work, we focus on conditional diffusion models, for which the reverse ODE in(2)becomes: ํํ = ํ(ํ,ํก)โ 1 2 ํ 2 (ํก)โ ํ log ํ ํก (ํ|ํ) ํํก.(5) 2.2 Evidence Lower Bound for Diffusion Models Letํ ode ํฝ (ํ)denote the distributions induced by the reverse ODE(2). Theorem 2 of [Son+21a] shows that, under certain regularity conditions, the log-likelihood satisfies: E ํ(ํ) [log ํ ode ํฝ (ํ)] | z Maximum Likelihood Estimation =โํฅ DSM (ํฝ; ํ 2 (ยท))+ ํถ,(6) whereํถis a constant independent ofํฝ. Equation (6) is analogous to the evidence lower bound (ELBO) in variational autoencoders, revealing a fundamental connection between DSM and maxi- mum likelihood estimation (MLE). This connection, enables likelihood-based training in diffusion models [Mar+23; Wal+24; Zhe+25]. 2.3 Classifier-Free Guidance Although the reverse ODE in(5)is theoretically guaranteed to sample from the target conditional distribution, its practical generation quality is often unsatisfactory. In practice, high-quality con- ditional generation requires modifying the standard reverse process with an additional guidance term, known as classifier-free guidance (CFG) [HS22], which leads to the perturbed reverse ODE: ํํ = ํ(ํ,ํก)โ 1 2 ํ 2 (ํก)(โ ํ log ํ ํก (ํ|ํ)+ ํพ(โ ํ log ํ ํก (ํ|ํ)โโ ํ log ํ ํก (ํ) | z CFG guidance ) ํํก,(7) whereํพcontrols the guidance strength. Intuitively, CFG sharpens the class- or conditional-specific structure by amplifying the difference between conditional and unconditional scores. However, the CFG-perturbed reverse sampling process(7)does not, in general, correspond to any known forward process [BN24]. Despite recent progress towards understanding CFG [Wu+24; Chi+24; BN24; Pav+25; LWQ25; LJ25; JSG25; Yan+26; Ven+26], its underlying mechanisms remain only partially understood. This post-hoc modification of the sampling procedure motivates the search for principled alternatives that reproduce CFG-like improvements while preserving theoretical consistency, which may, in turn, shed light on the mechanisms underlying CFG itself. 5 2.4 Direct Preference Optimization Direct preference optimization (DPO) is a widely used approach for aligning pretrained language or diffusion models with human preferences [Raf+23; Wal+24]. Under the Bradley-Terry (BT) model [BT52], the probability that a sample ํ ํค is preferred over ํ ํ given a condition ํ is: ํ(ํ ํค โป ํ ํ |ํ) = Sigmoid(ํ(ํ ํค |ํ)โ ํ(ํ ํ |ํ)),(8) whereSigmoid(ยท):=1/(1+ exp(โํ))denotes the Sigmoid function,ํ ํค andํ ํ denote the preferred and non-preferred samples, respectively, andํ(ํ|ํ)represents the underlying reward function re- flecting human preference. DPO parameterizes this reward asํ ํฝ (ํ|ํ):= ํฝ log ํ ํฝ (ํ|ํ)โํฝ log ํ ref (ํ|ํ) and estimatesํฝ via maximum likelihood estimation on the BT model (8): max ํฝ E (ํ,ํ ํค ,ํ ํ )โผํ log ํ(ํ ํค โป ํ ํ |ํ) := E (ํ,ํ ํค ,ํ ํ )โผํ log Sigmoid(ํฝ log ํ ํฝ (ํ ํค |ํ) ํ ref (ํ ํค |ํ) โ ํฝ log ํ ํฝ (ํ ํ |ํ) ํ ref (ํ ํ |ํ) ) , (9) whereํ ref denotes the pretrained base (reference) model,ํฝcontrols the strength of the KL regular- ization betweenํ ํฝ andํ ref , andํrepresents the preference dataset. As we show later, DPO can be naturally adapted to enhance conditional modeling in visual generative models, providing a contrastive mechanism that parallels our MCLR objective. 3 Method In this section, we first demonstrate that diffusion models learn conditional distributions that lack sufficient class distinctiveness ( section 3.1). To remedy this issue, we propose MCLR (sections 3.2, 3.3 and 3.5). For completeness, we also study a contrastive alternative obtained by adapting DPO to conditional generation (section 3.4). 3.1 Diffusion Models Lack Class-Specificity In theory, if the score functions are learned accurately, standard reverse diffusion sampling should produce samples from the target conditional distribution. In practice, however conditional sam- pling often fails to exhibit strong class-specific structure [LWQ25]. A common failure mode is that the conditional generations are weakly distinguishable across classes: when starting from the same initial noises, samples generated under different class conditions frequently share similar global layouts while class-discriminative features are attenuated or missing as shown in Figure 1(a) and Figure 2. This suggests that the learned conditional distributions are insufficiently distinguish- able from one another. We illustrate this phenomenon conceptually in Figure 1, where the learned class-conditional distributions of a trained base modelํ ref exhibit substantially less separation than the ground-truth data distribution ํ data . 3.2 Maximum Inter-Class Likelihood-Ratio Training Motivated by the preceding discussion, we propose improving conditional modeling by explicitly encouraging inter-class separation in conditional models, as shown conceptually in Figure 1(b). Letํ ํฝ (ํ|ํ)denote the modelโs conditional distribution for class conditionํ, whereํฝis the model parameter. We consider the following objective: max ํฝ E ํ,ํ(ํ|ํ) log ํ ํฝ (ํ|ํ)+ ํ 2 E ํ, ฬํ,ํโผํ(ยท|ํ),ํโผํ(ยท| ฬํ) log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) + log ํ ํฝ (ํ| ฬํ) ํ ํฝ (ํ|ํ) ,(10) 6 Class 350 Class 456 Base Model ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ Base Model 28056089612321288 ImageNet 512x512 Base Model 2244486728401008 MCLR Training Iterations (K images) Base Model Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ Class 617 Class 299 ImageNet 64x64 Iters= 0 Iters= 0 ํธ=ํ ํธ=ํ Figure 2: Effects of MCLR vs. CFG. We visualize the progressive emergence of class-specific structure under MCLR training and CFG. MCLR training induces effects analogous to increasing guidance strength in CFG, both substantially improving class-specific patterns. For each image block, samples from different classes are generated from the same initial noises. whereํand ฬํare two randomly sampled classes andํcontrols regularization strength. Compared to standard MLE (equivalently, DSM under appropriate weighting) in(6), eq.(10)explicitly encourages samples to have higher likelihood under their true class than under mismatched classes. Concretely, for a sampleํ โผ ํ ํฝ (ํ|ํ)(orํ โผ ํ ํฝ (ํ| ฬํ)), MCLR explicitly encourages it to have higher likelihood under true class than under mismatched classes. In doing so, MCLR drives the model to increase the inter-class likelihood ratio, thereby pushingํ ํฝ (ํ|ํ)to concentrate more probability mass in regions where the true class is favored over competing classes. Such regions typically correspond to samples with more pronounced class-specific features. Intuitively, MCLR encourages the model to fully exploit the label-conditioned information. Standard conditional model training simply feeds the conditional (class) labelํalongside the data to the model, optimizes the MLE (or DSM) objective, and relies on the deep networks to automatically discover and utilize the conditional structure. When labels are under-exploited, the conditional distributions can collapse and become weakly distinguishable, yieldingํ ํฝ (ํ|ํ)โ ํ ํฝ (ํ| ฬํ) regardless of the true class ofํ. Maximizing the log-likelihood ratiolog ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) directly penalizes this collapse and forces the model to discriminate between different conditions (classes) by leveraging the information encoded in the labels. Note that(10)admits the following equivalent form, which we use throughout unless otherwise stated: max ํฝ E ํ, ํ(ํ|ํ) log ํ ํฝ (ํ|ํ) + ํ E ํ, ฬํ, ํโผํ(ํ|ํ) h log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) i | z MCLR Regularization .(11) It can also be written as: 7 max ํฝ E ํ, ํ(ํ|ํ) log ํ ํฝ (ํ|ํ) + ํ E ํ, ํโผํ(ยท|ํ),ํโผํ(ยท) h log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ|ํ) i | z MCLR Regularization (Form I) ,(12) where ํโผ ํ(ํ) denotes sampling from the unconditional density ํ(ํ) := E ํ [ํ(ํ|ํ)]. Fine-tuning with MCLR. When a pretrained (suboptimal) modelํ ref (ํ)that lacks class specificity is available, we can fine-tune it using MCLR in combination with KL regularization: max ํฝ โE ํ ํท KL ํ ref (ํ|ํ)โฅํ ํฝ (ํ|ํ) + ํE ํ, ฬํ,ํโผํ(ํ|ํ) log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) .(13) In the next subsection, we analyze the optimal solution induced by(11)and(13), which provides insight into MCLRโs effects. 3.3 Theoretical Analysis of MCLR We begin by analyzing the optimization problem(11). Defineโ(ํ|ํ):= ํ(ํ|ํ)+ ํ ํ(ํ|ํ)โ ํ(ํ) . We make the following assumptions. Assumption 1. The function โ(ํ|ํ) has compact support; that is, supp โ(ํ|ํ) โ ํพ, |ํพ| <โ.(14) Assumption 2. Forโํ โ ํพ, ํ ํฝ (ํ|ํ)โฅ ํฟ > 0.(15) Note that Assumption 1 is mild in practice, as image data typically occupies a bounded pixel range. Assumption 2 is a regularity assumption that ensures the log-likelihood is well-defined, avoiding the singularity that occurs when evaluatinglog ํ ํฝ (ํ|ํ)at zero density. Sinceํ ํฝ (ํ|ํ) integrates to one over ํพ, we necessarily have ํฟโค 1 |ํพ| . In the following, we further assume ํฟ < 1 |ํพ| . Theorem 1. Based on the two assumptions above, the optimal solution to (11) is: ํ ํฝ โ (ํ|ํ) = ( max โ(ํ|ํ) ํ(ํ) , ํฟ , ํ โ ํพ, 0, ํ โ ํพ, (16) where ํ(ํ) is the normalizing constant. The proof is provided in section A.1. Intuitively, the optimal conditional distributionํ ํฝ โ (ํ|ํ) induced by MCLR is proportional toโ(ํ|ํ)wheneverโ(ํ|ํ) > ํ(ํ)ํฟ, while being clipped to the floorํฟelsewhere. In the limitํฟโ0, the floor disappears and the optimal distribution approaches: ํ ํฝ โ (ํ|ํ) = โ + (ํ|ํ) โซ ํ โ + (ํ|ํ)ํํ ,(17) whereโ + (ํ|ํ):= max โ(ํ|ํ),0 , such that the negative part ofโ(ํ|ํ)is truncated to zero and then normalized to form a valid distribution. Note that(16)and(17)can be interpreted as the 8 "sum-of-difference" distribution: MCLR reshapesํ(ํ|ํ)by adding to it the difference between ํ(ํ|ํ)andํ(ํ), such that the regions whereํ(ํ|ํ) > ํ(ํ)(i.e., samples with strong class-specific features) are amplified, and the regions whereํ(ํ|ํ) < ํ(ํ)(i.e., ambiguous samples lying near class boundaries) are suppressed. For the fine-tuning objective in(13), it is easy to show that the optimal solution admits the same form as(16), but in this caseโ(ํ|ํ) = ํ ref (ํ|ํ)+ํ(ํ(ํ|ํ)โํ(ํ)). This directly leads to the following corollary (the proof is provided in section A.2). Corollary 1. If the base model ํ ref satisfies the mixture error model: ํ ref (ํ|ํ) =(1โ ํ)ํ(ํ|ํ)+ ํ ํ(ํ),(18) whereํ โ [0,1], then finetuningํ ref (ํ|ํ)with MCLR regularization(13)recovers the ground truth conditional distribution ํ(ํ|ํ). The mixture error model(18)posits that the base model suffers from cross-class "leakage"; specifically, the learned conditional densityํ ref (ํ|ํ)is a convex combination of the ground-truth conditional distributionํ(ํ|ํ)and the unconditional counterpartํ(ํ), which corresponds to a weighted average over all class-conditional distributions. MCLR addresses this leakage by explicitly subtracting a portion of the unconditional component from the base model, thereby suppressing the influence of competing classes. Notably, similar mixture-based error models have been employed in the design of guidance methods [Kou+25]. 3.4 Adapting DPO for Improved Conditional Modeling The core principle behind MCLR is to improve conditional modeling by encouraging the model to exploit class-dependent structures through inter-class contrast. This idea can also be instantiated via other contrastive objectives such as DPO. Intuitively, given a condition (or prompt)ํand a pair consisting of a human-preferred sampleํ ํค and a non-preferred sampleํ ํ , the DPO objective(9) increases the relative density assigned by the fine-tuned modelํ ํฝ (ํ|ํ)toํ ํค , while decreasing its density onํ ํ , relative to the base modelํ ref (ํ|ํ). To adapt DPO for improving class specificity, we treat samples from the targetํas preferred (ํ ํค ) and samples from other randomly selected classes as non-preferred (ํ ํ ). This leads to the following objective: min ํฝ โE ํ, ฬํ, ํ ํค โผํ(ํ|ํ), ํ ํ โผํ(ํ| ฬํ) log Sigmoid ํฝ log ํ ํฝ (ํ ํค |ํ) ํ ref (ํ ํค |ํ) โ ํฝ log ํ ํฝ (ํ ํ |ํ) ํ ref (ํ ํ |ํ) .(19) We refer to(19)as Conditional Contrastive DPO (C-DPO). Similar to MCLR, C-DPO objective also admits a closed-form solution, as stated in the following theorem (the proof is provided in section B.2). Theorem 2. Under certain regularity conditions, the optimal solution to (19) is: ํ ํฝ โ (ํ|ํ) = 1 ฬ ํ(ํ) ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ,(20) where ฬ ํ(ํ) = โซ ํ ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ํํ is the normalizing constant. 9 Comparison between MCLR and C-DPO. Despite sharing the same inter-class contrastive mech- anism, MCLR and C-DPO differ in how they modify the base model. Unlike MCLRโs additive modification via the density differenceํ(ํ|ํ)โ ํ(ํ), C-DPO reweights the base model multi- plicatively by a density ratio term ํ(ํ|ํ) ํ(ํ) 1 ํฝ , thereby amplifying regions whereํ(ํ|ํ) > ํ(ํ)while suppressing regions whereํ(ํ|ํ) < ํ(ํ). This distinction has important theoretical consequences. In particular, the multiplicative form adopted by C-DPO can be overly aggressive. Consider a point ฬํsuch thatํ( ฬํ)โ0 whileํ( ฬํ|ํ) >0, a situation that naturally arises whenํare minority classes. In this case, the ratio ํ( ฬํ|ํ) ํ( ฬํ) 1/ํฝ becomes ill-conditioned, strongly amplifying the density at ฬํand potentially driving the learned conditional distribution toward degenerate or unstable solutions. By contrast, the optimal solution (16) induced by MCLR remains well-behaved. Equivalence between C-DPO and CCA. The optimal C-DPO solution(20)is known as the gamma-powered distribution. This distribution was initially conjectured to characterize the effect of classifier-free guidance (CFG) [HS22], but was later shown not to correspond to the true CFG dynamics [Kar+24a; BN24]. Interestingly, the same gamma-powered distribution also arises as the optimal solution of Conditional Contrastive Alignment (CCA) [Che+25b], a recently proposed method for autoregressive models (see section C). Our analysis therefore establishes a previously unrecognized equivalence between C-DPO and CCA at the level of their induced optimal distribu- tions. Empirically, as we demonstrate in section G.3, C-DPO matches or outperforms CCA while requiring fewer hyperparameters, making it simpler to deploy in practice. 3.5 Approximating Log-Likelihood with ELBO Both MCLR(11)and C-DPO(19)require access to the log-likelihood. While the log-likelihood can be computed exactly for autoregressive models [Tia+24], it is not directly available for diffusion models. We therefore approximate the log-likelihood using ELBO(6), so that the MCLR objective in (11) becomes equivalent to (21): min ํฝ ํฅ DSM (ํฝ, ํ; ํ 2 (ยท)) + ํE ํ, ฬํ,ํกโผํฐ[0,ํ] ํ(ํ|ํ), ํ 0ํก (ํ ํก |ํ) h ํ 2 (ํก) โฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 โโฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i , (21) which can be estimated with Monte Carlo sampling. Although the ELBO holds exactly only under a uniform time schedule and the specific weightingํ 2 (ํก), following standard practice in the diffusion model literature, we treat these terms as tunable design choices. Specifically, we adopt a customized time sampling distribution ํ(ํก) and replace ํ 2 (ํก) with a chosen weighting function ํค(ํก). Interpreting MCLR from Denoising Perspective. Moreover, according to the equivalence between score function and optimal MMSE denoiser, the score network can be parameterized asํ ํฝ (ํ,ํก, ํ) = ํ ํฝ (ํ;ํ(ํก),ํ)โํ ํ 2 (ํก) , whereํ(ํก)is the standard deviation of additive noise at timeํก(see section E.1 for details). As a result, the MCLR regularization with a customized training time schedule and adaptive weighting becomes equivalent to: E ํ, ฬํ,ํกโผํ(ํก) ํ(ํ|ํ), ํ 0ํก (ํ ํก |ํ) h ํค(ํก) โฅํโํ ํฝ (ํ ํก ; ํ(ํก), ํ)โฅ 2 2 โ โฅํโํ ํฝ (ํ ํก ; ํ(ํก), ฬํ)โฅ 2 2 i .(22) 10 Eq.(22)provides an intuitive interpretation of MCLR from a denoising perspective: it trains conditional denoisersํ(ํ;ํ(ํก), ํ)under an additional margin-like constraint. Specifically, for each sampleํ โผ ํ(ํ|ํ), the denoiser associated with the correct conditionํis encouraged to produce a better reconstruction than denoisers corresponding to mismatched conditions ฬํ. In practice, we implement MCLR using this denoising formulation. Remarks on the ELBO Approximation. Despite a standard practice in the literature, when re- placing log-likelihood with ELBO, the resulting objective(21)does not necessarily correspond to the original likelihood formulation(11), since the ELBO does not enforce the regularity conditions required for the parameterized score function to define a valid score field. Consequently, ELBO- approximated MCLR should be viewed as an approximate likelihood-ratio maximization procedure for diffusion models. This issue does not arise for autoregressive models, where likelihoods are available exactly. 4 CFG as an Alignment Algorithm: A Mechanistic Interpretation We now show that classifier-free guidance (CFG) is not merely an inference-time heuristic, but the exact optimal solution of an alignment objective. In particular, we prove that the CFG-guided score in (7) coincides with the unique minimizer of a sample-adaptive weighted MCLR objective. As established in the previous sections, the MCLR formulation in(11)(and its equivalent form(12)) can be interpreted as a contrastive alignment objective, structurally analogous to methods such as DPO. The result below therefore provides a formal characterization of CFG as an inference- time alignment algorithm that approximately optimizes an inter-class likelihood-ratio objective. 4.1 Formal Equivalence between CFG and Weighted MCLR We establish the following equivalence. The proof is provided in section D.1. Theorem 3. For any time sampling distribution ํ(ํก) and weighting function ํค(ํก), the CFG-guided score ํ cfg (ํ ํก ,ํก, ํ) :=โ ํ ํก log ํ ํก (ํ ํก |ํ)+ ํ โ ํ ํก log ํ ํก (ํ ํก |ํ)โโ ํ ํก log ํ ํก (ํ ํก ) is the unique minimizer of a sample-adaptive weighted ELBO-approximated MCLR objective: ํ cfg (ยท) = arg min ํ ํฝ (ยท) E ํ,ํกโผํ(ํก), ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก)โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i + ํE ํ, ฬํ,ํกโผํ(ํก) ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 โ ํ ํก (ํ ํก | ฬํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i . (23) The MCLR regularization term in(23)differs from that of standard MCLR objective(21)in that the negative component is reweighted by a sample-adaptive likelihood ratio ํ ํก (ํ ํก | ฬํ) ํ ํก (ํ ํก ) . This adaptive weighting transforms the standard MCLR into one whose minimizer coincides exactly with the CFG-guided score. Hence, CFG can be interpreted as approximately solving an inter-class likelihood-ratio alignment problem at inference time, where the approximation arises from (i) the ELBO-based training objective and (i) the adaptive weighting. Furthermore, recent empirical works [Fra+25; Jin+25; Che+25c; JWL26] have observed that CFG-style inference-time guidance often produces effects similar to those of training-time alignment 11 methods. The equivalence established above provides a precise theoretical explanation for this phenomenon. Lastly, we note that the MCLR regularization term in (23) also admits an equivalent form: E ํ,ํกโผํ(ํก),ํโผํ(ยท|ํ),ํโผํ(ยท) ํ 0ํก (ํ ํก |ํ),ํ 0ํก (ํ ํก |ํ) h ํค(ํก) โฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 โ ํ ํก (ํ ํก |ํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i . (24) 4.2 Understanding CFG-Variants through the Alignment Lens The equivalence above provides a unified perspective for interpreting CFG variants. By modifying the source distributions from which positive and negative samples are drawn in the contrastive objective, one recovers several existing guidance mechanisms as special cases. For example in(24), ifํis sampled from a stronger model andํis sampled from a weaker model, the resulting optimal solution recovers the score of Autoguidance [Kar+24a]. Similarly, ifํ is sampled from a reinforcement-learning fine-tuned model andํis sampled from the base model, the induced solution corresponds to the score used in CFGRL [Fra+25]. Thus, MCLR provides a unified alignment interpretation for a broad class of guidance algorithms. We summarize this unified framework in Figure 3 and defer a detailed generalization and discussion to section D.2. Contrastive Inter-distribution Alignment: Establishing the Equivalence between CFG and MCLR Inference time guidance-based method (CFG-variant): Amplify Likelihood-ratio at inference ํ !"#$%&'(&)* ํ * ,ํก:=โ ํ ! log ํ * , (ํ * )+ํโ ํ ! log ํ ํ , ํ ํ ํ * $ (ํ * ) Training-time alignment (MCLR): Internalize alignment into training max . ํธ / " (ํ) log ํ . (ํ)+ํํธ ํโผ/ " โ ,ํโผ/ # (โ ) [log ํ ํฝ (ํ) ํ ํฝ (ํ) ] CFG and its variants ํ ! ํํ " ํ CFG Autoguidance CFGRL ํํํ ํ(ํ) RL-fine-tuned model Base model Stronger model Weaker mo del Approximate equivalence: MCLR โguidance-based method under (i)adaptive weighting (i)ELBO approximation Figure 3: A Unified Framework Connecting CFG Variants with Contrastive Alignment. CFG- based methods can be interpreted as implicitly optimizing a likelihood-ratioโbased contrastive alignment objective between two distributions ํ + (ํ) and ํ โ (ํ) at inference time. 5 Related Work Our work relates to recent advances in conditional generative modeling and guidance in diffusion models. We provide a more comprehensive discussion in section H. 5.1 Visual Generation without Guidance. Several recent works aim to induce CFG-like behavior by modifying the training objective, rather than applying classifier-free guidance (CFG) at inference time [Che+25b; Che+25a; Tan+25]. Among them, Conditional Contrastive Alignment (CCA) [Che+25b] is the most closely related to our work. CCA learns a gamma-powered distribution via Noise Contrastive Estimation [GH10]. In contrast, MCLR provides stronger empirical performance in diffusion models and achieves 12 cometitive performance in autoregressive settings. Moreover, we reveal a previously unexplored theoretical equivalence between DPO and CCA. Another related approach is Guidance-Free Training (GFT) [Che+25a; Tan+25], which aims to reproduce CFG-induced score functions through a modified denoising score matching (DSM) objective. While GFT mimics the functional form of CFG during training, our MCLR formulation instead reveals the underlying contrastive likelihood-ratio structure implicit in CFG, offering a clearer mechanistic interpretation. Several additional works [Yan+24; Lee+25; KMS24; YAA25] employ class-wise contrastive objectives to improve conditional generation. However, these methods mainly focus on empirical improvements and lack a formal characterization of the underlying theoretical properties. Finally, Direct Discriminative Optimization (DDO) [Zhe+25] contrasts real and synthetic sam- ples rather than class-conditional distributions, and serves as a baseline in our experiments. Al- though MCLR, as a unified framework (discussed in section D.2), can be extended to real-synthetic contrastive settings, we leave this direction for future investigation. 5.2 Inference-Time Alignment via Guidance. Recent works [Fra+25; Jin+25; Che+25c; JWL26] observe that CFG-style inference-time guidance can produce effects resembling those of training-time alignment methods, including reinforcement learning-based approaches. These studies provide empirical evidence that guidance may implicitly induce alignment-like behavior. However, existing analyses are largely heuristic and often rely on unrealistic assumptions such as the guided score corresponds to that of the gamma-powered distribution(20), an assumption that has been proven incorrect [Kar+24a; BN24]. As a result, a rigorous theoretical connection between CFG and alignment objectives remains incomplete. By establishing the equivalence between CFG and a weighted MCLR objective, our work provides a formal mechanistic interpretation of CFG as an inference-time alignment algorithm. To the best of our knowledge, this is the first result that rigorously connects CFG-style guidance with likelihood- ratio-based alignment objectives. 6 Experimental Results In this section, we empirically evaluate the effectiveness of the proposed method. Our experiments demonstrate that: (i) MCLR substantially improves conditional generation quality and outperforms existing training-time baselines; and (i) MCLR achieves performance comparable to CFG, exhibit- ing a similar fidelityโdiversity trade-off and producing similar qualitative effects. Due to space constraints, we present only a subset of the results here and defer a more comprehensive evaluation to sections F and G. 6.1 Experimental Setups Practical Implementation. In our experiments, we focus on fine-tuning pretrained models. The KL-regularized objective in(13)requires sampling from the base model, which incurs significant computational overhead. Although one can combine denoising score matching (DSM) with MCLR as in(11), we empirically observe no performance gains from this formulation (see section G.1). Therefore, throughout the paper we fine-tune diffusion models using(22)alone. This simplified formulation yields strong empirical performance while reducing hyperparameter sensitivity, as MCLR introduces only a single hyperparameter, i.e., the learning rate. 13 The theoretical equivalence between MCLR and CFG requires an adaptive weighting scheme as in(23), which is intractable in practice. While this weight could in principle be approximated via an ELBO approximation, we instead adopt the standard (without adaptive weighting) MCLR objective. Despite this simplification, we find that standard MCLR achieves comparable quantitative performance to CFG and produces similar qualitative effects. Datasets, Models, and Baselines. We evaluate MCLR alongside several baselines, including CCA, C-DPO, DDO, and CFG, on both diffusion and visual autoregressive models. For diffusion models, we fine-tune pretrained EDM2 models [Kar+24b] on ImageNet-64ร64 and ImageNet-512ร512. For visual autoregressive models, we fine-tune VAR-d24 model [Tia+24] on ImageNet-256ร256. Evaluation Metrics. We evaluate generative performance usingFrรฉchet Distance(FD) [Heu+17], Precision and Recall [Kyn+19], and Inception score (IS) [Sal+16]. FD measures distributional alignment, IS favors class-discriminative samples with high prediction confidence, Precision reflects sample fidelity and Recall measures diversity. FD, Precision, and Recall can be computed using either Inception or DINOv2 features. For diffusion models, we primarily report FD computed with DINOv2 features (FD DINOv2 ). While FID (Inception-based FD) is widely used in the diffusion literature, we find that for strong pretrained model such as EDM2, it can be insensitive to perceptually meaningful improvements. In particular, although both CFG and MCLR lead to visually pronounced quality improvements, the FID score often does not improve and may even degrade. In contrast, FD DINOv2 consistently captures these improvements, aligning with prior findings that it correlates more strongly with human evaluations [Ste+23]. For visual autoregressive models, both FID and FD DINOv2 improve consistently. Precision and Recall exhibit consistent trends across feature extractors; we therefore report results computed with Inception features in the main text and defer DINOv2-based results to section F. 6.2 Overall Algorithmic Behavior Progressive Class Separation and FidelityโDiversity Trade-off. We first analyze the training dynamics induced by MCLR and their effect on conditional generation. Qualitatively, as shown in Figures 2 and 8 to 19, at early stages of training, images generated from the same initial noises share similar global structures across different class conditions, indicating weak class-conditional modeling by the base model. As training proceeds, generated images gradually develop distinct class-specific structures, reflecting increasing inter-class separation and improved conditional modeling. This behavior is also reflected quantitatively. As shown in Figure 4(b,d), both Inception Score (IS) and Precision increase steadily during training, indicating that the generated samples become more class-discriminative and visually faithful. However, excessive training reduces within-class diversity, as reflected by a decrease in Recall ( Figure 4(e)). As a consequence, FD DINOv2 initially improves as conditional modeling strengthens, but later deteriorates when diversity decreases, as shown in Figure 4(a), revealing a characteristic fidelityโdiversity trade-off. Interestingly, this behavior closely mirrors the effect of increasing the guidance scale in CFG, where stronger guidance improves class fidelity at the cost of diversity. Similar trends are observed for other contrastive alignment objectives such as C-DPO and CCA. Following prior works, we therefore report results at the checkpoint achieving the best FD DINOv2 score in table 1. 14 (g) (i) (h) (j) (b) (e) (c) (f) (a) (d) ImageNet-512 EDM2-LImageNet-256 VAR-d24 Figure 4: Quantitative Results for EDM2-L and VAR-d24. (a), (b), (d), and (e) show the evolution of FD, Inception Score, Precision, and Recall, respectively, as functions of training iterations. (c), (g), and (h) illustrate the FDโIS trade-offs, while (f), (i), and (j) depict the PrecisionโRecall trade-offs. For EDM2 models, we evaluate classifier-free guid- ance (CFG) scalesํพ โ 0.1,0.2,0.3,0.4,0.5,0.7,0.9,1,1.5,2.0,3.0. For Var-d24 model,ํพ โ 0.5,0.8,1.1,1.5,1.7,2.0,2.5,3.0,4.0,5.0,7.0,10.0,15.0. Start denotes the performance of the base model, while End denotes the model obtained after a fixed finetuning duration. MCLR Outperforms Training-time Baselines including C-DPO and CCA. (i) Diffusion Models. For diffusion models, MCLR achieves substantially better best-case FD DINOv2 scores than CCA, C-DPO, and DDO on ImageNet (see table 1). Moreover, as shown in Figure 4 (c,f), MCLR traverses a significantly wider fidelityโdiversity trade-off region steadily with a faster training speed. In contrast, CCA and C-DPO converge early at suboptimal local minima and exhibit zigzag optimization trajectories, as evidenced by slowly improving or stalled learning curves. This behavior suggests that these objectives are more difficult to optimize in diffusion models. (i) Autoregressive Models. In contrast to diffusion models, MCLR, C-DPO, and CCA achieve comparable performance improvements on VAR-d24, substantially improving both FD DINOv2 and FID (see Figure 4(g,h)). We hypothesize that this difference arises from how likelihood is computed in each framework. In diffusion models, the likelihood is only approximated, and C-DPO and CCA additionally require estimating the base-model likelihood, introducing extra variance during training. In contrast, autoregressive models provide exact likelihoods, resulting in lower-variance optimization and more stable behavior. Despite this, MCLR consistently achieves higher precision in later training stages, as shown in Figure 4(i,j). MCLR Achieves Comparable Performance as CFG Quantitatively. We next compare MCLR with classifier-free guidance (CFG). Overall, the two approaches achieve comparable performance, but exhibit different trade-offs across evaluation metrics. (i) Diffusion Models. For EDM2 models, CFG generally exhibits a better FDโIS trade-off and achieves a lower best-case FD DINOv2 . Nevertheless, this gap is moderate, particularly for the EDM2-L model, where CFG attains a best-case FD DINOv2 of 39.86 compared to 42.50 for MCLR. When evaluated using PrecisionโRecall, MCLR exhibits competitive performance relative to CFG. Specifically, as shown in Figure 4(f), MCLR matches CFG in the high-recall regime corresponding to early training stages, and attains a substantially higher best-case precision in the high-precision regime at later training stages, where CFG begins to produce images with oversaturated colors. Furthermore, applying CFG on top of an MCLR-fine-tuned model further narrows the performance 15 Table 1: Quantitative results for MCLR, CFG and training-time baselines. For each algorithm, the metrics are reported at the model checkpoint achieving the best case FD DINOv2 . Precision and Recall are computed using Inception features. The best and second-best results are highlighted in bold and underline, respectively. MethodNFEFD DINOv2 โPrec.โRec.โISโ ImageNet (64ร64) EDM2-S6395.200.7050.61460.43 +CFG12643.750.8000.565127.40 +DDO6372.970.6890.64265.84 +CCA6362.360.7620.55776.13 +C-DPO6360.980.7840.53686.11 +MCLR (Ours)6352.69 0.8000.50590.68 ImageNet (256ร256) VAR-d2410182.120.6720.62367.92 +CFG2045.08 0.7980.542100.70 +CCA1046.820.8730.44898.92 +C-DPO1046.630.8810.433100.12 +MCLR (Ours)1044.310.8930.404100.84 ImageNet (512ร512) EDM2-L6367.700.7530.610250.07 +CFG12639.860.8440.512360.30 +DDO6349.470.7370.652268.72 +CCA6351.450.7900.574281.45 +C-DPO6351.920.8120.544298.89 +MCLR (Ours)6342.50 0.8490.492332.02 gap between the two methods, yielding higher best-case Inception Score and Precision on both EDM2-S and EDM2-L models (see Figure 4(c,f) and Figure 5(c,f,i)). (i) Autoregressive Models. For VAR-d24 model, MCLR achieves a similar FDโIS trade-off to CFG in terms of both FD DINOv2 and FID, with CFG exhibiting a slightly better best-case FID. Consistent with diffusion models, MCLR outperforms CFG in the PrecisionโRecall trade-off and achieves a higher best-case precision. MCLR Exhibits Similar Qualitative Effect as CFG. Qualitatively, MCLR and CFG produce highly similar visual effects, both substantially enhancing class-specific structures in the generated images, as shown in Figures 2 and 8 to 19. These observations are consistent with our theoretical analysis in section 4, which interprets CFG as an implicit contrastive alignment method. In particular, both MCLR and CFG improve conditional modeling by leveraging inter-class contrastive signals. The key distinction lies in how this mechanism is applied: MCLR internalizes it during training, whereas CFG introduces it at inference time. Equivalence between CCA and C-DPO. Finally, we observe that C-DPO consistently matches or outperforms CCA on both EDM2 and VAR models. This empirical observation aligns with our theoretical analysis in section 3.4, which establishes the equivalence between these two objectives. 16 7 Discussion and Conclusions In this work, we introduce MCLR, a training-time objective that improves conditional modeling in visual generative models by explicitly encouraging inter-class separation. Through extensive experiments, we demonstrate that MCLR consistently outperforms existing training-time baselines and achieves effects similar to classifier-free guidance (CFG), substantially improving the visual fidelity of conditional generation without requiring inference-time guidance and therefore enabling faster inference. Beyond empirical improvements, our analysis reveals a close connection between MCLR and CFG. In particular, we show that the CFG-guided score corresponds to the optimal solution of a weighted MCLR objective, providing a mechanistic interpretation of CFG as an implicit contrastive alignment algorithm. This connection sheds new light on CFG as approximately optimizing an inter-class likelihood-ratio objective. Limitations and Future Directions. As a training-time method, MCLR produces a fixed model after training. Although it exhibits a fidelityโdiversity trade-off similar to that of CFG, it lacks the flexibility of inference-time guidance to dynamically adjust this trade-off between generation quality and diversity through customized guidance strength. Moreover, CFG typically achieves stronger best-case performance in terms of FD and Inception Score. Bridging the gap between training-time objectives and inference-time guidance remains an important direction for future work. More fundamentally, these observations raise a broader question: Should alignment be performed at training time, or is inference-time guidance a more effective paradigm? Recent studies [Fra+25; Jin+25; Che+25c; JWL26] suggest that inference-time guidance can, in certain settings, outperform training-based alignment algorithms. These observations point toward a promising research direction: developing alignment algorithms that operate directly at inference time. Exploring this possibility may lead to a broader shift from traditional training-time alignment toward inference-time alignment paradigms. Acknowledgment We acknowledge funding support from NSF CCF-2212066, NSF CCF- 2212326, NSF IIS 2402950, and ONR N000142512339. This research used the Delta advanced computing and data resource which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta is a joint effort of the University of Illinois Urbana-Champaign and its National Center for Supercomputing Applications[Boe+23]. 17 References [And82]Brian DO Anderson. โReverse-time diffusion equation modelsโ. In: Stochastic Processes and their Applications 12.3 (1982), p. 313โ326. [BN24]Arwen Bradley and Preetum Nakkiran. โClassifier-free guidance is a predictor-correctorโ. In: arXiv preprint arXiv:2408.09000 (2024). [Boe+23]Timothy J Boerner, Stephen Deems, Thomas R Furlani, Shelley L Knuth, and John Towns. โAccess: Advancing innovation: Nsfโs advanced cyberinfrastructure coordina- tion ecosystem: Services & supportโ. In: Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good. 2023, p. 173โ176. [BT52]Ralph Allan Bradley and Milton E Terry. โRank analysis of incomplete block designs: I. The method of paired comparisonsโ. In: Biometrika 39.3/4 (1952), p. 324โ345. [BV04]Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. [Che+25a]Huayu Chen, Kai Jiang, Kaiwen Zheng, Jianfei Chen, Hang Su, and Jun Zhu. โVisual Generation Without Guidanceโ. In: Forty-second International Conference on Machine Learning. 2025. [Che+25b]Huayu Chen, Hang Su, Peize Sun, and Jun Zhu. โToward Guidance-Free AR Visual Generation via Condition Contrastive Alignmentโ. In: The Thirteenth International Conference on Learning Representations. 2025. [Che+25c]Min Cheng, Fatemeh Doudi, Dileep Kalathil, Mohammad Ghavamzadeh, and Pan- ganamala R Kumar. โDiffusion Blend: Inference-Time Multi-Preference Alignment for Diffusion Modelsโ. In: arXiv preprint arXiv:2505.18547 (2025). [Chi+24]Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. โWhat does guidance do? A fine-grained analysis in a simple settingโ. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024. [Ess+24]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Mรผller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. โScaling rectified flow transformers for high-resolution image synthesisโ. In: Forty-first international conference on machine learning. 2024. [Fra+25]Kevin Frans, Seohong Park, Pieter Abbeel, and Sergey Levine. โDiffusion guidance is a controllable policy improvement operatorโ. In: arXiv preprint arXiv:2505.23458 (2025). [GH10]Michael Gutmann and Aapo Hyvรคrinen. โNoise-contrastive estimation: A new esti- mation principle for unnormalized statistical modelsโ. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Con- ference Proceedings. 2010, p. 297โ304. [Heu+17] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. โGans trained by a two time-scale update rule converge to a local nash equilibriumโ. In: Advances in neural information processing systems 30 (2017). [HJA20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. โDenoising diffusion probabilistic modelsโ. In: Advances in neural information processing systems 33 (2020), p. 6840โ6851. [HS22]Jonathan Ho and Tim Salimans. โClassifier-free diffusion guidanceโ. In: arXiv preprint arXiv:2207.12598 (2022). 18 [Jin+25]Luozhijie Jin, Zijie Qiu, Jie Liu, Zijie Diao, Lifeng Qiao, Ning Ding, Alex Lamb, and Xipeng Qiu. โInference-time alignment control for diffusion models with reinforce- ment learning guidanceโ. In: arXiv preprint arXiv:2508.21016 (2025). [JSG25]Cheng Jin, Qitan Shi, and Yuantao Gu. โStage-wise Dynamics of Classifier-Free Guid- ance in Diffusion Modelsโ. In: arXiv preprint arXiv:2509.22007 (2025). [JWL26]Zhou Jiang, Yandong Wen, and Zhen Liu. โRethinking Preference Alignment for Diffusion Models with Classifier-Free Guidanceโ. In: arXiv preprint arXiv:2602.18799 (2026). [Kar+22]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. โElucidating the design space of diffusion-based generative modelsโ. In: Advances in neural information processing systems 35 (2022), p. 26565โ26577. [Kar+24a]Tero Karras, Miika Aittala, Tuomas Kynkรคnniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. โGuiding a diffusion model with a bad version of itselfโ. In: Advances in Neural Information Processing Systems 37 (2024), p. 52996โ53021. [Kar+24b] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. โAnalyzing and improving the training dynamics of diffusion modelsโ. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, p. 24174โ24184. [KMS24]Zahra Kadkhodaie, Stรฉphane Mallat, and Eero P Simoncelli. โFeature-guided score dif- fusion for sampling conditional densitiesโ. In: arXiv preprint arXiv:2410.11646 (2024). [Kou+25]Felix Koulischer, Florian Handke, Johannes Deleu, Thomas Demeester, and Luca Am- brogioni. โFeedback guidance of diffusion modelsโ. In: arXiv preprint arXiv:2506.06085 (2025). [Kyn+19] Tuomas Kynkรคnniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. โImproved precision and recall metric for assessing generative modelsโ. In: Advances in neural information processing systems 32 (2019). [Lee+25]Jaa-Yeon Lee, Byunghee Cha, Jeongsol Kim, and Jong Chul Ye. โAligning text to image in diffusion models is easier than you thinkโ. In: arXiv preprint arXiv:2503.08250 (2025). [Lip+23]Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. โFlow Matching for Generative Modelingโ. In: The Eleventh International Conference on Learning Representations. 2023. [Liu+25]Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. โFlow-grpo: Training flow matching models via online rlโ. In: arXiv preprint arXiv:2505.05470 (2025). [LJ25]Gen Li and Yuchen Jiao. โProvable Efficiency of Guidance in Diffusion Models for General Data Distributionโ. In: Forty-second International Conference on Machine Learning. 2025. [LWQ25] Xiang Li, Rongrong Wang, and Qing Qu. โTowards Understanding the Mechanisms of Classifier-Free Guidanceโ. In: arXiv preprint arXiv:2505.19210 (2025). [Mar+23] Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. โA Variational Perspec- tive on Solving Inverse Problems with Diffusion Modelsโ. In: The Twelfth International Conference on Learning Representations. 2023. 19 [Miy+61]Koichi Miyasawa et al. โAn empirical Bayes estimator of the mean of a normal popu- lationโ. In: Bull. Inst. Internat. Statist 38.181-188 (1961), p. 1โ2. [OLV18]Aaron van den Oord, Yazhe Li, and Oriol Vinyals. โRepresentation learning with contrastive predictive codingโ. In: arXiv preprint arXiv:1807.03748 (2018). [Pav+25]Krunoslav Lehman Pavasovic, Jakob Verbeek, Giulio Biroli, and Marc Mezard. โClassifier- Free Guidance: From High-Dimensional Analysis to Generalized Guidance Formsโ. In: arXiv preprint arXiv:2502.07849 (2025). [PX23] William Peebles and Saining Xie. โScalable diffusion models with transformersโ. In: Proceedings of the IEEE/CVF international conference on computer vision. 2023, p. 4195โ 4205. [Raf+23]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. โDirect preference optimization: Your language model is secretly a reward modelโ. In: Advances in Neural Information Processing Systems 36 (2023), p. 53728โ53741. [Ram+22]Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. โHi- erarchical Text-Conditional Image Generation with CLIP Latentsโ. In: arXiv preprint arXiv:2204.06125 (2022). [Rom+22] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjรถrn Om- mer. โHigh-resolution image synthesis with latent diffusion modelsโ. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022, p. 10684โ 10695. [Sah+22]Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. โPhotorealistic text-to-image diffusion models with deep language understandingโ. In: Advances in neural information processing systems 35 (2022), p. 36479โ36494. [Sal+16]Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. โImproved techniques for training gansโ. In: Advances in neural information processing systems 29 (2016). [Son+21a] Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. โMaximum likelihood training of score-based diffusion modelsโ. In: Advances in neural information processing systems 34 (2021), p. 1415โ1428. [Son+21b] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. โScore-Based Generative Modeling through Stochastic Differ- ential Equationsโ. In: International Conference on Learning Representations. 2021. [Ste+23]George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. โExposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelsโ. In: Advances in Neural Information Processing Systems 36 (2023), p. 3732โ3784. [Tan+25]Zhicong Tang, Jianmin Bao, Dong Chen, and Baining Guo. โDiffusion models without classifier-free guidanceโ. In: arXiv preprint arXiv:2502.12154 (2025). [Tia+24]Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. โVisual autore- gressive modeling: Scalable image generation via next-scale predictionโ. In: Advances in neural information processing systems 37 (2024), p. 84839โ84865. 20 [Ven+26]Enrico Ventura, Beatrice Achilli, Luca Ambrogioni, and Carlo Lucibello. โEmergence of Distortions in High-Dimensional Guided Diffusion Modelsโ. In: arXiv preprint arXiv:2602.00716 (2026). [Vin11]Pascal Vincent. โA Connection Between Score Matching and Denoising Autoencodersโ. In: Neural Computation 23.7 (2011), p. 1661โ1674. doi: 10.1162/NECO_a_00142. [Wal+24]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Pu- rushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. โDiffusion model alignment using direct preference optimizationโ. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, p. 8228โ8238. [Wu+24]Yuchen Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei. โTheoretical insights for diffusion guidance: A case study for Gaussian mixture modelsโ. In: Forty- first International Conference on Machine Learning. 2024. [YAA25] Junno Yun, Yaลar Utku Alรงalar, and Mehmet Akรงakaya. โNo Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Modelsโ. In: arXiv preprint arXiv:2509.21565 (2025). [Yan+24]Divin Yan, Lu Qi, Vincent Tao Hu, Ming-Hsuan Yang, and Meng Tang. โTraining class- imbalanced diffusion model via overlap optimizationโ. In: arXiv preprint arXiv:2402.10821 (2024). [Yan+26]Ruofeng Yang, Yiyu Qiu, Bo Jiang, Cheng Chen, and Shuai Li. Elucidating Guidance in Variance Exploding Diffusion Models: Fast Convergence and Better Diversity. OpenReview. https://openreview.net/forum?id=tWbasgJA68. 2026. [Yu+25] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. โRepresentation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkโ. In: The Thirteenth International Conference on Learning Representations. 2025. [Zhe+25]Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. โDirect discriminative optimization: Your likelihood-based visual generative model is secretly a gan discriminatorโ. In: arXiv preprint arXiv:2503.01103 (2025). 21 Appendices A Theoretical Analysis of MCLR A.1 Main Theorem In this section, we provide the proof for Theorem 1. We first restate the assumptions and theorem. Assumption 1. The function โ(ํ|ํ) := ํ(ํ|ํ)+ ํ(ํ(ํ|ํ)โ ํ(ํ)) has compact support; that is, supp โ(ํ|ํ) โ ํพ, |ํพ| <โ.(25) Assumption 2. Forโํ โ ํพ, ํ ํฝ (ํ|ํ)โฅ ํฟ > 0,ํฟ < 1 |ํพ| .(26) Theorem 1. Under the two assumptions, the optimal solution to (11) is: ํ ํฝ โ (ํ|ํ) = ( max โ(ํ|ํ) ํ(ํ) , ํฟ , ํ โ ํพ, 0, ํ โ ํพ, (27) where ํ(ํ) is the normalizing constant. Proof. Without loss of generality, we consider discrete classes in this proof; the theorem extends straightforwardly to continuous classes. Suppose there areํclassesํ ํ ํ ํ=1 , each class has prior probability ํ(ํ ํ ) and ร ํ ํ=1 ํ(ํ ํ ) = 1, then the training objective (11) takes the following form: max ํฝ โ(ํฝ) := E ํ,ํ(ํ|ํ) log ํ ํฝ (ํ|ํ) + ํE ํ, ฬํ,ํโผํ(ํ|ํ) log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) (28) = max ํฝ ํ ร ํ=1 ํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) + ํ ํ ร ํ=1 ํ(ํ ํ ) ํ ร ํ=1 ํ(ํ ํ )E ํโผํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) (29) โํ ํ ร ํ=1 ํ(ํ ํ ) ํ ร ํ=1 ํ(ํ ํ )E ํโผํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) .(30) Note that we can decompose overall objectiveโ(ํฝ) as: โ(ํฝ) = ํ ร ํ=1 โ ํ (ํฝ),(31) whereโ ํ (ํฝ) is the amount contributed by ํ ํฝ (ํ|ํ ํ ): โ ํ (ํฝ) = ํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) + ํํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) (32) โํํ(ํ ํ ) ํ ร ํ=1 ํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) (33) = ํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) + ํํ(ํ ํ )E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) โ ํํ(ํ ํ )E ํ(ํ) log ํ ํฝ (ํ|ํ ํ ) . (34) 22 Note that we can optimizeํฟ ํ (ํฝ)for eachํ โ1, ... , ํto get the optimal conditional distribution ํ(ํ|ํ ํ ) independently: max ํฝ โ ํ (ํฝ)โ max ํ ํฝ (ยท|ํ ํ ) E ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) + ํE ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) โ ํE ํ(ํ) log ํ ํฝ (ํ|ํ ํ ) (35) = max ํ ํฝ (ยท|ํ ํ ) โซ ํพ log ํ ํฝ (ํ|ํ ํ ) ํ(ํ|ํ ํ )+ ํ(ํ(ํ|ํ ํ )โ ํ(ํ)) ํํ. (36) In what follows, we drop the subscript ํ, so that the optimization problem becomes: max ํ ํฝ (ยท|ํ) โซ ํพ log ํ ํฝ (ํ|ํ) ํ(ํ|ํ)+ ํ(ํ(ํ|ํ)โ ํ(ํ)) ํํ(37) = max ํ ํฝ (ยท|ํ) โซ ํพ log ํ ํฝ (ํ|ํ)โ(ํ|ํ)ํํ(38) Note that(38)shares the similar form as KL divergence, butโ(ํ|ํ)is not a valid probability distribution, since there could existsํsuch thatโ(ํ|ํ) <0. In this case, settingํ ํฝ (ํ|ํ) =0 makes(38)approaches+โ, hence the optimization problem does not have an attainable optimum. To make the optimization problem well-posed, we impose Assumption 2. Define the sets ํพ + (ํ) =ํ โ ํพ : โ(ํ|ํ) > 0, ํพ โ (ํ) =ํ โ ํพ : โ(ํ|ํ)โค 0.(39) Under Assumption 2, it is straightforward to show the optimal distributionํ ํฝ โ (ํ|ํ)must attain the lower boundํฟforโํ โ ํพ โ (ํ); otherwise increasing the density will decrease the objective(38). Furthermore forํ โ ํพ ํ , whereโ(ํ|ํ) =0, the optimal density must satisfyํ ํฝ โ (ํ|ํ) =0 for almost everyํ โ ํพ ํ ; otherwise, probability mass could be shifted fromํพ ํ toํพ + (ํ)to further increase the objective(38). Therefore, the remaining optimization concerns the density onํพ + (ํ), which is the optimal solution to the following constrained optimization problem: min ํ ํฝ (ยท|ํ) โซ ํพ + (ํ) โ log ํ ํฝ (ํ|ํ)โ(ํ|ํ)ํํ(40) s.t. โ ํ ํฝ (ํ|ํ)+ ํฟโค 0, โํ โ ํพ + (ํ)(41) โซ ํพ + (ํ) ํ ํฝ (ํ|ํ)ํํโ ํ(ํ) = 0,(42) where ํ(ํ) = 1โ โซ ํพ โ (ํ) ํฟํํ = 1โ ํฟ|ํพ โ (ํ)|.(43) Since ํฟ < 1 |ํพ| by assumption, we have 0 < ํ(ํ)โค 1. Note that this optimization problem is convex inํ ํฝ (ํ|ํ). Treatingํ ํฝ (ํ|ํ)for eachํas opti- mization variables, the constraints are affien and Slaterโs condition holds. Therefore strong duality applies, and the optimal solution can be characterized by the KKT conditions [BV04]. Define the Lagrangian as: โ(ํ ํฝ (ยท|ํ), ํ,ํข(ยท)) = โซ ํพ + (ํ) โ log ํ ํฝ (ํ|ํ)โ(ํ|ํ)ํํ+ ํ โซ ํพ + (ํ) ํ ํฝ (ํ|ํ)ํํโ ํ(ํ) (44) + โซ ํพ + (ํ) ํข(ํ)(โํ ํฝ (ํ|ํ)+ ํฟ)ํํ,(45) 23 where ํ and ํ(ยท) are the dual variables. For eachํ โ ํพ + (ํ), treatingํ ํฝ โ (ํ|ํ)as a pointwise optimization variable, letํ ํฝ โ (ํ|ํ),ํ โ and ํข โ (ํ)be the corresponding optimal primal and dual variables. Applying stationary condition of the KKT conditions, we have: โ ํ ํฝ โ (ํ|ํ) โ(ํ ํฝ โ (ํ|ํ), ํ โ ,ํข โ (ํ)) = 0(46) โโ โ(ํ|ํ) ํ ํฝ โ (ํ|ํ) + ํ โ โ ํข โ (ํ) = 0.(47) Based on (47), we consider the following two cases: โข Suppose ํข โ (ํ) = 0, we have ํ ํฝ โ (ํ|ํ) = โ(ํ|ํ) ํ โ . โขSupposeํข โ (ํ) โ 0, by complementary slackness, we haveํ ํฝ โ (ํ|ํ) = ํฟandํข โ (ํ) = ํ โ โ โ(ํ|ํ) ํฟ . Applying dual feasibility ํข โ (ํ)โฅ 0, we have โ(ํ|ํ)โค ํ โ ํฟ. Note that the above two cases can be combined as: ํ ํฝ โ (ํ|ํ) = max โ(ํ|ํ) ํ โ , ํฟ , โํ โ ํพ + (ํ).(48) Next, we prove ํ โ exists, i.e., (48) is normalizable. By applying the primal feasibility, we have: โซ ํพ + (ํ) ํ ํฝ โ (ํ|ํ)ํํ = ํ(ํ)(49) โ โซ ํพ + (ํ)โฉโ(ํ|ํ)โคํ โ ํฟ ํฟํํ+ โซ ํพ + (ํ)โฉโ(ํ|ํ)>ํ โ ํฟ) โ(ํ|ํ) ํ โ ํํ = ํ(ํ).(50) Define: ํด(ํ) := โซ ํพ + (ํ)โฉโ(ํ|ํ)โคํํฟ) ํฟํํ+ โซ ํพ + (ํ)โฉโ(ํ|ํ)>ํํฟ โ(ํ|ํ) ํ ํํ.(51) Suppose 0 < ํ 1 < ํ 2 , we have: ํด(ํ 2 )โ ํด(ํ 1 ) = โซ ํพ + (ํ)โฉํ 1 ํฟโคโ(ํ|ํ)โคํ 2 ํฟ ํฟํํโ โซ ํพ + (ํ)โฉํ 1 ํฟโคโ(ํ|ํ)โคํ 2 ํฟ โ(ํ|ํ) ํ 1 ํํ(52) + โซ ํพ + (ํ)โฉโ(ํ|ํ)>ํ 2 ํฟ ( โ(ํ|ํ) ํ 2 โ โ(ํ|ํ) ํ 1 )ํํ(53) = โซ ํพ + (ํ)โฉํ 1 ํฟโคโ(ํ|ํ)โคํ 2 ํฟ (ํฟโ โ(ํ|ํ) ํ 1 )ํํ+ โซ ํพ + (ํ)โฉโ(ํ)>ํ 2 ํฟ ( โ(ํ|ํ) ํ 2 โ โ(ํ|ํ) ํ 1 )ํํ(54) < 0,(55) which impliesํด(ํ)is a monotonically decreasing function ofํ >0. By the assumptionํฟ < 1 |ํพ| , we have ํฟ|ํพ โ (ํ)|+ ํฟ|ํพ + (ํ)| < 1, which implies ํฟ|ํพ + (ํ)| < ํ(ํ). Since: lim ํโ0 ํด(ํ) =+โ, lim ํโ+โ ํด(ํ) = ํฟ|ํพ + (ํ)|,(56) as long asํฟ|ํพ + (ํ)| < ํ(ํ), by the intermediate value theorem, there exists a finite dual optimal pointํ โ >0 such thatํด(ํ โ ) = ํ(ํ), i.e., the primal feasibility(50)holds. Hence,ํ ํฝ โ (ํ)in(48)is a 24 valid, normalizable probability distribution. Since the exact value ofํ โ is dependent on the specific condition ํ, we replace it with the notation ํ(ํ), which leads to the final result: ํ ํฝ โ (ํ|ํ) = ๏ฃฑ ๏ฃด ๏ฃด ๏ฃด๏ฃฒ ๏ฃด ๏ฃด ๏ฃด ๏ฃณ max โ(ํ|ํ) ํ(ํ) , ํฟ , ํ โ ํพ + (ํ), ํฟ, ํ โ ํพ โ (ํ), 0, ํ โ ํพ. (57) , which can be further simplified as: ํ ํฝ โ (ํ|ํ) = ( max โ(ํ|ํ) ํ โ , ํฟ , ํ โ ํพ, 0, ํ โ ํพ. (58) In the limit ํฟโ 0, the optimal distribution approaches: ํ ํฝ โ (ํ|ํ) = โ + (ํ|ํ) โซ ํ โ + (ํ|ํ)ํํ , โ + (ํ|ํ) := max โ(ํ|ํ), 0 (59) which simply zeros out the negative part ofโ(ํ|ํ)and renormalizes it as a valid distribution. This completes the proof.โก A.2 Fine-tuning with MCLR Given a base modelํ ref (ํ)that lacks class specificity, we may fine-tune it using MCLR combined with KL regularization: max ํฝ โE ํ ํท KL ํ ref (ํ|ํ)||ํ ํฝ (ํ|ํ) + ํE ํ, ฬํ,ํโผํ(ํ|ํ) log ํ ํฝ (ํ|ํ) ํ ํฝ (ํ| ฬํ) .(60) Similar to(35), we can get the optimal conditional distributionํ(ํ|ํ ํ )for eachํ โ1, ... , ํby solving the following optimization problem: arg max ํ ํฝ (ยท|ํ ํ ) โํท KL ํ ref (ํ|ํ ํ )||ํ ํฝ (ํ|ํ ํ ) +ํE ํ(ํ|ํ ํ ) log ํ ํฝ (ํ|ํ ํ ) โ ํE ํ(ํ) log ํ ํฝ (ํ|ํ ํ ) (61) = arg max ํ ํฝ (ยท|ํ ํ ) โซ ํพ log ํ ํฝ (ํ|ํ ํ ) ํ ref (ํ|ํ ํ )+ ํ(ํ(ํ|ํ ํ )โ ํ(ํ)) ํํ.(62) Note that optimization problem(62)shares the same structure as(36), hence by lettingโ(ํ|ํ):= ํ ref (ํ|ํ)+ ํ(ํ(ํ|ํ)โ ํ(ํ)) and under the same compact-support Assumption 1, we get the same optimal solution as in stated in Theorem 1. Importantly, in the fine-tuning setting, under a mixture error model, MCLR recovers the ground truth conditional distribution, as stated in the following corollary. Corollary 1. If the base model satisfies the mixture error model: ํ ref (ํ|ํ) =(1โ ํ)ํ(ํ|ํ)+ ํํ(ํ),(63) then fine-tuningํ ref (ํ|ํ)with MCLR objective(13)recovers the ground truth conditional distribution ํ(ํ|ํ). Proof. Under the mixture-error model (63), the optimization problem (62) becomes: arg max ํ ํฝ (ยท|ํ ํ ) โซ ํพ log ํ ํฝ (ํ|ํ ํ )ํ(ํ|ํ ํ )ํํ = arg min ํ ํฝ (ยท|ํ ํ ) ํท KL (ํ(ํ|ํ ํ )||ํ ํฝ (ํ|ํ ํ )),(64) which has optimal solutionํ ํฝ (ํ|ํ ํ ) = ํ(ํ|ํ ํ ). Note that in this case, the proof does not depend on Assumption 1 and Assumption 2. This completes the proof.โก 25 B Theoretical Analysis of C-DPO B.1 Basics of DPO Reward Modeling. For a given promptํand two associated outputsํ ํค andํ ํ , where the subscripts โํคโ stands for โwinningโ while โํโ stands for โlosingโ, implying thatํ ํค is preferred overํ ํ , DPO models the human preference distribution with the Bradley-Terry (BT) model [BT52]: ํ(ํ ํค โป ํ ํ |ํ) = exp(ํ โ (ํ ํค |ํ)) exp(ํ โ (ํ ํค |ํ))+ exp(ํ โ (ํ ํ |ํ)) = Sigmoid(ํ โ (ํ ํค |ํ)โ ํ โ (ํ ํ |ํ))(65) whereํ โ (ํ|ํ)is the underlying optimal reward function for promptํ. Intuitively, the preferred samplesํ ํค should have higher reward values compared to the non-preferred samplesํ ํ . Assuming access to a dataset of sampled comparisonsํฎ =(ํ ํ , ํ ํ ํค , ํ ํ ํ ) ํ ํ=1 , one can learn the optimal reward function via maximum likelihood estimation: ํ โ = arg max ํ E (ํ,ํ ํค ,ํ ํ )โผํฎ log ํ(ํ ํค โป ํ ํ |ํ) (66) = arg min ํ โE (ํ,ํ ํค ,ํ ํ )โผํ log Sigmoid(ํ(ํ ํค |ํ)โ ํ(ํ ํ |ํ)) .(67) RL fine-tuning Phase. Assuming access to the optimal preference reward functionํ โ , one can fine-tune a base modelํ ref (ํ|ํ)to align with the preference dataset by optimizing the following objective: max ํฝ E ํ E ํโผํ ํฝ (ํ|ํ) ํ โ (ํ|ํ) โํฝํท ํพํฟ (ํ ํฝ (ํ|ํ)||ํ ref (ํ|ํ)) ,(68) whereํฝcontrols the KL-regularization strength. Intuitively, objective(68)encourages the fine- tuned model to achieve high reward value in expectation, and at the same time not deviate too much from the base model. DPO Objective. Under certain regularity conditions, the optimal solution to the fine-tuning objective (68) admits the following closed-form: ํ ํฝ โ (ํ|ํ) = 1 ํ(ํ) ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ)),(69) whereํ(ํ)is a partition function for normalizing the density. To prove this, consider optimizing the fine-tuning objective (68) for a target condition ํ: arg max ํฝ E ํโผํ ํฝ (ํ|ํ) ํ โ (ํ|ํ) โ ํฝํท ํพํฟ (ํ ํฝ (ํ|ํ)||ํ ref (ํ|ํ))(70) = arg max ํฝ E ํโผํ ํฝ (ํ|ํ) ํ โ (ํ|ํ)โ ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) (71) = arg min ํฝ E ํโผํ ํฝ (ํ|ํ) log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) โ 1 ํฝ ํ โ (ํ|ํ) (72) = arg min ํฝ E ํโผํ ํฝ (ํ|ํ) log ํ ํฝ (ํ|ํ) 1 ํ(ํ) ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ)) โ log ํ(ํ) , (73) 26 where ํ(ํ) is the partition function that normalizes ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ)): ํ(ํ) = โซ ํ ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ))ํํ,(74) such thatํ โ (ํ|ํ):= 1 ํ(ํ) ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ))is a valid probability distribution. Sinceํ(ํ)is independent ofํฝ, the fine-tuning objective (73) is equivalent to: min ํฝ ํท KL (ํ ํฝ (ํ|ํ)||ํ โ (ํ|ํ)),(75) which achieves its minimum value 0 if and only if: ํ ํฝ (ํ|ํ) = ํ โ (ํ|ํ) = 1 ํ(ํ) ํ ref (ํ|ํ) exp( 1 ํฝ ํ โ (ํ|ํ)).(76) With some algebra, the optimal reward function can be expressed with ํ ํฝ โ (ํ|ํ): ํ โ (ํ|ํ) = ํฝ log ํ ํฝ โ (ํ|ํ) ํ ref (ํ|ํ) + ํฝ log ํ(ํ).(77) The relationship(77)between the optimal reward and the optimal fine-tuned distribution suggests a convenient parameterization of the reward model: ํ ํฝ (ํ|ํ) = ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) .(78) Substitute (78) into (67) results in the DPO objective: arg min ํฝ โE (ํ,ํ ํค ,ํ ํ )โผํ log Sigmoid(ํฝ log ํ ํฝ (ํ ํค |ํ) ํ ref (ํ ํค |ํ) โ ํฝ log ํ ํฝ (ํ ํ |ํ) ํ ref (ํ ํ |ํ) ) .(79) In this way, one can directly fine-tune the base model without explicitly modeling the reward. B.2 Improving Conditional Modeling with C-DPO To adapt DPO for improving class specificity, we may treat samples from the target classํas preferred data (ํ ํค ) and samples from other randomly selected classes as non-preferred data (ํ ํ ). This leads to the following objective: min ํฝ โE ํ, ฬํ,ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ| ฬํ) log Sigmoid(ํฝ log ํ ํฝ (ํ ํค |ํ) ํ ref (ํ ํค |ํ) โ ํฝ log ํ ํฝ (ํ ํ |ํ) ํ ref (ํ ํ |ํ) ) (80) = min ํฝ โE ํ,ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ) log Sigmoid(ํฝ log ํ ํฝ (ํ ํค |ํ) ํ ref (ํ ํค |ํ) โ ํฝ log ํ ํฝ (ํ ํ |ํ) ํ ref (ํ ํ |ํ) ) ,(81) which admits a closed-form solution as stated in the following theorem. Theorem 2. Under certain regularity conditions, the optimal solution to (81) is: ํ ํฝ โ (ํ|ํ) = 1 ฬ ํ(ํ) ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ,(82) where ฬ ํ(ํ) = โซ ํ ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ํํ is the normalizing constant. 27 Proof.From(67)to(69), it is clear that the optimal solution to C-DPO(81)is fully determined by the base model and the optimal solution to the following reward modeling objective: ํ โ = arg min ํ โE ํ,ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ) log Sigmoid(ํ(ํ ํค |ํ)โ ํ(ํ ํ |ํ)) .(83) Note thatํ โ is the collection of optimal rewardsํ โ (ยท|ํ ํ )for eachํ ํ โ ํ ํ ํ ํ=1 . Without loss of generality, we drop the subscript and solve for the optimal reward for a single target ํ: ํ โ (ยท|ํ) = arg min ํ(ยท|ํ) โE ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ) log Sigmoid(ํ(ํ ํค |ํ)โ ํ(ํ ํ |ํ)) (84) = arg max ํ(ยท|ํ) โซ ํ ํค โซ ํ ํ log Sigmoid(ํ(ํ ํค |ํ)โ ํ(ํ ํ |ํ))ํ(ํ ํค |ํ)ํ(ํ ํ )ํํ ํค ํํ ํ .(85) Note that for any arbitrary pair of points(ํ 1 , ํ 2 ), they contribute to the integral(85)for the following amount: log Sigmoid ( ํ(ํ 1 |ํ)โ ํ(ํ 2 |ํ) ) ํ(ํ 1 |ํ)ํ(ํ 2 )+ log Sigmoid ( ํ(ํ 2 |ํ)โ ํ(ํ 1 |ํ) ) ํ(ํ 2 |ํ)ํ(ํ 1 ), (86) henceํ โ (ยท|ํ)is an optimal solution to(85)if it maximizes(86)forโ(ํ 1 , ํ 2 ). To find suchํ โ (ยท|ํ), letโs define ํค = ํ(ํ 1 |ํ)โ ํ(ํ 2 |ํ) and solve the following optimization problem: max ํ โ(ํค) := log Sigmoid(ํค)ํ(ํ 1 |ํ)ํ(ํ 2 )+ log Sigmoid(โํค)ํ(ํ 2 |ํ)ํ(ํ 1 ).(87) Note that: โ ํค โ(ํค) = Sigmoid(โํค)ํ(ํ 1 |ํ)ํ(ํ 2 )โ Sigmoid(ํค)ํ(ํ 2 |ํ)ํ(ํ 1 ),(88) which implies the stationary point ํค โ must satisfy: Sigmoid(ํค โ )ํ(ํ 2 |ํ)ํ(ํ 1 ) = Sigmoid(โํค โ )ํ(ํ 1 |ํ)ํ(ํ 2 )(89) โ Sigmoid(ํค โ ) ํ(ํ 2 |ํ)ํ(ํ 1 )+ ํ(ํ 1 |ํ)ํ(ํ 2 ) = ํ(ํ 1 |ํ)ํ(ํ 2 )(90) โ Sigmoid(ํค โ ) = ํ(ํ 1 |ํ)ํ(ํ 2 ) ํ(ํ 2 |ํ)ํ(ํ 1 )+ ํ(ํ 1 |ํ)ํ(ํ 2 ) (91) โ Sigmoid(ํค โ ) = 1 1+ ํ(ํ 2 |ํ)ํ(ํ 1 ) ํ(ํ 1 |ํ)ํ(ํ 2 ) (92) โ Sigmoid(ํค โ ) = 1 1+ exp โ log ํ(ํ 1 |ํ)ํ(ํ 2 ) ํ(ํ 1 )ํ(ํ 2 |ํ) (93) โ ํค โ = log ํ(ํ 1 |ํ)ํ(ํ 2 ) ํ(ํ 1 )ํ(ํ 2 |ํ) ,(94) which implies: ํ โ (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) + ํพ,(95) where ํพ is any finite constant. Moreover, since: โ 2 ํค โ(ํค) =โSigmoid(ํค)Sigmoid(โํค)ํ(ํ 1 |ํ)ํ(ํ 2 )โ Sigmoid(โํค)Sigmoid(ํค)ํ(ํ 2 |ํ)ํ(ํ 1 ) (96) < 0,(97) 28 we knowโ(ํค)is concave andํค โ is the global maximizer and consequentlyํ โ (ํ|ํ)is the optimal reward function. Since this optimal reward function is consistent for any arbitrary pair of points (ํ 1 , ํ 2 ), it is the global reward function that maximizes the full objective (84). Substitute (95) into (69), we obtain the optimal solution to C-DPO: ํ ํฝ โ (ํ|ํ) = 1 ฬ ํ(ํ) ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ,(98) where ฬ ํ(ํ) = โซ ํ ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ํํ is the normalizing constant. This completes the proof. โก Remark. Note that from(90)to(91), we requireํ(ํ 2 |ํ)ํ(ํ 1 )+ ํ(ํ 1 |ํ)ํ(ํ 2 ) >0 forโํ 1 , ํ 2 , which holds true if bothํ(ํ|ํ)andํ(ํ)have full support onR ํ . Next, we discuss the corner cases where this assumption doesnโt hold. 1. Suppose ํ(ํ 1 |ํ) = 0 but ํ(ํ 1 ) > 0, then we have: โ(ํค) = log Sigmoid ํ(ํ 2 |ํ)โ ํ(ํ 1 |ํ) ํ(ํ 2 |ํ)ํ(ํ 1 ).(99) In this case, the objective is maximized if ํ โ (ํ 1 |ํ) =โโ. 2. Suppose ํ(ํ 1 |ํ) > 0 but ํ(ํ 1 ) = 0, then we have: โ(ํค) = log Sigmoid ํ(ํ 1 |ํ)โ ํ(ํ 2 |ํ) ํ(ํ 1 |ํ)ํ(ํ 2 ).(100) In this case, the objective is maximized if ํ โ (ํ 1 |ํ) =+โ. 3. Supposeํ(ํ 1 |ํ) =0 andํ(ํ 1 ) =0, thenํ 1 will never be sampled and it does not contribute to the overall objective. In such case,ํ โ (ํ 1 |ํ)is undefined and will implicitly depend on the practical parameterization of the reward model. The first two cases are already covered by(82). In particular, case 2 can lead to non-normalizable issue, as in this caseํ ํฝ โ (ยท|ํ)will have infinite density atํ 1 . Although sinceํ(ํ 1 ) = E ํ ํ(ํ 1 |ํ) , it will not be zero givenํ(ํ 1 |ํ) >0. It is highly likely in practice, there existsํ 1 such thatํ(ํ 1 |ํ) >0 butํ(ํ 1 )โ0. In this regime, the ratio ํ(ํ|ํ) ํ(ํ) 1 ํฝ blows up, forcingํ ํฝ โ (ยท|ํ)to essentially place all its mass on such ํ, which again leads to non-normalizable issue. Error Model for C-DPO. As with the MCLR case, C-DPO recovers the ground truth conditional distribution under an appropriate error model, as stated in the following corollary. Corollary 2. If the base model satisfies: ํ ref (ํ|ํ)โ ํ(ํ|ํ) 1โ 1 ํฝ ํ(ํ) 1 ํฝ (101) then fine-tuningํ ref (ํ|ํ)with the C-DPO objective(19)recovers the ground-truth conditional distri- bution ํ(ํ|ํ). Proof. Substitute (101) into (98), we get ํ ํฝ โ (ํ|ํ) = ํ(ํ|ํ), which completes the proof.โก 29 C Theoretical Analysis of CCA In section B.2, weโve shown that the underlying optimal reward function induced by the C-DPO objective(83)has the form of the log likelihood-ratio(95). Interestingly, the same reward function can be obtained by minimizing the following optimization problem: ํ โ (ยท|ํ) = arg max ํ(ยท|ํ) E ํ(ํ|ํ) log Sigmoid(ํ(ํ|ํ))+ E ํ(ํ) log Sigmoid(โํ(ํ|ํ))(102) = arg max ํ(ยท|ํ) โซ ํ log Sigmoid(ํ(ํ|ํ))ํ(ํ|ํ)+ log Sigmoid(โํ(ํ|ํ))ํ(ํ) ํํ.(103) Since the objective(103)decomposes pointwise overํ, the optimal reward function can be obtained by maximizing the integrand for each ํ independently: log Sigmoid(ํ(ํ|ํ))ํ(ํ|ํ)+ log Sigmoid(โํ(ํ|ํ))ํ(ํ).(104) To find ํ โ (ยท|ํ), letโs define ํค = ํ(ํ|ํ) and solve the following optimization problem: max ํค โ(ํค) := log Sigmoid(ํค)ํ(ํ|ํ)+ log Sigmoid(โํค)ํ(ํ).(105) Note that: โ ํค โ(ํค) := Sigmoid(โํค)ํ(ํ|ํ)โ Sigmoid(ํค)ํ(ํ),(106) which implies the stationary point ํค โ is: ํค โ = ํ โ (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) .(107) Furthermore, since: โ 2 ํค โ(ํค) =โSigmoid(ํค)Sigmoid(โํค)ํ(ํ|ํ)โ Sigmoid(ํค)Sigmoid(โํค)ํ(ํ)(108) < 0,(109) we knowโ(ํค)is concave andํ โ (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) is the unique maximizer. Hence, weโve proved that the optimization problems(83)and(103)lead to the same optimal reward up to an additive constant. Therefore, by parameterizing the reward function as: ํ ํฝ (ํ|ํ) = ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) (110) , and substitute this in (103), we get the following optimization problem: ํ ํฝ โ (ยท|ํ) = arg max ํ ํฝ (ยท|ํ) E ํ(ํ|ํ) log Sigmoid ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) +E ํ(ํ) log Sigmoid โํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) , (111) which has the global maximizer: ํฝ log ํ ํฝ โ (ํ|ํ) ํ ref (ํ|ํ) = ํ โ (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) (112) โ ํ ํฝ โ (ํ|ํ) = ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ .(113) 30 One issue with the optimization objective(111)is that its optimal solution(113)is not a valid probability distribution because it is not normalized. To alleviate this issue, we consider a slightly modified version of (103) by introducing an additional constant ํ: ํ โ (ยท|ํ) = arg max ํ(ยท|ํ) E ํ(ํ|ํ) log Sigmoid(ํ(ํ|ํ))+ ํE ํ(ํ) log Sigmoid(โํ(ํ|ํ)).(114) With similar proof technique, it can be shown that: ํ โ (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) + log 1 ํ .(115) Substitute (110) in (114), we get the following optimization problem: ํ ํฝ โ (ยท|ํ) = arg max ํ ํฝ (ยท|ํ) E ํ(ํ|ํ) log Sigmoid ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) +ํE ํ(ํ) log Sigmoid โํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) , (116) which has global maximizer: ํฝ log ํ ํฝ โ (ํ|ํ) ํ ref (ํ|ํ) = log ํ(ํ|ํ) ํ(ํ) + log 1 ํ (117) โ ํ ํฝ โ (ํ|ํ) = ํ ref (ํ|ํ) ํ(ํ|ํ) ํ(ํ) 1 ํฝ ( 1 ํ ) 1 ํฝ .(118) In this case, ํ ํฝ โ (ํ|ํ) is a valid probability distribution as long as: ํ 1 ํฝ = โซ ํ ํ ref (ํ|ํ)( ํ(ํ|ํ) ํ(ํ) ) 1 ํฝ ํํ.(119) Note that optimal solution to(116)exactly coincides with that of C-DPO(19). The formu- lation in(116), known as the Conditional Contrastive Alignment (CCA), which is essentially a combination of the Noise Contrastive Estimation (NCE) [GH10] and a special model parameteri- zation(115). This objective is first proposed by [Che+25b] for improving the generation quality of visual autoregressive models without relying on CFG. It should be noted that CCA is theoretically correct only whenํis fixed to a particular constant, and it must be tuned as a hyperparameter in practice. In contrast, we demonstrate that C-DPO fine-tuning recovers the same optimal solution without introducing this additional hyperparameter ํ. DTheoretical Analysis of the Equivalence between CFG and Weighted MCLR D.1 Proof of Theorem 3 In this section, we provide the proof for Theorem 3. To begin with, we introduce the following lemma: Lemma 1. Letํ(ํ)be the clean data distribution,ํ ํก (ํ)be the marginal distribution ofํ(ํก), andํ 0ํก (ํ ํก |ํ) be the transition density from ํ(0) to ํ(ํก) as defined in section 2.1, then โ ํ ํก log ํ ํก (ํ ํก ) = E ํ(ํ|ํ ํก ) โ ํ ํก log ํ 0ํก (ํ ํก |ํ) .(120) 31 Proof. โ ํ ํก log ํ ํก (ํ ํก ) = โ ํ ํก ํ ํก (ํ ํก ) ํ ํก (ํ ํก ) (121) = โ ํ ํก โซ ํ ํ(ํ)ํ 0ํก (ํ ํก |ํ)ํํ ํ ํก (ํ ํก ) (122) = โซ ํ ํ(ํ)ํ 0ํก (ํ ํก |ํ)โ ํ ํก ํ 0ํก (ํ ํก |ํ) ํ ํก (ํ ํก )ํ 0ํก (ํ ํก |ํ) ํํ(123) = โซ ํ ํ(ํ|ํ ํก )โ ํ ํก log ํ 0ํก (ํ ํก |ํ)ํํ(124) = E ํ(ํ|ํ ํก ) โ ํ ํก log ํ 0ํก (ํ ํก |ํ) .(125) This completes the proof.โก We now restate the main theorem and proceed with the proof. Theorem 3. For any time sampling distribution ํ(ํก) and weighting function ํค(ํก), the CFG-guided score ํ cfg (ํ ํก ,ํก, ํ) :=โ ํ ํก log ํ ํก (ํ ํก |ํ)+ ํ โ ํ ํก log ํ ํก (ํ ํก |ํ)โโ ํ ํก log ํ ํก (ํ ํก ) is the unique minimizer of a sample-adaptive weighted ELBO-approximated MCLR objective (21): ํ cfg (ยท) = arg min ํ ํฝ (ยท) E ํ,ํกโผํ(ํก), ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก)โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i + ํE ํ, ฬํ,ํกโผํ(ํก) ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 โ ํ ํก (ํ ํก | ฬํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i . (126) Proof. First, note optimization problem (126) is equivalent to: min ํ ํฝ (ยท) (1+ ํ)E ํ,ํกโผํ(ํก), ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) ํค(ํก)||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 (127) โ ํE ํ, ฬํ,ํกโผํ(ํก) ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) ํ ํก (ํ ํก | ฬํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i .(128) Note that (128) can be further simplified as: โํE ฬํ,ํกโผํ(ํก), ํโผํ(ํ), ํฅ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) ํ ํก (ํ ํก | ฬํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i .(129) Hence the original optimization problem (126) is equivalent to min ํ ํฝ (ยท) (1+ ํ)E ํ,ํกโผํ(ํก), ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) ํค(ํก)||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ ํE ํ,ํกโผํ(ํก), ํโผํ(ํ), ํฅ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) ํ ํก (ํ ํก |ํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i , (130) which coincides exactly with (24). 32 In order to solve(130), since the objective decomposes acrossํกandํ, the optimization can be performed independently for each pair(ํก, ํ). Hence, we can fix a specific pair ofํกandํ, and optimize the corresponding score ํ ํฝ (ํ, ํ, ํ) independently: min ํ ํฝ (ยท,ํก,ํ) (1+ ํ)E ํโผํ(ํ|ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) ||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ ํE ํโผํ(ํ), ํฅ ํก โผํ 0ํก (ํ ํก |ํ) h ํ ํก (ํ ํก |ํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i (131) , โ min ํ ํฝ (ยท,ํก,ํ) (1+ ํ)E ํ ํก โผํ ํก (ํ ํก |ํ), ํโผํ(ํ|ํ ํก ,ํ) ||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ ํE ํ ํก โผํ ํก (ํ ํก ), ํโผํ(ํ|ํ ํก ) h ํ ํก (ํ ํก |ํ) ํ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i (132) โ min ํ ํฝ (ยท,ํก,ํ) (1+ ํ)E ํ ํก โผํ ํก (ํ ํก |ํ), ํโผํ(ํ|ํ ํก ,ํ) ||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ ํE ํ ํก โผํ ํก (ํ ํก |ํ), ํโผํ(ํ|ํ ํก ) h โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 i . (133) From(131)to(132)we use the Bayes rule:ํ(ํ|ํ)ํ 0ํก (ํ ํก |ํ) = ํ(ํ, ํ ํก |ํ) = ํ ํก (ํ ํก |ํ)ํ(ํ|ํ ํก , ํ). From(132) to (133) we use the identity: ํ ํก (ํ ํก )ํ(ํ|ํ ํก ) ํ ํก (ํ ํก |ํ) ํ ํก (ํ ํก ) = ํ ํก (ํ ํก |ํ)ํ(ํ|ํ ํก ). Note that: E ํโผํ(ํ|ํ ํก ,ํ) ||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 =||ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ 2ํ ํฝ (ํ ํก ,ํก, ํ) ํ E ํโผํ(ํ|ํ ํก ,ํ) โ ํ ํก log ํ 0ํก (ํ ํก |ํ) + ํถ 1 =||ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ 2ํ ํฝ (ํ ํก ,ํก, ํ) ํ โ ํ ํก log ํ ํก (ํ ํก |ํ)+ ํถ 1 , (134) whereํถ 1 is a constant independent ofํฝ, and the second equality follows from Lemma 1. Similarly, we have E ํโผํ(ํ|ํ ํก ) ||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 =||ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ 2ํ ํฝ (ํ ํก ,ํก, ํ) ํ โ ํ ํก log ํ ํก (ํ ํก )+ ํถ 2 , (135) , where ํถ 2 is a constant independent ofํฝ. Substituting (134) and (135) into (133), the optimization problem becomes equivalent to: min ํ ํฝ (ยท,ํก,ํ) E ํ ํก โผํ ํก (ํ ํก |ํ) ||ํ ํฝ (ํ ํก ,ํก, ํ)|| 2 2 โ 2ํ ํฝ (ํ ํก ,ํก, ํ) ํ (1+ ํ)โ ํ ํก log ํ ํก (ํ ํก |ํ)โ ํโ ํ ํก log ํ ํก (ํ ํก ) (136) โ min ํ ํฝ (ยท,ํก,ํ) E ํ ํก โผํ ํก (ํ ํก |ํ) ||ํ ํฝ (ํ ํก ,ํก, ํ)โ ํ cfg (ํ ํก ,ํก, ํ)|| 2 2 + ํถ 3 , (137) where ํถ 3 is a constant independent ofํฝ. Therefore, the optimal solution is: ํ ํฝ โ (ํ ํก ,ํก, ํ) = ํ cfg (ํ ํก ,ํก, ํ)(138) =โ ํ ํก log ํ ํก (ํ ํก |ํ)+ ํ โ ํ ํก log ํ ํก (ํ ํก |ํ)โโ ํ ํก log ํ ํก (ํ ํก ) .(139) This completes the proof. โก 33 D.2Extensions: CFG Variants under the Alignment Framework The equivalence between CFG and weighted MCLR provides a unified perspective for interpreting CFG variants. This perspective is formalized in the following corollary. Corollary 3. Consider two distributionsํ + (ํ)andํ โ (ํ). For any time sampling distributionํ(ํก)and weighting function ํค(ํก), a generalized CFG-style score: ํ cfg-variant (ํ ํก ,ํก) :=โ ํ ํก log ํ + ํก (ํ ํก )+ ํ(โ ํ ํก log ํ + ํก (ํ ํก )โโ ํ ํก log ํ โ ํก (ํ ํก ))(140) is the unique minimizer of the following MCLR-style optimization problem: min ํ ํฝ (ยท) (1+ ํ)E ํกโผํ(ํก), ํโผํ + (ํ), ํ ํก โผํ 0ํก (ํ ํก |ํ) ํค(ํก)||โ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก)|| 2 2 โ ํE ํกโผํ(ํก), ํโผํ โ (ํ), ํฅ ํก โผํ 0ํก (ํ ํก |ํ) h ํค(ํก) ํ + ํก (ํ ํก ) ํ โ ํก (ํ ํก ) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก)โฅ 2 2 i (141) The proof is omitted, as it follows directly by adapting the proof of Theorem 3. The optimization problem (141) shares the same structure as the ELBO-approximated weighted MCLR (130), with the only difference that positive samples are drawn fromํ + (ํ)and negative samples fromํ โ (ํ). This formulation unifies a broad class of CFG-style methods, as illustrated below. โขStandard CFG. Letํ + (ํ) = ํ(ํ|ํ)andํ โ (ํ) = ํ(ํ). This recovers the standard CFG and corresponds to the weighted MCLR formulation discussed in previous sections. โข Autoguidance. Letํ + (ํ)be the distribution induced by a strong diffusion model andํ โ (ํ)the distribution induced by a weaker model. We recover the Autoguidance [Kar+24a]. โข Inference-Time Alignment Guidance. Letํ + (ํ)be the distribution induced by a diffusion model fine-tuned on high-reward data andํ โ (ํ)the distribution induced by the base model (or a low- reward fine-tuned model). We recover a family of inference-time alignment methods [Fra+25; Jin+25; Che+25c; JWL26]. Lastly, note that(141)without adaptive weighting is the ELBO-based approximation of the following likelihood-based objective, which is presented in Figure 3: max ํฝ E ํ + (ํ) log ํ ํฝ (ํ) + ํ E ํโผํ + (ยท),ํโผํ โ (ยท) log ํ ํฝ (ํ) ํ ํฝ (ํ) | z MCLR Regularization .(142) E Practical Implementation Details E.1 Approximating Log-Likelihood with ELBO Implementing MCLR requires access to the log-likelihood, which is not directly available for diffusion models. We therefore approximate the log-likelihood using the evidence lower bound (ELBO) in(6). In the following, we describe how this approximation is used in MCLR, C-DPO, and CCA. Before doing so, we state the following fact. 34 Equivalence between Score Function and MMSE Denoisers. For practical diffusion models, the drift coefficientํ(ยท)in(1)takes the formํ(ํ,ํก) = ํ(ํก)ํwhereํ(ยท):Rโ R. As a result, the corresponding transition distribution is Gaussian and can be written as [Kar+22]: ํ 0ํก (ํ ํก |ํ) =ํฉ(ํ ํก ;ํ (ํก)ํ, ํ 2 (ํก)ํ 2 (ํก)ํฐ)(143) , whereํฉ(ํ;ํ,ํบ)denotes the Gaussian density with meanํand covarianceํบevaluated atํ, ํ (ํก) = exp( โซ ํก 0 ํ(ํ)ํํ) andํ(ํก) = q โซ ํก 0 ํ 2 (ํ) ํ 2 (ํ) ํํ. The score of this transition distribution is therefore given by: โ ํ ํก log ํ 0ํก (ํ ํก |ํ) = ํ (ํก)ํโ ํ ํก ํ 2 (ํก)ํ 2 (ํก) .(144) Accordingly, the score network can be parameterized in terms of a denoiserํ ํฝ (ยท) as: ํ ํฝ (ํ,ํก, ํ) = ํ ํฝ (ํ ํก ;ํ 2 (ํก)ํ(ํก), ํ)โ ํ ํก ํ 2 (ํก)ํ 2 (ํก) .(145) Without loss of generality, we set ํ (ํก) = 1, under which the DSM objective in (3) becomes: 1 2 โซ ํ 0 E ํ,ํ(ํ|ํ),ํ 0ํก (ํ ํก |ํ) [ํค(ํก)โฅ ํ ํฝ (ํ ํก ; ํ(ํก), ํ)โ ํ ํ 2 (ํก) โฅ 2 2 ]ํํก = ฬ ํค(ํก)E ํ,ํกโผํฐ[0,ํ],ํ(ํ|ํ),ํ 0ํก (ํ ํก |ํ) ||ํ ํฝ (ํ ํก ; ํ(ํก), ํ)โ ํ|| 2 2 , (146) where ฬ ํค(ํก) = ํํค(ํก) 2ํ 4 (ํก) This shows that DSM is equivalent to training the MMSE denoiserํ ํฝ (ยท;ํ(ํก), ํ) for data from class ํ corrupted by additive Gaussian noise with standard deviation ํ(ํก). In this setting, the score function is related to the MMSE denoiser via Tweedieโs formula [Miy+61]: โ ํ log ํ ํก (ํ | ํ) = ํ(ํ; ํ(ํก), ํ)โ ํ ํ 2 (ํก) ,(147) whereํ(ํ;ํ(ํก), ํ)denotes the MMSE denoiser andํ ํก (ํ|ํ) = โซ ํ 0ํก (ํ|ํ 0 )ํ data (ํ 0 )ํํ 0 is the marginal distribution at timeํก. We are now ready to present the practical ELBO-approximated objectives of MCLR, C-DPO, and CCA for diffusion models. ELBO-Approximated MCLR for Diffusion Models. Substitute ELBO(6)into(11), the MCLR regularized DSM becomes: min ํฝ ํฅ DSM (ํฝ; ํ 2 (ยท)) + ํE ํ, ฬํ,ํกโผํฐ[0,ํ] ํ(ํ|ํ), ํ 0ํก (ํ ํก |ํ) h ํ 2 (ํก) โฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 โโฅโ ํ ํก log ํ 0ํก (ํ ํก | ํ)โ ํ ํฝ (ํ ํก ,ํก, ฬํ)โฅ 2 2 i . (148) Using the denoiser parameterization of the score network, together with a customized training-time noise sampling distribution ํ(ํก) and adaptive weighting ํค(ํก), the MCLR objective becomes: E ํ, ฬํ,ํกโผํ(ํก) ํ(ํ|ํ), ํ 0ํก (ํ ํก |ํ) h ํค(ํก) โฅํโํ ํฝ (ํ ํก ; ํ(ํก), ํ)โฅ 2 2 โ โฅํโํ ํฝ (ํ ํก ; ํ(ํก), ฬํ)โฅ 2 2 i , (149) which can be approximated with Monte Carlo sampling during training. Rather than parameterizing the noise level through the time variableํก โผ ํ(ํก), one may equivalently define a training noise 35 distribution directly overํ, denoted byํ(ํ)and a corresponding noise adaptive weightingํค(ํ). Under this formulation, (149) can be rewritten as: E ํ, ฬํ,ํโผํฉ(0,ํฐ) ํโผํ(ํ),ํ(ํ|ํ) h ํค(ํ) โฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 โ โฅํโํ ํฝ (ํ+ ํํ; ํ, ฬํ)โฅ 2 2 i . (150) As discussed in section G.1, combining DSM with MCLR regularization does not lead to benefit, hence we directly fine-tune pretrained model using (150). ELBO-Approximated C-DPO for Diffusion Models. The C-DPO algorithm requires access to the log-likelihood ratio for individual data point. Similar to MCLR, we approximate it with the ELBO: log ํ ํฝ (ํ) ํ ref (ํ) โ ํ 2 E ํกโผํฐ[0,ํ],ํ 0ํก (ํ ํก |ํ) h ํ 2 (ํก) โโฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ํฝ (ํ ํก ,ํก, ํ)โฅ 2 2 +(151) โฅโ ํ ํก log ํ 0ํก (ํ ํก |ํ)โ ํ ref (ํ ํก ,ํก, ํ)โฅ 2 2 i .(152) Using the denoiser parameterization of the score network, together with a customized training noise distribution and adaptive weighting, the log-likelihood ratio takes the following form: log ํ ํฝ (ํ) ํ ref (ํ) โ E ํโผํ train (ํ),ํโผํฉ(0,ํฐ) ํค(ํ)(โโฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 +โฅํโํ ref (ํ+ ํํ; ํ, ํ)โฅ 2 2 ) . (153) Define: ฮ(ํ, ํ,ํ, ํ) :=โฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 โโฅํโํ ref (ํ+ ํํ; ํ, ํ)โฅ 2 2 .(154) By substituting (153) into DPO objective (19), we get the following optimization problem: min ํฝ โE ํ, ฬํ,ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ| ฬํ) log Sigmoid(ํฝE ํ,ํ [ํค(ํ)(โฮ(ํ ํค , ํ,ํ, ํ)+ ฮ(ํ ํ , ํ,ํ, ํ))]) . (155) Since the log-Sigmoid function is concave, applying Jensenโs inequality by moving the expecta- tion outside yields the following upper bound of objective (155): min ํฝ โE ํ, ฬํ,ํ ํค ,ํ ํ ,ํ,ํ log Sigmoid(ํฝํค(ํ)(โฮ(ํ ํค , ํ,ํ, ํ)+ ฮ(ํ ํ , ํ,ํ, ํ))) ,(156) which serves as our final training objective. CCA for Diffusion Models. The Conditional Contrastive Alignment (CCA) [Che+25b] objective takes the following form: max ํฝ E ํ,ํ(ํ|ํ) log Sigmoid ํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) +ํE ํ, ฬํ,ํ(ํ| ฬํ) log Sigmoid โํฝ log ํ ํฝ (ํ|ํ) ํ ref (ํ|ํ) .(157) To adapt this optimization problem to diffusion models, we again approximate the log-likelihood with ELBO, resulting in the following optimization problem: max ํฝ E ํ, ฬํ,ํ ํค โผํ(ํ|ํ),ํ ํ โผํ(ํ| ฬํ) log Sigmoid ํฝE ํ,ํ โํค(ํ)ฮ(ํ ํค , ํ,ํ, ํ) +ํ log Sigmoid ํฝE ํ,ํ ํค(ํ)ฮ(ํ ํ , ํ,ํ, ํ) . (158) Since the log-sigmoid function is concave, applying Jensenโs inequality by moving the expectation outside yields the following lower bound of objective (158): max ํฝ E ํ,ํ ํค ,ํ ํ ,ํ,ํ log Sigmoid โํฝํค(ํ)ฮ(ํ ํค , ํ,ํ, ํ) +ํ log Sigmoid ํฝํค(ํ)ฮ(ํ ํ , ํ,ํ, ํ) , (159) which serves as our final training objective. 36 E.2 Building Training Data from a Minibatch Computing MCLR requires constructing contrastive tuples(ํ, ํ, ฬํ), whereํ โผ ํ(ํ|ํ)and ฬํdenotes a randomly chosen alternative class. Similarly, C-DPO and CCA require constructing preference- style tuples(ํ ํค , ํ ํ , ํ). In practice, we build these tuples directly from each training minibatch (ํ ํ , ํ ํ ) ํ ํ=1 . We consider two strategies, described below. Approach 1: Buildingํpairs. The simplest strategy constructs one contrastive (or preference) tuple per sample in the minibatch. For MCLR, given a sample(ํ, ํ), we randomly select another label ฬํfrom the same minibatch, forming a tuple(ํ, ํ, ฬํ). Repeating this process independently for eachํ(with replacement) yieldsํtuples in total. The same strategy applies to C-DPO and CCA, where we construct ํ tuples(ํ ํค , ํ ํ , ํ) from the minibatch. Approach 2: Building ํํพ pairs. Alternatively, we can construct multiple contrastive tuples per sample. For MCLR, given each(ํ, ํ), we randomly selectํพalternative labels ฬํ ํ ํพ ํ=1 from the same minibatch, yieldingํพtuples(ํ, ํ, ฬํ ํ ). Repeating this procedure for allํsamples results in ํํพtuples in total. An analogous strategy is applied to C-DPO and CCA, producingํํพtuples (ํ ํค , ํ ํ , ํ) per minibatch. Compared to Approach 1, Approach 2 increases the number of training tuples constructed from each minibatch, allowing better exploitation of inter-class contrastive information. In our experi- ments, we adopt Approach 2 for EDM-based diffusion models, where it consistently yields improved quantitative performance. For VAR models, we use Approach 1 due to its lower computational overhead and comparable empirical performance. E.3 Overall Algorithm We are now ready to present the overall algorithm. The algorithm for MCLR is given in algorithm 1, while the algorithms for C-DPO and CCA are presented in algorithm 2. E.4 Hyperparameters For each method, we carefully tune hyperparameters using grid search to ensure the best per- formance. The full set of hyperparameter configurations will be made publicly available upon publication. F Additional Experimental Results In this section, we present additional experimental results that complement and extend those in the main text. In section 6, we focus on validating the following three claims: โขMCLR induces progressive class separation, leading to a fidelityโdiversity trade-off analogous to that produced by increasing the guidance strength in classifier-free guidance (CFG). โข MCLR matches or outperforms training-time baselines, including C-DPO and CCA. โข MCLR achieves performance comparable to CFG. Due to space constraints, the main text reports quantitative results only for ImageNet-512ร512 with EDM2-L and ImageNet-256ร256 with VAR-d24. Here, we provide the complete set of experi- mental results on ImageNet-64ร64 with EDM2-S, ImageNet-512ร512 with EDM2-L, and ImageNet- 256ร256 with VAR-d24. 37 Algorithm 1 MCLR Require:Pre-trained base modelํ ํฝ 0 ; noise-adaptive weight functionํค(ยท); training noise schedule ํ(ํ); learning rateํผ; training dataset representingํ(ํ, ํ);ํพ โฅ1 (number of mismatched class labels per sample in Approach 2) 1: Initializeํฝโํฝ 0 2: for each training iteration do 3: Sample a minibatch (x ํ , c ํ ) ํ ํ=1 with(ํ ํ , ํ ํ )โผ ํ(ํ, ํ). 4: Sampleํ ํ ํ ํ=1 with ํ ํ โผ ํ(ํ) 5: Sampleํ ํ ํ ํ=1 withํ ํ โผํฉ(0, ํฐ) 6: Construct contrastive tuples from the minibatch: 7: if Approach 1 then 8:Set ํ โ ํ 9:For each ํ, sample ฬํ ํ from minibatch labels with ฬํ ํ โ ํ ํ 10:Define tuples(ํ ํ , ํ ํ , ฬํ ํ , ํ ํ ,ํ ํ ) ํ ํ=1 11: end if 12: if Approach 2 then 13:Set ํ โ ํํพ 14:For each ํ and each ํ = 1, ... , ํพ, sample ฬํ ํ,ํ from minibatch with ฬํ ํ,ํ โ ํ ํ 15:Define tuples(ํ ํ,ํ , ํ ํ,ํ , ฬํ ํ,ํ , ํ ํ,ํ ,ํ ํ,ํ ) where (ํ ํ,ํ , ํ ํ,ํ ) =(ํ ํ , ํ ํ ) and(ํ ํ,ํ ,ํ ํ,ํ ) =(ํ ํ ,ํ ํ ) 16: end if 17: โโ 0 18: for each tuple(ํ, ํ, ฬํ, ํ,ํ) do 19: โ+= ํค(ํ) โฅํ ํฝ (ํ+ ํํ; ํ, ํ)โ ํโฅ 2 2 โโฅํ ํฝ (ํ+ ํํ; ํ, ฬํ)โ ํโฅ 2 2 20: end for 21:ํฝโํฝโ ํผโ ํฝ โ 22: end for 23: returnํ ํฝ F.1 Progressive Class Separation and the Fidelity-Diversity Trade-off. Training with MCLR induces notable progressive class separation, which gives rise to a fidelityโ diversity trade-off similar that observed when increasing the guidance scale in CFG. This behavior is quantitatively supported by the following observations. (i) Increased Precision and Inception Score. As shown in Figure 5(b,d,g) for ImageNet-64, Fig- ure 7(c,f,i) for ImageNet-256 and Figure 6(b,d,g) for ImageNet-512, continued training with MCLR leads to a progressive increase in Inception Score, indicating increasingly class-discriminative gen- erations. This is accompanied by a corresponding increase in Precision, reflecting improved image fidelity. (i) Decreased Recall. Conversely, as shown in Figure 5(e,h), Figure 6(g,j), and Figure 7(e,h), excessive training reduces within-class diversity, which manifests as a decrease in Recall (see also Figure 4(e)). Taken together, these effects result in FDโIS and PrecisionโRecall trade-offs that closely mirror those induced by CFG, as illustrated in Figure 5(c,f,i), Figure 6(c,f,i), and Figure 7(d,e,h,k). Quali- tatively, as shown in Figures 2 and 8 to 19, continued MCLR training leads to gradually enhanced distinct, class-specific visual characteristics in generated samples. 38 F.2 MCLR Outperforms Training-time Baselines. Diffusion Models. As shown in Figures 5 and 6, for EDM models trained on ImageNet, MCLR achieves substantially better best-case FD DINOv2 scores than CCA, C-DPO. Moreover, MCLR traverses significantly wider FDโIS and PrecisionโInception trade-offs region with a faster training speed. In contrast, CCA and C-DPO often converge early, exhibit zigzag training trajectories, and become trapped in suboptimal local minima, as evidenced by slowly improving or stalled learning curves. Autoregressive Models. As shown in Figure 7, MCLR, C-DPO, and CCA yield comparable performance improvement on VAR-d24. Specifically, we see MCLR consistently achieves higher precision (see Figure 7(f,h,i,k)) and inception score (see Figure 7(c)) in later stages of training. F.3 MCLR Achieves Comparable Performance as CFG. Diffusion Models. For EDM models trained on ImageNet, CFG generally exhibits a better FDโIS trade-off and achieves a lower best-case FD DINOv2 than MCLR. Nevertheless, this gap is moderate for EDM2-L model, where CFG attains a best-case FD DINOv2 of 39.86 compared to 42.50 for MCLR as shown in Figure 6(c). Moreover, for EDM2-L model, when evaluated using PrecisionโRecall, MCLR demonstrates competitive and in some regimes superior behavior. As shown in Figure 6(f,g), MCLR matches CFG in the high-recall regime (early training stages) and achieves a substantially higher best-case precision in the high-precision regime (later training stages), where CFG begins to generate images with oversaturated colors. Applying CFG on top of an MCLR-fine-tuned model further alleviates the performance gap between MCLR and CFG, yielding higher best-case Inception Score and Precision on both EDM2-S and EDM2-L models, as shown in Figure 5(c,f,i) and Figure 6(c,f,i). Qualitatively, MCLR and CFG often produce similar effects, both significantly enhancing class- specific structures in the generated images, as shown in Figures 2 and 8 to 19. This similarity is expected, as both methods improve conditional modeling through inter-class contrastive signals. The key distinction is that MCLR internalizes this mechanism during training, whereas CFG applies it at inference time. Autoregressive Models. For VAR-d24 model, MCLR achieves a similar FDโIS trade-off to CFG in terms of both FD DINOv2 and FID, with CFG exhibiting a slightly better best-case FID. Consistent with diffusion models, MCLR outperforms CFG in the PrecisionโRecall trade-off and achieves a higher best-case precision. G Ablation Study G.1 MCLR does not Benefit from DSM. In section 3.2 we motivate MCLR as an additional regularization term applied on top of the standard denoising score matching (DSM) objective (or maximum likelihood estimation) as in(11), or alternatively on top of the KL-divergence-based objective in(13). While the latter requires drawing samples from the base model during training, which can be computationally prohibitive, it is natural to consider combining DSM with MCLR as in(11). Intuitively, one may expect the DSM objective to mitigate excessive effects of fine-tuning, such as mode collapse, that are commonly observed in post-training objectives [Liu+25]. 39 (b) (e) (c) (f) (a) (d) ImageNet-64 EDM2-S (g)(h)(i) Figure 5: Quantitative Results for EDM2-S trained on ImageNet-64ร64. (a), (b), (d), (e), (g), (h) show the evolution of FD, Inception Score, Precision (calculated with Inception features), and Recall (calculated with Inception features), Precision (calculated withDINOv2features), and Recall (calculated withDINOv2features), respectively, as functions of training iterations. (c) shows the FDโIS trade-offs, while (f), (i) depict the PrecisionโRecall trade-offs calculated with Inception andDINOv2features respectively. We evaluate classifier-free guidance (CFG) scales ํพโ0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 0.9, 1, 1.5, 2.0, 3.0. However, perhaps surprisingly, we find that combining DSM with MCLR does not yield perfor- mance improvements. As shown in Figure 20, adding an explicit DSM term consistently harms the best achievable performance. Concretely, we fine-tune an EDM2-S base model on ImageNet-64ร64 using the following objective with varying ํฝ: ํฝ E ํ,ํโผํฉ(0,ํฐ) ํโผํ(ํ),ํ(ํ|ํ) h ํค(ํ) โฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 i | z DSM Objective + E ํ, ฬํ,ํโผํฉ(0,ํฐ) ํโผํ(ํ),ํ(ํ|ํ) h ํค(ํ) โฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 โ โฅํโํ ํฝ (ํ+ ํํ; ํ, ฬํ)โฅ 2 2 i | z MCLR Regularization . (160) Empirically, as shown in Figure 20, increasingํฝleads to slower convergence and degrades both FD DINOv2 and Inception Score. As a result, throughout the paper we fine-tune diffusion models using the MCLR regularization alone. This choice not only yields strong empirical performance but 40 (b) (e) (c) (f) (a) (d) ImageNet-512 EDM2-L (g)(h)(i) Figure 6: Quantitative Results for EDM2-L trained on ImageNet-512ร512. (a), (b), (d), (e), (g), (h) show the evolution of FD, Inception Score, Precision (calculated with Inception features), and Recall (calculated with Inception features), Precision (calculated withDINOv2features), and Recall (calculated withDINOv2features), respectively, as functions of training iterations. (c) shows the FDโIS trade-offs, while (f), (i) depict the PrecisionโRecall trade-offs calculated with Incpetion andDINOv2features respectively. We evaluate classifier-free guidance (CFG) scales ํพโ0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 0.9, 1, 1.5, 2.0, 3.0. also simplifies hyperparameter tuning, as MCLR introduces only a single hyperparameter, i.e., the learning rate. G.2 Hyperparameter Sensitivity of C-DPO The hyperparameterํฝin the C-DPO objective(19)controls the strength of the KL regularization between the base model and the fine-tuned model. We study its effect on EDM2-S trained on ImageNet-64ร64. As shown in Figure 21, C-DPO exhibits relatively stable performance across a wide range ofํฝvalues. In particular, smallerํฝvalues achieve marginally better best-case FD DINOv2 and Inception Score. G.3 Hyperparameter Sensitivity of CCA The hyperparameterํin the CCA objective(159)controls the strength of the contrastive term, with largerํplacing greater emphasis on penalizing the non-preferred sampleํ ํ . We study its effect on EDM2-S trained on ImageNet-64ร64, fixingํฝ =0.001. As shown in Figure 22,ํhas little impact on the best achievable FD DINOv2 ; however, smallerํvalues lead to more stable and 41 smoother convergence. In contrast, largerํvalues can yield slightly higher best-case Inception Scores, but often cause training instability and eventual collapse when training is prolonged. In practice, jointly tuningํฝandํis challenging. Although CCA and C-DPO share the same theoretical optimal solution, C-DPO requires tuning only a single hyperparameterํฝ. Empirically, CCA does not outperform C-DPO under careful tuning as shown in Figure 22. Therefore, we recommend C-DPO over CCA in practice. H Discussion on Related Works Our work contributes to a growing line of research [Che+25b; Che+25a; Tan+25] that seeks to induce CFG-like effect by modifying the standard training objective, rather than applying CFG at inference time. The most closely related approach is CCA [Che+25b], which aims to learn the gamma-powered distribution(20)in autoregressive models using Noise Contrastive Estimation (NCE) [GH10]. While CCA also considers DPO as a baseline and reports inferior performance, our theoretical analysis demonstrates that C-DPO and CCA admit the same optimal solution, and are therefore fundamentally equivalent at the population level. With a correct implementation, we find that C-DPO consistently matches or exceeds the empirical performance of CCA while requiring fewer hyperparameters and simpler optimization. We further extend CCA to diffusion models by approximating log-likelihoods via the ELBO and include this variant as a baseline in our experiments. A detailed theoretical and empirical analysis of CCA is provided in section C. Another relevant direction is Guidance-Free Training (GFT) [Che+25a; Tan+25], which aims to directly train diffusion models to reproduce CFG-induced score functions. GFT relies on a reparameterization heuristic and inherits the same theoretical ambiguities associated with CFG itself. In contrast, the proposed MCLR objective provides a clearer mechanistic interpretation of CFG. Besides these approaches, several works sought to enhance conditional generative modeling through class-wise contrastive mechanisms. [Yan+24] introduce a contrastive objective to im- prove the modeling of tail classes; [Lee+25] employ an InfoNCE-style loss [OLV18] to strengthen text-image alignment in diffusion models; [KMS24] and [YAA25] regularize diffusion models by encouraging class separation in feature space. In contrast to MCLR, C-DPO and CCA, these ap- proaches are primarily heuristic and do not provide a theoretical characterization of the conditional distribution induced by their objectives. Lastly, beyond class-wise contrast, another complementary strategy for improving generative modeling is to contrast high-quality (โrealโ) data against low-quality (โsyntheticโ) samples. Zheng et al. [Zhe+25] propose Direct Discriminative Optimization (DDO), which employs the same contrastive objective as CCA, but replaces class-conditioned contrast with a realโsynthetic discrimi- nation signal. We include DDO as one of our baselines. 42 Algorithm 2 C-DPO and CCA Require:Pre-trained base modelํ ํฝ 0 ; noise-adaptive weight functionํค(ยท); training noise schedule ํ(ํ); learning rateํผ; KL regularization strengthํฝ; CCA hyperparameterํ; training dataset representing ํ(ํ, ํ); ํพโฅ 1 (number of mismatched class labels per sample in Approach 2) 1: Initializeํฝโํฝ 0 ,ํ ref โํ ํฝ 0 2: for each training iteration do 3: Sample a minibatch (x ํ , c ํ ) ํ ํ=1 with(ํ ํ , ํ ํ )โผ ํ(ํ, ํ). 4: Sampleํ ํ ํ ํ=1 with ํ ํ โผ ํ(ํ) 5: Sampleํ ํ ํ ํ=1 withํ ํ โผํฉ(0, ํฐ) 6: Construct contrastive tuples from the minibatch: 7: if Approach 1 then 8:Set ํ โ ํ 9:For each ํ, sample ฬํ ํ from minibatch samples with label ฬํ ํ โ ํ ํ 10:Define tuples(ํ ํ , ฬํ ํ , ํ ํ , ํ ํ ,ํ ํ ) ํ ํ=1 11: end if 12: if Approach 2 then 13:Set ํ โ ํํพ 14: For eachํand eachํ =1, ... , ํพ, sample ฬํ ํ,ํ from minibatch samples with label ฬํ ํ,ํ โ ํ ํ 15:Define tuples(ํ ํ,ํ , ฬํ ํ,ํ , ํ ํ,ํ , ํ ํ,ํ ,ํ ํ,ํ ) where (ํ ํ,ํ , ํ ํ,ํ ) =(ํ ํ , ํ ํ ) and(ํ ํ,ํ ,ํ ํ,ํ ) =(ํ ํ ,ํ ํ ) 16: end if 17: โโ 0 18: for each tuple(ํ, ฬํ, ํ, ํ,ํ) do 19:ฮ(ํ ํค , ํ,ํ, ํ) :=โฅํโํ ํฝ (ํ+ ํํ; ํ, ํ)โฅ 2 2 โโฅํโํ ref (ํ+ ํํ; ํ, ํ)โฅ 2 2 20:ฮ(ํ ํ , ํ,ํ, ํ) :=โฅ ฬํโํ ํฝ ( ฬํ+ ํํ; ํ, ํ)โฅ 2 2 โโฅ ฬํโํ ref ( ฬํ+ ํํ; ํ, ํ)โฅ 2 2 21:if C-DPO then 22:โ+=โ log Sigmoid(ํฝํค(ํ)(โฮ(ํ ํค , ํ,ํ, ํ)+ ฮ(ํ ํ , ํ,ํ, ํ))) 23:end if 24:if CCA then 25:โโ= log Sigmoid โํฝํค(ํ)ฮ(ํ ํค , ํ,ํ, ํ) +ํ log Sigmoid ํฝํค(ํ)ฮ(ํ ํ , ํ,ํ, ํ) 26:end if 27: end for 28:ํฝโํฝโ ํผโ ํฝ โ 29: end for 30: returnํ ํฝ 43 (b) (g) (d) (h) (a) (f) ImageNet-256 VAR-d24 (i)(j)(k) (c)(e) Figure 7: Quantitative Results for VAR-d24 trained on ImageNet-64ร64. (a), (b), (c), (f), (g), (i), (j) show the evolution of FD DINOv2 , FID, Inception Score, Precision (calculated with Inception features), and Recall (calculated with Inception features), Precision (calculated withDINOv2 features), and Recall (calculated withDINOv2features), respectively, as functions of training iterations. (d), (e) shows the FDโIS trade-offs, while (h), (k) depict the PrecisionโRecall trade-offs calculated with Inception andDINOv2features respectively. We evaluate classifier-free guidance (CFG) scales ํพโ0.5, 0.8, 1.1, 1.5, 1.7, 2.0, 2.5, 3.0, 4.0, 5.0, 7.0, 10.0, 15.0. 44 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters= 784Iters= 224Iters= 504 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters = 1792Iters = 448Iters = 896 Increasing CFG Scale ( ํพ )CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 64x64 EDM2-S ImageNet 512x512 EDM2-L (a) (b) Figure 8: Comparison between CFG and MCLR for Golden Retriever (Class 207). (a,b) demon- strate the progressive evolution of generated samples on ImageNet-64ร64 and ImageNet-512ร512, respectively. Increasing the CFG scaleํพ(top rows) and progressive MCLR training (bottom rows) produce similar effects, both enhancing class-specific structures in the generated images. 45 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters= 784Iters= 224Iters= 504 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 64x64 EDM2-S (a) ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters = 1792Iters = 448Iters = 896 Increasing CFG Scale ( ํพ )CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 512x512 EDM2-L (b) Figure 9: Comparison between CFG and MCLR for Warthog (Class 343). (a,b) demonstrate the progressive evolution of generated samples on ImageNet-64ร64 and ImageNet-512ร512, respectively. Increasing the CFG scaleํพ(top rows) and progressive MCLR training (bottom rows) produce similar effects, both enhancing class-specific structures in the generated images. 46 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters= 784Iters= 224Iters= 504 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters = 1792Iters = 448Iters = 896 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 64x64 EDM2-S ImageNet 512x512 EDM2-L (a) (b) Figure 10: Comparison between CFG and MCLR for Trumpet (Class 513). (a,b) demonstrate the progressive evolution of generated samples on ImageNet-64ร64 and ImageNet-512ร512, respectively. Increasing the CFG scaleํพ(top rows) and progressive MCLR training (bottom rows) produce similar effects, both enhancing class-specific structures in the generated images. 47 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters= 784Iters= 224Iters= 504 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters = 1792Iters = 448Iters = 896 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 64x64 EDM2-S ImageNet 512x512 EDM2-L (a) (b) Figure 11: Comparison between CFG and MCLR for Cowboy Hat (Class 515). (a,b) demonstrate the progressive evolution of generated samples on ImageNet-64ร64 and ImageNet-512ร512, re- spectively. Increasing the CFG scaleํพ(top rows) and progressive MCLR training (bottom rows) produce similar effects, both enhancing class-specific structures in the generated images. 48 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters= 784Iters= 224Iters= 504 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ํพ = 0.3 ํพ = 0.7 ํพ = 1 Iters = 1792Iters = 448Iters = 896 Increasing CFG Scale ( ํพ ) CFG MCLR Training Iterations (K images) MCLR Base model Base model ํพ = 0 Iters= 0 ImageNet 64x64 EDM2-S (a) ImageNet 512x512 EDM2-L (b) Figure 12: Comparison between CFG and MCLR for Lab Coat (Class 617). (a,b) demonstrate the progressive evolution of generated samples on ImageNet-64ร64 and ImageNet-512ร512, respectively. Increasing the CFG scaleํพ(top rows) and progressive MCLR training (bottom rows) produce similar effects, both enhancing class-specific structures in the generated images. 49 Base Model Iters = 02244486728401008 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ ImageNet 64x64 Figure 13: Comparison between CFG and MCLR. Left and right figures demonstrate the progressive evolution of generated samples for MCLR and CFG on ImageNet-64ร64, respectively, with all images initialized from the same random noise. Increasing the CFG scaleํพand progressive MCLR training produce similar effects, both enhancing class-specific structures in the generated images. Base Model Iters = 02244486728401008 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ ImageNet 64x64 Figure 14: Comparison between CFG and MCLR. Same as Figure 13, but with a different initial random noise. 50 Base Model Iters = 0 2244486728401008 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ ImageNet 64x64 Figure 15: Comparison between CFG and MCLR. Same as Figure 13, but with a different initial random noise. MCLR Training Iterations (K images) Class 350 Class 456 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ Base Model Iters = 0 28056089612321288 ImageNet 512x512 Class 96 Class 299 Figure 16: Comparison between CFG and MCLR. Same as Figure 13, but for ImageNet-512x512 with a different initial random noise. 51 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ Base Model Iters = 028056089612321512 Figure 17: Comparison between CFG and MCLR. Same as Figure 13, but for ImageNet-512x512 with a different initial random noise. Base Model Iters = 028056089612321512 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ ImageNet 512x512 Figure 18: Comparison between CFG and MCLR. Same as Figure 13, but for ImageNet-512x512 with a different initial random noise. 52 MCLR Training Iterations (K images) Class 31 Class 96 Class 513 Class 515 Class 207 Base Model ํธ=ํ Increasing CFG Scale (ํธ) ํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํํธ=ํ.ํ Base Model Iters = 0 28056089612321512 ImageNet 512x512 Figure 19: Comparison between CFG and MCLR. Same as Figure 13, but for ImageNet-512x512 with a different initial random noise. (a)(b) Figure 20: Effects of Adding DSM to MCLR. (a,b) compare fine-tuning diffusion models using MCLR alone (ํฝ =0) with variants that additionally include a DSM loss, evaluated on the EDM2-S model trained on ImageNet-64ร64. Performance is reported in terms of FD DINOv2 and Inception Score. Adding an explicit DSM term does not improve performance and but results in slower training speed and degrades the best achievable scores. 53 (a)(b) Figure 21: Effects ofํฝin C-DPO. (a,b) compare fine-tuning diffusion models using C-DPO with differentํฝevaluated on the EDM2-S model trained on ImageNet-64ร64. Performance is reported in terms of FD DINOv2 and Inception Score. C-DPO achieves stable performance under differentํฝ, with a smaller ํฝ leads to marginally better performance on selected metrics. (a)(b) Figure 22: Effects ofํin CCA. (a,b) compare fine-tuning diffusion models using CCA with different ํevaluated on the EDM2-S model trained on ImageNet-64ร64. Performance is reported in terms of FD DINOv2 and Inception Score. Note with the sameํฝ, C-DPO achieves comparable performance as CCA while at the same time has less hyperparameters. 54