Paper deep dive
SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration
Renye Yan, Jikang Cheng, You Wu, Bojin Huang, Wei Peng, Zongwei Wang, Ling Liang, Yimao Cai
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/12/2026, 2:53:30 AM
Summary
The paper introduces SwiftExplorer, a training-free plugin for diffusion models that addresses distribution collapse and inefficient guidance. It employs an Inheritance-Restart exploration mechanism to maintain diversity and a Quality-Efficiency arbitration mechanism to dynamically stop generation when marginal reward gains are insufficient, thereby improving alignment, diversity, and computational efficiency.
Entities (7)
Relation Signals (5)
SwiftExplorer → uses → Inheritance-Restart
confidence 95% · SwiftExplorer introduces an Inheritance-Restart exploration mechanism that prevents premature convergence
SwiftExplorer → uses → Quality-Efficiency Arbitration
confidence 95% · SwiftExplorer employs a Quality-Efficiency screening mechanism to filter out ineffective signals
Inheritance-Restart → improves → Diversity
confidence 90% · Inheritance-Restart exploration mechanism... increases the likelihood of high-reward trajectories... balances diversity and fidelity
SwiftExplorer → outperforms → DyMO
confidence 90% · SwiftExplorer... outperforms existing baselines in computational efficiency... DyMO yield more compact clustering
Quality-Efficiency Arbitration → reduces → Compute Costs
confidence 90% · reduces computation by dynamically stopping generation when completeness and marginal reward gain are optimal
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Diffusion models have general generative abilities but struggle to align with specific objectives. Fine-tuning can improve alignment, yet its training cost is often prohibitive. This led to training-free methods that apply objective-guided terms in sampling to bias the generation distribution toward designated regions, e.g., high-reward areas. However, these methods face two issues: (1) the strong directional bias narrows the pretrained distribution and generation diversity, and (2) indiscriminate constant guidance fails to prune redundant signals, hurting both quality and efficiency. To address the above challenges, we propose SwiftExplorer, a plugin that mitigates distribution collapse caused by excessive diversity loss and reduces compute costs. First, we adopt an Inheritance-Restart exploration mechanism to avoid early convergence, while exploration also increases the likelihood of high-reward trajectories. Additionally, it balances diversity and fidelity, adding diversity without causing a distribution over-shift. Second, our Quality-Efficiency arbitration mechanism improves guidance by removing incorrect signals, and it reduces computation by dynamically stopping generation when completeness and marginal reward gain are optimal. In an extensive number of experiments and different types of evaluation metrics, the proposed SwiftExplorer achieves excellent performance on all metrics, including preference, fidelity, diversity, and richness.
Tags
Links
- Source: https://arxiv.org/abs/2609.06651v1
- Canonical: https://arxiv.org/abs/2609.06651v1
Trouble viewing inline? Open PDF directly →
Full Text
73,057 characters extracted from source content.
Expand or collapse full text
SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration CCS: Computing methodologies Computer vision problemsCCS: Generation Optimization Renye Yan Note: Renye Yan and Jikang Cheng contributed equally to this work. Affiliation: Peking University, Beijing, China , Jikang Cheng Affiliation: Peking University, Beijing, China , You Wu Affiliation: Nanjing University, Nanjing, China , Baojin Huang Affiliation: Huazhong Agricultural University, Wuhan, China , Wei Peng Affiliation: Stanford University, Stanford, California, USA , Zongwei Wang Affiliation: Peking University, Beijing, China , Ling Liang Note: Ling Liang and Yimao Cai are the corresponding authors. Affiliation: Peking University, Beijing, China and Yimao Cai Affiliation: Peking University, Beijing, China Abstract. Diffusion models have general generative abilities but struggle to align with specific objectives. Fine-tuning can improve alignment, yet its training cost is often prohibitive. This led to training-free methods that apply objective-guided terms in sampling to bias the generation distribution toward designated regions, e.g., high-reward areas. However, these methods face two issues: (1) the strong directional bias narrows the pretrained distribution and generation diversity, and (2) indiscriminate constant guidance fails to prune redundant signals, hurting both quality and efficiency. To address the above challenges, we propose SwiftExplorer, a plugin that mitigates distribution collapse caused by excessive diversity loss and reduces compute costs. First, we adopt an Inheritance-Restart exploration mechanism to avoid early convergence, while exploration also increases the likelihood of high-reward trajectories. Additionally, it balances diversity and fidelity, adding diversity without causing a distribution over-shift. Second, our Quality–Efficiency arbitration mechanism improves guidance by removing incorrect signals, and it reduces computation by dynamically stopping generation when completeness and marginal reward gain are optimal. In an extensive number of experiments and different types of evaluation metrics, the proposed SwiftExplorer achieves excellent performance on all metrics, including preference, fidelity, diversity, and richness. Keywords: Language-Vision Conditional Generation, Training-free Guidance 1. INTRODUCTION Figure 1. Feature visualization via t-SNE. The features are extracted by a general self-supervised decoder (i.e., DINOv2) from images generated by different methods. With the guidance targets of both Ours and DyMO having achieved comparable scores, the distribution results indicate that existing SOTA guidance methods (i.e., DyMO) yield more compact clustering, narrowing the generated distribution, which in turn reduces the diversity of the generated data. This observation is substantiated by the Silhouette Coefficient (Rousseeuw, 1987), a clustering evaluation metric, which demonstrates its convergence and limited diversity. In contrast, our method maintains a broader generative distribution while achieving comparable target reward scores (see Tab. 1). Diffusion models (Ho et al., 2020; Ramesh et al., 2022; Rombach et al., 2022; Zhang et al., 2026b; Lu et al., 2026; Yan et al., 2025; Yan et al., 2026b; Yan et al., 2026a; Li et al., 2026; Ling et al., 2026; Ma et al., 2026; Shen et al., 2025), as one of the most powerful generative frameworks, demonstrate exceptional generative capabilities by training on various large-scale datasets. This is evident in the model’s capability to generate both high-quality images and a diverse range of semantic and stylistic content. However, in practical applications, it is not only necessary to generate realistic images but also to ensure that the results align with specific target requirements, such as aesthetic preferences. This need for goal alignment is becoming a core research problem, posing the challenge of effectively translating the general capabilities of diffusion models into customized generation. Early research primarily relied on training strategies (Black et al., 2023; Fan et al., 2024; Yang et al., 2024; Wallace et al., 2024). By fine-tuning diffusion models on specific preference objectives, alignment performance could be significantly improved. However, such methods generally face challenges of high training costs and poor transferability (Kim et al., 2025; Xie and Gong, 2025). Whenever user requirements or objectives change, diffusion models must be finetuned again, which limits their scalability and transferability in practical applications. To address this, researchers proposed training-free methods (Chung et al., 2022; Kim et al., 2025; Deckers et al., 2024; Tang et al., 2024; Xie and Gong, 2025), which do not modify the model’s weight but incorporate external signals during generation to achieve goal-guided results. The advantages of such approaches are evident as they require no additional training costs and can quickly adapt to different objective demands. Consequently, training-free guidance has become a crucial diffusion model alignment method. More detailed Related Works can be found in Appendix 1. However, existing training-free methods still suffer from two typical limitations: (1) overly narrow generative distributions. To enhance the alignment between generated results and target objectives, training-free methods should incorporate target-guided terms during sampling. These methods gradually shift the generative distribution toward specific regions, such as high-reward subspaces. Although these approaches effectively enhance alignment, their excessive reliance on a consistent guidance direction easily leads to reward hacking (Tang et al., 2024), which manifests as a rapid narrowing of the pretrained distribution, ultimately reducing image diversity (Ho and Salimans, 2022; Tang et al., 2024). (2) unselective, inefficient guidance. Current methods apply guidance terms indiscriminately throughout the entire generation process, lacking the ability to identify and manage ineffective or redundant signals. This not only results in unnecessary computational waste but may also introduce noise interference due to ineffective guidance, ultimately compromising both alignment quality and generation efficiency. To address the above challenges, we propose SwiftExplorer, an enhanced plugin compatible with existing training-free alignment methods. (1) SwiftExplorer introduces an Inheritance-Restart exploration mechanism that prevents premature convergence of generation trajectories to a single mode through probabilistic perturbation and path screening. Simultaneously, we introduce a progress-aware adaptive trade-off to enhance exploration while preserving generation fidelity (see Fig. 1). It promotes early-stage exploration to improve diversity and the discovery of high-reward trajectories. In later stages, it encourages convergence to preserve the pretrained distribution. (2) To improve guidance efficiency, SwiftExplorer employs a Quality-Efficiency screening mechanism to filter out ineffective signals, ensuring that each guidance step contributes meaningfully. Additionally, by integrating an adaptive factor and considering the diminishing marginal utility of rewards, our method dynamically cuts guidance in later generations to avoid redundant optimization. The Quality-Efficiency mechanism effectively reduces computational overhead while maintaining alignment quality. Experiments show that while ensuring leading performance on both fidelity and alignment objectives, SwiftExplorer effectively improves generative diversity and outperforms existing baselines in computational efficiency. Our contributions are as follows: • We propose an Inheritance-Restart exploration mechanism to enhance the generation diversity of existing methods. • We introduce a Quality-Efficiency mechanism to drop invalid signals, thereby improving guidance and reducing costs. • We achieve two trade-offs: Diversity and Fidelity evaluation during generation progress. Quality and Efficiency adjustment on marginal reward gains. 2. Method Figure 2. Overall framework of the proposed plug-and-play SwiftExplorer. SwiftExplorer is a plugin enhancing training-free methods in exploration and efficiency. Sec. 2.1 covers preliminaries. Sec. 2.2 elaborates on the Inheritance-Restart exploration mechanism, achieving an adaptive trade-off between Diversity and Fidelity. Sec. 2.3 introduces a Quality–Efficiency arbitration mechanism guiding early quality and late efficiency. The SwiftExplorer framework is shown in Fig. 2, with algorithmic details in Appendix. 4. 2.1. Reward-Guided Diffusion Alignment 2.1.1. Generative Process of Diffusion Models The discrete-time generative process (or reverse process) aims to produce a sample by denoising a latent variable tx_t iteratively. This process requires two components: a predefined noise schedule βtt=0T\ _t\_t=0^T and a well-trained score function θ(t,t)s_θ(x_t,t). We define αt=1−βt _t=1- _t and α¯t=∏i=1tαi α_t= _i=1^t _i. The generation starts by sampling an initial latent variable from a standard normal distribution: T∼(,)x_T (0,I) Then, for each timestep t from T to 00, the sample is iteratively refined using the update rule: (1) t−1=1αt(t+βtθ(t,t))+βtt,x_t-1= 1 _t (x_t+ _ts_θ(x_t,t) )+ _tz_t, where t∼(,)z_t (0,I) is a standard Gaussian noise vector independently sampled at each timestep. The final output after T steps is the generated sample 0x_0. For convenience, we also define a function fθ(t,t)f_θ(x_t,t) that provides a direct estimate of the clean sample 0|tx_0|t, from any noisy intermediate tx_t (Chung et al., 2022; Bansal et al., 2023; Xie and Gong, 2025): (2) 0|t=fθ(t,t)=1α¯t(t+(1−α¯t)θ(t,t)).x_0|t=f_θ(x_t,t)= 1 α_t (x_t+(1- α_t)s_θ(x_t,t) ). 2.1.2. Iterative training-free Refinement with Gradient Guidance To guide the generation process towards a specific objective, we can incorporate the gradient of a reward function ℛR (Li et al., 2025). The reward function is defined in the data domain (i.e., it operates on the estimated clean sample 0|tx_0|t by Eq. 2. The approach of predicting 0|tx_0|t using Eq. 2 for reward calculation is widely utilized in the field (Xie and Gong, 2025; Bansal et al., 2023)). The guidance is injected by adjusting the current sample tx_t in the direction of the reward gradient. Specifically, before applying the update in Eq. 1, we compute a guided sample tx_t: (3) t←t+w∇tℛ(fθ(t,t)),x_t _t+w _x_tR(f_θ(x_t,t)), where w is a guidance scale factor, ℛ(⋅)R(·) denotes the reward function for inference-time guidance. The above process could be seen as one step of optimization. For simplicity, we can define optimization with m steps of Eq. 3 as: (4) ~t=g(t,ℛ,m). x_t=g(x_t,R,m). Replacing tx_t in Eq. 1 with ~t x_t, the new update rule becomes: (5) t−1=1αt(~t+βtθ(~t,t))+βtt.x_t-1= 1 _t ( x_t+ _ts_θ( x_t,t) )+ _tz_t. 2.2. Inheritance–Restart Exploration mechanism To mitigate the issue of narrowed generative distribution caused by overly concentrated guidance direction in training-free methods, SwiftExplorer equips an Inheritance-Restart exploration mechanism. This mechanism selectively introduces exploratory supplementary terms based on the generation progress, thereby preventing premature convergence of generative trajectories to a single mode and increasing the probability of discovering high-reward paths. 2.2.1. Exploration Supplement Variable During the generation step t, we introduce an exploration supplement variable ϵt(i) ε^(i)_t, which allows the generated trajectories to maintain a richer distribution. Specifically, we adopt Monte Carlo sampling to obtain n candidates ϵt(i)∼(0,σt2I),i=1,…,n ε^(i)_t (0, _t^2I), i=1,…,n. After incorporating ϵt(i) ε^(i)_t, the original xtx_t without exploration is updated into a set x^t(i)\ x_t^(i)\: (6) ^t(i)=t+ϵt(i). x_t^(i)=x_t+ ε^(i)_t. Subsequently, each candidate ^t(i) x_t^(i) is mapped to its predicted reconstruction ^0|t(i)=fθ(^t(i),t) x_0|t^(i)=f_θ( x_t^(i),t), and is evaluated via the objective reward function ℛ(^0|t(i))R( x_0|t^(i)). After m rounds based on Eq. 4, we obtain a final set of candidates ~t(i)=g(t^(i),ℛ,m)\ x_t^(i)\=g(\ x_t^(i)\,R,m). We then define the optimal generation trajectory policy as: (7) i⋆=argmaxi∈1,…,nℛ(~0|t(i)),t⋆=~t(i⋆).i = _i∈\1,…,n\R( x_0|t^(i)), _t = x_t^(i ). This process selects the optimal candidate t⋆x_t as the sample of generation step t, which is adopted for the following generations. In summary, the injection of the exploration supplementary term expands the local search space. The selection policy picks the optimal generation trajectory with the highest alignment potential, thereby increasing diversity while maximizing alignment rewards. 2.2.2. Inheritance–Restart Technique In Sec. 2.2.1, the exploration supplementary term expands the generation space. Building upon this, we apply an Inheritance-Restart technique to this term. This technique dynamically regulates exploration behavior based on the target reward improvement, preserving high-quality explorations while eliminating ineffective adjustments. Consequently, the exploration term enhances diversity, while the Inheritance-Restart technique ensures that exploration consistently improves the target reward. Specifically, the reward changing for the supplementary term at generation step t is defined as: (8) Δℛt(i⋆)=ℛ(0|t⋆)−ℛ(0|t). _t^(i )=R(x_0|t )-R(x_0|t). (1): If Δℛt(i⋆)>0 _t^(i )>0, the exploration still yields a positive gain for the alignment reward. In this case, the latent change from the previous step is carried over to the next step t−1t-1, where ϵt−1=t⋆−t ε_t-1=x_t -x_t. Under this inheritance strategy, the exploration candidate set contains only this inherited signal, i.e., setting n=1n=1 in Eq. 6, and optimization continues via Eq. 4. (2): If Δℛt(i⋆)<0 _t^(i )<0, the exploration gain becomes saturated or starts to decline. In this case, the restart mechanism is activated: n exploration candidates are resampled in the next step t−1t-1 as Eq. 6, and the exploration term is updated through the optimization in Eq. 4 and the selection step in Eq. 7. The inheritance-restart technique uses reward changes as a criterion to enhance exploration diversity and reduce guidance overhead. Moreover, it ensures the generation alignment with target objective. 2.2.3. Diversity and Fidelity Adaptive Trade-off Diffusion models face the dual demands of diversity and fidelity. Diversity expands the coverage of potential generation distribution, while fidelity ensures the results are credible and semantically consistent with the conditioning. However, these two objectives often exhibit a trade-off, making their careful balance crucial for achieving high-quality results (Dhariwal and Nichol, 2021). To address this, we propose an adaptive coordination technique that dynamically adjusts the emphasis based on the generation progress: it prioritizes exploration in the early stages to foster diversity and gradually increases the strength of regularization in the later stages to ensure fidelity. Unlike approaches that use predefined static parameters, the adaptation technique can flexibly accommodate the dynamic nature of the generation process. To this end, we construct a Control Network (This network outputs a simple noise scorer, analogous to scoring models such as PickScore and Aesthetic Score, and is therefore used as a pretrained model. Experimental support is provided in Tab. 8, with details in Appendix 2.2.), which is designed to assess the denoising progress of the intermediate state in real-time during the generation process. Specifically, the control network is defined as a mapping function: hθp:t↦pth_ _p:x_t p_t, where t∈ℝdx_t ^d denotes the latent state at generation step t. pt∈[0,1]p_t∈[0,1] represents the probability of the current latent state close to the clean image 0x_0, which reflects the degree of the current state’s generation progress. The control network is obtained by minimizing a binary cross-entropy loss as the supervision signal (see Appendix 2.1 for details). We utilize ptp_t as an adaptive weighting to balance exploration diversity and generation fidelity. In the first optimization round, the combined effect of the exploration and alignment signal gradients is written as: (9) ^t(i)=t+(1−pt)⋅ϵ(i)−pt⋅∇tL2(^t,t)⏟Diversity-Fidelity Trade-off+∇tℛ(0|t), x_t^(i)=x_t+ (1-p_t)· ε^(i)-p_t· _x_tL_2( x_t,x_t)_Diversity-Fidelity Trade-off+ _x_tR(x_0|t), Figure 3. Visual impression on the advanced flow-matching-based models. Our results exhibit finer details, more diverse colors, and improved aesthetic style. where ϵ(i) ε^(i) is the exploration supplement variable and ∇tℛ(0|t) _x_tR(x_0|t) represents the reward-based gradient guidance. We introduce L2 regularization to constrain the exploration scope. Then, based on the iteration defined in Eq. 4, we iteratively introduce guidance to obtain the final state ~t(i)=g(^t(i),ℛ−L2,m) x_t^(i)=g( x_t^(i),R-L_2,m). In summary, the adaptive trade-off encourages exploratory diversity in the early generation stages. In the later stages, the continuous increase in the regularization constraint prompts the generation process to transition from exploration to stable convergence, thereby ensuring high-fidelity results. Through this design, SwiftExplorer achieves an adaptive trade-off between creativity and stability. 2.3. Quality–Efficiency Arbitration mechanism Table 1. Main comparisons with SoTA. All metrics are obtained with SDv15 as backbone, Pick-a-pic as prompt set, and PickScore as reward for guidance. Orange denotes gradient-based training methods. Green denotes reinforcement-learning-based training methods. Blue denotes training-free methods. Method Preference Fidelity Diversity Richness #Top2 PS↑ AES↑ IR↑ HPS↑ FID↓ CLIP↑ iFS↑ LPIPS↑ IS↑ TCE↑ BRI↓ NIQE↓ SE↑ SDv15 20.48 5.412 0.181 0.262 - 0.243 - 0.654 23.79 38.05 18.66 5.401 11.26 3 Diff-DPO 20.87 5.551 0.443 0.271 109.1 0.244 0.795 0.639 22.77 39.20 15.12 4.463 11.02 1 Diff-KTO 20.83 5.585 0.599 0.272 101.3 0.240 0.801 0.634 22.70 39.12 26.25 4.361 11.22 0 SPO 20.76 5.613 0.282 0.218 78.76 0.241 0.855 0.649 23.18 39.32 25.67 4.103 11.09 1 DRaFT 22.52 5.697 0.779 0.271 84.47 0.231 0.856 0.572 20.10 37.85 35.64 5.336 10.43 2 DDPO 21.79 5.704 0.196 0.212 147.5 0.242 0.539 0.629 20.01 39.18 12.46 4.328 11.74 1 DPOK 20.97 5.661 0.583 0.272 99.30 0.242 0.820 0.641 22.52 39.35 12.28 4.608 11.23 2 DNO 20.87 5.479 0.425 0.271 79.96 0.239 0.851 0.591 19.94 38.88 17.23 4.757 11.07 0 TTScale 22.14 5.636 0.619 0.272 66.93 0.240 0.863 0.652 23.73 38.90 15.05 4.447 11.28 2 FKSteer 22.56 5.696 0.539 0.278 73.83 0.243 0.868 0.650 23.76 39.42 13.93 4.092 11.59 2 DAS 22.34 5.658 0.632 0.283 95.42 0.240 0.844 0.625 21.01 39.19 14.36 4.631 11.14 1 DyMO 22.79 5.694 0.709 0.279 90.56 0.239 0.849 0.627 20.97 39.29 16.18 4.545 11.29 1 Ours 22.91 5.707 0.812 0.280 71.36 0.242 0.891 0.654 23.71 39.37 10.92 2.886 12.07 12 2.3.1. Guidance Screening Existing methods apply guidance signals indiscriminately throughout the entire sampling process, which can lead to ineffective guidance and undermine alignment effectiveness. To address this, SwiftExplorer introduces a guidance signal selection technique that dynamically discriminates the signal’s utility. Let the optimal guidance signal generated at step t be t(i⋆) γ_t^(i ), which represents the reward gradient corresponding to t⋆x_t , i.e., the optimal candidate. We then define a selection operator ℱF: (10) ℱ(t(i⋆))=Δℛt(i∗)>0.F( γ_t^(i ))=\ ^(i^*)_t>0\. With the proposed screening process, a guidance signal is retained only if it contributes to a positive reward gain; otherwise, it is discarded, and the original tx_t from the base diffusion model is used. This reward-margin-based selection strategy ensures the quality of the guidance signals for generation. 2.3.2. Quality and Compute Adaptive Trade-off Unlike existing methods that rigidly couple guidance with the generation steps, SwiftExplorer regulates the use of guidance signals through the dual control of target reward marginal gain and generation progress ptp_t. This approach aims to reduce computational costs while ensuring alignment quality. Guidance is stopped when it is insignificant to reward improvement and the denoising meets the required threshold, thereby avoiding redundant costs and preventing over-guidance. We first denote the step t reward ℛt(i∗)(⋅)R^(i^*)_t(·) as rtr_t. Then, the smoothed trend of reward gain is defined as: (11) Δr~t=(1−ϕ)Δr~t+1+ϕ(rt−rt+1), r_t=(1-φ)\, r_t+1+φ\,(r_t-r_t+1), where ϕ∈(0,1]φ∈(0,1]. If and only if pt=1p_t=1 and Δr~t≤δr r_t≤ _r, the reward return margin has been reached and the early stopping is triggered, where δr≥0 _r≥ 0 is the minimal tolerance gain. We define the early-stopping indicator function as: (12) Estop(t)=[pt=1∧Δr~t≤δr]∈0,1.E_stop(t)=I\! [p_t=1\ \ r_t≤ _r ]∈\0,1\. Exploration and guidance stop when Estop(t)=1E_stop(t)=1 and continue when Estop(t)=0E_stop(t)=0. This criterion ensures triggers early stopping only when both conditions are met: ptp_t guarantees the state is ‘sufficiently clean’, while Δr~t r_t ensures that ‘further exploration guidance yields no significant reward improvement’. By requiring both conditions simultaneously, this technique eliminates redundant guidance in the later stages of generation without compromising target alignment quality, thereby reducing computational cost. 3. Experiments 3.1. Experimental Setup Datasets. We employ Pick-a-pic (Kirstain et al., 2023), HPSv2 (Wu et al., 2023), GenEval (Ghosh et al., 2023), and animals (Black et al., 2023) datasets as our basic test bed. The prompts in Pick-a-Pic are relatively complex, abstract, and surreal, whereas those in HPSv2 are more realistic and aligned with the real world. GenEval includes specified counting and composition tasks. Optimization Objectives and Metrics. We deploy three reward functions as the optimization objectives, that is, PickScore (Kirstain et al., 2023) (PS), Aesthetic Score (Schuhmann et al., 2022) (AES), and ImageReward (Xu et al., 2023a) (IR). To quantitatively evaluate the generative performance as extensively as possible, we introduce 4 evaluating dimensions with 13 distinct metrics, including: Aesthetic Preference: AES, PS, IR, and HPSv2 (Wu et al., 2023). Image Fidelity: ClipScore (Radford et al., ) (Clip), Fréchet Inception Distance (FID) (Heusel et al., 2017), and improved F1 Score (iFS) (Kynkänniemi et al., 2019). Generative Diversity: LPIPS (Zhang et al., 2018), TCE (Ibarrola and Grace, 2024), and Inception Score (Salimans et al., 2016) (IS). Compositional Richness: NIQE (Mittal et al., 2012b), BRISQUE (BRI) (Mittal et al., 2012a), and Spectral Entropy (SE) (Liu et al., 2014). Details of each metric can be found in Appendix. 3.3. Baselines. For comprehensive comparison, we introduce three types of State-of-The-Art (SoTA) diffusion-based alignment methods, including: Gradient-based Training: Diff-DPO (Wallace et al., 2024), Diff-KTO (Li et al., 2024), SPO (Liang et al., 2024), and DRaFT (Clark et al., 2023). RL-based Training: DDPO (Black et al., 2023), DPOK (Fan et al., 2023). Training Free: DAS (Zhang et al., 2026a), TTScale (Ma et al., 2025), FKSteer (Singhal et al., 2025), DNO (Tang et al., 2024), and DyMO (Xie and Gong, 2025). All baselines are strictly reproduced based on their official code and settings within our evaluation benchmark. Backbones. We adopt two types of widely-adopted backbones: Unet-based models: SDv14, SDv15, SDv21, SD XL1.0 (XL) and Flow-matching models: SD 3.0-medium (SD3), SD3.5-medium (SD3.5). Figure 4. Wall-clock runtime comparison under consistent experimental conditions. Table 2. Results on HPSv2 dataset. Method IR PS CLIP LPIPS TCE NIQE↓ SDv15 0.231 21.57 0.249 0.643 39.62 4.656 Diff-DPO 0.571 22.03 0.245 0.619 37.97 4.471 SPO 0.300 21.58 0.249 0.595 39.37 4.172 DDPO 0.678 22.26 0.245 0.607 38.96 3.973 DNO 0.446 21.79 0.246 0.575 39.14 4.523 DAS 0.684 23.09 0.246 0.653 39.31 4.133 DyMO 0.805 23.46 0.246 0.610 39.57 4.088 Ours 0.872 23.79 0.247 0.671 39.92 2.595 Table 3. Comparison results with advanced flow matching models. Gdt represents the baseline that directly applies the reward gradient to the latent code during inference. Ours is designed based on Gdt with the proposed plugin components. Method SD3 SD3.5 PS CLIP LPIPS NIQE↓ Top1 PS CLIP LPIPS NIQE↓ Top1 Base 21.62 0.238 0.645 4.312 1 21.90 0.238 0.621 4.553 1 +Gdt 22.17 0.239 0.636 4.304 0 22.95 0.235 0.616 4.174 0 +Ours 22.57 0.241 0.614 2.990 3 23.93 0.238 0.625 3.794 4 Figure 5. Performance-Computation Trade-off. Figure 6. User study for plug-and-play effectiveness. Figure 7. Plug-and-Play effectiveness. While maintaining the Fidelity and Preference, our method significantly enhances Diversity, Richness, and Speed across all cases, such as the “background and pizza filling diversity” in the first row, and “improved Van Gogh style” in the second row. Table 4. Ablation study. For each component, ESV and M-ESV represent exploration supplement variable and with Monte Carlo sampling. I&R and EarS denote Inheritance-Restart technique and Early Stopping. GuiS denotes Guidance Screening. Var Dive Fide Effi Qual IR PS CLIP LPIPS TCE NIQE↓ Time↓ #Top2 ESV M-ESV L2 I&R EarS GuiS Base 0.709 22.79 0.239 0.627 39.29 4.454 44.92 0 V1 ✓ 0.739 22.12 0.236 0.659 39.30 2.897 46.31 1 V2 ✓ ✓ 0.734 22.46 0.240 0.636 39.30 3.379 51.24 0 V3 ✓ ✓ 0.757 22.75 0.240 0.641 39.32 2.939 67.08 0 V4 ✓ ✓ ✓ 0.746 22.73 0.241 0.640 39.29 3.039 48.56 0 V5 ✓ ✓ ✓ ✓ 0.759 22.66 0.242 0.645 39.28 3.120 40.91 2 V6 ✓ ✓ ✓ ✓ 0.774 22.95 0.241 0.653 39.35 2.847 44.73 4 Ours ✓ ✓ ✓ ✓ ✓ 0.812 22.91 0.242 0.654 39.37 2.886 37.28 7 Figure 8. Generative diversity examples with the same prompt and different seeds. Figure 9. Diversity-fidelity adaptive trade-off (Ours, red dotted lines) with different fixed intensities. 3.2. Comparisons Against SoTA Methods To demonstrate the effectiveness of SwiftExplorer (SwiftE), we carefully reproduce and evaluate 12 baselines with PickScore as the objective on SDv15 and pick-a-pic datasets. Since SwiftE is an efficient plugin, this section represents deploying SwiftE to DyMO. Then, we introduce 13 metrics to assess from four dimensions. As shown in Tab.1, our method achieves the best overall performance in all dimensions, achieving Top2 performance in 12 metrics. Moreover, since the prompts in the Pick-a-pic dataset are relatively abstract and surreal, we further conduct an experiment on the photorealistic HPSv2 dataset for comprehensive evaluation. In Tab.2, all methods perform better because realistic objects are more likely to be seen during training and hence easier to generate. our method still achieves the best results on five metricsand maintains comparable fidelity. The qualitative results in Appendix Fig. 12 further demonstrate the superiority of our method. In Appendix. 3.4 & Tab. 7, to assess the impact of random seeds on result stability, we compute the coefficient of variation (CoV) of each metric for the training-free baseline across different seeds. Comparisons with more advanced SD models. In Appendix Tab. 11, we employ our method and other SoTA on the SD-XL backbone, and introduce the advanced SDv35 as an additional baseline. We also implement it on the advanced SD3 and SD3.5 backbones with PS as the reward and Pick-a-pic as the dataset. Quantitatively, as shown in Tab. 3, our method can still effectively improve the reward score while maintaining the alignment and diversity. Qualitatively, in Fig. 3, it can be clearly observed that our method has achieved improved human preference in aesthetics. 3.3. Plug-and-Play Effectiveness In Fig. 7, we discuss the plugin effectiveness of SwiftExplorer. Specifically, we present side-by-side comparisons of results with and without SwiftE. Gdt represents backbones with Gradient guidance. We choose PS, CLIP, LPIPS, 1NIQE 1NIQE, and image-per-minute as the representations of Preference, Fidelity, Diversity, Richness, and Speed. The results clearly show that SwiftE effectively enhances the diversity of baseline methods and reduces computational cost while improving other quality dimensions. 3.4. Analysis on Inference Efficiency Comparisons on Wall-clock Runtime. Here, we measure and compare the time consumption of all methods under consistent experimental conditions. To investigate the effectiveness in real-world applications, we conduct comparisons on wall-clock runtime with details. As shown in Fig. 4, we present the time consumption of running the entire script. Firstly, their preparation time are similar and once prepared, the script can conduct arbitrary times of inference. Then, it can be observed that our inference time is much faster than the baseline DyMO, while Base has the smallest inference time since it includes no optimization or guidance. Performance-Computation Trade-off. To investigate the relation between computational cost and performance, we first deploy extra mean number of Function Evaluated (nFE) to quantify the computational cost in a unified manner, which represents the number of extra iterations or sampling in one denoising step. Then, we compare our method with iteration-based DyMO and sampling-based DAS, where the maximum iteration and sampling particle number are set to 2,4,8,16. Notably, SwiftExplorer includes both sampling and iteration, we decouple these factors and design two variants of our method, that is, Ours-i (iteration) and Ours-p (particles). In Fig. 5, our results can first save nFE because we conduct early stopping. Second, the curves of our results consistently surpass the baselines, indicating the consistent superiority of SwiftExplorer. 3.5. Ablation Study Overall Ablation. SwiftExplorer aims to balance the trade-offs between diversity-fidelity and quality-efficiency, and thus can be divided into four categories based on the objectives. Specifically, Diversity (Dive) includes directly adding exploration and adding Monte Carlo exploration. Fidelity (Fide) refers to L2 constraints for regularization. Efficiency (Effi) includes the I&R and early stopping. Guidance Screening aims to enhance Quality (Qual). Then, we conducted detailed ablations in Tab. 4, where the proposed diversity-fidelity tradeoff effectively improves the diversity of generated images while maintaining consistency. Meanwhile, the I&R mechanism and early stopping greatly reduce computational overhead with almost no impact on performance. GuiS improves inference efficiency and performance by removing invalid guidance. In Appendix. 3.7, we further analyze the mutual correlation among different proposed components based on the ablation results. Analysis on Generative Diversity. The generative diversity is compared among SD-XL, Diff-DPO, DyMO, and Ours by generating images with 40 seeds. As shown in Fig. 8, our method exhibits improved diversity while maintaining the impressive preference score and precise prompt-image alignment. We have also investigated exploration diversity during inference in Appendix 3.6. Analysis on Diversity-Fidelity Trade-off. In Fig. 9, we examine the Diversity-Fidelity Trade-off by fixing exploration and regularization levels instead of using dynamic ptp_t. Results show that stronger regularization undermines preference without enhancing diversity, while higher exploration reduces diversity through image distortion. In contrast, our method consistently surpasses baselines and achieves a balanced trade-off between richness and alignment. 3.6. User Study To subjectively evaluate the generative quality of different methods, we conduct a user study based on the images from the Plug-and-Play experiments in Sec. 3.3. Specifically, we recruit five subjects without prior knowledge to select the preferred image within paired images with the corresponding prompt as the reference. As shown in Fig. 6, our method consistently performs as an effective plugin that achieves superior quality compared to the implemented original models. User Study details are provided in Appendix 3.10. 4. Conclusion In this work, we address the challenges in aligning diffusion models with specific objectives without incurring prohibitive fine-tuning costs. We introduce SwiftExplorer, a plugin that mitigates distribution collapse and enhances efficiency through an Inheritance-Restart exploration mechanism and a Quality–Efficiency arbitration framework. These innovations prevent early convergence, increase exploration for high-reward trajectories, and improve guidance by eliminating irrelevant signals. Experiments show consistent gains in quality, diversity, and efficiency over existing methods. References Abramson et al. (2024) J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature 630 (8016), p. 493–500. Cited by: §1.1. Bansal et al. (2023) A. Bansal, H. Chu, A. Schwarzschild, S. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein Universal guidance for diffusion models. In Computer Vision and Pattern Recognition, p. 843–852. Cited by: §2.1.1, §2.1.2. Black et al. (2023) K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301. Cited by: §1.3, §1, §3.1. Chen et al. (2020) N. Chen, Y. Zhang, H. Zen, R. J. Weiss, M. Norouzi, and W. Chan Wavegrad: estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713. Cited by: §1.1. Chung et al. (2022) H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687. Cited by: §1, §2.1.1. Clark et al. (2023) K. Clark, P. Vicol, K. Swersky, and D. J. Fleet Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309.17400. Cited by: §1.2, §3.1. Deckers et al. (2024) N. Deckers, J. Peters, and M. Potthast Manipulating embeddings of stable diffusion prompts. In International Joint Conference on Artificial Intelligence, p. 7636–7644. Cited by: §1. Dhariwal and Nichol (2021) P. Dhariwal and A. Nichol Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems 34, p. 8780–8794. Cited by: §1.4, §2.2.3. Ding et al. (2025) X. Ding, L. Yu, X. Li, Z. Tu, H. Chen, J. Hu, and Z. Chen RaSS: improving denoising diffusion samplers with reinforced active sampling scheduler. In Computer Vision and Pattern Recognition Conference, p. 12923–12933. Cited by: §1.1. Fan et al. (2023) Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee Dpok: reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems 36, p. 79858–79885. Cited by: §1.3, §3.1. Fan et al. (2024) Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems 36. Cited by: §1. Ghosh et al. (2023) D. Ghosh, H. Hajishirzi, and L. Schmidt Geneval: an object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems 36, p. 52132–52152. Cited by: §3.1. Gu et al. (2024) Y. Gu, Z. Wang, Y. Yin, Y. Xie, and M. Zhou Diffusion-rpo: aligning diffusion models through relative preference optimization. arXiv preprint arXiv:2406.06382. Cited by: §1.2. Heusel et al. (2017) M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems 30. Cited by: 2nd item, §3.1. Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33, p. 6840–6851. Cited by: §1. Ho and Salimans (2022) J. Ho and T. Salimans Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. Cited by: §1. Ibarrola and Grace (2024) F. Ibarrola and K. Grace Measuring diversity in co-creative image generation. arXiv preprint arXiv:2403.13826. Cited by: 2nd item, §3.1. Kim et al. (2025) S. Kim, M. Kim, and D. Park Test-time alignment of diffusion models without reward over-optimization. arXiv preprint arXiv:2501.05803. Cited by: §1.3, §1.4, §1. Kirstain et al. (2023) Y. Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy Pick-a-pic: an open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems. Cited by: 2nd item, §3.1. Kong et al. (2020) Z. Kong, W. Ping, J. Huang, K. Zhao, and B. Catanzaro Diffwave: a versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761. Cited by: §1.1. Kynkänniemi et al. (2019) T. Kynkänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems 32. Cited by: 3rd item, §3.1. Li et al. (2026) H. Li, T. Ren, X. Ma, C. Qing, Z. Fang, S. He, Z. Guo, H. Wu, J. Tian, Y. Zou, et al. VideoCoCo: code-as-cot for physically-consistent video generation via an agentic dual-engine system. arXiv preprint arXiv:2607.27380. Cited by: §1. Li et al. (2024) S. Li, K. Kallidromitis, A. Gokul, Y. Kato, and K. Kozuka Aligning diffusion models by optimizing human utility. Advances in Neural Information Processing Systems 37, p. 24897–24925. Cited by: §1.1, §3.1. Li et al. (2025) Y. Li, Z. Zhou, Z. Peng, J. Dong, H. You, R. Yan, S. Wen, Y. Tian, and T. Huang A preference-driven methodology for efficient code generation. IEEE Transactions on Artificial Intelligence. Cited by: §2.1.2. Liang et al. (2024) Z. Liang, Y. Yuan, S. Gu, B. Chen, T. Hang, J. Li, and L. Zheng Step-aware preference optimization: aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.04314. Cited by: §3.1. Ling et al. (2026) Z. Ling, X. Yu, R. Yan, J. Cheng, Z. Wang, Q. Shuai, and C. Zou GenTrack: physical alignment for robot-native motion generation and zero-shot humanoid tracking. arXiv preprint arXiv:2608.01410. Cited by: §1. Liu et al. (2022) J. Liu, C. Li, Y. Ren, F. Chen, and Z. Zhao Diffsinger: singing voice synthesis via shallow diffusion mechanism. In AAAI conference on artificial intelligence, Vol. 36, p. 11020–11028. Cited by: §1.1. Liu et al. (2014) L. Liu, B. Liu, H. Huang, and A. C. Bovik No-reference image quality assessment based on spatial and spectral entropies. Signal Processing: Image communication 29 (8), p. 856–863. Cited by: 3rd item, §3.1. Lu et al. (2026) L. Lu, X. Chen, M. Guo, S. Li, J. Wang, and Y. Shi Chordedit: one-step low-energy transport for image editing. arXiv preprint arXiv:2602.19083. Cited by: §1. Ma et al. (2025) N. Ma, S. Tong, H. Jia, H. Hu, Y. Su, M. Zhang, X. Yang, Y. Li, T. Jaakkola, X. Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732. Cited by: §1.4, §3.1. Ma et al. (2026) Q. Ma, J. Cheng, L. Liang, Z. Yu, T. Huang, and R. Yan Can we perform online rl for image editing without editing rewards?. arXiv preprint arXiv:2608.22780. Cited by: §1. Mittal et al. (2012a) A. Mittal, A. K. Moorthy, and A. C. Bovik No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing 21 (12), p. 4695–4708. Cited by: 2nd item, §3.1. Mittal et al. (2012b) A. Mittal, R. Soundararajan, and A. C. Bovik Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters 20 (3), p. 209–212. Cited by: 1st item, §3.1. Qiao et al. (2022) Z. Qiao, W. Nie, A. Vahdat, T. F. Miller I, and A. Anandkumar Dynamic-backbone protein-ligand structure prediction with multiscale generative diffusion models. arXiv preprint arXiv:2209.15171 1. Cited by: §1.1. [35] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, Cited by: 1st item, §3.1. Ramesh et al. (2022) A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1 (2), p. 3. Cited by: §1. Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer High-resolution image synthesis with latent diffusion models. In Computer Vision and Pattern Recognition, p. 10684–10695. Cited by: §1. Rousseeuw (1987) P. J. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, p. 53–65. Cited by: Figure 1, Figure 1. Salimans et al. (2016) T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen Improved techniques for training gans. Advances in Neural Information Processing Systems 29. Cited by: 3rd item, §3.1. Schuhmann et al. (2022) C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. Laion-5b: an open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems. Cited by: 1st item, §3.1. Shen et al. (2025) H. Shen, J. Cheng, R. Yan, Z. Wang, W. Peng, and B. Huang When generative replay meets evolving deepfakes: dual confusion-aware regularization for incremental face forgery detection. arXiv preprint arXiv:2511.18436. Cited by: §1. Singhal et al. (2025) R. Singhal, Z. Horvitz, R. Teehan, M. Ren, Z. Yu, K. McKeown, and R. Ranganath A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848. Cited by: §1.4, §3.1. Sutton (2018) R. S. Sutton Reinforcement learning: an introduction. A Bradford Book. Cited by: §1.3. Tang et al. (2024) Z. Tang, J. Peng, J. Tang, M. Hong, F. Wang, and T. Chang Inference-time alignment of diffusion models with direct noise optimization. arXiv preprint arXiv:2405.18881. Cited by: §1.4, §1, §1, §3.1. Wallace et al. (2024) B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik Diffusion model alignment using direct preference optimization. In Computer Vision and Pattern Recognition, p. 8228–8238. Cited by: §1, §3.1. Watson et al. (2023) J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, R. J. Ragotte, L. F. Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature 620 (7976), p. 1089–1100. Cited by: §1.1. Wu et al. (2023) X. Wu, Y. Hao, K. Sun, Y. Chen, F. Zhu, R. Zhao, and H. Li Human preference score v2: a solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341. Cited by: 4th item, §3.1. Wu et al. (2024) X. Wu, Y. Hao, M. Zhang, K. Sun, Z. Huang, G. Song, Y. Liu, and H. Li Deep reward supervisions for tuning text-to-image diffusion models. In European Conference on Computer Vision, p. 108–124. Cited by: §1.2. Xie and Gong (2025) X. Xie and D. Gong DyMO: training-free diffusion model alignment with dynamic multi-objective scheduling. In Computer Vision and Pattern Recognition, p. 13220–13230. Cited by: §1.4, §1, §2.1.1, §2.1.2, §3.1. Xu et al. (2023a) J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong Imagereward: learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems. Cited by: 3rd item, §3.1. Xu et al. (2023b) M. Xu, A. S. Powers, R. O. Dror, S. Ermon, and J. Leskovec Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning, p. 38592–38610. Cited by: §1.1. Xu et al. (2022) M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang Geodiff: a geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923. Cited by: §1.1. Yan et al. (2025) R. Yan, J. Cheng, Y. Gan, S. Sun, Y. Wu, Y. Yang, L. Ling, J. Lin, Y. Zhu, J. Zhou, et al. Entropy-adaptive diffusion policy optimization with dynamic step alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 1924–1934. Cited by: §1. Yan et al. (2026a) R. Yan, J. Cheng, S. Sun, Y. Sun, Y. Wu, W. Peng, Z. Wang, L. Liang, J. Xing, and Y. Cai Do less, achieve more: do we need every-step optimization for rl fine-tuning of diffusion models?. arXiv preprint arXiv:2605.15855. Cited by: §1. Yan et al. (2026b) R. Yan, J. Cheng, Y. Wu, L. Liang, W. Peng, A. V. Vasilakos, Q. Zhao, Y. Zhang, E. Adeli, K. M. Pohl, et al. Pixel-space diffusion transformers. arXiv preprint arXiv:2607.17585. Cited by: §1. Yang et al. (2024) K. Yang, J. Tao, J. Lyu, C. Ge, J. Chen, W. Shen, X. Zhu, and X. Li Using human feedback to fine-tune diffusion models without any reward model. In Computer Vision and Pattern Recognition, p. 8941–8951. Cited by: §1. Yuan et al. (2024) H. Yuan, Z. Chen, K. Ji, and Q. Gu Self-play fine-tuning of diffusion models for text-to-image generation. Advances in Neural Information Processing Systems 37, p. 73366–73398. Cited by: §1.1. Zhang et al. (2018) R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang The unreasonable effectiveness of deep features as a perceptual metric. In Computer Vision and Pattern Recognition, p. 586–595. Cited by: 1st item, §3.1. Zhang et al. (2026a) X. Zhang, H. Lin, H. Ye, J. Y. Zou, J. Ma, Y. Liang, and Y. Du Inference-time scaling of diffusion models through classical search. In International Conference on Learning Representations, Vol. 2026, p. 39133–39166. Cited by: §3.1. Zhang et al. (2026b) X. Zhang, C. Zhang, Z. Li, X. Wang, S. Cai, M. Lao, Y. Guo, and H. Zhuang Rep deep & machine learning: exemplar-free continual video action recognition via slow-fast collaborative learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 36075–36083. Cited by: §1. Appendix Figure 10. Showcase of generated samples. The images from our model achieve state-of-the-art performance. 1. Related Work 1.1. Diffusion Model Diffusion models have achieved remarkable progress in generative modeling. Their stability and gradual refinement allow them to approximate complex distributions, producing samples with both high fidelity and diversity. They have shown strong generalization across domains, including text-to-image (Li et al., 2024; Yuan et al., 2024; Ding et al., 2025), speech (Kong et al., 2020; Chen et al., 2020; Liu et al., 2022), molecular design (Xu et al., 2022; Xu et al., 2023b), and protein generation (Watson et al., 2023; Abramson et al., 2024; Qiao et al., 2022). However, since these models are typically pretrained on broad datasets lacking task-specific signals, they often fail to reflect user-desired distributions in specialized applications. This gap highlights the need for alignment:enabling customized optimization toward target distributions without losing generality. 1.2. Gradient-based Fine-tuning Methods Direct gradient-based fine-tuning is intuitively feasible for enhancing diffusion models. For examples, DRaFT (Clark et al., 2023) performs differentiable reward optimization for diffusion models by backpropagating reward gradients through sampling steps, forming a unified gradient-based framework for preference-aligned fine-tuning; DRTune (Wu et al., 2024) stabilizes reward-supervised diffusion training by applying stop-gradient to denoiser inputs and uniformly sampling K timesteps, enabling efficient supervision across all sampling depths; Diffusion-RPO (Gu et al., 2024) extends relative preference optimization to diffusion models by conducting stepwise preference contrasts with CLIP-based multimodal weighting to learn coherent cross-semantic preference patterns. All these methods improve preference alignment with limited stability, scalability, and robustness under diverse real-world undifferentiated reward signals. 1.3. Fine-tuning Methods Existing alignment approaches fall into two categories: fine-tuning (Black et al., 2023; Fan et al., 2023) and training-free methods. The mainstream fine-tuning , for example, relies on reinforcement learning (RL) (Sutton, 2018) to adjust diffusion model weights. RL greatly improves alignment to downstream targets but faces challenges: rewards are only available after the full denoising process, making them sparse and prone to reward hacking. This often leads to higher reward scores but reduced diversity and poor distribution coverage. Moreover, RL-based fine-tuning comes with a high computational cost (Kim et al., 2025). 1.4. Traing-free Methods Training-free methods (Kim et al., 2025; Xie and Gong, 2025) provide an alternative: instead of changing diffusion model weights, they bias the sampling process at inference to guide results toward target objectives. For examples, TTScale (Ma et al., 2025) boosts generation quality by shifting extra inference compute from longer denoising chains to noise-search guided by a lightweight evaluator. FKSteer (Singhal et al., 2025) enables inference-time controllability by using particle-based weighting and resampling to steer diffusion trajectories without fine-tuning. This avoids costly retraining and reduces computation. First, prior work lacks an adaptive mechanism to balance diversity and fidelity (Dhariwal and Nichol, 2021). Second, existing training-free methods often overemphasize target guidance, forcing trajectories into narrow regions and reducing coverage of the pretrained distribution. While less prone to severe reward hacking, this effect still harms diversity. Further, guidance signals are typically applied indiscriminately across all steps (Tang et al., 2024), without filtering ineffective or saturated signals. This overuse increases inference cost and amplifies noise, sometimes even causing negative guidance effects. 2. Method Supplement 2.1. Control Network We assign y=0y=0 to samples close to pure noise Tx_T, and y=1y=1 to samples close to the target 0x_0, pretraining the network via the loss defined as follows. (13) ℒ(θ)=−(t,y)[ylogfθ(t)+(1−y)log(1−fθ(t))].L(θ)=-E_(x_t,y) [y f_θ(x_t)+(1-y) (1-f_θ(x_t)) ]. 2.2. Generalizability of Pre-trained Control Network Considering training a control network could undermine the claim of “training-free” of the proposed SwiftExplorer, it is necessary to demonstrate that the Control Network could also be seen as a pre-trained scorer, which can be pre-trained once and be generalizable for all. Therefore, as shown in Tab. 8, we pre-train the control network on the simple animal set and predict steps on other datasets. The results demonstrate that the network could easily generalize to other datasets with similar effective performance, which is caused by the fixed scheduler having already pre-set the noise level for different steps in a coarse manner. 3. Additional Experiment results 3.1. Results with alternative optimization targets. We also conduct experiments by replacing PickScore with Aesthetic score as the optimization target to evaluate the adaptability of SwiftExplorer. As shown in Fig. 11, guidance with the aesthetic score demonstrates our overall superiority in aesthetic preference and generative diversity. Then, in the visual results, it could be clearly observed that our generated images contain richer details, better alignment, and improved aesthetic impressions. Figure 11. Results with Aesthetic Score as the optimization objective. 3.2. Detailed hyper-parameter setting Table 5. Hyper-parameters and metrics in our experiment. Notably, only n and δr _r are newly introduced by our method; all other hyper-parameters follow DDIM or the baseline methods. Name Description Value η eta parameter for the DDIM sampler 1.0 w classifier-free guidance weight 5.0 mdnom_dno Iteration Number for DNO 20 mdymom_dymo Iteration Number for DyMO adaptive ndasn_das Particle number for DAS 4 mpmp mixed precision fp16 n (SwiftExplorer-unique) Exploration supplement number 2 δr _r (SwiftExplorer-unique) Reward enhancement threshold 0.1 Table 6. Results of Restart Count and Early Stop across HPSv2 and Pick-a-Pic with SDv15 and SD-XL backbones. Avg. Value Base HPSv2 Pick-a-Pic SDv15 SD-XL SDv15 SD-XL Restart Count 50 4.37 8.55 7.32 10.59 Guiding Step 50 37.35 44.21 40.67 44.30 The full list of hyper-parameters in our paper is shown in Table 5. 3.3. Metric We provide a detailed illustration of each introduced metric: • Aesthetic Preference – AES (Aesthetic Score) AES is a metric used for evaluating the aesthetic quality of an image, typically by training a model to predict human aesthetic ratings (Schuhmann et al., 2022). – PS (PickScore) PS is a method based on human selection preferences for rating the aesthetic quality of images (Kirstain et al., 2023). – IR (ImageReward) IR is a metric used for assessing image quality, often in the optimization process of image generation models (Xu et al., 2023a). – HPSv2 (Human Preference Score v2) HPSv2 is a preference-based metric that predicts human preferences for generated images. This model fine-tunes the CLIP model on the HPD v2 dataset. HPSv2 excels in multiple styles, including animation, concept art, paintings, and photographs (Wu et al., 2023). • Image Fidelity – ClipScore ClipScore is a model-based image scoring metric that evaluates the similarity between a generated image and its textual description. It calculates the cosine similarity between the image and text embeddings. The metric is highly correlated with human judgment and does not require reference text (Radford et al., ). – Fréchet Inception Distance (FID) FID is a metric used to evaluate the quality of generated images. It compares the mean and covariance of the features extracted from real and generated images using the Inception v3 model. A lower FID value indicates that the generated image is closer to real images (Heusel et al., 2017). – Improved F1 Score (iFS) iFS is an improvement over the traditional F1 score, aiming to provide a better evaluation of generative model performance. It optimizes the balance between precision and recall for more accurate assessments (Kynkänniemi et al., 2019). • Generative Diversity – LPIPS (Learned Perceptual Image Patch Similarity) LPIPS is a metric for evaluating perceptual similarity between images. It compares the feature activations of images in pre-trained convolutional neural networks (e.g., VGG). A higher LPIPS value indicates lower perceptual similarity and a higher diversity(Zhang et al., 2018). – TCE (Truncated CLIP Entropy) Truncated CLIP Entropy (TCE) is a measure of semantic diversity of a set of generated images, computed in the joint image–text embedding space of CLIP. It is defined by first mapping each image xix_i to a vector i=CLIPimg(xi)∈ℝdz_i=CLIP_ img(x_i) ^d, then forming the empirical covariance matrix Σ of these vectors, extracting its top-k eigenvalues λ1≥⋯≥λk _1≥·s≥ _k, and finally computing TCEk=12∑i=1klogλi.TCE_k= 12 _i=1^k _i. TCE gives a tractable proxy for how “spread out” the images are in the CLIP semantic space. (Ibarrola and Grace, 2024). – Inception Score (IS) IS is a metric used to evaluate the quality of generated images. It calculates the entropy of the class distribution of images in a pre-trained Inception v3 model. Higher IS values indicate that the generated images are both clear and diverse (Salimans et al., 2016). • Compositional Richness – NIQE (Natural Image Quality Evaluator) NIQE is a no-reference image quality assessment metric. It evaluates the image quality by measuring the distance between the natural scene statistics of the image and a natural image database of undistorted images (Mittal et al., 2012b). A lower NIQE value indicates better image quality. – BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator) BRISQUE is another no-reference image quality metric. It evaluates image quality by analyzing spatial-domain features. It does not rely on reference images and is suitable for a variety of image quality assessment tasks (Mittal et al., 2012a). – Spectral Entropy (SE) SE is a metric for evaluating the spectral properties of images. It calculates the entropy value of an image’s frequency spectrum to assess the complexity and richness of image details. Higher SE values generally indicate more detailed and textured images (Liu et al., 2014). Figure 12. The visual comparisons of different methods on SDv15. For photorealistic prompts from HPSv2, our results contain richer details while maintaining image-prompt alignment. For surreal prompts from Pick-a-Pic, we exhibit much more diverse patterns and colors. Table 7. CoV (%) comparison of methods across four evaluation metrics. Method Preference Fidelity Diversity Richness DNO 0.88 0.52 0.27 1.27 DAS 0.96 0.81 0.30 1.36 DyMO 0.62 0.86 0.28 1.15 Ours 0.65 0.79 0.28 1.41 3.4. Supplementary Statistical Significance Since the generative quality is highly related to the initial noise decided by the seeds, we further conduct experiments with 5 different seeds and calculate their average Coefficient of Variation (CoV). The metrics and evaluation protocol are strictly following Tab. 1. As shown in Tab. 7, it can be observed that all methods exhibit similarly low CoV across various seeds, which indicates that the guidance-based training-free scaling methods perform relatively stably. These results further demonstrate that the performance reported in Tab. 1 is statistically significant. Table 8. Mean step numbers predicted by the control network across different datasets. The control network is trained on the simple animal and directly generalizes to other datasets. Method Total SimpleAnimal Pick-a-Pic HPSv2 GenEval SD15 50 37.3 40.9 39.8 38.1 SDXL 30 24.7 24.9 25.1 24.6 Figure 13. Feature visualization comparison with DAS. 3.5. Supplementary Distribution Comparison with DAS In Fig. 1, we compare the feature distribution with the iteration-based DyMO, thus demonstrating the diversity of our method. Here, we conduct a similar experiment to evaluate the sampling-based baseline, i.e., DAS. As shown in Fig. 13, although the sampling-based DAS has richer diversity compared to DyMO, it still tightens the distribution range of the original backbone. In contrast, our result exhibits consistent diversity, which is demonstrated by the wider visualized distribution and the lower Silhouette Coefficient value. Figure 14. The proposed ESV facilitates exploration during the denoising process. Left: ESV enables our method to identify a broader range of patterns and adjust the trajectory toward the “teddy bear lantern” during intermediate stages. Right: Our method exhibits greater posture variation throughout denoising. While this variation is not directly responsible for the superior final generation, it reflects an increased level of exploration. 3.6. Analysis on Exploration. To demonstrate the exploration effect of introducing ESV, we present the intermediate results during generation. As shown in Fig. 14, our method exhibits superior exploration effect during denoising, implying the improved generative quality. 3.7. Further Analysis on Ablation Study The proposed SwiftExplorer is designed considering two trade-offs, that is, Diversity-Fidelity and Efficiency-Quality trade-offs. Therefore, different components are introduced with paired correlations. Specifically, introducing M-ESV can introduce exploration and thus improve diversity, while L2 can maintain the fidelity to prevent over-exploration that leads to the collapse of the denoising process. Therefore, without ESV that encourages exploration, solely deploying L2 is unnecessary. Then, both I&R and EarS are introduced to enhance the generation quality based on the exploration. Meanwhile, GuiS builds upon the previously introduced exploration, as direct guidance typically provides a straightforward path to higher rewards, making GuiS ineffective in such cases. In contrast, with ESV, guidance is not based on the current state but rather incorporates an exploratory shift, allowing GuiS to effectively filter out harmful optimization. As for the intensity of different components in the trade-offs, we present Fig. 9 to demonstrate the superiority of our adaptive strategy. Overall, the components of our approach are interdependent and mutually reinforcing, collectively contributing to the superior performance of our image generation scheme in terms of inference-time scaling. Table 9. Results with SDv14. Method IR PS CLIP LPIPS TCE NIQE↓ SDv14 0.744 20.62 0.240 0.671 40.32 4.6312 DDPO 1.056 21.20 0.240 0.609 39.61 4.554 DNO 0.915 21.85 0.239 0.597 39.24 4.875 DAS 1.217 22.58 0.238 0.610 39.56 5.451 DyMO 1.100 23.14 0.235 0.615 39.63 4.550 Ours 1.230 23.39 0.242 0.676 41.10 3.494 Table 10. Results with SDv21. Method IR PS CLIP LPIPS TCE NIQE↓ SDv21 0.541 20.63 0.280 0.644 39.69 5.672 DDPO 0.794 21.25 0.274 0.622 38.94 4.953 DNO 0.566 20.91 0.277 0.589 39.13 5.551 DAS 0.817 22.29 0.280 0.656 39.55 5.150 DyMO 0.921 22.62 0.279 0.623 39.62 5.176 Ours 0.980 22.91 0.280 0.677 40.07 3.695 3.8. Results on more SD backbones. In addition to SDv15 and SD-XL presented in the main paper, we provide the results on SDv14 and SDv21-turbo. As shown in Tab. 10 and Tab. 10, our method also leads in all metrics. To further validate robustness on larger and stronger backbones, we additionally report results on advanced models in Tab. 11. The same trend remains clear: our method consistently improves preference-related quality while preserving fidelity and diversity. Compared with optimization-heavy baselines, it achieves a better balance between visual richness and prompt alignment, indicating that the plugin design remains effective when scaling to stronger pretrained models. Table 11. Results with advanced models. Method IR PS CLIP LPIPS TCE NIQE↓ SD-XL 0.715 21.45 0.236 0.675 40.18 4.553 Diff-DPO 1.010 22.05 0.236 0.651 39.45 4.440 SPO 1.179 22.81 0.232 0.556 40.42 3.900 SDv35 1.193 21.90 0.238 0.621 39.42 4.873 DNO 0.924 22.58 0.238 0.581 39.34 4.783 DAS 1.171 23.07 0.237 0.594 39.36 5.365 DyMO 1.079 24.34 0.233 0.609 39.55 4.510 Ours 1.201 24.56 0.236 0.679 40.99 3.573 3.9. Exhibited adaptive values for computation reduction. Here, we provide the exhibited average values of restart counts and early stop positions. Specifically, we calculate the average restart counts and average number of guiding steps on HPSv2 and Pick-a-Pic datasets with SDv15 and SD-XL. As shown in Tab. 6, it can be observed that HPSv2 has smaller values of both variables compared to Pick-a-Pic, while SDv15 has smaller values than SD-XL. This is because HPSv2 has more photorealistic images that are easier than surreal one for denoising, while SDv15’s lesser robustness makes it more amenable to external guidance. Algorithm 1 Algorithm of Swift Diversity Exploration (SwiftExplorer) Input: Total Inference Denoising Step OPENT)T); Initialized Latent: Tx_T; Pre-trained score function θ(⋅)s_θ(·). Initialize Inheritance Flag finh=Falsef_inh=False; for timestep t∈T,…,1t∈\T,...,1\ do execute one denoising step t=1αt+1(t+1∗+βt+1θ(t+1∗,t+1))+βt+1t+1.x_t= 1 _t+1 (x^*_t+1+ _t+1s_θ(x^*_t+1,t+1) )+ _t+1z_t+1. considering early-stopping Estop(t)=[pt=1∧Δr~t≤δr]∈0,1.E_stop(t)=I\! [p_t=1\ \ r_t≤ _r ]∈\0,1\. if Estop(t)==0E_stop(t)==0 then inheritance-restart mechanism for exploration variable if finhf_inh then inherit previous exploration variable ϵt(i)=t+1⋆−t+1,i=1 ε_t^(i)=x_t+1 -x_t+1, i=1 else sample new Monte Carlo exploration set ϵt(i)\ ε^(i)_t\ ϵt(i)∼(0,σt2I),i=1,…,n ε^(i)_t (0, _t^2I), i=1,…,n generate candidates with Diversity-Fidelity trade-off ^t(i)=t+(1−pt)⋅ϵ(i)−pt⋅∇tL2(^t,t)⏟Diversity-Fidelity Trade-off+∇tℛ(0|t), x_t^(i)=x_t+ (1-p_t)· ε^(i)-p_t· _x_tL_2( x_t,x_t)_Diversity-Fidelity Trade-off+ _x_tR(x_0|t), iteratively optimizing candidates ~t(i)=g(^t(i),ℛ−L2,m) x_t^(i)=g( x_t^(i),R-L_2,m) obtain the optimal candidates t⋆=~t(i⋆),i⋆=argmaxi∈1,…,nℛ(~0|t(i)). _t = x_t^(i ), i = _i∈\1,…,n\R( x_0|t^(i)). verify optimization condition Δℛt(i⋆)=ℛ(0|t⋆)−ℛ(0|t), _t^(i )=R(x_0|t )-R(x_0|t), if Δℛt(i⋆)>0 _t^(i )>0 then finh=Truef_inh=True else t∗=tx^*_t=x_t finh=Falsef_inh=False Output: Optimized Inference Result 0x_0. 3.10. User Study Details. In this study, we assessed the plug-and-play effectiveness of our method through a human evaluation, where five evaluators compared our method to four competing models: DNO, DyMO XL, Gdt, and DyMO. The evaluation was method-blind, with participants unaware of which model generated each image. The evaluation criteria is: • Win: Our method was preferred. • Tie: Both methods were considered equally effective, hard to decide distinguishable preference. • Lose: Our method was less preferred. Each participant compared the generated images based on overall quality, with the results shown in Fig. 6. The preference for our method across different models demonstrates its superior performance in plug-and-play integration. 4. Algorithm of the proposed SwiftExplorer For improved delivery of the proposed SwiftExplorer, we show its detailed algorithm in Alg. 1. 5. Showcase Prompt Table Please refers to Tab. 12, 14, 15, 16, and 13. Table 12. Detailed prompts used for generated images in Fig. 10 - Part 1 Image Prompt Fig. 1, Row 1, Col 1 ballet dancer, insanely detailed, photorealistic, 8k, perfect composition, volumetric lighting, natural complexion, award-winning professional photography, taken with Canon EOS 5D Mark IV, 85m, mindblowing, masterpiece Fig. 1, Row 1, Col 2 A fluffy bunny as Rapunzel, with long golden ears flowing down from a tall enchanted tower, glowing lanterns in the night sky, warm fairytale atmosphere Fig. 1, Row 1, Col 3 A dog in sportswear lifting tiny dumbbells at the gym, determined expression, humorous fitness illustration Fig. 1, Row 1, Col 4 A boy superhero landing on the ground in classic “hero pose,” debris and glowing sparks flying around, comic action shot Fig. 1, Row 1, Col 5 A phoenix rising up from ashes Fig. 1, Row 2, Col 1 A young girl standing on a rooftop, blowing dandelions that transform into glowing comets, shooting across the night sky, dreamy fantasy artwork Fig. 1, Row 2, Col 2 A small hedgehog as the Frog Prince, wearing a tiny crown while sitting on a lilypad, a kind-hearted swan princess leaning close, surrounded by glowing fireflies, magical fairytale illustration Fig. 1, Row 2, Col 3 a white polar bear cub wearing sunglasses sits in a meadow with flowers. Fig. 1, Row 2, Col 4 A cat surfing on a giant wave at sunset, wearing cool shades, cinematic sports illustration Fig. 1, Row 2, Col 5 A sunflower in full bloom under golden sunlight, tiny dewdrops sparkling on its petals, cinematic macro fantasy illustration Table 13. Detailed prompts used for generated images in Fig. 9 Image Prompt Fig. 9, Row 1 Heart shaped balloon Fig. 9, Row 2 An apple on a table Fig. 9, Row 3 A castle in the sky, clouds, sunset, explosion Fig. 9, Row 4 Harry potter as a cat, pixar style, octane render, HD, high-detail Table 14. Detailed prompts used for generated images in Fig. 10 - Part 2 Image Prompt Fig. 1, Row 3, Col 1 A warrior standing at the edge of a glowing crater, surrounded by swirling cosmic energy, their silhouette outlined against the birth of a new star, ultimate epic fantasy art Fig. 1, Row 3, Col 2 A boy lying on the grass in a field, listening to music with glowing headphones, fireflies surrounding him Fig. 1, Row 3, Col 3 A little girl painting a rainbow bridge from the classroom window into the sky, playful magical fairytale art, hopeful and inspiring Fig. 1, Row 3, Col 4 A group of playful penguins throwing glowing snowballs at each other, each snowball turning into sparkling stars when it explodes, magical fairytale scene Fig. 1, Row 3, Col 5 Giant rubber duck floating in the ocean with a small island on its back, surrounded by tropical palm trees and crystal clear water, bright and sunny day, calm seas, vivid colors, cinematic lighting, high detail Fig. 1, Row 3, Col 6 A cozy library built inside an ancient oak tree, warm lights glowing through round windows, whimsical fairytale healing atmosphere Fig. 1, Row 3, Col 7 A brave boy carrying a glowing lantern, releasing trails of light that form a golden sunrise, cinematic epic fantasy style Table 15. Detailed prompts used for generated images in Fig. 11 Image Prompt Fig. 11, Row 1 transylvania castle on hiltop and dusk bats and scifi Fig. 11, Row 2 fantasy character portrait digital painting, anime style, detailed with beautiful emotive lighting suggesting personality, and background that suggests character backstory Fig. 11, Row 3 Women in Saree playing Holi Fig. 11, Row 4 a house burning at night Table 16. Detailed prompts used for generated images in Fig. 7 Image Prompt Fig. 7, Row 1 a slice of pizza floating through space with stars in the background Fig. 7, Row 2 A Great Dane dog in the style of Vincent Van Gogh