Paper deep dive
Best-of-$N$ Guidance for Test-time Diffusion Alignment
Richard Lee Kim, Yeongmin Kim, Gyuwon Sim, Taekyu Kim, Minsang Park, Il-chul Moon
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/7/2026, 8:20:18 AM
Summary
The paper introduces Best-of-N Guidance (BoNG), a test-time alignment method for diffusion models that integrates Best-of-N (BoN) sampling principles directly into the reverse diffusion process. Unlike vanilla BoN which selects the best sample only after generation, BoNG performs online selection among denoising particles and uses the highest-reward particle as a guidance signal to steer the entire population toward higher-reward regions. This approach improves both the quality of the single best sample and the average quality of multiple outputs, achieving state-of-the-art performance with significant speedups compared to existing sample-based guidance methods.
Entities (8)
Relation Signals (8)
BoNG → isbasedon → BoN
confidence 95% · We propose Best-of-N Guidance (BoNG), a novel method that integrates the principle of BoN sampling directly into the reverse diffusion process.
BoNG → supports → Multi-output capability
confidence 95% · BoNG also supports multi-output capability
BoNG → outperforms → SMC
confidence 92% · BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling.
BoNG → outperforms → BoN
confidence 92% · BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling.
BoNG → achievesspeedup → 1.6x
confidence 90% · BoNG also supports multi-output capability, achieving 1.3× ImageReward score of the latest sample-based guidance method with a 1.6× speedup.
BoN → haslimitation → Single-output setting
confidence 90% · Consequently, BoN sampling does not improve the average alignment of generated samples and is primarily suited to single-output settings.
SMC → haslimitation → Particle collapse
confidence 90% · SMC is prone to particle collapse.
BoNG → improves → ImageReward
confidence 90% · BoNG also supports multi-output capability, achieving 1.3× ImageReward score of the latest sample-based guidance method
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Diffusion models achieve strong generative performance but often struggle to align generated samples with human preferences measured by a reward model. A simple yet effective algorithm for test-time alignment is Best-of-$N$ (BoN) sampling, which draws $N$ i.i.d. samples from a pre-trained diffusion model and outputs the single highest-reward sample. Despite its empirical success, BoN makes limited use of reward information, as it is incorporated only at the final selection stage without influencing the reverse diffusion trajectory during sampling. Consequently, BoN sampling does not improve the average alignment of generated samples and is primarily suited to single-output settings. We propose Best-of-$N$ Guidance (BoNG), a novel method that integrates the principle of BoN sampling directly into the reverse diffusion process. BoNG performs online BoN selection over denoising particles and adjusts the reverse diffusion process to steer the particle population toward higher-reward regions during generation. Specifically, by introducing an asymmetric guidance interaction among denoising particles, BoNG uses the current BoN particle as a guidance signal to the rest of the particle population. This particle-level interaction reshapes the sampling process toward higher-reward regions, enabling BoNG to improve not only the final best sample beyond Vanilla BoN sampling, but also the average quality of generated samples. Over 36 empirical comparisons, BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling. BoNG also supports multi-output capability, achieving 1.3$\times$ ImageReward score of the latest sample-based guidance method with a 1.6$\times$ speedup. We release the code at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.05108v2
- Canonical: https://arxiv.org/abs/2610.05108v2
Trouble viewing inline? Open PDF directly →
Full Text
101,169 characters extracted from source content.
Expand or collapse full text
Best-of-N Guidance for Test-time Diffusion Alignment Richard Lee Kim Affiliation: KAIST Email: swimskykim@kaist.ac.kr Yeongmin Kim Affiliation: KAIST Email: alsdudrla10@kaist.ac.kr Gyuwon Sim Affiliation: KAIST Email: gkwlaks4886@kaist.ac.kr Taekyu Kim Affiliation: KAIST Email: taekyu@kaist.ac.kr Minsang Park Affiliation: KAIST Email: pagemu@kaist.ac.kr Il-Chul Moon Affiliation: KAIST Email: icmoon@kaist.ac.kr Affiliation: summary.ai Abstract Diffusion models achieve strong generative performance but often struggle to align generated samples with human preferences measured by a reward model. A simple yet effective algorithm for test-time alignment is Best-of-N (BoN) sampling, which draws N i.i.d. samples from a pre-trained diffusion model and outputs the single highest-reward sample. Despite its empirical success, BoN makes limited use of reward information, as it is incorporated only at the final selection stage without influencing the reverse diffusion trajectory during sampling. Consequently, BoN sampling does not improve the average alignment of generated samples and is primarily suited to single-output settings. We propose Best-of-N Guidance (BoNG), a novel method that integrates the principle of BoN sampling directly into the reverse diffusion process. BoNG performs online BoN selection over denoising particles and adjusts the reverse diffusion process to steer the particle population toward higher-reward regions during generation. Specifically, by introducing an asymmetric guidance interaction among denoising particles, BoNG uses the current BoN particle as a guidance signal to the rest of the particle population. This particle-level interaction reshapes the sampling process toward higher-reward regions, enabling BoNG to improve not only the final best sample beyond Vanilla BoN sampling, but also the average quality of generated samples. Over 36 empirical comparisons, BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling. BoNG also supports multi-output capability, achieving 1.3× ImageReward score of the latest sample-based guidance method with a 1.6× speedup. We release the code at https://github.com/aailab-kaist/BoNG. 1 Introduction Diffusion models Sohl-Dickstein et al. (2015); Ho et al. (2020); Song et al. (2021) demonstrate strong generative capabilities, leading to a growing number of real-world applications that produce outputs conditioned on user inputs Microsoft Team (2026); xAI Team (2026). However, the generated outputs still do not fully align with user preferences in terms of input alignment and sample fidelity. To address this issue, reward functions Xu et al. (2023); Wu et al. (2023); Ma et al. (2025) that evaluate alignment with user conditions and output quality have been incorporated into the generation process. Recently, test-time alignment methods have shown superior performance compared to parameter-scaling Podell et al. (2024); Labs (2024) or fine-tuning approaches Black et al. (2024); Wallace et al. (2024), attracting increasing research interest in this direction Snell et al. (2024); Singhal et al. (2025); Kim et al. (2026). Best-of-N (BoN) sampling and Sequential Monte Carlo (SMC) Singhal et al. (2025); Li et al. (2025); Skreta et al. (2025) are widely used test-time alignment methods for diffusion models. However, both approaches have inherent limitations: BoN selects only a single sample, while SMC is prone to particle collapse. As a result, they are primarily suited to scenarios where only one sample is ultimately served. In contrast, guidance-based methods enable the generation of diverse samples by independently adjusting the denoising direction for each sample, making them applicable to settings that require multiple outputs. However, early approaches, such as gradient guidance Kim et al. (2023); Bansal et al. (2024); Na et al. (2025), require neural backward passes with differentiable rewards, leading to substantial computational overhead and are prone to reward hacking. Sample-based guidance is emerging as a promising direction, as it can generate diverse samples without requiring neural backward passes. However, it often becomes a computational bottleneck, since iterative rollout sampling requires generating final samples at each time step. Consequently, this line of research has focused on reducing the cost of rollout Potaptchik et al. (2026a); Holderrieth et al. (2026); Potaptchik et al. (2026b); Kim et al. (2026), and has recently achieved promising results in both performance and efficiency Kim et al. (2026). However, it still incurs rollout cost, resulting in 2–3× higher sampling cost compared to vanilla sampling. We propose Best-of-N Guidance (BoNG), a rollout-free, sample-based guidance method inspired by BoN sampling. Unlike Vanilla BoN which uses the reward only once to select the best sample after generation, BoNG performs this selection online and injects it into the reverse diffusion process as a guidance signal (Figure 1). Specifically, we replace the iterative rollout process by selecting the highest-reward expected denoised sample among multiple particles at several intermediate timesteps. This approach steers all samples toward the direction of the highest-reward sample without incurring additional feed-forward computations of the score network. We theoretically analyze its connection to the reward-tilted distribution targeted by prior work, and empirically demonstrate gains over Vanilla BoN, SMC, and prior guidance methods with minimal additional inference cost. Figure 1: Best-of-N Guidance (BoNG) (Left) Vanilla BoN sampling selects the highest-reward sample only after generation. (Middle) BoNG performs online BoN selection (Algorithm 1) and uses the selected sample to guide the remaining particles (Algorithm 2). (Right) BoNG consistently improves single-output performance over Vanilla BoN and SMC across various sample budgets. 2 Related Work BoN sampling and approaches based on reward-tilted distributions have been widely studied in test-time diffusion alignment. Given a pre-trained diffusion model pθ(0)p_θ(x_0) and a human reward function r(0)r(x_0), the reward-tilted distribution is defined as pθr(0)∝pθ(0)exp(r(0))p^r_θ(x_0) p_θ(x_0) (r(x_0)). To approximate this distribution, previous work has studied how to estimate the intractable expected future reward rt(t):=pθ(0|t)[exp(r(0))]r_t(x_t):=E_p_θ(x_0|x_t) [ (r(x_0)) ] for a noisy sample tx_t, which induces the intermediate distribution pθr(t)∝pθ(t)rt(t)p_θ^r(x_t) p_θ(x_t)r_t(x_t) Uehara et al. (2024); Domingo-Enrich et al. (2025). SMC methods Singhal et al. (2025); Li et al. (2025); Skreta et al. (2025) generate multiple samples from the base diffusion model and perform resampling based on intermediate reward values. However, SMC suffers from particle collapse, which degrades the quality of the best sample in single-output generation and limits sample diversity in multi-output settings. Gradient-guidance methods address this issue by enabling i.i.d. sampling by approximating the target time-dependent score function. However, these methods require either training diffusion time-conditioned reward models Kim et al. (2023) or performing multiple neural backward passes with Taylor approximation Bansal et al. (2024); Na et al. (2025) at every denoising step, resulting in significant latency and memory overhead. Sample-based guidance methods Potaptchik et al. (2026a); Kim et al. (2026) have recently emerged as a promising direction because they avoid particle collapse and do not require neural backward propagation for guidance. Tilt Potaptchik et al. (2026a) proposes a gradient-free method for incorporating expected future rewards into the reverse process. However, it remains computationally expensive because it requires generating multiple samples from tx_t to 0x_0 at every timestep t, known as backward rollout. GlassFlow Holderrieth et al. (2026) improves efficiency by reducing the number of rollout steps through ODE-based sampling, and MFM Potaptchik et al. (2026b) further accelerates this process by replacing the ODE solver with a distilled model. LiDAR Kim et al. (2026) proposes a forward rollout strategy for estimating intermediate rewards. This reduces per-step rollout sampling to a single marginal rollout. LiDAR significantly improves sample-based guidance and achieves state-of-the-art performance while being around 10× faster than gradient-guidance methods. However, it still requires additional lookahead sampling, making it 2–3× slower than vanilla diffusion sampling. The goal of this work is to develop an efficient and effective sample-based guidance method inspired by the idea of BoN sampling. 3 Preliminary 3.1 Diffusion Models A diffusion model Sohl-Dickstein et al. (2015); Ho et al. (2020); Song et al. (2021) consists of a pair of stochastic processes: a forward process and a reverse process. The forward diffusion process injects noise into the data 0∼p(0)x_0 p(x_0) to produce a perturbed sample tx_t via a Gaussian kernel: p(t|0)=(t,0,σt2),p(x_t|x_0)=N(x_t;x_0, _t^2I), for every t∈[0,T]t∈[0,T], with a corresponding gradually increasing noise scale σt _t. This perturbation produces the marginal distribution p(t)p(x_t), and the terminal noise scale σT _T is chosen to be sufficiently large so that p(T)p(x_T) is close to a pure Gaussian prior. The reverse process generates a data 0x_0 from a noise T∼p(T)x_T p(x_T), and is often represented as a stochastic differential equation Anderson (1982); Song et al. (2021): dt=[(t,t)−g2(t)∇tlogp(t)]dt+g(t)dwt, dx_t= [f(x_t,t)-g^2(t) _x_t p(x_t) ]dt+g(t)dw_t, (1) where the drift f and volatility g terms are defined to correspond to the noise schedule σt _t, and wtw_t is a standard Wiener process. The diffusion model constructs the reverse process by estimating the score function ∇tlogp(t)≈∇tlogpθ(t) _x_t p(x_t)≈ _x_t p_θ(x_t) using a score model trained with denoising score matching Vincent (2011); Song and Ermon (2019) or noise prediction Ho et al. (2020) loss functions. 3.2 Test-time Human Alignment Problem Setup: With the advancement of diffusion models, many pre-trained models are now available as open-source Rombach et al. (2022); Podell et al. (2024); Labs (2024), enabling generation conditioned on user input c. However, the final generated sample 0x_0 often does not align well with the user input c. Accordingly, we assume access to a pre-trained diffusion model pθ(0|)p_θ(x_0|c) and a human preference reward function r:0×→ℝr:X_0×C , and aim to obtain samples 0x_0 such that reward r(0,)r(x_0,c) is high, by manipulating the inference process. Here, 0X_0 denotes the data space at time t=0t=0, and C denotes the user’s input condition space. 3.2.1 Best-of-N Sampling Best-of-N (BoN) sampling is a post-hoc selection procedure that draws N i.i.d. samples from the base distribution pθ(0∣)p_θ(x_0 c) and returns a single sample with the highest reward. In finite N, the BoN sample from the base distribution is defined as: 0BoN:=argmax0ii=1Nr(0i,),0i∼i.i.d.pθ(0∣) x_0^BoN:= \x_0^i\_i=1^Nargmax\;\;r(x_0^i,c), x_0^i i.i.d. p_θ(x_0 c) (2) BoN sampling is a remarkably strong inference-time alignment strategy in practice Ichihara et al. (2025), since it can substantially improve the quality of the final selected sample by increasing N, without modifying the underlying generative model. However, it remains fundamentally inefficient: the sampling process is done by Eq. 1, reward information is not used during the generation process. As a result, obtaining a sample from a high-reward region may require large N. Moreover, BoN sampling is inherently designed to return a single best output, and therefore does not directly address the multi-output setting Microsoft Team (2026); xAI Team (2026) in which all N generated candidates are expected to be of high quality. 3.2.2 Sampling from a Reward-tilted Distribution A line of research Singhal et al. (2025); Holderrieth et al. (2026); Kim et al. (2026) aims to sample from the reward-tilted distribution for alignment defined as: pθr(0|)∝pθ(0|)exp(λr(0,)), p_θ^r(x_0|c) p_θ(x_0|c) (λ r(x_0,c)), (3) where λ is a scalar hyperparameter that controls the strength of the reward. However, since this goal is defined on the final generated sample 0x_0, it is challenging to determine how to modify the iterative denoising process in Eq. 1. We follow a popular intermediate formulation from prior work Uehara et al. (2024); Domingo-Enrich et al. (2025); Holderrieth et al. (2026) and adopt the score-based formulation of Kim et al. (2026) as: ∇tlogpθr(t|)=∇tlogpθ(t|)+∇trtλ(t,), _x_t p_θ^r(x_t|c)= _x_t p_θ(x_t|c)+ _x_tr_t^λ(x_t,c), (4) where rtλ(t,):=logpθ(0|t,)[exp(λr(0,))]r_t^λ(x_t,c):= _p_θ(x_0|x_t,c) [ (λ r(x_0,c)) ] is expected future reward at particle tx_t. Denoising via Eq. 1 with the score replaced by Eq. 4 yields samples from the reward-tilted distribution in Eq. 3. Note that computing rtλr_t^λ is generally intractable, so most previous work focuses on how to approximate this term efficiently and accurately. Although this reverse-process modification is well suited for multi-output generation, existing approximations typically require costly gradient computation Bansal et al. (2024); Na et al. (2025) or rollout sampling Potaptchik et al. (2026a); Kim et al. (2026), making them less efficient for test-time alignment. 4 Method To mitigate the inefficiency and limited capability in multi-output settings, we refine the idea of Best-of-N sampling as a diffusion-based guidance scheme. A natural approach is to inject the reward information of 0BoNx_0^BoN in Eq. 2 directly into the reverse diffusion process, but doing so requires samples from 0X_0. Following prior works Bansal et al. (2024); Na et al. (2025); Singhal et al. (2025), we use one-step denoised samples as an alternative to approximate 0BoNx_0^BoN, and guide the particle population towards it. We construct our Best-of-N guidance algorithm in two steps. 4.1 Best-of-N Guidance (BoNG) Step 1: Best-of-N target in 0X_0. Let tii=1N\x_t^i\_i=1^N denote the current particle population. For each particle, we compute its one-step denoised sample ^0i:=pθ(0∣ti,)[0]=ti+σt2∇tilogpθ(ti∣). x_0^i:=E_p_θ(x_0 x_t^i,c)[x_0]=x_t^i+ _t^2 _x_t^i p_θ(x_t^i c). (5) These one-step denoised samples, ^0ii=1N\ x_0^i\_i=1^N, define a candidate set in 0X_0. We then perform Best-of-N selection over this set and define i∗=argmaxr(^0i,),^0BoN:=^0i∗,tBoN:=ti∗. i^*= iargmax\;\;r( x_0^i,c), x_0^BoN:= x_0^i^*, x_t^BoN:=x_t^i^*. (6) Step 2: Adjust each particle toward ^0BoN x_0^BoN. We define the BoN target score for each particle so that its one-step denoised sample matches ^0BoN x_0^BoN. Comparing this target with the one-step denoised sample in Eq. 5, we obtain ∇tilogpθBoN(ti∣)⏟BoN target score:=∇tilogpθ(ti∣)+1σt2(^0BoN−^0i)⏟:=∇tirtBoN(ti,)(BoN guidance term). _x_t^i p_θ^BoN(x_t^i c)_BoN target score:= _x_t^i p_θ(x_t^i c)+ 1 _t^2 ( x_0^BoN- x_0^i )_:=\, _x_t^ir_t^BoN(x_t^i,c)\ (BoN guidance term). (7) The Best-of-N Guidance (BoNG) method is defined by using the BoN guidance term to construct the BoN target score at each timestep t. See Algorithms 1 and 2. The BoN target score ∇tilogpθBoN(ti∣) _x_t^i p_θ^BoN(x_t^i c) in Eq. 7 admits a simple interpretation. For tBoNx_t^BoN, the BoN guidance term ∇tirtBoN(ti,) _x_t^ir_t^BoN(x_t^i,c) vanishes since (^0BoN−^0BoN)/σt2=0 ( x_0^BoN- x_0^BoN )/ _t^2=0, and thus tBoNx_t^BoN is denoised according to the original reverse diffusion process without additional guidance. For any particle tix_t^i with i≠i∗i≠ i^*, the update can be interpreted as the sum of two terms: ∇tilogpθBoN(ti∣)=∇tBoNlogpθ(tBoN∣)⏟BoN Score+1σt2(tBoN−ti)⏟Positive Guidance. _x_t^i p_θ^BoN(x_t^i c)= _x_t^BoN p_θ(x_t^BoN c)_BoN Score+ 1 _t^2 (x_t^BoN-x_t^i )_Positive Guidance. (8) This decomposition shows that BoNG induces an asymmetric interaction among particles: each particle tix_t^i with i≠i∗i≠ i^* is attracted toward the current BoN particle by positive guidance, while simultaneously following the score-induced denoising direction of the BoN trajectory by BoN score. Overall, BoNG embeds the strong selection behavior of Best-of-N sampling via a guidance term without additional feed-forward computations of the score network. Figure 2: Dynamics of BoNG. (Left) Algorithm Summary. (Right) Decomposition of BoN target score in Eq. 7. Algorithm 1 Best-of-N selection Input: tii=1N,t,,pθ,r\x_t^i\_i=1^N,\,t,\,c,\,p_θ,\,r 1. for i=1,…,Ni=1,…,N do ^0i←pθ(0∣ti,)[0] x_0^i _p_θ(x_0 x_t^i,c)[x_0] end for 2. Select i∗i^* according to Eq. 6 3. tBoN←ti∗x_t^BoN←x_t^i^*, ^0BoN←^0i∗ x_0^BoN← x_0^i^* Output: tBoN,^0BoN,^0ii=1Nx_t^BoN,\, x_0^BoN,\,\ x_0^i\_i=1^N Algorithm 2 One-step BoNG update Input: tii=1N,t,,pθ,r,σt\x_t^i\_i=1^N,\,t,\,c,\,p_θ,\,r,\, _t 1. Obtain tBoN,^0BoN,^0ii=1Nx_t^BoN,\, x_0^BoN,\,\ x_0^i\_i=1^N from Algorithm 1 2. for i=1,…,Ni=1,…,N do Update tix_t^i to t−1ix_t-1^i using Eq. 7 end for Output: t−1ii=1N\x_t-1^i\_i=1^N 4.2 Connection to Prior Sample-based Test-time Alignment Methods We now connect BoNG to prior alignment methods that sample from the reward-tilted distribution in Eq. 3. Lemma 4.1 and Lemma 4.2 provide a general analysis of the reward-tilted distribution that holds for any λ>0λ>0: they reduce reward guidance to estimating the tilted posterior mean and give its self-normalized importance sampling (SNIS) estimator. Theorem 4.3 then connects this general analysis to BoNG, showing that hard Best-of-N selection realizes the large-λ limit of the SNIS estimator, with Algorithms 1 and 2 stating the resulting procedure. Lemma 4.1. The gradient of the expected future reward can be expressed as ∇trtλ(t,)=1σt2(mλr(t)−pθ(0∣t,)[0]),mλr(t):=pθr(0∣t,)[0]. _x_tr_t^λ(x_t,c)= 1 _t^2 (m_λ^r(x_t)-E_p_θ(x_0 x_t,c)[x_0] ), m_λ^r(x_t):=E_p_θ^r(x_0 x_t,c)[x_0]. (9) Lemma 4.1 follows from the Fisher identity and related score-based formulations (Kim et al., 2026; Potaptchik et al., 2026a); its role is preparatory. It shows that reward guidance requires no reward derivatives and reduces to estimating the reward-tilted posterior mean mλr(t)m_λ^r(x_t). Derivation is in Section A.1. We now provide a Monte Carlo approximation scheme of mλr(t)m_λ^r(x_t) via self-normalized importance sampling (Swaminathan and Joachims, 2015) (SNIS) by the following lemma, which holds for any λ>0λ>0. Lemma 4.2. Let q(0)q(x_0) be any proposal distribution whose support is 0X_0. Then, given proposal samples ^01,…,^0N∼i.i.d.q(0) x_0^1,…, x_0^N i.i.d. q(x_0), the reward-tilted posterior mean mλr(t)m_λ^r(x_t) admits the self-normalized importance sampling approximation mλr(t)≈m^N,λr(t):=∑i=1Nαi(λ)^0i, m_λ^r(x_t)≈ m_N,λ^r(x_t):= _i=1^N _i^(λ) x_0^i, (10) where αi(λ)=softmaxi(logpθ(^0i∣t,)−logq(^0i)+λr(^0i,)). _i^(λ)=softmax_i\! ( p_θ( x_0^i x_t,c)- q( x_0^i)+λ r( x_0^i,c) ). Proof is in Section A.2. Lemma 4.2 provides a unified view of prior sample-based guidance methods: LiDAR (Kim et al., 2026) corresponds to the choice q(0)=pθ(0∣)q(x_0)=p_θ(x_0 c), while Tilt (Potaptchik et al., 2026a) uses q(0)=pθ(0∣t,)q(x_0)=p_θ(x_0 x_t,c). Although these methods differ in their choice of proposal distribution, both approximate mλr(t)m_λ^r(x_t) through weighted proposal samples. As λ→∞λ→∞, the softmax weights concentrate on the highest-reward proposal sample, with αi(λ)→1 _i^(λ)→ 1 for the maximizer and αi(λ)→0 _i^(λ)→ 0 otherwise, so the SNIS estimator reduces to the highest-reward sample in the proposal set. The following theorem shows that, in this large-λ limit, the process target score in Eq. 4 coincides with the BoN target score of BoNG in Eq. 7 when the proposal set is given by the one-step denoised samples ^0ii=1N\ x_0^i\_i=1^N with ^0i=pθ(0∣ti,)[0] x_0^i=E_p_θ(x_0 x_t^i,c)[x_0] in Eq. 5. Figure 3: Qualitative comparison of generated samples from Vanilla, SMC, and BoNG (SD v1.5 DDPM 100 step sampler). Red boxes denote Best-of-N samples for each method. Vanilla sampling often fails to generate images which align well with specific prompts such as “below a vase” and “a blue potted plant”. SMC generates better aligned images, but resampling leads to premature particle collapse. In contrast, BoNG generates samples that remain well aligned with the prompt while preserving diverse layouts and appearances. Theorem 4.3. Let ^0ii=1N\ x_0^i\_i=1^N be the proposal set of one-step denoised samples in Eq. 5 with a unique maximizer i∗=argmax1≤i≤Nr(^0i,)i^*= _1≤ i≤ Nr( x_0^i,c), and let ^0BoN:=^0i∗ x_0^BoN:= x_0^i^*. Then m^N,λr(ti)→^0BoN m_N,λ^r(x_t^i)→ x_0^BoN as λ→∞λ→∞, and the SNIS approximation of the reward-guidance gradient in Eq. 9 converges to the BoN guidance term for each particle: ∇tirtλ(ti,)≈1σt2(m^N,λr(ti)−^0i)→λ→∞∇tirtBoN(ti,). _x_t^ir_t^λ(x_t^i,c)\;≈\; 1 _t^2 ( m_N,λ^r(x_t^i)- x_0^i )\; \;λ→∞\;\; _x_t^ir_t^BoN(x_t^i,c). (11) Proof is in Section A.3. Theorem 4.3 shows that BoNG realizes the large-λ limit of sample-based guidance toward the reward-tilted distribution, while eliminating the costly forward or backward rollouts of prior methods. Here, the one-step Tweedie estimates serve as a surrogate proposal set rather than exact i.i.d. proposal samples. The same construction also admits a finite-λ relaxation: replacing the hard selection with the soft SNIS weights αi(λ) _i^(λ) of Lemma 4.2 yields a guidance target that interpolates between the average of the proposal set and the BoN target, so that λ directly controls the strength of reward tilting. We refer to Appendix E for the full derivation, the approximation gaps of this soft variant, and a λ sweep verifying its predicted reward–diversity scaling. 4.3 Instantiation of BoNG BoNG can be applied flexibly during inference time, which allows the method to interpolate between aggressive reward optimization and diversity preservation. Similarly to SMC Singhal et al. (2025), we instantiate BoNG by specifying a guidance timestep window, i.e. the subset of reverse diffusion steps at which the BoN guidance term is activated. The guidance window determines how often the particle population is steered toward the current Best-of-N target during reverse diffusion. Applying BoNG over more timesteps yields stronger steering toward high-reward regions, whereas restricting the window leaves more denoising steps governed by the original diffusion process. In this sense, the guidance timestep window in BoNG plays a role analogous to the reward-strength hyperparameter λ in prior methods Bansal et al. (2024); Na et al. (2025); Li et al. (2025) targeting the reward-tilted distribution in Eq. 3. Consistent with Theorem 4.3, increasing the guidance window corresponds to a more reward-tilted target distribution. In addition, following prior guidance methods Chung et al. (2024); Ho and Salimans (2021), we introduce a guidance scale multiplying the BoN guidance term in Eq. 7. This provides an additional degree of control over the magnitude of guidance at each timestep. Further analysis of BoNG is provided in Appendices C and D.1. 5 Experiments We use GenEval Ghosh et al. (2023) prompts for evaluation and ImageReward (IR) Xu et al. (2023) as the reward function during inference time. We report IR, CLIP Score Hessel et al. (2021), and GenEval score for all methods. For single-output performance we compare against Vanilla BoN sampling and Sequential Monte Carlo (SMC) Singhal et al. (2025), since both methods are designed for best-sample generation under N, with comparable computational cost. We report the performance of the BoN sample (i.e., highest IR particle) accordingly. For multi-output performance we compare against representative guidance-based baselines, including gradient-guidance Bansal et al. (2024); Na et al. (2025) and sample-based guidance Kim et al. (2026) methods. For each method, we report the average reward of 4 generated samples. See Appendix B for more details. Table 1: Best-of-N (BoN) performance on GenEval prompts with different numbers of generated images N, using SD v1.5 Rombach et al. (2022) and SDXL Podell et al. (2024) backbones. ImageReward (IR) is used as the reward for all methods. Bold entries denote the top-performing results within each backbone group. Performance (↑)( ) Backbone Method N=2 N=2 N=3 N=3 N=4 N=4 IR CLIP GenEval IR CLIP GenEval IR CLIP GenEval SD v1.5 (0.9B) Vanilla 0.317 0.277 0.473 0.545 0.281 0.496 0.682 0.283 0.520 (w/ DDIM 50 steps) SMC 0.317 0.277 0.473 0.532 0.281 0.512 0.636 0.282 0.509 BoNG 0.407 0.280 0.480 0.659 0.283 0.507 0.793 0.285 0.535 SD v1.5 (0.9B) Vanilla 0.396 0.279 0.481 0.605 0.282 0.513 0.742 0.284 0.525 (w/ DDPM 100 steps) SMC 0.396 0.279 0.481 0.675 0.283 0.511 0.826 0.286 0.529 BoNG 0.511 0.280 0.491 0.749 0.284 0.521 0.894 0.286 0.555 SDXL (2.6B) Vanilla 0.899 0.285 0.578 1.060 0.288 0.600 1.140 0.289 0.613 (w/ DDIM 50 steps) SMC 0.899 0.285 0.578 1.011 0.287 0.596 1.093 0.289 0.612 BoNG 0.911 0.287 0.573 1.073 0.289 0.594 1.149 0.292 0.606 SDXL (2.6B) Vanilla 1.037 0.286 0.600 1.162 0.289 0.617 1.228 0.290 0.634 (w/ DDPM 100 steps) SMC 1.037 0.286 0.600 1.191 0.288 0.623 1.264 0.290 0.631 BoNG 1.059 0.287 0.597 1.194 0.289 0.618 1.259 0.290 0.648 Figure 4: Reward–time trade-off. The reported time denotes the average time required to generate four images. The cross marker on each curve corresponds to the performance reported in Table 2. Table 2: Average performance on GenEval prompts with SD v1.5 backbone. Metrics are averaged over the 4 generated images per prompt, and cost is measured for generating 4 images (5 for Best-4-of-5) on a single A100 GPU. Bold entries indicate the best results. Method Performance (↑)( ) Cost (↓)( ) IR CLIP GenEval Time (s) Mem. (G) Vanilla -0.001 0.271 0.426 7.07 8.90 Vanilla (250 steps) 0.003 0.272 0.430 17.42 8.90 Vanilla Best-4-of-5 0.180 0.275 0.453 8.83 9.97 UG Bansal et al. (2024) 0.326 0.262 0.355 58.36 28.16 DATE Na et al. (2025) 0.364 0.274 0.438 32.89 24.71 LiDAR Kim et al. (2026) 0.384 0.278 0.478 13.41 8.90 BoNG 0.506 0.280 0.511 8.06 8.90 5.1 Single-Output Performance As shown in Table 1, BoNG consistently outperforms baselines across different backbone models with varying sample budget N. Over 36 comparisons, BoNG achieves the highest score in 29 cases, ranking first in 80.56% of all comparisons against baselines. Even with 2 particles, BoNG shows steep performance gains over baselines and further shows superior performance across N=2,3,4N=2,3,4. In particular, BoNG with N=4N=4 outperforms Vanilla BoN with N=8N=8 in GenEval (see Figure 1). This advantage scales to larger budgets where BoNG continues to outperform SMC and Vanilla BoN (see Section 5.2). Overall, these results demonstrate that even with small N budget, BoNG outputs a high reward sample compared to Vanilla BoN and SMC. BoNG is also applicable to modern flow-matching backbones, retaining its advantage on FLUX.1-dev (12B) Labs (2024) (Section 5.2). Why does BoNG outperform SMC? SMC also steers the generative process by incorporating reward information by interaction of particles, similar to BoNG. The main difference is that SMC relies on resampling, whereas BoNG does not. In SMC, even if denoising starts from N independent particles, repeated resampling based on intermediate rewards can collapse the particle population onto only a few trajectories, or even a single one. Such particle collapse reduces exploration and degrades the quality of the final Best-of-N sample. In contrast, BoNG preserves all particles and instead performs Best-of-N selection in the predicted clean-sample space at each intermediate timestep. The selected high-reward target is then used as a guidance signal so that the particle population is steered to high-reward regions without particle copy. As a result, BoNG maintains a better exploration-exploitation tradeoff than SMC, which leads to stronger final BoN performance under the same particle budget. Moreover, since guidance and resampling act on different components of the particle system, the two are composable: applying BoNG on top of SMC improves both the mean and the BoN reward at every particle count (Section F.4). 5.2 Multi-Output Performance As shown in Table 2, BoNG achieves the best performance across all three metrics, demonstrating that its benefit is not limited to improving only the final selected sample (BoN) but also for overall samples. More importantly, these gains are obtained with very small test-time overhead. Compared with prior guidance methods, BoNG shows the strongest average performance while remaining close to vanilla diffusion sampling in inference time. Why does BoNG outperform prior guidance methods? These results are consistent with Theorem 4.3, which shows that BoNG can also be interpreted as a sample-based guidance method targeting the reward-tilted distribution Eq. 3. Its strong empirical performance comes from combining this connection with the core strength of Best-of-N sampling: selecting a single high-reward output from N generated candidates. Rather than applying Best-of-N sampling only after generation, BoNG injects the same principle directly into reverse diffusion by repeatedly using the current Best-of-N target to steer the denoising particle population toward higher-reward regions. At the same time, unlike prior sample-based guidance methods, BoNG does not require additional forward or backward rollouts to construct guidance samples. Instead, it directly reuses the base scores and one-step denoised samples of particles already present in the denoising process. As a result, BoNG inherits the strong best-sample behavior of Best-of-N sampling while avoiding the computational overhead of prior guidance methods, leading to a substantially better efficiency performance tradeoff of the overall quality of multiple generated samples. Figure 5: Reward-Diversity trade-off of BoNG and SMC. (a) Average (b) BoN Figure 6: Average and BoN scores over four particles under different reward mixing weights (IR : CLIP). Table 3: Guidance with HPSv2 Wu et al. (2023) as the reward model (SD v1.5, DDPM 100 steps, N=4N=4). Mean (↑)( ) BoN (↑)( ) Method HPSv2 IR CLIP GenEval HPSv2 IR CLIP GenEval Vanilla 0.263 −0.018-0.018 0.272 0.427 0.287 0.488 0.282 0.517 BoNG 0.276 0.316 0.277 0.487 0.288 0.495 0.282 0.514 Figure 7: BoN performance across large sample budget N (SD v1.5, DDPM 100 steps). Table 4: Comparison with LiDAR under matched end-to-end wall-clock time (SD v1.5, DDPM 100 steps, δ=5δ=5 for LiDAR; BoN performance). Bold entries indicate the best results. Performance (↑)( ) Cost (↓)( ) Method IR CLIP GenEval Time (s) LiDAR (N=2N=2, n=50n=50) 0.734 0.284 0.515 10.59 LiDAR (N=3N=3, n=25n=25) 0.760 0.284 0.515 8.92 LiDAR (N=4N=4, n=9n=9) 0.866 0.286 0.549 8.24 BoNG (N=4N=4) 0.894 0.286 0.555 8.06 Table 5: BoN performance on FLUX.1-dev (12B) Labs (2024) with the flow-matching Euler 28-step sampler. N=2N=2 N=3N=3 N=4N=4 Method IR CLIP GenEval IR CLIP GenEval IR CLIP GenEval Vanilla 1.220 0.285 0.671 1.303 0.287 0.687 1.353 0.287 0.696 BoNG 1.252 0.285 0.680 1.341 0.287 0.696 1.370 0.286 0.700 5.3 Analysis Reward-Diversity Trade-off against SMC. Similar to BoNG, SMC can also be adapted to the multi-output setting. SMC does it by tuning the reward-weighting parameter λ that determines particle potentials, whereas BoNG controls the guidance time window in Section 4.3. Section 5.2 shows that BoNG achieves a consistently better reward-diversity Pareto frontier than SMC. SMC incorporates reward information through resampling, which necessarily duplicates some particles and removes others, while BoNG incorporates reward information through guidance and thus modifies particle trajectories more gradually. Consequently, BoNG preserves diversity more effectively while maintaining strong reward improvement, giving it a clear advantage in the multi-output setting. Efficiency–Performance comparison among gradient guidance methods. The guidance baselines in Table 2 incur substantial additional computation: UG and DATE require backward passes through the score and reward models, and LiDAR requires additional lookahead rollouts with score-network evaluations (see Section D.2 for the full cost breakdown). BoNG avoids both sources of overhead, yet achieves better average performance than LiDAR. We attribute this to the construction of the guidance signal: LiDAR relies on random lookahead samples, whereas BoNG steers all particles toward the current highest-reward denoising particle. Table 2 extends the efficiency–performance Pareto points of Table 2, showing how performance changes as more computation is used; BoNG achieves the steepest performance gain relative to the amount of computation. Performance with different reward functions. Section 5.2 shows results when BoNG is guided by CLIP, ImageReward (IR), or their weighted combination. To account for the different reward scales, we Z-score normalize each reward before applying the mixing weights. As the CLIP weight increases, the generated samples achieve higher CLIP scores, whereas increasing the IR weight improves ImageReward scores. This trend shows that BoNG can effectively leverage other reward functions as guidance signals. Furthermore, combining IR and CLIP yields a clear trade-off curve between the two metrics. The mixed rewards produce frontier points, indicating that reward mixing can improve the balance across complementary evaluation metrics. These results suggest that BoNG provides a reward-agnostic guidance mechanism that can operate with different reward functions and can further benefit from combining multiple rewards. We further test HPSv2 (Wu et al., 2023) as the guidance reward in Section 5.2, where BoNG improves both the mean and the BoN scores over Vanilla, confirming that the guidance mechanism transfers to a different human-preference reward. Results with a heavier reward model are provided in Section F.3. Budget-matched comparison with LiDAR. Table 1 compares methods whose end-to-end cost stays close to that of Vanilla BoN sampling. LiDAR (Kim et al., 2026), a representative sample-based guidance method, can also serve the single-output setting, but its cost is higher: independently of N, it must first generate n lookahead samples in advance, each requiring δ score-network evaluations, and its performance is governed by n (see Section D.2 for the full cost breakdown). We therefore compare BoNG against LiDAR under matched end-to-end wall-clock time, trading off N against n for LiDAR. As shown in Section 5.2, BoNG achieves the best results on every metric with the shortest generation time. This is because BoNG constructs its guidance signal from the N particles currently being sampled, whereas LiDAR relies on separately generated lookahead samples. Additional experiments are provided in Section F.2. Scalability of BoNG. BoNG scales with both the sample budget and the backbone. Section 5.2 shows that on SD v1.5 (DDPM 100 steps), BoNG consistently outperforms SMC and Vanilla BoN for N∈8,16,32N∈\8,16,32\, and its BoN performance keeps improving with N, inheriting the scaling behavior of Best-of-N sampling. BoNG also benefits from a stronger generative process: in Table 1, its performance increases as the backbone grows from SD v1.5 (0.9B) to SDXL (2.6B) and as the sampler becomes more accurate (DDIM 50 to DDPM 100 steps). This trend extends to flow-matching backbones, to which BoNG applies directly since it requires only the one-step denoised estimate in Eq. 5. On FLUX.1-dev (12B) (Labs, 2024), Section 5.2 shows that BoNG continues to improve over Vanilla BoN at every N, confirming that its gains carry over to a larger and stronger backbone. 6 Conclusion We propose BoNG, a test-time alignment method that integrates Best-of-N sampling directly into the reverse diffusion process of diffusion models. By performing online Best-of-N selection among denoising particles, BoNG steers particles to high reward regions during generation rather than using rewards only for post-hoc selection. We further show that BoNG is theoretically connected to sample-based guidance for reward-tilted diffusion sampling, while avoiding the costly rollout procedure required by prior methods. Empirically, BoNG improves both single-output (BoN) performance and average multi-output quality, achieving strong alignment gains with minimal additional inference cost. These results demonstrate that BoNG is an efficient and flexible framework for test-time diffusion alignment. Limitations and Future Work. BoNG assumes that the reward model faithfully reflects the target preference; an inaccurate reward model can steer the particles toward unwanted regions, a limitation shared by all reward-based alignment methods. As future work, the finite-λ relaxation in Appendix E, which replaces the hard Best-of-N selection with a soft-weighted posterior mean, opens a promising direction: exploring this relaxation in greater depth, for example by scaling λ to control the strength of alignment, would be a meaningful extension of BoNG. Acknowledgments and Disclosure of Funding This work was supported by the IITP (Institute of Information & Communications Technology Planning & Evaluation)-ITRC (Information Technology Research Center) grant funded by the Korea government (Ministry of Science and ICT) (IITP-2026-RS-2024-00437268, 50%), and by AI Technology Development for Commonsense Extraction, Reasoning, and Inference from Heterogeneous Data (IITP) funded by the Ministry of Science and ICT (RS-2022-I220077, 50%). References Anderson (1982) B. D. Anderson Reverse-time diffusion equation models. Stochastic Processes and their Applications 12 (3), p. 313–326. Cited by: §3.1. Bansal et al. (2024) A. Bansal, H. Chu, A. Schwarzschild, R. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein Universal guidance for diffusion models. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §B.3, §C.2, §C.3, §1, §2, §3.2.2, §4.3, §4, Table 2, §5. Black et al. (2024) K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1. Chung et al. (2024) H. Chung, J. Kim, G. Y. Park, H. Nam, and J. C. Ye Cfg++: manifold-constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2406.08070. Cited by: §C.3, §4.3. Domingo-Enrich et al. (2025) C. Domingo-Enrich, M. Drozdzal, B. Karrer, and R. T. Q. Chen Adjoint matching: fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2, §3.2.2. Ghosh et al. (2023) D. Ghosh, H. Hajishirzi, and L. Schmidt Geneval: an object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems 36, p. 52132–52152. Cited by: §B.4, §C.3, §5. Hessel et al. (2021) J. Hessel, A. Holtzman, M. Forbes, R. Le Bras, and Y. Choi CLIPScore: a reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 7514–7528. Cited by: §B.4, Figure 8, Figure 8, §5. Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel Denoising diffusion probabilistic models. Advances in neural information processing systems 33, p. 6840–6851. Cited by: §1, §3.1, §3.1. Ho and Salimans (2021) J. Ho and T. Salimans Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, Cited by: §B.1, §C.3, §4.3. Holderrieth et al. (2026) P. Holderrieth, U. Singer, T. Jaakkola, R. T. Q. Chen, Y. Lipman, and B. Karrer GLASS flows: efficient inference for reward alignment of flow and diffusion models. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §3.2.2, §3.2.2. Ichihara et al. (2025) Y. Ichihara, Y. Jinnai, T. Morimura, K. Ariu, K. Abe, M. Sakamoto, and E. Uchibe Evaluation of best-of-n sampling strategies for language model alignment. External Links: 2502.12668, Link Cited by: §3.2.1. Kim et al. (2023) D. Kim, Y. Kim, S. J. Kwon, W. Kang, and I. Moon Refining generative process with discriminator guidance in score-based diffusion models. In Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, p. 16567–16598. Cited by: §1, §2. Kim et al. (2026) Y. Kim, D. Shin, B. Na, M. Park, R. L. Kim, and I. Moon Lookahead sample reward guidance for test-time scaling of diffusion models. In Forty-third International Conference on Machine Learning, Cited by: §B.3, §C.3, §1, §1, §2, §3.2.2, §3.2.2, §3.2.2, §4.2, §4.2, §5.3, Table 2, §5. Labs (2024) B. F. Labs FLUX. Note: https://github.com/black-forest-labs/flux Cited by: §B.1, §1, §3.2, §5.1, §5.2, §5.3. Li et al. (2025) X. Li, Y. Zhao, C. Wang, G. Scalia, G. Eraslan, S. Nair, T. Biancalani, S. Ji, A. Regev, S. Levine, and M. Uehara Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §C.2, §1, §2, §4.3. Ma et al. (2025) Y. Ma, Y. Shui, X. Wu, K. Sun, and H. Li HPSv3: towards wide-spectrum human preference score. External Links: 2508.03789, Link Cited by: §F.3, §1. Microsoft Team (2026) Microsoft Team Note: https://w.bing.com/images/create Cited by: §1, §3.2.1. Na et al. (2025) B. Na, M. Park, G. Sim, D. Shin, H. Bae, M. Kang, S. J. Kwon, W. Kang, and I. Moon Diffusion adaptive text embedding for text-to-image diffusion models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §B.3, §C.2, §1, §2, §3.2.2, §4.3, §4, Table 2, §5. Podell et al. (2024) D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach SDXL: improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §B.1, §1, §3.2, Table 1. Potaptchik et al. (2026a) P. Potaptchik, C. Lee, and M. S. Albergo Tilt matching for scalable sampling and fine-tuning. In Forty-third International Conference on Machine Learning, Cited by: §1, §2, §3.2.2, §4.2, §4.2. Potaptchik et al. (2026b) P. Potaptchik, A. Saravanan, A. Mammadov, A. Prat, M. S. Albergo, and Y. W. Teh Meta flow maps enable scalable reward alignment. In Forty-third International Conference on Machine Learning, Cited by: §1, §2. Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10684–10695. Cited by: §B.1, §3.2, Table 1. Sadat et al. (2024) S. Sadat, O. Hilliges, and R. M. Weber Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In The Thirteenth International Conference on Learning Representations, Cited by: §C.3. Singhal et al. (2025) R. Singhal, Z. Horvitz, R. Teehan, M. Ren, Z. Yu, K. McKeown, and R. Ranganath A general framework for inference-time scaling and steering of diffusion models. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §B.2, §B.3, §C.1, §F.4, §F.4, §1, §1, §2, §3.2.2, §4.3, §4, §5. Skreta et al. (2025) M. Skreta, T. Akhound-Sadegh, V. Ohanesian, R. Bondesan, A. Aspuru-Guzik, A. Doucet, R. Brekelmans, A. Tong, and K. Neklyudov Feynman-kac correctors in diffusion: annealing, guidance, and product of experts. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §1, §2. Snell et al. (2024) C. Snell, J. Lee, K. Xu, and A. Kumar Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314. Cited by: §1. Sohl-Dickstein et al. (2015) J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, p. 2256–2265. Cited by: §1, §3.1. Song and Ermon (2019) Y. Song and S. Ermon Generative modeling by estimating gradients of the data distribution. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, p. 11918–11930. Cited by: §3.1. Song et al. (2021) Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, Cited by: §1, §3.1, §3.1. Swaminathan and Joachims (2015) A. Swaminathan and T. Joachims The self-normalized estimator for counterfactual learning. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28, p. . External Links: Link Cited by: §4.2. Uehara et al. (2024) M. Uehara, Y. Zhao, K. Black, E. Hajiramezanali, G. Scalia, N. L. Diamant, A. M. Tseng, T. Biancalani, and S. Levine Fine-tuning of continuous-time diffusion models as entropy-regularized control. arXiv preprint arXiv:2402.15194. Cited by: §2, §3.2.2. Vincent (2011) P. Vincent A connection between score matching and denoising autoencoders. Neural computation 23 (7), p. 1661–1674. Cited by: §3.1. Wallace et al. (2024) B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8228–8238. Cited by: §1. Wu et al. (2023) X. Wu, Y. Hao, K. Sun, Y. Chen, F. Zhu, R. Zhao, and H. Li Human preference score v2: a solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341. Cited by: §F.3, §1, §5.2, §5.3. xAI Team (2026) xAI Team Note: https://grok.com/imagineAccessed: 2026-01 Cited by: §1, §3.2.1. Xu et al. (2023) J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong Imagereward: learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems 36, p. 15903–15935. Cited by: §B.3, §B.4, Figure 8, Figure 8, §1, §5. Appendix A Proofs and Derivations A.1 Derivation of Lemma 4.1 See 4.1 Proof. We have a forward Gaussian Kernel and its Stein score as: pt(t|0)=1σt2πexp(−(t−0)22σt2),∇tlogpt(t|0)=−t−0σt2.p_t(x_t|x_0)= 1 _t 2π \! (- (x_t-x_0)^22 _t^2 ), _x_t p_t(x_t|x_0)=- x_t-x_0 _t^2. With Bayes’ rule, we can rewrite expected future reward as follows: pθ(0∣t,)[exp(λr(0,))] _p_θ(x_0 x_t,c) [ (λ r(x_0,c)) ] =∫pθ(0|t,)exp(λr(0,))d0 = p_θ(x_0|x_t,c) (λ r(x_0,c))dx_0 (12) =∫pθ(0∣)pt(t∣0)∫pθ(0∣)pt(t∣0)d0exp(λr(0,))d0 = p_θ(x_0 c)\,p_t(x_t x_0) p_θ(x_0 c)\,p_t(x_t x_0)\,dx_0 (λ r(x_0,c))dx_0 (13) =∫pθ(0∣)pt(t∣0)exp(λr(0,))d0∫pθ(0∣)pt(t∣0)d0 = p_θ(x_0 c)\,p_t(x_t x_0) (λ r(x_0,c))dx_0 p_θ(x_0 c)\,p_t(x_t x_0)\,dx_0 (14) =zr(t)z0(t), = z_r(x_t)z_0(x_t), (15) where zr(t) z_r(x_t) :=∫pθ(0∣)pt(t∣0)exp(λr(0,))d0 := p_θ(x_0 c)\,p_t(x_t x_0)\, (λ r(x_0,c))\,dx_0 (16) z0(t) z_0(x_t) :=∫pθ(0∣)pt(t∣0)d0=pθ(t|). := p_θ(x_0 c)\,p_t(x_t x_0)\,dx_0=p_θ(x_t|c). (17) The gradient of the expected future reward ∇trtλ(t,) _x_tr_t^λ(x_t,c) can be expressed as ∇trtλ(t,) _x_tr_t^λ(x_t,c) =∇tlogpθ(0∣t,)[exp(λr(0,))] = _x_t _p_θ(x_0 x_t,c) [ (λ r(x_0,c)) ] (18) =∇tlogzr(t)−∇tlogz0(t). = _x_t z_r(x_t)- _x_t z_0(x_t). (19) The first term ∇tlogzr(t) _x_t z_r(x_t) can be expressed as follows: ∇tlogzr(t) _x_t z_r(x_t) =∇tzr(t)zr(t) = _x_tz_r(x_t)z_r(x_t) (20) =∫pθ(0∣)exp(λr(0,))∇tpt(t∣0)zr(t)d0 = p_θ(x_0 c) (λ r(x_0,c)) _x_tp_t(x_t x_0)z_r(x_t)dx_0 (21) =∫pθ(0∣)exp(λr(0,))pt(t∣0)∇tpt(t∣0)pt(t∣0)zr(t)d0 = p_θ(x_0 c) (λ r(x_0,c))p_t(x_t x_0) _x_tp_t(x_t x_0)p_t(x_t x_0)z_r(x_t)dx_0 (22) =∫pθ(0∣)pt(t∣0)z0(t)z0(t)zr(t)exp(λr(0,))∇tlogpt(t∣0)d0 = p_θ(x_0 c)p_t(x_t x_0)z_0(x_t) z_0(x_t)z_r(x_t) (λ r(x_0,c)) _x_t p_t(x_t x_0)dx_0 (23) =∫pθ(0∣t,)exp(λr(0,))pθ(0∣t,)[exp(λr(0,))]⏟:=pθr(0∣t,)∇tlogpt(t∣0)d0 = p_θ(x_0 x_t,c) (λ r(x_0,c))E_p_θ(x_0 x_t,c) [ (λ r(x_0,c)) ]_:=p^r_θ(x_0 x_t,c) _x_t p_t(x_t x_0)dx_0 (24) =pθr(0∣t,)[∇tlogpt(t∣0)]. =E_p^r_θ(x_0 x_t,c) [ _x_t p_t(x_t x_0) ]. (25) We can rewrite the second term ∇tlogz0(t) _x_t z_0(x_t) as follows: ∇tlogz0(t) _x_t z_0(x_t) =∇tz0(t)z0(t) = _x_tz_0(x_t)z_0(x_t) (26) =∫pθ(0∣)∇tpt(t∣0)z0(t)d0 = p_θ(x_0 c) _x_tp_t(x_t x_0)z_0(x_t)dx_0 (27) =∫pθ(0∣)pt(t∣0)∇tpt(t∣0)pt(t∣0)z0(t)d0 = p_θ(x_0 c)p_t(x_t x_0) _x_tp_t(x_t x_0)p_t(x_t x_0)z_0(x_t)dx_0 (28) =∫pθ(0∣)pt(t∣0)pθ(t|)∇tlogpt(t∣0)d0 = p_θ(x_0 c)p_t(x_t x_0)p_θ(x_t|c) _x_t p_t(x_t x_0)dx_0 (29) =pθ(0|t,)[∇tlogpt(t∣0)]. =E_p_θ(x_0|x_t,c) [ _x_t p_t(x_t x_0) ]. (30) Substituting the two identities above into Eq. 19 and simplifying yields ∇trtλ(t,) _x_tr_t^λ(x_t,c) =pθr(0|t,)[∇tlogpt(t∣0)]−pθ(0|t,)[∇tlogpt(t∣0)] =E_p^r_θ(x_0|x_t,c) [ _x_t p_t(x_t x_0) ]-E_p_θ(x_0|x_t,c) [ _x_t p_t(x_t x_0) ] (31) =pθr(0|t,)[−t−0σt2]−pθ(0|t,)[−t−0σt2] =E_p^r_θ(x_0|x_t,c) [- x_t-x_0 _t^2 ]-E_p_θ(x_0|x_t,c) [- x_t-x_0 _t^2 ] (32) =−1σt2(t−pθr(0|t,)[0])+1σt2(t−pθ(0|t,)[0]) =- 1 _t^2 (x_t-E_p^r_θ(x_0|x_t,c)[x_0] )+ 1 _t^2 (x_t-E_p_θ(x_0|x_t,c)[x_0] ) (33) =1σt2(pθr(0|t,)[0]−pθ(0|t,)[0]) = 1 _t^2 (E_p^r_θ(x_0|x_t,c)[x_0]-E_p_θ(x_0|x_t,c)[x_0] ) (34) =1σt2(mλr(t)−pθ(0|t,)[0]). = 1 _t^2 (m_λ^r(x_t)-E_p_θ(x_0|x_t,c)[x_0] ). (35) This completes the proof. ∎ A.2 Derivation of Lemma 4.2 See 4.2 Proof. We derive the self-normalized importance sampling representation of the reward-tilted posterior mean. Specifically, we obtain mλr(t) m_λ^r(x_t) =pθr(0∣t,)[0]=∫pθ(0∣t,)exp(λr(0,))0d0pθ(0∣t,)[exp(λr(0,))] =E_p_θ^r(x_0 x_t,c)[x_0]= p_θ(x_0 x_t,c) (λ r(x_0,c))x_0dx_0E_p_θ(x_0 x_t,c) [ (λ r(x_0,c)) ] (36) =pθ(0∣t,)[0exp(λr(0,))]pθ(0∣t,)[exp(λr(0,))]. = E_p_θ(x_0 x_t,c) [x_0 \! (λ r(x_0,c) ) ]E_p_θ(x_0 x_t,c) [ \! (λ r(x_0,c) ) ]. (37) Let q(0)q(x_0) be any proposal distribution whose support contains that of pθ(0∣t,)p_θ(x_0 x_t,c). By change of measure, both the numerator and denominator in Eq. 37 can be rewritten under q(0)q(x_0), yielding mλr(t)=q(0)[pθ(0∣t,)q(0)0exp(λr(0,))]q(0)[pθ(0∣t,)q(0)exp(λr(0,))]. m_λ^r(x_t)= E_q(x_0) [ p_θ(x_0 x_t,c)q(x_0)x_0 \! (λ r(x_0,c) ) ]E_q(x_0) [ p_θ(x_0 x_t,c)q(x_0) \! (λ r(x_0,c) ) ]. (38) Now let ^01,…,^0N∼i.i.d.q(0) x_0^1,…, x_0^N i.i.d. q(x_0). Applying Monte Carlo approximation to both expectations in Eq. 38, we obtain mλr(t) m_λ^r(x_t) ≈1N∑i=1Npθ(^0i∣t,)q(^0i)^0iexp(λr(^0i,))1N∑i=1Npθ(^0i∣t,)q(^0i)exp(λr(^0i,)) ≈ 1N _i=1^N p_θ( x_0^i x_t,c)q( x_0^i) x_0^i \! (λ r( x_0^i,c) ) 1N _i=1^N p_θ( x_0^i x_t,c)q( x_0^i) \! (λ r( x_0^i,c) ) (39) =∑i=1Npθ(^0i∣t,)q(^0i)^0iexp(λr(^0i,))∑i=1Npθ(^0i∣t,)q(^0i)exp(λr(^0i,)). = _i=1^N p_θ( x_0^i x_t,c)q( x_0^i) x_0^i \! (λ r( x_0^i,c) ) _i=1^N p_θ( x_0^i x_t,c)q( x_0^i) \! (λ r( x_0^i,c) ). (40) Define the unnormalized importance weights wi(λ):=pθ(^0i∣t,)q(^0i)exp(λr(^0i,)). w_i^(λ):= p_θ( x_0^i x_t,c)q( x_0^i) \! (λ r( x_0^i,c) ). (41) Then Eq. 40 can be written as mλr(t)≈∑i=1Nwi(λ)∑j=1Nwj(λ)^0i. m_λ^r(x_t)≈ _i=1^N w_i^(λ) _j=1^Nw_j^(λ) x_0^i. (42) Therefore, if we define αi(λ):=wi(λ)∑j=1Nwj(λ), _i^(λ):= w_i^(λ) _j=1^Nw_j^(λ), (43) we obtain the self-normalized importance sampling estimator m^N,λr(t):=∑i=1Nαi(λ)^0i. m_N,λ^r(x_t):= _i=1^N _i^(λ) x_0^i. (44) Finally, taking logarithms of the unnormalized weights gives logwi(λ)=logpθ(^0i∣t,)−logq(^0i)+λr(^0i,). w_i^(λ)= p_θ( x_0^i x_t,c)- q( x_0^i)+λ r( x_0^i,c). (45) Hence the normalized weights admit the softmax form αi(λ)=softmaxi(logpθ(^0i∣t,)−logq(^0i)+λr(^0i,)), _i^(λ)=softmax_i\! ( p_θ( x_0^i x_t,c)- q( x_0^i)+λ r( x_0^i,c) ), (46) which proves the result. ∎ A.3 Derivation of Theorem 4.3 See 4.3 Proof. By Eq. 9, the target process score in Eq. 4 depends on the reward-tilted posterior mean mλr(t)m_λ^r(x_t). By Lemma 4.2, this posterior mean admits the self-normalized importance sampling approximation mλr(t)≈m^N,λr(t)=∑i=1Nαi(λ)^0i,m_λ^r(x_t)≈ m_N,λ^r(x_t)= _i=1^N _i^(λ) x_0^i, where αi(λ)=softmaxi(logpθ(^0i∣t,)−logq(^0i)+λr(^0i,)). _i^(λ)=softmax_i\! ( p_θ( x_0^i x_t,c)- q( x_0^i)+λ r( x_0^i,c) ). Let i∗=argmax1≤i≤Nr(^0i,),^0BoN:=^0i∗,i^*= _1≤ i≤ Nr( x_0^i,c), x_0^BoN:= x_0^i^*, which is unique by assumption. Then for any i≠i∗i≠ i^*, αi(λ)αi∗(λ) _i^(λ) _i^*^(λ) =exp(logpθ(^0i∣t,)−logq(^0i)+λr(^0i,)) = ( p_θ( x_0^i x_t,c)- q( x_0^i)+λ r( x_0^i,c) ) ×exp(−logpθ(^0i∗∣t,)+logq(^0i∗)−λr(^0i∗,)) × (- p_θ( x_0^i^* x_t,c)+ q( x_0^i^*)-λ r( x_0^i^*,c) ) =Ciexp(λ(r(^0i,)−r(^0i∗,))), =C_i (λ (r( x_0^i,c)-r( x_0^i^*,c) ) ), (47) where Ci:=exp(logpθ(^0i∣t,)−logq(^0i)−logpθ(^0i∗∣t,)+logq(^0i∗))C_i:= ( p_θ( x_0^i x_t,c)- q( x_0^i)- p_θ( x_0^i^* x_t,c)+ q( x_0^i^*) ) is independent of λ. Since i∗i^* is the unique maximizer, we have r(^0i,)−r(^0i∗,)<0for all i≠i∗.r( x_0^i,c)-r( x_0^i^*,c)<0 all i≠ i^*. Therefore, αi(λ)αi∗(λ)→λ→∞0for all i≠i∗. _i^(λ) _i^*^(λ) [λ→∞]0 all i≠ i^*. Since ∑i=1Nαi(λ)=1 _i=1^N _i^(λ)=1, it follows that αi(λ)→λ→∞0(i≠i∗),αi∗(λ)→λ→∞1. _i^(λ) [λ→∞]0 (i≠ i^*), _i^*^(λ) [λ→∞]1. Hence, m^N,λr(t)=∑i=1Nαi(λ)^0i→λ→∞^0i∗=^0BoN. m_N,λ^r(x_t)= _i=1^N _i^(λ) x_0^i [λ→∞] x_0^i^*= x_0^BoN. (48) Now evaluate at a query particle t=tix_t=x_t^i. By the proposal construction in Eqs. 5 and 6, pθ(0∣ti,)[0]=^0i,^0BoN=^0i∗.E_p_θ(x_0 x_t^i,c)[x_0]= x_0^i, x_0^BoN= x_0^i^*. Substituting the SNIS approximation into Eq. 9 evaluated at tix_t^i, we obtain ∇tirtλ(ti,)≈1σt2(m^N,λr(ti)−^0i). _x_t^ir_t^λ(x_t^i,c)≈ 1 _t^2 ( m_N,λ^r(x_t^i)- x_0^i ). (49) By Eq. 48, the right-hand side converges as λ→∞λ→∞ to 1σt2(^0BoN−^0i)=∇tirtBoN(ti,), 1 _t^2 ( x_0^BoN- x_0^i )= _x_t^ir_t^BoN(x_t^i,c), which is the BoN guidance term in Eq. 7. This establishes Eq. 11 and proves the theorem. ∎ Appendix B Experimental Settings B.1 Experiment Setup We use SD v1.5 [22] and SDXL [19] backbones with CFG [9] scale 7.5 for all experiments. For each backbone, we experiment with the DDPM 100-step sampler and the DDIM 50-step sampler (probability-flow ODE) in Table 1, the latter being substantially faster. We additionally evaluate on the flow-matching backbone FLUX.1-dev [14], using the flow-matching Euler sampler with 28 steps and guidance scale 3.5. All results are averaged over three random seeds, except FLUX.1-dev, which uses a single seed due to computational cost. B.2 BoNG Configuration BoNG has three hyperparameters: the window position tstartt_start, the window size K, and the guidance scale s. The selection protocol and the corresponding sensitivity analyses are given in Appendices C and D; here we record the values used in the main experiments. For a compute budget comparable to the SMC baseline [24], which performs five resampling steps, we set the number of guided transitions to K≤4K≤ 4 in all configurations of Table 1. A guidance window [tstart,tend][t_start,t_end] denotes the set of guided transitions t→t+1t→ t+1 for tstart≤t<tendt_start≤ t<t_end; e.g., [40,44][40,44] denotes the four transitions 40→41,…,43→4440→ 41,…,43→ 44 (K=4K=4). The guidance windows are [40,44][40,44] for SD v1.5 with DDPM 100 steps (where step 40 corresponds to t=0.6t=0.6, with t=1t=1 the prior and t=0t=0 the data), [21,23][21,23] for SD v1.5 with DDIM 50 steps, [24,28][24,28] for SDXL with DDPM 100 steps, [20,22][20,22] for SDXL with DDIM 50 steps, [12,14][12,14] for FLUX with Euler 28 steps. The guidance scale is s=6s=6 for SD v1.5, s=4s=4 for SDXL, s=2s=2 for FLUX. Table 2 uses the same configuration as Table 1. B.3 Baseline Configuration We apply the shared settings of Section B.1 to all baselines. In Table 1, Vanilla runs standard diffusion sampling N times and selects the best sample according to ImageReward [36]. The baseline denoted SMC is FK-Steering [24] under its recommended text-to-image configuration: max potential, λ=10λ=10, five evenly spaced resampling steps, adaptive resampling, and intermediate rewards on denoised estimates. Best-k-of-N involves no guidance: N=5N=5 samples are generated with the base model, and the top k=4k=4 samples by reward are returned. For Table 2, we follow the standard practice of quoting the reported numbers for LiDAR [13], UG [2], and DATE [18] from the concurrent work LiDAR, which achieves state-of-the-art performance in the multi-output setting. Please refer to Sections B.1 and B.2 of LiDAR for the corresponding configurations. For transparency, we additionally reproduce all baselines of Table 2 in our unified codebase under the same configurations and report the results in Section F.1. B.4 Evaluation Metrics We report the metrics used throughout our experiments. ImageReward [36]: a learned human-preference reward model for text-to-image generation. It assigns a scalar preference score to each generated image conditioned on the text prompt, and we use it to evaluate overall image quality and prompt satisfaction from a human-centric perspective. CLIP Score [7]: measures the semantic alignment between the text prompt and the generated image by computing the similarity between their CLIP embeddings. It provides a reference-free estimate of prompt-image correspondence, although it may be less sensitive to fine-grained compositional errors. GenEval [6]: an object-centric evaluation benchmark for text-to-image generation. It uses detection-based checks to assess whether generated images correctly satisfy prompt-specified entities, attributes, counts, and spatial relations, making it useful for measuring compositional accuracy. We measure the diversity of generated samples by computing pairwise distances in the CLIP image-embedding space, following the evaluation protocol of FK-Steering.11 1 https://github.com/zacharyhorvitz/D-Diffusion-Steering/blob/main/text_to_image/fkd_diffusers/rewards.py Appendix C Hyperparameter Selection Protocol of BoNG BoNG has three hyperparameters: the window position tstartt_start, the window size K, and the guidance scale s. For a new backbone or sampler, we set them by the following protocol: • Window position tstartt_start: place the window where the correlation between the Tweedie-estimate reward r(^0)r( x_0) and the final reward becomes reliable (≈0.6≈ 0.6). • Window size K: choose K by the compute budget, at or below the resampling budget of the SMC baseline. • Guidance scale s: tune s∈[1,6]s∈[1,6] to select the desired point on the reward–diversity frontier, at no extra compute. This procedure was followed for every configuration in the paper. The remainder of this section justifies each rule with controlled experiments: Section C.1 shows why the window position trades off the accuracy of the guidance signal against the steerability of the samples, Section C.2 shows that the window size moves gradually and monotonically along the reward–diversity frontier, and Section C.3 shows that the guidance scale sweeps the same frontier at zero additional cost. C.1 Window Position A meaningful guidance signal requires the one-step denoised estimate ^0BoN x_0^BoN to be aligned with the final reward, so we place the window near the point where the correlation between r(^0)r( x_0) and the final reward exceeds 0.60.6 (cf. Figure 5 of FK-Steering [24]). Placing the window too late is ineffective for the opposite reason: by then the high-level features of each sample are already determined, so particles committed to low-reward regions can no longer be steered. Table 6: Ablation over the window position tstartt_start (SD v1.5, DDPM 100 steps, N=4N=4, K=4K=4, s=6s=6; BoN performance). Window IR (↑)( ) CLIP (↑)( ) GenEval (↑)( ) Vanilla 0.713 0.284 0.532 [20,24][20,24] 0.753 0.282 0.523 [40,44][40,44] 0.875 0.286 0.559 [60,64][60,64] 0.849 0.287 0.526 [80,84][80,84] 0.866 0.287 0.528 Table 6 reports an ablation over tstartt_start with all other factors fixed (SD v1.5, DDPM 100 steps, N=4N=4, K=4K=4, s=6s=6, single seed; BoN performance). The metric-wise breakdown reveals why the optimum sits mid-range: window placement trades off the accuracy of the guidance signal against the steerability of the samples. An early window ([20,24][20,24]) fails on both metrics because the Tweedie-reward correlation is still low. In a late window ([80,84][80,84]), the correlation is at its highest and ImageReward improves accordingly; however, the high-level compositional content that GenEval measures is already determined by that stage, so accurate guidance can refine samples but can no longer re-steer particles committed to compositionally weaker configurations, and GenEval does not improve over Vanilla. The mid-range window [40,44][40,44], where the correlation first becomes reliable while the high-level content can still be shaped, is the only regime that improves all metrics simultaneously, which is precisely what the window-position rule above is designed to find. C.2 Window Size The window size K determines the additional cost of BoNG, which scales linearly as NK(D+R)NK(D+R) (see Section D.2). Enlarging K applies the reward signal over more transitions and correspondingly reduces the influence of the base diffusion process, concentrating the particles in high-reward regions; in this sense, K plays a role analogous to the reward-strength hyperparameter λ in prior methods [2, 18, 15]. Table 7 reports a controlled scan over K (SD v1.5, DDPM 100 steps, s=1s=1, tstart=40t_start=40). Both effects are gradual and monotone: reward increases and diversity decreases as K grows. At the main-experiment setting K=4K=4, the reward improves over Vanilla sampling with no loss of diversity (0.319 vs. 0.313), indicating that the main experiments operate in a regime free of particle collapse. To keep the compute budget below that of the SMC baseline, we set K=4K=4 for 100-step sampling and K=2K=2 for 50-step sampling. Table 7: Controlled scan over the window size K (SD v1.5, DDPM 100 steps, N=4N=4, s=1s=1, tstart=40t_start=40). Window K mean IR (↑)( ) Diversity (↑)( ) Vanilla – −0.001-0.001 0.313 [40,44][40,44] 4 0.019 0.319 [40,50][40,50] 10 0.252 0.316 [40,55][40,55] 15 0.465 0.262 [40,60][40,60] 20 0.576 0.216 [40,65][40,65] 25 0.641 0.176 [40,70][40,70] 30 0.678 0.146 [40,75][40,75] 35 0.707 0.121 C.3 Guidance Scale CFG [9] improves image fidelity without increasing inference time or the number of function evaluations (NFEs) by introducing a guidance scale ω∈ℝω with ω≥1ω≥ 1 into the conditional score function, as defined in Eq. 50. In the same spirit, following prior guidance methods that expose an analogous scale [2, 13], we introduce a guidance scale s∈ℝs with s≥1s≥ 1 on the BoN guidance term, which controls how strongly each non-BoN particle (tix_t^i with i≠i∗i≠ i^*) is steered, as described in Eq. 51. ∇tlogpθω(t|)=∇tlogpθ(t)+ω×pθ(0|t,)[0]−pθ(0|t)[0]σt2 _x_t p_θ^ω(x_t|c)= _x_t p_θ(x_t)+ω× E_p_θ(x_0|x_t,c)[x_0]-E_p_θ(x_0|x_t)[x_0] _t^2 (50) ∇tlogpθBoN(t|)=∇tlogpθ(t|)+s×pθ(0|tBoN,)[0]−pθ(0|t,)[0]σt2 _x_t p_θ^BoN(x_t|c)= _x_t p_θ(x_t|c)+s× E_p_θ(x_0|x_t^BoN,c)[x_0]-E_p_θ(x_0|x_t,c)[x_0] _t^2 (51) Within the protocol, the scale s complements K: since it only rescales the guidance term, it sweeps the same reward–diversity frontier at zero additional compute. Table 8 reports a scan at the fixed main-experiment window [40,44][40,44] (K=4K=4): increasing s raises the mean reward while diversity decreases gradually, so s selects the operating point on the frontier once tstartt_start and K are fixed. Table 8: Scan over the guidance scale s (SD v1.5, DDPM 100 steps, N=4N=4, window [40,44][40,44], K=4K=4). s mean IR (↑)( ) Diversity (↑)( ) Vanilla −0.001-0.001 0.313 1 0.019 0.319 2 0.224 0.315 3 0.490 0.255 4 0.473 0.245 6 0.506 0.224 We further compare the sensitivity of BoNG to s against UG and LiDAR, which expose the same hyperparameter, on GenEval [6] test prompts. Figure 8 reports the ImageReward–CLIP score trade-off as s varies, and we make three observations. First, in the range s∈[1,7.5]s∈[1,7.5], UG exhibits reward-hacking behavior: as s increases, ImageReward improves while the CLIP score decreases. In contrast, LiDAR and BoNG achieve more favorable trade-offs across the two metrics. Second, BoNG achieves a stronger ImageReward–CLIP score trade-off than LiDAR. Finally, when s=10s=10, we observe a degradation of BoNG in CLIP score due to issues such as off-manifold steering and color saturation, consistent with the findings of CFG++ [4] and APG [23]; this motivates the protocol range s∈[1,6]s∈[1,6]. Qualitative results are shown in Figure 9, where we present generations for five randomly selected prompts as s varies from 11 to 1010. Figure 8: ImageReward [36] and CLIP score [7] trade-offs between UG, LiDAR, and BoNG. Across all methods, we report results under guidance scale s∈1,2.5,5,7.5,10s∈\1,2.5,5,7.5,10\. Guidance scale Figure 9: Changes in Best-of-N samples of BoNG as the guidance scale s∈1,2,3,4,5,6,7,8,9,10s∈\1,2,3,4,5,6,7,8,9,10\ varies. The prompts from the top row to the bottom row are “a photo of a pizza”, “a photo of a blue elephant”, “a photo of a cow left of a stop sign”, “a photo of a baseball glove right of a bear”, and “a photo of a yellow car and an orange toothbrush”. Appendix D Additional Analysis of BoNG D.1 Effect of Guidance Timestep Window The guidance timestep window jointly determines reward optimization, diversity preservation, and inference-time overhead. As discussed in Sections 4.3 and 5.3, the window plays the role of λ in the reward-tilted distribution: applying BoNG over more timesteps yields stronger reward optimization, but may reduce sample diversity by making the generation process closer to sampling from a highly reward-tilted distribution. At the same time, the window size sets the additional cost NK(D+R)NK(D+R) of BoNG (Section D.2). Section 5.2 shows the resulting reward–diversity trade-off, Table 7 confirms that both effects are gradual and monotone, and Table 2 shows that BoNG achieves the best performance–efficiency trade-off among the compared methods; the selection rules for the window are given in Appendix C. D.2 Computational Cost BoNG introduces no extra score network evaluations over Vanilla diffusion sampling. Its additional cost comes only from constructing the BoN target at guided timesteps, which requires a VAE decoder pass and a reward model pass for each denoising particle. For N particles and K guided timesteps, letting D and R denote the cost of one VAE decoder and reward model forward pass, respectively, the additional function evaluations are NK(D+R)NK(D+R). Table 9 compares each method’s additional cost over Vanilla sampling in hardware-independent units. SMC and BoNG add no additional score-network evaluations; their overhead comes solely from scoring denoised estimates, NS(D+R)NS(D+R) and NK(D+R)NK(D+R), respectively. UG and DATE evaluate the decoder and reward at every step, NT(D+R)NT(D+R), and additionally require backward passes through the score network, decoder, and reward model to obtain reward gradients. LiDAR generates lookahead samples: each of the n lookaheads costs δ score-network evaluations plus one decoder and one reward pass, adding nδ+n(D+R)nδ+n(D+R) in total. Since K≤S≪TK≤ S T in all our experiments (K=4K=4, S=5S=5), BoNG incurs the smallest additional cost among the steering methods, with no gradient computation. Table 9: Additional cost over Vanilla sampling in hardware-independent units. D: VAE-decoder pass, R: reward pass; T: diffusion steps; S: resampling steps of SMC; n, δ: lookahead samples and score evaluations per lookahead of LiDAR. Method Score-net VAE decoder Reward Backward SMC – N⋅SN· S N⋅SN· S – UG / DATE – N⋅TN· T N⋅TN· T required LiDAR n⋅δn·δ n n – BoNG – N⋅KN· K N⋅KN· K – To make the comparisons concrete, Table 10 reports the wall-clock breakdown of one guided denoising step on SD v1.5 (N=16N=16, mean of 5 runs). Table 10: Wall-clock breakdown of one guided denoising step (SD v1.5, N=16N=16, mean of 5 runs). Component Time (ms) Per particle (ms) Time % TFLOPs TFLOPs / particle UNet fwd + CFG 238.88 14.93 20.6% 25.705 1.607 VAE decode 522.98 32.69 45.1% 40.232 2.515 Postprocess (tensor→ ) 167.94 10.50 14.5% 0.000 0.000 Reward (ImageReward) 230.81 14.43 19.9% 2.205 0.138 Regarding memory, we report peak GPU memory during generation in GiB in Table 2. All methods share an identical configuration (backbone, resolution, N=4N=4, GPU), so the differences do not stem from image size or encoding. Notably, sample count alone does not predict the measurements: UG and DATE store no extra samples yet require roughly 3×3× the memory of Vanilla, while LiDAR stores n=50n=50 lookahead latents yet matches Vanilla’s footprint. We instead characterize the measurements through the memory decomposition itself. With the same networks resident across methods, peak memory is approximately M≈Pscore+Pdec+Preward+N⋅AM≈ P_score+P_dec+P_reward+N· A, where only the activation term A differs: gradient guidance (UG, DATE) retains the activation graph for backpropagation (AbwdA_bwd), whereas Vanilla, LiDAR, and BoNG discard activations layer by layer (Afwd≪AbwdA_fwd A_bwd). BoNG retains no graph and allocates nothing beyond the N particles of Vanilla sampling, so its memory footprint equals Vanilla’s by construction, independently of hardware. D.3 Anchor-Switching Ablation To test whether a single particle dominates the guidance signal, we tracked the identity of the Best-of-N anchor tBoNx_t^BoN at every guided transition (window [40,44][40,44]: three consecutive transitions per prompt) across all 553 GenEval prompts (total of 1,659 transitions) across 3 runs. We report the mean, median, and standard deviation of the switch rate of the 3 runs. Switch rate 1.01.0 implies that for every transition tBoNx_t^BoN was changed, and 00 implies that tBoNx_t^BoN is not changed at all. The experiment shows that the anchor is dynamically re-elected from the population rather than fixed: no single trajectory dominates the guidance target, and the guidance signal genuinely emerges from particle interaction. Table 11: Anchor-switching statistics over 553 GenEval prompts (1,659 transitions), 3 runs. Anchor switching (553 prompts, 1,659 transitions) Value Mean 0.2656 Median 0.2737 Std 0.0161 Appendix E Finite-λ Soft Relaxation of BoNG BoNG applies hard Best-of-N selection over the one-step denoised candidates and therefore corresponds to the large-λ limit described in Theorem 4.3. However, Lemma 4.2 itself holds for every finite λ>0λ>0, and the same rollout-free one-step Tweedie construction admits a natural finite-λ relaxation, which we derive and evaluate in this section. We emphasize that this relaxation is a variant beyond the hard-selection algorithm evaluated in the main experiments; all results in the main text correspond to the hard-selection BoNG. Derivation. Choosing the proposal distribution as q(0)=pθ(0∣)q(x_0)=p_θ(x_0 c), the SNIS weights of Lemma 4.2 for a query particle tix_t^i can be written, using Bayes’ rule, as softmaxj(logpt(ti∣0j)+λr(0j,)), _j ( p_t (x_t^i x_0^j )+λ\,r (x_0^j,c ) ), (52) where 0j∼pθ(0∣)x_0^j p_θ(x_0 c) and pt(t∣0)p_t(x_t x_0) is the tractable Gaussian forward kernel; the finite-λ estimator is thus computable in principle when exact proposal samples are available. Obtaining such samples, however, requires full reverse-process rollouts. To construct a rollout-free relaxation, we replace the exact samples with the one-step Tweedie estimates already available from the current particle population, ^0j=pθ(0∣tj,)[0] x_0^j=E_p_θ(x_0 x_t^j,c)[x_0], which gives the soft weights α~ij(λ)=softmaxj(logpt(ti∣^0j)+λr(^0j,)), α_ij^(λ)=softmax_j ( p_t (x_t^i x_0^j )+λ\,r ( x_0^j,c ) ), (53) and the corresponding particle-specific relaxed guidance target m^N,λr(ti)=∑j=1Nα~ij(λ)^0j. m^r_N,λ(x_t^i)= _j=1^N α_ij^(λ)\, x_0^j. (54) Rather than selecting only the highest-reward candidate, this target softly combines the candidate set according to both the forward compatibility with the query particle and the reward. Smaller values of λ distribute the guidance across multiple candidates, while larger values increasingly concentrate the weights on the highest-reward candidate; assuming a unique reward maximizer, the hard-selection BoNG update is recovered as λ→∞λ→∞, consistent with Theorem 4.3. Approximation gaps. This construction remains approximate in two respects. First, the one-step Tweedie estimates are deterministic posterior means rather than exact proposal samples from pθ(0∣)p_θ(x_0 c). Second, once particle interaction has been applied, the guided particles need not exactly follow the original marginal pθ(t∣)p_θ(x_t c). Consequently, the exact importance-sampling interpretation of Lemma 4.2 does not directly transfer to this rollout-free relaxation: BoNG admits a rollout-free finite-λ relaxation, rather than exactly targeting the finite-λ reward-tilted distribution. Empirical verification of the predicted scaling. The finite-λ analysis predicts that λ traces the reward–diversity trade-off, with hard-selection BoNG recovered at the large-λ end of the spectrum. To verify this, we implement the soft-weighted relaxation, replacing hard Best-of-N selection with the softmax weights α~ij(λ) α_ij^(λ) over the one-step denoised candidates, with the same guidance window as the main experiments (SD v1.5, DDPM 100 steps). Table 12 reports the mean ImageReward and diversity as λ varies. Increasing λ raises the mean reward while diversity decreases gradually, which is precisely the behavior the finite-λ analysis predicts. These results suggest a soft-weighted family of BoNG variants beyond the hard-selection algorithm evaluated in this paper, which we view as a natural direction for future work opened up by our framework. Table 12: Soft-weighted finite-λ relaxation of BoNG under varying λ (N=4N=4). Backbone Method Mean IR (↑)( ) Diversity (↑)( ) SD v1.5 (0.9B) (w/ DDPM 100 steps) Vanilla −0.001-0.001 0.313 λ=100λ=100 0.001 0.313 λ=500λ=500 0.004 0.311 λ=1000λ=1000 0.131 0.300 λ=5000λ=5000 0.564 0.234 λ=10000λ=10000 0.595 0.214 Appendix F Additional Experiment Results This section supplements the main results along both evaluation axes. For the multi-output setting of Table 2, Section F.1 provides a unified-codebase reproduction with the SMC entry and diversity added, and Section F.4 shows that BoNG composes with SMC. For the single-output setting of Table 1, Section F.2 extends the matched wall-clock comparison against LiDAR in Section 5.3 to a large sample budget, and Section F.3 tests a heavier guidance reward beyond the HPSv2 result in Section 5.2. F.1 Unified-Codebase Reproduction of Table 2 For transparency, we reproduce all baselines of Table 2 in our unified codebase, following the LiDAR paper’s configurations for the UG and DATE baselines, and additionally report the SMC entry and the diversity of each method. Table 13 reports the results. The relative ordering of Table 2 is preserved under reproduction. SMC attains a higher mean reward than BoNG in this table, but at a substantially lower diversity (0.155 vs. 0.224), since resampling duplicates high-scoring ancestors; this makes it unsuitable for the multi-output setting that Table 2 targets, where N distinct outputs must be served. Table 13: Unified-codebase reproduction of Table 2 with the SMC entry and diversity added. Method IR (↑)( ) CLIP (↑)( ) GenEval (↑)( ) Diversity (↑)( ) Time (s) (↓)( ) Mem. (GiB) (↓)( ) Vanilla −0.001-0.001 0.271 0.426 0.313 7.07 8.9 UG 0.271 0.261 0.364 0.461 57.89 28.2 DATE 0.382 0.275 0.450 0.327 32.68 24.7 LiDAR (δ=5δ=5, n=50n=50) 0.394 0.278 0.473 0.280 13.57 8.9 SMC 0.641 0.283 0.517 0.155 8.23 8.9 BoNG 0.506 0.280 0.511 0.224 8.06 8.9 F.2 Comparison with LiDAR under Matched Wall-Clock Budgets Section 5.2 in Section 5.3 compares BoNG with LiDAR under matched end-to-end wall-clock time at N=4N=4. Here we extend the comparison to a large sample budget. At N=32N=32, we set LiDAR to n=100n=100 and δ=5δ=5 so that its additional cost nδ+n(D+R)nδ+n(D+R) consumes similar time as the NK(D+R)NK(D+R) cost of BoNG (see Section D.2), and compare the average performance over the N samples in Table 14. BoNG outperforms LiDAR on every metric with a shorter end-to-end time. This is because LiDAR’s performance gain is independent of N and relies on n, whereas BoNG inherits the scaling behavior of Best-of-N sampling as N increases. Table 14: Budget-matched comparison with LiDAR at N=32N=32. Bold entries indicate the best results. Performance (↑)( ) Cost (↓)( ) Backbone Method IR CLIP GenEval Time (s) SD v1.5 (0.9B) (w/ DDPM 100 steps) Vanilla −0.012-0.012 0.272 0.425 49.5 LiDAR (n=100n=100, δ=5δ=5) 0.405 0.281 0.478 61.4 BoNG 0.854 0.285 0.538 56.1 F.3 Performance with a Heavier Reward Model Section 5.2 in Section 5.3 shows that BoNG transfers to HPSv2 [34] as the guidance reward. To further test whether the guidance mechanism holds under a heavier reward backbone, we run Vanilla, SMC, and BoNG with HPSv3 [16] as the guidance reward (N=4N=4, SD v1.5, DDPM 100 steps) and report the BoN performance in Table 15. BoNG improves the HPSv3 reward over Vanilla, confirming that the guidance mechanism transfers to a heavier reward backbone, while SMC attains a slightly higher HPSv3 score in this setting. We also observe that ImageReward decreases under HPSv3 guidance for both SMC and BoNG; we attribute this to HPSv3 using a non-CLIP encoder, so that the two reward models emphasize different features. Table 15: Guidance with HPSv3 as the reward model (N=4N=4; BoN performance). Bold entries indicate the best results. Backbone Method HPSv3 (↑)( ) IR (↑)( ) CLIP (↑)( ) GenEval (↑)( ) SD v1.5 (0.9B) (w/ DDPM 100 steps) Vanilla 7.287 0.440 0.281 0.520 SMC 7.418 0.439 0.280 0.520 BoNG 7.306 0.399 0.279 0.512 F.4 Composing BoNG with SMC Inference-time steering methods fall into two distinct classes [24]: reward-guidance methods, which steer particles by modifying the sampling dynamics, and particle-based resampling methods, which steer by reweighting and resampling intermediate particles with potentials. BoNG shares a structural connection with SMC, as both maintain N interacting particles and evaluate intermediate rewards on denoised estimates, but the two belong to different classes. BoNG is a reward-guidance method, in which the reward signal enters through the guidance term sσt2(^0BoN−^0i) s _t^2 ( x_0^BoN- x_0^i ) added to the score function, whereas SMC is a potential-based resampling method, in which the reward signal enters through the resampling weights GtG_t while the transition kernel is left unchanged. Concretely, while the outer loop of BoNG resembles that of SMC (propagate N particles, score their denoised estimates with the reward), SMC applies multinomial resampling based on GtG_t, so particles are copied and deleted, whereas BoNG applies no resampling and instead modifies the score function. In this sense, BoNG brings the strength of the guidance class, with no particle copying or deletion, to the black-box reward setting. This distinction has a direct practical consequence: since guidance and resampling act on different components of the particle system (the transition kernel and the resampling weights, respectively), they are orthogonal and composable. To verify this empirically, we apply BoNG on top of SMC and report ImageReward with N particles in Table 16, for both the mean over the N returned samples and the Best-of-N sample. Composing BoNG with SMC improves both the mean and the Best-of-N reward at every particle count, with no reward gradients and no additional score-network evaluations. This empirically confirms that BoNG occupies the proposal-generator slot of the FK-IPS framework [24] and complements resampling-based steering. Table 16: Composing BoNG with SMC. Bold entries indicate the best results. Mean IR (↑)( ) BoN IR (↑)( ) Backbone Particle Count SMC +BoNG SMC +BoNG SD v1.5 (0.9B) (w/ DDPM 100 steps) N=2N=2 −0.039-0.039 0.276 0.377 0.417 N=3N=3 0.463 0.549 0.679 0.684 N=4N=4 0.641 0.706 0.823 0.833 Appendix G Qualitative Comparison Figure 10 shows the Best-of-N samples of Vanilla, SMC, and BoNG for various prompts under the configurations of Table 1. BoNG produces samples that are better aligned with the prompts across backbones and samplers. Figure 10: Qualitative comparison of Best-of-N samples from Vanilla, SMC, and BoNG. Appendix H Pseudocode of BoNG Algorithm 3 presents the full sampling procedure of BoNG, combining the Best-of-N selection of Algorithm 1 and the guided update of Algorithm 2 into a single loop over the reverse diffusion process. Algorithm 3 BoNG: full sampling procedure 1: Input: prompt c, number of particles N, diffusion steps T, guidance scale s, guidance window [tstart,tend][t_start,t_end], reward r, pretrained noise network ϵθ _θ 2: Initialize Ti∼(,)x_T^i (0,I) for i=1,…,Ni=1,…,N 3: for t=T,…,1t=T,…,1 do 4: ϵi←ϵθ(ti,t,) ε^i← _θ(x_t^i,t,c) for i=1,…,Ni=1,…,N 5: if tstart≤t<tendt_start≤ t<t_end then 6: ^0i←(ti−σtϵi)/α¯t x_0^i← (x_t^i- _t ε^i )/ α_t for i=1,…,Ni=1,…,N ⊳ Alg. 1 7: Ri←r(Decode(^0i),)R^i← r (Decode( x_0^i),c ) for i=1,…,Ni=1,…,N 8: i∗←argmaxiRii^*← _iR^i, ^0BoN←^0i∗ x_0^BoN← x_0^i^* 9: ϵi←ϵi−α¯ts(^0BoN−^0i)/σt ε^i← ε^i- α_t\,s\, ( x_0^BoN- x_0^i )/ _t for i=1,…,Ni=1,…,N ⊳ Alg. 2 10: end if 11: t−1i←SchedulerStep(ϵi,ti,t)x_t-1^i ( ε^i,x_t^i,t) for i=1,…,Ni=1,…,N 12: end for 13: Output: 0ii=1N\x_0^i\_i=1^N