Paper deep dive
FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling
Shivanshu Shekhar, Sagnik Mukherjee, Jia Yi Zhang, Tong Zhang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/10/2026, 3:43:07 AM
Summary
Fleming-Viot Diffusion (FVD) is an inference-time alignment method for diffusion models that replaces traditional Sequential Monte Carlo (SMC) multinomial resampling with a birth-death mechanism. This approach mitigates diversity collapse and lineage degeneracy by decoupling selection from replication, allowing for efficient, parallelizable reward-tilted sampling without requiring model retraining or expensive value function approximations.
Entities (5)
Relation Signals (3)
Fleming-Viot Diffusion → replaces → multinomial resampling
confidence 95% · FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment.
Fleming-Viot Diffusion → improves → FK-Diffusion
confidence 90% · FVD substantially improves the reward-diversity tradeoff... resolves the collapse observed in FK-Diffusion.
Fleming-Viot Diffusion → targets → reward-tilted distribution
confidence 90% · FVD yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomial resampling or closely related resampling schemes, which can still reduce diversity and lead to lineage collapse under strong selection pressure. Inspired by Fleming-Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment. To handle cases where rewards are only approximately available and naive rebirth would collapse deterministic trajectories, FVD integrates independent reward-based survival decisions with stochastic rebirth noise. This yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions, all without requiring value function approximation or costly rollouts. FVD is fully parallelizable and scales efficiently with inference compute. Empirically, it achieves substantial gains across settings: on DrawBench it outperforms prior methods by 7% in ImageReward, while on class-conditional tasks it improves FID by roughly 14-20% over strong baselines and is up to 66 times faster than value-based approaches.
Tags
Links
- Source: https://arxiv.org/abs/2604.06779v1
- Canonical: https://arxiv.org/abs/2604.06779v1
Trouble viewing inline? Open PDF directly →
Full Text
71,283 characters extracted from source content.
Expand or collapse full text
FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling Shivanshu Shekhar1 Sagnik Mukherjee1 Jia Yi Zhang2 Tong Zhang1 1 Siebel School of Computing and Data Science 2 Department of Statistics University of Illinois Urbana-Champaign shekhar6, sagnikm3, jyzhang5, tozhang@illinois.edu Abstract We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomial resampling or closely related resampling schemes, which can still reduce diversity and lead to lineage collapse under strong selection pressure. Inspired by Fleming–Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment. To handle cases where rewards are only approximately available and naive rebirth would collapse deterministic trajectories, FVD integrates independent reward-based survival decisions with stochastic rebirth noise. This yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions—all without requiring value function approximation or costly rollouts. FVD is fully parallelizable and scales efficiently with inference compute. Empirically, it achieves substantial gains across settings: on DrawBench it outperforms prior methods by 7% in ImageReward, while on class-conditional tasks it improves FID by roughly 14–20% over strong baselines and is up to 66× faster than value-based approaches. 1 Introduction Diffusion models [15, 31] have become a dominant paradigm for generative modelling, achieving state-of-the-art performance across modalities including images [15, 22, 24], video [39, 13], and language [2, 26]. In many practical settings, however, generation must satisfy objectives beyond reproducing the training distribution. A common requirement is reward alignment, where generated samples should remain on the learned data manifold while maximizing a reward function r:→ℝr:X that captures human preferences, task objectives, or domain-specific constraints. Existing approaches to reward alignment broadly fall into two categories. RL-based fine-tuning These approaches formulate alignment as a reinforcement learning objective applied to a pretrained generative model: ℒRLHF=τ∼πθ[R(τ)]−βKL(πθ∥πθref),L_RLHF=E_τ _θ [R(τ) ]-β\,KL( _θ\| _ _ref), (1) where πθref _ _ref is a reference model and the KL term regularizes deviations from the pretrained distribution. Several works instantiate this framework using policy gradient methods [3, 12], direct preference optimization [38, 34], or direct reward optimization [6]. While effective, these methods require expensive fine-tuning of the diffusion model and must be repeated whenever the reward function changes. Inference-time alignment An alternative direction avoids retraining by modifying the sampling procedure of a frozen diffusion model to target the distribution π∗(x)∝pθ(x)exp(λr(x)),π^*(x) p_θ(x) (λ r(x)), (2) where pθp_θ denotes the pretrained diffusion model and λ≥0λ≥ 0 controls the strength of reward alignment. These methods trade additional inference-time computation for flexibility, enabling alignment with new rewards without retraining. Several families of inference-time methods have recently been proposed. Gradient-based approaches [1, 5] bias the denoising trajectory using reward gradients, but require differentiable reward models and can destabilize the denoising process by pulling trajectories off the data manifold. SMC methods [4, 10, 28] maintain a population of particles and resample according to estimated rewards, offering a principled probabilistic framework but suffering from diversity collapse under aggressive resampling. Search-based methods [20, 19] perform local reward-guided exploration but do not scale naturally with compute. Value-function methods [17] learn a value function via Monte Carlo rollouts and sample greedily, but the rollouts are computationally expensive and difficult to batch, making them impractical at scale. A key failure mode of SMC-based methods is diversity collapse: as optimization pressure increases, the final population descends from only a small number of ancestral particles, producing over-optimized samples that deviate from the prior. We demonstrate this concretely in Section 6 for FK-Diffusion (Strongest SMC baseline) [28], where multinomial resampling aggressively prunes trajectories early in the denoising chain. FVD addresses this by replacing multinomial resampling with a Fleming–Viot-style birth–death mechanism tailored to diffusion alignment. Adapting Fleming–Viot population control to diffusion denoising requires handling two diffusion-specific challenges: rewards are only approximately available at intermediate timesteps, and naive rebirth would collapse deterministic DDIM trajectories. By combining independent reward-based survival decisions with stochastic rebirth, FVD yields a softer, variance-reducing population dynamics that preserves trajectory diversity throughout denoising, aggregates posterior mass more broadly, requires no learned approximations or expensive rollouts, and remains fully parallelizable. We show in Section 6 that this resolves the collapse observed in FK-Diffusion (FKD), and in Section 5 that it yields consistent gains across both prompt-conditioned, class-conditioned and prompt-free reward settings. Our main contributions are: 1. Fleming–Viot population control for diffusion alignment. We replace multinomial resampling in particle-based diffusion alignment with a Fleming–Viot-style birth–death mechanism tailored to diffusion denoising, where rewards are only approximately available at intermediate timesteps and naive rebirth would collapse deterministic trajectories. By combining independent reward-based survival decisions, uniform donor selection, and stochastic rebirth noise, FVD reduces offspring variance, mitigates lineage collapse, and improves sample quality over FKD. 2. Adaptive control of alignment strength. We show that the fraction of removed particles αt=ndead/K _t=n_dead/K is monotonic in λ, enabling a Robbins–Monro update λ←λ−ηk(αt−α∗)λ←λ- _k( _t-α^*) that automatically adjusts selection pressure during sampling. This replaces manual tuning of λ with a simple and interpretable target absorption rate α∗α^*. 3. Empirical validation of the resampling bottleneck. Through lineage analysis, reward-ranked removal statistics, and comparisons across class-conditional posterior sampling and text-to-image alignment, we show that FVD substantially improves the reward-diversity tradeoff while retaining the parallel efficiency of particle-based inference. 2 Related Work Inference-time Alignment. Inference-time alignment steers frozen pretrained diffusion models towards target distributions without modifying model parameters. These methods treat the pretrained model as a fixed prior and guide the sampling trajectory to favor outputs with higher reward. Inference-time alignment is useful when retraining is computationally infeasible or when new objectives arise post-training. Existing approaches fall into the following categories: gradient-based, particle-based, search-based, and value-based methods. Gradient-based Guidance. A common alignment technique is to modify the reverse diffusion update using gradients of a reward. Gradient-based methods perturb the mean of the denoising distribution using the gradient of the value or score function, steering model towards the desired objective [1, 5, 14, 32]. This is closely related to classifier guidance [9] and has been applied to incorporate semantic constraints, reward models, and preference signals during sampling. Variants of these methods improve gradient estimates using Monte Carlo sampling or other approximations [30]. While effective for differentiable rewards, gradient guidance is inapplicable to non-differentiable or discrete-state settings and introduces per-step gradient cost that scales with model size. Particle-based Methods. Particle-based methods [36, 10, 28, 33] propagate multiple candidate trajectories (particles) simultaneously through the diffusion process and resamples them at intermediate steps according to potential functions that approximate the soft value. Sequential Monte Carlo (SMC) enables reward-guided generation with frozen model weights. SMC is theoretically guaranteed to recover the target distribution given exact potentials and infinite particles [8]. The core bottleneck for sample quality in particle-based diffusion alignment is that the repeated resampling step can dramatically reduce diversity, leading to particle degeneracy when a few high-weight trajectories dominate the population. Search-based Methods. Recent work explores search-based strategies that generate multiple candidate denoising transitions and select those with higher reward estimates [20, 19] . Some methods perform greedy local search, while others treat inference as a search problem over noise or trajectories using random, zero-order, or path-based refinement with verifier feedback [21]. More advanced approaches employ tree-based exploration [41, 42, 16, 40], to expand and evaluate multiple trajectories. While these methods can improve sample quality, they often rely on shallow look-ahead or heuristic selection and may overfit to verifier biases when search becomes too aggressive [21]. Value Function-based Methods. Value Function-based methods estimate the soft value function at intermediate states and use it to bias the sampling process. Diffusion Tree Sampling (DTS) [17] casts the reverse process as a finite-horizon tree and applies soft-value Monte Carlo Tree Search (MCTS), backing up terminal rewards via the soft Bellman equation to refine value estimates across rollouts . This yields asymptotically exact samples and strong compute efficiency relative to baselines. However, the sequential tree-building loop limits parallelism: at matched NFE budgets DTS is much slower than fully parallel particle methods [17]. Our Perspective. Our work builds on particle-based inference-time alignment, viewing diffusion sampling through a sequential Monte Carlo lens. Unlike gradient-based methods, it does not require differentiable rewards or incur per-step gradient costs. In contrast to standard SMC approaches, which suffer from diversity collapse due to multinomial resampling, we use a Fleming–Viot birth–death process that decouples selection from replication and preserves diversity. Compared to search-based methods, we avoid explicit trajectory expansion, and unlike value-based methods, we do not learn value functions, eliminating additional modeling overhead while scaling efficiently with compute. 3 Preliminaries Diffusion Models. A diffusion model defines a forward noising process q(xt∣x0)=(xt;α¯tx0,(1−α¯t)I),q(x_t x_0)=N\! (x_t;\, α_t\,x_0,\,(1- α_t)I ), where α¯t=∏s=1tαs α_t= _s=1^t _s. A neural network ϵθ(xt,t) _θ(x_t,t) is trained to predict the injected noise, implicitly learning the score of the data distribution. Given a noisy sample xtx_t, the Tweedie estimate of the clean sample is x^0(xt,t)=xt−1−α¯tϵθ(xt,t)α¯t, x_0(x_t,t)= x_t- 1- α_t\, _θ(x_t,t) α_t, (3) which corresponds to the posterior mean [x0∣xt]E[x_0 x_t] [29]. Sampling is typically performed using DDIM [29]. We write DDIMη(xt,t)DDIM_η(x_t,t) for the DDIM update from xtx_t to xt−1x_t-1 using stochasticity parameter η∈[0,1]η∈[0,1], with η=0η=0 denoting the deterministic update and η>0η>0 injecting scheduler-scaled Gaussian noise. When the stochasticity parameter η=0η=0, DDIM produces a deterministic trajectory xT→xT−1→⋯→x0x_T→ x_T-1→·s→ x_0 with update xt−1=α¯t−1x^0(xt,t)+1−α¯t−1ϵθ(xt,t).x_t-1= α_t-1\, x_0(x_t,t)+ 1- α_t-1\, _θ(x_t,t). (4) In this deterministic setting the only randomness arises from the initial noise xT∼(0,I)x_T (0,I). Reward-Tilted Target Distribution. Let r:→ℝr:X denote a reward function. Inference-time alignment aims to sample from the reward-tilted distribution p∗(x0)∝pθ(x0)exp(λr(x0)),p^*(x_0) p_θ(x_0) (λ r(x_0)), (5) where pθp_θ is the distribution induced by the pretrained diffusion model and λ≥0λ≥ 0 controls the strength of alignment. Since intermediate states xtx_t are highly noisy—especially at early timesteps—the true reward r(x0)r(x_0) is not directly accessible during sampling. In practice, methods rely on proxy evaluations using the Tweedie estimate x^0(xt,t) x_0(x_t,t) or approximate the reward via partial or full rollouts to the final sample. Particle-Based Inference. Particle methods approximate complex target distributions using a population of interacting samples. A set of K particles evolves through alternating selection and mutation steps: selection reweights trajectories according to a potential function that favours high-quality samples, while mutation propagates each particle via the underlying dynamics. A standard formulation is given by the Feynman–Kac framework [8], which defines a sequence of measures πt∗t=0T\π^*_t\_t=0^T as πt∗(xt:T)∝πT(xT)∏s=tT−1Gs(xs:T)Ms(xs∣xs+1),π^*_t(x_t:T) _T(x_T) _s=t^T-1G_s(x_s:T)\,M_s(x_s x_s+1), (6) where Gs:T+1−s→ℝ+G_s:X^T+1-s _+ is a potential function and MsM_s is the mutation kernel. Sequential Monte Carlo (SMC) approximates πt∗π^*_t using K weighted particles (xt(i),wt(i))i=1K\(x_t^(i),w_t^(i))\_i=1^K, with weights updated according to wt(i)∝wt+1(i)Gt(xt:T(i))Mt(xt(i)∣xt+1(i))qt(xt(i)∣xt+1(i)),w_t^(i) w_t+1^(i) G_t(x_t:T^(i))\,M_t(x_t^(i) x_t+1^(i))q_t(x_t^(i) x_t+1^(i)), (7) where qtq_t is the proposal distribution. When the proposal matches the prior transition, i.e. qt=Mtq_t=M_t, the update simplifies to wt(i)∝wt+1(i)Gt(xt:T(i))w_t^(i) w_t+1^(i)G_t(x_t:T^(i)). To mitigate weight degeneracy, usually a resampling step is periodically applied, replacing low-weight particles with copies of high-weight ones. The resulting approximation is captured by the empirical path measure π^tK=K−1∑i=1Kδxt:T(i) π_t^K=K^-1 _i=1^K _x_t:T^(i), where δxt:T(i) _x_t:T^(i) is the Dirac measure at the trajectory xt:T(i)x_t:T^(i). For any test function f on path space, integration with respect to π^tK π_t^K reduces to the particle average K−1∑i=1Kf(xt:T(i))K^-1 _i=1^Kf(x_t:T^(i)), providing a concrete interpretation of the particle system as a distribution. Under mild regularity conditions, π^tK π_t^K converges to πt∗π^*_t as K→∞K→∞ [8]. Fleming–Viot Particle Systems. The Fleming–Viot (FV) process [11, 7] is an interacting particle system originally developed for simulating conditioned stochastic processes. Unlike SMC methods, which rely on importance weights and suffer from weight degeneracy, FV maintains a constant-size population through a birth–death mechanism that avoids explicit weighting. At each step t, each particle i independently undergoes a death event with probability dt(i)=ϕ(Gt(xt:T(i)),Gt(xt:T(k))k=1K)∈[0,1),d_t^(i)=φ\! (G_t(x_t:T^(i)),\,\G_t(x_t:T^(k))\_k=1^K )∈[0,1), (8) where Gt:T+1−t→ℝ+G_t:X^T+1-t _+ is a potential function and ϕφ is a monotone rule that assigns lower death probabilities to higher-potential particles. Each particle independently realizes a binary death outcome Dt(i)∼Bernoulli(dt(i))D_t^(i) (d_t^(i)). When a particle dies, it is immediately reborn by copying a donor particle j sampled uniformly from the survivors, j∼Uniform(k:Dt(k)=0),xt(i)←xt(j).j \! (\k:D_t^(k)=0\ ), x_t^(i)← x_t^(j). (9) This decouples selection and replication: GtG_t influences only survival, while donor selection is weight-free. As a result, FV concentrates mass on high-potential regions without the large offspring-count variance of multinomial resampling. Since deaths are independent Bernoulli trials, offspring variance is O(1)O(1) per particle rather than O(K)O(K), significantly reducing lineage collapse and preserving trajectory diversity. Under mild conditions, the empirical path measure K−1∑i=1Kδxt:T(i)K^-1 _i=1^K _x_t:T^(i) is expected to converge to the Feynman–Kac path measure πt∗π^*_t as K→∞K→∞, consistent with propagation-of-chaos results for related Fleming–Viot particle systems [11, 8]; see Appendix B. We adapt the Fleming–Viot process to diffusion denoising for inference-time alignment in the next section, yielding a stable and diversity-preserving alternative to standard SMC resampling that forms the basis of our method. 4 FV-Diffusion Algorithm 1 FVD: Adaptive Fleming–Viot Diffusion Sampling 0: Frozen model pθp_θ, reward r, particle count K, potential family Gt(⋅;λ)t∈\G_t(·;λ)\_t , schedule T, initial λ0 _0, target α∗α^*, rebirth ηrebirth _rebirth, cap αmax _ , floor δfloor _floor, subsample temperature τ 1: Sample xT(i)∼(0,I)x_T^(i) (0,I) for i=1,…,Ki=1,…,K 2: for t=T,T−1,…,1t=T,T-1,…,1 do 3: xt−1(i)←DDIMη=0(xt(i),t)x_t-1^(i) _η=0(x_t^(i),t) for all i 4: if t∈t then 5: gi←Gt(xt:T(i);λ)g_i← G_t(x_t:T^(i);λ); si←fi(Gt(xt))s_i← f_i(G_t(x_t)); di←[ui>si]d_i 1[u_i>s_i], ui∼U(0,1)u_i U(0,1) 6: Cap: while ∑idi>⌊αmaxK⌋ _id_i> _ K , revive highest-potential dead particle 7: for each dead particle i do 8: j∼Uniform(k:dk=0)j (\k:d_k=0\); xt−1(i)←DDIMηrebirth(xt(j),t)x_t-1^(i) _ _rebirth(x_t^(j),t) 9: end for 10: αt←(∑idi)/K _t←( _id_i)/K 11: if std(loggi)≥δfloorstd(\ g_i\)≥ _floor then 12: λ←clip(λ−ηk(αt−α∗),λmin,λmax)λ (λ- _k( _t-α^*),\, _ ,\, _ ) 13: end if 14: end if 15: end for 16: ri←r(x0(i))r_i← r(x_0^(i)) for all i 17: Draw NevalN_eval images ∝exp(ri/τ) (r_i/τ) from x0(i),rii=1K\x_0^(i),r_i\_i=1^K 18: return Selected images FV-Diffusion (FVD) generates samples by evolving K parallel DDIM trajectories while applying Fleming–Viot resampling at a predefined subset of timesteps ⊂1,…,TT⊂\1,…,T\. At each resampling step, particles are selectively removed and reborn according to positive per-step potentials Gt(xt:T)G_t(x_t:T), concentrating the particle population in high-potential regions while preserving diversity across trajectories. General Per-Step Potential Framework. We define FVD for an arbitrary family of positive per-step potentials Gt:T+1−t→ℝ+,t∈,G_t:X^T+1-t _+, t , and write the corresponding reward-twisted path measure in terms of the cumulative product of these potentials. To ensure the procedure targets the correct terminal distribution, the cumulative potential should satisfy ∏s∈Gs(xs:T)=exp(λr(x0)), _s G_s(x_s:T)= \! (λ\,r(x_0) ), (10) in the sense that the accumulated intermediate estimates recover the terminal reward [28]. In practice, the intermediate product in (10) is only approximate because the Tweedie proxies x^0(xt(i),t) x_0(x_t^(i),t) are not exactly equal to x0x_0. We therefore define a terminal correction potential G0(x0:T(i))=(∏s∈Gs(xs:T(i)))−1⋅exp(λr(x0(i))),G_0(x_0:T^(i))= ( _s G_s(x_s:T^(i)) )^-1· \! (λ\,r(x_0^(i)) ), (11) so that the full product satisfies G0(x0:T)∏s∈Gs(xs:T)=exp(λr(x0)).G_0(x_0:T) _s G_s(x_s:T)= \! (λ\,r(x_0) ). Thus G0G_0 cancels the accumulated intermediate weights and replaces them with the true terminal reward, leaving the target distribution exactly preserved. Potential-Based Survival. At each resampling step, particle i survives with probability si(t)=Gt(xt:T(i))maxjGt(xt:T(j)),s_i^(t)= G_t(x_t:T^(i)) _jG_t(x_t:T^(j)), (12) which ensures si(t)∈(0,1]s_i^(t)∈(0,1] and guarantees that at least one particle survives. Death events are sampled independently: particle i dies if ui∼U(0,1)u_i U(0,1) satisfies ui>si(t)u_i>s_i^(t). This contrasts with the multinomial resampling used in SMC-based methods [28], where all K successors are sampled jointly and offspring counts exhibit O(K)O(K) variance. Independent Bernoulli deaths reduce this variance to O(1)O(1) per particle (Propositions 1 and 2). To prevent excessive particle loss, we enforce ndead≤⌊αmaxK⌋n_dead≤ _ K . If more particles die in a step, the highest-potential dead particles are revived until the cap is satisfied. Each remaining dead particle then selects a donor uniformly from the surviving set donor(i)∼Uniform(j:Dj=0)donor(i) (\j:D_j=0\). The potential GtG_t affects only the survival decision in (12); donor selection is weight-free. As a result the potential is applied exactly once per step and the procedure remains consistent with the intended target distribution. Exponential Reward Instantiation. For the experiments in this paper, we instantiate the per-step potentials using the Tweedie reward proxy with Gt(xt:T)=exp(λ||r(x^0(xt,t))),G_t(x_t:T)= \! ( λ|T|\,r\! ( x_0(x_t,t) ) ), (13) so that each of the |||T| resampling steps contributes an equal share of the total alignment strength. Under this choice, the total alignment strength is simply λ. In Appendix B we provide an informal argument that, in the large-population regime, this procedure asymptotically targets π∗(x0)∝pθ(x0)exp(λr(x0))π^*(x_0) p_θ(x_0) (λ\,r(x_0)). The full generic procedure is summarized in Algorithm 1, and all experiments below use the instantiation (13). Survival Probabilities Under the Experimental Instantiation. Under our instantiation (13), particle i survives at step t∈t with probability si(t)=exp(λ||(ri−rmax)),ri=r(x^0(xt(i),t)),rmax=maxjrj,s_i^(t)= \! ( λ|T|\,(r_i-r_ ) ), r_i=r\! ( x_0(x_t^(i),t) ), r_ = _jr_j, (14) which is exactly the normalized potential rule (12) specialized to (13). Stochastic Rebirth. Because DDIM with η=0η=0 produces deterministic trajectories, directly copying a donor state would cause reborn particles to follow identical paths. To avoid this collapse, reborn particles instead re-run the DDIM update from the donor’s noisy state xt(j)x_t^(j) using a non-zero noise level ηrebirth>0 _rebirth>0: xt−1(i)=α¯t−1x^0(xt(j),t)+1−α¯t−1−σt2ϵθ(xt(j),t)+σtεi,εi∼(0,I),x_t-1^(i)= α_t-1\, x_0(x_t^(j),t)+ 1- α_t-1- _t^2\, _θ(x_t^(j),t)+ _t\, _i, _i (0,I), (15) where σt=ηrebirth(1−α¯t−1)βt/(1−α¯t) _t= _rebirth (1- α_t-1) _t/(1- α_t) follows the scheduler variance. Surviving particles continue their η=0η=0 trajectories unchanged. Adaptive λ via Robbins–Monro [23]. The absorption rate αt=1K∑i=1Kdt(i) _t= 1K _i=1^Kd_t^(i) is the fraction of particles killed at step t, and controls selection pressure: a high αt _t aggressively prunes low-potential particles but risks losing diversity, while a low αt _t is more conservative but may allow poor trajectories to survive. Under (14), the expected absorption fraction is [αt]=1K∑i=1K(1−exp(λ||(ri−rmax))),E[ _t]= 1K _i=1^K (1- \! ( λ|T|(r_i-r_ ) ) ), (16) which is strictly monotone increasing in λ for any reward distribution with positive spread (Proposition 3). Moreover, as the reward spread across particles shrinks over time, the expected absorption rate decreases for fixed λ, so the selection mechanism automatically becomes less aggressive near locally homogeneous populations. This monotonicity need not hold for arbitrary potential families, so the controller is specific to our exponential reward instantiation. In that setting, it allows a Robbins–Monro update targeting a desired absorption rate α∗α^*: λ←clip(λ−ηk(αt−α∗),λmin,λmax),ηk=η01+γk.λ \! (λ- _k( _t-α^*),\; _ ,\; _ ), _k= _01+γ k. (17) The update reduces λ when too many particles are removed and increases it otherwise, making the target absorption rate α∗α^* the primary user-controlled parameter — a direct, reward-scale-independent means of specifying selection pressure. 5 Experiments We present the main empirical results of the paper in this section, evaluating FVD across two settings of increasing complexity. We begin with class-conditional posterior sampling on MNIST and CIFAR-10, and then move to large-scale text-to-image generation using Stable Diffusion as the base model. Unless stated otherwise, all results are averaged over five random seeds. For the exponential potential family in Eq. (13), the alignment parameter used by FVD is simply λ. We set λ=1.0λ=1.0 across all methods for fair comparison; when adaptive updates are enabled, this value denotes the initialization. Dataset → MNIST MNIST even/odd CIFAR-10 Algorithm ↓ FID(↓ ) MMD(↓ ) [logr(x)]E[ r(x)](↑ ) Div(↑ ) FID(↓ ) MMD(↓ ) [logr(x)]E[ r(x)](↑ ) Div(↑ ) FID(↓ ) MMD(↓ ) [logr(x)]E[ r(x)](↑ ) Div(↑ ) DPS 0.9150.915 (4.487)(4.487) 0.3430.343 (0.248)(0.248) −0.160-0.160 (0.132)(0.132) 0.4860.486 (0.059)(0.059) 0.2000.200 (0.376)(0.376) 0.3860.386 (0.430)(0.430) −0.015-0.015 (0.011)(0.011) 0.6020.602 (0.051)(0.051) 0.3430.343 (0.050)(0.050) 1.2241.224 (0.427)(0.427) −0.117-0.117 (0.059)(0.059) 0.5280.528 (0.021)(0.021) FK 0.0340.034 (0.022)(0.022) 0.1480.148 (0.129)(0.129) −0.025-0.025 (0.013)(0.013) 0.4580.458 (0.055)(0.055) 0.0170.017 (0.004)(0.004) 0.0710.071 (0.029)(0.029) −0.005-0.005 (0.002)(0.002) 0.6150.615 (0.056)(0.056) 0.2410.241 (0.068)(0.068) 0.9170.917 (0.359)(0.359) −0.055-0.055 (0.031)(0.031) 0.5140.514 (0.030)(0.030) TDS 0.0920.092 (0.052)(0.052) 0.4280.428 (0.359)(0.359) −0.012-0.012 (0.031)(0.031) 0.4290.429 (0.056)(0.056) 0.1400.140 (0.065)(0.065) 0.6800.680 (0.345)(0.345) −0.001-0.001 (0.001)(0.001) 0.5830.583 (0.067)(0.067) 0.4110.411 (0.181)(0.181) 2.1162.116 (1.384)(1.384) −0.035-0.035 (0.035)(0.035) 0.4820.482 (0.036)(0.036) DAS 0.0260.026 (0.012)(0.012) 0.0970.097 (0.077)(0.077) −0.017-0.017 (0.017)(0.017) 0.4570.457 (0.056)(0.056) 0.0280.028 (0.009)(0.009) 0.1310.131 (0.079)(0.079) −0.005-0.005 (0.003)(0.003) 0.6140.614 (0.062)(0.062) 0.2130.213 (0.060)(0.060) 0.8080.808 (0.256)(0.256) −0.204-0.204 (0.193)(0.193) 0.5270.527 (0.021)(0.021) DTS 0.0190.019 (0.002)(0.002) 0.1000.100 (0.016)(0.016) −0.020-0.020 (0.001)(0.001) 0.4940.494 (0.003)(0.003) 0.0140.014 (0.004)(0.004) 0.0750.075 (0.029)(0.029) −0.015-0.015 (0.004)(0.004) 0.6160.616 (0.053)(0.053) 0.1800.180 (0.036)(0.036) 0.7440.744 (0.183)(0.183) −0.301-0.301 (0.098)(0.098) 0.5400.540 (0.019)(0.019) FVD (ours) 0.0140.014 (0.005)(0.005) 0.0770.077 (0.045)(0.045) −0.012-0.012 (0.003)(0.003) 0.4670.467 (0.054)(0.054) 0.0100.010 (0.003)(0.003) 0.0510.051 (0.020)(0.020) −0.004-0.004 (0.001)(0.001) 0.6150.615 (0.060)(0.060) 0.1440.144 (0.031)(0.031) 0.6670.667 (0.233)(0.233) −0.283-0.283 (0.414)(0.414) 0.5290.529 (0.022)(0.022) Table 1: Quantitative comparison on MNIST and CIFAR-10. We report mean ± standard deviation over 5 seeds after 10610^6 NFEs. Metrics include FID, MMD, expected reward, and diversity (Div). Best values per column are underlined, and values within 5% of the best are highlighted. FVD consistently achieves the best or near-best performance across all settings. Posterior Sampling under Class Conditioning. Figure 1: Qualitative comparison on CIFAR-10 (class: car). Samples generated with 10610^6 NFEs using reward-weighted final selection. FKD and TDS exhibit pronounced mode collapse, yielding visually similar samples, while DPS produces out-of-distribution images with degraded fidelity. In contrast, FVD and DTS preserve significantly higher diversity while maintaining alignment with the data distribution. Figure 2: Scaling behavior with increasing compute (NFEs). We plot FID as a function of the number of function evaluations (NFEs) on CIFAR-10 and MNIST. FVD consistently outperforms FKD and DTS across all compute budgets and settings, demonstrating favorable scaling while maintaining full parallelism. We evaluate on the task of class-conditional posterior sampling, where the goal is to draw samples from p(∣c)∝pθ()p(c∣),p(x c) p_θ(x)\,p(c ), with pθ()p_θ(x) an unconditional diffusion prior and p(c∣)p(c ) a pretrained classifier. Following [17], we evaluate on MNIST and CIFAR-10 across all 10 classes, using the log-classifier likelihood as the reward, r()=logp(c∣)r(x)= p(c ). For MNIST we additionally consider a multimodal setting in which labels are grouped into even digits evenC_even and odd digits oddC_odd, with reward r()=logmaxc∈p(c∣)r(x)= _c \,p(c ); results are reported averaged over both groups. We compare FVD against DTS [17], FKD [28], TDS [36], DAS [18], and DPS [5]. Table 1 reports mean and standard deviation across five seeds at 10610^6 NFEs, with metrics computed over 5000 generated samples. For each method we report the best performance under either reward-weighted or uniform final sampling. Figure 2 further examines how sample quality scales with compute by plotting FID against NFEs; DPS and TDS are excluded from this analysis due to high cross-seed variance that obscures meaningful comparison with stronger baselines. Results. On single-class MNIST, FVD achieves the lowest FID and MMD alongside the highest reward across all baselines, while maintaining competitive diversity. This trend holds in the multimodal MNIST (even/odd) setting and extends to CIFAR-10, where FVD again attains lower FID and MMD with competitive reward and diversity. Notably, TDS achieves high rewards but at the cost of diversity; Figure 1 reveals that this is attributable to mode collapse rather than genuine alignment with the target posterior. Figure 2 shows that FVD scales consistently with increasing compute, outperforming both FKD and DTS at higher NFE budgets. This is a particularly notable finding, as particle-based methods usually exhibit noisy scaling behavior [17] (see Figures 2 and 3); our results suggest that with appropriate design they can match the scaling efficiency of value-based approaches while remaining fully parallelizable. Qualitative results on CIFAR-10 are shown in Figure 1 for a fixed target class (car), generated with 10610^6 NFEs under reward-weighted final sampling. FKD and TDS produce near-identical samples, indicating severe diversity collapse, whereas FVD and DTS maintain substantially greater sample diversity. DPS often produces samples that lie outside the support of the base diffusion model, leading to degraded quality metrics despite exhibiting high diversity, consistent with prior observations [17]. Text-to-Image Generation. (a) Aesthetic score vs. NFEs. On the simple animals benchmark, FKD attains the highest raw rewards but exhibits clear overfitting [27], while FVD achieves competitive rewards with better visual fidelity. (b) ImageReward vs. NFEs. On DrawBench (100 prompts), FVD consistently outperforms FKD and DTS across compute budgets, demonstrating stronger scaling under prompt-conditioned evaluation. Figure 3: Reward scaling with compute in text-to-image generation. All methods are evaluated under matched NFE budgets, and each point reports the benchmark average of the best reward obtained per prompt. FVD achieves strong performance across both reward models, avoiding the over-optimization artifacts observed in FKD while offering substantially better runtime efficiency than DTS. In this setting, we study inference-time alignment for text-to-image generation, where the goal is to improve prompt adherence and perceptual quality without modifying the underlying diffusion model. This problem is particularly challenging due to the high dimensionality of the output space and potential mismatches between reward models and the data distribution. We evaluate FVD against FKD (best SMC baseline) and DTS (value-based baseline) using Stable Diffusion v1.5 [24] as the generative prior pθ(x∣y)p_θ(x y). Following [17], we consider two benchmarks. (1) DrawBench [25] consists of diverse prompts, evaluated using ImageReward [37], which captures both prompt alignment and human preferences (we use 100 prompts due to compute constraints). (2) Aesthetic Optimization [3], where prompts correspond to 45 simple animal categories and the LAION Aesthetic Predictor [35] is used as the reward, measuring visual quality independent of prompt correctness. All methods are evaluated under matched compute budgets by fixing the number of function evaluations (NFEs) per prompt. Results. We report the average reward across prompts, where for each prompt we select the highest-reward sample generated by each method. We further analyze how performance scales with compute in Figure 3. On the aesthetic benchmark (Figure 3(a)), FKD achieves the highest raw rewards but exhibits clear over-optimization, producing samples that deviate from the underlying data distribution (see Figure 8). In contrast, FVD achieves slightly lower rewards while maintaining substantially better visual fidelity, resulting in a more favorable reward–quality trade-off. On DrawBench (Figure 3(b)), FVD consistently outperforms both FKD and DTS across all compute budgets, demonstrating stronger scaling under prompt-conditioned evaluation. Qualitative results in Appendix Figures 8 and 9 further support these findings: FKD tends to produce artifact-prone or over-optimized samples, whereas FVD maintains better alignment with both the prompt and the base model distribution. We attribute this behavior to the less aggressive selection mechanism in FVD, which preserves a diverse set of candidate trajectories and avoids premature collapse to narrow high-reward modes. Unlike value-based methods, which rely on learning a value function from sampled trajectories—introducing approximation error and potential bias—FVD directly uses reward evaluations, avoiding this source of error while scaling more reliably with compute. 6 Discussion Particle Death Analysis. (a) Death rate per resampling step (b) Number of distinct lineages over time Figure 4: Particle collapse dynamics under FKD vs. FVD (K=1000K=1000, λ=1.0λ=1.0, CIFAR-10). Left: FVD maintains a consistently lower death rate throughout denoising, whereas FKD exhibits aggressive particle removal. Right: FVD preserves significantly more distinct ancestral lineages, avoiding the rapid collapse observed in FKD. Overall, FVD retains ∼10× \!10× more lineages, demonstrating improved diversity and stability. A central design goal of FVD is to mitigate particle collapse while maintaining effective selection pressure. Figure 4 compares the evolution of particle populations under FVD and FKD during denoising. We observe that FVD consistently maintains a lower death rate across resampling steps and preserves substantially more distinct lineages — where a lineage is a chain of particles connected by survival or cloning events back to a unique initial noise sample — over time. Quantitatively, for K=1000K=1000 and λ=1.0λ=1.0, FKD collapses to only 5 distinct lineages, whereas FVD retains 52, corresponding to a 10×10× improvement in population diversity. This behavior aligns with the theoretical prediction of Proposition 1, which shows that multinomial resampling eliminates an expected fraction 1/e1/e of the population in a single step under the uniform case. In contrast, the independent Bernoulli survival mechanism in FVD significantly reduces variance in offspring counts, preventing such catastrophic collapse. Beyond aggregate collapse, FVD also exhibits qualitatively different selection dynamics. While FKD removes particles aggressively and often eliminates high-reward trajectories, FVD applies more selective pressure, preferentially pruning low-reward particles while preserving promising candidates. This results in a more stable exploration of the posterior distribution and avoids premature convergence to a small set of modes. Additional statistics analyzing reward-ranked particle removal and per-step death distributions are provided in Appendix A.1, where we show that FVD consistently concentrates removals among low-reward particles while preserving high-reward trajectories. Inference Efficiency. Figure 5: FID vs. wall-clock time on CIFAR-10. FVD achieves lower FID across all time budgets and is ∼66× \!66× faster than DTS at matched NFEs. Notably, even at its longest runtime, DTS fails to reach the worst FID achieved by FVD, highlighting the substantial efficiency gap. Inference-time efficiency is a key consideration for practical deployment, as methods that significantly increase wall-clock time are often infeasible despite improvements in sample quality. Value-based approaches such as DTS [17] rely on constructing a search tree over the denoising trajectory. This process is inherently sequential: each node expansion depends on value estimates computed at earlier steps, limiting opportunities for parallelization across samples. As a result, DTS incurs substantial runtime overhead, particularly at large NFE budgets. In contrast, FVD operates on a population of K particles that evolve independently under the diffusion process, with only lightweight resampling operations coupling them. This structure allows full parallelization across particles, making FVD significantly more efficient in practice. Figure 5 compares FID as a function of wall-clock time on CIFAR-10. Across all time budgets, FVD consistently achieves lower FID than DTS. At matched NFE budgets, FVD is approximately 66×66× faster. Notably, even after DTS exceeds the maximum runtime observed for FVD, it still fails to match the highest FID achieved by FVD. We note that DTS amortizes part of its computational cost by caching the search tree across multiple queries, whereas FVD must rerun sampling for each query. Despite this advantage, DTS remains significantly slower, highlighting the inherent efficiency benefits of particle-based inference. Ablation on Target Absorption Rate α∗α^*. (a) Mean reward vs. α∗α^* (b) MMD vs. α∗α^* Figure 6: Effect of target absorption rate α∗α^* on reward–diversity trade-off (CIFAR-10, 10610^6 NFEs). Left: Mean reward increases and variance decreases with α∗α^* as stronger selection pressure favors high-reward samples. Right: MMD is minimized at intermediate α∗α^*, indicating optimal distributional coverage; large α∗α^* leads to increased MMD due to mode collapse. These results show that α∗α^* provides a direct and interpretable tradeoff over reward maximization and sample diversity. The Robbins–Monro controller adapts the regularization parameter λ at each resampling step to match the observed absorption fraction to a user-specified target α∗α^*. This makes α∗α^* the primary parameter governing selection pressure in FVD. We perform an ablation over α∗∈0.1,0.3,0.5,0.7,0.9α^*∈\0.1,0.3,0.5,0.7,0.9\ on CIFAR-10, while keeping the NFE budget fixed at 10610^6, with initial λ0=1 _0=1 and identical learning rate schedules across all runs. Results are shown in Figure 6. At low values of α∗α^*, selection pressure is weak: a large fraction of particles survive regardless of reward, resulting in limited guidance from the reward function. Consequently, we observe low mean reward, high variance, and elevated MMD, indicating that the generated distribution remains close to the unguided prior. As α∗α^* increases, selection becomes more discriminative, improving both reward alignment and distributional quality. Performance peaks around α∗=0.5α^*=0.5, where mean reward is maximized and MMD is minimized, suggesting an optimal balance between exploitation and diversity. For larger values of α∗α^*, the selection process becomes overly aggressive. A large fraction of particles are removed at each step, leading to reduced diversity and collapse toward a small number of high-reward modes. While mean reward may continue to increase, MMD rises significantly, reflecting poorer coverage of the target distribution. Overall, this ablation highlights a clear reward–diversity trade-off controlled by α∗α^*. Adaptive vs. Fixed λ. Figure 7: Effect of adaptive vs. fixed λ on FID. Adaptive λ provides consistent improvements across a wide range of initializations, with the largest gains observed when the fixed λ is poorly calibrated. We compare the proposed adaptive λ scheme based on Robbins–Monro updates against fixed λ across λ0∈0.1,0.3,0.5,0.7,0.9 _0∈\0.1,0.3,0.5,0.7,0.9\ on CIFAR-10, while keeping all other hyperparameters constant. Here λ0 _0 denotes the initial value of the controller and, in the fixed baseline, the constant value used throughout sampling. These values can therefore be interpreted directly as the overall alignment strength. As shown in Figure 7, the benefit of adaptive λ is limited near the best fixed setting, λ0≈0.3 _0≈ 0.3, where the constant value is already reasonably calibrated and requires minimal correction. However, as λ0 _0 deviates from this range, the advantage of adaptation becomes more pronounced. In particular, at λ0=0.7 _0=0.7, fixed λ induces overly strong selection pressure, leading to premature loss of diversity and worse results; the adaptive controller mitigates this by reducing λ during sampling, resulting in significant improvements in FID. At λ0=0.9 _0=0.9, the relative gain from adaptation decreases slightly. We hypothesize that, at this level of selection pressure, the particle system becomes overly selective early in the process, and with only four resampling steps, the trajectories diverge too far for the controller to effectively correct. 7 Conclusions & Limitations We introduced FVD, an inference-time alignment method for diffusion models based on the Fleming–Viot particle system. By replacing multinomial resampling with independent Bernoulli survival and uniform donor selection, FVD bounds the variance of each particle’s survival decision by O(1)O(1) and mitigates the lineage collapse observed in prior particle-based methods. A Robbins–Monro controller adapts the selection pressure λ online to match a target absorption rate α∗α^*, eliminating the need for manual tuning. We provide heuristic justification that, in the large-population regime, the idealized process at any fixed choice of the hyperparameter λ asymptotically targets π∗(x0)∝pθ(x0)exp(λr(x0))π^*(x_0) p_θ(x_0) (λ r(x_0)) for the exponential potential family used in our experiments. Empirically, FVD improves FID and distributional quality over strong baselines on class-conditional tasks, outperforms FKD and DTS on DrawBench, preserves up to 10×10× more distinct lineages, and is approximately 66×66× faster than DTS at matched NFE budgets. On the aesthetic benchmark, it achieves a more favorable reward–quality trade-off than FKD rather than the highest raw reward. These results highlight the effectiveness of simple, fully parallel particle-based inference with adaptive selection. That being said, we do not explore extensions of FVD to alternative generative frameworks such as flow matching or consistency models, and our method continues to rely on Tweedie’s estimates, which can be noisy at early timesteps. Additionally, we do not consider multi-objective alignment settings with competing reward functions. We leave these directions for future work. References [1] A. Bansal, H. Chu, A. Schwarzschild, S. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein (2023) Universal guidance for diffusion models. External Links: 2302.07121, Link Cited by: §1, §2. [2] T. Bie, M. Cao, K. Chen, L. Du, M. Gong, Z. Gong, Y. Gu, J. Hu, Z. Huang, Z. Lan, C. Li, C. Li, J. Li, Z. Li, H. Liu, L. Liu, G. Lu, X. Lu, Y. Ma, J. Tan, L. Wei, J. Wen, Y. Xing, X. Zhang, J. Zhao, D. Zheng, J. Zhou, J. Zhou, Z. Zhou, L. Zhu, and Y. Zhuang (2025) LLaDA2.0: scaling up diffusion language models to 100b. External Links: 2512.15745, Link Cited by: §1. [3] K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine (2024) Training diffusion models with reinforcement learning. External Links: 2305.13301, Link Cited by: §1, §5. [4] G. Cardoso, Y. J. E. Idrissi, S. L. Corff, and E. Moulines (2023) Monte carlo guided diffusion for bayesian linear inverse problems. External Links: 2308.07983, Link Cited by: §1. [5] H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye (2024) Diffusion posterior sampling for general noisy inverse problems. External Links: 2209.14687, Link Cited by: §1, §2, §5. [6] K. Clark, P. Vicol, K. Swersky, and D. J. Fleet (2024) Directly fine-tuning diffusion models on differentiable rewards. External Links: 2309.17400, Link Cited by: §1. [7] B. Cloez and M. Thai (2016) Fleming-viot processes : two explicit examples. External Links: 1603.04670, Link Cited by: §3. [8] P. Del Moral (2004-05) Feynman-kac formulae: genealogical and interacting particle systems with applications. Vol. 100. External Links: ISBN 0387202684, Document Cited by: item 2, Appendix B, §2, §3, §3, §3. [9] P. Dhariwal and A. Nichol (2021) Diffusion models beat gans on image synthesis. External Links: 2105.05233, Link Cited by: §2. [10] Z. Dou and Y. Song (2024) Diffusion posterior sampling for linear inverse problem solving: a filtering perspective. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1, §2. [11] S. N. Ethier and T. G. Kurtz (1993) Fleming–viot processes in population genetics. SIAM Journal on Control and Optimization 31 (2), p. 345–386. External Links: Document, Link, https://doi.org/10.1137/0331019 Cited by: item 2, Appendix B, §3, §3. [12] Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee (2023) DPOK: reinforcement learning for fine-tuning text-to-image diffusion models. External Links: 2305.16381, Link Cited by: §1. [13] Google DeepMind (2025) Veo 3. Google. Note: https://aistudio.google.com/models/veo-3Accessed: 2026-03-15 Cited by: §1. [14] Y. He, N. Murata, C. Lai, Y. Takida, T. Uesaka, D. Kim, W. Liao, Y. Mitsufuji, J. Z. Kolter, R. Salakhutdinov, and S. Ermon (2023) Manifold preserving guided diffusion. External Links: 2311.16424, Link Cited by: §2. [15] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. External Links: 2006.11239, Link Cited by: §1. [16] Z. Huang, K. Ramnath, Y. Chen, A. Feng, S. Woo, B. Srinivasan, Z. Xu, K. Zhou, S. Wang, H. Ding, and L. L. Cheong (2026) Diffusion language model inference with monte carlo tree search. External Links: 2512.12168, Link Cited by: §2. [17] V. Jain, K. Sareen, M. Pedramfar, and S. Ravanbakhsh (2025) Diffusion tree sampling: scalable inference-time alignment of diffusion models. External Links: 2506.20701, Link Cited by: §1, §2, §5, §5, §5, §5, §6. [18] S. Kim, M. Kim, and D. Park (2025) Test-time alignment of diffusion models without reward over-optimization. External Links: 2501.05803, Link Cited by: §5. [19] X. Li, M. Uehara, X. Su, G. Scalia, T. Biancalani, A. Regev, S. Levine, and S. Ji (2025) Dynamic search for inference-time alignment in diffusion models. External Links: 2503.02039, Link Cited by: §1, §2. [20] X. Li, Y. Zhao, C. Wang, G. Scalia, G. Eraslan, S. Nair, T. Biancalani, S. Ji, A. Regev, S. Levine, and M. Uehara (2024) Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding. External Links: 2408.08252, Link Cited by: §1, §2. [21] N. Ma, S. Tong, H. Jia, H. Hu, Y. Su, M. Zhang, X. Yang, Y. Li, T. Jaakkola, X. Jia, and S. Xie (2025) Inference-time scaling for diffusion models beyond scaling denoising steps. External Links: 2501.09732, Link Cited by: §2. [22] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach (2023) SDXL: improving latent diffusion models for high-resolution image synthesis. External Links: 2307.01952, Link Cited by: §1. [23] (2002) Robbins-monro algorithm. In Stochastic Approximation and Its Applications, p. 1–24. External Links: ISBN 978-0-306-48166-6, Document, Link Cited by: §4. [24] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. External Links: 2112.10752, Link Cited by: §1, §5. [25] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. (2022) Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems 35, p. 36479–36494. Cited by: §5. [26] S. S. Sahoo, M. Arriola, Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, and V. Kuleshov (2024) Simple and effective masked diffusion language models. External Links: 2406.07524, Link Cited by: §1. [27] S. Shekhar, S. Singh, and T. Zhang (2025) SEE-dpo: self entropy enhanced direct preference optimization. External Links: 2411.04712, Link Cited by: Figure 8, Figure 8, 3(a), 3(a). [28] R. Singhal, Z. Horvitz, R. Teehan, M. Ren, Z. Yu, K. McKeown, and R. Ranganath (2025) A general framework for inference-time scaling and steering of diffusion models. External Links: 2501.06848, Link Cited by: §1, §2, §4, §4, §5. [29] J. Song, C. Meng, and S. Ermon (2022) Denoising diffusion implicit models. External Links: 2010.02502, Link Cited by: §3. [30] J. Song, A. Vahdat, M. Mardani, and J. Kautz (2023) Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, External Links: Link Cited by: §2. [31] Y. Song and S. Ermon (2020) Generative modeling by estimating gradients of the data distribution. External Links: 1907.05600, Link Cited by: §1. [32] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. External Links: 2011.13456, Link Cited by: §2. [33] B. L. Trippe, J. Yim, D. Tischer, D. Baker, T. Broderick, R. Barzilay, and T. Jaakkola (2023) Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. External Links: 2206.04119, Link Cited by: §2. [34] B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik (2023) Diffusion model alignment using direct preference optimization. External Links: 2311.12908, Link Cited by: §1. [35] Z. J. Wang, E. Montoya, D. Munechika, H. Yang, B. Hoover, and D. H. Chau (2022) Large-scale prompt gallery dataset for text-to-image generative models. arXiv:2210.14896 [cs]. External Links: Link Cited by: §5. [36] L. Wu, B. Trippe, C. Naesseth, D. Blei, and J. P. Cunningham (2023) Practical and asymptotically exact conditional sampling in diffusion models. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 31372–31403. External Links: Link Cited by: §2, §5. [37] J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong (2023) ImageReward: learning and evaluating human preferences for text-to-image generation. External Links: 2304.05977, Link Cited by: §5. [38] K. Yang, J. Tao, J. Lyu, C. Ge, J. Chen, Q. Li, W. Shen, X. Zhu, and X. Li (2024) Using human feedback to fine-tune diffusion models without any reward model. External Links: 2311.13231, Link Cited by: §1. [39] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, D. Yin, Y. Zhang, W. Wang, Y. Cheng, B. Xu, X. Gu, Y. Dong, and J. Tang (2025) CogVideoX: text-to-video diffusion models with an expert transformer. External Links: 2408.06072, Link Cited by: §1. [40] J. Yoon, H. Cho, D. Baek, Y. Bengio, and S. Ahn (2026) Monte carlo tree diffusion for system 2 planning. External Links: 2502.07202, Link Cited by: §2. [41] T. Zhang, J. Pan, R. Feng, and T. Wu (2026) VFScale: intrinsic reasoning through verifier-free test-time scalable diffusion model. External Links: 2502.01989, Link Cited by: §2. [42] J. Zhao, Z. Wang, Y. Liao, C. Zhang, and Y. Ye (2026) Controllable graph generation with diffusion models via inference-time tree search guidance. External Links: 2510.10402, Link Cited by: §2. Appendix A Additional Experiments Figure 8: Qualitative comparison on aesthetic optimization. Samples from each method for a 50K NFE budget. As emphasized earlier, FKD achieves a high aesthetic score but produces samples that have overfit to the reward model and do not make sense [27], a symptom of over-optimization. FVD generates visually appealing images that remain faithful to the prompt. Figure 9: Qualitative comparison on DrawBench (ImageReward). Best samples generated by each method for each prompt at a 50K NFE budget. FVD produces samples with higher prompt alignment and perceptual quality compared to DTS and FKD, consistent with the quantitative results in Figure 3(b). A.1 Particle Death Analysis We provide a detailed analysis of particle collapse behavior to complement the results in the main text. Table 2 summarizes key statistics, while Figure 10 visualizes the distribution of reward ranks among removed particles. FKD exhibits near-uniform removal across reward ranks, with a mean rank of 0.4550.455 and 24.6%24.6\% of removed particles belonging to the top 30%30\% of the reward distribution (rank >0.7>0.7). This indicates that particle removal is largely stochastic and not sufficiently guided by reward, leading to the elimination of many high-quality trajectories. In contrast, FVD concentrates removals among low-reward particles, with a lower mean rank of 0.3960.396 and only 17.3%17.3\% of removed particles in the top 30%30\%. This demonstrates that the survival mechanism in FVD effectively prioritizes high-reward trajectories while selectively pruning less promising ones. These results confirm that FVD achieves a more desirable balance between exploration and exploitation: it maintains diversity by preserving multiple high-reward lineages while still applying sufficient pressure to guide the population toward high-probability regions of the target distribution. Table 2: Particle collapse statistics. K=1000K=1000, λ=1.0λ=1.0 (fixed), CIFAR-10 class 1. FVD significantly reduces collapse, preserves more lineages, and applies more selective (reward-aware) pruning compared to FKD. Metric FKD FVD Final distinct lineages 5 52 Expected uncapped losses (1−1/e)⋅K(1-1/e)·K 632 N/A Mean death rate per step 0.696 0.514 Mean reward rank of killed (↓ ) 0.455 0.396 Frac. killed with rank >0.7>0.7 (↓ ) 0.246 0.173 (a) FKD (b) FVD Figure 10: Reward-rank distribution of removed particles per resampling step (K=1000K=1000, λ=1.0λ=1.0, CIFAR-10 class 1). FKD removes particles across the entire reward spectrum, including many high-reward samples, indicating weak selection specificity. In contrast, FVD concentrates removals among low-reward particles, preserving high-reward trajectories and maintaining a more faithful approximation to the target distribution. A.2 Effect of Stochastic Rebirth on Sample Quality When a particle dies in FVD, it is reborn by copying a donor particle and reapplying the DDIM update with noise controlled by rebirth_eta∈[0,1] rebirth\_eta∈[0,1]. When rebirth_eta=0 rebirth\_eta=0, rebirth is deterministic, producing an exact clone that follows the same trajectory as its donor. Figure 11: FID vs. rebirth_eta for FVD on CIFAR-10 (class 0) using 10510^5 NFEs, averaged over five seeds (shaded region = ± std). Deterministic rebirth (rebirth_eta=0 rebirth\_eta=0) leads to poor FID due to diversity collapse, while moderate stochasticity (rebirth_eta=0.4 rebirth\_eta=0.4) yields the best performance. For rebirth_eta>0 rebirth\_eta>0, rebirth becomes stochastic, causing the new particle to deviate from the donor and explore a nearby denoising path, thereby reintroducing diversity into the population. Figure 11 shows FID as a function of rebirth_eta, averaged over five seeds on CIFAR-10 (class 0) with 10510^5 NFEs. Deterministic (rebirth_eta=0 rebirth\_eta=0) and near-deterministic (rebirth_eta=0.2 rebirth\_eta=0.2) settings result in the worst FID, indicating that cloning leads to rapid loss of diversity across resampling steps. Introducing moderate stochasticity (rebirth_eta=0.4 rebirth\_eta=0.4) achieves the best performance, suggesting that a small amount of noise is sufficient to maintain trajectory diversity without degrading reward guidance. For larger values, performance remains largely stable over rebirth_eta∈[0.4,1.0] rebirth\_eta∈[0.4,1.0], indicating that the benefits of stochastic rebirth quickly saturate and the method is not sensitive to the exact choice once sufficient noise is introduced. Appendix B Informal Theoretical Justification and Derivation This appendix gives an informal justification for why an idealized version of FVD should approximate a reward-tilted target distribution. Our goal is not to provide a complete convergence proof for the full practical algorithm, but rather to explain the limiting picture that motivates the method. In particular, the discussion below analyzes the process at a fixed choice of the hyperparameter λ, with no death cap and no adaptive update of λ. We also ignore the additional stochastic rebirth perturbation used in Algorithm 1, and instead analyze the underlying Fleming–Viot selection mechanism at the path-measure level. Lemma 1. Consider drawing K path samples xt:T(i)∼πt(xt:T)x_t:T^(i) _t(x_t:T) for i=1,…,Ki=1,…,K. Apply an idealized Fleming–Viot resampling step in which particle i survives with probability si(t)=Gt(xt:T(i))maxjGt(xt:T(j)),s_i^(t)= G_t(x_t:T^(i)) _jG_t(x_t:T^(j)), and each rejected particle is replaced by copying the full path of a uniformly chosen survivor. Let the resulting resampled paths be x~t:T(i) x_t:T^(i), with distribution π~t π_t. Then, in the large-K limit, the empirical law of the resampled particles is proportional to the GtG_t-tilted path measure π~t(x~t:T)∝πt(x~t:T)Gt(x~t:T). π_t( x_t:T) _t( x_t:T)\,G_t( x_t:T). We state this result without proof. Informally, the acceptance step biases the surviving population proportionally to Gt(xt:T)G_t(x_t:T), while uniform donor selection preserves this tilted empirical law after the constant-population refill. Analogous mean-field and propagation-of-chaos arguments appear in prior analyses of related interacting particle systems [8, 11]. B.1 Main Claim Using Lemma 1, we can prove the following result. Claim 1 (Informal large-population limit). Consider the idealized version of FVD with particle count K, resampling schedule T, and a fixed choice of the hyperparameter λ. Assume: 1. the number of particles satisfies K→∞K→∞; 2. the dependencies induced by the birth–death interaction become negligible in the large-K limit, as predicted by the usual mean-field / propagation-of-chaos theory for interacting particle systems [8, 11]. Then the empirical measure of particles produced by FVD should converge to a reward-tilted distribution of the form πt∗(xt:T)∝pθ(xt:T)∏s=tT−1Gs(xs:T), _t^*(x_t:T) p_θ(x_t:T) _s=t^T-1G_s(x_s:T), where pθp_θ is the probability of the diffusion path xt:Tx_t:T. In particular, this implies that π0∗(x0:T)∝pθ(x0:T)exp(λr(x0))⇒π0∗(x0)∝pθ(x0)exp(λr(x0)). _0^*(x_0:T) p_θ(x_0:T) (λ r(x_0)) _0^*(x_0) p_θ(x_0) (λ r(x_0)). We only present an informal proof sketch, by induction on t=T,T−1,…,0t=T,T-1,…,0. At t=Tt=T, no potential has yet been applied, so πT∗(xT)=pθ(xT) _T^*(x_T)=p_θ(x_T) holds. Now at t<Tt<T, assume that the claim holds at t+1,…,Tt+1,…,T. For each particle i, as K→∞K→∞, we have πt+1(xt+1:T(i))→πt+1∗(xt+1:T(i))∝pθ(xt+1:T(i))∏s=t+1T−1Gs(xs:T(i)). _t+1(x_t+1:T^(i))→ _t+1^*(x_t+1:T^(i)) p_θ(x_t+1:T^(i)) _s=t+1^T-1G_s(x_s:T^(i)). After one reverse diffusion step from xt+1(i)x_t+1^(i) to xt(i)x_t^(i) using the diffusion path law pθ(xt∣xt+1:T)p_θ(x_t x_t+1:T), and before FV resampling, we therefore have πt(xt:T(i))∝pθ(xt:T(i))∏s=t+1T−1Gs(xs:T(i)), _t(x_t:T^(i)) p_θ(x_t:T^(i)) _s=t+1^T-1G_s(x_s:T^(i)), since the new reverse transition only extends the path from xt+1:Tx_t+1:T to xt:Tx_t:T. The idealized FV step then applies the additional weighting factor Gt(xt:T(i))G_t(x_t:T^(i)). Hence, by Lemma 1, after resampling we obtain πt∗(xt:T(i))∝pθ(xt:T(i))∏s=tT−1Gs(xs:T(i)), _t^*(x_t:T^(i)) p_θ(x_t:T^(i)) _s=t^T-1G_s(x_s:T^(i)), which finishes the induction. This argument should be interpreted only as motivation for the mean-field limit at fixed λ. The practical algorithm in Algorithm 1 additionally includes capped deaths, adaptive updates of λ, and stochastic rebirth at the state level rather than exact path copying; these modifications improve robustness in practice. Appendix C Probabilistic Analysis of Particle Collapse Proposition 1 (Multinomial collapse probability). Let w1,…,wKw_1,…,w_K be positive weights such that ∑i=1Kwi=1 _i=1^Kw_i=1. Consider K samples drawn with replacement from the categorical distribution defined by w. The probability that all K draws select the same ancestor is ∑i=1KwiK _i=1^Kw_i^K. For a fixed ancestor i, the probability that all K draws land on i is wiKw_i^K, and these events are disjoint across i. In the uniform case wi=1/Kw_i=1/K, this becomes ∑i=1K(1K)K=K1−K, _i=1^K ( 1K )^K=K^1-K, which tends to zero as K→∞K→∞. However, the more relevant quantity for diversity collapse is not complete monopoly by a single ancestor, but the expected number of distinct ancestors represented after resampling. In the uniform case, the expected number of distinct ancestors selected in K draws with replacement is [distinct ancestors]=K(1−(1−1K)K)→K→∞K(1−e−1)≈0.632K.E[distinct ancestors]=K (1- (1- 1K )^K ) K→∞K(1-e^-1)≈ 0.632\,K. (18) Equivalently, the expected number of ancestors that receive no offspring is K(1−1K)K→K→∞Ke,K (1- 1K )^K K→∞ Ke, so asymptotically a fraction 1/e≈36.8%1/e≈ 36.8\% of the population is eliminated at each multinomial resampling step, regardless of K. This 1/e1/e loss is a fundamental consequence of sampling with replacement and does not vanish as the population size grows. Proof. For a fixed ancestor i, the probability that all K draws land on i is wiKw_i^K. Since these events are disjoint across i, the probability that all draws select the same ancestor is ∑i=1KwiK _i=1^Kw_i^K. In the uniform case this becomes ∑i=1K(1K)K=K1−K, _i=1^K ( 1K )^K=K^1-K, which tends to zero as K→∞K→∞. For the diversity-loss statement, a fixed particle i receives no offspring in K draws from the uniform categorical distribution with probability (1−1/K)K(1-1/K)^K. Therefore, the expected number of particles with zero offspring is K(1−1K)K.K (1- 1K )^K. Taking K→∞K→∞ and using limK→∞(1−1/K)K=e−1 _K→∞(1-1/K)^K=e^-1 yields K(1−1K)K→Ke.K (1- 1K )^K→ Ke. Hence the fraction of the population lost in one multinomial resampling step converges to 1/e1/e, or approximately 36.8%36.8\%, independent of K. ∎ Proposition 2 (Bernoulli variance bound). Let s1,…,sK∈(0,1]s_1,…,s_K∈(0,1] be survival probabilities, and let Bi∼Bernoulli(si)B_i (s_i) be independent survival indicators. Define Nsurv=∑i=1KBi.N_surv= _i=1^KB_i. Then: (i) For each particle, Var(Bi)=si(1−si)≤1/4Var(B_i)=s_i(1-s_i)≤ 1/4, so the variance of an individual survival decision is O(1)O(1) and does not depend on K. (i) The total survival count satisfies Var(Nsurv)=∑i=1Ksi(1−si)≤K/4.Var(N_surv)= _i=1^Ks_i(1-s_i)≤ K/4. Proof. For a Bernoulli random variable Bi∼Bernoulli(si)B_i (s_i), we have Var(Bi)=si(1−si).Var(B_i)=s_i(1-s_i). The function f(s)=s(1−s)f(s)=s(1-s) is maximized at s=1/2s=1/2, where f(1/2)=1/4f(1/2)=1/4. Hence Var(Bi)≤1/4Var(B_i)≤ 1/4 for all si∈[0,1]s_i∈[0,1], establishing (i). For (i), independence gives Var(Nsurv)=∑i=1KVar(Bi)=∑i=1Ksi(1−si)≤K4.Var(N_surv)= _i=1^KVar(B_i)= _i=1^Ks_i(1-s_i)≤ K4. ∎ Remark 1. For comparison, multinomial resampling with K draws from K categories also exhibits O(K)O(K) variability in diversity retention. In particular, for the uniform case, the number of distinct ancestors selected has mean K(1−(1−1K)K),K (1- (1- 1K )^K ), and variance K(1−(1−1K)K)−K2(1K−1K(1−1K)K)2,K (1- (1- 1K )^K )-K^2 ( 1K- 1K (1- 1K )^K )^2, which is O(K)O(K). Although this appears similar to the O(K)O(K) upper bound in Proposition 2(i), the key distinction is structural: under Bernoulli deaths, each particle faces an independent survival decision with per-particle variance bounded by 1/41/4, whereas under multinomial resampling the outcomes of all K particles are jointly coupled in a single draw. This coupling is what produces the catastrophic 1/e1/e loss rate that persists regardless of population size. Proposition 3 (Natural scaling of selection pressure). Let si(t)=exp(λ||(ri−rmax))s_i^(t)= \! ( λ|T|(r_i-r_ ) ) as in Eq. (14). The expected absorption fraction at step t satisfies [αt]=1K∑i=1K(1−exp(λ||(ri−rmax))).E[ _t]= 1K _i=1^K (1- \! ( λ|T|(r_i-r_ ) ) ). (19) Let Δt=rmax(t)−rmin(t) _t=r_ ^(t)-r_ ^(t) denote the reward range at timestep t. Then: (i) For any λ≥0λ≥ 0, [αt]≤1−exp(−λ||Δt)E[ _t]≤ 1- \! (- λ|T| _t ). (i) As the reward range collapses, Δt→0 _t→ 0, we have [αt]→0E[ _t]→ 0 for any fixed λ: selection pressure vanishes automatically. (i) [αt]E[ _t] is strictly increasing in λ for any reward distribution with Δt>0 _t>0, making it a valid signal for the Robbins–Monro controller. Proof. Eq. (19) follows directly from linearity of expectation, since each Bi∼Bernoulli(1−si(t))B_i (1-s_i^(t)) independently. (i) Since ri−rmax≥−Δtr_i-r_ ≥- _t for all i, we have 1−exp(λ||(ri−rmax))≤1−exp(−λ||Δt).1- \! ( λ|T|(r_i-r_ ) )≤ 1- \! (- λ|T| _t ). Averaging over i gives [αt]≤1−exp(−λ||Δt).E[ _t]≤ 1- \! (- λ|T| _t ). (i) As Δt→0 _t→ 0, all ri→rmaxr_i→ r_ , so si(t)=exp(λ||(ri−rmax))→exp(0)=1s_i^(t)= \! ( λ|T|(r_i-r_ ) )→ (0)=1 for all i, giving [αt]→0E[ _t]→ 0. This explains the empirical observation that almost no particles die at early noisy timesteps where Tweedie estimates are unreliable and rewards cluster tightly. (i) Differentiating Eq. (19) with respect to λ: ∂[αt]∂λ=1K∑i=1Krmax−ri||exp(λ||(ri−rmax))≥ 0, ∂\,E[ _t]∂λ= 1K _i=1^K r_ -r_i|T|\, \! ( λ|T|(r_i-r_ ) )\;≥\;0, (20) with equality only if ri=rmaxr_i=r_ for all i, i.e. all rewards are identical. For any distribution with positive spread Δt>0 _t>0, the derivative is strictly positive, confirming monotonicity and the identifiability of the Robbins–Monro signal. ∎