Paper deep dive
PartiCam: Camera Controlled Video Generation with Reward Guidance
Amine Ouasfi, Runjia Li, Junlin Han, Eric Marchand, Philip H. S. Torr, Adnane Boukhayma
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present PartiCam, a training-free Particle filtering rooted method for improved Camera controlled video generation. Generating videos that follow a precisely specified camera trajectory remains challenging for large video diffusion models. Training-free approaches are backbone-agnostic and avoid the need to construct large camera-annotated datasets by steering pretrained models toward the desired camera motion at test time. This enables the generation of camera-controlled video data that can subsequently be used to train camera-conditioned video diffusion models. Existing sampling-based guidance approaches often suffer from unstable trajectories: they either explore too broadly and fail to respect the target camera motion or collapse early and lose visual diversity over time. We introduce a global-local refinement framework for diffusion reward guidance, enabling accurate and consistent camera control during video generation. Our method builds on Sequential Monte-Carlo (SMC) guidance, but introduces a local refinement stage based on particle filtered resampling. Experiments show large improvements in camera trajectory adherence, reduced drift, and better visual quality, without requiring model retraining.
Tags
Links
- Source: https://arxiv.org/abs/2609.39504v1
- Canonical: https://arxiv.org/abs/2609.39504v1
Trouble viewing inline? Open PDF directly →
Full Text
65,033 characters extracted from source content.
Expand or collapse full text
PartiCam: Camera Controlled Video Generation with Reward Guidance Amine Ouasfi Affiliation: INRIA, Univ. Rennes, CNRS, IRISA Runjia Li Affiliation: University of Oxford Junlin Han Affiliation: Meta Affiliation: University of Oxford Eric Marchand Affiliation: INRIA, Univ. Rennes, CNRS, IRISA Philip H.S. Torr Affiliation: University of Oxford Adnane Boukhayma Affiliation: INRIA, Univ. Rennes, CNRS, IRISA Figure 1: Our method improves camera controlled video generation. We show a top-down view of the estimated camera trajectories from the videos generated with our method, its baselines and the Ground-truth. We show improvement over TrajectoryCrafter [82], and NVS-Solver[78] using both SVD (top) [7] and CogVideo (Bottom) [33] backbones. 1 Abstract We present PartiCam, a training-free Particle filtering rooted method for improved Camera controlled video generation. Generating videos that follow a precisely specified camera trajectory remains challenging for large video diffusion models. Training-free approaches are backbone-agnostic and avoid the need to construct large camera-annotated datasets by steering pretrained models toward the desired camera motion at test time. This enables the generation of camera-controlled video data that can subsequently be used to train camera-conditioned video diffusion models. Existing sampling-based guidance approaches often suffer from unstable trajectories: they either explore too broadly and fail to respect the target camera motion or collapse early and lose visual diversity over time. We introduce a global–local refinement framework for diffusion reward guidance, enabling accurate and consistent camera control during video generation. Our method builds on Sequential Monte-Carlo (SMC) guidance, but introduces a local refinement stage based on particle filtered resampling. Experiments show large improvements in camera trajectory adherence, reduced drift, and better visual quality, without requiring model retraining. 2 Introduction Recent advances in multimodal generation via diffusion and flow models have revolutionized creative fields, giving rise to video generation models [18, 53, 55, 11, 83] of impressive quality alongside emerging zero-shot generalization capabilities [68]. Yet these models often lack fine-grained control mechanisms. Building generation models that are action-conditioned, causally and physically controllable, and 3D-grounded represents a key step toward the vision of comprehensive world models. Open-source latent video models such as SVD [7], CogVideo [33, 75], Wan [67], and Hunyuan [41], etc. provide a practical and manageable testbed for developing ideas that can scale to larger models and broader modalities. Among the most important axes of control in visual data is camera viewpoint, which accounts for a dominant source of variation in images and videos and arguably offers the most intuitive form of control for these modalities. The task of camera trajectory control in video generation from a single reference image, sparse views, or a few frames of a dynamic scene is closely analogous to few-shot novel view synthesis [48, 37]: the video model serves as a spatio-temporal prior that enables educated completion of unobserved viewpoints. The notorious difficulty of these tasks in classical computer vision and graphics underscores their inherent challenges, chief among them ensuring 3D and 4D scene consistency alongside photorealistic inpainting of occluded regions. Camera control can be achieved by training conditional models from scratch or via adapter like fine-tuning [29, 82]. However, the scarcity of large-scale paired supervision, prohibitive computational training costs, and out-of-distribution (OOD) generalization failures on challenging geometries or novel camera trajectories can make such approaches expensive or suboptimal. A further, more practical concern is obsolescence: open-source latent video models are evolving rapidly, and a control module fine-tuned for one backbone is bound to that backbone’s architecture, latent space, and conditioning interface, so it risks being superseded before it is widely adopted. Training-free methods, by contrast, are largely backbone-agnostic and inherit the improvements of each new generation of base models at no additional training cost. This motivates training-free, test-time alternatives, which have become an increasingly active direction. Beyond their immediate utility, we argue that such methods are also a prerequisite for the training-based approaches they appear to compete with: a reliable test-time procedure is a generator of paired camera-annotated video data, whose samples can be distilled back into model weights. This mirrors the trajectory of LLM post-training, where sampling from a model and fine-tuning on the verified subset, i.e. rejection sampling fine-tuning [85, 84, 60], converts an expensive inference-time procedure into cheap model capability, and where the filter proved far easier to construct than the generator. One of the established approaches to training-free camera control is score modulation (e.g. [78]): at each denoising step, the score function is adjusted using warped input views as scene priors, steering the latent trajectory toward the target camera pose. We build on this formulation without loss of generality as our underlying control mechanism. Despite its effectiveness, score modulation operates over a single denoising trajectory. Errors made early in sampling, when an unlucky noise realization places the model on a poor generative path, compound through subsequent steps and become increasingly difficult to correct, leading to persistent camera drift even as the number of inference steps increases. One natural remedy is to introduce stochastic restarts [73] during sampling: by re-injecting forward-diffusion noise into an intermediate latent and re-denoising, the model is given an opportunity to escape poor generative trajectories and recover camera consistency. However, unrestricted restarts are unstable in practice and can severely degrade generation quality by discarding useful structure accumulated during denoising. To stabilize restarts, we propose equipping them with reward-guided resampling: at each restart, we draw N perturbed candidate continuations via noising-denoising, score each against a camera alignment reward, and retain the best according to these scores. Our formulation can support both differentiable and non-differentiable reward functions, enabling flexible integration of arbitrary camera consistency or visual quality metrics. This local procedure, which we term PF-Restart, substantially improves trajectory coherence. Yet because it operates independently for each candidate, it provides no mechanism to leverage a broader population of trajectories for global exploration, and is therefore still susceptible to collective drift when all candidates enter a similarly poor region of latent space. We address this by embedding local PF-Restart within a global Sequential Monte Carlo (SMC) framework. Rather than steering a single trajectory, our method maintains a population of K candidate video trajectories (i.e. particles) which are jointly propagated through the denoising process and globally reweighted and resampled according to how well they satisfy the target camera trajectory and desired visual properties. High-reward particles are preferentially retained and diversified via local PF-Restart restarts, whereas low-reward ones are discarded, giving the sampler a principled mechanism to continuously reallocate compute toward geometrically coherent and visually appealing trajectories. This combination of global population filtering and local reward-guided refinement empirically mitigates early drift, avoids collapse into low-quality or geometrically inconsistent solutions, and preserves sample diversity – limitations that plague both score modulation and unrestricted restart strategies individually. Importantly, our method is not limited to the training-free setting: it can be applied on top of trained camera control baselines (e.g. [29, 82]), mitigating some of their OOD generalization failures on challenging geometries or out-of-distribution trajectories. We validate our approach through quantitative and qualitative evaluation on video reshooting from images and videos for static and dynamic scenes, following the experimental protocol of NVSSolver [78]. Our method achieves state-of-the-art performance across these benchmarks, yielding superior visual results and stronger consistency with the desired camera trajectory, with improvements over both popular training-free [78] and training-based [82] methods. In summary, our contributions are: • A training-free strategy for enhanced camera control in video model generation via reward-guidance combining global SMC trajectory exploration with local particle filtering steered restarts (PF-Restart). • Analysis of reward functions that benefit the task of camera control in this context. • Demonstrated improvements over both training-free and training-based camera control state-of-the-art methods, hedging against score modulation drift and OOD generalization limits respectively. • State-of-the-art performance on video re-shooting benchmarks across single-image and monocular video settings. 3 Related Work Controllable Diffusion Models. Denoising diffusion probabilistic models [31] and efficient samplers such as DDIM [58] and DPM-Solver [45] have become the dominant generative paradigm for images and video, with latent diffusion models [57] enabling scalable high-resolution synthesis. Training-based controlled generation paradigms including ControlNet [87], IP-Adapter [76], T2I-Adapter [50], and classifier-free guidance [32] require paired training data and considerable computational resources. In contrast, training-free Guidance enable control without parameter update and with fractional cost. Inference-time steering of diffusion models generally falls into three categories: noise/attention manipulation, gradient-based guidance, and particle filtering. Attention-based methods (e.g. [34, 42, 1, 30, 49]) are effective for image and video editing but difficult to adapt to complex geometric 3D constraints. Gradient-based methods (e.g. [pseudoinverse, 16, 81, 5]) utilize auxiliary energy functions to steer the denoising process. DPS [13], for instance injects differentiable likelihoods directly into the reverse process. However, computing gradients through a video diffusion model at every step is computationally expensive and prone to high variance. Our method belongs to the third category, using reward-weighted Sequential Monte Carlo (SMC) to steer the generation without requiring expensive gradient backpropagation. Stochastic Sampling and Restart Mechanisms. Song et al. [59] establish predictor-corrector samplers under the SDE framework, showing that mid-trajectory Langevin corrections reduce deviation from the data manifold. CCDF [15] enforces conditioning via partial forward–reverse noise cycles, and MCG [14] projects corrector steps onto the data manifold. RePaint [47] alternates noising and denoising to harmonize inpainted regions with their surroundings; Restart [73] periodically re-injects noise to escape low-quality basins; ZigZag [4] parameterises the schedule as learnable forward–backward passes. All act as local annealing but lack principled weighting across candidates. Our PF-Restart component formalises this as particle filtering with reward-weighted resampling at each selected timestep, turning ad hoc noise injection into a principled selection procedure. Inference-Time Alignment and Sequential Monte Carlo. A growing body of work frames conditional diffusion as reward alignment via Sequential Monte Carlo (SMC) [22, 19], motivated by the shift toward post-training optimisation [62]. Tweedie’s formula [23] enables reward evaluation throughout the denoising chain without model updates. TDS [70] constructs twisted proposals targeting a reward-weighted posterior; Dou and Song [21] connect DPS to exact Kalman filtering. Kim et al. [39] combine SMC with adaptive resampling to avoid reward over-optimisation, and Kim et al. [38] scale inference-time compute for flow models. Ψ -Sampler [77] shows that Gaussian prior initialisation wastes particle diversity in low-reward regions, and proposes pCNL as a dimension-robust, gradient-informed MCMC sampler to draw initial particles from the reward-aware posterior, yielding consistent gains on layout, counting, and aesthetic tasks. SMC has also been applied to protein design [61] and language decoding [51]. Training-based aligners such as DPOK [25], DDPO [6], and RAFT [20] require parameter updates and are orthogonal to our approach. Our global SMC stage builds on TDS weighting and shares Ψ -Sampler’s motivation of targeting reward-relevant regions early in the chain, extended to the video latent space for camera trajectory alignment. Camera-Controlled Generation. Stand-alone models like NeRF [48] and 3DGS [37] can learn scene-specific 3D representations renderable from any viewpoint given calibrated multi-view images. Dynamic versions can render dynamic scenes [46, 69, 56, 54] by modeling typically a canonical 3D and deformation fields jointly. Data and regularization priors result in versions that are less likely to break under sparse input views [52, 74, 79, 10, 72, 9, 12]. However, reconstruction-based approaches are limited to observed geometry and struggle with large viewpoint extrapolation or disocclusions. Some of these limitations can be mitigated with hybrid methods distilling image or video diffusion priors into explicit 3D representations [64, 2, 44]. Feed-forward multi-view diffusion models synthesize new viewpoints from one or few images [63, 27, 8, 24, 43]. While requiring costly multi-view training data, many remain restricted to static or short-baseline settings with limited temporal consistency. 4 Method We propose a probabilistic control framework for diffusion-based video generation models, designed to steer the generation process toward a desired camera trajectory. Our method operates in the latent space of the diffusion model, maintaining and refining a population of stochastic trajectories. The procedure combines a global Sequential Monte Carlo (SMC) component for trajectory exploration and a local refinement component that merges particle filtering with Restarts [47, 73, 4]. 4.1 Problem Setting Let θD_θ denote a video diffusion model with parameters θ, which defines a stochastic generative process over a sequence of latent states along the sampling trajectory 0:T=(0,…,T)z_0:T=(z_0,…,z_T): pθ(0:T)=p(T)∏t=0T−1pθ(t∣t+1),p_θ(z_0:T)=p(z_T) _t=0^T-1p_θ(z_t _t+1), and the generated video is decoded from the latent 0z_0 as 0=θ(0)x_0=G_θ(z_0). We aim to sample trajectories that follow a target camera path and exhibit desired properties such as background consistency, motion smoothness, high aesthetic quality, quantified by reward functions R(t)=∑i=1αiRi(t),R(z_t)\;=\; _i=1 _iR_i(x_t), where each RiR_i is a user-defined reward measuring alignment between the generated views and the intended camera trajectory or motion constraints, weighted by αi _i. 4.2 Preliminaries: Score Modulation for Camera Control To control video diffusion models at inference time, we adopt score modulation via a latent transformation that adjusts the current latent tz_t based on the input view and the target camera pose. Methods such as NVS-Solver [78] apply this principle by warping the conditioning image to the target viewpoint and injecting the warped features into the denoising update. Throughout the paper, we follow this formulation and treat the score of θD_θ as the modulated score. Please refer to [78] for exhaustive details. A fundamental limitation of Score Modulation methods is that errors made early in sampling compound and become difficult to correct later, even when increasing the number of inference steps. When the early noise realization places sampling on a poor generative trajectory, the model often remains trapped there, resulting in camera drift confirmed by high ATE scores in our experiments (see Table 2). One way to mitigate this problem efficiently is to introduce stochastic perturbations in the form of Restarts [73] during sampling to recover correct camera trajectories by escaping such failure modes. Yet, in practice, unrestricted restarts are unstable and can degrade generation quality. To stabilize this mechanism, we propose to incorporate inference-time alignment [62] and convert the stochastic exploration into a structured filtering procedure. We employ Sequential Monte Carlo (SMC) as an inference-time guidance mechanism. SMC naturally filters trajectory candidates by assigning each particle an importance weight derived from the reward and resampling the population so that high-reward trajectories, i.e. those consistent with the target camera motion and desired visual quality, are preferentially retained. This filtering is applied globally across a population of particles throughout the diffusion process, allowing the sampler to continuously steer generation toward trajectories that remain both geometrically coherent and visually appealing. In parallel, each particle undergoes a local restart procedure: from its latent state tz_t, we draw NrN_r perturbations by injecting forward-diffusion noise and then denoise each one back to time t. These proposals constitute a local candidate set on which we again apply reward-weighted resampling during the backward update. The combination of global population filtering and local restart-based refinement mitigates early drift, avoids collapse into low-quality or geometrically inconsistent trajectories, and yields temporally stable video samples that follow the prescribed motion while maintaining high aesthetic fidelity. Figure 2: One denoising step with Global SMC. 4.3 Global Trajectory Exploration via SMC We aim to sample trajectories from a reward-weighted diffusion path distribution [22, 70]. : p(0:T∣R)∝pθ(0:T)exp(βR(0)),p(z_0:T R) p_θ(z_0:T) \! (β R(x_0) ), (1) where 0:Tz_0:T denotes the latent diffusion trajectory, pθ(0:T)p_θ(z_0:T) is the diffusion prior induced by the reverse process, and R(0)R(x_0) is an aesthetic reward evaluated on the decoded video 0x_0. To approximate this distribution we employ a Sequential Monte Carlo sampler with K particles: (t(k),wt(k))k=1K,\(z_t^(k),w_t^(k))\_k=1^K, which represents a weighted approximation of the intermediate target distributions. Sequential target distributions. Following the SMC framework, we define a sequence of intermediate targets: γt(t:T)∝pθ(t:T)ψt(t), _t(z_t:T) p_θ(z_t:T)\, _t(z_t), (2) where ψt(t) _t(z_t) is a twisting potential representing the expected future reward conditioned on the current latent state: ψt(t)=[exp(βR(0))|t]. _t(z_t)=E\! [ (β R(x_0))\, |\,z_t ]. (3) In practice we approximate this potential by exp(βRt(t)) (β R_t(x_t)), where tx_t is decoded from the Tweedie estimate ^0(t) z_0(z_t).. Propagation. At each diffusion timestep particles are propagated using the reverse diffusion transition: t−1(k)∼pθ(t−1∣t(k)),z_t-1^(k) p_θ(z_t-1 _t^(k)), (4) which corresponds to one denoising update of the diffusion model. Weight update. Particle weights are updated using the incremental SMC importance weight: w~t−1(k)=wt(k)ψt−1(t−1(k))ψt(t(k)). w_t-1^(k)=w_t^(k) _t-1(z_t-1^(k)) _t(z_t^(k)). (5) Because we propose from the reverse transition itself, qt=pθ(t−1∣t)q_t=p_θ(z_t-1 _t), the prior and proposal densities cancel in the twisted incremental weight, leaving only the ratio of consecutive potentials. Using the practical reward approximation, this reduces to the following w~t−1(k)=wt(k)exp(βRt−1(t−1(k))−βRt(t(k))). w_t-1^(k)=w_t^(k) \! (β R_t-1(x_t-1^(k))-β R_t(x_t^(k)) ). (6) Weights are normalized accordingly: wt−1(k)=w~t−1(k)∑jw~t−1(j).w_t-1^(k)= w_t-1^(k) _j w_t-1^(j). (7) Resampling. To avoid particle degeneracy we perform resampling whenever the effective sample size is below a predefined threshold. Particles are then resampled according to their normalized weights. This global SMC filtering step is applied every nsmcn_smc iterations to continuously bias the trajectory population toward high-reward regions while maintaining stochastic diversity across particles. 4.4 Local Refinement via Guided Restarts To further improve local exploration around each trajectory we introduce a refinement step inspired by Restarts [73, 47, 17] and particle filtering. At selected timesteps ref⊆1,…,TT_ref \1,…,T\, each particle spawns NrN_r candidate latents via a short noising–denoising cycle: ~t(k,n)=Denoise(Noise(t(k),σt+1)),n=1,…,Nr. z_t^(k,n)=Denoise\! (Noise(z_t^(k), _t+1) ), n=1,…,N_r. (8) This one step restart results in a diverse set of candidates that form a local proposal distribution around the current particle state. Local evaluation. Each proposal is scored using the reward w~t(k,n)∝exp(βRt(~t(k,n))), w_t^(k,n) \! (β R_t( x_t^(k,n)) ), where ~t(k,n) x_t^(k,n) denotes the decoded frame corresponding to ~t(k,n) z_t^(k,n). The weights are normalized within each particle’s proposal set rt(k,n)=w~t(k,n)∑m=1Nrw~t(k,m).r_t^(k,n)= w_t^(k,n) _m=1^N_r w_t^(k,m). Local resampling. A refined latent state is finally selected as follows: t(k)←~t(k,n∗),n∗∼Cat(rt(k,1),…,rt(k,NrCLOSE)).z_t^(k)← z_t^(k,n ), n (r_t^(k,1),…,r_t^(k,N_r)). (9) This allows to locally explore the latent neighborhood while preserving high-rewards. Since the noising–denoising cycle approximately preserves the diffusion marginal at t [73, 47], resampling the candidates ~t(k,n) z_t^(k,n) according to rt(k,n)r_t^(k,n) leaves the particle approximately distributed as pθ(⋅∣t+1(k))ψt(⋅)p_θ(· _t+1^(k))\, _t(·), i.e. the twisted proposal, at t∈reft _ref. Refinement thus concentrates proposals in high-reward regions, flattening the incremental weights of Eq. 6 and reducing particle degeneracy at fixed K. We keep the weight update unchanged, so that wt(k)w_t^(k) only reflects the selected candidate and not the quality of the local candidate set it was drawn from [26]. For more clarity, we provide an overall algorithm description in the supplementary material. 5 Implementation Details Our method consists of two complementary components: a local refinement module (PF-Restart) and a global exploration module based on Sequential Monte Carlo (SMC). The local refinement step corrects camera trajectories through restart operations combined with reward-guided resampling, while the global component maintains a set of reward-weighted trajectories that are periodically resampled to preserve diversity. For local refinement, we use the camera reward, as this stage is specifically designed to correct the camera trajectory. In contrast, the global SMC exploration relies on the aesthetic reward. As shown in our ablations, this separation provides a favorable trade-off between aesthetic quality and camera accuracy. In practice, we use Nr=2N_r=2 local candidates and K=2K=2 global particles, and run the diffusion model for 100100 iterations. To ensure reliable reward estimates, both local and global refinement are activated starting at iteration 88. Local refinement is applied until iteration 3232, while global SMC continues until iteration 4040. Both global and local updates are performed every 44 iterations. The local refinement step is repeated 4 times at each iteration. These design choices allow us to maintain the overall inference time below that of NVS-Solver while preserving both trajectory accuracy and visual quality. 6 Experiments Datasets We evaluate our method on both static and dynamic scenes following the experimental setting introduced by NVS-Solver[78]. The static scenes consist of six scenes from Tanks and Temples dataset [40] and three additional scenes selected by [78] to cover both outdoor and indoor environments. On the other hand dynamic scenes consist of nine monocular videos capturing both urban and natural settings. Metrics We evaluate our approach in terms of camera accuracy and visual quality. For camera accuracy, we use Particle-SFM [88] to estimate the camera trajectory of the generated videos and compute relative translation (RPE-T) and rotation (RPE-R) errors [28] as well as absolute trajectory error (ATE) [28]. We used FID to evaluate for visual quality of our results. Baselines We compare to state-of-the-art methods including reconstruction ones SparseGS [72], Text2NeRF [86], Def-Gaussian [46], 4DGS [69], in addition to feedforward methods Photo-NVS [80], 3D-aware [71], MotionCtrl [66], NVS-Solver [78], TrajectoryCrafter [82]. Unless stated differently, our method represents our guidance applied to NVS-Solver [78] with Cogvideo [33] backbone at inference time. Additional results can be found in the supplementary material. Figure 3: Qualitative comparison of different methods on NVS from single image of static scenes. We compare to methods NVS-Solver [78], NVS-Solver DGS [78], MotionCtrl [66], Text2NeRF [86]. Table 1: Quantitative comparison of different methods on NVS from single image of static scenes. For all metrics, the lower, the better. Methods Overfitting FID ATE RPE-T RPE-R SparseGS [72] ✓ 369.19 – – – Text2NeRF [86] ✓ 187.05 2.223 0.718 0.107 Photo-NVS [80] × 193.87 7.64 1.19 1.45 3D-aware [71] × 217.19 2.836 1.258 1.662 MotionCtrl [66] × 179.24 3.851 0.705 0.835 NVS-S (DGS) [78] × 166.50 4.533 0.810 0.742 NVS-S (Post) [78] × 165.12 0.767 0.156 0.170 Ours × 121.56 0.526 0.122 0.144 We evaluate the camera control in static scenes by generating novel-views from single image following prescribed camera trajectories. A quantitative comparison is presented in Table 1, while Figure 3 offers a visual comparison to state-of-the-art methods. Our approach, based on reward guidance outperforms other methods in terms of camera accuracy as well as in visual quality. For camera accuracy, our method is significantly better at preserving the global trajectories, as evidenced by ATE errors compared to NVS-Solver. In terms of visual quality, our method is able to generate better structures in the occluded regions, as reflected by the FID results. Figure 1 shows an additional visual comparison in this setting. We illustrate cases where our strategy can improve on top of a training-free method (NVS-Solver), as well as a state-of-the-art training-based method (TrajectoryCrafter) as backbone, both in synthesis quality and input camera trajectory adherence. 6.1 Camera Control in Dynamic Scenes Figure 4: Qualitative comparison of different methods on NVS from monocular videos of dynamic scenes. We compare to methods 4DGS [69], MotionCtrl [66], NVS-Solver [78], NVS-Solver DGS [78]. Table 2: Quantitative comparison of different methods on NVS from monocular videos of dynamic scenes. For all metrics, the lower, the better. Methods Overfitting FID ATE RPE-T RPE-R Def-Gaussian [46] ✓ 115.82 1.813 0.678 0.613 4DGS [69] ✓ 74.34 2.087 0.625 0.825 3D-aware [71] × 159.03 3.100 1.343 1.368 MotionCtrl [66] × 70.35 3.384 1.069 0.653 NVS-S (DGS) [78] × 37.973 2.236 0.691 0.446 NVS-S (Post) [78] × 39.86 2.308 0.725 0.400 Ours × 31.86 0.807 0.061 0.414 We evaluate camera control on dynamic scenes using monocular videos as input while enforcing prescribed camera trajectories. Quantitative results are reported in Table 2, and qualitative comparisons are shown in Figure 4. Our reward-guided sampling strategy consistently improves camera accuracy compared to existing approaches. In particular, methods such as NVS-Solver (training-free) and MotionCtrl (training-based) often struggle to follow the target trajectory in the presence of scene dynamics and occlusions. In contrast, our method achieves substantially lower Absolute Trajectory Error (ATE), as well as lower relative pose errors (RPE-T and RPE-R), indicating more accurate and faithful recovery of the desired camera motion. Beyond trajectory accuracy, our approach also improves the visual quality of the generated videos. By exploring multiple candidate trajectories during sampling and reallocating computation toward high-reward solutions, the method better handles disocclusions and dynamic regions. As a result, the generated videos exhibit sharper structures, more consistent geometry, and fewer motion artifacts, leading to more realistic and temporally coherent renderings. 6.2 Ablation Studies (a) FID (b) ATE (c) RPE Translation (d) RPE Rotation Figure 5: Ablation results of our method on Static Scenes. We use Aesthetic reward for Global SMC and Camera reward for local refinement. The number of local candidates is denoted as NrN_r while the number of global particles is referred to as K. Table 3: Ablation study of our method with respect to the choice of the backbone video model for NVS from monocular videos of dynamic scenes. For all metrics, the lower, the better. Methods FID ATE RPE-T RPE-R NVS-S (CogVideo) [78] 41.08 0.981 0.128 0.431 Ours (CogVideo) 31.86 0.807 0.061 0.414 NVS-S (SVD) [78] 39.86 2.308 0.725 0.400 Ours (SVD) 38.26 1.912 0.521 0.489 TrajectoryCrafter [82] 29.86 0.767 0.095 0.460 Ours (TrajectoryCrafter) 30.01 0.712 0.070 0.428 We conduct an ablative analysis on the static scenes from the NVS-Solver benchmark [78] using right-sweep trajectories, evaluating each component of our method using ATE, RPE-T, RPE-R, and FID. Results are summarised in Figure 5. Effect of restart sampling Starting from the NVS-Solver baseline (ATE: 1.56, RPE-T: 0.52, RPE-R: 0.27, FID: 133.96), introducing a simple restart without reward guidance already yields meaningful improvements across all metrics (ATE: 1.09, RPE-T: 0.27, RPE-R: 0.24, FID: 117.07). However, as evidenced by the wide interquartile ranges in Figure 5, plain restart suffers from high variance across scenes: while some scenes benefit substantially, others show little or no improvement. This indicates that without any reward signal to guide resampling, noise injection makes the generation process unstable. Effect of reward-guided local refinement (PF-Restart) Incorporating reward-guided resampling into the restart procedure (PF-Restart) significantly tightens this variance while further improving trajectory accuracy. With Nr=2N_r=2 local candidates, PF-Restart reduces ATE to 0.76 and RPE-T to 0.17 . Increasing to Nr=4N_r=4 pushes these to 0.54 and 0.14 respectively, while maintaining FID at 114.15 in both settings. Compared to unguided restart, the reward-guided resampling yields a notably more compact distribution of per-scene errors, confirming that reward alignment stabilizes the local search. Effect of global SMC exploration Global Sequential Monte Carlo sampling alone (K=4) reduces ATE to 1.03 and RPE-R to 0.19, but fails to achieve satisfactory performance on translation accuracy (RPE-T: 0.31 ) and actually degrades FID relative to NVS-Solver (135.84 vs. 133.96). This counterintuitive finding highlights a fundamental limitation of population-level exploration without local correction: while global diversity prevents early collapse into poor basins, it does not provide the fine-grained per-particle guidance needed to correct individual trajectory errors. This result confirms that global SMC alone is insufficient. Complementarity of global and local components Combining global SMC with local PF-Restart, our full method achieves the best overall performance and validates the complementarity of the two mechanisms. With K=2K=2 and Nr=2N_r=2, using the aesthetic reward for global exploration and the camera reward for local refinement, the joint method reaches RPE-T: 0.09 , RPE-R: 0.09, ATE: 0.37, and FID: 103.82, which is a 6× reduction in translation error and 4× reduction in ATE relative to NVS-Solver, while simultaneously improving visual quality. Increasing NrN_r to 4 does not monotonically help: ATE slightly degrades from 0.37 to 0.46 and RPE-R worsens from 0.09° to 0.14°, suggesting that local restarting with more particle budget can undermine global trajectory consistency when the control signal is not strong enough. The sweet spot in our setting is K=2K=2,Nr=2N_r=2. Effect of reward design A crucial choice for our method is the choice of the reward function we use to guide the sampling. We experimented in the paper with camera error using VGGT [65] to estimate trajectories in addition to aesthetic reward used in [36]. In our default configuration, the global stage uses the aesthetic reward to preserve visual diversity across particles, while the local stage uses the camera reward to correct geometric drift. We additionally evaluate a variant where the camera reward drives both stages ( K=2K=2,Nr=4N_r=4). Replacing the aesthetic reward with the camera reward in the global SMC yields the best absolute ATE of 0.28: a further 24% improvement over the aesthetic-global variant at the same NrN_r and also improves RPE-R from 0.14 to 0.11 relative to the aesthetic variant. This comes at a minor cost in visual quality (FID: 103.06 vs. 102.51) and a slight regression in RPE-R relative to the K=2K=2,Nr=2N_r=2 default (0.11 vs. 0.09). These results reveal a clear trade-off: using the camera reward globally sharpens trajectory adherence but reduces the diversity of particle proposals. The default separation with aesthetic reward globally and camera reward locally provides a good balance between geometric accuracy and visual quality, and is the configuration adopted throughout our main experiments. Similarly, we can see that for SCG using the camera reward significantly improves the camera metrics (ATE: 1.49 vs 1.25). For PF-Restart, we noted that they both perform equally well for scenes without large occlusion as in this case the quality of the video is enough to guide the model to generate meaningful content in the occluded regions. However when the motion is ambiguous due to large occlusions, Camera reward shows significantly better results improving the Absolute Trajectory Error (ATE) from 1.00 to 0.71. Comparison to other possible guidance methods These methods differ in how they estimate reward, how they sample candidate trajectories, and how they select the best samples. We seek a method that can quickly explore the space of diffusion trajectories and accurately select the best ones. TDS [70] and Ψ -Sampler [77] are the closest to our method but they require differentiating through the reward function and to backpropagate through the diffusion model which is impractical for reward functions like the camera reward and recent video diffusion models like Cog-Video or Wan. On the other hand, SCG [35] has the advantage of maintaining one global particle and sampling candidates using the reverse SDE which results in less diverse samples throughout the iterations. With aesthetic reward it performs the worst of all evaluated configurations on trajectory metrics (RPE-T: 0.71, ATE: 1.49), confirming that a non-geometric reward is insufficient to guide camera-accurate generation under this sampler. SCG with camera reward recovers partially (RPE-T: 0.25, ATE: 1.25) but still falls far behind our full method. Consequently, beyond the choice of the reward, our approach lies in the bi-level SMC framework itself: global exploration and local reward-guided refinement together achieve what neither component nor SCG can accomplish alone. Backbone Our proposed method can be applied to any latent video diffusion model. Throughout the paper, we primarily used CogVideo [33]. To evaluate the generality of our approach, we perform an ablation study by applying our method to SVD [7] and TrajectoryCrafter [82]. As shown in Table 3, our method consistently improves upon NVS-Solver both in the cases where the video model backbone is SVD and CogVideo. This result highlights the effectiveness of our Camera Control strategy in mitigating the weaknesses of score modulation. On the other hand, TrajectoryCrafter, which is a finetuned version of CogVideo specifically designed for camera control, already achieves very competitive performance even without additional guidance. Interestingly, when applying our method to TrajectoryCrafter, we observe further improvements in camera accuracy while maintaining comparable visual quality. 6.3 Mitigating limitations of TrajectoryCrafter[82] Training based camera controlled video generation methods can display visual artefacts, inconsistencies and misalignment with the input camera trajectory, which can be framed as generalization issues. We show here examples where our method can help recover from such failures. We challenge the state-of-the-art training based method TrajectoryCrafter[82] with harder camera trajectories at test time, following the camera trajectory sampling in [78]. Figure 7 shows our inference-time improvement over this method using some of the demo videos of TrajectoryCrafter[82]. 6.4 Mitigating limitations of ReCamMaster[3] Training based method ReCamMaster[3] displays visual artifacts especially under challenging camera trajectories. Figure 6 shows our inference-time improvement over this method. Figure 6: Qualitative comparison in monocular video reshooting of dynamic scenes to ReCamMaster [3]. Figure 7: Qualitative comparison in monocular video reshooting of dynamic scenes to TrajectoryCrafter [82]. We We show a two views of the estimated camera trajectories from the videos generated with our method, TrajectoryCrafter and the Ground-truth. 7 Limitations Our method can require a larger denoising budget due to maintaining multiple candidate trajectories. Conversely, reward guidance can also accelerate convergence by steering sampling toward better solutions early. Overall, the approach represents a reasonable trade-off when generation quality and adherence to the control signal are prioritized. Additionally, the effectiveness of the framework depends on the design of the reward function, which may require task-specific tuning. 8 Conclusion We presented PartiCam, a training-free framework for improving camera trajectory control in diffusion-based video generation. Our approach combines global Sequential Monte Carlo (SMC) trajectory exploration with local particle-filtered restarts (PF–Restart), enabling reward-guided sampling that balances exploration and refinement during the denoising process. This formulation mitigates common failure modes of existing approaches, including early trajectory drift in score-modulated sampling and limited robustness to OOD camera motions. By maintaining a population of candidate trajectories and reallocating sampling effort toward high-reward solutions, our method produces videos that better adhere to the desired camera trajectory while maintaining visual quality and temporal coherence. Beyond camera control, the proposed framework provides a general mechanism for reward-guided steering of diffusion processes without requiring gradient backpropagation or model retraining. We believe this population-based inference strategy may extend naturally to other controlled generation tasks and forms a promising direction for future work. References [1] Y. Alaluf, D. Garibi, O. Patashnik, H. Averbuch-Elor, and D. Cohen-Or (2024) Cross-image attention for zero-shot appearance transfer. In ACM SIGGRAPH 2024 conference papers, p. 1–12. Cited by: §3. [2] S. Bahmani, T. Shen, J. Ren, J. Huang, Y. Jiang, H. Turki, A. Tagliasacchi, D. B. Lindell, Z. Gojcic, S. Fidler, H. Ling, J. Gao, and X. Ren (2025) Lyra: generative 3d scene reconstruction via video diffusion model self‑distillation. arXiv preprint arXiv:2509.19296. Cited by: §3. [3] J. Bai, M. Xia, X. Fu, X. Wang, L. Mu, J. Cao, Z. Liu, H. Hu, X. Bai, P. Wan, and D. Zhang (2025) ReCamMaster: camera-controlled generative rendering from a single video. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Note: Best Paper Finalist Cited by: Figure 6, Figure 6, §6.4, §6.4. [4] L. Bai, S. Shao, Z. Zhou, Z. Qi, Z. Xu, H. Xiong, and Z. Xie (2024) Zigzag diffusion sampling: diffusion models can self-improve via self-reflection. External Links: 2412.10891, Link Cited by: §3, §4. [5] A. Bansal, H. Chu, A. Schwarzschild, R. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein (2024) Universal guidance for diffusion models. In International Conference on Learning Representations (ICLR), Cited by: §3. [6] K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine (2024) Training diffusion models with reinforcement learning. In International Conference on Learning Representations (ICLR), Cited by: §3. [7] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, and V. Jampani (2023) Stable video diffusion: scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127. Cited by: Figure 1, Figure 1, §2, §6.2. [8] C. Cao, C. Yu, S. Liu, F. Wang, X. Xue, and Y. Fu (2025) MVGenMaster: scaling multi‑view generation from any image via 3d priors enhanced diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6045–6056. Cited by: §3. [9] D. Charatan, S. L. Li, A. Tagliasacchi, and V. Sitzmann (2024) PixelSplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 19457–19467. Cited by: §3. [10] A. Chen, Z. Xu, F. Zhao, X. Zhang, F. Xiang, J. Yu, and H. Su (2021) MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), p. 14104–14113. External Links: Document Cited by: §3. [11] S. Chen, C. Ge, Y. Zhang, et al. (2025) Goku: flow based video generative foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 23516–23527. Cited by: §2. [12] Y. Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T. Cham, and J. Cai (2024) Mvsplat: efficient 3d gaussian splatting from sparse multi-view images. In European conference on computer vision, p. 370–386. Cited by: §3. [13] H. Chung, J. Kim, M. T. McCann, M. L. Klasky, and J. C. Ye (2023) Diffusion posterior sampling for general noisy inverse problems. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §3. [14] H. Chung, B. Sim, D. Ryu, and J. C. Ye (2022) Improving Diffusion Models for Inverse Problems Using Manifold Constraints. In Advances in Neural Information Processing Systems 35, p. 25683–25696. External Links: Document Cited by: §3. [15] H. Chung, B. Sim, and J. C. Ye (2022) Come-closer-diffuse-faster: accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §3. [16] G. Daras, H. Chung, C. Lai, Y. Mitsufuji, J. C. Ye, P. Milanfar, A. G. Dimakis, and M. Delbracio (2024) A survey on diffusion models for inverse problems. External Links: 2410.00083, Link Cited by: §3. [17] B. De Bruyne and F. Mori (2023) Resetting in stochastic optimal control. Physical Review Research 5 (1), p. 013122. Cited by: §4.4. [18] G. DeepMind (2025) Veo 3: video generation with native audio by google deepmind. Note: https://deepmind.google/en/models/veo/Accessed: 2025-11-13 Cited by: §2. [19] P. Del Moral, A. Doucet, and A. Jasra (2006) Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology 68 (3), p. 411–436. Cited by: §3. [20] H. Dong, W. Xiong, D. Goyal, R. Pan, S. Diao, J. Zhang, K. Shum, and T. Zhang (2023) RAFT: reward ranked finetuning for generative foundation model alignment. External Links: 2304.06767 Cited by: §3. [21] Z. Dou and Y. Song (2024) Diffusion posterior sampling for linear inverse problem solving: a filtering perspective. In International Conference on Learning Representations (ICLR), Cited by: §3. [22] A. Doucet, N. De Freitas, N. J. Gordon, et al. (2001) Sequential monte carlo methods in practice. Springer. Cited by: §3, §4.3. [23] B. Efron (2011) Tweedie’s formula and selection bias. Journal of the American Statistical Association 106 (496), p. 1602–1614. Cited by: §3. [24] N. Elata, B. Kawar, Y. Ostrovsky‑Berman, M. Farber, and R. Sokolovsky (2025) Novel view synthesis with pixel‑space diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 26756–26766. Cited by: §3. [25] Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee (2023) DPOK: reinforcement learning for fine-tuning text-to-image diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §3. [26] P. Fearnhead, O. Papaspiliopoulos, and G. O. Roberts (2008) Particle filters for partially observed diffusions. Journal of the Royal Statistical Society: Series B 70 (4), p. 755–777. Cited by: §4.4. [27] R. Gao, A. Hołyński, P. Henzler, A. Brussee, R. Martin‑Brualla, P. P. Srinivasan, J. T. Barron, and B. Poole (2024) CAT3D: create anything in 3d with multi‑view diffusion models. Advances in Neural Information Processing Systems. Cited by: §3. [28] P. Goel, S. I. Roumeliotis, and G. S. Sukhatme (1999) Robust localization using relative and absolute position estimates. In Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No. 99CH36289), Vol. 2, p. 1134–1140. Cited by: §6. [29] H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang (2024) CameraCtrl: enabling camera control for text-to-video generation. External Links: 2404.02101 Cited by: §2, §2. [30] A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or (2023) Prompt-to-prompt image editing with cross attention control. In International Conference on Learning Representations (ICLR), Cited by: §3. [31] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §3. [32] J. Ho and T. Salimans (2021) Classifier-free diffusion guidance. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, Cited by: §3. [33] W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang (2023) CogVideo: large‑scale pretraining for text‑to‑video generation via transformers. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: Figure 1, Figure 1, §11, §2, §6, §6.2. [34] L. Hua, F. Liu, J. Su, X. Miao, Z. Ouyang, Z. Wang, R. Hu, Z. Wen, B. Zhai, Y. Long, et al. (2025) Attention in diffusion model: a survey. arXiv preprint arXiv:2504.03738. Cited by: §3. [35] Y. Huang, A. Ghatare, Y. Liu, Z. Hu, Q. Zhang, C. S. Sastry, S. Gururani, S. Oore, and Y. Yue (2024) Symbolic music generation with non-differentiable rule guided diffusion. arXiv preprint arXiv:2402.14285. Cited by: §6.2. [36] Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2024) VBench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §6.2. [37] B. Kerbl, G. Kopanas, T. Leimkuehler, and G. Drettakis (2023) 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics 42 (4), p. 1–14. External Links: Document Cited by: §2, §3. [38] J. Kim, T. Yoon, J. Hwang, and M. Sung (2025) Inference-time scaling for flow models via stochastic generation and rollover budget forcing. External Links: 2503.19385 Cited by: §3. [39] S. Kim, M. Kim, and D. Park (2025) Test-time alignment of diffusion models without reward over-optimization. In International Conference on Learning Representations (ICLR), Cited by: §3. [40] A. Knapitsch, J. Park, Q. Zhou, and V. Koltun (2017) Tanks and temples: benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG) 36 (4), p. 1–13. Cited by: §6. [41] W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, K. Wu, Q. Lin, J. Yuan, Y. Long, A. Wang, A. Wang, C. Li, D. Huang, F. Yang, H. Tan, H. Wang, J. Song, J. Bai, J. Wu, J. Xue, J. Wang, K. Wang, M. Liu, P. Li, S. Li, W. Wang, W. Yu, X. Deng, Y. Li, Y. Chen, Y. Cui, Y. Peng, Z. Yu, Z. He, Z. Xu, Z. Zhou, Z. Xu, Y. Tao, Q. Lu, S. Liu, D. Zhou, H. Wang, Y. Yang, D. Wang, Y. Liu, J. Jiang, and C. Zhong (2024) HunyuanVideo: a systematic framework for large video generative models. ArXiv abs/2412.03603. External Links: Link Cited by: §2. [42] J. Koo, P. Guerrero, C. P. Huang, D. Ceylan, and M. Sung (2025) Videohandles: editing 3d object compositions in videos using video generative priors. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 17692–17701. Cited by: §3. [43] J. Kwak, E. Dong, Y. Jin, H. Ko, S. Mahajan, and K. M. Yi (2024) ViVid‑1‑to‑3: novel view synthesis with video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6775–6785. Cited by: §3. [44] C. Lin, P. Pan, B. Yang, Z. Li, and Y. Mu (2025) DiffSplat: repurposing image diffusion models for scalable gaussian splat generation. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §3. [45] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu (2022) DPM-Solver: a fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §3. [46] Z. Lu, X. Guo, L. Hui, T. Chen, M. Yang, X. Tang, F. Zhu, and Y. Dai (2024) 3D geometry-aware deformable gaussian splatting for dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 8900–8910. Cited by: §3, §6, Table 2. [47] A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. V. Gool (2022) RePaint: inpainting using denoising diffusion probabilistic models. External Links: 2201.09865, Link Cited by: §3, §4.4, §4.4, §4. [48] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng (2020) NeRF: representing scenes as neural radiance fields for view synthesis. In Proceedings of the European Conference on Computer Vision (ECCV), p. 478–495. Cited by: §2, §3. [49] R. Mokady, A. Hertz, K. Aberman, Y. Pritch, and D. Cohen-Or (2023) NULL-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6038–6047. Cited by: §3. [50] C. Mou, X. Wang, L. Xie, Y. Wu, J. Zhang, Z. Qi, and Y. Shan (2024) T2I-Adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §3. [51] S. Mudgal, J. Lee, H. Ganapathy, Y. Li, T. Wang, Y. Huang, Z. Chen, H. Cheng, M. Collins, T. Strohman, et al. (2023) Controlled decoding from language models. External Links: 2310.17022 Cited by: §3. [52] M. Niemeyer, J. T. Barron, B. Mildenhall, M. S. M. Sajjadi, A. Geiger, and N. Radwan (2022) RegNeRF: regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 5470–5480. Cited by: §3. [53] OpenAI (2024) Sora: text‑to‑video generation by openai. Note: https://openai.com/research/soraAccessed: 2025‑11‑13 Cited by: §2. [54] K. Park, U. Sinha, P. Hedman, J. T. Barron, S. Bouaziz, D. B. Goldman, R. Martin-Brualla, and S. M. Seitz (2021) HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields. ACM Trans. Graph. 40 (6). External Links: ISSN 0730-0301, Link, Document Cited by: §3. [55] M. Platforms (2024) Movie gen: a cast of media foundation models for video, audio, and image generation. Note: https://ai.meta.com/research/moviegenAccessed: 2025‑11‑13 Cited by: §2. [56] A. Pumarola, E. Corona, G. Pons-Moll, and F. Moreno-Noguer (2021) D-NeRF: Neural Radiance Fields for Dynamic Scenes. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10313–10322. External Links: Document Cited by: §3. [57] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §3. [58] J. Song, C. Meng, and S. Ermon (2021) Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), Cited by: §3. [59] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations (ICLR), Cited by: §3. [60] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom (2023) Llama 2: open foundation and fine-tuned chat models. External Links: 2307.09288, Link Cited by: §2. [61] B. L. Trippe, J. Yim, D. Tischer, D. Baker, T. Broderick, R. Barzilay, and T. S. Jaakkola (2023) Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem. In International Conference on Learning Representations (ICLR), Cited by: §3. [62] M. Uehara, Y. Zhao, C. Wang, X. Li, A. Regev, S. Levine, and T. Biancalani (2025) Inference-time alignment in diffusion models with reward-guided generation: tutorial and review. arXiv preprint arXiv:2501.09685. Cited by: §3, §4.2. [63] V. Voleti, C. Yao, M. Boss, A. Letts, D. Pankratz, D. Tochilkin, C. Laforte, R. Rombach, and V. Jampani (2024) SV3D: novel multi-view synthesis and 3D generation from a single image using latent video diffusion. In European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, Vol. 15059, p. 439–457. External Links: Document Cited by: §3. [64] H. Wang, F. Liu, J. Chi, and Y. Duan (2025) VideoScene: distilling video diffusion model to generate 3d scenes in one step. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 16475–16485. Cited by: §3. [65] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny (2025) VGGT: visual geometry grounded transformer. External Links: 2503.11651, Link Cited by: §6.2. [66] Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P. Luo, and Y. Shan (2024) MotionCtrl: a unified and flexible motion controller for video generation. In ACM SIGGRAPH Conference Proceedings, Cited by: Figure 8, Figure 8, Figure 9, Figure 9, Figure 3, Figure 3, Figure 4, Figure 4, §6, Table 1, Table 2. [67] WanTeam, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, and Y. Feng (2025) Wan: open and advanced large‑scale video generative models. External Links: 2503.20314, Link Cited by: §2. [68] T. Wiedemer, Y. Li, P. Vicol, et al. (2025) Video models are zero-shot learners and reasoners. arXiv preprint arXiv:2509.20328. Cited by: §2. [69] G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang (2024) 4D gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 20310–20320. Cited by: Figure 8, Figure 8, Figure 9, Figure 9, §3, Figure 4, Figure 4, §6, Table 2. [70] L. Wu, B. L. Trippe, C. A. Naesseth, D. M. Blei, and J. P. Cunningham (2024) Practical and asymptotically exact conditional sampling in diffusion models. External Links: 2306.17775, Link Cited by: §3, §4.3, §6.2. [71] J. Xiang, J. Yang, B. Huang, and X. Tong (2023) 3D‑aware image generation using 2d diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 2383–2393. Cited by: §6, Table 1, Table 2. [72] H. Xiong, S. Muttukuru, R. Upadhyay, P. Chari, and A. Kadambi (2023) SparseGS: real‑time 360° sparse view synthesis using gaussian splatting. arXiv preprint arXiv:2312.00206. Cited by: §3, §6, Table 1. [73] Y. Xu, M. Deng, X. Cheng, Y. Tian, Z. Liu, and T. Jaakkola (2023) Restart sampling for improving generative processes. External Links: 2306.14878, Link Cited by: §2, §3, §4.2, §4.4, §4.4, §4. [74] J. Yang, M. Pavone, and Y. Wang (2023) FreeNeRF: improving few‑shot neural rendering with free frequency regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §3. [75] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, et al. (2024) CogVideoX: text‑to‑video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072. Cited by: §2. [76] H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang (2023) IP-adapter: text compatible image prompt adapter for text-to-image diffusion models. ArXiv abs/2308.06721. External Links: Link Cited by: §3. [77] T. Yoon, Y. Min, K. Yeo, and M. Sung (2025) Ψ -Sampler: initial particle sampling for SMC-based inference-time reward alignment in score models. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2506.01320 Cited by: §3, §6.2. [78] M. You, Z. Zhu, H. Liu, and J. Hou (2025) NVS-solver: video diffusion model as zero-shot novel view synthesizer. In International Conference on Learning Representations (ICLR), Cited by: Figure 1, Figure 1, Figure 8, Figure 8, Figure 9, Figure 9, §11, §2, §2, §4.2, Figure 3, Figure 3, Figure 4, Figure 4, §6, §6, §6.2, §6.3, Table 1, Table 1, Table 2, Table 2, Table 3, Table 3. [79] A. Yu, V. Ye, M. Tancik, and A. Kanazawa (2021) PixelNeRF: neural radiance fields from one or few images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4578–4587. Cited by: §3. [80] J. J. Yu, F. Forghani, K. G. Derpanis, and M. A. Brubaker (2023) Long‑term photometric consistent novel view synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 7094–7104. Cited by: §6, Table 1. [81] J. Yu, Y. Wang, C. Zhao, B. Ghanem, and J. Zhang (2023) FreeDoM: training-free energy-guided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §3. [82] M. Yu, W. Hu, J. Xing, and Y. Shan (2025) TrajectoryCrafter: redirecting camera trajectory for monocular videos via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Note: Oral Cited by: Figure 1, Figure 1, §2, §2, §2, Figure 7, Figure 7, §6, §6.2, §6.3, §6.3, Table 3. [83] S. Yu, M. Hahn, D. Kondratyuk, et al. (2025) MALT diffusion: memory-augmented latent transformers for any-length video generation. External Links: 2502.12632 Cited by: §2. [84] Z. Yuan, H. Yuan, C. Li, G. Dong, K. Lu, C. Tan, C. Zhou, and J. Zhou (2023) Scaling relationship on learning mathematical reasoning with large language models. External Links: 2308.01825, Link Cited by: §2. [85] E. Zelikman, Y. Wu, J. Mu, and N. Goodman (2022) STaR: bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 15476–15488. External Links: Document, Link Cited by: §2. [86] J. Zhang, X. Li, Z. Wan, C. Wang, and J. Liao (2023) Text2NeRF: text‑driven 3d scene generation with neural radiance fields. arXiv preprint arXiv:2305.11588. Cited by: Figure 3, Figure 3, §6, Table 1. [87] L. Zhang, A. Rao, and M. Agrawala (2023) Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §3. [88] W. Zhao, S. Liu, H. Guo, W. Wang, and Y. Liu (2022) Particlesfm: exploiting dense point trajectories for localizing moving cameras in the wild. In European Conference on Computer Vision, p. 523–542. Cited by: §6. 9 Supplementary Material 10 Algorithm Algorithm 1 summarizes our proposed method. Algorithm 1 SMC Diffusion with Local Guided Restarts 1: diffusion model pθp_θ, reward R, particles K, local proposals NrN_r, refinement steps refT_ref, NpfN_pf repetitions of local refinement. 2: Initialize T(k)∼(0,I)z_T^(k) (0,I), wT(k)=1/Kw_T^(k)=1/K 3: for t=T,…,1t=T,…,1 do 4: if t∈reft _ref then 5: for k=1,…,Npfk=1,…,N_pf do 6: for k=1,…,Kk=1,…,K do 7: Generate ~t(k,n)=Denoise(Noise(t(k),σt+1)) z_t^(k,n)=Denoise(Noise(z_t^(k), _t+1)), n=1…Nrn=1… N_r 8: Score rt(k,n)∝exp(βRt(~t(k,n)))r_t^(k,n) (β R_t( x_t^(k,n))) 9: Sample t(k)←~t(k,n∗)z_t^(k)← z_t^(k,n ), n∗∼Cat(rt(k,:))n (r_t^(k,:)) 10: end for 11: end for 12: end if 13: for k=1,…,Kk=1,…,K do 14: Sample t−1(k)∼pθ(t−1∣t(k))z_t-1^(k) p_θ(z_t-1 _t^(k)) 15: Update w~t−1(k)←wt(k)exp(βRt−1(t−1(k))−βRt(t(k))) w_t-1^(k)← w_t^(k) (β R_t-1(x_t-1^(k))-β R_t(x_t^(k))) 16: end for 17: Normalize weights wt−1(k)∝w~t−1(k)w_t-1^(k) w_t-1^(k); resample if ESS below threshold 18: end for 19: return 0(k)k=1K\x_0^(k)\_k=1^K 11 Qualitative results We provide additional qualitative results and comparisons in this section. Figures 8 and 9 complement Figure 4 in the main paper. While the main paper shows only a single-frame comparison with our method building on CogVideo [33] based NVS-Solver [78], here we present additional frames for each method. Figure 8: Qualitative comparison of different methods on monocular video reshooting of dynamic scenes. We compare to methods 4DGS [69], MotionCtrl [66], NVS-Solver [78], NVS-Solver DGS [78]. Figure 9: Qualitative comparison of different methods on monocular video re-shooting of dynamic scenes. We compare to methods 4DGS [69], MotionCtrl [66], NVS-Solver [78], NVS-Solver DGS [78].