Paper deep dive
GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior
Yajiao Xiong, Youyu Luan, Xiaoyu Zhou, Yongtao Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Single-image novel view synthesis remains challenging because the underlying 3D geometry is highly ambiguous. Recent diffusion-based approaches produce plausible results, but they often struggle to preserve the geometric structure and spatial coherence of foreground objects. We present GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. Specifically, GenNVS models foreground objects and the background with 3D Gaussian Splatting and aligns them through a coarse-to-fine geometric optimization process to form a unified 3D scene. This scene conditions a video diffusion model through the proposed Dual-Stream Masking mechanism, which guides synthesis by jointly exploiting rendered validity masks and geometry-aware warping. Experimental results show that GenNVS performs favorably against recent methods in both visual quality and geometric accuracy, while naturally supporting flexible scene editing.
Tags
Links
- Source: https://arxiv.org/abs/2609.34579v1
- Canonical: https://arxiv.org/abs/2609.34579v1
Trouble viewing inline? Open PDF directly →
Full Text
49,744 characters extracted from source content.
Expand or collapse full text
GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior Yajiao Xiong Affiliation: Peking University Email: asdlkj@stu.pku.edu.cn Youyu Luan Affiliation: Peking University Email: krm@stu.pku.edu.cn Yongtao Wang22footnotemark: 2 Affiliation: Peking University Email: wyt@pku.edu.cn Xiaoyu Zhou Affiliation: Peking University Email: xyrain.zhou@gmail.com Abstract Single-image novel view synthesis remains challenging because the underlying 3D geometry is highly ambiguous. Recent diffusion-based approaches produce plausible results, but they often struggle to preserve the geometric structure and spatial coherence of foreground objects. We present GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. Specifically, GenNVS models foreground objects and the background with 3D Gaussian Splatting and aligns them through a coarse-to-fine geometric optimization process to form a unified 3D scene. This scene conditions a video diffusion model through the proposed Dual-Stream Masking mechanism, which guides synthesis by jointly exploiting rendered validity masks and geometry-aware warping. Experimental results show that GenNVS performs favorably against recent methods in both visual quality and geometric accuracy, while naturally supporting flexible scene editing. Novel View Synthesis with Geometry-Enhanced Prior. We synthesize novel views from a single image (left) by constructing a disentangled 3D prior (middle). This prior conditions a video diffusion model to generate a long-range, geometrically consistent view sequence (right). The output demonstrates the effectiveness of our prior-based pipeline. 11footnotetext: Corresponding author. 1 Introduction Novel view synthesis (NVS) from a single image is a core problem in computer vision and graphics, with applications ranging from augmented reality to digital content creation. Despite significant progress in 3D scene representations, such as Neural Radiance Fields [22] and 3D Gaussian Splatting (3DGS) [15], existing methods typically rely on dense multi-view observations and struggle to extrapolate to novel viewpoints from a single observation. Recent advances in diffusion models [11] suggest a new avenue for single-image NVS, with approaches [1] leveraging video diffusion prior to synthesize plausible camera motion from sparse inputs. However, without explicit 3D prior, these models often generate views with geometric distortions and temporal inconsistencies, underscoring the difficulty of achieving high-fidelity, consistent NVS from a single image. Effective NVS critically depends on strong 3D geometric prior. Prior approaches generally follow two paradigms. The first encodes camera poses as high-level embeddings to condition diffusion models [10, 1]. While simple, this implicit conditioning provides limited geometric supervision and can fail under out-of-distribution settings or complex trajectories. The second reconstructs an explicit 3D representation (e.g., point clouds or 3DGS) from the input image and uses it as a structural prior [32, 24]. Although this introduces stronger 3D awareness, single-image reconstruction remains fundamentally ambiguous and often yields incomplete or distorted geometry, which then propagates errors into the synthesized views. Existing methods typically use point clouds or 3DGS reconstructed from a single image as geometric prior, treating the scene holistically. This introduces two challenges. First, single-image reconstruction often yields unreliable geometry, which weakens the 3D prior and shifts the burden to the generative model to correct structural errors. Second, reconstructed scenes frequently contain large holes in unseen regions, relying on the generative model to inpaint missing content and often leading to geometric distortions. Figure 1: Geometric prior from entangled scene modeling [32] and our disentangled formulation. Motivated by the geometric fidelity of recent object-level 3D generative methods [7], we revisit single-image NVS from the perspective of where reliable geometry can be obtained. Salient objects (foreground) often require more faithful, object-centric reconstruction, while the remaining regions (background) mainly require coherent global layout. This motivates a foreground-background disentangled formulation: we separately recover object geometry and scene layout, and then merge them into a unified 3D scene prior that offers both geometric accuracy and structural coverage, providing stronger guidance for view synthesis and improving geometric consistency. Figure 1 contrasts prior from foreground-background entangled modeling [32] with our disentangled formulation, which yields a more complete and reliable 3D prior for novel view synthesis. Building on this insight, we introduce GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. It separately reconstructs foreground objects and the background, and fuses them into a coherent scene representation that provides stronger geometry guidance to a video diffusion model. Crucially, we additionally provide visibility/validity cues from the rendered prior during conditioning, encouraging view-consistent generation and reducing geometric artifacts. The disentangled representation also naturally enables flexible editing by modifying foreground objects while maintaining consistent novel views. The main contributions of this work are: • A geometry-enhanced NVS framework, GenNVS, that leverages a disentangled 3D prior by aligning foreground and background into a coherent 3D scene. • Foreground Gaussian Alignment (FGA), a coarse-to-fine alignment module combining GS-ICP registration with depth-aware optimization. • A Dual-Stream Masking mechanism for video diffusion models that leverages the rendered geometric prior to encourage 3D-consistent novel view synthesis. • Empirical validation on standard benchmarks, showing favorable performance in foreground geometry consistency and naturally supporting flexible scene editing. 2 Related Work Figure 2: Overview of GenNVS. Given a single input image, our framework first generates separate 3DGS for foreground objects and background. These are aligned into a unified 3D scene through coarse-to-fine geometric optimization. The aligned prior then conditions a video diffusion model via a Dual-Stream Masking mechanism, yielding high-fidelity and geometrically consistent novel view sequences. Feed-Forward Novel View Synthesis. Feed-forward models have reshaped scene reconstruction by enabling novel view synthesis from sparse inputs without test-time optimization. These approaches typically encode an input image into a latent representation, lift it into 3D space, and decode it in a single forward pass to produce target views or explicit 3D scene representations. One prominent line of work focuses on direct reconstruction from limited observations. Methods such as LRM [13], PixelSplat [6], and MVSplat [8] regress scene representations in a single forward pass. LRM [13], for example, employs a large-scale Transformer to map images into triplane features. Approaches such as VGGT [28] reconstruct point clouds using vision transformers trained on large-scale datasets, while Anysplat [14] and HunyuanWorld-Mirror [20] further extend this paradigm by directly predicting 3D Gaussian Splatting (3DGS) representations. While highly efficient, these methods effectively operate as deterministic interpolators, limiting their ability to reason about unobserved regions and often leading to severe artifacts under occlusion or large camera motion. To overcome these limitations, another line of work introduces generative prior from visual foundation models to hallucinate unseen content. Representative methods include SA3D [5], SAM3D [7], and SplatFlow [9], with SAM3D [7] demonstrating 3D object generation from a single image. Despite their strong generalization capability, these approaches lack explicit global geometric constraints, frequently resulting in multi-view inconsistencies and fragmented scene-level structure. Diffusion-Based Novel View Synthesis. Recent advances in novel view synthesis increasingly leverage pre-trained diffusion models for their powerful generative prior, particularly for plausibly hallucinating unobserved regions. Early methods build upon text-to-image diffusion models, such as Zero-1-to-3 [19], which condition generation on a source image and relative camera pose using specialized attention mechanisms. More recent approaches cast novel view synthesis as conditional video generation [31, 10, 1], better exploiting temporal coherence for continuous camera trajectories. These methods can be broadly categorized based on how geometric information is incorporated. Implicit Camera Encoding. This category conditions video diffusion models directly on numerical camera parameters to steer viewpoint trajectories. For instance, ReCamMaster [1] embeds camera poses into the model’s feature space via adapter modules or modified attention layers. While effective for short-range motions, such implicit conditioning provides only weak geometric supervision, often resulting in inaccurate camera control, spatial drift, and accumulated distortion under long-range or complex trajectories. Explicit Geometry Guidance. An alternative strategy incorporates explicit 3D scene representations as conditioning prior to enforce stronger geometric constraints. ViewCrafter [32] guides video diffusion using point clouds rendered from monocular reconstruction models [29]. Recently, Free360 [3] separates foreground and background to handle depth and occlusion better, while VMem [16] introduces a learnable geometric memory for cross-view feature retrieval. However, these methods remain fundamentally limited by the ambiguity of single-image 3D reconstruction: errors in the initial geometry propagate through the diffusion process, constraining fidelity and view consistency. 3 Method Figure 2 illustrates the overall architecture of our framework, which consists of disentangled foreground and background generation, foreground Gaussian alignment, and geometry-guided novel view synthesis. Figure 3: Training pipeline with Dual-Stream Masking. A coarse 3DGS reconstructed from the video is rendered into a warped image sequence and a validity mask. During training, the mask latent is concatenated with both the noisy video latent and the warped latent along the channel dimension, and the two streams are fused along the temporal dimension for denoising. 3.1 Disentangling Foreground and Background Given a reference image IrefI_ref, our goal is to produce separate 3D prior for foreground objects and background. This disentanglement enables high-fidelity generation of foreground objects while ensuring a coherent spatial context. Foreground Prior Generation. We first segment salient foreground objects from the input image. A Vision-Language Model [2] provides category labels, which guide a segmentation model to produce precise binary masks Mii=1N\M_i\_i=1^N for N objects. For each masked object, we apply a 3D generation model [7] to obtain a complete 3D Gaussian Splatting representation: fg-geni=(μji,Σji,αji,cji)j=1Ji,G_fg-gen^i= \ ( _j^i, _j^i, _j^i,c_j^i ) \_j=1^J_i, (1) where μji∈ℝ3 _j^i ^3 is the Gaussian center, Σji _j^i the covariance, αji _j^i the opacity, and cjic_j^i the spherical harmonic color coefficients. The full foreground prior is fg-gen=⋃i=1Nfg-geniG_fg-gen= _i=1^NG_fg-gen^i. Background Reconstruction and Layout Estimation. For complementary scene geometry, we use a monocular feed-forward model [20] to produce a holistic 3DGS of the scene, denoted feedG_feed. Using the foreground masks Mi\M_i\, we split this into foreground and background components via projection and masking: fg-feedi _fg-feed^i =g∈feed∣proj(g)∈Mi, =\g _feed (g)∈ M_i\, (2) bg-feed _bg-feed =feed∖⋃i=1Nfg-feedi. =G_feed _i=1^NG_fg-feed^i. (3) In this formulation, bg-feedG_bg-feed provides direct geometric prior for the background. We further estimate a coarse layout to guide object placement. For each object i, we compute an axis-aligned 3D bounding box from fg-feediG_fg-feed^i, parameterized by its center cib_c^i and size sib_s^i. The aggregate layout ℒ=(ci,si)i=1NL=\(b_c^i,b_s^i)\_i=1^N provides an initial spatial configuration, which guides the subsequent geometric alignment of foreground objects into the unified scene. Figure 4: Qualitative comparisons on the DL3DV and Mip-NeRF 360 datasets. We evaluate novel view sequences generated by our method against VMem [16], GenWarp [25], and HW-Mirror [20]. GenNVS produces more geometrically consistent and temporally stable results. Red circles highlight geometric inconsistencies, and blue circles indicate unseen holes. 3.2 Foreground Gaussian Alignment The independently generated foreground fg-genG_fg-gen exists in a coordinate system separate from the aligned background bg-feedG_bg-feed. To construct a unified 3D scene prior, we align each foreground object to the scene through a coarse-to-fine two-stage process. For the i-th object, we define a rigid transformation TiT^i parameterized by a scale factor si∈ℝs^i , a translation vector ti∈ℝ3t^i ^3, and a rotation matrix Ri∈SO(3)R^i∈ SO(3), such that Ti(g)=si⋅(Rig)+tiT^i(g)=s^i·(R^ig)+t^i for a Gaussian g. Coarse Alignment. We first coarsely position each foreground object using the estimated layout ℒL. For the i-th object, we initialize scaling factor sis^i and translation tit^i such that the bounding box of fg-geniG_fg-gen^i aligns with the corresponding layout box sib_s^i and cib_c^i. To further refine the placement, we perform point cloud registration between the generated object and its feed-forward counterpart. We extract the visible portion of fg-geniG_fg-gen^i from the reference view by thresholding the accumulated opacity during rendering, then uniformly sample points from these visible Gaussians to form a point cloud PvisiP_vis^i. Similarly, we sample points from fg-feediG_fg-feed^i to obtain PfeediP_feed^i. The Iterative Closest Point (ICP) algorithm is applied to compute an incremental rigid transformation: (ΔRi,Δti)=argmin∑p∈PvisiR,t‖Rp+t−NNPfeedi(Rp+t)‖22,( R^i, t^i)= _R,t _p∈ P_vis^i\|Rp+t-N_P_feed^i(Rp+t)\|_2^2, (4) where NNP(x)N_P(x) finds the nearest neighbor of point x in point cloud P. This incremental transformation is then composed with the existing one: Ri←ΔRiRiR^i← R^iR^i, ti←ΔRiti+Δtit^i← R^it^i+ t^i. Fine-grained Optimization. Following coarse alignment, we refine the transformation parameters TiT^i for each object through a per-object optimization that enforces photometric, geometric, and mask consistency. Let C^i C^i, D^i D^i, and A^i A^i denote the color, depth, and alpha map rendered from the transformed foreground Gaussians ~fg-geni G_fg-gen^i, respectively. The corresponding ground-truth color CrefiC_ref^i, depth DrefiD_ref^i, and binary mask MiM^i are derived from the feed-forward reconstruction fg-feediG_fg-feed^i and the segmentation mask. We define a composite loss function with four terms. The color and depth losses ensure appearance and geometry alignment within the object region: ℒrgbi _rgb^i =‖Mi⊙(C^i−Crefi)‖1, =\|M^i ( C^i-C_ref^i)\|_1, (5) ℒdepthi _depth^i =‖Mi⊙(D^i−Drefi)‖1. =\|M^i ( D^i-D_ref^i)\|_1. (6) where ⊙ denotes element-wise multiplication. The mask alignment loss uses mean squared error to match the rendered silhouette: ℒmaski=‖A^i−Mi‖22.L_mask^i=\| A^i-M^i\|_2^2. (7) To penalize both holes inside the mask and overflow outside it, we employ two ReLU-based regularization terms: ℒholei=ReLU(Mi−A^i), _hole^i=ReLU(M^i- A^i), (8) ℒoverflowi=ReLU(A^i−Mi). _overflow^i=ReLU( A^i-M^i). (9) The total fine-grained loss is a weighted combination: ℒfinei=∑kλkℒki,L_fine^i= _k _kL_k^i, (10) where k∈rgb,depth,mask,hole,overflowk∈\rgb,depth,mask,hole,overflow\ denotes the different loss terms with their corresponding weights λk _k. We minimize ℒfineiL_fine^i independently for each object to obtain the final transformation parameters. The resulting precisely aligned foreground Gaussians are denoted as ¯fg-geni G_fg-gen^i. The complete and consistent 3DGS scene prior is assembled as: scene=bg-feed∪⋃i=1N¯fg-geni.G_scene=G_bg-feed∪ _i=1^N G_fg-gen^i. (11) 3.3 Geometry-Guided Novel View Synthesis With the unified 3DGS prior sceneG_scene, we synthesize novel view sequences using a video diffusion model. Our core contribution is a Dual-Stream Masking conditioning mechanism that leverages rendered geometric prior to ensure 3D-consistent synthesis. Model Architecture and Conditioning. We build upon a pretrained text-to-video model based on a Diffusion Transformer (DiT) [23, 26]. The model is conditioned on explicit geometric cues derived from sceneG_scene. Given a target camera trajectory, we render two guidance sequences: a warped RGB video warp∈ℝ3×T×H×WV_warp ^3× T× H× W and a corresponding binary mask video mask∈ℝ1×T×H×WV_mask ^1× T× H× W. The warped video is encoded by a frozen VAE encoder ℰE to produce warpc_warp, while the mask is resized to the latent resolution to obtain maskc_mask. The Dual-Stream Masking mechanism constructs the model input as illustrated in fig. 3. The mask latent maskc_mask is first concatenated with the noisy latent tz_t along the channel dimension, and separately with the warped latent warpc_warp. These two concatenated features are then joined along the temporal dimension, forming the complete input to the denoising network. This design provides explicit, view-aligned geometric and structural guidance at each denoising step. Training and Inference. During training, we use multiview video data with known camera poses. For a training clip with frames gtI_gt and poses Ct\C_t\, we first reconstruct a coarse 3DGS scene trainG_train from the first frame using a monocular model [20], as shown in fig. 3. We then render trainG_train along the trajectory to obtain the training conditions warpV_warp and maskV_mask. The warped video is encoded to ^warp c_warp, while the mask is interpolated to ^mask c_mask. The ground-truth video is encoded as latents 0=ℰ(gt)z_0=E(I_gt). Following the flow-matching framework [18], we construct a linear interpolation path from the data 0z_0 to a standard Gaussian noise ϵ∼(0,)ε (0,I): τ=(1−τ)0+τϵ,z_τ=(1-τ)z_0+τε, (12) where τ∈[0,1]τ∈[0,1] is the timestep. The condition is formed by the proposed Dual-Stream Masking mechanism. The model θv_θ is trained to predict the vector field that transports τz_τ along the probability flow, with supervision applied only to the first half of the temporal output (the original noisy stream): ℒ=0,ϵ,τ[ =E_z_0,ε,τ [ ∥ϵ−0 \|ε-z_0 (13) −θ(τ,τ;^warp,^mask)[:,:,:T]∥22], -v_θ(z_τ,τ;\, c_warp, c_mask)[:,:,:T]\|_2^2 ], where [:,:,:T][:,:,:T] denotes slicing the first T frames along the temporal dimension. This objective corresponds to learning the conditional velocity field under a straight-line interpolation path, which is a common special case of flow matching. During inference, given a reference image, we obtain the aligned prior sceneG_scene and a target trajectory. We render the conditions, process them accordingly, and apply the same Dual-Stream Masking process to guide the iterative denoising, synthesizing a high-fidelity and geometrically consistent novel view sequence. Table 1: Quantitative comparison on DL3DV dataset. Single-view novel view synthesis results on DL3DV dataset with diverse environments and complex camera trajectories. Our method demonstrates favorable geometric consistency and image quality. Method PSNR↑ SSIM↑ LPIPS↓ FID↓ RdistR_dist↓ TdistT_dist↓ HW-Mirror [20] 17.18 0.41 0.28 20.81 1.607 0.467 GenWarp [25] 14.38 0.18 0.47 28.68 1.811 0.464 VMem [16] 16.32 0.30 0.34 19.17 1.888 0.460 RecamMaster [1] 17.60 0.44 0.33 18.82 1.805 0.465 ViewCrafter [32] 18.72 0.50 0.26 16.38 1.320 0.472 CameraCtrl [10] 16.74 0.43 0.32 22.10 1.680 0.465 MotionCtrl [31] 15.28 0.41 0.30 20.46 1.772 0.478 SEVA [34] 19.07 0.49 0.28 15.70 1.157 0.433 Ours 19.39 0.53 0.24 14.21 1.142 0.451 4 Experiments 4.1 Experimental Setup Implementation Details. The video diffusion model is fine-tuned from Wan2.1-1.3B [26]. We train on the filtered SpatialVid-HQ dataset [27] at a resolution of 394×538394× 538. We use AdamW optimizer [21] with a base learning rate of 1×10−51× 10^-5 and a cosine annealing schedule. All experiments are conducted on 8 NVIDIA A100-80G GPUs, and more training details are provided in appendix A. During inference, we adopt the flow matching scheduler with 50 denoising steps and incorporate classifier-free guidance [12]. Evaluation Datasets and Metrics. We evaluate on two challenging benchmarks, Mip-NeRF 360 [4] and DL3DV [17]. Standard image quality metrics including PSNR, SSIM [30], and LPIPS [33] are reported. To assess geometric correctness, we compute camera pose errors as in [29]. Specifically, relative poses (Rgen,tgen)(R_gen,t_gen) between generated novel views are estimated using DUST3R [29] and compared to the ground truth poses (Rgt,tgt)(R_gt,t_gt). The rotation distance RdistR_dist and translation distance TdistT_dist are defined as: Rdist R_dist =arccos(0.5(tr(RgenRgtT)−1)), = (0.5(tr(R_genR_gt^T)-1) ), (14) Tdist T_dist =‖tgt−tgen‖2. =\|t_gt-t_gen\|_2. (15) where tr(⋅)tr(·) is the matrix trace. The translation is normalized by the distance to the furthest frame. These metrics emphasize complementary aspects of the single-image NVS problem. Image-space metrics measure whether each synthesized frame remains faithful to the target view appearance, while the pose distances evaluate whether the generated sequence preserves a coherent camera-induced 3D structure. This distinction is important because a video model can produce visually plausible frames while still drifting geometrically over time. We therefore report both families of metrics and interpret the results jointly rather than optimizing for a single score. In addition, a user study is conducted to obtain human assessment of the proposed approach. Participants are tasked with selecting the most preferred novel-view sequence in terms of visual realism and geometric consistency, after viewing the results of our method and the baseline methods. Details are provided in appendix C. 4.2 Evaluation Results We evaluate our method against several state-of-the-art feed-forward and generative approaches for novel view synthesis, including HunyuanWorld-Mirror [20], GenWarp [25], VMem [16], RecamMaster [1], ViewCrafter [32], CameraCtrl [10], MotionCtrl [31], and SEVA [34]. Quantitative Comparison. As shown in tables 1 and 2, GenNVS consistently achieves strong quantitative performance on both benchmarks. It improves perceptual quality with higher PSNR and SSIM and lower LPIPS and FID, while also maintaining strong trajectory accuracy with low camera pose errors. These results suggest that our method enhances both visual fidelity and geometric consistency, leading to more reliable novel view synthesis under challenging scenes and camera motions. The improvement is most pronounced on metrics that depend on cross-view structure. Compared with methods that primarily condition generation through camera embeddings or monocular reconstructions, GenNVS reduces both rotation and translation errors. This indicates that the generated views are not only sharp independently but also more compatible as a sequence. The gains are also consistent across the two datasets, suggesting that the disentangled prior does not overfit to a particular scene type or camera distribution. Table 2: Quantitative comparison on Mip-NeRF 360 dataset. Evaluation on Mip-NeRF 360 dataset further validates the robustness of our approach in handling complex indoor and outdoor scenes. GenNVS obtains favorable results against baselines in both perceptual quality and geometric accuracy. Method PSNR↑ SSIM↑ LPIPS↓ FID↓ RdistR_dist↓ TdistT_dist↓ HW-Mirror [20] 15.90 0.20 0.35 16.25 1.098 0.420 GenWarp [25] 14.59 0.33 0.47 29.33 1.184 0.629 VMem [16] 19.04 0.47 0.30 16.81 2.150 0.481 RecamMaster [1] 17.86 0.48 0.34 15.16 1.881 0.432 ViewCrafter [32] 16.75 0.48 0.39 19.23 1.067 0.595 CameraCtrl [10] 16.74 0.43 0.30 22.10 1.680 0.465 MotionCtrl [31] 15.28 0.33 0.32 20.46 1.772 0.478 SEVA [34] 19.15 0.51 0.33 14.80 0.924 0.423 Ours 19.43 0.54 0.27 13.50 0.848 0.408 Qualitative Comparison. Figure 4 shows visual results on both datasets. GenNVS alleviates common issues such as temporal flickering and background distortion, and maintains sharp foreground details. It generates geometrically stable foregrounds with natural background integration, demonstrating robustness even under large viewpoint changes. The user study in fig. 6 compares our method with baselines in terms of Overall Aesthetic (O.A.), Overall Consistency (O.C.), Foreground Aesthetic (F.A.), and Foreground Consistency (F.C.), showing that our method is consistently preferred. Qualitatively, the main difference lies in how errors accumulate along the camera path. Baseline video diffusion methods often correct local appearance but gradually change the shape, scale, or attachment of foreground objects, especially when the camera reveals partially unseen regions. Feed-forward geometry-based methods provide stronger structure, but their initial geometry may contain holes or entangled foreground-background artifacts that become visible after rendering. By aligning object-level foreground priors with the scene-level background, GenNVS supplies a denser geometric scaffold before diffusion, which reduces the amount of structure that must be hallucinated by the generator. 4.3 Ablation Studies We conduct ablation studies to examine the contribution of key designs in GenNVS, as shown in table 3. We compare four configurations: generation without foreground-background disentanglement, with disentanglement but without coarse alignment, with disentanglement and coarse alignment but without fine alignment, and the full model. The full model achieves the best overall quality while maintaining a favorable efficiency-accuracy trade-off. Furthermore, we showcase the scene editing capability of GenNVS. Figure 5: Ablation study on alignment stages. Qualitative comparisons demonstrate the necessity of both the coarse and fine alignment stages. Table 3: Ablation studies on Mip-NeRF 360 dataset. We evaluate the contribution of each component. The full model achieves the best quality while maintaining a trade-off in efficiency. PSNR ↑ LPIPS ↓ Time/obj (s) w/o disent. 15.90 0.35 * w/o coarse align. 18.39 0.22 177.45 w/o fine align. 19.65 0.18 1.32 GenNVS 21.75 0.16 22.88 Disentangled Foreground-Background Generation. We evaluate the proposed method with a variant that generates the scene holistically without separating foreground and background. As shown in table 3, this variant leads to a significant drop in all quality metrics, confirming that artifacts and holes in the entangled prior degrade novel view synthesis. Qualitatively, our disentangled prior is also denser and more geometrically complete, as further illustrated in the fig. 9. Coarse-to-Fine Alignment. We ablate our two-stage alignment by evaluating three configurations: coarse alignment only, fine alignment only, and our full pipeline. Table 3 and fig. 5 show that both incomplete variants perform poorly compared to the complete strategy. Removing coarse alignment leads to unstable convergence and lower accuracy, while skipping fine optimization leaves visible misalignments. Our complete coarse-to-fine approach achieves the best balance between quality and efficiency. Efficiency Analysis. To analyze the efficiency of our framework, we provide a detailed breakdown of the inference time over 73 frames in table 4. The coarse alignment stage is fast and provides a reliable initial pose estimate. The subsequent fine-grained optimization, while more computationally involved, remains highly efficient in practice as the optimization per-object can be performed in parallel. It optimizes only the rigid transformation for each object, which requires few learnable parameters. This runtime profile reflects the intended division of labor in the pipeline. The feed-forward components provide an initial scene hypothesis, the alignment stage corrects the spatial relationship between independently generated assets, and the diffusion model focuses on view synthesis rather than full geometry recovery. In practice, the additional alignment cost is small compared with diffusion sampling and provides a favorable trade-off: a modest amount of test-time geometric optimization substantially improves the stability of the generated video. Table 4: Analysis of inference time. The table details the processing time per stage, including disentangled generation, coarse and fine alignment (per object), diffusion-based rendering, and the total end-to-end latency. Disent. gen. Coarse (/obj) Fine (/obj) Diff. Total Time (s) 20.15 1.28 21.60 60.10 103.13 Figure 6: User study results. We evaluate our method along with VMem [16], GenWarp [25], HW-Mirror [20] on O.A., O.C., F.A., and F.C.. Our method performs favorably against baselines, receiving over 40% of votes as the preferred approach in each category. Scene Editing. Benefiting from its fully disentangled and explicitly editable 3D Gaussian representation, GenNVS inherently supports flexible scene manipulation. Users can directly edit the scene at the component level—such as replacing, removing, or adding a foreground object—after which the framework consistently regenerates all novel views from the updated 3D prior, as visualized in fig. 7. Figure 7: Scene editing results. Replacing the foreground "lego" with a "tank" preserves visual and geometric consistency across synthesized novel views. 5 Conclusion This work introduces GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. By separately modeling foreground and background, aligning them with a coarse-to-fine strategy, and conditioning a video diffusion model with Dual-Stream Masking, our method produces geometrically consistent novel views and supports flexible scene editing. The experiments show that explicitly improving the 3D prior before diffusion leads to better visual quality, more stable foreground geometry, and lower camera-pose errors across challenging benchmark scenes. The central observation behind our approach is that single-image NVS benefits from separating the sources of geometric reliability. Object-level generation provides stronger foreground structure, while feed-forward scene reconstruction provides global layout and background context. Aligning these two sources yields a prior that is more useful than either component alone, because it reduces holes, object drift, and foreground-background entanglement before the video model begins synthesis. This design also exposes an editable intermediate representation, making scene manipulation a natural consequence of the pipeline rather than an additional post-processing step. References [1] J. Bai, M. Xia, X. Fu, X. Wang, L. Mu, J. Cao, Z. Liu, H. Hu, X. Bai, P. Wan, et al. (2025) Recammaster: camera-controlled generative rendering from a single video. arXiv preprint arXiv:2503.11647. Cited by: §1, §1, §2, §2, Table 1, §4.2, Table 2. [2] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. (2025) Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §3.1. [3] C. Bao, X. Zhang, Z. Yu, J. Shi, G. Zhang, S. Peng, and Z. Cui (2025) Free360: layered gaussian splatting for unbounded 360-degree view synthesis from extremely sparse and unposed views. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 16377–16387. Cited by: §2. [4] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman (2022) Mip-nerf 360: unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 5470–5479. Cited by: §4.1. [5] J. Cen, Z. Zhou, J. Fang, W. Shen, L. Xie, D. Jiang, X. Zhang, Q. Tian, et al. (2023) Segment anything in 3d with nerfs. Advances in Neural Information Processing Systems 36, p. 25971–25990. Cited by: §2. [6] D. Charatan, S. L. Li, A. Tagliasacchi, and V. Sitzmann (2024) Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 19457–19467. Cited by: §2. [7] X. Chen, F. Chu, P. Gleize, K. J. Liang, A. Sax, H. Tang, W. Wang, M. Guo, T. Hardin, X. Li, et al. (2025) Sam 3d: 3dfy anything in images. arXiv preprint arXiv:2511.16624. Cited by: §1, §2, §3.1. [8] Y. Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T. Cham, and J. Cai (2024) Mvsplat: efficient 3d gaussian splatting from sparse multi-view images. In European Conference on Computer Vision, p. 370–386. Cited by: §2. [9] H. Go, B. Park, J. Jang, J. Kim, S. Kwon, and C. Kim (2025) Splatflow: multi-view rectified flow model for 3d gaussian splatting synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 21524–21536. Cited by: §2. [10] H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang (2024) Cameractrl: enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101. Cited by: Figure 11, Figure 11, Appendix D, §1, §2, Table 1, §4.2, Table 2. [11] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33, p. 6840–6851. Cited by: §1. [12] J. Ho and T. Salimans (2022) Classifier-free diffusion guidance. arXiv preprint. Cited by: §4.1. [13] Y. Hong, K. Zhang, J. Gu, S. Bi, Y. Zhou, D. Liu, F. Liu, K. Sunkavalli, T. Bui, and H. Tan (2023) Lrm: large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400. Cited by: §2. [14] L. Jiang, Y. Mao, L. Xu, T. Lu, K. Ren, Y. Jin, X. Xu, M. Yu, J. Pang, F. Zhao, et al. (2025) AnySplat: feed-forward 3d gaussian splatting from unconstrained views. arXiv preprint arXiv:2505.23716. Cited by: §2. [15] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis (2023) 3D gaussian splatting for real-time radiance field rendering.. ACM Transactions on Graphics (ToG) 42 (4), p. 139–1. Cited by: §1. [16] R. Li, P. Torr, A. Vedaldi, and T. Jakab (2025) VMem: consistent interactive video scene generation with surfel-indexed view memory. arXiv preprint arXiv:2506.18903. Cited by: §2, Figure 4, Table 1, Figure 6, §4.2, Table 2. [17] L. Ling, Y. Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y. Lu, et al. (2024) Dl3dv-10k: a large-scale scene dataset for deep learning-based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22160–22169. Cited by: Figure 10, §4.1. [18] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2022) Flow matching for generative modeling. arXiv preprint arXiv:2210.02747. Cited by: §3.3. [19] R. Liu, R. Wu, B. Van Hoorick, P. Tokmakov, S. Zakharov, and C. Vondrick (2023) Zero-1-to-3: zero-shot one image to 3d object. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 9298–9309. Cited by: §2. [20] Y. Liu, Z. Min, Z. Wang, J. Wu, T. Wang, Y. Yuan, Y. Luo, and C. Guo (2025) WorldMirror: universal 3d world reconstruction with any-prior prompting. arXiv preprint arXiv:2510.10726. Cited by: Appendix A, §2, Figure 4, §3.1, §3.3, Table 1, Figure 6, §4.2, Table 2. [21] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §4.1. [22] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng (2021) Nerf: representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65 (1), p. 99–106. Cited by: §1. [23] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4195–4205. Cited by: §3.3. [24] X. Ren, T. Shen, J. Huang, H. Ling, Y. Lu, M. Nimier-David, T. Müller, A. Keller, S. Fidler, and J. Gao (2025) Gen3c: 3d-informed world-consistent video generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6121–6132. Cited by: §1. [25] J. Seo, K. Fukuda, T. Shibuya, T. Narihira, N. Murata, S. Hu, C. Lai, S. Kim, and Y. Mitsufuji (2024) Genwarp: single image to novel views with semantic-preserving generative warping. Advances in Neural Information Processing Systems 37, p. 80220–80243. Cited by: Figure 4, Table 1, Figure 6, §4.2, Table 2. [26] T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z. Wu, and Z. Liu (2025) Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: Appendix A, §3.3, §4.1. [27] J. Wang, Y. Yuan, R. Zheng, Y. Lin, J. Gao, L. Chen, Y. Bao, Y. Zhang, C. Zeng, Y. Zhou, et al. (2025) Spatialvid: a large-scale video dataset with spatial annotations. arXiv preprint arXiv:2509.09676. Cited by: Appendix A, §4.1. [28] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny (2025) Vggt: visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 5294–5306. Cited by: §2. [29] S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud (2024) Dust3r: geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 20697–20709. Cited by: §2, §4.1. [30] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13 (4), p. 600–612. Cited by: §4.1. [31] Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P. Luo, and Y. Shan (2024) Motionctrl: a unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, p. 1–11. Cited by: Figure 11, Figure 11, Appendix D, §2, Table 1, §4.2, Table 2. [32] W. Yu, J. Xing, L. Yuan, W. Hu, X. Li, Z. Huang, X. Gao, T. Wong, Y. Shan, and Y. Tian (2024) Viewcrafter: taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048. Cited by: Figure 1, §1, §1, §2, Table 1, §4.2, Table 2. [33] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 586–595. Cited by: §4.1. [34] J. Zhou, H. Gao, V. Voleti, A. Vasishta, C. Yao, M. Boss, P. Torr, C. Rupprecht, and V. Jampani (2025) Stable virtual camera: generative view synthesis with diffusion models. arXiv preprint arXiv:2503.14489. Cited by: Figure 11, Figure 11, Appendix D, Table 1, §4.2, Table 2. Appendix Appendix A Model Details Data Processing. We train our model on the SpatialVid-HQ [27] dataset, a high-quality video collection comprising diverse scenes with precise camera trajectories, textual descriptions, and motion annotations. Based on metrics such as motion score and dynamic ratio, we filter the dataset and retain 354,944 video clips for training. Since camera poses are provided only for keyframes, we first compute full-frame trajectories via linear interpolation. For each video, we use the first frame to predict an initial 3D Gaussian Splatting (3DGS) scene with HunyuanWorld-Mirror [20]. This scene is then rendered along the interpolated camera path to generate two conditioning sequences: a warped RGB video warpV_warp and a corresponding binary validity mask maskV_mask, which indicate regions with reliable geometric projection, obtained by thresholding the rendered alpha values at 0.8. Training Configuration. Our model is based on the Wan2.1 1.3B text-to-video diffusion transformer [26]. To accommodate the dual-stream input introduced by our conditioning mechanism, we fine-tune the patch embedding layer and all 3D-Attention blocks, while keeping all other parameters frozen. Training is performed at a resolution of 394×538394× 538, chosen to match the optimal operating point of the geometry estimator. We use the AdamW optimizer with a learning rate of 1×10−51× 10^-5, a global batch size of 8 (1 per GPU), with a total training time of approximately 72 hours. The model is trained for 40k steps on 8 NVIDIA A100 GPUs. Appendix B Additional Analysis Figure 8: Comparison of 3D prior from a single image. Top row: initial point cloud from a monocular model. Bottom row: refined results after disentangled foreground-background generation and coarse-to-fine alignment. Figure 9: Viewpoint variation ranges supported by our method in the additional large-viewpoint-change results. Viewpoint Parameter Range Pitch ±60∘± 60 Yaw ±40∘± 40 Translation ∼20% 20\% scene scale Density and Completeness of the Geometric Prior. We further compare the 3D prior produced by a holistic monocular reconstruction pipeline and our disentangled generation strategy. As shown in fig. 9, the initial point cloud obtained from feed-forward monocular reconstruction is sparse and exhibits substantial missing regions. After disentangled foreground-background generation and coarse-to-fine alignment, our method produces a substantially denser and more complete 3DGS prior, with improved foreground geometry and better spatial coherence with the background. This enhanced geometric prior provides more reliable guidance for novel view synthesis. Sensitivity to Loss Weights. We use a relatively stable set of loss weights during the prior generation stage. To assess the sensitivity of the model to these hyperparameters, we perturb each loss weight by 30%30\% individually and evaluate the rendered outputs from the input viewpoint on 5 scenes. The results are summarized in table 5. The performance remains reasonably stable under these perturbations, indicating that the prior generation process is robust to moderate changes in loss balancing. Table 5: Sensitivity analysis of loss weights in the prior generation stage. Best values are highlighted in bold. Metric Default Lrgb↑30%L_rgb 30\% Ldepth↑30%L_depth 30\% Lmask↑30%L_mask 30\% Lhole↑30%L_hole 30\% Loverflow↑30%L_overflow 30\% PSNR 24.03 24.18 23.82 23.65 21.47 23.95 SSIM 0.83 0.84 0.81 0.80 0.74 0.82 LPIPS 0.18 0.19 0.16 0.21 0.28 0.18 Scaling Up with Many Objects. Our method treats per-object generation and refinement as independent processes, enabling efficient parallel execution when multiple foreground objects are selected. Since each object is optimized independently and then aligned with the overall point cloud through a coarse-to-fine refinement stage, error accumulation is effectively mitigated. For example, the bottom-left case in fig. 4 shows a scene in which four objects (chairs and a table) are selected as foreground, and no significant error accumulation is observed. Additional multi-object examples are also provided in fig. 10. These results suggest that our framework scales favorably to more complex scenes with multiple edited objects. Figure 10: Additional results on the DL3DV [17] dataset. Figure 11: Additional comparisons with CameraCtrl [10], MotionCtrl [31], and SEVA [34]. Our method shows stronger geometric consistency. Appendix C User Study We conducted a controlled user study to evaluate the perceptual quality of the novel views synthesized by different methods. The study involved 90 participants of various ages (18–60) and professional backgrounds. Participants are presented with novel view sequences of the same scenes generated by our method and several baselines. They are asked to select the most preferred result based on four explicit criteria: overall realism, foreground object realism, overall consistency across views, and foreground consistency. Our method received a strong and consistent preference across all categories, with a significant majority of participants selecting it as the preferred option. These results provide clear empirical evidence that our approach produces outputs perceived as more realistic and geometrically coherent than existing techniques. Appendix D Additional Results Figure 12: Generalization to Diverse Artistic Styles. We evaluate our framework on out-of-distribution artistic inputs, including hand-drawn sketches, game scenes, and anime imagery. Results show that our method synthesizes high-quality, stylized novel views while maintaining strict multi-view 3D consistency, highlighting that our disentangled prior and conditioning mechanism provide strong geometric guidance without sacrificing adaptation to varied visual styles. Figure 13: Results under boundary viewpoint changes. Additional Visualizations. We provide extensive qualitative results in fig. 10 on the DL3DV dataset. These results further demonstrate our model’s ability to recover plausible geometric and textural details in heavily occluded regions. Notably, they highlight the superior geometric consistency of foreground objects, even under challenging camera motions, a key advantage of our disentangled generation and alignment strategy. We further compare against CameraCtrl [10], MotionCtrl [31], and SEVA [34] in fig. 11. The results likewise show that our method better preserves geometric consistency and handles occluded regions more reliably. Generalization to Out-of-Domain Data. We also evaluate our framework on a curated set of images from the web, computer games, and animated media, which differ significantly from the training distribution. As shown in fig. 12, our method successfully generates plausible and consistent novel view sequences across these diverse inputs. This demonstrates strong generalization beyond photorealistic scenes to a wide range of visual styles and scene types. Results under Large Viewpoint Changes. We additionally provide results under larger viewpoint changes in fig. 13. Our method can reliably handle viewpoint variations within ranges in fig. 9: Limitations. While GenNVS advances single-image novel view synthesis, it shares certain limitations common to the field. Our framework assumes static scenes and does not model dynamic elements (e.g., moving people or flowing water). Additionally, the quality of the final output depends partially on the accuracy of the initial monocular 3D reconstruction.