Paper deep dive
ObjectSplat: Improving Mesh Fidelity and Interactivity for 3D Scenes via Object-Level Mesh Splatting
Minhas Kamal, Hiranya Garbha Kumar, Mahedi Kamal, Balakrishnan Prabhakaran
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 9/15/2026, 8:38:22 AM
Summary
The paper introduces ObjectSplat, a method for 3D scene reconstruction that improves mesh fidelity and interactivity by decomposing scenes into individual objects and a background before reconstruction. Unlike monolithic splatting methods, ObjectSplat segments instances from each frame, inpaints the background, and reconstructs each component independently using mesh splatting. This approach yields over a 5% gain in F-score, better novel-view synthesis, and supports object-wise modifiability, addressing the limitations of holistic reconstruction where unobserved regions cause artifacts and editing is difficult.
Entities (8)
Relation Signals (6)
ObjectSplat → improves → mesh fidelity
confidence 95% · Our method significantly improves mesh fidelity (over a 5% gain in F-score)
ObjectSplat → uses → Mesh Splatting
confidence 95% · reconstruct each instance and the background independently with mesh splatting
ObjectSplat → supports → object-wise modifiability
confidence 90% · supporting object-wise modifiability and interactivity
ObjectSplat → comparesfavorablyagainst → 3D Gaussian Splatting
confidence 85% · Holistic methods like traditional splatting... provide no such object structure... moving an object can cause the holistically-optimized scene to break
ObjectSplat → usesforsegmentation → SAM 3
confidence 80% · video segmenter that follows objects through the capture (SAM 3 [4])
ObjectSplat → usesforinpainting → Marigold
confidence 75% · Following Marigold [38], we keep the VAE encoder and decoder frozen and fine-tune only the UNet
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Splatting-based algorithms reconstruct photorealistic, real-time-renderable, and mesh-exportable 3D scenes from regular images, but they represent a scene as a single monolithic field. Therefore, the reconstruction has no object-level structure, leaving it infeasible for downstream editing or interaction. Moreover, regions that are never directly observed in the input scans are contaminated by the surrounding texture and left uncorrected, capping both mesh fidelity and novel-view synthesis. We propose a decompose-before-reconstruct approach: we segment the instances out of every frame, consider the remaining as background and inpaint it, reconstruct each instance and the background independently with mesh splatting, and compose them into a single scene. Our method significantly improves mesh fidelity (over a 5\% gain in F-score) and novel-view synthesis, while supporting object-wise modifiability and interactivity. The code will be made publicly available.
Tags
Links
- Source: https://arxiv.org/abs/2608.30423v1
- Canonical: https://arxiv.org/abs/2608.30423v1
Trouble viewing inline? Open PDF directly →
Full Text
67,343 characters extracted from source content.
Expand or collapse full text
ObjectSplat: Improving Mesh Fidelity and Interactivity for 3D Scenes via Object-Level Mesh Splatting Minhas Kamal Hiranya Garbha Kumar Mahedi Kamal Balakrishnan Prabhakaran Affiliation: State University of New York at Albany, USA Affiliation: mxkamal, hgkumar, mkamal3, bprabhakaran @albany.edu Abstract Splatting-based algorithms reconstruct photorealistic, real-time-renderable, and mesh-exportable 3D scenes from regular images, but they represent a scene as a single monolithic field. Therefore, the reconstruction has no object-level structure, leaving it infeasible for downstream editing or interaction. Moreover, regions that are never directly observed in the input scans are contaminated by the surrounding texture and left uncorrected, capping both mesh fidelity and novel-view synthesis. We propose a decompose-before-reconstruct approach: we segment the instances out of every frame, consider the remaining as background and inpaint it, reconstruct each instance and the background independently with mesh splatting, and compose them into a single scene. Our method significantly improves mesh fidelity (over a 5% gain in F-score) and novel-view synthesis, while supporting object-wise modifiability and interactivity. The code will be made publicly available. Figure 1: Demonstration of our method’s capabilities. (a) Qualitative Comparison: On a novel test view, the baseline holistic reconstruction (top) exhibits significant floating artifacts and texture degradation. In contrast, our object-level approach (bottom) produces a cleaner, more geometrically accurate result. (b) Interactivity: Our object-decomposed representation enables direct manipulation of scene elements, including scaling (purse), appearance editing (shoe), duplication (banana), and rigid transformation (hand). In our pipeline, as backgrounds are inpainted before reconstruction, objects can be moved from their original positions without leaving holes. 1 Introduction Radiance-field methods have transformed the way photorealistic 3D scenes are reconstructed from 2D scans. Neural Radiance Fields (NeRF) [23] first demonstrated that a scene could be encoded as a continuous volumetric function and rendered from novel views with excellent realism, but its implicit formulation is slow to train and query. 3D Gaussian Splatting (3DGS) [19] replaced the neural volume with an explicit set of anisotropic Gaussians and a tile-based rasterizer, achieving comparable quality at real-time framerates. However, 3DGS represents a scene as an unstructured cloud of semi-transparent primitives that float in space and never explicitly model a surface, making its output ill-suited to the polygonal meshes that traditional rendering pipelines and game engines consume. Several subsequent works [12, 45] try to improve the surface quality of 3DGS. However, 2D Gaussian Splatting (2DGS) [16] takes a direct approach, collapsing primitives onto oriented surface-aligned disks. Following 2DGS, triangle splatting [14] and mesh splatting [13] methods produce explicit meshes that can be imported into 3D engines. Interactivity - selecting an object and then moving, duplicating, or removing it - requires that a scene be organized into discrete, semantically meaningful objects. This is not merely a structural preference; interaction itself creates new demands. For instance, moving an object reveals its previously unobserved surfaces, making robust novel-view synthesis critical for a seamless user experience. Holistic methods like traditional splatting, however, provide no such object structure. They reconstruct the environment as a single continuous field, a “soup” of primitives with no notion of where one object ends and another begins. Consequently, attempting to move an object can cause the holistically-optimized scene to break at the seams, leaving behind corrupted geometry or undefined holes. Although some existing methods attempt to recover object structures after reconstruction, they often produce contact-point artifacts and fail to solve the underlying issue: unobserved areas, now exposed by interaction, are typically filled with noise from the initial joint optimization. We take a different approach: rather than segmenting a scene after it is reconstructed, we decompose before we reconstruct. Our starting observation concerns why holistic reconstruction limits fidelity in the first place. When a scene is fit as a single field, regions that are never directly observed (the underside of a table, the wall behind a cabinet) receive no corrective gradient and are instead filled in by whatever texture surrounds them, leaving stray, mis-colored primitives that degrade the recovered geometry. Reconstructing each object in isolation removes this failure mode: with the background and every other object masked away, the optimizer has nothing to fit but the object itself, and no opportunity to place primitives where the object is not. We do a 1:7 train-test split (sparse view) for evaluating our methods ability for novel and unseen view synthesis. Our comparison against vanilla mesh splatting show that per-object decomposition yields significant improvements in mesh fidelity (measured by F-score against ground-truth geometry) together with modest gains in photorealism. We attribute this to the per-object problem being fundamentally easier: individual objects converge in fewer than half the iterations a full scene requires during training. Finally, we study completion as a route to still-higher fidelity, and find that today’s obstacle is as much evaluation as method. Our reported metrics exclude every form of completion: diffusion background inpainting and generative amodal completion [41] are not view-consistent enough across frames to train on, and even a deterministic flat-surface completion lowers the measured F-score because the ground-truth meshes are themselves incomplete exactly where completion acts, so a recovered surface is scored against a ground truth that lacks it. Assessing completion fairly therefore needs geometry-complete ground truth, such as realistic synthetic scenes, which this setting still lacks. We report this, alongside the consistency limits of automatic multi-view segmentation, to guide future work. In summary, our contributions are: • A novel method for object-level reconstruction via mesh splatting, built on a decompose-before-reconstruct principle. Our approach incorporates a new optimization strategy that exploits the opacity of primitives to prune extraneous geometry, leading to substantial improvements in mesh fidelity and novel-view synthesis while producing an editable, object-centric scene. • Empirical gains over vanilla mesh splatting in mesh fidelity and novel-view synthesis, with the observation that per-object optimization converges in under half the iterations of a holistic reconstruction. • A systematic analysis of the key bottlenecks in decomposition-based reconstruction. We examine the critical role of cross-view consistency across the segmentation, completion, and inpainting modules, and identify fundamental limitations in current datasets and evaluation metrics that hinder progress. 2 Related Works Table 1: A capability comparison of competing methods. Our work is the only approach that jointly provides an object-decomposed, mesh-accurate reconstruction with photorealistic appearance and a completed background from sparse views. Legend: ✓ = supported, ∼ = partial support (e.g., per-edit or runtime only), × = not supported. Method Mesh-accurate Photorealistic Object-decomposed Completed background Sparse-view Interactive 2DGS [16] ✓ ✓ × × × × SuGaR [12] ✓ ✓ × × × ✓ Gaussian Opacity Fields [45] ✓ × × × × × Mesh / Triangle Splatting [13, 14] ✓ ✓ × × × ✓ Gaussian Grouping [43] × ✓ ✓ ∼ × ✓ VR-GS [18] × ✓ ✓ ∼ × ✓ GScream [39] × ✓ × ∼ × × InFusion [22] × ✓ × ∼ × ✓ MAtCha [11] ✓ ✓ × × ✓ × Object-centric 2DGS [32] ✓ ✓ ∼ × × ✓ Split&Splat [26] × ✓ ✓ × × ✓ Ours ✓ ✓ ✓ ✓ ✓ ✓ 2.1 Monolithic Reconstruction with Splatting Neural Radiance Fields [23] and 3D Gaussian Splatting (3DGS) [19] established radiance fields as a means of photorealistic, real-time novel-view synthesis, but neither yields the explicit surfaces that downstream graphics pipelines require. Two distinct strategies have since emerged to recover geometry. The first extracts a mesh from a trained 3DGS field as a post-process: SuGaR [12] regularizes Gaussians toward the surface and meshes them via Poisson reconstruction, Gaussian Opacity Fields [45] define a level-set opacity field for adaptive surface extraction, and planar- and depth-based variants [6, 46] further sharpen the recovered geometry. These methods yield accurate geometry, but they optimize shape rather than appearance: the extracted mesh is uncolored, and re-attaching texture from the underlying Gaussians afterward is non-trivial and lossy, which limits the usefulness of the output in rendering and editing pipelines that expect textured assets. The second strategy makes the surface primitive itself the object of optimization, so that geometry and appearance are learned jointly. 2D Gaussian Splatting (2DGS) [16] collapses each primitive onto an oriented, surface-aligned disk, yielding view-consistent geometry that meshes cleanly, while triangle- and mesh-based splatting [14, 13] optimize photorealistic triangle or mesh primitives end-to-end and produce textured, engine-ready meshes directly. Crucially, however, every method in both categories reconstructs the scene as a single, monolithic field with no object-level structure (Table 1, top block). Our work builds on the second strategy, specifically the mesh-splatting instance. 2.2 Post-Hoc Segmentation and Shared-Field Editing To make a reconstructed scene editable, a body of work attaches object semantics to the primitives after reconstruction training. One line lifts 2D masks (typically from a promptable segmenter such as SAM [20]) into the 3D field, either by optimally assigning masks to primitives (FlashSplat [36]) or by distilling per-primitive identity and language features during training (Gaussian Grouping [43], SAGA [5], Feature-3DGS [48], LangSplat [29]). Because a single primitive straddling an object boundary is shared between neighbours, selecting or editing one object perturbs the other and produces artifacts at contact points; SAGD [15] identifies these boundary primitives explicitly but resolves them only by splitting within the shared field. Building on such segmentations, interactive systems let objects be moved, removed, or simulated (Point’n Move [17] for click-based manipulation, VR-GS [18] for physics-driven deformation, and text- or physics-based editors [7, 42]), yet each operates on a shared 3DGS field, so moving an object exposes background regions the reconstruction never modelled, which are then filled per edit at inference time rather than reconstructed once (Table 1, second block). Almost all of this work targets 3DGS, whose primitives lack the mesh accuracy of the 2D-splatting representations above. Closest to our setting are two concurrent, unpublished preprints. Object-centric 2DGS [32] reconstructs a single isolated object on the mesh-accurate 2D-splatting representation, but removes—rather than completes—its background and handles only one object at a time. Split&Splat [26] instead segments a scene into multiple objects, reconstructs each independently, and merges them, making it the most similar in spirit to our pipeline; however, it operates on 3DGS, leaves the background exposed by removed objects unfilled, and does not target mesh fidelity or the sparse-view regime. Neither, moreover, provides a public implementation, precluding a direct experimental comparison. Our method differs from both by reconstructing every object and a completed background as independent mesh splats, and by doing so under sparse views. 2.3 Inpainting and Amodal Completion A related line studies how to fill the regions revealed when an object is removed. In the NeRF setting, SPIn-NeRF [24] established the mask-and-inpaint paradigm, supervising the masked region with a 2D inpainter. Gaussian-domain methods extend this to splats: GScream [39] regularizes the geometry and features of the filled region for cross-view consistency, InFusion [22] learns depth completion from a diffusion prior to place new primitives at correct depths, and RefFusion [25] adapts a diffusion inpainter to a reference view and distills it into the 3D field. These methods fill the hole left by a single removal at edit time; they do not produce a persistent, fully completed background that supports arbitrary rearrangement of many objects, which is what our pipeline reconstructs up front. Figure 2: Overview of our pipeline. Every pixel of every posed frame is assigned to exactly one component: an object or the background. Objects and the background, which for our qualitative results is inpainted but is excluded from quantitative evaluation (see Sec. 4.4), are reconstructed independently; the components are then composed by union into a single, separable scene. A complementary question is how to recover the parts of an object that are themselves occluded. Amodal completion methods such as Amodal3R [41] hallucinate an object’s hidden geometry from its visible extent and could, in principle, further improve the fidelity and completeness of each per-object reconstruction. In practice, however, we find that current amodal methods are not sufficiently consistent across views: injecting their inconsistent completions into the per-object optimization degrades rather than improves it. We therefore treat object completion as a promising but not-yet-reliable direction rather than a component of our pipeline. 2.4 Sparse-View Radiance-Field Reconstruction Because dense captures are impractical in many settings, a growing body of work reconstructs radiance fields from sparse views. Regularization-based methods add geometric priors to few-shot 3DGS (FSGS [49] densifies Gaussians guided by proximity and monocular depth, and DNGaussian [21] normalizes depth at global and local scales), while pose-free and feed-forward approaches such as InstantSplat [10] and MVSplat [8] initialize geometry directly from stereo or cost-volume priors, and diffusion-prior methods such as ReconFusion [40] synthesize pseudo-observations at unseen viewpoints. Most of these optimize novel-view RGB quality on a monolithic scene; MAtCha [11] is a notable exception in targeting mesh-quality geometry from few views, but it too reconstructs the scene as a whole. None pairs sparse-view robustness with per-object, mesh-accurate decomposition, the regime in which we report our largest gains (Table 1, bottom block). Across these four lines of work, the same gap recurs. Methods that achieve high mesh fidelity reconstruct the scene as a single monolithic field; methods that recover object structure segment an already-reconstructed field, inheriting contact-point artifacts and disocclusion holes that are filled per edit, almost always on 3DGS rather than a mesh-accurate representation, and, because the underlying reconstruction is already fixed, unable to improve its mesh fidelity; inpainting and amodal completion address those holes only one edit at a time, or not yet reliably; and sparse-view methods improve few-shot reconstruction but leave the scene holistic and appearance-oriented. Consequently, no prior method in Table 1 satisfies all of these axes at once. We close this gap by inverting the usual order: instead of reconstructing a scene and then decomposing it, we decompose the multi-view observations first and reconstruct each object, alongside a completed background, as independent mesh splats that merge into a single interactive scene. We detail this method in Section 3. 3 Methodology Our method reconstructs a scene not as a single field but as a collection of independently optimized objects and a background, assembled into one environment. The input is a casually captured video of a static scene together with per-frame camera poses; in practice we recover poses with global structure-from-motion [28, 35], though any posed image set suffices. We build on mesh splatting [13], a surface-splatting representation whose primitives are opaque, spherical-harmonic-shaded triangles initialized from the sparse point cloud. One property of this representation is central to our approach: although primitives may be partially transparent early in optimization, they are driven opaque as training proceeds, so a fitted surface cannot be seen through. The organizing principle of the pipeline is a partition of image space. Across the posed frames, every pixel is assigned to exactly one component: one of the scene’s objects, or the background. Each component is reconstructed independently from the pixels it owns before all components are composed into one scene (Fig. 2). Two properties follow directly and motivate the design. First, because the components partition the images (no pixel is supervised twice, and none is left unsupervised), their union covers exactly what a holistic reconstruction would have covered, so the composed scene is free of holes by construction. Second, because each object is a self-contained component in the shared world frame, it remains separable after the fact: it can be selected, moved, or removed without disturbing its neighbors. 3.1 Scene Decomposition The partition begins with a set of per-frame instance masks that are mutually exclusive: each pixel names at most one object. We obtain these masks in one of two ways, depending on what the capture provides. When the scene ships with a ground-truth 3D reconstruction carrying per-object annotations, as in ScanNet++ [44], we rasterize the annotated geometry into every camera and read off, at each pixel, the object owning the nearest surface. This yields masks with three convenient properties: occlusion is resolved automatically by the depth test, so a chair behind a table simply cedes those pixels to the table; object identity is shared across all frames by construction, so no cross-frame association is needed; and, because each pixel stores a single owner, the masks are mutually exclusive by representation. Their only imperfection is inherited from the source scan (where it is incomplete, a few pixels are left unowned), which the background construction below absorbs. We use this path for our main results, as it isolates the contribution of the reconstruction method from any error in the segmentation. The second regime targets the practical setting where no such annotation exists. Here we obtain masks with a video segmenter that follows objects through the capture (SAM 3 [4], prompted with the scene’s object categories), or with an off-the-shelf video object tracker [9, 30, 4] that propagates a set of initial masks across all frames, so that object identity is carried through the video rather than re-inferred and re-associated per frame. This removes the dependence on ground-truth geometry, but such trackers are imperfect in two ways that matter downstream. First, their mask boundaries are not perfectly consistent across views: the same object is delineated slightly differently from frame to frame, and these inconsistencies propagate into the reconstruction. Second, and more damaging, a tracker occasionally merges distinct objects under a single identity (an object together with the surface it rests on, say), so that one object’s component is trained on pixels belonging to two scene objects at once. Its reconstruction then fuses their geometry, which breaks separability for interactivity and, because the component is now fit against inconsistent multi-object supervision, degrades mesh and even visual fidelity. We report this regime as a step toward a fully automatic pipeline. 3.2 Image-Space Partition and Background Completion Given the masks, we turn each frame into training images for the components that appear in it. For every retained object we write a full-frame image in which only the object’s own pixels are kept and all others are masked out. We keep the full frame rather than a tight crop so that each object is optimized against the scene’s true cameras; it is the mask, not the framing, that restricts what it learns from. The background is then the per-frame complement of the objects: every pixel not claimed by an object trained in that frame. This makes the partition explicit (each pixel belongs to exactly one component, an object or the background) and is precisely what guarantees the composed scene is gap-free. Removing the objects, however, leaves the background unobserved wherever an object stood and was never seen from another view. Because these pixels are simply absent from the background’s training images rather than supervised as empty, they leave a hole in the reconstructed room only where the capture never revealed the surface behind an object. For interactive use we complete such regions so that no gap is exposed when an object is relocated, using an image inpainter adapted from a pretrained latent diffusion model [34]. Following Marigold [38], we keep the VAE encoder and decoder frozen and fine-tune only the UNet; we extend its three-channel input with six additional channels, three for the RGB input and three for the hole mask, and generate training data synthetically from indoor imagery [31] by randomly masking parts of each image. Because the inpainter operates per frame, its completions are not consistent from one view to the next, and training a reconstruction on them would inject that disagreement into the geometry. We therefore use inpainting only to render the final composed scene hole-free for interactivity and qualitative novel-view figures (Fig. 1); it is excluded from all quantitative results, which train the background on its observed pixels alone. Completion need not stop at the background. Separability introduces a gap of its own: a surface an object rests on, such as a tabletop beneath a keyboard, is reconstructed with a hole where the occluder blocked it, exposed the moment the occluder is moved. We posit that completing such occluded object geometry should further improve reconstruction. Generative amodal completion is the natural tool, but we find that its cross-view inconsistency proves far too damaging for geometry-rich objects and collapses the per-object optimization. To test the premise without that instability, we apply a deterministic proxy on the tractable case of flat supporting surfaces: we recover the occluded region by binary hole-filling of the surface’s mask [37] and inpaint the exposed pixels with Navier–Stokes inpainting [2]. This yields a relatively hole-free supporting surface for interactive use, but we keep it out of our headline metrics: because the ground-truth meshes are themselves incomplete where these surfaces are occluded, a completed surface is scored against a ground truth that lacks it, so the measured F-score drops even where the recovered geometry is plausible. We report this analysis, and the geometry-complete data it would take to evaluate completion properly, in the appendix. 3.3 Independent Reconstruction and Background Randomization Each object and the background are reconstructed by the same, unmodified mesh-splatting optimizer, differing only in the masked images they are shown. Every component is optimized against the shared cameras under an identical protocol (the same resolution and iteration budget), and we keep, per component, the checkpoint that scores best on its own held-out views. Because isolated objects converge at very different rates, and far sooner than a holistic reconstruction of the whole scene (refer to appendix for details), this per-component selection matters. Since the components are independent, they are trained in parallel. Training a surface splat on a masked object, however, exposes a subtlety of the representation that we exploit. Mesh splatting renders a pixel p by compositing, front to back, the N triangles that cover it [13]. Each triangle tnt_n contributes its spherical-harmonic color tnc_t_n scaled by its opacity otno_t_n (the minimum of its three vertex opacities) and a spatial coverage window In()∈[0,1]I_n(p)∈[0,1], attenuated by the transmittance of the triangles in front of it. Compositing the primitives over a background color b gives C()=∑n=1Ntnwn()+T()C(p)= _n=1^Nc_t_n\,w_n(p)\;+\;T(p)\,b (1) wn()=otnIn()∏i=1n−1(1−otiIi())w_n(p)=o_t_nI_n(p)\!\! _i=1^n-1\! (1-o_t_iI_i(p) ) (2) where T()=∏n=1N(1−otnIn())T(p)= _n=1^N (1-o_t_nI_n(p) ) is the residual transmittance reaching the background. Writing the total foreground weight as W()=∑nwn()=1−T()W(p)= _nw_n(p)=1-T(p) and the weight-averaged foreground color as ¯()=W()−1∑ntnwn() c(p)=W(p)^-1 _nc_t_nw_n(p), Eq. (1) is simply C()=W()¯()+(1−W())C(p)=W(p)\, c(p)+ (1-W(p) )b. As optimization matures the vertex opacities saturate (otn→1o_t_n\!→\!1), so a covered pixel becomes opaque and its geometry can no longer be thinned by lowering opacity. When an object is trained in isolation, every pixel in the masked-out region Ω should render as empty space, that is, as the background itself. Supervising these pixels toward b, the loss over Ω is ℒΩ=∑∈Ω∥C()−∥2=∑∈ΩW()2∥¯()−∥2L_ \;=\; _p∈ C(p)-b ^2\;=\; _p∈ W(p)^2\, c(p)-b ^2 (3) With a fixed background, Eq. (3) is minimized trivially by painting the stray primitives the background color (¯= c=b), which leaves constant-colored triangles littering Ω and corrupting the object. We instead resample b from a uniform distribution at every iteration. Taking the expectation over b (mean ¯ b, per-channel variance σ2>0 _b^2>0) [ℒΩ]=∑∈ΩW()2(∥¯()−¯∥2+σ2)E_b [L_ ]\;=\; _p∈ W(p)^2 ( c(p)- b ^2+ _b^2 ) (4) which, because σ2>0 _b^2>0 and the colors are bounded, is minimized only as W()→0W(p)\!→\!0 for all ∈Ωp∈ : no fixed foreground color can track a background that changes every step. Since the opacities are already saturated and the coverage In()I_n(p) of a triangle over a pixel it overlaps cannot vanish, W()W(p) cannot be driven down by making primitives fainter; the only way to satisfy Eq. (4) is to remove the primitives covering Ω . Background randomization thus turns the masked region into a pressure that deletes every primitive not supported by the object. This requires only setting the renderer’s background color b per iteration, which mesh splatting’s differentiable renderer already exposes. It is also what makes the method strong at novel-view synthesis from few captures: a holistic reconstruction sheds such stray, environment-corrupted primitives only once enough viewpoints expose them as inconsistent, whereas background randomization removes them from a single view regardless of how many are available. Table 2: Comparison of meshsplatting and segmented-meshsplatting across scenes in sparse view (train:test = 1:7) Method Metric 5a269ba6fe 08bd80ce2a 39f36da05b 69e5939669 c0a81452 ef18cf0708 fb564c935d fe5fe0a8a4 00a231a370 08bbbdcc3d Average meshsplatting (30k) SSIM↑ 0.748 0.782 0.794 0.820 0.618 0.687 0.728 0.702 0.819 0.769 0.747 PSNR↑ 20.207 21.731 23.034 22.422 14.100 17.656 20.297 18.868 20.751 21.362 20.043 LPIPS↓ 0.367 0.310 0.288 0.277 0.429 0.414 0.375 0.385 0.322 0.322 0.349 F-score↑ 0.412 0.498 0.390 0.521 0.275 0.359 0.400 0.426 0.497 0.437 0.421 meshsplatting (best checkpoint) SSIM↑ 0.769 0.786 0.788 0.811 0.675 0.714 0.740 0.717 0.841 0.777 0.762 PSNR↑ 20.545 22.168 22.742 22.024 15.931 18.557 20.577 19.219 21.415 21.421 20.460 LPIPS↓ 0.358 0.306 0.297 0.293 0.415 0.402 0.372 0.371 0.297 0.328 0.344 F-score↑ 0.417 0.514 0.417 0.548 0.299 0.366 0.435 0.435 0.505 0.456 0.439 segmented-meshsplatting SSIM↑ 0.778 0.785 0.786 0.810 0.687 0.716 0.749 0.712 0.847 0.759 0.763 PSNR↑ 20.793 21.737 22.594 21.792 16.486 18.768 20.816 18.921 21.705 20.935 20.455 LPIPS↓ 0.339 0.294 0.293 0.284 0.409 0.398 0.353 0.366 0.283 0.332 0.335 F-score↑ 0.487 0.554 0.452 0.581 0.367 0.401 0.468 0.485 0.543 0.491 0.483 This pruning is also the mechanism behind our fidelity gains, and it clarifies why decomposition helps at all. When a whole scene is fit at once, surfaces that are never directly observed (the underside of a table, the wall behind a cabinet) receive no corrective supervision and are left filled with whatever texture the surrounding geometry happened to deposit, a defect no further training removes because nothing observes it. Reconstructing an object in isolation ameliorates this failure mode: with every other surface masked away and the masked region actively pruned, the optimizer has nothing to fit but the object, and no way to place primitives where the object is not. Finally, decomposition makes each reconstruction far easier: a single object presents much less geometric and appearance variation than a full room and reaches its best checkpoint in fewer than half the iterations a whole scene requires (refer appendix for details), so few views already constrain it well. Composition is then trivial. Because every component was fitted in the same world frame against a partition of the same photographs, their union is the scene: we concatenate the per-object and background primitives into a single splat, with no joint re-optimization and no blending. What each component contributes to a rendered pixel is decided by the renderer’s ordinary depth ordering, exactly as if the primitives had been optimized together. The result is one scene that renders as a whole, yet in which every object remains an identifiable, separable group of primitives available for downstream selection, editing, or export. 4 Experiments 4.1 Setup Datasets. Our approach needs two things of a dataset: per-frame instance masks consistently associated with the same 3D objects across views, and a ground-truth mesh for measuring geometric fidelity. ScanNet++ [44] dataset satisfies both: its per-object 3D annotations rasterize into every frame as mutually consistent masks (Sec. 3.1), and its meshes let us measure F-score directly (scene selection in the appendix). All main results use ScanNet++. We also tried MipNeRF-360 [1], but it has no ground-truth mesh and no instance annotations, and even state-of-the-art video segmentation associates objects inconsistently across its frames, degrading the per-object optimization; we relegate its results to the appendix as evidence the method runs without ground-truth masks. Novel-view protocol. To probe reconstruction from few views we invert the usual split: whereas vanilla mesh splatting trains on 7 of every 8 frames and tests on the 8th, we train on 1 and test on the other 7, spreading the single training view evenly through the capture. Every method sees the same 1-in-8 training frames and is scored on the same held-out 7. Because the held-out frames differ from the training view by changes in perspective, this measures novel-view synthesis quality, sharpened by the scarcity of training views. Metrics. We report visual fidelity with PSNR, SSIM, and LPIPS [47], and geometric fidelity with F-score against the ground-truth scan. Baselines. Our primary baseline is the original mesh splatting implementation [13], for which we report results from the best-performing checkpoint for each scene (evaluated at both a fixed 30k-iteration, as is common in related literature, and at convergence). A direct quantitative comparison to the closest concurrent works (Sec. 2.2) was precluded by the absence of public implementations or results. Implementation. As a preprocessing step, we use COLMAP [35] to undistort the ScanNet++ images. Each scene component is then trained independently for 30k iterations using the unmodified mesh-splatting optimizer. For the final scene composition, we select the best-performing checkpoint for each individual component. Components are independent and trained in parallel across GPUs. Reported metrics use decomposition and background randomization: the background is trained on its observed pixels, and neither the diffusion inpainter nor the deterministic surface completion of Sec. 3.2 is applied. Both are used only for the qualitative interactivity results and are analyzed in the appendix. 4.2 Results Table 2 reports ScanNet++ results across ten scenes. Our object-decomposed reconstruction improves geometric fidelity (F-score) over vanilla mesh splatting on every scene, at both the 30k and best-checkpoint settings, while PSNR and SSIM stay comparable and LPIPS improves on most scenes: decomposition sharpens geometry substantially and appearance modestly. The circular table in Fig. 1(a) also ablates background randomization; removing it introduces contaminated geometry and textures to the underside, confirming that the pruning it induces is a primary source of the geometric gain rather than a byproduct of decomposition alone. Under the conventional dense split (7 train, 1 test) the same comparison shows a 2% improvement in F1 scores while visual fidelity remains comparable, and the method also runs without ground-truth annotations on MipNeRF-360; both are reported in the appendix. 4.3 Analysis Why decomposition helps novel-view synthesis. The gain comes from how stray, environment-corrupted object primitives are removed. A holistic reconstruction can trim them, but only given enough viewpoints to expose them as inconsistent; as views grow scarce it loses that signal, whereas background randomization prunes them from a single view, and so regardless of view count. Decomposition also eases optimization: each object is a far simpler target and converges in fewer than half the iterations of a full scene (refer to the appendix). We analyze deterministic surface completion, which is excluded from the results above, in the appendix. 4.4 Interactivity Because every object is a separable group of primitives in a shared world frame, the composed scene supports downstream editing directly. Fig. 1 shows objects moved, removed, and rearranged, and the exported meshes brought into a standard game engine. For these qualitative results we fill regions revealed behind relocated objects with the diffusion inpainter of Sec. 3.2, making the scene hole-free for interaction; because its completions are not cross-view-consistent, it is used only here and not in our quantitative evaluation (see appendix and Sec. 5). 5 Limitations and Future Work Our largest bottleneck is cross-view consistency. Decomposing a scene and reconstructing its parts independently only works when each part is delineated the same way across every frame, and four modules must each supply that consistency: object segmentation, cross-frame association, and occluded-surface completion, and background inpainting. Each falls short today. Automatic video segmentation and tracking produce boundaries that drift from frame to frame and occasionally merge distinct objects, which is why our headline results fall back on ground-truth masks; even the annotation-rasterized masks of ScanNet++ are not perfectly consistent, as the source scans are incomplete (see appendix). Diffusion background inpainting and generative amodal completion [41] are likewise not multi-view-consistent enough to train a reconstruction on, so our quantitative results exclude both. Surface completion is excluded for a second reason: the ground-truth meshes are themselves incomplete precisely in the occluded regions completion targets, so a completed surface is scored as a false positive against a ground truth that lacks it, and the measured F-score falls even where the recovered geometry is plausible. We therefore cannot presently confirm completion’s benefit through geometric metrics, only its qualitative role in producing hole-free interactive scenes. The absence of a suitable dataset is itself a limitation on two fronts. No dataset we found supplies consistent per-frame instance masks tied to 3D objects alongside a ground-truth mesh except ScanNet++, which constrains both evaluation and the scenes we can study; and measuring completion fairly would additionally require geometry-complete ground truth, such as realistic synthetic CAD scenes, which do not yet exist for this setting. Building such a dataset is an important direction for making completion-based methods measurable. A second set of limitations is inherited from the underlying mesh-splatting representation. The recovered meshes are not watertight, so they cannot be driven directly by a physics engine; their textures bake in the capture’s lighting, precluding accurate relighting; and their surfaces are spikily and haphazardly triangulated, which costs visual fidelity on close inspection and yields inaccurate contact geometry for interaction. Our reconstructions also assume a static scene and depend on recovered camera poses, and per-object optimization adds cost, though it is largely offset by training components in parallel. Improving the mesh quality of the base representation would compound directly with the fidelity gains from decomposition. 6 Conclusion We presented a decompose-before-reconstruct approach that reconstructs a scene as independently optimized objects and a background over a partition of image space, instantiated on mesh splatting. Its central ingredient is a background-randomized training scheme that, by exploiting the non-transparency of mesh-splatting primitives, prunes stray geometry during optimization and drives our gains in mesh fidelity and novel-view synthesis, with the largest margins when input views are few. The same decomposition yields a separable, mesh-exportable scene that is directly usable in downstream applications. Our analysis of completion shows that its benefit is presently hard to measure rather than absent: a recovered surface is penalized against ground-truth meshes that are themselves incomplete. The cross-view inconsistency of current generative methods for inpainting, completion, and segmentation remains a primary bottleneck, indicating that future gains will come from developing more consistent models and the geometry-complete datasets needed to train them. References [1] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman (2022) Mip-nerf 360: unbounded anti-aliased neural radiance fields. CVPR. Cited by: Appendix E, §4.1. [2] M. Bertalmio, A. L. Bertozzi, and G. Sapiro (2001) Navier-stokes, fluid dynamics, and image and video inpainting. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Vol. 1, p. I–I. Cited by: §3.2. [3] G. Bradski (2000) The OpenCV Library. Dr. Dobb’s Journal of Software Tools. Cited by: Appendix A. [4] N. Carion, L. Gustafson, Y. Hu, S. Debnath, R. Hu, D. S. Coll-Vinent, C. Ryali, K. V. Alwala, H. Khedr, A. Huang, J. Lei, T. Ma, B. Guo, A. Kalla, M. Marks, J. Greer, M. Wang, P. Sun, R. Rädle, T. Afouras, E. Mavroudi, K. Xu, T. Wu, Y. Zhou, L. Momeni, R. HAZRA, S. Ding, S. Vaze, F. Porcher, F. Li, S. Li, A. Kamath, H. K. Cheng, P. Dollar, N. Ravi, K. Saenko, P. Zhang, and C. Feichtenhofer (2026) SAM 3: segment anything with concepts. In The Fourteenth International Conference on Learning Representations, Cited by: Appendix B, §3.1. [5] J. Cen, J. Fang, C. Yang, L. Xie, X. Zhang, W. Shen, and Q. Tian (2025) Segment any 3d gaussians. In Proceedings of the AAAI conference on artificial intelligence, Vol. 39, p. 1971–1979. Cited by: §2.2. [6] D. Chen, H. Li, W. Ye, Y. Wang, W. Xie, S. Zhai, N. Wang, H. Liu, H. Bao, and G. Zhang (2024) Pgsr: planar-based gaussian splatting for efficient and high-fidelity surface reconstruction. IEEE Transactions on Visualization and Computer Graphics 31 (9), p. 6100–6111. Cited by: §2.1. [7] Y. Chen, Z. Chen, C. Zhang, F. Wang, X. Yang, Y. Wang, Z. Cai, L. Yang, H. Liu, and G. Lin (2024) Gaussianeditor: swift and controllable 3d editing with gaussian splatting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21476–21485. Cited by: §2.2. [8] Y. Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T. Cham, and J. Cai (2024) Mvsplat: efficient 3d gaussian splatting from sparse multi-view images. In European conference on computer vision, p. 370–386. Cited by: §2.4. [9] H. K. Cheng, S. W. Oh, B. Price, A. Schwing, and J. Lee (2023) Tracking anything with decoupled video segmentation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), p. 1316–1326. Cited by: §3.1. [10] Z. Fan, W. Cong, K. Wen, K. Wang, J. Zhang, X. Ding, D. Xu, B. Ivanovic, M. Pavone, G. Pavlakos, et al. (2024) Instantsplat: sparse-view gaussian splatting in seconds. arXiv preprint arXiv:2403.20309. Cited by: §2.4. [11] A. Guédon, T. Ichikawa, K. Yamashita, and K. Nishino (2025) Matcha gaussians: atlas of charts for high-quality geometry and photorealism from sparse views. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 6001–6011. Cited by: §2.4, Table 1. [12] A. Guédon and V. Lepetit (2024) Sugar: surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 5354–5363. Cited by: §1, §2.1, Table 1. [13] J. Held, S. Son, R. Vandeghen, D. Rebain, M. Gadelha, Y. Zhou, A. Cioppa, M. C. Lin, M. V. Droogenbroeck, and A. Tagliasacchi (2025) MeshSplatting: differentiable rendering with opaque meshes. arXiv. External Links: Document, 2512.06818 [cs] Cited by: Appendix C, §1, §2.1, Table 1, §3.3, §3, §4.1. [14] J. Held, R. Vandeghen, A. Deliege, A. Hamdi, D. Rebain, S. Giancola, A. Cioppa, A. Vedaldi, B. Ghanem, A. Tagliasacchi, et al. (2025) Triangle splatting for real-time radiance field rendering. In Thirteenth International Conference on 3D Vision, Cited by: §1, §2.1, Table 1. [15] X. Hu, Y. Wang, L. Fan, C. Luo, J. Fan, Z. Lei, Q. Li, J. Peng, and Z. Zhang (2026) SAGD: boundary-enhanced segment anything in 3d gaussian via gaussian decomposition. IEEE Transactions on Image Processing. Cited by: §2.2. [16] B. Huang, Z. Yu, A. Chen, A. Geiger, and S. Gao (2024) 2D gaussian splatting for geometrically accurate radiance fields. In SIGGRAPH 2024 Conference Papers, External Links: Document Cited by: §1, §2.1, Table 1. [17] J. Huang, H. Yu, J. Zhang, and H. Nait-Charif (2024) Point’n move: interactive scene object manipulation on gaussian splatting radiance fields. IET Image Processing 18 (12), p. 3507–3517. Cited by: §2.2. [18] Y. Jiang, C. Yu, T. Xie, X. Li, Y. Feng, H. Wang, M. Li, H. Lau, F. Gao, Y. Yang, et al. (2024) Vr-gs: a physical dynamics-aware interactive gaussian splatting system in virtual reality. In ACM SIGGRAPH 2024 conference papers, p. 1–1. Cited by: §2.2, Table 1. [19] B. Kerbl, G. Kopanas, T. Leimkuehler, and G. Drettakis (2023) 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42 (4). External Links: ISSN 0730-0301, Document Cited by: §1, §2.1. [20] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W. Lo, et al. (2023) Segment anything. In 2023 IEEE/CVF international conference on computer vision (ICCV), p. 3992–4003. Cited by: §2.2. [21] J. Li, J. Zhang, X. Bai, J. Zheng, X. Ning, J. Zhou, and L. Gu (2024) Dngaussian: optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 20775–20785. Cited by: §2.4. [22] Z. Liu, H. Ouyang, Q. Wang, K. L. Cheng, J. Xiao, K. Zhu, N. Xue, Y. Liu, P. Luo, and Y. Cao (2026) Infusion: inpainting 3d gaussians via learning depth completion from diffusion prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 5726–5738. Cited by: §2.3, Table 1. [23] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng (2021) NeRF: representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65 (1), p. 99–106. External Links: ISSN 0001-0782, Link, Document Cited by: §1, §2.1. [24] A. Mirzaei, T. Aumentado-Armstrong, K. G. Derpanis, J. Kelly, M. A. Brubaker, I. Gilitschenski, and A. Levinshtein (2023) Spin-nerf: multiview segmentation and perceptual inpainting with neural radiance fields. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 20669–20679. Cited by: §2.3. [25] A. Mirzaei, R. De Lutio, S. W. Kim, D. Acuna, J. Kelly, S. Fidler, I. Gilitschenski, and Z. Gojcic (2024) Reffusion: reference adapted diffusion models for 3d scene inpainting. arXiv preprint arXiv:2404.10765. Cited by: §2.3. [26] L. Monchieri, E. Camuffo, F. Barbato, P. Zanuttigh, and S. Milani (2026) Split&Splat: zero-shot panoptic segmentation via explicit instance modeling and 3d gaussian splatting. arXiv preprint arXiv:2602.03809. Cited by: §2.2, Table 1. [27] E. Ozguroglu, R. Liu, D. Surís, D. Chen, A. Dave, P. Tokmakov, and C. Vondrick (2024) Pix2gestalt: amodal segmentation by synthesizing wholes. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3931–3940. External Links: Document Cited by: §D.1. [28] L. Pan, D. Baráth, M. Pollefeys, and J. L. Schönberger (2025) Global structure-from-motion revisited. In Computer Vision – ECCV 2024, A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol (Eds.), p. 58–77. External Links: ISBN 978-3-031-73661-2, Document Cited by: §3. [29] M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister (2024) Langsplat: 3d language gaussian splatting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 20051–20060. Cited by: §2.2. [30] N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, et al. (2025) Sam 2: segment anything in images and videos. In International Conference on Learning Representations, Vol. 2025, p. 28085–28128. Cited by: §3.1. [31] M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M. A. Bautista, N. Paczan, R. Webb, and J. M. Susskind (2021) Hypersim: a photorealistic synthetic dataset for holistic indoor scene understanding. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), p. 10892–10902. External Links: ISBN 978-1-6654-2812-5, Link, Document Cited by: Figure 5, Figure 5, §D.1, §3.2. [32] M. Rogge and D. Stricker (2025) Object-centric 2d gaussian splatting: background removal and occlusion-aware pruning for compact object models. arXiv preprint arXiv:2501.08174. Cited by: §2.2, Table 1. [33] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), p. 10674–10685. Cited by: §D.1. [34] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), p. 10674–10685. Cited by: Figure 5, Figure 5, §3.2. [35] J. L. Schonberger and J. Frahm (2016) Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 4104–4113. Cited by: §3, §4.1. [36] Q. Shen, X. Yang, and X. Wang (2024) Flashsplat: 2d to 3d gaussian splatting segmentation solved optimally. In European Conference on Computer Vision, p. 456–472. Cited by: §2.2. [37] P. Soille et al. (1999) Morphological image analysis: principles and applications. Vol. 2, Springer. Cited by: §3.2. [38] M. Viola, K. Qu, N. Metzger, B. Ke, A. Becker, K. Schindler, and A. Obukhov (2025) Marigold-dc: zero-shot monocular depth completion with guided diffusion. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 5359–5370. Cited by: Figure 5, Figure 5, §D.1, §3.2. [39] Y. Wang, Q. Wu, G. Zhang, and D. Xu (2024) Learning 3d geometry and feature consistent gaussian splatting for object removal. In European conference on computer vision, p. 1–17. Cited by: §2.3, Table 1. [40] R. Wu, B. Mildenhall, P. Henzler, K. Park, R. Gao, D. Watson, P. P. Srinivasan, D. Verbin, J. T. Barron, B. Poole, et al. (2024) Reconfusion: 3d reconstruction with diffusion priors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 21551–21561. Cited by: §2.4. [41] T. Wu, C. Zheng, F. Guan, A. Vedaldi, and T. Cham (2025) Amodal3r: amodal 3d reconstruction from occluded 2d images. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 9181–9193. Cited by: §1, §2.3, §5. [42] T. Xie, Z. Zong, Y. Qiu, X. Li, Y. Feng, Y. Yang, and C. Jiang (2024) Physgaussian: physics-integrated 3d gaussians for generative dynamics. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4389–4398. Cited by: §2.2. [43] M. Ye, M. Danelljan, F. Yu, and L. Ke (2024) Gaussian grouping: segment and edit anything in 3d scenes. In European conference on computer vision, p. 162–179. Cited by: §2.2, Table 1. [44] C. Yeshwanth, Y. Liu, M. Nießner, and A. Dai (2023) ScanNet++: a high-fidelity dataset of 3d indoor scenes. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), p. 12–22. External Links: ISBN 979-8-3503-0718-4, Link, Document Cited by: Appendix A, Appendix B, §3.1, §4.1. [45] Z. Yu, T. Sattler, and A. Geiger (2024) Gaussian opacity fields: efficient adaptive surface reconstruction in unbounded scenes. ACM Transactions on Graphics (ToG) 43 (6), p. 1–13. Cited by: §1, §2.1, Table 1. [46] B. Zhang, C. Fang, R. Shrestha, Y. Liang, X. Long, and P. Tan (2026) Rade-gs: rasterizing depth in gaussian splatting. ACM Transactions on Graphics 45 (2), p. 1–14. Cited by: §2.1. [47] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF conference on computer vision and pattern recognition, p. 586–595. Cited by: §4.1. [48] S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P. Chari, S. You, Z. Wang, and A. Kadambi (2024) Feature 3dgs: supercharging 3d gaussian splatting to enable distilled feature fields. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21676–21685. Cited by: §2.2. [49] Z. Zhu, Z. Fan, Y. Jiang, and Z. Wang (2024) Fsgs: real-time few-shot view synthesis using gaussian splatting. In European conference on computer vision, p. 145–163. Cited by: §2.4. Appendix A Scene Selection For our experimental evaluation, we select a subset of 10 scenes from the ScanNet++ [44] dataset. To ensure a rigorous and fair assessment of our decomposition and reconstruction pipeline, these scenes were curated based on the following two primary criteria: • Ground-Truth Geometry: The chosen scenes exhibit high-quality, relatively complete ground truth (GT) meshes, which is critical for accurate quantitative geometric evaluation and generation of complete 2D masks via backprojection. • Object Composition: The environments contain a sufficient number of distinct, decomposable object instances that are highly relevant to our methodology (specifically for ScanNet++: chairs, tables, sofas, coffee tables, benches, PCs, and pool tables). Furthermore, to specifically evaluate the usefulness of OpenCV’s [3] Navier-Stokes-based deterministic surface completion, we consider a narrower subset of 4 scenes. Because our heuristic for identifying fillable occlusion holes is intentionally conservative to avoid aggressive geometric hallucinations, we focus on scenes featuring planar surfaces (e.g., desks, tables, etc). Appendix B Modality Disparity and Backprojection Mask Incompleteness A key challenge in leveraging 3D annotations for 2D mask generation is the fundamental modality disparity between high-resolution RGB images and sensor-reconstructed 3D meshes. As ScanNet++ [44] is a real-world dataset, the GT 3D instance annotations are defined on a reconstructed mesh that is inherently imperfect, containing local geometric defects, scanning artifacts, and missing surfaces. While back-projecting these 3D annotations onto the 2D image plane, these underlying mesh defects propagate directly into the 2D domain, illustrated in Fig. 3, yielding incomplete masks that fail to cover the objects entirely. While our method’s capacity to improve mesh and visual fidelity in sparse-view settings largely absorbs these degradations, rendering the net impact on our sparse-view reconstruction minor, this modality gap nonetheless imposes an upper bound on performance that a geometrically pristine 3D annotation source would resolve. Figure 3: Limitations of 3D-to-2D mask projection due to incomplete geometry in ScanNet++. Due to missing geometry and reconstruction artifacts in the 3D meshes (visible as checkerboard background patterns indicating holes), projecting 3D segmentations to 2D viewports yields incomplete masks and false visibility boundaries. Specific failure cases include: (a) occluded under-desk regions and thin structures such as chair legs, and (b) transparent/specular surfaces (e.g., window panes) as well as dark, non-Lambertian surfaces (e.g., leather sofa folds). Albeit this mask incompleteness, the GT 3D annotations remain multi-view consistent, which is critical for current splatting methods. Cross-view consistency issues get significantly exacerbated when transitioning to ML-based segmentation pipelines (e.g., using 2D video tracking models such as SAM3 [4]). Consequently, we utilize the rasterized ground-truth annotations from ScanNet++ for our primary evaluations. Figure 4: Training Graph. Individual objects reach their best-scoring checkpoints in fewer iterations than the whole-scene reconstruction. Plus, objects converge at markedly different rates from one another primarily depending on their shape simplicity. This figure shows training and validation graphs (PSNR, SSIM, LPIPS, L1=m*PSNR+n*SSIM+o*LPIPS) throughout the 30k optimization iterations. Appendix C Per-Component Training Convergence The variance in training dynamics, observed in Fig. 4, is heavily governed by the physical scale of the target: smaller object instances possess lower geometric and textural complexity compared to large supporting structures or dense backgrounds, thereby requiring significantly fewer optimization iterations to converge. This disparity in convergence rates makes our per-component training strategy highly advantageous, as it allows us to perform best-checkpoint selection individually for each object, avoiding the sub-optimal compromises and over-fitting typical of uniform holistic training. We note that we select our best checkpoint after the Restricted Delaunay Triangulation (RDT) step (11K-th iteration in Mesh-Splatting [13]) which is run only once during the entire training routine and improves the connectivity of the triangle soup. Appendix D Background Inpainting and Amodal Completion D.1 Network Architecture Figure 5: Adapted stable diffusion (SD) [34] architecture for object completion and background inpainting. We condition on incomplete rgb and mask for the denoising operation, similar to Marigold [38]. Here, we take a pretrained SDv1.5 and modify the input layer of the U-Net. The network is retrained by synthetically generated data from the Hypersim [31] dataset for both background inpainting (Fig. 6) and object completion (Fig. 7); the text encoder and the image encoder-decoder (VAE) are kept frozen. During inference, we employ a variant of the Classifier-Free Guidance approach. To recover unobserved background regions (inpainting) previously occluded by foreground objects, as well as to generate hidden geometry for partially occluded foreground instances (amodal completion), we design a unified latent diffusion-based architecture. While the structural design of our network takes inspiration from the dense prediction framework introduced in Marigold [38], we adapt it for: background inpainting and amodal object completions. To leverage generalized image priors, we start with the pre-trained Stable Diffusion v1.5 [33]. During the training phase, only the Diffusion U-Net is gets updated. For this retraining, we utilize synthetic indoor environments from the Hypersim dataset [31], which provides diverse, photorealistic scenes for learning complex geometries and lighting variations. Though background inpainting and amodal completion are significantly different tasks, we use the same architecture for both models. Background inpainting model is simply trained on randomly masked images. However, generating data for amodal completion is tricky: we use a method similar to [27]. All other auxiliary networks, including the VAE encoder, VAE decoder, and text encoders, remain strictly frozen. This selective fine-tuning approach preserves the robust foundational visual priors of the pre-trained latent space, while allowing the U-Net to specialize on our tasks. Figure 6: Multi-view inconsistency in 2D background inpainting. While the diffusion model fills the masked regions (left), its independent 2D generation produces inconsistent textures for the same physical area across different views (right). In (a) and (c) the model hallucinates grid patterns but of different color, and (b) has a flat, untextured gray surface. These severe frame-to-frame structural and textural variations create conflicting gradients for splatting methods degrading the 3D mesh optimization. D.2 Performance and Optimization Limitations While our diffusion-based model is capable of generating visually plausible background and amodal completions for isolated frames, it inherently struggles to maintain strict multi-view consistency. As illustrated for background inpainting in Fig. 6, and for object completion in Fig. 7, applying the 2D generative model independently across varying perspectives results in severe spatial inconsistencies. For instance, when generating the occluded regions of a hexagonal wooden table (Fig. 7), the model inconsistently predicts structural elements like missing legs, while introducing severe texture noise and geometric warping across different frames. Figure 7: Multi-view inconsistency in 2D amodal completion. While applying independent amodal completion to occluded objects across different frames, the generative model produces inconsistent predictions. (a) and (b) has missing legs, and (c) has severe texture noise and geometric warping. These inconsistencies get propagated and corrupts the splat training. Because our 3D splatting optimization relies heavily on photometric consistency across overlapping views, these frame-to-frame generative variations act as contradictory supervisory signals. If injected directly into the optimization pipeline, these conflicting gradients force the model to average over inconsistent appearances, which actively degrades both the reconstructed mesh geometry and the overall texture fidelity. Consequently, while 2D generative completion serves as a valuable tool for downstream interactive scene editing and visualization, both background inpainting and amodal completion are expressly excluded from our optimization pipeline during quantitative evaluations to prevent geometric corruption. Appendix E MipNeRF-360 Results ScanNet++ supplies both consistent per-object masks and a ground-truth mesh, so it carries our main results. To show that the method still runs without ground-truth annotations, we also report MipNeRF-360 [1], whose masks are obtained through the automatic video-segmentation path of Section Methodology (Scene Decomposition). Because MipNeRF-360 provides no ground-truth masks, the cross-view inconsistency of automatic segmentation causes a marginal loss of visual fidelity relative to the ground-truth-mask setting; with no ground-truth mesh available, we cannot evaluate its effect on geometric fidelity. However, we present qualitative reconstruction results on various objects from the Mip-NeRF 360 dataset in Fig. 8. Figure 8: Robust mesh and visual reconstruction across extreme perspective changes on Mip-NeRF 360. Our approach maintains highly consistent geometry and texture quality from both observed views (left) and completely unobserved, novel viewpoints (right). Despite significant shifts in perspective, the extracted 3D structures remain complete and artifact-free, demonstrating the generalization capability and spatial consistency of the reconstruction across diverse objects. Appendix F Surface Completion The surface completion proposed in our methodology is designed to recover occluded regions of supporting structures (e.g., tabletops, desks) that are shadowed by foreground object instances. Fig. 9 shows results from this method. By patching these occlusions, we reconstruct a relatively complete background mask suitable for downstream interactive editing. To evaluate this module, we conduct a comparative analysis on the completion subset of ScanNet++ (the four scenes containing prominent planar structures; see Sec. A), comparing our final reconstructions with and without surface completion. For the completion pipeline, we utilize OpenCV’s Navier-Stokes-based inpainting method (cv2.inpaint) over the projected planar surfaces. Figure 9: Qualitative results of deterministic surface completion. Original reconstructed supporting surfaces exhibit occlusion holes and broken edges where foreground objects previously stood (left). The OpenCV deterministic completion method successfully repairs these planar regions (right). As shown in Table 3, executing surface completion results in a marginal decrease in 3D F-score, while leaving 2D rendering metrics (PSNR, SSIM, and LPIPS) virtually unchanged. The invariance of the visual metrics is expected: during evaluation, the segmented foreground objects are rendered in their original positions, thereby re-occluding the completed regions and preventing them from contributing to the image-space loss. We hypothesize that the slight decline in F-score is an artifact of the evaluation target rather than a reflection of geometric degradation. Sensor-based ground-truth meshes (such as those in ScanNet++) are themselves incomplete in occluded regions, meaning that a physically plausible, reconstructed surface is penalized as a false positive when measured against a ground truth that lacks geometry in those coordinates. Fully validating the fidelity of completed surfaces requires a geometry-complete ground truth—such as realistic, synthetic CAD-modeled environments—which does not currently exist at the scale and complexity of ScanNet++. We view the development of such complete synthetic benchmarks as an essential prerequisite for the quantitative evaluation of completion-based reconstruction techniques. Table 3: Effect of deterministic surface completion. Quantitative comparison (F-score, PSNR, SSIM, and LPIPS) on 4 selected ScanNet++ scenes, demonstrating the impact of deterministic flat-surface completion. without completion with completion Scene SSIM↑ PSNR↑ LPIPS↓ F1↑ SSIM↑ PSNR↑ LPIPS↓ F1↑ 08bd80ce2a 0.785 21.737 0.294 0.554 0.785 21.771 0.296 0.550 69e5939669 0.810 21.792 0.284 0.581 0.811 21.811 0.283 0.580 fe5fe0a8a4 0.713 18.921 0.366 0.485 0.711 18.842 0.367 0.484 08bbbdcc3d 0.759 20.935 0.332 0.491 0.755 20.708 0.338 0.491 Table 4: Dense-view reconstruction results on ScanNet++. Per-scene and average quantitative metrics (SSIM, PSNR, LPIPS, and F-score) comparing vanilla mesh splatting and our decomposition method under a dense 7:1 train-to-test split. Method Metric 5a269ba6fe 08bd80ce2a 39f36da05b 69e5939669 c0a81452 ef18cf0708 fb564c935d fe5fe0a8a4 00a231a370 08bbbdcc3d Average meshsplatting (30k) SSIM↑ 0.849 0.834 0.798 0.846 0.850 0.776 0.794 0.799 0.887 0.869 0.830 PSNR↑ 24.163 25.343 23.874 24.029 22.762 22.459 23.389 22.167 24.702 25.563 23.845 LPIPS↓ 0.262 0.279 0.305 0.273 0.285 0.385 0.333 0.319 0.256 0.240 0.294 F-score↑ 0.466 0.599 0.471 0.603 0.338 0.411 0.509 0.547 0.605 0.560 0.511 Ours SSIM↑ 0.852 0.802 0.790 0.812 0.850 0.769 0.776 0.779 0.877 0.833 0.814 PSNR↑ 23.704 23.200 23.482 22.261 23.633 21.255 22.121 21.037 23.462 23.535 22.769 LPIPS↓ 0.242 0.298 0.312 0.304 0.279 0.374 0.341 0.323 0.248 0.271 0.300 F-score↑ 0.525 0.555 0.492 0.585 0.412 0.466 0.492 0.458 0.502 0.547 0.503 Appendix G Dense-View Experiments Our primary results focus on sparse views to evaluate our method’s capability for unseen view synthesis. Here, we additionally present results from dense views to demonstrate its scene reconstruction quality, which is on par with vanilla mesh splatting. In our dense-view experiments (detailed in Table 4), we observe that our method performs marginally below holistic mesh splatting. We hypothesize that this performance characteristic is governed by a trade-off between mask quality and view density: • Dense-View Regime: With highly redundant viewpoints, the vanilla mesh splatting baseline can leverage the dense multi-view optimization constraints to effectively prune erroneous geometry on its own. Consequently, the relative margin of improvement from our decomposition vanishes, and the cumulative negative impact of noisy, incomplete boundaries across numerous frames begins to dominate the performance trade-off. • Sparse-View Regime: Conversely, under severe view constraints, the vanilla baseline lacks the redundant perspective supervision necessary to self-prune erroneous geometry, resulting in severe floaters and corrupted surfaces. Under these conditions, the explicit geometric regularization and floater elimination provided by our object-level decomposition heavily outweigh the local errors introduced by imperfect masks, resulting in a substantial net improvement in both mesh and rendering quality without any hyper-parameter tuning.