Paper deep dive
GS-VLA: Plug-and-Play Viewpoint Canonicalization for Frozen VLA Policies via Gaussian Splatting
Yechan Park, HyunJin Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 8/21/2026, 4:46:24 AM
Summary
The paper introduces GS-VLA, a lightweight, plug-and-play framework that enhances the robustness of frozen Vision-Language-Action (VLA) policies to viewpoint shifts by using 3D Gaussian Splatting for novel-view synthesis. By leveraging a 'Locality' assumption, the method reformulates viewpoint adaptation as a localized disocclusion task, allowing a 4M-parameter U-Net to predict Gaussian attributes that warp observations to a canonical view without retraining the policy. This approach significantly recovers performance lost due to camera perturbations across various policies, task suites, and perturbation scales.
Entities (8)
Relation Signals (6)
GS-VLA → uses → 3D Gaussian Splatting
confidence 95% · This paper proposes a lightweight, plug-and-play framework... first approach to directly leverage 3D Gaussian-based novel-view synthesis
GS-VLA → improvesrobustnessof → VLA Policies
confidence 92% · improves robustness to viewpoint shifts in Vision-Language-Action (VLA) policies without policy retraining.
GS-VLA → evaluatedon → LIBERO
confidence 90% · Our experiments show that even a small displacement of the camera mount can reduce the success rate on the LIBERO benchmark
GS-VLA → basedon → Locality Assumption
confidence 88% · Under a Locality assumption... viewpoint normalization reduces to a scene- and policy-independent disocclusion task.
GS-VLA → consistsof → U-Net
confidence 85% · Our work implements this idea with a 4M-parameter 3D-Gaussian canonicalizer... consisting of a 4M-parameter U-Net
GS-VLA → outperforms → AnyCamVLA
confidence 80% · we introduce a Locality-based reformulation that reduces the number of parameters by 42x compared with AnyCamVLA.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper proposes a lightweight, plug-and-play framework that improves robustness to viewpoint shifts in Vision-Language-Action (VLA) policies without policy retraining. To our knowledge, this is the first approach to directly leverage 3D Gaussian-based novel-view synthesis for observation-space adaptation in VLA policies. Current VLA performance relies on the implicit assumption that training and deployment camera configurations are identical. Our experiments show that even a small displacement of the camera mount can reduce the success rate on the LIBERO benchmark from about 90% to about 10% in the worst case. Prior approaches, such as large-scale fine-tuning or generative data augmentation, are computationally expensive and risk catastrophic forgetting. To address this, viewpoint shifts are reformulated as a localized novel-view synthesis problem. Under a Locality assumption, that camera perturbations remain within a small bounded region relative to the workspace, viewpoint normalization reduces to a scene- and policy-independent disocclusion task. Our work implements this idea with a 4M-parameter 3D-Gaussian canonicalizer prepended to a frozen VLA policy. Without modifying policy weights, GS-VLA improves performance across three orthogonal axes: (1) Policy architectures, (2) Unseen task suites, and (3) Perturbation scales. These results show that a lightweight visual module can recover a large fraction of the performance lost under viewpoint shift, without policy retraining.
Tags
Links
- Source: https://arxiv.org/abs/2608.19066v1
- Canonical: https://arxiv.org/abs/2608.19066v1
Trouble viewing inline? Open PDF directly →
Full Text
67,717 characters extracted from source content.
Expand or collapse full text
GS-VLA: Plug-and-Play Viewpoint Canonicalization for Frozen VLA Policies via Gaussian Splatting Yechan Park Affiliation: Dankook University Affiliation: Yongin, Republic of Korea Email: yechan3219@dankook.ac.kr HyunJin Kim Affiliation: Dept. of E, Dankook University Affiliation: Yongin, Republic of Korea Email: hyunjin2.kim@gmail.com Affiliation: (Corresponding author) Abstract This paper proposes a lightweight, plug-and-play framework that improves robustness to viewpoint shifts in Vision-Language-Action (VLA) policies without policy retraining. To our knowledge, this is the first approach to directly leverage 3D Gaussian-based novel-view synthesis for observation-space adaptation in VLA policies. Current VLA performance relies on the implicit assumption that training and deployment camera configurations are identical. Our experiments show that even a small displacement of the camera mount can reduce the success rate on the LIBERO benchmark from about 90% to about 10% in the worst case. Prior approaches, such as large-scale fine-tuning or generative data augmentation, are computationally expensive and risk catastrophic forgetting. To address this, viewpoint shifts are reformulated as a localized novel-view synthesis problem. Under a Locality assumption, that camera perturbations remain within a small bounded region relative to the workspace, viewpoint normalization reduces to a scene- and policy-independent disocclusion task. Our work implements this idea with a 4M-parameter 3D-Gaussian canonicalizer prepended to a frozen VLA policy. Without modifying policy weights, GS-VLA improves performance across three orthogonal axes: (1) Policy architectures, (2) Unseen task suites, and (3) Perturbation scales. These results show that a lightweight visual module can recover a large fraction of the performance lost under viewpoint shift, without policy retraining. 1 Introduction Vision-Language-Action (VLA) models [3, 44, 32, 16, 15, 2, 11, 34, 7] have become a standard policy class for general manipulation. They perform strongly on benchmarks such as LIBERO [17] and CALVIN [21] and can transfer to real-world robots without additional modification. However, these successes often rely on an implicit assumption: the visual conditions at deployment closely match those used during training. In practice, even small deviations in camera viewpoint can lead to substantial performance degradation, highlighting viewpoint robustness as an important open challenge for reliable real-world deployment. However, improving robustness through additional data collection and policy fine-tuning is often impractical in real deployment settings, where new demonstrations are costly or unavailable. Also, there is a risk of policy collapse; trying to fine-tune the model for new camera views might ruin its original manipulation skills—a problem often called catastrophic forgetting. More fundamentally, performance drops under viewpoint shifts do not necessarily reflect failures in high-level reasoning or control capabilities; rather, they may stem from a mismatch between deployment-time observations and those encountered during training. This raises a broader question: should robustness to viewpoint variation be learned entirely within policy parameters, or can it be achieved by adapting the observation space instead? The latter observation space adaptation is not only possible but also cost-effective. In many real-world setups, camera changes are usually small and stay within a limited range. In practice, these perturbations are often small relative to the workspace, typically amounting to only a few to a few tens of centimeters. We refer to this practical property as Locality. Under this assumption, viewpoint changes affect only a limited portion of the image, while most pixels remain geometrically recoverable. Specifically, most of the visual content can be mapped to the canonical view using depth information and camera parameters. This means that learning is mainly needed for a narrow missing region, which can be treated as an inpainting problem. Importantly, this problem depends more on the visual domain than on the particular scene or policy. In this sense, viewpoint canonicalization is not a full novel-view-synthesis problem, but a simpler local adaptation problem that is largely independent of the scene and policy. We treat Locality as an empirically testable assumption. This makes viewpoint adaptation much more practical. Under Locality, there is no need for a large view-synthesis model designed for arbitrary scenes and camera baselines. Instead, a lightweight canonicalization module can be placed before a frozen VLA policy to restore aligned observations. This plug-and-play design recovers a substantial portion of the lost performance across different policy architectures, task suites, and perturbation scales without any policy retraining. The main contributions of this paper are as follows: • A new formulation of viewpoint adaptation for VLA policies: We show that viewpoint adaptation for frozen VLA policies can be addressed in observation space rather than in policy parameters, and formalize this perspective through Locality, a practical premise that reduces the problem to local disocclusion handling. • The first Gaussian-based canonicalization module for VLA policies: To the best of our knowledge, this is the first work to directly use a 3D Gaussian-based module for observation-space adaptation in VLA policies, thereby enabling efficient viewpoint canonicalization without policy retraining. The resulting module is lightweight and efficient to train, consisting of a 4M-parameter U-Net [27] adapter that predicts 14-dimensional Gaussian attributes and converges in only 5 hours on a single RTX A5000. • Generalization across policies, tasks, and perturbation scales: We demonstrate that a single checkpoint transfers across multiple policy architectures, unseen task suites, and large camera perturbations, recovering substantial performance without axis-specific re-training. • Empirical analysis of the recovery mechanism: We provide empirical evidence that most of the target view is handled by deterministic geometric mapping, while learning is needed only for a spatially localized residual region. Taken together, the above contributions suggest that viewpoint robustness for frozen VLA policies can be addressed effectively through lightweight observation-space adaptation, without modifying the policy itself. The rest of the paper is organized as follows. Firstly, prior works on VLA robustness and novel-view synthesis are reviewed. We then present our Locality-based formulation and the proposed 3D Gaussian canonicalization module. Finally, we evaluate the method across multiple policies, task suites, and perturbation scales, and provide an analysis of the recovery mechanism. 2 Related Works VLA Foundation Policies. Recent VLA models [3, 44, 2, 11, 16, 15, 25, 34, 7, 32, 42, 41, 18, 40, 13] unified multi-modal inputs under large Transformer backbones trained on massive cross-embodiment datasets. Although these models achieved ≥90%≥ 90\% success on LIBERO [17], their performances were predicated on static training camera setups. Unlike works that modified internal activations, we treat these multi-billion-parameter models as frozen, black-box policies and improve viewpoint robustness entirely in the observation space. This preserves the integrity of the policy while avoiding the costs of retraining. Camera-Robust Robot Learning. Standard approaches to viewpoint robustness relied on retraining policies under augmented camera distributions. SPARTN [43] utilized NeRF [22] to generate novel-view trajectories for behavior cloning. The concurrent AnyCamVLA [10] similarly froze the policy but relied on LVSM [12] synthesizer with dual-view inputs, with evaluation limited to narrow perturbations. On the other hand, we introduce a Locality-based reformulation that reduces the number of parameters by 42× compared with AnyCamVLA. Feed-Forward Novel-View Synthesis. Novel-view synthesis (NVS) generates images from new camera poses given one or more input views. Per-scene radiance fields [22, 23, 1] achieve high fidelity but require per-scene optimization, while feed-forward predictors [38, 33] generalize across scenes. More recently, diffusion-based generators [28, 35] synthesize plausible views even under large baselines, at the cost of tens to hundreds of millions of parameters. Our contribution lies not in proposing a new NVS architecture but in reformulating the problem itself: by restricting the task to a bounded ε -ball under the Locality premise, we reduce capacity requirements by one to two orders of magnitude. 3D Gaussian Splatting Variants. Building on 3DGS [14] and its differentiable rasterizer [37], recent feed-forward variants predict per-pixel Gaussians from one or more views [31, 30, 38, 6] and improve rendering quality and structure [39, 20]. Flash3D in particular leverages monocular depth priors but targets cross-scene generalization.Our canonicalizer follows the pixel-aligned line but specializes the task to in-distribution camera perturbations, so the U-Net only needs to shape per-pixel Gaussians within an ε -ball around the training pose rather than synthesize arbitrary views. Image Inpainting. Filling disocclusions resembles image inpainting [29, 26], but state-of-the-art inpainters require tens of millions of parameters to hallucinate semantic content over free-form holes. In our setting, ε -bounded camera shifts produce only O(ε )-thin holes along depth discontinuities, so no hallucination is needed and a 4M-parameter U-Net suffices VLA Robustness Benchmarks. LIBERO-Plus [8] recently showed that camera viewpoint is a major source of fragility for state-of-the-art VLAs. In response to this challenge, we introduce an observation-space remedy and evaluate its effectiveness across three axes: policy, task suite, and perturbation scale. Unlike concurrent approaches targeting other robustness dimensions, such as SpatialVLA [25], our observation-space remedy is orthogonal to internal policy modifications and can be combined with them to achieve more comprehensive robustness. 3 Proposed method This section presents our formulation of the Locality assumption as a practical module. We first define the objective of the canonicalizer, then show that Locality allows a computationally expensive novel-view synthesizer to be replaced with a lightweight U-Net. Additionally, the training procedure and the module deployment are described. 3.1 Motivation A VLA policy trained at a fixed camera pose tends to collapse once the camera moves at deployment; in our worst LIBERO suite, the success rate decreases from 92.4%92.4\% to 9.3%9.3\%. A natural remedy is to preprocess the observation back to the canonical view before feeding it to the policy. Although it seems like we need to solve a complex NVS problem, the changes in the camera position are actually very small. In practical settings, rig-mounted cameras typically drift by only a few centimeters, such that the majority of pixels remain visible after the pose change, while only a narrow region near depth discontinuities becomes newly exposed. That band has area O(ρ)O(ρ) in the perturbation magnitude ρ. A geometric argument for this area bound and the underlying geometry are detailed in Appendix A.1. Therefore, the core challenge of the learning problem reduces to inpainting a thin, newly disoccluded band along depth-discontinuity curves rather than synthesizing a general novel view. This makes our canonicalizer a natural fit: rather than generating the canonical image from scratch, it geometrically transports the visible source content into the canonical frame, leaving learning to handle only the narrow disoccluded regions. As a result, the learnable component remains highly lightweight—a 44M-parameter U-Net is all we need, which is over 25×25× smaller than generic feed-forward NVS predictors [12, 30, 31]. We embed this same locality structure inside the architecture: each source pixel is back-projected through its depth to anchor a Gaussian primitive, the U-Net shapes each primitive (depth-bounded center offset, anisotropic scale, orientation, opacity, and color residual), and differentiable α-splatting [37] composes them at the canonical pose, jointly resolving occlusion ordering and blending across the disoccluded bands. Architectural details and the 14-dim descriptor are provided in Section 3.3. 3.2 Problem definition Setup and notation. We first define the input-output interface of the viewpoint canonicalizer fϕf_φ. A frozen VLA policy πθ _θ is trained at a canonical pose (the camera pose seen during πθ _θ training), C⋆=(R⋆,t⋆)∈SE(3)C =(R ,t ) (3). Here, SE(3)SE(3) denotes the group of 3D rigid transformations, represented by a rotation and a translation without scale, and SO(3)SO(3) denotes the group of 3D rotation matrices. We write R⋆∈SO(3)R (3) for the camera rotation and t⋆∈ℝ3t ^3 for its world-space position. The camera intrinsics are given by the pinhole matrix K∈ℝ3×3K ^3× 3, which encodes the focal lengths and principal point. At deployment, the camera may move to a different source pose Cs=(Rs,ts)C_s=(R_s,t_s). For each frame, the canonicalizer receives an RGB image Is∈[0,1]H×W×3I_s∈[0,1]^H× W× 3, a per-pixel metric depth map Ds∈ℝ>0H×WD_s _>0^H× W aligned with IsI_s, and the camera parameters (Cs,C⋆,K)(C_s,C ,K). Here, Ds(u,v)D_s(u,v) denotes the depth, in meters, of the 3D point observed at pixel (u,v)(u,v) in IsI_s. Throughout the paper, we use the camera-to-world convention: a 3D point Xc∈ℝ3X_c ^3 expressed in the camera frame is mapped to the world frame by Xw=RXc+tX_w=RX_c+t. The goal of the canonicalizer is to predict the image that would have been observed from the canonical pose, which is formulated as: fϕ:(Is,Ds,Cs,C⋆,K)⟼I^⋆∈[0,1]H×W×3,f_φ:(I_s,D_s,C_s,C ,K) I ∈[0,1]^H× W× 3, (1) where I^⋆ I is the predicted canonical-view image. The frozen policy then acts on this canonicalized observation, i.e., a=πθ(I^⋆)a= _θ( I ). The policy parameters θ are never updated, and fϕf_φ operates strictly on a per-frame basis, without temporal aggregation, multi-view input, or per-scene optimization. Locality assumption. We formalize the Locality assumption described in Introduction Section. The deployment pose is assumed to remain within a bounded neighborhood of the canonical pose, with translation tolerance εt>0 _t>0 and rotation tolerance εR>0 _R>0. The Locality set, ℬ(C⋆,εt,εR)B(C ; _t, _R), is defined as: ℬ(C⋆;εt,εR):=(R,t):∥t−t⋆∥2≤εt,∥log((R⋆)⊤R)∥2≤εR,B(C ; _t, _R):= \(R,t): t-t _2≤ _t,\; \! ((R ) R ) _2≤ _R \, (2) where Cs∈ℬ(C⋆,εt,εR).C_s (C ; _t, _R). (3) In (2), the term ∥log((R⋆)⊤R)∥2 \! ((R ) R ) _2 represents the geodesic angle between the current rotation R and the canonical rotation R⋆R . In our experiments, we relate the rotation limit εR _R to the translation limit εt _t via a first-order scaling derived from the motion-field equation under a pinhole camera. Whereas translation-induced image motion scales inversely with depth, rotation induces depth-independent motion [19, 9]. Therefore, we use the approximation εR≈εt/dmin _R≈ _t/d_ , where dmind_ denotes the minimum scene depth. This allows us to control the overall camera perturbation using a single parameter, εt _t. To measure the total pose change, we define a normalized pose distance, ρ, as: ρ=‖ts−t⋆‖dmin+‖log(R⋆⊤Rs)‖,ρ= \|t_s-t \|d_ +\| (R R_s)\|, (4) which combines translation and rotation into a unified scalar. Under our Locality assumption, the fraction of newly exposed regions, η(ρ)η(ρ), grows linearly to leading order as η(ρ)=αparρpar+αfrρfr+O(ρ2)η(ρ)= _par\, _par+ _fr\, _fr+O(ρ^2) for ρ≤1ρ≤ 1 in matched units, with ρpar=‖ts−t⋆‖/dmin _par=\|t_s-t \|/d_ and ρfr=‖log(R⋆⊤Rs)‖ _fr=\| (R R_s)\|. The two leading coefficients depend only on scene geometry: αpar _par on the silhouette length Lpx⋆L _px and the relative depth contrast (dmax−dmin)/dmax(d_ -d_ )/d_ , and αfr _fr on the image-frame perimeter. We give a geometric argument for this bound, together with the explicit forms of αpar _par and αfr _fr, in Appendix A.1. We test this scaling empirically along three additional settings: Section 4.3 sweeps the perturbation scale εt∈5,15,30,50,100,200cm _t∈\5,15,30,50,100,200\\,cm; Section 4.5 stresses calibration noise; and Appendix A.2 verifies the linearity of η(ρ)η(ρ) directly. 3.3 Pixel-aligned 3D-Gaussian architecture The canonicalizer fϕf_φ is realized as follows. Under the Locality assumption, we only need to handle a thin disocclusion band, so we use a compact U-Net rather than a generic NVS backbone [14, 30, 31], which can reduce computational costs. A symmetric U-Net takes the channel-wise concatenation [Is∥Ds][I_s\,\|\,D_s] as input and is FiLM-modulated [24] by a pose-pair embedding, which is formulated as zpose=MLP(Cs,C⋆)∈ℝ128z_pose=MLP(C_s,C ) ^128. At every source pixel, the decoder predicts a 1414-dimensional Gaussian descriptor consisting of a position residual (Δμ μ), an anisotropic log-scale (logs s), an orientation quaternion (q), an opacity (α), and a color residual (Δc c). All heads are zero-initialized so that fϕf_φ starts exactly at the geometric warp. The Gaussian center is anchored at the world-space back-projection Xw=RsK−1Ds(u,v)[u,v,1]⊤+tsX_w=R_sK^-1D_s(u,v)[u,v,1] +t_s and is allowed only a depth-bounded residual, μ=Xw+cΔμDs(u,v)tanh(Δμ),cΔμ=0.1,μ\;=\;X_w\;+\;c_ μ\,D_s(u,v)\, ( μ), c_ μ=0.1, (5) which caps displacement at 10%10\% of local depth. This keeps the residual from drifting away from the geometric base, which we found to be important for stability. One Gaussian per source pixel, with RGB Is(u,v)+ΔcI_s(u,v)+ c, are rasterized at the canonical pose C⋆C by gsplat’s [37] differentiable α-blender, I^⋆(p)=∑i∈(p)Tiαici I (p)= _i (p)T_i _ic_i with Ti=∏j<i(1−αj)T_i= _j<i(1- _j). Occlusion is resolved by depth-sorting, and the α-blender itself fills the disocclusion band, so we do not need a separate inpainting head. The canonicalizer only requires 4M parameters, roughly 840 × smaller than finetuning π0.5 _0.5 [11] and 42 × smaller than AnyCamVLA [10]. 3.4 Training We intentionally avoid specialized training techniques so that the observed gains can be attributed to the Locality reduction rather than to the training method. We train on only one suite so that the remaining suites can serve as unseen test domains for cross-suite evaluation. The objective is the standard 3DGS [14] :0.20.8\!:\!0.2 L1L_1/SSIM reconstruction loss: ℒ=0.8‖I^⋆−I⋆‖1+0.2(−SSIM).L=0.8\| I \!-\!I \|_1+0.2 (1\!-\!SSIM ). (6) 4 Experiments In our experiments, we evaluate GS-VLA in terms of magnitude, generality, and deployment realism: we quantify the extent to which a canonicalizer recovers the performance lost under perturbations, relative to an upper bound given by the unperturbed (best-case) performance and a lower bound corresponding to the perturbed setting without canonicalization. We also evaluate whether a single checkpoint generalizes across different policies, benchmark suites, and perturbation levels ε . Additionally, we assess whether the performance gains persist under realistic deployment conditions, specifically in the presence of simultaneous translation–rotation drift and extrinsic-pose calibration error. All experiments use a single checkpoint trained once on libero_spatial at εt=100 _t=100 cm; no axis-specific retraining is performed. 4.1 Experimental setup Benchmark. We evaluate on the four LIBERO suites [17]—libero_spatial, libero_object, libero_goal, and libero_10—which cover diverse manipulation scenarios including spatial reasoning, object-centric interactions, and long-horizon tasks. Performance is measured by task success rate over multiple episodes. Camera randomization. We add random Gaussian noise to both the camera’s position and its look-at point around the canonical pose. The parameter εt _t controls the size of this noise. We then adjust the camera’s orientation so it directly faces the new look-at point. If a shifted camera falls below the table, we discard it and sample a new one. Our default setting uses εt≈100cm _t≈ 100\,cm, and we apply more noise in the sideways and vertical directions (orthogonal to the view) than in the depth direction. To test the model’s robustness at different noise levels, we simply scale this default setting. Figure 1 visualizes the resulting camera distribution. Finally, in order to isolate the effects of calibration errors, we inject independent location and rotation noise into the canonicalizer’s input. This simulates real-world scenarios in which the camera pose assumed by the system deviates slightly from the true pose. Figure 1: Visualization of the randomized camera distribution. Policies. The main policy is π0.5 _0.5 (lerobot/pi05_libero_finetuned) [4]. For cross-policy transfer, we additionally evaluate OpenVLA-OFT [15], RynnVLA-002 [5], and XVLA [42], where all policies remain frozen throughout. Because rollout uses each policy’s native action-chunk configuration, we compare Δ rather than absolute SR across policies. Baselines. We evaluate two baselines. No canon passes the perturbed observation to the policy without canonicalization. Fwd-warp applies the closed-form depth-warp and zeros out the disocclusion region, which isolates the geometric component. Finally, GS-VLA denotes our proposed 4M-parameter canonicalizer detailed in Section 3.3. 4.2 Experimental results Our first question is how much of the performance drop caused by camera perturbations can be recovered by a single canonicalizer across three critical axes: policy, suite, and perturbation magnitude. Table 1 summarizes these headline results. A single GS-VLA checkpoint consistently improves success rates (SR) across every evaluated setting. Consistent with our locality assumption, the gains are largest precisely where the unprotected baseline policy fails most severely. In these cases, a larger fraction of the raw input pixels is out-of-distribution, leaving greater room for the canonicalizer to restore useful visual structure. We break down the recovery along the three generalization axes: • Cross-policy generalization: GS-VLA effectively acts as a plug-and-play module for various architectures. As shown in Table 1(a), every frozen policy benefits without any fine-tuning. For instance, the pilot XVLA policy jumps from a near-total failure of 1.4%1.4\% to 81.0%81.0\% (+79.6 p). Similarly, the 7B-parameter OpenVLA-OFT recovers dramatically to 81.6%81.6\% (+61.8 p). Notably, the magnitude of the benefit anti-correlates with the policy’s intrinsic robustness, suggesting that the canonicalizer serves as a broadly effective protective layer across policies. • Cross-suite generalization: Because the scene structure depends only on silhouette geometry and not on task semantics, our module transfers zero-shot to unseen environments. As shown in Table 1(b), the most striking gain occurs on the long-horizon libero_10 suite, boosting π0.5 _0.5 from 9.3%9.3\% to 72.1%72.1\% (+62.8 p). We attribute this especially large improvement to reduced error accumulation over long trajectories: by keeping the inpainted region small at each step, the locality property limits frame-to-frame drift and prevents errors from compounding over time. • Perturbation-scale robustness: On the core training setting (π0.5 _0.5 at εt=100 _t=100 cm), GS-VLA improves SR from 42.6%42.6\% to 86.8%86.8\% (+44.2 p). Even under extreme noise (εt=200 _t=200 cm) where the baseline falls to 35.7%35.7\%, GS-VLA maintains high robustness at 78.5%78.5\% (+42.8 p), as seen in Table 1(c). Table 1: Three-axis generalization from a single checkpoint. GS-VLA is trained once on libero_spatial at εt=100 _t=100 cm, transfers without retraining across (a) policies, (b) suites, and (c) perturbation scales. Within each block,by descending Δ in (a) and (b), by descending εt _t in (c). Boldface indicates the best result in each row, while bold Δ highlights the headline cell of each block. The Fwd-warp column reports a depth-only forward-warp baseline where measured. Full per-cell matrices in Table 10 and Section 4.3. Policy / Setting Suite No canon Fwd-warp GS-VLA (ours) Δ (p) SR (%) SR (%) SR (%) (a) Cross-policy generalization (εt=100 _t=100 cm, 500500 ep) XVLA spatial 1.41.4 19.719.7 81.081.0 +79.6+79.6 OpenVLA-OFT (7 B) object 19.819.8 30.530.5 81.681.6 +61.8+61.8 RynnVLA-002 spatial 25.825.8 37.837.8 78.678.6 +52.8+52.8 π0.5 _0.5 (3.4 B) spatial 42.642.6 56.556.5 86.886.8 +44.2+44.2 (b) Cross-suite generalization (π0.5 _0.5, εt=100 _t=100 cm, 10001000 ep) π0.5 _0.5 libero_10 9.39.3 22.922.9 72.172.1 +62.8+62.8 π0.5 _0.5 spatial 42.642.6 56.556.5 86.886.8 +44.2+44.2 π0.5 _0.5 object 64.064.0 74.674.6 90.290.2 +26.2+26.2 π0.5 _0.5 goal 66.366.3 72.472.4 92.292.2 +25.9+25.9 (c) Perturbation-scale robustness (π0.5×spatial _0.5\!×\! spatial, 10001000 ep) εt=200 _t=200 cm spatial 35.735.7 49.549.5 78.578.5 +42.8+42.8 εt=100 _t=100 cm spatial 42.642.6 56.556.5 86.886.8 +44.2+44.2 εt=50 _t=50 cm spatial 60.960.9 66.066.0 87.287.2 +13.7+13.7 εt=5 _t=5 cm spatial 85.285.2 87.587.5 88.088.0 +2.8+2.8 4.3 Scaling analysis In order to know how performance changes as the perturbation grows, we sweep εt∈5,15,30,50,100,200 _t∈\5,15,30,50,100,200\ cm across all suites (Tables 2, 12). Three distinct regimes emerge: on spatial and object, the unprotected policy stays near the performance ceiling for small εt _t, with the canonicalizer’s benefit (Δ ) growing sharply only past 5050 cm. On goal, the gap remains consistently large (+16+16 to +30+30 p) across all values of εt _t, indicating that the policy is highly sensitive even to modest camera drift. On libero_10, the canonicalizer outperforms the baseline across all perturbation levels, with the gain peaking at a +62.8 p gain at εt=100 _t=100 cm before slightly tapering off at 200200 cm as the perturbation approaches the Locality boundary (i.e., as ρ→1ρ→ 1). Within the εt≤100 _t≤ 100 cm range, the Δ(εt) ( _t) curve is approximately linear, and the per-suite slopes match our predicted αpar _par ratios to within 11%11\%. This trend is consistent with the O(ρ)O(ρ) bound in Appendix A.1: larger failures of the unprotected policy leave more room for the canonicalizer to recover performance. Table 2: εt _t-sweep on π0.5×libero_spatial _0.5\!×\! libero\_spatial. The baseline drops 49.549.5 p across the sweep; GS-VLA drops only 9.59.5 p. Term Δ grows approximately linearly in εt _t within the Locality bound, matching Appendix A.1; the largest cell (εt=100 _t=100 cm) is in bold. Method \ εt _t (cm) 55 1515 3030 5050 100100 200200 No canon 85.285.2 82.782.7 80.680.6 73.573.5 42.642.6 35.735.7 GS-VLA (ours) 88.088.0 88.688.6 87.687.6 87.287.2 86.886.8 78.578.5 Δ (p) +2.8+2.8 +5.9+5.9 +7.0+7.0 +13.7+13.7 +44.2+44.2 +42.8+42.8 4.4 Joint translation–rotation So far, we mainly varied translation and kept rotation small. To check that the picture survives once translation and rotation move together, we cross ε∈100,200 ∈\100,200\ cm with yaw θ∈10∘,20∘θ∈\10 ,20 \ in Table 3. Δ stays in the [+21,+42][+21,+42] p range across all four cells. Two trends are visible. First, larger ε actually widens Δ : the baseline drops faster than the canonicalizer, so the gap between them grows. Second, larger θ narrows Δ , which we attribute to undersampling of large rotations at training time — a natural target for follow-up work, since it can be addressed entirely on the data side without changing the architecture. Table 3: Joint translation × rotation. on π0.5×libero_spatial _0.5\!×\! libero\_spatial. Entries: No canon → GS-VLA (Δ ), in %\% SR. The bold Δ flags the largest gain (+42.2+42.2 p). Yaw θ ε=100 =100 cm ε=200 =200 cm θ=10∘θ=10 →81.850.0\!→\!81.8 (+31.8+31.8) →78.035.8\!→\!78.0 (+42.2+42.2) θ=20∘θ=20 →66.545.5\!→\!66.5 (+21.0+21.0) →69.334.8\!→\!69.3 (+34.5+34.5) 4.5 Extrinsic-pose noise The canonicalizer assumes that the deployment pose is known. In practice, it is only known up to calibration error, so we ask how much error is tolerated. To answer this, we inject (T cm,R∘)(T cm,R ) of pose noise on top of ε=100 =100 cm, applied only to the pose seen by the canonicalizer (Table 4). Within roughly 33 cm and 3∘3 of calibration error, the canonicalizer remains stable, with a three-suite mean drop of at most 55 p — comparable to ordinary camera re-mounting tolerances. Pushed to 55 cm and 5∘5 , the long-horizon libero_10 suite begins to suffer (−23.8-23.8 p), which marks the practical limit of the calibration noise GS-VLA can absorb. Intrinsics noise is far less damaging: across the full focal-length sweep we report in Table 15, |Δ|| | stays within 2.82.8 p, so for the perturbation scales we care about, errors in the pinhole intrinsics matter much less than errors in the extrinsic pose. Table 4: Calibration stress. Pose noise (T cm,R∘)(T\, cm,R ) injected on top of ε=100 =100 cm and applied only to the pose seen by the canonicalizer. The shaded zero-noise row is the GS-VLA reference; degradation is graceful up to ∼3 \!3 cm/3∘3 , with long-horizon libero_10 the most sensitive at 55 cm/5∘5 . Calib. noise (T,R)(T,R) object spatial goal libero_10 SR (%) SR (%) SR (%) SR (%) (0,0)(0,0) 89.289.2 79.979.9 88.588.5 65.465.4 (1,1)(1,1) 87.387.3 78.278.2 86.686.6 63.463.4 (2,2)(2,2) 86.186.1 75.075.0 86.586.5 59.959.9 (3,3)(3,3) 84.084.0 76.076.0 85.485.4 51.651.6 (5,5)(5,5) 78.478.4 70.370.3 81.081.0 41.641.6 Δ(5,5) _(5,5) −10.8-10.8 −9.6-9.6 −7.5-7.5 −23.8-23.8 5 Ablations This section isolates which parts of the canonicalizer matter, and whether the picture changes when we swap out the depth source. Capacity. A natural worry with a 44 M-parameter module is that more capacity would help. Doubling the base width to C=64C=64 (1515 M parameters) raises reconstruction PSNR from 26.8426.84 to 28.428.4 dB but actually lowers SR to 85.2%85.2\% (Table 13). Beyond about 4M-parameters, reconstruction quality and policy success decouple: the extra capacity goes into texture detail that the policy never reads, so it does not translate into higher SR. This supports our intentionally compact design. Data. Training set size shows the same kind of saturation. Sweeping pair counts in 10k,25k,50k\10k,25k,50k\ yields 8484–87%87\% SR, with the signal already saturated by about 1010k pairs (Table 14). This is consistent with the low intrinsic learning complexity that Locality predicts — the inpainting region is small, so it does not take many examples to cover it. Depth substitution. GS-VLA depends on calibrated metric depth, taken from a sensor or pre-calibrated estimator in our main experiments. Replacing this with zero-shot DepthAnything V2 [36], even with per-image analytical scale/shift calibration, yields only 44.5%44.5\% SR — the geometric base is no longer metric-aligned and per-frame calibration cannot recover the absolute scale required by (5). A short fine-tune of the depth backbone on 5050 k LIBERO frames (3 epochs) bridges most of this gap, reaching 71.7%71.7\% SR (−15.1-15.1 p from the GT-depth ceiling at 86.8%86.8\% and +27.2+27.2 p over calibration alone, Table 16). Beyond this point, neither doubling the depth-encoder capacity (ViT-S→ -B→ -L), appending a 0.8M-parameter UNet residual refinement head, nor passing end-to-end gradients through the splat module produced further SR gains; the refinement head in particular collapsed to a zero residual, indicating that the fine-tuned depth is already L1L_1-optimal at the resolution our pipeline consumes. We therefore read the remaining −15-15 p gap as a structural distribution mismatch and the splat module’s information bottleneck, rather than a depth-accuracy limitation. 6 Limitation We close with the boundaries within which our results should be read. Single-view input. The canonicalizer in this paper consumes a single source frame and its calibrated depth, so any scene content that the source camera does not observe at all cannot be recovered by the in-painter. We expect a multi-view extension — for example, fusing a wrist-mounted camera or a previous-step frame — to lift this ceiling without changing the underlying Locality reduction, and we leave that direction to future work. Locality boundary on rotation. The translation budget in our default setting is already large (εt=100 _t=100 cm relative to a ∼0.7 \!0.7 m workspace), and adding meaningful rotation on top of such translations frequently rotates the camera off the workspace entirely. We could not run a clean rotation sweep beyond the joint (εt,θ)( _t,θ) cells reported in Table 3 for this reason. Inspecting the failure-case rollouts, we found that a non-trivial fraction of the sampled cameras do not contain the workspace inside their frustum at all, which marks the practical ceiling of the perturbation distribution we used rather than a property of the canonicalizer itself. Real-world evaluation. All experiments in this paper are run inside the LIBERO simulator. Validating GS-VLA on a real robot requires both an external metric-depth source (per the previous point) and the engineering effort to instrument a physical rig, neither of which we were able to put in place within the resource and personnel constraints of this project. We therefore report the simulator results as a strong but ultimately preliminary signal, and treat a real-robot replication as the natural follow-up. 7 Conclusion We have presented GS-VLA, a 4M-parameter 3D-Gaussian canonicalizer that improves the camera-viewpoint robustness of frozen VLA policies without any policy updates. The design follows from a Locality reduction: under bounded perturbations, viewpoint canonicalization becomes a scene -and policy-independent disocclusion problem over an O(ρ)O(ρ)-area region ( A.1; the η(ρ)η(ρ) linearity link is confirmed at R2≥0.99R^2\!≥\!0.99, with the full ε→η→residual→Δ \!→\!η\!→\!residual\!→\! chain closing at R2∈0.99, 0.81, 1.00R^2\!∈\!\0.99,\,0.81,\,1.00\). The practical consequence is that a single checkpoint trained on libero_spatial at εt=100 _t=100 cm transfers without modification across 44 policies, 44 LIBERO suites (with a 4-suite mean of +39.8+39.8 p), and the full range εt∈[5,200] _t∈[5,200] cm, while still tolerating roughly 33 cm and 3∘3 of calibration noise. We see this as evidence for a broader reframing: “make a VLA camera-robust” can be turned from an axis-specific retraining problem into a visual-domain premise that a small observation-space module is enough to absorb. The same template should extend naturally to embodiment and lighting variation, once the corresponding “local” structure for those axes is identified. References [1] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman (2022) Mip-nerf 360: unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5470–5479. Cited by: §2. [2] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) pi_0pi\_0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164. Cited by: §1, §2. [3] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. (2022) Rt-1: robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817. Cited by: §1, §2. [4] R. Cadene, S. Aliberts, F. Capuano, M. Aractingi, A. Zouitine, P. Kooijmans, J. Choghari, M. Russi, C. Pascal, S. Palma, et al. (2026) Lerobot: an open-source library for end-to-end robot learning. arXiv preprint arXiv:2602.22818. Cited by: §A.3.3, §4.1. [5] J. Cen, S. Huang, Y. Yuan, K. Li, H. Yuan, C. Yu, Y. Jiang, J. Guo, X. Li, H. Luo, et al. (2025) Rynnvla-002: a unified vision-language-action and world model. arXiv preprint arXiv:2511.17502. Cited by: §4.1. [6] Y. Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T. Cham, and J. Cai (2024) Mvsplat: efficient 3d gaussian splatting from sparse multi-view images. In European conference on computer vision, p. 370–386. Cited by: §2. [7] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), p. 1684–1704. Cited by: §1, §2. [8] S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. (2025) Libero-plus: in-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626. Cited by: §2. [9] R. Hartley and A. Zisserman (2003) Multiple view geometry in computer vision. Cambridge university press. Cited by: §3.2. [10] H. Heo, S. Woo, S. M. Kim, J. Kim, J. Lee, Y. Lee, and Y. M. Kim (2026) AnyCamVLA: zero-shot camera adaptation for viewpoint robust vision-language-action models. arXiv preprint arXiv:2603.05868. Cited by: §2, §3.3. [11] P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. (2025) pi0.5pi_0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054. Cited by: §1, §2, §3.3. [12] H. Jin, H. Jiang, H. Tan, K. Zhang, S. Bi, T. Zhang, F. Luan, N. Snavely, and Z. Xu (2024) Lvsm: a large view synthesis model with minimal 3d inductive bias. arXiv preprint arXiv:2410.17242. Cited by: §2, §3.1. [13] T. Ke, N. Gkanatsios, and K. Fragkiadaki (2024) 3d diffuser actor: policy diffusion with 3d scene representations. arXiv preprint arXiv:2402.10885. Cited by: §2. [14] B. Kerbl, G. Kopanas, T. Leimkühler, G. Drettakis, et al. (2023) 3d gaussian splatting for real-time radiance field rendering.. ACM Trans. Graph. 42 (4), p. 139–1. Cited by: §2, §3.3, §3.4. [15] M. J. Kim, C. Finn, and P. Liang (2025) Fine-tuning vision-language-action models: optimizing speed and success. arXiv preprint arXiv:2502.19645. Cited by: §1, §2, §4.1. [16] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. (2024) Openvla: an open-source vision-language-action model. arXiv preprint arXiv:2406.09246. Cited by: §1, §2. [17] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) Libero: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems 36, p. 44776–44791. Cited by: §1, §2, §4.1. [18] S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu (2024) Rdt-1b: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864. Cited by: §2. [19] H. C. Longuet-Higgins and K. Prazdny (1980) The interpretation of a moving retinal image. Proceedings of the Royal Society of London. Series B. Biological Sciences 208 (1173), p. 385–397. Cited by: §3.2. [20] T. Lu, M. Yu, L. Xu, Y. Xiangli, L. Wang, D. Lin, and B. Dai (2024) Scaffold-gs: structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 20654–20664. Cited by: §2. [21] O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard (2022) Calvin: a benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters 7 (3), p. 7327–7334. Cited by: §1. [22] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng (2021) Nerf: representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65 (1), p. 99–106. Cited by: §2, §2. [23] T. Müller, A. Evans, C. Schied, and A. Keller (2022) Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG) 41 (4), p. 1–15. Cited by: §2. [24] E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville (2018) Film: visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §3.3. [25] D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al. (2025) Spatialvla: exploring spatial representations for visual-language-action model. arXiv preprint arXiv:2501.15830. Cited by: §2, §2. [26] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10684–10695. Cited by: §2. [27] O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, p. 234–241. Cited by: 2nd item. [28] K. Sargent, Z. Li, T. Shah, C. Herrmann, H. Yu, Y. Zhang, E. R. Chan, D. Lagun, L. Fei-Fei, D. Sun, et al. (2024) Zeronvs: zero-shot 360-degree view synthesis from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9420–9429. Cited by: §2. [29] R. Suvorov, E. Logacheva, A. Mashikhin, A. Remizova, A. Ashukha, A. Silvestrov, N. Kong, H. Goka, K. Park, and V. Lempitsky (2022) Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, p. 2149–2159. Cited by: §2. [30] S. Szymanowicz, E. Insafutdinov, C. Zheng, D. Campbell, J. F. Henriques, C. Rupprecht, and A. Vedaldi (2025) Flash3d: feed-forward generalisable 3d scene reconstruction from a single image. In 2025 International Conference on 3D Vision (3DV), p. 670–681. Cited by: §2, §3.1, §3.3. [31] S. Szymanowicz, C. Rupprecht, and A. Vedaldi (2024) Splatter image: ultra-fast single-view 3d reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10208–10217. Cited by: §2, §3.1, §3.3. [32] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. (2024) Octo: an open-source generalist robot policy. arXiv preprint arXiv:2405.12213. Cited by: §1, §2. [33] Q. Wang, Z. Wang, K. Genova, P. P. Srinivasan, H. Zhou, J. T. Barron, R. Martin-Brualla, N. Snavely, and T. Funkhouser (2021) Ibrnet: learning multi-view image-based rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4690–4699. Cited by: §2. [34] J. Wen, Y. Zhu, J. Li, Z. Tang, C. Shen, and F. Feng (2025) Dexvla: vision-language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05855. Cited by: §1, §2. [35] R. Wu, B. Mildenhall, P. Henzler, K. Park, R. Gao, D. Watson, P. P. Srinivasan, D. Verbin, J. T. Barron, B. Poole, et al. (2024) Reconfusion: 3d reconstruction with diffusion priors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 21551–21561. Cited by: §2. [36] L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao (2024) Depth anything v2. Advances in Neural Information Processing Systems 37, p. 21875–21911. Cited by: §5. [37] V. Ye, R. Li, J. Kerr, M. Turkulainen, B. Yi, Z. Pan, O. Seiskari, J. Ye, J. Hu, M. Tancik, et al. (2025) Gsplat: an open-source library for gaussian splatting. Journal of Machine Learning Research 26 (34), p. 1–17. Cited by: §A.3.3, §2, §3.1, §3.3. [38] A. Yu, V. Ye, M. Tancik, and A. Kanazawa (2021) Pixelnerf: neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4578–4587. Cited by: §2, §2. [39] Z. Yu, A. Chen, B. Huang, T. Sattler, and A. Geiger (2024) Mip-splatting: alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 19447–19456. Cited by: §2. [40] M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine (2024) Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693. Cited by: §2. [41] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705. Cited by: §2. [42] J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y. Feng, Y. Zheng, J. Zou, Y. Chen, J. Zeng, et al. (2025) X-vla: soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274. Cited by: §2, §4.1. [43] A. Zhou, M. J. Kim, L. Wang, P. Florence, and C. Finn (2023) Nerf in the palm of your hand: corrective augmentation for robotics via novel-view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 17907–17917. Cited by: §2. [44] B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. (2023) Rt-2: vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, p. 2165–2183. Cited by: §1, §2. Appendix A Appendix A.1 Proposition 1 of locality reduction Under the Locality assumption Cs∈ℬ(C⋆,εt,εR)C_s (C , _t, _R) of Section 4.1, the canonical view partitions into a region recoverable by the closed-form depth-warp (the world point Xw=RsK−1Ds(u,v)[u,v,1]⊤+tsX_w\!=\!R_sK^-1D_s(u,v)[u,v,1] \!+\!t_s re-projected to the canonical pixel via ΠK(R⋆⊤(Xw−t⋆)) _K(R (X_w\!-\!t )), no learnable parameters) and a thin disocclusion band of fractional area η∈[0,1]η∈[0,1] along depth edges, which has no defining input pixel and must be filled by a learned prior. Proposition 1 (Locality Reduction). Assume piecewise-smooth depth d∈[dmin,dmax]d∈[d_ ,d_ ] and silhouette edges that meet view rays transversely. Decompose ρ=ρpar+ρfrρ= _par+ _fr with ρpar=‖ts−t⋆‖/dmin _par=\|t_s-t \|/d_ and ρfr=‖log(R⋆⊤Rs)‖ _fr=\| (R R_s)\|. Then for every ρ≤1ρ≤ 1, clowρ≤η(Cs,C⋆)≤αparρpar+αfrρfr+C2ρ2,c_low\,ρ\;≤\;η(C_s,C )\;≤\; _par\, _par+ _fr\, _fr+C_2\,ρ^2, where αpar=Lpx⋆πApx⋅dmax−dmindmax,αfr=PpxApx, _par= L _pxπ\,A_px· d_ -d_ d_ , _fr= P_pxA_px, Lpx⋆L _px is the canonical-view silhouette arc-length, PpxP_px is the image-frame perimeter, Apx=HWA_px=HW the image area, and clow,C2c_low,C_2 depend only on scene-regularity bounds. The two-term form makes explicit that the parallax contribution scales with translation and depth contrast, while the frame contribution scales with rotation and image perimeter. Notation. We denote the parallax and frame contributions to η as ηpar _par and ηfr _fr respectively, with η=ηpar+ηfrη= _par+ _fr. Pure translation has ηfr≠0 _fr≠ 0 as well (translation also induces some image-plane motion); pure rotation has ηpar=0 _par=0 since rotation introduces no depth-parallax. The upper bound integrates a parallax shadow band along each silhouette; a matching lower bound via Cauchy–Schwarz on the silhouette-normal field certifies that the linear rate cannot be improved. Empirical validation in Appendix A.2 confirms the linear regime to ρ≈0.5ρ≈ 0.5 with R2≥0.99R^2\!≥\!0.99. Corollary 1 (Same-domain transfer). Since αpar _par and αfr _fr depend only on silhouette–depth and image-frame geometry, not on texture, lighting, or policy, a canonicalizer trained on one suite transfers to others in the same visual domain. Verified in Section 4.2 and 1: the same checkpoint improves every policy and every off-training suite. Success-rate coupling. A residual-aware Lipschitz argument composes the area bound into a success-rate drop Δ(ρ)=Lπ⋅crecon⋅(αparρpar+αfrρfr)+O(ρ2) (ρ)=L_π· c_recon·( _par _par+ _fr _fr)+O(ρ^2), where LπL_π is the policy’s local Lipschitz constant on canonical inputs and creconc_recon the residual–η slope of fϕf_φ. The full chain ε→η→residual→Δ \!→\!η\!→\!residual\!→\! closes numerically in Appendix A.2 (Table 6). A.2 Empirical validation of proposition 1 We test whether the Locality picture really holds at the level of individual quantities, not only at the level of Δ . Proposition A.1 is tested directly under controlled synthetic perturbations: 100100 canonical frames combined with 1010 perturbations each, using ground-truth depth and no policy in the loop. Linearity. The first prediction is that η should grow linearly in ρ inside the Locality ball. We find η(ρ)η(ρ) is linear with R2≥0.99R^2≥ 0.99 for ρ≤0.5ρ≤ 0.5 across translation, rotation, and joint perturbation modes, and across both spatial and object. On every (mode, suite) cell the slope satisfies η(ρ)/ρ∈[0.27,1.09]η(ρ)/ρ∈[0.27,1.09], which confirms the two-sided bound of Proposition A.1. Departure from linearity begins around ρ≈0.5ρ≈ 0.5, well inside the validity range the proposition predicts. Scene decomposition. A second, sharper prediction comes from Corollary A.1: the rotation contribution to η depends only on image-plane perimeter and should therefore be scene-invariant, while the translation contribution depends on silhouette length L⋆L and should not be. The data follow this split closely (Table 5). The rotation slope is essentially the same on both suites (1.0841.084 on spatial vs. 1.0891.089 on object, a 0.4%0.4\% discrepancy), while the translation slope varies by about 60%60\%. Predicting the per-suite translation-slope ratio from L⋆⋅κL ·κ matches the observed value to within 11%11\%. Silhouette hugging. The geometry implied by Proposition A.1 also predicts where the disocclusion pixels live: along depth silhouettes, not spread uniformly across the image. We see exactly that. Occluded pixels lie 55–77 px from the nearest depth edge on both suites, against 2727–3333 px for uniform-random pixels — a roughly ×5\!× tighter localization, in line with the geometry the proposition predicts. End-to-end chain. Finally, we verify that the bound on η propagates all the way to the success-rate drop Δ . On 200200 pairs the residual–η link is well captured by a single line (Pearson r=0.90r=0.90, R2=0.81R^2=0.81, crecon=0.049c_recon=0.049). Composing the full chain ε→η→residual→Δ \!→\!η\!→\!residual\!→\! implies a slope of L≈213L≈ 213 p per L1L_1-residual unit. To check that this is physically plausible we estimate the policy’s local Lipschitz behaviour from the released π0.5 _0.5 weights using power iteration: log∏l(1+Ll)=389 _l(1+L_l)=389 across 421421 weight matrices, comfortably above the value implied by L. All four links close numerically with R2∈0.99,0.81,1.00R^2∈\0.99,0.81,1.00\ (Table 6). Taken together, these checks confirm that the Locality picture is not just consistent with the headline numbers but actually predicts the intermediate quantities they decompose into. A.2.1 Locality slopes by mode × suite Direct test of Proposition A.1 and Corollary A.1. We fit η(ρ)η(ρ) separately for translation, rotation, and joint perturbations on spatial and object; numerical discussion is in Section A.2. Table 5: Empirical slopes of η(ρ)η(ρ). Per-mode slopes for parallax (ηpar _par), frame (ηfr _fr), and total (ηtot _tot). Silhouette-hugging: occluded pixels average 5.055.05/6.786.78 px from depth edges (spatial/object) vs. 26.7726.77/33.1033.10 px for uniform-random pixels (×5\!× tighter). Object suite uses translation- and rotation-only fits; joint mode pending. Mode ηpar _par ηfr _fr ηtot _tot R2R^2 libero_spatial (ρ≤0.5ρ≤ 0.5) Translation 0.0820.082 0.2810.281 0.3630.363 0.9990.999 Rotation <0.001<0.001 1.0841.084 1.0841.084 0.9950.995 Joint 0.0380.038 0.6070.607 0.6450.645 0.9940.994 libero_object (ρ≤0.5ρ≤ 0.5) Translation 0.0500.050 – – – Rotation – 1.0891.089 – – A.2.2 Success-rate coupling chain Quantitative check of the full ε→η→residual→Δ \!→\!η\!→\!residual\!→\! chain proposed in Appendix A.1. The geometric (R2=0.99R^2=0.99) and policy-Lipschitz (R2=1.00R^2=1.00) links are tight; the η→residualη\!→\!residual link is the loosest at R2=0.81R^2=0.81, but the composite end-to-end fit still closes at R2=0.996R^2=0.996 within the linear regime (ε≤50 ≤ 50 cm). Table 6: Success-rate coupling chain (Appendix A.1). Each individual link and the end-to-end drop fit linearly with high R2R^2. The implied L≈213L≈ 213 p per L1L_1-residual unit lies inside the spectral-norm bound for π0.5 _0.5 (log∏l(1+Ll)=389 _l(1+L_l)=389). Link Quantity Slope R2R^2 ε→η →η joint η/ρη/ρ slope (Tab. 5) ≈0.645≈0.645 0.990.99 η→residualη predictor linearity crecon=0.049c_recon=0.049 0.810.81 residual→Δresidual→ policy Lipschitz L≈213L≈ 213 p/L1 1.001.00 ε→Δ → (composite) end-to-end (ε≤50 ≤ 50) 6.676.67 p/ρ 0.9960.996 A.3 Environments A.3.1 Implementation detail We summarize the architecture, training configuration, and runtime cost in Tables 7–9, and describe data composition and the software stack. Table 7: Canonicalizer architecture. With the listed initialization, fϕf_φ starts at the closed-form forward warp at iteration 0 and learns only a residual. Component Specification Backbone Symmetric U-Net, base width C=32C=32, 4 down-/up-sampling levels Conv block FiLM-modulated ConvBlock at every scale Pose embedding zpose=MLP([ts,qs,t⋆,q⋆])∈ℝ128z_pose\!=\!MLP([t_s,q_s,t ,q ]) ^128, 2-layer Input [Is∥Ds]∈ℝ4×H×W[I_s\,\|\,D_s] ^4× H× W, H=W=256H=W=256 Per-pixel output 14-dim Gaussian descriptor (Δμ,logs,q,α,Δc)( μ, s,q,α, c) Centre clamp μ=Xw+0.1Dstanh(Δμ)μ=X_w+0.1\,D_s\, ( μ) (≤10%≤ 10\% of local depth) Initialization residual heads zero-init; opacity bias set so that α=1α=1 at step 0 Parameters 4,047,8244,047,824 (≈CLOSE(≈ 4M); encoder ≈60%≈ 60\%, decoder ≈40%≈ 40\% Table 8: Training configuration. Only the canonicalizer parameters are updated; the policy is never touched. No discriminator or perceptual GAN loss is used. Hyperparameter Value Optimizer AdamW (β1=0.9 _1=0.9, β2=0.999 _2=0.999, weight decay 10−410^-4) Base learning rate ×10−55\!×\!10^-5 LR schedule 500500-step linear warmup (start factor 0.010.01), then cosine to 00 Batch size 1616 Epochs 5050 Gradient clipping global norm 1.01.0 L1L_1 : SSIM weight 0.8:0.20.8:0.2 Scale cap weight λs _s 10−310^-3 on ReLU(logs−2)ReLU( s-2) Position reg. λμ _μ 10−210^-2 on |Δμ|/Ds| μ|/D_s Background augmentation b∼([0,1]3)b ([0,1]^3), resampled per step Image augmentation horizontal flip only Random seed 4242; ±0.6± 0.6 p variation across 33 independent seeds Hardware ×1\!× RTX A5000 (24 GB), peak memory ≈18≈ 18 GB Wall-clock ≈5≈ 5 h (50×50050× 500 s) Best val. PSNR 26.8426.84 dB Table 9: Inference latency. The downstream policy keeps its native action-chunk schedule (e.g., 5050 for π0.5 _0.5), so the canonicalizer cost is amortized across many actions per frame. Softmax splatting is not used in the released checkpoint. Stage Per-frame cost (A5000) U-Net forward ≈5≈ 5 ms Differentiable α-splat (gsplat) ≈8≈ 8 ms Total canonicalizer fϕf_φ ≈ 13 ms (≈75≈ 75 Hz) Rasterizer config 45∘45 vertical FOV, bilinear splatting A.3.2 Data setup We train on 50,00050,000 source→ pairs sampled from libero_spatial (10 tasks, multiple seeds per task), with 49,80049,800 used for training and 200200 held out for validation. Each pair is generated by rolling out the simulator at the canonical camera, then re-rendering the same scene under a perturbed camera Cs∼ℬ(C⋆,εt=100cm,εR)C_s (C ,\, _t=100\,cm,\, _R) with εR≈εt/dmin _R≈ _t/d_ . We deliberately did not use colour jitter, multi-suite mixing, or per-task balancing, so the cross-suite results in Section 4.2 are zero-shot. A.3.3 Software framework PyTorch 2.42.4, CUDA 12.112.1, gsplat [37] 0.1.x0.1.x, robosuite 1.41.4 for LIBERO simulation, and lerobot [4] for the policy interface. Random seeds, full hyperparameter dumps, and the training command lines used to produce every released checkpoint are bundled with the released code. A.4 Additional experimental results and ablation studies A.4.1 Full cross-policy × suite matrix This appendix expands Table 1(a) into the full policy × suite matrix at εt=100 _t=100 cm. Every cell is positive, and the largest gains track baselines closest to total failure — consistent with Locality predicting larger headroom where more pixels go out-of-distribution. Table 10: Full cross-policy × suite matrix with εt=100 _t=100 cm. Bold Δ values denote the absolute increase in success rate (percentage points) for key configurations, such as XVLA (+79.6+79.6 p), π0.5 _0.5 on libero_10 (+62.8+62.8 p), and OpenVLA-OFT on object-centric tasks (+61.8+61.8 p). Policy Suite No canon GS-VLA (ours) Δ (p) π0.5 _0.5 (3.4 B) spatial 42.642.6 86.886.8 +44.2+44.2 object 64.064.0 90.290.2 +26.2+26.2 goal 66.366.3 92.292.2 +25.9+25.9 libero_10 9.39.3 72.172.1 +62.8+62.8 OpenVLA-OFT (7 B) spatial 87.287.2 92.092.0 +4.8+4.8 object 19.819.8 81.681.6 +61.8+61.8 goal 80.480.4 91.091.0 +10.6+10.6 libero_10 8.88.8 47.047.0 +38.2+38.2 RynnVLA-002 spatial 25.825.8 78.678.6 +52.8+52.8 goal 29.029.0 75.875.8 +46.8+46.8 libero_10 53.453.4 65.265.2 +11.8+11.8 XVLA spatial 1.41.4 81.081.0 +79.6+79.6 goal 5.45.4 80.080.0 +74.6+74.6 A.4.2 GS-VLA absolute SR across suites (ε -sweep) Companion to Table 2: GS-VLA absolute SR for all four suites as εt _t varies. object and goal stay within ∼2 \!2 p of their small-ε ceilings up to εt=100 _t=100 cm; the noticeable drop at 200200 cm marks the per-suite Locality boundary. Table 11: GS-VLA absolute SR (%) across suites as εt _t varies. object and goal stay within ∼2 \!2 p of their εt=5 _t=5 ceiling up to εt=100 _t=100 cm; the main drop appears at εt=200 _t=200 cm—the per-suite Locality boundary. Suite (GS-VLA) εt=5 _t=5 1515 3030 5050 100100 200200 spatial 88.088.0 88.688.6 87.687.6 87.287.2 86.886.8 78.578.5 object 92.292.2 92.292.2 91.491.4 92.792.7 90.290.2 84.784.7 goal 93.993.9 93.693.6 92.492.4 93.293.2 92.092.0 85.785.7 libero_10 71.971.9 72.772.7 73.473.4 70.870.8 72.172.1 52.952.9 A.4.3 Magnitude sweep Stress test pushing position σ and look-at σ jointly to ×2\!× and ×3\!× the default. Δ stays large and grows monotonically — No canon decays faster than the canonicalizer, so the gap widens with magnitude until the Locality boundary. Table 12: Magnitude sweep on π0.5×libero_spatial _0.5\!×\! libero\_spatial. While the performance drop (Δ ) increases with perturbation magnitude, the canonicalizer exhibits significantly slower decay compared to the No canon baseline. This gap (utility) widens progressively until the Locality boundary. †The εt=15 _t=15 row follows the single-scalar protocol from Table 2. ‡Labels ×2\!×/×3\!× are nominal; position σ is capped to remain within the workspace, and lookat σ is fixed at ×1\!×. Condition pos σ (m) lookat σ No canon GS-VLA (ours) Δ (p) εt=15 _t=15 (low)† n/a n/a 82.782.7 88.688.6 +5.9+5.9 ×1\!× (default, εt=100 _t=100) (0.3,0.5,0.25)(0.3,0.5,0.25) 0.060.06 42.642.6 86.886.8 +44.2+44.2 ×2\!× (bigrand)‡ (0.6,0.8,0.5)(0.6,0.8,0.5) 0.060.06 41.641.6 80.880.8 +39.2+39.2 ×3\!× (extreme)‡ (0.9,1.2,0.75)(0.9,1.2,0.75) 0.060.06 34.434.4 76.176.1 +41.7+41.7 A.4.4 Capacity ablation We sweep base width C∈16,24,32,64C∈\16,24,32,64\ to test whether 44 M parameters are simply too few. Reconstruction PSNR keeps improving with C, but SR peaks at C=32C=32 and drops slightly at C=64C=64 — the policy does not consume the extra texture detail, supporting the intentionally compact design. Table 13: Capacity ablation. Reconstruction PSNR keeps improving with C, but SR peaks at C=32C=32. Beyond ∼4 \!4 M parameters, extra capacity goes to texture detail the policy does not consume. Base C Params PSNR (dB) SR (%) 1616 ∼1 \!1 M 25.7725.77 – 2424 ∼2 \!2 M 26.3326.33 – 32 (ours) 4 M 26.8426.84 86.886.8 6464 1515 M 28.4028.40 85.285.2 A.4.5 Data-size ablation Training-set sweep over N∈10k,25k,50kN∈\10k,25k,50k\ pairs. The signal saturates by ∼25 \!25 k pairs, with 5050 k offering only ∼2 \!2 p headroom over 1010 k — consistent with the small intrinsic learning complexity that Locality predicts (the inpainting region is narrow). Table 14: Data-size ablation (C=32C=32, ε=100 =100 cm). The signal saturates near 2525k pairs, consistent with the low intrinsic learning complexity that Locality predicts. Pairs N PSNR (dB) SR (%) 1010k 26.0626.06 ∼84 \!84 2525k 26.3826.38 ∼85 \!85 50k (ours) 26.8426.84 86.886.8 A.4.6 Intrinsics robustness (fovy sweep) Companion to the extrinsic-noise stress in Table 4: we instead inject focal-length error of ±5%± 5\% and ±10%± 10\% on top of ε=100 =100 cm. Worst-case |Δ|=2.8| |=2.8 p — about ×4\!× smaller than the extrinsic-pose budget, indicating that intrinsics calibration is much less critical for deployment than extrinsic calibration. Table 15: Intrinsics stress on libero_object, 10001000 ep, atop ε=100 =100 cm extrinsic. Tolerance is ∼× \!4\!× tighter than for extrinsic noise in Table 4. fovy SR (%) Δ vs. ref −10%-10\% (zoom-in) 86.486.4 −2.8-2.8 −5%-5\% 87.987.9 −1.3-1.3 ref (0%0\%) (ours) 89.289.2 – +5%+5\% 89.189.1 −0.1-0.1 +10%+10\% (zoom-out) 89.089.0 −0.2-0.2 A.4.7 Depth-substitution ablation Companion to the Depth substitution paragraph in Section 5. The question is whether GT depth can be replaced by a learned monocular estimator without retraining the canonicalizer. We sweep the depth pipeline along three axes at fixed canonicalizer (C=32C=32, ε=100 =100 cm) on libero_spatial: (i) calibration strategy applied to off-the-shelf DepthAnything V2, (i) fine-tuning capacity (ViT-S/B/L on 5050 k LIBERO frames), and (i) pipeline modifications atop the best fine-tuned backbone (residual refinement head, end-to-end depth gradient, confidence gating). The sweep is designed to isolate where the gap to the 86.8%86.8\% GT-depth ceiling closes and where it stops moving — the first axis tests whether calibration alone is enough, the second whether a larger depth model is enough, and the third whether downstream-aware refinement is enough. Table 16: Depth-substitution ablation (libero_spatial, ε=100 =100 cm, 10001000 ep unless marked). Calibration alone caps at 44.5%44.5\%; a fine-tune reaches 71.7%71.7\%. Larger encoders, a refinement head, and end-to-end gradients all fail to improve further — the residual −15-15 p gap is not a depth-accuracy bottleneck. Depth source Encoder val L1L_1 SR (%) GT depth (upper bound) (ours) – – 86.886.8 Calibration only (no fine-tune) DA V2, raw inverse ViT-S – 40.940.9 DA V2, metric indoor ViT-S – 44.544.5 Fine-tuned on 5050 k LIBERO pairs (3 ep, L1L_1 + edge-aware) DA V2-S, FT ViT-S 0.00920.0092 68.668.6 DA V2-B, FT (ours) ViT-B 0.00830.0083 71.771.7 DA V2-L, FT ViT-L 0.00730.0073 67.367.3 Pipeline modifications atop DA V2-B FT + UNet residual refinement (0.80.8 M) ViT-B 0.00830.0083 zero residual + end-to-end depth gradient (wanchor=0.5w_anchor=0.5) ViT-B 0.03760.0376 66.166.1