Paper deep dive
Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
Hongxing Li, Yixin Li, Dingming Li, Zixuan Wang, Yuchen Yan, Wenqi Zhang, Weiming Lu, Yongliang Shen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/11/2026, 2:09:42 AM
Summary
The paper introduces GPD (Geometry-Privileged Distillation), a method to enhance spatial reasoning in Vision-Language Models (VLMs) by using 3D geometric cues (depth, semantics, BEV) as privileged information during on-policy self-distillation. Unlike existing methods that either modify architecture for 3D inputs or rely solely on outcome rewards, GPD routes question-relevant 3D cues to a teacher model to provide token-level supervision for incorrect reasoning trajectories, while the deployed student model remains RGB-only. GPD outperforms baselines like GRPO and answer-privileged OPSD on benchmarks such as VSI-Bench, MindCube, SPAR-Bench, MMSI-Bench, and ViewSpatial.
Entities (13)
Relation Signals (12)
GPD → evaluatedon → MindCube
confidence 95% · 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial
GPD → evaluatedon → ViewSpatial
confidence 95% · 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial
GPD → evaluatedon → MMSI-Bench
confidence 95% · 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial
GPD → evaluatedon → SPAR-Bench
confidence 95% · 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial
GPD → evaluatedon → VSI-Bench
confidence 95% · On the 4B backbone, GPD achieves 57.1 on VSI-Bench
GPD → improves → Spatial Reasoning
confidence 95% · GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks.
GPD → uses → 3D cues
confidence 95% · For each question, depth, semantic, and bird’s-eye-view (BEV) cues are rendered as compact text and routed to the teacher
GPD → augments → GRPO
confidence 90% · a privileged KL, applied only to incorrect trajectories, augments GRPO
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incorrect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.
Tags
Links
- Source: https://arxiv.org/abs/2610.12355v1
- Canonical: https://arxiv.org/abs/2610.12355v1
Trouble viewing inline? Open PDF directly →
Full Text
36,851 characters extracted from source content.
Expand or collapse full text
Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models Hongxing Li 1,∗ , Yixin Li 1,∗ , Dingming Li 1 , Zixuan Wang 1 , Yuchen Yan 1 , Wenqi Zhang 1 , Weiming Lu 1 , Yongliang Shen 1,† 1 Zhejiang University hongxing.li, syl@zju.edu.cn Abstract Spatial reasoning remains a persistent weakness of vision- language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and la- tency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a mis- judged depth or direction can be corrected only by the scene’s true geometry, which the 3D-scanned sources of spatial train- ing corpora already provide. We propose GPD (Geometry- Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each ques- tion, depth, semantic, and bird’s-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incor- rect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR- Bench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restrict- ing distillation to incorrect trajectories. Our code is available at https://github.com/ZJU-REAL/GPD. 1 Introduction Spatial reasoning requires a vision-language model (VLM) to judge distance, direction, and layout, as in planning a route, grasping the nearest object, or estimating how far a sofa is from a door. Benchmarks consistently show that these judgments remain difficult for current VLMs (Yang et al. 2025a; Wang et al. 2025; Zhang et al. 2025; Yang et al. 2025b; Li et al. 2025a). The difficulty stems from the input: RGB frames do not directly provide geometric evidence, so the model must infer depth and layout from appearance alone. Two lines of work address this gap. Explicit 3D meth- ods integrate geometric encoders or representations into the model (Chen et al. 2024; Cheng et al. 2024; Wu et al. 2026; Zheng, Huang, and Wang 2025), or invoke percep- tion and reconstruction tools at test time (Han et al. 2025; Chen et al. 2026; Cho et al. 2026; Qi et al. 2025), at the ∗ These authors contributed equally. † Corresponding author. cost of architectural changes and additional inference over- head. Training-based methods adapt the model on spatial question-answering data through supervised fine-tuning or outcome-based reinforcement learning (Ouyang et al. 2025; Li et al. 2025b; Ma et al. 2025); RGB-only inference is pre- served, but supervision comes only from the final answer, so a geometric error along the reasoning chain, such as a wrong depth order or a flipped left–right relation, is neither localized nor corrected. The two lines thus sit on opposite sides of a trade-off: explicit 3D methods gain geometric evidence but add infer- ence overhead, while training-based methods keep inference RGB-only but train without geometric supervision. Yet at training time, geometry is readily available: spatial corpora derived from 3D scans provide depth, semantics, and layout directly, and off-the-shelf models can estimate the same sig- nals otherwise. These 3D signals cannot serve as inputs at deployment, but they can supervise training as privileged information. The question is how to turn this privilege into supervision that an RGB-only model can internalize. A natural starting point is on-policy self-distillation (OPSD) (Zhao et al. 2026; Yang et al. 2026), which converts privilege into supervision: the student samples responses from the task input alone, while the same model, additionally given the privileged context, scores each sampled token as a teacher. In spatial reasoning, however, errors originate in perception: the model misjudges depth, position, or extent, then reasons coherently over the wrong facts. The reference answer, the standard privilege in OPSD, cannot correct such errors: a misjudged depth or direction can be corrected only by the scene’s true geometry. The privilege must in- stead supply geometric evidence matched to the question: counting, distance, and layout each rest on different evidence. We introduce GPD, an on-policy self-distillation frame- work that provides the teacher with question-relevant ge- ometric evidence. For each query, GPD constructs textual cues from depth, semantic segmentation, and bird’s-eye-view (BEV) layouts, and presents the routed cues to the teacher together with the reference answer. The student samples tra- jectories from RGB observations and the question, while the same VLM, conditioned on the privileged context, re-scores these trajectories token by token. GPD combines this distilla- tion signal with GRPO and applies the privileged KL only to incorrect trajectories, concentrating process supervision on arXiv:2610.12355v1 [cs.CV] 8 Oct 2026 Outcome Guided 3DPI GRPOLoss 푦 ! 푅 ! 퐴 ! OPSDLoss 퐾퐿(휋 ! , 휋 " ) Policy Model + Privilege Guided VLM Input + 3DEncoder Output GRPOLoss 푦 ! 푅 ! 퐴 ! SFTLoss −푙표푔휋 ! (푦|푝) or Policy Model Optimization Explicit 3D Method Training-Based Method GPD (Ours) Architectural Modification Limited Geometric Guidance Geometry-Aware Supervision Figure 1: Comparison of three paradigms for spatial reasoning in VLMs: explicit 3D methods that require architectural modification, training-based methods that rely on outcome-only supervision, and GPD that introduces question-routed 3D cues as training-time privilege. trajectories that require correction. The privilege thus enters only through the training signal, and the deployed model re- mains RGB-only. Figure 1 illustrates the distinction between GPD and the two prior paradigms. We evaluate GPD with Qwen3-VL-Instruct backbones at 2B and 4B scales on five spatial reasoning benchmarks. On the 4B backbone, GPD achieves 57.1 on VSI-Bench (Yang et al. 2025a) and 37.6 average across MindCube (Wang et al. 2025), SPAR (Zhang et al. 2025), MMSI (Yang et al. 2025b), and ViewSpatial (Li et al. 2025a), outperforming both GRPO and answer-privileged OPSD. Ablations confirm the effec- tiveness of each component. Further analysis indicates that the geometric cues have been internalized by the student model. Together, these results establish question-routed geo- metric privilege as effective supervision for spatial reasoning. Our contributions are threefold: • We formulate geometry-aware privileged self-distillation for spatial reasoning, where the teacher is equipped with spatially grounded evidence, enabling meaningful token- level distillation along incorrect reasoning trajectories. • We propose GPD, equipping an OPSD teacher with question-routed depth, semantic, and BEV cues, con- centrating token-level distillation on incorrect trajectories while retaining RGB-only inference. • Experiments across two model scales and five bench- marks demonstrate consistent improvements over GRPO and answer-privileged OPSD, with ablations validating the key design choices. 2 Related Work Spatial Reasoning in VLMs. Existing work improves spa- tial reasoning in VLMs through explicit 3D augmentation or training-based adaptation. Explicit methods integrate geo- metric encoders or representations into VLMs (Chen et al. 2024; Cheng et al. 2024; Wu et al. 2026), or use auxiliary perception, reconstruction, and tool-based pipelines at infer- ence time (Han et al. 2025; Chen et al. 2026; Cho et al. 2026). Training-based methods instead apply supervised fine-tuning or outcome-based reinforcement learning to spatial question- answering data (Ouyang et al. 2025; Li et al. 2025b; Ma et al. 2025). GPD connects these directions by using 3D cues as teacher-side training privilege, providing geometry-aware token-level supervision while leaving the student architecture and inference inputs unchanged. On-policy Self-distillation. Knowledge distillation trans- fers information through teacher distributions (Hinton, Vinyals, and Dean 2015). OPSD extends this idea to on- policy reasoning by conditioning a teacher on privileged information and re-scoring student rollouts token by to- ken (Zhao et al. 2026; Yang et al. 2026). Recent studies ex- amine the stability and refinement of this supervision (Kim and Lee 2026; Shen et al. 2026; Kaur et al. 2026), while multimodal variants use privileged crops, zoomed views, or visual-thought traces (Yuan et al. 2026; Cai et al. 2026; Li et al. 2026). GPD instead constructs question-routed textual privilege from depth, semantics, and BEV layouts, and ap- plies distillation only to incorrect trajectories. 3 Method 3.1 Overview of GPD GPD addresses a core tension in spatial reasoning: 3D ge- ometric cues such as depth, layout, and object positions are highly informative for training, yet unavailable at inference when only RGB images and text are observed. Rather than discarding this information or requiring it at test time, GPD uses it as training-time privilege to guide policy learning. As illustrated in Figure 2, the method proceeds in two stages. First, for each training sample, GPD constructs com- pact textual summaries of depth, semantic, and BEV cues offline, then routes only the question-relevant subset to form a privileged context for the teacher. Second, during train- ing, the same VLM plays two roles: as a student conditioned only on RGB and question, it samples on-policy trajecto- ries; as a privileged teacher additionally conditioned on the routed 3D cues and reference answer, it re-scores those tra- jectories token by token. Training combines outcome-based GRPO (Shao et al. 2024) with privilege-guided distillation applied selectively to incorrect trajectories, leaving the infer- ence interface unchanged. Stage1: Offline 3D Privilege Construction Query Measuringfromthe closestpointsofeach,how farapartarethebicycle andtheguitarinmeters? Question- Conditioned Routing Routed3DCue Privilege Assembly + <scene_context> •Objects •Cam-BEV ... </scene_context> <reference answer> 4.4 </reference answer> 3D Privilege Information Stage 2: Geometry-Privileged Self-Distillation RGB Query Student Context Teacher Context RGB Query 3DPI Shared Params 푦 ! 푦 " 푦 # 푅 ! 푅 " 푅 # · Rollout Reward Function Grouped Normalize 퐴 ! 퐴 " 퐴 # · 휋 $ (푡 ! ) 흅 푺 흅 푻 휋 $ (푡 " ) · 휋 $ (푡 % ) 휋 % (푡 ! )휋 % (푡 " ) · 휋 % (푡 % ) KLDivergence Token-LevelGRPOLoss Token-LevelOPSDLoss 푹 풊 < 휏 ? GRPOLoss+흀×OPSDLoss OptimizationObjective 풚 풊 :푡 " 푡 # ...푡 $ Depth SemanticsBEV 휋 ! (#|푝 " ) 휋 ! (#|푝 # ) Figure 2: Overview of GPD. Stage 1 constructs 3D privilege offline and routes question-relevant cues to form the teacher context. Stage 2 trains the same VLM as student and teacher: the student samples on-policy trajectories optimized by GRPO; the teacher provides token-level KL supervision selectively to incorrect trajectories as the OPSD loss. The two losses are jointly optimized. 3.2 Offline 3D Privilege Construction We represent each training example as (v,q,a ⋆ ), where v denotes the RGB observations, q the question, and a ⋆ the an- swer. For each example, GPD constructs an available subset of three 3D cues offline: depth, semantic, and BEV. Textual 3D Cue Extraction. When aligned 3D annota- tions are available, we derive depth, semantic, and BEV cues directly from them; otherwise, we estimate depth via Depth Anything 3 (Lin et al. 2025a) and obtain semantic instance masks and labels via Grounded-SAM-2 (Ren et al. 2024a), with BEV cues included only when reliable cam- era poses and scene-layout information are available. All cues are computed once and fixed throughout training. Let M ⊆ depth, semantic, BEV denote the modalities avail- able for a given example. For each m∈M, a fixed template converts its cue into a textual summarys m : depth records ob- ject distances and near–far order, semantic lists objects with their image regions, and BEV describes top-down relations. Frame identifiers are retained for multi-frame inputs. Question-Conditioned Routing. Different spatial ques- tions require different forms of geometric evidence: counting questions rely on semantic and visibility cues, whereas dis- tance and navigation questions benefit from depth and BEV. Injecting all modalities indiscriminately may introduce ir- relevant context and dilute the teacher signal. GPD therefore performs question-conditioned routing offline, using the base VLM to select a relevant subsetr(q)⊆M for each question. The routed scene context is c scene (q) = concat(s m : m∈ r(q)),(1) and the complete privilege provided to the teacher is c priv (q,a ⋆ ) = concat(c scene (q),a ⋆ ).(2) We present an example of the teacher privilege as follows. Example of Teacher Privilege <scene_context> Room layout: chair(west), table(center), sofa(northwest), door(east), wall(north) [frame_00] Depth near→far: chair(1.39m)→table(2.08m)→sofa(2.64m)→door(3.05m) Objects: chair [right-bottom, 5%], table [center, 11%], sofa [left-middle, 18%], door [right-middle, 3%] Cam-BEV: chair(left-front, 1.4m), table(front, 2.1m), sofa(left-front, 2.6m), door(right-front, 3.1m) </scene_context> <reference_answer> A </reference_answer> Token-Level Teacher Scoring <think>1.bedisontherightsideoftheroom,nearthewall.2.chairisat thedeskinthecenter,alsonearthebed.3.microwaveisinthekitchenontheotherside oftheroom.4.IfIstandnearthebedandfacethechair,Ilookingtowardthedesk.5. thisview,thekitchenappearsofftotheright,sothemicrowaveshouldbeonthechair's right.6.Therefore,theanswerisA.right.</think><answer>A</answer> TheThe The am From Question:Nearthebed,lookingatthe chair:isthemicrowaveonthechair's leftorright? StudentAnswer:A.Right ReferenceAnswer:B.Left AgreementScore window Figure 3: Token-level teacher scoring on a student trajectory. The student answers incorrectly (A. Right) while the reference answer is B. Left. The teacher assigns high agreement scores to spatially grounded tokens and progressively rejects tokens where the reasoning goes astray. 3.3 Geometry-Privileged Self-Distillation During training, the same VLM serves as both stu- dent and teacher under different conditioning contexts. The student receives only the RGB observations and question, p S = concat(v,q), while the teacher addi- tionally receives the offline-constructed privilege, p T = concat(v,q,c priv (q,a ⋆ )). Outcome-Guided Policy Optimization. For each train- ing example, the student samples a group of G responses y i G i=1 ∼ π θ (· | p S ) using only the student context p S , where each response y i = (t 1 ,t 2 ,...,t T ) consists of T to- kens. A group-normalized advantage A i is computed from the outcome reward R i of y i . We then optimize the following GRPO objective over all sampled responses L GRPO (θ) =−E y i G i=1 " 1 G G X i=1 1 T T X j=1 min ρ i,j A i , clip(ρ i,j , 1± ε)A i − β 1 G G X i=1 D KL [π θ ∥π ref ] i # , (3) where ρ i,j = π θ (t j | t <j ,p S )/π θ old (t j | t <j ,p S ) is the token-level importance sampling ratio, and D KL [π θ ∥π ref ] = P T j=1 [logπ θ (t j | t <j ,p S )− logπ ref (t j | t <j ,p S )] is the token-level KL divergence with the frozen reference policy π ref . Here, β controls the strength of regularization toward the reference policy. Privilege-Guided Policy Distillation. At each token po- sition j, the teacher and student induce conditional distri- butions π θ (· | t <j ,p T ) and π θ (· | t <j ,p S ) respectively. The per-token reverse KL divergence is expensive to com- pute due to the full-vocabulary summation. We therefore take a single-sample estimate on the student-sampled token t j ∼ π θ (·| t <j ,p S ), yielding the token-level teacher–student log-probability gap ∆ i,j = logπ θ (t j | t <j ,p T )− logπ θ (t j | t <j ,p S ), (4) and its corresponding reverse KL estimate d i,j = exp(∆ i,j )− ∆ i,j − 1. The distillation loss is then aggre- gated over all valid tokens and responses as L OPSD (θ) = G X i=1 T X j=1 u i,j d i,j . G X i=1 T X j=1 u i,j , (5) where u i,j ∈ 0, 1 indicates a valid response token. This aligned comparison provides token-level supervision along the student’s own reasoning trajectory, rather than on teacher- generated rollouts. Figure 3 illustrates the token-level teacher scoring on a representative incorrect trajectory. Error-Gated Joint Objective. Directly applying distilla- tion on all trajectories is suboptimal: when the student already produces correct responses, the teacher signal may introduce noise or conflicting gradients. We therefore gate L OPSD to incorrect trajectories only, leaving correct ones unaffected. To further suppress occasionally large distillation signals, we aggregate the OPSD loss over valid tokens from incorrect tra- jectories within each optimization micro-batch and clip the resulting scalar at 1.5 before scaling: ̃ L OPSD = min X i∈E T X j=1 u i,j d i,j . X i∈E T X j=1 u i,j , 1.5 , (6) whereE = i : R i < τ denotes the set of incorrect trajec- tories and we set τ = 0.5. The final joint objective is L =L GRPO + λ ̃ L OPSD ,(7) where λ controls the strength of the distillation signal. ModelAvg. Numerical QuestionMultiple-choice Question Obj. Cnt Abs. Dist. Obj. Size Room Size Rel. Dist. Rel. Dir. Route Plan. Appr. Order General-Purpose VLMs GPT-4o (Hurst et al. 2024)34.0 46.25.343.838.237.041.331.528.5 Gemini-3-Pro (Team et al. 2024)52.5 38.037.872.744.159.955.745.966.0 LLaVA-OneVision-72B (Li et al. 2024) 40.3 43.523.957.637.542.539.932.544.6 Qwen2.5-VL-7B (Bai et al. 2025b)35.8 43.515.148.541.136.340.128.433.7 3D-Augmented Methods Spatial-MLLM-4B (Wu et al. 2026)47.3 65.635.564.240.641.347.934.049.2 Think3D-4B (Zhang et al. 2026)-----44.739.036.761.2 Outcome-Supervised Methods SpaceR-7B (Ouyang et al. 2025)44.5 63.230.060.337.639.745.631.448.2 SpatialLadder-3B (Li et al. 2025b)45.7 63.534.361.743.945.444.835.636.4 Controlled Training Variants Qwen3-VL-2B (Bai et al. 2025a)51.3 59.942.170.354.051.743.127.361.7 + GRPO51.3 60.241.271.547.950.844.629.964.4 + OPSD50.8 64.140.869.448.151.444.729.957.8 + GPD51.7 61.242.171.348.751.845.029.464.2 Qwen3-VL-4B (Bai et al. 2025a)55.4 67.543.074.361.352.747.933.562.9 + GRPO56.4 70.244.474.961.751.349.933.565.5 + OPSD55.7 68.246.073.858.151.848.636.662.6 + GPD57.1 70.644.874.962.354.250.734.064.9 Table 1: VSI-Bench results across numerical and multiple-choice spatial reasoning subtasks. For controlled training variants, bold indicates the best result within each backbone. MethodMindCube SPAR MMSI ViewSpatial Avg. GPT-4o38.836.4 30.332.634.5 Qwen2.5-VL-7B29.330.2 25.937.930.8 SpatialLadder-3B43.532.9 27.439.935.9 Qwen3-VL-2B31.731.8 24.336.631.1 + GRPO35.838.5 19.836.732.7 + OPSD31.230.9 21.736.830.2 + GPD38.438.221.136.833.6 Qwen3-VL-4B21.535.2 27.739.130.9 + GRPO33.041.4 28.739.635.7 + OPSD21.936.3 27.540.431.5 + GPD38.043.429.439.837.6 Table 2: Results on MindCube, SPAR-Bench, MMSI-Bench, and ViewSpatial. For controlled training variants, bold indi- cates the best result within each backbone. 4 Experiments 4.1 Experimental Setup Baselines and Benchmarks. We compare GPD with general-purpose VLMs (Hurst et al. 2024; Team et al. 2024; Li et al. 2024; Bai et al. 2025b), 3D-augmented methods (Wu et al. 2026; Zhang et al. 2026), and outcome-supervised methods (Ouyang et al. 2025; Li et al. 2025b). For con- trolled comparison, we train GRPO and OPSD on the same data and backbone (Shao et al. 2024; Zhao et al. 2026). We evaluate on five benchmarks (Yang et al. 2025a; Wang et al. 2025; Zhang et al. 2025; Yang et al. 2025b; Li et al. 2025a): VSI-Bench covers eight video-based spatial reasoning sub- tasks; MindCube tests mental spatial modeling from lim- ited views; SPAR-Bench evaluates 3D object relations and scene geometry; MMSI-Bench assesses multi-image spatial intelligence; and ViewSpatial-Bench evaluates perspective- dependent spatial understanding. Training Dataset. We train on 10K VSI, 4K SPAR, and 1K MindCube training-split samples. For VSI and SPAR, which are derived from ScanNet scenes (Dai et al. 2017), privileged information is generated directly from 3D annotations. For MindCube, which lacks aligned 3D ground truth, we estimate depth with Depth Anything 3 (Lin et al. 2025b) and obtain semantic instance cues with Grounded-SAM-2 (Ren et al. 2024b). All methods and ablations share the same training mixture and precomputed privileged information. Training Details. All experiments are based on Qwen3- VL-Instruct backbones at 2B and 4B scales (Bai et al. 2025a). We train for one epoch with a learning rate of 1× 10 −6 , a rollout group size of n = 8, and a privileged-distillation weight of λ = 0.003. 4.2 Main Results Results on VSI-Bench. Table 1 compares GPD against general-purpose VLMs, 3D-augmented methods, outcome- supervised methods, and controlled training variants on VSI- Bench. With Qwen3-VL-4B, GPD achieves the highest av- erage accuracy of 57.1 among controlled training variants, improving over the base model by 1.7 points, GRPO by 0.7 points, and OPSD by 1.4 points. With Qwen3-VL-2B, GPD Base OPSD GRPO AnswerOnly Routed3DTextOnly Full3DText+Answer Routed3DImages +Answer Routed3DText +Answer(GPD) Figure 4: Ablation studies on Qwen3-VL-4B. Left: Privilege-component ablation. Grey bars represent no-signal or single-signal baselines; blue bars represent variants that combine outcome reward with privileged KL. Right: Privileged KL weight sweep. The dashed grey line marks GRPO and the dashed orange line marks the all-trajectories variant at λ = 0.003. similarly achieves the best controlled average of 51.7, out- performing the base model and GRPO by 0.4 points and OPSD by 0.9 points. Across both scales, the improvements on geometry-dependent subtasks align with routed 3D priv- ilege providing depth order and spatial layout cues that are difficult to recover from RGB alone. Results on Additional Benchmarks. Table 2 reports results on MindCube, SPAR-Bench, MMSI-Bench, and ViewSpatial. With Qwen3-VL-4B, GPD achieves the high- est controlled average of 37.6, improving over GRPO by 1.9 points and OPSD by 6.1 points. GPD obtains the best result among controlled variants on MindCube (38.0 vs. 33.0 for GRPO, +5.0 points), SPAR (43.4 vs. 41.4 for GRPO, +2.0 points), and MMSI (29.4 vs. 28.7 for GRPO, +0.7 points). With Qwen3-VL-2B, GPD again achieves the best controlled average of 33.6, outperforming GRPO by 0.9 points and OPSD by 3.4 points, with the largest gain on MindCube (38.4 vs. 35.8 for GRPO, +2.6 points). 4.3 Ablation Study Effect of Privilege Components. Figure 4 (left) ablates the privilege design on Qwen3-VL-4B, reporting Avg. as the mean over VSI, MindCube, SPAR, MMSI and ViewSpatial. Adding the reference answer alone reaches 40.2 (+0.4 points over GRPO), and adding routed 3D text alone reaches 40.5 (+0.7 points). Combining both yields 41.5, confirming that the two sources are complementary: the answer specifies the target outcome while the 3D context supplies geometric evi- dence. Full 3D text injection (39.8) falls below routed text by 1.7 points despite containing more information, confirming that indiscriminate injection introduces noise. Routed images reach 40.4, below routed text (41.5) by 1.1 points, suggesting that structured text aligns geometric cues with the symbolic reasoning trajectory better than visual overlays. Effect of Privileged KL. Figure 4 (right) sweeps the priv- ileged KL weight λ and its application scope with all other components fixed. Applying privileged KL to all trajecto- ries rather than incorrect ones alone reduces Avg. by 0.9 points to 40.6, consistent with noisy teacher signals dis- turbing already-correct trajectories. Among incorrect-only variants, Avg. rises from 39.6 at λ = 0.001 to 41.5 at λ = 0.003 (+1.7 points over GRPO), then decreases to 40.2 at λ = 0.004 and 39.6 at λ = 0.005. Too small a weight pro- vides insufficient process-level correction; too large a weight over-constrains the policy and degrades performance. 4.4 Further Analysis Zero-Shot Privilege Injection. Before RL training, we run a controlled zero-shot study on a 1000-question VSI-590K subset to decide how 3D privilege should be presented. We evaluate Qwen3-VL-4B-Instruct using identical RGB ob- servations and, when applicable, the same offline question- conditioned routing over depth, semantic segmentation, and BEV, without providing the reference answer. Figure 5 (left) summarizes the results. All image-based variants and full text injection remain below the RGB-only baseline (49.1): full 3D images drop to 47.4, while routed images and full text both reach 48.3. Routed text is the only variant that ex- ceeds the baseline, reaching 52.2 (+3.1 points), motivating its use as the privilege form in GPD. Routing Distribution. Figure 5 (right) shows the normal- ized routing distribution across VSI task categories. The pat- tern aligns well with geometric intuition: object counting is routed entirely to semantic maps, absolute distance relies predominantly on depth, room size is dominated by BEV, and relative direction uses both semantic and BEV. This align- ment confirms that the question-conditioned router selects relevant 3D cues rather than injecting every modality. Training Dynamics. Figure 6 compares GPD and GRPO training curves on Qwen3-VL-4B. Both methods show sim- ilar validation accuracy in early steps, but GPD gradually pulls ahead and continues to improve while GRPO plateaus. The entropy plot reveals why: GPD reaches a higher entropy peak earlier than GRPO, indicating that privileged KL en- courages broader exploration in the early training phase. As entropy levels align in later steps, GPD maintains its accu- racy advantage, suggesting that the spatial reasoning modes shaped by privileged supervision are retained. Figure 5: Privilege injection analysis on Qwen3-VL-4B-Instruct. Left: Zero-shot overall accuracy under different privilege forms on a VSI-590K subset. Right: Normalized routing weight distribution across VSI task categories and 3D modalities. 0255075100 40 45 50 55 Validation Accuracy GPD GRPO 0255075100 0.4 0.6 0.8 Policy Entropy Training Step Figure 6: Training dynamics of GPD and GRPO on Qwen3- VL-4B. Left: validation accuracy. Right: policy entropy. 525456 Accuracy 54.9 57.0 +2.1 56.3 56.9 +0.6 GRPOGPD Figure 7: Comparison of test-time 3D privilege dependence between GRPO and GPD. Privilege Internalization. To test whether GPD internal- izes 3D cues during training, we evaluate models trained on Qwen3-VL-4B with and without privilege injection at test time, as shown in Figure 7. GPD (56.3) already outper- forms GRPO (54.9) under RGB-only inference, and gains less when routed 3D text is injected at test time (+0.6 vs. +2.1 for GRPO). The smaller privilege gap suggests that GPD internalizes geometric cues during training rather than remaining dependent on them at inference. Case Study. Figure 8 shows a representative case where GPD answers correctly and GRPO fails. GPD correctly local- izes the camera, maps the left-forward motion to the spatial frame, and grounds the fountain via cross-view correspon- dence. GRPO misidentifies the fountain’s position relative to Question:Basedonthesefour, witheachcameraalignedwith roomwallsandpartially capturingthesurroundings:IfI amstandingatthesamespotand facingthesamedirectionas showninimage2,thenIturnleft andmoveforward,willIget closertothestonefountain? Options:A.YesB.No 3 1 24 GPDReasoningProcess:<think>Inimage2,thecameraispositioned totheleftofthecentraljar,facingthewallwiththedecorativescreen. Whentheuserturnsleftandmovesforwardfromthisposition,theyare movingtowardtheleftsideoftheframe,whichiswherethestone fountainislocatedinimage1.Thestonefountainisvisibleinimage1, whichisdirectlytotheleftofthecurrentviewpointinimage2. Therefore,movingleftandforwardfromthepositioninimage2willbring theuserclosertothestonefountain.</think><answer>A</answer> LocalizationMapping Cross-viewGrounding GRPOReasoningProcess:...Thefountainislocatedtotherightof thejar,andmovingleftwouldbringmeclosertothewallandaway fromthefountain... Figure 8: Case study comparing GPD and GRPO on a multi- view spatial question. the jar and reaches the wrong conclusion. This illustrates a failure mode of outcome-only supervision, where a geomet- rically incorrect intermediate judgment goes uncorrected in the absence of process-level geometric guidance. 5 Conclusion We present GPD, which augments GRPO with an OPSD teacher conditioned on question-routed depth, semantic, and BEV textual cues together with the reference answer. The teacher applies token-level KL supervision selectively to in- correct student trajectories, enabling the student to inter- nalize geometric reasoning during training while retaining RGB-only inference. Across both model scales and bench- marks, GPD improves overall accuracy over GRPO and out- performs answer-privileged OPSD. References Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. 2025a. Qwen3-VL Technical Report. arXiv preprint arXiv:2511.21631. Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; et al. 2025b. Qwen2.5-VL Technical Report. arXiv preprint arXiv:2502.13923. Cai, Y.; Liu, J.; Liu, Y.; Deng, H.; Yao, L.; Zheng, Y.; Ouyang, K.; Li, Z.; Wang, Z.; Sun, X.; Bai, H.; and Li, X. 2026. Thinking Without Images: Internalizing Visual Ma- nipulation with On-Policy Self-Distillation. arXiv preprint arXiv:2606.08719. Chen, B.; Xu, Z.; Kirmani, S.; Ichter, B.; Driess, D.; Florence, P.; Sadigh, D.; Guibas, L.; and Xia, F. 2024. SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14455– 14465. Chen, Z.; Lu, X.; Zheng, Z.; Li, P.; He, L.; Zhou, Y.; Shao, J.; Zhuang, B.; and Sheng, L. 2026. Geometrically-constrained agent for spatial reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 38689–38699. Cheng, A.-C.; Yin, H.; Fu, Y.; Guo, Q.; Yang, R.; Kautz, J.; Wang, X.; and Liu, S. 2024. SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models. In Advances in Neural Information Processing Systems. Cho, S.; Hachiuma, R.; Badki, A.; Su, H.; Lee, B.-K.; Song, C. H.; Liu, S.; Radhakrishnan, S.; Kim, S.; Wang, Y.-C. F.; et al. 2026. SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning. arXiv preprint arXiv:2606.13673. Dai, A.; Chang, A. X.; Savva, M.; Halber, M.; Funkhouser, T.; and Nießner, M. 2017. ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 5828–5839. Han, Y.; Zhou, E.; Rong, S.; An, J.; Wang, P.; Wang, Z.; Chi, C.; Sheng, L.; and Zhang, S. 2025. TIGeR: Tool- Integrated Geometric Reasoning in Vision-Language Models for Robotics. arXiv preprint arXiv:2510.07181. Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distill- ing the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531. Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Rad- ford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. Kaur, S.; Ri, N.; He, Y.; Fowl, L.; and Arora, S. 2026. Re- thinking On-Policy Self-Distillation for Thinking Models. arXiv preprint arXiv:2607.05184. Kim, J.; and Lee, D. 2026. OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Mod- els. arXiv preprint arXiv:2605.06188. Li, B.; Zhang, Y.; Guo, D.; Zhang, R.; Li, F.; Zhang, H.; Zhang, K.; Zhang, P.; Li, Y.; Liu, Z.; et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326. Li, D.; Li, H.; Wang, Z.; Yan, Y.; Zhang, H.; Chen, S.; Hou, G.; Jiang, S.; Zhang, W.; Shen, Y.; et al. 2025a. Viewspatial- bench: Evaluating multi-perspective spatial localization in vision-language models. arXiv preprint arXiv:2505.21500. Li, H.; Li, D.; Wang, Z.; Yan, Y.; Wu, H.; Zhang, W.; Shen, Y.; Lu, W.; Xiao, J.; and Zhuang, Y. 2025b. Spatialladder: Progressive training for spatial reasoning in vision-language models. arXiv preprint arXiv:2510.08531. Li, P.; Gao, Z.; Zhang, L.; Huang, M.; Li, Y.; Xu, F.; and Liu, J. 2026. Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reason- ing. arXiv preprint arXiv:2606.18974. Lin, H.; Chen, S.; Liew, J.; Chen, D. Y.; Li, Z.; Shi, G.; Feng, J.; and Kang, B. 2025a. Depth anything 3: Recovering the vi- sual space from any views. arXiv preprint arXiv:2511.10647. Lin, H.; Chen, S.; Liew, J.; Chen, D. Y.; Li, Z.; Shi, G.; Feng, J.; and Kang, B. 2025b. Depth Anything 3: Recov- ering the Visual Space from Any Views. arXiv preprint arXiv:2511.10647. Ma, W.; Chou, Y.-C.; Liu, Q.; Wang, X.; de Melo, C.; Xie, J.; and Yuille, A. 2025. SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning. arXiv preprint arXiv:2504.20024. Ouyang, K.; Liu, Y.; Wu, H.; Liu, Y.; Zhou, H.; Zhou, J.; Meng, F.; and Sun, X. 2025. Spacer: Reinforcing mllms in video spatial reasoning. arXiv preprint arXiv:2504.01805. Qi, Z.; Zhang, Z.; Fang, Y.; Wang, J.; and Zhao, H. 2025. Gpt4scene: Understand 3d scenes from videos with vision- language models. arXiv preprint arXiv:2501.01428. Ren, T.; Liu, S.; Zeng, A.; Lin, J.; Li, K.; Cao, H.; Chen, J.; Huang, X.; Chen, Y.; Yan, F.; et al. 2024a. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159. Ren, T.; Liu, S.; Zeng, A.; Lin, J.; Li, K.; Cao, H.; Chen, J.; Huang, X.; Chen, Y.; Yan, F.; et al. 2024b. Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks. arXiv preprint arXiv:2401.14159. Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeekMath: Pushing the Limits of Mathemati- cal Reasoning in Open Language Models. arXiv preprint arXiv:2402.03300. Shen, Z.; Tong, J.; Yan, S.; Shen, C.; Chen, H.; Ye, W.; Hu, X.; Miao, R.; Wang, H.; Zhao, J.; Chen, G.; and Ye, J. 2026. Purified OPSD: On-Policy Self-Distillation Without Losing How to Think. arXiv preprint arXiv:2607.02234. Team, G.; Georgiev, P.; Lei, V. I.; Burnell, R.; Bai, L.; Gulati, A.; Tanzer, G.; Vincent, D.; Pan, Z.; Wang, S.; et al. 2024. Gemini 1.5: Unlocking multimodal understand- ing across millions of tokens of context. arXiv preprint arXiv:2403.05530. Wang, Q.; Yin, B.; Zhang, P.; Zhang, J.; Wang, K.; Wang, Z.; Zhang, J.; Chandrasegaran, K.; Liu, H.; Krishna, R.; Xie, S.; Li, M.; Wu, J.; and Fei-Fei, L. 2025. MindCube: Spa- tial Mental Modeling from Limited Views. arXiv preprint arXiv:2506.21458. Wu, D.; Liu, F.; Hung, Y.-H.; and Duan, Y. 2026. Spatial- mllm: Boosting mllm capabilities in visual-based spatial in- telligence. Advances in neural information processing sys- tems, 38: 13569–13597. Yang, C.; Qin, C.; Si, Q.; Chen, M.; Gu, N.; Yao, D.; Lin, Z.; Wang, W.; Wang, J.; and Duan, N. 2026. Self-distilled rlvr. arXiv preprint arXiv:2604.03128. Yang, J.; Yang, S.; Gupta, A. W.; Han, R.; Fei-Fei, L.; and Xie, S. 2025a. Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR). Yang, S.; Xu, R.; Xie, Y.; Yang, S.; Li, M.; Lin, J.; Zhu, C.; Chen, X.; Duan, H.; Yue, X.; Lin, D.; Wang, T.; and Pang, J. 2025b. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence. arXiv preprint arXiv:2505.23764. Yuan, Q.; Lou, J.; Yu, X.; Lin, H.; Sun, L.; Han, X.; and Lu, Y. 2026. Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation. arXiv preprint arXiv:2605.18740. Zhang, J.; Chen, Y.; Zhou, Y.; Xu, Y.; Huang, Z.; Mei, J.; Chen, J.; Yuan, Y.-J.; Cai, X.; Huang, G.; Quan, X.; Xu, S.; and Li, Z. 2025. From Flatland to Space: Teaching Vision- Language Models to Perceive and Reason in 3D. arXiv preprint arXiv:2503.22976. Zhang, Z.; Wu, Y.; Jia, L.; Wang, Y.; Zhang, Z.; Li, Y.; Ran, B.; Zhang, F.; Sun, Z.; Yin, Z.; et al. 2026. Think3D: Thinking with Space for Spatial Reasoning. arXiv preprint arXiv:2601.13029. Zhao, S.; Xie, Z.; Liu, M.; Huang, J.; Pang, G.; Chen, F.; and Grover, A. 2026. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. arXiv preprint arXiv:2601.18734. Zheng, D.; Huang, S.; and Wang, L. 2025. Video-3d llm: Learning position-aware video representation for 3d scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8995–9006.