Paper deep dive

Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models

Tuo Liang, Disheng Liu, Nengbo Wang, Vipin Chaudhary, Yu Yin

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 84

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Grounding is a core capability of spatial vision-language models, yet most existing work focuses only on where a referred object is. Many 3D tasks also require knowing how it is oriented. Although existing 3D VLMs may predict oriented boxes, box pose does not explicitly capture object-centric orientation or symmetry-induced ambiguities. We introduce orientation grounding, a referring grounding task that predicts an object's 6D orientation and axial symmetry from a language or box query in single-view or multi-view scenes. To support this task, we construct ReferOri, with 331K multi-view and 387K single-view orientation-grounding queries obtained through scalable reconstruction, consistency checking, and human verification. We further present OG-VLM, which adapts a 3D VLM with structured box/orientation outputs, sign and symmetry tokens, and geometry-aware auxiliary losses. Across single-view and multi-view benchmarks, OG-VLM substantially outperforms orientation-aware VLM baselines and surpasses object-level orientation foundation models on scene-level referring benchmarks, showing that explicit orientation grounding is a distinct and learnable capability beyond localization. Downstream results validate its benefit for orientation-related spatial reasoning.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

80,613 characters extracted from source content.

Expand or collapse full text

Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models Tuo Liang1 Disheng Liu1 Nengbo Wang1 Vipin Chaudhary1 Yu Yin1 1Case Western Reserve University Abstract Grounding is a core capability of spatial vision-language models, yet most existing work focuses only on where a referred object is. Many 3D tasks also require knowing how it is oriented. Although existing 3D VLMs may predict oriented boxes, box pose does not explicitly capture object-centric orientation or symmetry-induced ambiguities. We introduce orientation grounding, a referring grounding task that predicts an object’s 6D orientation and axial symmetry from a language or box query in single-view or multi-view scenes. To support this task, we construct ReferOri, with 331K multi-view and 387K single-view orientation-grounding queries obtained through scalable reconstruction, consistency checking, and human verification. We further present OG-VLM, which adapts a 3D VLM with structured box/orientation outputs, sign and symmetry tokens, and geometry-aware auxiliary losses. Across single-view and multi-view benchmarks, OG-VLM substantially outperforms orientation-aware VLM baselines and surpasses object-level orientation foundation models on scene-level referring benchmarks, showing that explicit orientation grounding is a distinct and learnable capability beyond localization. Downstream results validate its benefit for orientation-related spatial reasoning. 1 Introduction Recent progress in spatial vision-language models (VLM), especially 3D VLM, has made grounding a central interface between language and geometry. Most existing grounding benchmarks and models focus on localization: given a referring expression, the model predicts a 2D or 3D box that identifies where the object is (Hong et al., 2023; Chen et al., 2024b; Zhang et al., 2024a; Li et al., 2025; Arnaud et al., 2025). More recent systems further use grounded objects as intermediate representations for spatial reasoning and 3D VQA (Ma et al., 2025b; Chen et al., 2025b). This trend makes grounding more than a supporting component; it is becoming a prerequisite for reliable spatial understanding. Localization alone, however, only specifies part of an object’s state. Many spatial and scene-level tasks (Linghu et al., 2024; Ma et al., 2025a) also require object-centric orientation: as shown in Fig 1, model may know there is a chair in front of himself but doesn’t understand the true direction he can sit down. This information is not equivalent to the pose of an enclosing box. Although recent 3D VLMs (Zheng et al., 2025; Linghu et al., 2026) use oriented bounding-box supervision from datasets such as KITTI, SUN RGB-D, nuScenes, ARKitScenes, and EmbodiedScan (Geiger et al., 2013; Song et al., 2015; Caesar et al., 2020; Baruch et al., 2021; Wang et al., 2024a), the resulting orientation remains coupled to box geometry, coordinate conventions, and localization quality. In contrast, object-centric orientation asks how the referred object itself is directed, which often depends on part structure, local shape, symmetry and environment. Figure 1: Orientation grounding extends localization grounding from predicting where a referred object is to estimating how it is oriented. ReferOri provides single-view and multi-view annotations with object boxes, 6D orientation, and axial symmetry labels, enabling OG-VLM to ground both object position and object-centric direction. Despite the practical importance of orientation, dedicated studies on orientation grounding remain limited. Existing spatial VLMs can sometimes infer coarse directions from category priors or canonical poses (Ma et al., 2022; Azuma et al., 2022; Chen et al., 2024b), but such behavior is fragile under non-canonical object poses, partial occlusion, and symmetric or near-symmetric shapes. Studying orientation as an explicit grounding target therefore exposes a different capability from localization: the model must both identify the referred object and recover its object-centric axes from visual and geometric evidence. In this work, we define orientation grounding as a natural next step beyond localization grounding toward more comprehensive 3D world understanding. Given a queried single-view or multi-view 3D scene and either a referring expression or an object box, the goal is to predict the target object’s 6D orientation together with axial symmetry labels along its canonical axes. This formulation raises three challenges: referring orientation labels are scarce, symmetry introduces inherent orientation ambiguities, and autoregressive VLMs are prone to sign flips and inconsistent binary predictions in structured numeric outputs. To address these challenges, we introduce ReferOri, a referring orientation-grounding dataset built by linking existing referring datasets with scalable scene and object reconstruction. ReferOri combines multi-view consistency checks with human verification to produce object orientation and axial symmetry annotations, yielding 331K multi-view and 387K single-view training queries and supporting both scene-level referring grounding and object-centric orientation estimation. Building on ReferOri, we present Orientation-Grounding VLM (OG-VLM), an adaptation of an existing 3D VLM for orientation grounding. OG-VLM introduces explicit <bbox> and <ori> output spans, dedicated tokens for numerical signs and symmetry labels, and lightweight geometry-aware heads for box, orientation, and symmetry supervision. Experiments show substantial gains over VLM-based spatial grounding/reasoning models and object-level orientation foundation models, demonstrating that orientation grounding is a distinct and learnable capability beyond localization grounding alone. In this paper, our contributions can be summarized as follows: • We introduce orientation grounding as a referring 3D grounding task that extends localization grounding to object-centric orientation and axial symmetry, and construct ReferOri, a scalable benchmark spanning single- and multi-view scenes through an automated data construction pipeline. • We present OG-VLM, an adaptation of an existing 3D VLM for orientation grounding that combines structured spatial output tokens with geometry-aware losses for box, orientation, and symmetry supervision. • We conduct extensive experiments across multi-view, single-view, and orientation-specific benchmarks, showing consistent gains over orientation-aware VLM baselines and object-level orientation foundation models, while downstream evaluations further demonstrate the benefit of orientation grounding for orientation-related spatial reasoning. 2 Related Work 2.1 3D Referring Grounding 3D referring grounding links language to scene geometry by localizing queried objects. Prior work develops datasets and models for predicting target boxes from reconstructed scenes or multi-view observations (Chen et al., 2020; Zhang et al., 2023b; Hong et al., 2023; Chen et al., 2024b; Zhang et al., 2024a; Li et al., 2025; Arnaud et al., 2025), while recent spatial VLMs further use grounding as an intermediate representation for reasoning and 3D question answering (Ma et al., 2025b; Chen et al., 2025b; Liu et al., 2025a). These works establish localization as a foundation for spatial understanding, but mainly predict the position and extent of referred objects. We retain this referring setting while extending the target to object-centric orientation and symmetry. 2.2 Orientation Dataset Existing orientation datasets mainly fall into three categories. Object-level 3D datasets (Reizenstein et al., 2021; Collins et al., 2022; Wu et al., 2023; Ahmadyan et al., 2021) provide canonical geometry and object-centric poses, but typically lack scene context. Single-view scene-level datasets (Xiang et al., 2016; Xiang et al., 2014; Ma et al., 2024) often infer pose through CAD matching, making them dependent on external 3D priors and sensitive to occlusion, truncation, and viewpoint ambiguity. Multi-view scene-level datasets (Geiger et al., 2013; Song et al., 2015; Caesar et al., 2020; Zhang et al., 2024b) provide oriented boxes or pose annotations, but mainly target detection and pose estimation rather than referring orientation grounding. ARKitScenes (Baruch et al., 2021) and EmbodiedScan (Wang et al., 2024a) offer rich scene geometry and can support referring-oriented training after adaptation. Our pipeline extends this line by constructing large-scale orientation and symmetry annotations directly under referring settings for both single-view and multi-view data. 2.3 Orientation Estimation Non-VLM Related. Earlier approaches (Avetisyan et al., 2019; Ma et al., 2024; Chen et al., 2024a) recovered object orientation through computer-aided design (CAD) matching and alignment in images or 3D scenes, relying on large 3D CAD repositories and remaining sensitive to occlusion, truncation, and structural variation. Later methods moved toward direct pose or orientation estimation from visual input, including PoseCNN (Xiang et al., 2017), the OnePose series (Sun et al., 2022; He et al., 2022), the GenPose series (Zhang et al., 2023a; Zhang et al., 2024b), POPE (Fan et al., 2024), and the Orient Anything series (Wang et al., 2024b; Wang et al., 2026). More recent reconstruction-based methods, such as One2Any (Liu et al., 2025b) and SAM3D (Chen et al., 2025a), estimate orientation by reconstructing object geometry. These works mainly formulate orientation as pose estimation, CAD alignment, or geometry recovery, whereas we study it as a grounding target under language and scene context. VLM-Related. Recent VLM-based works have begun to incorporate orientation into spatial understanding, but it is rarely treated as a first-class grounding target with dedicated representation and supervision. SpatialReasoner (Ma et al., 2025b) uses orientation grounding to support downstream reasoning, yet inaccurate object-centric orientation can become a bottleneck for orientation-dependent reasoning, as reflected in our later evaluations. SOFAR (Qi et al., 2025) uses a separate semantic-orientation predictor for spatial reasoning and object manipulation, but focuses on object-level semantic directions rather than scene-level referring orientation grounding. VG-LLM (Zheng et al., 2025) predicts orientation as part of 9D oriented boxes, coupling rotation with box geometry and Euler-angle parameterization. More generally, query-conditioned object-centric orientation and symmetry-induced ambiguities remain insufficiently modeled under a unified cross-category reference frame in cluttered scenes. We address these issues with explicit referring orientation grounding, dedicated orientation/symmetry supervision, and geometry-aware modeling. 3 Methodology 3.1 Task Formulation Given a scene observation X and a query q, orientation grounding predicts the referred object’s spatial state ^=(B^,R^,S^) A=( B, R, S), where B^∈ℝ6 B ^6 is the predicted 3D bounding box, R^∈SO(3) R∈ SO(3) is the predicted object-centric rotation in the scene/world coordinate system, and S^∈0,13 S∈\0,1\^3 denotes the predicted axial symmetry along the canonical x, y, and z axes. We use =(B,R,S)A=(B,R,S) to denote the corresponding ground-truth annotation throughout the paper. The query can be a natural-language referring expression or a box query. The model predicts rotation using the continuous 6D representation of R (Zhou et al., 2019), i.e., the first two axes of the rotation basis, which is converted to R^∈SO(3) R∈ SO(3). Symmetry matters because several orientations can be perceptually equivalent for symmetric objects. We therefore define the symmetry-aware discrepancy dS^(R,R^)=minQ∈⁡(S^)⁡arccos⁡(tr⁡(R^(RQ)⊤)−12),d_ S(R, R)= _Q ( S) ( tr\! ( R(RQ) )-12 ), Algorithm 1 Orientation annotation pipeline 1: Referred sample (I,b)(I,b) or multi-view sample (It,bt)t=1T\(I_t,b_t)\_t=1^T 2: Orientation grounding annotation =(B,R,S)A=(B,R,S) 3: if single-view then 4: Estimate Tworld←camT_world with Map Anything 5: Obtain object mask M from b using SAM2 6: Reconstruct object point cloud P and pose Tcam←objT_cam with SAM3D 7: Estimate symmetry labels S from P 8: Compute world-frame orientation R 9: Construct 3D box B 10: else 11: Select the top-2 most informative views 12: Estimate two candidate world-frame orientations R1,R2\R_1,R_2\ using the same SAM2–SAM3D pipeline 13: if ∠⁡(R1,R2)<τ (R_1,R_2)<τ then 14: Set R←R1R← R_1 15: else 16: Apply lightweight human correction on a coarse annotation to obtain R 17: end if 18: Estimate S and construct B 19: end if 20: return =(B,R,S)A=(B,R,S) When no symmetry ambiguity is present, ⁡(S^)=IG( S)=\I\. This formulation separates orientation errors from equivalent poses induced by object symmetry. We apply the same symmetry-aware treatment at inference time. 3.2 Data Construction Figure 2: Overview of ReferOri dataset structure. ReferOri attaches object-centric orientation and symmetry annotations to existing referring data. For single-view scenes, we use RefCOCO (Yu et al., 2016). For multi-view scenes, we annotate ScanNet (Dai et al., 2017) objects and transfer the resulting annotations to ScanNet-based referring datasets for training and evaluation. Following the notation in Sec. 3.1, each referred object is annotated as =(B,R,S)A=(B,R,S), where B is the 3D box, R∈SO(3)R∈ SO(3) is the world-frame object orientation, and S∈0,13S∈\0,1\^3 denotes axial symmetry. Algorithm 1 summarizes the overall annotation procedure, with the single-view and multi-view branches detailed below. Single-view construction. Given an image I and a referred bounding box b, we first use Map Anything (Keetha et al., 2025) to estimate scene depth and camera parameters, yielding a camera-to-world transformation Tworld←camT_world . We then apply SAM2 (Ravi et al., 2024) to refine the target mask M, and feed (I,M)(I,M) into SAM3D (Chen et al., 2025a) to reconstruct an object point cloud P together with an object-to-camera transformation Tcam←objT_cam . Based on P, we estimate the axial symmetry labels S, and treat the canonical coordinate system of the reconstructed object as an orientation proxy. The final world-frame orientation is computed as Robjworld=Rworld←camRcam←objR_obj^world=R_world \,R_cam . Multi-view construction. For ScanNet scenes, we select the two most informative views of the referred object, approximated by the largest visible mask area. We estimate object orientation independently from both views using the same SAM2–SAM3D pipeline and transform the results into the shared world frame using the native multi-view camera parameters. If the angular discrepancy between the two estimates is below a threshold τ, we directly accept the annotation; otherwise, we retain one estimate as a coarse annotation and apply lightweight human correction. Verification. We validate annotation quality through human inspection. For single-view data, five rounds of 100 randomly sampled annotations are reviewed for referent correctness, orientation consistency with the source image and reconstructed geometry, and coordinate consistency, yielding a 96% human-verified correctness rate. For multi-view ScanNet data, all ∼ 10K object annotations are manually inspected and corrected when necessary. We additionally audit 800 images with two independent evaluators, obtaining 97.5% inter-annotator agreement. Detailed verification protocols and annotation-module validation are provided in Appendix B.6. The resulting structure of ReferOri after annotation and verification is illustrated in Fig. 2. Figure 3: Overview of OG-VLM. Given multi-view frames, a single frame, or a 3D point cloud together with a text or box query, the VLM backbone generates structured grounding outputs with separate <bbox> and <ori> spans. Lightweight heads attached to the corresponding tokens provide localization, orientation, and symmetry supervision through geometry-aware losses. 3.3 OG-VLM Overview OG-VLM predicts the 3D box, orientation, and axial symmetry of a queried object (Fig. 3) from single-/multi-view input or a point cloud, together with a language or box query. Built on GS-Reasoner (Chen et al., 2025b), it introduces two extensions: separate <bbox> and <ori> spans with dedicated sign and symmetry tokens (Sec. 3.3.1), and lightweight heads on span-closing hidden states for geometry-aware box, orientation, and symmetry supervision (Sec. 3.3.2). 3.3.1 Special Token Design for Orientation Grounding GS-Reasoner represents 3D boxes as plain-text coordinates inside a <bbox> span. Extending this format to orientation introduces three issues: interference between box and orientation values, severe errors from sign flips in rotation components, and inconsistent free-form symmetry labels. Following prior LLM-based grounding models (Peng et al., 2023; Chen et al., 2024b), we address these issues with separate structured spans and dedicated sign and symmetry tokens. To improve the stability of structured spatial prediction, we explicitly disentangle localization and orientation outputs using Y=[ybbox,yori]Y=[\,y^bbox,\,y^ori\,], where ybbox=<bbox>(~)</bbox>y^bbox= <bbox>( b) </bbox>, and yori=<ori>(~,)</ori>y^ori= <ori>( o,s) </ori>. Here, ∈ℝ6b ^6 denotes the 3D bounding box, R∈SO(3)R∈ SO(3) denotes the object orientation, ∈ℝ6o ^6 is the continuous 6D representation of R used in training, and ∈0,13s∈\0,1\^3 denotes the axial symmetry labels. Rather than generating signed values as free-form text, we factorize each scalar v of b and o into a sign token and its magnitude, and we write each symmetry label as a dedicated token: v↦(σ⁡(v),|v|),σ⁡(v)=<+>,v≥0,<->,v<0,sa↦<s_yes>,sa=1,<s_no>,sa=0.v (σ(v),\,|v| ), σ(v)= cases <+>,&v≥ 0,\\ <->,&v<0, cases s_a cases <s\_yes>,&s_a=1,\\ <s\_no>,&s_a=0. cases -10.00002pt (1) This design reduces interference between box and orientation prediction while better matching token-based language modeling. Explicit sign tokens separate coarse spatial direction from numeric magnitude, reducing severe errors caused by sign flips. Discrete symmetry tokens likewise avoid unstable free-form binary generation and improve output consistency. 3.3.2 Geometry-Aware Loss for Grounding The language-modeling loss treats each numeric token as an independent class and therefore does not explicitly reflect geometric distance. For example, predictions with substantially different coordinate errors may receive similar token-level penalties when they differ from the target by a similar number of tokens. Moreover, the language-modeling objective alone does not enforce geometric constraints, such as orthogonality between the predicted orientation axes. To provide explicit geometric supervision, we attach separate lightweight MLP heads to the hidden states of the corresponding tokens for box, orientation, and symmetry prediction. These heads provide task-specific supervision for each spatial component (i.e., ℒboxL_box, ℒoriL_ori, ℒsymL_sym) and augment the language-modeling objective: ℒ=ℒLM+λboxℒbox+λoriℒori+λsymℒsym.L=L_LM+ _boxL_box+ _oriL_ori+ _symL_sym. (2) Box loss. From the hidden state at the </bbox> token, we regress a 3D box B=[xmin,ymin,zmin,xmax,ymax,zmax]∈ℝ6B=[x_ ,y_ ,z_ ,x_ ,y_ ,z_ ] ^6, and define ℒbox=αgiouℒGIoU+αscaleℒscale+αcenterℒcenter,L_box= _giouL_GIoU+ _scaleL_scale+ _centerL_center, -3.00003pt (3) where ℒGIoUL_GIoU, ℒscaleL_scale, and ℒcenterL_center enforce overlap-aware alignment, relative size consistency, and normalized center consistency, respectively. Orientation loss. The ground-truth orientation lies in SO(3)SO(3), while we predict its continuous 6D representation as two 3D axes ^, r, f. Let ,r,f denote the ground-truth right and front axes. We define ℒori=βalignℒalign+βorthoℒortho,L_ori= _alignL_align+ _orthoL_ortho, -10.00002pt (4) where ℒalign=12[(1−cos⁡(^,)+ϵ)λ+(1−cos⁡(^,)+ϵ)λ],ℒortho=|cos⁡(^,^)|.L_align= 12 [ (1- ( r,r)+ε )^λ+ (1- ( f,f)+ε )^λ ], _ortho=| ( r, f)|. -5.0pt (5) The two terms encourage axis alignment and mutual orthogonality, respectively. The adaptive exponent λ preserves sensitivity in the small-error regime. At inference, Gram–Schmidt orthogonalization converts the predicted axes into a valid rotation matrix. Symmetry loss. We further apply a cosine-based symmetry regularizer, ℒsym=1−cos⁡(^,),L_sym=1- ( s,s), -5.0pt (6) where s and s denote the predicted and ground-truth symmetry vectors, respectively. Overall, the proposed loss supplements token-level supervision with explicit geometric constraints, improving box consistency, axis alignment, and symmetry-aware grounding. Detailed formulations of ℒGIoUL_GIoU, ℒscaleL_scale, ℒcenterL_center, as well as the adaptive schedule of λ, are provided in Appendix E. 4 Experiments 4.1 Experimental Settings Training Settings. We use the proposed ReferOri as training data, including 387K single-view and 331K multi-view queries. We consider three progressively enhanced settings: Only data, which fine-tunes the base model on our constructed orientation-grounding data; + Special token, which introduces our special-token representation; and + Geometric loss, which further incorporates our geometry-aware loss. Further details on dataset construction, hyperparameter selection, and implementation can be found in Appendix E. Benchmarks and Metrics. We evaluate on both single-view and multi-view orientation-grounding benchmarks. The single-view benchmarks include RefCOCO∗, RefCOCO+∗, RefCOCOg∗ (Yu et al., 2016), and Ori-COCO (Wang et al., 2024b), while the multi-view benchmarks include ScanRefer∗ (Chen et al., 2020), Multi3DRefer∗ (Zhang et al., 2023b), and ARKitScenes (Baruch et al., 2021). Here, (∗*) denotes benchmarks whose orientation annotations are generated using our annotation pipeline. In total, five benchmarks are annotated by our pipeline, whereas OriCOCO and ARKitScenes provide native orientation annotations. Since neither dataset relies on our annotation process, we use them exclusively for out-of-distribution (OOD) and zero-shot evaluation. For continuous-orientation benchmarks, we report Acc@15 and Acc@30, i.e., the percentage of predictions whose orientation error is within 15∘15 and 30∘30 , respectively. For fair comparison, all methods’ native orientation representations are converted to rotation matrices in a shared right-handed reference frame and scored using the same symmetry-aware SO(3)SO(3) geodesic metric. Ori-COCO is evaluated using its native eight-sector classification accuracy because it only provides discrete orientation labels. Per-baseline conversion details are provided in Appendix C.3. 4.2 Main Results Table 1: Results on multi- and single-view scenes. Qualitative results appear in Fig. 5 in the appendix. Models Multi-View Scene Single-View Scene ARKitScenes ScanRefer∗ Multi3DRefer∗ Ori-COCO RefCOCO∗ RefCOCO+∗ RefCOCOg∗ Acc15 ↑ Acc30 ↑ Acc15 ↑ Acc30 ↑ Acc15 ↑ Acc30 ↑ Acc ↑ Acc15 ↑ Acc30 ↑ Acc15 ↑ Acc30 ↑ Acc15 ↑ Acc30 ↑ Spatial Reasoner – – – – – – 14.9 4.5 14.5 4.5 14.0 3.9 13.1 VG-LLM 10.1 17.4 17.8 26.2 21.5 33.5 – – – – – – – Ours (only data) 43.1 45.4 63.1 66.6 68.1 70.6 60.1 31.1 60.9 31.1 60.8 28.8 57.8 Ours (+special token) 48.2 50.5 65.2 68.0 70.5 73.1 62.2 31.6 63.3 31.7 61.5 29.7 59.1 Ours (+geometric loss) 50.5 51.6 66.1 70.1 71.5 74.2 64.8 32.0 63.9 32.2 62.1 30.2 60.5 As shown in Table 1, current spatial VLMs remain largely dominated by localization-oriented grounding, although grounding has become increasingly central to spatial understanding and reasoning. Existing baselines with partial orientation capability, including SpatialReasoner (Ma et al., 2025b) and VG-LLM (Zheng et al., 2025), perform poorly on benchmarks despite being designed for single-view reasoning and multi-view oriented-box prediction, respectively. This demonstrates our hypothesis that advances in localization grounding and grounding-guided reasoning do not automatically translate into reliable object-centric orientation grounding. Predicting oriented boxes or relying on implicit orientation cues is insufficient to recover how an object itself is oriented. Moreover, representing orientation solely with three Euler angles is limited by convention dependence, discontinuities, and axis coupling, which can hinder stable learning and accurate grounding. In contrast, our method consistently achieves large gains across both single-view and multi-view benchmarks, indicating that orientation grounding requires explicit task formulation, dedicated supervision, and tailored modeling. These results support our central claim that orientation grounding is a natural step beyond localization grounding toward deeper 3D world understanding. Localization results (in appendix Table 12) further show that our designs also improve 3D bounding box grounding. 4.3 Comparison with Orientation Foundation Model Table 2: Comparison with orientation foundation models. Models ARKitScenes ScanRefer∗ Multi3DRefer∗ Acc@15 Acc@30 Acc@15 Acc@30 Acc@15 Acc@30 Orientation foundation models Orient Anything 18.3 35.8 17.4 34.5 16.9 33.1 Orient Anything V2 19.8 43.3 15.8 37.4 16.9 35.6 Ours (+geometric loss) 50.5 51.6 66.1 70.1 71.5 74.2 We further compare our OG-VLM with recent orientation foundation models (Table 2). Unlike OG-VLM, which takes a queried multi-view 3D scene as input and performs orientation grounding in a scene-centric setting, existing orientation foundation models (Wang et al., 2024b; Wang et al., 2026) assume access to a cropped target object for object-centric orientation estimation. Since object localization is assumed to be given, the latter setting is substantially less challenging and may require additional detection or segmentation modules in practice. Nevertheless, OG-VLM achieves superior performance across both in-domain and OOD benchmarks. Notably, although trained exclusively on ScanNet-based multi-view data, it generalizes well to ARKitScenes, indicating that it learns transferable orientation representations rather than dataset-specific coordinate conventions. We attribute this advantage to OG-VLM’s explicit grounding supervision and scene-aware reasoning, which enable it to jointly localize the queried target and infer its orientation from multi-view observations and surrounding scene context. 5 Analysis 5.1 Axis-Wise Error Analysis Table 3: Axis-wise mean orientation error (in degrees). X, Y, and Z denote the right, front, and up axes, respectively. Dataset Ablation X (↓ ) Y (↓ ) Z (↓ ) RefCOCO+∗ only data 37.9 40.8 17.8 special token 35.3 38.3 17.6 geometric loss 35.1 38.0 17.3 ScanRefer∗ only data 38.4 38.6 0.1 special token 38.1 38.2 0.1 geometric loss 37.1 37.2 0.1 As shown in Table 1, progressively adding our designs yields consistent gains, with the combination of special tokens and geometric loss performing best overall. To identify where these gains arise, we analyze axis-wise mean orientation error on RefCOCO+∗ and ScanRefer∗ in Table 3. Improvements mainly occur on the X and Y axes, corresponding to the object’s right and front directions, which are also the more challenging components of orientation grounding. In contrast, Z-axis error is substantially smaller and varies little across settings, especially on ScanRefer∗. This is expected because most real-scene objects are approximately upright, making the up axis strongly constrained by gravity. Thus, our method mainly improves the harder object-centric horizontal directions, while the vertical axis is already relatively easy. The consistent X/Y error reduction from only data to special token and then geometric loss further shows that both structured output design and geometry-aware supervision improve fine-grained orientation alignment rather than merely benefiting trivial axes. 5.2 What Affects Orientation Grounding? Beyond the main benchmark results, we further analyze several factors that influence orientation grounding, including input depth, object symmetry, and localization quality. These analyses help characterize how visual and geometric evidence, object properties, and localization accuracy affect orientation prediction. We additionally study the effect of the number of input views in Appendix D.2. Table 4: Effect of depth on RefCOCO+∗. For localization, we report the accuracy at IoU thresholds of 25% and 50%. For orientation grounding, we report the accuracy of predictions whose orientation error falls within 15∘15 and 30∘30 , as well as the mean orientation error. Model Setting Localization Orientation Acc 25% Acc 50% Mean Err. (∘) Acc@15 Acc@30 only data w depth 68.7 36.3 29.6 31.1 60.8 w/o depth 1.2 0.0 34.3 24.7 52.3 special token w depth 68.2 35.6 29.5 30.9 61.5 w/o depth 0.8 0.0 35.0 23.6 52.1 geometric loss w depth 67.8 35.9 30.5 29.0 58.7 w/o depth 2.5 0.1 33.4 24.4 53.7 Depth. We conduct the depth ablation in the single-view setting to isolate its effect from multi-view geometric consistency. As shown in Table 4, removing depth causes a dramatic drop in 3D box grounding (i.e., localization) but a smaller degradation in orientation grounding. This suggests that depth is crucial for metric properties such as object scale, extent, and 3D location, whereas orientation can still be partially inferred from appearance, local shape, and semantic priors. Nevertheless, the consistent drops in Acc@15 and Acc@30 show that depth remains important for precise orientation grounding. Overall, 3D box grounding relies more heavily on metric reconstruction, while orientation can additionally exploit visual and semantic cues, which also helps explain why VLMs can acquire partial orientation understanding from image-language supervision. Table 5: Effect of object symmetry on orientation grounding. Dataset Model Symmetric Non-symmetric Acc@15 Acc@30 Acc@15 Acc@30 RefCOCO+∗ only data 24.41 53.76 34.41 66.56 special token 26.76 53.05 34.44 67.36 geometric loss 28.87 58.69 35.31 68.79 ScanRefer∗ only data 64.23 66.37 59.54 64.31 special token 65.97 68.40 58.37 63.69 geometric loss 66.77 69.03 60.50 63.80 Object symmetry. Symmetry directly determines whether multiple orientations should be treated as equivalent. We therefore compare symmetric and non-symmetric objects across datasets. As shown in Table 5, RefCOCO+ exhibits lower accuracy on symmetric objects, suggesting greater orientation ambiguity under limited single-view geometry. In contrast, symmetric objects achieve slightly higher accuracy on ScanRefer across all settings. We hypothesize that richer multi-view geometry reduces viewpoint ambiguity, while symmetry-aware labels admit a larger set of equivalent orientations. Consequently, symmetric objects benefit more from multi-view reasoning, whereas asymmetric objects remain more sensitive to local structure, occlusion, and grounding noise. Effect of Symmetry and Localization Constraints. To disentangle genuine orientation-grounding ability from the effects of symmetry handling and localization errors, we compare the same model under three evaluation settings that progressively introduce symmetry adjustment and localization constraints. As shown in Table 6, removing symmetry adjustment lowers Acc@15 by 2.7 points on ScanRefer∗ and 7.2 points on Multi3DRefer∗, but by less than 0.2 points on the RefCOCO family. The relatively modest gains indicate that the reported orientation performance is not primarily driven by symmetry adjustment; the model retains substantial orientation-grounding capability without it, while explicit symmetry modeling remains useful for intrinsically ambiguous orientations. Requiring joint correctness with localization causes a much larger drop, particularly on ScanRefer∗ and Multi3DRefer∗, because localization and orientation must succeed simultaneously and errors from either component accumulate. This distinction helps isolate orientation quality from localization failures during evaluation. We therefore treat the joint metric as a diagnostic complement to orientation-only evaluation, while the remaining gap suggests that jointly acquiring multiple grounding capabilities remains challenging for current spatial VLMs. Table 6: Comparison of three evaluation settings for OG-VLM. w/o symmetry-adjusted directly computes orientation accuracy without symmetry handling; w/ symmetry-adjusted accounts for symmetry-equivalent orientations when computing Acc@15 and Acc@30; and Joint additionally requires the predicted box to satisfy a localization IoU threshold of 0.25. ScanRefer∗ Multi3DRefer∗ RefCOCO∗ RefCOCO+∗ RefCOCOg∗ Acc@15 Acc@30 Acc@15 Acc@30 Acc@15 Acc@30 Acc@15 Acc@30 Acc@15 Acc@30 w/o symmetry-adjusted 63.4 66.9 64.3 71.8 31.9 63.8 32.1 61.1 30.1 55.4 w/ symmetry-adjusted 66.1 70.1 71.5 74.2 32.0 63.9 32.2 62.1 30.2 60.5 Joint w/ IoU≥ 0.25 45.2 47.8 43.0 45.7 25.8 50.2 24.5 47.7 21.5 39.1 5.3 Does Orientation Grounding Benefit Downstream Spatial Reasoning? In front ofOn the leftViewpointAvg.002020404042.442.438.138.19.39.3303049.749.747.947.922.222.24040Accuracy (%)GS-ReasonerOurs Figure 4: Results on three Orientation subsets of 3DSRBench (In front of, On the left, and Viewpoint). Avg. denotes the aggregate performance. A central question is whether orientation grounding provides information that is useful for downstream spatial reasoning. Importantly, neither model is trained on these downstream benchmarks, allowing us to evaluate whether the learned orientation capability helps downstream tasks without benchmark-specific supervision. We first evaluate on the Orientation subset of 3DSRBench (Ma et al., 2025a), which is a benchmark designed to assess the 3D spatial reasoning capabilities of VLMs. As shown in Fig. 4, our orientation-grounded model consistently improves over the baseline model (i.e., GS-Reasoner (Chen et al., 2025b)), increasing the overall accuracy from 30.0% to 40.0%. These results indicate that, even without 3DSRBench-specific training, strengthening the model’s orientation-grounding capability can directly improve spatial reasoning that relies on an object’s intrinsic reference frame. Table 7: Downstream situated spatial reasoning results on MSQA. Model EM@1 ↑ EM@1-Strict ↑ GS-Reasoner 29.02 18.05 Ours 32.64 28.18 We further evaluate on MSQA (Linghu et al., 2024), which provides a complementary setting for situated 3D spatial reasoning, where directional relations must be interpreted under a spatial reference frame rather than as isolated geometric attributes. We follow the official evaluation protocol and report Exact Match (EM)@1 and EM@1-Strict under the same benchmark evaluation setting (Table 7). Although the improvements are less directly tied to object-centric relations than on 3DSRBench, our model still improves both EM@1 and EM@1-Strict, suggesting that directional understanding also transfers to situated spatial reasoning. Together, these results suggest that orientation grounding serves as a complementary geometric cue for spatial reasoning across diverse downstream reasoning scenarios, particularly when object orientation is part of the spatial context. By providing a query-conditioned, object-centric frame of reference, it captures information that localization alone cannot recover, especially for symmetric objects and orientation-dependent referring expressions where valid orientations depend on the query. More downstream evaluation protocols are provided in Appendix C.4. 6 Conclusion We introduced orientation grounding as a referring 3D grounding task that extends localization to object-centric orientation and axial symmetry, and constructed ReferOri, a scalable single- and multi-view benchmark and training resource. We show that existing spatial VLMs can acquire strong orientation-grounding capability through explicit supervision, structured outputs, and lightweight geometry-aware losses. Experiments demonstrate substantial gains over orientation-aware VLMs and object-level orientation foundation models, while downstream evaluations further show benefits for orientation-related spatial reasoning. We hope this work encourages more complete spatial grounding beyond object location alone. Future work can build richer spatial reasoning datasets upon comprehensive spatial grounding representations to further study grounding-to-reasoning transfer. We also discuss the limitations of the current formulation and evaluation in Appendix A. References Achlioptas et al. (2020) P. Achlioptas, A. Abdelreheem, F. Xia, M. Elhoseiny, and L. Guibas Referit3d: neural listeners for fine-grained 3d object identification in real-world scenes. In European conference on computer vision, p. 422–440. Cited by: §E.1. Ahmadyan et al. (2021) A. Ahmadyan, L. Zhang, A. Ablavatski, J. Wei, and M. Grundmann Objectron: a large scale dataset of object-centric videos in the wild with pose annotations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 7822–7831. Cited by: §2.2. Arnaud et al. (2025) S. Arnaud, P. McVay, A. Martin, A. Majumdar, K. M. Jatavallabhula, P. Thomas, R. Partsey, D. Dugas, A. Gejji, A. Sax, et al. Locate 3d: real-world object localization via self-supervised learning in 3d. arXiv preprint arXiv:2504.14151. Cited by: §E.1, §1, §2.1. Avetisyan et al. (2019) A. Avetisyan, M. Dahnert, A. Dai, M. Savva, A. X. Chang, and M. Nießner Scan2cad: learning cad model alignment in rgb-d scans. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, p. 2614–2623. Cited by: §2.3. Azuma et al. (2022) D. Azuma, T. Miyanishi, S. Kurita, and M. Kawanabe Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 19129–19139. Cited by: §1. Baruch et al. (2021) G. Baruch, Z. Chen, A. Dehghan, T. Dimry, Y. Feigin, P. Fu, T. Gebauer, B. Joffe, D. Kurz, A. Schwartz, et al. Arkitscenes: a diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. arXiv preprint arXiv:2111.08897. Cited by: §C.3, §1, §2.2, §4.1. Caesar et al. (2020) H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom Nuscenes: a multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 11621–11631. Cited by: §1, §2.2. Chen et al. (2020) D. Z. Chen, A. X. Chang, and M. Nießner Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, p. 202–221. Cited by: §E.1, §2.1, §4.1. Chen et al. (2024a) J. Chen, Z. Zhou, M. Sun, R. Zhao, L. Wu, T. Bao, and Z. He Zeropose: cad-prompted zero-shot object 6d pose estimation in cluttered scenes. IEEE Transactions on Circuits and Systems for Video Technology 35 (2), p. 1251–1264. Cited by: §2.3. Chen et al. (2025a) X. Chen, F. Chu, P. Gleize, K. J. Liang, A. Sax, H. Tang, W. Wang, M. Guo, T. Hardin, X. Li, et al. Sam 3d: 3dfy anything in images. arXiv preprint arXiv:2511.16624. Cited by: §B.5, Table 8, §2.3, §3.2. Chen et al. (2024b) Y. Chen, S. Yang, H. Huang, T. Wang, R. Xu, R. Lyu, D. Lin, and J. Pang Grounded 3d-llm with referent tokens. arXiv preprint arXiv:2405.10370. Cited by: §1, §1, §2.1, §3.3.1. Chen et al. (2025b) Y. Chen, Z. Qi, W. Zhang, X. Jin, L. Zhang, and P. Liu Reasoning in space via grounding in the world. arXiv preprint arXiv:2510.13800. Cited by: §E.1, §1, §2.1, §3.3, §5.3. Collins et al. (2022) J. Collins, S. Goel, K. Deng, A. Luthra, L. Xu, E. Gundogdu, X. Zhang, T. F. Y. Vicente, T. Dideriksen, H. Arora, et al. Abo: dataset and benchmarks for real-world 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 21126–21136. Cited by: §2.2. Dai et al. (2017) A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner Scannet: richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 5828–5839. Cited by: §3.2. Fan et al. (2024) Z. Fan, P. Pan, P. Wang, Y. Jiang, D. Xu, and Z. Wang Pope: 6-dof promptable pose estimation of any object in any scene with one reference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7771–7781. Cited by: §2.3. Geiger et al. (2013) A. Geiger, P. Lenz, C. Stiller, and R. Urtasun Vision meets robotics: the kitti dataset. The international journal of robotics research 32 (11), p. 1231–1237. Cited by: §1, §2.2. He et al. (2022) X. He, J. Sun, Y. Wang, D. Huang, H. Bao, and X. Zhou Onepose++: keypoint-free one-shot object pose estimation without cad models. Advances in Neural Information Processing Systems 35, p. 35103–35115. Cited by: §2.3. Hong et al. (2023) Y. Hong, H. Zhen, P. Chen, S. Zheng, Y. Du, Z. Chen, and C. Gan 3d-llm: injecting the 3d world into large language models. Advances in Neural Information Processing Systems 36, p. 20482–20494. Cited by: §1, §2.1. Keetha et al. (2025) N. Keetha, N. Müller, J. Schönberger, L. Porzi, Y. Zhang, T. Fischer, A. Knapitsch, D. Zauss, E. Weber, N. Antunes, et al. Mapanything: universal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509.13414. Cited by: §3.2. Li et al. (2025) R. Li, S. Li, L. Kong, X. Yang, and J. Liang Seeground: see and ground for zero-shot open-vocabulary 3d visual grounding. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 3707–3717. Cited by: §1, §2.1. Linghu et al. (2026) X. Linghu, J. Huang, B. Jia, and S. Huang 3D-rft: reinforcement fine-tuning for video-based 3d scene understanding. arXiv preprint arXiv:2603.04976. Cited by: §1. Linghu et al. (2024) X. Linghu, J. Huang, X. Niu, X. Ma, B. Jia, and S. Huang Multi-modal situated reasoning in 3d scenes. Advances in Neural Information Processing Systems 37, p. 140903–140936. Cited by: §C.4, §1, §5.3. Liu et al. (2025a) D. Liu, T. Liang, Z. Hu, J. Peng, Y. Lu, Y. Xu, Y. Fu, and Y. Yin Deconstructing spatial intelligence in vision-language models. Cited by: §2.1. Liu et al. (2025b) M. Liu, S. Li, A. Chhatkuli, P. Truong, L. Van Gool, and F. Tombari One2Any: one-reference 6d pose estimation for any object. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 6457–6467. Cited by: §2.3. Ma et al. (2025a) W. Ma, H. Chen, G. Zhang, Y. Chou, J. Chen, C. De Melo, and A. Yuille 3dsrbench: a comprehensive 3d spatial reasoning benchmark. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 6924–6934. Cited by: §C.4, §1, §5.3. Ma et al. (2025b) W. Ma, Y. Chou, Q. Liu, X. Wang, C. de Melo, J. Xie, and A. Yuille Spatialreasoner: towards explicit and generalizable 3d spatial reasoning. arXiv preprint arXiv:2504.20024. Cited by: 1st item, §1, §2.1, §2.3, §4.2. Ma et al. (2024) W. Ma, G. Zhang, Q. Liu, G. Zeng, A. Kortylewski, Y. Liu, and A. Yuille Imagenet3d: towards general-purpose object-level 3d understanding. Advances in Neural Information Processing Systems 37, p. 96127–96149. Cited by: §2.2, §2.3. Ma et al. (2022) X. Ma, S. Yong, Z. Zheng, Q. Li, Y. Liang, S. Zhu, and S. Huang Sqa3d: situated question answering in 3d scenes. arXiv preprint arXiv:2210.07474. Cited by: §1. Peng et al. (2023) Z. Peng, W. Wang, L. Dong, Y. Hao, S. Huang, S. Ma, and F. Wei Kosmos-2: grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824. Cited by: §3.3.1. Qi et al. (2025) Z. Qi, W. Zhang, Y. Ding, R. Dong, X. Yu, J. Li, L. Xu, B. Li, X. He, G. Fan, et al. Sofar: language-grounded orientation bridges spatial reasoning and object manipulation. arXiv preprint arXiv:2502.13143. Cited by: §2.3. Ravi et al. (2024) N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, et al. Sam 2: segment anything in images and videos. arXiv preprint arXiv:2408.00714. Cited by: §3.2. Reizenstein et al. (2021) J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny Common objects in 3d: large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, p. 10901–10911. Cited by: §2.2. Song et al. (2015) S. Song, S. P. Lichtenberg, and J. Xiao Sun rgb-d: a rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 567–576. Cited by: §1, §2.2. Sun et al. (2022) J. Sun, Z. Wang, S. Zhang, X. He, H. Zhao, G. Zhang, and X. Zhou Onepose: one-shot object pose estimation without cad models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6825–6834. Cited by: §2.3. Wang et al. (2025) J. Wang, Z. Kang, H. Wang, H. Jiang, J. Li, B. Wu, Y. Wang, J. Ran, X. Liang, C. Feng, et al. Vgr: visual grounded reasoning. arXiv preprint arXiv:2506.11991. Cited by: §E.2. Wang et al. (2024a) T. Wang, X. Mao, C. Zhu, R. Xu, R. Lyu, P. Li, X. Chen, W. Zhang, K. Chen, T. Xue, et al. Embodiedscan: a holistic multi-modal 3d perception suite towards embodied ai. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 19757–19767. Cited by: §1, §2.2. Wang et al. (2024b) Z. Wang, Z. Zhang, T. Pang, C. Du, H. Zhao, and Z. Zhao Orient anything: learning robust object orientation estimation from rendering 3d models. arXiv preprint arXiv:2412.18605. Cited by: Table 8, 3rd item, §C.3, §2.3, §4.1, §4.3. Wang et al. (2026) Z. Wang, Z. Zhang, J. Xu, J. Wang, T. Pang, C. Du, H. Zhao, and Z. Zhao Orient anything v2: unifying orientation and rotation understanding. arXiv preprint arXiv:2601.05573. Cited by: Table 8, 3rd item, §C.2, §2.3, §4.3. Wu et al. (2023) T. Wu, J. Zhang, X. Fu, Y. Wang, J. Ren, L. Pan, W. Wu, L. Yang, J. Wang, C. Qian, et al. Omniobject3d: large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 803–814. Cited by: §2.2. Xiang et al. (2016) Y. Xiang, W. Kim, W. Chen, J. Ji, C. Choy, H. Su, R. Mottaghi, L. Guibas, and S. Savarese Objectnet3d: a large scale database for 3d object recognition. In European conference on computer vision, p. 160–176. Cited by: §2.2. Xiang et al. (2014) Y. Xiang, R. Mottaghi, and S. Savarese Beyond pascal: a benchmark for 3d object detection in the wild. In IEEE winter conference on applications of computer vision, p. 75–82. Cited by: §2.2. Xiang et al. (2017) Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox Posecnn: a convolutional neural network for 6d object pose estimation in cluttered scenes. arxiv 2017. arXiv preprint arXiv:1711.00199 10. Cited by: §2.3. Yu et al. (2016) L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg Modeling context in referring expressions. In European conference on computer vision, p. 69–85. Cited by: §E.1, §3.2, §4.1. Zhang et al. (2024a) H. Zhang, H. Li, F. Li, T. Ren, X. Zou, S. Liu, S. Huang, J. Gao, Leizhang, C. Li, et al. Llava-grounding: grounded visual chat with large multimodal models. In European Conference on Computer Vision, p. 19–35. Cited by: §1, §2.1. Zhang et al. (2024b) J. Zhang, W. Huang, B. Peng, M. Wu, F. Hu, Z. Chen, B. Zhao, and H. Dong Omni6dpose: a benchmark and model for universal 6d object pose estimation and tracking. In European Conference on Computer Vision, p. 199–216. Cited by: §2.2, §2.3. Zhang et al. (2023a) J. Zhang, M. Wu, and H. Dong Genpose: generative category-level object pose estimation via diffusion models. arXiv preprint arXiv:2306.10531. Cited by: §2.3. Zhang et al. (2025) M. Zhang, X. Li, J. Xu, K. Zhou, H. Bae, Y. Shen, C. Xiong, and H. Dong Sr3d: unleashing single-view 3d reconstruction for transparent and specular object grasping. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 15816–15823. Cited by: §E.1. Zhang et al. (2023b) Y. Zhang, Z. Gong, and A. X. Chang Multi3drefer: grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 15225–15236. Cited by: §E.1, §2.1, §4.1. Zheng et al. (2025) D. Zheng, S. Huang, Y. Li, and L. Wang Learning from videos for 3d world: enhancing mllms with 3d vision geometry priors. arXiv preprint arXiv:2505.24625. Cited by: 2nd item, §1, §2.3, §4.2. Zhou et al. (2019) Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li On the continuity of rotation representations in neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5745–5753. Cited by: §3.1. AI Use Statement Large language models were used only for language polishing and improving writing clarity. They were not used for research design, experiments, results, or scientific conclusions. All assisted text was reviewed by the authors. Appendix A Limitations Our annotation pipeline may introduce noise through imperfect segmentation, reconstruction ambiguity, and coordinate transformation. Manual verification and an independent annotation audit indicate high annotation consistency, although residual errors remain, particularly for ambiguous or weakly directional objects. In addition, a noticeable gap remains between our model and reconstruction-based object-level orientation estimates, suggesting that the current VLM has not yet fully absorbed the geometric cues available from reconstruction. Although our model performs strongly on multi-view orientation grounding, dedicated orientation foundation models remain stronger in object-centric single-view orientation estimation. The two settings are also not directly equivalent: our model reasons over queried scenes, whereas orientation foundation models typically operate on cropped objects or masks. Our goal is therefore not to replace these models, but to extend orientation understanding from isolated objects to referring grounding in single- and multi-view scenes. Bridging these complementary capabilities remains an important direction for future work. Our model is obtained by fine-tuning an existing 3D VLM, and we do not comprehensively evaluate whether this adaptation preserves all capabilities of the base model. Incorporating orientation grounding during pretraining may provide a more unified way to learn localization, orientation, and reasoning. Our robustness analyses also reveal residual reliance on upright-scene and language priors, motivating rotation-aware training, explicit gravity conditioning, and visually unanswerable examples. Finally, the current downstream evaluation focuses on orientation-related spatial reasoning rather than full embodied systems such as manipulation or navigation. Evaluating whether improved orientation grounding translates to broader embodied performance remains future work. Appendix B Data Construction Details This appendix provides additional details for the construction of our orientation grounding data, including the single-view and multi-view pipelines, symmetry estimation, coordinate alignment, annotation-module validation, human verification, and failure cases. B.1 Single-View Construction Pipeline Our single-view construction pipeline starts from existing referring grounding data with object-level bounding boxes. Given an image I and a referred object box b, we first filter out severely blurred images to avoid unstable reconstruction and ambiguous geometry. We then apply a metric-scale monocular 3D scene reconstruction model to estimate scene depth and camera parameters. This yields a camera-to-world transformation Tworld←camT_world together with a coarse 3D scene representation in a unified metric frame. Next, we refine the object region using a 2D segmentation model. Specifically, we use the referred box b as input to obtain an object mask M, which provides a more precise object boundary than the original bounding box. The masked object, together with the original image, is fed into an object-level 3D reconstruction model, which produces: • an object point cloud P, • an object canonical coordinate system, • and an object-to-camera transformation Tcam←objT_cam . We treat the canonical coordinate system of the reconstructed object as an orientation proxy for the object itself. Let Rcam←objR_cam denote the rotation component of Tcam←objT_cam , and let Rworld←camR_world denote the rotation component of Tworld←camT_world . The object orientation in the world frame is computed as Robjworld=Rworld←camRcam←obj.R_obj^world=R_world \,R_cam . The final rotation is stored as the ground-truth object orientation in SO(3)SO(3), and converted into a continuous 6D representation during training. Based on the reconstructed point cloud P, we further estimate the object’s axial symmetry labels along the x, y, and z axes. The final annotation tuple for each referred object is =(B,R,S),A=(B,R,S), where B is the 3D bounding box, R∈SO(3)R∈ SO(3) is the world-frame object orientation, and S∈0,13S∈\0,1\^3 denotes the symmetry labels. B.2 Multi-View Construction Pipeline For multi-view scenes, we exploit view consistency to improve annotation reliability. Given a referred object instance in a multi-view scene, we first identify the two most informative views. In practice, we approximate informativeness by the visible object area and select the top two frames with the largest mask area or bounding-box area. For each selected frame, we estimate the object orientation independently using the same object-level reconstruction pipeline as in the single-view setting. Each orientation estimate is then transformed into the common scene world frame using the native camera parameters provided by the multi-view scene. Let the resulting orientations be R1worldR_1^world and R2worldR_2^world. We compute their angular discrepancy Δ⁡(R1,R2)=arccos⁡(tr⁡(R1R2⊤)−12). (R_1,R_2)= ( tr(R_1R_2 )-12 ). If Δ⁡(R1,R2)<τ (R_1,R_2)<τ, where τ=5∘τ=5 , we directly accept the annotation. Otherwise, we retain one prediction as a coarse annotation and ask a human annotator to correct it. This strategy greatly reduces manual effort. Instead of labeling orientation from scratch, annotators only need to adjust a coarse but already plausible initialization. B.3 Symmetry Estimation We estimate axial symmetry labels directly from the reconstructed point cloud P. For each canonical axis a∈x,y,za∈\x,y,z\, we reflect the point cloud across the corresponding symmetry plane and measure the geometric consistency between the reflected point set and the original point cloud. If the discrepancy falls below a predefined threshold, we assign the corresponding axis symmetry label as positive. Formally, let Refa()Ref_a(P) denote the reflected point cloud across axis a, and let d⁡(⋅,⋅)d(·,·) denote a bidirectional point-set discrepancy. We mark axis a as symmetric if d⁡(,Refa())<ϵa.d (P,Ref_a(P) )< _a. This produces the final symmetry label vector S=(sx,sy,sz),sa∈0,1.S=(s_x,s_y,s_z), s_a∈\0,1\. In practice, symmetry estimation is reliable for most rigid and structurally regular objects, but can become noisy under severe reconstruction artifacts, heavy occlusion, or near-symmetric shapes. B.4 Coordinate Alignment and Sanity Checks Our pipeline combines outputs from scene-level and object-level reconstruction modules, which may adopt different coordinate conventions. To ensure consistency, we explicitly verify: • coordinate convention compatibility across modules, • global axis flips, • handedness mismatch, • and world-frame alignment after transformation. These checks are not conducted only in image space. During verification, we inspect both the original 2D image and the final reconstructed 3D scene with the transformed object orientation. This allows us to detect systematic issues that may not be obvious from a 2D projection alone, such as left-right inversion or incorrect up-axis assignment. We find that such sanity checks are important in practice, especially when combining independently trained reconstruction modules. B.5 Annotation Module Validation Since SAM3D (Chen et al., 2025a) provides the object-level orientation proposals used in our annotation pipeline, we evaluate its orientation estimates on Ori-COCO as an external reference. As shown in Table 8, SAM3D achieves 96.3% accuracy, compared with 72.4% and 86.4% for Orient Anything V1 and V2, respectively. This evaluation provides an external sanity check on the orientation proposals used for large-scale annotation. Table 8: Validation of the annotation module on Ori-COCO (Acc). Model Acc Orient Anything V1 (Wang et al., 2024b) 72.4 Orient Anything V2 (Wang et al., 2026) 86.4 SAM3D (Chen et al., 2025a) 96.3 B.6 Human Verification Protocol For single-view annotations, we evaluate reliability through repeated random human inspection. We conduct five rounds, each with 100 randomly sampled examples. During verification, annotators inspect: • the original 2D image, • the target object region, and • the reconstructed 3D scene with the projected object orientation. A sample is counted as correct only when the referred object is correctly identified and its annotated orientation is visually consistent with both the source image and reconstructed geometry. Annotators additionally check coordinate consistency, global axis flips, and handedness mismatches. Across the five rounds, 96% of the sampled annotations are judged correct on average, which we report as the human-verified correctness rate rather than accuracy against ground-truth annotations. Multi-view annotation protocol. For ScanNet multi-view data, where reconstruction is less reliable, reconstruction outputs are treated only as annotation proposals. All object annotations are manually inspected. Annotators check whether the proposed canonical frame and symmetry label are consistent with the available multi-view evidence and reconstructed geometry. Incorrect proposals are corrected, while ambiguous or low-confidence cases are independently re-annotated by a second annotator. Remaining disagreements are resolved through joint inspection, and samples without reliable consensus are discarded. For weakly directional objects, symmetry is assigned only when alternative directions are genuinely equivalent; if a unique direction exists but cannot be reliably determined, the annotation is corrected or removed. Independent audit. We further audit 200 images from each of ScanNet, RefCOCO, RefCOCO+, and RefCOCOg, for a total of 800 images. Two evaluators independently judge whether each full annotation—including the referent, orientation frame, and symmetry label—is correct, yielding 1,600 binary judgments. Table 9 reports percentage agreement and Fleiss’ κ. Agreement ranges from 96.0% to 98.8%, with 97.5% overall. The lower κ values despite high raw agreement are consistent with the prevalence effect under the strong imbalance toward annotations judged as correct. Table 9: Independent annotation audit: agreement between two evaluators. Dataset Agreement Fleiss’ κ ScanNet 96.00% 0.479 RefCOCO 96.27% 0.409 RefCOCO+ 98.79% 0.436 RefCOCOg 98.68% 0.493 Overall 97.45% 0.454 B.7 Advantages of the Pipeline Our construction pipeline has three main advantages. First, it is scalable. It builds upon existing referring data, segmentation, and reconstruction modules, allowing orientation annotations to be generated at large scale without exhaustive manual labeling. Second, it supports both single-view and multi-view settings. This enables us to cover both natural-image scenarios and richer 3D scene understanding benchmarks within a unified formulation. Third, it is extensible. Since the pipeline is modular, future advances in scene reconstruction and object reconstruction can be directly incorporated to improve annotation quality. In this sense, the quality of the benchmark can continue to improve with stronger underlying 3D perception models. B.8 Failure Cases and Limitations Although effective in practice, the pipeline still has several limitations. Reconstruction ambiguity. The canonical coordinate system produced by object-level reconstruction is used as an orientation proxy, but it is not guaranteed to perfectly match human-defined intrinsic orientation in all cases. This issue is most pronounced for highly symmetric objects or objects with weak directional cues. Segmentation errors. Errors in the 2D object mask may propagate to the object reconstruction stage, which can in turn affect both orientation and symmetry estimation. Occlusion and incomplete geometry. Heavy occlusion may lead to incomplete or distorted reconstructions, making symmetry estimation and orientation inference less reliable. Cross-module noise. The final annotation quality depends on the consistency of scene reconstruction, object reconstruction, and coordinate transformation. Although we explicitly check for major mismatches, residual noise may still remain. We therefore regard the constructed annotations as scalable supervision with bounded noise, rather than perfectly noise-free labels. Appendix C Evaluation Protocol Details C.1 Unified Rotation Conversion All continuous-orientation benchmarks are evaluated after converting each method’s native output into a 3×33× 3 rotation matrix in the same right-handed reference frame (x right, y forward, z up): • SpatialReasoner (Ma et al., 2025b) predicts a front direction under an assumed world-up direction; we complete it to a full rotation matrix and map it into the shared frame. • VG-LLM (Zheng et al., 2025) predicts Euler angles; we compose them into a rotation matrix following its published convention. • Orient Anything V1/V2 (Wang et al., 2024b; Wang et al., 2026) predict azimuth–polar–rotation angles; we convert them into rotation matrices following the Orient Anything convention. • OG-VLM predicts a continuous 6D rotation representation; we orthogonalize it via Gram–Schmidt and map it into the same reference frame. After conversion, all methods are evaluated using the same SO(3)SO(3) geodesic error, so differences in native orientation parameterization do not introduce representation-specific evaluation metrics. C.2 Symmetry-Aware Evaluation The symmetry-aware evaluation in Sec. 3.1 uses the model-predicted axial symmetry labels rather than ground-truth symmetry. For OG-VLM, the predicted labels S^=(s^x,s^y,s^z) S=( s_x, s_y, s_z) determine the set of symmetry-equivalent transformations ⁡(S^)G( S), and the orientation error is computed as the minimum SO(3)SO(3) geodesic distance over this set. For baselines that do not provide compatible axial symmetry predictions, including SpatialReasoner and VG-LLM, we use the identity equivalence set =IG=\I\, corresponding to no symmetry adjustment. Thus, ground-truth symmetry labels are never used to relax orientation errors during evaluation. Our axis-specific symmetry representation differs from the rotational-order-k representation used by Orient Anything V2 (Wang et al., 2026). As shown in Table 6, symmetry adjustment produces dataset-dependent improvements, while the relatively modest changes on most benchmarks indicate that the reported orientation performance is not primarily driven by symmetry handling. C.3 Benchmark Splits and Metrics RefCOCO/RefCOCO+/RefCOCOg, ScanRefer, and Multi3DRefer are constructed by converting the test splits of the corresponding referring datasets into orientation-grounding test sets while preserving their original referring structure. Ori-COCO is adopted from Orient Anything (Wang et al., 2024b). For ARKitScenes (Baruch et al., 2021), which already provides oriented 3D bounding-box annotations, we directly use its validation split following common practice. For all continuous benchmarks, every method’s native output is converted into a rotation matrix in the same right-handed reference frame before evaluation. Specifically, we convert Euler angles for VG-LLM, a front direction under an assumed world-up for SpatialReasoner, azimuth–polar–rotation angles for Orient Anything, and the continuous 6D rotation representation used by our method. All predictions are then evaluated using the same symmetry-aware SO(3)SO(3) geodesic metric. Ori-COCO is treated separately because it does not provide fine-grained SO(3)SO(3) annotations; instead, its labels partition the horizontal orientation space into eight 45∘45 sectors. Following its native protocol, we bin each predicted orientation into one of these sectors and report classification accuracy. Using Acc@15/Acc@30 on this benchmark would primarily reflect label granularity rather than method quality. Accordingly, Ori-COCO results in Table 1 are reported under its native metric. Table 8 uses Ori-COCO to validate SAM3D as our annotation module under the original Orient Anything protocol, whereas Table 1 uses it as an independently annotated external benchmark. C.4 Downstream Evaluation Protocol 3DSRBench. For 3DSRBench (Ma et al., 2025a), we evaluate the Orientation subset, including In front of, On the left, and Viewpoint, following the benchmark’s official evaluation protocol. The benchmark uses CircularEval, which evaluates each question under multiple answer-choice permutations and counts it as correct only when the model remains correct across the permutations. Its test set also includes FlipEval-augmented examples to reduce left/right response bias. We report accuracy for each relation type and the aggregate accuracy over the Orientation subset. MSQA. For MSQA (Linghu et al., 2024), we follow the MSR3D evaluation protocol and report EM@1 and EM@1-Strict. Predictions and reference answers are normalized by lowercasing, whitespace normalization, punctuation removal, spelling correction, number-word conversion, and article removal where applicable. EM@1-Strict requires an exact match with at least one normalized reference answer, whereas EM@1 additionally accepts containment matches between the normalized prediction and reference answers. Empty predictions receive zero under both metrics, and dataset-level scores are averaged over all evaluated questions. Neither model receives benchmark-specific QA supervision or task-specific fine-tuning on 3DSRBench or MSQA. Appendix D Additional Results and Analyses D.1 Sensitivity to Image-Plane Rotation Table 10: Sensitivity of OG-VLM to image-plane rotation of the input view. Axis-wise and overall mean orientation errors are in degrees; Acc@15/Acc@30 are reported in %. Rotation X Err. Y Err. Z Err. Mean Err. Acc@15 Acc@30 90∘90 69.0 51.3 83.9 60.2 0.05 0.20 180∘180 40.1 44.5 18.7 42.3 16.71 37.32 270∘270 68.4 52.1 83.6 60.3 0.05 0.25 We evaluate OG-VLM on inputs rotated in the image plane by 90∘90 , 180∘180 , and 270∘270 . As shown in Table 10, 90∘90 and 270∘270 rotations break the alignment between image-top and world-up and increase the Z-axis error to roughly 84∘84 , collapsing Acc@15; a 180∘180 rotation preserves the vertical axis and mainly increases horizontal facing errors. This reveals a reliance on upright-scene priors; rotation-aware augmentation or explicit gravity conditioning are natural remedies, which we leave to future. D.2 Input Frames Ablation We study how the number of input frames affects orientation grounding on Multi3DRefer. As shown in Table 11, performance drops from multi-view to single-view, and further to language-only input, showing that orientation grounding depends strongly on visual and geometric evidence. Multi-view input provides complementary cues for resolving occlusion and structural ambiguity, while single-view input offers only limited local appearance and shape information. Table 11: Results on Multi3DRefer∗ under different settings. Model Settings Acc@15 Acc@30 Ours (only data) multi-view 68.08 70.64 single-view 58.43 61.46 language 20.00 40.00 Ours (+special token) multi-view 70.50 73.06 single-view 56.88 60.33 language 8.04 10.80 Ours (+geometric loss) multi-view 71.50 74.20 single-view 64.25 66.92 language 22.22 22.22 Notably, the model still outputs orientation predictions even without visual input. Such non-zero performance is expected for pretrained VLMs, which retain category-level and scene-level language priors. Our goal in this ablation is therefore not to eliminate these priors, but to quantify the additional benefit of visual evidence, particularly multi-view observations, for orientation estimation. The language-only performance mainly reflects residual priors rather than genuine grounding ability; while they may support coarse guesses from object categories or common scene configurations, they are insufficient for robust orientation grounding in ambiguous or non-canonical cases. D.3 Downstream Spatial Reasoning Analysis The three orientation relations in 3DSRBench probe complementary uses of the grounded object-centric frame. In front of primarily depends on the intrinsic front axis, while On the left depends on the intrinsic right/left axis. Viewpoint additionally requires relating the object’s intrinsic frame to the camera frame, making it a more challenging reference-frame transformation. The consistent gains across all three relations therefore suggest that improved orientation grounding benefits multiple forms of orientation-dependent spatial reasoning rather than a single relation type. D.4 Localization Results Table 12: Localization grounding results at Acc25% on ScanRefer, Multi3DRefer, RefCOCO, RefCOCO+, and RefCOCOg. Model ScanRefer Multi3DRefer RefCOCO RefCOCO+ RefCOCOg Acc 25% Acc 25% Acc 25% Acc 25% Acc 25% Ours (only data) 55.68 55.43 71.54 68.91 62.37 Ours (+special token) 58.95 57.38 71.84 69.26 62.42 Ours (+geometric loss) 61.37 60.08 72.67 70.31 63.38 We additionally report localization grounding results to examine whether adding explicit orientation supervision preserves the base model’s localization capability. These results complement the orientation-only evaluation in the main paper by showing the behavior of the localization component under the same training variants. D.5 Qualitative Comparison with Baselines Figure 5 shows that our predictions better align with ground-truth orientations than SpatialReasoner on RefCOCO+ and VG-LLM on ScanNet across both single-view and multi-view settings. Figure 5: Qualitative orientation-grounding results on RefCOCO+ and ScanNet. For RefCOCO+, gray arrows denote ground truth, while orange and red arrows denote predictions from Ours and SpatialReasoner, respectively. For ScanNet, solid axes denote ground truth and dashed axes denote predictions from Ours and VG-LLM, respectively. Appendix E Training and Geometry-Aware Loss Details This appendix provides the detailed formulation of the geometry-aware losses used in Sec. 3.3.2, including the box loss, orientation loss, symmetry loss, and the implementation details of the corresponding prediction heads. E.1 Dataset Construction and Training Details For multi-view scenes, we construct orientation-grounding data from Locate3D (Arnaud et al., 2025), SR3D (Zhang et al., 2025), NR3D (Achlioptas et al., 2020), ScanRefer (Chen et al., 2020), and Multi3DRefer (Zhang et al., 2023b). For single-view scenes, we use RefCOCO (Yu et al., 2016). For each referred object, we construct a language query from the original referring expression and a box query from the corresponding bounding box. The resulting dataset contains 331K multi-view queries and 387K single-view queries, with the two query types approximately balanced at a 1:1 ratio. We use GS-Reasoner (Chen et al., 2025b) as the base model. Training is performed on 4×H200 GPUs, with each full training run taking approximately 120 hours. Loss coefficients and additional optimization details are summarized in Sec. E.7. E.2 Prediction Heads To explicitly disentangle localization and orientation learning, we adopt multiple lightweight prediction heads on top of the hidden states, following the spirit of VGR (Wang et al., 2025), which shows that geometric supervision can be effectively introduced through small MLP heads over language model representations. Concretely, we use: • a box head on the hidden state at the </bbox> token to regress a 3D box, • an orientation head on the hidden state at the </ori> token to regress the continuous 6D orientation representation, • and a symmetry head to predict symmetry-related geometry. This decoupling reduces interference between localization and orientation prediction, while allowing each sub-task to receive task-specific geometric supervision. E.3 Detailed Box Loss For 3D grounding, the model predicts an axis-aligned box B=[xmin,ymin,zmin,xmax,ymax,zmax]∈ℝ6.B=[x_ ,y_ ,z_ ,x_ ,y_ ,z_ ] ^6. (7) Let BpB_p and BgB_g denote the predicted and ground-truth boxes, respectively. The box loss in Eq. (3) consists of three terms: ℒbox=αgiouℒGIoU+αscaleℒscale+αcenterℒcenter.L_box= _giouL_GIoU+ _scaleL_scale+ _centerL_center. (8) GIoU loss. We first define the 3D IoU as IoU⁡(Bp,Bg)=|Bp∩Bg||Bp∪Bg|.IoU(B_p,B_g)= |B_p∩ B_g||B_p∪ B_g|. (9) Let C denote the smallest enclosing box covering both BpB_p and BgB_g. The generalized IoU is GIoU⁡(Bp,Bg)=IoU⁡(Bp,Bg)−|C|−|Bp∪Bg||C|,GIoU(B_p,B_g)=IoU(B_p,B_g)- |C|-|B_p∪ B_g||C|, (10) and the corresponding loss is ℒGIoU=1−GIoU⁡(Bp,Bg).L_GIoU=1-GIoU(B_p,B_g). (11) Scale loss. Let p=maxp−minp,g=maxg−mings_p=b^p_ -b^p_ , _g=b^g_ -b^g_ (12) denote the predicted and ground-truth box sizes. We measure the relative scale discrepancy along each axis as ℒscale=13∑k∈x,y,z|sp,k−sg,k|max⁡(sp,k,sg,k)+ϵ.L_scale= 13 _k∈\x,y,z\ |s_p,k-s_g,k| (s_p,k,s_g,k)+ε. (13) Center loss. Let p=minp+maxp2,g=ming+maxg2c_p= b^p_ +b^p_ 2, _g= b^g_ +b^g_ 2 (14) denote the predicted and ground-truth box centers, and let dg=‖maxg−ming‖2d_g=\|b^g_ -b^g_ \|_2 (15) be the diagonal length of the ground-truth box. We define the normalized center loss as ℒcenter=min⁡(‖p−g‖2dg+ϵ,1).L_center= ( \|c_p-c_g\|_2d_g+ε,1 ). (16) E.4 Detailed Orientation Loss The ground-truth orientation is represented as a rotation R∈SO(3)R∈ SO(3), while training adopts its continuous 6D representation. From the hidden state at the </ori> token, the orientation head regresses two 3D vectors 1,2∈ℝ3.a_1,a_2 ^3. (17) We then apply Gram–Schmidt orthogonalization to recover two orthonormal axes r and f, corresponding to the predicted right and front directions. Let ,r,f denote the ground-truth right and front axes. As defined in Eq. (4), the orientation loss is ℒori=βalignℒalign+βorthoℒortho.L_ori= _alignL_align+ _orthoL_ortho. (18) Axis alignment loss. We define ℒalign=12[(1−cos⁡(^,)+ϵ)λ+(1−cos⁡(^,)+ϵ)λ].L_align= 12 [ (1- ( r,r)+ε )^λ+ (1- ( f,f)+ε )^λ ]. (19) Compared with a plain cosine loss, the exponent λ allows us to adapt the sensitivity of the alignment term during training. When the alignment error becomes sufficiently small, decreasing λ prevents the loss from saturating too early and preserves optimization sensitivity in the low-error regime. Orthogonality loss. To ensure that the predicted axes remain geometrically valid, we use ℒortho=|cos⁡(^,^)|.L_ortho=| ( r, f)|. (20) This term penalizes non-orthogonal predictions and stabilizes the recovered orientation basis. E.5 Adaptive Exponent Schedule In practice, the exponent λ in Eq. (19) is adaptively reduced during training: λ=1,ℒalign>τ1,1/2,τ2<ℒalign≤τ1,1/4,ℒalign≤τ2,λ= cases1,&L_align> _1,\\ 1/2,& _2<L_align≤ _1,\\ 1/4,&L_align≤ _2, cases (21) where τ1 _1 and τ2 _2 are predefined thresholds. Intuitively, this strategy makes the optimization initially focus on coarse orientation alignment, and later remain sensitive to fine-grained directional refinement. E.6 Detailed Symmetry Loss To preserve axis-level structural regularity, we further supervise the predicted symmetry representation. Let s and s denote the predicted and ground-truth symmetry vectors, respectively. We use a cosine-based loss: ℒsym=1−cos⁡(^,).L_sym=1- ( s,s). (22) This loss complements the token-level symmetry prediction by encouraging the model to preserve geometric symmetry structure in a continuous feature space. In this sense, token-level decoding captures the discrete output format, while the auxiliary symmetry loss provides additional geometric regularization. E.7 Hyperparameters and Implementation Details The default coefficients are set as αgiou=0.5,αscale=0.25,αcenter=0.25,βalign=0.8,βortho=0.2. _giou=0.5, _scale=0.25, _center=0.25, _align=0.8, _ortho=0.2. For the overall objective, λbox,λori,λsym _box, _ori, _sym are linearly warmed up during training. In addition, after step 2000, span-aware weighting is applied to the language modeling loss, with weight 33 for tokens inside the <bbox> and <ori> spans and weight 11 elsewhere.

Instant research discovery