Paper deep dive
Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models
Shengli Zhou, Minghang Zheng, Feng Zheng, Yang Liu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/27/2026, 1:15:22 AM
Summary
QuatRoPE is a novel positional embedding method for Large Language Models (LLMs) that enables scalable 3D spatial reasoning by encoding absolute 3D coordinates as holistic vectors and transforming them into pairwise relative positions via quaternion rotations in attention layers. It addresses the scalability issues of quadratic input length and the limitations of absolute position encoding, while the Isolated Gated RoPE Extension (IGRE) minimizes interference with existing language-based positional embeddings.
Entities (4)
Relation Signals (3)
QuatRoPE → enhances → Spatial Reasoning
confidence 95% · QuatRoPE, a novel 3D positional encoding that explicitly models objects’ pairwise relative positions through quaternion rotations, enhancing the spatial understanding for 3D LLMs.
ASR → evaluates → Spatial Reasoning
confidence 95% · we introduce a diagnostic benchmark, the Attribute-free Spatial Reasoning (ASR) benchmark, to isolate and more directly probe a model’s spatial reasoning capabilities.
IGRE → minimizesinterferencewith → Language RoPE
confidence 95% · IGRE, which effectively limits QuatRoPE's influence to object-related tokens, thereby minimizing interference with the LLM's existing positional embeddings
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spatial reasoning focuses on locating target objects based on spatial relations in 3D scenes, which plays a crucial role in developing intelligent embodied agents. Due to the limited availability of 3D scene-language paired data, it is challenging to train models with strong reasoning ability from scratch. Previous approaches have attempted to inject 3D scene representations into the input space of Large Language Models (LLMs) and leverage the pretrained comprehension and reasoning abilities for spatial reasoning. However, models encoding absolute positions struggle to extract spatial relations from prematurely fused features, while methods explicitly encoding all spatial relations (which is quadratic in the number of objects) as input tokens suffer from poor scalability. To address these limitations, we propose QuatRoPE, a novel positional embedding method with an input length that is linear to the number of objects, and explicitly calculates pairwise spatial relations through the dot product in attention layers. QuatRoPE's holistic vector encoding of 3D coordinates guarantees a high degree of spatial consistency, maintaining fidelity to the scene's geometric integrity. Additionally, we introduce the Isolated Gated RoPE Extension (IGRE), which effectively limits QuatRoPE's influence to object-related tokens, thereby minimizing interference with the LLM's existing positional embeddings and maintaining the LLM's original capabilities. Extensive experiments demonstrate the effectiveness of our approaches. The code and data are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.24721v1
- Canonical: https://arxiv.org/abs/2603.24721v1
Trouble viewing inline? Open PDF directly →
Full Text
63,615 characters extracted from source content.
Expand or collapse full text
Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models Shengli Zhou1 Minghang Zheng2 Feng Zheng1 Yang Liu2,3🖂^2,3 1Department of Computer Science and Engineering, Southern University of Science and Technology 2Wangxuan Institute of Computer Technology, Peking University 3State Key Laboratory of General Artificial Intelligence, Peking University zhousl2022@mail.sustech.edu.cn, minghang, yangliu@pku.edu.cn, f.zheng@ieee.org Work done as an intern at Peking University. 🖂 Corresponding author. Abstract Spatial reasoning focuses on locating target objects based on spatial relations in 3D scenes, which plays a crucial role in developing intelligent embodied agents. Due to the limited availability of 3D scene-language paired data, it is challenging to train models with strong reasoning ability from scratch. Previous approaches have attempted to inject 3D scene representations into the input space of Large Language Models (LLMs) and leverage the pretrained comprehension and reasoning abilities for spatial reasoning. However, models encoding absolute positions struggle to extract spatial relations from prematurely fused features, while methods explicitly encoding all spatial relations (which is quadratic in the number of objects) as input tokens suffer from poor scalability. To address these limitations, we propose QuatRoPE, a novel positional embedding method with an input length that is linear to the number of objects, and explicitly calculates pairwise spatial relations through the dot product in attention layers. QuatRoPE’s holistic vector encoding of 3D coordinates guarantees a high degree of spatial consistency, maintaining fidelity to the scene’s geometric integrity. Additionally, we introduce the Isolated Gated RoPE Extension (IGRE), which effectively limits QuatRoPE’s influence to object-related tokens, thereby minimizing interference with the LLM’s existing positional embeddings and maintaining the LLM’s original capabilities. Extensive experiments demonstrate the effectiveness of our approaches. The code and data are available at https://github.com/oceanflowlab/QuatRoPE. Figure 1: (a) In QuatRoPE, we embed the absolute 3D position of each object to the corresponding token, thus limiting the input length linear to object count. (b) By leveraging a dedicated rotation scheme, when tokens with 3D position embedding interact in the attention layer of the LLM, the absolute coordinates are transformed into pairwise relative positions, empowering spatial reasoning. (c) In previous methods, as the positions are decoupled into individual coordinates, when the coordinate of some axis is close (as marked in green), the attention score is incorrectly inflated. (d) QuatRoPE encodes positions as holistic vectors, correctly representing spatial relations. 1 Introduction Spatial reasoning refers to the process of locating a target object according to its spatial relations with other objects (i.e., anchor objects) in the scene. Such a process is the core step for solving 3D Vision-Language (3D VL) tasks, including 3D Visual Grounding (3D VG) and 3D Visual Question-Answering (3D VQA). As the process of spatial reasoning is based on the spatial relations between objects, the accurate perception of inter-object spatial relations is a prerequisite for acquiring a strong spatial reasoning ability. Thus, a core challenge in spatial reasoning is effectively encoding and computing object relations. Due to the scarcity of 3D scene-text paired data, training a model with a strong spatial reasoning capability from scratch is challenging. With the development of Large Language Models (LLMs), previous works [13, 14, 34] have integrated point cloud representations with natural language, leveraging LLMs’ large-scale pretrained reasoning abilities to perform spatial reasoning on 3D scenes [21]. In these works, the models represent scene layouts using either absolute or relative object positions. (1) Absolute position encoding incorporates objects’ 3D coordinates as part of their features [14, 38, 9]. However, absolute coordinates carry little inherent meaning since the origin and orientation in 3D scenes have no natural physical definition, despite preserving geometric relationships between objects. Moreover, since absolute positional encoding does not explicitly represent relative geometry, models must laboriously learn these relations from limited data. This challenge is further compounded by premature feature fusion, which obstructs LLMs from extracting positions and computing pairwise object relationships. (2) For methods that directly encode pair-wise object relations using additional input tokens, the length of the LLM’s input sequence grows quadratically with object count, which can easily exceed the input limits of many LLMs (e.g., the InteriorGS [25] dataset contains an average of over 554 objects per scene, yielding over 153,181 relations). While pruning strategies, such as 3DGraphLLM’s [34] KNN approach, reduce tokens by keeping only nearby objects, this risks omitting critical relations since spatial proximity does not ensure relevance, potentially causing errors in spatial reasoning. In contrast to previous approaches, we propose QuatRoPE, which uses only O(n)O(n) input tokens while preserving all O(n2)O(n^2) spatial relations (where n is the number of objects in the scene), supporting scalability and avoiding erroneous pruning. As shown in Fig. 1 (a) and (b), the core idea of QuatRoPE is to inject explicit absolute positional encodings for all object-related tokens111Object-related tokens: LLM’s input tokens for objects’ 2D/3D features and identifiers like <obj001>. and leverage the Transformer’s attention mechanism to convert absolute positions into relative relationships during query-key dot products. Specifically, we apply quaternion rotations to query and key vectors based on the corresponding objects’ 3D coordinates. By constructing specific mathematical formulations for rotation, the dot product (i.e., attention score) between two rotated vectors depends solely on their relative positions in the 3D scene, efficiently providing pairwise spatial relations for LLM. Additionally, as shown in Fig. 1 (d), QuatRoPE encodes object coordinates as holistic vectors (instead of encoding the coordinate on each axis independently). Such an approach prevents inflated attention scores from small coordinate differences on single axes, accurately representing spatial layouts. The scarcity of 3D scene-text paired data also makes it difficult to train LLMs with dual RoPE (i.e., language RoPE and QuatRoPE) from scratch. At the same time, when applying QuatRoPE on LLMs with Language RoPE, both RoPEs rotate query and key vectors, yielding interference and hindering position perception for both text and objects. To address this issue, we further propose Isolated Gated RoPE Extension (IGRE). In IGRE, object-related tokens are extended with QuatRoPE-specific dimensions (zero-padded for other tokens), isolating QuatRoPE from language RoPE. Also, IGRE ensures that attention scores only adjust to reflect relative positions when two object tokens interact through the dot product (i.e., gated), preserving the LLM’s original linguistic capabilities. While benchmarks like SQA3D [19], ScanRefer [5], and Multi3DRef [35] evaluate aspects of spatial understanding, they are not designed to—and thus are inherently limited in—purely assessing spatial reasoning. In these tasks, language descriptions often intertwine spatial relationships with non-spatial cues, such as object categories or attributes. This makes it difficult to determine whether a model’s success stems from true spatial comprehension or from simply recognizing semantic or visual features. To address this deficiency, we introduce a diagnostic benchmark, the Attribute-free Spatial Reasoning (ASR) benchmark, to isolate and more directly probe a model’s spatial reasoning capabilities. In our benchmark, we select ScanQA’s [2] uniquely-answerable object-name questions, filter out those revealing target attributes to enforce spatial reasoning, and convert them into 3D VG format to eliminate language generation biases. By such an approach, the ASR benchmark can make a fair and rigorous comparison of spatial reasoning. Across all these benchmarks, our approach consistently outperforms strong baselines, showing that QuatRoPE provides effective positional cues for spatial understanding. In summary, our contributions are as follows: (1) We propose QuatRoPE, a novel 3D positional encoding that explicitly models objects’ pairwise relative positions through quaternion rotations, enhancing the spatial understanding for 3D LLMs. (2) We propose IGRE to combine QuatRoPE with the language RoPE to reduce interference. (3) We construct a challenging benchmark ASR for exclusively evaluating 3D spatial understanding. (4) We achieve consistent and large-margin gains on ASR and multiple existing 3D VL benchmarks, validating the effectiveness. 2 Related Work 2.1 3D VL Tasks on Spatial Reasoning 3D Vision-Language (3D VL) refers to multi-modal tasks that are solved by combining 3D scenes and natural language, such as 3D Visual Grounding (3D VG) [16] and 3D Visual Question-Answering (3D VQA) [36, 33, 18]. Previously, ScanRefer [5] introduced the task of single-object 3D VG, where the model finds an object based on a text description (e.g., locating “the bottle on top of the table”); Multi3DRef [35] extended this task to cases where the number of ground-truth objects varies, further testing the model’s spatial reasoning skills. For 3D VQA, ScanQA [2] was the first to define the task of answering questions about 3D scenes (e.g., answering “What color is the object under the chair and next to the lamp?”). SQA3D [19] further developed this into situated question-answering, where models answer questions from a specific viewpoint, which better aligns with the practical requirements for applications such as intelligent robots. Other datasets, such as Nr3D and Sr3D [1], have also defined variants of these 3D VL tasks. However, current benchmarks in these tasks fail to directly reflect models’ spatial reasoning ability, as objects’ attributes (e.g., category, color, and shape) in language descriptions can help models locate target objects without spatial reasoning. In contrast, we propose a diagnostic benchmark that omits all attributes of the target objects, thereby evaluating models’ spatial reasoning abilities exclusively. 2.2 3D LLMs for Spatial Reasoning When solving spatial reasoning tasks, models should be able to precisely perceive the spatial relation between objects to obtain the correct answer. Due to the scarcity of 3D scene-text paired data, previous works have leveraged the perception and reasoning capabilities of LLMs to enhance spatial reasoning. Among these works, 3D-LLM [13] represents the 3D entire scene as a holistic feature. Though such an encoding approach can preserve the scene layout, the compact representation loses details and entangles objects’ features, which requires the model to identify objects and impedes object-level spatial reasoning. To facilitate object-level reasoning, LEO [15] and Chat-Scene [14] segment the scene into objects and encode the feature of each object as input tokens. Despite their promising performance, they struggle to extract spatial relations between objects from absolute positions that are prematurely fused with geometric features. To solve this problem, 3DGraphLLM [34] utilizes additional input tokens to explicitly represent spatial relations between objects. Additionally, since the number of relations is quadratic to the object count (which can easily exceed LLMs’ input limits), 3DGraphLLM employs a K-Nearest-Neighbors (KNN) strategy, encoding only the spatial relations between each object and its nearest objects. However, this approach is error-prone as proximity does not always indicate task-relevant importance. In contrast, we propose QuatRoPE that encodes 3D positions on each object-related token. Through a dedicated embedding scheme, it converts absolute coordinates to pairwise relative positions of all objects via query-key dot products in attention layers. Such a method not only ensures robustness to global rotations and translations but also mitigates error-prone pruning. 2.3 Rotary Positional Embeddings Rotary Positional Embedding (RoPE) [26] enhances transformers by encoding relative positions through complex rotations of query/key vector segments of 2 components. Each segment is rotated by mθim _i (where m is the absolute position and θi _i is the frequency), making attention scores depend only on position differences. By this mechanism, the dot products of query and key vectors are only related to the difference in position, transforming absolute positions into relative positions. Currently, RoPE has become foundational in various LLMs, including LLaMA [10] and QWen [3]. For multi-modal data (e.g., images), M-RoPE [27] extends RoPE by grouping segments for multi-position embedding. Video-RoPE [30] further introduces Low-frequency Temporal Allocation to focus on long-range dependencies along the time axis, and Diagonal Layout to maintain spatial-textual position consistency. However, 3D scenes pose unique challenges: existing methods overemphasize proximity along individual axes. When two objects have similar coordinates on one axis (despite being distant overall), these methods inflate attention scores due to incorrectly amplified dimension-wise products in segment groups corresponding to the axis. This creates false “nearby” associations between objects, impairing the model’s understanding of spatial relationships. In contrast, QuatRoPE encodes coordinates as integrated vectors by rotating each individual dimension of the query and key vectors according to the corresponding 3D coordinates. This approach ensures attention scores increase only when objects are truly proximate in 3D space, effectively representing scene layouts. 3 Method 3.1 Baseline Models Revisited To utilize LLMs for perceiving and reasoning on scene information, previous 3D LLMs [14, 34] have aligned and injected point cloud features into LLMs. In these works, the pipeline is as follows: The model is input with a point cloud and textual instructions. To begin with, the model utilizes either ground-truth segmentations or predictions from off-the-shelf segmentation models [23, 28, 17] to segment the point cloud into a series of objects, thereby facilitating the model’s ability to perform object-level reasoning. For each object, its features (e.g., 3D geometric feature calculated by PointNet++ [22]) are projected into the input space of LLM through projection layers. Additionally, a set of object identifiers is defined and trained to fit within the input space of LLMs (e.g., <obj005> represents the fifth object in the scene enumeration order), helping the model refer to specific objects in the scene. Finally, the project features and object identifiers are parsed into LLMs as input tokens (each object-related token corresponds to a single object), along with other language tokens for prompts and questions. Inside the LLM, the feature vectors of tokens serve as the input embeddings for the first self-attention layer. Thus, when the model calculates attention scores between tokens, it also forms attention associations between objects based on their features. In previous works that prematurely fuse absolute coordinates into objects’ overall features, the spatial information in feature vectors is implicit and sparse, weakening the relative spatial information in the association. Therefore, in QuatRoPE, we enhance the spatial information by providing an explicit encoding on each object-related token, representing its absolute position in the scene. When object-related tokens interact in the attention layers of the LLM, the absolute positions can be further transformed into relative spatial cues between objects, empowering the model’s understanding of spatial relations. 3.2 QuatRoPE To provide the LLM with pairwise spatial relations between objects, while constraining the number of input tokens to be linear to the number of objects, we propose QuatRoPE. The core mechanism of QuatRoPE is to first encode the corresponding object’s absolute coordinates on object-related tokens, and then calculate pairwise relative positions between objects during the dot products for query and key vectors in attention layers. Initially, given that spatial reasoning operates at the object level, we represent each object’s 3D position through its bounding box center. To facilitate relative-position calculation via dot products in attention layers, we encode absolute positions using rotations. Such an approach transforms absolute coordinates into relative positions, since dot products reflect angle differences. Specifically, the query vectors and the key vectors in self-attention layers are grouped into 3D segments, represented each as a pure quaternion (denoted as q→ q and k→ k with zero real part), and apply quaternion rotation before each attention layer. Let m→ m and n→ n be the absolute 3D coordinates of the objects corresponding to query vector q→ q and key vector k→ k, and f(x→,p→)f( x, p) be the function for rotating the query or key vector x→ x according to the corresponding absolute 3D position p→ p. Since the attention score should only relate to the relative position (i.e., m→−n→ m- n), the rotation function f should satisfy for some ternary function g: ⟨f(q→,m→),f(k→,n→)⟩=g(q→,k→,m→−n→) <f( q, m),f( k, n) >=g( q, k, m- n) (1) When encoding coordinates independently, the proximity of coordinates on a single axis can mislead the model into amplifying attention scores for objects that are indeed far away. Thus, QuatRoPE embeds the coordinates as a unified vector, i.e., the components of the query and key vectors are adjusted based on the object’s position rather than its coordinate along a specific axis. Since coordinates are 3D vectors, we leverage quaternion rotation with three degrees of freedom to embed them. Formally, the rotation function can be expressed via Euler angle decomposition as: f(q→,m→)=Q(m→)q→Q−1(m→)Q(m→)=Qz(mz)Qy(my)Qx(mx)Qx(mx)=cos[θx(mx)/2]+i^sin[θx(mx)/2]Qy(my)=cos[θy(my)/2]+j^sin[θy(my)/2]Qz(mz)=cos[θz(mz)/2]+k^sin[θz(mz)/2] casesf( q, m)=Q( m)~ q~Q^-1( m)\\ Q( m)=Q_z(m_z)~Q_y(m_y)~Q_x(m_x)\\ Q_x(m_x)= [ _x(m_x)/2 ]+ i [ _x(m_x)/2 ]\\ Q_y(m_y)= [ _y(m_y)/2 ]+ j [ _y(m_y)/2 ]\\ Q_z(m_z)= [ _z(m_z)/2 ]+ k [ _z(m_z)/2 ] cases (2) where Q’s are rotation matrices and θ’s are unary functions. Through Equation (2), we transform the requirement in QuatRoPE (i.e., converting absolute coordinates to relative positions via dot products) into deriving θ’s that satisfy Equation (1). To solve the equation, we transform the dot product into the real part of the product of the rotation functions to yield a form with multiplication between the rotation matrices (i.e., Q−1(m→)Q^-1( m) and Q(n→)Q( n)). ⟨f(q→,m→),f(k→,n→)⟩= <f( q, m),f( k, n) >= ℜ[f(q→,m→)f∗(k→,n→)] [f( q, m)f^*( k, n)] (3) = = ℜ[Q(m→)q→Q−1(m→)Q(n→)k→∗Q−1(n→)] [Q( m)~ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)] where k→∗ k^* denotes the conjugate of quaternion k→ k, and ℜ denotes the real part of the quaternion. To pair every Q(m→)Q( m) with Q(n→)Q( n), according to the real-part invariance of quaternion rotation (i.e., ℜ(Q−1kQ)=ℜ(k) (Q^-1kQ)= (k)), left multiplying Q−1(m→)Q^-1( m) and right multiplying Q(m→)Q( m) yields: ⟨f(q→,m→),f(k→,n→)⟩=ℜ[q→Q−1(m→)Q(n→)k→∗Q−1(n→)Q(m→)] <f( q, m),f( k, n) >= [ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)~Q( m)] (4) According to Equation (1), since ⟨f(q→,m→),f(k→,n→)⟩ <f( q, m),f( k, n) > should only relate to m→−n→ m- n, we have Q−1(n→)Q(m→)=Q(m→−n→)Q^-1( n)~Q( m)=Q( m- n). The equation further yields that unary functions θx,θy _x, _y, and θz _z should be linear (as detailed in the appendix). Thus, an approximate solution for QuatRoPE is: f(q→,m→)=Q(m→)q→Q−1(m→)Q(m→)=Qz(mz)Qy(my)Qx(mx)Qx(mx)=cos[mxθx(1)/2]+i^sin[mxθx(1)/2]Qy(my)=cos[myθy(1)/2]+j^sin[myθy(1)/2]Qz(mz)=cos[mzθz(1)/2]+k^sin[mzθz(1)/2] casesf( q, m)=Q( m)~ q~Q^-1( m)\\ Q( m)=Q_z(m_z)~Q_y(m_y)~Q_x(m_x)\\ Q_x(m_x)= [m_x _x(1)/2]+ i [m_x _x(1)/2]\\ Q_y(m_y)= [m_y _y(1)/2]+ j [m_y _y(1)/2]\\ Q_z(m_z)= [m_z _z(1)/2]+ k [m_z _z(1)/2] cases (5) where θx(1) _x(1), θy(1) _y(1), and θz(1) _z(1) are frequencies for quaternion rotations. According to Equation (1), as we perform rotation by q→:=f(q→,m→) q:=f( q, m) and k→:=f(k→,n→) k:=f( k, n) before each attention layer, the attention scores between object-related tokens reflect their relative positions. By such an approach, QuatRoPE can effectively convey relative positional information for LLMs to perform spatial reasoning. Moreover, for objects that are spatially close in a scene, their QuatRoPE embeddings are similar, resulting in larger attention scores. This behavior aligns with the human cognitive mechanism of Maxim of Relation [11]. For example, when referring to “the window to the left of the door,” if multiple windows exist at varying distances, humans typically imply the one closest to the door. Such alignment consequently enhances the LLM’s ability to comprehend implicit references in natural language. 3.3 Isolated Gated RoPE Extension Although QuatRoPE can effectively provide spatial information for LLMs to utilize, training point cloud-based 3D LLMs presents new challenges. Due to the scarcity of 3D scene-text paired data, training an LLM with language RoPE and QuatRoPE from scratch is impractical. However, simply applying QuatRoPE along with language RoPE may cause interference as they simultaneously perform rotation on query and key vectors. Table 1: Results for the comparative experiments, the best scores obtained by using ground-truth or predicted segmentation are underlined, and the overall best scores are in bold. By applying QuatRoPE, our models have achieved consistent gains under all metrics. Model Detector / ScanRefer Multi3DRef SQA3D Segmentation Acc@0.25 Acc@0.5 Multi@0.25 Multi@0.5 F1@0.25 F1@0.5 EM@1 ScanRefer [5] VoteNet 39.0 26.1 32.1 21.3 – – – 3DJCG [4] VoteNet 49.6 37.3 41.4 30.8 – 26.6 – Vil3DRef [7] PointGroup 47.9 37.7 40.3 30.7 – – – D3Net [6] PointGroup – 37.9 – 30.1 – 32.2 – VPP-Net [24] Group-free 55.7 43.3 50.3 39.0 – – – AugRefer [29] Group-free 55.7 44.0 50.0 39.1 – – – M3DRef-CLIP [35] PointGroup – 44.7 – 36.8 42.8 38.4 – MA2TransVG [31] Group-free 57.9 45.7 53.8 41.4 – – – 3D-VisTA [37] Mask3D 50.6 45.8 43.7 39.1 – – 48.5 3DSyn [32] Mask3D 52.3 46.2 – – – – – TSP3D [12] N/A 56.5 46.7 – – – – – PQ3D [38] PQ3D Promptable – 51.2 – 46.2 – 50.1 47.1 BridgeQA [20] VoteNet – – – – – – 52.9 Scene-LLM [9] N/A – – – – – – 53.6 Chat-Scene-1B [14] GT 50.7 50.3 42.7 42.3 53.3 52.9 50.7 Chat-Scene-1B + QuatRoPE (Ours) GT 55.4 55.0 47.8 47.4 58.1 57.7 53.1 3DGraphLLM-1B [34] GT 55.9 55.8 47.9 47.7 58.6 58.4 51.1 3DGraphLLM-1B + QuatRoPE (Ours) GT 58.3 58.2 50.8 50.6 60.7 60.5 53.2 Chat-Scene-7B [14] Mask3D 55.5 50.2 47.8 42.9 57.1 52.4 54.6 Chat-Scene-7B + QuatRoPE (Ours) Mask3D 57.8 52.2 51.1 45.7 59.5 54.8 54.7 3DGraphLLM-7B [34] Mask3D 57.0 51.3 – – 60.1 55.4 53.1 3DGraphLLM-7B + QuatRoPE (Ours) Mask3D 58.2 52.5 54.3 49.2 60.6 56.0 55.2 Table 2: Results on our spatial reasoning benchmark. Our model achieves significant and consistent gains across various settings. Model LLM Acc @ 0.25 Gain Acc @ 0.5 Gain Chat-Scene [14] Llama-3.2-1B-Instruct 22.92 – 22.92 – Chat-Scene + QuatRoPE (Ours) Llama-3.2-1B-Instruct 27.38 4.46 (19.48%) 27.38 4.46 (19.48%) 3DGraphLLM [34] Llama-3.2-1B-Instruct 25.89 – 25.60 – 3DGraphLLM + QuatRoPE (Ours) Llama-3.2-1B-Instruct 29.76 3.87 (14.94%) 29.76 4.17 (16.28%) 3DGraphLLM [34] Llama-3-8B-Instruct 37.50 – 36.90 – 3DGraphLLM + QuatRoPE (Ours) Llama-3-8B-Instruct 41.96 4.46 (11.90%) 41.96 5.06 (13.71%) Table 3: Results for ablation study on different composition approaches, the best scores of each baseline are marked in bold. RoPE Composition Approach ScanRefer SQA3D Acc @ 0.25 Acc @ 0.5 Multi @ 0.25 Multi @ 0.5 EM @ 1 Baseline: Chat-Scene [14] None 50.72 50.33 42.69 42.29 50.72 Trans-Additive 53.12 52.79 45.48 45.14 52.96 IGRE (Ours) 55.44 55.00 47.81 47.36 53.14 Baseline: 3DGraphLLM [34] None 55.92 55.75 47.92 47.74 51.09 Trans-Additive 53.68 53.38 45.94 45.64 51.55 IGRE (Ours) 58.30 58.15 50.77 50.60 53.20 Table 4: Results for ablation study on different RoPE methods, the best scores of each baseline are marked in bold. Explicit Positional Encoding Approach ScanRefer SQA3D Acc @ 0.25 Acc @ 0.5 Multi @ 0.25 Multi @ 0.5 EM @ 1 Baseline: Chat-Scene [14] None 50.72 50.33 42.69 42.29 50.72 Raw Coordinates 52.26 52.01 44.41 44.17 51.40 M-RoPE 54.30 53.92 46.44 46.10 51.55 QuatRoPE (Ours) 55.44 55.00 47.81 47.36 53.14 Baseline: 3DGraphLLM [34] None 55.92 55.75 47.92 47.74 51.09 Raw Coordinates 3.60 3.44 3.57 3.46 35.50 M-RoPE 57.69 57.48 50.07 49.83 53.14 QuatRoPE (Ours) 58.30 58.15 50.77 50.60 53.20 Meanwhile, directly applying QuatRoPE rotations to query and key vectors also introduces erroneous associations between object-related tokens and non-object tokens (e.g., tokens for system prompts, questions, instructions, and relations). While RoPE-based positional encodings can represent arbitrary positions or coordinates, they inherently cannot express the concept that “non-object tokens do not correspond to a position in the 3D coordinate system.” Even if non-object tokens are left unrotated, it is equivalent to positioning them at (0,0,0)(0,0,0). Such a configuration misleadingly biases the model to disproportionately attend to relationships between non-object tokens and objects near the coordinate origin. Figure 2: An illustration of ASR’s construction pipeline. To address these problems, we introduce Isolated Gated RoPE Extension (IGRE). For an object-related token, we apply QuatRoPE on a base vector and concatenate it to the query/key vector. For a non-object token, we concatenate a zero vector to pad the query/key vector to the same dimension as object-related tokens. By this approach, we isolate the components rotated by language RoPE and QuatRoPE, thus reducing the interference between multiple RoPEs. Additionally, as non-object tokens are zero-padded, the “non-existence” of non-object tokens in the 3D scene can be well-represented. Also, under such a representation, when a query or key vector that belongs to a non-object token is involved in the dot product, the padded zeros ensure that element-wise products in these dimensions are 0, keeping the attention scores unchanged. Thus, IGRE can constrain QuatRoPE’s adjustments on attention scores between object-related tokens, maximizing the retention of the pretrained LLM’s capabilities in understanding natural language and performing reasoning. QuatRoPE’s adjustments to attention scores are gated within the dot products with both query and key vectors from object-related tokens. Therefore, IGRE can maximally reduce interference and preserve LLM’s ability to understand natural language and perform reasoning. 3.4 Attribute-free Spatial Reasoning Benchmark Though previous datasets in 3D VL tasks can reflect models’ spatial reasoning abilities, none of them can fully isolate the impact of other model abilities on the final scores. Under 3D VL tasks, descriptions of object attributes (e.g., category, color, and shape) often entangle with those for spatial relations, improperly facilitating the model an unintended bypass of spatial reasoning. For example, for a 3D VG task locating “the red chair under the window and next to the table” while there is only one red chair in the scene, the model can obtain the answer by recognizing the red chair, rather than locating it through its relations with the window and the table. To address these problems, we propose the Attribute-free Spatial Reasoning (ASR) benchmark. First, we select a series of 3D VQA questions with unique answers in ScanQA that ask for the name of the object. Then, we filter out questions that do not provide any other attributes of the target object, requiring the model to obtain the answer through spatial reasoning (e.g., “What is the object in front of the tall white shelf?”). Finally, we convert these queries into a 3D VG format (e.g., “The object in front of the tall white shelf”), where the model only needs to perform multiple-choice selection between objects in the scene, eliminating the impact of different language generation abilities between models. The pipeline for constructing our ASR benchmark is illustrated in Fig. 2. Through attribute-free questions and the 3D VG format setting of our benchmark, we ensure fair and rigorous comparisons of models’ spatial reasoning capabilities. 4 Experiments 4.1 Experimental Settings We validate the effectiveness of our method through experiments using strong point cloud-based 3D LLM Chat-Scene [14] and model 3DGraphLLM [34] as baselines. All models are trained on a combined dataset composed of ScanRefer [5], Multi3DRef [35], ScanQA [2], SQA3D [19], Scan2Cap [8], ReferIt3D [1], and Chat-Scene’s object alignment task. During training, LLMs are fine-tuned with LoRA (rank r=16r=16 and scaling factor α=16α=16) at a learning rate of 2×10−52× 10^-5. For the 3DGraphLLM baseline, we adopt the same setting, pruning scene graphs using KNN with k=2k=2. To evaluate spatial reasoning ability, we test models on 3D VG benchmarks, including ScanRefer and Multi3DRef (as they involve precise perception of objects’ spatial relations and locating objects according to instructions), and Situated 3D VQA benchmark SQA3D. 4.2 Comparative Experiments In this experiment, we aim to verify the effectiveness and generalizability of the proposed methods. We utilize Llama-3.2-1B-Instruct and Vicuna-7B-v1.5 as the LLM for Chat-Scene and 3DGraphLLM baselines, and apply QuatRoPE through IGRE to these models. Finally, we compare their performance with specialist and generalist models on multiple datasets to evaluate the gain for spatial reasoning. The results in Table 1 demonstrate that our method outperforms baselines across all metrics, particularly on 3D VG tasks that require higher spatial reasoning abilities. The results further indicate that QuatRoPE can effectively convey spatial information by providing relative object positions, verifying the correctness of our approaches. 4.3 Spatial Reasoning Ability Verification In the previous experiment, the improved scores across datasets generally indicate that the proposed methods can enhance models’ spatial reasoning abilities. To exclusively demonstrate QuatRoPE’s effectiveness in enhancing models’ spatial reasoning ability, we utilize our ASR benchmark for further evaluation. We conduct zero-shot comparisons between models with and without QuatRoPE. The results are shown in Table 2. The results in the table demonstrate that our model has achieved consistent and substantial gains throughout different experimental settings, directly verifying that the proposed method can enhance models’ performance in spatial reasoning. The results further indicate that our method can effectively provide useful relative spatial information to LLMs for solving 3D VL tasks. 4.4 Ablation Study To compare different RoPE settings, including composition approaches (i.e., IGRE and traditional additive approach) and RoPE methods, we perform an ablation study on these factors. Specifically, we utilize baseline models with Llama-3.2-1B-Instruct as the LLM. Then, we apply QuatRoPE via different composition methods, namely, Trans-Additive (where QuatRoPE and language RoPE simultaneously rotate query/key vectors, but with inverse frequencies to lower interference) and IGRE. Results are shown in Table 3. The results demonstrate that among different composition methods, IGRE surpasses the baseline and the model using the Trans-Additive approach under all metrics. Particularly, IGRE has obtained significant improvements on the 3D VG dataset ScanRefer, where spatial reasoning is the key to locating objects based on spatial relations. The results verify that IGRE can separate QuatRoPE from language RoPE better and minimize interference. Moreover, we also compare the performance of different positional encoding approaches (i.e., without explicit encoding, directly adding raw (x,y,z)(x,y,z) coordinates to feature vectors, M-RoPE [27], and QuatRoPE) using IGRE. The results are shown in Table 4. Models with explicit positional encoding outperform baseline models in most cases, suggesting that explicit positional encoding generally improves scene understanding. However, adding raw coordinates introduces absolute positions into the feature vectors and prevents them from being transformed into relative positions during the attention mechanism. Thus, such an approach disrupts models’ understanding of scene layouts, especially in models like 3DGraphLLM that rely heavily on input tokens to understand them. Among RoPE-based approaches, QuatRoPE outperforms M-RoPE across all metrics, demonstrating that encoding coordinates as holistic vectors can convey spatial relations between objects and represent scene layout more effectively. Table 5: Verification of advantage in holistic encoding. δ 3DGraphLLM-1B + QuatRoPE Gain 1 (All) 93.72 94.65 0.93 0.5 92.28 94.21 1.93 0.3 91.47 94.31 2.84 0.2 93.21 96.30 3.09 0.1 92.39 96.74 4.35 0.05 84.62 92.31 7.69 Figure 3: Qualitative results on the ScanRefer dataset. Target objects are correctly grounded by QuatRoPE (green), whereas the baseline Chat-Scene produces incorrect predictions (red). 4.5 Verification of Advantage in Holistic Encoding In previous RoPE, each axis is treated independently, causing close coordinates on a single axis to falsely appear “nearby” and disrupt attention. To address this issue, QuatRoPE encodes positions as holistic vectors. To verify the effectiveness of such a design, we re-split the ScanRefer [5] dataset according to the severity of the “false nearby” issue. Since such an issue occurs when the coordinate difference is small on some axis, severity is defined by the aspect ratio minΔx,ΔymaxΔx,Δy<δ \ x, y\ \ x, y\<δ for anchor-target object position differences (Δx,Δy,Δz)( x, y, z), with lower δ denoting more severe cases. As in Tab. 5, QuatRoPE outperforms the 3DGraphLLM baseline in all cases, and its advantage increases with severity, verifying the effectiveness. 4.6 Qualitative Results Finally, we visualize several cases in ScanRefer to demonstrate the effectiveness of our approach. The qualitative results are illustrated in Fig. 3. In these cases, as relative positions between objects are well represented, models can better align spatial information with descriptions like “surrounded by” and “next to”. Notably, in Case (c), while both doors are to the machine’s right, the correct one is closer (which can be explained by humans’ preference for the “Maxim of Relation” [11] in linguistics). Such a case also indicates that, as QuatRoPE correctly increases attention scores for proximate objects, models can align with human implication better, enabling models to understand and predict human-like spatial reasoning patterns across multimodal tasks. 5 Conclusion In this paper, we propose QuatRoPE, a positional embedding that encodes objects’ positions to tokens and leverages the attention mechanism to transform absolute positions into objects’ pairwise spatial relations. To minimize QuatRoPE’s interference with language RoPE, we further propose IGRE for separating dimensions for RoPEs and constraining QuatRoPE’s effect to object-related tokens. Extensive experiments demonstrate the effectiveness of QuatRoPE and IGRE. Moreover, through our ASR benchmark, we verify that our method can achieve large gains in spatial reasoning across various baselines, offering a solution for enhancing the spatial reasoning ability of 3D LLMs. Acknowledgements. This work was supported by the grants from the Beijing Natural Science Foundation 4252040, the Beijing Nova Program, and the National Natural Science Foundation of China 62372014. References [1] P. Achlioptas, A. Abdelreheem, F. Xia, M. Elhoseiny, and L. J. Guibas (2020) ReferIt3D: neural listeners for fine-grained 3d object identification in real-world scenes. In 16th European Conference on Computer Vision (ECCV), Cited by: §2.1, §4.1. [2] D. Azuma, T. Miyanishi, S. Kurita, and M. Kawanabe (2022) ScanQA: 3d question answering for spatial scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2.1, §4.1. [3] J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y. Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y. Zhang, Z. Zhang, C. Zhou, J. Zhou, X. Zhou, and T. Zhu (2023) Qwen technical report. External Links: 2309.16609, Link Cited by: §2.3. [4] D. Cai, L. Zhao, J. Zhang, L. Sheng, and D. Xu (2022-06) 3DJCG: a unified framework for joint dense captioning and visual grounding on 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 16464–16473. Cited by: Table 1. [5] D. Z. Chen, A. X. Chang, and M. Nießner (2020) Scanrefer: 3d object localization in rgb-d scans using natural language. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, p. 202–221. Cited by: Appendix C, §1, §2.1, Table 1, §4.1, §4.5. [6] D. Z. Chen, Q. Wu, M. Nießner, and A. X. Chang (2022) D3 3net: A unified speaker-listener architecture for 3d dense captioning and visual grounding. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXII, S. Avidan, G. J. Brostow, M. Cissé, G. M. Farinella, and T. Hassner (Eds.), Lecture Notes in Computer Science, Vol. 13692, p. 487–505. External Links: Link, Document Cited by: Table 1. [7] S. Chen, M. Tapaswi, P. Guhur, C. Schmid, and I. Laptev (2022) Language conditioned spatial relation reasoning for 3d object grounding. In NeurIPS, Cited by: Table 1. [8] Z. Chen, A. Gholami, M. Niessner, and A. X. Chang (2021-06) Scan2Cap: context-aware dense captioning in rgb-d scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3193–3203. Cited by: §4.1. [9] R. Fu, J. Liu, X. Chen, Y. Nie, and W. Xiong (2025-02) Scene-llm: extending language model for 3d visual reasoning. In Proceedings of the Winter Conference on Applications of Computer Vision (WACV), p. 2195–2206. Cited by: §1, Table 1. [10] A. Grattafiori, A. Dubey, A. Jauhri, et al. (2024) The llama 3 herd of models. External Links: 2407.21783, Link Cited by: §2.3. [11] H. P. Grice (1975) Logic and conversation. Syntax and Semantics 3, p. 41–58. External Links: Link Cited by: §3.2, §4.6. [12] W. Guo, X. Xu, Z. Wang, J. Feng, J. Zhou, and J. Lu (2025-06) Text-guided sparse voxel pruning for efficient 3d visual grounding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), p. 3666–3675. Cited by: Table 1. [13] Y. Hong, H. Zhen, P. Chen, S. Zheng, Y. Du, Z. Chen, and C. Gan (2023) 3D-llm: injecting the 3d world into large language models. NeurIPS. Cited by: §1, §2.2. [14] H. Huang, Y. Chen, Z. Wang, R. Huang, R. Xu, T. Wang, L. Liu, X. Cheng, Y. Zhao, J. Pang, et al. (2024) Chat-scene: bridging 3d scene and large language models with object identifiers. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada. Cited by: Table 6, §B.1, Table 8, Table 10, Table 9, Appendix C, §1, §2.2, §3.1, Table 1, Table 1, Table 2, Table 3, Table 4, §4.1. [15] J. Huang, S. Yong, X. Ma, X. Linghu, P. Li, Y. Wang, Q. Li, S. Zhu, B. Jia, and S. Huang (2024) An embodied generalist agent in 3d world. In Proceedings of the International Conference on Machine Learning (ICML), Cited by: §2.2. [16] X. Li, J. Liu, Y. Guo, H. Dong, and Y. Liu (2025) 3D weakly supervised visual grounding at category and instance levels. In Proceedings of the International Conference on Robotics and Automation, Cited by: §2.1. [17] J. Luo, Z. Zhao, and Y. Liu (2025) Zero shot domain adaptive semantic segmentation by synthetic data generation and progressive adaptation. In Proceedings of the International Conference on Intelligent Robots and Systems, Cited by: §3.1. [18] Y. Lyu, X. Qin, X. Du, et al. (2025) Multi-path reasoning for multi-hop question answering over knowledge graph. Chinese Journal of Electronics 34 (2), p. 642–648. External Links: Document Cited by: §2.1. [19] X. Ma, S. Yong, Z. Zheng, Q. Li, Y. Liang, S. Zhu, and S. Huang (2023) SQA3D: situated question answering in 3d scenes. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2.1, §4.1. [20] W. Mo and Y. Liu (2024) Bridging the gap between 2d and 3d visual question answering: a fusion approach for 3d vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: Table 1. [21] Y. Peng, Z. Wang, G. Li, et al. (2026) A survey on fine-grained multimodal large language models. Chinese Journal of Electronics. Note: In press External Links: Document Cited by: §1. [22] C. R. Qi, L. Yi, H. Su, and L. J. Guibas (2017) PointNet++: deep hierarchical feature learning on point sets in a metric space. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Red Hook, NY, USA, p. 5105–5114. External Links: ISBN 9781510860964 Cited by: §3.1. [23] J. Schult, F. Engelmann, A. Hermans, O. Litany, S. Tang, and B. Leibe (2023) Mask3D: Mask Transformer for 3D Semantic Instance Segmentation. International Conference on Robotics and Automation (ICRA). Cited by: §3.1. [24] X. Shi, Z. Wu, and S. Lee (2024-06) Viewpoint-aware visual grounding in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14056–14065. Cited by: Table 1. [25] M. T. Inc. SpatialVerse Research Team (2025) InteriorGS: a 3d gaussian splatting dataset of semantically labeled indoor scenes. Note: https://huggingface.co/datasets/spatialverse/InteriorGS Cited by: §1. [26] J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu (2024) RoFormer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. External Links: ISSN 0925-2312, Document, Link Cited by: §2.3. [27] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin (2024) Qwen2-vl: enhancing vision-language model’s perception of the world at any resolution. External Links: 2409.12191, Link Cited by: §2.3, §4.4. [28] X. Wang, P. Wu, C. Zhang, et al. (2025) An extensible hierarchical multimodal semantic segmentation network for underwater scenarios. Chinese Journal of Electronics 34 (6), p. 1861–1872. External Links: Document Cited by: §3.1. [29] X. Wang, N. Zhao, Z. Han, D. Guo, and X. Yang (2025) AugRefer: advancing 3d visual grounding via cross-modal augmentation and spatial relation-based referring. CoRR abs/2501.09428. External Links: Link, Document, 2501.09428 Cited by: Table 1. [30] X. Wei, X. Liu, Y. Zang, X. Dong, P. Zhang, Y. Cao, J. Tong, H. Duan, Q. Guo, J. Wang, et al. (2025) VideoRoPE: what makes for good video rotary position embedding?. In International Conference on Machine Learning, Cited by: §2.3. [31] C. Xu, Y. Han, R. Xu, L. Hui, J. Xie, and J. Yang (2024) Multi attributes interactions matters for 3d visual grounding. In CVPR, Cited by: Table 1. [32] D. Yang, Z. Xu, W. Mo, Q. Chen, S. Huang, and Y. Liu (2024) 3D vision and language pretraining with large-scale synthetic data. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-24, Cited by: Table 1. [33] J. Yang, M. Ma, Y. Li, et al. (2025) VQALS: a video question answering method in low-light scenes based on illumination correction and feature enhancement. Chinese Journal of Electronics 34 (4), p. 1300–1308. External Links: Document Cited by: §2.1. [34] T. Zemskova and D. Yudin (2024) 3DGraphLLM: combining semantic graphs and large language models for 3d scene understanding. External Links: 2412.18450v2, Link Cited by: Table 6, §B.1, §1, §2.2, §3.1, Table 1, Table 1, Table 2, Table 2, Table 3, Table 4, §4.1. [35] Y. Zhang, Z. Gong, and A. X. Chang (2023-10) Multi3DRefer: grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 15225–15236. Cited by: §1, §2.1, Table 1, §4.1. [36] S. Zhou, Y. Liu, and F. Zheng (2025) Learn 3d vqa better with active selection and reannotation. In Proceedings of the 33rd ACM International Conference on Multimedia, M ’25, New York, NY, USA, p. 4610–4618. External Links: ISBN 9798400720352, Link, Document Cited by: §2.1. [37] Z. Zhu, X. Ma, Y. Chen, Z. Deng, S. Huang, and Q. Li (2023-10) 3D-vista: pre-trained transformer for 3d vision and text alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 2911–2921. Cited by: Table 1. [38] Z. Zhu, Z. Zhang, X. Ma, X. Niu, Y. Chen, B. Jia, Z. Deng, S. Huang, and Q. Li (2024) Unifying 3d vision-language understanding via promptable queries. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XLIV, Berlin, Heidelberg, p. 188–206. External Links: ISBN 978-3-031-72783-2, Link, Document Cited by: §1, Table 1. Appendix A Mathematical Derivation for QuatRoPE In this section, we provide a detailed mathematical derivation for QuatRoPE. Let m→ m and n→ n be the absolute 3D coordinates of the objects corresponding to query vector q→ q and key vector k→ k, and f(x→,p→)f( x, p) be the function for rotating the query or key vector x→ x with a corresponding 3D position p→ p. Since the attention score should only relate to the relative position (i.e., m→−n→ m- n), the rotation function f should satisfy: ⟨f(q→,m→),f(k→,n→)⟩=g(q→,k→,m→−n→) <f( q, m),f( k, n) >=g( q, k, m- n) (6) In QuatRoPE, we embed the coordinates as a holistic vector by applying quaternion rotations to query and key vectors. Formally, the rotation function can be expressed as: f(q→,m→)=Q(m→)q→Q−1(m→)Q(m→)=Qz(mz)Qy(my)Qx(mx)Qx(mx)=cos[θx(mx)/2]+i^sin[θx(mx)/2]Qy(my)=cos[θy(my)/2]+j^sin[θy(my)/2]Qz(mz)=cos[θz(mz)/2]+k^sin[θz(mz)/2] casesf( q, m)=Q( m)~ q~Q^-1( m)\\ Q( m)=Q_z(m_z)~Q_y(m_y)~Q_x(m_x)\\ Q_x(m_x)= [ _x(m_x)/2 ]+ i [ _x(m_x)/2 ]\\ Q_y(m_y)= [ _y(m_y)/2 ]+ j [ _y(m_y)/2 ]\\ Q_z(m_z)= [ _z(m_z)/2 ]+ k [ _z(m_z)/2 ] cases (7) where Q’s are rotation matrices and θ’s are unary functions. Through Equation (7), we transform the requirement in QuatRoPE (i.e., converting absolute coordinates to relative positions via dot products) into deriving θ’s that satisfy Equation (6). To solve the equation, we transform the dot product into the real part of the product of the rotation functions to yield a form with multiplication between the rotation matrices (i.e., Q−1(m→)Q^-1( m) and Q(n→)Q( n)). ⟨f(q→,m→),f(k→,n→)⟩ <f( q, m),f( k, n) > (8) = = ℜ[f(q→,m→)f∗(k→,n→)] [f( q, m)f^*( k, n)] = = ℜ[Q(m→)q→Q−1(m→)Q(n→)k→Q−1(n→)¯] [Q( m)~ q~Q^-1( m)~ Q( n)~ k~Q^-1( n)] = = ℜ[Q(m→)q→Q−1(m→)Q(n→)k→∗Q−1(n→)] [Q( m)~ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)] where k→∗ k^* denotes the conjugate of quaternion k→ k, and ℜ denotes the real part of the quaternion. To pair every Q(m→)Q( m) with Q(n→)Q( n), according to the real-part invariance of quaternion rotation, after left multiplying Q−1(m→)Q^-1( m) and right multiplying Q(m→)Q( m), Equation (8) yields: ⟨f(q→,m→),f(k→,n→)⟩ <f( q, m),f( k, n) > (9) = = ℜ[Q(m→)q→Q−1(m→)Q(n→)k→∗Q−1(n→)Q(m→)Q−1(m→)] [Q( m)~ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)~Q( m)~Q^-1( m)] = = ℜ[q→Q−1(m→)Q(n→)k→∗Q−1(n→)Q(m→)] [ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)~Q( m)] According to Equation (6), ⟨f(q→,m→),f(k→,n→)⟩ <f( q, m),f( k, n) > should only relate to m→−n→ m- n, the following equation should hold: ℜ[q→Q−1(m→)Q(n→)k→∗Q−1(n→)Q(m→)]=g(q→,k→,m→−n→) [ q~Q^-1( m)~Q( n)~ k^*~Q^-1( n)~Q( m)]=g( q, k, m- n) (10) Thus, Q(m→−n→)=Q−1(n→)Q(m→)Q( m- n)=Q^-1( n)~Q( m) (11) i.e., Qz(mz−nz)Qy(my−ny)Qx(mx−nx) Q_z(m_z-n_z)~Q_y(m_y-n_y)~Q_x(m_x-n_x) (12) = = Qx−1(nx)Qy−1(ny)Qz−1(nz)Qz(mz)Qy(my)Qx(mx) Q_x^-1(n_x)~Q_y^-1(n_y)~Q_z^-1(n_z)~Q_z(m_z)~Q_y(m_y)~Q_x(m_x) When m→=n→=0→ m= n= 0, we have Q(0→)Q(0→)=Q(0→)Q( 0)~Q( 0)=Q( 0). Thus, Q(0→)=1Q( 0)=1. According to Equation (7), 1= 1= Q(0→) Q( 0) (13) = = Qz(0)Qy(0)Qx(0) Q_z(0)~Q_y(0)~Q_x(0) = = [cos(θz(0)2)+k^sin(θz(0)2)] [ ( _z(0)2 )+ k ( _z(0)2 ) ] [cos(θy(0)2)+j^sin(θy(0)2)] [ ( _y(0)2 )+ j ( _y(0)2 ) ] [cos(θx(0)2)+i^sin(θx(0)2)] [ ( _x(0)2 )+ i ( _x(0)2 ) ] Consider the real part of the equation above, we have: 1= 1= ℜ[cos(θz(0)2)+k^sin(θz(0)2)] \ [ ( _z(0)2 )+ k ( _z(0)2 ) ] . (14) [cos(θy(0)2)+j^sin(θy(0)2)] [ ( _y(0)2 )+ j ( _y(0)2 ) ] [cos(θx(0)2)+i^sin(θx(0)2)] . [ ( _x(0)2 )+ i ( _x(0)2 ) ] \ = = cos(θz(0)2)cos(θy(0)2)cos(θx(0)2) ( _z(0)2 ) ( _y(0)2 ) ( _x(0)2 ) +k^j^i^sin(θz(0)2)sin(θy(0)2)sin(θx(0)2) + k j i ( _z(0)2 ) ( _y(0)2 ) ( _x(0)2 ) = = cos(θx(0)2)cos(θy(0)2)cos(θz(0)2) ( _x(0)2 ) ( _y(0)2 ) ( _z(0)2 ) +sin(θx(0)2)sin(θy(0)2)sin(θz(0)2) + ( _x(0)2 ) ( _y(0)2 ) ( _z(0)2 ) Also, since the imaginary part of Equation (13) is 0, either all cosines or all sines are equal to 0. Therefore cos(θx(0)2)=cos(θy(0)2)=cos(θz(0)2)=1 ( _x(0)2 )= ( _y(0)2 )= ( _z(0)2 )=1 (15) or sin(θx(0)2)=sin(θy(0)2)=sin(θz(0)2)=1 ( _x(0)2 )= ( _y(0)2 )= ( _z(0)2 )=1 (16) Consider the first solution, let mx=my=nx=ny=0m_x=m_y=n_x=n_y=0, Equation (12) yields: Table 6: Comparison between fixed and learnable base vectors for rotation. Model Base Vector ScanRefer SQA3D Multi3dRef Acc @ 0.25 Acc @ 0.5 EM @ 1 F1 @ 0.25 F1 @ 0.5 Chat-Scene [14] Fixed 55.44 55.00 53.14 58.09 57.72 Learnable 54.47 54.14 52.84 57.96 57.74 3DGraphLLM [34] Fixed 58.30 58.15 53.20 60.70 60.52 Learnable 56.89 56.64 52.68 60.67 60.51 Table 7: Comparison between different frequencies. Frequency ScanRefer SQA3D Multi3dRef Acc @ 0.25 Acc @ 0.5 EM @ 1 F1 @ 0.25 F1 @ 0.5 0.3 (Default) 58.30 58.15 60.70 60.52 53.20 0.1 (Small) 54.55 54.39 58.02 57.90 51.99 1.0 (Large) 53.41 53.14 56.28 55.99 52.18 Qz(mz−nz)Qy(0−0)Qx(0−0) Q_z(m_z-n_z)~Q_y(0-0)~Q_x(0-0) (17) = = Qx−1(0)Qy−1(0)Qz−1(nz)Qz(mz)Qy(0)Qx(0) Q_x^-1(0)~Q_y^-1(0)~Q_z^-1(n_z)~Q_z(m_z)~Q_y(0)~Q_x(0) i.e., Qz(mz−nz)=Qz−1(nz)Qz(mz)Q_z(m_z-n_z)=Q_z^-1(n_z)~Q_z(m_z) (18) For Equation (18), the left-hand side Qz(mz−nz) Q_z(m_z-n_z) (19) = = cos(θz(mz−nz)2)+sin(θz(mz−nz)2)k ( _z(m_z-n_z)2 )+ ( _z(m_z-n_z)2 ) k while the right-hand side Qz−1(nz)Qz(mz) Q_z^-1(n_z)~Q_z(m_z) (20) = = [cos(θz(nz)/2)−k^sin(θz(nz)/2)] [ ( _z(n_z)/2 )- k ( _z(n_z)/2 ) ] [cos(θz(mz)/2)+k^sin(θz(mz)/2)] [ ( _z(m_z)/2 )+ k ( _z(m_z)/2 ) ] = = [cos(θz(nz)/2)cos(θz(mz)/2) [ ( _z(n_z)/2 ) ( _z(m_z)/2 ) . +sin(θz(nz)/2)sin(θz(mz)/2)] .+ ( _z(n_z)/2 ) ( _z(m_z)/2 ) ] +[cos(θz(nz)/2)sin(θz(mz)/2) + [ ( _z(n_z)/2 ) ( _z(m_z)/2 ) . −sin(θz(nz)/2)cos(θz(mz)/2)]k - . ( _z(n_z)/2 ) ( _z(m_z)/2 ) ] k = = cos(θz(mz)−θz(nz)2)+sin(θz(mz)−θz(nz)2)k ( _z(m_z)- _z(n_z)2 )+ ( _z(m_z)- _z(n_z)2 ) k By Equation (19) and Equation (20), we have θz(mz)−θz(nz)=θz(mz−nz) _z(m_z)- _z(n_z)= _z(m_z-n_z) (21) When mz=nzm_z=n_z, Equation (21) yields: θz(0) _z(0) =θz(mz−nz) = _z(m_z-n_z) (22) =θz(mz)−θz(nz) = _z(m_z)- _z(n_z) =0 =0 Then, for any t∈ℤt , we have θz(t)= _z(t)= θz(t−1)+θz(1) _z(t-1)+ _z(1) (23) = = θz(t−2)+θz(1)+θz(1) _z(t-2)+ _z(1)+ _z(1) = = ⋯ ·s = = θz(0)+tθz(1) _z(0)+t _z(1) = = tθz(1) t _z(1) Moreover, for any t,p∈ℤt,p and (t,p)=1(t,p)=1 (i.e., tp∈ℚ tp ), we have θz(t)= _z(t)= θz(t(p−1)p)+θz(tp) _z ( t(p-1)p )+ _z ( tp ) (24) = = θz(t(p−2)p)+θz(tp)+θz(tp) _z ( t(p-2)p )+ _z ( tp )+ _z ( tp ) = = ⋯ ·s = = pθz(tp) p _z ( tp ) and hence θz(tp)=1pθz(t)=tpθz(1) _z ( tp )= 1p _z(t)= tp _z(1) (25) Also, since the embedding should be continuous with respect to the position, θz _z should be continuous, and the solution to θz _z is θz(z)=zθz(1),z∈ℝ _z(z)=z _z(1),z (26) Let nz=mz=0n_z=m_z=0, according to Equation (12), we have Qy(my−ny)Qx(mx−nx) Q_y(m_y-n_y)~Q_x(m_x-n_x) (27) = = Qx−1(nx)Qy−1(ny)Qy(my)Qx(mx) Q_x^-1(n_x)~Q_y^-1(n_y)~Q_y(m_y)~Q_x(m_x) Similarly, the above equation yields θy(y)=yθy(1),y∈ℝ _y(y)=y _y(1),y (28) Again, let ny=my=nz=mz=0n_y=m_y=n_z=m_z=0, Equation (27) yields Qx(mx−nx)=Qx−1(nx)Qx(mx)Q_x(m_x-n_x)=Q_x^-1(n_x)~Q_x(m_x) (29) and thus θx(x)=xθx(1),x∈ℝ _x(x)=x _x(1),x (30) In conclusion, an approximate solution for QuatRoPE is: f(q→,m→)=Q(m→)q→Q−1(m→)Q(m→)=Qz(mz)Qy(my)Qx(mx)Qx(mx)=cos[mxθx(1)2]+i^sin[mxθx(1)2]Qy(my)=cos[myθy(1)2]+j^sin[myθy(1)2]Qz(mz)=cos[mzθz(1)2]+k^sin[mzθz(1)2] casesf( q, m)=Q( m)~ q~Q^-1( m)\\ Q( m)=Q_z(m_z)~Q_y(m_y)~Q_x(m_x)\\ Q_x(m_x)= [ m_x _x(1)2 ]+ i [ m_x _x(1)2 ]\\ Q_y(m_y)= [ m_y _y(1)2 ]+ j [ m_y _y(1)2 ]\\ Q_z(m_z)= [ m_z _z(1)2 ]+ k [ m_z _z(1)2 ] cases (31) where θx(1) _x(1), θy(1) _y(1), and θz(1) _z(1) are frequencies for quaternion rotations. According to Equation (6), as we perform rotation by q→:=f(q→,m→) q:=f( q, m) and k→:=f(k→,n→) k:=f( k, n) before each attention layer, the attention scores between object-related tokens reflect their relative positions. By such an approach, QuatRoPE can effectively convey relative positional information for LLMs to perform spatial reasoning. Appendix B Experimental Settings Table 8: Qualitative Results Description Chat-Scene [14] QuatRoPE (Ours) This is a brown chair. It is turned toward the end of the table. Box-shaped footstool with a tarnished red color. There are 6 footstools stacked, 3 on the bottom row and 3 on the top. This is located on the bottom row in the middle. A blue towel that is hanging on the glass shower door. The towel is in the middle of the three towels hanging on the shower handle. This is a black office chair. It is facing the desk corner. B.1 Base Vector for Rotation In IGRE, the quaternion rotation of QuatRoPE is applied to the base vector to obtain the positional embedding. In this section, we compare the performance between using (1,0,0)(1,0,0) as a fixed base vector and the strategy of using a learnable base vector. Then we train and evaluate these approaches on Chat-Scene-1B [14] and 3DGraphLLM-1B [34], and the results are shown in Table 6. The results indicate that learnable base vectors do not achieve better results. Such outcomes may result from the difficulty of learning base vectors, as these vectors have a significant impact on subsequent layers. Therefore, in our model, we set the base vector as (1,0,0)(1,0,0), which is also more computationally efficient. B.2 Choice of Rotation Frequency In the experiments, rotation frequency is set to 0.3 (untuned, consistent across all datasets) to avoid two issues shown in Tab. 7: (a) Small frequencies lead to small rotation angles, weakening feature vector influence and hindering learning. (b) Large frequencies cause the “wrapping” problem—large coordinate differences may produce similar rotation angles, misleading the model with incorrect scene layouts. Given the maximum coordinate difference of 10, frequency is set to π10≈0.3 π10≈ 0.3, ensuring all rotations lie in the same semi-circle and larger coordinate differences correspond to larger angle differences. Additionally, the error introduced by the non‑commutativity of the Euler angle decomposition sequence is proportional to the square of the frequency. Thus, selecting a small frequency (e.g., 0.3) also makes QuatRoPE closer to the requirement of Equation (6). Appendix C Qualitative Results Table 9: Qualitative Results (Continued) Description Chat-Scene [14] QuatRoPE (Ours) It is a brown chair with armrests and four legs. It is directly under a blackboard. Case 1: This door appears to be the front door to the apartment. If you walk through the apartment and past the bathroom, you will encounter this door. The door is black and has a small window. Case 2: The door is rectangular in shape and has a small window on the upper portion. The door is located to the right of the bath area. Chat-Scene fails under both cases. The small rounded table. The table is next to the couch end. It is a tall gray trash can. The trash can is under the left side of the counter, to the left of the door when you enter. Table 10: Qualitative Results (Continued) Description Chat-Scene [14] QuatRoPE (Ours) Stand in front of the free-standing board in the room. Looking down the side of the table closest to you, it is the second chair down the row. Case 1: The monitor is next to the leftmost window. The monitor is black and rectangular. Case 2: The monitor is on the silver table. The monitor is the closest to the window. The bookshelf is between another bookshelf and a red wall. The bookshelf is brown and rectangular. The Ottoman is in the back, middle of the room. There is an identical ottoman to the right of it. In this section, we provide additional qualitative results to illustrate the effectiveness of QuatRoPE. The qualitative results are obtained from Chat-Scene-1B’s [14] predictions on the validation split of the ScanRefer dataset [5]. The cases in Tables 8 - 10 demonstrate that QuatRoPE can effectively provide precise relative positions between objects. By providing explicit spatial relations between objects, models can directly perceive the scene layout without extracting and calculating objects’ positions from prematurely fused features. Such a method significantly reduces the cost of training models to learn spatial reasoning, enabling them to achieve better performance.