Paper deep dive
Journey Operators for Structured Multi-Axis Composition
Mahesh Godavarti
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/4/2026, 11:08:55 AM
Summary
The paper introduces a framework for structured multi-axis composition using 'journey operators' derived from axis-step generators. It demonstrates that path-independent composition requires commuting generators, recovering Rotary Position Embedding (RoPE) and its variants. The authors propose JoFormer, a model integrating this composition into attention, and establish theoretical links to state-space models (SSMs) and algebraic structures like the DFT.
Entities (10)
Relation Signals (6)
Commuting Generators → enables → Path Independence
confidence 96% · composition and movement across axes are path-independent precisely when the axis transformations commute
Journey Operator → recovers → Rotary Position Embedding
confidence 95% · When the transformations are fixed, our framework recovers Rotary Position Embedding (RoPE)
JoFormer → uses → Journey Operator
confidence 95% · Finally, we use this theory to design JoFormer... relate it to attention
Axis-Step Generator → defines → Journey Operator
confidence 92% · the journey operator is the product of per-axis transformations along that path
JoFormer → relatesto → State Space Models
confidence 90% · relate it to attention and state-space models (SSMs)
Journey Operator → unifies → Standard attention
confidence 85% · This notation unifies standard attention... SSMs... and JoFormer
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Many kinds of data have structure along one or more axes: words in a sentence, pixels in an image, nodes in a tree, frames in audio, or cells in a 3D volume. Along one axis, order matters: "the dog bit the man" is different from "the man bit the dog." Across independent axes, however, neither composition nor movement should depend on the order of axes: in an image, composing right then down should give the same result as composing down then right, and moving right then down should describe the same relative position as moving down then right. We develop a framework for modeling this kind of multi-axis structure. Each data item carries its content together with a small transformation for each axis. A path connecting two positions defines a journey; the journey operator is the product of per-axis transformations along that path, governing both how data composes along the path and how relative position is described. When the transformations are fixed, our framework recovers Rotary Position Embedding (RoPE) and its multi-dimensional variants. When they depend on the data, the model gains a content-adaptive positional inductive bias. We show exactly when these paths are well-defined: both composition and movement across axes are path-independent precisely when the axis transformations commute. We also prove that, under the stated toral-frame symmetry, cocycle, bilinearity, and norm-preservation assumptions, the resulting pairwise scoring rule must take the form of block-wise rotations, explaining why RoPE-like methods arise naturally. Finally, we use this theory to design JoFormer, a model for value aggregation, and relate it to attention and state-space models (SSMs). Initial experiments across vision, language, and length generalization suggest that these inductive biases can have observable consequences in practice.
Tags
Links
- Source: https://arxiv.org/abs/2607.26775v1
- Canonical: https://arxiv.org/abs/2607.26775v1
Trouble viewing inline? Open PDF directly →
Full Text
96,898 characters extracted from source content.
Expand or collapse full text
Journey Operators for Structured Multi-Axis Composition Mahesh Godavarti A Carrot, Inc Abstract Many kinds of data have structure along one or more axes: words in a sentence, pixels in an image, nodes in a tree, frames in audio, or cells in a 3D volume. Along one axis, order matters: “the dog bit the man” is different from “the man bit the dog.” Across independent axes, however, neither composition nor movement should depend on the order of axes: in an image, composing right then down should give the same result as composing down then right, and moving right then down should describe the same relative position as moving down then right. We develop a framework for modeling this kind of multi-axis structure. Each data item carries its content together with a small transformation for each axis. A path connecting two positions defines a journey; the journey operator is the product of per-axis transformations along that path, governing both how data composes along the path and how relative position is described. When the transformations are fixed, our framework recovers Rotary Position Embedding (RoPE) and its multi-dimensional variants. When they depend on the data, the model gains a content-adaptive positional inductive bias. We show exactly when these paths are well-defined: both composition and movement across axes are path-independent precisely when the axis transformations commute. We also prove that, under the stated toral-frame symmetry, cocycle, bilinearity, and norm-preservation assumptions, the resulting pairwise scoring rule must take the form of block-wise rotations, explaining why RoPE-like methods arise naturally. Finally, we use this theory to design JoFormer, a model for value aggregation, and relate it to attention and state-space models (SSMs). Initial experiments across vision, language, and length generalization suggest that these inductive biases can have observable consequences in practice. 1 Introduction Any data that can be indexed—sequences, images, audio, volumetric arrays, trees—composes along multiple axes. In the one-dimensional (1D) case, a sequence like [abcd][a\;b\;c\;d] can be formed by composing sub-sequences—for example, by concatenating [ab][a\;b] with [cd][c\;d], or [a][a] with [bcd][b\;c\;d]. Algebraic tools such as non-commutative semigroups or free groups have long provided principled ways to model such 1D compositional structure (Rudolph and Giesbrecht, 2010). In contrast, there exists no broadly accepted algebraic framework for modeling two-dimensional (2D) composition. Consider the array [abcd] [ smallmatrixa&b\\ c&d smallmatrix ]: it can be composed vertically, by stacking [ab][a\;b] over [cd][c\;d], or horizontally, by placing [ac] [ smallmatrixa\\ c smallmatrix ] next to [bd] [ smallmatrixb\\ d smallmatrix ]. Such multiple valid composition paths do not fit neatly into existing algebraic systems, which are typically designed for linear (1D) structured data. Even in the 1D case, aligning algebraic formalisms with the needs of modern machine learning architectures can be problematic—for example, representing tokens as matrices instead of vectors breaks the core assumptions of the attention mechanism in transformers, which relies fundamentally on vector operations. This tension reveals a deeper issue: the absence of an algebraic framework that can both align with vector-based learning architectures and naturally support composition along multiple axes. We present such an algebraic framework, built on one operation that models composition in 1D and extends naturally to multiple axes. In 1D, each data point carries both content and a single orthogonal transformation—its axis-step generator. Composing two data points applies the first point’s axis-step generator to rotate the second’s content, and multiplies their axis-step generators together. This is associative but non-commutative: the first element’s axis-step generator acts on the second’s content, not vice versa, so order matters. Composing an entire sequence accumulates these transformations: each value is transformed by the product of all preceding axis-step generators (Section˜2.1). In nD, each datum carries one axis-step generator per axis. Composing two data points along a given axis applies that axis’s axis-step generator to rotate the second point’s content into the first’s frame, accumulates the axis-step generator for that axis, and leaves all other axes unchanged. For multi-axis data like images, we want two kinds of path independence: composition itself should not depend on the order of axes (composing right-then-down equals down-then-right), and any movement should depend only on the initial and final positions, not on the path taken. This is the inductive bias—it encodes the assumption that independent axes of real-world data commute. Our framework captures both: when per-axis axis-step generators commute (the flat regime), composition is path-independent and the journey—the relative transformation between two positions—depends only on displacement (Theorem˜2.2). In 1D with a single axis-step generator, this recovers RoPE; content-dependent axis-step generators yield data-dependent composition (JoFormer-projected). This compositional structure is absent from standard attention, which aggregates values without non-commutative composition. Bringing composition into attention—applying the journey operator to values before aggregation—is the architecture (JoFormer) developed in Section˜3. Initial experiments across vision, language, and length generalization are consistent with the framework’s value (Section˜4). With fixed rotations, the framework is a generalized DFT with learned frequencies. RoPE (Su et al., 2021), 2D-RoPE (Heo et al., 2024), tree PEs (Shiv and Quirk, 2019), and RotatE (Sun et al., 2019) emerge as special cases; the value-path formula provides common notation for standard attention, SSMs, and JoFormer (Section˜F.2). Contributions. 1. We introduce a compositional framework (Section˜2.1): in 1D, each data point carries content and a single axis-step generator (an orthogonal transformation); in nD, each data point carries one axis-step generator per axis. Composition along any axis is a single associative operation. 2. From this framework we derive several theoretical results: (a) path-independent composition requires commuting axis-step generators (Theorem˜2.2); (b) under natural axioms (bilinearity, toral frame invariance, cocycle, norm preservation), the only compatible mechanism is block-diagonal SO(2)d/2SO(2)^d/2 rotation (Theorem˜2.4); if the full orthogonal group O(d)O(d) is the symmetry instead, positional structure collapses (Theorem˜2.5); (c) in the flat (commutative) regime, value transport via the journey operator is translation equivariant (Theorem˜2.8); (d) the DFT is a special case of the compositional embedding, and learned frequencies yield a generalized Fourier transform (Proposition˜C.1). 3. These constraints lead to JoFormer (Section˜3), an architecture that brings composition into attention via data-dependent projected angles and value-side rotation. 4. A value-path hierarchy (Section˜F.2) places standard attention, SSMs, and JoFormer on a common spectrum—the SSM recurrence unrolls into the same value-path formula (Proposition˜F.1)—clarifying what each family composes and what it discards. 5. Experiments on vision, language modeling, and length generalization suggest the framework has practical value (Section˜4). 2 Algebraic Framework and Theoretical Analysis 2.1 Compositional Data Points Everything in this paper flows from one operation. In 1D, each data point is a pair (,R)(v,R): content ∈ℝdv ^d plus an orthogonal transformation R∈G≤O(d)R∈ G (d) (the group of norm-preserving d×d× d matrices)—its axis-step generator. Given two such points (,A)(a,A) and (,B)(b,B)—where ,∈ℝda,b ^d are contents and A,B∈GA,B∈ G are axis-step generators—their composition is: (,A)∘(,B):=(+A,AB).(a,A) (b,B)\;:=\;(a+Ab,\;AB). (1) The content of the composite is a plus b rotated into a’s frame by A; the axis-step generator of the composite is the product ABAB. This operation is associative with identity (,I)(0,I), but non-commutative: the first element’s axis-step generator acts on the second’s content, so order matters. Axis-specific composition. For data indexed by D axes, each datum carries content and one axis-step generator per axis: (,R1,…,RD)(v,R_1,…,R_D). Given two data points with contents ,∈ℝda,b ^d at axis-k positions nkn_k and mkm_k respectively, composing along axis k applies the first point’s axis-k generator to rotate the second’s content and accumulates that axis-step generator, leaving other axes unchanged: (,R1n1,…,Rknk,…,RDnD)∘k(,R1n1,…,Rkmk,…,RDnD) (a,R_1^n_1,…,R_k^n_k,…,R_D^n_D) _k(b,R_1^n_1,…,R_k^m_k,…,R_D^n_D) =(+Rknk,R1n1,…,Rknk+mk,…,RDnD). =(a+R_k^n_kb,\;R_1^n_1,…,R_k^n_k+m_k,…,R_D^n_D). Crucially, ∘k _k is defined only when the two operands agree on all axes i≠ki≠ k—they must share the same exponents (n1,…,nk−1,nk+1,…,nD)(n_1,…,n_k-1,n_k+1,…,n_D). Composition therefore operates over tiles: 1D slices of the grid at fixed coordinates on the remaining D−1D-1 axes. One cannot, for instance, compose elements from different rows of a 2D grid along the column axis. When RkRl=RlRkR_kR_l=R_lR_k for all axis pairs, the regime is flat: composition is path-independent (composing along axis k then l gives the same result as l then k), and the relative transformation between two positions depends only on their displacement, not on which path connects them (Theorem˜2.2). Both properties follow from the single condition of commuting axis-step generators. When axis-step generators do not commute, the regime is curved: composition depends on the order of axes, and relative transformations depend on absolute position. Sequence unrolling. In 1D, composing a sequence [e1,…,eT][e_1,…,e_T] with et=(t,Rt)e_t=(v_t,R_t) gives: e1∘e2∘⋯∘eT=(∑i=1T(∏j=1i−1Rj)i,∏j=1TRj).e_1 e_2 ·s e_T\;=\; (\, _i=1^T ( _j=1^i-1R_j )v_i,\;\; _j=1^TR_j ). (2) The content part ∑i(∏j<iRj)i _i( _j<iR_j)v_i is a weighted sum where each value is rotated by all preceding transformations—position is encoded through accumulated rotation. The journey operator. The journey operator from position j to position k is defined as the composition of axis-step generators along the path connecting them. In 1D, traveling from j to k (with j>kj>k) means composing the generators at positions k,k+1,…,j−1k,k+1,…,j-1 in sequence. Absolute operators. Separately, define the absolute operator at position i as Ai=∏j=1i−1RjA_i= _j=1^i-1R_j—so A1=IA_1=I, A2=R1A_2=R_1, A3=R1R2A_3=R_1R_2, and so on. The unrolled content (2) can then be rewritten as ∑iAii _iA_iv_i: each value iv_i is expressed in the global frame (position 1’s frame) via AiA_i. Journey in terms of absolute operators. We can show that the path-defined journey equals Ak−1AjA_k^-1A_j. To see this: the absolute operator AjA_j maps jv_j into the global frame, and Ak−1A_k^-1 maps from the global frame into position k’s frame. Their composition Pj→k=Ak−1AjP_j→ k=A_k^-1A_j therefore transforms jv_j into k’s frame—exactly what the path-defined journey does. When all axis-step generators are identical (Rt=R_t=R), the absolute operator is Ai=Ri−1A_i=R^i-1 and the journey reduces to Pj→k=R−(k−1)Rj−1=Rj−kP_j→ k=R^-(k-1)R^j-1=R^j-k, a pure function of displacement. In nD, the absolute operator at position n is A=R1n1⋯RDnDA_n=R_1^n_1·s R_D^n_D; the formal multi-axis treatment is in Section˜2.2. Value-path formula (1D). The unrolled content gives a sum of rotated values, each weighted equally. In an attention mechanism, different positions contribute differently: position j’s value is weighted by an attention weight αkj _kj (how much position k attends to position j). Combining the journey operator with selective attention yields the value-path formula: k=∑jαkjPj→kj,c_k= _j _kj\,P_j→ k\,v_j, (3) where Pj→k=Ak−1AjP_j→ k=A_k^-1A_j is the journey from j to k. This notation unifies standard attention (P=IP=I, no compositional structure), SSMs (P=∏AxtP=Π A_x_t, content-dependent recurrence with uniform weights), and JoFormer (P=Ak−1AjP=A_k^-1A_j with full attention weights). 2D composition and value-path formula. For a 2D grid of data points e(i,j)=((i,j),Rx,Ry)e_(i,j)=(v_(i,j),R_x,R_y), we compose along rows first (1D composition along x): each row i gives ∑jRxj(i,j) _jR_x^\,j\,v_(i,j) by the 1D unrolling (2). Composing the row results along columns (1D composition along y) applies RyiR_y^\,i to each row’s content: ∑iRyi∑jRxj(i,j)=∑i,jRyiRxj(i,j)=∑i,jA(i,j)(i,j), _iR_y^\,i _jR_x^\,j\,v_(i,j)\;=\; _i,jR_y^\,i\,R_x^\,j\,v_(i,j)\;=\; _i,jA_(i,j)\,v_(i,j), where A(i,j)=RyiRxjA_(i,j)=R_y^\,i\,R_x^\,j is the absolute operator at position (i,j)(i,j). When the axis-step generators commute, composing columns-then-rows gives the same result (since RyiRxj=RxjRyiR_y^\,iR_x^\,j=R_x^\,jR_y^\,i). This generalizes directly to D axes: composing along each axis in turn yields ∑A _nA_n\,v_n with A=∏k=1DRknkA_n= _k=1^DR_k^\,n_k, independent of the order of axes when generators commute. The journey from (i′,j′)(i ,j ) to (i,j)(i,j) is: P(i′,j′)→(i,j)=A(i,j)−1A(i′,j′)=Rx−jRy−iRyi′Rxj′.P_(i ,j )→(i,j)=A_(i,j)^-1\,A_(i ,j )=R_x^-j\,R_y^-i\,R_y^\,i \,R_x^\,j . When the axis-step generators commute (RxRy=RyRxR_xR_y=R_yR_x), this simplifies to Ryi′−iRxj′−jR_y^\,i -i\,R_x^\,j -j—a pure function of displacement. The value-path formula becomes: (i,j)=∑(i′,j′)α(i,j),(i′,j′)Ryi′−iRxj′−j(i′,j′).c_(i,j)= _(i ,j ) _(i,j),(i ,j )\;R_y^\,i -i\,R_x^\,j -j\;v_(i ,j ). (4) This generalizes directly to D axes: the absolute operator at position =(n1,…,nD)n=(n_1,…,n_D) is A=∏k=1DRknkA_n= _k=1^DR_k^\,n_k, and the journey (with commuting axis-step generators) is the multi-axis displacement operator P′→=∏k=1DRknk′−nkP_n = _k=1^DR_k^\,n _k-n_k. The hierarchy is developed in Section˜F.2. 2.2 Path Transport and Commutativity We now formalize how composing axis-step generators along different paths may or may not yield the same result. Definition 2.1 (Multi-axis composition and path transport). For D axes with axis-step generators R1,…,RD∈G≤O(d)R_1,…,R_D∈ G (d), a position =(n1,…,nD)∈ℤDn=(n_1,…,n_D) ^D has absolute operator A=R1n1⋯RDnDA_n=R_1^n_1·s R_D^n_D (as in Section˜2.1). For a word π=k1⋯kmπ=k_1·s k_m over axes 1,…,D\1,…,D\ (a path), define the path transport: T(π)=Rkm⋯Rk1.T(π)=R_k_m·s R_k_1. (5) The path transport T(π)T(π) composes generators step by step; different orderings of the same axis steps may yield different composites—the transport is path-dependent in general. Under the canonical axis ordering, the composite transport from m to n evaluates to A−1A_n^-1A_m, but this equals the transport along every other path only when generators commute. This motivates the central theorem. Theorem 2.2 (Path independence ⇔ commuting generators). If you compose along a multi-axis grid using per-axis axis-step generators, the result is independent of the order you take the steps (path-independent) if and only if the axis-step generators commute. When they commute, the journey operator P→=A−1AP_m =A_n^-1A_m depends only on displacement—it is fully determined by composition. Formal statement and proof in Section˜A.1. Remark 2.3 (Connection to interchange laws). In higher category theory, the interchange law (x∘iy)∘j(z∘iw)=(x∘jz)∘i(y∘jw)(x _iy) _j(z _iw)=(x _jz) _i(y _jw) states that two composition operations are compatible. Path independence is the geometric manifestation of this algebraic condition: the interchange law holds for group-valued elements iff generators commute iff transport is path-independent. 2.3 Journey Operator and Score/Value Attention As derived in Section˜2.1, composing axis-step generators along each axis yields absolute operators Ai,Aj∈G≤O(d)A_i,A_j∈ G (d) at each position. In the flat (commutative) regime established by Theorem˜2.2, the journey operator arising from composition reduces to: Pj→i=Ai−1Aj.P_j→ i=A_i^-1A_j. (6) This operator structures both the score and value sides of attention: Score: Att(i,j)=qi⊤Pj→ikj=(Aiqi)⊤(Ajkj), (i,j)=q_i P_j→ i\,k_j=(A_iq_i) (A_jk_j), (7) Value: i=Ai−1∑jαijAjvj=∑jαijPj→ivj. _i=A_i^-1 _j _ij\,A_j\,v_j= _j _ij\,P_j→ i\,v_j. (8) For orthogonal AiA_i: (Aiqi)⊤(Ajkj)=qi⊤Ai−1Ajkj=qi⊤Pj→ikj(A_iq_i) (A_jk_j)=q_i A_i^-1A_jk_j=q_i P_j→ ik_j, confirming consistency. Score computation rotates q and k by their respective absolute operators; value computation rotates each vjv_j by AjA_j then inverse-rotates the output by Ai−1A_i^-1. RoPE as a special case. For a sequence, At=RtA_t=R^t with R=diag(R(θ1),…,R(θd/2))R=diag(R( _1),…,R( _d/2)) being a block-diagonal rotation. Then Pj→i=R−iRj=Rj−iP_j→ i=R^-iR^j=R^j-i, and the score becomes (Riqi)⊤(Rjkj)=qi⊤Rj−ikj(R^iq_i) (R^jk_j)=q_i R^j-ik_j—exactly RoPE (Su et al., 2021). Standard RoPE uses only the score side; our framework additionally rotates values. 2D-RoPE as a special case. For a 2D grid with generators Rx,RyR_x,R_y acting on disjoint subspaces (first d/4d/4 planes for x, remaining for y): A(m,n)=RxmRyn=diag(R(ω1m),…,R(ωd/4m),R(ϕ1n),…,R(ϕd/4n))A_(m,n)=R_x^mR_y^n=diag(R( _1m),…,R( _d/4m),R( _1n),…,R( _d/4n)). Since the generators act on disjoint planes, RxRy=RyRxR_xR_y=R_yR_x—path independence is satisfied by construction. 2.4 Toral Classification We classify all bilinear, norm-preserving, cocycle-compatible attention mechanisms under toral frame symmetry T=SO(2)d/2T=SO(2)^d/2. The result: the score matrix must be a block-diagonal SO(2)d/2SO(2)^d/2 rotation—each of d/2d/2 planes gets an independent angle θk(s)−θk(s′) _k(s)- _k(s ). This recovers RoPE (Su et al., 2021), 2D-RoPE (Heo et al., 2024), and tree PEs (Shiv and Quirk, 2019) as special cases. If the full orthogonal group O(d)O(d) is the symmetry instead, positional structure collapses entirely. Full axioms (bilinearity, toral frame invariance, cocycle, norm preservation), formal statements, and proofs are in Section˜A.2. Theorem 2.4 (Toral Classification). Under bilinearity, toral frame invariance, cocycle compositionality, and norm preservation, the score matrix must be a block-diagonal rotation: each of d/2d/2 planes gets an independent angle θk(s)−θk(s′) _k(s)- _k(s ), and the full operator is their direct sum. This is the unique form compatible with the symmetry contract. Formal statement and proof in Section˜A.3. Theorem 2.5 (O(d)O(d) Impossibility). If the full orthogonal group O(d)O(d) is the frame symmetry (instead of the torus), every score matrix reduces to a scalar multiple of the identity—positional structure is completely lost. Proof in Section˜A.4. 2.5 Flat/Curved Dichotomy and the V Rotation Prediction Definition 2.6 (Flat and curved). A multi-axis compositional framework is flat if its per-axis axis-step generators commute ([Ri,Rj]=0[R_i,R_j]=0 for all i≠ji≠ j), and curved otherwise. All block-diagonal SO(2)d/2SO(2)^d/2 methods are flat; dense rotation matrices (like LieRE) are generically curved. In the flat regime, Pj→ivj=R(Δp)vjP_j→ iv_j=R( p)v_j depends only on relative displacement—the journey operator is fully determined by composition. Definition 2.7 (Translation equivariance). For a shift ∈ℤDu ^D, define the translation operator (Tv)=v−(T_uv)_m=v_m-u. An aggregation rule F mapping value fields v to output fields c is translation equivariant if F(Tv)+=F(v)F(T_uv)_n+u=F(v)_n for all ,n,u: shifting inputs shifts outputs by the same amount. Theorem 2.8 (Relative-displacement equivariance of value transport). When per-axis generators commute (flat regime) and attention weights depend only on relative displacement, applying the journey operator A−1A_n^-1A_m to values before aggregation gives a translation-equivariant rule: shifting all positions by the same amount shifts outputs identically. When generators do not commute, the value transport becomes path-dependent—it depends on absolute positions, not just displacement. Formal statement and proof in Section˜A.5. Remark 2.9 (Scope: operator-level, not whole-layer equivariance). In a standard transformer, attention weights αij=softmaxj(qi⊤Pj→ikj/d) _ij=softmax_j(q_i P_j→ ik_j/ d) depend on token content through qi=WQxiq_i=W_Qx_i and kj=WKxjk_j=W_Kx_j, so they are not purely displacement-dependent. Theorem˜2.8 therefore does not imply that the full attention layer is translation equivariant. What the theorem does justify is the value-side operator choice: once absolute operators are determined by composition (Section˜2.1), Ai−1AjA_i^-1A_j is the canonical cocycle-compatible journey that yields displacement-dependent value transport when weights happen to be displacement-only. In the general content-dependent case, no full equivariance guarantee remains, but the value transform is still norm-preserving and uses the same toral operator family. We test the flat-regime hypothesis experimentally in Section˜4.1. 2.6 Value-Path Hierarchy and Relation to SSMs The value-path formula (3), introduced in Section˜2.1 as a direct consequence of the composition operation, provides common notation for standard attention, SSMs, and JoFormer. This is a notation-level bridge rather than an equivalence: Mamba’s transition matrices are not generally toral, attention weights are normalized, and the forward SSM kernel differs from the attention journey by convention. The hierarchy, SSM bridge proposition, and detailed analysis are in Section˜F.2. 3 JoFormer Architecture The algebraic framework motivates a concrete architecture: the JoFormer (Journey-based Transformer), which implements data-dependent journey operators on both the score and value sides. 3.1 Design Principles Three inductive biases follow from the theory: 1. Commutativity: use block-diagonal SO(2)d/2SO(2)^d/2 rotations (flat, satisfying path independence). Under the full modeling contract (bilinearity, toral-frame invariance, cocycle, norm preservation), this is the only compatible structure (Theorem˜2.4). The choice of T=SO(2)d/2T=SO(2)^d/2 as the symmetry group is an assumption of that contract, not a consequence. 2. V rotation: apply Pj→iP_j→ i to values (justified by the flat/curved analysis—the journey operator arising from composition gives displacement-dependent value transport when axis-step generators commute). 3. Data dependence: compute angles θ(x)θ(x) from content, enriching the value-path beyond fixed positional functions—an SSM-like data-dependent value transform, though without the recurrent product over intermediate states. What the theory does and does not justify. The core claim is that standard attention lacks a structural inductive bias on the value side: commutative summation discards compositional context. Theorems˜2.2 and 2.8 prove that the journey operator—arising directly from composition—fills this gap, giving displacement-dependent value transport exactly in the flat/commutative regime with relative attention weights. JoFormer-fixed and JoFormer-learned preserve this contract. JoFormer-projected keeps the same norm-preserving toral operator family but makes angles content-dependent; it is therefore a theory-motivated architecture, not a direct translation-equivariance corollary. A detailed breakdown of which theorem supports which variant is in Table˜8 (Section˜F.1). 3.2 Architecture Variants All variants share the same attention computation: Score: αij∝exp((R(θi)qi)⊤(R(θj)kj)/d), _ij ((R( _i)q_i) (R( _j)k_j)/ d ), (9) Value: ci=R(θi)−1∑jαijR(θj)vj. c_i=R( _i)^-1 _j _ij\,R( _j)\,v_j. (10) The variants differ in how angles θi _i are computed: JoFormer-fixed. Angles are linear in position: θk(l)(t)=ωk⋅t _k^(l)(t)= _k· t (same as RoPE frequencies). This is equivalent to RoPE on Q/K plus V rotation and inverse rotation on output. Implements the journey value path Pj→i=R−iRj=Rj−iP_j→ i=R^-iR^j=R^j-i. JoFormer-learned. Per-layer learned frequency vectors ω(l)∈ℝd/2ω^(l) ^d/2 define θ(l)(t)=t⋅ω(l)θ^(l)(t)=t·ω^(l). This is equivalent to RoPE with per-layer learned frequencies instead of fixed geometric spacing. Enables layer-specific frequency selection while maintaining the linear-in-position structure. JoFormer-projected. Per-layer MLP angle projectors compute angles from the residual stream: θ(l)(x)=W2(l)GELU(W1(l)LN(x)),θ^(l)(x)=W_2^(l)\,GELU(W_1^(l)\,LN(x)), (11) with W1(l)∈ℝd×dW_1^(l) ^d× d, W2(l)∈ℝd/2×dW_2^(l) ^d/2× d. Angles are content-dependent: computed fresh at each layer from the current representation. This provides an SSM-like data-dependent value transform: Pj→iP_j→ i depends on the current residual representations at positions i and j, but does not implement the recurrent product over intermediate states. The MLP adds ∼1.5d2 1.5d^2 parameters per layer (small relative to the 12d212d^2 for attention + FFN). In practice, JoFormer-projected requires softmax attention (softplus is unstable with data-dependent angles, as unbounded weights accumulate across layers) and a lower learning rate (≤2×10−4≤ 2×10^-4). Attention ordering for vision. JoFormer uses K⊤QK Q (not Q⊤KQ K) attention ordering in vision applications. Since qi⊤Pj→ikj=(Aiqi)⊤(Ajkj)=q^i⊤k^jq_i P_j→ ik_j=(A_iq_i) (A_jk_j)= q_i k_j, both orderings yield equivalent scores; we adopt K⊤QK Q as a convention matching the reference-factored structure k^j⊤q^i k_j q_i. 4 Experiments Each experiment is a single-seed sanity check for a different theoretical hypothesis; none are benchmark claims. MNIST (Section˜E.1): monoidal compression. CIFAR-100/ImageNet: V rotation. Wikipedia LM: value-path hierarchy. Length generalization: projected angles. 4.1 CIFAR-100 V Rotation and Scaling Setup. Two experimental setups test V rotation for vision. (A) ViT-Tiny (D=384D=384, 12 layers, 6 heads, ∼14.9 14.9M params, patch 4×44×4, CIFAR-100, 200 epochs, Adam lr=10−4=10^-4, cosine annealing, H100, seed=42) using LieRE’s framework (Ostmeier et al., 2024). (B) Smaller ViT (4 layers, 4 heads, patch 4×44×4, CIFAR-100, 300 epochs, cosine lr=10−3=10^-3, dropout=0.1, weight decay, mixup, cutout) at D∈32,64,128,256D∈\32,64,128,256\ with fully deterministic GPU-resident training. Both setups use exact reproducibility (fixed seeds, no DataLoader workers, torch-op augmentation on GPU). Methods. Nine PE variants organized on two axes: frequency type (fixed RoPE vs. learned) and V rotation (Q/K only vs. Q/K/V with inverse rotation on output). The axial factorization splits dimensions into disjoint y- and x-subspaces, guaranteeing commutativity. Results: V rotation is consistently beneficial in these paired single-seed CIFAR protocols. Table 1: V rotation effect on CIFAR-100 (paired comparisons, Setup B, D=256, deterministic single run). V rotation helps in all five paired comparisons; the synergy with learnable frequencies is largest. Approach Q/K only Q/K/V Δ Axial learned monoidal_axial: 60.92% joformer_axial: 63.27% +2.35% Axial learned (per-layer) 60.63% 62.76% +2.13% Axial fixed rope2d: 61.39% joformer_old: 61.85% +0.46% Combined fixed rope2dv2: 55.53% joformer_fixed: 56.66% +1.13% Combined learned (per-layer) 60.85% 61.17% +0.32% V rotation helps consistently across D=32–256 (Table˜6 in Section˜E.3), with V rotation and learnable frequencies synergistic (+1.88%+1.88\% at D=256). In the ViT-Tiny LieRE framework, axial-dense V rotation gains +0.51%+0.51\%, while LieRE64’s dense (curved) rotation drops 1.23%1.23\% (Appendix˜E)—consistent with the flat-regime hypothesis. 4.2 ImageNet ViT-S Scale Validation ViT-S (D=384D=384, 12 layers, 6 heads, ∼22 22M params, DeiT-I recipe (Touvron et al., 2022), 300 epochs, single seed; details in Appendix˜E). Table 2: ImageNet-1K ViT-S (DeiT-I, 300 epochs, single seed). JoFormer adds V rotation + inverse rotation to RoPE2D. Method Top-1 Acc Top-5 Acc RoPE2D (axial, Q/K only) 80.71% 95.26% JoFormer (axial, Q/K/V) 81.11% 95.53% Δ +0.40%+0.40\% +0.27%+0.27\% The +0.40%+0.40\% gap is consistent from epoch 180 onward. Single-seed: a scale sanity check, not a conclusive ImageNet improvement. 4.3 JoFormer Wikipedia Language Modeling Full English Wikipedia (∼983 983M byte-pair encoding (BPE) tokens, vocab=8K, block_size=512, 200K iters; details in Appendix˜E). Table 3: Wikipedia LM validation perplexity (PPL; 200K iters, vocab=8K, full wiki). JoFormer-projected leads in all configurations tested (single seed). Lower = better. Config RoFormer JoF-fixed JoF-learned JoF-projected n100, L2 7.24 6.67 6.43 6.15 n200, L2 5.82 5.51 5.42 5.17 n200, L4 5.36 5.07 5.01 4.72 n250, L4 5.10 4.85 4.74 4.55 n500, L2 5.02 4.82 4.73 4.58 n500, L4 4.67 4.42 4.37 4.32 JoFormer-projected leads in all configurations, consistent with the value-path hierarchy (which does not prove an optimization ordering). 4.4 Length Generalization 163M params (D=768, 16 layers, 8 heads), OpenWebText, trained on block_size=512. Architecture: 5 windowed layers (window=32) + 1 full-attention NoPE layer. JoFormer-projected uses 200K total iterations (150K fixed-angle ++ 50K projected-angle fine-tuning); the RoPE baseline trains for 150K iterations. The additional 50K iterations are needed to convert fixed angles to content-dependent projected angles (details in Appendix˜E). Table 4: Length generalization from 512 training length (163M params, OWT, single seed). Ratio = PPL@4096 / PPL@512; lower ratio = better extrapolation. JoFormer-projected uses 200K total iterations (150K fixed ++ 50K projected fine-tuning); RoPE trains for 150K. Model PPL@512 PPL@1024 PPL@2048 PPL@4096 Ratio JoFormer-projected 25.55 25.17 24.73 26.01 1.02× JoFormer fixed 26.32 34.32 64.74 115.96 4.41× RoPE 26.82 41.90 90.10 168.67 6.29× JoFormer-projected extrapolates well (1.02×1.02× ratio vs. 6.29×6.29× for RoPE); JoFormer-fixed is intermediate (4.41×4.41×), suggesting data-dependent angles—not just V rotation—are consistent with improved length generalization. The staged recipe worked more reliably than training projected angles from scratch in our setup. 5 Related Work RoPE (Su et al., 2021), 2D-RoPE (Heo et al., 2024), and ComRoPE (Yu et al., 2025) operate on Q/K only; our compositional framework extends to the value side via the journey operator, with commutativity as the iff condition for displacement-dependent value transport (Theorem˜2.8). Shaw et al. (Shaw et al., 2018) inject relative information into values via learned additive embeddings; the proposed value operation instead uses the same orthogonal cocycle transport as scores, inheriting norm preservation and the target-frame interpretation. LieRE (Ostmeier et al., 2024) uses dense (curved) rotations—expressive, but composition is path-dependent. S4 (Gu et al., 2022) and Mamba (Gu and Dao, 2023) implement non-trivial value-path kernels; our hierarchy (Section˜F.2) provides common notation. ALiBi (Press et al., 2022) uses additive biases; toral scores span a d-dimensional trigonometric polynomial space (Section˜A.7)—a function-class separation, not a dominance claim. 6 Discussion and Conclusion We started from one operation: composing data points that carry both content and axis-step generators. This operation defines journey operators along paths, and we showed that journey operators can be expressed in terms of absolute operators as Ak−1AjA_k^-1A_j. From this foundation, a chain of results follows. Path-independent composition requires commuting axis-step generators (Theorem˜2.2). Under natural axioms, the only compatible mechanism is block-diagonal SO(2)d/2SO(2)^d/2 rotation (Theorem˜2.4), explaining why RoPE-like methods arise naturally. In the flat regime, value transport via the journey operator is displacement-dependent (Theorem˜2.8), giving a concrete design rule: apply the journey to values, not just scores. Standard attention discards this compositional structure entirely. JoFormer brings it back by applying the journey operator to values before aggregation. The value-path formula (3) places standard attention, SSMs, and JoFormer on a common spectrum, clarifying what each composes and what it discards (Section˜F.2). For practitioners, V rotation is a zero-parameter change to RoPE: apply the same rotation to values and inverse-rotate the output. Our experiments across vision (∼2% 2\% on CIFAR-100, +0.40%+0.40\% on ImageNet), language modeling, and length generalization suggest the framework has practical value. All comparisons are single-seed sanity checks, not benchmark claims. Open directions include extending the toral classification to non-abelian frame groups, making the value-path hierarchy quantitative, and finding simpler length-generalization recipes than the staged JoFormer-projected training. Limitations. The toral classification assumes multiplicity-free representations; extending it to non-abelian frame groups remains open. The theory motivates projected angles and value-side rotation but does not prove that they help—they are theory-motivated, not theory-proved (Table˜8). Full-layer equivariance requires additional assumptions beyond operator-level analysis (Remark˜2.9). All experimental comparisons are single-seed sanity checks at small-to-moderate scale; we do not claim state-of-the-art results, and the approach has not been validated on large-scale pretraining. Broader Impact. This work is primarily theoretical—it provides an algebraic framework for understanding and designing positional mechanisms in transformers. We do not foresee specific negative societal consequences beyond those common to general-purpose sequence and vision architectures. References Gu and Dao [2023] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. Gu et al. [2022] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022. Heo et al. [2024] Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun. Rotary position embedding for vision transformer. In European Conference on Computer Vision, 2024. Ostmeier et al. [2024] Sophie Ostmeier, Brian Coste, Mischa Jung, Justin Luitjens, Boris Bornstein, Luciano Del Corro, Dhruv Mahajan, Patrice Simard, Arindam Nori, and Nanyun Peng. LieRE: Generalizing rotary position encodings. In International Conference on Machine Learning, 2024. Press et al. [2022] Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length generalization. In International Conference on Learning Representations, 2022. Rudolph and Giesbrecht [2010] Sebastian Rudolph and Eugenie Giesbrecht. Compositional matrix-space models of language. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 907–916, 2010. Shaw et al. [2018] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 464–468, 2018. Shiv and Quirk [2019] Vighnesh Shiv and Chris Quirk. Novel positional encodings to enable tree-based transformers. In Advances in Neural Information Processing Systems, volume 32, 2019. Su et al. [2021] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864, 2021. Sun et al. [2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. RotatE: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations, 2019. Touvron et al. [2022] Hugo Touvron, Matthieu Cord, and Hervé Jégou. DeiT I: Revenge of the ViT. arXiv preprint arXiv:2204.07118, 2022. Yu et al. [2025] Hao Yu, Tangyu Jiang, Shuning Jia, Shannan Yan, Shunning Liu, Haolong Qian, Guanghao Li, Shuting Dong, and Chun Yuan. ComRoPE: Scalable and robust rotary position embedding parameterized by trainable commuting angle matrices. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. Appendix A Full Proofs A.1 Proof of Path Independence (Theorem˜2.2) Formal statement. Let R1,…,RD∈GR_1,…,R_D∈ G be axis-step generators. The following are equivalent: (i) T(π)T(π) depends only on the count vector (|π|1,…,|π|D)(|π|_1,…,|π|_D) of π (path independence). (i) Every elementary square commutes: T(ij)=T(ji)T(ij)=T(ji) for all i≠ji≠ j. (i) Per-axis generators commute: RiRj=RjRiR_iR_j=R_jR_i for all i≠ji≠ j. When these hold, the relative operator P→=A−1A=R1m1−n1⋯RDmD−nDP_m =A_n^-1A_m=R_1^m_1-n_1·s R_D^m_D-n_D depends only on the displacement −m-n (source minus target)—the journey is fully determined by the composition structure. Proof. (i)⇒ (i): Any two words π,π′π,π with the same count vector are related by a sequence of adjacent transpositions (this is the well-known fact that adjacent transpositions generate the symmetric group, applied to each pair of adjacent letters in the word). Consider an adjacent transposition swapping letters at positions p,p+1p,p+1: π=…kpkp+1…π=… k_p\,k_p+1… becomes π′=…kp+1kp…π =… k_p+1\,k_p…. Since T(π)=Rkm⋯Rkp+1Rkp⋯Rk1T(π)=R_k_m·s R_k_p+1R_k_p·s R_k_1 and the transposition only affects the product Rkp+1RkpR_k_p+1R_k_p, we need Rkp+1Rkp=RkpRkp+1R_k_p+1R_k_p=R_k_pR_k_p+1. If kp=kp+1k_p=k_p+1 this is trivial; otherwise it follows from (i). Composing all adjacent transpositions: T(π)=T(π′)T(π)=T(π ). (i)⇒ (i): Words ijij and jiji have the same count vector (1i,1j)(1_i,1_j) (one step along each axis). By (i): T(ij)=T(ji)T(ij)=T(ji). (i)⇒ (i): By definition, T(ij)=RjRiT(ij)=R_jR_i (rightmost letter acts first) and T(ji)=RiRjT(ji)=R_iR_j. Equality T(ij)=T(ji)T(ij)=T(ji) gives RjRi=RiRjR_jR_i=R_iR_j, i.e., RiRj=RjRiR_iR_j=R_jR_i. Path independence of relative operators: When (i) holds, A=R1n1⋯RDnDA_n=R_1^n_1·s R_D^n_D is well-defined (independent of multiplication order by commutativity). Then P→=A−1A=RD−nD⋯R1−n1R1m1⋯RDmD=R1m1−n1⋯RDmD−nDP_m =A_n^-1A_m=R_D^-n_D·s R_1^-n_1R_1^m_1·s R_D^m_D=R_1^m_1-n_1·s R_D^m_D-n_D, depending only on −m-n. Conversely, if P→P_m depends only on −m-n, then in particular Pj→i+j=P→iP_e_j _i+e_j=P_0 _i. The LHS is (RiRj)−1Rj=Rj−1Ri−1Rj(R_iR_j)^-1R_j=R_j^-1R_i^-1R_j; the RHS is Ri−1R_i^-1. Equality gives Rj−1Ri−1Rj=Ri−1R_j^-1R_i^-1R_j=R_i^-1, hence RiRj=RjRiR_iR_j=R_jR_i. ∎ A.2 Toral Classification: Full Development This subsection provides the full axioms, scope discussion, and supporting material for the toral classification summarized in Section˜2.4. In this subsection, Bs,s′B_s,s denotes the pairwise score matrix (the journey Ps′→sP_s → s); it is not the absolute operator AsA_s used elsewhere. Scope of the classification. The theorem does not derive the torus from first principles. It characterizes all bilinear, norm-preserving, cocycle-compatible attention mechanisms after selecting the maximal connected abelian (commutative) compact frame group T=SO(2)d/2T=SO(2)^d/2 acting with one non-repeated 2D plane per factor. The axiom choice has a design rationale: bilinearity restricts to multiplicative PEs; toral frame invariance selects T (full O(d)O(d) collapses positional structure, Theorem˜2.5); cocycle enforces compositionality; norm preservation prevents rescaling. The result is conditional: it determines what is compatible with this contract, not what is optimal. The novelty is not Schur’s lemma itself, but the identification of this symmetry contract as exactly the contract under which journey-based score/value transport reduces to RoPE-style block rotations. The four axioms below formalize a concrete question: if the only unobservable frame changes are independent rotations of each 2D coordinate plane (i.e., the frame group is the torus T=SO(2)d/2T=SO(2)^d/2), what is the most general attention mechanism consistent with that symmetry? Axiom 1 (Bilinearity). For each position pair (s,s′)(s,s ), there exists Bs,s′∈ℝd×dB_s,s ^d× d such that Att(q,k,s,s′)=q⊤Bs,s′kAtt(q,k,s,s )=q B_s,s k. Axiom 2 (Toral Frame Invariance). For all M∈T=SO(2)d/2M∈ T=SO(2)^d/2: Att(Mq,Mk,s,s′)=Att(q,k,s,s′)Att(Mq,Mk,s,s )=Att(q,k,s,s ). This states that rotating Q and K by the same toral element does not change scores. Axiom 3 (Cocycle). Bs,s=IdB_s,s=I_d (self-attention is unmodified) and Bs,s′Bs′,s′=Bs,s′B_s,s B_s ,s =B_s,s (composition). Equivalently: non-degeneracy (each Bs,s′B_s,s invertible) plus transitivity. Axiom 4 (Norm preservation). Each Bs,s′B_s,s preserves the Euclidean norm: ‖Bs,s′x‖=‖x‖\|B_s,s x\|=\|x\| for all x. This prevents positional encoding from rescaling attention scores. Lemma A.1 (Cocycle factorization). Under Axiom 3, fix a reference position s0s_0 and define Rs=Bs,s0R_s=B_s,s_0. Then Bs,s′=RsRs′−1B_s,s =R_sR_s ^-1. Proof. By the cocycle property: Bs,s0=Bs,s′Bs′,s0B_s,s_0=B_s,s B_s ,s_0, hence Bs,s′=Bs,s0Bs′,s0−1=RsRs′−1B_s,s =B_s,s_0B_s ,s_0^-1=R_sR_s ^-1. ∎ The derivation applies standard commutant and Schur machinery to multiplicity-free representations; the contribution is identifying toral frame invariance as the right contract, from which RoPE, 2D-RoPE, and tree PEs emerge as special cases. Orientation convention. Here Bs,s′=Ps′→s=As−1As′B_s,s =P_s → s=A_s^-1A_s under the notation contract. The factorization Bs,s′=RsRs′−1B_s,s =R_sR_s ^-1 uses gauge variables Rs=Bs,s0R_s=B_s,s_0. With reference s0=0s_0=0 and At=RtA_t=R^t: Rs=Bs,0=As−1A0=R−sR_s=B_s,0=A_s^-1A_0=R^-s, so the gauge angle is θk(s)=−ωks _k(s)=- _ks (negated relative to the absolute angle). The formula θk(s)−θk(s′)=ωk(s′−s) _k(s)- _k(s )= _k(s -s) then matches Ps′→s=Rs′−sP_s → s=R^s -s. If repeated isotypic components or non-abelian frame groups are allowed, the commutant—and therefore the classification—changes. Theorem˜2.4 recovers RoPE [Su et al., 2021] (gauge angle θk(t)=−ωkt _k(t)=- _kt, giving score angle ωk(s′−s) _k(s -s)), 2D-RoPE [Heo et al., 2024] (axis-partitioned planes), simple depth-indexed tree encodings [Shiv and Quirk, 2019], and dot-product attention (θk≡0 _k≡ 0) as special cases. A.3 Proof of the Toral Classification (Theorem˜2.4) Formal statement. Assume d is even. Let T=SO(2)d/2T=SO(2)^d/2 act on ℝdR^d as the direct sum of the standard 2D representation of each factor, one factor per coordinate plane (each SO(2)SO(2) factor acts on exactly one 2-plane; there are no repeated isotypic components). Under Axioms 1–4 with this frame group, the pairwise score matrix is: Bs,s′=diag(R(θ1(s)−θ1(s′)),…,R(θd/2(s)−θd/2(s′))),B_s,s =diag\! (R ( _1(s)- _1(s ) ),…,R ( _d/2(s)- _d/2(s ) ) ), (12) where R(ϕ)=(cosϕ−sinϕsinϕcosϕ)R(φ)= ( smallmatrix φ&- φ\\ φ& φ smallmatrix ) and θk:S→ℝ _k:S assigns an angle per rotation plane per position. Proof. Step 1 (Commutant structure): The commutant of T=SO(2)d/2T=SO(2)^d/2 in ℝd×dR^d× d consists of block-diagonal matrices with 2×22× 2 blocks. Each block carries the standard 2D representation of the corresponding SO(2)SO(2) factor, which is irreducible over ℝR. By Schur’s lemma, the real commutant of an irreducible real representation is ℝR, ℂC, or ℍH. For the 2D representation of SO(2)SO(2), the commutant is spanℝI2,J≅ℂspan_R\I_2,J\ where J=(0−110)J= ( smallmatrix0&-1\\ 1&0 smallmatrix ). Off-diagonal blocks between different SO(2)SO(2) factors vanish because distinct irreducible representations have zero intertwining operators. Step 2 (Frame invariance ⇒ commutant): Axiom 2: M⊤Bs,s′M=Bs,s′M B_s,s M=B_s,s for all M∈TM∈ T. Since each M∈TM∈ T is orthogonal (M⊤=M−1M =M^-1), this becomes M−1Bs,s′M=Bs,s′M^-1B_s,s M=B_s,s , i.e., Bs,s′M=MBs,s′B_s,s M=MB_s,s for all M∈TM∈ T. Thus Bs,s′B_s,s lies in the commutant Comm(T)=X:XM=MX∀M∈TComm(T)=\X:XM=MX\;∀\,M∈ T\. Each 2×22× 2 block has the form akI2+bkJ=ρkR(ψk)a_kI_2+b_kJ= _kR( _k) with ρk=ak2+bk2 _k= a_k^2+b_k^2. Step 3 (Norm preservation): Axiom 4: ‖Bs,s′x‖=‖x‖\|B_s,s x\|=\|x\|. Applied to each 2×22× 2 block: ρk=1 _k=1, giving pure rotation R(ψk)R( _k). Step 4 (Cocycle factorization): Axiom 3: Bs,s′=RsRs′−1B_s,s =R_sR_s ^-1 for some Rs∈TR_s∈ T. Each block: R(ψk(s,s′))=R(θk(s))R(−θk(s′))=R(θk(s)−θk(s′))R( _k(s,s ))=R( _k(s))R(- _k(s ))=R( _k(s)- _k(s )). ∎ A.4 O(d)O(d) Impossibility Formal statement. Under full orthogonal symmetry H=O(d)H=O(d) with d≥2d≥ 2, frame invariance forces Bs,s′=αs,s′IdB_s,s = _s,s I_d. Attention collapses to position-dependent rescaling (by Schur’s lemma on the irreducible standard representation). Proof of Theorem˜2.5. The standard representation of O(d)O(d) on ℝdR^d is irreducible for d≥2d≥ 2 (any invariant subspace is either 0\0\ or ℝdR^d, since O(d)O(d) acts transitively on unit vectors). By Schur’s lemma, its real commutant is ℝ⋅IdR· I_d. Frame invariance forces Bs,s′∈Comm(O(d))=ℝ⋅IdB_s,s (O(d))=R· I_d. ∎ A.5 Proof of V-Rotation Equivariance (Theorem˜2.8) Formal statement. Let positions lie in ℤDZ^D with orthogonal axis-step generators R1,…,RDR_1,…,R_D and absolute operators (from composition) A=R1n1⋯RDnDA_n=R_1^n_1·s R_D^n_D. Assume attention weights are functions only of relative displacement, α=α(−) _nm=α(m-n). Define value aggregation with transport: =∑α(−)A−1Av.c_n= _mα(m-n)\,A_n^-1A_m\,v_m. (a) If the generators commute, then A−1A_n^-1A_m depends only on −m-n (by Theorem˜2.2), and the aggregation rule is translation equivariant in the sense of Definition˜2.7. (b) If some generators do not commute, there exist ,,m,n,u such that A+−1A+≠A−1A_n+u^-1A_m+u≠ A_n^-1A_m, so value transport is path-dependent (it depends on absolute positions, not only displacement). The transport is still a valid operation arising from composition; it simply does not reduce to a pure function of displacement. Proof. (a) Commutativity gives A−1A=R1m1−n1⋯RDmD−nDA_n^-1A_m=R_1^m_1-n_1·s R_D^m_D-n_D, depending only on −m-n. For translation equivariance: F(Tv)+=∑α(−)A+−1Av−F(T_uv)_n+u= _mα(m-n-u)\,A_n+u^-1A_m\,v_m-u. Substituting ′=−m =m-u: =∑′α(′−)A+−1A′+v′= _m α(m -n)\,A_n+u^-1A_m +u\,v_m . By commutativity, A+−1A′+=A−1A′A_n+u^-1A_m +u=A_n^-1A_m (both depend only on ′−m -n). Hence F(Tv)+=F(v)F(T_uv)_n+u=F(v)_n. (b) Suppose Ra,RbR_a,R_b do not commute (relabel axes so a<ba<b). Let =bn=e_b, =a+bm=e_a+e_b, and =−bu=-e_b. Under the canonical order A=RbA_n=R_b, A=RaRbA_m=R_aR_b, so A−1A=Rb−1RaRbA_n^-1A_m=R_b^-1R_aR_b. After shifting: A+−1A+=A−1Aa=RaA_n+u^-1A_m+u=A_0^-1A_e_a=R_a. These are equal iff Rb−1RaRb=RaR_b^-1R_aR_b=R_a, i.e., RaRb=RbRaR_aR_b=R_bR_a—contradicting the assumption. ∎ Remark A.2 (Noncommuting counterexample). Take D=2D=2 with Ra=R(π/4)R_a=R(π/4), Rb=(100−1)R_b= ( smallmatrix1&0\\ 0&-1 smallmatrix ) (a reflection). The journey from (1,1)(1,1) to (0,1)(0,1) is A(0,1)−1A(1,1)=Rb−1RaRbA_(0,1)^-1A_(1,1)=R_b^-1R_aR_b, a reflection-conjugated rotation. After shifting by =−bu=-e_b: the journey from (1,0)(1,0) to (0,0)(0,0) is A−1Aa=RaA_0^-1A_e_a=R_a, a plain rotation. These differ: the same displacement gives different value operators depending on absolute position. A.6 SSM Bridge (Proposition˜F.1) Proof. Mamba’s recurrence ht=Axtht−1+Bxtxth_t=A_x_th_t-1+B_x_tx_t unrolls by induction: hT=∑t=1T(∏s=t+1TAxs)Bxtxth_T= _t=1^T ( _s=t+1^TA_x_s )B_x_tx_t. Define Pt→T=∏s=t+1TAxsP_t→ T= _s=t+1^TA_x_s and vt=Bxtxtv_t=B_x_tx_t. Then hT=∑t=1TPt→Tvth_T= _t=1^TP_t→ Tv_t, which has the form of (3) with αTt=1 _Tt=1 (uniform, unnormalized weights) and content-dependent path operators. When Axt∈SO(2)d/2A_x_t (2)^d/2, each prefix product is a block-diagonal rotation. The associative scan Pt→T=AT⋅AT−1⋯At+1P_t→ T=A_T· A_T-1·s A_t+1 can be computed in O(logN)O( N) parallel depth using the standard parallel prefix algorithm. ∎ A.7 ALiBi Expressivity Separation Proposition A.3 (Representational limitations of additive linear biases). Let d≥4d≥ 4 be even. Compare: • ALiBi [Press et al., 2022]: AttALiBi(q,k,n)=q⊤k−m⋅nAtt_ALiBi(q,k,n)=q k-m· n, where n=s−s′n=s-s is the signed relative position and m>0m>0 is a per-head slope. • Toral mechanism (Theorem˜2.4): Atttoral(q,k,n)=q⊤RθnkAtt_toral(q,k,n)=q R_θ^n\,k, where Rθ=diag(R(θ1),…,R(θd/2))R_θ=diag(R( _1),…,R( _d/2)). Then: (a) Score dimensionality gap. For fixed q,kq,k, the ALiBi score is affine in n: AttALiBi=q⊤k−mnAtt_ALiBi=q k-mn. The toral score is a trigonometric polynomial: Atttoral(q,k,n)=∑b=1d/2αbcos(nθb)+βbsin(nθb),Att_toral(q,k,n)= _b=1^d/2 _b (n _b)+ _b (n _b), where αb=q2b−1k2b−1+q2bk2b _b=q_2b-1k_2b-1+q_2bk_2b and βb=q2b−1k2b−q2bk2b−1 _b=q_2b-1k_2b-q_2bk_2b-1. The ALiBi positional signal spans a 11-dimensional function space; the toral signal spans a d-dimensional space. (b) Strict separation. For any period p≥3p≥ 3, the periodic attention pattern ϕ(n)=cos(2πn/p)φ(n)= (2π n/p) is representable by the toral mechanism but not by any ALiBi head. (c) Approximate subsumption. On any bounded context |n|≤N|n|≤ N, the toral mechanism can ϵε-approximate any ALiBi attention pattern for any ϵ>0ε>0. (d) Value-path deficiency. ALiBi modifies only attention logits, leaving Pj→k=IP_j→ k=I for all j,kj,k. The value aggregation under ALiBi is order-blind: standard attention with ALiBi computes k=∑jαkjvjc_k= _j _kjv_j with no positional modulation of values, regardless of the slope m. This separation does not imply that either mechanism is uniformly better for all tasks; it identifies positional functions representable by one class and not the other. Proof. (a) The b-th 2×22× 2 rotation block of RθnR_θ^n contributes αbcos(nθb)+βbsin(nθb) _b (n _b)+ _b (n _b) to q⊤Rθnkq R_θ^nk, with αb,βb _b, _b as stated. For distinct θb _b, the d/2d/2 cosine and d/2d/2 sine terms are linearly independent functions of n, spanning a d-dimensional space. ALiBi contributes only the single linear function n↦−mn -mn. (b) Set θ1=2π/p _1=2π/p and q=k=e1∈ℝdq=k=e_1 ^d. Then Atttoral(e1,e1,n)=cos(2πn/p)Att_toral(e_1,e_1,n)= (2π n/p), which equals ϕ(n)φ(n) for all n. For ALiBi, q⊤k−mn=cos(2πn/p)q k-mn= (2π n/p) for all n would require a bounded periodic function to equal an unbounded affine function—impossible. (c) The key identity is sin(nθ)/θ→n (nθ)/θ→ n uniformly on |n|≤N|n|≤ N as θ→0θ→ 0, with error |n−sin(nθ)/θ|≤|n|3θ2/6|n- (nθ)/θ|≤|n|^3θ^2/6. Choose θ1=6ϵ/(mN3) _1= 6ε/(mN^3) and q,kq,k in block 1 so that β1=−m/θ1 _1=-m/ _1 and α1=0 _1=0. Then |β1sin(nθ1)−(−mn)|=m|n−sin(nθ1)/θ1|≤mN3θ12/6=ϵ| _1 (n _1)-(-mn)|=m|n- (n _1)/ _1|≤ mN^3 _1^2/6=ε for all |n|≤N|n|≤ N. The constant c0=q⊤kc_0=q k is matched by the remaining blocks. (d) ALiBi adds a scalar bias to the attention logit; it does not transform value vectors. Hence Pj→k=IP_j→ k=I identically. ∎ A.8 Cocycle from Primitives The cocycle axiom (Bs,s=IB_s,s=I, Bs,s′Bs′,s′=Bs,s′B_s,s B_s ,s =B_s,s ) follows from two simpler primitives: (P1) Non-degeneracy: each Bs,s′B_s,s is invertible. (P2) Composability: Bs,s′=Bs,s′⋅Bs′,s′B_s,s =B_s,s · B_s ,s . Proof: Set s′=s =s in (P2): Bs,s′=Bs,sBs,s′B_s,s =B_s,sB_s,s ; invertibility of Bs,s′B_s,s gives Bs,s=IB_s,s=I. Conversely, cocycle ⇒ composability is by definition, and Bs,s′⋅Bs′,s=Bs,s=IB_s,s · B_s ,s=B_s,s=I gives invertibility with inverse Bs,s′−1=Bs′,sB_s,s ^-1=B_s ,s. Appendix B Additional Theoretical Results This section collects additional theoretical results that complement the core framework. B.1 DFT Sufficiency for White Sources Proposition B.1 (DFT sufficiency for white sources). Fix K=d/2K=d/2 distinct frequencies θ1,…,θK∈[0,2π) _1,…, _K∈[0,2π) and let N→∞N→∞. If the source is white (S(ω)=S0S(ω)=S_0 for all ω), then any such set asymptotically achieves the same mutual information I(X;Y)→d2log(1+NS0/σ2)I(X;Y)→ d2 (1+NS_0/σ^2). In particular, equispaced DFT frequencies θk=2πk/K _k=2π k/K achieve this bound. Proof. Since K is fixed and the frequencies are distinct, their pairwise separation δ>0δ>0 is constant. By the large-sieve inequality, ΦθHΦθ/N→IK _θ^H _θ/N→ I_K as N→∞N→∞, so I(X;Y)→∑k=1Klog(1+NS(θk)/σ2)I(X;Y)→ _k=1^K (1+NS( _k)/σ^2). When S(ω)=S0S(ω)=S_0 is constant, every term equals log(1+NS0/σ2) (1+NS_0/σ^2) regardless of the θk _k, giving I(X;Y)→d2log(1+NS0/σ2)I(X;Y)→ d2 (1+NS_0/σ^2). ∎ B.2 MSE Gain of Learned Frequencies Proposition B.2 (MSE gain of learned frequencies). Fix K=d/2K=d/2 and let N→∞N→∞. For a source with non-uniform power spectral density (PSD) S(ω)S(ω) whose K largest values occur at distinct frequencies θ1∗,…,θK∗ _1^*,…, _K^*, the asymptotic MSE gap between optimal learned frequencies and fixed DFT frequencies satisfies: MSEDFT−MSEopt≥ 0(N→∞),MSE_DFT-MSE_opt\;≥\;0 (N→∞), with equality if and only if the DFT frequencies already sample the K largest values of S(ω)S(ω). Proof. With K fixed and frequencies distinct, the large-sieve gives ΦθHΦθ/N→IK _θ^H _θ/N→ I_K, so the MSE for estimating X from Y=ΦθX+ZY= _θX+Z reduces to MSEθ=∑k=1Kσ2S(θk)σ2+NS(θk)MSE_θ= _k=1^K σ^2S( _k)σ^2+NS( _k). Since each term is a decreasing function of S(θk)S( _k), the MSE is minimized by choosing θ1∗,…,θK∗ _1^*,…, _K^* at the K largest values of S(ω)S(ω). The DFT places frequencies at θk=2πk/K _k=2π k/K, which generically miss these peaks. The inequality follows, with equality exactly when the DFT grid coincides with the spectral maxima. ∎ Together, Propositions˜B.1 and B.2 explain the MNIST results (Section˜E.1): for white sources the DFT suffices, but for structured signals (images), learned frequencies select task-relevant spectral peaks that the DFT grid generically misses. B.3 Context-Sensitive Separation Proposition B.3 (Context-sensitive separation). Let =(x1,…,xN)x=(x_1,…,x_N) and ′x be two sequences that differ only at position s, with xs≠xs′x_s≠ x_s . Fix positions j<s<kj<s<k. (a) Content-independent operators. If Pj→k=Rk−jP_j→ k=R^k-j, then the value-path transform applied to vjv_j at output position k is the same for both sequences: Pj→k=Pj→k′P_j→ k=P _j→ k. Intervening content is invisible to the value path. (b) Content-dependent operators. If Pj→k=∏t=jk−1R~xtP_j→ k= _t=j^k-1 R_x_t where R~xt=h(xt)∈GL(d) R_x_t=h(x_t) (d) is a learned function of token content, and R~xs≠R~xs′ R_x_s≠ R_x_s , then Pj→k≠Pj→k′P_j→ k≠ P _j→ k. For any vjv_j outside a proper subspace of ℝdR^d (i.e., for generic inputs), the value-path outputs differ: Pj→kvj≠Pj→k′vjP_j→ k\,v_j≠ P _j→ k\,v_j. Proof. Part (a) is immediate: Rk−jR^k-j depends only on positions, not content. For part (b), factor the shared portions: Pj→k=LR~xsMP_j→ k=L\, R_x_s\,M and Pj→k′=LR~xs′MP _j→ k=L\, R_x_s \,M, where L=∏t=js−1R~xtL= _t=j^s-1 R_x_t and M=∏t=s+1k−1R~xtM= _t=s+1^k-1 R_x_t are common invertible factors. Then Pj→k−Pj→k′=L(R~xs−R~xs′)MP_j→ k-P _j→ k=L ( R_x_s- R_x_s )M. Since L,M∈GL(d)L,M (d) and R~xs≠R~xs′ R_x_s≠ R_x_s , the difference is a nonzero linear map, so its kernel is a proper subspace of ℝdR^d. ∎ This result motivates JoFormer-projected: content-dependent journey operators allow the value path to encode contextual information that fixed-angle operators cannot capture, providing a formal separation between the JoFormer-fixed and JoFormer-projected variants. B.4 Shift-Invariance and Order-Awareness We show that the compositional embedding simultaneously achieves two desirable properties: invariance to uniform shifts (for translation-invariant tasks) and sensitivity to element ordering (for sequence-dependent tasks). Definition B.4 (m-Representations). For a sequence of length N with embeddings a1,…,aN∈ℝda_1,…,a_N ^d, choose a window length m. For each window starting at position k, form the window embedding: sk=∑i=1mRi−1ak+i−1,s_k= _i=1^mR^i-1a_k+i-1, (13) where R∈O(d)R (d) is a fixed block-diagonal rotation. Partition sks_k into K=d/2K=d/2 blocks sk,1,…,sk,Ks_k,1,…,s_k,K and define the magnitude vector: vk=(‖sk,1‖,…,‖sk,K‖)∈ℝK.v_k= (\|s_k,1\|,…,\|s_k,K\| ) ^K. (14) Under circular boundary conditions (indices modulo N), the circular m-representation is v∘=∑k=1Nvkv = _k=1^Nv_k. Theorem B.5 (Shift-invariance). For orthonormal R∈O(d)R (d), the circular m-representation v∘v is invariant under uniform circular shifts of the input along any axis. Proof. A uniform circular shift by one replaces ata_t with at+1modNa_t+1 N in each window. The shifted window at position k contains elements ak+1,…,ak+ma_k+1,…,a_k+m (mod N), which is the original window at position k+1k+1. So sk′=sk+1modNs_k =s_k+1 N, and the multiset of window embeddings skk=1N\s_k\_k=1^N is unchanged. Hence the sum of magnitude vectors v∘=∑kvkv = _kv_k is invariant. ∎ Theorem B.6 (Order-awareness). Let R∈O(d)R (d) be a block-diagonal rotation matrix with K=d/2K=d/2 blocks having angles θ1,…,θK _1,…, _K, at least one of which satisfies θℓ/π∉ℚ _ /π . For elements a1,…,am∈ℝda_1,…,a_m ^d in general position, the magnitude vector v=(‖s(1)‖,…,‖s(K)‖)v=(\|s^(1)\|,…,\|s^(K)\|) of the window embedding s=∑i=1mRi−1ais= _i=1^mR^i-1a_i distinguishes different permutations: if σ≠idσ , then generically v(a1,…,am)≠v(aσ(1),…,aσ(m))v(a_1,…,a_m)≠ v(a_σ(1),…,a_σ(m)). Proof. In the ℓ -th block with angle θℓ _ , represent the block component of aia_i as zi(ℓ)∈ℂz_i^( ) , so s(ℓ)=∑i=1mωℓi−1zi(ℓ)s^( )= _i=1^m _ ^i-1\,z_i^( ) with ωℓ=eθℓ _ =e^i _ . For a fixed non-identity permutation σ, define the collision set σ=(a1,…,am)∈ℝdm:‖s(ℓ)‖2=‖s′(ℓ)‖2for all ℓC_σ=\(a_1,…,a_m) ^dm:\|s^( )\|^2=\|s ( )\|^2\;for all \, where s′(ℓ)=∑iωℓi−1zσ(i)(ℓ)s ( )= _i _ ^i-1z_σ(i)^( ). Each equation is polynomial, so σC_σ is an algebraic variety. It suffices to show σ≠ℝdmC_σ ^dm. Choose block ℓ with θℓ/π∉ℚ _ /π and set ω=ωℓω= _ . The difference of squared magnitudes is a Hermitian form: |s|2−|s′|2=∗H,Hij=ωi−j−ωσ−1(i)−σ−1(j).|s|^2-|s |^2=z^*H\,z, H_ij=ω^i-j-ω^σ^-1(i)-σ^-1(j). For σ≠idσ , there exist i,ji,j with i−j≠σ−1(i)−σ−1(j)i-j≠σ^-1(i)-σ^-1(j) (otherwise σ−1(i)=i+cσ^-1(i)=i+c for constant c, forcing c=0c=0, contradicting σ≠idσ ). Since θℓ/π∉ℚ _ /π , the element ω has infinite order, so Hij≠0H_ij≠ 0 and the Hermitian form is not identically zero. Hence σC_σ is a proper algebraic subvariety of measure zero. The union over all m!−1m!-1 non-identity permutations is still measure zero. ∎ B.5 Structured Concatenation Definition B.7 (Axis-k concatenation). Given two compositional embeddings X=(a;R1n1,…,Rknk,…,RDnD)X=(a;\;R_1^n_1,…,R_k^n_k,…,R_D^n_D) and Y=(b;R1n1,…,Rkmk,…,RDnD)Y=(b;\;R_1^n_1,…,R_k^m_k,…,R_D^n_D) that share the same axis-step generators RiR_i and the same exponents on all axes i≠ki≠ k, their concatenation along axis k is: X⊕kY=(a+Rknkb;R1n1,…,Rknk+mk,…,RDnD).X _kY= (a+R_k^n_k\,b;\;\;R_1^n_1,…,R_k^n_k+m_k,…,R_D^n_D ). (15) Theorem B.8 (Concatenation invertibility). Given X⊕kYX _kY, the axis dimensions (nk,mk)(n_k,m_k), and invertibility of RkR_k, either component is uniquely recoverable from the other: (a) Given a (the value of X): b=(Rknk)−1((X⊕kY)value−a)b=(R_k^n_k)^-1 ((X _kY)_value-a ). (b) Given b (the value of Y): a=(X⊕kY)value−Rknkba=(X _kY)_value-R_k^n_k\,b. Proof. From (15), the value component is c=a+Rknkbc=a+R_k^n_kb. Since Rk∈GL(d)R_k (d), the matrix RknkR_k^n_k is invertible, so given a: b=(Rknk)−1(c−a)b=(R_k^n_k)^-1(c-a), and given b: a=c−Rknkba=c-R_k^n_kb. ∎ Appendix C Classical Transforms and Fixed-Operator Limits The monoidal embedding E=∑t=0N−1RtvtE= _t=0^N-1R^tv_t directly recovers Fourier-style spectral features when R is a fixed block rotation. Fixed-sign Hadamard/Walsh constructions are useful analogies and limiting cases, but they do not satisfy the same connected toral SO(2)d/2SO(2)^d/2 contract used in Theorem˜2.4. C.1 Discrete Fourier Transform Proposition C.1 (DFT as compositional embedding). The discrete Fourier transform is a special case of the compositional embedding: 1. In 1D, composing n data points with a shared block-diagonal rotation R=diag(R(2πkn))k=0n−1R=diag(R( 2π kn))_k=0^n-1 yields an embedding whose k-th block encodes (ℜ(Xk),ℑ(Xk))⊤( (X_k), (X_k)) , where XkX_k is the k-th DFT coefficient. 2. In 2D, applying the 1D construction along rows then columns recovers the full 2D DFT. 3. Replacing fixed frequencies θk=2πk/n _k=2π k/n with learned θk _k gives a generalized Fourier transform with task-optimal spectral components. Proof. One-dimensional case. Consider n input embeddings (vi,R)(v_i,R) for i=1,…,ni=1,…,n, all sharing the same transformation R. Their composition yields (V,Rn)(V,R^n) where V=∑i=1nRi−1vi.V= _i=1^nR^i-1v_i. (16) Let R be a real 2n×2n2n× 2n block-diagonal matrix consisting of n rotation blocks. The k-th block (k=0,…,n−1k=0,…,n-1) is Rk=(cos2πkn−sin2πknsin2πkncos2πkn),R=diag(R0,R1,…,Rn−1),R_k= pmatrix \! 2π kn&- \! 2π kn\\[4.0pt] \! 2π kn& \! 2π kn pmatrix, R=diag(R_0,R_1,…,R_n-1), so Rn=I2nR^n=I_2n (each RknR_k^n is a rotation by 2πk2π k). For input vectors, take vi=(ai,0,ai,0,…,ai,0)⊤∈ℝ2nv_i=(a_i,0,a_i,0,…,a_i,0) ^2n, repeating (ai,0)⊤(a_i,0) across each block. Because R is block-diagonal, Ri−1R^i-1 rotates the k-th block of viv_i by angle (i−1)⋅2πk/n(i-1)· 2π k/n: Ri−1vi|block k=(aicos2πk(i−1)naisin2πk(i−1)n).R^i-1v_i |_block k= pmatrixa_i \! 2π k(i-1)n\\[6.0pt] a_i \! 2π k(i-1)n pmatrix. Summing over i, the k-th block of V is V|block k=(∑i=1naicos2πk(i−1)n∑i=1naisin2πk(i−1)n),k=0,1,…,n−1.V |_block k= pmatrix _i=1^na_i \! 2π k(i-1)n\\[15.00002pt] _i=1^na_i \! 2π k(i-1)n pmatrix, k=0,1,…,n-1. (17) Defining the DFT coefficients Xk=∑i=1naiej2π(i−1)k/nX_k= _i=1^na_i\,e^j2π(i-1)k/n, we have V|block k=(ℜ(Xk),ℑ(Xk))⊤V |_block k=( (X_k),\; (X_k)) . The compositional embedding exactly encodes all n DFT coefficients. Two-dimensional case. Consider an n×n× n array ai,ja_i,j. Apply the 1D construction along each row i, yielding output WiW_i whose k-th block is Wi|block k=(ℜ(Xi,k(row))ℑ(Xi,k(row))),where Xi,k(row)=∑m=1nai,mej2π(m−1)k/n.W_i |_block k= pmatrix \! (X_i,k^(row) )\\[3.0pt] \! (X_i,k^(row) ) pmatrix, X_i,k^(row)= _m=1^na_i,m\,e^j2π(m-1)k/n. Then apply the same construction along the column index i for each frequency k. The result for the p-th block is Yp,k=∑i=1nXi,k(row)ej2π(i−1)p/n=∑i=1n∑m=1nai,mexp(j2πn[(m−1)k+(i−1)p]),Y_p,k= _i=1^nX_i,k^(row)\,e^j2π(i-1)p/n= _i=1^n _m=1^na_i,m\, \! (j 2πn [(m-1)k+(i-1)p ] ), which is the (p,k)(p,k) entry of the 2D DFT. The double sum is separable: first n independent 1D DFTs along the rows, then n along the columns. This is exactly what the compositional embedding with the same rotation R along both axes achieves. Learned frequencies. Setting θk=2πk/N _k=2π k/N recovers the standard DFT. Learned θk _k select task-optimal spectral components—a generalized Fourier transform where the basis frequencies are optimized for the downstream task rather than fixed to uniform spacing. ∎ C.2 Discrete Cosine Transform The discrete cosine transform (DCT) can be obtained similarly to the DFT by choosing a real-valued cosine basis. The 1D DCT-I of length n is: Ck=∑i=1naicos(π(i−1)(k−1)n),k=1,…,n.C_k= _i=1^na_i \! ( π(i-1)(k-1)n ), k=1,…,n. This is the real part of a DFT applied to an even extension of the sequence. In our framework, one sets the operator R to perform rotations with half-frequency increments (angles πk/nπ k/n instead of 2πk/n2π k/n) so that the embedding sum yields CkC_k on the output vector. Because these cosine rotations commute, the transform is separable across dimensions. In particular, an n×n× n 2D DCT is achieved by applying the 1D DCT along the rows and then along the columns (or vice versa), using the same commuting operators. C.3 Discrete Sine Transform The discrete sine transform (DST) uses a sine basis instead of cosines. The DST-I is: Sk=∑i=1naisin(πikn+1),k=1,…,n.S_k= _i=1^na_i \! ( π ikn+1 ), k=1,…,n. This corresponds to the imaginary part of the DFT for an odd extension of the data. In our embedding model, the DST is realized by choosing operators that introduce half-sample shifts and sign flips to generate sine terms. Since these sine-based rotations also commute, the multi-dimensional DST is performed by applying the 1D DST along each axis. C.4 Hadamard Transform The Hadamard transform can be written as Hnx=∑i=1nRi−1viH_nx= _i=1^nR^i-1v_i for specific R and viv_i (assuming n=2mn=2^m). Definition of R. Let R=diag(1,−1,1,−1,…,1,−1)R=diag(1,-1,1,-1,…,1,-1), so R2=IR^2=I. Definition of viv_i. For input x=(x1,…,xn)⊤x=(x_1,…,x_n) , define vi=xi⋅(ei+ei+n/2)if 1≤i≤n/2,xi⋅(ei−n/2−ei)if n/2<i≤n,v_i= casesx_i·(e_i+e_i+n/2)&if 1≤ i≤ n/2,\\ x_i·(e_i-n/2-e_i)&if n/2<i≤ n, cases where eje_j is the j-th standard basis vector. Base case (n=2n=2). H2=(111−1)H_2= ( smallmatrix1&1\\ 1&-1 smallmatrix ), R=(100−1)R= ( smallmatrix1&0\\ 0&-1 smallmatrix ), v1=x1(1,1)⊤v_1=x_1(1,1) , v2=x2(1,−1)⊤v_2=x_2(1,-1) . Then v1+Rv2=(x1+x2,x1−x2)⊤=H2xv_1+Rv_2=(x_1+x_2,\;x_1-x_2) =H_2x. Inductive step. Assume the claim holds for n=2kn=2^k. For n′=2n =2n, write x=(x(1)⊤,x(2)⊤)⊤x=(x^(1) ,x^(2) ) . By Sylvester’s construction, H2nx=(Hn(x(1)+x(2)),Hn(x(1)−x(2)))⊤H_2nx=(H_n(x^(1)+x^(2)),\;H_n(x^(1)-x^(2))) . Both terms can be written as ∑iRni−1vi _iR_n^i-1v_i by the inductive assumption, completing the induction. C.5 Walsh Transform The Walsh transform Wn=PHnW_n=PH_n is obtained by reordering the Hadamard matrix rows so that sign changes (sequency) increase monotonically, where P is a Gray-code permutation matrix. Given the Hadamard embedding R,vi\R,v_i\, define R′=PRP−1R =PRP^-1 and vi′=Pviv _i=Pv_i. Then ∑iR′i−1vi′=P(∑iRi−1vi)=PHn(P−1x)=Wnx. _iR ^i-1v _i=P ( _iR^i-1v_i )=P\,H_n(P^-1x)=W_nx. C.6 Learnable Generalization All classical transforms above use fixed operators; the compositional framework generalizes them by learning R (or equivalently the angles θk _k) from data. The magnitude ‖E‖2=∑t,t′vt⊤Rt′−tvt′\|E\|^2= _t,t v_t R^t -tv_t depends only on pairwise relative positions (shift-invariant), yet permuting the sequence changes ‖E‖2\|E\|^2 (order-sensitive). This explains why learned monoidal embeddings outperform DFT at low dimension (Section˜E.1): they select exactly the frequencies needed for the task, rather than committing to uniform spacing. Appendix D Application Sketches The multi-axis compositional framework extends naturally beyond sequences and images. We sketch four application domains. D.1 Audio and Spectrograms Audio spectrograms are naturally arranged on a time–frequency grid. Assign one composition operator ∘t _t for the temporal axis and another ∘f _f for the frequency axis, giving each spectrogram cell a compositional embedding: Eaudio=∑t,fRtimetRfreqfvt,f.E_audio= _t,fR_time^\,t\,R_freq^\,f\,v_t,f. When RtimeR_time and RfreqR_freq commute, the journey operator P(t′,f′)→(t,f)=Rtimet′−tRfreqf′−fP_(t ,f )→(t,f)=R_time^t -tR_freq^f -f depends only on the time–frequency displacement, enabling models to incorporate both temporal and spectral locality in a principled way. The same construction applies to EEG or other time–frequency representations. D.2 Video A video is an H×W×TH× W× T grid (two spatial dimensions plus time). Introducing a temporal composition operator ∘time _time in addition to the spatial operators gives a 3-axis embedding: Evideo=∑i,j,tRrowiRcoljRtimetvi,j,t.E_video= _i,j,tR_row^\,i\,R_col^\,j\,R_time^\,t\,v_i,j,t. By Theorem˜2.2, when the three generators commute, composing spatially within each frame and then advancing in time yields the same result as first advancing each pixel in time and then composing spatially. This coherence ensures consistent representations of motion regardless of composition order. D.3 Multimodal Fusion For multimodal data (e.g., text ++ image ++ audio), assign one composition axis per modality with generator RmodR_mod, plus intra-modality axes for sequence or spatial structure. The cross-modal journey Ptext→image=Rmod−1⋅Atext−1AimageP_text =R_mod^-1· A_text^-1A_image provides a principled alignment operator between modalities, with commutativity ensuring that fusing text-then-audio gives the same result as audio-then-text. D.4 Multi-Axis Alignment Given two compositional embeddings X and Y, the optimal alignment along axis k can be found via inner product maximization: s∗=argmaxs⟨X,RksY⟩.s^*= _s\; X,\;R_k^s\,Y . (18) This is a cross-correlation computed in the rotated embedding space. For D-dimensional alignment, apply shifts s1,…,sDs_1,…,s_D on each axis: Y′=R1s1⋯RDsDY =R_1^s_1·s R_D^s_DY. Because the RiR_i commute, the order of shifts does not matter. This enables operations like sequence alignment or embedding fusion by sliding one structure relative to another. Appendix E Extended Experimental Details E.1 MNIST Monoidal Embedding Setup. The monoidal embedding demonstrates the compositional framework on image classification. Each pixel pijp_ij of a 28×2828× 28 image is represented as a data point with content pij⋅p_ij·e (pixel intensity times a d-dimensional basis vector e) and two axis-step generators: RxR_x for the horizontal axis and RyR_y for the vertical axis. Both axis-step generators are block-diagonal rotations Rx=diag(R(θ1x),…,R(θd/2x))R_x=diag(R(θ^x_1),…,R(θ^x_d/2)), Ry=diag(R(θ1y),…,R(θd/2y))R_y=diag(R(θ^y_1),…,R(θ^y_d/2)), sharing the same block structure so that RxRy=RyRxR_xR_y=R_yR_x (flat regime). The full image embedding is the 2D composition over all pixels: Eimage=∑i=027∑j=027pijRyiRxj.E_image= _i=0^27 _j=0^27p_ij\;R_y^\,i\,R_x^\,j\;e. Each pixel’s content is rotated by the product of its row and column axis-step generators, and the results are summed into a single d-dimensional vector. Because the axis-step generators commute, this composition is path-independent: composing rows-then-columns gives the same embedding as columns-then-rows. In each 2D block (identified with ℂC), the k-th component reduces to E(k)=∑i,jpijei(jθkx+iθky)E^(k)= _i,jp_ij\,e^i(jθ^x_k+iθ^y_k)—a 2D Fourier-like transform with learned frequencies (θkx,θky)(θ^x_k,θ^y_k). With fixed frequencies θkx=2πk/dθ^x_k=2π k/d, θky=2πk/dθ^y_k=2π k/d, this recovers the standard 2D DFT; with learned frequencies, the model selects task-optimal spectral components. The d-dimensional embedding EimageE_image is fed into an MLP classifier (128 hidden units, ReLU, 10-class softmax) trained end-to-end with cross-entropy loss. Training: Adam lr=10−3=10^-3, 50 epochs, batch=128, no augmentation. The only learnable parameters in the embedding itself are the d angles (θkx,θky)k=1d/2(θ^x_k,θ^y_k)_k=1^d/2; all representational capacity comes from the compositional structure. Results. Table 5: MNIST accuracy by embedding method and dimension (single seed). Monoidal embeddings learn task-optimal spectral components; DFT uses fixed Fourier frequencies. Method d=2d=2 d=8d=8 d=32d=32 d=784d=784 Monoidal (learned) 55.2% 86.4% 96.5% 97.5% DFT (fixed) 21.0% 75.3% 95.5% — MLP baseline 97.2% (128 hidden, d=784d=784) The advantage is most dramatic at low dimension: at d=2d=2, monoidal achieves 55.2% (well above chance 10%) while DFT achieves only 21.0%. At d=32d=32, monoidal reaches 96.5%—within 0.7% of the full MLP baseline (97.2%) despite using 24×24× fewer input dimensions. This is consistent with the core insight: learned angles select task-relevant spectral components that fixed Fourier bases cannot match. The advantage diminishes at high d (where frequency allocation is less constrained), suggesting that the monoidal structure’s benefit is in efficient spectral selection. E.2 CIFAR-100 LieRE Experiment Architecture: ViT-Tiny with D=384D=384, 12 layers, 6 heads, head_dim=64, ∼14.9 14.9M params. Patch size: 4×4→8×8=644× 4→ 8× 8=64 patches + CLS token. Training: Adam lr=10−4=10^-4, cosine annealing 200 epochs, bf16-mixed precision, seed=42, NVIDIA H100 PCIe. Augmentation: RandomCrop(32, padding=4), RandomHorizontalFlip, Normalize([0.5071, 0.4867, 0.4408], [0.2675, 0.2565, 0.2761]). Extended results at 400 epochs: Method Q/K only Q/K/V V effect Axial-dense 70.61% 71.12% +0.51%+0.51\% LieRE64 70.64% 69.41% −1.23%-1.23\% Setup B full reproducibility. Optimizer: AdamW (β1=0.9 _1=0.9, β2=0.999 _2=0.999, ε=10−8 =10^-8, weight decay=0.1=0.1). Batch size: 128. MLP ratio: 4× (hidden dim =4D=4D). Dropout: 0.1; no drop-path or stochastic depth. Schedule: cosine annealing from lr=10−3=10^-3 to 0 over 300 epochs, no warmup. Mixup: α=0.8α=0.8; label smoothing 0. Augmentation: random crop (pad 4, reflect), random horizontal flip (50%), RandAugment (n=2n=2, m=9m=9), random erasing (50% probability, 16×1616×16 zeroed patch). Normalization: mean =[0.4914,0.4822,0.4465]=[0.4914,0.4822,0.4465], std =[0.2470,0.2435,0.2616]=[0.2470,0.2435,0.2616]. Seed: 42; torch.manual_seed ++ cuda.manual_seed_all. Deterministic: entire dataset GPU-resident; batches via torch.randint (no DataLoader workers). Train/test: standard CIFAR-100 split (50K/10K), no validation hold-out. E.3 CIFAR-100 Scaling Results Table 6: CIFAR-100 accuracy scaling: joformer_axial (learned axial + V rotation) vs. rope2d (fixed axial, Q/K only) and all 9 variants. Setup B, 300 epochs cosine, single seed. Model D=32D=32 D=64D=64 D=128D=128 D=256D=256 joformer_axial 52.77 61.33 66.67 63.27 joformer (combined + V) 52.43 58.83 66.19 62.10 joformer_old (fixed + V) 50.54 59.13 66.10 61.85 monoidal_axial (learned, no V) 51.75 59.10 64.81 60.92 rope2d (fixed, no V) 50.91 59.23 64.22 61.39 learned (additive PE) 52.23 57.26 60.74 55.49 In these paired single-seed CIFAR protocols, the joformer_axial advantage over rope2d is consistent at ∼2% 2\% across all scales: +1.86%+1.86\% (D=32), +2.10%+2.10\% (D=64), +2.45%+2.45\% (D=128), +1.88%+1.88\% (D=256). At D=128, the top three models are all V-rotation variants (66.67, 66.19, 66.10), separated from non-V models (64.81, 64.22) by a clear ∼2% 2\% gap. In this single-seed grid, joformer_axial leads at every checkpoint from epoch 50 onward—not a late-stage effect. Key finding: V rotation × learnable frequencies is synergistic. Neither V rotation alone (joformer_old vs. rope2d: +0.46%+0.46\% at D=256) nor learnable frequencies alone (monoidal_axial vs. rope2d: −0.47%-0.47\%) provides the full benefit. Together (joformer_axial vs. rope2d: +1.88%+1.88\%), they interact synergistically. The mechanism: V rotation gives the model a richer position-dependent value transformation; learnable frequencies adapt to leverage it. Per-layer frequencies don’t help. Shared frequencies across layers outperform per-layer variants (e.g., joformer_axial 63.27% vs. per-layer 62.76% at D=256). D=256 non-monotonicity. The D=256 models are not uniformly better than D=128 under this fixed training recipe, so the table should be read as a paired PE comparison at each width rather than as an optimized scaling law. Consistent positional encoding helps the residual stream maintain coherent position information across layers. E.4 ImageNet ViT-S Model: ViT-S (D=384D=384, 12 layers, 6 heads, patch=16, img=224). Recipe: DeiT-I (AdamW lr=10−3=10^-3, wd=0.05, cosine schedule + 5-epoch warmup, 300 epochs, batch=1024, RandAugment(9, 0.5), Mixup 0.8, CutMix 1.0, Random Erasing 0.25, Label Smoothing 0.1, Stochastic Depth 0.1, AMP fp16). Hardware: 2× NVIDIA H100 PCIe. JoFormer: K⊤QK Q ordering, V rotation = RjvjR_jv_j followed by output inverse Ri−1ciR_i^-1c_i. Training time: ∼72 72 hours for RoPE2D, ∼94 94 hours for JoFormer in this ViT-S setup. E.5 Wikipedia Language Modeling Data: Full English Wikipedia dump (28.8M lines, ∼983 983M BPE tokens with vocab=8000). Architecture: standard transformer with rotary attention, FFN ratio 4×, pre-norm (LayerNorm before attention and FFN). Training: AdamW (lr=2×10−4=2× 10^-4, β1=0.9 _1=0.9, β2=0.95 _2=0.95, wd=0.01), cosine schedule to lr=min2×10−5_ =2× 10^-5, batch=32, block_size=512, gradient accumulation=1, AMP bf16. JoFormer-projected MLP: input LayerNorm → Linear(d, d) → GELU → Linear(d, d/2d/2) (angles). Evaluation: validation PPL on held-out 1% of data. E.6 Length Generalization Architecture: 5 windowed layers (window=32) + 1 full-attention NoPE layer. The windowed layers use either RoPE or JoFormer variants; the NoPE layer has no positional encoding. JoFormer-projected recipe (staged; in our setup this worked more reliably than training projected angles from scratch): Stage 1: JoFormer-fixed (RoPE angles + V rotation), lr=5×10−4=5× 10^-4, 100K iters. Stage 2: Continue fixed-angle training at lr=2×10−4=2× 10^-4, 50K iters. Stage 3: Fine-tune to projected angles (zero-initialize angle projector weights so initial angles match fixed), lr=5×10−5=5× 10^-5, 50K iters. The first two stages (150K total) match the RoPE baseline iteration count; stage 3 is a fine-tuning phase that converts fixed angles to content-dependent projected angles. Evaluation: 200 iterations per length, fixed seed, lengths 512, 1024, 2048, 4096, 8192. E.7 Parameter and Wall-Clock Overhead Table 7: JoFormer parameter overhead relative to RoPE2D baseline. PE params are additional learnable parameters in the positional encoding; V rotation adds no new parameters. ImageNet ViT-S training times on 2× H100 PCIe, 300 epochs (unoptimized rotation ops). Variant Extra PE params Wall-clock (ImageNet) RoPE2D (baseline) 0 ∼72 72h JoFormer-fixed (V rot.) 0 ∼94 94h JoFormer-learned (axial) 2nheads×dhead/22n_heads× d_head/2 per layer ∼94 94h JoFormer-projected MLP: d→d→d/2d→d→d/2 per layer — The ImageNet ViT-S training overhead reflects unoptimized rotation ops in a standard ViT pipeline. JoFormer-projected was tested only on Wikipedia LM (not ImageNet); its MLP projector adds O(d2)O(d^2) parameters per layer. Appendix F Architecture Variants and Value-Path Hierarchy F.1 Variant-Scope Table Table 8: Which theorem supports which architecture variant. Only fixed and learned linear-position variants inherit the relative-displacement theorem; projected angles are a separate, theory-motivated architecture. Variant What is proved What is not proved JoFormer-fixed Covered by Theorem˜2.8 when weights are relative-displacement functions. Value operator depends only on displacement (arises from composition). Does not prove a performance gain. JoFormer-learned Same as fixed if angles remain linear in position with learned frequencies. Frequencies are learned, but the theorem does not choose optimal frequencies. JoFormer-projected Still uses commuting block-diagonal rotations and norm-preserving value transport. Not a translation-equivariant relative-displacement PE, because angles depend on content/residual state. It is an architecture motivated by the value-path view, not a direct corollary of Theorem˜2.8. F.2 Value-Path Hierarchy and SSM Bridge The journey operator on the value side (8) gives the attention value-path formula: k=∑jαkjPj→kvj.c_k= _j _kj\,P_j→ k\,v_j. (3) For SSMs, the value-path kernel is oriented in the forward recurrence direction. This kernel is not necessarily the same object as the attention journey Pj→k=Ak−1AjP_j→ k=A_k^-1A_j unless one adapts the address convention (e.g., At=R−tA_t=R^-t). We therefore write SSM forward kernels as Kj→kK_j→ k in this subsection to distinguish them from the attention journey P. The value-path formula reveals a hierarchy: (a) Standard Transformer: Pj→k=IP_j→ k=I for all j,kj,k. Values are aggregated without positional modulation—the model is order-blind on the value side. (b) Linear SSM (S4 [Gu et al., 2022]): Kj→k=Rk−jK_j→ k=R^k-j. Values carry position-dependent phase shifts, enabling the recurrence ht=Rht−1+vth_t=Rh_t-1+v_t. (c) Selective SSM (Mamba [Gu and Dao, 2023]): Kj→k=∏t=j+1kAxtK_j→ k= _t=j+1^kA_x_t. Each transition depends on input content. (d) JoFormer (attention journey): Pj→k=Ak−1AjP_j→ k=A_k^-1A_j with full attention weights αkj _kj. Combines content-dependent value paths with selective attention. Proposition F.1 (SSM recurrences in value-path notation). Mamba’s selective recurrence ht=Axtht−1+Bxtxth_t=A_x_th_t-1+B_x_tx_t unrolls to: hT=∑t=1T(∏s=t+1TAxs)Bxtxth_T= _t=1^T ( _s=t+1^TA_x_s )B_x_tx_t. Defining the forward kernel Kt→T=∏s=t+1TAxsK_t→ T= _s=t+1^TA_x_s and vt=Bxtxtv_t=B_x_tx_t, this gives the value-path formula hT=∑tKt→Tvth_T= _tK_t→ T\,v_t with uniform attention weights. When Axt∈SO(2)d/2A_x_t (2)^d/2, the prefix products are block-diagonal rotations computable via parallel scan in O(logN)O( N) depth. Note: Mamba’s actual AxtA_x_t matrices are not generally in SO(2)d/2SO(2)^d/2, and attention weights are normalized (unlike the uniform weights here). The forward kernel Kt→TK_t→ T and the attention journey Pt→T=AT−1AtP_t→ T=A_T^-1A_t coincide when At=R−tA_t=R^-t; otherwise they differ by a sign convention. This embedding highlights structural similarities but is not a full equivalence. JoFormer uses the attention journey Pj→i=Ai−1AjP_j→ i=A_i^-1A_j on values, combining full attention selectivity (the score side) with content-dependent value paths (the value side). It does not implement the full product over intermediate states, but rather a single content-dependent rotation per layer. NeurIPS Paper Checklist 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: The abstract states five contributions (interchange prediction, toral classification, JoFormer architecture, value-path hierarchy, experiments) all substantiated with theorems and experimental results in the paper body. 2. Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Section 6 explicitly lists four limitations: single-seed experiments, three-stage training recipe complexity, stability requirements, and speed penalty. 3. Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: All axioms are explicitly stated (Axioms 1–4). Main theorems have proof sketches in the body and complete proofs in Appendix A. The norm-preservation axiom is stated before the toral classification theorem. 4. Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results? Answer: [Yes] Justification: Appendix E provides complete hyperparameters, architectures, training recipes, hardware, seeds, and stability requirements for all experiments. 5. Open access to data and code Question: Does the paper provide open access to the data and code? Answer: [No] Justification: Code will be released upon acceptance. All experiments use public datasets (CIFAR-100, ImageNet-1K, OpenWebText, MNIST, English Wikipedia). 6. Experimental setting/details Question: Does the paper specify all the training and test details necessary to understand the results? Answer: [Yes] Justification: Full details in Appendix E: optimizer, learning rate, schedule, batch size, augmentation, architecture, epochs/iterations, hardware, seed, and critical stability parameters. 7. Experiment statistical significance Question: Does the paper report error bars or other appropriate statistical significance information? Answer: [No] Justification: Most experiments use a single seed due to computational constraints. We acknowledge this explicitly in Section 6 and frame experiments as controlled tests of algebraic predictions rather than definitive performance comparisons. CIFAR-100 uses deterministic training ensuring exact reproducibility. 8. Experiments compute resources Question: Does the paper provide sufficient information on compute resources? Answer: [Yes] Justification: Hardware specified in Appendix E: NVIDIA H100 PCIe for CIFAR-100 and ImageNet, GPU details for language modeling experiments. 9. Code of ethics Question: Does the research conform with the NeurIPS Code of Ethics? Answer: [Yes] Justification: This is foundational research on positional encoding theory with no direct negative societal applications. 10. Broader impacts Question: Does the paper discuss potential societal impacts? Answer: [N/A] Justification: This paper develops algebraic theory for attention mechanisms—foundational research with no direct path to negative societal impacts. 11. Safeguards Question: Does the paper describe safeguards for responsible release? Answer: [N/A] Justification: No models or datasets with misuse potential are released. 12. Licenses for existing assets Question: Are existing assets properly credited with licenses respected? Answer: [Yes] Justification: CIFAR-100, ImageNet-1K, OpenWebText, MNIST, and Wikipedia are properly cited. LieRE is credited. 13. New assets Question: Are new assets well documented? Answer: [N/A] Justification: No new datasets or models are released in this submission. 14. Crowdsourcing and research with human subjects Question: For crowdsourcing experiments, does the paper include instructions and compensation details? Answer: [N/A] Justification: No human subjects or crowdsourcing involved. 15. Institutional review board (IRB) approvals Question: Does the paper describe potential risks and IRB approvals? Answer: [N/A] Justification: No human subjects research. 16. Declaration of LLM usage Question: Does the paper describe LLM usage if it is a core method component? Answer: [N/A] Justification: LLMs were not used as a component of the research methodology.