Paper deep dive
Causal and Interpretable Structures in LLM Compositional Tasks
Gurbir Arora, Toni J. B. Liu, Jiajun Bao, Raphaël Sarfati, Christopher J. Earls
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/30/2026, 7:41:30 AM
Summary
This paper investigates how Large Language Models (LLMs) represent and process relational information in compositional tasks involving cyclic concepts (months, hours, weekdays, musical notes). Using functional ANOVA decomposition on activation ensembles, the authors identify geometric structures corresponding to pairwise and three-way token interactions. They demonstrate a layerwise progression where intermediate layers (around layer 15) utilize pairwise joint representations (e.g., B-A), while later layers (around layer 18) utilize three-way joint representations (e.g., C+B-A) to predict the next token. Causal interventions (ablation and replacement) confirm that these specific interaction terms are causally relevant at their respective depths. Restricting models to only causally relevant joint representations improves next-token prediction accuracy.
Entities (11)
Relation Signals (7)
LLaMA-3.1-8B → usedinstudy → Functional ANOVA Decomposition
confidence 98% · In the main text, all results are reported using base Llama-3.1-8B... We use the functional analysis of variance (ANOVA) decomposition
Pairwise Interaction → geometricallyorganizedby → Pairwise Difference (B-A)
confidence 96% · H_AB^r gets organized by gamma in early layers... gamma=B-A
Three-way Interaction → geometricallyorganizedby → Answer (C+B-A)
confidence 96% · H_ABC^r becomes organized by D at layer 18... D=C+gamma
Pairwise Interaction → causallyrelevantatlayer → 15-17
confidence 95% · ablating H_AB^r produces the largest decrease in top-3 accuracy around layers 15–17
Three-way Interaction → causallyrelevantatlayer → 17-18
confidence 95% · ablating H_ABC^r sharply decreases accuracy after layer 17... transition... around layers 17–18
Ensemble Ablation → demonstrates → Causal Relevance
confidence 94% · Causal interventions and activation patching have been used... to test the functional role
Restricting to Causally Relevant Representations → improves → Next-Token Prediction Accuracy
confidence 93% · restricting the models to such causally relevant joint representations improves next-token prediction accuracy.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy.
Tags
Links
- Source: https://arxiv.org/abs/2609.35970v1
- Canonical: https://arxiv.org/abs/2609.35970v1
Trouble viewing inline? Open PDF directly →
Full Text
95,924 characters extracted from source content.
Expand or collapse full text
Causal and Interpretable Structures in LLM Compositional Tasks Gurbir Arora Affiliation: Cornell University, USA Toni J.B. Liu Affiliation: Cornell University, USA Jiajun Bao Affiliation: Cornell University, USA Raphaël Sarfati Christopher J. Earls Affiliation: Goodfire AI, USA Correspondence:g.arora@cornell.edu Abstract Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy. 1 Introduction Many tasks require a large language model (LLM) to use relationships between multiple input tokens. Consider the prompt, “Hi John, the conference is scheduled between January 13 and March 13. This is the same time as between May 13 and”. The correct next-token prediction depends crucially on three tokens: January, March, and May. Let A, B, and C denote these three tokens, respectively, and define γ=B−A(mod 12)γ=B-A\;(mod\;12). The correct answer is then D=C+γ(mod 12)D=C+γ\;(mod\;12). The individual representations of A, B, and C lie on the low-dimensional manifold associated with calendar months (Engels et al., 2025). We ask whether computationally relevant quantities arising jointly from these tokens, such as γ or D, also have geometric structure, and whether that structure is used in the next-token prediction. We vary A,B,CA,B,C independently over calendar months while keeping everything else fixed. Across this ensemble of prompts, the last-token activation varies with A, B, and C individually and with the particular combinations in which they co-occur. These sources of variation are superposed in the full activation ensemble. We use the functional analysis of variance (ANOVA) decomposition to separate the resulting activation ensemble into terms associated with individual variables, pairwise interactions, and their three-way interaction (Hoeffding, 1948). We treat the interaction terms as geometric objects and study their organization and causal role across the layers of an LLM. We study this question across several cyclic concepts: months, hours, weekdays, and musical notes. The cyclic structure gives us known relational quantities. In particular, γ=B−Aγ=B-A is a pairwise relation, while the answer D=C+γD=C+γ depends jointly on all three inputs. Importantly, γ is never supplied explicitly and must instead be inferred from A and B. This allows us to ask how the joint dependence associated with γ and D is represented and used across network depth. Our contributions are: (a) Geometric structure. We find that the pairwise interaction between A and B is organized by γ in the middle layers. Other pairwise interactions are also organized by their corresponding pairwise differences. The three-way interaction between A, B, and C becomes organized by D at a later depth. (b) Causality. We show that the representation associated with A,BA,B is causally relevant in the middle layers, while the representation associated with A,B,CA,B,C becomes causally relevant at later layers. (c) Improved performance. Retaining only the causally relevant three-way interaction improves the performance on the task. (d) Relationship transfer. We show that the relationship encoded by interaction terms transfers across cyclic domains, such as from hours to months. 1.1 Related work Geometry of concepts. Prior work has identified low-dimensional geometric structure associated with individual concepts in latent activations (Engels et al., 2025; Modell et al., 2025; Park et al., 2025; Karkada et al., 2026; Hu et al., 2026; Prieto et al., 2026). Other work has studied how representational geometry evolves during computation and how interventions along that geometry affect model behavior (Gurnee et al., 2025; Wurgaft et al., 2026; Sarfati et al., 2026). We instead study the geometry and causality of joint dependence between multiple tokens. Figure 1: Two-dimensional visualization using the first two principal components for the activation ensemble Φr ^r and its decomposed interaction ensembles HABrH_AB^r and HABCrH_ABC^r for the prompt template in Section 2.1 using Llama-3.1-8B. Here, γ=B−Amod12γ=B-A 12 and the expected answer is D=C+γmod12D=C+γ 12. Before fitting the principal components, we exclude prompts satisfying A=BA=B, B=CB=C, or C=AC=A (see Appendix B.3). The bottom left of each panel shows the coloring according to either γ or D. HABrH_AB^r gets organized by γ in early layers. Until layer 17, HABCrH_ABC^r is unstructured but becomes organized by D at layer 18. See Section 3 for a quantitative analysis. Relational representations and arithmetic computation. Prior work has shown that latent activations contain structured representations of relations between inputs, including approximately linear transformations, as well as distributed representations that bind entities or variables to relational roles (Hernandez et al., 2024; Merullo et al., 2024; Feng and Steinhardt, 2024; Davies et al., 2023; Dai et al., 2026; Todd et al., 2026; Wang et al., 2024). Related work on arithmetic has identified structured numerical representations and computational mechanisms, including Fourier and periodic structure in transformers trained on modular arithmetic and in pretrained language models (Nanda et al., 2023; Furuta et al., 2024; Zhou et al., 2024; Kantamneni and Tegmark, 2025; Levy and Geva, 2025). A closely related work is that of Feucht et al. (2026), who study tasks involving an explicit offset γ and a concept C from a cyclic domain. They identify a universal base-10 addition mechanism in Llama-3.1-8B by characterizing MLP neurons associated with the task’s Fourier structure. In contrast, we work at the representation level. The relation γ in our setting is not supplied explicitly, but arises from the joint dependence of two independently varied inputs. We isolate the component of the residual-stream representation associated with this joint dependence and study its geometry and causal role across layers. Causal interventions. The presence of structured or decodable information in an activation does not by itself establish that the model uses that information to perform the task. Causal interventions and activation patching have been used in several works to test the functional role of such representations (Vig et al., 2020; Geiger et al., 2021; Geiger et al., 2024; Zhang and Nanda, 2024; Arora et al., 2024). Our experiments similarly intervene directly on joint representations of multiple input tokens. 2 Setup In the main text, all results are reported using base Llama-3.1-8B (Llama Team, 2024b), which contains 32 layers indexed by ℓ∈0,…,31 ∈\0,…,31\. In the appendices, we reproduce all our results on base models: Llama-3.2-3B (Llama Team, 2024a), Qwen-2.5-7B (Qwen Team, 2024), Qwen-3-8B (Qwen Team, 2025), Gemma-2-9B (Gemma Team, 2024), Gemma-3-12B (Gemma Team, 2025), Mistral-small-24B-Base-2501 (Mistral AI Team, 2025). In each model, we analyze residual-stream activations after every transformer block. 2.1 Construction of prompts Several other works have used controlled prompt variations to reveal manifolds associated with individual concepts (Engels et al., 2025; Kantamneni and Tegmark, 2025). We propose a similar experimental design, but we aim to elicit a relationship between concepts. We consider several cyclic concepts—months, weekdays, hours, musical notes—and let X be the associated vocabulary. In the main text, we discuss the months domain, :=January,…,DecemberX:=\January,…,December\. Consider the variables (A,B,C)∈3(A,B,C) ^3 along with the promptHi name, the noun is scheduled between A dates and B dates. This is the same time as between C dates and Here, name is chosen from the most common names of the past century (Social Security Administration, n.d.), noun varies over synonyms of conference, while dates ∈1,…,28∈\1,…,28\ (see Appendix A.1 for a list). We call each combination of (,,)( |name|, |noun|, |dates|) a replicate and label it by r. These replicates define prompt variations that leave the underlying task unchanged. We further require that all replicate variables tokenize to the same length so that positional offsets do not introduce a confound. We randomly subsample 10 such replicates. Unless stated otherwise, we average over the replicates when we report any quantity. We design prompts associated with each cyclic concept such that the answer to the task is the next token. We measure the performance by computing the top-11 and top-33 accuracy, where top-k accuracy is the fraction of prompts for which the correct answer features in the k largest logits. The top-11 accuracy of Llama-3.1-8B is 62.2%±5.1%62.2\%± 5.1\%, and the top-33 accuracy is 85.4%±3.3%85.4\%± 3.3\%, where the variation is reported as the standard deviation across replicates r11 1 The accuracies are computed over all prompts with distinct variables (A,B,C)(A,B,C). See Appendix B.3.. Because top-33 accuracy varies less with larger offsets of γ=B−A≳6γ=B-A 6 than top-11 accuracy (Appendix A), we use top-33 accuracy as the primary performance metric in the main text. The analogous results using top-11 accuracy are reported in the appendices. 2.2 Ensemble decomposition Let ϕℓr(A,B,C)∈ℝdmodel φ^r_ (A,B,C) ^d_model denote the last-token activation at layer ℓ for replicate r. We define the corresponding activation ensemble as Φℓr:=(ϕℓr(A,B,C))(A,B,C)∈3. ^r_ := ( φ^r_ (A,B,C) )_(A,B,C) ^3. (1) Since we report all quantities as a function of layer, we drop the label ℓ for brevity. We decompose each activation as ϕr(A,B,C)=r⏟mean+Ar+Br+Cr⏟additive terms+ABr+BCr+CAr+ABCr⏟interaction terms, φ^r(A,B,C)= μ^r_mean+ h_A^r+ h_B^r+ h_C^r_additive terms+ h_AB^r+ h_BC^r+ h_CA^r+ h_ABC^r_interaction terms, (2) where every term is defined as an average with respect to the ensemble Φr ^r: r μ^r :=ABC(ϕr), :=E_ABC( φ^r), (3) Ar h_A^r :=BC(ϕr)−r, :=E_BC( φ^r)- μ^r, ABr h_AB^r :=C(ϕr)−r−Ar−Br, :=E_C( φ^r)- μ^r- h_A^r- h_B^r, ABCr h_ABC^r :=ϕr−r−Ar−Br−Cr−ABr−BCr−CAr. := φ^r- μ^r- h_A^r- h_B^r- h_C^r- h_AB^r- h_BC^r- h_CA^r. Analogously to Equation 1, we define the ensemble associated with decomposed terms as HAr=(Ar)A∈,HABr=(ABr)(A,B)∈2,andHABCr=(ABCr)(A,B,C)∈3.H_A^r= ( h_A^r )_A , H_AB^r= ( h_AB^r )_(A,B) ^2, H_ABC^r= ( h_ABC^r )_(A,B,C) ^3. The remaining terms are defined by cyclic permutation of the expressions above. Intuitively, r μ^r is the center of the activation ensemble, HArH_A^r captures the dependence on A alone, HABrH_AB^r captures the joint dependence on A and B that remains after averaging over C and removing their individual effects, and HABCrH_ABC^r contains the residual three-way dependence after accounting for all lower-order terms. We refer to HABrH_AB^r, HBCrH_BC^r, HCArH_CA^r, and HABCrH_ABC^r as the interaction ensembles. We emphasize that Equation 2 is an exact identity rather than an approximation. It is known as the functional ANOVA decomposition (Hoeffding, 1948; Owen, 2013). The decomposed components are pairwise orthogonal under the ensemble-averaged inner product (Appendix B). We do not apply the inferential machinery of ANOVA, since the associated questions of statistical significance are orthogonal to the goal of this paper. Figure 1 shows the first two principal components of Φr ^r, HABrH_AB^r, and HABCrH_ABC^r. The remainder of the paper analyzes the interaction ensembles as geometric objects to study their structure and causal role across network depth. 3 Geometric structure of interaction terms The ANOVA decomposition described in Equation 2 breaks the ensemble into its constituent parts. Figure 1 visually suggests that HABrH_AB^r can be parametrized by γ=B−Aγ=B-A and similarly HABCrH_ABC^r can be parametrized by D=C+γD=C+γ. In this section, we first rigorously quantify the dependence of interaction terms on γ or D by exploiting the cyclic nature of A,B,CA,B,C using the discrete Fourier transform in Section 3.1. Second, we analyze the norm of vectors in each interaction ensemble in Section 3.2 to check for ensembles that are highly organized but have negligible norm overall. 3.1 Fourier Transform We perform a discrete Fourier transform (DFT) of every interaction term for every replicate r. Consider HABrH_AB^r as an example. Its DFT is given by ^kAkBr=∑A,BrABe−2πi(kAA+kBB)/12, h_k_Ak_B^r= _A,B h^r_ABe^-2π i(k_AA+k_BB)/12, (4) where we identify the months with A,B∈0,…,11A,B∈\0,…,11\ and kA,kBk_A,k_B are the associated Fourier modes. Let γ=B−Aγ=B-A. If ABr h_AB^r depends only on γ, that is, ABr≡(γ) h_AB^r≡ f(γ), then its DFT ^kAkBr=∑γ(γ)e−2πikBγ/12∑Ae−2πi(kA+kB)A/12 h_k_Ak_B^r= _γ f(γ)e^-2π ik_Bγ/12 _Ae^-2π i(k_A+k_B)A/12 vanishes unless kA+kB=0mod12k_A+k_B=0 12. Thus concentration on modes satisfying kA=−kBk_A=-k_B22 2 Strictly, the condition is kA+kB=0mod 12k_A+k_B=0\;mod\;12; all frequency equalities are understood modulo 12. quantifies the extent to which HABrH_AB^r is organized according to γ. Therefore we define mode fraction as m(HABr):=∑kA=−kB‖^kAkBr‖22∑kA,kB‖^kAkBr‖22.m(H_AB^r):= _k_A=-k_B h^r_k_Ak_B _2^2 _k_A,k_B h^r_k_Ak_B _2^2. (5) The mode fraction m(HABr)∈[0,1]m(H^r_AB)∈[0,1] measures the degree of organization of HABrH_AB^r by γ, with m=1m=1 when the ensemble depends on γ alone. We define an analogous mode fraction for HBCrH_BC^r, HCArH_CA^r, and HABCrH_ABC^r to probe for their dependence on γ′=C−Bγ =C-B, γ′=A−Cγ =A-C, and D=C+B−AD=C+B-A respectively. For the latter, the modes under investigation are (kA,kB,kC)=(−k,k,k)(k_A,k_B,k_C)=(-k,k,k). Dependence on γ cannot occur in terms lower-order than HABrH_AB^r, and dependence on D cannot occur in terms lower-order than HABCrH_ABC^r. Thus, if the residual stream contains structure organized by γ or D, that structure must reside in HABrH_AB^r or HABCrH_ABC^r, respectively. Consistently, we find large mode fractions for both interaction ensembles but at different depths. Importantly, we also find that all second-order interaction terms are strongly organized by their corresponding pairwise differences, and that HABCrH_ABC^r undergoes a sharp increase in organization by D around layers 17–18 (Figure 2(a)). At the same depth, the mode fractions of HBCrH_BC^r and HCArH_CA^r begin to decrease, while m(HABr)m(H_AB^r) remains large throughout the depth of the transformer. Results across models and domains appear in Appendix C. Control experiments detailed in Appendix E suggest that the pairwise-difference organization of the second-order interaction ensembles can arise from the co-occurrence of the corresponding tokens. This is in agreement with Karkada et al. (2026). In contrast, HABCrH_ABC^r is not organized by D when the task structure is removed from the prompt, even when all three tokens A,B,CA,B,C are present. Figure 2: Mode fraction and energy fraction of the interaction ensembles across layers of Llama-3.1-8B. We zero out vectors for which A, B, and C are not pairwise distinct before computing both fractions (Appendix B.3). (a) Mode fraction measures the organization of HABrH_AB^r, HBCrH_BC^r, and HCArH_CA^r by their corresponding pairwise differences, and of HABCrH_ABC^r by D=C+B−AD=C+B-A. Under a random null model, the expected mode fraction for second-order interactions is 0.090.09 and that for third-order interaction is 0.0080.008. (b) Energy fraction measures the fraction of mean-subtracted activation energy attributable to each interaction ensemble. Throughout this work, colored bands show the central 80% interval across 10 replicates, and vertical gray bands highlight the layers discussed in the text. 3.2 Energy fraction An ensemble can have a high mode fraction even if all of its vectors are close to zero. Directly comparing the norms of interaction terms across layers is misleading because ϕr φ^r grows substantially with depth, with most of this growth coming from the mean r μ^r. Therefore we need a measure of relative magnitude for each ensemble across layers. For S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\, we define energy fraction e(HSr)e(H^r_S) as33 3 See Appendix D for a relation between ANOVA decomposition and our definition of energy fraction. e(HSr)=S(‖Sr‖22)ABC(‖ϕr(A,B,C)−r‖22)e(H_S^r)= E_S ( h^r_S _2^2 )E_ABC ( φ^r(A,B,C)- μ^r _2^2 ) (6) Figure 2(b) shows the energy fraction of each interaction ensemble. The energy fraction of HABrH_AB^r peaks at layer 15 and starts decreasing around layers 17–18. Around the same depth, the energy fraction of HABCrH_ABC^r increases dramatically. This coincides with the sharp increase in organization of HABCrH_ABC^r by D, as found in the previous Section 3.1. Together, these observations hint at a transition from a pairwise representation of the input relation B−AB-A to a three-way representation aligned with the answer C+B−AC+B-A, which we test causally in the following section. 4 Causal use of interaction terms The previous section establishes that the interaction ensembles are highly organized by pairwise differences or by D, depending on the interaction ensemble and the layer. To test whether these ensembles are used in downstream computation, we intervene directly on the activation of the last token (Zhang and Nanda, 2024) using its ANOVA decomposition. In the first set of experiments in Section 4.1, we intervene on the last token at a particular layer to remove an interaction term. In Section 4.2, we instead remove all terms except for the mean and an interaction term. After each intervention, we let the model run without any further modification and score the performance. Figure 3: Top row shows the result of ensemble ablation experiments, while the bottom row shows corresponding ensemble replacement experiments. The horizontal dashed line shows the baseline performance. Dotted line at 25% is the chance of getting the correct answer under a uniform prior. (a) Ablating HABrH_AB^r produces the largest decrease in top-3 accuracy around layers 15–17, whereas ablating HABCrH_ABC^r sharply decreases accuracy after layer 17. (b) Ablating HBCrH_BC^r or HCArH_CA^r has little effect on performance across model depth. (c) Replacement experiments show a complementary transition where retaining HABrH_AB^r preserves performance in the middle layers, while retaining HABCrH_ABC^r preserves performance in later layers. (d) Retaining HBCrH_BC^r or HCArH_CA^r sharply decreases performance from layer 15 onward. 4.1 Ensemble Ablations We define ensemble ablation of an interaction ensemble HSrH_S^r by subtracting the corresponding interaction vector Sr h_S^r from the activation of the last token: ϕ~r(A,B,C)=ϕr(A,B,C)−Sr, φ^r(A,B,C)= φ^r(A,B,C)- h^r_S, (7) for all A,B,C∈A,B,C and S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\. Figure 3 shows the results of ensemble ablation experiments for every interaction term. We see that ablating HBCrH_BC^r or HCArH_CA^r barely changes the top-33 accuracy relative to the baseline (Figure 3(b)). In contrast, ablating HABrH_AB^r or HABCrH_ABC^r causes a drastic change in the model’s performance, but at layers 15 and 18 respectively (Figure 3(a)). 4.2 Ensemble Replacement In a complementary set of experiments, we perform ensemble replacement by retaining a single interaction ensemble together with the mean. Concretely, we replace the activation of the last token with ϕ~r(A,B,C)=r+Sr, φ^r(A,B,C)= μ^r+ h^r_S, (8) for all A,B,C∈A,B,C and S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\. Figure 3(d) shows that retaining HBCrH_BC^r or HCArH_CA^r alone induces a sharp decrease in top-3 accuracy around layers 14–15. However, Figure 3(c) shows that retaining HABrH_AB^r preserves performance relative to the unmodified baseline until around layer 17, after which performance decreases sharply. Because we intervene only at the last-token position and at a particular layer, the model may recompute task-relevant information downstream of the intervention. We therefore also include a r μ^r-only baseline to measure this recovery. Before layer 17, retaining HABCrH_ABC^r gives performance similar to the r μ^r-only baseline. After layer 17, however, retaining HABCrH_ABC^r substantially improves performance over the r μ^r-only baseline and even exceeds the unmodified baseline. Together, ensemble ablation and replacement experiments show a transition in causal relevance from HABrH_AB^r to HABCrH_ABC^r around layers 17–18. Between layers 15–17, ablating HABrH_AB^r causes a sharp decrease in performance while retaining HABrH_AB^r preserves it. The same aforementioned properties hold for HABCrH_ABC^r starting around layers 17–18. This transition occurs at the same depth where Section 3 finds that HABCrH_ABC^r becomes organized according to the answer D, the energy fraction of HABrH_AB^r starts decreasing, and the energy fraction of HABCrH_ABC^r rises sharply. Although HBCrH_BC^r and HCArH_CA^r are also strongly organized by their corresponding pairwise differences (Section 3), ablating them has little effect on the performance, and retaining them does not reproduce the layerwise replacement behavior of HABrH_AB^r. Additional ablation and replacement experiments for different models and domains are in Appendices G and H, respectively. Appendix F further shows that ablating HABrH_AB^r at layer 15 prevents the later emergence of HABCrH_ABC^r. 5 Steering So far, we have shown that the ensembles HABrH_AB^r and HABCrH_ABC^r are causally relevant, but this does not tell us if the model uses their organization according to γ or D. If the model indeed uses this organization, then changing γ→γ+δγ→γ+δ should shift the predicted answer by δ, that is, D→D+δD→ D+δ. In this section, we test this by steering along γ and D. There are multiple ways to steer using HABrH_AB^r: shifting B→B+δB→ B+δ such that ABr→A,B+δr h_AB^r→ h_A,B+δ^r, shifting A→A−δA→ A-δ such that ABr→A−δ,Br h_AB^r→ h_A-δ,B^r, or using an averaged vector that depends only on γ. Results using a combination of the first two strategies are reported in Appendix I. Here, we describe the last strategy. We define an averaged vector ¯γr h^r_γ by44 4 The chosen steering vector ¯γr h^r_γ is closely related to the definition of the mode fraction in Equation 5. We show this correspondence in Appendix B.1. ¯γr:=AB−A=γABABr. h^r_γ:= E_AB_B-A=γ h_AB^r. (9) To steer, we replace the activation ϕr φ^r with ϕ~r φ^r according to ϕ~r(A,B,C)=ϕr(A,B,C)+α(¯γ+δr−¯γr). φ^r(A,B,C)= φ^r(A,B,C)+α ( h^r_γ+δ- h^r_γ ). (10) Figure 4: Steering using different vectors as a function of layer at α=1α=1. The median is evaluated among all the prompts for each layer and δ. (a) We see that steering using ¯γ h_γ is most effective between layers 15–17. (b) In contrast, steering using ¯D h_D becomes effective from layer 18 onward. (c) Top-3 accuracy for δ=4δ=4 after steering. The dashed line shows the unmodified model’s top-3 accuracy with respect to the target answer D+δD+δ. Similarly, we define the steering vector ¯D h_D for the ensemble HABCrH_ABC^r as ¯Dr:=ABC+B−A=DABCABCr h^r_D:= E_ABC_C+B-A=D h_ABC^r (11) and steer using Equation 10 after replacing ¯γr h^r_γ with ¯Dr h^r_D. The steering method is equivalent to the difference-of-means steering, analogous to that of Rimsky et al. (2024) and Subramani et al. (2022) (see Appendix B.2). Let D∗=D+δD^*=D+δ be the target answer. To measure the effect of steering, we compute the top-3 logit margin m3m_3, defined as the difference between the target logit zD∗z_D^* and the third-largest non-target logit z3z_3: m3=zD∗−z3.m_3=z_D^*-z_3. (12) Thus m3≥0m_3≥ 0 iff the target logit is in top-3. Finally, we take the difference between steered and baseline m3m_3 to get Δm3=m3steered−m3baseline. m_3=m_3^steered-m_3^baseline. (13) In Figures 4(a–b), we report medianΔm3median\; m_3 for steering using ¯γr h_γ^r and ¯Dr h_D^r at α=1α=1, and Figure 4(c) shows the top-3 accuracy after steering for δ=4δ=4. Steering using ¯γr h^r_γ becomes effective around layers 14–15, but around layers 17–18, its effect decreases and steering using ¯Dr h^r_D becomes effective. 6 Cross-domain transfer Our preceding experiments show that HABrH_AB^r is causally relevant in the middle layers while HABCrH_ABC^r is relevant in the later layers. We now want to understand whether one interaction vector extracted from one context can substitute for the corresponding vector in another context performing the same underlying task. Prior work has shown that relational and task-level representations can be reused across contexts (Wang et al., 2024; Todd et al., 2024). We focus on the interaction ensemble HABrH_AB^r here; the performance of transplanting HABCrH_ABC^r varies across model and domain combinations. Appendix J shows that HABCrH_ABC^r can be transplanted across domains when their subspaces are aligned. We will use domain-1 to refer to the original set of prompts that we have discussed so far and we use domain-2 for the following set of prompts:Hi name, the noun is scheduled between day A o’clock and day B o’clock. This is the same time as between day C o’clock and day where A,B,C∈(2)=1,…,12A,B,C ^(2)=\1,…,12\ and (name, noun, day) are replicate variables (Appendix A.1). We then decompose the domain-2 ensemble according to Section 2.2. We transplant an interaction ensemble by replacing each domain-1 interaction vector with the corresponding domain-2 interaction vector indexed by the same values of the underlying cyclic variables. We consider ϕ~r1,(1)(A,B,C)=ϕr1,(1)(A,B,C)−ABr1,(1)+ABr2,(2), φ^r_1,(1)(A,B,C)= φ^r_1,(1)(A,B,C)- h^r_1,(1)_AB+ h^r_2,(2)_AB, (14) where superscripts (1),(2)(1),(2) refer to the domain and r1,r2r_1,r_2 refer to the corresponding replicate. Analogous to the ensemble replacement experiments in Section 4.2, we also consider ϕ~r1,(1)(A,B,C)=r1,(1)+ABr2,(2). φ^r_1,(1)(A,B,C)= μ^r_1,(1)+ h_AB^r_2,(2). (15) In both cases, the model runs unmodified after the intervention and we score relative to domain-1. Figure 5(a) shows that replacing an ablated domain-1 interaction ensemble with the corresponding domain-2 ensemble restores the performance lost under ablation. Figure 5(b) shows that the domain-2 interaction ensemble also reproduces the layerwise trend observed in the replacement experiment in Section 4.2. Figure 5: Shaded bands show the central 80% interval across 10 randomly sampled replicate pairs (r1,r2)(r_1,r_2). (a) Transplanting the interaction ensemble HAB(2)H_AB^(2) from domain-2 to domain-1 preserves performance. (b) Transplanting and retaining only the domain-1 mean (1) μ^(1) and domain-2 interaction ensemble HAB(2)H^(2)_AB also preserves performance compared to the dashed blue within-domain replacement experiments. (c) Cross-domain steering using ¯γ+δr2,(2)−¯γr1,(1) h_γ+δ^r_2,(2)- h_γ^r_1,(1) at α=1α=1. We next ask whether the organization according to γ also transfers across domains. Using the steering vector defined in Equation 9, we steer using the difference between the domain-1 vector for γ and the domain-2 vector for γ+δγ+δ: ϕ~r1,(1)(A,B,C)=ϕr1,(1)(A,B,C)+α(¯γ+δr2,(2)−¯γr1,(1)). φ^r_1,(1)(A,B,C)= φ^r_1,(1)(A,B,C)+α ( h_γ+δ^r_2,(2)- h_γ^r_1,(1) ). (16) If the organization by γ transfers between two domains, then the answer should shift accordingly. Figure 5(c) shows that steering vectors constructed across the two domains produce a steering pattern similar to that in Section 5. 7 Discussion We started by asking whether it is possible to separate variation due to individual concepts from variation arising through their joint dependence in an activation ensemble. We showed that using the functional ANOVA decomposition is a simple yet powerful technique to accomplish this task. The resulting interaction ensembles exhibit clear geometric structure, but not all geometrically organized ensembles are causally necessary for the task. For the ensembles that are causally relevant, their organization can be exploited to predictably steer the model. We also found two results that we did not anticipate. First, retaining only the mean and the third-order interaction can outperform the unmodified model. Our interventions show that HABCrH_ABC^r is used by the LLM to output the correct answer, while other interaction ensembles either are unused or perform a different function. Therefore, it may happen that those other interaction ensembles interfere destructively in the computation in the last few layers. Second, transplanting HABH_AB between domains that perform the same underlying task recovers much of the performance lost under ablation. This suggests that the intermediate representation of the inferred relation is sufficiently compatible across domains to support downstream computation. Neither observation would have been accessible in the full activation ensemble without first isolating the joint dependence from the remaining components. Our results also suggest a possible connection to the neuron-level analysis of Feucht et al. (2026). They identify a small set of MLP neurons at layer 18 that are associated with Fourier components of the arithmetic representation in Llama-3.1-8B. In our analysis, the same depth is where the three-way interaction HABCrH_ABC^r sharply increases in energy fraction, becomes organized by D, and becomes causally relevant. While we do not analyze individual neurons here, the interaction decomposition may provide a way to identify the layers in which such structure emerges and may also provide a model-agnostic starting point for neuron-level analyses. Limitations Scope. We have studied a controlled setting with three variables A,B,CA,B,C that are drawn from the same cyclic concept whose composition has a definitive arithmetic answer. A natural extension is to allow A,B,CA,B,C to vary over different, potentially non-cyclic concepts whose composition may not have a well-defined answer. Exponential scaling. As we increase the number of variables p, the number of prompts required to furnish the ANOVA decomposition scales exponentially ∼||p |X|^p. Acknowledgments This research is funded in part by the Gordon and Betty Moore Foundation through Grant GBMF13901 to Cornell to support the work of G.A. AI use statement We have used generative AI (GPT-5.6 and GPT-6) to code, proofread the paper to improve readability, correct typos and grammatical errors. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. References Arora et al. (2024) A. Arora, D. Jurafsky, and C. Potts CausalGym: benchmarking causal interpretability methods on linguistic tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 14638–14663. External Links: Link, Document Cited by: §1.1. Dai et al. (2026) Q. Dai, B. Heinzerling, and K. Inui Cell-based representation of relational binding in language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), M. Liakata, V. P. Moreira, J. Zhang, and D. Jurgens (Eds.), San Diego, California, United States, p. 47464–47524. External Links: Link, Document, ISBN 979-8-89176-390-6 Cited by: §1.1. Davies et al. (2023) X. Davies, M. Nadeau, N. Prakash, T. R. Shaham, and D. Bau Discovering variable binding circuitry with desiderata. External Links: 2307.03637, Link Cited by: §1.1. Engels et al. (2025) J. Engels, E. J. Michaud, I. Liao, W. Gurnee, and M. Tegmark Not all language model features are one-dimensionally linear. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1.1, §1, §2.1. Feng and Steinhardt (2024) J. Feng and J. Steinhardt How do language models bind entities in context?. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1.1. Feucht et al. (2026) S. Feucht, T. Haklay, U. Bhalla, D. Wurgaft, C. Rager, R. Sarfati, J. Merullo, T. McGrath, O. Lewis, E. S. Lubana, T. Fel, and A. Geiger Arithmetic in the wild: llama uses base-10 addition to reason about cyclic concepts. External Links: 2605.01148, Link Cited by: §1.1, §7. Furuta et al. (2024) H. Furuta, G. Minegishi, Y. Iwasawa, and Y. Matsuo Towards empirical interpretation of internal circuits and properties in grokked transformers on modular polynomials. External Links: 2402.16726, Link Cited by: §1.1. Geiger et al. (2021) A. Geiger, H. Lu, T. Icard, and C. Potts Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. W. Vaughan (Eds.), Vol. 34, p. 9574–9586. External Links: Link Cited by: §1.1. Geiger et al. (2024) A. Geiger, Z. Wu, C. Potts, T. Icard, and N. D. Goodman Finding alignments between interpretable causal variables and distributed neural representations. External Links: 2303.02536, Link Cited by: §1.1. Gemma Team (2024) Gemma Team Gemma 2: improving open language models at a practical size. External Links: 2408.00118, Link Cited by: §2. Gemma Team (2025) Gemma Team Gemma 3 technical report. Kaggle. External Links: Link Cited by: §2. Gurnee et al. (2025) W. Gurnee, E. Ameisen, I. Kauvar, J. Tarng, A. Pearce, C. Olah, and J. Batson When models manipulate manifolds: the geometry of a counting task. Transformer Circuits Thread. External Links: Link Cited by: §1.1. Hernandez et al. (2024) E. Hernandez, A. S. Sharma, T. Haklay, K. Meng, M. Wattenberg, J. Andreas, Y. Belinkov, and D. Bau Linearity of relation decoding in transformer language models. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1.1. Hoeffding (1948) W. Hoeffding A Class of Statistics with Asymptotically Normal Distribution. The Annals of Mathematical Statistics 19 (3), p. 293 – 325. External Links: Document, Link Cited by: §1, §2.2. Hu et al. (2026) Z. Hu, L. Niu, and S. Varma Language models represent and transform concepts with shared geometry. In Mechanistic Interpretability Workshop at ICML 2026, External Links: Link Cited by: §1.1. Kantamneni and Tegmark (2025) S. Kantamneni and M. Tegmark Language models use trigonometry to do addition. External Links: 2502.00873, Link Cited by: §1.1, §2.1. Karkada et al. (2026) D. Karkada, D. J. Korchinski, A. Nava, M. Wyart, and Y. Bahri Symmetries in language statistics shape the geometry of model representations. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1.1, §3.1. Levy and Geva (2025) A. A. Levy and M. Geva Language models encode numbers using digit representations in base 10. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, p. 385–395. External Links: Link, Document, ISBN 979-8-89176-190-2 Cited by: §1.1. Llama Team (2024a) Llama Team Llama-3.2-3B. Note: Hugging Face model card External Links: Link Cited by: §2. Llama Team (2024b) Llama Team The llama 3 herd of models. CoRR abs/2407.21783. External Links: Link, Document, 2407.21783 Cited by: §2. Merullo et al. (2024) J. Merullo, C. Eickhoff, and E. Pavlick Language models implement simple Word2Vec-style vector arithmetic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), K. Duh, H. Gomez, and S. Bethard (Eds.), Mexico City, Mexico, p. 5030–5047. External Links: Link, Document Cited by: §1.1. Mistral AI Team (2025) Mistral AI Team Mistral-Small-24B-Base-2501. Note: https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501Model card Cited by: §2. Modell et al. (2025) A. Modell, P. Rubin-Delanchy, and N. Whiteley The origins of representation manifolds in large language models. External Links: 2505.18235, Link Cited by: §1.1. Nanda et al. (2023) N. Nanda, L. Chan, T. Lieberum, J. Smith, and J. Steinhardt Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §1.1. Owen (2013) A. B. Owen Monte carlo theory, methods and examples. https://artowen.su.domains/mc/. Cited by: §2.2. Park et al. (2025) K. Park, Y. J. Choe, Y. Jiang, and V. Veitch The geometry of categorical and hierarchical concepts in large language models. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1.1. Prieto et al. (2026) L. Prieto, E. Stevinson, M. Barsbey, T. Birdal, and P. A. M. Mediano From data statistics to feature geometry: how correlations shape superposition. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1.1. Qwen Team (2024) Qwen Team Qwen2.5: a party of foundation models. External Links: Link Cited by: §2. Qwen Team (2025) Qwen Team Qwen3 technical report. External Links: 2505.09388, Link Cited by: §2. Rimsky et al. (2024) N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 15504–15522. External Links: Link, Document Cited by: §5. Sarfati et al. (2026) R. Sarfati, E. Bigelow, D. Wurgaft, S. Boppana, J. Merullo, A. Geiger, O. Lewis, T. McGrath, and E. S. Lubana The shape of beliefs: geometry, dynamics, and interventions along representation manifolds of language models’ posteriors. External Links: 2602.02315, Link Cited by: §1.1. Social Security Administration (n.d.) Social Security Administration Top names over the last 100 years. Note: Accessed in 2026 External Links: Link Cited by: Table 2, §2.1. Subramani et al. (2022) N. Subramani, N. Suresh, and M. Peters Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, S. Muresan, P. Nakov, and A. Villavicencio (Eds.), Dublin, Ireland, p. 566–581. External Links: Link, Document Cited by: §5. Todd et al. (2026) E. Todd, J. Brinkmann, R. Gandikota, and D. Bau In-context algebra. In International Conference on Learning Representations, C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (Eds.), Vol. 2026, p. 80695–80729. External Links: Link Cited by: §1.1. Todd et al. (2024) E. Todd, M. Li, A. S. Sharma, A. Mueller, B. C. Wallace, and D. Bau Function vectors in large language models. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §6. Vig et al. (2020) J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, Y. Singer, and S. Shieber Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 12388–12401. External Links: Link Cited by: §1.1. Wang et al. (2024) Z. Wang, B. Whyte, and C. Xu Locating and extracting relational concepts in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 4818–4832. External Links: Link, Document Cited by: §1.1, §6. Wurgaft et al. (2026) D. Wurgaft, C. Rager, M. Kowal, S. Feucht, U. Bhalla, T. Haklay, E. Bigelow, R. Sarfati, J. Merullo, N. Goodman, T. Fel, A. Geiger, and E. S. Lubana Manifold steering reveals the shared geometry of neural network representation and behavior. In Third Conference on Language Modeling, External Links: Link Cited by: §1.1. Zhang and Nanda (2024) F. Zhang and N. Nanda Towards best practices of activation patching in language models: metrics and methods. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1.1, §4. Zhou et al. (2024) T. Zhou, D. Fu, V. Sharan, and R. Jia Pre-trained large language models use fourier features to compute addition. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §1.1. Appendix A Models, Prompts, and Capabilities In this work, we evaluate the robustness of our results across seven base models ranging from 3B to 24B parameters: Llama-3.2-3B, Llama-3.1-8B, Qwen-2.5-7B, Qwen-3-8B, Gemma-2-9B, Gemma-3-12B, Mistral-small-24B-Base-2501. Table 1 lists all the prompts for each model, while Table 4 shows their corresponding performances using top-1 and top-3 accuracy as metrics. Here, we report performance using all prompts, including cases with coincident variables, that is, when A=BA=B, B=CB=C, or C=AC=A. We use two conditions to fix the prompt template: (a) The model should be capable of answering the problem via the next token. (b) All variations over A,B,CA,B,C must tokenize to the same width. Therefore we exclude evaluation of the hours domain for the Qwen, Gemma, and Mistral families since all of them use a single digit tokenizer. A.1 List of variables Tables 2 and 3 show the values of variables used for replicate variables and A,B,CA,B,C respectively. A.2 Capabilities Figure 6 shows the top-1 and top-3 accuracy of each model indexed by γ over prompt templates in Table 1. Table 4 lists the aggregate top-1 and top-3 accuracy over replicates. Table 1: Prompt templates used for each model and cyclic domain. A,B,CA,B,C and the replicate variables vary as described in Tables 2 and 3. N/A indicates domains excluded because the cyclic values do not tokenize to a common width. Model Domain Prompt template Llama-3.2-3B months Hi name, the noun is scheduled from A dates to B dates. This is the same as time from C dates to hours Hi name, the noun is scheduled from day A o’clock to day B o’clock. This is the same time as from day C o’clock to day weekdays Hi name, the noun is scheduled from A to B. This is the same as time from C to music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Llama-3.1-8B months Hi name, the noun is scheduled between A dates and B dates. This is the same time as between C dates and hours Hi name, the noun is scheduled between day A o’clock and day B o’clock. This is the same time as between day C o’clock and day weekdays Hi name, the noun is scheduled between A and B. This is the same as time between C and music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Qwen-2.5-7B months Hello name, the noun is scheduled from A datesDouble to B datesDouble. This is the same as duration from C datesDouble to hours N/A weekdays Hi name, the noun is scheduled from A to B. This is the same time as from C to music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Qwen-3-8B months Hello name, the noun is scheduled from A datesDouble to B datesDouble. This is the same time as from C datesDouble to hours N/A weekdays Hi name, the noun is scheduled from A to B. This is the same time as from C to music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Gemma-2-9B months Hello name, the noun is scheduled from A to B. This is the same as duration from C to hours N/A weekdays Hi name, the noun is scheduled from A to B. This is the same time as from C to music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Gemma-3-12B months Hello name, the noun is scheduled from A to B. This is the same time as from C to hours N/A weekdays Hi name, the noun is scheduled from A to B. This is the same time as from C to music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Mistral-small-24B-Base-2501 months Hi name, the noun is scheduled from A to B. This is the same as time from C to hours N/A weekdays Hi name, the noun is scheduled between A and B. This is the same as time between C and music Hi name, on the musical scale, the interval from the note A to the note B is the same as the interval from the note C to the note Figure 6: Top-1 and top-3 accuracy as a function of γ=B−Aγ=B-A across models and cyclic concepts. Columns correspond to models and rows to concepts. Curves show accuracy averaged across the 10 sampled replicates, with shaded bands showing the central 80% interval. Hours are omitted for Qwen, Gemma, and Mistral because the relevant numerical values do not tokenize to a common width. Top-3 accuracy is generally more stable across γ than top-1 accuracy. Appendix B Theoretical background This appendix collects properties of the functional ANOVA decomposition used throughout the paper and establishes the identities underlying the mode fraction, the energy fraction and steering analyses. Let us revisit the functional ANOVA decomposition defined in the main text, ϕr(A,B,C)=r+Ar+Br+Cr+ABr+BCr+CAr+ABCr, φ^r(A,B,C)= μ^r+ h_A^r+ h_B^r+ h_C^r+ h_AB^r+ h_BC^r+ h_CA^r+ h_ABC^r, where r μ^r :=ABC(ϕr), :=E_ABC( φ^r), Ar h_A^r :=BC(ϕr)−r, :=E_BC( φ^r)- μ^r, ABr h_AB^r :=C(ϕr)−r−Ar−Br, :=E_C( φ^r)- μ^r- h_A^r- h_B^r, ABCr h_ABC^r :=ϕr−r−Ar−Br−Cr−ABr−BCr−CAr. := φ^r- μ^r- h_A^r- h_B^r- h_C^r- h_AB^r- h_BC^r- h_CA^r. To standardize notation, we use ∅r=r h^r_ = μ^r. Using these definitions, it immediately follows that each non-constant decomposed term has zero mean with respect to each variable on which it depends. Lemma 1 (Zero mean of ANOVA terms). For every nonempty S⊆A,B,CS \A,B,C\ and every X∈SX∈ S, X[Sr]=.E_X\! [ h_S^r ]= 0. (17) Proof. Consider Ar h_A^r: A(Ar) _A( h_A^r) =A(BC(ϕr)−r) =E_A (E_BC( φ^r)- μ^r ) =ABC(ϕr)−r =E_ABC( φ^r)- μ^r =r−r = μ^r- μ^r =0. =0. Similarly, for ABr h_AB^r A(ABr) _A( h_AB^r) =A(C(ϕr)−r−Ar−Br) =E_A (E_C( φ^r)- μ^r- h_A^r- h_B^r ) =AC(ϕr)−r−A(Ar)−Br =E_AC( φ^r)- μ^r-E_A( h_A^r)- h_B^r =(r+Br)−r−0−Br = ( μ^r+ h_B^r )- μ^r-0- h_B^r =0. =0. The same argument holds for B(AB)=0E_B( h_AB)=0. Finally, consider ABCr h_ABC^r A(ABCr) _A( h_ABC^r) =A(ϕr−r−Ar−Br−Cr−ABr−BCr−CAr) =E_A ( φ^r- μ^r- h_A^r- h_B^r- h_C^r- h_AB^r- h_BC^r- h_CA^r ) =A(ϕr)−r−A(Ar)−Br−Cr−A(ABr)−BCr−A(CAr) =E_A( φ^r)- μ^r-E_A( h_A^r)- h_B^r- h_C^r-E_A( h_AB^r)- h_BC^r-E_A( h_CA^r) =(r+Br+Cr+BCr)−r−0−Br−Cr−0−BCr−0 = ( μ^r+ h_B^r+ h_C^r+ h_BC^r )- μ^r-0- h_B^r- h_C^r-0- h_BC^r-0 =0. =0. Similarly, B(ABCr)=C(ABCr)=0E_B( h_ABC^r)=E_C( h_ABC^r)=0. In general, it can be proved by induction. ∎ Definition 1 (Ensemble-averaged form). For S,T⊆A,B,CS,T \A,B,C\ and two decomposed ensembles HSrH_S^r and HTrH_T^r with vectors Sr∈HSr h_S^r∈ H_S^r and Tr∈HTr h_T^r∈ H_T^r, define the ensemble-averaged form as ⟨HSr,HTr⟩ens:=ABC[⟨Sr,Tr⟩] H_S^r,H_T^r _ens:=E_ABC [ h_S^r, h_T^r ] (18) where the inner product on the RHS is the standard Euclidean inner product defined over ℝdmodelR^d_model. Table 2: Replicate variables used in the experiments. Variable Values name Each name should tokenize to one token. There are 194 approved names for each Llama, Qwen, and Mistral model, and 200 for each Gemma model from the list of 200 most common names of the last century (Social Security Administration, n.d.). noun conference, meeting, forum, summit dates 1,…,28 datesDouble 10,…,28 day Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday Table 3: Values assigned to the cyclic variables A, B, and C in each domain. Months and hours have cycle length N=12N=12, while weekdays and musical notes have N=7N=7. Domain Values of A, B, and C Months January, February, March, April, May, June, July, August, September, October, November, December Hours 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 Weekdays Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday Music C, D, E, F, G, A, B Table 4: Top-1 and top-3 accuracy by model and domain. Values are mean ± standard deviation across 10 replicates, computed over all prompts, including coincident values of A, B, and C. Model Performance metric Months Hours Weekdays Music Llama-3.2-3B Top-1 accuracy 53.4±2.253.4± 2.2% 78.5±3.278.5± 3.2% 78.3±2.578.3± 2.5% 70.9±0.370.9± 0.3% Top-3 accuracy 72.2±2.872.2± 2.8% 95.2±1.195.2± 1.1% 96.8±0.896.8± 0.8% 91.9±0.591.9± 0.5% Llama-3.1-8B Top-1 accuracy 66.0±3.966.0± 3.9% 79.9±1.879.9± 1.8% 78.8±1.178.8± 1.1% 79.3±0.779.3± 0.7% Top-3 accuracy 84.8±2.684.8± 2.6% 94.7±1.594.7± 1.5% 98.6±0.398.6± 0.3% 90.2±0.490.2± 0.4% Qwen-2.5-7B Top-1 accuracy 62.7±7.062.7± 7.0% N/A 75.4±2.475.4± 2.4% 89.5±0.689.5± 0.6% Top-3 accuracy 82.5±6.882.5± 6.8% N/A 98.7±0.898.7± 0.8% 99.7±0.299.7± 0.2% Qwen-3-8B Top-1 accuracy 68.5±4.568.5± 4.5% N/A 67.5±3.467.5± 3.4% 84.9±1.784.9± 1.7% Top-3 accuracy 86.5±2.686.5± 2.6% N/A 94.0±1.094.0± 1.0% 99.5±0.399.5± 0.3% Gemma-2-9B Top-1 accuracy 58.3±5.958.3± 5.9% N/A 85.4±2.685.4± 2.6% 70.7±0.670.7± 0.6% Top-3 accuracy 78.8±6.278.8± 6.2% N/A 98.8±1.798.8± 1.7% 87.0±0.387.0± 0.3% Gemma-3-12B Top-1 accuracy 66.5±1.866.5± 1.8% N/A 85.5±1.485.5± 1.4% 75.1±1.375.1± 1.3% Top-3 accuracy 84.1±1.784.1± 1.7% N/A 98.3±1.198.3± 1.1% 94.0±0.894.0± 0.8% Mistral-small-24B-Base-2501 Top-1 accuracy 76.7±2.776.7± 2.7% N/A 79.5±0.779.5± 0.7% 76.7±0.776.7± 0.7% Top-3 accuracy 94.3±1.694.3± 1.6% N/A 95.7±0.795.7± 0.7% 99.1±0.399.1± 0.3% Proposition 1 (Inner product). Under conditions of Definition 1, Equation 18 satisfies the properties of an inner product: 1. Symmetry. ⟨HSr,HTr⟩ens=⟨HTr,HSr⟩ens H_S^r,H_T^r _ens= H_T^r,H_S^r _ens. 2. Linearity. ⟨αHSr+βHTr,HUr⟩ens=α⟨HSr,HUr⟩ens+β⟨HTr,HUr⟩ens α H_S^r+β H_T^r,H_U^r _ens=α H_S^r,H_U^r _ens+β H_T^r,H_U^r _ens for any α,β∈ℝα,β . 3. Positive definiteness. ⟨HSr,HSr⟩ens≥0 H_S^r,H_S^r _ens≥ 0 and the equality holds iff HSr=0H_S^r=0. Proof. Symmetry. By symmetry of the Euclidean inner product, ⟨HSr,HTr⟩ens=ABC[⟨Sr,Tr⟩]=ABC[⟨Tr,Sr⟩]=⟨HTr,HSr⟩ens. H_S^r,H_T^r _ens=E_ABC [ h_S^r, h_T^r ]=E_ABC [ h_T^r, h_S^r ]= H_T^r,H_S^r _ens. Linearity. For any α,β∈ℝα,β and ensembles HSr,HTr,HUrH_S^r,H_T^r,H_U^r, linearity of the Euclidean inner product and of expectation gives ⟨αHSr+βHTr,HUr⟩ens α H_S^r+β H_T^r,H_U^r _ens =ABC[⟨αSr+βTr,Ur⟩] =E_ABC [ α h_S^r+β h_T^r, h_U^r ] =α⟨HSr,HUr⟩ens+β⟨HTr,HUr⟩ens. =α H_S^r,H_U^r _ens+β H_T^r,H_U^r _ens. Positive definiteness. For every ensemble HSrH_S^r, ⟨HSr,HSr⟩ens=ABC[‖Sr‖22]≥0. H_S^r,H_S^r _ens=E_ABC [\| h_S^r\|_2^2 ]≥ 0. A nonnegative random variable has expectation zero if and only if it is zero. Hence ⟨HSr,HSr⟩ens=0⟺Sr=0. H_S^r,H_S^r _ens=0 h_S^r=0. This is equivalent to HSr=0H_S^r=0 in the ensemble space. Thus the form satisfies all three inner product axioms. ∎ Proposition 2 (Orthogonality of the ANOVA terms). Under the full ensemble, any two distinct terms in the decomposition are orthogonal under the inner product in Definition 1: ⟨HSr,HTr⟩ens=0,S≠T. H_S^r,H_T^r _ens=0, S≠ T. (19) Here, S,T⊆A,B,CS,T \A,B,C\ Proof. Let S≠TS≠ T. Without loss of generality, choose a variable X∈S∖TX∈ S T. Since Tr h_T^r does not depend on X, we may average over X first: ⟨HSr,HTr⟩ens H_S^r,H_T^r _ens =ABC∖X[⟨X[Sr],Tr⟩] =E_\ABC\ \X\ [ _X [ h_S^r ], h_T^r ] =0, =0, where the final equality follows from Lemma 1. Hence all distinct terms are pairwise orthogonal. ∎ Corollary 1 (Energy decomposition). Under the conditions of Proposition 2, ABC‖ϕr(A,B,C)−∅r‖22=∑∅≠S⊆A,B,CS‖Sr‖22.E_ABC \| φ^r(A,B,C)- h_ ^r \|_2^2= _ ≠ S \A,B,C\E_S \| h_S^r \|_2^2. (20) Proof. By Equation 2, ϕr−∅r=∑∅≠S⊆A,B,CSr. φ^r- h_ ^r= _ ≠ S \A,B,C\ h_S^r. Expanding the squared norm and applying Proposition 2 eliminates all cross terms: ABC‖ϕr−∅r‖22 _ABC \| φ^r- h_ ^r \|_2^2 =∑∅≠S⊆A,B,CABC‖Sr‖22 = _ ≠ S \A,B,C\E_ABC \| h_S^r \|_2^2 =∑∅≠S⊆A,B,CS‖Sr‖22, = _ ≠ S \A,B,C\E_S \| h_S^r \|_2^2, where the second equality follows because Sr h_S^r depends only on the variables contained in S. ∎ B.1 Mode fraction and cluster analysis In Section 3 of the main text, we defined mode fraction using the DFT. It can instead be defined directly without invoking the DFT. Here, we show the equivalence. Figure 1 suggests that the interaction ensembles form clusters indexed by γ or D. Definition 2 (Cluster vectors). For HABrH_AB^r, we define the centroid of the cluster indexed by γ as ¯γr:=[ABr|B−A=γ]. h_γ^r:=E\! [ h_AB^r\, |\,B-A=γ ]. (21) Similarly, for HABCrH_ABC^r, we define the centroid of the cluster indexed by D as ¯Dr:=[ABCr|C+B−A=D]. h_D^r:=E\! [ h_ABC^r\, |\,C+B-A=D ]. (22) Note that these are the same vectors used in the steering experiments in Section 5. We now define the corresponding projection operators for each ensemble. Definition 3 (Projection operator). Let PγP_γ be the projection operator that operates on HABrH_AB^r to give the corresponding cluster vector: PγHABr:=¯γrP_γH_AB^r:= h_γ^r (23) Similarly, we define PDP_D as PDHABCr:=¯DrP_DH_ABC^r:= h_D^r (24) Thus, PγHABrP_γH_AB^r is constant over pairs with the same value of γ, while PDHABCrP_DH_ABC^r is constant over triples with the same value of D. Now we define cluster fraction using these projection operators and show that it is equivalent to the definition of mode fraction in Section 3. Definition 4 (Cluster fraction). Let c(Hr)c(H^r) be the cluster fraction defined by c(HABr):=γ‖PγHABr‖22AB‖ABr‖22=γ‖¯γr‖22AB‖ABr‖22 c(H_AB^r):= E_γ \|P_γH_AB^r \|_2^2E_AB \| h_AB^r \|_2^2= E_γ \| h_γ^r \|_2^2E_AB \| h_AB^r \|_2^2 (25) and c(HABCr):=D‖PDHABCr‖22ABC‖ABCr‖22=D‖¯Dr‖22ABC‖ABCr‖22 c(H_ABC^r):= E_D \|P_DH_ABC^r \|_2^2E_ABC \| h_ABC^r \|_2^2= E_D \| h_D^r \|_2^2E_ABC \| h_ABC^r \|_2^2 (26) Proposition 3 (Equivalence between cluster fraction and mode fraction). If c(Hr)c(H^r) is the cluster fraction as in Definition 4 and m(Hr)m(H^r) is the mode fraction as defined in Section 3, then c(HSr)=m(HSr),c(H_S^r)=m(H_S^r), (27) for S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\. Proof. We suppress the superscript r for readability. First consider HABH_AB. Let AB:=PγHAB=¯γ. p_AB:=P_γH_AB= h_γ. Here, γ=B−Aγ=B-A, and hence B=A+γB=A+γ. The discrete Fourier transform (DFT) of p is ^kAkB p_k_Ak_B =∑A,BABe−2πi(kAA+kBB)/N = _A,B p_ABe^-2π i(k_AA+k_BB)/N =∑A,γ¯γe−2πi[(kA+kB)A+kBγ]/N = _A,γ h_γe^-2π i[(k_A+k_B)A+k_Bγ]/N =(∑Ae−2πi(kA+kB)A/N)(∑γ¯γe−2πikBγ/N). = ( _Ae^-2π i(k_A+k_B)A/N ) ( _γ h_γe^-2π ik_Bγ/N ). Here, N is the length of the cycle; N=12N=12 for months and hours, while N=7N=7 for weekdays and musical notes. The sum over A vanishes unless kA+kB=0k_A+k_B=0. Therefore, ^kAkB=whenever kA≠−kB. p_k_Ak_B= 0 k_A≠-k_B. For frequencies (kA,kB)=(−k,k)(k_A,k_B)=(-k,k), ^−k,k p_-k,k =N∑γ¯γe−2πikγ/N =N _γ h_γe^-2π ikγ/N (28) =∑A,γA,A+γe−2πikγ/N = _A,γ h_A,A+γe^-2π ikγ/N (29) =∑A,BABe−2πik(B−A)/N = _A,B h_ABe^-2π ik(B-A)/N (30) =^−k,k, = h_-k,k, (31) where the second equality uses ¯γ=1N∑A,A+γ. h_γ= 1N _A h_A,A+γ. Thus, the Fourier transform of PγHABP_γH_AB agrees with that of HABH_AB on the modes kA=−kBk_A=-k_B and vanishes on all other modes. Consider c(HAB)c(H_AB): c(HAB) c(H_AB) =AB‖AB‖22AB‖AB‖22 = E_AB \| p_AB \|_2^2E_AB \| h_AB \|_2^2 =∑kA=−kB‖^kAkB‖22∑kA,kB‖^kAkB‖22 = _k_A=-k_B \| p_k_Ak_B \|_2^2 _k_A,k_B \| h_k_Ak_B \|_2^2 =∑kA=−kB‖^kAkB‖22∑kA,kB‖^kAkB‖22 = _k_A=-k_B \| h_k_Ak_B \|_2^2 _k_A,k_B \| h_k_Ak_B \|_2^2 =m(HAB), =m(H_AB), where we get the second equality using Parseval’s identity and the third equality using Equation 31. A similar proof holds for c(HABC)=m(HABC)c(H_ABC)=m(H_ABC). ∎ B.2 Steering vector Finally, we show the correspondence between the choice of steering vector as in Section 5 and the conditional mean of the activation ensemble. Proposition 4. If ¯γr h_γ^r is the vector according to Definition 2, then ¯γr=ABC[ϕr(A,B,C)|B−A=γ]−r. h_γ^r=E_ABC [ φ^r(A,B,C)\;|\;B-A=γ ]- μ^r. (32) Similarly, ¯Dr=ABC[ϕr(A,B,C)|C+B−A=D]−r. h_D^r=E_ABC [ φ^r(A,B,C)\;|\;C+B-A=D ]- μ^r. (33) Proof. To see this, start with the conditional average over ϕr φ^r: [ϕr|B−A=γ] \! [ φ^r\, |\,B-A=γ ] =r+[Ar|B−A=γ]+[Br|B−A=γ] = μ^r+E\! [ h_A^r\, |\,B-A=γ ]+E\! [ h_B^r\, |\,B-A=γ ] +[Cr|B−A=γ]+[ABr|B−A=γ] +E\! [ h_C^r\, |\,B-A=γ ]+E\! [ h_AB^r\, |\,B-A=γ ] +[BCr|B−A=γ]+[CAr|B−A=γ] +E\! [ h_BC^r\, |\,B-A=γ ]+E\! [ h_CA^r\, |\,B-A=γ ] +[ABCr|B−A=γ]. +E\! [ h_ABC^r\, |\,B-A=γ ]. Using Lemma 1, we get [Ar|B−A=γ]=[Br|B−A=γ]=[Cr|B−A=γ]=,E\! [ h_A^r\, |\,B-A=γ ]=E\! [ h_B^r\, |\,B-A=γ ]=E\! [ h_C^r\, |\,B-A=γ ]= 0, and [BCr|B−A=γ]=[CAr|B−A=γ]=[ABCr|B−A=γ]=.E\! [ h_BC^r\, |\,B-A=γ ]=E\! [ h_CA^r\, |\,B-A=γ ]=E\! [ h_ABC^r\, |\,B-A=γ ]= 0. Hence, [ϕr|B−A=γ]=r+[ABr|B−A=γ].E\! [ φ^r\, |\,B-A=γ ]= μ^r+E\! [ h_AB^r\, |\,B-A=γ ]. By definition, ¯γr=[ABr|B−A=γ], h_γ^r=E\! [ h_AB^r\, |\,B-A=γ ], and therefore ¯γr=[ϕr|B−A=γ]−r. h_γ^r=E\! [ φ^r\, |\,B-A=γ ]- μ^r. A similar proof holds for ¯Dr h_D^r. ∎ Therefore, we get the following identities using Proposition 4: ¯γ+δr−¯γr=[ϕr|B−A=γ+δ]−[ϕr|B−A=γ], h_γ+δ^r- h_γ^r=E\! [ φ^r\, |\,B-A=γ+δ ]-E\! [ φ^r\, |\,B-A=γ ], (34) ¯D+δr−¯Dr=[ϕr|C+B−A=D+δ]−[ϕr|C+B−A=D]. h_D+δ^r- h_D^r=E\! [ φ^r\, |\,C+B-A=D+δ ]-E\! [ φ^r\, |\,C+B-A=D ]. (35) Thus, the steering using ¯γ+δr−¯γr h_γ+δ^r- h_γ^r is equivalent to the difference-of-means steering. B.3 Coincident variables We always compute the interaction ensembles using the full set of prompts, including cases with A=BA=B, B=CB=C, or C=AC=A. Table 5 reports the mean norm of the interaction vectors grouped by γ=B−Aγ=B-A, γ′=C−Bγ =C-B, and γ′=A−Cγ =A-C for a representative ensemble using the months prompt template at layer 15 of Llama-3.1-8B. The vectors corresponding to γ=0,γ′=0,γ′=0γ=0,\;γ =0,\;γ =0 have substantially larger norms than the rest of the ensemble and can therefore bias quantitative results toward coincident-variable cases. We exclude such prompts when reporting the results in the main text. Unless stated otherwise, we repeat our analyses without this exclusion in the following appendices. This also serves as a sanity check that our conclusions are not driven by the coincident-variable cases. Table 5: Mean norm of each interaction vector categorized by γ=B−Aγ=B-A, γ′=C−Bγ =C-B, and γ′=A−Cγ =A-C, respectively, for a single replicate of the months prompt template at layer 15 of Llama-3.1-8B. The interaction vectors associated with zero difference have substantially larger norms than the remaining classes. This motivates the exclusion of coincident-variable prompts from the quantitative analyses in the main text. Ensemble 0 1 2 3 4 5 6 7 8 9 10 11 HABH_AB 7.445 5.875 3.899 3.568 3.144 2.551 2.391 2.307 2.595 2.700 3.188 4.023 HBCH_BC 4.343 1.921 0.918 0.961 0.971 1.021 0.853 0.976 1.022 1.378 1.377 1.575 HCAH_CA 6.448 3.013 1.311 1.457 1.478 1.624 1.476 1.487 1.263 1.220 1.589 3.189 Appendix C Mode fraction Figure 7: Mode fraction of the interaction ensembles across models and cyclic domains. Columns correspond to models and rows to domains; shaded bands show the central 80% interval across 10 replicates. The vertical gray band marks the transition in causal relevance from HABrH_AB^r to HABCrH_ABC^r identified by the ablation and replacement experiments. Across all models and prompt templates, the pairwise interaction ensembles are strongly organized by their corresponding pairwise differences before HABCrH_ABC^r becomes organized by D. The transition in HABCrH_ABC^r is more distributed across depth in the Gemma models. The main text reports the layerwise organization of the interaction ensembles for Llama-3.1-8B on the months domain. Here, we test whether the same pattern generalizes across models and cyclic domains. Recall the definition of mode fraction for HABrH_AB^r as defined in the main text: m(HABr):=∑kA=−kB‖^kAkBr‖22∑kA,kB‖^kAkBr‖22.m(H_AB^r):= _k_A=-k_B h^r_k_Ak_B _2^2 _k_A,k_B h^r_k_Ak_B _2^2. (36) Similarly, we define the corresponding mode fractions for HBCrH_BC^r and HCArH_CA^r using modes satisfying kB+kC=0k_B+k_C=0 and kC+kA=0k_C+k_A=0 respectively. For HABCrH_ABC^r, the mode fraction is defined using modes (kA,kB,kC)=(−k,k,k)(k_A,k_B,k_C)=(-k,k,k). Figure 7 shows the mode fractions for all interaction ensembles using the prompts and models as detailed in Table 1. Across all models and prompt templates, the second-order interaction ensembles become strongly organized by their corresponding pairwise differences before the three-way interaction becomes organized by D. The increase in m(HABCr)m(H_ABC^r) occurs near the same relative depth within a particular model across domains. The transition is sharp for most models but is distributed across several layers in the Gemma models. Notice that we report Llama-3.1-8B on the months domain again: we include prompts with coincident variables and we see that the mode fraction is inflated as a result of their inclusion. Appendix D Energy Fraction Figure 8: Energy fraction of the interaction ensembles across models and cyclic domains. Columns correspond to models and rows to domains; shaded bands show the central 80% interval across 10 replicates. The vertical gray band marks the transition in causal relevance identified by the ablation and replacement experiments. Across all models and prompt templates, the energy fraction of HABrH_AB^r decreases near the same depth at which the energy fraction of HABCrH_ABC^r increases. Recall Corollary 1 from Appendix B: A,B,C‖ϕr(A,B,C)−r‖22=∑S‖Sr‖22,E_A,B,C \| φ^r(A,B,C)- μ^r \|_2^2= _SE_S \| h_S^r \|^2_2, (37) where S ranges over non-empty subsets of A,B,C\A,B,C\. This motivates us to define the energy fraction of each ensemble as e(HSr)=S(‖Sr‖22)ABC(‖ϕr(A,B,C)−r‖22),with∑Se(HSr)=1.e(H_S^r)= E_S ( h^r_S _2^2 )E_ABC ( φ^r(A,B,C)- μ^r _2^2 ), _Se(H_S^r)=1. (38) We next test whether the layerwise redistribution of energy fraction observed in the main text generalizes across models and domains. Figure 8 shows the energy fraction for each interaction ensemble. Across all models and prompt templates, the energy fraction of HABrH_AB^r decreases near the same depth at which the energy fraction of HABCrH_ABC^r increases. As with mode fraction, this transition is spread over several layers in the Gemma models. Appendix E Controls for Llama-3.1-8B The strong organization of the pairwise interaction ensembles by their corresponding pairwise differences does not by itself imply that this structure is task-specific. We therefore test whether similar organization arises when the cyclic tokens co-occur without the relational computation used in the main task. We design three control prompts as listed in Table 6. For the non-cyclic control, we construct a template similar to the Llama-3.1-8B months template but use A,B,C∈A,B,C∈ berry, apple, orange, banana, peach, pear, fig, mango, lemon, lime, cherry, plum instead of the usual calendar months. The off-by-1 control retains A,B,CA,B,C as calendar months, but the answer depends only on C; A and B therefore co-occur in the prompt without contributing to the answer. The list control also retains calendar months but removes the arithmetic relation entirely; A,B,CA,B,C are presented as items in a list and the model is asked to return B. Figure 9: Mode and energy fractions for three control prompt templates using Llama-3.1-8B. The non-cyclic control replaces months with an unordered vocabulary, while the off-by-1 and list controls retain calendar months but remove the relational computation of the main task. In the latter two controls, the pairwise interaction ensembles remain strongly organized by their corresponding pairwise differences despite having a small energy fraction. In contrast, HABCrH_ABC^r remains negligible in both mode and energy fraction across all controls. All results reported here exclude vectors corresponding to the coincident-variable cases. In Figure 9, we have computed the mode and energy fractions for the control prompt templates. In all three controls, the mode and energy fractions of HABCrH_ABC^r remain close to zero. In contrast, when calendar months co-occur in the off-by-1 and list controls, HABrH_AB^r, HBCrH_BC^r, and HCArH_CA^r can still have large mode fractions, but retain a small energy fraction. This suggests that pairwise-difference geometry can arise from co-occurrence of cyclic tokens alone, whereas the D-organized interaction ensemble HABCrH_ABC^r requires the task structure. Table 6: Control prompt templates for Llama-3.1-8B. Control Prompt template Non-cyclic Hello name, the noun is scheduled between A dates and B dates. This is the same time as between C dates and Off by 1 Hello name, the noun is scheduled between A dates and B dates. The month that comes after C dates is List Hello name, the talks are on A dates, on B dates, and on C dates. The second talk is on Appendix F Causal handoff Section 4 shows that HABrH_AB^r and HABCrH_ABC^r are causally relevant at different depths, but this does not establish whether the later HABCrH_ABC^r representation depends on the earlier HABrH_AB^r representation. To test this, we ablate HABrH_AB^r at layer 15 of Llama-3.1-8B on the months domain, allow the model to run normally thereafter, and recompute the ANOVA decomposition at every subsequent layer. We then compare the mode and energy fractions of the resulting HABCrH_ABC^r with those of the unmodified model. Figure 10 shows the comparison of mode and energy fractions between the ablated and the unmodified ensemble for HABCrH_ABC^r. In the unmodified case, both the mode fraction and energy fraction of HABCrH_ABC^r rise sharply around layer 18. After ablating HABrH_AB^r at layer 15, neither transition occurs at that depth. The mode fraction recovers slightly in later layers, but the energy fraction remains substantially reduced. Thus, the emergence of the later HABCrH_ABC^r representation depends on the earlier HABrH_AB^r representation. Figure 10: Downstream HABCrH_ABC^r after ablating HABrH_AB^r at layer 15. Solid curves show the mode and energy fractions of the recomputed HABCrH_ABC^r after the intervention; dashed curves show the corresponding quantities in the unmodified model. The vertical gray band marks layer 18, where HABCrH_ABC^r normally becomes organized by D and increases sharply in energy fraction. (a) After ablating HABrH_AB^r, the mode fraction does not undergo its normal increase at layer 18, although some organization recovers later but never regains the unmodified baseline. (b) The energy fraction likewise fails to increase at layer 18 and remains substantially below that of the unmodified ensemble. These results show that the normal emergence of HABCrH_ABC^r depends on the earlier HABrH_AB^r representation. Appendix G Ensemble ablation We next test whether the layerwise ablation pattern from the main text generalizes across models and cyclic concepts. We conduct the ensemble ablation experiments for all prompts and models in Table 1. Recall that we intervene on the last token position at a particular layer using ϕ~r(A,B,C)=ϕr(A,B,C)−Sr, φ^r(A,B,C)= φ^r(A,B,C)- h^r_S, (39) for all A,B,C∈A,B,C and S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\. Figures 12 and 13 report the results of the ensemble ablation experiments using top-1 and top-3 accuracy as metrics, respectively. We observe that across every model and domain, the layerwise behavior of interaction ensembles under ablation remains the same. Ablating HABrH_AB^r always affects the accuracy at an earlier network depth than ablating HABCrH_ABC^r. However, ablating HBCrH_BC^r or HCArH_CA^r has little to no effect on performance. Ablating HABCrH_ABC^r at later layers decreases the performance dramatically, with performance dropping to near chance when it is ablated near the final layers. Appendix H Ensemble replacement We repeat the ensemble replacement experiments across all models and prompt templates using the following intervention ϕ~r(A,B,C)=r+Sr, φ^r(A,B,C)= μ^r+ h^r_S, (40) for all A,B,C∈A,B,C and S∈AB,BC,CA,ABCS∈\AB,BC,CA,ABC\. Figures 14 and 15 show the corresponding results using top-1 and top-3 accuracy respectively. Retaining HBCrH_BC^r or HCArH_CA^r together with the mean performs similarly to the μrμ^r-only baseline. Retaining HABrH_AB^r preserves performance in intermediate layers but becomes insufficient later in depth, while retaining HABCrH_ABC^r becomes effective at later layers and can exceed the unmodified baseline. Together with Appendix G, these results show that the transition from HABrH_AB^r to HABCrH_ABC^r is reproduced by both ablation and replacement interventions across models and prompt templates. Appendix I Steering We report steering results across all models and prompt templates using the averaged vectors introduced in Section 5. For S∈γ,DS∈\γ,D\, we intervene according to ϕ~r(A,B,C)=ϕr(A,B,C)+α(¯S+δr−¯Sr). φ^r(A,B,C)= φ^r(A,B,C)+α ( h^r_S+δ- h^r_S ). (41) To compare steering effects across models, domains, and layers, we report the normalizedmedianΔm3normalized\;median\; m_3. We normalize by Δm3max=maxS,δ,r,ℓ|Δm3r,ℓ(¯S+δr−¯Sr)|. m_3^max= max_S,δ,r, | m_3^r, ( h^r_S+δ- h^r_S) |. (42) Figure 16 shows the resulting steering profiles at α=1α=1. Steering with ¯γr h_γ^r is effective in intermediate layers, whereas steering with ¯Dr h_D^r becomes effective at later layers. These regions coincide with the layers where HABrH_AB^r and HABCrH_ABC^r are found to be causally relevant in Appendices G and H. I.1 Sensitivity to steering strength The experiments in the main text use α=1α=1. To test whether the observed steering effects depend strongly on this choice, Figure 11 shows the medianΔm3median\; m_3 as a function of α at representative layers for ¯γr h_γ^r and ¯Dr h_D^r. In both cases, the steering effect varies smoothly with α. Figure 11: Median Δm3 m_3 as a function of steering strength α for the averaged steering vectors. (a) Steering with ¯γr h_γ^r at layer 15. (b) Steering with ¯Dr h_D^r at layer 22. Both panels use the months prompt template with Llama-3.1-8B. I.2 Symmetrized pointwise steering The averaged vectors above depend only on the relation label γ or D. As a complementary intervention, we construct prompt-specific counterfactual interaction vectors by averaging over the coordinate changes that produce the same shift in the corresponding relation. For HABrH_AB^r, both B→B+δB→ B+δ and A→A−δA→ A-δ change γ→γ+δγ→γ+δ and therefore D→D+δD→ D+δ. We define ~AB|δr=12(A,B+δr+A−δ,Br). h^r_AB δ= 12 ( h^r_A,B+δ+ h^r_A-δ,B ). (43) Similarly, each of A→A−δA→ A-δ, B→B+δB→ B+δ, and C→C+δC→ C+δ changes D to D+δD+δ, so we define ~ABC|δr=13(A−δ,B,Cr+A,B+δ,Cr+A,B,C+δr). h^r_ABC δ= 13 ( h^r_A-δ,B,C+ h^r_A,B+δ,C+ h^r_A,B,C+δ ). (44) When δ=0δ=0, these expressions reduce to the original interaction vectors. We steer using ϕ~r(A,B,C)=ϕr(A,B,C)+α(~S|δr−Sr), φ^r(A,B,C)= φ^r(A,B,C)+α ( h^r_S δ- h^r_S ), (45) where S∈AB,ABCS∈\AB,ABC\. Figure 17 shows the normalizedmedianΔm3normalized\;median\; m_3 obtained using this intervention. The same layerwise pattern appears as with the averaged steering vectors: steering using ~AB|δr h^r_AB δ is effective in intermediate layers, whereas steering using ~ABC|δr h^r_ABC δ becomes effective at later layers. Appendix J Cross-domain transfer We report the cross-domain transfer experiments across models and domains. Figures 18 and 19 show top-1 and top-3 accuracy after transplanting HABrH_AB^r from one domain into another following ablation of the corresponding domain-1 interaction. Figures 20 and 21 show the corresponding replacement experiments, in which the domain-1 mean is retained together with the domain-2 interaction. Figure 22 shows cross-domain steering using ¯γ h_γ, including δ=0δ=0 as a control. Direct transplantation of HABrH_AB^r generally recovers part of the performance lost under ablation, although the degree of recovery depends on the source and target domains. In particular, transfer between the months and hours domains is asymmetric for the Llama models: months→ decreases performance relative to the unmodified model, whereas hours→ improves the performance (Figures 18 and 19). In contrast, direct transfer of HABCrH_ABC^r is substantially less consistent across domains (Figures 23 and 24). J.1 Alignment of interaction subspaces The inconsistent direct transfer of HABCrH_ABC^r may arise because different domains use different residual-stream subspaces for the same underlying interaction structure. We therefore align the interaction subspaces before transplantation. We perform the alignment in a q=N−1q=N-1 dimensional subspace and divide the replicates into a training set (70%), denoted by rtr_t, and a held-out test set (30%), denoted by rer_e. For domain c∈1,2c∈\1,2\, let WAB(c)=[ABrt,(c)]∈ℝtN2×dmodel,W_AB^(c)= [ h_AB^r_t,(c) ] ^tN^2× d_model, (46) and similarly, WABC(c)=[ABCrt,(c)]∈ℝtN3×dmodel.W_ABC^(c)= [ h_ABC^r_t,(c) ] ^tN^3× d_model. (47) Both WAB(c)W_AB^(c) and WABC(c)W_ABC^(c) have zero mean by Lemma 1. We describe the alignment procedure for WABW_AB; the same procedure is applied to WABCW_ABC. Consider transplanting the domain-2 interaction into domain-1. For each domain, we first compute WAB(c)=U(c)Σ(c)(V(c))T,W_AB^(c)=U^(c) ^(c) (V^(c) )^T, (48) where the columns of V(c)V^(c) are the right singular vectors. Let Vq(c)∈ℝdmodel×qV_q^(c) ^d_model× q contain the leading q=N−1q=N-1 right singular vectors. We project the interaction ensembles into these subspaces: ZAB(c)=WAB(c)Vq(c)∈ℝtN2×q.Z_AB^(c)=W_AB^(c)V_q^(c) ^tN^2× q. (49) We then align the domain-2 coordinates to domain-1 by solving the orthogonal Procrustes problem RAB∗=argminRTR=q‖ZAB(2)R−ZAB(1)‖F2,R_AB^*= R^TR= I_qarg\;min Z_AB^(2)R-Z_AB^(1) _F^2, (50) where RAB∗∈ℝq×qR_AB^* ^q× q is the orthogonal transformation that best aligns the two coordinate systems. Transforming a domain-2 interaction vector into the domain-1 residual-stream basis then consists of projection onto the domain-2 subspace, alignment in the q-dimensional coordinate system, and reconstruction in the domain-1 basis. We therefore define Q=Vq(2)RAB∗(Vq(1))T∈ℝdmodel×dmodel.Q=V_q^(2)R_AB^* (V_q^(1) )^T ^d_model× d_model. (51) We evaluate the learned alignment only on held-out replicates. For the aligned ablation-transplant experiment, we use ϕ~re,(1)(A,B,C)=ϕre,(1)(A,B,C)−ABre,(1)+ABre,(2)Q. φ^r_e,(1)(A,B,C)= φ^r_e,(1)(A,B,C)- h_AB^r_e,(1)+ h_AB^r_e,(2)Q. (52) For the corresponding aligned replacement experiment, we use ϕ~re,(1)(A,B,C)=re,(1)+ABre,(2)Q. φ^r_e,(1)(A,B,C)= μ^r_e,(1)+ h_AB^r_e,(2)Q. (53) Figures 25 and 26 show the aligned ablation-transplant results for both HABrH_AB^r and HABCrH_ABC^r, while Figures 27 and 28 show the corresponding replacement experiments. Alignment improves cross-domain transfer for both interaction ensembles and produces a particularly large improvement for HABCrH_ABC^r, whose direct transfer is otherwise inconsistent across domains. This suggests that part of the failure of direct HABCrH_ABC^r transplantation can be explained by differences in the residual-stream subspaces used by different domains. Figure 12: Top-1 accuracy after ablating an interaction ensemble at the last-token position, shown as a function of layer. The vertical shaded bands mark the model-specific intermediate layers where ablating HABrH_AB^r produces its largest drop in performance. Across models and prompts, ablating HBCrH_BC^r or HCArH_CA^r has little effect on accuracy, whereas ablating HABrH_AB^r or HABCrH_ABC^r substantially degrades performance. The effect of ablating HABrH_AB^r occurs earlier in depth than the effect of ablating HABCrH_ABC^r. Figure 13: Top-3 accuracy after ablating each interaction ensemble at the last-token position, shown as a function of layer. Across models and prompt templates, ablating HBCrH_BC^r or HCArH_CA^r has little effect on performance, whereas ablating HABrH_AB^r or HABCrH_ABC^r substantially decreases accuracy. The effect of ablating HABrH_AB^r consistently occurs earlier in depth than the effect of ablating HABCrH_ABC^r. Figure 14: Top-1 accuracy after replacing the activation of the last token with only the mean r μ^r and an interaction ensemble, shown as a function of layer. The vertical shaded bands mark the intermediate layers where ablating HABrH_AB^r produced the largest drop in performance, as shown in Figures 12 and 13. Replacing with HBCrH_BC^r, HCArH_CA^r, or HABCrH_ABC^r substantially degrades performance at the first vertical band, whereas replacing with HABrH_AB^r does not affect performance until the second vertical band. At later layers, retaining HABCrH_ABC^r preserves performance and can exceed the unmodified baseline. Figure 15: Top-3 accuracy after replacing the last-token activation with the ensemble mean r μ^r and one interaction term. Across models and prompt templates, retaining HABrH_AB^r preserves performance in intermediate layers but becomes insufficient later in depth, while retaining HABCrH_ABC^r becomes effective at later layers and can outperform the unmodified baseline. Retaining HBCrH_BC^r or HCArH_CA^r generally degrades the performance beginning from the first vertical band. Figure 16: Heatmap of steering using ¯γ h_γ and ¯D h_D across every model and prompt. Color indicates normalizedmedianΔm3normalized\;median\; m_3. Steering with ¯γ h_γ is most effective in intermediate layers, with boldface layer labels marking the layers where HABrH_AB^r was found to be causally relevant. At later layers, steering with ¯D h_D becomes effective. Figure 17: Heatmap of steering using ~AB|δ h_AB|δ and ~ABC|δ h_ABC|δ across every model and prompt. Color indicates normalizedmedianΔm3normalized\;median\; m_3. As with the steering results in Figure 16, steering using ~AB|δ h_AB|δ is effective in intermediate layers, while steering using ~ABC|δ h_ABC|δ becomes effective at later layers. Figure 18: Top-1 accuracy for cross-domain ablation transplant of HABrH_AB^r. Solid curves show performance after replacing an ablated domain-1 interaction vector with the corresponding domain-2 vector; dashed blue curves show ablation without transplantation and dashed black curves show the unmodified baseline. Successful transfer is indicated by recovery from the dashed blue ablation curve toward the baseline. Direct transplantation generally recovers performance, although the amount of recovery depends on the model and cross-domain pair. Figure 19: Top-3 accuracy for direct cross-domain ablation transplant of HABrH_AB^r. As in Figure 18, transplantation generally recovers part of the performance lost under ablation, with variation across model and cross-domain pairs. Figure 20: Top-1 accuracy for cross-domain replacement using HABrH_AB^r. Solid curves retain the domain-1 mean together with the domain-2 interaction vector; dashed blue curves show the corresponding within-domain replacement experiment. Solid and dashed trajectories follow each other, which indicates that the domain-2 interaction can substitute for the domain-1 interaction in downstream computation. Figure 21: Top-3 accuracy for direct cross-domain replacement using HABrH_AB^r. Across all models and cross-domain pairs, the transplanted domain-2 interaction reproduces the layerwise behavior of the corresponding within-domain replacement. Figure 22: Cross-domain steering using vectors extracted from domain-2. Color indicates normalized median Δm3 m_3. Steering is strongest in the same intermediate layers where HABrH_AB^r is causally relevant. The δ=0δ=0 case provides a non-trivial control: replacing the domain-1 relation vector with the corresponding domain-2 vector should preserve the original answer if the relation representation transfers across domains. Figure 23: Top-3 accuracy for cross-domain ablation transplant of HABCrH_ABC^r. Unlike HABrH_AB^r, direct transfer of the three-way interaction is not consistently successful across domains. Recovery in performance is observed for the months↔ transfers in the Llama models, but most other model–domain pairs show little improvement over ablation. Figure 24: Top-3 accuracy for cross-domain replacement using HABCrH_ABC^r. Consistent with Figure 23, the ability of a domain-2 HABCrH_ABC^r to substitute for its domain-1 counterpart depends strongly on the model and domain pair. Figure 25: Top-1 accuracy for cross-domain ablation transplant after aligning the interaction subspaces using training replicates and evaluating on held-out replicates. Solid curves show aligned domain-2 interaction vectors transplanted into domain 1; dashed curves show the corresponding ablation and unmodified baselines. Alignment improves transfer for both HABrH_AB^r and HABCrH_ABC^r, although the degree of recovery varies across models and domain pairs. Figure 26: Top-3 accuracy for cross-domain ablation transplant after interaction-subspace alignment. Alignment improves recovery over the ablated baseline for both HABrH_AB^r and HABCrH_ABC^r, with the largest qualitative change occurring for HABCrH_ABC^r, whose direct transfer was often weak. Figure 27: Top-1 accuracy for cross-domain replacement after aligning the interaction subspaces. Solid curves retain the domain-1 mean together with an aligned domain-2 interaction vector; dashed curves show the corresponding within-domain replacement. Alignment allows the transplanted interaction to reproduce the within-domain replacement trajectory more closely, particularly for HABCrH_ABC^r. Figure 28: Top-3 accuracy for cross-domain replacement after interaction-subspace alignment. As in the top-1 results in Figure 27, alignment improves agreement between cross-domain and within-domain replacement, especially for HABCrH_ABC^r, although the degree of recovery depends on the model and domain.