Paper deep dive
Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data
Marios Papamichalis, Regina Ruane
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/18/2026, 4:16:28 AM
Summary
This paper argues that standard attention metrics (cosine similarity, Jensen-Shannon divergence, Shannon entropy) are subcompositionally incoherent because they conflate 'sink' token attention (usually the first token) with 'content' token attention. The authors propose treating attention rows as compositional data using Aitchison geometry, which orthogonally separates sink mass from content distribution. This separation reveals that many observed phenomena, such as entropy collapse during training, are artifacts of the sink growing rather than attention sharpening. The Aitchison distance and its content component provide invariant, coherent metrics for comparing attention heads.
Entities (10)
Relation Signals (8)
Aitchison distance → separates → sink term
confidence 95% · the Aitchison distance splits orthogonally into a sink term and a content term
Aitchison distance → separates → content term
confidence 95% · the Aitchison distance splits orthogonally into a sink term and a content term
attention rows → aremodeledas → compositions
confidence 94% · This is precisely a composition in the sense of 1: a vector of positive parts carrying only relative information.
Compositional Data Analysis → provides → subcompositional coherence
confidence 93% · Compositional data analysis (CoDA) identifies the properties at stake: perturbation invariance and subcompositional coherence
sink token → causes → metric reversal
confidence 92% · This choice can reverse conclusions... 17--47% of verdicts about which of two heads is more similar flip with the convention
Shannon Entropy → conflates → content entropy
confidence 90% · Shannon entropy combines marginal sink mass with conditional content entropy
Shannon Entropy → conflates → sink mass
confidence 90% · Shannon entropy combines marginal sink mass with conditional content entropy
Entropy Collapse → iscausedby → sink growing
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Each row of a transformer's attention matrix is a probability distribution over tokens, and in trained models most of that probability lands on a single \emph{sink} token, usually the first. Standard tools for comparing attention rows (cosine similarity, Jensen--Shannon divergence, Shannon entropy) therefore hinge on a choice papers rarely report: keep the sink, or drop it and renormalize. This choice can reverse conclusions. On ten pretrained models from five families, 17--47% of verdicts about which of two heads is more similar flip with the convention, and the most prominent structure in a standard BERT head-clustering pipeline is an artifact of it. The reason is that one-number summaries mix two questions: how much attention the sink takes, and how the rest is divided among the content tokens. Treating rows as compositional data separates them exactly: the Aitchison distance splits orthogonally into a sink term and a content term, entropy splits by an exact identity, and the content distance is characterized by invariances the transformer itself possesses. The separation matters in practice: most measured entropy collapse during training is the sink growing, not attention sharpening (30% of the drop at 70M parameters, 95% at 1B, 79% at 1.4B), and pruning heads with the wrong channel can inflate perplexity more than a hundredfold. We map where each convention is safe, test a frozen out-of-sample predictor (one confirmation, one abstention, one failure), and release code regenerating every number.
Tags
Links
- Source: https://arxiv.org/abs/2608.14712v1
- Canonical: https://arxiv.org/abs/2608.14712v1
Trouble viewing inline? Open PDF directly →
Full Text
185,103 characters extracted from source content.
Expand or collapse full text
Which Question Is Your Attention Metric Answering? Attention Rows as Compositional DataThanks: Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, 3733 Spruce Street, Philadelphia, PA 19104-6340, ruanej@wharton.upenn.edu Marios Papamichalis Thanks: Human Nature Lab, Yale University, New Haven, CT 06511, marios.papamichalis@yale.edu Affiliation: Human Nature Lab, Yale University Affiliation: and Affiliation: Regina Ruane Affiliation: Department of Statistics and Data Science, The Wharton School, Affiliation: University of Pennsylvania Abstract Each row of a transformer’s attention matrix is a probability distribution over tokens, and in trained models most of that probability lands on a single sink token, usually the first. Standard tools for comparing attention rows (cosine similarity, Jensen–Shannon divergence, Shannon entropy) therefore hinge on a choice papers rarely report: keep the sink, or drop it and renormalize. This choice can reverse conclusions. On ten pretrained models from five families, 17–47% of verdicts about which of two heads is more similar flip with the convention, and the most prominent structure in a standard BERT head-clustering pipeline is an artifact of it. The reason is that one-number summaries mix two questions: how much attention the sink takes, and how the rest is divided among the content tokens. Treating rows as compositional data separates them exactly: the Aitchison distance splits orthogonally into a sink term and a content term, entropy splits by an exact identity, and the content distance is characterized by invariances the transformer itself possesses. The separation matters in practice: most measured entropy collapse during training is the sink growing, not attention sharpening (30% of the drop at 70M parameters, 95% at 1B, 79% at 1.4B), and pruning heads with the wrong channel can inflate perplexity more than a hundredfold. We map where each convention is safe, test a frozen out-of-sample predictor (one confirmation, one abstention, one failure), and release code regenerating every number. 1 Introduction Figure 1: The reversal in one construction (Theorem 1). Three attention rows over sink,a,b\sink,a,b\: p=(0.90,0.08,0.02)p=(0.90,0.08,0.02), q=(0.70,0.24,0.06)q=(0.70,0.24,0.06), r=(0.90,0.02,0.08)r=(0.90,0.02,0.08). (a) With the sink column, p and r share the same sink mass while q differs. (b) Dropping the sink and renormalizing, a routine, usually undocumented preprocessing choice, makes p′p and q′q identical (both allocate content 4:14:1 to a:ba:b) while r′r mirrors them. (c) Cosine, JS, and Euclidean distance all say r is the closer head with the sink and q without it: the scientific conclusion is an artifact of the convention. The Aitchison distance returns the same verdict in both views, and its content part dA⟂d_A is numerically identical. Softmax attention produces, for every query position, a probability distribution over keys, a point on the simplex. This is precisely a composition in the sense of 1: a vector of positive parts carrying only relative information. The interpretability and training-stability literatures routinely compare and summarize these rows. 4 cluster BERT heads by the JS divergence between attention rows; cosine similarity between attention maps is pervasive; 26 track the Shannon entropy of attention rows to diagnose entropy collapse; head-role taxonomies and pruning scores (23; 16; 18) summarize populations of attention rows with Euclidean averages. These statistics are valid for their intended targets, but full-row comparisons need not preserve conclusions after a coordinate is removed and the row is renormalized, and Shannon entropy combines marginal sink mass with conditional content entropy. Compositional data analysis (CoDA) identifies the properties at stake: perturbation invariance and subcompositional coherence (1; 2; 7). CoDA critiques of Euclidean statistics date back four decades. What makes them consequential here is a second, recent empirical fact: attention mass concentrates overwhelmingly on a few sink coordinates, chiefly the first or ⟨bos⟩ token (24; 22; 10). In a typical Llama-3.1-405B prompt roughly 80%80\% of the attention mass sits on the ⟨bos⟩ token (3), and the fraction of sink heads grows with model scale (10). A sink is exactly the kind of dominant shared part whose inclusion or exclusion CoDA warns changes Euclidean and divergence-based comparisons. Whether an analysis keeps the sink column, or masks it and renormalizes, is today an undocumented analyst choice, and this choice can determine the answer: two heads that divide their non-sink attention identically can be scored as distant, and a head with the opposite content preference ranked their nearest neighbor, purely because all three share a large sink (Figure 1). Two estimands. Full-row statistics answer questions about total allocation, sink included; sink-dropped statistics answer questions about allocation conditional on ignoring the sink. Both are legitimate; the failure mode is mixing them under an unstated convention, which is what one-number summaries do when a shared sink holds most of the mass, and these summaries feed head taxonomies, pruning decisions, and training-stability monitors. Our claim is not that the classical metrics are wrong but that they answer a convention-dependent question; we separate the channels exactly, characterize the content channel’s canonical metric from the transformer’s own symmetries, and measure what the separation changes. Concurrent 2026 work arrives at the same geometry for different questions (§2); the measurement problem, the sink/content separation, and the convention audit are, to our knowledge, new. Contributions. • Diagnosis. We identify the keep-versus-drop sink convention as an unreported analyst choice and prove it reverses cosine, JS, and Euclidean closer-head verdicts on a positive-measure set (Theorem 1). • Toolkit. An exact orthogonal split of the Aitchison distance into sink and content terms, an exact entropy identity H=Hb(s)+(1−s)H(π)H=H_b(s)+(1-s)H(π), invariance statements for the transformer’s own nuisance transformations, and a five-axiom characterization with a quotient version for the content distance (Lemmas 1, 2; Prop. 2; Thm. 2). • Failure geometry. In the sink-dominated regime all classical pairwise dissimilarities collapse into vanishing bands while the content diagnostics are unaffected (Theorem 3; Prop. 3). • Measurement. 1717–47%47\% of closer-head verdicts flip on ten pretrained models, tracking curves computed before any model was run; prospectively specified re-analyses on taxonomies (including Clark et al.’s pipeline on its full support), scale, and training checkpoints are scored against their predictions (§3–5.3). • Function. As pruning criteria every metric fails somewhere; we map the regimes across ten models, show each criterion is stable along a different axis, and test a frozen regime predictor out of sample, reporting one confirmation, one abstention, and one failure (§3). 2 Related work Attention-analysis metrics. 4 compute head–head distances as summed JS divergences between attention rows and cluster the result; 13 taxonomize attention patterns and study redundancy; cosine similarity between flattened attention maps is a default in many analyses. Entropy collapse and stability. 26 define attention entropy row-wise and tie pathologically low entropy to training instability; 6 study rank collapse of pure attention. Attention sinks and registers. StreamingLLM identifies the first-token sink and its role in windowed inference (24); 5 find high-norm register tokens in ViTs; 22 trace sinks to massive activations acting as implicit attention biases; 10 measure how the fraction of sink heads scales; 3 explain why models attend to the first token and quantify the mass involved. Head roles and pruning. 23 taxonomize heads and prune with L0L_0 gates; 16 prune most heads at test time; 18 characterize induction heads. Compositional data analysis. The log-ratio approach originates with 1; 2; ILR coordinates and balances come from 7; 20 is the standard treatment; zero handling follows 15; 9; 8 quantify and reappraise subcompositional incoherence. 21 bring log-contrasts to deep models for compositional inputs. Concurrent compositional views of attention. Three concurrent works arrive at this geometry from other directions. 11 introduce the Aitchison distance to quantify within-row token distinguishability in long contexts and derive a linear relation between temperature scaling and Aitchison distance, independently corroborating our powering axiom. 14 identifies the row-centered attention logit with the CLR transform and studies spectral invariants of the resulting field. 25 construct tree-aligned orthonormal bases of the Aitchison simplex and note the isomorphism between shift-equivalent logits and the CLR hyperplane. None studies the measurement problem addressed here: cross-head similarity, entropy diagnostics, taxonomies, and pruning under sink conventions (17 embed graph nodes as compositions for learned representations). Information geometry (Fisher–Rao) equips the simplex with a different metric with different invariances; it is not perturbation-invariant and does not decompose sink from content, which are the properties the sink problem demands (§6). To our knowledge, this is the first work to bring log-ratio geometry to the measurement methodology of attention analysis, and the first to separate the sink and content channels within it. 3 Aitchison geometry of attention rows Compositions. Let D=p∈ℝ>0D:∑ipi=1S^D=\p ^D_>0: _ip_i=1\ denote the (interior of the) simplex with D parts, and (x)=x/∑ixiC(x)=x/ _ix_i the closure operation. A softmax attention row over D keys is a point of DS^D (structural zeros from masking are handled below). The simplex is a (D−1)(D-1)-dimensional real vector space under perturbation p⊕q=(p1q1,…,pDqD)p q=C(p_1q_1,…,p_Dq_D) and powering α⊙p=(p1α,…,pDα)α p=C(p_1^α,…,p_D^α), with identity the uniform composition e and inverse ⊖p=(1/p1,…,1/pD) p=C(1/p_1,…,1/p_D) (1; 20). Writing g(p)g(p) for the geometric mean of the parts, the centered log-ratio is clr(p)=(logpig(p))i (p)= ( p_ig(p) )_i, mapping DS^D isomorphically onto the hyperplane ℋ=x∈ℝD:∑ixi=0H=\x ^D: _ix_i=0\, and an isometric log-ratio (ILR) map is ilr(p)=V⊤clr(p) (p)=V (p) for any orthonormal contrast basis V∈ℝD×(D−1)V ^D×(D-1), V⊤V=IV V=I, V⊤=0V 1=0 (7). The Aitchison inner product, norm, and distance are the Euclidean ones pulled back through clr (equivalently any ilr ): dA(p,q)=∥clr(p)−clr(q)∥2=∥ilr(p)−ilr(q)∥2.d_A(p,q)\;=\; (p)- (q) _2\;=\; (p)- (q) _2. (1) Subcompositions and coherence. For S⊆1,…,DS \1,…,D\ the subcomposition p(S)=((pi)i∈S)p^(S)=C ((p_i)_i∈ S ) discards the other parts and renormalizes, exactly what an analyst does when masking the sink column. Subcompositional coherence demands that conclusions about the parts in S not depend on whether the analysis was run on p or on p(S)p^(S); a dissimilarity is subcompositionally dominant if δ(p(S),q(S))≤δ(p,q)δ (p^(S),q^(S) )≤δ(p,q) (2; 9). In ILR coordinates, taking a subcomposition is an orthogonal projection (17, Lemma 3.1), so dAd_A is dominant and log-ratios among the parts of S are untouched by the operation. Cosine, JS, and Euclidean distance are neither subcompositionally coherent nor dominant, and none is stable under sink removal, as §4.1 makes constructive. Sink/content notation. We index the sink coordinate as 00 and the remaining content coordinates as 1,…,D−11,…,D-1; any row factors uniquely as p=(s,(1−s)π)p=(s,(1-s)π) with sink mass s=p0s=p_0 and content composition π∈D−1π ^D-1. We write p′=p(1,…,D−1)=πp =p^(\1,…,D-1\)=π for the sink-dropped, renormalized row. (Multiple sinks group identically; Appendix 5.) Structural zeros and masking. Causal masking creates structural zeros, which log-ratios forbid; the principled route is comparison on the common unmasked support, a subcomposition, hence an orthogonal projection, and our protocol additionally fixes the key support across aggregated query positions (Appendix 5). Residual numerical zeros are handled by multiplicative replacement with ε=10−6 =10^-6 (15; 19), and we report sensitivity to ε∈10−5,10−6,10−7 ∈\10^-5,10^-6,10^-7\. 4 Theory 4.1 The standard toolkit is subcompositionally incoherent Write c(p,q)c(p,q) for cosine similarity, JS(p,q)JS(p,q) for the Jensen–Shannon divergence (natural log), ∥p−q∥ p-q for Euclidean distance, and H(p)=−∑ipilogpiH(p)=- _ip_i p_i for Shannon entropy. Theorem 1 (Sink-driven ranking reversal). Fix D≥3D≥ 3, index rows x∈Dx ^D by 0,…,D−10,…,D-1 with coordinate 00 the sink, and write R0(x):=(x1,…,xD−1)/(1−x0)∈D−1R_0(x):=(x_1,…,x_D-1)/(1-x_0) ^D-1. Fix s′∈(0,1)s ∈(0,1) and π∈D−1π ^D-1, for each permutation τ of the content coordinates with τπ≠πτπ≠π and s∈(s′,1)s∈(s ,1) set ps:=(s,(1−s)π)p_s:=(s,(1-s)π), q:=(s′,(1−s′)π)q:=(s ,(1-s )π), and rs,τ:=(s,(1−s)τπ)r_s,τ:=(s,(1-s)τπ); fix one such σ and write π~:=σπ π:=σπ and rs:=rs,σr_s:=r_s,σ. Then: (i) R0(ps)=R0(q)=πR_0(p_s)=R_0(q)=π and R0(rs)=π~R_0(r_s)= π, so after sink removal q is strictly closer to psp_s than rsr_s under cosine, JSJS, Euclidean distance, and every metric on D−1S^D-1. (i) There is a threshold s∗=s∗(π,s′)∈(s′,1)s_ =s_ (π,s )∈(s ,1), uniform over all permutations τ with τπ≠πτπ≠π, such that for every s∈(s∗,1)s∈(s_ ,1) and every such τ the with-sink verdict is simultaneously reversed for all three classical measures: c(ps,rs,τ)>c(ps,q)c(p_s,r_s,τ)>c(p_s,q), JS(ps,rs,τ)<JS(ps,q)JS(p_s,r_s,τ)<JS(p_s,q), and ∥ps−rs,τ∥<∥ps−q∥ p_s-r_s,τ < p_s-q . (i) The set of triples in (D)3(S^D)^3 on which all three classical verdicts flip between the two conventions is open and has positive 3(D−1)3(D-1)-dimensional Lebesgue measure in the affine hull. (iv) dA⟂(x,y):=dA(R0(x),R0(y))d_A (x,y):=d_A(R_0(x),R_0(y)) is a pseudometric on DS^D (a metric on D/∼0S^D/ _0, x∼0yx _0y iff R0(x)=R0(y)R_0(x)=R_0(y)); on the family, dA⟂(ps,q)=0d_A (p_s,q)=0 and dA⟂(ps,rs)=dA(π,π~)=:B>0d_A (p_s,r_s)=d_A(π, π)=:B>0 under both conventions. Full dAd_A satisfies dA(ps,q)=αDlogs(1−s′)s′(1−s)d_A(p_s,q)= _D s(1-s )s (1-s) (αD=(D−1)/D _D= (D-1)/D) and dA(ps,rs)=Bd_A(p_s,r_s)=B, so it ranks q closer exactly when s<sAs<s_A for an explicit cutoff sA∈(s′,1)s_A∈(s ,1); the Table 1 witness satisfies s<sAs<s_A, which is why full dAd_A agrees across conventions there. (Proof: Appendix B.) Table 1: Exact witness for Theorem 1 with p=(0.90,0.08,0.02)p=(0.90,0.08,0.02), q=(0.70,0.24,0.06)q=(0.70,0.24,0.06), r=(0.90,0.02,0.08)r=(0.90,0.02,0.08): q shares p’s content ratio (4:14:1) and r shares p’s sink mass with mirrored content. “Closer” means larger cosine similarity or smaller dissimilarity. All three classical verdicts reverse; the JS and Euclidean values also violate subcompositional dominance (JS(p′,r′)>JS(p,r)JS(p ,r )>JS(p,r), ∥p′−r′∥>∥p−r∥ p -r > p-r ). with sink sink dropped metric d(p,q)d(p,q) d(p,r)d(p,r) closer d(p′,q′)d(p ,q ) d(p′,r′)d(p ,r ) closer cosine 0.96930.9693 0.99560.9956 r 1.00001.0000 0.47060.4706 q reversal JS 0.03240.0324 0.01930.0193 r 0.00000.0000 0.19270.1927 q reversal Euclidean 0.25920.2592 0.08490.0849 r 0.00000.0000 0.84850.8485 q reversal dAd_A 1.10221.1022 1.96051.9605 q 0.00000.0000 1.96051.9605 q stable The witness entries are exact rationals; every displayed quantity is evaluated in closed form in Appendices B and C. Proposition 1 (Entropy inversion under sink removal). For every D≥3D≥ 3 there exist p,q∈Dp,q ^D with H(p)<H(q)H(p)<H(q) but H(p′)>H(q′)H(p )>H(q ): entropy-collapse comparisons, across heads, layers, or training steps, can invert with the analyst’s convention. A witness at D=3D=3 is P=(1920,140,140)P= ( 1920, 140, 140 ), Q=(12,9992000,12000)Q= ( 12, 9992000, 12000 ): H(P)≈0.233<0.697≈H(Q)H(P)≈ 0.233<0.697≈ H(Q), yet H(P′)=log2>H(Q′)≈0.0079H(P )= 2>H(Q )≈ 0.0079. The convention-invariant content statistics agree with the conditional view: H(P′)=log2H(P )= 2 is maximal and CA⟂(P)=0C_A (P)=0 (content exactly uniform), versus H(Q′)≈0.008H(Q )≈ 0.008 and CA⟂(Q)≈4.88C_A (Q)≈ 4.88 (content nearly degenerate). (Proof and sharpness: Appendix B.) The mechanism is the chain rule H(p)=Hb(s)+(1−s)H(π)H(p)=H_b(s)+(1-s)H(π) (Appendix B): a growing sink drives H down regardless of what the content distribution does, so H conflates two unrelated quantities, sink strength and content spread, that the Aitchison geometry separates exactly (Lemma 1). The constructive remedy is to report the pair (s,H(π))(s,H(π)) in place of H(p)H(p); CA⟂C_A is a complementary log-ratio dispersion of the same content channel, not an entropy substitute (Remark 1). 4.2 A coherent toolkit, and why it is essentially unique Definition 1 (Coherent attention diagnostics). For rows p,q∈Dp,q ^D with sink part 00: (a) the head signature of a set of rows pt\p_t\ is their Aitchison mean cen(pt)=(exp(1T∑tlogpt)) (\p_t\)=C ( ( 1T _t p_t ) ), the Fréchet mean under dAd_A; (b) head dissimilarity is dAd_A of (1), and the direction similarity is the ILR-cosine ρA(p,q)=⟨clrp,clrq⟩∥clrp∥∥clrq∥ _A(p,q)= p, q p q ; (c) the sink balance is b(p)=D−1Dlogp0g(p1,…,pD−1)b(p)= D-1D\, p_0g(p_1,…,p_D-1); (d) the content distance is dA⟂(p,q)=dA(p′,q′)d_A (p,q)=d_A(p ,q ) and the content log-ratio dispersion is CA⟂(p)=∥clr(p′)∥2=dA(p′,e)C_A (p)= (p ) _2=d_A(p ,e), the Aitchison distance of the content to uniform (it orders by log-ratio dispersion, not by majorization; Remark 1); (e) dispersion of a set of rows is the total Aitchison variance 1n∑idA(pi,cen)2 1n _id_A (p_i, )^2. Lemma 1 (Pythagorean sink decomposition). Let D≥2D≥ 2, p,q∈Dp,q ^D with designated sink 00, and b(⋅)b(·) the sink balance of Definition 1(c). In CLR space the exact orthogonal identity clr(p)=b(p)ub+Jclr(p′) (p)=b(p)\,u_b+J (p ) holds, where ubu_b is the unit balance direction and J is the isometric zero-extension of the content CLR space; hence dA(p,q)2=dA⟂(p,q)2+(b(p)−b(q))2.d_A(p,q)^2=d_A (p,q)^2+ (b(p)-b(q) )^2. There is a sink-adapted ILR basis whose first coordinate is b(⋅)b(·) and whose remaining D−2D-2 coordinates are ILR coordinates of the sink-dropped composition. Consequently dA⟂d_A and CA⟂C_A depend only on the content composition: varying the sink mass with the content held fixed changes neither, and both take identical values whether computed in the balance-orthogonal coordinates of the full rows or after dropping the sink and re-closing. (Proof: Appendix C.) Proposition 2 (Invariances and equivariances). With softmax rows p=(eℓ)p=C(e ), q=(eℓ′)q=C(e ): (i) a shared finite logit bias u acts as perturbation by cu=(eu)c_u=C(e^u), and dAd_A and dA⟂d_A are invariant under it; CA⟂C_A (for D≥3D≥ 3), Euclidean distance, JSJS, cosine, and ρA _A are not. (i) a temperature τ acts as powering by 1/τ1/τ: dAd_A, dA⟂d_A , and CA⟂C_A scale by 1/τ1/τ (so one common temperature preserves all rankings and ties), total Aitchison variance scales by 1/τ21/τ^2, and ρA _A is invariant to separate positive per-head temperatures applied uniformly within each head (row-dependent temperatures can change it). (i) sink coherence: dA⟂(p,q)d_A (p,q) and CA⟂(p)C_A (p) are independent of the sink masses; full dAd_A is not sink-invariant but decomposes exactly (Lemma 1). (iv) a subcomposition on S (|S|≥2|S|≥ 2) is an orthogonal projection in CLR space, so dA(p(S),q(S))≤dA(p,q)d_A(p^(S),q^(S))≤ d_A(p,q); hard masks require re-closure on a common positive support; and (v) for D≥3D≥ 3, Euclidean distance, JSJS, and 1−cos1- are not subcompositionally dominant. (Full statements with all qualifications, proofs, and scope: Appendix C.) The axioms are the transformer’s own knobs. Given the operational meaning the transformer itself assigns to ⊕ (shared logit biases) and ⊙ (temperature), four interpretable axioms force the Aitchison distance, and the invariance axioms among them are transformations the architecture performs. A quotient version characterizes the content distance itself (Lemma 2). Theorem 2 (Characterization of the Aitchison distance). Let δ:D×D→ℝ≥0δ:S^D×S^D _≥ 0 satisfy: (A1) δ is a metric; (A2) perturbation invariance: δ(c⊕p,c⊕q)=δ(p,q)δ(c p,\,c q)=δ(p,q) for all c; (A3) powering homogeneity: δ(α⊙p,α⊙q)=|α|δ(p,q)δ(α p,\,α q)=|α|\,δ(p,q) for all α∈ℝα ; (A4) the parallelogram law: writing N(v)=δ(v,e)N(v)=δ(v,e), N(v⊕w)2+N(v⊖w)2=2N(v)2+2N(w)2\;N(v w)^2+N(v w)^2=2N(v)^2+2N(w)^2; (A5) permutation invariance: δ(σp,σq)=δ(p,q)δ(σ p,σ q)=δ(p,q) for all σ∈SDσ∈ S_D. Then δ=cdAδ=c\,d_A for some constant c>0c>0. The proof (Appendix C) reduces δ to a norm via A2, upgrades it to an inner-product norm via A4 by 12, and pins the inner product to a multiple of the CLR one via A5 and irreducibility of the permutation representation on the sum-zero hyperplane. The axioms are tight: N1(v)=∥clrv∥1N_1(v)= v _1 satisfies A1–A3 and A5 but not A4 (Remark C.3). A2 and A3 are exactly invariance to shared logit biases and consistency under temperature, the two nuisance transformations the transformer parameterization itself supplies. Cross-dimensional comparisons need one common calibration constant, under which dominance, the sink decomposition, and the bias/temperature behavior follow rather than being assumed (Corollary 2, Remark C.2). 4.3 Consequences: dynamic range, taxonomies, and training curves Three consequences sharpen what §5 measures. Theorem 3 (Dynamic-range collapse in the sink-dominated regime). Fix D≥3D≥ 3 and s0∈(12,1)s_0∈( 12,1). (i) If p,q∈Dp,q ^D have sink masses at least s0s_0, then, strictly, 1−c(p,q)<(1−s0)2s02+(1−s0)2,∥p−q∥<2(1−s0),JS(p,q)<(1−s0)log2,1-c(p,q)< (1-s_0)^2s_0^2+(1-s_0)^2, p-q < 2\,(1-s_0), (p,q)<(1-s_0) 2, and each right-hand side is the exact, nonattained supremum (the attainable values fill [0,⋅)[0,·)); the cosine band shrinks quadratically, ∼(1−s0)2 (1-s_0)^2. (i) The content-preserving sink transformations Ttr:=(t,(1−t)r′)T_tr:= (t,(1-t)r ) leave the entire dA⟂d_A dissimilarity matrix and all CA⟂C_A values of any finite head family exactly unchanged; in particular, sinks can strengthen arbitrarily while the content geometry stays fixed. (Proof: Appendix D.) Concretely, heads with sink mass ≥0.8≥ 0.8 (Llama-3.2-1B’s mean is 0.840.84) have all pairwise cosine similarities above 0.940.94, Euclidean distances below 0.290.29, and JSJS below 0.140.14 nats: the classical dynamic range shrinks into a band comparable to estimation noise, so the orderings inside it, nearest neighbors, redundancy ranks, cluster merges, carry vanishing information (a mechanism candidate for the pruning catastrophe of §3). Proposition 3 (Taxonomies: stability and instability). (i) For any finite labelled head set, the dA⟂d_A dissimilarity matrix computed with the sink retained equals, entrywise, the dAd_A matrix of the explicitly sink-dropped heads. Hence every clustering rule, deterministic or randomized, whose sole data-dependent input is this labelled matrix returns identical outputs in the two analyses (equally, in distribution and under a common coupling, when randomized). (i) There is a nonempty open set of head triples of positive 3(D−1)3(D-1)-dimensional measure on which, simultaneously for 1−cos1- , JSJS, and Euclidean distance, the unique closest pair is (p,r)(p,r) with the sink and (p,q)(p,q) after the drop; single- and complete-linkage clustering at K=2K=2 therefore return p,rq\p,r\\q\ versus p,qr\p,q\\r\. The exact rational witness is Table 1. (Proof: Appendix D.) Corollary 1 (Checkpoint entropy decomposition and sink-only collapse). Along any checkpoint family pt=(st,(1−st)πt)p_t= (s_t,(1-s_t) _t ), H(pt)=Hb(st)+(1−st)H(πt)H(p_t)=H_b(s_t)+(1-s_t)H( _t) pointwise. If the content is fixed, πt≡π _t≡π, then CA⟂(pt)C_A (p_t) is constant while s↦H(s,(1−s)π)s H (s,(1-s)π ) is strictly decreasing on [12,1)[ 12,1) (its unique maximizer is θπ=(1+eH(π))−1<12 _π= (1+e^H(π) )^-1< 12); with uniform content the apparent collapse can approach logD D with CA⟂≡0C_A ≡ 0. This is a possibility result, not a claim that sink mass grows along any empirical trajectory. (Proof: Appendix D.) §5.3 turns all three into prospectively specified measurements. 5 Experiments All numbers in this section are exact evaluations, Monte Carlo estimates, or single-seed model measurements produced by the released code with fixed seeds; §5.1 isolates the mechanism in silico and §3 measures it on pretrained models. 5.1 Exact constructions and calibrated simulations Exact witnesses. Table 1 and Figure 1 evaluate the Theorem 1 witness; Proposition 1 is instantiated in its statement; every invariance claim of Proposition 2 is verified to machine precision in the released tests. How often do verdicts flip? Figure 3 samples head triples with sink structure calibrated to reported statistics (3; 10) and asks the basic analyst’s question, which of q,rq,r is closer to p?, under both sink conventions. At mean sink mass s¯=0.5/0.7/0.9 s=0.5/0.7/0.9 the conventions disagree on 36/39/41%36/39/41\% (cosine), 23/31/40%23/31/40\% (JS), and 27/37/45%27/37/45\% (Euclidean) of triples, growing with sink mass toward the 50%50\% chance level. The content distance dA⟂d_A is identical in both pipelines by Lemma 1, hence 0%0\% everywhere. The positive-measure region of Theorem 1 is not thin at realistic sink levels; stars overlay the §3 measurements at each model’s signature sink mass. Figure 2: Entropy conflates sink strength with content spread; CA⟂C_A separates them. Synthetic 12-layer models, 4,0004,000 rows per layer. (a) Sink mass grows 0.06→0.850.06→ 0.85 with content log-ratio dispersion held fixed: mean Shannon entropy falls 2.71→0.822.71→ 0.82 nats (−70%-70\%, a textbook “entropy collapse”) while CA⟂C_A moves 48.1→48.048.1→ 48.0 (−0.3%-0.3\%): the collapse is phantom. (b) Content genuinely collapses while the sink recedes 0.70→0.050.70→ 0.05: entropy drifts 1.73→1.271.73→ 1.27 (−27%-27\%, reads as mild) while CA⟂C_A rises 10.0→70.410.0→ 70.4 (7×7×): the real collapse is masked. Collapse diagnostics. Figure 2 constructs two 12-layer scenarios. In Scenario (a), only the sink grows; Shannon entropy reports a 70%70\% collapse while content log-ratio dispersion is flat, exactly the failure Proposition 1 predicts, and the mode to which entropy-collapse monitoring (26) is exposed when sinks strengthen (10). In Scenario (b), the content genuinely collapses while the sink recedes; entropy barely moves. CA⟂C_A responds in both cases because it is a statistic of the content subcomposition only. 5.2 Pretrained models across five model families Figure 3: Calibrated reversal curves and ten measured models. Curves: Monte Carlo closer-head disagreement rates between sink conventions for cosine, JS, and Euclidean distance, computed before any pretrained model was run (solid α=0.1α=0.1, dashed α=1α=1). Stars: the ten models, placed at the sink mass of the signatures the distances are computed on; mean absolute deviation from the solid curves is 4.84.8 points (max 1313). The dA⟂d_A rates are zero identically (Lemma 1), plotted as a code check. Protocol (released; two consumer GPUs, a few hours, no training; ten models, GPT-2, Pythia 70M–1.4B, Qwen2.5-1.5B, Llama-3.2-1B/3B, ViT-B/16; three resampling seeds each, varying data order and triple sampling for one fixed public checkpoint per model). Extraction: forward hooks on the attention softmax; for LMs, query positions t∈[w,T]t∈[w,T] restricted to the first W=64W=64 keys (a fixed common support; no zero imputation needed), with key 00 (the ⟨bos⟩ /first token) as the sink column; for ViT-B/16, the CLS column as sink. Aggregation: per-head Aitchison mean over query positions and inputs (Definition 1(a)). Measurements: (1) closer-head ranking-reversal rates over 50,00050,000 head triples between the with-sink and sink-dropped conventions, for cosine/JS/Euclidean versus dA⟂d_A ; (2) layer-wise mean H, H(π)H(π), and sink mass; (3) redundancy pruning at matched sparsity, prune the m∈10,20,30%m∈\10,20,30\\% most redundant heads (nearest-neighbor distance) under JS, under the content distance dA⟂d_A , and under the full dAd_A (sink balance included), comparing perplexity (LMs) and top-1 agreement with the unpruned model (ViT, so no labels are needed); (4) an ε sweep over 10−5,10−6,10−7\10^-5,10^-6,10^-7\. Data: 2,0002,000 WikiText-103 validation sequences (LMs), 2,0482,048 COCO val2017 images (ViT); details in Appendix 5. Table 2: Measured convention-dependence on ten pretrained models (mean± over three seeds). Reversal == % of 50,00050,000 head triples whose closer-head verdict flips between sink conventions (bit-identical across the ε sweep). Rates rise with signature sink mass (Figure 3). Hell./FR: Hellinger reversal; Fisher–Rao is identical since both are strictly monotone in the Bhattacharyya coefficient, so every ranking verdict coincides. Collapse disagr.: fraction of adjacent-layer transitions on which H and the content entropy H(π)H(π) move in opposite directions. Sink (row/sig.): arithmetic row-level mean and the sink mass of the Aitchison-mean signatures the distances use. The content-distance column is omitted: its rate is zero by construction (Lemma 1). ranking-reversal rate Model sink (row/sig.) cosine JS Euclid. Hell./FR collapse disagr. GPT-2 (124M) 0.68/0.780.68/0.78 36.1±0.136.1 ± 0.1% 35.0±0.235.0 ± 0.2% 38.1±0.238.1 ± 0.2% 34.4±0.234.4 ± 0.2% 27.3±0.027.3 ± 0.0% Llama-3.2-1B 0.84/0.940.84/0.94 46.1±0.246.1 ± 0.2% 44.1±0.244.1 ± 0.2% 47.2±0.347.2 ± 0.3% 43.6±0.243.6 ± 0.2% 46.7±0.046.7 ± 0.0% Qwen2.5-1.5B 0.68/0.820.68/0.82 43.9±0.143.9 ± 0.1% 42.5±0.142.5 ± 0.1% 44.3±0.144.3 ± 0.1% 42.3±0.142.3 ± 0.1% 29.6±0.029.6 ± 0.0% Llama-3.2-3B 0.86/0.950.86/0.95 45.0±0.345.0 ± 0.3% 43.7±0.343.7 ± 0.3% 46.7±0.346.7 ± 0.3% 43.5±0.343.5 ± 0.3% 37.0±0.037.0 ± 0.0% ViT-B/16 0.04/0.070.04/0.07 22.4±0.122.4 ± 0.1% 17.3±0.117.3 ± 0.1% 23.2±0.123.2 ± 0.1% 18.1±0.118.1 ± 0.1% 9.1±0.09.1 ± 0.0% Pythia-70M 0.08/0.430.08/0.43 33.5±0.233.5 ± 0.2% 30.6±0.230.6 ± 0.2% 31.0±0.231.0 ± 0.2% 30.3±0.230.3 ± 0.2% 40.0±0.040.0 ± 0.0% Pythia-160M 0.28/0.690.28/0.69 43.2±0.243.2 ± 0.2% 43.9±0.243.9 ± 0.2% 44.3±0.344.3 ± 0.3% 43.4±0.243.4 ± 0.2% 18.2±0.018.2 ± 0.0% Pythia-410M 0.52/0.740.52/0.74 43.5±0.243.5 ± 0.2% 42.4±0.242.4 ± 0.2% 43.8±0.243.8 ± 0.2% 42.1±0.242.1 ± 0.2% 21.7±0.021.7 ± 0.0% Pythia-1B 0.59/0.700.59/0.70 41.0±0.241.0 ± 0.2% 40.3±0.240.3 ± 0.2% 41.3±0.241.3 ± 0.2% 40.3±0.140.3 ± 0.1% 33.3±0.033.3 ± 0.0% Pythia-1.4B 0.64/0.800.64/0.80 41.7±0.241.7 ± 0.2% 40.4±0.140.4 ± 0.1% 42.4±0.242.4 ± 0.2% 40.1±0.140.1 ± 0.1% 30.4±0.030.4 ± 0.0% Table 3: Redundancy pruning at 20%20\% sparsity, all ten models (mean± , three resampling seeds; 10/20/30%10/20/30\% sweep in Table 10; random and total-dAd_A baselines in Table 11). LMs: perplexity, lower is better; ViT: top-1 agreement with the unpruned model, a stability rather than accuracy metric, higher is better. JS′ is JS on the sink-dropped rows; Hellinger on the content rows selects identical prune sets (Appendix E.1). “-L1” restricts neighbors to the same layer and defers a head whose nearest neighbor is already selected (one representative of each mutual pair is retained). Bold: best of the five criteria; the regime structure is described in the text. Model base JS JS′ dA⟂d_A JS-L1 dA⟂d_A -L1 GPT-2 (124M) 43.5±0.443.5 ± 0.4 74.5±0.774.5 ± 0.7 106.3±0.9106.3 ± 0.9 104.7±0.7104.7 ± 0.7 70.6±0.770.6 ± 0.7 448.5±271.1448.5 ± 271.1 Llama-3.2-1B 15.3±0.115.3 ± 0.1 2094.1±139.72094.1 ± 139.7 70.5±9.670.5 ± 9.6 63.6±4.163.6 ± 4.1 616.6±30.2616.6 ± 30.2 45.0±3.545.0 ± 3.5 Qwen2.5-1.5B 14.8±0.114.8 ± 0.1 111.5±0.7111.5 ± 0.7 28.1±0.228.1 ± 0.2 30.3±0.730.3 ± 0.7 34.3±1.134.3 ± 1.1 26.9±1.126.9 ± 1.1 Llama-3.2-3B 12.1±0.112.1 ± 0.1 310.8±3.8310.8 ± 3.8 25.3±0.125.3 ± 0.1 40.2±1.640.2 ± 1.6 1733.2±358.71733.2 ± 358.7 23.4±3.023.4 ± 3.0 ViT-B/16 1.0001.000 0.578±0.0030.578 ± 0.003 0.557±0.0150.557 ± 0.015 0.566±0.0130.566 ± 0.013 0.572±0.0040.572 ± 0.004 0.581±0.0060.581 ± 0.006 Pythia-70M 724.4±10.7724.4 ± 10.7 661.7±16.5661.7 ± 16.5 1253.6±19.01253.6 ± 19.0 1354.5±16.11354.5 ± 16.1 1002.3±15.91002.3 ± 15.9 1340.3±24.51340.3 ± 24.5 Pythia-160M 131.4±0.7131.4 ± 0.7 142.1±1.8142.1 ± 1.8 736.8±15.7736.8 ± 15.7 721.5±25.8721.5 ± 25.8 150.4±11.3150.4 ± 11.3 476.4±58.6476.4 ± 58.6 Pythia-410M 32.4±0.132.4 ± 0.1 75.2±0.575.2 ± 0.5 135.7±11.4135.7 ± 11.4 194.8±5.0194.8 ± 5.0 50.7±0.750.7 ± 0.7 77.1±13.777.1 ± 13.7 Pythia-1B 21.8±0.121.8 ± 0.1 44.9±0.144.9 ± 0.1 63.0±1.963.0 ± 1.9 63.0±1.963.0 ± 1.9 37.6±0.537.6 ± 0.5 47.4±8.347.4 ± 8.3 Pythia-1.4B 20.4±0.120.4 ± 0.1 33.4±0.333.4 ± 0.3 593.4±41.4593.4 ± 41.4 377.8±8.8377.8 ± 8.8 28.1±0.128.1 ± 0.1 41.6±2.241.6 ± 2.2 Figure 4: Measured layer-wise collapse decomposed. Mean Shannon entropy H (red) versus content entropy H(π)H(π) (green), sink mass dotted (seed 0); shaded bands mark adjacent-layer transitions on which the two disagree in sign. Where the sink strengthens with depth, H reports deepening collapse that the content channel does not show: Proposition 1’s conflation observed in pretrained models. Results: measurement. Table 3 confirms the theory on ten pretrained models across five families. Row-level sink masses on the fixed 6464-key support span 0.040.04 (ViT CLS) to 0.860.86, corroborating 3; within each family sink mass grows with scale. The signatures the distances are computed on concentrate further, geometric averaging amplifying a shared sink (row/signature masses in Table 3; Pythia-70M 0.08→0.430.08→ 0.43). Indexed by the signature sink, reversal rates rise from 1717–23%23\% on ViT to 4444–47%47\%, approaching the 50%50\% chance level, on the Llamas, and track the calibrated curves of Figure 3 with mean absolute deviation 4.84.8 points (max 1313); reversal under dA⟂d_A is zero by construction (verified to the bit); classical rates are ε -invariant with resampling sd ≤0.4≤ 0.4 points. The support choice is not decisive: the window retains 4848–71%71\% of row mass, reversal moves at most 22 points across W∈32,64,128W∈\32,64,128\, and dA⟂d_A distances correlate at ρ≥0.87ρ≥ 0.87 across windows (Appendix E.3). Entropy and the content entropy H(π)H(π) disagree in sign on 99–47%47\% of layer transitions across models (Figure 4 shows three). (A measured Llama triple realizes Theorem 1 verbatim: Fig. 7.) Appendix E.1 audits the estimator (tail coordinates carry ≤4.6%≤ 4.6\% of squared dA⟂d_A ; dropped-row JS agrees on coarse ranking but not on nearest neighbors or taxonomies). Figure 5: Prospectively specified downstream outcomes. (a) reversal tracks sink mass, which strengthens with scale within each family. (b) at Pythia-410M H collapses while H(π)H(π) barely moves; the share 1−ΔH(π)/ΔH1- H(π)/ H is 30/53/70/95/79%30/53/70/95/79\% across 70M–1.4B, rising to 95%95\% at 1B (79%79\% at 1.4B). Results: pruning. Table 3 is a regime map. Under the global protocol, conditioning out the sink is the decisive step and the choice of geometry within the content channel is secondary: on Llama-3.2-1B, with-sink JS collapses (15.3→2094±14015.3→ 2094±140) while dropped-row JS (70.5±9.670.5±9.6) and dA⟂d_A (63.6±4.163.6±4.1) both avoid the failure, and dropped-row JS wins outright on Llama-3.2-3B (25.325.3 versus 40.240.2); the full dAd_A, re-mixing the sink channel, fails like JS (2480±2112480±211), and random orderings beat global dA⟂d_A on 1515 of 3030 sweep cells. The separation appears under a protocol closer to practice (same-layer neighbors, one representative kept per redundant pair): dA⟂d_A -L1 is the best of all nine criteria on every strong-sink model (45.045.0, 26.926.9, 23.423.4 on Llama-1B/Qwen/Llama-3B; 0.5810.581 on ViT), whereas each classical criterion fails severely on at least one of them (JS′-L1 reaches 26422642 on Llama-3.2-1B; JS-L1 reaches 17331733 on Llama-3.2-3B). dA⟂d_A -L1 in turn fails on GPT-2 (449449), and with-sink JS(-L1) wins wherever sinks are weak. Across resampling seeds the dA⟂d_A -L1 sets are the less stable (Jaccard 0.360.36–0.770.77 versus 0.920.92–1.001.00 for JS-L1) even where their outcomes are stable: each criterion is stable along a different axis. Sink mass alone does not predict the regime (GPT-2 and Qwen2.5 share s=0.68s=0.68 with opposite verdicts); the cross-head dispersion of the sink share separates all nine LMs (coefficient of variation, CV, ≤0.36≤ 0.36 in the content regime, ≥0.42≥ 0.42 in the JS regime); a frozen out-of-sample test on three unseen models returned one confirmation, one boundary abstention, and one failure, so the rule is presented as descriptive, not validated (Appendix E.2). The experiments support two claims: measurements feeding any criterion are convention-dependent under classical metrics (Table 3); which channel carries the functional signal is model-dependent, a question the decomposition makes precise. Two stress tests separate the Aitchison choice from simply dropping the sink: rescaling the content logits by an unobservable temperature τ∈[0.5,2]τ∈[0.5,2] leaves every dA⟂d_A verdict and prune set bit-identical (Proposition 2) while dropped-row JS changes up to 38%38\% of its top-20%20\% prune-set membership, and across sink definitions (first-1 versus first-4) dA⟂d_A is the less sensitive on every model, with nested-projection dominance verified (Appendix E.2). 5.3 Prospectively specified downstream tests Are conclusions built on these measurements convention-dependent too? Three predictions were stated before the runs (in the released repository; no external registry); we report each outcome. (1) Taxonomies. Clark-style hierarchical head clustering (agglomerative, K∈4,6,8K∈\4,6,8\) on BERT-base, GPT-2, and Llama-3.2-1B (4). Prediction (Proposition 3): classical between-convention ARIs sit materially below 11, decreasing with sink mass; the dA⟂d_A partitions are identical by construction. Outcome: at 6464 keys, ARIs sit at or below chance on all three (Table 5). On the full 128128-token support with 4’s own pipeline (JS, average linkage), cross-convention ARI is 0.010.01–0.050.05, and the SEP-head block (6868/144144 heads) drops from Jaccard 0.870.87–0.920.92 to ≈0.48≈ 0.48 when the sink is dropped (Appendix E.3): its most prominent structure is the convention. (2) Scale. Sink mass and reversal rate across Pythia 70M–1.4B; prediction (Theorem 3; 10): both increase with parameters. Outcome: sink mass is monotone within each family and reversal tracks it near the pre-computed curves (Fig. 5a). (3) Training curves. H versus CA⟂C_A across Pythia checkpoints; prediction (Corollary 1): during sink emergence H falls with CA⟂C_A approximately flat. Outcome on the frozen endpoint: not confirmed. CA⟂C_A changes by 57/72/85%57/72/85\% at 70M/160M/410M (16%16\% at 1B, 42%42\% at 1.4B), failing at the three smaller sizes and approximately holding at 1B. Exploratory analysis via the exact identity H=Hb(s)+(1−s)H(π)H=H_b(s)+(1-s)H(π) (Fig. 5b): at 410M H collapses 3.63→1.523.63→ 1.52 while H(π)H(π) moves 0.60.6 nats, and at 1B H(π)H(π) moves 0.070.07 nats through a 1.61.6-nat collapse; the share 1−ΔH(π)/ΔH1- H(π)/ H (the fraction of the drop not attributable to a fall in H(π)H(π)) is 30/53/70/95/79%30/53/70/95/79\% across 70M–1.4B, rising to 95%95\% at 1B and remaining high (79%79\%) at 1.4B (signed decomposition: Appendix E.2). 6 Conclusion Attention rows are compositions, and the field’s standard summaries mix a sink question with a content question under an unreported convention. We separated the channels exactly, showed the separation changes published-style conclusions (verdict flips near chance, a clustering whose main structure is the convention, a collapse that is mostly sink), and mapped when each channel matters functionally. What the theory does and does not claim. Theorems 1–2 are statements about measurement: for relative allocation among content tokens, the classical toolkit answers a convention-dependent question, the Aitchison toolkit a well-posed one. We make no claim that computation “uses” log-ratios. Limitations. The scale law rests on one family; the frozen regime test returned one confirmation, one abstention, one failure (OPT-125M: uniformly high sink, benign pruning, a third behavior the map lacks); dA⟂d_A -L1 selects unstable sets across resampling; audits are signature-level rather than per-row. Future work: fitting dλ2=(dA⟂)2+λ(Δb)2d_λ^2=(d_A )^2+λ( b)^2 from held-out functional evidence, per-row audits, and value-weighted extensions. References Aitchison (1982) J. Aitchison The statistical analysis of compositional data. Journal of the Royal Statistical Society: Series B (Methodological) 44 (2), p. 139–160. Cited by: §1, §2, §3. Aitchison (1992) J. Aitchison On criteria for measures of compositional difference. Mathematical Geology 24 (4), p. 365–379. Cited by: §1, §2, §3. Barbero et al. (2025) F. Barbero, A. Arroyo, X. Gu, C. Perivolaropoulos, M. Bronstein, P. Veličković, and R. Pascanu Why do llms attend to the first token?. arXiv preprint arXiv:2504.02732. Cited by: §E.5, §1, §2, §5.1, §5.2. Clark et al. (2019) K. Clark, U. Khandelwal, O. Levy, and C. D. Manning What does bert look at? an analysis of bert’s attention. In Proceedings of the 2019 ACL workshop BlackboxNLP: analyzing and interpreting neural networks for NLP, p. 276–286. Cited by: §E.3, §1, §2, §5.3. Darcet et al. (2024) T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski Vision transformers need registers. In International conference on learning representations, Vol. 2024, p. 2632–2652. Cited by: §E.5, §2. Dong et al. (2021) Y. Dong, J. Cordonnier, and A. Loukas Attention is not all you need: pure attention loses rank doubly exponentially with depth. In International conference on machine learning, p. 2793–2803. Cited by: §2. Egozcue et al. (2003) J. J. Egozcue, V. Pawlowsky-Glahn, G. Mateu-Figueras, and C. Barcelo-Vidal Isometric logratio transformations for compositional data analysis. Mathematical geology 35 (3), p. 279–300. Cited by: §1, §2, §3. Greenacre et al. (2023) M. Greenacre, E. Grunsky, J. Bacon-Shone, I. Erb, and T. Quinn Aitchison’s compositional data analysis 40 years on: a reappraisal. Statistical Science 38 (3), p. 386–410. Cited by: Remark C.3, §2. Greenacre (2011) M. Greenacre Measuring subcompositional incoherence. Mathematical Geosciences 43 (6), p. 681–693. Cited by: §2, §3. Gu et al. (2025) X. Gu, T. Pang, C. Du, Q. Liu, F. Zhang, C. Du, Y. Wang, and M. Lin When attention sink emerges in language models: an empirical view. In International Conference on Learning Representations, Vol. 2025, p. 97114–97144. Cited by: §E.5, §1, §2, §5.1, §5.1, §5.3. [11] S. Hilton-Jones, T. J. Norman, and Z. Zhu Modelling attention with aitchison geometry: token distinguishability and temperature scaling. In Forty-third International Conference on Machine Learning, Cited by: §2. Jordan and Neumann (1935) P. Jordan and J. v. Neumann On inner products in linear, metric spaces. Annals of Mathematics 36 (3), p. 719–723. Cited by: §4.2. Kovaleva et al. (2019) O. Kovaleva, A. Romanov, A. Rogers, and A. Rumshisky Revealing the dark secrets of bert. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 4365–4374. Cited by: §2. Lee (2026) W. Lee On the invariants of softmax attention. arXiv preprint arXiv:2605.02907. Cited by: §2. Martín-Fernández et al. (2003) J. A. Martín-Fernández, C. Barceló-Vidal, and V. Pawlowsky-Glahn Dealing with zeros and missing values in compositional data sets using nonparametric imputation. Mathematical Geology 35 (3), p. 253–278. Cited by: §2, §3. Michel et al. (2019) P. Michel, O. Levy, and G. Neubig Are sixteen heads really better than one?. Advances in neural information processing systems 32. Cited by: §1, §2. Nakis et al. (2026) N. Nakis, C. Kosma, P. Promponas, M. Chatzianastasis, and G. Nikolentzos Aitchison embeddings for learning compositional graph representations. arXiv preprint arXiv:2605.00716. Cited by: §2, §3. Olsson et al. (2022) C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.11895. Cited by: §1, §2. Palarea-Albaladejo and Martín-Fernández (2015) J. Palarea-Albaladejo and J. A. Martín-Fernández ZCompositions—r package for multivariate imputation of left-censored data under a compositional approach. Chemometrics and Intelligent Laboratory Systems 143, p. 85–96. Cited by: §3. Pawlowsky-Glahn et al. (2015) V. Pawlowsky-Glahn, J. J. Egozcue, and R. Tolosana-Delgado Modeling and analysis of compositional data. Cited by: §2, §3. Quinn et al. (2020) T. Quinn, D. Nguyen, S. Rana, S. Gupta, and S. Venkatesh Deepcoda: personalized interpretability for compositional health data. In International Conference on Machine Learning, p. 7877–7886. Cited by: §2. Sun et al. (2024) M. Sun, X. Chen, J. Z. Kolter, and Z. Liu Massive activations in large language models. arXiv preprint arXiv:2402.17762. Cited by: §1, §2. Voita et al. (2019) E. Voita, D. Talbot, F. Moiseev, R. Sennrich, and I. Titov Analyzing multi-head self-attention: specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th annual meeting of the association for computational linguistics, p. 5797–5808. Cited by: §1, §2. Xiao et al. (2024) G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis Efficient streaming language models with attention sinks. In International Conference on Learning Representations, Vol. 2024, p. 21875–21895. Cited by: §1, §2. Yamada et al. (2026) D. Yamada, Q. Zhang, T. Pence, B. B. Bendlin, F. Rey, and V. Singh Tree-structured orthonormal decomposition of the aitchison simplex.. Arxiv, p. arXiv–2606. Cited by: §2. Zhai et al. (2023) S. Zhai, T. Likhomanenko, E. Littwin, D. Busbridge, J. Ramapuram, Y. Zhang, J. Gu, and J. M. Susskind Stabilizing transformer training by preventing attention entropy collapse. In International conference on machine learning, p. 40770–40803. Cited by: §1, §2, §5.1. Appendix A Worked example and the geometry of the reversal The witness in full. The triple of Figure 1 and Table 1 is p=(0.90,0.08,0.02)p=(0.90,0.08,0.02), q=(0.70,0.24,0.06)q=(0.70,0.24,0.06), r=(0.90,0.02,0.08)r=(0.90,0.02,0.08) over parts (sink,a,b)(sink,a,b). In the parameterization of Theorem 1: π=(0.8,0.2)π=(0.8,0.2), π~=(0.2,0.8) π=(0.2,0.8) (the transposition of the two content parts), s=0.90s=0.90, s′=0.70s =0.70. All values in Table 1 are exact evaluations (code: experiments/exact_examples.py). The sink balances are b(p)=b(r)=2.5422b(p)=b(r)=2.5422 and b(q)=1.4400b(q)=1.4400, and the Pythagorean decomposition of Lemma 1 verifies to machine precision: dA(p,q)2=1.214868=0+(2.5422−1.4400)2,dA(p,r)2=3.843624=1.96052+0.d_A(p,q)^2=1.214868=0+(2.5422-1.4400)^2, d_A(p,r)^2=3.843624=1.9605^2+0. The (p,q)(p,q) discrepancy is entirely sink balance; the (p,r)(p,r) discrepancy is entirely content. Cosine, JS, and Euclidean distance mix the two components in a renormalization-dependent way, which is precisely what the reversal exploits. Figure 6: The reversal region is not thin. Fixing p and r of the witness and sweeping the third row q over the interior of the simplex (20,00020,000 Dirichlet(1,1,1)(1,1,1) samples with all parts >0.004>0.004), the cosine verdict about which of q,rq,r is closer to p differs between the with-sink and sink-dropped conventions on 77.9%77.9\% of positions (red). How large is the flip region for one anchor pair? Figure 6 fixes (p,r)(p,r) and sweeps q: the cosine verdict is convention-dependent on 77.9%77.9\% of the simplex. This complements the positive-measure statement of Theorem 1(i): near strong sinks, convention-dependence is the typical case, not the exception. Transparency note: total dAd_A versus content dA⟂d_A . In the Monte Carlo of Figure 3, the total Aitchison distance and the content distance dA⟂d_A disagree about the closer head on 0.1%0.1\% of triples, at every sink level. The two are different, explicitly declared estimands; whenever they disagree the discrepancy is exactly the sink-balance term of Lemma 1, in contrast to the undisclosed convention-dependence of cosine/JS/Euclidean. An analyst who wants sink-inclusive similarity reports dAd_A; one who wants content similarity reports dA⟂d_A ; both are stable under the keep-vs-drop choice. Figure 7: A measured Llama-3.2-1B head triple realizing Theorem 1. Found automatically by find_real_triple.py from the saved seed-0 signatures: with the sink, cosine, JS, and Euclidean distance all rank r closer to p; after dropping the sink and re-closing, all three rank q closer; the dA⟂d_A verdict is identical in both pipelines. Appendix B Proofs for Section 4.1 Standing convention. For every integer m≥2m≥ 2, let m:=x∈(0,∞)m:⊤x=1S^m:=\x∈(0,∞)^m:1 x=1\. For x∈mx ^m, let logx x denote the componentwise logarithm and write clrm(x):=logx−1m(⊤logx). _m(x):= x- 1m (1 x )1. Whenever dAd_A is applied to two m-part compositions, it denotes the unrescaled CLR distance dA(x,y):=∥clrm(x)−clrm(y)∥2d_A(x,y):= _m(x)- _m(y) _2; the dimension of the CLR map is determined by the arguments of dAd_A, so full-row and sink-removed distances use clrD _D and clrD−1 _D-1, respectively. This is a metric on mS^m: if clrm(x)=clrm(y) _m(x)= _m(y), then logx−logy=c x- y=c1 for some c∈ℝc , so x=ecyx=e^cy; since both vectors sum to one, c=0c=0 and x=yx=y. All logarithms are natural, and H(x):=−∑jxjlogxjH(x):=- _jx_j x_j with 0log0:=00 0:=0. Proof of Theorem 1. Put t:=1−s,u:=1−s′,a:=∥π∥22,β:=⟨π,π~⟩,Δ:=a−β.t:=1-s, u:=1-s , a:= π _2^2, β:= π, π , :=a-β. Because π~ π is a permutation of π, ∥π~∥22=a π _2^2=a. Cauchy–Schwarz gives β≤aβ≤ a, and equality would force π~ π to be a positive scalar multiple of π. Since both vectors sum to one, equality would imply π~=π π=π, contrary to assumption. Thus 0<a<1,0<β<a,0<Δ<a,∥π−π~∥22=2Δ.0<a<1, 0<β<a, 0< <a, π- π _2^2=2 . Here 1−a=2∑i<jπiπj>0,1-a=2 _i<j _i _j>0, because D−1≥2D-1≥ 2 and every coordinate of π is positive. Proof of (i). The identities for R0(ps),R0(q),R0(rs)R_0(p_s),R_0(q),R_0(r_s) follow directly from the definition. Hence R0(ps)=R0(q)R_0(p_s)=R_0(q) and R0(ps)≠R0(rs)R_0(p_s)≠ R_0(r_s). Metric definiteness proves the Euclidean and arbitrary-metric assertions. Jensen–Shannon divergence is zero if and only if its arguments agree. Finally, c(R0(ps),R0(q))=1,c(R0(ps),R0(rs))=βa<1.c (R_0(p_s),R_0(q) )=1, c (R_0(p_s),R_0(r_s) )= βa<1. Proof of (i). Direct calculation gives ∥ps−q∥22 p_s-q _2^2 =(s−s′)2(1+a)=(u−t)2(1+a), =(s-s )^2(1+a)=(u-t)^2(1+a), (2) ∥ps−rs∥22 p_s-r_s _2^2 =2Δt2, =2 t^2, (3) c(ps,rs) c(p_s,r_s) =(1−t)2+βt2(1−t)2+at2=1−Δt2(1−t)2+at2. = (1-t)^2+β t^2(1-t)^2+at^2=1- t^2(1-t)^2+at^2. (4) Also, c(ps,q)=(1−t)s′+tua(1−t)2+at2(s′)2+au2.c(p_s,q)= (1-t)s +tua (1-t)^2+at^2 (s )^2+au^2. (5) For Jensen–Shannon divergence, mpq:=ps+q2=(s+s′2,t+u2π).m_pq:= p_s+q2= ( s+s 2, t+u2π ). Consequently, KL(ps∥mpq) (p_s\|m_pq) =slog2s+s′+tlog2t+u, =s 2ss+s +t 2tt+u, KL(q∥mpq) (q\|m_pq) =s′log2s′s+s′+ulog2ut+u. =s 2s s+s +u 2ut+u. Therefore JS(ps,q)=JSBer(s,s′),JS(p_s,q)=JS_ Ber(s,s ), (6) where JSBer(v,w):=JS((v,1−v),(w,1−w)).JS_ Ber(v,w):=JS ((v,1-v),(w,1-w) ). Similarly, mpr:=ps+rs2=(s,tπ+π~2),m_pr:= p_s+r_s2= (s,t π+ π2 ), and direct substitution gives JS(ps,rs)=tJS(π,π~).JS(p_s,r_s)=t\,JS(π, π). (7) Set γ:=s′(s′)2+a(1−s′)2∈(0,1),η:=1−γ>0,γ:= s (s )^2+a(1-s )^2∈(0,1), η:=1-γ>0, and define ε∗:=minu2,u28log2,η41+a,η8a,s∗:=1−ε∗. _ := \ u2, u^28 2, η4 1+a, η8a \, s_ :=1- _ . Every entry in the minimum is strictly positive. Moreover, ε∗≤u/2<u _ ≤ u/2<u, so s∗∈(s′,1)s_ ∈(s ,1). This threshold depends on s′s and on π only through a=∥π∥22a= π _2^2; it is therefore uniform over all admissible permutations. Suppose s>s∗s>s_ , so 0<t<ε∗0<t< _ . Use TV(P,Q):=12∥P−Q∥1.TV(P,Q):= 12 P-Q _1. If P,QP,Q are Bernoulli laws with parameters s,s′s,s and M=(P+Q)/2M=(P+Q)/2, then TV(P,M)=TV(Q,M)=|s−s′|2.TV(P,M)=TV(Q,M)= |s-s |2. Pinsker’s inequality applied to both terms gives JSBer(s,s′)≥(s−s′)22=(u−t)22.JS_ Ber(s,s )≥ (s-s )^22= (u-t)^22. Furthermore, JS(π,π~)≤log2JS(π, π)≤ 2. Indeed, if m=(π+π~)/2m=(π+ π)/2, then mi≥πi/2m_i≥ _i/2 and mi≥π~i/2m_i≥ π_i/2, so each KL divergence in the definition of JSJS is at most log2 2. Consequently, JS(ps,q)≥(u−t)22>u28>tlog2≥JS(ps,rs).JS(p_s,q)≥ (u-t)^22> u^28>t 2 (p_s,r_s). The Euclidean identities, together with Δ<a<1 <a<1 and t<u/2t<u/2, give ∥ps−rs∥2 p_s-r_s _2 =t2Δ<t2a<u22a<u21+a =t 2 <t 2a< u2 2a< u2 1+a <(u−t)1+a=∥ps−q∥2. <(u-t) 1+a= p_s-q _2. It remains to compare cosine similarities. Let e0=(1,0,…,0)e_0=(1,0,…,0) and x^=x/∥x∥2 x=x/ x _2 for x≠0x≠ 0. Since t<u/2<1/2t<u/2<1/2, (1−t)2+at2>14,(1-t)^2+at^2> 14, and equation (4) gives c(ps,rs)>1−4at2>1−η2.c(p_s,r_s)>1-4at^2>1- η2. Moreover, γ=⟨e0,q^⟩γ= e_0, q and |c(ps,q)−γ| |c(p_s,q)-γ | =|⟨p^s−e0,q^⟩|≤∥p^s−e0∥2 = | p_s-e_0, q |≤ p_s-e_0 _2 ≤2∥ps−e0∥2=2t1+a<η2. ≤ 2 p_s-e_0 _2=2t 1+a< η2. Here ∥p^s−e0∥2 p_s-e_0 _2 ≤∥p^s−ps∥2+∥ps−e0∥2 ≤ p_s-p_s _2+ p_s-e_0 _2 =|1−∥ps∥2|+∥ps−e0∥2 =|1- p_s _2|+ p_s-e_0 _2 ≤2∥ps−e0∥2, ≤ 2 p_s-e_0 _2, where the last inequality is the reverse triangle inequality. Since 1−η/2=γ+η/21-η/2=γ+η/2, c(ps,rs)>1−η2=γ+η2>c(ps,q).c(p_s,r_s)>1- η2=γ+ η2>c(p_s,q). This proves all three with-sink inequalities. Part (i) then gives the simultaneous reversals. The argument above applies verbatim to every permutation τ with τπ≠πτπ≠π (replace π~ π by τπτπ); since there are finitely many such τ, taking s∗s_ to be the maximum of the per-τ thresholds yields the threshold of (i), uniform over τ. Proof of (i). For z=(x,y,w)∈(D)3z=(x,y,w)∈(S^D)^3, define F1(z) F_1(z) :=c(x,w)−c(x,y), :=c(x,w)-c(x,y), F2(z) F_2(z) :=c(R0(x),R0(y))−c(R0(x),R0(w)), :=c (R_0(x),R_0(y) )-c (R_0(x),R_0(w) ), F3(z) F_3(z) :=JS(x,y)−JS(x,w), :=JS(x,y)-JS(x,w), F4(z) F_4(z) :=JS(R0(x),R0(w))−JS(R0(x),R0(y)), :=JS (R_0(x),R_0(w) )-JS (R_0(x),R_0(y) ), F5(z) F_5(z) :=∥x−y∥22−∥x−w∥22, := x-y _2^2- x-w _2^2, F6(z) F_6(z) :=∥R0(x)−R0(w)∥22−∥R0(x)−R0(y)∥22. := R_0(x)-R_0(w) _2^2- R_0(x)-R_0(y) _2^2. Then ℛD=⋂j=16Fj−1((0,∞)).R_D= _j=1^6F_j^-1 ((0,∞) ). Squaring a nonnegative distance does not change a strict comparison. Moreover, 1−x0=∑i=1D−1xi>01-x_0= _i=1^D-1x_i>0 on DS^D, so R0R_0 is smooth there. Cosine similarity, Jensen–Shannon divergence, and squared Euclidean distance are continuous on the relevant positive simplices. Thus every FjF_j is continuous. Choosing any s∈(s∗,1)s∈(s_ ,1), parts (i)–(i) give (ps,q,rs)∈ℛD,(p_s,q,r_s) _D, so ℛDR_D is nonempty. Let D:=x∈ℝD:∑i=0D−1xi=1.A_D:= \x ^D: _i=0^D-1x_i=1 \. Because D=D∩(0,∞)D,S^D=A_D∩(0,∞)^D, the set (D)3(S^D)^3 is open in the 3(D−1)3(D-1)-dimensional affine space D3A_D^3. Hence ℛDR_D is open in D3A_D^3. Every nonempty open subset of that affine space contains a nondegenerate relative Euclidean ball and therefore has positive intrinsic 3(D−1)3(D-1)-dimensional Lebesgue measure. This proves part (i). Proof of (iv). Because dAd_A is a metric and R0R_0 is a map, its pullback dA⟂d_A is nonnegative and symmetric and satisfies the triangle inequality. Moreover, dA⟂(x,y)=0⟺R0(x)=R0(y)⟺x∼0y.d_A (x,y)=0 R_0(x)=R_0(y) x _0y. Therefore dA⟂d_A is a pseudometric on DS^D, and dA,0([x],[y]):=dA⟂(x,y)d_A,0([x],[y]):=d_A (x,y) defines a genuine metric on D/∼0S^D/ _0. It is well defined because x∼0x′x _0x and y∼0y′y _0y imply R0(x)=R0(x′)R_0(x)=R_0(x ) and R0(y)=R0(y′)R_0(y)=R_0(y ). The identities for dA⟂d_A follow from its definition and part (i). Moreover, B>0B>0 because dAd_A is a metric and π≠π~π≠ π. Put Ls:=log(s(1−s′)s′(1−s))>0.L_s:= \! ( s(1-s )s (1-s) )>0. Direct centering of the coordinatewise log-ratio gives clrD(ps)−clrD(q)=LsD(D−1,−1,…,−1). _D(p_s)- _D(q)= L_sD(D-1,-1,…,-1). Therefore dA(ps,q)=‖clrD(ps)−clrD(q)‖2=αDLs=αD(logit(s)−logit(s′)).d_A(p_s,q)= _D(p_s)- _D(q) _2= _DL_s= _D (logit(s)-logit(s ) ). Next set hi:=logπiπ~i.h_i:= _i π_i. Because π~ π is a permutation of π, ∑i=1D−1hi=log(∏i=1D−1πi∏i=1D−1π~i)=0. _i=1^D-1h_i= \! ( _i=1^D-1 _i _i=1^D-1 π_i )=0. Consequently, clrD(ps)−clrD(rs)=(0,h1,…,hD−1), _D(p_s)- _D(r_s)=(0,h_1,…,h_D-1), while clrD−1(π)−clrD−1(π~)=(h1,…,hD−1). _D-1(π)- _D-1( π)=(h_1,…,h_D-1). Thus dA(ps,rs)=dA(π,π~)=B.d_A(p_s,r_s)=d_A(π, π)=B. Let E:=exp(B/αD)>1.E:= (B/ _D)>1. Then sA=s′E1−s′+s′E,s_A= s E1-s +s E, and sA−s′=s′(1−s′)(E−1)1−s′+s′E>0,1−sA=1−s′1−s′+s′E>0.s_A-s = s (1-s )(E-1)1-s +s E>0, 1-s_A= 1-s 1-s +s E>0. Hence sA∈(s′,1)s_A∈(s ,1). Moreover, LsA=logE=BαD,dsLs=1s(1−s)>0.L_s_A= E= B _D, ddsL_s= 1s(1-s)>0. Therefore dA(ps,q)<Bd_A(p_s,q)<B exactly when s<sAs<s_A, with equality at s=sAs=s_A and the reverse inequality when s>sAs>s_A. In the special case D=3D=3, for the Table 1 values, (h1,h2)=(log4,−log4),Ls=log(9/10)(3/10)(7/10)(1/10)=log277.(h_1,h_2)=( 4,- 4), L_s= (9/10)(3/10)(7/10)(1/10)= 277. Hence B=2log4,dA(ps,q)=23log277.B= 2 4, d_A(p_s,q)= 23 277. Finally, 23log277<23log4<2log4, 23 277< 23 4< 2 4, which completes the proof. ∎ Remark B.1 (Scope of the Aitchison conclusion). On the full simplex, dA⟂(x,y)=0⟺R0(x)=R0(y).d_A (x,y)=0 R_0(x)=R_0(y). The map R0R_0 is surjective because R0((1/2,π/2))=πR_0 ((1/2,π/2) )=π for every π∈D−1π ^D-1. Therefore Φ:D/∼0⟶D−1,Φ([x]):=R0(x), :S^D/ _0 ^D-1, ([x]):=R_0(x), is a well-defined bijection. By the definition of dA⟂d_A , it preserves distances, so the metric quotient is isometric to (D−1,dA)(S^D-1,d_A). Full dAd_A is a genuine metric, but its ranking is not generally invariant under sink deletion. Indeed, dA(ps,q)⟶∞as s↑1,dA(ps,rs)=B.d_A(p_s,q) ∞ s 1, d_A(p_s,r_s)=B. Thus the unqualified claim that full dAd_A always ranks q closer in both views is false. It holds for the Table 1 example because s<sAs<s_A there. Proof of Proposition 1. We first establish the entropy identity for the sink–content factorization. Let r=(s,(1−s)π),s∈(0,1),π∈D−1,r= (s,(1-s)π ), s∈(0,1), π ^D-1, and define the binary entropy Hb(s):=−slogs−(1−s)log(1−s).H_b(s):=-s s-(1-s) (1-s). Since ∑j=1D−1πj=1 _j=1^D-1 _j=1, direct expansion gives H(r) H(r) =−slogs−∑j=1D−1(1−s)πjlog((1−s)πj) =-s s- _j=1^D-1(1-s) _j \! ((1-s) _j ) =−slogs−(1−s)log(1−s)−(1−s)∑j=1D−1πjlogπj =-s s-(1-s) (1-s)-(1-s) _j=1^D-1 _j _j =Hb(s)+(1−s)H(π). =H_b(s)+(1-s)H(π). (8) Fix D≥3D≥ 3, and put m:=D−1≥2m:=D-1≥ 2. Define π(p):=(1m,…,1m)∈m,H(π(p))=logm.π^(p):= ( 1m,…, 1m ) ^m, H (π^(p) )= m. For 0<ε<1/m0< <1/m, define πε(q):=(1−(m−1)ε,ε,…,ε)∈mπ^(q)_ := (1-(m-1) , ,…, ) ^m and set ηε:=H(πε(q)) _ :=H (π^(q)_ ). The vector πε(q)π^(q)_ has m strictly positive coordinates that sum to one, and ηε=−(1−(m−1)ε)log(1−(m−1)ε)−(m−1)εlogε⟶0 _ =- (1-(m-1) ) (1-(m-1) )-(m-1) 0 as ε↓0 0. Indeed, the first term tends to zero by continuity of −xlogx-x x at x=1x=1, while the second tends to zero by xlogx→0x x→ 0 as x↓0x 0. Because m≥2m≥ 2 and every coordinate of πε(q)π^(q)_ lies in (0,1)(0,1), we also have ηε>0 _ >0. We may therefore fix ε∈(0,1/m) ∈(0,1/m) such that 0<ηε<log20< _ < 2. For s∈(0,1)s∈(0,1), set ps:=(s,(1−s)π(p)),q:=(12,12πε(q)).p_s:= (s,(1-s)π^(p) ), q:= ( 12, 12π^(q)_ ). Both vectors belong to DS^D. By (8), H(ps)=Hb(s)+(1−s)logm⟶0(s↑1),H(p_s)=H_b(s)+(1-s) m 0 (s 1), whereas H(q)=Hb(12)+12ηε=log2+12ηε>0.H(q)=H_b\! ( 12 )+ 12 _ = 2+ 12 _ >0. Since H(q)>0H(q)>0 is fixed, there exists s0∈(0,1)s_0∈(0,1), sufficiently close to one, such that H(ps0)<H(q)H(p_s_0)<H(q). Set p:=ps0p:=p_s_0. Removing the sink and re-closing gives p′=π(p)p =π^(p) and q′=πε(q)q =π^(q)_ , and hence H(p′)=logm≥log2>ηε=H(q′).H(p )= m≥ 2> _ =H(q ). This proves the asserted inversion for every D≥3D≥ 3. We now verify the displayed D=3D=3 witness by exact inequalities. Direct re-closure gives P′=(12,12),Q′=(9991000,11000).P = ( 12, 12 ), Q = ( 9991000, 11000 ). For P, H(P)=1920log2019+120log40,H(P)= 1920 2019+ 120 40, so 20H(P)=log[40(2019)19].20H(P)= \! [40 ( 2019 )^19 ]. Since (1+119)19<e<3, (1+ 119 )^19<e<3, we obtain 40(2019)19<120<220.40 ( 2019 )^19<120<2^20. Taking logarithms and dividing by 2020 yields H(P)<log2H(P)< 2. Applying (8) to Q gives H(Q)=log2+12Hb(11000)>log2.H(Q)= 2+ 12H_b\! ( 11000 )> 2. Thus H(P)<H(Q)H(P)<H(Q) exactly. Next, Hb′(t)=log1−t,Hb′(t)=−1t(1−t)<0(0<t<1).H_b (t)= 1-tt, H_b (t)=- 1t(1-t)<0 (0<t<1). Hence HbH_b is strictly concave and has its unique maximum log2 2 at t=1/2t=1/2. Since 1/1000≠1/21/1000≠ 1/2, H(Q′)=Hb(11000)<log2=H(P′).H(Q )=H_b\! ( 11000 )< 2=H(P ). Finally, for a binary composition u=(a,b)u=(a,b), clr2(u)=(12logab,−12logab). _2(u)= ( 12 ab,- 12 ab ). It follows that clr2(P′)=(0,0),clr2(Q′)=(12log999,−12log999), _2(P )=(0,0), _2(Q )= ( 12 999,- 12 999 ), and therefore CA⟂(P)=0,CA⟂(Q)=log9992.C_A (P)=0, C_A (Q)= 999 2. This completes the verification of the witness and the concentration values. ∎ Remark B.2 (Sharpness and scope of the entropy inversion). The restriction D≥3D≥ 3 is necessary. If D=2D=2, every sink-dropped composition equals (1)(1), and therefore H(p′)=H(q′)=0H(p )=H(q )=0 for all p,q∈2p,q ^2. Moreover, an entropy inversion necessarily uses unequal sink masses. Indeed, if p=(s,(1−s)πp)p= (s,(1-s) _p ) and q=(s,(1−s)πq)q= (s,(1-s) _q ), then (8) implies H(p)−H(q)=(1−s)(H(p′)−H(q′)).H(p)-H(q)=(1-s) (H(p )-H(q ) ). Because 1−s>01-s>0, the entropy differences have the same sign. Finally, for a fixed normalized content composition π, CA⟂(s,(1−s)π)=∥clrD−1(π)∥2,0<s<1.C_A (s,(1-s)π )= _D-1(π) _2, 0<s<1. Thus CA⟂C_A is independent of the mass assigned to the fixed designated sink when π is held fixed, and it equals the CLR norm computed directly from the re-closed content composition. This does not assert invariance under changing which coordinate is designated as the sink, nor does it identify CA⟂C_A with the CLR norm of the full composition. Appendix C Proofs for Section 4.2 Table 4: Invariance matrix. Perm. == permutation invariance; Perturb. == invariance to a shared logit bias (Prop. 2(i)); Temp. == global-temperature ranking preservation; Sink == coherent under keep-vs-drop of the sink column; Dom. == subcompositional dominance. † : dAd_A is 1/τ1/τ-equivariant, so rankings are exact; ρA _A is invariant even to per-head temperatures. ‡ : dAd_A decomposes exactly (Lemma 1); dA⟂,CA⟂d_A ,C_A are exactly sink-invariant. “n/a”: not applicable. Perm. Perturb. (⊕ ) Temp. Sink Dom. Euclidean ✓ × × × × cosine ✓ × × × × JS ✓ × × × × Shannon H ✓ × × × n/a dAd_A ✓ ✓ ✓† ✓‡ ✓ ρA _A (ILR-cosine) ✓ × ✓† n/a n/a dA⟂d_A , CA⟂C_A ✓ ✓ ✓† ✓‡ ✓ Standing notation. All logarithms are natural. For every positive vector x=(x0,…,xm−1)x=(x_0,…,x_m-1), write (x):=x/∑ixiC(x):=x/ _ix_i. For every integer m≥1m≥ 1, let mS^m be the open m-part simplex, em:=1mme_m:= 1m1_m, g(x1,…,xm):=(∏i=1mxi)1/m,clrm(x):=(logxi−m−1∑jlogxj)i,g(x_1,…,x_m):= ( _i=1^mx_i )^1/m, _m(x):= ( x_i-m^-1 _j x_j )_i, and throughout dA(x,y):=∥clrm(x)−clrm(y)∥2d_A(x,y):= _m(x)- _m(y) _2 for x,y∈mx,y ^m, with no dimension-dependent rescaling. For D≥2D≥ 2 and p,q∈Dp,q ^D with designated sink coordinate 00, p′:=(p1,…,pD−1),b(p):=D−1Dlogp0g(p1,…,pD−1),p :=C(p_1,…,p_D-1), b(p):= D-1D p_0g(p_1,…,p_D-1), dA⟂(p,q):=dA(p′,q′),CA⟂(p):=∥clrD−1(p′)∥2.d_A (p,q):=d_A(p ,q ), C_A (p):= _D-1(p ) _2. Perturbation, powering, and the Aitchison center are x⊕y:=((xiyi)i)x y:=C((x_iy_i)_i), α⊙x:=((xiα)i)α x:=C((x_i^α)_i), and cen(x1,…,xn):=(exp1n∑tlogxt) (x_1,…,x_n):=C( \ 1n _t x_t\), all componentwise. When x≠emx≠ e_m and y≠emy≠ e_m, ρA(x,y):=⟨clrm(x),clrm(y)⟩/(∥clrm(x)∥2∥clrm(y)∥2) _A(x,y):= _m(x), _m(y) /( _m(x) _2 _m(y) _2), and for nonzero Euclidean vectors cos(x,y):=x⊤y/(∥x∥2∥y∥2) (x,y):=x y/( x _2 y _2). For m=1m=1, set 1=(1)S^1=\(1)\ and regard its CLR space and ILR coordinate vector as zero-dimensional, so its CLR vector, norm, and all distances are zero. Proof of Lemma 1. For m≥1m≥ 1, let ℋm:=x∈ℝm:m⊤x=0.H_m:= \x ^m:1_m x=0 \. The map clrm _m takes values in ℋmH_m. Define ub:=D−1D(1,−1D−1,…,−1D−1)∈ℋDu_b:= D-1D (1,- 1D-1,…,- 1D-1 ) _D and W:=(0,z):z∈ℋD−1⊂ℋD,J:ℋD−1⟶W,Jz=(0,z).W:=\(0,z):z _D-1\ _D, J:H_D-1 W, Jz=(0,z). The map J is a linear isometry. Moreover, ∥ub∥22=D−1D(1+D−1(D−1)2)=1, u_b _2^2= D-1D (1+ D-1(D-1)^2 )=1, and for every z∈ℋD−1z _D-1, ⟨ub,Jz⟩=−1D(D−1)∑i=1D−1zi=0. u_b,Jz =- 1 D(D-1) _i=1^D-1z_i=0. Since dimW=D−2 W=D-2 and dimℋD=D−1 _D=D-1, it follows that ℋD=spanub⊕W,spanub⟂W.H_D=span\u_b\ W, \u_b\ W. (9) Write uniquely p=(s,(1−s)π),s=p0∈(0,1),π=p′∈D−1.p=(s,(1-s)π), s=p_0∈(0,1), π=p ^D-1. Because the entries of ubu_b sum to zero, the common centering term in clrD(p) _D(p) vanishes in its inner product with ubu_b. Hence ⟨clrD(p),ub⟩ _D(p),u_b =D−1D(logp0−1D−1∑i=1D−1logpi) = D-1D ( p_0- 1D-1 _i=1^D-1 p_i ) =D−1Dlogp0g(p1,…,pD−1)=b(p). = D-1D p_0g(p_1,…,p_D-1)=b(p). (10) Let PWP_W denote Euclidean orthogonal projection onto W. For x∈ℋDx _D, (PWx)0=0,(PWx)i=xi−1D−1∑j=1D−1xj,1≤i≤D−1.(P_Wx)_0=0, (P_Wx)_i=x_i- 1D-1 _j=1^D-1x_j, 1≤ i≤ D-1. Applying this to x=clrD(p)x= _D(p) gives, for i≥1i≥ 1, (PWclrD(p))i (P_W _D(p))_i =logpi−1D−1∑j=1D−1logpj = p_i- 1D-1 _j=1^D-1 p_j =logπi−1D−1∑j=1D−1logπj=(clrD−1(π))i. = _i- 1D-1 _j=1^D-1 _j=( _D-1(π))_i. Therefore PWclrD(p)=JclrD−1(p′).P_W _D(p)=J _D-1(p ). (11) Equations (9)–(11) yield the exact orthogonal identity clrD(p)=b(p)ub+JclrD−1(p′). \; _D(p)=b(p)u_b+J _D-1(p ).\; (12) Apply (12) to p and q, subtract, and use the orthogonality in (9) and the isometry of J. Then dA(p,q)2 d_A(p,q)^2 =∥clrD(p)−clrD(q)∥22 = _D(p)- _D(q) _2^2 =(b(p)−b(q))2+∥clrD−1(p′)−clrD−1(q′)∥22 = (b(p)-b(q) )^2+ _D-1(p )- _D-1(q ) _2^2 =(b(p)−b(q))2+dA⟂(p,q)2, = (b(p)-b(q) )^2+d_A (p,q)^2, which proves the Pythagorean identity. To verify the ILR assertion, let w2,…,wD−1w_2,…,w_D-1 be any orthonormal basis of W; the list is empty when D=2D=2. Then (ub,w2,…,wD−1)(u_b,w_2,…,w_D-1) is an orthonormal contrast basis of ℋDH_D and hence defines an ILR map. Its first coordinate is b(p)b(p) by (10). The vectors J−1w2,…,J−1wD−1J^-1w_2,…,J^-1w_D-1 form an orthonormal basis of ℋD−1H_D-1, and (11) gives ⟨clrD(p),wj⟩=⟨clrD−1(p′),J−1wj⟩,2≤j≤D−1. _D(p),w_j = _D-1(p ),J^-1w_j , 2≤ j≤ D-1. Thus the remaining coordinates are precisely an ILR coordinate system for p′p . In particular, dA⟂(p,q)=∥PW(clrD(p)−clrD(q))∥2,CA⟂(p)=∥PWclrD(p)∥2.d_A (p,q)= P_W( _D(p)- _D(q)) _2, C_A (p)= P_W _D(p) _2. The first expression depends only on p′p and q′q , and the second only on p′p ; neither depends on a sink mass. Equation (11) also makes the projection claim precise: it is a statement in CLR space under J, not a Euclidean projection of probability vectors. ∎ Exact evaluation of the Table 1 witness. For p=(910,225,150),q=(710,625,350),r=(910,150,225),p= ( 910, 225, 150 ), q= ( 710, 625, 350 ), r= ( 910, 150, 225 ), interpreting the decimal entries in Table 1 as these exact rational numbers, one has p′=q′=(45,15),r′=(15,45),p =q = ( 45, 15 ), r = ( 15, 45 ), and b(p)=b(r) b(p)=b(r) =23log452≈2.542174604636, = 23 452≈ 2.542174604636, b(q) b(q) =23log356≈1.439964055745. = 23 356≈ 1.439964055745. Consequently, the pure-sink and pure-content identities are dA⟂(p,q) d_A (p,q) =0, =0, dA(p,q)2 d_A(p,q)^2 =23log2277≈1.214868094088, = 23 ^2 277≈ 1.214868094088, b(p)−b(r) b(p)-b(r) =0, =0, dA(p,r)2 d_A(p,r)^2 =dA(p′,r′)2=2log24=8log22≈3.843624111346, =d_A(p ,r )^2=2 ^24=8 ^22≈ 3.843624111346, with dA(p,r)=2log4≈1.960516286937.d_A(p,r)= 2 4≈ 1.960516286937. Thus rounded decimal displays must use ≈; the exact equalities are the logarithmic identities above. Full statement of Proposition 2 Let D≥2D≥ 2, let ℓ,ℓ′∈ℝD , ^D be finite logit rows, and put p=(eℓ)p=C(e ), q=(eℓ′)q=C(e ), where exponentials, products, and powers are componentwise. (i) Shared finite logit bias equals perturbation. For every finite u∈ℝDu ^D, with cu=(eu)c_u=C(e^u), (eℓ+u)=p⊕cu,(eℓ′+u)=q⊕cu,C(e +u)=p c_u, (e +u)=q c_u, and dA(p⊕cu,q⊕cu)=dA(p,q)d_A(p c_u,q c_u)=d_A(p,q). For a fixed designated sink, dA⟂d_A has the same shared-perturbation invariance. If D≥3D≥ 3, CA⟂C_A is not perturbation-invariant in general; when D=2D=2, it is identically zero. Euclidean distance, Jensen–Shannon divergence, and cosine similarity likewise fail to be invariant under every shared perturbation. The directional similarity ρA _A is also not perturbation-invariant in general, even when it is defined both before and after perturbation. (i) Temperature equals powering. For every finite temperature τ>0τ>0, (eℓ/τ)=1τ⊙pC(e /τ)= 1τ p and (eℓ′/τ)=1τ⊙qC(e /τ)= 1τ q, and hence dA(1τ⊙p,1τ⊙q)=1τdA(p,q),dA⟂(1τ⊙p,1τ⊙q)=1τdA⟂(p,q),CA⟂(1τ⊙p)=1τCA⟂(p).d_A ( 1τ p, 1τ q )= 1τd_A(p,q), d_A ( 1τ p, 1τ q )= 1τd_A (p,q), C_A ( 1τ p )= 1τC_A (p). Thus a single common temperature applied to every member of a family preserves all weak and strict rankings among the resulting distances or concentration values; these are equivariances, not value invariances. For a nonempty finite collection P=(p1,…,pn)P=(p_1,…,p_n), define VarA(P):=1n∑tdA(pt,cen(P))2Var_A(P):= 1n _td_A(p_t, (P))^2. Then VarA(1τ⊙p1,…,1τ⊙pn)=1τ2VarA(P),Var_A ( 1τ p_1,…, 1τ p_n )= 1τ^2Var_A(P), where the center on the left is recomputed after powering. Whenever p≠eDp≠ e_D and q≠eDq≠ e_D, so that ρA(p,q) _A(p,q) is defined, separate positive temperatures cancel: for every τh,τh′>0 _h, _h >0, ρA(1τh⊙p,1τh′⊙q)=ρA(p,q). _A ( 1 _h p, 1 _h q )= _A(p,q). More formally, for h∈1,2h∈\1,2\ let nh≥1n_h≥ 1 and pht∈Dp_ht ^D, put Ph:=(ph1,…,phnh)P_h:=(p_h1,…,p_hn_h) and sh:=cen(Ph)≠eDs_h:= (P_h)≠ e_D, and give every row in head h the same exponent αh>0 _h>0. If sh(αh):=cen(αh⊙ph1,…,αh⊙phnh)s_h^( _h):= ( _h p_h1,…, _h p_hn_h), then sh(αh)=αh⊙sh,ρA(s1(α1),s2(α2))=ρA(s1,s2).s_h^( _h)= _h s_h, _A (s_1^( _1),s_2^( _2) )= _A(s_1,s_2). Thus one positive temperature per head cancels in ρA _A when the rows have a common support and the two resulting signatures are nonuniform. This conclusion can fail for row-dependent temperatures. Under unequal head temperatures or row-dependent temperatures, the separate-scaling cancellation asserted here is specific to ρA _A. No general invariance or ranking-preservation claim is made for dAd_A, dA⟂d_A , CA⟂C_A , or Aitchison variance. (i) Sink coherence. For a fixed designated sink, write p=(s,(1−s)π)p=(s,(1-s)π) and q=(t,(1−t)η)q=(t,(1-t)η) with π,η∈D−1π,η ^D-1. Then dA⟂(p,q)=dA(π,η)d_A (p,q)=d_A(π,η) and CA⟂(p)=∥clrD−1(π)∥2C_A (p)= _D-1(π) _2. These values are independent of s and t and are identical whether computed in the balance-orthogonal coordinates of the full compositions or after dropping the sink and re-closing. Full dAd_A is not itself sink-invariant; Lemma 1 gives its exact sink–content decomposition. (iv) Subcomposition and masking. Let S⊆0,…,D−1S \0,…,D-1\ with |S|≥2|S|≥ 2, and define p(S)=((pi)i∈S)p^(S)=C((p_i)_i∈ S) and q(S)=((qi)i∈S)q^(S)=C((q_i)_i∈ S). After canonically embedding the |S||S|-part CLR space into ℋDH_D, subcomposition is an orthogonal projection and dA(p(S),q(S))≤dA(p,q).d_A (p^(S),q^(S) )≤ d_A(p,q). Coordinate deletion is literal in an S-adapted ILR basis; in an arbitrary fixed ILR basis the same map is generally a non-diagonal orthogonal projection. Outside the standing finite-logit hypothesis, let p~ p and q~ q be hard-masked probability rows in the closed simplex with positive supports Tp~T_ p and Tq~T_ q. If S=Tp~∩Tq~S=T_ p∩ T_ q has at least two parts, dA(p~(S),q~(S))d_A( p^(S), q^(S)) (of the re-closed restrictions) is a well-defined within-common-support comparison. If the two supports differ, this comparison discards the support mismatch. Since structural-zero rows do not lie in the open simplex, their full Aitchison distance is undefined, and no contraction inequality relative to that nonexistent full distance is asserted. If positive pre-mask rows in DS^D are available and the same mask retains S, the preceding contraction inequality does apply to those pre-mask rows and their S-subcompositions. (v) Dominance. Part (iv) shows that dAd_A is subcompositionally dominant. For every D≥3D≥ 3, Euclidean distance, Jensen–Shannon divergence JSJS (not its square root), and cosine dissimilarity 1−cos1- are not subcompositionally dominant. For D=2D=2, there is no nontrivial proper subcomposition containing at least two parts. Proof of Proposition 2. For (i), closure is unchanged by multiplication by a common positive scalar. Therefore (eℓ+u) (e +u) =((eℓieui)i=0D−1) =C ((e _ie^u_i)_i=0^D-1 ) =((pi(cu)i)i=0D−1)=p⊕cu, =C ((p_i(c_u)_i)_i=0^D-1 )=p c_u, and similarly for q. Since clr(a⊕b)=clr(a)+clr(b) (a b)= (a)+ (b), we obtain dA(p⊕cu,q⊕cu) d_A(p c_u,q c_u) =∥clr(p)+clr(cu)−clr(q)−clr(cu)∥2 = (p)+ (c_u)- (q)- (c_u) _2 =dA(p,q). =d_A(p,q). For the content distance, let cu′c_u be the sink-dropped and re-closed composition. Directly from the definition of perturbation, (p⊕cu)′=p′⊕cu′,(q⊕cu)′=q′⊕cu′.(p c_u) =p c_u , (q c_u) =q c_u . Applying the same CLR perturbation identity in dimension D−1D-1 (including the stated zero-dimensional convention when D=2D=2) gives dA⟂(p⊕cu,q⊕cu)=dA⟂(p,q)d_A (p c_u,q c_u)=d_A (p,q). In contrast, CA⟂(p⊕cu)=∥clrD−1(p′)+clrD−1(cu′)∥2.C_A (p c_u)= _D-1(p )+ _D-1(c_u ) _2. For D≥3D≥ 3, this equals CA⟂(p)C_A (p) for every p if and only if clrD−1(cu′)=0 _D-1(c_u )=0, equivalently, the content coordinates of u are all equal. Thus the analogous invariance under arbitrary shared perturbations is false for CA⟂C_A . Indeed, let p=eDp=e_D and c=(1,2,1,…,1)c=C(1,2,1,…,1), where coordinate 00 is the sink and the factor 22 is on a content coordinate. Then CA⟂(p)=0,(p⊕c)′=(2,1,…,1),CA⟂(p⊕c)>0.C_A (p)=0, (p c) =C(2,1,…,1), C_A (p c)>0. At D=3D=3, the last value is exactly (log2)/2( 2)/ 2. When D=2D=2, the content simplex has one part and CA⟂≡0C_A ≡ 0. To prove failure of the corresponding classical invariances, fix distinct p,q∈Dp,q ^D, choose a coordinate j, let ut=tju_t=te_j, and put ct=(eut)c_t=C(e^u_t). Since all entries of p and q are positive, ct⊕p⟶j,ct⊕q⟶j(t→∞).c_t p _j, c_t q _j (t→∞). Using the standard continuous extension of JSJS to the closed simplex (with 0log0=00 0=0), and continuity of Euclidean distance and cosine similarity at probability vectors, gives ∥ct⊕p−ct⊕q∥2 c_t p-c_t q _2 ⟶0, 0, JS(ct⊕p,ct⊕q) (c_t p,c_t q) ⟶0, 0, cos(ct⊕p,ct⊕q) (c_t p,c_t q) ⟶1. 1. For p≠qp≠ q, the original Euclidean distance and JSJS divergence are strictly positive, whereas the original cosine similarity is strictly less than one. Consequently none is invariant under every shared perturbation. The same conclusion holds for ρA _A. Fix 0≠v∈ℋD0≠ v _D and set p=(ev),q=(e−v),c=(e2v).p=C(e^v), q=C(e^-v), c=C(e^2v). Then clr(p)=v (p)=v, clr(q)=−v (q)=-v, and clr(c)=2v (c)=2v, so all relevant directional similarities are defined and ρA(p,q)=−1,ρA(p⊕c,q⊕c)=ρA((e3v),(ev))=1. _A(p,q)=-1, _A(p c,q c)= _A (C(e^3v),C(e^v) )=1. For (i), set α=1/τ>0α=1/τ>0. Then (eℓ/τ)=((eℓi)α)=α⊙pC(e /τ)=C((e _i)^α)=α p. The elementary identity clr(α⊙p)=αclr(p) (α p)=α (p) (13) implies dA(α⊙p,α⊙q)=αdA(p,q)d_A(α p,α q)=α d_A(p,q). Moreover, dropping and re-closing commutes with powering: (α⊙p)′=α⊙p′(α p) =α p . Applying (13) in the (D−1)(D-1)-part content simplex proves the displayed scaling identities for dA⟂d_A and CA⟂C_A . For the variance assertion, put xt=clr(pt)x_t= (p_t) and x¯=n−1∑t=1nxt x=n^-1 _t=1^nx_t. From the definition of the Aitchison center, clr(cen(P))=x¯ ( (P))= x. After common powering by α, the CLR vectors and their recomputed center are αxtα x_t and αx¯α x. Hence 1n∑t=1n∥αxt−αx¯∥22=α21n∑t=1n∥xt−x¯∥22, 1n _t=1^n α x_t-α x _2^2=α^2 1n _t=1^n x_t- x _2^2, which proves the factor 1/τ21/τ^2. If p,q≠eDp,q≠ e_D and α,β>0α,β>0, then ⟨αclr(p),βclr(q)⟩∥αclr(p)∥2∥βclr(q)∥2=ρA(p,q), α (p),β (q) α (p) _2 β (q) _2= _A(p,q), which proves invariance of ρA _A to separate positive temperatures. For the aggregation qualification, let sh=cen(ph1,…,phnh)s_h= (p_h1,…,p_hn_h) and let αh=1/τh>0 _h=1/ _h>0 be common to all rows in head h. Applying (13) and the center identity above gives cen(αh⊙ph1,…,αh⊙phnh)=αh⊙sh. ( _h p_h1,…, _h p_hn_h)= _h s_h. Thus separate positive head temperatures cancel in ρA _A whenever the resulting signatures are nonuniform. With row-dependent coefficients αht _ht, the new center has CLR vector nh−1∑tαhtclr(pht)n_h^-1 _t _ht (p_ht), which need not be a scalar multiple of clr(sh) (s_h). An explicit failure is available in every D≥2D≥ 2. Fix 0≠v∈ℋD0≠ v _D. Let the first head contain p11=(e2v),p12=(e−v),p_11=C(e^2v), p_12=C(e^-v), and let the second head consist of p21=(ev)p_21=C(e^v). Before row-dependent powering, their signatures satisfy clr(s1)=12v,clr(s2)=v,ρA(s1,s2)=1. (s_1)= 12v, (s_2)=v, _A(s_1,s_2)=1. Now use exponents α11=1/4 _11=1/4 and α12=1 _12=1 in the first head, leaving the second head unchanged. The new first-head signature s~1 s_1 has clr(s~1)=12(14(2v)−v)=−14v,ρA(s~1,s2)=−1. ( s_1)= 12 ( 14(2v)-v )=- 14v, _A( s_1,s_2)=-1. Hence row-dependent positive temperatures can change ρA _A. For (i), closure of the content block gives p′=πp =π and q′=ηq =η. The two displayed identities follow immediately from the definitions dA⟂(p,q):=dA(p′,q′)d_A (p,q):=d_A(p ,q ) and CA⟂(p):=∥clrD−1(p′)∥2C_A (p):= _D-1(p ) _2. Their equality with the full balance-orthogonal calculation is (11). The final qualification follows from the Pythagorean identity: the balance term generally changes when sink masses change. For (iv), define WS:=w∈ℝD:wi=0 for i∉S,∑i∈Swi=0⊂ℋD,W_S:= \w ^D:w_i=0 for i∉ S,\ _i∈ Sw_i=0 \ _D, and let IS:ℋ|S|→WSI_S:H_|S|→ W_S be zero-extension, using the inherited coordinate order on S. This is a linear isometry. If PWSP_W_S is orthogonal projection onto WSW_S, then for every x∈ℋDx _D, (PWSx)i=xi−|S|−1∑j∈Sxj,i∈S,0,i∉S.(P_W_Sx)_i= casesx_i-|S|^-1 _j∈ Sx_j,&i∈ S,\\ 0,&i∉ S. cases Applying this to x=clrD(p)x= _D(p) shows that ISclr|S|(p(S))=PWSclrD(p),ISclr|S|(q(S))=PWSclrD(q).I_S _|S|(p^(S))=P_W_S _D(p), I_S _|S|(q^(S))=P_W_S _D(q). (14) Subtracting the identities in (14), using the isometry of ISI_S, and using contractivity of an orthogonal projection gives dA(p(S),q(S)) d_A(p^(S),q^(S)) =∥PWS(clrD(p)−clrD(q))∥2 = P_W_S( _D(p)- _D(q)) _2 ≤∥clrD(p)−clrD(q)∥2=dA(p,q). ≤ _D(p)- _D(q) _2=d_A(p,q). This proves both the projection identity and dominance. The masking qualifications in the statement follow because CLR coordinates exist only for strictly positive parts. For (v), first consider D=3D=3 and use the Table 1 compositions p=(910,225,150),r=(910,150,225),S=1,2.p= ( 910, 225, 150 ), r= ( 910, 150, 225 ), S=\1,2\. Then p(S)=(45,15),r(S)=(15,45).p^(S)= ( 45, 15 ), r^(S)= ( 15, 45 ). For Euclidean distance, ∥p−r∥2=3250<325=∥p(S)−r(S)∥2. p-r _2= 3 250< 3 25= p^(S)-r^(S) _2. For cosine dissimilarity, direct calculation gives ∥p∥22=∥r∥22=20422500,p⊤r=20332500, p _2^2= r _2^2= 20422500, p r= 20332500, and ∥p(S)∥22=∥r(S)∥22=1725,(p(S))⊤r(S)=825. p^(S) _2^2= r^(S) _2^2= 1725, (p^(S)) r^(S)= 825. Consequently, 1−cos(p,r)=92042<917=1−cos(p(S),r(S)).1- (p,r)= 92042< 917=1- (p^(S),r^(S)). For a,ba,b in a common simplex, let JS(a,b):=12KL(a∥a+b2)+12KL(b∥a+b2).JS(a,b):= 12KL (a \| a+b2 )+ 12KL (b \| a+b2 ). Put π=(45,15),π~=(15,45),mc=(12,12),π= ( 45, 15 ), π= ( 15, 45 ), m_c= ( 12, 12 ), and let Jπ:=45log85+15log25>0.J_π:= 45 85+ 15 25>0. By symmetry, KL(π∥mc)=KL(π~∥mc)=Jπ,JS(π,π~)=Jπ.KL(π\|m_c)=KL( π\|m_c)=J_π, (π, π)=J_π. Here p=(910,110π),r=(910,110π~),p= ( 910, 110π ), r= ( 910, 110 π ), and their midpoint is (9/10,(1/10)mc)(9/10,(1/10)m_c). Therefore KL(p∥(p+r)/2)=KL(r∥(p+r)/2)=110Jπ,KL (p\|(p+r)/2 )=KL (r\|(p+r)/2 )= 110J_π, and hence JS(p,r)=110Jπ<Jπ=JS(p(S),r(S)).JS(p,r)= 110J_π<J_π=JS(p^(S),r^(S)). It remains to extend the counterexample to every D>3D>3. Put k=D−3k=D-3, choose ε∈(0,1/k) ∈(0,1/k), set a=1−kε∈(0,1)a=1-k ∈(0,1), and append k equal parts: p¯=(ap0,ap1,ap2,ε,…,ε⏟k),r¯=(ar0,ar1,ar2,ε,…,ε⏟k). p=(ap_0,ap_1,ap_2, ,…, _k), r=(ar_0,ar_1,ar_2, ,…, _k). These are strictly positive D-part compositions, and their subcompositions on S=1,2S=\1,2\ remain p(S)p^(S) and r(S)r^(S). The appended parts agree, so ∥p¯−r¯∥2=a∥p−r∥2<∥p(S)−r(S)∥2 p- r _2=a p-r _2< p^(S)-r^(S) _2 and, because the common appended parts contribute zero to both KL terms while the factor a cancels inside each logarithm, KL(p¯∥p¯+r¯2) ( p \| p+ r2 ) =aKL(p∥p+r2), =aKL (p \| p+r2 ), KL(r¯∥p¯+r¯2) ( r \| p+ r2 ) =aKL(r∥p+r2). =aKL (r \| p+r2 ). Consequently, JS(p¯,r¯)=aJS(p,r)<JS(p(S),r(S)).JS( p, r)=aJS(p,r)<JS(p^(S),r^(S)). Furthermore, 1−cos(p¯,r¯) 1- ( p, r) =a2(∥p∥22−p⊤r)a2∥p∥22+kε2 = a^2( p _2^2-p r)a^2 p _2^2+k ^2 <∥p∥22−p⊤r∥p∥22=92042<917=1−cos(p(S),r(S)). < p _2^2-p r p _2^2= 92042< 917=1- (p^(S),r^(S)). The first inequality is strict because kε2>0k ^2>0. Thus, for every D≥3D≥ 3, each classical dissimilarity can increase strictly after taking a subcomposition, so none is subcompositionally dominant. ∎ Remark C.1 (Scope of the invariance claims). A hard mask uses an offset −∞-∞ and is not an interior perturbation, so it lies outside Proposition 2(i) on the full simplex. One must first restrict and re-close on a common positive support. Within that reduced simplex, finite shared offsets are again governed by part (i); if positive pre-mask rows and a common retained support are available, part (iv) supplies contraction relative to their full distance. With unequal structural-zero supports, the common-support comparison discards the mismatch and no full Aitchison distance between the masked rows exists. Sink coherence always refers to a fixed designated sink (or to a simultaneous relabeling of both the composition and the sink designation). In particular, full dAd_A is decomposable but not sink-invariant; dA⟂d_A is perturbation-invariant, whereas CA⟂C_A is not when D≥3D≥ 3 (and is identically zero when D=2D=2); and dA⟂d_A and CA⟂C_A are temperature-equivariant rather than value-invariant. Full statement and proof of Theorem 2 Here mS^m is the open m-part simplex and m S_m the symmetric group. Under the operations above, the map Tm:=clrm:(m,⊕,⊙)→(ℋm,+,⋅)T_m:= _m:(S^m, , )→(H_m,+,·) is a linear isomorphism with inverse Tm−1(x)=(expx)T_m^-1(x)=C( x); for σ∈mσ∈ S_m we use the action (σp)i=pσ−1(i)(σ p)_i=p_σ^-1(i). The full statement of Theorem 2 is: if δD:D×D→[0,∞) _D:S^D×S^D→[0,∞) satisfies (A1) δD _D is a metric; (A2) δD(c⊕p,c⊕q)=δD(p,q) _D(c p,c q)= _D(p,q) for all c; (A3) δD(α⊙p,α⊙q)=|α|δD(p,q) _D(α p,α q)=|α| _D(p,q) for all α∈ℝα ; (A4) with N(v):=δD(v,eD)N(v):= _D(v,e_D), N(v⊕w)2+N(v⊖w)2=2N(v)2+2N(w)2N(v w)^2+N(v w)^2=2N(v)^2+2N(w)^2 for all v,wv,w; and (A5) δD(σp,σq)=δD(p,q) _D(σ p,σ q)= _D(p,q) for every σ∈Dσ∈ S_D; then there is a unique constant κD>0 _D>0 with δD=κDdA _D= _Dd_A on DS^D, and conversely every κDdA _Dd_A satisfies (A1)–(A5). Proof of Theorem 2. We give the complete argument. Reduction to a norm. Because T=clrDT= _D is bijective, the identities clrD(p⊕q)=clrD(p)+clrD(q),clrD(α⊙p)=αclrD(p) _D(p q)= _D(p)+ _D(q), _D(α p)=α _D(p) transport the usual vector-space laws on ℋDH_D to (D,⊕,⊙)(S^D, , ), whose zero is eDe_D. Applying (A2) with c=⊖qc= q gives δD(p,q)=δD(p⊖q,eD)=N(p⊖q). _D(p,q)= _D(p q,e_D)=N(p q). (15) By (A1), N(v)≥0N(v)≥ 0 and N(v)=0N(v)=0 if and only if v=eDv=e_D. Since α⊙eD=eDα e_D=e_D, (A3) yields N(α⊙v)=|α|N(v)N(α v)=|α|N(v). Finally, the triangle inequality in (A1) and perturbation invariance give N(v⊕w) N(v w) =δD(v⊕w,eD) = _D(v w,e_D) ≤δD(v⊕w,w)+δD(w,eD) ≤ _D(v w,w)+ _D(w,e_D) =δD(v,eD)+δD(w,eD)=N(v)+N(w), = _D(v,e_D)+ _D(w,e_D)=N(v)+N(w), where the third line follows by perturbing the first distance by ⊖w w. Hence N is a norm on the Aitchison vector space. The parallelogram identity. Assumption (A4) is exactly the parallelogram identity for N. The real Jordan–von Neumann theorem therefore implies that Bδ(v,w):=14N(v⊕w)2−N(v⊖w)2B_δ(v,w):= 14 \N(v w)^2-N(v w)^2 \ (16) is an inner product and that N(v)2=Bδ(v,v)N(v)^2=B_δ(v,v). Transport to CLR coordinates. Let T:=clrDT:= _D. Transporting BδB_δ through the linear isomorphism T:D→ℋDT:S^D _D gives an inner product on ℋDH_D. Because ℋDH_D is finite-dimensional, there is a unique Euclidean-self-adjoint positive-definite operator M:ℋD→ℋDM:H_D _D such that Bδ(v,w)=⟨T(v),MT(w)⟩2(v,w∈D).B_δ(v,w)= T(v),MT(w) _2 (v,w ^D). (17) Permutation invariance forces isotropy. Let PσP_σ denote the permutation matrix corresponding to σ∈Dσ∈ S_D, and put Qσ:=Pσ|ℋDQ_σ:=P_σ|_H_D. The space ℋDH_D is invariant under PσP_σ, and QσQ_σ is orthogonal. Coordinate permutations are Aitchison-linear: they commute with ⊕ , ⊖ , and ⊙ , and T(σv)=QσT(v)T(σ v)=Q_σT(v). Since σeD=eDσ e_D=e_D, assumption (A5) gives N(σv)=N(v)N(σ v)=N(v). Polarization in (16) consequently gives Bδ(σv,σw)=Bδ(v,w)B_δ(σ v,σ w)=B_δ(v,w), and hence Qσ∗MQσ=M.Q_σ^*MQ_σ=M. As QσQ_σ is orthogonal, M commutes with every QσQ_σ. For completeness, the coordinate-permutation representation on ℋDH_D is irreducible over ℝR. Indeed, let U be a nonzero linear subspace of ℋDH_D that is invariant under every coordinate permutation, and choose 0≠x∈U0≠ x∈ U. Not all coordinates of x can be equal, because D⊤x=01_D x=0; hence xi≠xjx_i≠ x_j for some i≠ji≠ j. If (ij)(ij) is the corresponding transposition and ie_i denotes the iith standard basis vector of ℝDR^D, then x−P(ij)x=(xi−xj)(i−j)∈U.x-P_(ij)x=(x_i-x_j)(e_i-e_j)∈ U. Thus i−j∈Ue_i-e_j∈ U. Permuting coordinates shows that k−ℓ∈Ue_k-e_ ∈ U for every k≠ℓk≠ , and these vectors span ℋDH_D. Therefore U=ℋDU=H_D. By the finite-dimensional spectral theorem, M has a real eigenvalue λ and a nonzero eigenspace EλE_λ. Since M commutes with all QσQ_σ, the space EλE_λ is invariant under every coordinate permutation. Irreducibility forces Eλ=ℋDE_λ=H_D, so M=λIℋD.M=λ I_H_D. (18) Positive definiteness gives λ>0λ>0. Conclusion and converse. Equations (15), (17), and (18) yield δD(p,q)2 _D(p,q)^2 =N(p⊖q)2 =N(p q)^2 =λ‖T(p⊖q)‖22 =λ \|T(p q) \|_2^2 =λ‖T(p)−T(q)‖22=λdA(p,q)2. =λ \|T(p)-T(q) \|_2^2=λ d_A(p,q)^2. Thus δD=κDdA _D= _Dd_A with κD:=λ>0 _D:= λ>0. Since D≥2D≥ 2, dAd_A is nonzero for some pair, so κD _D is unique. Conversely, fix κD>0 _D>0 and define δD(p,q):=κD‖T(p)−T(q)‖2 _D(p,q):= _D\|T(p)-T(q)\|_2. Since T is injective and Euclidean distance is a metric, (A1) holds. The CLR identities give δD(c⊕p,c⊕q) _D(c p,c q) =κD‖T(c)+T(p)−T(c)−T(q)‖2=δD(p,q), = _D\|T(c)+T(p)-T(c)-T(q)\|_2= _D(p,q), δD(α⊙p,α⊙q) _D(α p,α q) =κD‖α(T(p)−T(q))‖2=|α|δD(p,q), = _D\|α(T(p)-T(q))\|_2=|α|\, _D(p,q), which prove (A2) and (A3). Moreover, N(v)=κD‖T(v)‖2N(v)= _D\|T(v)\|_2, so the Euclidean parallelogram identity gives N(v⊕w)2+N(v⊖w)2 N(v w)^2+N(v w)^2 =κD2(‖T(v)+T(w)‖22+‖T(v)−T(w)‖22) = _D^2 (\|T(v)+T(w)\|_2^2+\|T(v)-T(w)\|_2^2 ) =2N(v)2+2N(w)2, =2N(v)^2+2N(w)^2, which is (A4). Finally, because every permutation matrix is orthogonal, δD(σp,σq)=κD‖Qσ(T(p)−T(q))‖2=δD(p,q), _D(σ p,σ q)= _D\|Q_σ(T(p)-T(q))\|_2= _D(p,q), which is (A5). ∎ Corollary 2 (Consequences under a dimension-compatible calibration). Assume D≥3D≥ 3 and let δD _D satisfy the hypotheses of Theorem 2. Write δD=κDdA _D= _Dd_A and, for each 2≤m<D2≤ m<D, define the compatible common-calibration distance δm[κD](r,s):=κDdA(r,s) _m^[ _D](r,s):= _Dd_A(r,s) for r,s∈mr,s ^m (with dAd_A the m-part CLR distance). Also put δD[κD]:=δD _D^[ _D]:= _D. Then: (i) Scale invariance. For x,y∈(0,∞)Dx,y∈(0,∞)^D, define δ¯D(x,y):=δD((x),(y)) δ_D(x,y):= _D(C(x),C(y)). For all a,b>0a,b>0, δ¯D(ax,by)=δ¯D(x,y) δ_D(ax,by)= δ_D(x,y). Thus δ¯D δ_D is a scale-invariant pseudometric on the positive cone; it is not a metric there, since proportional vectors have distance zero. (i) Subcompositional dominance. If S⊆1,…,DS \1,…,D\ has m:=|S|≥2m:=|S|≥ 2, let r(S):=((ri)i∈S)r^(S):=C((r_i)_i∈ S) for r∈Dr ^D, with the coordinates in their inherited order. Then δm[κD](p(S),q(S))≤δD(p,q). _m^[ _D](p^(S),q^(S))≤ _D(p,q). (i) Exact sink decomposition. Relabel the D coordinates by 0,1,…,D−10,1,…,D-1, with 00 the sink, and for any r∈Dr ^D put r′:=((,,,,,))r :=C((r_1,…,r_D-1)). Define bδ(r):=κDb(r)b_δ(r):= _D\,b(r) (with b the sink balance) and the content pseudometric δD⟂(p,q):=δD−1[κD](p′,q′)=κDdA(p′,q′) _D (p,q):= _D-1^[ _D](p ,q )= _Dd_A(p ,q ). Then δD(p,q)2=δD⟂(p,q)2+(bδ(p)−bδ(q))2. _D(p,q)^2= _D (p,q)^2+ (b_δ(p)-b_δ(q) )^2. (19) More precisely, with Wc:=x∈ℋD:x0=0W_c:=\x _D:x_0=0\ and PWcP_W_c Euclidean orthogonal projection onto WcW_c, δD⟂(p,q)=κD‖PWc(clrD(p)−clrD(q))‖2=κDdA(p′,q′). _D (p,q)= _D \|P_W_c ( _D(p)- _D(q) ) \|_2= _Dd_A(p ,q ). Thus the orthogonal projection recenters the kept CLR coordinates and is not mere deletion of the sink coordinate. Moreover, δD⟂ _D and the content log-ratio dispersion δ(r):=δD−1[κD](r′,eD−1)K_δ(r):= _D-1^[ _D](r ,e_D-1) depend only on content log-ratios and not on the sink mass. The full distance δD _D need not be sink-invariant, because it also contains the balance term in (19). (iv) Shared coordinate-bias invariance and temperature equivariance. Let smD(ℓ):=(expℓ)sm_D( ):=C( ). For all ℓ,ℓ′,u∈ℝD , ,u ^D and τ>0τ>0, δD(smD(ℓ+u),smD(ℓ′+u)) _D (sm_D( +u),sm_D( +u) ) =δD(smD(ℓ),smD(ℓ′)), = _D (sm_D( ),sm_D( ) ), δD(smD(ℓ/τ),smD(ℓ′/τ)) _D (sm_D( /τ),sm_D( /τ) ) =τ−1δD(smD(ℓ),smD(ℓ′)). =τ^-1 _D (sm_D( ),sm_D( ) ). Consequently, one common temperature applied to every row preserves all pairwise-distance rankings and ties. For n≥1n≥ 1 and rows p(1),…,p(n)∈Dp^(1),…,p^(n) ^D, with cen the Aitchison center and Vδ(p(1),…,p(n)):=1n∑j=1nδD(p(j),cen(p(1),…,p(n)))2,V_δ(p^(1),…,p^(n)):= 1n _j=1^n _D\! (p^(j), (p^(1),…,p^(n)) )^2, one has, writing cu:=smD(u)c_u:=sm_D(u), Vδ(cu⊕p(1),…,cu⊕p(n))=Vδ(p(1),…,p(n)),Vδ(1τ⊙p(1),…,1τ⊙p(n))=τ−2Vδ(p(1),…,p(n)).V_δ(c_u p^(1),…,c_u p^(n))=V_δ(p^(1),…,p^(n)), V_δ\! ( 1τ p^(1),…, 1τ p^(n) )=τ^-2V_δ(p^(1),…,p^(n)). Finally, put Nδ(r):=δD(r,eD)N_δ(r):= _D(r,e_D) and, for v,w∈Dv,w ^D, define Bδ(v,w):=14Nδ(v⊕w)2−Nδ(v⊖w)2,ρδ(v,w):=Bδ(v,w)Nδ(v)Nδ(w)(v,w≠eD).B_δ(v,w):= 14 \N_δ(v w)^2-N_δ(v w)^2 \, _δ(v,w):= B_δ(v,w)N_δ(v)N_δ(w) (v,w≠ e_D). Then ρδ(v,w)=⟨clrD(v),clrD(w)⟩2‖clrD(v)‖2‖clrD(w)‖2, _δ(v,w)= _D(v), _D(w) _2\| _D(v)\|_2\| _D(w)\|_2, and, for all α,β>0α,β>0, ρδ(α⊙v,β⊙w)=ρδ(v,w) _δ(α v,β w)= _δ(v,w): the cosine is invariant under separate positive powerings, equivalently arbitrary separate temperatures τv,τw>0 _v, _w>0. A shared nonconstant coordinate-bias vector generally does not preserve the cosine. Proof. Part (i) follows from (ax)=(x)C(ax)=C(x) and (by)=(y)C(by)=C(y). For part (i), put ai:=logpiqi,a¯D:=1D∑i=1Dai,a¯S:=1m∑i∈Sai,a_i:= p_iq_i, a_D:= 1D _i=1^Da_i, a_S:= 1m _i∈ Sa_i, so that z:=clrD(p)−clrD(q)z:= _D(p)- _D(q) satisfies zi=ai−a¯Dz_i=a_i- a_D. Also define WS:=x∈ℋD:xi=0 for i∉SW_S:=\x _D:x_i=0 for i∉ S\. The orthogonal projection of z onto WSW_S is (PWSz)i=ai−a¯S,i∈S,0,i∉S.(P_W_Sz)_i= casesa_i- a_S,&i∈ S,\\[4.0pt] 0,&i∉ S. cases Indeed, the displayed vector belongs to WSW_S, and, for every y∈WSy∈ W_S, ⟨z−PWSz,y⟩2=(a¯S−a¯D)∑i∈Syi=0. z-P_W_Sz,y _2=( a_S- a_D) _i∈ Sy_i=0. Its restriction to the coordinates in S is precisely clrm(p(S))−clrm(q(S)) _m(p^(S))- _m(q^(S)), and it vanishes on ScS^c. Therefore dA(p(S),q(S))=‖PWSz‖2≤‖z‖2=dA(p,q).d_A(p^(S),q^(S))=\|P_W_Sz\|_2≤\|z\|_2=d_A(p,q). Multiplication by the common positive constant κD _D proves part (i). For part (i), after the stated relabeling define ub:=D−1D(1,−1D−1,…,−1D−1),Wc:=x∈ℋD:x0=0.u_b:= D-1D (1,- 1D-1,…,- 1D-1 ), W_c:=\x _D:x_0=0\. Then ‖ub‖2=1\|u_b\|_2=1, ub⟂Wcu_b W_c, and ℋD=spanub⊕⟂WcH_D=span\u_b\ W_c. Direct calculation gives ⟨clrD(r),ub⟩2=b(r) _D(r),u_b _2=b(r) and PWcclrD(r)=(0,clrD−1(r′)1,…,clrD−1(r′)D−1).P_W_c _D(r)= (0, _D-1(r )_1,…, _D-1(r )_D-1 ). Applying Pythagoras to clrD(p)−clrD(q) _D(p)- _D(q) therefore yields dA(p,q)2=dA(p′,q′)2+(b(p)−b(q))2.d_A(p,q)^2=d_A(p ,q )^2+ (b(p)-b(q) )^2. Multiplying by κD2 _D^2 and using the definitions of δD⟂ _D and bδb_δ proves (19). The displayed formula for PWcclrD(r)P_W_c _D(r) proves the remaining sink claims. In particular, the zero-padded vector on the right is the orthogonal projection; simple deletion of the zeroth CLR coordinate would not give this vector without the displayed recentering. For part (iv), coordinatewise calculation gives smD(ℓ+u)=smD(u)⊕smD(ℓ),smD(ℓ/τ)=1τ⊙smD(ℓ).sm_D( +u)=sm_D(u) _D( ), _D( /τ)= 1τ _D( ). The two distance identities follow from (A2) and (A3). Moreover, clrD(cen(p(1),…,p(n)))=1n∑j=1nclrD(p(j)). _D\! ( (p^(1),…,p^(n)) )= 1n _j=1^n _D(p^(j)). Consequently, for every c∈Dc ^D and α∈ℝα , cen(c⊕p(1),…,c⊕p(n))=c⊕cen(p(1),…,p(n)), (c p^(1),…,c p^(n))=c (p^(1),…,p^(n)), cen(α⊙p(1),…,α⊙p(n))=α⊙cen(p(1),…,p(n)). (α p^(1),…,α p^(n))=α (p^(1),…,p^(n)). Applying the two distance identities term by term proves the formulas for VδV_δ. Finally, Theorem 2 and (18) give Bδ(v,w)=κD2⟨clrD(v),clrD(w)⟩2.B_δ(v,w)= _D^2 _D(v), _D(w) _2. For α,β>0α,β>0, bilinearity and absolute homogeneity give Bδ(α⊙v,β⊙w)=αβBδ(v,w),Nδ(α⊙v)=αNδ(v).B_δ(α v,β w)=αβ B_δ(v,w), N_δ(α v)=α N_δ(v). The factor κD2 _D^2, and then the positive powering factors, cancel upon normalizing, proving the cosine claims. To see that every nonconstant shared coordinate bias can change a cosine, fix c≠eDc≠ e_D and put y:=clrD(c)≠0y:= _D(c)≠ 0. Surjectivity of the CLR map provides v,w∈Dv,w ^D with CLR vectors y/2y/2 and −y/2-y/2, respectively. Then ρδ(v,w)=−1 _δ(v,w)=-1, whereas ρδ(c⊕v,c⊕w)=1 _δ(c v,c w)=1. ∎ Remark C.2 (Why the compatibility and scaling clauses are necessary). Theorem 2 is a fixed-dimensional result. If metrics are separately postulated in different dimensions, the theorem gives δm=κmdA _m= _md_A (in each dimension m), but (A1)–(A5) do not force the constants κm _m to agree. In general, the projection inequality gives only δm(p(S),q(S))≤κmκDδD(p,q). _m(p^(S),q^(S))≤ _m _D\, _D(p,q). Thus κm≤κD _m≤ _D is sufficient for dominance, while a common calibration is the clean equality convention used above. For example, let δ3=dA _3=d_A, δ2=100dA _2=100\,d_A, q=e3q=e_3, p=((,,,,,))p=C(( (a), (-a),1)) with a≠0a≠ 0, and take S=1,2S=\1,2\. Then dA(p,q)=dA(p(S),q(S))=2|a|d_A(p,q)=d_A(p^(S),q^(S))= 2\,|a| (three- and two-part CLR distances, respectively), and therefore δ2(p(S),q(S))=1002|a|>2|a|=δ3(p,q). _2(p^(S),q^(S))=100 2\,|a|> 2\,|a|= _3(p,q). Thus cross-dimensional dominance fails even though both metrics separately satisfy (A1)–(A5). The common-calibration family in the corollary is therefore an explicit compatibility convention, not a consequence of the fixed-D axioms alone. Likewise, the unscaled balance identity must be adjusted when κD≠1 _D≠ 1: δD(p,q)2=κD2dA(p′,q′)2+κD2(b(p)−b(q))2 _D(p,q)^2= _D^2d_A(p ,q )^2+ _D^2(b(p)-b(q))^2, equivalently one uses δD⟂=κDdA _D = _Dd_A on contents and bδ=κDb_δ= _Db, as in the corollary. If instead an independently calibrated distance δD−1=κD−1dA _D-1= _D-1d_A is used, the exact identity is δD(p,q)2=(κDκD−1)2δD−1(p′,q′)2+(bδ(p)−bδ(q))2. _D(p,q)^2= ( _D _D-1 )^2 _D-1(p ,q )^2+ (b_δ(p)-b_δ(q) )^2. Finally, shared-bias invariance applies when both endpoints of a distance, or all rows and their center, receive the same perturbation. It does not imply that the one-row concentration dA(p′,eD−1)d_A(p ,e_D-1), or the CLR cosine, is invariant under an arbitrary nonconstant content bias with the reference eD−1e_D-1 held fixed. Remark C.3 (Tightness of the axioms). N1(v)=∥clrv∥1N_1(v)= v _1 satisfies (A1)–(A3) and (A5): it is a norm on the Aitchison vector space (hence induces a perturbation-invariant metric), it is absolutely homogeneous under powering because clr(α⊙v)=αclrv (α v)=α v, and the ℓ1 _1 norm is permutation-symmetric. It fails (A4) whenever D≥3D≥ 3: for u=(1,−1,0,…)u=(1,-1,0,…) and v=(1,0,−1,0,…)v=(1,0,-1,0,…) in ℋH, N1(u)=N1(v)=2N_1(u)=N_1(v)=2, N1(u+v)=4N_1(u+v)=4, N1(u−v)=2N_1(u-v)=2, and 16+4≠2⋅4+2⋅416+4≠ 2· 4+2· 4. So the parallelogram axiom is not implied by the others; dropping it admits genuinely different coherent geometries, still perturbation- and permutation-invariant and temperature-homogeneous, among which (A4) selects the unique Euclidean one. This is the precise sense in which “quasi-coherent” log-ratio alternatives [8] relate to dAd_A. Appendix D Proofs for Section 4.3 Throughout this section, coordinate 00 is the designated sink, x′x denotes the sink-dropped, re-closed content composition of x∈Dx ^D (D≥3D≥ 3), H(x):=−∑jxjlogxjH(x):=- _jx_j x_j with 0log0:=00 0:=0, and dA⟂(x,y):=dA(x′,y′)d_A (x,y):=d_A(x ,y ), CA⟂(x):=∥clrD−1(x′)∥2C_A (x):= _D-1(x ) _2 with no dimension-dependent rescaling. Full statement of Theorem 3 Fix an integer D≥3D≥ 3 and s0∈(12,1)s_0∈( 12,1), and designate coordinate 00 as the sink coordinate. (i) For nonzero x,y∈ℝDx,y ^D, write c(x,y):=⟨x,y⟩/(∥x∥2∥y∥2)c(x,y):= x,y /( x _2 y _2), and let JSJS denote the equal-weight Jensen–Shannon divergence formed from Kullback–Leibler divergence with natural logarithms. If p,q∈Dp,q ^D satisfy p0,q0≥s0p_0,q_0≥ s_0, then 1−c(p,q) 1-c(p,q) <(1−s0)2s02+(1−s0)2≤(1−s0s0)2, < (1-s_0)^2s_0^2+(1-s_0)^2≤ ( 1-s_0s_0 )^2, ∥p−q∥2 p-q _2 <2(1−s0)≤5(1−s0), < 2\,(1-s_0)≤ 5\,(1-s_0), JS(p,q) (p,q) <(1−s0)log2≤32(1−s0). <(1-s_0) 2≤ 32(1-s_0). All three sharp endpoints are nevertheless approached from within the open simplex. More precisely, supp,q∈Dp0,q0≥s01−c(p,q) _ subarraycp,q ^D\\ p_0,q_0≥ s_0 subarray\1-c(p,q)\ =(1−s0)2s02+(1−s0)2, = (1-s_0)^2s_0^2+(1-s_0)^2, supp,q∈Dp0,q0≥s0∥p−q∥2 _ subarraycp,q ^D\\ p_0,q_0≥ s_0 subarray p-q _2 =2(1−s0), = 2\,(1-s_0), supp,q∈Dp0,q0≥s0JS(p,q) _ subarraycp,q ^D\\ p_0,q_0≥ s_0 subarrayJS(p,q) =(1−s0)log2. =(1-s_0) 2. Thus none of the three suprema is attained. In fact, their respective ranges are exactly [0,(1−s0)2s02+(1−s0)2),[0,2(1−s0)),[0,(1−s0)log2), [0, (1-s_0)^2s_0^2+(1-s_0)^2 ), [0, 2(1-s_0) ), [0,(1-s_0) 2 ), and their upper endpoints are sharp. In particular, all three range widths tend to zero as s0↑1s_0 1, while the cosine range has the quadratic asymptotic behavior (1−s0)2s02+(1−s0)2∼(1−s0)2 (1-s_0)^2s_0^2+(1-s_0)^2 (1-s_0)^2 as s0↑1s_0 1. (i) For r∈Dr ^D, define the conditional content composition and the content-preserving sink-mass transformation by r′:=(r1,…,rD−1)1−r0,Ttr:=(t,(1−t)r′),0<t<1.r := (r_1,…,r_D-1)1-r_0, T_tr:= (t,(1-t)r ), 0<t<1. Then, for arbitrary r,z∈Dr,z ^D and t,τ∈(0,1)t,τ∈(0,1), dA⟂(Ttr,Tτz)=dA⟂(r,z),CA⟂(Ttr)=CA⟂(r),CA⟂(Tτz)=CA⟂(z).d_A (T_tr,T_τz)=d_A (r,z), C_A (T_tr)=C_A (r), C_A (T_τz)=C_A (z). Consequently, for every integer n≥1n≥ 1, every r(1),…,r(n)∈Dr^(1),…,r^(n) ^D, and every t1,…,tn∈(0,1)t_1,…,t_n∈(0,1), dA⟂(Ttir(i),Ttjr(j))=dA⟂(r(i),r(j))(1≤i,j≤n),CA⟂(Ttir(i))=CA⟂(r(i))(1≤i≤n).d_A (T_t_ir^(i),T_t_jr^(j) )=d_A (r^(i),r^(j) ) (1≤ i,j≤ n), C_A (T_t_ir^(i) )=C_A (r^(i) ) (1≤ i≤ n). Hence composition-specific changes of sink mass that preserve each conditional content composition leave the entire content-distance matrix and all content-concentration values unchanged. In particular, choosing r0(i)<ti<1r^(i)_0<t_i<1 increases every designated sink mass while leaving the conditional-content geometry exactly unchanged. Proof of Theorem 3. Fix p,q∈Dp,q ^D with p0,q0≥s0p_0,q_0≥ s_0. Put ε:=1−s0 :=1-s_0, and set sp:=p0,sq:=q0,πp:=(p1,…,pD−1)1−sp,πq:=(q1,…,qD−1)1−sq.s_p:=p_0, s_q:=q_0, _p:= (p_1,…,p_D-1)1-s_p, _q:= (q_1,…,q_D-1)1-s_q. Because p,q∈Dp,q ^D, the vectors πp,πq _p, _q are strictly positive probability vectors with D−1≥2D-1≥ 2 coordinates. Hence p=(sp,(1−sp)πp),q=(sq,(1−sq)πq),p= (s_p,(1-s_p) _p ), q= (s_q,(1-s_q) _q ), and ∥πp∥2<1,∥πq∥2<1,⟨πp,πq⟩>0. _p _2<1, _q _2<1, _p, _q >0. Cosine dissimilarity. We have ⟨p,q⟩ p,q =spsq+(1−sp)(1−sq)⟨πp,πq⟩, =s_ps_q+(1-s_p)(1-s_q) _p, _q , ∥p∥22 p _2^2 =sp2+(1−sp)2∥πp∥22≤sp2+(1−sp)2, =s_p^2+(1-s_p)^2 _p _2^2≤ s_p^2+(1-s_p)^2, and the analogous norm inequality holds for q. Define h(s):=s2+(1−s)2,0<s<1.h(s):= s s^2+(1-s)^2, 0<s<1. Since sp,sq<1s_p,s_q<1 and ⟨πp,πq⟩>0 _p, _q >0, c(p,q) c(p,q) >spsq∥p∥2∥q∥2 > s_ps_q p _2 q _2 ≥spsqsp2+(1−sp)2sq2+(1−sq)2=h(sp)h(sq). ≥ s_ps_q s_p^2+(1-s_p)^2 s_q^2+(1-s_q)^2=h(s_p)h(s_q). Moreover, h′(s)=1−ss2+(1−s)23/2>0.h (s)= 1-s\s^2+(1-s)^2\^3/2>0. Since sp,sq≥s0s_p,s_q≥ s_0, it follows that c(p,q)>h(s0)2=s02s02+(1−s0)2,c(p,q)>h(s_0)^2= s_0^2s_0^2+(1-s_0)^2, which proves the first strict bound. The coarser bound follows because s02+(1−s0)2≥s02s_0^2+(1-s_0)^2≥ s_0^2. Euclidean distance. Let a:=1−spa:=1-s_p and b:=1−sqb:=1-s_q. Then 0<a,b≤ε0<a,b≤ , and ∥p−q∥22 p-q _2^2 =(a−b)2+∥aπp−bπq∥22 =(a-b)^2+ a _p-b _q _2^2 =(a−b)2+a2∥πp∥22+b2∥πq∥22−2ab⟨πp,πq⟩ =(a-b)^2+a^2 _p _2^2+b^2 _q _2^2-2ab _p, _q <(a−b)2+a2+b2 <(a-b)^2+a^2+b^2 =2(a2+b2−ab). =2(a^2+b^2-ab). If a≥ba≥ b, then a2+b2−ab=a2−b(a−b)≤a2;a^2+b^2-ab=a^2-b(a-b)≤ a^2; if b≥ab≥ a, the symmetric argument gives a2+b2−ab≤b2a^2+b^2-ab≤ b^2. Therefore ∥p−q∥22<2maxa,b2≤2ε2, p-q _2^2<2 \a,b\^2≤ 2 ^2, which proves the second strict bound; the displayed 5 5-bound is the immediate numerical relaxation 2≤5 2≤ 5. Jensen–Shannon divergence. Let e0,…,eD−1e_0,…,e_D-1 be the standard coordinate vectors in ℝDR^D, and define u:=p−s0e0ε,v:=q−s0e0ε.u:= p-s_0e_0 , v:= q-s_0e_0 . The assumptions p0,q0≥s0p_0,q_0≥ s_0 imply that u,vu,v have nonnegative coordinates, and ∑i=0D−1ui=∑i=0D−1vi=1−s0ε=1. _i=0^D-1u_i= _i=0^D-1v_i= 1-s_0 =1. Thus u,vu,v are probability vectors in the closed simplex, and p=s0e0+εu,q=s0e0+εv.p=s_0e_0+ u, q=s_0e_0+ v. Set w:=u+v2,m:=p+q2=s0e0+εw.w:= u+v2, m:= p+q2=s_0e_0+ w. Then supp(u)∪supp(v)=supp(w)supp(u) (v)=supp(w). We use the standard closed-simplex convention KL(x∥y):=∑i:xi>0xilogxiyiKL(x\|y):= _i:x_i>0x_i x_iy_i when supp(x)⊆supp(y)supp(x) (y), and set the divergence equal to +∞+∞ otherwise. Thus all divergences below are finite. Joint convexity of Kullback–Leibler divergence gives KL(p∥m) (p\|m) ≤s0KL(e0∥e0)+εKL(u∥w)=εKL(u∥w), ≤ s_0KL(e_0\|e_0)+ (u\|w)= (u\|w), KL(q∥m) (q\|m) ≤s0KL(e0∥e0)+εKL(v∥w)=εKL(v∥w). ≤ s_0KL(e_0\|e_0)+ (v\|w)= (v\|w). Consequently, JS(p,q)≤εJS(u,v).JS(p,q)≤ (u,v). For any closed-simplex probability vectors x,yx,y, put z=(x+y)/2z=(x+y)/2. Whenever xi>0x_i>0, logxizi=log2xixi+yi≤log2, x_iz_i= 2x_ix_i+y_i≤ 2, with equality if and only if yi=0y_i=0. Hence KL(x∥z)≤log2KL(x\|z)≤ 2, and the analogous conclusion holds with x,yx,y interchanged. It follows that JS(x,y)≤log2,JS(x,y)=log2⟺supp(x)∩supp(y)=∅.JS(x,y)≤ 2, (x,y)= 2 (x) (y)= . For every i=1,…,D−1i=1,…,D-1, however, ui=piε>0,vi=qiε>0.u_i= p_i >0, v_i= q_i >0. Thus the supports of u and v overlap, and therefore JS(p,q)≤εJS(u,v)<εlog2=(1−s0)log2.JS(p,q)≤ (u,v)< 2=(1-s_0) 2. This proves the third strict bound; log2<32 2< 32 gives the stated coarser bound. Sharpness. The assumption D≥3D≥ 3 provides two distinct nonsink coordinates. In the closed simplex, let p∗:=s0e0+εe1,q∗:=s0e0+εe2,m∗:=p∗+q∗2.p^*:=s_0e_0+ e_1, q^*:=s_0e_0+ e_2, m^*:= p^*+q^*2. Direct calculation gives 1−c(p∗,q∗) 1-c(p^*,q^*) =ε2s02+ε2, = ^2s_0^2+ ^2, ∥p∗−q∗∥2 p^*-q^* _2 =2ε, = 2\, , KL(p∗∥m∗) (p^*\|m^*) =KL(q∗∥m∗)=εlog2, =KL(q^*\|m^*)= 2, and hence JS(p∗,q∗)=εlog2JS(p^*,q^*)= 2. To approximate this pair from within DS^D, fix 0<δ<1/(D−1)0<δ<1/(D-1) and, for j∈1,…,D−1j∈\1,…,D-1\, define (πp,δ)j:=1−(D−2)δ,j=1,δ,j≠1,(πq,δ)j:=1−(D−2)δ,j=2,δ,j≠2.( _p,δ)_j:= cases1-(D-2)δ,&j=1,\\ δ,&j≠ 1, cases ( _q,δ)_j:= cases1-(D-2)δ,&j=2,\\ δ,&j≠ 2. cases Set pδ:=(s0,επp,δ),qδ:=(s0,επq,δ).p_δ:= (s_0, _p,δ ), q_δ:= (s_0, _q,δ ). Each πp,δ _p,δ and πq,δ _q,δ is a strictly positive probability vector, so pδ,qδ∈Dp_δ,q_δ ^D, and (pδ,qδ)⟶(p∗,q∗)as δ↓0.(p_δ,q_δ) (p^*,q^*) δ 0. Cosine dissimilarity and Euclidean distance are continuous at this closed-simplex limit. Jensen–Shannon divergence is also continuous there because JS(x,y)=H(x+y2)−H(x)+H(y)2,JS(x,y)=H\! ( x+y2 )- H(x)+H(y)2, with 0log0:=00 0:=0, and t↦−tlogt -t t is continuous on [0,1][0,1] under this convention. The three upper endpoints are therefore the exact suprema. Their nonattainment follows from the strict bounds already proved. Finally, all three dissimilarities are nonnegative: for cosine dissimilarity this follows from Cauchy–Schwarz, for Euclidean distance it is immediate, and for Jensen–Shannon divergence it follows from nonnegativity of Kullback–Leibler divergence. Each takes the value zero when p=qp=q. The feasible set of pairs is convex and hence path connected, and each of the three dissimilarities is continuous on it. The image of this set under each dissimilarity is therefore an interval. Because that interval contains zero, has the displayed supremum, and does not contain its supremum, it is exactly the corresponding half-open interval stated in part (i). This proves all assertions in part (i). For part (i), fix r,z∈Dr,z ^D and t,τ∈(0,1)t,τ∈(0,1). For j=1,…,D−1j=1,…,D-1, one has rj′>0r _j>0, and ∑j=1D−1rj′=∑j=1D−1rj1−r0=1. _j=1^D-1r _j= _j=1^D-1r_j1-r_0=1. Thus r′r is a strictly positive probability vector, and the same holds for z′z . Moreover, all coordinates of TtrT_tr are strictly positive and ∑j=0D−1(Ttr)j=t+(1−t)∑j=1D−1rj′=1. _j=0^D-1(T_tr)_j=t+(1-t) _j=1^D-1r _j=1. Hence Ttr∈DT_tr ^D, and similarly Tτz∈DT_τz ^D. Directly, (Ttr)′=(1−t)r′1−t=r′,(Tτz)′=(1−τ)z′1−τ=z′.(T_tr) = (1-t)r 1-t=r , (T_τz) = (1-τ)z 1-τ=z . Therefore dA⟂(Ttr,Tτz) d_A (T_tr,T_τz) =dA((Ttr)′,(Tτz)′)=dA(r′,z′)=dA⟂(r,z), =d_A ((T_tr) ,(T_τz) )=d_A(r ,z )=d_A (r,z), CA⟂(Ttr) C_A (T_tr) =∥clrD−1((Ttr)′)∥2=∥clrD−1(r′)∥2=CA⟂(r), = _D-1((T_tr) ) _2= _D-1(r ) _2=C_A (r), and the same calculation yields CA⟂(Tτz)=CA⟂(z)C_A (T_τz)=C_A (z). Applying these identities to every pair (i,j)(i,j) and every index i in a finite family proves the two finite-family claims. Finally, if r0(i)<ti<1r^(i)_0<t_i<1, then (Ttir(i))0=ti>r0(i) (T_t_ir^(i) )_0=t_i>r^(i)_0, which proves the last assertion. The restrictions ti∈(0,1)t_i∈(0,1) ensure that every transformed vector remains in the open simplex; no endpoint assertion is required. ∎ Full statement of Proposition 3 Fix an integer D≥3D≥ 3, index the sink coordinate by 00, and, for x=(x0,…,xD−1)∈Dx=(x_0,…,x_D-1) ^D, write x′:=(x1,…,xD−1)=(x1,…,xD−1)/(1−x0)∈D−1x :=C(x_1,…,x_D-1)=(x_1,…,x_D-1)/(1-x_0) ^D-1. (i) Let h1,…,hn∈Dh_1,…,h_n ^D, n≥2n≥ 2, be a finite labelled collection of heads. Define the content-distance matrix computed without first modifying the stored heads and the Aitchison-distance matrix computed after explicit sink removal by Δijcontent:=dA⟂(hi,hj),Δijexplicit-drop:=dA(hi′,hj′). ^content_ij:=d_A (h_i,h_j), ^explicit -drop_ij:=d_A(h_i ,h_j ). Then Δcontent=Δexplicit-drop ^content= ^explicit -drop entrywise. Consequently, any deterministic clustering rule of the form (Δ,η)A( ;η), whose sole data-dependent input is the labelled dissimilarity matrix and whose ancillary specification η, including matrix-level preprocessing, linkage, hyperparameters, tie-breaking, and stopping rules, is held fixed, gives identical outputs in the two analyses. In particular, the rule may not additionally inspect the raw heads, their sink masses, or their ambient dimension. If the same measurable randomized implementation is written as (Δ,U,η)A( ,U;η), and in both analyses U has conditional law K(du∣Δ,η)K(du ,η), then the outputs have the same distribution: for every measurable output event B, both output probabilities equal ∫(Δ,u;η)∈BK(du∣Δ,η). 1\! \A( ,u;η)∈ B \K(du ,η). Moreover, there exists a common-randomness coupling: draw one U∼K(⋅∣Δ,η)U K(\,· ,η) and use it in both analyses; under this coupling the outputs are equal almost surely. This identity concerns dA⟂d_A ; it does not assert that the ordinary full-composition distance dA(hi,hj)d_A(h_i,h_j) is unchanged by sink removal. (i) For probability vectors x,yx,y of the same finite length, possibly with zero coordinates, define c(x,y):=⟨x,y⟩/(‖x‖2‖y‖2)c(x,y):= x,y /(\|x\|_2\|y\|_2), dcos(x,y):=1−c(x,y)d_ (x,y):=1-c(x,y), dE(x,y):=‖x−y‖2d_E(x,y):=\|x-y\|_2, and JS(x,y):=H(x+y2)−12H(x)−12H(y)=12KL(x∥x+y2)+12KL(y∥x+y2),JS(x,y):=H\! ( x+y2 )- 12H(x)- 12H(y)= 12KL\! (x \| x+y2 )+ 12KL\! (y \| x+y2 ), where KL(x∥y):=∑ℓ:xℓ>0xℓlog(xℓ/yℓ)KL(x\|y):= _ :x_ >0x_ (x_ /y_ ), with KL(x∥y)=+∞KL(x\|y)=+∞ if xℓ>0x_ >0 and yℓ=0y_ =0 for some ℓ ; in the displayed Jensen–Shannon formula the midpoint has support containing those of both arguments, so both relative entropies are finite. There exists a nonempty relatively open set D⊂(D)3U_D⊂(S^D)^3 having positive relative 3(D−1)3(D-1)-dimensional Lebesgue measure in the affine hull, and hence positive 3(D−1)3(D-1)-dimensional Hausdorff measure, such that, for every (p,q,r)∈D(p,q,r) _D, simultaneously for δ∈dcos,JS,dEδ∈\d_ ,JS,d_E\, the unique closest pair with the sink retained is (p,r)(p,r), whereas the unique closest pair after sink removal is (p,q)(p,q). Hence, for each of the three dissimilarities, both standard single-linkage and standard complete-linkage agglomerative clustering, initialized at singleton clusters and stopped at K=2K=2, return keep=p,r,q,drop=p,q,r.P_keep= \\p,r\,\q\ \, _drop= \\p,q\,\r\ \. For D=3D=3, the set 3U_3 contains the exact witness p=150(45,4,1),q=150(35,12,3),r=150(45,1,4).p= 150(45,4,1), q= 150(35,12,3), r= 150(45,1,4). Proof of Proposition 3. For part (i), the standing definition gives, for every i,ji,j, Δijcontent=dA⟂(hi,hj)=dA(hi′,hj′)=Δijexplicit-drop. ^content_ij=d_A (h_i,h_j)=d_A(h_i ,h_j )= ^explicit -drop_ij. Thus both analyses supply exactly the same labelled matrix to the same clustering implementation. The deterministic outputs are therefore equal. In the randomized case, the displayed integral in the statement depends only on the common pair (Δ,η)( ,η), so the two output laws coincide. If the same draw U is used in both analyses, the two arguments of A agree pointwise, and hence the coupled outputs are equal almost surely. For part (i), first establish nonemptiness in every dimension D≥3D≥ 3. Choose π∈D−1π ^D-1 whose first two coordinates are unequal, and let π~ π be obtained by interchanging those two coordinates. Fix a∈(0,1)a∈(0,1), and, for s∈(a,1)s∈(a,1), define ps=(s,(1−s)π),q=(a,(1−a)π),rs=(s,(1−s)π~).p_s= (s,(1-s)π ), q= (a,(1-a)π ), r_s= (s,(1-s) π ). Then ps′=q′=πp_s =q =π and rs′=π~≠πr_s = π≠π. Each of dcosd_ , JSJS, and dEd_E is nonnegative and vanishes on probability vectors precisely when its two arguments are equal. This is immediate for dEd_E. For dcosd_ , it follows from the equality case of Cauchy–Schwarz, because proportional unit-sum probability vectors must be equal. For JSJS, it follows from the strict concavity of H. Hence, for every δ∈dcos,JS,dEδ∈\d_ ,JS,d_E\, δ(ps′,q′)=0<δ(ps′,rs′)=δ(q′,rs′).δ(p_s ,q )=0<δ(p_s ,r_s )=δ(q ,r_s ). Let e0=(1,0,…,0)e_0=(1,0,…,0). As s↑1s 1, both psp_s and rsr_s converge to e0e_0, whereas q≠e0q≠ e_0. Euclidean and cosine dissimilarities are continuous on the closed simplex; for cosine, the denominator never vanishes there. The entropy definition of JSJS, together with 0log0=00 0=0, shows that JSJS is also continuous on the closed simplex. Therefore, for each such δ, δ(ps,rs)⟶0,δ(ps,q)⟶δ(e0,q)>0,δ(q,rs)⟶δ(q,e0)>0.δ(p_s,r_s) 0, δ(p_s,q) δ(e_0,q)>0, δ(q,r_s) δ(q,e_0)>0. Because the family of dissimilarities is finite, there exists a common s∗∈(a,1)s_*∈(a,1), sufficiently close to 11, such that, simultaneously for all three, δ(ps∗,rs∗)<minδ(ps∗,q),δ(q,rs∗).δ(p_s_*,r_s_*)< \δ(p_s_*,q),δ(q,r_s_*) \. Define DU_D to be the set of triples (p,q,r)∈(D)3(p,q,r)∈(S^D)^3 satisfying, for every δ∈dcos,JS,dEδ∈\d_ ,JS,d_E\, δ(p,r) δ(p,r) <δ(p,q), <δ(p,q), δ(p,r) δ(p,r) <δ(q,r), <δ(q,r), δ(p′,q′) δ(p ,q ) <δ(p′,r′), <δ(p ,r ), δ(p′,q′) δ(p ,q ) <δ(q′,r′). <δ(q ,r ). The preceding construction shows that (ps∗,q,rs∗)∈D(p_s_*,q,r_s_*) _D, so DU_D is nonempty. The sink-removal map is continuous on the open simplex because 1−x0>01-x_0>0, and all three dissimilarities are continuous there. Thus DU_D, being defined by finitely many strict inequalities, is relatively open in (D)3(S^D)^3. Every nonempty relatively open subset of this 3(D−1)3(D-1)-dimensional open set in its affine hull contains a relative open ball. It therefore has positive relative Lebesgue measure and positive 3(D−1)3(D-1)-dimensional Hausdorff measure. It remains to verify the displayed D=3D=3 witness exactly. For Euclidean distance, ‖p−r‖22=182500<1682500=‖p−q‖22<2222500=‖q−r‖22.\|p-r\|_2^2= 182500< 1682500=\|p-q\|_2^2< 2222500=\|q-r\|_2^2. Thus (p,r)(p,r) is the unique Euclidean closest pair. Multiplying all three vectors by 5050, which does not change cosine similarity, gives P=(45,4,1)P=(45,4,1), Q=(35,12,3)Q=(35,12,3), R=(45,1,4)R=(45,1,4). Consequently, c(p,r)=20332042,c(p,q)=16262042⋅1378,c(q,r)=15992042⋅1378.c(p,r)= 20332042, c(p,q)= 1626 2042· 1378, c(q,r)= 1599 2042· 1378. The last two quantities have the same positive denominator. Moreover, all three displayed cosine similarities are positive, and therefore c(p,r)>c(p,q)c(p,r)>c(p,q) is equivalent, after squaring, to 20332⋅1378−16262⋅2042=296601850>0.2033^2· 1378-1626^2· 2042=296601850>0. This comparison and 1626>15991626>1599 imply c(p,r)>c(p,q)>c(q,r)c(p,r)>c(p,q)>c(q,r). Thus (p,r)(p,r) is also the unique closest pair under dcos=1−cd_ =1-c. We next record the two Jensen–Shannon mixture identities used below. Let u,vu,v be probability vectors of the same size, put w=(u+v)/2w=(u+v)/2, and let s,t∈(0,1)s,t∈(0,1). Then KL((s,(1−s)u)∥(s,(1−s)w))=(1−s)KL(u∥w),KL\! ((s,(1-s)u) \|(s,(1-s)w) )=(1-s)KL(u\|w), and the analogous identity holds with v. It follows that JS((s,(1−s)u),(s,(1−s)v))=(1−s)JS(u,v).JS ((s,(1-s)u),(s,(1-s)v) )=(1-s)JS(u,v). Next put s¯=(s+t)/2 s=(s+t)/2. The midpoint of (s,(1−s)u)(s,(1-s)u) and (t,(1−t)u)(t,(1-t)u) is (s¯,(1−s¯)u)( s,(1- s)u), and direct summation gives KL((s,(1−s)u)∥(s¯,(1−s¯)u))=KL((s,1−s)∥(s¯,1−s¯)).KL\! ((s,(1-s)u) \|( s,(1- s)u) )=KL ((s,1-s)\|( s,1- s) ). Applying the analogous identity to t and averaging yields JS((s,(1−s)u),(t,(1−t)u))=JS((s,1−s),(t,1−t)).JS ((s,(1-s)u),(t,(1-t)u) )=JS ((s,1-s),(t,1-t) ). For the witness, put π+=(45,15),π−=(15,45). _+= ( 45, 15 ), _-= ( 15, 45 ). The first mixture identity and direct calculation give JS(p,r) (p,r) =110JS(π+,π−)=110(45log85+15log25)=150log(81923125)<150. = 110JS( _+, _-)= 110 ( 45 85+ 15 25 )= 150 \! ( 81923125 )< 150. Indeed, 81923125<6524=∑k=041k!<e. 81923125< 6524= _k=0^4 1k!<e. Because p and q have the same conditional content composition, the second mixture identity gives JS(p,q)=JS((910,110),(710,310)).JS(p,q)=JS\! ( ( 910, 110 ), ( 710, 310 ) ). The midpoint of these two binary probability vectors is m=(45,15)m= ( 45, 15 ), and each endpoint has total-variation distance 1/101/10 from m, where TV(u,v):=12‖u−v‖1TV(u,v):= 12\|u-v\|_1. Pinsker’s inequality for natural logarithms, KL(u∥v)≥2TV(u,v)2KL(u\|v)≥ 2TV(u,v)^2, therefore gives JS(p,q)≥12(150+150)=150>JS(p,r).JS(p,q)≥ 12 ( 150+ 150 )= 150>JS(p,r). Now define the deterministic coarse-graining T(x0,x1,x2):=(x0,x1+x2)T(x_0,x_1,x_2):=(x_0,x_1+x_2). Since T(x+y2)=Tx+Ty2T\! ( x+y2 )= Tx+Ty2, the data-processing inequality applied to each relative-entropy term defining Jensen–Shannon divergence gives JS(q,r) (q,r) ≥JS(Tq,Tr)=JS((710,310),(910,110))=JS(p,q)>JS(p,r). (Tq,Tr)=JS\! ( ( 710, 310 ), ( 910, 110 ) )=JS(p,q)>JS(p,r). The last equality uses symmetry of JSJS and the second mixture identity. Thus (p,r)(p,r) is the unique Jensen–Shannon closest pair. After sink removal, p′=q′=(45,15),r′=(15,45).p =q = ( 45, 15 ), r = ( 15, 45 ). The (p,q)(p,q) dissimilarity is therefore zero. Since p′=q′p =q , the common dissimilarity of each of the other two pairs is dcos(p′,r′) d_ (p ,r ) =917>0, = 917>0, JS(p′,r′) (p ,r ) =15log(81923125)>0, = 15 \! ( 81923125 )>0, dE(p′,r′) d_E(p ,r ) =325>0. = 3 25>0. Hence (p,q)(p,q) is uniquely closest after sink removal under all three dissimilarities. Finally, with three singleton clusters, both single linkage and complete linkage assign to a pair of clusters exactly the underlying pairwise dissimilarity. Their first merge is therefore the unique closest pair. Stopping immediately after that merge, at K=2K=2, gives the two asserted partitions. ∎ Full statement of Corollary 1 Fix an integer D≥3D≥ 3, let ∅≠ be a checkpoint index set, and let pt∈Dp_t ^D for every t∈t . Write uniquely pt=(st,(1−st)πt),st=(pt)0∈(0,1),πt=pt′∈D−1,p_t= (s_t,(1-s_t) _t ), s_t=(p_t)_0∈(0,1), _t=p_t ^D-1, and define Hb(s):=−slogs−(1−s)log(1−s)H_b(s):=-s s-(1-s) (1-s). Then, for every t∈t , H(pt)=Hb(st)+(1−st)H(πt).H(p_t)=H_b(s_t)+(1-s_t)H( _t). (20) Suppose that, for some π∈D−1π ^D-1, πt=π _t=π for every t∈t . Then CA⟂(pt)=‖clrD−1(π)‖2C_A (p_t)=\| _D-1(π)\|_2 is independent of t. If fπ(s):=H(s,(1−s)π),s∈(0,1),θπ:=11+exp(H(π)),f_π(s):=H (s,(1-s)π ), s∈(0,1), _π:= 11+ (H(π)), then fπf_π is strictly increasing on (0,θπ](0, _π] and strictly decreasing on [θπ,1)[ _π,1), with unique maximizer θπ _π on (0,1)(0,1). In particular, θπ<1/2 _π<1/2, and, for any t,u∈t,u , 12≤st<su<1⟹H(pu)<H(pt)andCA⟂(pu)=CA⟂(pt). 12≤ s_t<s_u<1 H(p_u)<H(p_t) C_A (p_u)=C_A (p_t). Moreover, let uD−1:=(1D−1,…,1D−1)u_D-1:=( 1D-1,…, 1D-1) and ps:=(s,(1−s)uD−1)p_s:= (s,(1-s)u_D-1 ) for s∈(0,1)s∈(0,1). Then ps′=uD−1,CA⟂(ps)=0,H(p1/D)=logD,lims↑1H(ps)=0.p_s =u_D-1, C_A (p_s)=0, H(p_1/D)= D, _s 1H(p_s)=0. The map s↦H(ps)s H(p_s) is strictly decreasing on [1/D,1)[1/D,1), and sups∈(1/D,1)H(p1/D)−H(ps)=logD. _s∈(1/D,1) \H(p_1/D)-H(p_s) \= D. Consequently, for every ε∈(0,logD) ∈(0, D), there exists sε∈(1/2,1)s_ ∈(1/2,1) such that H(p1/D)−H(psε)>logD−εH(p_1/D)-H(p_s_ )> D- , even though the conditional content composition and CA⟂C_A are unchanged between p1/Dp_1/D and psεp_s_ . Thus, variation in sink mass along a path with fixed conditional content composition can produce an apparent entropy collapse relative to the content-only diagnostic, of size arbitrarily close to the nonattained supremum logD D. This is a possibility result; it does not by itself assert that sink mass increases along any empirical training trajectory. Proof of Corollary 1. Since pt∈Dp_t ^D, one has 0<st<10<s_t<1. Therefore πt=((pt)1,…,(pt)D−1)1−st∈D−1 _t= ((p_t)_1,…,(p_t)_D-1 )1-s_t ^D-1 is well defined, and the displayed representation of ptp_t is unique. Using ∑i=1D−1πt,i=1 _i=1^D-1 _t,i=1, we obtain H(pt) H(p_t) =−stlogst−∑i=1D−1(1−st)πt,ilog((1−st)πt,i) =-s_t s_t- _i=1^D-1(1-s_t) _t,i ((1-s_t) _t,i ) =−stlogst−(1−st)log(1−st)−(1−st)∑i=1D−1πt,ilogπt,i =-s_t s_t-(1-s_t) (1-s_t)-(1-s_t) _i=1^D-1 _t,i _t,i =Hb(st)+(1−st)H(πt), =H_b(s_t)+(1-s_t)H( _t), which proves (20). This is a pointwise identity, so it requires no topology or ordering on T. If πt=π _t=π for every t, then pt′=πp_t =π, and the definition of content Aitchison concentration gives CA⟂(pt)=‖clrD−1(pt′)‖2=‖clrD−1(π)‖2.C_A (p_t)=\| _D-1(p_t )\|_2=\| _D-1(π)\|_2. Furthermore, fπ(s)=Hb(s)+(1−s)H(π)f_π(s)=H_b(s)+(1-s)H(π), and hence fπ′(s)=log1−s−H(π),fπ′(s)=−1s(1−s)<0,0<s<1,f_π (s)= 1-ss-H(π), f_π (s)=- 1s(1-s)<0, 0<s<1, so fπf_π is strictly concave. Because s↦log((1−s)/s)s ((1-s)/s) is strictly decreasing on (0,1)(0,1), fπ′(s)=0⇔1−s=exp(H(π))⇔s=11+exp(H(π))=θπ.f_π (s)=0 1-ss= (H(π)) s= 11+ (H(π))= _π. Consequently, fπ′(s)>0,0<s<θπ,fπ′(s)=0,s=θπ,fπ′(s)<0,θπ<s<1. casesf_π (s)>0,&0<s< _π,\\ f_π (s)=0,&s= _π,\\ f_π (s)<0,& _π<s<1. cases This proves the asserted strict monotonicity and unique maximizer. Since D−1≥2D-1≥ 2 and every coordinate of π is strictly positive, H(π)>0H(π)>0, so θπ<1/2 _π<1/2. Therefore, if 12≤st<su<1 12≤ s_t<s_u<1, then both sts_t and sus_u lie in the strictly decreasing region of fπf_π, giving H(pu)=fπ(su)<fπ(st)=H(pt)H(p_u)=f_π(s_u)<f_π(s_t)=H(p_t). The equality of the two content log-ratio dispersions was established above. Finally, H(uD−1)=log(D−1)H(u_D-1)= (D-1) and clrD−1(uD−1)=0 _D-1(u_D-1)=0, and ps′=uD−1p_s =u_D-1. Therefore CA⟂(ps)=0C_A (p_s)=0 for every s∈(0,1)s∈(0,1). The corresponding threshold is θuD−1=11+exp(log(D−1))=1D. _u_D-1= 11+ ( (D-1))= 1D. The preceding monotonicity result shows that s↦H(ps)s H(p_s) is strictly decreasing on [1/D,1)[1/D,1). At s=1/Ds=1/D, every coordinate of psp_s equals 1/D1/D, so H(p1/D)=logDH(p_1/D)= D. Moreover, H(ps)=Hb(s)+(1−s)log(D−1)⟶0as s↑1.H(p_s)=H_b(s)+(1-s) (D-1) 0 s 1. Since H(ps)>0H(p_s)>0 for every s∈(0,1)s∈(0,1), it follows that H(p1/D)−H(ps)<logDH(p_1/D)-H(p_s)< D for every s∈(1/D,1)s∈(1/D,1). On the other hand, H(p1/D)−H(ps)⟶logDH(p_1/D)-H(p_s) D as s↑1s 1. Therefore sups∈(1/D,1)H(p1/D)−H(ps)=logD. _s∈(1/D,1) \H(p_1/D)-H(p_s) \= D. The supremum is not attained because s=1s=1 is excluded and H(ps)>0H(p_s)>0 for every admissible s. Given ε∈(0,logD) ∈(0, D), choose sε∈(1/2,1)s_ ∈(1/2,1), sufficiently close to 11, such that H(psε)<εH(p_s_ )< . Then H(p1/D)−H(psε)=logD−H(psε)>logD−ε,H(p_1/D)-H(p_s_ )= D-H(p_s_ )> D- , as claimed. ∎ Remark 1 (CA⟂C_A orders by dispersion, not majorization). CA⟂C_A is not Schur-convex and is not an entropy substitute. For p=(0.800,0.199,0.001)p=(0.800,0.199,0.001) and q=(0.780,0.219,0.001)q=(0.780,0.219,0.001), p majorizes q and H(p)≈0.507<0.533≈H(q)H(p)≈ 0.507<0.533≈ H(q), yet ∥clr(p)∥2≈4.988<5.000≈∥clr(q)∥2 (p) _2≈ 4.988<5.000≈ (q) _2: near-zero coordinates dominate the log-ratio norm. Accordingly the paper uses CA⟂C_A as a dispersion statistic of the content log-ratios and routes concentration and collapse claims through the exact decomposition H(p)=Hb(s)+(1−s)H(π)H(p)=H_b(s)+(1-s)H(π), whose content term H(π)H(π) carries the entropy order. Lemma 2 (Quotient characterization of dA⟂d_A ). Define p∼qp q iff R0p=R0qR_0p=R_0q (equal content subcompositions). Then dA⟂d_A is the quotient metric induced by dAd_A on /∼S/ : dA⟂(p,q)=minp∗∼p,q∗∼qdA(p∗,q∗)d_A (p,q)= _p^* p,\;q^* qd_A(p^*,q^*), and Theorem 2 applied on the content simplex characterizes dA⟂d_A as the unique metric on the quotient satisfying A1–A4 for content transformations. Proof. By Lemma 1, dA(p∗,q∗)2=dA⟂(p,q)2+(b(p∗)−b(q∗))2d_A(p^*,q^*)^2=d_A (p,q)^2+ (b(p^*)-b(q^*) )^2 for any representatives, since dA⟂d_A depends only on the content classes. The balance b can be set freely while holding the content fixed (vary the sink share), so the minimum over representatives zeroes the second term and equals dA⟂(p,q)d_A (p,q); it is attained. The quotient space with this metric is isometric to the Aitchison geometry of the content simplex, on which Theorem 2 applies verbatim. ∎ Appendix E Experimental details Table 5: Measured taxonomy stability across sink conventions (the dA⟂d_A column is omitted: its partitions are identical across conventions by construction, Proposition 3(i), with ARI =1=1 verified in code) (mean ARI over K∈4,6,8K∈\4,6,8\, average linkage; reanalyze_clustering.py). The dA⟂d_A column is exact by Proposition 3(i). Model 1−cos1- JS Euclid. BERT-base 0.0240.024 0.0660.066 0.0300.030 GPT-2 (124M) −0.014-0.014 0.0220.022 −0.061-0.061 Llama-3.2-1B −0.030-0.030 −0.082-0.082 −0.086-0.086 ViT-B/16 evaluation uses 2,0482,048 COCO val2017 images with a label-free top-1 agreement metric against the unpruned model; labeled ImageNet-val substitutes directly in the released script via IMAGENET_DIR. E.1 Are attention rows compositions? Tail mass, rival geometries, and the content channel A fair objection to the compositional stance is causal: the row’s downstream effect is the value mixture ∑iaivi _ia_iv_i, so absolute mass matters and a coordinate at 10−1710^-17 moves nothing, while log-ratios weight it. Three answers. First, scope: our claims concern the row as an analysis object, which is what the literature computes cosine, JS, and entropy on; the sink balance b(p)b(p) retains the absolute sink share, and functional relevance is tested by the pruning experiments of §5, not assumed. Second, a direct audit of the estimator. For each model we decompose the squared content distance between seed-0 head signatures by coordinate magnitude: coordinate pairs whose smaller entry is below 10−610^-6 carry at most 4.6%4.6\% of squared dA⟂d_A (GPT-2) and below 0.7%0.7\% on every other model, and essentially 0%0\% below 10−910^-9 (Table 6). No coordinate has median signature value below 10−610^-6 in any model, so no tail-amalgamation set exists at that threshold and the shares above bound the maximal tail influence; sub-threshold values are shared across heads, which is why their log-ratio differences nearly cancel. Third, rival geometries measured rather than dismissed: Hellinger and Fisher–Rao closer-head verdicts flip between sink conventions at rates of 0.180.18 to 0.440.44, the same range as cosine, JS, and Euclidean distance. The same signatures address the central methodological question: how much of the observed stability is conditioning out the sink, and how much is Aitchison geometry? Within the content channel, JS on sink-dropped rows and dA⟂d_A agree strongly on coarse ranking (Spearman ρ≥0.96ρ≥ 0.96) but disagree on fine structure: nearest-neighbor identities differ for 1313 to 27%27\% of heads, and the induced taxonomies differ with cross-ARI as low as 0.330.33 on Llama-3.2-1B (Table 6). Conditioning out the sink is therefore the dominant stabilizer. The reversal phenomenon is also not an artifact of feeding classical metrics Aitchison-mean signatures: recomputed on arithmetic-mean signatures (their native aggregation, released alongside), classical reversal rates remain 0.180.18–0.450.45, within 0.150.15 of the Aitchison-mean rates on every model (Table 7). The revised protocol ran the functional comparison, and we report it either way it fell: under the global protocol dropped-row JS matches dA⟂d_A on Llama-3.2-1B and beats it on Llama-3.2-3B; under the same-layer keep-one protocol dA⟂d_A is best on every strong-sink model while dropped-row JS fails catastrophically on Llama-3.2-1B (Table 3). Two equivalences simplify the rival columns: Hellinger and Fisher–Rao are strictly monotone in the Bhattacharyya coefficient, so all of their ranking verdicts, and hence reversal rates, coincide exactly; and as pruning criteria, Hellinger-on-content selects the identical nearest neighbors and prune sets as dropped-row JS on every model, because for nearby distributions all f-divergences share the same local quadratic form while the Aitchison metric does not. Table 6: Estimator and rival-geometry audit (seed-0 signatures). Hell./FR == closer-head reversal rate between sink conventions under Hellinger and Fisher–Rao. Share == fraction of squared dA⟂d_A carried by coordinate pairs with smaller entry below 10−610^-6. Last three columns compare dropped-row JS with dA⟂d_A on the content channel: Spearman correlation of pairwise distances, nearest-neighbor agreement, and mean cross-ARI of the induced taxonomies over K∈4,6,8K∈\4,6,8\. Model Hell. rev. FR rev. share <10−6<10^-6 ρ N agr. cross-ARI GPT-2 (124M) 0.350.35 0.340.34 0.0460.046 0.9570.957 0.730.73 0.400.40 Llama-3.2-1B 0.430.43 0.440.44 0.0030.003 0.9780.978 0.810.81 0.330.33 Qwen2.5-1.5B 0.430.43 0.420.42 0.0000.000 0.9930.993 0.860.86 0.630.63 Llama-3.2-3B 0.440.44 0.440.44 0.0000.000 0.9710.971 0.800.80 0.600.60 ViT-B/16 0.190.19 0.180.18 0.0000.000 0.9980.998 0.850.85 0.770.77 Pythia-70M 0.300.30 0.300.30 0.0000.000 0.9700.970 0.810.81 0.870.87 Pythia-160M 0.430.43 0.430.43 0.0070.007 0.9580.958 0.740.74 0.540.54 Pythia-410M 0.420.42 0.420.42 0.0000.000 0.9910.991 0.840.84 0.530.53 Pythia-1B 0.410.41 0.400.40 0.0040.004 0.9960.996 0.870.87 0.600.60 Pythia-1.4B 0.400.40 0.400.40 0.0000.000 0.9930.993 0.840.84 0.830.83 Table 7: Aggregation robustness: with-sink versus sink-dropped reversal rates (seed 0) computed on arithmetic-mean signatures (the classical metrics’ native aggregation) and on Aitchison-mean signatures. arithmetic mean Aitchison mean Model cosine JS Euclid. cosine JS Euclid. GPT-2 (124M) 0.330.33 0.300.30 0.350.35 0.360.36 0.350.35 0.380.38 Llama-3.2-1B 0.400.40 0.370.37 0.410.41 0.460.46 0.440.44 0.470.47 Qwen2.5-1.5B 0.400.40 0.370.37 0.410.41 0.440.44 0.430.43 0.450.45 Llama-3.2-3B 0.440.44 0.410.41 0.450.45 0.450.45 0.440.44 0.470.47 ViT-B/16 0.210.21 0.180.18 0.220.22 0.220.22 0.180.18 0.230.23 Pythia-70M 0.320.32 0.280.28 0.310.31 0.330.33 0.310.31 0.310.31 Pythia-160M 0.340.34 0.290.29 0.320.32 0.430.43 0.440.44 0.440.44 Pythia-410M 0.400.40 0.380.38 0.400.40 0.440.44 0.430.43 0.440.44 Pythia-1B 0.350.35 0.340.34 0.360.36 0.410.41 0.400.40 0.420.42 Pythia-1.4B 0.430.43 0.400.40 0.430.43 0.420.42 0.400.40 0.420.42 E.2 Convention stakes measured: temperature and the sink set Temperature. Axiom A3 (powering) is the softmax temperature: rescaling the content logits by τ maps each content row π to (πτ)C(π^τ). By Proposition 2, dA⟂d_A is exactly equivariant (dA⟂→τdA⟂d_A →τ\,d_A ), so every ranking verdict, nearest neighbor, and prune set is invariant; we verify this to the bit on all ten models. Dropped-row JS is not: over τ∈0.5,0.7,1.5,2τ∈\0.5,0.7,1.5,2\ its closer-head verdicts flip by up to 6.6%6.6\% and its top-20%20\% prune set retains as little as 62%62\% of its τ=1τ=1 membership (Table 8). Effective logit scale is not observable to an analyst comparing models or pipelines, so a criterion whose selections depend on it answers a scale-relative question; dA⟂d_A does not. The sink set. Dropping the sink requires deciding which columns are the sink; first token and first four tokens are both used in the literature. This is a second convention layer: between the two definitions, dropped-row JS flips 11 to 21%21\% of verdicts and dA⟂d_A flips 11 to 18%18\%, with dA⟂d_A uniformly less sensitive on every model (Table 8). Both changes are changes of estimand; what distinguishes the Aitchison side is that the definitions are exactly related: the content distances are nested orthogonal projections, and we verify the implied dominance (dA⟂d_A on the smaller content set never exceeds dA⟂d_A on the larger) with no violations across all head pairs of all ten models. No analogous relation exists for JS. Which regime? A dispersion predictor. Mean sink mass does not predict which pruning regime a model is in (GPT-2 and Qwen2.5 share s=0.68s=0.68 with opposite verdicts). The cross-head dispersion of the sink share does, for the nine language models: the content-regime LMs have coefficient of variation at most 0.360.36 (Llama-3.2-1B 0.110.11, Llama-3.2-3B 0.080.08, Qwen2.5 0.360.36) and the JS-regime LMs at least 0.420.42 (GPT-2 0.420.42, Pythias 0.430.43–1.031.03). A near-uniform sink carries no between-head information, so mixing it into distances only adds noise; a heterogeneous sink marks genuine head classes. ViT, where the criteria tie within noise, does not fit the LM pattern. A complementary stability probe: across the three resampling seeds, the top-20%20\% prune sets selected by dA⟂d_A -L1 overlap with Jaccard 0.360.36–0.770.77 while the JS-L1 sets are essentially frozen (Jaccard 0.920.92–1.001.00 on LMs), even though the dA⟂d_A -L1 outcomes are the stable ones on strong-sink models. Each criterion is stable along a different axis: dA⟂d_A under the analyst’s conventions (temperature, sink set), JS under data resampling; on GPT-2 the dA⟂d_A -L1 set instability coincides with its one catastrophic cell. This makes the weighted family dλ2=(dA⟂)2+λ(Δb)2d_λ^2=(d_A )^2+λ\,( b)^2 natural, with λ growing with the b-channel’s informativeness; fitting λ from held-out functional evidence is left to future work, and we note that the with-sink JS criterion itself lies outside this family. Held-out test, specified before running. We freeze the rule and the model list here, prior to any evaluation: for opt-125m, distilgpt2, and SmolLM2-135M we will compute the cross-head CV of the signature sink share and predict, with the thresholds above, that CV≤0.36CV≤ 0.36 places the model in the content regime (dA⟂d_A -L1 best or tied-best among the six criteria, with at least one classical criterion exceeding four times base perplexity) and CV≥0.42CV≥ 0.42 in the JS regime (with-sink JS(-L1) best); for 0.36<CV<0.420.36<CV<0.42 the predictor abstains. The released prediction script logs its output before any pruning evaluation exists, and the outcomes will be reported whichever way they fall. Outcomes. The predictions were logged before any evaluation (timestamps in the released heldout_predictions.json); measured CVs: OPT-125M 0.280.28, DistilGPT2 0.510.51, SmolLM2-135M 0.420.42. DistilGPT2 (JS regime predicted): confirmed. With-sink JS wins (105.3±0.8105.3±0.8 on base 67.467.4) and dropped-row JS is catastrophic (1161±241161±24, 17×17× base). SmolLM2-135M: the predictor abstained at the boundary; the outcome fell on the JS side (JS-L1 best at 83.5±0.283.5±0.2 on base 24.824.8; every content-conditioned criterion 99–54×54× base). Its sink is layer-bimodal, 0.010.01–0.150.15 in the first twelve layers and 0.680.68–0.920.92 thereafter, which is what places its cross-head CV at the threshold. OPT-125M (content regime predicted): the prediction failed. Global content criteria do beat with-sink JS (59.259.2/59.359.3 versus 63.363.3 on base 46.746.7), but JS-L1 wins outright (57.1±0.557.1±0.5) while dA⟂d_A -L1 sits mid-pack with high seed variance (68.5±7.768.5±7.7), and no criterion is catastrophic at 20%20\% (worst 2.1×2.1×; dropped-JS-L1 reaches 6.9×6.9× only at 30%30\%). The scorecard on non-abstentions is one confirmation and one failure: the dispersion rule as frozen does not transfer, and OPT exhibits a third behavior the nine-model map lacks, a uniformly high sink with a benign pruning landscape. We accordingly present the rule as descriptive of the training fleet, not as a validated decision procedure. A second-family checkpoint sweep (e.g. OLMo) is excluded by compute budget, so the checkpoint analysis of Fig. 5b currently rests on one model family; we state this as a limitation. Exact decomposition of the checkpoint collapse. With Δ the change from the first checkpoint to the entropy minimum, the identity gives the signed split ΔH=ΔHb(s)+Δ[(1−s)H(π)] H= H_b(s)+ [(1-s)H(π)] exactly (residual 00 to machine precision). Across 70M/160M/410M/1B/1.4B: ΔH=−0.78/−1.55/−2.11/−1.58/−2.01 H=-0.78/-1.55/-2.11/-1.58/-2.01, ΔHb=+0.18/+0.26/+0.23/+0.29/+0.25 H_b=+0.18/+0.26/+0.23/+0.29/+0.25, and Δ[(1−s)H(π)]=−0.96/−1.81/−2.34/−1.88/−2.26 [(1-s)H(π)]=-0.96/-1.81/-2.34/-1.88/-2.26 nats. The binary term rises as s approaches 1/21/2 while the weighted content term carries the entire fall, and within that term the conditional entropy H(π)H(π) itself moves little; the main-text share 1−ΔH(π)/ΔH1- H(π)/ H isolates exactly this, the fraction of the drop not attributable to a drop in H(π)H(π). Table 8: Convention stakes (seed-0 signatures). Temperature: worst-case dropped-JS verdict flip rate and top-20%20\% prune-set retention over τ∈0.5,0.7,1.5,2τ∈\0.5,0.7,1.5,2\ (the dA⟂d_A values are exactly 00 and 11 on every model, verified to the bit, and are omitted). Sink set: closer-head verdict flip rate between the first-token and first-four-token sink definitions. temperature (JS′) sink set flips Model flip (max) set retention (min) JS′ dA⟂d_A GPT-2 (124M) 0.0660.066 0.690.69 0.0640.064 0.0390.039 Llama-3.2-1B 0.0460.046 0.870.87 0.1540.154 0.1280.128 Qwen2.5-1.5B 0.0270.027 0.960.96 0.0860.086 0.0750.075 Llama-3.2-3B 0.0400.040 0.900.90 0.2050.205 0.1760.176 ViT-B/16 0.0170.017 0.930.93 0.0100.010 0.0120.012 Pythia-70M 0.0560.056 0.700.70 0.1040.104 0.0730.073 Pythia-160M 0.0660.066 0.620.62 0.1110.111 0.0710.071 Pythia-410M 0.0330.033 0.920.92 0.1090.109 0.0890.089 Pythia-1B 0.0230.023 0.920.92 0.1120.112 0.1030.103 Pythia-1.4B 0.0270.027 0.900.90 0.0960.096 0.0850.085 E.3 Full-support Clark reanalysis and window robustness Clark et al. on the full support. We rebuild 4’s head-clustering pipeline with no window truncation: 512512 sequences of exactly 128128 WikiText tokens ([CLS] ++ 126126 wordpieces ++ [SEP]), so every row shares one support and the sink set is \CLS, SEP\; per-head Aitchison-mean signatures; JS distances and average-linkage clustering as in the original, plus cosine and dA⟂d_A . Of 144144 heads, 6868 place majority mass on SEP and 1212 on CLS. Between the keep and drop conventions the JS clusterings agree at chance (ARI 0.010.01–0.050.05 over K∈4,5,6,8K∈\4,5,6,8\; cosine 0.000.00–0.060.06). The SEP-head block is recovered by the with-sink JS clustering at Jaccard 0.870.87–0.920.92 and collapses to 0.460.46–0.490.49 after dropping; the content clusterings (dA⟂d_A , convention-invariant by construction) recover it only at 0.470.47–0.500.50. The structure the pipeline is best known for is therefore a property of the sink channel, visible under exactly one convention. Table 9: Clark-style clustering on the full 128128-token support (BERT-base; sink set \CLS, SEP\). Cross == between-convention ARI; SEP Jac. == Jaccard of the SEP-head set with its best-matching cluster. JS cosine dA⟂d_A K cross SEP Jac. (keep) SEP Jac. (drop) cross cross SEP Jac. 44 0.050.05 0.920.92 0.490.49 0.060.06 11 0.470.47 55 0.040.04 0.870.87 0.490.49 0.010.01 11 0.480.48 66 0.010.01 0.870.87 0.460.46 0.010.01 11 0.500.50 88 0.030.03 0.870.87 0.480.48 0.000.00 11 0.500.50 Window robustness and mass coverage. Extracting at W∈32,64,128W∈\32,64,128\ on GPT-2, Llama-3.2-1B, and Pythia-410M: the retained mass of the window before reclosure is 0.490.49–0.610.61 (GPT-2), 0.600.60–0.710.71 (Llama), 0.480.48–0.670.67 (Pythia-410M); JS reversal rates move by at most 2.12.1 points across the sweep (0.33/0.34/0.350.33/0.34/0.35, 0.39/0.40/0.400.39/0.40/0.40, 0.40/0.40/0.410.40/0.40/0.41 respectively), and dA⟂d_A pairwise distances between consecutive windows correlate at Spearman ρ=0.87ρ=0.87–0.980.98. The fixed-support choice shifts the estimand (each W conditions on a different retained set) but not the phenomenon. E.4 Synthetic experiments (Section 5.1) All synthetic results use numpy with seed 77 and run in seconds on CPU (experiments/run_synthetic.py); the exact witnesses and the machine-precision checks of Lemma 1 and Proposition 2 are in experiments/exact_examples.py. Head-signature sampler. A row is p=(s,(1−s)π)p=(s,(1-s)π) with sink mass s∼(s¯,0.10)s ( s,0.10) clipped to [0.02,0.98][0.02,0.98] and content π∼Dirichlet(αD−1)π ( 1_D-1), D=64D=64. The peaked setting α=0.1α=0.1 mimics concentrated attention; α=1.0α=1.0 (dashed in Figure 3) is the diffuse control. Ranking-disagreement Monte Carlo (Figure 3). For each s¯ s on a grid from 0.100.10 to 0.900.90, sample 20,00020,000 independent triples (p,q,r)(p,q,r), and record whether the verdict of “is q or r closer to p?” differs between the with-sink and sink-dropped pipelines, separately for cosine, JS, Euclidean distance, and dAd_A (total) versus dA⟂d_A . Headline rates at s¯=0.5/0.7/0.9 s=0.5/0.7/0.9 (peaked content): cosine 36.4/39.0/41.1%36.4/39.0/41.1\%, JS 23.4/31.0/39.7%23.4/31.0/39.7\%, Euclidean 27.3/36.8/44.6%27.3/36.8/44.6\%; dA⟂d_A is 0%0\% by construction, and total-dAd_A-versus-dA⟂d_A is 0.1%0.1\% at every grid point (see the transparency note in Appendix A). Collapse scenarios (Figure 2). Twelve “layers,” 4,0004,000 rows per layer, s-noise 0.030.03. Scenario (a): sℓ=0.05+0.80ς(1.1(ℓ−5))s_ =0.05+0.80\, (1.1( -5) ) with ς the logistic function and content log-ratio dispersion fixed (αℓ≡0.15 _ ≡ 0.15); measured H falls 2.71→0.822.71→ 0.82 nats while CA⟂C_A moves 48.13→47.9848.13→ 47.98. Scenario (b): sℓs_ linear 0.70→0.050.70→ 0.05 while αℓ _ decays geometrically 1.0→0.021.0→ 0.02; H drifts 1.73→1.271.73→ 1.27 while CA⟂C_A rises 10.04→70.4310.04→ 70.43. Reversal wedge (Figure 6). Fix (p,r)(p,r) of the witness; sample 20,00020,000 points q∼Dirichlet(1,1,1)q (1,1,1) with all parts >0.004>0.004; the cosine verdict differs between conventions on 77.9%77.9\% of them. E.5 Protocol for pretrained models (Section 3) Implemented in experiments/run_real_models.py (PyTorch + transformers + datasets); ten models, three resampling seeds each, on two RTX 4090 GPUs; the measurement passes take minutes per model and the pruning evaluations a few hours in total. Reported standard deviations are population estimates (ddof 00) over the three seeds throughout. Models and data. GPT-2 (gpt2, 124M, 12 layers × 12 heads, fp32); Llama-3.2-1B (meta-llama/Llama-3.2-1B, 16 layers × 32 heads, GQA, fp16); ViT-B/16 (google/vit-base-patch16-224, 12 layers × 12 heads, fp32). LM data: 2,0002,000 sequences of length ≥192≥ 192 tokens from the WikiText-103 validation split; ViT data: 2,0482,048 COCO val2017 images (public, unlabeled; the script accepts any image folder, and the ViT pruning metric is top-1 agreement with the unpruned model, so labels are not required, labeled ImageNet-val substitutes directly). Row extraction on a fixed common support. Attention probabilities are captured from the softmax output (via output_attentions=True or forward hooks). For LMs, for every query position t≥wt≥ w we keep the attention over the first W=64W=64 keys only and re-close on that support. This makes the support identical across all query positions, sequences, and heads, so the main measurements require no zero imputation, while retaining the sink column, key 00 (the ⟨bos⟩ /first token; a flag groups the first 44 keys as a sink block instead, following the StreamingLLM observation, in which case the sink balance is the corresponding SBP balance). For ViT-B/16 the support is all 197197 tokens (no masking) and the sink group is the CLS column, optionally augmented with the highest-norm register-like outlier tokens [5]. Aggregation. Each head’s signature is the Aitchison mean of its collected rows (Definition 1(a)); residual numerical zeros (none on the fixed support, possible under the optional variable-support ablation) are multiplicatively replaced at ε=10−6 =10^-6 before taking logs, with the sweep ε∈10−5,10−6,10−7 ∈\10^-5,10^-6,10^-7\ reported. Measurements. (1) Reversal rates: sample 50,00050,000 head triples uniformly (within and across layers); report the fraction whose closer-head verdict flips between the with-sink and sink-dropped pipelines, for cosine, JS, Euclidean, and dA⟂d_A (identically 00; reported as a check). (2) Collapse curves: per layer, mean row entropy H (with sink) versus mean CA⟂C_A ; report the fraction of adjacent-layer steps on which the two disagree in sign. (3) Redundancy pruning at matched sparsity: define a head’s redundancy as its distance to the nearest other head; prune the m∈10,20,30%m∈\10,20,30\\% most redundant heads under JS, under the content distance dA⟂d_A , and under the full dAd_A including the sink balance (matched counts), via head_mask where supported and output-zeroing hooks otherwise; evaluate perplexity on 100100k held-out WikiText tokens (LMs) and top-1 agreement with the unpruned model on the 2,0482,048 images (ViT). (4) Sink statistics: mean sink mass per model and per layer, for context against 3, 10. Runtime. On two RTX 4090s the full protocol (all models, all measurements) completes in roughly 1515–2525 minutes wall clock with the released parallel launcher; a single comparable GPU takes about one hour end to end. E.6 Protocol for the prospectively specified tests (Section 5.3) (1) Taxonomies (reanalyze_clustering.py). Objects are per-head Aitchison-mean signatures on the fixed 6464-key support, collected exactly as in the main protocol (decoder models reuse the saved signatures of the §3 runs; for BERT-base we collect encoder rows for queries t∈[0,64)t∈[0,64) renormalized to the first 6464 keys over 1,0001,000 WikiText-103 sequences of length 128128, sink == [CLS]). For each dissimilarity δ∈1−cos,JS,Euclid.,dA⟂,dAδ∈\1- ,JS,Euclid.,d_A ,d_A\ we form the full pairwise matrix under both conventions and run agglomerative clustering (average, complete, and single linkage; NumPy implementation released) cut at K∈4,6,8K∈\4,6,8\. Stability is the Adjusted Rand Index between the two conventions’ partitions; Table 5 reports the mean over K under average linkage, and the released script prints all linkage × K cells. The dA⟂d_A matrices are asserted equal entrywise at runtime (Proposition 3(i)). (2) Scale (scale_trend.py after PYTHIA=1 run_all.sh). Adds EleutherAI Pythia 70M,160M,410M,1B,1.4B\70M,160M,410M,1B,1.4B\ under the unchanged main protocol and plots measured sink mass versus parameters and reversal rate versus sink mass over the pre-computed calibration curves of Figure 3. (3) Training curves (checkpoints_pythia.py). Pythia-160M at revisions step512, 1000, 2000, 4000, 8000, 16000, 32000, 64000, 128000, 143000; 256256 held-out sequences per checkpoint, queries t∈[64,128)t∈[64,128) on the fixed 6464-key support; we report mean row entropy H, mean content log-ratio dispersion CA⟂C_A , and mean sink mass per checkpoint, and shade checkpoint transitions where sign(ΔH)≠sign(−ΔCA⟂)sign( H) (- C_A ) (roughly 3030–4545 minutes on one consumer GPU). E.7 Measured results: full pruning sweep and ε sensitivity Table 10 reports the complete pruning sweep behind Table 3. Two observations beyond the main text: on Llama-3.2-1B, JS-guided pruning is already catastrophic at 10%10\% sparsity (15.3→358.015.3→ 358.0), while dA⟂d_A -guided pruning remains within a factor 5.25.2 of baseline even at 30%30\% (79.879.8); and the full-dAd_A criterion is the lowest at 10%10\% on Llama (18.218.2) but degrades sharply at higher sparsity there, consistent with sink-similar heads being redundant in small numbers but not in bulk. All reversal rates in Table 3 are bit-identical across ε∈10−5,10−6,10−7 ∈\10^-5,10^-6,10^-7\, as expected: the fixed common support makes multiplicative replacement inactive for these measurements, so ε enters only through numerical clipping. Figure 4 is regenerated from the released per-model JSONs by experiments/make_fig4_v2.py. Table 10: Full pruning sweep at 10/20/30%10/20/30\% sparsity (mean± over three seeds). LMs: perplexity (lower better); ViT: top-1 agreement (higher better; base =1=1 by definition). All rates in Table 3 are bit-identical across ε∈10−5,10−6,10−7 ∈\10^-5,10^-6,10^-7\, as the fixed-support protocol predicts. Model sparsity base JS dA⟂d_A dAd_A (total) GPT-2 (124M) 10% 43.5±0.443.5 ± 0.4 50.5±0.550.5 ± 0.5 88.9±0.988.9 ± 0.9 61.5±0.761.5 ± 0.7 20% 74.5±0.774.5 ± 0.7 104.7±0.7104.7 ± 0.7 104.8±0.8104.8 ± 0.8 30% 85.2±1.085.2 ± 1.0 122.3±1.1122.3 ± 1.1 117.4±1.6117.4 ± 1.6 Llama-3.2-1B 10% 15.3±0.115.3 ± 0.1 358.0±31.1358.0 ± 31.1 45.6±2.945.6 ± 2.9 18.2±0.418.2 ± 0.4 20% 2094.1±139.72094.1 ± 139.7 63.6±4.163.6 ± 4.1 2480.3±210.52480.3 ± 210.5 30% 2961.7±150.02961.7 ± 150.0 79.8±7.779.8 ± 7.7 51742.6±6382.951742.6 ± 6382.9 Qwen2.5-1.5B 10% 14.8±0.114.8 ± 0.1 56.4±0.256.4 ± 0.2 19.6±0.119.6 ± 0.1 22.6±0.122.6 ± 0.1 20% 111.5±0.7111.5 ± 0.7 30.3±0.730.3 ± 0.7 32.3±0.232.3 ± 0.2 30% 162.1±0.8162.1 ± 0.8 49.0±0.449.0 ± 0.4 46.7±0.346.7 ± 0.3 Llama-3.2-3B 10% 12.1±0.112.1 ± 0.1 49.3±0.249.3 ± 0.2 16.4±0.116.4 ± 0.1 14.4±0.114.4 ± 0.1 20% 310.8±3.8310.8 ± 3.8 40.2±1.640.2 ± 1.6 36.4±0.636.4 ± 0.6 30% 525.0±12.7525.0 ± 12.7 178.4±5.4178.4 ± 5.4 13636.1±1182.813636.1 ± 1182.8 ViT-B/16 10% 1.0001.000 0.652±0.0100.652 ± 0.010 0.679±0.0120.679 ± 0.012 0.647±0.0060.647 ± 0.006 20% 0.578±0.0030.578 ± 0.003 0.566±0.0130.566 ± 0.013 0.573±0.0050.573 ± 0.005 30% 0.407±0.0020.407 ± 0.002 0.466±0.0210.466 ± 0.021 0.425±0.0060.425 ± 0.006 Pythia-70M 10% 724.4±10.7724.4 ± 10.7 663.3±14.5663.3 ± 14.5 1006.0±16.91006.0 ± 16.9 1020.1±9.71020.1 ± 9.7 20% 661.7±16.5661.7 ± 16.5 1354.5±16.11354.5 ± 16.1 1484.9±11.31484.9 ± 11.3 30% 775.5±16.6775.5 ± 16.6 1683.9±19.41683.9 ± 19.4 2483.2±40.52483.2 ± 40.5 Pythia-160M 10% 131.4±0.7131.4 ± 0.7 129.6±1.3129.6 ± 1.3 220.9±6.3220.9 ± 6.3 212.7±1.0212.7 ± 1.0 20% 142.1±1.8142.1 ± 1.8 721.5±25.8721.5 ± 25.8 512.4±27.5512.4 ± 27.5 30% 150.2±2.4150.2 ± 2.4 884.7±30.0884.7 ± 30.0 1071.4±18.51071.4 ± 18.5 Pythia-410M 10% 32.4±0.132.4 ± 0.1 33.0±0.133.0 ± 0.1 91.7±1.991.7 ± 1.9 38.9±0.438.9 ± 0.4 20% 75.2±0.575.2 ± 0.5 194.8±5.0194.8 ± 5.0 101.6±4.4101.6 ± 4.4 30% 132.2±2.4132.2 ± 2.4 128.2±2.5128.2 ± 2.5 131.6±5.6131.6 ± 5.6 Pythia-1B 10% 21.8±0.121.8 ± 0.1 40.3±0.140.3 ± 0.1 75.7±3.475.7 ± 3.4 28.5±0.628.5 ± 0.6 20% 44.9±0.144.9 ± 0.1 63.0±1.963.0 ± 1.9 57.5±3.157.5 ± 3.1 30% 54.9±0.254.9 ± 0.2 61.2±0.961.2 ± 0.9 134.0±8.9134.0 ± 8.9 Pythia-1.4B 10% 20.4±0.120.4 ± 0.1 23.0±0.123.0 ± 0.1 76.7±2.476.7 ± 2.4 51.2±2.551.2 ± 2.5 20% 33.4±0.333.4 ± 0.3 377.8±8.8377.8 ± 8.8 142.8±11.3142.8 ± 11.3 30% 61.5±0.661.5 ± 0.6 1703.7±126.81703.7 ± 126.8 268.8±26.1268.8 ± 26.1 Table 11: Additional pruning baselines at 20%20\% sparsity (mean± over three resampling seeds; two random draws shown individually). Total dAd_A re-mixes the sink balance; JS′-L1 is dropped-row JS under the same-layer keep-one protocol; the 10/30%10/30\% values are in the released JSONs. Model total dAd_A JS′-L1 random A/B GPT-2 (124M) 104.8±0.8104.8 ± 0.8 268.0±3.1268.0 ± 3.1 314314/171171 Llama-3.2-1B 2480.3±210.52480.3 ± 210.5 2642.2±242.62642.2 ± 242.6 187187/296296 Qwen2.5-1.5B 32.3±0.232.3 ± 0.2 28.4±0.528.4 ± 0.5 3030/3535 Llama-3.2-3B 36.4±0.636.4 ± 0.6 24.1±0.524.1 ± 0.5 3939/309309 ViT-B/16 0.573±0.0050.573 ± 0.005 0.562±0.0110.562 ± 0.011 0.5200.520/0.5350.535 Pythia-70M 1484.9±11.31484.9 ± 11.3 2051.1±27.02051.1 ± 27.0 15561556/10851085 Pythia-160M 512.4±27.5512.4 ± 27.5 463.9±122.0463.9 ± 122.0 312312/282282 Pythia-410M 101.6±4.4101.6 ± 4.4 65.1±1.265.1 ± 1.2 6969/7171 Pythia-1B 57.5±3.157.5 ± 3.1 41.2±2.041.2 ± 2.0 8585/6969 Pythia-1.4B 142.8±11.3142.8 ± 11.3 43.5±5.243.5 ± 5.2 5454/4747