Paper deep dive
Hierarchical Grading in Large Language Models
T. Shaska
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/28/2026, 4:03:49 AM
Summary
The paper introduces Graded Large Language Models (GLLMs), an algebraic framework that applies a grading to the representation space of transformers to reduce sample complexity. By propagating weighted scalar actions through embeddings and attention, GLLMs align the model's geometry with the hierarchical structure of language data. The optimal grades are determined by a convex program based on offline estimable profiles of target energy and data variance, allowing the model to compile to a standard transformer with identical inference cost but improved training efficiency.
Entities (9)
Relation Signals (6)
Graded Large Language Models → extends → Transformer
confidence 95% · The construction extends the theory of graded neural networks and graded transformers to autoregressive language models
Graded Large Language Models → compilesto → Transformer
confidence 93% · Because the grading is absorbed into the learned parameters after training, every GLLM compiles to a standard transformer of identical architecture and inference complexity.
Graded Large Language Models → optimizes → Sample Complexity
confidence 92% · Grading creates no information: it converts a correct prior about the target into reduced sample complexity
Graded Large Language Models → utilizes → Geometric Invariant Theory
confidence 90% · The governing geometric picture is that of geometric invariant theory.
Optimal Grades → determinedby → Bhattacharyya Affinity
confidence 88% · The entire benefit of grading is therefore the Bhattacharyya affinity between where the target places its weight and where the data places its variance.
Multi-Level Graded Embeddings → partof → Graded Large Language Models
confidence 87% · Sec. 4 introduces Multi-Level Graded Embeddings (MLGE)... Sec. 5 extends the grading to the training objective
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce Graded Large Language Models (GLLMs), an algebraic framework that equips the representation space of a transformer with a grading and propagates the induced weighted scalar action through embeddings, self-attention, and the training objective. The construction extends the theory of graded neural networks and graded transformers to autoregressive language models while preserving expressive power, asymptotic computational complexity, and inference cost. The governing geometric picture is that of geometric invariant theory. The benefit of a grading is expressed by a Kempf--Ness functional on the grading torus; the grades that improve upon the uniform architecture form an open convex cone whose membership is decided by a Hilbert--Mumford-type criterion pairing a grade direction against two measurable profiles of the target and the data; the optimal grades are the coincidence point of two moment maps, given in closed form; and the ordinary transformer appears as a semistable isotropic point on the boundary of the cone: one member of a larger graded family rather than a distinguished optimum. Separately, for level-stratified targets we prove a minimax separation between the graded prior and its absence: over all estimators the risks of the graded and uniform target classes separate throughout an explicit window of sample sizes, by a factor that decays exponentially in the number of levels under geometric stratification. Both profiles are estimable offline, so the optimal grades solve a convex program certified before training begins. Because the grading is absorbed into the learned parameters after training, every GLLM compiles to a standard transformer of identical architecture and inference complexity.
Tags
Links
- Source: https://arxiv.org/abs/2607.22757v1
- Canonical: https://arxiv.org/abs/2607.22757v1
Trouble viewing inline? Open PDF directly →
Full Text
196,388 characters extracted from source content.
Expand or collapse full text
Hierarchical Grading in Large Language Models T. Shaska Department of Computer Science and Engineering, Oakland University, Rochester, MI 48309 shaska@oakland.edu Abstract. We introduce Graded Large Language Models (GLLMs), an algebraic framework that equips the representation space of a transformer with a grading and propagates the induced weighted scalar action through embeddings, self-attention, and the training objective. The construction extends the theory of graded neural networks and graded transformers to autoregressive language models while preserving expressive power, asymptotic computational complexity, and inference cost. The governing geometric picture is that of geometric invariant theory. The benefit of a grading is expressed by a Kempf–Ness functional on the grading torus; the grades that improve upon the uniform architecture form an open convex cone whose membership is decided by a Hilbert–Mumford-type criterion pairing a grade direction against two measurable profiles of the target and the data; the optimal grades are the coincidence point of two moment maps, given in closed form; and the ordinary transformer appears as a semistable isotropic point on the boundary of the cone: one member of a larger graded family rather than a distinguished optimum. Separately, for level-stratified targets we prove a minimax separation between the graded prior and its absence: over all estimators the risks of the graded and uniform target classes separate throughout an explicit window of sample sizes, by a factor that decays exponentially in the number of levels under geometric stratification. Both profiles are estimable offline, so the optimal grades solve a convex program certified before training begins. Because the grading is absorbed into the learned parameters after training, every GLLM compiles to a standard transformer of identical architecture and inference complexity. We conclude with the grade-selection procedure for domains without canonical gradings, instantiated for natural language, and a pre-registered validation program whose empirical results will be reported in a companion manuscript. Key words and phrases: graded vector spaces, graded transformers, large language models, hierarchical attention, sample complexity, minimax lower bounds, inductive bias 2020 Mathematics Subject Classification: 68T07, 68T50, 68Q32, 62C20, 16W50 1. Introduction Large Language Models (LLMs) have moved from statistical n-gram estimators to transformer systems of O(1011)O(10^11) parameters trained on O(1013)O(10^13) tokens, and they now generate relatively fluent text, perform multi-step tasks, and support code synthesis and scientific work; see [radford2019language] among many other sources. This progress came almost entirely from scaling; see [kaplan2020scaling]. The architecture behind it is uniform: every token occupies the same space, every coordinate of that space carries the same weight, and every attention head computes over a flat, isotropic geometry in which no direction is special. Language, in contrast, is hierarchical. Subword units compose into words, words into phrases and clauses, clauses into sentences, sentences into discourse, and discourse into pragmatic inference. A model with no representation of this hierarchy must recover it from data. The cost of that recovery is the question that Sec. 2 is organised around. The answer is given mainly in two statements. The first is the classical conjecture that an architecture without a structural prior must discriminate among the Ω(nL) (n^L) parse trees a depth-L grammar admits over inputs of length n, at a sample cost exponential in L (Conj. 1). This conjecture motivated the graded program, but no result below depends on it. The second is weaker, can be measured on an annotated corpus without training a model, and is the hypothesis the framework rests on: in language, the features that decide hierarchical targets are not the features that carry the corpus variance (Assumption 1). This work introduces Graded Large Language Models (GLLMs), which supply the missing prior through the algebra of the representation space rather than through new architectural components. The construction builds on a line of work on graded vector spaces in machine learning: artificial neural networks over graded spaces [2024-02], graded neural networks with graded neurons, activations, and losses [sh-89], and the Graded Transformer framework comprising the Linearly Graded Transformer (LGT) and the Exponentially Graded Transformer (EGT) [sh-95]. The object underlying all of them is a graded vector space ℝnR^n_q: a copy of ℝnR^n equipped with a grade tuple =(q0,…,qn−1)q=(q_0,…,q_n-1) and the induced scalar action λ⋆x=(λq0x0,…,λqn−1xn−1),λ>1.λ x=(λ^q_0x_0,…,λ^q_n-1x_n-1), λ>1. A grade tuple states which coordinates matter and by how much. Assigning grades to the dimensions of an embedding space and propagating the action through attention, embeddings, and the loss amplifies the subspaces that carry syntactic heads, semantic nuclei, and pragmatic signal, and reduces the weight of the rest. Nothing is removed: the grading changes the metric of the representation space and leaves the computation over it intact, so expressive power, parallelisability, and autoregressive decoding are all preserved. What changes is the geometry the model searches over. The construction has three parts and a cost analysis. Sec. 3 develops exponentially graded multi-head self-attention (EG-MHSA) within causal decoder-only stacks and establishes three facts that locate the advantage. Grading leaves the asymptotic cost of attention unchanged at O(n2d+nd2)O(n^2d+nd^2) per layer (Prop. 4). The grading matrix is absorbed into the projection weights by an invertible substitution (Rem. 1), so a trained GLLM compiles to a standard transformer of identical architecture and carries its prior at zero deployment cost (Cor. 1); no variant in Tab. 1 has this property, since recurrence and sparsity encode their bias in the computation and pay for it at every forward pass. At a fixed norm budget the graded constraint set is an ellipsoid whose axes are aligned with the grades and which strictly contains the uniform ball (Prop. 5), so the advantage cannot come from representability: a larger class at a common budget hurts generalization rather than helping it. Together these facts place the advantage in the norm required to represent a given target, and hence in the data required to learn it. Sec. 4 introduces Multi-Level Graded Embeddings (MLGE), decomposing ℝd=⨁l=0L−1VlR^d= _l=0^L-1V_l into level-specific subspaces, and carries out that analysis. Sec. 5 extends the grading to the training objective and characterizes exactly when it remains consistent for the corpus conditional (Prop. 9). Sec. 8 states the program that will test the resulting predictions, and Sec. 6 assembles the results into the framework’s main purpose: a procedure that selects, certifies, and prices grades in domains that supply none. The analysis yields the paper’s central quantity. Write α for the profile of the target’s energy across the graded basis and τ for the profile of the data’s variance (Def. 2). Then the Rademacher-based sample complexity of the graded class stands to that of the uniform class in the ratio Λ(g) (g) =(∑j=1dαjgj−2)(∑j=1dτjgj2), = ( _j=1^d _j\,g_j^-2 ) ( _j=1^d _j\,g_j^2 ), ming∈ℝ>0dΛ(g) _g _>0^d (g) =(∑j=1dαjτj)2=BC(α,τ)2≤1, = ( _j=1^d _j _j )^2=BC(α,τ)^2≤ 1, with equality exactly when α=τα=τ (Prop. 7). The entire benefit of grading is therefore the Bhattacharyya affinity between where the target places its weight and where the data places its variance. Grading creates no information: it converts a correct prior about the target into reduced sample complexity at this rate, and an incorrect prior into increased sample complexity at the same rate. Under geometric stratification of the profiles the ratio decays exponentially in the number of levels (Cor. 3). This is the only route by which any exponential-in-L statement enters this work, and it enters as a hypothesis, not as a theorem. Prop. 8 reads that quantity as a condition on the grades themselves, and it is the paper’s main positive result. The grades at which the graded bound is strictly smaller form an open convex set, invariant under translation along 1, so restricting to non-negative rationals costs nothing. The set is entered, to first order, exactly when ⟨,α−τ⟩>0, ,α-τ >0, that is, high grades on coordinates where the target’s energy exceeds the data’s variance and low grades elsewhere. The set is nonempty precisely when α≠τα≠τ, and the standard transformer, which sets g=g=1, lies on its boundary: the gain there is exactly 11 and the gradient is 2(logλ)(τ−α)≠02( λ)(τ-α)≠ 0. Isotropy is therefore not a local optimum of the sample-complexity landscape. Its gauge orbit is the unique set of grades at which the gain equals 11 for every corpus and every target—the choice that ignores the profiles—and a descending direction is available from it in the direction the two profiles determine. Both profiles can be estimated offline, so the sign of that inner product is checkable before any pretraining begins (Rem. 17). The optimal grades within the region are then the solution of a convex program rather than the result of a search (Cor. 4), projected to the clipped region admitted by Prop. 10. The gain attained at grades computed from estimated profiles degrades only to second order in the estimation error (Lem. 2): the gradient of the gain vanishes at the optimum, so the framework is protected exactly where protection is needed. The geometry of this selection problem admits a natural interpretation in geometric invariant theory, developed in Sec. 6.7. The admissible set is an open convex cone; the criterion ⟨,α−τ⟩>0 ,α-τ >0 is a Hilbert–Mumford-type pairing that decides which directions descend from the uniform point; and the ordinary transformer sits on the boundary as a semistable point of the grading action. The detailed development is deferred to that subsection so that the statistical content can be stated first in elementary language. Three facts fix the location of the claim. The advantage is not one of expressivity: by Remarks 1 and 3 graded and ungraded models realise the same function class, and the content of grading lies in parameterization, initialisation, and implicit bias. It is also not one of optimisation: exponential rescaling degrades the smoothness constant by a factor λqmax≥1λ^q_ ≥ 1, so the stationarity guarantee available for a graded model is weaker than for its ungraded counterpart (Lemmas 5 and 5.1). This asymmetry is the substance of the claim, and both of its sides are priced by the same constant: at clip λqmax≤Cλ^q_ ≤ C the penalty in optimisation steps is at most C, uniformly in L and in the profiles, while the saving in tokens is at most C2C^2 (Rem. 16, Prop. 10). The price is linear in the clip and the purchase quadratic, so the trade is favourable at every setting, and the exponential regime of Cor. 3 is reached as the clip opens. Grading spends a cheap resource—compute per step—to save an expensive one—high-quality tokens. The advantage lives in the norm required to represent a given target, and hence in the data required to learn it, which is where Propositions 7 and 8 place it in closed form. Prop. 7 compares two upper bounds, for a graded embedding composed with a linear read-out. Since graded and ungraded models realise the same functions, the separation the framework claims is between the graded prior and its absence rather than between two hypothesis classes. What converts the comparison of bounds into that separation is a matching minimax lower bound over the uniform target class, with the estimator unrestricted. Thm. 4.1 proves it in the level-stratified regime of Cor. 3: at squared loss and over all estimators, the ratio of minimax risks is Θ(Λ(g)) ( (g)) throughout an explicit window of sample sizes, so in the regime the planned configurations occupy the separation is established. The general case is Conj. 2, and Sec. 8.4 records what its proof requires; Rem. 11 records the scope. Two limitations are resolved in this manuscript—the adaptivity cost of estimated grades by Lem. 2 and the lower bound in the stratified regime by Thm. 4.1—and each of the remaining ones names the tool its repair would require. The costs are small and are computed rather than asserted. Grading preserves the asymptotic complexity of attention exactly (Prop. 4). The training overhead is dominated by the level projections at d2d^2 parameters, giving 0.33%0.33\% at 345M and 0.24%0.24\% at 7B for the configurations of Sec. 8.1.1; this follows from d2≪Nd^2 N at those scales and must be recomputed at other scales. At deployment the overhead is zero: by Corollaries 1 and 3 a trained GLLM compiles to a standard transformer of identical architecture and identical inference cost, so the training figure is an upper bound on the total lifetime cost of the prior. The framework’s governing hypothesis is settled before any model is trained, and the predictions are staged so that the cheapest test comes first. Both profiles can be estimated offline, so BC(α,τ)BC(α,τ) is reported per task in advance; a value near 11 on the hierarchical tasks falsifies Assumption 1, and with it every conditional prediction, at the cost of two estimates. The same two estimates certify every candidate grading against the criterion above, one inner product apiece. Configurations, baselines, benchmarks, and the selection ladder of Tab. 2 follow, with arms of identical architecture and budget distinguished only by what their grades consulted and ordered in advance by their certified inner products; each prediction is marked as established or as conditional on named hypotheses. The ordering is fixed before training, which makes it a prediction rather than a fit, and Sec. 8 specifies the runs that test it. The framework extends in a direction the present construction does not reach. The grading of Sec. 4 is block-diagonal with respect to ⨁lVl _lV_l and therefore transports nothing between levels; directed level-to-level transport requires block-triangular morphisms ϕl′←l:Vl→Vl′ _l ← l:V_l→ V_l internal to the graded hidden space. The morphic structure of [sh-111] supplies them, and subsumes external-tool paradigms as a special case via functorial internalization; MLGE does not. 2. Background: Language Models and the Uniform Transformer In this section we fix the notation and review the classical results of the graded constructions introduced in [2024-02, sh-89, sh-95, sh-111]. The central question, made precise in Conj. 1 and Assumption 1, is what the transformer’s uniform treatment of positions and feature dimensions costs and where that cost is located. 2.1. Language Models and the Autoregressive Objective Definition 1. Let V be a finite vocabulary. A language model is a parameterized probability distribution pθp_θ on ∗V^*. For w=(w1,…,wT)∈Tw=(w_1,…,w_T) ^T the chain rule gives the autoregressive factorization (1) pθ(w)=∏t=1Tpθ(wt∣w<t),w<t:=(w1,…,wt−1),p_θ(w)= _t=1^Tp_θ(w_t w_<t), w_<t:=(w_1,…,w_t-1), reducing the task to estimation of the conditionals [shannon1951prediction, bengio2003neural]. Quality is reported as perplexity, the exponentiated per-token cross-entropy (2) PPL(w)=exp(−1T∑t=1Tlogpθ(wt∣w<t)).PPL(w)= \! (- 1T _t=1^T p_θ(w_t w_<t) ). Every architecture in this paper, graded or not, is a parameterization of Eq. 1; grading acts on the representation of w<tw_<t and on the weight each conditional carries in the loss, and leaves the factorization itself intact. The direct empirical objective for Eq. 1 is causal language modeling, (3) ℒCLM(θ)=−1n∑t=1nlogpθ(xt∣x<t).L_CLM(θ)=- 1n _t=1^n p_θ(x_t x_<t). Lemma 1. Write pt:=p(⋅∣x<t)p_t:=p(· x_<t) for the data conditional and p^t:=pθ(⋅∣x<t) p_t:=p_θ(· x_<t) for the model’s. The population form of Eq. 3 decomposes as (4) [ℒCLM(θ)]=1n∑t=1n(H(pt)+DKL(pt∥p^t)),E\! [L_CLM(θ) ]= 1n _t=1^n (H(p_t)+D_KL(p_t\,\|\, p_t) ), so the objective is bounded below by the conditional entropy of the data, attained exactly when p^t=pt p_t=p_t for all t. Proof. Take expectations in Eq. 3 over xt∼ptx_t p_t and add and subtract H(pt)H(p_t) term by term. ∎ Eq. 4 is the object the graded loss of Sec. 5 modifies: per-token weights λqtλ^q_t re-weight its summands, leaving the form of the decomposition intact while changing which positions dominate the gradient. Which re-weightings preserve the population minimizer is exactly the consistency question answered by Prop. 9. Eq. 3 is non-convex in θ, and no convergence guarantee to a global minimizer is known for transformer parameterizations; standard non-convex SGD analysis controls stationarity, mint≤T‖∇ℒ‖2=O(1/T), _t≤ TE\| \|^2=O(1/ T), not the value of Eq. 4. This gap is inherited, not repaired, by grading, and Thm. 5.1 quantifies the direction in which grading moves it. 2.2. The Transformer Let n be the sequence length, d the model dimension, H the number of heads, dk=dv=d/Hd_k=d_v=d/H the per-head dimension, and X∈ℝn×dX ^n× d a layer input; see [vaswani2017attention] for the standard transformer and [sh-95] for the graded transformer. Tokens enter through an embedding matrix We∈ℝ||×dW_e ^|V|× d; subword vocabularies are built by byte-pair encoding or unigram segmentation [sennrich2016neural, kudo2018sentencepiece], and tying the output projection to We⊤W_e couples the generation and representation geometries; the graded softmax of Sec. 5 acts on precisely this tied projection. Segmentation is chosen by corpus frequency and carries no record of morphological or syntactic role. Tokenisation is thus the first place hierarchy is discarded, and the level-0 subspace V0V_0 of Sec. 4 is the first point at which it is reintroduced. Since attention is permutation-equivariant in the rows of X, position is supplied explicitly, classically by the sinusoidal encoding (5) PE(t,j)=sin(t/ 10000 2⌊j/2⌋/d),j even,cos(t/ 10000 2⌊j/2⌋/d),j odd,PE(t,j)= cases \! (t\,/\,10000^\,2 j/2 /d ),&j even,\\[4.30554pt] \! (t\,/\,10000^\,2 j/2 /d ),&j odd, cases added to the embeddings; Eq. 16 is its graded refinement. With projections WQh,WKh∈ℝd×dkW_Q_h,W_K_h ^d× d_k, WVh∈ℝd×dvW_V_h ^d× d_v, and WO∈ℝHdv×dW_O ^Hd_v× d, multi-head self-attention is (6) MHSA(X)=Concat(Head1,…,HeadH)WO,Headh=Attention(XWQh,XWKh,XWVh), splitMHSA(X)&=Concat(Head_1,…,Head_H)\,W_O,\\ Head_h&=Attention(XW_Q_h,XW_K_h,XW_V_h), split where, with the causal mask Mcausal∈0,−∞n×nM_causal∈\0,-∞\^n× n, (Mcausal)ij=−∞(M_causal)_ij=-∞ for j>ij>i, enforcing Eq. 1, (7) Attention(Q,K,V)=softmax(QK⊤dk+Mcausal)V.Attention(Q,K,V)=softmax\! ( QK d_k+M_causal )V. The scaling dk−1/2d_k^-1/2 is dictated by the variance of the logits: for q,kq,k with independent unit-variance entries, Var(q⊤k)=dkVar(q k)=d_k, so unscaled logits drive the softmax toward a one-hot distribution with vanishing gradient as dkd_k grows. Eq. 7 computes the bilinear form q⊤kq k, the Euclidean inner product on ℝdkR^d_k, in which no coordinate is distinguished. Replacing this form by a graded one is the single modification of Sec. 3.1. Each position independently passes through a feed-forward network, (8) FFN(x)=max(0,xW1+b1)W2+b2,W1∈ℝd×dff,W2∈ℝdff×d,FFN(x)= (0,\,xW_1+b_1)W_2+b_2, W_1 ^d× d_f,\ W_2 ^d_f× d, conventionally with dff=4d_f=4d; gated variants such as SwiGLU adjust dffd_f to hold the parameter count fixed [shazeer2020glu]. Sublayers are wrapped in residual connections and normalisation, in the pre-norm ordering x′=x+Sublayer(LN(x))x =x+Sublayer(LN(x)), which bounds gradients at initialisation independently of depth [xiong2020pre]. Normalisation is by LayerNorm [ba2016layer] or, standard in current open models, RMSNorm [zhang2022rms], (9) RMSNorm(x)=γ⊙x1d∑jxj2+ϵ.RMSNorm(x)=γ x 1d _jx_j^2+ε. Proposition 1. In a pre-norm block with dff=4d_f=4d and dk=dv=d/Hd_k=d_v=d/H, the FFN accounts for 2/32/3 of the block’s parameters, up to O(d)O(d) terms. Proof. Attention contributes WQ,WK,WV,WOW_Q,W_K,W_V,W_O, each d×d× d after concatenating heads, hence 4d24d^2; the FFN contributes W1∈ℝd×4dW_1 ^d× 4d and W2∈ℝ4d×dW_2 ^4d× d, hence 8d28d^2. Biases and normalisation parameters are O(d)O(d), and 8d2/12d2=2/38d^2/12d^2=2/3. ∎ Prop. 1 calibrates every overhead claim in this paper: a graded parameter budget of O(d+Hdk)O(d+Hd_k) per block is negligible against the 12d212d^2 established here, and the full accounting for the proposed configurations appears in Sec. 8.1. Proposition 2. One layer of Eq. 6 together with Eq. 8 costs O(n2d+nd2)O(n^2d+nd^2) time. Proof. Per head, logits and value weighting cost O(n2dk)O(n^2d_k); over H heads with dk=d/Hd_k=d/H this is O(n2d)O(n^2d). Projections, concatenation, and Eq. 8 contribute O(nd2)O(nd^2). ∎ Prop. 4 shows grading preserves this bound exactly. 2.3. Scaling Laws Cross-entropy loss under Eq. 3 follows fitted power laws in non-embedding parameters N, data D, and compute C≈6NDC≈ 6ND [kaplan2020scaling]; the compute budget C of this subsection is standard notation and is distinct from the grade clip C of Sec. 8.1.1. The parameterization of Hoffmann et al. [hoff], (10) L(N,D)=E+ANα+BDβ,E≈1.69,α≈0.34,β≈0.28,L(N,D)=E+ AN^α+ BD^β, E≈ 1.69,\ α≈ 0.34,\ β≈ 0.28, yields the compute-optimal allocation below. Proposition 3. Minimising Eq. 10 subject to 6ND=C6ND=C gives N∗∝Cβ/(α+β)N^* C^β/(α+β) and D∗∝Cα/(α+β)D^* C^α/(α+β); for α≈βα≈β both exponents approach 1/21/2, realised empirically at roughly 2020 tokens per parameter. Proof. Substituting D=C/(6N)D=C/(6N) into Eq. 10 and differentiating in N gives −αAN−α−1+βB(6N/C)βN−1=0-α AN^-α-1+β B(6N/C)^βN^-1=0, whence Nα+β∝CβN^α+β C^β. ∎ Two qualifications govern the use made of Eq. 10 here. First, it is a fitted regularity with no derivation from Eq. 3; it constrains the loss, not the capabilities the loss induces, and whether capability changes smoothly or abruptly with scale is disputed [wei2022emergent, schaeffer2023mirage]. Second, Eq. 10 aggregates all positions into one scalar: a token carrying a clause boundary and a token carrying a function word contribute identically. The graded loss of Sec. 5 reweights the summands of Eq. 4 by token grade, which plausibly changes the constants of Eq. 10 rather than its form; whether it changes the exponents is an empirical question this paper does not settle. 2.4. The Uniformity Principle and Its Cost Every component above treats its inputs uniformly. Attention Eq. 7 scores all pairs through one Euclidean form; the FFN Eq. 8 applies one map to all positions; the loss Eq. 3 weights all positions equally; and the scaling law Eq. 10 aggregates them into one scalar. We call this the uniformity principle: all tokens, parameters, and feature dimensions carry equal structural status. Its geometric expression is that the representation space ℝdR^d is isotropic: no coordinate direction is distinguished, so no direction can carry structural meaning that the architecture must respect. Formally, Eq. 7 is equivariant under the orthogonal group acting on ℝdkR^d_k jointly through the projections, and in particular under the unweighted scalar action λ⋅xλ· x; it is equivariant under the weighted action λ⋆x=(λqjxj)λ x=(λ^q_jx_j) only in the degenerate case q=(1,…,1)q=(1,…,1). When data carries a weighted structure with q≠(1,…,1)q≠(1,…,1), the architecture is committed to the wrong symmetry, and the structure must be recovered from data if at all. Existing architectural variants relax uniformity only as a function of positional offset. Tab. 1 classifies them by the structural information their inductive bias can express. Variant Key mechanism Examples Structural bias Decoder-only Causal autoregression GPT series, Llama None (emergent) Encoder–decoder Bidirectional encoding T5, BART None (task-supervised) State space Selective linear recurrence Mamba-2 Recency only Sparse attention Restricted mask support Longformer, BigBird Locality only Hybrid SSM–attention Interleaved recurrence/attention Jamba Recency ++ locality Graded (this paper) Exponential subspace scaling Sections 3 and 4 Explicit, multi-level Table 1. Architecture variants by structural inductive bias [gu2023mamba, gu2024mamba2, peng2023rwkv, raffel2020exploring, radford2018improving]. “Recency” and “locality” denote biases that are functions of positional offset |i−j||i-j| alone; no existing variant conditions on structural role. Grading acts on the metric of the representation space, sparsity and recurrence on the mask, and the two compose. The cost of uniformity that motivates this paper is compositional, and we record it in two statements with different levels of support. The first is the classical conjecture, which this paper does not prove. Conjecture 1. An L-level context-free grammar admits Ω(nL) (n^L) parse trees over inputs of length n. The sample complexity of an architecture with no structural prior, trained by Eq. 3 to a fixed error on structures of depth L, grows exponentially in L. Conj. 1 is consistent with the observed accuracy decay on k-nested arithmetic, k-hop inference, and k-deep call stacks, and it motivated the graded program; but the results of this paper neither prove it nor depend on it. What the theory developed in Sec. 4 requires is strictly weaker, and unlike Conj. 1 it is directly measurable on annotated corpora. Assumption 1 (Profile Divergence). For hierarchical prediction targets in natural language, the target energy profile α and the data energy profile τ of Def. 2 are far apart in Bhattacharyya affinity: structurally decisive features carry little of the corpus variance, and high-variance lexical features carry little of the decision. Assumption 1 is the empirical hypothesis on which the benefit of grading rests. Prop. 7 converts it into an exact sample-complexity ratio, Λ⋆=BC(α,τ)2 =BC(α,τ)^2, and Cor. 3 shows that under geometric stratification of the profiles the ratio decays exponentially in the number of levels; this is the only route by which any exponential-in-L statement enters this work, and it enters as a hypothesis, not as a theorem. Both profiles can be estimated offline, so Assumption 1 is testable before any model is trained; Sec. 7 carries out the construction of the estimators and Sec. 8 stages the measurement first. 3. Graded Attention in Decoder-Only Architectures Sec. 2 closed with a deficit stated in two forms: Conj. 1, which this paper does not prove, and Assumption 1, which it makes precise and quantifies. The transformer’s representation space is isotropic, so hierarchical competence must be recovered from data. The remainder of the paper addresses the deficit by changing the metric on the representation space rather than the computation performed over it. This section develops exponentially graded multi-head self-attention within causal decoder-only stacks and establishes three facts. Grading leaves the asymptotic cost of attention unchanged (Prop. 4). It acts by an invertible change of parameters (Rem. 1), so a trained GLLM compiles to a standard transformer and carries its prior at zero deployment cost (Cor. 1), a property none of the variants in Tab. 1 shares. And at a fixed norm budget the graded constraint set is an ellipsoid whose axes are aligned with the grades (Prop. 5), strictly containing the uniform ball; the advantage therefore cannot come from representability, since a larger class at fixed budget hurts generalisation. Together these facts place the advantage in the norm required to represent a given target, and hence in the data required to learn it. Sec. 4 decomposes the embedding space into level-specific subspaces via Multi-Level Graded Embeddings and computes that quantity exactly (Prop. 7), characterises in closed form the grades at which the graded bound is the smaller one (Prop. 8), and gives the behaviour under stratified profiles (Cor. 3). Sec. 5 extends the grading to the training objective and characterises its consistency (Prop. 9). Sec. 8 states the program that will test the resulting predictions, and Sec. 6 assembles the results into a procedure that selects, certifies, and prices the grades in domains where they are not given. Three distinct grade objects appear in what follows and should not be conflated: per-head feature grades qh∈ℚ≥0dkq_h _≥ 0^d_k (this section), per-level embedding grades ql∈ℚ≥0dlq_l _≥ 0^d_l (Sec. 4), and per-token loss grades qt∈ℚ≥0q_t _≥ 0 (Sec. 5). All take values in ℚ≥0Q_≥ 0, and all act through the same scalar action λ⋆x=(λqixi)λ x=(λ^q_ix_i) with a common base λ>1λ>1. We work in the decoder-only paradigm of the GPT series [radford2019language], with multi-head self-attention and the causal mask as fixed in Eqs. 6 and 7: queries Qh=XWQhQ_h=XW_Q_h, keys Kh=XWKhK_h=XW_K_h, and values Vh=XWVhV_h=XW_V_h in ℝn×dkR^n× d_k for each head h∈1,…,Hh∈\1,…,H\, with dk=dv=d/Hd_k=d_v=d/H and the mask McausalM_causal enforcing Eq. 1. Every position and every feature dimension enters Eq. 7 identically: the cost is O(n2d+nd2)O(n^2d+nd^2) per layer by Prop. 2, and no coordinate of ℝdkR^d_k is distinguished from any other. 3.1. Exponentially Graded Multi-Head Self-Attention In GLLMs we replace MHSA by Exponentially Graded Multi-Head Self-Attention (EG-MHSA), built from the Exponentially Graded Transformer of [sh-95]. Each head carries its own grading of the per-head space ℝdk=⨁j=1dkℝejR^d_k= _j=1^d_kRe_j, assigning to the basis vector eje_j a grade qh,j∈ℚ≥0q_h,j _≥ 0. The grading transformation is the diagonal matrix Mqh,λ=diag(λqh,1,…,λqh,dk)∈ℝdk×dk,M_q_h,λ=diag\! (λ^q_h,1,…,λ^q_h,d_k ) ^d_k× d_k, with fixed base λ>1λ>1. Taking λ=e1/dkλ=e^1/d_k gives λ≈1.008λ≈ 1.008 at dk=128d_k=128 and λ≈1.016λ≈ 1.016 at dk=64d_k=64, so that the clipped grade range qmax=dklnCq_ =d_k C of Sec. 8.1.1 spans a spread of order dkd_k. The base is chosen small precisely because the grades are not, and the clip C, not the base, is the constant that prices the construction (Rem. 18). The graded queries, keys, and values are obtained by right-multiplication: (11) Qh′=QhMqh,λ,Kh′=KhMqh,λ,Vh′=VhMqh,λ.Q_h =Q_hM_q_h,λ, K_h =K_hM_q_h,λ, V_h =V_hM_q_h,λ. The graded head is (12) Headh′=softmax(Qh′(Kh′)⊤dk+Mcausal)Vh′.Head_h =softmax\! ( Q_h (K_h ) d_k+M_causal )V_h . Since Mqh,λM_q_h,λ is diagonal and hence symmetric, (Kh′)⊤=Mqh,λKh⊤(K_h ) =M_q_h,λK_h , so the pre-softmax logits are Qh′(Kh′)⊤dk=QhMqh,λ2Kh⊤dk,Mqh,λ2=diag(λ2qh,1,…,λ2qh,dk), Q_h (K_h ) d_k= Q_hM_q_h,λ^2K_h d_k, M_q_h,λ^2=diag\! (λ^2q_h,1,…,λ^2q_h,d_k ), so that agreement between query and key along eje_j is weighted by λ2qh,jλ^2q_h,j: the grading acts on the bilinear form defining attention, replacing the Euclidean inner product of Eq. 7 on ℝdkR^d_k by the graded form ⟨u,v⟩qh=u⊤Mqh,λ2v u,v _q_h=u M_q_h,λ^2v. The multi-head output is EG-MHSA(X)=Concat(Head1′,…,HeadH′)WO.EG -MHSA(X)=Concat(Head_1 ,…,Head_H )\,W_O. The grading induces a filtration of the per-head space by grade, V≤q(h)=⨁j:qh,j≤qℝej,V≤q(h)⊆V≤q′(h) for q≤q′,V_≤ q^(h)= _j\,:\,q_h,j≤ qRe_j, V_≤ q^(h) V_≤ q ^(h) for q≤ q , which is the algebraic object that linguistic stratification is to be mapped onto: low grades for lexical and morphological features, high grades for phrasal and discourse-level ones [2024-02, sh-89]. Proposition 4. EG-MHSA has the same asymptotic time and space complexity as standard MHSA, namely O(n2d+nd2)O(n^2d+nd^2) per layer including feed-forward components. Proof. The grading step Eq. 11 applies 3H3H diagonal multiplications, each costing O(ndk)O(nd_k), for a total of O(nHdk)=O(nd)O(nHd_k)=O(nd). The matrix Mqh,λ2M_q_h,λ^2 is precomputable per head in O(dk)O(d_k). The attention computation Eq. 12 is structurally identical to standard MHSA: O(n2dk)O(n^2d_k) per head for the logits and O(n2dk)O(n^2d_k) per head for the softmax and value weighting, giving O(n2d)O(n^2d) over H heads. Concatenation and the output projection add O(nd2)O(nd^2), and the feed-forward block O(nd2)O(nd^2). Summing gives O(n2d+nd2)O(n^2d+nd^2), matching Prop. 2. ∎ Remark 1. Because Mqh,λM_q_h,λ is invertible and WQh,WKh,WVhW_Q_h,W_K_h,W_V_h are unconstrained, the substitution W~∙h=W∙hMqh,λ W_ h=W_ hM_q_h,λ is a bijection of the parameter space carrying Eq. 12 to a standard head. EG-MHSA and MHSA therefore realise the same set of functions. In this precise sense the grading changes the metric of the representation space and leaves the computation over it intact, and two consequences follow, running in opposite directions. On one side, no separation is available from representability. On the other, an invertible reparameterization can be applied once and then discarded, which gives Cor. 1. What grading fixes is a parameterization, and with it an initialisation, an implicit bias under gradient descent, and, once the projection weights are norm-constrained, an effective hypothesis class whose geometry is computed in Prop. 5. This is a statement about which functions are reachable and preferred, not about which are representable, and the informative comparison fixes the target rather than the budget (Prop. 7). Corollary 1. Let θ be the parameters of a trained GLLM whose attention sublayers are EG-MHSA with grades qh\q_h\. The map θ↦θ~θ θ given by W~∙h=W∙hMqh,λ W_ h=W_ hM_q_h,λ for ∙∈Q,K,V ∈\Q,K,V\ yields a standard decoder-only transformer of identical architecture computing the identical function on every input. The grades need not be stored at inference, and the diagonal multiplications of Eq. 11 need not be performed. Proof. Immediate from the bijection of Rem. 1, applied once and offline. Since Mqh,λM_q_h,λ is diagonal with strictly positive entries, W~∙h W_ h is well defined and the substitution violates no constraint on θ. ∎ Remark 2. The graded prior is paid for once, during training, and costs nothing afterwards. This separates grading from every variant classified in Tab. 1: selective recurrence, restricted mask support, and hybrid schemes encode their inductive bias in the computation, pay for it at every forward pass, and do not compile to the architecture they modify. A metric can be absorbed into the weights, while a mask cannot. The training overhead accounted in Sec. 8.1.3 is therefore an upper bound on the total lifetime cost of the prior, and Sec. 8 reports zero inference overhead as an established consequence of the construction rather than an experimental outcome. Proposition 5. Fix grades qhh=1H\q_h\_h=1^H and a budget B>0B>0, and let ℱBgraded=X↦EG-MHSA(X):∥W∙h∥F≤B∀∙∈Q,K,V,∀h,ℱBunif=X↦MHSA(X):∥W∙h∥F≤B∀∙∈Q,K,V,∀h. splitF_B^graded&= \X -MHSA(X)\;:\;\|W_ h\|_F≤ B\ \ ∀\, ∈\Q,K,V\,\ ∀ h \,\\ F_B^unif&= \X (X)\;:\;\|W_ h\|_F≤ B\ \ ∀\, ∈\Q,K,V\,\ ∀ h \. split In the effective coordinates W~∙h=W∙hMqh,λ W_ h=W_ hM_q_h,λ of Rem. 1, the budget constraint defining ℱBgradedF_B^graded is the ellipsoid ∑j=1dkλ−2qh,j‖W~∙hej‖22≤B2, _j=1^d_kλ^-2q_h,j \| W_ h\,e_j \|_2^2\;≤\;B^2, with semi-axis Bλqh,jBλ^q_h,j along the j-th column, whereas that defining ℱBunifF_B^unif is the Frobenius ball of radius B. Consequently ℱBunif⊆ℱBgraded,F_B^unif _B^graded, with strict inclusion whenever some qh,j>0q_h,j>0, and the Rademacher complexity of the graded class is weakly greater at every fixed B. Proof. By the substitution of Rem. 1, a graded head with weights W∙hW_ h computes the same function as a standard head with effective weights W~∙h=W∙hMqh,λ W_ h=W_ hM_q_h,λ. Under this bijection the constraint ‖W∙h‖F≤B\|W_ h\|_F≤ B becomes ‖W~∙hMqh,λ−1‖F≤B\| W_ hM_q_h,λ^-1\|_F≤ B, which is the displayed ellipsoid; its semi-axis along eje_j is Bλqh,jBλ^q_h,j. Since qh,j≥0q_h,j≥ 0 and λ>1λ>1, every semi-axis is at least B, so the ellipsoid contains the Frobenius ball of radius B, strictly in the j-th direction whenever qh,j>0q_h,j>0. Passing from weight sets to function classes preserves the inclusion, and any W~∙h W_ h with ‖W~∙h‖F=B\| W_ h\|_F=B supported on a coordinate with qh,j>0q_h,j>0 can be scaled by λqh,jλ^q_h,j to lie in the graded class but not the uniform one, giving strictness. Monotonicity of Rademacher complexity under inclusion of hypothesis classes gives the final claim. ∎ Corollary 2. No capacity bound favouring the graded model is available at fixed B. Prop. 5 identifies the geometric object grading produces: an ellipsoidal constraint set, its axes aligned with the grades and its eccentricity along eje_j equal to λqh,jλ^q_h,j. Reading the proposition as Cor. 2 alone understates it. The informative comparison fixes the target rather than the budget, asks each class for the smallest budget at which it represents that target, and evaluates the capacity bounds at those budgets; Prop. 7 carries this out and identifies the sample-complexity ratio in closed form, and Prop. 8 characterises exactly the grades at which that ratio favours the graded bound. The ellipsoid is also what makes the comparison sharp: it is the geometry on which a matching minimax lower bound is posed, and on it the separation between the graded prior and its absence, rather than a ratio of upper bounds, is obtained. Thm. 4.1 establishes that separation in the level-stratified regime, over all estimators, throughout the window of Lem. 4; Conj. 2 states the general case, and Sec. 8.4 records what its proof requires. 4. Hierarchical Grading for Language Modeling Linguistic structure is layered. Subword units aggregate into words, words compose into phrases and clauses, clauses into sentences, sentences into discourse, and discourse into pragmatic inference; the observation is common to the generative and model-theoretic traditions in linguistics, and we take it as given rather than argue it here. Classical LLMs approximate this stratification implicitly, through patterns emergent in unannotated corpora [radford2019language]. Their embedding space is isotropic: no coordinate direction is distinguished, so no direction can be reserved for a linguistic level. We propose Multi-Level Graded Embeddings (MLGE), which give the stratification an explicit home in the geometry of the embedding space, extending the graded vector space formalism of [2024-02, sh-89, sh-95] to autoregressive language modeling. Sec. 4.2 then quantifies the construction: it establishes the exact sample-complexity ratio obtained by grading (Prop. 7), characterises in closed form the grades at which the graded model is favoured and locates the standard transformer on the boundary of that set (Prop. 8), and shows the ratio decays exponentially in depth under stratified profiles (Cor. 3), converting Assumption 1 into a computable, falsifiable quantity. 4.1. The Graded Embedding Space Let E=(ε1,…,εn)⊤∈ℝn×dE=( _1,…, _n) ^n× d be the input embedding matrix for a token sequence y=(y1,…,yn)∈ny=(y_1,…,y_n) ^n, with εt=We[yt]∈ℝd _t=W_e[y_t] ^d and learnable weights We∈ℝ||×dW_e ^|V|× d as in Sec. 2.2. We decompose the embedding space as an orthogonal direct sum ℝd=⨁l=0L−1Vl,Vl≅ℝdl,∑l=0L−1dl=d,R^d= _l=0^L-1V_l, V_l ^d_l, _l=0^L-1d_l=d, with dl=d/Ld_l=d/L under uniform allocation, where VlV_l is reserved for features salient to linguistic level l: V0V_0 for subword and morphological structure, V1V_1 for syntactic constituency, V2V_2 for sentential semantics, VL−1V_L-1 for discourse and pragmatics. Level-specific components are El=EPl∈ℝn×dlE_l=EP_l ^n× d_l, where Pl∈ℝd×dlP_l ^d× d_l has orthonormal columns spanning VlV_l, initialised by principal component analysis of features annotated for level l by an offline constituency parser. When the PlP_l are exactly orthonormal with mutually orthogonal ranges, ∑lPlPl⊤=Id _lP_lP_l =I_d and the reconstruction E=∑lElPl⊤E= _lE_lP_l is exact. During training the columns are only approximately orthonormal, and we penalise Ωorth(P)=∑l,l′‖Pl⊤Pl′−δll′Idl‖F2, _orth(P)= _l,l \|P_l P_l - _l I_d_l \|_F^2, so that the reconstruction error is controlled by Ωorth(P) _orth(P) and hence by the penalty weight. The error is a function of optimisation, not of L, and no rate in L is claimed. Each VlV_l carries a grading tuple ql=(ql,1,…,ql,dl)∈ℚ≥0dlq_l=(q_l,1,…,q_l,d_l) _≥ 0^d_l, whose entries are rational so that they may record discrete structural depths exactly. The associated grading transformation is Mql,λ=diag(λql,1,…,λql,dl)∈ℝdl×dl,M_q_l,λ=diag\! (λ^q_l,1,…,λ^q_l,d_l ) ^d_l× d_l, with the base λ>1λ>1 fixed once for the whole architecture, as in Sec. 3. The graded embeddings are (13) E′=∑l=0L−1ElMql,λPl⊤=EG,G:=∑l=0L−1PlMql,λPl⊤,E = _l=0^L-1E_l\,M_q_l,λ\,P_l =E\,G, G:= _l=0^L-1P_lM_q_l,λP_l , which in the graded basis is the block-diagonal map ⨁lMql,λ _lM_q_l,λ. Evaluating Eq. 13 costs O(ndL)O(ndL). Remark 3. G is symmetric positive definite, hence invertible, and WeW_e is unconstrained. The substitution W~e=WeG W_e=W_eG is therefore a bijection of the parameter space carrying Eq. 13 to an ungraded embedding, and MLGE realises exactly the function class of a standard embedding layer. As in Rem. 1, two consequences follow in opposite directions. On one side, the construction adds no capacity, so no separation is available from representability. On the other, the substitution extends Cor. 1 from the attention sublayers to the embedding layer, so the whole of the graded construction, attention, embeddings, and positional encoding alike, is folded into the weights at the end of training and costs nothing at inference. What the construction contributes is its parameterization: G fixes an initialisation aligned with annotated linguistic levels, and constrains the geometry along which gradient descent moves. Proposition 6. Let η:=maxl≠l′‖Pl⊤Pl′‖opη:= _l≠ l \|P_l P_l \|_op and λqmax:=maxl,jλql,jλ^q_ := _l,jλ^q_l,j. Then G of Eq. 13 satisfies G(Vl)⊆VlG(V_l) V_l exactly when η=0η=0, and in general ‖G−⨁l=0L−1Mql,λ‖op≤Lλqmaxη. \|G- _l=0^L-1M_q_l,λ \|_op\;≤\;L\,λ^q_ \,η. Proof. When η=0η=0 the ranges of the PlP_l are mutually orthogonal, so Pl′⊤GPl′=Mql′,λP_l GP_l =M_q_l ,λ and Pl′⊤GPl=0P_l GP_l=0 for l≠l′l≠ l ; thus G is block-diagonal in the graded basis and preserves each VlV_l. In general, the off-diagonal block indexed by (l′,l)(l ,l) is Pl′⊤PlMql,λPl⊤Pl′P_l P_lM_q_l,λP_l P_l , of operator norm at most λql,maxη2≤λqmaxηλ^q_l, η^2≤λ^q_ η for η≤1η≤ 1, and there are at most L such blocks in any row. Summing gives the bound. ∎ Remark 4. Prop. 6 is a consistency check, not a capability claim: it states that the intended block structure survives approximate orthogonality of the PlP_l, with degradation linear in L and in the grading amplitude λqmaxλ^q_ . It is the first place where large grades are seen to cost something. 4.2. Sample Complexity Under Grading Two routes to a separation are closed before we begin, and closing them fixes the location of the third. By Rem. 3 graded and ungraded embeddings realise the same functions, so nothing follows from representability. By Prop. 5 the graded constraint set at a common norm budget is an ellipsoid strictly containing the uniform ball, with weakly greater Rademacher complexity, so nothing favouring the graded model follows at fixed B either. The informative comparison fixes the target rather than the budget. Each class is asked for the smallest norm at which it represents a given f∗f^*, and the resulting bound is evaluated at that norm. The grading then acts on two quantities at once: it lowers the norm required for targets aligned with it, and it raises the effective radius of the data. The theorem below is the exact trade-off between them. Prop. 8 then reads that trade-off as a condition on the grades themselves, and identifies the standard transformer as the point at which the trade-off is declined. Throughout this subsection the analysis is carried out for the graded linear map x↦⟨u,Gx⟩x u,Gx realised by MLGE composed with a single linear read-out. It is a statement about the embedding layer and its immediate consumer, not about the full stack; the extension is discussed in Rem. 11. Definition 2 (Energy Profiles). Let P be a distribution on ℝd×ℝR^d×R with σj2:=[xj2]∈(0,∞) _j^2:=E[x_j^2]∈(0,∞) for every j, and let f∗(x)=⟨w∗,x⟩f^*(x)= w^*,x with wj∗≠0w_j^*≠ 0 for every j. Writing Δd−1 ^d-1 for the probability simplex in ℝdR^d, the target profile α∈Δd−1α∈ ^d-1 and the data profile τ∈Δd−1τ∈ ^d-1 are αj=(wj∗)2‖w∗‖22,τj=σj2∑k=1dσk2. _j= (w_j^*)^2\|w^*\|_2^2, _j= _j^2 _k=1^d _k^2. Both are supported on the graded basis of Eq. 13, both are probability vectors, and both are strictly positive in every coordinate. A coordinate on which w∗w^* vanishes contributes nothing to α and is optimally suppressed by gj→0g_j→ 0; such coordinates are deleted from the analysis rather than carried through it. Definition 3 (Graded and Uniform Classes at a Target). For B>0B>0 set ℋBgraded=x↦⟨u,Gx⟩:u∈ℝd,‖u‖2≤B,ℋBunif=x↦⟨w,x⟩:w∈ℝd,‖w‖2≤B. splitH_B^graded&= \x u,Gx \;:\;u ^d,\ \|u\|_2≤ B \,\\ H_B^unif&= \x w,x \;:\;w ^d,\ \|w\|_2≤ B \. split The representation budgets of f∗f^* are Bgraded(f∗)=min‖u‖2:Gu=w∗=‖G−1w∗‖2,Bunif(f∗)=‖w∗‖2,B^graded(f^*)= \\|u\|_2:Gu=w^* \= \|G^-1w^* \|_2, B^unif(f^*)=\|w^*\|_2, the smallest budgets at which each family contains f∗f^*. Proposition 7. Let α and τ be as in Def. 2 and let ℓ be a loss that is LℓL_ -Lipschitz in its first argument. Define the grading gain Λ(g)=(∑j=1dαjgj−2)(∑j=1dτjgj2),g∈ℝ>0d. (g)= ( _j=1^d _j\,g_j^-2 ) ( _j=1^d _j\,g_j^2 ), g _>0^d. Then: (i) Λ is invariant under g↦cg cg for every c>0c>0, and Λ()=1 (1)=1; (i) the Rademacher-based sample complexity of reaching excess risk ε over ℋBgraded(f∗)gradedH^graded_B^graded(f^*) stands to that over ℋBunif(f∗)unifH^unif_B^unif(f^*) in the ratio Λ(g) (g), uniformly in ε and f∗f^*; (i) the gain is minimised at gj⋆∝αj1/4τj−1/4,g_j \; \; _j^1/4\, _j^-1/4, and the optimal value is the squared Bhattacharyya affinity of the two profiles, (14) Λ⋆:=ming∈ℝ>0dΛ(g)=(∑j=1dαjτj)2=BC(α,τ)2; := _g _>0^d (g)= ( _j=1^d _j _j )^2=BC(α,τ)^2; (iv) Λ⋆≤1 ≤ 1, with equality if and only if α=τα=τ. Proof. (i) Replacing g by cgcg multiplies the first factor by c−2c^-2 and the second by c2c^2. At g=g=1 the factors are ∑jαj=1 _j _j=1 and ∑jτj=1 _j _j=1. (i) Fix B>0B>0 and a sample x1,…,xmx_1,…,x_m. Writing zi=Gxiz_i=Gx_i, the class ℋBgradedH_B^graded is the B-norm-bounded linear class in the variable z, so its empirical Rademacher complexity is ℜ^m(ℋBgraded)=1mϵsup‖u‖2≤B⟨u,∑i=1mϵizi⟩=Bmϵ‖∑i=1mϵiGxi‖2≤Bm(ϵ‖∑i=1mϵiGxi‖22)1/2=Bm(∑i=1m‖Gxi‖22)1/2, split R_m (H_B^graded )&= 1m\,E_ε _\|u\|_2≤ B u, _i=1^m _iz_i = Bm\,E_ε \| _i=1^m _iGx_i \|_2\\ &≤ Bm (E_ε \| _i=1^m _iGx_i \|_2^2 )^1/2= Bm ( _i=1^m\|Gx_i\|_2^2 )^1/2, split the last equality because [ϵiϵi′]=δii′E[ _i _i ]= _i . Taking expectations over the sample and using ‖Gx‖22=∑jgj2σj2E\|Gx\|_2^2= _jg_j^2 _j^2, ℜm(ℋBgraded)≤Bm(∑j=1dgj2σj2)1/2. R_m (H_B^graded )≤ B m ( _j=1^dg_j^2 _j^2 )^1/2. Setting g=g=1 gives the uniform case. Evaluating each bound at its own representation budget, and using ‖G−1w∗‖22=‖w∗‖22∑jαjgj−2\|G^-1w^*\|_2^2=\|w^*\|_2^2 _j _jg_j^-2 from Def. 2, Bgraded(f∗)(∑jgj2σj2)1/2=‖w∗‖2(∑kσk2)1/2(∑jαjgj−2)1/2(∑jτjgj2)1/2=Bunif(f∗)(∑kσk2)1/2Λ(g)1/2. splitB^graded(f^*) ( _jg_j^2 _j^2 )^1/2&=\|w^*\|_2 ( _k _k^2 )^1/2 ( _j _jg_j^-2 )^1/2 ( _j _jg_j^2 )^1/2\\ &=B^unif(f^*) ( _k _k^2 )^1/2 (g)^1/2. split For an LℓL_ -Lipschitz loss the standard symmetrisation and contraction estimate of Bartlett and Mendelson [bartlett2002rademacher] bounds the excess risk of empirical risk minimisation over a class ℋH containing f∗f^* by 2Lℓℜm(ℋ)+O(log(1/δ)/m)2L_ R_m(H)+O( (1/δ)/m), so the sample size sufficient for excess risk ε scales as the square of the product of budget and radius. That product carries the factor Λ(g)1/2 (g)^1/2 in the graded case and 11 in the uniform case, and the ratio of sample sizes is Λ(g) (g), independently of ε and of ‖w∗‖2\|w^*\|_2. (i) Substitute tj=gj2>0t_j=g_j^2>0, so that Λ=(∑jαjtj−1)(∑jτjtj) = ( _j _jt_j^-1 ) ( _j _jt_j ). By Cauchy–Schwarz applied to the vectors with entries αj/tj _j/t_j and τjtj _jt_j, (∑j=1dαjτj)2=(∑j=1dαjtj⋅τjtj)2≤(∑j=1dαjtj)(∑j=1dτjtj)=Λ, ( _j=1^d _j _j )^2= ( _j=1^d _jt_j· _jt_j )^2≤ ( _j=1^d _jt_j ) ( _j=1^d _jt_j )= , so BC(α,τ)2BC(α,τ)^2 is a lower bound. Equality in Cauchy–Schwarz holds exactly when the two vectors are proportional, that is αj/tj=κτjtj _j/t_j=κ\, _jt_j for some κ>0κ>0 and all j, giving tj=κ−1/2(αj/τj)1/2t_j=κ^-1/2( _j/ _j)^1/2 and hence gj=tj1/2∝αj1/4τj−1/4g_j=t_j^1/2 _j^1/4 _j^-1/4. This g lies in ℝ>0dR_>0^d because αj _j and τj _j are positive by Def. 2, so the bound is attained and Eq. 14 follows. (iv) Cauchy–Schwarz again, now on αj _j against τj _j, gives ∑jαjτj≤(∑jαj)1/2(∑jτj)1/2=1, _j _j _j≤ ( _j _j )^1/2 ( _j _j )^1/2=1, so Λ⋆≤1 ≤ 1. Equality requires αj=cτj _j=c _j for all j; squaring and summing forces c=1c=1, hence α=τα=τ. Conversely α=τα=τ gives ∑jαjτj=∑jαj=1. _j _j _j= _j _j=1. ∎ Remark 5. Three points fix the standing of Prop. 7(i). First, the two bounds are produced by the identical argument, the same symmetrisation, the same contraction, the same evaluation at the representation budget, with the grading entering only through the data radius and the budget. The ratio Λ(g) (g) therefore compares like with like, and is not an artefact of one bound being derived more loosely than the other. Second, the confidence term O(log(1/δ)/m)O( (1/δ)/m) of the excess-risk bound is class-independent and common to both sides, so at any fixed δ it cancels from the comparison of sufficient sample sizes to leading order; the ratio statement concerns the complexity-dominated regime, which is the budget-active regime made precise in Rem. 12. Third, at its representation budget each class contains f∗f^* on the boundary of its constraint set, so empirical risk minimisation over either class is well posed and the bound of [bartlett2002rademacher] applies to both without modification. Remark 6. Prop. 7 identifies the entire benefit of grading with a single scalar: the affinity between where the target places its weight and where the data places its variance. Grading buys nothing when the two profiles agree, and its value grows as they separate. In this exact sense grading does not create information. It converts a correct prior about the target into a reduction in sample complexity at the computable rate Eq. 14, and an incorrect prior at the same rate in the opposite direction: any g with Λ(g)>1 (g)>1 is admissible and costs data. Assumption 1 is exactly the assertion that these profiles diverge for language. Deep structural features are decisive for the target and rare in the corpus, with high αj _j and low τj _j, while lexical features carry the bulk of the variance and comparatively little of the decision. An isotropic architecture, which by construction sets g=g=1 and Λ=1 =1, forgoes the gain Eq. 14 whatever its size; Prop. 8 locates it precisely, on the boundary of the set of grades at which the graded bound is the smaller one. Both profiles can be estimated from an annotated corpus without training a model, so Assumption 1 is testable in advance of any experiment; Sec. 8.2 includes this measurement in the program. Proposition 8. Let α,τ∈Δd−1α,τ∈ ^d-1 be the profiles of Def. 2, related to the grades by gj=λqjg_j=λ^q_j, and write Λ(q)=(∑j=1dαjλ−2qj)(∑j=1dτjλ2qj) (q)= ( _j=1^d _jλ^-2q_j ) ( _j=1^d _jλ^2q_j ) for the gain of Prop. 7 expressed in the grades. Define the admissible set +=q∈ℝd:Λ(q)<1,Q_+= \q ^d\;:\; (q)<1 \, the grades at which the graded bound is strictly smaller than the uniform one. Then: (i) (Gauge invariance) Λ(q+c)=Λ(q) (q+c1)= (q) for every c∈ℝc . Hence +Q_+ is invariant under translation along 1, and +≠∅Q_+≠ implies +∩ℚ≥0d≠∅Q_+ _≥ 0^d≠ : restricting the grades to non-negative rationals, as Sec. 3 does throughout, costs nothing. (i) (Convexity) q↦logΛ(q)q (q) is convex on ℝdR^d, and +Q_+ is open and convex. (i) (Nonemptiness, and the position of the standard transformer) Λ()=1 (0)=1, and +≠∅Q_+≠ if and only if α≠τα≠τ. When α≠τα≠τ, the uniform grade q=q=0 — which by Remarks 1 and 3 is the standard transformer — lies on ∂+ _+. (iv) (First-order criterion) ∇qlogΛ|q==2(logλ)(τ−α) _q |_q=0=2( λ)\,(τ-α), and for v∈ℝdv ^d the ray εv:ε>0\ v: >0\ meets +Q_+ if and only if (15) ⟨v,α−τ⟩>0. v,α-τ >0. In that case ε>0:εv∈+\ >0: v _+\ is an interval (0,εmax(v))(0, _ (v)). (v) (Gain along an admissible direction) Let v satisfy Eq. 15, and set D=⟨v,α−τ⟩D= v,α-τ and S=Varα(v)+Varτ(v)S=Var_α(v)+Var_τ(v), where Varp(v)=∑jpjvj2−(∑jpjvj)2Var_p(v)= _jp_jv_j^2- ( _jp_jv_j )^2. Then S>0S>0 and logΛ(εv)=−2(logλ)Dε+ 2(logλ)2Sε2+O(ε3), ( v)=-2( λ)\,D\, \;+\;2( λ)^2S\, ^2\;+\;O( ^3), so the quadratic truncation is minimised at ε⋆=D/(2Slogλ) =D/(2S λ), with value −D2/(2S)-D^2/(2S) independent of λ. (vi) (Global optimum) minq∈ℝdΛ(q)=BC(α,τ)2 _q ^d (q)=BC(α,τ)^2, attained exactly on the gauge orbit of the q⋆q of Prop. 7(i). Proof. Set uj=2qjlogλu_j=2q_j λ, an invertible linear change of variable since λ>1λ>1, and write A(u)=∑j=1dαje−uj,T(u)=∑j=1dτjeuj,Φ(u)=logA(u)+logT(u)=logΛ.A(u)= _j=1^d _je^-u_j, T(u)= _j=1^d _je^u_j, (u)= A(u)+ T(u)= . By Def. 2 both α and τ lie in Δd−1 ^d-1 and are strictly positive. For u∈ℝdu ^d define the tilted vectors αj(u)=αje−ujA(u),τj(u)=τjeujT(u),α^(u)_j= _je^-u_jA(u), τ^(u)_j= _je^u_jT(u), both in Δd−1 ^d-1, and for p∈Δd−1p∈ ^d-1 write C(p)=diag(p)−pp⊤C(p)=diag(p)-p , so that v⊤C(p)v=Varp(v)≥0v C(p)\,v=Var_p(v)≥ 0. Differentiating, ∇uΦ(u)=τ(u)−α(u),∇u2Φ(u)=C(α(u))+C(τ(u)), split _u (u)&=τ^(u)-α^(u),\\ ∇^2_u (u)&=C (α^(u) )+C (τ^(u) ), split the term logA A contributing C(α(u))C(α^(u)) to the Hessian because the two sign reversals arising from u↦−u -u cancel. At u=u=0 we have α()=α^(0)=α and τ()=τ^(0)=τ. (i) Under q↦q+cq q+c1 we get u↦u+c′u u+c 1 with c′=2clogλc =2c λ, so A↦e−c′A e^-c A and T↦ec′T e^c T, leaving the product ATAT fixed. For the second claim, let q∈+q _+ and set q′=q−(minjqj)q =q-( _jq_j)1, so that q′≥0q ≥ 0 and Λ(q′)=Λ(q)<1 (q )= (q)<1. Since Λ is continuous and ℚdQ^d is dense in ℝdR^d, some rational q′≥0q ≥ 0 near q′q satisfies Λ(q′)<1 (q )<1. (i) ∇u2Φ∇^2_u is a sum of two matrices of the form C(p)C(p) and is therefore positive semidefinite, so Φ is convex in u and hence in q, the map q↦uq u being linear. Then +=Φ<0Q_+=\ <0\ is a strict sublevel set of a convex function, hence convex, and open by continuity. (i) Λ()=(∑jαj)(∑jτj)=1 (0)= ( _j _j ) ( _j _j )=1. By Prop. 7(i) and (iv), minqΛ(q)=BC(α,τ)2≤1 _q (q)=BC(α,τ)^2≤ 1 with equality if and only if α=τα=τ; hence +≠∅Q_+≠ exactly when α≠τα≠τ. In that case ∉+0 _+, while by (iv) the direction v=α−τv=α-τ satisfies ⟨v,α−τ⟩=‖α−τ‖22>0, v,α-τ =\|α-τ\|_2^2>0, so every neighbourhood of 0 meets +Q_+; thus ∈∂+0∈ _+. (iv) The gradient at u=u=0 is τ−ατ-α, and ∇q=2(logλ)∇u, _q=2( λ) _u, which gives the stated formula. Put ϕ(ε)=logΛ(εv)φ( )= ( v), convex by (i), with ϕ(0)=0φ(0)=0 and ϕ′(0)=⟨v,∇qlogΛ()⟩=−2(logλ)⟨v,α−τ⟩.φ (0)= v, _q (0) =-2( λ) v,α-τ . If ⟨v,α−τ⟩>0 v,α-τ >0 then ϕ′(0)<0φ (0)<0 and ϕ(ε)<0φ( )<0 for all small ε>0 >0. If ⟨v,α−τ⟩≤0 v,α-τ ≤ 0 then ϕ′(0)≥0φ (0)≥ 0, and convexity gives ϕ(ε)≥ϕ(0)+ϕ′(0)ε≥0φ( )≥φ(0)+φ (0) ≥ 0 for every ε>0 >0, so the ray never meets +Q_+. This proves Eq. 15. The set ε>0:ϕ(ε)<0\ >0:φ( )<0\ is the intersection of an interval, by convexity of ϕφ, with (0,∞)(0,∞), and contains all small ε>0 >0; it is therefore of the form (0,εmax(v))(0, _ (v)). (v) That S>0S>0 follows from D≠0D≠ 0: since τj>0 _j>0 for every j, Varτ(v)=0Var_τ(v)=0 forces v∈ℝv 1, and then D=⟨v,α−τ⟩=0D= v,α-τ =0 because α and τ are both probability vectors. Taylor expansion of ϕφ at 0 gives ϕ(ε)=ϕ′(0)ε+12ϕ′(0)ε2+O(ε3)φ( )=φ (0) + 12φ (0) ^2+O( ^3) with ϕ′(0)=−2(logλ)Dφ (0)=-2( λ)D and ϕ′(0)=v⊤∇q2logΛ()v=(2logλ)2v⊤[C(α)+C(τ)]v=4(logλ)2S,φ (0)=v _q^2 (0)\,v=(2 λ)^2\,v [C(α)+C(τ) ]v=4( λ)^2S, which is the stated expansion. Minimising aε2+bεa ^2+b with a=2(logλ)2S>0a=2( λ)^2S>0 and b=−2(logλ)Db=-2( λ)D gives ε⋆=−b/(2a)=D/(2Slogλ) =-b/(2a)=D/(2S λ) and minimum value −b2/(4a)=−D2/(2S)-b^2/(4a)=-D^2/(2S), in which λ cancels. (vi) The value and the minimizer are Prop. 7(i), transported through gj=λqjg_j=λ^q_j; the gauge orbit is the fibre of the scaling invariance recorded in (i). ∎ Remark 7. Four consequences are worth noting. First, Eq. 15 is the condition on the weights that the framework has needed. Grading improves the bound, to first order, exactly when the grades correlate positively with α−τα-τ: high grades on coordinates where the target’s energy exceeds the data’s variance, low grades where it does not. Assumption 1 thereby becomes an inner product one can compute. Second, the standard transformer is not a local optimum of the sample-complexity landscape. It sits at Λ=1 =1 on the boundary of +Q_+ with gradient 2(logλ)(τ−α)≠02( λ)(τ-α)≠ 0 whenever α≠τα≠τ, so a strictly descending direction exists at q=q=0 and Eq. 15 exhibits it. The isotropic architecture forgoes the gain of Eq. 14 not because the gain is unavailable at its location, since the gradient there is nonzero, but because the gauge orbit of q=q=0 is the unique set of grades at which Λ=1 =1 for every pair of profiles. It is the choice that ignores the profiles. Third, only the sign of ⟨q,α^−τ^⟩ q, α- τ need survive estimation error, not the magnitudes of α α and τ τ. This is what makes the coarse part-of-speech initialisation of Sec. 8.1.2 defensible: it does not attempt to locate q⋆q , only to enter +Q_+. Fourth, by (i) the offline selection of grades from estimated profiles is the minimisation of a convex function with no spurious stationary points, and by (i) the non-negativity and rationality demanded of the grades throughout this paper are free. The selection problem, on the full space and on the clipped region the configurations admit, is solved in Cor. 4 and Prop. 10 respectively. Λ is a ratio of upper bounds, and Prop. 8 characterises when the graded bound beats the uniform bound. In the level-stratified regime of Cor. 3 the comparison is a separation between the graded prior and its absence: Thm. 4.1 establishes the two-sided minimax ratio there, over all estimators. For general profiles the corresponding statement is Conj. 2, and Rem. 11 records the scope. Corollary 3. Suppose both profiles are uniform within levels: αj=al/dl _j=a_l/d_l and τj=sl/dl _j=s_l/d_l for j∈Vlj∈ V_l, with a,s∈ΔL−1a,s∈ ^L-1. Then Λ⋆=BC(a,s)2=(∑l=0L−1alsl)2, =BC(a,s)^2= ( _l=0^L-1 a_ls_l )^2, independently of the allocation dl\d_l\, and the optimal grade is constant on each level with ql⋆logλ=14logal−14logsl+const.q_l \, λ= 14 a_l- 14 s_l+const. If moreover the target concentrates geometrically towards the deepest level and the data decays geometrically away from the shallowest, al∝νL−1−la_l ν^L-1-l and sl∝μls_l μ^l with 0<μ<ν<10<μ<ν<1, then ql⋆q_l is affine in l with positive slope, and Λ⋆=Θ(νL)(L→∞). = \! (ν^L ) (L→∞). Proof. For the first claim, ∑j∈Vlαjτj=dl⋅(al/dl)(sl/dl)=alsl _j∈ V_l _j _j=d_l· (a_l/d_l)(s_l/d_l)= a_ls_l, and summing over l gives BC(α,τ)=BC(a,s)BC(α,τ)=BC(a,s); the dld_l cancel. The grade formula is loggj⋆=14logαj−14logτj+const g_j = 14 _j- 14 _j+const from Prop. 7(i), in which the logdl d_l terms cancel between αj _j and τj _j. For the second, write Za=∑l=0L−1νL−1−l=(1−νL)/(1−ν)Z_a= _l=0^L-1ν^L-1-l=(1-ν^L)/(1-ν) and Zs=(1−μL)/(1−μ)Z_s=(1-μ^L)/(1-μ). Then logal−logsl=−l(logν+logμ)+const a_l- s_l=-l( ν+ μ)+const, which is affine in l with positive coefficient since logν,logμ<0 ν, μ<0; the grade claim follows. Setting r=μ/ν∈(0,1)r= μ/ν∈(0,1), BC(a,s)=1ZaZs∑l=0L−1ν(L−1−l)/2μl/2=ν(L−1)/2ZaZs⋅1−rL1−r.BC(a,s)= 1 Z_aZ_s _l=0^L-1ν^(L-1-l)/2μ^l/2= ν^(L-1)/2 Z_aZ_s· 1-r^L1-r. As L→∞L→∞ we have Za→(1−ν)−1Z_a→(1-ν)^-1, Zs→(1−μ)−1Z_s→(1-μ)^-1 and rL→0r^L→ 0, so BC(a,s)∼ν(L−1)/2⋅(1−ν)(1−μ)1−μ/ν,BC(a,s) ν^(L-1)/2· (1-ν)(1-μ)1- μ/ν, and squaring gives Λ⋆=Θ(νL) = (ν^L). ∎ Remark 8. Cor. 3 is an unconstrained statement: the optimal grade is affine in l, so its dynamic range grows linearly in L, and realising Λ⋆=Θ(νL) = (ν^L) requires grades that exceed any fixed clip once L is large. At the clipped configurations of Sec. 8.1.1 the attainable gain is bounded by C2C^2 (Prop. 10), and the exponential regime is reached as the clip opens (Rem. 18). The corollary quantifies that trajectory; it does not describe the planned configurations. Remark 9. Under the hypotheses of Cor. 3 the criterion Eq. 15 reduces to a condition on L numbers rather than d: a level-constant grade vj=vlv_j=v_l for j∈Vlj∈ V_l is admissible exactly when ∑l=0L−1vl(al−sl)>0 _l=0^L-1v_l(a_l-s_l)>0. Under the geometric profiles this holds for every v affine and increasing in l, so the linguistic ordering itself, with deeper levels receiving higher grades, is enough to enter +Q_+ without any estimate of ν or μ. Lemma 2. Let α^,τ^∈Δd−1 α, τ∈ ^d-1 be strictly positive estimates of the profiles of Def. 2, and let g g be the plug-in optimal grades g^j∝α^j1/4τ^j−1/4 g_j α_j^1/4\, τ_j^-1/4 of Prop. 7(i). Set δj=logα^jαj−logτ^jτj,δ¯=1d∑j=1dδj. _j= α_j _j- τ_j _j, δ= 1d _j=1^d _j. Then logΛ(g^)≤logBC(α,τ)2+14∑j=1d(δj−δ¯)2. ( g)\;≤\; (α,τ)^2\;+\; 14 _j=1^d ( _j- δ )^2. The attained gain is therefore insensitive to first order to estimation error in the profiles: the degradation is quadratic in the centred log-errors, and vanishes at rate the square of the profile error rather than the profile error itself. Proof. Work in the variables uj=2qjlogλ=2loggju_j=2q_j λ=2 g_j of Prop. 8 and write Φ(u)=logΛ (u)= . The optimal grades of Prop. 7(i) correspond to uj⋆=12log(αj/τj)+c⋆u _j= 12 ( _j/ _j)+c , and at u⋆u the tilted profiles of Prop. 8 coincide, αj(u⋆)=αjτjBC(α,τ)=τj(u⋆),α^(u )_j= _j _jBC(α,τ)=τ^(u )_j, so ∇uΦ(u⋆)=τ(u⋆)−α(u⋆)=0 _u (u )=τ^(u )-α^(u )=0: the optimum is a stationary point. For every u the Hessian satisfies ∇u2Φ(u)=C(α(u))+C(τ(u))⪯2I,∇^2_u (u)=C (α^(u) )+C (τ^(u) ) 2I, since for any p∈Δd−1p∈ ^d-1 one has C(p)=diag(p)−pp⊤⪯diag(p)⪯IC(p)=diag(p)-p (p) I. Taylor expansion with integral remainder along the segment from u⋆u to u u, using the vanishing gradient and the uniform Hessian bound, gives Φ(u^)≤Φ(u⋆)+‖u^−u⋆‖22. ( u)\;≤\; (u )+\| u-u \|_2^2. The plug-in grades correspond to u^j=12log(α^j/τ^j)+c u_j= 12 ( α_j/ τ_j)+ c, so u^j−uj⋆=12δj+const u_j-u _j= 12 _j+const, and by the gauge invariance of Prop. 8(i) the representative u⋆u on the optimal orbit may be chosen to minimise the distance, giving minc∈ℝ‖12δ+c‖22=14∑j=1d(δj−δ¯)2. _c \| 12δ+c1 \|_2^2= 14 _j=1^d ( _j- δ )^2. Since Φ(u⋆)=logBC(α,τ)2 (u )= (α,τ)^2 by Prop. 8(vi), the claim follows. ∎ Remark 10. The program estimates the profiles on an annotated corpus disjoint from the pretraining sample, and this disjointness is what closes the adaptivity question in the present setting. Conditional on g g, the grades are a fixed deterministic quantity with respect to the training sample, so Prop. 7(i) applies verbatim with Λ(g^) ( g) in place of Λ(g) (g) and no selection term arises. The whole cost of not knowing α is therefore the degradation Lem. 2 bounds, and that degradation is quadratic in the profile error because the gradient of the gain vanishes at the optimum: the framework is protected at exactly the point where protection is needed. Certification asks for less still, since by Rem. 7 only the sign of one inner product must survive the estimation. Remark 11. Four limitations bound what Prop. 7 establishes; two of them are resolved in this manuscript, one of them in the regime the program occupies, and the remaining ones name the tool their repair would require. First, the analysis is for the graded embedding composed with a linear read-out. The multi-layer statement requires propagating the argument through a layerwise Lipschitz analysis of the graded stack, and the resulting bound compounds across depth; the extension is open. Second, both sides of (i) are upper bounds, and a ratio of upper bounds becomes a separation only when a matching minimax lower bound over the uniform target class meets it. Thm. 4.1 supplies that bound in the level-stratified regime of Cor. 3: at squared loss and over all estimators, the ratio of minimax risks is Θμ,ν(Λ(g)) _μ,ν( (g)) throughout the window of Lem. 4, so in the regime the planned configurations occupy the separation between the graded prior and its absence is established. The ellipsoidal constraint geometry of Prop. 5 is the packing set on which that bound is posed. What remains open is the general-profile, Lipschitz-loss case, stated as Conj. 2 in the regime fixed by Rem. 12. Third, g⋆g depends on α, which is unknown. Lem. 2 and Rem. 10 resolve this for the present setting: profiles estimated on data disjoint from the training sample incur no adaptivity cost beyond the quadratic degradation of the lemma, which vanishes to first order at the optimum. What remains open is only the fully adaptive variant in which grades and read-out are fit on the same sample. Fourth, Cor. 3 assumes geometric profiles. That assumption is an empirical refinement of Assumption 1, not a theorem, and it is the assumption on which every exponential-in-L statement in this work rests; Rem. 8 bounds what it delivers at fixed clip. Conjecture 2. Let x∼N(0,diag(σ12,…,σd2))x N (0,diag( _1^2,…, _d^2) ) and y=f∗(x)+ξy=f^*(x)+ξ with ξ∼N(0,σξ2)ξ N(0, _ξ^2) independent of x, in the setting of Def. 2, and let ℓ be a loss that is LℓL_ -Lipschitz in its first argument, as in Prop. 7; write ℰ(f^,f∗)=[ℓ(f^(x),y)]−[ℓ(f∗(x),y)]E( f,f^*)=E [ ( f(x),y) ]-E [ (f^*(x),y) ] for the excess risk. Fix a target f∗f^* with profile α and let g⋆g be the optimal grades of Prop. 7(i). Define the target classes carrying no prior and the graded prior respectively, ℱunif=⟨w,⋅⟩:‖w‖2≤Bunif(f∗),ℱgraded=⟨w,⋅⟩:‖G−1w‖2≤Bgraded(f∗), splitF^unif&= \ w,· \;:\;\|w\|_2≤ B^unif(f^*) \,\\ F^graded&= \ w,· \;:\; \|G^-1w \|_2≤ B^graded(f^*) \, split with G at grades g⋆g and both budgets calibrated at f∗f^* as in Def. 3, and the minimax excess risk of a target class ℱF at sample size m, over all estimators, ℛm(ℱ)=inff^supf∗∈ℱ[ℰ(f^,f∗)],mℱ(ε)=minm:ℛm(ℱ)≤ε.R_m(F)= _ f\, _f^* \,E [E( f,f^*) ], m^F( )= \m\;:\;R_m(F)≤ \. Then there exist universal constants 0<c1≤c20<c_1≤ c_2 such that, for every ε in the budget-active window of Rem. 12, c1⋅BC(α,τ)2≤mℱgraded(ε)mℱunif(ε)≤c2⋅BC(α,τ)2,c_1·BC(α,τ)^2\;≤\; m^F^graded( )m^F^unif( )\;≤\;c_2·BC(α,τ)^2, so the gain Eq. 14 is attained up to universal constants and is a genuine separation between the graded prior and its absence, not an artefact of the upper bounds of Prop. 7. Conj. 2 is stated for general profiles and Lipschitz losses. In the level-stratified regime of Cor. 3, at squared loss, it is a theorem: Thm. 4.1 establishes the two-sided ratio, in risk form at each sample size of the window of Lem. 4, and that regime is the one every planned configuration of Sec. 8.1.1 occupies. What the conjecture asserts beyond the theorem is the general case, and the remark below fixes the regime in which it can hold. Remark 12. Two features of the statement are forced, and each replaces a formulation that cannot hold. First, the infimum runs over all estimators and the two classes enter as classes of targets: by Remarks 1 and 3 the graded and uniform hypothesis classes coincide as sets of functions, so a comparison that restricted the estimator would compare a quantity with itself. What differs between the two sides is what is known about the target in advance, which is exactly the content of a prior, and the conjecture quantifies that prior. Second, the comparison cannot be asymptotic in ε→0 → 0 at fixed d: below the parametric floor of order σξ2d/m _ξ^2d/m the norm constraint ceases to bind, both minimax problems reduce to the same unconstrained d-dimensional one, and the ratio tends to 11 for every pair of classes with nonempty interior. Each class has a budget-active interval of ε on which its minimax risk exceeds its parametric floor by a fixed factor, equivalently sample sizes below the saturation scale at which its constraint stops binding, and the budget-active window of the conjecture is the intersection of the two intervals, since the two-sided ratio requires both constraints to bind. Only in that window is the geometry of Prop. 5 visible in the rates. In the stratified regime the window is no longer a description but a set: Lem. 4 exhibits it as W, proves it nonempty at d≥3L/σξd≥ 3L/ _ξ, and proves that in the joint scaling d,m→∞d,m→∞ it widens rather than closes. For general profiles the same two requirements, nonemptiness under Assumption 1 and widening under the joint scaling, are the conditions the conjecture’s window must satisfy. The route Thm. 4.1 takes is the one the general case must extend. The lower bound is an Assouad packing of sign patterns with the amplitude allocation chosen to hold the per-coordinate testing error constant, run in the anisotropic Gaussian design directly and converted into the bound by Fano’s inequality (Lem. 3); the window on which both budgets bind is exhibited and sized by Lem. 4; and the upper bound in the matching form is carried by the envelope rather than imported from Prop. 7(i). Extending the argument beyond level-homogeneous designs and squared loss is what the general case requires, and Sec. 8.4 records it as the remaining theoretical target alongside the layerwise extension. 4.3. The Minimax Envelope Throughout this subsection the design is Gaussian and level-homogeneous: x∼N(0,Σ)x N(0, ) with Σ=diag(σ12,…,σd2) =diag( _1^2,…, _d^2) constant on levels, σj2=σ(l)2 _j^2= _(l)^2 for j∈Vlj∈ V_l and dimVl=dl V_l=d_l, and y=⟨w∗,x⟩+ξy= w^*,x +ξ with ξ∼N(0,σξ2)ξ N(0, _ξ^2) independent of x. The loss is squared, so the excess risk of a predictor f f at the target f∗=⟨w∗,⋅⟩f^*= w^*,· is ‖f^−f∗‖L2(Px)2\| f-f^*\|_L^2(P_x)^2, and for a class ℱF of targets the minimax excess risk at sample size m is ℛm(ℱ)=inff^supf∗∈ℱ‖f^−f∗‖L2(Px)2R_m(F)= _ f _f^* E\,\| f-f^*\|_L^2(P_x)^2, the infimum over all estimators, as in Conj. 2. For a diagonal matrix D=diag(DlIdl)D=diag(D_lI_d_l) with Dl>0D_l>0 and a budget B>0B>0 set ℱ(B,D)=⟨w,⋅⟩:w∈ℝd,‖Dw‖2≤B,capl=σ(l)2B2Dl2,ε2=σξ2m.F(B,D)= \ w,· :w ^d,\ \|Dw\|_2≤ B \, _l= _(l)^2B^2D_l^2, ^2= _ξ^2m. The choice D=IdD=I_d, B=Bunif(f∗)B=B^unif(f^*) realises the uniform target class, and Dl=gl−1D_l=g_l^-1, B=Bgraded(f∗)B=B^graded(f^*) the graded one; caplcap_l is the prediction-norm energy the budget can place at level l if spent there entirely, and the two classes differ only through their caps. Lemma 3. Let S=l:dlε2≤caplS=\l:d_l ^2 _l\. There exist constants 0<c(L)≤C(L)0<c(L)≤ C(L) depending only on L such that: (i) for every m≥1m≥ 1, ℛm(ℱ(B,D))≥c(L)∑l=0L−1mincapl,dlε2;R_m (F(B,D) )\ ≥\ c(L) _l=0^L-1 \cap_l,\ d_l ^2 \; (i) for every m≥2∑l∈Sdl+4m≥ 2 _l∈ Sd_l+4, ℛm(ℱ(B,D))≤C(L)∑l=0L−1mincapl,dlε2.R_m (F(B,D) )\ ≤\ C(L) _l=0^L-1 \cap_l,\ d_l ^2 \. Proof. Write μl=mincapl,dlε2 _l= \cap_l,d_l ^2\ and M=∑lμlM= _l _l. (i) Fix c0=(log2)/64c_0=( 2)/64 and set, for each level, kl=mindl,⌊capl/(c0Lε2)⌋k_l= \d_l,\ _l/(c_0L ^2) \ and δl2=c0ε2/σ(l)2 _l^2=c_0 ^2/ _(l)^2, and let ⊆ℝdW ^d consist of the vectors supported on the first klk_l coordinates of each VlV_l with entries ±δl± _l. Every w∈w is feasible: ‖Dw‖22=∑lklDl2δl2=c0ε2∑lklB2capl≤B2L∑l1=B2.\|Dw\|_2^2= _lk_lD_l^2 _l^2=c_0 ^2 _lk_l B^2cap_l≤ B^2L _l1=B^2. For w,w′∈w,w with Hamming distance ρ(w,w′)ρ(w,w ) between their sign patterns, the level-homogeneity of Σ and the constant-testing-error allocation σ(l)2δl2=c0ε2 _(l)^2 _l^2=c_0 ^2 give ‖Σ1/2(w−w′)‖22=∑lσ(l)2(2δl)2ρl(w,w′)=4c0ε2ρ(w,w′),KL(Pw⊗m∥Pw′⊗m)=m2σξ2‖Σ1/2(w−w′)‖22≤2c0K,K:=∑lkl, split\| ^1/2(w-w )\|_2^2&= _l _(l)^2\,(2 _l)^2\, _l(w,w )=4c_0 ^2\,ρ(w,w ),\\ KL (P_w m\, \|\,P_w m )&= m2 _ξ^2\,\| ^1/2(w-w )\|_2^2≤ 2c_0K, K:= _lk_l, split so the weighted packing problem collapses to an unweighted one on ±1K\± 1\^K. Suppose first K≥32K≥ 32. By the Varshamov–Gilbert bound there is ′⊆W with log|′|≥(K/8)log2 |W |≥(K/8) 2 and pairwise Hamming distance at least K/8K/8, hence pairwise L2(Px)L^2(P_x)-separation at least c0Kε2/2c_0K ^2/2. Fano’s inequality then yields ℛm≥c0Kε28(1−2c0K+log2(K/8)log2)≥c016Kε2,R_m\ ≥\ c_0K ^28 (1- 2c_0K+ 2(K/8) 2 )\ ≥\ c_016\,K ^2, the last step by 2c0K=(K/32)log22c_0K=(K/32) 2 and log2≤(K/32)log2 2≤(K/32) 2 at K≥32K≥ 32. When capl≥c0Lε2cap_l≥ c_0L ^2 the floor in klk_l is at least half its argument, so klε2≥12mindlε2,capl/(c0L)≥μl/(2max1,c0L);k_l ^2≥ 12 \d_l ^2,\ cap_l/(c_0L) \≥ _l/(2 \1,c_0L\); when capl<c0Lε2cap_l<c_0L ^2 the level contributes μl≤capl<c0Lε2≤(c0L/32)Kε2 _l _l<c_0L ^2≤(c_0L/32)\,K ^2. Summing over the two kinds of level, M≤(2max1,c0L+c0L2/32)Kε2=:C1(L)Kε2,M≤ (2 \1,c_0L\+c_0L^2/32 )K ^2=:C_1(L)\,K ^2, so ℛm≥(c0/(16C1(L)))MR_m≥ (c_0/(16\,C_1(L)) )M and (i) follows in this case. If K<32K<32, then for every l either dl<32d_l<32 or capl<32c0Lε2cap_l<32c_0L ^2, so μl≤32max1,c0Lmincapl,ε2 _l≤ 32 \1,c_0L\ \cap_l, ^2\ for every l and M≤C′(L)maxlmincapl,ε2M≤ C (L) _l \cap_l, ^2\ with C′(L)=32Lmax1,c0LC (L)=32L \1,c_0L\; a two-point argument with w=0w=0 against w=δejw=δ e_j for j∈Vl∗j∈ V_l^* at the maximising level, with σ(l∗)2δ2=mincapl∗,c0ε2 _(l^*)^2δ^2= \cap_l^*,c_0 ^2\, is feasible, has KL≤c0/2KL≤ c_0/2 over the m samples, and by Le Cam’s two-point bound gives ℛm≥c5mincapl∗,c0ε2≥c5c0mincapl∗,ε2≥(c5c0/C′(L))M,R_m≥ c_5 \cap_l^*,c_0 ^2\≥ c_5c_0 \cap_l^*, ^2\≥ (c_5c_0/C (L) )M, with c5c_5 an absolute constant. (i) Let VS=⨁l∈SVlV_S= _l∈ SV_l with DS=dimVS=∑l∈SdlD_S= V_S= _l∈ Sd_l, and let f f be ordinary least squares on the coordinates of VSV_S, extended by zero on the remaining levels. Since Σ is diagonal, the omitted covariates are independent of the retained ones, so conditional on the retained design the omitted signal ⟨wSc∗,xSc⟩ w^*_S^c,x_S^c is centred noise of variance ∑l∉Sσ(l)2‖wl∗‖22 _l∉ S _(l)^2\|w^*_l\|_2^2, independent across observations, and the regression of y on xSx_S has noise variance σξ2+∑l∉Sσ(l)2‖wl∗‖22 _ξ^2+ _l∉ S _(l)^2\|w^*_l\|_2^2. The excess risk decomposes into the approximation term ∑l∉Sσ(l)2‖wl∗‖22 _l∉ S _(l)^2\|w^*_l\|_2^2 and the estimation term, which for Gaussian design and m≥2DS+4m≥ 2D_S+4 is at most 2(σξ2+∑l∉Sσ(l)2‖wl∗‖22)DS/m2 ( _ξ^2+ _l∉ S _(l)^2\|w^*_l\|_2^2 )D_S/m. The budget constraint gives σ(l)2‖wl∗‖22≤capl _(l)^2\|w^*_l\|_2^2 _l uniformly over the class, and DS/m≤1/2D_S/m≤ 1/2, so ℛm≤∑l∉Scapl+ 2DSε2+∑l∉Scapl≤ 3∑l=0L−1mincapl,dlε2,R_m\ ≤\ _l∉ Scap_l\ +\ 2\,D_S\, ^2\ +\ _l∉ Scap_l\ ≤\ 3 _l=0^L-1 \cap_l,\ d_l ^2 \, using capl=μlcap_l= _l for l∉Sl∉ S and dlε2=μld_l ^2= _l for l∈Sl∈ S. ∎ Remark 13. The lower bound of Lem. 3 holds at every sample size; the upper bound is established for m≥2∑l∈Sdl+4m≥ 2 _l∈ Sd_l+4, the regime in which every level worth estimating is estimable. The band m≍∑l∈Sdlm _l∈ Sd_l, in which a level crosses its estimability threshold and its worth-estimating threshold at comparable sample sizes, is not covered by the argument above, and the window on which the separation of Thm. 4.1 is established must be located outside it; Lem. 4 carries this out. Throughout the remainder of this subsection the profiles are geometric and the allocation uniform: al=νL−1−l/Zaa_l=ν^L-1-l/Z_a and sl=μl/Zss_l=μ^l/Z_s with 0<μ<ν<10<μ<ν<1 and Za=∑k=0L−1νkZ_a= _k=0^L-1ν^k, Zs=∑k=0L−1μkZ_s= _k=0^L-1μ^k; dl=d/Ld_l=d/L for every l; and the calibrations ‖w∗‖2=1\|w^*\|_2=1, trΣ=1tr =1, so that σ(l)2=(L/d)sl _(l)^2=(L/d)\,s_l. The grade vector is level-constant, g=(g0,…,gL−1)g=(g_0,…,g_L-1), and is required to satisfy 1≤g0≤g1≤⋯≤gL−1,gl+1gl≤(νμ)−1/4(0≤l≤L−2),1≤ g_0≤ g_1≤·s≤ g_L-1, g_l+1g_l≤(νμ)^-1/4 (0≤ l≤ L-2), a condition met by the uniform grades g=g=1, by the fourth-root grades of Prop. 7(i) evaluated on the geometric profiles, and by their clipped form gC⋆g _C of Prop. 10, since clipping only decreases consecutive ratios. Under these conventions the caps of Lem. 3 are caplunif=Ldsl,caplgr=Ldslgl2∑k=0L−1akgk−2,ε2=σξ2m.cap_l^unif= Ld\,s_l, _l^gr= Ld\,s_l\,g_l^2 _k=0^L-1a_kg_k^-2, ^2= _ξ^2m. Lemma 4. Set r=μ/ν∈(0,1)r= μ/ν∈(0,1), msat=d0σξ2cap0unif=d2σξ2L2s0,W=m∈ℕ:4≤m≤12msat.m_sat= d_0\, _ξ^2cap_0^unif= d^2\, _ξ^2L^2\,s_0, W= \m :4≤ m≤ 12\,m_sat \. Then there is an absolute constant c4∈(0,1)c_4∈(0,1) such that: (i) (Non-emptiness and width.) W≠∅W≠ whenever d≥3L/σξd≥ 3L/ _ξ, and |W|=Θ(d2σξ2/L2)|W|= (d^2 _ξ^2/L^2 ) as d/L→∞d/L→∞ at fixed (μ,ν,σξ)(μ,ν, _ξ); in the joint scaling d,m→∞d,m→∞ the window widens, as Rem. 12 requires of the regime. (i) (Full saturation.) For every m∈Wm∈ W, every level l, and both classes, capl≤12dlε2cap_l≤ 12\,d_l\, ^2; in particular the set S of Lem. 3 is empty for both classes and the regime restriction of Rem. 13 is vacuous on W. (i) (Risk pinning.) For each class and every m∈Wm∈ W, c4maxlcapl≤ℛm(ℱ(B,D))≤maxlcapl,c_4\, _lcap_l\;≤\;R_m (F(B,D) )\;≤\; _lcap_l, the upper bound attained by the zero estimator, whose worst-case risk over the ellipsoid ‖Dw‖2≤B\\|Dw\|_2≤ B\ is exactly maxlcapl _lcap_l. (iv) (Ratio.) Under the grade condition both cap sequences are geometric with ratio at most r, both maxima sit at l=0l=0, and uniformly for m∈Wm∈ W, c4(1−r)Λ(g)≤ℛm(ℱgr)ℛm(ℱunif)≤1c4(1−μ)Λ(g),c_4\,(1-r)\; (g)\;≤\; R_m (F^gr )R_m (F^unif )\;≤\; 1c_4\,(1-μ)\; (g), with Λ(g)=(∑lalgl−2)(∑lslgl2) (g)= ( _la_lg_l^-2 ) ( _ls_lg_l^2 ) the gain of Prop. 7. The constants depend only on (μ,ν)(μ,ν) and are uniform in L, d, and m∈Wm∈ W. At the fourth-root grades the ratio is Θμ,ν(BC(a,s)2)=Θμ,ν(νL) _μ,ν (BC(a,s)^2 )= _μ,ν(ν^L), and at gC⋆g _C it is Θμ,ν(ΛC⋆) _μ,ν( _C). (v) (Parametric floor.) For m≥max2d+4,d0σξ2/minlcaplm≥ \2d+4,\;d_0 _ξ^2/ _lcap_l \ the set S is full for both classes, both risks are ΘL(dε2) _L(d\, ^2) by Lem. 3, and the ratio is ΘL(1) _L(1): the graded gain concentrates on the window W, where the budget is active. On the intermediate band (12msat,d0σξ2/minlcapl) ( 12m_sat,\,d_0 _ξ^2/ _lcap_l ) no two-sided comparison against Λ(g) (g) is asserted. Proof. Since g is nondecreasing, g02∑kakgk−2=∑kak(g0/gk)2≤∑kak=1g_0^2 _ka_kg_k^-2= _ka_k(g_0/g_k)^2≤ _ka_k=1, so maxlcaplgr=cap0gr=Lds0g02∑kakgk−2≤Lds0=cap0unif, _lcap_l^gr\;=\;cap_0^gr\;=\; Ld\,s_0\,g_0^2 _ka_kg_k^-2\;≤\; Ld\,s_0\;=\;cap_0^unif, where the first equality holds because sl+1gl+12/(slgl2)=μ(gl+1/gl)2≤μ(νμ)−1/2=r<1s_l+1g_l+1^2/(s_lg_l^2)=μ\,(g_l+1/g_l)^2≤μ\,(νμ)^-1/2=r<1: both cap sequences are geometric with ratio at most r (ratio μ≤rμ≤ r in the uniform case) and are maximised at l=0l=0. (i) For m≤12msatm≤ 12m_sat we have cap0unif≤12d0ε2cap_0^unif≤ 12d_0 ^2 by the definition of msatm_sat, and every other cap of either class is at most cap0unifcap_0^unif by the display above, while dl=d0d_l=d_0 for every l. (i) W≠∅W≠ requires msat≥8m_sat≥ 8, that is d2≥8L2s0/σξ2d^2≥ 8L^2s_0/ _ξ^2, and s0=(1−μ)/(1−μL)≤1s_0=(1-μ)/(1-μ^L)≤ 1 makes d≥3L/σξd≥ 3L/ _ξ sufficient. The width claim is immediate from the formula for msatm_sat. (i) Upper bound: the zero estimator has excess risk ‖Σ1/2w‖22=∑jσj2wj2\| ^1/2w\|_2^2= _j _j^2w_j^2, whose supremum over ‖Dw‖2≤B\\|Dw\|_2≤ B\ is maxjσj2B2/Dj2=maxlcapl _j _j^2B^2/D_j^2= _lcap_l, attained on the maximising coordinate. Lower bound: run the packing of the proof of Lem. 3(i) on the single level l=0l=0, with k0=mind0,⌊cap0/(c0ε2)⌋k_0= \d_0,\ _0/(c_0 ^2) \ and σ(0)2δ02=c0ε2 _(0)^2 _0^2=c_0 ^2; feasibility uses the full budget at one level, k0D02δ02=c0ε2k0B2/cap0≤B2k_0D_0^2 _0^2=c_0 ^2k_0B^2/cap_0≤ B^2. If k0≥32k_0≥ 32, then either k0=d0k_0=d_0, in which case k0ε2=d0ε2≥2cap0k_0 ^2=d_0 ^2≥ 2cap_0 by (i), or k0=⌊cap0/(c0ε2)⌋≥32k_0= _0/(c_0 ^2) ≥ 32, in which case cap0≥32c0ε2cap_0≥ 32\,c_0 ^2 and k0≥cap0/(2c0ε2)k_0 _0/(2c_0 ^2), so k0ε2≥cap0/(2c0)≥2cap0k_0 ^2 _0/(2c_0)≥ 2cap_0; in either case Fano gives ℛm≥(c0/16)k0ε2≥(c0/8)cap0R_m≥(c_0/16)\,k_0 ^2≥(c_0/8)\,cap_0. If k0<32k_0<32, then either cap0<32c0ε2cap_0<32\,c_0 ^2 or d0<32d_0<32, and in the latter case cap0≤12d0ε2<16ε2cap_0≤ 12d_0 ^2<16\, ^2 by (i); in both cases mincap0,c0ε2≥(c0/32)cap0 \cap_0,c_0 ^2\≥(c_0/32)\,cap_0, and the two-point argument of Lem. 3(i) at level 0 with amplitude σ(0)2δ2=mincap0,c0ε2 _(0)^2δ^2= \cap_0,c_0 ^2\ gives ℛm≥c5mincap0,c0ε2≥(c5c0/32)cap0R_m≥ c_5 \cap_0,c_0 ^2\≥(c_5c_0/32)\,cap_0. Take c4c_4 the smaller of the two constants. (iv) Dividing the two-sided bounds of (i), c4cap0grcap0unif≤ℛm(ℱgr)ℛm(ℱunif)≤1c4cap0grcap0unif,cap0grcap0unif=g02∑kakgk−2.c_4\, cap_0^grcap_0^unif\;≤\; R_m(F^gr)R_m(F^unif)\;≤\; 1c_4\, cap_0^grcap_0^unif, cap_0^grcap_0^unif=g_0^2 _ka_kg_k^-2. It remains to compare g02∑kakgk−2g_0^2 _ka_kg_k^-2 with Λ(g) (g). Since ∑lsl=1 _ls_l=1, g02∑kakgk−2=Λ(g)⋅g02∑lslgl2,s0g02≤∑lslgl2≤s0g021−r, splitg_0^2 _ka_kg_k^-2\;&=\; (g)· g_0^2 _ls_lg_l^2,\\ s_0\,g_0^2\;≤\; _ls_lg_l^2\;≤\; s_0\,g_0^21-r, split the second line because the summands slgl2s_lg_l^2 are geometric with ratio at most r. Hence g02/∑lslgl2g_0^2/ _ls_lg_l^2 lies in [ 1−r, 1/s0]⊆[ 1−r, 1/(1−μ)][\,1-r,\;1/s_0\,] [\,1-r,\;1/(1-μ)\,], which gives the display. For the fourth-root grades, Cor. 3 gives Λ(g⋆)=BC(a,s)2=Θ(νL) (g )=BC(a,s)^2= (ν^L), and Λ(gC⋆)=ΛC⋆ (g _C)= _C by Prop. 10. (v) For such m every level of both classes satisfies dlε2≤capld_l ^2 _l, so S is full, the envelope of Lem. 3 evaluates to ΘL(∑ldlε2)=ΘL(dε2) _L( _ld_l ^2)= _L(d\, ^2) for both classes, and the ratio is bounded above and below by C(L)/c(L)C(L)/c(L) and its inverse: past the saturation point the budget is inactive for both classes and the comparison localises to W. The band assertion is a scope statement, not a theorem, and records where the separation is to be read off. ∎ Theorem 4.1. Assume the setting of this subsection: Gaussian level-homogeneous design with squared loss, geometric profiles al=νL−1−l/Zaa_l=ν^L-1-l/Z_a and sl=μl/Zss_l=μ^l/Z_s with 0<μ<ν<10<μ<ν<1, uniform allocation dl=d/Ld_l=d/L, the calibrations ‖w∗‖2=1\|w^*\|_2=1 and trΣ=1tr =1, and a level-constant grade vector g satisfying the monotonicity and ratio condition stated before Lem. 4. Suppose d≥3L/σξd≥ 3L/ _ξ, so that the window W of Lem. 4 is nonempty. Then there exist constants 0<cμ,ν≤Cμ,ν0<c_μ,ν≤ C_μ,ν depending only on (μ,ν)(μ,ν), uniform in L, d, and m, such that for every m∈Wm∈ W, cμ,νΛ(g)≤ℛm(ℱgr)ℛm(ℱunif)≤Cμ,νΛ(g),c_μ,ν\, (g)\;≤\; R_m (F^gr )R_m (F^unif )\;≤\;C_μ,ν\, (g), with Λ(g) (g) the gain of Prop. 7. In particular: (i) at the fourth-root grades g⋆g of Prop. 7(i) the ratio is Θμ,ν(BC(a,s)2)=Θμ,ν(νL) _μ,ν (BC(a,s)^2 )= _μ,ν(ν^L); (i) at the clipped grades gC⋆g _C of Prop. 10 the ratio is Θμ,ν(ΛC⋆) _μ,ν ( _C ), with ΛC⋆<1 _C<1 whenever a≠sa≠ s. The gain Eq. 14 is therefore attained on W up to constants depending only on (μ,ν)(μ,ν), over all estimators, and is a separation between the graded prior and its absence rather than a comparison of the upper bounds of Prop. 7. Proof. The two-sided bound is Lem. 4(iv), with cμ,ν=c4(1−r)c_μ,ν=c_4(1-r) and Cμ,ν=(c4(1−μ))−1C_μ,ν= (c_4(1-μ) )^-1, where c4c_4 is the constant of Lem. 4(i) and r=μ/νr= μ/ν; both grade vectors named in (i) and (i) satisfy the grade condition, as recorded before Lem. 4, so the bound applies to them. The value Λ(g⋆)=BC(a,s)2=Θ(νL) (g )=BC(a,s)^2= (ν^L) is Cor. 3, and Λ(gC⋆)=ΛC⋆ (g _C)= _C is Prop. 10. For the strict inequality in (i), a≠sa≠ s gives +≠∅Q_+≠ by Prop. 8(i); by Rem. 9 a level-constant grade increasing in l satisfies the criterion of Prop. 8(iv), so εv∈+ v _+ for all small ε>0 >0 along such a direction v, and for small ε the point εv v lies in the box CQ_C as well; hence CQ_C meets +Q_+ and the minimum over it is strictly below 11. Nonemptiness of W at d≥3L/σξd≥ 3L/ _ξ is Lem. 4(i). ∎ Thm. 4.1 establishes the separation of Conj. 2 in the regime the program occupies. Level-stratified geometric profiles are the hypothesis of Cor. 3 and the setting of every planned configuration, and g⋆g and gC⋆g _C are the grades those configurations carry; the loss is squared, the curved case in which minimax risk is classically posed. The statement is a risk ratio at each sample size of the window, which is the form Rem. 12 shows to be well posed: past saturation both risks sit on the parametric floor and the ratio is ΘL(1) _L(1) by Lem. 4(v), so the theorem localises the gain to W rather than diminishing it, and W is the token-limited regime the framework is built for. What remains open is the general-profile, Lipschitz-loss case, and Conj. 2 states it. 4.4. Graded Positional Encoding Let l(j)∈0,…,L−1l(j)∈\0,…,L-1\ denote the level of global dimension j, and i(j)i(j) its local index within Vl(j)V_l(j). The graded positional encoding scales the standard sinusoidal encoding Eq. 5 by the grade of the dimension it occupies: (16) PE′(t,j)=λql(j),i(j)sin(t10000 2⌊i(j)/2⌋/dl(j)),i(j) even,cos(t10000 2⌊i(j)/2⌋/dl(j)),i(j) odd.PE (t,j)=λ^q_l(j),\,i(j) cases \! ( t10000^\,2 i(j)/2 /d_l(j) ),&i(j) even,\\[8.61108pt] \! ( t10000^\,2 i(j)/2 /d_l(j) ),&i(j) odd. cases The wavelength is computed within the level, so that each VlV_l carries a complete positional basis at its own resolution rather than a fragment of a global one. The input to the first layer is X0=E′+PE′∈ℝn×dX_0=E +PE ^n× d. 4.5. The Graded Forward Pass The stack applies K pre-norm blocks in the ordering of Sec. 2.2, with EG-MHSA in place of MHSA: (17) Xℓ′=Xℓ−1+EG-MHSAℓ(LN(Xℓ−1)),Xℓ=Xℓ′+FFNℓ(LN(Xℓ′)), splitX_ &=X_ -1+EG -MHSA_ (LN(X_ -1) ),\\ X_ &=X_ +FFN_ (LN(X_ ) ), split for ℓ=1,…,K =1,…,K, with layer normalisation as in [ba2016layer]. The feed-forward block admits a level-decomposed variant, FFNℓ(X)=∑l=0L−1GeGLUl(XPl)Pl⊤,FFN_ (X)= _l=0^L-1GeGLU_l\! (XP_l )P_l , using the gated linear unit of [shazeer2020glu]. We record what this costs. The decomposed block is block-diagonal with respect to ⨁lVl _lV_l and therefore performs no cross-level mixing; all interaction between levels must then occur in attention. This is a substantive architectural restriction and not merely a reparameterization, and it is in tension with the informal reading of the filtration V≤l=⨁l′≤lVl′V_≤ l= _l ≤ lV_l as a channel by which lower levels feed higher ones. A filtration is a nested family of subspaces; on its own it enforces no propagation. Directed level-to-level transport requires block-triangular maps ϕl′←l:Vl→Vl′ _l ← l V_l→ V_l rather than block-diagonal ones; the morphic structure of [sh-111] supplies them, and MLGE does not. At position t the final hidden state zt∈ℝdz_t ^d is mapped to vocabulary logits by Wvoc⊤zt∈ℝ||W_voc z_t ^|V|, with Wvoc∈ℝd×||W_voc ^d×|V|, and (18) p(yt∣y<t)=softmax(Wvoc⊤zt+κqout),qout∈ℚ≥0||,κ>0,p(y_t y_<t)=softmax\! (W_voc z_t+κ\,q_out ), q_out _≥ 0^|V|,\ κ>0, which multiplies the unnormalised probability of token v by eκqout,ve^κ q_out,v and is therefore monotone in the grade, uniformly in ztz_t. 5. Training Graded LLMs Training a GLLM retains the autoregressive next-token objective of classical LLMs [radford2019language] and modifies the weight each token carries in it. The standard objective is the unweighted negative log-likelihood of Eq. 3. The graded objective attaches to position t a weight λqtλ^q_t recording the structural salience of that position, reweighting the summands of the decomposition Eq. 4. This section defines the objective, establishes what it converges to, and states precisely which of the framework’s claims it can and cannot support. We state the conclusion at the outset, because the section’s structure follows from it. Grading is a statistical device, and its whole force is spent on the resource that is scarce. It restricts the effective hypothesis class in a direction aligned with hierarchical targets, and the reduction in the data required to reach a given generalisation level is a consequence of that restriction, mediated by Prop. 7 and located exactly by Prop. 8. It buys nothing in step count: exponential rescaling worsens the conditioning of the objective, and we claim no acceleration. This distinction is central to the claim. Extra optimisation steps are cheap while extra high-quality tokens are not, so a device that spends conditioning to buy sample efficiency spends a cheap resource on an expensive one; Rem. 16 prices the two sides with the same constant, the step penalty at the clip C and the token purchase at C2C^2. Sec. 8 measures the two separately for exactly that reason. 5.1. The Graded Objective and What It Minimises The base objective weights the per-token log-loss exponentially by a grade: (19) ℒgrade(y,p^;q)=−∑t=1nλqtlogp^(yt∣y<t),q=(q1,…,qn)∈ℚ≥0n,L_grade(y, p;q)=- _t=1^nλ^q_t p(y_t y_<t), q=(q_1,…,q_n) _≥ 0^n, with p p the model’s predicted conditional from Eq. 18 and λ>1λ>1 the grading base fixed in Sec. 3. Everything depends on what qtq_t is allowed to depend on, and the following proposition draws the line. Proposition 9. Let p⋆(⋅∣y<t)p_ (· y_<t) denote the true conditional and let p p range over all of Δ||−1 ^|V|-1. (1) If qt=q(y<t)q_t=q(y_<t) depends only on the context, then the population minimizer of Eq. 19 is p^⋆(⋅∣y<t)=p⋆(⋅∣y<t) p_ (· y_<t)=p_ (· y_<t). The objective is a reweighting of contexts and is consistent. (2) If qt=q(yt)q_t=q(y_t) depends only on the realised target, then the population minimizer is the exponentially tilted conditional p^⋆(v∣y<t)=λq(v)p⋆(v∣y<t)∑u∈λq(u)p⋆(u∣y<t)≠p⋆(v∣y<t), p_ (v y_<t)= λ^q(v)\,p_ (v y_<t) _u λ^q(u)\,p_ (u y_<t)\;≠\;p_ (v y_<t), and the objective is inconsistent for p⋆p_ ; the tilt is the same at every context. (3) If qt=q(yt,y<t)q_t=q(y_t,y_<t) depends on the realised target and its context — a constituency parse depth, for instance — then the population minimizer is the context-dependent tilted conditional p^⋆(v∣y<t)=λq(v,y<t)p⋆(v∣y<t)∑u∈λq(u,y<t)p⋆(u∣y<t)≠p⋆(v∣y<t), p_ (v y_<t)= λ^q(v,\,y_<t)\,p_ (v y_<t) _u λ^q(u,\,y_<t)\,p_ (u y_<t)\;≠\;p_ (v y_<t), and the objective is inconsistent for p⋆p_ ; the tilt varies with the context. Proof. For fixed context y<ty_<t, write wv=λqtw_v=λ^q_t for the weight attached to the outcome v and minimise −∑vwvp⋆(v)logp^(v)- _vw_v\,p_ (v) p(v) over the simplex. The Lagrangian stationarity condition −wvp⋆(v)/p^(v)+μ=0-w_vp_ (v)/ p(v)+μ=0 gives p^(v)=wvp⋆(v)/μ p(v)=w_vp_ (v)/μ, and μ is fixed by normalisation, yielding p^(v)=wvp⋆(v)/∑uwup⋆(u) p(v)=w_vp_ (v)/ _uw_up_ (u). In case (1) wv≡λq(y<t)w_v≡λ^q(y_<t) is constant in v and cancels. In case (2) wv=λq(v)w_v=λ^q(v) does not, and is the same function of v at every context. In case (3) wv=λq(v,y<t)w_v=λ^q(v,\,y_<t) does not cancel either, and the resulting tilt is a different function of v at each context. ∎ Remark 14. Case (2) of Prop. 9 is a constraint that the architecture can match. Write the graded softmax Eq. 18 as p^(v∣y<t)∝eκqout,vs(v∣y<t) p(v y_<t) e^κ q_out,v\,s(v y_<t), where s is the conditional realised by the ungraded head. Setting κ=lnλκ= λ and qout,v=q(v)q_out,v=q(v) makes the architectural tilt of Eq. 18 coincide with the objective tilt of Prop. 9(2), so that the population minimizer satisfies s=p⋆s=p_ : the underlying model is calibrated to the corpus, and the grading is carried entirely by the output head. Under this matching, and only under it, the graded objective and the graded architecture are compatible with the standard language-modelling objective. Case (3) admits no such matching. The tilt of Prop. 9(3) is a different function of v at each context, whereas qoutq_out in Eq. 18 is a single fixed vector, applied identically at every context; no choice of κ and qoutq_out can therefore absorb a context-dependent tilt. A grade that consults the target and its context jointly, a parse depth being the canonical example, leaves the trained model uncalibrated, with the miscalibration at each context equal to the Kullback–Leibler divergence between the corpus conditional and its tilt at that context. Prop. 9(3) rules out the parse-depth initialisation qt=depth(yt∣y<t)q_t=depth(y_t y_<t) outright: parse depth is a function of the target and its context jointly, so it falls under case (3), where by Rem. 14 no matching is available. We therefore take qt=qout,ytq_t=q_out,y_t with qoutq_out a per-type grade estimated offline, which is a function of the token alone, falls under case (2), and admits the cancellation. Part-of-speech classes supply a natural default: high grades for open-class types, low for closed-class function words. A parse-depth grading is retained in the program only as an intentionally uncalibrated ablation; by Prop. 9(3) its perplexity gap measures the tilt, not the inductive bias. 5.2. Regularisation of the Grades The trainable parameters are θ=We,Wvoc,WQh,WKh,WVhh=1H,WO,Pll=0L−1,qh,ql,qout.θ=\W_e,W_voc,\W_Q_h,W_K_h,W_V_h\_h=1^H,W_O,\P_l\_l=0^L-1,\q_h\,\q_l\,q_out\. Collect all grades into q. The full objective is (20) ℒtotal(θ)=y∼[w(y)ℒgrade(y,p^;q)]+νmag‖22+νorthΩorth(P)−νdiv∑1≤h<h′≤H‖qh−qh′‖22, splitL_total(θ)&=E_y \! [w(y)\,L_grade(y, p;q) ]+ _mag\|q\|_2^2+ _orth\, _orth(P)\\ & - _div _1≤ h<h ≤ H \|q_h-q_h \|_2^2, split with νmag,νorth,νdiv>0 _mag, _orth, _div>0, and Ωorth _orth the orthogonality penalty of Sec. 4. Three points on Eq. 20 deserve emphasis. First, the diversity term enters with a negative sign and is therefore repulsive: it rewards heads whose grade tuples differ. This is required for consistency with the framework, since the graded bilinear forms of Sec. 3.1 are head-specific only when the qhq_h differ, and Sec. 8 predicts head specialisation as an observable. An attractive penalty ∑h‖qh−q¯‖22 _h\|q_h- q\|_2^2 has its minimum at qh≡q¯q_h≡ q, at which point every head carries the identical grading and the construction degenerates to a global rescaling; such a term would penalise precisely the phenomenon the framework predicts. Second, the repulsion must be bounded, and the magnitude term νmag‖22 _mag\|q\|_2^2 supplies the cap. Writing =(q1,…,qH)q=(q_1,…,q_H) with ‖22=∑h‖qh‖22\|q\|_2^2= _h\|q_h\|_2^2, the polarisation identity ∑1≤h<h′≤H‖qh−qh′‖22=H∑h=1H‖qh‖22−‖∑h=1Hqh‖22 _1≤ h<h ≤ H \|q_h-q_h \|_2^2=H _h=1^H\|q_h\|_2^2- \| _h=1^Hq_h \|_2^2 shows that the two quadratic terms of Eq. 20 contribute together νmag‖22−νdiv∑h<h′‖qh−qh′‖22=(νmag−νdivH)‖22+νdiv‖∑hqh‖22, _mag\|q\|_2^2- _div _h<h \|q_h-q_h \|_2^2= ( _mag- _divH )\|q\|_2^2+ _div \| _hq_h \|_2^2, a quadratic form with eigenvalue 2νmag2 _mag on the head-constant directions and 2(νmag−νdivH)2( _mag- _divH) on their orthogonal complement. We therefore impose (21) νmag>νdivH, _mag> _div\,H, which is exactly the condition under which the pair is coercive, and hence Eq. 20 bounded below in q, and simultaneously the condition under which it is convex. The bound is tight: taking H=2H=2 and q1=−q2=vq_1=-q_2=v gives 2(νmag−2νdiv)‖v‖222( _mag-2 _div)\|v\|_2^2, which is unbounded below as ‖v‖2→∞\|v\|_2→∞ whenever νmag<νdivH _mag< _divH. Third, data filtering, if desired, must act inside the expectation, through the per-sequence weight w(y)∈[0,1]w(y)∈[0,1]; an additive penalty depending only on the data has zero gradient and no effect on training. Remark 15. Prop. 9(1) applies to w(y)w(y): reweighting whole sequences leaves the per-context minimizer unchanged, so w may be set from a parse-validity check without disturbing consistency. The grades entering Eq. 20 are refined end-to-end, but they are not found there: they are selected offline from estimated profiles, by the convex program of Cor. 4 and its clipped form Prop. 10, both developed in Sec. 6. 5.3. Optimisation Optimisation uses AdamW [loshchilov2017decoupled], with raw moments mk=β1mk−1+(1−β1)gk,vk=β2vk−1+(1−β2)gk⊙gk,gk:=∇θℒtotal(θk), splitm_k&= _1m_k-1+(1- _1)\,g_k,\\ v_k&= _2v_k-1+(1- _2)\,g_k g_k, g_k:= _θL_total( _k), split bias corrections m^k=mk/(1−β1k) m_k=m_k/(1- _1^k) and v^k=vk/(1−β2k) v_k=v_k/(1- _2^k), and update θk+1=θk−ηkm^kv^k+ϵ−ηkωθk, _k+1= _k- _k m_k v_k+ε- _k\,ω\, _k, taking β1=0.9 _1=0.9, β2=0.95 _2=0.95, ω=0.1ω=0.1, ϵ=10−8ε=10^-8. The learning rate follows the cosine schedule of [loshchilov2017sgdr], ηk=ηmin+12(ηmax−ηmin)(1+cos(πk/T)), _k= _ + 12 ( _ - _ ) (1+ (π k/T) ), with ηmax=6×10−4 _ =6× 10^-4 and ηmin=10−5 _ =10^-5. Gradient clipping must be stated carefully, since the point of clipping is to remove the grade dependence rather than to accommodate it. We clip the global gradient norm to a constant independent of λ and of qmaxq_ : a threshold set at λqmaxλ^q_ would rescale with the very amplification it is meant to control and would impose no constraint at all. Together with the magnitude penalty of Eq. 20 under Eq. 21 and the grade clipping of Sec. 8.1.1, this keeps the effective step size bounded uniformly in the grades. Distributed training uses standard sharded data parallelism with optimiser-state partitioning [rajbhandari2020zero] and pipeline parallelism across devices [huang2019gpipe]. Lemma 5. Suppose each per-token loss ℓt(θ)=−logp^(yt∣y<t) _t(θ)=- p(y_t y_<t) is ςt _t-smooth on a region Θ , and write ςunif:=∑t=1nςt _unif:= _t=1^n _t for the resulting smoothness constant of the unweighted sum. Then on Θ the graded loss Eq. 19 is ςgrade _grade-smooth with ςgrade≤λqmaxςunif,qmax:=maxtqt, _grade\;≤\;λ^q_ \, _unif, q_ := _tq_t, and the regularisation terms of Eq. 20 contribute an additive constant cregc_reg depending on νmag,νorth,νdiv,H _mag, _orth, _div,H and the diameter of Θ , but not on λ or q. Proof. For θ,θ′∈Θθ,θ ∈ , ‖∇ℒgrade(θ)−∇ℒgrade(θ′)‖2≤∑t=1nλqt‖∇ℓt(θ)−∇ℓt(θ′)‖2≤(∑t=1nλqtςt)‖θ−θ′‖2≤λqmaxςunif‖θ−θ′‖2, split \| _grade(θ)- _grade(θ ) \|_2&≤ _t=1^nλ^q_t \|∇ _t(θ)-∇ _t(θ ) \|_2\\ &≤ ( _t=1^nλ^q_t _t )\|θ-θ \|_2≤λ^q_ _unif\|θ-θ \|_2, split using the triangle inequality, the per-token hypothesis, and λqt≤λqmaxλ^q_t≤λ^q_ . The magnitude and diversity penalties are quadratics in q whose Hessians have operator norm at most 2νmag2 _mag and 2νdivH2 _divH respectively, by the eigenvalue computation of Sec. 5.2, and Ωorth _orth is a polynomial in the entries of P with gradient Lipschitz on the bounded region Θ ; none of these constants involves λ or q. ∎ Theorem 5.1. Suppose ℒtotalL_total is ς -smooth on the region visited, stochastic gradients are unbiased with variance at most σ2σ^2 at batch size b, and the step size satisfies η≤1/ςη≤ 1/ . Then after T steps of stochastic gradient descent, mink≤T‖∇θℒtotal(θk)‖22≤2(ℒtotal(θ0)−infℒtotal)ηT+ςησ2b, _k≤ TE \| _θL_total( _k) \|_2^2\;≤\; 2 (L_total( _0)- _total )η T+ \,η\,σ^2b, and by Lem. 5 the smoothness constant obeys ς≤λqmaxςunif+creg ≤λ^q_ \, _unif+c_reg, with ςunif _unif the corresponding constant for the ungraded objective. Proof. By ς -smoothness, for consecutive iterates θk+1=θk−ηg^k _k+1= _k-η\, g_k with g^k g_k the stochastic gradient, ℒtotal(θk+1)≤ℒtotal(θk)−η⟨∇ℒtotal(θk),g^k⟩+ςη22‖g^k‖22.L_total( _k+1) _total( _k)-η _total( _k), g_k + η^22\| g_k\|_2^2. Taking expectations conditional on θk _k, using unbiasedness [g^k]=∇ℒtotal(θk)E[ g_k]= _total( _k) and the variance bound ‖g^k‖22≤‖∇ℒtotal(θk)‖22+σ2/bE\| g_k\|_2^2≤\| _total( _k)\|_2^2+σ^2/b, [ℒtotal(θk+1)]≤[ℒtotal(θk)]−η(1−ςη2)‖∇ℒtotal(θk)‖22+ςη2σ22b.E [L_total( _k+1) ] [L_total( _k) ]-η (1- η2 )E\| _total( _k)\|_2^2+ η^2σ^22b. Since η≤1/ςη≤ 1/ gives 1−ςη/2≥1/21- η/2≥ 1/2, rearranging and summing over k=0,…,T−1k=0,…,T-1 telescopes the loss terms, and dividing by ηT/2η T/2 bounds the minimum by the average: mink≤T‖∇ℒtotal(θk)‖22≤1T∑k=0T−1‖∇ℒtotal(θk)‖22≤2(ℒtotal(θ0)−infℒtotal)ηT+ςησ2b. _k≤ TE\| _total( _k)\|_2^2≤ 1T _k=0^T-1E\| _total( _k)\|_2^2≤ 2 (L_total( _0)- _total )η T+ ησ^2b. The smoothness comparison is Lem. 5. ∎ Remark 16. The factor λqmax≥1λ^q_ ≥ 1 enters Thm. 5.1 as a price: grading degrades the smoothness constant and therefore the stationarity bound, and no reading of this theorem yields a convergence speedup. The price and the purchase are controlled by the same constant. The grade clipping of Sec. 8.1.1 enforces λqmax≤Cλ^q_ ≤ C with C=2C=2 at the planned configurations, so Thm. 5.1 degrades by at most a factor of C, uniformly in L and in the profiles; the token purchase at the same clip is at most C2C^2 by Prop. 10, and the exponential regime of Cor. 3 is reached as the clip opens (Rem. 18). A price linear in the clip against a purchase quadratic in it is the trade the framework makes, favourable at every setting, and it is the reason the two are reported separately in Sec. 8 rather than folded into a single figure. Three restrictions on scope. The bound is over stationarity and not suboptimality, since ℒtotalL_total is nonconvex in θ and no global guarantee is available. The theorem is stated for stochastic gradient descent, while Sec. 5.3 specifies AdamW; Adam-family methods admit no unconditional convergence guarantee even on convex problems, as shown by Reddi, Kale, and Kumar, so the statement is a discipline on the objective rather than on the optimiser actually deployed. And Lem. 5 bounds the smoothness through per-token constants, so ςunif _unif is the sum of per-token smoothness constants rather than the (possibly smaller) constant of the summed objective; the comparison is honest but not tight. Consequently, any prediction of reduced training tokens to a fixed generalisation target must be routed through Prop. 7 and Prop. 8, and must not be attributed to this theorem. 5.4. Pretraining and Fine-Tuning Pretraining is planned on The Pile [gao2020pile], approximately 800 GB of diverse text, at sequence lengths n∈[1024,8192]n∈[1024,8192]. Per-type grades qoutq_out are estimated once, offline, by the clipped selection problem of Prop. 10 applied to part-of-speech statistics over the corpus, certified against Rem. 17, and refined end-to-end thereafter under the constraint of Rem. 14. Fine-tuning uses direct preference optimisation [rafailov2023direct]. Writing x for the prompt, ywy_w and yly_l for the preferred and rejected responses, πθ _θ for the policy and πref _ref for the reference, (22) ℒDPO=−(x,yw,yl)∼pref[logσ(βDPOlogπθ(yw∣x)πref(yw∣x)−βDPOlogπθ(yl∣x)πref(yl∣x))],L_DPO=-\,E_(x,y_w,y_l) _pref [ σ\! ( _DPO _θ(y_w x) _ref(y_w x)- _DPO _θ(y_l x) _ref(y_l x) ) ], with βDPO>0 _DPO>0 the inverse-temperature of the implicit reward and prefD_pref the preference dataset. A graded variant weights the token-level contributions to logπθ(y∣x) _θ(y x) by their grades, concentrating preference signal on structurally salient positions rather than distributing it uniformly across a response. The grades used must be the qoutq_out of Rem. 14 and no others: a weighting that depends on the realised token and its context reintroduces the tilt of Prop. 9(3) into the implicit reward, where no cancellation is available to absorb it. Subject to that constraint the variant is a construction whose behaviour the program does not yet test. Sec. 6 assembles the selection results of this paper into the general procedure, Sec. 7 instantiates that procedure for language, and Sec. 8 states the experimental program that tests the resulting predictions. 6. Grade Selection The results of Sections 3, 4 and 5 assign to every grade vector a price and a value: a conditioning penalty bounded by Lem. 5, and a sample-complexity gain computed by Prop. 7 and located by Prop. 8. This section assembles those results into the framework’s main application. Grades are neither a hyperparameter to sweep nor a taxonomy to posit; they are the solution of an optimisation problem whose inputs are measurable before training. The procedure below computes them, certifies them, and specifies the experiment that tests the computation. The distinction that organises the section is between domains that carry canonical grades and domains that do not. Weighted projective coordinates and graded rings arrive with their grades attached: the weights of the Igusa invariants are (2,4,6,10)(2,4,6,10) by theorem, not by choice, and in such settings the machinery of this paper adds a consistency check rather than a decision. Language carries no canonical grading; which coordinates of an embedding space deserve amplification is exactly the question no theorem of linguistics answers, and it is the question Prop. 7(i) answers in closed form. The contribution of the framework is educated grade selection in domains where the grades are not given. 6.1. The Selection Procedure The procedure has six steps, and each names the result that supports it. (1) Fix the basis. The profiles of Def. 2 are defined relative to the graded basis of Eq. 13. In language this is the embedding basis carrying the level projections Pl\P_l\; in a domain with canonical structure it is the given one. (2) Estimate the data profile. τ^j τ_j is the per-coordinate variance over the corpus, normalised to the simplex, as in Sec. 7.2. This is a corpus statistic and requires no model. (3) Estimate the target profile. α^j α_j is the normalised squared weight of a linear probe trained on the annotated target, reported together with the probe’s inductive bias, as in Sec. 7.2. (4) Test divergence. Compute BC(α^,τ^)BC( α, τ). By Prop. 7(iv) a value near 11 means the profiles agree and grading buys nothing in this domain at this target; the procedure terminates here, at the cost of two estimates. This step is the domain filter, and it is where Assumption 1 is tested rather than assumed. (5) Certify candidates. A proposed grade vector q, whether taxonomic, statistical, or computed, is admissible to first order exactly when ⟨q,α^−τ^⟩>0 q, α- τ >0, by Prop. 8(iv). The inner product is one line of arithmetic and is reported before training. (6) Compute the optimum. The grades minimising the bound are the solution of the convex program of Cor. 4 below, projected to the clipped region by Prop. 10. The output is qC⋆q _C, the educated grading at clip C; by Lem. 2 the gain it attains degrades only to second order in the estimation error of steps (2) and (3). Steps (2)–(4) cost no pretraining, step (5) costs an inner product, and step (6) is a geometric program in d variables. The entire selection is performed, published, and falsifiable before the first training step, which is the property the staging of Sec. 8 is built around. 6.2. The Offline Selection Problem The grades entering Eq. 20 are refined end-to-end, but they are not found there. They are selected offline from estimated profiles, and that problem is well posed in a way the end-to-end problem is not. Corollary 4. Let α^,τ^∈Δd−1 α, τ∈ ^d-1 be strictly positive estimates of the profiles of Def. 2, and consider the offline selection problem minq∈ℝ≥0dlogΛ(q)+νmag‖q‖22,Λ(q)=(∑j=1dα^jλ−2qj)(∑j=1dτ^jλ2qj). _q _≥ 0^d\ \ (q)+ _mag\|q\|_2^2 \, (q)= ( _j=1^d α_j\,λ^-2q_j ) ( _j=1^d τ_j\,λ^2q_j ). Then the objective is strictly convex on the convex feasible set ℝ≥0dR_≥ 0^d, so the minimizer is unique and every stationary point is the global minimum. In the variables tj=λ2qjt_j=λ^2q_j the function Λ is a posynomial and the problem is a geometric program. As νmag→0 _mag→ 0 the value tends to logBC(α^,τ^)2 ( α, τ)^2; for νmag>0 _mag>0 the penalty breaks the gauge invariance of Prop. 8(i) and selects the minimum-norm representative of the optimal orbit. The version of this problem on the clipped region the architecture admits is solved in Prop. 10, with the same convex structure and a saturated form of the same optimum. Proof. Convexity of q↦logΛ(q)q (q) is Prop. 8(i), and νmag‖q‖22 _mag\|q\|_2^2 is strictly convex, so the sum is strictly convex; ℝ≥0dR_≥ 0^d is convex, whence uniqueness and the coincidence of stationary and global minima. For the posynomial claim, ∑jα^jtj−1 _j α_jt_j^-1 and ∑jτ^jtj _j τ_jt_j are posynomials in t∈ℝ>0dt _>0^d and a product of posynomials is a posynomial. The limiting value is Prop. 8(vi), and the representative in ℝ≥0dR_≥ 0^d with minjqj=0 _jq_j=0 supplied by Prop. 8(i) has finite norm, so the penalty is finite along the orbit and selects its minimum-norm point. ∎ Remark 17. Cor. 4 has a consequence available before any pretraining run begins. By Prop. 8(iv), a proposed initialisation qinitq^init, say the part-of-speech grades of Sec. 8.1.2, lies in the admissible cone +Q_+, to first order, exactly when ⟨qinit,α^−τ^⟩>0, q^init, α- τ >0, a single inner product between the proposed grades and the estimated profiles. Neither quantity requires a trained model: τ τ is a corpus statistic and α α a probe weight, both computed in Sec. 7.2. A GLLM may therefore be certified in advance to begin inside the region where its sample-complexity bound is the smaller one, at the cost of one dot product, and Cor. 4 then locates the optimum within that region by convex programming rather than by search. The magnitude of the gain depends on the accuracy of α α and τ τ, controlled by Lem. 2; the sign does not, and only the sign is needed for certification. At the clipped configurations the relevant optimum is the qC⋆q _C of Prop. 10, and the attainable gain is bounded by C2C^2 accordingly. The end-to-end refinement is a different matter. Eq. 20 is nonconvex in θ, and no counterpart of Cor. 4 is available for it. The role of the offline problem is to supply a certified starting point, not to replace training. 6.3. Selection Under Clipping The architecture is trained under a grade clip λqmax≤Cλ^q_ ≤ C, with C=2C=2 at the configurations of Sec. 8.1.1, and the selection problem must be solved on the region the clip admits. The clipped problem retains the structure of the unclipped one, and the clip prices the gain exactly. Proposition 10. Fix C=λqmax≥1C=λ^q_ ≥ 1 and let C=q∈ℝd:0≤qj≤qmaxQ_C=\q ^d:0≤ q_j≤ q_ \. Write ΛC⋆=minq∈CΛ(q) _C= _q _C (q) and rj=14logλlog(αj/τj)r_j= 14 λ ( _j/ _j). (i) The minimum is attained, and the problem is convex: logΛ is convex on the compact convex set CQ_C. (i) Every minimizer has the saturated form qj⋆=minqmax,max0,rj+cq_j \;=\; \q_ ,\, \0,\;r_j+c\ \ for some constant c∈ℝc : the fourth-root law of Prop. 7(i) on the interior coordinates, saturation at the box on the rest. (i) The gain is bounded by the clip: ΛC⋆≥maxBC(α,τ)2,C−2, _C\;≥\; \BC(α,τ)^2,\;C^-2 \, and the bound C−2C^-2 is approached as the profiles concentrate on separate coordinates. Reaching the regime Λ⋆=Θ(νL) = (ν^L) of Cor. 3 therefore requires C≥ν−L/2C≥ν^-L/2: the exponential-in-L gain is the limit of a growing clip, at the conditioning price Lem. 5 attaches to it. Proof. (i) CQ_C is a product of intervals, hence compact and convex, and logΛ is convex by Prop. 8(i) and continuous. (i) In the variables uj=2qjlogλu_j=2q_j λ of Prop. 8, the Karush–Kuhn–Tucker conditions for the box read ∂jΦ(u)=τj(u)−αj(u)=μj−μj+ _j (u)=τ^(u)_j-α^(u)_j= _j^-- _j^+ with μj±≥0 _j^±≥ 0 supported on the active constraints. On an interior coordinate both multipliers vanish and αje−uj/A(u)=τjeuj/T(u) _je^-u_j/A(u)= _je^u_j/T(u), giving uj=12log(αj/τj)+12log(T(u)/A(u))u_j= 12 ( _j/ _j)+ 12 \! (T(u)/A(u) ), which is qj=rj+cq_j=r_j+c with c=14logλlog(T(u⋆)/A(u⋆))c= 14 λ \! (T(u )/A(u ) ) common to all interior coordinates. On a coordinate active at the lower bound the condition ∂jΦ≥0 _j ≥ 0 holds exactly when the interior law would give rj+c≤0r_j+c≤ 0, and symmetrically at the upper bound, which is the saturated form. Since Λ is invariant along 1 by Prop. 8(i), the minimizer set is a segment in that direction intersected with CQ_C, and every point of it has the stated form under the corresponding shift of c. (i) The bound BC(α,τ)2BC(α,τ)^2 is the unconstrained minimum of Prop. 7(i). For the second bound, every feasible gj=λqjg_j=λ^q_j lies in [1,C][1,C], so Λ(q)=(∑j=1dαjgj−2)(∑j=1dτjgj2)≥C−2(∑jαj)⋅1⋅(∑jτj)=C−2. (q)= ( _j=1^d _jg_j^-2 ) ( _j=1^d _jg_j^2 )\;≥\;C^-2 ( _j _j )· 1· ( _j _j )=C^-2. For tightness take d=2d=2, α→(1,0)α→(1,0), τ→(0,1)τ→(0,1), and q=(qmax,0)q=(q_ ,0): the first factor tends to C−2C^-2 and the second to 11. For the final claim, ΛC⋆≤νL _C≤ν^L forces C−2≤νLC^-2≤ν^L. ∎ Remark 18. Prop. 10(i) and Lem. 5 are controlled by the same constant, and together they give the framework’s cost–benefit statement in final form: at clip C the conditioning penalty is at most C and the sample-complexity gain is at most C2C^2. The price is linear in the clip and the purchase quadratic, so the trade is favourable at every setting, and the planned configuration C=2C=2 buys a gain of at most 44 at a penalty of at most 22. The exponential regime of Cor. 3 is not available at fixed clip; it is reached as the clip grows, with both sides of the trade priced at every point along the way. The experiments of Sec. 8 are accordingly designed to detect a bounded effect at C=2C=2 and to sweep C as an explicit arm. 6.4. Level Reduction Under the stratification of Cor. 3 the selection problem collapses from d variables to L, and this reduction is what makes step (6) implementable at scale. Proposition 11. Suppose the profiles are uniform within levels in the sense of Cor. 3: αj=al/dl _j=a_l/d_l and τj=sl/dl _j=s_l/d_l for j∈Vlj∈ V_l, with a,s∈ΔL−1a,s∈ ^L-1 strictly positive. Then every minimizer of Λ over ℝdR^d, and every minimizer over the clipped box CQ_C of Prop. 10, is constant on each level. Consequently the selection problem is the box-constrained convex program in L variables minv∈[0,qmax]LΛL(v),ΛL(v)=(∑l=0L−1alλ−2vl)(∑l=0L−1slλ2vl), _v∈[0,q_ ]^L\ _L(v), _L(v)= ( _l=0^L-1a_lλ^-2v_l ) ( _l=0^L-1s_lλ^2v_l ), whose solution v⋆v determines q⋆q by qj⋆=vl⋆q _j=v _l for j∈Vlj∈ V_l. At L=4L=4 the program has four variables, whatever d may be. Proof. Work in the variables uj=2qjlogλu_j=2q_j λ and write Φ(u)=logA(u)+logT(u) (u)= A(u)+ T(u) as in the proof of Prop. 8, with A(u)=∑jαje−ujA(u)= _j _je^-u_j and T(u)=∑jτjeujT(u)= _j _je^u_j. Let Σ be the group of coordinate permutations σ with σ(Vl)=Vlσ(V_l)=V_l for every l. Under the stratification hypothesis α and τ are constant on each level, hence Σ -invariant, so Φ(σ⋅u)=Φ(u) (σ· u)= (u) for every σ∈Σσ∈ ; and both feasible sets, ℝdR^d and the box, are Σ -invariant and convex. Let ℳM denote the set of minimizers, nonempty by Prop. 10(i) in the clipped case and by Prop. 7(i) in the unclipped one. Since Φ is convex and the feasible set is convex, ℳM is convex. If u⋆∈ℳu then σ⋅u⋆∈ℳσ· u for every σ∈Σσ∈ , so the average u¯=|Σ|−1∑σ∈Σσ⋅u⋆ u=| |^-1 _σ∈ σ· u lies in ℳM, and u¯ u is constant on each level because it is a Σ -average. It remains to show that every minimizer is of this form. Let u1,u2∈ℳu^1,u^2 . The segment [u1,u2][u^1,u^2] lies in the feasible set by convexity and in ℳM by convexity of Φ , on which Φ is therefore constant; a convex function constant along a segment has vanishing second directional derivative along it, so (u2−u1)⊤∇2Φ(u)(u2−u1)=0(u^2-u^1) ∇^2 (u)(u^2-u^1)=0 at every u of the segment. By the proof of Prop. 8, ∇2Φ(u)=C(α(u))+C(τ(u))∇^2 (u)=C(α^(u))+C(τ^(u)) with v⊤C(p)v=Varp(v)v C(p)v=Var_p(v), and both tilted vectors α(u),τ(u)α^(u),τ^(u) are strictly positive because α and τ are. Hence Varα(u)(u2−u1)=0Var_α^(u)(u^2-u^1)=0, which forces u2−u1∈ℝu^2-u^1 1. Every minimizer is therefore u¯+c u+c1 for some c∈ℝc , and is constant on each level. For the reduced form, evaluate Λ on a level-constant q: ∑j∈Vl(al/dl)λ−2vl=alλ−2vl _j∈ V_l(a_l/d_l)λ^-2v_l=a_lλ^-2v_l and likewise for τ, giving ΛL _L. Convexity of v↦logΛL(v)v _L(v) is Prop. 8(i) restricted to the level-constant subspace, and [0,qmax]L[0,q_ ]^L is compact and convex. ∎ 6.5. The Selection Ladder The experiment that tests educated selection is not a two-arm comparison but a ladder. Every arm is a GLLM of identical architecture, data, and optimisation budget, distinguished only by what its grades consulted; the standard transformer is the bottom rung, the arm whose grades consulted nothing, and by the gauge invariance of Prop. 8(i) its admissibility inner product is 0 identically, with Prop. 8(i) reading that zero as the boundary position of the uniform grades. Arm Grades consult ⟨q,α^−τ^⟩ q, α- τ Licence Uniform nothing 0 exactly Prop. 8(i),(i) Random noise ≈0≈ 0, either sign control Taxonomic the level ordering measured Rem. 9 Part-of-speech corpus statistics measured Sec. 8.1.2 Educated (qC⋆q _C) both estimated profiles maximal among arms Prop. 10 Oracle the held-out target diagnostic ceiling — Table 2. The selection ladder. All arms share architecture, data, and budget; only the provenance of the grades varies. The third column is computed offline and published before training. The prediction, conditional on Assumption 1 and the scope of Rem. 11, is an ordering: the token-efficiency measurement of Sec. 8.2.2 ranks the arms as the third column of Tab. 2 ranks them. The ordering is fixed by offline measurement before any arm is trained, which makes the prediction pre-registered by construction and sharper than any pairwise comparison: a graded model beating an ungraded one admits many explanations, while six arms ordering by a number computed in advance admits few. The outcomes partition cleanly. If the ordering holds, the first-order reading of Prop. 8 survives the passage from the single-layer analysis to the full stack, and the selection procedure works as claimed. If the educated arm falls below the part-of-speech arm, the quadratic degradation of Lem. 2 has exceeded the refinement the estimates purchased, on estimates whose error the bootstrap of Sec. 7.2 reports in advance. If all arms tie, either the profiles fail to diverge, which is caught at step (4) before training at negligible cost, or the single-layer analysis fails at depth, which is the first limitation of Rem. 11. Each outcome points to a specific result. 6.6. Scope of the Procedure The procedure applies wherever its first three steps are executable: a basis, second moments, and a target admitting a probe. Domains sort into three classes against that criterion. Domains with canonical grades, such as weighted projective spaces, graded rings, and the weighted moduli of algebraic curves, carry the answer to step (6) already, and there the procedure inverts into a consistency test: the estimated q⋆q should correlate with the canonical weights, and a failure of that correlation would indicate a target that does not respect the graded structure. The machine-learning treatment of the genus-two moduli space is the natural site for this check, since the weights (2,4,6,10)(2,4,6,10) and the invariant-theoretic targets are both exact and the profiles are computable without estimation error. Domains without canonical grades but with measurable profiles are where the procedure is the contribution, and language is the motivating case: the levels are real, their coordinates are not given, and Sec. 7 instantiates the procedure for it, with Sec. 8 running the ladder. Domains where step (3) has no probe target, such as end-to-end control where no annotated intermediate defines α, lie outside the procedure’s present scope, and extending it there requires a surrogate for the target profile before anything else. 6.7. The Admissible Cone in Geometric Invariant Theory The structure assembled in Prop. 8, the gauge invariance along 1, the convexity of logΛ , the boundary position of the uniform grades, and the first-order criterion Eq. 15, is an instance of a classical one. Under the exponential change of coordinates the gain becomes a Kempf–Ness functional for an action of the grading torus, the admissible set becomes the negativity region of a homogeneous hypersurface, the optimal grades of Prop. 7(i) become the coincidence point of two moment maps, and Eq. 15 becomes the descent criterion of geometric invariant theory at the uniform point [mumford-GIT, kempf-ness]. This subsection records the dictionary; no result elsewhere in the paper depends on it, and its purpose is to place the selection theory of this section in the same geometry as the canonically graded domains of Sec. 6.6, where the stability theory is classical and the grades arrive by theorem. Let =(ℂ×)dT=(C^×)^d act diagonally on ℂdC^d by t⋅z=(tjzj)t· z=(t_jz_j), with maximal compact (S1)d(S^1)^d and Lie-algebra coordinates u∈ℝdu ^d acting through exp(u/2)⋅z=(euj/2zj). (u/2)· z=(e^u_j/2z_j). For a strictly positive p∈Δd−1p∈ ^d-1 set zp=(p11/2,…,pd1/2)∈ℂdz_p=(p_1^1/2,…,p_d^1/2) ^d and define the Kempf–Ness functional of zpz_p, ψp(u)=log‖exp(u/2)⋅zp‖2=log∑j=1dpjeuj,∇ψp(u)=μ(exp(u/2)⋅zp), split _p(u)&= \| (u/2)· z_p \|^2= _j=1^dp_j\,e^u_j,\\ ∇ _p(u)&=μ ( (u/2)· z_p ), split where μ(z)=(|zj|2/‖z‖2)j∈Δd−1μ(z)= (|z_j|^2/\|z\|^2 )_j∈ ^d-1 is the moment map of the compact torus, with image in the simplex. Lemma 6. In the variables uj=2qjlogλu_j=2q_j λ of Prop. 8, the log-gain is the Kempf–Ness functional of the pair (zα,zτ)∈ℂd×ℂd(z_α,z_τ) ^d×C^d under the anti-diagonal action t⋅(x,y)=(t−1x,ty)t·(x,y)=(t^-1x,\,t\,y): Φ(u)=logΛ=ψα(−u)+ψτ(u),∇Φ(u)=μ(eu/2zτ)−μ(e−u/2zα)=τ(u)−α(u), split (u)= &= _α(-u)+ _τ(u),\\ ∇ (u)&=μ (e^u/2z_τ )-μ (e^-u/2z_α )=τ^(u)-α^(u), split so the tilted profiles of Prop. 8 are the moment maps of the two factors, and the Hessian ∇u2Φ=C(α(u))+C(τ(u))∇^2_u =C(α^(u))+C(τ^(u)) is the sum of their moment-map derivatives. Proof. By definition ψα(−u)=log∑jαje−uj=logA(u) _α(-u)= _j _je^-u_j= A(u) and ψτ(u)=logT(u) _τ(u)= T(u), so the sum is log(AT)=logΛ (AT)= . Differentiating, ∂jψα(−u)=−αje−uj/A(u)=−αj(u) _j _α(-u)=- _je^-u_j/A(u)=-α^(u)_j and ∂jψτ(u)=τj(u) _j _τ(u)=τ^(u)_j; moreover μ(e−u/2zα)j=αje−uj/∑kαke−uk=αj(u)μ(e^-u/2z_α)_j= _je^-u_j/ _k _ke^-u_k=α^(u)_j, and likewise for τ. The Hessian identity is the computation in the proof of Prop. 8. ∎ Proposition 12. Let α,τα,τ be the profiles of Def. 2 and set tj=λ2qjt_j=λ^2q_j. (i) (The boundary hypersurface.) In the coordinates t∈ℝ>0dt ^d_>0 the admissible set is +=F<0∩ℝ>0dQ_+=\F<0\ ^d_>0, where F(t)=(∑j=1dαj∏k≠jtk)(∑l=1dτltl)−∏k=1dtkF(t)= ( _j=1^d _j _k≠ jt_k ) ( _l=1^d _l\,t_l )- _k=1^dt_k is homogeneous of degree d. The homogeneity is the gauge invariance of Prop. 8(i): the translation q↦q+cq q+c1 is the scaling t↦λ2ct λ^2ct, so +Q_+ is a cone in the exact sense, invariant under ℝ>0R_>0-scaling, and ∂+ _+ descends to the positive real locus of the projective hypersurface V(F)⊂ℙd−1V(F) ^d-1. The uniform point [1:⋯:1][1:·s:1] lies on V(F)V(F), with ∇F()=τ−α∇ F(1)=τ-α, and is a smooth point of it whenever α≠τα≠τ. (i) (Coercivity and existence.) For a one-parameter subgroup v∈ℝdv ^d of T, the recession slope of the Kempf–Ness functional along v is ρ(v):=lims→∞Φ(sv)s=maxjvj−minjvj,ρ(v):= _s→∞ (sv)s= _jv_j- _jv_j, strictly positive for every v∉ℝv 1 because both profiles are strictly positive by Def. 2. The functional Φ is therefore coercive on ℝd/ℝR^d/R1, hence proper modulo the gauge direction, and the pair (zα,zτ)(z_α,z_τ) is polystable for the quotient torus /ℂ×T/C^×1; the Kempf–Ness theorem then yields existence of a minimizer and uniqueness modulo gauge, recovering Prop. 8(vi). The recession slope ρ is the coercivity certificate and is symmetric under v↦−v -v; the sign-sensitive Hilbert–Mumford-type criterion that decides admissibility is the first-order pairing of (iv), not ρ. (i) (The optimum as a moment-map coincidence.) The minimizer uj⋆=12log(αj/τj)+cu _j= 12 ( _j/ _j)+c is characterised by the coincidence of the two moment maps, μ(e−u⋆/2zα)=μ(eu⋆/2zτ)=β,βj=αjτjBC(α,τ),μ (e^-u /2z_α )=μ (e^u /2z_τ )=β, _j= _j _jBC(α,τ), with attained value Φ(u⋆)=logBC(α,τ)2 (u )= (α,τ)^2: the fourth-root law of Prop. 7(i) is the balancing of the two moment maps, and the profile β is their common value. (iv) (The first-order criterion as the descent criterion.) At u=u=0 the derivative of Φ along v is ⟨v,τ−α⟩ v,τ-α , so Eq. 15 is precisely the condition that v be a Kempf–Ness descent direction at the uniform point. This pairing is sign-sensitive, reversing under v↦−v -v, and is the Hilbert–Mumford-type object of the dictionary: the character it pairs against is the moment-map defect τ−ατ-α, which is the normal to V(F)V(F) at the uniform point by (i), and the full gauge orbit ℝR1 of the uniform grades lies in Λ=1\ =1\. Proof. (i) With A(t)=∑jαjtj−1A(t)= _j _jt_j^-1 and T(t)=∑lτltlT(t)= _l _lt_l, clearing denominators gives Λ−1=F(t)/∏ktk -1=F(t)/ _kt_k on ℝ>0dR^d_>0, so Λ<1 <1 is F<0F<0 there. The first factor of F is homogeneous of degree d−1d-1, the second of degree 11, and the monomial of degree d. Scaling t↦st st with s=λ2cs=λ^2c corresponds to q↦q+cq q+c1, which is Prop. 8(i). At t=t=1, F()=Λ()−1=0,F(1)= (1)-1=0, and ∇F=(∇tΛ)∏ktk+(Λ−1)∇t∏ktk∇ F=( _t ) _kt_k+( -1)\, _t _kt_k evaluates at t=t=1 to ∇tΛ() _t (1), whose j-th entry is −αjT()+A()τj=τj−αj;- _jT(1)+A(1) _j= _j- _j; this is nonzero exactly when α≠τα≠τ, which is smoothness of V(F)V(F) at the uniform point. (i) As s→∞s→∞, ψτ(sv)=smaxjvj+O(1) _τ(sv)=s _jv_j+O(1) and ψα(−sv)=−sminjvj+O(1) _α(-sv)=-s _jv_j+O(1), since each sum is dominated by its extremal exponent and all coefficients are strictly positive; the slope is ρ(v)=maxjvj−minjvjρ(v)= _jv_j- _jv_j, vanishing exactly on ℝR1. A convex function with strictly positive recession slope in every direction transverse to 1 is coercive, hence proper, on ℝd/ℝR^d/R1, so a minimizer exists; uniqueness modulo ℝR1 follows as in the proof of Prop. 11, since vanishing of the Hessian form along a segment of minimizers forces the segment into ℝR1. Symmetry of ρ under v↦−v -v is immediate from the displayed formula, which distinguishes it from a Hilbert–Mumford weight. (i) At u⋆u , αje−uj⋆=αjτje−c _je^-u _j= _j _j\,e^-c, so A(u⋆)=e−cBC(α,τ)A(u )=e^-c\,BC(α,τ) and αj(u⋆)=αjτje−ce−cBC(α,τ)=αjτjBC(α,τ);α^(u )_j= _j _j\,e^-ce^-c\,BC(α,τ)= _j _jBC(α,τ); symmetrically τjeuj⋆=αjτjec _je^u _j= _j _j\,e^c, T(u⋆)=ecBC(α,τ)T(u )=e^c\,BC(α,τ), and τj(u⋆)=αjτj/BC(α,τ)τ^(u )_j= _j _j/BC(α,τ). The moment maps coincide at β and ∇Φ(u⋆)=0∇ (u )=0 by Lem. 6. The value is ψα(−u⋆)+ψτ(u⋆)=logA(u⋆)+logT(u⋆)=(logBC−c)+(logBC+c)=logBC(α,τ)2. _α(-u )+ _τ(u )= A(u )+ T(u )= ( -c )+ ( +c )= (α,τ)^2. (iv) The derivative is ⟨∇Φ(),v⟩=⟨τ−α,v⟩ ∇ (0),v = τ-α,v by Lem. 6, and Λ(c)=1 (c1)=1 for every c is Prop. 8(i); the normal direction is ∇F()=τ−α∇ F(1)=τ-α from (i). The pairing ⟨τ−α,v⟩ τ-α,v reverses sign under v↦−v -v, unlike the recession slope ρ of (i). ∎ Remark 19. Two statements coexist in Prop. 12 and operate at different points of the dictionary. The first is the solvability of the selection problem, certified by (i): the recession slope ρ(v)ρ(v) is strictly positive along every one-parameter subgroup transverse to the gauge direction, because both profiles have full support, so the Kempf–Ness functional is coercive modulo gauge. This is what makes the minimisation of Cor. 4 well posed: the minimisation has a minimizer, unique modulo gauge, rather than an infimum escaping to infinity along a direction of vanishing slope. A coordinate on which α or τ vanished would supply such a direction, and Def. 2 deletes those coordinates for this reason: full support of the profiles is the analytic form of stability, and the deletion convention of Def. 2 is its enforcement. The sign-sensitive criterion that classifies directions, as opposed to the symmetric slope that certifies coercivity, is the first-order pairing of (iv). The second concerns the uniform grades, and it is a statement about a point of the orbit rather than about the orbit’s type: the gauge orbit ℝR1 lies inside the level set Λ=1\ =1\, which by (i) is the boundary hypersurface V(F)V(F), while the moment-map defect ∇Φ()=τ−α∇ (0)=τ-α is nonzero whenever the profiles differ. The uniform point is thus positioned the way a strictly semistable point is positioned in a GIT quotient, on the boundary of the favourable locus with its entire orbit trapped there, and with the destabilising data, here the profile difference, naming the direction of escape. The analogy is positional and not literal, since the pair itself is stable; what the position encodes is that the uniform grades are the unique gauge orbit on which the Kempf–Ness functional is critical for no pair of profiles and equal to its reference value for every pair, which is Prop. 8(i) read through the dictionary. Remark 20. The admissible cone is not a nullcone, and the comparison should be drawn at exactly its strength. A nullcone is Zariski-closed, cut out by the ideal of positive-degree invariants, generally non-convex, and stratified; +Q_+ is open, semialgebraic, defined by the single inequality F<0F<0, and log-convex in the grade variables, and the convexity is the property the selection of Cor. 4 runs on. What the dictionary identifies is the complementary structure: the gauge orbit of the uniform grades sits inside the boundary hypersurface in the way the orbit of a strictly semistable point sits inside the boundary of the stable locus, as Rem. 19 makes precise, and Eq. 15 is a Hilbert–Mumford-shaped pairing that reads admissibility of a one-parameter subgroup off a fixed character of the torus. The Hilbert–Mumford-type object of the dictionary is the sign-sensitive first-order pairing of Prop. 12(iv); the recession slope of Prop. 12(i) is not such an object but its coercivity certificate, symmetric under reversal of the one-parameter subgroup. The clipped problem of Prop. 10 is, in this language, the Kempf–Ness minimisation restricted to a box in the Lie algebra, and the saturated form of Prop. 10(i) is the facial structure of that box under the moment-map flow. Remark 21. The graded program originates in the invariant theory of binary forms, and the stability theory above closes that circle at its source. For degree-d binary forms under SL2SL_2, the Hilbert–Mumford criterion reduces, after conjugating the one-parameter subgroup into the maximal torus, to the weight computation μ(f,λ)=max2i−d:ai≠0μ(f,λ)= \2i-d:a_i≠ 0\, a sign-sensitive pairing of the kind that appears in Prop. 12(iv); stability is equivalent to every root having multiplicity below d/2d/2, and the invariant ring is graded with canonical weights, the weights (2,4,6,10)(2,4,6,10) above being the sextic instance. The equivalence class of a form is its weighted moduli point in the weighted projective space carrying those weights as grades, and the entire stability theory is read off that graded point: a form is unstable exactly when its moduli point vanishes, and, over a number field, semistable over a residue field exactly when the prime does not divide the point’s coordinates, with semistability quantified by the weighted height of the point [eshaska]. In both settings a graded coordinate system converts stability into an arithmetic condition on a single point: there, divisibility and weighted height of the moduli point decide residual semistability; here, positivity of the recession slope of Prop. 12(i), guaranteed by full support of the profiles, gives solvability of the selection problem. The direction of inference is complementary: the binary-form setting fixes the grades, given by theorem as the weights of the invariants, and studies the stability of points; this section fixes the point, the pair of profiles, and selects the grades. The consistency test of Sec. 6.6 sits at the intersection: in a canonically graded domain the estimated optimum of Cor. 4 is predicted to recover the weights the invariant theory prescribes. Remark 22. The domains of Sec. 6.6 that carry canonical grades, weighted projective spaces and graded rings among them, are precisely those in which the torus action and its linearisation are given by theorem and the moment-map data need not be estimated. The present construction equips a domain without canonical grades with the same objects from measurement: the two profiles of Def. 2 determine the pair (zα,zτ)(z_α,z_τ), the estimated moment maps determine the descent direction at the uniform point, and the selection of Cor. 4 is the Kempf–Ness minimisation for that pair. The consistency test of Sec. 6.6 for the canonically graded case is, in this language, the statement that the estimated moment-map coincidence point should recover the canonical weights. 7. Case Study: Hierarchical Grading for Natural Language Sec. 6 states the selection procedure in the generality the framework supports: fix a basis, estimate two profiles, test their divergence, certify a candidate, compute the optimum. This section carries it out for language. Each subsection below is one step of Sec. 6.1, taken to the point at which it can be implemented and reproduced, and the section as a whole is the input the program of Sec. 8 uses: it fixes the graded basis the configurations use, the grades every arm of Tab. 2 carries, and the effect size the runs must detect. Two things distinguish this stage from the runs it precedes. It is offline in the strict sense: a treebank, a matrix of second moments, and a convex program in four variables. And it is falsifiable on its own terms, since step (4) can terminate the program before a single parameter is initialised. The estimates themselves are not reported here; what is reported is the construction, and the consequences that follow from it in closed form. 7.1. The Level Taxonomy and the Graded Basis Step (1) of Sec. 6.1 asks for the basis in which the profiles of Def. 2 are read. Language supplies no canonical one, which is exactly the situation the framework addresses; what it does supply is a taxonomy, and the basis is built to carry it. We take L=4L=4, matching the configurations of Sec. 8.1.1: V0V_0 for subword and morphological structure, V1V_1 for syntactic constituency and dependency, V2V_2 for sentential semantics, V3V_3 for discourse and pragmatics. Each level is given a feature map φl _l sending a token in context to a vector of annotations drawn from a corpus that carries them. For l=0l=0 these are the segmentation identity, affix class, part-of-speech tag, and orthographic case. For l=1l=1 they are the label of the incoming dependency arc, the signed distance to the head, the labels of the constituent ancestors, and the depth of the token in the parse. For l=2l=2 they are the predicate–argument role, the polarity of the governing negation, and the scope index of the governing quantifier. For l=3l=3 they are coreference-chain membership, the discourse-relation label attaching the containing clause to its neighbour, and the position of the sentence within the document. The Penn Treebank and the Universal Dependencies corpora supply l∈0,1l∈\0,1\; a proposition-bank annotation supplies l=2l=2; a coreference-annotated corpus and a discourse treebank supply l=3l=3. The projections Pl\P_l\ are then fixed by principal component analysis: for each level, the annotated feature φl _l is regressed onto a reference representation of width d, the residual covariance of the fitted directions is formed, and Pl∈ℝd×dlP_l ^d× d_l is taken to span its leading dld_l eigenvectors. Gram–Schmidt across levels enforces mutual orthogonality of the ranges at initialisation, which is the hypothesis η=0η=0 of Prop. 6; thereafter the columns are learnable and the orthogonality penalty of Sec. 4 controls the drift, with Prop. 6 bounding the residual. Remark 23. The construction above requires a representation of width d before the model that will carry it exists, and the circularity is real rather than apparent: at initialisation WeW_e is isotropic by construction, so the profiles read in its coordinates are uniform and carry no information. Two routes resolve it, and the program uses both, at different steps. For steps (2)–(5), the divergence test and the certification, the reference is a fixed public encoder of matching width, or static type embeddings projected to ℝdR^d. This costs no pretraining, which is what makes the falsification test of Sec. 7.3 available before the first GPU-hour. The profiles so obtained are the reference’s, and transporting them to the GLLM’s own geometry is an assumption, not a theorem. For step (6), the educated grading qC⋆q _C, the reference is the embedding matrix of the uniform arm itself. This is exact, since the profiles are then read in precisely the space in which the comparison of Prop. 7 is made, and it is not an added cost: the uniform arm is the control of Tab. 2 and must be trained regardless. What makes the cheap route sufficient for its purpose is Prop. 8(iv). Certification turns on the sign of an inner product, not on the magnitudes of α α and τ τ, and a coarse reference that gets the sign right certifies as well as an exact one. 7.2. Estimating the Profiles Steps (2) and (3) produce the two probability vectors on which everything downstream depends. Both are computed in the basis of Sec. 7.1 and both are estimated on held-out data. The data profile is a corpus statistic and requires no model. Writing εt∈ℝd _t ^d for the reference representation of the token at position t and ε¯ for its mean over the corpus, the estimator is the normalised per-coordinate second moment in the graded basis, τ^j=σ^j2∑k=1dσ^k2,σ^j2=1N∑t=1N⟨ej,εt−ε¯⟩2, τ_j= σ_j^2 _k=1^d σ_k^2, σ_j^2= 1N _t=1^N e_j, _t- ^2, with ej\e_j\ the columns of [P0∣⋯∣PL−1][P_0 ·s P_L-1]. Tokens are drawn from the pretraining corpus [gao2020pile] so that τ τ is the variance profile of the data the runs will actually see, and N is set by the width of the resulting confidence interval rather than by convention. The target profile is a probe weight. For a hierarchical target y, such as acceptability, dependency-arc label, or nesting depth, a linear probe w w is fitted on the annotated treebank in the same basis, and α^j=w^j2‖w^‖22. α_j= w_j^2\| w\|_2^2. The probe is fitted with ridge regularisation and the penalty is selected by cross-validation on a split disjoint from the one on which α α is reported, so that the shrinkage does not itself manufacture the concentration that Assumption 1 asserts. Three properties of the estimate must be reported with it. It identifies α only up to the probe’s inductive bias, and a probe that cannot express the target reports the profile of what it can express instead. Coordinates on which w w vanishes are deleted rather than carried, as Def. 2 requires. And the ridge penalty biases α α towards uniformity, so the estimate is conservative for the framework’s purposes: it understates divergence rather than overstating it. The quantity that must survive estimation error is the sign of ⟨q,α^−τ^⟩ q, α- τ , not the value of either profile, and it is reported with a bootstrap over the treebank sentences: resample, refit, recompute, and report the fraction of resamples on which the sign is preserved. This is the only uncertainty quantification the certification of Sec. 7.4 requires. The magnitude of BC(α^,τ^)BC( α, τ), which the effect-size statement does require, carries its own interval from the same bootstrap. 7.3. The Divergence Test Step (4) is the domain filter, and it is where Assumption 1 is tested rather than assumed. The program reports BC(α^,τ^)=∑j=1dα^jτ^jBC( α, τ)= _j=1^d α_j\, τ_j per task, together with the level-aggregated profiles (a^,s^)( a, s) of Cor. 3 obtained by summing α α and τ τ within levels, and a goodness-of-fit check of the geometric hypothesis al∝νL−1−la_l ν^L-1-l, sl∝μls_l μ^l on which every exponential-in-L statement of this work rests. By Prop. 7(iv) a value near 11 says the profiles agree, the gain of grading is 11, and the procedure terminates for that task at the cost of two estimates. By Assumption 1 the hierarchical targets of Sec. 8.2.3 are the tasks at which it should not be near 11, and the lexically dominated contrast tasks are the tasks at which it should be: the framework predicts a spread across tasks, and a uniform value across tasks would be as informative against it as a uniform value near 11. This is the cheapest disconfirmation available anywhere in the program, and the staging of Sec. 8 exists so that it is taken first. 7.4. Certifying the Candidate Gradings Step (5) costs one inner product per candidate. By Rem. 17, a proposed grading qinitq^init enters the admissible cone +Q_+ of Prop. 8, to first order, exactly when ⟨qinit,α^−τ^⟩>0 q^init, α- τ >0, and the program reports the quantity for every arm of Tab. 2 before pretraining begins. A negative value falsifies that candidate and not the framework, and the corrected grades are then supplied by Sec. 7.5 by convex programming rather than by search. Two arms are settled in advance of the measurement, and in opposite directions. The uniform arm scores ⟨,α^−τ^⟩=0 0, α- τ =0 identically, whatever the profiles turn out to be, and by the gauge invariance of Prop. 8(i) so does every uniform grade cc1. This is not an artefact of the comparison but the content of Prop. 8(i): the standard transformer sits on ∂+ _+ with gradient 2(logλ)(τ−α)≠02( λ)(τ-α)≠ 0, so a strictly descending direction exists at its location and the profiles determine it. The isotropic architecture forgoes the gain of Prop. 7 not because the profiles are unfavourable to it but because its grade vector does not read them. Remark 24. The taxonomic arm is admissible a priori, before any estimate exists. By Rem. 9, under the level-stratified profiles of Cor. 3 a level-constant grading vj=vlv_j=v_l for j∈Vlj∈ V_l satisfies the criterion Eq. 15 exactly when ∑l=0L−1vl(al−sl)>0 _l=0^L-1v_l(a_l-s_l)>0, and under the geometric hypothesis this holds for every v affine and increasing in l, with no estimate of ν, of μ, or of the profiles at all. The linguistic ordering is therefore sufficient on its own: assigning higher grades to deeper levels, which is what the taxonomy of Sec. 7.1 does by construction, enters +Q_+. What the estimates of Sec. 7.2 buy is not admissibility but the location of the optimum within it, and the circularity of Rem. 23 therefore cannot reach the framework’s positive claim: it reaches only the sharpness of the grading, not its direction. 7.5. Computing the Educated Grading Step (6) is a convex program, and for language it is a small one. The unclipped problem of Cor. 4 and the clipped problem of Prop. 10 are posed in d variables, 10241024 at GLLM-Medium and 40964096 at GLLM-7B, but under the stratification the taxonomy imposes they collapse by Prop. 11 to the L-variable program ΛL _L, with L=4L=4 at the planned configurations. Prop. 11 is what makes the selection implementable rather than merely well posed. The program it names is solved by projected gradient descent with coordinatewise clipping as the projection, using the gradient supplied by the proof of Prop. 8 in level form, ∂vllogΛL(v)=2logλ(sl(v)−al(v)),al(v)=alλ−2vl∑kakλ−2vk,sl(v)=slλ2vl∑kskλ2vk, split ∂ v_l _L(v)&=2 λ\, (s^(v)_l-a^(v)_l ),\\ a^(v)_l= a_lλ^-2v_l _ka_kλ^-2v_k, s^(v)_l&= s_lλ^2v_l _ks_kλ^2v_k, split which is the difference of two tilted level profiles and vanishes at the interior optimum exactly when they coincide. Four variables, a closed-form gradient, a convex objective with no spurious stationary points by Prop. 8(i), and a projection that is a clip: the educated grading of Tab. 2 is the output of a computation that runs in milliseconds once (a^,s^)( a, s) are in hand. By Prop. 10(i) the solution has the saturated form, the fourth-root law of Prop. 7(i) on the interior levels and the box on the rest, and the solver is verified against that form. 7.6. The Predicted Effect Size The output of the procedure is a number the runs of Sec. 8 must detect, and the theory fixes it in closed form once the level profiles are named. Under the geometric hypothesis of Cor. 3 the profiles are governed by two parameters, ν for the concentration of the target towards the deepest level and μ for the decay of the data away from the shallowest, and step (4) of Sec. 6.1 estimates them. Tab. 3 records what Prop. 11 and Prop. 10 deliver across the range those estimates may fall in, at the planned L=4L=4 and C=2C=2. (ν,μ)(ν,μ) Λ⋆=BC(a,s)2 =BC(a,s)^2 ΛC⋆ _C at C=2C=2 token gain 1/ΛC⋆1/ _C gC⋆=(g0,g1,g2,g3)g _C=(g_0,g_1,g_2,g_3) (0.5,0.2)(0.5,0.2) 0.2790.279 0.4240.424 2.362.36 (1.00, 1.31, 2.00, 2.00)(1.00,\ 1.31,\ 2.00,\ 2.00) (0.6,0.3)(0.6,0.3) 0.4590.459 0.5310.531 1.881.88 (1.00, 1.30, 2.00, 2.00)(1.00,\ 1.30,\ 2.00,\ 2.00) (0.7,0.3)(0.7,0.3) 0.5340.534 0.5820.582 1.721.72 (1.00, 1.32, 1.95, 2.00)(1.00,\ 1.32,\ 1.95,\ 2.00) (0.8,0.4)(0.8,0.4) 0.7000.700 0.7080.708 1.411.41 (1.00, 1.27, 1.69, 2.00)(1.00,\ 1.27,\ 1.69,\ 2.00) Table 3. The gain at L=4L=4 and clip C=2C=2, as a function of the free parameters (ν,μ)(ν,μ) of Cor. 3. No corpus enters this table. It is arithmetic: Λ⋆ is the closed form of Cor. 3, and ΛC⋆ _C is the four-variable program of Prop. 11 solved on the box, with gC⋆=λqC⋆g _C=λ^q _C the resulting per-level multiplier. The pair (ν,μ)(ν,μ) is what step (4) of Sec. 6.1 estimates; the table says what each possible estimate would imply. Remark 25. Three observations follow from the table. The predicted effect is bounded and it is large enough to see. Across the range, the token gain runs from 1.4×1.4× to 2.4×2.4×, which is the quantity Sec. 8.2.2 measures and the magnitude the runs must be powered for. It is not the exponential regime of Cor. 3, and Rem. 18 says why: at C=2C=2 the exponential is unreachable by construction, and what Tab. 3 quantifies is the first point of that trajectory. The regime of the table, geometric level-stratified profiles at level-constant clipped grades, is moreover exactly the hypothesis set of Thm. 4.1, so each tabulated gain is attained up to constants depending only on (ν,μ)(ν,μ) as a ratio of minimax risks over all estimators, throughout the window of Lem. 4: the effect size the runs are powered for is backed by the two-sided separation, not only by the upper bounds of Prop. 7. The clip is the binding constraint, and it binds hardest exactly where the profiles diverge most. At (0.8,0.4)(0.8,0.4) the clipped gain is within about 1%1\% of the unclipped one; at (0.5,0.2)(0.5,0.2) the clip costs a third of it, and the top two levels saturate at g=Cg=C. The pattern is Rem. 18 made quantitative: the more the corpus rewards grading, the more the clip withholds, and the C-sweep of Sec. 8.1.1 is the arm that recovers it. The optimal grading is stable in a way the profiles are not. Across the whole range the multiplier on the syntactic level sits between 1.271.27 and 1.321.32, and the discourse level saturates at C throughout. The ordering of gC⋆g _C, increasing in l, is invariant across every entry, which is Rem. 24 recovered numerically: the estimates move the magnitudes and leave the direction alone. The bound ΛC⋆≥maxBC(α,τ)2,C−2 _C≥ \BC(α,τ)^2,C^-2\ of Prop. 10(i) is a lower bound and is not tight at these parameters; at (0.6,0.3)(0.6,0.3) it gives 0.4590.459 against the true 0.5310.531. The effect size the program is powered for is therefore the solution of the reduced program and not the bound. This is the operative use of Prop. 11: the bound is what the theory guarantees, and the four-variable program is what the experiment must detect. 7.7. Cost of the Offline Stage The accounting closes the case study. Steps (2) and (3) are one pass over a corpus for a covariance and one ridge regression on a treebank of order 10610^6 tokens. Step (4) is a sum of d square roots. Step (5) is one inner product per arm of Tab. 2, six in total. Step (6) is the four-variable convex program of Prop. 11. Against the 100100B tokens and 150150K steps of Sec. 8.1.5, the whole of it is free, and it is free in the sense that matters: it does not consume the resource the framework is spending its conditioning to buy. That asymmetry is the case study’s main point. The procedure of Sec. 6 publishes, before the first GPU-hour, a per-task affinity that can falsify Assumption 1, a certificate for each candidate grading, an ordering of the arms of Tab. 2, and the effect size of Tab. 3 that the runs must then detect. Whatever the runs return, the prediction they test was fixed in advance by a computation anyone can repeat, and that is a property the ungraded baseline cannot have, not because its profiles are unfavourable, but because the gauge invariance of Prop. 8(i) puts its admissibility inner product at 0 identically. 8. The Experimental Programme Sections 3, 4 and 5 developed the Graded Large Language Model: graded attention preserving the asymptotic cost of standard attention (Prop. 4) and compiling to a standard transformer at deployment (Cor. 1), multi-level graded embeddings with an exact sample-complexity accounting (Propositions 7 and 3) and a closed-form characterisation of the grades that purchase it (Prop. 8), and a graded training objective whose consistency conditions are characterised in Prop. 9. This section specifies the training program that will test the framework, states the predictions the theory supports, and marks explicitly the hypotheses on which each depends. No experiments have been conducted at the time of writing, and this section reports none. The program is staged so that the cheapest test comes first. Its first stage is a measurement rather than an experiment: both profiles of Def. 2 can be estimated offline, so the Bhattacharyya affinity BC(α,τ)BC(α,τ) that Prop. 7 identifies with the entire gain of grading is reported per task, and every candidate grading is certified against Eq. 15 at the cost of one inner product, before any model is pretrained. That stage is the selection procedure of Sec. 6, carried out for language in Sec. 7: the arms of the ladder are certified, the educated grading qC⋆q _C is computed by Propositions 10 and 11, and the effect size the runs below must detect is fixed in advance. The present section takes those quantities as given: it fixes the configurations, pretrains at 345M and 7B, and evaluates. A framework whose governing hypothesis is measurable before its first GPU-hour should be tested in that order. We state the status of each prediction explicitly. Several predictions, in particular any reduction in the data required to reach a given generalisation target, follow from Propositions 7 and 8 only under hypotheses that are empirical, not mathematical: that the energy profiles of language diverge (Assumption 1), and that the scope limitations of Rem. 11 do not overturn the single-layer analysis at the full stack. Those predictions are marked conditional below and name the hypotheses they depend on. 8.1. Planned Experimental Setup 8.1.1. Model configurations Two scales are planned. GLLM-Medium (∼ 345M parameters) mirrors GPT-2-medium at 24 layers, d=1024d=1024, H=16H=16, with EG-MHSA substituting standard MHSA and MLGE substituting standard embeddings at L=4L=4 levels and uniform allocation dl=256d_l=256. GLLM-7B (∼ 7B parameters) mirrors LLaMA-7B at 32 layers, d=4096d=4096, H=32H=32, with L=4L=4 and dl=1024d_l=1024. Both use the base λ=e1/dkλ=e^1/d_k fixed in Sec. 3.1, with grade clipping enforcing λqmax≤Cλ^q_ ≤ C at C=2C=2. The clip is the framework’s single cost–benefit parameter (Rem. 18): it caps the optimisation penalty of Thm. 5.1 at a factor of C, uniformly in L and in the profiles, and caps the token gain at C2C^2 by Prop. 10. At GLLM-Medium the clip is swept over C∈1.5,2,4C∈\1.5,2,4\ as an explicit arm, tracing the price–purchase curve of Rem. 18 rather than reporting a single point on it. 8.1.2. Grade initialisation Grades are initialised from part-of-speech statistics computed offline over the pretraining corpus, assigning high grades to open-class types and low grades to closed-class function words, and are certified against Eq. 15 as in Sec. 7.4. Per-type initialisation is not a matter of convenience: Prop. 9(3) establishes that a grade depending on the realised target and its context jointly, a constituency parse depth for instance, renders the training objective inconsistent for the corpus conditional, and by Rem. 14 the resulting context-dependent tilt admits no matching by the fixed output grades. Per-type grades fall under Prop. 9(2), the case in which the matching is available. A parse-depth initialisation appears only in the ablation arm described in Sec. 5.1. 8.1.3. Parameter overhead The overhead comprises the projection matrices Pl\P_l\ at ∑lddl=d2 _ld\,d_l=d^2 parameters, the head grades qh\q_h\ at KHdkKHd_k, the level grades ql\q_l\ at d, and the output grades qoutq_out at |||V|, so the total is O(d2+KHdk+||)O(d^2+KHd_k+|V|) and is dominated by d2d^2. At GLLM-Medium this is 1.05M+24.6K+1K+50.3K≈1.13M1.05M+24.6K+1K+50.3K≈ 1.13M, or 0.33%0.33\% of 345M345M; at GLLM-7B it is 16.8M+131K+4K+32K≈17.0M16.8M+131K+4K+32K≈ 17.0M, or 0.24%0.24\% of 7B7B. The overhead is below one percent at both scales and negligible against the 12d212d^2 per block of Prop. 1, but this follows from d2≪Nd^2 N at these configurations rather than from any sublinearity in d, and must be recomputed rather than assumed at other scales. By Corollaries 1 and 3 the figure is a training cost only, and it is an upper bound on the lifetime cost of the prior. 8.1.4. Baselines and the ladder The arms are those of the selection ladder, Tab. 2: uniform (the standard transformer), random-grade, taxonomic, part-of-speech, educated (qC⋆q _C), and oracle, all of identical architecture, data, compute, and optimisation budget, distinguished only by what their grades consulted. The uniform arm is realised as identically sized GPT-2-Medium and LLaMA-7B models with standard MHSA and embeddings. The random-grade arm is the control that isolates the prior from the parameterization: by Prop. 8(iv) a grade direction drawn independently of α−τα-τ has ⟨v,α−τ⟩ v,α-τ of either sign, so it is predicted to straddle the uniform arm rather than to improve on it, and any gain the informed arms show over it is attributable to the prior and not to the extra parameters of Sec. 8.1.3. Two further baselines sit outside the ladder: a probe-augmented uniform model, adding syntactic probing classifiers over frozen representations to indicate what post-hoc structural annotation recovers without architectural grading, and a linear-grading variant evaluated alongside the exponential to separate their contributions. 8.1.5. Training GLLM-Medium is pretrained on a 100B-token subset of The Pile [gao2020pile] for 150K steps at batch size 512. GLLM-7B is pretrained on 300B tokens. Both use AdamW [loshchilov2017decoupled] with ηmax=6×10−4 _ =6× 10^-4 and the cosine schedule of [loshchilov2017sgdr], under the objective Eq. 20 with the boundedness condition Eq. 21 of Sec. 5.2. Distributed training uses sharded data parallelism with optimiser-state partitioning [rajbhandari2020zero] and pipeline parallelism [huang2019gpipe]. 8.1.6. Context length Training sequence lengths must be set by the evaluation the configuration is intended to support. Discourse-level grading at l=3l=3 is the setting in which MLGE is expected to matter most, and it cannot be assessed at a context of 1024 or 2048 tokens; nor does the graded sinusoidal encoding Eq. 16 extrapolate beyond training length. Long-context evaluation therefore requires training at the target context, with the compute that implies, and a rotary or otherwise extrapolable graded positional encoding. Absent that, the long-context benchmarks below are out of scope for these configurations and the corresponding predictions are untestable. 8.2. Planned Benchmarks 8.2.1. Intrinsic metrics Held-out perplexity Eq. 2 on Penn Treebank and WikiText-103 gives short-range coverage; PG-19 gives long-document coverage at contexts the configurations support. Perplexity is the unweighted loss, and the graded model does not optimise it. The configurations of Sec. 8.1.2 enforce the matched-tilt condition of Rem. 14, under which the graded and unweighted objectives share a population minimizer and the perplexity comparison is therefore made on equal terms. The parse-depth arm of the component ablation of Sec. 8.2.5 is the single arm in which that matching is unavailable, by Prop. 9(3), and there any perplexity gap is attributed to the grade-tilt, the Kullback–Leibler divergence between the corpus conditional and its context-dependent tilt, and not to the inductive bias. Every perplexity comparison states which condition holds. 8.2.2. Sample efficiency The quantity Prop. 7 speaks to is tokens, not steps, and the program measures it directly. For each task of Sec. 8.2.3 we record the held-out error of every arm as a function of pretraining tokens consumed, and report the ratio of token counts at which each first attains a fixed target error. Prop. 7(i) predicts that ratio to be Λ(g) (g), estimated in advance by Sec. 7.2 and bounded below by C−2C^-2 at the clipped configurations (Prop. 10); the scope of Rem. 11 means the prediction is of the ordering and the sign, not of the constant. This is the measurement on which the framework’s central claim stands, and it is reported separately from steps-to-loss for the reason given in Rem. 16. 8.2.3. Hierarchical tasks CoLA (linguistic acceptability) and MultiRC (multi-sentence reading comprehension) are the primary targets, rewarding syntactic sensitivity and compositional inference respectively. BoolQ, CB, and WiC provide contrast at lower structural demand, distinguishing gains specific to hierarchy from gains distributed across tasks. 8.2.4. Long-context tasks Subject to the context-length condition of Sec. 8.1.6, SCROLLS [shaham2022scrolls], NarrativeQA, and QUALITY test discourse-level grading. 8.2.5. Ablation protocol Two ablations run orthogonally. The first isolates each graded component: EG-MHSA alone; MLGE alone; graded loss alone; and their combinations. It also carries the parse-depth arm, the intentionally uncalibrated grading of Sec. 8.1.2, retained to measure the tilt cost that Prop. 9(3) predicts. The second is the selection ladder of Tab. 2, varying only the provenance of the grades across the six arms of Sec. 8.1.4: uniform, random, taxonomic level-constant, POS-based, the educated qC⋆q _C of Prop. 10 computed from the estimated profiles, and oracle. Each arm reports its admissibility inner product ⟨qinit,α^−τ^⟩ q^init, α- τ from Sec. 7.4 alongside its result, so that the ladder tests Eq. 15 directly: the arms are predicted to order by that inner product, an arm with a negative value is predicted to underperform the uniform baseline, and the interpretation of each possible outcome is set out in Sec. 6.5. For the random arm the inner product is of either sign on any given draw, so its predicted position is fixed only once its draw is measured; the aggregate prediction is the straddle of Sec. 8.1.4. The level count is swept over L∈1,2,4,8,16L∈\1,2,4,8,16\; the choice L=4L=4 reflects a linguistic taxonomy, and the sweep is exploratory rather than confirmatory. 8.2.6. Interpretability Let goldA_gold be the set of gold dependency arcs in a held-out treebank and Ah,ijA_h,ij the attention weight from token i to token j in head h. Define the structural attention mass (23) SAM(h)=1|gold|∑(i,j)∈goldAh,ij,SAM(h)= 1|A_gold| _(i,j) _goldA_h,ij, the mean attention weight placed on gold arcs. This is a recall-like quantity, not a precision, and it is not normalised against the attention placed off gold arcs; it should be reported alongside the total attention mass per head. Grade-stratified accuracy Acc(l∗)Acc(l^*) partitions benchmark instances by maximum dependency nesting depth l∗l^*. 8.3. Predictions Every prediction below names the result it follows from. 8.3.1. Zero inference overhead (established). By Corollaries 1 and 3, the grading transformations are absorbed into the projection and embedding weights by an invertible change of parameters. A trained GLLM therefore compiles to a standard transformer of identical architecture and identical inference cost, and the 0.33%0.33\% and 0.24%0.24\% training overheads of Sec. 8.1.3 vanish entirely at deployment. This follows from the construction and is not subject to experimental uncertainty. By Rem. 2 it distinguishes grading from every variant of Tab. 1, each of which carries its inductive bias in the computation and pays for it at every forward pass. 8.3.2. The educated grading is certifiably admissible (established given the profile estimates). By Prop. 8(iv) and Rem. 17, a candidate grading enters the cone +Q_+ to first order exactly when ⟨qinit,α^−τ^⟩>0 q^init, α- τ >0, and we predict a strictly positive value for the POS and educated arms on CoLA and MultiRC. Only the sign must survive the estimation error of Sec. 7.2; the magnitudes need not. The ungraded baseline scores 0 on this quantity identically, by the gauge invariance of Prop. 8(i), and cannot score otherwise; Prop. 8(i) is the interpretation of that zero, as the boundary position of the uniform grades. The prediction is reported before pretraining and is falsified by a non-positive value. 8.3.3. Selection-ladder ordering (conditional on Assumption 1 and the scope of Rem. 11). The arms of Tab. 2 are predicted to rank, in the token-efficiency measurement of Sec. 8.2.2, in the order of their admissibility inner products, computed offline in Sec. 7 and published before training. This is the central prediction of the program and the sharpest, because an ordering fixed in advance is far harder to satisfy by accident than any single pairwise comparison. The partition of outcomes and what each would establish is given in Sec. 6.5. 8.3.4. Hierarchical task gains (conditional on Assumption 1 and the scope of Rem. 11). Gains are predicted to concentrate where the estimated BC(α,τ)BC(α,τ) is smallest, by Assumption 1 on CoLA and MultiRC, and to diminish on lexically dominated tasks, where the profiles approach agreement and Prop. 7(iv) forces Λ⋆→1 → 1. That contrast is what makes the gain attributable to hierarchy rather than to the parameterization, and the measurements of Sec. 7 turn it from a qualitative expectation into a quantitative one: the predicted ordering of per-task gains is the ordering of the measured affinities, fixed in advance of training, with each gain bounded by C2C^2 at the clipped configurations (Prop. 10). The measurement is that of Sec. 8.2.2, in tokens to a fixed error. The gain lives in the norm-controlled regime, which is the regime the runs occupy: Eq. 20 is optimised under the decoupled weight decay of Sec. 5.3, so the comparison of Prop. 7, with each class evaluated at the smallest budget at which it represents the target, is the one the experiment realises. 8.3.5. Depth-stratified robustness (conditional on Assumption 1 and the scope of Rem. 11). Given Prop. 7 and profiles that diverge in the sense of Assumption 1, the graded model’s accuracy is predicted to decay more slowly in nesting depth l∗l^* than the ungraded baseline’s, at equal data. The prediction is qualitative: no result in this manuscript supports a rate for Acc(l∗)Acc(l^*), and the sample-complexity ratio of Prop. 7 governs the data required to reach a target error, not the shape of the accuracy-versus-depth curve. The two are distinct quantities and should not be conflated. 8.3.6. Head specialisation (conditional on Eq. 20). The repulsive diversity term of Eq. 20 rewards distinct grade tuples, so heads are predicted to acquire distinguishable gradings and, if the linguistic prior is informative, to align differentially with dependency categories, observable as SAM(h)SAM(h) varying systematically with qhq_h. This is a prediction of the regulariser, not of the algebraic structure of EG-MHSA, which by Rem. 1 is silent on the question. Under an attractive penalty the prediction reverses. 8.3.7. Resource separation (established). Theorems 5.1 and 5 give ς≤λqmaxςunif+creg ≤λ^q_ _unif+c_reg with λqmax≥1λ^q_ ≥ 1: grading degrades the smoothness constant. We therefore predict that graded training does not reduce, and may increase, the optimisation steps to a fixed training loss at matched hyperparameters. The two resources are priced by the same constant and in the framework’s favour: the step-count penalty is capped at C by the grade clipping of Sec. 8.1.1, uniformly in L, while the token saving is capped at C2C^2 by Prop. 10, a price linear in the clip against a purchase quadratic in it (Rem. 18), with the exponential regime of Cor. 3 reached as the clip opens. The C-sweep of Sec. 8.1.1 traces this curve directly. Since steps are the cheap resource and high-quality tokens the expensive one, the trade favours the graded model at every setting of the clip. Any reduction in tokens to a fixed generalisation target arises from Propositions 7 and 8 and is visible as a gap in held-out rather than training loss. These are different measurements and the program reports them separately, in Sec. 8.2.2 and Sec. 8.2.1 respectively. 8.4. Remaining Theoretical Work The framework’s central quantity is settled in the regime the program occupies. Thm. 4.1 establishes the separation of Conj. 2 for level-stratified geometric profiles at squared loss: a minimax lower bound over the uniform target class, with the estimator unrestricted, met by the matching upper bound of Lem. 3 throughout the window of Lem. 4, so that on that window the ratio Λ(g) (g) of Prop. 7 is a genuine separation between the graded prior and its absence rather than a comparison of bounds. What remains is the general case: Conj. 2 for arbitrary profiles and Lipschitz losses, whose proof requires extending the packing argument of Lem. 3 beyond level-homogeneous designs. It is independent of the experiments above and could be completed before any of them. Three further directions follow from limitations recorded above. The first limitation of Rem. 11 calls for the layerwise extension of Prop. 7 through the graded stack. Sec. 4.5 records that the level-decomposed feed-forward block performs no cross-level mixing, and that directed transport between levels requires block-triangular maps rather than block-diagonal ones; the morphic structure of [sh-111] supplies them. And Sec. 8.1.6 makes an extrapolable graded positional encoding a precondition for testing discourse-level grading at all, which is the setting in which Cor. 3 predicts the largest effect at open clip. References