Paper deep dive
An Active-Bottleneck Mechanism for Weak-to-Strong Generalization
Mohammad Zeinalpour, Amir Najafi
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Weak-to-strong generalization (W2SG) occurs when a student trained on a teacher's predictions outperforms that teacher. We study when this happens under fully converged, ridgeless two-stage learning, with no early stopping, no explicit regularization, and no assumption that the student is more expressive than the teacher. In two-stage linear regression, a teacher is fit from $n$ labeled examples and a student is trained solely on the teacher's predictions on $m$ fresh, unlabeled inputs. Although both stages share the same hypothesis class and the same training rule, we show that the student outperforms the teacher exactly when $m$ lies in an explicit intermediate range: too few pseudo-labels leave the student without enough signal, too many let it inherit the teacher's noise. Under power-law covariance, we derive this range in closed form as a function of the spectral decay and noise level, including regimes where the improving region splits into two disjoint intervals of $m$. We then study a random-feature model in which the student has strictly more features than the teacher, and identify two regimes, again given by explicit thresholds: one where improvement occurs only for $m$ in a bounded interval, and one where it occurs only once the student width $N_S$ exceeds an explicit threshold. Both regimes are governed by a single "active-bottleneck" principle: whichever of $m$ or $N_S$ is scarcer controls how much teacher error is filtered out, while increasing the other resource only reduces estimation noise. Together, these results show that finite data and finite width can themselves regularize a two-stage learner, with no explicit mechanism doing so.
Tags
Links
- Source: https://arxiv.org/abs/2609.33835v1
- Canonical: https://arxiv.org/abs/2609.33835v1
Trouble viewing inline? Open PDF directly →
Full Text
302,488 characters extracted from source content.
Expand or collapse full text
An Active-Bottleneck Mechanism for Weak-to-Strong Generalization Mohammad Zeinalpour Amir Najafi †thanks: Corresponding author, Email: amir.najafi@sharif.edu Department of Computer Engineering, Sharif University of Technology Abstract Weak-to-strong generalization (W2SG) occurs when a student trained on a teacher’s predictions outperforms that teacher. We study when this happens under fully converged, ridgeless two-stage learning, with no early stopping, no explicit regularization, and no assumption that the student is more expressive than the teacher. In two-stage linear regression, a teacher is fit from n labeled examples and a student is trained solely on the teacher’s predictions on m fresh, unlabeled inputs. Although both stages share the same hypothesis class and the same training rule, we show that the student outperforms the teacher exactly when m lies in an explicit intermediate range: too few pseudo-labels leave the student without enough signal, too many let it inherit the teacher’s noise. Under power-law covariance, we derive this range in closed form as a function of the spectral decay and noise level, including regimes where the improving region splits into two disjoint intervals of m. We then study a random-feature model in which the student has strictly more features than the teacher, and identify two regimes, again given by explicit thresholds: one where improvement occurs only for m in a bounded interval, and one where it occurs only once the student width NSN_S exceeds an explicit threshold. Both regimes are governed by a single active-bottleneck principle: whichever of m or NSN_S is scarcer controls how much teacher error is filtered out, while increasing the other resource only reduces estimation noise. Together, these results show that finite data and finite width can themselves regularize a two-stage learner, with no explicit mechanism doing so. Contents 1 Introduction 2 Related Work 3 Preliminaries 4 Case I: Ridgeless Regression 4.1 Model and Training Procedure 4.2 Stage-I Projection Geometry 4.3 Power-Law Covariance and Two Regimes for m 4.4 Asymptotic W2SG Phase Diagram 5 Case I: The Random-Feature Model 5.1 Model and Two-Stage Training 5.2 Two Stage-I Resources, Three Regimes 5.3 Asymptotic W2SG Phase Diagram 6 Experimental Results 6.1 Case I: Ridgeless Regression 6.1.1 Phase Diagram 6.1.2 Finite-Size Scaling 6.2 Case I: Random-Feature Regression 6.2.1 Phase Diagram 6.2.2 Separation of the Two Phase Mechanisms 6.2.3 Negative Control 7 Conclusion References A Two-Stage Minimum-Norm Interpolation Theory A.1 Deterministic Equivalence A.2 Two-Stage Linear Regression with Gaussian Covariates A.3 Stage I: Ridgeless Regression A.4 Stage I Risk Derivation (Distilled Stage-I Sample) A.5 Proportional high-dimensional asymptotics A.5.1 Limiting Stage-I-to-Stage-I risk improvement A.6 Sparse-growing Stage-I sample size A.7 Ultra-sparse Stage-I extensions A.7.1 Fixed Stage-I sample count A.7.2 Analytic sign results for the fixed-m gain B Two-Stage Random-Feature Interpolation Theory B.1 Deterministic equivalence B.2 Defining Random-Feature Model B.3 Theory of DE for stage I B.4 Stage I: linear-random-feature derivation conditional on Stage I B.5 Positive-signal-fraction regime: assumptions and phase diagram B.6 Critical Boundaries and Finite-Dimensional Effects 1 Introduction As AI systems approach and eventually exceed human ability on complex tasks, humans will need to supervise models more capable than themselves. Burns et al. (2023) propose a way to study this problem today: fine-tune a strong pretrained model using only the predictions of a weaker, already fine-tuned one, and ask whether the strong model can outperform its supervisor. They call this weak-to-strong generalization (W2SG), and show that it occurs empirically — a GPT-4-level model supervised only by a GPT-2-level model can recover a substantial fraction of the performance gap between them. The finding is counterintuitive: the strong model is trained explicitly to fit the weak model’s labels, yet it does not merely imitate them. Existing theoretical explanations for W2SG typically rely on some form of asymmetry between the two stages. Either the student is assumed to have strictly greater representational capacity than the teacher (Xue et al., 2025; Dong et al., 2025); or the student is trained with heavier or differently structured regularization than the teacher (Moniri and Hassani, 2025); or the student is stopped early, before it has had the chance to fully absorb the teacher’s errors (Medvedev et al., 2025; Geng et al., 2026). Each of these mechanisms gives the student some structural advantage that the teacher does not share. This raises a natural question: is such an asymmetry necessary? Can a student with exactly the teacher’s capacity, trained by exactly the teacher’s rule, run to exact convergence with no ridge penalty at all, still outperform a teacher it was trained only to imitate? We show that the answer is yes, and that a single, simple resource is enough to cause it: the number of pseudo-labeled examples, m, that the student sees in its own training stage. In our first setting, both the teacher and the student solve ridgeless linear regression over the same hypothesis class, and the student is trained to convergence on m fresh inputs labeled by the teacher. Because m is finite, the student’s minimum-norm solution is constrained to an m-dimensional subspace of parameter space; it does not have room to reproduce the teacher exactly. This subspace can retain more of the teacher’s useful signal than its noise, or the reverse, depending on m. Under power-law covariance, we make this trade-off completely explicit: we solve for the critical values of m — as a fixed threshold, or as a limiting ratio m/pm/p when m scales with the ambient dimension p — that separate the regime where the student is worse than the teacher from the regime where it is better, and we show that this improving region can itself be a single interval, all of (0,p)(0,p), or two disjoint intervals of m, depending on the noise level and the covariance decay rate. This single-resource result suggests something more general is at work: a shared resource between the two training stages must be constrained to an intermediate range — tight enough to keep out the teacher’s error, loose enough to let the target signal through — for W2SG to occur. We call such a resource, whichever it turns out to be, an active bottleneck, and in our second setting we test this idea against a genuine capacity asymmetry. We give the student strictly more random features than the teacher, so that the student is a truly stronger model in the usual sense, and ask whether pseudo-sample size still matters once this asymmetry is present. It does, but its role now shares the stage with a second resource, student width. We identify two regimes, again with explicit thresholds: in one, improvement holds only when m falls inside a bounded interval, which we solve for exactly; in the other, improvement requires the student width NSN_S to exceed an explicit threshold, and now more pseudo-labels are beneficial rather than harmful. The two regimes are governed by a single principle: whichever resource, m or NSN_S, is scarcer at a given point in parameter space is the one that determines how much of the teacher’s error is filtered out; the other resource, once it is not the bottleneck, only reduces estimation noise and can be increased freely. This is the sense in which our two settings tell one story: Case I is the special case of this principle in which only one resource, m, is available to bind, and student and teacher share the same capacity; Case I makes the principle explicit and shows it survives, in modified form, once a genuine capacity asymmetry is introduced. Our emphasis on an intermediate, rather than merely large, pseudo-sample size is not only a theoretical curiosity: large-scale empirical studies of distillation scaling laws have observed the same non-monotonicity directly in trained language models, where increasing the number of teacher-labeled examples eventually collapses W2SG even between models with a large capacity gap (Busbridge et al., 2025). We discuss the relationship between this empirical finding and our theoretical account in Section 2. Contributions. • W2SG at ridgeless convergence with no capacity asymmetry. In two-stage linear regression with identical teacher and student hypothesis classes, we show that a student trained to convergence on the teacher’s own predictions can strictly outperform that teacher, with no early stopping and no explicit regularization at either stage. • An explicit sample-size phase diagram. Under power-law covariance, we solve for the critical pseudo-sample size(s) separating improvement from non-improvement, as fixed thresholds and as limiting ratios m/pm/p, and characterize when the improving region is a single interval, extends to the full overparameterized range, or splits into two disjoint intervals of m (Theorem 4.4). • A random-feature model with a genuine student advantage, and an active-bottleneck principle. We extend the analysis to random features with NS>NTN_S>N_T, and give explicit critical thresholds in m and in NSN_S separating the sample-bottleneck and feature-bottleneck regimes (Theorem 5.4, Corollary 5.6). Both regimes instantiate the same active-bottleneck principle: the scarcer of the two Stage-I resources controls teacher-error filtering, while the other resource only reduces estimation noise. We discuss the relationship of these results to the broader W2SG literature — in particular to alternative mechanisms based on early stopping, asymmetric regularization, representation geometry, and overparameterization, and to empirical scaling-law evidence at LLM scale — in Section 2. 2 Related Work A growing theoretical literature explains W2SG through different lenses. We organize it along three axes bearing directly on our contribution: what mechanism drives improvement, whether the student has a capacity advantage over the teacher, and which resource, if any, is identified as the source of the effect. Mechanisms proposed for W2SG. One line of work gives structural conditions guaranteeing improvement. Charikar et al. (2024) relate improvement to the student’s misfit to the teacher’s labels under realizability and convexity; Mulgund and Pabbaraju (2025) extend this to Bregman divergences, and Xu et al. (2025) give a bias–variance decomposition connecting improvement to how well the student approximates a posterior-mean teacher. Lang et al. (2024) use expansion-type conditions under which the student corrects and extends the teacher’s pseudo-label coverage, and Shin et al. (2025) study how easy/hard data overlap supports this. Somerstep et al. (2025) cast W2SG as latent-concept transfer. These frameworks isolate properties of a fixed predictor pair — misfit, coverage, alignment — sufficient for improvement. We instead fix a standard training procedure at both stages and ask how the resources supplied to it determine improvement, without assuming any property of the resulting predictors. A second line explains W2SG through training dynamics. Medvedev et al. (2025) show that early-stopped population gradient flow in random-feature regression yields improvement, because finite training time suppresses teacher error before the student fully absorbs it; Geng et al. (2026) show a related effect for finite-time logistic training. Both rely on stopping before convergence. We study the complementary regime: our student always converges, and the constraint that produces improvement is on the data or features it receives, not on training time. Symmetric versus asymmetric capacity. Several analyses assume the student has an advantage the teacher lacks. Moniri and Hassani (2025) show a student can compensate for a teacher that is under-regularized relative to it, so asymmetric regularization is itself the source of improvement. Xue et al. (2025) characterize teacher-error propagation through principal-kernel geometry, allowing the student’s representation to differ favorably from the teacher’s. Dong et al. (2025) derive an inherited-variance decomposition governed by teacher–student representation overlap, again asymmetric. Oh et al. (2026) let the student be a nonlinear network trained beyond a linear convolutional teacher. In each case, some asymmetry — regularization, representation quality, or nonlinearity — does part of the work. Our Case I removes this axis entirely: teacher and student share one hypothesis class, one ridgeless rule, and one convergence guarantee. Case I reintroduces capacity asymmetry deliberately, via NS>NTN_S>N_T, to test whether a resource-based mechanism survives alongside it. Resource-based explanations and scaling. Closest to our framing are analyses treating sample size or feature count as explicit, finite parameters. Ildiz et al. (2025) characterize surrogate predictors that improve a downstream ridgeless learner relative to training directly on the target; their benchmark is improvement over direct target training, whereas ours is improvement over the teacher Stage I actually produces. Wu et al. (2026) derive deterministic equivalents for random-feature ridge regression and find improved scaling exponents via a joint choice of sample size, width, and ridge; larger sample sizes and widths are generally beneficial once the ridge is tuned. Our Case I student is ridgeless throughout, and on the sample-bottleneck branch W2SG holds only inside a bounded interval of m — more pseudo-labels eventually destroy it. Wu and Sahai (2025) study a structurally similar two-stage ridgeless pipeline, but their source of improvement is a built-in asymmetry: the student’s covariates are strictly stronger than the teacher’s throughout. Our Case I needs no such asymmetry at all, and our Case I treats a representational advantage as a separate, explicit parameter analyzed jointly with m. Yao et al. (2026) show a shared pretrained initialization can induce W2SG in a single-index model — a different, complementary source of advantage rather than a competing explanation. None of these works identifies a single scarce resource governing a sharp transition between improvement and its absence. Overparameterization: helpful, harmful, or beside the point? A further group discusses overparameterization directly without explaining why it helps or hurts. Wu et al. (2026) show that, with ridge regularization, the interplay between the ridge and the degree of overparameterization can improve W2SG scaling laws. Dong et al. (2025) find, in kernel regression with an assumed teacher–student discrepancy, that W2SG occurs both overparameterized-ridgeless and underparameterized-with-ridge. Both leave open why overparameterization itself helps. Moniri and Hassani (2025) argue the opposite: without regularization, an overparameterized student loses Stage-I information that a regularized student would retain, which they treat as harmful. Our Case I agrees that information is lost — this is exactly Proposition 4.1 — but shows the loss is not intrinsically harmful. Whether it helps depends on which directions of parameter space it comes from, which depends on the covariance structure. Under power-law covariance, Theorem 4.4 shows this same loss produces >0 W2SG>0 throughout the overparameterized range for α≤2α≤ 2, and on an explicit sub-range for α>2α>2. Overparameterization is neither uniformly good nor bad here: it is the mechanism, and its effect depends on where the discarded information lives spectrally. Evidence from LLM-scale distillation. Busbridge et al. (2025) report an extensive empirical study of distillation scaling laws and find that the number of teacher-labeled examples m has a non-monotone effect: beyond a point, more pseudo-labels cause overfitting to the teacher and drive W2SG toward zero, even with a large capacity gap remaining. This matches the upper half of our own phase diagrams and is, to our knowledge, the first evidence that this shape holds at LLM scale rather than only in linear models. We do not see this as being in tension with our interval characterization: they report the shape empirically, with no mechanism or explicit threshold, which is exactly what Theorems 4.4 and 5.4 supply; and an interval with a finite upper endpoint already implies ”large m is eventually bad,” so the two claims agree rather than compete. What is new here. No prior analysis combines all three features of our setting: (i) teacher and student trained by the same procedure to exact convergence, with no early stopping or ridge; (i) in Case I, no capacity or regularization asymmetry; and (i) an explicit, closed-form characterization of the resource level — m in Case I, jointly m and NSN_S in Case I — separating improvement from its absence, rather than a sufficient condition or scaling exponent. A resource constraint shared between the two stages can itself regularize the student, and this constraint must be active — neither too loose nor too tight — for improvement to occur; the non-monotonicity of Busbridge et al. (2025) suggests this picture is not an artifact of the linear models we analyze. 3 Preliminaries Notation. For a positive integer d, let [d]:=1,…,d[d]:=\1,…,d\. Vectors are written using lowercase letters, such as x, β, and w, whereas matrices are written using uppercase letters, such as X, U, and Σ . All vectors are column vectors. We use A†A , range(A)range(A), and row(A)row(A) to denote the Moore–Penrose pseudoinverse, column space, and row space of A, respectively. The notation IdI_d denotes the d×d× d identity matrix, and diag(⋅)diag(·) denotes a diagonal or block-diagonal matrix. We use Tr(⋅)Tr(·) for the non-normalized trace of a matrix and tr(⋅)tr(·) for the normalized trace. For a symmetric matrix A, we write λ1(A)≥λ2(A)≥⋯ _1(A)≥ _2(A)≥·s for its eigenvalues in nonincreasing order. We use ∥⋅∥op\|·\|_op for the operator norm and ∥⋅∥Σ\|·\|_ for the covariance-weighted norm ‖v‖Σ2:=v⊤Σv\|v\|_ ^2:=v v. Subscripts T and S refer to teacher and student quantities, respectively. Unless stated otherwise, asymptotic relations are taken as p→∞p→∞. For positive sequences apa_p and bpb_p, we write ap∼bpa_p b_p if ap/bp→1a_p/b_p→ 1, ap=O(bp)a_p=O(b_p) if ap/bpa_p/b_p remains bounded, ap=o(bp)a_p=o(b_p) if ap/bp→0a_p/b_p→ 0, and ap=Θ(bp)a_p= (b_p) if both ap=O(bp)a_p=O(b_p) and bp=O(ap)b_p=O(a_p). Two-stage learning. We study a two-stage prediction pipeline that is common to both settings analyzed in this paper; each setting specializes it in a different way, and we defer every specialization — the choice of hypothesis class, learning rule, and noise model — to Sections 4 and 5. A target coefficient β⋆∈ℝpβ ^p is fixed throughout a given problem instance; it may itself be drawn from some distribution independently of everything else described below, in which case all expectations we define are understood to also average over β⋆β . An input x∈ℝpx ^p is drawn from a distribution PxP_x, and observations (x,y)(x,y) satisfy ψ⋆(x):=x⊤β⋆,y=ψ⋆(x)+ε,ψ (x):=x β , y=ψ (x)+ , (1) for some label noise ε , independent of x, whose distribution is specified separately in each case we study. Stage I produces a teacher predictor h^T∈ℋT h_T _T from n labeled observations n:=(xi,yi)i=1nD_n:=\(x_i,y_i)\_i=1^n drawn from (1), via some learning rule T A_T, so that h^T=T(n) h_T= A_T(D_n). The hypothesis class ℋTH_T may itself depend on auxiliary randomness independent of nD_n — for instance, a random feature embedding — in which case h^T h_T is understood to be conditional on that randomness as well. Stage I produces a student predictor h^S∈ℋS h_S _S using no labeled data of its own. Instead, m fresh, unlabeled inputs ~m:=x~ii=1m X_m:=\ x_i\_i=1^m are drawn i.i.d. from PxP_x, independently of nD_n and of any randomness underlying ℋTH_T, and are labeled by the teacher itself: y~i:=h^T(x~i) y_i:= h_T( x_i) for i=1,…,mi=1,…,m. Writing ~m:=(x~i,y~i)i=1m D_m:=\( x_i, y_i)\_i=1^m for the resulting pseudo-labeled dataset, the student is h^S:=S(~m)∈ℋS h_S:= A_S( D_m) _S for some learning rule S A_S; as with ℋTH_T, the class ℋSH_S may depend on auxiliary randomness independent of ~m D_m. Throughout, the student sees only the teacher’s predictions, never β⋆β , ε , or nD_n directly. Population risk and the W2SG gain. For a predictor h, its population risk against the target coefficient is ℛ(h,β⋆):=x∼Px[(h(x)−x⊤β⋆)2],R(h;β ):=E_x P_x [ (h(x)-x β )^2 ], (2) where the test point x is independent of all training data and of any auxiliary randomness underlying h. We write the corresponding averaged teacher and student risks as ℛ¯T:=[ℛ(h^T,β⋆)],ℛ¯S:=[ℛ(h^S,β⋆)], R_T:=E\! [R( h_T;β ) ], R_S:=E\! [R( h_S;β ) ], (3) where each expectation is taken over every source of randomness entering the corresponding predictor: β⋆β (if random), the Stage-I data nD_n and label noise, any randomness underlying ℋTH_T, and, for ℛ¯S R_S, additionally the Stage-I inputs ~m X_m and any randomness underlying ℋSH_S. The weak-to-strong generalization gain is :=ℛ¯T−ℛ¯S, W2SG:= R_T- R_S, (4) and we say weak-to-strong generalization occurs when >0 W2SG>0: the student, trained only on the teacher’s own predictions and with no access to labeled data, achieves strictly lower expected risk than the teacher it was trained to imitate. Sections 4 and 5 instantiate this template in two ways that differ in every specialized choice — hypothesis class, learning rule, and noise model — but share the risk and gain defined here. 4 Case I: Ridgeless Regression We first instantiate the two-stage template of Section 3 in the setting where the teacher and student share the same linear hypothesis class, the same feature space, and the same ridgeless training rule. Ridgeless linear regression is not a toy warm-up here: even in this classical setting, minimum-norm interpolation is known to exhibit genuinely surprising behavior — double descent, benign overfitting, and sharp phase transitions in the proportional asymptotic regime — and it has been the object of substantial recent theoretical attention in its own right (Hastie et al., 2022; Cheng and Montanari, 2024; Han and Xu, 2026; Wu and Xu, 2020; Richards et al., 2021; Loureiro et al., 2021). We use it here because it lets us isolate a single mechanism for W2SG in its purest form: with teacher and student identical in every respect except the data each one sees, any improvement of the student over the teacher must come entirely from the pseudo-sample size m. Prior analyses of W2SG in ridgeless regression have focused on representation mismatch, scaling behavior, and optimization dynamics between teacher and student (Dong et al., 2025; Wu and Sahai, 2025; Wu et al., 2026). We instead remove every such source of asymmetry: teacher and student share one feature space and one hypothesis class, and both are trained to exact convergence with zero ridge. We show that W2SG nonetheless occurs, governed entirely by m. 4.1 Model and Training Procedure We specialize the data-generating process of (1) to Gaussian covariates, x∼(0,Σ)x (0, ) with Σ∈ℝp×p ^p× p positive definite, and to Gaussian label noise ε∼(0,σ2) (0,σ^2) in Stage I, independent of x. The target coefficient β⋆β is random, independent of all training and test randomness, with [β⋆]=0,[β⋆β⋆⊤]=Ip.E[β ]=0, [β β ]=I_p. (5) We do not need to assume a specific distribution for β⋆β beyond these two moments: every quantity we study — the teacher and student risks, and the W2SG gain — is a linear or quadratic functional of β⋆β through [β⋆β⋆⊤]E[β β ] alone, so any isotropic distribution with these moments (Gaussian, uniform on a sphere of the matching radius, or otherwise) yields identical results. Both stages share the linear hypothesis class ℋ:=hβ:x↦x⊤β,β∈ℝpH:=\h_β:x x β,\ β ^p\. Given X=[x1|⋯|xn]⊤∈ℝn×pX=[x_1|·s|x_n] ^n× p and y=(y1,…,yn)⊤y=(y_1,…,y_n) , the teacher is the minimum-norm interpolator β^T=X†y,h^T(x)=x⊤β^T. β_T=X y, h_T(x)=x β_T. (6) Independently of Stage I, draw x~i∼i.i.d.(0,Σ) x_i i.i.d. 2.5[1]$ $N(0, ) for i=1,…,mi=1,…,m, form the noiseless pseudo-labels y~i=x~i⊤β^T y_i= x_i β_T, and write X~=[x~1|⋯|x~m]⊤ X=[ x_1|·s| x_m] , y~=(y~1,…,y~m)⊤ y=( y_1,…, y_m) . The student is again the minimum-norm interpolator of its own (pseudo-labeled) data, β^S=X~†y~=X~†X~β^T,h^S(x)=x⊤β^S. β_S= X y= X X β_T, h_S(x)=x β_S. (7) For Gaussian covariates, the population risk (2) of any hβ∈ℋh_β takes the standard closed form ℛ(hβ,β⋆)=‖Σ1/2(β−β⋆)‖22,R(h_β;β )=\| ^1/2(β-β )\|_2^2, (8) used throughout the ridgeless-regression literature cited above. The averaged risks (3) therefore specialize to ℛ¯T=β⋆,X,ε[ℛ(h^T,β⋆)],ℛ¯S=β⋆,X,ε,X~[ℛ(h^S,β⋆)]. R_T=E_β ,X, \! [R( h_T;β ) ], R_S=E_β ,X, , X\! [R( h_S;β ) ]. (9) 4.2 Stage-I Projection Geometry Because X~ X is drawn independently of everything determining β^T β_T — in particular, independently of X, ε , and β⋆β — we can study the statistical behavior of the Stage-I design and the statistical behavior of the teacher separately, and only combine them at the level of the final risk. This separation is what makes Lemma 4.2 below useful: it is a statement purely about X~ X, valid for any fixed β^T β_T, and can therefore be applied after averaging over Stage I. The minimum-norm student solution satisfies β^S=PX~β^T,PX~:=X~†X~, β_S=P_ X β_T, P_ X:= X X, (10) where PX~P_ X is the orthogonal projector onto the row space of X~ X. Under our Gaussian design, once m≥pm≥ p this projector is the identity almost surely, so the student exactly reproduces the teacher and the two-stage procedure is statistically inert. The only regime in which Stage I can do anything other than copy the teacher is therefore the overparameterized one, m<pm<p, and we restrict attention to it from here on. Proposition 4.1 (Necessity of a finite-sample bottleneck). If m≥pm≥ p, then β^S=β^T β_S= β_T almost surely and =0 W2SG=0. Hence W2SG in Case I requires m<pm<p. Proof. For m≥pm≥ p, the Gaussian design X~ X has full column rank almost surely because Σ≻0 0. Therefore X~†X~=Ip X X=I_p, so β^S=β^T β_S= β_T and =0 W2SG=0. ∎ When m<pm<p, PX~P_ X has rank exactly m: Stage I retains the component of β^T β_T in an m-dimensional random subspace — the row space of X~ X — and discards its component in the (p−m)(p-m)-dimensional orthogonal complement. The sample size m controls the dimension of this retained subspace; the realized covariates X~ X control its orientation. This is the source of the trade-off at the heart of Case I: increasing m lets more of the teacher’s useful signal reach the student, but by the same mechanism it also lets more of the teacher’s error reach the student. Covariance geometry biases which directions this random subspace preferentially retains. The next lemma makes this precise in expectation: retention is not a hard cutoff at the top m eigenvalues of Σ , but a soft preference for higher-variance directions, with a total retention budget fixed by m. Lemma 4.2 (Expected spectral retention). Let 1≤m<p1≤ m<p and Σ=Vdiag(λ1,…,λp)V⊤ =Vdiag( _1,…, _p)V , with V=[v1,…,vp]V=[v_1,…,v_p] orthogonal and λ1≥⋯≥λp>0 _1≥·s≥ _p>0. Define ℓi:=X~‖PX~vi‖22 _i:=E_ X\|P_ Xv_i\|_2^2. Then X~[PX~]=Vdiag(ℓ1,…,ℓp)V⊤,1>ℓ1≥⋯≥ℓp>0,∑i=1pℓi=m.E_ X[P_ X]=Vdiag( _1,…, _p)V , 1> _1≥·s≥ _p>0, _i=1^p _i=m. Proof is outlined in Appendix A(see Lemma A.6). Thus m sets a total expected retention budget across all p directions, while Σ determines how that budget is allocated: higher-variance directions are retained with no smaller expected weight than lower-variance ones, but every direction, including the top eigendirection, retains a strictly positive expected fraction below one. A finite-sample bottleneck therefore attenuates both signal and teacher error, never one exclusively; whether the net effect helps or hurts the student is exactly what the risk analysis below resolves. 4.3 Power-Law Covariance and Two Regimes for m We now specialize Σ to a power-law spectrum. Non-isotropy is essential to the mechanism: under an isotropic covariance, the projection of Lemma 4.2 retains every direction with the same expected weight m/pm/p, attenuating signal and teacher error identically, so W2SG cannot occur without an explicit ridge. A power-law spectrum gives the graded, coordinate-dependent separation between signal and noise-dominated directions that a finite-sample projection needs to filter one from the other. We use it both for this reason and because it is the standard testbed for learning curves and scaling laws in high-dimensional regression (Caponnetto and De Vito, 2007; Steinwart et al., 2009; Spigler et al., 2020; Cui et al., 2021; Bordelon et al., 2024; Defilippis et al., 2026; Wu et al., 2026), smoother than the spiked-covariance model used for a related purpose by Wu and Sahai (2025). We conjecture the same qualitative behavior holds for other sufficiently graded non-isotropic spectra, though not for every non-isotropic Σ . Assumption 4.3 (Power-law spectrum and asymptotic scaling). The population covariance has eigenvalues λi(Σ)=i−α _i( )=i^-α for i=1,…,pi=1,…,p, with 1<α<1+21<α<1+ 2. As p→∞p→∞, the Stage-I sample size satisfies p/n→ϕn∈(1,∞)p/n→ _n∈(1,∞), and the Stage-I sample size m follows one of two regimes: • Proportional regime: m→∞m→∞ with p/m→ϕm∈(1,∞)p/m→ _m∈(1,∞), so m=Θ(p)m= (p); • Fixed-sample regime: m is held at a fixed finite value as p→∞p→∞, so m/p→0m/p→ 0. These two regimes are not two arbitrary sub-cases: they probe genuinely different scales at which a critical pseudo-sample size can appear. A critical threshold that is O(1)O(1) in absolute terms is invisible in the proportional regime, where m is already growing linearly with p; a critical threshold that is Θ(p) (p) is invisible in the fixed-sample regime, where m never leaves a bounded range. We need both to see the full picture. The restriction α<1+2α<1+ 2 is a technical requirement for the phase-diagram analysis below (not for Lemma 4.2); we discuss α>1+2α>1+ 2, where the picture is only partially resolved, in Appendix A. 4.4 Asymptotic W2SG Phase Diagram Under Assumption 4.3, the teacher and student risks admit deterministic equivalents: asymptotically exact high-dimensional approximations, in the sense of Atanasov et al. (2026), that replace the random resolvents governing ℛ¯T R_T and ℛ¯S R_S by explicit functions of Σ and the two sample ratios (also, see Atanasov et al. (2025) for two point DE). We write pDE W2SG_p^DE for the resulting deterministic equivalent of the finite-p gain p W2SG_p; the full derivation, including the standard Stage-I risk formula in terms of a self-consistent ridge parameter τT _T and its associated coefficient γT∈(0,1) _T∈(0,1) (already known from the ridgeless-regression literature cited above), is given in Appendix A. Here we state only what is needed to read the theorem: γT _T is a scalar, determined by α and ϕn _n alone, that measures how much of the Stage-I estimator’s error is variance (driven by σ2σ^2) rather than bias. The two regimes of Assumption 4.3 produce different phase pictures, which we state informally here and prove in full in Appendix A (Theorems A.13, A.20, A.21 for the proportional regime; Theorems A.26, A.38, A.45 for the fixed-sample regime). In the proportional regime, m=Θ(p)m= (p) and we state results directly for the p→∞p→∞ limit DE:=limp→∞pDE W2SG^DE:= _p→∞ W2SG_p^DE; in the fixed-sample regime, m stays finite as p→∞p→∞ and we state results for the corresponding limit Gm:=limp→∞pDEG_m:= _p→∞ W2SG_p^DE, which still depends on m. Theorem 4.4 (W2SG phase diagram, informal). Suppose Assumption 4.3 holds and σ2>0σ^2>0. (a) Proportional regime (m=Θ(p)m= (p)). (i) If 1<α≤21<α≤ 2, then DE>0 W2SG^DE>0 for every ϕm>1 _m>1 — that is, for every pseudo-sample size m<pm<p growing proportionally with p, with no further condition on σ2σ^2 or on ϕm _m. (i) If 2<α≤1+22<α≤ 1+ 2, there is a unique critical ratio ϕm,crit _m,crit (solution of a fixed-point equation, see Appendix A), depending on α, ϕn _n, and σ2σ^2 but requiring no restriction on σ2σ^2, such that DE>0 W2SG^DE>0 if and only if ϕm<ϕm,crit _m< _m,crit, equivalently m>mcritFP(p)∼p/ϕm,crit=Θ(p)m>m_crit^FP(p) p/ _m,crit= (p). (b) Fixed-sample regime (m=O(1)m=O(1)). Here pDE W2SG_p^DE converges, as p→∞p→∞ with m fixed, to an explicit limit GmG_m depending only on m, α, and σ2γT/(1−γT)σ^2 _T/(1- _T). (i) If 1<α<21<α<2: for σ2σ^2 below an explicit threshold σc2(α,ϕn) _c^2(α, _n), Gm<0G_m<0 at m=1m=1; and Gm→(2−α)σ2γT/(1−γT)>0G_m→(2-α)σ^2 _T/(1- _T)>0 as m→∞m→∞, so at least one crossing m∗m_* exists. For σ2σ^2 below a second, smaller threshold σ02(α,ϕn)<σc2(α,ϕn) _0^2(α, _n)< _c^2(α, _n), this crossing is unique, and as σ2↓0σ^2 0 it is located at mcritHP∼[cα(1−γT)(2−α)σ2γT]1α−1,cα:=[παcscπα]α,m_crit^HP [ c_α(1- _T)(2-α)σ^2 _T ] 1α-1, c_α:= [ πα πα ]^α, (11) and pDE→Gm>0 W2SG_p^DE→ G_m>0 for every m>mcritHPm>m_crit^HP. (i) If 2<α≤5/22<α≤ 5/2: there is an explicit threshold σsafe2(α,ϕn) _safe^2(α, _n) such that, for σ2≤σsafe2(α,ϕn)σ^2≤ _safe^2(α, _n), Gm≤0G_m≤ 0 for every fixed m≥1m≥ 1: in this low-noise regime, no fixed pseudo-sample size produces W2SG at all. Explicit formulas for σsafe,σ0,σc _safe, _0, _c can be found in appendix A Two qualitative points are worth drawing out explicitly. First, part (a) shows that when the pseudo-sample size scales with the ambient dimension, a single, unconditional statement is possible for α≤2α≤ 2: W2SG is guaranteed for every m<pm<p, with no bottleneck fine-tuning needed at all; the bottleneck only becomes a genuine constraint, with a sharp critical threshold, once α>2α>2. Second, part (b) shows that the fixed- and proportional-sample regimes are not merely two proof techniques for the same fact — they can disagree qualitatively. For 2<α≤5/22<α≤ 5/2 and sufficiently small noise, no fixed m ever produces W2SG, yet part (a)(i) guarantees a Θ(p) (p)-scale pseudo-sample size that does. The bottleneck effect, in this regime, is a genuinely proportional phenomenon: it requires m to grow with p, not merely to be large in absolute terms. Remark 4.5 (Beyond the small-noise threshold, and disconnected regions). The explicit formula (11) for mcritHPm_crit^HP is an asymptotic statement as σ2↓0σ^2 0, though in practice it tracks the true critical m closely even at moderate noise levels. For noise above the thresholds in Theorem 4.4, teacher error can be concentrated enough near specific high-variance directions — rather than spread smoothly across the spectrum — that the Stage-I projection improves on the teacher even at very small m. Combined with the proportional-scale threshold of part (a), the set of pseudo-sample sizes producing W2SG can then split into a window near m=O(1)m=O(1) and a separate window near m=Θ(p)m= (p). This is not merely an artifact of two disconnected proof techniques: the intermediate scaling m→∞m→∞ with m/p→0m/p→ 0, analyzed in Appendix A (Section A.6), contains no critical m at all, so our fixed-sample and proportional results together already give a complete picture across every scaling of m with p. 5 Case I: The Random-Feature Model Case I shows that a representational advantage is not necessary for W2SG: with teacher and student sharing one hypothesis class, the pseudo-sample size m alone was enough to act as the bottleneck. We now ask a different question. Suppose the student genuinely is a stronger model, with access to a richer representation than the teacher. Does the bottleneck mechanism survive this asymmetry, and if so, is m still the resource that plays the bottleneck role, or can that role pass to a different resource entirely? This section shows that the mechanism survives, and that the bottleneck can indeed be either resource: the pseudo-sample size m, as in Case I, or the student’s feature width NSN_S, which has no counterpart in Case I at all. Which resource is active, and where the handoff between them occurs, is the content of Theorem 5.4 below. 5.1 Model and Two-Stage Training We adopt the linear random-feature model of Medvedev et al. (2025), which gives teacher and student each their own independently sampled random-feature map — a standard device for endowing a linear model with a controllable, finite amount of representational capacity, and the natural setting in which to let the student have genuinely more capacity than the teacher. We keep their model exactly, but ask a different question of it: rather than showing that early stopping in Stage I produces W2SG, as Medvedev et al. (2025) do, we train the student to full convergence and ask whether the finiteness of its two resources — pseudo-sample size and feature width — is by itself enough. This makes our result complementary to theirs: two different mechanisms, early stopping versus a resource bottleneck, both sufficient for W2SG in the same underlying model. Target and covariance structure. Let x∼(0,Σ)x (0, ), where Σ:=diag(Ik,δpIp−k) :=diag(I_k, _pI_p-k): the first k coordinates have unit variance and the remaining p−kp-k have variance δp∈(0,1) _p∈(0,1), a sequence that we allow to depend on p and whose asymptotic rate (relative to p−kp-k) will matter once we take p→∞p→∞ in Section 5.3. We refer to the first block as the strong coordinates and the second as the weak coordinates. The target coefficient β⋆∈ℝpβ ^p is deterministic — unlike Case I, where β⋆β was random with isotropic second moment — and is supported entirely on the strong block, so βi⋆=0 _i =0 for i>ki>k: the signal we want the student to recover lives entirely in the high-variance directions, while the weak block is pure background. We normalize ‖β⋆‖Σ2=1\|β \|_ ^2=1. There is no label noise in this section (ε≡0 ≡ 0 in (1)); the only source of teacher error here is a finite random-feature representation, not observation noise, and isolating this source is the point of the model. Stage I: teacher. Let UT∈ℝNT×pU_T ^N_T× p have i.i.d. (0,1)N(0,1) entries; UTU_T is drawn once and then held fixed throughout Stage I — it plays the role of a fixed, randomly initialized feature map, not a parameter that Stage I’s learning rule optimizes over. Conditional on UTU_T, the teacher’s feature extractor is fT(x)=UTx∈ℝNTf_T(x)=U_Tx ^N_T, and its hypothesis class is ℋT(UT):=hT,w:x↦x⊤UT⊤w:w∈ℝNTH_T(U_T):=\h_T,w:x x U_T w:w ^N_T\. Rather than fitting n noisy labeled samples as in Case I, the teacher here is idealized as the population risk minimizer within ℋT(UT)H_T(U_T) — the n→∞n→∞ limit of Stage I training, which lets us isolate representation bias as the sole source of teacher error, uncontaminated by any Stage-I estimation variance: wT⋆:=argminw∈ℝNT‖w‖2subject toUTΣUT⊤w=UTΣβ⋆.w_T := _w ^N_T~\|w\|_2 to U_T U_T w=U_T β . (12) The induced coefficient in the original input space is βT⋆:=UT⊤wT⋆ _T :=U_T w_T , and the teacher predictor is hT⋆(x):=x⊤βT⋆h_T (x):=x _T . Because wT⋆w_T solves the exact population problem, hT⋆h_T ’s only error, conditional on UTU_T, is approximation bias from the finiteness of NTN_T; the only remaining randomness, across different draws of UTU_T, is randomness in which NTN_T-dimensional representation the teacher happened to receive, not sample noise. Stage I: student. Independently of UTU_T, draw fresh covariates x~1,…,x~m∼i.i.d.(0,Σ) x_1,…, x_m i.i.d. 2.5[1]$ $N(0, ) and assign pseudo-labels y~i:=hT⋆(x~i)=x~i⊤βT⋆ y_i:=h_T ( x_i)= x_i _T , giving the Stage-I dataset ~m D_m of Section 3, with design matrix X~ X and label vector y~ y. The student is given its own independent random-feature map, US∈ℝNS×pU_S ^N_S× p with i.i.d. (0,1)N(0,1) entries, independent of UTU_T and of ~m D_m, with feature extractor fS(x)=USxf_S(x)=U_Sx and hypothesis class ℋS(US)H_S(U_S) defined analogously to ℋT(UT)H_T(U_T). This is the crucial departure from Case I: the student’s representation is no longer tied to the teacher’s, and the student’s width NSN_S is a resource entirely under our control, with no counterpart on the teacher side. Once NSN_S is allowed to differ from NTN_T — in particular, once NS>NTN_S>N_T — the student is a genuinely stronger model in the ordinary sense of having more capacity, and it is exactly this second resource, alongside m, that gives Case I its two possible bottlenecks. Because y~ y is generated by hT⋆∉ℋS(US)h_T _S(U_S) in general, it need not be exactly interpolable by the student; Stage I therefore performs minimum-norm least-squares regression, rather than minimum-norm interpolation as in Case I: w^S=argminw∈ℝNS‖w‖2subject tow∈argminv∈ℝNS‖X~US⊤v−y~‖22, w_S= _w ^N_S~\|w\|_2 to w∈ _v ^N_S~ \| XU_S v- y \|_2^2, (13) with induced coefficient β^S:=US⊤w^S β_S:=U_S w_S and student predictor h^S(x):=x⊤β^S h_S(x):=x β_S. Risks. The population risk (2) again takes the closed form ℛ(hβ,β⋆)=(β−β⋆)⊤Σ(β−β⋆)R(h_β;β )=(β-β ) (β-β ), so the averaged risks of (3) specialize to ℛ¯T=UT[ℛ(hT⋆,β⋆)],ℛ¯S=UT,US,~m[ℛ(h^S,β⋆)], R_T=E_U_T\! [R(h_T ;β ) ], R_S=E_U_T,U_S, X_m\! [R( h_S;β ) ], (14) where ℛ¯S R_S additionally averages over Stage-I sample randomness and over the student’s own representation USU_S. 5.2 Two Stage-I Resources, Three Regimes As in Case I, whether Stage I can do anything nontrivial at all is governed by a rank condition — but here there are two independent Stage-I quantities, m and NSN_S, that can each be the scarcer one relative to p. Definition 5.1 (Stage-I bottleneck regimes). The Stage-I regime is determined by which of m, NSN_S, and p is smallest: • the sample-bottleneck regime: m<minNS,pm< \N_S,p\; • the feature-bottleneck regime: NS<minm,pN_S< \m,p\; • the no-bottleneck regime: p<minm,NSp< \m,N_S\. Proposition 5.2 (No-bottleneck regime). If p<minm,NSp< \m,N_S\, then h^S=hT⋆ h_S=h_T almost surely, so ℛ¯S=ℛ¯T R_S= R_T and =0 W2SG=0. Proof. Since m>pm>p and NS>pN_S>p, both X~∈ℝm×p X ^m× p and US∈ℝNS×pU_S ^N_S× p have full column rank almost surely; hence range(US⊤)=ℝprange(U_S )=R^p, and because y~=X~βT⋆ y= X _T , the unique minimizer of the Stage-I loss is β^S=βT⋆ β_S= _T , giving h^S=hT⋆ h_S=h_T , ℛ¯S=ℛ¯T R_S= R_T, and =0 W2SG=0. ∎ Exactly as in Case I, then, W2SG is only possible once at least one Stage-I resource is scarce relative to p — both being abundant just reproduces the teacher exactly. What Case I adds is that this scarcity can come from either resource, and the two subsections below show that it produces genuinely different behavior depending on which one it is. 5.3 Asymptotic W2SG Phase Diagram We work under the same style of deterministic-equivalent analysis introduced in Section 4.4, now applied jointly to the two Stage-I resources, and let all five relevant quantities grow proportionally with p. Assumption 5.3 (Proportional asymptotic scaling). Let dp:=p−kd_p:=p-k, and assume kp→ϑ,dpδp→η,NTp→ρT,mp→ρm,NSp→ρS, kp→ , d_p _p→η, N_Tp→ _T, mp→ _m, N_Sp→ _S, (15) where ϑ,ρT∈(0,1) , _T∈(0,1), η∈(0,∞)η∈(0,∞), and ρm,ρS∈(0,∞) _m, _S∈(0,∞). We consider the regular fixed-ratio regime, in which ρT≠ϑ _T≠ , ρm≠ρS _m≠ _S, ρm,ρS≠ϑ _m, _S≠ , and ρm,ρS≠1 _m, _S≠ 1; the nontrivial excluded boundary cases are treated in Section B.6 of Appendix B. Here ϑ is the asymptotic fraction of coordinates carrying signal, and ρT,ρm,ρS _T, _m, _S are the three Stage-I-relevant resource ratios of Definition 5.1, now expressed in the proportional limit. As in Case I, we use deterministic equivalents: pDE W2SG_p^DE denotes the high-dimensional asymptotic approximation to the finite-p gain p W2SG_p, in the same sense introduced in Section 4.4; the full derivation is in Appendix B. Theorem 5.4 (W2SG phase diagram). Suppose Assumption 5.3 holds. Define ΛT:=ϑ−ρT,ρT<ϑ,(1−ϑ)(ρT−ϑ)1−ρT,ρT>ϑ. _T:= cases - _T,& _T< ,\\[2.84526pt] (1- )( _T- )1- _T,& _T> . cases (16) (i) Necessary condition. If minρm,ρS<ϑ \ _m, _S\< , then pDE<0 W2SG_p^DE<0: if either Stage-I resource falls below the signal fraction, the student cannot even recover the signal, let alone improve on the teacher. (i) Sample-bottleneck regime (ϑ<ρm<min1,ρS < _m< \1, _S\, so m, not NSN_S, is the scarcer Stage-I resource). Let ΔDE:=(ρS+3ϑ−ΛT)2−16ϑρS _DE:=( _S+3 - _T)^2-16 _S. If ΔDE>0 _DE>0, then pDE>0⇔ρm∈(ρS+3ϑ−ΛT−ΔDE4,ρS+3ϑ−ΛT+ΔDE4)∩(ϑ,min1,ρS). W2SG_p^DE>0 _m∈ ( _S+3 - _T- _DE4, _S+3 - _T+ _DE4 )∩ ( , \1, _S\ ). (17) If ΔDE≤0 _DE≤ 0, there is no positive region on this branch. This is the direct analogue of the Case I phase diagram: m must lie in a bounded window for the student to improve on the teacher. (i) Feature-bottleneck regime (ϑ<ρS<min1,ρm < _S< \1, _m\, so NSN_S, not m, is now the scarcer Stage-I resource). Here pDE>0⇔ρS−ϑ>ΛTandρm>2ρS(ρS−ϑ)ρS−ϑ−ΛT. W2SG_p^DE>0 _S- > _T _m> 2 _S( _S- ) _S- - _T. (18) Unlike (i), the condition on ρm _m is a half-line: once NSN_S is the active bottleneck and satisfies ρS−ϑ>ΛT _S- > _T, more pseudo-labels are unambiguously good, and W2SG occurs for every m large enough. The handoff between the two regimes in the theorem is governed entirely by which of m,NSm,N_S is smaller relative to p: regime (i) is exactly ρm<ρS _m< _S (with both above ϑ ), and regime (i) is exactly ρS<ρm _S< _m. There is no third, intermediate mechanism — whichever resource is scarcer is the one whose value must be tuned into a window (or past a threshold) for W2SG to occur, and the theorem’s two branches are this dichotomy made precise. Remark 5.5 (Reading the phase diagram). Three consequences of Theorem 5.4 are worth stating explicitly. First, combining (i) and (i), any positive-W2SG region forces ρS>ϑ+ΛT>ρT _S> + _T> _T: the student must have asymptotically more random features than the teacher. This is not an assumption we imposed — it falls out of the theorem. It confirms that a genuine capacity advantage is necessary for W2SG once representations differ, but the theorem shows it is not sufficient: satisfying ρS>ϑ+ΛT _S> + _T only opens the door to a bottleneck condition on m still needing to hold. Second, the qualitative character of the two regimes differs, not just their formulas. In the sample-bottleneck regime (i), the pseudo-sample size itself must be tuned to a bounded window, exactly as in Case I. In the feature-bottleneck regime (i), by contrast, m only needs to clear a threshold, after which it is purely beneficial: the sample-side trade-off of Case I disappears once NSN_S takes over as the active constraint. Third, this asymmetry reflects a general division of labor between the two resources: whichever resource is the active bottleneck governs how much of the teacher’s approximation error the student filters out, while the other resource, once it is not the bottleneck, only reduces estimation noise around whatever the bottleneck resource already lets through — which is why it can be safely increased without limit. The next corollary rearranges Theorem 5.4 to hold ρm _m fixed and vary ρS _S instead, exhibiting the same two regimes from the feature-width side. Corollary 5.6 (Phase diagram in student width). Suppose the conditions of Theorem 5.4 hold and fix ρm>ϑ _m> . (i) Feature-bottleneck regime (ϑ<ρS<min1,ρm < _S< \1, _m\). Let DS:=(ρm+2ϑ)2−8ρm(ϑ+ΛT)D_S:=( _m+2 )^2-8 _m( + _T). If DS>0D_S>0, pDE>0⇔ρS∈(ρm+2ϑ−DS4,ρm+2ϑ+DS4)∩(ϑ,min1,ρm); W2SG_p^DE>0 _S∈ ( _m+2 - D_S4, _m+2 + D_S4 )∩ ( , \1, _m\ ); (19) if DS≤0D_S≤ 0, there is no positive region on this branch. The feature-bottleneck positive region is thus a bounded interval in ρS _S — too little width starves the student, too much lets it overfit the teacher’s bias. (i) Sample-bottleneck regime (ϑ<ρm<min1,ρS < _m< \1, _S\). If ρm≤2ϑ _m≤ 2 , there is no positive region on this branch. If ρm>2ϑ _m>2 , pDE>0⇔ρS>ρm(2ρm−3ϑ+ΛT)ρm−2ϑ. W2SG_p^DE>0 _S> _m(2 _m-3 + _T) _m-2 . (20) The sample-bottleneck positive region is thus a half-line in ρS _S: once m is already the active bottleneck, additional student width is purely beneficial. Corollary 5.6 is the mirror image of Theorem 5.4: interval versus half-line swaps sides depending on which variable is held fixed and which is scanned, but the underlying division is the same one identified in Remark 5.5 — whichever resource is currently the bottleneck needs a window; the other only needs to clear a threshold. Together, Theorem 5.4 and Corollary 5.6 are what we mean by the active-bottleneck principle: the scarcer of the two Stage-I resources determines how much teacher error the student filters out and must sit at an intermediate value to do so well, while the other resource, once identified as non-bottleneck, can be increased freely and only serves to reduce estimation noise. This is the sense in which Case I is the single-resource special case of the same phenomenon: with only m available to bind, there is no ”other resource” to play the free role, and the sample-bottleneck branch of Theorem 5.4 is what survives. 6 Experimental Results We validate the theory against Monte Carlo simulations by comparing the finite-p deterministic equivalents with empirically measured W2SG. The experiments are organized around the two theoretical settings studied in the paper. For Case I, we examine the ridgeless-regression phase diagram on the two sides of the critical exponent α=2α=2 and then study the finite-size behavior associated with the fixed-m and proportional scalings. For Case I, we validate the random-feature phase diagram, separate the sample- and feature-bottleneck mechanisms, and include a negative-control experiment in which the theory predicts no admissible positive-W2SG region. (a) σ2=0.5σ^2=0.5. (b) σ2=2σ^2=2. Figure 1: Case I with 1<α<21<α<2. Numerical validation at α=1.8α=1.8, p=400p=400, and n=200n=200 using 10001000 Monte Carlo trials. In each subfigure, the top panel compares the finite-p deterministic equivalent with empirical W2SG and its 95%95\% confidence interval; the bottom-left panel shows the fixed-m limit GmG_m; and the bottom-right panel shows the proportional-regime limit as a function of the Stage-I sampling ratio. 6.1 Case I: Ridgeless Regression We first consider the ridgeless-regression setting of Theorem 4.4. We examine the W2SG phase behavior on the two sides of the critical exponent α=2α=2, comparing the finite-p deterministic equivalent with Monte Carlo simulations and with the fixed-m and proportional asymptotic limits. We then study finite-size scaling to illustrate how these two asymptotic regimes emerge under different scalings of the Stage-I sample size m. 6.1.1 Phase Diagram We first validate the Case I deterministic equivalent and phase diagram at p=400p=400 and n/p=0.5n/p=0.5. We consider α=1.8α=1.8 and α=2.2α=2.2, representing the two sides of the critical exponent α=2α=2, and use σ2∈0.5,2σ^2∈\0.5,2\. For each setting, we compare the finite-p deterministic equivalent with empirical Monte Carlo W2SG and also display the fixed-m and proportional asymptotic limits. (a) σ2=0.5σ^2=0.5. (b) σ2=2σ^2=2. Figure 2: Case I with α>2α>2. Numerical validation at α=2.2α=2.2, p=400p=400, and n=200n=200 using 10001000 Monte Carlo trials. The panel layout is the same as in Figure 1: finite-p deterministic equivalent versus empirical W2SG on top, fixed-m limit on the bottom left, and proportional-regime limit on the bottom right. Regime 1<α<21<α<2. Figure 1 considers α=1.8α=1.8 and contrasts two noise levels. At σ2=0.5σ^2=0.5 (Figure 1(a)), the finite-p deterministic equivalent is negative for the smallest pseudo-sample sizes and crosses zero at approximately m=15.25m=15.25; the empirical curve crosses at approximately m=17.39m=17.39. The fixed-sample limit exhibits the same eventual sign change, but at the larger value m≈24.47m≈ 24.47. By contrast, the proportional-regime limit is positive throughout the displayed interval of fixed sampling ratios m/p>0m/p>0. These observations illustrate the distinction between the two asymptotic scalings in Theorem 4.4: small fixed m can remain unfavorable even though every fixed positive proportional ratio is favorable when α≤2α≤ 2. At the higher noise level σ2=2σ^2=2 (Figure 1(b)), the gain is positive throughout all three displayed regimes. The finite-p curve is nevertheless strongly non-monotone: it decreases sharply at very small m, rises over an intermediate range, and then decreases toward zero as m approaches p. The fixed-m and proportional limits display the same broad spectral trade-off at their respective scales. In both noise settings, the finite-p deterministic equivalent follows the Monte Carlo mean closely, including through the small-m transition and the non-monotone region. (a) α=1.8α=1.8, σ2=0.5σ^2=0.5. (b) α=1.8α=1.8, σ2=2σ^2=2. (c) α=2.2α=2.2, σ2=0.5σ^2=0.5. (d) α=2.2α=2.2, σ2=2σ^2=2. Figure 3: Finite-size convergence in Case I. Finite-p deterministic equivalents for p∈200,400,800,1000p∈\200,400,800,1000\ with n/p=0.5n/p=0.5, compared with the two asymptotic limits in Theorem 4.4. In each subfigure, the left panel keeps the absolute Stage-I sample size m fixed and compares pDE(m) W2SG^DE_p(m) with the fixed-m limit GmG_m (purple dashed curve). The right panel keeps the sampling ratio ρm=m/p _m=m/p fixed and compares the finite-p curves with the proportional-regime limit (purple dashed curve). The four subfigures use the same (α,σ2)(α,σ^2) pairs as the main Case I experiments. Regime α>2α>2. Figure 2 repeats the comparison at α=2.2α=2.2. At σ2=0.5σ^2=0.5 (Figure 2(a)), the fixed-m limit GmG_m remains negative throughout the displayed range, whereas the proportional limit is negative for small sampling ratios and becomes positive only after a critical ratio ρm,crit≈0.0840 _m,crit≈ 0.0840. At p=400p=400, this predicts a critical pseudo-sample size pρm,crit≈33.6p _m,crit≈ 33.6, essentially coinciding with the finite-p deterministic-equivalent crossing m≈33.64m≈ 33.64. The empirical crossing, m≈38.36m≈ 38.36, is slightly shifted at this finite dimension but exhibits the same transition. This is the clearest numerical example of the qualitative disagreement between the fixed-sample and proportional regimes for α>2α>2: a fixed number of pseudo-samples remains unfavorable, while a sufficiently large Θ(p) (p)-scale pseudo-sample size yields positive W2SG. At σ2=2σ^2=2 (Figure 2(b)), the phase structure is richer. The finite-p deterministic equivalent is positive at the very smallest m, becomes negative over an intermediate small-m interval, and becomes positive again, with zero crossings at approximately m=3.73m=3.73 and m=28.38m=28.38. The empirical curve shows the same disconnected structure, with crossings near m=2.54m=2.54 and m=29.14m=29.14. The fixed-m limit likewise changes sign near m=3.23m=3.23 and is negative thereafter over the displayed range, whereas the proportional limit again crosses from negative to positive near ρm≈0.084 _m≈ 0.084. This is the finite-dimensional manifestation of the disconnected positive-W2SG regions discussed in Remark 4.5: improvement can occur at very small m and again at proportional scale, while an intermediate range remains unfavorable. 6.1.2 Finite-Size Scaling Figure 4: Two-stage random-feature regression. Empirical and deterministic-equivalent W2SG for p=400p=400, ϑ=0.05 =0.05, η=2η=2, ρT=0.04 _T=0.04, over 200200 Monte Carlo trials. Left: W2SG versus ρm=m/p _m=m/p at fixed ρS=0.8 _S=0.8. Right: W2SG versus ρS=NS/p _S=N_S/p at fixed ρm=0.5 _m=0.5. Blue: finite-p master deterministic equivalent. Orange: empirical mean with 95%95\% confidence band. Green lines: predicted bounded positive-W2SG interval. Purple line: threshold of the predicted positive half-line. Gray dotted line: the interpolation boundary m=NSm=N_S. Red dotted line: the signal threshold ρ=ϑρ= . Gray dashed line: the ambient-rank boundary ρ=1ρ=1. Figure 3 checks directly that the two asymptotic formulas arise from two different large-p scalings of the same finite-p deterministic equivalent. In the left panels, m is held fixed as p increases. The finite-p curves move toward the fixed-sample limit GmG_m; for α=1.8α=1.8 and σ2=0.5σ^2=0.5, this is visible in the rightward motion of the finite-p zero crossing toward the crossing of GmG_m. For α=2.2α=2.2 and σ2=0.5σ^2=0.5, increasing p instead drives the fixed-m curve downward toward the negative limiting branch, reinforcing that fixed m does not access the positive proportional regime. In the right panels, m/pm/p is held fixed. Away from the small-ρm _m boundary, the finite-p curves collapse rapidly onto the proportional-limit prediction as p increases. For α=1.8α=1.8, the limiting proportional curve remains positive for both noise levels, in agreement with Theorem 4.4(a)(i). For α=2.2α=2.2, the curves converge to the proportional phase transition predicted by Theorem 4.4(a)(i). The σ2=2σ^2=2 setting has the largest finite-size corrections near ρm=0 _m=0, precisely where the proportional scaling approaches the fixed-m/sparse boundary. Taken together, the four scaling experiments suggest that the asymptotic behavior of the deterministic equivalent depends critically on how the Stage-I sample size m scales with the ambient dimension p. In particular, fixed-m and proportional-m limits capture different high-dimensional regimes, highlighting the need to analyze distinct scalings of m when characterizing W2SG asymptotically. 6.2 Case I: Random-Feature Regression We next examine the random-feature setting of Theorem 5.4 and Corollary 5.6, where the Stage-I learner is controlled by both the pseudo-sample size m and the student feature width NSN_S. We first validate the predicted phase diagram, then isolate the two bottleneck mechanisms through a branch-separation experiment, and finally consider a negative-control setting in which the theory predicts no admissible positive-W2SG region. Figure 5: Case I branch separation. A configuration in which the two scans realize different components of the phase diagram. Left: the bounded positive interval in ρm _m is absent, while the positive half-line remains. Right: a bounded positive interval in ρS _S survives. The plotting conventions are the same as in Figure 4. Results use 500500 Monte Carlo trials. 6.2.1 Phase Diagram Figure 4 tests both branches of Theorem 5.4 and Corollary 5.6 at p=400p=400, ϑ=0.05 =0.05, η=2η=2, ρT=0.04 _T=0.04. Sweeping ρm _m at fixed ρS=0.8 _S=0.8 (left panel) recovers the sample-bottleneck branch: a bounded positive-W2SG interval, exactly as in Theorem 5.4(i). Sweeping ρS _S at fixed ρm=0.5 _m=0.5 (right panel) recovers the feature-bottleneck branch instead: a bounded interval in ρS _S per Corollary 5.6(i), with positivity resuming as a half-line once ρS _S grows large enough to make m the active bottleneck again. In both panels, the empirical curve tracks the finite-p master deterministic equivalent closely away from the internal interpolation boundary m=NSm=N_S, where the risk is singular and finite-dimensional fluctuations are strongly amplified, exactly as predicted by the exact interpolation classification of Appendix B. 6.2.2 Separation of the Two Phase Mechanisms Figure 5 separates the two phase mechanisms by choosing a configuration in which one bounded interval disappears while another branch survives. The headline setting displays several positive branches simultaneously. To verify that these branches are genuinely distinct rather than different representations of the same phenomenon, we take p=400p=400, ϑ=0.2 =0.2, η=2η=2, and ρT=0.16 _T=0.16, with ρS=0.8 _S=0.8 in the ρm _m-scan and ρm=2.25 _m=2.25 in the ρS _S-scan. Figure 6: Case I negative control. A configuration in which the fixed Stage-I resource lies below the signal threshold, so neither scan admits an admissible positive-W2SG region. Left: W2SG as a function of ρm=m/p _m=m/p at fixed ρS _S. Right: W2SG as a function of ρS=NS/p _S=N_S/p at fixed ρm _m. In both scans the finite-p master deterministic equivalent remains strictly negative throughout the displayed range and closely follows the empirical Monte Carlo mean. The gray dotted line denotes the interpolation boundary m=NSm=N_S and the red dotted line denotes the signal threshold ρ=ϑρ= . Shaded bands are empirical 95%95\% confidence intervals. For the left scan in Figure 5, the interval discriminant satisfies ΔDE<0 _DE<0, so no bounded positive-ρm _m interval exists. Nevertheless, the half-line branch remains admissible, with the asymptotic threshold at ρm≃1.714 _m 1.714. The finite-p deterministic-equivalent and empirical first zero crossings occur at approximately ρmDE≃1.827 _m^DE 1.827 and ρmemp≃1.902 _m^emp 1.902, respectively. In contrast, the right scan has DS>0D_S>0 and retains a bounded interval ρS∈(0.252,1) _S∈(0.252,1). The corresponding finite-p and empirical transitions, approximately 0.2940.294 and 0.2960.296, are nearly coincident. This experiment therefore isolates the two geometric components of the phase diagram: the bounded interval can disappear without eliminating the positive half-line, while the complementary scan can simultaneously retain its bounded positive branch. 6.2.3 Negative Control Figure 6 provides a negative control in which the branch constraints exclude all positive-W2SG regions. We retain p=400p=400, ϑ=0.05 =0.05, η=2η=2, and ρT=0.04 _T=0.04, but fix both Stage-I resources below the signal threshold, ρSfixed=ρmfixed=0.025<ϑ _S^fixed= _m^fixed=0.025< . Although the raw quadratic discriminants are positive in the configuration of Figure 6, their formal roots lie outside the admissible bottleneck branches once the signal-threshold and active-resource constraints are enforced. Consequently, neither scan contains a positive-W2SG interval or half-line. Consistent with this prediction, both the deterministic equivalent and the empirical Monte Carlo curves remain strictly below zero throughout the scanned domains. This negative control is useful because it shows that the phase theory does not merely fit observed sign changes: it also predicts settings in which improvement is absent altogether. 7 Conclusion We showed that weak-to-strong generalization can arise from a single, simple mechanism — a finite resource shared between two training stages, constrained to an intermediate range — without early stopping, without explicit regularization, and, in Case I, without any capacity advantage for the student at all. In ridgeless linear regression with identical teacher and student hypothesis classes, the pseudo-sample size m alone determines whether the student improves on its teacher, and we gave the explicit critical values of m, in both the fixed-sample and proportional regimes, that separate improvement from its absence. In the random-feature setting, where the student genuinely has more capacity than the teacher, the same trade-off persists but can be governed by either of two resources, pseudo-sample size or student width, and we identified an active-bottleneck principle unifying both regimes: whichever resource is scarcer controls how much teacher error is filtered out, while the other resource, once identified as non-binding, can be increased freely. Several directions remain open. Our power-law and two-block covariance models isolate the mechanism cleanly, but we only conjecture, rather than prove, that a comparably graded non-isotropic spectrum produces the same qualitative phase transitions beyond these specific families. The boundary cases excluded from Assumption 5.3 in Case I, and the regime α>1+2α>1+ 2 in Case I, are only partially resolved. More broadly, we view the active-bottleneck principle as plausibly extending beyond the two linear models studied here — to nonlinear students, to other resources such as training time, and, ultimately, to the large-scale settings where W2SG was first observed and where recent empirical work already points to the same non-monotonic dependence on pseudo-sample size that our theory predicts. AI Use Statement Generative AI tools were used to assist with literature discovery, organization and language editing of the abstract, introduction, and related-work discussion, and to provide feedback on manuscript structure and experimental design. All citations, mathematical claims, proofs, code, and reported results were independently verified by the authors. The authors take responsibility for the final content. References Atanasov et al. (2025) A. Atanasov, B. Bordelon, J. A. Zavatone-Veth, C. Paquette, and C. Pehlevan Two-point deterministic equivalence for stochastic gradient dynamics in linear models. arXiv preprint arXiv:2502.05074. Cited by: §A.1, §B.1, Lemma B.1, §4.4. Atanasov et al. (2026) A. Atanasov, J. A. Zavatone-Veth, and C. Pehlevan Scaling and renormalization in high-dimensional regression. Journal of Statistical Mechanics: Theory and Experiment 2026 (4), p. 043404. Cited by: §A.1, §A.3, §A.3, §A.3, §A.3, §A.4, §A.4, §A.4, Lemma A.1, Lemma A.1, Lemma A.1, §B.4, §B.4, §B.4, §B.4, §B.4, §B.4, §B.4, §B.4, Theorem B.4, §4.4. Bordelon et al. (2024) B. Bordelon, A. Atanasov, and C. Pehlevan A dynamical model of neural scaling laws. arXiv preprint arXiv:2402.01092. Cited by: §4.3. Burns et al. (2023) C. Burns, P. Izmailov, J. H. Kirchner, B. Baker, L. Gao, L. Aschenbrenner, Y. Chen, A. Ecoffet, M. Joglekar, J. Leike, et al. Weak-to-strong generalization: eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.09390. Cited by: §1. Busbridge et al. (2025) D. Busbridge, A. Shidani, F. Weers, J. Ramapuram, E. Littwin, and R. Webb Distillation scaling laws. arXiv preprint arXiv:2502.08606. Cited by: §1, §2, §2. Caponnetto and De Vito (2007) A. Caponnetto and E. De Vito Optimal rates for the regularized least-squares algorithm. Foundations of Computational mathematics 7 (3), p. 331–368. Cited by: §4.3. Charikar et al. (2024) M. Charikar, C. Pabbaraju, and K. Shiragur Quantifying the gain in weak-to-strong generalization. Advances in neural information processing systems 37, p. 126474–126499. Cited by: §2. Cheng and Montanari (2024) C. Cheng and A. Montanari Dimension free ridge regression. The Annals of Statistics 52 (6), p. 2879–2912. Cited by: §A.3, §4. Cui et al. (2021) H. Cui, B. Loureiro, F. Krzakala, and L. Zdeborová Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime. Advances in Neural Information Processing Systems 34, p. 10131–10143. Cited by: §4.3. Defilippis et al. (2026) L. Defilippis, Y. Xu, J. Girardin, V. Erba, E. Troiani, L. Zdeborová, B. Loureiro, and F. Krzakala Scaling laws and spectra of shallow neural networks in the feature learning regime. In International Conference on Learning Representations, Vol. 2026, p. 108893–108934. Cited by: §4.3. Dong et al. (2025) Y. Dong, Y. Li, Y. Li, J. D. Lee, and Q. Lei Discrepancies are virtue: weak-to-strong generalization through lens of intrinsic dimension. arXiv preprint arXiv:2502.05075. Cited by: §1, §2, §2, §4. Geng et al. (2026) S. Geng, D. Hansen, and J. Li Weak-to-strong generalization is nearly inevitable (in linear models). arXiv preprint arXiv:2605.05742. Cited by: §1, §2. Han and Xu (2026) Q. Han and X. Xu The distribution of ridgeless least squares interpolators. Journal of Machine Learning Research 27 (23), p. 1–94. Cited by: §A.3, §4. Hastie et al. (2022) T. Hastie, A. Montanari, S. Rosset, and R. J. Tibshirani Surprises in high-dimensional ridgeless least squares interpolation. Annals of statistics 50 (2), p. 949. Cited by: §A.3, §4. Ildiz et al. (2025) M. Ildiz, H. Gozeten, E. Taga, M. Mondelli, and S. Oymak High-dimensional analysis of knowledge distillation: weak-to-strong generalization and scaling laws. In International Conference on Learning Representations, Vol. 2025, p. 27181–27220. Cited by: §A.4, §2. Lang et al. (2024) H. Lang, D. Sontag, and A. Vijayaraghavan Theoretical analysis of weak-to-strong generalization. Advances in neural information processing systems 37, p. 46837–46880. Cited by: §2. Loureiro et al. (2021) B. Loureiro, C. Gerbelot, H. Cui, S. Goldt, F. Krzakala, M. Mezard, and L. Zdeborová Learning curves of generic features maps for realistic datasets with a teacher-student model. Advances in Neural Information Processing Systems 34, p. 18137–18151. Cited by: §A.3, §4. Medvedev et al. (2025) M. Medvedev, K. Lyu, D. Yu, S. Arora, Z. Li, and N. Srebro Weak-to-strong generalization even in random feature networks, provably. arXiv preprint arXiv:2503.02877. Cited by: §1, §2, §5.1. Moniri and Hassani (2025) B. Moniri and H. Hassani On the mechanisms of weak-to-strong generalization: a theoretical perspective. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, Cited by: §1, §2, §2. Mulgund and Pabbaraju (2025) A. Mulgund and C. Pabbaraju Relating misfit to gain in weak-to-strong generalization beyond the squared loss. arXiv preprint arXiv:2501.19105. Cited by: §2. Oh et al. (2026) J. Oh, J. Song, and C. Yun From linear to nonlinear: provable weak-to-strong generalization through feature learning. Advances in Neural Information Processing Systems 38, p. 169216–169285. Cited by: §2. Richards et al. (2021) D. Richards, J. Mourtada, and L. Rosasco Asymptotics of ridge (less) regression under general source condition. In International Conference on Artificial Intelligence and Statistics, p. 3889–3897. Cited by: §A.3, §4. Shin et al. (2025) C. Shin, J. Cooper, and F. Sala Weak-to-strong generalization through the data-centric lens. In International Conference on Learning Representations, Vol. 2025, p. 22039–22077. Cited by: §2. Somerstep et al. (2025) S. Somerstep, F. Maia Polo, M. Banerjee, Y. Ritov, M. Yurochkin, and Y. Sun A transfer learning framework for weak to strong generalization. In International Conference on Learning Representations, Vol. 2025, p. 100714–100748. Cited by: §2. Spigler et al. (2020) S. Spigler, M. Geiger, and M. Wyart Asymptotic learning curves of kernel methods: empirical data versus teacher–student paradigm. Journal of Statistical Mechanics: Theory and Experiment 2020 (12), p. 124001. Cited by: §4.3. Steinwart et al. (2009) I. Steinwart, D. R. Hush, C. Scovel, et al. Optimal rates for regularized least squares regression.. In COLT, p. 79–93. Cited by: §4.3. Wu and Sahai (2025) D. Wu and A. Sahai Provable weak-to-strong generalization via benign overfitting. In International Conference on Learning Representations, Vol. 2025, p. 88280–88316. Cited by: §2, §4.3, §4. Wu and Xu (2020) D. Wu and J. Xu On the optimal weighted ℓ2 _2 regularization in overparameterized linear regression. In Advances in Neural Information Processing Systems, Vol. 33, p. 10112–10123. Cited by: §A.3, §4. Wu et al. (2026) D. Wu, L. Chen, T. Misiakiewicz, and M. Mondelli Improved scaling laws via weak-to-strong generalization in random feature ridge regression. arXiv preprint arXiv:2603.05691. Cited by: §2, §2, §4.3, §4. Xu et al. (2025) G. Xu, W. Yao, Z. Wang, and Y. Liu On the emergence of weak-to-strong generalization: a bias-variance perspective. arXiv preprint arXiv:2505.24313. Cited by: §2. Xue et al. (2025) Y. Xue, J. Li, and B. Mirzasoleiman Representations shape weak-to-strong generalization: theoretical insights and empirical predictions. arXiv preprint arXiv:2502.00620. Cited by: §1, §2. Yao et al. (2026) W. Yao, W. Zhaoyang, G. Xu, C. Qian, D. Liu, Z. Wang, Y. Liu, and Y. Xu On the blessing of pre-training in weak-to-strong generalization. arXiv preprint arXiv:2605.05710. Cited by: §2. Appendix A Two-Stage Minimum-Norm Interpolation Theory This section develops the theoretical analysis of the two-stage minimum-norm interpolation model used in Section 4. In this setting, the teacher and student operate in the same ambient linear hypothesis space, but are trained on different samples: Stage I fits a ridgeless estimator to noisy labeled data, while Stage I fits a second ridgeless estimator to fresh covariates pseudo-labeled by the Stage-I teacher. The central question is whether this second interpolation step can improve population prediction risk despite receiving no information beyond the teacher-generated labels. We first develop the deterministic-equivalent description of the two-stage procedure in the proportional high-dimensional regime. After recording the resolvent deterministic equivalence used throughout the analysis, we derive the Stage-I prediction-risk formula and then analyze Stage I conditionally on the realized teacher. This separates the student risk into the Stage-I error carried into the second stage, the additional error generated by Stage-I interpolation, and the cross term produced by the interaction between the two stages. A weighted ridgeless pseudoinverse deterministic equivalent then allows the remaining Stage-I randomness to be evaluated. Under the additional isotropic random-teacher assumption, these calculations yield a fully deterministic expression for the Stage-I-to-Stage-I risk improvement in terms of the population spectrum and the effective Stage-I and Stage-I resolution parameters. We then specialize the deterministic-equivalent formula to a power-law covariance spectrum and study its asymptotic sign. The fixed-point equations for the effective resolutions are reduced to limiting integral equations, and the trace terms in the W2SG gain are analyzed separately before being combined into a scalar sign criterion. This leads to the proportional W2SG phase diagram and reveals a qualitative change at the spectral-decay exponent α=2α=2: the regime 1<α≤21<α≤ 2 exhibits positive deterministic-equivalent gain throughout the nontrivial proportional Stage-I range, whereas for α>2α>2 the gain can change sign as the Stage-I sampling ratio varies. The subsequent analysis characterizes the existence and, where established, uniqueness of these proportional transitions. Finally, we move beyond fixed proportional Stage-I sampling. We first consider a sparse-growing Stage-I sample size and derive the corresponding deterministic-equivalent limit. We then study the ultra-sparse regime in which the Stage-I sample count is fixed while the ambient dimension diverges. Because this regime lies outside the standard proportional deterministic-equivalence theorem, its analysis is stated under an explicit continuation assumption. Within that framework, we derive the fixed-m gain, identify noise- and spectral-decay-dependent sign transitions, obtain critical and large-m asymptotics, and give explicit conditions for W2SG and non-W2SG behavior. Together, these results describe how the benefit of the second interpolation stage depends on spectral decay, label noise, and the amount of pseudo-labeled Stage-I data across proportional, sparse-growing, and fixed-sample regimes. A.1 Deterministic Equivalence We begin by stating the deterministic-equivalence results that form the basis of our analysis Atanasov et al. [2026], Atanasov et al. [2025]. These results apply in proportional high-dimensional asymptotic regimes, where the sample size and feature dimension grow at comparable rates. Any additional assumptions required in the subsequent derivations are stated explicitly. Lemma A.1 (Resolvent deterministic equivalence). We adopt the notation and definition of strong deterministic equivalence used by [Atanasov et al., 2026, Sec. I.E.3]. In particular, the symbol ≃ below has the same test-matrix meaning as in that reference. Let App≥1\A_p\_p≥ 1 be a sequence of random p×p× p matrices and let Bpp≥1\B_p\_p≥ 1 be a sequence of deterministic p×p× p matrices. Following [Atanasov et al., 2026, Sec. I.E.3], we write Ap≃BpA_p B_p if, for every deterministic test matrix MpM_p satisfying supp‖Mp‖op<∞, _p\|M_p\|_op<∞, one has tr(ApMp)tr(BpMp)→p→∞ℙ1. tr(A_pM_p)tr(B_pM_p) [p→∞]P1. The same notion is used in [Atanasov et al., 2026, Eqs. (18)–(20)] to obtain strong deterministic equivalents for empirical-covariance resolvents. For the sample covariance matrix, this framework yields the following deterministic equivalents in the proportional high-dimensional regime: (Σ^p+λI)−1≃τλ(Σp+τI)−1.( _p+λ I)^-1 τλ( _p+τ I)^-1. Σ^p(Σ^p+λI)−1≃Σp(Σp+τI)−1. _p( _p+λ I)^-1 _p( _p+τ I)^-1. where xi∼(0,Σp),Σ^p=1np∑i=1npxixi⊤,p,np→∞,pnp→ϕ∈(0,∞).x_i (0, _p), _p= 1n_p _i=1^n_px_ix_i , p,n_p→∞, pn_p→φ∈(0,∞). Here λ>0λ>0 is the ridge regularization parameter and τ=τ(λ)τ=τ(λ) denotes the corresponding renormalized ridge parameter in the proportional regime. Its precise characterization will be introduced later in the derivation. The ridgeless regime is obtained, when appropriate, by subsequently taking the limit λ↓0λ 0. In the overparameterized regime ϕ>1φ>1, τ(λ)τ(λ) may remain nonzero in this limit, whereas in the underparameterized regime ϕ<1φ<1, one has τ(λ)→0τ(λ)→ 0. We now introduce the two-stage ridgeless linear-regression model with Gaussian covariates that will be used throughout the subsequent analysis. A.2 Two-Stage Linear Regression with Gaussian Covariates Let • n be the number of data points in Stage I, • m be the number of data points in Stage I, • p be the dimension of the feature space. We consider the linear model y=Xβ⋆+ε,y=Xβ + , (21) where X∈ℝn×pX ^n× p is the design matrix, β⋆∈ℝpβ ^p is the true parameter vector, and ε∈ℝn ^n is the label-noise vector. For the time being, β⋆β is treated as deterministic; later, when a random-teacher model is required, the corresponding distributional assumptions will be stated explicitly. The rows xi⊤x_i of X satisfy xi∼i.i.d.(0,Σ),x_i .i.d. N(0, ), where Σ∈ℝp×p ^p× p is the population covariance matrix, assumed positive definite, Σ≻0 0. The noise components are independent of the design and satisfy εi∼i.i.d.(0,σ2). _i .i.d. N(0,σ^2). For an independent test covariate x∼(0,Σ)x (0, ) and any β^∈ℝp β ^p, we define the population prediction risk by ℛ(hβ^):= (h_ β):= x[(x⊤β^−x⊤β⋆)2]=(β^−β⋆)⊤Σ(β^−β⋆)=‖Σ1/2(β^−β⋆)‖22. _x [ (x β-x β )^2 ]=( β-β ) ( β-β )= \| ^1/2( β-β ) \|_2^2. (22) A.3 Stage I: Ridgeless Regression Stage-I ridgeless regression. In Stage I, we fit the ridgeless least-squares estimator β^T:=X†y, β_T:=X y, (23) where X†X denotes the Moore–Penrose pseudoinverse of X. For Gaussian covariates, the design matrix has maximal rank almost surely. Therefore, when n≥pn≥ p, β^T=(X⊤X)−1X⊤y, β_T=(X X)^-1X y, (24) which is the ordinary least-squares estimator. When p>np>n, β^T=X⊤(XX⊤)−1y, β_T=X (X )^-1y, (25) which is the unique minimum-Euclidean-norm solution of Xβ=y.Xβ=y. Thus, in the overparameterized regime p>np>n, the ridgeless least-squares estimator coincides with the minimum-norm interpolator. Define the empirical covariance matrix Σ^:=1nX⊤X∈ℝp×p. := 1nX X ^p× p. For fixed p, the law of large numbers gives Σ^→Σ → as n→∞n→∞ under the usual moment assumptions. In the proportional high-dimensional regime considered below, however, Σ does not in general converge to Σ in operator norm, and its effect must instead be described through deterministic equivalents. An equivalent covariance-form representation of (23) is β^T=Σ^†(1nX⊤y), β_T= ( 1nX y ), (26) where Σ^† denotes the Moore–Penrose pseudoinverse. A substantial body of recent work has developed both asymptotic and non-asymptotic risk characterizations for the minimum ℓ2 _2-norm interpolator, corresponding to ridgeless regression in the overparameterized regime p≥np≥ n Hastie et al. [2022], Cheng and Montanari [2024], Han and Xu [2026], Wu and Xu [2020], Richards et al. [2021], Loureiro et al. [2021]. To analyze the Stage-I population risk in the proportional high-dimensional regime, we use the strong deterministic-equivalence framework introduced in Lemma A.1, following Atanasov et al. [2026]. In particular, the random resolvents involving Σ that appear in the risk calculation are replaced by their deterministic equivalents in the bounded-test-matrix sense. Since ℛ(hβ^T)R(h_ β_T) already averages over an independent test covariate, we further average over the randomness of the training sample and define ℛ¯T:=X,ε[ℛ(hβ^T)]. R_T:=E_X, \! [R(h_ β_T) ]. This quantity corresponds to the average generalization error considered in [Atanasov et al., 2026, Sec. I.A–B]. Applying the deterministic-equivalent resolvent identities of Lemma A.1 yields ℛ¯T≃DEℛ¯TDE, R_T DE R_T^DE, where ≃DE DE at the scalar level denotes asymptotic equality in the proportional high-dimensional limit, following the notation used in [Atanasov et al., 2026, Eq. (24)]. The explicit expression for ℛ¯TDE R_T^DE derived below is the ridgeless specialization of the ridge-regression generalization-error formula in [Atanasov et al., 2026, Sec. I.B, Eqs. (24)–(25)]. Theorem A.2 (Stage-I risk deterministic equivalent). The averaged Stage-I population risk admits the deterministic equivalent ℛ¯TDE= R_T^DE= 11−γT[τT 2β⋆⊤Σ(Σ+τTIp)−2β⋆+σ2γT], \; 11- _T [ _T^\,2\,β ( + _TI_p)^-2β +σ^2\, _T ], (27) where γT:=pn1p∑i=1pλi2(λi+τT)2=1n∑i=1pλi2(λi+τT)2, _T:= pn\, 1p _i=1^p _i^2( _i+ _T)^2= 1n _i=1^p _i^2( _i+ _T)^2, and, in the ridgeless overparameterized regime, τT>0 _T>0 is determined by ∑i=1pλiλi+τT=n. _i=1^p _i _i+ _T=n. Proof. This follows directly from the ridgeless specialization of the generalization-error deterministic equivalent in [Atanasov et al., 2026, Sec. I.B, Eqs. (24)–(25)]. ∎ A.4 Stage I Risk Derivation (Distilled Stage-I Sample) Stage-I data model. Draw a fresh independent dataset x~ii=1m∼i.i.d.(0,Σ),X~∈ℝm×p,\ x_i\_i=1^m .i.d. N(0, ), X ^m× p, and define the distilled labels, with no additional label noise, by y~:=X~β^T. y:= X β_T. Stage-I ridgeless estimator. In Stage I, we fit the ridgeless least-squares estimator β^S=argminβ∈ℝp1m‖X~β−y~‖22,m≥p,argminβ∈ℝp‖β‖2:X~β=y~,m<p. β_S= cases _β ^p 1m\| Xβ- y\|_2^2,&m≥ p,\\[11.99998pt] _β ^p \\|β\|_2: Xβ= y \,&m<p. cases Using y~=X~β^T, y= X β_T, the student estimator can be written as β^S=argminβ∈ℝp1m‖X~β−X~β^T‖22=(X~⊤X~)−1X~⊤X~β^T=β^T,m≥p,argminβ∈ℝp‖β‖2:X~β=X~β^T=X~⊤(X~X~⊤)−1X~β^T,m<p. β_S= cases _β ^p 1m \| Xβ- X β_T \|_2^2=( X X)^-1 X X\, β_T= β_T,&m≥ p,\\[13.99995pt] _β ^p \\|β\|_2: Xβ= X β_T \= X ( X X )^-1 X\, β_T,&m<p. cases When Σ~ is singular (e.g., m<pm<p), β^S β_S is understood as the minimum-norm interpolating solution. Since β^S β_S depends on both the Stage-I training sample (X,ε)(X, ) and the fresh Stage-I design X~ X, we define the averaged Stage-I population risk by ℛ¯S:=X,ε,X~[ℛ(hβ^S)]. R_S:=E_X, , X\! [R(h_ β_S) ]. To analyze the Stage-I population risk in the proportional high-dimensional regime, we again use the strong deterministic-equivalence framework introduced in Lemma A.1, following Atanasov et al. [2026]. In particular, conditional on β^T β_T, the random resolvents involving Σ~ are replaced by their deterministic equivalents in the bounded-test-matrix sense. We therefore write ℛ¯S≃DEℛ¯SDE, R_S DE R_S^DE, where ℛ¯SDE R_S^DE denotes the deterministic equivalent of the averaged Stage-I population risk. We first derive the Stage-I risk while leaving the remaining expectation over the Stage-I estimator β^T β_T unevaluated. The resulting expression depends on moments of β^T β_T. In the subsequent derivation, these moments are rewritten as weighted matrix functionals that are compatible with the deterministic-equivalence framework. Theorem A.3 (Stage-I risk deterministic equivalent). The Stage-I deterministic-equivalent risk is ℛ¯SDE= R_S^DE= β^T[‖β^T−β⋆‖Σ2⏟Stage-I error carried over+τS21−γS∑i=1pλi(β^T,i)2(λi+τS)2⏟Stage-I distillation error−2∑i=1pλiτSλi+τSβ^T,i(β^T,i−βi⋆)⏟cross term], _ β_T [ \| β_T-β \|_ ^2_Stage-I error carried over+ _S^21- _S _i=1^p _i( β_T,i)^2( _i+ _S)^2_Stage-I distillation error- 2 _i=1^p _i _S _i+ _S β_T,i ( β_T,i- _i )_cross term ], (28) where λii=1p\ _i\_i=1^p are the eigenvalues of Σ , β^T,i β_T,i and βi⋆ _i denote the corresponding coordinates in the eigenbasis of Σ , and γS:=pm⋅1p∑i=1pλi2(λi+τS)2=1m∑i=1pλi2(λi+τS)2. _S:= pm· 1p _i=1^p _i^2( _i+ _S)^2= 1m _i=1^p _i^2( _i+ _S)^2. In the ridgeless overparameterized regime, τS>0 _S>0 is determined by ∑i=1pλiλi+τS=m. _i=1^p _i _i+ _S=m. Consequently, define the averaged Stage-I-to-Stage-I risk improvement by p:=ℛ¯T−ℛ¯S≃DEpDE, W2SG_p:= R_T- R_S DE W2SG_p^DE, where pDE:=ℛ¯TDE−ℛ¯SDE. W2SG_p^DE:= R_T^DE- R_S^DE. At the deterministic-equivalent level, the Stage-I error carried over cancels, yielding pDE= W2SG_p^DE= β^T[2∑i=1pλiτSλi+τSβ^T,i(β^T,i−βi⋆)−τS21−γS∑i=1pλi(β^T,i)2(λi+τS)2]. _ β_T [2 _i=1^p _i _S _i+ _S β_T,i ( β_T,i- _i )- _S^21- _S _i=1^p _i( β_T,i)^2( _i+ _S)^2 ]. (29) Proof. Define the Stage-I empirical covariance matrix by Σ~:=1mX~⊤X~∈ℝp×p. := 1m X X ^p× p. For the deterministic-equivalent calculation, we first introduce the auxiliary ridge estimator β^S,λS=argminβ∈ℝp1m‖X~β−y~‖22+λS‖β‖22. β_S, _S= _β ^p \ 1m\| Xβ- y\|_2^2+ _S\|β\|_2^2 \. Using y~=X~β^T y= X β_T, this is equivalently β^S,λS=argminβ∈ℝp1m‖X~(β−β^T)‖22+λS‖β‖22. β_S, _S= _β ^p \ 1m\| X(β- β_T)\|_2^2+ _S\|β\|_2^2 \. Although our Stage-I estimator is ridgeless, we keep λS>0 _S>0 throughout the deterministic-equivalent calculation, following Atanasov et al. [2026], and take the limit λS↓0 _S 0 only at the end. For λS>0 _S>0, β^S,λS= β_S, _S= (X~⊤X~+mλSIp)−1X~⊤y~=(X~⊤X~+mλSIp)−1X~⊤X~β^T=(Σ~+λSIp)−1Σ~β^T. ( X X+m _SI_p)^-1 X y=( X X+m _SI_p)^-1 X X\, β_T=( + _SI_p)^-1 \, β_T. (30) Equivalently, β^S,λS=(Ip−λS(Σ~+λSIp)−1)β^T. β_S, _S= (I_p- _S( + _SI_p)^-1 ) β_T. Hence, defining the Stage-I update by ΔS,λS:=β^S,λS−β^T _S, _S:= β_S, _S- β_T, we obtain ΔS,λS=−λS(Σ~+λSIp)−1β^T. _S, _S=- _S( + _SI_p)^-1 β_T. Two-stage population risk decomposition. Write β^S,λS−β⋆=ΔS,λS+(β^T−β⋆). β_S, _S-β = _S, _S+( β_T-β ). Expanding the population risk and conditioning first on β^T β_T gives ℛ¯S,λS= R_S, _S= β^T[‖β^T−β⋆‖Σ2+X~[‖ΔS,λS‖Σ2∣β^T]+2X~[(β^T−β⋆)⊤ΣΔS,λS∣β^T]]. _ β_T [\| β_T-β \|_ ^2+E_ X [\| _S, _S\|_ ^2 β_T ]+2\,E_ X [( β_T-β ) _S, _S β_T ] ]. (31) Deterministic equivalence for Stage I conditional on β^T β_T. Condition on β^T β_T and view Stage I as ridge regression with fixed teacher β^T β_T and zero label noise. In the proportional limit, applying the deterministic-equivalence framework of Lemma A.1 to the Stage-I resolvent terms, in the same manner as in the Stage-I risk derivation, yields X~[(β^S,λS−β^T)⊤Σ(β^S,λS−β^T)|β^T]≃DEτS21−γS∑i=1pλi(β^T,i)2(λi+τS)2,E_ X [( β_S, _S- β_T) ( β_S, _S- β_T)\, |\, β_T ] DE _S^21- _S _i=1^p _i( β_T,i)^2( _i+ _S)^2, where γS=pm⋅1p∑i=1pλi2(λi+τS)2. _S= pm· 1p _i=1^p _i^2( _i+ _S)^2. The renormalized ridge parameter τS _S is related to the explicit ridge parameter λS _S through the corresponding fixed-point equation τS(1−1m∑i=1pλiλi+τS)=λS. _S (1- 1m _i=1^p _i _i+ _S )= _S. Thus, in the ridgeless overparameterized limit λS↓0 _S 0 with τS>0 _S>0, ∑i=1pλiλi+τS=m. _i=1^p _i _i+ _S=m. For the cross term, Lemma A.1 gives the strong deterministic equivalence Σ~(Σ~+λSIp)−1≃DEΣ(Σ+τSIp)−1. ( + _SI_p)^-1 DE ( + _SI_p)^-1. Hence, conditional on β^T β_T, the Stage-I estimator is a ridge-regression estimator with fixed teacher β^T β_T. Applying the corresponding deterministic equivalent from Atanasov et al. [2026] gives X~[β^S,λS∣β^T]≃DEΣ(Σ+τSIp)−1β^T.E_ X [ β_S, _S β_T ] DE ( + _SI_p)^-1 β_T. Consequently, X~[ΔS,λS∣β^T]≃DE−τS(Σ+τSIp)−1β^T.E_ X [ _S, _S β_T ] DE - _S( + _SI_p)^-1 β_T. Therefore, X~[(β^T−β⋆)⊤ΣΔS,λS∣β^T]≃DE−τS(β^T−β⋆)⊤Σ(Σ+τSIp)−1β^T=−∑i=1pλiτSλi+τSβ^T,i(β^T,i−βi⋆). _ X [( β_T-β ) _S, _S β_T ] DE - _S( β_T-β ) ( + _SI_p)^-1 β_T=- _i=1^p _i _S _i+ _S β_T,i( β_T,i- _i ). (32) Final Stage-I risk. Substituting the deterministic equivalents above into (31) and then taking the ridgeless limit λS↓0 _S 0 yields ℛ¯S R_S ≃DEβ^T[‖β^T−β⋆‖Σ2⏟Stage-I error carried over+τS21−γS∑i=1pλi(β^T,i)2(λi+τS)2⏟Stage-I distillation error−2∑i=1pλiτSλi+τSβ^T,i(β^T,i−βi⋆)⏟cross term]. DE E_ β_T [ \| β_T-β \|_ ^2_Stage-I error carried over+ _S^21- _S _i=1^p _i( β_T,i)^2( _i+ _S)^2_Stage-I distillation error- 2 _i=1^p _i _S _i+ _S β_T,i( β_T,i- _i )_cross term ]. (33) This is precisely the deterministic equivalent ℛ¯SDE R_S^DE stated in (28). ∎ Lemma A.4 (Weighted ridgeless pseudoinverse deterministic equivalent). Let B=BpB=B_p be a deterministic matrix with uniformly bounded operator norm. Let τT>0 _T>0 be determined by 1nTr[Σ(Σ+τTIp)−1]=1, 1nTr\! [ ( + _TI_p)^-1 ]=1, (34) and define γT:=1nTr[Σ2(Σ+τTIp)−2]. _T:= 1nTr\! [ ^2( + _TI_p)^-2 ]. (35) Then XTr[BΣ^†]≃DE11−γTTr[BΣ(Σ+τTIp)−2].E_XTr\! [B ] DE 11- _TTr\! [B ( + _TI_p)^-2 ]. (36) Proof. We start from Lemma A.1. For λ>0λ>0, that lemma gives (Σ^+λIp)−1≃DEτ(λ)λ(Σ+τ(λ)Ip)−1,( +λ I_p)^-1 DE τ(λ)λ( +τ(λ)I_p)^-1, (37) where the renormalized ridge τ(λ)τ(λ) is determined by τ1−1nTr[Σ(Σ+τIp)−1]=λ.τ \1- 1nTr\! [ ( +τ I_p)^-1 ] \=λ. (38) Weighted pseudoinverse deterministic equivalent. Starting from the finite-ridge relation (A.1), differentiate with respect to λ. Since −dλ[Σ^(Σ^+λIp)−1]=Σ^(Σ^+λIp)−2,- ddλ [ ( +λ I_p)^-1 ]= ( +λ I_p)^-2, and, by the chain rule, −dλ[Σ(Σ+τ(λ)Ip)−1]=dτdλΣ(Σ+τ(λ)Ip)−2,- ddλ [ ( +τ(λ)I_p)^-1 ]= dτdλ ( +τ(λ)I_p)^-2, testing against B gives XTr[BΣ^(Σ^+λIp)−2]≃DEdτdλTr[BΣ(Σ+τ(λ)Ip)−2]. _XTr\! [B ( +λ I_p)^-2 ] DE dτdλTr\! [B ( +τ(λ)I_p)^-2 ]. (39) It remains to evaluate dτ/dλdτ/dλ. From (38), λ=τ−τnTr[Σ(Σ+τIp)−1].λ=τ- τnTr\! [ ( +τ I_p)^-1 ]. Differentiating with respect to τ gives dλdτ=1−1nTr[Σ(Σ+τIp)−1]+τnTr[Σ(Σ+τIp)−2]. dλdτ=1- 1nTr\! [ ( +τ I_p)^-1 ]+ τnTr\! [ ( +τ I_p)^-2 ]. Using Σ(Σ+τIp)−1−τΣ(Σ+τIp)−2=Σ2(Σ+τIp)−2, ( +τ I_p)^-1-τ ( +τ I_p)^-2= ^2( +τ I_p)^-2, this simplifies to dλdτ=1−1nTr[Σ2(Σ+τIp)−2]. dλdτ=1- 1nTr\! [ ^2( +τ I_p)^-2 ]. (40) Hence, as λ↓0λ 0 and τ(λ)→τTτ(λ)→ _T, dτdλ|λ↓0=11−γT, . dτdλ |_λ 0= 11- _T, where γT=1nTr[Σ2(Σ+τTIp)−2]. _T= 1nTr\! [ ^2( + _TI_p)^-2 ]. Finally, by spectral calculus, limλ↓0Σ^(Σ^+λIp)−2=Σ^†. _λ 0 ( +λ I_p)^-2= . Taking λ↓0λ 0 in (39) therefore yields XTr[BΣ^†]≃DE11−γTTr[BΣ(Σ+τTIp)−2],E_XTr\! [B ] DE 11- _TTr\! [B ( + _TI_p)^-2 ], which proves the lemma. ∎ We next derive a fully deterministic expression for the Stage-I-to-Stage-I risk improvement. A related two-stage ridgeless-regression setting is studied in [Ildiz et al., 2025, Definition 3], where the Stage-I surrogate sample and Stage-I target sample may have different population covariances and noise levels. Their analysis therefore accommodates a more general distribution-shift setting and also provides a generalization bound under additional boundedness assumptions. Some of those assumptions, including boundedness conditions involving the inverse covariance, are not imposed in the present analysis. Accordingly, we work instead with deterministic-equivalent formulas under the spectral assumptions stated above and do not claim a corresponding non-asymptotic generalization bound. Theorem A.5 (Fully deterministic Stage-I-to-Stage-I risk improvement). Assume, in addition, that the teacher is random and independent of the design and label noise and is isotropic: [β⋆]=0,[β⋆β⋆⊤]=Ip.E[β ]=0, [β β ]=I_p. Then the teacher-averaged Stage-I-to-Stage-I risk improvement admits the deterministic equivalent β⋆[p]≃DEpDE,E_β \! [ W2SG_p ] DE W2SG_p^DE, where pDE= W2SG_p^DE= −τS21−γSTr[Σ2(Σ+τSIp)−2(Σ+τTIp)−1] - _S^21- _S [ ^2( + _SI_p)^-2( + _TI_p)^-1 ] + + 2σ2τSn(1−γT)Tr[Σ2(Σ+τSIp)−1(Σ+τTIp)−2] 2σ^2 _Sn(1- _T) [ ^2( + _SI_p)^-1( + _TI_p)^-2 ] − - σ2τS2n(1−γT)(1−γS)Tr[Σ2(Σ+τSIp)−2(Σ+τTIp)−2]. σ^2 _S^2n(1- _T)(1- _S) [ ^2( + _SI_p)^-2( + _TI_p)^-2 ]. (41) Since all matrices in (41) are functions of Σ , they are simultaneously diagonalizable. Thus, the same deterministic equivalent can be written in the eigenbasis of Σ as pDE= W2SG_p^DE= −τS21−γS∑i=1pλi2(λi+τS)2(λi+τT) - _S^21- _S _i=1^p _i^2( _i+ _S)^2( _i+ _T) + + 2σ2τSn(1−γT)∑i=1pλi2(λi+τS)(λi+τT)2 2σ^2 _Sn(1- _T) _i=1^p _i^2( _i+ _S)( _i+ _T)^2 − - σ2τS2n(1−γT)(1−γS)∑i=1pλi2(λi+τS)2(λi+τT)2. σ^2 _S^2n(1- _T)(1- _S) _i=1^p _i^2( _i+ _S)^2( _i+ _T)^2. (42) Proof. To complete the derivation of the Stage-I-to-Stage-I risk improvement, we must average the Stage-I deterministic-equivalent expression over the remaining Stage-I randomness. Direct coordinate-wise application of deterministic equivalence to the random coordinates β^T,i β_T,i is not justified by Lemma A.1, which is formulated in terms of deterministic test matrices and trace functionals. We therefore rewrite the coordinate expression in terms of weighted quadratic forms that are compatible with the deterministic-equivalence framework of Lemma A.1. Since all matrices appearing in the Stage-I deterministic equivalent are functions of the population covariance Σ , define AS:=τSΣ(Σ+τSIp)−1,A_S:= _S ( + _SI_p)^-1, (43) and DS:=2τSΣ(Σ+τSIp)−1−τS21−γSΣ(Σ+τSIp)−2.D_S:=2 _S ( + _SI_p)^-1- _S^21- _S ( + _SI_p)^-2. (44) Indeed, in the eigenbasis of Σ , [AS]ii=λiτSλi+τS,[A_S]_i= _i _S _i+ _S, whereas [DS]ii=2λiτSλi+τS−τS21−γSλi(λi+τS)2.[D_S]_i=2 _i _S _i+ _S- _S^21- _S _i( _i+ _S)^2. Therefore, the Stage-I expression (29) can be written as β⋆[p]≃DE _β \! [ W2SG_p ] DE [β^T⊤DSβ^T]−2[β⋆⊤ASβ^T]. [ β_T D_S β_T ]-2E [β A_S β_T ]. (45) Recall that Σ^=1nX⊤X,PT=Σ^†Σ = 1nX X,P_T= and the Stage-I labels satisfy y=Xβ⋆+ε,[ε]=0,[εε⊤]=σ2In.y=Xβ + , [ ]=0, [ ]=σ^2I_n. (46) The Stage-I ridgeless estimator can be written as β^T=Σ^†X⊤yn=Σ^†Σ^β⋆+Σ^†X⊤εn. β_T= X yn= β + X n. (47) Since Σ is symmetric positive semidefinite, PTP_T is an orthogonal projector and hence PT⊤=PT,PT2=PTP_T =P_T,P_T^2=P_T. Thus, β^T=PTβ⋆+Σ^†X⊤εn. β_T=P_Tβ + X n. (48) We first evaluate [β^T⊤DSβ^T].E [ β_T D_S β_T ]. Conditioning on X and using (48), we have β^Tβ^T⊤=PTβ⋆β⋆⊤PT+PTβ⋆ε⊤XnΣ^†+Σ^†X⊤εnβ⋆⊤PT+Σ^†X⊤εε⊤Xn2Σ^†. β_T β_T =P_Tβ β P_T+P_Tβ Xn + X nβ P_T+ X Xn^2 . (49) Because β⋆β is independent of the design and noise, [β⋆β⋆⊤]=IpE[β β ]=I_p, and [ε]=0E[ ]=0, the two mixed terms vanish. Therefore, β⋆,ε[β^Tβ^T⊤∣X]=PT+σ2n2Σ^†X⊤XΣ^†=PT+σ2nΣ^†Σ^Σ^†=PT+σ2nΣ^†, _β , [ β_T β_T X ]=P_T+ σ^2n^2 X X =P_T+ σ^2n =P_T+ σ^2n , (50) where the last equality uses the Moore–Penrose identity Σ^†Σ^Σ^†=Σ^†. = . Consequently, [β^T⊤DSβ^T]= [ β_T D_S β_T ]= XTr[DSβ⋆,ε[β^Tβ^T⊤∣X]]=XTr(DSPT)+σ2nXTr[DSΣ^†]. _X [D_SE_β , [ β_T β_T X ] ]=E_X (D_SP_T)+ σ^2nE_X [D_S ]. (51) From (48), [β⋆⊤ASβ^T]= [β A_S β_T ]= [β⋆⊤ASPTβ⋆]+[β⋆⊤ASΣ^†X⊤εn]. [β A_SP_Tβ ]+E [β A_S X n ]. (52) The second term vanishes because the label noise is independent and zero mean. Hence, [β⋆⊤ASβ^T]= [β A_S β_T ]= Xβ⋆[β⋆⊤ASPTβ⋆∣X]=XTr[ASPT[β⋆β⋆⊤]]=XTr(ASPT). _XE_β [β A_SP_Tβ X ]=E_X [A_SP_TE[β β ] ]=E_X (A_SP_T). (53) Substituting (51) and (53) into (45) gives β⋆[p]≃DE _β \! [ W2SG_p ] DE XTr(DSPT)+σ2nXTr[DSΣ^†]−2XTr(ASPT). _X (D_SP_T)+ σ^2nE_X [D_S ]-2E_X (A_SP_T). (54) We now evaluate the three remaining Stage-I expectations in (54): XTr(DSPT),XTr(DSΣ^†),XTr(ASPT).E_X (D_SP_T),E_X (D_S ),E_X (A_SP_T). At this stage, ASA_S and DSD_S are deterministic matrices because they depend only on Σ and on the Stage-I deterministic-equivalent parameters τS _S and γS _S. Thus, all remaining randomness is contained in the Stage-I design X through Σ^=1nX⊤X,PT=Σ^†Σ^. = 1nX X,P_T= . Recall that τT>0 _T>0 is determined in the ridgeless overparameterized regime by 1nTr[Σ(Σ+τTIp)−1]=1, 1n [ ( + _TI_p)^-1 ]=1, (55) or equivalently ∑i=1pλiλi+τT=n. _i=1^p _i _i+ _T=n. Also recall γT:=1nTr[Σ2(Σ+τTIp)−2]=1n∑i=1p(λiλi+τT)2. _T:= 1n [ ^2( + _TI_p)^-2 ]= 1n _i=1^p ( _i _i+ _T )^2. (56) For a fixed proportional Stage-I aspect ratio ϕS>1 _S>1, 1−γS1- _S remains bounded away from zero. Moreover, 0⪯τSΣ(Σ+τSIp)−1⪯Σ,0 _S ( + _SI_p)^-1 , and 0⪯τS2Σ(Σ+τSIp)−2⪯Σ.0 _S^2 ( + _SI_p)^-2 . Hence, under the assumed uniform bound on ‖Σ‖op\| \|_op, both ASA_S and DSD_S have uniformly bounded operator norm. Next, observe that PT=Σ^†Σ^=limλ↓0Σ^(Σ^+λIp)−1.P_T= = _λ 0 ( +λ I_p)^-1. Taking the ridgeless limit in the shrinkage deterministic equivalent (A.1) therefore gives, for every deterministic matrix B with uniformly bounded operator norm, XTr(BPT)≃DETr[BΣ(Σ+τTIp)−1].E_X (BP_T) DE \! [B ( + _TI_p)^-1 ]. (57) Applying (57) with B=DSB=D_S gives XTr(DSPT)≃DETr[DSΣ(Σ+τTIp)−1].E_X (D_SP_T) DE \! [D_S ( + _TI_p)^-1 ]. (58) Similarly, taking B=ASB=A_S gives XTr(ASPT)≃DETr[ASΣ(Σ+τTIp)−1].E_X (A_SP_T) DE \! [A_S ( + _TI_p)^-1 ]. (59) Finally, applying Lemma A.4 with B=DSB=D_S gives XTr[DSΣ^†]≃DE11−γTTr[DSΣ(Σ+τTIp)−2].E_X [D_S ] DE 11- _T [D_S ( + _TI_p)^-2 ]. (60) Therefore, σ2nXTr[DSΣ^†]≃DEσ2n(1−γT)Tr[DSΣ(Σ+τTIp)−2]. σ^2nE_X [D_S ] DE σ^2n(1- _T) [D_S ( + _TI_p)^-2 ]. (61) Substituting (58), (61), and (59) into (54) yields β⋆[p]≃DE _β \! [ W2SG_p ] DE Tr[DSΣ(Σ+τTIp)−1]+σ2n(1−γT)Tr[DSΣ(Σ+τTIp)−2]−2Tr[ASΣ(Σ+τTIp)−1]. [D_S ( + _TI_p)^-1 ]+ σ^2n(1- _T) [D_S ( + _TI_p)^-2 ]-2 [A_S ( + _TI_p)^-1 ]. (62) Recall that AS=τSΣ(Σ+τSIp)−1,A_S= _S ( + _SI_p)^-1, and DS=2τSΣ(Σ+τSIp)−1−τS21−γSΣ(Σ+τSIp)−2.D_S=2 _S ( + _SI_p)^-1- _S^21- _S ( + _SI_p)^-2. Therefore, DS−2AS=−τS21−γSΣ(Σ+τSIp)−2.D_S-2A_S=- _S^21- _S ( + _SI_p)^-2. (63) The first and third terms of (62) can therefore be combined as Tr[DSΣ(Σ+τTIp)−1]−2Tr[ASΣ(Σ+τTIp)−1]=Tr[(DS−2AS)Σ(Σ+τTIp)−1] [D_S ( + _TI_p)^-1 ]-2 [A_S ( + _TI_p)^-1 ]= [(D_S-2A_S) ( + _TI_p)^-1 ] = = −τS21−γSTr[Σ2(Σ+τSIp)−2(Σ+τTIp)−1]. - _S^21- _S [ ^2( + _SI_p)^-2( + _TI_p)^-1 ]. (64) For the noise contribution, substituting the explicit form of DSD_S into (61) gives σ2n(1−γT)Tr[DSΣ(Σ+τTIp)−2] σ^2n(1- _T) [D_S ( + _TI_p)^-2 ] = = 2σ2τSn(1−γT)Tr[Σ2(Σ+τSIp)−1(Σ+τTIp)−2]−σ2τS2n(1−γT)(1−γS)Tr[Σ2(Σ+τSIp)−2(Σ+τTIp)−2]. 2σ^2 _Sn(1- _T) [ ^2( + _SI_p)^-1( + _TI_p)^-2 ]- σ^2 _S^2n(1- _T)(1- _S) [ ^2( + _SI_p)^-2( + _TI_p)^-2 ]. (65) Combining (64) and (65) gives β⋆[p]≃DE _β \! [ W2SG_p ] DE −τS21−γSTr[Σ2(Σ+τSIp)−2(Σ+τTIp)−1]+2σ2τSn(1−γT)Tr[Σ2(Σ+τSIp)−1(Σ+τTIp)−2] - _S^21- _S [ ^2( + _SI_p)^-2( + _TI_p)^-1 ]+ 2σ^2 _Sn(1- _T) [ ^2( + _SI_p)^-1( + _TI_p)^-2 ] − - σ2τS2n(1−γT)(1−γS)Tr[Σ2(Σ+τSIp)−2(Σ+τTIp)−2]. σ^2 _S^2n(1- _T)(1- _S) [ ^2( + _SI_p)^-2( + _TI_p)^-2 ]. (66) The right-hand side is precisely pDE W2SG_p^DE in (41). Finally, because every matrix in this expression is a function of Σ , diagonalizing Σ gives (42). ∎ Lemma A.6 (Average spectral preference). Assume 1≤m<p1≤ m<p. Let Σ=Vdiag(λ1,…,λp)V⊤,λ1≥⋯≥λp>0, =Vdiag( _1,…, _p)V , _1≥·s≥ _p>0, where V=[v1,…,vp]V=[v_1,…,v_p] is orthogonal. Define Pm:=X~†X~,β^S:=Pmβ^T.P_m:= X X, β_S:=P_m β_T. Then the following statements hold: The expected projector satisfies X~[Pm]=Vdiag(ℓ1,…,ℓp)V⊤,E_ X[P_m]=Vdiag( _1,…, _p)V , where ℓi:=X~[vi⊤Pmvi]=X~‖Pmvi‖22 _i:=E_ X[v_i P_mv_i]=E_ X\|P_mv_i\|_2^2 obey 1>ℓ1≥⋯≥ℓp>0,∑i=1pℓi=m.1> _1≥·s≥ _p>0, _i=1^p _i=m. Thus higher-variance covariance eigendirections have greater expected retention, but are not subject to an exact coordinate cutoff. Proof. Write X~V=GΛ1/2 XV=G ^1/2, where Λ=diag(λ1,…,λp) =diag( _1,…, _p) and G=[g1,…,gp]G=[g_1,…,g_p] has independent (0,Im)N(0,I_m) columns. In the covariance eigenbasis, V⊤PmV=Λ1/2G⊤(GΛG⊤)−1GΛ1/2.V P_mV= ^1/2G (G G )^-1G ^1/2. Sign symmetry of each Gaussian column implies that every off-diagonal entry of this matrix has zero expectation. For a diagonal entry, define A−i:=∑j≠iλjgjgj⊤,ai:=gi⊤A−i−1gi.A_-i:= _j≠ i _jg_jg_j , a_i:=g_i A_-i^-1g_i. Because p−1≥mp-1≥ m, A−iA_-i is positive definite almost surely. The Sherman–Morrison identity gives (V⊤PmV)ii=λiai1+λiai.(V P_mV)_i= _ia_i1+ _ia_i. Hence 0<ℓi<10< _i<1. For fixed G, this expression increases with λi _i and decreases with any λj _j, j≠ij≠ i. Comparing two configurations with λi _i and λj _j interchanged, and using exchangeability of the Gaussian columns, therefore gives λi≥λj⇒ℓi≥ℓj _i≥ _j _i≥ _j. Finally, ∑i=1pℓi=[Tr(Pm)]=m, _i=1^p _i=E[Tr(P_m)]=m, because PmP_m is a rank-m orthogonal projector. ∎ Interpretation. The pseudo-labeled sample size m determines the dimension of the teacher-parameter subspace retained by Stage-I training: an m-dimensional, sample-dependent subspace is preserved, while its (p−m)(p-m)-dimensional orthogonal complement is eliminated. The retained subspace is the row space of X~ X, whereas the eliminated subspace is its null space. Importantly, these are sample-dependent directions and do not generally coincide with the covariance eigenvectors. In the covariance eigenbasis, the same mechanism appears as preferential retention of higher-variance directions in expectation, with total expected retention ∑i=1pℓi=m. _i=1^p _i=m. Thus, m determines the dimension of the retained subspace, while the sampled covariates and the population covariance determine its orientation. Assumption A.7 (Power-law covariance). We assume that the population covariance follows the power-law model Σ=diag(1,2−α,…,p−α),λi=i−α,α>1, =diag (1,2^-α,…,p^-α ), _i=i^-α, α>1, (67) where λii=1p\ _i\_i=1^p are the eigenvalues of Σ . Assumption A.8 (Proportional high-dimensional asymptotics). We work in the proportional high-dimensional regime ϕT,p:=pn⟶ϕT>1,ϕS,p:=pm⟶ϕS>1, _T,p:= pn _T>1, _S,p:= pm _S>1, (68) as p,n,m→∞p,n,m→∞. Thus both Stage I and Stage I remain overparameterized, with asymptotically nonzero sample-to-dimension ratios. In this regime, the deterministic-equivalence framework of Lemma A.1 applies under the corresponding proportional-asymptotic assumptions. The preceding spectral-preference result provides intuition for how Stage I can modify the Stage-I estimator even in the absence of explicit regularization: the Stage-I row-space projection preferentially retains directions associated with larger population variance. We now turn from this qualitative interpretation to a quantitative analysis. Under Assumptions A.7 and A.8, we specialize the deterministic-equivalent expression (62) to the power-law covariance model. A.5 Proportional high-dimensional asymptotics Theorem A.9 (Power-law limit of the deterministic-equivalent risk improvement). Under Assumptions A.7 and A.8, let pDE W2SG_p^DE denote the finite-p deterministic-equivalent Stage-I-to-Stage-I risk improvement. Then pDE⟶ W2SG_p^DE σ2ϕTsS1−γT∫01xα[2+(2−11−γS)sSxα](1+sSxα)2(1+sTxα)2x. σ^2 _Ts_S1- _T _0^1 x^α [2+ (2- 11- _S )s_Sx^α ](1+s_Sx^α)^2(1+s_Tx^α)^2\,dx. (69) Here sT,sS>0s_T,s_S>0 are the unique solutions of ∫01dx1+saxα=1ϕa,a∈T,S, _0^1 dx1+s_ax^α= 1 _a, a∈\T,S\, (70) and γa=ϕa∫01dx(1+saxα)2,a∈T,S. _a= _a _0^1 dx(1+s_ax^α)^2, a∈\T,S\. (71) Proof. For clarity, throughout this proof we use τa,p,γa,p,a∈T,S, _a,p, _a,p,a∈\T,S\, to denote the corresponding finite-p quantities. Below, we define the rescaled ridge sa,p:=pατa,p,s_a,p:=p^α _a,p, whose limit is denoted by sas_a, while γa _a denotes the limit of γa,p _a,p. Since the ridgeless fixed-point equation is ∑i=1pi−αi−α+τa,p=pϕa,p, _i=1^p i^-αi^-α+ _a,p= p _a,p, using sa,p=pατa,ps_a,p=p^α _a,p and dividing by p gives 1p∑i=1pi−αi−α+τa,p=1p∑i=1p11+sa,p(i/p)α=1ϕa,p. 1p _i=1^p i^-αi^-α+ _a,p= 1p _i=1^p 11+s_a,p(i/p)^α= 1 _a,p. (72) Hence sa,p⟶sa,a∈T,S,s_a,p s_a,~a∈\T,S\, where sa>0s_a>0 is the unique solution of ∫01dx1+saxα=1ϕa. _0^1 dx1+s_ax^α= 1 _a. (73) Consequently, τa,p=sap−α+o(p−α). _a,p=s_a\,p^-α+o(p^-α). (74) Similarly, γa,p=ϕa,p∑i=1p(i−αi−α+τa,p)2=ϕa,p1p∑i=1p1(1+sa,p(i/p)α)2. _a,p= _a,pp _i=1^p ( i^-αi^-α+ _a,p )^2= _a,p 1p _i=1^p 1 (1+s_a,p(i/p)^α )^2. (75) Therefore, γa,p⟶γa:=ϕa∫01dx(1+saxα)2. _a,p _a:= _a _0^1 dx(1+s_ax^α)^2. (76) Using (73), 1−γa=ϕa∫01[11+saxα−1(1+saxα)2]x=ϕasa∫01xα(1+saxα)2x>0. 1- _a= _a _0^1 [ 11+s_ax^α- 1(1+s_ax^α)^2 ]dx= _as_a _0^1 x^α(1+s_ax^α)^2\,dx>0. (77) Hence (1−γa,p)−1(1- _a,p)^-1 remains bounded for all sufficiently large p. For the remainder of the proof, let xi,p:=ip.x_i,p:= ip. Then i−α=p−αxi,p−α,i^-α=p^-αx_i,p^-α, and i−αi−α+τa,p=11+sa,pxi,pα. i^-αi^-α+ _a,p= 11+s_a,px_i,p^α. Consider Tr[DSΣ(Σ+τT,pIp)−1]. [D_S ( + _T,pI_p)^-1 ]. Since all matrices are diagonal in the eigenbasis of Σ , Tr[DSΣ(Σ+τT,pIp)−1]=2τS,p∑i=1pi−αi−α+τS,pi−αi−α+τT,p [D_S ( + _T,pI_p)^-1 ]=2 _S,p _i=1^p i^-αi^-α+ _S,p i^-αi^-α+ _T,p − - τS,p21−γS,p∑i=1pi−α(i−α+τS,p)2i−αi−α+τT,p. _S,p^21- _S,p _i=1^p i^-α(i^-α+ _S,p)^2 i^-αi^-α+ _T,p. (78) The two summands satisfy τS,pi−αi−α+τS,pi−αi−α+τT,p= _S,p i^-αi^-α+ _S,p i^-αi^-α+ _T,p= p−αsS,p(1+sS,pxi,pα)(1+sT,pxi,pα), p^-α s_S,p(1+s_S,px_i,p^α)(1+s_T,px_i,p^α), (79) and τS,p2i−α(i−α+τS,p)2i−αi−α+τT,p= _S,p^2 i^-α(i^-α+ _S,p)^2 i^-αi^-α+ _T,p= p−αsS,p2xi,pα(1+sS,pxi,pα)2(1+sT,pxi,pα). p^-α s_S,p^2x_i,p^α(1+s_S,px_i,p^α)^2(1+s_T,px_i,p^α). (80) Therefore, Tr[DSΣ(Σ+τT,pIp)−1] [D_S ( + _T,pI_p)^-1 ] = = p1−α[2sS,p1p∑i=1p1(1+sS,pxi,pα)(1+sT,pxi,pα)−sS,p21−γS,p1p∑i=1pxi,pα(1+sS,pxi,pα)2(1+sT,pxi,pα)]. p^1-α [2s_S,p 1p _i=1^p 1(1+s_S,px_i,p^α)(1+s_T,px_i,p^α)- s_S,p^21- _S,p 1p _i=1^p x_i,p^α(1+s_S,px_i,p^α)^2(1+s_T,px_i,p^α) ]. (81) By Riemann-sum convergence, =p1−α[2sS∫01dx(1+sSxα)(1+sTxα)−sS21−γS∫01xα(1+sSxα)2(1+sTxα)x]+o(p1−α). =p^1-α [2s_S _0^1 dx(1+s_Sx^α)(1+s_Tx^α)- s_S^21- _S _0^1 x^α(1+s_Sx^α)^2(1+s_Tx^α)\,dx ]+o(p^1-α). (82) Since α>1α>1, this term vanishes as p→∞p→∞. We next consider σ2n(1−γT,p)Tr[DSΣ(Σ+τT,pIp)−2]. σ^2n(1- _T,p) [D_S ( + _T,pI_p)^-2 ]. Expanding DSD_S gives σ2n(1−γT,p)Tr[DSΣ(Σ+τT,pIp)−2]=2σ2τS,pn(1−γT,p)∑i=1pi−αi−α+τS,pi−α(i−α+τT,p)2 σ^2n(1- _T,p) [D_S ( + _T,pI_p)^-2 ]= 2σ^2 _S,pn(1- _T,p) _i=1^p i^-αi^-α+ _S,p i^-α(i^-α+ _T,p)^2 − - σ2τS,p2n(1−γT,p)(1−γS,p)∑i=1pi−α(i−α+τS,p)2i−α(i−α+τT,p)2. σ^2 _S,p^2n(1- _T,p)(1- _S,p) _i=1^p i^-α(i^-α+ _S,p)^2 i^-α(i^-α+ _T,p)^2. (83) Since i−α(i−α+τT,p)2=pαxi,pα(1+sT,pxi,pα)2, i^-α(i^-α+ _T,p)^2=p^α x_i,p^α(1+s_T,px_i,p^α)^2, (84) we have τS,pi−αi−α+τS,pi−α(i−α+τT,p)2= _S,p i^-αi^-α+ _S,p i^-α(i^-α+ _T,p)^2= sS,pxi,pα(1+sS,pxi,pα)(1+sT,pxi,pα)2, s_S,p x_i,p^α(1+s_S,px_i,p^α)(1+s_T,px_i,p^α)^2, (85) and τS,p2i−α(i−α+τS,p)2i−α(i−α+τT,p)2= _S,p^2 i^-α(i^-α+ _S,p)^2 i^-α(i^-α+ _T,p)^2= sS,p2xi,p2α(1+sS,pxi,pα)2(1+sT,pxi,pα)2. s_S,p^2 x_i,p^2α(1+s_S,px_i,p^α)^2(1+s_T,px_i,p^α)^2. (86) Using 1n=ϕT,p, 1n= _T,pp, we obtain σ2n(1−γT,p)Tr[DSΣ(Σ+τT,pIp)−2]=2σ2ϕT,psS,p1−γT,p1p∑i=1pxi,pα(1+sS,pxi,pα)(1+sT,pxi,pα)2 σ^2n(1- _T,p) [D_S ( + _T,pI_p)^-2 ]= 2σ^2 _T,ps_S,p1- _T,p 1p _i=1^p x_i,p^α(1+s_S,px_i,p^α)(1+s_T,px_i,p^α)^2 −σ2ϕT,psS,p2(1−γT,p)(1−γS,p)1p∑i=1pxi,p2α(1+sS,pxi,pα)2(1+sT,pxi,pα)2. - σ^2 _T,ps_S,p^2(1- _T,p)(1- _S,p) 1p _i=1^p x_i,p^2α(1+s_S,px_i,p^α)^2(1+s_T,px_i,p^α)^2. (87) Taking p→∞p→∞ yields σ2n(1−γT,p)Tr[DSΣ(Σ+τT,pIp)−2]⟶2σ2ϕTsS1−γT∫01xα(1+sSxα)(1+sTxα)2x σ^2n(1- _T,p) [D_S ( + _T,pI_p)^-2 ] 2σ^2 _Ts_S1- _T _0^1 x^α(1+s_Sx^α)(1+s_Tx^α)^2\,dx −σ2ϕTsS2(1−γT)(1−γS)∫01x2α(1+sSxα)2(1+sTxα)2dx. - σ^2 _Ts_S^2(1- _T)(1- _S) _0^1 x^2α(1+s_Sx^α)^2(1+s_Tx^α)^2\,dx. (88) Thus, unlike the signal-dependent contribution, the Stage-I noise contribution remains of order one. Finally, −2Tr[ASΣ(Σ+τT,pIp)−1]=−2τS,p∑i=1pi−αi−α+τS,pi−αi−α+τT,p=−2p1−αsS,p[1p∑i=1p1(1+sS,pxi,pα)(1+sT,pxi,pα)]. -2 [A_S ( + _T,pI_p)^-1 ]=-2 _S,p _i=1^p i^-αi^-α+ _S,p i^-αi^-α+ _T,p=-2p^1-αs_S,p [ 1p _i=1^p 1(1+s_S,px_i,p^α)(1+s_T,px_i,p^α) ]. (89) Hence −2Tr[ASΣ(Σ+τT,pIp)−1]=−2sSp1−α∫01dx(1+sSxα)(1+sTxα)+o(p1−α), -2 [A_S ( + _T,pI_p)^-1 ]=-2s_Sp^1-α _0^1 dx(1+s_Sx^α)(1+s_Tx^α)+o(p^1-α), (90) and therefore this term also vanishes because α>1α>1. Moreover, the leading 2sS2s_S contribution in (82) cancels with (90). Thus, their combined contribution is Tr[DSΣ(Σ+τT,pIp)−1]−2Tr[ASΣ(Σ+τT,pIp)−1] [D_S ( + _T,pI_p)^-1 ]-2 [A_S ( + _T,pI_p)^-1 ] = = −sS21−γSp1−α∫01xα(1+sSxα)2(1+sTxα)dx+o(p1−α). - s_S^21- _Sp^1-α _0^1 x^α(1+s_Sx^α)^2(1+s_Tx^α)\,dx+o(p^1-α). (91) Since α>1α>1, the entire signal-dependent contribution vanishes in the proportional limit. A.5.1 Limiting Stage-I-to-Stage-I risk improvement Combining the preceding calculations gives pDE= W2SG_p^DE= 2σ2ϕTsS1−γT∫01xα(1+sSxα)(1+sTxα)2x−σ2ϕTsS2(1−γT)(1−γS)∫01x2α(1+sSxα)2(1+sTxα)2x 2σ^2 _Ts_S1- _T _0^1 x^α(1+s_Sx^α)(1+s_Tx^α)^2\,dx- σ^2 _Ts_S^2(1- _T)(1- _S) _0^1 x^2α(1+s_Sx^α)^2(1+s_Tx^α)^2\,dx − - sS21−γSp1−α∫01xα(1+sSxα)2(1+sTxα)x+o(1). s_S^21- _Sp^1-α _0^1 x^α(1+s_Sx^α)^2(1+s_Tx^α)\,dx+o(1). (92) Because α>1α>1, p1−α⟶0,p^1-α 0, and therefore pDE⟶ W2SG_p^DE 2σ2ϕTsS1−γT∫01xα(1+sSxα)(1+sTxα)2x−σ2ϕTsS2(1−γT)(1−γS)∫01x2α(1+sSxα)2(1+sTxα)2x. 2σ^2 _Ts_S1- _T _0^1 x^α(1+s_Sx^α)(1+s_Tx^α)^2\,dx- σ^2 _Ts_S^2(1- _T)(1- _S) _0^1 x^2α(1+s_Sx^α)^2(1+s_Tx^α)^2\,dx. (93) Combining the two terms over the common denominator (1+sSxα)2(1+s_Sx^α)^2 gives (69). Define Ay:=∫01xyα(1+sSxα)2(1+sTxα)2x.A_y:= _0^1 x^yα(1+s_Sx^α)^2(1+s_Tx^α)^2\,dx. (94) Then pDE⟶DE:= W2SG_p^DE W2SG^DE:= σ2ϕTsS1−γT[2A1+(2−11−γS)sSA2]. σ^2 _Ts_S1- _T [2A_1+ (2- 11- _S )s_SA_2 ]. (95) Define Jα:=2A1+(2−11−γS)sSA2.J_α:=2A_1+ (2- 11- _S )s_SA_2. (96) Since ϕT>1,sS>0,1−γT>0, _T>1,s_S>0,1- _T>0, the sign of the limiting deterministic-equivalent gain satisfies sgn(DE)=sgn(Jα)whenever σ2>0.sgn ( W2SG^DE )=sgn(J_α) σ^2>0. Equivalently, for σ2>0σ^2>0, DE>0⟺Jα>0. W2SG^DE>0 J_α>0. If σ2=0σ^2=0, then instead DE=0. W2SG^DE=0. Under the power-law covariance model, the proportional limit of the deterministic-equivalent risk improvement is therefore determined by α, ϕT _T, ϕS _S, and σ2σ^2 through sTs_T, sSs_S, γT _T, and γS _S. ∎ Lemma A.10 (Monotonicity and limiting values of γS _S). For every α>1α>1, the function γS _S is strictly increasing as a function of ϕS−1∈(0,1). _S^-1∈(0,1). Moreover, limϕS−1↓0γS=α−1α,limϕS−1↑1γS=1. _ _S^-1 0 _S= α-1α, _ _S^-1 1 _S=1. (97) Consequently: 1. if 1<α<21<α<2, there exists a unique ϕS,γ−1(α)∈(0,1) _S,γ^-1(α)∈(0,1) such that γS<12for 0<ϕS−1<ϕS,γ−1(α), _S< 12 0< _S^-1< _S,γ^-1(α), and γS>12for ϕS,γ−1(α)<ϕS−1<1; _S> 12 _S,γ^-1(α)< _S^-1<1; 2. if α=2α=2, γS>12∀ϕS−1∈(0,1), _S> 12 ∀\, _S^-1∈(0,1), while γS↓12(ϕS−1↓0); _S 12 ( _S^-1 0); 3. if α>2α>2, γS>12∀ϕS−1∈(0,1). _S> 12 ∀\, _S^-1∈(0,1). Proof. Recall that sS>0s_S>0 is determined implicitly by ∫01dx1+sSxα=1ϕS, _0^1 dx1+s_Sx^α= 1 _S, (98) and γS=ϕS∫01dx(1+sSxα)2. _S= _S _0^1 dx(1+s_Sx^α)^2. (99) We first derive a convenient closed-form relation for γS _S. Observe that dx(x1+sxα)=11+sxα−αsxα(1+sxα)2. ddx ( x1+sx^α )= 11+sx^α- α sx^α(1+sx^α)^2. Integrating over x∈[0,1]x∈[0,1] gives 11+s=∫01dx1+sxα−α∫01sxα(1+sxα)2x. 11+s= _0^1 dx1+sx^α-α _0^1 sx^α(1+sx^α)^2\,dx. Using sxα(1+sxα)2=11+sxα−1(1+sxα)2, sx^α(1+sx^α)^2= 11+sx^α- 1(1+sx^α)^2, we obtain ∫01dx(1+sxα)2=α−1α∫01dx1+sxα+1α(1+s). _0^1 dx(1+sx^α)^2= α-1α _0^1 dx1+sx^α+ 1α(1+s). (100) Setting s=sSs=s_S and using (98) and (99), we find γS= _S= ϕS[α−1α1ϕS+1α(1+sS)]=α−1α+ϕSα(1+sS). _S [ α-1α 1 _S+ 1α(1+s_S) ]= α-1α+ _Sα(1+s_S). (101) Equivalently, since ϕS=(∫01dx1+sSxα)−1, _S= ( _0^1 dx1+s_Sx^α )^-1, we may write γS=α−1α+1α[(1+sS)∫01dx1+sSxα]−1. _S= α-1α+ 1α [(1+s_S) _0^1 dx1+s_Sx^α ]^-1. (102) We next establish monotonicity. First, ds∫01dx1+sxα=−∫01xα(1+sxα)2dx<0. dds _0^1 dx1+sx^α=- _0^1 x^α(1+sx^α)^2\,dx<0. Hence ϕS−1=∫01dx1+sSxα _S^-1= _0^1 dx1+s_Sx^α is a strictly decreasing function of sSs_S. It therefore remains to show that γS _S is also strictly decreasing as a function of sSs_S. By (102), it is enough to show that [(1+s)∫01dx1+sxα]−1 [(1+s) _0^1 dx1+sx^α ]^-1 is strictly decreasing in s. Its logarithmic derivative is dslog[((1+s)∫01dx1+sxα)−1]=−11+s+∫01xα(1+sxα)2x∫01dx1+sxα. dds [ ((1+s) _0^1 dx1+sx^α )^-1 ]=- 11+s+ _0^1 x^α(1+sx^α)^2\,dx _0^1 dx1+sx^α. (103) Now, ∫01dx1+sxα−(1+s)∫01xα(1+sxα)2x=∫011+sxα−(1+s)xα(1+sxα)2x=∫011−xα(1+sxα)2x>0. _0^1 dx1+sx^α-(1+s) _0^1 x^α(1+sx^α)^2\,dx= _0^1 1+sx^α-(1+s)x^α(1+sx^α)^2\,dx= _0^1 1-x^α(1+sx^α)^2\,dx>0. (104) Therefore, ∫01xα(1+sxα)2x∫01dx1+sxα<11+s. _0^1 x^α(1+sx^α)^2\,dx _0^1 dx1+sx^α< 11+s. Substituting this into (103) shows that the logarithmic derivative is strictly negative. Hence γS _S is strictly decreasing in sSs_S. Since both γS _S and ϕS−1 _S^-1 are strictly decreasing functions of sSs_S, it follows that γS _S is a strictly increasing function of ϕS−1∈(0,1). _S^-1∈(0,1). We now determine the endpoint limits. As sS↓0s_S 0, ∫01dx1+sSxα⟶1, _0^1 dx1+s_Sx^α 1, so ϕS−1↑1. _S^-1 1. Moreover, from (99), γS=ϕS∫01dx(1+sSxα)2⟶1. _S= _S _0^1 dx(1+s_Sx^α)^2 1. Thus, limϕS−1↑1γS=1. _ _S^-1 1 _S=1. Next consider sS→∞s_S→∞. With the substitution u=sS1/αx,u=s_S^1/αx, we have ∫01dx1+sSxα= _0^1 dx1+s_Sx^α= sS−1/α∫0sS1/αdu1+uα. s_S^-1/α _0^s_S^1/α du1+u^α. (105) Since α>1α>1, Define Cα:=∫0∞du1+uα<∞,C_α:= _0^∞ du1+u^α<∞, and therefore ∫01dx1+sSxα∼CαsS−1/α. _0^1 dx1+s_Sx^α C_αs_S^-1/α. In particular, ϕS−1↓0. _S^-1 0. Furthermore, (1+sS)∫01dx1+sSxα∼CαsS 1−1/α⟶∞,(1+s_S) _0^1 dx1+s_Sx^α C_αs_S^\,1-1/α ∞, because α>1α>1. Hence, by (102), γS⟶α−1α. _S α-1α. Thus, limϕS−1↓0γS=α−1α. _ _S^-1 0 _S= α-1α. It remains to distinguish the three cases. If 1<α<21<α<2, then α−1α<12<1. α-1α< 12<1. Since γS _S is continuous and strictly increasing in ϕS−1 _S^-1, there exists a unique ϕS,γ−1(α)∈(0,1) _S,γ^-1(α)∈(0,1) such that γS=12. _S= 12. Consequently, γS<12for 0<ϕS−1<ϕS,γ−1(α), _S< 12 0< _S^-1< _S,γ^-1(α), whereas γS>12for ϕS,γ−1(α)<ϕS−1<1. _S> 12 _S,γ^-1(α)< _S^-1<1. If α=2α=2, then limϕS−1↓0γS=12. _ _S^-1 0 _S= 12. Strict monotonicity therefore implies γS>12∀ϕS−1∈(0,1), _S> 12 ∀\, _S^-1∈(0,1), with γS↓12as ϕS−1↓0. _S 12 _S^-1 0. Finally, if α>2α>2, then α−1α>12. α-1α> 12. Since this is already the lower endpoint limit of γS _S, strict monotonicity gives γS>12∀ϕS−1∈(0,1). _S> 12 ∀\, _S^-1∈(0,1). This proves all three cases. ∎ Lemma A.11 (Removal of the Stage-I weight). For every sT,sS>0s_T,s_S>0, A2A1≤B2(sS)B1(sS). A_2A_1≤ B_2(s_S)B_1(s_S). (106) where By(sS)=∫01xyα(1+sSxα)2xB_y(s_S)= _0^1 x^yα (1+s_Sx^α )^2\,dx (107) Proof. Define w(x):=xα(1+sSxα)2,h(x):=1(1+sTxα)2,w(x):= x^α(1+s_Sx^α)^2, h(x):= 1(1+s_Tx^α)^2, and let μ be the probability measure on [0,1][0,1] defined by dμ(x)=w(x)dx∫01w(t)t.dμ(x)= w(x)\,dx _0^1w(t)\,dt. Then, by the definitions of A1A_1 and A2A_2, A2A1=μ[xαh(x)]μ[h(x)]. A_2A_1= E_μ\! [x^αh(x) ]E_μ\! [h(x) ]. Since xαx^α is increasing on [0,1][0,1], whereas h(x)h(x) is decreasing, the two functions are oppositely monotone. Hence Covμ(xα,h(x))≤0,Cov_μ\! (x^α,h(x) )≤ 0, and therefore μ[xαh(x)]≤μ[xα]μ[h(x)].E_μ\! [x^αh(x) ] _μ\! [x^α ]E_μ\! [h(x) ]. Since μ[h(x)]>0E_μ[h(x)]>0, division gives A2A1≤μ[xα]. A_2A_1 _μ[x^α]. Finally, μ[xα]=∫01x2α(1+sSxα)2x∫01xα(1+sSxα)2x=B2(sS)B1(sS).E_μ[x^α]= _0^1 x^2α(1+s_Sx^α)^2\,dx _0^1 x^α(1+s_Sx^α)^2\,dx= B_2(s_S)B_1(s_S). This proves (106). ∎ Lemma A.12 (Scalar positivity up to the critical exponent). Let 1<α≤2.1<α≤ 2. Then, for every s>0s>0, Gα(s):= G_α(s):= ∫01dx1+sxα−(∫01dx(1+sxα)2)(2−∫01dx1+sxα)>0. _0^1 dx1+sx^α- ( _0^1 dx(1+sx^α)^2 ) (2- _0^1 dx1+sx^α )>0. (108) Proof. We first derive an identity relating the two integrals appearing in GαG_α. Since dx(x1+sxα)=11+sxα−αsxα(1+sxα)2, ddx ( x1+sx^α )= 11+sx^α- α sx^α(1+sx^α)^2, integration over [0,1][0,1] gives 11+s=∫01dx1+sxα−α∫01sxα(1+sxα)2x. 11+s= _0^1 dx1+sx^α-α _0^1 sx^α(1+sx^α)^2\,dx. Using sxα(1+sxα)2=11+sxα−1(1+sxα)2, sx^α(1+sx^α)^2= 11+sx^α- 1(1+sx^α)^2, we obtain ∫01dx(1+sxα)2=α−1α∫01dx1+sxα+1α(1+s). _0^1 dx(1+sx^α)^2= α-1α _0^1 dx1+sx^α+ 1α(1+s). (109) Moreover, ds∫01dx1+sxα=−∫01xα(1+sxα)2dx, dds _0^1 dx1+sx^α=- _0^1 x^α(1+sx^α)^2\,dx, and hence sds∫01dx1+sxα=∫01dx(1+sxα)2−∫01dx1+sxα.s dds _0^1 dx1+sx^α= _0^1 dx(1+sx^α)^2- _0^1 dx1+sx^α. (110) Combining this with (109) gives sds∫01dx1+sxα=1α[11+s−∫01dx1+sxα].s dds _0^1 dx1+sx^α= 1α [ 11+s- _0^1 dx1+sx^α ]. (111) We first examine GαG_α near s=0s=0. Expanding the integrands gives ∫01dx1+sxα= _0^1 dx1+sx^α= 1−sα+1+s22α+1+O(s3), 1- sα+1+ s^22α+1+O(s^3), (112) ∫01dx(1+sxα)2= _0^1 dx(1+sx^α)^2= 1−2sα+1+3s22α+1+O(s3). 1- 2sα+1+ 3s^22α+1+O(s^3). (113) Therefore, Gα(s)= G_α(s)= [2(α+1)2−12α+1]s2+O(s3)=2−(α−1)2(α+1)2(2α+1)s2+O(s3). [ 2(α+1)^2- 12α+1 ]s^2+O(s^3)= 2-(α-1)^2(α+1)^2(2α+1)s^2+O(s^3). (114) For 1<α≤21<α≤ 2, 2−(α−1)2>0,2-(α-1)^2>0, and consequently Gα(s)>0for all sufficiently small s>0.G_α(s)>0 all sufficiently small s>0. (115) Suppose, toward a contradiction, that GαG_α has a positive zero, and let s⋆>0s_ >0 denote its first positive zero. Then Gα(s⋆)=0,Gα(s)>0for 0<s<s⋆.G_α(s_ )=0, G_α(s)>0 0<s<s_ . Define u:=1−∫01dx1+s⋆xα.u:=1- _0^1 dx1+s_ x^α. Since s⋆>0s_ >0, 0<u<1.0<u<1. Thus, ∫01dx1+s⋆xα=1−u. _0^1 dx1+s_ x^α=1-u. The condition Gα(s⋆)=0G_α(s_ )=0 then implies ∫01dx(1+s⋆xα)2=1−u1+u. _0^1 dx(1+s_ x^α)^2= 1-u1+u. (116) Define b:=1−(α−1)uα.b:= 1-(α-1)uα. Combining (109) and (116) gives 1+s⋆=1+uα(1−u)b.1+s_ = 1+uα(1-u)b. (117) Equivalently, s⋆=u(1+b)(1−u)b.s_ = u(1+b)(1-u)b. (118) We next note that b>0b>0. If 1<α<21<α<2, then b=1−(α−1)uα>2−α>0,b= 1-(α-1)uα> 2-α>0, whereas, if α=2α=2, b=1−u2>0.b= 1-u2>0. Hence b>0for all 1<α≤2.b>0 all 1<α≤ 2. (119) Differentiating the definition of GαG_α and using (109), (111), (116), (117), and (118), a direct simplification yields Gα′(s⋆)=u(1−u)2b(αb2+2−α)(1+u)2(1+b).G_α (s_ )= u(1-u)^2b (α b^2+2-α )(1+u)^2(1+b). (120) If 1<α<21<α<2, then αb2+2−α>0.α b^2+2-α>0. If α=2α=2, then αb2+2−α=2b2>0.α b^2+2-α=2b^2>0. Together with 0<u<1,b>0,0<u<1,b>0, this implies Gα′(s⋆)>0.G_α (s_ )>0. On the other hand, s⋆s_ is the first zero reached from a region where Gα(s)>0G_α(s)>0. Since GαG_α is differentiable, this requires Gα′(s⋆)≤0,G_α (s_ )≤ 0, which is a contradiction. Therefore GαG_α has no positive zero. Since it is positive for all sufficiently small s>0s>0, we conclude that Gα(s)>0∀s>0.G_α(s)>0 ∀\,s>0. ∎ Theorem A.13 (Global proportional positivity for 1<α≤21<α≤ 2). Let 1<α≤2,ϕT>1,ϕS>1,σ2>0.1<α≤ 2, _T>1, _S>1,σ^2>0. Then DE>0. W2SG^DE>0. (121) for every nontrivial proportional Stage-I sampling fraction 0<ϕS−1<1.0< _S^-1<1. Proof. We distinguish the two possible signs of the Stage-I coefficient. If γS≤1/2 _S≤ 1/2, then 1−2γS1−γS≥0. 1-2 _S1- _S≥ 0. Since A1>0A_1>0 and A2>0A_2>0, equation (96) immediately gives DE>0 W2SG^DE>0 It remains to consider the case γS>1/2 _S>1/2. In this case, (96) shows that DE(ϕT,ϕS)>0 W2SG^DE( _T, _S)>0 is equivalent to 2γS−11−γSsSA2A1<2. 2 _S-11- _S\,s_S A_2A_1<2. (122) By Lemma A.11, A2A1≤B2(sS)B1(sS). A_2A_1≤ B_2(s_S)B_1(s_S). Recall that ϕS−1=∫01dx1+sSxα, _S^-1= _0^1 dx1+s_Sx^α, and γS=ϕS∫01dx(1+sSxα)2. _S= _S _0^1 dx(1+s_Sx^α)^2. Therefore, γS=∫01dx(1+sSxα)2∫01dx1+sSxα. _S= _0^1 dx(1+s_Sx^α)^2 _0^1 dx1+s_Sx^α. (123) It follows that 2γS−11−γS=2∫01dx(1+sSxα)2−∫01dx1+sSxα∫01dx1+sSxα−∫01dx(1+sSxα)2. 2 _S-11- _S= 2 _0^1 dx(1+s_Sx^α)^2- _0^1 dx1+s_Sx^α _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2. (124) Next, directly from the definitions of B1B_1 and B2B_2, ∫01dx1+sSxα−∫01dx(1+sSxα)2= _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2= sSB1(sS), s_SB_1(s_S), (125) 1−2∫01dx1+sSxα+∫01dx(1+sSxα)2= 1-2 _0^1 dx1+s_Sx^α+ _0^1 dx(1+s_Sx^α)^2= sS2B2(sS). s_S^2B_2(s_S). (126) Hence sSB2(sS)B1(sS)=1−2∫01dx1+sSxα+∫01dx(1+sSxα)2∫01dx1+sSxα−∫01dx(1+sSxα)2. s_S B_2(s_S)B_1(s_S)= 1-2 _0^1 dx1+s_Sx^α+ _0^1 dx(1+s_Sx^α)^2 _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2. (127) Combining (124), (127), and Lemma A.11, we obtain 2γS−11−γSsSA2A1 2 _S-11- _S\,s_S A_2A_1 ≤(2∫01dx(1+sSxα)2−∫01dx1+sSxα)(1−2∫01dx1+sSxα+∫01dx(1+sSxα)2)(∫01dx1+sSxα−∫01dx(1+sSxα)2)2. ≤ (2 _0^1 dx(1+s_Sx^α)^2- _0^1 dx1+s_Sx^α ) (1-2 _0^1 dx1+s_Sx^α+ _0^1 dx(1+s_Sx^α)^2 ) ( _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2 )^2. (128) We now apply Lemma A.12. A direct algebraic expansion gives 2(∫01dx1+sSxα−∫01dx(1+sSxα)2)2 2 ( _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2 )^2 −(2∫01dx(1+sSxα)2−∫01dx1+sSxα)(1−2∫01dx1+sSxα+∫01dx(1+sSxα)2) - (2 _0^1 dx(1+s_Sx^α)^2- _0^1 dx1+s_Sx^α ) (1-2 _0^1 dx1+s_Sx^α+ _0^1 dx(1+s_Sx^α)^2 ) = = ∫01dx1+sSxα−(∫01dx(1+sSxα)2)(2−∫01dx1+sSxα). _0^1 dx1+s_Sx^α- ( _0^1 dx(1+s_Sx^α)^2 ) (2- _0^1 dx1+s_Sx^α ). (129) By Lemma A.12, applied at s=sSs=s_S, the right-hand side is strictly positive. Hence (2∫01dx(1+sSxα)2−∫01dx1+sSxα)(1−2∫01dx1+sSxα+∫01dx(1+sSxα)2)(∫01dx1+sSxα−∫01dx(1+sSxα)2)2<2. (2 _0^1 dx(1+s_Sx^α)^2- _0^1 dx1+s_Sx^α ) (1-2 _0^1 dx1+s_Sx^α+ _0^1 dx(1+s_Sx^α)^2 ) ( _0^1 dx1+s_Sx^α- _0^1 dx(1+s_Sx^α)^2 )^2<2. (130) Together with (128), this proves (122). Therefore, DE(ϕT,ϕS)>0 W2SG^DE( _T, _S)>0 also in the case γS>1/2 _S>1/2. Thus, DE>0for all 1<α≤2,ϕT>1,ϕS>1. W2SG^DE>0 all 1<α≤ 2, _T>1, _S>1. ∎ The situation changes qualitatively when α>2α>2. By Lemma A.10, γS>12∀ϕS−1∈(0,1), _S> 12 ∀\, _S^-1∈(0,1), and hence DE>0⇔2A1>2γS−11−γSsSA2. W2SG^DE>0 2A_1> 2 _S-11- _Ss_SA_2. (131) Unlike the case 1<α≤21<α≤ 2, this inequality is not automatically satisfied. We first prove that it necessarily fails near the sparse edge. Proposition A.14 (Sparse proportional negativity for α>2α>2). Let α>2,ϕT>1,σ2>0.α>2, _T>1,σ^2>0. Then there exists ε0=ε0(α,ϕT)>0 _0= _0(α, _T)>0 such that 0<ϕS−1<ε0⟹DE(ϕT,ϕS)<0.0< _S^-1< _0 W2SG^DE( _T, _S)<0. (132) Proof. The sparse proportional limit corresponds to ϕS−1↓0⟺sS→∞. _S^-1 0 s_S→∞. From Lemma A.10, γS⟶α−1α, _S α-1α, and hence 1−2γS1−γS⟶2−α<0. 1-2 _S1- _S 2-α<0. (133) Write h1(x):=1(1+sTxα)2.h_1(x):= 1(1+s_Tx^α)^2. Then A1=∫01xα(1+sSxα)2h1(x)x.A_1= _0^1 x^α(1+s_Sx^α)^2h_1(x)\,dx. Multiplying by sSs_S, sSA1=∫01sSxα(1+sSxα)2h1(x)x.s_SA_1= _0^1 s_Sx^α(1+s_Sx^α)^2h_1(x)\,dx. For every x>0x>0, sSxα(1+sSxα)2⟶0, s_Sx^α(1+s_Sx^α)^2 0, and 0≤sSxα(1+sSxα)2≤14.0≤ s_Sx^α(1+s_Sx^α)^2≤ 14. Thus dominated convergence yields sSA1⟶0.s_SA_1 0. (134) Similarly, sS2A2=∫01(sSxα)2(1+sSxα)2h1(x)x.s_S^2A_2= _0^1 (s_Sx^α)^2(1+s_Sx^α)^2h_1(x)\,dx. For every x>0x>0, (sSxα)2(1+sSxα)2⟶1, (s_Sx^α)^2(1+s_Sx^α)^2 1, while the ratio is bounded by 11. Therefore sS2A2⟶C1(sT):=∫01dx(1+sTxα)2>0.s_S^2A_2 C_1(s_T):= _0^1 dx(1+s_Tx^α)^2>0. (135) Multiplying by sSs_S, sSJα=2sSA1+1−2γS1−γSsS2A2.s_SJ_α=2s_SA_1+ 1-2 _S1- _S\,s_S^2A_2. Using (133), (134), and (135), sSJα⟶(2−α)C1(sT)<0.s_SJ_α (2-α)C_1(s_T)<0. Therefore Jα<0J_α<0 for all sufficiently large sSs_S. Since ϕS−1 _S^-1 is a strictly decreasing function of sSs_S, there exists ε0=ε0(α,ϕT)>0 _0= _0(α, _T)>0 such that 0<ϕS−1<ε0⟹Jα<0.0< _S^-1< _0 J_α<0. Because σ2>0σ^2>0, the sign relation (96) then gives DE(ϕT,ϕS)<0. W2SG^DE( _T, _S)<0. ∎ Thus for every α>2α>2, there always exist proportional ratios p/mp/m sufficiently large for which the W2SG condition fails. We next determine when a positive region necessarily occurs near the opposite boundary ϕS−1↑1 _S^-1 1. Define the near-square quantities By(sT):= B_y(s_T):= ∫01xyα(1+sTxα)2x, _0^1 x^yα(1+s_Tx^α)^2\,dx, (136) and Lα(ϕT):=2B1(sT)−(α+1)B2(sT).L_α( _T):=2B_1(s_T)-(α+1)B_2(s_T). (137) Proposition A.15 (Near-square sign for α>2α>2). Let α>2,ϕT>1.α>2, _T>1. Then limϕS−1↑1Jα(ϕT,ϕS)=Lα(ϕT). _ _S^-1 1J_α( _T, _S)=L_α( _T). (138) Consequently: 1. if Lα(ϕT)>0L_α( _T)>0, then Jα>0J_α>0 for all ϕS−1<1 _S^-1<1 sufficiently close to 11; 2. if Lα(ϕT)<0L_α( _T)<0, then Jα<0J_α<0 for all ϕS−1<1 _S^-1<1 sufficiently close to 11; 3. if Lα(ϕT)=0L_α( _T)=0, the leading near-square limit is degenerate and a higher-order expansion is required. Proof. As ϕS−1↑1, _S^-1 1, the Stage-I fixed point gives sS↓0.s_S 0. From Fα(s)=1−sα+1+O(s2),Hα(s)=1−2sα+1+O(s2),F_α(s)=1- sα+1+O(s^2), H_α(s)=1- 2sα+1+O(s^2), we obtain γS=Hα(sS)Fα(sS)=1−sSα+1+O(sS2). _S= H_α(s_S)F_α(s_S)=1- s_Sα+1+O(s_S^2). Thus sS2γS−11−γS⟶α+1.s_S 2 _S-11- _S α+1. (139) Moreover, dominated convergence gives A1⟶B1(sT),A2⟶B2(sT).A_1 B_1(s_T),A_2 B_2(s_T). Since γS>1/2 _S>1/2 for α>2α>2, Jα=2A1−2γS−11−γSsSA2.J_α=2A_1- 2 _S-11- _Ss_SA_2. Hence Jα⟶2B1(sT)−(α+1)B2(sT)=Lα(ϕT).J_α 2B_1(s_T)-(α+1)B_2(s_T)=L_α( _T). The sign conclusions follow by continuity. ∎ Combining the sparse and near-square results immediately gives a crossing criterion. Theorem A.16 (Existence of both W2SG and non-W2SG proportional ratios for α>2α>2). Assume α>2,ϕT>1,σ2>0,α>2, _T>1,σ^2>0, and suppose Lα(ϕT)>0.L_α( _T)>0. Then there exist 0<ϕS,−1<ϕS,+−1<10< _S,-^-1< _S,+^-1<1 such that DE(ϕT,ϕS,−)<0,DE(ϕT,ϕS,+)>0. W2SG^DE( _T, _S,-)<0, W2SG^DE( _T, _S,+)>0. Equivalently, there exists a proportional ratio p/mp/m for which the W2SG condition fails and another proportional ratio p/mp/m for which it holds. Moreover, by continuity, there exists at least one ϕS,⋆−1∈(0,1) _S, ^-1∈(0,1) such that DE(ϕT,ϕS,⋆)=0. W2SG^DE( _T, _S, )=0. Proof. By Proposition A.14, DE(ϕT,ϕS)<0 W2SG^DE( _T, _S)<0 for all sufficiently small ϕS−1>0 _S^-1>0. By Proposition A.15 and Lα(ϕT)>0L_α( _T)>0, DE(ϕT,ϕS)>0 W2SG^DE( _T, _S)>0 for all ϕS−1<1 _S^-1<1 sufficiently close to 11. Choose one point from each region. Since DE(ϕT,ϕS) W2SG^DE( _T, _S) is continuous as a function of ϕS−1∈(0,1) _S^-1∈(0,1), the intermediate value theorem gives at least one zero between them. ∎ The condition Lα(ϕT)>0L_α( _T)>0 is automatically satisfied for a nontrivial range above the critical exponent. Corollary A.17 (An unconditional crossing range above α=2α=2). If 2<α≤1+2,2<α≤ 1+ 2, then Lα(ϕT)>0L_α( _T)>0 for every ϕT>1 _T>1. Consequently, if in addition σ2>0σ^2>0, then for every 2<α≤1+22<α≤ 1+ 2 and every ϕT>1 _T>1, there exist both positive-W2SG and negative-W2SG proportional Stage-I ratios. Proof. Consider the probability measure dν(x)=xαdx∫01tαt.dν(x)= x^α\,dx _0^1t^α\,dt. Then B2(sT)B1(sT)=ν[xα(1+sTxα)−2]ν[(1+sTxα)−2]. B_2(s_T)B_1(s_T)= E_ν [x^α(1+s_Tx^α)^-2 ]E_ν [(1+s_Tx^α)^-2 ]. Since xαx^α is increasing and (1+sTxα)−2(1+s_Tx^α)^-2 is strictly decreasing for sT>0s_T>0, the same covariance argument as in Lemma A.11 gives the strict inequality B2(sT)B1(sT)<∫01x2αx∫01xαx=α+12α+1. B_2(s_T)B_1(s_T)< _0^1x^2α\,dx _0^1x^α\,dx= α+12α+1. (140) Therefore Lα(ϕT)B1(sT)= L_α( _T)B_1(s_T)= 2−(α+1)B2(sT)B1(sT)>2−(α+1)22α+1=2−(α−1)22α+1. 2-(α+1) B_2(s_T)B_1(s_T)>2- (α+1)^22α+1= 2-(α-1)^22α+1. (141) For 2<α<1+2,2<α<1+ 2, the right-hand side is strictly positive. At α=1+2,α=1+ 2, the final non-strict lower bound equals zero, but (140) is strict because sT>0s_T>0. Hence Lα(ϕT)>0L_α( _T)>0 also at the endpoint. ∎ For α>1+2,α>1+ 2, the condition Lα(ϕT)>0L_α( _T)>0 may still hold, depending on the Stage-I aspect ratio ϕT _T. The preceding theorem therefore remains applicable, but positivity near the square boundary is no longer implied by α alone. If Lα(ϕT)≤0,L_α( _T)≤ 0, the sparse region remains rigorously negative, but the present argument does not rule out the possibility of a positive component at an intermediate proportional ratio. We next strengthen the preceding existence result by proving uniqueness of the proportional W2SG transition whenever Lα(ϕT)>0L_α( _T)>0. Specifically, we show that the gain crosses zero exactly once as the Stage-I sampling fraction varies from the sparse boundary to the near-square boundary. The argument relies on two auxiliary results: a Stieltjes representation for the squared Stage-I fixed-point map and a strict curvature property for a corresponding Stieltjes quotient. These are established in Lemmas A.18 and A.19, respectively, and are then used to prove Theorem A.20. Lemma A.18 (Stieltjes representation of the squared Stage-I fixed-point map). Let 0<ℓ<12,0< < 12, and define Fℓ(s):=ℓ∫01yℓ−11+syy,s>0.F_ (s):= _0^1 y -11+sy\,dy, s>0. Set Cℓ:= C_ := ∫01uℓ−1−u−ℓ1−uu, _0^1 u -1-u^- 1-u\,du, (142) qℓ(y):= q_ (y):= Cℓyℓ+∑k=0∞yk+1k+1−ℓ,0<y<1, C_ y + _k=0^∞ y^k+1k+1- , 0<y<1, (143) wℓ(y):= w_ (y):= 2ℓ2yℓ−1qℓ(y). 2 ^2y -1q_ (y). (144) Then Cℓ>0,wℓ(y)>0(0<y<1),C_ >0, w_ (y)>0 (0<y<1), and ∫01wℓ(y)y=1. _0^1w_ (y)\,dy=1. Moreover, Fℓ(s)2=∫01wℓ(y)1+syy,s>0.F_ (s)^2= _0^1 w_ (y)1+sy\,dy, s>0. (145) Thus Fℓ2F_ ^2 is itself a Stieltjes transform of a probability density on (0,1)(0,1). Proof. The proof has three steps: first we establish positivity of the density, then we compute all moments of wℓw_ , and finally we compare those moments with the power-series coefficients of Fℓ2F_ ^2. Step 1: Positivity of CℓC_ and wℓw_ . Since 0<ℓ<1/20< <1/2, we have 1−2ℓ>01-2 >0. For 0<u<10<u<1, uℓ−1−u−ℓ=uℓ−1(1−u1−2ℓ)>0.u -1-u^- =u -1 (1-u^1-2 )>0. Hence the integrand in (142) is positive on (0,1)(0,1). It is also integrable at both endpoints. Therefore Cℓ>0.C_ >0. Using 11−u=∑k=0∞uk,0<u<1, 11-u= _k=0^∞u^k, 0<u<1, and monotone convergence, we obtain the useful series representation Cℓ=∑k=0∞(1k+ℓ−1k+1−ℓ)>0.C_ = _k=0^∞ ( 1k+ - 1k+1- )>0. (146) Every term in the definition of qℓ(y)q_ (y) is therefore positive, so qℓ(y)>0,0<y<1.q_ (y)>0, 0<y<1. Since 2ℓ2yℓ−1>02 ^2y -1>0, it follows that wℓ(y)>0,0<y<1.w_ (y)>0, 0<y<1. Step 2: Moments of wℓw_ . For n≥0n≥ 0, define Mn:=∫01ynwℓ(y)y.M_n:= _0^1y^nw_ (y)\,dy. By (144) and Tonelli’s theorem, Mn= M_n= 2ℓ2∫01yn+ℓ−1(Cℓyℓ+∑k=0∞yk+1k+1−ℓ)y=2ℓ2[Cℓn+2ℓ+∑k=0∞1(k+1−ℓ)(n+k+ℓ+1)]. 2 ^2 _0^1y^n+ -1 (C_ y + _k=0^∞ y^k+1k+1- )\,dy=2 ^2 [ C_ n+2 + _k=0^∞ 1(k+1- )(n+k+ +1) ]. (147) Now use the partial-fraction identity 1(k+1−ℓ)(n+k+ℓ+1)=1n+2ℓ(1k+1−ℓ−1n+k+ℓ+1). 1(k+1- )(n+k+ +1)= 1n+2 ( 1k+1- - 1n+k+ +1 ). (148) Substituting (148) into (147) gives Mn= M_n= 2ℓ2n+2ℓ[Cℓ+∑k=0∞(1k+1−ℓ−1n+k+ℓ+1)]. 2 ^2n+2 [C_ + _k=0^∞ ( 1k+1- - 1n+k+ +1 ) ]. (149) Using (146), the terms involving (k+1−ℓ)−1(k+1- )^-1 cancel, so Mn= M_n= 2ℓ2n+2ℓ∑k=0∞(1k+ℓ−1n+k+ℓ+1)=2ℓ2n+2ℓ∑k=0n1k+ℓ. 2 ^2n+2 _k=0^∞ ( 1k+ - 1n+k+ +1 )= 2 ^2n+2 _k=0^n 1k+ . (150) The last expression can be symmetrized by observing that 1(k+ℓ)(n−k+ℓ)=1n+2ℓ(1k+ℓ+1n−k+ℓ). 1(k+ )(n-k+ )= 1n+2 ( 1k+ + 1n-k+ ). Summing over k=0,…,nk=0,…,n yields ∑k=0n1(k+ℓ)(n−k+ℓ)=2n+2ℓ∑k=0n1k+ℓ. _k=0^n 1(k+ )(n-k+ )= 2n+2 _k=0^n 1k+ . Hence Mn=ℓ2∑k=0n1(k+ℓ)(n−k+ℓ).M_n= ^2 _k=0^n 1(k+ )(n-k+ ). (151) In particular, setting n=0n=0 gives ∫01wℓ(y)y=M0=ℓ21ℓ2=1. _0^1w_ (y)\,dy=M_0= ^2 1 ^2=1. Thus wℓw_ is a probability density on (0,1)(0,1). Step 3: Matching the power series. For |s|<1|s|<1, 11+sy=∑n=0∞(−s)nyn. 11+sy= _n=0^∞(-s)^ny^n. Therefore Fℓ(s)= F_ (s)= ℓ∫01yℓ−1∑n=0∞(−s)nyny=∑n=0∞ℓ(−s)n+ℓ. _0^1y -1 _n=0^∞(-s)^ny^n\,dy= _n=0^∞ (-s)^nn+ . (152) Squaring (152) and using the Cauchy product gives Fℓ(s)2= F_ (s)^2= ∑n=0∞(−s)n[ℓ2∑k=0n1(k+ℓ)(n−k+ℓ)]=∑n=0∞(−s)nMn, _n=0^∞(-s)^n [ ^2 _k=0^n 1(k+ )(n-k+ ) ]= _n=0^∞(-s)^nM_n, (153) where the last equality follows from (151). Since Mn=∫01ynwℓ(y)y,M_n= _0^1y^nw_ (y)\,dy, we obtain Fℓ(s)2= F_ (s)^2= ∫01wℓ(y)∑n=0∞(−sy)ny=∫01wℓ(y)1+syy,|s|<1. _0^1w_ (y) _n=0^∞(-sy)^n\,dy= _0^1 w_ (y)1+sy\,dy, |s|<1. (154) Finally, both sides of (145) are real analytic for s>−1s>-1. Since they agree on the nonempty interval (−1,1)(-1,1), the identity theorem extends the equality to the whole connected domain s>−1s>-1, and hence in particular to every s>0s>0. ∎ Lemma A.19 (Strict curvature at stationary points of a Stieltjes quotient). Let w be integrable and positive almost everywhere on (0,1)(0,1). Suppose r is continuous on [0,1][0,1], satisfies r(0)=r(1)=0,r(y)>0(0<y<1),r(0)=r(1)=0, r(y)>0 (0<y<1), and is strictly increasing up to a unique point in (0,1)(0,1) and strictly decreasing afterward. Define M(s):=∫01w(y)1+syy,N(s):=∫01r(y)w(y)1+syy,M(s):= _0^1 w(y)1+sy\,dy, N(s):= _0^1 r(y)w(y)1+sy\,dy, and Q(s):=N(s)M(s).Q(s):= N(s)M(s). Then every stationary point s0>0s_0>0 of Q satisfies Q′(s0)<0.Q (s_0)<0. Consequently, Q has at most one stationary point on (0,∞)(0,∞). Proof. Suppose Q′(s0)=0Q (s_0)=0, and set c:=Q(s0)c:=Q(s_0). Since c is a weighted average of r with a positive weight, 0<c<max[0,1]r.0<c< _[0,1]r. By strict unimodality of r, there are exactly two points 0<u<v<10<u<v<1 such that r(u)=r(v)=cr(u)=r(v)=c, and r−cr-c has sign pattern −,+,−-\,, +\,, - on the three intervals determined by u and v. Put g(y):=(r(y)−c)w(y),T(s):=N(s)−cM(s).g(y):=(r(y)-c)w(y), T(s):=N(s)-cM(s). Since T=M(Q−c)T=M(Q-c) and Q′(s0)=0Q (s_0)=0, T(s0)=T′(s0)=0.T(s_0)=T (s_0)=0. Hence ∫01g(y)1+s0yy=0,∫01yg(y)(1+s0y)2y=0. _0^1 g(y)1+s_0y\,dy=0, _0^1 yg(y)(1+s_0y)^2\,dy=0. (155) The sign pattern of g implies ∫01g(y)(y−u)(y−v)(1+s0y)3y<0. _0^1 g(y)(y-u)(y-v)(1+s_0y)^3\,dy<0. (156) Using the identity (y−u)(y−v)(1+s0y)3= (y-u)(y-v)(1+s_0y)^3= uv1+s0y−(u+v+2s0uv)y(1+s0y)2+(1+s0u)(1+s0v)y2(1+s0y)3, uv1+s_0y- (u+v+2s_0uv)y(1+s_0y)^2+ (1+s_0u)(1+s_0v)y^2(1+s_0y)^3, the first two terms vanish after integration by (155). Since (1+s0u)(1+s0v)>0(1+s_0u)(1+s_0v)>0, (156) therefore gives T′(s0)=2∫01y2g(y)(1+s0y)3y<0.T (s_0)=2 _0^1 y^2g(y)(1+s_0y)^3\,dy<0. Differentiating T=M(Q−c)T=M(Q-c) twice at s0s_0 yields M(s0)Q′(s0)=T′(s0)<0.M(s_0)Q (s_0)=T (s_0)<0. Thus Q′(s0)<0Q (s_0)<0. If Q had two stationary points, both would be strict local maxima. The minimum of Q on the closed interval joining them would then be attained at an interior point, producing a stationary point with nonnegative second derivative, a contradiction. Hence Q has at most one stationary point. ∎ Theorem A.20 (Uniqueness of the proportional transition for α>2α>2). Assume α>2,ϕT>1,σ2>0,Lα(ϕT)>0.α>2, _T>1,σ^2>0,L_α( _T)>0. Let sT>0s_T>0 be the Stage-I proportional fixed-point parameter. Then there exists a unique sS,crit>0s_S,crit>0 such that Jα(ϕT,ϕS)=0⟺sS=sS,crit.J_α( _T, _S)=0 s_S=s_S,crit. (157) Equivalently, there exists a unique proportional critical ratio ϕm,crit:=[∫01dx1+sS,critxα]−1>1 _m,crit:= [ _0^1 dx1+s_S,critx^α ]^-1>1 (158) such that DE(ϕT,ϕS)>0,1<ϕS<ϕm,crit,=0,ϕS=ϕm,crit,<0,ϕS>ϕm,crit. W2SG^DE( _T, _S) cases>0,&1< _S< _m,crit,\\[2.84526pt] =0,& _S= _m,crit,\\[2.84526pt] <0,& _S> _m,crit. cases (159) Equivalently, in terms of the Stage-I sampling fraction, DE(ϕT,ϕS)>0⟺ϕS−1>ϕm,crit−1. W2SG^DE( _T, _S)>0 _S^-1> _m,crit^-1. (160) Proof. Fix α and ϕT _T, and write t:=sT,s:=sS.t:=s_T,s:=s_S. Define F(s):=∫01dx1+sxαF(s):= _0^1 dx1+sx^α and Bt(s):=∫01xα(1+sxα)(1+txα)2x.B_t(s):= _0^1 x^α(1+sx^α)(1+tx^α)^2\,dx. (161) By the Stage-I fixed-point equation, F(s)=ϕS−1.F(s)= _S^-1. (162) Moreover, F(s)+sF′(s)=∫01dx(1+sxα)2,F(s)+sF (s)= _0^1 dx(1+sx^α)^2, and therefore γS=F+sF′F,1−γS=−sF′F. _S= F+sF F, 1- _S=- sF F. (163) In particular, F′(s)<0F (s)<0. Differentiating (161) under the integral sign gives Bt′(s)=−A2,Bt(s)+sBt′(s)=A1.B_t (s)=-A_2, B_t(s)+sB_t (s)=A_1. (164) Using (96), (163), and (164), we obtain Jα=2Bt+s1−γSBt′=2Bt−F′Bt′. J_α=2B_t+ s1- _SB_t =2B_t- FF B_t . (165) Now define Rt(s):=Bt(s)F(s)2.R_t(s):= B_t(s)F(s)^2. (166) A direct differentiation of (166) and comparison with (165) gives Jα=−F(s)3F′(s)Rt′(s).J_α=- F(s)^3F (s)R_t (s). (167) Since F>0F>0 and F′<0F <0, the prefactor in (167) is strictly positive. Thus signJα=signRt′.signJ_α=signR_t . (168) It remains to prove that Rt′R_t has at most one zero. Put ℓ:=1α∈(0,1/2). := 1α∈(0,1/2). The change of variables y=xαy=x^α gives F(s)=ℓ∫01yℓ−11+syyF(s)= _0^1 y -11+sy\,dy (169) and Bt(s)=ℓ∫01yℓ(1+ty)2(1+sy)y.B_t(s)= _0^1 y (1+ty)^2(1+sy)\,dy. (170) Let qℓq_ and wℓw_ be defined by (143)–(144). By Lemma A.18, F(s)2=∫01wℓ(y)1+syy.F(s)^2= _0^1 w_ (y)1+sy\,dy. Define rt(y):=y2ℓ(1+ty)2qℓ(y),0<y<1.r_t(y):= y2 (1+ty)^2q_ (y), 0<y<1. (171) Then (170) becomes Bt(s)=∫01rt(y)wℓ(y)1+syy,B_t(s)= _0^1 r_t(y)w_ (y)1+sy\,dy, and hence Rt(s)=∫01rt(y)wℓ(y)1+syy∫01wℓ(y)1+syy.R_t(s)= _0^1 r_t(y)w_ (y)1+sy\,dy _0^1 w_ (y)1+sy\,dy. (172) We next show that rtr_t is strictly unimodal. From (143), qℓ(y)∼Cℓyℓ(y↓0),q_ (y) C_ y (y 0), while qℓ(y)⟶∞(y↑1),q_ (y) ∞ (y 1), because the series in (143) diverges at y=1y=1. Thus rtr_t extends continuously to [0,1][0,1] with rt(0)=rt(1)=0,rt(y)>0(0<y<1).r_t(0)=r_t(1)=0,r_t(y)>0(0<y<1). Write y=evy=e^v, v<0v<0. Then qℓ(ev)=Cℓeℓv+∑k=0∞e(k+1)vk+1−ℓ.q_ (e^v)=C_ e v+ _k=0^∞ e^(k+1)vk+1- . The series and its derivatives converge locally uniformly for v<0v<0. Writing its positive terms as ajeλjva_je _jv and normalizing them to probabilities πj(v)=ajeλjvqℓ(ev), _j(v)= a_je _jvq_ (e^v), we obtain d2dv2logqℓ(ev)=∑jπj(v)λj2−(∑jπj(v)λj)2>0, d^2dv^2 q_ (e^v)= _j _j(v) _j^2- ( _j _j(v) _j )^2>0, because the exponents ℓ,1,2,… ,1,2,… are distinct. Consequently, d2dv2logrt(ev)=−2tev(1+tev)2−d2dv2logqℓ(ev)<0. d^2dv^2 r_t(e^v)=- 2te^v(1+te^v)^2- d^2dv^2 q_ (e^v)<0. (173) Hence logrt(ev) r_t(e^v) is strictly concave. Since rtr_t is positive on (0,1)(0,1) and vanishes at both endpoints, it has a unique interior maximum and is strictly increasing before that point and strictly decreasing afterward. Lemma A.19, applied to (172), now implies Rt′(s0)=0⟹Rt′(s0)<0.R_t (s_0)=0 R_t (s_0)<0. (174) In particular, RtR_t has at most one stationary point, and by (168), JαJ_α has at most one zero as a function of sSs_S. It remains only to establish existence. By the assumption Lα(ϕT)>0,L_α( _T)>0, Proposition A.15 gives Jα>0J_α>0 for sS>0s_S>0 sufficiently small. On the other hand, Proposition A.14 gives Jα<0J_α<0 for sSs_S sufficiently large. Continuity therefore gives at least one zero sS,crits_S,crit, and the preceding argument shows that this zero is unique. Moreover, Jα>0,0<sS<sS,crit,=0,sS=sS,crit,<0,sS>sS,crit.J_α cases>0,&0<s_S<s_S,crit,\\[2.84526pt] =0,&s_S=s_S,crit,\\[2.84526pt] <0,&s_S>s_S,crit. cases (175) Finally, by (162) and F′(s)<0F (s)<0, ϕS−1 _S^-1 is strictly decreasing in sSs_S, equivalently ϕS _S is strictly increasing in sSs_S. Thus (175) is exactly the sign pattern in (159). The prefactor relating DE(ϕT,ϕS) W2SG^DE( _T, _S) and JαJ_α in (96) is strictly positive for σ2>0σ^2>0, so the same sign pattern holds for the proportional deterministic-equivalent W2SG improvement. ∎ Theorem A.21 (Proportional W2SG phase diagram). Assume α>1,ϕT>1,σ2>0,0<ϕS−1<1.α>1, _T>1,σ^2>0,0< _S^-1<1. Then the proportional deterministic-equivalent W2SG limit has the following behavior. (i) Subcritical and critical decay regime: 1<α≤21<α≤ 2. If 1<α<21<α<2, there exists a unique ϕS,γ−1(α)∈(0,1) _S,γ^-1(α)∈(0,1) such that γS=12. _S= 12. For 0<ϕS−1≤ϕS,γ−1(α),0< _S^-1≤ _S,γ^-1(α), the coefficient 1−2γS1−γS 1-2 _S1- _S is nonnegative, so positivity follows directly from (96). For ϕS,γ−1(α)<ϕS−1<1, _S,γ^-1(α)< _S^-1<1, one has γS>1/2 _S>1/2, but Theorem A.13 still gives 2A1>2γS−11−γSsSA2.2A_1> 2 _S-11- _Ss_SA_2. At the critical exponent α=2α=2, there is no interior γS=1/2 _S=1/2 crossing. Instead, γS>12∀ϕS−1∈(0,1), _S> 12 ∀\, _S^-1∈(0,1), with γS↓12as ϕS−1↓0. _S 12 _S^-1 0. Nevertheless, Theorem A.13 again yields 2A1>2γS−11−γSsSA2.2A_1> 2 _S-11- _Ss_SA_2. Consequently, 1<α≤2⟹DE(ϕT,ϕS)>0∀ϕS−1∈(0,1).1<α≤ 2 W2SG^DE( _T, _S)>0 ∀\, _S^-1∈(0,1). (176) (i) Supercritical unconditional transition regime: 2<α≤1+22<α≤ 1+ 2. In this regime, γS>12∀ϕS−1∈(0,1), _S> 12 ∀\, _S^-1∈(0,1), and W2SG is governed by 2A1>2γS−11−γSsSA2.2A_1> 2 _S-11- _Ss_SA_2. By Proposition A.14, DE(ϕT,ϕS)<0 W2SG^DE( _T, _S)<0 for all sufficiently small ϕS−1>0 _S^-1>0. By Corollary A.17, Lα(ϕT)>0for every ϕT>1,L_α( _T)>0 every _T>1, including the endpoint α=1+2α=1+ 2. Hence Proposition A.15 gives DE(ϕT,ϕS)>0 W2SG^DE( _T, _S)>0 for ϕS−1 _S^-1 sufficiently close to 11. Finally, Theorem A.20 implies that there is a unique proportional critical ratio ϕm,crit>1 _m,crit>1 such that DE(ϕT,ϕS)>0,1<ϕS<ϕm,crit,=0,ϕS=ϕm,crit,<0,ϕS>ϕm,crit. W2SG^DE( _T, _S) cases>0,&1< _S< _m,crit,\\[2.84526pt] =0,& _S= _m,crit,\\[2.84526pt] <0,& _S> _m,crit. cases Equivalently, DE(ϕT,ϕS)>0⟺ϕS−1>ϕm,crit−1. W2SG^DE( _T, _S)>0 _S^-1> _m,crit^-1. (i) Conditional supercritical regime: α>1+2α>1+ 2. The sparse proportional region remains negative. If Lα(ϕT)>0,L_α( _T)>0, then Theorem A.20 again yields a unique negative-to-positive proportional transition. If Lα(ϕT)≤0,L_α( _T)≤ 0, the present analysis proves negativity near the sparse proportional boundary but does not determine whether a positive W2SG region exists at an intermediate proportional sampling fraction. A.6 Sparse-growing Stage-I sample size Assumption A.22 (Sparse-growing Stage-I DE continuation). Assume the power-law covariance model of Assumption A.7. Stage I remains in the proportional overparameterized regime, pn⟶ϕT>1,γT,p⟶γT∈(0,1). pn _T>1,~ _T,p _T∈(0,1). Stage I follows the sparse-growing regime p→∞,m=mp→∞,mpp→0.p→∞,~m=m_p→∞,~ m_pp→ 0. Along this sequence, we assume that the finite-p deterministic-equivalent expression (41) continues to apply, with τS,p>0 _S,p>0 determined by ∑i=1pλiλi+τS,p=mp, _i=1^p _i _i+ _S,p=m_p, (177) and γS,p=1mp∑i=1p(λiλi+τS,p)2. _S,p= 1m_p _i=1^p ( _i _i+ _S,p )^2. This continuation assumption is not implied by the standard proportional deterministic-equivalence theorem, because mp/p→0m_p/p→ 0. Theorem A.23 (Sparse-growing DE W2SG limit). Under Assumption A.22, pDE⟶(2−α)σ2γT1−γT. W2SG_p^DE (2-α)σ^2 _T1- _T. (178) Consequently, if σ2>0σ^2>0, the limiting gain is positive for 1<α<21<α<2, zero for α=2α=2, and negative for α>2α>2. If σ2=0σ^2=0, the limiting gain is zero for every α>1α>1. Proof. Let Cα:=∫0∞du1+uα,cα:=Cα.C_α:= _0^∞ du1+u^α, c_α:=C_α^α. Rescaling (177) by i=mpxi=m_px gives mpατS,p⟶cα.m_p^α _S,p c_α. (179) In particular, τS,p∼cαmp−α⟶0, _S,p c_αm_p^-α 0, not ∞. The same Riemann-sum argument gives γS,p⟶∫0∞dx(1+cαxα)2=α−1α. _S,p _0^∞ dx(1+c_αx^α)^2= α-1α. (180) The last equality follows from the identity used in Lemma A.10, after rescaling the integral to [0,∞)[0,∞). For the remainder of the proof, write ri,p:=2τS,pλi+τS,p−11−γS,p(τS,pλi+τS,p)2r_i,p:=2 _S,p _i+ _S,p- 11- _S,p ( _S,p _i+ _S,p )^2 and define the Stage-I noise weights wi,p:=σ2n(1−γT,p)(λiλi+τT,p)2.w_i,p:= σ^2n(1- _T,p) ( _i _i+ _T,p )^2. (181) Define the limiting Stage-I noise mass by VTnoise:=σ2γT1−γT.V_T^noise:=σ^2 _T1- _T. Then ∑i=1pwi,p=σ2γT,p1−γT,p⟶VTnoise. _i=1^pw_i,p=σ^2 _T,p1- _T,p V_T^noise. (182) Using (41), the gain can be written as pDE=Tpteach+∑i=1pri,pwi,p, W2SG_p^DE=T_p^teach+ _i=1^pr_i,pw_i,p, (183) where Tpteach=−τS,p21−γS,p∑i=1pλi2(λi+τS,p)2(λi+τT,p).T_p^teach=- _S,p^21- _S,p _i=1^p _i^2( _i+ _S,p)^2( _i+ _T,p). The teacher term vanishes. Indeed, 0≤−Tpteach≤τS,p21−γS,p∑i=1pλi(λi+τS,p)2.0≤-T_p^teach≤ _S,p^21- _S,p _i=1^p _i( _i+ _S,p)^2. If ai,p=λi/(λi+τS,p)a_i,p= _i/( _i+ _S,p), then the fixed-point equation implies mp(1−γS,p)=τS,p∑i=1pλi(λi+τS,p)2.m_p(1- _S,p)= _S,p _i=1^p _i( _i+ _S,p)^2. Consequently, 0≤−Tpteach≤mpτS,p=O(mp1−α)⟶0.0≤-T_p^teach≤ m_p _S,p=O(m_p^1-α) 0. (184) It remains to treat the noise term. The pointwise limit at a fixed index is not useful here: because τS,p→0 _S,p→ 0, it would give τS,p/(λi+τS,p)→0 _S,p/( _i+ _S,p)→ 0. Instead, the weights wi,pw_i,p are asymptotically carried by indices of order p. More precisely, limδ↓0lim supp→∞∑i≤δpwi,p=0, _δ 0 _p→∞ _i≤δ pw_i,p=0, by the Stage-I Riemann-sum limit. On the complementary set i≥δpi≥δ p, (179) gives supi≥δpλiτS,p≤O((mpp)α)⟶0. _i≥δ p _i _S,p≤ O\! ( ( m_pp )^α ) 0. Hence, uniformly for i≥δpi≥δ p, ri,p⟶2−11−(α−1)/α=2−α.r_i,p 2- 11-(α-1)/α=2-α. The coefficients ri,pr_i,p are uniformly bounded for all sufficiently large p because (180) keeps 1−γS,p1- _S,p bounded away from zero. Splitting the weighted sum at δpδ p, then sending p→∞p→∞ and δ↓0δ 0, yields ∑i=1pri,pwi,p⟶(2−α)VTnoise. _i=1^pr_i,pw_i,p (2-α)V_T^noise. (185) Combining (183), (184), and (185) proves (178). ∎ A.7 Ultra-sparse Stage-I extensions We next consider the fixed-Stage-I-sample regime, which lies outside the standard proportional asymptotic setting. Assumption A.24 (Fixed-m deterministic-equivalent continuation). Assume the power-law covariance model of Assumption A.7. Stage I remains in the proportional overparameterized regime, pn⟶ϕT>1,γT,p⟶γT∈(0,1), pn _T>1, _T,p _T∈(0,1), while the Stage I sample size m≥1m≥ 1 is fixed as p→∞p→∞. Along this sequence, we assume that the finite-p deterministic-equivalent expression (41) continues to apply, with τm,S,p>0 _m,S,p>0 determined by ∑i=1pλiλi+τm,S,p=m, _i=1^p _i _i+ _m,S,p=m, (186) and γm,S,p=1m∑i=1p(λiλi+τm,S,p)2. _m,S,p= 1m _i=1^p ( _i _i+ _m,S,p )^2. This continuation assumption is not implied by the proportional deterministic-equivalence theorem, which requires a nonvanishing Stage I sample-to-dimension ratio. A.7.1 Fixed Stage-I sample count Lemma A.25 (Fixed-m spectral parameters). For every fixed m≥1m≥ 1, there is a unique τm,S>0 _m,S>0 satisfying ∑i=1∞λiλi+τm,S=m. _i=1^∞ _i _i+ _m,S=m. (187) Moreover, τm,S,p⟶τm,S,γm,S,p⟶γm,S:=1m∑i=1∞(λiλi+τm,S)2, _m,S,p _m,S, _m,S,p _m,S:= 1m _i=1^∞ ( _i _i+ _m,S )^2, where 0<γm,S<10< _m,S<1. Define qm,S:=2−11−γm,S=1−2γm,S1−γm,S.q_m,S:=2- 11- _m,S= 1-2 _m,S1- _m,S. (188) Proof. Let f(τ):=∑i=1∞λiλi+τ.f(τ):= _i=1^∞ _i _i+τ. Because α>1α>1, this series is finite for every τ>0τ>0. It is continuous and strictly decreasing, with f(τ)→∞f(τ)→∞ as τ↓0τ 0 and f(τ)→0f(τ)→ 0 as τ→∞τ→∞. Thus (187) has a unique solution. Monotone convergence of the finite partial sums, together with uniqueness of the limiting root, gives τm,S,p→τm,S _m,S,p→ _m,S. The convergence of γm,S,p _m,S,p follows by dominated convergence. Finally, with ai=λi/(λi+τm,S)a_i= _i/( _i+ _m,S), one has 0<ai<10<a_i<1 and ∑iai=m _ia_i=m. Therefore 0<∑iai2<∑iai=m,0< _ia_i^2< _ia_i=m, which proves 0<γm,S<10< _m,S<1. ∎ Theorem A.26 (Fixed-m deterministic-equivalent W2SG gain). Under Assumption A.24, for every fixed integer m≥1m≥ 1, pDE⟶Gm:=−mτm,S+qm,Sσ2γT1−γT. W2SG_p^DE G_m:=-m _m,S+q_m,Sσ^2 _T1- _T. (189) Proof. Write pDE=Tm,pteach+∑i=1prm,i,pwi,p, W2SG_p^DE=T_m,p^teach+ _i=1^pr_m,i,pw_i,p, where wi,p:= w_i,p:= σ2n(1−γT,p)(λiλi+τT,p)2, σ^2n (1- _T,p ) ( _i _i+ _T,p )^2, Tm,pteach= T_m,p^teach= −τm,S,p21−γm,S,p∑i=1pλi2(λi+τm,S,p)2(λi+τT,p), - _m,S,p^21- _m,S,p _i=1^p _i^2( _i+ _m,S,p)^2( _i+ _T,p), rm,i,p= r_m,i,p= 2τm,S,pλi+τm,S,p−11−γm,S,p(τm,S,pλi+τm,S,p)2. 2 _m,S,p _i+ _m,S,p- 11- _m,S,p ( _m,S,p _i+ _m,S,p )^2. Since τT,p→0 _T,p→ 0 under the proportional Stage-I scaling (74), Lemma A.25 and dominated convergence give Tm,pteach T_m,p^teach ⟶−τm,S21−γm,S∑i=1∞λi(λi+τm,S)2. - _m,S^21- _m,S _i=1^∞ _i( _i+ _m,S)^2. (190) The fixed-point identity implies m(1−γm,S)= m(1- _m,S)= ∑i=1∞[λiλi+τm,S−(λiλi+τm,S)2]=τm,S∑i=1∞λi(λi+τm,S)2. _i=1^∞ [ _i _i+ _m,S- ( _i _i+ _m,S )^2 ]= _m,S _i=1^∞ _i( _i+ _m,S)^2. (191) Substitution into (190) yields Tm,pteach⟶−mτm,S.T_m,p^teach -m _m,S. (192) For the noise term, rm,i,pr_m,i,p is uniformly bounded for large p, and Lemma A.25 gives limK→∞lim supp→∞supK<i≤p|rm,i,p−qm,S|=0, _K→∞ _p→∞ _K<i≤ p|r_m,i,p-q_m,S|=0, because λi→0 _i→ 0. For each fixed K, ∑i=1Kwi,p⟶0,∑i=1pwi,p=σ2γT,p1−γT,p⟶VTnoise. _i=1^Kw_i,p 0, _i=1^pw_i,p=σ^2 _T,p1- _T,p V_T^noise. Splitting the sum at K therefore yields ∑i=1prm,i,pwi,p⟶qm,SVTnoise. _i=1^pr_m,i,pw_i,p q_m,SV_T^noise. (193) Combining (192) and (193) proves (189). ∎ A.7.2 Analytic sign results for the fixed-m gain We now analyze the sign of the fixed-m limiting gain as a continuous function of the real variable m>0m>0. Throughout this subsection, assume α>1,λi=i−α,i∈ℕ:=1,2,….α>1, _i=i^-α, i :=\1,2,…\. Let 0<γT<1,cT:=γT1−γT>0,V:=cTσ2,σ2≥0.0< _T<1, c_T:= _T1- _T>0, V:=c_Tσ^2, σ^2≥ 0. (194) For every real m>0m>0, consider the fixed-point equation fα(τ):=∑i=1∞11+τiα=m.f_α(τ):= _i=1^∞ 11+τ i^α=m. (195) Lemma A.27 below shows that this equation has a unique solution τ>0τ>0; we denote that solution by τm _m. Set ai(m):= a_i(m):= 11+τmiα, 11+ _mi^α, γm:= _m:= 1m∑i=1∞ai(m)2, 1m _i=1^∞a_i(m)^2, ηm:= _m:= 1m∑i=1∞ai(m)3, 1m _i=1^∞a_i(m)^3, (196) Dm:= D_m:= m(1−γm), m(1- _m), Bm:= B_m:= mτm, m _m, (197) qm:= q_m:= 1−2γm1−γm=2−mDm, 1-2 _m1- _m=2- mD_m, Gm:= G_m:= −Bm+Vqm. -B_m+Vq_m. (198) For integer m, uniqueness of the fixed-point solution implies τm=τm,S,γm=γm,S,qm=qm,S. _m= _m,S,~ _m= _m,S,~q_m=q_m,S. Consequently, for integer m, GmG_m coincides with the limiting deterministic-equivalent gain in Theorem A.26. Fixed-point regularity and exact identities Lemma A.27 (Existence, uniqueness, and differentiability). For every real m>0m>0, (195) has a unique solution τm>0 _m>0. The functions used in (196)–(198) are continuously differentiable, and 0<γm<1,Dm>0,qm<1.0< _m<1, D_m>0, q_m<1. (199) Moreover, τm′= _m = −τmDm, - _mD_m, (200) ai′(m)= a_i (m)= ai(m)(1−ai(m))Dm, a_i(m)(1-a_i(m))D_m, (201) Bm′= B_m = −τmγm1−γm<0, - _m _m1- _m<0, (202) mγm′= m _m = γm+γm2−2ηm1−γm. _m+ _m^2-2 _m1- _m. (203) Proof. For each τ>0τ>0, the defining series is bounded above by τ−1∑i−α<∞τ^-1 _ii^-α<∞. It is continuous and strictly decreasing in τ, tends to zero as τ→∞τ→∞, and tends to infinity as τ↓0τ 0. The last claim follows by retaining an arbitrarily large finite number of terms, each tending to one. These properties prove existence and uniqueness. On any compact subinterval of (0,∞)(0,∞), the summands and their τ-derivatives are dominated by summable constant multiples of i−αi^-α. Termwise differentiation and the inverse function theorem therefore apply. Since 0<ai<10<a_i<1 and ∑iai=m _ia_i=m, 0<∑iai2<∑iai=m,0< _ia_i^2< _ia_i=m, which proves (199). Also, −τmfα′(τm)=∑iai(1−ai)=m−∑iai2=Dm.- _mf_α ( _m)= _ia_i(1-a_i)=m- _ia_i^2=D_m. Differentiating (195) yields (200); then the chain rule gives (201) and Bm′=τm+mτm′=τm−τm1−γm,B_m = _m+m _m = _m- _m1- _m, which is (202). Finally, dm∑iai2=2Dm∑iai2(1−ai)=2(γm−ηm)1−γm. ddm _ia_i^2= 2D_m _ia_i^2(1-a_i)= 2( _m- _m)1- _m. Differentiating mγm=∑iai2m _m= _ia_i^2 and rearranging proves (203). ∎ Definition A.28 (Moment expression and noise thresholds). Define Km:=γm2(2−γm)−ηm.K_m:= _m^2(2- _m)- _m. (204) On the set where qm>0q_m>0, define thresholds in V and in σ2σ^2 by Tα(m):=Bmqm,Hα(m):=BmcTqm=Tα(m)cT.T_α(m):= B_mq_m, H_α(m):= B_mc_Tq_m= T_α(m)c_T. (205) Proposition A.29 (Exact sign criterion and threshold derivative). For every m≥1m≥ 1, qm≤0 q_m≤ 0 ⟹Gm<0for every σ2≥0, G_m<0 every σ^2≥ 0, (206) qm>0 q_m>0 ⟹sgnGm=sgn(σ2−Hα(m)). _m=sgn (σ^2-H_α(m) ). (207) Where qm>0q_m>0, one has Hα′(m)Hα(m)=Tα′(m)Tα(m)=2Kmm(1−γm)2(1−2γm). H_α (m)H_α(m)= T_α (m)T_α(m)= 2K_mm(1- _m)^2(1-2 _m). (208) Proof. Since Bm>0B_m>0, (206) follows directly from (198). If qm>0q_m>0, factor the gain as Gm=cTqm(σ2−Hα(m)).G_m=c_Tq_m (σ^2-H_α(m) ). For the derivative identity, first rewrite (203) as mγm′=γm(1−2γm)+2Km1−γm.m _m = _m(1-2 _m)+ 2K_m1- _m. (209) By (202), Bm′Bm=−γm(1−γm),qm′qm=−γm′(1−γm)(1−2γm). B_m B_m=- _mm(1- _m), q_m q_m=- _m (1- _m)(1-2 _m). Subtracting the latter identity from the former and using (209) proves (208). ∎ Integral comparisons and asymptotics Define Cα:=∫0∞dx1+xα=παsin(π/α).C_α:= _0^∞ \,dx1+x^α= πα (π/α). (210) The integral is evaluated by u=xαu=x^α and Euler’s beta integral. Lemma A.30 (Explicit fixed-point bounds). For every m>0m>0, (Cαm+1)α<τm<(Cαm)α, ( C_αm+1 )^α< _m< ( C_αm )^α, (211) and mα−14<Dm<m+1α+14. mα- 14<D_m< m+1α+ 14. (212) Consequently, Cαm(m+1)α<Bm<Cαm1−α.C_α^α m(m+1)^α<B_m<C_α^αm^1-α. (213) Proof. Let g(x):=11+τmxα.g(x):= 11+ _mx^α. Since g is strictly decreasing on (0,∞)(0,∞), the standard integral comparison is strict: ∑i≥1g(i)<∫0∞g(x)x<g(0)+∑i≥1g(i). _i≥ 1g(i)< _0^∞g(x)\,dx<g(0)+ _i≥ 1g(i). Using ∑i≥1g(i)=m,g(0)=1,∫0∞g(x)dx=Cατm−1/α, _i≥ 1g(i)=m, g(0)=1, _0^∞g(x)\,dx=C_α _m^-1/α, we obtain m<Cατm−1/α<m+1.m<C_α _m^-1/α<m+1. (214) Solving these inequalities for τm _m gives (211). Since Bm=mτmB_m=m _m, multiplication by m>0m>0 gives (213). For the bounds on DmD_m, define h(x):=τmxα(1+τmxα)2.h(x):= _mx^α(1+ _mx^α)^2. The function h increases from zero to its maximum 1/41/4 and then decreases to zero. For such an absolutely continuous unimodal function, |∑i≥1h(i)−∫0∞h(x)x|≤maxh=14. | _i≥ 1h(i)- _0^∞h(x)\,dx |≤ h= 14. (215) To justify this bound, writing x:=x−⌊x⌋\x\:=x- x gives ∑i≥1h(i)−∫0∞h(x)x=∫0∞xh′(x)x. _i≥ 1h(i)- _0^∞h(x)\,dx= _0^∞\x\h (x)\,dx. On the increasing part, the last integral contributes between zero and maxh h, while on the decreasing part it contributes between −maxh- h and zero. This proves (215). Moreover, h(x)=−xg′(x)α.h(x)=- xg (x)α. Hence integration by parts gives ∫0∞h(x)dx=1α∫0∞g(x)dx=Cατm−1/α. _0^∞h(x)\,dx= 1α _0^∞g(x)\,dx= C_α _m^-1/α. Since Dm=∑i≥1h(i),D_m= _i≥ 1h(i), equation (215) yields Cατm−1/α−14≤Dm≤Cατm−1/α+14. C_α _m^-1/α- 14≤ D_m≤ C_α _m^-1/α+ 14. Combining these inequalities with the strict comparison (214), mα<Cατm−1/α<m+1α, mα< C_α _m^-1/α< m+1α, gives mα−14<Dm<m+1α+14, mα- 14<D_m< m+1α+ 14, which proves (212). ∎ Lemma A.31 (Moment and gain asymptotics). Fix α>1α>1 and write b=1−1/αb=1-1/α. As m→∞m→∞, τm _m ∼Cαm−α, C_α^αm^-α, Bm B_m ∼Cαm1−α, C_α^αm^1-α, (216) γm _m ⟶b, b, ηm _m ⟶b(1+b)2, b(1+b)2, (217) qm q_m ⟶2−α, 2-α, Km K_m ⟶b(1−b)(2b−1)2. b(1-b)(2b-1)2. (218) In particular, limm→∞Gm=(2−α)V. _m→∞G_m=(2-α)V. (219) Proof. The first asymptotic in (216) follows by squeezing in (211); the second follows by multiplication by m. For the remaining moments, let zm=τm−1/α→∞z_m= _m^-1/α→∞ and Sk(α,z)=∑i≥11(1+(i/z)α)k,Cα,k=∫0∞dx(1+xα)k.S_k(α,z)= _i≥ 1 1(1+(i/z)^α)^k, C_α,k= _0^∞ \,dx(1+x^α)^k. Integral comparison gives 0≤zCα,k−Sk(α,z)≤1.0≤ zC_α,k-S_k(α,z)≤ 1. Integration of the derivative of x(1+xα)−kx(1+x^α)^-k over (0,∞)(0,∞) yields Cα,k+1=(1−1kα)Cα,k.C_α,k+1= (1- 1kα )C_α,k. Since Cα,1=CαC_α,1=C_α, this gives Cα,2=bCα,Cα,3=b(1+b)2Cα.C_α,2=bC_α, C_α,3= b(1+b)2C_α. At z=zmz=z_m, S1=mS_1=m, γm=S2/S1 _m=S_2/S_1, and ηm=S3/S1 _m=S_3/S_1. Taking ratios proves (217). Substitution in the definitions of qmq_m and KmK_m gives (218). Finally, Bm→0B_m→ 0, proving (219). ∎ The initial noise threshold Lemma A.32 (Positivity of q1q_1 in the exponent range of interest). For 1<α≤5/21<α≤ 5/2, one has q1>0q_1>0. Proof. If 1<α≤21<α≤ 2, then fα(1)=∑i≥111+iα≥∑i≥111+i2>∑i≥11i(i+1)=1.f_α(1)= _i≥ 1 11+i^α≥ _i≥ 1 11+i^2> _i≥ 1 1i(i+1)=1. The strict inequality holds termwise for i≥2i≥ 2. Hence τ1>1 _1>1. It follows that ai(1)≤a1(1)<1/2a_i(1)≤ a_1(1)<1/2, so γ1=∑iai(1)2≤a1(1)∑iai(1)=a1(1)<1/2. _1= _ia_i(1)^2≤ a_1(1) _ia_i(1)=a_1(1)<1/2. For 2<α≤5/22<α≤ 5/2, we first prove τ1>2/3 _1>2/3 using a finite rational lower bound. Let (k1,…,k10)=(100,142,174,200,224,245,265,283,300,317).(k_1,…,k_10)=(100,142,174,200,224,245,265,283,300,317). Each ki2≥10000ik_i^2≥ 10000i, so i≤ki/100 i≤ k_i/100. Therefore fα(2/3)≥∑i=11011+(2/3)i5/2≥∑i=110150150+i2ki>101100>1.f_α(2/3)≥ _i=1^10 11+(2/3)i^5/2≥ _i=1^10 150150+i^2k_i> 101100>1. The penultimate inequality is an inequality between the displayed finite sum of rational numbers and 101/100101/100, verified by bringing the fractions to a common denominator. Thus τ1>2/3 _1>2/3. Consequently, a1(1)<3/5a_1(1)<3/5 and ai(1)≤a2(1)<3/11a_i(1)≤ a_2(1)<3/11 for i≥2i≥ 2. Using ∑iai(1)=1 _ia_i(1)=1 gives γ1≤a1(1)2+311(1−a1(1))≤129275<12. _1≤ a_1(1)^2+ 311 (1-a_1(1) )≤ 129275< 12. Here the quadratic is bounded by its maximum at the endpoints of [0,3/5][0,3/5]. In both cases γ1<1/2 _1<1/2, hence q1>0q_1>0. ∎ Definition A.33 (Initial critical noise). For 1<α≤5/21<α≤ 5/2, define σc2:=Hα(1)=τ1cTq1>0. _c^2:=H_α(1)= _1c_Tq_1>0. (220) This is the noise level at which G1=0G_1=0. It is not, by definition, a threshold guaranteeing the same sign for every m. Corollary A.34. For 1<α≤5/21<α≤ 5/2, G1<0,0≤σ2<σc2,=0,σ2=σc2,>0,σ2>σc2.G_1 cases<0,&0≤σ^2< _c^2,\\ =0,&σ^2= _c^2,\\ >0,&σ^2> _c^2. cases For σ2<σc2σ^2< _c^2, continuity gives Gm<0G_m<0 on some real interval [1,1+δ)[1,1+δ), with δ>0δ>0. Proof. Use G1=cTq1(σ2−σc2)G_1=c_Tq_1(σ^2- _c^2) and cTq1>0c_Tq_1>0. ∎ The regime 1<α<21<α<2 Theorem A.35 (Negative initial gain and positive tail). Suppose 1<α<21<α<2 and 0<σ2<σc20<σ^2< _c^2. Then Gm<0G_m<0 near m=1m=1, and Gm>0G_m>0 for every sufficiently large m. In particular, the continuous extension has at least one zero in (1,∞)(1,∞). Proof. The initial sign follows from Corollary A.34. Since V>0V>0 and 2−α>02-α>0, Lemma A.31 gives Gm→(2−α)V>0G_m→(2-α)V>0. The intermediate value theorem proves the existence of a zero. ∎ Remark A.36. Theorem A.35 does not prove uniqueness of the zero. It does not exclude additional changes of sign at intermediate m. Positive noise is essential: for σ2=0σ^2=0, Gm=−Bm<0G_m=-B_m<0 for every m. Corollary A.37 (An explicit sufficient positive region). Suppose 1<α<21<α<2 and V>0V>0. If m≥α(α+2)4(2−α)andm>[2Cα(2−α)V]1/(α−1),m≥ α(α+2)4(2-α) m> [ 2C_α^α(2-α)V ]^1/(α-1), (221) then Gm>0G_m>0. Proof. The first inequality implies m>α/4m>α/4. By (212), qm=2−mDm≥2−αm−α/4≥2−α2.q_m=2- mD_m≥ 2- α mm-α/4≥ 2-α2. The last inequality is algebraically equivalent to the first condition in (221). Using (213), Gm≥−Cαm1−α+2−α2V>0G_m≥-C_α^αm^1-α+ 2-α2V>0 under the second condition. ∎ Theorem A.38 (Unique transition for sufficiently small positive noise). Fix 1<α<21<α<2 and cT>0c_T>0. There exists 0<σ02<σc20< _0^2< _c^2 such that, for every 0<σ2≤σ020<σ^2≤ _0^2, there is a unique real m∗>1m_*>1 with Gm<0,1≤m<m∗,=0,m=m∗,>0,m>m∗.G_m cases<0,&1≤ m<m_*,\\ =0,&m=m_*,\\ >0,&m>m_*. cases (222) A valid choice is σ02=BMcT, _0^2= B_Mc_T, (223) where M≥1M≥ 1 is any fixed number such that qm>0q_m>0 and Km<0K_m<0 for all m≥Mm≥ M. Such an M exists and depends only on α. Proof. Let b=1−1/α∈(0,1/2)b=1-1/α∈(0,1/2). By Lemma A.31, qm→2−α>0,Km→b(1−b)(2b−1)2<0.q_m→ 2-α>0, K_m→ b(1-b)(2b-1)2<0. Thus M with the stated properties exists. On [M,∞)[M,∞), Proposition A.29 implies Tα′(m)<0T_α (m)<0. Also, Tα(m)∼Cα2−αm1−α⟶0.T_α(m) C_α^α2-αm^1-α 0. (224) Now take 0<V≤BM0<V≤ B_M. Because BmB_m is decreasing and qm<1q_m<1, for 1≤m≤M1≤ m≤ M one has Gm=−Bm+Vqm<−Bm+V≤−BM+V≤0.G_m=-B_m+Vq_m<-B_m+V≤-B_M+V≤ 0. At M, 0<qM<10<q_M<1 gives Tα(M)>BM≥VT_α(M)>B_M≥ V. The continuous strictly decreasing function TαT_α therefore takes the value V exactly once on (M,∞)(M,∞). Factoring Gm=qm[V−Tα(m)]G_m=q_m[V-T_α(m)] on this tail proves (222). Finally, since BM≤B1B_M≤ B_1 and 0<q1<10<q_1<1, σ02=BMcT≤B1cT<B1cTq1=σc2. _0^2= B_Mc_T≤ B_1c_T< B_1c_Tq_1= _c^2. ∎ Corollary A.39 (Small-noise location of the unique transition). Under Theorem A.38, with α and cTc_T fixed, m∗∼[Cα(2−α)cTσ2]1/(α−1)as σ2↓0.m_* [ C_α^α(2-α)c_Tσ^2 ]^1/(α-1) σ^2 0. (225) Proof. The equation Tα(m∗)=VT_α(m_*)=V and strict positivity of TαT_α at every finite point on [M,∞)[M,∞) imply m∗→∞m_*→∞ as V↓0V 0. Substitute m=m∗m=m_* into (224) and solve the resulting asymptotic relation for m∗m_*. ∎ The critical fixed-m regime at α=2α=2 The case α=2α=2 is critical because the leading large-m limit Gm⟶(2−α)VG_m (2-α)V vanishes at α=2α=2. Consequently, this leading-order limit does not determine the eventual sign of GmG_m, and a higher-order analysis is required. At α=2α=2, we can carry out this analysis explicitly using closed-form summation identities. Here, “fixed-m” refers to the order of limits: we first take p→∞p→∞ with m fixed and then analyze the resulting limiting family as m→∞m→∞. At α=2α=2, retain the definitions Gm=−Bm+Vqm,Bm=mτm,qm=1−2γm1−γm,G_m=-B_m+Vq_m,~B_m=m _m,~q_m= 1-2 _m1- _m, where τm>0 _m>0 is determined by ∑i=1∞11+τmi2=m. _i=1^∞ 11+ _mi^2=m. (226) Lemma A.40 (Exact hyperbolic representation at α=2α=2). Let α=2α=2 and m>0m>0. Define zm:=πτm,sm:=2m+1.z_m:= π _m,s_m:=2m+1. Then zm>0z_m>0 is the unique solution of zmcothzm=sm.z_m z_m=s_m. (227) Moreover, defining δm:=sm2−zm2=zm2csch2zm, _m:=s_m^2-z_m^2=z_m^2csch^2z_m, (228) one has 0<δm<1,0< _m<1, and Dm= D_m= m(1−γm)=sm−δm4, m(1- _m)= s_m- _m4, (229) qm= q_m= 2(1−δm)sm−δm, 2(1- _m)s_m- _m, (230) Bm= B_m= π2(sm−1)2(sm2−δm). π^2(s_m-1)2(s_m^2- _m). (231) In particular, qm>0q_m>0 for every m>0m>0. Proof. For a>0a>0, the classical identity ∑i=1∞1i2+a2=πacoth(πa)−12a2 _i=1^∞ 1i^2+a^2= π a (π a)-12a^2 (232) implies, with a=zm/πa=z_m/π, ∑i=1∞11+τmi2=zm2π2∑i=1∞1i2+(zm/π)2=zmcothzm−12. _i=1^∞ 11+ _mi^2= z_m^2π^2 _i=1^∞ 1i^2+(z_m/π)^2= z_m z_m-12. (233) Using (226) therefore yields zmcothzm=2m+1=sm,z_m z_m=2m+1=s_m, which proves (227). To establish uniqueness, define f(z):=zcothz,z>0.f(z):=z z,~z>0. Then f′(z)=cothz−zcsch2z=sinhzcoshz−zsinh2z>0, f (z)= z-zcsch^2z= z z-z ^2z>0, (234) because sinh(2z)>2z (2z)>2z for every z>0z>0. Furthermore, limz↓0f(z)=1,limz→∞f(z)=∞. _z 0f(z)=1, _z→∞f(z)=∞. Since sm=2m+1>1s_m=2m+1>1, equation (227) therefore has a unique positive solution. Next, since coth2z−1=csch2z, ^2z-1=csch^2z, we obtain sm2−zm2=zm2csch2zm=δm.s_m^2-z_m^2=z_m^2csch^2z_m= _m. Moreover, sinhz>z z>z for every z>0z>0, so 0<z2csch2z=z2sinh2z<1.0<z^2csch^2z= z^2 ^2z<1. Hence 0<δm<1.0< _m<1. To obtain DmD_m, define M(τ):=∑i=1∞11+τi2.M(τ):= _i=1^∞ 11+τ i^2. Then Dm=∑i=1∞ai(1−ai)=−τmM′(τm),ai:=11+τmi2.D_m= _i=1^∞a_i(1-a_i)=- _mM ( _m), a_i:= 11+ _mi^2. Since M(τ)=zcothz−12,z=πτ−1/2,M(τ)= z z-12,~z=πτ^-1/2, and dzdτ=−z2τ, dzdτ=- z2τ, we obtain −τM′(τ)= -τ M (τ)= z4(cothz−zcsch2z). z4 ( z-zcsch^2z ). (235) Evaluating at z=zmz=z_m gives Dm=zmcothzm−zm2csch2zm4=sm−δm4.D_m= z_m z_m-z_m^2csch^2z_m4= s_m- _m4. Since qm=2−mDm,m=sm−12,q_m=2- mD_m,~m= s_m-12, we obtain qm=2−2(sm−1)sm−δm=2(1−δm)sm−δm.q_m=2- 2(s_m-1)s_m- _m= 2(1- _m)s_m- _m. Because 0<δm<10< _m<1, this also proves qm>0.q_m>0. Finally, τm=π2zm2=π2sm2−δm, _m= π^2z_m^2= π^2s_m^2- _m, and therefore Bm=mτm=π2(sm−1)2(sm2−δm).B_m=m _m= π^2(s_m-1)2(s_m^2- _m). ∎ Lemma A.41 (Second-order large-m expansion at α=2α=2). Under the assumptions of Lemma A.40, as m→∞m→∞, qm= q_m= 1m−12m2+O(m−3), 1m- 12m^2+O(m^-3), (236) Bm= B_m= π24m−π24m2+O(m−3), π^24m- π^24m^2+O(m^-3), (237) and consequently Gm=V−π2/4m+π2/4−V/2m2+O(m−3).G_m= V-π^2/4m+ π^2/4-V/2m^2+O(m^-3). (238) Proof. From (227), sm−zm=zm(cothzm−1)=2zme2zm−1.s_m-z_m=z_m( z_m-1)= 2z_me^2z_m-1. Since sm→∞s_m→∞ and z↦zcothz z z is strictly increasing and unbounded, we have zm→∞z_m→∞. Hence sm−zm→0,s_m-z_m→ 0, so zm∼sm∼2m.z_m s_m 2m. It follows that sm−zm=O(me−4m),s_m-z_m=O\! (me^-4m ), and therefore δm=(sm−zm)(sm+zm)=O(m2e−4m). _m=(s_m-z_m)(s_m+z_m)=O\! (m^2e^-4m ). In particular, δm=o(m−k) _m=o(m^-k) for every fixed k>0k>0. Using (230), qm=2(1−δm)2m+1−δm=22m+1+o(m−3). q_m= 2(1- _m)2m+1- _m= 22m+1+o(m^-3). (239) Expanding (2m+1)−1(2m+1)^-1 gives qm=1m−12m2+O(m−3).q_m= 1m- 12m^2+O(m^-3). Similarly, from (231), Bm=π2(2m)2((2m+1)2−δm)=π2m(2m+1)2+o(m−3)=π24m−π24m2+O(m−3). B_m= π^2(2m)2 ((2m+1)^2- _m )= π^2m(2m+1)^2+o(m^-3)= π^24m- π^24m^2+O(m^-3). (240) Finally, substituting these expansions into Gm=−Bm+VqmG_m=-B_m+Vq_m gives Gm=V−π2/4m+π2/4−V/2m2+O(m−3),G_m= V-π^2/4m+ π^2/4-V/2m^2+O(m^-3), as claimed. ∎ Corollary A.42 (Critical tail-noise threshold at α=2α=2). Let V=σ2γT1−γT,V=σ^2 _T1- _T, and define Vc:=π24,σtail,c2:=π241−γTγT.V_c:= π^24, _tail,c^2:= π^24 1- _T _T. (241) Then: 1. if σ2<σtail,c2,σ^2< _tail,c^2, then Gm<0G_m<0 for all sufficiently large m; 2. if σ2>σtail,c2,σ^2> _tail,c^2, then Gm>0G_m>0 for all sufficiently large m; 3. at the critical value σ2=σtail,c2,σ^2= _tail,c^2, the first-order term vanishes and Gm=π28m2+O(m−3).G_m= π^28m^2+O(m^-3). (242) Consequently, Gm>0G_m>0 for all sufficiently large m. Proof. The first two claims follow immediately from the coefficient of m−1m^-1 in (238). At the critical value V=π24,V= π^24, the coefficient of m−1m^-1 vanishes, while the coefficient of m−2m^-2 becomes π24−12π24=π28>0. π^24- 12 π^24= π^28>0. Hence Gm=π28m2+O(m−3),G_m= π^28m^2+O(m^-3), and therefore Gm>0G_m>0 for all sufficiently large m. ∎ The preceding results show why α=2α=2 is a genuine critical exponent. For every fixed positive proportional Stage-I sampling fraction and σ2>0σ^2>0, the proportional deterministic-equivalent gain remains strictly positive. By contrast, in the sparse-growing regime the order-one limit vanishes at α=2α=2, while in the fixed-m continuation the large-m behavior is controlled by the critical tail-noise level σtail,c2=π241−γTγT. _tail,c^2= π^24 1- _T _T. Thus, at the critical exponent the sign is determined by a higher-order balance that is invisible in the leading sparse-growing limit. The regime 2<α≤5/22<α≤ 5/2 For α>2α>2, the large-m behavior is qualitatively different from the case 1<α<21<α<2. Indeed, since qm⟶2−α<0,q_m 2-α<0, the gain is eventually negative independently of the noise level. The next lemma gives an explicit finite point beyond which this occurs. Lemma A.43 (Noise-independent negative tail). For every α>2α>2, define Mα:=α+42(α−2)=2+α/2α−2.M_α:= α+42(α-2)= 2+α/2α-2. (243) Then, for every real m≥Mαm≥ M_α, qm≤0,Gm<0for every σ2≥0.q_m≤ 0, G_m<0 every σ^2≥ 0. (244) Proof. From (212), Dm≤m+1α+14=4m+α+44α.D_m≤ m+1α+ 14= 4m+α+44α. If m≥Mαm≥ M_α, then 4m+α+44α≤m2, 4m+α+44α≤ m2, and therefore Dm≤m/2D_m≤ m/2. Hence qm=2−mDm≤0.q_m=2- mD_m≤ 0. The conclusion Gm<0G_m<0 now follows from (206). ∎ We next obtain an explicit lower bound for the threshold Tα(m)T_α(m) at the integer points for which qm>0q_m>0. Lemma A.44 (Explicit integer threshold bound). Suppose 2<α≤5/22<α≤ 5/2. For every integer m≥2m≥ 2 such that qm>0q_m>0, Tα(m)>Cαα+123α+1(4−α).T_α(m)>C_α^α α+123^α+1(4-α). (245) Proof. By Lemma A.43, qm>0q_m>0 implies m<Mαm<M_α. Moreover, the upper bound on DmD_m gives qm=2−mDm≤2−4αm4m+α+4=:q¯α(m).q_m=2- mD_m≤ 2- 4α m4m+α+4=: q_α(m). Thus 0<qm≤q¯α(m),q¯α(m)=2(α+4)−4(α−2)m4m+α+4.0<q_m≤ q_α(m), q_α(m)= 2(α+4)-4(α-2)m4m+α+4. (246) By the strict lower bound in (213), Bm>Cαm(m+1)α.B_m>C_α^α m(m+1)^α. Using also 0<qm≤q¯α(m)0<q_m≤ q_α(m), we obtain Tα(m)=Bmqm>Cαm(m+1)α1q¯α(m)=:Lα(m),T_α(m)= B_mq_m>C_α^α m(m+1)^α 1 q_α(m)=:L_α(m), where Lα(m)=Cα4(α−2)m(4m+α+4)(m+1)α(Mα−m).L_α(m)= C_α^α4(α-2) m(4m+α+4)(m+1)^α(M_α-m). (247) To locate the smallest value relevant to integer m≥2m≥ 2, define m0,α:=α+44(α−1).m_0,α:= α+44(α-1). For 2<α≤5/22<α≤ 5/2, 1<m0,α<32<2.1<m_0,α< 32<2. Furthermore, Lα′(m)Lα(m)=1m+44m+α+4+1Mα−m−αm+1. L_α (m)L_α(m)= 1m+ 44m+α+4+ 1M_α-m- αm+1. (248) Writing d:=α−2∈(0,1/2],t:=m−m0,α≥0,d:=α-2∈(0,1/2],t:=m-m_0,α≥ 0, and putting (248) over its positive common denominator, its numerator is, up to a positive factor, Pd(t)= P_d(t)= 64d(d+1)3t3+16d(d+1)2(d+2)(d+7)t2+4(2−d)(d+1)(d+2)(d+6)t+(3−d)(d+2)2(d+6)2. 64d(d+1)^3t^3+16d(d+1)^2(d+2)(d+7)t^2+4(2-d)(d+1)(d+2)(d+6)t+(3-d)(d+2)^2(d+6)^2. Every coefficient is strictly positive for 0<d≤1/20<d≤ 1/2. Hence Lα′(m)>0L_α (m)>0 for m0,α<m<Mα.m_0,α<m<M_α. Since every integer under consideration satisfies 2≤m<Mα,2>m0,α,2≤ m<M_α,2>m_0,α, it follows that Lα(m)≥Lα(2).L_α(m)≥ L_α(2). Finally, q¯α(2)=6(4−α)α+12, q_α(2)= 6(4-α)α+12, so Lα(2)=Cα23αα+126(4−α)=Cαα+123α+1(4−α).L_α(2)=C_α^α 23^α α+126(4-α)=C_α^α α+123^α+1(4-α). Therefore Tα(m)>Lα(m)≥Lα(2)=Cαα+123α+1(4−α),T_α(m)>L_α(m)≥ L_α(2)=C_α^α α+123^α+1(4-α), which proves (245). ∎ Theorem A.45 (Explicit no-W2SG bound for integer sample counts). Suppose 2<α≤5/22<α≤ 5/2 and cT>0c_T>0. Define σsafe2:=minσc2,CαcTα+123α+1(4−α). _safe^2:= \ _c^2,\, C_α^αc_T α+123^α+1(4-α) \. (249) Then 0≤σ2≤σsafe2⟹Gm≤0for every m∈ℕ.0≤σ^2≤ _safe^2 G_m≤ 0 every m . (250) More precisely, Gm<0for every integer m≥2G_m<0 every integer m≥ 2 (251) throughout the whole range 0≤σ2≤σsafe20≤σ^2≤ _safe^2. Furthermore, 0≤σ2<σsafe2⟹Gm<0for every m∈ℕ.0≤σ^2< _safe^2 G_m<0 every m . (252) At the endpoint σ2=σsafe2σ^2= _safe^2, equality Gm=0G_m=0 can occur only at m=1m=1, and only when σsafe2=σc2 _safe^2= _c^2. Proof. First consider m=1m=1. Since σsafe2≤σc2, _safe^2≤ _c^2, the initial sign characterization gives G1≤0whenever 0≤σ2≤σsafe2.G_1≤ 0 0≤σ^2≤ _safe^2. Moreover, G1<0G_1<0 if σ2<σsafe2,σ^2< _safe^2, while at the endpoint σ2=σsafe2σ^2= _safe^2 equality can occur only if σsafe2=σc2. _safe^2= _c^2. Now let m≥2m≥ 2 be an integer. If qm≤0q_m≤ 0, then Gm<0G_m<0 for every σ2≥0σ^2≥ 0 by (206). It remains to consider qm>0q_m>0. By Lemma A.44, Hα(m)=Tα(m)cT>CαcTα+123α+1(4−α).H_α(m)= T_α(m)c_T> C_α^αc_T α+123^α+1(4-α). Since σsafe2≤CαcTα+123α+1(4−α), _safe^2≤ C_α^αc_T α+123^α+1(4-α), we obtain the strict inequality σsafe2<Hα(m). _safe^2<H_α(m). Hence, for every 0≤σ2≤σsafe20≤σ^2≤ _safe^2, σ2<Hα(m).σ^2<H_α(m). The pointwise sign criterion (207) therefore yields Gm<0.G_m<0. Thus Gm<0G_m<0 for every integer m≥2m≥ 2, while G1≤0G_1≤ 0. This proves (250) and (251). For σ2<σsafe2σ^2< _safe^2, one also has G1<0G_1<0, which proves (252). The final endpoint statement follows from the defining property of σc2 _c^2 at m=1m=1. ∎ Remark A.46. The bound σsafe2 _safe^2 is an explicit sufficient noise threshold that rules out positive fixed-m W2SG gain for every integer Stage I sample count. At the endpoint σ2=σsafe2σ^2= _safe^2, the gain remains strictly negative for every m≥2m≥ 2; the only possible zero is at m=1m=1, when σsafe2=σc2 _safe^2= _c^2. No claim is made that σsafe2 _safe^2 is the largest possible threshold with this property. Appendix B Two-Stage Random-Feature Interpolation Theory This section develops the two-stage random-feature theory for Case I. The teacher and student operate through independent random linear feature maps, so the Stage-I predictor is constrained simultaneously by the pseudo-labeled sample size and by the dimension of the student feature space. The resulting weak-to-strong generalization behavior is therefore governed not only by the quality of the Stage-I teacher, but also by the interaction among the signal dimension, the teacher width, the number of Stage-I samples, and the student width. The analysis proceeds in two layers. We first derive a master deterministic-equivalent description of the two-stage procedure. After recording the two-point structured-Wishart deterministic equivalent needed for second-order risk calculations, we formulate the random-feature model and analyze Stage I through its exact projection geometry and deterministic-equivalent strong- and weak-block energies. We then condition on the Stage-I teacher and derive the Stage-I random-feature regression problem. Averaging successively over the Stage-I samples and the student features yields the corresponding effective-resolution equations, mimic and cross terms, and ultimately a master deterministic equivalent for the teacher-minus-student prediction-risk gap. We next specialize this master expression to the positive-signal- fraction regime. The resulting phase diagram separates the sample-bottleneck, feature-bottleneck, and no-bottleneck branches and identifies the parameter regions in which the deterministic-equivalent weak-to-strong gain is positive or negative. In particular, the analysis isolates the role of the Stage-I mismatch through a single effective quantity and shows how the location of the active Stage-I bottleneck relative to the signal dimension determines the form of the phase boundary. Finally, we examine the principal regimes excluded from the regular fixed-ratio phase diagram. We give an exact finite-dimensional classification of the Stage-I interpolation boundary, including its integrability threshold and its termination at the ambient no-bottleneck boundary. We then describe the p p critical window around the signal dimension. Within the master deterministic equivalent, this analysis also reveals an asymmetry under joint teacher–student criticality: the sample-bottleneck branch remains unfavorable, whereas a positive weak-to-strong region may survive on the feature-bottleneck branch. B.1 Deterministic equivalence In addition to Lemma A.1 from Appendix A, the derivation of the two-random-feature ridgeless-regression results requires the following two-point deterministic-equivalence lemma, introduced in Atanasov et al. [2025]. Lemma B.1 (Two-point deterministic equivalence for structured Wishart matrices). Let Ap⪰0A_p 0 be deterministic with uniformly bounded operator norm, let Zp∈ℝn×pZ_p ^n× p have independent standard Gaussian entries, and define qp:=pn,Wp:=1nZp⊤Zp,A^p:=Ap1/2WpAp1/2.q_p:= pn, W_p:= 1nZ_p Z_p, A_p:=A_p^1/2W_pA_p^1/2. Assume that qp→q∈(0,∞)q_p→ q∈(0,∞). For λ,λ′>0λ,λ >0, let κp,λ,κp,λ′>0 _p,λ, _p,λ >0 denote the renormalized ridges determined by κp,λ[1−qptr(Ap(Ap+κp,λIp)−1)]=λ, _p,λ [1-q_ptr (A_p(A_p+ _p,λI_p)^-1 ) ]=λ, and the analogous equation with λ′λ . Define T^p,λ:=A^p(A^p+λIp)−1,Gp,λ:=(Ap+κp,λIp)−1,Tp,λ:=ApGp,λ, T_p,λ:= A_p( A_p+λ I_p)^-1, G_p,λ:=(A_p+ _p,λI_p)^-1, T_p,λ:=A_pG_p,λ, with Gp,λ′G_p,λ and Tp,λ′T_p,λ defined similarly. Then, for every deterministic matrix MpM_p satisfying supp‖Mp‖op<∞ _p\|M_p\|_op<∞, the two-point deterministic equivalent of [Atanasov et al., 2025, Eq. (A.3) and the subsequent free-product specialization] specializes to T^p,λMpT^p,λ′ T_p,λM_p T_p,λ ≃Tp,λMpTp,λ′+κp,λκp,λ′Gp,λApGp,λ′qptr[ApGp,λMpGp,λ′]1−qptr[Ap2Gp,λGp,λ′]. T_p,λM_pT_p,λ + _p,λ _p,λ G_p,λA_pG_p,λ q_ptr [A_pG_p,λM_pG_p,λ ]1-q_ptr [A_p^2G_p,λG_p,λ ]. (253) Here ≃ has the strong test-matrix meaning of Lemma A.1, and the denominator is assumed to remain bounded away from zero. In particular, for λ′=λ =λ, T^p,λMpT^p,λ T_p,λM_p T_p,λ ≃Tp,λMpTp,λ+κp,λ2Gp,λApGp,λqptr[ApGp,λMpGp,λ]1−qptr[Ap2Gp,λ2]. T_p,λM_pT_p,λ+ _p,λ^2G_p,λA_pG_p,λ q_ptr [A_pG_p,λM_pG_p,λ ]1-q_ptr [A_p^2G_p,λ^2 ]. (254) Since 0⪯T^p,λ⪯Ip0 T_p,λ I_p, the same normalized-trace deterministic equivalent remains valid after taking expectation over WpW_p. B.2 Defining Random-Feature Model In the second setting, we consider a linear random-feature regression model in which the teacher and student operate through random feature representations of possibly different dimensions. Unlike Case I, where the teacher and student share the same hypothesis space and the difference arises only from the two-stage training procedure, the weak-to-strong generalization phenomenon in this setting is affected by the interaction between the random feature representations and the Stage-I training procedure. Let the input distribution in the preliminaries be x∼Px=(0,Σp)x P_x=N(0, _p), where Σp=diag(1,…,1⏟k,δp,…,δp⏟p−k),0<δp<1. _p=diag ( 1,…,1_k, _p,…, _p_p-k ), 0< _p<1. Let β⋆∈ℝpβ ^p be supported on the first k coordinates, so βi⋆=0 _i =0 for i>ki>k. The target function and noiseless observation model are ψ⋆(x)=x⊤β⋆ψ (x)=x β and y=ψ⋆(x)y=ψ (x). Thus, for the labeled dataset n=(xi,yi)i=1nD_n=\(x_i,y_i)\_i=1^n introduced in the preliminaries, yi=xi⊤β⋆y_i=x_i β . We normalize the target so that [y2]=(β⋆)⊤Σpβ⋆=1E[y^2]=(β ) _pβ =1. The teacher and student feature extractors are random linear maps, which are defined through independent random matrices UTU_T and USU_S, respectively. Case I studies its population-limit analogue: conditional on UTU_T, the teacher minimizes population risk. Consequently, n does not enter the asymptotic analysis below. Stage I (teacher). Let UT:=[uT,1|…|uT,NT]⊤∈ℝNT×p,U_T:=[u_T,1|…|u_T,N_T] ^N_T× p, where uT,1,…,uT,NT∼i.i.d.(,Ip)u_T,1,…,u_T,N_T i.i.d. 2.5[1]$ $N(0,I_p). The matrix UTU_T is randomly drawn and is not learnable. Conditional on UTU_T, the teacher feature extractor is fT(x)=UTx∈ℝNTf_T(x)=U_Tx ^N_T, and the conditional teacher hypothesis class is ℋT(UT):=hT,w:x↦⟨w,fT(x)⟩=x⊤UT⊤w:w∈ℝNT.H_T(U_T):= \h_T,w:x w,f_T(x) =x U_T w:w ^N_T \. The population version of the teacher learning rule TA_T returns the minimum-Euclidean-norm population-risk minimizer: wT⋆=argminw∈ℝNT‖w‖2subjecttow∈argminv∈ℝNTx[(x⊤UT⊤v−x⊤β⋆)2].w_T = _w ^N_T\|w\|_2 ~to w∈ _v ^N_TE_x\! [ (x U_T v-x β )^2 ]. Define the induced coefficient in the original input space by β^T=UT⊤wT⋆ β_T=U_T w_T . The resulting teacher predictor is h^T=hT,wT⋆ h_T=h_T,w_T , with h^T(x)=x⊤β^T h_T(x)=x β_T. Stage I (student). Independently of UTU_T, generate fresh feature vectors x~1,…,x~m∼i.i.d.(,Σp) x_1,…, x_m i.i.d. 2.5[1]$ $N(0, _p), and assign pseudo-labels using the teacher predictor: y~i≜h^T(x~i)=x~i⊤β^T,i∈[m]. y_i h_T( x_i)= x_i β_T, i∈[m]. The resulting Stage-I dataset is the pseudo-labeled sample from the preliminaries, ~m=(x~i,y~i)i∈[m] D_m=\( x_i, y_i)\_i∈[m]. Define the corresponding input matrix by X~≜[x~1|…|x~m]⊤∈ℝm×p X [ x_1|…| x_m] ^m× p and label vector y~≜[y~1,…,y~m]⊤∈ℝm y [ y_1,…, y_m] ^m. The student uses an independent random feature matrix US≜[uS,1|…|uS,NS]⊤∈ℝNS×p,U_S [u_S,1|…|u_S,N_S] ^N_S× p, where uS,1,…,uS,NS∼i.i.d.(,Ip)u_S,1,…,u_S,N_S i.i.d. 2.5[1]$ $N(0,I_p). The matrix USU_S is independent of UTU_T and the Stage-I samples. Conditional on USU_S, the student feature extractor is fS(x)=USx∈ℝNSf_S(x)=U_Sx ^N_S. The conditional student hypothesis class is ℋS(US):=hS,w:x↦⟨w,fS(x)⟩=x⊤US⊤w:w∈ℝNS.H_S(U_S):= \h_S,w:x w,f_S(x) =x U_S w:w ^N_S \. The Stage-I training rule SA_S corresponds to ridgeless least-squares regression in the random-feature space. We take the minimum-norm least-squares solution w^S=(X~US⊤)†y~. w_S= ( XU_S ) y. Equivalently, w^S=argminw∈ℝNS‖w‖2subjecttow∈argminv∈ℝNS‖X~US⊤v−y~‖22. w_S= _w ^N_S\|w\|_2 ~to w∈ _v ^N_S \| XU_S v- y \|_2^2. No direct interpolation constraint is imposed here, since y~ y need not lie in range(X~US⊤)range( XU_S ). The induced coefficient and student predictor are β^S=US⊤w^S β_S=U_S w_S, h^S=hS,w^S h_S=h_S, w_S, and h^S(x)=x⊤β^S h_S(x)=x β_S. For a linear predictor hβ(x):=x⊤βh_β(x):=x β, the squared-loss population risk in (2) specializes to ℛ(hβ)=x[(x⊤β−x⊤β⋆)2]=(β−β⋆)⊤Σp(β−β⋆).R(h_β)=E_x [(x β-x β )^2 ]=(β-β ) _p(β-β ). Accordingly, in the notation of the preliminaries, ℛ¯T,p=[ℛ(h^T)] R_T,p=E\! [R( h_T) ] and ℛ¯S,p=[ℛ(h^S)] R_S,p=E\! [R( h_S) ]. The expectation in ℛ¯T,p R_T,p is over UTU_T, whereas the expectation in ℛ¯S,p R_S,p is over UTU_T, USU_S, and ~m D_m. The finite-dimensional W2SG gain is p:=ℛ¯T,p−ℛ¯S,p. W2SG_p:= R_T,p- R_S,p. (255) Thus, W2SG occurs whenever p>0 W2SG_p>0. B.3 Theory of DE for stage I Lemma B.2 (Exact Stage-I estimator and projection geometry). Let AT,p:=UTΣp1/2,PT,p:=AT,p⊤(AT,pAT,p⊤)†AT,p.A_T,p:=U_T _p^1/2, P_T,p:=A_T,p (A_T,pA_T,p ) A_T,p. The minimum-norm population estimator and its predictor are β^T=UT⊤(UTΣpUT⊤)†UTΣpβ⋆,h^T(x):=x⊤β^T. β_T=U_T (U_T _pU_T ) U_T _pβ , h_T(x):=x β_T. In whitened coordinates, θT:=Σp1/2β^T=PT,pθ⋆,θ⋆:=Σp1/2β⋆, _T:= _p^1/2 β_T=P_T,pθ , θ := _p^1/2β , where PT,pP_T,p is the Euclidean orthogonal projector onto row(AT,p)row(A_T,p). Consequently, ℛ¯T,p=1−‖θT‖22=1−ST,p−VT,p, R_T,p=1-E\| _T\|_2^2=1-S_T,p-V_T,p, where ST,p:=‖θT,str‖22,VT,p:=‖θT,wk‖22.S_T,p:=E\| _T,str\|_2^2,V_T,p:=E\| _T,wk\|_2^2. Moreover, [(βstr⋆)⊤θT,str]=ST,p+VT,p.E[(β _str) _T,str]=S_T,p+V_T,p. Proof. Exact Stage-I population estimator. For a candidate coefficient β=UT⊤w,β=U_T w, the population squared loss is ℒT(w)=x[(x⊤UT⊤w−x⊤β⋆)2]=x[(x⊤(UT⊤w−β⋆))2]=(UT⊤w−β⋆)⊤Σp(UT⊤w−β⋆), _T(w)=E_x [ (x U_T w-x β )^2 ]=E_x [ (x (U_T w-β ) )^2 ]=(U_T w-β ) _p(U_T w-β ), (256) where we used [xx⊤]=Σp.E[x ]= _p. Differentiating (256) with respect to w gives ∇wℒT(w)=2UTΣp(UT⊤w−β⋆). _wL_T(w)=2U_T _p(U_T w-β ). Hence every minimizer satisfies the normal equation UTΣpUT⊤w=UTΣpβ⋆.U_T _pU_T w=U_T _pβ . (257) Choosing the minimum-Euclidean-norm solution gives wT⋆=(UTΣpUT⊤)†UTΣpβ⋆.w_T =(U_T _pU_T ) U_T _pβ . Therefore the exact Stage-I coefficient in the original input coordinates is β^T=UT⊤(UTΣpUT⊤)†UTΣpβ⋆. β_T=U_T (U_T _pU_T ) U_T _pβ . (258) Equation (258) is a finite-p identity; no deterministic-equivalent approximation has yet been used. Introduce whitened coordinates θ:=Σp1/2β,θ⋆:=Σp1/2β⋆.θ:= _p^1/2β,θ := _p^1/2β . Since the teacher is supported on the strong block, θ⋆=(βstr⋆0),‖θ⋆‖22=1.θ = pmatrixβ _str\\ 0 pmatrix, \|θ \|_2^2=1. The population prediction risk becomes ordinary Euclidean squared distance: ℛ(hβ):= (h_β):= (β−β⋆)⊤Σp(β−β⋆)=‖Σp1/2(β−β⋆)‖22=‖θ−θ⋆‖22. (β-β ) _p(β-β )= \| _p^1/2(β-β ) \|_2^2=\|θ-θ \|_2^2. (259) Define the whitened Stage-I feature matrix AT,p:=UTΣp1/2∈ℝNT×p.A_T,p:=U_T _p^1/2 ^N_T× p. (260) Multiplying (258) by Σp1/2 _p^1/2 gives θT:=Σp1/2β^T= _T:= _p^1/2 β_T= Σp1/2UT⊤(UTΣpUT⊤)†UTΣpβ⋆=AT,p⊤(AT,pAT,p⊤)†AT,pθ⋆. _p^1/2U_T (U_T _pU_T ) U_T _pβ =A_T,p (A_T,pA_T,p ) A_T,pθ . (261) Thus θT=PT,pθ⋆,PT,p:=AT,p⊤(AT,pAT,p⊤)†AT,p. _T=P_T,pθ , P_T,p:=A_T,p (A_T,pA_T,p ) A_T,p. (262) The matrix PT,pP_T,p is exactly the Euclidean orthogonal projector onto row(AT,p)⊆ℝp.row(A_T,p) ^p. Indeed, PT,p⊤=PT,p,PT,p2=PT,p,P_T,p =P_T,p,P_T,p^2=P_T,p, by the Moore–Penrose identities. Consequently, θ⋆−θT⟂θT,θ - _T _T, and therefore (θ⋆−θT)⊤θT=0.(θ - _T) _T=0. (263) By Pythagoras, ‖θ⋆−θT‖22= \|θ - _T\|_2^2= ‖θ⋆‖22−‖θT‖22=1−‖θT‖22. \|θ \|_2^2-\| _T\|_2^2=1-\| _T\|_2^2. (264) Taking expectation over UTU_T, the exact Stage-I risk is therefore ℛ¯T,p=1−‖θT‖22. R_T,p=1-E\| _T\|_2^2. (265) This exact projection identity will later allow us to determine the total Stage-I second moment without a separate random matrix calculation. Split the whitened Stage-I estimator according to the two covariance blocks: θT=(θT,strθT,wk),θT,str∈ℝk,θT,wk∈ℝp−k. _T= pmatrix _T,str\\ _T,wk pmatrix, _T,str ^k, _T,wk ^p-k. Define the aggregate strong and weak prediction energies ST,p:=‖θT,str‖22,VT,p:=‖θT,wk‖22.S_T,p:=E\| _T,str\|_2^2, V_T,p:=E\| _T,wk\|_2^2. (266) Because ‖θT‖22=‖θT,str‖22+‖θT,wk‖22,\| _T\|_2^2=\| _T,str\|_2^2+\| _T,wk\|_2^2, equation (265) yields the exact finite-p identity ℛ¯T,p=1−ST,p−VT,p. R_T,p=1-S_T,p-V_T,p. (267) Moreover, expanding (263) blockwise gives (βstr⋆−θT,str)⊤θT,str−‖θT,wk‖22=0.(β _str- _T,str) _T,str-\| _T,wk\|_2^2=0. Hence (βstr⋆−θT,str)⊤θT,str=‖θT,wk‖22.(β _str- _T,str) _T,str=\| _T,wk\|_2^2. (268) Equivalently, (βstr⋆)⊤θT,str=‖θT,str‖22+‖θT,wk‖22.(β _str) _T,str=\| _T,str\|_2^2+\| _T,wk\|_2^2. Taking expectation gives [(βstr⋆)⊤θT,str]=ST,p+VT,p.E[(β _str) _T,str]=S_T,p+V_T,p. (269) The identities (267) and (269) are exact and will be repeatedly used in the Stage-I analysis. ∎ Theorem B.3 (Stage-I deterministic equivalents). Assume the proportional high-dimensional regime specified in the preliminaries and the regularity conditions required by the one-point and two-point structured-Wishart deterministic-equivalent results. Let τT,p=0 _T,p=0 when NT≥pN_T≥ p; when NT<pN_T<p, let τT,p>0 _T,p>0 be the unique solution of dfp(τT,p)=NTp,dfp(τ):=1p[k1+τ+(p−k)δpδp+τ].df_p( _T,p)= N_Tp, _p(τ):= 1p [ k1+τ+ (p-k) _p _p+τ ]. Define aT,p:=11+τT,p,cT,p:=aT,p−aT,p2,a_T,p:= 11+ _T,p,c_T,p:=a_T,p-a_T,p^2, and ΘT,p:=[1+k(δp+τT,p)2(p−k)δp(1+τT,p)2]−1. _T,p:= [1+ k( _p+ _T,p)^2(p-k) _p(1+ _T,p)^2 ]^-1. Then ℛ¯T,p≃DEℛ¯T,pDE:=τT,p1+τT,p, R_T,p DE R_T,p^DE:= _T,p1+ _T,p, β^T≃DE(aT,pβstr⋆0),E β_T DE pmatrixa_T,pβ _str\\ 0 pmatrix, and VT,p≃DEVT,pDE:=cT,pΘT,p,ST,p≃DEST,pDE:=aT,p−cT,pΘT,p.V_T,p DE V_T,p^DE:=c_T,p _T,p, S_T,p DE S_T,p^DE:=a_T,p-c_T,p _T,p. Proof. Structured-Wishart resolvent and effective ridge. We now pass from the exact projector representation to its high-dimensional deterministic equivalent. For j=1,…,NTj=1,…,N_T, define the effective teacher-feature vector zT,j:=Σp1/2uT,j.z_T,j:= _p^1/2u_T,j. Since uT,j∼(0,Ip)u_T,j (0,I_p), zT,j∼(0,Σp).z_T,j (0, _p). Moreover, the j-th row of AT,p=UTΣp1/2A_T,p=U_T _p^1/2 is zT,j⊤z_T,j . Therefore, Σ^T,p:=1NTAT,p⊤AT,p=1NT∑j=1NTzT,jzT,j⊤=Σp1/2(1NTUT⊤UT)Σp1/2. _T,p:= 1N_TA_T,p A_T,p= 1N_T _j=1^N_Tz_T,jz_T,j = _p^1/2 ( 1N_TU_T U_T ) _p^1/2. (270) Thus Σ^T,p _T,p is the empirical covariance of NTN_T independent samples from (0,Σp)N(0, _p). In particular, Σ^T,p=Σp,E _T,p= _p, and Σ^T,p _T,p is a structured Wishart matrix with population covariance Σp _p. To verify the projector identity algebraically, let AT,p=QT,pDT,pRT,p⊤A_T,p=Q_T,pD_T,pR_T,p be a thin singular-value decomposition, where DT,p=diag(d1,…,dr),r=rank(AT,p),D_T,p=diag(d_1,…,d_r),r=rank(A_T,p), and every dj>0d_j>0. Then PT,p= P_T,p= AT,p⊤(AT,pAT,p⊤)†AT,p=RT,pDT,pQT,p⊤(QT,pDT,p2QT,p⊤)†QT,pDT,pRT,p⊤ A_T,p (A_T,pA_T,p ) A_T,p=R_T,pD_T,pQ_T,p (Q_T,pD_T,p^2Q_T,p ) Q_T,pD_T,pR_T,p = = RT,pDT,pDT,p−2DT,pRT,p⊤=RT,pRT,p⊤. R_T,pD_T,pD_T,p^-2D_T,pR_T,p =R_T,pR_T,p . (271) Moreover, Σ^T,p=RT,p(DT,p2NT)RT,p⊤, _T,p=R_T,p ( D_T,p^2N_T )R_T,p , so for every fixed bare ridge λ>0λ>0, Σ^T,p(Σ^T,p+λIp)−1= _T,p( _T,p+λ I_p)^-1= RT,pdiag(d12d12+NTλ,…,dr2dr2+NTλ)RT,p⊤. R_T,pdiag ( d_1^2d_1^2+N_Tλ,…, d_r^2d_r^2+N_Tλ )R_T,p . (272) Since limλ↓0dj2dj2+NTλ=1,j=1,…,r, _λ 0 d_j^2d_j^2+N_Tλ=1, j=1,…,r, equations (271) and (272) give PT,p=limλ↓0Σ^T,p(Σ^T,p+λIp)−1.P_T,p= _λ 0 _T,p( _T,p+λ I_p)^-1. (273) We now apply Lemma A.1(equation A.1). The population covariance in this application is Σp _p. Consequently, the empirical teacher-feature covariance resolvent has the deterministic equivalent Σ^T,p(Σ^T,p+λIp)−1 _T,p( _T,p+λ I_p)^-1 ≃DEΣp(Σp+κT,λIp)−1. DE _p( _p+ _T,λI_p)^-1. (274) where the renormalized ridge κT,λ>0 _T,λ>0 is determined by κT,λ[1−pNTdfp(κT,λ)]=λ. _T,λ [1- pN_Tdf_p( _T,λ) ]=λ. (275) Here dfp(τ):=tr[Σp(Σp+τIp)−1],df_p(τ):=tr [ _p( _p+τ I_p)^-1 ], (276) For the two-block covariance, Σp=diag(Ik,δpIp−k), _p=diag (I_k, _pI_p-k ), so dfp(τ)=1p[k1+τ+(p−k)δpδp+τ].df_p(τ)= 1p [ k1+τ+ (p-k) _p _p+τ ]. (277) We will also need the squared shrinkage trace dfp(2)(τ):=tr[Σp2(Σp+τIp)−2],df_p^(2)(τ):=tr [ _p^2( _p+τ I_p)^-2 ], (278) which in the present two-block model becomes dfp(2)(τ)=1p[k(1+τ)2+(p−k)δp2(δp+τ)2].df_p^(2)(τ)= 1p [ k(1+τ)^2+ (p-k) _p^2( _p+τ)^2 ]. (279) Suppose first that NT<p.N_T<p. Then the ridgeless limit remains on the positive effective-ridge branch. Sending λ↓0λ 0 in (275) gives κT,λ[1−pNTdfp(κT,λ)]⟶0. _T,λ [1- pN_Tdf_p( _T,λ) ] 0. Since the limiting effective ridge is positive in the underparameterized branch, its limit τT,p>0 _T,p>0 must satisfy 1−pNTdfp(τT,p)=0.1- pN_Tdf_p( _T,p)=0. Therefore, k1+τT,p+(p−k)δpδp+τT,p=NT,NT<p k1+ _T,p+ (p-k) _p _p+ _T,p=N_T, N_T<p (280) Because dfp(τ)df_p(τ) is continuous and strictly decreasing from 11 to 00, equation (280) has a unique positive solution. If instead NT≥p,N_T≥ p, then AT,pA_T,p has full column rank almost surely, so PT,p=IpP_T,p=I_p and Stage I recovers the target coefficient β⋆β exactly. In this branch we set τT,p=0. _T,p=0. From the exact projection representation, ℛ¯T,p=1−[(θ⋆)⊤PT,pθ⋆]. R_T,p=1-E [(θ ) P_T,pθ ]. Using (273) and the structured-Wishart deterministic equivalent, the ridgeless projector is replaced at leading order by Σp(Σp+τT,pIp)−1. _p( _p+ _T,pI_p)^-1. Therefore ℛ¯T,p R_T,p ≃DE1−(θ⋆)⊤Σp(Σp+τT,pIp)−1θ⋆. DE 1-(θ ) _p( _p+ _T,pI_p)^-1θ . (281) The target coefficient lies entirely in the unit-eigenvalue block, hence (θ⋆)⊤Σp(Σp+τIp)−1θ⋆=11+τ‖θ⋆‖22=11+τ.(θ ) _p( _p+τ I_p)^-1θ = 11+τ\|θ \|_2^2= 11+τ. So we have ℛ¯T,p≃DEℛ¯T,pDE:=τT,p1+τT,p. R_T,p DE R_T,p^DE:= _T,p1+ _T,p. (282) It is convenient to define aT,p:=11+τT,p.a_T,p:= 11+ _T,p. Then ℛ¯T,pDE=1−aT,p. R_T,p^DE=1-a_T,p. (283) Combining this with the exact identity ST,p+VT,p=1−ℛ¯T,pS_T,p+V_T,p=1- R_T,p already shows that the total Stage-I prediction energy satisfies ST,p+VT,p≃DEaT,p.S_T,p+V_T,p DE a_T,p. (284) We first determine the form of the mean exactly from the rotational symmetries of the model. The deterministic equivalent will only be needed afterward to determine the remaining scalar coefficient. For clarity, write the Stage-I estimator as a function of the random feature matrix: β^T(UT)=UT⊤(UTΣpUT⊤)†UTΣpβ⋆. β_T(U_T)=U_T (U_T _pU_T ) U_T _pβ . Let D∈O(p)D∈ O(p) be any orthogonal matrix satisfying DΣpD⊤=Σp,Dβ⋆=β⋆.D _pD = _p,Dβ =β . The first identity implies that DΣp=ΣpD _p= _pD. Evaluating the estimator at UTDU_TD gives β^T(UTD)= β_T(U_TD)= D⊤UT⊤(UTDΣpD⊤UT⊤)†UTDΣpβ⋆ D U_T (U_TD _pD U_T ) U_TD _pβ = = D⊤UT⊤(UTΣpUT⊤)†UTΣpDβ⋆ D U_T (U_T _pU_T ) U_T _pDβ = = D⊤UT⊤(UTΣpUT⊤)†UTΣpβ⋆ D U_T (U_T _pU_T ) U_T _pβ = = D⊤β^T(UT). D β_T(U_T). (285) Because UTU_T has independent isotropic Gaussian rows, UTD=UT.U_TD d=U_T. Combining this distributional invariance with (285) yields β^T(UT)=D⊤β^T(UT). β_T(U_T) d=D β_T(U_T). (286) We now apply this identity to rotations of the weak block. For any O∈O(p−k)O∈ O(p-k), let DO:=(Ik00O).D_O:= pmatrixI_k&0\\ 0&O pmatrix. Since Σp=(Ik00δpIp−k),β⋆=(βstr⋆0), _p= pmatrixI_k&0\\ 0& _pI_p-k pmatrix,β = pmatrixβ _str\\ 0 pmatrix, we have DOΣpDO⊤=Σp,DOβ⋆=β⋆.D_O _pD_O = _p,D_Oβ =β . Define μT,p:=β^T=(μT,str,pμT,wk,p). _T,p:=E β_T= pmatrix _T,str,p\\ _T,wk,p pmatrix. Taking expectations in (286) gives μT,p=DO⊤μT,p, _T,p=D_O _T,p, and therefore μT,wk,p=O⊤μT,wk,pfor every O∈O(p−k). _T,wk,p=O _T,wk,p every O∈ O(p-k). Choosing O=−Ip−kO=-I_p-k gives μT,wk,p=−μT,wk,p, _T,wk,p=- _T,wk,p, so the weak component of the mean vanishes exactly: [β^T,wk]=μT,wk,p=0.E[ β_T,wk]= _T,wk,p=0. (287) To determine the direction of the strong mean, consider DR:=(R00Ip−k),D_R:= pmatrixR&0\\ 0&I_p-k pmatrix, where R∈O(k)R∈ O(k) satisfies Rβstr⋆=βstr⋆.Rβ _str=β _str. Because the strong covariance block equals IkI_k, we again have DRΣpDR⊤=Σp,DRβ⋆=β⋆.D_R _pD_R = _p, D_Rβ =β . Thus μT,str,p=R⊤μT,str,p _T,str,p=R _T,str,p for every strong-block rotation that fixes βstr⋆β _str. Such rotations act arbitrarily on the orthogonal complement of βstr⋆β _str. Hence their common fixed subspace is spanβstr⋆.span\β _str\. It follows that, for some scalar bT,pb_T,p, β^T=(bT,pβstr⋆0).E β_T= pmatrixb_T,pβ _str\\ 0 pmatrix. (288) This conclusion is exact for every finite p: symmetry determines the direction of the mean, but not yet the scalar bT,pb_T,p. We next determine bT,pb_T,p. Since ‖βstr⋆‖22=(β⋆)⊤Σpβ⋆=1,\|β _str\|_2^2=(β ) _pβ =1, equation (288) implies bT,p= b_T,p= (βstr⋆)⊤[β^T,str]=[(θ⋆)⊤θT]=[(θ⋆)⊤PT,pθ⋆]. (β _str) E[ β_T,str]=E [(θ ) _T ]=E [(θ ) P_T,pθ ]. (289) Following the ridgeless branch in (273) and (274) gives PT,p≃DEΣp(Σp+τT,pIp)−1.P_T,p DE _p( _p+ _T,pI_p)^-1. In the strong deterministic-equivalence definition, choose the deterministic test matrix Mp:=θ⋆(θ⋆)⊤.M_p:=θ (θ ) . Then Tr(PT,pMp)=(θ⋆)⊤PT,pθ⋆.Tr(P_T,pM_p)=(θ ) P_T,pθ . Because PT,pP_T,p is an orthogonal projector, 0≤(θ⋆)⊤PT,pθ⋆≤‖θ⋆‖22=1,0≤(θ ) P_T,pθ ≤\|θ \|_2^2=1, so the corresponding convergence in probability also passes to the expectation. Therefore bT,p b_T,p ≃DE(θ⋆)⊤Σp(Σp+τT,pIp)−1θ⋆=11+τT,p‖θ⋆‖22=11+τT,p=aT,p. DE (θ ) _p( _p+ _T,pI_p)^-1θ = 11+ _T,p\|θ \|_2^2= 11+ _T,p=a_T,p. (290) Equivalently, the same conclusion follows entirely from the exact cross identity: bT,p=[(βstr⋆)⊤θT,str]=ST,p+VT,p≃DEaT,p.b_T,p=E[(β _str) _T,str]=S_T,p+V_T,p DE a_T,p. Consequently, β^T≃DE(aT,pβstr⋆0).E β_T DE pmatrixa_T,pβ _str\\ 0 pmatrix. (291) Finally, using the exact mean form, (β^T)⊤Σp(β^T)= (E β_T) _p(E β_T)= bT,p2(βstr⋆)⊤Ikβstr⋆=bT,p2. b_T,p^2(β _str) I_kβ _str=b_T,p^2. (292) Together with bT,p≃DEaT,pb_T,p DE a_T,p, this gives (β^T)⊤Σp(β^T)≃DEaT,p2.(E β_T) _p(E β_T) DE a_T,p^2. (293) Define the exact second-moment matrix T,p:=[β^Tβ^T⊤]. M_T,p:=E [ β_T β_T ]. Partition it according to the strong and weak covariance blocks: T,p=(s,psw,pws,pww,p). M_T,p= pmatrix M_s,p& M_sw,p\\ M_ws,p& M_w,p pmatrix. For every O∈O(p−k)O∈ O(p-k), the matrix DOD_O defined above satisfies β^T(UTDO)=DO⊤β^T(UT),UTDO=UT. β_T(U_TD_O)=D_O β_T(U_T),U_TD_O d=U_T. Therefore T,p= M_T,p= [β^T(UTDO)β^T(UTDO)⊤]=DO⊤[β^T(UT)β^T(UT)⊤]DO=DO⊤T,pDO. [ β_T(U_TD_O) β_T(U_TD_O) ]=D_O E [ β_T(U_T) β_T(U_T) ]D_O=D_O M_T,pD_O. (294) Expanding the right-hand side blockwise gives DO⊤T,pDO=(s,psw,pO⊤ws,pO⊤w,pO).D_O M_T,pD_O= pmatrix M_s,p& M_sw,pO\\ O M_ws,p&O M_w,pO pmatrix. Comparison with T,p M_T,p yields sw,pO=sw,p,O⊤ws,p=ws,p, M_sw,pO= M_sw,p, O M_ws,p= M_ws,p, and O⊤w,pO=w,pfor every O∈O(p−k).O M_w,pO= M_w,p every O∈ O(p-k). Taking O=−Ip−kO=-I_p-k in the first two identities gives sw,p=−sw,p,ws,p=−ws,p, M_sw,p=- M_sw,p, M_ws,p=- M_ws,p, so sw,p=ws,p=0. M_sw,p= M_ws,p=0. The last invariance identity states that the weak–weak block is unchanged by conjugation with every orthogonal matrix. Sign-flip matrices force all off-diagonal entries to vanish, and permutation matrices force all diagonal entries to be equal. Therefore there is a scalar νT,p≥0 _T,p≥ 0 such that w,p=νT,pIp−k. M_w,p= _T,pI_p-k. It remains to determine νT,p _T,p. On the weak block, θT,wk=δp1/2β^T,wk, _T,wk= _p^1/2 β_T,wk, and hence VT,p= V_T,p= ‖θT,wk‖22=δp‖β^T,wk‖22=δpTr(w,p)=δpνT,p(p−k). \| _T,wk\|_2^2= _pE\| β_T,wk\|_2^2= _pTr( M_w,p)= _p _T,p(p-k). (295) Thus νT,p=VT,p(p−k)δp. _T,p= V_T,p(p-k) _p. On the strong block, whitening has no effect because the corresponding eigenvalue of Σp _p is 11: θT,str=β^T,str. _T,str= β_T,str. Therefore ST,p=‖β^T,str‖22=Tr(s,p).S_T,p=E\| β_T,str\|_2^2=Tr( M_s,p). Writing T,str,p:=s,p, M_T,str,p:= M_s,p, we arrive at the exact finite-p block structure T,p=(T,str,p00VT,p(p−k)δpIp−k),TrT,str,p=ST,p. M_T,p= pmatrix M_T,str,p&0\\[2.84526pt] 0& V_T,p(p-k) _pI_p-k pmatrix, M_T,str,p=S_T,p. (296) No assumption such as k=o(p)k=o(p) has been used. The equivariance, vanishing cross blocks, weak-block isotropy, and normalization above are exact for every finite p. Let μT,p:=β^T _T,p:=E β_T and define the covariance of the Stage-I estimator by ΓT,p:=T,p−μT,pμT,p⊤. _T,p:= M_T,p- _T,p _T,p . Its total prediction-norm trace is Tr(ΣpΓT,p)= ( _p _T,p)= [β^T⊤Σpβ^T]−μT,p⊤ΣpμT,p. [ β_T _p β_T]- _T,p _p _T,p. (297) The first term equals ST,p+VT,p=1−ℛ¯T,pS_T,p+V_T,p=1- R_T,p exactly. Using (284) and (293), we obtain Tr(ΣpΓT,p)≃DEaT,p−aT,p2.Tr( _p _T,p) DE a_T,p-a_T,p^2. Define cT,p:=aT,p−aT,p2=τT,p(1+τT,p)2.c_T,p:=a_T,p-a_T,p^2= _T,p(1+ _T,p)^2. Thus cT,pc_T,p is the deterministic-equivalent total projection-induced fluctuation energy. Knowing the scalar total fluctuation energy cT,pc_T,p does not yet identify its location. To separate the strong and weak contributions, we first write down the precise random quantity that must be evaluated; only after that do we invoke the two-point deterministic-equivalent lemma. Let Dp⪰0D_p 0 be a deterministic prediction-energy weight and define ℱD,p:=[(β^T−μT,p)⊤Dp(β^T−μT,p)],μT,p:=β^T.F_D,p:=E\! [( β_T- _T,p) D_p( β_T- _T,p) ], _T,p:=E β_T. (298) Thus ℱD,pF_D,p is the amount of Stage-I covariance measured in the quadratic norm generated by DpD_p. Expanding the centered quadratic form gives the exact identity ℱD,p=[β^T⊤Dpβ^T]−μT,p⊤DpμT,p.F_D,p=E[ β_T D_p β_T]- _T,p D_p _T,p. (299) We now express the first term through the random projector. Recall that β^T=Σp−1/2PT,pθ⋆. β_T= _p^-1/2P_T,pθ . Define MD,p:=Σp−1/2DpΣp−1/2.M_D,p:= _p^-1/2D_p _p^-1/2. Then, using PT,p⊤=PT,pP_T,p =P_T,p, β^T⊤Dpβ^T= β_T D_p β_T= (θ⋆)⊤PT,pΣp−1/2DpΣp−1/2PT,pθ⋆=(θ⋆)⊤PT,pMD,pPT,pθ⋆. (θ ) P_T,p _p^-1/2D_p _p^-1/2P_T,pθ =(θ ) P_T,pM_D,pP_T,pθ . (300) Consequently, the random matrix that must be understood is [PT,pMD,pPT,p].E[P_T,pM_D,pP_T,p]. This is a two-point object: the same random projector appears on both sides of MD,pM_D,p. A one-point equivalent for PT,pP_T,p is not sufficient, because in general [PT,pMD,pPT,p]≠(PT,p)MD,p(PT,p).E[P_T,pM_D,pP_T,p]≠(EP_T,p)M_D,p(EP_T,p). The difference between these two matrices is exactly the contribution of the projector fluctuations. To place this random object in the resolvent setting, regularize the projector. For λ>0λ>0, define T^T,λ:=Σ^T,p(Σ^T,p+λIp)−1. T_T,λ:= _T,p( _T,p+λ I_p)^-1. By (273), T^T,λ⟶PT,pas λ↓0. T_T,λ P_T,p λ 0. Define the associated regularized coefficient and its mean by β^T,λ:=Σp−1/2T^T,λθ⋆,μT,p,λ:=β^T,λ. β_T,λ:= _p^-1/2 T_T,λθ , _T,p,λ:=E β_T,λ. Then the regularized weighted fluctuation is exactly ℱD,p(λ):= _D,p(λ):= [(β^T,λ−μT,p,λ)⊤Dp(β^T,λ−μT,p,λ)]=(θ⋆)⊤[T^T,λMD,pT^T,λ]θ⋆−μT,p,λ⊤DpμT,p,λ. \! [( β_T,λ- _T,p,λ) D_p( β_T,λ- _T,p,λ) ]=(θ ) E\! [ T_T,λM_D,p T_T,λ ]θ - _T,p,λ D_p _T,p,λ. (301) We now invoke Lemma B.1. Suppose first that NT<pN_T<p, and set qT,p:=pNT.q_T,p:= pN_T. For fixed λ>0λ>0, write GT,λ:=(Σp+κT,λIp)−1,T,λ:=ΣpGT,λ.G_T,λ:=( _p+ _T,λI_p)^-1,T_T,λ:= _pG_T,λ. The equal-ridge specialization of the two-point lemma gives, for every deterministic MpM_p of uniformly bounded operator norm, [T^T,λMD,pT^T,λ] \! [ T_T,λM_D,p T_T,λ ] ≃DET,λMD,pT,λ+κT,λ2GT,λΣpGT,λqT,ptr[ΣpGT,λMD,pGT,λ]1−qT,ptr[Σp2GT,λ2]. DE T_T,λM_D,pT_T,λ+ _T,λ^2G_T,λ _pG_T,λ q_T,ptr [ _pG_T,λM_D,pG_T,λ ]1-q_T,ptr [ _p^2G_T,λ^2 ]. (302) The first term on the right-hand side is the squared-mean contribution. Indeed, the one-point deterministic equivalent gives μT,p,λ≃DEΣp−1/2T,λθ⋆, _T,p,λ DE _p^-1/2T_T,λθ , and hence μT,p,λ⊤DpμT,p,λ _T,p,λ D_p _T,p,λ ≃DE(θ⋆)⊤T,λΣp−1/2DpΣp−1/2T,λθ⋆=(θ⋆)⊤T,λMD,pTT,λθ⋆. DE (θ ) T_T,λ _p^-1/2D_p _p^-1/2T_T,λθ =(θ ) T_T,λM_D,pT_T,λθ . (303) Therefore the first term in (302) cancels the squared-mean term in (301). The second term is precisely the leading covariance contribution. We apply the two-point deterministic equivalent only to the strong- and weak-block prediction-energy weights defined below. Both commute with Σp _p, and their whitened weights MD,p=Σp−1/2DpΣp−1/2M_D,p= _p^-1/2D_p _p^-1/2 are orthogonal coordinate projectors. In particular, ‖MD,p‖op=1\|M_D,p\|_op=1 uniformly in p. We first take the high-dimensional deterministic equivalent at fixed λ>0λ>0, and then send λ↓0λ 0. Write GT,p:=(Σp+τT,pIp)−1,γT,p:=qT,ptr[Σp2GT,p2].G_T,p:=( _p+ _T,pI_p)^-1, _T,p:=q_T,ptr [ _p^2G_T,p^2 ]. The surviving covariance term gives ℱD,p _D,p ≃DEτT,p2(θ⋆)⊤GT,pΣpGT,pθ⋆qT,ptr[ΣpGT,pMD,pGT,p]1−γT,p. DE _T,p^2(θ ) G_T,p _pG_T,pθ q_T,ptr [ _pG_T,pM_D,pG_T,p ]1- _T,p. (304) We now simplify every factor. Since θ⋆θ is supported on the strong block, on which Σp=Ik _p=I_k, (θ⋆)⊤GT,pΣpGT,pθ⋆=1(1+τT,p)2.(θ ) G_T,p _pG_T,pθ = 1(1+ _T,p)^2. Since DpD_p, Σp _p, and GT,pG_T,p commute, tr[ΣpGT,pMD,pGT,p]=tr[DpGT,p2].tr [ _pG_T,pM_D,pG_T,p ]=tr[D_pG_T,p^2]. It remains to simplify the denominator. The ridgeless fixed-point equation is qT,ptr[ΣpGT,p]=1.q_T,ptr[ _pG_T,p]=1. Using ΣpGT,p=Σp2GT,p2+τT,pΣpGT,p2, _pG_T,p= _p^2G_T,p^2+ _T,p _pG_T,p^2, we obtain 1= 1= qT,ptr[ΣpGT,p]=qT,ptr[Σp2GT,p2]+qT,pτT,ptr[ΣpGT,p2]=γT,p+qT,pτT,ptr[ΣpGT,p2]. q_T,ptr[ _pG_T,p]=q_T,ptr[ _p^2G_T,p^2]+q_T,p _T,ptr[ _pG_T,p^2]= _T,p+q_T,p _T,ptr[ _pG_T,p^2]. Therefore 1−γT,p=qT,pτT,ptr[ΣpGT,p2].1- _T,p=q_T,p _T,ptr [ _pG_T,p^2 ]. Substituting these three simplifications into (304) yields ℱD,p _D,p ≃DEτT,p21(1+τT,p)2qT,ptr[DpGT,p2]qT,pτT,ptr[ΣpGT,p2]=τT,p(1+τT,p)2tr[DpGT,p2]tr[ΣpGT,p2]. DE _T,p^2 1(1+ _T,p)^2 q_T,ptr[D_pG_T,p^2]q_T,p _T,ptr[ _pG_T,p^2]= _T,p(1+ _T,p)^2 tr[D_pG_T,p^2]tr[ _pG_T,p^2]. Since cT,p=τT,p(1+τT,p)2,GT,p2=(Σp+τT,pIp)−2,c_T,p= _T,p(1+ _T,p)^2,G_T,p^2=( _p+ _T,pI_p)^-2, the desired weighted covariance formula is ℱD,p≃DEcT,ptr[Dp(Σp+τT,pIp)−2]tr[Σp(Σp+τT,pIp)−2].F_D,p DE c_T,p tr [D_p( _p+ _T,pI_p)^-2 ]tr [ _p( _p+ _T,pI_p)^-2 ]. (305) Equation (305) says that the total fluctuation energy is cT,pc_T,p, while the trace ratio determines the fraction assigned to the directions selected by DpD_p. If NT≥pN_T≥ p, then PT,p=IpP_T,p=I_p almost surely and cT,p=0c_T,p=0, so every weighted covariance vanishes identically. We now choose DpD_p to select the two blocks. For the strong prediction-energy weight, Dstr,p=(Ik000),D_str,p= pmatrixI_k&0\\ 0&0 pmatrix, the corresponding trace is Lstr,p(τ):=tr[Dstr,p(Σp+τIp)−2]=kp1(1+τ)2,L_str,p(τ):=tr [D_str,p( _p+τ I_p)^-2 ]= kp 1(1+τ)^2, (306) For the weak prediction-energy weight, Dwk,p=(000δpIp−k),D_wk,p= pmatrix0&0\\ 0& _pI_p-k pmatrix, we obtain Lwk,p(τ):=tr[Dwk,p(Σp+τIp)−2]=p−kpδp(δp+τ)2.L_wk,p(τ):=tr [D_wk,p( _p+τ I_p)^-2 ]= p-kp _p( _p+τ)^2. (307) Because Dstr,p+Dwk,p=Σp,D_str,p+D_wk,p= _p, the denominator in (305) is exactly Lstr,p(τ)+Lwk,p(τ).L_str,p(τ)+L_wk,p(τ). Define the weak fraction of the Stage-I fluctuation energy by ΘT,p:=Lwk,p(τT,p)Lstr,p(τT,p)+Lwk,p(τT,p). _T,p:= L_wk,p( _T,p)L_str,p( _T,p)+L_wk,p( _T,p). (308) Substituting (306)–(307) gives ΘT,p= _T,p= p−kpδp(δp+τT,p)2kp1(1+τT,p)2+p−kpδp(δp+τT,p)2=[1+k(δp+τT,p)2(p−k)δp(1+τT,p)2]−1. p-kp _p( _p+ _T,p)^2 kp 1(1+ _T,p)^2+ p-kp _p( _p+ _T,p)^2= [1+ k( _p+ _T,p)^2(p-k) _p(1+ _T,p)^2 ]^-1. (309) Hence ΘT,p=[1+k(δp+τT,p)2(p−k)δp(1+τT,p)2]−1. _T,p= [1+ k( _p+ _T,p)^2(p-k) _p(1+ _T,p)^2 ]^-1. (310) The weak mean vanishes exactly. Hence ℱDwk,p= _D_wk,p= [β^T,wk⊤(δpIp−k)β^T,wk]=δp‖β^T,wk‖22=VT,p. \! [ β_T,wk ( _pI_p-k) β_T,wk ]= _pE\| β_T,wk\|_2^2=V_T,p. Therefore (305) and (308) give VT,p≃DEVT,pDE:=cT,pΘT,p.V_T,p DE V_T,p^DE:=c_T,p _T,p. (311) For the strong weight, the same formula gives ℱDstr,p≃DEcT,p(1−ΘT,p).F_D_str,p DE c_T,p(1- _T,p). The strong block contains both this fluctuation energy and the deterministic mean energy aT,p2,a_T,p^2, and therefore ST,p S_T,p ≃DEaT,p2+cT,p(1−ΘT,p)=aT,p2+(aT,p−aT,p2)−cT,pΘT,p=aT,p−cT,pΘT,p. DE a_T,p^2+c_T,p(1- _T,p)=a_T,p^2+(a_T,p-a_T,p^2)-c_T,p _T,p=a_T,p-c_T,p _T,p. (312) Therefore ST,p≃DEST,pDE:=aT,p−cT,pΘT,p.S_T,p DE S_T,p^DE:=a_T,p-c_T,p _T,p. (313) Combining (311) and (313), VT,pDE= V_T,p^DE= cT,pΘT,p, c_T,p _T,p, (314) ST,pDE= S_T,p^DE= aT,p−cT,pΘT,p. a_T,p-c_T,p _T,p. As a consistency check, ST,pDE+VT,pDE=aT,p,S_T,p^DE+V_T,p^DE=a_T,p, so 1−ST,pDE−VT,pDE=1−aT,p=τT,p1+τT,p=ℛ¯T,pDE,1-S_T,p^DE-V_T,p^DE=1-a_T,p= _T,p1+ _T,p= R_T,p^DE, exactly as required by the projection identity. ∎ B.4 Stage I: linear-random-feature derivation conditional on Stage I We now derive Stage I by specializing the linear-random-feature calculation of [Atanasov et al., 2026, Sec. IV]. Conditional on Stage I, set b:=β^Tb:= β_T and regard b∈ℝpb ^p as deterministic. Throughout Stage I, SE_S denotes expectation over the Stage-I randomness (US,X~)(U_S, X), conditional on the Stage-I teacher b. Define FS:=1NSUS⊤∈ℝp×NS,Σ~:=1mX~⊤X~.F_S:= 1 N_SU_S ^p× N_S, := 1m X X. Because the pseudo-labels are y~=X~b, y= Xb, the induced input-space coefficient of the minimum-norm ridgeless student is β^S=FS(X~FS)†X~b. β_S=F_S( XF_S) Xb. (315) Introduce a ridge λ>0λ>0 in the normalized feature coordinates. Define v^S,λ:= v_S,λ:= (FS⊤Σ~FS+λINS)−1FS⊤Σ~b, (F_S F_S+λ I_N_S )^-1F_S b, β^S,λ:= β_S,λ:= FSv^S,λ=HS,λb, F_S v_S,λ=H_S,λb, (316) where HS,λ:=FS(FS⊤Σ~FS+λINS)−1FS⊤Σ~.H_S,λ:=F_S (F_S F_S+λ I_N_S )^-1F_S . Let GS:=FSFS⊤.G_S:=F_SF_S . The push-through identity A(BA+λI)−1=(AB+λI)−1A(BA+λ I)^-1=(AB+λ I)^-1A gives HS,λ= H_S,λ= GSΣ~(GSΣ~+λIp)−1=Ip−λ(GSΣ~+λIp)−1. G_S (G_S +λ I_p)^-1=I_p-λ(G_S +λ I_p)^-1. (317) Therefore the regularized displacement from the fixed teacher is dS,λ:=β^S,λ−b=−QS,λb,QS,λ:=λ(GSΣ~+λIp)−1.d_S,λ:= β_S,λ-b=-Q_S,λb, Q_S,λ:=λ(G_S +λ I_p)^-1. (318) Thus Stage I requires two different random-matrix objects: S[QS,λ] _S[Q_S,λ] for the cross term, the cross term, (319) S[QS,λ⊤ΣpQS,λ] _S [Q_S,λ _pQ_S,λ ] for the mimic term. the mimic term. (320) The first is a one-point resolvent problem; the second is a two-point or source-differentiated resolvent problem. For the ridgeless displacement dS:=β^S−b,d_S:= β_S-b, we have β^S−β⋆=(b−β⋆)+dS. β_S-β =(b-β )+d_S. Consequently, ℛ(h^S)= ( h_S)= (b−β⋆)⊤Σp(b−β⋆)+dS⊤ΣpdS+2(b−β⋆)⊤ΣpdS. (b-β ) _p(b-β )+d_S _pd_S+2(b-β ) _pd_S. (321) Define ℛSmim(b):=S[dS⊤ΣpdS],Scross(b):=2S[(b−β⋆)⊤ΣpdS].R_S^mim(b):=E_S [d_S _pd_S ],C_S^cross(b):=2E_S [(b-β ) _pd_S ]. Then the conditional identity is S[ℛ(h^S)]=ℛ(hb)+ℛSmim(b)+Scross(b).E_S[R( h_S)]=R(h_b)+R_S^mim(b)+C_S^cross(b). (322) Define the feature-dressed population covariance ΣF,S:=Σp1/2GSΣp1/2. _F,S:= _p^1/2G_S _p^1/2. For a positive semidefinite p×p× p matrix A, write dfA,p(1)(κ):= _A,p^(1)(κ):= tr[A(A+κIp)−1], [A(A+κ I_p)^-1 ], dfA,p(2)(κ):= _A,p^(2)(κ):= tr[A2(A+κIp)−2]. [A^2(A+κ I_p)^-2 ]. (323) At fixed FSF_S, the sample average in [Atanasov et al., 2026, Eq. (35)] introduces the first effective ridge κ^1,λ(FS)>0 κ_1,λ(F_S)>0, determined by κ^1,λ(FS)[1−pmdfΣF,S,p(1)(κ^1,λ(FS))]=λ. κ_1,λ(F_S) [1- pmdf_ _F,S,p^(1)\! ( κ_1,λ(F_S) ) ]=λ. (324) The corresponding two-resolvent stability parameter is γ^1,λ(FS):=pmdfΣF,S,p(2)(κ^1,λ(FS)). γ_1,λ(F_S):= pmdf_ _F,S,p^(2)\! ( κ_1,λ(F_S) ). (325) The positive-ridge mimic quadratic is exactly dS,λ⊤ΣpdS,λ= d_S,λ _pd_S,λ= λ2b⊤(Σ~GS+λIp)−1Σp(GSΣ~+λIp)−1b. λ^2b ( G_S+λ I_p)^-1 _p(G_S +λ I_p)^-1b. (326) To evaluate this product, introduce T~b,F(κ1):=b⊤Σp1/2(ΣF,S+κ1Ip)−1Σp1/2b. T_b,F( _1):=b _p^1/2( _F,S+ _1I_p)^-1 _p^1/2b. (327) The source-differentiation calculation of [Atanasov et al., 2026, Eqs. (34)–(36)] gives X~[dS,λ⊤ΣpdS,λ|FS,b]≃DE−κ^1,λ(FS)21−γ^1,λ(FS)∂κ1T~b,F(κ1)|κ1=κ^1,λ(FS).E_ X\! [d_S,λ _pd_S,λ\, |\,F_S,b ] DE - κ_1,λ(F_S)^21- γ_1,λ(F_S) ∂ _1 T_b,F( _1) |_ _1= κ_1,λ(F_S). (328) The derivative generates the two copies of the population resolvent, while (1−γ^1,λ(FS))−1(1- γ_1,λ(F_S))^-1 is the corresponding two-point stability factor. We now average over FSF_S. Let d1,λ:=dfp(κ2,λ),d2,λ:=dfp(2)(κ2,λ),d_1,λ:=df_p( _2,λ),d_2,λ:=df_p^(2)( _2,λ), where dfpdf_p and dfp(2)df_p^(2) are the Stage-I degrees-of-freedom functions. The feature average in [Atanasov et al., 2026, Eqs. (37)–(38)] introduces a second effective ridge κ2,λ=κ¯1,λSGS(−d1,λ), _2,λ= κ_1,λS_G_S(-d_1,λ), (329) where SGSS_G_S is the limiting S-transform of GS=FSFS⊤G_S=F_SF_S . For FS=1NSUS⊤,F_S= 1 N_SU_S , the matrix GSG_S is a white Wishart Gram matrix with aspect ratio p/NSp/N_S. Its S-transform, evaluated at −d1,λ-d_1,λ, is SGS(−d1,λ)=11−(p/NS)d1,λ.S_G_S(-d_1,λ)= 11-(p/N_S)d_1,λ. (330) Consequently, κ2,λ=κ¯1,λ1−(p/NS)d1,λ. _2,λ= κ_1,λ1-(p/N_S)d_1,λ. (331) After averaging over FSF_S, introduce the deterministic first effective ridge κ¯1,λ:=λ1−(p/m)d1,λ. κ_1,λ:= λ1-(p/m)d_1,λ. (332) Eliminating κ¯1,λ κ_1,λ between (331) and (332) gives the central nested-ridge equation λ=κ2,λ[1−pmd1,λ][1−pNSd1,λ],d1,λ=dfp(κ2,λ).λ= _2,λ [1- pmd_1,λ ] [1- pN_Sd_1,λ ], d_1,λ=df_p( _2,λ). (333) The first bracket is the sample renormalization and the second is the random-feature renormalization. To state the feature-averaged mimic formula, define αS,λ:=dlogκ2,λdlogκ¯1,λ. _S,λ:= d _2,λd κ_1,λ. The feature-averaged version of the sample stability parameter is γ¯1,λ:=pmd1,λ[1−d1,λ−d2,λd1,λαS,λ]. γ_1,λ:= pmd_1,λ [1- d_1,λ-d_2,λd_1,λ _S,λ ]. (334) For a deterministic teacher vector b, define Tb(κ):=b⊤Σp(Σp+κIp)−1b,T_b(κ):=b _p( _p+κ I_p)^-1b, so that −Tb′(κ)=b⊤Σp(Σp+κIp)−2b.-T_b (κ)=b _p( _p+κ I_p)^-2b. Define the positive-ridge conditional mimic risk by ℛS,λmim(b):=S[dS,λ⊤ΣpdS,λ].R_S,λ^mim(b):=E_S [d_S,λ _pd_S,λ ]. (335) Specializing [Atanasov et al., 2026, Eq. (40)] to zero label noise gives the deterministic equivalent ℛS,λmim(b)≃DE _S,λ^mim(b) DE −κ2,λ2Tb′(κ2,λ)1−γ¯1,λαS,λ+κ2,λTb(κ2,λ)1−γ¯1,λ(1−αS,λ). - _2,λ^2T_b ( _2,λ)1- γ_1,λ _S,λ+ _2,λT_b( _2,λ)1- γ_1,λ(1- _S,λ). (336) This formula is the explicit result of first averaging over the Stage-I samples and then averaging over the random features. The same two subordinations applied to the one-point quantity in (319) give S[QS,λ]≃DEκ2,λ(Σp+κ2,λIp)−1.E_S[Q_S,λ] DE _2,λ( _p+ _2,λI_p)^-1. (337) Define the deterministic Stage-I ridgeless effective ridge by τS,p:=limλ↓0κ2,λ. _S,p:= _λ 0 _2,λ. Equation (333) shows algebraically that this limit can be reached in three distinct branches. No bottleneck. If p<minm,NS,p< \m,N_S\, both brackets in (333) remain positive at κ2,λ=0 _2,λ=0. Hence τS,p=0. _S,p=0. In fact, the conclusion is exact: FSF_S has full row rank and X~ X has full column rank almost surely, so β^S=b. β_S=b. Feature bottleneck. If NS<minm,p,N_S< \m,p\, the feature bracket vanishes while the sample bracket remains positive. Thus τS,p>0 _S,p>0 is the unique solution of dfp(τS,p)=NSp.df_p( _S,p)= N_Sp. In this branch, κ¯1,λ↓0,κ2,λ⟶τS,p>0, κ_1,λ 0, _2,λ _S,p>0, so αS,λ⟶0,γ¯1,λ⟶NSm. _S,λ 0, γ_1,λ N_Sm. Taking the ridgeless limit in (336) yields ℛSmim(b)≃DEτS,pTb(τS,p)1−NS/m.R_S^mim(b) DE _S,pT_b( _S,p)1-N_S/m. (338) This is exactly the zero-noise specialization of [Atanasov et al., 2026, Eq. (42)]. Sample bottleneck. If m<minp,NS,m< \p,N_S\, the sample bracket vanishes while the feature bracket remains positive. Therefore τS,p>0 _S,p>0 is the unique solution of dfp(τS,p)=mp.df_p( _S,p)= mp. Let γS,psamp:=pmdfp(2)(τS,p). _S,p^samp:= pmdf_p^(2)( _S,p). The overparameterized ridgeless formula [Atanasov et al., 2026, Eq. (44)], with zero label noise, gives ℛSmim(b)≃DE _S^mim(b) DE −τS,p2Tb′(τS,p)1−γS,psamp+τS,pTb(τS,p)dlogSGS(−d)dlogd|d=m/p. - _S,p^2T_b ( _S,p)1- _S,p^samp+ _S,pT_b( _S,p) d S_G_S(-d)d d |_d=m/p. (339) For the white random features, SGS(−d)=11−(p/NS)d,S_G_S(-d)= 11-(p/N_S)d, and therefore dlogSGS(−d)dlogd|d=m/p= . d S_G_S(-d)d d |_d=m/p= (p/NS)d1−(p/NS)d|d=m/p=m/NS1−m/NS. . (p/N_S)d1-(p/N_S)d |_d=m/p= m/N_S1-m/N_S. (340) Substitution gives ℛSmim(b)≃DE _S^mim(b) DE −τS,p2Tb′(τS,p)1−(p/m)dfp(2)(τS,p)+τS,pTb(τS,p)(m/NS)1−m/NS. - _S,p^2T_b ( _S,p)1-(p/m)df_p^(2)( _S,p)+ _S,pT_b( _S,p)(m/N_S)1-m/N_S. (341) The first term is the sample-projection contribution. Its stability denominator comes from the two-point/source differentiation in the data average. The second term is the finite-random-feature contribution and comes from the logarithmic derivative of the white Wishart S-transform. In particular, the denominator 1−pmdfp(2)(τS,p)1- pmdf_p^(2)( _S,p) does not follow merely by differentiating the scalar function dfp(τ)df_p(τ); it is the two-point stability factor in [Atanasov et al., 2026, Eq. (44)]. Define the positive-ridge conditional cross term by S,λcross(b):= _S,λ^cross(b):= 2S[(b−β⋆)⊤ΣpdS,λ]=−2(b−β⋆)⊤ΣpS[QS,λ]b, 2E_S [(b-β ) _pd_S,λ ]=-2(b-β ) _pE_S[Q_S,λ]b, (342) Using (337) and then sending λ↓0λ 0, Scross(b)≃DE−2τS,p(b−β⋆)⊤Σp(Σp+τS,pIp)−1b.C_S^cross(b) DE -2 _S,p(b-β ) _p( _p+ _S,pI_p)^-1b. (343) Set b=β^Tb= β_T. Throughout this subsection, TE_T denotes expectation over the Stage-I randomness UTU_T. Define the exact Stage-I-averaged Stage-I contributions by ℳS,p:= _S,p:= T[ℛSmim(β^T)], _T [R_S^mim( β_T) ], S,pcross:= _S,p^cross:= T[Scross(β^T)]. _T [C_S^cross( β_T) ]. (344) Recall ST,p=‖θT,str‖22,VT,p=‖θT,wk‖22,S_T,p=E\| _T,str\|_2^2,V_T,p=E\| _T,wk\|_2^2, where θT=Σp1/2β^T. _T= _p^1/2 β_T. Because β^T,str=θT,str,δp‖β^T,wk‖22=VT,p, β_T,str= _T,str, _pE\| β_T,wk\|_2^2=V_T,p, the conditional target functions average exactly as T[Tβ^T(τ)]= _T[T_ β_T(τ)]= ST,p1+τ+VT,pδp+τ=:T,p(1)(τ), S_T,p1+τ+ V_T,p _p+τ=:T_T,p^(1)(τ), (345) T[−Tβ^T′(τ)]= _T[-T_ β_T (τ)]= ST,p(1+τ)2+VT,p(δp+τ)2=:T,p(2)(τ). S_T,p(1+τ)^2+ V_T,p( _p+τ)^2=:T_T,p^(2)(τ). (346) The cross target has a stronger exact simplification. Define ℬT,p(τ):=T[(β^T−β⋆)⊤Σp(Σp+τIp)−1β^T].B_T,p(τ):=E_T [( β_T-β ) _p( _p+τ I_p)^-1 β_T ]. The exact Stage-I projection identity gives (θT,str−βstr⋆)⊤θT,str=−‖θT,wk‖22.( _T,str-β _str) _T,str=-\| _T,wk\|_2^2. Hence the strong part of ℬT,p(τ)B_T,p(τ) is −VT,p1+τ,- V_T,p1+τ, whereas the weak part is VT,pδp+τ. V_T,p _p+τ. Therefore ℬT,p(τ)=VT,pΔp(τ),Δp(τ):=1δp+τ−11+τ.B_T,p(τ)=V_T,p _p(τ), _p(τ):= 1 _p+τ- 11+τ. (347) Since 0<δp<10< _p<1, one has Δp(τ)>0 _p(τ)>0. Let T,p(r),DE(τ),r∈1,2,T_T,p^(r),DE(τ),r∈\1,2\, denote (345)–(346) with ST,p,VT,pS_T,p,V_T,p replaced by their Stage-I deterministic equivalents. ℳS,pDE=0,p<minm,NS,τS,pT,p(1),DE(τS,p)1−NS/m,NS<minm,p,τS,p2T,p(2),DE(τS,p)1−(p/m)dfp(2)(τS,p)+τS,pT,p(1),DE(τS,p)(m/NS)1−m/NS,m<minp,NS.M_S,p^DE= cases0,&p< \m,N_S\,\\[8.53581pt] _S,pT_T,p^(1),DE( _S,p)1-N_S/m,&N_S< \m,p\,\\[14.22636pt] _S,p^2T_T,p^(2),DE( _S,p)1-(p/m)df_p^(2)( _S,p)+ _S,pT_T,p^(1),DE( _S,p)(m/N_S)1-m/N_S,&m< \p,N_S\. cases (348) Likewise, (343) and (347) give S,pcross≃DES,pcross,DE:=−2τS,pVT,pDEΔp(τS,p).C_S,p^cross DE C_S,p^cross,DE:=-2 _S,pV_T,p^DE _p( _S,p). (349) Theorem B.4 (Stage-I risk and weak-to-strong gain). Under the proportional-regime assumptions of the Stage-I theorem and the linear-random-feature deterministic-equivalent assumptions of [Atanasov et al., 2026, Sec. IV], ℛ¯S,p≃DEℛ¯S,pDE:=ℛ¯T,pDE+ℳS,pDE+S,pcross,DE. R_S,p DE R_S,p^DE:= R_T,p^DE+M_S,p^DE+C_S,p^cross,DE. (350) Consequently, p≃DEpDE:=ℛ¯T,pDE−ℛ¯S,pDE. W2SG_p DE W2SG_p^DE:= R_T,p^DE- R_S,p^DE. (351) Proof. Averaging (322) over Stage I gives the exact identity ℛ¯S,p=ℛ¯T,p+ℳS,p+S,pcross. R_S,p= R_T,p+M_S,p+C_S,p^cross. Substitute the Stage-I risk deterministic equivalent, (348), and (349) to obtain (350). Finally, from p=ℛ¯T,p−ℛ¯S,p, W2SG_p= R_T,p- R_S,p, replacing the teacher and student risks by their deterministic equivalents yields p≃DEpDE=ℛ¯T,pDE−ℛ¯S,pDE, W2SG_p DE W2SG_p^DE= R_T,p^DE- R_S,p^DE, which is (351). ∎ B.5 Positive-signal-fraction regime: assumptions and phase diagram Assumption B.5 (Proportional asymptotic scaling). Let dp:=p−kd_p:=p-k and assume kp→ϑ,dpδp→η,NTp→ρT,mp→ρm,NSp→ρS, kp→ , d_p _p→η, N_Tp→ _T, mp→ _m, N_Sp→ _S, (352) where ϑ,ρT∈(0,1) , _T∈(0,1), η∈(0,∞)η∈(0,∞), and ρm,ρS∈(0,∞) _m, _S∈(0,∞). We consider the regular fixed-ratio regime in which ρT≠ϑ _T≠ , ρm≠ρS _m≠ _S, ρm,ρS≠ϑ _m, _S≠ , and ρm,ρS≠1 _m, _S≠ 1. We exclude these critical boundaries from the fixed-ratio analysis; the relevant finite-dimensional and critical-scaling phenomena are discussed separately in Section B.6. Theorem B.6 (Positive-signal-fraction phase diagram within the master DE). Under Assumption B.5 and the conditions of the Stage-I and Stage-I deterministic equivalents, let pDE W2SG_p^DE denote the master deterministic equivalent of the teacher-minus-student prediction risk. Define ΛT:=ϑ−ρT,0<ρT<ϑ,(1−ϑ)(ρT−ϑ)1−ρT,ϑ<ρT<1. _T:= cases - _T,&0< _T< ,\\[2.84526pt] (1- )( _T- )1- _T,& < _T<1. cases (353) (a) Student bottleneck below the signal dimension. If minρm,ρS<ϑ \ _m, _S\< , then pDE<0 W2SG_p^DE<0 for all sufficiently large p. (b) Sample bottleneck above the signal dimension. Suppose ϑ<ρm<min1,ρS < _m< \1, _S\. If the left-hand side of (ρm−2ϑ)ρS−ρm(2ρm−3ϑ+ΛT)>0( _m-2 ) _S- _m(2 _m-3 + _T)>0 (354) is nonzero, then pDE>0 W2SG_p^DE>0 for all sufficiently large p if and only if (354) holds. Equivalently, pDE>0 W2SG_p^DE>0 if and only if 2ρm2−(ρS+3ϑ−ΛT)ρm+2ϑρS<0.2 _m^2-( _S+3 - _T) _m+2 _S<0. (355) Define DSB:=(ρS+3ϑ−ΛT)2−16ϑρSD_SB:=( _S+3 - _T)^2-16 _S. If DSB>0D_SB>0, let ρm,±:=ρS+3ϑ−ΛT±DSB4. _m,±:= _S+3 - _T± D_SB4. Then the open positive phase on the sample-bottleneck branch is ρm∈(ρm,−,ρm,+)∩(ϑ,min1,ρS) _m∈( _m,-, _m,+)∩( , \1, _S\). If DSB≤0D_SB≤ 0, or if this intersection is empty, there is no open positive phase on this branch. (c) Feature bottleneck above the signal dimension. Suppose ϑ<ρS<min1,ρm < _S< \1, _m\. If the left-hand side of (ρm−2ρS)(ρS−ϑ)−ρmΛT>0( _m-2 _S)( _S- )- _m _T>0 (356) is nonzero, then pDE>0 W2SG_p^DE>0 for all sufficiently large p if and only if (356) holds. Equivalently, if ρS−ϑ−ΛT>0 _S- - _T>0, the open positive phase is ρm>2ρS(ρS−ϑ)ρS−ϑ−ΛT. _m> 2 _S( _S- ) _S- - _T. (357) If ρS−ϑ−ΛT≤0 _S- - _T≤ 0, then pDE<0 W2SG_p^DE<0 for all sufficiently large p throughout the feature-bottleneck branch. (d) No student bottleneck. If 1<minρm,ρS1< \ _m, _S\, then, under the almost-sure full-rank conditions of the two-stage model, β^S=β^T,p=0 β_S= β_T, W2SG_p=0 almost surely for all sufficiently large p. Proof. Under Assumption B.5, δp∼η/((1−ϑ)p)→0 _p η/((1- )p)→ 0. By the Stage-I master deterministic equivalent, pDE=2τS,pVT,pDEΔp(τS,p)−ℳS,pDE. W2SG_p^DE=2 _S,pV_T,p^DE _p( _S,p)-M_S,p^DE. (358) Stage-I asymptotics. The equation dfp(τT,p)=NT/pdf_p( _T,p)=N_T/p is equivalent to NTpτT,p2+[NTp−kp+δp(NTp+kp−1)]τT,p−(1−NTp)δp=0. N_Tp _T,p^2+ [ N_Tp- kp+ _p ( N_Tp+ kp-1 ) ] _T,p- (1- N_Tp ) _p=0. If 0<ρT<ϑ0< _T< , its positive root satisfies τT,p→(ϑ−ρT)/ρT _T,p→( - _T)/ _T, whereas if ϑ<ρT<1 < _T<1, writing τT,p=δptT,p _T,p= _pt_T,p gives τT,p/δp→(1−ρT)/(ρT−ϑ) _T,p/ _p→(1- _T)/( _T- ). Substitution into the Stage-I deterministic equivalents yields (ST,pDE,VT,pDEδp)⟶(ρTϑ,(1−ϑ)ρTϑ(ϑ−ρT)),0<ρT<ϑ,(1,1−ρTρT−ϑ),ϑ<ρT<1. (S_T,p^DE, V_T,p^DE _p ) cases ( _T , (1- ) _T ( - _T) ),&0< _T< ,\\[8.53581pt] (1, 1- _T _T- ),& < _T<1. cases Hence, with ΛT _T as in (353), δpST,pDEVT,pDE⟶ΛT1−ϑ,VT,pDE⟶0,δp2ST,pDEVT,pDE⟶0. _pS_T,p^DEV_T,p^DE _T1- , V_T,p^DE 0, _p^2S_T,p^DEV_T,p^DE 0. (359) Bottleneck below the signal dimension. Write sT:=limp→∞ST,pDE>0s_T:= _p→∞S_T,p^DE>0. First suppose ρm<minϑ,ρS _m< \ , _S\. Then τS,p→(ϑ−ρm)/ρm _S,p→( - _m)/ _m and dfp(2)(τS,p)→ρm2/ϑdf_p^(2)( _S,p)→ _m^2/ . The beneficial term in (358) tends to zero, while substitution into the Stage-I mimicry term gives ℳS,pDE⟶sTϑ−ρmϑρSρS−ρm,M_S,p^DE s_T - _m _S _S- _m, and therefore pDE⟶−sTϑ−ρmϑρSρS−ρm<0. W2SG_p^DE -s_T - _m _S _S- _m<0. If instead ρS<minϑ,ρm _S< \ , _m\, then τS,p→(ϑ−ρS)/ρS _S,p→( - _S)/ _S, and the same substitution on the feature-bottleneck branch gives pDE⟶−sTϑ−ρSϑρmρm−ρS<0. W2SG_p^DE -s_T - _S _m _m- _S<0. This proves part (a). Bottleneck above the signal dimension. For an active Stage-I bottleneck ratio ρ∈(ϑ,1)ρ∈( ,1), the fixed-point equation gives τS,pδp⟶1−ρ−ϑ,τS,pΔp(τS,p)⟶A(ρ):=1−ρ1−ϑ. _S,p _p 1-ρ- , _S,p _p( _S,p) A(ρ):= 1-ρ1- . Using (359), τS,pT,p(1),DE(τS,p)VT,pDE⟶A(ρ)(1+ΛTρ−ϑ),τS,p2T,p(2),DE(τS,p)VT,pDE⟶A(ρ)2. _S,pT_T,p^(1),DE( _S,p)V_T,p^DE A(ρ) (1+ _Tρ- ), _S,p^2T_T,p^(2),DE( _S,p)V_T,p^DE A(ρ)^2. (360) On the sample-bottleneck branch ϑ<ρm<min1,ρS < _m< \1, _S\, take ρ=ρmρ= _m. Moreover, dfp(2)(τS,p)⟶ϑ+(ρm−ϑ)21−ϑ,1−pmdfp(2)(τS,p)⟶(ρm−ϑ)(1−ρm)ρm(1−ϑ).df_p^(2)( _S,p) + ( _m- )^21- , 1- pmdf_p^(2)( _S,p) ( _m- )(1- _m) _m(1- ). Together with m/(NS−m)→ρm/(ρS−ρm)m/(N_S-m)→ _m/( _S- _m), substitution into (358) gives pDEVT,pDEA(ρm) W2SG_p^DEV_T,p^DEA( _m) ⟶2−ρmρm−ϑ−ρmρS−ρm(1+ΛTρm−ϑ)=(ρm−2ϑ)ρS−ρm(2ρm−3ϑ+ΛT)(ρm−ϑ)(ρS−ρm). 2- _m _m- - _m _S- _m (1+ _T _m- )= ( _m-2 ) _S- _m(2 _m-3 + _T)( _m- )( _S- _m). (361) Since VT,pDE>0V_T,p^DE>0, A(ρm)>0A( _m)>0, and (ρm−ϑ)(ρS−ρm)>0( _m- )( _S- _m)>0, the eventual sign of pDE W2SG_p^DE is the sign of the numerator. This gives (354), equivalently (355); the discriminant and root characterization in part (b) follow from the quadratic formula. On the feature-bottleneck branch ϑ<ρS<min1,ρm < _S< \1, _m\, take ρ=ρSρ= _S in (360). Since m/(m−NS)→ρm/(ρm−ρS)m/(m-N_S)→ _m/( _m- _S), substitution into (358) yields pDEVT,pDEA(ρS) W2SG_p^DEV_T,p^DEA( _S) ⟶2−ρmρm−ρS(1+ΛTρS−ϑ)=(ρm−2ρS)(ρS−ϑ)−ρmΛT(ρS−ϑ)(ρm−ρS). 2- _m _m- _S (1+ _T _S- )= ( _m-2 _S)( _S- )- _m _T( _S- )( _m- _S). (362) Since VT,pDE>0V_T,p^DE>0, A(ρS)>0A( _S)>0, and (ρS−ϑ)(ρm−ρS)>0( _S- )( _m- _S)>0, the eventual sign of pDE W2SG_p^DE is again the sign of the numerator. This gives (356). Rearranging yields (357) when ρS−ϑ−ΛT>0 _S- - _T>0; if this quantity is nonpositive, the limiting numerator is strictly negative throughout the feature-bottleneck branch. This proves parts (b) and (c). No student bottleneck. If 1<minρm,ρS1< \ _m, _S\, then eventually m>pm>p and NS>pN_S>p. Under the stated full-rank conditions, X~ X has full column rank and FSF_S has full row rank. Hence FSF_S is surjective, so there exists v0v_0 with FSv0=β^TF_Sv_0= β_T, and the minimum Stage-I residual is zero. Any least-squares minimizer v^S v_S therefore satisfies X~(FSv^S−β^T)=0 X(F_S v_S- β_T)=0. Since ker(X~)=0 ( X)=\0\, it follows that FSv^S=β^TF_S v_S= β_T. Hence β^S=β^T,p=0 β_S= β_T, W2SG_p=0 almost surely for all sufficiently large p. This proves part (d). ∎ B.6 Critical Boundaries and Finite-Dimensional Effects The regular fixed-ratio phase diagram excludes the interpolation boundary, the signal-rank threshold, and the ambient no-bottleneck boundary. We record here the two boundary phenomena most relevant to the W2SG phase diagram. Exact interpolation singularity. Let rp:=minm,NS,gp:=|m−NS|,r_p:= \m,N_S\,g_p:=|m-N_S|, and assume that the Stage-I coefficient b=β^Tb= β_T is nonzero almost surely and has finite population risk. Theorem B.7 (Exact Stage-I interpolation-boundary classification). The following finite-dimensional statements hold. 1. If rp≥pr_p≥ p, then both Stage-I maps have sufficient rank to identify the teacher coefficient: β^S=β^T β_S= β_T almost surely. Consequently, ℛ¯S,p=ℛ¯T,p R_S,p= R_T,p, and p=0 W2SG_p=0. 2. If rp<pr_p<p and gp≤1g_p≤ 1, then the relevant inverse-Wishart first moment diverges. Under the nondegeneracy conditions of the two-stage model, ℛ¯S,p=+∞ R_S,p=+∞ and p=−∞ W2SG_p=-∞. 3. If rp<pr_p<p and gp≥2g_p≥ 2, the Stage-I expected risk is finite. Its excess contribution contains the inverse-Wishart amplification factor 1gp−1. 1g_p-1. Proof. If m≥pm≥ p and NS≥pN_S≥ p, then X~ X has full column rank and the student feature map has full row rank almost surely. The Stage-I normal equations therefore identify the teacher coefficient uniquely, giving β^S=β^T β_S= β_T. If NS<mN_S<m and NS<pN_S<p, conditioning on the student feature map reduces Stage I to an NSN_S-dimensional Gaussian least-squares problem. Its variance contains the factor NSm−NS−1. N_Sm-N_S-1. If m<NSm<N_S and m<pm<p, conditioning instead on the Stage-I sample gives the dual factor 1NS−m−1. 1N_S-m-1. These expectations are infinite when |m−NS|≤1|m-N_S|≤ 1 and finite when |m−NS|≥2|m-N_S|≥ 2, proving the result. ∎ In particular, m=NS<pm=N_S<p gives ℛ¯S,p=+∞ R_S,p=+∞, whereas the exceptional point m=NS=pm=N_S=p belongs to the exact no-bottleneck regime and satisfies β^S=β^T,p=0. β_S= β_T, W2SG_p=0. Thus, the internal interpolation singularity terminates when both Stage-I resources reach ambient dimension. The signal threshold has a p p critical window. Assume ϑp:=kp→ϑ∈(0,1),(p−k)δp→η∈(0,∞), _p:= kp→ ∈(0,1),(p-k) _p→η∈(0,∞), and εp:=δp. _p:= _p. Define A(z):=z2+4ϑ(1−ϑ)−z2ϑ,L(z):=z2+4ϑ(1−ϑ).A(z):= z^2+4 (1- )-z2 , L(z):= z^2+4 (1- ). Let ρp _p denote a dimension-to-ambient ratio, such as m/pm/p, NS/pN_S/p, or NT/pN_T/p. If ρp−ϑpεp→z _p- _p _p→ z and the corresponding effective resolution satisfies dfp(τp)=ρp,df_p( _p)= _p, then τpεp→A(z). _p _p→ A(z). Because pεp∼ηp1−ϑ,p _p η p1- , the transition around the signal dimension occurs in a dimension window of order p p. We next consider the jointly critical regime in which both the teacher width and the active Stage-I resource approach the signal dimension within their respective p p windows. Suppos NT−kpεp→zT,minm,NS−kpεp→zS, N_T-kp _p→ z_T, \m,N_S\-kp _p→ z_S, and define AS:=A(zS),LS:=L(zS),vTcrit:=1−ϑL(zT).A_S:=A(z_S), L_S:=L(z_S), v_T^crit:= 1- L(z_T). Within the master deterministic equivalent, the two Stage-I bottleneck branches then behave as follows. If m is the active bottleneck and NS/p→ρS>ϑN_S/p→ _S> , then pDE⟶−ϑvTcritLS<0. W2SG_p^DE - v_T^critL_S<0. If NSN_S is the active bottleneck and m/p→ρm>ϑm/p→ _m> , then pDEεp⟶2vTcrit−ρmρm−ϑ(AS+vTcrit). W2SG_p^DE _p 2v_T^crit- _m _m- (A_S+v_T^crit ). Consequently, the leading critical feature-bottleneck gain is positive exactly when (ρm−2ϑ)vTcrit>ρmAS.( _m-2 )v_T^crit> _mA_S. Thus joint signal criticality is asymmetric: the sample-bottleneck branch remains unfavorable, while a nonempty positive W2SG region may survive on the feature-bottleneck branch. These statements concern the master deterministic equivalent; equality in the displayed sign condition requires a higher-order expansion.