Paper deep dive
Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification
Kunyu Wang, Dehan Wang, Wenjun Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/27/2026, 3:00:20 AM
Summary
This paper analyzes norm-constrained linear classification under Euclidean adversarial perturbations within a Gaussian model featuring a low-dimensional informative subspace and an independent noise tail. The authors prove that for bounded ramp loss, if a principal-space witness exists with risk below one-half, near-optimal predictors must have small tail weights. They establish unique population minimizers under isotropic principal covariance and derive localized finite-sample generalization guarantees governed by principal dimension and total tail energy, showing that excess risk and tail usage decrease with sample size.
Entities (11)
Relation Signals (6)
Kunyu Wang → affiliatedwith → Chinese University of Hong Kong
confidence 99% · Kunyu Wang1 ... The Chinese University of Hong Kong
Dehan Wang → affiliatedwith → Chinese University of Hong Kong
confidence 99% · Dehan Wang1 ... The Chinese University of Hong Kong
Wenjun Chen → affiliatedwith → Columbia University
confidence 99% · Wenjun Chen2 ... Columbia University
Bounded Ramp Loss → usedin → Adversarial Classification
confidence 95% · For bounded ramp loss, we prove that a principal-space witness... forces every near-optimal predictor to have small tail weight.
Sample Size → decreases → Excess Risk
confidence 93% · Experiments ... show decreasing excess risk ... with sample size
Principal Subspace → controls → Tail Weight
confidence 90% · a principal-space witness with risk below one half forces every near-optimal predictor to have small tail weight.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We study norm-constrained linear classification under Eu clidean adversarial perturbations in a Gaussian model with a low-dimen sional informative subspace and an independent noise tail. For bounded ramp loss, we prove that a principal-space witness with risk below one half forces every near-optimal predictor to have small tail weight. A path-specific density bound yields constants without requiring positive tail variance. Under isotropic principal covariance, we establish a unique population minimizer and joint local growth. Boundary normalization then removes the common attack penalty from centered margins, giving localized finite-sample guarantees governed by principal dimension and total tail energy. Globalized growth removes the entrance condition at weaker constants; a model-aware comparison retains local guarantees. Experiments with twenty paired repetitions show decreasing excess risk and tail use with sample size, and nearly unchanged behavior when tail dimension grows at fixed total energy. Pure-noise controls and optimizer diagnostics clarify the scope and limitations of these conclusions.
Tags
Links
- Source: https://arxiv.org/abs/2609.23688v1
- Canonical: https://arxiv.org/abs/2609.23688v1
Trouble viewing inline? Open PDF directly →
Full Text
41,247 characters extracted from source content.
Expand or collapse full text
Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification Kunyu Wang1 E-mail: kunyuwang@link.cuhk.edu.hk Dehan Wang1 E-mail: dhwang@link.cuhk.edu.hk Wenjun Chen2 E-mail: wc2741@columbia.edu Abstract We study norm-constrained linear classification under Euclidean adversarial perturbations in a Gaussian model with a low-dimensional informative subspace and an independent noise tail. For bounded ramp loss, we prove that a principal-space witness with risk below one half forces every near-optimal predictor to have small tail weight. A path-specific density bound yields constants without requiring positive tail variance. Under isotropic principal covariance, we establish a unique population minimizer and joint local growth. Boundary normalization then removes the common attack penalty from centered margins, giving localized finite-sample guarantees governed by principal dimension and total tail energy. Globalized growth removes the entrance condition at weaker constants; a model-aware comparison retains local guarantees. Experiments with twenty paired repetitions show decreasing excess risk and tail use with sample size, and nearly unchanged behavior when tail dimension grows at fixed total energy. Pure-noise controls and optimizer diagnostics clarify the scope and limitations of these conclusions. Keywords: Adversarial classification, spectral tails, local complexity, excess risk, Gaussian mixtures †tocauthor: Kunyu Wang, Dehan Wang, Wenjun Chen†institute: The Chinese University of Hong Kong , and Columbia University 1 Introduction Many observed coordinates may contain little label information, yet a learner trained in the full space can use them to fit a finite sample. Under adversarial perturbations, these coefficients also affect the worst-case margin. We study whether full-space training controls the weight assigned to an independent, uninformative tail, and how that structural control affects generalization. We study norm-constrained linear scores, a full-space Euclidean adversary and bounded ramp loss. Signal lies in a principal Gaussian block; the orthogonal tail is independent noise. The principal subspace defines an analytical oracle but is not supplied to full-space fitting. Deleting a noise coefficient need not improve a nonconvex bounded surrogate. A principal-space witness with risk below 1/21/2 places near-optimal predictors in a favorable margin region, where tail deletion reduces the attack penalty without a harmful variance change. We prove joint growth under isotropy and obtain a fast-rate bound with explicit local constants after an entrance check. A second version globalizes the growth inequality and removes the entrance condition at the cost of weaker constants. Related Work. Adversarial Rademacher bounds for linear and nonlinear classes were developed in [2, 12]; robust sample complexity can differ substantially from ordinary learning [10]. Dan et al. [6] establish sharp statistical guarantees for robust Gaussian classification. Javanmard and Soltanolkotabi [8] characterize classification accuracies in a proportional high-dimensional limit. Their estimator, loss or asymptotic regime differs from our finite-sample, fixed-radius ramp analysis. Chen et al. [5] study benign overfitting under adversarial training with an exponential objective. Tanner et al. [11] analyze feature blocks with heterogeneous usefulness and robustness. Their predictive but vulnerable blocks should not be identified with our independent zero-signal tail. Robust low-dimensional representations and latent models are studied in [1, 7]; here the learner performs no preliminary projection. Our statistical argument follows local empirical-process methods [4]. Adversarial fast rates already appear in [9]; our contribution is to verify the centered-loss variance and geometry conditions for this nonsmooth ramp model. Finally, surrogate calibration requires separate assumptions [3]; the bounds below do not identify ramp and robust classification excess risks. Our contribution is explicit deletion growth, including zero tail variance, and its connection to finite-sample bounds through isotropic joint geometry and normalization. We claim neither the first adversarial fast rate nor a global nonconvex solver. 2 Model and Gaussian Risk Let Y be uniform on −1,+1\-1,+1\. For r,k≥1r,k≥ 1, observe X=(Z,τG),Z=Yμ+ξ,ξ∼N(0,Λ),G∼N(0,Ik),X=(Z,τ G), Z=Yμ+ξ, ξ N(0, ), G N(0,I_k), (1) where Y,ξ,GY,ξ,G are independent. Write w=(a,b)∈ℝr+kw=(a,b) ^r+k and B=w:∥w∥2≤B,Γz=Λ+μμ⊤,T=∥X∥22=trΓz+τ2k.W_B=\w: w _2≤ B\, _z= +μ , T=E X _2^2= _z+τ^2k. The decomposition is an analytical coordinate system, justified by orthogonal invariance of the attack and constraint. There is no intercept. Here principal means signal-bearing, not necessarily the leading covariance eigenspace. The theorem covers a broader block model; the nearly low-rank subregime has τ2≪λmin(Γz)τ^2 _ ( _z). Small individual tail eigenvalues and small aggregate energy τ2kτ^2k are distinct properties. For a full-space attack of radius ε , the exact worst-case margin and ramp loss are Mw(x,y)=yw⊤x−ε∥w∥2,ϕγ(t)=min1,max0,1−t/γ.M_w(x,y)=yw x- w _2, _γ(t)= \1, \0,1-t/γ\\. (2) Let S=((Xi,Yi))i=1nS=((X_i,Y_i))_i=1^n be an independent and identically distributed sample from (1). Set ℓw=ϕγ(Mw) _w= _γ(M_w), R(w)=ℓwR(w)=E _w, Rn(w)=n−1∑iℓw(Xi,Yi)R_n(w)=n^-1 _i _w(X_i,Y_i), R⋆=minBR = _W_BR and ℰ(w)=R(w)−R⋆E(w)=R(w)-R . Compactness and continuity give existence of a population minimizer. All risks being compared use the same B,γ,εB,γ, . Counting zero margins as errors gives ℙ(Mw≤0)≤R(w)P(M_w≤ 0)≤ R(w); this pointwise domination does not compare the two excess risks. Assumption 1 The parameters B,γ,εB,γ, are positive, 0≤τ≤τmax<∞0≤τ≤ _ <∞, and 0<λ−Ir⪯Λ⪯λ+Ir0< _-I_r _+I_r. Assumption 2 For some Δ>0 >0, a feasible w0=(a0,0)w_0=(a_0,0) satisfies R(w0)≤1/2−2ΔR(w_0)≤ 1/2-2 . The witness exists exactly when B(∥μ∥2−ε)>γ/2B( μ _2- )>γ/2: this condition makes the aligned boundary vector a witness, and any witness must have mean margin above γ/2γ/2. The maximum feasible mean margin is B(∥μ∥2−ε)+B( μ _2- )_+, which exceeds this positive threshold exactly under the stated condition. The signed margin is Gaussian with νw=a⊤μ−ε∥w∥2,vw=a⊤Λa+τ2∥b∥22. _w=a μ- w _2, v_w=a a+τ^2 b _2^2. (3) Indeed, conditional on either value of Y, (Yξ,YG)(Yξ,YG) has the same centered joint Gaussian law. Thus, for v>0v>0, R(w)=Fγ(νw,vw),Fγ(ν,v)=1γ∫0γΦ(t−νv)t.R(w)=F_γ( _w,v_w), F_γ(ν,v)= 1γ _0^γ ( t-ν v )\,dt. (4) For v=0v=0, use Fγ(ν,0)=ϕγ(ν)F_γ(ν,0)= _γ(ν). Formula (4) follows from ϕγ(z)=γ−1∫0γz≤tdt _γ(z)=γ^-1 _0^γ1\z≤ t\\,dt and Tonelli’s theorem. Lemma 1 For v>0v>0, writing p(ν,v)=Φ((γ−ν)/v)−Φ(−ν/v)p(ν,v)= ((γ-ν)/ v)- (-ν/ v), ∂νFγ _νF_γ =−p(ν,v)/γ<0, =-p(ν,v)/γ<0, (5) ∂vFγ _vF_γ =φ((γ−ν)/v)−φ(ν/v)2γv. = ((γ-ν)/ v)- (ν/ v)2γ v. (6) Moreover, Fγ(ν,v)<1/2F_γ(ν,v)<1/2 iff ν>γ/2ν>γ/2, and ∂vFγ≥0 _vF_γ≥ 0 iff ν≥γ/2ν≥γ/2. The risk threshold also holds at v=0v=0. Proof Differentiate (4) under the integral on compact sets with positive variance. The variance derivative integrates the identity ∂vΦ(t−νv)=12v∂tφ(t−νv). _v ( t-ν v )= 12 v _t ( t-ν v ). Symmetry gives Fγ(γ/2,v)=1/2F_γ(γ/2,v)=1/2 and (5) gives the threshold. Since the normal density decreases with absolute argument, the sign in (6) follows from (γ−ν)2≤ν2(γ-ν)^2≤ν^2 iff ν≥γ/2ν≥γ/2. The deterministic case follows directly from the ramp definition.∎ 3 Tail Localization and a Global Learning Bound Let m=∥μ∥2m= μ _2 and define the following constants under the witness condition: vmax=B2maxλ+,τmax2,κ=γ2πvmaxexp(−(m−ε)22λ−),cb=εκ2γB>0. splitv_ &=B^2 \ _+, _ ^2\,\\ κ&= γ 2π v_ (- (m- )^22 _- ), c_b= κ2γ B>0. split (7) This is a bound on actual deletion paths. It exploits coupling between mean and variance, rather than minimizing normal interval mass over a rectangle containing unattainable parameter pairs. It is independent of k for fixed structural parameters, including a fixed τmax _ . Theorem 3.1 (Tail-Weight Localization) Under Assumptions 1 and 2, every feasible w=(a,b)w=(a,b) with ℰ(w)≤ΔE(w)≤ satisfies R(a,b)−R(a,0)≥cb∥b∥22,∥b∥22≤ℰ(w)/cb.R(a,b)-R(a,0)≥ c_b b _2^2, b _2^2 (w)/c_b. (8) Every population minimizer has b=0b=0. For a fixed principal distribution, R⋆=min∥a∥2≤BR(a,0)R = _ a _2≤ BR(a,0) is independent of k and τ. Proof Near-optimality gives R(w)≤R(w0)+Δ≤1/2−ΔR(w)≤ R(w_0)+ ≤ 1/2- , hence νw>γ/2 _w>γ/2. Let q=∥a∥2q= a _2 and β=∥b∥2β= b _2. Then q>γ/(2m)>0q>γ/(2m)>0. Along wu=(a,ub)w_u=(a, u\,b), 0≤u≤10≤ u≤ 1, νu=a⊤μ−εq2+uβ2,vu=a⊤Λa+uτ2β2. _u=a μ- q^2+uβ^2, v_u=a a+uτ^2β^2. Every path point is feasible and γ/2<νu≤(m−ε)qγ/2< _u≤(m- )q, λ−q2≤vu≤vmax _-q^2≤ v_u≤ v_ . For t∈[0,γ]t∈[0,γ], |t−νu|≤νu|t- _u|≤ _u, so (t−νu)2/vu≤(m−ε)2/λ−(t- _u)^2/v_u≤(m- )^2/ _-. The normal density on [0,γ][0,γ] is therefore at least κ/γκ/γ, giving p(νu,vu)≥κp( _u,v_u)≥κ. By the chain rule and Lemma 1, duFγ(νu,vu) dduF_γ( _u,v_u) =εβ2p(νu,vu)2γq2+uβ2+τ2β2∂vFγ(νu,vu) = β^2p( _u,v_u)2γ q^2+uβ^2+τ^2β^2 _vF_γ( _u,v_u) ≥εκ2γBβ2=cbβ2. ≥ κ2γ Bβ^2=c_bβ^2. Integrating proves the deletion bound. Feasibility of (a,0)(a,0) gives R(a,0)≥R⋆R(a,0)≥ R , proving the second inequality. Applying it to a minimizer yields b=0b=0. No division by zero occurs, even when τ=0τ=0, because q>0q>0 and the principal covariance is nondegenerate.∎ Global Concentration. For 0<η<10<η<1, define Un(η)=2B(T+ε)γn+log(2/η)2n.U_n(η)= 2B( T+ )γ n+ (2/η)2n. (9) With probability at least 1−η1-η, simultaneously over BW_B, |R(w)−Rn(w)|≤Un(η).|R(w)-R_n(w)|≤ U_n(η). (10) To verify this, use the no-outer-absolute-value Rademacher convention. For signs σi _i, the margin process is bounded by Bn‖∑iσiYiXi‖2+εBn|∑iσi|≤B(T+ε)n. BnE \| _i _iY_iX_i \|_2+ BnE | _i _i |≤ B( T+ ) n. Scalar contraction contributes 1/γ1/γ, and symmetrization contributes 2. Each one-sided loss deviation changes by at most 1/n1/n when one observation changes. Applying bounded differences to the two sides with failure probabilities η/2η/2 proves (10). Gaussian inputs need not be bounded because the loss is bounded. If a feasible output satisfies Rn(w^)≤infw∈BRn(w)+ρ,ρ≥0,R_n( w)≤ _w _BR_n(w)+ρ, ρ≥ 0, (11) then on this event ℰ(w^)≤min1,2Un(η)+ρE( w)≤ \1,2U_n(η)+ρ\. When 2Un(η)+ρ≤Δ2U_n(η)+ρ≤ , Theorem 3.1 controls its tail weight. The tolerance in (11) concerns the global empirical minimum, not a stationary point. Growing Dimension. Suppose B,γ,ε,λ−,λ+,τmaxB,γ, , _-, _+, _ are fixed, ∥μn∥2 _n _2 is uniformly bounded, all problems have a common positive witness gap, and ρn→0 _n→ 0. If rn+τn2kn=o(n)r_n+ _n^2k_n=o(n), then ℰn(w^n)→0E_n( w_n)→ 0 and ∥b^n∥2→0 b_n _2→ 0 in probability. Indeed, set ηn=n−2 _n=n^-2 in (9); then Un(ηn)→0U_n( _n)→ 0. Formula (7) provides a common positive cbc_b, so eventual entry and (8) prove both conclusions. For example, fixed rnr_n, kn=n2k_n=n^2 and τn=1/n _n=1/n are allowed. Failure of this sufficient energy condition does not establish impossibility for every learner. 4 Isotropic Geometry and Boundary Normalization Retain Assumption 1 and now take σz>0 _z>0 with Λ=σz2Ir,μ=mu⋆,∥u⋆∥2=1,B(m−ε)>γ/2. = _z^2I_r, μ=mu_ , u_ _2=1, B(m- )>γ/2. (12) Use λ−=λ+=σz2 _-= _+= _z^2 in (7). Let g(q)=R(qu⋆,0)g(q)=R(qu_ ,0) and qmin=γ/(2m)q_ =γ/(2m). Define crad=−g′(B)>0,ca=mincradB,κm2γB.c_rad=-g (B)>0, c_a= \ c_radB, κ m2γ B \. (13) Theorem 4.1 (Joint Local Growth) Suppose Assumption 1 and (12) hold. Then w⋆=(Bu⋆,0)w =(Bu_ ,0) is the unique minimizer. Choose Δ>0 >0 with R(w⋆)≤1/2−2ΔR(w )≤ 1/2-2 . Every feasible w=(a,b)w=(a,b) with ℰ(w)≤ΔE(w)≤ satisfies ℰ(w)≥ca∥a−Bu⋆∥22+cb∥b∥22.E(w)≥ c_a a-Bu_ _2^2+c_b b _2^2. (14) For r=1r=1, the stronger choice ca=crad/Bc_a=c_rad/B is valid. Proof Theorem 3.1 removes the tail of any optimum. For a≠0a≠ 0, write q=∥a∥2q= a _2 and c=a⊤u⋆/qc=a u_ /q. Isotropy gives R(a,0)=1γ∫0γΦ(t/q−mc+εσz)t,R(a,0)= 1γ _0^γ ( t/q-mc+ _z )\,dt, which is strictly decreasing in c for fixed q. Along the aligned direction, −g′(q)=1γσz∫0γ/qzφ(z−(m−ε)σz)z>0.-g (q)= 1γ _z _0^γ/qz ( z-(m- ) _z )\,dz>0. (15) Thus q=Bq=B is optimal, proving uniqueness. The integral in (15) decreases with q, so −g′(q)≥−g′(B)=crad-g (q)≥-g (B)=c_rad on [qmin,B][q_ ,B]. For a near-optimal w, q>qminq>q_ by the previous proof. Decompose exactly ℰ(a,b)= (a,b)= [R(a,b)−R(a,0)] [R(a,b)-R(a,0)] +[R(a,0)−R(qu⋆,0)]+[g(q)−g(B)]. +[R(a,0)-R(qu_ ,0)]+[g(q)-g(B)]. The first term is at least cb∥b∥22c_b b _2^2. For the second, the variance remains σz2q2 _z^2q^2 during alignment, while the mean increases from ma⊤u⋆−εqma u_ - q to (m−ε)q(m- )q. Each intermediate mean lies above γ/2γ/2 and below (m−ε)q(m- )q; the path density bound in the previous proof therefore applies. Integrating (5) and then (15) gives ℰ(a,b)≥cb∥b∥22+crad(B−q)+κmγ(q−a⊤u⋆).E(a,b)≥ c_b b _2^2+c_rad(B-q)+ κ mγ(q-a u_ ). (16) Finally, ∥a−Bu⋆∥22=(B−q)2+2B(q−a⊤u⋆) a-Bu_ _2^2=(B-q)^2+2B(q-a u_ ) and (B−q)2≤B(B−q)(B-q)^2≤ B(B-q) prove (14). If r=1r=1, the local region forces positive alignment, so the angular term is unnecessary.∎ Isotropy keeps variance fixed during alignment; general anisotropic principal growth is not verified here. Shared Classification Optimum. Under (12), w⋆w also minimizes robust classification error. Indeed, for q=∥a∥2q= a _2, s=∥w∥2s= w _2 and positive mean margin, a⊤μ−εsσz2q2+τ2∥b∥22≤mq−εsσzq≤m−εσz. a μ- s _z^2q^2+τ^2 b _2^2≤ mq- s _zq≤ m- _z. Nonpositive mean margin gives error at least 1/21/2, including degenerate margins. Hence the minimum is Φ(−(m−ε)/σz) (-(m- )/ _z), attained by every (cu⋆,0)(cu_ ,0) with 0<c≤B0<c≤ B. The two risks share an optimizer but not their optimal values or scaling behavior; pointwise loss domination alone supplies no excess-risk conversion. Corollary 1 (One-Dimensional Sphere Tail Bound) Under Theorem 4.1, let r=1r=1, ∥w∥2=B w _2=B and ℰ(w)≤ΔE(w)≤ . Then ℰ(w)≥cb,tail∥b∥22,cb,tail=cb+crad2B.E(w)≥ c_b,tail b _2^2, c_b,tail=c_b+ c_rad2B. (17) Proof Orient the scalar coordinate along u⋆u_ . The local region gives a>0a>0, and (16) gives ℰ(a,b)≥cb∥b∥22+crad(B−a)E(a,b)≥ c_b b _2^2+c_rad(B-a). On the sphere, B−a=∥b∥22/(B+a)≥∥b∥22/(2B)B-a= b _2^2/(B+a)≥ b _2^2/(2B).∎ This is a separate tail inequality. Substituting cb,tailc_b,tail into the joint-growth ellipsoid while keeping cac_a unchanged would reuse the radial contribution without justification. The local complexity below retains (14) and its original constants. Proposition 1 (Boundary Normalization) Fix a unit vector v0v_0 independently of the sample. Define SB(w)=Bw/∥w∥2S_B(w)=Bw/ w _2 for w≠0w≠ 0 and SB(0)=Bv0S_B(0)=Bv_0. For every feasible w and every (x,y)(x,y), ℓSB(w)(x,y)≤ℓw(x,y) _S_B(w)(x,y)≤ _w(x,y). The ball and sphere B∂=w:∥w∥2=BW_B^∂=\w: w _2=B\ have the same empirical and population infima. Normalizing an output satisfying (11) preserves its tolerance ρ. Proof For nonzero w, let c=B/∥w∥2≥1c=B/ w _2≥ 1. Homogeneity gives Mcw=cMwM_cw=cM_w. For all real t, ϕγ(ct)≤ϕγ(t) _γ(ct)≤ _γ(t): nonpositive margins stay in the loss-one region, and positive margins cannot incur greater loss. At w=0w=0, the original loss equals one. Equality of infima and preservation of the tolerance follow from sphere inclusion and this pointwise inequality.∎ On the sphere the penalty is the common constant −εB- B. Consequently (10) holds there with the improved quantity Un∂(η)=2BTγn+log(2/η)2n.U_n^∂(η)= 2B Tγ n+ (2/η)2n. (18) To see this, enclose the sphere class by ϕγ(yw⊤x−εB) _γ(yw x- B) indexed over the ball, retaining the constant shift even for interior parameters. Contraction now applies to the linear score class. This refinement does not remove attack dependence from the risk or geometric constants. 5 Localized Risk and a Post-Fit Comparison This section assumes the isotropic conditions and witness gap of Theorem 4.1 and gives the sharper sphere result. Proposition 1 makes it applicable to normalized approximate empirical minimizers over the ball. Let hw=ℓw−ℓw⋆h_w= _w- _w and ℋ(s)=hw:w∈B∂,ℰ(w)≤sH(s)=\h_w:w _B^∂,E(w)≤ s\ for 0≤s≤Δ0≤ s≤ . Use ℜn(ℱ)=S,σsupf∈ℱn−1∑iσif(Xi,Yi) R_n(F)=E_S,σ _f n^-1 _i _if(X_i,Y_i). Define K2=1γ2(trΓzca+τ2kcb),V=1γ2max‖Γz‖opca,τ2cb.K^2= 1γ^2 ( _zc_a+ τ^2kc_b ), V= 1γ^2 \ \| _z\|_opc_a, τ^2c_b \. (19) Lemma 2 (Local Statistical Inputs) For 0≤s≤Δ0≤ s≤ , ℜn(ℋ(s))≤Ks/n R_n(H(s))≤ K s/n. Also hw2≤Vℰ(w)Eh_w^2≤ VE(w) whenever ℰ(w)≤ΔE(w)≤ . Proof Put δw=w−w⋆δ w=w-w and D=diag(caIr,cbIk)D= (c_aI_r,c_bI_k). By (14), δw⊤Dδw≤ℰ(w)δ w Dδ w (w). Both parameter norms are B, so the centered margin is exactly Yδw⊤XYδ w X. Enlarge the local difference set to the ellipsoid δ⊤Dδ≤sδ Dδ≤ s. Its signed linear supremum is (s/n)∥D−1/2∑iσiYiXi∥2( s/n)\|D^-1/2 _i _iY_iX_i\|_2. Its expected square inside the norm is ntr(D−1Γ)n (D^-1 ), with Γ=diag(Γz,τ2Ik) = ( _z,τ^2I_k). Jensen’s inequality and sample-wise Lipschitz contraction around the fixed margin Mw⋆M_w give the complexity bound. For the second moment, hw2≤γ−2δw⊤Γδw≤Vδw⊤Dδw≤Vℰ(w).Eh_w^2≤γ^-2δ w δ w≤ Vδ w Dδ w≤ VE(w). The variance bound follows from this second-moment bound, not from boundedness alone.∎ For n≥2n≥ 2 and 0<η<10<η<1, let Jn=⌈log2n⌉,tn=log2(Jn+1)η,An=1+64K2+(8V+16/3)tnn. splitJ_n&= _2n , t_n= 2(J_n+1)η,\\ A_n&= 1+64K^2+(8V+16/3)t_nn. split (20) Lemma 3 (Uniform Localized Deviation) With probability at least 1−η/21-η/2, simultaneously for every sphere predictor with e=ℰ(w)≤Δe=E(w)≤ , (P−Pn)hw≤e/2+An/2.(P-P_n)h_w≤ e/2+A_n/2. (21) Here P and PnP_n denote population and empirical averaging. Proof For s>0s>0, consider ℋ(mins,Δ)H( \s, \). Its functions take values in [−1,1][-1,1], have variance at most VsVs and expected complexity at most Ks/nK s/n. The first inequality of (4, Theorem 2.1), with its tuning parameter equal to one, gives suph(P−Pn)h≤4Ks/n+2Vst/n+8t3n _h(P-P_n)h≤ 4K s/n+ 2Vst/n+ 8t3n (22) with failure probability at most e−te^-t. All suprema are measurable: the relevant sets are compact and the losses continuously parameterized. Apply (22) at sj=2j/ns_j=2^j/n, j=0,…,Jnj=0,…,J_n, with t=tnt=t_n. The union has failure probability at most η/2η/2. For any local predictor, select j=0j=0 if e≤1/ne≤ 1/n, otherwise the smallest j with sj≥es_j≥ e. Such a j exists since sJn≥1≥es_J_n≥ 1≥ e. With x=maxe,1/nx= \e,1/n\, sj≤2xs_j≤ 2x, hence (P−Pn)hw≤(42K+2Vtn)x/n+8tn/(3n).(P-P_n)h_w≤(4 2K+2 Vt_n) x/n+8t_n/(3n). Write C=42K+2VtnC=4 2K+2 Vt_n. Young’s inequality gives Cx/n≤x/2+C2/(2n)C x/n≤ x/2+C^2/(2n) and C2≤64K2+8VtnC^2≤ 64K^2+8Vt_n. Since x≤e+1/nx≤ e+1/n, this is (21).∎ Theorem 5.1 (Post-Fit Localized Risk) With probability at least 1−η1-η, simultaneously for every w∈B∂w _B^∂, define gS⋆(w)=[Rn(w)−Rn(w⋆)]+.g_S (w)=[R_n(w)-R_n(w )]_+. (23) If 2Un∂(η/2)+gS⋆(w)≤Δ2U_n^∂(η/2)+g_S (w)≤ , then ℰ(w)≤min1, 2Un∂(η/2)+gS⋆(w),An+2gS⋆(w).E(w)≤ \1,\,2U_n^∂(η/2)+g_S (w),\,A_n+2g_S (w)\. (24) The joint parameter bound (14) holds on the same event. In particular, for w~=SB(w^) w=S_B( w) satisfying (11), one may replace gS⋆g_S by ρ in the condition and bound. Proof Intersect the sphere global deviation event with failure probability η/2η/2 and the event of Lemma 3. On their intersection, for every sphere predictor, ℰ(w)≤Rn(w)−Rn(w⋆)+2Un∂(η/2)≤gS⋆(w)+2Un∂(η/2).E(w)≤ R_n(w)-R_n(w )+2U_n^∂(η/2)≤ g_S (w)+2U_n^∂(η/2). The entrance condition places that same predictor in the local region. Using Pnhw≤gS⋆(w)P_nh_w≤ g_S (w) in (21) gives ℰ(w)≤gS⋆(w)+ℰ(w)/2+An/2E(w)≤ g_S (w)+E(w)/2+A_n/2. Rearranging proves (24). The event is uniform before w or its gap is chosen, so a data-dependent gap is allowed and the two events need not be independent. For an approximate empirical minimizer, normalization preserves the tolerance and gS⋆(w~)≤ρg_S ( w)≤ρ.∎ Corollary 2 (Entry-Free Risk Bound) Under the same assumptions, set c~a=minca,Δ/(4B2) c_a= \c_a, /(4B^2)\ and c~b=mincb,Δ/(4B2) c_b= \c_b, /(4B^2)\. Define K~,V~ K, V by (19) with these constants, and t~n=logJn+1η,A~n=1+64K~2+(8V~+16/3)t~n. t_n= J_n+1η, A_n= 1+64 K^2+(8 V+16/3) t_nn. With probability at least 1−η1-η, simultaneously on the sphere, ℰ(w)≤min1,A~n+2gS⋆(w).E(w)≤ \1, A_n+2g_S (w)\. (25) No entrance check is required; normalized approximate empirical minimizers may replace gS⋆g_S by ρ. Proof The reduced-constant growth inequality holds on the whole ball. For ℰ(w)≤ΔE(w)≤ , apply Theorem 4.1; otherwise its proposed right-hand side is at most Δ∥w−w⋆∥22/(4B2)≤Δ<ℰ(w) w-w _2^2/(4B^2)≤ <E(w). Lemma 2 therefore holds for 0≤s≤10≤ s≤ 1 with the reduced constants. Repeat the proof of Lemma 3, replacing the cutoff Δ by 1 and tnt_n by t~n t_n. Its union failure probability is now η, with no global event needed. Thus (P−Pn)hw≤ℰ(w)/2+A~n/2(P-P_n)h_w (w)/2+ A_n/2; use Pnhw≤gS⋆(w)P_nh_w≤ g_S (w) and rearrange.∎ With fixed m,σz,B,γ,ε,τmax,Δm, _z,B,γ, , _ , , both sets of geometric constants are independent of r,kr,k. Since trΓz=rσz2+m2 _z=r _z^2+m^2 and ‖Γz‖op=σz2+m2\| _z\|_op= _z^2+m^2, both statistical terms have order (r+τ2k+log((1+logn)/η))/n(r+τ^2k+ ((1+ n)/η))/n, plus the comparison gap or optimization tolerance. Entry enables the original local constants, not this rate order itself. The iterated logarithm comes from the finite grid. The comparison is model-aware: w⋆,T,Δw ,T, and the constants are supplied by the synthetic model, not generally known from real data. Neither fitting nor initialization uses w⋆w . A zero gap means only Rn(w)≤Rn(w⋆)R_n(w)≤ R_n(w ), not global empirical optimality. 6 Controlled Experiments The baseline is r=1r=1, k=32k=32, m=2m=2, σz=B=γ=1 _z=B=γ=1, τ=0.15τ=0.15, n=512n=512 and evaluation attack ε=0.25 =0.25. Each of 22 settings uses 20 independent samples, paired across full-space robust ramp, principal-space oracle robust ramp, full-space ordinary ramp, normalized robust logistic, and the normalized signed mean B∑iYiXi/∥∑iYiXi∥2B _iY_iX_i/ _iY_iX_i _2. The last is both a standalone baseline and a shared initialization. Except for ordinary ramp and the training-budget sweep, training uses attack 0.250.25. Optimization and Evaluation. We use SciPy 1.17.1 SLSQP with supplied gradients, ftol=10−10=10^-10, at most 1000 iterations, and 12 starts: the signed-mean direction at radius B plus 11 random directions with radii uniform on [B/4,B][B/4,B]. A signed mean of norm at most 10−1410^-14 is replaced by a random start. The piecewise ramp gradient averages over all n samples, with contributions only for 0<M<γ0<M<γ (zero at either kink). The norm-penalty gradient is set to zero at ∥w∥2≤10−14 w _2≤ 10^-14 and to εw/∥w∥2 w/ w _2 otherwise. This is an implementation convention, not differentiability at the kinks. Every terminal candidate is projected to feasibility; the smallest training objective is selected regardless of optimizer status, then normalized. All 1,760 selected main-grid fits succeeded. Across robust-ramp starts, the median objective range was 0.86610.8661; 9.63% of random starts ended at loss one, versus no signed-mean starts. Success does not certify (11). For zero-based setting j and repetition q, the sample seed is 20260919+100000j+q20260919+100000j+q, and optimizer seeds add 5000050000 plus the method index in the order above (excluding signed mean). Settings are ordered by the sample-size, fixed-τ dimension, fixed-energy dimension, signal, training-attack and principal-dimension sweeps; values increase within each sweep. Thus methods share samples, whereas separate sweeps use independent draws even at identical parameters. Population ramp risks and robust margin errors are evaluated from (3)–(4); the latter is Φ(−νw/vw) (- _w/ v_w) for vw>0v_w>0. The population reference is analytic in the executed grid. Witness settings use Theorem 4.1; the pure-noise and threshold controls are treated separately. We save every raw and normalized coefficient vector, all optimizer starts, and the source hash, seed and dependency versions. Independent Gaussian quadrature and reconstruction of all 3,520 coefficient vectors agreed with recorded metrics to within 2.23×10−162.23× 10^-16. Normalization never increases ramp loss but increased the logistic training objective by more than 10−910^-9 in 29 of 440 fits (maximum 0.195080.19508); this method is labelled normalized robust logistic. Positive scaling preserves robust margin signs, not surrogate risks. Only exact zero invokes v0=e1v_0=e_1; otherwise max-absolute-coordinate scaling avoids underflow, without a positive norm threshold. 6.1 Sample Size and Tail Energy Excess risk and tail weight decrease with n (Fig. 1, Table 1), approximately fourfold per quadrupling over most of this grid. Four sample sizes do not identify an exponent or validate bound constants. Table 1: Normalized robust ramp, 20 repetitions. Gn=min1,2Un∂(η/2)+gS⋆G_n= \1,2U_n^∂(η/2)+g_S \ at η=0.05η=0.05; all recorded gaps are zero. AnA_n is shown only after entry; a dash means inapplicable n Mean ℰE Mean ∥b^∥22 b _2^2 GnG_n AnA_n 128 0.003268 0.038356 1.000000 — 512 0.000742 0.009011 0.553621 — 2048 0.000218 0.002669 0.276811 — 8192 0.000051 0.000630 0.138405 1.152511 Figure 1: Sample-size sweep, including the signed-mean baseline: means and standard errors over 20 paired samples. Vertical axes are logarithmic above 10−610^-6 and linear near zero, retaining exact oracle zeros without an added offset At fixed τ=0.15τ=0.15, increasing k from 8 to 512 increases total tail energy from 0.180.18 to 11.5211.52, excess risk from 0.0002330.000233 to 0.0115740.011574, and squared tail norm from 0.0028510.002851 to 0.1243520.124352. In contrast, setting τ=0.72/kτ= 0.72/k keeps energy fixed. At k=8,32,128,512k=8,32,128,512, the excess risks are 0.0009050.000905, 0.0008510.000851, 0.0008320.000832 and 0.0008410.000841; squared tail norms remain between 0.010070.01007 and 0.010500.01050 (Fig. 2). These controlled comparisons support aggregate tail energy as a more informative quantity than coordinate count alone. The robust classification error in the sample-size sweep changes only from 0.0405570.040557 to 0.0400670.040067, illustrating why risk and coefficient diagnostics are reported separately. Figure 2: Dimension and energy controls, with the same five methods and vertical scales as Fig. 1. Top: fixed τ=0.15τ=0.15. Bottom: fixed kτ2=0.72kτ^2=0.72. Bars are standard errors over 20 paired samples 6.2 Entry, Constants and Boundary Controls All 80 robust-ramp fits in the sample-size sweep have gS⋆=0g_S =0. Only n=8192n=8192 satisfies the sufficient sphere entrance condition: 2Un∂(η/2)=0.138405<Δ=0.1925032U_n^∂(η/2)=0.138405< =0.192503. The confidence statement is for one sample and fixed configuration, not joint 95% coverage of the grid. The path constant is κ=0.0862773κ=0.0862773, giving cb=0.0107847c_b=0.0107847. At n=8192n=8192, η=0.05η=0.05, the local term is 1.336701.33670 with the general isotropic cac_a, or 1.152511.15251 with the sharper scalar ca=0.111634c_a=0.111634. The entry-free term is A~n=1.90874 A_n=1.90874: removing entry does not make the bound numerically useful here. The operative Theorem 5.1 bound remains 0.1384050.138405. Dividing by cbc_b or the stronger cb,tail=0.0666018c_b,tail=0.0666018 gives 12.833512.8335 or 2.07812.0781, both worse than the feasibility cap B2=1B^2=1. Structural tightness is different. Using the actual unrounded population excesses at n=8192n=8192 gives ℰ¯/cb,tail=0.000772 E/c_b,tail=0.000772, versus observed ∥b∥22¯=0.000630 b _2^2=0.000630. Figure 3 checks the inequality for all 80 fitted sample-size predictors. In this configuration, the statistical risk bound is much looser than the structural conversion. This uses known population risk after fitting, not a training-time certificate. Figure 3: Direct structural checks. Left: 80 fitted predictors and the scalar sphere lower bound (logarithmic axes); all satisfy ℰ≤ΔE≤ . Right: zero-tail-variance deletion paths at fixed a=0.7a=0.7, 0≤b≤0.70≤ b≤ 0.7, with m=2m=2, σz=B=γ=1 _z=B=γ=1. Positive attack alone creates a tail cost; without attack it vanishes For a paired τ=0τ=0 control, we reuse the 20 n=512n=512 labels, principal features and optimizer seeds, changing only tail variance. All 80 selected control fits succeeded; normalized full-space robust ramp has exactly zero tail weight and excess risk in all 20 samples. Independent quadrature and reconstruction of its 160 raw/normalized method vectors agree within 1.39×10−171.39× 10^-17. The fixed-a paths in Fig. 3 isolate the attack penalty from the sphere’s principal-weight reallocation. With common evaluation budget 0.250.25, training budgets 0,0.25,0.50,0.25,0.5 yield mean squared tail norms 0.024422,0.009204,0.0045440.024422,0.009204,0.004544. Increasing r from 1 to 4 to 16 raises excess risk from 0.0007800.000780 to 0.0017710.001771 to 0.0076040.007604, while tail weight stays near 0.0090.009. These sweeps separate the training penalty and principal estimation cost from tail energy; they do not establish a universal advantage of larger attacks. The signal sweep uses m∈0,0.75,1.25,2m∈\0,0.75,1.25,2\. The first two settings lack a positive witness: at m=0.75m=0.75, B(m−ε)=γ/2B(m- )=γ/2 and the aligned ramp risk is 1/21/2, although its robust classification error is Φ(−0.5)=0.30854<1/2 (-0.5)=0.30854<1/2. The witness condition concerns the favorable ramp region, not the possibility of nontrivial robust classification. At m=0m=0, ordinary binary classification error is 1/21/2, while the fitted robust margin error is approximately 0.5990.599. Initialization Versus the Fitted Predictor. The normalized empirical signed mean is already one of the initializations. Comparing it with the selected normalized robust-ramp output on the same samples shows lower training risk but higher population risk and tail weight in all 80 sample-size trials. At n=512n=512, mean training risk changes from 0.1186930.118693 to 0.1180300.118030, while excess risk rises from 0.00002840.0000284 to 0.00074200.0007420 and squared tail norm from 0.00034840.0003484 to 0.00901110.0090111. This directly exposes the empirical fitting incentive to use tail noise. In this grid, the simple initialization generalizes better; the theory characterizes full-space empirical fitting rather than asserting its superiority. Why the Witness Matters. For pure noise X∼N(0,Σ)X N(0, ), (4) along w=cuw=cu has integrand Φ((t/c+ε)/u⊤Σu) ((t/c+ )/ u u), decreasing in c>0c>0 and positive directional variance. Thus, when λmax(Σ)>0 _ ( )>0, minimizers have norm B in a top-eigenvalue direction. Unconditional tail deletion is false: for μ=0μ=0, Λ=1 =1, a=0.1a=0.1, τ=2τ=2, ε=0.2 =0.2, B=γ=1B=γ=1, changing b from 0 to 0.5 lowers risk from 0.9693110.969311 to 0.7174340.717434. The witness is absent. 7 Discussion A tail-free population optimum does not prevent finite-sample tail use. The witness-based deletion inequality and isotropic geometry yield statistical bounds, while the experiments expose empirical fitting incentives relative to a simple initialization. The finite-sample constants remain conservative; general anisotropic growth and global optimization guarantees are unresolved. No same-task minimax lower bound is established: a principal-subspace oracle can remove the tail entirely. A vanishing attack, informative tail or non-Euclidean geometry requires separate arguments. Our bounds concern ramp excess risk. Although the isotropic model shares an optimal direction with robust classification, a quantitative excess-risk conversion has not been established here. References (1) Awasthi, P., Chatziafratis, V., Chen, X., Vijayaraghavan, A.: Adversarially robust low dimensional representations. In: Proceedings of the 34th Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 134, p. 237–325. PMLR (2021). URL https://proceedings.mlr.press/v134/awasthi21a.html (2) Awasthi, P., Frank, N., Mohri, M.: Adversarial learning guarantees for linear hypotheses and neural networks. In: Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 119, p. 431–441. PMLR (2020). URL https://proceedings.mlr.press/v119/awasthi20a.html (3) Bao, H., Scott, C., Sugiyama, M.: Calibrated surrogate losses for adversarially robust classification. In: Proceedings of the Thirty Third Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 125, p. 408–451. PMLR (2020). URL https://proceedings.mlr.press/v125/bao20a.html (4) Bartlett, P.L., Bousquet, O., Mendelson, S.: Local Rademacher complexities. The Annals of Statistics 33(4), 1497–1537 (2005). DOI 10.1214/009053605000000282. URL https://arxiv.org/abs/math/0508275 (5) Chen, J., Cao, Y., Gu, Q.: Benign overfitting in adversarially robust linear classification. In: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, Proceedings of Machine Learning Research, vol. 216, p. 313–323. PMLR (2023). URL https://proceedings.mlr.press/v216/chen23b.html (6) Dan, C., Wei, Y., Ravikumar, P.: Sharp statistical guarantees for adversarially robust Gaussian classification. In: Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 119, p. 2345–2355. PMLR (2020). URL https://arxiv.org/abs/2006.16384 (7) Javanmard, A., Mehrabi, M.: Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoff. Operations Research 72(3), 1016–1030 (2024). DOI 10.1287/opre.2022.0162. URL https://arxiv.org/abs/2110.11950. First published online in 2023 (8) Javanmard, A., Soltanolkotabi, M.: Precise statistical analysis of classification accuracies for adversarial training. The Annals of Statistics 50(4), 2127–2156 (2022). DOI 10.1214/22-AOS2180. URL https://arxiv.org/abs/2010.11213 (9) Mustafa, W., Lei, Y., Kloft, M.: On the generalization analysis of adversarial learning. In: Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 162, p. 16,174–16,196. PMLR (2022). URL https://proceedings.mlr.press/v162/mustafa22a.html (10) Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., Madry, A.: Adversarially robust generalization requires more data. In: Advances in Neural Information Processing Systems, vol. 31 (2018). URL https://arxiv.org/abs/1804.11285 (11) Tanner, K., Vilucchio, M., Loureiro, B., Krzakala, F.: A high dimensional statistical model for adversarial training: Geometry and trade-offs. In: Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 258, p. 2530–2538. PMLR (2025). URL https://proceedings.mlr.press/v258/tanner25a.html (12) Yin, D., Ramchandran, K., Bartlett, P.L.: Rademacher complexity for adversarially robust generalization. In: Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, p. 7085–7094. PMLR (2019). URL https://arxiv.org/abs/1810.11914