Paper deep dive
Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks
Andrew Cheng, Ali Eslamian, Jie Cheng, Mehdi Zargham, Qiang Cheng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/14/2026, 4:37:34 AM
Summary
This paper introduces Random Mapping Networks (RaMaN), a framework for training neural networks via low-dimensional reparameterization. It derives an orientation-resolved quadratic master formula to predict the latent dimension required for successful training based on curvature spectrum and displacement profiles. RaMaN utilizes structured Hadamard or seed-regenerated Gaussian maps to reduce memory overhead from O(P) to O(d) and eliminates the need for exhaustive dimension sweeps.
Entities (5)
Relation Signals (4)
Random Mapping Networks → uses → Hadamard Mapping
confidence 95% · RaMaN supports orientation-resolved selection... instantiated using scalable map families, including structured Hadamard mappings
orientation-resolved quadratic master formula → predicts → random-slice residual
confidence 93% · master formula that predicts the random-slice residual from both the curvature spectrum and the reference-to-solution displacement profile
Random Mapping Networks → reduces → optimizer-state memory
confidence 90% · reduce optimizer-state memory from O(P) to O(d)
orientation-resolved quadratic master formula → recovers → Gaussian-width quadratic bound
confidence 85% · in a conservative radius-only specialization, recovers the earlier Gaussian-width quadratic bound
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Neural networks can often be trained or fine-tuned through random low-dimensional reparameterization, where a small latent vector is mapped into a full parameter update by a frozen random map. This raises a practical question: how large must the latent search space be to reach a low-loss region? We first express the known accessibility transition in an equivalent conic form, centered for compact convex targets at the statistical dimension of the polar cone. Our main theoretical contribution is an orientation-resolved quadratic master formula that predicts the random-slice residual from both the curvature spectrum and the reference-to-solution displacement profile. It yields a self-consistent isotropic-orientation predictor and, in a conservative radius-only specialization, recovers the earlier Gaussian-width quadratic bound. Building on this analysis, we introduce Random Mapping Networks (RaMaN), which instantiate the predicted latent dimension using structured Hadamard or seed-regenerated Gaussian maps. These constructions avoid the O(dP) storage of dense random maps and reduce optimizer-state memory from O(P) to O(d). We also develop matrix-free curvature approximations and sweep-free dimension selection. Across controlled quadratic and neural-curvature experiments, the orientation-resolved predictor closely tracks measured transition locations and outperforms orientation-agnostic approximations when displacement direction matters. End-to-end experiments further show sharp, protocol-dependent training transitions across image and language models.
Tags
Links
- Source: https://arxiv.org/abs/2608.12597v1
- Canonical: https://arxiv.org/abs/2608.12597v1
Trouble viewing inline? Open PDF directly →
Full Text
175,930 characters extracted from source content.
Expand or collapse full text
Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks Andrew Cheng† Affiliation: Department of Computer Science, Tsinghua University, Beijing, China Affiliation: Department of Computer Science, The University of Manchester, Manchester, UK Ali Eslamian† Affiliation: Department of Computer Science, University of Kentucky, Lexington, KY, USA Jie Cheng Affiliation: Department of Computer Science, Miami University, Oxford, OH, USA Mehdi Zargham Affiliation: Department of Computer Science, University of Dayton, Dayton, OH, USA Qiang Cheng* Affiliation: Department of Computer Science, University of Kentucky, Lexington, KY, USA Affiliation: Institute for Biomedical Informatics, University of Kentucky, Lexington, KY, USA Abstract Neural networks can often be trained or fine-tuned using random low-dimensional reparameterization: instead of optimizing all model parameters directly, one optimizes a small latent vector that is mapped into a full parameter update through a frozen random map. This procedure restricts training to a randomly chosen low-dimensional search space and raises a practical question: how large must this search space be to access parameters that achieve low loss? Random low-dimensional training is known to exhibit a sharp accessibility transition governed by the geometry of the low-loss region. We first express this phenomenon in an equivalent conic form, in which the transition for compact convex targets is centered at the statistical dimension of the polar cone. Our main theoretical contribution is an operational quadratic analysis that goes beyond this geometric characterization. We derive an orientation-resolved master formula that predicts the random-slice residual from both the curvature spectrum and the reference-to-solution displacement profile, thereby making explicit how displacement orientation relative to curvature affects the required latent dimension. The formula yields a new self-consistent isotropic-orientation predictor and, in its conservative radius-only specialization, recovers the earlier Gaussian-width quadratic bound. Building on this analysis, we introduce Random Mapping Networks (RaMaN), a scalable framework that instantiates the predicted latent dimension d using structured Hadamard mappings or seed-regenerated Gaussian maps for a network with P parameters. These constructions eliminate the O(dP)O(dP) frozen-map storage required by dense random generators and reduce optimizer-state memory from O(P)O(P) to O(d)O(d). We further develop matrix-free curvature approximations and sweep-free dimension-selection procedures, and design a benchmark that measures accessibility transitions across tasks, architectures, and map families while accounting for trainable parameters, frozen-map storage, optimizer-state memory, checkpoint size, and runtime. Across controlled quadratic and neural-curvature experiments, the orientation-resolved predictor closely tracks measured transition locations and substantially improves over orientation-agnostic approximations in settings where displacement direction matters; end-to-end experiments further demonstrate sharp, protocol-dependent training transitions across image and language models. † authors contributed equally to this work. *Corresponding author: Qiang Cheng, qiang.cheng@uky.edu 1 Introduction Modern deep neural networks are typically trained by directly optimizing all parameters of a high-dimensional weight vector θ∈ℝPθ ^P, where P may range from millions to billions or more. Although this direct parameterization is highly expressive, it is also expensive: optimizers must maintain gradients and auxiliary states for every parameter, checkpoints scale with P, and unrestricted optimization may overfit when the effective degrees of freedom required by the task are much smaller than the ambient parameter dimension. This gap between the ambient dimension of the model and the effective dimension of the optimization problem motivates low-dimensional reparameterization. In low-dimensional reparameterization, the full parameter vector θ∈ℝPθ ^P is not optimized directly. Instead, it is generated from a much smaller trainable latent vector z∈ℝdz ^d, with d≪Pd P, through a fixed or partially fixed mapping θ=gω(z),θ=g_ω(z), where ω denotes random, structured, or otherwise non-trainable parameters of the mapping. Equivalently, one may write θ=θref+Δθ(z),θ= _ref+ θ(z), where θref _ref is a reference point, such as a random initialization, a pretrained model, or a pilot solution, and Δθ(z) θ(z) is a constrained parameter displacement generated from the low-dimensional latent variable z. Training then optimizes only z, while the target network performs its standard forward computation using the generated weights θ(z)θ(z). This idea has a simple geometric interpretation. Direct training searches over the full parameter space ℝPR^P, whereas low-dimensional reparameterization restricts training to the image Im(gω)=gω(z):z∈ℝd,Im(g_ω)=\g_ω(z):z ^d\, where ω denotes the frozen random draw defining the map. When gωg_ω is smooth, this image has at most d local degrees of freedom, because the Jacobian of gωg_ω at any z∈ℝdz ^d has rank at most d. Such a restriction can be beneficial if the low-loss region is wide in most parameter directions and constrained only along a relatively small number of important directions. In this case, the low-dimensional image need not cover the full ambient space ℝPR^P; it only needs enough degrees of freedom to satisfy these effective constraints. We refer to the corresponding number of constrained directions as the effective codimension of the low-loss region. This viewpoint provides a geometric explanation for empirical observations that neural networks can often be trained or fine-tuned successfully in surprisingly small random subspaces [1, 14]. However, the central challenge is not merely whether a low-dimensional random search space can reach a low-loss region. Prior work has already shown that random-subspace trainability exhibits a sharp geometric transition as the latent dimension increases [14, 11]. The practically important question is whether this transition can be predicted from quantities that are accessible without performing an exhaustive sweep over latent dimensions. In particular, given a frozen map gω:ℝd→ℝPg_ω:R^d ^P drawn from a random or structured random family, we would like to determine a dimension d for which Im(gω)∩Sε≠∅Im(g_ω)∩ S_ ≠ with high probability over ω, where Sε=θ:ℒ(θ)≤ℒ⋆+εS_ = \θ:L(θ) + \ is an ε -low-loss sublevel set. A related algorithmic question is whether gradient-based optimization over the resulting latent variables can exploit such an accessible low-loss region under realistic training budgets and map constructions. Recent approaches provide substantial evidence that optimizing a small number of trainable variables can sometimes approach or match full-parameter training, including intrinsic-dimension training [14], low-dimensional fine-tuning [1], random-basis parameterizations [16, 9], and Mapping Networks [20]. Mapping Networks, for example, optimize a low-dimensional latent vector z∈ℝdz ^d that modulates a randomly initialized mapping network with frozen base weights to generate the P≫dP d parameters of a target network. These methods establish the practical feasibility of low-dimensional parameterizations, but they do not by themselves provide a prospective rule for selecting the latent dimension from the local geometry of the target loss. Larsen et al. [11] provided an important geometric explanation for this phenomenon. They characterized random-subspace trainability through the angular Gaussian width of a low-loss sublevel set viewed from the initialization, relating the observed transition to the geometry of the target region. This characterization, however, is difficult to use prospectively: the Gaussian width of an unknown neural-network low-loss region is generally not directly available before performing the dimension sweeps one would like to avoid. Moreover, in the quadratic setting, the resulting radius-only approximation depends on the Hessian spectrum and the scalar distance from the reference point to a minimum, but does not explicitly resolve how that displacement is oriented relative to the curvature eigendirections. This leaves an operational gap between a geometric characterization of the transition and a computable predictor that can be used for latent-dimension selection. We address this gap in two stages. First, for compact convex target sets, we express the known random-slice accessibility transition in an equivalent conic form, in which its location is characterized by the statistical dimension of the polar cone generated by the shifted low-loss region. This formulation provides a convenient effective-codimension interpretation and a bridge to the local quadratic analysis. Our main theoretical contribution is then an orientation-resolved quadratic predictor. For a positive-semidefinite local curvature model, we derive a master formula that predicts the random-slice residual using both the curvature spectrum and the complete reference-to-solution displacement profile. Consequently, two problems with the same Hessian spectrum and the same displacement norm can have different predicted latent dimensions when their displacements are oriented differently relative to the curvature eigenspaces. A new self-consistent isotropic-orientation predictor follows by imposing an equal-energy displacement model, while an orientation-uniform radius-only specialization recovers the earlier quadratic bound of Larsen et al. [11]. The latter provides a conservative choice when directional displacement information is unavailable. This local quadratic setting is both analytically tractable and empirically relevant: studies of deep-network Hessians commonly report a small number of large outlier eigenvalues together with a bulk of near-zero eigenvalues, corresponding to a few stiff directions and many nearly flat directions [19, 5, 17]. Because exact spectral information is unavailable at modern network scale, we further develop matrix-free approximations based on leading curvature directions and stochastic estimates of the unresolved spectral mass. These approximations support practical latent-dimension selection without requiring a full sweep over candidate dimensions. Building on this analysis, we introduce Random Mapping Networks (RaMaN), a scalable framework that couples latent-dimension selection with fixed random low-dimensional reparameterizations. RaMaN supports orientation-resolved selection when an estimated displacement profile is available, radius-based selection when only a displacement scale is known, and nested adaptive expansion when neither estimate is sufficiently reliable. The selected dimension is instantiated using scalable map families, including structured Hadamard mappings (SHM), motivated by fast structured Hadamard constructions such as SRHT and Fastfood [2, 21, 12], and seed-regenerated Gaussian maps. A global seed-regenerated Gaussian construction realizes the uniformly random subspace assumed by Theorem 1, while the default layer-wise Gaussian construction corresponds to the independent layer-wise setting of Corollary 3 under its product-structure assumptions. SHM provides a structured, memory-efficient alternative whose empirical behavior is evaluated separately. These constructions avoid storing a dense P×dP× d random mapping object and reduce optimizer-state memory from O(P)O(P) to O(d)O(d). Our goal is not to claim that all large neural networks can be trained through only a few thousand latent variables. Rather, we ask when a substantial reduction in trainable dimension is geometrically plausible, whether the required dimension can be predicted from measurable local curvature and displacement information, and how map structure, optimizer, and training protocol affect the resulting transition. This perspective turns low-dimensional reparameterization from a purely empirical dimension-sweep procedure into a framework with explicit, falsifiable predictions. Our main contributions are summarized as follows: • Theory. Building on the Gaussian-width characterization of random-subspace training, we recast the accessibility transition in an equivalent conic/statistical-dimension form and derive a new orientation-resolved quadratic master formula. Unlike prior radius-only quadratic bounds, the master-formula predictor depends jointly on the curvature spectrum and the reference-to-solution displacement profile. Two useful specializations of this predictor follow: a new self-consistent isotropic-orientation predictor, and an orientation-uniform predictor that recovers the earlier radius-only bound as a special case. • Method. We develop Random Mapping Networks (RaMaN), which couple curvature- and displacement-informed latent-dimension selection with scalable frozen random reparameterizations. RaMaN supports orientation-resolved and radius-based sweep-free selection together with nested adaptive dimension expansion that preserves the previously active random search space. To instantiate the selected dimension without storing a dense P×dP× d map, we develop a seeded Structured Hadamard Mapping (SHM) based on nested selection of Hadamard basis directions, cropHDIΩcrop\,HDI_ , and a seed-regenerated Gaussian construction whose active columns form prefixes of a fixed virtual Gaussian matrix. These constructions preserve the fixed-slice interpretation underlying the theory while providing scalable matrix-free training. • Measurement. We design a benchmark for measuring the phase transition of random low-dimensional training across tasks, architectures, and mapping families. The benchmark evaluates the orientation-resolved master-formula predictor and its isotropic-orientation and orientation-uniform specializations, compares Gaussian and structured random-map families and practical dimension-selection strategies, and reports not only trainable parameters but also frozen-map storage, optimizer-state memory, checkpoint size, and wall-clock cost. 2 Related Work Low-dimensional reparameterization and intrinsic dimension. A growing line of work suggests that the number of degrees of freedom required to train or adapt a neural network can be much smaller than its ambient parameter dimension. Li et al. [14] introduced the notion of the intrinsic dimension of an objective landscape by training networks within randomly oriented linear subspaces and measuring the subspace dimension at which successful solutions first appear. Their large-scale experiments also used structured Fastfood projections [12] to avoid explicitly storing dense random projection matrices, motivating the use of scalable structured maps for random-subspace training. Li et al. provided important empirical evidence for a sharp dimension-dependent transition but did not derive a geometric theory predicting its location. Larsen et al. [11] subsequently provided such a geometric explanation using Gordon’s escape-through-a-mesh theorem. They defined the local angular dimension of a low-loss sublevel set as the squared Gaussian width of its spherical projection about the initialization and related the threshold training dimension to the complement of this quantity. For quadratic wells, they further derived a radius-only bound based on the Hessian spectrum and the distance from the initialization to a minimum. Our work builds directly on this geometric perspective. For compact convex targets, we express the same accessibility geometry through the statistical dimension of the cone generated by the shifted low-loss set and its polar. Our main distinction is operational: we derive an orientation-resolved quadratic master formula that retains the complete reference-to-solution displacement profile relative to the curvature eigendirections. Its self-consistent isotropic-orientation specialization provides a new predictor, whereas its conservative orientation-uniform specialization recovers the radius-only quadratic bound of Larsen et al. Aghajanyan et al. [1] extended the intrinsic-dimension viewpoint to language-model fine-tuning, showing that pretrained models can often be adapted through very low-dimensional random reparameterizations. VeRA [10] further reduces the trainable parameter count in parameter-efficient fine-tuning by sharing frozen random low-rank matrices across layers and learning only small scaling vectors. These studies demonstrate the practical utility of low-dimensional and frozen-random parameterizations, while the required latent dimension is typically selected or assessed empirically rather than predicted from the local curvature and displacement geometry of the target loss. Random-basis and generator-based parameterizations. Gressmann et al. [7] studied neural-network optimization in low-dimensional random bases and introduced independent random projections for different parts of a network together with on-demand pseudorandom generation from shared seeds to reduce projection storage. Their optimization strategy differs fundamentally from the frozen random-slice setting considered here: they found that keeping a projection fixed could impair optimization and instead redraw the random subspace during training. RaMaN retains a fixed random map because its geometric question concerns whether a particular frozen random search space can access a prescribed low-loss region; seed regeneration is used as a scalable implementation mechanism. Other methods parameterize neural weights through frozen random bases. PRANC [16] represents a model as a linear combination of randomly initialized frozen basis networks, training only the combination coefficients. NOLA [9] applies a related idea to parameter-efficient fine-tuning by representing low-rank LoRA updates [8] as linear combinations of random basis matrices. These approaches reduce trainable parameters and checkpoint size but do not provide a prospective rule for selecting the dimension of a fixed random search space from the local geometry of the target objective. Mapping Networks [20] take a generator-based approach in which a compact trainable latent representation is mapped into the much higher-dimensional parameter space of a target network. This provides further evidence that target-network parameters can be controlled through far fewer trainable degrees of freedom. However, such a generator-based construction does not by itself characterize the accessibility threshold of a particular frozen random parameterization. Moreover, a dense linear random map from ℝdR^d to ℝPR^P, when used to realize such a low-dimensional parameterization directly, requires O(dP)O(dP) frozen storage. RaMaN therefore combines dimension prediction with matrix-free structured or seed-regenerated map realizations, rather than treating random-map generation itself as the principal contribution. Conic geometry and random intersection theory. Our theoretical development builds on tools from high-dimensional convex geometry and Gaussian process theory. Gordon’s escape-through-a-mesh theorem provides foundational probability bounds for when random subspaces avoid a subset of the sphere [6]. Conic integral geometry sharpens this picture for convex cones: random cone intersections undergo phase transitions governed by statistical dimension [3]. Related Gaussian-width and high-dimensional probability tools are summarized by Vershynin [22] and have played central roles in convex optimization, compressed sensing, and high-dimensional statistics. Larsen et al. [11] brought this random-intersection viewpoint directly to neural-network training through the Gaussian width of a loss sublevel set viewed from the initialization. Our conic analysis should therefore be viewed as a refinement and reformulation of this geometric foundation rather than as the first random-intersection characterization of low-dimensional training. For compact convex targets, the spherical set used in the Gaussian-width analysis is the spherical section of the cone generated by the shifted target, allowing the transition to be expressed equivalently through the statistical dimension of that cone or its polar. This effective-codimension formulation provides the geometric foundation for our main extension: an orientation-resolved quadratic residual analysis that turns the otherwise difficult-to-evaluate geometric threshold into an operational predictor based on curvature and displacement information. Together with its isotropic-orientation and orientation-uniform specializations, this predictor motivates the sweep-free dimension-selection procedures used by RaMaN. 3 Methodology 3.1 Problem Setup and Random Reparameterization Let θ∈ℝPθ ^P denote the parameter vector of the target network, where P is the total number of parameters, and let ℒ(θ)L(θ) denote the training objective. We consider low-dimensional reparameterizations of the form θ=gω(z),z∈ℝd,d≪P,θ=g_ω(z), z ^d, d P, where z is trainable and ω denotes random, structured, or otherwise non-trainable parameters of the mapping. Equivalently, for fine-tuning or residual parameterization, we write θ(z)=θref+Δθ(z),θ(z)= _ref+ θ(z), where θref _ref is a reference parameter vector, such as a pretrained model, a random initialization, or a pilot solution. In the geometric analysis below, we write θ0:=θref _0:= _ref for this reference point. For a target tolerance ε>0 >0, define the low-loss sublevel set Sε:=θ:ℒ(θ)≤ℒ⋆+ε,S_ := \θ:L(θ) + \, where ℒ⋆L denotes the best achievable loss within the parameter region under consideration. For a set ⊆ℝPA ^P, define its conic hull by cone():=ta:t≥0,a∈¯,cone(A):= \ta:t≥ 0,\;a \, where the closure ensures that the resulting cone is closed. Given the reference point θ0 _0, we write Cε:=cone(Sε−θ0).C_ :=cone(S_ - _0). Geometrically, CεC_ contains the rays from θ0 _0 toward the low-loss region. Its polar cone is Cε∘:=v∈ℝP:⟨v,u⟩≤0for all u∈Cε.C_ := \v ^P: v,u ≤ 0\ for all u∈ C_ \. For a general neural-network objective, SεS_ and CεC_ need not be convex. The conic phase-transition results below therefore apply to localized compact convex targets. In particular, in the local quadratic setting we use S=Sε∩B¯(θ0,Rloc)S=S_ ∩ B( _0,R_loc) and write C=cone(S−θ0).C=cone(S- _0). For the compact convex targets used in our theorems, this generated cone is already closed, so no additional closure is required (Lemma 1). For a closed convex cone C, its statistical dimension is δ(C):=g∼(0,IP)[‖ΠC(g)‖22],δ(C):=E_g (0,I_P) [\| _C(g)\|_2^2 ], where ΠC(g) _C(g) denotes the Euclidean projection of g onto C. Statistical dimension plays the role of an effective dimension for a convex cone. Its polar counterpart δ(C∘)δ(C ) will characterize the effective number of constrained directions that a random low-dimensional reparameterization must capture. Remark 1 (Relation to the local angular dimension of Larsen et al.). For the localized compact convex target S, define its spherical view from θ0 _0 by Σθ0(S):=θ−θ0‖θ−θ0‖2:θ∈S. _ _0(S):= \ θ- _0\|θ- _0\|_2:θ∈ S \. Since C=cone(S−θ0)C=cone(S- _0), Σθ0(S)=C∩P−1. _ _0(S)=C ^P-1. Larsen et al. [11] characterize random-subspace trainability through the squared Gaussian width of this spherical set, which they term the local angular dimension. For a closed convex cone, squared Gaussian width and statistical dimension satisfy w2(C∩P−1)≤δ(C)≤w2(C∩P−1)+1w^2\! (C ^P-1 )≤δ(C)≤ w^2\! (C ^P-1 )+1 [3]. Thus, the statistical-dimension formulation used here provides an equivalent conic refinement of the earlier Gaussian-width perspective: the corresponding Gaussian-width and statistical-dimension transition centers differ by at most one dimension. We use this conic formulation as a convenient geometric foundation for the orientation-resolved quadratic analysis developed below. With this notation, we study two related questions. First, can the latent dimension required for a frozen random search space to access a prescribed low-loss region be predicted from measurable properties of the local loss geometry, rather than identified through an exhaustive sweep over d? Second, once such a low-loss region is accessible within the random parameterization, how effectively can gradient-based optimization over z reach it under practical training conditions? 3.2 Critical Dimension and Hessian Predictors We first state the geometric accessibility transition underlying random low-dimensional reparameterization and then address the more operational question of predicting its location from local curvature and displacement information. Choosing d too small makes access to a prescribed low-loss region unlikely, whereas choosing it unnecessarily large weakens the computational advantages of low-dimensional training. Figure 1: Overview of Random Mapping Networks (RaMaN). (a) Random low-dimensional training is formulated as an intersection problem: a random affine slice from θ0 _0 succeeds when it intersects the localized target S, with C=cone(S−θ0)C=cone(S- _0) and the conic transition centered at dconic=δ(C∘)d_conic=δ(C ). (b) In the local quadratic regime, the transition is estimated from the curvature spectrum and reference-to-solution displacement using the orientation-resolved master-formula predictor, with isotropic-orientation and radius-only effective-rank specializations when less directional information is available. (c) RaMaN instantiates the predicted latent dimension with frozen matrix-free maps, including Structured Hadamard Mapping (SHM) and seed-regenerated Gaussian maps. Only the latent variables zll=1L\z_l\_l=1^L are trained; each generates a layer-wise update Δθ(l)=βlℛl(ϕl(zl)) θ^(l)= _lR_l( _l(z_l)), reducing dense-map storage and optimizer-state memory. The conic formulation characterizes the accessibility transition through the effective codimension of the localized low-loss region, quantified by the statistical dimension of the corresponding polar cone. As discussed in Remark 1, this provides an equivalent statistical-dimension refinement of the Gaussian-width perspective of Larsen et al. [11]. Because this geometric quantity is generally intractable for neural-network losses, our main theoretical development focuses on the local quadratic regime. In this setting, we derive an orientation-resolved master formula that depends jointly on the curvature spectrum and the reference-to-solution displacement profile. A new self-consistent isotropic-orientation predictor follows as one specialization, while the conservative orientation-uniform specialization recovers the earlier radius-only quadratic bound. These results motivate the practical three-mode dimension-selection procedure in Algorithm 1. The general conic intersection result for compact convex target sets is stated and proved in Appendix A. The theory is stated for a positive-semidefinite Hessian at a local minimum. Away from such a point, we use either the positive spectral part of the Hessian or a positive-semidefinite curvature surrogate, such as the Gauss–Newton or Fisher matrix. These substitutions are practical approximations rather than consequences of Theorem 1. Theorem 1 (Conic transition for a localized quadratic target). Let θ†θ be a local minimizer with local quadratic model ℒ~(θ)=ℒ(θ†)+12(θ−θ†)⊤H(θ−θ†),H⪰0, L(θ)=L(θ )+ 12(θ-θ ) H(θ-θ ), H 0, and, for ε>0 >0 and localization radius Rloc>0R_loc>0, define S=θ:12(θ−θ†)⊤H(θ−θ†)≤ε∩B¯(θ0,Rloc),S= \θ: 12(θ-θ ) H(θ-θ )≤ \∩ B( _0,R_loc), assumed nonempty, with reference point θ0∉S _0∉ S. Let C=t(θ−θ0):t≥0,θ∈S.C= \t(θ- _0):t≥ 0,\;θ∈ S \. Let E be a uniformly random d-dimensional subspace of ℝPR^P, and let A∈ℝP×dA ^P× d be any matrix with range(A)=Erange(A)=E, so that the random slice is θ0+E=θ(z):=θ0+Az:z∈ℝd, _0+E= \θ(z):= _0+Az:z ^d \, where z∈ℝdz ^d is the trainable latent vector. Equivalently, A may be taken to have i.i.d. (0,1)N(0,1) entries, in which case A has rank d almost surely and E=range(A)E=range(A) is uniformly distributed; the intersection event depends only on E, not on the particular basis A. The slice reaches S if and only if E∩C≠0E∩ C≠\0\. Moreover, for every η∈(0,1)η∈(0,1), the intersection probability is at least 1−η1-η when d≥P−δ(C)+aηP,d≥ P-δ(C)+a_η P, and at most η when d≤P−δ(C)−aηP,d≤ P-δ(C)-a_η P, where aη=8log(4/η).a_η= 8 (4/η). Thus, the conic transition is centered at dconic:=P−δ(C)=δ(C∘),d_conic:=P-δ(C)=δ(C ), within an O(P)O( P) transition window (Theorem 2, proved in Appendix A). Implications of the theorem. Figure 1 summarizes the progression from the geometric phase-transition result to its quadratic predictors and scalable RaMaN implementation. The identity dconic=δ(C∘)d_conic=δ(C ) gives the key geometric interpretation: the transition is governed by the effective codimension of the low-loss set as seen from θ0 _0, namely the effective number of constrained directions that the random slice must satisfy, rather than directly by the ambient parameter dimension. Small-dimensional accessibility is therefore predicted only in the fat-cone regime, δ(C)≈Pδ(C)≈ P, where S is wide in most parameter directions and narrow along relatively few directions. Whether this fat-cone regime holds depends on the localized geometry of the low-loss set relative to the reference point θ0 _0. In the quadratic model, such a regime is favored by a Hessian spectrum with a small number of large eigenvalues and a bulk of near-zero eigenvalues, although the actual transition also depends on the displacement orientation, loss tolerance, and localization radius. Under this spectral structure, the quadratic surrogates introduced below can predict a transition at a latent dimension substantially smaller than P. The orientation-resolved master formula incorporates both the Hessian spectrum and the displacement profile, while the orientation-uniform predictor reff(ε,R)r_eff( ,R) provides a conservative spectrum-based mean-level estimate when only a displacement radius is available. This qualitative spectral profile is widely reported in empirical studies of deep-network Hessians [19, 5, 17], and our benchmark tests whether the associated quadratic surrogates predict the measured random-slice transitions. Thus, the conic theorem identifies the geometric quantity controlling random-slice accessibility, but it does not by itself provide a practical way to evaluate that quantity for a neural-network loss. The quadratic predictors developed next address this operational problem. Remark 2 (Two extreme geometries). Two examples delimit the regimes. (i) Round target. If S is a ball of radius ρ centered at distance D>ρD>ρ from θ0 _0, then C is a circular cone with half-angle α=arcsin(ρ/D).α= (ρ/D). Its statistical dimension is approximately [3] δ(C)≈Psin2α=Pρ2D2,δ(C)≈ P ^2α=P ρ^2D^2, so dconic≈P(1−ρ2D2).d_conic≈ P (1- ρ^2D^2 ). Thus, random slices are essentially ineffective for small, well-rounded targets: for a fixed point at distance D from θ0 _0, a Haar-random d-dimensional slice through θ0 _0 has expected squared distance D2(1−d/P)D^2(1-d/P) from that point, giving the typical scale DP−dP.D P-dP. (i) Rank-k quadratic residual. If H=λ(Ik⊕0P−k),H=λ(I_k 0_P-k), then the unlocalized quadratic residual depends only on the k stiff directions. Under the isotropic displacement model, the least-squares/master-formula transition can be computed exactly and is controlled by k rather than by P; see Appendix B.5. This example illustrates how a small effective curvature dimension can produce a low-dimensional random-slice residual transition. For the localized conic theorem, however, δ(C∘)δ(C ) also depends on the reference point, tolerance, and localization radius. Empirical Hessian studies of deep networks often exhibit the qualitative local-curvature structure underlying case (i), namely a small number of stiff directions and many nearly flat directions. Our experiments test whether this local spectral structure yields the small quadratic random-slice transition predicted by the Hessian-based surrogates. The conic transition center dconic=δ(C∘)d_conic=δ(C ) is geometric but generally not directly computable from standard neural-network diagnostics. We therefore seek an operational quadratic surrogate that retains information discarded by radius-only approximations, in particular the orientation of the reference-to-solution displacement relative to the curvature eigenspaces. Definition 1 (Orientation-resolved master-formula predictor). Let H=∑i=1Pλivivi⊤,λi≥0,H= _i=1^P _iv_iv_i , _i≥ 0, be the eigendecomposition of the positive-semidefinite curvature operator, and let Δ0:=θ†−θ0,Δi:=⟨Δ0,vi⟩. _0:=θ - _0, _i:= _0,v_i . Thus, Δi _i is the component of the reference-to-solution displacement along the iith curvature eigenvector. Let r:=rank(H),εq:=2ε,r:=rank(H), _q:=2 , where ε is the loss-excess tolerance in the quadratic model and εq _q is the corresponding quadratic-form tolerance. For each candidate dimension d∈(0,r)d∈(0,r), let κ(d)>0κ(d)>0 denote the unique solution of d=tr[H(H+κ(d)I)−1]=∑i=1Pλiλi+κ(d).d=tr\! [H (H+κ(d)I )^-1 ]= _i=1^P _i _i+κ(d). (1) The orientation-resolved master-formula predictor, a deterministic-equivalent mean-level approximation to the expected minimum quadratic residual at latent dimension d, is ρ^MF(d,Δ0):=∑i=1Pκ(d)λiλi+κ(d)Δi2. ρ_MF(d; _0):= _i=1^P κ(d) _i _i+κ(d) _i^2. (2) The corresponding predicted critical dimension is d^MF:=mind∈1,…,r−1:ρ^MF(d,Δ0)≤εq. d_MF:= \d∈\1,…,r-1\: ρ_MF(d; _0)≤ _q \. (3) If the reference point already satisfies Δ0⊤HΔ0≤εq _0 H _0≤ _q, we set d^MF=0 d_MF=0. If the set in (3) is empty, we set d^MF=r d_MF=r. The predictor in Definition 1 is orientation resolved: two displacements having the same Euclidean norm can yield substantially different predicted dimensions if their energy is distributed differently across the curvature eigenvectors. Directions with large λi _i are more costly because displacement along them contributes more strongly to the quadratic residual and must therefore be represented more accurately by the random search subspace. The master formula is a mean-level predictor rather than an exact high-probability guarantee. Its derivation, given in Appendix B, is exact up to the deterministic-equivalent approximation for the expected Gaussian-span projector. Accordingly, d^MF d_MF estimates the quadratic random-slice transition but is distinct from the rigorous conic transition center dconic=δ(C∘)d_conic=δ(C ) of Theorem 1. The orientation-resolved predictor in Definition 1 requires the complete displacement profile Δii=1P\ _i\_i=1^P. When this directional information is unavailable, the master formula yields two useful spectrum-based specializations. Definition 2 (Specialized Hessian predictors). Adopt the notation of Definition 1. Let R>0R>0 denote a displacement-radius bound satisfying ‖Δ0‖2≤R,\| _0\|_2≤ R, and let r:=rank(H).r:=rank(H). When the displacement norm is known exactly, we take R=‖Δ0‖2R=\| _0\|_2. For d∈(0,r)d∈(0,r), let κ(d)>0κ(d)>0 be determined by (1). All expressions involving κ(d)κ(d) at d=0d=0 and d=rd=r are interpreted by continuity, corresponding respectively to κ→+∞κ→+∞ and κ→0κ→ 0. (i) Isotropic-orientation predictor. Under the isotropic-orientation model with displacement norm R, Δi2=R2P,i=1,…,P, _i^2= R^2P, i=1,…,P, the orientation-resolved master formula reduces to ρ^iso(d)=R2P∑i=1Pκ(d)λiλi+κ(d)=R2Pκ(d)d. ρ_iso(d)= R^2P _i=1^P κ(d) _i _i+κ(d)= R^2P\,κ(d)d. The isotropic-orientation predictor diso⋆d _iso is determined by diso⋆=∑i=1Pλiλi+κ⋆,κ⋆diso⋆=2εPR2.d _iso= _i=1^P _i _i+κ , κ d _iso= 2 PR^2. (4) (i) Orientation-uniform predictor. For every displacement satisfying ‖Δ0‖2≤R\| _0\|_2≤ R, the master formula obeys ρ^MF(d,Δ0) ρ_MF(d; _0) =∑i=1Pκ(d)λiλi+κ(d)Δi2 = _i=1^P κ(d) _i _i+κ(d) _i^2 ≤κ(d)∑i=1PΔi2 ≤κ(d) _i=1^P _i^2 ≤κ(d)R2. ≤κ(d)R^2. Imposing the sufficient condition κ(d)R2≤εqκ(d)R^2≤ _q gives the orientation-uniform predictor dunif(ε,R) d_unif( ,R) :=∑i=1Pλiλi+εq/R2 := _i=1^P _i _i+ _q/R^2 (5) =∑i=1PλiR2λiR2+εq. = _i=1^P _iR^2 _iR^2+ _q. We equivalently denote this effective-rank predictor by reff(ε,R):=dunif(ε,R).r_eff( ,R):=d_unif( ,R). (6) Remark 3 (Recovery of the quadratic radius bound of Larsen et al.). Larsen et al. [11] derive, for a positive-definite quadratic well, the approximation dlocal(ε,R)≳∑iri2R2+ri2,ri2=2ελi.d_local( ,R) _i r_i^2R^2+r_i^2, r_i^2= 2 _i. The corresponding complementary threshold expression is P−∑iri2R2+ri2=∑iλiR2λiR2+2ε=reff(ε,R),P- _i r_i^2R^2+r_i^2= _i _iR^2 _iR^2+2 =r_eff( ,R), where εq=2ε _q=2 in Definition 1. Thus, the orientation-uniform specialization of the master formula recovers the earlier radius-only quadratic expression of Larsen et al., while the orientation-resolved predictor developed here extends it by retaining the full displacement profile Δii=1P\ _i\_i=1^P. The three predictors correspond to different levels of displacement information. The master-formula predictor d^MF d_MF uses the complete orientation profile Δii=1P\ _i\_i=1^P. The isotropic predictor diso⋆d _iso replaces that profile with an equal-energy orientation model, whereas the orientation-uniform predictor dunif=reffd_unif=r_eff removes the orientation dependence through a uniform upper bound. Consequently, the isotropic-orientation predictor is sharper when its equal-energy assumption is a reasonable approximation, whereas the orientation-uniform predictor applies when only a displacement-radius estimate is available. The proof of Theorem 1 applies the conic kinematic formula to the cone C=cone(S−θ0)C=cone(S- _0) associated with the localized quadratic low-loss set S. The corresponding general result for compact convex target sets is stated and proved in Appendix A. Because direct computation of δ(C∘)δ(C ) is generally intractable for neural-network losses, Definitions 1 and 2 provide operational quadratic-model surrogates for the corresponding accessibility transition. These results motivate the three-mode dimension-selection procedure developed next. When a reliable displacement profile is available, we use the orientation-resolved master-formula predictor; when only a displacement radius is available, we use the orientation-uniform predictor dunif=reffd_unif=r_eff; and when neither estimate is reliable, we use nested adaptive latent-dimension expansion. The isotropic-orientation predictor diso⋆d _iso is retained primarily as a sharper diagnostic predictor when its equal-energy orientation model is appropriate. 3.3 Latent Dimension Selection For practical curvature estimation, all eigenvalues, eigenvectors, and trace estimates are computed from the same positive-semidefinite curvature operator. This operator is the Hessian when it is positive semidefinite, its positive spectral part when explicitly estimated, or a standard positive-semidefinite surrogate such as the generalized Gauss–Newton or Fisher matrix. For notational simplicity, we continue to denote this operator by H. The resulting quantities are practical surrogates for the quadratic predictors of Definition 1; when H is replaced by a positive-semidefinite surrogate, they should not be interpreted as direct estimators of the rigorous conic quantity dconic=δ(C∘)d_conic=δ(C ). Define the unresolved spectral mass by T^tail:=[tr^H−∑i=1kλi]+, T_tail:= [ tr\,H- _i=1^k _i ]_+, where [x]+:=maxx,0[x]_+:= \x,0\. We model the P−kP-k uncomputed eigenvalues by their common average λ^tail:=T^tailP−k,k<P. λ_tail:= T_tailP-k, k<P. (7) When k=Pk=P, all tail terms below are defined to be zero. This equal-tail spectral representation is exact when the unresolved eigenvalues are equal. Within the deterministic-equivalent master formula, it also avoids the artificial saturation that can arise when all unresolved spectral contributions are linearized at zero curvature. Mode 1: orientation-resolved selection. Suppose that a pilot, related-task, or diagnostic solution θ θ provides the displacement estimate Δ^0:=θ^−θref. _0:= θ- _ref. Its estimated components in the leading curvature eigenspace are Δ^i:=⟨Δ^0,vi⟩,i=1,…,k, _i:= _0,v_i , i=1,…,k, and the remaining displacement energy is R^tail2:=[‖Δ^0‖22−∑i=1kΔ^i2]+. R_tail^2:= [\| _0\|_2^2- _i=1^k _i^2 ]_+. (8) For each candidate dimension d, let κ^(d)>0 κ(d)>0 solve d=∑i=1kλiλi+κ^(d)+(P−k)λ^tailλ^tail+κ^(d).d= _i=1^k _i _i+ κ(d)+(P-k) λ_tail λ_tail+ κ(d). (9) The corresponding practical master formula is ρ^MF(k)(d,Δ^0) ρ_MF^(k)(d; _0) :=∑i=1kκ^(d)λiλi+κ^(d)Δ^i2 := _i=1^k κ(d) _i _i+ κ(d) _i^2 (10) +κ^(d)λ^tailλ^tail+κ^(d)R^tail2. + κ(d) λ_tail λ_tail+ κ(d) R_tail^2. Define the estimated rank of the equal-tail curvature representation by r^:=#i≤k:λi>0+(P−k) 1λ^tail>0. r:=\#\i≤ k: _i>0\+(P-k)\,1\ λ_tail>0\. (11) For interior dimensions d∈1,…,r^−1d∈\1,…, r-1\, κ^(d)>0 κ(d)>0 is determined by (9). At the two endpoints, we define the practical residual by continuity: ρ^MF(k)(0,Δ^0):=∑i=1kλiΔ^i2+λ^tailR^tail2,ρ^MF(k)(r^,Δ^0):=0. ρ_MF^(k)(0; _0):= _i=1^k _i _i^2+ λ_tail R_tail^2, ρ_MF^(k)( r; _0):=0. The raw orientation-resolved estimate is then d~MF:=mind∈0,…,r^:ρ^MF(k)(d,Δ^0)≤εq. d_MF:= \d∈\0,…, r\: ρ_MF^(k)(d; _0)≤ _q \. (12) Since ρ^MF(k)(d,Δ^0) ρ_MF^(k)(d; _0) is nonincreasing in d, the smallest qualifying interior dimension can be found efficiently by integer bisection. Mode 2: orientation-uniform radius selection. If the full displacement profile is unavailable but a displacement-radius estimate or upper bound R is available, we use the equal-tail approximation to the orientation-uniform effective-rank predictor: r^effeq(ε,R) r_eff^eq( ,R) :=∑i=1kλiR2λiR2+εq := _i=1^k _iR^2 _iR^2+ _q (13) +(P−k)λ^tailR2λ^tailR2+εq. +(P-k) λ_tailR^2 λ_tailR^2+ _q. When R is a genuine upper bound on ‖θ†−θref‖2\|θ - _ref\|_2, this retains the orientation-uniform interpretation of Definition 2. When R is estimated empirically, the resulting quantity should instead be regarded as a radius-calibrated practical predictor. The equal-tail representation of the uncomputed spectrum remains an additional approximation. The compact three-mode selection procedure is summarized in Algorithm 1. The orientation-resolved and radius modes use matrix-free curvature estimates of the leading spectrum and unresolved spectral mass, while the adaptive mode uses nested latent-dimension expansion that preserves previously active random directions and latent coordinates. Algorithm 1 Three-mode latent-dimension selection for RaMaN: compact version 0: Reference parameters θref _ref, parameter dimension P, loss tolerance ε , multiplicative safety factor γ≥1γ≥ 1, additive calibration margin b≥0b≥ 0, optional curvature operator H, optional displacement estimate Δ^0 _0, optional radius estimate R, and adaptive-expansion controls. 1: Set εq←2ε _q← 2 . 2: if a reliable curvature operator H and displacement estimate Δ^0 _0 are available then 3: Estimate the leading curvature eigenpairs and unresolved spectral tail. 4: Compute the orientation-resolved estimate d~MF d_MF using (8)–(12). 5: Set d←minP,⌈γd~MF+b⌉,mode←MF.d← \P, γ d_MF+b \, . 6: else if a reliable curvature operator H and radius estimate R are available then 7: Estimate the leading curvature eigenvalues and unresolved spectral tail. 8: Compute r^effeq(ε,R) r_eff^eq( ,R) using (13). 9: Set d←minP,⌈γr^effeq(ε,R)+b⌉,mode←Radius.d← \P, γ r_eff^eq( ,R)+b \, . 10: else 11: Apply nested adaptive expansion to obtain d and an adaptive success/failure flag. 12: Set mode←Adaptivemode . 13: end if 14: return Selected latent dimension d, selection mode, and, for the adaptive mode, the success/failure flag. Algorithm 1 separates prediction from fallback. The orientation-resolved mode is preferred when a pilot or related-task displacement provides a reliable estimate of the displacement orientation relative to the curvature eigenspaces. The radius mode discards this directional information and is therefore more conservative when R is a valid displacement bound. The adaptive mode makes no quadratic-model prediction and instead identifies a sufficient dimension through successive training runs. The multiplicative factor γ and additive margin b convert the raw mean-level predictor into a practical operating dimension. They are calibration parameters rather than consequences of the conic theorem or high-probability guarantees for the master formula. The conic theorem provides a rigorous transition window centered at dconic=δ(C∘)d_conic=δ(C ), but it does not control the deterministic-equivalent approximation error, the spectral-tail approximation, or the accuracy of a pilot displacement estimate. Nestedness is essential in the adaptive mode. When the dimension is expanded from d to dnewd_new, all previously active random directions and latent coordinates must remain unchanged. For a seed-regenerated Gaussian map, this is achieved by treating the active matrix as a column prefix of a fixed virtual Gaussian matrix generated from the same seed. For SHM, the coordinate sets are likewise generated as prefixes of one fixed seeded ordering: Ω(d)=(ω1,…,ωd),Ω(dnew)=(ω1,…,ωdnew),d<dnew, (d)=( _1,…, _d), (d_new)=( _1,…, _d_new), d<d_new, where Ω(d) (d) is a prefix of Ω(dnew) (d_new), while the sign operator and Hadamard transform are retained unchanged. These constructions ensure ℛdnew([zd;0])=ℛd(zd).R_d_new([z_d;0])=R_d(z_d). Resampling the Gaussian columns or SHM coordinate ordering after an expansion would change the underlying random subspace rather than enlarge the existing one. For the current HDIHDI SHM implementation, the sign pattern is also kept fixed to preserve exact parameterization continuity; its active signs can be absorbed into the latent coordinates and therefore do not change the reachable SHM subspace. For AdamW dimension sweeps and adaptive expansion, we initialize the learning-rate schedule across latent dimensions using α(d)=α(d0)d0d.α(d)=α(d_0) d_0d. This scaling is motivated by the dimension dependence of the induced parameter-space update under the Gaussian map. For AdamW, however, adaptive moment normalization modifies the exact step scaling, so this rule should be regarded as an optimization heuristic rather than as part of the geometric dimension-selection theory. Remark 4 (Conservative versus equal-tail correction). The equal-tail correction in (13) avoids the saturation that can occur when the unresolved spectral contribution is linearized. If a conservative upper estimate is required, it may be replaced by r^tailupper:=minP−k,R2εqT^tail. r_tail^upper:= \P-k, R^2 _q T_tail \. This follows from λR2λR2+εq≤λR2εq. λ R^2λ R^2+ _q≤ λ R^2 _q. The upper correction is conservative but may substantially overestimate the required dimension. The equal-tail correction is typically less conservative, but its accuracy depends on how well the average unresolved eigenvalue represents the spectral tail. 3.4 Random Mapping Networks As summarized in Figure 1, RaMaN instantiates the selected latent dimensions using frozen matrix-free random maps. Given a reference parameter vector θref _ref, it trains only the low-dimensional latent vectors zll=1L\z_l\_l=1^L. 3.4.1 Scalable Random Map Instantiations A dense random map from ℝdR^d to ℝPR^P requires storing a frozen P×dP× d matrix, which is prohibitive for modern networks. RaMaN avoids this persistent storage bottleneck using either seed-regenerated Gaussian maps or Structured Hadamard Mapping (SHM). In the idealized i.i.d. Gaussian construction, a global seed-regenerated Gaussian map realizes the uniformly random subspace assumed in Theorem 1, while the default layer-wise construction corresponds to the independent layer-wise setting of Corollary 3 under its product-structure assumptions. SHM provides a structured, memory-efficient alternative whose behavior is evaluated empirically rather than identified with the Haar-random model of the theorem. We therefore consider two default instantiations: SHM RaMaN, which uses a seeded random selection of Hadamard basis directions implemented through a fast Walsh–Hadamard transform, and seed-regenerated Gaussian RaMaN, which regenerates Gaussian projection blocks on the fly from fixed random seeds. We now specify the layer-wise RaMaN parameterization. Let the target network have parameter vector θ=[θ(1),…,θ(L)],θ(l)∈ℝPl,P=∑l=1LPl.θ= [θ^(1),…,θ^(L) ], θ^(l) ^P_l, P= _l=1^LP_l. RaMaN parameterizes the target weights as θ(z)=θref+Δθ(z),θ(z)= _ref+ θ(z), where θref _ref is either a random initialization for training from scratch or a pretrained parameter vector for fine-tuning. The update Δθ(z) θ(z) is generated from low-dimensional trainable latent variables rather than learned directly in ℝPR^P. In the layer-wise form used by default, each layer receives its own latent vector zl∈ℝdl,dl≪Pl,z_l ^d_l, d_l P_l, and the corresponding parameter update is Δθ(l)=βlℛl(ϕl(zl)),l=1,…,L. θ^(l)= _lR_l ( _l(z_l) ), l=1,…,L. Here ℛl:ℝdl→ℝPlR_l:R^d_l ^P_l is a frozen random map, βl>0 _l>0 is a layer-wise update scale, and ϕl _l is an optional low-dimensional nonlinear modulation. Unless otherwise stated, we set βl=1 _l=1 for all layers. In the simplest and default version, ϕl(zl)=zl, _l(z_l)=z_l, yielding a linear random reparameterization. When nonlinear modulation is used, we take ϕl(zl)=zl+αlψl(zl), _l(z_l)=z_l+ _l _l(z_l), where ψl _l is a small frozen nonlinear map satisfying ψl(0)=0,Dψl(0)=0. _l(0)=0, D _l(0)=0. Consequently, Dϕl(0)=ID _l(0)=I, so the local Jacobian at the origin is determined by the random map ℛlR_l, while the nonlinear term acts as a controlled higher-order perturbation. The parameter αl _l controls the strength of this modulation. In the nonlinear SHM ablation used in our experiments, ψl(z)=tanh(z)−z _l(z)= (z)-z and αl=0.5 _l=0.5, giving ϕl(z)=0.5z+0.5tanh(z). _l(z)=0.5z+0.5 (z). Layer-wise latent-dimension selection. The latent dimensions dll=1L\d_l\_l=1^L may be selected by applying the curvature-based modes of Algorithm 1 either to individual layer blocks or to the full parameter space. Let Hl=∑iλi(l)vi(l)(vi(l))⊤H_l= _i _i^(l)v_i^(l) (v_i^(l) ) denote the llth diagonal block of the positive-semidefinite curvature operator, where Hl∈ℝPl×PlH_l ^P_l× P_l. Treating the layers independently is a block-diagonal curvature approximation because cross-layer curvature terms are omitted. Let εl≥0 _l≥ 0 be a layer-wise loss budget satisfying ∑l=1Lεl≤ε _l=1^L _l≤ under this block-diagonal quadratic approximation, and define εq,l:=2εl _q,l:=2 _l. The layer-wise budgets εl\ _l\ are user-specified subject to ∑lεl≤ε _l _l≤ . In the absence of reliable blockwise information, our default implementation instead selects a global d and uses the proportional allocation in (16). When a reliable pilot, related-task, or diagnostic displacement is available, define Δ^0(l):=θ^(l)−θref(l). _0^(l):= θ^(l)- _ref^(l). We then apply the orientation-resolved mode of Algorithm 1 to the llth curvature block and loss budget. Let d~MF(l) d_MF^(l) denote the resulting raw master-formula estimate. The calibrated layer-wise dimension is dl=minPl,⌈γd~MF(l)+b⌉.d_l= \P_l, γ d_MF^(l)+b \. (14) When the displacement orientation is unavailable but a layer-wise displacement-radius estimate or upper bound RlR_l is available, we instead apply the orientation-uniform radius mode of Algorithm 1. Let r^eff,leq(εl,Rl) r_eff,l^eq( _l,R_l) denote the corresponding equal-tail estimate. Then dl=minPl,⌈γr^eff,leq(εl,Rl)+b⌉.d_l= \P_l, γ r_eff,l^eq( _l,R_l)+b \. (15) Thus, (14) uses the estimated displacement orientation, whereas (15) discards this directional information and is conservative when RlR_l is a valid upper bound. When RlR_l is estimated empirically rather than known as an upper bound, the resulting quantity should be regarded as a radius-calibrated practical predictor. Under the product-structure assumptions of Corollary 3, a separate theorem-level simultaneous intersection statement can be obtained by assigning failure probability η/Lη/L to each layer and applying a union bound. This probabilistic statement concerns the conic threshold conditions of the corollary and should not be interpreted as a guarantee for the calibrated master-formula dimensions in (14) or (15). If reliable blockwise curvature information is unavailable, we first use Algorithm 1 to select a global dimension d. For the default layer-wise RaMaN parameterization, we then allocate that total dimension approximately in proportion to the layer sizes: dl≈dPlP,∑l=1Ldl=d,d_l≈ d P_lP, _l=1^Ld_l=d, (16) with integer rounding adjusted to preserve the total dimension. This proportional allocation is a practical heuristic and does not, by itself, inherit the layer-wise probabilistic statement of Corollary 3. When neither reliable displacement nor radius estimates are available, we use the adaptive mode of Algorithm 1. In the layer-wise implementation, expansion must be nested within every affected layer: previously active random directions and latent coordinates are preserved, and newly added latent coordinates are initialized to zero. For general neural-network losses, where cross-layer curvature and target-set interactions need not vanish, both the block-diagonal predictor and the proportional allocation in (16) are practical approximations whose accuracy is evaluated empirically. Structured Hadamard Mapping RaMaN. The first scalable instantiation uses a Hadamard-based structured mapping (SHM), motivated by fast structured projection methods based on Walsh–Hadamard transforms, including SRHT [2, 21] and Fastfood [12]. Li et al. [14] also used Fastfood to realize scalable random-subspace experiments. Our SHM construction is not itself an SRHT; rather, it uses a seeded random selection of Hadamard basis directions together with a fast Walsh–Hadamard implementation. For each layer l, let P~l P_l be the smallest power of two satisfying P~l≥Pl. P_l≥ P_l. Let HP~l∈ℝP~l×P~lH_ P_l P_l× P_l be the normalized Walsh–Hadamard matrix, and let DlD_l be a fixed random diagonal Rademacher sign matrix of the same size. Using a fixed layer-wise seed, sample once an ordered sequence Ωlmax=(ωl,1,…,ωl,dl,max) _l = ( _l,1,…, _l,d_l, ) of distinct indices from 1,…,P~l\1,…, P_l\, where dl,max≤Pld_l, ≤ P_l is the maximum latent dimension allowed for layer l. Equivalently, Ωlmax _l may be taken as the first dl,maxd_l, entries of a seeded uniformly random permutation of 1,…,P~l\1,…, P_l\. At active dimension dld_l, define Ωl(dl):=(ωl,1,…,ωl,dl). _l(d_l):= ( _l,1,…, _l,d_l ). Thus, each Ωl(dl) _l(d_l) has the distribution of a uniformly sampled ordered dld_l-tuple of distinct indices, while dimensions are nested: dl<dl,new⟹Ωl(dl) is a prefix of Ωl(dl,new).d_l<d_l,new _l(d_l) is a prefix of _l(d_l,new). Define the coordinate-injection operator IΩl(dl):ℝdl→ℝP~lI_ _l(d_l):R^d_l P_l by [IΩl(dl)u]j=ua,j=ωl,a for some a∈1,…,dl,0,j∉ωl,1,…,ωl,dl. [I_ _l(d_l)u ]_j= casesu_a,&j= _l,a for some a∈\1,…,d_l\,\\ 0,&j∉\ _l,1,…, _l,d_l\. cases Because the coordinate sequence itself is sampled uniformly, no separate permutation operator is required. The SHM random map is defined by ℛl,dlSHM(u)=P~lPlcropPl(HP~lDlIΩl(dl)u),u∈ℝdl.R_l,d_l^SHM(u)= P_lP_l\,crop_P_l (H_ P_lD_lI_ _l(d_l)u ), u ^d_l. The factor P~l/Pl P_l/P_l normalizes the Euclidean norm of each cropped Hadamard basis column: before this scaling, a normalized Hadamard column restricted to the first PlP_l coordinates has squared norm Pl/P~lP_l/ P_l. Because DlD_l is diagonal, DlIΩl(dl)=IΩl(dl)DΩl(dl),D_lI_ _l(d_l)=I_ _l(d_l)D_ _l(d_l), where DΩl(dl):=IΩl(dl)∗DlIΩl(dl)D_ _l(d_l):=I_ _l(d_l)^*D_lI_ _l(d_l) is an invertible dl×dld_l× d_l diagonal sign matrix. Therefore, range(cropPlHP~lDlIΩl(dl)) (crop_P_lH_ P_lD_lI_ _l(d_l) ) =range(cropPlHP~lIΩl(dl)). =range (crop_P_lH_ P_lI_ _l(d_l) ). Thus, for the default identity modulation, the randomness of the reachable SHM search subspace is determined by the seeded selection Ωl(dl) _l(d_l) of Hadamard basis directions. We retain DlD_l for implementation convenience, not as an additional source of subspace randomization. The sign operator DlD_l, the Hadamard transform HP~lH_ P_l, and the ordered coordinate sequence Ωlmax _l are fixed throughout training. Consequently, for dl<dl,newd_l<d_l,new, ℛl,dl,newSHM([u;0])=ℛl,dlSHM(u).R_l,d_l,new^SHM ([u;0] )=R_l,d_l^SHM(u). (17) This prefix construction is required by adaptive dimension expansion: resampling Ωl _l after an expansion would replace the existing structured search subspace rather than enlarge it. The sign pattern is also retained unchanged to preserve exact parameterization continuity, although, as shown above, its active signs can be absorbed into the latent coordinates and do not alter the reachable SHM subspace. The cropping operation removes the padded coordinates. The map is evaluated from right to left without materializing either DlD_l or HP~lH_ P_l. The operator IΩl(dl)I_ _l(d_l) is implemented as a sparse coordinate scatter, DlD_l as elementwise multiplication by a fixed or seed-regenerated Rademacher sign vector, and HP~lH_ P_l using an in-place fast Walsh–Hadamard transform. Consequently, the map requires O(P~llogP~l)O( P_l P_l) computation and O(P~l)O( P_l) transform working memory, rather than persistent storage of a dense P~l×dl P_l× d_l random matrix. The backward pass applies the adjoints of the forward operators in reverse order. Given gl=∇Δθ(l)ℒ∈ℝPl,g_l= _ θ^(l)L ^P_l, we first zero-pad glg_l to length P~l P_l and then compute ∇ulℒ=βlP~lPlIΩl(dl)∗DlHP~lpadP~l(gl)∈ℝdl. _u_lL= _l P_lP_l\,I_ _l(d_l)^*D_lH_ P_lpad_ P_l(g_l) ^d_l. Here, IΩl(dl)∗:ℝP~l→ℝdlI_ _l(d_l)^*:R P_l ^d_l denotes the adjoint of the coordinate-injection operator IΩl(dl)I_ _l(d_l). It gathers the coordinates indexed by Ωl(dl) _l(d_l); specifically, for any x∈ℝP~lx P_l, [IΩl(dl)∗x]a=xωl,a,a=1,…,dl. [I_ _l(d_l)^*x ]_a=x_ _l,a, a=1,…,d_l. In the real Euclidean setting, padP~lpad_ P_l is the adjoint of cropPlcrop_P_l, while Dl∗=DlD_l^*=D_l and HP~l∗=HP~lH_ P_l^*=H_ P_l because DlD_l is a real diagonal sign operator and HP~lH_ P_l is the normalized symmetric Hadamard transform. Thus, both forward and backward SHM operations use fast transforms rather than dense matrix multiplication. Seed-regenerated Gaussian RaMaN. The second instantiation uses a Gaussian random map that is never stored. For each layer, define ℛlSG(u)=Alu,Al∈ℝPl×dl,R_l^SG(u)=A_lu, A_l ^P_l× d_l, where (Al)ij=1Plξij,ξij∼(0,1).(A_l)_ij= 1 P_l _ij, _ij (0,1). The normalization does not affect the Gaussian column space but controls the scale of the induced parameter update. Instead of materializing AlA_l, its entries are regenerated on demand using a counter-based pseudorandom number generator with fixed layer seed slseeds_l^seed. The counter indexing is chosen so that increasing dld_l exposes a column prefix of one fixed virtual Gaussian matrix, thereby preserving nestedness under adaptive dimension expansion. For memory-efficient evaluation, the rows of AlA_l are partitioned into BlB_l blocks: Al=[Al(1)Al(2)Al(Bl)],Al(b)∈ℝmb×dl,∑b=1Blmb=Pl.A_l= bmatrixA_l^(1)\\ A_l^(2)\\ \\ A_l^(B_l) bmatrix, A_l^(b) ^m_b× d_l, _b=1^B_lm_b=P_l. During the forward pass, each row block Al(b)A_l^(b) is regenerated to compute the corresponding output block Al(b)uA_l^(b)u. During the backward pass, the output gradient is partitioned conformably as gl=[gl(1)gl(2)gl(Bl)],gl(b)∈ℝmb,g_l= bmatrixg_l^(1)\\ g_l^(2)\\ \\ g_l^(B_l) bmatrix, g_l^(b) ^m_b, and the same row blocks are regenerated to accumulate Al⊤gl=∑b=1Bl(Al(b))⊤gl(b).A_l g_l= _b=1^B_l (A_l^(b) ) g_l^(b). Consequently, for ul=ϕl(zl)u_l= _l(z_l) and Δθ(l)=βlAlul θ^(l)= _lA_lu_l, the latent gradient is ∇zlℒ=βlDϕl(zl)⊤Al⊤gl,gl:=∇Δθ(l)ℒ. _z_lL= _lD _l(z_l) A_l g_l, g_l:= _ θ^(l)L. For the identity modulation ϕl(zl)=zl _l(z_l)=z_l, this reduces to ∇zlℒ=βlAl⊤gl. _z_lL= _lA_l g_l. Thus, in the idealized random-map model, seed-regenerated Gaussian RaMaN realizes the same i.i.d. Gaussian projection as a materialized dense map. In implementation, the projection requires no persistent Pl×dlP_l× d_l matrix: the persistent map state is the random seed, while temporary block storage is controlled by the chosen block size. 3.5 Training and Complexity Training objective. RaMaN trains only the latent variables z=zll=1L.z=\z_l\_l=1^L. The target network is used in the standard forward pass with parameters θ(z)θ(z). Given the training dataset D, the latent variables are optimized according to minzℒ(fθ(z),), _zL (f_θ(z);D ), where ℒL denotes the task-specific empirical loss evaluated on D. Gradients are propagated through the frozen random maps ℛlR_l: ∇zlℒ=βlDϕl(zl)⊤ℛl⊤∇Δθ(l)ℒ. _z_lL= _lD _l(z_l) R_l _ θ^(l)L. When ϕl _l is the identity, this reduces to ∇zlℒ=βlℛl⊤∇Δθ(l)ℒ. _z_lL= _lR_l _ θ^(l)L. Thus optimizer states are maintained only for the latent vectors zl\z_l\ rather than for all P target parameters. Remark 5 (Optimization on the slice). In the quadratic model of Theorem 1, with the linear slice θ(z)=θ0+Azθ(z)= _0+Az, optimization over z reduces to the convex least-squares problem minz12‖H1/2(Az−Δ0)‖22,Δ0:=θ†−θ0. _z 12 \|H^1/2(Az- _0) \|_2^2, _0:=θ - _0. Thus, whenever the slice intersects SεS_ , the global least-squares optimum attains loss within the prescribed tolerance. The convergence rate of gradient descent is governed by the spectrum of A⊤HA∈ℝd×dA HA ^d× d. For isotropic Gaussian A, one has [A⊤HA]=tr(H)PId,E[A HA]= tr(H)PI_d, suggesting that random reparameterization can also improve conditioning when the projected curvature concentrates around its expectation. A quantitative analysis of this effect is left to future work. Memory and computational complexity. Dense Mapping-Network implementations can require O(dP)O(dP) frozen mapping parameters in addition to the target-network parameters. This becomes prohibitive for large models because P may range from millions to billions. RaMaN removes this bottleneck. With SHM maps, the random projection is represented by signs, coordinate subsets, and Hadamard transforms, requiring O(P)O(P) memory or only random seeds when the random objects are regenerated. With seed-regenerated Gaussian maps, the projection has the same distribution as a dense Gaussian map but requires O(d)O(d) trainable memory plus O(1)O(1) seed storage. The optimizer-state memory is reduced from O(P)O(P) to O(d)O(d), or from O(Pl)O(P_l) to O(dl)O(d_l) per layer. Seed regeneration removes the O(Pldl)O(P_ld_l) storage cost but does not remove the corresponding dense arithmetic: each forward or backward application of the Gaussian map requires O(Pldl)O(P_ld_l) operations, in addition to random-number regeneration overhead. By contrast, the SHM requires O(P~llogP~l)O( P_l P_l) operations. Thus, the two RaMaN instantiations provide different memory–computation tradeoffs: the Gaussian variant exactly preserves the relevant random-subspace distribution in the settings described above, whereas the SHM variant offers substantially faster structured transforms. Importantly, RaMaN does not claim to reduce the number of parameters used by the target network during inference unless combined with orthogonal compression techniques such as pruning, quantization, or low-rank decomposition. Its primary advantages are reduced trainable dimension, reduced optimizer-state memory, reduced checkpoint size, and removal of the dense P×dP× d frozen-mapping memory wall. 3.6 RaMaN Algorithm A compact version of the RaMaN training algorithm is shown in Algorithm 2. Algorithm 2 RaMaN training: compact version 0: Reference parameters θref _ref, loss ℒL, training data D, latent dimensions dll=1L\d_l\_l=1^L, map type type∈SHM,SeedGaussiantype∈\SHM,SeedGaussian\. 1: For each layer l, initialize the trainable latent vector zl=0∈ℝdlz_l=0 ^d_l and a frozen random map ℛl:ℝdl→ℝPlR_l:R^d_l ^P_l. 2: repeat 3: Generate layer-wise updates Δθ(l)=βlℛl(ϕl(zl)),l=1,…,L. θ^(l)= _lR_l( _l(z_l)), l=1,…,L. 4: Set θ(l)=θref(l)+Δθ(l).θ^(l)= _ref^(l)+ θ^(l). 5: Evaluate ℒ(fθ(z),)L(f_θ(z);D) and update only the latent variables zll=1L\z_l\_l=1^L by backpropagating through the frozen maps ℛll=1L\R_l\_l=1^L. 6: until convergence 7: return Trained latents zll=1L\z_l\_l=1^L and the random seeds defining the frozen maps; generate θ(z)θ(z) on demand when needed. 4 Experimental Results We evaluate the geometric theory, its quadratic predictors, and the resulting Random Mapping Network (RaMaN) training procedure through twelve experiments. Experiments 1–3 examine the quadratic random-slice model: the existence and location of the phase transition, its dependence on displacement orientation, and its transfer to curvature operators obtained from trained neural networks. Experiment 4 tests whether end-to-end RaMaN optimization exhibits a comparable transition on MLP and CNN models. Experiments 5–6 evaluate the frozen-map families and the three-mode latent-dimension selection procedure. Experiments 7–9 extend the end-to-end study to a vision transformer, a pretrained language model, and a deep residual network. Experiments 10–11 examine the sensitivity of the measured training midpoint to the optimizer and the loss-excess tolerance. Finally, Experiment 12 applies the quadratic-probe pipeline to a ViT-scale curvature surrogate. Unless stated otherwise, d denotes the global latent dimension; for a layer-wise parameterization, it denotes the total latent dimension d=∑l=1Ldld= _l=1^Ld_l. 4.1 Experimental Protocol Quadratic hit criterion and empirical transition. For a positive-semidefinite curvature operator H, a reference-to-solution displacement Δ0=θ†−θ0 _0=θ - _0, and a random Gaussian map A∈ℝP×dA ^P× d, we evaluate the least-squares residual from (B.1), ρ(A)=minz∈ℝd(Az−Δ0)⊤H(Az−Δ0).ρ(A)= _z ^d(Az- _0) H(Az- _0). A trial is successful when ρ(A)≤εqρ(A)≤ _q, where εq=2ε _q=2 . For each candidate d, the empirical success probability p^(d) p(d) is the fraction of random-map trials satisfying this criterion. We denote by dpd_p the interpolated dimension at which p^(d)=p p(d)=p; in particular, d50d_50 is the empirical transition midpoint and d90−d10d_90-d_10 is its empirical width. Quadratic predictors. We compare the empirical transition with the three mean-level predictors defined in Section 3.2: the orientation-resolved master-formula predictor d^MF d_MF, the isotropic-orientation predictor diso⋆d _iso, and the orientation-uniform predictor dunif=reff(ε,R)d_unif=r_eff( ,R). When only leading Ritz eigenpairs and a stochastic trace estimate are available, we use the practical estimates d~MF d_MF and r^effeq r_eff^eq from Section 3.3. These predictors are tested directly in Experiments 1–3. Experiment 12 extends the same calculation to a ViT-scale GGN–Ritz surrogate. Because its measured midpoint and its practical predictors are computed from the same Ritz-plus-tail approximation, Experiment 12 is an internal consistency test of the surrogate rather than an independent validation of the true network curvature or of the end-to-end training midpoint. End-to-end success criterion. For RaMaN training, a run is counted as successful when its training loss reaches the prescribed target ℒtargetL_target used by the dimension-sweep protocol. The empirical training-success probability and d50d_50 are computed across random seeds. For the layer-wise map families, d always denotes ∑ldl _ld_l. For the AdamW sweeps, the learning-rate search is initialized using α(d)=α(d0)d0d,α(d)=α(d_0) d_0d, which approximately normalizes the induced parameter-space step across latent dimensions. This scaling is used as an initialization for per-d tuning rather than as an optimizer-independent rule; Experiment 10 treats SGD separately. Role of the full-parameter reference. Experiments 1–3 are designed to evaluate the quadratic transition predictors and do not constitute comparisons of RaMaN generalization against full-parameter training. In the end-to-end experiments, the full-parameter model primarily defines the training-loss target and provides a descriptive accuracy reference. Unless a protocol-matched control is explicitly stated, differences in test accuracy should therefore not be interpreted as compute-matched generalization comparisons. The corresponding d50d_50 values characterize the training transition under the specified reference and optimization protocol. Datasets, models, and roles. Table 1 summarizes the neural-network experiments. The smallest MNIST model admits a complete eigendecomposition of the positive spectral part of the Hessian. Larger curvature probes use the generalized Gauss–Newton (GGN) operator, leading Ritz eigenpairs, and a stochastic trace estimate. The ViT experiments include both end-to-end training sweeps and a separate quadratic probe based on a GGN–Ritz surrogate. The pretrained language-model experiment is an end-to-end fine-tuning study only. For bert-tiny, the embedding module is held fixed; consequently, we distinguish the full parameter count P from the reparameterized count PrepP_rep. Table 1: Neural-network settings used in the experiments. “Exact H+H_+” denotes a complete eigendecomposition of the positive spectral part of the Hessian; “GGN–Ritz” denotes a matrix-free GGN approximation based on leading Ritz eigenpairs and a stochastic trace estimate. Model Dataset Input/setting Experimental role Parameter count TinyMLP MNIST 14×1414× 14 Exact H+H_+; training P=6,634P=6,634 SmallMLP MNIST 28×2828× 28 GGN–Ritz probe P=109,386P=109,386 SmallCNN CIFAR-10 32×3232× 32 GGN–Ritz; training P=19,466P=19,466 ResNet-18 CIFAR-10/100 32×3232× 32 Training transition; generalization controls P≈11.2×106P≈ 11.2× 10^6 ViT-Tiny CIFAR-10 patch size 4 Training transition; GGN–Ritz probe P=1,806,538P=1,806,538 ViT-Tiny CIFAR-100 patch size 4 Training transition; GGN–Ritz probe P=1,823,908P=1,823,908 bert-tiny SST-2 Pretrained Fine-tuning transition P=4,386,178P=4,386,178 Prep=413,314P_rep=413,314 Data subsets and evaluation splits. Unless otherwise stated, all from-scratch image-classification experiments use fixed subsets of 10,000 training examples, with the same subset used for the full-parameter and RaMaN conditions. This protocol applies to the MNIST and CIFAR experiments in Experiments 3–7, 9–12, except that Experiments 10 and 11 reuse the saved runs from Experiment 4 and Experiment 12 reuses the ViT-Tiny data configuration from Experiment 7. For Experiment 8, we use 8,000 of the 67,349 SST-2 training examples and evaluate on all 872 validation examples. Test-set sizes, data augmentation, and the data subsets used to estimate Hessian or GGN curvature are specified below for the applicable experiments. Transition localization and reporting. An initial dimension grid brackets each transition, after which a denser local grid estimates d10d_10, d50d_50, and d90d_90. Unless otherwise stated, reported uncertainty bands and error bars summarize variation across the evaluated random seeds. 4.2 Controlled Quadratic Phase Transitions Experiment 1: transition location. We construct five synthetic positive-semidefinite quadratic instances whose spectra span low-rank, rapidly decaying, outlier-plus-bulk, and slowly decaying regimes. In every case, we use an equal-energy displacement, Δi2=R2/P _i^2=R^2/P. Figure 2 shows the empirical success curves, and Table 2 compares their midpoints with diso⋆d _iso and the orientation-uniform predictor dunifd_unif. Figure 2: Controlled quadratic phase transitions. Empirical success probability p^(d) p(d) as a function of latent dimension for five curvature spectra, using 300 independent random-slice trials per evaluated dimension. The first four cases use P=4,096P=4,096, whereas the slowly decaying power-law negative control, λi∝i−1 _i i^-1, uses P=1,024P=1,024. Under the equal-energy displacement used here, diso⋆d _iso closely matches the empirical midpoint d50d_50, whereas dunifd_unif is conservative to a spectrum-dependent degree. Table 2: Empirical transition midpoints for five controlled synthetic curvature spectra under the equal-energy displacement model. Here, rnum=#i:λi>10−14λmaxr_num=\#\i: _i>10^-14 _ \ is the numerical rank used by the residual solver, and dunif=reff(ε,R)d_unif=r_eff( ,R). The first four cases use P=4,096P=4,096, whereas the slowly decaying power-law control uses P=1,024P=1,024. Spectrum P rnumr_num d50d_50 diso⋆d _iso dunifd_unif diso⋆/d50d _iso/d_50 Block, k=128k=128 4,096 128 114.8 115.2 127.6 1.004 Power law, λi∝i−2 _i i^-2 4,096 4,096 13.8 13.9 241.3 1.007 Exponential, τ=50τ=50 4,096 1,612 180.9 180.7 335.5 0.999 Outliers with near-zero bulk 4,096 4,096 27.7 28.0 439.9 1.010 Power law, λi∝i−1 _i i^-1 1,024 1,024 685.8 686.8 763.6 1.002 Across the five spectra, whose empirical midpoints span almost a 50×50× range, the absolute relative error of diso⋆d _iso is at most 1.1%1.1\% and has median 0.36%0.36\%. The orientation-uniform predictor does not underestimate d50d_50 in these tests, but its ratio to d50d_50 ranges from approximately 1.11.1 to 17.517.5. Transition width and localization diagnostic. To separate transition width from transition location, we repeat the block-spectrum experiment at three ambient dimensions while holding the stiff rank k=128k=128 and the tolerance fraction fixed. Figure 3: Scaling and localization diagnostics. Left: empirical success curves for the block spectrum at three ambient dimensions. Middle: the measured width d90−d10d_90-d_10 remains approximately constant over the tested 16×16× range of P; the plotted P P curve is a normalized reference rather than a prediction that the measured width must follow. Right: norm of the unconstrained least-squares minimizer relative to ‖Δ0‖2\| _0\|_2. The smooth spectra remain below approximately three over the plotted range, whereas the block spectrum grows much larger near its hard rank boundary. Table 3: Transition width for the block spectrum as the ambient dimension increases. The approximately constant observed width is narrower than the general O(P)O( P) envelope in Theorem 1 and is compatible with the refined fat-cone window in Remark 7. P k d50d_50 diso⋆d _iso d90−d10d_90-d_10 1,024 128 114.5 115.2 12.4 4,096 128 114.3 115.2 12.0 16,384 128 114.4 115.2 12.9 The right panel of Figure 3 is a diagnostic rather than a validation of the localization assumption in Theorem 1. In particular, the unconstrained least-squares minimizer can lie well outside the ball of radius ‖Δ0‖2\| _0\|_2, especially for a spectrum with a hard rank boundary. Consequently, Experiments 1–3 test the quadratic residual predictors; a direct test of the localized conic theorem would additionally need to enforce or independently calibrate the search-radius constraint. 4.3 Orientation-Resolved Prediction Experiment 2: dependence on displacement orientation. We next vary the displacement direction while keeping its Euclidean norm fixed. The interpolation begins at a baseline random direction constrained to be orthogonal to the leading curvature eigenvector and ends at complete alignment with that stiff direction. The baseline endpoint is therefore not the equal-energy model used to define diso⋆d _iso. For each spectrum, the tolerance is calibrated once using the equal-energy reference displacement and then held fixed throughout the orientation sweep, so the curvature spectrum, displacement norm, and target sublevel set remain unchanged as α varies. Figure 4 shows how the empirical transition midpoint and the corresponding quadratic predictors change across the orientation sweep. Figure 4: Orientation dependence of the transition. Empirical midpoint d50d_50 as the displacement changes from a baseline random direction orthogonal to the leading curvature eigenvector (α=0α=0) to complete alignment with that stiff direction (α=1α=1). The shaded transition band spans [d10,d90][d_10,d_90], and the orientation-resolved master-formula prediction d^MF d_MF tracks the empirical midpoint throughout the sweep. Horizontal references show diso⋆d _iso and dunifd_unif; the former corresponds to the idealized equal-energy model Δi2=R2/P _i^2=R^2/P and therefore need not coincide with the α=0α=0 endpoint. Table 4: Accuracy of the orientation-resolved master-formula predictor over 30 orientation–spectrum combinations. Quantity Value Median absolute relative error 0.45% Maximum absolute relative error 5.36% Largest transition range induced by orientation 37×37× As shown in Figure 4, the measured transition changes substantially as the displacement becomes increasingly aligned with the stiff curvature direction, even though the curvature spectrum and R=‖Δ0‖2R=\| _0\|_2 remain fixed. Across the three spectra, orientation changes the transition midpoint by as much as a factor of 3737. Table 4 summarizes the prediction accuracy over all 30 orientation–spectrum combinations: d^MF d_MF has a median absolute relative error of 0.45%0.45\% and a maximum absolute relative error of 5.36%5.36\%. These results support retaining the full displacement profile when it can be estimated reliably and explain why a spectrum-only specialization can be either sharp or conservative depending on displacement orientation. 4.4 Quadratic Probes of Neural-Network Curvature Experiment 3: transfer to trained-network curvature. We apply the same quadratic hit test to curvature operators obtained from trained neural networks. The reference point is either the random initialization or a short pilot solution, and the diagnostic displacement points from that reference to the final full-parameter solution obtained under the prescribed training schedule. To quantify anisotropy, let ℐ1%I_1\% contain the largest-curvature one percent of eigen-directions and define mstiff:=∑i∈ℐ1%λiΔi2∑i=1PλiΔi2.m_stiff:= _i _1\% _i _i^2 _i=1^P _i _i^2. Values near one indicate that most of the curvature-weighted displacement is concentrated in a small set of stiff directions, whereas values near zero indicate that the displacement places little curvature-weighted mass in those directions and is instead distributed primarily over less-curved directions. Intermediate values reflect progressively mixed orientation between these two regimes. Figure 5 summarizes the representative empirical success curves together with cross-case predictor comparisons, and Table 5 reports the corresponding numerical transition midpoints and anisotropy scores. Figure 5: Quadratic probes of neural-network curvature. Top: representative empirical success curves for exact-Hessian and GGN–Ritz curvature probes. Bottom left: predicted versus measured transition midpoints across the six model/reference combinations. Bottom right: prediction-to-measurement ratios, where unity denotes perfect agreement. The orientation-resolved MF prediction remains close to the measured midpoint, whereas the equal-energy specialization diso⋆d _iso can substantially underestimate the transition when the displacement is concentrated in stiff directions. The radius-only predictor is conservative, particularly for the GGN–Ritz cases. Table 5: Quadratic phase-transition probes on trained neural networks. “MF prediction” denotes d^MF d_MF when the full spectrum is available and d~MF d_MF for the matrix-free GGN cases. Model / reference Curvature P R d50d_50 MF prediction diso⋆d _iso mstiffm_stiff TinyMLP / initialization Exact H+H_+ 6,634 9.540 1,478.3 1,480.4 998.1 0.105 TinyMLP / pilot Exact H+H_+ 6,634 3.300 1,378.6 1,374.9 985.1 0.305 SmallMLP / initialization GGN–Ritz 109,386 16.040 151.3 151.6 7.2 1.000 SmallMLP / pilot GGN–Ritz 109,386 6.073 109.7 110.4 0.0 1.000 SmallCNN / initialization GGN–Ritz 19,466 9.487 431.0 434.9 90.7 0.943 SmallCNN / pilot GGN–Ritz 19,466 4.862 743.9 742.1 309.3 0.854 As shown by the representative success curves in Figure 5, the empirical transition remains sharply defined after replacing synthetic quadratic spectra by curvature operators extracted from trained neural networks. The lower panels of Figure 5 further show that the orientation-resolved MF prediction closely follows the measured midpoint across all six verified cases, while diso⋆d _iso can substantially underestimate the required dimension and the radius-only predictor is typically conservative. Table 5 quantifies these trends. The MF prediction differs from the measured midpoint by at most 0.91%0.91\% across the six verified cases. By contrast, diso⋆d _iso underestimates the required dimension by as much as 21.0×21.0× and returns zero in one case. The observed range mstiff∈[0.105,1]m_stiff∈[0.105,1] spans markedly different orientation regimes, from displacements with relatively little curvature-weighted mass in the stiffest directions to cases in which that mass is almost entirely concentrated there. Thus, the agreement of the orientation-resolved predictor is not confined to a narrow class of displacement geometries, while the failures of the equal-energy specialization become more pronounced when the displacement is strongly aligned with stiff directions. Predictor-only scale test. The same matrix-free pipeline is applied to ResNet-18. The model has P=11,173,962P=11,173,962 parameters. Empirical verification is not feasible on the available 20 GB GPU because the diagnostic least-squares calculation scales as O(rd2)O(rd^2) time and O(rd)O(rd) memory per random trial. From a random initialization, the reported predictor values are diso⋆=8.6d _iso=8.6 and d~MF=437 d_MF=437. We treat this run as a scale diagnostic, not as a verified transition measurement. 4.5 End-to-End RaMaN Training on MLP and CNN Models Experiment 4: optimization transition. We next replace exact quadratic minimization by gradient-based RaMaN training. We compare a global seed-regenerated Gaussian map, which realizes the uniformly random subspace used by the theory, with the default layer-wise seed-regenerated Gaussian and Structured Hadamard Mapping (SHM) parameterizations. Figure 6 shows the resulting training-success transitions and test-accuracy curves, while Table 6 summarizes the corresponding empirical transition midpoints. Figure 6: End-to-end RaMaN transition on MLP and CNN models. Top: training-success probability as a function of latent dimension. Bottom: mean test accuracy, with the dash-dotted horizontal line denoting the corresponding full-parameter baseline. Vertical dashed lines indicate the empirical midpoint d50d_50 for each random-map construction. Both models exhibit a pronounced optimization transition, and test accuracy improves substantially once the transition region is crossed. For layer-wise maps, d=∑ldld= _ld_l. Table 6: Empirical training-transition midpoint and predictive performance by model and random-map construction. “Best RaMaN accuracy” denotes the highest mean test accuracy observed over the evaluated latent dimensions. The global Gaussian construction is omitted for SmallCNN because its direct per-step cost scales as O(dP)O(dP). Model Random-map construction d50d_50 Best RaMaN accuracy Full-parameter accuracy TinyMLP (P=6,634P=6,634) Global seed-regenerated Gaussian 939 90.67% 92.55% Layer-wise seed-regenerated Gaussian 939 91.09% 92.55% Layer-wise SHM 1,195 91.27% 92.55% SmallCNN (P=19,466P=19,466) Layer-wise seed-regenerated Gaussian 6,144 59.71% 61.25% Layer-wise SHM 3,072 55.56% 61.25% As shown in Figure 6, all evaluated RaMaN constructions display a clear optimization transition: below a model-dependent latent dimension, successful training is rare, whereas beyond the transition the training-success probability rises rapidly toward one. The lower panels of Figure 6 further show that test accuracy improves in parallel with this transition and approaches the full-parameter baseline more closely as d increases. Because the RaMaN sweeps and the full-parameter reference use different training budgets and tuning protocols, these accuracy curves are intended as descriptive references rather than as controlled generalization comparisons; the primary quantity measured in this experiment is the training-success transition. Table 6 quantifies both the transition location and the predictive performance achieved over the evaluated dimension range. For TinyMLP, the global seed-regenerated Gaussian map and the layer-wise seed-regenerated Gaussian map have the same empirical midpoint (d50=939d_50=939), while layer-wise SHM requires a somewhat larger latent dimension (d50=1,195d_50=1,195). All three constructions nevertheless attain best observed test accuracies within approximately two percentage points of the 92.55%92.55\% full-parameter baseline. At d50d_50, however, the transition criterion only indicates that the empirical probability of reaching the prescribed training-loss target is 0.50.5; test accuracy has not yet saturated and continues to improve as the latent dimension d increases beyond the transition. For SmallCNN, the ordering reverses: layer-wise SHM transitions earlier (d50=3,072d_50=3,072) than the layer-wise seed-regenerated Gaussian construction (d50=6,144d_50=6,144), although the latter achieves the higher best observed test accuracy over the evaluated dimension range (59.71%59.71\% versus 55.56%55.56\%). Thus, transition location and eventual predictive performance need not induce the same ordering across map families. 4.6 Ablation Studies Experiments 5–6 examine two practical design choices of RaMaN: the frozen-map family and the strategy for selecting or expanding the latent dimension. 4.6.1 Map-Family Accuracy and Cost Experiment 5: frozen-map implementation. All map families are trained at a fixed dimension above the largest measured transition midpoint for the corresponding model: d=1,912d=1,912 for TinyMLP and d=9,831d=9,831 for SmallCNN. We report accuracy together with measured frozen-map storage, optimizer-state memory, checkpoint size, peak GPU memory, and wall-clock time. The full-parameter rows are baselines measured under the ablation protocol and therefore need not coincide exactly with the references in Experiment 4. Figure 7 summarizes the resulting accuracy–systems-cost tradeoffs across map families, while Table 7 reports the corresponding numerical measurements. For the nonlinear SHM ablation, we use ϕl(z)=z+0.5(tanh(z)−z)=0.5z+0.5tanh(z). _l(z)=z+0.5 ( (z)-z )=0.5z+0.5 (z). This coordinate-wise map is bijective and therefore leaves the reachable SHM subspace unchanged; the ablation tests the effect of nonlinear latent modulation on optimization rather than an enlargement of the representable parameter set. The full-parameter row is included as a descriptive reference rather than as a compute-matched accuracy baseline; the primary comparisons in this ablation are among the random-map implementations at the same latent dimension. Figure 7: Map-family accuracy and systems cost. Test accuracy, checkpoint size, optimizer-state memory, and time per epoch for full-parameter training and four random-map parameterizations. Matrix-free maps eliminate dense frozen-map storage, while their accuracy and wall-clock cost remain model dependent. Error bars denote one standard deviation over the evaluated seeds. Table 7: Map-family ablation. Memory and checkpoint values are measured. For random-map parameterizations, accuracy is reported as mean ± one standard deviation over three seeds; each full-parameter baseline is a single reference run and is therefore reported without an uncertainty estimate. The full-parameter baseline directly optimizes all P model parameters without a random reparameterization map; all other rows optimize only the latent d variables. Model Parameterization Trainable parameters Frozen map (MB) Optimizer state (MB) Checkpoint (MB) Peak GPU (MB) Time (s/epoch) Test accuracy (%) TinyMLP MNIST Full parameters 6,634 0.00 0.05 0.03 18 0.08 93.4093.40 Dense Gaussian 1,912 45.40 0.02 45.41 109 0.11 89.83±0.4589.83± 0.45 Seed-regenerated Gaussian 1,912 0.00 0.02 0.01 137 0.18 90.17±0.6290.17± 0.62 SHM 1,912 0.00 0.02 0.01 19 0.36 89.52±0.3289.52± 0.32 SHM with nonlinear ϕl _l 1,912 0.00 0.02 0.01 19 0.40 89.55±0.4689.55± 0.46 SmallCNN CIFAR-10 Full parameters 19,466 0.00 0.16 0.08 47 0.13 60.1060.10 Dense Gaussian 9,831 267.69 0.08 267.73 405 0.20 58.48±0.6558.48± 0.65 Seed-regenerated Gaussian 9,831 0.00 0.08 0.04 364 0.56 57.63±1.1257.63± 1.12 SHM 9,831 0.00 0.08 0.04 91 0.74 54.73±1.2554.73± 1.25 SHM with nonlinear ϕl _l 9,831 0.00 0.08 0.04 91 0.81 54.18±0.7154.18± 0.71 Figure 7 illustrates the central implementation tradeoff: matrix-free parameterizations remove the potentially dominant frozen-map and checkpoint storage associated with a dense Gaussian map, but this storage advantage does not necessarily imply lower runtime or identical predictive performance. In particular, the accuracy and time costs of the different map families remain model dependent. Table 7 quantifies these differences. Seed regeneration differs from dense-Gaussian accuracy by only 0.340.34 percentage points on TinyMLP and 0.850.85 points on SmallCNN while eliminating storage of the P×dP× d random map. Relative to the dense Gaussian implementation, checkpoint size falls from 45.4145.41 to 0.010.01 MB on TinyMLP and from 267.73267.73 to 0.040.04 MB on SmallCNN. SHM differs from the dense Gaussian result by only 0.310.31 points on TinyMLP but by 3.753.75 points on SmallCNN, again indicating model-dependent behavior. The optional nonlinear modulation yields no resolved accuracy improvement over the corresponding SHM baseline at the evaluated seed counts. At these model sizes, however, the storage reductions do not translate into lower wall-clock time because map regeneration and per-layer transform overhead dominate. 4.6.2 Sweep-Free Latent-Dimension Selection Experiment 6: sweep-free dimension selection. We evaluate three strategies for obtaining a workable latent dimension without performing a full independent sweep. The oracle strategy uses curvature estimated at the final full-parameter solution together with the diagnostic radius R=‖θ†−θref‖2R=\|θ - _ref\|_2, whereas the pilot strategy uses the corresponding curvature and radius estimates from a short pilot run. The adaptive strategy uses nested adaptive expansion and requires neither quantity. For the oracle and pilot strategies, we compare the conservative linearized-tail correction of Remark 4 with the equal-tail approximation in (13), which is used by the practical default of Algorithm 1. Figure 8: Sweep-free dimension selection on TinyMLP/MNIST. Selected latent dimensions for the conservative linear-tail rule, the equal-tail default, and adaptive expansion. The horizontal dashed line marks the cross-map Gaussian reference d50ref=939d_50^ref=939 from Experiment 4. All three selection experiments use global SHM maps; oracle and pilot use curvature–radius estimates, whereas adaptive expansion increases d until the target-loss criterion is reached. Table 8: Sweep-free latent-dimension selection for TinyMLP/MNIST. The oracle and pilot strategies report both the conservative linear-tail selection and the equal-tail default. The reference d50ref=939d_50^ref=939 is the global seed-regenerated Gaussian midpoint from Experiment 4, whereas the selection experiments use global SHM maps; therefore, d/d50refd/d_50^ref is a cross-map reference ratio rather than a same-map overshoot factor. For oracle and pilot, verification is the fraction of retraining seeds that reach the target loss. Adaptive expansion is evaluated on its returned trajectory. Strategy R Conservative-tail d Default selected d d/d50refd/d_50^ref Verification Oracle radius 9.54 6,518 3,353 3.57 1.00 Pilot radius 7.26 4,602 2,779 2.96 1.00 Adaptive expansion – – 2,048 2.18 Reached Figure 8 and Table 8 compare the three sweep-free strategies. The conservative linear-tail correction remains highly conservative: the oracle and pilot strategies select d=6,518d=6,518 and d=4,602d=4,602, corresponding to approximately 98%98\% and 69%69\% of the full parameter count P=6,634P=6,634. Replacing this correction by the equal-tail default reduces the selected dimensions to 3,3533,353 and 2,7792,779, respectively, a reduction of approximately 4040–49%49\% while retaining successful retraining in all evaluated verification runs. Adaptive expansion reaches the target at the smallest selected dimension, d=2,048d=2,048, without requiring either a displacement profile or an explicit radius estimate. Its result, however, is obtained on the expansion trajectory itself rather than from an independent retraining ensemble and should therefore not be interpreted as a directly comparable success probability. All three strategies therefore provide feasible alternatives to a full dimension sweep, but with different tradeoffs. The equal-tail approximation substantially reduces the conservatism of the linearized tail correction, whereas adaptive expansion provides a curvature-free fallback when no reliable curvature–radius estimate is available. Because the sweep reference and the selection experiments use different random-map families, the ratios in Table 8 quantify relative scale only; they do not measure same-map excess dimension above the true transition. 4.7 Extensions to Larger and More Diverse Architectures Experiments 7–9 test whether the sharp end-to-end training transition from Experiment 4 persists for an attention-based model trained from scratch, a pretrained language model adapted from a nonrandom reference point, and a deep residual network. 4.7.1 Vision Transformer Training from Scratch Experiment 7: ViT-Tiny on CIFAR-10 and CIFAR-100. We train a patch-based vision transformer with patch size 4, embedding dimension 192, and depth 6. The model contains P=1,806,538P=1,806,538 parameters on CIFAR-10 and P=1,823,908P=1,823,908 on CIFAR-100. Each sweep evaluates seven latent dimensions, two layer-wise map families, and five random seeds. Figure 9: ViT-Tiny training transition on CIFAR-10. Empirical training-success probability (top) and test accuracy (bottom) as functions of total latent dimension d for layer-wise seed-regenerated Gaussian and SHM maps. The dashed line is the full-parameter reference under the reported baseline protocol. Figure 10: ViT-Tiny training transition on CIFAR-100. The same protocol as Figure 9. The transition occurs at a larger latent dimension than on CIFAR-10 despite the nearly identical parameter count, showing that the required dimension depends on the task and training objective rather than on P alone. Task Map construction d50d_50 d50/Pd_50/P Full-parameter accuracy (%) CIFAR-10 Layer-wise seed-regenerated Gaussian 597 3.31×10−43.31× 10^-4 35.55 CIFAR-10 Layer-wise SHM 384 2.13×10−42.13× 10^-4 35.55 CIFAR-100 Layer-wise seed-regenerated Gaussian 3,072 1.68×10−31.68× 10^-3 14.25 CIFAR-100 Layer-wise SHM 3,328 1.82×10−31.82× 10^-3 14.25 Table 9: Empirical training-transition midpoints for ViT-Tiny. The last column reports the full-parameter reference accuracy under the baseline protocol used to define the target loss. The reported transition values and reference accuracies are conditional on that protocol. Both tasks exhibit a sharp transition, but the transition location changes substantially with the dataset: the CIFAR-100 midpoint is approximately 5.1×5.1× higher than the CIFAR-10 midpoint for the Gaussian map and 8.7×8.7× higher for SHM. The map-family ordering also reverses across the two tasks at fixed architecture. The relative midpoints are approximately 2.1×10−42.1× 10^-4–1.8×10−31.8× 10^-3 of P, much smaller than those measured for TinyMLP and SmallCNN in Experiment 4. The accuracy curves exceed the original full-parameter reference after the transition, but this apparent advantage is sensitive to the reference training protocol. Retraining the full-parameter model under a protocol matched more closely to RaMaN raises the reference accuracy substantially and removes the apparent RaMaN advantage. Because the original ViT sweeps do not retain the per-seed losses needed to recompute the transition under the stricter matched target, we therefore interpret Experiment 7 primarily as evidence for the existence and approximate location of the training transition under the reported protocol, rather than as an accuracy comparison with full-parameter training. Runtime at ViT scale. The reported production sweeps required 150.8–155.4 minutes for the layer-wise seed-regenerated Gaussian map and 211.4–233.0 minutes for SHM on a single V100-32GB GPU. At the very small ratios d/Pd/P selected here, the fixed per-layer Hadamard-transform overhead dominates, so SHM is slower despite its more favorable scaling as d approaches P. 4.7.2 Pretrained Language-Model Fine-Tuning Experiment 8: bert-tiny on SST-2. We next set θref _ref to a pretrained bert-tiny checkpoint and fine-tune it on SST-2 through layer-wise random maps. The model contains P=4,386,178P=4,386,178 parameters. Because the word, position, and token-type embedding tensors account for most of the model and are held fixed at their pretrained values, only Prep=413,314P_rep=413,314 parameters are reparameterized. Thus, the relevant denominator for the reported latent fraction is PrepP_rep, not the full model size. Figure 11: RaMaN fine-tuning transition for bert-tiny on SST-2. Empirical training-success probability (top) and validation accuracy (bottom) as functions of total latent dimension. Word, position, and token-type embeddings remain fixed at their pretrained values. The dashed line is the full fine-tuning reference. Table 10: Empirical fine-tuning midpoint for bert-tiny/SST-2. The ratio is computed relative to the reparameterized budget Prep=413,314P_rep=413,314. The full fine-tuning reference has validation accuracy 76.83%76.83\%. Random-map construction d50d_50 d50/Prepd_50/P_rep Full val. accuracy Layer-wise seed-regenerated Gaussian 1,792 0.43% 76.83% Layer-wise SHM 2,560 0.62% 76.83% Both map families exhibit a sharp transition between the tested dimensions. The empirical midpoint uses only 0.430.43–0.62%0.62\% of the reparameterized budget, approximately 3535–75×75× smaller as a fraction than the 1414–32%32\% observed for TinyMLP and SmallCNN trained from scratch. This is qualitatively consistent with prior evidence that pretrained models can be adapted in very low-dimensional random subspaces [1, 14]. The comparison is not a controlled magnitude comparison, however: the model, dataset, initialization, training budget, frozen embedding tensors, and success criterion differ from the earlier experiments. Because only one small pretrained language model and one task are evaluated, these results should not be generalized to large language models without additional experiments. 4.7.3 Deep Residual Network Training Experiment 9: ResNet-18 on CIFAR-10 and CIFAR-100. We next examine whether the end-to-end training transition persists for a substantially deeper architecture. The GroupNorm-based ResNet-18 contains P=11,173,962P=11,173,962 parameters on CIFAR-10 and P=11,220,132P=11,220,132 on CIFAR-100. Because seed-regenerated Gaussian maps are prohibitively expensive for this architecture, we use the layer-wise SHM parameterization. Under the same transition-measurement protocol used in the preceding experiments, both tasks exhibit a sharp training-success transition at latent dimensions that are small fractions of the full parameter count. The measured midpoint is approximately 6×1036× 10^3 on CIFAR-10 and d50=10,752d_50=10,752 on CIFAR-100, corresponding to latent fractions of order 10−410^-4–10−310^-3 of P. These numerical midpoints should be interpreted as protocol-dependent end-to-end thresholds rather than intrinsic constants of the architecture or dataset. In particular, retraining the full-parameter reference under an optimization protocol matched to RaMaN substantially lowers the reference training loss, and none of the latent dimensions in the original ResNet-18 grid reaches the resulting stricter target. Consequently, a protocol-matched d50d_50 is not defined over the evaluated grid rather than simply being shifted to a larger measured value. The initially observed post-transition accuracy gap is not, by itself, evidence of an unavoidable cost of random reparameterization. We therefore repeated the CIFAR-10 comparison under a conventional training protocol with standard data augmentation, a 5,000-example validation split, and validation-selected learning rate and checkpoint. The epoch budget, optimizer, weight decay, cosine schedule, training data, and seed count were matched between RaMaN and full-parameter training. Because the learning-rate grids used in the original comparison placed both conditions at their respective grid boundaries, we additionally widened the learning-rate search for both conditions. Table 11: Protocol-matched ResNet-18/CIFAR-10 generalization comparison with 45,00045,000 training examples. Both RaMaN and full-parameter training use the same augmentation, validation split, epoch budget, optimizer, weight decay, and cosine schedule, with learning rate and checkpoint selected by validation performance from widened learning-rate grids. The generalization gap is training accuracy minus test accuracy. Results use three seeds. Method / d d/Pd/P Train acc. Test acc. Gen. gap Full parameters 100%100\% 98.89%98.89\% 89.95%89.95\% 8.98.9 RaMaN, 1,048,5761,048,576 9.4%9.4\% 96.33%96.33\% 90.33%90.33\% 6.06.0 RaMaN, 1,572,8641,572,864 14.1%14.1\% 99.22%99.22\% 90.61%90.61\% 8.68.6 RaMaN, 2,097,1522,097,152 18.8%18.8\% 98.13%98.13\% 90.36%90.36\% 7.87.8 Table 11 reports the resulting protocol-matched comparison at ntrain=45,000n_train=45,000. At d=1,048,576d=1,048,576, corresponding to only 9.4%9.4\% of the full parameter count, RaMaN reaches 90.33%90.33\% test accuracy, 0.380.38 percentage points above the 89.95%89.95\% full-parameter reference. This improves the RaMaN result at the same dimension by 1.641.64 percentage points relative to the original narrow-grid run. Notably, RaMaN attains this test accuracy despite lower training accuracy (96.33%96.33\% versus 98.89%98.89\%), resulting in a smaller generalization gap (6.06.0 versus 8.98.9 percentage points). Thus, the remaining accuracy deficit observed under the original learning-rate grid is not intrinsic to the random reparameterization: under the matched tuning protocol, the test-accuracy crossover occurs by d/P=9.4%d/P=9.4\%. Because the resulting test-accuracy margin is small and the selected full-parameter learning rate remains at the boundary of its search grid, we interpret this result primarily as evidence of accuracy parity and crossover rather than as a statistically resolved superiority claim. 4.8 Sensitivity Analyses Experiments 10–11 examine two factors involved in measuring an end-to-end training transition: the optimizer used to search the random slice and the loss-excess tolerance used to define training success. These experiments test the robustness of the observed transition and clarify which quantities must accompany a reported training midpoint d50d_50. 4.8.1 Optimizer Sensitivity Experiment 10: AdamW versus SGD. The end-to-end training midpoint combines two effects: whether the random slice contains a low-loss solution and whether the chosen optimizer finds it. To assess the optimization contribution to the observed transition, we repeat the layer-wise Gaussian and SHM sweeps using SGD with momentum 0.90.9, while holding the model, latent-dimension grid, random seeds, reference solution, and success tolerance fixed. Because the 1/d1/ d learning-rate scaling used for AdamW is not an optimizer-independent rule, SGD receives a wider per-d learning-rate search. Figure 12: Optimizer sensitivity of the SmallCNN transition. (a) Empirical training-success probability and (b) test accuracy versus total latent dimension under AdamW and SGD with momentum 0.90.9, for layer-wise seed-regenerated Gaussian and SHM maps. Both optimizers retain a sharp training transition, but SGD shifts its location toward larger d on SmallCNN. TinyMLP results, for which no corresponding shift is resolved on the evaluated grid, are summarized in Table 12. Table 12: Empirical training midpoint under AdamW and SGD with momentum 0.90.9 at the common success tolerance ε=0.05 =0.05. The comparison is resolved only to the spacing of the dimension grid. †The SGD/Gaussian estimate for SmallCNN is potentially upper biased because the selected learning rate reached the boundary of the search grid near the crossing. Model Layer-wise map AdamW d50d_50 SGD d50d_50 SGD/AdamW TinyMLP Seed-regenerated Gaussian 768 768 1.00 SHM 1,195 1,195 1.00 SmallCNN Seed-regenerated Gaussian 6,144 9,728†9,728 1.58 SHM 3,072 6,144 2.00 Figure 12 shows that the sharp end-to-end training transition persists under both optimizers on SmallCNN, but its location shifts toward larger latent dimensions under SGD. The corresponding test-accuracy curves remain broadly similar across optimizers, indicating that the main effect of the optimizer change in this experiment is on the dimension required to reach the prescribed training-loss criterion rather than on a qualitatively different accuracy trend. Table 12 quantifies the transition locations across both models. The two optimizers agree within the resolution of the evaluated grid on TinyMLP, whereas SGD requires approximately 1.61.6–2.0×2.0× more latent dimensions on SmallCNN. Thus, the existence of a sharp transition is not specific to AdamW, but the empirical end-to-end midpoint is optimizer dependent. End-to-end d50d_50 should therefore be reported together with the optimizer and tuning protocol. This dependence does not alter the optimizer-free least-squares transitions of Experiments 1–3 and 12. For TinyMLP, the optimizer-ablation sweep uses the common comparison grid rather than the denser transition-localization grid used in Experiment 4; hence its AdamW midpoint is a grid-level estimate and need not coincide exactly with the refined value reported there. 4.8.2 Sensitivity to the Loss-Excess Tolerance Experiment 11: varying ε . The end-to-end success event is defined by ℒ(θRaMaN)≤ℒ(θfull)+ε.L( _RaMaN) ( _full)+ . The main experiments use ε=0.05 =0.05. We evaluate alternative tolerances to determine whether the observed sharp transition depends on this particular choice of ε . Figure 13: Sensitivity to the success tolerance. (a–b) Representative AdamW/Gaussian training-success curves for five values of ε on TinyMLP and SmallCNN. (c–d) Empirical midpoint d50d_50 versus ε for both optimizers and both layer-wise map families. Relaxing the tolerance shifts the transition toward smaller d, while the representative success curves remain sharply localized across the tested range. Table 13: AdamW transition midpoint as the success tolerance varies over a 20×20× range, using the common comparison-grid sweeps from Experiment 10. All values are recomputed from the same saved per-seed training losses without retraining; “Spread” denotes the ratio of the largest to the smallest midpoint within each row. d50d_50 at tolerance ε Model Layer-wise map 0.01 0.02 0.05 0.10 0.20 Spread TinyMLP Seed-regenerated Gaussian 1,536 1,536 768 768 597 2.6×2.6× SHM 1,536 1,536 1,195 768 768 2.0×2.0× SmallCNN Seed-regenerated Gaussian 6,144 6,144 6,144 6,144 4,779 1.3×1.3× SHM 5,632 4,779 3,072 3,072 3,072 1.8×1.8× Figure 13(a–b) shows representative training-success curves as the loss-excess tolerance varies over ε∈0.01,0.02,0.05,0.10,0.20 ∈\0.01,0.02,0.05,0.10,0.20\. As expected, relaxing the tolerance shifts the transition toward smaller latent dimensions because every run that succeeds at a stricter tolerance also succeeds at a looser one. More importantly, the representative curves remain sharply localized throughout this 20×20× tolerance range rather than becoming diffuse as the success criterion changes. Figure 13(c–d) extends this comparison to both optimizers and both map families, while Table 13 reports the corresponding AdamW midpoints numerically. Across the tested tolerance range, the AdamW midpoint changes by factors of 1.31.3–2.62.6, depending on the model and map family. Thus, the existence of a sharp end-to-end transition is robust to the particular tolerance used to define success, although its numerical location is necessarily tolerance dependent. Empirical midpoints should therefore be compared only under a common ε . This tolerance dependence should not be interpreted as uncertainty in a midpoint measured at fixed ε . Likewise, predictor comparisons performed at a common tolerance remain well defined. If the quadratic predictors are compared across different tolerances, however, they must be recomputed using the corresponding quadratic tolerance εq=2ε _q=2 . 4.9 ViT-Scale Quadratic-Surrogate Diagnostic Experiment 12: quadratic probes for ViT-Tiny. Experiments 7–9 directly measure end-to-end training transitions at million-parameter scale but do not attach curvature-based predictors to those transitions. We therefore apply the quadratic-probe pipeline of Experiment 3 to the ViT-Tiny configurations used in Experiment 7. The probe uses the GGN operator, k=512k=512 leading Ritz eigenpairs, a stochastic trace estimate for the unresolved spectral tail, and the exact least-squares hit criterion applied to the resulting surrogate. The parameter counts match the corresponding end-to-end models exactly. Table 14: Practical quadratic predictors and measured transition midpoint for the ViT-scale GGN–Ritz surrogate. Here, d~MF d_MF is the practical orientation-resolved predictor, r^effeq r_eff^eq is the practical equal-tail orientation-uniform estimate, and “resolved fraction” is the fraction of squared displacement represented on the k=512k=512 computed Ritz directions. Reference Task P R Surrogate d50d_50 d~MF d_MF diso⋆d _iso r^effeq r_eff^eq Resolved fraction Initialization CIFAR-10 1,806,538 46.61 122.9 122.8 0.0 5,807.7 0.4% Pilot CIFAR-10 1,806,538 27.93 157.5 158.1 0.0 813.5 6.0% Initialization CIFAR-100 1,823,908 61.42 90.7 91.4 0.0 509.1 0.2% Pilot CIFAR-100 1,823,908 38.94 99.1 99.4 0.0 482.5 2.1% Figure 14: Quadratic-surrogate probes at ViT scale. The top row shows three representative success curves for random slices of the GGN–Ritz surrogate. The bottom-left panel compares practical predictions with the measured surrogate midpoint, and the bottom-right panel reports the prediction-to-measurement ratio. The close agreement of d~MF d_MF with the diagonal is an internal consistency result for the surrogate; it is not an independent validation of the true network curvature or of the end-to-end transition in Experiment 7. Within the GGN–Ritz surrogate, the practical master-formula predictor tracks the measured midpoint to within 0.04–0.79% across the four cells. The isotropic-orientation predictor evaluates to zero, whereas the practical orientation-uniform estimate exceeds the measured midpoint by factors of 4.94.9–47.347.3. This ordering is consistent with Experiment 3 and indicates that the equal-energy orientation model is inappropriate for these fitted surrogates. The result nevertheless requires a strict qualification. Only 0.2–6.0% of the squared displacement is resolved on the computed Ritz directions; the remaining 94–99.8% is represented by the tail model. Moreover, an independent exact-Hessian-vector spot check used only three trials at each of 0.5,1,2×d50\0.5,1,2\× d_50 and produced zero successes at the surrogate midpoint in the tested cells. The current data therefore support the predictor ordering and approximate scale of the surrogate transition, but they do not establish sub-percent predictor accuracy for the true ViT curvature. More exact probes concentrated near the surrogate midpoint are needed for that claim. 4.10 Implementation and Reproducibility Considerations Several controls materially affect the measured end-to-end transition. For AdamW, holding the latent learning rate fixed across d makes the induced parameter-space step grow approximately as d d and can produce a nonmonotone apparent success curve; the AdamW sweeps therefore use the reported dimension-normalized learning-rate initialization followed by per-d tuning. Experiment 10 shows that this scaling should not be transferred unchanged to SGD, for which a broader per-d search is used. A coarse uniform d-grid can also mislocate a sharp transition, so the reported midpoints use a two-stage grid or local interpolation around the crossing. Experiments 10–11 further show that an end-to-end midpoint is conditional on the optimizer and the loss-excess tolerance. These quantities should be reported whenever d50d_50 is quoted. They do not affect the optimizer-free least-squares transitions in the quadratic experiments, but a change in ε must be accompanied by the corresponding change εq=2ε _q=2 when a quadratic predictor is recomputed. Finally, adaptive map growth uses a fixed master seed and nested prefixes so that ℛdnew([zd;0])=ℛd(zd),R_d_new([z_d;0])=R_d(z_d), rather than resampling the previously trained random subspace. The original ViT-Tiny and ResNet-18 transition sweeps use the baseline reference protocol specified for those experiments, so their full-parameter accuracy comparisons should be interpreted conditionally on that protocol. Experiment 9 additionally includes a conventionally trained, protocol-matched ResNet-18 control to separate the training-transition measurement from post-transition generalization. In the language-model experiment, the embedding module is explicitly excluded from reparameterization, so the reported latent fraction is relative to PrepP_rep rather than the full model size. 5 Discussion and Conclusion Low-loss geometry, rather than ambient dimension, governs accessibility. Consistent with the geometric perspective of prior random-subspace analyses, the ambient parameter count P alone does not determine accessibility. Our quadratic analysis further shows that, beyond the curvature spectrum, the orientation of the reference-to-solution displacement can materially alter the predicted transition. For a compact convex low-loss region, the random-affine-slice analysis instead relates the transition to its effective codimension, formalized through the statistical dimension of the corresponding polar cone. In the local quadratic model, this geometric quantity depends not only on the curvature spectrum but also on the orientation of the reference-to-solution displacement relative to that spectrum. The orientation-resolved master formula makes this dependence explicit. Experiments 1–3 support this quadratic prediction across synthetic spectra and neural-network curvature operators: the orientation-resolved predictor closely tracks the measured quadratic transition, whereas spectrum-only or orientation-uniform specializations can be substantially less accurate or more conservative. These experiments validate the quadratic residual predictions rather than the full localized conic theorem; a direct empirical test of the latter would additionally require enforcing or independently calibrating its localization constraint. Geometric accessibility and end-to-end optimization are distinct. The end-to-end experiments show that the sharp transition predicted by the geometric picture has an empirical counterpart under gradient-based training. Pronounced training-success transitions appear across MLPs, CNNs, vision transformers, ResNet-18, and pretrained language-model fine-tuning, suggesting that the phenomenon is not restricted to a particular model class or initialization regime. At the same time, the numerical transition midpoint should not be interpreted as an intrinsic constant of the loss landscape. Unlike the quadratic least-squares transition, an end-to-end d50d_50 combines whether the random search space contains an acceptable solution with whether the chosen optimizer can find one. Experiments 10–11 show explicitly that its location depends on the optimizer, tuning protocol, and loss-excess tolerance, and the ResNet-18 controls further show sensitivity to the reference-training protocol. Thus, empirical d50d_50 values are most meaningful when accompanied by the map family, optimizer, success tolerance, and reference protocol under which they were measured. Training accessibility does not imply full-model generalization. A second distinction emerging from the experiments is between the dimension needed to reach a prescribed training-loss target and the dimension needed to approach the predictive performance of full-parameter training. It is useful to denote these conceptually different scales by dtraind_train and dgend_gen, respectively. The phase transition studied in this work concerns primarily dtraind_train. The ResNet-18 experiments show that substantially larger latent dimensions may be required before test accuracy approaches the full-parameter reference. At the same time, this generalization gap is not intrinsic to random reparameterization: under a conventional protocol with widened learning-rate tuning applied to both conditions, RaMaN at d=1,048,576d=1,048,576 (9.4%9.4\% of P) slightly exceeds the corresponding full-parameter test accuracy while exhibiting a smaller generalization gap. Thus, the dimension needed for training accessibility can be much smaller than the dimension needed for full-model predictive performance, but the latter need not approach P. Practical implications for RaMaN. The theory also provides a principled starting point for designing scalable random reparameterizations. Dense Gaussian maps preserve the ideal random subspace model but require an O(dP)O(dP) frozen mapping object. RaMaN replaces this storage bottleneck with seed-regenerated Gaussian or structured Hadamard maps while maintaining only d trainable latent variables and O(d)O(d) optimizer state. The experiments show, however, that no map family is uniformly preferable: Gaussian and SHM constructions can differ in transition location, predictive behavior, and runtime, and their ordering can reverse across models or datasets. Similarly, the sweep-free dimension-selection experiments show that the conservative spectral-tail correction can substantially overestimate the required dimension, while the equal-tail approximation provides a less conservative practical default and adaptive expansion provides a curvature-free alternative. Because these selection experiments and the independent transition reference use different map families, their numerical ratios should be interpreted as relative scales rather than same-map optimality guarantees. Limitations and future directions. Several limitations delimit the scope of the present results. First, the exact random-affine-slice theory applies to uniformly random Gaussian subspaces, whereas the layer-wise and SHM constructions used for scalable training introduce additional structure. Extending the theory directly to such product and structured random maps would reduce the current gap between the clean geometric model and the practical implementation. Second, the Hessian-based predictors rely on a local quadratic description of the low-loss geometry. Exact curvature information is already expensive for moderate networks, and the ViT-scale analysis necessarily relies on a GGN–Ritz approximation and a modeled spectral tail. Its close predictor agreement should therefore be viewed as an internal consistency result for that surrogate rather than as sub-percent validation of the true large-network curvature. Developing more reliable matrix-free curvature and displacement estimators is important for practical one-shot dimension selection at larger scale. A further open problem is to characterize dgend_gen, rather than only the accessibility threshold dtraind_train. The present theory explains when a random low-dimensional search space can intersect a low-loss region, but it does not determine which point within that region gradient optimization will select or how that choice affects generalization. Understanding this second scale will likely require combining the geometry developed here with optimization dynamics, implicit regularization, sample size, and properties of the data distribution. The same distinction is relevant to nonlinear random maps. In particular, a coordinate-wise bijective transformation ϕφ in a parameterization Aϕ(z)Aφ(z) leaves the reachable set equal to col(A)col(A) and therefore cannot enlarge the representable parameter space. Genuinely state-dependent mappings can bend the reachable manifold beyond a fixed linear subspace, but whether such additional geometry improves generalization enough to offset its effects on conditioning and optimization remains an open question. These directions, together with evaluation on larger pretrained models and more diverse tasks, provide natural extensions of the present framework. Conclusion. This work develops a geometric and algorithmic framework for understanding when neural networks can be trained through frozen random low-dimensional reparameterizations. Building on the effective-codimension view of random-subspace accessibility, our quadratic analysis identifies displacement orientation as an additional determinant of the latent dimension required for a random search space to access a low-loss region, and the resulting quadratic predictors are supported across controlled and neural-curvature experiments. RaMaN translates this perspective into scalable matrix-free random mappings that avoid the O(dP)O(dP) storage of dense random generators, while end-to-end experiments show that sharp training transitions persist across substantially different architectures and training regimes. At the same time, the experiments establish an important boundary on the interpretation of this phenomenon: the measured transition is conditional on the optimization and success protocol, and crossing it does not by itself guarantee full-model generalization. Taken together, these results provide a principled account of when random low-dimensional training becomes feasible, while separating that question from the distinct problems of optimization efficiency and predictive generalization. References [1] A. Aghajanyan, S. Gupta, and L. Zettlemoyer (2021) Intrinsic dimensionality explains the effectiveness of language model fine-tuning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, p. 7319–7328. External Links: Document, Link Cited by: §1, §1, §2, §4.7.2. [2] N. Ailon and B. Chazelle (2009) The fast johnson–lindenstrauss transform and approximate nearest neighbors. SIAM Journal on computing 39 (1), p. 302–322. Cited by: §1, §3.4.1. [3] D. Amelunxen, M. Lotz, M. B. McCoy, and J. A. Tropp (2014) Living on the edge: phase transitions in convex programs with random data. Information and Inference: A Journal of the IMA 3 (3), p. 224–294. External Links: Document Cited by: Appendix A, §2, Remark 1, Remark 2, Remark 7. [4] M. Dereziński, F. Liang, and M. W. Mahoney (2020) Exact expressions for double descent and implicit regularization via surrogate random design. In Advances in Neural Information Processing Systems, Vol. 33. Cited by: §B.2, §B.2. [5] B. Ghorbani, S. Krishnan, and Y. Xiao (2019) An investigation into neural net optimization via hessian eigenvalue density. In International Conference on Machine Learning, p. 2232–2241. Cited by: §1, §3.2. [6] Y. Gordon (1988) On milman’s inequality and random subspaces which escape through a mesh in ℝnR^n. In Geometric Aspects of Functional Analysis, Lecture Notes in Mathematics, Vol. 1317, p. 84–106. Cited by: §2, Remark 8. [7] F. Gressmann, Z. Eaton-Rosen, and C. Luschi (2020) Improving neural network training in low dimensional random bases. In Advances in Neural Information Processing Systems, Vol. 33. Cited by: §2. [8] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §2. [9] S. A. Koohpayegani, K. L. Navaneet, P. Nooralinejad, S. Kolouri, and H. Pirsiavash (2023) NOLA: compressing LoRA using linear combination of random basis. arXiv preprint arXiv:2310.02556. External Links: Link Cited by: §1, §2. [10] D. J. Kopiczko, T. Blankevoort, and Y. M. Asano (2024) VeRA: vector-based random matrix adaptation. In International Conference on Learning Representations, Cited by: §2. [11] B. W. Larsen, S. Fort, N. Becker, and S. Ganguli (2022) How many degrees of freedom do we need to train deep networks: a loss landscape perspective. In International Conference on Learning Representations, Cited by: §1, §1, §1, §2, §2, §3.2, Remark 1, Remark 3. [12] Q. Le, T. Sarlos, and A. Smola (2013) Fastfood - computing hilbert space expansions in loglinear time. In Proceedings of the 30th International Conference on Machine Learning, S. Dasgupta and D. McAllester (Eds.), Proceedings of Machine Learning Research, Vol. 28, Atlanta, Georgia, USA, p. 244–252. External Links: Link Cited by: §1, §2, §3.4.1. [13] O. Ledoit and S. Péché (2011) Eigenvectors of some large sample covariance matrix ensembles. Probability Theory and Related Fields 151, p. 233–264. External Links: Document Cited by: §B.2. [14] C. Li, H. Farkhoor, R. Liu, and J. Yosinski (2018) Measuring the intrinsic dimension of objective landscapes. In International Conference on Learning Representations, Cited by: §1, §1, §1, §2, §3.4.1, §4.7.2, Remark 7. [15] J. J. Moreau (1962) Décomposition orthogonale d’un espace hilbertien selon deux cônes mutuellement polaires. Comptes rendus hebdomadaires des séances de l’Académie des sciences 255, p. 238–240. Cited by: Appendix A. [16] P. Nooralinejad, A. Abbasi, S. A. Koohpayegani, K. Pourahmadi Meibodi, R. M. S. Khan, S. Kolouri, and H. Pirsiavash (2022) PRANC: pseudo RAndom networks for compacting deep models. arXiv preprint arXiv:2206.08464. External Links: Link Cited by: §1, §2. [17] V. Papyan (2020) Traces of class/cross-class structure pervade deep learning spectra. Journal of Machine Learning Research 21 (252), p. 1–64. Cited by: §1, §3.2. [18] F. Rubio and X. Mestre (2011) Spectral convergence for a general class of random matrices. Statistics & Probability Letters 81 (5), p. 592–602. Cited by: §B.2. [19] L. Sagun, U. Evci, V. U. Guney, Y. Dauphin, and L. Bottou (2017) Empirical analysis of the hessian of over-parametrized neural networks. arXiv preprint arXiv:1706.04454. Cited by: §1, §3.2. [20] L. Sen and S. Mukherjee (2026) Mapping networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §1, §2. [21] J. A. Tropp (2011) Improved analysis of the subsampled randomized hadamard transform. Advances in Adaptive Data Analysis 3 (01n02), p. 115–126. Cited by: §1, §3.4.1. [22] R. Vershynin (2018) High-dimensional probability: an introduction with applications in data science. Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press. External Links: Document Cited by: §2. Appendix A General Conic Intersection Theorem Throughout this appendix, S⊂ℝPS ^P denotes a target set that the random slice must reach, and θ0∈ℝP _0 ^P denotes the reference point. In the setting of the main text, S=Sε∩B¯(θ0,Rloc),S=S_ ∩ B( _0,R_loc), where Rloc>0R_loc>0 is the localization radius. The ball localization serves two purposes. First, it makes S compact whenever SεS_ is closed, which is exactly what is needed for the cone reduction below to be an exact equivalence (Remark 6 explains what can go wrong without localization). Second, it restricts the geometric accessibility statement to a bounded neighborhood of the reference point. The localization radius RlocR_loc should be distinguished from the displacement-radius quantity R=‖θ†−θ0‖2R=\|θ - _0\|_2 used in the quadratic predictors reff(ε,R)r_eff( ,R) and diso⋆d _iso. The former specifies the bounded target region for the rigorous conic theorem, whereas the latter measures the reference-to-solution displacement entering the quadratic surrogate. Lemma 1 (Exact reduction of affine-slice intersection to a conic intersection). Let S⊂ℝPS ^P be nonempty, compact, and convex, and let θ0∉S _0∉ S. Define C:=t(θ−θ0):t≥0,θ∈S.C\;:=\; \\,t(θ- _0)\;:\;t≥ 0,\ θ∈ S\, \. Then: (i) C is a closed convex cone (no closure operation is needed), and C contains no line; in particular, C≠0C≠\0\ and C is not a linear subspace. (i) For every linear subspace E⊆ℝPE ^P, (θ0+E)∩S≠∅⟺E∩C≠0.( _0+E)∩ S≠ E∩ C≠\0\. Proof. (i) Convexity. Let v1=t1(θ1−θ0)v_1=t_1( _1- _0) and v2=t2(θ2−θ0)v_2=t_2( _2- _0) with t1,t2≥0t_1,t_2≥ 0 and θ1,θ2∈S _1, _2∈ S, and let μ∈[0,1]μ∈[0,1]. Put s:=μt1+(1−μ)t2s:=μ t_1+(1-μ)t_2. If s=0s=0 then μv1+(1−μ)v2=0∈Cμ v_1+(1-μ)v_2=0∈ C. If s>0s>0 then μv1+(1−μ)v2=s(θ¯−θ0),θ¯:=μt1θ1+(1−μ)t2θ2s∈Sμ v_1+(1-μ)v_2=s\,( θ- _0), θ:= μ t_1 _1+(1-μ)t_2 _2s∈ S by convexity of S, so the combination lies in C. Closedness. Since S is compact and θ0∉S _0∉ S, we have ρ0:=dist(θ0,S)>0 _0:=dist( _0,S)>0, so the direction map φ(θ):=(θ−θ0)/‖θ−θ0‖2 (θ):=(θ- _0)/\|θ- _0\|_2 is continuous on S and its image D:=φ(S)⊂P−1D:= (S) ^P-1 is compact. Note C=tu:t≥0,u∈DC=\tu:t≥ 0,\ u∈ D\. Let vk=tkuk→v_k=t_ku_k→ v with tk≥0t_k≥ 0, uk∈Du_k∈ D. If v=0v=0 then v∈Cv∈ C. If v≠0v≠ 0 then tk=‖vk‖2→‖v‖2>0t_k=\|v_k\|_2→\|v\|_2>0, and by compactness of D a subsequence ukj→u∈Du_k_j→ u∈ D, whence v=‖v‖2u∈Cv=\|v\|_2\,u∈ C. No line. Suppose u≠0u≠ 0 with u∈Cu∈ C and −u∈C-u∈ C. Then u=s(θ1−θ0),−u=t(θ2−θ0)u=s( _1- _0), -u=t( _2- _0) for some θ1,θ2∈S _1, _2∈ S and necessarily s,t>0s,t>0. Adding these two identities gives s(θ1−θ0)+t(θ2−θ0)=0,s( _1- _0)+t( _2- _0)=0, and hence θ0=sθ1+tθ2s+t. _0= s _1+t _2s+t. Thus θ0∈S _0∈ S by convexity of S, contradicting θ0∉S _0∉ S. Since S≠∅S≠ and θ0∉S _0∉ S, there exists θ∈Sθ∈ S with θ−θ0≠0θ- _0≠ 0, and hence θ−θ0∈Cθ- _0∈ C. Thus C≠0C≠\0\. Therefore, a cone containing no line cannot be a nonzero linear subspace, and C is not a linear subspace. (i). (⇒ ) If θ∈(θ0+E)∩Sθ∈( _0+E)∩ S, then u:=θ−θ0∈Eu:=θ- _0∈ E, u=1⋅(θ−θ0)∈Cu=1·(θ- _0)∈ C, and u≠0u≠ 0 because θ0∉S _0∉ S. (⇐ ) If 0≠v∈E∩C0≠ v∈ E∩ C, write v=t(θ−θ0)v=t(θ- _0) with θ∈Sθ∈ S; since v≠0v≠ 0 we have t>0t>0, hence θ=θ0+v/t∈θ0+Eθ= _0+v/t∈ _0+E because E is a subspace. Thus θ∈(θ0+E)∩Sθ∈( _0+E)∩ S. ∎ Remark 6 (Why compactness matters: a closure pitfall for unbounded sublevel sets). If S is closed convex but unbounded, the set t(θ−θ0):t≥0,θ∈S\t(θ- _0):t≥ 0,\ θ∈ S\ need not be closed, and taking its closure can destroy the equivalence in Lemma 1(i). Example: in ℝ2R^2, let S=(x,y):x>0,y≥1/xS=\(x,y):x>0,\ y≥ 1/x\ (closed and convex) and θ0=(0,0)∉S _0=(0,0)∉ S. The ray generated by e1=(1,0)e_1=(1,0) lies in the closure of the cone but not in the cone itself, and the line θ0+spane1 _0+span\e_1\ never meets S. This situation is not exotic in our application: when the Hessian H⪰0H 0 has zero eigenvalues (flat directions), the quadratic sublevel set SεS_ is an unbounded cylinder. Let S=Sε∩B¯(θ0,Rloc)S=S_ ∩ B( _0,R_loc). The ball localization with radius RlocR_loc removes this issue. We recall the statistical dimension and the two properties used below. Definition 3 (Statistical dimension). For a closed convex cone C⊆ℝPC ^P, δ(C):=g∼(0,IP)[‖ΠC(g)‖22],δ(C):=E_g (0,I_P) [\,\| _C(g)\|_2^2\, ], where ΠC _C denotes Euclidean projection onto C. The polar cone is C∘:=v:⟨v,u⟩≤0∀u∈CC :=\v: v,u ≤ 0\ ∀ u∈ C\. Lemma 2 (Two standard facts). (a) If L is a linear subspace, δ(L)=dimLδ(L)= L. (b) (Moreau complementarity) For every closed convex cone C, δ(C)+δ(C∘)=P.δ(C)+δ(C )=P. Proof. (a) ΠL _L is the orthogonal projector onto L, so ‖ΠLg‖22=tr(ΠL)=dimLE\| _Lg\|_2^2=tr( _L)= L. (b) By the Moreau decomposition [15], every g splits as g=ΠC(g)+ΠC∘(g)g= _C(g)+ _C (g) with ⟨ΠC(g),ΠC∘(g)⟩=0 _C(g), _C (g) =0, hence ‖g‖22=‖ΠC(g)‖22+‖ΠC∘(g)‖22\|g\|_2^2=\| _C(g)\|_2^2+\| _C (g)\|_2^2. Taking expectations over g∼(0,IP)g (0,I_P) gives P=δ(C)+δ(C∘)P=δ(C)+δ(C ). ∎ Theorem 2 (Random affine-slice intersection: conic phase transition). Let S⊂ℝPS ^P be nonempty, compact, and convex, let θ0∉S _0∉ S, and let C be the closed convex cone of Lemma 1. Let E⊂ℝPE ^P be a uniformly random d-dimensional linear subspace; equivalently, E is the column space of a matrix A∈ℝP×dA ^P× d with i.i.d. (0,1)N(0,1) entries, which has rank d almost surely. Fix η∈(0,1)η∈(0,1) and set aη:=8log(4/η).a_η:= 8 (4/η). Then d+δ(C)≥P+aηP d+δ(C)\;≥\;P+a_η P ⟹ℙ[(θ0+E)∩S≠∅]≥ 1−η, [( _0+E)∩ S≠ ]\;≥\;1-η, (A.1) d+δ(C)≤P−aηP d+δ(C)\;≤\;P-a_η P ⟹ℙ[(θ0+E)∩S≠∅]≤η. [( _0+E)∩ S≠ ]\;≤\;η. (A.2) Consequently, the random-slice accessibility transition is centered at dconic:=P−δ(C)=δ(C∘),d_conic:=P-δ(C)=δ(C ), within an O(P)O( P) transition window, where the last equality follows from Lemma 2(b). Thus, dconicd_conic denotes the geometric transition center rather than an exact finite-dimensional critical dimension. Proof. By Lemma 1(i), the event (θ0+E)∩S≠∅( _0+E)∩ S≠ coincides with the event E∩C≠0E∩ C≠\0\. Write E=QE0E=QE_0, where E0E_0 is a fixed d-dimensional subspace and Q is a Haar-distributed random orthogonal matrix; this is the definition of a uniformly random subspace, and it agrees with the column-space description because the column space of an i.i.d. Gaussian matrix is rotation invariant. A linear subspace is a closed convex cone with δ(E0)=dδ(E_0)=d by Lemma 2(a). By Lemma 1(i), C is a closed convex cone that is not a linear subspace. The approximate kinematic formula of Amelunxen, Lotz, McCoy, and Tropp [3, Theorem I] applies to the pair (C,E0)(C,E_0), one of which is not a subspace, and states that for a Haar-random rotation Q: δ(C)+δ(E0)≤P−aηP⟹ℙ[C∩QE0≠0]≤η,δ(C)+δ(E_0)≤ P-a_η P\; \;P [C∩ QE_0≠\0\ ]≤η, δ(C)+δ(E0)≥P+aηP⟹ℙ[C∩QE0≠0]≥1−η.δ(C)+δ(E_0)≥ P+a_η P\; \;P [C∩ QE_0≠\0\ ]≥ 1-η. Substituting δ(E0)=dδ(E_0)=d yields (A.1)–(A.2), and Lemma 2(b) gives P−δ(C)=δ(C∘)P-δ(C)=δ(C ). ∎ Remark 7 (Sharper window in the fat-cone regime). Theorem 2 uses the dimension-free constant aηPa_η P from [3, Theorem I]. Because our random object is a subspace, the refined subspace form [3, Theorem 7.1] applies and gives a window governed by ω(C):=δ(C)∧δ(C∘)ω(C):= δ(C) δ(C ) instead of P P: for λ≥0λ≥ 0, d≥δ(C∘)+λd≥δ(C )+λ implies intersection with probability at least 1−pC(λ)1-p_C(λ), and d≤δ(C∘)−λd≤δ(C )-λ implies a miss with probability at least 1−pC(λ)1-p_C(λ), where pC(λ)=4exp(−λ2/8ω2(C)+λ)p_C(λ)=4 (- λ^2/8ω^2(C)+λ ). In the regime of interest, δ(C∘)=dconic≪Pδ(C )=d_conic P, so ω2(C)=dconicω^2(C)=d_conic and the transition width is O(dconiclog(1/η)+log(1/η)),O\! ( d_conic (1/η)+ (1/η) ), which can be far narrower than O(P)O( P) and is consistent with the sharp empirical transitions reported by Li et al. [14]. Concretely, pC(λ)≤ηp_C(λ)≤η holds for λ=max4log(4/η)ω(C), 16log(4/η)λ= \4 (4/η)\,ω(C),\,16 (4/η)\. Remark 8 (Nonconvex sublevel sets). For nonconvex SεS_ , the exact transition above does not apply, but two one-sided statements survive. A miss bound follows from Gordon’s escape-through-a-mesh theorem [6] applied to the spherical shadow Σ=(θ−θ0)/‖θ−θ0‖2:θ∈Sε =\(θ- _0)/\|θ- _0\|_2:θ∈ S_ \: if P−dP-d exceeds a constant multiple of w(Σ)2w( )^2, the slice misses with high probability. A hit bound follows by restricting to any compact convex subset of a single basin (e.g., its localized quadratic ellipsoid) and applying Theorem 2; a union of basins can only increase the intersection probability. Corollary 3 (Layer-wise slices under product structure). Suppose the localized target set contains a product S(1)×⋯×S(L)S^(1)×·s× S^(L), where each S(l)⊂ℝPlS^(l) ^P_l is nonempty, compact, and convex with θ0(l)∉S(l) _0^(l)∉ S^(l), and let ClC_l be the cone of Lemma 1 for (S(l),θ0(l))(S^(l), _0^(l)). Let E1,…,ELE_1,…,E_L be independent uniformly random subspaces of dimensions d1,…,dLd_1,…,d_L. If dl≥Pl−δ(Cl)+aη/LPl,l=1,…,L,d_l\;≥\;P_l-δ(C_l)+a_η/L P_l, l=1,…,L, then with probability at least 1−η1-η every layer slice θ0(l)+El _0^(l)+E_l intersects S(l)S^(l) simultaneously, so the product slice intersects the product target. Proof. Apply Theorem 2 to each layer with failure probability η/Lη/L and take a union bound. ∎ Appendix B Heuristic Derivation of the Master Formula and Its Specializations This section derives the orientation-resolved master formula and its isotropic-orientation and orientation-uniform specializations, and clarifies their precise status. The derivation is exact up to a single, clearly labeled random-matrix approximation (Approximation 1); we verify this approximation against an exactly solvable model in Section B.5. The resulting hierarchy contains three predictors: • the orientation-resolved master-formula predictor, which uses the complete displacement profile Δii=1P\ _i\_i=1^P relative to the curvature eigenvectors and is the primary quadratic-model predictor used by Mode 1 of Algorithm 1; • the isotropic-orientation predictor diso⋆d _iso, obtained by imposing the equal-energy model Δi2=R2/P _i^2=R^2/P. It is generally sharper than the orientation-uniform predictor and is used primarily as a diagnostic prediction when the isotropic-orientation model is appropriate; and • the orientation-uniform predictor dunif(ε,R)=reff(ε,R),d_unif( ,R)=r_eff( ,R), obtained by removing the displacement-orientation dependence through a uniform upper bound. It requires only the displacement radius R and is used by Mode 2 of Algorithm 1. The master-formula predictor is the most informative when a reliable displacement profile is available. The isotropic predictor provides a sharper falsifiable diagnostic under its equal-energy assumption, whereas dunif=reffd_unif=r_eff is the conservative radius-only choice when directional information is unavailable. All three remain mean-level quadratic-model predictors rather than high-probability consequences of the conic theorem. B.1 Exact reduction to a least-squares residual Work in the quadratic model of Theorem 1: H⪰0H 0 with eigendecomposition H=∑i=1Pλivivi⊤H= _i=1^P _iv_iv_i , minimizer θ†θ , reference point θ0 _0, displacement Δ0:=θ†−θ0,Δi:=⟨Δ0,vi⟩,‖Δ0‖2≤R. _0:=θ - _0, _i:= _0,v_i , \| _0\|_2≤ R. Consider the random slice θ(z)=θ0+Azθ(z)= _0+Az with A∈ℝP×dA ^P× d having i.i.d. (0,1/P)N(0,1/P) entries (the scaling is immaterial: it affects neither the column space nor the intersection event). Since 2(ℒ~(θ(z))−ℒ(θ†))=(Az−Δ0)⊤H(Az−Δ0)=‖Mz−b‖22,M:=H1/2A,b:=H1/2Δ0,2 ( L(θ(z))-L(θ ) )=(Az- _0) H(Az- _0)= \|Mz-b \|_2^2, M:=H^1/2A, b:=H^1/2 _0, minimizing over z gives the exact identity minz∈ℝd2(ℒ~(θ(z))−ℒ(θ†))=‖(I−PW)b‖22=:ρ(A),W:=col(M)=H1/2col(A), _z ^d2 ( L(θ(z))-L(θ ) )\;=\; \|(I-P_W)\,b \|_2^2\;=:\;ρ(A), W:=col(M)=H^1/2col(A), (B.1) where PWP_W is the orthogonal projector onto W. Let S~ε:=θ:12(θ−θ†)⊤H(θ−θ†)≤ε S_ := \θ: 12(θ-θ ) H(θ-θ )≤ \ denote the unlocalized quadratic sublevel set. Then (θ0+range(A))∩S~ε≠∅=ρ(A)≤2ε. \( _0+range(A))∩ S_ ≠ \\;=\; \ρ(A)≤ 2 \. (B.2) Everything so far is exact for the unlocalized quadratic sublevel set. To connect this residual criterion to the localized target S=S~ε∩B¯(θ0,Rloc)S= S_ ∩ B( _0,R_loc) used in Theorem 1, one additionally requires a least-squares solution z⋆z attaining (B.1) to satisfy ‖Az⋆‖2≤Rloc.\|Az \|_2≤ R_loc. Thus, the effective-rank derivation should be viewed as a predictor for the localized conic threshold when the least-squares solution lies inside the chosen localization ball. In experiments, this condition can be checked by monitoring the trained displacement norm ‖Δθ‖2\| θ\|_2. Note that b carries the full orientation dependence: bi=λiΔib_i= _i\, _i, so displacement along flat directions (λi=0 _i=0) contributes nothing, while displacement along stiff directions must be canceled by the random subspace W. B.2 Deterministic-equivalent approximation The columns of M are i.i.d. (0,H/P)N(0,H/P) vectors, so W is the span of d i.i.d. Gaussian vectors with covariance proportional to H. This subspace is not Haar distributed: it is biased toward large-λ directions, which is precisely why few latent dimensions suffice. Approximation 1 (Ridge-leverage surrogate for the Gaussian-span projector). Let A∈ℝP×dA ^P× d have i.i.d. (0,1/P)N(0,1/P) entries, and let W=col(H1/2A).W=col(H^1/2A). For d<rank(H)d<rank(H), we use the surrogate [PW]≈H(H+κI)−1,E[P_W]\;≈\;H(H+κ I)^-1, where κ=κ(d)>0κ=κ(d)>0 is chosen to preserve the exact projector trace: d=tr[H(H+κI)−1]=∑i=1Pλiλi+κ.d=tr\! [H(H+κ I)^-1 ]= _i=1^P _i _i+κ. (B.3) The approximation is supported by random-projection results under high-dimensional concentration conditions, but is used here as a finite-dimensional surrogate outside those regimes. Leave-one-out derivation. We next explain why the ridge-leverage form in Approximation 1 arises. The projector identities below are exact; the approximation enters only when the leave-one-out quadratic forms are replaced by a common deterministic equivalent. Define M:=H1/2A∈ℝP×d,W=col(M).M:=H^1/2A ^P× d, W=col(M). Let r=rank(H)r=rank(H) and write H=UrΛrUr⊤H=U_r _rU_r , where Λr≻0 _r 0. Then M=UrΛr1/2(Ur⊤A).M=U_r _r^1/2(U_r A). Since A is Gaussian and UrU_r has orthonormal columns, Ur⊤A∈ℝr×dU_r A ^r× d is also Gaussian and has full column rank d almost surely whenever d≤rd≤ r. Hence, for d<rank(H)d<rank(H), the columns of M are linearly independent almost surely, and the orthogonal projector onto W is PW=M(M⊤M)−1M⊤.P_W=M(M M)^-1M . Since M⊤M=A⊤HA,M M=A HA, we define Q:=A⊤HA∈ℝd×d,Q:=A HA ^d× d, and obtain PW=H1/2AQ−1A⊤H1/2.P_W=H^1/2AQ^-1A H^1/2. Because H is symmetric positive semidefinite, it admits an eigendecomposition H=UΛU⊤,Λ=diag(λ1,…,λP),H=U U , =diag( _1,…, _P), where U is orthogonal. We now express all quantities in this orthonormal eigenbasis. Under the coordinate transformation A~=U⊤A A=U A, the matrix A~ A has the same i.i.d. Gaussian distribution as A by rotational invariance. Hence, without loss of generality, we may work in coordinates in which H=diag(λ1,…,λP),H=diag( _1,…, _P), and, for notational simplicity, continue to denote the transformed Gaussian matrix by A. Let ai⊤∈ℝ1×da_i ^1× d denote the iith row of A. In these coordinates, Q=A⊤HA=∑j=1Pλjajaj⊤.Q=A HA= _j=1^P _ja_ja_j . Moreover, the iith row of H1/2AH^1/2A is λiai⊤ _i\,a_i . Therefore, (PW)ii (P_W)_i =ei⊤H1/2AQ−1A⊤H1/2ei =e_i H^1/2AQ^-1A H^1/2e_i =λiai⊤Q−1ai. = _ia_i Q^-1a_i. Thus, understanding [PW]E[P_W] reduces to approximating the quantities [λiai⊤Q−1ai],i=1,…,P.E\! [ _ia_i Q^-1a_i ], i=1,…,P. To separate the dependence of Q−1Q^-1 on the row aia_i, write Q=∑j=1Pλjajaj⊤=Q−i+λiaiai⊤,Q−i:=∑j≠iλjajaj⊤.Q= _j=1^P _ja_ja_j =Q_-i+ _ia_ia_i , Q_-i:= _j≠ i _ja_ja_j . For d<r:=rank(H)d<r:=rank(H), the matrix Q−iQ_-i is invertible almost surely: after removing the iith term, at least r−1≥dr-1≥ d independent Gaussian rows with positive weights remain when λi>0 _i>0, while all r>dr>d positively weighted rows remain when λi=0 _i=0. We may therefore define ti:=ai⊤Q−i−1ai.t_i:=a_i Q_-i^-1a_i. For λi>0 _i>0, applying the Sherman–Morrison identity to Q=Q−i+λiaiai⊤Q=Q_-i+ _ia_ia_i gives Q−1=Q−i−1−λiQ−i−1aiai⊤Q−i−11+λiai⊤Q−i−1ai.Q^-1=Q_-i^-1- _iQ_-i^-1a_ia_i Q_-i^-11+ _ia_i Q_-i^-1a_i. Sandwiching this identity between ai⊤a_i and aia_i yields ai⊤Q−1ai a_i Q^-1a_i =ti−λiti21+λiti =t_i- _it_i^21+ _it_i =ti1+λiti. = t_i1+ _it_i. Therefore, (PW)ii=λiai⊤Q−1ai=λiti1+λiti.(P_W)_i= _ia_i Q^-1a_i= _it_i1+ _it_i. The same identity holds trivially when λi=0 _i=0, since both sides then vanish. The advantage of this representation is that Q−iQ_-i depends only on the rows aj:j≠i\a_j:j≠ i\ and is therefore independent of aia_i. Conditional on Q−iQ_-i, the vector ai∼(0,Id/P)a_i (0,I_d/P) is independent of Q−iQ_-i. Therefore, the standard Gaussian quadratic-form identities give [ti∣Q−i]=1Ptr(Q−i−1),Var(ti∣Q−i)=2P2tr(Q−i−2).E[t_i Q_-i]= 1Ptr(Q_-i^-1), (t_i Q_-i)= 2P^2tr(Q_-i^-2). Consequently, tit_i concentrates around its conditional mean whenever Q−i−1Q_-i^-1 is not spectrally dominated by a small number of directions. Under standard high-dimensional regularity conditions—for example, sufficiently large stable rank rH:=tr(H)‖H‖r_H:= tr(H)\|H\| with rH/d>1r_H/d>1 bounded away from one—leave-one-out resolvent arguments further approximate the normalized traces P−1tr(Q−i−1)P^-1tr(Q_-i^-1) by a common deterministic scalar τ; see [4, Theorem 2 and its proof discussion]. We therefore make the approximation ti≈τ,i=1,…,P.t_i≈τ, i=1,…,P. Substituting ti≈τt_i≈τ into the exact leave-one-out identity gives [(PW)ii]≈λiτ1+λiτ=λiλi+κ,κ:=τ−1.E[(P_W)_i]≈ _iτ1+ _iτ= _i _i+κ, κ:=τ^-1. Gaussian sign symmetry makes the off-diagonal entries of [PW]E[P_W] vanish in the eigenbasis of H, producing [PW]≈H(H+κI)−1.E[P_W]≈ H(H+κ I)^-1. Finally, because tr(PW)=dtr(P_W)=d exactly, matching the trace gives (B.3). This leave-one-out argument is the finite-dimensional heuristic underlying Approximation 1; rigorous approximation bounds under stable-rank conditions are developed in [4]. The map κ↦∑iλi/(λi+κ)κ _i _i/( _i+κ) is strictly decreasing from rank(H)rank(H) (as κ→0κ→ 0) to 00 (as κ→∞κ→∞), so (B.3) defines a bijection between d∈(0,rankH)d∈(0,rankH) and κ∈(0,∞)κ∈(0,∞). Related deterministic-equivalent results for sample-covariance resolvents are developed in [18, 13]. Approximation 1 is exact when H∝IH I (a Haar subspace, where [PW]=(d/P)IE[P_W]=(d/P)I) and exact in the block-constant model of Section B.5. We use Approximation 1 as a finite-dimensional heuristic. The conic theorem provides a high-probability transition window centered at δ(C∘)δ(C ); it does not, by itself, control the approximation error of Approximation 1 or the fluctuations of ρ(A)ρ(A). Combining (B.1) with Approximation 1 yields the master formula [ρ(A)]≈b⊤(I−H(H+κI)−1)b=∑i=1Pκλi+κbi2=∑i=1Pκλiλi+κΔi2,E [ρ(A) ]\;≈\;b (I-H(H+κ I)^-1 )b\;=\; _i=1^P κ _i+κ\,b_i^2\;=\; _i=1^P κ _i _i+κ\, _i^2, (B.4) where Hvi=λiviHv_i= _iv_i, Δi:=⟨θ†−θ0,vi⟩ _i:= θ - _0,v_i is the component of the reference-to-solution displacement along the iith Hessian eigenvector, and bi=λiΔib_i= _i\, _i. The scalar κ=κ(d)>0κ=κ(d)>0 is determined by (B.3). The final expression in (B.4) defines the orientation-resolved mean-level residual predictor ρ^MF(d,Δ0):=∑i=1Pκ(d)λiλi+κ(d)Δi2. ρ_MF(d; _0):= _i=1^P κ(d) _i _i+κ(d) _i^2. The corresponding predicted critical dimension is d^MF:=mind∈1,…,r−1:ρ^MF(d,Δ0)≤εq, d_MF:= \d∈\1,…,r-1\: ρ_MF(d; _0)≤ _q \, (B.5) with the same endpoint conventions as in Definition 1: we set d^MF=0 d_MF=0 when Δ0⊤HΔ0≤εq _0 H _0≤ _q, and d^MF=r d_MF=r when the set in (B.5) is empty. Since εq=2ε _q=2 , this is equivalent to using [ρ(A)]≤2εE[ρ(A)]≤ 2 as a mean-level predictor of the transition. Converting this mean-level condition into a high-probability success guarantee would require separate concentration control for ρ(A)ρ(A). B.3 Isotropic-orientation specialization Under the typical-orientation model in which Δ0 _0 is spread evenly across the eigenbasis, Δi2=R2/P _i^2=R^2/P for all i, the master formula (B.4) becomes [ρ(A)]≈R2P∑i=1Pκλiλi+κ=R2Pκd(κ),E [ρ(A) ]\;≈\; R^2P _i=1^P κ\, _i _i+κ\;=\; R^2P\,κ\,d(κ), using ∑iκλi/(λi+κ)=κd(κ) _iκ _i/( _i+κ)=κ\,d(κ) from (B.3). Setting this equal to 2ε2 gives the self-consistent isotropic predictor diso⋆d _iso, defined by the pair of equations diso⋆=∑i=1Pλiλi+κ⋆,κ⋆diso⋆=2εPR2.d _iso= _i=1^P _i _i+κ , κ \,d _iso= 2 PR^2. (B.6) The system is monotone (larger d means smaller κ, hence smaller residual) and is solved numerically by bisection on κ: given κ, compute d(κ)d(κ) from (B.3) and check the sign of κd(κ)−2εP/R2κ\,d(κ)-2 P/R^2. Because κ⋆=2εP/(R2diso⋆)κ =2 P/(R^2d _iso) exceeds 2ε/R22 /R^2 by the factor P/diso⋆≫1P/d _iso 1 in the low-dimensional regime diso⋆≪Pd _iso P, the isotropic cutoff is much larger than the orientation-uniform cutoff, and therefore diso⋆≤dunif(ε,R),d _iso\;≤\;d_unif( ,R), often by a substantial margin. The interpretation is a smoothed tail-mass rule: under isotropy each direction carries excess loss λiR2/(2P) _iR^2/(2P), and the random subspace must (approximately) cancel the stiffest directions until the total remaining excess drops below ε — a tail-sum criterion — rather than canceling every direction whose individual worst-case excess λiR2 _iR^2 exceeds ε , which is what dunifd_unif does. Interpretation. diso⋆d _iso is the sharper falsifiable prediction for phase-transition experiments under random initialization, where the isotropic model is plausible (and directly testable by recording the eigenbasis profile of H1/2(θ0−θfinal)H^1/2( _0- _final)). B.4 Orientation-uniform specialization Since κλiλi+κ≤κ κ _i _i+κ≤κ for every i, the master formula (B.4) gives, uniformly over all orientations of the displacement with ‖Δ0‖2≤R\| _0\|_2≤ R, [ρ(A)]≤κ∑iΔi2≤κR2.E [ρ(A) ]\;≤\;κ _i _i^2\;≤\;κ R^2. Under the deterministic-equivalent approximation, imposing κR2≤εqκ R^2≤ _q ensures that the orientation-uniform residual surrogate does not exceed εq _q for any displacement satisfying ‖Δ0‖2≤R\| _0\|_2≤ R. Setting κ=εq/R2κ= _q/R^2 gives dunif(ε,R)=∑i=1Pλiλi+εq/R2=∑i=1PλiR2λiR2+εq.d_unif( ,R)= _i=1^P _i _i+ _q/R^2= _i=1^P _iR^2 _iR^2+ _q. (B.7) This orientation-uniform bound can be nearly tight when the displacement is concentrated along a sufficiently stiff direction. In particular, if Δ0=Rvj _0=Rv_j and λj≫εq/R2 _j _q/R^2, then κλjλj+κR2≈κR2, κ _j _j+κR^2≈κ R^2, so the required dimension approaches dunifd_unif. Intuitively, a random isotropic subspace cannot selectively target one prescribed stiff direction; accurately capturing that direction requires a sufficiently large ridge leverage, while also allocating leverage to directions of comparable or greater curvature. Interpretation. reff(ε,R)r_eff( ,R) is the orientation-uniform mean-level predictor. It is a conservative radius-only choice and can be approached when the displacement concentrates along sufficiently stiff curvature directions. B.5 Verification in an exactly solvable block model Let H=λ(Ik⊕0P−k)H=λ (I_k 0_P-k ): k stiff directions of equal curvature λ and P−kP-k exactly flat directions, with isotropic displacement Δi2=R2/P _i^2=R^2/P. Exact computation. Here b is supported on the stiff block with ‖b‖22=λkR2/P\|b\|_2^2=λ kR^2/P, and the columns of M are i.i.d. Gaussian vectors supported on the stiff block, so W is a uniformly random d-dimensional subspace of the k-dimensional stiff space (for d≤kd≤ k). For a fixed vector and a Haar d-subspace of ℝkR^k, ‖(I−PW)b‖22=(1−d/k)‖b‖22E\|(I-P_W)b\|_2^2=(1-d/k)\,\|b\|_2^2. Hence [ρ]=(1−dk)λkR2P=λ(k−d)R2P≤ 2ε⟺d≥k−2εPλR2.E[ρ]= (1- dk ) λ kR^2P= λ(k-d)R^2P\;≤\;2 d\;≥\;k- 2 Pλ R^2. Deterministic-equivalent route. Equation (B.3) gives d=kλ/(λ+κ)d=kλ/(λ+κ), i.e., κ=λ(k−d)/dκ=λ(k-d)/d, and the master formula (B.4) evaluates to [ρ]≈κλ+κ⋅kR2P=λk−dk⋅kR2P=λ(k−d)R2P,E[ρ]≈ κλ+κ· kR^2P=λ\, k-dk· kR^2P= λ(k-d)R^2P, in exact agreement. Solving (B.6) likewise returns diso⋆=[k−2εPλR2]+,d _iso= [k- 2 Pλ R^2 ]_+, with the resulting value rounded upward when an integer latent dimension is required. In this isotropic block model, the orientation-resolved master-formula prediction coincides with diso⋆d _iso. For comparison, the orientation-uniform dimension is dunif=kλR2λR2+2ε≈kd_unif= kλ R^2λ R^2+2 ≈ k when λR2≫ελ R^2 . The two specialized predictors diso⋆d _iso and dunifd_unif coincide in the strict-tolerance limit ε≪ε0:=λkR2/(2P) _0:=λ kR^2/(2P) and separate as ε approaches ε0 _0, where diso⋆→0d _iso→ 0 while dunifd_unif remains approximately k. B.6 Remarks Remark 9 (Ball localization is typically inactive). The least-squares minimizer z⋆z of (B.1) is expected to produce a parameter displacement Az⋆Az with norm comparable to the reference-to-solution displacement scale R in the regimes above; e.g., in the block model of Section B.5, one computes ‖Az⋆‖2=O(Rd/k)\|Az \|_2=O(R d/k) for the flat-direction component and O(R)O(R) overall. Thus, when the localization radius is chosen as Rloc=cR_loc=cR for a moderate constant c, the ball constraint is expected to be inactive. Empirically, ‖Δθ‖2\| θ\|_2 should be monitored during training as a check. Remark 10 (Rank saturation). Since b=H1/2Δ0∈range(H)b=H^1/2 _0 (H) always, taking d≥rank(H)d (H) drives ρ→0ρ→ 0 in the model; hence the zero-residual threshold is at most rank(H)rank(H), and (B.3) is used on the domain d<rank(H)d<rank(H). Remark 11 (Relation to the conic theorem). The event analyzed in this section is the quadratic-model analogue of the affine-slice intersection event in Theorem 2. More precisely, (B.2) gives the exact residual characterization for the unlocalized quadratic sublevel set S~ε S_ . When the corresponding least-squares solution also lies inside the localization ball, this event coincides with the localized intersection event used in the conic theorem. The conic theorem gives a rigorous transition window centered at the statistical-dimension quantity δ(C∘)δ(C ) associated with the localized cone C=cone((S~ε∩B¯(θ0,Rloc))−θ0).C=cone ( ( S_ ∩ B( _0,R_loc) )- _0 ). The three quantities derived in this section, d^MF,diso⋆,dunif=reff, d_MF, d _iso, d_unif=r_eff, should therefore be viewed as mean-level Hessian-based surrogates for this conic phase-transition center under different levels of displacement information. The orientation-resolved predictor d^MF d_MF retains the complete displacement profile Δi\ _i\ and is the primary prediction when that profile is available. The isotropic predictor diso⋆d _iso replaces the profile with the equal-energy model Δi2=R2/P _i^2=R^2/P, whereas dunif=reffd_unif=r_eff removes the orientation dependence through a uniform upper bound and requires only the displacement radius R. None of these mean-level predictors, by itself, inherits the high-probability guarantee of the conic theorem; their accuracy is tested empirically. Remark 12 (Experimental guidance). Phase-transition experiments should report all predictors supported by the available displacement information. When the complete displacement profile Δi\ _i\ is available or can be estimated reliably, d^MF d_MF is the primary prediction and should be compared directly with the measured critical dimension. The same experiments should also report diso⋆d _iso as a sharper diagnostic under the isotropic-orientation model and dunif=reffd_unif=r_eff as the conservative orientation-uniform, radius-only prediction. Under controlled isotropic displacements, the measured transition is expected to be close to diso⋆d _iso. As the displacement energy is increasingly concentrated along stiff curvature directions, d^MF d_MF should track the resulting change in the critical dimension, while dunifd_unif provides the orientation-uniform mean-level upper prediction. When only a reliable radius estimate R is available, dunifd_unif is the operational prediction used by Mode 2 of Algorithm 1.