Paper deep dive
Frozen Factor or Spectral Band? Disentangling Two Choices in Low-Rank LoRA
Adnan Slimane Ali, Ayoub Belfatmi, David Ngwe Pouth
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Spectral variants of low-rank adaptation (LoRA) choose both a subspace and which factor to freeze. We separate these choices by freezing the input factor A or output factor B on the top or bottom singular directions of pretrained weights, with learning rates selected separately. At rank 2, the same-band advantage of freezing A is larger than either within-factor band difference on all four task-model pairs with complete comparisons. Freezing B also trails comparable-budget free LoRA by 8-18 percentage points on five pairs spanning a formatting task and OpenBookQA. The A-frozen advantage persists in a single-GPU-model replication and within individual MLP module groups, including controls with equal or greater trainable counts for B frozen, and when A is frozen on a random orthonormal basis. The factor contrast weakens with rank. On OpenBookQA / Qwen2.5-1.5B at rank 16, PEFT's MiCA implementation trails comparable-budget LoRA by 3.08 points under a shared training recipe transferred from the MiCA paper. A trained oracle output subspace largely removes the low-rank deficit; partial warm-up gains recur across three direction seeds. The factor-versus-band ordering is descriptive; an approximate multiplicity audit weakens several earlier significance claims. These results extend known factor asymmetry by showing how its magnitude depends on spectral placement, rank and training conditions.
Tags
Links
- Source: https://arxiv.org/abs/2610.06621v1
- Canonical: https://arxiv.org/abs/2610.06621v1
Trouble viewing inline? Open PDF directly →
Full Text
113,685 characters extracted from source content.
Expand or collapse full text
Frozen Factor or Spectral Band? Disentangling Two Choices in Low-Rank LoRAThanks: Generative AI tools, including ChatGPT, assisted with drafting, editing, code review and analysis checks. The authors remain responsible for the experimental results, citations and final manuscript. Adnan Slimane Ali adnan.slimane-ali@student-cs.fr Affiliation: CentraleSupélec, Université Paris-Saclay Ayoub Belfatmi ayoub.belfatmi@polytechnique.edu Affiliation: École polytechnique, Institut Polytechnique de Paris David Ngwe Pouth david.ngwepouth@student-cs.fr Affiliation: CentraleSupélec, Université Paris-Saclay Abstract Spectral variants of low-rank adaptation (LoRA) choose both a subspace and which factor to freeze. We separate these choices by freezing the input factor A or output factor B on the top or bottom singular directions of pretrained weights, with learning rates selected separately. At rank 2, the same-band advantage of freezing A is larger than either within-factor band difference on all four task–model pairs with complete comparisons. Freezing B also trails comparable-budget free LoRA by 8–18 percentage points on five pairs spanning a formatting task and OpenBookQA. The A-frozen advantage persists in a single-GPU-model replication and within individual MLP module groups, including controls with equal or greater trainable counts for B frozen, and when A is frozen on a random orthonormal basis. The factor contrast weakens with rank. On OpenBookQA / Qwen2.5-1.5B at rank 16, PEFT’s MiCA implementation trails comparable-budget LoRA by 3.08 points under a shared training recipe transferred from the MiCA paper. A trained oracle output subspace largely removes the low-rank deficit; partial warm-up gains recur across three direction seeds. The factor-versus-band ordering is descriptive; an approximate multiplicity audit weakens several earlier significance claims. These results extend known factor asymmetry by showing how its magnitude depends on spectral placement, rank and training conditions. 1 Introduction LoRA adapts a pretrained weight matrix through a low-rank update BABA, where A maps the layer input into a small intermediate representation and B maps that representation to the output (Hu et al., 2022). Fixing one factor reduces the number of trainable parameters, but also fixes either the input or output subspace available to the update. A spectral construction makes another choice: which singular directions of the pretrained weight should define that subspace? Cross-method comparisons can entangle these choices. MiCA fixes the output factor on bottom singular directions (Rüdiger and Raschka, 2026); PiCa’s implementation fixes an input-side projector on top directions and shares the trainable output factor across layers (Hwang et al., 2026). These correspond to freezing B and A, respectively (Appendix C.3). Comparing them changes orientation, band and sharing at once. Their within-method band ablations, by contrast, hold orientation fixed. The asymmetry between LoRA factors is already known. Zhu et al. (2024) show that training B while fixing A is often more effective than the reverse, including experiments with spectral factors. LoRA-FA fixes A for memory efficiency (Zhang et al., 2023), and LoRA+ improves training by assigning a larger learning rate to B (Hayou et al., 2024a). The question we address is narrower: how large is the frozen-factor contrast relative to the top-versus-bottom spectral contrast, and how does that comparison change with rank? We use a two-by-two design: freeze A or B, each on the top or bottom band, while retaining the same pretrained model, data, adapted modules and training budget. Learning rates are selected separately for every configuration. We complement equal-rank comparisons with free LoRA at a nearby trainable-parameter count and a control that gives the B-frozen configuration more parameters than the A-frozen configuration. Figure 1 illustrates the main low-rank result. Figure 1: At rank 2, changing the frozen factor has a larger effect than switching spectral extremes. Points are mean test scores over five seeds at separately selected learning rates. The free-LoRA reference has rank 1 and a comparable, slightly larger trainable budget than rank-2 B-frozen LoRA. A-frozen LoRA has more trainable parameters at equal rank; Table 3 controls this advantage. The panels use different vertical scales. Lines connect configurations, not training trajectories. Full scores and uncertainty are in Tables 7 and 2. At rank 2, the factor contrast exceeds the band contrast on all four complete comparisons. The A-frozen advantage survives parameter-count and module-group controls and does not require a spectral input basis on the two pairs tested. The factor contrast decreases along the complete OpenBookQA rank sweep, becoming comparable to the band contrast at rank 16. Finally, a subspace extracted from trained LoRA greatly improves B-frozen performance, showing that its low-rank deficit depends on placement. Our contribution is to separate these design choices under optimization and budget controls and identify the limits of the resulting comparison. Factor asymmetry itself is not new. A follow-up evaluates PEFT’s MiCA implementation under a transferred recipe, without reproducing its original knowledge-acquisition benchmark or evaluating PiCa. 2 Experimental design 2.1 What is fixed when a factor is frozen? For a weight W0∈ℝdout×dinW_0 ^d_out× d_in, we use W=W0+sBA,A∈ℝr×din,B∈ℝdout×r,s=α/r=1.W=W_0+sBA, A ^r× d_in, B ^d_out× r, s=α/r=1. (1) The backbone remains fixed. With B frozen, every output update lies in col(B)col(B). With A frozen, the update can act only through the input features represented by row(A)row(A). These are different constraints, especially for rectangular layers. Let W0=UΣV⊤W_0=U V be its thin SVD, with m=min(dout,din)m= (d_out,d_in) singular values in decreasing order. The top band uses indices 1,…,r1,…,r and the bottom band uses m−r+1,…,m-r+1,…,m. No numerical-rank cutoff is applied. Thus “bottom” refers to the last directions returned by the thin SVD, not necessarily to strictly nonzero modes. The construction does not add null-space directions omitted by the thin decomposition. Table 1 defines the configurations. Singular vectors are used without singular-value scaling, and the trainable factor starts at zero. Every configuration therefore starts from the same effective weight W0W_0. The spectral vectors are computed by a full, non-randomized SVD in fp32 and cached in fp16. The latter makes orthonormality approximate in the implementation; Appendix C.2 describes the numerical checks. Configuration Initialization and constraint Trainable parameters/module B frozen, band S B=U:,SB=U_:,S fixed; train A from zero rdinrd_in A frozen, band S A=V:,S⊤A=V_:,S fixed; train B from zero rdoutrd_out Both trainable, band S Initialize B=U:,SB=U_:,S and A=0A=0; train both k(din+dout)k(d_in+d_out) Free LoRA Kaiming-initialized A, zero B; train both k(din+dout)k(d_in+d_out) Table 1: Controlled configurations. S is the top or bottom band. Frozen configurations use rank r; the two trainable configurations use either k=rk=r or a smaller rank giving a comparable parameter count. The spectral initialization of the both-trainable control does not subtract a component from W0W_0, and is therefore not a reproduction of PiSSA or MiLoRA. 2.2 Equal rank and comparable budget answer different questions An equal-rank comparison holds the maximum matrix rank fixed, but generally gives A-frozen and B-frozen adapters different parameter counts. Summed over all adapted modules, A-frozen adapters at the same rank contain 15.0% more parameters on Llama-3.2-1B and 26.8% more on Qwen2.5-1.5B. This follows from the models’ input and output widths, not from an additional learned component. For a second comparison, we select an integer rank k for free LoRA that gives a nearby trainable budget. At frozen rank 2, the reference has rank 1: its count exceeds the B-frozen count by 7.5% on Llama-3.2-1B and 13.4% on Qwen2.5-1.5B. At frozen rank 16, Qwen uses a rank-7 reference, within about 2.5% of the B-frozen count on the two Qwen sizes. Appendix C gives the counts. We call this a comparable-budget reference, rather than an exact budget match. It changes both the accessible update space and rank, so its gap cannot identify the isolated causal effect of freezing a factor. Reference choice matters: on Format / Llama-3.2-1B, equal-rank free LoRA scores 80.33% over five seeds with 2.15 times the B-frozen count (supplementary record, Table S14). The top-band deficit is then 15.90 points, versus 12.41 against the rank-1 reference. To test whether parameter count alone explains the A-frozen advantage, we also compare A frozen at rank 2 against B frozen at rank 3. This gives B frozen the larger count on two models. We do not infer memory, wall-clock or total-compute equivalence from trainable counts. 2.3 Tasks, models and optimization The central rank-2 results cover five task–model pairs: a synthetic formatting task on Llama-3.2-1B and Qwen2.5-1.5B, and OpenBookQA on Qwen2.5-1.5B, Qwen2.5-7B and Mistral-7B-v0.3. Both frozen orientations are evaluated on four pairs; the formatting experiment on Qwen2.5-1.5B has no A-frozen arm. ARC-Easy and ARC-Challenge provide additional benchmark checks. The complete A/B rank sweep covers ranks 2, 4, 8 and 16 on OpenBookQA with Qwen2.5-1.5B, with rank-16 checks on other pairs. Rank-32 results on ARC-Challenge concern B-frozen configurations only. The formatting task transforms CoNLL-2003 sentences into an invented structured output (Tjong Kim Sang and De Meulder, 2003). Its transformations include rewriting person names, sorting lists, spelling out an entity count and constructing an entity-initial signature. A strict verifier scores whether every required rule is satisfied on 614 test items. This is a measure of learning a structured output transformation, not a general reasoning benchmark. OpenBookQA and ARC use multiple-choice accuracy (Mihaylov et al., 2018; Clark et al., 2018). Each answer option is scored by mean conditional log-likelihood per token; the highest-scoring option is selected. OpenBookQA uses 500 test questions. Unless stated otherwise, experiments adapt the query, key, value and output projections and the MLP gate, up and down projections in every transformer layer. The standard budget is 600,000 supervised tokens, with AdamW, zero weight decay, gradient clipping at 1.0 and a cosine schedule. The effective batch is 32 sequences of at most 384 tokens. Formatting loss covers the target only; the default OpenBookQA and ARC loss covers packed question–answer text. An answer-only control is reported separately. Identical token budgets imply different numbers of steps and epochs across tasks (Appendix D). Learning rates are selected independently for each configuration on validation scores, normally using seeds 0 and 1, and test results are then reported on seeds 2–6. Rates span 10−410^-4 to 10−210^-2 in the main grid, with extensions when a maximum occurs at an edge. The formatting replication on Qwen uses reporting seeds 22–26. New module-group controls use eight reporting seeds, 2–9, and the MiCA follow-up uses a shared transferred recipe rather than separately tuned rates. Selection exceptions, residual grid-boundary cases and hardware differences are documented in Appendix D.3. Separate tuning reduces a known evaluation bias (Lee et al., 2026); it does not establish that every optimizer configuration has reached its global optimum. 2.4 Contrasts and uncertainty Write atop,abottoma_top,a_bottom for mean scores with A frozen and btop,bbottomb_top,b_bottom for scores with B frozen. To summarize the two-by-two comparison, we report F F =12[(atop−btop)+(abottom−bbottom)], = 12[(a_top-b_top)+(a_bottom-b_bottom)], (2) S S =12[(abottom−atop)+(bbottom−btop)]. = 12[(a_bottom-a_top)+(b_bottom-b_top)]. (3) F is the factor contrast averaged across the two bands; S is the signed band contrast averaged across the two frozen orientations. These are summaries of the tested configurations, not a decomposition of variance or a parameter-adjusted causal model. We also inspect each same-band factor contrast and each within-factor band difference, because averaging can hide interactions. Scores and differences in the tables use percent and percentage points, respectively. Most uncertainty estimates pair runs by training seed and condition on the fixed test set. The post hoc F,SF,S summaries use percentile-bootstrap 95% intervals over five seeds; these intervals are approximate and can be too narrow. Direct controls report paired-t intervals and, where specified, a Bonferroni decision rule within the two-band family. Displayed 95% intervals are nominal unless explicitly stated otherwise. They do not include uncertainty from validation-based learning-rate selection or from drawing a new test sample. Many campaign comparisons were specified in internal version-control records before their corresponding runs, after earlier experiments had already informed the research program. We call these prespecified campaign comparisons, not externally preregistered tests. The supplementary record dates these protocols and lists every historical test with its status. The factor-versus-band decomposition is post hoc, and the rank trend is reported descriptively. An approximate post hoc Holm audit of 145 historical test rows retains 46 of the 109 previously positive decisions. Some rows combine multiple contrasts or reconstruct p-values from bootstrap intervals, so this is a sensitivity analysis, not a certification of study-wide error control. In particular, the rank-2 OpenBookQA / Qwen2.5-1.5B direct A/B row no longer passes (adjusted p=0.062p=0.062). We retain the effect estimates and state campaign-specific decision rules explicitly. Appendix G.2 details the audit; Appendix G explains the older recovery statistic. 3 At rank 2, factor orientation has the larger contrast Across the four complete rank-2 comparisons, freezing A rather than B improves mean performance by 6.6–19.0 percentage points on the same band. By comparison, switching between the top and bottom bands changes the score by at most 5.2 points within either orientation. This ordering holds within each of the four pairs; it is not an artifact of averaging positive and negative band effects. The corresponding averaged contrasts are shown in Table 2 and Figure 2. Task / model Factor F Band S Interaction Format / Llama-3.2-1B 15.9615.96 −2.21-2.21 5.995.99 [15.13,16.86][15.13,16.86] [−3.24,−1.22][-3.24,-1.22] [3.36,8.60][3.36,8.60] OpenBookQA / Qwen2.5-1.5B 8.428.42 −2.46-2.46 3.723.72 [7.20,9.64][7.20,9.64] [−3.60,−1.60][-3.60,-1.60] [2.24,4.80][2.24,4.80] OpenBookQA / Qwen2.5-7B 11.7811.78 −3.46-3.46 2.602.60 [11.34,12.38][11.34,12.38] [−3.78,−3.14][-3.78,-3.14] [1.48,3.64][1.48,3.64] OpenBookQA / Mistral-7B 11.9811.98 −1.02-1.02 −2.28-2.28 [11.28,13.04][11.28,13.04] [−2.34,0.44][-2.34,0.44] [−3.16,−1.40][-3.16,-1.40] Table 2: Factor and band contrasts at rank 2, in percentage points. F and S are defined in Eqs. (2)–(3). The interaction is (abottom−atop)−(bbottom−btop)(a_bottom-a_top)-(b_bottom-b_top). Brackets are post hoc paired-bootstrap 95% intervals over five seeds, conditional on the test set and selected rates. They are not simultaneous confidence intervals, and no confirmatory test of F>|S|F>|S| is claimed. All four A/B configurations use the same rank; their parameter counts differ. Against comparable-budget free LoRA, the top- and bottom-band B-frozen configurations lose 8–18 points on the five central pairs (Table 7). On OpenBookQA with Qwen2.5-1.5B, the bottom-band B-frozen score is 42.20%, equal to the unadapted model’s score, while free LoRA reaches 54.56%. On the formatting task, the base model never produces a fully valid output, so the gap measures how well each adapter learns the transformation. ARC-Challenge is a useful boundary case: at rank 2, the corresponding B-frozen deficits are only 2.0 and 3.8 points, alongside a smaller gain from fine-tuning itself. 3.1 The parameter-count advantage does not explain the ordering Table 3 reverses the trainable-count advantage. B frozen at rank 3 has 18–22% more trainable parameters than A frozen at rank 2, yet trails it by 6.3–12.2 points across the two models and bands. Every contrast clears the prespecified two-band threshold. This rules out a simple explanation based on the total trainable count in these OpenBookQA experiments. It does not equalize allocation across module types or isolate input/output geometry. Model Band Extra count A(r=2r=2)−-B(r=3r=3) t Qwen2.5-1.5B Top 18.3%18.3\% 6.28[4.48,8.08]6.28\ [4.48,8.08] 9.719.71 Bottom 18.3%18.3\% 9.20[6.68,11.72]9.20\ [6.68,11.72] 10.1410.14 Qwen2.5-7B Top 22.2%22.2\% 9.00[8.26,9.74]9.00\ [8.26,9.74] 33.5433.54 Bottom 22.2%22.2\% 12.24[10.74,13.74]12.24\ [10.74,13.74] 22.5922.59 Table 3: Giving B frozen more parameters does not close the gap. OpenBookQA, seeds 2–6, separately selected learning rates. Differences are in percentage points with nominal paired-t 95% intervals. The prespecified decision threshold is t>3.495t>3.495 with four degrees of freedom, corresponding to a two-sided Bonferroni correction for two bands at family level 0.05. Exact counts and scores are in Appendix C. Spectral placement of A. Freezing A does not require a spectral basis in the two pairs tested. In a prespecified replication on new runs, a random orthonormal input basis beats random orthonormal B frozen at rank 2 by 12.08 points on Format / Llama-3.2-1B ([10.62,13.55][10.62,13.55], t=22.86t=22.86) and 8.64 on OpenBookQA / Qwen2.5-1.5B ([7.44,9.84][7.44,9.84], t=20.06t=20.06). Both pass the two-pair Bonferroni rule; displayed intervals are nominal 95%. Across these cells and OpenBookQA at rank 16, descriptive differences range from −0.75-0.75 to +2.80+2.80 points against spectral A frozen and from 0.00 to +0.59+0.59 against the free references. These are not equivalence tests. Earlier runs had already suggested the outcome; Appendix E.5 discloses their use and the protocol deviations. 3.2 The advantage persists within MLP module groups The preceding count control still combines modules with different shapes. We therefore repeat the comparison while adapting only the MLP down projections, or only its gate and up projections, on Format / Llama-3.2-1B and OpenBookQA / Qwen2.5-1.5B. All runs use RTX 4000 Ada GPUs, separately selected learning rates and eight reporting seeds. Within down projections, freezing A at rank 2 gives fewer trainable parameters than freezing B at rank 2, yet scores higher on both bands. Within gate and up projections, we raise the B-frozen rank to 8 on Llama and 12 on Qwen: its count then equals the A-frozen count on Llama and exceeds it by 2.9% on Qwen. All eight A-minus-B intervals remain above zero after Bonferroni correction (Table 4). Neither total count nor allocation across module types therefore explains the ordering, though geometry can affect its magnitude. In the formatting down projections, B-frozen scores fall to 29.38 and 12.30%; this cannot separate restricted capacity from optimization failure. Task / model Adapted modules A rank / B rank Top A−-B Bottom A−-B Format / Llama-1B Down 2 / 2 36.54[32.34,40.75]36.54\ [32.34,40.75] 55.46[52.19,58.72]55.46\ [52.19,58.72] Gate + up 2 / 8 12.56[10.59,14.53]12.56\ [10.59,14.53] 10.04[7.33,12.74]10.04\ [7.33,12.74] OBQA / Qwen-1.5B Down 2 / 2 3.30[1.76,4.84]3.30\ [1.76,4.84] 8.88[7.03,10.72]8.88\ [7.03,10.72] Gate + up 2 / 12 4.48[2.19,6.76]4.48\ [2.19,6.76] 6.00[4.30,7.70]6.00\ [4.30,7.70] Table 4: Within-module comparisons favor A frozen. Differences in percentage points, seeds 2–9. Each interval has level 1−0.05/8=99.375%1-0.05/8=99.375\%, giving 95% family coverage for the eight prespecified paired-t comparisons (t1−0.05/16,7=3.855t_1-0.05/16,7=3.855). B frozen has at least as many trainable parameters in every row. Appendix E.2 gives counts, scores and selection exceptions, including an A-frozen boundary optimum. 3.3 Training controls and remaining optimization effects Three controls test narrower alternatives. On Llama’s formatting task, quadrupling the supervised-token budget and retuning every learning rate reduces the cost of fixing B, but leaves equal-rank deficits of 12.0 and 18.2 points relative to training both factors. A fresh five-seed replication also retains the large gap; its factor contrast is 15.21 points versus an absolute band contrast of 2.96 points. On OpenBookQA with Qwen2.5-1.5B, supervising answers only preserves A-frozen advantages of 7.2 and 9.7 points. This last control holds the number of optimizer steps and the packed sequences fixed, while reducing supervised tokens by about sixfold; it tests loss masking rather than an equal-supervision training regime. Appendix E gives the measurements. Making both factors trainable from the same spectral initialization also restores most of the performance, including at the comparable-budget rank. Once B is trainable, however, it can leave the initial band. The disappearance of a band difference in this control is therefore expected and is not independent evidence that the original band was irrelevant. The data also leave room for optimization effects. On the formatting task, fixed singular-value scaling improves the top-band B-frozen score by 3.61 points, and a fixed scalar per module improves it by 2.31 points without changing the accessible subspace. The scalar is determined from the pretrained spectrum; it is not an extra learned parameter. Loss curves continue to decrease at the original training endpoint. These observations prevent interpreting the remaining gap as a measured approximation-error floor. A separate replication reruns all seven central formatting configurations on RTX 4000 Ada at their original selected rates. The A-frozen advantages are 13.16 and 19.67 points (top and bottom), and the free-LoRA gaps are 12.74 and 18.66 points. All retain their campaign decisions. This removes differences in GPU model as an explanation for this replication, while leaving the historical scores unchanged (Appendix E.1). 4 The factor contrast weakens with rank The rank sweep on OpenBookQA with Qwen2.5-1.5B provides the clearest test of the result’s range. The averaged factor contrast decreases from 8.42 points at rank 2 to 2.58 points at rank 16, while the absolute band contrast is 3.02 points at rank 16 (Figure 2). At that rank, the two contrasts are comparable and the averaged band contrast is slightly larger. The top-band A-frozen advantage is only 1.36 points and does not meet its campaign’s significance criterion. The bottom-band advantage remains 3.80 points. Figure 2: Factor and band contrasts become comparable at higher rank. Left: the rank-2 factor contrast exceeds the absolute band contrast on each complete task–model pair. Right: the factor contrast decreases along the OpenBookQA / Qwen2.5-1.5B sweep; the two estimates are close at rank 16. Values are descriptive contrasts of mean scores, without a test for their difference or for a crossover rank. The right panel does not imply a universal rank threshold. Tables 2 and 5 supply the values. A frozen minus B frozen Free LoRA minus B frozen Frozen rank Top Bottom Top Bottom F |S||S| 2 6.566.56 10.2810.28 8.048.04 12.3612.36 8.428.42 2.462.46 4 7.007.00 8.648.64 8.808.80 10.0010.00 7.827.82 0.380.38 8 6.366.36 6.886.88 5.525.52 8.888.88 6.626.62 3.103.10 16 1.361.36 3.803.80 2.082.08 6.326.32 2.582.58 3.023.02 Table 5: Rank dependence on OpenBookQA / Qwen2.5-1.5B. Differences in percentage points, seeds 2–6. The free reference uses ranks 1, 2, 4 and 7, respectively; its trainable count is 13.4% above B frozen at ranks 2–8 and 0.8% below at rank 16. F and S summarize the four frozen configurations. The top-band same-band contrast and reference gap rise slightly at rank 4, so strict monotonicity does not hold for every quantity. Scores and available direct-contrast intervals appear in Appendix B. This pattern is not universal across models. At rank 16 on OpenBookQA with Qwen2.5-7B, the factor contrast is 4.00 points and the absolute band contrast is 1.64 points. Both same-band A-frozen advantages remain positive under the specified tests. On ARC-Challenge with Qwen2.5-1.5B, the rank-16 top-band B-frozen configuration slightly exceeds free LoRA, whereas the bottom band trails by 5.00 points. At rank 32 on that task, the two B-frozen spectral scores differ by only 0.66 points; no A-frozen arm was evaluated there, so this observation cannot locate a factor-versus-band crossover. Across six task–model pairs at rank 16, the bottom-band B-frozen configuration trails the comparable-budget free reference by 1.48–6.51 points. These are descriptive gaps, not six independently confirmed deficits (Table 11). The rank-4 and rank-8 A/B comparisons come from only one task–model pair. Together, the evidence supports a strong low-rank effect whose magnitude depends on task and model, rather than a fixed ranking over all ranks. 4.1 MiCA under a transferred training recipe Does the rank-16 bottom-band deficit depend on our training recipe? We evaluate PEFT 0.21.0’s MiCA implementation on OpenBookQA / Qwen2.5-1.5B, transferring the learning rate and four-epoch schedule of MiCA’s rank-16 BLOGS configuration (Rüdiger and Raschka, 2026). We retain our data, 384-token packing, scoring rule and seven adapted module types. This tests MiCA in our setting, rather than reproducing the original benchmark. MiCA reaches 52.40%, 2.56 points above our bottom-band arm, but trails a fresh comparable-budget free-LoRA run by 3.72 points (Table 6). That comparison changes both recipe and training amount. After observing it, we specify and run a second test with free LoRA under the same transferred recipe as MiCA. Free LoRA still leads by 3.08 points, with a nominal paired-t 95% interval of [1.34,4.82][1.34,4.82]. The two methods then use the same 417,792 supervised tokens and 272 optimizer steps; free LoRA has 0.79% fewer trainable parameters. This follow-up supports a deficit under one shared recipe. It does not establish the best attainable performance of either method or reproduce MiCA’s acquisition–retention comparison. Appendix C.4 gives the settings and the adaptive sequence of tests. Configuration Training recipe Accuracy (%) Supervised tokens Steps Bottom B frozen, r=16r=16 Main 49.84 602,112 49 Free LoRA, r=7r=7 Main 56.12 602,112 49 PEFT MiCA, r=16r=16 Transferred 52.40 417,792 272 PEFT free LoRA, r=7r=7 Transferred 55.48 417,792 272 Table 6: MiCA and comparable-budget LoRA on OpenBookQA / Qwen2.5-1.5B. Means over seeds 2–6, all on RTX 4000 Ada. The free-LoRA-minus-MiCA contrasts are 3.72[1.58,5.86]3.72\ [1.58,5.86] points across recipes (t=4.83t=4.83) and 3.08[1.34,4.82]3.08\ [1.34,4.82] under the shared recipe (t=4.90t=4.90). Intervals are nominal 95%, conditional on the fixed test set. The final row was added after the first comparison and reuses its MiCA runs. 5 Can a frozen output subspace support good adaptation? The preceding comparisons could suggest that fixing B is inherently ineffective. A simple oracle experiment tests that interpretation. We first train a free LoRA at rank 2 on the same training data. For each module, we extract an orthonormal basis Q of its learned output-factor space, freeze a new adapter’s B on Q, and train its A from zero. This is an oracle with respect to access to a trained subspace, not access to test labels. The reason this is a useful diagnostic is algebraic. If the trained teacher has factors B∗,A∗B^*,A^* and B∗=QMB^*=QM, then B∗A∗=Q(MA∗).B^*A^*=Q(MA^*). (4) Thus the teacher’s update is representable with B=QB=Q fixed. This identity is not a new capacity theorem. The empirical question is whether retraining the remaining factor from zero finds a useful solution, and how informative the result is about cheaper choices of Q. On OpenBookQA with Qwen2.5-1.5B, oracle directions from two independently trained teachers yield 55.88% and 55.28% accuracy, compared with 46.52% for the top-band B-frozen configuration and 54.84% for a free rank-1 reference retrained in the diagnostic campaign. On Llama’s formatting task, one teacher basis gives 80.81%, close to the five-seed free-LoRA mean of 80.33% at rank 2. These results show that a fixed output basis can support high performance in the tested settings. They do not prove that optimization difficulty is absent for arbitrary bases. 5.1 Which placements are available before training? We compare the spectral extremes with an independently drawn random orthonormal output basis and the top two left singular vectors of the gradient of the training loss with respect to W0W_0. The gradient is estimated either from 64 training sequences or from the entire training set. All resulting adapters keep B fixed and train A; they do not implement LoRA-GA, LoRA-One or LoRA-SB. Neither random nor initial-gradient placement removes the observed deficit (Table 19). Accumulating the gradient over all training sequences does not improve this conclusion on OpenBookQA. On the formatting task, its gain over the top spectral band is 1.79 points, with a nominal 95% interval of [−0.84,4.42][-0.84,4.42]. This excludes the large recovery seen with the oracle in this experiment, but should not be read as a statement about all gradient-derived subspaces, ranks or optimizers. Figure 3: Training-derived output directions are more useful in these diagnostics. Gains relative to top-band B frozen, at rank 2, with nominal paired-t 95% intervals over five retraining seeds. Dashed lines mark the earlier free-LoRA gaps used as fixed campaign reference values: 8.04 and 12.41 points. Teacher-subspace uncertainty is not included. On the formatting task, the 64-sequence gradient and oracle use a retrained top-band reference, whereas the full-gradient and warm-up comparisons use the original reference; the two reference scores differ by 0.13 points. Full details are in Appendix F. 5.2 How early do useful directions emerge? A short warm-up of a free rank-2 LoRA provides an intermediate diagnostic. After a chosen fraction of training steps, we freeze its output basis and retrain A from zero over the remaining steps. The total number of steps is unchanged, but both factors train during warm-up. The original direction seed and two new seeds give the same qualitative pattern: longer warm-up produces larger mean gains at the tested 5%, 10% and 25% cuts (Figure 4). At 10%, gains over top-band B frozen span 4.12–4.64 points on OpenBookQA and 6.09–7.79 points on the formatting task. Each of the four new comparisons passes the Bonferroni correction for that family. Nevertheless, none of their corrected intervals establishes recovery of at least 70% of the fixed earlier free-LoRA gap. Appendix F.3 gives the conditional intervals and references. Figure 4: Warm-up gains recur across three direction seeds. Each point averages five retraining seeds for one learned output basis. Recovery is the gain over the original top-band B-frozen reference divided by its fixed free-LoRA gap (8.04 points on OpenBookQA; 12.41 on Format). Lines connect three tested cuts; they are not a fitted learning curve. The pattern is descriptive, and values above one can arise because the reference uses rank 1 while warm-up uses rank 2. Corrected intervals for the new 10% comparisons appear in Table 20. The full oracle requires a separate rank-2 teacher run followed by adapter retraining. Warm-up keeps the step count fixed but changes which factors train during those steps. Neither is a demonstrated gain in total compute or memory efficiency. Both show that the choice of fixed output subspace can matter much more than the difference between the two spectral extremes. 6 Related work Factor asymmetry and freezing. Zhu et al. (2024) are the closest precedent: they compare the two frozen orientations, study orthonormal factors and report a top-spectral B-frozen result essentially equal to a random B-frozen result on GLUE at rank 16. LoRA-FA (Zhang et al., 2023) and LoRA+ (Hayou et al., 2024a) also establish that the factors have distinct practical roles; initialization changes their optimization dynamics (Hayou et al., 2024b). Kratsios et al. (2025) study generalization of asymmetric adapters with frozen random factors. We extend the empirical comparison to both spectral extremes, separately tuned rates, very low ranks and controls that reverse the parameter-count advantage. We do not treat tuning or frozen-factor asymmetry themselves as new contributions. Pretrained spectral structure. PiSSA and MiLoRA initialize trainable low-rank factors from different parts of the pretrained spectrum (Meng et al., 2024; Wang et al., 2024a). MiCA keeps its output subspace fixed (Rüdiger and Raschka, 2026); PiCa uses a fixed input-side projector in its implementation together with parameter sharing (Hwang et al., 2026). LoRA-XS learns a small core within a spectral construction (Bałazy et al., 2024), while SVFT learns sparse coefficients between pretrained singular vectors (Lingam et al., 2024). Quercia et al. (2026) already show the value of intermediate components for performance–forgetting trade-offs in spectral initialization. Our interior-band observation is a secondary result about a frozen output subspace, not the discovery of intermediate spectral components. In speech adaptation, SSVD also keeps output-associated singular vectors fixed (Wang et al., 2025), underscoring the need to limit general claims about the cost of this constraint. Data-dependent and changing subspaces. LoRA-GA and LoRA-One use gradient information to initialize adapters whose factors remain trainable (Wang et al., 2024b; Zhang et al., 2025). LoRA-SB instead trains a small core between fixed factors obtained from an approximation to an initial update (Ponkshe et al., 2024). EVA and CorDA use activation or context information (Paischer et al., 2025; Yang et al., 2024). TLoRA constructs a task-aware input subspace from weights and input covariance, fixes A, and trains B with additional rank and scale allocation (Lin et al., 2026). These constructions differ from fixing only the leading left singular vectors of our initial-gradient estimate. GaLore refreshes gradient-projection subspaces during training (Zhao et al., 2024); Flora interprets low-rank adapters as gradient compressors (Hao et al., 2024); VeRA learns scaling vectors around shared random factors (Kopiczko et al., 2024). Our negative gradient diagnostic does not evaluate these complete methods. Recent work on factor-constrained continual learning likewise studies how factor constraints translate into effective updates, but targets historical-feature protection rather than our single-task top-versus-bottom comparison (Wang et al., 2026). Evaluation and optimization. Lee et al. (2026) show that separately tuning learning rates can remove apparent advantages of several LoRA variants. Our own selections illustrate the issue. On Format / Llama-3.2-1B at rank 2, bottom-band B frozen selects 5⋅10−35· 10^-3, half the top band’s 10−210^-2 (Table 14). It also selects a lower rate on OpenBookQA at ranks 4 and 8 on Qwen2.5-1.5B and rank 16 on Qwen2.5-7B. Historical common-rate comparisons can even reverse the ranking (Appendix D.3). Lee and Lee (2026) identify batch size as another important evaluation axis. Our low-rank B-frozen gaps survive the learning-rate search used here, but batch size remains fixed and not every training configuration is shown to converge. The conclusion concerns the reported training budgets and hyperparameters. 7 Discussion and limitations What the comparison identifies. At rank 2, the factor contrast exceeds the top/bottom contrasts on all four complete pairs. This ordering does not extend to arbitrary subspace placement, as the oracle shows. Equal-rank and comparable-budget comparisons control different quantities. Count and module-group controls support the A-frozen advantage, but scalar rescaling and longer training show that optimization also contributes to the measured deficits. Scope. The strongest evidence concerns Format and OpenBookQA. The larger models are tested on OpenBookQA, Mistral only at rank 2; complete rank-4 and rank-8 comparisons cover one pair. ARC-Challenge shows different higher-rank behavior. We establish neither a universal rank threshold nor a ranking of complete MiCA and PiCa methods. The MiCA follow-up covers one model, task and transferred recipe, without assessing acquisition–forgetting trade-offs. Uncertainty and selection. Five reporting seeds give limited precision for small effects and ratios. Intervals condition on test items; a seed-and-item bootstrap widened selected Format intervals by 1.8–2.6 times. OpenBookQA lacks the retained predictions for that analysis. Corrections apply within campaigns; the approximate 145-row Holm audit reduces historical positive decisions and excludes the new experiments. The decomposition remains exploratory. Retrospective power calculations often require 3–5-point band differences for 80% detection probability; non-significance does not establish equivalence. Oracle and warm-up intervals condition on each learned basis, and random-basis variability is not isolated from retraining randomness. Rate selection on two seeds adds uncertainty absent from these intervals. The random-A replication was prespecified but not blind to earlier outcomes. Hardware and reproducibility. Of 137 historical score cells, 86 use RTX 4000 Ada only and 51 mix GPU models; six individual GPU records are missing. The seven-arm Format replication uses RTX 4000 Ada throughout, strengthening that result without balancing other historical campaigns. Protocol timestamps establish chronology within the repository, not external registration. Exact reproduction requires the original data-building, training and evaluation artifacts in addition to the aggregates. 8 Conclusion At rank 2, the frozen-factor contrast exceeds the contrast between spectral extremes. The A-frozen advantage survives count and module controls and, on two pairs, a random input basis. It weakens with rank and varies by task and model. Learned output bases greatly improve B-frozen performance. Across complete all-module comparisons at ranks 2–8, means favor A frozen on both bands; free LoRA scores higher or within 1.4 points. Comparisons should report orientation, basis construction and scale, tuned hyperparameters, trainable counts and subspace-learning cost. References Bałazy et al. (2024) Klaudia Bałazy, Mohammadreza Banaei, Karl Aberer, and Jacek Tabor. LoRA-XS: Low-rank adaptation with extremely small number of parameters. arXiv preprint arXiv:2405.17604, 2024. Clark et al. (2018) Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. Grattafiori et al. (2024) Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Hao et al. (2024) Yongchang Hao, Yanshuai Cao, and Lili Mou. Flora: Low-rank adapters are secretly gradient compressors. In International Conference on Machine Learning, 2024. arXiv:2402.03293. Hayou et al. (2024a) Soufiane Hayou, Nikhil Ghosh, and Bin Yu. LoRA+: Efficient low rank adaptation of large models. In International Conference on Machine Learning, 2024. arXiv:2402.12354. Hayou et al. (2024b) Soufiane Hayou, Nikhil Ghosh, and Bin Yu. The impact of initialization on LoRA finetuning dynamics. In Advances in Neural Information Processing Systems, 2024. arXiv:2406.08447. He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision, 2015. arXiv:1502.01852. Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. arXiv:2106.09685. Hwang et al. (2026) Junseo Hwang, Wonguk Cho, and Taesup Kim. PiCa: Parameter-efficient fine-tuning with column space projection. In International Conference on Learning Representations, 2026. arXiv:2505.20211. Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, et al. Mistral 7B. arXiv preprint arXiv:2310.06825, 2023. Kopiczko et al. (2024) Dawid J. Kopiczko, Tijmen Blankevoort, and Yuki M. Asano. VeRA: Vector-based random matrix adaptation. In International Conference on Learning Representations, 2024. arXiv:2310.11454. Kratsios et al. (2025) Anastasis Kratsios, Tin Sum Cheng, Aurelien Lucchi, and Haitz Sáez de Ocáriz Borde. Sharp generalization bounds for foundation models with asymmetric randomized low-rank adapters. arXiv preprint arXiv:2506.14530, 2025. Lee and Lee (2026) Sangyoon Lee and Jaeho Lee. Beware of the batch size: Hyperparameter bias in evaluating LoRA. arXiv preprint arXiv:2602.09492, 2026. Lee et al. (2026) Yu-Ang Lee, Ching-Yun Ko, Pin-Yu Chen, and Mi-Yen Yeh. Learning rate matters: Vanilla LoRA may suffice for LLM fine-tuning. In ICML 2026 Workshop on High-dimensional Learning Dynamics (HiLD), 2026. https://openreview.net/forum?id=fQ79zHBsjm. Lin et al. (2026) Weicheng Lin, Yi Zhang, Jiawei Dang, and Liang-Jie Zhang. TLoRA: Task-aware low rank adaptation of large language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 29252–29269, 2026. doi:10.18653/v1/2026.acl-long.1348. Lingam et al. (2024) Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, and Sujay Sanghavi. SVFT: Parameter-efficient fine-tuning with singular vectors. arXiv preprint arXiv:2405.19597, 2024. Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. arXiv:1711.05101. Meng et al. (2024) Fanxu Meng, Zhaohui Wang, and Muhan Zhang. PiSSA: Principal singular values and singular vectors adaptation of large language models. In Advances in Neural Information Processing Systems, volume 37, 2024. arXiv:2404.02948. Mihaylov et al. (2018) Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Conference on Empirical Methods in Natural Language Processing, 2018. Paischer et al. (2025) Fabian Paischer, Lukas Hauzenberger, Thomas Schmied, Benedikt Alkin, Marc Peter Deisenroth, and Sepp Hochreiter. Parameter efficient fine-tuning via explained variance adaptation. In Advances in Neural Information Processing Systems, 2025. arXiv:2410.07170v5. Ponkshe et al. (2024) Kaustubh Ponkshe, Raghav Singhal, Eduard Gorbunov, Alexey Tumanov, Samuel Horváth, and Praneeth Vepakomma. Initialization using update approximation is a silver bullet for extremely efficient low-rank fine-tuning. arXiv:2411.19557, 2024. Quercia et al. (2026) Alessio Quercia, Arya Bangun, Ira Assent, and Hanno Scharr. Least but not last: Fine-tuning intermediate principal components for better performance-forgetting trade-offs. arXiv preprint arXiv:2602.03493, 2026. Qwen Team (2024) Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. Rüdiger and Raschka (2026) Sten Rüdiger and Sebastian Raschka. MiCA learns more knowledge than LoRA and full fine-tuning. arXiv preprint arXiv:2604.01694, 2026. Tjong Kim Sang and De Meulder (2003) Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147, 2003. Wang et al. (2024a) Hanqing Wang, Zeguan Xiao, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. MiLoRA: Harnessing minor singular components for parameter-efficient LLM finetuning. arXiv preprint arXiv:2406.09044, 2024. Wang et al. (2024b) Shaowen Wang, Linxi Yu, and Jian Li. LoRA-GA: Low-rank adaptation with gradient approximation. In Advances in Neural Information Processing Systems, 2024. arXiv:2407.05000. Wang et al. (2025) Pu Wang, Shinji Watanabe, and Hugo Van hamme. SSVD: Structured SVD for parameter-efficient fine-tuning and benchmarking under domain shift in ASR. arXiv preprint arXiv:2509.02830, 2025. Wang et al. (2026) YongShun Wang, JianLin Su, and Yong Ma. Bilinear optimization divergence: Diagnosing factor-constrained LoRA continual learning. arXiv preprint arXiv:2609.23594v1, 2026. Yang et al. (2024) Yibo Yang, Xiaojie Li, Zhongzhu Zhou, Shuaiwen Leon Song, Jianlong Wu, Liqiang Nie, and Bernard Ghanem. CorDA: Context-oriented decomposition adaptation of large language models for task-aware parameter-efficient fine-tuning. In Advances in Neural Information Processing Systems, 2024. arXiv:2406.05223. Zhang et al. (2023) Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. LoRA-FA: Memory-efficient low-rank adaptation for large language models fine-tuning. arXiv preprint arXiv:2308.03303v1, 2023. Zhang et al. (2025) Yuanhe Zhang, Fanghui Liu, and Yudong Chen. LoRA-One: One-step full gradient could suffice for fine-tuning large language models, provably and efficiently. In Proceedings of the 42nd International Conference on Machine Learning, PMLR 267, 2025. arXiv:2502.01235. Zhao et al. (2024) Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. GaLore: Memory-efficient LLM training by gradient low-rank projection. In International Conference on Machine Learning, 2024. arXiv:2403.03507. Zhu et al. (2024) Jiacheng Zhu, Kristjan Greenewald, Kimia Nadjahi, Haitz Sáez de Ocáriz Borde, Rickard Brüel Gabrielsson, Leshem Choshen, Marzyeh Ghassemi, Mikhail Yurochkin, and Justin Solomon. Asymmetry in low-rank adapters of foundation models. In Proceedings of the 41st International Conference on Machine Learning, 2024. arXiv:2402.16842. Appendix A Complete scores for the central comparisons Table 7 gives the mean scores underlying the rank-2 claims. Table 8 gives the controls that train both factors from a spectral initialization. The two tables use the same reporting seeds and selected-rate convention. A dash denotes an unrun configuration, not a zero score. Scores below are percentages; differences elsewhere are percentage points. Contrasts computed from the displayed means can differ by 0.01 points from contrasts computed before rounding. B frozen, r=2r=2 A frozen, r=2r=2 Free Base Task / model Top Bottom Top Bottom r=1r=1 Format / Llama-3.2-1B 64.43 59.22 77.39 78.18 76.84 0.00 Format / Qwen2.5-1.5B 62.57 63.91 — — 78.79 0.00 OpenBookQA / Qwen2.5-1.5B 46.52 42.20 53.08 52.48 54.56 42.20 OpenBookQA / Qwen2.5-7B 53.16 48.40 63.64 61.48 62.84 47.40 OpenBookQA / Mistral-7B 47.64 47.76 60.76 58.60 60.12 44.60 Table 7: Rank-2 mean test scores (%). Reporting seeds are 2–6, except 22–26 for Format / Qwen2.5-1.5B. The metric is valid-format accuracy or multiple-choice accuracy according to task. Base models are evaluated without adaptation. Table 2 gives the available uncertainty for the joint factor/band summaries. These means alone do not provide seed-level standard deviations. Both trainable, k=1k=1 Both trainable, k=2k=2 Task / model Top init. Bottom init. Top init. Bottom init. Format / Llama-3.2-1B 79.28 79.54 80.00 80.72 Format / Qwen2.5-1.5B 78.96 78.11 77.75 78.57 OpenBookQA / Qwen2.5-1.5B 55.24 54.96 55.76 55.28 OpenBookQA / Qwen2.5-7B 62.32 62.64 62.92 63.12 OpenBookQA / Mistral-7B 59.24 59.24 59.88 60.48 Table 8: Both-trainable controls (%). B starts on the indicated spectral band, A starts at zero, and both factors train. Rank 1 has the same count as the rank-1 free reference; rank 2 matches the frozen adapters’ rank but has substantially more trainable parameters. The band is an initialization, not a persistent constraint, in these controls. For the rank-2 A-frozen-minus-B-frozen contrasts on the larger models, the paired-t statistics are 29.48 and 25.53 on Qwen2.5-7B, and 18.44 and 26.02 on Mistral-7B (top and bottom, respectively). The direct parameter-count reversal provides independently specified comparisons with explicit intervals in Table 3. Neither the factor/band table nor the free-reference scores should be interpreted as a formal equivalence test between A-frozen and free LoRA. Appendix B Rank-dependent scores and boundary cases B frozen A frozen Free Ref. rank Task / model Rank Top Bottom Top Bottom OBQA / Qwen2.5-1.5B 4 46.44 45.24 53.44 53.88 55.24 2 8 49.92 46.56 56.28 53.44 55.44 4 16 54.08 49.84 55.44 53.64 56.16 7 OBQA / Qwen2.5-7B 16 60.20 56.76 62.40 62.56 63.20 7 ARC-C / Qwen2.5-1.5B 2 48.42 46.68 — — 50.44 1 16 49.70 44.52 49.22 49.82 49.52 7 32 48.24 47.58 — — 48.88 14 Table 9: Higher-rank comparisons and ARC-Challenge boundary cases (%). Seeds 2–6. OBQA denotes OpenBookQA and ARC-C denotes ARC-Challenge. A-frozen scores at rank 16 on Qwen2.5-1.5B are recovered algebraically from the reported B-frozen means and A-minus-B differences. They do not introduce new experiments. The ARC-Challenge base score is 43.60%. Model / task Rank Top A−-B Bottom A−-B Interpretation Qwen2.5-1.5B / OBQA 4 7.00[5.42,8.58]7.00\ [5.42,8.58] 8.64[6.90,10.38]8.64\ [6.90,10.38] Both pass 8 6.36[4.80,7.92]6.36\ [4.80,7.92] 6.88[4.18,9.58]6.88\ [4.18,9.58] Both pass Qwen2.5-7B / OBQA 16 2.20[0.69,3.71]2.20\ [0.69,3.71] 5.80[3.35,8.25]5.80\ [3.35,8.25] Both pass Table 10: Available direct-contrast uncertainty in the rank extension. Percentage points; nominal paired-t 95% intervals, five reporting seeds. “Pass” refers to the specified threshold t>3.495t>3.495 for the two bands within each campaign. No test of a universal monotonic trend or a common crossover rank is implied. Task / model, frozen rank 16 B top B bottom Free reference Free minus bottom Format / Llama-3.2-1B 77.56 75.54 82.05 (k=8k=8) 6.51 Format / Qwen2.5-1.5B 74.30 75.24 80.78 (k=7k=7) 5.54 OpenBookQA / Qwen2.5-1.5B 54.08 49.84 56.16 (k=7k=7) 6.32 OpenBookQA / Qwen2.5-7B 60.20 56.76 63.20 (k=7k=7) 6.44 ARC-Easy / Qwen2.5-1.5B 79.48 77.92 79.40 (k=7k=7) 1.48 ARC-Challenge / Qwen2.5-1.5B 49.70 44.52 49.52 (k=7k=7) 5.00 Table 11: All six rank-16 bottom-band gaps. Scores in percent and gaps in percentage points; seeds 2–6. The last column is a descriptive subtraction, not a set of six significant comparisons. ARC-Easy’s base score is 71.10%. On OpenBookQA / Qwen2.5-1.5B, both-trainable spectral controls at the comparable-budget rank score 55.40/55.20% at frozen rank 4, 55.04/54.20% at frozen rank 8, and 55.52/55.52% at frozen rank 16 (top/bottom initialization). At equal rank they score 55.04/54.20%, 56.36/55.56% and 57.24/55.28%, respectively. The rank-4/8 extension uses its own campaign runs for reused configurations. On OpenBookQA / Qwen2.5-7B at frozen rank 16, the corresponding values are 62.48/62.76% at reference rank 7 and 63.32/62.00% at equal rank 16. The rank-16 factor contrast on ARC-Challenge / Qwen2.5-1.5B is 2.41 points and the absolute band contrast is 2.29 points. The top-band A-minus-B contrast is −0.48-0.48 points, while the bottom-band contrast is +5.30+5.30. Thus an averaged factor contrast should not be mistaken for an advantage on each band. Appendix C Parameter counts, subspaces and numerical construction Let Din=∑ℓ,jdin(ℓ,j)D_in= _ ,jd_in^( ,j) and Dout=∑ℓ,jdout(ℓ,j)D_out= _ ,jd_out^( ,j), summing over adapted layers and module types. The counts are rDinrD_in for B frozen, rDoutrD_out for A frozen, and k(Din+Dout)k(D_in+D_out) for both-trainable LoRA. This comparison excludes the frozen backbone and the storage of fixed bases. For Qwen2.5-1.5B, Din=508,928D_in=508,928 and Dout=645,120D_out=645,120. The exact totals are: Configuration Rank-2 comparison Rank-16 comparison B frozen 1,017,856 (r=2r=2) 8,142,848 (r=16r=16) A frozen 1,290,240 (r=2r=2) 10,321,920 (r=16r=16) Free reference 1,154,048 (k=1k=1) 8,078,336 (k=7k=7) The rank-1 reference has 13.38% more parameters than B frozen at rank 2, while the rank-7 reference has 0.79% fewer than B frozen at rank 16. On Llama-3.2-1B, Dout/Din=1.15D_out/D_in=1.15, giving a 7.5% surplus for the half-rank free reference. On Qwen2.5-7B the per-layer sums are 40,448 and 49,664, respectively, giving a 2.53% deficit for the rank-7 reference against rank-16 B frozen. The parameter-reversal control in Table 3 has the following per-layer counts; multiplication by the common number of adapted layers leaves the ratios unchanged. Model A frozen, rank 2 B frozen, rank 3 B frozen, rank 4 Qwen2.5-1.5B 46,080 54,528 72,704 Qwen2.5-7B 99,328 121,344 161,792 The B-frozen rank-3 scores are 46.80/43.28% on Qwen2.5-1.5B and 54.64/49.24% on Qwen2.5-7B (top/bottom). Their selected rates are 5⋅10−3/2⋅10−35· 10^-3/2· 10^-3 and 2⋅10−3/5⋅10−42· 10^-3/5· 10^-4. Rank-4 B-frozen scores are 46.44/45.24% and 54.64/51.36%, respectively. Comparing these rank-4 scores with rank-2 A frozen is a descriptive extension of the rank-3 control. C.1 Reachable updates and optimization For an orthonormal fixed Q∈ℝdout×rQ ^d_out× r, QA:A∈ℝr×din=ΔW:col(ΔW)⊆col(Q).\QA:A ^r× d_in\=\ W:col( W) (Q)\. For an input basis P∈ℝdin×rP ^d_in× r with orthonormal columns, the analogous A-frozen set is BP⊤\BP \, whose row space lies in row(P⊤)row(P ). Given a target update T, its best Frobenius approximations in these spaces are QQ⊤TQQ T and TPP⊤TPP . These projection identities explain the constraints but do not identify the target update preferred by a nonlinear network or the solution found by AdamW. A change of basis Q↦QOQ QO with O orthogonal leaves the reachable update set unchanged. It need not leave coordinate-wise AdamW dynamics unchanged. This distinction also applies to small within-band rotations caused by finite-precision SVD computations. Column-sign changes can be absorbed by matching row-sign changes in A, but general rotations do not establish optimizer equivalence. C.2 SVD precision and the bottom band The complete thin SVD is computed in fp32; singular values are retained in fp32 and singular vectors are stored in fp16. With zero-based indices, a band at relative position β∈[0,1]β∈[0,1] begins at ⌊β(m−r)⌋ β(m-r) and includes the next r directions. The main two bands use β=0β=0 and β=1β=1. A random spectral-index baseline selects r columns of U without replacement; it is distinct from the random orthonormal baseline, which is independent of W0W_0. No numerical threshold removes small singular values in the reported runs. For a rectangular matrix, a thin SVD omits the additional structural null-space directions on the larger side. It may still contain zero or numerically near-zero singular values if the weight is rank deficient. Near-degenerate directions also make individual singular vectors less stable than their spanned subspace. An implementation check against PEFT 0.21.0’s MiCA initialization used a common fp32 SVD on 14 Qwen2.5-1.5B modules at rank 2 and found matching initialized factors, trainable parameters, outputs and gradients on A. With the actual fp16 caches, checks across all 308 adapted modules of Qwen2.5-1.5B and Llama-3.2-1B at ranks 2 and 16 found a maximum principal-angle sine of 5.8⋅10−35.8· 10^-3, with median values between 2⋅10−42· 10^-4 and 9⋅10−49· 10^-4. These are numerical subspace checks, not end-to-end equivalence tests of the training recipes. C.3 Relation to MiCA and PiCa The B-frozen bottom configuration follows MiCA’s parameterization: the left singular vectors at the bottom of the pretrained weight’s thin SVD define fixed B, and A starts at zero. The scaling, adapted modules, tasks, training lengths and hyperparameter search follow our protocol. The controlled arm reproduces this construction. The separate PEFT follow-up in Section 4.1 transfers selected training settings from Rüdiger and Raschka (2026), with the adaptations specified below. PiCa’s paper and PyTorch weight storage use transposed conventions. Its official LinearWithPiCa implementation computes the right singular vectors of a weight stored with shape (dout,din)(d_out,d_in), registers the top r rows as a fixed buffer P, and trains M∈ℝdout×rM ^d_out× r. Its effective update is MPMP. Consequently P corresponds to our A and M to our B. This can be inspected in pica/pica_layer.py of the official repository at revision 8fa738b70b0c03e7643515695f1e9e6785ec3778.11 1 https://github.com/hjunseoh/PiCa. The cited revision fixes the inspected implementation; it is not a revision of our experimental code. Its shared trainable factor is a further difference from our layer-specific adapters. PiCa’s Gemma-2B projection ablation is at rank 256, whereas our central A/B comparison is at rank 2. We did not execute PiCa and do not infer its relative benchmark performance from the present results. C.4 PEFT MiCA follow-up: transferred settings The PEFT arm uses init_lora_weights="mica", r=α=16r=α=16, fixed B and initially zero A. We take learning rate 5⋅10−45· 10^-4 and four epochs from the MiCA paper’s rank-16 Llama-2-7B BLOGS entry, together with its cosine schedule, 10% warm-up, weight decay 0.01, adapter dropout 0.05, gradient clipping 0.3 and batch size four without accumulation. We use AdamW and retain our OpenBookQA loader, 384-token blocks, precision and evaluator. The original study uses a 512-token limit and normally adapts query and value projections; we retain all seven module types. Its base-to-instruct weight composition is not part of this benchmark protocol. Hence “transferred recipe” denotes these explicit settings, not a full replication. The first campaign trains MiCA and reruns our bottom B-frozen and free-LoRA references on seeds 2–6. It specifies one two-sided paired test of free LoRA minus MiCA at level 0.05 (t>2.776t>2.776, four degrees of freedom). MiCA minus our bottom arm is 2.56[1.30,3.82]2.56\ [1.30,3.82] points, descriptively. The four-epoch and main recipes differ in both supervised tokens and optimizer steps (Table 6). The shared-recipe test was planned after this result. It uses PEFT’s standard initialization (init_lora_weights=True), r=α=7r=α=7, and the same data, modules and training settings as the MiCA arm. Its five new LoRA runs are paired with the existing five MiCA runs. The protocol fixes the same nominal 0.05 test before these new runs, but the comparison is an adaptively chosen follow-up, not an independent replication of MiCA. Free LoRA’s change from the main to the transferred recipe is −0.64[−1.41,0.13]-0.64\ [-1.41,0.13] points; this interval does not establish equivalence. Neither method is independently retuned in the shared-recipe test, and the transferred rate need not favor either method on this task. Appendix D Training, evaluation and selection details D.1 Training configuration The backbones belong to the Llama, Qwen2.5 and Mistral families (Grattafiori et al., 2024; Qwen Team, 2024; Jiang et al., 2023). The main configuration uses AdamW (Loshchilov and Hutter, 2019) with (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999), zero weight decay, clipping at 1.0, bf16 model weights and gradient checkpointing. The cosine schedule uses a 5% warm-up. The effective batch is two sequences per micro-batch with 16 accumulation steps. Setting α=rα=r maintains a scale of one across ranks. Free LoRA initializes A using the Kaiming construction (He et al., 2015) and B at zero; neither spectral trainable control modifies W0W_0 at initialization. The supervised-token counter is incremented at every forward pass, and training stops after the optimizer step that reaches the token budget. The accumulation counter spans epoch boundaries, with gradients cleared after each optimizer step. The standard runs require 274 steps (2.76 epochs) for the Llama formatting task, 49 steps (5.76 epochs) for OpenBookQA and 49 steps (14.93 epochs) for ARC-Challenge. Thus the 600,000-token budget is a nominal stopping target and can be exceeded by the final complete step. On OpenBookQA and ARC, training examples contain the question and the correct answer text, without distractor options. They are concatenated and packed into blocks of 384 tokens. Every token is supervised in the default setting. Evaluation uses the prompt “Question: … Answer:” and scores each candidate answer by score(a∣q)=1|a|∑t=1|a|logp(at∣q,a<t).score(a q)= 1|a| _t=1^|a| p(a_t q,a_<t). The reported metric is the fraction of questions for which the highest-scoring candidate is correct. Training uses each benchmark’s own training split. Public benchmark scores do not constitute a guarantee against pretraining contamination. D.2 Formatting task and verifier Sentences are drawn from CoNLL-2003 and retained when they contain at least three entities. Targets require person names in last-name-first order, alphabetically sorted lists, an entity count written in words, and entity initials in document order. The verifier enforces tags, single-line JSON, the required keys and their order, and the target’s transformation rules. A generation passes only if all checks succeed. The task therefore combines entity handling, transformations and serialization; “valid-format accuracy” is stricter than merely syntactic JSON validity. The training set used by the full-gradient diagnostic contains 3,178 sequences. Evaluation generates at most 128 tokens on 614 test items. Across main rank-2 configurations, 0.7–0.9% of generations reach that limit. Component-level checks place the larger bottom-band deficits in the signature and entity-count fields rather than the presence of keys. Exact reproduction of this custom metric requires the data generator, prompt template, split manifest and verifier implementation, not merely the verbal description. Runs predating verifier-version logging were checked where possible. Saved generations from 91 runs were rescored and matched their recorded scores. Fifteen reported runs without saved generations were repeated and replaced. Five repetitions matched the original score exactly; the other ten moved by −2.6-2.6 to +3.6+3.6 points, with a mean change of +0.1+0.1. This is consistent with run variability but does not by itself prove the absence of an earlier verifier difference. D.3 Learning-rate search and its exceptions The main initial grid is 10−4,5⋅10−4,10−3,2⋅10−3\10^-4,5· 10^-4,10^-3,2· 10^-3\, extended to 5⋅10−35· 10^-3 and 10−210^-2 when needed. Position sweeps additionally use 2⋅10−22· 10^-2. The rank-16 OpenBookQA search spans seven rates from 10−410^-4 to 2⋅10−22· 10^-2. Validation scores, normally averaged over seeds 0 and 1, select the rate. No test-set optimum replaces this selection in the main score tables. The main formatting selection uses a 400-item validation evaluation. The rank-2 bottom-band tie on Llama was additionally evaluated on selection seeds 30–32. The wider experimental record also contains two selections whose independence from reporting seeds was not established. The main rate-selection principle should therefore not be interpreted as a claim that every historical selection was strictly independent. The fresh-seed replication and the separately specified extension campaigns reduce reliance on these early choices without resolving them retrospectively. Some campaign grids end at a selected boundary: the rank-2 bottom B-frozen arm on Qwen2.5-7B and the top A-frozen arm on Mistral select the lowest tested rate. In the four-times-longer formatting control, top-initialized both-trainable LoRA selects the lowest rate and free LoRA the highest. These configurations retain their observed optima over the tested grids; an untested rate is not ruled out. The parameter-reversal and rank-extension campaigns have interior selected rates. In the answer-only campaign, the bottom A-frozen grid was extended to 2⋅10−22· 10^-2, after which 10−210^-2 remained the selected rate. For the complete rank extensions on OpenBookQA, the selected rates for B-top, B-bottom, A-top, A-bottom and the free reference, in that order, are (5,2,5,2,5)⋅10−3(5,2,5,2,5)· 10^-3 at rank 4 on Qwen2.5-1.5B, (5,1,2,2,2)⋅10−3(5,1,2,2,2)· 10^-3 at rank 8 on that model, and (1,0.5,1,0.5,0.5)⋅10−3(1,0.5,1,0.5,0.5)· 10^-3 at rank 16 on Qwen2.5-7B. Frozen ranks and reference ranks are distinguished in Table 9. A post hoc sensitivity analysis that chooses the best rate on reporting seeds is optimistic and is not a valid replacement for validation-based model selection. Several rank-16 auxiliary verdicts changed under that analysis, further motivating the descriptive treatment of small higher-rank effects. Lower selected rates and curvature. At rank 2 on the formatting task, the two-seed selections on Llama-3.2-1B and an additional model, Qwen2.5-0.5B, give bottom-band B frozen half the rate of the other B-frozen configurations (5⋅10−35· 10^-3 versus 10−210^-2). A five-seed validation selection supports a lower bottom-band rate on four of five Llama seeds, but only partly on OpenBookQA. Its reading rules were recorded three minutes after launch, before results were read (supplementary record, §S16.2). Post hoc Llama curvature measurements estimate the largest eigenvalue λmax _ of the training-loss Hessian H restricted to each configuration’s trainable directions. The estimator uses power iteration on H+cIH+cI, with the shift estimated in an unshifted pass, and retains the largest of five restarts. At initialization, the bottom-band median is 333, versus 29–33 for top, random-index and random orthonormal configurations. At step 274, bottom remains higher than top (23.33 versus 6.95). All 15 paired bottom-to-other ratios at initialization exceed two, while the product of λmax _ and the selected learning rate η is 5.1–5.7 times larger for bottom (supplementary record, §S23.3). This is consistent with the direction of the rate shift, but neither explains its magnitude nor establishes its cause. Lee et al. (2026) similarly relate method-specific optimal learning rates to the largest loss-Hessian eigenvalue across LoRA variants. The raw Hessian omits AdamW’s preconditioning, and the eigenvalue estimates have numerical uncertainty. Rankings at a common learning rate. Historical formatting experiments show reversals in the ranking by mean score on Llama-3.2-1B at ranks 2 and 4, and Qwen2.5-0.5B and Qwen2.5-1.5B at rank 2 (supplementary record, §S23.4). On Llama at rank 2, bottom leads the best other B-frozen mean by 2.15 points at 2⋅10−32· 10^-3, but trails the best other mean by 14.43 points at 10−210^-2. These extrema use different competitors and need not equal the seed-paired contrasts. The bottom advantage paired over seeds 2–6 at the lower rate is only 0.98 points, with a nominal paired-t 95% interval of [−1.4,3.4][-1.4,3.4], so this half of the reversal is not established. Both halves pass their original nominal paired-t tests (t>2.776t>2.776, four degrees of freedom) only on Qwen2.5-0.5B, where one reused cell predates a protocol that described it as new. The Llama rank-2 analysis is post hoc, and the other three configurations have prespecified campaign protocols. These observations motivate separate rate selection without establishing a universal reversal across tasks or ranks. D.4 Resolving four historical grid-boundary cases A follow-up evaluates 2⋅10−22· 10^-2 for the formatting task’s top, bottom and random-spectral-index arms at rank 1 and its random-spectral-index arm at rank 4. It reuses the original selection runs at other rates, adds seeds 0 and 1 at the new rate, and reruns seeds 2–6 only if selection changes. When the top two validation means differ by at most 0.005, both candidates receive seeds 30–32. A five-seed lead above 0.005 resolves the tie; otherwise the higher mean is retained and the tie declared unresolved. B-frozen configuration Earlier rate Revised rate Earlier score Revised score Top, rank 1 10−210^-2 2⋅10−22· 10^-2 51.07 48.34 Bottom, rank 1 10−210^-2 10−210^-2 45.47 45.47 Random spectral indices, rank 1 10−210^-2 10−210^-2 52.88 52.77 Random spectral indices, rank 4 10−210^-2 5⋅10−35· 10^-3 70.23 70.81 Table 12: Corrections to historical formatting scores (%). Llama-3.2-1B, reporting seeds 2–6. These are selection and aggregation corrections, not significance tests. The rank-1 random correction removes a run from a different configuration; its selected rate is unchanged. For top rank 1, validation rises from 53.25 to 55.63% at the new rate while test performance falls by 2.73 points. The selected rate remains at the new upper boundary; no further extension was specified. The top-minus-bottom test gap therefore falls from 5.60 to 2.87 points. Bottom rank 1 scores only 17.87% on validation at 2⋅10−22· 10^-2 and retains its old rate. Random rank 1 has a near-tie at first (58.37 versus 58.25%); the added selection seeds favor 10−210^-2, 57.40 versus 55.90%. For random rank 4, 2⋅10−22· 10^-2 scores zero on validation, while 5⋅10−35· 10^-3 beats 10−210^-2 (72.00 versus 71.25%). The earlier version reported 70.23% at 10−210^-2, a rate fixed in advance for a separate rank-4 measurement. The current selection uses configuration-consistent, salted random draws and selects 5⋅10−35· 10^-3, yielding 70.81%. The change therefore comes from applying this selection rule, rather than from the new upper rate. None of these four cells enters the central A/B rank sweep on Qwen. Appendix E Robustness controls and spectral-position checks Control and contrast Top Bottom Scope Fresh seeds: both trainable r=2r=2 minus B frozen r=2r=2 14.50 21.27 Format / Llama; seeds 120–124 Four-times tokens: same contrast 11.99 18.24 Format / Llama; all rates retuned Original-token counterpart 15.57 21.50 Same task, original campaign Answer-only: A frozen minus B frozen, r=2r=2 7.16 9.72 OBQA / Qwen-1.5B; steps fixed Square-module control: A frozen minus B frozen 39.3 33.6 Top arm meets collapse condition Table 13: Robustness controls (percentage points). The first three rows compare equal rank, not equal trainable counts. The square-module values use the underlying reported differences; subtraction of rounded scores can differ slightly. Its top-band number is descriptive and was excluded from the prespecified interpretation because the B-frozen score was below the collapse threshold. The four-times-longer formatting run uses 2.4 million supervised tokens. The equal-rank differences have paired-t statistics 12.26 and 16.57 for the top and bottom bands. The deficits against comparable-budget free LoRA are 9.74 and 15.54 points. This shows persistence over the tested extension, not convergence to an asymptotic floor. In the fresh-seed replication on one GPU type, gains of both-trainable rank-1 controls over B-frozen rank-2 controls are 14.85 [13.32,16.39][13.32,16.39] and 20.42 [17.48,23.37][17.48,23.37] points. The reference gaps in that campaign are 10.72 and 17.10 points. The complete frozen-factor comparison on these fresh seeds has F=15.21F=15.21 and S=−2.96S=-2.96 points, with interaction 6.846.84 [4.33,10.88][4.33,10.88] (paired-bootstrap 95% interval). Reusing selected rates tests replication at those rates, not independent reselection of the entire pipeline. The square-module control adapts only Llama’s query and output projections. At rank 2, both frozen orientations have 4,096 trainable parameters per adapted module. The top/bottom B-frozen scores are 33.7/37.1%, the A-frozen scores are 72.9/70.7%, and free LoRA scores 76.7%. Reporting seeds are 140–144 on one GPU type. The top B-frozen score lies below the prespecified 45% collapse cutoff. Both module types have near-degenerate bottom spectra, so the control is limited evidence about dimension-independent asymmetry. E.1 Seven-arm replication on one GPU model The replication repeats all seven central formatting arms on RTX 4000 Ada, using seeds 2–6 and each arm’s original selected rate. These runs are separate from both the original cells and the earlier fresh-seed replication. Configuration Rank Selected rate Original score Replication B frozen, top 2 10−210^-2 64.43 64.23 B frozen, bottom 2 5⋅10−35· 10^-3 59.22 58.31 A frozen, top 2 10−210^-2 77.39 77.39 A frozen, bottom 2 10−210^-2 78.18 77.98 Free LoRA 1 10−310^-3 76.84 76.97 Both trainable, top initialization 1 5⋅10−35· 10^-3 79.28 80.36 Both trainable, bottom initialization 1 5⋅10−35· 10^-3 79.54 79.67 Table 14: Single-GPU-model replication on Format / Llama-3.2-1B (%). All 35 runs use the original reporting seeds and selected rates. The historical means are retained alongside the replication. The free-minus-B contrasts have t=11.39t=11.39 and 13.8613.86; A-minus-B has t=11.49t=11.49 and 14.3614.36. Each clears the prespecified two-band threshold 3.495. Gains from training both factors at rank 1 are 16.12[13.20,19.04]16.12\ [13.20,19.04] and 21.37[19.09,23.64]21.37\ [19.09,23.64] points. Their nominal intervals lie above the campaign’s fixed thresholds 8.92 and 13.07 points, respectively. This recovery criterion retains the conditional interpretation in Appendix G. E.2 Within-module comparisons: counts and selected rates The two groups are the down projections alone and the gate plus up projections alone, each adapted in every layer. At rank 2, the per-module counts for B frozen versus A frozen are 16,384 versus 4,096 on Llama down projections and 17,920 versus 3,072 on Qwen down projections. For gate/up projections these counts exchange roles. Raising the B-frozen rank to 8 on Llama makes both orientations use 16,384 parameters per module; rank 12 on Qwen gives B frozen 18,432 versus 17,920 for A frozen. Across all adapted modules, the counts are: Model Group A frozen, rank 2 B frozen, comparison rank Free, rank 1 Llama-1B Down 65,536 262,144 (r=2r=2) 163,840 Gate + up 524,288 524,288 (r=8r=8) 327,680 Qwen-1.5B Down 86,016 501,760 (r=2r=2) 293,888 Gate + up 1,003,520 1,032,192 (r=12r=12) 587,776 Rates are selected on seeds 0 and 1 from 5⋅10−4,10−3,2⋅10−3,5⋅10−3,10−2,2⋅10−2\5· 10^-4,10^-3,2· 10^-3,5· 10^-3,10^-2,2· 10^-2\, with edge extensions along the 1–2–5 sequence and the tie rule of Appendix D.4. Reporting uses seeds 2–9. All runs use RTX 4000 Ada and the main 600,000-token target. Validation used 500 examples rather than the specified 400, consistently across this campaign’s arms and rates. Pair Group Configuration Score top / bottom Rate top / bottom Format / Llama-1B Down B frozen, r=2r=2 29.38 / 12.30 10−210^-2 / 10−210^-2 A frozen, r=2r=2 65.92 / 67.75 2⋅10−22· 10^-2 / 5⋅10−25· 10^-2 Free, r=1r=1 73.49 5⋅10−35· 10^-3 Gate + up B frozen, r=2r=2 39.15 / 40.53 5⋅10−25· 10^-2 / 2⋅10−22· 10^-2 A frozen, r=2r=2 78.14 / 75.53 5⋅10−35· 10^-3 / 2⋅10−22· 10^-2 B frozen, r=8r=8 65.57 / 65.49 10−210^-2 / 10−210^-2 Free, r=1r=1 78.62 5⋅10−35· 10^-3 OBQA / Qwen-1.5B Down B frozen, r=2r=2 45.85 / 40.75 5⋅10−35· 10^-3 / 10−210^-2 A frozen, r=2r=2 49.15 / 49.62 2⋅10−22· 10^-2 / 2⋅10−22· 10^-2 Free, r=1r=1 52.65 5⋅10−35· 10^-3 Gate + up B frozen, r=2r=2 42.08 / 44.10 10−210^-2 / 10−310^-3 A frozen, r=2r=2 53.33 / 51.95 5⋅10−35· 10^-3 / 5⋅10−35· 10^-3 B frozen, r=12r=12 48.85 / 45.95 2⋅10−22· 10^-2 / 2⋅10−22· 10^-2 Free, r=1r=1 53.55 5⋅10−35· 10^-3 Table 15: Within-module scores (%) and selected learning rates. Means over eight seeds. Free LoRA is a descriptive reference and is not budget matched to either frozen orientation. Table 4 reports the eight prespecified contrasts. Each group must first pass a fixed testability criterion: free LoRA must improve on the base model by at least two points. All four groups pass, with gains of 73.49 and 78.62 points on Format and 10.45 and 11.35 on OpenBookQA. This criterion differs from the earlier square-module collapse rule. The rank-2 gate/up A-minus-B contrasts are descriptive: 38.99/35.00 points on Format and 11.25/7.85 on OpenBookQA. Two selected rates remain at the upper tested boundary despite the protocol’s interior-maximum target: Format down A-bottom and Format gate/up B-top at rank 2. The latter enters only the descriptive equal-rank comparison. Three ties remain unresolved: Format down A-top, Format gate/up B-top at rank 8, and OpenBookQA gate/up B-bottom at rank 2. The free-LoRA ties on Format were resolved. The eight primary comparisons therefore support an advantage at the selected settings, not a comparison of fully optimized configurations. E.3 Answer-only loss Configuration Selected rate Answer-only score Full-loss score B frozen, top 5⋅10−35· 10^-3 48.36 46.52 B frozen, bottom 10−310^-3 45.76 42.20 A frozen, top 5⋅10−35· 10^-3 55.52 53.08 A frozen, bottom 10−210^-2 55.48 52.48 Free LoRA, rank 1 2⋅10−32· 10^-3 55.72 54.56 Table 16: Loss-mask control on OpenBookQA / Qwen2.5-1.5B. Scores in percent, seeds 2–6; frozen ranks are 2. The original packed blocks and 49 optimizer steps are retained. Full-loss scores are included for context and are not paired tests of an equal-supervision loss change. The loader contains 272 packed blocks, 4,945 answer markers and 4,945 supervised answers per epoch. Answer masking retains 17,300 of 104,448 tokens per epoch. A run supervises a median of 99,614 tokens versus about 602,112 under the full loss. A-minus-B differences are 7.16 [4.66,9.66][4.66,9.66] and 9.72 [7.23,12.21][7.23,12.21] points, with t=7.96t=7.96 and 10.8510.85. Both exceed the two-band threshold 3.495. This supports persistence of the asymmetry when question-token loss is removed, not equivalence of the two training objectives. E.4 Interior spectral positions and scaling On the formatting task with Llama at rank 2, the five positions β=0,0.25,0.5,0.75,1β=0,0.25,0.5,0.75,1 have mean scores 65.47, 63.68, 66.71, 67.68 and 59.28% over seeds 2–8 at separately selected rates. The corresponding across-seed standard deviations are approximately 1.1, 2.0, 1.9, 1.7 and 1.9 points. The best position is interior, and the full range is 8.40 points. This is larger than the top-versus-bottom contrast emphasized in the main design and cautions against interpreting it as a bound on all spectral placements. A separate fresh-seed extension, seeds 9–16, spans up to 9.53 points. Its detailed pairwise evidence is not used for the central claim. For the top-band scaling controls, fixed B=UrΣrB=U_r _r improves over B=UrB=U_r by 3.61 points (t=4.22t=4.22), and fixed B=Urσ¯B=U_r σ, where σ¯ σ is the band’s mean singular value, improves by 2.31 points (t=3.32t=3.32). Both constructions preserve the column space when the scaling is invertible. Their different optimization trajectories are why we do not equate an empirical score gap with a difference in representable updates. Figure 5: Training losses at the selected rates, reproduced from the experimental record. Mean over seeds 2–6, logged every ten optimizer steps. Top: OpenBookQA / Qwen2.5-1.5B, full packed-text loss. Bottom: Format / Llama-3.2-1B, target-only loss on a logarithmic scale. The two vertical axes are not comparable. “B trained” means both factors train from spectral initialization. Continued decline and cosine learning-rate decay prevent interpreting a visually flat tail as proof of convergence. E.5 Random input bases: a prespecified replication The dual_random_ortho arm freezes A on a random orthonormal basis of the input space and trains B from zero, without using the pretrained spectrum to construct A. Its B-frozen counterpart, random_ortho, fixes a random orthonormal output basis. The follow-up uses the main training setup on Format / Llama-3.2-1B and OpenBookQA / Qwen2.5-1.5B at rank 2, with a descriptive OpenBookQA extension at rank 16. Selection and scores. Validation on seeds 0–1 selects the rates in Table 17; seeds 2–6 report test performance. Format selects 2⋅10−22· 10^-2 at the initial upper boundary. The prescribed one-step extension to 5⋅10−25· 10^-2 diverges, leaving 2⋅10−22· 10^-2 as an interior maximum of the extended grid. In every cell the two leading validation means differ by more than 0.005, so no tie rule is invoked. Task, rank Rate Seed 2 Seed 3 Seed 4 Seed 5 Seed 6 Mean Format, 2 2⋅10−22· 10^-2 77.52 78.18 77.69 76.71 77.04 77.43 OpenBookQA, 2 10−210^-2 55.20 55.00 53.80 55.60 53.20 54.56 OpenBookQA, 16 2⋅10−32· 10^-3 56.20 56.20 56.20 55.80 57.80 56.44 P1, rank 2 Random A−-random B Nominal 95% interval t Format / Llama-3.2-1B +12.08+12.08 [10.62,13.55][10.62,13.55] 22.86 OpenBookQA / Qwen2.5-1.5B +8.64+8.64 [7.44,9.84][7.44,9.84] 20.06 Table 17: Random A-frozen replication. Top: test scores in percent. Bottom: seed-paired differences in percentage points against the previously reported random B-frozen runs (65.34% and 45.92%). Both P1 comparisons pass the two-sided Bonferroni rule over two pairs, |t|>3.495|t|>3.495 with four degrees of freedom. The displayed 95% intervals are not multiplicity corrected. Prespecified test and descriptive comparisons. P1 pairs the new A-frozen runs with the previously reported B-frozen runs using the same seed labels, 2–6. Both comparisons pass the fixed decision rule (Table 17): the A-frozen advantage does not require spectral placement of A on these two pairs. This equal-rank comparison retains the count asymmetry in Section 2.2; it is distinct from the count controls. Table 18 compares spectral A-frozen arms and the paper’s free-LoRA references. These comparisons were not prespecified as tests, are uncorrected and do not establish equivalence. The rank-16 results are descriptive only. Task, frozen rank Minus A-top Minus A-bottom Minus free LoRA Format, 2 +0.03+0.03 −0.75-0.75 +0.59+0.59 [−2.13,2.20][-2.13,2.20] [−1.92,0.42][-1.92,0.42] [−1.99,3.17][-1.99,3.17] OpenBookQA, 2 +1.48+1.48 +2.08+2.08 0.000.00 [−0.21,3.17][-0.21,3.17] [0.75,3.41][0.75,3.41] [−2.86,2.86][-2.86,2.86] OpenBookQA, 16 +1.00+1.00 +2.80+2.80 +0.28+0.28 [0.21,1.79][0.21,1.79] [1.96,3.64][1.96,3.64] [−1.63,2.19][-1.63,2.19] Table 18: Descriptive random-A contrasts, in percentage points. Nominal paired-t 95% intervals over seeds 2–6; no multiplicity correction or equivalence test. Spectral arms use the same rank as random A. Free LoRA uses ranks 1, 1 and 7, with counts chosen near B frozen, not exactly matched to A frozen (Appendix C). Reported contrasts retain source rounding and may differ by 0.01 points from subtraction of displayed means. Earlier runs and prior access to their results. The 49 random-A runs initially launched under PROTOCOL_freeze_A.md failed before training because of a misplaced check. The check was corrected on 26 September, documented in lora.py, and the runs completed after relaunch in the same directories. Two completed 49-run grids from that day use rank 2, seeds 0–6 and seven rates, with salted random draws and no separate subspace_seed. The Format grid is identified with that protocol; the OpenBookQA grid has the same design and overlaps its execution window, making its attribution likely but not certain. Validation-selected rates are 2⋅10−22· 10^-2 and 10−210^-2, giving reporting means 76.84% and 54.64%. The new means, 77.43% and 54.56%, are similar, but the seed-level scores differ: these are new runs, not reused observations. The Format results were read on 26 September: the three A-frozen means were within 1.34 points, and random A selected the upper boundary. On 3 October, the GPU-attribution audit displayed both earlier means incidentally because they coincided with published values. The earlier seed-level scores were read only on 4 October, after reading the new experiment’s results. The old runs are descriptive and excluded from P1; their partial prior visibility makes this a prespecified replication, not a blind discovery test. Declared deviations. The report declares three deviations: 1. Addendum pushed after selection submission. Planned before any run, it was pushed at 21:12:02, two minutes after selection was submitted (21:10), when no run had finished: before any result from this experiment and before any reporting run. 2. P1 is not a blind test. Its outcome was largely predictable from the earlier runs (addendum and preceding paragraph). The test, its threshold and its interpretation are unchanged. 3. Interrupted deployment. A new version overwrote the deployment script while it was executing; it stopped after submitting selection, without starting the driver. The driver was started manually at 21:12. The runs were unaffected: the state was checked before resuming, with no duplicate submissions. No other deviation is declared: the reference runs use salted draws, parameters conform to the protocol, there are no ties, and the single extension step follows the rule. The follow-up comprises 53 runs (36 selection, 2 extension and 15 reporting), using 5.2 GPU-hours against 6–7 planned. Protocol dates appear in Table 23. Appendix F Placement diagnostics: scores, references and uncertainty Where free LoRA writes. A geometric diagnostic uses saved rank-1 free-LoRA factors from five seeds on OpenBookQA / Qwen2.5-1.5B and Format / Llama-3.2-1B. Its reading rule was fixed before the OpenBookQA measurement and applied post hoc to Format (supplementary record, §S15, phase 1). A module type is classified as outside both tested rank-2 bands when its output-factor energy share in each band is below five times the isotropic reference 2/dout2/d_out, using layer medians averaged over seeds. Six of seven module types satisfy this rule on both pairs. The key projection is more aligned with the bottom band, but its energy share remains only 6.1% on OpenBookQA and 2.5% on Format. Thus the learned output factors are not concentrated in either tested extreme. This classification does not mean zero spectral overlap, and the strict all-seven-module trigger for the oracle follow-up was not met (Appendix F.2). It motivates testing learned placements but does not by itself explain their performance. Fixed output basis, rank 2 OpenBookQA / Qwen-1.5B Format / Llama-1B Top spectral band 46.52 64.43 Bottom spectral band 42.20 59.22 Random orthonormal 45.92 65.34 Initial gradient, 64 sequences 42.72 66.58 Initial gradient, all sequences 43.04 66.22 After 10% warm-up 51.16 72.21 Oracle, teacher seed 0 55.88 80.81 Oracle, teacher seed 7 55.28 — Free reference, rank 1 (original) 54.56 76.84 Free reference, rank 1 (diagnostic rerun) 54.84 76.87 Table 19: Placement scores (%). Five retraining seeds, 2–6. The initial three rows use the original campaign means. Subsequent diagnostics sometimes use rerun controls for paired differences, as detailed below. The oracle has an additional rank-2 teacher-training cost. The warm-up uses part of the step budget with both factors trainable. F.1 Initial gradients and accumulated gradients For each adapted module, the gradient of the training loss with respect to W0W_0 is measured at the unadapted model. Its top two left singular vectors define fixed B. The 64-sequence estimator uses sampling seed 1000. The accumulated estimator uses nested sets so that its first 64 sequences reproduce the original estimator; the full sets contain 272 packed sequences on OpenBookQA and 3,178 sequences on the formatting task. Gradients are accumulated per module in a separate process and reset after each micro-batch; no optimizer update is taken when constructing the basis. The construction records a double-precision SVD for this gradient diagnostic. The top-two share of gradient energy changes little as the sample increases: approximately 0.83 on OpenBookQA and 0.555–0.558 on the formatting task. Capturing much of the initial gradient energy therefore does not guarantee a useful fixed subspace throughout fine-tuning. It does not follow that the gradient is uninformative under a different rank, covariance metric or subspace-refresh rule. For the 64-sequence comparison, the top-band reference is 46.52% on OpenBookQA and a rerun score of 64.30% on the formatting task. The resulting gains are −3.80-3.80 [−5.03,−2.57][-5.03,-2.57] and +2.28+2.28 [0.94,3.62][0.94,3.62] points. The full-gradient comparison uses original top references, giving −3.48-3.48 [−4.65,−2.31][-4.65,-2.31] and +1.79+1.79 [−0.84,4.42][-0.84,4.42] points. Its selected rates are 2⋅10−32· 10^-3 and 5⋅10−35· 10^-3, respectively. F.2 Oracle and warm-up construction The oracle teacher is a free rank-2 LoRA trained on the same training data. An orthonormal basis of its output-factor column space is used for a new rank-2 B-frozen adapter, with A retrained from zero. OpenBookQA uses teacher seeds 0 and 7; the formatting task uses teacher seed 0. The first oracle campaign followed a declared launch decision after an earlier geometric diagnostic: the original trigger did not specify how to aggregate its seven module types, and its strict all-module interpretation was not met. Its stated comparison was retained. Consequently the oracle is presented as a follow-up diagnostic, without claiming an untouched preregistered discovery path. On OpenBookQA, the two teacher bases give gains over the original top reference of 9.36 [8.42,10.30][8.42,10.30] and 8.76 [7.66,9.86][7.66,9.86] points. On the formatting task, the gain over the rerun top reference is 16.51 [15.15,17.88][15.15,17.88] points. Intervals cover retraining seeds conditional on the chosen teacher. They do not average over a representative distribution of teacher subspaces. The original warm-up teacher uses learning rate 2⋅10−32· 10^-3 and direction seed 0 per task; the follow-up adds seeds 301 and 302. Checkpoints are taken after 5, 10 and 25% of steps; the output basis is then fixed and A retrained from zero over the remaining steps. The 10% cuts occur after five optimizer steps on OpenBookQA and 28 on the formatting task, corresponding to about 0.59 and 0.28 epochs. The subsequent B-frozen stage selects 5⋅10−35· 10^-3 on both tasks. The output-subspace extraction is checked against the corresponding full-teacher basis at the last step, with overlaps of 0.999 and 1.000. The 10% gains relative to original top references are 4.64 [3.95,5.33][3.95,5.33] and 7.79 [5.54,10.03][5.54,10.03] points. Additional 5% and 25% gains are 1.04/6.24 on OpenBookQA and 4.98/11.50 on the formatting task, without formal tests. These cuts are not independent teacher replications. They are consistent with improved placement during training but do not establish a monotonic population-level learning curve. F.3 Replication across warm-up direction seeds Two new rank-2 direction runs per task use seeds 301 and 302 at learning rate 2⋅10−32· 10^-3. The 5/10/25% cuts occur after steps 3/5/13 out of 49 on OpenBookQA and 14/28/69 out of 274 on Format. For each saved basis, five new B-frozen runs train A from zero for the remaining steps, using seeds 2–6 and the corresponding rate from the original warm-up selection. Thus each task now has three direction seeds, with five retraining seeds per cut and direction. All new runs use RTX 4000 Ada. Task Warm-up fraction Direction 0 Direction 301 Direction 302 OpenBookQA 5% 0.13 −0.01-0.01 0.00 10% 0.58 0.51[0.32,0.70]0.51\ [0.32,0.70] 0.55[0.35,0.75]0.55\ [0.35,0.75] 25% 0.78 0.70 0.81 Format 5% 0.40 0.45 0.23 10% 0.63 0.55[0.19,0.90]0.55\ [0.19,0.90] 0.49[0.25,0.73]0.49\ [0.25,0.73] 25% 0.93 0.90 1.07 Table 20: Recovery ratios across three direction seeds. The denominator remains the original fixed gap, 8.04 points on OpenBookQA and 12.41 on Format. Brackets for the four new 10% comparisons are paired-t intervals at level 1−0.05/4=98.75%1-0.05/4=98.75\%. Other entries are descriptive point estimates. A reported value of −0.00-0.00 is displayed as 0.00. The four new 10% gains are 4.12 and 4.44 points on OpenBookQA (t=11.63,11.97t=11.63,11.97), and 6.78 and 6.09 on Format (t=6.62,8.72t=6.62,8.72). All exceed t1−0.05/8,4=4.315t_1-0.05/8,4=4.315. None of the recovery intervals has a lower bound of at least 0.7 or an upper bound below 0.3; all remain inconclusive under the specified recovery rule. Direction-level means increase over the three cuts, but no inferential test of a monotonic population trend is claimed. The gains are reported against the original top-band controls, with mean scores 46.52% and 64.43%, using the campaign’s seed-paired statistics. No new top-band reference is trained in this follow-up; in particular, the Format reference is historical rather than the single-GPU-model replication. Recovery intervals use the fixed earlier denominator. Each interval describes retraining variability conditional on one learned basis and does not combine uncertainty across direction seeds. Warm-up directions and the initial gradient. A post hoc measurement, specified before execution and requiring no training, compares the OpenBookQA bases from direction seed 0 with the leading left singular subspaces of the initial gradient (supplementary record, §S24.7). For each orthonormal rank-2 basis Q, the energy share in gradient subspace UkU_k is ‖Uk⊤Q‖F2/‖Q‖F2\|U_k Q\|_F^2/\|Q\|_F^2, with isotropic reference k/doutk/d_out. The layer-median share in the top-two gradient directions ranges across module types from 0.12–0.38 after 5% of training, 0.10–0.33 after 10%, 0.07–0.24 after 25%, and 0.05–0.18 for the oracle. Every value exceeds its isotropic reference. Even the top 32 gradient directions capture only 0.14–0.47 of the oracle basis’s energy. Meanwhile, the recovered share P rises from 0.13 to 0.58, 0.78 and 1.16. Increasing recovery therefore accompanies decreasing alignment in this one-seed, one-pair diagnostic. It does not show that moving away from the gradient causes improvement, but it does show that leading-gradient alignment alone does not characterize the best-performing fixed basis. F.4 Fixed-reference recovery ratios The placement campaigns use fixed reference gaps G0=8.04G_0=8.04 points for OpenBookQA and G0=12.41G_0=12.41 points for the formatting task, measured in earlier campaigns. For a diagnostic gain D, the reported ratio is P=D/G0P=D/G_0. Because G0G_0 is treated as fixed, its intervals propagate uncertainty in D only. The values are: Diagnostic OpenBookQA Format Initial gradient (64) −0.47[−0.63,−0.32]-0.47\ [-0.63,-0.32] 0.18[0.08,0.29]0.18\ [0.08,0.29] Initial gradient (all) −0.43[−0.58,−0.29]-0.43\ [-0.58,-0.29] 0.14[−0.07,0.36]0.14\ [-0.07,0.36] 10% warm-up 0.58[0.49,0.66]0.58\ [0.49,0.66] 0.63[0.45,0.81]0.63\ [0.45,0.81] Oracle, teacher 0 1.16[1.05,1.28]1.16\ [1.05,1.28] 1.33[1.22,1.44]1.33\ [1.22,1.44] Oracle, teacher 7 1.09[0.95,1.23]1.09\ [0.95,1.23] — The campaigns classify a lower bound at least 0.7 as substantial recovery and an upper bound below 0.3 as little recovery; intermediate cases are inconclusive under that rule. The warm-up fails the first criterion on both tasks, while the full-gradient formatting interval crosses 0.3. The oracle clears 0.7 on all three teacher/task combinations. These are conditional thresholds, not unconditional confidence statements about a ratio of two estimated quantities. Control reruns change the actual reference gaps to 8.32 and 12.57 points. On OpenBookQA, rerun bottom B frozen scores 42.76%; the formatting record quotes 59.05% for bottom and 66.31% for random orthonormal. The later provenance audit could not identify a run set for the quoted 59.05% bottom score, which is not used in the main contrasts. The primary ratio retains the original fixed denominator, so these rerun values must not be silently substituted into it. A ratio above one is possible and does not measure a probability or an efficiency advantage. Appendix G Statistical interpretation and experimental provenance G.1 Unit of uncertainty and families of comparisons For a paired difference dsd_s across n=5n=5 seeds, a nominal 95% paired-t interval is d¯±t0.975,4sd/5 d± t_0.975,4\,s_d/ 5. The two-band controls use the more stringent decision threshold t0.9875,4≃3.495t_0.9875,4 3.495 for a two-sided familywise 0.05 comparison. A nominal interval excluding zero is not by itself the corrected decision rule. Tables state when such a rule applies. The factor/band summaries instead retain the original post hoc paired percentile-bootstrap intervals; we do not reconstruct seed-level samples from aggregate scores. Pairing by seed preserves the study’s matched-run structure. It does not remove a systematic GPU imbalance, or propagate the variability of an earlier rate-selection stage. The finite test sample is treated as fixed in the principal analyses. A separate two-level bootstrap on the formatting task resamples seeds and test items; for selected band contrasts and their interaction, intervals widen by 1.8–2.6 times. The B-frozen band contrast and interaction remain nonzero in that check, while the A-frozen band contrast does not. OpenBookQA did not retain the per-question predictions needed for the analogous paired item-level reanalysis. Question Evidence used here Inferential status Is the rank-2 factor contrast larger than the top/bottom contrast? Four complete A/B comparisons Post hoc descriptive comparison; seed-bootstrap intervals for component contrasts Can count or module allocation explain the advantage? Count reversal; within-module controls Two-band families for count reversal; eight-test family for module controls Does the result survive selected training controls? Fresh seeds, one GPU model, four-times tokens, answer-only loss Prespecified campaign comparisons; scope differs by control Does the A-frozen advantage need spectral placement of A? Random orthonormal A, two pairs, rank 2 Prespecified replication; both P1 tests pass two-pair Bonferroni; earlier outcome partly known Does the contrast weaken with rank? Complete rank sweep on one pair; rank-16 checks Descriptive trend; no universal crossover or strict monotonicity claim Does the rank-16 deficit persist with PEFT MiCA? Transferred recipe and shared-recipe follow-up Nominal paired tests; shared-recipe test planned after the first result Can better output placement support adaptation? Oracle, gradient and three warm-up direction seeds Conditional on each basis; four-test correction for new warm-up comparisons Table 21: Evidence map. The paper’s main empirical claim does not inherit confirmatory status merely because some underlying campaigns had prespecified protocols. G.2 Approximate multiplicity audit of the historical record A post hoc audit applies Holm’s step-down calculation at 0.05 to 145 rows of the earlier campaign test inventory. The inventory starts from 151 rows, excludes four descriptive rows and two selection outcomes, and counts a row once even when it contains several quantities. This population is larger than the comparisons retained in this manuscript and excludes the new training campaigns. The 151 inventory rows are in Tables S24–S27 of the supplementary record. Removing the four descriptive rows and two selection outcomes gives the 145 audited rows. For reported t statistics, the audit reconstructs p-values using the recorded degrees of freedom and sidedness. For percentile-bootstrap intervals it instead estimates a standard error from the distance to the relevant endpoint divided by 1.96, then uses a normal tail. Other interval-based decisions are inverted using their stated thresholds. Rows without a statistical test receive p=1p=1 while remaining explicitly outside inferential interpretation. If a row originally declared several components positive, the largest of their p-values is used; otherwise the smallest component value is used. One interval-based row also selects the smaller of two one-sided values. These data-dependent aggregation rules and approximate conversions prevent interpreting the output as rigorous error control over all underlying hypotheses. Under that calculation, 46 of 109 previously positive row decisions remain below 0.05, and 63 do not. The A-frozen advantages on Format / Llama and on both 7B models are retained, along with the Qwen-7B parameter-count control. The direct rank-2 Qwen-1.5B OpenBookQA row and its parameter-count control each have adjusted p=0.062p=0.062. The rank-8 and rank-16 A/B rows and the answer-only row also fail this global sensitivity threshold. The components originally declared positive in the five factor/band decomposition rows survive under the interval approximation; this does not supply a new test of F>|S|F>|S|. The transcription check finds one inconsistent original verdict, a row combining two tests with different thresholds. Six interval-converted rows lie near the Holm threshold. We therefore report the audit as sensitivity evidence and retain the original effect sizes and campaign-specific intervals, rather than relabeling every claim as globally confirmed. At the first Holm step, p≤0.05/145p≤ 0.05/145 corresponds to |t|≥11.34|t|≥ 11.34 for a two-sided test with four degrees of freedom (9.48 for a one-sided test); the threshold varies with degrees of freedom and step. G.3 Sensitivity of small band comparisons For historical band comparisons that did not pass their campaign criterion, we estimate the minimum detectable effect (MDE) at 80% power. For a paired-t test this is the mean difference required by a noncentral-t power calculation, using the observed standard deviation of seed-paired differences and the original significance level. These are retrospective sensitivity estimates with uncertain variance inputs, not proof that an undetected effect exists. Historical comparison Seeds Observed difference MDE Format / Llama, β=.75β=.75 minus top 7 2.21 3.36 Format / Llama, β=.75β=.75 minus β=.5β=.5 7 0.98 4.51 Format / Llama, β=.75β=.75 minus random orthonormal 7 2.42 3.92 Same random comparison, fresh seeds 8 3.36 4.84 Format / Llama, common-rate bottom advantage, r=2r=2 5 0.98 3.29 Format / Llama, common-rate bottom advantage, r=4r=4 5 2.25 3.50 OBQA / Qwen-1.5B, difference of band gaps, r=8r=8 5 0.52 4.16 OBQA / Qwen-7B, trained-versus-fixed B band-gap difference, r=16r=16 5 2.12 4.24 Table 22: Selected retrospective sensitivity estimates (percentage points). Historical selection rules and seed sets are retained. These rows are not all drawn from the central separately tuned two-by-two comparisons. Many five-to-eight-seed comparisons require 3–5 points for 80% power, although the range is wider across the full audit. Two older predicted bottom-band advantages instead have negative estimates: −2.88-2.88 points on rank-2 OpenBookQA (MDE 3.25) and −2.90-2.90 on ARC-Easy (MDE 1.02). Their sign contradicts the prediction; this is distinct from merely failing to detect a small positive gain. These are separate historical contrasts, not the band differences of the central table. The MDE alone is not a significance test of the reverse ordering. Two rows use normal approximations to bootstrap widths rather than noncentral-t inversion: the rank-2 spectral-position contrast (MDE 5.93 points) and the Mistral band component (1.99 points). Two further rows lack a reported mean difference and retain standardized MDEs only, 2.48 and 1.68. These declared limitations, and the use of observed rather than prospectively specified variability, rule out interpreting non-significance as equivalence. G.4 Why recovery is not a causal decomposition Earlier campaign summaries used G=L−bG=L-b, R=u−bR=u-b and I=L−uI=L-u, where L is free LoRA at the reference rank, u is a both-trainable spectral initialization at that rank, and b is a B-frozen score at the larger frozen rank. The identity G=R+IG=R+I holds by construction. It does not establish that R is a separately identified causal contribution of freezing, because moving from b to u also changes rank, parameterization and trainable count. The original recovery criterion asks whether a nominal interval for R lies above 0.7G00.7G_0, with G0G_0 fixed from an earlier campaign and a two-point floor. This tests performance against a fixed numerical benchmark. It does not propagate uncertainty or covariance in an estimated G, and is not a test of R−0.7GR-0.7G under joint sampling. Its strongest interpretation is that spectral initialization need not prevent a trainable adapter from approaching the free reference. The paper therefore places direct A/B comparisons and the parameter-count reversal ahead of this statistic. G.5 Prospective records and deviations Internal protocols specify configurations, seeds, thresholds and interpretations for many campaigns before the corresponding runs. The larger-model comparisons, fresh-seed replication, parameter reversal, rank extension, accumulated gradient, warm-up and answer-only controls have such records. Earlier findings informed which campaigns were proposed, and some records were committed shortly before launch. These are local version-control timestamps, not external immutable registrations. Table R1 of the supplementary record dates the protocols of the earlier campaigns. The broader campaign history includes reused runs, revised selections and machine-failure relaunches. A rank-8 Llama selection was replaced with fresh selection seeds 40–41 and reporting seeds 42–46 after an earlier unregistered sweep. The Qwen formatting replication uses seeds 22–26. Some exploratory common-rate and position comparisons reused earlier runs; they are not promoted to confirmatory evidence here. The oracle launch deviation is described in Appendix F. The new records and departures are summarized below; protocol counts do not make the post hoc synthesis confirmatory. The aggregation procedure checks evaluation size, adapted modules, clipping, norm rescaling, verifier version, random-subspace draw salt, model and LoRA scale before combining runs. Early data failed this check and were separated or replaced. It is a consistency check on recorded configurations, not a substitute for independent reproduction. Follow-up Pushed (2026) Scope and departures Boundary selection 03 Oct, 00:51 Reuses prior selection runs; three engine fixes precede all new runs. Single GPU model 03 Oct, 00:51 Seven arms rerun at their reported rates, seeds 2–6. PEFT MiCA 03 Oct, 00:51 Before runs, addendum distinguishes recipes and training amounts. Shared MiCA recipe 03 Oct, 08:36 Planned after the MiCA result; five new LoRA runs reuse MiCA controls. Module groups 03 Oct, 00:51 Validation uses 500 rather than 400 items; some boundary optima and ties remain. Warm-up directions 03 Oct, 00:51 New direction seeds 301/302; historical top controls and fixed reference gaps. Random A protocol 04 Oct, 21:02 Two rank-2 P1 comparisons and a descriptive rank-16 extension; earlier runs partly known. Random A addendum 04 Oct, 21:12 Pushed after selection submission but before any result; deviations in Appendix E.5. Table 23: Internal prospective records for the new training campaigns. According to the retained records, protocols preceded their new runs. The random-A addendum was pushed after selection submission but before any result. The shared-recipe test was planned after the first MiCA result. The no-training audits are explicitly post hoc. GPU attribution required two declared revisions after its initial execution: configuration grouping was corrected, then ambiguous rounded means were resolved from their table row and column. The final mapping was pushed on 3 October 2026 at 09:13. Matching a rounded score alone is insufficient: several distinct configurations share the same four-decimal mean. For two sensitivity rows, standardized effects replace the planned recovery of run-level variances because the corresponding cells could not be identified reliably. G.6 Hardware, retained artifacts and compute Training uses a shared cluster with RTX 4000 Ada, RTX 2000 Ada, RTX A5000, RTX A4000 and GeForce RTX 3090 GPUs. The final audit identifies 137 distinct historical score cells, classifying 86 as RTX 4000 Ada only and 51 as mixed-GPU. It resolves 133 of 134 distinct recorded score values. Its per-run counts are 577 RTX 4000 Ada, 39 RTX 3090, 33 RTX 2000 Ada, 27 RTX A5000 and 23 RTX A4000, with six missing records: three each in the original rank-2 Format B-top and B-bottom cells. These counts sum to 705 records, of which 699 have a known GPU; 577/699 is 82.5%. Classification follows available metadata, and assignment is not balanced retrospectively. The unmatched score is the 59.05% formatting bottom control quoted in an earlier diagnostic caption. Parameter-reversal and rank-extension campaigns use RTX 4000 Ada throughout, as do the 681 follow-up runs preceding the random-A replication. The seven-arm formatting replication directly addresses the two central cells with missing records (Appendix E.1). Re-evaluation on the original GPU model reproduced the checked scores, whereas 11 of 12 re-evaluations on another GPU model differed; observed shifts were 0.2–1.8 points. This association is consistent with hardware-dependent numerical behavior, but does not isolate its exact kernel-level cause. Seed variation captures random hardware effects only to the extent that assignment varies appropriately; it cannot rule out systematic confounding. Runs failing an SVD reconstruction check were excluded before training. Two earlier runs on a subsequently suspect node were repeated elsewhere and replaced under a rule fixed before inspecting replacement scores. Memory failures caused by concurrent cluster use also led to relaunches. Factors were not retained for every extension run because of storage constraints, limiting retrospective geometric analyses. The recorded software stack is PyTorch 2.14.0 with CUDA 13.0, Transformers 5.17.0, Datasets 5.0.1, Safetensors 0.8.0 and NumPy 2.2.6. Nineteen of the original 1,242 runs predate a cluster upgrade and used PyTorch 2.5.1 with CUDA 12.1. Two belong to reported cells; rerunning them on the newer version changed their scores by −0.16-0.16 and +0.65+0.65 points. The exact association between versions and runs requires the per-run manifests. The reported accounting is 1,242 original runs (256 GPU-hours), 539 later runs (179 GPU-hours) and 838 extension runs (116.1 GPU-hours): 2,619 training runs and approximately 551 GPU-hours. This includes selection and follow-up work beyond the reduced set of comparisons emphasized in this manuscript. It excludes failed or superseded runs and substantial non-training work such as SVD caches, some gradient measurements and evaluation. The broader exploratory project reports 6,545 completed executions and 1,069 GPU-hours, using an execution-based count that is not directly interchangeable with the campaign count. The first follow-up campaigns add 681 runs and 61.3 GPU-hours: boundary selection 24/3.4, single GPU model 35/5.0, PEFT MiCA 15/0.8, shared recipe 5/0.2, module groups 538/45.9, and warm-up seeds 64/6.0 (runs/hours). The random-A replication adds 53 runs and 5.2 GPU-hours (36 selection, 2 extension, 15 reporting), bringing the campaign-based total to 3,353 training runs and approximately 617.6 GPU-hours. The post hoc audits launch no training runs. These totals include research and selection costs, not just the cost of any one adapter. Reproducibility materials. This manuscript includes the retained aggregate scores, configuration definitions, parameter formulas and interpretation rules. Code, protocols and run records are provided in the supplementary material, including the custom-task generator and verifier. Exact reproduction additionally depends on the model and dataset revisions and on the split and selection manifests. The run records allow the seed-level intervals to be recomputed, but the recorded campaign chronology cannot be verified independently of the private repository.