Paper deep dive
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Jonathan Mei, Sang Hyub Kim, Oliver Knitter, Chi Chen, Martin Roetteler
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 10/1/2026, 8:30:30 AM
Summary
The paper introduces ShamAN-Q, a sub-1-bit post-training quantization method for Large Language Models (LLMs) that extends NanoQuant. It replaces NanoQuant's diagonal reconstruction geometry with a dense curvature metric derived from the Shampoo optimizer paradigm. By fitting a Kronecker product to the empirical Fisher information matrix via Kullback-Leibler minimization, ShamAN-Q forms a Mahalanobis reconstruction loss. This approach lowers WikiText-2 perplexity on Qwen3-Base models (0.6B, 1.7B, 4B) at approximately 1 bit per weight (bpw) while maintaining or improving zero-shot accuracy. At 0.8 bpw, it matches NanoQuant's performance at 1.0 bpw.
Entities (10)
Relation Signals (9)
ShamAN-Q → appliesto → Qwen3-Base
confidence 95% · On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity...
ShamAN-Q → extends → NanoQuant
confidence 95% · ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant...
ShamAN-Q → evaluateson → Eleuther LM Evaluation Harness
confidence 90% · matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness.
ShamAN-Q → evaluateson → WikiText-2
confidence 90% · ShamAN-Q lowers WikiText-2 perplexity...
ShamAN-Q → uses → Shampoo
confidence 90% · using a general paradigm popularized by the Shampoo optimizer.
ShamAN-Q → fits → Kronecker product
confidence 85% · ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix...
ShamAN-Q → replaces → diagonal reconstruction geometry
confidence 85% · replacing each its diagonal reconstruction geometry with a tractable dense curvature metric
ShamAN-Q → transforms → ADMM updates
confidence 80% · The continuous ADMM updates from NanoQuant become solutions to Sylvester equations
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback--Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at $\approx$1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at $\approx$0.8 bpw matches the published perplexity of NanoQuant at $\approx$1.0 bpw.
Tags
Links
- Source: https://arxiv.org/abs/2609.38521v1
- Canonical: https://arxiv.org/abs/2609.38521v1
Trouble viewing inline? Open PDF directly →
Full Text
70,508 characters extracted from source content.
Expand or collapse full text
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights Jonathan Mei Sang Hyub Kim Oliver Knitter Chi Chen & Martin Roetteler Affiliation: Applications R&D Affiliation: IonQ Affiliation: College Park, MD 20740, USA Email: jmei,sang,oliver.knitter,chi.chen,martin.roetteler@ionq.co Abstract We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback–Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at ≈ 1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at ≈ 0.8 bpw matches the published perplexity of NanoQuant at ≈ 1.0 bpw. 1 Introduction Figure 1: WikiText-2 perplexity (PPL) (left, lower is better) and zero-shot mean on reasoning tasks in Eleuther LM Evaluation Harness (right, higher is better) vs. Qwen3-Base size at a 1.0 bpw budget. ShamAN-Q matches or improves on NanoQuant in both PPL and zero-shot mean at every size. All points are from Table 1; only reported results are plotted. The energy consumption of language models is an omnipresent, growing concern, affecting all scales from those of frontier data centers down to small edge devices. Awareness of this cost grows alongside demand for data sovereignty, independence from model providers, and fast local intelligence. Beyond the cost of the computation itself, an often overlooked source of language model energy cost is the repeated movement of data. For causal language model inference, processing a prompt in parallel during prefill is often compute-bound. A long prompt produces a large effective batch size B, and assuming n×n× n matrices, the resulting matrix–matrix multiplication moves O(n2)O(n^2) data for O(Bn2)O(Bn^2) computation. Decoding, by contrast, is often bandwidth-bound, since each generated token requires transferring the entire active set of parameters from memory to the compute cores, before counting any host–device or device–device transfers required by models that do not fit on a single accelerator. In the extreme, batch size B=1B=1 reduces to matrix–vector multiplication with O(n2)O(n^2) data movement for only O(n2)O(n^2) computation. Shrinking the stored weights therefore directly attacks the dominant cost of decoding, and large language models are increasingly deployed under memory and bandwidth budgets that their weights alone exceed. Post-training quantization (PTQ) to 4 bits per weight (bpw) is now well established (Frantar et al., 2023), and even 2 bpw is maturing (Lin et al., 2024), but the sub-1-bit regime requires a structural change, since a scalar codebook cannot spend less than one bit on every weight. NanoQuant (Chong et al., 2026) instead represents each decoder linear layer by two low-rank binary factors and two short FP16 scale vectors (Eq. 1). By doing so, NanoQuant changes the dominant storage from one bit per matrix entry to r(m+n)r(m+n) bits per layer at factor rank r≫16r 16. At this compression level, nearly every weight is perturbed, and as a result accuracy depends not only on the size of the reconstruction error but also on which input and output directions absorb it. NanoQuant handles unequal channel scales through diagonal preconditioning, after which its optimization process minimizes a Frobenius reconstruction loss. This diagonal preconditioning acts as a proxy for second-order information specific to each channel, but it does not capture second-order relationships between channels. ShamAN-Q extends NanoQuant by replacing the channel-wise scaling with a dense metric that captures these channel correlations. In this regard, ShamAN-Q is to NanoQuant what Shampoo (Gupta et al., 2018) is to diagonal AdaGrad (Duchi et al., 2011). One-sided Hessian-aware PTQ methods such as GPTQ (Frantar et al., 2023) model input sensitivity while treating all output directions alike. The Fisher Information matrix of a weight matrix captures both, but its mn×mnmn× mn size is prohibitive for an m×nm× n weight. Approximating it with an n×n× n input statistic and an m×m× m output statistic makes this information tractable (Martens and Grosse, 2015) (§3.2). ShamAN-Q estimates dense curvature statistics from a small calibration set while limiting the influence of massive-activation tokens. As quantization changes the model, these statistics are regularized and refreshed; they simultaneously serve to guide low-rank reconstruction and rank allocation at NanoQuant’s total bit budget, thus preserving its binary representation and deployment format. At this budget, ShamAN-Q lowers WikiText-2 perplexity on Qwen3-Base from 27.56 to 22.96 at 0.6B, 19.21 to 16.72 at 1.7B, and 14.29 to 13.80 at 4B while matching or improving zero-shot accuracy on the Eleuther LM Eval Harness; at 0.8 bpw, ShamAN-Q at 0.6B matches NanoQuant’s published 1.0 bpw perplexity (§5). We now describe the primary contributions of this paper: • A dense two-sided reconstruction metric extension of NanoQuant for sub-1-bit PTQ: KL-Shampoo fit, and a fresh input statistic with periodic full refresh (§4.1, §4.3), resulting in a modified ADMM whose continuous updates are solutions to Sylvester equations. Other components, such as the Sign-Value Independent Decomposition (SVID) projection (Pouransari et al., 2020; Xu et al., 2020), dual updates, scale extraction, and packed format structures remain unchanged (§4.3). • A redistribution of NanoQuant’s uniform rank budget at bit parity from a combination of depth, layer type, and a measurement of reconstruction loss for each layer (§4.2). • Consistent decreases in perplexity paired with increases in zero-shot reasoning performance over NanoQuant on Qwen3-Base from 0.6B to 14B at identical ≲1 1-bit budgets, attributed to each component (§5). 2 Related Work Binary and Sub-1-bit Quantization of LLMs. Weight-only PTQ is already an established paradigm at 4 and 2 bpw (Frantar et al., 2023; Lin et al., 2024). Pushing further, binary methods either train with quantization-aware objectives, as in the cases of BitNet (Wang et al., 2023) and OneBit (Xu et al., 2024), or partially binarize and keep salient weights at higher precision, as in the case of PB-LLM (Shang et al., 2023); Tu et al. (2026), in contrast, revisits 1-bit optimization from pre-trained weights. Generally, in-place binarization with full-precision scales is bounded below by one bit per weight, and grouping metadata pushes the effective rate higher. NanoQuant (Chong et al., 2026) breaks this bound in the PTQ setting by factorizing each weight into two low-rank binary matrices and two scale vectors, initialized by a latent-binary ADMM under diagonal preconditioning and refined block by block (§3.1). Hessian-aware PTQ and Kronecker Curvature Statistics. GPTQ (Frantar et al., 2023) demonstrates a form of Hessian-aware PTQ by weighting the reconstruction loss with the input second moment, a one-sided Hessian surrogate that models input sensitivity while treating all output directions alike. The second order optimizers K-FAC (Martens and Grosse, 2015) and Shampoo (Gupta et al., 2018) approximate the layer Fisher by a Kronecker product of input and output statistics, and BaKron (Birnick and Saab, 2026) applies similar Kronecker-structured Hessians to weight quantization. Two estimators fit the Kronecker product to the empirical Fisher directly, the nearest Kronecker product (NKP) in Frobenius norm (Mei et al., 2023; Morwani et al., 2025) and the matrix-normal maximum-likelihood fit of KL-Shampoo (Lin et al., 2026; Sun and Wei, 2026). §3.2 presents both, and in ShamAN-Q we adopt the KL fit because of how it weights massive-activation tokens (Sun et al., 2024). ADMM for Structured Compression. The alternating direction method of multipliers (Boyd et al., 2011) suits objectives that pair a continuous reconstruction loss with a hard discrete constraint. NanoQuant introduces LB-ADMM, which decouples the low-rank reconstruction from the binary structure by using SVID-projected consensus variables. ShamAN-Q retains this effective decoupling and its Euclidean consensus penalties, so that only the continuous subproblems change. These subproblems change from right-sided normal equations (Eq. 5) to two-sided Sylvester equations. 3 Background 3.1 NanoQuant Representation and bit cost. NanoQuant (Chong et al., 2026) deploys each linear layer, with weight ∈ℝm×nW ^m× n and factor rank r, in the two-scale form ^=Diag(1)±1±1⊤Diag(2),±1∈±1m×r,±1∈±1n×r, W=Diag(s_1)\,U_± 1\,V_± 1 \,Diag(s_2), _± 1∈\± 1\^m× r, _± 1∈\± 1\^n× r, (1) using FP16 scale vectors of lengths m and n. Since the signs require r(m+n)r(m+n) bits, and the scales 16(m+n)16(m+n) bits, a rank-r layer costs bits(r)=(r+16)(m+n),BPW=(r+16)(m+n)/mn.bits(r)=(r+16)(m+n), =(r+16)(m+n)/mn. (2) Pipeline. Phase 1, global calibration, runs a calibration set through the full-precision teacher ℳM and collects, per linear layer and token t, the layer input t∈ℝnx_t ^n and output gradient t∈ℝm δ_t ^m. Phase 2, block reconstruction, processes decoder blocks in order. Within a block, Step 1 (TuneFP) tunes the remaining full-precision weights to reproduce the teacher’s block output from the already-quantized prefix’s activations, and is repeated between layer factorizations. Step 2 initializes each linear layer’s binary factors and scales by the LB-ADMM below. Step 3 (TuneLatentSTE) jointly tunes the block’s continuous latents ,U,V and scales 1,2s_1,s_2 under a weighted MSE block loss with the straight-through estimator (Bengio et al., 2013), after which ±1=sign()U_± 1= (U) and ±1=sign()V_± 1= (V) are packed. Phase 3 (TuneScalesKD), tuning only the scale vectors of every layer, minimizes the forward KL divergence between the next-token distributions of the teacher and quantization models. Diagonal preconditioning (Step 2-1). NanoQuant performs one measurement of the per-input-feature activation second moments and the per-output-feature output gradient second moments for the full-precision model, after robust token clipping, and shrinks them toward their mean, in2=1N∑t⊙t,out2=1N∑t⊙t,2←(1−γ)2+γmean(2) 1,d_in^2= 1N _tx_t _t, _out^2= 1N _t δ_t δ_t, ^2←(1-γ)\,d^2+γ\,mean(d^2)\,1, (3) with γ∈[0,1]γ∈[0,1]. These shrunken moments then produce ~in=Diag(in) D_in=Diag(d_in), ~out=Diag(out) D_out=Diag(d_out), the transformed target ~=~out~in W= D_outW D_in, and the loss ∥~∥F2=tr(⊤~out2~in2) E _F^2=tr (E D_out^2E D_in^2 ) where =−^E=W- W and ~=~out~in E= D_outE D_in. This loss comprises the two-sided metric of §3.2 with diagonal statistics. LB-ADMM (Step 2-2). The continuous optimization variables are ∈ℝm×rU ^m× r and ∈ℝn×rV ^n× r, and they correspond with consensus variables UZ_U and VZ_V, which are restricted to the SVID family d,r=Diag()Diag():∈ℝ+d,∈ℝ+r,∈±1d×rC_d,r=\Diag(a)SDiag(b):a _+^d,\ b _+^r,\ S∈\± 1\^d× r\. NanoQuant then solves min,,U,V=U,=V12∥~−⊤∥F2+λ2(∥F2+∥F2)+ℐm,r(U)+ℐn,r(V) _ subarraycU,V,Z_U,Z_V\\ U=Z_U,\ V=Z_V subarray\; 12 W-UV _F^2+ λ2 ( _F^2+ _F^2 )+I_C_m,r(Z_U)+I_C_n,r(Z_V) (4) by a scaled-dual ADMM, with ridge λ and a penalty ρk _k that grows linearly over K iterations. Taking αk=ρk+λ _k= _k+λ, with the consensus target Uk=Uk−UkC_U^k=Z_U^k- _U^k being formed using the scaled dual Uk _U^k, the U update is the right-sided r×r× r normal equation k+1((k)⊤k+αkr)=~k+ρkUk,U^k+1 ((V^k) V^k+ _kI_r )= WV^k+ _kC_U^k, (5) while the V update is analogous. As shown for UZ_U, the consensus variables are projected according to Uk+1=SVID(k+1+Uk)Z_U^k+1= SVID(U^k+1+ _U^k), where SVID takes the sign of its argument and the three largest singular values of its absolute value, and the scaled duals are updated as Uk+1=Uk+k+1−Uk+1 _U^k+1= _U^k+U^k+1-Z_U^k+1 (Appendix C). Scale extraction (Step 2-3). The final proxies K+UKU^K+ _U^K and K+VKV^K+ _V^K are magnitude-balanced. Their signs are become ±1,±1U_± 1,V_± 1, their row-wise mean absolute values give the scales, and the inverse of the diagonal maps return the scales to the original coordinates (Appendix C). 3.2 Kronecker Curvature Statistics: Shampoo and KL-Shampoo Empirical Fisher and the two-sided metric. For calibration token t, the weight gradient is the rank-one matrix tt⊤ δ_tx_t , and the empirical layer Fisher information, in the original coordinates of vec()vec(W), is given by ^cal=∑tvec(tt⊤)vec(tt⊤)⊤∈ℝmn×mn H_cal= _tvec( δ_tx_t )vec( δ_tx_t ) ^mn× mn. A Kronecker approximation can estimate this matrix as ^cal≈⊗ H_cal using an input-side statistic ∈ℝn×nA ^n× n and an output-side statistic ∈ℝm×mG ^m× m (Martens and Grosse, 2015; Gupta et al., 2018). We call these curvature statistics and reserve factors for the low-rank weight matrices U and V. Under column-major vectorization, A and G act respectively on input and output coordinates, and the metric induced on an error E is ‖,2:=tr(⊤)=vec()⊤(⊗)vec(). _G,A^2 :=tr\! (E GEA )=vec(E) (A )vec(E). (6) All computations use the two statistics directly, so the full mn×mnmn× mn Kronecker product never needs to be formed. NanoQuant uses as its loss the diagonal case =~in2A= D_in^2, =~out2G= D_out^2, which is equivalent to =nA=I_n, =mG=I_m in the transformed coordinates. Shampoo. Shampoo (Gupta et al., 2018) preconditions a matrix gradient ∇t=tt⊤ _t= δ_tx_t with two accumulated statistics, Sh=∑t∇t∇t⊤=∑t∥t∥2tt⊤,Sh=∑t∇t⊤∇t=∑t∥t∥2tt⊤,G_Sh= _t _t _t = _t _t ^2\, δ_t δ_t , _Sh= _t _t _t= _t δ_t ^2\,x_tx_t , (7) and updates according to ←−ηSh−1/4∇Sh−1/4W -η\,G_Sh^-1/4\,∇\,A_Sh^-1/4, which corresponds in vectorized form with the preconditioner (Sh1/2⊗Sh1/2)−1/2(A_Sh^1/2 _Sh^1/2)^-1/2. Morwani et al. (2025) show that the square of Shampoo’s approximation is one power-iteration step toward the optimal Kronecker approximation of ^cal H_cal, so the exponent 1/41/4 per statistic is the inverse square root of an approximate curvature. Estimators as token-weighted second moments. Kronecker fits of ^cal H_cal all share the form ∝∑twtxtt⊤G _tw^x_t\, δ_t δ_t , ∝∑twtδtt⊤A _tw^δ_t\,x_tx_t , differing only in their token weights. The nearest-Kronecker-product (NKP) fit (Mei et al., 2023; Morwani et al., 2025) minimizes ∥^cal−⊗∥F H_cal-A _F over positive semidefinite statistics, with fixed-point weights wtx=t⊤tw^x_t=x_t A\,x_t, wtδ=t⊤tw^δ_t= δ_t G\, δ_t; Shampoo’s statistics coem from its first alternation from =nA=I_n, =mG=I_m. The KL-Shampoo fit (Lin et al., 2026; Sun and Wei, 2026) is the matrix-normal maximum-likelihood estimate (,)∈argmin,≻DKL((,^cal)∥(,⊗)),tr()=n,(A,G)∈ _A,G 0D_KL\! (N(0, H_cal)\, \|\,N(0,A ) ), (A)=n, (8) where the trace constraint removes the scale ambiguity (c)⊗(c−1)=⊗(cA) (c^-1G)=A . Stationarity then gives the coupled fixed point =1n∑t(t⊤−1t)tt⊤,=1m∑t(t⊤−1t)tt⊤,G= 1n _t (x_t A^-1x_t )\, δ_t δ_t , = 1m _t ( δ_t G^-1 δ_t )\,x_tx_t , (9) so the token weights are the inverse quadratic forms. The KL-Shampoo optimizer tracks these statistics by an exponential moving average and updates ←−η−1/2∇−1/2W -η\,G^-1/2\,∇\,A^-1/2, with exponent 1/21/2 per statistic because ⊗A now approximates ^cal H_cal directly. 4 ShamAN-Q Figure 2: ShamAN-Q end to end. Phase 1 runs the calibration set through the frozen teacher, fits and shrinks the curvature statistics (γ,γ)(A_γ,G_γ) of every linear layer, and allocates ranks at bit parity. Phase 2 processes decoder blocks in order. The factorized prefix and the untouched full-precision suffix are frozen while block b is tuned (TuneFP), factorized layer by layer by Kron–LB–ADMM under the transported statistics (¯,¯)( A, G) (Figure 3), and refined by latent STE. The statistics of the unfactorized layers are refreshed on the partially quantized model after every Δ blocks. Phase 3 distills only the scale vectors against the teacher, with the sign matrices frozen. Flames mark trained parameters, snowflakes frozen ones, and blue blocks denote differences between ShamAN-Q and NanoQuant. ShamAN-Q keeps the same representation as NanoQuant and similarly adopts a three-phase pipeline (§3.1), but changes the geometry used to fit the low-rank factors at four interfaces. Curvature is estimated during the initial calibration (§4.1), used during rank allocation (§4.2), applied to the continuous factor updates, and refreshed with respect to the partially quantized model (§4.3). All components related to SVID, scale extraction, latent tuning, packing, and scale distillation are unchanged. Figure 2 shows the three phases and marks their changed stages, while Figure 3 details the curvature-aware ADMM fitting of one layer, which is the primary extension over NanoQuant. The complete pipeline is given in Appendix C. Figure 3: ShamAN-Q pipeline for one layer. Calibration fits and shrinks the curvature statistics, yielding the transported metric (¯,¯)( A, G) and diagonal maps (~in,~out)( D_in, D_out). Preconditioning maps W to ~ W. Kron–LB–ADMM returns the continuous weight factors (,)(U,V) and scale vectors (1,2)(s_1,s_2). The transported statistics alter only the continuous /U/V optimization component; Euclidean consensus leaves SVID, dual updates, scale extraction, and sign extraction unchanged from NanoQuant. 4.1 Dense Kronecker Curvature from Global Calibration Curvature estimation. Where NanoQuant keeps only the diagonal of ^cal H_cal (Eq. 3), ShamAN-Q keeps its full Kronecker structure; (,)(A,G) are fitted for each linear layer by three alternating passes of the KL-Shampoo fixed point (Eq. 9) over the calibration set, with Cholesky inverses. The KL fit is preferred to NKP for its token weighting (§3.2). The NKP weights t⊤tx_t A\,x_t are quartic in activation magnitude, so the few tokens corresponding with massive activations (Sun et al., 2024) dominate the leading eigenvalues. The KL weights t⊤−1tx_t A^-1x_t, in contrast, are bounded leverage scores, and the resulting statistics have condition numbers of order 40 instead of several hundred. The estimator consumes only the pairs (t,t)(x_t, δ_t) and never materializes ^cal H_cal. Scale-matched shrinkage. The fitted statistics are regularized before any coordinate change, γ=(1−γ)+γtr()nn,γ=(1−γ)+γtr()mm,γ∈[0,1].A_γ=(1-γ)A+γ\, tr(A)n\,I_n, _γ=(1-γ)G+γ\, tr(G)m\,I_m, γ∈[0,1]. (10) Each identity has the same mean eigenvalue as the statistic it regularizes, so shrinkage changes anisotropy without arbitrarily changing its scale: there is no additional damping term. The diagonal of the shrunken statistic is NanoQuant’s shrunken 2d^2 of Eq. 3, and NanoQuant’s robust token clipping is kept when the statistics are accumulated. The fit and shrinkage of this section form the calibration stage of Figure 3. They run once at calibration and at each periodic refresh, while the diagonal maps, transport, and tempering of §4.3 run per layer. 4.2 Non-uniform Rank Allocation at Bit Parity NanoQuant gives every layer the same bits per weight: setting BPW=βBPW=β in Eq. 2 yields rℓ⋆=βmℓnℓ/(mℓ+nℓ)−16,β=1,r _ =β\,m_ n_ /(m_ +n_ )-16, β=1, (11) giving rℓunir^uni_ when floored to a multiple of 32. ShamAN-Q keeps the total Nbits=∑ℓbitsℓ(rℓuni)N_bits= _ bits_ (r^uni_ ) (bit parity) and redistributes it across layers in steps of 32 with rank ceiling min(mℓ,nℓ) (m_ ,n_ ). Given bℓ∈1,…,Bb_ ∈\1,…,B\, the block of layer ℓ and its type t(ℓ)t( ) (q, k, v, o, gate, up, down), both allocators use the depth prior ωℓ=exp(κ(bℓ−1B−1−12))wt(ℓ) _ = (κ ( b_ -1B-1- 12 ) )\,w_t( ), with κ=0.6κ=0.6, so the last block gets more bits than the first by a factor of e0.6≈1.8e^0.6≈ 1.8. The hand table sets budgets βℓ∝βωℓ _ β\, _ —where the type weights wtw_t are contained in a range from 0.85 (q) to 1.15 (down), taken from the block-loss jump each layer type caused when binarized in the 4B logs—and rounds them to ranks meeting exactly NbitsN_bits. The measured sensitivity allocator instead probes each layer with a short dense-curvature LB-ADMM (§4.3) at three ranks, scores the result by the Gauss–Newton error Jℓ(r)=∥ℓ−^ℓ(r)∥γ,ℓ,γ,ℓ2J_ (r)= _ - W_ (r) ^2_G_γ, ,A_γ, using the layer’s own metric, fits a power law, multiplies its level by the depth prior with all wt=1w_t=1, and hands out 32-rank steps greedily by predicted loss decrease per bit. The probes rank the seven layer types within a block well, but under-value late blocks, which the prior supplies. Appendix B gives the type weights, rounding, and probe protocol. 4.3 Block Reconstruction Pipeline Blocks are processed in order, and within each block every linear layer is factorized according to NanoQuant’s per-layer schedule of §3.1. Step 1 (TuneFP) and Step 3 (TuneLatentSTE, sign extraction, and packing) are unchanged; ShamAN-Q changes only the preconditioning and the continuous solves inside Step 2. Periodic curvature refresh. The Fisher information is local to the full-precision weights, and the model drifts from that point as blocks are binarized. ShamAN-Q corrects for this at two time scales: a per-layer fresh input statistic inside Step 2-1, and a periodic refresh at block boundaries. For every Δ blocks, the statistics of all not-yet-factorized layers are re-estimated with one warm-started pass of Eq. 9 through the current partially quantized model ℳ M, so that later blocks see the statistics they will actually receive, and are shrunk with γ as before. Step 2-1: Dense-Curvature Preconditioning. NanoQuant measures ~in D_in once, on the full-precision model (Eq. 3). Under the block-wise schedule, by the time layer ℓ is factorized, its inputs come from an already quantized prefix and from full-precision layers that TuneFP has just modified. Immediately before ADMM for each layer, we therefore re-measure its input second moment on exactly those inputs, fresh=1N∑t=1Ntt⊤A_fresh= 1N _t=1^Nx_tx_t , tkaen over the N calibration tokens with no gradient weighting. Shrunken by Eq. 10, it then takes the place of γA_γ for that layer, while the output-side statistic γG_γ is that of either the calibration time or most recent refresh. The diagonal preconditioners are the square roots of the diagonals of the shrunken statistics, floored at ϵ>0ε>0 to keep both maps invertible (Eq. 18), while ~ W and ~ E are taken as in §3.1. Rather than refitting in these coordinates, we transport the same shrunken statistics by congruence, which preserves the Mahalanobis error exactly: ¯=~in−1γ~in−1,¯=~out−1γ~out−1,‖γ,γ2=‖~‖¯,¯2, A= D_in^-1A_γ D_in^-1, G= D_out^-1G_γ D_out^-1, _G_γ,A_γ^2= E _ G, A^2, (12) which is obtained by substituting =~out−1~~in−1E= D_out^-1 E D_in^-1 into Eq. 6 and cycling the trace. Notwithstanding the floor, the transported statistics are unit-diagonal correlation matrices, so the dense metric combines NanoQuant’s diagonal preconditioning with (¯,¯)( G, A). Thus NanoQuant is recovered exactly by ¯=n A=I_n and ¯=m G=I_m. The one-bit reconstruction error is large (∥≈0.4 ≈ 0.4–0.5∥0.5\, ), well outside the neighborhood in which the Fisher is an accurate curvature, so before the ADMM solver is applied, each transported statistic is replaced by its trace-preserving power (spectral tempering). Writing ¯=Diag()⊤ A=QDiag(c)Q , we then have ¯⟵∑ici∑icipDiag(p)⊤,p=12, A\; \; _ic_i _ic_i^\,p\;QDiag(c^\,p)Q , p= 12, (13) and likewise for ¯ G. This halves the log-condition number, and as a result, the data term trusts the dominant curvature directions less. The square root is the regret-optimal metric under an uncertain curvature. Tempering is free given the cached eigendecomposition. It leaves the identity unchanged, and beats both p=1p=1 and Shampoo’s p=14p= 14 (§5.4). All uses of ¯,¯ A, G below refer to the tempered statistics; their eigensystems and :=¯~¯M := G\, W\, A are cached per layer (Algorithm 1 in Appendix C). Step 2-2: Dense-Curvature LB-ADMM. Relative to NanoQuant’s objective (Eq. 4), ShamAN-Q replaces the Frobenius reconstruction term by the transported metric and keeps everything else. In the transformed coordinates it solves min,,U,V _U,V,Z_U,Z_V 12‖~−⊤‖¯,¯2+λ2(‖F2+‖F2)+ℐm,r(U)+ℐn,r(V) 12 W-UV _ G, A^2+ λ2 ( _F^2+ _F^2 )+I_C_m,r(Z_U)+I_C_n,r(Z_V) subject to=U,=V. to =Z_U, =Z_V. (14) with UZ_U and VZ_V in the SVID family of §3.1. The reconstruction term is Mahalanobis, but the two augmented consensus penalties are Euclidean. This keeps SVID, dual updates, the ρk _k schedule, and residuals identical to NanoQuant, so only the continuous solves change. Figure 3 follows one layer from calibration through preconditioning, the continuous Kron–LB–ADMM solve, and scale and sign extraction; Algorithm 2 in Appendix C lists the solver in full. Take αk=ρk+λ _k= _k+λ and let Uk=Uk−UkC_U^k=Z_U^k- _U^k be the consensus target formed with the scaled dual Uk _U^k. Holding kV^k fixed, stationarity of Eq. 14 gives ¯k+1Vk+αkk+1=k+ρkUk,Vk=(k)⊤¯k, G\,U^k+1R_V^k+ _kU^k+1=MV^k+ _kC_U^k, _V^k=(V^k) AV^k, (15) and the V update is obtained analogously, by exchanging the roles of (¯,)( G,U) and (¯,)( A,V) and using ⊤M in place of M. These are Mahalanobis Sylvester equations, rather than NanoQuant’s right-sided r×r× r normal equations (Eq. 5). Because the augmented penalties are Euclidean, the left curvature statistic does not cancel. Both have the form +α=CXR+ =B, with C a tempered, transported curvature statistic and ∈ℝr×rR ^r× r positive semidefinite. With the curvature eigensystem =Diag()⊤C=QDiag(c)Q cached once per layer and =Diag()⊤R=SDiag( ν)S recomputed for each update, the solution in the joint eigenbasis is =⊤,i,j=[⊤]i,j/(ciνj+α).X=QYS , _i,j= [Q BS ]_i,j/(c_i _j+α). (16) Because α=ρ+λ>0α=ρ+λ>0, every denominator is positive even for semidefinite statistics. We observe that taking ¯=m G=I_m and ¯=n A=I_n recovers Eq. 5. One eigendecomposition per statistic per layer comprises the added cost (Appendix D), and the SVID projection and its inner iteration count are independent of ¯,¯ A, G (Appendix C). Step 2-3: Unchanged Scale Extraction. The magnitude balancing and scale extraction that NanoQuant uses are applied verbatim to the final proxies in the transformed coordinates, followed by its un-preconditioning through the inverse diagonal maps (Appendix C). The resulting sign matrices initialize the latents ,U,V of Step 3, and the block loss, packing, and TuneScalesKD all come from NanoQuant as well (§3.1). 5 Experiments 5.1 Experimental Setup Implementation and Environment. ShamAN-Q is implemented on top of the released NanoQuant code and follows its protocol wherever §4 does not change it. The 0.6B to 4B results use a single NVIDIA A100-80GB GPU with fp64 eigendecompositions. The 8B and 14B results use fp32 eigendecompositions on two to four GPUs, which was verified to be lossless at 1.7B. Hyperparameters and the engineering measures that keep the dense recipe within budget are given in Appendix A. Models and Datasets. We quantize Qwen3-Base at 0.6B, 1.7B, and 4B, 8B, and 14B parameters (Qwen Team, 2025), calibrating on 128 WikiText-2 (Merity et al., 2016) sequences of 2048 tokens (seed 0). At 8B and 14B the curvature statistics and their refreshes use 512 sequences, while block reconstruction and distillation still use 128. We report both WikiText-2 perplexity at context 2048 and the mean zero-shot accuracy from the Eleuther LM Evaluation Harness (Gao et al., 2024). All numbers are single runs. Baselines. The baseline is NanoQuant at the same total number of bits, presented both as published (Chong et al., 2026) and as our own paper-faithful reproduction. Every ShamAN-Q model spends the same total number of bits as NanoQuant. That is, at the 1 bpw budget, 0.973 bpw for 0.6B and 0.986 bpw for 1.7B and 4B, and at 0.8 bpw, 0.773 / 0.783 / 0.793 / 0.794 bpw for 0.6B / 1.7B / 4B / 8B. At 0.55 bpw the actual values are 0.527 / 0.535 / 0.544 / 0.545 bpw, and the 8B curvature statistics use 128 sequences. Only the rank allocator (§4.2) differs across sizes. Next Token Prediction. Table 1 compares ShamAN-Q with NanoQuant at identical bit budgets. PPL is lower at every model size, and the gap narrows as the model grows. At 0.8 bpw the PPL margin over NanoQuant widens at 1.7B and 4B (13 to 26% and 3 to 11%), narrows to 0.4% at 8B, and at 0.6B and 1.7B ShamAN-Q at 0.8 bpw matches or beats NanoQuant at 1.0 bpw. Figure 1 shows both metrics across model sizes through 14B. The right block reports the zero-shot mean. NanoQuant publishes zero-shot accuracies for Qwen3 only at 8B, and at 4B the rank allocator trades perplexity against zero-shot accuracy (§5.2). Table 1: WikiText-2 perplexity and zero-shot mean across a suite of reasoning tasks for <<1 bpw Qwen3-Base models at the same total bits as NanoQuant (Chong et al., 2026), with both reported and reproduced values. The upper block uses a 1.0 bpw budget, the middle block a 0.8 bpw budget, and the lower block a 0.55 bpw budget. Unreported results are marked with –. PPL ↓ 0-shot mean ↑ bpw Method 0.6B 1.7B 4B 8B 14B 0.6B 1.7B 4B 8B 14B ≈ 1.0 NanoQuant (paper) 27.56 19.21 14.29 12.47 10.92 – – – 0.4894 – NanoQuant (repr.) 29.21 18.76 14.86 12.38 11.49 0.387 0.427 0.455 0.474 0.498 ShamAN-Q 22.96 16.72 13.80 11.82 10.91 0.427 0.448 0.464 0.497 0.506 ≈ 0.8 NanoQuant (paper) 33.79 25.31 19.33 14.83 12.88 – – – – – NanoQuant (repr.) 42.22 22.37 17.99 15.63 – 0.387 0.414 0.431 0.443 – ShamAN-Q 27.51 18.84 17.27 14.77 – 0.398 0.426 0.430 0.447 – ≈ 0.55 NanoQuant (paper) 52.94 33.74 32.86 20.04 17.06 – – – – – NanoQuant (repr.) 62.65 32.40 39.49 22.49 – 0.355 0.380 0.409 0.419 – ShamAN-Q 38.80 25.63 24.24 18.21 – 0.362 0.384 0.397 0.410 – 5.2 Rank Allocation at Bit Parity Table 4 compares the allocators of §4.2 at the same total bits. At 0.6B the measured sensitivity × depth prior is best, 2.5 perplexity below uniform ranks. At 4B the measured allocator gives 13.80 against 13.55 from the hand table but restores the zero-shot mean (0.463 vs. 0.450); both are single runs. The depth prior alone thus does not recover what the type weights in the hand table yields at that width, and the two allocators trade perplexity against zero-shot accuracy. Table 2: Rank allocators at the same bpw. WikiText-2 PPL with zero-shot mean. “Measured” is probed sensitivity × depth prior. Qwen3-Base Hand table Measured 4B 13.55 / 0.450 13.80 / 0.463 Table 3: End-to-end wall-clock time for reproduced NanoQuant and ShamAN-Q on Qwen3-Base at 11 bpw on 1× NVIDIA A100 80 GB. Size NanoQuant ShamAN-Q Overhead 0.6B 1 h 12 1 h 59 65%65\% 1.7B 1 h 18 2 h 15 73%73\% 4B 2 h 30 4 h 30 80%80\% Table 4: Cumulative attribution at Qwen3-1.7B-Base, uniform ranks, 0.986 bpw, WikiText-2 perplexity. Configuration PPL NanoQuant, published 19.21 NanoQuant, our reproduction 18.76 NKP statistics, untempered 19.21 ++ tempering p=12p= 12 17.46 ++ fresh input statistic and refresh 17.28 KL-Shampoo statistics, untempered 17.04 ++ tempering p=12p= 12 (ShamAN-Q) 16.72 5.3 Quantization Cost ShamAN-Q uses the same deployment format as NanoQuant, so inference kernels, memory footprint, and decoding throughput are unchanged. The cost is incurred during quantization, as shown in Table 4. Appendix D gives the asymptotic accounting and Appendix A the timing breakdown and cost measures. 5.4 Ablation Studies Table 4 adds the modifications of §4 cumulatively at 1.7B with uniform ranks. Our reproduction of NanoQuant is already 0.45 below the published number. A dense NKP fit without tempering gives nothing over the diagonal baseline. Tempering the same statistics comprises the single largest step, and replacing the NKP fit by the KL-Shampoo fit improves both the untempered and the tempered models. Only the tempering and rank-allocation steps are individually outside noise. The fresh input statistic and periodic refresh of §4.3 are a consistent gain at every block boundary, though their aggregate effect in Table 4 is within single-run noise. For the tempering exponent, p=1p=1 (none) and p=1/4p=1/4 (Shampoo) were both worse than p=1/2p=1/2 at 1.7B. 6 Conclusion We introduced ShamAN-Q, which replaces the diagonal reconstruction geometry in NanoQuant with a dense metric estimated by a KL-Shampoo-style Kronecker fit of the empirical Fisher Information matrix. We also identify useful processing techniques for robust application of the curvature estimated under uncertainty, including shrinkage, tempering, and periodic refreshes. The continuous ADMM updates of NanoQuant then become Sylvester equation solutions, which can be computed in a cached eigenbasis, while other components of NanoQuant, including crucially the packed format, remain unchanged, so inference can inherit the speed of the customized kernel. At the same total number of bits, ShamAN-Q lowers WikiText-2 PPL and increases zero-shot reasoning performance of ≲1 1 bpw Qwen3-Base models from 0.6B to 14B. Future work includes parallelizing the computation and using further approximations to the dense curvature statistics that yield fast matrix multiplication, both to reduce wall clock time. In addition, as the depth prior and calibration budget were tuned at 0.6B, tuning would likely help, and understanding scaling behavior is left as a promising direction for future investigation. AI use statement In this work, we used generative AI tools for implementing methods. We have not used generative AI tools for proposing or refining hypotheses, designing or providing feedback on research methodology or experiments, or interpreting results. Generating synthetic data sets, helping develop theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, assisting in the writing of proofs, assisting with translation, cleaning and reformating datasets, and supporting qualitative and thematic data analysis are not applicable to this work. Additionally, we used generative AI tools for creating scientific figures, suggesting experimental parameters, brainstorming, identifying some relevant literature, and some editing. We have reviewed all AI-assisted work. LLM-generated code was verified via tests reviewed by humans prior to implementation. AI-generated figures were produced as TikZ that was manually revised by humans, and AI-generated writing was manually reviewed by humans. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. Ethics statement This work is intended to reduce the energy consumption required to run AI models, which has been a globally increasing concern due to scarcity of resources, limited production capacity, and environmental impact. On the surface, we might expect that reducing the energy cost of model inference would have a positive impact on these issues. However, this should be considered in the context of Jevon’s paradox, where a decreasing energy cost for using an item results in increased aggregate usage due to more widespread access and adoption to the point that total energy consumption due to the item increases. This effect, while not guaranteed, remains a possibility here. Reproducibility statement To aid with reproducibility, we provide source code with configuration files to run experiments found in the paper, which we intend to open source as part of the publication. The implementation details of the algorithms and relevant hyperparameters are contained within §4 and appendices. In addition, in the appendices we have included some discussion of directions that were unsuccessful in our explorations. References Bengio et al. (2013) Y. Bengio, N. Léonard, and A. Courville Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432. Cited by: §3.1. Birnick and Saab (2026) J. Birnick and R. Saab BaKron: efficient quantization with kronecker-factored hessians. arXiv preprint arXiv:2608.06291. Cited by: §2. Boyd et al. (2011) S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3 (1), p. 1–122. Cited by: §2. Chong et al. (2026) H. Chong, D. Kim, C. Kim, and M. Choi NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1, §2, §3.1, §5.1, Table 1. Duchi et al. (2011) J. Duchi, E. Hazan, and Y. Singer Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12 (61), p. 2121–2159. External Links: Link Cited by: §1. Frantar et al. (2023) E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh GPTQ: accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations, Cited by: §1, §1, §2, §2. Gao et al. (2024) L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac’h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou The language model evaluation harness. Zenodo. External Links: Document, Link Cited by: §5.1. Gupta et al. (2018) V. Gupta, T. Koren, and Y. Singer Shampoo: preconditioned stochastic tensor optimization. In International Conference on Machine Learning, Cited by: §1, §2, §3.2, §3.2. Lin et al. (2024) J. Lin, J. Tang, H. Tang, S. Yang, W. Chen, W. Wang, G. Xiao, X. Dang, C. Gan, and S. Han AWQ: activation-aware weight quantization for LLM compression and acceleration. In Proceedings of Machine Learning and Systems (MLSys), Cited by: §1, §2. Lin et al. (2026) W. Lin, S. C. Lowe, F. Dangel, R. Eschenhagen, Z. Xu, and R. Grosse Understanding and improving Shampoo and SOAP via Kullback-Leibler Minimization. In International Conference on Learning Representations, C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (Eds.), Vol. 2026, p. 92799–92826. External Links: Link Cited by: §2, §3.2. Martens and Grosse (2015) J. Martens and R. Grosse Optimizing neural networks with kronecker-factored approximate curvature. In International Conference on Machine Learning, Cited by: §1, §2, §3.2. Mei et al. (2023) J. Mei, A. Moreno, and L. Walters Kradagrad: kronecker approximation-domination gradient preconditioned stochastic optimization. In Uncertainty in Artificial Intelligence, p. 1412–1422. Cited by: §2, §3.2. Merity et al. (2016) S. Merity, C. Xiong, J. Bradbury, and R. Socher Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843. Cited by: §5.1. Morwani et al. (2025) D. Morwani, I. Shapira, N. Vyas, E. Malach, S. Kakade, and L. Janson A new perspective on shampoo’s preconditioner. International Conference on Learning Representations 2025, p. 5802–5822 (en). Cited by: §2, §3.2, §3.2. Pouransari et al. (2020) H. Pouransari, Z. Tu, and O. Tuzel Least squares binary quantization of neural networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), p. 2986–2996. Cited by: 1st item. Qwen Team (2025) Qwen Team Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §5.1. Shang et al. (2023) Y. Shang, Z. Yuan, Q. Wu, and Z. Dong PB-LLM: partially binarized large language models. arXiv preprint arXiv:2310.00034. Cited by: §2. Sun et al. (2024) M. Sun, X. Chen, J. Z. Kolter, and Z. Liu Massive activations in large language models. In First Conference on Language Modeling (COLM), Note: arXiv:2402.17762 Cited by: §2, §4.1. Sun and Wei (2026) R. Sun and E. Wei Pro-KLShampoo: Projected KL-Shampoo with Whitening Recovered by Orthogonalization. arXiv preprint arXiv:2605.06316. Cited by: §2, §3.2. Tu et al. (2026) Z. Tu, J. Li, Y. Xi, S. Liu, C. Liu, H. Chen, J. Hu, and Y. Wang Rethinking 1-bit optimization leveraging pre-trained large language models. In Proceedings of the 43rd International Conference on Machine Learning (ICML), External Links: Link Cited by: §2. Wang et al. (2023) H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y. Wu, and F. Wei BitNet: scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453. Cited by: §2. Xu et al. (2020) J. Xu, X. Chen, S. Hu, J. Yu, X. Liu, and H. Meng Low-bit quantization of recurrent neural network language models using alternating direction methods of multipliers. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 7939–7943. Cited by: 1st item. Xu et al. (2024) Y. Xu, X. Han, Z. Yang, S. Wang, Q. Zhu, Z. Liu, W. Liu, and W. Che OneBit: towards extremely low-bit large language models. In Advances in Neural Information Processing Systems, Cited by: §2. Appendix A Implementation Details Hyperparameters. All runs follow the protocol used by NanoQuant wherever §4 does not change it: 128 WikiText-2 calibration sequences of 2048 tokens selected with seed 0 (512 for the curvature statistics at 8B and 14B), shrinkage γ=0.2γ=0.2, NanoQuant’s robust token clipping, K=400K=400 LB-ADMM iterations with the linear ρk _k schedule, two FP16 scale vectors per layer, and TuneFP/TuneLatentSTE/TuneScalesKD with learning rates 10−4/10−5/10−610^-4/10^-5/10^-6, batch sizes 4/1/14/1/1, 8 epochs each (at 8B and 14B the first two stages use 2, 4, 6, and 8 epochs for the q and k, v and o, gate, and up and down layers) with cosine decay, NanoQuant’s weighted MSE block loss, and forward-KL scale-only distillation. ShamAN-Q adds three KL-Shampoo passes at calibration, the tempering exponent p=12p= 12, the refresh period Δ=7 =7 for the 28-block 0.6B and 1.7B models and Δ=9 =9 for the 36-block 4B model and 8B models, and Δ=10 =10 for the 40-block 14B model, the fresh input statistic, and the rank allocator listed in Table 1. The measured allocator probes each layer at 0.5,1,1.5×rℓ⋆\0.5,1,1.5\× r _ at 0.6B and 1.7B and at 0.6,1,1.4×rℓ⋆\0.6,1,1.4\× r _ from 4B upward, with K=50K=50. Every n×n× n and m×m× m eigendecomposition in the reported runs is computed in fp64. Keeping the dense recipe within budget. The recipe costs more than NanoQuant per block: dense statistics per layer, one eigendecomposition per statistic per ADMM call, the fresh input measurement, the refresh passes, the rank probe, and late blocks whose allocated ranks reach 2560 at 4B (§5.3). With allocated ranks at 4B, ADMM accounts for 260 s of a 473 s block at block 7, and for about 40 % of the block with uniform ranks. Four measures keep this within budget. (i) The fp32 statistics of all layers do not fit next to the model at 4B (about 36 GB), so the calibration passes accumulate the Kronecker statistics on the GPU over layer groups sized to a memory budget instead of streaming to the CPU, reducing 4B calibration from about 5 h to about 35 min. The KL fit’s per-pass inverses are Cholesky solves on the accumulation device, reducing 1.7B calibration from 55 to 9 min. (i) The fresh input statistic of §4.3 is measured once per shared-input group and its eigendecomposition cached for the group’s ADMM calls, saving three of seven measurements and three large eigendecompositions per block. (i) The n×n× n eigendecompositions can run in fp32: the eigenvalues are clamped and the result feeds a sign projection, and 1.7B perplexity was unchanged (16.66 vs. 16.72 with fp64). The reported 1.7B and 4B results used fp64, so this is a cost measure only. (iv) Calibration statistics, the rank probe, every per-layer ADMM solution (content-addressed by weight bytes, curvature tensors, and ADMM settings), per-block checkpoints, and the pre-distillation model are cached, so an interrupted run resumes from its last block in minutes. Refresh costs are small, as demonstrated by how three refreshes cost 4 min in total at 1.7B. The rank probe cost is given in Appendix B. Appendix B Rank Allocation Details This appendix completes §4.2. Both allocators start from the uniform ranks rℓunir^uni_ of Eq. 11, keep their total NbitsN_bits, and move ranks in steps of 32 with ceiling min(mℓ,nℓ) (m_ ,n_ ). Hand table. The per-layer budget is βℓ=βζωℓ _ =β\,ζ\, _ with the depth prior ωℓ _ of §4.2 and type weights w=(q 0.85,k 0.90,v 1.10,o 1.00,gate 1.00,up 1.05,down 1.15)w=(q\ 0.85,\ k\ 0.90,\ v\ 1.10,\ o\ 1.00,\ gate\ 1.00,\ up\ 1.05,\ down\ 1.15), set from the median relative block-loss jump each layer type caused when binarized in the 4B logs, and ζ chosen so that ∑ℓβℓmℓnℓ=β∑ℓmℓnℓ _ _ m_ n_ =β _ m_ n_ . Ranks are obtained from βℓ _ by Eq. 11, floored to multiples of 32, and then moved in ±32± 32 steps, largest relative deviation from βℓ _ first, until the total equals NbitsN_bits. Measured sensitivity times depth prior. After calibration, every layer is solved by a short run of the dense-curvature LB-ADMM of §4.3 (K=50K=50) at the three candidate ranks 0.5,1,1.5×rℓ⋆\0.5,1,1.5\× r _ at 0.6B and 1.7B and 0.6,1,1.4×rℓ⋆\0.6,1,1.4\× r _ from 4B upward, and the deployed matrix is scored by the Gauss–Newton weight error Jℓ(r)=‖ℓ−^ℓ(r)‖γ,ℓ,γ,ℓ2,J_ (r)= _ - W_ (r) _G_γ, ,A_γ, ^2, (17) with the layer’s own shrunken statistics before the trace normalization, so that values are comparable across layers. A power law Jℓ(r)=eaℓr−ηℓJ_ (r)=e^a_ r^- _ is fitted through the three probes and its level multiplied by the depth prior of §4.2 with all wt=1w_t=1. Starting from rank 32 everywhere, 32-rank steps are handed to the layer with the largest predicted loss decrease per added bit, (Jℓ(r)−Jℓ(r+32))/(32(mℓ+nℓ)) (J_ (r)-J_ (r+32) )/ (32(m_ +n_ ) ), until NbitsN_bits is reached. The measured curves rank the seven layer types within a block well (Spearman 0.80.8–0.960.96 against logged block-loss jumps) but under-value late blocks, which is what the prior supplies. The probe shares each statistic’s eigendecomposition across its three candidate ranks and is cached, taking 707 s at 0.6B. Appendix C Algorithms This appendix lists the procedures of §4.3 and the unchanged NanoQuant components they call (§3.1), written in the transformed coordinates. Diagonal maps and transformed target (Step 2-1). With the maximum applied elementwise, the diagonal preconditioners are in _in =maxdiag(γ),ϵn, = \diag(A_γ), 1_n\, ~in D_in =Diag(in), =Diag(d_in), out _out =maxdiag(γ),ϵm, = \diag(G_γ), 1_m\, ~out D_out =Diag(out), =Diag(d_out), (18) and the transformed target, transformed error, and cached metric target are ~=~out~in,~=~out~in,:=¯~¯, W= D_outW D_in, E= D_outE D_in, := G\, W\, A, (19) where M uses the tempered transported statistics of Eq. 13 and is cached together with their eigensystems. Algorithm 1 lists the step. Algorithm 1 PrepareMetricTarget for one layer (Step 2-1). 0: W, shrunken statistics (γ,γ)(A_γ,G_γ), with γA_γ from the fresh measurement of §4.3, floor ϵε, tempering exponent p 1: Form (~in,~out)( D_in, D_out) by Eq. 18 2: ~←~out~in W← D_outW D_in 3: Transport to (¯,¯)( A, G) by Eq. 12, do not refit 4: Eigendecompose ¯,¯ A, G and temper both by Eq. 13 5: ←¯~¯M← G W A, cache the tempered eigensystems 6: return ~ W, both diagonal maps, and the tempered metric state Continuous block updates (Step 2-2). Set αk=ρk+λ _k= _k+λ. Holding kV^k fixed, define Vk=(k)⊤¯k,Uk=k+ρkUk,R_V^k=(V^k) AV^k, _U^k=MV^k+ _kC_U^k, (20) so that Eq. 15 reads ¯k+1Vk+αkk+1=Uk G\,U^k+1R_V^k+ _kU^k+1=B_U^k. After this update, define Uk+1=(k+1)⊤¯k+1,Vk=⊤k+1+ρkVk,R_U^k+1=(U^k+1) GU^k+1, _V^k=M U^k+1+ _kC_V^k, (21) which yields ¯k+1Uk+1+αkk+1=Vk. A\,V^k+1R_U^k+1+ _kV^k+1=B_V^k. (22) Both systems are solved by Eq. 16, and these two solves are denoted by KronUSolve and KronVSolve in Algorithm 2. Euclidean proxy projection. For a proxy ∈ℝd×rP ^d× r, SVID computes =sign(),(,σ,)=TopSingularTriplet(||),S= (P), (a,σ,b)= TopSingularTriplet(|P|), (23) with nonnegative unit vectors ,a,b and zeros in P mapped to +1+1 by sign . Defining =σp= σa and =σq= σb, the projected proxy is SVID()=(σ⊤)⊙=Diag()Diag(). SVID(P)=( ) =Diag(p)SDiag(q). (24) The triplet is obtained by the same power iteration as NanoQuant. Since the consensus penalties in Eq. 14 are Euclidean, neither Eq. 24 nor its inner iteration count depends on ¯ A or ¯ G. Scale extraction (Step 2-3). The final proxies U,VP_U,P_V live in the transformed coordinates of Eq. 19. NanoQuant’s magnitude balancing (its Eq. 7) and scale extraction (its Eq. 8) are applied to them verbatim, yielding sign matrices U,VS_U,S_V and transformed scale vectors ~1,~2 s_1, s_2. The return to the original weight coordinates is 1=out−1⊙~1,2=in−1⊙~2,s_1=d_out^-1 s_1, _2=d_in^-1 s_2, (25) which is exactly NanoQuant’s un-preconditioning step with the diagonal maps of Eq. 18. Finalize in Algorithm 2 denotes these operations. Algorithm 2 Kron-LB-ADMM for one transformed layer (Step 2-2). 0: Prepared state from Algorithm 1, rank r, iterations K, penalty schedule ρk _k, ridge λ 1: Initialize 0,0U^0,V^0 as in NanoQuant 2: U0←SVID(0)Z_U^0← SVID(U^0), V0←SVID(0)Z_V^0← SVID(V^0) 3: Initialize scaled duals U0,V0 _U^0, _V^0 4: for k=0,…,K−1k=0,…,K-1 do 5: Uk←Uk−UkC_U^k _U^k- _U^k, Vk←Vk−VkC_V^k _V^k- _V^k 6: k+1←KronUSolve(k,Uk,¯,¯,,ρk,λ)U^k+1← KronUSolve(V^k,C_U^k; G, A,M, _k,λ) 7: k+1←KronVSolve(k+1,Vk,¯,¯,,ρk,λ)V^k+1← KronVSolve(U^k+1,C_V^k; G, A,M, _k,λ) 8: Uk+1←SVID(k+1+Uk)Z_U^k+1← SVID(U^k+1+ _U^k) 9: Vk+1←SVID(k+1+Vk)Z_V^k+1← SVID(V^k+1+ _V^k) 10: Uk+1←Uk+k+1−Uk+1 _U^k+1← _U^k+U^k+1-Z_U^k+1 11: Vk+1←Vk+k+1−Vk+1 _V^k+1← _V^k+V^k+1-Z_V^k+1, update Euclidean residuals 12: end for 13: U←K+UKP_U ^K+ _U^K, V←K+VKP_V ^K+ _V^K 14: return Finalize(U,V,~out,~in) Finalize(P_U,P_V, D_out, D_in) End-to-end pipeline. Algorithm 3 shows the complete pipeline. Calibration statistics are first collected from the full-precision teacher, so every initial KL-Shampoo fit is defined in the original layer coordinates, and ranks are allocated once. After TuneFP and the fresh measurement, ~ W, the transported and tempered statistics, and M are recomputed for the current target. Latent tuning and final knowledge distillation act on ,,1,2U,V,s_1,s_2 exactly as in NanoQuant. Algorithm 3 Dense-curvature ShamAN-Q. 0: Full-precision teacher ℳM, calibration set calX_cal, bit budget β and allocator (§4.2), γ,ϵ,p,Δγ,ε,p, , ADMM parameters K,ρ,λK,ρ,λ, tuning budgets Tpre,Tpost,TglobT_pre,T_post,T_glob 1: // Phase 1: global calibration and rank allocation 2: Collect (t,t)(x_t, δ_t) for every linear layer on calX_cal through ℳM 3: for each linear layer ℓ do 4: (,)(ℓ)←KL-Shampoo(A,G)^( )← KL-Shampoo by Eq. 9, three passes, shrink by Eq. 10 5: end for 6: Allocate ranks rℓ\r_ \ at bit parity (§4.2), probing with K=50K=50 if the measured allocator is used 7: // Phase 2: block reconstruction 8: ℳ^←ℳ M 9: for blocks b=1,…,Bb=1,…,B do 10: if b>1b>1 and (b−1)modΔ=0(b-1) =0 then 11: Refresh (,)(ℓ)(A,G)^( ) of all unfactorized layers with one warm-started pass of Eq. 9 through ℳ M, shrink 12: end if 13: b←ℳ^<b(cal)X_b← M_<b(X_cal), b←ℬbFP(b)Y_b _b^FP(X_b) 14: TuneFP(b,b,b,Tpre) TuneFP(b,X_b,Y_b;T_pre) // Step 1 15: for shared-input groups g∈bg∈ b (q k v, o, gate up, down) do 16: Measure fresh(g)A_fresh^(g) (§4.3) on the current inputs of g, shrink by Eq. 10 17: for linear layers ℓ∈g ∈ g with weight (ℓ)W^( ) do 18: PrepareMetricTarget (Algorithm 1) with γ←fresh,γ(g)A_γ _fresh,γ^(g) and the current γ(ℓ)G_γ^( ) // Step 2-1 19: (,,1,2)(ℓ)←Kron-LB-ADMM(U,V,s_1,s_2)^( )← Kron-LB-ADMM at rank rℓr_ by Algorithm 2 // Steps 2-2, 2-3 20: end for 21: end for 22: TuneLatentSTE(b,b,b,Tpost) TuneLatentSTE(b,X_b,Y_b;T_post) over ,,1,2U,V,s_1,s_2 // Step 3 23: for linear layers ℓ∈b ∈ b do 24: ±1(ℓ)←sign((ℓ))U_± 1^( )← (U^( )), ±1(ℓ)←sign((ℓ))V_± 1^( )← (V^( )) 25: PackBinary(±1(ℓ),±1(ℓ)) PackBinary(U_± 1^( ),V_± 1^( )) 26: end for 27: end for 28: // Phase 3: model reconstruction 29: TuneScalesKD(ℳ^,ℳ,cal,Tglob) TuneScalesKD( M,M,X_cal;T_glob) over 1,2s_1,s_2 30: return ℳ M Appendix D Complexity Analysis Table 5 separates calibration, preparation, and one outer ADMM sweep for a layer with N calibration tokens. The dense setup includes forming M and eigendecomposing the two transported statistics. Table 5: Per-layer costs for an m×nm× n weight, factor rank r, and N calibration tokens. Component NanoQuant diagonal geometry ShamAN-Q dense geometry Curvature state Θ(m+n) (m+n) Θ(m2+n2) (m^2+n^2) statistics and eigensystems Calibration fit Θ(N(m+n)) (N(m+n)) Θ(N(m2+n2)+m3+n3) (N(m^2+n^2)+m^3+n^3) per pass, three passes Fresh input statistic – Θ(Nn2) (Nn^2) per shared-input group Refresh – one fit pass over unfactorized layers every Δ blocks Rank probe – three K=50K=50 ADMM solves per layer, once Setup Θ(mn) (mn) Θ(m3+n3+m2n+mn2) (m^3+n^3+m^2n+mn^2) One outer sweep Θ(mnr+(m+n)r2+r3) (mnr+(m+n)r^2+r^3) Θ(mnr+(m2+n2)r+(m+n)r2+r3) (mnr+(m^2+n^2)r+(m+n)r^2+r^3) SVID inner work Θ(Kin(m+n)r) (K_in(m+n)r) unchanged Each layer performs two continuous solves per outer iteration and two SVID projections at initialization plus two per iteration. For representation x∈diagonal,densex∈\diagonal,dense\, its reconstruction runtime can be decomposed as Tx,ℓ=Sx,ℓ+Kout,ℓCx,ℓouter+(Kout,ℓ+1)CℓSVID+Ndiag,ℓDx,ℓ,T_x, =S_x, +K_out, C_x, ^outer+(K_out, +1)C_ ^SVID+N_diag, D_x, , (26) where S is setup, CouterC^outer is one pair of continuous solves, CSVIDC^SVID is one pair of proxy projections, and D covers diagnostic evaluations. This accounting makes the one-time dense setup explicit rather than hiding it in the iteration cost. In practice, the (m2+n2)r(m^2+n^2)r term does not dominate: at 4B, where allocated ranks reach r≈1500r≈ 1500–25002500 and each layer performs 800 r×r× r eigendecompositions over K=400K=400 iterations, removing every n2rn^2r product changes per-layer ADMM time by only 5–8 %, so the per-iteration r×r× r eigendecomposition dominates instead (Appendix E). Appendix E Design Alternatives Not Adopted The following variants were evaluated and are not part of ShamAN-Q. Variants within noise. A dense Mahalanobis block loss for TuneFP and TuneLatentSTE (NanoQuant’s weighted MSE is kept), a spike-plus-flat projection of the curvature statistics, distilling the latent binaries in the KD stage, feature distillation, trainable norm weights, and best-epoch selection in the KD stage, a logit-level objective for the last blocks, lifting the rank ceiling, and spending the rounding remainder to reach exactly 1.0 bpw were all within single-run noise. An explicit per-rank middle scale vector between the two sign matrices (§4.3), carried from ADMM through tuning and KD, was 3.5–5 perplexity worse at 0.6B: a mean-one rebalanced export of the same vector was within noise, and a sweep of its learning rate was flat to negative. On the 0.6B four-block screen used for tuning-budget changes (block-3 perplexity 14.15 for the control, single-run noise about 0.2), a plateau stop that ends an epoch loop once the block loss improves by less than 10−210^-2 was within noise (14.23), while running one TuneFP round per shared-input group (four rounds per block instead of seven) cost +0.28+0.28, outside noise, so every round matters, including those between layers that read the same activation. Speed variants. An inexact Sylvester solve (a stale eigenbasis as preconditioner with Rayleigh, QR, and conjugate-gradient rungs) saved about 10 % of ADMM time at 1.7B and nothing at 4B, and early stopping of ADMM never fires: at iteration 400 every layer still flips between 30 and 20,000 signs per iteration under the linear ρk _k schedule. A low-rank-plus-identity Sylvester step for spectrally projected statistics, which keeps the ksk_s largest and kdk_d smallest eigenvalues of each transported statistic exactly and replaces the rest by their geometric mean so that each update costs O(nkr+nr2)O(nkr+nr^2) instead of O(n2r)O(n^2r) with k=ks+kdk=k_s+k_d, saved only 5–8 % of 4B ADMM time, because the r×r× r eigendecomposition and not the n2rn^2r products dominates at those widths (Appendix D): the projection itself cost +0.10+0.10 to +0.34+0.34 perplexity at block 3 with k=64k=64 per side and was within noise at k=256k=256. It is not used.