Paper deep dive
Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization
I Kennedy, T Kennedy
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/30/2026, 7:03:20 AM
Summary
The paper introduces RAM (Resource-Aware Mixed-precision), a data-free quantization framework that uses single random Gaussian probes to estimate quantization error sensitivity. It demonstrates that quantization error is spectrally flat, allowing accurate estimation with few probes. RAM allocates bit-widths via a knapsack solver based on propagated probe scores, outperforming calibration-based methods like GPTQ and HAWQ-V2 across various model sizes (8B to 400B) without requiring calibration data.
Entities (10)
Relation Signals (7)
RAM+ → appliesto → Qwen3.5-35B-A3B
confidence 95% · Isolated and propagated scores rank tensors independently on Qwen3.5-35B-A3B
RAM+ → appliesto → Qwen3-8b
confidence 95% · On Qwen3-8B the propagated probe ties HAWQ-V2 at matched bytes.
Quantization Error → hasproperty → Spectral Flatness
confidence 95% · round-to-nearest error is spectrally flat.
RAM+ → uses → Random Gaussian Probes
confidence 95% · A single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of a layer's quantization error.
RAM+ → outperforms → GPTQ
confidence 90% · at matched bytes on Qwen3.8-27B, a block-output probe beats... an oracle that allocates from the real-activation objective.
RAM+ → ties → HAWQ-V2
confidence 90% · On Qwen3-8B the propagated probe ties HAWQ-V2 at matched bytes.
RAM+ → uses → Knapsack Solver
confidence 90% · A knapsack solver allocates bits under an exact byte budget
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of a layer's quantization error. The estimator is well-behaved because round-to-nearest error is spectrally flat. Across 1,683 tensors from a 35B MoE and a 9B dense model, effective dimensionality is 0.93 to 0.96 times the i.i.d. noise value of the same shape, and on the MoE the median is unchanged from 2-bit to 8-bit. The probe coefficient of variation is predictable from tensor shape. One probe measures per-tensor sensitivity to within 4 to 7%; twenty probes reach 1.3 to 1.4%.RAM applies the propagated form of this estimator to budget-targeted mixed-precision quantization with no calibration data. Gaussian probes carrying the network's own input statistics score every tensor at six bit-widths. A knapsack solver allocates bits under an exact byte budget, with guardrails against catastrophic 2-bit assignments. One probe pass serves any budget. Isolated and propagated scores rank tensors independently on Qwen3.5-35B-A3B (Spearman -0.01), yet the propagated probe rank-correlates 0.81 to 0.83 with the GPTQ layer objective from real activations, while the isolated estimator is uncorrelated with it. That objective is the wrong allocation target: at matched bytes on Qwen3.8-27B, a block-output probe beats a vendor IQ3_M mix and an oracle that allocates from the real-activation objective. On Qwen3-8B the propagated probe ties HAWQ-V2 at matched bytes. Across seven architectures from 8B to 122B, with probe timing up to a 400B model in nine minutes on one workstation, RAM reaches 3.5 to 13.6% lower median WikiText-2 perplexity than size-comparable uniform 4-bit builds on the tested MoE models. (Black Sheep Ai this http URL)
Tags
Links
- Source: https://arxiv.org/abs/2609.33923v2
- Canonical: https://arxiv.org/abs/2609.33923v2
Trouble viewing inline? Open PDF directly →
Full Text
107,563 characters extracted from source content.
Expand or collapse full text
Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization at 400B Scale I. Kennedy Affiliation: (http://baa.ai Blacksheep ai.) T. Kennedy Affiliation: (Auckland, New Zealand) Abstract A single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of a layer’s quantization error (Theorem 1). This estimator is far better behaved than Monte-Carlo intuition suggests, because of a structural fact about quantization error that we measure in closed form: the error spectrum is flat. Across 1,683 weight tensors of a 35B-parameter MoE and a 9B dense model, the effective dimensionality deff=Tr(Δ⊤Δ)2/‖Δ⊤Δ‖F2d_eff=Tr( )^2/\| \|_F^2 of the round-to-nearest error Δ is 0.930.93 to 0.960.96 times the value mn/(m+n)mn/(m+n) attained by an i.i.d. noise matrix of the same shape, and on the MoE model its median is the same to three digits from 2-bit to 8-bit. Flatness makes the probe estimator calibrated: its coefficient of variation is 2/deff 2/d_eff, predictable from tensor shape before touching the weights, and it matches measurement to 0.1%0.1\% at the median. One probe measures per-tensor sensitivity to within 44 to 7%7\%, twenty probes to within 1.31.3 to 1.4%1.4\%, and averaging converges at the i.i.d. rate against exact ground truth. RAM (Resource-Aware Mixed-precision) applies the propagated form of the same estimator to budget-targeted quantization. Gaussian probes carrying the network’s own input statistics score every tensor at six candidate bit-widths, which by a one-line extension of the theorem estimates the layer-wise objective that GPTQ minimizes with calibration data; a multiple-choice knapsack solver allocates bit-widths under an exact byte budget, and divergence guardrails block catastrophic 2-bit assignments. One probe pass serves any budget. The isolated and propagated signals are different quantities: they rank tensors independently on Qwen3.5-35B-A3B (Spearman −0.01-0.01), and calibration transfers only partly (per-sequence CV 0.16 to 2.4 against 0.04 to 0.07), yet rank agreement with an independent reference at the allocator’s 50 sequences is 0.97 to 1.00 because the propagated signal spans ten times the range across tensors. Propagation supplies the missing input statistics: on Qwen3-8B, Qwen3.5-9B and Qwen3.8-27B the propagated probe’s estimate of the layer-wise calibration objective rank-correlates 0.83, 0.81 and 0.83 with the objective computed from real activations (domain ceiling 0.99), while the isolated estimator is uncorrelated with it (−0.07-0.07 to −0.15-0.15). That objective is nevertheless the wrong quantity to allocate from: at matched bytes on Qwen3.8-27B, allocating llama.cpp k-quant types from the block-output probe signal beats the vendor IQ3_M mix by 5.5% perplexity and beats an oracle that allocates from the real-activation objective by 1.6%, because the block signal carries the downstream weighting the per-tensor objective lacks. Fed through one allocator at matched bytes on Qwen3-8B, the propagated probe ties calibration-based HAWQ-V2 Hessian-trace sensitivity on WikiText-2 (paired p=0.96p=0.96), both beat the isolated estimator by 0.6%0.6\% (p<10−15p<10^-15), and all three tie on MMLU; on Qwen3.5-35B-A3B the isolated and propagated allocations tie on perplexity and the isolated one leads MMLU by 1.11.1 points. The probe pass takes 36 to 539 seconds on a single Apple Silicon machine for models from 8B to 400B parameters (15 to 803 GB in BF16). On the four MoE models where uniform 4-bit builds of comparable size exist, RAM builds reach 3.5 to 13.6% lower median WikiText-2 perplexity. On Qwen3.5-35B-A3B, RAM beats a GPTQ-style “protect attention” allocation by 1.7% perplexity at 4.8% smaller size, and at 33.6 GB it scores 73.1% on MMLU against 72.1% for the BF16 model. Latent probes that pass through the compression path recover the sensitivity of post-bottleneck weights in Multi-head Latent Attention models, where isotropic probes fail. Pre-quantized models are published at https://huggingface.co/baa-ai. 1 Introduction Post-training quantization (PTQ) is the main route to running large language models (LLMs) on consumer hardware. GPTQ [13], AWQ [25], and SqueezeLLM [20] compress well, and all three need a representative calibration dataset. Calibration data may be unavailable for proprietary models, the chosen distribution may not match the deployment domain, and calibration itself costs compute: on a 400B-parameter Mixture-of-Experts (MoE) model, GPTQ-style calibration loads the full model and runs forward passes over hundreds of sequences, which takes hours on high-end hardware. Data-free approaches [32, 37, 4] avoid calibration but either apply uniform bit-widths or rank tensors by a single weight statistic (kurtosis [1], the Frobenius norm of the rounding error [37]) with hand-tuned thresholds. Threshold-based allocation produces one output size regardless of the deployment memory target. We take a different route. Instead of summarizing weight matrices with statistical features, we measure how quantization error propagates through each layer using random functional probes: A random Gaussian vector passed through a linear layer provides an unbiased estimate of the squared Frobenius norm of the quantization error, the same quantity that calibration-based methods estimate using real activations. This observation, formalized in Theorem 1, says that a single random probe estimates in expectation the quantity that calibration-based methods estimate with data. Unbiasedness alone would be useless if the estimator’s variance were large. The second half of the story is a structural property of quantization error that we measure rather than assume: round-to-nearest quantization error is spectrally flat. Its effective dimensionality deff(Δ⊤Δ)d_eff( ) is within a few percent of the value an i.i.d. noise matrix of the same shape would have (mn/(m+n)mn/(m+n), the Marchenko–Pastur prediction), which is hundreds to thousands of significant directions for transformer-sized tensors. Because the single-probe coefficient of variation is 2/deff 2/d_eff (Lemma 4), flatness places the estimator in its best-case concentration regime: one probe yields a 44 to 7%7\% measurement of per-tensor sensitivity and twenty probes yield 1.31.3 to 1.4%1.4\%. Building on this, RAM (Resource-Aware Mixed-precision) is a complete data-free mixed-precision quantization framework. This paper makes the following contributions: 1. Theoretical guarantee: We prove that [⊤Δ⊤Δ]=‖Δ‖F2E[x \,x]=\| \|_F^2 for Gaussian x, where Δ=− =W- W is the quantization error matrix (Theorem 1, §3.1). 2. Measured spectral flatness: We compute the effective dimensionality of quantization error in closed form on 1,683 tensors across two architectures and four bit-widths: deff=0.93d_eff=0.93 to 0.96×mn/(m+n)0.96× mn/(m+n), the i.i.d.-noise value. The resulting per-probe accuracy 2/deff 2/d_eff is predictable from tensor shape alone and verified to 0.1%0.1\% median accuracy (§3.2). 3. Honest convergence: Probe averaging converges at the i.i.d. rate (p−0.50p^-0.50 measured against exact closed-form ground truth with disjoint probe blocks); an earlier version of this work reported a faster rate that was an artifact of a nested probe reference (§4.3). 4. Propagated functional probes: Random Gaussian probes propagated through the network score every tensor at six candidate bit-widths by output-space divergence, in 36 seconds to 9 minutes per model with no calibration data (§3.3). The propagated probe is the same estimator applied to the input- and downstream-weighted error operator (Proposition 3), which with the network’s own input statistics is the calibration objective of GPTQ and OBQ without data. 5. Isolated and propagated sensitivities are different quantities: the two signals rank tensors independently on Qwen3.5-35B-A3B (Spearman −0.01-0.01 over 390 tensors) and calibration transfers only partly: per-sequence CV 0.16 to 2.4 against 0.04 to 0.07, yet rank agreement at the allocator’s 50 sequences is 0.97 to 1.00 because the propagated signal spans ten times the range across tensors (§4.4, §5.1). 6. Data-free estimate of the calibration objective: the propagated probe’s quadratic form rank-correlates 0.83 (Qwen3-8B), 0.81 (Qwen3.5-9B) and 0.83 (Qwen3.8-27B, with the pipeline’s adaptation rule switched off) with the GPTQ layer objective under real activations, against −0.07-0.07 to −0.15-0.15 for the isolated estimator and 0.25 for HAWQ-V2; the block-output cosine the pipeline uses reaches only 0.50 and 0.11. Allocating from the objective itself, with real data or without, nevertheless loses to the block-output signal at matched bytes on Qwen3.8-27B (§4.5). 7. Matched-budget ablations: through one allocator at one budget, the isolated and propagated allocations tie on Qwen3.5-35B-A3B on perplexity, with the isolated one 1.11.1 MMLU points ahead; on Qwen3-8B the propagated probe ties calibration-based HAWQ-V2 (paired p=0.96p=0.96), both beat the isolated estimator by 0.6%0.6\% perplexity (p<10−15p<10^-15), and all three tie on MMLU (§4.7, §4.8). 8. Budget-targeted allocation: Per-tensor scores feed a multiple-choice knapsack solver under an explicit byte budget, with divergence guardrails against catastrophic 2-bit assignments; one probe pass serves any budget (§3.5). 9. Seven-architecture evaluation: Quality results on models from 8B to 122B parameters (15 to 250 GB in BF16), and probe timing up to 400B parameters (803 GB), covering dense, MoE with up to 256 experts, FP8-native, and MLA architectures (§4). 10. Blanket “protect attention” rules lose to measured sensitivity: A GPTQ-style allocation that leaves all attention and shared-expert tensors unquantized is both larger and worse than the probe allocation; the probes rank attention output/value projections and MoE routers most sensitive and routed experts least, so bytes go to specific tensors rather than to a tensor class (§4.2, §5.3). 11. MLA probe coverage: We identify and resolve a limitation of functional probes on Multi-head Latent Attention (MLA) [8] architectures, where compressed KV projections create sensitivity blind spots (§3.7). 2 Related Work Calibration-based PTQ. GPTQ [13] applies layerwise Optimal Brain Quantization using Hessian information from calibration data, processing one column at a time with error compensation. AWQ [25] identifies salient weight channels via activation-aware scaling. SqueezeLLM [20] combines sensitivity-based non-uniform quantization with dense-and-sparse decomposition. SpQR [9] isolates outlier weights for full-precision storage. QuIP [6] and AQLM [12] apply random orthogonal transformations and additive codebooks respectively. All require representative calibration data. Data-free quantization. EasyQuant [32] proposes data-free PTQ via weight distribution analysis. MXQ [37] uses the Frobenius norm of quantization error as a single sensitivity metric for mixed-precision allocation. HQQ [4] uses half-quadratic splitting for fast uniform quantization. HIGGS [26] applies Hadamard rotation via the linearity theorem. Lee et al. [23] proposed a data-free mixed-precision sensitivity metric for vision networks with synthetic labelled data. Concurrent calibration-free allocators for LLMs rank layers or experts by weight-spectrum statistics: NSDS [38] combines numerical and structural layer sensitivities, AlphaQ [33] scores experts with a heavy-tailed self-regularization exponent, and BitsMoE [39] allocates by spectral energy of an SVD factorization. RAM differs in deriving its signal from the output-space effect of quantizing each tensor under random inputs, and in reporting a measured calibration guarantee for that signal. Sensitivity-based mixed-precision. HAWQ [11] and HAWQ-V2 [10] allocate bit-widths by per-tensor Hessian-trace sensitivity, with the trace estimated by Hutchinson’s method on calibration data; §4.8 compares RAM against HAWQ-V2 at matched bytes. LLM-MQ [24] formulates mixed-precision allocation as an integer program with calibration-based sensitivities. SliM-LLM [16] achieves group-wise mixed precision via salience-driven allocation, and CherryQ [7] identifies high-impact parameters; both use calibration data. MC-MoE [15] combines expert quantization with dynamic pruning for MoE models, and Q-Strata [22] allocates across MoE blocks with a model-level objective evaluated on the assembled quantized model. GAMMA [34] and MixQuant [28] share RAM’s goal of one offline analysis that serves any budget; GAMMA learns module preferences with a hidden-state reconstruction objective, and MixQuant marginalizes each layer’s distortion over random upstream quantization configurations. CASA [35] shows that scalar sensitivity proxies in the multiple-choice knapsack formulation can distort inter-module rankings when the input and output Hessian factors are ill-conditioned; RAM’s propagated probes address the input side by using the network’s own upstream statistics. Quantization-error structure. LQER [36] applies an activation-induced scale matrix to drive the singular-value distribution of the quantization error toward a distribution that a low-rank correction can absorb. Our measurement concerns the unscaled round-to-nearest error and finds the opposite regime: its spectrum is flat, so no low-rank summary captures it, and that flatness is what makes random probes accurate. Stochastic trace estimation. Hutchinson’s estimator [17] approximates Tr(A)Tr(A) via [⊤A]E[z Az] with Gaussian or Rademacher probes; Bekas et al. [5] extend it to the diagonal, and Avron and Toledo [3] give concentration bounds. Theorem 1 applies this machinery with A=Δ⊤ΔA= , where Δ is the quantization residual. The Johnson–Lindenstrauss lemma [18] gives the same conclusion from the projection side: random projections preserve norm structure, so probe-based rankings track the true sensitivity ordering. Fisher information [2] and Hessian-based sensitivity [13] are the classical importance measures; both need data or activations. 3 Method RAM operates in three stages: (1) score every tensor at every candidate bit-width with functional probes, (2) apply divergence guardrails, and (3) solve the budget-constrained bit-width allocation. The pipeline needs only the pretrained weights: no calibration data, no gradients, no forward passes over text. Section 3.1 analyses the probe estimator on an isolated tensor, where its statistics can be computed in closed form; §3.3 describes the propagated version the allocator uses. 3.1 Functional Probes on an Isolated Tensor Consider a linear layer f()=f(x)=Wx with weight matrix ∈ℝm×nW ^m× n. Let W denote the quantized weight and Δ=− =W- W the quantization error matrix. The output perturbation for input x is δ()=Δδ(x)= \,x, with squared norm ‖δ()‖2=⊤Δ⊤Δ\|δ(x)\|^2=x \,x. Rather than estimating this quantity with calibration data, we draw functional probes ∼(,n)x (0,I_n) and compute the probe sensitivity Sprobe(,^)=1p∑j=1p‖Δj‖2,j∼(,n),S_probe(W, W)= 1p _j=1^p\| \,x_j\|^2, _j (0,I_n), (1) where p is the number of probes. Theorem 1 (Unbiased probe sensitivity). Let ∼(,n)x (0,I_n) and Δ∈ℝm×n ^m× n. Then [⊤Δ⊤Δ]=Tr(Δ⊤Δ)=‖Δ‖F2.E\! [x \,x ]=Tr( )=\| \|_F^2. (2) That is, a single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of the quantization error. Proof. Let A=Δ⊤Δ∈ℝn×nA= ^n× n. Since A is symmetric positive semidefinite, [⊤A] [x Ax] =[∑i,jAijxixj]=∑i,jAij[xixj] =E\! [ _i,jA_ijx_ix_j ]= _i,jA_ij\,E[x_ix_j] (3) =∑i,jAijδij=∑iAii=Tr(A)=Tr(Δ⊤Δ)=‖Δ‖F2, = _i,jA_ij\, _ij= _iA_i=Tr(A)=Tr( )=\| \|_F^2, (4) where the second equality uses [xixj]=δijE[x_ix_j]= _ij for ∼(,)x (0,I). ∎ Corollary 2 (Normalized probe sensitivity). The per-probe ratio ‖Δ‖2/(‖2+ε)\| \|^2/(\|Wx\|^2+ ) is a consistent (but not unbiased, since [A/B]≠[A]/[B]E[A/B] [A]/E[B] in general) estimator of ‖Δ‖F2/‖F2=NRMSE2\| \|_F^2/\|W\|_F^2=NRMSE^2. The bias is O(1/p)O(1/p) because ‖2\|Wx\|^2 concentrates around ‖F2\|W\|_F^2. Remark 1 (Connection to Hutchinson’s estimator). Theorem 1 is a special case of Hutchinson’s trace estimator [17, 5]: Tr(A)=[⊤A]Tr(A)=E[z Az] for any random vector z with [⊤]=E[zz ]=I. For Gaussian probes the variance is Var[⊤A]=2‖A‖F2=2‖Δ⊤Δ‖F2Var[x Ax]=2\|A\|_F^2=2\| \|_F^2 (Lemma 4), which is small relative to Tr(A)2Tr(A)^2 when A has many comparable eigenvalues, that is, when the error spectrum is flat. Section 3.2 shows by direct measurement that quantization error is in this favorable regime. Remark 2 (Why probe at all when ‖Δ‖F2\| \|_F^2 is computable?). For an isolated tensor, ‖Δ‖F2\| \|_F^2 can be computed directly, and by Theorem 1 the isotropic probe estimates nothing more than that. The probe formulation earns its place in two ways. First, it generalizes to non-identity input covariance: the allocator feeds each layer the probes propagated through the preceding layers (§3.3) and MLA blocks receive probes passed through their own compression path (§3.7), so the measured perturbation reflects the directions the network can produce rather than all n directions equally. Second, the multi-probe sample variance is a free per-tensor uncertainty estimate, which flags the rare tensors whose error is not flat (§E). Proposition 3 (Weighted-input estimator). Let ∼(,Σ)x (0, ) with Σ⪰0 0, and let the measured perturbation be =Δ δ=M for a fixed linear map M. Write =ΔΣ1/2K=M ^1/2. Then ‖2=‖F2=Tr(Δ⊤⊤ΔΣ),Var‖2=2‖⊤‖F2,E\| δ\|^2=\|K\|_F^2=Tr\! ( M M \, ), \| δ\|^2=2\|K K\|_F^2, (5) so the single-probe coefficient of variation is 2/deff(⊤) 2/d_eff(K K). Proof. Write =Σ1/2x= ^1/2z with ∼(,)z (0,I). Then ‖2=⊤⊤\| δ\|^2=z K K\,z, and Theorem 1 and Lemma 4 apply with A=⊤A=K K. ∎ Two readings of Proposition 3 matter for what follows. With =M=I and Σ=[⊤] =E[xx ] the second moment of a layer’s inputs, ‖F2=Tr(ΔΣΔ⊤)\|K\|_F^2=Tr( ) is the layer-wise reconstruction objective that GPTQ and OBQ minimize with calibration data, ‖Δ‖F2=Tr(Δ⊤Δ⊤)\| \|_F^2=Tr( \,XX ), up to the sample count. A probe drawn from the network’s own propagated input distribution therefore estimates the calibration objective without data (§3.3). And with M the Jacobian of the rest of the layer at the reference input, the proposition describes the perturbation the allocator measures to first order in Δ . Both readings come with a warning: the flatness measured in §3.2 is a property of Δ⊤Δ , and deff(⊤)d_eff(K K) can be far smaller when Σ or M concentrate energy in a few directions. Whether the propagated signal is calibrated is therefore a separate measurement (§4.4). 3.2 Spectral Flatness of Quantization Error How many probes are needed? Theorem 1 guarantees unbiasedness with a single probe, but if the error spectrum were concentrated on a few directions, a single probe would be noisy. The variance of the p-probe estimator is controlled by the effective dimensionality of the error spectrum. Definition 1 (Effective dimensionality). For a positive semidefinite matrix A with eigenvalues λ1≥λ2≥⋯≥λn≥0 _1≥ _2≥·s≥ _n≥ 0, the effective dimensionality (spectral participation ratio) is deff(A)=Tr(A)2‖A‖F2=(∑iλi)2∑iλi2.d_eff(A)= Tr(A)^2\|A\|_F^2= ( _i _i )^2 _i _i^2. (6) By Lemma 4, Var[1p∑j=1pj⊤Aj]=2‖A‖F2p=2Tr(A)2p⋅deff(A),Var\! [ 1p _j=1^px_j Ax_j ]= 2\|A\|_F^2p= 2\,Tr(A)^2p· d_eff(A), (7) so the single-probe coefficient of variation is 2/deff 2/d_eff: the estimator concentrates when deffd_eff is large, that is, when error energy is spread over many directions. A rank-one error would give CV=2≈141%CV= 2≈ 141\%; a flat spectrum over 10310^3 directions gives CV≈4.5%CV≈ 4.5\%. Direct measurement. Rather than assuming a spectral shape, we measure it. For every weight tensor of Qwen3.5-35B-A3B (1,317 tensors: attention, linear attention, dense MLP, fused MoE experts, vision blocks) and Qwen3.5-9B (366 tensors), at each of b∈2,3,4,8b∈\2,3,4,8\ bits (RTN, group 64), we compute ‖Δ‖F2\| \|_F^2 and ‖Δ⊤Δ‖F2\| \|_F^2 in closed form via the small-side Gram matrix, with no estimation involved. Three facts emerge (Table 15, §E): 1. Quantization error is spectrally flat. For an m×nm× n i.i.d. noise matrix, deff(Δ⊤Δ)=mn/(m+n)d_eff( )=mn/(m+n) in expectation (the Marchenko–Pastur participation ratio). Measured quantization error attains 0.93×0.93× this value at the median on the 35B MoE and 0.96×0.96× on the 9B dense model (IQR [0.86,0.97][0.86,0.97] and [0.90,0.98][0.90,0.98]). The largest tensors sit closest to the ceiling: the fused expert stacks (262,144×2,048)(262,144× 2,048) predict deff=2,032.1d_eff=2,032.1 and measure 2,031.72,031.7. 2. Flatness is bit-width invariant. On the 35B model the median deffd_eff at 2, 3, 4, and 8 bits is 386.1386.1, 386.0386.0, 386.1386.1, and 386.2386.2: the spectral shape of RTN error does not change as its magnitude shrinks 64×64×. On the 9B model the four medians lie within 3.5%3.5\% of one another (1,4241,424 to 1,4761,476). 3. Probe accuracy is predictable from shape alone. Since deff≈0.93mn/(m+n)d_eff≈ 0.93\,mn/(m+n), the per-probe CV 2/deff 2/d_eff can be computed before touching the weights. At 4-bit the measured per-probe CV matches the prediction with median ratio 0.9990.999 (IQR [0.965,1.033][0.965,1.033]) on the 35B model and 1.0001.000 (IQR [0.968,1.032][0.968,1.032]) on the 9B model, and the 200-probe mean matches the exact ‖Δ‖F2\| \|_F^2 with median ratio 0.99980.9998 and 1.00001.0000. The estimator is calibrated, not merely unbiased. Flatness is what RTN’s structure implies: within each group of 64 weights the rounding residuals are near-independent, sub-quantization-step fluctuations, so Δ behaves like a noise matrix rather than a low-rank perturbation. The consequence is the title claim: a single random probe is not a noisy Monte-Carlo sample but a 44 to 7%7\% measurement, with error bars known in advance. 3.3 Propagated Probes: The Allocation Signal The allocator scores tensors inside the full network rather than in isolation. It draws a batch of p=50p=50 probe sequences of s=8s=8 positions, ∈ℝp×s×dX ^p× s× d with entries i.i.d. (0,1)N(0,1), feeds them to the first decoder layer, and thereafter feeds each layer the reference output of the layer before it. For each weight tensor iW_i in layer ℓ and each candidate bit-width b∈2,3,4,5,6,8b∈\2,3,4,5,6,8\, the pipeline quantizes the tensor with RTN at group size 64, runs the layer on its input batch, compares the perturbed output Y with the reference output Y, and restores the original weight. The score is the cosine distance between the flattened outputs, averaged over probe sequences: divi(b)=1−⟨,^⟩‖‖^‖.div_i(b)=1- , Y \|Y\|\,\| Y\|. (8) For a small perturbation =^− δ= Y-Y, the cosine distance equals ‖⟂‖2/(2‖2)\| δ_ \|^2/(2\|Y\|^2) up to third-order terms, where ⟂ δ_ is the component of δ orthogonal to Y. It is therefore a normalized output-perturbation energy of the same kind as the NRMSE2 of Corollary 2, with two differences that are the point of propagation: the input covariance is the one the preceding layers produce rather than the identity, and the perturbation is measured after the layer’s nonlinearity and residual path rather than at the linear output. By Proposition 3 the propagated probe is the same estimator applied to the input- and downstream-weighted error operator; §4.4 measures its per-probe variance and convergence, and §5.1 shows that it ranks tensors differently from the isolated estimator. Two secondary signals ride on the same pass. On the final decoder layer the pipeline also records the fraction of probe positions whose argmax token under the language-model head changes when the tensor is quantized (the flip rate) and adds it to the score with weight 0.10.1. Between layers it monitors the cosine distance between consecutive reference outputs; if three consecutive layers exceed three times the running median, it switches from propagated to independent probes for the remaining layers. This switch triggered on Qwen3.5-35B-A3B and Qwen3-30B-A3B and not on Qwen3-8B; on the hybrid Qwen3.8-27B it fires at layer 32, and §4.5 shows that there it harms the estimate it was meant to protect. 3.4 Divergence Guardrails Dense models expose a failure mode of budget-driven allocation: every tensor has small divergence, so the greedy solver fills a tight budget with 2-bit assignments whose individually small errors compound across dozens of tensors. RAM applies three guardrails to the candidate set before and after allocation: 1. Absolute 2-bit veto. If divi(2)>10−4div_i(2)>10^-4, the 2-bit option is removed from tensor i’s candidate set. On the models we tested the threshold separates MoE expert stacks (2-bit divergence of order 2×10−52× 10^-5) from dense MLP tensors (order 2×10−42× 10^-4). 2. At most 30% of parameters at 2-bit. If the allocation exceeds this, the highest-divergence 2-bit tensors are upgraded until it does not. 3. At least 3.5 average bits. If breached, every 2-bit tensor is upgraded to 4-bit. At budgets below about 30% of the BF16 size on dense models and MoE models with at most 128 experts, we also raise the minimum candidate bit-width to 4 (§4.1); on these models 3-bit expert or MLP tensors cost more perplexity than the bytes they save. 3.5 Budget-Targeted Allocation via MCKP A design goal of RAM is exact budget targeting: the user specifies a memory limit (“fit in 24 GB”) and the system returns the lowest-divergence allocation that satisfies it. Threshold-based approaches produce one output size regardless of the hardware, and uniform quantization offers a few discrete size points (4-bit, 8-bit). Given per-tensor scores and guardrail-filtered candidate sets iC_i, RAM solves a Multiple-Choice Knapsack Problem (MCKP): min∑i=1Tbidivi(bi)s.t.∑i=1Tsizei(bi)≤(1−ρ)B,bi∈i, _\b_i\ _i=1^Tdiv_i(b_i) .t. _i=1^Tsize_i(b_i)≤(1-ρ)\,B, b_i _i, (9) where sizei(b)=nib/8+⌊ni/g⌋⋅4size_i(b)=n_ib/8+ n_i/g · 4 bytes for b<16b<16 (packed weights plus one 16-bit scale and bias per group of g=64g=64), B is the user’s byte budget, and ρ=0.1ρ=0.1 reserves a share of the budget for the tensors the allocator does not score: embeddings, the language-model head, and normalization weights, which the converter leaves at 16-bit, plus runtime metadata. Tensors with fewer than 1,024 elements are also left at 16-bit. Greedy solver. Every tensor starts at its lowest admissible bit-width. The solver then repeatedly applies the upgrade (bi→bi′)(b_i→ b_i ) with the largest divergence reduction per added byte, η=Δdiv/Δsizeη= / , until no upgrade fits the remaining budget. Because the scores are budget-independent, the same probe manifest is re-solved for any target size; the Qwen3.5-35B-A3B builds at 21, 23, 25, and 34 GB in §4.12 all reuse one 188-second probe pass. 3.6 Pipeline Summary Algorithm 1 RAM: propagated-probe analysis and budget-targeted allocation 0: Model directory, byte budget B, probes p, positions s, group size g 0: Per-tensor manifest (i,bi)\(i,b_i)\ 1: Phase 1: Probe sensitivity analysis 2: ←X← p×sp× s probe vectors ∼(,d) (0,I_d) 3: for each decoder layer ℓ (loaded one at a time in lazy mode) do 4: ←layerℓ()Y _ (X) reference output 5: for each weight tensor iW_i in layer ℓ with ≥1,024≥ 1,024 elements do 6: for each candidate bit-width b∈2,3,4,5,6,8b∈\2,3,4,5,6,8\ do 7: ^i←Q(i,b,g) W_i← Q(W_i;b,g); ^←layerℓ() Y _ (X) with ^i W_i in place; restore iW_i 8: divi(b)←1−⟨,^⟩/(‖‖^‖)div_i(b)← 1- , Y /(\|Y\|\| Y\|) (+ 0.1×0.1× flip rate on the last layer) 9: end for 10: end for 11: ←X unless the adaptation monitor has switched to independent probes 12: Free layer weights 13: end for 14: Phase 2: Guardrails i←b:b≠2 or divi(2)≤10−4C_i←\b:b≠ 2 or div_i(2)≤ 10^-4\ 15: Phase 3: Allocation Solve Eq. 9 greedily; enforce the 30% 2-bit cap and 3.5-bit floor 16: Write manifest JSON The pipeline (Algorithm 1) processes decoder layers sequentially. In lazy mode it loads one layer’s weights at a time and frees them after scoring, so models larger than memory can be analysed (Table 11). The output manifest maps each tensor to its assigned bit-width and is consumed as a quantization predicate during model conversion. 3.7 MLA-Aware Probe Coverage Multi-head Latent Attention (MLA) [8], used in GLM-4.7-Flash and DeepSeek-V2, compresses key-value projections through a low-rank bottleneck: =KcompressK=W_KW_compressx, where compress∈ℝdc×dW_compress ^d_c× d projects to a compressed latent space (dc≪d_c d). A probe drawn in the dcd_c-dimensional latent space with isotropic covariance spends its energy on all dcd_c directions with equal weight, whereas at inference the up-projection KW_K only ever sees the normalized image of compressW_compress. The mismatch mis-estimates the sensitivity of the post-bottleneck weights, and on GLM-4.7-Flash the resulting allocation is markedly worse (§4.9). RAM addresses this with two-stage probing. For each MLA attention block: (1) the compression weights compressW_compress are probed with the standard d-dimensional propagated probes (they see the full hidden dimension). (2) For the up-projection weights K∈ℝdout×dcW_K ^d_out× d_c that operate in the compressed space, we compute latent probes by passing the propagated probes through the block’s own compression weight and latent normalization layer: latent=Normlatent(compress).x_latent=Norm_latent\! (W_compress\,x ). (10) These dcd_c-dimensional latent probes score KW_K via ‖Klatent−^Klatent‖2\|W_Kx_latent- W_Kx_latent\|^2. Weights whose input dimension equals the hidden size (for example the output projection) keep full-dimensional probes. After probing all tensors, the full unquantized block output propagates to the next layer. MLA detection is automatic: if the model config contains kv_lora_rank <d<d, two-stage probing is activated. On GLM-4.7-Flash, 94 of 701 scored tensors received latent probes. 4 Experiments We evaluate RAM on seven model families using WikiText-2 perplexity (test split, 128 sequences of 2,048 tokens, seed 42). All experiments run on one Apple Mac Studio with an M2 Ultra (192 GB unified memory); probing, conversion, and inference use MLX [14]. We report median per-sequence perplexity as the primary metric because a few outlier sequences dominate the mean (§4.11). Unless stated otherwise, RAM builds use 50 propagated probe sequences of 8 positions, RTN at group size 64, and the six candidate bit-widths of §3.3. Models. Three categories: (i) MoE: Qwen3.5-35B-A3B [30] (256 experts), Qwen3-30B-A3B (128 experts), MiniMax-M2.5 (256 experts, native FP8), Llama-4-Scout [27] (16 experts), Qwen3.5-122B-A10B (128 experts); (i) MoE with MLA: GLM-4.7-Flash; and (i) dense: Qwen3-8B. Quality is evaluated on these seven models (8B to 122B parameters); probe timing is additionally reported for Llama-4-Maverick (400B parameters, 803 GB). Baselines. Uniform 4-bit RTN builds with group size 64 or 128, uniform 8-bit, a GPTQ-style “protect attention” allocation, and a calibration-based AWQ build on the dense model. 4.1 Main Results Table 1 presents the comparison across all seven architectures. Uniform 4-bit has one fixed size per model, so the two builds are not always size-matched; the table lists both sizes. On the four MoE models whose uniform build is within 13% of the RAM size (Qwen3.5-35B-A3B, MiniMax-M2.5, Llama-4-Scout, GLM-4.7-Flash), RAM lowers median perplexity by 3.5 to 13.6%. On Qwen3.5-122B-A10B and Qwen3-8B the RAM builds are larger than uniform 4-bit and the comparison measures the practitioner’s chosen budget rather than the allocator. On Qwen3-30B-A3B the tight budget with the 4-bit minimum admits no upgrades, so the “RAM” build is uniform 4-bit at group size 64 with unquantized embeddings; its advantage over the group-128 baseline is a group-size effect, not a mixed-precision one. Table 1: Main results across seven architectures. WikiText-2 test, 128 sequences, 2,048 tokens, seed 42; median perplexity. Δ is relative to the uniform 4-bit build in the same row; the uniform size is listed because uniform 4-bit builds have a fixed size. Probe time is wall-clock seconds for the full analysis pass on the M2 Ultra. aBuilt with the 4-bit minimum (§3.4); the allocation is uniform 4-bit at group size 64. bUniform build evaluated on 256 sequences. cRAM build is not size-matched to the uniform build. Uniform group size: g128 for Qwen3.5-35B-A3B and Qwen3.5-122B-A10B, g64 for Qwen3-8B, community 4-bit builds elsewhere. Model Arch BF16 (GB) RAM (GB) RAM Med. PPL Unif. 4b (GB) Unif. 4b Med. PPL Δ Time Qwen3.5-35B-A3B MoE 256exp 69.3 19.5 6.585 17.2 6.928 −-4.9% 188 s Qwen3-30B-A3Ba MoE 128exp 61.1 16.8 8.877 — 9.158 −-3.1% 53 s MiniMax-M2.5 MoE 256exp FP8 230.1 112.3 9.070 119.8 9.399b −-3.5% 301 s Llama-4-Scout MoE 16exp 217.3 56.0 7.806 56.9 8.219b −-5.0% 214 s Qwen3.5-122B-A10Bc MoE 128exp 250.2 107.6 5.327 60.4 5.601 −-4.9% 345 s GLM-4.7-Flash MoE + MLA 60.0 16.2 8.700 — 10.075 −-13.6% 99 s Qwen3-8Bc Dense 15.3 7.3 9.628 4.3 9.982 −-3.5% 36 s Scaling with model size. The method runs unchanged from a 15 GB dense model to 250 GB MoE models (Qwen3.5-122B-A10B; MiniMax-M2.5 at 230 GB in FP8). The largest gain, 13.6% on GLM-4.7-Flash, comes from MLA-aware probe coverage (§4.9). Budget targeting. Because the scores are budget-independent, the allocator is re-run at any target size from one manifest. The Qwen3.5-35B-A3B builds at four budgets in §4.12 share a single 188-second probe pass. Uniform quantization offers a few discrete size points; GPTQ and AWQ require a calibration run per target size. Probe time. The analysis pass takes 36 seconds on the smallest model (Qwen3-8B) and 345 seconds on the largest evaluated for quality (Qwen3.5-122B-A10B, 250 GB). An 803 GB model (Llama-4-Maverick) is analysed in 539 seconds (Table 11). 4.2 Comparison with a GPTQ-Style Heuristic A common strategy in GPTQ and AWQ deployments is to protect attention (keep it at high precision) while quantizing the MLP and expert tensors. We test this rule against RAM’s allocation on Qwen3.5-35B-A3B. The GPTQ-style build leaves attention and shared-expert tensors unquantized and applies 4-bit RTN at group size 128 everywhere else. Table 2: RAM vs. a GPTQ-style “protect attention” allocation on Qwen3.5-35B-A3B. All builds use RTN; the comparison isolates the allocation. The GPTQ-style build keeps attention and shared-expert tensors at 16-bit and everything else at 4-bit g128. Method Size (GB) Median PPL Δ vs BF16 BF16 69.3 6.494 — RAM (probe allocation) 19.5 6.585 ++1.4% GPTQ-style (protect attention) 20.5 6.697 ++3.1% Uniform 4-bit g64 17.4 6.764 ++4.2% Uniform 4-bit g128 17.2 6.928 ++6.7% RAM reaches ++1.4% degradation vs BF16 at 19.5 GB; the GPTQ-style build reaches ++3.1% at the larger size of 20.5 GB. The heuristic spends its bytes on a tensor class: it lifts every attention and shared-expert tensor to 16-bit, including those the probes rank as tolerant, and leaves the routed experts at a flat 4-bit. RAM spends the same budget on specific tensors. Its Qwen3.5-35B-A3B manifest keeps 93 of 260 attention tensors and all 40 routers at 16-bit, puts most shared-expert tensors at 8-bit, and spreads the routed expert stacks over 2, 3, 4, and 5 bits (Appendix C). Section 5.3 reports the sensitivity ordering behind this allocation. 4.3 Probe Convergence Against Exact Ground Truth Because ‖Δ‖F2\| \|_F^2 is computable in closed form, convergence of the isolated-tensor estimator can be measured against exact ground truth rather than against a converged probe reference. For each tensor we draw 200 independent probes, form p-probe estimates of NRMSE2=‖Δ‖F2/‖F2NRMSE^2=\| \|_F^2/\|W\|_F^2 from disjoint probe blocks, and compare the induced cross-tensor ranking with the exact one (Table 3). Table 3: Probe convergence against exact closed-form ground truth at 4-bit (disjoint probe blocks; mean over blocks). Relative MAE of the sensitivity estimate decays with log-log slope −0.50-0.50 on Qwen3.5-35B-A3B and −0.51-0.51 on Qwen3.5-9B, the i.i.d. averaging rate, from a single-probe base set by 2/deff 2/d_eff. Rank agreement is limited by the tightness of the sensitivity distribution rather than by probe noise: at 4-bit the normalized sensitivities of all tensors span 0.410.41 dex (sd 0.0650.065 dex) on the 35B model and 0.360.36 dex (sd 0.0490.049 dex) on the 9B model. Qwen3.5-35B-A3B (1,317 tensors) Qwen3.5-9B (366 tensors) Probes (p) Spearman ρ Rel. MAE Spearman ρ Rel. MAE 1 0.671 6.2% 0.627 5.5% 5 0.855 2.8% 0.864 2.5% 10 0.903 2.0% 0.921 1.8% 20 0.938 1.4% 0.957 1.3% 50 0.968 0.90% 0.982 0.75% 100 0.981 0.64% 0.991 0.53% Convergence is i.i.d. The relative MAE decays with log-log slope −0.50-0.50 (35B) and −0.51-0.51 (9B), the textbook p−1/2p^-1/2 rate. An earlier version of this work reported a slope of −1.43-1.43; that number was an artifact of measuring convergence against a 500-probe reference whose probe bank contained the evaluated probes, so estimate and reference shared samples and the MAE fell to zero at p=500p=500 by construction. Measured against exact ground truth with disjoint probes there is no super-convergence, and none is needed: the single-probe estimate already starts at 6%6\% relative error because deffd_eff is large, and 20 probes reach 1.4%1.4\%. Rank agreement is spread-limited, not noise-limited. At 4-bit the exact normalized sensitivities are tightly clustered (sd 0.0650.065 dex, range 0.410.41 dex on the 35B model). With 20 probes the per-tensor estimate error (1.4%1.4\%) is far below this spread, so the surviving rank disagreements (ρ=0.94ρ=0.94) are swaps between tensors whose true sensitivities differ by less than the estimate error. Whether such swaps matter for the final model is an end-to-end question that this measurement does not answer; §6 returns to it. Practical implication. For the isolated estimator, practitioners can bound the probe count needed for a target accuracy in advance from tensor shapes alone, via 2/(pdeff) 2/(p\,d_eff) with deff≈0.93mn/(m+n)d_eff≈ 0.93\,mn/(m+n). 4.4 Calibration of the Propagated Signal Proposition 3 says the propagated probe is a Hutchinson estimator of the weighted operator K; whether it is calibrated depends on deff(⊤)d_eff(K K), which involves the network’s own input second moment and downstream Jacobian and is not available in closed form. It can be measured. We re-ran the propagated pass of §3.3 with 200 probe sequences on three models, recording the per-sequence divergence of every tensor at 2, 4, and 8 bits, and scored p-sequence estimates from the first 100 sequences against the mean of the other 100 (Table 4; 2- and 8-bit agree with 4-bit to the digits shown). Table 4: Calibration of the propagated signal at 4-bit (200 probe sequences of 8 positions) next to the isolated estimator from §3.2. CV is the per-probe-sequence coefficient of variation of a tensor’s divergence (median over tensors, IQR in brackets); deffd_eff is 2/CV22/CV^2 for the propagated rows and the measured value for the isolated rows. Spread is the standard deviation of log10 _10 of the signal across tensors. ρp _p is the Spearman rank correlation between a p-probe estimate and a disjoint reference; MAE50 is the median relative error at p=50p=50, the allocator’s setting. Qwen3-8B was not part of the isolated measurement. Model Signal Tensors CV [IQR] deffd_eff CV(‖j‖)CV(\|Y_j\|) Spread (dex) ρ1 _1 ρ50 _50 MAE50 Qwen3.5-9B isolated 366 0.037 1,476 — 0.049 0.627 0.982 0.75% Qwen3.5-9B propagated 312 0.16 [0.09, 0.22] 79 0.04 1.06 0.994 1.000 1.3% Qwen3.5-35B-A3B isolated 1,317 0.072 386 — 0.065 0.671 0.968 0.90% Qwen3.5-35B-A3B propagated 590 0.24 [0.13, 0.59] 34 0.03 0.86 0.954 0.998 2.6% Qwen3-8B propagated 324 2.45 [1.60, 3.16] 0.3 1.67 0.64 0.742 0.967 19% The cosine signal is the perturbation energy of Proposition 3. The first-order identity 1−cos=‖⟂‖2/(2‖2)1- =\| δ_ \|^2/(2\|Y\|^2) holds per sequence to a median relative deviation of 2×10−42× 10^-4 or better on all three models, and the two quantities rank tensors identically (Spearman ≥0.995≥ 0.995). Calibration transfers only partly, and the shortfall is model-dependent. On Qwen3.5-9B and Qwen3.5-35B-A3B the propagated operator is far less flat than the isolated error (deffd_eff 79 against 1,476 and 34 against 386), so a single sequence measures a tensor to 16 to 24% rather than 4 to 7%; fifty sequences still bring the median tensor within 1.0 to 1.7% of the 200-sequence value. On Qwen3-8B the propagated signal is not calibrated at all: the per-sequence CV is 2.4, an implied deffd_eff below one, and the 50-sequence mean carries a median error of 19% (90th percentile 41%). The cause is visible in the reference outputs. From the third decoder layer onward the residual-stream norm of Qwen3-8B varies by a factor of 1.7 across Gaussian probe sequences (Table 4, CV(‖j‖)CV(\|Y_j\|)), against 0.03 to 0.04 on the two Qwen3.5 models: a few sequences excite very large activations, and the per-sequence divergence inherits their heavy tail. This is the concentration of energy in few directions that Proposition 3 warns about, produced by Σ and M rather than by Δ . Ranking survives because the spread is wide. The propagated signal spans 0.64 to 1.06 dex across tensors, ten to twenty times the 0.05 to 0.07 dex of the isolated estimator, so per-tensor noise that would scramble the isolated ranking leaves the propagated ranking intact. At the allocator’s 50 sequences the rank agreement with a disjoint reference is 0.998 (35B), 1.000 (9B), and 0.967 (8B); a single sequence already gives 0.95 and 0.99 on the two Qwen3.5 models. The relative error decays with probe count at the i.i.d. rate on all three (log-log slopes −0.50-0.50, −0.45-0.45, and −0.46-0.46 for the 9B, 35B, and 8B models), as Proposition 3 predicts for any fixed operator. The isolated estimator is calibrated but narrow; the propagated signal is noisy but wide. For models that develop large activations under random inputs, more sequences or a median aggregate would be the remedy; we have not tested either. 4.5 The Propagated Probe Estimates the Calibration Objective Proposition 3 makes a claim that can be checked directly: if the propagated probes carry the network’s own input statistics, the quadratic form they estimate should track the layer-wise objective that calibration-based methods compute from real activations. We measured both on every linear tensor of Qwen3-8B, Qwen3.5-9B, and the 27B hybrid Qwen3.8-27B. The real objective is Qreal(t,b)=∑x‖Δb‖2/∑x‖2Q_real(t,b)= _x\| _bx\|^2/ _x\|Wx\|^2 over the inputs the tensor receives when the model runs causally on 32 sequences of 512 tokens from the WikiText-2 training split (never the test split used for perplexity), with a second real-text domain (Tulu-3 SFT dialogue) as a ceiling on how far the objective itself moves between domains. QprobeQ_probe is the same ratio over the pipeline’s propagated probes (200 sequences of 8 positions, no mask, same adaptation rule), and Qiso=‖Δb‖F2/‖F2Q_iso=\| _b\|_F^2/\|W\|_F^2 is the isolated estimator. Table 5 reports rank correlations across tensors at 4-bit. Table 5: Does the propagated probe estimate the calibration objective? Spearman rank correlation across linear tensors at 4-bit between the objective under real activations (QrealQ_real, WikiText-2 train) and each signal. “Domain ceiling” is QrealQ_real on WikiText-2 against QrealQ_real on Tulu-3 dialogue. “Block cosine” is the pipeline’s own signal (§3.3), measured at the layer output. Ratio is the median per-tensor Qprobe/QrealQ_probe/Q_real with IQR. Model Tensors QprobeQ_probe QisoQ_iso HAWQ-V2 Block cosine Domain ceiling Ratio Qwen3-8B 252 0.83 −0.07-0.07 0.25 0.50 0.99 1.29 [1.05, 1.48] attention 144 0.90 −0.03-0.03 0.36 — — 1.35 MLP 108 0.71 0.08 0.07 — — 1.13 Qwen3.5-9B 248 0.81 −0.06-0.06 — 0.11 0.99 1.12 [0.92, 1.43] attention 32 0.88 −0.04-0.04 — — — 0.99 linear attention 120 0.72 −0.05-0.05 — — — 1.25 MLP 96 0.87 0.08 — — — 1.05 Qwen3.8-27B, adaptation rule on (fired at layer 32) 496 0.27 −0.15-0.15 — — — 1.19 [0.96, 1.76] layers 0–31 (propagated) 248 0.79 −0.11-0.11 — — — 1.14 layers 32–63 (independent probes) 248 −0.22-0.22 −0.17-0.17 — — — 1.62 Qwen3.8-27B, adaptation rule off 496 0.83 −0.15-0.15 — — — 1.15 [0.89, 1.34] attention 64 0.91 0.23 — — — 1.48 linear attention 240 0.76 −0.23-0.23 — — — 1.06 MLP 192 0.80 0.10 — — — 1.16 layers 0–31 248 0.79 −0.11-0.11 — — — 1.14 layers 32–63 248 0.85 −0.17-0.17 — — — 1.15 Propagation supplies the second moment. With real activations the ranking of tensors by quantization damage is stable across domains (0.99), and the propagated probe reproduces it at 0.830.83 and 0.810.81 on the two smaller models and at 0.830.83 on the 27B hybrid once its adaptation rule is switched off (0.88 to 0.91 within full attention), with no data at all. The isolated estimator, which is what Theorem 1 certifies, is uncorrelated with the real objective (−0.07-0.07, −0.06-0.06, −0.15-0.15): the second moment of the inputs, not the rounding error, decides which tensors matter, and only the propagated probe has it. HAWQ-V2’s Hessian trace, which folds in the loss curvature, tracks the layer objective at 0.25 on Qwen3-8B, so the matched-perplexity tie of §4.8 is between two signals that both see the input statistics and differ in what else they weight. The probe over-estimates the objective in absolute terms by 12 to 29% at the median, because Gaussian probes over-excite the few high-energy channels of the residual stream (the top eight channels carry 29% of probe energy against 17% of real energy on Qwen3-8B, and 28% against 22% on Qwen3.8-27B), but the ranking survives. The adaptation rule destroys the estimate, and switching it off restores it. On Qwen3.8-27B the pipeline’s adaptation monitor (§3.3) fired at layer 32 and replaced the propagated probes with fresh isotropic ones for the remaining 32 layers. With the rule on, the overall correlation is only 0.270.27, and the reason is visible by depth: 0.790.79 for the propagated half, −0.22-0.22 for the independent half, where the probe signal collapses onto the isolated estimator (ρ=0.75ρ=0.75 between them), which is what an isotropic input must produce. We re-ran the pass with the rule disabled and nothing else changed. The overall correlation rises to 0.830.83, the value measured on Qwen3-8B; layers 32–63 move from −0.22-0.22 to 0.850.85 and layers 0–31 stay at 0.790.79, where the rule had not yet fired. Within classes the estimate is 0.910.91 for full attention, 0.800.80 for the MLP and 0.760.76 for the 240 gated-DeltaNet projections; the one projection it does not track is the β-gate input (0.060.06 over 48 tensors of shape 48×512048× 5120, 0.05% of the linear parameters). The switch was introduced to stabilise the block-output cosine on hybrid stacks; for the objective estimate it is exactly wrong, because it throws away the input statistics that make the estimate work. The under-excitation of high-energy channels we had first attributed to this model was an artefact of the same rule: with propagated inputs throughout, the top eight channels carry 28% of probe energy against 22% of real energy, the same mild over-excitation as on Qwen3-8B. A better estimate of the objective is not a better allocation signal. The pipeline’s block-output cosine tracks the real objective at only 0.500.50 (8B) and 0.110.11 (9B), against 0.830.83 and 0.810.81 for the propagated quadratic form at the linear output, and the quadratic form is cheaper: for a stored input batch it is one matrix product per candidate bit-width, with no re-execution of the layer. The natural next step is to allocate with it, and we tested that on Qwen3.8-27B in a setting that reuses none of the pipeline’s own build path. Each signal goes to the same greedy knapsack over llama.cpp’s k-quant types (Q2_K to Q8_0, one type per tensor); the budget is bisected until the file size matches llama.cpp’s own IQ3_M mix within 1.7%; llama-quantize then builds the model from the vendor BF16 file with one importance matrix (100 chunks of 512 tokens) shared by every arm. The signals decide the 304 tensors of the feed-forward blocks, the sixteen full-attention blocks and the DeltaNet output projections (10.4 of the 12.6 GB); embeddings, output head, norms and, through a tensor-naming gap in our bridge that we found after the runs, the 192 DeltaNet input projections (1.9 GB) take llama.cpp’s default types, identical in every arm. With the naming fixed and the file matched to the IQ3_M byte count exactly, the ensemble arm reaches 6.018 whether the signal controls those projections or llama.cpp does (6.020), from a manifest probed without the adaptation rule; with the rule on, the same rebuild reaches 6.115, because the isotropic probes it substitutes from layer 32 under-protect the mid-depth feed-forward and DeltaNet gate tensors. Perplexity is on the first 40 chunks of 2,048 tokens of WikiText-2 test. Every arm is scored on the same chunks, so Table 6 reports, next to the mean, the number of chunks on which an arm beats the vendor mix and the number on which it beats the best arm. The seven arms are the two vendor mixes; the pipeline’s propagated cosine; that cosine combined with an isolated-tensor structural signal, the geometric mean of the cosine and one minus the linear centered kernel alignment [21] between a tensor’s outputs before and after quantization on 32 Gaussian probes, which is the signal behind the released builds of this model; the propagated quadratic form with the adaptation rule on and off; and QrealQ_real itself, an oracle that allocates from the real-activation objective computed on WikiText-2 training text. Table 6: Allocating from each signal at the byte count of llama.cpp’s IQ3_M mix, Qwen3.8-27B, k-quant types with a shared importance matrix. Perplexity on 40 chunks of 2,048 tokens of WikiText-2 test (mean ± standard error over chunks). “Chunks” counts the chunks, out of 40, on which the arm beats IQ3_M and on which it beats the cosine-and-CKA arm. The oracle is the only arm that uses text. Signal Data File (GB) Perplexity vs. IQ3_M vs. best IQ3_M (llama.cpp mix) none 12.58 6.336 ± 0.073 — 3 Q3_K_M (llama.cpp mix) none 13.30 6.207 ± 0.075 31 3 Propagated block cosine none 12.72 6.041 ± 0.071 36 6 Block cosine × isolated CKA none 12.79 5.991 ± 0.071 37 — Quadratic form, adaptation rule on none 12.61 6.204 ± 0.075 34 0 Quadratic form, adaptation rule off none 12.66 6.238 ± 0.076 30 1 QrealQ_real (oracle) WikiText-2 train 12.80 6.087 ± 0.073 37 5 The block-output signals win, and the oracle loses to them. The cosine-and-CKA arm reaches 5.991 against 6.336 for IQ3_M at the same bytes, better on 37 of 40 chunks, and Q3_K_M needs 5.7% more bytes to reach 6.207; the cosine alone reaches 6.041. The oracle allocates from the exact quantity that calibration-based methods optimize, reaches 6.087, and loses to the cosine-and-CKA arm on 35 of 40 chunks (Wilcoxon p=10−5p=10^-5) and to the cosine alone on 29 of 40 (p=0.004p=0.004). Its data-free estimate does worse still, at 6.238 with the adaptation rule off and 6.204 with it on, and the oracle beats the rule-off estimate on 39 of 40 chunks, so the estimate is imperfect as an allocation signal; but the gap that matters is the one between the oracle and the block-output signals, and it has the opposite sign from the one a rank correlation of 0.83 suggests. The reason is what the two quantities normalize by. QrealQ_real divides a tensor’s output perturbation by that tensor’s own output energy, so it says how badly the tensor rounds and nothing about how much its output matters to the block: in the terms of Proposition 3 it has =M=I. The block cosine measures the perturbation against the block output, residual stream included, so a tensor whose contribution is small relative to the residual scores low however badly it rounds; it is Proposition 3 with M the Jacobian of the rest of the block. The allocations show the difference. Over the tensors the signals control, the cosine-and-CKA arm spends 2.7 to 2.8 bits per weight on layers 0 to 31 and 4.3 to 4.5 on layers 32 to 63, and gives the sixteen full-attention blocks 4.1 bits; the oracle is nearly flat by depth (3.2 to 3.9 bits), gives full attention 3.4 bits, and spends on the DeltaNet output projections instead (4.4 against 3.8). The two agree on the type of 53% of those tensors. So the propagated quadratic form is the right way to estimate the calibration objective without data, and the calibration objective is the wrong quantity to allocate from. The signal the allocator needs is the downstream-weighted form, which the block cosine approximates at the cost of a layer re-execution and which the quadratic form could compute at one matrix product if M were supplied. We have not built that, and the allocation test covers one model and one quantizer family. 4.6 Dense Model at a Tight Budget: Guardrails and a Negative Result Table 7 reports Qwen3-8B (dense, 15.3 GB) at a 4 GB target and at a 6 GB target. The 4 GB target is a negative result for RAM. Both builds overshoot the target because the 10% reserve of §3.5 under-estimates the cost of this model’s unquantized embedding and language-model head (each 151,936×4,096151,936× 4,096 at 16-bit, 2.5 GB together), and both are worse than uniform 4-bit g64 at a smaller 4.3 GB. Within the pair, the build restricted to 2,4,8\2,4,8\ bits with the 2-bit veto beats the six-width build: on this dense model the 3-bit MLP tensors that the six-width solver prefers cost more perplexity than the bytes they save, which motivated the 4-bit minimum of §3.4. At the 6 GB target (7.35 GB actual, 48% of BF16) the six-width allocation, 55% of it at 5-bit, matches BF16 within 0.2%0.2\%. Table 7: Qwen3-8B (dense) at a tight 4 GB target and a 6 GB target. Both 4 GB builds overshoot the target and trail uniform 4-bit g64; the 6 GB build matches BF16. “2-bit” is the share of parameters assigned 2-bit. Mean and median over the same 128 sequences. Build Size (GB) 2-bit Mean PPL Median PPL BF16 15.3 0% 9.624 9.648 Uniform 4-bit g64 4.3 0% 9.965 9.982 RAM 4 GB target, 2,4,8\2,4,8\ + veto 5.8 7.4% 10.231 10.299 RAM 4 GB target, six widths + veto 5.7 1.4% 10.473 10.494 RAM 6 GB target, six widths 7.3 0% 9.642 9.628 4.7 Allocating with the Isolated Estimator If the isolated estimator is calibrated and the propagated one is not, why allocate with the propagated one? We answer by allocation. On Qwen3.5-35B-A3B we computed the exact isolated NRMSE2 of every scored tensor at all six candidate bit-widths with the pipeline’s own quantizer, and fed it to the same greedy solver at the same 21 GB budget with the same candidate sets (the 2-bit veto decisions were copied from the propagated manifest), so that the cost signal is the only difference. Both manifests were then converted and evaluated under one runtime (Table 8). Table 8: Isolated versus propagated allocation on Qwen3.5-35B-A3B at a 21 GB target: same solver, budget, quantizer, and candidate sets; only the cost signal differs. Upper: parameter-weighted mean bit-width by tensor class. Lower: the two builds converted and evaluated under the same runtime; the propagated row of Table 1 is repeated for reference. The two allocations assign the same bit-width to 50% of tensors. Attention Routers Shared experts Routed experts Other Propagated signal (avg bits) 8.5 16.0 8.0 3.8 11.8 Isolated estimator (avg bits) 5.7 8.0 8.0 3.9 10.6 Build Size (GB) Median PPL MMLU Propagated (Table 1 build) 19.5 6.585 0.657 Propagated (rebuilt, same runtime) 19.5 6.544 0.668 Isolated estimator 19.5 6.550 0.679 The isolated estimator, ranking by rounding error per unit of weight energy, spends less on attention (5.7 against 8.5 average bits, with 30 attention tensors at 2-bit against 22) and on routers (8-bit against 16-bit), and returns the bytes to the routed expert stacks (3.9 against 3.8 bits; 109 of 120 stacks at 4-bit against 74). On WikiText-2 the two builds are indistinguishable: median perplexity 6.5446.544 against 6.5506.550 and mean 6.6226.622 against 6.6256.625 at the same 19.5019.50 GB. That gap is a tenth of the shift the runtime upgrade alone produced on the propagated manifest (6.5856.585 under mlx_lm 0.30.4 in Table 1 to 6.5446.544 under 0.31.3, same weights), so perplexity does not separate the two signals at this budget. Both agree on the coarse structure, small tensors up and expert stacks down, and the size-normalized upgrade rule of the greedy solver enforces that structure whichever signal supplies the costs. On MMLU the isolated allocation scores 0.6790.679 against 0.6680.668 for the propagated rebuild (±0.004± 0.004 each; the runtime upgrade moved the propagated build from 0.6570.657), a 1.11.1-point margin of about two standard errors in favour of the calibrated isolated estimator, consistent with its expert stacks sitting at 4-bit rather than 3-bit (109 of 120 against 74). At this budget on this model, the signal that the theory covers allocates at least as well as the signal the pipeline used. 4.8 Matched Comparison with HAWQ-V2 The natural calibration-based comparator is HAWQ-V2 [10], which scores tensor t at bit-width b by Ωt(b)=(TrHt/nt)‖t−Qb(t)‖F2 _t(b)=(Tr\,H_t/n_t)\,\|W_t-Q_b(W_t)\|_F^2, with the Hessian trace estimated by Hutchinson’s method through double back-propagation of the language-model loss on calibration text. Proposition 3 says the propagated probe estimates a related but different quadratic form (the input second moment and downstream Jacobian, without the loss curvature), so the comparison asks whether the loss curvature buys anything at matched bytes. Protocol. We use our HAWQ-V2 implementation from earlier work: 16 WikiText-2 sequences of 256 tokens, 8 Rademacher probes each (128 Hessian–vector products), bfloat16, one seed, and the same group-64 RTN quantizer for the reconstruction term. The comparison runs on Qwen3-8B, the paper’s dense model, at its 6 GB target; the double back-propagation does not fit the 35B MoE on our hardware (its peak already exceeded 64 GB at 14B), which is itself part of the case for a data-free signal. All three signals, propagated, isolated, and HAWQ-V2, go through the same greedy solver at the same budget with the same candidate sets (2-bit veto copied from the propagated manifest, 205 of the 252 linear tensors), and the 72 normalization weights are pinned to the propagated allocation, so the flexible tensor set and its byte total (within 0.02%0.02\%) are identical. The three manifests were converted and evaluated under one runtime; perplexity differences are tested per sequence with a paired t-test on the 128 WikiText-2 sequences. Table 9: Three signals through one allocator on Qwen3-8B at a 6 GB target (252 flexible tensors, 72 norms pinned, byte totals within 0.02%0.02\%). Upper: parameter-weighted mean bit-width by projection; agreement is the share of flexible tensors given the same bit-width as the propagated allocation, and ρ4 _4 the Spearman correlation of the signal’s 4-bit cost with the propagated one. Lower: the three builds evaluated under one runtime (WikiText-2, 128 sequences of 2,048 tokens; MMLU 5-shot, full test set). “Lower” counts sequences on which the build beats the propagated build; p is the paired t-test. Signal q k v o gate up down Agreement ρ4 _4 Propagated probe (data-free) 5.3 6.4 7.8 6.3 5.4 5.5 6.0 — — Isolated estimator (data-free) 6.0 8.0 8.0 6.0 5.5 5.4 5.7 50% 0.04 HAWQ-V2 (calibration) 5.6 6.6 7.6 5.4 5.6 5.8 5.6 41% 0.19 Build Size (GB) Median PPL Mean PPL Lower / p vs. propagated MMLU Propagated probe 7.3 9.630 9.641 — 0.749 Isolated estimator 7.3 9.737 9.698 20 / <10−15<\!10^-15 0.749 HAWQ-V2 7.3 9.656 9.641 63 / 0.96 0.750 All three signals agree on the coarse shape, value projections high and gate/up projections low, and disagree tensor by tensor: HAWQ-V2 matches the propagated allocation on 41% of flexible tensors and the isolated estimator on 50%, with 4-bit cost rank correlations of 0.19 and 0.04 against the propagated signal. HAWQ-V2 spends least on the output projection (5.4 bits against 6.3) and most on the up projection (5.8 against 5.5). On WikiText-2 the propagated probe and HAWQ-V2 are indistinguishable: mean perplexity 9.6419.641 for both, medians 9.6309.630 and 9.6569.656, and per sequence the propagated build is lower on 65 of 128 sequences and the HAWQ-V2 build on 63 (paired t-test p=0.96p=0.96). The isolated estimator trails both: median 9.7379.737, higher than the propagated build on 108 of 128 sequences (p<10−15p<10^-15), a 0.6%0.6\% gap in mean perplexity, and the same margin against HAWQ-V2 (106 of 128). Read with §4.7, the two ablations agree. On the MoE model, where the routed expert stacks hold 96% of the scored parameters and every signal puts them low, the cost signal does not change the 21 GB build. On the dense model, where the flexible tensors are comparably sized and the allocation is decided tensor by tensor, the propagated probe recovers what the loss-curvature signal knows, without gradients or data, and the calibrated isolated estimator does not. On MMLU the three builds are indistinguishable at 0.7490.749, 0.7490.749, and 0.7500.750 (±0.003± 0.003): at this budget every allocation preserves the dense model’s factual recall, and perplexity is the only metric that separates the signals. The HAWQ-V2 signal uses a single Hutchinson seed; the comparison therefore establishes parity of the data-free probe with one draw of the gradient estimator, not superiority over it. 4.9 MLA Probe Coverage Table 10 compares probe strategies on GLM-4.7-Flash, which uses Multi-head Latent Attention with a compressed KV projection. All builds target 16 GB and use the same allocator; the MLA-aware manifest yields a 16.2 GB build and the two isotropic-probe manifests yield 14.6 GB builds, so part of the gap is size. Table 10: MLA probe strategies on GLM-4.7-Flash (MoE + MLA), 16 GB target. “Propagated” probes pass through the network without the latent stage; “adaptive” adds the divergence monitor of §3.3. The community uniform 4-bit build is the baseline. Strategy Size (GB) Median PPL Δ vs BF16 BF16 58.2 8.470 — Probe, MLA-aware (latent stage) 16.2 8.700 ++2.7% Probe, adaptive 14.6 9.354 ++10.4% Probe, propagated 14.6 9.486 ++12.0% Uniform 4-bit — 10.075 ++18.9% MLA-aware probes reach ++2.7% degradation vs BF16, against ++10.4% for the best isotropic-probe manifest and ++18.9% for uniform 4-bit. Isotropic probes in the latent space spend energy on directions the compression path never produces, so the scores they assign to the 94 post-bottleneck weights do not reflect inference-time behaviour and the allocator misplaces bytes; latent probes restrict the measurement to the active subspace. 4.10 Analysis Time Table 11 reports wall-clock analysis time for each model. For three models we also ran the rate-distortion alternative that scores each tensor by quantizing it at 13 (bit, group-size) configurations in isolation; the probe pass is 5 to 50 times faster. Table 11: RAM analysis time on the M2 Ultra. RD time is the measured wall-clock of a per-tensor rate-distortion scan at 13 configurations, where we ran it. Asterisk (*) marks lazy loading (one layer at a time). Model BF16 Size Probe Time RD Time Speedup Qwen3-8B 15 GB 36 s 195 s 5.4× Qwen3-30B-A3B 61 GB 53 s* 2,634 s 50× Qwen3.5-35B-A3B 69 GB 188 s — — GLM-4.7-Flash 60 GB 99 s 1,847 s 19× Llama-4-Scout 217 GB 214 s* — — MiniMax-M2.5 230 GB 301 s* — — Qwen3.5-122B-A10B 250 GB 345 s* — — Llama-4-Maverick 803 GB 539 s* — — Lazy loading. For models exceeding available memory, RAM loads one decoder layer at a time, so peak memory is set by the largest layer plus the probe batch rather than by the model. The run reports record peak memory of about 25 GB for the 217 to 250 GB models and about 109 GB for the 803 GB Llama-4-Maverick, which the 192 GB machine analyses in under 10 minutes. 4.11 Median vs. Mean Perplexity A handful of sequences can dominate corpus-level perplexity. On GLM-4.7-Flash the BF16 model has 5 outlier sequences among 128 (per-sequence perplexity above 25,000) that lift its mean to 11.5211.52 while its median is 8.4708.470; the uniform 4-bit build has 6 outliers, mean 14.7514.75, median 10.07510.075. The ordering is the same under both statistics, but the size of the gap under the mean is set by a few pathological sequences on which the BF16 model itself is near-random. We therefore report the median of per-sequence perplexity as the primary metric and give the mean alongside it where both exist (Table 7). 4.12 Downstream Task Evaluation To check that perplexity improvements carry over to a downstream task, we evaluate MMLU (5-shot, full test set) with the lm-evaluation-harness [31] on Qwen3.5-35B-A3B builds along a budget sweep, all allocated from the same probe manifest. Standard errors are those reported by the harness (about 0.350.35 percentage points at the full test-set size). Table 12: Downstream evaluation and allocation ablation on Qwen3.5-35B-A3B. Upper: at matched size (∼ 34 GB), RAM matches or beats uniform 8-bit on both metrics. Middle: budget sweep from one probe manifest; the 25 GB-target build is the smallest that beats uniform 4-bit on both metrics. Lower: allocation ablation at 19.5 GB. Config G forces all attention tensors to 8-bit and reduces the rest to fit; Config H forces all MLP and expert tensors to 4-bit and upgrades attention. MMLU standard errors are 0.00340.0034 to 0.00390.0039, except 0.00560.0056 for the 25 GB-target build. Build Size (GB) Med. PPL MMLU Note Reference points BF16 69.3 6.494 0.721 Uniform 8-bit g64 34.3 6.517 0.720 Uniform 4-bit g64 17.4 6.764 0.704 RAM budget sweep (one probe pass) RAM, 34 GB target 33.6 6.504 0.731 beats BF16 by 1.0 p RAM, 25 GB target 22.9 6.584 0.713 beats uniform 4-bit on both RAM, 23 GB target 21.2 6.525 0.683 PPL only RAM, 21 GB target 19.5 6.585 0.657 PPL only Allocation ablation at 19.5 GB G: force attention to 8-bit 19.5 6.598 0.645 H: force MLP/experts to 4-bit 19.5 6.622 0.589 At moderate budgets RAM wins on both metrics. At 33.6 GB (48% of BF16), the RAM build scores 0.7310.731 on MMLU against 0.7210.721 for BF16 and 0.7200.720 for uniform 8-bit at 34.3 GB, a 1.01.0 percentage-point margin over BF16 that is about two standard errors. PPL and MMLU cross over near a third of the BF16 size. Every RAM build in the sweep beats uniform 4-bit g64 on median perplexity. On MMLU, the 19.5 GB and 21.2 GB builds trail uniform 4-bit (0.6570.657 and 0.6830.683 vs 0.7040.704), and the 22.9 GB build (33% of BF16, 4.86 average bits) is the smallest that beats it (0.7130.713). The mechanism is visible in the manifests: the perplexity-optimal allocation at tight budgets pushes routed expert stacks to 3-bit, which average next-token prediction tolerates because each expert fires rarely, while MMLU queries every knowledge domain and therefore every expert. From about 23 GB upward the budget keeps the expert stacks at 4 to 5 bits. Both blanket rules lose. Config G (force every attention tensor to 8-bit, reduce the rest) is worse than RAM on both perplexity (6.5986.598 vs 6.5856.585, a gap inside the resolution of a 128-sequence median) and MMLU (0.6450.645 vs 0.6570.657, 1.21.2 points at 0.40.4-point standard errors). Config H (force MLP and expert tensors to 4-bit, upgrade attention) is worst on both (6.6226.622; 0.5890.589). Taking bytes from the expert and MLP tensors costs far more factual recall than taking them from attention, and adding bytes to every attention tensor helps less than adding them to the specific tensors the probes flag. Practical recommendation. At budgets of a third of BF16 and above, the RAM build dominates uniform quantization on both metrics. Below that, RAM still gives lower perplexity, and practitioners who prioritise knowledge-intensive tasks may prefer uniform 4-bit’s even distribution of precision across experts. 5 Analysis 5.1 Isolated and Propagated Sensitivities Are Different Quantities Proposition 3 says the two signals estimate different operators; the manifests show how different. On Qwen3.5-35B-A3B we matched the exact isolated 4-bit sensitivity ‖Δ‖F2/‖F2\| \|_F^2/\|W\|_F^2 from the spectral-flatness measurement (§E) with the propagated 4-bit divergence in the build manifest for the 390 tensors present in both, and computed the Spearman rank correlation across tensors (Table 13). Table 13: Rank agreement between the isolated exact estimator and the propagated allocator signal, Qwen3.5-35B-A3B, 4-bit, 390 matched tensors. The two signals order tensors independently overall and in opposite directions within attention. Tensor set n Spearman ρ All matched tensors 390 −0.01-0.01 Attention projections 190 −0.43-0.43 Shared-expert projections 120 0.060.06 Routed expert stacks 40 0.590.59 Routers 40 0.530.53 Overlap of the two top-30 lists 7 of 30 The isolated estimator’s 30 most sensitive tensors are 29 routers and one attention projection; the propagated signal’s are 22 attention projections and 8 routers. The isolated estimator sees a (256×2,048)(256× 2,048) router as the worst-quantized tensor per unit of weight energy, which it is; the propagated signal sees that a router’s rounding error barely moves the layer output under the inputs the network produces, while an attention output projection’s does. Neither is wrong about what it measures. The allocation results in §4 were produced by the propagated signal, and §4.7 reports what happens when the isolated estimator allocates instead. 5.2 What the Probe Measures That Weight Statistics Do Not Existing data-free methods rank tensors by per-tensor weight statistics: kurtosis [1], the Frobenius norm of the rounding error [37], spectral properties. Theorem 1 makes the honest position clear: an isotropic probe on an isolated tensor estimates ‖Δ‖F2\| \|_F^2 and nothing else, so it carries the same information as the Frobenius statistic, only with a known and predictable noise level. What the allocator’s signal adds is propagation. The input to layer ℓ is the reference output of the layers before it, so the probe covariance at each tensor is the one the network produces, and the perturbation is measured after the layer’s nonlinearity and residual path. Two tensors with identical ‖Δ‖F2\| \|_F^2 can then score differently because the network feeds them different inputs or damps their errors differently. The same mechanism is what the MLA fix exploits (§3.7): passing probes through the compression path is propagation applied inside a block. We have measured the calibration of the isolated estimator (§3.2); the propagated signal is validated only end-to-end, by the quality of the builds it produces. 5.3 Which Tensors Are Sensitive The propagated 4-bit divergence in the manifests orders tensor classes consistently across the MoE models, and differently from the “protect attention” folklore in one respect and from its opposite in another: • On Qwen3.5-35B-A3B, the 20 most sensitive tensors are 16 attention projections (output and value projections of the late full-attention layers, then the fused QKV projections of the linear-attention layers) and 4 MoE routers. Routers sit at the 90th percentile of sensitivity at the median, attention at the 65th, shared experts at the 46th, and routed expert stacks at the 34th. GLM-4.7-Flash and Qwen3-30B-A3B show the same ordering: routers and attention at the top, routed experts at the bottom. • On the dense Qwen3-8B the order flips: 14 of the 20 most sensitive tensors are MLP projections, and MLP tensors sit at the 61st percentile against the 42nd for attention. • The blanket rule fails on both models for the same reason. Sensitivity is a property of individual tensors, not of a class: on the 35B model 93 of 260 attention tensors earn 16-bit while 22 earn 2-bit, and the routed expert stacks that dominate the byte count spread over four bit-widths (Appendix C). Lifting a whole class, as Config G does, pays for tolerant tensors; crushing a whole class, as Config H does, hits the tensors that carry factual recall. 5.4 MoE Expert Handling For Mixture-of-Experts models, expert tensors are stored as 3-D arrays [E,din,dout][E,d_in,d_out] in fused switch modules where all E experts within a module share quantization parameters. RAM scores and allocates each fused (layer, projection) stack as one tensor, so its divergence is the propagated effect of quantizing all E experts together, its size is the sum over experts, and the solver’s output is directly implementable by the inference runtime. On Qwen3.5-35B-A3B this yields 120 expert stacks (40 layers, 3 projections) carrying 32.2 of the model’s 33.6 billion scored parameters. 6 Limitations and Discussion Data-free vs. calibration-based methods. RAM is data-free by design and operates in a different regime from calibration-based methods (GPTQ, AWQ, LLM-MQ, SliM-LLM). On Qwen3-8B (dense), AWQ reaches 9.7469.746 median perplexity at 4.44.4 GB using calibration data; RAM needs 7.37.3 GB to reach 9.6289.628, and at the tight 4 GB target it trails uniform 4-bit (§4.6). Calibration data captures activation-dependent sensitivity that random probes do not, and on a small dense model there is no tolerant parameter class for the allocator to exploit. Calibration-based methods in turn need a representative dataset, full-model forward passes, and hours of compute, whereas RAM completes in seconds to minutes on any model without data access. A hybrid, with RAM for the initial allocation and calibration-based refinement for critical tensors, is a natural next step. RTN base quantizer. RAM uses round-to-nearest (RTN) quantization. On 30 tensors of Qwen3.5-35B-A3B at 4-bit, HQQ [4] lowers per-tensor NRMSE by 2.96%2.96\% on average and a random Hadamard rotation [6] by 7.92%7.92\%, while a data-free Hessian proxy for OBQ [13] and a weight-only AdaRound [29] objective do not help (Appendix B). At the model level we have one data point: on Qwen3-8B at uniform 4-bit g64, HQQ-optimized weights evaluated in BF16 reach 9.7689.768 median perplexity against 9.9879.987 for RTN, a 2.2%2.2\% improvement that our packed-weight conversion did not preserve. Combining a better per-tensor quantizer with mixed-precision allocation remains open, and we make no claim that RTN is optimal. Isotropic probes and the propagated signal. Theorem 1 and the flatness measurement of §3.2 concern the isolated estimator with ∼(,)x (0,I). Proposition 3 extends the estimator to the propagated setting, but its variance depends on the network’s own input second moment and downstream Jacobian, and §4.4 shows that on one of three models the propagated signal is not calibrated per sequence. Its ranking is nevertheless stable at the allocator’s probe count on all three. The Gaussian probe distribution is a modelling choice: the MLA extension shows that architecture-specific probe shaping recovers signal when the covariance structure is known, and a general treatment using estimated input statistics from a small unlabelled corpus would bridge the data-free and calibration-based regimes. The estimate of the objective is good; the objective is the wrong target. Section 4.5 shows that the propagated quadratic form estimates the calibration objective far better than the block-output cosine the reported builds were allocated with, and that allocating from that objective, with real data or without, loses to the block-output cosine at matched bytes on Qwen3.8-27B. The measurement covers three non-MoE models (two of them hybrids) and the allocation test one model and one quantizer family; the MoE expert stacks, whose inputs are routed, were not measured. The downstream-weighted quadratic form, Proposition 3 with M the rest of the block, is the signal that would combine the estimate’s cost with the cosine’s allocation quality, and it is untested. On the hybrid 27B the adaptation rule must be disabled for the estimate to hold; with it off the three models agree. Rank swaps and end-to-end quality. Section 4.3 shows that residual rank disagreements at 20 probes are between tensors with near-equal isolated sensitivity. Equal objective cost does not imply equal end-to-end quality: two allocations with the same total divergence can differ in perplexity. We have not measured the perplexity spread across probe seeds for the reported builds, and we do not claim allocation stability beyond the objective. Dense model gains. On the single dense model tested (Qwen3-8B) the picture is mixed: RAM matches BF16 at 48% of its size but loses to uniform 4-bit at a tight budget. MoE models have high per-tensor sensitivity heterogeneity, which gives the allocator room to work; dense models have more uniform sensitivity and a smaller vocabulary-relative budget for the unquantized embedding tables. Downstream evaluation. We evaluate on WikiText-2 perplexity and MMLU on one model family. Perplexity and MMLU within a few percent of BF16 do not certify behaviour on procedural or agentic tasks: in companion work we found that compressed models that pass such guards can still invent steps in multi-step execution [19]. A broader multi-task evaluation, and a behavioural one, would strengthen the quality claims here. 7 Conclusion We presented RAM, a data-free mixed-precision quantization framework built on a calibrated estimator. Gaussian probes provide unbiased estimates of the squared Frobenius norm of quantization error (Theorem 1), and because quantization error is spectrally flat, with measured effective dimensionality within 44 to 7%7\% of the i.i.d.-noise value mn/(m+n)mn/(m+n), a single probe measures per-tensor sensitivity to 44 to 7%7\% with error bars predictable from tensor shape alone. Propagating the probes through the network applies the same estimator to the input- and downstream-weighted error operator, which is the calibration objective of GPTQ without data, and a multiple-choice knapsack solver converts one probe pass into a build at any byte budget. The two signals are different quantities: they rank tensors independently, the propagated one is calibrated per sequence on two of three models and not on the third, and its ranking is stable at fifty sequences on all three because it spans ten times the range across tensors. The propagated form is a data-free estimate of the calibration objective itself: its per-tensor values rank-correlate 0.8 with the objective under real activations, where the isolated estimator does not correlate at all. Through one allocator at matched bytes, the calibrated isolated estimator allocates at least as well as the propagated signal on the MoE model and 0.6%0.6\% worse in perplexity on the dense model, where the propagated probe ties gradient-based HAWQ-V2 without gradients or data. Across seven architectures, RAM builds beat size-comparable uniform 4-bit builds by 3.5 to 13.6% in median perplexity on the MoE models, beat a GPTQ-style heuristic at a smaller size, and reach BF16-level MMLU at half the BF16 size, with no calibration data, no gradients, and no GPU cluster. The analysis pass scales to an 803 GB model in nine minutes on one workstation. Two findings reach beyond RAM. First, sensitivity is a property of tensors, not of tensor classes: the probes rank attention output projections and MoE routers most sensitive and routed experts least, and both “protect attention” and “deprotect MLP” rules lose to per-tensor allocation. Second, median per-sequence perplexity is the safer headline statistic; a few outlier sequences on which even the BF16 model is near-random can dominate the mean. References [1] M. S. Akhondzadeh, A. Bojchevski, E. Eleftheriou, and M. Dazzi (2025) KurTail: kurtosis-based LLM quantization. arXiv preprint arXiv:2503.01483. External Links: Link Cited by: §1, §5.2. [2] S. Amari (1998) Natural gradient works efficiently in learning. Neural Computation 10 (2), p. 251–276. External Links: Document, Link Cited by: §2. [3] H. Avron and S. Toledo (2011) Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. Journal of the ACM 58 (2), p. 1–34. External Links: Document, Link Cited by: §A.2, §2. [4] H. Badri and A. Shaji (2024) Half-quadratic quantization of large machine learning models. Note: Mobius Labs, https://github.com/mobiusml/hqq Cited by: Appendix B, §1, §2, §6. [5] C. Bekas, E. Kokiopoulou, and Y. Saad (2007) An estimator for the diagonal of a matrix. Applied Numerical Mathematics 57 (11–12), p. 1214–1229. External Links: Document, Link Cited by: §2, Remark 1. [6] J. Chee, Y. Cai, V. Kuleshov, and C. De Sa (2024) QuIP: 2-bit quantization of large language models with guarantees. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §2, §6. [7] W. Cui and Q. Wang (2024) Cherry on top: parameter heterogeneity and quantization in large language models. arXiv preprint arXiv:2404.02837. External Links: Link Cited by: §2. [8] DeepSeek-AI (2024) DeepSeek-V2: a strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434. External Links: Link Cited by: item 11, §3.7. [9] T. Dettmers, R. Svirschevski, V. Egiazarian, D. Kuznedelev, E. Frantar, S. Ashkboos, A. Borzunov, T. Hoefler, and D. Alistarh (2024) SpQR: a sparse-quantized representation for near-lossless LLM weight compression. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §2. [10] Z. Dong, Z. Yao, Y. Cai, D. Arfeen, A. Gholami, M. W. Mahoney, and K. Keutzer (2020) HAWQ-V2: Hessian aware trace-weighted quantization of neural networks. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §2, §4.8. [11] Z. Dong, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer (2019) HAWQ: Hessian AWare quantization of neural networks with mixed-precision. In International Conference on Computer Vision (ICCV), External Links: Document, Link Cited by: §2. [12] V. Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, and D. Alistarh (2024) Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06118. External Links: Link Cited by: §2. [13] E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh (2023) GPTQ: accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §1, §2, §2, §6. [14] A. Hannun, J. Digani, A. Katharopoulos, and R. Collobert (2023) MLX: an array framework for Apple silicon. Note: Apple Machine Learning Research, https://github.com/ml-explore/mlx Cited by: §4. [15] W. Huang, Y. Liao, J. Liu, R. He, H. Tan, S. Zhang, H. Li, S. Liu, and X. Qi (2024) Mixture compressor for mixture-of-experts LLMs gains more. arXiv preprint arXiv:2410.06270. External Links: Link Cited by: §2. [16] W. Huang, H. Qin, Y. Liu, Y. Li, X. Liu, L. Benini, M. Magno, and X. Qi (2024) SliM-LLM: salience-driven mixed-precision quantization for large language models. arXiv preprint arXiv:2405.14917. External Links: Link Cited by: §2. [17] M. F. Hutchinson (1989) A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Communications in Statistics – Simulation and Computation 18 (3), p. 1059–1076. External Links: Document, Link Cited by: §2, Remark 1. [18] W. B. Johnson and J. Lindenstrauss (1984) Extensions of Lipschitz mappings into a Hilbert space. In Contemporary Mathematics, Vol. 26, p. 189–206. External Links: Document, Link Cited by: §2. [19] I. Kennedy and T. Kennedy (2026) Fidelity is not safety: gently-compressed LLMs pass every data-free quality guard yet invent procedure steps in agentic execution. arXiv preprint arXiv:2607.28196. External Links: Link Cited by: §6. [20] S. Kim, C. Hooper, A. Gholami, Z. Dong, X. Li, S. Shen, M. W. Mahoney, and K. Keutzer (2024) SqueezeLLM: dense-and-sparse quantization. In International Conference on Machine Learning (ICML), External Links: Link Cited by: §1, §2. [21] S. Kornblith, M. Norouzi, H. Lee, and G. Hinton (2019) Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning (ICML), External Links: Link Cited by: §4.5. [22] D. Lee, S. Chu, and H. O. Song (2026) Q-Strata: hierarchical bit allocation for mixed-precision quantization of mixture-of-experts LLMs. arXiv preprint arXiv:2608.30564. External Links: Link Cited by: §2. [23] D. Lee, M. Cho, S. Lee, J. Song, and C. Choi (2021) Data-free mixed-precision quantization using novel sensitivity metric. arXiv preprint arXiv:2103.10051. External Links: Link Cited by: §2. [24] S. Li, X. Ning, K. Hong, T. Liu, L. Wang, X. Li, K. Zhong, G. Dai, H. Yang, and Y. Wang (2023) LLM-MQ: mixed-precision quantization for efficient LLM deployment. In NeurIPS 2023 Workshop on Efficient Natural Language and Speech Processing (ENLSP), External Links: Link Cited by: §2. [25] J. Lin, J. Tang, H. Tang, S. Yang, W. Chen, W. Wang, G. Xiao, X. Dang, C. Gan, and S. Han (2024) AWQ: activation-aware weight quantization for on-device LLM compression and acceleration. In Conference on Machine Learning and Systems (MLSys), External Links: Link Cited by: §1, §2. [26] V. Malinovskii, A. Panferov, I. Ilin, H. Guo, P. Richtárik, and D. Alistarh (2024) Pushing the limits of large language model quantization via the linearity theorem. arXiv preprint arXiv:2411.17525. External Links: Link Cited by: §2. [27] Meta AI (2025) The Llama 4 herd: the beginning of a new era of natively multimodal AI innovation. Note: https://ai.meta.com/blog/llama-4-multimodal-intelligence/ Cited by: §4. [28] A. Misra, M. Agrawal, A. Jain, and T. Abdelzaher (2026) MixQuant: adaptive mixed-precision quantization for large language models. arXiv preprint arXiv:2607.23047. External Links: Link Cited by: §2. [29] M. Nagel, R. A. Amjad, M. van Baalen, C. Louizos, and T. Blankevoort (2020) Up or down? Adaptive rounding for post-training quantization. In International Conference on Machine Learning (ICML), External Links: Link Cited by: Appendix B, §6. [30] Qwen Team (2025) Qwen3.5-35B-A3B model card. Note: Hugging Face, https://huggingface.co/Qwen/Qwen3.5-35B-A3B Cited by: §4. [31] L. Sutawika, H. Schoelkopf, L. Gao, B. Abbasi, S. Biderman, J. Tow, et al. (2024) EleutherAI/lm-evaluation-harness: v0.4.3. Note: Zenodo External Links: Document, Link Cited by: §4.12. [32] H. Tang, Y. Sun, D. Wu, K. Liu, J. Zhu, and Z. Kang (2023) EasyQuant: an efficient data-free quantization algorithm for LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), External Links: Document, Link Cited by: §1, §2. [33] W. Yang, Y. Ma, A. Conzelmann, X. Zheng, M. W. Mahoney, T. K. Rusch, et al. (2026) AlphaQ: calibration-free bit allocation for mixture-of-experts quantization. arXiv preprint arXiv:2606.04980. External Links: Link Cited by: §2. [34] Z. Yao, H. Zhao, H. Wang, and X. Han (2026) GAMMA: global bit allocation for mixed-precision models under arbitrary budgets. arXiv preprint arXiv:2605.18475. External Links: Link Cited by: §2. [35] A. Yoshida and Y. Ichikawa (2026) Beyond scalar sensitivity: activation-aware mixed-precision LLM quantization with cross-layer refinement. arXiv preprint arXiv:2609.25916. External Links: Link Cited by: §2. [36] C. Zhang, J. Cheng, G. A. Constantinides, and Y. Zhao (2024) LQER: low-rank quantization error reconstruction for LLMs. arXiv preprint arXiv:2402.02446. External Links: Link Cited by: §2. [37] F. Zhang, Y. Liu, W. Li, X. Wang, and Q. Bai (2025) A mixed quantization approach for data-free quantization of LLMs. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence (ICAART), p. 353–363. External Links: Document, Link Cited by: §1, §2, §5.2. [38] H. Zhang, X. Chen, Z. Su, X. Liang, J. Xiong, W. Xu, et al. (2026) Beyond outliers: a data-free layer-wise mixed-precision quantization approach driven by numerical and structural dual-sensitivity. arXiv preprint arXiv:2603.17354. External Links: Link Cited by: §2. [39] J. Zhao, Z. Teng, M. Fan, T. Ma, W. Ren, S. Chen, et al. (2026) BitsMoE: efficient spectral energy-guided bit allocation for MoE LLM quantization. arXiv preprint arXiv:2606.00079. External Links: Link Cited by: §2. Appendix A Proof Details and Variance Bounds A.1 Variance of the Probe Estimator Lemma 4 (Variance bound). For ∼(,n)x (0,I_n) and symmetric positive semidefinite A=Δ⊤ΔA= : Var[⊤A]=2‖A‖F2=2∑i,jAij2.Var[x Ax]=2\|A\|_F^2=2 _i,jA_ij^2. (11) Proof. Using the fourth-moment identity for Gaussians: [(⊤A)2] [(x Ax)^2] =[∑i,j,k,lAijAklxixjxkxl] =E\! [ _i,j,k,lA_ijA_klx_ix_jx_kx_l ] (12) =∑i,j,k,lAijAkl[xixjxkxl]. = _i,j,k,lA_ijA_kl\,E[x_ix_jx_kx_l]. (13) For Gaussian variables, [xixjxkxl]=δijδkl+δikδjl+δilδjkE[x_ix_jx_kx_l]= _ij _kl+ _ik _jl+ _il _jk. Therefore [(⊤A)2] [(x Ax)^2] =∑i,j,k,lAijAkl(δijδkl+δikδjl+δilδjk) = _i,j,k,lA_ijA_kl( _ij _kl+ _ik _jl+ _il _jk) (14) =Tr(A)2+2Tr(A2)=Tr(A)2+2‖A‖F2. =Tr(A)^2+2Tr(A^2)=Tr(A)^2+2\|A\|_F^2. (15) Since Var[⊤A]=[(⊤A)2]−([⊤A])2=Tr(A)2+2‖A‖F2−Tr(A)2=2‖A‖F2Var[x Ax]=E[(x Ax)^2]-(E[x Ax])^2=Tr(A)^2+2\|A\|_F^2-Tr(A)^2=2\|A\|_F^2. ∎ Corollary 5 (Relative standard deviation). The coefficient of variation of a single-probe estimate is CV=2‖A‖F2Tr(A)=2deff(A).CV= 2\|A\|_F^2Tr(A)= 2d_eff(A). (16) For p probes, CVp=CV/pCV_p=CV/ p. With the measured median deff=386d_eff=386 on Qwen3.5-35B-A3B, a single probe gives CV1=2/386≈7.2%CV_1= 2/386≈ 7.2\% and p=20p=20 gives CV20≈1.6%CV_20≈ 1.6\%; on Qwen3.5-9B (median deff=1,476d_eff=1,476), CV1≈3.7%CV_1≈ 3.7\% and CV20≈0.8%CV_20≈ 0.8\%. These predictions match the empirical CVs with median ratio 0.9990.999 and 1.0001.000 (Table 15). A.2 Concentration and the Role of Spectral Flatness Concentration bounds for quadratic forms (Hanson–Wright; see also Avron and Toledo [3]) control the deviation of ‖Δ‖2/‖Δ‖F2\| \|^2/\| \|_F^2 in terms of the spectrum of A=Δ⊤ΔA= , with the effective dimensionality playing the role of the sample size: Pr[|‖Δ‖2‖Δ‖F2−1|>ε]≤2exp(−cdeff(A)min(ε,ε2)) \! [ | \| \|^2\| \|_F^2-1 |> ]≤ 2 \! (-c\,d_eff(A)\, ( , ^2) ) (17) for an absolute constant c. A Johnson–Lindenstrauss-style bound in the ambient dimension would be valid only for an exactly isotropic error; the honest parameter is deffd_eff, and the measurement in §E shows deffd_eff is within a few percent of its isotropic-noise ceiling. For a typical deff≈400d_eff≈ 400 to 1,5001,500, a single probe gives a (1±0.1)(1± 0.1)-approximation with high probability, matching the observed per-probe CVs of 44 to 7%7\%. A.3 Effective Dimensionality of Quantization Error: Measured An earlier version of this work conjectured deff≪min(m,n)d_eff (m,n) (spatially correlated, outlier-dominated error) and inferred deff≈30d_eff≈ 30 to 5050 from an apparent faster-than-p p convergence slope. Direct measurement (§E) refutes both: the convergence slope artifact is explained in §4.3, and the true effective dimensionality is close to the maximal value attained by an i.i.d. noise matrix of the same shape, deff(Δ⊤Δ)≈(0.93 to 0.96)⋅mnm+n,d_eff( )≈(0.93 to 0.96)· mnm+n, (18) where mn/(m+n)mn/(m+n) is the expected participation ratio of a Marchenko–Pastur spectrum with aspect ratio n/mn/m (it equals n/2n/2 for square matrices and approaches n for very tall ones). Group-wise RTN error behaves like white noise because within each 64-weight group the rounding residuals are effectively independent sub-step perturbations; outliers and inter-group correlation shave only a few percent off the noise ceiling. The deficit is slightly larger for MoE expert stacks quantized with shared group parameters (median ratio 0.930.93) than for dense tensors (0.960.96), and a small tail of structured tensors (the 3232-row linear-attention projections, one vision-tower output projection) fall well below the ceiling. These are the tensors where more probes are needed; because the multi-probe sample variance is a free empirical estimate of each tensor’s CV (Remark 2), they are detected at probe time at no additional cost. Appendix B Alternative Per-Tensor Quantizers We evaluated alternatives to RTN as the per-tensor quantizer on 30 tensors of Qwen3.5-35B-A3B at 4-bit with group size 64, measuring per-tensor NRMSE. HQQ (Half-Quadratic Quantization). HQQ [4] lowers per-tensor NRMSE by 2.96%2.96\% on average (30 of 30 tensors improved). At the model level, on Qwen3-8B at uniform 4-bit g64, HQQ-optimized weights evaluated in BF16 reach 9.7689.768 median perplexity against 9.9879.987 for RTN. Re-packing the HQQ weights into the runtime’s 4-bit format re-rounds them and loses the gain (10.33810.338), so integrating HQQ into the pipeline requires a format-aware conversion that we have not built. Hadamard rotation. A random orthogonal rotation lowers NRMSE by 7.92%7.92\% on average (30 of 30 tensors improved), the largest per-tensor improvement of any method tested. Deployment requires custom dequantization kernels (y=Wq(H⊤x)y=W_q(H x)) that our runtime does not provide, so we did not evaluate it at the model level. Data-free OBQ. Approximating the Hessian as H≈W⊤WH≈ W W worsens reconstruction (++0.50% NRMSE, 0 of 18 tensors improved). The weight-only Hessian is a poor proxy for the activation Hessian; OBQ needs calibration data. Data-free AdaRound. Replacing AdaRound’s [29] activation-weighted objective with weight-only MSE changes NRMSE by exactly 0.000%0.000\%: without activation data the optimal rounding direction for each element is round-to-nearest, which RTN already computes. Where bits are allocated matters more, in our experiments, than how each tensor is rounded, but the HQQ data point shows that a better rounder can add a further percent or two at the model level. Appendix C Allocation by Tensor Class Table 14 summarises the manifests behind the Qwen3.5-35B-A3B (21 GB target), GLM-4.7-Flash (16 GB target, MLA-aware), and Qwen3-8B (6 GB target) builds of Table 1. “Attention” includes all attention and linear-attention projections; “Routed experts” are the fused per-layer expert stacks; “Other” covers small convolution and gating tensors above the 1,024-element threshold. Table 14: RAM allocation by tensor class, read from the build manifests. Counts are tensors at each bit-width; the last column is the parameter-weighted mean bit-width of the class. Embeddings, language-model heads, and norms are unquantized and not listed. Model Class Tensors Params 2b 3b 4b 5b 6b 8b/16b Avg bits Qwen3.5-35B-A3B Attention 260 1.28B 22 0 0 3 13 129/93 8.5 Router 40 0.02B 0 0 0 0 0 0/40 16.0 Shared expert 160 0.13B 0 0 0 0 3 139/18 8.0 Routed experts 120 32.2B 3 31 74 12 0 0/0 3.8 Other 40 <<0.01B 0 0 0 0 0 21/19 11.8 GLM-4.7-Flash Attention 329 1.02B 0 5 12 17 26 131/138 9.0 Router 46 0.01B 0 0 0 0 0 0/46 16.0 Shared expert 138 0.43B 0 5 9 12 20 15/77 11.5 Routed experts 138 27.8B 0 58 80 0 0 0/0 3.6 Dense MLP + other 50 0.06B 0 0 1 0 0 23/26 — Qwen3-8B Attention 180 1.51B 0 0 7 27 57 72/17 6.0 Dense MLP 108 5.44B 0 0 0 68 26 14/0 5.6 Other 36 <<0.01B 0 0 0 0 0 16/20 12.4 Appendix D Evaluation Protocol WikiText-2 evaluation. All perplexity evaluations use the WikiText-2 test split with 128 evaluation sequences of 2,048 tokens, seed 42, and no sliding window (each sequence is evaluated independently), except the two community uniform builds marked in Table 1, which were evaluated on 256 sequences. Our evaluator reports both mean and median perplexity. Median perplexity. For each sequence we compute the per-sequence perplexity as exp(mean cross-entropy loss) (mean cross-entropy loss). The reported median is the 50th percentile of these values. The mean is the standard corpus-level perplexity (exponential of the average cross-entropy across all tokens). Hardware. All experiments use a single Apple Mac Studio with M2 Ultra (192 GB unified memory, 24-core CPU, 76-core GPU). No multi-GPU or distributed setup is required. Quantization. All quantization uses group-wise round-to-nearest with affine scaling, group size 64 unless otherwise specified, and packed integer storage with a 16-bit scale and bias per group. Appendix E Direct Measurement of Quantization-Error Spectra Protocol. For every weight tensor with both dimensions ≥32≥ 32 (embeddings and LM head excluded; 3-D fused expert stacks (E,dout,din)(E,d_out,d_in) reshaped to (Edout,din)(E\,d_out,d_in)), we compute the RTN error Δ=−Q(,b) =W-Q(W;b) at b∈2,3,4,8b∈\2,3,4,8\ (group 64) in float32 and evaluate, in closed form via the small-side Gram matrix, S=‖Δ‖F2S=\| \|_F^2, Q=‖Δ⊤Δ‖F2Q=\| \|_F^2, and deff=S2/Qd_eff=S^2/Q. In the same pass we draw 200 independent Gaussian probes per tensor and record each single-probe estimate ‖Δj‖2\| _j\|^2. The full measurement takes 127 s for Qwen3.5-35B-A3B (1,317 tensors) and 41 s for Qwen3.5-9B (366 tensors) on a single Apple Silicon machine. Table 15: Measured effective dimensionality of quantization error at 4-bit. d^eff=mn/(m+n) d_eff=mn/(m+n) is the expected participation ratio of an i.i.d. noise matrix of the same shape. CVpred=2/deff_pred= 2/d_eff; CVemp_emp is the sample CV over 200 independent probes. Unbiasedness: median ratio of the 200-probe mean to the exact ‖Δ‖F2\| \|_F^2. deffd_eff Probe calibration (median) Model median IQR deff/d^effd_eff/ d_eff med. [IQR] CVemp_emp/CVpred_pred s¯200/‖Δ‖F2 s_200/\| \|_F^2 Qwen3.5-35B-A3B 386 [354, 405] 0.93 [0.86, 0.97] 0.999 0.9998 Qwen3.5-9B 1,476 [722, 2,709] 0.96 [0.90, 0.98] 1.000 1.0000 By tensor type (Qwen3.5-35B-A3B, 4-bit, medians over layers). Fused MoE expert stacks (262,144×2,048)(262,144× 2,048): deff=2,031.7d_eff=2,031.7 vs. noise prediction 2,032.12,032.1 (the tall-matrix limit deff→nd_eff→ n); expert down-projections (524,288×512)(524,288× 512): 511.5511.5 vs. 511.5511.5. Full-attention QW_Q (8,192×2,048)(8,192× 2,048): 1,5191,519 vs. 1,6381,638. Linear-attention QKV projections (8,192×2,048)(8,192× 2,048): 1,5651,565 vs. 1,6381,638; linear-attention output projections (2,048×4,096)(2,048× 4,096): 1,0261,026 vs. 1,3651,365. Shared-expert and K/VW_K/W_V projections (512×2,048)(512× 2,048): 309309 to 374374 vs. 410410. MoE routers (256×2,048)(256× 2,048): 206206 vs. 228228. The strongest deviations from the noise ceiling are the 3232-row linear-attention a/ba/b projections (deff≈25d_eff≈ 25 vs. 31.531.5) and a single vision-tower MLP output projection with deff=83d_eff=83 against a prediction of 909909: small or structurally atypical tensors, detectable at probe time via the multi-probe sample variance. Bit-width invariance. Median deffd_eff at 2/3/4/8 bits: 386.1/386.0/386.1/386.2386.1/386.0/386.1/386.2 on the 35B model and 1,469/1,424/1,476/1,4571,469/1,424/1,476/1,457 on the 9B model. The spectral shape of RTN error is independent of its magnitude across a 64×64× range of error power on the MoE model and varies by under 4%4\% on the dense model. Flatness is a property of the rounding mechanism, not of a particular precision. The MAE convergence slope is between −0.49-0.49 and −0.51-0.51 at every bit-width on both models. Data. Raw per-tensor measurements (exact ‖Δ‖F2\| \|_F^2, ‖Δ⊤Δ‖F2\| \|_F^2, ‖W‖F2\|W\|_F^2, 200 single-probe samples per tensor per bit-width) and the measurement script are archived at RAM/results/deff/ in the project repository. Appendix F Reproduction Instructions Pre-quantized models are published at https://huggingface.co/baa-ai. The analysis and conversion scripts are experiments/probe_allocator_v2.py, experiments/probe_mla.py, and experiments/convert_probe_model.py in the project repository; the main results reproduce with: # Probe analysis (50 propagated probes, six bit-widths, group 64) python experiments/probe_allocator_v2.py \ --model /path/to/bf16-model --budget-gb 21 \ --num-probes 50 --lazy --output manifest.json # add --min-bits 4 for dense or <=128-expert models below ~30% of BF16 # Convert using the manifest python experiments/convert_probe_model.py \ --model /path/to/bf16-model --manifest manifest.json \ --output /path/to/quantized # Evaluate perplexity (mean and median) python eval_perplexity.py --model /path/to/quantized \ --num-sequences 128 --seq-length 2048 --seed 42 Software versions. Python 3.12.0, mlx_lm 0.30.4, PyTorch 2.6.0, SciPy 1.17.0, safetensors 0.6.2; lm-evaluation-harness 0.4.11 for MMLU. Hardware. Apple Mac Studio, M2 Ultra, 192 GB unified memory, macOS 15.5.