Paper deep dive
Few Bits, One Law: Toward W2A4KV2
Kai Yi, Tarek Elgamal, Sruthikesh Surineni, Vignesh Vivekraja, Soumyadeep Ghosh, Steven Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/8/2026, 7:13:27 AM
Summary
The paper introduces CanonQ, a unified quantization-aware training (QAT) framework for extreme low-bit Large Language Model (LLM) compression. CanonQ addresses the challenges of jointly quantizing weights, activations, and KV caches by separating source canonicalization (using fixed rotations and energy normalization) from task-aware adaptation. This allows the reuse of frozen Gaussian-reference codebooks across layers and models. Theoretical bounds are provided for codebook transfer error and gradient bias via a normalization-aware straight-through estimator. Empirical results show significant improvements in perplexity and zero-shot accuracy for models like LLaMA3 and Qwen3 under W2A4KV2 compression.
Entities (16)
Relation Signals (16)
Kai Yi → affiliatedwith → Meta AI
confidence 98% · Kai Yi Affiliation: Meta AI
CanonQ-Omni → usescompression → W2A4KV2
confidence 96% · The strongest gains arise under joint W2A4KV2 compression: across LLaMA3-1B/3B/8B, CanonQ-Omni achieves
CanonQ-Omni → appliedto → Llama3-3B
confidence 95% · across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28x lower WikiText-2 perplexity
CanonQ-Omni → appliedto → LLaMA3-8B
confidence 95% · across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28x lower WikiText-2 perplexity
CanonQ-Omni → appliedto → LLaMA3-1B
confidence 95% · across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28x lower WikiText-2 perplexity
CanonQ-Omni → evaluatedon → WikiText-2
confidence 95% · achieves up to 14.28x lower WikiText-2 perplexity
CanonQ → improves → WikiText-2 perplexity
confidence 95% · achieves up to 14.28x lower WikiText-2 perplexity
CanonQ → appliedto → MobileLLM-Pro-1B
confidence 93% · on instruction-tuned MobileLLM-Pro-1B at W2A16KV16, CanonQ achieves relative improvements
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Extreme low-bit LLM compression is most challenging when weights, activations, and KV caches are quantized together: their distributions differ, and quantization errors interact throughout the network. We introduce CanonQ, a unified quantization-aware training framework that addresses these challenges by separating source canonicalization from task-aware adaptation. Fixed rotations and energy normalization map heterogeneous tensor sources to canonical coordinates, enabling frozen Gaussian-reference codebooks to be reused across layers and models. Joint training then adapts the network to the coupled errors of weight, activation, and cache quantization within a common scalar/vector interface. We bound frozen-codebook transfer error and local task loss, and derive an exact normalization-aware straight-through Jacobian that links quantization distortion to gradient bias. The strongest gains arise under joint W2A4KV2 compression: across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28x lower WikiText-2 perplexity and up to 57.9% higher mean zero-shot accuracy than prior state-of-the-art and representative quantization baselines. The benefits extend to Qwen3-1.7B, code generation, and mathematical reasoning: on instruction-tuned MobileLLM-Pro-1B at W2A16KV16, CanonQ achieves relative improvements of 41.7% in HumanEval pass@1 and 39.1% in GSM8K exact match over the strongest evaluated quantization baseline.
Tags
Links
- Source: https://arxiv.org/abs/2610.09202v1
- Canonical: https://arxiv.org/abs/2610.09202v1
Trouble viewing inline? Open PDF directly →
Full Text
100,203 characters extracted from source content.
Expand or collapse full text
Few Bits, One Law: Toward W2A4KV2 Kai Yi Affiliation: Meta AI Tarek Elgamal Affiliation: Meta AI Sruthikesh Surineni Affiliation: Meta AI Vignesh Vivekraja Affiliation: Meta AI Soumyadeep Ghosh Affiliation: Meta AI Steven Li Affiliation: Meta AI Abstract Extreme low-bit LLM compression is most challenging when weights, activations, and KV caches are quantized together: their distributions differ, and quantization errors interact throughout the network. We introduce CanonQ, a unified quantization-aware training framework that addresses these challenges by separating source canonicalization from task-aware adaptation. Fixed rotations and energy normalization map heterogeneous tensor sources to canonical coordinates, enabling frozen Gaussian-reference codebooks to be reused across layers and models. Joint training then adapts the network to the coupled errors of weight, activation, and cache quantization within a common scalar/vector interface. We bound frozen-codebook transfer error and local task loss, and derive an exact normalization-aware straight-through Jacobian that links quantization distortion to gradient bias. The strongest gains arise under joint W2A4KV2 compression: across LLaMA3-1B/3B/8B, CanonQ-Omni achieves up to 14.28×14.28× lower WikiText-2 perplexity and up to 57.9%57.9\% higher mean zero-shot accuracy than prior state-of-the-art and representative quantization baselines. The benefits extend to Qwen3-1.7B, code generation, and mathematical reasoning: on instruction-tuned MobileLLM-Pro-1B at W2A16KV16, CanonQ achieves relative improvements of 41.7%41.7\% in HumanEval pass@1 and 39.1%39.1\% in GSM8K exact match over the strongest evaluated quantization baseline. †correspondence: kaiyi96@meta.com Figure 1: Schematic overview of CanonQ. Fixed source-specific transforms and normalization map tensor blocks to canonical coordinates. Frozen Gaussian-reference codebooks are reused across layers and models for matching (b,d)(b,d). Joint QAT adapts the model to coupled quantization errors at W2A4KV2. 1 Introduction LLM inference is increasingly limited by memory movement: weights dominate model storage and matrix-multiplication bandwidth, activations affect prefill, and the KV cache grows with batch size and context length. Quantization is therefore central to deployment, yet the most attractive regimes, which include W1/W2 weights, A4 activations, and KV2 caches, remain difficult. Post-Training Quantization (PTQ) is effective at moderate precision but often collapses when several components are compressed simultaneously (dettmers2022llmint8; xiao2023smoothquant; frantar2022gptq; lin2024awq; shao2024omniquant; egiazarian2024aqlm; malinovskii2024pv). Quantization-Aware Training (QAT) can recover quality by adapting the model to the quantized computation (liu2024llmqat; liu2025paretoq; chen2025efficientqat; li2026winq; yi2026jacquant), but the result still depends on the discrete representation to which the model is trained. At such low precision, that representation is not a minor implementation choice. Uniform quantization uses equally spaced reconstruction levels, even though weights, activations, and KV states have different distributional shapes. A data-fitted codebook can accommodate those differences, but ties the representation to the tensors or calibration data used to construct it. Joint compression makes the choice harder: weights shape the activation distribution, and quantized activations generate the keys and values that must themselves survive a two-bit cache. This motivates a common quantization interface that supports adaptation across the coupled computation. We ask: can a small quantization interface be designed once, then reused while an LLM learns to operate through it? Our approach separates two decisions. Source-specific transforms and normalization determine the coordinates presented to the quantizer; a source-independent reference determines the reconstruction alphabet. QAT then adapts the network to the resulting fixed codec. We introduce CanonQ around this principle: canonicalize the source, then train the model to use the codec. A fixed source-specific Hadamard–diagonal–permutation (HDP) transform mixes each block, and energy normalization removes its magnitude. The codebooks and HDP maps stay fixed throughout QAT. CanonQ-W applies this interface to weights; CanonQ-Omni applies it to the complete W/A/K/V computation. Our central finding is that the benefit of canonical quantization becomes most pronounced when weights, activations, and KV caches are compressed together. Our contributions are: (1) A unified framework for joint W/A/KV quantization. We introduce CanonQ, which unifies scalar and vector quantization of weights, activations, and KV caches through a common canonical interface. Frozen reference codebooks are reusable across layers and models, while joint QAT adapts the network to interacting quantization errors across the full computation. (2) A unified theory linking source alignment to optimization. We establish a quantitative chain from source mismatch to codebook distortion, local task loss, and optimization error. Wasserstein bounds characterize when frozen reference codebooks transfer to actual tensor distributions, while an exact normalization-aware straight-through estimator (STE) Jacobian identity reveals how assignment distortion controls backward sensitivity and gradient bias. We further derive distortion-dependent stationarity guarantees for a smooth reference objective, providing a theoretical foundation for the canonical training interface. (3) Large quality gains under extreme joint compression. At W2A4KV2, CanonQ-Omni achieves up to 14.28×14.28× lower WikiText-2 perplexity and up to 57.9%57.9\% higher mean zero-shot accuracy than strong quantization baselines across LLaMA3-1B/3B/8B. Further evaluations demonstrate transfer to Qwen3-1.7B, substantial coding and GSM8K improvements under W2A16KV16, and stable relative perplexity within the 8K continued-QAT training range. 2 Related Work Low-bit optimization and codebooks. PTQ methods including GPTQ, AWQ, and OmniQuant improve low-bit reconstruction (frantar2022gptq; lin2024awq; shao2024omniquant); LLM-QAT, EfficientQAT, ParetoQ, WinQ, and JacQuant adapt models through quantized training (liu2024llmqat; chen2025efficientqat; liu2025paretoq; li2026winq; yi2026jacquant). Binary and ternary methods likewise make representation a first-order design choice (wang2023bitnet; ma2024era; shang2023pbllm; huang2024billm; xu2024onebit). Lloyd–Max and Linde–Buzo–Gray provide classical scalar and vector constructions (max1960quantizing; lloyd1982least; linde1980algorithm); modern non-uniform, additive, and lattice approaches include NF4, SqueezeLLM, AQLM, PV-Tuning, and QuIP# (dettmers2023qlora; kim2024squeezellm; egiazarian2024aqlm; malinovskii2024pv; tseng2024quipsharp). LC-QAT is a close weight-only QAT comparison: it represents two-bit quantized weights through a learned affine map over discrete vectors and avoids explicit codebook lookup in the training forward pass (wang2026lcqat). CanonQ instead freezes the reference table, retains nearest-codeword assignment, and trains all enabled W/A/K/V paths. Rotations, canonicalization, and caches. Randomized rotations support distributed compression and LLM incoherence processing (vargaftik2021drive; vargaftik2022eden; chee2023quip; tseng2024quipsharp). QuaRot and SpinQuant suppress outliers or learn rotations (ashkboos2024quarot; liu2024spinquant); KIVI and KVQuant exploit cache-specific structure (liu2024kivi; hooper2024kvquant). TurboQuant separates norm from direction and constructs scalar quantizers for rotation-induced marginals; its inner-product variant additionally quantizes a residual with QJL (zandieh2025turboquant). This is the closest source-modeling connection. We study task-aware adaptation to a fixed canonical codec, allow joint tuple assignment, and quantify the implemented HDP map’s mismatch from its design reference. Building on joint four-bit compression exemplified by QuaRot, CanonQ targets the more aggressive W1/W2-A4/A8-KV2 regime through a reusable canonical interface and joint task-aware adaptation. 3 CanonQ: Unified W/A/KV Quantization As illustrated in Fig. 1, CanonQ separates source-specific coordinates from source-independent reconstruction choices. Let u∈ℝgSu ^g_S be a block from S∈W,A,K,VS∈\W,A,K,V\. Draw and fix the orthogonal map RS=PSHDSR_S=P_SHD_S, where H is a normalized Hadamard matrix, DSD_S contains random signs, and PSP_S is a permutation. Each source shares its map across layers; keys and values share one KV map. With ρS(u)=‖u‖2+ϵ _S(u)=\|u\|_2+ε, define z=gSRSu/ρS(u).z= g_SR_Su/ _S(u). (1) Rotation redistributes energy; normalization removes magnitude. For ϵ=0ε=0, the representation is invariant to positive rescaling of nonzero u. The Gaussian law Qd=(0,Id)Q_d=N(0,I_d) provides a reusable codebook design reference, motivated by the large-gSg_S behavior of Haar-rotated sphere coordinates (diaconis1984asymptotics). The canonicalization defect in Def. 1 quantifies deviations under the implemented structured HDP map. 3.1 Reusable Codebooks in Canonical Coordinates For b bits per scalar and tuple dimension d, the population target is the best K=2bdK=2^bd-point quantizer of QdQ_d, Cb,d⋆∈argmin1≤|C|≤2bd1dV∼Qddist(V,C)2.C_b,d ∈ _1≤|C|≤ 2^bd 1dE_V Q_ddist(V,C)^2. (2) Generalized Lloyd/LBG iterations approximate this target offline; we denote the deployed numerical table by Cb,dcan:=C^b,dC^can_b,d:= C_b,d and retain its finite fitting gap εvq _vq in Eq. 6. For d=1d=1 the construction is Lloyd–Max; for d>1d>1 assignment is genuinely joint rather than a Cartesian product of scalar decisions. For example, W2 with d=4d=4 uses 256 four-dimensional codewords, so one 8-bit index represents four weights. A table depends only on (b,d)(b,d) and stays frozen across groups, layers, models, and QAT updates. Group energies and source maps handle scale and coordinates, while QAT adapts the model without refitting each tensor’s alphabet. The shared construction therefore accommodates different raw W/A/K/V distributions. Partition z into consecutive tuples vj∈ℝdSv_j ^d_S, with dS|gSd_S g_S. The codec is v^j=ΠCbS,dScan(vj),u^=γS,ℓρS(u)gSRS⊤concatj(v^j). v_j= _C^can_b_S,d_S(v_j), u= _S, _S(u) g_SR_S concat_j( v_j). (3) Let q=concatj(v^j)q=concat_j( v_j) denote the assigned canonical block. The nonnegative gain γS,ℓ _S, adjusts its scale after assignment; clipping and decoding metadata remain source-specific. Here gSg_S is the normalization-block size and dSd_S is the assignment dimension. Assignment error ‖q−z‖2\|q-z\|_2 and reconstruction error ‖γS,ℓq−z‖2\| _S, q-z\|_2 therefore describe different quantities. With identity assignment and unit gain, the codec recovers u exactly. 3.2 Joint Adaptation to Quantization Errors Nearest-codeword assignment uses the identity STE, v~j=vj+sg(ΠCbS,dScan(vj)−vj),minθ,ϕ(x,y)∼ℓ(fQC(θ,ϕ)(x),y), v_j=v_j+sg\! ( _C^can_b_S,d_S(v_j)-v_j ), _θ,φE_(x,y) \! (f_Q_C(θ;φ)(x),y ), where sgsg stops gradients, θ denotes shadow-model parameters, and ϕφ contains enabled trainable quantizer parameters. The reported canonical activation and KV paths use energy normalization and frozen tables without an additional learned, absmax, or percentile clipping parameter. All low-bit paths in CanonQ-Omni are active during QAT, so training sees the coupled errors that the evaluation operator will introduce. Evaluation re-quantizes the shadow weights and uses the labeled activation/cache path. The norm is recomputed from the current block. Differentiating it in both encoding and decoding makes the block STE differ from the identity, as derived in Sec. 4.3. 3.3 Scalar/Vector Variants and Computational Cost CanonQ-W canonicalizes weights and retains uniform quantizers for enabled A/KV paths; CanonQ-Omni canonicalizes every enabled source. We also implement VQLLM-QAT, a raw-space control with group-fitted weight codebooks, the same tuple dimension and index rate, nearest-codeword assignment, and STE. This control changes transformation and table fitting together. Sources literally share a table only when (b,d)(b,d) matches; otherwise they share the construction. The suffix d denotes dWd_W; reported canonical A/K/V paths use dA=dK=dV=1d_A=d_K=d_V=1. Each tuple uses bdbd index bits. A shared table occupies 2bddqC2^bddq_C bits for qCq_C-bit components. Exhaustive assignment costs O(gS2bd)O(g_S2^bd) arithmetic per block and the Hadamard transform costs O(gSloggS)O(g_S g_S). These arithmetic costs motivate small d. Group metadata, table bytes, and unquantized modules are accounted for in Sec. A.4. 4 Theory: From Source Alignment to Optimization The fixed interface raises three questions: when do reference codebooks transfer, how do quantization residuals affect local task loss, and how does distortion influence training? 4.1 Bounding Frozen-Codebook Transfer Error Let Qd=(0,Id)Q_d=N(0,I_d). Haar-rotated normalized blocks converge to this reference in W2W_2 for fixed d (Prop. 1). The implemented R=PHDR=PHD is a structured orthogonal transform. For blocks independent of its random signs, Prop. 2 establishes sub-Gaussian coordinates, a marginal Gaussian-approximation bound controlled by ‖u‖33/‖u‖23\|u\|_3^3/\|u\|_2^3, and an exact mean tuple-covariance defect controlled by g‖u‖44/‖u‖24g\|u\|_4^4/\|u\|_2^4. Our transfer analysis explicitly accounts for deviations from the Gaussian reference through the checkpoint-specific joint tuple law. Write ω=(θ,ϕ)ω=(θ,φ) and let PS,dωP_S,d^ω denote the post-HDP tuple law induced by checkpoint ω under a fixed sampling rule over layers, groups, tokens, heads, and tuple positions. Definition 1 (Canonicalization defect). For any tuple law P with finite second moment, define Δd(P):=1dW2(P,Qd),ΔS,d(ω):=Δd(PS,dω). _d(P):= 1 dW_2(P,Q_d), _S,d(ω):= _d(P_S,d^ω). (4) This checkpoint-specific defect measures mismatch in the full joint tuple law, including coordinate dependence, structured-transform effects, and ϵ>0ε>0 normalization. It connects the reference-codebook design to the tensor distributions encountered during QAT. For a K-point codebook C⊂ℝdC ^d, let DP,d(C):=1dV∼Pdist(V,C)2,ℛP,d(C):=DP,d(C),D_P,d(C):= 1dE_V Pdist(V,C)^2, _P,d(C):= D_P,d(C), (5) and let ℛP,b,d⋆:=inf1≤|C|≤2bdℛP,d(C)R_P,b,d := _1≤|C|≤ 2^bdR_P,d(C). The numerical canonical table C^b,d C_b,d is allowed a squared-error fitting gap DQd,d(C^b,d)≤(ℛQd,b,d⋆)2+εvq.D_Q_d,d( C_b,d)≤(R_Q_d,b,d )^2+ _vq. (6) Theorem 1 (Frozen-codebook transfer). Let AbA_b be any 2b2^b-level scalar alphabet and AbdA_b^d its Cartesian product. Then ℛQd,b,d⋆≤ℛQd,d(Abd),ℛQd,d(C^b,d)≤ℛQd,b,d⋆+εvq.R_Q_d,b,d _Q_d,d(A_b^d), _Q_d,d( C_b,d) _Q_d,b,d + _vq. (7) For every actual tuple law P with finite second moment, ℛP,d(C^b,d)−ℛP,b,d⋆≤2Δd(P)+εvq.R_P,d( C_b,d)-R_P,b,d ≤ 2 _d(P)+ _vq. (8) More directly, define the reference margin mQ(Abd,C^b,d):=ℛQd,d(Abd)−ℛQd,d(C^b,d)m_Q(A_b^d, C_b,d):=R_Q_d,d(A_b^d)-R_Q_d,d( C_b,d). Then ℛP,d(C^b,d)−ℛP,d(Abd)≤2Δd(P)−mQ(Abd,C^b,d).R_P,d( C_b,d)-R_P,d(A_b^d)≤ 2 _d(P)-m_Q(A_b^d, C_b,d). (9) Hence a validated lower bound on the reference shaping gain transfers whenever it exceeds twice a validated upper bound on the canonicalization defect. The bound separates two routes to reliable transfer: better reference-table fitting reduces εvq _vq, while better canonical coordinates reduce Δd(P) _d(P). The margin condition in Eq. 9 quantifies when a reference reconstruction advantage survives deployment mismatch. These assignment results use γ=1γ=1; Lem. 3 extends the accounting to assign-then-gain decoding. Tuple dimension affects reference distortion, fitting error, and joint source mismatch together. We next incorporate downstream sensitivity to connect these representation effects to local task loss. 4.2 Bounding Local Task Loss from Quantization Let ΠC(d) _C^(d) apply C tuple by tuple to z=gRu/ρ(u)z= gRu/ρ(u). The implementation first assigns against C and then applies a nonnegative source-specific gain γ. Its actual canonical residual and block perturbation are therefore ηC,γ(z):=γΠC(d)(z)−z,eC,γ(u):=u^−u=ρ(u)gR⊤ηC,γ(z). _C,γ(z):=γ _C^(d)(z)-z, e_C,γ(u):= u-u= ρ(u) gR _C,γ(z). (10) Quantization errors also interact across sources. For a linear block, write W^=W+EW W=W+E_W and a^=a+eA a=a+e_A. Then W^a^−Wa=EWa+WeA+EWeA. W a-Wa=E_Wa+We_A+E_We_A. The cross term, and the analogous query/key/value terms in attention (Lem. 4), motivate joint adaptation. Theorem 2 (Gain-aware local task-loss certificate). Consider a single-site intervention that replaces block u by u+eC,γ(u)u+e_C,γ(u) while holding the other sites fixed. Suppose ℒL is C3C^3 on a neighborhood of the joining segment, ‖∇2ℒ(u)‖op≤Λu\|∇^2L(u)\|_op≤ _u, and ‖∇3ℒ(w)‖op≤M3\|∇^3L(w)\|_op≤ M_3 on that segment. Then |ℒ(u+eC,γ)−ℒ(u)| |L(u+e_C,γ)-L(u)| ≤ρ(u)g‖∇ℒ(u)‖2‖ηC,γ(z)‖2 ≤ ρ(u) g\| (u)\|_2\| _C,γ(z)\|_2 +ρ(u)2Λu2g‖ηC,γ(z)‖22+M3ρ(u)36g3/2‖ηC,γ(z)‖23. + ρ(u)^2 _u2g\| _C,γ(z)\|_2^2+ M_3ρ(u)^36g^3/2\| _C,γ(z)\|_2^3. (11) The certificate applies to a smooth downstream continuation or a hard-quantized branch whose downstream assignments remain fixed throughout a neighborhood of the intervention segment. It retains the local gradient term explicitly, covering activation and KV interventions away from coordinate-wise stationarity, and separates first-order sensitivity, curvature, and higher-order effects. For random blocks, set Z=gRU/ρ(U)Z= gRU/ρ(U) and choose ΛU≥‖∇2ℒ(U)‖op _U≥\|∇^2L(U)\|_op. The quadratic contribution is bounded by one half of the sensitivity-weighted distortion DS,γcurv(C):=1g[ρ(U)2ΛU‖ηC,γ(Z)‖22],D^curv_S,γ(C):= 1gE [ρ(U)^2 _U\| _C,γ(Z)\|_2^2 ], (12) without residual-isotropy assumptions. Expected certificates are finite when the displayed sensitivity-weighted moments are finite. Common-cut interventions retain cross-site Hessian terms; causally ordered sites require telescoping with the smoothness condition checked at each step (Cor. 1). 4.3 Quantization Distortion Controls STE Error The hard nearest-codeword objective is generally discontinuous at cell boundaries. We therefore compare its STE with a smooth gain-preserving reference. Set s(u)=ρ(u)/gs(u)=ρ(u)/ g, q=ΠC(d)(z)q= _C^(d)(z), and δa=‖q−z‖2/g _a=\|q-z\|_2/ g. If the backward pass uses Dq/Dz=IDq/Dz=I and treats γ as independent of u, for u≠0u≠ 0, its surrogate Jacobian is JSTE(u)=γ[I+R⊤(q−z)∇s(u)⊤],∥JSTE−γI∥op=γδa.J_STE(u)=γ [I+R (q-z)∇ s(u) ], \|J_STE-γ I\|_op=γ _a. (13) Thus, an identity STE at assignment induces a distortion-dependent block correction of rank at most one. If the group energy is detached in both encoding and decoding, the local surrogate Jacobian becomes γIγ I instead; additional clipping introduces its own derivative. For a downstream loss f with LfL_f-Lipschitz gradient, this identity yields the explicit bias certificate ∥gSTE−γ∇f(γu)∥2≤γδa∥∇f(γu)∥2+γ2(1+δa)Lfρ(u)δa=:BC(u).\|g_STE-γ∇ f(γ u)\|_2≤γ _a\|∇ f(γ u)\|_2+γ^2(1+ _a)L_fρ(u) _a=:B_C(u). (14) For the uniform tuple law PuP_u of a fixed group, δa(u)2=DPu,d(C) _a(u)^2=D_P_u,d(C). Under matching random-group/tuple sampling, δa(U)2=DP,d(C)E _a(U)^2=D_P,d(C). If gains, energies, downstream gradients, and assignment residuals are bounded on the analyzed region, then BC≤KδaB_C≤ K _a and BC(U)2≤K2DP,d(C)EB_C(U)^2≤ K^2D_P,d(C) (Cor. 3). A trained gain contributes an additional Jacobian-column defect ρ(u)δaρ(u) _a. These give codec-dependent sufficient conditions, rather than assuming alignment independently of the quantizer (yin2019understanding). For the full network, let F be the L-smooth gain-preserving reference obtained by replacing all enabled hard quantizers (including any uniform control paths) by the identity. Let GtG_t be the stochastic STE direction, t[⋅]=[⋅∣ℱt]E_t[·]=E[· _t] condition on the pre-minibatch history, and bt=tGt−∇F(ωt)b_t=E_tG_t-∇ F( _t). Under backward-map stability, integrability, and valid differentiation under expectation, Lem. 5 lifts the codec-dependent errors to a full-network bias certificate ‖bt‖2≤Bt\|b_t\|_2≤ B_t. Theorem 3 (Distortion-controlled STE stationarity). Assume F⋆>−∞F_ >-∞, ‖bt‖2≤Bt\|b_t\|_2≤ B_t, and the conditional STE variance satisfies t‖Gt−tGt‖22≤σ2E_t\|G_t-E_tG_t\|_2^2≤σ^2. For ωt+1=ωt−αtGt _t+1= _t- _tG_t with deterministic steps 0<αt≤1/(4L)0< _t≤ 1/(4L) and ST=∑t<TαtS_T= _t<T _t, 1ST∑t<Tαt‖∇F(ωt)‖22≤2(F(ω0)−F⋆)ST+52ST∑t<TαtBt2+Lσ2ST∑t<Tαt2. 1S_T _t<T _tE\|∇ F( _t)\|_2^2≤ 2(F( _0)-F_ )S_T+ 52S_T _t<T _tEB_t^2+ Lσ^2S_T _t<T _t^2. (15) With αt=c/T≤1/(4L) _t=c/ T≤ 1/(4L), the bound separates O(T−1/2)O(T^-1/2) optimization and sampling terms from the codec-dependent gradient-bias contribution. This establishes distortion-dependent stationarity bounds for the smooth gain-preserving reference under the stated SGD update rule. The appendix further develops a conditional PL bound and Voronoi-margin stability. Connection to evidence. The analysis separates source mismatch, sensitivity-weighted reconstruction error, and assignment distortion. Scalar W1W_1/KS diagnostics assess marginal alignment; zero-update/QAT comparisons test adaptation to the fixed codec; precision and dimension sweeps establish how these choices affect model quality. 5 Experiments The evaluation first locates the benefit across compression settings, then examines source alignment and adaptation, and finally tests longer-context likelihood, code generation, and mathematical reasoning. 5.1 Evaluation Setup We evaluate LLaMA3-1B/3B/8B and Qwen3-1.7B (llama3; yang2025qwen3), using 1B for the broad component study. Standard QAT uses FineWeb-Edu, sequence length 2048, effective batch size 64, AdamW, and up to 240K updates (≈31.5≈ 31.5B tokens). Learning rates are selected from 1,2,4×10−5\1,2,4\× 10^-5 on held-out validation data. The 8B CanonQ-Omni configuration reuses the selected 3B setting. The context study separately continues QAT at 8192 for 40K steps. We report WikiText-2 perplexity (PPL) (merity2017pointer) and the unweighted mean zero-shot accuracy over eight benchmarks. Accuracy is reported in percent; task definitions and evaluation details appear in Sec. A.1. The instruction-tuned extension reports greedy coding pass@1 and GSM8K final-answer exact match under a separate protocol (Sec. A.5) (austin2021program; chen2021evaluating; cobbe2021training). ParetoQ denotes its weight quantizer combined with the uniform A/KV paths specified in Sec. A.2. Uniform A4/KV2 uses 15/3 active levels; canonical scalar tables use 16/4 at the same index rate. Comparisons evaluate complete trained configurations, including these differences. Full settings and per-task results appear in Apps. A and B.3. Each reported configuration uses one training seed. Table 1: Joint-compression and cross-model results. Columns report PPL ↓ and eight-task accuracy ↑ . The suffix denotes dWd_W; canonical A/K/V paths use d=1d=1. CanonQ-Omni uses canonical KV quantization in the headline KV2 results. Full task scores appear in Tabs. 11, 12 and 13. LLaMA3-1B, quantized KV W2A8KV2 W2A4KV2 Method PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ FP Model 9.75 58.54 9.75 58.54 RTN (PTQ) 5.8e5 37.31 5.5e5 36.72 GPTQ (PTQ) 5.3e4 38.23 9.5e4 37.83 QuaRot (PTQ) 3.4e4 37.39 4.1e4 38.11 KIVI 1.1e5 34.85 3.2e5 35.16 ParetoQ 47.58 45.13 90.57 40.47 VQLLM-QAT-d4 34.35 45.35 76.51 42.32 CanonQ-W-d1 30.07 46.41 64.40 42.49 CanonQ-W-d2 30.19 47.56 64.48 42.95 CanonQ-W-d4 31.27 46.91 76.19 41.97 CanonQ-Omni-d1 15.70 53.45 15.42 53.56 CanonQ-Omni-d2 16.43 52.67 15.90 53.16 LLaMA3-3B W2A4 W2A4KV2 Method PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ FP Model 7.73 65.11 7.73 65.11 RTN (PTQ) 4.2e5 35.74 1.1e6 36.78 GPTQ (PTQ) 7.9e4 38.12 8.9e4 37.88 QuaRot (PTQ) 6.7e2 37.23 1.5e4 37.95 ParetoQ 17.88 53.15 92.64 39.39 VQLLM-QAT-d4 30.71 46.30 52.82 42.07 CanonQ-Omni-d1 10.66 61.16 10.93 58.37 CanonQ-Omni-d2 10.50 61.14 11.16 60.04 Qwen3-1.7B and LLaMA3-8B Q3-1.7B (W2A4) L3-8B (W2A4KV2) Method PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ FP Model 16.33 58.10 6.14 70.14 ParetoQ 43.56 42.35 134.36 37.87 CanonQ-Omni-d1 17.80 50.96 9.41 59.81 5.2 Large Quality Gains at W2A4KV2 The central result is the large separation under W2A4KV2 (Tab. 1). On LLaMA3-1B, CanonQ-Omni-d1 reaches 15.42 PPL and 53.56% mean accuracy, versus 90.57 and 40.47% for ParetoQ: 5.87×5.87× lower PPL and a 13.09-point accuracy gain. Relative to weight-only canonicalization with CanonQ-W-d1, it lowers PPL by 4.18×4.18× and raises accuracy by 11.07 points. This direct comparison locates the larger benefit in extending the canonical interface to activations and caches. The gains persist at larger scales. On 3B, CanonQ-Omni-d2 reaches 11.16/60.04 (PPL/accuracy), versus 92.64/39.39 for ParetoQ: 8.30×8.30× lower PPL and a 20.65-point gain. At 8B, CanonQ-Omni-d1 reaches 9.41/59.81 versus 134.36/37.87, giving 14.28×14.28× lower PPL and 57.9%57.9\% higher accuracy. To reduce tuning costs, we reuse the best training hyperparameters identified on 3B for 8B. We expect a dedicated 8B hyperparameter search to further improve accuracy and narrow the gap to FP. On Qwen3-1.7B W2A4, CanonQ-Omni-d1 obtains 17.80/50.96 versus 43.56/42.35, improving PPL by 2.45×2.45× and accuracy by 8.61 points. Together, these results show that the same frozen-codebook construction supports joint quantization across model scales and transfers to a second model family. 5.3 Gains Grow Beyond Weight-Only Quantization Weight-only quantization gives modest differences. At 1B W2A16, ParetoQ and CanonQ-W-d2 achieve similar quality: 12.66/55.53 and 12.86/55.45 (PPL/accuracy), respectively (Tab. 2). At W1A16, CanonQ-W-d1 lowers PPL from 18.13 to 16.93, while dW=2d_W=2 gives the highest accuracy, 51.81%. At matched dW=4d_W=4, canonical coding improves raw-space VQLLM-QAT from 22.36/47.90 to 17.47/51.17, showing a clearer benefit from canonical coordinates and reference codebooks at one-bit weight precision. Activation compression produces the larger separation. At W2A8, CanonQ-Omni-d1 improves ParetoQ’s 18.22/51.09 to 13.25/55.60. At W2A4, weight-only canonicalization remains near 35 PPL and 44% accuracy, whereas CanonQ-Omni-d1 reaches 13.84/54.71: 2.35×2.35× lower PPL and a 10.59-point gain over ParetoQ. All eight tasks improve, with gains from 1.51 points on BoolQ to 23.62 on HellaSwag (Tabs. 10 and 3). Moving from A8 to A4 raises canonical-path PPL by only 0.59 and lowers accuracy by 0.89 points, versus 14.35 PPL and 6.97 points for ParetoQ. Extending the interface to activations thus preserves quality as activation precision tightens. Table 2: LLaMA3-1B results without KV-cache quantization. Best and second-best non-FP results in each column are shaded and underlined, respectively. Full task scores appear in Tabs. 9 and 10. (a) Weight-only quantization W1A16 W2A16 Method PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ FP Model 9.75 58.54 9.75 58.54 RTN (PTQ) 4.2e8 33.7 1.2e6 36.40 GPTQ (PTQ) 7.9e4 38.03 1.2e4 38.21 QuaRot (PTQ) 1.3e4 38.34 3.2e3 38.25 ParetoQ 18.13 51.55 12.66 55.53 VQLLM-QAT-d4 22.36 47.90 13.77 55.12 VQLLM-QAT-d8 20.10 49.85 N/A N/A CanonQ-W-d1 16.93 51.42 12.94 55.11 CanonQ-W-d2 17.41 51.81 12.86 55.45 CanonQ-W-d4 17.47 51.17 12.96 55.45 CanonQ-W-d8 18.79 50.56 N/A N/A (b) Weight–activation quantization W1A8 W2A8 W2A4 Method PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ PPL ↓ Acc. ↑ FP Model 9.75 58.54 9.75 58.54 9.75 58.54 RTN (PTQ) 1.1e6 34.94 1.2e6 36.14 1.1e6 36.78 GPTQ (PTQ) 1.0e5 38.09 1.3e4 38.16 6.9e4 36.86 QuaRot (PTQ) 1.3e4 38.36 3.2e3 38.42 4.6e3 38.02 ParetoQ 21.27 48.65 18.22 51.09 32.57 44.12 VQLLM-QAT-d4 25.98 46.74 16.99 52.73 42.65 43.45 VQLLM-QAT-d8 25.37 47.47 N/A N/A N/A N/A CanonQ-W-d1 23.54 48.37 14.38 54.76 35.16 44.14 CanonQ-W-d2 21.23 49.45 13.49 53.82 35.20 43.66 CanonQ-W-d4 22.10 48.75 16.01 52.95 35.53 44.53 CanonQ-W-d8 20.54 49.59 N/A N/A N/A N/A CanonQ-Omni-d1 17.76 51.25 13.25 55.60 13.84 54.71 CanonQ-Omni-d2 17.47 49.96 13.42 54.63 14.51 54.53 Vector dimension changes the tradeoff. At 1B W2A4KV2, scalar CanonQ-Omni-d1 leads dW=2d_W=2 on both metrics. At 3B, dW=2d_W=2 raises accuracy by 1.67 points while PPL increases from 10.93 to 11.16. The reference-law comparison in Thm. 1 explains the reconstruction choices available at a fixed index rate; task-level results determine which choice is useful after adaptation. Together, these results support selecting dimension by validation quality and lookup cost within the shared interface. 5.4 Canonicalization and QAT Play Complementary Roles We next examine the complementary roles of source canonicalization and model adaptation through distribution diagnostics, zero-update comparisons, and cache-path ablations. HDP aligns dynamic-source marginals. Layer-mean W1W_1 distances to the Gaussian reference fall from 0.101 to 0.010 for activations, 0.060 to 0.026 for keys, and 0.030 to 0.018 for values; weights change from 0.015 to 0.017 (Fig. 2). The largest distributional change is in activations, which also show the strongest separation between weight-only and full-stack canonicalization at A4. Keys and values move closer to the same reference, while weights need little marginal correction. These marginal measurements support the use of a shared reference construction for heterogeneous W/A/K/V sources. The fixed codec benefits from task-aware adaptation. At 1B W2A4, applying the scalar canonical W/A codec with zero updates gives 7043.43/38.83, versus 13.84/54.71 after QAT (Tab. 7). PPL falls approximately 509×509× while the table stays fixed. The codec supplies reconstruction choices, but the pretrained network must still adapt to using them throughout the computation. Marginal alignment and task-aware training thus address different parts of the problem. The KV2 zero-update comparison also changes the cache path and measures their combined effect. The cache path also matters. The uniform-KV CanonQ-Omni-d1 ablation reaches 22.82/47.72 at W2A8KV2 and 21.65/48.30 at W2A4KV2, compared with 15.70/53.45 and 15.42/53.56 for canonical KV. The corresponding accuracy gains are 5.73 and 5.26 points (Tab. 8), so the cache contribution persists at both activation precisions. At 1B, moving from W2A4 to W2A4KV2 costs the complete canonical path 1.58 PPL and 1.15 accuracy points, compared with 58.00 PPL and 3.65 points for ParetoQ. The ablation evaluates the complete canonical cache path, combining transformation, active levels, normalization, and backward behavior. 5.5 Stable Relative Perplexity Within the 8K Training Range After 8K continued QAT for 40K updates, we evaluate LLaMA3-1B W2A4KV2 at 2K/4K/8K. At 8K, CanonQ-Omni achieves 14.86 PPL, versus 39.50 for CanonQ-W and 45.63 for ParetoQ. Its PPL relative to the same-length FP reference stays within 1.691.69–1.74×1.74×, showing a stable relative quality gap as the evaluation window grows within the trained range. Full results and protocol appear in App. C. Table 3: Code generation and mathematical reasoning on MobileLLM-Pro-Instruct-1B W2A16KV16. Scores are percentages, with correct-task counts in parentheses. MBPP/HumanEval report greedy pass@1; GSM8K reports final-answer exact match (EM). Quantized settings share the teacher and evaluation step. GSM8K also guides learning-rate selection; see Sec. A.5. Benchmark Metric Tasks FP base ParetoQ VQLLM-QAT-d4 CanonQ-d1 MBPP pass@1 500 46.4 (232) 28.0 (140) 27.8 (139) 32.2 (161) HumanEval pass@1 164 60.4 (99) 43.9 (72) 33.5 (55) 62.2 (102) GSM8K EM 1319 53.8 (709) 34.3 (453) 32.7 (431) 47.8 (630) 5.6 Gains Extend to Code Generation and Reasoning We evaluate MobileLLM-Pro-Instruct-1B at W2A16KV16 using the shared-teacher protocol in Sec. A.5. These experiments extend the evaluation to generated solutions under weight quantization. Mathematical reasoning. On GSM8K, CanonQ-d1 solves 630 of 1,319 problems (47.8%), versus 453 for ParetoQ and 431 for VQLLM-QAT-d4. This gives a 39.1%39.1\% relative improvement over the stronger quantized baseline and retains 88.9%88.9\% of the FP score. GSM8K guides learning-rate selection as specified in Sec. A.5. Code generation. CanonQ-d1 achieves 32.2% MBPP and 62.2% HumanEval greedy pass@1 (Tab. 3), exceeding ParetoQ by 4.2 and 18.3 points. The HumanEval gain is a 41.7%41.7\% relative improvement. The FP reference scores 60.4% (paired comparison: p=0.74p=0.74). Neither coding benchmark guides selection, and CanonQ leads the evaluated quantized baselines on all three generation benchmarks. 6 Discussion and Limitations A shared interface for coupled errors. CanonQ separates reusable reconstruction choices from task-aware adaptation. Canonicalization makes a fixed reference alphabet useful across heterogeneous sources, while joint QAT adapts the network to their interacting errors. The larger gains under joint compression, together with the alignment and zero-update studies, support this division of roles. These results favor selecting the codec under the complete target precision configuration: methods with similar weight-only quality separate sharply when activations and caches are also quantized. Evaluating the intended W/A/KV format therefore captures interactions that weight-only comparisons leave untested. Choosing scalar or vector assignment. The interface supports both choices at a fixed index rate. Scalar tables give the strongest reported 1B W2A4KV2 result; vector assignment improves 3B accuracy with a small PPL tradeoff. Dimension is therefore a practical choice guided by validation quality and lookup cost. Joint tuple statistics and sensitivity-weighted residuals offer concrete quantities for explaining these differences. Deployment and further evaluation. The present study establishes model quality at extreme low bit rates. Deployment evaluation will complement these results with exported-model size, KV-storage, and prefill/decode throughput measurements for fused kernels, accounting for group metadata and unquantized modules (Sec. A.4). Evaluation beyond the 8K training range, reasoning benchmarks with independent hyperparameter selection, and repeated training seeds are directions for extending this evidence. 7 Conclusion We introduced CanonQ, a unified framework for weight, activation, and KV-cache quantization using canonical coordinates and frozen reference codebooks. Our theory links source mismatch to reconstruction, task loss, and normalization-aware STE bias. Experiments establish substantial quality gains under W2A4KV2 across model scales, with further improvements in coding and reasoning and stable relative perplexity within the 8K training range. References Contents 1 Introduction 2 Related Work 3 CanonQ: Unified W/A/KV Quantization 3.1 Reusable Codebooks in Canonical Coordinates 3.2 Joint Adaptation to Quantization Errors 3.3 Scalar/Vector Variants and Computational Cost 4 Theory: From Source Alignment to Optimization 4.1 Bounding Frozen-Codebook Transfer Error 4.2 Bounding Local Task Loss from Quantization 4.3 Quantization Distortion Controls STE Error 5 Experiments 5.1 Evaluation Setup 5.2 Large Quality Gains at W2A4KV2 5.3 Gains Grow Beyond Weight-Only Quantization 5.4 Canonicalization and QAT Play Complementary Roles 5.5 Stable Relative Perplexity Within the 8K Training Range 5.6 Gains Extend to Code Generation and Reasoning 6 Discussion and Limitations 7 Conclusion References A Experimental Details A.1 Models, Training, and Evaluation A.2 Quantization Paths and Baselines A.3 HDP Maps and Frozen Tables A.4 Index Rate and Stored Footprint A.5 Code Generation and Mathematical Reasoning B Supplementary Experimental Results B.1 Marginal Alignment and Task Adaptation B.2 Canonical versus Uniform Cache Quantization B.3 Per-task Results across Precisions and Models C Context-Length Robustness after 8K Continued QAT D Theory Details and Proofs D.1 The Haar Reference and the Implemented Hadamard Map D.2 Root-Distortion Stability and Frozen-Table Transfer D.3 Gain-Aware Task-Loss Bounds D.4 The STE Jacobian Induced by the Canonical Codec D.5 Distortion-Controlled Optimization Consequences D.6 Voronoi Margins, Code Switching, and Index Stability Appendix A Experimental Details A.1 Models, Training, and Evaluation We use pretrained LLaMA-3.2-1B/3B, LLaMA3-8B, and Qwen3-1.7B checkpoints with their native tokenizers (llama3; yang2025qwen3). The 1B model supports the broad precision and component study; the larger models test transfer of the same frozen-table construction. Standard QAT starts from the corresponding full-precision checkpoint and samples a continuous FineWeb-Edu stream. The architecture, vocabulary, and positional encoding are unchanged. Training excludes the evaluation datasets. Optimization. We train on eight H100/A100-80G GPUs, with eight sequences per GPU. The standard recipe uses 2048-token sequences, effective batch size 64, and at most 240K updates, or approximately 31.5B tokens. We use AdamW with β=(0.9,0.999)β=(0.9,0.999), ϵ=10−8ε=10^-8, zero weight decay, and global gradient-norm clipping at 1.01.0. A cosine schedule follows 1000 linear-warmup steps and decays to zero. Learning rates are selected from 1,2,4×10−5\1,2,4\× 10^-5 on held-out validation data; enabled scale and gain parameters share the weights’ learning rate. For the reported 8B CanonQ-Omni configuration, we reuse the best training hyperparameters selected at 3B to reduce tuning costs. The training seed is 42. The 8K/40K continuation and instruction-tuned coding and reasoning extension use the separate protocols in Apps. C and A.5. Language modeling and downstream tasks. For WikiText-2, we concatenate tokenized test documents without added separators, split the stream into non-overlapping 2048-token windows, and drop the trailing partial window. PPL is the exponential of mean next-token negative log-likelihood over the scored tokens. All methods use the same tokenized corpus and evaluation implementation. Zero-shot accuracy is the unweighted mean over ARC-Easy and ARC-Challenge (clark2018think), BoolQ (clark2019boolq), PIQA (bisk2020piqa), SIQA (sap2019socialiqa), HellaSwag (zellers2019hellaswag), OpenBookQA (mihaylov2018can), and WinoGrande (sakaguchi2021winogrande), using common prompts and answer normalization across methods. Evaluation re-quantizes shadow weights and keeps the trained activation/cache operators enabled. Each result represents one training seed unless stated otherwise. A.2 Quantization Paths and Baselines Tab. 4 summarizes which sources each method quantizes. A16/KV16 denotes a floating-point path; other labels specify index bits. A canonical (b,d)(b,d) table has 2bd2^bd vector codewords, so a bdbd-bit index represents d scalars. Canonical A/K/V paths use d=1d=1 throughout; method suffixes specify dWd_W. Table 4: Quantization paths. “Canonical” means HDP, energy normalization, and a frozen Gaussian-reference table. All QAT paths are active during training and evaluation. Method Weights Activations KV cache ParetoQ learned uniform grid dynamic uniform dynamic uniform VQLLM-QAT group-fitted raw codebook dynamic uniform dynamic uniform CanonQ-W canonical dynamic uniform dynamic uniform CanonQ-Omni canonical canonical canonical HDP+LM PTQ canonical, no updates canonical, no updates uniform if enabled Baseline scope. ParetoQ supplies the weight quantizer, extended here with the uniform A/KV operators below (liu2025paretoq). VQLLM-QAT fits raw-space weight tables by k-means/LBG; matching its index rate and tuple dimension to CanonQ-W changes both source coordinates and table construction. RTN and GPTQ specify weight PTQ procedures; their low-A/KV rows evaluate composite configurations. KIVI specifies a cache codec with inherited W/A quantization (frantar2022gptq; ashkboos2024quarot; liu2024kivi). These composite rows measure end-to-end configurations and do not isolate the original weight or cache algorithm. Uniform weight and dynamic-source quantizers. The ParetoQ weight path uses a learned per-row scale α: W1 reconstructs αsign(w) (w), while W2 rounds to α4−3,−1,1,3 α4\-3,-1,1,3\. For a dynamic A/KV group x, the implemented symmetric operator is M=maxi|xi|,s=2b−1−1M+ϵ,x^=round(sx)s+ϵ.M= _i|x_i|, s= 2^b-1-1M+ε, x= round(sx)s+ε. (16) Its active cardinalities are 15/255/3 at A4/A8/KV2, compared with 16/256/4 for canonical scalar tables. Consequently, comparisons at the same index rate can still differ in cardinality, normalization, and backward behavior. Canonical paths. All sources use the codec in Eqs. 1 and 3, with source-specific (gS,bS,dS,RS,γS,ℓ)(g_S,b_S,d_S,R_S, _S, ). Activations and KV states use energy normalization without an additional learned, absmax, or percentile clipping parameter. K and V are quantized independently and decoded to their original coordinates before attention. Assignment uses the identity STE, with the energy differentiated through encoding and decoding as in Eq. 13. A source with a fixed unit gain sets γS,ℓ=1 _S, =1. A.3 HDP Maps and Frozen Tables Grouping and rotations. Canonical groups contain gW=gA=128g_W=g_A=128 scalar coordinates and gK=gV=64g_K=g_V=64. Each source uses one fixed map RS=PSHDSR_S=P_SHD_S across layers. Seeds are sS=s0+oSs_S=s_0+o_S, with s0=0s_0=0 and offsets oW=0o_W=0, oA=1o_A=1, oK=oV=2o_K=o_V=2; a seeded PRNG draws signs and then the permutation. Thus K and V share a map, while W/A/KV maps differ. The normalized fast Hadamard transform uses hammer.ops.fast_hadamard_transform with a PyTorch butterfly fallback. Power-of-two groups require no padding. Quantized linear layers include attention q,k,v,oq,k,v,o and MLP w1,w2,w3w_1,w_2,w_3 projections; token embeddings and the language-model head remain unrotated. Offline construction and assignment. Scalar tables are fitted to (0,1)N(0,1) using double-precision Lloyd–Max iterations; vector tables use Lloyd/LBG assignments and centroid updates. The fitting gap εvq _vq in Eq. 6 explicitly accounts for numerical codebook optimization. The scalar W1 levels are ±0.7978846± 0.7978846; the scalar W2/KV2 levels are approximately ±0.4528± 0.4528 and ±1.5104± 1.5104. Exact A4/A8 levels and vector centers are serialized with the configuration. Tables remain frozen through QAT and evaluation. For W1, the evaluated dimensions d∈1,2,4,8d∈\1,2,4,8\ give K∈2,4,16,256K∈\2,4,16,256\; for W2, d∈1,2,4d∈\1,2,4\ gives K∈4,16,256K∈\4,16,256\. Scalar A4/A8/KV2 tables have 16/256/4 entries. Sources literally share a table only when (b,d)(b,d) and the codewords match. Scalar assignment uses sorted midpoint thresholds and torch.bucketize with right=False, so midpoint ties select the lower level. Vector assignment uses argmin and selects the lowest index on ties. The same rule is used during training and evaluation. A.4 Index Rate and Stored Footprint The label W2A4KV2 specifies an index-rate tuple of 2/4/2/2; it excludes decoding factors, table storage, and unquantized modules. For source S with NSN_S scalar values, the stored rate and once-per-model allocation are bSstored b_S^stored =bSidx+BSdecode+BSlocal-table+BSauxNS, =b_S^idx+ B_S^decode+B_S^local -table+B_S^auxN_S, (17) Bresident B^resident =Bglobal-table+Brotation+Bconstant. =B^global -table+B^rotation+B^constant. (18) Here auxiliary bits include offsets, padding, and alignment. Globally shared objects are counted once; a group-fitted codebook contributes to local-table storage. Decoding metadata. A group of g values with r stored qsq_s-bit decoding scalars adds rqs/grq_s/g bits per value. A static canonical weight group needs the folded factor αg=γgρg/g _g= _g _g/ g in u^g=αgRg⊤z^g u_g= _gR_g z_g. Storing this product requires one scalar; retaining norm and gain separately requires two. Table 5 gives both cases for the declared scalar group sizes. The rates are representation accounting, not measured exported bytes. Table 5: Canonical W2A4KV2 storage before padding and unquantized modules. Each decoding scalar is 16 bits; r counts independent factors after folding. A rates apply only to materialized quantized activation buffers. Global tables are separate. Source Index bits Group g Stored bits, r=1r=1 Stored bits, r=2r=2 Weights 2 128 2.125 2.250 Activations 4 128 4.125 4.250 Keys / values 2 64 2.250 2.500 The same accounting gives 2.25 bits per value for a symmetric two-bit cache with one 16-bit scale per 64 values. An asymmetric cache with a scale and offset uses 2+(qs+qz)/g2+(q_s+q_z)/g bits per value; at g=32g=32 and qs=qz=16q_s=q_z=16, this is 3.0. Per-row ParetoQ weight scales instead contribute 16∑ℓmℓ/∑ℓmℓnℓ16 _ m_ / _ m_ n_ bits per weight for matrices of shape mℓ×nℓm_ × n_ . Total storage therefore accounts for both index bits and method-specific metadata. Shared and group-fitted tables. A canonical (b,d)(b,d) table with qCq_C-bit components occupies 2bddqC2^bddq_C bits. At 16 bits per component, W2-d1/d2/d4 tables require 8/64/2048 bytes and an A4-d1 table requires 32 bytes. Each distinct exported table is counted once. In contrast, a raw-space table shared by each block of 128 output rows contributes BWlocal-table=∑ℓ⌈mℓ/128⌉ 2bddqC.B_W^local -table= _ m_ /128 \,2^bddq_C. (19) These table-sharing blocks differ from the 128-scalar canonical normalization groups. Any separately retained raw-codebook normalization factors add their own metadata; gains absorbed into raw codewords are not counted twice. Deployment implications. HDP maps can be regenerated from seeds without storing dense matrices; exported signs, permutations, and replicated tables still occupy resident bytes. KV decoding factors persist with cached indices. Activation factors can remain on chip in a fused kernel, so their contribution depends on whether an activation buffer is materialized. The relevant deployment quantity is Mtotal(B,L)=MW+Mresident+MKV(B,L)+MApeakM_total(B,L)=M_W+M_resident+M_KV(B,L)+M_A^peak, where MWM_W includes unquantized modules. Actual memory and speed also depend on packing, lookup, and kernel boundaries. A.5 Code Generation and Mathematical Reasoning Tab. 3 evaluates MobileLLM-Pro-Instruct-1B at W2A16KV16. Token embeddings and output/language-model heads remain full precision. CanonQ-d1, VQLLM-QAT-d4, and ParetoQ use canonical scalar, raw vector, and uniform weight quantizers, respectively. All three arms start from the same instruction-tuned FP checkpoint and use it as a full-precision distillation teacher. Checkpoint and learning-rate selection. All quantized methods are evaluated at step 279236. For each method with a learning-rate sweep, GSM8K exact match selects the setting used for all three benchmarks. The reported reasoning score therefore reflects GSM8K-guided selection, while both coding benchmarks evaluate the selected model without contributing to selection. Coding metrics and decoding. Evaluation uses a dialog prompt: HumanEval is zero-shot with maximum prompt/generation lengths 2000/1024, while MBPP is three-shot with maximum generation length 256. Greedy decoding generates one sample per task, with seed 42 and batch size 32. The FP reference uses the same harness and decoding settings. Generated code runs in an isolated subprocess with a five-second timeout; a timeout or nonzero exit is a failure. Thus pass@1 is the fraction of solved tasks, and parentheses in Tab. 3 give integer counts. Mathematical-reasoning metric. GSM8K exact match measures final-answer correctness on 1,319 problems. FP, ParetoQ, VQLLM-QAT-d4, and CanonQ-d1 solve 709, 453, 431, and 630 problems, respectively. We display scores to one decimal place and compute improvements from the counts before rounding. CanonQ gives net gains of 177 and 199 correct answers over the quantized baselines. Relative to ParetoQ, this is 100(630−453)/1319=13.4100(630-453)/1319=13.4 accuracy points or 100(630/453−1)=39.1%100(630/453-1)=39.1\% relative improvement. CanonQ retains 630/709=88.9%630/709=88.9\% of the FP score and recovers (630−453)/(709−453)=69.1%(630-453)/(709-453)=69.1\% of the baseline-to-FP gap. Uncertainty and interpretation. On HumanEval, CanonQ and FP solve 102 and 99 tasks, with 20 solved only by CanonQ and 17 only by FP. Exact McNemar testing gives p=0.74p=0.74; the paired-bootstrap 95% interval for the accuracy difference is [−5.5,9.1][-5.5,9.1] points. This interval spans both signs, and no equivalence margin was tested. On MBPP, the discordant counts are 17 and 88 (p<10−12p<10^-12), consistent with the 14.2-point FP gap. These comparisons quantify variation across benchmark tasks under greedy decoding; training-seed variation is a separate quantity. For GSM8K, the available aggregate counts support accuracy and count comparisons; paired significance testing requires per-problem outcomes. Appendix B Supplementary Experimental Results B.1 Marginal Alignment and Task Adaptation We test the scalar-marginal consequence of canonicalization using a trained LLaMA3-1B W2A4KV2 checkpoint. Weights come from the checkpoint; activations and KV states are captured at quantizer inputs on a fixed calibration batch. Each comparison uses identical groups, with gW=gA=128g_W=g_A=128 and gK=gV=64g_K=g_V=64, and identical energy normalization. Thus znorot=gSx/(‖x‖2+ϵ)z_norot= g_Sx/(\|x\|_2+ε) and zHDP=RSznorotz_HDP=R_Sz_norot differ only by rotation. Measurements precede codeword assignment. For each layer we compute scalar W1W_1 distance to (0,1)N(0,1) and the KS distance supt|F^(t)−Φ(t)| _t| F(t)- (t)|. Table 6 averages layers equally. HDP reduces activation W1W_1 by 90%, key W1W_1 by 57%, and value W1W_1 by 40%; weights are already close and show a small increase. The range of source-wise means narrows from 0.0150.015–0.1010.101 to 0.0100.010–0.0260.026. This explains the shared marginal reference in Fig. 2. These statistics assess marginal alignment. Estimating ΔS,d _S,d requires a W2W_2 assessment of the corresponding tuple law, including coordinate dependence when d>1d>1. Figure 2: Paired scalar marginals before and after HDP, using identical normalized groups. Dynamic A/K/V sources approach the Gaussian reference; weights remain near it. These scalar diagnostics illustrate alignment with the shared design reference. Full statistics appear in Tab. 6. Table 6: Paired scalar-marginal distances to (0,1)N(0,1) for a trained LLaMA3-1B W2A4KV2 checkpoint. Values are layer means ± standard deviations, not uncertainty across training seeds. Both conditions use the same groups and normalization. W1W_1 to (0,1)N(0,1) KS to (0,1)N(0,1) Source No rotation HDP No rotation HDP Weights W 0.015±0.0080.015± 0.008 0.017±0.0120.017± 0.012 0.006±0.0030.006± 0.003 0.010±0.0060.010± 0.006 Activations A 0.101±0.0550.101± 0.055 0.010±0.0040.010± 0.004 0.039±0.0190.039± 0.019 0.005±0.0020.005± 0.002 Keys K 0.060±0.0170.060± 0.017 0.026±0.0130.026± 0.013 0.035±0.0110.035± 0.011 0.014±0.0060.014± 0.006 Values V 0.030±0.0140.030± 0.014 0.018±0.0080.018± 0.008 0.016±0.0060.016± 0.006 0.009±0.0030.009± 0.003 Adaptation to a fixed codec. The zero-update control in Tab. 7 enables the canonical W/A codec directly on the pretrained 1B model, with scalar tables, group size 128, rotation seed 0, and no gradient updates or layer reconstruction. At W2A4, QAT reduces PPL from 7043.43 to 13.84 and raises accuracy by 15.88 points without changing table capacity. The result demonstrates the importance of model adaptation to this fixed codec. The KV2 pair additionally changes uniform KV to canonical KV, so its improvement reflects both adaptation and the cache path. Table 7: Zero-update canonical PTQ and trained CanonQ-Omni-d1 on LLaMA3-1B. The W2A4 pair shares the W/A codec; the KV2 pair also changes the cache codec. Format Method / cache path PPL ↓ Avg. ZS ↑ FP Full precision 9.75 58.54 W2A4 HDP+LM PTQ 7043.43 38.83 W2A4 CanonQ-Omni-d1 QAT 13.84 54.71 W2A4KV2 HDP+LM PTQ, uniform KV 8868.91 36.76 W2A4KV2 CanonQ-Omni-d1 QAT, canonical KV 15.42 53.56 B.2 Canonical versus Uniform Cache Quantization Tab. 8 compares cache codecs while keeping W/A quantization canonical. The uniform-KV variant uses the dynamic uniform operator in Sec. A.2. Canonical KV improves both PPL and eight-task accuracy at A8 and A4, extending the benefit beyond activation handling. This comparison evaluates the complete cache design, including its transform, active levels, normalization, and backward graph. Table 8: Cache-path ablation for LLaMA3-1B CanonQ-Omni-d1. W/A paths remain canonical; the comparison changes the complete cache path. Format Cache path PPL ↓ Avg. ZS ↑ W2A8KV2 uniform 22.82 47.72 canonical 15.70 53.45 W2A4KV2 uniform 21.65 48.30 canonical 15.42 53.56 B.3 Per-task Results across Precisions and Models Tables 9–13 report the complete eight-task scores. ARC-e/c, HellaS., OBQA, and Wino. abbreviate ARC-Easy/Challenge, HellaSwag, OpenBookQA, and WinoGrande. The mean weights the eight tasks equally. Weight-only and activation precision. In Tab. 9, 1B W2A16 gives closely matched results for ParetoQ and CanonQ-W-d2: 12.66/55.53 versus 12.86/55.45 (PPL/accuracy). At W1A16, CanonQ-W-d1 has the lowest PPL, while dW=2d_W=2 has the highest accuracy. The matched-dW=4d_W=4 comparison improves raw-space VQLLM-QAT from 22.36/47.90 to 17.47/51.17. In Tab. 10, extending canonicalization to activations preserves low PPL when the uniform activation path degrades: at W2A4, CanonQ-Omni-d1 achieves 13.84 PPL against 35.16 for CanonQ-W-d1. Table 9: LLaMA3-1B weight-only quantization. All task scores are zero-shot accuracies (%); PPL is measured on WikiText-2. Method ARC-e ARC-c BoolQ PIQA SIQA HellaS. OBQA Wino. Avg. ZS ↑ PPL ↓ FP16 Base 64.57 42.85 65.35 75.05 44.91 64.31 50.00 61.25 58.54 9.75 W2A16 RTN (PTQ) 27.19 24.69 54.03 50.00 35.89 25.43 23.05 50.94 36.40 1.2e6 GPTQ (PTQ) 26.95 25.31 61.66 49.12 36.67 25.47 29.69 50.78 38.21 1.2e4 QuaRot (PTQ) 28.12 24.69 62.35 50.34 36.23 26.72 28.12 49.45 38.25 3.2e3 ParetoQ 64.60 39.69 61.05 72.75 42.79 56.74 47.85 58.75 55.53 12.66 VQLLM-QAT-d4 62.89 40.62 62.65 72.61 42.63 54.43 46.09 59.06 55.12 13.77 CanonQ-W-d1 64.38 38.05 62.71 72.36 43.21 56.19 46.68 57.27 55.11 12.94 CanonQ-W-d2 62.77 41.02 63.91 71.68 43.21 56.35 45.31 59.38 55.45 12.86 CanonQ-W-d4 64.34 41.09 61.51 72.41 43.46 56.46 47.66 56.64 55.45 12.96 W1A16 RTN (PTQ) 25.0 22.5 37.6 49.5 32.9 25.0 27.1 49.6 33.7 4.2e8 GPTQ (PTQ) 28.59 22.73 62.32 50.29 36.62 25.29 28.71 49.69 38.03 7.9e4 QuaRot (PTQ) 28.16 26.02 62.41 48.58 37.16 24.80 28.71 50.86 38.34 1.3e4 ParetoQ 59.73 35.62 61.54 68.65 42.29 46.48 43.75 54.38 51.55 18.13 VQLLM-QAT-d4 53.55 31.72 60.22 65.33 40.87 40.03 37.11 54.38 47.90 22.36 VQLLM-QAT-d8 57.77 36.80 59.83 66.65 41.60 42.93 40.62 52.58 49.85 20.10 CanonQ-W-d1 58.95 35.55 61.66 68.65 42.77 47.20 42.97 53.59 51.42 16.93 CanonQ-W-d2 60.66 36.41 61.96 68.60 41.70 46.76 42.19 56.17 51.81 17.41 CanonQ-W-d4 58.79 36.25 61.99 68.16 41.75 46.98 40.82 54.61 51.17 17.47 CanonQ-W-d8 57.77 34.63 59.68 68.10 41.63 44.85 42.38 55.47 50.56 18.79 Table 10: LLaMA3-1B with quantized weights and activations, retaining the floating-point cache. The three panels vary weight and activation precision. Method ARC-e ARC-c BoolQ PIQA SIQA HellaS. OBQA Wino. Avg. ZS ↑ PPL ↓ FP base 64.57 42.85 65.35 75.05 44.91 64.31 50.00 61.25 58.54 9.75 W2A8 RTN (PTQ) 26.91 25.16 51.47 50.29 36.43 25.72 23.05 50.08 36.14 1.2e6 GPTQ (PTQ) 26.68 25.39 62.08 48.39 37.11 25.56 29.69 50.39 38.16 1.3e4 QuaRot (PTQ) 28.71 24.38 62.44 50.15 36.08 26.74 28.91 49.92 38.42 3.2e3 ParetoQ 58.99 35.73 61.56 68.16 41.43 45.72 42.58 54.53 51.09 18.22 VQLLM-QAT-d4 61.36 37.42 61.20 70.10 42.69 49.91 43.75 55.39 52.73 16.99 CanonQ-W-d1 63.56 40.71 61.41 71.01 43.50 54.06 45.51 58.36 54.76 14.38 CanonQ-W-d2 64.15 39.95 59.92 71.07 43.04 55.67 43.16 53.59 53.82 13.49 CanonQ-W-d4 61.06 39.86 62.11 70.20 42.49 50.69 43.36 53.83 52.95 16.01 CanonQ-Omni-d1 65.84 40.20 62.35 72.14 43.85 55.99 47.85 56.56 55.60 13.25 CanonQ-Omni-d2 61.99 38.01 64.17 72.04 43.04 56.63 43.75 57.42 54.63 13.42 W1A8 RTN (PTQ) 28.24 25.47 39.39 51.46 36.04 24.70 23.63 50.55 34.94 1.1e6 GPTQ (PTQ) 27.77 23.05 62.38 50.98 36.47 25.09 29.69 49.30 38.09 1.0e5 QuaRot (PTQ) 28.01 25.62 62.41 48.88 37.30 24.89 28.32 51.41 38.36 1.3e4 ParetoQ 55.62 32.35 61.92 66.11 41.13 40.65 40.82 50.62 48.65 21.27 VQLLM-QAT-d4 51.90 31.50 62.35 63.95 40.27 36.76 35.94 51.25 46.74 25.98 VQLLM-QAT-d8 51.94 32.26 60.77 63.85 41.08 39.13 38.87 51.88 47.47 25.37 CanonQ-W-d1 55.83 33.78 60.16 65.14 41.48 41.07 36.72 52.73 48.37 23.54 CanonQ-W-d2 55.83 36.57 61.83 65.41 41.23 42.37 39.84 52.50 49.45 21.23 CanonQ-W-d4 54.73 32.77 61.68 66.22 40.47 41.30 41.99 50.86 48.75 22.10 CanonQ-W-d8 54.65 34.88 59.83 67.40 42.19 43.75 40.43 53.59 49.59 20.54 CanonQ-Omni-d1 58.15 36.99 60.25 68.10 42.54 47.21 42.19 54.53 51.25 17.76 CanonQ-Omni-d2 57.39 34.54 57.98 68.37 42.09 46.71 39.84 52.73 49.96 17.47 W2A4 RTN (PTQ) 26.60 26.02 54.24 50.34 36.82 25.30 23.83 51.09 36.78 1.1e6 GPTQ (PTQ) 27.54 24.06 51.20 52.15 37.06 24.79 25.98 52.11 36.86 6.9e4 QuaRot (PTQ) 26.45 25.31 60.61 51.27 37.35 25.81 27.34 50.00 38.02 4.6e3 ParetoQ 47.76 27.87 58.68 61.96 40.57 31.66 32.23 52.27 44.12 32.57 VQLLM-QAT-d4 45.99 27.62 60.13 61.31 41.03 30.71 31.05 49.77 43.45 42.65 CanonQ-W-d1 45.90 27.96 59.38 60.34 40.98 31.39 34.38 52.81 44.14 35.16 CanonQ-W-d2 47.76 27.79 56.64 61.75 39.67 31.56 33.01 51.09 43.66 35.20 CanonQ-W-d4 47.30 29.48 60.53 60.78 40.12 32.01 34.77 51.25 44.53 35.53 CanonQ-Omni-d1 63.89 39.10 60.19 71.39 43.40 55.28 46.09 58.36 54.71 13.84 CanonQ-Omni-d2 64.40 39.78 62.04 70.64 42.24 54.85 45.90 56.41 54.53 14.51 Joint compression at 1B. Table 11 shows that CanonQ-Omni-d1 retains similar quality at A8 and A4 with KV2: 15.70/53.45 and 15.42/53.56. The corresponding ParetoQ results fall from 47.58/45.13 to 90.57/40.47. At W2A4KV2, CanonQ-Omni-d1 improves all eight tasks over ParetoQ; scalar canonical tables suffice for this operating point. Table 11: LLaMA3-1B joint W/A/KV quantization. CanonQ-Omni uses canonical KV quantization; other QAT methods use uniform KV. Method ARC-e ARC-c BoolQ PIQA SIQA HellaS. OBQA Wino. Avg. ZS ↑ PPL ↓ FP16 Base 64.57 42.85 65.35 75.05 44.91 64.31 50.00 61.25 58.54 9.75 W2A8KV2 RTN (PTQ) 26.88 22.50 56.16 51.07 37.26 25.30 27.54 51.80 37.31 5.8e5 GPTQ (PTQ) 27.42 26.41 61.24 49.76 37.70 25.72 25.59 52.03 38.23 5.3e4 QuaRot (PTQ) 25.86 24.06 61.90 51.95 36.43 24.81 25.00 49.14 37.39 3.4e4 KIVI 27.73 23.36 37.59 52.34 38.67 26.75 23.83 48.52 34.85 1.1e5 ParetoQ 47.76 29.81 61.89 62.98 40.68 31.72 34.96 51.25 45.13 47.58 VQLLM-QAT-d4 46.33 29.81 61.98 61.75 39.67 34.56 34.96 53.75 45.35 34.35 CanonQ-W-d1 51.65 31.67 61.41 64.33 40.32 37.06 33.59 51.25 46.41 30.07 CanonQ-W-d2 53.34 33.61 57.19 65.79 40.07 38.06 39.65 52.81 47.56 30.19 CanonQ-W-d4 50.76 31.25 61.59 64.71 40.47 37.57 37.89 51.02 46.91 31.27 CanonQ-Omni-d1 60.22 38.85 60.53 70.26 43.90 53.56 45.51 54.77 53.45 15.70 CanonQ-Omni-d2 60.73 34.97 62.65 70.15 43.25 50.52 44.14 54.92 52.67 16.43 W2A4KV2 RTN (PTQ) 28.83 25.55 54.33 50.59 36.08 25.11 23.24 50.00 36.72 5.5e5 GPTQ (PTQ) 28.12 23.91 58.77 52.64 37.30 25.23 24.41 52.27 37.83 9.5e4 QuaRot (PTQ) 27.58 25.16 61.75 51.17 36.28 24.83 25.98 52.11 38.11 4.1e4 KIVI 26.48 25.16 37.80 52.69 37.40 25.80 25.00 50.94 35.16 3.2e5 ParetoQ 40.41 25.17 54.34 56.57 39.21 28.13 30.08 49.84 40.47 90.57 VQLLM-QAT-d4 41.72 25.93 61.95 58.41 39.62 28.84 33.40 48.67 42.32 76.51 CanonQ-W-d1 41.47 28.29 58.10 59.38 39.47 28.78 32.42 52.03 42.49 64.40 CanonQ-W-d2 42.57 29.90 56.10 60.78 39.92 29.22 36.33 48.83 42.95 64.48 CanonQ-W-d4 42.02 25.25 57.68 58.14 39.31 28.28 32.62 52.42 41.97 76.19 CanonQ-Omni-d1 61.32 40.79 62.14 70.20 42.99 52.23 43.95 54.84 53.56 15.42 CanonQ-Omni-d2 63.47 36.74 59.34 70.20 43.30 52.48 44.34 55.39 53.16 15.90 Weight-only and joint compression at 3B. Table 12 places all 3B precisions under one FP reference. The W2A16 difference is small: CanonQ-W-d2 reaches 9.19/63.02 versus 9.29/62.84 for ParetoQ. Under W2A4KV2, the separation grows to 11.16/60.04 versus 92.64/39.39. Dimension changes the PPL–accuracy tradeoff: dW=1d_W=1 yields lower PPL (10.93), while dW=2d_W=2 raises accuracy from 58.37 to 60.04. Neither dimension dominates both metrics. Table 12: LLaMA3-3B: weight-only, weight–activation, and joint W/A/KV quantization. The same FP reference applies to all three panels. Accuracy is in percent; lower PPL is better. Method ARC-e ARC-c BoolQ PIQA SIQA HellaS. OBQA Wino. Avg. ZS ↑ PPL ↓ FP base 72.99 49.84 75.12 78.12 48.29 74.05 53.52 68.91 65.11 7.73 W2A16 RTN (PTQ) 27.11 25.94 47.30 50.34 36.91 25.45 25.78 49.30 36.02 3.6e5 GPTQ (PTQ) 27.34 23.83 61.54 51.12 35.35 26.10 29.49 48.83 37.95 7.3e3 QuaRot (PTQ) 28.01 25.70 56.43 50.24 37.45 25.61 26.17 50.08 37.46 3.3e2 ParetoQ 73.97 50.00 68.42 75.83 46.78 68.30 55.47 63.91 62.84 9.29 CanonQ-W-d1 73.31 50.08 69.02 75.99 46.63 68.54 51.76 64.22 62.44 9.37 CanonQ-W-d2 73.56 49.38 72.48 75.78 45.95 68.42 53.32 65.23 63.02 9.19 W2A4 RTN (PTQ) 27.23 24.84 46.30 50.54 37.60 25.34 24.80 49.30 35.74 4.2e5 GPTQ (PTQ) 28.05 25.08 61.57 51.51 37.84 24.99 27.54 48.36 38.12 7.9e4 QuaRot (PTQ) 27.93 25.47 55.05 51.71 37.84 26.29 24.80 48.75 37.23 6.7e2 ParetoQ 62.80 40.03 55.98 68.10 44.46 53.89 45.90 54.06 53.15 17.88 VQLLM-QAT-d4 49.32 30.24 61.86 63.09 40.12 34.92 37.70 53.12 46.30 30.71 CanonQ-W-d1 62.50 36.64 60.82 69.38 42.09 50.72 38.67 53.52 51.79 16.52 CanonQ-W-d2 64.10 37.58 59.53 68.80 42.44 52.99 45.31 53.98 53.09 18.12 CanonQ-Omni-d1 72.17 46.88 69.17 75.70 44.56 66.26 52.73 61.80 61.16 10.66 CanonQ-Omni-d2 72.26 47.47 67.84 76.08 46.22 66.14 50.78 62.34 61.14 10.50 W2A4KV2 RTN (PTQ) 27.42 25.62 50.84 50.44 37.35 25.15 26.37 51.02 36.78 1.1e6 GPTQ (PTQ) 27.62 26.33 59.83 51.07 36.77 25.45 27.54 48.44 37.88 8.9e4 QuaRot (PTQ) 27.81 24.84 61.75 50.34 36.04 25.59 27.73 49.53 37.95 1.5e4 ParetoQ 37.29 23.82 49.91 55.23 39.26 26.18 32.23 51.17 39.39 92.64 VQLLM-QAT-d4 45.14 27.20 44.54 60.51 41.08 30.78 34.77 52.58 42.07 52.82 CanonQ-W-d1 39.02 26.52 44.39 55.50 39.21 26.94 28.52 52.11 39.03 76.28 CanonQ-W-d2 43.75 30.07 55.31 59.05 39.11 34.06 33.98 52.97 43.54 30.24 CanonQ-Omni-d1 68.29 44.09 67.05 74.46 44.20 63.18 45.70 60.00 58.37 10.93 CanonQ-Omni-d2 70.62 45.62 69.20 74.61 45.46 64.38 50.78 59.61 60.04 11.16 Transfer across model family and scale. In Tab. 13, CanonQ-Omni-d1 improves every task over ParetoQ on both Qwen3-1.7B and LLaMA3-8B. The largest gains occur on HellaSwag: 17.93 and 42.68 points, respectively. Improvements also span the remaining reasoning and commonsense tasks, so the aggregate gain reflects broad recovery with task-dependent magnitudes. The different PPL and accuracy gaps to FP reinforce the value of evaluating likelihood and task performance together. Table 13: Cross-family and scale transfer: Qwen3-1.7B W2A4 and LLaMA3-8B W2A4KV2. Qwen3-1.7B, W2A4 Method ARC-e ARC-c BoolQ PIQA SIQA HellaS. OBQA Wino. Avg. ZS ↑ PPL ↓ FP base 68.79 41.48 79.12 71.39 44.73 59.51 38.67 61.09 58.10 16.33 ParetoQ 41.01 25.25 60.76 59.43 41.48 28.67 30.66 51.56 42.35 43.56 CanonQ-Omni-d1 58.11 34.12 62.38 67.13 43.04 46.60 43.95 52.34 50.96 17.80 LLaMA3-8B, W2A4KV2 FP base 80.95 57.88 83.34 81.14 49.34 79.50 55.56 73.43 70.14 6.14 ParetoQ 36.36 26.71 39.98 55.33 37.91 26.15 30.36 50.16 37.87 134.36 CanonQ-Omni-d1 70.86 42.04 71.30 73.80 45.26 68.83 43.65 62.74 59.81 9.41 Where the aggregate gain comes from. Figure 3 connects the precision sweep to its task-level pattern. As compression extends from W2A8 to W2A4KV2, CanonQ-Omni-d1 stays within 13.25–15.42 PPL, while the other QAT paths degrade. In W2A4, all eight tasks improve over ParetoQ; gains range from 1.51 points on BoolQ to 23.62 on HellaSwag. Thus the mean improvement is distributed across tasks, although its size depends strongly on the benchmark. Figure 3: Quality under joint compression on LLaMA3-1B. (a) CanonQ-Omni-d1 retains low PPL as activation and cache precision decrease; the vertical axis is logarithmic. (b) Its W2A4 improvement over ParetoQ spans all eight tasks, with uneven gain magnitudes. Points summarize separately trained configurations; task scores appear in Tab. 10. Appendix C Context-Length Robustness after 8K Continued QAT Protocol. All context-length results use LLaMA3-1B W2A4KV2, continued QAT with 8192-token sequences for 40K updates, and evaluation at 2048/4096/8192 tokens. The FP reference is the base checkpoint evaluated at each length. Table 14 reports PPL and the same-length ratio rM(L)=PPLM(L)/PPLFP(L)r_M(L)=PPL_M(L)/PPL_FP(L). Table 14: LLaMA3-1B W2A4KV2: all quantized methods receive 8K continued QAT for 40K updates. WikiText-2 PPL and its ratio to the same-length FP reference. PPL ↓ × FP ↓ Method 2K 4K 8K 2K 4K 8K FP reference 9.71 9.21 8.74 1.00 1.00 1.00 ParetoQ 50.42 46.67 45.63 5.19 5.07 5.22 CanonQ-W 43.16 40.24 39.50 4.44 4.37 4.52 CanonQ-Omni 16.86 15.55 14.86 1.74 1.69 1.70 Relative quality remains stable across the tested lengths. CanonQ-Omni improves from 16.86 to 15.55 to 14.86 PPL as the evaluation window grows. Its reductions relative to ParetoQ are 2.99×2.99×, 3.00×3.00×, and 3.07×3.07×; at 8K, it also improves over CanonQ-W by 2.66×2.66×. FP improves from 9.71 to 9.21 to 8.74, leaving CanonQ-Omni at 1.691.69–1.74×1.74× FP across lengths, versus 4.374.37–4.52×4.52× for CanonQ-W and 5.075.07–5.22×5.22× for ParetoQ. The ordering is consistent at all three lengths. Interpretation. All evaluation windows lie within the 8K continued-QAT training range. The experiment measures language-modeling quality across these windows. Quantized models receive continued QAT, while the FP checkpoint serves as a fixed same-length reference; the resulting ratios combine training and quantization effects. Longer windows provide additional preceding tokens and can change the scored-token set. We therefore assess context-length robustness through the consistent method ordering and approximately stable same-length FP ratio. Appendix D Theory Details and Proofs Throughout this appendix, Qd=(0,Id)Q_d=N(0,I_d) and all tuple laws have finite second moment. Wasserstein distance is induced by the Euclidean norm. For a finite codebook C⊂ℝdC ^d, write dC(v)=dist(v,C)d_C(v)=dist(v,C) and let ΠC(v) _C(v) denote nearest-codeword assignment with the deterministic tie rule used by the implementation. The main transfer result uses root mean-squared distortion because it is stable under W2W_2 without bounded support. D.1 The Haar Reference and the Implemented Hadamard Map Proposition 1 (Joint Haar reference law). Let R be Haar-random on (g) O(g), let u≠0u≠ 0 be fixed, and let 1≤d<g1≤ d<g. The first d coordinates of V=gRu/‖u‖2V= g\,Ru/\|u\|_2 have density pg,d(v)=Γ(g/2)πd/2gd/2Γ((g−d)/2)(1−‖v‖22g)(g−d−2)/2,‖v‖2<g,p_g,d(v)= (g/2)π^d/2g^d/2 ((g-d)/2) (1- \|v\|_2^2g )^(g-d-2)/2, \|v\|_2< g, (20) The law is independent of the direction of u, and for fixed d it converges to QdQ_d in W2W_2 as g→∞g→∞. Proof. Set q=u/‖u‖2q=u/\|u\|_2. Haar invariance makes RqRq uniform on g−1S^g-1. If G∼(0,Ig)G (0,I_g), then Rq=G/‖G‖2Rq d=G/\|G\|_2. For X=(Rq)1:dX=(Rq)_1:d, ‖X‖22=∑j≤dGj2∑j≤gGj2∼Beta(d2,g−d2).\|X\|_2^2= _j≤ dG_j^2 _j≤ gG_j^2 \! ( d2, g-d2 ). Rotational invariance in the first d coordinates and normalization in polar coordinates give fX(x)=Γ(g/2)πd/2Γ((g−d)/2)(1−‖x‖22)(g−d−2)/2,‖x‖2≤1.f_X(x)= (g/2)π^d/2 ((g-d)/2)(1-\|x\|_2^2)^(g-d-2)/2, \|x\|_2≤ 1. The change of variables v=gxv= g\,x yields Eq. 20. Finally, V=G1:d‖G‖22/g⇒(0,Id)V d= G_1:d \|G\|_2^2/g (0,I_d) by the law of large numbers and Slutsky’s theorem. Moreover, ‖V‖22=d=Qd‖V‖22E\|V\|_2^2=d=E_Q_d\|V\|_2^2 for every g. Weak convergence together with convergence of second moments is equivalent to W2W_2 convergence, which proves the stated metric claim. ∎ The preceding proposition motivates the reference law but does not describe the implemented transform. The next result applies directly to one random-sign Hadamard map and exposes the input-dependent quantities that must be checked. Proposition 2 (One-HDP coordinate and covariance diagnostics). Let H∈±g−1/2g×gH∈\± g^-1/2\^g× g be orthogonal, let D=diag(ϵ1,…,ϵg)D=diag( _1,…, _g) with independent Rademacher signs, and let P be an independent uniform permutation. For a fixed unit vector q independent of (D,)(D, P), define Z~=gHDq Z= g\,HDq. Every fixed Hadamard row satisfies DZ~i=0,DZ~i2=1,PrD(|Z~i|≥t)≤2e−t2/2,E_D Z_i=0, _D Z_i^2=1, _D(| Z_i|≥ t)≤ 2e^-t^2/2, (21) and, for a universal Berry–Esseen constant CBEC_BE, supx|PrD(Z~i≤x)−Φ(x)|≤CBE∑j=1g|qj|3≤CBE‖q‖∞. _x| _D( Z_i≤ x)- (x)|≤ C_BE _j=1^g|q_j|^3≤ C_BE\|q\|_∞. (22) If I=(i1,…,id)I=(i_1,…,i_d) is the ordered set of distinct Hadamard rows selected by the first d positions of P, the deployed first tuple is Z1:d:=Z~IZ_1:d:= Z_I, and ΣI(q):=D[Z~IZ~I⊤∣I,q] _I(q):=E_D[ Z_I Z_I I,q]. Then ‖ΣI(q)−Id‖F2=d(d−1)g‖q‖44−1g−1.E_ P\| _I(q)-I_d\|_F^2=d(d-1) g\|q\|_4^4-1g-1. (23) Proof. Writing the unnormalized Hadamard signs as sij∈±1s_ij∈\± 1\ gives Z~i=∑j=1gsijϵjqj Z_i= _j=1^gs_ij _jq_j. The mean and variance in Eq. 21 follow from independence and ∑jqj2=1 _jq_j^2=1. Moreover, DeλZ~i=∏jcosh(λqj)≤exp(λ22∑jqj2)=eλ2/2.E_De^λ Z_i= _j (λ q_j)≤ \! ( λ^22 _jq_j^2 )=e^λ^2/2. The standard Chernoff argument gives the stated two-sided tail. Applying the Berry–Esseen inequality to the independent summands sijϵjqjs_ij _jq_j proves the first inequality in Eq. 22; the second uses ∑j|qj|3≤‖q‖∞∑jqj2 _j|q_j|^3≤\|q\|_∞ _jq_j^2. For two distinct selected rows i≠ki≠ k, [ΣI(q)]ik=∑j=1gsijskjqj2,[ΣI(q)]ii=1.[ _I(q)]_ik= _j=1^gs_ijs_kjq_j^2, [ _I(q)]_i=1. Fix i and average the squared off-diagonal expression over k≠ik≠ i. Hadamard-column orthogonality gives ∑k=1g(∑jsijskjqj2)2 _k=1^g ( _js_ijs_kjq_j^2 )^2 =∑j,ℓsijsiℓqj2qℓ2∑kskjskℓ=g∑jqj4. = _j, s_ijs_i q_j^2q_ ^2 _ks_kjs_k =g _jq_j^4. The k=ik=i term equals (∑jqj2)2=1( _jq_j^2)^2=1, so a uniformly sampled distinct pair has expected squared covariance (g‖q‖44−1)/(g−1)(g\|q\|_4^4-1)/(g-1). There are d(d−1)d(d-1) ordered off-diagonal entries, which proves Eq. 23. ∎ Prop. 2 characterizes random-seed behavior for blocks independent of the sampled transform. For an adaptively trained checkpoint, Thm. 1 instead uses its actual post-HDP tuple law through Δd(P) _d(P). The distinction is concrete: after fixing R, choosing u=R⊤e1u=R e_1 gives Ru=e1Ru=e_1; sparse fixed blocks can also have non-Gaussian one-RHT laws. Related worst-case analyses establish sufficient multi-RHT constructions for scalar and vector guarantees (benbasat2026rht). CanonQ uses the random-seed bounds as structural diagnostics and the checkpoint-specific defect to quantify transfer of the frozen reference codebook. Lemma 1 (Normalization regularization defect). Sample a nonzero block U∈ℝgU ^g by the chosen group law, then sample one of its g/dg/d tuples uniformly. Let P0P_0 and PϵP_ε be the tuple laws under normalization by ‖U‖2\|U\|_2 and ‖U‖2+ϵ\|U\|_2+ε, respectively, using the same fixed orthogonal map. Then W2(Pϵ,P0)d≤[(ϵ‖U‖2+ϵ)2]1/2,Δd(Pϵ)≤Δd(P0)+[(ϵ‖U‖2+ϵ)2]1/2. W_2(P_ε,P_0) d≤ [E ( ε\|U\|_2+ε )^2 ]^1/2, _d(P_ε)≤ _d(P_0)+ [E ( ε\|U\|_2+ε )^2 ]^1/2. (24) Proof. Couple each regularized tuple to the corresponding unregularized tuple of the same block. Writing a=‖U‖2/(‖U‖2+ϵ)a=\|U\|_2/(\|U\|_2+ε) gives Zϵ=aZ0Z_ε=aZ_0. Orthogonality implies ‖Z0‖22=g\|Z_0\|_2^2=g for the full block, so conditional mean squared tuple error is d(1−a)2d(1-a)^2. This coupling bounds W22W_2^2; the triangle inequality gives the second claim. ∎ The bound isolates the regularizer from the HDP defect, but does not bound the latter. Zero blocks require the implementation’s explicit convention; the differentiable-norm results below apply to nonzero blocks. D.2 Root-Distortion Stability and Frozen-Table Transfer Lemma 2 (Root distortion is W2W_2-stable). For any finite nonempty codebook C and any tuple laws P,QP,Q with finite second moment, |ℛP,d(C)−ℛQ,d(C)|≤1dW2(P,Q).|R_P,d(C)-R_Q,d(C)|≤ 1 dW_2(P,Q). (25) Consequently, |ℛP,b,d⋆−ℛQ,b,d⋆|≤W2(P,Q)/d|R_P,b,d -R_Q,b,d |≤ W_2(P,Q)/ d. Proof. For any coupling (X,Y)(X,Y) of (P,Q)(P,Q), the distance to a fixed set is one-Lipschitz. The reverse triangle inequality in L2L_2 therefore gives d|ℛP,d(C)−ℛQ,d(C)| d\,|R_P,d(C)-R_Q,d(C)| =|‖dC(X)‖L2−‖dC(Y)‖L2| = |\|d_C(X)\|_L_2-\|d_C(Y)\|_L_2 | ≤‖dC(X)−dC(Y)‖L2≤‖X−Y‖L2. ≤\|d_C(X)-d_C(Y)\|_L_2≤\|X-Y\|_L_2. Taking the infimum over couplings proves Eq. 25. Applying this inequality to every feasible C, taking infima, and exchanging P,QP,Q proves the optimal-distortion statement. ∎ Proof of Thm. 1. The product alphabet AbdA_b^d has (2b)d=2bd(2^b)^d=2^bd entries, so it is feasible in the definition of ℛQd,b,d⋆R_Q_d,b,d . This proves the first part of Eq. 7. From Eq. 6 and a+e≤a+e a+e≤ a+ e, ℛQd,d(C^b,d)≤ℛQd,b,d⋆+εvq,R_Q_d,d( C_b,d) _Q_d,b,d + _vq, which proves its second part. Let Δ=Δd(P) = _d(P). By Lem. 2, ℛP,d(C^b,d)≤ℛQd,d(C^b,d)+Δ,ℛP,b,d⋆≥ℛQd,b,d⋆−Δ.R_P,d( C_b,d) _Q_d,d( C_b,d)+ , _P,b,d _Q_d,b,d - . Subtracting and using the fitting gap yields Eq. 8. Applying the same two inequalities to C^b,d C_b,d and AbdA_b^d gives ℛP,d(C^b,d)−ℛP,d(Abd) _P,d( C_b,d)-R_P,d(A_b^d) ≤ℛQd,d(C^b,d)−ℛQd,d(Abd)+2Δ _Q_d,d( C_b,d)-R_Q_d,d(A_b^d)+2 =2Δ−mQ(Abd,C^b,d), =2 -m_Q(A_b^d, C_b,d), which is Eq. 9. ∎ Root distortion gives a transfer bound under finite second moments. Squared distance is not globally Lipschitz on the Gaussian reference’s unbounded support, so a bounded-support W1W_1 argument would not cover this codec. Lemma 3 (Correction for assign-then-gain decoding). Define ℛP,d(γ)(C):=(1d‖V−γΠC(V)‖22)1/2,BC:=maxc∈C‖c‖2.R_P,d^(γ)(C):= ( 1dE\|V-γ _C(V)\|_2^2 )^1/2, B_C:= _c∈ C\|c\|_2. For the implementation that assigns with C and scales only after assignment, |ℛP,d(γ)(C)−ℛP,d(1)(C)|≤|γ−1|BCd.|R_P,d^(γ)(C)-R_P,d^(1)(C)|≤ |γ-1|B_C d. (26) Proof. Apply the reverse triangle inequality in L2(P)L_2(P) to V−γΠC(V)V-γ _C(V) and V−ΠC(V)V- _C(V). Their difference is (1−γ)ΠC(V)(1-γ) _C(V), whose norm is at most |1−γ|BC|1-γ|B_C. Division by d d gives Eq. 26. ∎ Notice that γΠC(V)γ _C(V) is generally not the Euclidean projection onto γCγ C: the assignment boundaries remain those of C. Thus same-rate dominance cannot simply be reused at non-unit gain. An exact projection onto the scaled table would instead assign by γΠC(V/γ)γ _C(V/γ) for γ>0γ>0. D.3 Gain-Aware Task-Loss Bounds For a hard-quantized network, downstream code-cell switches can make the loss discontinuous as the intervention moves. Theorem 2 is valid on a smooth continuation or a neighborhood in which downstream assignments remain fixed and other operations are C3C^3. Its smoothness hypothesis must be checked; freezing the codebook does not imply it. For activation/KV interventions, stationarity in model parameters does not imply a small gradient in the intervention coordinates. Proof of Thm. 2. Abbreviate e=eC,γ(u)e=e_C,γ(u) and η=ηC,γ(z)η= _C,γ(z). Taylor’s theorem gives ℒ(u+e)−ℒ(u)=∇ℒ(u)⊤e+12e⊤∇2ℒ(u)e+r3,|r3|≤M36∥e∥23.L(u+e)-L(u)= (u) e+ 12e ∇^2L(u)e+r_3, |r_3|≤ M_36\|e\|_2^3. Orthogonality and Eq. 10 imply ‖e‖2=ρ(u)‖η‖2/g\|e\|_2=ρ(u)\|η\|_2/ g. Cauchy–Schwarz and the operator-norm Hessian bound then give exactly Eq. 11. ∎ For a random block U, the magnitude of the quadratic term obeys |12eC,γ(U)⊤∇2ℒ(U)eC,γ(U)|≤12DS,γcurv(C).E | 12e_C,γ(U) ∇^2L(U)e_C,γ(U) |≤ 12D^curv_S,γ(C). This follows by applying the pointwise operator-norm inequality before taking expectations. It requires neither residual isotropy nor an almost-sure bound on ρ(U)2ΛUρ(U)^2 _U. Corollary 1 (Simultaneous quantization at a common graph cut). Let e=(e1,…,em)e=(e_1,…,e_m) concatenate intervention coordinates exposed at a common cut of the computation graph, and regard ℒL as a function of their concatenation. If the full Hessian and third derivative satisfy the same operator-norm bounds as in Thm. 2, then |ℒ(x+e)−ℒ(x)|≤‖∇ℒ(x)‖2‖e‖2+Λx2‖e‖22+M36‖e‖23.|L(x+e)-L(x)|≤\| (x)\|_2\|e\|_2+ _x2\|e\|_2^2+ M_36\|e\|_2^3. (27) Proof. Apply the same Taylor argument to the concatenated variable. The quadratic form e⊤∇2ℒ(x)e ∇^2L(x)e contains every cross-site block ei⊤Hijeje_i H_ije_j; Eq. 27 bounds the full form and does not discard these interactions. For causally dependent internal sites, apply this result sequentially and telescope the loss differences rather than treating the sites as coordinates of one cut. ∎ Lemma 4 (Coupled errors in linear and attention blocks). For a linear map with W^=W+EW W=W+E_W and a^=a+eA a=a+e_A, ‖W^a^−Wa‖2≤‖EW‖op‖a‖2+‖W‖op‖eA‖2+‖EW‖op‖eA‖2.\| W a-Wa\|_2≤\|E_W\|_op\|a\|_2+\|W\|_op\|e_A\|_2+\|E_W\|_op\|e_A\|_2. For attention with head dimension h, define S=QK⊤/hS=QK / h, A=softmaxrow(S)A=softmax_row(S), and Y=AVY=AV. Let Q^=Q+EQ Q=Q+E_Q, K^=K+EK K=K+E_K, V^=V+EV V=V+E_V, with corresponding S S, A A and Y Y. On the same attention mask, ‖S^−S‖F \| S-S\|_F ≤1h(‖EQ‖F‖K‖op+‖Q‖op‖EK‖F+‖EQ‖F‖EK‖op), ≤ 1 h (\|E_Q\|_F\|K\|_op+\|Q\|_op\|E_K\|_F+\|E_Q\|_F\|E_K\|_op ), ‖Y^−Y‖F \| Y-Y\|_F ≤12‖S^−S‖F‖V‖op+‖A^‖op‖EV‖F. ≤ 12\| S-S\|_F\|V\|_op+\| A\|_op\|E_V\|_F. Masked logits are compared only on their common active entries. These are deterministic local bounds; no independence or isotropy of the errors is assumed. Proof. Expanding (W+EW)(a+eA)−Wa(W+E_W)(a+e_A)-Wa and applying the triangle inequality gives the first result. Similarly, S^−S=(EQK⊤+QEK⊤+EQEK⊤)/h S-S=(E_QK +QE_K +E_QE_K )/ h; the Frobenius/operator-norm product inequalities give the logit bound. For a probability vector p, the softmax Jacobian is J=diag(p)−pp⊤J=diag(p)-p . Its absolute row sum is 2pi(1−pi)≤1/22p_i(1-p_i)≤ 1/2; symmetry therefore implies ‖J‖op≤1/2\|J\|_op≤ 1/2. Integrating along the line segment between each pair of unmasked logit rows gives ‖A^−A‖F≤12‖S^−S‖F\| A-A\|_F≤ 12\| S-S\|_F. Finally, Y^−Y=(A^−A)V+A^EV Y-Y=( A-A)V+ AE_V, which proves the output bound. A fixed causal mask is handled by applying the same argument on each row’s active coordinates. ∎ D.4 The STE Jacobian Induced by the Canonical Codec The following exact calculation is the method-specific part of the optimization analysis. It assumes the normalization scale is differentiated, as in the stated fake-quantization graph, and that the nonnegative gain is an independent parameter when differentiating with respect to u. Proposition 3 (Exact normalized-codebook STE Jacobian). Let u≠0u≠ 0, set s(u)=ρ(u)/g=(‖u‖2+ϵ)/gs(u)=ρ(u)/ g=(\|u\|_2+ε)/ g, z=Ru/s(u)z=Ru/s(u), q=ΠC(d)(z)q= _C^(d)(z), and u^=γs(u)R⊤q u=γ s(u)R q for γ≥0γ≥ 0. Under the identity STE Dq/Dz=IDq/Dz=I, the surrogate Jacobian is JSTE(u)=γ[I+R⊤(q−z)∇s(u)⊤].J_STE(u)=γ [I+R (q-z)∇ s(u) ]. (28) With the assignment defect δa(u)=‖q−z‖2/g _a(u)=\|q-z\|_2/ g and reconstruction defect δr(u)=‖γq−z‖2/g _r(u)=\|γ q-z\|_2/ g, ‖JSTE−γI‖op=γδa,‖JSTE‖op≤γ(1+δa),‖u^−u‖2=ρ(u)δr.\|J_STE-γ I\|_op=γ _a, \|J_STE\|_op≤γ(1+ _a), \| u-u\|_2=ρ(u) _r. (29) The deviation from the gain-preserving reference output γuγ u is ‖u^−γu‖2=γρ(u)δa\| u-γ u\|_2=γρ(u) _a. If γ is trained, the gain-column defect is ‖∂γu^−∂γ(γu)‖2=‖sR⊤(q−z)‖2=ρ(u)δa. \| _γ u- _γ(γ u) \|_2=\|sR (q-z)\|_2=ρ(u) _a. (30) Proof. Differentiate z=Ru/sz=Ru/s using the identity STE: dz=Rdu/s−zds/sdz=R\,du/s-z\,ds/s. Since sR⊤z=usR z=u, du d u =γ(R⊤qds+sR⊤dz) =γ (R q\,ds+sR dz ) =γ(du+R⊤(q−z)ds), =γ (du+R (q-z)\,ds ), which proves Eq. 28. For u≠0u≠ 0, ‖∇s(u)‖2=1/g\|∇ s(u)\|_2=1/ g. The matrix A(u)=R⊤(q−z)∇s(u)⊤A(u)=R (q-z)∇ s(u) has rank at most one and satisfies ‖A(u)‖op=‖q−z‖2/g=δa\|A(u)\|_op=\|q-z\|_2/ g= _a. Hence JSTE−γI=γA(u)J_STE-γ I=γ A(u) has operator norm γδaγ _a, and the Jacobian bounds follow. Finally, u^−u=sR⊤(γq−z),u^−γu=γsR⊤(q−z), u-u=sR (γ q-z), u-γ u=γ sR (q-z), and orthogonality gives the two forward-error identities. Finally, ∂γu^=sR⊤q _γ u=sR q and ∂γ(γu)=u=sR⊤z _γ(γ u)=u=sR z, which gives Eq. 30. ∎ The two defects differ because the current implementation assigns in the unscaled table C and applies γ afterward. Assignment distortion controls the STE Jacobian relative to its gain-scaled identity, whereas reconstruction distortion controls the actual error relative to u. If z is partitioned into g/dg/d tuples VjV_j and P is obtained by sampling one of them uniformly, then δa(u)2=1g∑j=1g/d‖ΠC(Vj)−Vj‖22=DP,d(C). _a(u)^2= 1g _j=1^g/d\| _C(V_j)-V_j\|_2^2=D_P,d(C). (31) Thus Thm. 1 directly bounds the RMS assignment defect that appears in the STE Jacobian, subject only to the local sensitivity factors in the next corollary. Corollary 2 (Codec-controlled STE bias and alignment). Let f be a downstream loss whose gradient is LfL_f-Lipschitz. Define gSTE=JSTE⊤∇f(u^)g_STE=J_STE ∇ f( u). Relative to the gain-preserving smooth continuation u↦f(γu)u f(γ u), ∥gSTE−γ∇f(γu)∥2 \|g_STE-γ∇ f(γ u)\|_2 ≤γδa‖∇f(γu)‖2+γ2(1+δa)Lfρ(u)δa=:BC(u). ≤γ _a\|∇ f(γ u)\|_2+γ^2(1+ _a)L_fρ(u) _a=:B_C(u). (32) Consequently, with gref=γ∇f(γu)g_ref=γ∇ f(γ u), ⟨gSTE,gref⟩≥‖gref‖2(‖gref‖2−BC(u)). g_STE,g_ref ≥\|g_ref\|_2 (\|g_ref\|_2-B_C(u) ). (33) If gref≠0g_ref≠ 0 and BC(u)≤κ‖gref‖2B_C(u)≤κ\|g_ref\|_2 for some κ<1κ<1, then cos∠(gSTE,gref)≥1−κ1+κ. (g_STE,g_ref)≥ 1-κ1+κ. (34) Proof. Add and subtract JSTE⊤∇f(γu)J_STE ∇ f(γ u): gSTE−γ∇f(γu) g_STE-γ∇ f(γ u) =(JSTE⊤−γI)∇f(γu)+JSTE⊤[∇f(u^)−∇f(γu)]. =(J_STE -γ I)∇ f(γ u)+J_STE [∇ f( u)-∇ f(γ u)]. Use Eq. 29, gradient Lipschitzness, and ‖u^−γu‖2=γρ(u)δa\| u-γ u\|_2=γρ(u) _a to obtain Eq. 32. Cauchy–Schwarz yields Eq. 33. Under the relative-error condition, ‖gSTE‖2≤(1+κ)‖gref‖2\|g_STE\|_2≤(1+κ)\|g_ref\|_2 and the inner product is at least (1−κ)‖gref‖22(1-κ)\|g_ref\|_2^2, which proves Eq. 34. ∎ Corollary 3 (RMS distortion controls the blockwise STE certificate). Suppose on the analyzed region that 0≤γ≤γ¯0≤γ≤ γ, ρ(u)≤ρ¯ρ(u)≤ ρ, ‖∇f(γu)‖2≤G\|∇ f(γ u)\|_2≤ G, and δa(u)≤δ¯ _a(u)≤ δ. With K:=γ¯G+γ¯2(1+δ¯)Lfρ¯K:= γG+ γ^2(1+ δ)L_f ρ, one has BC(u)≤Kδa(u)B_C(u)≤ K _a(u). If P is the uniformly sampled tuple law of the group, then BC(u)2≤K2DP,d(C).B_C(u)^2≤ K^2D_P,d(C). (35) For a random group U, taking expectation gives BC(U)2≤K2DPU,d(C)EB_C(U)^2≤ K^2ED_P_U,d(C); the tuple law must use the same group-sampling distribution. Proof. Apply the regional bounds to Eq. 32, then use Eq. 31. ∎ From source mismatch to STE bias. For the numerical reference table C=C^b,dC= C_b,d, Lems. 2 and 6 give ℛP,d(C)≤ℛQd,d(C)+Δd(P)≤ℛQd,b,d⋆+εvq+Δd(P).R_P,d(C) _Q_d,d(C)+ _d(P) _Q_d,b,d+ _vq+ _d(P). Under the regional bounds of Cor. 3 and matching group/tuple sampling, it follows that [BC(U)2]≤K2(ℛQd,b,d⋆+εvq+Δd(P))2.E\! [B_C(U)^2 ]≤ K^2\! (R _Q_d,b,d+ _vq+ _d(P) )^2. (36) This makes the roles of reference distortion, numerical fitting, source mismatch, and downstream sensitivity explicit in one blockwise bias certificate. Comparing with the gain-scaled downstream gradient evaluated at the unquantized input u, the same decomposition and Eq. 29 give ∥gSTE−γ∇f(u)∥2≤γδa∥∇f(u)∥2+γ(1+δa)Lfρ(u)δr.\|g_STE-γ∇ f(u)\|_2≤γ _a\|∇ f(u)\|_2+γ(1+ _a)L_fρ(u) _r. (37) Assignment error δa _a controls the Jacobian defect, while reconstruction error δr _r controls forward error relative to u. Optimizing only the latter can leave the former large when gain is applied after assignment. Proposition 4 (Cellwise shadow objective). Let f:ℝd→ℝf:R^d have LfL_f-Lipschitz gradient and let VkV_k be the open Voronoi cell of codeword ckc_k. Define Cf(z):=f(ck)+⟨∇f(ck),z−ck⟩,z∈Vk.S_Cf(z):=f(c_k)+ ∇ f(c_k),z-c_k , z∈ V_k. Inside each cell, ∇Cf(z)=∇f(ΠC(z)) _Cf(z)=∇ f( _C(z)), exactly the identity-STE direction for a fixed-scale quantizer, and |Cf(z)−f(z)|≤Lf2‖z−ΠC(z)‖22.|S_Cf(z)-f(z)|≤ L_f2\|z- _C(z)\|_2^2. (38) Proof. The gradient statement follows because CfS_Cf is affine on VkV_k. The smoothness remainder for the first-order expansion of f at ckc_k gives Eq. 38. ∎ The cellwise construction relates local STE modeling error to codebook distortion. It can be discontinuous across cell boundaries, so its guarantee is local to each cell. For a full network, let ℬξ(ω,e)B_ξ(ω,e) denote the backward map when the concatenated perturbation vector e includes forward residuals, local STE-Jacobian deviations, and trainable-quantizer Jacobian-column errors, with ℬξ(ω,0)=∇fξ(ω)B_ξ(ω,0)=∇ f_ξ(ω). The following elementary lemma explains how the sitewise certificates aggregate. Lemma 5 (From aggregate codec errors to global STE bias). Assume ξ∇fξ(ω)=∇F(ω)E_ξ∇ f_ξ(ω)=∇ F(ω), ξ‖e(ξ)‖22<∞E_ξ\|e(ξ)\|_2^2<∞, and the required gradients are integrable. If, on the analyzed region, ‖ℬξ(ω,e)−ℬξ(ω,0)‖2≤κ‖e‖2,\|B_ξ(ω,e)-B_ξ(ω,0)\|_2≤κ\|e\|_2, (39) then for G(ω,ξ)=ℬξ(ω,e(ξ))G(ω;ξ)=B_ξ(ω,e(ξ)) and F(ω)=ξfξ(ω)F(ω)=E_ξf_ξ(ω), ‖ξG(ω,ξ)−∇F(ω)‖22≤κ2ξ‖e(ξ)‖22.\|E_ξG(ω;ξ)-∇ F(ω)\|_2^2≤κ^2E_ξ\|e(ξ)\|_2^2. (40) The exact blockwise bound in Eq. 32 supplies a sharper local value when the normalization graph is modeled explicitly. Proof. Jensen’s inequality followed by Eq. 39 gives ‖ξ[G−∇fξ]‖22≤ξ‖G−∇fξ‖22≤κ2ξ‖e(ξ)‖22. \|E_ξ[G-∇ f_ξ]\|_2^2 _ξ\|G-∇ f_ξ\|_2^2≤κ^2E_ξ\|e(ξ)\|_2^2. ∎ D.5 Distortion-Controlled Optimization Consequences For the SGD update in Thm. 3, the following analysis separates optimization and sampling error from the codec-dependent bias certificate. The result concerns the smooth reference objective under the stated update rule; the experimental optimizer is AdamW. Proof of Thm. 3. Let gt=∇F(ωt)g_t=∇ F( _t), G¯t=tGt G_t=E_tG_t, and bt=G¯t−gtb_t= G_t-g_t. L-smoothness gives tF(ωt+1)≤F(ωt)−αt⟨gt,gt+bt⟩+Lαt22t‖Gt‖22.E_tF( _t+1)≤ F( _t)- _t g_t,g_t+b_t + L _t^22E_t\|G_t\|_2^2. (41) The variance condition implies t‖Gt‖22≤2‖gt‖22+2‖bt‖22+σ2.E_t\|G_t\|_2^2≤ 2\|g_t\|_2^2+2\|b_t\|_2^2+σ^2. Also, −⟨gt,bt⟩≤14‖gt‖22+‖bt‖22- g_t,b_t ≤ 14\|g_t\|_2^2+\|b_t\|_2^2. Substituting both inequalities into Eq. 41 and using Lαt≤1/4L _t≤ 1/4 yields tF(ωt+1)≤F(ωt)−αt2‖gt‖22+5αt4‖bt‖22+Lαt2σ22.E_tF( _t+1)≤ F( _t)- _t2\|g_t\|_2^2+ 5 _t4\|b_t\|_2^2+ L _t^2σ^22. (42) Take total expectation, sum from t=0t=0 to T−1T-1, use F(ωT)≥F⋆F( _T)≥ F_ , and substitute the codec-bias bound. Multiplying the resulting inequality by two and dividing by STS_T gives Eq. 15. ∎ For the constant schedule αt=c/T≤1/(4L) _t=c/ T≤ 1/(4L), ST=cTS_T=c T and ∑t<Tαt2=c2 _t<T _t^2=c^2. The optimization and sampling terms in Eq. 15 therefore decay as T−1/2T^-1/2, leaving the weighted codec-bias certificate. If that certificate also vanishes, the bound establishes stationarity of the smooth reference; otherwise it gives a residual upper bound determined by the codec and sensitivity conditions. Corollary 4 (Local PL bound with codec-dependent residual). Suppose, in addition, that the iterates remain in a region satisfying the PL inequality 12‖∇F(ω)‖22≥μ(F(ω)−F⋆), 12\|∇ F(ω)\|_2^2≥μ(F(ω)-F_ ), (43) and that ‖bt‖22≤B¯2\|b_t\|_2^2≤ B^2. For a constant 0<α≤min1/(4L),1/μ0<α≤ \1/(4L),1/μ\, [F(ωT)−F⋆]≤(1−μα)T(F(ω0)−F⋆)+5B¯24μ+Lασ22μ.E[F( _T)-F_ ]≤(1-μα)^T(F( _0)-F_ )+ 5 B^24μ+ Lασ^22μ. (44) Proof. Apply Eq. 43 to Eq. 42 and take total expectations: Δt+1≤(1−μα)Δt+5α4B¯2+Lα2σ22,Δt:=[F(ωt)−F⋆]. _t+1≤(1-μα) _t+ 5α4 B^2+ Lα^2σ^22, _t:=E[F( _t)-F_ ]. Unrolling the geometric recurrence and upper-bounding the finite geometric sum by (μα)−1(μα)^-1 proves Eq. 44. ∎ D.6 Voronoi Margins, Code Switching, and Index Stability Lemma 6 (Assignment-switch margin). For a point v with unique nearest codeword ca(v)c_a(v), define mC(v):=minj≠a(v)‖v−cj‖22−‖v−ca(v)‖222‖cj−ca(v)‖2.m_C(v):= _j≠ a(v) \|v-c_j\|_2^2-\|v-c_a(v)\|_2^22\|c_j-c_a(v)\|_2. (45) Set mC(v)=0m_C(v)=0 when the nearest-codeword assignment is tied. Then mC(v)m_C(v) is the distance from v to the boundary of its Voronoi cell. If ‖v′−v‖2<mC(v)\|v -v\|_2<m_C(v), then a(v′)=a(v)a(v )=a(v). Consequently, if ‖V′−V‖2≤r\|V -V\|_2≤ r almost surely, Pr(a(V′)≠a(V))≤Pr(mC(V)≤r). (a(V )≠ a(V))≤ (m_C(V)≤ r). (46) Proof. For j≠a(v)j≠ a(v), the bisector between ca(v)c_a(v) and cjc_j is a hyperplane. The signed Euclidean distance from v to that hyperplane is precisely the corresponding ratio in Eq. 45. The minimum is therefore the distance to the boundary of the current cell. The open ball of that radius lies inside the cell, proving both claims. ∎ For the idealized ϵ=0ε=0 normalization z(u)=gRu/‖u‖2z(u)= gRu/\|u\|_2, if ‖h‖2≤‖u‖2/2\|h\|_2≤\|u\|_2/2, then ‖z(u+h)−z(u)‖2≤2g‖h‖2‖u‖2.\|z(u+h)-z(u)\|_2≤ 2 g\|h\|_2\|u\|_2. Let zj(u)∈ℝdz_j(u) ^d denote tuple j. A sufficient condition that one shadow update preserve every tuple index is 2g‖h‖2‖u‖2<minjmC(zj(u)). 2 g\|h\|_2\|u\|_2< _jm_C(z_j(u)). This inequality certifies unchanged assignments when it holds. Violating it permits, but does not imply, switching or chattering. Corollary 5 (Stability of the code-index distribution). Let aCa_C be the nearest-codeword index map, let P,QP,Q be tuple laws, and let V∼QV Q. Then TV((aC)#P,(aC)#Q)≤infr>0W2(P,Q)2r2+Q(mC(V)≤r).TV((a_C)_\#P,(a_C)_\#Q)≤ _r>0 \ W_2(P,Q)^2r^2+Q(m_C(V)≤ r) \. (47) Proof. Take a coupling (X,Y)(X,Y) of (P,Q)(P,Q). If ‖X−Y‖2≤r\|X-Y\|_2≤ r and mC(Y)>rm_C(Y)>r, Lem. 6 gives aC(X)=aC(Y)a_C(X)=a_C(Y). Hence Pr(aC(X)≠aC(Y))≤Pr(‖X−Y‖2>r)+Q(mC(Y)≤r). (a_C(X)≠ a_C(Y))≤ (\|X-Y\|_2>r)+Q(m_C(Y)≤ r). Markov’s inequality bounds the first term by ‖X−Y‖22/r2E\|X-Y\|_2^2/r^2. Total variation between pushforwards is at most the mismatch probability under any coupling. Infimizing first over couplings and then over r proves Eq. 47. ∎ Code-index stability is governed jointly by transport distance and the probability mass near Voronoi boundaries. The bound separates these contributions: W2W_2 controls displacement between tuple laws, while the boundary term controls the sensitivity of discrete assignments to that displacement.