Paper deep dive
Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition
Calvin Ang, Sungyoon Kim, Mert Pilanci
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/23/2026, 12:06:27 PM
Summary
The paper presents a theoretical framework for optimal scalar quantization in matrix multiplication, deriving a high-rate asymptotic expansion for mean-squared error (MSE). It identifies optimal companding point densities based on conditional second moments and proves a correlation-driven phase transition in the optimal density for Gaussian multiplicative pairs. The method is validated through synthetic experiments and applications in quantized least squares and large language model activation quantization.
Entities (6)
Relation Signals (2)
Calvin Ang → authored → Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition
confidence 100% · Paper title and author list
Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition → evaluatedon → GPT-2
confidence 95% · Provide experimental validation of our results... on key and query activations on the GPT-2 and Qwen3 family of models.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We study entrywise scalar quantization of two matrices prior to multiplication. Given $A\in R^{m\times k}$ and $B\in R^{k\times n}$, we quantize entries of $A$ and $B$ independently using scalar quantizers with $K_X$ and $K_Y$ levels per entry, and form $\widehat C=\widehat A\,\widehat B$. The objective is to minimize the matrix multiplication mean-squared error (MSE) $E[\|{AB-\widehat A\widehat B}\|_F^2]$ under a pair-i.i.d.\ inner-product model. In the high-resolution regime $K_X,K_Y\to\infty$, we derive a sharp $K^{-2}$ asymptotic expansion for $\mathcal{E}$, identify the exact optimal leading constants, and characterize asymptotically optimal quantization center densities in terms of conditional second moments. We then specialize to correlated Gaussian multiplicative pairs, obtaining a closed-form optimal point density \[ \lambda^\star(u)\ \propto\ \exp\!\left(-\frac{u^2}{6}\right)\bigl((1-\rho^2)+\rho^2u^2\bigr)^{1/3}, \qquad u=\frac{x}{\sigma_X}, \] with the same form for $y/\sigma_Y$, and prove a correlation-driven phase transition: the density is unimodal at the origin for $|\rho|\leq 1/\sqrt{3}$ and becomes bimodal for $|\rho|>1/\sqrt{3}$ with peaks at $u_{\mathrm{peak}}=\pm\sqrt{3-1/\rho^2}$. We show our method's applicability in synthetic experiments such as matrix multiplication quantization and least squares optimization, as well as quantization of large language model key and query activations.
Tags
Links
- Source: https://arxiv.org/abs/2603.19559v1
- Canonical: https://arxiv.org/abs/2603.19559v1
Trouble viewing inline? Open PDF directly →
Full Text
71,580 characters extracted from source content.
Expand or collapse full text
Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition Calvin Ang Stanford University &Sungyoon Kim11footnotemark: 1 Stanford University &Mert Pilanci Stanford University Equal contribution. Abstract We study entrywise scalar quantization of two matrices prior to multiplication. Given A∈ℝm×kA ^m× k and B∈ℝk×nB ^k× n, we quantize entries of A and B independently using scalar quantizers with KXK_X and KYK_Y levels per entry, and form C^=A^B C= A\, B. The objective is to minimize the matrix multiplication mean-squared error (MSE) ℰ≜[‖AB−A^B^‖F2]E [\|AB- A B\|_F^2] under a pair-i.i.d. inner-product model. In the high-resolution regime KX,KY→∞K_X,K_Y→∞, we derive a sharp K−2K^-2 asymptotic expansion for ℰE, identify the exact optimal leading constants, and characterize asymptotically optimal quantization center densities in terms of conditional second moments. We then specialize to correlated Gaussian multiplicative pairs, obtaining a closed-form optimal point density λ⋆(u)∝exp(−u26)((1−ρ2)+ρ2u2)1/3,u=xσX,λ (u)\ \ \! (- u^26 ) ((1-ρ^2)+ρ^2u^2 )^1/3, u= x _X, with the same form for y/σYy/ _Y, and prove a correlation-driven phase transition: the density is unimodal at the origin for |ρ|≤1/3 ρ ≤ 1/ 3 and becomes bimodal for |ρ|>1/3 ρ >1/ 3 with peaks at upeak=±3−1/ρ2u_peak=± 3-1/ρ^2. We show our method’s applicability in synthetic experiments such as matrix multiplication quantization and least squares optimization, as well as quantization of large language model key and query activations. 1 Introduction Quantized matrix multiplication is central in modern machine learning inference and scientific computing. In many deployments, one does not care about entrywise reconstruction error of operands A and B; rather, the relevant distortion is the error induced after multiplication Ordentlich and Polyanskiy (2025). This perspective is closely connected to classical high-rate theory Bennett (1948) and to the broader quantization literature Gersho and Gray (2012), which show that optimal quantizer design depends fundamentally on the downstream distortion criterion. At the same time, low-precision quantization has become a practical necessity in modern GPU inference: recent hardware and software stacks increasingly rely on ultra-low-precision formats such as NVFP4 NVIDIA Technical Blog (2025), INT8 Dettmers et al. (2022), and FP8 Micikevicius et al. (2022) to improve throughput, memory efficiency, and energy efficiency for large-scale matrix multiplication workloads arising in large language model inference . These trends make it especially important to understand quantization schemes that are optimal for the product itself, rather than for the separate reconstruction of the input matrices. However, matrix multiplication introduces a distinct bilinear distortion structure: the error contributed by one operand is filtered by the other, and the importance of each entry depends jointly on both factors. Related recent work has studied quantized matrix multiplication in different settings, including nested lattice quantization Ordentlich and Polyanskiy (2025). In contrast, our focus is on the optimal design of scalar quantizers specifically for matrix multiplication, deriving high-rate laws tailored to product distortion and showing how multiplicative structure and statistical dependence fundamentally change the optimal point density. This paper asks: Which scalar quantizers for entries of A and B minimize the expected Frobenius MSE of the product computed from quantized operands? Contributions. • We derive a high-rate characterization of the optimal achievable matrix multiplication MSE, proving a sharp K−2K^-2 scaling law with an exact leading constant. • We reduce the matrix objective to two weighted scalar MSE quantization problems driven by conditional second moments, and we identify the optimal companding point densities. • For correlated Gaussian multiplicative pairs, we obtain closed-form optimal densities and prove a correlation-induced unimodal-to-bimodal “phase transition”. • Provide experimental validation of our results on synthetically generated matrices for matrix multiplication and quantized least squares as well as on key and query activations on the GPT-2 and Qwen3 family of models. 2 High-Rate Analysis of Matrix Multiplication MSE This section derives the high-rate reduction from matrix multiplication MSE to two decoupled weighted scalar criteria, and states the resulting sharp K−2K^-2 constant. All asymptotic steps are justified in Appendix A and Appendix B. 2.1 Problem Formulation Let A∈ℝm×kA ^m× k, B∈ℝk×nB ^k× n, and C=ABC=AB. Let QXQ_X and QYQ_Y be scalar quantizers with |range(QX)|≤KX (Q_X) ≤ K_X and |range(QY)|≤KY (Q_Y) ≤ K_Y. Define A^iℓ=QX(Aiℓ),B^ℓj=QY(Bℓj),C^=A^B^. A_i =Q_X(A_i ), B_ j=Q_Y(B_ j), C= A\, B. We measure performance by the matrix multiplication MSE ℰ(QX,QY)≜[‖C−C^‖F2].E(Q_X,Q_Y) [ C- C _F^2 ]. The quantizer design is governed by the distribution of multiplicative entry-pairs feeding each inner product. We assume each pairs (X,Y)(X,Y) in the matrix multiplication has identical joint distributions. Assumption 1 (Pair-i.i.d. inner products). For each output entry Cij=∑ℓ=1kAiℓBℓjC_ij= _ =1^kA_i B_ j, the pairs (Aiℓ,Bℓj)ℓ=1k\(A_i ,B_ j)\_ =1^k are i.i.d. copies of a generic pair (X,Y)(X,Y) with finite moments up to order 4+ϵ4+ε for some ϵ>0ε>0. Because of this, the entries Ciji∈[m],j∈[n]\C_ij\_i∈[m],j∈[n] are identically distributed (although dependent). Under Assumption 1, when we denote (X,Y)∼(X,Y) we have ℰ(QX,QY)=nm(Xi,Yi)∼[(∑i=1kXiYi−∑i=1kX^iY^i)2].E(Q_X,Q_Y)=nm\ E_(X_i,Y_i) [ ( _i=1^kX_iY_i- _i=1^k X_i Y_i )^2 ]. We use the standard fixed-rate identification RX≜log2KX,RY≜log2KY,R_X _2K_X, R_Y _2K_Y, and analyze the high-resolution regime KX,KY→∞K_X,K_Y→∞ (or equivalently, RX,RY→∞R_X,R_Y→∞). 2.2 Sharp High-rate Constant and Optimal Point Densities Let D≜XY−X^Y^D XY- X Y for a single pair (X,Y)(X,Y). Independence across samples (Xi,Yi)(X_i,Y_i) yields (Xi,Yi)∼[(∑i=1kXiYi−∑i=1kX^iY^i)2]=k[D2]+k(k−1)[D]2.E_(X_i,Y_i) [( _i=1^kX_iY_i- _i=1^k X_i Y_i)^2 ]=k\,E[D^2]+k(k-1)E[D]^2. (1) Appendix A proves that, for the companding quantizers used in the high-rate design, [D]=O(KX−2+KY−2)E[D]=O(K_X^-2+K_Y^-2). Consequently the second term in (1) is o(KX−2+KY−2)o(K_X^-2+K_Y^-2) and does not affect the K−2K^-2 leading constant. Define quantization errors eX≜X−X^e_X X- X and eY≜Y−Y^e_Y Y- Y. Then D=XY−X^Y^=YeX+XeY−eXeY.D=XY- X Y=Ye_X+Xe_Y-e_Xe_Y. (2) Squaring produces D2 D^2 =Y2eX2+X2eY2+eX2eY2+2XYeXeY−2YeX2eY−2XeXeY2. =Y^2e_X^2+X^2e_Y^2+e_X^2e_Y^2+2XYe_Xe_Y-2Ye_X^2e_Y-2Xe_Xe_Y^2. (3) Appendix A shows that the mixed terms on the second line have expectation O(KX−2KY−2)O(K_X^-2K_Y^-2) for the companding quantizers used in the high-rate design. Since KX−2KY−2=o(KX−2+KY−2)K_X^-2K_Y^-2=o(K_X^-2+K_Y^-2) as KX,KY→∞K_X,K_Y→∞, we obtain the sharp leading reduction [D2]=[Y2eX2]+[X2eY2]+o(KX−2+KY−2).E[D^2]=E[Y^2e_X^2]+E[X^2e_Y^2]+o(K_X^-2+K_Y^-2). (4) Because eXe_X is a deterministic function of X and eYe_Y is a deterministic function of Y, conditioning yields [Y2eX2]=[[Y2∣X]eX2],[X2eY2]=[[X2∣Y]eY2].E[Y^2e_X^2]=E\! [E[Y^2 X]\ e_X^2 ], [X^2e_Y^2]=E\! [E[X^2 Y]\ e_Y^2 ]. Define the conditional second moments as wX(x)≜[Y2∣X=x],wY(y)≜[X2∣Y=y].w_X(x) [Y^2 X=x], w_Y(y) [X^2 Y=y]. (5) Then (1) and (4), along with the fact that [D]=O(KX−2+KY−2)E[D]=O(K_X^-2+K_Y^-2) imply the two-scalar high-rate reduction ℰ(QX,QY) (Q_X,Q_Y) =mnk([wX(X)(X−QX(X))2]+[wY(Y)(Y−QY(Y))2]) =mnk (E[w_X(X)(X-Q_X(X))^2]+E[w_Y(Y)(Y-Q_Y(Y))^2] ) +o(KX−2+KY−2). +o(K_X^-2+K_Y^-2). (6) Now let’s denote fXf_X and fYf_Y the marginal densities of X and Y, and further assume a regularity condition. These conditions are used to justify the asymptotic analysis and to derive weighted optimal rates. Assumption 2 (High-rate regularity). The pair (X,Y)(X,Y) admits a joint density fX,Yf_X,Y that is twice continuously differentiable. The functions fX,fY,wX,wYf_X,f_Y,w_X,w_Y are continuous and strictly positive on ℝR, and ∫ℝ(fX(x)wX(x))1/3dx<∞,∫ℝ(fY(y)wY(y))1/3dy<∞. _R(f_X(x)w_X(x))^1/3\,dx<∞, _R(f_Y(y)w_Y(y))^1/3\,dy<∞. Moreover, the conditional mean functions μY|X(x)≜[Y∣X=x] _Y|X(x) [Y X=x] and μX|Y(y)≜[X∣Y=y] _X|Y(y) [X Y=y] are continuously differentiable and the integrability conditions used in Appendix A and Appendix B hold.111These conditions are mild for common smooth models (e.g., jointly Gaussian pairs) and are stated explicitly where they are used. Now denote the integrals in ˜2 as IX≜∫−∞(fX(x)wX(x))1/3dx,IY≜∫−∞(fY(y)wY(y))1/3dy.I_X _-∞^∞(f_X(x)w_X(x))^1/3\,dx, I_Y _-∞^∞(f_Y(y)w_Y(y))^1/3\,dy. (7) An application of Cauchy-Schwartz gives the optimal high-rate quantizer for matrix multiplication. Theorem 1 (High-rate optimal matrix multiplication MSE). Under Assumptions 1–2, infQX,QYℰ(QX,QY)=mnk(IX312KX2+IY312KY2)+o(1KX2+1KY2). _Q_X,Q_Y\ E(Q_X,Q_Y)=mnk ( I_X^312\,K_X^2+ I_Y^312\,K_Y^2 )+o\! ( 1K_X^2+ 1K_Y^2 ). (8) Moreover, asymptotically optimal KXK_X- and KYK_Y-level quantizers are companding quantizers with point densities λX⋆(x)=(fX(x)wX(x))1/3IX,λY⋆(y)=(fY(y)wY(y))1/3IY. _X (x)= (f_X(x)w_X(x))^1/3I_X, _Y (y)= (f_Y(y)w_Y(y))^1/3I_Y. (9) Proof. Appendix A proves the decoupling (2.2) with a remainder o(KX−2+KY−2)o(K_X^-2+K_Y^-2) and shows that the bias term in (1) is negligible at the K−2K^-2 scale. Appendix B proves a weighted scalar high-rate theorem (including both achievability and converse) and identifies the unique optimizing point densities. Combining these two appendices yields (8) and (9). ∎ Corollary 1 (Rate form and optimal bit split). Let RX=log2KXR_X= _2K_X and RY=log2KYR_Y= _2K_Y. Then the leading term in (8) can be written as infQX,QYℰ(QX,QY)=mnk(αX 2−2RX+αY 2−2RY)+o(2−2RX+2−2RY), _Q_X,Q_Y\ E(Q_X,Q_Y)=mnk ( _X\,2^-2R_X+ _Y\,2^-2R_Y )+o(2^-2R_X+2^-2R_Y), where αX≜IX3/12 _X I_X^3/12 and αY≜IY3/12 _Y I_Y^3/12. If the pair rate R≜RX+RYR R_X+R_Y is fixed and large, then the minimizer of the leading term satisfies RX⋆=R2+14log2(αXαY),RY⋆=R2−14log2(αXαY),R_X = R2+ 14 _2\! ( _X _Y ), R_Y = R2- 14 _2\! ( _X _Y ), equivalently KX/KY=(αX/αY)1/4K_X/K_Y=( _X/ _Y)^1/4. 2.3 Special Case: Correlated Gaussian Assume the multiplicative pair (X,Y)(X,Y) is bivariate Gaussian: [XY]∼([00],[σX2ρσXσYρσXσYσY2]),ρ∈(−1,1). bmatrixX\\ Y bmatrix \! ( bmatrix0\\ 0 bmatrix, bmatrix _X^2&ρ _X _Y\\ ρ _X _Y& _Y^2 bmatrix ), ρ∈(-1,1). (10) Applying Theorem˜1 to the Gaussian case enables a specialization to Gaussian joint densities. See Appendix C for a proof. Corollary 2 (Closed-form asymptotically optimal point density). Under (12), the optimal companding point densities in (9) take the closed form λX⋆(x) _X (x) =exp(−x26σX2)((1−ρ2)+ρ2x2σX2)1/3∫−∞exp(−t26σX2)((1−ρ2)+ρ2t2σX2)1/3dt, = \! (- x^26 _X^2 ) ((1-ρ^2)+ρ^2 x^2 _X^2 )^1/3 _-∞^∞ \! (- t^26 _X^2 ) ((1-ρ^2)+ρ^2 t^2 _X^2 )^1/3\,dt, λY⋆(y) _Y (y) =exp(−y26σY2)((1−ρ2)+ρ2y2σY2)1/3∫−∞exp(−t26σY2)((1−ρ2)+ρ2t2σY2)1/3dt. = \! (- y^26 _Y^2 ) ((1-ρ^2)+ρ^2 y^2 _Y^2 )^1/3 _-∞^∞ \! (- t^26 _Y^2 ) ((1-ρ^2)+ρ^2 t^2 _Y^2 )^1/3\,dt. If we write the normalized variable as u=x/σXu=x/ _X, we have λ⋆(u)∝exp(−u26)((1−ρ2)+ρ2u2)1/3.λ (u)\ \ \! (- u^26 ) ((1-ρ^2)+ρ^2u^2 )^1/3. The density has a qualitative difference compared to Gaussian companders: specifically, the optimal λ∗λ^* goes through a phase transition as ρ increases: at first it has a single mode, but as ρ increases two modes appear. See Fig.˜1 for a schematic, and Appendix D for a proof. Theorem 2 (Unimodal-to-bimodal transition at |ρ|=1/3 ρ =1/ 3). Let λ⋆(u)λ (u) be the optimal density shape above. 1. If ρ2<1/3ρ^2<1/3, then λ⋆(u)λ (u) is unimodal with a unique global maximum at u=0u=0. 2. If ρ2>1/3ρ^2>1/3, then λ⋆(u)λ (u) is bimodal: u=0u=0 is a strict local minimum and the two symmetric global maxima occur at upeak=±3−1ρ2.u_peak=± 3- 1ρ^2. 3. If ρ2=1/3ρ^2=1/3, the curvature at u=0u=0 vanishes (critical splitting point). Figure 1: Optimal density phase transition. When ρ=0ρ=0, there is only a single mode. As ρ increases, an additional mode emerges at the critical value |ρ|=1/3 ρ =1/ 3. At last, direct substitution gives the high-rate loss for the proposed matmul quantization. Corollary 3 (Gaussian high-rate constant for matrix multiplication MSE). Let KX=KY=K_X=K_Y=K. Define J(ρ)≜∫−∞exp(−u26)((1−ρ2)+ρ2u2)1/3du.J(ρ) _-∞^∞ \! (- u^26 ) ((1-ρ^2)+ρ^2u^2 )^1/3\,du. Then limK→∞K2infQX,QY[‖AB−A^B^‖F2]=mknσX2σY262πJ(ρ)3. _K→∞K^2\ _Q_X,Q_Y\ E [ AB- A B _F^2 ]= mkn\, _X^2 _Y^26 2π\ J(ρ)^3. A closed form for J(ρ)J(ρ) is given in Appendix E. 3 Experiments We now empirically evaluate the performance of our derived quantization method with a variety of other state-of-the-art quantization methods on synthetic and real-world data. 3.1 Synthetic Experiments: Matrix Multiplication First, we compare the performance of our optimal matrix multiplication quantizer with various commonly used quantizers for synthetically generated matrices that conform to our correlated Gaussian model. For this experiment, we test a variety of ρ values and compare the quantization error in relative Frobenius norm for the matrices. 3.1.1 Alternate Quantizer Descriptions 1. MatMul-Opt (Ours). The theory-optimal companding quantizer derived from Corollary 2 for the correlated Gaussain case. Bin boundaries ti\t_i\ are placed by inverting the CDF of the optimal point density λ∗(u)∝exp(−u26)[(1−ρ2)+ρ2u2]1/3,u=x/σ,λ^*(u) \! (- u^26 ) [(1-ρ^2)+ρ^2u^2 ]^1/3, u=x/σ, (11) so that equal probability mass of λ∗λ^* falls in each bin. Separate quantizers are built for A and B using their empirical standard deviations σ^A σ_A and σ^B σ_B and the known correlation ρ. 2. Gaussian Compander. A companding quantizer whose point density is the ρ=0ρ=0 special case of the optimal density from standard scalar high-rate theory. 3. Lloyd-Max. The iteratively computed optimal scalar quantizer for a Gaussian source X∼(0,σ2)X (0,σ^2) Lloyd (1982). Starting from quantile-midpoint initialization, the algorithm alternates between (i) setting boundaries to midpoints of adjacent reconstruction levels and (i) setting levels to the conditional mean of |phi|phi over each bin, until convergence in ℓ∞ ^∞ norm up to 200200 iterations. 4. Uniform. A symmetric uniform quantizer with K evenly-spaced reconstruction levles on [−c,c][-c,c]. The clip value c is selected by a grid search over c∈[1.5σ^, 5.0σ^]c∈[1.5 σ,\,5.0 σ] (36 points) to minimize empirical MSE on entries of the matrix being quantized. Separate clip values are calibrated for A and B. 5. μ-Law. The ITU-T G.711 μ-law compander Jayant and Noll (1984) with μ=255μ=255, adapted to continuous-valued inputs. The companding function f(x)=sgn(x)ln(1+μ|x/c|)ln(1+μ)f(x)=sgn(x)\, (1+μ|x/c|) (1+μ) maps inputs to a uniform grid of K levels in the logarithmically compressed domain; the inverse map recovers reconstruction levels in the original domain. The clip is fixed at c=4σ^c=4 σ. μ-law compression places more quantization levels near zero, which improves SNR for speech-like signals but is not matched to the matrix-multiplication task. 6. A-Law. The ITU-T G.711 A-law compander Jayant and Noll (1984) with A=87.6A=87.6, also adapted to continuous inputs. The two-piece companding function is linear for |x/c|<1/A|x/c|<1/A and logarithmic otherwise: f(x)=sgn(x)⋅A|x/c|1+lnA|x/c|<1A,1+ln(A|x/c|)1+lnA1A≤|x/c|≤1. f(x)=sgn(x)· cases A\,|x/c|1+ A&|x/c|< 1A,\\[6.0pt] 1+ (A\,|x/c|)1+ A& 1A≤|x/c|≤ 1. cases K uniform levels are placed in the companded domain with clip c=4σ^c=4 σ. A-law and μ-law share the same logarithmic motivation and serve as classical telecommunications baselines. 7. NF4 (Normal Float 4-bit). A K-level fixed codebook whose points are the quantile midpoints of (0,1)N(0,1): ci=Φ−1((i+12)/K)c_i= ^-1\! ((i+ 12)/K ) for i=0,…,K−1i=0,…,K-1, normalized so that maxi|ci|=1 _i|c_i|=1, with endpoints clamped to ±1± 1. This is an approximation of the QLoRA NF4 codebook Dettmers et al. (2023), which places levels at Gaussian quantile midpoints to minimize MSE for normally-distributed weights. A per-matrix scalar scale is selected by grid search over [0.5σ^, 4.0σ^][0.5 σ,\,4.0 σ] (60 points) to minimize empirical MSE. 8. NV FP4 (E2M1). NVIDIA’s 4-bit floating-point format NVIDIA Technical Blog (2025) with 1 sign bit, 2 exponent bits, and 1 mantissa bit with exponent bias 1. This yields 15 distinct finite values: 0,±0.5,±1.0,±1.5,±2.0,±3.0,±4.0,±6.0\0,\,± 0.5,\,± 1.0,\,± 1.5,\,± 2.0,\,± 3.0,\,± 4.0,\,± 6.0\. Nearest-neighbor quantization is performed after scaling the codebook by a per-matrix scalar selected by grid search over [0.25σ^, 4.0σ^][0.25 σ,\,4.0 σ] (60 points) to minimize empirical MSE. 3.1.2 Results We obtained the following results averaged over 500500 generated A∈ℝ128×256A ^128× 256 and B∈ℝ256×128B ^256× 128 according to our correlated Gaussian model. From this, we can see that our optimal matrix multiplication quantizer consistently outperforms other commonly used quantization methods for our specified tasks. Figure 2: Performance of our optimal quantizer vs. other commonly used quantizers. We include 95%95\% error bars but used enough synthetic data that they are barely visible. 3.2 Application in Quantized Least Squares We consider solving the least squares problem minW‖XW−Y‖F2, _W\|XW-Y\|_F^2, where X∈ℝn×d,W,W∗∈ℝd×mX ^n× d,W,W^* ^d× m, Y=XW∗+ϵZY=XW^*+ε Z and the entries of X,W∗X,W^* satisfy ˜1, and the joint distribution [XW∗]∼([00],[σX2ρσXσYρσXσYσY2]),ρ=0.6. bmatrixX\\ W^* bmatrix \! ( bmatrix0\\ 0 bmatrix, bmatrix _X^2&ρ _X _Y\\ ρ _X _Y& _Y^2 bmatrix ), ρ=0.6. (12) Imagine X is so large so that it cannot fit into a single GPU. A natural solution to this would be distributing the matrix over multiple GPUs then solving the problem by appropriately aggregating subproblems, but it may be timely as the loading/deloading time of the data matrix could be a bottleneck in such settings. Hence quantizing X and Y to load them onto a GPU then solving the quantized least squares, minW‖QXW−QY‖F2, _W\|Q_XW-Q_Y\|_F^2, may make sense when solving the problem faster could be much more important than solving the problem exactly. We apply the proposed quantization to the above problem and show that we can have better accuracy in terms of ‖W−W∗‖F\|W-W^*\|_F under the same bit budget. We compare three different schemes, where • Scheme 1: quantize X, Y with Gaussian high-rate quantizer, simply considering the marginal distribution. • Scheme 2: sweep through possible ρ∈[−1,1]ρ∈[-1,1] and plot the loss with the best ρ. • Scheme 3: estimate ρ with ρ^=1ndm∑i=1d∑j=1m(XW¯)ij, ρ= 1ndm _i=1^d _j=1^m(X W)_ij, where W¯ W is a solution to the subproblem minW‖X¯W−Y¯‖F2. _W\| XW- Y\|_F^2. Here X¯ X, Y¯ Y are row subsampled matrices of X,YX,Y. Figure 3: Comparison of three different schemes for solving quantized least squares. The y axis is the difference between ground truth and the quantized solution ‖W−W∗‖F\|W-W^*\|_F. The results show that using nonzero ρ can improve the accuracy of quantized least squares. Sweeping along the possible ρ is better than using a ρ estimate, especially for lower bit budgets. The main reason for this is that for bits=3,4, ρ=−0.9ρ=-0.9 is the optimal ρ for quantized least squares, which is very different from the ground truth ρ=0.6ρ=0.6. Such observation implies that there could be a different reason for why the correlation-aware quantization algorithm works better in the least squares setting. For higher rates the optimal ρ becomes similar to ground truth ρ, and is better than naive Gaussian. 3.3 Quantization of Transformer-based Models Activation quantization is critical for efficient inference in Transformer-based models. We compare our method against INT8 quantization Dettmers et al. (2022) and FP8 quantization Micikevicius et al. (2022). In all cases, we apply per-token scaling to Q and K before quantization and use K=256K=256 quantization levels (8 bits). We empirically observe non-trivial correlation between entries of the query and key activations within attention heads (Figure 4). This motivates applying our correlation-aware matrix multiplication quantizer independently to each attention head. For evaluation, we measure the relative Frobenius error of the pre-softmax attention logits: ‖QK⊤−Q^K^⊤‖F‖QK⊤‖F, QK - Q K _F QK _F, where Q Q and K K denote quantized activations. This metric directly captures distortion in the attention logits. Figure 4: Estimated ρ value for each layer and head in GPT-2 Small 3.3.1 Models and Data We evaluate on the GPT-2 family of models Radford et al. (2019) where there key and query activations are directly multiplied in addition so some Qwen3 models Yang et al. (2025) which apply rotary embeddings to the key and query activations before multiplying. For the Qwen3 models we tune ρ based on the key and query activations after passing through the rotary embeddings. Sequences are drawn from WikiText-2 Merity et al. (2016). We use 64 non-overlapping sequences of length 128 from the training split for evaluation. We hold out 32 sequences from the validation split for calibration. Per-head statistics are computed by concatenating all evaluation sequences. 3.3.2 Activation Collection We extract Q and K at each attention layer using PyTorch forward hooks. For GPT-2, we hook the fused c_attn module and split the output into Q, K, and V. For Qwen3, we hook the q_norm, k_norm, and rotary_emb submodules per layer; the post-normalization Q and K are buffered, and rotary position embeddings are applied in NumPy to obtain post-RoPE activations. 3.3.3 ρ-tuning For each attention head, we tune ρ∈[0,0.95]ρ∈[0,0.95] on the calibration set by grid search over 40 evenly spaced values, minimizing the relative Frobenius error above. For all methods, we use the same per-token ℓ∞ _∞ scaling, i.e., we normalize each token vector so all entries lie in [−1,1][-1,1]. Given a tuned ρ, we construct reproduction points using the closed-form density from Corollary 2. 3.3.4 Results Table 1 reports the fraction of heads for which ρ-tuned achieves lower relative Frobenius error than the baseline quantizer (a “win”). Table 1: Win rate of ρ-tuned vs. INT8 and FP8 across GPT-2 sizes (win = lower relative Frobenius error per head). Model ρ-tuned vs FP8 (% wins) ρ-tuned vs INT8 (% wins) GPT-2 Small (12 layers, 12 heads) 100% 96.5% GPT-2 Medium (24 layers, 16 heads) 100% 97.9% GPT-2 Large (36 layers, 20 heads) 100% 100% GPT-2 XL (48 layers, 25 heads) 100% 99.7% Qwen3-0.6B (28 layers, 8 heads) 98.7% 65.6% Qwen3-1.7B (28 layers, 8 heads) 98.7% 59.4% Qwen3-8B (36 layers, 8 heads) 86.1% 42.7% Across model sizes, ρ-tuned matches or improves upon both INT8 and FP8, with the largest gains observed in larger models for the GPT-2 family of models while the opposite trend was observed on the Qwen3 family. We saw the performance degrade for larger Qwen3 models with the tuned ρ algorithm losing to INT8 for Qwen3-8B. The tuned values of ρ tended to be higher than the estimated values of ρ (see figure 5), which indicates some level of model misspecification in our correlated Gaussian model. Figure 5: Tuned |ρ||ρ| for GPT-small This misspecification seemed to hurt most in the Qwen3 family of models, which we posit is due to the nature of the rotary embeddings and their effect on the activation distributions. Limitations and Future Work. We plan to build upon our method so that it performs better on key and query quantization in models with rotary embeddings. We will also incorporate outlier-handling techniques commonly used in LLM quantization Dettmers et al. (2022) and report downstream task metrics in addition to logit-level distortion. 4 Conclusion We derived a sharp high-rate characterization of optimal scalar quantization for matrix multiplication MSE under a pair-i.i.d. inner-product model. The leading error constant decomposes into two weighted scalar quantization problems with weights given by conditional second moments, leading to explicit optimal companding densities. For correlated Gaussian multiplicative pairs we obtained a closed-form density and proved a sharp unimodal-to-bimodal phase transition at |ρ|=1/3 ρ =1/ 3. We also benchmarked our derived quantizer on synthetically generated matrices for matrix multiplication and quantized least squares as well as on key and query activation quantization of the GPT-2 family of models. Appendix A Proof of the Decoupling and High-Rate Expansion This appendix provides a complete proof of the reduction (2.2) and the negligibility of the bias term in (1) at the K−2K^-2 scale. A.1 Exact variance decomposition Let Dℓ≜XℓYℓ−X^ℓY^ℓD_ X_ Y_ - X_ Y_ . Then S−S^=∑ℓ=1kDℓ.S- S= _ =1^kD_ . Because (Xℓ,Yℓ)\(X_ ,Y_ )\ are i.i.d. and quantized entrywise, the DℓD_ are i.i.d. with the same law as D. Hence [(S−S^)2]=∑ℓ=1k[Dℓ2]+2∑ℓ<r[Dℓ][Dr]=k[D2]+k(k−1)([D])2,E[(S- S)^2]= _ =1^kE[D_ ^2]+2 _ <rE[D_ ]E[D_r]=kE[D^2]+k(k-1)(E[D])^2, which is (1). A.2 Quantizer model and elementary cell identities Throughout this appendix we analyze companding quantizers QX=QKX,λXQ_X=Q_K_X, _X and QY=QKY,λYQ_Y=Q_K_Y, _Y constructed as in Section 2, with λX,λY _X, _Y continuously differentiable and strictly positive. For QK,λQ_K,λ, let boundaries xi=G−1(i/K)x_i=G^-1(i/K) and reproduction points ri=G−1((i−12)/K)r_i=G^-1((i- 12)/K), and denote the iith cell by Ii=[xi−1,xi)I_i=[x_i-1,x_i) with width Δi=xi−xi−1 _i=x_i-x_i-1 and midpoint mi≜(xi−1+xi)/2m_i (x_i-1+x_i)/2. Lemma 1 (Cell width formula). For each i=1,…,Ki=1,…,K, there exists ξi∈Ii _i∈ I_i such that Δi=1Kλ(ξi). _i= 1K\,λ( _i). (13) Proof. By definition G(xi)−G(xi−1)=1/KG(x_i)-G(x_i-1)=1/K. Since G is differentiable with derivative G′(x)=λ(x)G (x)=λ(x), the mean value theorem gives a point ξi∈(xi−1,xi) _i∈(x_i-1,x_i) such that 1K=G(xi)−G(xi−1)=G′(ξi)(xi−xi−1)=λ(ξi)Δi. 1K=G(x_i)-G(x_i-1)=G ( _i)(x_i-x_i-1)=λ( _i) _i. Rearranging yields (13). ∎ Lemma 2 (Reproduction points are second-order close to cell midpoints). Assume λ is continuously differentiable. Then for each fixed compact interval [−M,M][-M,M] there exists CM<∞C_M<∞ and KMK_M such that for all K≥KMK≥ K_M and all cells IiI_i intersecting [−M,M][-M,M], |ri−mi|≤CMΔi2. r_i-m_i ≤ C_M\, _i^2. (14) Consequently, ∫xi−1xi(x−ri)dx _x_i-1^x_i(x-r_i)\,dx =Δi(mi−ri)=O(Δi3), = _i(m_i-r_i)=O( _i^3), (15) ∫xi−1xi(x−ri)2dx _x_i-1^x_i(x-r_i)^2\,dx =Δi312+O(Δi5). = _i^312+O( _i^5). uniformly over such cells. Proof. Let g≜G−1g G^-1 and u≜(i−12)/Ku (i- 12)/K, so that xi−1=g(u−1/(2K))x_i-1=g(u-1/(2K)), xi=g(u+1/(2K))x_i=g(u+1/(2K)), and ri=g(u)r_i=g(u). We know that g is twice continuously differentiable by the inverse function theorem, so taking the second-order Taylor expansion of g around u gives g(u±12K)=g(u)±g′(u)2K+g′(ζ±)8K2g\! (u± 12K )=g(u)± g (u)2K+ g ( _±)8K^2 for some ζ± _± between u and u±1/(2K)u± 1/(2K). Averaging the “++” and “−-” expansions yields mi=xi−1+xi2=g(u)+g′(ζ+)+g′(ζ−)16K2,m_i= x_i-1+x_i2=g(u)+ g ( _+)+g ( _-)16K^2, hence mi−ri=g′(ζ+)+g′(ζ−)16K2.m_i-r_i= g ( _+)+g ( _-)16K^2. We note that by Lemma 1, Δi=1Kλ(ξi). _i= 1Kλ( _i). Since λ is continuous and strictly positive, it is bounded away from 0 on [−M,M][-M,M], so Δi=Θ(1/K) _i= (1/K) uniformly over cells intersecting [−M,M][-M,M]. Therefore, on a compact [−M,M][-M,M], the corresponding u values lie in a compact subset of (0,1)(0,1) for all sufficiently large K (since cells intersecting [−M,M][-M,M] have both endpoints in g−1([−M−C/K,M+C/K])g^-1([-M-C/K,M+C/K]) as a consequence of our previous statement), so g′g is bounded there. Thus |mi−ri|=O(1/K2)=O(Δi2) m_i-r_i =O(1/K^2)=O( _i^2) uniformly over cells intersecting [−M,M][-M,M], proving (14). Finally, ∫xi−1xi(x−ri)dx _x_i-1^x_i(x-r_i)\,dx =[(x−ri)22]xi−1xi=(xi−ri)2−(xi−1−ri)22 = [ (x-r_i)^22 ]_x_i-1^x_i= (x_i-r_i)^2-(x_i-1-r_i)^22 =Δi(mi−ri) = _i(m_i-r_i) which together with (14) gives the first identity in (15). For the second identity, write (x−ri)2=(x−mi)2+(mi−ri)2+2(x−mi)(mi−ri)(x-r_i)^2=(x-m_i)^2+(m_i-r_i)^2+2(x-m_i)(m_i-r_i) and integrate over [xi−1,xi][x_i-1,x_i]. The cross term integrates to zero by symmetry around mim_i, giving ∫xi−1xi(x−ri)2dx _x_i-1^x_i(x-r_i)^2\,dx =∫xi−1xi(x−mi)2dx+Δi(mi−ri)2 = _x_i-1^x_i(x-m_i)^2\,dx+ _i(m_i-r_i)^2 =Δi312+O(Δi5), = _i^312+O( _i^5), since ∫xi−1xi(x−mi)2dx=Δi3/12 _x_i-1^x_i(x-m_i)^2\,dx= _i^3/12 exactly and (mi−ri)2=O(Δi4)(m_i-r_i)^2=O( _i^4). ∎ A.3 Bias term is negligible at the K−2K^-2 scale We prove the key estimate [D]=O(KX−2+KY−2)E[D]=O(K_X^-2+K_Y^-2). Lemma 3 (A weighted first-moment bound for companders). Let X have density f and let g:ℝ→ℝg:R be continuously differentiable. Let QK,λQ_K,λ be a companding quantizer with continuously differentiable point density λ>0λ>0. Assume that q(x)≜g(x)f(x)q(x) g(x)f(x) is twice continuously differentiable. Then [g(X)(X−QK,λ(X))]=O(1K2),E [g(X)\,(X-Q_K,λ(X)) ]=O\! ( 1K^2 ), (16) as K→∞K→∞. Proof. Let the cells be Ii=[xi−1,xi)I_i=[x_i-1,x_i) with reproduction rir_i, width Δi _i, and midpoint mim_i as in Section A.2. Write e(x)≜x−QK,λ(x)e(x) x-Q_K,λ(x), so that e(x)=x−rie(x)=x-r_i for x∈Iix∈ I_i. Then [g(X)e(X)]=∑i=1K∫Iiq(x)(x−ri)dx.E[g(X)e(X)]= _i=1^K _I_iq(x)(x-r_i)\,dx. Fix M>0M>0 and split the sum into cells intersecting [−M,M][-M,M] (“interior cells”) and the remaining tail cells (“exterior cells”). Step 1: Interior cells. For an interior cell Ii⊂[−M−η,M+η]I_i⊂[-M-η,M+η] (for a fixed small η>0η>0 and K large enough), expand q around the midpoint mim_i: q(x)=q(mi)+q′(mi)(x−mi)+12q′(ζi,x)(x−mi)2,q(x)=q(m_i)+q (m_i)(x-m_i)+ 12q ( _i,x)(x-m_i)^2, for some ζi,x∈Ii _i,x∈ I_i. Multiply by (x−ri)=(x−mi)+(mi−ri)(x-r_i)=(x-m_i)+(m_i-r_i) and integrate over IiI_i: ∫Iiq(x)(x−ri)dx=Ti,0+Ti,1+Ti,2, _I_iq(x)(x-r_i)\,dx=T_i,0+T_i,1+T_i,2, where Ti,0 T_i,0 ≜q(mi)∫Ii(x−ri)dx, q(m_i) _I_i(x-r_i)\,dx, Ti,1 T_i,1 ≜q′(mi)∫Ii(x−mi)(x−ri)dx, q (m_i) _I_i(x-m_i)(x-r_i)\,dx, Ti,2 T_i,2 ≜12∫Iiq′(ζi,x)(x−mi)2(x−ri)dx. 12 _I_iq ( _i,x)(x-m_i)^2(x-r_i)\,dx. We bound each term using Lemma 2. First, Lemma 2 gives ∫Ii(x−ri)dx=O(Δi3) _I_i(x-r_i)\,dx=O( _i^3). Since q(mi)q(m_i) is bounded on [−M−η,M+η][-M-η,M+η], we have Ti,0=O(Δi3)T_i,0=O( _i^3) uniformly over interior cells. Second, note that (x−mi)(x−ri)=(x−mi)2+(mi−ri)(x−mi)(x-m_i)(x-r_i)=(x-m_i)^2+(m_i-r_i)(x-m_i) and ∫Ii(x−mi)dx=0 _I_i(x-m_i)\,dx=0 by symmetry. Hence ∫Ii(x−mi)(x−ri)dx=∫Ii(x−mi)2dx=Δi312. _I_i(x-m_i)(x-r_i)\,dx= _I_i(x-m_i)^2\,dx= _i^312. Since q′(mi)q (m_i) is bounded on [−M−η,M+η][-M-η,M+η], it follows that Ti,1=O(Δi3)T_i,1=O( _i^3) uniformly over interior cells. Third, on IiI_i we have |x−mi|≤Δi/2 x-m_i ≤ _i/2 and |x−ri|≤|x−mi|+|mi−ri|≤Δi/2+O(Δi2)=O(Δi) x-r_i ≤ x-m_i + m_i-r_i ≤ _i/2+O( _i^2)=O( _i). Since q′q is bounded on [−M−η,M+η][-M-η,M+η], we obtain |Ti,2| T_i,2 ≤C∫Ii|x−mi|2|x−ri|dx ≤ C _I_i x-m_i ^2\, x-r_i \,dx ≤C′Δi∫Ii(x−mi)2dx ≤ C _i _I_i(x-m_i)^2\,dx =C′ΔiΔi312 =C _i _i^312 =O(Δi4). =O( _i^4). Combining Ti,0=O(Δi3)T_i,0=O( _i^3), Ti,1=O(Δi3)T_i,1=O( _i^3), and Ti,2=O(Δi4)T_i,2=O( _i^4) gives ∫Iiq(x)(x−ri)dx=O(Δi3)uniformly over interior cells. _I_iq(x)(x-r_i)\,dx=O( _i^3) over interior cells. Summing over all interior cells and using Lemma 1 (which gives Δi=O(1/K) _i=O(1/K) on compacts), we obtain ∑i:Ii∩[−M,M]≠∅∫Iiq(x)(x−ri)dx _i:\,I_i∩[-M,M]≠ _I_iq(x)(x-r_i)\,dx =O(∑i:Ii∩[−M,M]≠∅Δi3) =O\! ( _i:\,I_i∩[-M,M]≠ _i^3 ) =O(1K2), =O\! ( 1K^2 ), because there are O(K)O(K) interior cells and each has Δi3=O(1/K3) _i^3=O(1/K^3) uniformly. Step 2: Tail cells. Let AM=x:|x|>MA_M=\x: x >M\. Using Cauchy–Schwarz, |∫AMq(x)(x−Q(x))dx| _A_Mq(x)(x-Q(x))\,dx ≤(∫AMq(x)2f(x)−1dx)1/2 ≤ ( _A_Mq(x)^2f(x)^-1\,dx )^1/2 ×(∫AMf(x)(x−Q(x))2dx)1/2. × ( _A_Mf(x)(x-Q(x))^2\,dx )^1/2. The second factor is the (unweighted) MSE on the tail set and is bounded by [(X−Q(X))2]1/2=O(1/K)E[(X-Q(X))^2]^1/2=O(1/K) for companding quantizers (this is a special case of Appendix B with w≡1w≡ 1). The first factor can be made arbitrarily small by choosing M large enough because q2/f=g2fq^2/f=g^2f is integrable whenever [g(X)2]<∞E[g(X)^2]<∞, and in our application g is a conditional mean with finite second moment by Assumption 1. Therefore, for any ε>0 >0 we can pick M so that the tail contribution is at most ε/K /K in absolute value uniformly in K. Step 3: Combine. For this fixed M and all sufficiently large K, [g(X)e(X)]=O(1K2)+ε1K.E[g(X)e(X)]=O\! ( 1K^2 )+ 1K. Letting ε↓0 0 and then K→∞K→∞ yields (16). ∎ Proposition 1 (Bias order for the product error). Let D=XY−X^Y^D=XY- X Y with X^=QKX,λX(X) X=Q_K_X, _X(X) and Y^=QKY,λY(Y) Y=Q_K_Y, _Y(Y) as above. Then [D]=O(KX−2+KY−2).E[D]=O(K_X^-2+K_Y^-2). (17) Consequently, k(k−1)([D])2=o(KX−2+KY−2)asKX,KY→∞.k(k-1) (E[D] )^2=o(K_X^-2+K_Y^-2) K_X,K_Y→∞. Proof. Write eX=X−X^e_X=X- X and eY=Y−Y^e_Y=Y- Y. Using (2), [D]=[YeX]+[XeY]−[eXeY].E[D]=E[Ye_X]+E[Xe_Y]-E[e_Xe_Y]. Since eXe_X is a function of X only, [YeX]=[[Y∣X]eX]=[μY|X(X)eX].E[Ye_X]=E [E[Y X]\,e_X ]=E[ _Y|X(X)\,e_X]. Apply Lemma 3 with g=μY|Xg= _Y|X and QKX,λXQ_K_X, _X to obtain [YeX]=O(KX−2)E[Ye_X]=O(K_X^-2). Similarly, [XeY]=O(KY−2)E[Xe_Y]=O(K_Y^-2). For the final term, Cauchy–Schwarz gives |[eXeY]|≤[eX2][eY2]. [e_Xe_Y] ≤ E[e_X^2]\,E[e_Y^2]. The unweighted companding MSE satisfies [eX2]=O(KX−2)E[e_X^2]=O(K_X^-2) and [eY2]=O(KY−2)E[e_Y^2]=O(K_Y^-2) (a special case of Appendix B), hence [eXeY]=O((KXKY)−1)E[e_Xe_Y]=O((K_XK_Y)^-1). Using 2/(KXKY)≤1/KX2+1/KY22/(K_XK_Y)≤ 1/K_X^2+1/K_Y^2 (AM–GM), this is also O(KX−2+KY−2)O(K_X^-2+K_Y^-2). Combining the three bounds yields (17). Finally, squaring (17) gives ([D])2=O((KX−2+KY−2)2)=o(KX−2+KY−2)(E[D])^2=O((K_X^-2+K_Y^-2)^2)=o(K_X^-2+K_Y^-2), completing the proof. ∎ A.4 Dominant terms in [D2]E[D^2] We now prove that the mixed terms in (3) have expectation O(KX−2KY−2)O(K_X^-2K_Y^-2). Lemma 4 (A generic mixed-cell bound). Let ϕ:ℝ2→ℝφ:R^2 be twice continuously differentiable. Consider companding quantizers QX=QKX,λXQ_X=Q_K_X, _X and QY=QKY,λYQ_Y=Q_K_Y, _Y with continuosly differentiable point densities. Let rir_i and sjs_j denote the reproduction points and let Δi _i and δj _j denote the corresponding cell widths for QXQ_X and QYQ_Y respectively. Assume the function Mϕ(x,y)≜max|α|+|β|=2|∂xα∂yβϕ(x,y)|M_φ(x,y) _|α|+|β|=2 _x^α _y^βφ(x,y) is integrable with respect to fX,Y(x,y)dxdyf_X,Y(x,y)\,dx\,dy when weighted by (λX(x)λY(y))−2( _X(x) _Y(y))^-2 on compact sets (as made explicit in the proof below). Then, as KX,KY→∞K_X,K_Y→∞, ∑i=1KX∑j=1KY∫Ii∫Jjϕ(x,y)(x−ri)(y−sj)dydx=O(1KX2KY2). _i=1^K_X _j=1^K_Y _I_i _J_jφ(x,y)\,(x-r_i)(y-s_j)\,dy\,dx=O\! ( 1K_X^2K_Y^2 ). (18) Proof. Let Ii=[xi−1,xi)I_i=[x_i-1,x_i) and Jj=[yj−1,yj)J_j=[y_j-1,y_j) be the cells in x and y. Fix M>0M>0 and split the double sum into (i) rectangles intersecting [−M,M]2[-M,M]^2 and (i) rectangles in the complement. We bound both contributions. Step 1: Rectangles intersecting a compact set. For a rectangle Ii×JjI_i× J_j intersecting [−M,M]2[-M,M]^2, pick the center point (ri,sj)(r_i,s_j) and perform a second-order Taylor expansion of ϕφ around (ri,sj)(r_i,s_j): ϕ(x,y) φ(x,y) =ϕ(ri,sj)+ϕx(ri,sj)(x−ri) =φ(r_i,s_j)+ _x(r_i,s_j)(x-r_i) +ϕy(ri,sj)(y−sj)+Ri,j(x,y), + _y(r_i,s_j)(y-s_j)+R_i,j(x,y), where the remainder satisfies |Ri,j(x,y)|≤12Mϕ(ζi,j(x,y))((x−ri)2+(y−sj)2) R_i,j(x,y) ≤ 12M_φ( _i,j(x,y)) ((x-r_i)^2+(y-s_j)^2 ) for some ζi,j(x,y)∈Ii×Jj _i,j(x,y)∈ I_i× J_j. Multiply the Taylor expansion by (x−ri)(y−sj)(x-r_i)(y-s_j) and integrate over Ii×JjI_i× J_j. The constant term contributes ϕ(ri,sj)(∫Ii(x−ri)dx)(∫Jj(y−sj)dy).φ(r_i,s_j) ( _I_i(x-r_i)\,dx ) ( _J_j(y-s_j)\,dy ). By Lemma 2, each one-dimensional integral is O(Δi3)O( _i^3) and O(δj3)O( _j^3) on a compact region, hence the constant term contribution is O(Δi3δj3)O( _i^3 _j^3). The ϕx _x term contributes ϕx(ri,sj)(∫Ii(x−ri)2dx)(∫Jj(y−sj)dy)=O(Δi3δj3), _x(r_i,s_j) ( _I_i(x-r_i)^2\,dx ) ( _J_j(y-s_j)\,dy )=O( _i^3 _j^3), since ∫Ii(x−ri)2dx=O(Δi3) _I_i(x-r_i)^2\,dx=O( _i^3) by (15). The ϕy _y term is analogous. For the remainder term, note that on Ii×JjI_i× J_j we have |x−ri|=O(Δi) x-r_i =O( _i) and |y−sj|=O(δj) y-s_j =O( _j), so |Ri,j(x,y)(x−ri)(y−sj)|≤CMϕ(ζi,j(x,y))(Δi3δj+Δiδj3). R_i,j(x,y)(x-r_i)(y-s_j) ≤ C\,M_φ( _i,j(x,y)) ( _i^3 _j+ _i _j^3 ). Integrating over Ii×JjI_i× J_j contributes at most C(Δi3δj+Δiδj3) C ( _i^3 _j+ _i _j^3 ) ∫Ii∫JjMϕ(ζi,j(x,y))dydx≤ _I_i _J_jM_φ( _i,j(x,y))\,dy\,dx≤ C′Δiδj(Δi2+δj2)supIi×JjMϕ. C _i _j ( _i^2+ _j^2 ) _I_i× J_jM_φ. On compact sets, supIi×JjMϕ _I_i× J_jM_φ is bounded and Δi,δj=O(1/KX),O(1/KY) _i, _j=O(1/K_X),O(1/K_Y), so this remainder contribution is also O(Δi3δj3)O( _i^3 _j^3). Combining all pieces, we have shown that for rectangles intersecting [−M,M]2[-M,M]^2, ∫Ii∫Jjϕ(x,y)(x−ri)(y−sj)dydx=O(Δi3δj3), _I_i _J_jφ(x,y)(x-r_i)(y-s_j)\,dy\,dx=O( _i^3 _j^3), uniformly. Summing over the O(KXKY)O(K_XK_Y) rectangles intersecting [−M,M]2[-M,M]^2 and using Δi=O(1/KX) _i=O(1/K_X), δj=O(1/KY) _j=O(1/K_Y) on compacts yields a total compact contribution of order O(KXKY⋅KX−3KY−3)=O(KX−2KY−2)O(K_XK_Y· K_X^-3K_Y^-3)=O(K_X^-2K_Y^-2). Step 2: Tail rectangles. On the complement of [−M,M]2[-M,M]^2 we bound the entire integral by absolute values. Since |x−ri|≤|x|+|ri| x-r_i ≤ x + r_i and similarly for y−sjy-s_j, and since (X,Y)(X,Y) has finite (4+ϵ)(4+ε) moments, by choosing M large we can make the tail probability Pr((X,Y)∉[−M,M]2) ((X,Y)∉[-M,M]^2) arbitrarily small. The detailed bound follows the same truncation logic as in Lemma 3: first control the tail by moments and then let M→∞M→∞. This yields a tail contribution that is o(KX−2KY−2)o(K_X^-2K_Y^-2). Step 3: Combine and conclude. Let the rectangles which intersect [−M,M]2[-M,M]^2 be C:=(i,j):Ii×Jj∩[−M,M]2≠∅C:=\(i,j):I_i× J_j∩[-M,M]^2≠ \ and ψi,j(x,y):=ϕ(x,y)(x−ri)(y−sj) _i,j(x,y):=φ(x,y)\,(x-r_i)(y-s_j). Then ∑i=1KX∑j=1KY∫Ii∫Jjψi,j(x,y)dydx _i=1^K_X _j=1^K_Y _I_i _J_j _i,j(x,y)\,dy\,dx =∑(i,j)∈C∫Ii∫Jjψi,j(x,y)dydx = _(i,j)∈ C _I_i _J_j _i,j(x,y)\,dy\,dx +∑(i,j)∉C∫Ii∫Jjψi,j(x,y)dydx + _(i,j) ∈ C _I_i _J_j _i,j(x,y)\,dy\,dx and from Step 1 we know that ∑(i,j)∈C∫Ii∫Jjψi,j(x,y)dydx=O(KX−2KY−2) _(i,j)∈ C _I_i _J_j _i,j(x,y)\,dy\,dx=O(K_X^-2K_Y^-2). From Step 2 we know that ∫Ii∫Jjψi,j(x,y)dydx _I_i _J_j _i,j(x,y)\,dy\,dx =o(KX−2KY−2),∀(i,j)∉C =o(K_X^-2K_Y^-2),~~∀(i,j) ∈ C ⟹∑(i,j)∉C∫Ii∫Jjψi,j(x,y)dydx _(i,j) ∈ C _I_i _J_j _i,j(x,y)\,dy\,dx =O(KXKY)o(KX−2KY−2) =O(K_XK_Y)o(K_X^-2K_Y^-2) =O(KX−2KY−2). =O(K_X^-2K_Y^-2). Thus we see that ∑i=1KX∑j=1KY∫Ii∫Jjψi,j(x,y)dydx _i=1^K_X _j=1^K_Y _I_i _J_j _i,j(x,y)\,dy\,dx =O(KX−2KY−2) =O(K_X^-2K_Y^-2) which is exactly (18). ∎ Proposition 2 (Mixed terms in [D2]E[D^2] are O(KX−2KY−2)O(K_X^-2K_Y^-2)). Let D be as in Proposition 1. Then the mixed terms in (3) satisfy [eX2eY2] [e_X^2e_Y^2] =O(KX−2KY−2), =O(K_X^-2K_Y^-2), (19) [XYeXeY] [XYe_Xe_Y] =O(KX−2KY−2), =O(K_X^-2K_Y^-2), (20) [YeX2eY] [Ye_X^2e_Y] =O(KX−2KY−2), =O(K_X^-2K_Y^-2), (21) [XeXeY2] [Xe_Xe_Y^2] =O(KX−2KY−2). =O(K_X^-2K_Y^-2). (22) Consequently, [D2] [D^2] =[Y2eX2]+[X2eY2]+O(KX−2KY−2) =E[Y^2e_X^2]+E[X^2e_Y^2]+O(K_X^-2K_Y^-2) =[Y2eX2]+[X2eY2]+o(KX−2+KY−2). =E[Y^2e_X^2]+E[X^2e_Y^2]+o(K_X^-2+K_Y^-2). Proof. Each expectation can be written as a double sum over rectangles Ii×JjI_i× J_j. (i) [XYeXeY]E[XYe_Xe_Y]. Write [XYeXeY]=∑i=1KX∑j=1KY∫Ii∫Jjxy(x−ri)(y−sj)fX,Y(x,y)dydx.E[XYe_Xe_Y]= _i=1^K_X _j=1^K_Y _I_i _J_jxy(x-r_i)(y-s_j)f_X,Y(x,y)\,dy\,dx. Apply Lemma 4 with ϕ(x,y)=xyfX,Y(x,y)φ(x,y)=xyf_X,Y(x,y) to obtain (20). (i) [YeX2eY]E[Ye_X^2e_Y] and [XeXeY2]E[Xe_Xe_Y^2]. For [YeX2eY]E[Ye_X^2e_Y] we write [YeX2eY]=∑i,j∫Ii∫Jjy(x−ri)2(y−sj)fX,Y(x,y)dydx.E[Ye_X^2e_Y]= _i,j _I_i _J_jy(x-r_i)^2(y-s_j)f_X,Y(x,y)\,dy\,dx. Fix i and view (x−ri)2(x-r_i)^2 as a bounded factor of order O(Δi2)O( _i^2) on IiI_i; then apply the same second-order Taylor argument in y around sjs_j (as in Lemma 4) to gain an extra factor δj3 _j^3. Summing over i,ji,j yields O(KX−2KY−2)O(K_X^-2K_Y^-2). The term [XeXeY2]E[Xe_Xe_Y^2] is symmetric. (i) [eX2eY2]E[e_X^2e_Y^2]. Similarly, [eX2eY2]=∑i,j∫Ii∫Jj(x−ri)2(y−sj)2fX,Y(x,y)dydx.E[e_X^2e_Y^2]= _i,j _I_i _J_j(x-r_i)^2(y-s_j)^2f_X,Y(x,y)\,dy\,dx. On each rectangle, (x−ri)2=O(Δi2)(x-r_i)^2=O( _i^2) and (y−sj)2=O(δj2)(y-s_j)^2=O( _j^2), and integrating over the rectangle produces a factor Δiδj _i _j. Thus each rectangle contributes O(Δi3δj3)O( _i^3 _j^3) and summing yields (19). (iv) Conclusion. Insert (19)–(22) into (3). Since KX−2KY−2=o(KX−2+KY−2)K_X^-2K_Y^-2=o(K_X^-2+K_Y^-2), the stated expansion follows. ∎ A.5 Conditional weights and conclusion Using Proposition 2 in (3) yields [D2]=[Y2eX2]+[X2eY2]+o(KX−2+KY−2).E[D^2]=E[Y^2e_X^2]+E[X^2e_Y^2]+o(K_X^-2+K_Y^-2). Conditioning on X and Y gives [Y2eX2]=[wX(X)eX2],[X2eY2]=[wY(Y)eY2],E[Y^2e_X^2]=E[w_X(X)e_X^2], [X^2e_Y^2]=E[w_Y(Y)e_Y^2], and combining with Proposition 1 in (1) yields (2.2). Appendix B Proof of the Weighted Scalar High-Rate Theorem We prove the weighted high-rate companding theorem used in Theorem 1, including a converse that rules out a better K−2K^-2 constant for any sequence of K-level scalar quantizers. B.1 Scalar setting Let X have density f on ℝR and let w:ℝ→(0,∞)w:R→(0,∞) be continuous. For a quantizer Q with at most K reproduction points, define the weighted MSE D(Q)≜[w(X)(X−Q(X))2]=∫ℝf(x)w(x)(x−Q(x))2dx.D(Q) [w(X)(X-Q(X))^2 ]= _Rf(x)w(x)(x-Q(x))^2\,dx. Let DK⋆≜inf|range(Q)|≤KD(Q)D_K _ (Q) ≤ KD(Q) and define h(x)≜f(x)w(x)h(x) f(x)w(x). We assume throughout that h is continuous and strictly positive, and that I≜∫ℝh(x)1/3dx<∞.I _Rh(x)^1/3\,dx<∞. (23) B.2 Nearest-neighbor form Lemma 5 (Nearest-neighbor regions). Fix reproduction points r1<⋯<rKr_1<…<r_K. Among all quantizers using this codebook, the minimizer of D(Q)D(Q) assigns each x to the nearest reproduction point in squared error. Hence optimal quantizers can be taken to have interval cells. Proof. We note that by w>0w>0 w(x)(x−Q(x))2 w(x)(x-Q(x))^2 ≥w(x)(x−rix)2 ≥ w(x)(x-r_i_x)^2 where ix≜argmini(x−ri)2i_x _i\(x-r_i)^2\ so that we achieve the minimum D(Q)D(Q) when Q is chosen to assign x to its nearest neighbor among the reproduction points. Furthermore, in one dimension, these intervals Q−1(ri)=[xi−1,xi)Q^-1(r_i)=[x_i-1,x_i) are characterized by xi≜ri+ri+12x_i r_i+r_i+12. ∎ B.3 Achievability: Bennett integral for a fixed point density Fix a continuous point density λ>0λ>0 with ∫ℝλ=1 _Rλ=1 and construct QK,λQ_K,λ. Denote its cells by Ii=[xi−1,xi)I_i=[x_i-1,x_i) with width Δi=xi−xi−1 _i=x_i-x_i-1, midpoint mim_i, and reproduction point rir_i. Lemma 6 (Bennett integral limit for companders). Assume λ is continuously differentiable and ∫ℝh(x)/λ(x)2dx<∞ _Rh(x)/λ(x)^2\,dx<∞. Then limK→∞K2D(QK,λ)=112∫ℝh(x)λ(x)2dx. _K→∞K^2D(Q_K,λ)= 112 _R h(x)λ(x)^2\,dx. (24) Proof. Write D(QK,λ)=∑i=1K∫Iih(x)(x−ri)2dx.D(Q_K,λ)= _i=1^K _I_ih(x)(x-r_i)^2\,dx. Fix M>0M>0 and decompose the sum into cells that intersect [−M,M][-M,M] and cells that do not. We handle these two contributions separately. Step 1: Compact contribution. For a cell IiI_i intersecting [−M,M][-M,M], Lemma 2 gives ∫Ii(x−ri)2dx=Δi312+O(Δi5), _I_i(x-r_i)^2\,dx= _i^312+O( _i^5), uniformly. Since h is uniformly continuous on a slightly enlarged compact interval and Δi→0 _i→ 0 uniformly on that compact region (by Lemma 1), there exists ξi∈Ii _i∈ I_i such that ∫Iih(x)(x−ri)2dx=h(ξi)(Δi312+O(Δi5)). _I_ih(x)(x-r_i)^2\,dx=h( _i) ( _i^312+O( _i^5) ). Multiply by K2K^2 and use Lemma 1 to write Δi=1/(Kλ(ηi)) _i=1/(Kλ( _i)) for some ηi∈Ii _i∈ I_i: K2∫Iih(x)(x−ri)2dx=112h(ξi)λ(ηi)2Δi+O(h(ξi)K2Δi),K^2 _I_ih(x)(x-r_i)^2\,dx= 112 h( _i)λ( _i)^2 _i+O\! ( h( _i)K^2 _i ), because K2Δi5=(K2Δi3)Δi2=O(Δi2)Δi=O(K−2)ΔiK^2 _i^5=(K^2 _i^3) _i^2=O( _i^2) _i=O(K^-2) _i on compacts where Δi=O(1/K) _i=O(1/K). Summing over all cells intersecting [−M,M][-M,M] gives K2∑i:Ii∩[−M,M]≠∅∫Iih(x)(x−ri)2dx=112∑i:Ii∩[−M,M]≠∅h(ξi)λ(ηi)2Δi+oM(1),K^2 _i:\,I_i∩[-M,M]≠ _I_ih(x)(x-r_i)^2\,dx= 112 _i:\,I_i∩[-M,M]≠ h( _i)λ( _i)^2 _i+o_M(1), (25) where oM(1)→0o_M(1)→ 0 as K→∞K→∞ for each fixed M. Because ξi,ηi∈Ii _i, _i∈ I_i and maxi:Ii∩[−M,M]≠∅Δi→0 _i:\,I_i∩[-M,M]≠ _i→ 0, continuity implies h(ξi)λ(ηi)2=h(ζi)λ(ζi)2+o(1) h( _i)λ( _i)^2= h( _i)λ( _i)^2+o(1) for some ζi∈Ii _i∈ I_i. Therefore the right-hand side of (25) is a Riemann sum for ∫−MMh(x)/λ(x)2dx _-M^Mh(x)/λ(x)^2\,dx, and we conclude that limK→∞K2∑i:Ii∩[−M,M]≠∅∫Iih(x)(x−ri)2dx=112∫−MMh(x)λ(x)2dx. _K→∞K^2 _i:\,I_i∩[-M,M]≠ _I_ih(x)(x-r_i)^2\,dx= 112 _-M^M h(x)λ(x)^2\,dx. Step 2: Tail contribution. Since h/λ2h/λ^2 is integrable by assumption, we can choose M large enough such that ∫|x|>Mh(x)λ(x)2dx≤ε. _ x >M h(x)λ(x)^2\,dx≤ . A similar bound (using nonnegativity of the integrand and the same cell-width identity as above) shows that for all K, 0≤K2∑i:Ii∩[−M,M]=∅∫Iih(x)(x−ri)2dx≤112∫|x|>Mh(x)λ(x)2dx+o(1)≤ε12+o(1).0≤ K^2 _i:\,I_i∩[-M,M]= _I_ih(x)(x-r_i)^2\,dx≤ 112 _ x >M h(x)λ(x)^2\,dx+o(1)≤ 12+o(1). Letting ε↓0 0 and then K→∞K→∞ gives that the tail contribution vanishes in the K2K^2-scaled limit. Step 3: Combine and conclude. Combining the compact and tail contributions and then letting M→∞M→∞ yields (24). ∎ B.4 Optimize over λ (Hölder inequality) Given Lemma 6, the leading constant for a fixed point density λ is J(λ)≜∫ℝh(x)λ(x)2dx,subject toλ>0,∫ℝλ=1.J(λ) _R h(x)λ(x)^2\,dx, to λ>0,\ _Rλ=1. Let a(x)≜h(x)1/3a(x) h(x)^1/3. Write a(x)=(a(x)λ(x)2/3)λ(x)2/3.a(x)= ( a(x)λ(x)^2/3 )λ(x)^2/3. Hölder with exponents 33 and 3/23/2 gives ∫ℝa≤(∫ℝa3λ2)1/3(∫ℝλ)2/3=J(λ)1/3. _Ra≤ ( _R a^3λ^2 )^1/3 ( _Rλ )^2/3=J(λ)^1/3. Therefore J(λ)≥(∫a)3=I3J(λ)≥( a)^3=I^3, with equality if and only if λ(x)∝a(x)λ(x) a(x), i.e., λ⋆(x)=h(x)1/3∫ℝh(t)1/3dt=h(x)1/3I.λ (x)= h(x)^1/3 _Rh(t)^1/3\,dt= h(x)^1/3I. (26) Combining with Lemma 6 gives the achievability statement lim supK→∞K2DK⋆≤I312. _K→∞K^2D_K ≤ I^312. (27) B.5 Converse: no smaller K−2K^-2 constant is possible We now prove the matching lower bound lim infK→∞K2DK⋆≥I3/12 _K→∞K^2D_K ≥ I^3/12. Lemma 7 (A lower bound on the distortion over a compact interval). Fix M>0M>0 and define the truncated distortion DM(Q)≜∫−MMh(x)(x−Q(x))2dx.D_M(Q) _-M^Mh(x)(x-Q(x))^2\,dx. Let hmin,M≜infx∈[−M,M]h(x)h_ ,M _x∈[-M,M]h(x), which is strictly positive by continuity. Then for any K-level quantizer Q, DM(Q)≥112∑ℓ=1Lhℓ,min|Jℓ|3,D_M(Q)\ ≥\ 112 _ =1^Lh_ , \, J_ ^3, (28) where Jℓ=1L\J_ \_ =1^L is the partition of [−M,M][-M,M] induced by the quantizer cells (each JℓJ_ is an interval on which Q is constant), L≤KL≤ K, and hℓ,min≜infx∈Jℓh(x)h_ , _x∈ J_ h(x). Proof. On each induced interval JℓJ_ the quantizer output is a constant reproduction point, say Q(x)=rℓQ(x)=r_ . Since h(x)≥hℓ,minh(x)≥ h_ , for x∈Jℓx∈ J_ , ∫Jℓh(x)(x−rℓ)2dx≥hℓ,min∫Jℓ(x−rℓ)2dx. _J_ h(x)(x-r_ )^2\,dx≥ h_ , _J_ (x-r_ )^2\,dx. For an interval of length |Jℓ| J_ , the function r↦∫Jℓ(x−r)2dxr _J_ (x-r)^2\,dx is minimized at the midpoint of JℓJ_ , with minimum value |Jℓ|3/12 J_ ^3/12. Therefore the integral is at least |Jℓ|3/12 J_ ^3/12 for any rℓr_ , giving ∫Jℓh(x)(x−rℓ)2dx≥hℓ,min12|Jℓ|3. _J_ h(x)(x-r_ )^2\,dx≥ h_ , 12 J_ ^3. Summing over ℓ=1,…,L =1,…,L yields (28). ∎ Lemma 8 (Approximating ∫−MMh1/3 _-M^Mh^1/3 by a partition of small mesh). Fix M>0M>0 and let ωM(δ) _M(δ) denote the modulus of continuity of h1/3h^1/3 on [−M,M][-M,M]: ωM(δ)≜supx,y∈[−M,M]|x−y|≤δ|h(x)1/3−h(y)1/3|. _M(δ) _ subarraycx,y∈[-M,M]\\ |x-y|≤δ subarray h(x)^1/3-h(y)^1/3 . Then for any partition Jℓ=1L\J_ \_ =1^L of [−M,M][-M,M] into intervals with maximum length at most δ, ∑ℓ=1Lhℓ,min1/3|Jℓ|≥∫−MMh(x)1/3dx− 2MωM(δ). _ =1^Lh_ , ^1/3\, J_ \ ≥\ _-M^Mh(x)^1/3\,dx\ -\ 2M\, _M(δ). (29) Proof. Fix an interval JℓJ_ . For any x∈Jℓx∈ J_ , we have hℓ,min1/3≥h(x)1/3−ωM(δ)h_ , ^1/3≥ h(x)^1/3- _M(δ) because supt∈Jℓ|h(t)1/3−h(x)1/3|≤ωM(δ) _t∈ J_ h(t)^1/3-h(x)^1/3 ≤ _M(δ) and |Jℓ|≤δ J_ ≤δ. Integrating over x∈Jℓx∈ J_ yields hℓ,min1/3|Jℓ|≥∫Jℓh(x)1/3dx−ωM(δ)|Jℓ|.h_ , ^1/3 J_ ≥ _J_ h(x)^1/3\,dx- _M(δ) J_ . Summing over ℓ and using ∑ℓ|Jℓ|=2M _ J_ =2M gives (29). ∎ Lemma 9 (Compact-set converse constmmant). Fix M>0M>0 and define IM≜∫−MMh(x)1/3dx.I_M _-M^Mh(x)^1/3\,dx. Then lim infK→∞K2DK⋆≥IM312. _K→∞K^2D_K \ ≥\ I_M^312. (30) Proof. Since D(Q)≥DM(Q)D(Q)≥ D_M(Q) for every quantizer Q, we have DK⋆≥inf|range(Q)|≤KDM(Q)D_K ≥ _ (Q) ≤ KD_M(Q), and it is enough to lower bound the latter. Fix δ>0δ>0. Consider any K-level quantizer Q, and let Jℓ=1L\J_ \_ =1^L be the induced partition of [−M,M][-M,M] (as in Lemma 7), with L≤KL≤ K. Case 1: the partition mesh exceeds δ. If maxℓ|Jℓ|>δ _ J_ >δ, then Lemma 7 and hℓ,min≥hmin,Mh_ , ≥ h_ ,M imply DM(Q)≥hmin,M12δ3.D_M(Q)≥ h_ ,M12δ^3. Case 2: the partition mesh is at most δ. If maxℓ|Jℓ|≤δ _ J_ ≤δ, then Lemma 7 gives DM(Q)≥112∑ℓ=1L(hℓ,min1/3|Jℓ|)3.D_M(Q)≥ 112 _ =1^L (h_ , ^1/3 J_ )^3. Apply Hölder to the nonnegative numbers aℓ≜hℓ,min1/3|Jℓ|a_ h_ , ^1/3 J_ : (∑ℓ=1Laℓ)3≤(∑ℓ=1Laℓ3)L2≤(∑ℓ=1Laℓ3)K2, ( _ =1^La_ )^3≤ ( _ =1^La_ ^3 )L^2≤ ( _ =1^La_ ^3 )K^2, so ∑ℓ=1Laℓ3≥(∑ℓ=1Laℓ)3/K2 _ =1^La_ ^3≥( _ =1^La_ )^3/K^2. Therefore DM(Q)≥112K2(∑ℓ=1Lhℓ,min1/3|Jℓ|)3.D_M(Q)≥ 112K^2 ( _ =1^Lh_ , ^1/3 J_ )^3. By Lemma 8, the bracketed term is at least IM−2MωM(δ)I_M-2M _M(δ). Hence DM(Q)≥112K2(IM−2MωM(δ))3.D_M(Q)≥ 112K^2 (I_M-2M _M(δ) )^3. Combine the two cases. We have shown that for every K-level quantizer Q, DM(Q)≥minhmin,M12δ3,112K2(IM−2MωM(δ))3.D_M(Q)≥ \ h_ ,M12δ^3,\ 112K^2 (I_M-2M _M(δ) )^3 \. Taking the infimum over all Q with at most K levels preserves the inequality. Multiply by K2K^2 and let K→∞K→∞. For fixed δ, the second term in the minimum dominates for large K, yielding lim infK→∞K2DK⋆≥112(IM−2MωM(δ))3. _K→∞K^2D_K ≥ 112 (I_M-2M _M(δ) )^3. Finally let δ↓0δ 0. Since h1/3h^1/3 is uniformly continuous on [−M,M][-M,M], ωM(δ)→0 _M(δ)→ 0, and we obtain (30). ∎ Theorem 3 (Weighted scalar high-rate constant and optimal density). Under (23), limK→∞K2DK⋆=112(∫ℝ(f(x)w(x))1/3dx)3. _K→∞K^2D_K = 112 ( _R(f(x)w(x))^1/3\,dx )^3. (31) Moreover, the unique minimizing point density for the Bennett functional is (26), and the sequence of companders QK,λ⋆Q_K,λ achieves the limit in (31). Proof. The achievability bound (27) follows from Lemma 6 and the Hölder minimization. For the converse, Lemma 9 implies that for every M, lim infK→∞K2DK⋆≥IM312,IM=∫−MMh(x)1/3dx. _K→∞K^2D_K ≥ I_M^312, I_M= _-M^Mh(x)^1/3\,dx. Letting M→∞M→∞ and using monotone convergence (since h1/3≥0h^1/3≥ 0 and integrable) gives limM→∞IM=I _M→∞I_M=I, hence lim infK→∞K2DK⋆≥I3/12 _K→∞K^2D_K ≥ I^3/12. Together with (27) this yields (31). The optimality and uniqueness of λ⋆λ follow from the equality condition in Hölder. ∎ Appendix C Proof of Theorem 2 By Theorem 1, λX⋆(x)∝(fX(x)wX(x))1/3 _X (x) (f_X(x)w_X(x))^1/3 and λY⋆(y)∝(fY(y)wY(y))1/3 _Y (y) (f_Y(y)w_Y(y))^1/3. For joint Gaussian (12), the conditional law satisfies Y∣X=x∼(ρσYσXx,σY2(1−ρ2)).Y X=x \! (ρ _Y _Xx,\ _Y^2(1-ρ^2) ). Hence wX(x)=[Y2∣X=x] w_X(x)=E[Y^2 X=x] =Var(Y∣X=x)+([Y∣X=x])2 =Var(Y X=x)+(E[Y X=x])^2 =σY2(1−ρ2)+ρ2σY2σX2x2. = _Y^2(1-ρ^2)+ρ^2 _Y^2 _X^2x^2. (32) Similarly, wY(y)=σX2(1−ρ2)+ρ2σX2σY2y2.w_Y(y)= _X^2(1-ρ^2)+ρ^2 _X^2 _Y^2y^2. (33) The marginal X∼(0,σX2)X (0, _X^2) has density fX(x)∝exp(−x2/(2σX2))f_X(x) (-x^2/(2 _X^2)), so (fX(x)wX(x))1/3∝exp(−x26σX2)((1−ρ2)+ρ2x2σX2)1/3,(f_X(x)w_X(x))^1/3\ \ \! (- x^26 _X^2 ) ((1-ρ^2)+ρ^2 x^2 _X^2 )^1/3, and normalization yields the stated λX⋆ _X . The same reasoning with (33) yields λY⋆ _Y . Appendix D Proof of Theorem 2 Let u=x/σXu=x/ _X. Up to an additive constant, the log-density is ℓ(u)=−u26+13log((1−ρ2)+ρ2u2). (u)=- u^26+ 13 ((1-ρ^2)+ρ^2u^2 ). Differentiate: ℓ′(u)=u[−13+2ρ23((1−ρ2)+ρ2u2)]. (u)=u [- 13+ 2ρ^23((1-ρ^2)+ρ^2u^2) ]. Thus u=0u=0 is always stationary. For u≠0u≠ 0, stationarity requires −13+2ρ23((1−ρ2)+ρ2u2)=0⇔(1−ρ2)+ρ2u2=2ρ2⇔u2=3−1ρ2.- 13+ 2ρ^23((1-ρ^2)+ρ^2u^2)=0 (1-ρ^2)+ρ^2u^2=2ρ^2 u^2=3- 1ρ^2. Real nonzero stationary points exist iff ρ2>1/3ρ^2>1/3, and then they are ±3−1/ρ2± 3-1/ρ^2. The curvature at the origin is ℓ′(0)=3ρ2−13(1−ρ2). (0)= 3ρ^2-13(1-ρ^2). If ρ2<1/3ρ^2<1/3, then ℓ′(0)<0 (0)<0 and, with no other stationary points, the density is unimodal with maximum at 0. If ρ2>1/3ρ^2>1/3, then ℓ′(0)>0 (0)>0 so 0 is a strict local minimum and the only other stationary points are the two symmetric maxima computed above, yielding bimodality. At ρ2=1/3ρ^2=1/3, ℓ′(0)=0 (0)=0 (critical point). Appendix E Closed Form for the Normalizer Integral Define J(ρ)=∫−∞e−u2/6((1−ρ2)+ρ2u2)1/3du.J(ρ)= _-∞^∞e^-u^2/6 ((1-ρ^2)+ρ^2u^2 )^1/3\,du. By even symmetry and the substitution t=u2t=u^2 (so du=12t−1/2dt\,du= 12t^-1/2\,dt), J(ρ)=∫0∞e−t/6((1−ρ2)+ρ2t)1/3t−1/2dt.J(ρ)= _0^∞e^-t/6 ((1-ρ^2)+ρ^2t )^1/3t^-1/2\,dt. (34) Factoring (1−ρ2)(1-ρ^2) and rescaling t yields, for ρ≠0ρ≠ 0, J(ρ)=π(1−ρ2)5/6|ρ|U(12,116,1−ρ26ρ2),J(ρ)= π\,(1-ρ^2)^5/6 ρ \,U\! ( 12, 116, 1-ρ^26ρ^2 ), (35) where U(a,b,z)U(a,b,z) is Tricomi’s confluent hypergeometric function. Also J(0)=6πJ(0)= 6π. References [1] W. R. Bennett (1948) Spectra of quantized signals. Bell System Technical Journal 27 (3), p. 446–472. External Links: Document Cited by: §1. [2] T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer (2022) LLM.int8(): 8-bit matrix multiplication for transformers at scale. External Links: 2208.07339, Link Cited by: §1, §3.3.4, §3.3. [3] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer (2023) Qlora: efficient finetuning of quantized llms. Advances in neural information processing systems 36, p. 10088–10115. Cited by: item 7. [4] A. Gersho and R. M. Gray (2012) Vector quantization and signal compression. Vol. 159, Springer Science & Business Media. Cited by: §1. [5] N. S. Jayant and P. Noll (1984) Digital coding of waveforms: principles and applications to speech and video. Vol. 2, Prentice-Hall Englewood Cliffs, NJ. Cited by: item 5, item 6. [6] S. Lloyd (1982) Least squares quantization in pcm. IEEE transactions on information theory 28 (2), p. 129–137. Cited by: item 3. [7] S. Merity, C. Xiong, J. Bradbury, and R. Socher (2016) Pointer sentinel mixture models. External Links: 1609.07843, Link Cited by: §3.3.1. [8] P. Micikevicius, D. Stosic, N. Burgess, M. Cornea, P. Dubey, R. Grisenthwaite, S. Ha, A. Heinecke, P. Judd, J. Kamalu, N. Mellempudi, S. Oberman, M. Shoeybi, M. Siu, and H. Wu (2022) FP8 formats for deep learning. External Links: 2209.05433, Link Cited by: §1, §3.3. [9] NVIDIA Technical Blog (2025-06-24) Introducing nvfp4 for efficient and accurate low-precision inference. Note: NVIDIA Technical Blog Cited by: §1, item 8. [10] O. Ordentlich and Y. Polyanskiy (2025) Optimal quantization for matrix multiplication. IEEE Transactions on Information Theory. Cited by: §1. [11] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. (2019) Language models are unsupervised multitask learners. OpenAI blog 1 (8), p. 9. Cited by: §3.3.1. [12] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025) Qwen3 technical report. External Links: 2505.09388, Link Cited by: §3.3.1.