Paper deep dive
OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization
Neriah Ben David, Ori Meir, Or Ordentlich
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In this work we develop a novel quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4). For each linear projection it uses a diagonal smoothing matrix whose entries are optimized to minimize the squared matrix-product quantization error under NVFP4, taking into account the rounding procedure that is used (either round-to-nearest, or GPTQ-style successive interference cancellation). This requires performing joint optimization on the smoothing entries as well as the block scales, which is facilitated by analyzing a multiplicative-dither FP4 quantizer instead of the fixed deterministic one. Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94-97\% of vendor NVFP4 prefill throughput on the measured workloads. Our code is available in this https URL
Tags
Links
- Source: https://arxiv.org/abs/2610.08231v1
- Canonical: https://arxiv.org/abs/2610.08231v1
Trouble viewing inline? Open PDF directly →
Full Text
102,181 characters extracted from source content.
Expand or collapse full text
OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization Neriah Ben David Ori Meir & Or Ordentlich †thanks: The authors are listed in alphabetical order. Affiliation: School of Computer Science and Engineering Affiliation: Hebrew University of Jerusalem Affiliation: Jerusalem, Israel Email: neriah.bendavid,ori.meir,or.ordentlich@mail.huji.ac.il Abstract NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In this work we develop a novel quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4). For each linear projection it uses a diagonal smoothing matrix whose entries are optimized to minimize the squared matrix-product quantization error under NVFP4, taking into account the rounding procedure that is used (either round-to-nearest, or GPTQ-style successive interference cancellation). This requires performing joint optimization on the smoothing entries as well as the block scales, which is facilitated by analyzing a multiplicative-dither FP4 quantizer instead of the fixed deterministic one. Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94-97% of vendor NVFP4 prefill throughput on the measured workloads. Our code is available here. 1 Introduction Low-bit data formats are gaining much popularity for large language models (LLMs) inference, as they require less memory for storing and running the model, decrease the traffic from and to memory, and allow to execute matrix multiplication (MatMul) faster through optimized hardware. The FP4 datatype stands out as a prominent representative of such formats which is heavily used in models running on NVIDIA’s Blackwell series and its subsequent generations. Unfortunately, using FP4 instead of higher resolution formats such as FP8 often comes with the cost of reduced accuracy. The goal of this paper is to develop a quantization scheme for NVFP4 that jointly optimizes a diagonal smoothing transformation, as well as the weight block scales in order to maintain maximal accuracy. At first thought, it is not at all clear that diagonal smoothing is useful for FP4 quantization. Traditionally, the main differentiator between INT constellations and FP constellations was the dynamic range. An INT-R constellation has fixed spacing between constellation points, and its dynamic range is therefore proportional to 2R2^R. Absmax scaling is often used to make sure the input fits within the dynamic range, and the corresponding per-entry mean-squared error (MSE) in representing x∈ℝnx ^n in INT-R is consequently proportional to ‖x‖∞2⋅2−2R\|x\|^2_∞· 2^-2R Ordentlich and Polyanskiy (2026a). For this reason, one of the key aspects in INT-based quantization techniques is controlling the infinity norm of the quantizer’s input. Diagonal scaling can help balance the infinity norms of vectors participating in MatMul, and indeed one of the simplest and most efficient techniques for INT-quantized MatMul is SmoothQuant Xiao et al. (2023). For FP constellations with E exponent bits, the dynamic range is proportional to 22E2^2^E, and can be made “essentially unbounded” with as few as E=4E=4 exponent bits, as in the standard E4M3 FP8 datatype. The MSE in representing the iith entry xix_i of x∈ℝnx ^n is therefore proportional to |xi|2⋅2−2M|x_i|^2· 2^-2M, where M is the number of mantissa bits, provided that xix_i is inside the constellation’s dynamic range Ordentlich and Polyanskiy (2026a). For this reason, quantization to FP is typically considered simpler, and requires less pre and post processing than quantization to INT constellations. In particular, whenever E is large enough so that dynamic range is not a limiting factor, SmoothQuant-style diagonal scaling is quite useless, as [(QFP(Xi⋅α)QFP(Yi/α)−XiYi)2]E[(Q_FP(X_i·α)Q_FP(Y_i/α)-X_iY_i)^2] is proportional to [|Xi|2|Yi|2]2−2ME[|X_i|^2|Y_i|^2]2^-2M for all α>0α>0 provided that the distribution of (Xi,Yi)(X_i,Y_i) is “sufficiently smooth” and Xi⋅αX_i·α and Yi/αY_i/α are within the quantizer’s dynamic range with high probability. The picture is different for the FP4 E2M1 format since with E=2E=2 the dynamic range is too small to be ignored. To combat the effect of limited dynamic range, FP4 is usually applied with microscaling as in MXFP4 or NVFP4, where a group of g entries are also given a common scale γ. In MXFP4 g=32g=32 and γ is represented in E8M0 format, whereas in NVFP4 g=16g=16 and γ is represented in E4M3 format. Here we focus on NVFP4, which is becoming a common choice for models running on NVIDIA hardware, though some of the techniques we develop may also be useful for MXFP4 as well. The scale γ can be used to ensure that the input to the FP4 quantizer never exceeds 66, which is the largest value in the FP4 constellation. In particular, for x∈ℝ16x ^16 the default absmax-to-6 rule sets γ≈‖x‖∞/6γ≈\|x\|_∞/6 such that ‖γ−1x‖∞≈6\|γ^-1x\|_∞≈ 6. This rule takes care of the upper limit of the dynamic range: it ensures that no entry of γ−1xγ^-1x is large enough to saturate the quantizer. However, with this rule all entries with magnitude 2424 times smaller than ‖x‖∞\|x\|_∞ will be quantized to 0, so that the total distortion can be quite large when x is largely unbalanced. Furthermore, even for balanced vectors, there are multiple choices of γ for which all entries of γ−1xγ^-1x are within the dynamic range of the FP4 constellation, and careful optimization of γ may yield significant reduction of MSE over the absmax-to-6 rule. This work develops an NVFP4 quantization method that utilizes a diagonal matrix A∈ℝn×nA ^n× n for “smoothing” the weight matrix W∈ℝm×nW ^m× n to WA−1WA^-1 and the activation matrix X∈ℝn×kX ^n× k to AXAX, as in SmoothQuant. The clear advantage of diagonal A with respect to more general structures for A, e.g. block-rotation Egiazarian et al. (2026), PCA directions aligning Chen et al. (2025), tensor of dense small matrices Sun et al. (2025), is that diagonal pre-processing is cheap and can easily be done in runtime for the activations. In fact, in some layers it can be fused into the LayerNorm or RMSnorm. In contrast to SmoothQuant where A is used for balancing infinity norms, the role of the matrix A here is more intricate. It should balance the distribution of all entries within the same groups (1616 consecutive entries) in the rows of W and in the columns of X, so that the corresponding E4M3 scales can then be used to minimize the MSE. Developing a mathematical formulation that accurately captures this objective is one of the main challenges addressed in this work. This is achieved by analyzing a randomized multiplicatively dithered FP4 quantizer instead of the deterministic one, which is actually used by our scheme. The distortion incurred by the randomized quantizer admits a relatively simple form using the function ϕ(⋅)φ(·) introduced below (See Figure 1), and is closely related to the distortion incurred by the deterministic quantizer (Lemma 1). This in turn enables to jointly optimize the smoothing weight matrix and the block scales. Using the fixed weights and calibration activations, we optimize the diagonal entries of A and the block scales offline in two stages. In the first stage we jointly optimize the diagonal entries of A and the block scales with respect to a dithered FP4 quantizer, whose corresponding MSE loss is smooth and amenable to optimization. Crucially, the loss takes into account whether round-to-nearest (RTN) or successive-interference-cancellation (SIC) will be used for the actual weight quantization. The resulting A is retained, while the weight scales are finely optimized over the E4M3 grid in the second stage. Activation scales must instead be determined in real time; our default is absmax-to-6. Our main contributions are: • We develop an expression that approximates the expected squared Frobenius norm of the matrix-product error under randomized FP4 quantization for diagonally smoothed NVFP4 quantization, as a function of the smoothing parameters and the group scales. This expression takes into account the rounding procedure (RTN/SIC) for W, and is amenable to efficient joint optimization of all participating parameters. • Relying on this approximation, we develop a quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4) quantization. This scheme uses the approximation to perform joint continuous optimization of diagonal smoothing and block scales, followed by discrete E4M3 weight-scale selection. The optimization criterion for the discrete weight-scale depends on the calibration second-moments as well as on the rounding method (RTN/SIC). The activation group scales are set using absmax-to-6. • Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94-97% of vendor NVFP4 prefill throughput on the measured workloads. • We provide an add-on package for LLM Compressor AI and vLLM Project (2024) that implements our OSFP4 compression method and exports quantized models as vLLM checkpoints, together with a vLLM plugin that support the additional diagonal smoothing layers. 1.1 Related Work Calibration and error compensation. Post-training quantization uses calibration data to preserve layer outputs without retraining the original model. GPTQ (Frantar et al., 2023) uses second-order information to compensate quantization errors through updates to unquantized weights. This method has demonstrated empirical benefits for various quantization codebooks, including for FP and in particular NVFP4 constellations Egiazarian et al. (2026). Diagonal scaling has proved useful for INT constellations: SmoothQuant (Xiao et al., 2023) balances the quantization difficulty of weights and activations by redistributing their magnitudes, while AWQ (Lin et al., 2024) uses activation statistics to scale and protect important weight channels. OmniQuant (Shao et al., 2024) learns such scalings as part of a broader family of equivalent transformations, jointly with clipping parameters, through reconstruction objectives. These methods establish the value of calibration, scaling, and feedback. WaterSIC (Lifar et al., 2026) combines all three and finds a diagonal smoothing matrix which, when combined with GPTQ/SIC, obtains near-information-theoretic optimal weight-only entropy coded integer quantization. Another recent work (Avisdris et al., 2027) addresses optimal diagonal smoothing for weights and activations INT quantization using GPTQ is. It turns out that NVFP4 requires a markedly different criterion for choosing the diagonal smoothing matrix. OSFP4 jointly adapts smoothing and block scales to the NVFP4 grid and the chosen rounding procedure. Transforms for low-bit inference. Prior work demonstrated the value of randomized rotations for low-bit quantization (Chee et al., 2023; Ashkboos et al., 2024; Liu et al., 2025). For NVFP4, however, full-dimensional rotations were found less beneficial and can even hurt RTN accuracy, whereas MR-GPTQ (Egiazarian et al., 2026) benefits from rotations within quantization groups together with format-specific GPTQ optimizations. OSFP4 avoids rotations and instead uses cheaper diagonal transformations, jointly optimized (offline) with the block scales. Despite this simpler transformation, our W-SIC/X-RTN configuration achieves higher average W4A4 accuracy than MR-GPTQ under the common evaluation protocol in Table 1. NVFP4 scale selection. Recent work shows that absmax scaling leaves substantial room for improvement. Four Over Six (4/6) (Cook et al., 2026) adaptively selects between absmax-to-4 and absmax-to-6 scaling for each block. ScaleSearch (Gupta et al., 2026) searches block scales to reduce reconstruction error, and ScaleSweep (Lin and Wan, 2026) derives bounded search ranges for MSE and weighted MSE objectives. SOAR (Bao et al., 2026) combines joint global/block-scale optimization with decoupled encoding and decoding scales. H-Scale (Yu et al., 2026) uses calibration activation second moments to select E4M3 block scales by minimizing a weighted-mean-squared weight reconstruction error. OSFP4 combines accurate scale selection with diagonal smoothing, jointly optimizing both while accounting for the applied rounding procedure (RTN/SIC). Online activation-scale search is complementary to this procedure; our default uses absmax activation scaling. Quantization-error models. Our analysis builds on high-rate models of floating-point error and covariance-aware quantized matrix multiplication (Ordentlich and Polyanskiy, 2026a; Ordentlich and Polyanskiy, 2026b). We use multiplicative randomization to obtain a smooth FP4 error function that retains finite-range effects, and derive distinct objectives for RTN and SIC, with or without activation quantization. This makes the intended feedback procedure part of the smoothing optimization and connects the randomized loss to the subsequent deterministic scale search. 2 Loss Functions 2.1 Smoothed FP4 quantization error Denote by FP4=0,±12,±1,±32,±2,±3,±4,±6 _FP4= \0,± 12,± 1,± 32,± 2,± 3,± 4,± 6 \ (1) the points in the FP4 E2M1 constellation, and for x∈ℝx let QFP4(x)=argminx^∈FP4|x−x^| Q_FP4(x)= _ x _FP4|x- x| (2) be the deterministic nearest-neighbor quantizer of x to FP4. Inspired by the dithered absmax FP quantizer from Ordentlich and Polyanskiy (2026a) we consider the following randomized FP4 quantizer Q~FP4(x)=2UQFP4(2−Ux), Q_FP4(x)=2^UQ_FP4 (2^-Ux ), (3) where U∼Uniform([0,1))U ([0,1)) is statistically independent of x, and its corresponding normalized MSE function is ϕ(x)=1x2(x−Q~FP4(x))2. φ(x)= 1x^2E (x- Q_FP4(x) )^2. (4) We set ϕ(0)=1φ(0)=1 by continuous extension. The following lemma provides two key properties of Q~FP4(x) Q_FP4(x) and ϕ(x)φ(x). Lemma 1 1. For any x∈ℝnx ^n there exists a fixed β∈[1,2)β∈[1,2) such that ‖x−β⋅QFP4(β−1x)‖2≤∑i=1nxi2ϕ(xi). \|x-β· Q_FP4(β^-1x)\|^2≤ _i=1^nx_i^2φ(x_i). (5) 2. ϕ(x)φ(x) is an even function. Its minimum is ϕmin=[(1−2−(U+1)round(2U+1))2]≈0.010723 _min=E[(1-2^-(U+1)round(2^U+1))^2]≈ 0.010723 and it attains this minimum on the entire interval ℐFP4=[7/4,7]I_FP4=[7/4,7]. A proof is given in Appendix A. Figure 1 plots ϕ(x)φ(x) as well as the non-smoothed ϕexact(x)=1x2(x−QFP4(x))2 _exact(x)= 1x^2E (x-Q_FP4(x) )^2. The plateau in ℐFP4I_FP4 is visibly clear. Figure 1: Normalized mean-squared error of randomized FP4 quantization, ϕ(x)=[(x−Q~FP4(x))2]x2φ(x)= E[(x- Q_FP4(x))^2]x^2, and its deterministic counterpart, ϕexact(x)=(x−QFP4(x))2x2 _exact(x)= (x-Q_FP4(x))^2x^2. 2.2 Matrix-product error under RTN and SIC Let W∈ℝm×nW ^m× n and X∈ℝn×kX ^n× k, where the columns of X are calibration activations, and define ΣX=XX⊤/k _X=X /k. We partition the inner dimension into L=n/16L=n/16 groups: Wℓ=W(:,1+16(ℓ−1):16ℓ)W_ =W(:,1+16( -1):16 ) and Xℓ=X(1+16(ℓ−1):16ℓ,:)X_ =X(1+16( -1):16 ,:), with Aℓ=diag(αℓ(1),…,αℓ(16))A_ = diag( _ ^(1),…, _ ^(16)). Write ΓWℓ=γWℓ(i)i=1m _W_ = \ _W_ ^(i) \_i=1^m and ΓXℓ=γXℓ(j)j=1k _X_ = \ _X_ ^(j) \_j=1^k for the row and column scales. The matrices entering the FP4 quantizers are Wℓ′=diag(ΓWℓ)−1WℓAℓ−1,Xℓ′=AℓXℓdiag(ΓXℓ)−1. W _ = diag( _W_ )^-1W_ A_ ^-1, X _ =A_ X_ diag( _X_ )^-1. (6) For either deterministic or randomized quantization, let the dequantized matrices W^ℓ W_ and X^ℓ X_ approximate WℓAℓ−1W_ A_ ^-1 and AℓXℓA_ X_ , respectively. Denote their errors by eWℓe_W_ and eXℓe_X_ and concatenate them into eWe_W and eXe_X. The matrix-product error is e=W^X^−WX=eWAX+WA−1eX+eWeX≈eWAX+WA−1eX. e= W X-WX=e_WAX+WA^-1e_X+e_We_X≈ e_WAX+WA^-1e_X. (7) Here we neglect the second-order term eWeXe_We_X. To construct the optimization loss, we now analyze the randomized quantizer Q~FP4 Q_FP4 from (3); the final quantization procedure, however, uses the deterministic quantizer QFP4Q_FP4, as described in Section 3. We will optimize A,ΓW,ΓXA, _W, _X in order to optimize the distortion of the randomized quantizer, then fix the obtained A, and afterwards refine ΓW _W with respect to the deterministic quantizer. Since the randomized quantizer’s relative MSE ϕφ is minimized on ℐFP4I_FP4, the rough goal is to choose A,ΓW,ΓXA, _W, _X so that as many entries as possible of the matrices in (6) have magnitudes in this interval. In accordance with Lemma 1, we model the randomized FP4 errors as Q~FP4(y)=y+y2ϕ(y)Z, Q_FP4(y)=y+ y^2φ(y)Z, (8) where Z has zero mean and unit variance, with independent noise across entries and blocks. These are modeling assumptions, rather than exact unbiasedness and independence properties of the quantization errors. The complete derivations are given in Appendix I. Round to nearest (RTN) quantization: To analyze RTN, we apply the randomized quantizer to the scaled signal without feedback, and multiply the output by the inverse scaling. Specifically, given A,ΓWA, _W under RTN quantization the entries of the weight matrix W are quantized to W^ℓFP4=Q~FP4(diag(ΓWℓ)−1WℓAℓ−1),so thatW^ℓ=diag(ΓWℓ)Q~FP4(diag(ΓWℓ)−1WℓAℓ−1). W_ ^FP4= Q_FP4 ( diag( _W_ )^-1W_ A^-1_ ),~~so that~~ W_ = diag( _W_ ) Q_FP4 ( diag( _W_ )^-1W_ A^-1_ ). (9) With the block scales γWℓ(i)i=1m \γ^(i)_W_ \_i=1^m and smoothing coefficients αℓ(r)r=116 \ _ ^(r) \_r=1^16 defined above, we obtain eWℓ(i,r)=W^ℓ(i,r)−Wℓ(i,r)αℓ(r)=(Wℓ(i,r)αℓ(r))2ϕ(Wℓ(i,r)γWℓ(i)αℓ(r))Zℓ(i,r), e_W_ (i,r)= W_ (i,r)- W_ (i,r)α^(r)_ = ( W_ (i,r)α^(r)_ )^2φ ( W_ (i,r) _W_ ^(i)α^(r)_ )Z_ (i,r), (10) where Zℓ(i,r)\Z_ (i,r)\ are iid zero-mean, unit-variance random variables. Let LWℓRTN(ΓWℓ,Aℓ)=‖eWℓAℓXℓ‖F2, L^RTN_W_ ( _W_ ,A_ )=E\|e_W_ A_ X_ \|_F^2, (11) where Xℓ,Wℓ,ΓWℓ,AℓX_ ,W_ , _W_ ,A_ are assumed fixed and the expectation is only with respect to the randomization of Q~FP4(⋅) Q_FP4(·), that is, with respect to the iid random variables Zℓ(i,r)\Z_ (i,r)\. It immediately follows that LWℓRTN(ΓWℓ,Aℓ) L^RTN_W_ ( _W_ ,A_ ) =∑j=1k∑i=1m∑r=116Wℓ2(i,r)Xℓ2(r,j)⋅ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)) = _j=1^k _i=1^m _r=1^16W^2_ (i,r)X^2_ (r,j)·φ ( W_ (i,r) _W_ ^(i)α^(r)_ ) =k∑r=116σXℓ2(r)∑i=1mWℓ2(i,r)⋅ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)), =k _r=1^16σ^2_X_ (r) _i=1^mW^2_ (i,r)·φ ( W_ (i,r) _W_ ^(i)α^(r)_ ), (12) where σXℓ2(r)=1k∑j=1kXℓ2(r,j)σ^2_X_ (r)= 1k _j=1^kX^2_ (r,j) (we also have σXℓ2(r)=ΣX(r+16(ℓ−1),r+16(ℓ−1))σ^2_X_ (r)= _X(r+16( -1),r+16( -1))). We assume that the random variables Zℓ(⋅,⋅)\Z_ (·,·)\ corresponding to different blocks are also independent. This implies LWRTN=‖eWAX‖F2=∑ℓ=1LLWℓRTN(ΓWℓ,Aℓ). L^RTN_W=E\|e_WAX\|_F^2= _ =1^LL^RTN_W_ ( _W_ ,A_ ). (13) If activations are also quantized, applying RTN with the randomized FP4 quantizer gives X^ℓFP4=Q~FP4(AℓXℓdiag(ΓXℓ)−1),so thatX^ℓ=Q~FP4(AℓXℓdiag(ΓXℓ)−1)diag(ΓXℓ), X_ ^FP4= Q_FP4 (A_ X_ diag( _X_ )^-1 ),~~so that~~ X_ = Q_FP4 (A_ X_ diag( _X_ )^-1 ) diag( _X_ ), (14) and eXℓ(r,j)=X^ℓ(r,j)−αℓ(r)Xℓ(r,j)=(αℓ(r)Xℓ(r,j))2ϕ(αℓ(r)γXℓ(j)Xℓ(r,j))Z~ℓ(r,j), e_X_ (r,j)= X_ (r,j)-α^(r)_ X_ (r,j)= (α^(r)_ X_ (r,j) )^2φ ( α^(r)_ _X_ ^(j)X_ (r,j) ) Z_ (r,j), (15) where γXℓ(j)j=1k \γ^(j)_X_ \_j=1^k denote the activation block scales, and Z~ℓ(r,j)\ Z_ (r,j)\ are iid zero-mean unit-variance random variables statistically independent of everything. Similar to the derivation above, we obtain LXℓRTN(ΓXℓ,Aℓ)=‖WℓAℓ−1eXℓ‖F2=m∑r=116σWℓ2(r)∑j=1kXℓ2(r,j)⋅ϕ(αℓ(r)γXℓ(j)Xℓ(r,j)), L^RTN_X_ ( _X_ ,A_ )=E\|W_ A^-1_ e_X_ \|_F^2=m _r=1^16σ^2_W_ (r) _j=1^kX^2_ (r,j)·φ ( α^(r)_ _X_ ^(j)X_ (r,j) ), (16) where σWℓ2(r)=1m∑i=1mWℓ2(i,r)σ^2_W_ (r)= 1m _i=1^mW^2_ (i,r). We also have LXRTN=‖WA−1eX‖F2=∑ℓ=1LLXℓRTN(ΓXℓ,Aℓ). L^RTN_X=E\|WA^-1e_X\|_F^2= _ =1^LL^RTN_X_ ( _X_ ,A_ ). (17) Successive interference cancellation (SIC): Note that ‖eWAX‖F2=tr((eWAX)(eWAX)⊤)=ktr(eWAΣXA⊤eW⊤). \|e_WAX\|_F^2= tr ((e_WAX)(e_WAX) )=k tr (e_WA _XA e_W ). (18) Let U∈ℝn×nU ^n× n be an upper triangular matrix such that ΣX=U⊤U _X=U U, and denote Y=UA⊤(WA−1)⊤=UW⊤∈ℝn×m. Y=UA (WA^-1) =UW ^n× m. (19) We have ‖eWAX‖F2=k‖UA⊤eW⊤‖F2=k‖Y−UAW^⊤‖F2. \|e_WAX\|_F^2=k\|UA e_W \|_F^2=k\|Y-UA W \|_F^2. (20) Even when A and the E4M3 scaling matrices ΓWℓ=1L\ _W_ \_ =1^L are fixed, the problem of choosing W^FP4∈FP4m×n W^FP4 _FP4^m× n such as to minimize (20) is computationally difficult due to the coupling the matrix UAUA introduces between different columns of W W (equivalently, rows of W^⊤ W ). As common in the literature, instead of attempting an optimal solution, we resort to the SIC/GPTQ approach. Taking into account the structure of the NVFP4 constellation, this boils down to quantizing the weight columns successively, starting from the last, and feeding the error back to the remaining columns, as in Algorithm 2 in Appendix I. The analysis of LWSIC=k‖Y−UAW^⊤‖F2L_W^SIC=kE\|Y-UA W \|_F^2 under the SIC algorithm is a bit complicated, even under the randomized FP4 quantization error model (8). The reason is that Z entries from time t affect Y(t′)Y^(t ) for t′<t <t due to the feedback nature of the quantization process. We therefore make a high-resolution quantization assumption and assume the energy of the (filtered) quantization noise is negligible next to the signal (entries of Y) energy. Under this assumption we have that Y(t)(t,:)≈U(t,t)⋅(W(:,t))⊤Y^(t)(t,:)≈ U(t,t)·(W(:,t)) . Under this additional approximation, LWℓSIC(ΓWℓ,Aℓ)=k∑r=116Uℓ2(r)∑i=1mWℓ2(i,r)ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)), L^SIC_W_ ( _W_ ,A_ )=k _r=1^16U_ ^2(r) _i=1^mW_ ^2(i,r)φ ( W_ (i,r) _W_ ^(i) _ ^(r) ), (21) where Uℓ(r)=U(16(ℓ−1)+r,16(ℓ−1)+r)U_ (r)=U(16( -1)+r,16( -1)+r). The only difference from LWℓRTN(ΓWℓ,Aℓ)L^RTN_W_ ( _W_ ,A_ ) is replacing σXℓ2(r)σ^2_X_ (r) by Uℓ2(r)U_ ^2(r). The latter is the variance of the optimal prediction error of coordinate t=16(ℓ−1)+rt=16( -1)+r from coordinates t+1:nt+1:n of a Gaussian vector with covariance ΣX _X, and hence Uℓ2(r)≤σXℓ2(r)U_ ^2(r)≤σ^2_X_ (r). This distinction allows us to optimize A taking into account that SIC will be used for the actual quantization. Total quadratic Distortion: Using (12) (16) and (21) we obtain the following loss functions for randomized NVFP4 quantization, as a function of A and the E4M3 scales, under the various quantization setups: LW−RTN L^W-RTN =∑ℓ=1LLWℓRTN(ΓWℓ,Aℓ) = _ =1^LL^RTN_W_ ( _W_ ,A_ ) (22) LW−SIC L^W-SIC =∑ℓ=1LLWℓSIC(ΓWℓ,Aℓ) = _ =1^LL^SIC_W_ ( _W_ ,A_ ) (23) LW−RTN,X−RTN L^W-RTN,X-RTN =∑ℓ=1L[LWℓRTN(ΓWℓ,Aℓ)+LXℓRTN(ΓXℓ,Aℓ)] = _ =1^L [L^RTN_W_ ( _W_ ,A_ )+L^RTN_X_ ( _X_ ,A_ ) ] (24) LW−SIC,X−RTN L^W-SIC,X-RTN =∑ℓ=1L[LWℓSIC(ΓWℓ,Aℓ)+LXℓRTN(ΓXℓ,Aℓ)]. = _ =1^L [L^SIC_W_ ( _W_ ,A_ )+L^RTN_X_ ( _X_ ,A_ ) ]. (25) Note that in all expressions the dependency on A and the E4M3 scales completely decouples to L different functions, that depend on different parameters. Therefore, the optimization on Aℓ,ΓWℓ,ΓXℓ\A_ , _W_ , _X_ \ decouples to L independent optimization problems. 3 The OSFP4 Quantization Algorithm We are now ready to describe how the Optimized Smoothing and Scaling for NVFP4 (OSFP4) quantization algorithm works. This algorithm has four different configurations: W−RTN,W−SIC,W−RTN/X−RTNW-RTN,W-SIC,W-RTN/X-RTN or W−SIC/X−RTNW-SIC/X-RTN, corresponding to either W4A16 or W4A4, and to whether W is quantized using RTN or using SIC. The parameters for each linear projection WXWX are optimized independently. We describe the procedure for a single projection operation. Our quantization pipeline is as follows. Calibration set: For all 44 configurations we first collect k1k_1 activation vectors x1,…,xk1x_1,…,x_k_1 and set ΣX=1k1∑j=1k1xjxj⊤ _X= 1k_1 _j=1^k_1x_jx_j . Since computing ΣX _X does not require many resources even when k1k_1 is large, we typically use large k1k_1, as is standard in calibration based quantization. The terms LW−RTN,LW−SICL^W-RTN,L^W-SIC depend on the calibration set only through ΣX _X, and for weight-only quantization we therefore keep only ΣX _X and set k=k1k=k_1. In the setups involving activations quantization, the term LX−RTNL^X-RTN also participates in the loss function, and the complexity of optimizing this term grows with k. Even though the optimization is performed offline, we will limit k to modest values, such that the quantization procedure will not require too many resources. We therefore take k<k1k<k_1 and select a random subset =j1,…,jkJ=\j_1,…,j_k\ of k calibration vectors and set X=[xj1|⋯|xjk]∈ℝn×kX=[x_j_1|·s|x_j_k] ^n× k. Optimization of A (and initial guess for ΓW) _W): Depending on the configuration W−RTN,W−SIC,W−RTN/X−RTNW-RTN,W-SIC,W-RTN/X-RTN or W−SIC/X−RTNW-SIC/X-RTN we construct the relevant loss function from (22)-(25) (note that the first two depend only on ΣX _X, whereas the last two also depend on X). Then, for every ℓ=1,…,L =1,…,L we perform joint optimization on Aℓ,ΓWℓA_ , _W_ and also ΓXℓ _X_ whenever relevant. As described above, all loss functions (22)-(25) decouple to L independent terms, so optimizing for each ℓ separately is optimal. The optimization is iterative and is initialized with A=InA=I_n and the ΓW,ΓX _W, _X corresponding to absmax-to-6 scaling. The precise details of the optimization iterations are given in Appendix B. To illustrate the benefit of this optimization over the standard initialization values, Figure 3 in Appendix B shows how much LW−SIC,X−RTNL^W-SIC,X-RTN improved from its initial value along the different layers of Llama-3.1-8B-Instruct. The value of A found by optimization is the final value used for quantization. The value found for ΓW _W will be used to dictate the search space for the next step. Even in the setups W−RTN/X−RTNW-RTN/X-RTN or W−SIC/X−RTNW-SIC/X-RTN The value of ΓX _X is irrelevant for the actual quantization of the activations, and will be determined in real-time based on the values of the actual activations computed in inference. Optimization of ΓW _W: The value of ΓW _W found in the previous step was optimized assuming the randomized quantizer Q~FP4 Q_FP4 is used, whose output is generally not in FP4C_FP4. Since in practice we are restricted to output W^FP4∈FP4m×n W^FP4 _FP4^m× n, the randomized quantizer is not applicable, and we should use QFP4(⋅)Q_FP4(·) instead. Nevertheless, using part 1 of Lemma 1, for any fixed y∈ℝ16y ^16 and γWℓ(i) _W_ ^(i) there must be some βWℓ(i)∈[1,2) _W_ ^(i)∈[1,2) such that the distortion for the deterministic γWℓ(i)βWℓ(i)QFP4(yγWℓ(i)βWℓ(i)) _W_ ^(i) _W_ ^(i)Q_FP4 ( y _W_ ^(i) _W_ ^(i) ) is no larger than the expected distortion of the randomized quantizer γWℓ(i)Q~FP4(yγWℓ(i)) _W_ ^(i) Q_FP4 ( y _W_ ^(i) ). In other words, the effective deterministic scale γ′=γWℓ(i)βWℓ(i)γ = _W_ ^(i) _W_ ^(i) lies in [γWℓ(i),2γWℓ(i)][ _W_ ^(i),2 _W_ ^(i)], and the deterministic quantizer γ′QFP4(yγ′)γ Q_FP4 ( yγ ) attains no larger distortion than the expected distortion of the randomized quantizer γWℓ(i)Q~FP4(yγWℓ(i)) _W_ ^(i) Q_FP4 ( y _W_ ^(i) ). In fact, the distortion for the optimal γ′γ may be substantially smaller than that of the randomized quantizer, as it can always place at least one of the entries of y on a point in FP4C_FP4, cancelling its quantization error completely. Consequently, for each ℓ=1,…,L =1,…,L and i=1,…,mi=1,…,m we determine the final scale by trying all valid scale values in an interval containing [γWℓ(i),2γWℓ(i)][ _W_ ^(i),2 _W_ ^(i)]. We can only use scales in scaleC_scale, the set of positive E4M3 numbers. Consequently, not all values in [γWℓ(i),2γWℓ(i)][ _W_ ^(i),2 _W_ ^(i)] can be used, and we therefore choose the best scale in scale+(γWℓ(i))=scale∩[γWℓ(i)1.2,γWℓ(i)0.3]C_scale^+( _W_ ^(i))=C_scale∩ [ _W_ ^(i)1.2, _W_ ^(i)0.3 ]. The fine-grained scale-selection procedure is described in Appendix C. Projection-level comparisons of the predicted loss, randomized quantization, and final deterministic scale selection for all four configurations are given in Appendix E. Optimization of ΓX _X: We first need to multiply the activation X by the diagonal matrix A. This is handled in the LayerNorm whenever it exists, or via an additional diagonal scaling layer when there is no LayerNorm. Under W−RTNW-RTN and W−SICW-SIC the activation X is not quantized, whereas under W−RTN/X−RTNW-RTN/X-RTN and W−SIC/X−RTNW-SIC/X-RTN the activations also need to be quantized. Unlike the quantization of W which is done offline, the quantization of X is done in inference time and should therefore be sufficiently fast. Our default is to compute a global scale for X based on the calibration set, and then quantize each 1616 consecutive entries using absmax-to-6. The fixed global scale allows to perform this quantization online, even before the entire tensor of activations was computed. This is the default choice in our experimental results. We nevertheless mention that recent work Gupta et al. (2026); Lin and Wan (2026) advocates searching a small bank of E4M3 values around the absmax-to-6 value even for the activations, and shows that the overhead this search incurs on the quantization time is quite reasonable (Gupta et al., 2026, Table 6). This optimization is referred to as ScaleSearch/ScaleSweep. 4 Experimental Results We evaluate whether OSFP4 improves NVFP4 accuracy, how its components contribute, and whether these gains retain the efficiency of native FP4 inference. We focus here on Llama-3.1-8B-Instruct; Appendix F contains results for additional models. Locally prepared models share the same source checkpoint and calibration set; all models are evaluated through vLLM on an RTX 5090. We report W4A4 and W4A16 separately and distinguish absmax activation scaling from methods that search activation scales during inference, which typically incur larger overhead. Full task results, calibration settings, and implementation details are given in Appendix D. Accuracy under a common evaluation protocol. Table 1 summarizes the mean accuracy on MMLU-CoT, GSM8K, HellaSwag, and WinoGrande. With absmax activation scaling, OSFP4 W-SIC/X-RTN achieves 77.03%77.03\%, compared with 76.34%76.34\% for FP-Quant GPTQ and 76.18%76.18\% for MR-GPTQ, and has the highest average among the evaluated methods in this group. The weight-only configuration reaches 78.42%78.42\%, versus 78.00%78.00\% for H-Scale and 79.22%79.22\% for BF16. Thus, the accuracy improvement is present both with and without activation quantization. Online activation-scale search is complementary to OSFP4, and was evaluated mainly for enabling a fair comparison with ScaleSweep Lin and Wan (2026). Note that even with absmax activation scaling OSFP4 W-SIC/X-RTN scores better than ScaleSweepMSE+GPTQ, whereas the latter does optimize the activation scale online. Method Avg. BF16 baseline BF16 79.22 W4A4: absmax activation scaling RTN (FP-Quant) 75.65 GPTQ (FP-Quant) 76.34 MR-GPTQ (FP-Quant) 76.18 SOAR 76.60 H-Scale (W4A4 extension) 76.67 NVIDIA released checkpoint† 76.11 OSFP4 W-RTN / X-RTN 76.36 OSFP4 W-SIC / X-RTN 77.03 W4A4: online activation-scale search ScaleSweepMSE 76.62 ScaleSweepMSE + GPTQ 76.69 4over6 76.34 OSFP4 W-RTN / X-ScaleSweepMSE 76.96 OSFP4 W-SIC / X-ScaleSweepMSE 77.09 W4A16: weights only RTN 77.67 H-Scale 78.00 OSFP4 W-RTN 78.15 OSFP4 W-SIC 78.42 †Separately released checkpoint; preparation is uncontrolled. Bold: best quantized average in each group. Table 1: Mean accuracy (%) on Llama-3.1-8B-Instruct. Full scores: Table 3. Table 2: WikiText-2 perplexity (↓ ). W4A4 uses absmax activation scales. Configuration W4A16 W4A4 BF16 reference 7.22 7.22 Absmax RTN (A=IA=I) 7.55 7.88 MSE scales (A=IA=I) 7.52 7.85 OSFP4 W-RTN 7.42 7.77 SIC (A=IA=I, absmax) 7.45 7.77 OSFP4 W-SIC 7.39 7.67 Figure 2: Layer 12:value, W-RTN/W4A16. Full three-projection comparison: Figure 4. Contribution of smoothing, scale selection, and SIC. Table 2 compares the full procedure with simpler configurations. Optimizing only the weight scales by unweighted MSE provides a modest perplexity improvement over absmax. OSFP4 W-RTN optimizes A and selects the weight scales using weighted MSE yielding a more significant improvement. Combining the optimized transformation and scale selection with SIC gives the lowest perplexity among the NVFP4 configurations in both precision settings. In W4A4, perplexity decreases from 7.887.88 for absmax RTN to 7.677.67 for OSFP4 W-SIC, compared with 7.777.77 for SIC using A=IA=I and absmax scales. Appendix H further details an ablation study of the MSE for projection-output error across all seven projections, for all variants. The full method (optimized A, scales and SIC) attains the smallest MSE across all layers and compared schemes. These configurations assess the combined procedure rather than isolating the contribution of A alone. Figure 3 in Appendix B shows the reduction in the joint optimization objective across projections. From predicted loss to deterministic quantization. Figure 2 examines the two-stage procedure on the value projection in layer 1212. For W-RTN/W4A16, the mean error of the randomized quantizer agrees with the predicted loss LW−RTN(A∗,ΓW∗)L^W-RTN(A^*, _W^*) to within 1%1\%. The final deterministic E4M3 scale search reduces mean error to approximately half the prediction and improves over absmax weight scaling at the same optimized A. This supports using the smoothed loss to guide optimization, followed by fine-grained deterministic scale selection. Appendix E provides details and more extensive results in this line. Inference efficiency. On an NVIDIA B200, OSFP4 retains approximately 9494–97%97\% of the vendor NVFP4 prefill throughput across the measured workloads. This modest gap is primarily attributable to the explicit diagonal smoothing required before the attention output and MLP down projections, where the transformation cannot be absorbed into a preceding LayerNorm. Consistent with this explanation, a control that sets A=IA=I while retaining the same runtime path achieves essentially the same throughput as OSFP4. Appendix G reports the full workload sweep and timing protocol. AI use statement In this work, we used generative AI tools for assistance with writing and exposition, for writing code and running experiments, and for assistance in implementation of the scale optimization algorithm, as described in Appendix B. We have not used generative AI tools for any original idea. The conceptualization of this project, the idea to use the dithered quantizer for developing a convenient loss function, and the mathematical development of the loss function are solely due to the human authors. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. Reproducibility statement We provide a complete reproduction package for the OSFP4 experiments reported in this paper. It contains the experiment scripts, configurations, software dependency versions, GPU specifications, saved measurements, and commands for regenerating the reported tables and figures. The package is available in an OSFP4 reproduction repository. References AI and vLLM Project (2024) LLM Compressor External Links: Link Cited by: 4th item. Ashkboos et al. (2024) S. Ashkboos, A. Mohtashami, M. L. Croci, B. Li, P. Cameron, M. Jaggi, D. Alistarh, T. Hoefler, and J. Hensman QuaRot: outlier-free 4-bit inference in rotated LLMs. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §1.1. Avisdris et al. (2027) N. Avisdris, E. Badash, Y. Herzberg, and O. Ordentlich SmoothGPTQ: optimizing diagonal scaling for GPTQ weight-activation quantization. Note: Submitted Cited by: §1.1. Bao et al. (2026) C. Bao, X. Yan, Z. Li, G. Qin, G. Yu, and Y. Zhang SOAR: scale optimization for accurate reconstruction in NVFP4 quantization. arXiv preprint arXiv:2605.12245. External Links: Link Cited by: §D.2, §1.1. Chee et al. (2023) J. Chee, Y. Cai, V. Kuleshov, and C. De Sa QuIP: 2-bit quantization of large language models with guarantees. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §1.1. Chen et al. (2025) J. Chen, V. Egiazarian, R. L. Castro, T. Hoefler, and D. Alistarh Wush: near-optimal adaptive transforms for llm quantization. arXiv preprint arXiv:2512.00956. Cited by: §1. Cook et al. (2026) J. Cook, J. Guo, G. Xiao, Y. Lin, K. Wyss, M. Nazemi, A. Mishra, C. del Mundo, T. Blankevoort, and S. Han Four over six: more accurate NVFP4 quantization with adaptive block scaling. arXiv preprint arXiv:2512.02010. External Links: Link Cited by: §D.2, §1.1. Egiazarian et al. (2026) V. Egiazarian, R. Castro, D. Kuznedelev, A. Panferov, E. Kurtic, S. Pandit, A. Marques, M. Kurtz, S. Ashkboos, T. Hoefler, and D. Alistarh Bridging the gap between promise and performance for microscaling fp4 quantization. In International Conference on Learning Representations, Vol. 2026, p. 113529–113563. Cited by: §D.2, §1.1, §1.1, §1. Frantar et al. (2023) E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh GPTQ: accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations, External Links: Link Cited by: §1.1. Gupta et al. (2026) T. Gupta, H. Prairie, X. Wu, R. Abhyankar, Q. Wu, A. Silveria, P. Ponnusamy, J. Wang, B. Athiwaratkun, L. Song, T. Dao, D. Y. Fu, and C. De Sa Search your block floating point scales!. Proceedings of Machine Learning and Systems 8, p. 1383–1398. External Links: Link Cited by: §1.1, §3. Lifar et al. (2026) E. Lifar, S. Savkin, O. Ordentlich, and Y. Polyanskiy WaterSIC: information-theoretically (near) optimal linear layer quantization. arXiv preprint arXiv:2603.04956. External Links: Link Cited by: §1.1. Lin et al. (2024) J. Lin, J. Tang, H. Tang, S. Yang, W. Chen, W. Wang, G. Xiao, X. Dang, C. Gan, and S. Han AWQ: activation-aware weight quantization for LLM compression and acceleration. In Proceedings of Machine Learning and Systems, External Links: Link Cited by: §1.1. Lin and Wan (2026) L. Lin and X. Wan ScaleSweep: accurate NVFP4 post-training quantization of LLMs via block scale initialization. arXiv preprint arXiv:2606.07618. External Links: Link Cited by: §D.2, §1.1, §3, §4. Liu et al. (2025) Z. Liu, C. Zhao, I. Fedorov, B. Soran, D. Choudhary, R. Krishnamoorthi, V. Chandra, Y. Tian, and T. Blankevoort SpinQuant: LLM quantization with learned rotations. In International Conference on Learning Representations, External Links: Link Cited by: §1.1. Ordentlich and Polyanskiy (2026a) O. Ordentlich and Y. Polyanskiy High-rate quantized matrix multiplication i. IEEE BITS the Information Theory Magazine. Cited by: Appendix A, §1.1, §1, §1, §2.1. Ordentlich and Polyanskiy (2026b) O. Ordentlich and Y. Polyanskiy High-rate quantized matrix multiplication I. arXiv preprint arXiv:2605.13768. External Links: Link Cited by: §1.1. Savkin et al. (2025) S. Savkin, E. Porat, O. Ordentlich, and Y. Polyanskiy NestQuant: nested lattice quantization for matrix products and llms. arXiv preprint arXiv:2502.09720. Cited by: §F.1. Shao et al. (2024) W. Shao, M. Chen, Z. Zhang, P. Xu, L. Zhao, Z. Li, K. Zhang, P. Gao, Y. Qiao, and P. Luo OmniQuant: omnidirectionally calibrated quantization for large language models. In International Conference on Learning Representations, External Links: Link Cited by: §1.1. Sun et al. (2025) Y. Sun, R. Liu, H. Bai, H. Bao, K. Zhao, Y. Li, J. Hu, X. Yu, L. Hou, C. Yuan, X. Jiang, W. Liu, and J. Yao FlatQuant: flatness matters for LLM quantization. In Proceedings of the 42nd International Conference on Machine Learning, External Links: Link Cited by: §1. Xiao et al. (2023) G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han Smoothquant: accurate and efficient post-training quantization for large language models. In International conference on machine learning, p. 38087–38099. Cited by: §1.1, §1. Yu et al. (2026) H. Yu, Z. Li, D. Liu, and J. Zhang H-Scale: hessian-guided scale refinement for NVFP4 sub-byte LLM inference. arXiv preprint arXiv:2608.28113. External Links: Link Cited by: §D.2, §1.1. Appendix A Proof of Lemma 1 We first express the smoothed relative error as an integral over a unit interval in logarithmic coordinates. Proposition 1 (Integral representation of the smoothed relative error) For every t∈ℝt , ϕ(2t) φ(2^t) =∫t−1tg(v)v, = _t-1^tg(v)\,dv, g(v) g(v) =(1−2−vQFP4(2v))2. = (1-2^-vQ_FP4(2^v) )^2. (26) Proof. Substituting (3) into the definition of ϕφ and using U∼Uniform([0,1))U ([0,1)) gives ϕ(2t) φ(2^t) =U[(1−2U−tQFP4(2t−U))2] =E_U [ (1-2^U-tQ_FP4(2^t-U) )^2 ] =∫01(1−2u−tQFP4(2t−u))2u. = _0^1 (1-2^u-tQ_FP4(2^t-u) )^2\,du. (27) The substitution v=t−uv=t-u, with du=−dvdu=-dv, yields (26). Proof of Lemma 1. The first part follows since the minimum is upper bounded by the expectation, taking β=2Uβ=2^U. The fact that ϕ(−x)=ϕ(x)φ(-x)=φ(x) follows immediately from QFP4(−x)=−QFP4(x)Q_FP4(-x)=-Q_FP4(x). It remains to show that ϕ(x)≥ϕminφ(x)≥ _min and that this minimum is attained for all x∈ℐFP4x _FP4. To that end, we first resort to the infinite exponent model of Ordentlich and Polyanskiy (2026a, Section I.B). In particular, define the “E∞M1E∞ M1” constellation CE∞M1=0∪±2k,±32 2k:k∈ℤ, C_E∞ M1=\0\∪ \± 2^k,\ ± 32\,2^k:k \, (28) which is clearly a superset of FP4C_FP4. Defining Q∞(x)=argminx^∈E∞M1|x−x^|, Q_∞(x)= _ x _E∞ M1|x- x|, (29) it therefore follows that g∞(v) g_∞(v) ≜(1−2−vQ∞(2v))2=2−2v(2v−Q∞(2v))2 (1-2^-vQ_∞(2^v) )^2=2^-2v (2^v-Q_∞(2^v) )^2 ≤2−2v(2v−QFP4(2v))2=g(v),∀v∈ℝ. ≤ 2^-2v (2^v-Q_FP4(2^v) )^2=g(v),~~∀ v . (30) Consequently ϕ(2t)=∫t−1tg(v)v≥∫t−1tg∞(v)v=(a)∫12g∞(v)v=(b)ϕmin, φ(2^t)= _t-1^tg(v)\,dv≥ _t-1^tg_∞(v)\,dv (a)= _1^2g_∞(v)\,dv (b)= _min, (31) where in (a)(a) we have used the fact that 2−(v+z)Q∞(2(v+z))=2−vQ∞(2v)2^-(v+z)Q_∞(2^(v+z))=2^-vQ_∞(2^v) for all v∈ℝv and z∈ℤz (since 2zCE∞M1=CE∞M12^z C_E∞ M1= C_E∞ M1). We now establish equality (b)(b) above . This follows since for any x∈[2,4)x∈[2,4) we have that Q∞(x)=round(x)Q_∞(x)=round(x), which implies also that for any 1≤v<21≤ v<2 it holds that g∞(v)=(1−2−vround(2v))2g_∞(v)=(1-2^-vround(2^v))^2. Consequently ∫12g∞(v)v=∫12(1−2−vround(2v))2v=∫01(1−2−(u+1)round(2(u+1)))2u=ϕmin _1^2g_∞(v)\,dv= _1^2(1-2^-vround(2^v))^2\,dv= _0^1(1-2^-(u+1)round(2^(u+1)))^2\,du= _min (32) as required. The bound (31) holds with equality whenever (30) holds with equality. Equality in (30) in turn holds for all v for which Q∞(2v)=QFP4(2v)Q_∞(2^v)=Q_FP4(2^v). Noting that CE∞M1=…,34, 1,32, 2, 3, 4, 6, 8,… C_E∞ M1=\…, 34,\ 1,\ 32,\ 2,\ 3,\ 4,\ 6,\ 8,…\ (33) we see that Q∞(y)=QFP4(y)∀y∈[7/8,7]. Q_∞(y)=Q_FP4(y)~~~∀ y∈[7/8,7]. (34) In particular, for any x∈ℐFP=[7/4,7]x _FP=[7/4,7] we have that Q∞(x)=QFP4(x)Q_∞(x)=Q_FP4(x) on the entire interval [x/2,x][x/2,x]. Consequently, setting tx=log2(x)t_x= _2(x) we see that for any x∈ℐFP=[7/4,7]x _FP=[7/4,7] it holds that g(v)=g∞(v)g(v)=g_∞(v) for all v∈[tx−1,tx]v∈[t_x-1,t_x] and therefore (30) holds with equality for all x∈ℐFP=[7/4,7]x _FP=[7/4,7], as claimed. Appendix B Implementation of the Scale Optimization This appendix specifies the continuous optimization of A, ΓW _W, and, when activations are quantized, ΓX _X. It also describes how the resulting parameters determine the FP32 tensor scales used for deployment. The continuous optimization and the subsequent selection of E4M3 weight scales are separate steps: the former uses the smoothed loss ϕφ, whereas the latter evaluates deterministic FP4 quantization error. Calibration statistics and parameterization. For a weight matrix W∈ℝm×nW ^m× n, let k1k_1 be the number of calibration activation vectors used to estimate ΣX _X. For W4A4, the activation term uses a fixed subset of k≤k1k≤ k_1 complete activation vectors, sampled uniformly without replacement. This subset is selected once for the mapping and reused in every optimization iteration. Weight-only optimization requires ΣX _X but does not retain an activation matrix in its objective. When several projections share one smoothing transformation, their weight matrices are stacked along the output dimension and optimized as one mapping; m then denotes the combined number of output rows. For each group ℓ , the trainable parameters are the natural logarithms aℓr a_ r =logαℓ(r), = _ ^(r), r r =1,…,16, =1,…,16, bℓi b_ i =logγWℓ(i), = _W_ ^(i), i i =1,…,m, =1,…,m, cℓj c_ j =logγXℓ(j), = _X_ ^(j), j j =1,…,k. =1,…,k. (35) The cℓjc_ j parameters are omitted for W4A16. Scales are reconstructed by exponentiation, ensuring positivity. We initialize αℓ(r)=1 _ ^(r)=1, γWℓ(i)=maxr|Wℓ(i,r)|/6 _W_ ^(i)= _r|W_ (i,r)|/6, and γXℓ(j)=maxr|Xℓ(r,j)|/6 _X_ ^(j)= _r|X_ (r,j)|/6, with positive numerical floors for zero-valued blocks. The weights, calibration activations, and second-moment statistics are held fixed throughout this optimization. The loss and trainable log parameters use FP32 arithmetic. Objective and normalization. Define the fixed weight metric dℓr=ΣX(t,t),for weight RTN,U(t,t)2,for weight SIC,t=16(ℓ−1)+r. d_ r= cases _X(t,t),&for weight RTN,\\ U(t,t)^2,&for weight SIC, cases t=16( -1)+r. (36) For robustness, the implementation computes the SIC factor from the damped second moment, U⊤U=ΣX+λIn,λ=0.01tr(ΣX)n, U U= _X+λ I_n, λ=0.01\, tr( _X)n, (37) as is common in GPTQ. The per-group losses evaluated in the implementation are Wℓ _W_ =1m∑r=116∑i=1mdℓrWℓ(i,r)2ϕ(Wℓ(i,r)αℓ(r)γWℓ(i)), = 1m _r=1^16 _i=1^md_ rW_ (i,r)^2φ\! ( W_ (i,r) _ ^(r) _W_ ^(i) ), (38) Xℓ _X_ =1mk∑r=116∑j=1k(∑i=1mWℓ(i,r)2)Xℓ(r,j)2ϕ(αℓ(r)Xℓ(r,j)γXℓ(j)). = 1mk _r=1^16 _j=1^k ( _i=1^mW_ (i,r)^2 )X_ (r,j)^2φ\! ( _ ^(r)X_ (r,j) _X_ ^(j) ). (39) Thus, the scalar objective is =∑ℓWℓJ= _ J_W_ for W4A16 and =∑ℓ(Wℓ+Xℓ)J= _ (J_W_ +J_X_ ) for W4A4. These are the corresponding losses in Section 2, normalized by mkmk, with dℓr\d_ r\ in the weight metric estimated using all k1k_1 calibration vectors and the activation term averaged over the selected k vectors. The coefficients dℓrWℓ(i,r)2d_ rW_ (i,r)^2 and Xℓ(r,j)2∑iWℓ(i,r)2X_ (r,j)^2 _iW_ (i,r)^2 are computed once. All groups are evaluated together in a batched computation; their parameters are disjoint, so summing their objectives introduces no coupling between groups. Differentiable evaluation of ϕφ. We evaluate the smoothed relative error through a lookup table indexed by t=log2|x|t= _2|x|. The table has spacing h=2−12h=2^-12 on [−4,8][-4,8], with entries ϕ(2tq)φ(2^t_q) at tq=−4+qht_q=-4+qh. Each entry is computed using the identity proved in Proposition 1, by analytically integrating ϕ(2t)=∫t−1t(1−2−vQFP4(2v))2v, φ(2^t)= _t-1^t (1-2^-vQ_FP4(2^v) )^2\,dv, (40) splitting the integral at the FP4 decision boundaries. Table construction uses double precision and the cached table used by the optimizer is FP32. Values between adjacent entries are obtained by linear interpolation in t. Recall that the trainable variables are the log-scales a,b,ca,b,c in (35). For nonzero inputs, t is affine in these variables, so their gradients follow directly from the interpolation slope with respect to t via the chain rule, with a factor of 1/ln21/ 2 converting natural logarithms to base two. Automatic differentiation treats the integer cell index as fixed and differentiates through the interpolation fraction, the logarithm of the magnitude, and the exponential scale parameterization. Consequently, optimization uses the derivative of the interpolated surrogate, without sampling the randomized quantizer or differentiating through deterministic FP4 rounding. The implementation clamps the logarithmic argument to [−4,8−h][-4,8-h] before interpolation; outside this interval the lookup contribution has zero derivative with respect to its argument. At x=0x=0 the lookup is finite and its loss coefficient is zero. Adam iterations. We jointly update all the log parameters using Adam for 8080 iterations with a fixed learning rate η=0.12η=0.12, moment coefficients (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999), and ϵ=10−8ε=10^-8. Weight decay is zero and AMSGrad is disabled. Let θs _s collect the log parameters a,b,ca,b,c after iteration s, with c omitted for weight-only quantization, and let gs=∇θ(θs−1)g_s= _θJ( _s-1). Adam maintains exponentially weighted moving averages psp_s of the gradients and vsv_s of their elementwise squares. The updates are ps p_s =β1ps−1+(1−β1)gs, = _1p_s-1+(1- _1)g_s, vs v_s =β2vs−1+(1−β2)gs⊙2, = _2v_s-1+(1- _2)g_s 2, p^s p_s =ps/(1−β1s), =p_s/(1- _1^s), v^s v_s =vs/(1−β2s), =v_s/(1- _2^s), θs,j _s,j =θs−1,j−ηp^s,jv^s,j+ϵ,j=1,…,dim(θ), = _s-1,j-η\, p_s,j v_s,j+ε, j=1,…, (θ), (41) where j indexes the trainable log parameters, p0=v0=0p_0=v_0=0, and all operations are elementwise. The quantities p^s p_s and v^s v_s correct the moving averages for their zero initialization. Each iteration clears the previous gradients, evaluates the entire fixed objective, backpropagates, and updates the parameters simultaneously. There is no alternating minimization, learning-rate schedule, gradient clipping, or early stopping. The final iterate is retained, rather than the iterate with the smallest observed loss. The procedure seeks a low-loss solution of the nonconvex surrogate and does not impose monotonic descent. Scale normalization and deployment precision. The objective is invariant under a common rescaling within each group. After the last Adam update, we exploit this freedom by setting sℓ s_ =exp(1m∑i=1mlogγWℓ(i)), = \! ( 1m _i=1^m _W_ ^(i) ), αℓ(r) _ ^(r) ←sℓαℓ(r),γWℓ(i)←γWℓ(i)/sℓ,γXℓ(j)←sℓγXℓ(j). ← s_ _ ^(r), _W_ ^(i)← _W_ ^(i)/s_ , _X_ ^(j)← s_ _X_ ^(j). (42) This sets the geometric mean of the continuous weight scales to one while preserving both Wℓ(i,r)/(αℓ(r)γWℓ(i))W_ (i,r)/( _ ^(r) _W_ ^(i)) and αℓ(r)Xℓ(r,j)/γXℓ(j) _ ^(r)X_ (r,j)/ _X_ ^(j). The normalization is applied once, after optimization. We then round the diagonal entries of A to the weight dtype when the transformation is folded into a preceding normalization layer, or to BF16 when explicit runtime smoothing is required. The subsequent deterministic E4M3 weight-scale selection uses this rounded A. Rounding and discrete scale selection are not part of the Adam iterations. FP32 tensor scales. The global tensor scales are distinct from the block scales optimized above and are not additional Adam parameters. For weights, the saved FP32 global scale is explicitly set to one. After the normalization in (42), the weight-scale search selects E4M3 values directly, with the compensating scale already incorporated into A. For W4A4 activations, the optimized ΓX _X is used only to guide the choice of A and is discarded for deployment. After applying the final rounded diagonal transformation A⋆A_ , we recompute the calibration maximum MX=max1≤j≤k1‖A⋆xj‖∞. M_X= _1≤ j≤ k_1\|A_ x_j\|_∞. (43) This uses the full calibration statistics, not only the subset used in the activation loss. The maximum can be recovered exactly from stored per-channel absolute maxima, since the diagonal scaling is fixed. The saved FP32 global quantization multiplier is gX=448⋅6MX,sX=gX−1=MX448⋅6, g_X= 448· 6M_X, s_X=g_X^-1= M_X448· 6, (44) where 448448 and 66 are the maximum positive E4M3 and E2M1 values, respectively, and sXs_X denotes the corresponding dequantization scale. Numerical safeguards replace nonfinite multipliers by one. This global scale remains fixed during inference. The E4M3 block scales are computed dynamically from each actual group of 1616 smoothed activation entries using absmax-to-6 with this global multiplier. Weight-only configurations do not require activation quantization scales. (a) Loss improvement for attention projections. (b) Loss improvement for MLP projections Figure 3: Ratio between LW−SIC,X−RTNL^W-SIC,X-RTN with default initialized values, and the optimized value, along the 32 layers of Llama-3.1-8B-instruct. The ratio is plotted for each one of the 7 projections. Appendix C Detailed Fine-Grained E4M3 Scale-Selection Procedure After the continuous optimization and scale normalization in Appendix B, we choose the final weight scales from the candidate sets scale+(γWℓ(i))C_scale^+( _W_ ^(i)) defined in the main text. For each normalized continuous scale γ=γWℓ(i)γ= _W_ ^(i), the implementation searches representable E4M3 candidate scales γ′γ in [γ/1.2,γ/0.3][γ/1.2,γ/0.3]. The rounded smoothing coefficients αℓ(r) _ ^(r) are held fixed throughout the search. Denote by ΓWℓfinal=diag(γWℓfinal,(1),….,γWℓfinal,(m)) ^final_W_ = diag(γ^final,(1)_W_ ,....,γ^final,(m)_W_ ) the final E4M3 scales used for representing WℓW_ in NVFP4 format. For the W−RTNW-RTN and W−RTN/X−RTNW-RTN/X-RTN setups we set γWℓfinal,(i) γ^final,(i)_W_ =argminγ∈scale+(γWℓ(i))∑r=116σXℓ2(r)(Wℓ(i,r)−γ⋅αℓ(r)⋅QFP4(Wℓ(i,r)γ⋅αℓ(r)))2,∀i=1,…,m, = _γ _scale^+( _W_ ^(i)) _r=1^16σ^2_X_ (r) (W_ (i,r)-γ· _ ^(r)· Q_FP4 ( W_ (i,r)γ·α^(r)_ ) )^2,~~∀ i=1,…,m, (45) and then set W^ℓFP4(i,:)=QFP4(Wℓ(i,:)Aℓ−1γWℓfinal,(i)). W^FP4_ (i,:)=Q_FP4 ( W_ (i,:)A_ ^-1γ^final,(i)_W_ ). (46) In the W−SICW-SIC and W−SIC/X−RTNW-SIC/X-RTN setups the dependence of the MSE on ΓWℓfinal ^final_W_ is more involved because of the feedback nature of the SIC quantization algorithm. Under RTN, the choice of γ affects the quantization errors corresponding to 1616 entries in the corresponding block in an independent manner, and is therefore optimized by (45). On the other hand, under SIC, errors in quantizing earlier coordinates propagate to the quantization of future coordinates by the SIC feedback loop. Consequently, optimizing ΓWℓfinalℓ=1L \ ^final_W_ \_ =1^L is less straightforward. We use Algorithm 1 in order to choose ΓWℓfinalℓ=1L \ ^final_W_ \_ =1^L, and quantize W in the same run. The main challenge in choosing γ is that we do not know what will be the input to the quantizer before the feedback quantization procedure was executed. Algorithm 1 chooses the scale γ block by block, and relies on the decisions on W W that were already committed in the previous blocks. When optimizing the scale for block ℓ , it further estimates the effect of intra-block feedback-filtered quantization by approximating W^ℓ=Wℓ W_ =W_ only for the current block. Algorithm 1 NVFP4 SIC with Optimized E4M3 Inputs: Y∈ℝn×mY ^n× m, upper triangular U∈ℝn×nU ^n× n, diagonal matrices Aℓ=1L=n/16\A_ \_ =1^L=n/16 with Aℓ=diag(αℓ(1),…,αℓ(16))A_ = diag (α^(1)_ ,…,α^(16)_ ), and initial diagonal scaling matrices ΓWℓ=1L=n/16\ _W_ \_ =1^L=n/16 with ΓWℓ=diag(γWℓ(1),…,γWℓ(m)) _W_ = diag (γ^(1)_W_ ,…,γ^(m)_W_ ) Outputs: WSIC∈FP4n×mW_SIC _FP4^n× m and scale matrices ΓWℓfinalℓ=1L=n/16\ ^final_W_ \_ =1^L=n/16 Y(n)←Y^(n)← Y for ℓ=L:1 =L:1 do ℐℓ=1+16(ℓ−1),…,16ℓI_ =\1+16( -1),…,16 \ ⊳ The set of 1616 indices participating in ℓ block Ublock=U(ℐℓ,ℐℓ)U_block=U(I_ ,I_ ) ⊳ The ℓ 16×1616× 16 block in the diagonal of U Ublockcausal=Ublock−diag(Ublock)U^causal_block=U_block- diag(U_block) ⊳ Making UblockU_block strictly upper triangular Yblock←Y(16ℓ)(ℐℓ,:)Y_block← Y^(16 )(I_ ,:) Yblockapprox←Yblock−UblockcausalWℓ⊤Y_block^approx← Y_block-U^causal_blockW _ ⊳ Approximating input to the quantizer before applying feedback iteration for i=1:mi=1:m do γWℓfinal,(i)←argminγ∈scale+(γWℓ(i))γ^final,(i)_W_ ← _γ _scale^+( _W_ ^(i)) ∑a=116(Yblockapprox(a,i)−γαℓ(a)Ublock(a,a)⋅QFP4(Yblockapprox(a,i)γαℓ(a)Ublock(a,a)))2~~~~~~~~~~~~~~~ _a=1^16 (Y_block^approx(a,i)-γ _ ^(a)U_block(a,a)· Q_FP4 ( Y_block^approx(a,i)γα^(a)_ U_block(a,a) ) )^2 end for ΓWℓfinal=diag(γWℓfinal,(1),….,γWℓfinal,(m)) ^final_W_ = diag(γ^final,(1)_W_ ,....,γ^final,(m)_W_ ) for r=16:1r=16:1 do t←16(ℓ−1)+rt← 16( -1)+r WSIC(t,:)←QFP4(Y(t)(t,:)(ΓfinalWℓ)−1αℓ(r)U(t,t))W_SIC(t,:)← Q_FP4 ( Y^(t)(t,:) ( ^final_W_ )^-1α^(r)_ U(t,t) )⊳ For optimization use Q~FP4 Q_FP4, for actual quantization use QFP4Q_FP4 Y(t−1)←Y(t)−αℓ(r)U(:,t)⋅WSIC(t,:)ΓWℓfinalY^(t-1)← Y^(t)- _ ^(r)U(:,t)· W_SIC(t,:) ^final_W_ ⊳ U(:,t)U(:,t) is the ttth column of U end for end for Appendix D Native NVFP4 Comparison: Experimental Setup Table 3 gives the full task results underlying Table 1. All 18 rows share one evaluation protocol. We distinguish W4A4 methods using absmax activation scaling, W4A4 methods searching activation scales online, and weight-only W4A16 methods. Bold entries indicate the best score within each group. Table 3: Native NVFP4 accuracy on Llama-3.1-8B-Instruct under the common protocol in Appendix D. All entries are percentages; recovery is relative to the measured BF16 baseline. Bold indicates the best score within each group. FP-Quant rows use author-provided quantization with our native serving adapter; H-Scale W4A4 is our extension. †NVIDIA’s separately released checkpoint has uncontrolled preparation. The online-search group includes 4over6, which selects activation scales by reconstruction error during inference. OSFP4 activation-ScaleSweep rows retain their corresponding X-RTN weights, A, and tensor scales. Precision Method MMLU-CoT GSM8K HellaSwag WinoGrande Avg. Recovery % Baseline BF16 72.77 85.97 80.41 77.74 79.22 100.00 Weights and activations: absmax activation scaling W4A4 RTN (FP-Quant) 68.61 80.29 78.87 74.82 75.65 95.48 W4A4 GPTQ (FP-Quant) 68.86 80.67 78.78 77.03 76.34 96.35 W4A4 MR-GPTQ (FP-Quant) 69.84 81.20 78.16 75.53 76.18 96.16 W4A4 SOAR 69.14 82.11 78.85 76.32 76.60 96.69 W4A4 H-Scale (W4A4 extension) 69.74 81.35 79.03 76.56 76.67 96.78 W4A4 NVIDIA released checkpoint† 68.49 80.89 78.75 76.32 76.11 96.07 W4A4 OSFP4 W-RTN / X-RTN 69.21 82.03 78.83 75.37 76.36 96.39 W4A4 OSFP4 W-SIC / X-RTN 69.73 82.34 79.09 76.95 77.03 97.23 Weights and activations: online activation-scale search W4A4 ScaleSweepMSE 68.99 82.18 78.92 76.40 76.62 96.72 W4A4 ScaleSweepMSE + GPTQ 69.53 82.94 78.51 75.77 76.69 96.80 W4A4 4over6 69.56 82.11 78.56 75.14 76.34 96.36 W4A4 OSFP4 W-RTN / X-ScaleSweepMSE 69.72 83.55 79.36 75.22 76.96 97.14 W4A4 OSFP4 W-SIC / X-ScaleSweepMSE 69.56 83.40 79.10 76.32 77.09 97.31 Weights only W4A16 RTN 70.69 83.85 79.36 76.80 77.67 98.04 W4A16 H-Scale 71.06 83.85 79.88 77.19 78.00 98.45 W4A16 OSFP4 W-RTN 71.13 83.47 79.77 78.22 78.15 98.64 W4A16 OSFP4 W-SIC 71.53 84.15 80.01 77.98 78.42 98.98 D.1 Model and calibration All locally prepared models start from the same BF16 Llama-3.1-8B-Instruct checkpoint and use the same calibration set: 1024 sequences of 2048 tokens sampled from FineWeb-Edu. All seven linear projections in each decoder layer are quantized. Embeddings, normalization parameters, the language-model head, and KV caches remain in BF16. NVFP4 uses E2M1 values with one E4M3 block scale per 16 input channels and an FP32 tensor scale. W4A4 also quantizes activations in groups of 16, whereas W4A16 retains BF16 activations. No fine-tuning or distillation is performed for the locally prepared models. D.2 Methods and implementation provenance Author-provided implementations. The rows labeled RTN (FP-Quant), GPTQ (FP-Quant), and MR-GPTQ (FP-Quant) use the authors’ quantization implementation (Egiazarian et al., 2026), run on our common checkpoint and calibration data. We convert the resulting models for native vLLM evaluation, preserving their quantized weights, scales, and transforms. These rows therefore use author-provided quantization with our serving integration, rather than the original paper’s PyTorch evaluation backend. MR-GPTQ includes block Hadamard transforms and activation-based column ordering; GPTQ uses the original column order. All three use absmax activation scaling. SmoothQuant is omitted because the corresponding implementation from the FP-Quant authors was unavailable. Our implementations of competing methods. The weight-only RTN control, ScaleSweepMSE, ScaleSweepMSE+GPTQ, 4over6, SOAR, and H-Scale rows use our own implementations of the respective procedures. ScaleSweepMSE (Lin and Wan, 2026) searches weight and activation block scales by reconstruction MSE; ScaleSweepMSE+GPTQ selects weight scales before the GPTQ feedback pass and holds them fixed during feedback. 4over6 (Cook et al., 2026) chooses between absmax-to-4 and absmax-to-6 scales for both weights and activations. SOAR (Bao et al., 2026) jointly refines tensor and block scales with separate encoding and decoding scales. H-Scale (Yu et al., 2026) selects weight scales using activation-weighted reconstruction error. SOAR and the H-Scale W4A4 extension use absmax activation scaling. H-Scale is originally a weight-only method; its W4A4 row is our extension, not a result reported by its authors. These independently implemented competitors use no rotations or column reordering. OSFP4 and activation-scale ablations. The four OSFP4 configurations use the optimization and deterministic scale selection described in Appendices B and C. The diagonal transformation is folded into preceding normalization layers where possible and applied explicitly before the attention output and MLP down projections otherwise. The two activation-ScaleSweepMSE ablations reuse the corresponding OSFP4 W4A4 checkpoints, changing only the online activation block-scale rule. Their weights, diagonal transformations, and tensor scales remain fixed, so these rows isolate the effect of activation-scale selection. Released NVIDIA checkpoint. We also evaluate the separately released nvidia/Llama-3.1-8B-Instruct-NVFP4 checkpoint under the same task protocol. Its calibration and preparation are outside our controlled procedure, so it serves as a released-model reference rather than an isolated comparison of quantization algorithms. The publishable code package contains the exact asset identities, calibration sampling procedure, software environments, method settings, conversion and validation procedures, and saved results needed to reproduce the table. Appendix E Projection-Level Distortion and Deterministic Scale Selection We compare the predicted randomized loss with measured projection errors for four configurations: W-RTN, W-SIC, W-RTN/X-RTN, and W-SIC/X-RTN. The first two retain unquantized activations (W4A16); the last two quantize both operands (W4A4). We use three fixed projections of Llama-3.1-8B-Instruct: the value projection in layer 1212, the attention output projection in layer 2626, and the MLP up-projection in layer 55, with zero-based layer indices. Each configuration and projection is optimized independently. The measurements concern local projection error on inputs captured from the BF16 model, rather than error propagated through an end-to-end quantized model. Empirical activation distribution. We use the checkpoint and FineWeb-Edu calibration-token cache specified in Appendix D. From the 10241024 cached document windows of length 20482048, we select 128128 distinct windows uniformly without replacement and 256256 distinct token positions within each window. Causal BF16 forward passes, without a chat template or generated-token decoding, provide N=32768N=32768 activation vectors xt∈ℝnx_t ^n for each projection. Here n=4096n=4096 and the output dimensions are m=1024m=1024, 40964096, and 1433614336 for 12v12v, 26o26o, and 5up5up, respectively. The prompt/token pairs are identical across all configurations and projections. We regard x as uniform on this empirical population and use the same population for optimization and measurement; this is a calibration diagnostic, not a held-out generalization test. Fitted parameters and comparisons. For each projection and configuration, we apply OSFP4 as described in Section 3 to obtain A,ΓWA, _W and, for W4A4, ΓX _X. We then keep these fitted parameters fixed and compare one predicted-loss reference with three measured error distributions. Predicted loss (dashed line). For each configuration we plot L¯=L(A,ΓW,ΓX)Nm, L= L(A, _W, _X)Nm, (47) where L is the corresponding loss from (22)–(25), with ΓX _X omitted for W4A16. This prediction uses the optimized continuous scales, before deterministic E4M3 scale selection, and is the same reference for all curves in a panel. Deterministic quantization (blue and orange). Writing W W for the reconstruction of WA−1WA^-1 and x^t x_t for that of AxtAx_t, each histogram contains the per-activation output MSE dt=1m‖W^x^t−Wxt‖22.d_t= 1m\| W x_t-Wx_t\|_2^2. (48) The two curves test different scale-selection rules: • W4A16: Blue uses OSFP4’s deterministic weight-scale selection; orange replaces it with absmax-to-6 weight scales. Both retain the same fitted A, tensor scale, and RTN or SIC rounding procedure, with x^t=Axt x_t=Ax_t. For SIC, both curves use Algorithm 1, with its scale-selection step replaced by absmax-to-6 for the orange curve. • W4A4: Both deterministic curves use the same OSFP4-quantized weights, including the final E4M3 weight scales selected by the RTN or SIC procedure. Blue uses ScaleSweepMSE activation block scales, whereas orange uses absmax-to-6. The diagonal transformation A and all tensor scales remain fixed, so the comparison isolates activation block-scale selection. Randomized pseudo-MSE (green). Using the fitted continuous scales, we apply the randomized quantizer to weights and, in W4A4, activations, retaining SIC feedback where applicable. To estimate the expectation over the random dither, we repeat this procedure 3232 times with fresh independent dithers, sharing each randomized weight matrix across all activation samples. For each activation, we average the output MSE over these repetitions. The green histogram shows the distribution of these averages across activations. The full W-RTN/W4A16 comparison is shown in Figure 4; its layer 1212 value projection is shown separately in Figure 2 in the main text. The remaining configurations follow. Table 4 shows that the mean randomized pseudo-MSE closely matches the predicted loss across all configurations and projections, with ratios of 0.960.96–1.011.01. Deterministic OSFP4 achieves approximately half the predicted loss, with ratios of 0.480.48–0.530.53 (using ScaleSweepMSE activation scales in W4A4). Figure 4: W-RTN (W4A16). Blue/orange: OSFP4/absmax-to-6 weight scales at fixed A. Green: randomized pseudo-MSE; dashed: predicted loss. Figure 5: W-SIC (W4A16). Blue/orange: OSFP4/absmax-to-6 weight scales at fixed A. Green: randomized pseudo-MSE; dashed: predicted loss. Figure 6: W-RTN/X-RTN (W4A4). Blue/orange: ScaleSweepMSE/absmax-to-6 activation scales with identical OSFP4 weights. Green: randomized pseudo-MSE; dashed: predicted loss. Figure 7: W-SIC/X-RTN (W4A4). Blue/orange: ScaleSweepMSE/absmax-to-6 activation scales with identical OSFP4 weights. Green: randomized pseudo-MSE; dashed: predicted loss. Table 4: Mean measured MSE relative to predicted loss. OSFP4 W4A4 uses ScaleSweepMSE activation scales. Randomized mean / prediction OSFP4 mean / prediction Configuration 12v 26o 5up 12v 26o 5up W-RTN 1.0085 1.0068 1.0038 0.5072 0.5062 0.5267 W-SIC 0.9807 0.9595 0.9913 0.5149 0.5088 0.5320 W-RTN/X-RTN 1.0075 1.0008 0.9994 0.5035 0.4798 0.5019 W-SIC/X-RTN 0.9950 0.9876 0.9932 0.5063 0.4902 0.5076 Appendix F Additional Models This appendix reports additional model results. F.1 Llama-3-8B perplexity Published comparison results are taken from Table 2 of Savkin et al. (2025). The OSFP4, GPTQ and RTN results are added here. Table 5: WikiText-2 perplexity comparison for Llama-3-8B. Precision Method Llama-3-8B Baseline BF16 6.14 W4A16 NestQuant 6.31 NestQuantM 6.35 RTN 6.54 GPTQ 6.37 W-RTN 6.43 W-SIC 6.31 W4A4 SpinQuant 7.10 OstQuant 7.24 DuQuant 8.06 RTN 6.88 GPTQ 6.68 W-RTN/X-RTN 6.74 W-SIC/X-RTN 6.59 W4A4KV4 NestQuant 6.63 NestQuantM 6.82 F.2 Qwen3-8B accuracy Table 6: Evaluation comparison on Qwen3-8B. All quantized configurations use the NVFP4 format. Precision Method MMLU-CoT GSM8K HellaSwag WinoGrande Avg. Recovery % Baseline BF16 71.64 90.98 76.60 70.64 77.46 100.00 Weights and activations W4A4 RTN (FP-Quant) 67.72 89.31 74.68 69.46 75.29 97.20 W4A4 GPTQ (FP-Quant) 70.94 89.92 74.63 69.61 76.28 98.48 W4A4 MR-GPTQ (FP-Quant) 73.60 89.76 74.92 68.43 76.68 98.99 W4A4 QAT+Had16 (FP-Quant) 71.29 89.16 74.63 70.32 76.35 98.57 W4A4 OSFP4 W-RTN / X-RTN 73.22 90.14 74.69 69.30 76.84 99.20 W4A4 OSFP4 W-SIC / X-RTN 70.49 89.61 75.24 70.01 76.34 98.55 Weights only W4A16 OSFP4 W-RTN 72.28 90.75 75.76 70.96 77.44 99.97 W4A16 OSFP4 W-SIC 69.87 90.07 76.05 70.09 76.52 98.79 F.3 Qwen3-30B-A3B-Instruct accuracy For each MoE layer, we use expert-specific diagonal smoothing matrices rather than sharing a single transformation across experts. During calibration, we preserve the model’s natural MoE routing, so each expert is calibrated only on the tokens routed to it. At runtime, after routing, Agate/upA_gate/up is applied to the shared input activations of the selected expert’s gate and up projections, while AdownA_down is applied to the input activations of its down projection. For OSFP4 W4A4, activation global scales would need to be computed from smoothed calibration activations separately for each expert’s shared gate/up input and down input. Inference would then require a vLLM plugin that selects the appropriate scale after routing. Because this route-aware serving path was not implemented in our experiments, we report only W4A16 results for this model. For this evaluation, we measure every row once using the same seven-task protocol. We evaluate C-Eval, LiveBench, MMLU-Redux, AIME 2025, ARC-Challenge, BBH, and GPQA Diamond. Generation uses temperature 0.7, top-p 0.8, top-k 20, repetition penalty 1.0, presence penalty 1.5, and at most 32,768 output tokens. AIME 2025 uses 16 independent generations per problem. We report the unweighted mean of the seven task scores. Table 7: Evaluation comparison on Qwen3-30B-A3B-Instruct. Precision Method C-Eval LiveBench MMLU-R AIME 2025 ARC-C BBH GPQA-D Avg. Recovery % Baseline BF16 86.97 70.41 88.19 63.96 95.14 90.77 55.05 78.64 100.00 Weights only W4A16 RTN 86.57 67.11 87.63 57.71 95.56 89.42 61.11 77.87 99.02 W4A16 RTN + H-Scale 86.37 68.22 87.67 61.46 95.56 89.54 57.07 77.98 99.16 W4A16 GPTQ + H-Scale 86.22 67.93 87.72 59.58 95.22 90.06 55.56 77.47 98.51 W4A16 MR-GPTQ + H-Scale 86.51 67.24 87.82 60.42 95.14 89.71 54.04 77.27 98.26 W4A16 OSFP4 W-RTN 86.70 69.44 87.61 60.21 95.14 90.08 58.08 78.18 99.42 W4A16 OSFP4 W-SIC 86.39 67.77 87.77 57.92 94.71 89.96 57.58 77.44 98.47 F.4 Gemma-4-31B-IT accuracy For this evaluation, we use the no-thinking setting and seed counts described in Red Hat AI’s Gemma-4-31B-IT-NVFP4 model card on Hugging Face as a reference for our local measurements. Every row uses zero-shot, text-only prompts and the Gemma chat template with thinking disabled. We generate responses with vLLM using a 32,768-token context, temperature 1.0, top-p 0.95, and top-k 64. We score GPQA Diamond (GPQA-D), MATH-500, and AIME 2025 with LightEval, and IFEval with lm-eval’s prompt-level strict metric. Scores are averaged over three seeds for GPQA-D, MATH-500, and IFEval, and eight seeds for AIME 2025. The RTN (RedHatAI) row uses RedHatAI’s publicly released NVFP4 checkpoint. All table entries, including BF16, are our local measurements. Table 8: Evaluation comparison on Gemma-4-31B-IT. Precision Method GPQA-D MATH-500 IFEval strict AIME 2025 Avg. Recovery % Baseline BF16 78.28 89.20 90.94 67.92 81.59 100.00 Weights and activations W4A4 RTN (RedHatAI) 76.60 88.07 90.14 64.17 79.74 97.74 W4A4 MR-GPTQ (FP-Quant) 74.58 88.20 91.00 63.75 79.38 97.30 W4A4 OSFP4 W-RTN / X-RTN 75.76 90.40 90.51 63.75 80.11 98.19 W4A4 OSFP4 W-SIC / X-RTN 78.28 91.87 90.76 62.08 80.75 98.97 Appendix G Inference Efficiency Table 9 compares the inference throughput of OSFP4, the vendor NVFP4 RTN baseline, NVFP4 with unit smoothing, and BF16 on Llama-3.1-8B-Instruct (NVIDIA B200, vLLM 0.28.0). Each workload uses a fixed synthetic batch of random-token prompts. Prefill emits one token and measures time to first token. Decode is measured with 256 output tokens and with one output token; subtracting the two durations removes the common prefill contribution. For each method and workload, we perform three independent timing repetitions, each consisting of 30 inference runs. The smoothing transformation A is applied at runtime only on o_proj and down_proj. End-to-end inference includes attention with a BF16 KV cache, activation quantization, output writes, and scheduling overhead in addition to FP4 matrix multiplication. The measured gains therefore depend on the workload. OSFP4 closely matches the unit-smoothing control across the sweep, suggesting that its gap to the vendor NVFP4 path primarily reflects the runtime smoothing implementation rather than the values of A. Table 9: Inference throughput on Llama-3.1-8B-Instruct (NVIDIA B200). Workload Throughput Batch Prompt Out BF16 NVFP4 NVFP4 A=IA=I OSFP4 Prefill: input throughput (thousand tokens/s), measured to first output token 1 512 1 60.2 81.0 78.1 78.4 8 512 1 82.5 177.0 165.4 166.5 32 512 1 86.5 192.4 180.8 180.6 1 2048 1 77.0 153.1 145.0 144.9 8 2048 1 85.0 183.9 173.7 174.0 1 8192 1 75.0 144.6 137.8 138.0 Decode: output throughput across the batch (tokens/s), excluding prefill 1 128 256 274 321 312 312 8 128 256 2183 2627 2548 2547 32 128 256 8448 9936 9664 9642 Appendix H Component Ablation We conduct an ablation study on Llama-3.1-8B-Instruct under W4A16 and W4A4 quantization. All configurations are calibrated on 1,0241,024 FineWeb sequences of 2,0482,048 tokens. Perplexity and projection-output errors are evaluated on the WikiText-2 test set. For W4A16, we compare AbsMax, optimal E4M3 scale according to unweighted MSE, our W-RTN method, W-SIC with A=IA=I and standard AbsMax weight scales, and our W-SIC method. For W4A4, we compare AbsMax, optimal E4M3 scale for W, with A=IA=I and standard AbsMax scaling for X, our W-RTN method, W-SIC with A=IA=I and standard AbsMax weight and activation scales, and our W-SIC method. For each method, we report the relative MatMul MSE for the seven projection types p∈q,k,v,o,gate,up,downp∈\q,k,v,o,gate,up,down\. Let Yt,pY_t,p denote the full-precision output of projection p in Transformer layer t, and let Y^t,pmethod Y_t,p^method and Y^t,pFP8RTN Y_t,p^FP8~RTN denote the corresponding outputs produced by the evaluated method and the FP8 RTN reference, respectively. We compute the squared Frobenius-error ratio independently in each Transformer layer and report its mean over all 3232 layers: Rp R_p =132∑t=132‖Y^t,pmethod−Yt,p‖F2‖Y^t,pFP8RTN−Yt,p‖F2,forp∈q,k,v,o,gate,up,down. = 132 _t=1^32 \| Y_t,p^method-Y_t,p \|_F^2 \| Y_t,p^FP8~RTN-Y_t,p \|_F^2,~~for~~p∈\q,k,v,o,gate,up,down\. (49) The W4A16 configurations use a W8A16 FP8 RTN reference, whereas the W4A4 configurations use a W8A8 FP8 RTN reference. The table reports the seven relative MatMul MSE values and WikiText-2 perplexity for each configuration. Table 10: NVFP4 component ablation on Llama-3.1-8B-Instruct: relative projection MSE and WikiText-2 perplexity. Precision Method Relative MMSE ↓ PPL ↓ q k v o gate up down Baseline BF16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 7.22 Weights only W8A16 RTN 1.00 1.00 1.00 1.00 1.00 1.00 1.00 7.24 W4A16 RTN 10.27 8.68 14.30 12.59 12.22 12.64 12.52 7.55 W4A16 MSE 8.12 6.93 12.01 9.28 9.08 9.39 9.43 7.52 W4A16 W-RTN 5.29 4.42 8.53 8.16 8.48 8.95 8.10 7.42 W4A16 SIC 5.88 4.73 9.05 5.82 9.00 9.53 9.19 7.45 W4A16 W-SIC 3.63 2.89 6.08 4.02 6.45 6.86 6.29 7.39 Weights and activations W8A8 RTN 1.00 1.00 1.00 1.00 1.00 1.00 1.00 7.27 W4A4 RTN 10.86 8.97 16.18 12.67 12.41 12.85 12.45 7.88 W4A4 MSE 9.76 8.09 15.00 10.96 10.80 11.20 10.81 7.85 W4A4 W-RTN/X-RTN 7.05 6.19 10.12 10.12 10.38 10.75 9.77 7.77 W4A4 SIC 8.52 6.88 13.54 9.25 10.75 11.28 10.68 7.77 W4A4 W-SIC/X-RTN 6.17 5.38 8.94 8.03 9.33 9.72 8.82 7.67 Appendix I Detailed Loss Derivations and SIC Procedure Denote by scaleC_scale all the positive points in the FP8 E4M3 constellation, and recall the definition of FP4C_FP4 from (1). Let W∈ℝm×nW ^m× n and X∈ℝn×kX ^n× k. We generally think of X as a random vector with k=1k=1, but for optimization of the diagonal scaling matrix A, we will denote by X the matrix holding k calibration activations in ℝnR^n, so that k>1k>1. We also define ΣX=1kXX⊤∈ℝn×n _X= 1kX ^n× n. Let L=n/16L=n/16 and assume for simplicity that L is an integer. We partition W to L matrices of shape m×16m× 16 so that W=[W1|⋯|WL],Wℓ=W(:,1+16(ℓ−1):16ℓ)forℓ=1,…,L. W=[W_1|·s|W_L],~~~W_ =W(:,1+16( -1):16 )~for~ =1,…,L. (50) Under NVFP4 quantization we represent each WℓW_ as W^ℓ=diag(ΓWℓ)⋅W^ℓFP4,whereΓWℓ∈scalem,W^ℓFP4∈FP4m×16. W_ = diag( _W_ )· W_ ^FP4,~~where~ _W_ _scale^m, W_ ^FP4 _FP4^m× 16. (51) Similarly, X=[X1⋮XL],Xℓ=X(1+16(ℓ−1):16ℓ,:)forℓ=1,…,L. X= [ array[]@c@X_1\\ \\ X_L array ],~~~X_ =X(1+16( -1):16 ,:)~for~ =1,…,L. When activations are also quantized, we represent each XℓX_ as X^ℓ=X^ℓFP4⋅diag(ΓXℓ),whereΓXℓ∈scalek,X^ℓFP4∈FP416×k. X_ = X_ ^FP4· diag( _X_ ),~~where~ _X_ _scale^k, X_ ^FP4 _FP4^16× k. (55) In this work we use a diagonal smoothing matrix A∈ℝn×nA ^n× n so that we actually quantize WA−1WA^-1 and (possibly) AXAX. We keep using the notation W^ℓ\ W_ \ and X^ℓ\ X_ \ for the corresponding NVFP4 representation even when A≠InA≠ I_n, in which case W^ℓ W_ can be very far from WℓW_ , and same for X. We also express the matrix A as a block diagonal matrix with blocks of size 16×1616× 16 so that A=[A10⋯00A2⋯0⋱00⋯AL]. A= bmatrixA_1&0&·s&0\\ 0&A_2&·s&0\\ & & & \\ 0&0&·s&A_L bmatrix. (56) The MatMul Quantization Error: Let e e =W^X^−WX=∑ℓ=1L(W^ℓXℓ^−(WℓAℓ−1)(AℓXℓ)) = W X-WX= _ =1^L ( W_ X_ -(W_ A_ ^-1)(A_ X_ ) ) (57) =∑ℓ=1L(WℓAℓ−1+eWℓ)(AℓXℓ+eXℓ)−((WℓAℓ−1)(AℓXℓ)) = _ =1^L(W_ A_ ^-1+e_W_ )(A_ X_ +e_X_ )- ((W_ A_ ^-1)(A_ X_ ) ) (58) =∑ℓ=1LeWℓAℓXℓ+WℓAℓ−1eXℓ+eWℓeXℓ = _ =1^Le_W_ A_ X_ +W_ A_ ^-1e_X_ +e_W_ e_X_ (59) where eWℓ=W^ℓ−(WℓAℓ−1)e_W_ = W_ -(W_ A_ ^-1) and eXℓ=X^ℓ−(AℓXℓ)e_X_ = X_ -(A_ X_ ). Setting eW=[eW1|⋯|eWL],eX=[eX1⋮eXL], e_W=[e_W_1|·s|e_W_L],~~~e_X= [ array[]@c@e_X_1\\ \\ e_X_L array ], we obtain e=eWAX+WA−1eX+eWeX. e=e_WAX+WA^-1e_X+e_We_X. (63) We further neglect the third term which has much smaller energy than the sum of the first two under NVFP4 quantization, and approximate e≈eWAX+WA−1eX. e≈ e_WAX+WA^-1e_X. (64) Randomized FP4 quantization error: We assume the randomized quantizer Q~FP4(⋅) Q_FP4(·) from (3) is used. We model it as a channel with input y∈ℝy and output Q~FP4(y)∈ℝ Q_FP4(y) satisfying Q~FP4(y)=y+y2ϕ(y)Z, Q_FP4(y)=y+ y^2φ(y)Z, (65) where Z is a random variable with zero mean and unit variance, statistically independent of everything. For a matrix F∈ℝk1×k2F ^k_1× k_2 the notation Q~FP4(F) Q_FP4(F) corresponds to applying the randomized quantizer Q~FP4(⋅) Q_FP4(·) on each component of F. Round to nearest (RTN) quantization: The simplest way to use the randomized quantizer is to feed to it the scaled signal, and multiply the output by the inverse scaling. This is referred to as RTN quantization. Specifically, given A,ΓWA, _W under RTN quantization the entries of the weight matrix W are quantized to W^ℓFP4=Q~FP4(diag(ΓWℓ)−1WℓAℓ−1),so thatW^ℓ=diag(ΓWℓ)Q~FP4(diag(ΓWℓ)−1WℓAℓ−1). W_ ^FP4= Q_FP4 ( diag( _W_ )^-1W_ A^-1_ ),~~so that~~ W_ = diag( _W_ ) Q_FP4 ( diag( _W_ )^-1W_ A^-1_ ). (66) Denoting γWℓ(i)i=1m\γ^(i)_W_ \_i=1^m and αℓ(r)r=116\ _ ^(r)\_r=1^16 for the diagonal entries of ΓWℓ _W_ and AℓA_ , respectively, we obtain eWℓ(i,r)=W^ℓ(i,r)−Wℓ(i,r)αℓ(r)=(Wℓ(i,r)αℓ(r))2ϕ(Wℓ(i,r)γWℓ(i)αℓ(r))Zℓ(i,r), e_W_ (i,r)= W_ (i,r)- W_ (i,r)α^(r)_ = ( W_ (i,r)α^(r)_ )^2φ ( W_ (i,r) _W_ ^(i)α^(r)_ )Z_ (i,r), (67) where Zℓ(i,r)\Z_ (i,r)\ are iid zero-mean, unit-variance random variables. Let LWℓRTN(ΓWℓ,Aℓ)=‖eWℓAℓXℓ‖F2, L^RTN_W_ ( _W_ ,A_ )=E\|e_W_ A_ X_ \|_F^2, (68) where Xℓ,Wℓ,ΓWℓ,AℓX_ ,W_ , _W_ ,A_ are assumed fixed and the expectation is only with respect to the randomization of Q~FP4(⋅) Q_FP4(·), that is, with respect to the iid random variables Zℓ(i,r)\Z_ (i,r)\. It immediately follows that LWℓRTN(ΓWℓ,Aℓ) L^RTN_W_ ( _W_ ,A_ ) =∑i=1m∑j=1k∑r=116Wℓ2(i,r)Xℓ2(r,j)⋅ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)) = _i=1^m _j=1^k _r=1^16W^2_ (i,r)X^2_ (r,j)·φ ( W_ (i,r) _W_ ^(i)α^(r)_ ) =k∑r=116σXℓ2(r)∑i=1mWℓ2(i,r)⋅ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)), =k _r=1^16σ^2_X_ (r) _i=1^mW^2_ (i,r)·φ ( W_ (i,r) _W_ ^(i)α^(r)_ ), (69) where σXℓ2(r)=1k∑j=1kXℓ2(r,j)σ^2_X_ (r)= 1k _j=1^kX^2_ (r,j) (we also have σXℓ2(r)=ΣX(r+16(ℓ−1),r+16(ℓ−1))σ^2_X_ (r)= _X(r+16( -1),r+16( -1))). We assume that the random variables Zℓ(⋅,⋅)\Z_ (·,·)\ corresponding to different blocks are also independent. This implies LWRTN=‖eWAX‖F2=∑ℓ=1LLWℓRTN(ΓWℓ,Aℓ). L^RTN_W=E\|e_WAX\|_F^2= _ =1^LL^RTN_W_ ( _W_ ,A_ ). (70) If activations are also quantized, applying RTN with the randomized FP4 quantizer gives X^ℓFP4=Q~FP4(AℓXℓdiag(ΓXℓ)−1),so thatX^ℓ=Q~FP4(AℓXℓdiag(ΓXℓ)−1)diag(ΓXℓ), X_ ^FP4= Q_FP4 (A_ X_ diag( _X_ )^-1 ),~~so that~~ X_ = Q_FP4 (A_ X_ diag( _X_ )^-1 ) diag( _X_ ), (71) and eXℓ(r,j)=X^ℓ(r,j)−αℓ(r)Xℓ(r,j)=(αℓ(r)Xℓ(r,j))2ϕ(αℓ(r)γXℓ(j)Xℓ(r,j))Z~ℓ(r,j), e_X_ (r,j)= X_ (r,j)-α^(r)_ X_ (r,j)= (α^(r)_ X_ (r,j) )^2φ ( α^(r)_ _X_ ^(j)X_ (r,j) ) Z_ (r,j), (72) where γXℓ(j)j=1k\γ^(j)_X_ \_j=1^k denote the diagonal entries of ΓXℓ _X_ , and Z~ℓ(r,j)\ Z_ (r,j)\ are iid zero-mean unit-variance random variables statistically independent of everything. Similar to the derivation above, we obtain LXℓRTN(ΓXℓ,Aℓ)=‖WℓAℓ−1eXℓ‖F2=m∑r=116σWℓ2(r)∑j=1kXℓ2(r,j)⋅ϕ(αℓ(r)γXℓ(j)Xℓ(r,j)), L^RTN_X_ ( _X_ ,A_ )=E\|W_ A^-1_ e_X_ \|_F^2=m _r=1^16σ^2_W_ (r) _j=1^kX^2_ (r,j)·φ ( α^(r)_ _X_ ^(j)X_ (r,j) ), (73) where σWℓ2(r)=1m∑i=1mWℓ2(i,r)σ^2_W_ (r)= 1m _i=1^mW^2_ (i,r). We also have LXRTN=‖WA−1eX‖F2=∑ℓ=1LLXℓRTN(ΓXℓ,Aℓ). L^RTN_X=E\|WA^-1e_X\|_F^2= _ =1^LL^RTN_X_ ( _X_ ,A_ ). (74) Successive interference cancellation (SIC): Note that ‖eWAX‖F2=tr((eWAX)(eWAX)⊤)=ktr(eWAΣXA⊤eW⊤). \|e_WAX\|_F^2= tr ((e_WAX)(e_WAX) )=k tr (e_WA _XA e_W ). (75) Let U∈ℝn×nU ^n× n be an upper triangular matrix such that ΣX=U⊤U _X=U U, and denote Y=UA⊤(WA−1)⊤=UW⊤∈ℝn×m. Y=UA (WA^-1) =UW ^n× m. (76) We have ‖eWAX‖F2=k‖UA⊤eW⊤‖F2=k‖Y−UAW^⊤‖F2. \|e_WAX\|_F^2=k\|UA e_W \|_F^2=k\|Y-UA W \|_F^2. (77) Even when A and the E4M3 scaling matrices ΓWℓ=1L\ _W_ \_ =1^L are fixed, the problem of choosing W^FP4∈FP4m×n W^FP4 _FP4^m× n such as to minimize (77) is computationally difficult due to the coupling the matrix UAUA introduces between different columns of W W (equivalently, rows of W^⊤ W ). As common in the literature, instead of attempting an optimal solution, we resort to the SIC/GPTQ approach. Taking into account the structure of the NVFP4 constellation, this boils down to choosing WSIC∈FP4n×mW_SIC _FP4^n× m as in Algorithm 2, and then setting W^FP4=WSIC⊤ W^FP4=W _SIC and W^ℓ=ΓWℓW^ℓFP4 W_ = _W_ W_ ^FP4. Algorithm 2 NVFP4 SIC Inputs: Y∈ℝn×mY ^n× m, upper triangular U∈ℝn×nU ^n× n, diagonal matrices Aℓ=1L=n/16\A_ \_ =1^L=n/16 with Aℓ=diag(αℓ(1),…,αℓ(16))A_ = diag (α^(1)_ ,…,α^(16)_ ), and E4M3 diagonal scaling matrices ΓWℓ=1L=n/16\ _W_ \_ =1^L=n/16 with ΓWℓ=diag(γWℓ(1),…,γWℓ(m)) _W_ = diag (γ^(1)_W_ ,…,γ^(m)_W_ ) Outputs: WSIC∈FP4n×mW_SIC _FP4^n× m Y(n)←Y^(n)← Y for ℓ=L:1 =L:1 do for r=16:1r=16:1 do t←16(ℓ−1)+rt← 16( -1)+r WSIC(t,:)←QFP4(Y(t)(t,:)Γ−1Wℓαℓ(r)U(t,t))W_SIC(t,:)← Q_FP4 ( Y^(t)(t,:) ^-1_W_ α^(r)_ U(t,t) )⊳ For optimization use Q~FP4 Q_FP4, for actual quantization use QFP4Q_FP4 Y(t−1)←Y(t)−αℓ(r)U(:,t)⋅WSIC(t,:)ΓWℓY^(t-1)← Y^(t)- _ ^(r)U(:,t)· W_SIC(t,:) _W_ ⊳ U(:,t)U(:,t) is the ttth column of U end for end for The analysis of LWSIC=k‖Y−UAW^⊤‖F2L_W^SIC=kE\|Y-UA W \|_F^2 under the SIC algorithm is a bit complicated, even under the randomized FP4 quantization error model (65). The reason is that Z entries from time t affect Y(t′)Y^(t ) for t′<t <t due to the feedback nature of the quantization process. We therefore make a high-resolution quantization assumption and assume the energy of the (filtered) quantization noise is negligible next to the signal (entries of Y) energy. Under this assumption we have that Y(t)(t,:)≈U(t,t)⋅(W(:,t))⊤. Y^(t)(t,:)≈ U(t,t)·(W(:,t)) . (78) With this assumption a straightforward calculation gives (under the randomized FP4 quantization error model (65)) for fixed Aℓ\A_ \ and ΓWℓ\ _W_ \ LWSIC=k∑ℓ=1L∑r=116Uℓ2(r)∑i=1mWℓ2(i,r)ϕ(Wℓ(i,r)γWℓ(i)αℓ(r))=∑ℓ=1LLWSIC(ΓWℓ,Aℓ), L_W^SIC=k _ =1^L _r=1^16U^2_ (r) _i=1^mW^2_ (i,r)φ ( W_ (i,r) _W_ ^(i)α^(r)_ )= _ =1^LL_W^SIC( _W_ ,A_ ), (79) where Uℓ(r)=U(16(ℓ−1)+r,16(ℓ−1)+r)U_ (r)=U(16( -1)+r,16( -1)+r) and LWSIC(ΓWℓ,Aℓ)=k∑r=116Uℓ2(r)∑i=1mWℓ2(i,r)ϕ(Wℓ(i,r)γWℓ(i)αℓ(r)). L_W^SIC( _W_ ,A_ )=k _r=1^16U^2_ (r) _i=1^mW^2_ (i,r)φ ( W_ (i,r) _W_ ^(i)α^(r)_ ). (80) Note that the only difference between LWSIC(ΓWℓ,Aℓ)L_W^SIC( _W_ ,A_ ) and LWRTN(ΓWℓ,Aℓ)L_W^RTN( _W_ ,A_ ) is replacing σXℓ2(r)σ^2_X_ (r) in the latter with Uℓ2(r)U^2_ (r) in the former. Since Uℓ2(r)U^2_ (r) is the variance of the optimal prediction error of X(t)X(t) - the ttth coordinate of X∼(0,ΣX)X (0, _X), t=16(ℓ−1)+rt=16( -1)+r - from X(t+1:n)X(t+1:n) and σXℓ2(r)σ^2_X_ (r) is the variance of X(t)X(t), we have Uℓ2(r)≤σXℓ2(r)U^2_ (r)≤σ^2_X_ (r) for all ℓ,r ,r. Total quadratic Distortion: Using (69) (73) and (80) we obtain the following loss functions for randomized NVFP4 quantization, as a function of A and the E4M3 scales, under the various quantization setups: LW−RTN L^W-RTN =∑ℓ=1LLWℓRTN(ΓWℓ,Aℓ) = _ =1^LL^RTN_W_ ( _W_ ,A_ ) (81) LW−SIC L^W-SIC =∑ℓ=1LLWℓSIC(ΓWℓ,Aℓ) = _ =1^LL^SIC_W_ ( _W_ ,A_ ) (82) LW−RTN,X−RTN L^W-RTN,X-RTN =∑ℓ=1L[LWℓRTN(ΓWℓ,Aℓ)+LXℓRTN(ΓXℓ,Aℓ)] = _ =1^L [L^RTN_W_ ( _W_ ,A_ )+L^RTN_X_ ( _X_ ,A_ ) ] (83) LW−SIC,X−RTN L^W-SIC,X-RTN =∑ℓ=1L[LWℓSIC(ΓWℓ,Aℓ)+LXℓRTN(ΓXℓ,Aℓ)]. = _ =1^L [L^SIC_W_ ( _W_ ,A_ )+L^RTN_X_ ( _X_ ,A_ ) ]. (84) Note that in all expressions the dependency on A and the E4M3 scales completely decouples to L different functions, that depend on different parameters. Therefore, the optimization on Aℓ,ΓWℓ,ΓXℓ\A_ , _W_ , _X_ \ decouples to L independent optimization problems.