Paper deep dive
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Chethan Reddy G. P
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/16/2026, 5:41:26 AM
Summary
The paper introduces ExTernD, a novel post-training quantization (PTQ) method that decomposes LLM weight matrices into expanded-rank ternary factors (B, D, C). Unlike fixed-bit quantization, ExTernD uses a continuous inner-rank multiplier (μ) to progressively correct quantization errors, enabling accuracy to approach full precision (bf16) arbitrarily closely. The method employs a greedy Alternating Least Squares (ALS) algorithm, enhanced with batched GPU processing and importance weighting via llama.cpp's imatrix. Evaluated on models like Qwen3.5-4B and Gemma-4-E2B, ExTernD matches Q4_K accuracy at ~5.2-5.5 effective bits per weight while maintaining multiplication-free inference and continuous sparsity control.
Entities (10)
Relation Signals (8)
ExTernD → decomposes → LLM Weight Matrix
confidence 95% · factorization of each LLM weight matrix A∈ℝm×n into A≈Bdiag(D)C
ExTernD → comparedagainst → Q4_K
confidence 90% · ExTernD matches Q4_K's per-matrix accuracy at 5.2–5.5 effective bpw
Qwen3.5-4b → evaluatedon → WikiText-2
confidence 90% · a full Qwen3.5-4B conversion at μ = 3 reaches 10.10 wikitext-2 perplexity
ExTernD → evaluatedon → Qwen3.5-4b
confidence 90% · ExTernD matches Q4_K's per-matrix accuracy at 5.2–5.5 effective bpw on Gemma-4-E2B and Qwen3.5-4B
ExTernD → isa → Post-training Quantization
confidence 90% · We introduce ExTernD ... a post-training factorization of each LLM weight matrix
ExTernD → utilizes → Alternating Least Squares
confidence 90% · ExTernD and a greedy ALS algorithm for it
llama.cpp → provides → imatrix
confidence 85% · Given per-input-channel second moments h from a llama.cpp importance matrix (imatrix)
imatrix → usedin → Importance-Weighted ALS
confidence 85% · importance-weighted variant using llama.cpp imatrix statistics
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix $A \in \mathbb{R}^{m \times n}$ into $A \approx B \mathrm{diag}(D) C$ with ternary factors $B \in \{-1,0,+1\}^{m \times k}$, $C \in \{-1,0,+1\}^{k \times n}$ and a real scale vector $D \in \mathbb{R}^k$. The inner rank $k = \mu \min(m,n)$ is deliberately expanded beyond full rank ($\mu > 1$), so that components past full rank correct the quantization error of earlier ones. We prove the residual decreases monotonically in $k$ and can be driven below any $\varepsilon > 0$: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do. Memory and compute scale continuously with $\mu$, and factor sparsity continuously with a threshold $\tau$, so an accuracy target is hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K's per-matrix accuracy at 5.2-5.5 effective bpw (5.1-5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at $\mu = 3$ reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%), placing it near the Q4_K/Q5_K accuracy band at ~5.7 effective bpw.
Tags
Links
- Source: https://arxiv.org/abs/2607.13511v1
- Canonical: https://arxiv.org/abs/2607.13511v1
Trouble viewing inline? Open PDF directly →
Full Text
26,680 characters extracted from source content.
Expand or collapse full text
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level Chethan Reddy G.P. chethanreddygp@proton.me (July 2026 (early preprint)) Abstract We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix A∈ℝm×nA ^m× n into A≈Bdiag(D)CA≈ B\,diag(D)\,C with ternary factors B∈−1,0,+1m×kB∈\-1,0,+1\^m× k, C∈−1,0,+1k×nC∈\-1,0,+1\^k× n and a real scale vector D∈ℝkD ^k. The inner rank k=μmin(m,n)k=μ (m,n) is deliberately expanded beyond full rank (μ>1μ>1), so that components past full rank correct the quantization error of earlier ones. We prove the residual decreases monotonically in k and can be driven below any ε>0 >0: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do. Memory and compute scale continuously with μ, and factor sparsity continuously with a threshold τ, so an accuracy target is hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K’s per-matrix accuracy at 5.2–5.5 effective bpw (5.1–5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at μ=3μ=3 reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%+3.2\%), placing it near the Q4_K/Q5_K accuracy band at ∼5.7 5.7 effective bpw. 1 Introduction Ternary weights are attractive: matrix multiplication degenerates into additions and subtractions, and storage approaches log23≈1.58 _23≈ 1.58 bits per weight. BitNet [1, 2] shows that 1-bit and 1.58-bit LLMs can be trained from scratch; ternary weight networks [3] and trained ternary quantization [4] established the format earlier for vision models. But a single ternary plane carries at most 1.58 bits of information per weight, so post-training projection of a full-precision matrix onto one ternary plane loses too much: reasoning collapses [5]. PTQTP [5] takes the important step of using two trit-planes, a superposition of two ternary matrices with row-wise scales, and shows this restores much of the lost expressiveness without retraining. But two planes is still a fixed capacity budget with a ceiling it cannot get past, and there is nothing special about two, or about any fixed number of planes. ExTernD replaces a fixed count of full-size planes with a factored representation whose inner dimension k is a free parameter: A≈Bdiag(D)CA≈ B\,diag(D)\,C with both factors ternary. Setting k=μmin(m,n)k=μ (m,n) (typically μ=2μ=2–3.53.5, but unbounded) gives the representation as much capacity as needed; each additional component fits the residual left by all previous ones, so capacity is spent exactly where the ternary constraint hurt and accuracy is not capped (Sec. 2.1, proof in Appendix A). All of this is strictly post-training: the factorization sees one weight matrix at a time and never a gradient. Mainstream post-training quantization (GPTQ [7], AWQ [8], llama.cpp k-quants) sits at a small number of discrete accuracy/cost points: 3, 4, 5 bits per weight. Because k is a real-valued rank rather than a bit-width, ExTernD’s accuracy/cost trade-off is instead continuous: the inner-rank multiplier μ (storage and compute both scale linearly with it, Sec. 2.6) and a sparsity threshold τ (which sets the fraction of zeros in B and C, e.g. τ=0.7→∼41%τ=0.7→ 41\% zeros, τ=1.0→∼57%τ=1.0→ 57\%, τ=2.0→∼87%τ=2.0→ 87\%) can each be set per matrix to any real value, letting a target accuracy be hit exactly rather than rounded up to the next bit-width. Contributions: (i) ExTernD and a greedy ALS algorithm for it, with a proof (Appendix A) that its error falls below any ε>0 >0 as μ grows, making it to our knowledge the only ternary quantization that can provably reach any target accuracy, up to bf16, and hence beat any fixed-plane-count method at large enough μ; (i) a batched block-ALS GPU algorithm with decorrelated targets, matching sequential quality at ∼16× 16× speed; (i) an importance-weighted variant using llama.cpp imatrix statistics, including a non-obvious correction to the V-step; (iv) an empirical study on Gemma-4-E2B, Qwen3.5-4B and Granite-4.0-h-tiny, including a full end-to-end model conversion. 2 Method 2.1 Expanded inner rank Given A∈ℝm×nA ^m× n, we seek A≈A^=Bdiag(D)C,B∈−1,0,1m×k,C∈−1,0,1k×n,D∈ℝk.A\;≈\; A\;=\;B\,diag(D)\,C, B∈\-1,0,1\^m× k,\;C∈\-1,0,1\^k× n,\;D ^k. (1) Quality is reported as energy preserved, E=1−‖A−A^‖F2/‖A‖F2E=1-\|A- A\|_F^2/\|A\|_F^2. With real factors, k=min(m,n)k= (m,n) suffices (SVD). With ternary factors each rank-one component dibici⊤d_i\,b_ic_i is a crude, quantized object; at k=min(m,n)k= (m,n) the representation hits an information ceiling well short of full precision. The core idea is to keep adding components past full rank: component i is fitted to the residual R=A−∑j<idjbjcj⊤R=A- _j<id_jb_jc_j , i.e. to the accumulated quantization error of its predecessors. The multiplier μ=k/min(m,n)μ=k/ (m,n) becomes the accuracy dial. Crucially this only works through sequential deflation: solving for all k>min(m,n)k> (m,n) components jointly is underdetermined and fails completely (0–50% energy in our ablation, vs. 96–100% with deflation). Because μ is unbounded and the residual is monotone in it (Appendix A), the accuracy ceiling that a fixed plane count imposes does not exist here: any target accuracy is reachable by construction rather than by luck of the fit. 2.2 Ternarization operator and continuous sparsity For u∈ℝmu ^m, the adaptive mean-threshold ternarization with scale τ is Tτ(u)i=sign(ui)⋅[|ui|>τ⋅1m∑j|uj|],T_τ(u)_i=sign(u_i)·1\! [\,|u_i|>τ· 1m _j|u_j|\, ], (2) keeping the single largest entry if the result would be all-zero. τ controls factor sparsity continuously and almost matrix-independently. The optimal ternary projection argmint,α‖u−αt‖2 _t,α\|u-α t\|_2 has a closed form (sort |u||u|, keep the top-s⋆s prefix maximizing 1s∑i≤s|u|(i) 1 s _i≤ s|u|_(i)) but empirically ties T0.7T_0.7, so we use the cheaper threshold form. 2.3 Greedy sequential ALS With residual R←AR← A, for i=1,…,ki=1,…,k: 1. Alternating ternary fit (15 iterations): v←Tτ(R⊤u)v← T_τ(R u), u←Tτ(Rv)u← T_τ(Rv). 2. Optimal scale: di=u⊤Rv‖u‖22‖v‖22d_i= u Rv\|u\|_2^2\,\|v\|_2^2, the least-squares minimizer of ‖R−duv⊤‖F\|R-d\,uv \|_F. 3. Deflate: R←R−diuv⊤R← R-d_i\,uv ; store B:,i=uB_:,i=u, Ci,:=v⊤C_i,:=v . Step 2 guarantees ‖R‖F2\|R\|_F^2 never increases (Appendix A), which is what makes μ a well-behaved dial: more components monotonically means less error, all the way to exact recovery. 2.4 Batched block ALS (GPU) Extracting components in blocks of b (default 256, capped at ⌊min(m,n)/8⌋ (m,n)/8 ) turns per-component vector work into block matmuls. The naive batching, ternarizing R⊤UR U column-wise, collapses (99.5% → 50% energy at b=256b=256): every column chases the same dominant residual direction. The fix is to ternarize the jointly decorrelated least-squares target: V←Tτ([(U⊤U+ϵI)−1U⊤R]⊤),U←Tτ(RV(V⊤V+ϵI)−1),V← T_τ\! ( [(U U+ε I)^-1U R ] ), U← T_τ\! (R\,V\,(V V+ε I)^-1 ), (3) so each component is fitted against the residual minus its block-mates’ contributions (implicit within-block deflation; an exact block solve in place of Gauss–Seidel). After the alternating iterations, all b scales are solved jointly and exactly: [(U⊤U)⊙(V⊤V)]d=diag(U⊤RV), [(U U) (V V) ]\,d=diag(U RV), (4) then R←R−Udiag(d)V⊤R← R-Udiag(d)V . Deflation across blocks stays fully sequential. Block width must respect b≲min(m,n)/8b (m,n)/8 or the ternary Gram matrices become ill-conditioned. Refinement sweeps. After extraction, revisiting each block (add its contribution back to R, re-run the block fit, re-deflate) improves energy monotonically: ≈+0.4≈+0.4 points on hard matrices after 10 sweeps, and a 4–8% reduction in the multiplier needed for fixed energy (larger at high τ). 2.5 Importance-weighted ALS Frobenius error weights all input channels equally; activations do not. Given per-input-channel second moments h from a llama.cpp importance matrix (imatrix), we minimize ∑ijh~j(Aij−A^ij)2 _ij h_j\,(A_ij- A_ij)^2 with h~=h/max(h)+λ h=h/ (h)+λ. The U-step and the joint scale solve generalize directly (W=R⊙h~W=R h as target, V⊤diag(h~)V diag( h)V Gram matrices); they are the exact weighted normal equations. The V-step does not: solving it against W loses 0.3–1.8 points of weighted energy versus the unweighted algorithm, because the h~ h rescaling starves low-importance channels of support under a threshold that spans the whole component. In a column-weighted objective h~ h cancels in the V-step, so the correct V-target is the plain residual R. λ interpolates continuously between the uniform (λ→∞λ→∞) and pure-imatrix (λ=0λ=0) objectives; λ=0λ=0 matches what llama.cpp imatrix quants optimize and is stable in practice. 2.6 Cost model Inference computes y=B(diag(D)(Cx))y=B(diag(D)(Cx)): k(m+n)k(m+n) ternary add/subtracts plus k multiplies, versus mnmn multiply-adds, a ratio of μ(m+n)/max(m,n)μ\,(m+n)/ (m,n) that is linear (hence continuous) in μ. With sparse mask+sign packing (a 1-bit zero/nonzero mask per stored element plus one sign bit per nonzero, bpw=2−sparsitybpw=2-sparsity), storage per original weight is bpweff=μ⋅m+nmax(m,n)⋅(2−sparsity),bpw_eff=μ· m+n (m,n)·(2-sparsity), (5) again linear in μ and continuous in τ through the sparsity. This packing is near the entropy floor for unstructured supports (mask entropy H(0.43)≈0.99H(0.43)≈ 0.99 bits; signs incompressible). 3 Results Setup: matrices from google/gemma-4-E2B, Qwen/Qwen3.5-4B, and IBM Granite-4.0-h-tiny. Every experiment in this paper (decompositions, multiplier searches, the full-model conversion, and all perplexity evaluations) was run on a single AMD MI50 32 GB GPU (torch/ROCm). The Q4_K baseline is a faithful torch port of llama.cpp’s quantize_row_q4_K_ref (4.5 bpw). 3.1 Energy vs. multiplier; batched = sequential Gemma-4-E2B matrix μ=2μ=2 μ=2.5μ=2.5 μ=2.5μ=2.5 + 10 sweeps up_proj L30 (12288×1536) 96.18 98.32 98.75 down_proj L23 (1536×12288) 95.81 98.10 98.59 down_proj L10 (1536×6144) 97.52 99.00 99.19 q_proj L7 (2048×1536) 99.49 99.85 99.87 k_proj L6 (256×1536) 97.02 98.73 99.00 Table 1: Energy preserved (%) at τ=0.7τ=0.7. Attention projections need much less inner rank than MLPs. The batched algorithm (Sec. 2.4) matches the sequential one within ±0.2± 0.2 points on every matrix while running the six-matrix suite in 10.5 s vs. 170.9 s (∼16× 16×). On Qwen3.5-4B at μ=2.5μ=2.5, τ=0.7τ=0.7, 10 sweeps: MLPs 99.21–99.26%, v_proj 99.52%, o_proj 99.78%. Same algorithm, easier model; achievable energy at fixed μ is strongly model-dependent. 3.2 The two dials interact favorably Sweeping τ at a fixed 99%-energy target: sparsity rises continuously (41% at τ=0.7τ=0.7 to 87% at τ=2.0τ=2.0) while the required μ rises (up_proj: 2.82 → 7.21). The nonzero budget μ×μ×density falls monotonically (1.65 → 0.87): sparser factors with more components need fewer total nonzeros for equal energy, up to a convergence cliff near τ=2.5τ=2.5. Under mask+sign packing (Eq. 5), where zeros also cost a mask bit, the optimum is interior at τ=1.0τ=1.0 (∼ 57% sparsity). 3.3 Effective bits at matched Q4_K accuracy Q4_K achieves a strikingly uniform 99.4–99.5% energy on every matrix of both models. Choosing μ per matrix to match it exactly: iso-Q4_K μ (τ=1.0τ=1.0) bpweff Q4_K Gemma-4-E2B (5 matrices) 1.97–3.43 5.26–5.51 4.5 Qwen3.5-4B (6 matrices) 2.25–3.03 5.22–5.53 4.5 Table 2: Effective bits per original weight at Q4_K-matched energy, no sweeps. The ∼20% 20\% gap is uniform across shapes and models: Qwen is easier in absolute energy, but Q4_K improves by the same margin, so the gap is a property of the algorithm, not the test model. Attention consistently needs far less rank (μ≈2.0μ≈ 2.0–2.72.7) than MLPs (≈3.0≈ 3.0–3.43.4). Because the packing is near the entropy floor, closing this gap is a multiplier problem, not a coding problem: parity requires ∼17% 17\% lower iso-quality μ. Refinement sweeps give ∼5% 5\%; importance weighting (next) gives another ∼4% 4\% independently. 3.4 Importance weighting With the corrected V-step (Sec. 2.5) at λ=0λ=0, weighting helps most where the imatrix is skewed: on Granite-4.0-h-tiny, v_proj weighted energy 96.55% → 98.95% (+2.40+2.40); flat-imatrix tensors are unchanged. On Qwen3.5-4B, the rank multiplier needed for 99.5% weighted energy drops 2–8% on MLP gate/up and v_proj, and 4.5–7.4% on gated-DeltaNet projections, the best territory, since their qkv/out input importance is strongly skewed. Redoing the iso-Q4_K accounting under the weighted metric: mean bpweff 5.49 → 5.26 (−4.2%-4.2\%), best tensors 5.08–5.14, shrinking the Q4_K gap from ∼22% 22\% to ∼17% 17\% overall and 13–15% on the best tensors. 3.5 End-to-end model conversion All 200 language-model linear layers of Qwen3.5-4B (96 MLP, 32 attention, 72 gated-DeltaNet) were decomposed with the weighted algorithm at a fixed μ=3μ=3, τ=1.0τ=1.0, λ=0λ=0, no sweeps, in 19.9 minutes total on one GPU. Per-tensor energy: plain min/mean 98.52/99.45%, weighted 99.34/99.65%. model bpw wikitext-2 PPL Δ bf16 16 9.782 n/a Q5_K_M (tuned mix) ∼ 5.7 9.882 +1.0%+1.0\% Q4_K_M (tuned mix) ∼ 4.9 9.930 +1.5%+1.5\% Q5_K pure + imatrix 5.51 9.948 +1.7%+1.7\% Q4_K pure + imatrix 4.51 10.015 +2.4%+2.4\% ternary decomposition, μ=3μ=3 ∼ 5.7 eff. 10.099 +3.2%+3.2\% Table 3: First full end-to-end validation (llama-perplexity, 580 chunks, same imatrix for all calibrated rows). The decomposed model is coherent; at matched bits and calibration it currently trails pure Q5_K by ∼ 1.5 points of relative PPL. None of the known levers were applied to this conversion: refinement sweeps, per-matrix μ allocation (iso-quality μ spreads 2.3–3.0 across tensors while this run used a flat μ=3μ=3), or τ/λτ/λ tuned on PPL. The pure ladder also shows that the tuned Q4_K_M mix beats pure Q5_K despite fewer bits: per-tensor budget allocation is worth more than 0.6 uniform bpw for k-quants, and per-matrix μ allocation, trivially expressible here because μ is continuous, is the analogous untapped lever for the decomposition. 3.6 Comparison with Fixed and Stacked Ternary Formats To isolate the structural capacity of ExTernD, we compare its weight-reconstruction energy against two recent post-training ternarization methods: PT2-LLM [6], which uses a fixed single ternary plane (∼1.58 1.58 bpw), and Progressive Trit-Plane Approximation (PTQTP) [5], which uses a dense superposition of K ternary planes. To match bit budgets precisely, we evaluate PTQTP at group size G=128G=128 and set the ExTernD component count to match the number of ternary elements (k=Kmnm+nk=K mnm+n). The 1.58-bit Bottleneck. PT2-LLM uses an iterative ternary fitting (ITF) algorithm to optimize a single ternary matrix with row-wise and block-wise scales. When applied purely to the weight matrix, it preserves only ∼78–81% 78--81\% of the original energy. This confirms our hypothesis: a fixed single ternary plane lacks the structural capacity to accurately represent a dense LLM weight matrix, forcing such methods to heavily rely on activation-aware scaling to mask the underlying quantization error. Scaling to Stacked Planes. PTQTP addresses this bottleneck by stacking multiple dense ternary planes. At low bit-budgets (K=2K=2, ∼3.4 3.4 bpw), PTQTP significantly outperforms ExTernD by 1.8–2.6 percentage points of energy preservation, as direct superposition preserves dense structures better than a rank-deficient (μ≈1.5μ≈ 1.5) factorization. However, as the bit budget scales to K=4K=4 (∼6.8 6.8 bpw), ExTernD’s inner rank expands toward and past the matrix’s full rank (μ>3μ>3), structurally approaching perfect reconstruction. At this regime, ExTernD slightly edges out PTQTP in energy (∼99.72% 99.72\% vs 99.59%99.59\%), while PTQTP hits diminishing returns due to its rigid stacked-discrete-value formulation. The Sparsity Advantage. Crucially, ExTernD provides a continuous sparsity threshold τ. When forcing ExTernD to favor sparsity (τ=1.0τ=1.0), it achieves a massive 57% sparsity compared to PTQTP’s natural ∼34% 34\% sparsity and PT2-LLM’s ∼46% 46\% sparsity, while maintaining highly competitive accuracy (only a ∼0.1% 0.1\% drop in energy compared to PTQTP at K=4K=4). Imatrix-Weighted Objective. When minimizing the imatrix-weighted Frobenius error while strictly matching the number of ternary elements to PTQTP’s K=4K=4, PTQTP shifts its dense assignments to perfectly fit the most important channels, spiking its weighted energy to ∼99.85% 99.85\%. Under this naive element-matching, ExTernD sits at a competitive ∼99.58% 99.58\% when constrained to τ=1.0τ=1.0. Iso-BPW Parity. Matching the number of ternary elements penalizes ExTernD by ignoring its massive sparsity advantage. A fairer comparison matches the effective bits-per-weight (BPW), utilizing Shannon entropy to account for zero-density. PTQTP at K=4K=4 (∼33% 33\% sparsity) requires 6.686.68 effective BPW. When we increase ExTernD’s inner rank μ to precisely match this 6.686.68 BPW footprint, ExTernD achieves virtually identical energy preservation (∼99.84% 99.84\% at τ=1.0τ=1.0, 56%56\% sparsity) while maintaining a significantly sparser and more hardware-friendly format (Table 5). This decisively proves that ExTernD’s factorization structurally matches or exceeds dense stacked planes when normalized for actual information capacity, eliminating the ceiling of fixed-width methods. Fixed Stacked Planes (PTQTP) Factorization (ExTernD, K=4K=4 equiv.) Matrix (Qwen3.5-4B) PT2-LLM K=2K=2 K=4K=4 (Weighted) τ=1.0τ=1.0 (Weighted) τ=0.7τ=0.7 (Weighted) ‘L30.up_proj‘ 81.63% 97.45% 99.85% 99.56% 99.70% ‘L4.up_proj‘ 81.74% 97.51% 99.85% 99.56% 99.73% ‘L10.down_proj‘ 81.38% 97.22% 99.85% 99.60% 99.73% ‘L23.down_proj‘ 81.57% 97.41% 99.85% 99.58% 99.71% ‘L7.v_proj‘ 81.14% 96.88% 99.85% 99.60% 99.73% ‘L31.o_proj‘ 78.16% 97.23% 99.83% 99.56% 99.66% Mean Sparsity ∼ 46% ∼ 33% ∼ 35% ∼ 57% ∼ 40% Table 4: Comparison of weight energy preservation and sparsity. PT2-LLM demonstrates the ∼81% 81\% hard limit of a single ternary plane. PTQTP scales accuracy by stacking dense planes, while ExTernD scales by expanding the inner rank of sparse ternary factors. At high capacity (K=4K=4), ExTernD matches dense stacked accuracy while unlocking 57%57\% sparsity. Stacked Planes Factorization (ExTernD, Iso-BPW) Matrix (Qwen3.5-4B) PTQTP K=4K=4 (6.686.68 bpw) τ=1.0τ=1.0 (56%56\% sparse) τ=0.7τ=0.7 (40%40\% sparse) ‘L30.up_proj‘ 99.85% 99.85% (μ=3.6μ=3.6) 99.81% (μ=3.3μ=3.3) ‘L4.up_proj‘ 99.85% 99.84% (μ=3.6μ=3.6) 99.80% (μ=3.3μ=3.3) ‘L10.down_proj‘ 99.85% 99.86% (μ=3.6μ=3.6) 99.82% (μ=3.3μ=3.3) ‘L23.down_proj‘ 99.85% 99.85% (μ=3.6μ=3.6) 99.81% (μ=3.3μ=3.3) ‘L7.v_proj‘ 99.85% 99.85% (μ=3.3μ=3.3) 99.81% (μ=3.0μ=3.0) ‘L31.o_proj‘ 99.83% 99.83% (μ=2.8μ=2.8) 99.78% (μ=2.6μ=2.6) Mean Energy 99.85% 99.84% 99.80% Table 5: Iso-BPW Comparison. When allowing ExTernD to expand its inner rank μ to match the actual information-theoretic bit footprint (6.68 BPW) of PTQTP K=4K=4, ExTernD achieves exact parity in imatrix-weighted energy preservation, despite being nearly twice as sparse (56%56\% vs 33%33\%). 4 Deployment notes The factors store today in llama.cpp’s TQ2_0 (2.06 bpw; its per-block fp16 scales absorb D into C’s row scales for free), and bitnet.cpp-style LUT kernels [9] apply directly to both. Emerging multiplication-free ASICs stand to exploit this the most, and the inner-rank expansion is designed with them in mind: the 3.43.4–6×6× op-count growth reads as slowdown only on multiplier-based datapaths, where a ternary add buys nothing over a multiply–add. 5 Limitations Everything here ran on a single AMD MI50 32 GB GPU, which bounds the evaluation. We have not tested above 4B or on matrices larger than ∼9216×2560 9216× 2560, so the behaviour of μ at 30B–70B+ is unknown. Evaluation is perplexity-only: no downstream benchmarks, no KLD, no long-context checks. We compare only against fixed-capacity baselines at the tensor level, but full end-to-end evaluation against established ternary schemes [5, 2] is the most valuable missing experiment: our structural and representational claims against them are proven (Sec. 2.1), but end-to-end task accuracy is not yet measured. The known levers (sweeps ∼5% 5\%, weighting ∼4% 4\%, per-matrix rank allocation) are unstacked, and no fused kernels exist. QAT is untried and is the obvious next lever: a short straight-through pass on an ExTernD initialization should recover accuracy, plausibly faster than QAT on a conventionally quantized model, since rank expansion hands the optimizer μ(m+n)/max(m,n)μ\,(m+n)/ (m,n) times as many trainable ternary entries per layer, already initialized near target. Whether STE gradients behave through two chained ternary factors is untested. Matching Q4_K or Q5_K bit-for-bit is not the objective; those are a familiar yardstick. The objective is a viable ternary LLM, where accuracy is set by μ rather than capped by the format, and this is best read as a step towards highly accurate ternary quantization rather than a finished recipe. 6 Conclusion Expanding the inner rank turns ternary representation from a fixed, lossy format with a hard accuracy ceiling into a family that provably converges to full precision as k grows. This is what makes ternary LLMs viable: accuracy becomes a dial (μ, continuous, with τ for sparsity) rather than a ceiling, so multiplication-free kernels and multiplication-free hardware can be pointed at any accuracy target, to our knowledge the only known way to get there. A simple greedy ALS already lands within ∼ 17–20% of Q4_K’s bit efficiency at matched accuracy, with identified, unstacked levers of comparable total size, while keeping inference multiplication-free. Appendix A Monotone residual decrease and convergence Proposition 1 (Monotone decrease). Let RiR_i be the residual before extracting component i, and let u,vu,v be any nonzero ternary vectors produced by the fit. With the optimal scale di=u⊤Riv/(‖u‖2‖v‖2)d_i=u R_iv/(\|u\|^2\|v\|^2), ‖Ri+1‖F2=‖Ri−diuv⊤‖F2=‖Ri‖F2−(u⊤Riv)2‖u‖22‖v‖22≤‖Ri‖F2,\|R_i+1\|_F^2=\|R_i-d_iuv \|_F^2=\|R_i\|_F^2- (u R_iv)^2\|u\|_2^2\,\|v\|_2^2\;≤\;\|R_i\|_F^2, with strict decrease whenever u⊤Riv≠0u R_iv≠ 0. Proof. Expand ‖R−duv⊤‖F2=‖R‖F2−2du⊤Rv+d2‖u‖2‖v‖2\|R-d\,uv \|_F^2=\|R\|_F^2-2d\,u Rv+d^2\|u\|^2\|v\|^2 and substitute the minimizing d. ∎ Proposition 2 (Convergence to arbitrary accuracy). Augment each step to take whichever of (a) the ALS solution and (b) the best single-entry pair u=ei⋆,v=sign((R)i⋆j⋆)ej⋆u=e_i ,v=sign((R)_i j )\,e_j at the largest-magnitude residual entry gives the larger decrease. Then ‖Rk‖F2≤(1−1mn)k‖A‖F2⟶ 0.\|R_k\|_F^2\;≤\; (1- 1mn )^k\,\|A\|_F^2\; \;0. Proof. Option (b) with ‖u‖=‖v‖=1\|u\|=\|v\|=1 removes (u⊤Rv)2=maxijRij2≥‖R‖F2/mn(u Rv)^2= _ijR_ij^2≥\|R\|_F^2/mn. The augmented step removes at least this much, giving the geometric bound. ∎ Corollary 1. For every ε>0 >0 there is a finite inner rank k with ‖A−Bdiag(D)C‖F<ε\|A-Bdiag(D)C\|_F< : the decomposition reaches the bf16 weights arbitrarily closely, with error monotone in k. No fixed number of ternary planes has this property; the expansion is what removes the information bottleneck. In practice the safeguard never activates, since the ALS decrease is far larger than the single-entry bound, and empirical convergence is much faster than geometric in mnmn (99% energy at μ≈2.5μ≈ 2.5–33). References [1] H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y. Wu, F. Wei. BitNet: Scaling 1-bit Transformers for Large Language Models. arXiv:2310.11453, 2023. [2] S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, F. Wei. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits. arXiv:2402.17764, 2024. [3] F. Li, B. Liu, X. Wang, B. Zhang, J. Yan. Ternary Weight Networks. arXiv:1605.04711, 2016. [4] C. Zhu, S. Han, H. Mao, W. J. Dally. Trained Ternary Quantization. ICLR, 2017. arXiv:1612.01064. [5] H. Xiao, R. Yang, Q. Yang, W. Xu, Z. Li, Y. Su, Z. Liu, H. Yang, N. Wong. PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models. arXiv:2509.16989, 2025. [6] X. Yan, W. Wu, Z. Wu, S. Yang, Z. Gao, M. Huang. PT2-LLM: Post-Training Ternarization for Large Language Models. arXiv:2510.03267, 2025. [7] E. Frantar, S. Ashkboos, T. Hoefler, D. Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. arXiv:2210.17323, 2022. [8] J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, S. Han. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. MLSys, 2024. arXiv:2306.00978. [9] J. Wang et al. Bitnet.cpp: Efficient Inference Framework for 1-bit LLMs. https://github.com/microsoft/BitNet, 2024.