Paper deep dive

High-probability guarantees for linear accessibility in feature superposition

Enrico Vompa

Year: 2026Venue: arXiv preprintArea: stat.MLType: PreprintEmbeddings: 52

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 9/10/2026, 2:55:14 AM

Summary

This paper establishes high-probability bounds for linear accessibility in neural network feature superposition by framing the problem as compressed sensing. It proves that the required dimension scales linearly with the number of active features (d=O(k log m)) under subgaussian noise, improving upon prior worst-case quadratic limits. The authors validate these theoretical bounds using Gaussian-tail approximations and numerical tests, demonstrating that linear decoding is feasible under specific geometric and noise constraints, with implications for sparse autoencoders and neural interpretability.

Entities (10)

Enrico Vompa · author · 100%Feature Superposition · concept · 98%Compressed Sensing · methodology · 95%Linear Decoding · methodology · 95%Tallinn University of Technology · organization · 95%Elhage et al. · citation · 90%Garg et al. · citation · 90%LLMs · model · 90%RMSNorm · technique · 90%Sparse Autoencoders · model · 90%

Relation Signals (6)

Enrico Vompa → affiliatedwith → Tallinn University of Technology

confidence 95% · Affiliation: Tallinn University of Technology, Estonia

Linear Accessibility → modeledas → Compressed Sensing

confidence 95% · By framing linear accessibility as a compressed sensing problem

Dimension Scaling → scalesas → O(k log m)

confidence 95% · proving the sufficient dimension scales linearly (d=O_{ε}(k log m))

Feature Superposition → constrainedby → Cross-feature interference

confidence 90% · cross-feature interference constrains the linear accessibility of simultaneously active features

LLMs → exhibit → Feature Superposition

confidence 90% · neural networks leverage feature superposition... LLMs can represent features in denser superposition

Sparse Autoencoders → evaluatedby → Linear Representation Hypothesis

confidence 85% · providing a framework for evaluating sparse autoencoders... These results quantify the geometric constraints of the linear representation hypothesis

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear accessibility of simultaneously active features. By framing linear accessibility as a compressed sensing problem, we derive high-probability bounds for fixed supports under subgaussian noise, proving the sufficient dimension scales linearly ($d=O_{\varepsilon}(k \log m)$) rather than prior worst-case quadratic limits. We then validate these bounds across system parameters through Gaussian-tail approximations. These results quantify the geometric constraints of the linear representation hypothesis, providing a framework for evaluating sparse autoencoders, compositional generalization, and neural interpretability.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

49,538 characters extracted from source content.

Expand or collapse full text

High-probability guarantees for linear accessibility in feature superposition Enrico Vompa envomp@taltech.e Affiliation: Applied Artificial Intelligence Group Affiliation: Tallinn University of Technology, Estonia Abstract Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear accessibility of simultaneously active features. By framing linear accessibility as a compressed sensing problem, we derive high-probability bounds for fixed supports under subgaussian noise, proving the sufficient dimension scales linearly (d=Oε(klog⁡m)d=O_ (k m)) rather than prior worst-case quadratic limits. We then validate these bounds across system parameters through Gaussian-tail approximations. These results quantify the geometric constraints of the linear representation hypothesis, providing a framework for evaluating sparse autoencoders, compositional generalization, and neural interpretability. 1 Introduction Stochastic separation theorems show that in high dimensions, any given point in a random set can be separated from the others by a hyperplane with high probability, even if the number of points in the set grows exponentially with the dimension (Sidorov and Zolotykh, 2020). Though traditionally applied to single features, these concentration principles extend to multiple simultaneous features. As formalized by compressed sensing, where a vast number of features (m) can be encoded into a smaller dimensional space (d) assuming a sparse subset of support size k is active, neural networks leverage feature superposition to represent more features than dimensions (Elhage et al., 2022). This manifests as the model mapping related concepts closer together and pushing unrelated ones apart, either driven by the training objective or emerging naturally. In reality, frequent co-occurrence of features such as “Christmas” and “December” results in them having high similarity, which produces constructive interference and helps when decoding such features (Prieto et al., 2026). However, being overly reliant on such constructive interference works against true compositional reasoning, where the model must occasionally decode arbitrary combinations of “Christmas in July”. Under a linear decoding scenario, we apply the subgaussian Hoeffding inequality to derive high-probability bounds for arbitrary supports, showing that feature superposition scales linearly with active features, improving the prior worst-case quadratic limits (Garg et al., 2026). Additionally, we employ Gaussian-tail approximations to estimate failure rates in order to demonstrate that the theory applies in practice. Even though LLM spaces are globally not uniform (Ethayarajh, 2019), concept subspaces are (Cai et al., 2021), and can be effectively approximated by Gaussian distributions (Zhao et al., 2025). It is this very local uniformity that gives rise to representational geometry (linear representation hypothesis) in vision encoders (Radford et al., 2021) and LLMs (Park et al., 2025; Park et al., 2024), which encode features into composite structures (e.g., categorical, hierarchical, spatial, relational, etc). Our contributions are as follows: - We derive high-probability decoding guarantees for fixed supports in a noisy environment, showing that the required dimension scales as d=Oε(klog⁡m)d=O_ (k m) for fixed error and failure tolerances when the noise satisfies σ=Oε(1/log⁡m)σ=O_ (1/ m). - We interpret these results in the context of feature superposition. 2 Theory: compressed sensing framework We model the problem in a compressed sensing framework, where a high-dimensional sparse vector is mapped to a lower-dimensional measurement space. 2.1 Linear versus nonlinear encoding Definition 1 (Linear encoding with independent noise). Let x∈[−1,1]mx∈[-1,1]^m be a k-sparse input signal, let A=[a1,…,am]∈ℝd×mA=[a_1,…,a_m] ^d× m be a dictionary of unit ℓ2 _2-norm feature embeddings (‖ai‖2=1\|a_i\|_2=1), and let z∈ℝdz ^d denote an independent observation noise vector. The linear encoding maps the input and noise to a compressed d-dimensional sketch y via: y=Ax+z=∑j∈Sxjaj+zy=Ax+z= _j∈ Sx_ja_j+z where S⊆[m]S [m] represents the active feature support of size |S|≤k|S|≤ k. Under this formulation, the encoded representation y is simply a weighted sum of the active feature embeddings. An alternative to this framework is encoding nonlinear representations. Linearly encoded representations possess a wider margin of error during decoding, operating as a probabilistic structure. Geometrically, decoding a linear sketch is akin to measuring the cosine similarity between nearly orthogonal feature directions; slight measurement errors yield proportionally small perturbations in the recovered state. In contrast, nonlinear encoding strategies, such as storing the k largest coefficients alongside their indices, can achieve a high compression of d=O⁡(k)d=O(k) (Ba et al., 2010). However, slight errors during decoding of such structures (e.g., a corrupted index mapping) can completely break the structure, resulting in disparate representations rather than proportional deviations. Such a discontinuous, non-differentiable representation space renders gradient-based optimization intractable. 2.2 History of nonlinear decoding Classical compressed sensing demonstrates that a k-sparse vector in ℝmR^m can be embedded into approximately d=O⁡(klog⁡(m/k))d=O(k (m/k)) measurements, assuming a known measurement matrix (Candes and Tao, 2005). Here, recovery can be achieved in polynomial time using nonlinear decoding algorithms such as basis pursuit (ℓ1 _1-minimization) (Candès et al., 2006), which extends to noisy environments, provided the noise z is bounded by ‖z‖ℓ2≤ε||z||_ _2≤ (Candès, 2008). Assuming the measurement matrix satisfies the restricted isometry property with a constant δ2k<2−1 _2k< 2-1, this recovery is guaranteed to be stable in both the ℓ1/ℓ1 _1/ _1 and ℓ2/ℓ1 _2/ _1 regimes (Candès, 2008), with a matching lower bound in these regimes (Ba et al., 2010). Figure 1: Compressed sensing encoding and decoding pipeline. 2.3 Linear decoding Linear decoding requires features to be linearly accessible, meaning they are recovered by a fixed linear decoder. As opposed to relatively clean recovery as in nonlinear decoding, linear decoding may produce cross-feature interference on both inactive and active coordinates (Stevinson et al., 2025). Depending on its sign, this interference can create spurious inactive estimates or distort active estimates. Decoding can therefore be framed as the existence of a threshold which separates active features from inactive ones. Definition 2 (Linear decoding). Given a compressed sketch y∈ℝdy ^d and a dictionary A∈ℝd×mA ^d× m, linear decoding is defined by the single-step matched-filter projection: x^=A⊤y x=A y such that the estimated coefficient for any feature i∈[m]i∈[m] is recovered via the inner product x^i=⟨ai,y⟩ x_i= a_i,y . The geometric limits of linear decoding are invariant to the magnitude of the compressed state y. By the bilinearity of the inner product, scaling the compressed state by a factor α>0α>0 scales the entire decoding projection equally ⟨ai,αy⟩=α⁡⟨ai,y⟩ a_i,α y =α a_i,y . Because the compressed state is a linear combination of active features (y=∑xjajy=Σ x_ja_j), this scaling magnifies both the target signal and the interference by the same factor. Figure 2 visualizes the recovery bounds in the empirical high-probability and worst-case scenarios, where a separation threshold could reside. Figure 2: Signal magnitudes and recovery bounds in an RMSNorm (Zhang and Sennrich, 2019) vector space. 2.4 The worst-case scenario of linear decoding Requiring the linear decoder to satisfy ‖A⊤Ax−x‖∞<ε\|A Ax-x\|_∞< uniformly for every k-sparse vector x∈[−1,1]mx∈[-1,1]^m yields an upper bound d=Oε(k2log⁡m)d=O_ (k^2 m); a nearly matching lower bound of d=Ωε(k2log⁡klog⁡mk)d= _ \! ( k^2 k mk ) applies even when allowing any arbitrary linear decoder B⊤B (Garg et al., 2026). This k2k^2 scaling is a geometric limitation representing how effectively low-rank projections can suppress cross-feature interference uniformly over all combinatorially many k-sparse inputs. 2.5 The empirical high-probability linear decoding scenario By analyzing the joint probability space of the dictionary, signal, and noise, we can bound the total cross-feature interference and noise. We partition the error tolerance ε , allocating a fraction α∈(0,1)α∈(0,1) to noise and (1−α)(1-α) to interference. A subgaussian tail bound controls both the interference IiI_i and the projected noise. Because subgaussian tails decay exponentially fast, applying a union bound across all features guarantees with high probability that both components remain bounded within allocated budget. To bound the maximum interference, we partition the sum of individual tail probabilities into two: ℙA,x(maxi∈ℐ⁡|Ii|≥(1−α)ε) _A,x ( _i |I_i|≥(1-α) ) (1) ≤2(m−|S|)exp⁡(−d(1−α)2ε22|S|)⏟inactive+|S|≥2⋅2|S|exp⁡(−d(1−α)2ε22(|S|−1))⏟active ≤ 2(m-|S|) (- d(1-α)^2 ^22|S| )_inactive+ I_|S|≥ 2· 2|S| (- d(1-α)^2 ^22(|S|-1) )_active ≤2mexp⁡(−d(1−α)2ε22|S|)≤δint ≤ 2m (- d(1-α)^2 ^22|S| )≤ _int The inactive term bounds the interference on inactive coordinates, where the interference is generated by all |S||S| active features, whose erroneous inclusion would produce false positives. The active term bounds the interference on active coordinates, where the interference is generated by the remaining |S|−1|S|-1 active features, whose erroneous exclusion would produce false negatives. The union bound includes both terms, so the resulting condition controls the interference failures relevant to both error types. Lemma 1 (Interference constraint). Let d,m,k∈ℕ>0d,m,k _>0 with 1≤|S|≤k≤m1≤|S|≤ k≤ m for an arbitrarily chosen index set S⊆[m]S [m]. Suppose the dictionary columns A=[a1,…,am]A=[a_1,…,a_m] are mean-zero and sampled independently and uniformly from the unit sphere Sd−1S^d-1, and the active signal coefficients (xj)j∈S(x_j)_j∈ S are drawn from a joint distribution Px∈SP_x _S satisfying |xj|≤1|x_j|≤ 1 independently of A (with xj=0x_j=0 for j∉Sj∉ S). For any error threshold ε>0 >0, fraction α∈(0,1)α∈(0,1), and failure probability δint∈(0,1) _int∈(0,1), if the dimension d satisfies: d≥2|S|(1−α)2ε2ln⁡(2mδint)d≥ 2|S|(1-α)^2 ^2 ( 2m _int ) then the maximum cross-feature interference is bounded by (1−α)ε(1-α) with probability at least 1−δint1- _int. For fixed α and δint _int independent of m,k,εm,k, , substituting |S|≤k|S|≤ k yields the dimension scaling dreq=Oε(klog⁡m)d_req=O_ (k m), mirroring the subset-incoherence and subgaussian concentration results from hyperdimensional computing theory (Thomas et al., 2021). Lemma 2 (Noise constraint). Let the dictionary A and signal x satisfy the conditions of Lemma 1. Suppose the observation noise z=(z1,…,zd)z=(z_1,…,z_d) is drawn from a product distribution Pz∈σP_z _σ independent of (A,x)(A,x), with independent, mean-zero coordinates satisfying the subgaussian norm bound ‖zn‖ψ2≤σ\|z_n\|_ _2≤σ for σ>0σ>0. For any fraction α∈(0,1)α∈(0,1), error tolerance ε>0 >0, and noise failure budget δnoise∈(0,1) _noise∈(0,1), if the subgaussian scale satisfies: σ≤cαεln⁡(2m/δnoise)σ≤ cα (2m/ _noise) then projected noise is bounded by αεα with probability at least 1−δnoise1- _noise, where c>0c>0 is an absolute constant. For fixed α and δnoise _noise, this bounds the maximum certified noise scales as σmax=Oε(1/log⁡m) _max=O_ (1/ m). Theorem 1 (Fixed-support decoding guarantee). Let ε>0 >0 and α∈(0,1)α∈(0,1), and let the failure budget be partitioned into δint,δnoise∈(0,1) _int, _noise∈(0,1) such that δint+δnoise=Δ<1 _int+ _noise= <1. If the dimension d and noise scale σ satisfy Lemma 1 and Lemma 2, respectively, then: ∀S⊆[m]with 1≤|S|≤k,supPx∈SsupPz∈σℙA,x,z(‖x^−x‖∞≥ε)≤Δ∀\,S [m]\ with\ 1≤|S|≤ k, _P_x _S _P_z _σP_A,x,z (\| x-x\|_∞≥ )≤ Meaning, for any arbitrary fixed support S across all valid distributions, the maximum estimation error exceeds ε with a joint failure probability of at most Δ . Corollary 1 (High-probability support recovery). Given the ℓ∞ _∞ error bound ‖x^−x‖∞≤ε\| x-x\|_∞≤ from Theorem 1 and a known minimum active coefficient xmin≤minj∈S⁡|xj|x_ ≤ _j∈ S|x_j|, support recovery via the thresholding rule S^=i:|x^i|>τ S=\i:| x_i|>τ\ is guaranteed if there exists a threshold τ satisfying: εinactive<xmin−εactive⟹εinactive⏟maxi∉S⁡|x^i|<τ<xmin−εactive⏟minj∈S⁡|x^j| _inactive<x_ - _active _inactive_ _i∉ S| x_i|<τ< x_ - _active_ _j∈ S| x_j| This proof is not a uniform guarantee over all combinatorially many supports. Full proof is in Appendix A. 3 Numerical tests: evaluating theory In our numerical tests (unless specified otherwise), all dictionary columns are drawn from Gaussian distribution and L2-normalized to unit length. For each sample, k active features are selected uniformly at random with coefficients set to 11, yielding noiseless observations. As the Hoeffding inequality provides a conservative bound for any dictionary uniformly distributed on the unit sphere Sd−1S^d-1, we can obtain a more precise estimate for the failure probability δgauss _gauss by modeling the interference terms independently (Equation 1) and computing their Gaussian approximation (Appendix A.3). Because this formulation bounds the tail-matching factor from below by a constant (0.50.5), the Gaussian approximation occasionally falls below the Hoeffding bound but never exceeds it. This approximation effectively captures the scaling dynamics of the system parameters. In configurations where the number of features is small relative to the dimension size, it yields a high probability of successful decoding; and instead when dimension size is relatively smaller, the probability decreases accordingly. With minimum active coefficient xmin=1x_ =1, the empirical maximum deviations are computed as maxi∈S⁡|1.0−x^i| _i∈ S|1.0- x_i| and maxi∉S⁡|x^i| _i∉ S| x_i| for active and inactive terms, respectively, from which we derive the empirical failure rate δemp _emp. As δ approaches zero, the term ln⁡(2m/δ) (2m/δ) grows indefinitely. To prevent finite-sample Monte Carlo noise from distorting the failure-rate ratio δgauss/δemp _gauss/ _emp, we restrict our evaluation to regimes with at least 100100 empirical failures. Because the empirical failure count NfailN_fail for rare events approximates a Poisson distribution, its relative statistical error (coefficient of variation) is defined by the ratio of the standard deviation to the mean, σ/μ≈Nfail/Nfail=1/Nfailσ/μ≈ N_fail/N_fail=1/ N_fail (Rubinstein and Kroese, 2016). By requiring Nfail≥100N_fail≥ 100, the estimated coefficient of variation of the empirical denominator is at most 1/100=10%1/ 100=10\%, reducing the influence of sampling noise on the failure-rate ratio. Meaning, 40,00040,000 trials enables evaluating high-probability decoding regimes down to a minimum failure rate of 0.25%0.25\% (a 99.75%99.75\% success rate). As recovery success approaches 100%, the δemp _emp converges to δgauss _gauss (Figure 3), demonstrating that the Gaussian approximation closely estimates the required dimension for high-probability decoding, providing empirical support that Hoeffding inequality governs the guarantee for high-probability (low failure-probability) linear accessibility. Similar results for Gaussian and Laplace11 1 Though mean-zero Laplace distribution is subexponential, because we project the dictionary columns onto a unit sphere, it becomes bounded and subsequently subgaussian. We also require uniformity, however. Here, central limit theorem comes into play which states that the interference asymptotically approaches a normal distribution, justifying the Gaussian approximation. distributions are shown in Appendix C. Figure 3: Tightness of Gaussian approximation for high-probability decoding on Rademacher distribution. 40,00040,000 trials were run for every combination of sparsity k∈[2,24]k∈[2,24], dictionary size m∈512,1024,2048,4096m∈\512,1024,2048,4096\ and dimension d∈64,128,256,512d∈\64,128,256,512\, evaluated against various error thresholds ε∈0.1,0.2,…,0.9 ∈\0.1,0.2,…,0.9\. When k=2k=2 and active coefficients have equal magnitude, a projection ⟨ai,aj⟩ a_i,a_j equally contributes to the failures of target i and j, so the union bound counts the same underlying event twice (negligible for larger k). 3.1 Support recovery Figures 4 and 5 evaluate success by the existence of a valid separating threshold (Corollary 1). In general vector space, where active features are simply summed together, the magnitude of the superposed state grows unbounded as the number of simultaneous features (k) increases. Subsequently, applying RMSNorm constrains the superposed state vector to a fixed magnitude of d d. Despite this difference in magnitudes, relative signal-to-interference ratios are identical as normalization scales the target signal and the background geometric interference by the same factor. Figure 4: Recovery of general and RMSNorm vector spaces across dimensions and sparsities. Our proofs partition the error tolerance ε into noise budget (αεα ) and interference budget ((1−α)ε(1-α) ). In practice, success simply requires that the sum of interference and noise remains below the tolerance ε . Because we model interference and additive noise as independent variables, Bienaymé’s identity dictates that the variance of their sum equals the sum of their individual variances (we derive the variance proxy for interference in Equation 14). Meaning, the system exhibits budget borrowing, where high interference can consume the total tolerance ε , leaving no room for noise, and vice versa. Figure 5: Recovery from noisy environment. 3.2 Representation learning In unsupervised dictionary learning, the optimization must simultaneously discover the feature directions and identify the unknown k-sparse activation support (Figure 6). OMP (orthogonal matching pursuit) greedily orthogonalizes the projection of linearly most accessible feature. LASSO (least absolute shrinkage and selection operator) runs a convex optimization using ℓ1 _1 penalty simultaneously over all features, which is dependent on linear accessibility as its starting point. LISTA (learned iterative shrinkage-thresholding algorithm) mimics the behavior of LASSO through parameters, making it also dependent on linear accessibility. Figure 6: Dictionary learning using OMP, LASSO and LISTA. Figures present the median, 5th, 25th, 75th, and 95th percentiles of the matched cosine similarities against ground truth directions. Active feature coefficients are drawn from a normal distribution. Full methodology in Section B.1. Taken together, interference (which governs linear accessibility) bounds the success of both linear and nonlinear decoding regimes. Though, decoding strategies respond differently to coefficient distributions. 4 Discussion Given that monotonic activation functions (e.g., ReLU) do not increase the capacity of linear accessibility (Garg et al., 2026), this could explain why sparse autoencoders (SAEs), which rely on a single-step linear projection and ReLU, fail at compositional generalization within a compressed sensing framework when trained on LLM activations (Barin-Pacela et al., 2026). Because Transformers can provably implement LISTA-type iterative nonlinear decoding algorithms (Liu et al., 2025), they have the capacity to operate in a nonlinear decoding regime. Meaning, LLMs can represent features in denser superposition that remain inaccessible to the linear decoding regime of SAEs. 4.1 Unknown nonlinear transform A parallel line of work, known as World Models, has emerged which force nonlinear observations into a linear transform of the world’s latent structure (Klindt et al., 2026). This is best understood through structural features, which are compositional properties that require nonlinearity to be recovered (Vompa and Tammet, 2026). Rather than assuming representations are inherently nonlinear, we hypothesize it is more akin to an unsolved problem where the base features are represented linearly, while higher-order properties are derived via iterative bottom-up perception or top-down reasoning (Vompa et al., 2026). Here, new information is linearly added into the residual stream, though, the basis system may change (Vompa and Tammet, 2026). 4.2 Limitations The connection between linear accessibility and LLM mechanics remains speculative, relying on external empirical evidence. By the central limit theorem (Lindeberg, 1922), the mean-zero, variance-normalized interference asymptotically converges to a normal distribution as |S||S| grows; coupled with normalization (typically RMSNorm), it becomes bounded and subsequently subgaussian when representations are mean-zero. Though, LLM representations are not necessarily mean-zero. Consequently, our high-probability concentration proofs can apply to LLM representational geometries provided their assumptions hold. We describe the conditions where linear accessibility is possible, the extent LLMs utilize this remains unknown. 5 Conclusion We characterize the high-probability guarantees of linear accessibility in feature superposition, showing that decoding requires d=O⁡(klog⁡m)d=O(k m) dimensions. These results clarify the geometric limits of the linear representation hypothesis and suggest how learned superposition can support compositional reasoning. References Ba et al. (2010) K. D. Ba, P. Indyk, E. Price, and D. P. Woodruff Lower bounds for sparse recovery. In SODA, External Links: Link Cited by: §2.1, §2.2. Barin-Pacela et al. (2026) V. Barin-Pacela, S. Joshi, I. Camacho, S. Lacoste-Julien, and D. Klindt Stop probing, start coding: why linear probes and sparse autoencoders fail at compositional generalisation. In 2nd Workshop on Compositional Learning: Safety, Interpretability, and Agents, External Links: Link Cited by: §4. Cai et al. (2021) X. Cai, J. Huang, Y. Bian, and K. Church Isotropy in the contextual embedding space: clusters and manifolds. In International Conference on Learning Representations, External Links: Link Cited by: §1. Candès et al. (2006) E. J. Candès, J. K. Romberg, and T. Tao Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics 59 (8), p. 1207–1223. External Links: Document, Link Cited by: §2.2. Candès (2008) E. J. Candès The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique 346 (9), p. 589–592. External Links: ISSN 1631-073X, Document, Link Cited by: §2.2. Candes and Tao (2005) E. Candes and T. Tao Decoding by linear programming. In IEEE Transactions on Information Theory, External Links: Link Cited by: §2.2. Elhage et al. (2022) N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah Toy models of superposition. Transformer Circuits Thread. External Links: Link Cited by: §1. Ethayarajh (2019) K. Ethayarajh How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In EMNLP-IJCNLP, External Links: Link Cited by: §1. Garg et al. (2026) N. Garg, J. Kleinberg, and K. Peng How many features can a language model store under the linear representation hypothesis?. In COLT, External Links: Link Cited by: §1, §2.4, §4. Klindt et al. (2026) D. Klindt, Y. LeCun, and R. Balestriero When does lejepa learn a world model?. External Links: 2605.26379, Link Cited by: §4.1. Lindeberg (1922) J. W. Lindeberg Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung. Mathematische Zeitschrift 15, p. 211–225. External Links: Link Cited by: §4.2. Liu et al. (2025) R. Liu, R. Zhou, C. Shen, and J. Yang On the learn-to-optimize capabilities of transformers in in-context sparse recovery. In ICLR, External Links: Link Cited by: §4. Park et al. (2025) K. Park, Y. J. Choe, Y. Jiang, and V. Veitch The geometry of categorical and hierarchical concepts in large language models. In ICLR, External Links: Link Cited by: §1. Park et al. (2024) K. Park, Y. J. Choe, and V. Veitch The linear representation hypothesis and the geometry of large language models. In ICML, External Links: Link Cited by: §1. Prieto et al. (2026) L. Prieto, E. Stevinson, M. Barsbey, T. Birdal, and P. A. M. Mediano From data statistics to feature geometry: how correlations shape superposition. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. ICML. Cited by: §1. Rubinstein and Kroese (2016) R. Y. Rubinstein and D. P. Kroese Simulation and the monte carlo method. External Links: Document, Link Cited by: §3. Sidorov and Zolotykh (2020) S. Sidorov and N. Zolotykh Linear and fisher separability of random points in the d-dimensional spherical layer and inside the d-dimensional cube. IJCNN. Cited by: §1. Stevinson et al. (2025) E. Stevinson, L. Prieto, M. Barsbey, and T. Birdal Adversarial attacks leverage interference between features in superposition. In Mechanistic Interpretability Workshop at NeurIPS 2025, External Links: Link Cited by: §2.3. Thomas et al. (2021) A. Thomas, S. Dasgupta, and T. Rosing A theoretical perspective on hyperdimensional computing. Journal of Artificial Intelligence Research 72. External Links: Link Cited by: §2.5. Vershynin (2026) R. Vershynin High-dimensional probability: an introduction with applications in data science. 2 edition, Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press, Cambridge. Cited by: Appendix A. Vignat and Bhatnagar (2008) C. Vignat and S. Bhatnagar An extension of wick’s theorem. Statistics and Probability Letters. External Links: Link Cited by: §A.2. Vompa et al. (2026) E. Vompa, T. Tammet, and M. Vaishnav Beyond the linear separability ceiling: aligning representations in VLMs. Transactions on Machine Learning Research. External Links: Link Cited by: §4.1. Vompa and Tammet (2026) E. Vompa and T. Tammet The scaling properties of implicit deductive reasoning in transformers. External Links: Link Cited by: §4.1. Zhang and Sennrich (2019) B. Zhang and R. Sennrich Root mean square layer normalization. In NeurIPS, External Links: Link Cited by: Figure 2, Figure 2. Zhao et al. (2025) H. Zhao, H. Zhao, B. Shen, A. Payani, F. Yang, and M. Du Beyond single concept vector: modeling concept subspace in llms with gaussian distribution. ICLR. External Links: Link Cited by: §1. Appendix A Sufficient high-probability bound for linear decoding Grounded in the linear representation hypothesis, the dictionary A=[a1,…,am]∈ℝd×mA=[a_1,…,a_m] ^d× m encodes semantic features as normalized latent directions (‖ai‖2=1\|a_i\|_2=1). The observed state is a sparse linear superposition of these vectors y=Ax+zy=Ax+z, where z is additive observation noise. We analyze the linear decoder: x^=A⊤y x=A y (2) Let d,m,k∈ℕ>0d,m,k _>0 with 1≤k≤m1≤ k≤ m, where d is the ambient dimension (number of observations), m the total feature count, and k the maximum sparsity. Fix an arbitrary deterministic index set S⊆[m]S [m] with sparsity 1≤|S|≤k1≤|S|≤ k. We make the following assumptions throughout: - Dictionary columns a1,…,ama_1,…,a_m are mean-zero, and sampled independently and uniformly from the unit sphere Sd−1S^d-1. - The active signal coefficients (xj)j∈S(x_j)_j∈ S are drawn from any joint distribution Px∈SP_x _S, where SP_S denotes the broad class of all valid bounded coefficient distributions (|xj|≤1|x_j|≤ 1) that are independent of A. We set xj=0x_j=0 for j∉Sj∉ S. - Observation noise z=(z1,…,zd)z=(z_1,…,z_d) is drawn from a distribution Pz∈σP_z _σ, where σZ_σ denotes the class of product noise distributions independent of (A,x)(A,x) with independent, mean-zero coordinates satisfying the subgaussian norm bound ‖zn‖ψ2≤σ\|z_n\|_ _2≤σ, where σ>0σ>0. Our analysis of this decoding draws largely upon the high-dimensional probability techniques detailed in Vershynin (2026), from which the theorems utilized below are sourced. We fixed a support S to avoid the uniform-support combinatorial penalty, with inactive coordinates set to zero (xj=0x_j=0 for j∉Sj∉ S). For any target coordinate i∈[m]i∈[m], the number of interfering active features is denoted by si:=|S∖i|s_i:=|S \i\|. Because S is fixed, sis_i is a fixed quantity bounded by si=|S|−1s_i=|S|-1 when i∈Si∈ S, and si=|S|s_i=|S| when i∉Si∉ S. In either case, si≤|S|≤ks_i≤|S|≤ k. A.1 Error decomposition Let ε>0 >0 denote the maximum acceptable global decoding error. To analyze this error for a target coordinate i, we expand the observation model y=∑j∈Sxjaj+zy= _j∈ Sx_ja_j+z and subtract the true signal xix_i from the decoder estimate x^i=⟨ai,y⟩ x_i= a_i,y : x^i−xi x_i-x_i =⟨ai,y⟩−xi = a_i,y -x_i (3) =⟨ai,∑j∈Sxjaj+z⟩−xi = a_i, _j∈ Sx_ja_j+z -x_i =xi⟨ai,ai⟩+∑j∈S∖ixj⟨ai,aj⟩+⟨ai,z⟩−xi =x_i a_i,a_i + _j∈ S \i\x_j a_i,a_j + a_i,z -x_i =xi(⟨ai,ai⟩−1)+∑j∈S∖ixj⟨ai,aj⟩⏟Ii+⟨ai,z⟩⏟Ni =x_i( a_i,a_i -1)+ _j∈ S \i\x_j a_i,a_j _I_i+ a_i,z _N_i Because the dictionary vectors are normalized (‖ai‖2=1\|a_i\|_2=1), the self-term vanishes (xi(1−1)=0x_i(1-1)=0), isolating the decoding error to the cross-feature interference IiI_i and the projected noise NiN_i: x^i−xi=Ii+Ni x_i-x_i=I_i+N_i (4) We define ℐ:=i:si≥1I:=\i:s_i≥ 1\ as the subset of coordinates that face cross-talk (worst-case having a cardinality of m); for i∉ℐi , the interference is zero (si=0s_i=0 and Ii=0I_i=0). To separate the two contributions to the total error, we partition the global tolerance ε . We allocate a fraction α∈(0,1)α∈(0,1) to the noise, and (1−α)(1-α) to the interference. By the triangle inequality, if the total error exceeds ε , at least one component must necessarily exceed its allocated budget. This bounds the total failure event within the union of individual failures: |Ii+Ni|≥ε⊆|Ii|≥(1−α)ε∪|Ni|≥αε\|I_i+N_i|≥ \ \|I_i|≥(1-α) \∪\|N_i|≥α \ (5) Applying the union bound to these sets yields the decoupled probability bound: ℙ⁡(|Ii+Ni|≥ε)≤ℙ⁡(|Ii|≥(1−α)ε)+ℙ⁡(|Ni|≥αε)P(|I_i+N_i|≥ ) (|I_i|≥(1-α) )+P(|N_i|≥α ) (6) A.2 Interference concentration We evaluate the interference concentration in the joint probability space of the dictionary and the signal. For any fixed target coordinate i∈ℐi and any distribution Px∈SP_x _S, with S fixed, we condition on the signal x and the target vector aia_i realizations. Recall Ii=∑j∈S∖ixjUjI_i= _j∈ S \i\x_jU_j, where Uj=⟨ai,aj⟩U_j= a_i,a_j . Since aj∼Unif(Sd−1)a_j (S^d-1) and aia_i is a fixed unit vector, evaluating all moments allows us to bound the exponential moment. To do so, we count the ways to partition 2r2r factors into disjoint pairs. Through the sphere version of Wick’s theorem (Vignat and Bhatnagar, 2008), this count gives the even moments of UjU_j, which we then bound and substitute into the exponential series to bound its exponential moment. For each integer r≥1r≥ 1, Uj2r=∏k=12r⟨ai,aj⟩U_j^2r= _k=1^2r a_i,a_j . Let I2r=1,…,2rI_2r=\1,…,2r\ and let Π2r _2r be the set of partitions of I2rI_2r into r disjoint pairs. For a pairing σ∈Π2rσ∈ _2r, write σ⁡(k)σ(k) for the partner of k, and let I2r/σI_2r/σ contain one representative from each pair. Then: A[Uj2r∣ai]=Γ⁡(d/2)2rΓ(r+d/2)∑σ∈Π2r⏟|Π2r| terms∏k∈I2r/σ⟨ai,ai⟩⏟r factors, all 1=1⋅3⋯(2r−1)d(d+2)⋯(d+2r−2)≤(2r)!2rr!drE_A[U_j^2r a_i]= (d/2)2^r (r+d/2) _σ∈ _2r_ _2r terms _k∈ I_2r/σ a_i,a_i _r factors, all 1= 1· 3·s(2r-1)d(d+2)·s(d+2r-2)≤ (2r)!2^rr!\,d^r (7) Here Γ is the Gamma function, satisfying Γ⁡(z+1)=zΓ(z) (z+1)=z (z), and |Π2r| _2r denotes the number of pairings. Each pairing contributes 11 because ⟨ai,ai⟩=‖ai‖22=1 a_i,a_i =\|a_i\|_2^2=1. The final equality uses |Π2r|=1⋅3⋯(2r−1) _2r =1· 3·s(2r-1) and the Gamma recurrence to obtain the denominator d(d+2)⋯(d+2r−2)d(d+2)·s(d+2r-2). The inequality follows by bounding this denominator below by drd^r and using 1⋅3⋯(2r−1)=(2r)!/(2r!)1· 3·s(2r-1)=(2r)!/(2^rr!). To control the probability of large interference via the exponential moment method, we first bound A[eλUj∣ai]E_A[e^λ U_j a_i]. Since |Uj|≤1|U_j|≤ 1, we can expand the exponential and average its series term by term. The odd moments vanish by symmetry, and the even-moment bound above yields for every λ∈ℝλ (we will choose the exact value later): A[eλUj∣ai] _A[e^λ U_j a_i] =1+λ22!A[Uj2∣ai]+λ44!A[Uj4∣ai]+λ66!A[Uj6∣ai]+⋯ =1+ λ^22!E_A[U_j^2 a_i]+ λ^44!E_A[U_j^4 a_i]+ λ^66!E_A[U_j^6 a_i]+·s (8) =1+∑r=1∞λ2r(2r)!A[Uj2r∣ai] =1+ _r=1^∞ λ^2r(2r)!E_A[U_j^2r a_i] (9) ≤1+∑r=1∞λ2r(2r)!(2r)!2rr!dr ≤ 1+ _r=1^∞ λ^2r(2r)! (2r)!2^rr!\,d^r (10) =1+∑r=1∞1r!(λ22d)r=exp⁡(λ22d) =1+ _r=1^∞ 1r! ( λ^22d )^r= ( λ^22d ) (11) Conditional on x,aix,a_i, the interference IiI_i is symmetric about zero, so its two tails have equal probability. For t>0t>0 and λ>0λ>0, we apply Markov’s inequality (Proposition 1.6.2) to eλIie^λ I_i and multiply by two. Conditional independence factors the expectation into a product, and the exponential-moment bound with parameter λxjλ x_j bounds each factor. Thus: ℙA(|Ii|≥t∣x,ai) _A(|I_i|≥ t x,a_i) =2ℙA(Ii≥t∣x,ai)=2ℙA(eλIi≥eλt∣x,ai) =2P_A(I_i≥ t x,a_i)=2P_A(e^λ I_i≥ e^λ t x,a_i) (12) ≤2A[eλIi∣x,ai]eλt=2e−λt∏j∈S∖iA[eλxjUj∣x,ai] ≤ 2 E_A[e^λ I_i x,a_i]e^λ t=2e^-λ t _j∈ S \i\E_A[e^λ x_jU_j x,a_i] Substituting Equation 11, with parameter λxjλ x_j, into Equation 12 yields: ℙA(|Ii|≥t∣x,ai) _A(|I_i|≥ t x,a_i) ≤2e−λt∏j∈S∖iexp⁡((λxj)22d) ≤ 2e^-λ t _j∈ S \i\ ( (λ x_j)^22d ) (13) =2e−λtexp⁡(∑j∈S∖iλ2xj22d) =2e^-λ t ( _j∈ S \i\ λ^2x_j^22d ) =2exp⁡(−λt+λ22d∑j∈S∖ixj2). =2 (-λ t+ λ^22d _j∈ S \i\x_j^2 ). Using |xj|≤1|x_j|≤ 1 and ∑j∈S∖ixj2≤si _j∈ S \i\x_j^2≤ s_i, we obtain the variance proxy: 1d∑j∈S∖ixj2≤sid 1d _j∈ S \i\x_j^2≤ s_id (14) For si>0s_i>0, choosing λ=dt/siλ=dt/s_i minimizes the quadratic exponent −λt+λ2si/(2d)-λ t+λ^2s_i/(2d). Thus, substituting Equation 14 into Equation 13 and choosing λ=dt/siλ=dt/s_i, we obtain: ℙA(|Ii|≥t∣x,ai)≤2exp⁡(−dt2si+dt22si)=2exp⁡(−dt22si)P_A (|I_i|≥ t x,a_i )≤ 2 (- dt^2s_i+ dt^22s_i )=2 (- dt^22s_i ) (15) We did this instead of applying the subgaussian Hoeffding inequality (Theorem 2.7.3) to avoid an unspecified absolute constant by proving it is 0.50.5. Furthermore, Theorem 2.2.1 assumes Rademacher variables, so we couldn’t use it directly, but the steps in this proof follow the same exponential moment method. Because the right hand side of this bound is deterministic and independent of the specific realizations of x and aia_i, taking the expectation over x∼Pxx P_x and ai∼Unif(Sd−1)a_i (S^d-1) preserves the bound in the joint (A,x)(A,x) probability space: ℙA,x(|Ii|≥t)≤2exp⁡(−dt22si)P_A,x (|I_i|≥ t )≤ 2 (- dt^22s_i ) (16) Intuitively, if the failure probability satisfies this bound for every individual realization of the signal and target vector, the expected failure probability across all random signals and vectors preserves this exact same bound. As such, to bound the worst-case cross-talk across all valid coordinates i∈ℐi , we seek the probability that the maximum interference exceeds our allocated threshold t=(1−α)εt=(1-α) . Applying the union bound over ℐI transforms the maximum into a sum of individual tail probabilities: ℙA,x(maxi∈ℐ⁡|Ii|≥(1−α)ε)≤∑i∈ℐℙA,x(|Ii|≥(1−α)ε)P_A,x ( _i |I_i|≥(1-α) )≤ _i P_A,x (|I_i|≥(1-α) ) (17) Where maxi∈∅⁡|Ii|=0 _i∈ |I_i|=0. We evaluate this sum by partitioning ℐI into inactive coordinates (i∉Si∉ S, where si=|S|s_i=|S|) and active coordinates (i∈Si∈ S, where si=|S|−1s_i=|S|-1). This splits the sum into two components, yielding the joint interference constraint with failure probability at most δint _int: ℙA,x(maxi∈ℐ⁡|Ii|≥(1−α)ε) _A,x ( _i |I_i|≥(1-α) ) (18) ≤2(m−|S|)exp⁡(−d(1−α)2ε22|S|)⏟inactive+|S|≥2⋅2|S|exp⁡(−d(1−α)2ε22(|S|−1))⏟active ≤ 2(m-|S|) (- d(1-α)^2 ^22|S| )_inactive+ I_|S|≥ 2· 2|S| (- d(1-α)^2 ^22(|S|-1) )_active ≤2mexp⁡(−d(1−α)2ε22|S|)≤δint ≤ 2m (- d(1-α)^2 ^22|S| )≤ _int The inactive term controls the interference budget on inactive coordinates, where the interference is generated by all si=|S|s_i=|S| active features, whose erroneous inclusion would produce false positives. The active term controls the interference budget on active coordinates, where the interference is generated by the remaining si=|S|−1s_i=|S|-1 active features, whose erroneous exclusion would produce false negatives. Here, the term vanishes when |S|=1|S|=1. The union bound includes both terms, so the resulting condition controls the interference failures relevant to both error types. Inverting the rightmost inequality for d yields the Lemma 1. A.3 Gaussian approximation of δint _int Recall that interference Ii=∑j∈S∖ixjUjI_i= _j∈ S \i\x_jU_j, where Uj=⟨ai,aj⟩U_j= a_i,a_j , is a linear combination of 1D projections. Since noise is absent, we simplify (1−α)ε=ε(1-α) = . By the projective central limit theorem (Theorem 3.3.9), 1D marginals of vectors uniformly distributed on the unit sphere converge to a normal distribution. Because |xj|≤1|x_j|≤ 1, summing the independent projection variances of 1/d1/d across the sis_i active interfering features results in a worst-case total interference variance of σ2=si/dσ^2=s_i/d. Standardizing Ii/σ≈⁡(0,1)I_i/σ (0,1) and defining the normalized threshold t=ε/σ=εd/sit= /σ= d/s_i, we apply the Gaussian tail bound (Proposition 2.1.2) to approximate the two-sided failure probability: ℙ⁡(|Ii|≥ε)=ℙ⁡(|Ii/σ|≥t) (|I_i|≥ )=P(|I_i/σ|≥ t) ≈2(1t2π)e−t2/2 ≈ 2 ( 1t 2π )e^-t^2/2 (19) =2(siε2πd)exp⁡(−dε22si) =2 ( s_i 2π d ) (- d ^22s_i ) For comparison, we define a tail-matching factor by matching the Gaussian approximation to the form of the Hoeffding envelope (Equation 16). By allocating the entire error budget to the interference term, the threshold for the Hoeffding envelope becomes t=εt= . Subsequently isolating a tail-matching factor ctailc_tail yields: 2exp⁡(−ctaildε2si) 2 (-c_tail d ^2s_i ) =2(siε2πd)exp⁡(−dε22si) =2 ( s_i 2π d ) (- d ^22s_i ) (20) −ctaildε2si -c_tail d ^2s_i =ln⁡(siε2πd)−dε22si = ( s_i 2π d )- d ^22s_i ctail c_tail =0.5−sidε2ln⁡(siε2πd) =0.5- s_id ^2 ( s_i 2π d ) To understand how an absolute constant can bound the tail-matching factor from below, let’s substitute εd/si=t d/s_i=t back, yielding: ctail=0.5−1t2ln⁡(1t2π)c_tail=0.5- 1t^2 ( 1t 2π ) (21) Figure A.1: ctail(t)c_tail(t) is bounded from below by 0.50.5 for t=εd/si≥1/2πt= d/s_i≥ 1/ 2π. To compute the Gaussian approximation for δint _int, we plug ctailc_tail back into the Hoeffding envelope, and apply the union bound M∈(m−|S|),|S|M∈\(m-|S|),|S|\ over the exponent: δgauss=2Mexp⁡(−max⁡(0.5,0.5−sidε2ln⁡(siε2πd))dε2si) _gauss=2M (- (0.5,0.5- s_id ^2 ( s_i 2π d ) ) d ^2s_i ) (22) This approximation captures the scaling dynamics of the system parameters. The floor of 0.50.5 within the exponent ensures that the numerical estimate does not exceed the baseline Hoeffding bound. The ctailc_tail term is then utilized to occasionally get a better numerical estimate, yielding more accurate numerical results in regimes where the Gaussian tail decays faster than the worst-case Hoeffding envelope. In configurations where the number of features is small relative to the dimension size, the exponential decay yields a high probability of successful decoding; and instead when dimension size is relatively smaller, the probability decreases accordingly. A.4 Noise concentration Conditional on A, because the projected noise Ni=⟨ai,z⟩=∑n=1dai,nznN_i= a_i,z = _n=1^da_i,nz_n is a sum of independent, mean-zero subgaussian variables, we can apply the subgaussian Hoeffding inequality (Theorem 2.7.3), where c>0c>0 is a constant: ℙz(|Ni|≥t∣A)≤2exp⁡(−ct2∑n=1d‖ai,nzn‖ψ22)P_z (|N_i|≥ t A )≤ 2 (- ct^2 _n=1^d\|a_i,nz_n\|_ _2^2 ) (23) By the definition of our noise class σZ_σ, the subgaussian norm of each coordinate is ‖zn‖ψ2≤σ\|z_n\|_ _2≤σ. When factoring out ai,na_i,n and using the fact that aia_i is a unit vector (‖ai‖2=1\|a_i\|_2=1), we get: ∑n=1d‖ai,nzn‖ψ22≤σ2∑n=1dai,n2=σ2 _n=1^d\|a_i,nz_n\|_ _2^2≤σ^2 _n=1^da_i,n^2=σ^2 (24) Substituting Equation 24 into Equation 23 with our allocated error threshold t=αεt=α , we obtain: ℙz(|Ni|≥αε∣A)≤2exp⁡(−cα2ε2σ2)P_z (|N_i|≥α A )≤ 2 (- cα^2 ^2σ^2 ) (25) Applying a union bound over all m coordinates, analogous to Equation 17, yields the conditional noise constraint: ℙz(maxi∈[m]⁡|Ni|≥αε|A)≤∑i=1mℙz(|Ni|≥αε|A)≤2mexp⁡(−cα2ε2σ2)≤δnoiseP_z ( _i∈[m]|N_i|≥α \, |\,A )≤ _i=1^mP_z (|N_i|≥α \, |\,A )≤ 2m (- cα^2 ^2σ^2 )≤ _noise (26) Since this conditional bound holds uniformly for every realization of A, it also holds under the joint distribution of (A,z)(A,z), with failure probability at most δnoise _noise. Inverting the rightmost inequality for σ yields the Lemma 2. Appendix B Extended methodology B.1 Dictionary learning To measure dictionary learning across varying dimensions (d∈64,128,256d∈\64,128,256\) and sparsities (k∈[1,50)k∈[1,50)) for a dictionary of size m=1024m=1024, we employ stochastic alternating minimization using batches of N=50,000N=50,000 samples generated via Y=DtrueXY=D_trueX per iteration. Sparse codes are estimated using OMP (constrained to k non-zeros), LASSO (optimized using 50 iterations of FISTA with Nesterov acceleration and spectral norm-derived step sizes), and LISTA (unrolled for 10 iterations using analytical weight matrices derived from the current dictionary). The dictionary is then updated from accumulated sufficient statistics XXTXX^T and YXTYX^T using ridge-regularized Method of Optimal Directions (MOD) with a penalty of 10−410^-4; degenerate atoms (‖di‖2<10−8\|d_i\|_2<10^-8) are dynamically reinitialized from random batch samples. Recovery is evaluated up to sign and permutation ambiguities by applying the Hungarian matching algorithm to the cost matrix −|DtrueTDpred|-|D_true^TD_pred|. Appendix C Gaussian approximation on other distributions (a) Gaussian distribution (b) Laplace distribution Figure A.2: Tightness of Gaussian approximation for high-probability decoding on other distributions. 40,00040,000 trials were run for every combination of sparsity k∈[2,24]k∈[2,24], dictionary size m∈512,1024,2048,4096m∈\512,1024,2048,4096\ and dimension d∈64,128,256,512d∈\64,128,256,512\, evaluated against various error thresholds ε∈0.1,0.2,…,0.9 ∈\0.1,0.2,…,0.9\.

Instant research discovery