Paper deep dive
Hierarchical Prototype Emergence in Modern Hopfield Models
Aditya Cowsik, Adithya Sriram
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/16/2026, 7:58:14 AM
Summary
This paper investigates hierarchical prototype emergence in modern dense Hopfield networks with polynomial activation functions. The authors analytically derive conditions for the stability of memories at different levels of a hierarchical tree structure (from specific leaf memories to generalized root prototypes). They demonstrate that by tuning the activation sharpness (polynomial degree) and the number of memories, the network can transition between memorization (recovering specific leaf patterns) and generalization (recovering emergent ancestor prototypes). Experiments on Fashion-MNIST validate these theoretical findings, showing distinct phases where the network recovers specific images, intermediate prototypes (boots, shoes), or global roots depending on network parameters.
Entities (8)
Relation Signals (7)
Adithya Sriram → affiliatedwith → Stanford University
confidence 99% · Affiliation: Department of Physics, Stanford University
Aditya Cowsik → affiliatedwith → Stanford University
confidence 99% · Affiliation: Department of Physics, Stanford University
Aditya Cowsik → coauthorof → Adithya Sriram
confidence 95% · Aditya Cowsik and Adithya Sriram contributed equally to this work.
Dense Hopfield Network → evaluatedon → Fashion-MNIST
confidence 95% · Experiment on Fashion-MNIST... The dense associate memory built from this dataset
Dense Hopfield Network → uses → Polynomial Activation
confidence 95% · dense Hopfield network with polynomial activation
Polynomial Activation → controls → Prototype Reconstruction
confidence 90% · the sharpness, n, still controls which level of the hierarchy is stable.
Spurious Minima → enables → Generalization
confidence 88% · spurious minima generated in the energy landscape... are being explored as a mechanism behind generalization
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Hierarchical correlations are a universal feature of any realistic model of data, and the question of how associative memory models may learn these correlations and generalize beyond them to construct new sensible images is an important step towards understanding more complex modern architectures such as diffusion models. We consider a hierarchical model for memories which are sampled and stored in a dense Hopfield network with polynomial activation. We analytically derive conditions for each level of this hierarchy to be locally stable - that is they are local energy minima. We use prototype reconstruction as a minimal model of generalization and we find that it takes only a quasi-polynomial amount of information to generalize beyond particular memories and even particular groups in the hierarchy. We observe a qualitatively analogous phase diagram in the number of memories, sharpness of the activation function (polynomial degree) for data from Fashion-MNIST.
Tags
Links
- Source: https://arxiv.org/abs/2609.12079v1
- Canonical: https://arxiv.org/abs/2609.12079v1
Trouble viewing inline? Open PDF directly →
Full Text
150,536 characters extracted from source content.
Expand or collapse full text
Hierarchical Prototype Emergence in Modern Hopfield Models Aditya Cowsik Thanks: Aditya Cowsik and Adithya Sriram contributed equally to this work. Affiliation: Department of Physics, Stanford University, Stanford, CA 94305, USA Adithya Sriram Thanks: Aditya Cowsik and Adithya Sriram contributed equally to this work. Affiliation: Department of Physics, Stanford University, Stanford, CA 94305, USA September 10, 2026 Abstract Hierarchical correlations are a universal feature of any realistic model of data, and the question of how associative memory models may learn these correlations and generalize beyond them to construct new sensible images is an important step towards understanding more complex modern architectures such as diffusion models. We consider a hierarchical model for memories which are sampled and stored in a dense Hopfield network with polynomial activation. We analytically derive conditions for each level of this hierarchy to be locally stable – that is they are local energy minima. We use prototype reconstruction as a minimal model of generalization and we find that it takes only a quasi-polynomial amount of information to generalize beyond particular memories and even particular groups in the hierarchy. We observe a qualitatively analogous phase diagram in the number of memories, sharpness of the activation function (polynomial degree) for data from Fashion-MNIST. Introduction — The advent of transformer based language models has revolutionized generative artificial intelligence technologies [1, 2, 3, 4]. Despite the incredible progress in their capabilities, exactly how these architectures learn and are capable of generating sensible content is an area of intense scrutiny [5]. In the specific context of diffusion models and image generation, how the model is able to generalize images from its training set and create entirely new and creative out-of-distribution images from a simple query is a fascinating question, and mechanistic descriptions of this process are just beginning to emerge [6, 7, 8, 9, 10, 11]. At the same time as these novel architectures emerge, Hopfield networks and associative memory models have also seen a number of interesting innovations [12, 13, 14, 15, 16, 17]. These models are content-addressable self correcting memories and the original Hopfield networks were shown to be able to reliably store a number of memories which scaled linearly in the number of neurons [12]. More recently, dense associative memories have emerged as a modern generalization of the original Hopfield model [18]. In these networks, the quadratic energy function is replaced with a higher order polynomial or exponential activation function. This sharpens the basin of attraction around the stored memories and boosts the storage capacity to super-linear or even exponential in the number of neurons [18, 19, 20]. Figure 1: Schematic of the hierarchical memory structure we consider in this work. Patterns seen by the network μ ξ^μ are the leaves (pictures of shoes) and generalized patterns ℓg (reconstructed prototypes) are those which are further up in the correlation network. In the toy model we consider, patterns are derived from their direct ancestors by applying noise to a fraction p of the bits of the ancestor pattern (Eq. 2). While in diffusion models, the task is to generalize from the training set and produce new patterns, the task in associative memory models is exactly the opposite. Yet there is an intriguing analytical correspondence between the two, as it was shown that diffusion models and dense associative memories have related energy functions and similar properties as dynamical systems [21, 9, 22]. This motivates a perspective of interpolating between these two and exploring a memorization-generalization transition. Understanding the structure of Hopfield networks could help us understand when they merely reproduce memorized data, and when they can generalize beyond what they have already seen. This question is closely related to the notion of capacity in generalized Hopfield models. Indeed, the spurious minima generated in the energy landscape of dense associative memories are being explored as a mechanism behind generalization [9, 23]. However, it is insufficient to say that generalization happens precisely when we exceed a capacity threshold. We adopt this perspective and explore a model of dense associative memory in which the encoded data contains a latent hierarchical correlation structure. Data with latent hierarchical structure is very common. This type of data can be viewed as storing "example" patterns, such as images of shoes. These example patterns would have emerged from more generalized "concepts" which sit further up in the correlation structure. The images of shoes are examples of a generalized “footwear concept" which is itself an example of a generalized “clothing concept" (see Fig. 1). Under the assumption of this latent correlational structure, and using prototype reconstruction as a minimal model for generalization, we attempt to answer the following two questions: 1) With hierarchically correlated memories, when do dense associative memory models memorize and remember patterns? 2) Can these dense associative memory models recover the generalized patterns from the underlying correlation structure? We find that not only can the network reliably recover the stored patterns, but new “spurious” patterns emerge as minima in the energy landscape and many of these patterns are exactly those which are further up in the correlation structure. This competition between memorization and generalization can be tuned by the network parameters. Our work sheds light on how hierarchical features in data can be learned and how that is precisely related to memorization/forgetting in an exactly solvable model. Before continuing, we review some related work. We first note that Hopfield networks with correlated patterns have been studied in previous work, though these models consider only quadratic activation functions, whereas the dense activation function is the crucial ingredient of our work [24, 25, 26, 27]. In the context of dense associative memories, the idea of hierarchical organization has begun to receive interest. Ref. [28] studied models where the architecture itself involved hierarchically organized hidden layers of dense associative memories. Other related work by [29, 30] and concerns the use of a neural networks in retrieving an archetype memory or concept given many training examples. Our contribution beyond these is to consider arbitrary depth trees and analyze the relative stability of memories at different tree levels. We also show how new memories and patterns may be generated by the spurious minima. Model Description — We consider a system of binary neurons, each of which is denoted by a variable σi _i which can take on values ±1± 1 [12]. The state of the entire system is denoted ∈−1,1N σ∈\-1,1\^N. A pattern to be stored, or memory, is denoted as ξ, where the ithi th index ξi _i is the state of the ithi th neuron in the memory. We define the following as the energy function for the system: E()=−∑μ=1MF(μ⋅), E( σ)=- _μ=1^MF( ξ^μ· σ), (1) where F(x)F(x) is an activation function which here takes in as input the dot product between the memory and the current state of the system. Here, we will consider polynomial activation functions, i.e. F(x)=xnF(x)=x^n. Also, M is the total number of encoded memories. Recovery of memories happens by performing local descent starting at a probe point 0 σ^0 until a fixed point (local minimum) is reached. In the toy model of data we consider, the memories are correlated in a tree structure, see Fig. 1. The central root prototype is denoted in the figure as 0g^0. Derived from this pattern are successive layers of prototypes, each generating the next layer. This continues arbitrarily many times until finally one reaches the bottom layer of the tree. In our model, this tree is taken to be of depth h. From each prototype, we derive K−1K-1 memories and so the tree is degree K (from the root we derive K memories). We take each derived memory to be generated by slightly corrupting its prototype with a finite density of noise e. This noise is modeled with a random vector where each entry ei∈±1e_i∈\± 1\ is i.i.d. and ℙ(ei=−1)=pP(e_i=-1)=p. We restrict to 0<p<1/20<p<1/2. Then, for example, a level ℓ prototype ℓg , which is derived from a level ℓ−1 -1 prototype ℓ−1g -1 takes the form ℓ=ℓ−1∘ =g -1 (2) where here ∘ denotes element-wise multiplication, i.e. giℓ=giℓ−1eig _i=g -1_ie_i. For simplicity we maintain a uniform correlation between prototypes and derived memories and a uniform degree, i.e. p and K are uniform throughout the tree. Importantly, we initialize the network with only leaf memories ξ. That is, in the energy function Eq. 1, the network only penalizes states based on their Hamming distance from the leaf memories. We remark on one important point. Often times in Hopfield models, the patterns represent bit configurations, such as the states of pixels in a black and white image. In that case, if a bit in the state is flipped "on", then the corresponding pixel is lit up. The model we present could indeed correspond to such a system and is a natural way of viewing things, but it could also correspond to something more general. For instance, two images depicting similar objects may look completely uncorrelated from a pixel-to-pixel point of view, but are in fact highly correlated in some latent space of objects. Our model is intended to capture these types of correlational structures as well, if the right variables are used as inputs. Memory Stability and Emergent Minima — For any pattern, its stability to perturbations is given by the following energy gap: ΔE()=E(−2σi^i)−E(). E( σ)=E( σ-2 _i e_i)-E( σ). (3) This quantity measures the gap in energy between the state of the system σ and the state with the bit at position i flipped. If the energy gap is positive for all possible flips, then the pattern is indeed a local minima. The statistics of this gap inform us as to which patterns are local energy minima under the random ensemble in consideration. In particular, we are interested in whether the mean energy gap is positive, i.e. ⟨ΔE⟩>0 E >0, and in the squared coefficient of variation, (v)2(C^v)^2, i.e. ratio between the energy gap variance and the mean squared. This is akin to an inverse signal to noise ratio and it sets the probability that a pattern has a negative energy gap. This analysis yields the familiar linear memory capacities of traditional Hopfield models and the superlinear capacities of dense associative memories [12, 31, 18, 19]. Figure 2: The squared coefficient of variation, (ℓv)2(C^v_ )^2, can be made asymptotically small at all ℓ simultaneously with the appropriate choice of sharpness, n. (a) Varying ℓ at a fixed n=6n=6 demonstrates how the stability in the intermediate K regime is largely ℓ independent, but beyond the crossover, patterns closer to the leaves become highly unstable. At the crossover all levels are approximately equally stable. (b) Varying the activation power n at fixed depth (ℓ=5 =5) causes the coefficient of variation to diverge when K<q−2K<q^-2, reach a minimum for larger K before saturating as K→∞K→∞. (c) Optimal value of n, n∗n^* vs K. (d) Tuning n to the optimal value n∗(K)n^*(K) (as shown in (c)), which minimizes (ℓv)2(C^v_ )^2 given other variables are fixed, leads to (ℓv)2∼1/K(C^v_ )^2 1/K at all depths (here shown for ℓ=5 =5). When σ corresponds to an encoded leaf memory ξ, stability is synonymous with memorization – the network can recover a stored pattern. On the other hand, the second question posed in the introduction pertains to whether the model can understand and recover the underlying correlational structure in the data. In this case, we can assess the energy gap statistics for when σ is an ancestor prototype, e.g. ℓg . Stability of ℓg to perturbations is prototype reconstruction and is what we consider to be generalization. We now discuss evaluation of the statistics of Eq. 3 for various σ. There are three types of memories whose stabilities in which we are interested: 1) ancestor prototypes ℓg where 0<ℓ<h0< <h, 2) the leaf memories μ ξ^μ, and 3) the root 0g^0. As mentioned earlier, the stabilities of these states are governed by the sign of the mean, as well as (ℓv)2(C^v_ )^2. Here, the subscript ℓ denotes the level of the prototype and due the symmetry of the model, all prototypes of the same level have the same statistics. For the parameter regime q2h=ω(n/N)q^2h=ω(n/ N), the mean and the variance have the following forms for any probe point which is a tree node: ⟨ΔE⟩ℓ E _ =2nNn−1∑d=h−ℓh+ℓCℓ(1)(d)qnd =2nN^n-1 _d=h- ^h+ C^(1)_ (d)q^nd (4) Var[(ΔE)ℓ] [( E)_ ] =4n2N2n−2∑(s,t1,t2)Cℓ(2)((,,,,,)) =4n^2N^2n-2 _(s,t_1,t_2)C^(2)_ ((s,t_1,t_2)) ×qn(2s+t1+t2)−2s(1−q2s). × q^n(2s+t_1+t_2)-2s(1-q^2s). (5) In these expressions, q=1−2pq=1-2p. In the second line, s,t1,t2s,t_1,t_2 are the distances from the probe σ, target 1 and target 2 respectively to their median point in the tree. The Cℓ(⋅)C_ ^(·) are combinatorial coefficients. Within the parameter regime described above, these expressions are valid up to an o(1)o(1) relative error. Further evaluation of the above equations and determination of the Cℓ(⋅)C_ ^(·) requires sifting through many tedious combinatorics. To organize these computations, we developed a diagrammatic formalism in analogy with how Feynman diagrams help organize the combinatorial problem of perturbation theory in field theory. We describe this formalism in detail, derive the above equations, and establish rigorous control over the approximation errors in the supplementary material. From Eq. 4, as the combinatorial coefficients Cℓ(1)C^(1)_ are always positive, the mean energy gap is strictly positive for any set of parameter values. Therefore the stabilities are entirely determined by the coefficients of variation. We first discuss the case of the ancestor memories, where ℓ is in the range 0<ℓ<h0< <h. In Fig. 2(a), we plot the analytical expression for (0<ℓ<hv)2(C^v_0< <h)^2 calculated for 0<ℓ<h0< <h, in the large h limit. Recall that this ℓ corresponds to ancestor memories which were never explicitly seen by the network. We find three distinct regimes in the parameter space as a function of K, and its ratio to powers of q. First, when Kq2<1Kq^2<1, the ancestor memories at all ℓ are highly unstable, as evidenced by a (0<ℓ<hv)2(C^v_0< <h)^2 which quickly diverges as K approaches 0. This is readily understood because the network does not have enough training examples to reconstruct the ancestor memories. Once Kq2>1Kq^2>1, the squared coefficient of variation becomes finite and falls to a minimum, before beginning to increase again as K is further increased past K>q−(2n−1)K>q^-(2n-1). Eventually, in the large K limit, the squared coefficient of variation saturates to an ℓ dependent plateau. Notably, adding more data does not improve recall unless the model capacity (n) is increased commensurately. The height of this plateau diverges rapidly with ℓ , indicating that the large number of training examples sufficiently interfere with one another and serve to "wash out" the differences between the different ancestor memories. The intermediate regime of K between q−2q^-2 and q−(2n−1)q^-(2n-1) is the most interesting. Here, we observe that (0<ℓ<hv)2(C_0< <h^v)^2 decreases roughly as ∼K−1 K^-1 until a minimum point. For large ℓ , the value of K at which this minimum point occurs saturates to a value K∗∝q−4n/3K^* q^-4n/3 and inverting this relation yields the optimal n∗n^* Fig. 2(c). Furthermore, the actual value of (ℓv)2(C^v_ )^2 at K∗K^* may be driven downwards with increasing n. In Fig. 2(d) we increase K and tune n=n∗(K)n=n^*(K) and see that minimum value (ℓv)2(C^v_ )^2 decreases as roughly ∝K−1 K^-1. What we therefore find is that in this parameter regime, the probability for an ancestor prototype to be unstable is decreasing in K, and so long as n is increased commensurately, this probability may be decreased indefinitely towards zero (minima in Fig. 2(b)). This ability to suppress (0<ℓ<hv)2(C_0< <h^v)^2 with K and n indicates how we should scale these parameters with N. Indeed, if we take K to be an appropriate superlinear function of N and n∝logNn N, then (0<ℓ<hv)2(C_0< <h^v)^2 will vanish with N. Subsequently, a straightforward application of Chebyshev’s inequality reveals that the probability for an ancestor prototype to be unstable vanishes. Furthermore, the stated parameter regime (q2h=ω(n/N)q^2h=ω(n/ N)) means that this ability to suppress (0<ℓ<hv)2(C_0< <h^v)^2 is valid up to h∝logNh N. In other words, with NΘ(logN)+o(1)N ( N)+o(1) amount of training examples, the dense Hopfield network is able to reconstruct prototypes and generalize up to a depth logN N. Next, let us discuss the stability of the leaf memories μ ξ^μ and the root prototype 0g^0, for which ℓ=h =h and ℓ=0 =0 respectively. The leaf stability corresponds entirely to a recall phenomenon rather than that of generalization. In contrast to the ancestor prototypes, we find that (ℓ=hv)2(C^v_ =h)^2 monotonically increases with K (green dashed line in Fig. 2(a)), indicating that leaf memory stability is highest when K is small. Unlike with the ancestor prototypes, the correlations between training examples damages the network’s ability to recall them individually. On the other hand for the root prototype, (ℓ=0v)2(C^v_ =0)^2 decreases monotonically in K (orange dashed line in Fig. 2(a)). In this case, every training example reinforces the root, so its stability is only improved by an increase in K. Figure 3: Energy landscape of a hierarchical Hopfield network along a root-to-leaf walk, computed to leading order in N from the exact tree moments. The probe traverses each parent–child edge along a shortest Hamming path. Integer positions ℓ correspond to the prototypes patterns. The top row shows the mean energy while the bottom row shows the mean energy cost of the next spin flip of the walk. Shaded regions (dotted edges) denote ±1σ± 1σ fluctuations over the disorder ensemble. The ⟨E⟩ E band scales as N−1/2N^-1/2 (shown for N=103N=10^3), while the gap band is N-independent. Columns show the three stability regimes of the branching ratio K for h=7h=7, n=3n=3, and flip probability p=0.25p=0.25: (a),(d) K=6K=6, fluctuations destabilize all but the deepest levels (only leaves are stable minima); (b),(e) K=15K=15, the gap band clears zero after every node and all levels are stable; (c),(f) K=60K=60, only the root remains stable, the relative fluctuations grow with depth toward the large-K plateau Cv2→q−2ℓ−1C_v^2\!→\!q^-2 -1. Note the symmetric-log scales in (c),(f), reflecting the geometric growth of the root basin, ∼(Kqn)h (Kq^n)^h. Fig. 3 shows the mean energy landscape as we probe locations from the root to a leaf (following Hamming-distance geodesics from parent to child), passing through all intermediate nodes along the way. Each level is located at a minimum of the average energy, superimposed on an overall trend towards lower/higher energy as we move from the root to the leaves. Subfigure (a) shows that small K leads to greater stability at the leaf, and in general moving down the hierarchy, while (b) shows the opposite effect at larger K. Correspondingly subfigures (d) and (e) show the average energy difference corresponding to a spin-flip, moving from the higher to lower prototypes. In (a) K is too small for any but the leaf prototype to be stable which corresponds to the shaded variance of ΔE E overlapping the x-axis (it’s sign is uncertain) while in (b) the distribution is gapped from 0. (c) and (f) show that once K becomes too large stability towards the leaves collapse entirely due to interference and only the prototypes near the root are stable. The local minima have basis of attraction which extend approximately halfway (in Hamming distance) to the immediate parent or child nodes. The sizes of these neighborhoods don’t depend strongly on the parameters though the overall trend effects the degree to which points prefer to flow to lower or higher nodes on the tree. Experiment on Fashion-MNIST — We extend our analytics on the exactly solvable tree model to a real dataset which displays some features of hierarchical correlations, Fashion-MNIST [32]. These images carry a natural multi-level structure with each image belonging to one of ten defined classes and those classes sharing relative similarity relations with each other. Additionally this dataset largely lacks translation or rotation symmetry which our simplified model is not designed for, but which a more general model should simultaneously manage [7]. We comment on this further in the discussion. The dense associate memory built from this dataset shows that the sharpness, n, still controls which level of the hierarchy is stable. Though the theoretical investigation primarily discusses a scaling of training examples, they also imply how for different ℓ , the (ℓv)2(C^v_ )^2 may be adjusted with n (see Fig. 2(b)). We binarize (per-pixel dithering) the images and store them as memories in a dense associative memory with polynomial activation, initialize the dynamics at a single, ankle boot query image, and follow energy descent to a fixed point. We sweep the sharpness, and the number of images per class (which we analogize to the branching factor, K). This traces out the phase diagram of Fig. 4 where we color each point by Hamming distance between the recovered fixed point and the original query. We find four well-separated regimes corresponding to distinct levels of the latent hierarchy. For large n or small K the activation is sharp enough that the query is pinned to the query image (the leaf, red) and the network purely memorizes. As n is lowered, the basin flows to an emergent boot prototype and then to the shoe prototype, neither of which are stored explicitly but appear as minima built from the interference of many examples as predicted by the tree model. For the smallest n (left) the flow collapses onto the global root memory (purple star), the over-generalized fixed point. Near the lower triple point the recovered shoe prototype visibly borrows features from the boot prototype while remaining recognizably a shoe. Indeed the shoe prototype appears at a smaller K than the boot prototype indicating that the basin it belongs to encompasses both types of footwear. This progression with decreasing n mirrors the memorization-to-generalization crossover of the toy model, and the existence of stable intermediate phases is the real-data counterpart of the emergent ancestor memories. Figure 4: Phase diagram of the dense-associative memory dynamics on Fashion-MNIST initialized at an ankle boot image over a range of activation sharpnesses and group sizes. Color encodes the Hamming distance between the query and the recovered fixed point (red, circle is exactly zero). The five markers locate representative fixed points, shown as images on the right with matching shape and color. Four well-separated regions appear: the original image (leaf node), an emergent boot prototype, an emergent shoe prototype, and the blank root prototype. At the lower triple point (teal plus), the shoe prototype takes on some features of the boot prototype (e.g. a lift at the front of the shoe) while remaining largely consistent with the blue triangle shoe prototype. Discussion — In this work, we have explored a memorization-generalization transition that occurs in dense Hopfield networks, where the emergent memory patterns arise as a direct consequence of the underlying correlation structure in the data. When the data to be stored is correlated in hierarchical manner, then the ancestor patterns of the encoded memories themselves emerge as stable memories in the network. In this way, a type of generalization has occurred by way of prototype reconstruction, where encoded memories serve as “examples” for the network to learn a more general “concept.” We explore this idea both through a direct analytical signal-to-noise ratio calculation and by numerically probing the energy landscape. These analyses elucidate that the level of generalization is a function of the number of correlated patterns and the activation function, e.g. for some number of training examples below a number which depends on the strength of correlations, all ancestor prototypes have a finite probability to be stable minima. Finally, we qualitatively demonstrate some aspects of this phenomena in a model of real data, Fashion-MNIST. This work may be extended in numerous directions. First, we have used only a simple one layer network. It would be interesting to explore what would change under more complicated architectures, such as multilayer Hopfield networks and even hierarchical Hopfield networks [14, 28]. In addition, given the enormous performance boost granted by the usage of self-attention and transformers in modern diffusion model architectures, it would be curious to ask how these types of layers could be used in these associative memories. For example, we performed numerical experiments on Fashion-MNIST, where the emergent memories yielded sensible patterns but in more complicated datasets, for example those with spatial symmetries, we found that that emergent memories became washed out and blurry. We hypothesize that usage of more complex architectures could generalize more complex hierarchically correlated data into discernable patterns. Indeed capturing latent structure was part of the philosophy behind the Boltzmann machine as an improvement on the original Hopfield model [33]. This work could be a starting point towards understanding the behavior of diffusion models as well as the attention mechanism in transformers. We also remark on the similarity between our data model and the ultrametric structure of the Gibbs states in a full replica symmetry broken spin glass [34, 35, 36]. In these f-RSB models, this ultrametric structure is an emergent property of the ruggedness of the energy landscape and the presence of exponentially many (in system size) stable local minima. What is interesting is that in the model we present, we also obtain an ultrametric structure of stable local minima, but this structure is "hard-coded" by the data model. We believe an analysis of the contrast of our model to that of a broken replica symmetry spin glass is an intriguing question for future work. Acknowledgements — We acknowledge helpful discussions with Surya Ganguli, Federico Ghimenti, Mason Kamb, Dmitry Krotov, Aditya Mahadevan, and Akshat Pandey. A.S. was supported in part by the Office of Naval Research Young Investigator Program (ONR YIP) under Award Number N00014-24-1-2098 and in part by a Packard Fellowship in Science and Engineering (PI: Vedika Khemani). A.S. also acknowledges support from the NSF graduate research fellowship and the ARCS Scholar award. A.C. acknowledges funding from the Simons Foundation (Award ID: 560571). Claude was used to generate the cartoon in Fig. 1 and both Claude and ChatGPT were used to assist with and proofread the analytical calculations done by the authors. References [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need (2023), arXiv:1706.03762 [cs.CL] . [2] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), edited by J. Burstein, C. Doran, and T. Solorio (Association for Computational Linguistics, Minneapolis, Minnesota, 2019) p. 4171–4186. [3] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale (2021), arXiv:2010.11929 [cs.CV] . [4] W. Peebles and S. Xie, Scalable diffusion models with transformers (2023), arXiv:2212.09748 [cs.CV] . [5] J. Simon, D. Kunin, A. Atanasov, E. Boix-Adserà, B. Bordelon, J. Cohen, N. Ghosh, F. Guth, A. Jacot, M. Kamb, D. Karkada, E. J. Michaud, B. Ottlik, and J. Turnbull, There will be a scientific theory of deep learning (2026), arXiv:2604.21691 [stat.ML] . [6] J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics (2015), arXiv:1503.03585 [cs.LG] . [7] M. Kamb and S. Ganguli, arXiv preprint arXiv:2412.20292 (2024). [8] H. Hunt, M. Kamb, and S. Ganguli, An exact information theory of generalization phase transitions in bayesian diffusion models (2026), arXiv:2607.08041 [cs.LG] . [9] B. Pham, G. Raya, M. Negri, M. J. Zaki, L. Ambrogioni, and D. Krotov, Memorization to generalization: Emergence of diffusion models from associative memory (2026), arXiv:2505.21777 [cs.LG] . [10] M. Niedoba, B. Zwartsenberg, K. Murphy, and F. Wood, Towards a mechanistic explanation of diffusion model generalization (2025), arXiv:2411.19339 [cs.LG] . [11] H. Cui, C. Pehlevan, and Y. M. Lu, A solvable model of learning generative diffusion: theory and insights (2025), arXiv:2501.03937 [cs.LG] . [12] J. J. Hopfield, Proceedings of the National Academy of Sciences 79, 2554 (1982), https://w.pnas.org/doi/pdf/10.1073/pnas.79.8.2554 . [13] D. Krotov, B. Hoover, P. Ram, and B. Pham, Modern methods in associative memory (2025), arXiv:2507.06211 [cs.LG] . [14] H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve, V. Greiff, D. Kreil, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, Hopfield networks is all you need (2021), arXiv:2008.02217 [cs.NE] . [15] M. D. Gaudio, F. Ghimenti, and S. Ganguli, Short-term plasticity recalls forgotten memories through a trampoline mechanism (2026), arXiv:2511.22848 [q-bio.NC] . [16] L. Lufkin, T. Figliolia, B. Millidge, and K. Krishnamurthy, Hybrid associative memories (2026), arXiv:2603.22325 [cs.LG] . [17] J. Steinberg and H. Sompolinsky, Sci. Rep. 12, 21808 (2022). [18] D. Krotov and J. J. Hopfield, Advances in neural information processing systems 29 (2016). [19] C. Lucibello and M. Mézard, Phys. Rev. Lett. 132, 077301 (2024). [20] M. S. Kafraj, D. Krotov, and P. E. Latham, A biologically plausible dense associative memory with exponential capacity (2026), arXiv:2601.00984 [q-bio.NC] . [21] L. Ambrogioni, arXiv preprint arXiv:2309.17290 (2023). [22] X. Niu, B. Bai, L. Deng, and W. Han, Beyond scaling laws: Understanding transformer performance with associative memory (2024), arXiv:2405.08707 [cs.LG] . [23] B. Hoover, Z. Shi, K. Balasubramanian, D. Krotov, and P. Ram, Dense associative memory with epanechnikov energy (2026), arXiv:2506.10801 [cs.LG] . [24] V. S. Dotsenko, Physica A: Statistical Mechanics and its Applications 140, 410 (1986). [25] A. Engel, Journal of Physics A: Mathematical and General, 23, 2587 (1990). [26] E. Agliari, A. Barra, A. De Antoni, and A. Galluzzi, Neural Netw 38, 52 (2013). [27] S. Sargolzaei and L. Rueda, in New Frontiers in Associative Memory Workshop at the International Conference on Learning Representations (ICLR) (2025). [28] D. Krotov, Hierarchical associative memory (2021), arXiv:2107.06446 [cs.NE] . [29] E. Agliari, L. Albanese, F. Alemanno, A. Alessandrelli, A. Barra, F. Giannotti, D. Lotito, and D. Pedreschi, Physica A 626, 129076 (2023). [30] E. Agliari, F. Alemanno, A. Barra, and G. D. Marzo, The emergence of a concept in shallow neural networks (2021), arXiv:2109.00454 [cond-mat.dis-n] . [31] T. Geszti, Physical models of neural networks (World Scientific Publishing, Singapore, Singapore, 1990). [32] H. Xiao, K. Rasul, and R. Vollgraf, Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017), cs.LG/1708.07747 . [33] D. Ackley, G. Hinton, and T. Sejnowski, Cogn. Sci. 9, 147 (1985). [34] M. Mezard, G. Parisi, and M. A. Virasoro, Spin glass theory and beyond: An introduction to the replica method and its applications, World Scientific Lecture Notes In Physics (World Scientific Publishing, Singapore, Singapore, 1987). [35] D. J. Amit, H. Gutfreund, and H. Sompolinsky, Physical Review Letters 55, 1530 (1985). [36] N. Parga and M. A. Virasoro, in World Scientific Lecture Notes in Physics (WORLD SCIENTIFIC, 1986) p. 436–443. Supplemental Material for “Hierarchical Prototype Emergence in Modern Hopfield Models” Aditya Cowsik and Adithya Sriram Department of Physics, Stanford University, Stanford, CA 94305, USA S1 Moment Formulas and Diagrammatic Formalism In this section we will derive the analytical results for the energy gap statistics. To begin, let us first formally define the model described in the main text. Definition 1 (Dense Hopfield Model with Hierarchical Memories). Let 0∈−1,1Ng^0∈\-1,1\^N be an iid Bernoulli(1/2) random vector. From 0g^0, generate K new datapoints by sampling random noise vectors ∈−1,1Ne∈\-1,1\^N, where each element eie_i is i.i.d. according to a Bernoulli distribution with parameter p, i.e. ℙ(ei=−1)=pP(e_i=-1)=p. We always take 0<p<1/20<p<1/2. The new datapoints are generated by element-wise multiplication with the roots, 1=0∘g^1=g^0 . To form the next layer, generate K−1K-1 new datapoints for each 1g^1 by sampling random noise vectors according to the same distribution. This process is repeated until h layers are generated. The patterns in the final layer, the leaf patterns, are denoted as μ ξ^μ and there are overall K(K−1)h−1K(K-1)^h-1 leaf patterns. The energy function for the dense Hopfield model is given by E()=−∑μ=1M(μ⋅)n E( σ)=- _μ=1^M ( ξ^μ· σ )^n (S1) where n is an even integer. We seek to calculate the statistics of the energy gap: ⟨(ΔE)m⟩ ( E)^m =⟨[2∑μ=1M∑koddn(nk)(ξiμσi)(∑j≠iξjμσj)n−k]m⟩ = [2 _μ=1^M _k\ odd^nn k( _i^μ _i) ( _j≠ iξ^μ_j _j )^n-k ]^m (S2) We will utilize the following result that allows us to more easily calculate the mmth moment of the energy gap. Theorem 2 (Approximate mmth Moment of Energy Gap). Let ⟨(ΔE~)m⟩ ( E)^m be ⟨(ΔE~)m⟩=2mnm∑μ1,…,μm=1M⟨∏a=1m(ξiμaσi)⟩(∏amNn−1q(n−1)d(μa,)) ( E)^m =2^mn^m _ _1,…, _m=1^M\; _a=1^m\,( _i _a _i) ( _a^mN^n-1q^(n-1)d( ξ _a, σ) ) (S3) For q2h=ω(mn/N)q^2h=ω(mn/ N), the mmth moment of the energy gap (Eq. S2) is approximated by ⟨(ΔE)m⟩≤⟨(ΔE~)m⟩⋅(1+o(1)), ( E)^m \ ≤\ ( E)^m ·(1+o(1)), (S4) We will prove this theorem upon introduction of two helper lemmas, the first of which allows us to restrict to k=1k=1 in Eq. S2, and the second of which which controls expectation values of terms of the form (∑j≠iξjμσj)n−k ( _j≠ iξ^μ_j _j )^n-k. First, we expand out the part of the expectation value argument raised to the m: =2m∑μ1,…,μm=1M∑k1,…,kmalloddn⟨∏a=1m(nka)(ξiμaσi)(∑j≠iξjμaσj)n−ka⟩ =2^m _ _1,…, _m=1^M\; _ subarrayck_1,…,k_m\\ all\ odd subarray^n\; _a=1^mn k_a\,( _i _a _i) ( _j≠ iξ _a_j _j )^n-k_a (S5) =2m∑μ1,…,μm=1M∑k1,…,kmalloddnT, =2^m _ _1,…, _m=1^M\; _ subarrayck_1,…,k_m\\ all\ odd subarray^n\;T_k, (S6) where T T_k =⟨∏a=1m(nka)(ξiμaσi)(∑j≠iξjμaσj)n−ka⟩. = _a=1^mn k_a\,( _i _a _i) ( _j≠ iξ _a_j _j )^n-k_a . (S7) We may approximate this formula by only considering the term of the sum where all ka=1k_a=1, i.e. only using T(1,1,…,1)T_(1,1,…,1). For a particular parameter regime, this leads to a controlled relative error on the moments of the energy gap. Lemma 3 (All ka=1k_a=1 approximation). For q2h=ω(mn/N)q^2h=ω(mn/ N), the mmth moment of the energy gap (Eq. S2) is approximated by 2m∑μ1,…,μm=1MT(1,1,…1)≤⟨(ΔE)m⟩≤ 2m∑μ1,…,μm=1MT(1,1,…1)⋅(1+ε) 2^m _ _1,…, _m=1^M\;T_(1,1,… 1)\ ≤\ ( E)^m ≤\ 2^m _ _1,…, _m=1^M\;T_(1,1,… 1)· (1+ ) (S8) where T(1,1,…,1)T_(1,1,…,1) is given by Eq. S7 and setting =(1,1,…,1)k=(1,1,…,1) and ε=o(1) =o(1) is a small relative error vanishing with N. The proof of this is given in Section S3.1. The proof is not complicated but is rather tedious. Next, let us write out T(1,1,…,1)T_(1,1,…,1). T(1,1…,1)=nm⟨∏a=1m(ξiμaσi)⟩⟨∏a=1m(∑j≠iξjμaσj)n−1⟩ T_(1,1…,1)=n^m _a=1^m( _i _a _i) _a=1^m ( _j≠ iξ _a_j _j )^n-1 (S9) The first factor ⟨∏a=1m(ξiμaσi)⟩ _a=1^m( _i _a _i) is simple to deal with. Recall that q=(1−2p)q=(1-2p) so the overlap between any two nodes at a path-distance d is ⟨ξiAξjB⟩=δijqd. _i^A _j^B = _ijq^d. (S10) The variable d counts the number of edges on the unique simple path, also the shortest path, between A and B. Thus, ⟨ξjμaσj⟩ _j _a _j is a simple function of the distance between μa ξ _a and σ, i,e. ⟨ξjμaσj⟩=qd(μa,). _j _a _j =q^d( ξ _a, σ). (S11) Thus, to determine the overall term ⟨∏am(ξiμaσi)⟩ _a^m( _i _a _i) , we simply count all the links which appear an odd number of times along the paths from σ to each of the μa ξ _a. The second type of term, ⟨∏am(∑j≠iξjμaσj)n−1⟩ _a^m ( _j≠ i _j _a _j )^n-1 is more complicated. In theory this requires calculating higher moment correlators between patterns. However, this term is well approximated by instead taking powers of the mean of the argument. Lemma 4 (Bulk Propagator). The bulk propagator ⟨∏am(∑j≠iξjμaσj)n−1⟩ _a^m ( _j≠ i _j _a _j )^n-1 is well approximated by its mean when q2h=ω(mn/N)q^2h=ω(mn/ N), i.e. [(N−1)!(N−1−m(n−1))!∏amq(n−1)d(μa,)]≤⟨∏am(∑j≠iξjμaσj)n−1⟩≤[∏amNn−1q(n−1)d(μa,)](1+ϵ¯), [ (N-1)!(N-1-m(n-1))! _a^mq^(n-1)d( ξ _a, σ) ]≤ _a^m ( _j≠ i _j _a _j )^n-1 ≤ [ _a^mN^n-1q^(n-1)d( ξ _a, σ) ](1+ ε), (S12) where ϵ¯=o(1) ε=o(1). We present the proof of this lemma in Section S3.2. Much of it involves the same series of steps as the proof of Section S3.1. Putting these two together, we may prove Theorem 2. Proof of Theorem 2. Unfolding the mmth moment of the gap, we obtain ⟨(ΔE)m⟩ ( E)^m =2m∑μ1,…,μm=1M∑k1,…,kmalloddn⟨∏a=1m(nka)(ξiμaσi)(∑j≠iξjμaσj)n−ka⟩ =2^m _ _1,…, _m=1^M\; _ subarrayck_1,…,k_m\\ all\ odd subarray^n\; _a=1^mn k_a\,( _i _a _i) ( _j≠ iξ _a_j _j )^n-k_a (S13) ≤2mnm∑μ1,…,μm=1M⟨∏a=1m(ξiμaσi)⟩⟨∏a=1m(∑j≠iξjμaσj)n−1⟩(1+ε) ≤ 2^mn^m _ _1,…, _m=1^M\; _a=1^m( _i _a _i) _a=1^m ( _j≠ iξ _a_j _j )^n-1 (1+ ) (S14) ≤2mnm∑μ1,…,μm=1M⟨∏a=1m(ξiμaσi)⟩[∏amNn−1q(n−1)d(μa,)](1+ϵ¯)(1+ε) ≤ 2^mn^m _ _1,…, _m=1^M\; _a=1^m( _i _a _i) [ _a^mN^n-1q^(n-1)d( ξ _a, σ) ](1+ ε)(1+ ) (S15) ≤⟨(ΔE~)m⟩(1+o(1)). ≤ ( E)^m (1+o(1)). (S16) In line 2, we used Lemma 3 and in line 3, we used Lemma 4. ∎ S1.1 Diagrams and Feynman Rules To each term in Eq. S3, we may associate a diagram. Let us use m=2m=2 as an illustrative example. When m=2m=2, within each term, present are σ and two μ ξ^μ, which may or may not be the same leaf memory. Above, two example diagrams are shown. The first diagram on the left corresponds to a case in which σ (orange) is a leaf memory and the common ancestors between each pair of σ, μ1 ξ _1 (black) and μ2 ξ _2 (black) are the same (i.e. the root). The diagram on the right corresponds to a case in which the common ancestor between μ1 ξ _1 and μ2 ξ _2 is different than the common ancestor between σ and either of the ξ. We mark the presence of, in the term, a factor of the form (ξiμaσi)( _i _a _i) with a dashed line which between σ and μa ξ _a following the unique path between them on the tree. Similarly, the factor (∑j≠iξjμaσj)n−1 ( _j≠ i _j _a _j )^n-1 is represented on the diagram by a solid line which follows the same unique path between σ and μa ξ _a. Note that going forward, as the branches of the tree which do not lead to either σ or any ξ are unimportant to the diagram weight, we omit them. For example, this will allow us to represent the diagrams from above as tripods. From the discussion in the previous subsection, we can associate to the lines of a diagram the following weights: • Solid line (bulk propagator). Represents the extensive overlap raised to the (n−1)(n-1)th power: (∑j≠i⟨σjξjμ1⟩)n−1→Nn−1q(n−1)ℓ ( _j≠ i _jξ _1_j )^n-1→ N^n-1\,q^(n-1)\, in the large-N limit. A solid line between nodes A and B carries a factor =Nn−1q(n−1)d(A,B). to32.45pt to4pt -2.0pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 0 0 L 39.37 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @fill @drawpath@unclippedM 0 0 M 2.77 0 C 2.77 1.53 1.53 2.77 0 2.77 C -1.53 2.77 -2.77 1.53 -2.77 0 C -2.77 -1.53 -1.53 -2.77 0 -2.77 C 1.53 -2.77 2.77 -1.53 2.77 0 Z M 0 0stroke:none @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @fill @drawpath@unclippedM 39.37 0 M 42.14 0 C 42.14 1.53 40.9 2.77 39.37 2.77 C 37.84 2.77 36.6 1.53 36.6 0 C 36.6 -1.53 37.84 -2.77 39.37 -2.77 C 40.9 -2.77 42.14 -1.53 42.14 0 Z M 39.37 0stroke:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope = N^n-1\,q^(n-1)\,d(A,B). (S17) • Dashed line (single-spin propagator). Represents the single-site overlap ⟨σiξiμ1⟩=qℓ _iξ _1_i =q . A dashed line between A and B carries =qd(A,B). to32.45pt to4pt -2.0pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 0 0 L 39.37 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @fill @drawpath@unclippedM 0 0 M 2.77 0 C 2.77 1.53 1.53 2.77 0 2.77 C -1.53 2.77 -2.77 1.53 -2.77 0 C -2.77 -1.53 -1.53 -2.77 0 -2.77 C 1.53 -2.77 2.77 -1.53 2.77 0 Z M 0 0stroke:none @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @fill @drawpath@unclippedM 39.37 0 M 42.14 0 C 42.14 1.53 40.9 2.77 39.37 2.77 C 37.84 2.77 36.6 1.53 36.6 0 C 36.6 -1.53 37.84 -2.77 39.37 -2.77 C 40.9 -2.77 42.14 -1.53 42.14 0 Z M 39.37 0stroke:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope = q^d(A,B). (S18) As mentioned, to calculate ⟨∏am(ξiμaσi)⟩ _a^m( _i _a _i) , we must count the links which appear an odd number of times along the paths from σ to each of the μa ξ _a. This manifests in a simple cancellation rule for the dashed lines. Any dashed line passing through multiple nodes of the tree can be factored as the product of the nearest-neighbor dashed lines along the path. This means that two dashed lines, which overlap in any segment, can be reduced to a shorter set of non-overlapping dashed lines. See the following two examples. = , = to43.08pt to14.63pt 7.31319pt -7.31319pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 9.84 0 C 9.84 5.44 5.44 9.84 0 9.84 C -5.44 9.84 -9.84 5.44 -9.84 0 C -9.84 -5.44 -5.44 -9.84 0 -9.84 C 5.44 -9.84 9.84 -5.44 9.84 0 Z M 0 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.00.0pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 49.21 0 C 49.21 5.44 44.81 9.84 39.37 9.84 C 33.93 9.84 29.53 5.44 29.53 0 C 29.53 -5.44 33.93 -9.84 39.37 -9.84 C 44.81 -9.84 49.21 -5.44 49.21 0 Z M 39.37 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 [rgb]0,0,0 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @inpgf@ignorespaces @transformcm1.00.00.01.00.0pt2.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 2.77) @stroke @drawpath@unclippedM 10.12 0 L 29.25 0fill:none @inpgf@ignorespaces @closescope @begingroup@_scopebegin=1 [rgb]0,0,0 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @inpgf@ignorespaces @transformcm1.00.00.01.00.0pt-2.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 -2.77) @stroke @drawpath@unclippedM 10.12 0 L 29.25 0fill:none @inpgf@ignorespaces @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope \;=\; to43.08pt to14.63pt 7.31319pt -7.31319pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 9.84 0 C 9.84 5.44 5.44 9.84 0 9.84 C -5.44 9.84 -9.84 5.44 -9.84 0 C -9.84 -5.44 -5.44 -9.84 0 -9.84 C 5.44 -9.84 9.84 -5.44 9.84 0 Z M 0 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.00.0pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 49.21 0 C 49.21 5.44 44.81 9.84 39.37 9.84 C 33.93 9.84 29.53 5.44 29.53 0 C 29.53 -5.44 33.93 -9.84 39.37 -9.84 C 44.81 -9.84 49.21 -5.44 49.21 0 Z M 39.37 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 0) @hbox64 @closescope @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope , to85.76pt to71.53pt -21.13957pt -7.31319pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 to0.0pt @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 74.34 83.14 L 34.97 43.77fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 77.27 80.21 L 37.9 40.84fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 80.21 77.27 L 40.84 37.9fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 83.14 74.34 L 43.77 34.97fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 74.34 74.34 L 113.71 34.97fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 77.27 77.27 L 116.64 37.9fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 80.21 80.21 L 119.58 40.84fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 83.14 83.14 L 122.51 43.77fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 33.14 39.37 L 33.14 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 37.29 39.37 L 37.29 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash0.0pt @begingroup@stroke-dasharray=none,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 41.45 39.37 L 41.45 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 45.6 39.37 L 45.6 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 120.59 38.13 L 100.9 -1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 115.64 40.61 L 95.95 1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 120.59 40.61 L 140.27 1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 115.64 38.13 L 135.32 -1.24fill:none @inpgf@ignorespaces @closescope @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 88.58 78.74 C 88.58 84.18 84.18 88.58 78.74 88.58 C 73.3 88.58 68.9 84.18 68.9 78.74 C 68.9 73.3 73.3 68.9 78.74 68.9 C 84.18 68.9 88.58 73.3 88.58 78.74 Z M 78.74 78.74 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.056.90552pt56.90552pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 78.74 78.74) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 49.21 39.37 C 49.21 44.81 44.81 49.21 39.37 49.21 C 33.93 49.21 29.53 44.81 29.53 39.37 C 29.53 33.93 33.93 29.53 39.37 29.53 C 44.81 29.53 49.21 33.93 49.21 39.37 Z M 39.37 39.37 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt28.45276pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 39.37) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 127.95 39.37 C 127.95 44.81 123.55 49.21 118.11 49.21 C 112.67 49.21 108.27 44.81 108.27 39.37 C 108.27 33.93 112.67 29.53 118.11 29.53 C 123.55 29.53 127.95 33.93 127.95 39.37 Z M 118.11 39.37 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.085.35828pt28.45276pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 118.11 39.37) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#F8000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#F8000 @fillstroke @drawpath@unclippedM 49.21 0 C 49.21 5.44 44.81 9.84 39.37 9.84 C 33.93 9.84 29.53 5.44 29.53 0 C 29.53 -5.44 33.93 -9.84 39.37 -9.84 C 44.81 -9.84 49.21 -5.44 49.21 0 Z M 39.37 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 108.27 0 C 108.27 5.44 103.86 9.84 98.43 9.84 C 92.99 9.84 88.58 5.44 88.58 0 C 88.58 -5.44 92.99 -9.84 98.43 -9.84 C 103.86 -9.84 108.27 -5.44 108.27 0 Z M 98.43 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.071.1319pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 98.43 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 147.64 0 C 147.64 5.44 143.23 9.84 137.8 9.84 C 132.36 9.84 127.95 5.44 127.95 0 C 127.95 -5.44 132.36 -9.84 137.8 -9.84 C 143.23 -9.84 147.64 -5.44 147.64 0 Z M 137.8 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.099.58466pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 137.8 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope \;=\; to85.76pt to71.53pt -21.13957pt -7.31319pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 to0.0pt @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 80.7 76.78 L 41.33 37.41fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 76.78 80.7 L 37.41 41.33fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 80.7 80.7 L 120.07 41.33fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 76.78 76.78 L 116.15 37.41fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 42.14 39.37 L 42.14 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 36.6 39.37 L 36.6 0fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 120.59 38.13 L 100.9 -1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 115.64 40.61 L 95.95 1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 120.59 40.61 L 140.27 1.24fill:none @inpgf@ignorespaces @closescope @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @setdash3.0pt,3.0pt0.0pt @begingroup@stroke-dasharray=3.0pt,3.0pt,stroke-dashoffset=0.0pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 115.64 38.13 L 135.32 -1.24fill:none @inpgf@ignorespaces @closescope @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 88.58 78.74 C 88.58 84.18 84.18 88.58 78.74 88.58 C 73.3 88.58 68.9 84.18 68.9 78.74 C 68.9 73.3 73.3 68.9 78.74 68.9 C 84.18 68.9 88.58 73.3 88.58 78.74 Z M 78.74 78.74 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.056.90552pt56.90552pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 78.74 78.74) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 49.21 39.37 C 49.21 44.81 44.81 49.21 39.37 49.21 C 33.93 49.21 29.53 44.81 29.53 39.37 C 29.53 33.93 33.93 29.53 39.37 29.53 C 44.81 29.53 49.21 33.93 49.21 39.37 Z M 39.37 39.37 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt28.45276pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 39.37) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#808080 @fillstroke @drawpath@unclippedM 127.95 39.37 C 127.95 44.81 123.55 49.21 118.11 49.21 C 112.67 49.21 108.27 44.81 108.27 39.37 C 108.27 33.93 112.67 29.53 118.11 29.53 C 123.55 29.53 127.95 33.93 127.95 39.37 Z M 118.11 39.37 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.085.35828pt28.45276pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 118.11 39.37) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#F8000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#F8000 @fillstroke @drawpath@unclippedM 49.21 0 C 49.21 5.44 44.81 9.84 39.37 9.84 C 33.93 9.84 29.53 5.44 29.53 0 C 29.53 -5.44 33.93 -9.84 39.37 -9.84 C 44.81 -9.84 49.21 -5.44 49.21 0 Z M 39.37 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.028.45276pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 39.37 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 108.27 0 C 108.27 5.44 103.86 9.84 98.43 9.84 C 92.99 9.84 88.58 5.44 88.58 0 C 88.58 -5.44 92.99 -9.84 98.43 -9.84 C 103.86 -9.84 108.27 -5.44 108.27 0 Z M 98.43 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.071.1319pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 98.43 0) @hbox64 @closescope @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 147.64 0 C 147.64 5.44 143.23 9.84 137.8 9.84 C 132.36 9.84 127.95 5.44 127.95 0 C 127.95 -5.44 132.36 -9.84 137.8 -9.84 C 143.23 -9.84 147.64 -5.44 147.64 0 Z M 137.8 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.099.58466pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 137.8 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope The second example is the same diagram from the right hand side the prior figure, with the middle groups omitted. For the solid lines, the factor of Nn−1N^n-1 arises due to the sum over the spin variables. This sum does not factor as a product over the path, so the factor of Nn−1N^n-1 is associated with the whole line rather than being broken up along the path. On the other hand the factors of qn−1q^n-1 do factor along the path which is why they come with an exponent of d(A,B)d(A,B). In other words, for any node C on the path between A and B we can factor Nn−1q(n−1)d(A,B) N^n-1\,q^(n-1)d(A,B) = A B = to71.6pt to28.33pt 7.31319pt -21.01251pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 9.84 0 C 9.84 5.44 5.44 9.84 0 9.84 C -5.44 9.84 -9.84 5.44 -9.84 0 C -9.84 -5.44 -5.44 -9.84 0 -9.84 C 5.44 -9.84 9.84 -5.44 9.84 0 Z M 0 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.00.0pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.0-3.75pt-17.6795pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 -5.19 -24.46) @hbox64 @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 88.58 0 C 88.58 5.44 84.18 9.84 78.74 9.84 C 73.3 9.84 68.9 5.44 68.9 0 C 68.9 -5.44 73.3 -9.84 78.74 -9.84 C 84.18 -9.84 88.58 -5.44 88.58 0 Z M 78.74 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.056.90552pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 78.74 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.052.86212pt-17.6795pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 73.15 -24.46) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 [rgb]0,0,0 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 10.12 0 L 68.62 0fill:none @inpgf@ignorespaces @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope Nn−1(q(n−1)d(A,C)⋅q(n−1)d(C,B)) N^n-1 (q^(n-1)d(A,C)· q^(n-1)d(C,B) ) = A C B . = to100.05pt to28.33pt 7.31319pt -21.01251pt to0.0pt @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @setlinewidth @begingroup@stroke-width=0.4pt @inpgf@ignorespaces @begingroup@_scopebegin=1 @closescope to0.0pt @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 9.84 0 C 9.84 5.44 5.44 9.84 0 9.84 C -5.44 9.84 -9.84 5.44 -9.84 0 C -9.84 -5.44 -5.44 -9.84 0 -9.84 C 5.44 -9.84 9.84 -5.44 9.84 0 Z M 0 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.00.0pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 0 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.0-3.75pt-17.6795pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 -5.19 -24.46) @hbox64 @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 68.9 0 C 68.9 5.44 64.49 9.84 59.06 9.84 C 53.62 9.84 49.21 5.44 49.21 0 C 49.21 -5.44 53.62 -9.84 59.06 -9.84 C 64.49 -9.84 68.9 -5.44 68.9 0 Z M 59.06 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.042.67914pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 59.06 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.038.7479pt-17.6795pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 53.62 -24.46) @hbox64 @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @begingroup@stroke=#000000 @begingroup@fill=#000000 @fillstroke @drawpath@unclippedM 127.95 0 C 127.95 5.44 123.55 9.84 118.11 9.84 C 112.67 9.84 108.27 5.44 108.27 0 C 108.27 -5.44 112.67 -9.84 118.11 -9.84 C 123.55 -9.84 127.95 -5.44 127.95 0 Z M 118.11 0 @inpgf@ignorespaces @closescope @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.085.35828pt0.0pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 118.11 0) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @begingroup@_scopebegin=1 @transformcm1.00.00.01.081.31488pt-17.6795pt @begingroup@transform=matrix(1.0 0.0 0.0 1.0 112.52 -24.46) @hbox64 @closescope @closescope @begingroup@_scopebegin=1 [rgb]0,0,0 @setlinewidth @begingroup@stroke-width=0.8pt @inpgf@ignorespaces @stroke @drawpath@unclippedM 10.12 0 L 48.94 0 M 69.17 0 L 107.99 0fill:none @inpgf@ignorespaces @closescope @closescope @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @inpgf@ignorespaces @discardpath @closescope @closescope . We now enumerate the Feynman rules for evaluation of ⟨(ΔE~)m⟩ ( E)^m . 1. Indicate the probe point σ on the tree. 2. Choose m leaf nodes, one for each replica of ΔE E, labeled by leaf indices μ1,…,μm _1,…, _m. 3. Draw a dotted and solid line between the probe and each μa ξ _a, going through the tree. 4. Apply the dotted line cancellation rule repeatedly until no pairs of overlapping dotted lines remain. 5. Assign propagator weights: 2nNn−1q(n−1)d(,μa)2nN^n-1q^(n-1)\,d( σ, ξ _a) per solid line and qd(X,Y)q^d(X,Y) per remaining dashed line. Here X and Y range over every pair of nodes necessary to cover all dashed lines, with any covering set yielding the same result. 6. Sum over all placements of the leaf indices μ1,…,μm _1,…, _m. For a homogeneous tree this depends only on the hierarchy of most recent common ancestors of the nodes. Each of those possibilities is associated with a multiplicity that appears as a symmetry factor. It is convenient to group by these factors and multiply by the appropriate combinatorial factor. S2 First Two Moments S2.1 Diagrams and Combinatorics For ⟨ΔE⟩ E , there is a single leaf index μ. The diagram consists of one solid and one dashed line from σ to μ ξ^μ, contributing OPENNn−1qnd(,μCLOSE)N^n-1q^nd( σ, ξ^μ) in total. There is only a single diagram topology in this case and since only one line is present, there are no cancellations. Let the depth of the probe point be ℓ . The sum over all placements of the leaf indices is equivalent to a sum over the number of diagrams with distance d(,μ)=d( σ, ξ^μ)=d. Denote this number by Cℓ(d)C_ (d). Then, for our setting with a homogenous tree of height h and degree K Cℓ(1)(d)=(K−1)h−ℓ,d=h−ℓ(K−2)(K−1)d−ℓ+h2−1,h−ℓ<d<h+ℓ(K−1)h,d=h+ℓ C_ ^(1)(d)= cases(K-1)^h- ,& d=h- \\ (K-2)(K-1) d- +h2-1,& h- <d<h+ \\ (K-1)^h,& d=h+ cases (S19) With this counting, the first moment is ⟨ΔE⟩ℓ=2nNn−1qn(h−ℓ)∑x=0ℓCℓ(1)(h−ℓ+2x)q2nx. E _ =2n\,N^n-1q^n(h- ) _x=0 C^(1)_ (h- +2x)\,q^2nx. (S20) For m=2m=2, the diagrams all have a tripod topology. Every such diagram can be labeled by a tuple (s,t1,t2)∈ℤ≥03(s,t_1,t_2) _≥ 0^3, where each element of the tuple is the distance between the probe, target 1, or target 2, and the junction which is the median point between the three nodes. Then, given the position of the probe σ, we must count all such tripods. For a tuple (s,t1,t2)(s,t_1,t_2) we label the count Cℓ((,,,,,))C_ ((s,t_1,t_2)). This yields ⟨(ΔE~)2⟩ℓ ( E)^2 _ =4n2N2n−2∑(s,t1,t2)Cℓ(2)((,,,,,))qn(2s+t1+t2)−2s =4n^2N^2n-2 _(s,t_1,t_2)C^(2)_ ((s,t_1,t_2))q^n(2s+t_1+t_2)-2s (S21) The coefficient of variation will depend on the square of the mean. However, the square of the mean, just like the second moment, will depend on a sum over all positions of two targets. That is, expanded out, the square of the mean will take a form much like that of the second moment, in that it will have the exact same combinatoric structure. ⟨ΔE~⟩ℓ2 E _ ^2 =4n2N2n−2∑d1,d2Cℓ(1)(d1)Cℓ(1)(d2)qn(d1+d2) =4n^2N^2n-2 _d_1,d_2C_ ^(1)(d_1)C_ ^(1)(d_2)q^n(d_1+d_2) (S22) =4n2N2n−2∑(s,t1,t2)Cℓ(2)((,,,,,))qn(2s+t1+t2) =4n^2N^2n-2 _(s,t_1,t_2)C^(2)_ ((s,t_1,t_2))q^n(2s+t_1+t_2) (S23) Here, for any junction placement, d1=s+t1d_1=s+t_1 and d2=s+t2d_2=s+t_2 and so d1+d2=2s+t1+t2d_1+d_2=2s+t_1+t_2. The only difference between this expression and that of the second moment is the factor of q−2sq^-2s in the second moment. This allows us to write the variance in a more general form: ⟨ΔE~2⟩ℓ−⟨ΔE~⟩ℓ2 E^2 _ - E _ ^2 =4n2N2n−2 =4n^2N^2n-2 ×∑(s,t1,t2)Cℓ(2)((,,,,,))qn(2s+t1+t2)−2s(1−q2s) × _(s,t_1,t_2)C^(2)_ ((s,t_1,t_2))q^n(2s+t_1+t_2)-2s(1-q^2s) (S24) Dividing the variance by the mean squared, and taking a square root yields the coefficient of variation. In the above formulas, we have used the approximate formulas for the both the second moment and the mean squared. The relative errors we determined for the moments do not readily transfer to a relative error on the variance. However, through some additional work, we obtain a similar result. That is, Lemma 5 (Relative Error on Variance). For q2h=ω(n/N)q^2h=ω(n/ N), Var(ΔE)≤4n2N2(n−1)(1+o(1))∑μ1μ2Mqn(2s+t1+t2)−2s(1−q2s). ( E)≤ 4n^2N^2(n-1)(1+o(1)) _ _1 _2^M\;q^n(2s+t_1+t_2)-2s(1-q^2s). (S25) The proof of this is shown in Section S3.3. The proof involves many near identical manipulations to that of Lemma 3 and Lemma 4. The exact form of Cℓ(2)C_ ^(2) will depend on whether σ is a leaf, an internal node or the central root. We present the countings below. Note that if t1=t2=0t_1=t_2=0, the targets overlap and the probe is connected to the target via two solid lines. On the other hand if s=t1=0s=t_1=0, then the probe and one of the targets overlaps and the probe is connected to the other target via a solid line and a dashed line. For readability, if the subscript from t is omitted, then t1=t2t_1=t_2. Additionally, the t1≠t2t_1≠ t_2 cases are symmetric. If σ is itself a leaf, i.e. ℓ=h =h, Cℓ=h(2)=(K−1)2h−1(K−2),h=s=t(K−2)(K−3)(K−1)2s−2,s∈[1,h−1],t=s(K−2)(K−1)h+t−1,s∈[h+1,2h−1],s+t=2h(K−2)2(K−1)(t2+3s−4)/2,s∈[1,h−1],t1=s,t2∈[s+2,s+4,…,s+2(h−s−1)](K−2)(K−1)s+h−1,s∈[1,h−1],t1=s,t2=2h−s(K−2)2(K−1)(s+3t)/2−2,t∈[1,h−1],s∈[t+2,t+4,…,2h−t−2](K−2)(K−1)t2/2−1,t2∈[2,4,…,2(h−1)],s=t1=0(K−2)(K−1)s/2−1,s∈[2,4,…,2(h−1)],t=0(K−1)h,2h=t2,t1=s=0(K−1)h,2h=s,t=01,s=t1=t2=0.C^(2)_ =h= cases(K-1)^2h-1(K-2),& subarraylh=s=t subarray\\[2.0pt] (K-2)(K-3)(K-1)^2s-2,& subarrayls∈[1,h-1],\ t=s subarray\\[2.0pt] (K-2)(K-1)^h+t-1,& subarrayls∈[h+1,2h-1],\\ s+t=2h subarray\\[2.0pt] (K-2)^2(K-1)^(t_2+3s-4)/2,& subarrayls∈[1,h-1],\ t_1=s,\\ t_2∈[s+2,\,s+4,\\ 8.19447pt\ …,\,s+2(h-s-1)] subarray\\[2.0pt] (K-2)(K-1)^s+h-1,& subarrayls∈[1,h-1],\\ t_1=s,\ t_2=2h-s subarray\\[2.0pt] (K-2)^2(K-1)^(s+3t)/2-2,& subarraylt∈[1,h-1],\\ s∈[t+2,t+4,…,\\ 2h-t-2] subarray\\[2.0pt] (K-2)(K-1)^t_2/2-1,& subarraylt_2∈[2,4,…,2(h-1)],\\ s=t_1=0 subarray\\[2.0pt] (K-2)(K-1)^s/2-1,& subarrayls∈[2,4,…,2(h-1)],\\ t=0 subarray\\[2.0pt] (K-1)^h,& subarrayl2h=t_2,\ t_1=s=0 subarray\\[2.0pt] (K-1)^h,& subarrayl2h=s,\ t=0 subarray\\[2.0pt] 1,& subarrayls=t_1=t_2=0 subarray cases. (S26) When 0<ℓ<h0< <h, the combinatorics are C0<ℓ<h(2)=(K−1)2t−1(K−2),s=0,t=h−ℓ(K−1)s+2t−1(K−2),s∈[1,h−ℓ−1],s+t=h−ℓ(K−2)(K−3)(K−1)2t−2,s∈[1,ℓ−1],t−s=h−ℓ(K−2)(K−1)2h−1,s=ℓ,t=h(K−1)h+t−1(K−2),s∈[ℓ+1,h+ℓ−1],s+t=h+ℓ(K−2)2(K−1)(s+3t+h−ℓ)/2−2,t∈[1,h−1],s∈[|h−ℓ−t|+2,|h−ℓ−t|+4,…,h+ℓ−t−2](K−2)2(K−1)(t2+3t1)/2−2,s∈[1,ℓ−1],t1−s=h−ℓ,t2−t1∈[2,…,2(ℓ−s−1)](K−2)(K−1)(t2+3t1)/2−1,s=0,t1=h−ℓ,t2∈[h−ℓ,h−ℓ+2,…,h−ℓ+2(ℓ−1)](K−2)(K−1)h+t1−1,s∈[1,ℓ−1],t1−s=h−ℓ,t2−t1=2(ℓ−s)(K−1)t1+h,s=0,t1=h−ℓ,t2=h+ℓ(K−1)h−ℓ,s=h−ℓ,t=0(K−2)(K−1)(s+h−ℓ)/2−1,s∈[h−ℓ+2,h−ℓ+4,…,h−ℓ+2(ℓ−1)],t=0(K−1)h,s=h+ℓ,t=0.C^(2)_0< <h= cases(K-1)^2t-1(K-2),& subarrayls=0,\ t=h- subarray\\[2.0pt] (K-1)^s+2t-1(K-2),& subarrayls∈[1,h- -1],\\ s+t=h- subarray\\[2.0pt] (K-2)(K-3)(K-1)^2t-2,& subarrayls∈[1, -1],\\ t-s=h- subarray\\[2.0pt] (K-2)(K-1)^2h-1,& subarrayls= ,\ t=h subarray\\[2.0pt] (K-1)^h+t-1(K-2),& subarrayls∈[ +1,h+ -1],\\ s+t=h+ subarray\\[2.0pt] (K-2)^2(K-1)^(s+3t+h- )/2-2,& subarraylt∈[1,h-1],\\ s∈[|h- -t|+2,|h- -t|+4,…,\\ h+ -t-2] subarray\\[2.0pt] (K-2)^2(K-1)^(t_2+3t_1)/2-2,& subarrayls∈[1, -1],\ t_1-s=h- ,\\ t_2-t_1∈[2,…,2( -s-1)] subarray\\[2.0pt] (K-2)(K-1)^(t_2+3t_1)/2-1,& subarrayls=0,\ t_1=h- ,\\ t_2∈[h- ,\,h- +2,\\ 8.19447pt\ …,\,h- +2( -1)] subarray\\[2.0pt] (K-2)(K-1)^h+t_1-1,& subarrayls∈[1, -1],\ t_1-s=h- ,\\ t_2-t_1=2( -s) subarray\\[2.0pt] (K-1)^t_1+h,& subarrayls=0,\ t_1=h- ,\ t_2=h+ subarray\\[2.0pt] (K-1)^h- ,& subarrayls=h- ,\ t=0 subarray\\[2.0pt] (K-2)(K-1)^(s+h- )/2-1,& subarrayls∈[h- +2,\,h- +4,\\ 8.19447pt\ …,\,h- +2( -1)],\\ t=0 subarray\\[2.0pt] (K-1)^h,& subarrayls=h+ ,\ t=0 subarray cases. (S27) Finally, when ℓ=0 =0, the combinatorics are Cℓ=0(2)=K(K−1)s+2t−2(K−2),s∈[1,h−1],s+t=hK(K−1)2h−1,s=0,t=hK(K−1)h−1,s=h,t=0. C^(2)_ =0= casesK(K-1)^s+2t-2(K-2),& s∈[1,h-1],s+t=h\\ K(K-1)^2h-1,& s=0,t=h\\ K(K-1)^h-1,& s=h,t=0 cases. (S28) With these exact combinatorics calculated, we will now make the simplifying assumption that K≫3K 3. This is a reasonable approximation with large training data sets, and will be a valid upper bound on the variance. This will allow us to collapse many of the expressions above to much simpler, easier to manipulate forms, as everywhere we make the substitution (K−1),(K−2),(K−3)→K(K-1),(K-2),(K-3)→ K. The simplified combinatorial coefficients are, for the first moment, C~ℓ(1)=Kd+h−ℓ2,h−ℓ≤d≤h+ℓ. C_ ^(1)=K d+h- 2, h- ≤ d≤ h+ . (S29) For the second moment, when ℓ=h =h, we obtain C~ℓ=h(2)=K(s+3t)/2,t∈[0,h],s∈[t,t+2,…, 2h−t]K(t2+3t1)/2,t1=s∈[0,h−1],t2∈[s+2,s+4,…, 2h−s] C^(2)_ =h= casesK^(s+3t)/2,& subarraylt∈[0,h],\\ s∈[t,\,t+2,\,…,\,2h-t] subarray\\[2.0pt] K^(t_2+3t_1)/2,& subarraylt_1=s∈[0,h-1],\\ t_2∈[s+2,\,s+4,\,…,\,2h-s] subarray cases (S30) When 0<ℓ<h0< <h, we get C~0<ℓ<h(2)=K2(h−ℓ)+2s,s∈[1,ℓ],t=h−ℓ+sK2(h−ℓ)−s,s∈[0,h−ℓ],s+t=h−ℓKh+t−c,t∈[1,h−1],c∈[0,min(ℓ−1,h−t−1)]s=ℓ+h−t−2c,K(t2+3t1)/2,s∈[0,ℓ−1],t1=h−ℓ+s,t2∈[t1+2,t1+4,…, 2h−t1]K(s+h−ℓ)/2,s∈[h−ℓ+2,…,h+ℓ] C_0< <h^(2)= casesK^2(h- )+2s,& subarrayls∈[1, ],\ t=h- +s subarray\\[2.0pt] K^2(h- )-s,& subarrayls∈[0,h- ],\\ s+t=h- subarray\\[2.0pt] K^h+t-c,& subarraylt∈[1,h-1],c∈[0, ( -1,h-t-1)]\\ s= +h-t-2c, subarray\\[2.0pt] K^(t_2+3t_1)/2,& subarrayls∈[0, -1],\ t_1=h- +s,\\ t_2∈[t_1+2,\,t_1+4,\\ 8.19447pt\ …,\,2h-t_1] subarray\\[2.0pt] K^(s+h- )/2,& subarrayls∈[h- +2,\\ 8.19447pt\ …,\,h+ ] subarray cases (S31) And in the last case, when ℓ=0 =0, we get C~ℓ=0(2) C_ =0^(2) =K2h−s,s∈[0,h],s+t=h = casesK^2h-s,&s∈[0,h],\ s+t=h cases (S32) S2.2 Coefficient of Variation and Stability Recall that the statistics for the energy gap that we require to determine stability for a pattern is given by the mean, and the coefficient of variation. The mean energy gap must be positive, as a pattern can only be stable if perturbing it increases the energy. As explained in the main text, from the form of eq. Eq. S20, we observe that for prototypes of any ℓ , this is true. As a result, the relative stabilities of prototypes at different ℓ may be understood by studying the coefficient of variation, ℓvC^v_ . With the combinatorial coefficients determined, we use the approximate formula for the variance given by Lemma 5 to obtain an upper bound for (ℓv)2(C^v_ )^2. For the mean squared which is in the denominator of (ℓv)2(C^v_ )^2, one danger we must be cognizant of is that we cannot use the simplified combinatorics for C~ℓ(1) C_ ^(1), as this will obscure the relation in S24. We may generically write this as (ℓv)2≤∑(s,t1,t2)C~ℓ(2)((,,,,,))qn(2s+t1+t2)−2s(1−q2s)(∑d=h−ℓh+ℓCℓ(1)(d)qnd)2(1+o(1)),(C^v_ )^2\;≤\; \, _(s,t_1,t_2) C^(2)_ ((s,t_1,t_2) )q^n(2s+t_1+t_2)-2s(1-q^2s) ( _d=h- ^h+ C^(1)_ (d)\,q^n\,d )^2(1+o(1)), (S33) where the o(1)o(1) error is from Lemma 5. This expression is plotted in Fig. 2 for various ℓ . The rigorous upper bound on the coefficient of variation allows us to use Chebyshev’s inequality to obtain a bound on the probability that the energy gap is negative for some spin flip. That is, ℙ(⋃i=1NΔEi<0) ( _i=1^N E_i<0 ) ≤∑iNℙ(ΔE<0) ≤ _i^NP ( E<0 ) (S34) ≤Nℙ(|ΔE−⟨ΔE⟩|>⟨ΔE⟩) ≤ NP (| E- E |> E ) (S35) ≤N(Cℓv)2. ≤ N(C^v_ )^2. (S36) S2.2.1 Prototype Reconstruction 0<ℓ<h0< <h Stability of prototypes with 0<ℓ<h0< <h indicates prototype reconstruction, which we take to be a form of generalization. When K is large, holding other parameters constant, then each sum in Eq. S33 involving a coefficient in Eq. S31 which contributes a positive power of K is dominated by its largest term. In both the numerator and denominator, this largest term goes as KhK^h, which cancels out and so the limiting value becomes a constant: (0<ℓ<hv)2→K→∞q−2ℓ−1. (C^v_0< <h)^2\; \,K→∞\,\;\,q^-2 -1\,. (S37) This expression only vanishes as q→1q→ 1. However, in the q→1q→ 1 limit, all levels of the tree merge as the correlations become perfect, and the examples only serve to reinforce the recall of the single root memory. However, as we see in Fig. 2, up to a point, 0<ℓ<hvC^v_0< <h decreases with K. This occurs in a particular regime of K, namely q−2≪K≪q−2n+1q^-2 K q^-2n+1. In this regime, for appropriate choice of scaling K,nK,n and h with N, we show that the tail probability for there to be a spin, which when flipped lowers the energy, vanishes. Theorem 6 (Coefficient of Variation for Ancestor Prototypes). For q2h=ω(2n/N)q^2h=ω(2n/ N) and K≫3K 3, and q−2≪K≪q−2n+1q^-2 K q^-2n+1, (0<ℓ<hv)2≤exp(2(h−ℓ)K−1)⋅(1−q2)⋅[K2q4n−2+(Kq2)−1]⋅(1+ν)⋅(1+o(1)), (C_0< <h^v)^2≤ ( 2(h- )K-1 )·(1-q^2)· [K^2q^4n-2+(Kq^2)^-1 ]·(1+ν)·(1+o(1)), (S38) where ν=O(K2q4n−2+Kq2n+(Kq2)−1)ν=O(K^2q^4n-2+Kq^2n+(Kq^2)^-1) and the o(1)o(1) piece results from the relative error on the variance (Lemma 5). Furthermore, for K∝N⋅κ(N)K N·κ(N), where κ(N)κ(N) is any arbitrarily slow growing function of N, there exist positive constants cnc_n and chc_h, such that for n=cnlogNn=c_n N and h=chlogNh=c_h N, ℙ(⋃i=1NΔEi<0)≤o(1). ( _i=1^N E_i<0 )≤ o(1). (S39) Thus, any given ancestor prototype has a vanishing probability to be unstable. The proof of this theorem is given in Section S3.4. This theorem is the formal statement of the prototype reconstruction result of this work. This statement implies that every prototype in the hierarchy has a finite probability to be a stable local minimum in the energy landscape, leading to a finite density of emergent minima of the Hopfield model. The term in the brackets of Eq. S38 primarily controls the coefficient of variation. If we minimize the term in the brackets with respect to K, then we obtain K∗K^*, i.e. K∗∝q−4n/3K^* q^-4n/3. S2.2.2 Memorization: ℓ=h =h Let us now study the stability of the leaf node. As before, we seek to study limiting values. If we first take the large K limit, then by similar arguments to the case above, we obtain (ℓ=hv)2→K→∞q−2h−1, (C_ =h^v)^2\; \,K→∞\,\;q^-2h-1, (S40) which is simply Eq. S37 with the substitution ℓ=h =h. Similarly, in the h→∞h→∞ limit, we once again demand that Kq2n−1<1Kq^2n-1<1 to prevent divergence. This can be seen simply from the terms corresponding to the K2sK^2s coefficient. In order to obtain a finite expression when taking the h→∞h→∞ limit, we demand that Kq2n−1<1Kq^2n-1<1, as otherwise the infinite series in the above expressions will diverge. Unlike the interior prototype case however, if we take the K→∞K→∞ limit now, assuming Kq2n−1<1Kq^2n-1<1, then ℓ=hvC_ =h^v vanishes. To see this, we observe that in each sum involving a different coefficient from Eq. S30, the dominant term is the lowest term of the sum. After carrying out some algebra, one finds that the leading form of (ℓ=hv)2(C^v_ =h)^2 has the form limh→∞(ℓ=hv)2∝K2q4n−2+Kq4n−4. _h→∞(C^v_ =h)^2 K^2q^4n-2+Kq^4n-4. (S41) The above expression at large K, due to the assumption that Kq2n−1→0Kq^2n-1→ 0, vanishes as this condition requires q2n−1q^2n-1 to vanish quicker than K diverges. S2.2.3 Root Prototype: ℓ=0 =0 Finally, let us analyze the stability of the root node. Here we obtain (ℓ=0v)2 (C^v_ =0)^2 =∑s=0h(Kq2)−s−K−s = _s=0^h(Kq^2)^-s-K^-s (S42) As all leaves are derived from the same root, they all serve to reinforce it stability. For Kq2>1Kq^2>1, the sum in the expression above is convergent at large h. The h→∞h→∞ limit of the above expression is limh→∞(ℓ=0v)2=Kq2Kq2−1−K−1. _h→∞(C^v_ =0)^2= Kq^2Kq^2-1- KK-1. (S43) In the large K limit this expression vanishes. Of the cases we have studied this is the only case in which the coefficient of variation vanishes and so, the root is the only absolutely stable memory at fixed n,h,qn,h,q. The regime where Kq2<1Kq^2<1 is one where the correlations are sufficiently weak and there are not enough training examples for the network to take advantage of them in order to reinforce the root. In this regime, the coefficient of variation diverges. S3 Additional Proofs and Derivations A central object that will appear repeatedly in the upcoming calculations are expectation values of the following form: ⟨∑j1,1,…,j1,n−k1jm,1,…,j1,n−kmN∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩. _ subarraycj_1,1,…,j_1,n-k_1\\ \\ j_m,1,…,j_1,n-k_m subarray^N _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a . (S44) The sum in the expectation value may be split into a contribution which result from all j being distinct, and a contribution from at least two jjs being the same. The benefit of this is that, upon using linearity of expectation, the expectation value of terms in the first of these sums will factorize. ⟨∑j1,1,…,j1,n−k1jm,1,…,j1,n−kmN∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩ _ subarraycj_1,1,…,j_1,n-k_1\\ \\ j_m,1,…,j_1,n-k_m subarray^N _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a =∑all jr,a distinct∏a=1mq(n−ka)d(μa,)+∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩. = _all j_r,a distinct _a=1^mq^(n-k_a)d( ξ _a, σ)+ _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a . (S45) In order to facilitate the upcoming proofs, we will make use of the following helper lemma which places bounds on the second term on the RHS. We use the following shorthand notation da=d(μa)d_a=d( ξ _a σ). Lemma 7 (Bounding Collision Terms). The second term of Eq. S45 is non-negative and upper bounded by ∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩≤(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sR(q−2hN−mn⋅eRlog2)s. _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a ≤ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^R ( q^-2h N-mn· eR 2 )^s. (S46) Proof. Non-negativity is easily proven as the correlations are all positive and so the expectation values must all be greater than or equal to 00. To derive the upper bound, we will decompose the summation into colliding and distinct indices. That is, every term in the summation involves some set of indices among which there are collisions (i.e. the jr,aj_r,a are the same), and then the remaining indices are all distinct. The colliding indices need not all be the same, but for jr,aj_r,a in the set of colliding indices, there is at least one other jr′,a′j_r ,a for which jr,a=jr′,a′j_r,a=j_r ,a . With this logic, and setting R=∑a=1mn−kaR= _a=1^mn-k_a, the sum may be written as ∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩ _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a =∑sR∑S⊂(r,a):|S|=s∑π⊢S|B|≥2∀B∑uBB∈πv(a,r)(a,r)∉S distinct∏B∈π⟨∏(a,r)∈BξuBμaσuB⟩∏(a,r)∉S⟨ξv(a,r)μaσv(a,r)⟩. = _s^R _ subarraycS⊂\(r,a)\\\ :|S|=s subarray _ subarraycπ S\\ |B|≥ 2∀ B subarray _ subarraycu_B_B∈π\\ \v_(a,r)\_(a,r)∉ S\\ distinct subarray _B∈π _(a,r)∈ B _u_B _a _u_B _(a,r)∉ S _v_(a,r) _a _v_(a,r) . (S47) We use s to indicate the number of indices amongst whome there are collisions. Then, we choose a subset S of size s amongst all the tuples (r,a)(r,a). This subset S is further decomposed into subsets, and among those subsets, j(r,a)j_(r,a) is equal. Thus, S=⊔B∈πuBS= _B∈πu_B. Now, we shall upper bound this expression by replacing the expectation over the colliding indices with 11. With this, and using the dashed line propagator, we obtain ∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩ _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a ≤∑sR∑S⊂(r,a):|S|=s∑π⊢S|B|≥2∀B∑uBB∈πv(a,r)(a,r)∉S distinct∏(a,r)∉Sqda ≤ _s^R _ subarraycS⊂\(r,a)\\\ :|S|=s subarray _ subarraycπ S\\ |B|≥ 2∀ B subarray _ subarraycu_B_B∈π\\ \v_(a,r)\_(a,r)∉ S\\ distinct subarray _(a,r)∉ Sq^d_a (S48) =∑sR∑S⊂(r,a):|S|=s∑π⊢S|B|≥2∀B(N−1R−s+|π|)(R−s+|π|)!∏(a,r)∉Sqda = _s^R _ subarraycS⊂\(r,a)\\\ :|S|=s subarray _ subarraycπ S\\ |B|≥ 2∀ B subarrayN-1 R-s+|π|(R-s+|π|)! _(a,r)∉ Sq^d_a (S49) =(∏a=1mq(n−ka)da)∑sR∑S⊂(r,a):|S|=s∑π⊢S|B|≥2∀B(N−1R−s+|π|)(R−s+|π|)!∏(a,r)∈Sq−da = ( _a=1^mq^(n-k_a)d_a ) _s^R _ subarraycS⊂\(r,a)\\\ :|S|=s subarray _ subarraycπ S\\ |B|≥ 2∀ B subarrayN-1 R-s+|π|(R-s+|π|)! _(a,r)∈ Sq^-d_a (S50) ≤(∏a=1mq(n−ka)da)∑sRq−2hs(Rs)∑b=1⌊s/2⌋S2(s,b)(N−1R−s+b)(R−s+b)! ≤ ( _a=1^mq^(n-k_a)d_a ) _s^Rq^-2hsR s _b=1 s/2 S_2(s,b)N-1 R-s+b(R-s+b)! (S51) In the third line, we made the replacement ∏(a,r)∉Sqda=∏a=1mq(n−ka)da∏(a,r)∈Sq−da _(a,r)∉ Sq^d_a= _a=1^mq^(n-k_a)d_a _(a,r)∈ Sq^-d_a. In the fourth line, we use the fact that q−da≤q−2hq^-d_a≤ q^-2h for any dad_a. We also replace the sums with appropriate counts as the sums no longer depend on any other properties of the subsets beyond their size. SrS_r denotes the r-associated Stirling number of the second kind. Next we use the fact that s−b≥s/2s-b≥ s/2 to bound the product of binomial coefficients: (N−1R−s+b)(R−s+b)!≤(N−1R)R!(1N−R)s/2. N-1 R-s+b(R-s+b)!≤N-1 RR! ( 1N-R )^s/2. (S52) Substituting, we get ≤(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sRq−2hs(Rs)(1N−R)s/2∑b=1⌊s/2⌋S2(s,b) ≤ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^Rq^-2hsR s ( 1N-R )^s/2 _b=1 s/2 S_2(s,b) (S53) ≤(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sR(q−2hN−R⋅eRlogs)s ≤ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^R ( q^-2h N-R· eR s )^s (S54) To obtain the second line, we used the partition bound on S2S_2 and the binomial bound, and collected terms. Next, we use the fact that R≤mnR≤ mn to lower bound the N−R N-R term, and that logs≥log2 s≥ 2, which yields ≤(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sR(q−2hN−mn⋅eRlog2)s ≤ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^R ( q^-2h N-mn· eR 2 )^s (S55) ∎ S3.1 Proof of Lemma 3 Proof. With Eq. S7, we write the mmth moment as ⟨(ΔE)m⟩ ( E)^m =2m∑μ1,…,μm=1MT(1,1,…1)(1+∑k1,…,kmallodd≠(1,1…,1)nTT(1,1,…1)). =2^m _ _1,…, _m=1^M\;T_(1,1,… 1) (1+ _ subarrayck_1,…,k_m\\ all\ odd\\ ≠(1,1…,1) subarray^n T_kT_(1,1,… 1) ). (S56) The term in the parenthesis is the relative error. As the correlations are all positive, notice that T≥0T_k≥ 0 for any k. Thus the relative error is always positive, which yields the lower bound of the lemma. We can factorize T_k to yield T T_k =∏a=1m(nka)⟨∏a=1m(ξiμaσi)⟩⟨∏a=1m(∑j≠iξjμaσj)n−ka⟩ = _a=1^mn k_a\, _a=1^m( _i _a _i) _a=1^m ( _j≠ iξ _a_j _j )^n-k_a (S57) =∏a=1m(nka)⟨∏a=1m(ξiμaσi)⟩⟨∑j1,1,…,j1,n−k1jm,1,…,j1,n−kmN∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩. = _a=1^mn k_a\, _a=1^m( _i _a _i) _ subarraycj_1,1,…,j_1,n-k_1\\ \\ j_m,1,…,j_1,n-k_m subarray^N _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a . (S58) Following Eq. S45, we obtain =∏a=1m(nka)⟨∏a=1m(ξiμaσi)⟩[∑all jr,a distinct∏a=1mq(n−ka)da+∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩] = _a=1^mn k_a\, _a=1^m( _i _a _i) [ _all j_r,a distinct _a=1^mq^(n-k_a)d_a+ _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a ] (S59) We use the shorthand d(μa,)=dad( ξ _a, σ)=d_a. Let us now write the ratio between an arbitrary k case and the case of =(1,1,…,1)k=(1,1,…,1). ε=T(1,1,…,1) _k= T_kT_(1,1,…,1) =∏a=1m(nka)⟨∏a=1m(ξiμaσi)⟩[∑all jr,a distinct∏a=1mq(n−ka)da+∑≥2jr,a same⟨∏a=1m∏r=1n−kaξjr,aμaσjr,a⟩]∏a=1m(n1)⟨∏a=1m(ξiμaσi)⟩[∑all jr,a distinct∏a=1mq(n−1)da+∑≥2jr,a same⟨∏a=1m∏r=1n−1ξjr,aμaσjr,a⟩] = _a=1^mn k_a\, _a=1^m( _i _a _i) [ _all j_r,a distinct _a=1^mq^(n-k_a)d_a+ _≥ 2\ j_r,a same _a=1^m _r=1^n-k_a _j_r,a _a _j_r,a ] _a=1^mn 1\, _a=1^m( _i _a _i) [ _all j_r,a distinct _a=1^mq^(n-1)d_a+ _≥ 2\ j_r,a same _a=1^m _r=1^n-1 _j_r,a _a _j_r,a ] (S60) To simplify notation, set R=∑a=1mn−kaR= _a=1^mn-k_a. We use Lemma 7 now in two ways. First, we may remove the second term from the denominator, as it is positive, to obtain a valid upper bound. Second, we use Lemma 7 to upper bound the second term in the numerator. We additionally cancel the ⟨∏a=1m(ξiμaσi)⟩ _a=1^m( _i _a _i) from the numerator and denominator in the prefactor, and use the binomial bound, to obtain ε _k ≤(en)∑akanm∏amkaka⋅∑all jr,a distinct∏a=1mq(n−ka)da+(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sR(q−2hN−mn⋅eRlog2)s∑all jr,a distinct∏a=1mq(n−1)da ≤ (en) _ak_an^m _a^mk_a^k_a· _all j_r,a distinct _a=1^mq^(n-k_a)d_a+ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^R ( q^-2h N-mn· eR 2 )^s _all j_r,a distinct _a=1^mq^(n-1)d_a (S61) We next insert the appropriate counting factors for the sum over distinct indices. ε _k ≤(en)∑akanm∏amkaka⋅(N−1R)R!∏a=1mq(n−ka)da+(∏a=1mq(n−ka)da)(N−1R)R!⋅∑sR(q−2hN−mn⋅eRlog2)s(N−1m(n−1))(m(n−1))!∏a=1mq(n−1)da ≤ (en) _ak_an^m _a^mk_a^k_a· N-1 RR! _a=1^mq^(n-k_a)d_a+ ( _a=1^mq^(n-k_a)d_a )N-1 RR!· _s^R ( q^-2h N-mn· eR 2 )^sN-1 m(n-1)(m(n-1))! _a=1^mq^(n-1)d_a (S62) =(en)∑akanm∏amkaka⋅((N−1R)R!(N−1m(n−1))(m(n−1))!∏a=1mq(1−ka)da)⋅(1+∑sR(q−2hN−mn⋅eRlog2)s) = (en) _ak_an^m _a^mk_a^k_a· ( N-1 RR!N-1 m(n-1)(m(n-1))! _a=1^mq^(1-k_a)d_a )· (1+ _s^R ( q^-2h N-mn· eR 2 )^s ) (S63) ≤(en)∑akanm∏amkaka⋅((1/q)2h(N−mn))∑aka−m⋅(1+∑sR(q−2hN−mn⋅eRlog2)s) ≤ (en) _ak_an^m _a^mk_a^k_a· ( (1/q)^2h(N-mn) ) _ak_a-m· (1+ _s^R ( q^-2h N-mn· eR 2 )^s ) (S64) In the third line, we used the fact that q−da≤q−2hq^-d_a≤ q^-2h and the following: (N−1R)R!(N−1m(n−1))(m(n−1))! N-1 RR!N-1 m(n-1)(m(n-1))! =(N−1−(m(n−1))!CLOSE(N−1−R)! = (N-1-(m(n-1))!(N-1-R)! (S65) =1(N−m(n−1))(N−m(n−1)+1)…(N−1−R) = 1(N-m(n-1))(N-m(n-1)+1)…(N-1-R) (S66) ≤1(N−m(n−1))∑aka−m ≤ 1(N-m(n-1)) _ak_a-m (S67) ≤1(N−mn)∑aka−m. ≤ 1(N-mn) _ak_a-m. (S68) The prefactor in Eq. S64 vanishes under the assertion that q2h=ω(n/N)q^2h=ω(n/N). The geometric series in the brackets of Eq. S64 converges so long as the common ratio is less than 1/21/2. This will be true under the stronger assertion that q2h=ω(n/N)q^2h=ω(n/ N). Combined with the prefactor, the entire ratio becomes o(1)o(1). Let us define δ=n(1/q)2h(N−mn)δ= n(1/q)^2h(N-mn). Then, the overall ratio ε _k is now upper bounded by ε≤Cδ−m∏am(ekaδ)ka _k≤ Cδ^-m _a^m ( ek_aδ )^k_a (S69) where here, C is some constant which depends on the sum in the brackets of Eq. S64. We use this and return to the moment: ⟨(ΔE)m⟩ ( E)^m =2m∑μ1,…,μm=1MT(1,1,…1)(1+∑k1,…,kmallodd≠(1,1…,1)nε) =2^m _ _1,…, _m=1^M\;T_(1,1,… 1) (1+ _ subarrayck_1,…,k_m\\ all\ odd\\ ≠(1,1…,1) subarray^n _k ) (S70) ≤2m∑μ1,…,μm=1MT(1,1,…1)(1+Cδ−m∑k1,…,kmallodd≠(1,1…,1)n∏am(ekaδ)ka) ≤ 2^m _ _1,…, _m=1^M\;T_(1,1,… 1) (1+Cδ^-m _ subarrayck_1,…,k_m\\ all\ odd\\ ≠(1,1…,1) subarray^n _a^m ( ek_aδ )^k_a ) (S71) ≤2m∑μ1,…,μm=1MT(1,1,…1)(1+O(δ2)) ≤ 2^m _ _1,…, _m=1^M\;T_(1,1,… 1) (1+O(δ^2) ) (S72) As long as q2h=ω(mn/N)q^2h=ω(mn/ N), the O(δ2)O(δ^2) term becomes o(1)o(1) and so ⟨(ΔE)m⟩ ( E)^m =2m∑μ1,…,μm=1MT(1,1,…1)(1+o(1)), =2^m _ _1,…, _m=1^M\;T_(1,1,… 1) (1+o(1) ), (S73) i.e., the mmth moment is well approximated by the all ka=1k_a=1 term. ∎ S3.2 Proof of Lemma 4 Proof. Let us first expand the term in the expectation. ⟨∏am(∑j≠iξjμaσj)n−1⟩ _a^m ( _j≠ i _j _a _j )^n-1 =⟨∑j1,1,…,j1,n−1jm,1,…,j1,n−1N∏a=1m∏r=1n−1ξjr,aμaσjr,a⟩ = _ subarraycj_1,1,…,j_1,n-1\\ \\ j_m,1,…,j_1,n-1 subarray^N _a=1^m _r=1^n-1 _j_r,a _a _j_r,a (S74) Following Eq. S45, we split the sum on the RHS into distinct and collision terms. Counting the total number of distinct jr,aj_r,a and using Lemma 7, we obtain ≤(N−1m(n−1))(m(n−1))!∏a=1mq(n−1)da+(∏a=1mq(n−1)da)(N−1m(n−1))(m(n−1))!⋅∑sm(n−1)(q−2hN−mn⋅em(n−1)logs)s ≤N-1 m(n-1)(m(n-1))! _a=1^mq^(n-1)d_a+ ( _a=1^mq^(n-1)d_a )N-1 m(n-1)(m(n-1))!· _s^m(n-1) ( q^-2h N-mn· em(n-1) s )^s (S75) ≤(N−1)!(N−1−m(n−1))!∏a=1mq(n−1)da(1+∑sm(n−1)(q−2hN−mn⋅emnlogs)s). ≤ (N-1)!(N-1-m(n-1))! _a=1^mq^(n-1)d_a (1+ _s^m(n-1) ( q^-2h N-mn· emn s )^s ). (S76) We obtain that the relative error ϵ¯ ε is upper bounded by ϵ¯≤∑snm(q−2hemnlog2N−mn)s. ε≤ _s^nm ( q^-2hemn 2 N-mn )^s. (S77) This sum and therefore the relative error is o(1)o(1) under the constraint q2h=ω(mn/N)q^2h=ω(mn/ N). Subsequently, the entire propagator is ≤(N−1)!(N−1−m(n−1))!∏a=1mq(n−1)da(1+o(1)) ≤ (N-1)!(N-1-m(n-1))! _a=1^mq^(n-1)d_a (1+o(1) ) (S78) ≤[∏a=1mNn−1q(n−1)da](1+o(1)). ≤ [ _a=1^mN^n-1q^(n-1)d_a ] (1+o(1) ). (S79) In the final line, we have just upper bounded (N−1)!(N−1−m(n−1))! (N-1)!(N-1-m(n-1))! with Nm(n−1)N^m(n-1). As the relative error term is positive, the lower bound is obtained by lower bounding the prefactor. This yields ⟨∏am(∑j≠iξjμaσj)n−1⟩ _a^m ( _j≠ i _j _a _j )^n-1 ≥(N−1)!(N−1−m(n−1))!∏a=1mq(n−1)d(μa,). ≥ (N-1)!(N-1-m(n-1))! _a=1^mq^(n-1)d( ξ _a, σ). (S80) ∎ S3.3 Proof of Lemma 5 Proof. The overall variance may be expressed as Var(ΔE) ( E) =4∑μ1μ2M∑k1,k2,oddn(nk1)(nk2)[⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−k1(∑j≠iξjμ2σj)n−k2⟩ =4 _ _1 _2^M _k_1,k_2, \ odd^nn k_1n k_2 [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-k_1 ( _j≠ iξ _2_j _j )^n-k_2 . −⟨(ξiμ1σi⟩⟨(∑j≠iξjμ1σj)n−k1⟩⟨(ξiμ2σi⟩⟨(∑j≠iξjμ2σj)n−k2⟩]. .- ( _i _1 _i ( _j≠ i _j _1 _j )^n-k_1 ( _i _2 _i ( _j≠ i _j _2 _j )^n-k_2 ]. (S81) As before, we will attempt to approximate this expression with k1=k2=1k_1=k_2=1. The relative error there is ε~ _k =(nk1)(nk2)n2[⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−k1(∑j≠iξjμ2σj)n−k2⟩−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−k1⟩⟨(∑j≠iξjμ2σj)n−k2⟩][⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩] = n k_1n k_2n^2 [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-k_1 ( _j≠ iξ _2_j _j )^n-k_2 - ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-k_1 ( _j≠ i _j _2 _j )^n-k_2 ] [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 - ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ] (S82) ≤(nk1)(nk2)n2⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−k1(∑j≠iξjμ2σj)n−k2⟩[⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩] ≤ n k_1n k_2n^2 ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-k_1 ( _j≠ iξ _2_j _j )^n-k_2 [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 - ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ] (S83) =(nk1)(nk2)n2⟨(∑j≠iξjμ1σj)n−k1(∑j≠iξjμ2σj)n−k2⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩11−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ = n k_1n k_2n^2 ( _j≠ iξ _1_j _j )^n-k_1 ( _j≠ iξ _2_j _j )^n-k_2 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 11- ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 (S84) ≤ε(1−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩)−1. ≤ _k (1- ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 )^-1. (S85) In the second line, we used the fact that the correlations are positive to remove the subtraction from the numerator. Then, by factoring the expression in line 3, we obtain a prefactor which is identical in form to that in Lemma 3. We upper bound the prefactor using Lemma 3 to get to line 4. Continuing onwards, we now focus on the term in the parenthesis. To upper bound the variance, we must lower bound the term in the parenthesis. 1−⟨(ξiμ1σi)⟩⟨(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ 1- ( _i _1 _i) ( _i _2 _i) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 =1−qd(μ1,)+d(μ2,)⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩qd(μ1,μ2)⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ =1- q^d( ξ _1, σ)+d( ξ _2, σ) ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 q^d( ξ _1, ξ _2) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 (S86) =1−q2s⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ =1-q^2s ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 (S87) ≥1−q2s. ≥ 1-q^2s. (S88) In the first line we used the dashed line rule and in the second line, we used the tripod relation, where s is the distance from the probe σ to the median point. The third line is obtained by realizing that as all correlations in the problem are positive. Therefore, for even n and due to the independence of the sites, OPEN(∑j≠iξjμ1σj)n−1)( _j≠ i _j _1 _j)^n-1) and OPEN(∑j≠iξjμ2σj)n−1)( _j≠ i _j _2 _j)^n-1) are positively associated and the denominator must be at least as large as the numerator (see FKG inequality). Therefore, the fraction is upper bounded by 1. Thus the full term has the upper bound: ε~ _k ≤ε1−q2s. ≤ _k1-q^2s. (S89) Here, s will depend on μ1 _1 and μ2 _2. It is possible for s=0s=0. But as we will see, the contribution to the variance from these terms is 0. We return now to the full variance formula: Var(ΔE) ( E) ≤4n2∑μ1μ2,s≠0M[⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ ≤ 4n^2 _ _1 _2,s≠ 0^M [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 . −⟨(ξiμ1σi⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(ξiμ2σi⟩⟨(∑j≠iξjμ2σj)n−1⟩]×(1+11−q2s(μ1,μ2)∑k1,k2≠(1,1),oddnε) .- ( _i _1 _i ( _j≠ i _j _1 _j )^n-1 ( _i _2 _i ( _j≠ i _j _2 _j )^n-1 ]× (1+ 11-q^2s( _1, _2) _k_1,k_2≠(1,1), odd^n _k ) (S90) ≤4n2∑μ1μ2,s≠0M[⟨(ξiμ1σi)(ξiμ2σi)⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩ ≤ 4n^2 _ _1 _2,s≠ 0^M [\; ( _i _1 _i)( _i _2 _i) ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 . −⟨(ξiμ1σi⟩⟨(∑j≠iξjμ1σj)n−1⟩⟨(ξiμ2σi⟩⟨(∑j≠iξjμ2σj)n−1⟩]×(1+11−q2s(μ1,μ2)O(δ2)) .- ( _i _1 _i ( _j≠ i _j _1 _j )^n-1 ( _i _2 _i ( _j≠ i _j _2 _j )^n-1 ]× (1+ 11-q^2s( _1, _2)O(δ^2) ) (S91) =4n2∑μ1μ2,s≠0M[qt1+t2⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩−q2s+t1+t2⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩] =4n^2 _ _1 _2,s≠ 0^M [\;q^t_1+t_2 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 -q^2s+t_1+t_2 ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ] ×(1+11−q2s(μ1,μ2)O(δ2)) × (1+ 11-q^2s( _1, _2)O(δ^2) ) (S92) We must now control the term in the bracket. We will show that this term, divided by using the solid line approximation is 1 plus a small error term. qt1+t2⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩−q2s+t1+t2⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩qt1+t2⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1−q2s+t1+t2⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1 q^t_1+t_2 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 -q^2s+t_1+t_2 ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 q^t_1+t_2 ( _j≠ iξ _1_j _j ) ^n-1 ( _j≠ iξ _2_j _j ) ^n-1-q^2s+t_1+t_2 ( _j≠ iξ _1_j _j ) ^n-1 ( _j≠ iξ _2_j _j ) ^n-1 (S93) =⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩−q2s⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1(1−q2s) = ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 -q^2s ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _j≠ iξ _1_j _j ) ^n-1 ( _j≠ iξ _2_j _j ) ^n-1 (1-q^2s ) (S94) =⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−11−q2s⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩1−q2s = ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 ( _j≠ iξ _1_j _j ) ^n-1 ( _j≠ iξ _2_j _j ) ^n-1 1-q^2s ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 1-q^2s (S95) ≤(1+ϵ¯)1−q2s⟨(∑j≠iξjμ1σj)n−1⟩⟨(∑j≠iξjμ2σj)n−1⟩⟨(∑j≠iξjμ1σj)n−1(∑j≠iξjμ2σj)n−1⟩1−q2s ≤(1+ ε) 1-q^2s ( _j≠ i _j _1 _j )^n-1 ( _j≠ i _j _2 _j )^n-1 ( _j≠ iξ _1_j _j )^n-1 ( _j≠ iξ _2_j _j )^n-1 1-q^2s (S96) ≤(1+ϵ¯)1−q2s⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1(1+ϵ¯)1−q2s ≤(1+ ε) 1-q^2s ( _j≠ i _j _1 _j ) ^n-1 ( _j≠ i _j _2 _j ) ^n-1 ( _j≠ iξ _1_j _j ) ^n-1 ( _j≠ iξ _2_j _j ) ^n-1(1+ ε)1-q^2s (S97) =(1+ϵ¯)1−q2s(1+ϵ¯)−11−q2s =(1+ ε) 1-q^2s(1+ ε)^-11-q^2s (S98) =1+ϵ¯1−q2s =1+ ε1-q^2s (S99) In the second line, we canceled out qt1+t2q^t_1+t_2, in the third line we factorized the expression, and in the fourth and fifth lines, we applied Lemma 4. Finally, in the last line we omitted the 1−11+ϵ¯1- 11+ ε term. Once again returning to the full variance, we obtain Var(ΔE) ( E) ≤4n2∑μ1μ2,s≠0Mqt1+t2⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1(1−q2s(μ1,μ2))(1+ϵ¯1−q2s(μ1,μ2))×(1+11−q2s(μ1,μ2)O(δ2)). ≤ 4n^2 _ _1 _2,s≠ 0^M\;q^t_1+t_2 ( _j≠ i _j _1 _j ) ^n-1 ( _j≠ i _j _2 _j ) ^n-1(1-q^2s( _1, _2)) (1+ ε1-q^2s( _1, _2) )× (1+ 11-q^2s( _1, _2)O(δ^2) ). (S100) The factor (1−q2s)(1-q^2s) is 0 for s=0s=0, so even though the error terms are uncontrolled there, the overall expression is 0. So we are free to include those terms back into the summation. Since those terms are 0, we can upper bound the error terms involving s with s=1s=1. This ultimately yields Var(ΔE) ( E) ≤4n2[(1+ϵ¯1−q2)×(1+11−q2O(δ2))]∑μ1μ2Mqt1+t2⟨(∑j≠iξjμ1σj)⟩n−1⟨(∑j≠iξjμ2σj)⟩n−1(1−q2s) ≤ 4n^2 [ (1+ ε1-q^2 )× (1+ 11-q^2O(δ^2) ) ] _ _1 _2^M\;q^t_1+t_2 ( _j≠ i _j _1 _j ) ^n-1 ( _j≠ i _j _2 _j ) ^n-1(1-q^2s) (S101) ≤4n2N2(n−1)(1+o(1))∑μ1μ2Mqn(2s+t1+t2)−2s(1−q2s) ≤ 4n^2N^2(n-1)(1+o(1)) _ _1 _2^M\;q^n(2s+t_1+t_2)-2s(1-q^2s) (S102) ∎ S3.4 Proof of Theorem 6 Proof. The expression in the numerator of ℓvC_ ^v is the variance, which in simplified form is Var~[Δ(E)] Var[ (E)] =4n2N2n−2∑(s,t1,t2)C~ℓ(2)((,,,,,))qn(2s+t1+t2)−2s(1−q2s). =4n^2N^2n-2 _(s,t_1,t_2) C^(2)_ ((s,t_1,t_2) )q^n(2s+t_1+t_2)-2s(1-q^2s). (S103) We use Var~ Var to denote the variance approximated using Lemma 5 and the simplified combinatorics. We assume that q−2≪K≪q−2n+1q^-2 K q^-2n+1. Upon some algebraic manipulations, we expand Eq. S103 as Var~[Δ(E)] Var[ (E)] =4n2N2n−2(Kqn)2(h−ℓ)[∑s=1ℓ(Kq2n−1)2s(1−q2s)+∑s=0h−ℓ(Kq2)−s(1−q2s)+∑a=1ℓ(Kq2n−1)2a∑u=1h−ℓ+a−1(Kq2)−u(1−q2(a+u)) =4n^2N^2n-2(Kq^n)^2(h- ) [ _s=1 (Kq^2n-1)^2s(1-q^2s)+ _s=0^h- (Kq^2)^-s(1-q^2s)+ _a=1 (Kq^2n-1)^2a _u=1^h- +a-1(Kq^2)^-u (1-q^2(a+u) ) . +2∑s=1ℓ−1(Kq2n−1)2s(1−q2s)∑y=1ℓ−s(Kq2n)y+(Kq2)−(h−ℓ)∑x=1ℓ(Kq4n−4)x(1−q2(h−ℓ+2x))] .+2 _s=1 -1(Kq^2n-1)^2s(1-q^2s) _y=1 -s(Kq^2n)^y+(Kq^2)^-(h- ) _x=1 (Kq^4n-4)^x (1-q^2(h- +2x) ) ] (S104) Each of the sums in the bracket here are geometric series. Furthermore, by the assumption on K, the common ratio of each of the sums is less than one. Thus we may upper bound the entire expression by setting the top of each sum to be ∞. Our goal is to obtain an expression multiplied by a term of the form 1+o(1)1+o(1). To that end, we will factor out the first term from each sum. Note that each of the sums carries a term of the form 1−q2s1-q^2s so this may be factored out of the entire expression. This results in the following: Var~[Δ(E)] Var[ (E)] ≤4n2N2n−2(Kqn)2(h−ℓ)(1−q2)[(K2q4n−2)(1+λ1)+(Kq2)−1(1+λ2)+(Kq4n−4)(1+q2)(1+λ3) ≤ 4n^2N^2n-2(Kq^n)^2(h- )(1-q^2) [(K^2q^4n-2)(1+ _1)+(Kq^2)^-1(1+ _2)+(Kq^4n-4)(1+q^2)(1+ _3) +2(K3q6n−2)(1+λ4)+(Kq2)−(h−ℓ)(Kq4n−4)1−q2(h−ℓ+2)1−q2(1+λ5)], +2(K^3q^6n-2)(1+ _4)+(Kq^2)^-(h- )(Kq^4n-4) 1-q^2(h- +2)1-q^2(1+ _5) ], (S105) where in the above expression, each of the λi _i terms are λ1 _1 :=∑s=2∞(Kq2n−1)2s(1−q2s)(Kq2n−1)2(1−q2)≤∑s=2∞s(K2q4n−2)s−1=O(K2q4n−2) := _s=2^∞ (Kq^2n-1 )^2s (1-q^2s ) (Kq^2n-1 )^2 (1-q^2 )\;≤\; _s=2^∞s\, (K^2q^4n-2 )^s-1\;=\;O\! (K^2q^4n-2 ) (S106) λ2 _2 :=∑s=2∞(Kq2)−s(1−q2s)(Kq2)−1(1−q2)≤∑s=2∞s(Kq2)−(s−1)=O((Kq2)−1) := _s=2^∞ (Kq^2 )^-s (1-q^2s ) (Kq^2 )^-1 (1-q^2 )\;≤\; _s=2^∞s\, (Kq^2 )^-(s-1)\;=\;O\! ((Kq^2)^-1 ) (S107) λ3 _3 :=∑a,u≥ 1(a,u)≠(1,1)(Kq2n−1)2a(Kq2)−u(1−q2(a+u))(Kq2n−1)2(Kq2)−1(1−q4) := _ subarrayca,u\,≥\,1\\ (a,u)\,≠\,(1,1) subarray (Kq^2n-1 )^2a (Kq^2 )^-u (1-q^2(a+u) ) (Kq^2n-1 )^2 (Kq^2 )^-1 (1-q^4 ) ≤∑a,u≥ 1(a,u)≠(1,1)(a+u)(K2q4n−2)a−1(Kq2)−(u−1)=O(K2q4n−2+(Kq2)−1) \;≤\; _ subarrayca,u\,≥\,1\\ (a,u)\,≠\,(1,1) subarray(a+u)\, (K^2q^4n-2 )^a-1 (Kq^2 )^-(u-1)\;=\;O\! (K^2q^4n-2+(Kq^2)^-1 ) (S108) λ4 _4 :=∑s,y≥ 1(s,y)≠(1,1)(Kq2n−1)2s(1−q2s)(Kq2n)y(Kq2n−1)2(1−q2)(Kq2n) := _ subarraycs,y\,≥\,1\\ (s,y)\,≠\,(1,1) subarray (Kq^2n-1 )^2s (1-q^2s ) (Kq^2n )^y (Kq^2n-1 )^2 (1-q^2 ) (Kq^2n ) ≤∑s,y≥ 1(s,y)≠(1,1)s(K2q4n−2)s−1(Kq2n)y−1=O(K2q4n−2+Kq2n) \;≤\; _ subarraycs,y\,≥\,1\\ (s,y)\,≠\,(1,1) subarrays\, (K^2q^4n-2 )^s-1 (Kq^2n )^y-1\;=\;O\! (K^2q^4n-2+Kq^2n ) (S109) λ5 _5 :=∑x=2∞(Kq4n−4)x−11−q2(h−ℓ+2x)1−q2(h−ℓ+2)≤11−q6∑x=2∞(Kq4n−4)x−1=O(Kq4n−4) := _x=2^∞ (Kq^4n-4 )^x-1\, 1-q^2(h- +2x)1-q^2(h- +2)\;≤\; 11-q^6 _x=2^∞ (Kq^4n-4 )^x-1\;=\;O\! (Kq^4n-4 ) (S110) Here, for λ1 _1 to λ4 _4, we have used the inequality 1−q2s1−q2= 1+q2+⋯+q2s−2≤s 1-q^2s1-q^2\;=\;1+q^2+·s+q^2s-2\;≤\;s. We next notice that in the expression in the brackets of S105, each term involves a factor of K2q4n−2K^2q^4n-2 and/or Kq2Kq^2. We subsequently factor out [K2q4n−2+(Kq2)−1][K^2q^4n-2+(Kq^2)^-1]. Upon doing so, and also upper bounding 1−q2(h−ℓ+2)1−q2 1-q^2(h- +2)1-q^2 using the earlier inequality, we obtain Var~[Δ(E)] Var[ (E)] ≤4n2N2n−2(Kqn)2(h−ℓ)(1−q2)[K2q4n−2+(Kq2)−1]⋅[1+K2q4n−2λ1+(Kq2)−1λ2K2q4n−2+(Kq2)−1+(1+q2)(K2q4n−2)(Kq2)−1(1+λ3)K2q4n−2+(Kq2)−1 ≤ 4n^2N^2n-2(Kq^n)^2(h- )(1-q^2) [K^2q^4n-2+(Kq^2)^-1 ]· [1+ K^2q^4n-2 _1+(Kq^2)^-1 _2K^2q^4n-2+(Kq^2)^-1+ (1+q^2)(K^2q^4n-2)(Kq^2)^-1(1+ _3)K^2q^4n-2+(Kq^2)^-1 +2(K2q4n−2)(Kq2n)(1+λ4)K2q4n−2+(Kq2)−1+(Kq2)−(h−ℓ)(K2q4n−2)(Kq2)−1(h−ℓ+2)(1+λ5)K2q4n−2+(Kq2)−1] + 2(K^2q^4n-2)(Kq^2n)(1+ _4)K^2q^4n-2+(Kq^2)^-1+ (Kq^2)^-(h- )(K^2q^4n-2)(Kq^2)^-1(h- +2)(1+ _5)K^2q^4n-2+(Kq^2)^-1 ] (S111) ≤4n2N2n−2(Kqn)2(h−ℓ)(1−q2)[K2q4n−2+(Kq2)−1]⋅[1+λ1+λ2+(1+q2)(K2q4n−2)(1+λ3)+2(Kq2n)(1+λ4) ≤ 4n^2N^2n-2(Kq^n)^2(h- )(1-q^2) [K^2q^4n-2+(Kq^2)^-1 ]· [1+ _1+ _2+(1+q^2)(K^2q^4n-2)(1+ _3)+2(Kq^2n)(1+ _4) +(h−ℓ+2)(Kq2)−(h−ℓ)(K2q4n−2)(1+λ5)] +(h- +2)(Kq^2)^-(h- )(K^2q^4n-2)(1+ _5) ] (S112) ≤4n2N2n−2(Kqn)2(h−ℓ)(1−q2)[K2q4n−2+(Kq2)−1](1+ν). ≤ 4n^2N^2n-2(Kq^n)^2(h- )(1-q^2) [K^2q^4n-2+(Kq^2)^-1 ](1+ν). (S113) Here, we set ν to be ν ν :=λ1+λ2+(1+q2)(K2q4n−2)(1+λ3)+2(Kq2n)(1+λ4)+(h−ℓ+2)(Kq2)−(h−ℓ)(K2q4n−2)(1+λ5) := _1+ _2+(1+q^2)(K^2q^4n-2)(1+ _3)+2(Kq^2n)(1+ _4)+(h- +2)(Kq^2)^-(h- )(K^2q^4n-2)(1+ _5) (S114) =O(K2q4n−2+Kq2n+(Kq2)−1) =O(K^2q^4n-2+Kq^2n+(Kq^2)^-1) (S115) With this result in hand, we now determine the denominator to the squared coefficeint of variation, which is simply the mean squared. This quantity is lower bounded by a modification to the approximate formula (Theorem 2) using the lower bound in Lemma 4. ⟨ΔE⟩ℓ2 E _ ^2 ≥4n2((N−1)!(N−n)!)2((K−1)qn)2(h−ℓ)×[1+K−2K−1∑x=1ℓ−1((K−1)q2n)x+((K−1)q2n)ℓ]2 ≥ 4n^2 ( (N-1)!(N-n)! )^2 ((K-1)q^n )^2(h- )× [1+ K-2K-1 _x=1 -1 ((K-1)q^2n )^x+ ((K-1)q^2n ) ]^2 (S116) ≥4n2((N−1)!(N−n)!)2((K−1)qn)2(h−ℓ) ≥ 4n^2 ( (N-1)!(N-n)! )^2 ((K-1)q^n )^2(h- ) (S117) ≥4n2N2n−2(1−n(n−1)N)((K−1)qn)2(h−ℓ). ≥ 4n^2N^2n-2 (1- n(n-1)N ) ((K-1)q^n )^2(h- ). (S118) When we invert this in the squared coefficient of variation, the term in the parenthesis becomes a 1+o(1)1+o(1) factor, which combines with the relative error on the variance to remain 1+o(1)1+o(1). With this, we obtain the upper bound on squared coefficient of variation. (0<ℓ<hv)2 (C_0< <h^v)^2 ≤(K−1)2(h−ℓ)(1−q2)⋅[K2q4n−2+(Kq2)−1]⋅(1+ν)⋅(1+o(1)) ≤ ( KK-1 )^2(h- )(1-q^2)· [K^2q^4n-2+(Kq^2)^-1 ]·(1+ν)·(1+o(1)) (S119) ≤exp(2(h−ℓ)K−1)⋅[K2q4n−2+(Kq2)−1]⋅(1+ν)⋅(1+o(1)) ≤ ( 2(h- )K-1 )· [K^2q^4n-2+(Kq^2)^-1 ]·(1+ν)·(1+o(1)) (S120) Thus we obtain the inequality of the theorem. Now, we set K=cK⋅N⋅κ(N),n=cnlogNK=c_K· N·κ(N),\ n=c_n N and h=chlogNh=c_h N. Here, κ(N)κ(N) is an arbitrarily slowly growing function of N. For this choice of parameter scaling ν becomes o(1)o(1). First, our formulas are only valid for q2h=ω(n/N)q^2h=ω(n/ N). For this to be satisfied, we require limN→∞cnlogNq2chlogN→0 _N→∞ c_n N Nq^2c_h N→ 0, which subsequently requires chlog(1/q)<1/4. c_h (1/q)<1/4. (S121) Return now to Eq. S120, we focus on the term in the brackets, as this is what will be vanishing. K2q4n−2+(Kq2)−1 K^2q^4n-2+(Kq^2)^-1 =cK2N2q4cnlogN−2(κ(N))2+1cKN1κ(N)q2 =c_K^2N^2q^4c_n N-2(κ(N))^2+ 1c_KN^1κ(N)q^2 (S122) From Eq. S122, we may now use the union bound over all N spins. Note that the exponential prefactor also becomes 1+o(1)1+o(1) under this parameter scaling. ℙ(⋃i=1NΔEi<0) ( _i=1^N E_i<0 ) ≤N⋅(cK2N2q4cnlogN−2(κ(N))2+1cKN1κ(N)q2)(1+o(1)) ≤ N· (c_K^2N^2q^4c_n N-2(κ(N))^2+ 1c_KN^1κ(N)q^2 )(1+o(1)) (S123) When N multiples the second term in the parenthesis, that term decays with N due to the κ(N)κ(N) term in the denominator. As for the first term in the parenthesis, it too decays so long as cnlog(1/q)>34 c_n (1/q)> 34 (S124) This relation and Eq. S121 imply also that cn>3chc_n>3c_h. This proves the second statement in the theorem. ∎