Paper deep dive
A Compositional Theory of Curvature in Probabilistic Circuits
Hrithik Suresh, Sahil Sidheekh, Shelar Parth Vijay, Yasir Z, Sriraam Natarajan, Narayanan Chatapuram Krishnan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/14/2026, 5:24:40 AM
Summary
This paper introduces a compositional theory of curvature in Probabilistic Circuits (PCs), demonstrating that the global Hessian trace (sharpness) factorizes into a node's contextual usage (circuit flow) and its intrinsic local curvature. The authors argue that global sharpness regularization is often misspecified because it conflates these two factors, leading to depth bias and underfitting. They propose an adaptive sharpness-aware regularizer that targets nodes based on local curvature rather than global contribution, improving generalization while preserving closed-form EM updates.
Entities (7)
Relation Signals (8)
Probabilistic Circuits → supports → Exact Inference
confidence 95% · Probabilistic Circuits (PCs) are generative models that support exact inference
Hessian Trace → measures → Loss-Surface Curvature
confidence 92% · admit an exact and tractable measure of loss-surface curvature: the trace of the Hessian of the log-likelihood
Sum Node → contributesto → Hessian Trace
confidence 90% · each sum node’s contribution to the Hessian trace factorizes exactly
Hessian Trace → factorizesinto → Local Sharpness
confidence 90% · each sum node’s contribution to the Hessian trace factorizes exactly into its circuit flow... and a local sharpness term
Hessian Trace → factorizesinto → Circuit Flow
confidence 90% · each sum node’s contribution to the Hessian trace factorizes exactly into its circuit flow... and a local sharpness term
Adaptive Sharpness Aware Regularizer → improves → Generalization
confidence 90% · this targeted regularization recovers the generalization that global regularization sacrifices
Adaptive Sharpness Aware Regularizer → uses → local curvature
confidence 90% · penalizes nodes based on intrinsic local curvature
Global Sharpness Regularization → causes → Underfitting
confidence 88% · global sharpness regularization is depth biased and can lead to underfitting
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Probabilistic Circuits (PCs) are generative models that support exact inference and, unlike deep neural networks, admit an exact and tractable measure of loss-surface curvature: the trace of the Hessian of the log-likelihood. Recent work regularizes this trace globally to bias learning toward flatter, better generalizing optima. We show that treating sharpness as a global regularizer can be misspecified for PCs, whose curvature is inherently compositional. We prove that each sum node's contribution to the Hessian trace factorizes exactly into its circuit flow, which measures how heavily the node is used, and a local sharpness term determined by its output distribution. This decomposition provides insights into why global sharpness regularization is depth biased and can lead to underfitting. Building on it, we introduce an adaptive sharpness aware regularizer that penalizes nodes based on intrinsic local curvature and preserves closed form EM updates. We also show that empirically, this targeted regularization recovers the generalization that global regularization sacrifices while retaining the robustness and benefits of sharpness aware learning.
Tags
Links
- Source: https://arxiv.org/abs/2608.12869v1
- Canonical: https://arxiv.org/abs/2608.12869v1
Trouble viewing inline? Open PDF directly →
Full Text
83,333 characters extracted from source content.
Expand or collapse full text
A Compositional Theory of Curvature in Probabilistic Circuits Hrithik Suresh Sahil Sidheekh Thanks: Work done in part while Sahil was an intern at AT&T. Shelar Parth Vijay Yasir Z Sriraam Natarajan Narayanan Chatapuram Krishnan Abstract Probabilistic Circuits (PCs) are generative models that support exact inference and, unlike deep neural networks, admit an exact and tractable measure of loss-surface curvature: the trace of the Hessian of the log-likelihood. Recent work regularizes this trace globally to bias learning toward flatter, better generalizing optima. We show that treating sharpness as a global regularizer can be misspecified for PCs, whose curvature is inherently compositional. We prove that each sum node’s contribution to the Hessian trace factorizes exactly into its circuit flow, which measures how heavily the node is used, and a local sharpness term determined by its output distribution. This decomposition provides insights into why global sharpness regularization is depth biased and can lead to underfitting. Building on it, we introduce an adaptive sharpness aware regularizer that penalizes nodes based on intrinsic local curvature and preserves closed form EM updates. We also show that empirically, this targeted regularization recovers the generalization that global regularization sacrifices while retaining the robustness and benefits of sharpness aware learning. Introduction Probabilistic circuits (PCs) are generative models whose structural constraints make a broad class of probabilistic queries exactly tractable and computable in time linear in the circuit size (5). This distinguishes PCs from mainstream deep generative models such as GANs (9), VAEs (14), and normalizing flows (21), and makes them particularly useful in settings where exact and reliable inference is required, including constrained generation (35), image inpainting (19), robust representation learning (4), multimodal fusion (31), neuro-symbolic AI (1; 11) and algorithmic recourse (20; 26), among others. As PCs become deeper and more expressive, however, they are increasingly prone to overfitting in low-data regimes and may converge to sharp optima that generalize poorly (32). Sharpness-aware learning addresses this problem by biasing optimization toward flatter regions of the loss landscape (8). In deep neural networks, the relevant Hessian-based curvature is generally intractable and must be estimated through stochastic approximations (12; 27). Recent work (32) has shown that PCs are a rare exception in this context: their structure makes the Hessian trace of the negative log-likelihood exactly computable in a single forward–backward pass, enabling a tractable global trace regularizer that improves generalization when data are scarce. While this result established that sharpness can be computed and penalized efficiently in PCs, it treats the trace as a single global measure of flatness. In this work, we ask what this global curvature actually represents inside a PC, and whether it is always an appropriate learning signal. Our motivation comes from three empirical observations. First, in the high-data regime, global trace regularization reaches flatter optima but lowers both training and test log-likelihood (as illustrated in Figure 1), indicating under-fitting rather than improved generalization. Second, the node-wise contributions to the trace are highly concentrated, with a small fraction of sum nodes accounting for most of the total curvature. Third, restricting regularization to the largest-contributing nodes degrades performance further. These observations suggest that the issue is not only the overall strength of regularization, but how it is allocated across the circuit. Our central thesis is that these quantities separate exactly, and that separating them helps resolve the puzzle. Specifically, we show that a sum node’s contribution to the Hessian trace factorizes exactly into two semantically distinct terms: Tn()=Fn()2⏟contextual usage⋅tn()⏟local curvature,T_n(x)= F_n(x)^2_contextual usage· t_n(x)_local curvature, where FnF_n is the circuit flow through node n, measuring how strongly the node is used in explaining the input, while tn()t_n(x) is a local curvature measure that depends only on the node n. Thus, sharpness in a PC is local in origin and contextual in its global effect. A node may contribute strongly to the Hessian trace because it is locally sharp, because it lies on a high-flow path, or because both effects coincide. Consequently, ranking nodes by their global contribution need not identify the nodes with the largest intrinsic local curvature. Figure 1: Global Flatness Can Underfit: Local Curvature Determines Where to Regularize. Training trajectories of a PC on a 2D data distribution, from a shared initialization (star), shown over the (a) train NLL, (b) test NLL, and (c) sharpness surface. The unregularized model reaches a sharp optimum that generalizes poorly. The global trace regularizer’s uniform penalty helps achieve flattest region, but at the expense of under-fitting (poor train and test NLL), whereas our adaptive penalty targets nodes of high local curvature tnt_n and generalizes best, while also reducing sharpness. Both approaches use the same regularization strength μ. We first characterize and study both factors theoretically. For the local term, we show that the Hessian of a sum node is rank one and that its unique nonzero eigenvalue is exactly tnt_n. Consequently, tnt_n fully captures the node’s ambient second-order geometry. For the contextual term, we build on the standard circuit-flow recursion to analyze how structural usage shapes global curvature, and show that it causes a depth bias in the global trace toward shallow, high-flow nodes. We formalize this and derive an exact condition under which rankings by global contribution TnT_n and local curvature tnt_n disagree, explaining why selecting nodes by their global trace contribution can preferentially target highly used circuit components rather than intrinsically sharp mixtures. Finally, we show how this decomposition can be algorithmically useful. Since TnT_n conflates local curvature with structural usage, it is not a reliable quantity for deciding where regularization should act. We propose an adaptive sharpness regularizer use the empirical local curvature t^n t_n to gate the existing trace penalty, assigning stronger regularization to intrinsically sharper mixture nodes while preserving the tractable updates and linear-time complexity of the global method. The resulting learner retains the low-data gains of sharpness-aware training while reducing the under-fitting induced by uniform regularization at higher data scales. Background and Preliminaries Probabilistic Circuits A probabilistic circuit (PC) over random variables =X1,…,XdX=\X_1,…,X_d\ is a rooted directed acyclic graph in which each node n computes a distribution pnp_n, defined recursively: pn()=fn(),n an input node,∏c∈ch(n)pc(),n a product node,∑c∈ch(n)θncpc(),n a sum node,p_n(x)= casesf_n(x),&n an input node,\\ _c (n)p_c(x),&n a product node,\\ _c (n) _nc\,p_c(x),&n a sum node, cases (1) where fnf_n is a univariate input distribution, ch(n)ch(n) are the children of n, and the sum-node weights satisfy θnc≥0 _nc≥ 0 and ∑c∈ch(n)θnc=1 _c (n) _nc=1, so that the likelihood of a sample is simply the value computed at the root, p()=pr()p(x)=p_r(x). Product nodes represent factorizations over disjoint scopes (decomposability), while sum nodes represent convex mixtures over children with identical scope (smoothness). Together, these two structural conditions are what make a broad class of probabilistic queries tractable, computable exactly and in time linear in circuit size (5). This formalism is general enough to subsume arithmetic circuits (6), sum-product networks (24), PSDDs (15), and cutset networks (25) as special cases. Throughout, we write S for the set of sum nodes and collect their weights into =θnc θ=\ _nc\. Unless stated otherwise, we treat these sum-edge weights as untied free parameters, and measure curvature w.r.t this representation. Circuit Flows and EM Learning . While the upward evaluation of a PC computes the likelihood at the root, analyzing learning and curvature requires attributing this computation to individual nodes and edges. Circuit flows (18) provide this downward attribution by measuring how much probability mass the root node routes through a given node when explaining a particular sample, and it is this quantity that will allow us later to localize curvature to specific parts of the circuit. Flow propagates from the root downward, and at each node its value is determined by the type of the node’s parents. Setting Fr()=1F_r(x)=1 at the root, the flow at node n is Fn()=∑m∈pa(n)m is productFm()+∑m∈pa(n)m is sumFm()θmnpn()pm(),F_n(x)= _ subarraycm (n)\\ m is product subarrayF_m(x)\ + _ subarraycm (n)\\ m is sum subarrayF_m(x)\, _mn\, p_n(x)p_m(x), meaning a product parent passes its flow to each child undivided, whereas a sum parent attenuates the flow it passes to child n by the routing responsibility θmnpn()/pm() _mn\,p_n(x)/p_m(x). Equivalently, Fn()=∂logpr()/∂logpn()F_n(x)=∂ p_r(x)/∂ p_n(x): the flow is the sensitivity of the root log-likelihood to the node’s log-output. Flow at an edge (n,c)(n,c) leaving a sum node n is Fnc()=θncpc()pn()Fn(),F_nc(x)= _nc\, p_c(x)p_n(x)\,F_n(x), (2) and from this the gradient of the log-likelihood with respect to the corresponding sum weight follows directly, ∂logpr()/∂θnc=Fnc()/θnc∂ p_r(x)/∂ _nc=F_nc(x)/ _nc. We also write rnc()=θncpc()/pn()r_nc(x)= _nc\,p_c(x)/p_n(x) for the posterior routing responsibility of child c, which satisfies ∑c∈ch(n)rnc()=1 _c (n)r_nc(x)=1 and so behaves as a proper distribution over n’s children. Node and edge flows are obtained together from a single forward–backward pass over the circuit, and this same pass suffices to compute gradients and closed-form expectation-maximization (EM) updates in linear time (18). In the EM view, the edge flow Fnc()F_nc(x) is the expected number of times sample x traverses edge (n,c)(n,c), and the M-step simply aggregates these expectations across a dataset. Sharpness and Regularization in PCs Recent works have pushed PCs toward deeper, more expressive, and more scalable architectures while preserving tractable inference (23; 22; 28; 29; 17; 30). This increased capacity, however, also makes them more susceptible to overfitting, especially in limited-data regimes. Existing approaches have attempted to exploit the circuit’s tractable structure to address this issue via dropout (34), parameter smoothing, data softening, and entropy regularization (18). A complementary view studies overfitting through the geometry of the loss landscape. Flat minima have long been associated with better generalization (10; 13), motivating methods such as sharpness-aware minimization (8). This relationship is however not universal: sharpness can depend on the chosen parameterization, and its connection to generalization varies across models and tasks (7; 2). More recently, (32) brought this perspective to PCs, showing that overfitting can coincide with convergence to sharp optima and that the Hessian trace of the negative log-likelihood can be computed exactly for general PCs. They showed that as prp_r is multilinear in each sum-edge weight, the diagonal of the Hessian of the log-likelihood coincides with the squared first derivative along that weight, so the trace of the Hessian reduces to a sum of squared edge gradients. Writing the per-example negative log-likelihood (NLL) as ℓ(,)=−logpr() ( θ;x)=- p_r(x), the trace Tr(∇2ℓ(,))=∑n∈∑c∈ch(n)(Fnc()θnc)2≥ 0.Tr\! (∇^2_ θ\, ( θ;x) )= _n _c (n) ( F_nc(x) _nc )^2\;≥\;0. (3) quantifies the sharpness of the loss landscape at θ, i.e. curvature information, and is computable exactly and in time linear in circuit size. Overfitting in PCs is typically correlated with convergence to sharp optima, and incorporating this trace as a constraint in the EM M-step yields a closed-form, trace-regularized update for each sum weight, θnc=Nnc+Nnc2+4λμNnc2λ, _nc= N_nc+ N_nc^2+4λμ\,N_nc2λ, (4) where Nnc=∑iFnc(i)N_nc= _iF_nc(x_i) is the batch-aggregated expected count and λ,μ≥0λ,μ≥ 0 are the Lagrange multipliers enforcing, respectively, the simplex constraint and the regularization strength. This enables convergence to flatter optima that can generalize better. However, in this framework a single strength μ is applied uniformly to every sum node, we refer to this method as the global trace regularizer, and we take it as our baseline and point of departure. To reason about where in the circuit this trace originates, rather than only its aggregate value, we attribute it to individual sum nodes. The sample-wise global trace contribution of a sum node n collects the terms of Eq. (3) belonging to its outgoing edges, Tn()=∑c∈ch(n)(Fnc()θnc)2,T^n=1N∑i=1NTn(i)T_n(x)= _c (n) ( F_nc(x) _nc )^2,\ T_n= 1N _i=1^NT_n(x_i) (5) so that ∑n∈Tn() _n T_n(x) recovers the trace in Eq. (3) exactly, and T^n T_n is its empirical average at node n. The collection T^n\ T_n\ thus gives a node-level view of where the model’s sharpness actually resides, we analyze this in the next section. When Global Sharpness Fails The global trace regularizer was motivated by the observation that reducing sharpness improves generalization by enabling convergence to flatter optima (32). However, as we show in this section, treating sharpness as a single global measure obscures how curvature is distributed across the circuit and which components give rise to it. We identify three empirical observations that motivate a more structured view of sharpness and its regularization. Table 1: Mean percentage change in test NLL (Δ ), reduction in degree of overfitting (Δ ), and decrease in the loss-surface sharpness (Δ ) across 20 DEBD binary datasets, achieved by global-trace regularized HCLT, compared to non-regularized baseline, averaged over 5 runs. Data % Δ -NLL Δ Δ 1%1\% 7.127.12 8.818.81 23.0423.04 5%5\% 2.192.19 5.795.79 15.3215.32 10%10\% 0.270.27 2.732.73 10.4210.42 50%50\% −0.48-0.48 0.210.21 5.555.55 100%100\% −0.31-0.31 0.060.06 4.044.04 Figure 2: Training and test log-likelihood trajectories for the vanilla and global trace-regularized models on accidents, baudio, and ad using the full training data. Global trace regularization reduces the train-test gap but lowers both training and test likelihood, indicating underfitting rather than improved generalization. Observation 1: A flatter optimum need not be a better one. Table 1 shows that the global trace regularizer helps reach flatter optima but can nevertheless attain lower test log-likelihood than the unregularized model when sufficient data are available. This degradation is not explained by a widening train-test gap, as the corresponding train log-likelihood also decreases, as shown in Figure 2. It therefore drives learning toward solutions that are flatter yet fit both the training and test distributions less well, indicating underfitting. Reducing overall curvature is thus not sufficient, understanding how that curvature is important to preserve model capacity. Figure 3: Only a small fraction of sum nodes accounts for most of the global Hessian trace. Cumulative share of the total trace as sum nodes are ranked by their empirical contribution (T^n)( T_n). Across datasets, the distribution is highly concentrated: a small set of nodes dominates the global sharpness signal, while most nodes contribute negligibly. Observation 2: Global sharpness is concentrated in a small set of nodes. Examining how the global Hessian trace is distributed across the circuit reveals that the node-wise contributions (T^n)( T_n) defined in Eq (5) are highly nonuniform. As shown in Figure 3, a small minority (<10%)(<10\%) of sum nodes accounts for nearly the entire trace (99.99%)(99.99\%), while the contributions of other nodes are negligible. Global sharpness is therefore not spread evenly across the model, but concentrated in a small set of circuit components. This also reveals a limitation of using a single global regularization strength. The regularizer penalizes all sum nodes equally despite their markedly different contributions to the total curvature, and cannot distinguish the few nodes that dominate the sharpness signal from the many nodes that are already effectively flat. Figure 4: Targeting the largest global trace contributors worsens underfitting. Change in test log-likelihood relative to the unregularized model when trace regularization is applied to all sum nodes or restricted to progressively smaller fractions ranked by T^n T_n. Selecting fewer top-ranked nodes generally worsens performance, showing that global contribution is a poor criterion for allocating regularization. Observation 3: Targeting the largest contributors makes underfitting worse. The concentration observed above suggests a natural solution: restrict regularization to the nodes with the largest contributions to the global trace. If underfitting were caused primarily by applying the penalty to many negligible contributors that are already sufficiently flat, this strategy could preserve capacity while retaining curvature control. However, as shown in Figure 4, the test log-likelihood degrades further as the penalty is restricted to progressively fewer top-(T^n)( T_n) nodes. Thus, although (T^n)( T_n) measures a node’s contribution to global sharpness, it does not identify where regularization should act. Taken together, these observations suggest that neither greater flattening nor restricting regularization to fewer high-contribution nodes is sufficient to resolve the observed degradation. They instead raise a more fundamental question: what does a node’s global trace contribution actually measure, and why can it be a poor signal for allocating regularization? A Compositional Theory of Sharpness To address the above puzzle, we first show theoretically that a node’s global trace contribution factorizes exactly into a structural factor and a local curvature factor. We characterize them geometrically, and derive the conditions under which the two induce different node rankings. Together, these results formalize the thesis that curvature in a PC is generated locally and expressed globally through circuit flow. Exact Global-Local Decomposition (a) Contextual Usage (Fn)(F_n) (b) Intrinsic local curvature (tn)(t_n) (c) Global trace contribution (Tn)(T_n) (d) Flow-curvature interaction Figure 5: Compositional structure of sharpness in a trained probabilistic circuit. For each input, a sum node’s global trace contribution factorizes as Tn=Fn2tnT_n=F_n^2t_n, separating local curvature from its amplification through circuit flow. Panels (a)–(c) show the same circuit nodes colored by empirical contextual usage, local curvature, and global contribution, while panel (d) visualizes their interaction, showing that nodes with the large global curvature contribution need not have the large local curvature. Theorem 1. Consider any sum node n with strictly positive outgoing weights in a smooth, decomposable PC. For any input x, its global trace contribution Tn()T_n(x) factorizes as Tn()=Fn()2tn(),T_n(x)=F_n(x)^2\,t_n(x), tn()=∑c∈ch(n)(pc()/pn())2t_n(x)= _c (n)(p_c(x)/p_n(x))^2 Consequently, Tr(∇2ℓ(,))=∑n∈Fn()2tn().Tr ( _ θ^2 ( θ;x) )= _n F_n(x)^2\,t_n(x). The decomposition is exact and follows directly from the edge-flow factorization. Moreover, both factors are available from the same upward-downward computation used to evaluate circuit flows, so computing the decomposition adds no asymptotic cost. The two factors, however, have distinct interpretations. The term Fn()2F_n(x)^2 measures the node’s contextual usage: how strongly the circuit output depends on node n for input x. The term tn()t_n(x), as we will show, is the node’s local trace, determined entirely by the outputs of its local mixture. Hence, a large global contribution TnT_n may arise from large local curvature, large contextual usage, or both. A node that is frequently used can dominate the global trace even when its local mixture is not among the sharpest, while a locally sharp node can contribute little globally if little flow reaches it. Under uniform penalization, this can suppress high-usage components regardless of their local geometry, leading regularization to act where curvature is globally amplified rather than where it is intrinsically largest. Figure 5 makes the distinction concrete. The nodes that dominate the global trace in panel (c) align more closely with high contextual usage in panel (a) than with the largest local traces in panel (b). This mismatch motivates a closer examination of the two factors: we first characterize the local geometry summarized by tnt_n, and then study how FnF_n transports that geometry through the circuit. Rank-One Local Geometry The local factor tnt_n is more than a convenient scalar summary, and as we show below, it captures the entire local second-order geometry of a sum node. Consider a sum node n in isolation, with local negative log-output ℓn(n,)=−log(∑cθncpc()) _n( θ_n;x)=- ( _c _ncp_c(x) ), and write ρnc()=pc()/pn() _nc(x)=p_c(x)/p_n(x) for the output ratios, collected into the vector n ρ_n. Proposition 1 (Rank-one local Hessian). The gradient and Hessian of ℓn _n in the edge weights n θ_n are ∇nℓn=−n _ θ_n _n=- ρ_n and Hn()=∇n2ℓn=nn⊤H_n(x)=∇^2_ θ_n _n= ρ_n ρ_n . Hence, HnH_n is positive semidefinite and, whenever n≠0 ρ_n≠ 0, has rank one, with unique nonzero eigenvalue λmax(Hn)=∥n∥22=tn _ (H_n)= ρ_n _2^2=t_n. Consequently tn=Tr(Hn)=∥Hn∥2=∥Hn∥Ft_n=Tr(H_n)= H_n _2= H_n _F. Thus, the local second-order geometry of a sum node collapses to a single scalar, the quantity tnt_n, which is simultaneously the local hessian trace, the maximum ambient curvature, and the total Hessian magnitude. Context Propagation and the Depth Bias The contextual factor Fn2F_n^2 is what makes TnT_n and tnt_n differ, and it induces a systematic depth bias in the global trace: because flow is partitioned among children at every sum node on the way down from the root, shallow nodes accumulate more of it than deep ones, so the global contribution Tn=Fn2tnT_n=F_n^2t_n favors structurally shallow nodes regardless of their intrinsic curvature. To understand this bias better, recall a standard consequence of the circuit-flow recursion: product nodes transmit flow unchanged, whereas sum nodes scale it by posterior routing responsibilities. Unrolling the flow recursion thus shows that flow is in fact attenuated by the specific edges it traverses, rather than depth as such. Lemma 1. For a tree-structured PC, the node flow is the product of routing responsibilities along the unique root-to-n path π(r,n)π(r,n), taken over its sum edges (EsumE_sum) only: Fn()=∏e∈π(r,n)∩Esumre()F_n(x)= _e∈π(r,n)∩ E_sumr_e(x), with r(m,n′)=θmn′pn′/pmr_(m,n )= _mn \,p_n /p_m. For a DAG, Fn()=∑π∈Π(r,n)∏e∈π∩Esumre()F_n(x)= _π∈ (r,n) _e∈π∩ E_sumr_e(x), summing over all root-to-n paths Π(r,n) (r,n). Two consequences follow. First, product depth does not attenuate flow: only sum edges do, so the notion of “depth” relevant to sharpness is the number of upstream sum edges rather than the raw topological depth. Second, in a DAG a shared node can receive flow through several contexts, and converging paths may compensate for path-wise attenuation; flow therefore need not decay monotonically with depth. The following conditional bound however holds along tree paths. Corollary 1. Consider a tree-structured PC. If every sum-edge responsibility on the root-to-n path satisfies re()≤ρ<1r_e(x)≤ρ<1, then Fn()≤ρdΣ(n),F_n(x)≤ρ^d_ (n), and Tn()≤ρ2dΣ(n)tn(),T_n(x)≤ρ^2d_ (n)t_n(x), where dΣ(n)d_ (n) is the number of upstream sum edges. The corollary formalizes a mechanism in which the contextual factor can suppress a locally sharp node n reached only through many attenuating routing decisions: its curvature is present, but its global trace contribution is discounted geometrically in the number of upstream sum edges dΣ(n)d_ (n). Thus, the global trace can concentrate on shallow, heavily-used nodes, which we also confirm as illustrated in Figure 6. Global-Local Ranking Reversals The decomposition also implies that ranking nodes by their global sharpness contribution is not the same as ranking them by their intrinsic local curvature. A simple analysis gives us the following exact condition for disagreement. Proposition 2. For any two sum nodes i,ji,j, Ti>Tj⇔ti/tj>(Fj/Fi)2T_i>T_j t_i/t_j>(F_j/F_i)^2. In particular, a locally less curved node i with ti<tjt_i<t_j outranks a locally sharper node j whenever Fi/Fj>tj/tiF_i/F_j> t_j/t_i. The two orderings answer different questions: ranking by TnT_n identifies which parameter block strongly affects the root likelihood, whereas ranking by tnt_n identifies which local mixture computation is most sharply curved. Because TnT_n is amplified by the structural factor Fn2F_n^2, which is largest at shallow, heavily-used nodes, the top-TnT_n nodes need not be the locally sharp ones, accounting for the failure of top-TnT_n selection we saw earlier. The ranking reversals are not a theoretical edge case: in the trained circuit of Figure 5, the root node combines maximal usage with the lowest local trace among the labeled nodes yet attains the largest global contribution, while a deep node with local trace larger by nearly three orders of magnitude contributes almost nothing globally. Figure 6: Global trace contributions are biased toward early circuit partitions. Normalized global trace contribution across depth partitions on three representative benchmarks. The observed concentration near earlier partitions is consistent with attenuation through upstream sum-node routing, although raw depth alone does not determine flow. Figure 7: Local trace is lesser concentrated. Cumulative share of ∑nt^n _n t_n as nodes are ranked by t^n t_n. On the evaluated circuits, the local trace is distributed more broadly than the global contribution, indicating that part of the concentration in T^n T_n is introduced by contextual usage. Figure 8: Local-curvature selection preserves fit more effectively than global-contribution selection. Change in test log-likelihood relative to the unregularized model as regularization is applied to all sum nodes or restricted to progressively smaller fractions ranked by global contribution or local curvature. Selecting by global contribution degrades performance whereas selecting by local curvature preserves the baseline on baudio and improves it on ad. Adaptive Sharpness-Aware Learning The decomposition suggests that global contribution and local curvature should play different roles during learning. The global term T^n=^[Fn2tn] T_n= E[F_n^2t_n] measures a node’s contribution to root-level curvature, but also reflects its contextual usage. We therefore propose to use the local trace to determine where regularization should act, while retaining the global trace penalty as the quantity being controlled. This gives a simple modification of global sharpness-aware learning: each sum node receives a gate derived from its local curvature, and the existing trace penalty is scaled by that gate. Local-Curvature Gating Let t^n=1N∑i=1Ntn(i) t_n= 1N _i=1^Nt_n(x_i) denote the empirical local trace of node n. We assign each sum node a gate ωn∈[0,1] _n∈[0,1] and define R()=∑n∈ωnT^n=1N∑i=1N∑n∈ωn∑c∈ch(n)(Fnc(i)θnc)2R_ ω( θ)= _n _n T_n= 1N _i=1^N _n _n _c (n) ( F_nc(x_i) _nc )^2 The global trace regularizer is recovered when ωn=1 _n=1 for every node. We choose ωn=g(t^n) _n=g( t_n) with g monotone increasing, so that nodes with larger local curvature receive stronger regularization independently of their flow. To reduce sensitivity to the scale of t^n t_n, we use the bounded gate ωn=t^nmaxm∈t^m, _n= t_n _m t_m, (6) The node with the largest empirical local trace receives the full penalty, while the remaining nodes are scaled by their curvature relative to that maximum. Gates are recomputed from the current model and held fixed during each parameter update. This separates two roles that are coupled by the global regularizer: t^n t_n allocates regularization across nodes, while T^n T_n remains the curvature contribution being penalized. Table 2: Test log-likelihood on DEBD benchmark datasets (higher is better, mean ± std over 5 seeds). Best μ/method selected via validation L. Dataset Vanilla Global Trace Gated Regularization accidents -26.64± 0.02 -29.79± 0.13 -26.55± 0.01 ad -18.40± 0.09 -20.36± 0.05 -18.10± 0.04 baudio -39.62± 0.02 -42.45± 0.02 -39.62± 0.01 bbc -269.58± 0.41 -256.40± 0.10 -260.07± 0.26 bnetflix -56.22± 0.02 -59.67± 0.02 -56.21± 0.02 book -34.34± 0.04 -36.35± 0.03 -34.16± 0.01 c20ng -151.99± 0.13 -157.64± 0.05 -152.22± 0.13 cr52 -99.09± 1.15 -102.23± 1.45 -97.99± 0.70 cwebkb -154.71± 0.33 -157.30± 0.02 -152.90± 0.15 dna -87.78± 0.24 -81.33± 0.00 -81.28± 0.08 jester -52.82± 0.03 -55.74± 0.05 -52.82± 0.03 kdd -2.22± 0.01 -2.36± 0.00 -2.22± 0.00 kosarek -10.59± 0.01 -11.08± 0.04 -10.59± 0.02 msnbc -6.12± 0.01 -6.47± 0.01 -6.08± 0.00 msweb -9.73± 0.01 -10.40± 0.02 -9.73± 0.01 nltcs -6.00± 0.00 -6.48± 0.00 -6.00± 0.00 plants -12.68± 0.03 -15.99± 0.02 -12.67± 0.01 pumsb_star -22.78± 0.06 -28.29± 0.06 -22.75± 0.04 tmovie -42.16± 0.03 -48.84± 0.04 -42.21± 0.06 tretail -10.84± 0.01 -10.98± 0.01 -10.85± 0.01 Gated EM Update Let Nnc=∑iFnc(i)N_nc= _iF_nc(x_i) and Snc=∑iFnc(i)2S_nc= _iF_nc(x_i)^2 denote the expected edge count and edge-flow second moment. With fixed gates, directly optimizing the empirical trace penalty gives the following objective: max∑cnNnclogθnc−μωn∑cSncθnc2s.t.∑cθnc=1. _ θ_n\ _cN_nc _nc-μ _n _c S_nc _nc^2 .t. _c _nc=1. Its stationarity condition, Nnc/θnc−λ+2μωnSnc/θnc3=0N_nc/ _nc-λ+2μ _nS_nc/ _nc^3=0, is cubic in θnc _nc. To preserve the tractable EM updates of global trace-regularized learning, we apply the same surrogate as used by (32) with the node-specific strength μn=μωn _n=μ _n, which yields the following update equation. Proposition 3. For fixed gates ωnn∈\ _n\_n , under the surrogate used by global trace-regularized EM, the update for each outgoing weight of sum node n satisfies λnθnc2−Nncθnc−μωnNnc=0, _n _nc^2-N_nc _nc-μ _nN_nc=0, and its positive solution is given by θnc=Nnc+Nnc2+4λnμωnNnc2λn, _nc= N_nc+ N_nc^2+4 _nμ _nN_nc2 _n, (7) The update thus preserves the form and asymptotic complexity of the global method, changing only the effective node-wise strength, μ↦μωnμ μ _n. It recovers global trace regularization when ωn=1 _n=1 and approaches the unregularized EM update as ωn→0 _n→ 0. Though we adopt this simple gate as proof of concept, Prop. 3 holds for any monotone gate, and richer gate functions are natural directions for future work. Experiments We organize the experiments around three questions that connect the theory to the proposed method: (Q1) How strongly does contextual flow shape the distribution of global trace contributions? (Q2) Does selecting nodes by local curvature preserve model fit more effectively than selecting by global contribution? (Q3) Does global trace regularization underfit, and can adaptive local-curvature gating recover the lost capacity? Experimental setup. We evaluate on the 20 binary density-estimation benchmarks (DEBD) using Hidden Chow-Liu Trees (HCLTs) (18). All models are implemented in PyJuice (17) with a latent size 100100 and trained using EM. We compare the unregularized model, global trace regularization, and the proposed local-curvature-gated regularizer. Results are averaged over 55 random seeds, and the regularization strength μ is selected independently for each method using validation log-likelihood. Further experimental details are provided in the appendix. Q1: Anatomy of the global trace. We first examine how the factors in Tn=Fn2tnT_n=F_n^2t_n shape global sharpness. Figure 6 shows that global trace contributions concentrate in earlier circuit partitions, consistent with attenuation theory through upstream sum-node routing. Figure 7 further shows that local curvature is substantially less concentrated than global contribution: the top 10%10\% of nodes account for more than 99.99%99.99\% of T^n T_n, but it takes over 60%60\% of nodes to contribute the same for t^n t_n. Thus, concentration in the global trace is primarily caused by the contextual usage than local curvature. Q2: Controlled node selection. We next isolate the effect of the ranking criterion. Holding the regularization strength and selected fraction fixed, we apply the trace penalty to nodes chosen by global contribution T^n T_n or local curvature t^n t_n. Figure 8 compares the two criteria on baudio and ad. Restricting the penalty to progressively smaller sets of high-T^n T_n nodes reduces test log-likelihood on both datasets. In contrast, selecting by t^n t_n preserves the unregularized fit on baudio and improves it slightly on ad. These results show that global contribution and local curvature provide materially different signals for allocating regularization. Q3: Underfitting and recovery. Table 2 compares the three methods across all 20 DEBD datasets. Global trace regularization improves over the unregularized model on only 2 datasets and degrades performance on the remaining 18, indicating underfitting. In contrast, the proposed gated method outperforms global trace regularization and matches or improves upon the unregularized model on majority of the datasets. These results show that allocating the trace penalty according to local curvature substantially mitigates the loss of fit induced by uniform global regularization. Conclusion Overall, in this paper we studied sharpness aware learning in probabilistic circuits through a compositional lens. Our analysis showed that a node’s global curvature contribution separates exactly into contextual usage and intrinsic local curvature, providing insights into why global trace regularization can flatten the wrong parts of the model and induce underfitting. Guided by this decomposition, we introduced a gated regularizer that preserves tractable learning, while allocating regularization effectively to prevent underfitting. Future work involves developing richer gating functions that jointly account for local geometry and contextual influence. More broadly, the decomposition also opens up directions for curvature-aware model compression, targeted robustness interventions, and adaptive circuit design, where contextual usage and local sensitivity can jointly guide which components to preserve, regularize, prune, or expand. Acknowledgments SN and S gratefully acknowledge the generous support by the AFOSR award FA9550-23-1-0239, the ARO award W911NF2010224 and the DARPA Assured Neuro Symbolic Learning and Reasoning (ANSR) award HR001122S0039. CK and HS gratefully acknowledge Dr. Anji Liu for the discussions related to the work and CK, HS, SPV and YZ thank IIT Palakkad for the access to Madhava Cluster. References Ahmed et al. (2022) K. Ahmed, S. Teso, K. Chang, G. V. den Broeck, and A. Vergari Semantic probabilistic layers for neuro-symbolic learning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, Cited by: Introduction. Andriushchenko et al. (2023) M. Andriushchenko, F. Croce, M. Müller, M. Hein, and N. Flammarion A modern look at the relationship between sharpness and generalization. In Proceedings of the 40th International Conference on Machine Learning, Vol. 202, p. 840–902. Cited by: Sharpness and Regularization in PCs. Bekker et al. (2015) J. Bekker, J. Davis, A. Choi, A. Darwiche, and G. Van den Broeck Tractable learning for complex probability queries. Advances in Neural Information Processing Systems 28. Cited by: Appendix C. Braun et al. (2025) S. Braun, S. Sidheekh, A. Vergari, M. Mundt, S. Natarajan, and K. Kersting Tractable representation learning with probabilistic circuits. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: Introduction. Choi et al. (2020) Y. Choi, A. Vergari, and G. V. den Broeck Probabilistic circuits: a unifying framework for tractable probabilistic models. Cited by: Introduction, Probabilistic Circuits. Darwiche (2003) A. Darwiche A differential approach to inference in bayesian networks. Journal of the ACM (3), p. 280–305. Cited by: Probabilistic Circuits. Dinh et al. (2017) L. Dinh, R. Pascanu, S. Bengio, and Y. Bengio Sharp minima can generalize for deep nets. In International Conference on Machine Learning, p. 1019–1028. Cited by: Sharpness and Regularization in PCs. Foret et al. (2021) P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur Sharpness-aware minimization for efficiently improving generalization. In 9th International Conference on Learning Representations, 2021, Cited by: Introduction, Sharpness and Regularization in PCs. Goodfellow et al. (2014) I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio Generative adversarial nets. In Advances in Neural Information Processing Systems 27: Annual Conference, Cited by: Introduction. Hochreiter and Schmidhuber (1997) S. Hochreiter and J. Schmidhuber Flat minima. Neural computation (1), p. 1–42. Cited by: Sharpness and Regularization in PCs. Karanam et al. (2025) A. Karanam, S. Mathur, S. Sidheekh, and S. Natarajan A unified framework for human-allied learning of probabilistic circuits. In AAAI Conference on Artificial Intelligence, 2025, Vol. 39, p. 17779–17787. Cited by: Introduction. Kaur et al. (2023) S. Kaur, J. Cohen, and Z. C. Lipton On the maximum hessian eigenvalue and generalization. In Proceedings of Machine Learning Research, External Links: ISSN 26403498 Cited by: Introduction. Keskar et al. (2017) N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang On large-batch training for deep learning: generalization gap and sharp minima. In 5th International Conference on Learning Representations, 2017, Cited by: Sharpness and Regularization in PCs. Kingma and Welling (2014) D. P. Kingma and M. Welling Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, 2014, Cited by: Introduction. Kisa et al. (2014) D. Kisa, G. V. D. Broeck, A. Choi, and A. Darwiche Probabilistic sentential decision diagrams. In International Conference on Knowledge Representation and Reasoning, External Links: ISSN 23341033 Cited by: Probabilistic Circuits. Li et al. (2018) H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, p. 6391–6401. Cited by: Appendix D. Liu et al. (2024a) A. Liu, K. Ahmed, and G. V. den Broeck Scaling tractable probabilistic circuits: A systems perspective. In Forty-first International Conference on Machine Learning, 2024, Cited by: 1st item, Sharpness and Regularization in PCs, Experimental setup.. Liu and den Broeck (2021) A. Liu and G. V. den Broeck Tractable regularization of probabilistic circuits. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, 2021, p. 3558–3570. Cited by: 1st item, Probabilistic Circuits, Probabilistic Circuits, Sharpness and Regularization in PCs, Experimental setup.. Liu et al. (2024b) A. Liu, M. Niepert, and G. V. den Broeck Image inpainting via tractable steering of diffusion models. In The Twelfth International Conference on Learning Representations, 2024, Cited by: Introduction. Němeček et al. (2025) J. Němeček, T. Pevnỳ, and J. Marecek Generating likely counterfactuals using sum-product networks. In International Conference on Learning Representations, Vol. 2025, p. 69804–69835. Cited by: Introduction. Papamakarios et al. (2021) G. Papamakarios, E. T. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, p. 57:1–57:64. Cited by: Introduction. Peharz et al. (2020) R. Peharz, S. Lang, A. Vergari, K. Stelzner, A. Molina, M. Trapp, G. V. den Broeck, K. Kersting, and Z. Ghahramani Einsum networks: fast and scalable learning of tractable probabilistic circuits. In 37th International Conference on Machine Learning, 2020, Proceedings of Machine Learning Research, p. 7563–7574. Cited by: Sharpness and Regularization in PCs. Peharz et al. (2019) R. Peharz, A. Vergari, K. Stelzner, A. Molina, M. Trapp, X. Shao, K. Kersting, and Z. Ghahramani Random sum-product networks: A simple and effective approach to probabilistic deep learning. In Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, 2019, Proceedings of Machine Learning Research, p. 334–344. Cited by: Sharpness and Regularization in PCs. Poon and Domingos (2011) H. Poon and P. M. Domingos Sum-product networks: A new deep architecture. In Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, 2011, p. 337–346. Cited by: Probabilistic Circuits. Rahman et al. (2014) T. Rahman, P. Kothalkar, and V. Gogate Cutset networks: a simple, tractable, and scalable approach for improving the accuracy of chow-liu trees. In Machine Learning and Knowledge Discovery in Databases: European Conference, 2014, p. 630–645. Cited by: Probabilistic Circuits. Sabu et al. (2026) A. Sabu, V. S, and N. C. Krishnan PAR: plausibility-aware amortized recourse generation. External Links: 2601.17309, Link Cited by: Introduction. Sankar et al. (2021) A. R. Sankar, Y. Khasbage, R. Vigneswaran, and V. N. Balasubramanian A deeper look at the hessian eigenspectrum of deep neural networks and its applications to regularization. In Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, The Eleventh Symposium on Educational Advances in Artificial Intelligence, p. 9481–9488. Cited by: Introduction. Sidheekh et al. (2023) S. Sidheekh, K. Kersting, and S. Natarajan Probabilistic flow circuits: towards unified deep models for tractable probabilistic inference. In Uncertainty in Artificial Intelligence, 2023, Proceedings of Machine Learning Research, p. 1964–1973. Cited by: Sharpness and Regularization in PCs. Sidheekh and Natarajan (2024) S. Sidheekh and S. Natarajan Building expressive and tractable probabilistic generative models: A review. In Thirty-Third International Joint Conference on Artificial Intelligence, 2024, p. 8234–8243. Cited by: Sharpness and Regularization in PCs. Sidheekh and Natarajan (2026) S. Sidheekh and S. Natarajan Geometry-aware probabilistic circuits via voronoi tessellations. In Forty-third International Conference on Machine Learning, Cited by: Sharpness and Regularization in PCs. Sidheekh et al. (2025) S. Sidheekh, P. Tenali, S. Mathur, E. Blasch, K. Kersting, and S. Natarajan Credibility-aware multimodal fusion using probabilistic circuits. In The 28th International Conference on Artificial Intelligence and Statistics, Cited by: Introduction. Suresh et al. (2026) H. Suresh, S. Sidheekh, V. S. M. P, S. Natarajan, and N. C. Krishnan Tractable sharpness-aware learning of probabilistic circuits. Proceedings of the AAAI Conference on Artificial Intelligence 40 (30), p. 25736–25744. External Links: Document Cited by: Introduction, Introduction, Sharpness and Regularization in PCs, When Global Sharpness Fails, Gated EM Update. Van Haaren and Davis (2012) J. Van Haaren and J. Davis Markov network structure learning: a randomized feature generation approach. In AAAI Conference on Artificial Intelligence, Vol. 26, p. 1148–1154. Cited by: Appendix C. Ventola et al. (2023) F. Ventola, S. Braun, Z. Yu, M. Mundt, and K. Kersting Probabilistic circuits that know what they don’t know. In Uncertainty in Artificial Intelligence, 2023, Proceedings of Machine Learning Research, p. 2157–2167. Cited by: Sharpness and Regularization in PCs. Zhang et al. (2023) H. Zhang, M. Dang, N. Peng, and G. V. den Broeck Tractable control for autoregressive language generation. In International Conference on Machine Learning,2023, Proceedings of Machine Learning Research, p. 40932–40945. Cited by: Introduction. Supplementary Material: A Compositional Theory of Curvature in Probabilistic Circuits Appendix A Preliminaries and Notation We first revisit the definitions and notation used throughout the proofs, so that each subsequent argument can be read in isolation. Probabilistic circuits. We consider smooth and decomposable probabilistic circuits with alternating layers of sum and product nodes over a set of tractable input distributions. The root node r computes the model density pr()p_r(x). Each sum node n∈n computes a convex combination of its children, pn()=∑c∈ch(n)θncpc()p_n(x)= _c (n) _nc\,p_c(x), with nonnegative weights normalized on the simplex, ∑c∈ch(n)θnc=1 _c (n) _nc=1. We write ℓ(,)=−logpr() ( θ;x)=- p_r(x) for the per-sample negative log-likelihood (NLL) and study the trace of its Hessian with respect to the sum weights θ. Output ratios and routing responsibilities. For a sum node n and child c we define the output ratio ρnc()=pc()/pn() _nc(x)=p_c(x)/p_n(x) and the routing responsibility rnc()=θncρnc()r_nc(x)= _nc\, _nc(x). The responsibilities form a distribution over children, ∑c∈ch(n)rnc()=1 _c (n)r_nc(x)=1, and we collect the output ratios into the vector n()=(ρnc())c∈ch(n) ρ_n(x)=( _nc(x))_c (n). Circuit flow. The circuit flow measures the top-down usage of each node, and is defined by the node-type recursion below. Definition 1 (Circuit flow). The flow Fn()F_n(x) of a node n is Fn()=1,n=r(root),∑m∈pa(n)Fm(),na sum node,∑m∈pa(n)θmnpn()pm()Fm(),na product or input node,F_n(x)= cases1,&n=r\ (root),\\[4.0pt] _m (n)F_m(x),&n\ a sum node,\\[10.0pt] _m (n) _mn\, p_n(x)p_m(x)\,F_m(x),&n\ a product or input node, cases (8) where pa(n)pa(n) denotes the parents of n. By the alternating-layer property, the responsibility factor θmnpn/pm=rmn _mn\,p_n/p_m=r_mn attaches on exactly the sum edges of the circuit (edges out of a sum node), while product and input nodes inherit their parents’ flow undivided. Edge flow. The flow carried by the edge from sum node n to child c is Fnc()=θncρnc()Fn()=rnc()Fn()F_nc(x)= _nc\, _nc(x)\,F_n(x)=r_nc(x)\,F_n(x). It satisfies Fnc/θnc=ρncFnF_nc/ _nc= _nc\,F_n. Local and global trace. The local trace of sum node n is tn()=∑c∈ch(n)ρnc()2t_n(x)= _c (n) _nc(x)^2, and its global trace contribution is Tn()=∑c∈ch(n)(Fnc()/θnc)2T_n(x)= _c (n)(F_nc(x)/ _nc)^2. Theorem 1 shows these are related by the exact identity Tn=Fn2tnT_n=F_n^2\,t_n. Summing over sum nodes recovers the full NLL Hessian trace, Tr(∇2ℓ)=∑n∈Tn≥0Tr(∇^2_ θ )= _n T_n≥ 0. Empirical batch estimators are denoted t^n t_n and T^n T_n. Appendix B Proofs for Theoretical Results In this section, we provide complete, self-contained proofs for all of the key lemmas, propositions, and theorems stated in the main paper Theorem 2. Consider any sum node n with strictly positive outgoing weights in a smooth, decomposable PC. For any input x, its global trace contribution Tn()T_n(x) factorizes as Tn()=Fn()2tn(),T_n(x)=F_n(x)^2\,t_n(x), tn()=∑c∈ch(n)(pc()/pn())2t_n(x)= _c (n)(p_c(x)/p_n(x))^2 Consequently, Tr(∇2ℓ(,))=∑n∈Fn()2tn().Tr ( _ θ^2 ( θ;x) )= _n F_n(x)^2\,t_n(x). Proof. The global trace contribution of node n is the sum over its outgoing edges of the squared diagonal Hessian entries, which equal (Fnc()/θnc)2(F_nc(x)/ _nc)^2. Substituting the edge-flow identity Fnc/θnc=ρncFnF_nc/ _nc= _nc\,F_n from Section A, Tn()=∑c∈ch(n)(Fnc()θnc)2=∑c∈ch(n)(pc(x)pn(x)Fn(x))2.T_n(x)= _c (n) ( F_nc(x) _nc )^2= _c (n) ( p_c(x)p_n(x)\,F_n(x) )^2. (9) The flow Fn()F_n(x) does not depend on the child index c, so it factors out of the sum, Tn()=Fn(x)2∑c∈ch(n)(pc(x)pn(x))2.T_n(x)=F_n(x)^2\ _c (n) ( p_c(x)p_n(x) )^2. (10) Recognizing the remaining sum as the local trace tn()=∑c∈ch(n)ρnc()2t_n(x)= _c (n) _nc(x)^2 yields the factorization, Tn()=Fn(x)2tn().T_n(x)=F_n(x)^2\ t_n(x). (11) Summing over all sum nodes gives the full Hessian trace Tr(∇2ℓ)=∑n∈Fn2tn≥0Tr(∇^2_ θ )= _n F_n^2\,t_n≥ 0, since each term is a product of squares. ∎ Proposition 4 (Rank-one local Hessian). The gradient and Hessian of ℓn _n in the edge weights n θ_n are ∇nℓn=−n _ θ_n _n=- ρ_n and Hn()=∇n2ℓn=nn⊤H_n(x)=∇^2_ θ_n _n= ρ_n ρ_n . Hence, HnH_n is positive semidefinite and, whenever n≠0 ρ_n≠ 0, has rank one, with unique nonzero eigenvalue λmax(Hn)=∥n∥22=tn _ (H_n)= ρ_n _2^2=t_n. Consequently tn=Tr(Hn)=∥Hn∥2=∥Hn∥Ft_n=Tr(H_n)= H_n _2= H_n _F. Proof. ℓn(n,)=−log(∑cθncpc()) _n( θ_n;x)=- ( _c _ncp_c(x) ), ρnc()=pc()/pn() _nc(x)=p_c(x)/p_n(x), pn()=∑cθncpc()p_n(x)= _c _ncp_c(x) First order derivative ∂ℓn(n,)∂θnc=−∂log(∑cθncpc())∂θnc=−1pn()∂∑cθncpc()∂θnc ∂ _n( θ_n;x)∂ _nc= -∂ ( _c _ncp_c(x) )∂ _nc= -1p_n(x) ∂ _c _ncp_c(x)∂ _nc (12) ∂ℓn(n,)∂θnc=−(pc()pn()) ∂ _n( θ_n;x)∂ _nc=- ( p_c(x)p_n(x) ) (13) ⟹∇nℓn=−n _ θ_n _n=- ρ_n (14) Second order derivative ∂2ℓn(n,)∂θnc′θnc=−∂(pc()/pn())∂θnc′=−[pn()pc()′−pn()′pc()pn()2] ∂^2 _n( θ_n;x)∂ _nc _nc=- ∂ (p_c(x)/p_n(x) )∂ _nc =- [ p_n(x)\ p_c(x) -p_n(x) \ p_c(x)p_n(x)^2 ] (15) As pc()p_c(x) is independent of θnc′ _nc we get ∂(pc()/pn())∂θnc′=−[−pc′()pc()pn()2]=pc′()pn()pc()pn()=ρnc′ρnc ∂ (p_c(x)/p_n(x) )∂ _nc =- [ -p_c (x)\ p_c(x)p_n(x)^2 ]= p_c (x)p_n(x)\ p_c(x)p_n(x)= _nc \ _nc (16) Thus Hn=nn⊤H_n= ρ_n ρ_n . A nonzero outer product has rank one and unique nonzero eigenvalue ‖n‖22\| ρ_n\|_2^2. For a rank-one positive semidefinite matrix, the trace, spectral norm, and Frobenius norm all equal this eigenvalue. Trace of local Hessian Tr(Hn())=Tr(nn⊤)=∑c(pc()pn())2=tn()Tr(H_n(x))=Tr( ρ_n ρ_n )= _c ( p_c(x)p_n(x) )^2=t_n(x) (17) ∎ Proposition 5. For any two sum nodes i,ji,j, Ti>Tj⇔ti/tj>(Fj/Fi)2T_i>T_j t_i/t_j>(F_j/F_i)^2. In particular, a locally less curved node i with ti<tjt_i<t_j outranks a locally sharper node j whenever Fi/Fj>tj/tiF_i/F_j> t_j/t_i. Proof. Ti=Fi2ti,Tj=Fj2tjT_i=F_i^2\ t_i,T_j=F_j^2\ t_j Ti>Tj⇔Fi2ti>Fj2tjT_i>T_j F_i^2\ t_i>F_j^2\ t_j Ti>Tj⇔titj>(FjFi)2T_i>T_j t_it_j> ( F_jF_i )^2 ∎ Lemma 2. For a tree-structured PC, the node flow is the product of routing responsibilities along the unique root-to-n path π(r,n)π(r,n), taken over its sum edges (EsumE_sum) only: Fn()=∏e∈π(r,n)∩Esumre()F_n(x)= _e∈π(r,n)∩ E_sumr_e(x), with r(m,n′)=θmn′pn′/pmr_(m,n )= _mn \,p_n /p_m. For a DAG, Fn()=∑π∈Π(r,n)∏e∈π∩Esumre()F_n(x)= _π∈ (r,n) _e∈π∩ E_sumr_e(x), summing over all root-to-n paths Π(r,n) (r,n). Proof. We proceed by induction on the depth d of node n in the tree-structured PC. Base case (d=0d=0). Node n is the root r. By definition, Fr()=1F_r(x)=1. The product over an empty index set (no edges on the root-to-root path) is also 11, so the claim holds trivially. Inductive step. Assume the claim holds for every node at depth strictly less than d. Let n be a node at depth d, and let m=pa(n)m=pa(n) denote its unique parent ( PC is tree-structured). We consider two cases according to the type of the edge (m,n)(m,n). Case 1: (m,n)(m,n) is a product edge (i.e. m is a product node). By the flow recursion, product nodes propagate their flow without weighting: Fn()=Fm().F_n(x)\;=\;F_m(x). Since m is at depth d−1d-1, the inductive hypothesis gives Fm()=∏e∈π(r,m)∩Esumre().F_m(x)\;=\; _e\,∈\,π(r,m)\,∩\,E_sumr_e(x). Because (m,n)∉Esum(m,n)∉ E_sum, the sum-edge sets along the two paths coincide: π(r,n)∩Esum=π(r,m)∩Esumπ(r,n)∩ E_sum=π(r,m)∩ E_sum. Therefore Fn()=∏e∈π(r,n)∩Esumre().F_n(x)\ =\ _e\,∈\,π(r,n)\,∩\,E_sumr_e(x). (18) Case 2: (m,n)(m,n) is a sum edge (i.e. m is a sum node). By the flow recursion and the definition of routing responsibility r(m,n)()=θmnpn()/pm()r_(m,n)(x)= _mn\,p_n(x)/p_m(x): Fn()=θmnpn()pm()Fm()=r(m,n)()Fm().F_n(x)\;=\; _mn\, p_n(x)p_m(x)\,F_m(x)\;=\;r_(m,n)(x)\,F_m(x). Since m is at depth d−1d-1, the inductive hypothesis gives Fm()=∏e∈π(r,m)∩Esumre().F_m(x)\;=\; _e\,∈\,π(r,m)\,∩\,E_sumr_e(x). Because (m,n)∈Esum(m,n)∈ E_sum and π(r,n)=π(r,m)∪(m,n)π(r,n)=π(r,m)∪\(m,n)\, the new sum-edge factor yields Fn() F_n(x) =r(m,n)()⋅∏e∈π(r,m)∩Esumre() \;=\;r_(m,n)(x)· _e\,∈\,π(r,m)\,∩\,E_sumr_e(x) =∏e∈π(r,n)∩Esumre(). \;=\; _e\,∈\,π(r,n)\,∩\,E_sumr_e(x). Both cases establish the inductive step, so by strong induction the formula Fn()=∏e∈π(r,n)∩Esumre()F_n(x)\;=\; _e\,∈\,π(r,n)\,∩\,E_sumr_e(x) holds for every node n in the tree-structured PC. For general DAG structure Base case (d=0d=0). Node n is the root r. By definition Fr()=1F_r(x)=1. The only root-to-root path is the empty path π∅ _ ; its edge product equals 11 (empty product), and the sum over this single path equals 11. Hence the claim holds. Inductive step. Assume the claim holds for every node at depth strictly less than d≥1d≥ 1. Let n be a node at depth d with parent set pa(n)=m1,…,mkpa(n)=\m_1,…,m_k\, each parent at depth d−1d-1. By the inductive hypothesis, for every mi∈pa(n)m_i (n): Fmi()=∑π∈Π(r,mi)∏e∈π∩Esumre().F_m_i(x)\;=\; _π\,∈\, (r,\,m_i) _e\,∈\,π∩ E_sumr_e(x). (19) We consider two cases according to the type of n. Case 1: n is a sum node. By the alternating-layer property every parent mim_i is a product node, so every edge (mi,n)∉Esum(m_i,n)∉ E_sum. The flow recursion for a sum node gives Fn()=∑mi∈pa(n)Fmi().F_n(x)\;=\; _m_i\,∈\,pa(n)F_m_i(x). Substituting (19): Fn()=∑mi∈pa(n)∑π∈Π(r,mi)∏e∈π∩Esumre().F_n(x)\;=\; _m_i\,∈\,pa(n) _π\,∈\, (r,\,m_i) _e\,∈\,π∩ E_sumr_e(x). Every root-to-n path π∈Π(r,n)π∈ (r,n) passes through exactly one parent mi∈pa(n)m_i (n) and then traverses the product edge (mi,n)(m_i,n). Because (mi,n)∉Esum(m_i,n)∉ E_sum, adding this edge does not change the set of sum edges on the path: π∩Esum=π′∩Esum,where π′=π(r,mi) is the prefix of π up to mi.π∩ E_sum\;=\;π ∩ E_sum, π =π(r,m_i) is the prefix of π up to m_i. Hence the double sum exactly enumerates all root-to-n paths: Fn()=∑π∈Π(r,n)∏e∈π∩Esumre().F_n(x)\;=\; _π\,∈\, (r,n) _e\,∈\,π∩ E_sumr_e(x). Case 2: n is a product or input node. By the alternating-layer property every parent mim_i is a sum node, so every edge (mi,n)∈Esum(m_i,n)∈ E_sum carries routing responsibility r(mi,n)()=θminpn()/pmi()r_(m_i,n)(x)= _m_in\,p_n(x)/p_m_i(x). The flow recursion reads Fn()=∑mi∈pa(n)r(mi,n)()Fmi().F_n(x)\;=\; _m_i\,∈\,pa(n)r_(m_i,n)(x)\,F_m_i(x). Substituting (19): Fn()=∑mi∈pa(n)r(mi,n)()∑π′∈Π(r,mi)∏e∈π′∩Esumre().F_n(x)\;=\; _m_i\,∈\,pa(n)r_(m_i,n)(x) _π \,∈\, (r,\,m_i) _e\,∈\,π ∩ E_sumr_e(x). Every root-to-n path π∈Π(r,n)π∈ (r,n) decomposes uniquely as a prefix π′∈Π(r,mi)π ∈ (r,m_i) for exactly one mi∈pa(n)m_i (n) followed by the sum edge (mi,n)(m_i,n). Since (mi,n)∈Esum(m_i,n)∈ E_sum, appending this edge extends the edge product by exactly one factor: ∏e∈π∩Esumre()=r(mi,n)()⋅∏e∈π′∩Esumre(). _e\,∈\,π∩ E_sumr_e(x)\;=\;r_(m_i,n)(x)· _e\,∈\,π ∩ E_sumr_e(x). Re-indexing the double sum over all root-to-n paths therefore gives Fn() F_n(x) =∑mi∈pa(n)∑π′∈Π(r,mi)r(mi,n)()∏e∈π′∩Esumre() \;=\; _m_i\,∈\,pa(n)\; _π \,∈\, (r,\,m_i)r_(m_i,n)(x) _e\,∈\,π ∩ E_sumr_e(x) =∑π∈Π(r,n)∏e∈π∩Esumre(). \;=\; _π\,∈\, (r,n) _e\,∈\,π∩ E_sumr_e(x). Conclusion. Both cases establish the inductive step. By strong induction, Fn()=∑π∈Π(r,n)∏e∈π∩Esumre()F_n(x)\;=\; _π\,∈\, (r,n) _e\,∈\,π∩ E_sumr_e(x) ∎ Corollary 2. Consider a tree-structured PC. If every sum-edge responsibility on the root-to-n path satisfies re()≤ρ<1r_e(x)≤ρ<1, then Fn()≤ρdΣ(n),F_n(x)≤ρ^d_ (n), and Tn()≤ρ2dΣ(n)tn(),T_n(x)≤ρ^2d_ (n)t_n(x), where dΣ(n)d_ (n) is the number of upstream sum edges. Proof. Part 1: Bound on Fn()F_n(x). By the Lemma 1, the node flow at n in a tree-structured PC is the product of routing responsibilities over all sum edges on the unique root-to-n path: Fn()=∏e∈π(r,n)∩Esumre().F_n(x)\;=\; _e\,∈\,π(r,n)\,∩\,E_sumr_e(x). This product contains exactly dΣ(n)d_ (n) factors. By hypothesis, each factor satisfies re()≤ρ<1r_e(x)≤ρ<1, so: Fn()=∏e∈π(r,n)∩Esumre()≤ρdΣ(n).F_n(x)\;=\; _e\,∈\,π(r,n)\,∩\,E_sumr_e(x)\;≤\;ρ^\,d_ (n). Part 2: Bound on Tn()T_n(x). By the decomposition in Theorem 1: Tn()=Fn()2⋅tn().T_n(x)\;=\;F_n(x)^2· t_n(x). Since tn()≥0t_n(x)≥ 0 (it is a sum of squares), we may apply the flow bound directly: Tn()=Fn()2⋅tn()≤(ρdΣ(n))2⋅tn()=ρ 2dΣ(n)tn().T_n(x)\;=\;F_n(x)^2· t_n(x)\;≤\; (ρ^\,d_ (n) )^\!2· t_n(x)\;=\;ρ^\,2d_ (n)\,t_n(x). ∎ Adaptive Gated Regularization Proposition 6. For fixed gates ωnn∈\ _n\_n , under the surrogate used by global trace-regularized EM, the update for each outgoing weight of sum node n satisfies λnθnc2−Nncθnc−μωnNnc=0, _n _nc^2-N_nc _nc-μ _nN_nc=0, and its positive solution is given by θnc=Nnc+Nnc2+4λnμωnNnc2λn, _nc= N_nc+ N_nc^2+4 _nμ _nN_nc2 _n, (20) Proof. Adding the adaptive regularizer to the EM objective of the PC we obtain θ∗n⋅=argmaxθnc θ^*_n·= _ _nc ∑c∈ch(n)Nnc(x)logθnc Σ _c (n)N_nc(x)\> \, _nc subject to to ∑c∈ch(n)θnc=1, Σ _c (n) _nc=1, ωn(x)∑c∈ch(n)(Nnc(x)θnc)≤m _n(x)Σ _c∈ ch(n) ( N_nc(x) _nc )≤ m The Lagrangian formulation of the constrained maximization objective is ℒ(θn⋅,λ,μ)= ( _n·,λ,μ)= ∑c∈ch(n)Nnc(x)logθnc−λ(∑c∈ch(n)θnc−1) Σ _c (n)N_nc(x)\> \, _nc-λ (Σ _c (n) _nc-1 ) −μ(ωn(x)∑c∈ch(n)(Nnc(x)θnc)2−m) -μ ( _n(x)Σ _c (n) ( N_nc(x) _nc )^2-m ) Differentiating wrt θnc _nc we get, ∂ℒ(θn⋅,λ,μ)∂θnc=Nncθnc−λ+μ(ωnNncθnc2) ( _n·,λ,μ)∂ _nc= N_nc _nc-λ+μ ( _n N_nc _nc^2 ) equating this to 00 yields the quadratic equation λθnc2−Nncθnc−μωnNnc=0λ\ _nc^2-N_nc\ _nc-μ\ _nN_nc=0 whose solution obtained by the quadratic formula is θnc=Nnc+Nnc(x)2+4λμωnNnc2λ _nc= N_nc+ N_nc(x)^2+4λμ\ _nN_nc2λ ∎ Algorithm 1 Gated Local Trace EM for Probabilistic Circuits Algorithm 2 Gated Marginals EM for Probabilistic Circuits 1: Probabilistic circuit PC with sum-node parameters P=θnc(n,c)∈EP=\ _nc\_(n,c)∈ E; dataset D=x(i)i=1ND=\x^(i)\_i=1^N; regularization weight μ≥0μ≥ 0; simplex constraint weight λ>0λ>0; smoothing factor α∈(0,1]α∈(0,1]; monotone gate function g:ℝ≥0→[0,1]g:R_≥ 0→[0,1]; number of epochs E 2: Updated sum-node parameters P minimising the gated marginals regularised log-likelihood 3: Initialise: for each sum-node n, set θn⋅ _n· uniformly on its simplex 4: Set ωn←g(0) _n← g(0) for all sum-nodes n // gate weights; recomputed after first epoch 5: for epoch =1,…,E=1,…,E do // repeat until convergence or max epochs 6: E-step: Compute expected edge flows and node marginals 7: Run forward–backward passes on PC over D 8: Obtain edge flows Fnc(x)(n,c)∈E,x∈D \F_nc(x) \_(n,c)∈ E,\;x∈ D // O(|P||D|)O\! (|P|\,|D| ) time 9: Obtain node marginals pn(x),pc(x)n∈V,x∈D \p_n(x),\,p_c(x) \_n∈ V,\;x∈ D // available from the forward pass at no extra cost 10: Local Trace Computation (marginal ratio): 11: for all sum-nodes n do 12: Aggregate flows: F¯nc←∑x∈DFnc(x)\; F_nc← _x∈ DF_nc(x) 13: Compute dataset-averaged marginal ratio for each child c: r¯nc←1|D|∑x∈Dpc(x)pn(x) r_nc\;←\; 1|D| _x∈ D p_c(x)p_n(x) 14: Compute local trace via marginals: t^n←∑c∈ch(n)r¯nc 2=∑c∈ch(n)(1|D|∑x∈Dpc(x)pn(x))2 t_n\;←\; _c\,∈\,ch(n) r_nc^\;2\;=\; _c\,∈\,ch(n) ( 1|D| _x∈ D p_c(x)p_n(x) )^\!2 // depth-agnostic: no FnF_n factor; no θnc _nc dependence 15: end for 16: Gate Weight Update: 17: for all sum-nodes n do 18: ωn←g(t^n) _n← g\! ( t_n ) // e.g. g(t^n)=t^n/maxn′t^n′g( t_n)= t_n/ _n t_n (Proposed); 19: end for 20: M-step: Gated sharpness-aware parameter update 21: for all sum-nodes n do // updates are independent per node 22: for all child edges (n→c)(n\!→\!c) do 23: θ~nc←F¯nc+F¯nc 2+4λ(μωn)F¯nc2λ θ_nc\;←\; F_nc+ F_nc^\;2+4\,λ\,(μ\, _n)\, F_nc2\,λ // same closed form as global trace; effective strength μn=μωn _n=μ\, _n 24: end for 25: Normalise: θ~nc←θ~nc/∑c′∈ch(n)θ~nc′\; θ_nc← θ_nc / _c (n) θ_nc ∀c∈ch(n)∀\,c (n) // project onto probability simplex 26: end for 27: θ←(1−α)θ+αθ~θ←(1-α)\,θ+α\, θ // running-average smoothing for noisy mini-batch flows 28: end for 29: return P≡θP≡θ Appendix C Experimental Setup and Implementation Details In this section, we provide the details regarding the datasets, model architecture, hyperparameter used for the experimental results reported in the main paper. Table 3: Overview of the 2020 binary density estimation datasets, showing the number of variables and the number of instances in the training, validation, and test splits. Dataset Name #vars #train #valid #test nltcs 1616 1618116181 21572157 32363236 msnbc 1717 291326291326 3884338843 5826558265 kdd 6565 180092180092 1990719907 3495534955 plants 6969 1741217412 23212321 34823482 baudio 100100 1500015000 20002000 30003000 jester 100100 90009000 10001000 41164116 bnetflix 100100 1500015000 20002000 30003000 accidents 111111 1275812758 17001700 25512551 tretail 135135 2204122041 29382938 44084408 pumsb_star 163163 1226212262 16351635 24522452 dna 180180 16001600 400400 11861186 kosarek 190190 3337533375 44504450 66756675 msweb 294294 2944129441 32703270 50005000 tmovie 500500 45244524 10021002 591591 book 500500 87008700 11591159 17391739 cwebkb 839839 28032803 558558 838838 cr52 889889 65326532 10281028 15401540 c20ng 910910 1129311293 37643764 37643764 bbc 10581058 16701670 225225 330330 ad 15561556 24612461 327327 491491 Real World Data. We consider the standard suite of 2020 binary density estimation benchmark (33; 3). These include small to large domains such as nltcs (1616 variables), up to ad (15561556 variables). Table 3 summarizes the number of variables and data points in the train, validation, and test splits for each of the 2020 datasets. (a) Contextual usage F^n2 F_n^2 (b) Local curvature t^n t_n (c) Global contribution T^n T_n (d) T^n=F^n2t^n T_n= F_n^2\, t_n Figure 9: The curvature of a trained probabilistic circuit factorizes into usage and local sharpness. Each sum node contributes Tn=Fn2tnT_n=F_n^2\,t_n to the NLL Hessian trace. Panels (a)-(c) show the same circuit, with identical node coordinates, colored by (a) squared flow F^n2 F_n^2, (b) local trace t^n t_n, and (c) global contribution T^n T_n; panel (d) plots logF^n2 F_n^2 against logt^n t_n colored by logT^n T_n, with dashed iso-contribution lines of constant T^n T_n. The two factors are anticorrelated across the circuit: the locally sharpest nodes (dark in b) sit at low flow (light in a) and contribute little globally, while the high-usage root region drives the global trace despite modest local curvature. Nodes A, B, and C mark the three regimes: high-usage/low-local (A), low-usage/high-local (B), and high on both (C). Model Architectures • PyJuice (17), for our experiments on the binary density estimation datasets. We use the Hidden Chow-Liu Tree structure (18; 17) which is a generative probabilistic model that extends the classical Chow-Liu tree by introducing latent (hidden) variables to model complex dependencies among observed variables more effectively. The tree topology is learned from data using maximum‐likelihood (Chow-Liu algorithm). The observed variables are then pushed to the leaves, introducing latent variables to occupy the internal nodes, forming a latent tree structure. As a result, the learned structure can vary across datasets, adapting to the underlying statistical relationships. The latent size (num_latent) refers to the number of states each hidden variable can take, and it serves as a key hyperparameter that controls the model’s capacity. We set num_latent=100=100 for all our binary density estimation datasets. Appendix D Additional Results This section provides additional evidence for the empirical observations and method comparisons reported in the main paper. We first visualize the two components of the curvature decomposition and present a controlled synthetic example. We then report full-dataset results for underfitting, trace concentration, node-selection criteria, and performance across data regimes. Visualizing the Curvature Decomposition Figure 9 visualizes the empirical contextual usage, local curvature, and global trace contribution of the sum nodes in a trained circuit. In this example, several nodes with large local curvature receive little flow and therefore contribute weakly to the global trace. Conversely, nodes near the root receive substantial flow and can dominate the global contribution despite moderate local curvature. The labeled nodes illustrate the ranking reversal characterized in Proposition 2. Node A has the smallest local curvature among the three but the largest global contribution because it receives the most flow. Node B is locally sharper but contributes less globally because its contextual usage is small. The example illustrates why global contribution and local curvature provide different signals for allocating regularization. A 2D Synthetic Data Distribution We construct a two-dimensional example to illustrate how uniform and locally gated trace regularization can favor different regions of parameter space. The training set contains 5050 samples from a mixture of three isotropic Gaussians with standard deviation σ=0.25σ=0.25, centered at the vertices of an equilateral triangle. We additionally include eight noise samples distributed around a ring of radius 2.32.3, with radial jitter 0.120.12. The test set contains 800800 samples drawn only from the Gaussian mixture. We train a PC with Gaussian input distributions and 1616 sum and input units under three regimes: unregularized maximum likelihood, global trace regularization, and adaptive local-curvature gating. All methods use the same initialization, and both regularized methods use μ=0.20μ=0.20. Following 16, we project the optimization trajectories onto the plane spanned by the leading two principal components of the parameter iterates. We then evaluate the train NLL, test NLL, and exact Hessian trace over this plane. Figure 10 shows that the three methods converge to different regions. The unregularized model attains the lowest train NLL but higher test NLL and curvature. Global regularization reaches a flatter region but sacrifices both train and test fit. Adaptive gating retains lower curvature while reaching a region with better test fit than uniform global regularization. Figure 10: Vanilla, global, and adaptive regularization settle in distinct regions of parameter space. Vanilla reaches the lowest train NLL but a sharp, high-curvature region that generalizes poorly. Global regularization flattens curvature indiscriminately and underfits, while adaptive gating flattens only the non-generalizing sharpness, reaching a region with competitive test NLL at markedly lower sharpness. Underfitting at full data regime Figure 11 reports the training and test log-likelihood trajectories of the unregularized and global trace-regularized models across the DEBD benchmarks using the full training sets. On most datasets, global regularization lowers both training and test likelihood. The resulting degradation is therefore consistent with reduced model fit rather than a larger train–test gap. Figure 11: (Underfitting at full data regime)Training and test log-likelihood trajectories for the vanilla and global trace-regularized models DEBD using the full training data. Global trace regularization reduces the train-test gap but lowers both training and test likelihood, indicating underfitting rather than improved generalization. Sparsity of Global and Local Trace Figures 12 and 13 compare how global trace contribution and local curvature are distributed across sum nodes. The global contribution is highly concentrated across the evaluated datasets, with a small fraction of nodes accounting for most of the total trace. Local curvature is generally distributed across a broader set of nodes, indicating that contextual flow contributes to the concentration observed in the global trace. Figure 12: (Sparsity of Global trace) Cumulative share of the total trace as sum nodes are ranked by theirempirical contribution (T^n)( T_n). Across datasets, the distribution is highly concentrated: a small set of nodes dominates the global sharpness signal, while most nodes contribute negligibly. Figure 13: (Sparsity of local trace) Cumulative share of ∑nt^n _n t_n as nodes are ranked by t^n t_n. On the evaluated circuits, the local trace concentration is less severe than the global contribution, indicating that part of the concentration in T^n T_n is introduced by contextual usage. Thresholding based node selection results We next compare two criteria for selecting where trace regularization acts. In Figure 14, nodes are ranked by global contribution. Restricting the penalty to progressively smaller sets of high-contribution nodes generally fails to recover the unregularized likelihood and often degrades it further. Figure 15 instead ranks nodes by local curvature. Relative to global-contribution selection, local-curvature selection more consistently preserves the unregularized fit across the evaluated datasets. Figure 14: (Thresholding based node selection results) Change in test log-likelihood relative to the unregularized model when trace regularization is applied to all sum nodes or restricted to progressively smaller fractions ranked by T^n T_n. Selecting fewer top-ranked nodes generally worsens performance, showing that global contribution is a poor criterion for allocating regularization. Figure 15: (Thresholding based node selection results) Change in test log-likelihood relative to the unregularized model when trace regularization is applied to all sum nodes or restricted to progressively smaller fractions ranked by t^n t_n. Selecting fewer top-ranked nodes to regularize generally improves the performance, showing that t^n t_n is a better criterion for allocating regularization. Low Data Regime Experiments Table 4: Test log-likelihood comparison across data fractions for DEBD. For gated methods, the variant is selected per dataset/fraction by highest mean validation L. Dataset 25% Data 50% Data 100% Data Vanilla Global Trace Best Gated Vanilla Global Trace Best Gated Vanilla Global Trace Best Gated accidents -28.59 -29.83 -28.64 -27.13 -29.84 -27.12 -26.63 -29.69 -26.64 ad -28.55 -29.64 -28.86 -20.51 -22.07 -20.47 -18.30 -20.37 -18.22 baudio -41.13 -42.46 -41.16 -39.99 -42.40 -40.01 -39.63 -42.45 -39.63 bbc -379.30 -281.33 -377.54 -312.26 -262.64 -309.58 -269.69 -256.43 -269.39 bnetflix -59.06 -59.87 -59.24 -56.85 -59.83 -56.95 -56.22 -59.67 -56.21 book -37.91 -37.48 -37.45 -35.44 -36.75 -35.29 -34.37 -36.31 -34.23 c20ng -162.01 -160.21 -161.97 -154.92 -158.67 -154.94 -151.97 -157.60 -152.04 cr52 -107.98 -103.75 -106.04 -102.79 -102.07 -101.00 -99.35 -102.15 -100.26 cwebkb -204.80 -164.74 -202.88 -171.74 -159.32 -170.11 -154.78 -157.34 -154.89 dna -93.73 -82.61 -93.44 -90.35 -81.76 -90.07 -87.70 -81.35 -87.11 jester -57.38 -55.65 -57.43 -54.05 -55.90 -54.26 -52.82 -55.81 -52.88 kdd -2.22 -2.37 -2.21 -2.22 -2.36 -2.21 -2.23 -2.37 -2.22 kosarek -10.83 -11.11 -10.79 -10.64 -11.06 -10.67 -10.59 -11.09 -10.59 msnbc -6.10 -6.46 -6.08 -6.11 -6.47 -6.08 -6.12 -6.46 -6.08 msweb -10.00 -10.45 -9.97 -9.82 -10.39 -9.81 -9.72 -10.40 -9.74 nltcs -6.09 -6.50 -6.10 -6.02 -6.47 -6.02 -6.00 -6.48 -6.00 plants -12.97 -15.90 -12.96 -12.77 -15.97 -12.77 -12.67 -15.98 -12.66 pumsb_star -23.82 -28.75 -23.65 -22.79 -28.35 -22.77 -22.79 -28.35 -22.78 tmovie -51.23 -50.51 -50.89 -44.55 -49.57 -44.58 -42.17 -48.84 -42.11 tretail -11.31 -11.03 -11.27 -10.96 -11.00 -10.98 -10.84 -10.98 -10.85 Table 4 compares the unregularized model, global trace regularization, and the best gated variant across the evaluated data fractions. For each dataset and fraction, the gated variant is selected using validation log-likelihood. The results show that global regularization can improve performance for some datasets, particularly when data are limited, but frequently degrades likelihood as the available data increase. The gated variants more consistently preserve the fit while retaining gains on datasets where sharpness regularization remains beneficial.