Paper deep dive
Learning a Mixture of GFlowNets
Tiago da Silva, Amauri H. Souza, Salem Lahlou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/13/2026, 3:39:31 AM
Summary
The paper proposes a theoretical framework for learning a mixture of GFlowNets, introducing Continuously Indexed (CI) and Discretely Indexed (DI) variants. CI GFlowNets utilize random features to boost expressivity and stability in graph-structured tasks. DI GFlowNets encompass prior methods and lead to the novel Stratum-Conditioned (SC) GFlowNets, which decompose the state space using modular functions. SC GFlowNets enable parallel training, accelerate convergence, and improve mode coverage without significant computational overhead.
Entities (10)
Relation Signals (9)
SC GFlowNet → improves → learning convergence
confidence 95% · significantly speed up learning convergence and mode coverage
SC GFlowNet → isa → DI GFlowNet
confidence 95% · DI GFlowNets ... provide the foundation for the newly proposed Stratum-Conditioned (SC) GFlowNets
SC GFlowNet → supports → parallel training
confidence 95% · SC GFlowNets support centralized and component-wise embarrassingly parallel training
CI GFlowNet → uses → Random Features
confidence 95% · CI GFlowNets can be interpreted through the lens of a random features expansion
GFlowNet → uses → Trajectory Balance
confidence 93% · a popular choice being the trajectory balance (Malkin et al., 2022, TB)
SC GFlowNet → uses → Doob's h-transform
confidence 92% · This method, which is inspired by the Doob's h-transform of Markov chains
Mixture of GFlowNets → generalizes → CI GFlowNet
confidence 90% · we specialize into continuously (CI) and discretely indexed (DI) collections
Mixture of GFlowNets → generalizes → DI GFlowNet
confidence 90% · we specialize into continuously (CI) and discretely indexed (DI) collections
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Learning an ensemble of GFlowNets to sample from a discrete target distribution has become a common approach for achieving better state space exploration and convergence than that of a monolithic sampler. However, these methods often add a substantial runtime overhead to the base model, and their conceptual connection remains elusive. To address this, we first propose a general-purpose theoretical framework for describing a mixture of GFlowNets, which we specialize into continuously (CI) and discretely indexed (DI) collections. On the one hand, we show CI GFlowNets can be interpreted through the lens of a random features expansion, provably boosting the sampler's expressivity in graph-structured tasks and reducing learning instability via spectral shifting. On the other hand, we demonstrate DI GFlowNets encompass prior approaches for GFlowNet training and provide the foundation for the newly proposed Stratum-Conditioned (SC) GFlowNets. This method, which is inspired by the Doob's h-transform of Markov chains, decomposes the state space according to a prescribed modular function and restricts each component to sample from a distinct subset of it. Importantly, SC GFlowNets support centralized and component-wise embarrassingly parallel training, and we show both of them significantly speed up learning convergence and mode coverage without introducing any non-negligible extra computation.
Tags
Links
- Source: https://arxiv.org/abs/2610.07562v1
- Canonical: https://arxiv.org/abs/2610.07562v1
Trouble viewing inline? Open PDF directly →
Full Text
180,200 characters extracted from source content.
Expand or collapse full text
Learning a Mixture of GFlowNets Tiago da Silva Affiliation: MBZUAI, UAE Email: tiago.dasilva@mbzuai.ac.ae Amauri H. Souza Affiliation: Avra, Brazil & Federal Institute of Ceará, Brazil Email: amauri.holanda@avra.ai Salem Lahlou Affiliation: MBZUAI, UAE Email: salem.lahlou@mbzuai.ac.ae Abstract Learning an ensemble of GFlowNets to sample from a discrete target distribution has become a common approach for achieving better state space exploration and convergence than that of a monolithic sampler. However, these methods often add a substantial runtime overhead to the base model, and their conceptual connection remains elusive. To address this, we first propose a general-purpose theoretical framework for describing a mixture of GFlowNets, which we specialize into continuously (CI) and discretely indexed (DI) collections. On the one hand, we show CI GFlowNets can be interpreted through the lens of a random features expansion, provably boosting the sampler’s expressivity in graph-structured tasks and reducing learning instability via spectral shifting. On the other hand, we demonstrate DI GFlowNets encompass prior approaches for GFlowNet training and provide the foundation for the newly proposed Stratum-Conditioned (SC) GFlowNets. This method, which is inspired by the Doob’s h-transform of Markov chains, decomposes the state space according to a prescribed modular function and restricts each component to sample from a distinct subset of it. Importantly, SC GFlowNets support centralized and component-wise embarrassingly parallel training, and we show both of them significantly speed up learning convergence and mode coverage without introducing any non-negligible extra computation. 1 Introduction Suppose we are given an unnormalized distribution R over a discrete space X. A common problem in both statistical inference and probabilistic machine learning is to draw samples from X distributed according to R (Bishop, 2007; Neal and others, 2011; Blei and et al., 2017). When X is intractably large—e.g., the space of graphs containing N nodes or of subsets of a set with N elements for largish N—addressing this challenge often requires the use of approximate algorithms such as Markov chain Monte Carlo (MCMC) or variational inference (VI). Notably, MCMC suffers from slow convergence and correlated samples, especially in discrete spaces, and its correctness cannot be tractably assessed in general (Owen, 2013). VI, in contrast, reframes the sampling problem as that of finding the best approximation to R within a tractable family, thereafter enabling the generation of independent samples from a measurably correct distribution (Malkin et al., 2023; Zimmermann et al., 2023). As a major milestone of VI for discrete spaces, GFlowNets (Bengio et al., 2021; Lahlou et al., 2023) learn a sequential decision process that iteratively constructs each x∈x from an initial (empty) state in proportion to R(x)R(x). Since their conception, they have been successfully deployed into applications ranging from causal discovery (Deleu et al., 2022; Deleu et al., 2023) to LLM fine-tuning (Hu et al., 2023; Venkatraman et al., 2024) and phylogenetic inference (Zhou et al., 2024). Despite their potential, they suffer from insufficient state space exploration (Kim et al., 2025b; Madan et al., 2025), slow training convergence (Liu et al., 2024), and learning instability (Figure 10), often failing to simulate even simple multi-modal distributions defined over relatively small X (Figure 20). Figure 1: An ensemble of GFlowNets that, when averaged, matches the target. To mitigate this issue, recent works have proposed learning an ensemble of GFlowNets in which each component focuses on a nearly-unique subset of the state space (Yu et al., 2026a; Yu et al., 2026b; Dall’Antonia et al., 2026b); in doing so, they incentivize the exploration of unvisited high-probability regions and accelerate learning convergence. However, they often incur a substantial computational overhead—requiring several neural networks to be separately trained—and, despite being similar, lack a shared conceptual foundation. In this scenario, our first contribution is to introduce a unifying theory to describe a mixture of GFlowNets (Figure 1), shedding light on the applicability and limitations of such an approach. Drawing on this framework, we propose distinct algorithms that we show to both stabilize and accelerate learning convergence. In this context, we separate our presentation into continuously (CI) and discretely indexed (DI) mixtures of GFlowNets. On the one hand, CI GFlowNets learn an uncountable collection of samplers, each of which is associated to a real-valued vector whose value serves as an extra input to the underlying neural network parameterizing the sampler. We demonstrate this formulation is connected to the theory of random features for graph learning (Sato et al., 2021; Abboud et al., 2021), providing a computationally simple approach to provably boost the sampler’s expressivity in graph-structured tasks (Sections 4 and C). In addition, we show CI GFlowNets stabilize learning by shifting the spectrum of the loss function’s Hessian, reducing the dependence of the learned model on exogenous factors such as the initial value of the pseudorandom number generator (Proposition C.7). On the other hand, DI GFlowNets consist of a finite (or countably infinite) mixture of samplers. Each sampler learns in proportion to a re-weighted target; for the kkth component, this is x↦R(x)q(k|x)x R(x)q(k|x) for a posterior mixing distribution q(⋅|x)q(·|x) (see Section 5). In this sense, DI GFlowNets support two learning modalities: centralized, with an embedding of k being used as extra input to a component-wise shared model, and embarrassingly parallel (Neiswanger et al., 2014), with distinct components being independently trained across non-communicating processes. We prove this framework encompasses two prior methods, Subgraph Asynchronous Learning (Silva et al., 2025b) and Boosted GFlowNets (Dall’Antonia et al., 2026b), unifying otherwise disparate algorithms. As the simplest non-trivial instantiation of DI GFlowNets, we propose setting q(⋅|x)q(·|x) as a sparse distribution assigning zero probability to all but a few components. This effectively corresponds to defining a covering of the state space. We show, however, that conditioning a sampler to exclusively explore an arbitrary subset of X is NP-complete in general. We consequently introduce Stratum-Conditioning (SC) as a tractable approach for learning DI GFlowNets. Simply put, SC partitions the state space by binning the outputs of a modular and real-valued function defined on X, with each bin being called a stratum (hence the name). This function can be set to the trajectory length required for reaching x∈x , for instance, or as the distance between x∈x and a chosen reference point according to a prescribed metric (Section 5.1). We demonstrate the set of samplers learned in this way are connected via Doob’s h-transform of a base GFlowNet (Section 5.2). Our experiments highlight SC GFlowNets substantially improve both learning convergence and state space exploration across diverse benchmark tasks. Importantly, contrarily to recent algorithmic advances in the GFlowNet literature, SC adds no computational overhead to training, being as fast to train as a single-component sampler (see Figure 4 and Figures 11, 12 and 13). As with most of machine learning research, which historically started from simple algorithms that have been incrementally enhanced over time, we believe SC GFlowNets set a minimal baseline for learning a mixture of GFlowNets. In summary, our contributions are as follows. 1. We introduce an abstract framework for learning an ensemble of GFlowNets, demonstrating its usefulness in providing an unifying view of prior techniques and in designing novel algorithms. 2. We define continuously (CI) and discretely indexed (DI) GFlowNets, outlining their empirical and theoretical advantages over a single-component sampler. 3. We propose Stratum-Conditioned (SC) GFlowNets as an effective form of learning DI GFlowNets, supporting both centralized and parallel training. 4. We show that SC GFlowNets speed up convergence and mode discovery across benchmark tasks. 2 Preliminaries Definitions. We let X and S be finite spaces, with S corresponding to partial realizations of X. We also let so∈s_o be the initial state from which any x∈x can be constructed. An example the reader should keep in mind is when X is the set of subsets of size K of a given finite set and S is the set of subsets of size less than K, with so=∅s_o= . We call a state graph a directed acyclic graph having sos_o as the only state without incoming edges and x∈x as the only states without outgoing edges, which we call terminal, and such that there is a path from sos_o to every x∈x containing only states from S. Notationally, we let [N]=1,…,N[N]=\1,…,N\ for each positive integer N∈ℕN and s↝s′s s be the set of trajectories τ=(s,s1),(s1,s2),…,(sd,s′)τ=\(s,s_1),(s_1,s_2),…,(s_d,s )\ from s∈s to s′∈∪s . GFlowNets. A GFlowNet learns to stochastically construct x∈x from sos_o by following a policy pF:×(∪)→[0,1]p_F ×(S )→[0,1] such that pF(s,s′)>0p_F(s,s )>0 if and only if s→s′s→ s in the state graph. The objective is for the marginal distribution of pFp_F over X to match the unnormalized distribution R:→ℝ+R _+ from which we aim to sample, i.e., mF(x)≔∑τ∈so↝x∏(si,si+1)∈τpF(si,si+1)≔∑τ∈so↝xpF(so,τ)∝R(x).m_F(x) _τ∈ s_o x _(s_i,s_i+1)∈τp_F(s_i,s_i+1) _τ∈ s_o xp_F(s_o,τ) R(x). (1) As mF(x)m_F(x) is generally intractable, we introduce a backward policy pB(x,τ)p_B(x,τ) representing the probability of having followed τ given that x was sampled. We then search for (pF,pB)(p_F,p_B) satisfying a consistency condition, with a popular choice being the trajectory balance (Malkin et al., 2022, TB): Z⋅pF(so,τ)=R(x)⋅pB(x,τ)Z· p_F(s_o,τ)=R(x)· p_B(x,τ) and Z as a learned constant. In practice, pF(s,⋅)p_F(s,·) is parameterized as a neural network and this search is carried out via stochastic gradient descent. Please refer to Appendices A and I for a comprehensive background and recent advances on GFlowNets. 3 Learning a Mixture of GFlowNets Simply put, the central challenges in achieving Equation 1 are the problem’s under-determination, owning to the existence (in general) of infinitely many pairs (pF,pB)(p_F,p_B) abiding by any given consistency condition, and sample inefficiency, requiring extensive state graph exploration for finding a resolutive (pF,pB)(p_F,p_B). We will show a mixture formulation can naturally address both of them, focusing on the latter in the main text and elaborating on the former in Appendix C of the supplement. Towards this objective, we develop an abstract framework for learning a mixture of GFlowNets. For starters, we let Γ be an index space and pF(γ),pB(γ)γ∈Γ\p_F [t](γ),p_B [t](γ)\_γ∈ be the corresponding set of policies. Our goal is for the marginal distribution of these policies on each x∈x , when weightedly averaged over γ∈Γγ∈ , to match the target R(x)R(x). Formally, we assume Γ to be embedded in a topological measurable space equipped with a measure ν over its Borel σ-algebra. This allows us to set aside the specific properties of Γ , while accounting for common use cases via a unified language. Remark 3.1 (Γ can be either discrete or continuous). Common choices for Γ and ν are Γ=ℝdγ =R^d_γ with ν as the Lebesgue measure and Γ=[K] =[K] with ν as the counting measure (Appendices C and 5). Designing a mixture of GFlowNets. With this setup, we let pMp_M be a probability measure on Γ absolutely continuous with respect to ν. The subscript M indicates pMp_M is a mixing (prior) distribution over Γ . Our goal is for the marginal identity analogous to Equation 1 ∫ΓpM(γ)∑τ∈so↝xpF(γ)(so,τ)≔∫ΓpM(γ)mF(γ)(x)∝R(x) _ p_M(dγ) _τ∈ s_o xp_F [t](γ)(s_o,τ) _ p_M(dγ)m_F [t](γ)(x) R(x) (2) to be satisfied. As in Equation 1, we call mF(γ)(x)m_F [t](γ)(x) the marginal of pF(γ)(so,⋅)p_F [t](γ)(s_o,·) over X. Under this condition, we can sample from R by first drawing γ∼pM(dγ)γ p_M(dγ) and then sampling x∼mF(γ)(x)x m_F [t](γ)(x). Nevertheless, Equation 2 is doubly intractable: neither mF(γ)m_F [t](γ) nor its average under γ∼pMγ p_M can be computed efficiently. We circumvent this issue by introducing a mixing (posterior) distribution qM(dγ|x)q_M(dγ|x) and rewriting Equation 2 as11 1 Recall f(γ)μ(dγ)=g(γ)ν(dγ)f(γ)μ(dγ)=g(γ)ν(dγ) means ∫Bf(γ)μ(γ)=∫Bg(γ)ν(γ) _Bf(γ)μ(dγ)= _Bg(γ)ν(dγ) for any measurable set B. pM(dγ)mF(γ)(x)∝R(x)qM(dγ|x).p_M(dγ)m_F [t](γ)(x) R(x)q_M(dγ|x). (3) The role of qMq_M in the equation above is akin to that of responsibilities in expectation-maximization algorithms (Dempster et al., 1977), representing the probability a given x∈x was drawn from a γ-indexed sampler. As in Section 2, we turn Equation 3 into a learning objective—referred to as mixed trajectory balance (MTB)—by replacing the intractable marginal with tractable policies. Definition 3.2 (MTB condition). Let γ=(pF(γ),pB(γ)) g_γ= (p_F [t](γ),p_B [t](γ) ) and γ∈Γ\ g_γ\_γ∈ be a Γ -indexed set of GFlowNets, and pM(dγ)p_M(dγ) and qM(dγ|x)q_M(dγ|x) be prior and posterior mixing distributions over Γ . Then, pM(dγ)pF(γ)(so,τ)=R(x)ZqM(dγ|x)pB(γ)(x,τ)p_M(dγ)p_F [t](γ)(s_o,τ)= R(x)Zq_M(dγ|x)p_B [t](γ)(x,τ) for each x∈x and τ∈so↝xτ∈ s_o x is the mixed trajectory balance condition, with Z=∑x∈R(x)Z= _x R(x). The sufficiency of Definition 3.2 for Equation 2 is a direct consequence of (Malkin et al., 2022, Proposition 1). As it will soon become clear, it will be notationally convenient to define Z(dγ)=Z⋅pM(dγ) and R(x,dγ)=R(x)qM(dγ|x)Z(dγ)=Z· p_M(dγ) and R(x,dγ)=R(x)q_M(dγ|x) as unnormalized measures over Γ and ×ΓX× . This allows us to rewrite MTB as Z(dγ)⋅pF(γ)(so,τ)=R(x,dγ)⋅pB(γ)(x,τ),Z(dγ)· p_F [t](γ)(s_o,τ)=R(x,dγ)· p_B [t](γ)(x,τ), resembling the familiar TB condition. We also deduce from this that other balance conditions, such as Detailed Balance (DB) (Bengio et al., 2023) and SubTB (Madan et al., 2022), remain valid in our setting. Given a function F:s∈↦F(s,dγ)F s F(s,dγ) mapping s to a measure in Γ , for instance, the DB condition can be written as, for s∈s , s′∈∪s , and x∈x F(s,dγ)⋅pF(γ)(s,s′)=F(s′,dγ)pB(γ)(s′,s) and F(x,dγ)=R(x,dγ).F(s,dγ)· p_F [t](γ)(s,s )=F(s ,dγ)p_B [t](γ)(s ,s) and F(x,dγ)=R(x,dγ). This raises the question of how to select both Z(dγ)Z(dγ) and R(x,dγ)R(x,dγ), which will be addressed in Section 5. Under our abstract setting, however, we show both measures are intimately related. Proposition 3.3 (Relationship between Z(dγ)Z(dγ) and R(x,dγ)R(x,dγ)). Let γ∈Γ\ g_γ\_γ∈ and pMp_M and qMq_M be as in Definition 3.2, and Z(dγ)Z(dγ) and R(x,dγ)R(x,dγ) be as above. Then, Z(dγ)Z(dγ) and R(x,dγ)R(x,dγ) jointly satisfy Z(dγ)=∑x∈R(x,dγ) and R(x,dγ)=Z(dγ)⋅mF(γ)(x).Z(dγ)= _x R(x,dγ) and R(x,dγ)=Z(dγ)· m_F [t](γ)(x). Under Proposition 3.3, Z(dγ)Z(dγ) may be understood as the fraction of total probability mass (Z) being assigned to the model indexed by γ, while x↦R(x,dγ)x R(x,dγ) represents the (unnormalized) marginal distribution of such model over X. This supports the interpretation of Γ -indexing as a soft partition of X: each γ g_γ primarily focuses on a (potentially) distinct subset of X. Learning a GFlowNet ensemble. We learn a MTB-abiding Γ=(pM,qM,γ∈Γ) g_ \!=\!(p_M,q_M,\ g_γ\_γ∈ ) by optimizing ℒTB(τ,dγ,Γ)=(logZ(dγ)⋅pF(γ)(so,τ)R(x,dγ)⋅pB(γ)(x,τ))2L_TB(τ,dγ; g_ )= ( Z(dγ)· p_F [t](γ)(s_o,τ)R(x,dγ)· p_B [t](γ)(x,τ) )^2 (4) for all τ,γτ,γ; we call ℒTBL_TB the TB loss function (Malkin et al., 2022). As an aside, we notice in Definition B.2 that similar learning objectives from Bengio et al. (2023); Deleu et al. (2024), such as SubTB and DB, can also be seamlessly extended to our setting. From this, learning continues business as usual: we parameterize pF(γ)(s,⋅)p_F [t](γ)(s,·) and (optionally) pB(γ)(s,⋅)p_B [t](γ)(s,·) as neural networks receiving both s and γ as inputs and, defining exploratory (full support) distributions ρM(dγ) _M(dγ) and ρ(γ)(so,τ)ρ [t](γ)(s_o,τ), we solve the stochastic program minΓB∼ρM(⋅),τ∼ρ(γ)(so,⋅)[ℒTB(τ,B,Γ)] _ g_ E_B _M(·),τ ρ [t](γ)(s_o,·) [L_TB (τ,B; g_ ) ] (5) via stochastic gradient descent (Jordan et al., 2024). In the following sections, we present both specific instantiations and an empirical validation of the proposed framework. Remark 3.4 (Modularity). We also notice methods for learning both pFp_F and ρ in Equation 5 can be directly applied to our setting (Kim et al., 2025b; Madan et al., 2025; Dall’Antonia et al., 2026a). 4 Continuously Indexed GFlowNets The most straightforward form of Equation 5 assumes q(dγ|x)=pM(dγ)q(dγ|x)=p_M(dγ), i.e., each component is equally likely to generate x∈x . In this case, the ratio Z(dγ)/R(x,dγ)=Z/R(x) Z(dγ)R(x,dγ)= ZR(x) becomes γ-independent, and our problem reduces to minimizing the familiar TB loss averaged over γ∈Γγ∈ (Malkin et al., 2022). Although deceptively simple, this approach corresponds to augmenting our model with random features (RFs) when Γ=ℝdγ =R^d_γ. This is a well-established procedure for boosting the expressivity of graph neural networks (Abboud et al., 2021), often-used by GFlowNets in graph-structured spaces. Drawing on this, we show continuously indexed (CI) GFlowNets—as we call the RF-augmented sampler—can arbitrarily approximate any graph distribution. Remark 4.1 (CI GFlowNets as Universal Approximators). CI GFlowNets are strictly more expressive, in terms of which distributions they can realize, than GFlowNets (see Example C.3). In particular, when considering graph-structured tasks based on edge-additive generative processes, CI GFlowNets are universal approximators (Proposition C.5). We formalize this discussion in Appendix C, where we also demonstrate CI GFlowNets can have a smoothing effect on the loss surface (Bishop, 1995, as in), significantly reducing training instability. 5 Discretely Indexed GFlowNets To start our discussion on discrete GFlowNet mixtures, we first notice that the TB loss function in Equation 4 does not impose a preference of a non-trivial solution over a collapsed sampler in which Z(dγ)=Z(dγ)=Z and R(x,dγ)=R(x)R(x,dγ)=R(x) or, more generally, in which the ratio R(x,dγ)/Z(dγ) R(x,dγ)Z(dγ) does not depend on γ. Although a γ-independent target can exhibit beneficial properties, such as greater expressivity and a smoother loss surface (as in Section 4), this should be a design decision—achieved by setting qM(dγ|x)=pM(dγ)q_M(dγ|x)=p_M(dγ)—rather than a consequence of training. In practice, learning a non-collapsed mixture from Equation 5 requires constraining the search space for the mixing distributions qMq_M and pMp_M either via soft regularization or hard constraints. We advocate for the latter in this section, suggesting to fix Γ as a finite set and qM(dγ|x)q_M(dγ|x) as a sparse distribution assigning zero probability to most γ∈Γγ∈ . We also briefly elaborate on using a countably infinite Γ when discussing Boosted GFlowNets (Dall’Antonia et al., 2026b) in Appendix F. Discretely Indexed GFlowNets. We refer to the corresponding sampler as Discretely Indexed (DI) GFlowNets. As notational shortcuts, we let Γ=[K] =[K], and write the measures Z and R and qMq_M as Z(k)=ZkZ(\k\)=Z_k and R(x,k)=Rk(x)R(x,\k\)=R_k(x) and qM(k|x)=qk(x)q_M(\k\|x)=q_k(x), respectively. In addition, we let =(Zk)k∈[K]Z=(Z_k)_k∈[K] and ρk _k be the exploration policy for the mixture’s kkth component. Our objectives are to define qkq_k and ρk _k; as with single-component GFlowNets, we set Z as a learnable parameter. 5.1 Stratum-Conditioned GFlowNets In this context, we propose setting qk(x)q_k(x) according to a hand-crafted covering Xkk∈[K]\X_k\_k∈[K] of X: under this condition, qk(x)q_k(x) corresponds to a uniform distribution over the indices k for which kX_k contains x. We call each kX_k a stratum due to its construction as the pre-image of an interval with respect to a real-valued function on X; see Remark 5.5. This section formalizes the ensuing algorithm, called Stratum-Conditioned (SC) GFlowNets, explains how it can be implemented, and describes sufficient conditions for which sampling from kX_k is computationally tractable. Stratum Conditioning. The first requirement for tractability is for us to know how many sets N(x)N(x) in kk∈[K]\X_k\_k∈[K] contain a given x, allowing for the efficient computation of qk(x)q_k(x). As such, we define a tractable K-cover of X, denoted as TC(,K)TC(X,K). Definition 5.1 (Tractable K-Cover). Let K be a positive integer and X be a set. We say TC(,K)=kk=1KTC(X,K)=\X_k\_k=1^K is a Tractable K-Cover of X if the following conditions are satisfied. 1. TC(,K)TC(X,K) is an exact cover of X, i.e., each k⊆X_k and ⋃k=1Kk= _k=1^KX_k=X. 2. For each x∈x , the quantity N(x)≔#k∈[K]:x∈kN(x) \#\k∈[K] x _k\ is known22 2 An intractable cover, in contrast, could be built upon an expensive multiclass classifier f:→[K]f →[K].. The second condition allows efficient querying of qk(x)q_k(x), defined as qk(x)=N(x)−1q_k(x)=N(x)^-1 if x∈kx _k and qk(x)=0q_k(x)=0 otherwise. In practice, we often let TC(,K)TC(X,K) be a disjoint cover—a partition—of X; in this case, qk(x)=1q_k(x)=1 for the only k with x∈kx _k. A corollary of Proposition 3.3 cleanly characterizes both ZkZ_k and Rk(x)R_k(x) as a fraction of the probability masses assigned to kX_k and x under a TC. Corollary 5.2 (ZkZ_k as the mass within kX_k). Let TC(,K)=kk=1KTC(X,K)=\X_k\_k=1^K and [K] g_[K] be a GFlowNet ensemble satisfying the MTB condition in Definition 3.2. Then, denoting by [⋅] 1[·] the indicator function, Zk=∑x∈kR(x)N(x) and Rk(x)=R(x)N(x)⋅[x∈k].Z_k= _x _k R(x)N(x) and R_k(x)= R(x)N(x)· 1[x _k]. In particular, when N(x)N(x) is constant, the learned policy pF(k)p_F [t](k) generates samples in proportion to the restriction of R(x)R(x) to kX_k, a property that might be of independent interest when the covering has a meaningful semantic value for the problem at hand. Remark 5.3 (Rk(x)R_k(x) for a disjoint cover). Under the conditions of Corollary 5.2, assume N(x)=N(x)=N (e.g., N=1N=1 for a disjoint cover). Then, mF(k)(x)∝R(x)⋅[x∈k]m_F [t](k)(x) R(x)· 1[x _k]. We emphasize pF(k)k∈[K]\p_F [t](k)\_k∈[K] are parameterizable via a shared neural network architecture receiving that receives an embedding of k as input; they need not be completely separate samplers. To understand the benefits of partitioning, we consider the Lines domain (Silva et al., 2026), described next. Figure 2: SC GFlowNets for the Lines environment (N=256N=256). (Left) We show the TV distance between the learned and target distributions, emphasizing the faster convergence promoted by stratum-conditioning. (Middle) The marginal distribution of both models after 1000 training steps, and the target. (Right) The marginal distribution of SC GFlowNets for each stratum k∈1,2,3k∈\1,2,3\. Example 5.4 (Lines domain). We define =1,…,NS=\1,…,N\ and =1,…,N×⊤X=\1,…,N\×\ \ with N≥2N≥ 2 and ⊤ indicating the state is in X and not in S. The initial state is so=1s_o=1 and each transition corresponds to adding 11 to the current state if it is less than N or attaching ⊤ and terminating. Let 1=1,…,⌊N/3⌋,2=⌊N/3⌋+1,…,⌊2⋅N/3⌋, and 3=⌊2⋅N/3⌋+1,…,NX_1=\1,…, N3 \,\,X_2=\ N3 +1,…, 2· N3 \, and X_3=\ 2· N3 +1,…,N\ be the partition. We design pF(k)(s,⋅)p_F [t](k)(s,·) by masking out the terminating transition if s∉ks _k and masking out the successor transition if s+1∉ks+1 _k, i.e., pF(k)(s,s+1)∝[s+1∈k]⋅eϕ(s,k)⊤w1 and pF(k)(s,(s,⊤))∝[s∈k]⋅eϕ(s,k)⊤w2,p_F [t](k)(s,s+1) 1[s+1 _k]· e^φ(s,k) w_1 and p_F [t](k)(s,(s, )) 1[s _k]· e^φ(s,k) w_2, with ϕ:[N]×[K]→ℝdφ [N]×[K] ^d and w1,w2∈ℝdw_1,w_2 ^d being learnable parameters. Figure 2 shows the error curves measured in terms of total variation distance (TV) for (i) a GFlowNet and (i) a SC GFlowNet for the target distribution R(x)=exp−(βN)−1⋅miny∈3,N−2|x−y|R(x)= \-(β N)^-1· _y∈\3,N-2\|x-y|\ with N=256N=256 and β=0.1β=0.1. Notably, while SC GFlowNet quickly converges to the target distribution, a GFlowNet struggles to achieve a comparable goodness-of-fit, requiring over three times more training steps. How to design a tractable covering? A pivotal question for SC GFlowNets is how to define the mapping TC(,K)TC(X,K) for each X and K. Although distinct problems may benefit from idiosyncratic strategies, we provide next a blueprint for designing TC(,K)TC(X,K) in commonly encountered tasks. Remark 5.5 (Modulo-based TCTC). Example 5.4 hints on how to construct TCTC: we let kX_k be the set of states whose distance to a reference point is within a prescribed interval. As most state spaces of interest lack an inherent geometric structure, however, we use a modular function as a general proxy for distance. To make sense of such definition, recall a compositional object can be described as a multisubset of a component set C, i.e., each s∈∪s can be identified with s⊆s . In Example 5.4, for instance, =1,⊤C=\1, \ and s≡1,…,1s≡\\!\1,…,1\\!\ (s times). In this scenario, let ψ be a bounded modular function with ψ(c)≥0ψ(\c\)≥ 0 for all c∈c . We let k=x∈:k−1K(ψ()+1)≤ψ(x)<kK(ψ()+1).X_k= \x k-1K(ψ(C)+1)≤ψ(x)< kK(ψ(C)+1) \. When ψ(s)=#sψ(s)=\#s, we call TC(,K)TC(X,K) to be a length-based covering. In general, we let ψ(s)=#(s∩r)ψ(s)=\#(s∩ r) for a given reference set r⊆r ; if r=∅r= , the SC GFlowNet reduces to a GFlowNet. Importantly, while easily solved for some ψ such as the length-based one in Example 5.4, we show next that the decision problem of inferring whether kX_k is reachable from a state in S is NP-complete in general. This is required for masking out from pF(k)p_F [t](k) transitions leading to states for which kX_k becomes unreachable, ensuring the sampler does not become deadlocked in a zero-probability region. Proposition 5.6 (Sampling from a general TC is NP-complete). Let TC(,K)TC(X,K) be a modulo-based TC built upon a modular function ψ and k=x∈:ψ(x)∈[ak,bk)X_k=\x ψ(x)∈[a_k,b_k)\ for numbers (ak,bk)k=1K\(a_k,b_k)\_k=1^K. Let s∈⊂2s ⊂ 2^C. Given k∈[K]k∈[K], we consider the following decision problem, ∃s′∈2∖s s.t. s∪s′∈ and ψ(s∪s′)∈[ak,bk),∃ s ∈ 2^C s s.t. s∪ s and ψ(s∪ s )∈[a_k,b_k), (6) i.e., whether there is a terminal x∈kx _k reachable from s∈s . Then, if ψ is an arbitrary modular function evaluable in polynomial time on the size of its input, Equation 6 is NP-complete. We demonstrate Proposition 5.6 by reducing Equation 6 to the SubsetSum problem, which is NP-complete (Cormen et al., 2009). Rather than a strict limitation of SC GFlowNets, however, Proposition 5.6 should be read as a guiding design principle for TC(,K)TC(X,K), suggesting there are partitions for which sampling cannot be efficiently implemented. In contrast, drawing on Example 5.4, we show there is a specific class of functions ψ for which Equation 6 is decidable in polynomial time. Proposition 5.7 (Efficient sampling from symmetric junta-based TC). We call a function ψ:2→ℝψ 2^C a symmetric junta if there are a reference set r⊆2r 2^C and a function ξ:ℤ→ℝξ for which ψ(x)=ξ(#(x∩r))ψ(x)=ξ(\#(x∩ r)) for all x⊆x . We assume ψ can be evaluated in polynomial time. Then, if k=x∈:ψ(x)∈[ak,bk)X_k=\x ψ(x)∈[a_k,b_k)\ for k∈[K]k∈[K] is our TC(,K)TC(X,K), the decision problem in Equation 6 can be resolved in polynomial time with respect to #\#C. This remains true when we replace 22^C by the bounded multisubsets of C. As we will see in Sections 6 and G, symmetric junta-based TCs can be applied to a broad range of problems in the GFlowNet literature. Building on the solution to Equation 6, we can efficiently mask out undesirable transitions from each policy. In practice, this operation requires a negligible amount of arithmetic operations, adding an imperceptible extra cost to the sampling process. In Example 5.4, as an illustration, we solely verify whether s+1s+1 is within the boundaries of kX_k. 5.2 Complementary Characterization of SC GFlowNets Connection to Doob’s h-transform.33 3 This section is fairly technical and can be skipped by readers solely interested in our method. A natural question raised by our exposition is whether there is a relationship between each pF(k)p_F [t](k) for k∈[K]k∈[K]. We show in this section that, given a solution pFp_F to the original problem of sampling from x∈x in proportion to R(x)R(x) and any partition kk=1K\X_k\_k=1^K, a solution pF(k)p_F [t](k) to the optimization problem in Equation 5 can be derived from a Doob’s h-transform of pFp_F when TCTC is a disjoint cover. To understand this, we review Doob’s h-transform of Markov chains. Definition 5.8 (Doob’s h-transform). Let Stt≥1\S_t\_t≥ 1 be a Markov chain with state space ¯≔∪ S and transition kernel κ:¯×¯→[0,1]κ S× S→[0,1]. Let also h:¯→ℝh S be a harmonic function, i.e., h(s)=s′∼κ(s,⋅)[h(s′)]h(s)=E_s κ(s,·) [h(s ) ]. We call κ(h)(s,s′)=κ(s,s′)⋅h(s′)/h(s)κ [t](h)(s,s )=κ(s,s )· h(s )h(s) the Doob’s h-transform of κ. We refer the reader to (Rogers and Williams, 2000, Chapter I) for a comprehensive overview of Definition 5.8 and its applications to stochastic processes. This section shows that there is a harmonic function hkh_k for which pF(k)p_F [t](k) is the Doob’s hkh_k-transformation of pFp_F for each k∈[K]k∈[K]. To see this, we describe the generative process of a GFlowNet as a Markov chain having X as the absorbing set. Proposition 5.9 (GFlowNet as an absorbing Markov chain). Let pFp_F be a GFlowNet’s policy abiding by any balance condition and κF _F be a Markov kernel on ¯ S defined as κF(s,s′)=pF(s,s′) _F(s,s )=p_F(s,s ) if s∈s and κF(x,s′)=[s′=x] _F(x,s )= 1[s =x] if x∈x for each s′∈¯s ∈ S. Let St(f)t≥1\S_t [t](f)\_t≥ 1 be a Markov chain following κF _F, and let T=mint:St(f)∈T_X= \t S_t [t](f) \ be its hitting time on X. When pFp_F satisfies the TB, ℙκF[T<∞|so]=1 and ℙκF[ST(f)=x∈|so]=R(x)∑x∈R(x)P_ _F [T_X<∞|s_o ]=1 and P_ _F[S_T_X [t](f)=x |s_o]= R(x) _x R(x) with ℙκFP_ _F being the probability measure induced by the transition kernel κF _F. Equivalently, let pBp_B be a GFlowNet’s backward policy and κB _B a Markov kernel on ¯ S such that κB(s,s′)=pB(s,s′) _B(s,s )=p_B(s,s ) for s∈¯∖sos∈ S \s_o\ and s′∈¯s ∈ S and κB(so,s)=[s=so] _B(s_o,s)= 1[s=s_o]. As such, let St(b)t≥1\S_t [t](b)\_t≥ 1 follow κB _B; if Tso=mint:St(b)=so, then ℙκB[Tso<∞|x]=1 for every x∈.T_\s_o\= \t S_t [t](b)=s_o\, then P_ _B [T_\s_o\<∞|x ]=1 for every x . Under Proposition 5.9, we show that there is a transition kernel κF(k) _F [t](k) for which (i) kX_k is the only absorbing set, (i) each x∈kx _k is an absorbing state being arrived at with probability proportional to R(x)R(x), (i) and that κF(k) _F [t](k) is a Doob’s transformation of κF _F. For this, let St(f)t≥1\S_t [t](f)\_t≥ 1 be as above and Tk=mint:St(f)∈kT_k= \t S_t [t](f) _k \ be the hitting time on kX_k. We define hk(s)=ℙκF[Tk=mink′Tk′|s]h_k(s)=P_ _F [T_k= _k T_k |s ] as the probability that the Markov chain St(f)t≥1\S_t [t](f)\_t≥ 1 reaches kX_k before any other k′X_k when starting at s. The next lemma shows hkh_k is a harmonic function in the sense of Definition 5.8. Lemma 5.10. The functions hk:¯→ℝh_k S , k∈[K]k∈[K], are harmonic in S with respect to κF _F. They also satisfy hk(x)=[x∈k]h_k(x)= 1[x _k] for x∈x and, defining ZkZ_k and Z as in Corollary 5.2, hk(so)=Zk/Zh_k(s_o)= Z_kZ. Lemma 5.10 ensures Doob’s hkh_k-transform of κF _F is well-defined. We show in the next proposition that the kernels resulting from such transform sample each x∈kx _k in proportion to R(x)R(x). Proposition 5.11 (pF(k)p_F [t](k) is a Doob’s hkh_k-transform of pFp_F). Let κF _F be as in Proposition 5.9 and hk:¯→ℝh_k S be the harmonic function of Lemma 5.10 for k∈[K]k∈[K]. We define κF(k)(s,s′)=κF(s,s′)⋅hk(s′)/hk(s) and St(k)t≥1 _F [t](k)(s,s )= _F(s,s )· h_k(s )h_k(s) and \S_t [t](k)\_t≥ 1 as a Markov chain following κF(k) _F [t](k). Then, if Tm(k)=mint:St(k)∈mT_m [t](k)= \t S_t [t](k) _m \, ℙκF(k)[Tk(k)<∞=Tm(k)]=1 for m≠k and ℙκF(k)[STk(k)(k)=x]=R(x)∑x∈kR(x)P_ _F [t](k) [T_k^(k)<∞=T_m^(k) ]=1 for m≠ k and P_ _F [t](k) [S_T_k^(k) [t](k)=x ]= R(x) _x _kR(x) for all x∈kx _k. In particular, the policy function pF(k)p_F [t](k) defined as pF(k)(s,s′)=κF(k)(s,s′)p_F [t](k)(s,s )= _F [b](k)(s,s ) for s∈s and s′∈¯s ∈ S satisfies Equation 2, i.e., the marginal of pF(k)p_F [t](k) on X matches RkR_k up to a constant. In other words, Proposition 5.11 states pF(k)p_F [t](k) solves SC GFlowNet’s learning problem. 5.3 Sampling Partitions & Parallel Learning for SC GFlowNets Although Proposition 5.11 clearly connects each pF(k)p_F [t](k) to a base policy pFp_F, the quantity hk(x)h_k(x) cannot be efficiently computed in general even for known κF _F—let alone for the unknown solution to the GFlowNet’s balance equations. Besides, learning hkh_k from Lemma 5.10 can be impractical, as it would severely increase the per-transition computational cost. In practice, we parameterize our policies as in Example 5.4 and learn them by minimizing the objective function k∼ρM,τ∼ρk[ℒTB(τ,k,[K])].E_k _M,\,τ _k [L_TB (τ,\k\; g_[K] ) ]. (7) Active Partition Sampling. Equation 7 begs the question of how to sample k from [K][K] during training. Intuitively, we should select most often the k for which the probability mass under kX_k is largest, while also serendipitously picking the k such that kX_k is thought to only include low-probability states. As ZkZ_k serves as a surrogate for the probability mass under kX_k (Corollary 5.2), this intuition can be naturally operationalized by drawing k according to k∼ρM≔Categorical((1−ϵ)⋅/⊺+ϵ⋅K−1)k _M ((1-ε)· Z1 Z+ε· K^-1 ) with ϵ∈[0,1]ε∈[0,1] and 1 as a K-sized vector of ones. We emphasize more sophisticated approaches, such as curriculum-based sampling (Bengio et al., 2009; Laajil et al., 2025) focused on high-loss instead of high-mass components, could also be developed; however, we leave this for future research. We summarize both the training and inference procedures for SC GFlowNets in Algorithms 1 and 2. Embarrassingly Parallel Learning. Besides being trainable as a monolithic model through active partition sampling, SC GFlowNets enable distributed learning of GFlowNets without inter-process communication. We call this embarrassingly parallel learning: we learn K independent models, each of which generates samples from kX_k according to Rk(x)R_k(x). Afterwards, during inference, we pick each model in proportion to its learned partition function to generate samples from X. To separate SC GFlowNet from their asynchronous variants, we refer to the latter as ASC GFlowNets. Figure 3: Length-based SC GFlowNets trains faster than conventional GFlowNets in the Set Generation domain, requiring less than half the number of iterations to achieve the same accuracy. Figure 4: Average wall-clock times are similar for both samplers. Figure 5: SC GFlowNets converge substantially faster than GFlowNets in the Hypergrid domain, with distinct partitioning strategies (Length & Octant) yielding similar benefits. 6 Experiments Our experiments evaluate the effectiveness of SC GFlowNets. We consider the tasks of Set Generation, Hypergrid, and Ancestral Graphs, representing both classical and recent GFlowNet applications with tractable assessment metrics (Malkin et al., 2022; Malkin et al., 2023; Kim et al., 2023; Kim et al., 2025b), and defer Lazy Random Walk and Variable Selection, along with an evaluation of ASC GFlowNets and a comparison against Boosted GFlowNets (Dall’Antonia et al., 2026b), to Appendix G. Set generation. We let =s⊆[H]S=\s [H]\ for some H∈ℕH and =×⊤X=S×\ \ for the state space. We also define u:[H]→ℝ+u [H] _+ and let R(x)=∏e∈xu(e)R(x)= _e∈ xu(e) be the target distribution, which allows the probability πitem(e) _item(e) of e being a member of x for e∈[H]e∈[H] to be tractably computed. We evaluate the model via Erritem=12∑e∈[H]|π^item(e)−πitem(e)|Err_item= 12 _e∈[H]| π_item(e)- _item(e)|, with π^item π_item as the Monte Carlo estimate of πitem _item. For SC GFlowNets, we use a length-based partitioning with K=3K=3. Results for H∈24,48,64H∈\24,48,64\ in Figure 4 highlight the benefits of SC GFlowNet over its unpartitioned counterpart. In addition, we notice in Figure 4 the difference in runtime for both samplers is not significant. The main reason for this is that the cost of the extra steps required by the mixture model—active partition sampling and mask computation—is dwarfed by that of policy and reward evaluation. We observed similar runtime patterns across all our experiments. Hypergrid. The state space for Hypergrid is =0,…,H−1dS=\0,…,H-1\^d and =×⊤X=S×\ \. The initial state is so=s_o=0 and each transition from s∈s either adds 11 to one of s’s coordinates, respecting the boundaries of S, or interrupts the generative process, producing x=(s,⊤)x=(s, ). The target distribution, described in Appendix G, is shown in Figure 5. When designing SC GFlowNets, we consider both a length- (K=3K=3) and octant-based (K=4K=4) partitions, with the latter perfectly separating the four high-probability regions of X; see Figure 27 for illustrations. Results in Figure 5 confirm SC GFlowNets’s effectiveness even when using an imperfectly balanced partitioning. (a) BIC (Equation 20). (b) Erdős-Renyi model. Figure 6: SC GFlowNets enhance exploration (left) and speed up learning (right) for the Ancestral Graphs domain. We denote by H the number of variables (nodes). Ancestral graphs. This task was studied in (da Silva et al., 2026) for addressing the problem of causal discovery under latent confounding. In a nutshell, S contains all partially edge-labelled ancestral graphs (Richardson and Spirtes, 2002, AGs) with H nodes, while X contains all fully edge-labelled AGs. The generative process starts at a graph with unlabelled edges, and each transition labels a node pair with either ∅ , →, ←, or ↔ ; ∅ represents the absence of an edge. The objective is to generate high-scoring AGs x according to the Bayesian information criterion (BIC) based on a dataset modelled as a linear Gaussian structural equation model (Drton and Richardson, 2004); we set R(x)=exp−BIC(x)R(x)= \-BIC(x)\. As the distributional correctness of such method cannot be tractably measured for all but tiny H, we also consider an Erdős-Renyi model, for which edge marginals can be efficiently computed using Robinson’s formula (Robinson, 1973) and compared to that learned by our samplers (see Appendix G). In both cases, we partition X according to the number of non-∅ edges for SC GFlowNets. Figure 6 shows that our mixture model improves both exploration and convergence. 7 Conclusions The central questions we addressed in this work are why and how to learn a mixture of GFlowNets. We first developed a general-purpose framework for answering them, outlining the capabilities and limitations of such approach. In particular, we showed SC GFlowNets, which learn an ensemble of samplers restricted to subsets of the state space, provide substantial empirical benefits at a negligible computational cost. Future research should investigate better ways of operationalizing our framework and, specifically, develop heuristics for automatic and tractable partitioning of the state space. References Abboud et al. (2021) R. Abboud, İ. İ. Ceylan, M. Grohe, and T. Lukasiewicz The surprising power of graph neural networks with random node initialization. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), p. 2112–2118. Cited by: Example C.4, Appendix C, §H.10, §H.10, §1, §4. Bengio et al. (2021) E. Bengio, M. Jain, M. Korablyov, D. Precup, and Y. Bengio Flow network based generative models for non-iterative diverse candidate generation. In NeurIPS (NeurIPS), Cited by: §A.1, Example D.1, Appendix I, §1. Bengio et al. (2023) Y. Bengio, S. Lahlou, T. Deleu, E. J. Hu, M. Tiwari, and E. Bengio GFlowNet foundations. Journal of Machine Learning Research (JMLR). Cited by: Appendix I, §3, §3. Bengio et al. (2009) Y. Bengio, J. Louradour, R. Collobert, and J. Weston Curriculum learning. In International Conference on Machine Learning, Cited by: §5.3. Bishop (1995) C. M. Bishop Training with noise is equivalent to tikhonov regularization. Neural computation 7 (1), p. 108–116. Cited by: §4. Bishop (2007) C. M. Bishop Pattern recognition and machine learning. Springer. Cited by: §1. Blei et al. (2003) D. M. Blei, A. Y. Ng, and M. I. Jordan Latent dirichlet allocation. Journal of machine Learning research 3 (Jan), p. 993–1022. Cited by: §F.1. Blei and et al. (2017) D. M. Blei and et al. Variational inference: a review for statisticians. Journal of the American Statistical Association. Cited by: §1. Boussif et al. (2024) O. Boussif, L. N. Ezzine, J. D. Viviano, M. Koziarski, M. Jain, N. Malkin, E. Bengio, R. Assouel, and Y. Bengio Action abstractions for amortized sampling. External Links: 2410.15184, Link Cited by: Appendix I. Bradbury et al. (2018) JAX: composable transformations of Python+NumPy programs Cited by: §F.2, §G.2, Appendix G. Brunswic et al. (2024) L. Brunswic, Y. Li, Y. Xu, Y. Feng, S. Jui, and L. Ma A theory of non-acyclic generative flow networks. Proceedings of the AAAI Conference on Artificial Intelligence 38 (10), p. 11124–11131. External Links: ISSN 2159-5399, Link, Document Cited by: Appendix I. Caterini et al. (2021) A. Caterini, R. Cornish, D. Sejdinovic, and A. Doucet Variational inference with continuously-indexed normalizing flows. External Links: 2007.05426, Link Cited by: Appendix I. Chen et al. (2026) B. Chen, H. Ding, N. Shen, J. Huang, T. Guo, L. Liu, and M. Zhang AlphaSAGE: structure-aware alpha mining via gflownets for robust exploration. External Links: 2509.25055, Link Cited by: Appendix I. Cormen et al. (2009) T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein Introduction to algorithms. 3rd edition, MIT Press. External Links: ISBN 9780262033848 Cited by: §H.3, §5.1. Cornish et al. (2021) R. Cornish, A. L. Caterini, G. Deligiannidis, and A. Doucet Relaxing bijectivity constraints with continuously indexed normalising flows. External Links: 1909.13833, Link Cited by: Appendix I. da Silva et al. (2026) T. da Silva, B. Bazaluk, E. de Souza da Silva, A. Góis, S. Lahlou, D. Heider, S. Kaski, D. Mesquita, and A. H. Ribeiro Expert-aided causal discovery of ancestral graphs. Information Sciences 756, p. 123816. External Links: ISSN 0020-0255, Document Cited by: §G.2, §G.2, §6. Dall’Antonia et al. (2026a) P. Dall’Antonia, T. da Silva, D. Csillag, S. Lahlou, and D. Mesquita Avoid what you know: divergent trajectory balance for gflownets. External Links: 2602.17827, Link Cited by: §A.1, §A.2, §A.2, §G.1, Remark 3.4. Dall’Antonia et al. (2026b) P. Dall’Antonia, T. da Silva, D. A. de Souza, C. L. C. Mattos, and D. Mesquita Boosted gflownets: improving exploration via sequential learning. Cited by: §A.1, §A.2, §F.1, §F.1, §F.1, §F.2, §1, §1, §5, §6. Darema et al. (1988) F. Darema, D. A. George, V. A. Norton, and G. F. Pfister A single-program-multiple-data computational model for EPEX/FORTRAN. Parallel Computing 7 (1), p. 11–24. External Links: Document Cited by: §G.1. Deleu and Bengio (2023) T. Deleu and Y. Bengio Generative flow networks: a markov chain perspective. External Links: 2307.01422, Link Cited by: Appendix I. Deleu et al. (2022) T. Deleu, A. Góis, C. C. Emezue, M. Rankawat, S. Lacoste-Julien, S. Bauer, and Y. Bengio Bayesian structure learning with generative flow networks. In UAI, Cited by: Example D.1, Appendix I, §1. Deleu et al. (2023) T. Deleu, M. Nishikawa-Toomey, J. Subramanian, N. Malkin, L. Charlin, and Y. Bengio Joint Bayesian inference of graphical structure and parameters with a single generative flow network. In Advances in Neural Processing Systems (NeurIPS), Cited by: Appendix I, §1. Deleu et al. (2024) T. Deleu, P. Nouri, N. Malkin, D. Precup, and Y. Bengio Discrete probabilistic inference as control in multi-path environments. External Links: 2402.10309, Link Cited by: §3. Dempster et al. (1977) A. P. Dempster, N. M. Laird, and D. B. Rubin Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B 39, p. 1–38. External Links: Link Cited by: §3. Drton and Richardson (2004) M. Drton and T. S. Richardson Iterative conditional fitting for gaussian ancestral graph models. In Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI-04), p. 130–137. Cited by: §G.2, §6. Gelman and Meng (1998) A. Gelman and X. Meng Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Statistical Science 13 (2), p. 163–185. Cited by: Appendix I. George and McCulloch (1993) E. I. George and R. E. McCulloch Variable selection via Gibbs sampling. Journal of the American Statistical Association 88 (423), p. 881–889. External Links: Document Cited by: §G.1. Gritsaev et al. (2025) T. Gritsaev, N. Morozov, S. Samsonov, and D. Tiapkin Optimizing backward policies in gflownets via trajectory likelihood maximization. External Links: 2410.15474, Link Cited by: §A.1, §A.2. Gupta et al. (2026) A. Gupta, P. Barabanshchikova, V. K. Garg, S. Kaski, and T. Jaakkola Divide-and-denoise: a game-theoretic method for fairly composing diffusion models. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: Appendix I. Hastie et al. (2009) T. Hastie, R. Tibshirani, and J. Friedman The elements of statistical learning: data mining, inference, and prediction. 2nd edition, Vol. 2, Springer. Cited by: Appendix C. Heek et al. (2026) Flax: a neural network library and ecosystem for JAX External Links: Link Cited by: §G.2. Hoffman et al. (2010) M. D. Hoffman, D. M. Blei, and F. Bach Online learning for latent dirichlet allocation. In Advances in Neural Information Processing Systems, J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta (Eds.), Vol. 23, p. 856–864. External Links: Link Cited by: §F.1. Hu et al. (2023) E. J. Hu, M. Jain, E. Elmoznino, Y. Kaddar, and et al. Amortizing intractable inference in large language models. External Links: 2310.04363 Cited by: Appendix I, §1. Hu et al. (2026) R. Hu, Y. Zhang, Z. Li, and L. Huang Beyond squared error: exploring loss design for enhanced training of generative flow networks. External Links: 2410.02596, Link Cited by: §A.1. Jain et al. (2022) M. Jain, E. Bengio, A. Hernandez-Garcia, J. Rector-Brooks, B. F. P. Dossou, C. A. Ekbote, J. Fu, T. Zhang, M. Kilgour, D. Zhang, L. Simine, P. Das, and Y. Bengio Biological sequence design with GFlowNets. In International Conference on Machine Learning (ICML), Cited by: §A.2, §A.2. Jaynes (2003) E. T. JaynesG. L. Bretthorst (Ed.) Probability theory: the logic of science. Cambridge University Press, Cambridge. Cited by: §A.2. Jordan et al. (2024) K. Jordan, Y. Jin, V. Boza, Y. Jiacheng, F. Cesista, L. Newhouse, and J. Bernstein Muon: an optimizer for hidden layers in neural networks. External Links: Link Cited by: §G.2, §3. Khan and Nielsen (2018) M. E. Khan and D. Nielsen Fast yet simple natural-gradient descent for variational inference in complex models. External Links: 1807.04489, Link Cited by: §A.1. Kim et al. (2025a) H. Kim, S. Lee, and M. Oh Symmetry-aware gflownets. External Links: 2506.02685, Link Cited by: §A.1, §A.2, Appendix C. Kim et al. (2025b) M. Kim, S. Choi, T. Yun, E. Bengio, L. Feng, J. Rector-Brooks, S. Ahn, J. Park, N. Malkin, and Y. Bengio Adaptive teachers for amortized samplers. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.1, §A.2, §A.2, §G.1, §1, Remark 3.4, §6. Kim et al. (2024) M. Kim, J. Ko, T. Yun, D. Zhang, L. Pan, W. Kim, J. Park, E. Bengio, and Y. Bengio Learning to scale logits for temperature-conditional gflownets. External Links: 2310.02823, Link Cited by: §G.1. Kim et al. (2023) M. Kim, T. Yun, E. Bengio, D. Zhang, Y. Bengio, S. Ahn, and J. Park Local search gflownets. arXiv preprint arXiv:2310.02710. Cited by: §A.2, §G.1, §6. Kingma and Ba (2014) D. P. Kingma and J. Ba Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Cited by: §G.2. Kipf and Welling (2016) T. N. Kipf and M. Welling Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. Cited by: Example C.3. Laajil et al. (2025) A. Laajil, A. Shtanchaev, S. Muhammad, E. Moulines, and S. Lahlou Curriculum-augmented gflownets for mrna sequence generation. External Links: 2510.03811, Link Cited by: Example D.1, Appendix I, §5.3. Lahlou et al. (2023) S. Lahlou, T. Deleu, P. Lemos, D. Zhang, A. Volokhova, A. Hernández-García, L. N. Ezzine, Y. Bengio, and N. Malkin A theory of continuous generative flow networks. In ICML, Proceedings of Machine Learning Research, Vol. 202, p. 18269–18300. Cited by: §A.1, §1. Li et al. (2023) S. Li, I. Marinescu, and S. Musslick GFN-sr: symbolic regression with generative flow networks. External Links: 2312.00396, Link Cited by: Appendix I. Liu and Liu (2001) J. S. Liu and J. S. Liu Monte carlo strategies in scientific computing. Vol. 10, Springer. Cited by: Appendix I. Liu et al. (2024) S. Liu, P. J. Ramadge, and R. P. Adams Generative marginalization models. External Links: 2310.12920, Link Cited by: Example D.2, §1. Locatello et al. (2018) F. Locatello, R. Khanna, J. Ghosh, and G. Ratsch Boosting variational inference: an optimization perspective. In Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, A. Storkey and F. Perez-Cruz (Eds.), Proceedings of Machine Learning Research, Vol. 84, p. 464–472. Cited by: §F.1. Madan et al. (2025) K. Madan, A. Lamb, E. Bengio, G. Berseth, and Y. Bengio Towards improving exploration through sibling augmented GFlownets. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.1, §A.2, §1, Remark 3.4. Madan et al. (2022) K. Madan, J. Rector-Brooks, M. Korablyov, E. Bengio, M. Jain, A. C. Nica, T. Bosc, Y. Bengio, and N. Malkin Learning gflownets from partial episodes for improved convergence and stability. In International Conference on Machine Learning, Cited by: §A.1, Appendix B, §3. Malkin et al. (2022) N. Malkin, M. Jain, E. Bengio, C. Sun, and Y. Bengio Trajectory balance: improved credit assignment in GFlownets. In NeurIPS (NeurIPS), Cited by: §A.1, §A.1, Example D.2, §G.2, §H.8, §2, §3, §3, §4, §6. Malkin et al. (2023) N. Malkin, S. Lahlou, T. Deleu, X. Ji, E. Hu, K. Everett, D. Zhang, and Y. Bengio GFlowNets and variational inference. International Conference on Learning Representations (ICLR). Cited by: §A.1, §A.1, Appendix I, §1, §6. Morozov et al. (2025) N. Morozov, I. Maksimov, D. Tiapkin, and S. Samsonov Revisiting non-acyclic gflownets in discrete environments. External Links: 2502.07735, Link Cited by: Appendix I. Neal et al. (2011) R. M. Neal et al. MCMC using hamiltonian dynamics. Handbook of markov chain monte carlo. Cited by: §1. Neiswanger et al. (2014) W. Neiswanger, C. Wang, and E. P. Xing Asymptotically exact, embarrassingly parallel MCMC. In UAI, Cited by: §1. Owen (2013) A. B. Owen Monte carlo theory, methods and examples. Cited by: §1. Pan et al. (2023a) L. Pan, N. Malkin, D. Zhang, and Y. Bengio Better training of gflownets with local credit and incomplete trajectories. arXiv preprint arXiv:2302.01687. Cited by: §A.1. Pan et al. (2023b) L. Pan, D. Zhang, A. Courville, L. Huang, and Y. Bengio Generative augmented flow networks. In International Conference on Learning Representations (ICLR), Cited by: §A.1. Pandey et al. (2024) M. Pandey, G. Subbaraj, and E. Bengio GFlowNet pretraining with inexpensive rewards. arXiv preprint arXiv:2409.09702. Cited by: Appendix I. Pardo et al. (2022) F. Pardo, A. Tavakoli, V. Levdik, and P. Kormushev Time limits in reinforcement learning. External Links: 1712.00378, Link Cited by: §A.2. Richardson and Spirtes (2002) T. Richardson and P. Spirtes Ancestral graph markov models. The Annals of Statistics 30 (4), p. 962–1030. Cited by: §6. Robinson (1973) R. W. Robinson Counting labeled acyclic digraphs. In New Directions in the Theory of Graphs, F. Harary (Ed.), p. 239–273. External Links: ISBN 978-0-12-324255-6 Cited by: §G.2, §6. Rogers and Williams (2000) L. C. G. Rogers and D. Williams Diffusions, markov processes, and martingales: volume 1, foundations. 2nd edition, Cambridge Mathematical Library, Cambridge University Press. External Links: ISBN 9780521775946 Cited by: §5.2. Sato et al. (2021) R. Sato, M. Yamada, and H. Kashima Random features strengthen graph neural networks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), Cited by: Appendix C, §1. Schmidhuber (2020) J. Schmidhuber Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991). Neural Networks 127, p. 58–66. Cited by: §A.2. Shen et al. (2023) M. W. Shen, E. Bengio, E. Hajiramezanali, A. Loukas, K. Cho, and T. Biancalani Towards understanding and improving gflownet training. In International Conference on Machine Learning, Cited by: §A.2, §F.1. Silva et al. (2025a) T. Silva, R. B. Alves, E. de Souza da Silva, A. H. Souza, V. Garg, S. Kaski, and D. Mesquita When do GFlownets learn the right distribution?. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.2, Example C.3, Appendix C. Silva et al. (2025b) T. Silva, A. H. Souza, O. Rivasplata, V. Garg, S. Kaski, and D. Mesquita Generalization and distributed learning of GFlownets. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.2, Appendix E, Appendix E, §H.15, Appendix I, §1, footnote 4. Silva et al. (2026) T. Silva, E. S. Whitammer, and S. Lahlou Path-dependent discrete amortized inference. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §G.1, §5.1. Tiapkin et al. (2025) D. Tiapkin, A. Agarkov, N. Morozov, I. Maksimov, A. Tsyganov, T. Gritsaev, and S. Samsonov Gfnx: fast and scalable library for generative flow networks in jax. External Links: 2511.16592, Link Cited by: Appendix I. Tiapkin et al. (2024) D. Tiapkin, N. Morozov, A. Naumov, and D. Vetrov Generative flow networks as entropy-regularized rl. External Links: 2310.12934 Cited by: §A.1, §A.1, Appendix I. Venkatraman et al. (2024) S. Venkatraman, M. Jain, L. Scimeca, M. Kim, M. Sendera, M. Hasan, L. Rowe, S. Mittal, P. Lemos, and E. B. others Amortizing intractable inference in diffusion models for vision, language, and control. External Links: 2405.20971, Link Cited by: Appendix I, §1. Viviano et al. (2026) J. D. Viviano, O. G. Younis, S. Choi, V. Schmidt, Y. Bengio, and S. Lahlou Torchgfn: a pytorch gflownet library. External Links: 2305.14594, Link Cited by: Appendix I. Xu et al. (2019) K. Xu, W. Hu, J. Leskovec, and S. Jegelka How powerful are graph neural networks?. International Conference on Learning Representations (ICLR). Cited by: Example C.3. Yin and Zhou (2018) M. Yin and M. Zhou Semi-implicit variational inference. In International conference on machine learning, Cited by: Appendix I. Yu et al. (2026a) X. Yu, X. Wang, R. Zhu, Y. Zhang, and Y. Wang Exploring multiple high-scoring subspaces in generative flow networks. External Links: 2602.11491, Link Cited by: §1. Yu et al. (2026b) X. Yu, X. Wang, R. Zhu, Y. Zhang, and Y. Wang Partial gflownet: accelerating convergence in large state spaces via strategic partitioning. External Links: 2602.11498, Link Cited by: Appendix I, §1. Zhang et al. (2023) D. W. Zhang, C. Rainone, M. Peschl, and R. Bondesan Robust scheduling with gflownets. In International Conference on Learning Representations (ICLR), Cited by: Appendix I. Zhou et al. (2024) M. Y. Zhou, Z. Yan, E. Layne, N. Malkin, D. Zhang, M. Jain, M. Blanchette, and Y. Bengio PhyloGFN: phylogenetic inference with generative flow networks. In The Twelfth International Conference on Learning Representations, Cited by: §A.1, Appendix I, §1. Zimmermann et al. (2023) H. Zimmermann, F. Lindsten, J. van de Meent, and C. A. Naesseth A variational perspective on generative flow networks. Transactions on Machine Learning Research. Note: External Links: ISSN 2835-8856, Link Cited by: §1. Appendix A GFlowNets: Background & Challenges We provide a detailed background on GFlowNets, and review current challenges in effectively training these samplers, explaining how they can be naturally addressed by our mixture framework. A.1 Background This section reviews prior work on learning GFlowNets for discrete distributions, following Malkin et al. (2023); Lahlou et al. (2023), to which we refer for a comprehensive overview. GFlowNets. Let X be a finite set; we say X is a compositional space if there is a state graph (SG) =(,ℰ)G=(S,E) with vertices ⊃S and edges ℰE having the following properties. 1. G is a directed acyclic graph (DAG) with a single source, sos_o, and with X as the set of sink vertices. We call sos_o the initial state. 2. There is a (possibly non-unique) path from from each sos_o to each ∖soS \s_o\. We further assume, for computational reasons, that the number of children of each s∈s on G and the maximum trajectory length in G are exponentially smaller than |||X|. (Otherwise, every finite set X would be compositional by considering =so∪S=\s_o\ and ℰ=so→x:x∈E=\s_o→ x x \). Our objective is to generate x∈x in proportion to a positive function R:→ℝ+R _+. Sampling as learning. Towards this objective, we define pF:×(∪)→[0,1]p_F ×(S )→[0,1] (resp. pB:(∪)×→[0,1]p_B (S )×S→[0,1]) as the forward (resp. backward) policy on G representing the probability pF(s,s′)p_F(s,s ) of transitioning from s to a child s′s of s. The reason for pBp_B’s existence will become clearer shortly. We also define s↝s′s s as the set of trajectories in G from s to s′s . Our objective is for the marginal probability of pFp_F over X, when starting at sos_o, to match R, i.e., mF(x)≔∑τ∈so↝xτ=(so,s1),…,(sK,x)∏(s,s′)∈τpF(s,s′)≔∑τ∈so↝xpF(so,τ)∝R(x).m_F(x) _ subarraycτ∈ s_o x\\ τ=\(s_o,s_1),…,(s_K,x)\ subarray _(s,s )∈τp_F(s,s ) _τ∈ s_o xp_F(s_o,τ) R(x). (8) The trajectory length K in the equation above need not be the same for each τ. Importantly, we emphasize this equation merely integrates out from pFp_F intermediate variables from the joint distribution (so,…,x)(s_o,…,x) over trajectories. Due to Equation 8’s intractability, we estimate the summation above using pBp_B as a proposal for an importance-sampling-like scheme, that is, denoting Z=∑x∈R(x)Z= _x R(x), μF(so,x)=τ∼pB(x,⋅)[pF(so,τ)pB(x,τ)]=R(x)Z, _F(s_o,x)=E_τ p_B(x,·) [ p_F(s_o,τ)p_B(x,τ) ]= R(x)Z, with pB(x,τ)=∏(s,s′)∈τpB(s′,s)p_B(x,τ)= _(s,s )∈τp_B(s ,s). The condition pF(τ|so)/pB(τ|x)=R(x)/Z p_F(τ|s_o)p_B(τ|x)= R(x)Z is known as trajectory balance (TB) Malkin et al. (2022). To search for policies satisfying it, we parameterize pFp_F and (optionally) pBp_B as neural networks and solve the stochastic program minpF,pBℒ(pF,pB|ρ)≔τ∼ρ(logZ⋅pF(τ|so)pB(τ|x)⋅R(x))2 _p_F,p_BL(p_F,p_B|ρ) _τ ρ ( Z· p_F(τ|s_o)p_B(τ|x)· R(x) )^2 (9) via stochastic gradient descent. As the reader may have noticed, pBp_B was introduced arbitrarily; in general, there are infinitely many pairs (pF,pB)(p_F,p_B) satisfying the TB condition, a fact that will come up later in Section A.2 when we discuss mixtures of amortized samplers. Additional learning objectives, with distinct parameterizations and properties, have been proposed and extensively studied (Madan et al., 2022; Malkin et al., 2023; Tiapkin et al., 2024; Pan et al., 2023a; Hu et al., 2026). In practice, we let θ be the (joint) parameters of pFp_F, pBp_B, and logZ Z, αtt≥0\ _t\_t≥ 0 be a sequence of step sizes, and iteratively minimize θ(t)=minθθ⊺∇θ(t−1)ℒ^(θ|θ(t−1))+12αt‖θ−θ(t−1)‖22θ [t](t)= _θ _θ [t](t-1) L(θ|θ [t](t-1))+ 12 _t\|θ-θ [t](t-1)\|_2^2 (or a preconditioned variant of it (Khan and Nielsen, 2018)), with ℒ^(θ|θ(t−1)) L(θ|θ [t](t-1)) as an Monte Carlo estimate of ℒ(pF,pB|ρ(t−1))L(p_F,p_B|ρ [t](t-1)) when both pFp_F and pBp_B are instantiated according to θ and ρ(t−1)ρ [t](t-1) as a function of θ(t−1)θ [t](t-1) (Tiapkin et al., 2024; Kim et al., 2025b; Madan et al., 2025; Gritsaev et al., 2025; Dall’Antonia et al., 2026a). For example, the popular ϵε-greedy exploration (Bengio et al., 2021; Malkin et al., 2022; Pan et al., 2023b; Zhou et al., 2024; Kim et al., 2025a; Dall’Antonia et al., 2026b) uses ρ(t−1)(s,⋅)=(1−ϵ)⋅pF(t−1)(s,⋅)+ϵ⋅pU(s,⋅)ρ [t](t-1)(s,·)=(1-ε)· p_F [t](t-1)(s,·)+ε· p_U(s,·) with pU(s,⋅)p_U(s,·) as the policy function which selects the children of s uniformly at random and pF(t−1)(s,⋅)p_F [t](t-1)(s,·) as the pFp_F corresponding to θ(t−1)θ [t](t-1). A.2 Challenges in training GFlowNets To further motivate our work, we briefly describe current challenges in learning GFlowNets, and later argue they can be partly addressed through a mixture formulation of the sampling problem. Figure 7: Any α∈[1,2]α∈[1,2] solves the assignment problem in this state graph. The solution for uniform pBp_B is α=1.5α=1.5. The first challenge is that the optimization problem described in Equation 9 is underdetermined, as there is no pair (pF,pB)(p_F,p_B) uniquely satisfying the TB condition in general (unless there is a single trajectory from sos_o to each x, as in autoregressive sequence generation tasks Jain et al. (2022)). This is illustrated in Figure 7, which displays a family of pFp_F for which the marginal constraint in Equation 8 is satisfied, highlighting the existence of a continuum of solutions to the sampling problem. Challenge A.1 (Underdetermination). There are infinitely many pairs (pF,pB)(p_F,p_B) for which TB is satisfied and the marginal of pFp_F on X matches R, and it is unclear how to choose between them. Figure 8: KL between pBp_B and pUp_U for the Hypergrid domain. A common choice, inspired by the maximum entropy principle (Jaynes, 2003), is to set pB(s,⋅)p_B(s,·) as a uniform probability distribution over the parents of s in the SG (Shen et al., 2023). Alternatively, both pFp_F and pBp_B can be jointly learned (Gritsaev et al., 2025). However, deriving a pBp_B for which finding the corresponding pFp_F is easiest is not possible in general, and it is unclear which pair (pF,pB)(p_F,p_B) does gradient-based learning induce. We illustrate this in Figure 8 for the Hypergrid domain, showing the variability of KL(pB(s,⋅)||pU(s,⋅))KL(p_B(s,·)||p_U(s,·)) averaged over states s∈∖sos \s_o\ during training for 5 independent runs (the highlighted curve represents per-step averages). As we can observe, the learned pBp_B vary drastically solely by modifying the pseudorandom seed for sample generation—while maintaining both the parameter initialization and optimizer unchanged. A mixture model, as we will show, naturally addresses Challenge A.1 by learning a family of backward policies—hence avoiding strict commitment to a single, possibly unknown pBp_B. However, even when pBp_B is uniquely defined—e.g., in autoregressive sequence generation (Jain et al., 2022)—the underlying neural network might be incapable of finding an approximation to solution of the stochastic program in Equation 9. This problem, known as near state aliasing (Pardo et al., 2022), is particularly harmful for graph-structured domains (Silva et al., 2025a; Kim et al., 2025a). Challenge A.2 ((Near) state aliasing). A solution (pF,pB)(p_F,p_B) to Equation 9 might not be (accurately) realizable by the chosen family of parametric models. In Appendix C, we show how our method alleviates it by establishing a relationship between Random Network Initialization (RNI) in Graph Neural Networks (GNNs) and mixture modelling. Even an universal approximator, however, may struggle to search for a compatible pair (pF,pB)(p_F,p_B) simply due to the sheer size of the state space. The main reason for this is that, to accurately approximate the target distribution, the model has to reach its high-probability regions during training Kim et al. (2023); Kim et al. (2025b); Dall’Antonia et al. (2026a), an obstacle that typical ϵε-greedy strategies fail to overcome. Challenge A.3 (Insufficient exploration). Accurately learning (pF,pB)(p_F,p_B) requires visiting the target R’s high-probability regions during training, which are often diverse and sparsely distributed. To address Challenge A.3, recent works have proposed using boosting-style techniques (Dall’Antonia et al., 2026b) and artificial curiosity-inspired Schmidhuber (2020) exploratory models—which are discarded after training. These approaches, however, increase the per-step computational cost of conventional samplers by severalfold, requiring the learning and evaluation of one Kim et al. (2025b); Madan et al. (2025); Dall’Antonia et al. (2026a) or many additional models Silva et al. (2025b); Dall’Antonia et al. (2026b) throughout learning and inference. In contrast, in Section 6, we showed improved exploration is achievable by a mixture of GFlowNets—without relying on additional, independently parameterized costly models. Appendix B Mixture of GFlowNets We outline additional results on the learning of a mixture of GFlowNets: an extension of Madan et al. (2022)’s SubTB condition and the associated family of loss functions. We start with the former. Proposition B.1 (SubTB). For each s∈∪s , let F(s,dγ)F(s,dγ) be a measure on Γ such that F(s,dγ)=R(x,dγ)F(s,dγ)=R(x,dγ) for x∈x . We call F a flow function. If, for τ∈s↝s′τ∈ s s , F(s,dγ)pF(γ)(s,τ)=F(s′,dγ)pB(γ)(s′,τ),F(s,dγ)p_F [t](γ)(s,τ)=F(s ,dγ)p_B [t](γ)(s ,τ), then Equation 2 is satisfied. In addition, for each s∈s , Z(dγ)=F(so,dγ) and F(s,dγ)=∑x∈∑τ∈s↝xR(x,dγ)pB(γ)(x,τ).Z(dγ)=F(s_o,dγ) and F(s,dγ)= _x _τ∈ s xR(x,dγ)p_B [t](γ)(x,τ). We provide a demonstration for this and every other statement in the supplement in Appendix H, jointly with the statements in the main text. From Proposition B.1, the usual loss functions encountered in the GFlowNet literature can be readily derived. Definition B.2 (SubTB & DB & TB losses). Let Z(⋅)Z(·) and F(s,⋅)F(s,·) and R(x,⋅)R(x,·) be measures on Γ , as in Proposition B.1, for s∈s and x∈x , and pF(γ)p_F [t](γ) and pB(γ)p_B [t](γ) be Γ -indexed policies. We concisely let Γ=(Z,F,R,pF(γ),pB(γ)γ∈Γ) g_ =(Z,F,R,\p_F [t](γ),p_B [t](γ)\_γ∈ ). Given τ=(so,…,sN)τ=(s_o,…,s_N) with sN=x∈s_N=x , we define τi:j=(si,si+1,…,sj) _i:j=(s_i,s_i+1,…,s_j) and ℒSubTB(τ,dγ;Γ)=∑0≤i<j≤Nαij(logF(si,dγ)pF(γ)(si,τi:j)F(sj,dγ)pB(γ)(sj,τj:i))2L_SubTB(τ,dγ; g_ )= _0≤ i<j≤ N _ij ( F(s_i,dγ)p_F [t](γ)(s_i, _i:j)F(s_j,dγ)p_B [t](γ)(s_j, _j:i) )^2 as the SubTB loss, with (αij)i,j=0N( _ij)_i,j=0^N as positive numbers with ∑j>iαij=1 _j>i _ij=1. Similarly, ℒDB(τ,dγ,Γ)=1N∑0≤i≤N−1(logF(si,dγ)pF(γ)(si,si+1)F(si+1,dγ)pB(γ)(si+1,si))2L_DB(τ,dγ; g_ )= 1N _0≤ i≤ N-1 ( F(s_i,dγ)p_F [t](γ)(s_i,s_i+1)F(s_i+1,dγ)p_B [t](γ)(s_i+1,s_i) )^2 is the DB loss. In conclusion, we let the TB loss be ℒTB(τ,dγ,Γ)=(Z(dγ)pF(γ)(so,τ)R(x,dγ)pB(γ)(x,τ))2.L_TB(τ,dγ; g_ )= ( Z(dγ)p_F [t](γ)(s_o,τ)R(x,dγ)p_B [t](γ)(x,τ) )^2. Appendix C Continuously Indexed GFlowNets In this section, we elaborate on the design of a continuously indexed mixture of GFlowNets. As we will show, this approach can drastically boost the expressivity of GNN-parameterized samplers (Kim et al., 2025a) without requiring expensive heuristics such as look-ahead search, as proposed in (Silva et al., 2025a). In doing so, we designedly use a collapsed mixing posterior distribution—q(dγ|x)=q(dγ)q(dγ|x)=q(dγ) for every x∈x —an approach we refer to as random features (RFs) for GFlowNets. The reason for this is that either Z(dγ)Z(dγ) or q(dγ|x)q(dγ|x) are generally intractable measures when Γ is uncountable, as we also demonstrate. Continuous mixtures of GFlowNets. We henceforth consider Γ=ℝd =R^d and ν as the Lebesgue measure, and we assume both qM(dγ|x)q_M(dγ|x) (resp. R(x,dγ)R(x,dγ), F(s,dγ)F(s,dγ)) and pM(dγ)p_M(dγ) (resp. Z(dγ)Z(dγ)) are absolutely continuous with respect to it. As such, we write R(x,γ)R(x,γ) and F(s,γ)F(s,γ) and Z(γ)Z(γ) for their corresponding densities. It is an obvious consequence of Proposition B.1 that the balance conditions can be written as a function of these densities instead of the previously introduced measures. We highlight this in the following corollary. Remark C.1 (SubTB). When Γ=ℝd =R^d and F(s,dγ)F(s,dγ) and Z(dγ)Z(dγ) are absolutely continuous with respect to the base Lebesgue measure ν, the SubTB condition is F(s,γ)pF(γ)(s,τ)=F(s′,γ)pB(γ)(s′,τ)F(s,γ)p_F [t](γ)(s,τ)=F(s ,γ)p_B [t](γ)(s ,τ) for each s,s′∈∪s,s and τ∈s↝s′τ∈ s s and ν-almost surely for γ∈Γγ∈ . As per Remark C.1, we learn F(s,γ)F(s,γ) as a real-valued neural network-parameterized positive function receiving both s∈s and γ∈Γγ∈ as inputs; recall F(x,γ)=R(x,γ)F(x,γ)=R(x,γ) for x∈x . The core issue in doing so, however, is that we may not be able to sample from the prior mixing distribution during inference when the function Z(γ)Z(γ) is defined arbitrarily. Conversely, if we let pM(γ)p_M(γ) have a simple form (e.g., Gaussian) and qM(γ|x)q_M(γ|x) be flexibly learned, the latter distribution may collapse into the former. This can be understood via the following corollary of Proposition B.1. Corollary C.2 (Relationship between pMp_M and qMq_M). Assume pM,qM,pF(γ),pB(γ)p_M,q_M,p_F [t](γ),p_B [t](γ) satisfy the SubTB condition in Remark C.1. Then, pM(γ)=∑x∈R(x)Z⋅qM(γ|x) and qM(γ|x)=Z⋅pM(γ)⋅mF(γ)(x)R(x).p_M(γ)= _x R(x)Z· q_M(γ|x) and q_M(γ|x)= Z· p_M(γ)· m_F [t](γ)(x)R(x). The first equation in Corollary C.2 shows that, if qM(γ|x)q_M(γ|x) has a tractable, non-trivial form, then the solution for pM(γ)p_M(γ) is a combinatorial mixture of all qγ(γ|x)q_γ(γ|x)’s for x∈x weighted by R(x)R(x), which is the distribution we are aiming to sample from in the first place. The second equation, on the other hand, suggests that if pF(γ)(s,⋅)p_F [t](γ)(s,·) is too flexible a function of γ and s—able to perfectly mimic the target R(x)R(x)—then the optimal form for qM(γ|x)q_M(γ|x) reduces to pM(γ)p_M(γ). As we also discussed in Section 5, reduction is not necessarily undesirable, as we will show later; however, we believe a x-independent qMq_M should be an algorithmic design decision rather than an implicit consequence of training. Should the goal be to have meaningfully different pF(γ)p_F [t](γ) for each γ with the objective of, for instance, improving exploration, Corollary C.2 suggests that sensible structural constraints should be imposed on the relationship between pF(γ)p_F [t](γ) and γ, a venue we pursued in Section 5. Random Features (RFs) for GFlowNets. In this scenario, we describe the potential benefits of setting qM(γ|x)=pM(γ)q_M(γ|x)=p_M(γ) by connecting the ensuing framework with the theory of RFs for GNNs Abboud et al. (2021); Sato et al. (2021). We first recall the expressivity limitations of message passing GNN-parameterized GFlowNets in the following example. Example C.3 (Limitations of GNN-based GFlowNets). We consider the following state graph (Figure 9) with initial state sos_o being followed by x1x_1 and x2x_2, each of them a 66-node graph. We notice that x1x_1 and x2x_2 correspond to xox_o with additional edges (a,b)(a,b) and (a,c)(a,c), respectively, and we write x1=so∪(a,b)x_1=s_o∪\(a,b)\ and x2=so∪(a,c)x_2=s_o∪\(a,c)\. This is the simplest setting a GFlowNet can be applied to. Figure 9: State graph for Example C.3. As in (Silva et al., 2025a), we let h:(,n)↦ℝdh (G,n) ^d be a 1-WL GNN Kipf and Welling (2016); Xu et al. (2019) yielding a representation for a node n in a graph G and σ:ℝd→ℝσ ^d be a learnable real-valued function. We parameterize the policy function according to pF(s,s∪(n1,n2))∝expσ(h(s,n1)+h(s,n2))p_F(s,s∪\(n_1,n_2)\) \σ(h(s,n_1)+h(s,n_2)) \ (10) for nodes n1,n2n_1,n_2 such that there is no edge between n1,n2n_1,n_2 in s, and 00 otherwise. Under these conditions, unless the target R satisfies R(x1)=R(x2)R(x_1)=R(x_2), there is no instantiation of pFp_F for which the marginal distribution of pFp_F over x1,x2\x_1,x_2\ matches R—as h cannot distinguish node b from node c. When augmenting h with random node-wise features γ, however, the policy function in Equation 10 acquires the capability of approximating any target with high-probability over the draw of γ. The intuition is that a RF-augmented 1-WL GNN will only be unable to separate nodes b and c in Figure 9 if their corresponding features match—an event of null probability since Γ is continuous. Example C.4 (RFs boost the expressivity of GNN-based GFlowNets). We let h:(,n,γn)↦ℝdh (G,n, _n) ^d be a RF-augmented 1-WL GNN with γn∼(0,1) _n (0,1). Then, if pF(γ)(s,s∪(n1,n2))∝expσ(h(s,n1,γn1)+h(s,n2,γn2))p_F [t](γ)(s,s∪\(n_1,n_2)\) \σ(h(s,n_1, _n_1)+h(s,n_2, _n_2)) \ (11) as in Equation 10, it follows from (Abboud et al., 2021, Theorem 1) that for each ϵ>0ε>0 and δ∈(0,1)δ∈(0,1) there is a GNN h such that pFp_F approximates a given target with accuracy ϵε and probability 1−δ1-δ over the drawing of γ. More broadly, we show in the next that a GFlowNet parameterized as in Equation 11 can arbitrarily approximate with high probability any given distribution defined on an edge-additive generative process for graph-structured objects. For this, we define TV(p,q)TV(p,q) as the total variation distance between (possibly unnormalized) probability distributions p and q. Proposition C.5 (RFs GNN-based GFlowNets are Universal Approximators). Let (pF(γ),pB(γ))γ∈Γ\(p_F [t](γ),p_B [t](γ))\_γ∈ be a GFlowNet augmented with RFs γ∈Γγ∈ and defined over a generative process over graphs such that each transition corresponds to either adding an edge or stopping. We define the following hypothesis classes. 1. ℋt=h:(,n,γn)↦ℝd,σ:ℝd→ℝ,d∈ℕH_t=\h (G,n, _n) ^d,\,σ ^d ,\,d \ containing 1-WL GNNs h and a real-valued function σ. 2. ℋs=ξ:(,n:n∈)↦yH_s=\ξ (G,\x_n n \) y\ containing readout functions mapping a graph and its node embeddings to y∈ℝy . We parameterize pF(γ)p_F [t](γ) for edge addition, respectively, as in Equation 11, assigning also a probability proportional to expξ(s,h(s,n):n∈s) \ξ(s,\h(s,n) n∈ s\)\ for interrupting the generation at s∈s , and fix pB(γ)p_B [t](γ). Then, for any target distribution R, there is a h∈ℋth _t and a ξ∈ℋsξ _s such that the marginal distribution mF(γ)m_F [t](γ) satisfies TV(mF(γ),R)≤ϵTV(m_F [t](γ),R)≤ε with probability larger than 1−δ1-δ over the draw of γ. With this in mind, a thorough investigation of the benefits of RF GFlowNets is left for future research. An extensive analysis of how to usefully define pMp_M is of particular importance. RFs & Stability. Besides improving the expressivity of GNN-based GFlowNets, we also demonstrate RFs impose an implicit Tikhonov regularization (Hastie et al., 2009) on the learning objective that mitigate the instability of pBp_B observed in Figure 8 in Section A.2. To understand this, we recall that, if L(θ)L(θ) is a loss function associated with parameters θ and θ⋆θ is a local minimum of L, L(θ)=L(θ⋆)+12(θ−θ⋆)⊤H(θ⋆)(θ−θ⋆)+(‖θ−θ⋆‖3)L(θ)=L(θ )+ 12(θ-θ ) H(θ )(θ-θ )+O(\|θ-θ \|^3) with H(θ⋆)=∇θ2L(θ⋆)H(θ )= _θ^2L(θ ) representing L’s Hessian at θ⋆θ . From this, we observe that H(θ⋆)H(θ ) characterizes the local geometry of L around θ⋆θ ; when H(θ⋆)H(θ ) is nearly singular, the loss surface around θ⋆θ is almost flat, a configuration that first-order gradient-based methods struggle to navigate. We next show that incorporating RFs into the policies of a GFlowNet has the effect of increasing the eigenvalues of the loss function’s Hessian, reducing the geometry’s flatness and—as we will empirically illustrate—reducing the functional variability of the learned (pF,pB)(p_F,p_B) originating solely from parameter initialization. We start with the following lemma. Lemma C.6 (Local Approximation for RF-augmented Loss). We define Γ=pF(γ),pB(γ)γ∈Γ g_ =\p_F [t](γ),p_B [t](γ)\_γ∈ as a GFlowNet augmented with RFs γ. We assume γ is drawn from a zero-centered distribution pMp_M with covariance matrix σ2Iσ^2I for some σ>0σ>0. Then, γ∼pM[ℒTB(τ,γ,Γ)]=ℒTB(τ,0,Γ)+12σ2⋅tr(Hγ(τ,0))+(σ4),E_γ p_M [L_TB(τ,γ; g_ ) ]=L_TB(τ,0; g_ )+ 12σ^2·tr (H_γ(τ,0) )+O(σ^4), (12) with trtr being the trace operator, HγH_γ representing the Hessian of ℒTB(τ,γ,Γ)L_TB(τ,γ; g_ ) with respect to γ, and ℒTBL_TB as in Definition B.2. Lemma C.6 follows from a Taylor approximation of the loss function around γ=0γ=0 and the fact that the odd moments of γ null out. Notably, the quantity ℒTB(τ,0,Γ)L_TB(τ,0; g_ ) corresponds to the loss of an unaugmented GFlowNet. By averaging τ out of Equation 12, we show that the trace term within it acts as a regularizer. Proposition C.7 (RFs as Regularizers). Let Γ g_ be as in Lemma C.6 and o g_o be the GFlowNet corresponding to γ=0γ=0. We define ℒTB(Γ)=τ∼ργ∼pM[ℒTB(τ,γ,Γ)],ℒTB(o)=τ∼ρ[ℒTB(τ,0,Γ)],L_TB( g_ )=E_τ ρE_γ p_M [L_TB(τ,γ; g_ ) ],\,L_TB( g_o)=E_τ ρ [L_TB(τ,0; g_ ) ], and ℒReg(Γ)=τ∼ρ[tr(Hγ(τ,0))]L_Reg( g_ )=E_τ ρ [tr(H_γ(τ,0)) ] for an exploratory policy ρ. Also, let θ be the parameters of Γ g_ , and let HTB=∇θ2ℒTB(Γ)H_TB= _θ^2L_TB( g_ ), Ho=∇θ2ℒTB(o)H_o= _θ^2L_TB( g_o), and HReg=∇θ2ℒReg(Γ)H_Reg= _θ^2L_Reg( g_ ). In this setting, HTB=Ho+σ22⋅HReg+(σ4).H_TB=H_o+ σ^22· H_Reg+O(σ^4). (13) Figure 10: Random features act as a regularizer for GFlowNet training. Importantly, Equation 13 suggests σ should be set to a small positive number to ensure the reminder term in (σ4)O(σ^4) is negligible. To measure the effects of our method in practice, we calculate the variance of avgKL(pB||pU)=1||∑s∈KL[pB(s,⋅)||pU(s,⋅)]avgKL(p_B||p_U)= 1|S| _s KL[p_B(s,·)||p_U(s,·)], with pUp_U being an uniform backward policy. We train a continuously indexed (CI) GFlowNet with Γ=ℝ8 =R^8 and γ∼(0,0.152⋅I)γ (0,0.15^2· I) and a conventional GFlowNet for the Hypergrid environment (see Section 6). As in Figure 8, we fix both the architecture and the parameter initialization of the neural networks underlying pF(γ)p_F [t](γ) and pB(γ)p_B [t](γ), and compute the per-step variance of the KL divergence by solely modifying the initial pseudorandom seed of the exploratory policy. We repeat this process across 5 independent runs. In conformity with Proposition C.7, we observe in Figure 10 that RFs have a stabilizing effect on the learning problem, smoothing the loss surface and increasing the model’s robustness to the randomness of sample generation—as emphasized by the reduced variance of avgKL(pB||pU)avgKL(p_B||p_U). Appendix D Discretely Indexed GFlowNets We complement Section 5 by introducing further examples of how to partition the state space, besides the abstract modulo-based technique described in Remark 5.5. Example D.1 (Length-based TCTC). Let d(x)d(x) be the trajectory-size, or description-length, for generating a x∈x . Common GFlowNet applications such as sequence design (Laajil et al., 2025), variable selection, drug discovery (Bengio et al., 2021), and structure learning (Deleu et al., 2022) often have d(x)∈[dmin,dmax]d(x)∈[d_min,d_max] with both dmind_min and dmaxd_max known. Under this condition, we can define TC(,K)=kk=1KTC(X,K)=\X_k\_k=1^K for K≤D≔dmax−dmin+1K≤ D d_max-d_min+1 as k=x∈:dmin(k)≔dmin+k−1K⋅D≤d(x)<dmax(k)≔dmin+kK⋅D.X_k= \x d_min [t](k) d_min+ k-1K· D≤ d(x)<d_max [t](k) d_min+ kK· D \. The policy pF(k)p_F [t](k) can be simply implemented by masking out terminating transitions leading to x for which d(x)∉[dmin(k),dmax(k))d(x)∉[d_min [t](k),d_max [t](k)). Example D.2 (Hamming weight-based TCTC). Let =1,0dX=\1,0\^d, which is often the cases in applications such as bit sequences (Malkin et al., 2022) and Ising model simulation (Liu et al., 2024). We define w(x)=∑1≤i≤dxiw(x)= _1≤ i≤ dx_i as the Hamming weight of x and TC(,K)=kk=1KTC(X,K)=\X_k\_k=1^K as k=x∈:w(x)∈[ak,bk)X_k=\x w(x)∈[a_k,b_k)\ with a1=0a_1=0, bK=d+1b_K=d+1, and ⋃k=1K[ak,bk)=[0,d+1) _k=1^K[a_k,b_k)=[0,d+1). We implement pF(k)p_F [t](k) by pruning transitions resulting in states for which the condition w(x)∈[ak,bk)w(x)∈[a_k,b_k) is unattainable. The tasks above can be easily seen to fall into the symmetric-junta category we outlined in Proposition 5.7. Example D.1 straightforwardly follows from picking ψ(s)ψ(s) to be equal to the number of components defining s. Example D.2, on the other hand, follows from letting s be represented as a subset of 1,0×[d]\1,0\×[d] and defining ψ(s)=#(s∩r)ψ(s)=\#(s∩ r) with the reference set r=(1,1),…,(1,d)r=\(1,1),…,(1,d)\ corresponding to a sequence containing only 11s. In either case, a tractable implementation for SC GFlowNets is admissible. Implementation of SC GFlowNets. In addition, to further clarify the implementation of SC GFlowNets, we provide a high-level description of its training and inference in the pseudocode in Algorithms 1 and 2, respectively, alongside detailed and efficient computer code for reproducibility. Algorithm 1 Training of SC GFlowNets K, =(Zk)k∈[K]Z=(Z_k)_k∈[K] trainingIterationstrainingIterations, ϵ>0,batchSizeε>0,\,batchSize SC GFlowNet [K] g_[K] getPolicies:k,[K]↦pF(k),pB(k)getPolicies k, g_[K] \p_F^(k),p_B^(k)\ for _\_ in range(trainingIterations)range(trainingIterations) do Samples←Samples←\\ parfor _\_ in range(batchSize)range(batchSize) do Z←(1−ϵ)⋅KT+ϵ⋅Kp_Z←(1-ε)· Z1_K^TZ+ε· 1_KK k∼Categorical(Z)k (p_Z ) pF(k),pB(k)←getPolicies(k,[K])p_F^(k),p_B^(k) (k, g_[K]) #\# Sample trajectory from pF(k)p_F^(k) τ∼ϵ-greedy ofpF(k)(so,⋅)τ ε-greedy of\,p_F^(k)(s_o,·) Samples←Samples∪(τ,k)Samples ∪\(τ,k)\ end parfor loss←lossFunction(Samples,[K],)loss (Samples, g_[K],Z) [K],←GradientStep(loss,[K],) g_[K],Z (loss, g_[K],Z) end for Algorithm 2 Inference with SC GFlowNets K, =(Zk)k∈[K]Z=(Z_k)_k∈[K] SC GFlowNet [K] g_[K] getPolicies:k,[K]↦pF(k),pB(k)getPolicies k, g_[K] \p_F^(k),p_B^(k)\ k∼Categorical(KT)k ( Z1_K^TZ ) pF(k),pB(k)←getPolicies(k,[K])p_F^(k),p_B^(k) (k, g_[K]) #\# Sample trajectory τ∼pF(k)(so,⋅)τ p_F^(k)(s_o,·) #\# Get terminal state from τ x←TerminalState(τ)x (τ) return x Appendix E Revisiting Subgraph Asynchronous Learning (SAL) This section and the next highlight how our framework can be instantiated to accommodate prior algorithms for learning GFlowNets. We start with SAL (Silva et al., 2025b). Similarly to SC GFlowNets, SAL defines a fixed-horizon covering (FHC)44 4 Silva et al. (2025b) called it a fixed-horizon partition (FHP); however, the underlying sets need not be disjoint, as the word partition suggests. of X, referred to as FHC(,K)FHC(X,K). Definition E.1 (Fixed-Horizon Covering). We call FHC(,K)=o∪kk=1KFHC(X,K)=X_o∪\X_k\_k=1^K a fixed-horizon covering (FHC) if the following conditions are satisfied. 1. There are disjoint subsets kk=1K\S_k\_k=1^K of S such that kX_k contains all states in X reachable from some s∈ks _k. 2. Each s∈ks _k is within a fixed-distance d to sos_o, and all states within distance d of sos_o are contained in some kS_k (fixed-horizon). 3. oX_o is the subset of X whose elements’ to sos_o to x is smaller than d. We call distance from s to sos_o the length of the shortest-path from sos_o and s. Drawing on Definition E.1, each partition enforces a so-called Amortized Trajectory Balance (ATB), defined as F(s,k)pF(k)(s,τ)=R(x)pB(x,τ) for all k∈[K],s∈k,x∈k, and τ∈s↝x,F(s,k)p_F [t](k)(s,τ)=R(x)p_B(x,τ) for all k∈[K],\,s _k,\,x _k, and τ∈ s x, (14) and then Z⋅pF(o)(s,τ′)=F(s,k)pB(s,τ′)Z· p_F [t](o)(s,τ )=F(s,k)p_B(s,τ ) and Z⋅pF(o)(s,τ′)=R(x)pB(s,τ′)Z· p_F [t](o)(s,τ )=R(x)p_B(s,τ ) for k∈[K]k∈[K], s∈ks _k, x∈ox _o, and τ∈so↝sτ∈ s_o s and τ′∈so↝xτ ∈ s_o x. In other words, each model first enforces a balance condition within its own partition according to a shared (uniform) backward policy pBp_B, and their learned flow functions F(⋅,k)F(·,k) are later used as the target distribution of the centralized model’s (pF(o)p_F [t](o)) own learning objective. We next show that a solution to SAL’s amortized TB can be interpreted as a DI GFlowNet satisfying its own MTB in Definition 3.2. We do this by introducing a Doob h-transform (see Section 5.2) that maps pF(o)p_F [t](o) and pF(k)p_F [t](k) into a policy function that strictly goes through kS_k when sampling from kX_k. With this in mind, we let Tk(f)=mint:St(f)∈k and Tk(b)=mint:St(b)∈kT_k [t](f)= \t S_t [t](f) _k\ and T_k [t](b)= \t S_t [t](b) _k\ be the hitting times on kS_k of the Markov chains introduced in Proposition 5.9, and hk(s)=ℙκF[Tk(f)=mink′∈[K]Tk′(f)<∞|s] and gk(s)=ℙκB[Tk(b)=mink′∈[K]Tk′(b)<∞|s]h_k(s)=P_ _F [T_k [t](f)= _k ∈[K]T_k [t](f)<∞ |s ] and g_k(s)=P_ _B [T_k [t](b)= _k ∈[K]T_k [t](b)<∞ |s ] be the corresponding probabilities that a Markov chain starting at s∈¯s∈ S and following κF _F (resp. κB _B) arrives at kS_k prior to arriving at any other k′S_k with k′≠k ≠ k. We first show that both hkh_k and gkg_k are harmonic functions. Lemma E.2. Both hkh_k and gkg_k are harmonic functions on ¯∖k S _k for k∈[K]k∈[K]. A core property required by the next Proposition E.4 is that both hk(so)h_k(s_o) and gk(x)g_k(x) for x∈x can be interpreted as probability distributions over [K][K]. As we emphasize next, this intuitively follows from the fact that kk=1K\S_k\_k=1^K forms a boundary between sos_o and the set X of terminal states. Lemma E.3. The functions hkh_k and gkg_k in Lemma E.2 satisfy ∑0≤k≤Khk(so)=1 and ∑0≤k≤Kgk(x)=1 for x∈ _0≤ k≤ Kh_k(s_o)=1 and _0≤ k≤ Kg_k(x)=1 for x with ho(so)=ℙκF[Tk(f)=∞∀k∈[K]|so]h_o(s_o)=P_ _F [T_k^(f)=∞∀ k∈[K] |s_o ] and go(x)=ℙκB[Tk(b)=∞∀k∈[K]|x]g_o(x)=P_ _B [T_k^(b)=∞\,∀ k∈[K] |x ] referring to the case when the generative process terminates on oX_o (see Definition E.1). Building on Lemmas E.2 and E.3, we demonstrate in Proposition E.4 that SAL can be viewed as an instance of a DI GFlowNet by showing that a solution to Equation 14 can be uniquely represented as a solution to the mixture learning problem in Definition 3.2 with a specific choice of ZkZ_k and q(k|x)q(k|x). This suggests yet another form of designing a DI GFlowNet. It should nonetheless be noticed that, contrarily to SC GFlowNets, SAL is an inherently distributed algorithm—as neither hkh_k nor gkg_k in Lemma E.2 can be tractably computed in general. Proposition E.4 (SAL as a DI GFlowNet). Let pF(k)k=0K\p_F [t](k)\_k=0^K and pBp_B be the solutions to SAL’s balance equations having d as the distance from sos_o to kk=1K\S_k\_k=1^K. We define p~F(k)(s,s′)=pF(o)(s,s′)⋅hk(s′)hk(s) if dist(so,s′)≤d,pF(k)(s,s′) otherwise, p_F [t](k)(s,s )= casesp_F [t](o)(s,s )· h_k(s )h_k(s) if dist(s_o,s )≤ d,\\ p_F [t](k)(s,s ) otherwise, cases with distdist representing the shortest-path distance in the underlying state graph, and p~B(k)(s,s′)=pB(s,s′) if dist(so,s′)≤d,pB(s,s′)⋅gk(s′)gk(s) otherwise, p_B [t](k)(s,s )= casesp_B(s,s ) if dist(s_o,s )≤ d,\\ p_B(s,s )· g_k(s )g_k(s) otherwise, cases for k≥1k≥ 1 and p~B(o)(s,s′)=pB(s,s′) p_B [t](o)(s,s )=p_B(s,s ) for k=0k=0. Then, for k∈0,…,Kk∈\0,…,K\ and each trajectory τ from sos_o to x∈kx _k going through kS_k, Zkp~F(k)(so,τ)=R(x)q(k|x)p~B(k)(x,τ)Z_k p_F [t](k)(s_o,τ)=R(x)q(k|x) p_B [t](k)(x,τ) with Z=∑x∈R(x)Z= _x R(x) and Zk=Z⋅hk(so)Z_k=Z· h_k(s_o) and q(k|x)=gk(x)q(k|x)=g_k(x). In simpler terms, Proposition E.4 highlights SAL can be cast into our mixture learning framework by reframing hk(so)h_k(s_o) and gk(x)g_k(x) as our prior and posterior mixing distributions, an interpretation supported by the normalization property established in Lemma E.3. Notably, the central statements on the correctness of SAL in (Silva et al., 2025b) can be immediately derived from Proposition E.4, highlighting the generality of DI GFlowNets as a theoretical tool for devising GFlowNet training algorithms with proven correctness. The next section emphasizes this point further. Appendix F Revisiting Boosted GFlowNets This section is separated into two parts. The first one shows how Boosted GFlowNets can be understood through the lens of our abstract framework. The second one evaluates Boosted GFlowNets against our proposed SC GFlowNets, highlighting the computational amenability of the latter when compared to the former. F.1 Boosted GFlowNets satisfy the Mixed Trajectory Balance As with SC GFlowNets and SAL, Boosted GFlowNets learn distinct samplers focusing on nearly-disjoint regions of the state space (Locatello et al., 2018; Dall’Antonia et al., 2026b). However, instead of relying on amortization (as in SC GFlowNets) or parallelization (SAL) over fixed structural primitives, Boosted GFlowNets adaptively learn a residual target distribution in a sequential fashion. Their connection to DI GFlowNets, which will be developed in this section, will reveal our framework’s applicability in the context of countably infinite mixtures. To see this, we briefly recall the algorithmic design of Boosted GFlowNets. Let kk≥1≔(Zk,pF(k),pB(k))k≥1\ g_k\_k≥ 1 \(Z_k,p_F [t](k),p_B [t](k))\_k≥ 1 be GFlowNets and, for τ∈so↝xτ∈ s_o x, define R(0)(x,τ)=0 and R(k)(x,τ)=R(k−1)(x,τ)+Z(k)pF(k)(so,τ)pB(k)(x,τ)R [t](0)(x,τ)=0 and R [t](k)(x,τ)=R [t](k-1)(x,τ)+ Z [t](k)p_F [t](k)(s_o,τ)p_B [b](k)(x,τ) for k≥2k≥ 2. Intuitively, R(k)(x,τ)R [t](k)(x,τ) represents an estimate of the accumulated probability mass on x by the first k models. An implicit assumption of Boosted GFlowNets, which is compatible with the observed empirical behavior of GFlowNets (Shen et al., 2023), is that each GFlowNet underestimates the total probability mass in any given region. Assumption F.1 (Underallocation of probability mass). After trained, we assume assume the first k-th GFlowNets satisfy R(k)(x,τ)≤R(x)R [t](k)(x,τ)≤ R(x) for all x and τ∈so↝xτ∈ s_o x. Under Assumption F.1, each GFlowNet is trained by minimizing the boosted trajectory balance (BTB) loss function, which replaces R(x)R(x) by the residual target R(x)−R(K)(x,τ)R(x)-R [t](K)(x,τ) with τ drawn from the corresponding backward policy pB(K)(x,⋅)p_B [b](K)(x,·). Definition F.2 (Boosted Trajectory Balance Condition & Loss). Let kk=1K\ g_k\_k=1^K be a sequence of GFlowNets with kk=1K−1\ g_k\_k=1^K-1 satisfying Assumption F.1. By defining R(k)(x)=τ∼pB(k)(x,⋅)[R(k)(x,τ)],R [t](k)(x)=E_τ p_B [t](k)(x,·) [R [t](k)(x,τ) ], we call K-th Boosted Trajectory Balance (BTB) condition ZK⋅pF(K)(so,τ)=(R(x)−R(K−1)(x))⋅pB(K)(x,τ),Z_K· p_F [t](K)(s_o,τ)=(R(x)-R [t](K-1)(x))· p_B [t](K)(x,τ), which is enforced through the loss function ℒBTB(τ|kk=1K)=(logZK⋅pF(K)(so,τ)(R(x)−R(K−1)(x))⋅pB(K)(x,τ))2,L_BTB (τ |\ g_k\_k=1^K )= ( Z_K· p_F [t](K)(s_o,τ)(R(x)-R [t](K-1)(x))· p_B [b](K)(x,τ) )^2, which we refer to as BTB loss. The intuition for the BTB loss is that, when the the first K−1K-1 GFlowNets collapse onto a subset of the state space, the K-th GFlowNet will—through the influence of the residual target—mostly ignore such region. In practice, (Dall’Antonia et al., 2026b) proposes several techniques for stabilizing training when the quantity R(x)−R(K)(x)R(x)-R [t](K)(x) becomes overly small, to which the interested reader is referred to for further details; however, none of them are relevant to our discussion. It should be clear from Definition F.2 and Assumption F.1 that Boosted GFlowNets only sample from the target distribution when the trajectory-wise residual target vanishes and that the BTB loss can only be non-biasedly evaluated if we can compute R(K−1)(x)R^(K-1)(x) exactly. The latter is satisfied when R(K−1)(x,τ)=R(x)R^(K-1)(x,τ)=R(x) for every τ in the support of pFp_F, and 00 otherwise; equivalently, when the on-policy expectation of the BTB loss under pF(K−1)p_F^(K-1) vanishes. This was the main condition for (Dall’Antonia et al., 2026b, Theorem 1). We explicitly state these behaviors in our next assumption. Assumption F.3 (Convergence of residuals & Collapse). For all x∈x and τ∈so↝xτ∈ s_o x, the series R(k)k≥1\R [t](k)\_k≥ 1 satisfies limk→∞R(k)(x,τ)=R(x) _k→∞R [t](k)(x,τ)=R(x). Also, there is a subset kX_k of X such that R(k)(x,τ)=R(k)(x)R^(k)(x,τ)=R^(k)(x) for each τ∈so↝xτ∈ s_o x and x∈kx _k, and R(k)(x,τ)=0R^(k)(x,τ)=0 otherwise. We now show that Assumptions F.1 and F.3 allow us to characterize Boosted GFlowNets as a solution to the MTB equations in Definition 3.2 based on a countably infinite mixing distribution. Importantly, as with classical topic modelling (Blei et al., 2003; Hoffman et al., 2010) based on Dirichlet processes, we typically choose a K and truncate the prior mixing distribution to be supported on [K][K]. Dall’Antonia et al. (2026b) sets K to 22 or 33 in their experiments. Proposition F.4 (Boosted GFlowNets as DI GFlowNets). We define kk≥1≔(Zk,pF(k),pB(k))k≥1\ g_k\_k≥ 1 \(Z_k,p_F [t](k),p_B [t](k))\_k≥ 1 as GFlowNets satisfying Assumptions F.1 and F.3. By letting Rk(x)=R(k)(x)−R(k−1)(x) and q(k|x)=Rk(x)R(x),R_k(x)=R [t](k)(x)-R [t](k-1)(x) and q(k|x)= R_k(x)R(x), then the following properties are satisfied. 1. q(k|x)q(k|x) is a probability distribution, i.e., ∑k≥1q(k|x)=1 _k≥ 1q(k|x)=1 for each x∈x , 2. The sequence kk≥1\ g_k\_k≥ 1 satisfies a MTB condition Zk⋅pF(k)(so,τ)=R(x)q(k|x)pB(k)(x,τ)Z_k· p_F [t](k)(s_o,τ)=R(x)q(k|x)p_B [t](k)(x,τ) for k≥1k≥ 1, with ∑k≥1Zk=Z _k≥ 1Z_k=Z. Once again, the correctness of Boosted GFlowNets can be directly inferred from Corollary 5.2 and Proposition F.4. However, contrarily to SC GFlowNets, which support both amortization and distributed training, Boosted GFlowNets can only be trained via sequential learning. Also, in contrast to both SC GFlowNets and SAL, Boosted GFlowNets cannot be trained with an near-embarrassingly parallel algorithm, primarily due to the unbounded support of the mixing distribution and the dependence of each model on its predecessors’ residuals. In addition, we notice SC GFlowNets are the most foundational instantiation of DI GFlowNets when compared to both SAL and Boosted GFlowNets: stratum-conditioning could be used for training each booster in Boosted GFlowNets and each sampler in SAL, but neither of these algorithms could be used to learn SC GFlowNets without sacrificing its monolithicness, which we deem to be one of its fundamental properties—a single model can be used for learning the entire mixture. F.2 Experiments We now present several experiments to support our claim that (i) Boosted GFlowNets introduce a substantial computational overhead over the base model, which is not observed for SC GFlowNets, and (i) SC GFlowNets often converge faster than Boosted GFlowNets. (a) Runtimes per training step. (b) FLOPs per training step. Figure 11: SC GFlowNets run both faster (left) and more compute-efficiently (right) than Boosted GFlowNets for the Set Generation domain. Throughout this section, runtimes represent per-step averages over 33 independent runs, each with 600600 iterations. (a) Runtimes per training step. (b) FLOPs per training step. Figure 12: SC GFlowNets are more compute-efficient than Boosted GFlowNets in the Hypergrid domain, requiring substantially fewer FLOPs per gradient step during training. (a) Runtimes per training step. (b) FLOPs per training step. Figure 13: SC GFlowNets is far more efficient than Boosted GFlowNets in the Lazy Random Walk domain (Rings corresponds to the Rings target; GM, to Gaussian Mixture). Computational profile. We start by showcasing the cost of Boosted GFlowNets is substantially larger than of SC GFlowNets in terms of both runtime and FLOPs per training step for the Set Generation, Hypergrid, and Lazy Random Walk domains. To evaluate the latter, we use JAX’s cost_analysis method for compiled functions (Bradbury et al., 2018). Results are displayed in Figures 11, 12 and 13. Notably, SC GFlowNets are up to roughly 1.6×1.6× faster than Boosted GFlowNets per step (for Set Generation), and execute up to 50%50\% fewer FLOPs per training step. The reasons for this are clear: as discussed before, our method learns an amortized policy over the mixture’s components, while boosting requires training independent neural networks in a sequential fashion, having a monotonically increasing cost in the number of boosters (components). Importantly, during our experiments, we proceeded as suggested by Dall’Antonia et al. (2026b) and allocated a prescribed number of iterations equally among a predefined number of boosters for training; automatic sampler elicitation remains an unresolved issue for Boosted GFlowNets. Figure 14: SC GFlowNets converge substantially faster than Boosted GFlowNets in the Set Generation domain, particularly for larger state spaces. See also Figure 4 for ErritemErr_item’s definition. Figure 15: SC GFlowNets exhibit faster learning convergence when compared to Boosted GFlowNets, while also being far more compute-efficient (Figure 12). (a) Rings. (b) Gaussian Mixture. Figure 16: On top of being more computationally efficient (Figure 13), SC GFlowNets also learn a more accurate distributional approximation to the target relatively to Boosted GFlowNets in both the Rings and Gaussian Mixture variants of the Lazy Random Walk domain. Learning convergence. In addition to being more compute-efficient, SC GFlowNets often lead to faster learning convergence in the considered tasks. This is highlighted in Figures 14, 15 and 16. The intuition for this is that, while both Boosted and SC GFlowNets enhance state space exploration by design, hence improving the rate of discovery of high-probability regions, our method does so by stably partitioning the state space, ensuring consistent, non-random exploration of specific subsets of it. In contrast, boosting is built on a more delicate approach that is is reliant upon accurate training of several independent neural networks, and can be biased—unless Assumptions F.1 and F.3 are satisfied. Appendix G Experimental Details & Further Experiments This section extends empirical analysis in Section 6 with additional domains, target distributions, and TC’s constructions. In addition, we provide further experimental details for reproducing our results. We also attach the computer code, written using JAX (Bradbury et al., 2018), we used for training and evaluating each of the described models on the considered tasks. G.1 More Experiments (a) L=18L=18. (b) L=24L=24. Figure 17: SC GFlowNets significantly accelerate learning convergence when approximating the generative process in Equation 15, with greater improvements being observed for larger state spaces. TVitemTV_item and TVlengthTV_length are defined in Equation 17. Autoregressive sequence generation. We train a GFlowNet to reproduce the following generative process of token sequences (t1,…,tl)(t_1,…,t_l) over a finite vocabulary, t1∼pinit,l∼plength,ti|ti−1∼pkernel(ti−1,⋅) for i∈2,…l.t_1 p_init,\,l p_length,\,t_i|t_i-1 p_kernel(t_i-1,·) for i∈\2,… l\. (15) We let [V]≔1,…,V[V] \1,…,V\ be the support of pinitp_init (vocabulary) and pkernel(t,⋅)p_kernel(t,·) for t∈[V]t∈[V] and 0∪[L]\0\∪[L] be that of plengthp_length. The target distribution associated with =(t1,…,tl)t=(t_1,…,t_l) is R()=pinit(t1)plength(l)∏2≤i≤lpkernel(ti−1,ti).R(t)=p_init(t_1)p_length(l) _2≤ i≤ lp_kernel(t_i-1,t_i). (16) The GFlowNet is built upon a generative process starting at an empty sequence and iteratively deciding either to add a token from [V][V] or to stop the generation, and the SC GFlowNet is designed according to Example D.1 with K=3K=3 partitions. Due to its simplicity, many quantities of interest from the model in Equation 16 can be tractably computed, including the marginal distribution over sequence lengths and the value of tit_i for i∈[L]i∈[L]. We denote these by π(l)π(l) and π(i,v)π(i,v) for i∈[L]i∈[L] and v∈[V]v∈[V], respectively, and evaluate our model with TVlength=12∑0≤l≤L|π^(l)−π(l)| and TVitem=12max∑1≤i≤Lv∈[V]|π^(i,v)−π(i,v)|,TV_length= 12 _0≤ l≤ L| π(l)-π(l)| and TV_item= 12 _v∈[V] _1≤ i≤ L| π(i,v)-π(i,v)|, (17) with π π being a Monte Carlo estimate of π based on the learned sampler. We illustrate the evolution of these metrics throughout training in Figure 17. As we can observe, SC GFlowNets significantly improve learning convergence. (a) L=18L=18. (b) L=24L=24. Figure 18: Embarrassingly parallel counterpart of Figure 17, comparing ASC GFlowNet against the best of K randomly initialized and independently trained GFlowNets with matched computational cost. ASC GFlowNet accelerates learning convergence. We also evaluate ASC GFlowNets on this domain, training K=3K=3 separate samplers in parallel on distinct subsets of the state space. We show the results in Figure 18. Again, we notice our partitioning technique significantly speeds up training. Figure 19: Accuracy of SC GFlowNets as a function of the number of components in the mixture. Another question that comes up when designing SC GFlowNets is how to select the number of partitions. Figure 19 shows that, for this domain, there is no significant difference between choosing K∈2,3,4,5,10K∈\2,3,4,5,10\. This may not always be the case, however; see Figure 26. We discuss this further when describing the Lazy Random Walk domain. Importantly, the question of whether one should use ASC or SC GFlowNets depends primarily on the available computational resources. ASC GFlowNets, being massively parallelizable, have access to larger training sets and greater model capacity than any individual SC GFlowNet, being expected to both improve state space exploration and learning convergence, should the extra processing power be available. From an implementational perspective, as the samplers are architecturally identical, this parallelization can be directly attained by using tools designed to express single-program, multiple-data (SPMD) computational models (Darema et al., 1988), such as jax.pmap. Lazy Random Walk. The state space for the Lazy Random Walk domain is =(,t):∈−H,…,Hd and 0≤t<TS=\(x,t) ∈\-H,…,H\^d and 0≤ t<T\ and =(,T):∈−H,…,HdX=\(x,T) ∈\-H,…,H\^d\ for given positive integers H, d, and T. The generative process starts at so=(,0)s_o=(0,0), and each iteration adds either 11, −1-1, or 00 to a chosen coordinate of x—as long as it remains within the boundary of S—and increments t by one until t=Tt=T. As in (Dall’Antonia et al., 2026a; Silva et al., 2026), we consider d=2d=2, H=16H=16, and T=32T=32. Our target distributions, Rings and Gaussian Mixture, are shown in Figure 20. Notably, as the trajectory size is the same for each (,T)(x,T), the length-based partitioning of Example D.1 cannot be used for Lazy Random Walk. Instead, we propose a uniform distance-based partitioning using the function ψ(,T)=‖1ψ(x,T)=\|x\|_1 in Remark 5.5 with K=3K=3. We compute the TV distance between the learned and target distributions to measure a sampler’s goodness-of-fit. (a) Rings. (b) Gaussian Mixture. Figure 20: SC GFlowNets improve goodness-of-fit for the Lazy Random Walk task, exhaustively covering the target distribution’s high probability regions even for an imperfect state space partitioning (as for the Gaussian Mixture; see Figure 21). (a) Rings. (b) Gaussian Mixture. Figure 21: SC GFlowNet’s marginal mF(k)(so,x)m_F [t](k)(s_o,x) (recall Equation 1) in each x∈x for both the (a) Rings and (b) Gaussian Mixture target distributions of the Lazy Random Walk domain. Figure 20 shows both the model’s accuracy throughout training and a snapshot of the learned distribution for both SC and a traditional GFlowNet. In addition, we present in Figure 21 each component pF(k)p_F [t](k) for k∈1,2,3k∈\1,2,3\ of the SC GFlowNet. We notice that our mixture modelling framework both accelerates learning convergence and improves fitness—even when the chosen partitioning does not separate the target distribution’s high probability regions, as in the Gaussian Mixture example. The intuition for this is that the sampler will naturally avoid being pulled back to and bogged in previously visited modes during training when we constraint the future value of ‖1\|x\|_1 at t=Tt=T. We also compare ASC to SC GFlowNets in Figure 24 for both examples; as expected, ASC GFlowNets find a distinctly better fit to the target—particularly in the case of Rings, for which each component receives an easier target with connected high-probability regions. (a) Rings. (b) Rings. (c) Gaussian Mixture. (d) Gaussian Mixture. Figure 22: Learned log-partition functions by ASC GFlowNets in the Lazy Random Walk domain. We recall that, for Rings, k=3k=3 is the starved component, containing negligible probability mass; for Gaussian Mixture, it is k=1k=1. As expected, ASC GFlowNets learn the correct values (a, c); however, the approximation of logZk Z_k for regions with near-zero probability is imperfect (b, d), as sizeable deviations in logZk Z_k in this case have little effect on distributional accuracy. (a) Rings. (b) Gaussian Mixture. Figure 23: KL between prior mixing distributions. Importantly, due to having a tractably enumerable state space, the Lazy Random Walk domain allows us to efficiently evaluate the log-partitions functions. This provides another road for evaluating our model. To see this, we denote by logZ^k Z_k and logZ Z the learned partition functions, and by logZk Z_k and logZ Z their ground-truth values for k∈[K]k∈[K]. We consider the metrics KL[pM||p^M]≔∑1≤k≤KZkZ⋅logZk/Z^k/Z^,(ZkZ⋅|logZ^k−logZk|), and (|logZ^k−logZk|),KL[p_M|| p_M] _1≤ k≤ K Z_kZ· Z_kZ Z_k Z,\, ( Z_kZ· | Z_k- Z_k | ), and ( | Z_k- Z_k | ), for k∈[K]k∈[K], which respectively evaluate the accuracy of the learned prior mixing distribution and the relative and absolute errors of the learned partition functions. We separate relative and absolute metrics as, in practice, if a ZkZ_k is negligible when compared to other Zk′Z_k , small discrepancies in ZkZ_k will result in large deviations for logZ^k Z_k. As an example, the distributions with logits (1,2,−10)(1,2,-10) and (1.05,1.99,−15)(1.05,1.99,-15) are effectively the same, despite the large difference in the values of the third component. We display these values in Figures 22 and 23, with a comparison between SC and ASC GFlowNets according to them in the latter. (a) Rings. (b) Gaussian Mixture. Figure 24: ASC GFlowNets converge faster than their synchronous counterparts in the Lazy Random Walk task, achieving a more accurate approximation in fewer training iterations. (a) Log-predictive posterior density of the observed data (LPPD). (b) Average unnormalized target density of the 10 most likely encountered states. Figure 25: SC GFlowNets improve learning convergence (left) and accelerate the discovery of high-probabiltiy states (right) for the Bayesian variable selection task; H represents the number of variables (columns) in the dataset. (a) Rings. (b) Four Gaussians. Figure 26: SC GFlowNets’ distributional accuracy for distinct numbers of partitions. We once again consider the problem of choosing K. Figure 26 highlights the dependence of SC GFlowNets’s accuracy on K for the Lazy Random Walk domain. Although SC GFlowNets improve upon a monolithic sampler for all evaluated choices of K≥2K≥ 2, there is also a noticeable difference between them. In view of this, we provide two separate suggestions for selecting this hyperparameter. The first is to always let K=3K=3 components, should this choice be compatible with the chosen TC; despite its simplicity, we found setting K=3K=3 to consistently achieve an accurate approximation to the target distributions we considered. The second is to rely upon standard hyperparameter selection protocols, such as carrying out short training runs and measuring the sampler’s performance afterwards, to pick K. With this in mind, future research should investigate principled approaches for algorithmic picking of K. Nonetheless, a useful intuition to have is that the complexity of the function learned by our component-wise amortized policy increases with K, due to which we advise against setting too large a value for the number of components in SC GFlowNets. The main takeaway from these experiments, however, is that designing an effective covering for SC GFlowNets does not require fine-grained prior knowledge about the shape of the target distribution. (a) Length-based TC. (b) Octant-based TC. Figure 27: Component-wise learned distributions for both considered SC GFlowNets in Figure 5. Variable Selection. We consider the task of Bayesian variable selection (George and McCulloch, 1993). Given a data set =(x(i),y(i))i=1nD=\(x [t](i),y [t](i))\_i=1^n with x(i)∈ℝHx [t](i) ^H and y(i)∈ℝy [t](i) , we let p(y|x,S,β)=(y|βSTxS,σ2)p(y|x,S,β)=N(y| _S^Tx_S,σ^2) be defined according to a linear model with observational noise σ and coefficients β∈ℝHβ ^H and features S⊆[H]S [H]. We here denote by xSx_S the vector (x(i))i∈S(x [t](i))_i∈ S composed of the indices i in S, with a similar definitiong for βS _S. In this setting, we train a sampler to draw from the marginal posterior distribution over S, i.e., R(S)≔p(S|)=∫ℝHp(S,β|)β,R(S) p(S|D)= _R^Hp(S,β|D)dβ, with p(S,β|)p(S,β|D) being the usual Bayesian posterior for S and β. We notice p(S|)p(S|D) can be computed in closed form up to a normalizing constant independent of S for a linear model. After training, we evaluate the samplers according to the predictive posterior distribution of the observed data, as it is usually done in Bayesian modelling, and to the average unnormalized log-posterior of the 10 most likely variable sets, a common approach for assessing exploration effectiveness in the GFlowNet literature, e.g., (Kim et al., 2024; Kim et al., 2023; Kim et al., 2025b). We show both metrics in Figure 25, with the SC GFlowNet being designed as in the Set Generation domain. Notably, our mixture model improves both learning and the rate at which modes are encountered during training. G.2 Experimental Details We provide further details for how each domain was implemented, including model architecture, mixture-encoding details, optimizer, training iterations, and batch size per gradient step. We ran all our experiments in a MacBook Pro M4 with 16 GB of RAM, and we implemented our models using Jax’s Flax library (Bradbury et al., 2018; Heek et al., 2026). All reported results, including error bars, represent averages and standard deviations computed over three independent runs. In every experiment, we used the Muon optimizer (Jordan et al., 2024) for the neural network’s parameters, and Adam (Kingma and Ba, 2014) for the logZ Z and logZkk∈[K]\ Z_k\_k∈[K] parameters. Unless stated otherwise, we parameterized the policies pFp_F and pBp_B of the GFlowNet using an MLP with two 64-dimensional layers, pF(s,⋅)=Softmax(MLP(ι(s))⊙(s)),p_F(s,·)=Softmax (MLP( (s)) (s) ), with ι(s):→ℝ64 (s) ^64 being a learned vector embedding for the state, and (s)∈1,−∞fm(s)∈\1,-∞\^f being masking function that ensures the sampler is constrained to the target region (either X or kX_k, in the case of SC GFlowNets); f is the maximum number of children a given state s has in the underlying state graph. We implement a similar representation for pBp_B, using a shared encoder and simply changing the output layer and the mask. In its centralized form, SC GFlowNets are implemented as pF(γ)(s,⋅)=Softmax(MLP(ι(s)⊕ιγ(γ))⊙(s,γ)).p_F^(γ)(s,·)=Softmax(MLP( (s) _γ(γ)) (s,γ)). (18) Again, (s,γ)m(s,γ) is a masking function, which depends on γ in the case of SC GFlowNets but not in the case of RF GFlowNets; ⊕ represents concatenation. To ensure matching computational costs and similar parameter counts for both SC GFlowNets and GFlowNets, we let ι:→ℝ64−dγ ^64-d_γ and ιγ:Γ→ℝdγ _γ ^d_γ, with dγd_γ being the dimension of the learned representation for γ. For SC GFlowNets, ιγ _γ is a learned dictionary from [K][K] to ℝdγR^d_γ. For RF GFlowNets, ιγ _γ is a linear layer. We set dγ=8d_γ=8 in all experiments, which we found to be consistently effective. ASC GFlowNets, on the other hand, was trained with K identical copies of Equation 18; as each model learns to sample from a distinct kX_k, information about k was not required to be explicitly encoded into the neural network. Set generation. We considered sets of size 24,48,64\24,48,64\, and trained each sampler for 30003000, 50005000, 1000010000 iterations, respectively, using a batch size of 6464 trajectories. Hypergrid. We trained the models for H∈16,32H∈\16,32\, both of which over 1600016000 iterations and a batch size of 6464 trajectories per step. The reward function is the same as in (Malkin et al., 2022), i.e., logR()=log(12∏i=1d(ai>0.5)+2∏i=1d(0.6<ai<0.8)+ρ), R(x)= ( 12 _i=1^d 1(a_i>0.5)+2 _i=1^dI(0.6<a_i<0.8)+ρ ), (19) with ρ=10−3ρ=10^-3 and ai=2⋅i⋅(H−1)−1−1a_i=2·x_i·(H-1)^-1-1. Ancestral Graphs. We considered two distinct tasks under the Ancestral Graphs domain. The first, considered also in (da Silva et al., 2026), considered graphs with H∈15,20H∈\15,20\ nodes and used the BIC for scoring each AG, R(x)=−BIC(x)=2logp^(|x)−|x|Elogn−2|x|Elog|x|N,R(x)=-BIC(x)=2 p(D|x)-|x|_E n-2|x|_E |x|_N, (20) in which p p is the maximum likelihood of D given x, computed via Drton and Richardson (2004)’s Iteratively Reweighted Least Squares (IRLS) algorithm, |x|N|x|_N is the number of nodes (H) in x, |x|E|x|_E, the number of edges, and D was drawn from a randomly parameterized linear Gaussian structural equation model with H variables. We used the same data generative process as in (da Silva et al., 2026). The second task consisted of simulating the Erdős Renyi model, whose unnormalized density is defined as R(x)=α|xE|.R(x)=α^|x_E|. We restrict the GFlowNet’s generative process to directed acyclic graphs (DAGs) by masking out bidirected edges. Under this condition, both the partition function ZH(α)Z_H(α) of R(x)R(x) and the probability that there is an edge between each pair of nodes (the same for all of them, due to symmetry) can be computed in closed form and compared against that learned by our sampler. We notice that these quantities are actually related, and both of them can be computed efficiently by adapting Robinson’s formula (Robinson, 1973), which we quickly review. Fix any pair (a,b)(a,b) of nodes. Let HX_H be the space of all DAGs with H nodes. The probability that there is an edge from a to b in the graph is EH(α)=∑x∈,(a→b) in xα|xE|ZH(α) with ZH(α)=∑x∈Hα|xE|.E_H(α)= _x ,(a→ b) in xα^|x_E|Z_H(α) with Z_H(α)= _x _Hα^|x_E|. To compute the latter, pick any subset of size d of nodes, which will be deemed roots of the DAG, for d∈1,…,Hd∈\1,…,H\; in a DAG, there is always at least one. There are (Hd) Hd ways of choosing them. From them, we may pick each of the H−dH-d remaining nodes to decide whether there is an edge between them and the root nodes, totaling a maximum of d(H−d)d(H-d) edges. Given this choice, the remaining H−dH-d nodes may form any DAG. In our summation, the corresponding term is m(d,α)=(Hd)⋅ZH−d(α)⋅∑0≤k≤d(H−d)(d(H−d)k)αk=(Hd)⋅ZH−d(α)⋅(1+α)d(H−d).m(d,α)= Hd· Z_H-d(α)· _0≤ k≤ d(H-d) d(H-d)kα^k= Hd· Z_H-d(α)·(1+α)^d(H-d). However, summing this expression over d∈1,…,Hd∈\1,…,H\ over-counts certain graphs; e.g., the edgeless graph is counted several times when k=0k=0 for each d. More comprehensively, let RH,dR_H,d be the set of all DAGs having exactly d nodes as roots, and let c(d,α)=∑x∈RH,dα|x|Ec(d,α)= _x∈ R_H,dα^|x|_E; our objective is to compute ∑1≤d≤Hc(d,α) _1≤ d≤ Hc(d,α). However, the expression above evaluates m(d,α)=∑d≤j≤H(jd)⋅c(j,α),m(d,α)= _d≤ j≤ H jd· c(j,α), since RH,jR_H,j appears (jd) jd times when evaluating m(d,α)m(d,α); to see this, fix a set n=n1,…,njn=\n_1,…,n_j\ of j nodes, and notice that our computation above considers the case in which every n∈n is root each time we pick any d-sized subset of nn, i.e., (jd) jd times, for d≤jd≤ j, as our computation considers DAGs with at least d roots. We can circumvent this issue by instead computing ∑1≤d≤H(−1)d+1m(d,α) _1≤ d≤ H(-1)^d+1m(d,α) =∑1≤d≤H∑d≤j≤H(−1)d+1⋅(jd)⋅c(j,α) = _1≤ d≤ H _d≤ j≤ H(-1)^d+1· jd· c(j,α) =∑1≤j≤Hc(j,α)⋅∑1≤d≤j(−1)d+1⋅(jd) = _1≤ j≤ Hc(j,α)· _1≤ d≤ j(-1)^d+1· jd =∑1≤j≤Hc(j,α)⋅(1−∑0≤d≤j(−1)d⋅(jd)) = _1≤ j≤ Hc(j,α)· (1- _0≤ d≤ j(-1)^d· jd ) =∑1≤j≤Hc(j,α)(1−(1−1)j)=∑1≤j≤Hc(j,α). = _1≤ j≤ Hc(j,α) (1-(1-1)^j )= _1≤ j≤ Hc(j,α). In conclusion, ZH(α)=∑1≤d≤H(−1)d+1⋅(Hd)⋅ZH−d(α)⋅(1+α)d(H−d).Z_H(α)= _1≤ d≤ H(-1)^d+1· Hd· Z_H-d(α)·(1+α)^d(H-d). This can be computed recursively via the boundary condition Z1(α)=1Z_1(α)=1. To evaluate EH(α)E_H(α), we notice the probability of existing an edge from a to b is equal, by symmetry, to the expected number of edges divided by the total number of edges. On the other hand, the number of edges follows the exponential family |x|E↦exp|x|Elogα−logZH(α)|x|_E \|x|_E α- Z_H(α) \ with cumulant logZH(α) Z_H(α); as a consequence, its expected value can be computed as the derivative of the cumulant, x[|x|E]=dlogZH(α)dlogα=α⋅dlogZH(α)dα,E_x[|x|_E]= d Z_H(α)d α=α· d Z_H(α)dα, which can be computed efficiently through our previously discussed recursive formula. The probability of there being an edge between any pair of nodes under Erdős-Renyi’s model is then EH(α)=1H(H−1)⋅α⋅dlogZH(α)dα.E_H(α)= 1H(H-1)·α· d Z_H(α)dα. During training, we compare EH(α)E_H(α) with the edge-wise probabilities learned by our sampler, estimated via Monte Carlo with 20482048 samples; the average L1L_1 between both quantities is the value reported in Figure 6, called MarginalEdgeErrorMarginal\,Edge\,Error and evaluated as MarginalEdgeError=2H(H−1)∑n1,n2∈[H],n1≠n2|EH(α)−π^(n1,n2)|Marginal\,Edge\,Error= 2H(H-1) _n_1,n_2∈[H],n_1≠ n_2|E_H(α)- π(n_1,n_2)| with π^(n1,n2) π(n_1,n_2) being the referred Monte Carlo estimate. Autoregressive sequence generation. We set plengthp_length to an uniform distribution and both pinitp_init and pkernelp_kernel to randomly sampled distribution with logits drawn according a standard multivariate normal model of dimension L∈18,24L∈\18,24\. We train the samplers for 80008000 iterations in both cases. Lazy Random Walk. We consider the follow distributions for the Rings and Gaussian Mixture targets, respectively, logRRings()=log(∑k=1Kwkexp(−12(‖2−rks)2)), R_Rings(x)= ( _k=1^Kw_k (- 12 ( \|x\|_2-r_ks )^\!2 ) ), with K=2K=2, r1=0.2r_1=0.2, r2=0.8r_2=0.8, w1=0.6w_1=0.6, w2=0.4w_2=0.4, and s=0.05s=0.05, and logRGaussianMixture()=[log(∑j=14exp(−12(‖−j‖22σ)2))−log(4πσ2π)], R_ GaussianMixture(x)= [ ( _j=1^4 (- 12 ( \|x- μ_j\|_2^2σ )^\!2 ) )- (4πσ 2π ) ], with σ=0.1σ=0.1 and μ1=(−0.8,−0.8) _1=(-0.8,-0.8), μ2=(−0.8,0.8) _2=(-0.8,0.8), μ3=(0.8,−0.8) _3=(0.8,-0.8), and μ4=(0.8,0.8) _4=(0.8,0.8). In both cases, we normalize x to [−1,1][-1,1] through the transformation ↦H−1x H^-1x. Variable Selection. We consider H∈24,48H∈\24,48\ and train both models for 50005000 and 1000010000 iterations, respectively, and we used a batch size of 6464 trajectories per training step. Appendix H Proofs We provide detailed proofs for each theoretical result in both the main text and the supplement. H.1 Proof of Proposition 3.3 As each γ g_γ abides by the MTB condition, we observe that Z(dγ)⋅pF(γ)(so,τ)=R(x,dγ)⋅pB(γ)(x,τ)Z(dγ)· p_F [t](γ)(s_o,τ)=R(x,dγ)· p_B [t](γ)(x,τ) for each γ∈Γγ∈ and trajectory τ from sos_o to x. We then infer that, by marginalizing τ∈so↝xτ∈ s_o x, Z(dγ)∑τ∈so↝xpF(so,τ)=R(x,dγ)∑τ∈so↝xpB(γ)(x,τ)⏟=1=R(x,dγ).Z(dγ) _τ∈ s_o xp_F(s_o,τ)=R(x,dγ) _τ∈ s_o xp_B [t](γ)(x,τ)_=1=R(x,dγ). In addition, the left-hand side of the above equation is exactly mF(γ)(x)m_F [t](γ)(x) (by definition). Hence, Z(dγ)mF(γ)(x)=R(x,dγ).Z(dγ)m_F [t](γ)(x)=R(x,dγ). This shows the second equality in Proposition 3.3. By summing the above equation over x∈x and noticing that mF(γ)m_F [t](γ) is a probability measure on X, we obtain Z(dγ)=Z(dγ)∑x∈mF(γ)(x)⏟=1=∑x∈R(x,dγ).Z(dγ)=Z(dγ) _x m_F [t](γ)(x)_=1= _x R(x,dγ). This demonstrates the first equality in Proposition 3.3. H.2 Proof of Corollary 5.2 Under the notation of Proposition 3.3, we observe that Rk(x)=R(x,k)=R(x)qk(x)=R(x)N(x)−1R_k(x)=R(x,\k\)=R(x)q_k(x)=R(x)N(x)^-1 if x∈kx _k, and R(x,k)=0R(x,\k\)=0 otherwise. This proves the second equality in Corollary 5.2. The first equality can be derived by recalling that Zk≔∑x∈kRk(x)Z_k _x _kR_k(x) and hence Zk=∑x∈kR(x)N(x).Z_k= _x _k R(x)N(x). This is exactly the first equality in Corollary 5.2. H.3 Proof of Proposition 5.6 We demonstrate there are numbers ak,bkk∈[K]\a_k,b_k\_k∈[K] and a function ψ for which the problem ∃s′∈2∖ss.t. s∪s′∈ and ψ(s∪s′)∈[ak,bk)∃ s ∈ 2^C ss.t. s∪ s and ψ(s∪ s )∈[a_k,b_k) is equivalent to SubsetSum. We assume =[C]C=[C] for some C∈ℕC . We first notice that since ψ is modular, i.e., ψ(s∪c)=ψ(s)+ψ(c)ψ(s∪\c\)=ψ(s)+ψ(\c\) for each c∈∖sc s, we may write ψ(s)=ψ(∅)+∑c∈sψ(c).ψ(s)=ψ( )+ _c∈ sψ(\c\). As such, we define ηc=ψ(c)≥0 _c=ψ(\c\)≥ 0 for conciseness and assume ψ(∅)=0ψ( )=0 without loss of generality. In this case, ψ(s)ψ(s) can be computed in linear time in the size of s. We also assume ηc _c are positive integers and fix s=∅s= . Under these conditions, our decision problem in Equation 6 becomes ∃s∈2:∑c∈sηc∈[ak,bk).∃ s∈ 2^C _c∈ s _c∈[a_k,b_k). Let there be a k for which bk=ak+1b_k=a_k+1 with both aka_k and bkb_k positive integers; as ηc _c are also integers, the above sum can only be satisfied when ∑c∈sηc=ak _c∈ s _c=a_k for some subset s of C. This is exactly the subset sum problem (with positive inputs), which is NP-complete (Cormen et al., 2009). In particular, deciding upon the reachability of kX_k from a given s∈s is NP-complete in general. H.4 Proof of Proposition 5.7 We show the decision problem in Equation 6 can be efficiently solved when ψ is a symmetric junta by relying upon the fact that, since C is finite, there are only so many values realizable by ψ, all of which can be quickly verified in polynomial time. Fix any s∈s with n=#(s∩r)n=\#(s∩ r). We notice n≤#(∩r)n≤\#(C∩ r). Also, ψ(s)=ξ(n)ψ(s)=ξ(n) by definition. Our decision problem then becomes ∃s′∈2∖s:ψ(s∪s′)∈[ak,bk).∃ s ∈ 2^C s ψ(s∪ s )∈[a_k,b_k). As s and s′s are disjoint, #((s∪s′)∩r)=#(s∩r)+#(s′∩r)\#((s∪ s )∩ r)=\#(s∩ r)+\#(s ∩ r). Similarly, #(s′∩r)≤#(∩r)−n\#(s ∩ r)≤\#(C∩ r)-n. And not only this, but #(s′∩r)\#(s ∩ r) can also assume any value within 0,…,#(∩r)−n\0,…,\#(C∩ r)-n\ as s′∈2∖s ∈ 2^C s. As such, deciding whether the above problem is solvable is equivalent to deciding whether there is a m∈0,…,#(∩r)−nm∈\0,…,\#(C∩ r)-n\ for which ξ(n+m)∈[ak,bk)ξ(n+m)∈[a_k,b_k) which can (by definition) be executed efficiently as ψ is assumed to be efficiently evaluable in polynomial time. To see that the result remains true if we replace 22^C by the set of bounded multisubsets of C, we define ℳ(B)=c1,…,ck:c1,…,ck∈ and k≤BM(B)=\\\!\c_1,…,c_k\\!\ c_1,…,c_k and k≤ B\ as the set of multisubsets of C of size up to B. We emphasize the operator s↦#s \#s counts repeated entries as distinct in s. We assume B is polynomial on #\#C. In this case, the above argument applies by replacing 22^C with ℳ(B)M(B) and noticing that it is enough to verify whether, given s∈s , there is a m∈0,…,B−#(s∩r)m∈\0,…,B-\#(s∩ r)\ for which ξ(#(s∩r)+m)∈[ak,bk)ξ(\#(s∩ r)+m)∈[a_k,b_k), which can again be done efficiently. H.5 Proof of Proposition 5.9 Let St(f)t≥1\S_t [t](f)\_t≥ 1 be the Markov chain defined by κF _F, and let L be the maximum trajectory length in the GFlowNet’s state graph. For conciseness, we will denote by ℙκF[⋅]P_ _F[·] the probability measure corresponding to the case in which the forward Markov chain starts sos_o, i.e., ℙκF[⋅|so]P_ _F[·|s_o]. It is clear that T≤LT_X≤ L and, hence, ℙκF[T<∞|s]=1P_ _F[T_X<∞|s]=1 for any s. We now proceed to show that ST(f)S_T_X [t](f) is distributed according to the target R(x)R(x). We start by noticing by ℙκF[ST(f)=x]=∑1≤t≤LℙκF[T=t]ℙκF[ST(f)=x|T=t].P_ _F[S_T_X [t](f)=x]= _1≤ t≤ LP_ _F[T_X=t]P_ _F [S_T_X [t](f)=x|T_X=t ]. and, by the definition of conditional probabilities, ℙκF[ST(f)=x|T=t]=∑(s1,…,st−1,x)∈S¯tκF(st−1,x)⋅∏1≤i≤t−2κF(si,si+1)∑x′∈∑(s1,…,st−1,x′)∈S¯tκF(st−1,x′)⋅∏1≤i≤t−2κF(si,si+1).P_ _F [S_T_X [t](f)=x|T_X=t ]= _(s_1,…,s_t-1,x)∈ S^t _F(s_t-1,x)· _1≤ i≤ t-2 _F(s_i,s_i+1) _x _(s_1,…,s_t-1,x )∈ S^t _F(s_t-1,x )· _1≤ i≤ t-2 _F(s_i,s_i+1). This equation follows from the definition of T_X and the Markov property. In addition, if there is a (si,si+1)(s_i,s_i+1) for which the transition si→si+1s_i→ s_i+1 is not in the underlying GFlowNet’s state graph, the probability of (so,…,st−1,x)(s_o,…,s_t-1,x) is zero by design (as κF _F matches pFp_F); hence, the above ratio can be written as ∑τ∈so↝x,|τ|=tpF(so,τ)∑x′∈∑τ′∈so↝x′,|τ′|=tpF(so,τ′), _τ∈ s_o x,|τ|=tp_F(s_o,τ) _x _τ ∈ s_o x ,|τ |=tp_F(s_o,τ ), with |τ|=t|τ|=t stating that the length (number of states) of the trajectory τ is t. A similar reasoning shows that ℙκF[T=t] _ _F[T_X=t] =∑x′∈∑τ∈so↝x′,|τ|=tpF(so,τ). = _x _τ∈ s_o x ,|τ|=tp_F(s_o,τ). Thus, ℙκF[ST(f)=x]=∑1≤t≤L∑τ∈so↝x,|τ|=tpF(so,τ)=∑τ∈so↝xpF(so,τ)=mF(x).P_ _F[S_T_X [t](f)=x]= _1≤ t≤ L _τ∈ s_o x,|τ|=tp_F(s_o,τ)= _τ∈ s_o xp_F(s_o,τ)=m_F(x). When the GFlowNet abides a balance condition, this implies ℙκF[ST(f)=x]=R(x)∑x∈R(x),P_ _F [S_T_X [t](f)=x ]= R(x) _x R(x), as stated by Proposition 5.9. Correspondingly, we observe that ℙκB[Tso<∞|x]=1P_ _B [T_\s_o\<∞|x ]=1 since each trajectory starting at x must eventually reach sos_o, and the maximum distance between sos_o and any x∈x is L. This shows the second part of Proposition 5.9. H.6 Proof of Lemma 5.10 We show the function hkh_k defined as hk(s)=ℙκF[Tk=mink′Tk′|s]h_k(s)=P_ _F [T_k= _k T_k |s ] for s∈s is harmonic. For this, we observe that s′∼κF(s,⋅)[hk(s′)] _s _F(s,·) [h_k(s ) ] =∑s′∈¯κF(s,s′)ℙκF[Tk=mink′Tk′|s′] = _s ∈ S _F(s,s )P_ _F [T_k= _k T_k |s ] =∑s′∈¯κF(s,s′)ℙκF[Tk<∞|s′] = _s ∈ S _F(s,s )P_ _F [T_k<∞|s ] =ℙκF[Tk<∞|s]=ℙκF[Tk=mink′Tk′|s]=hk(s). =P_ _F[T_k<∞|s]=P_ _F [T_k= _k T_k |s ]=h_k(s). The first equation follows from definition. The second and fourth equations are a consequence of the fact that kX_k is an absorbing set of the Markov chain dictated by κF _F and hence that kX_k being arrived at before k′X_k for k≠k′k≠ k implies that TkT_k is the only finite hitting time among Tkk∈[K]\T_k\_k∈[K]. The third equation follows from the law of total probability, and the fifth equation follows from the definition of hkh_k. This demonstrates hkh_k is an harmonic function for k∈[K]k∈[K]. The fact that hk(x)=1h_k(x)=1 for x∈kx _k and hk(x)=0h_k(x)=0 for x∈∖kx _k follows from the definition of hkh_k as the probability of the Markov process reaching at kX_k before any k′X_k for k′≠k ≠ k. Similarly, hk(so)h_k(s_o) is the probability that κF _F arrives at kX_k before arriving at any other k′X_k for k′≠k ≠ k; as the underlying GFlowNet is assumed to abide by any of its balance conditions, the probability of the former is proportional to Zk=∑x∈kR(x)Z_k= _x _kR(x), and of the latter, proportional to Z=∑x∈R(x)Z= _x R(x). This shows hk(so)=∑x∈kR(x)∑x∈R(x)=ZkZ.h_k(s_o)= _x _kR(x) _x R(x)= Z_kZ. We also provide a direct derivation of this fact in Section H.7. H.7 Proof of Proposition 5.11 We show that the marginal distribution of the transformed kernel κF(k) _F [t](k), as defined in Proposition 5.9, matches the restriction of R(x)R(x) to kX_k up to a normalizing constant. As before, we denote by ℙκF[⋅]P_ _F[·] the probability measure ℙκF[⋅|so]P_ _F[·|s_o] corresponding to the scenario in which the forward Markov chain starts at sos_o. We first notice that, since hkh_k is harmonic, ∑s′∈¯κF(k)(s,s′)=∑s′∈¯κF(s,s′)⋅hk(s′)hk(s)=hk(s)hk(s)=1. _s ∈ S _F [t](k)(s,s )= _s ∈ S _F(s,s )· h_k(s )h_k(s)= h_k(s)h_k(s)=1. We then demonstrate that, under κF(k) _F [t](k), the probability of any path that arrives at mX_m before kX_k for a m≠km≠ k is zero. To see this, let (so,s1…,st−1,x)(s_o,s_1…,s_t-1,x) for x∉kx _k; then, hk(x)=0h_k(x)=0 and there is a smallest index i≥1i≥ 1 for which hk(si)=0h_k(s_i)=0. (By the definition of κF _F, hk(so)>0h_k(s_o)>0 for each k∈[K]k∈[K]). The transition from si−1s_i-1 to sis_i under κF(k) _F [t](k) thus has probability zero. This shows that, under κF(k) _F [t](k), the Markov chain arrives at kX_k before any mX_m for m≠km≠ k with probability one. In particular, dropping the superscript (k)(k) from TmT_m for clarity, ℙκF(k)[Tk<∞=Tm]=1.P_ _F [t](k) [T_k<∞=T_m ]=1. This is Proposition 5.11’s first statement. To show that ℙκF(k)[STk(k)=x]=R(x)∑x∈kR(x),P_ _F [t](k) [S_T_k [t](k)=x ]= R(x) _x _kR(x), we proceed as in the demonstration of Proposition 5.9. Since ℙκF(k)[STk(k)=x] _ _F [t](k) [S_T_k [t](k)=x ] =∑1≤t≤LℙκF(k)[Tk=t]ℙκF(k)[STk(k)=x|Tk=t] = _1≤ t≤ LP_ _F [t](k)[T_k=t]P_ _F [t](k) [S_T_k [t](k)=x|T_k=t ] =∑1≤t≤LℙκF(k)[Tk=t]∑τ∈so↝x,|τ|=tκF(k)(so,τ)∑x′∈k∑τ′∈so↝x′,|τ′|=tκF(k)(so,τ′), = _1≤ t≤ LP_ _F [t](k) [T_k=t ] _τ∈ s_o x,|τ|=t _F [t](k)(s_o,τ) _x _k _τ ∈ s_o x ,|τ |=t _F [t](k)(s_o,τ ), in which we define κF(k)(so,τ)=∏(s,s′)∈τκF(k)(s,s′) _F [t](k)(s_o,τ)= _(s,s )∈τ _F [t](k)(s,s ), and hk(x)=1h_k(x)=1 and hk(so)>0h_k(s_o)>0 for each k∈[K]k∈[K] and x∈kx _k, a telescoping product shows that ∏(s,s′)∈τκF(k)(s,s′)=1hk(so)∏(s,s′)∈τκF(s,s′)=1hk(so)⋅κF(so,τ)=1hk(so)⋅pF(so,τ) _(s,s )∈τ _F [t](k)(s,s )= 1h_k(s_o) _(s,s )∈τ _F(s,s )= 1h_k(s_o)· _F(s_o,τ)= 1h_k(s_o)· p_F(s_o,τ) for any τ∈so↝xτ∈ s_o x with x∈kx _k. As a consequence, ∑τ∈so↝x,|τ|=tκF(k)(so,τ)∑x′∈k∑τ′∈so↝x′,|τ′|=tκF(k)(so,τ′)=∑τ∈so↝x,|τ|=tpF(so,τ)∑x′∈k∑τ′∈so↝x′,|τ′|=tpF(so,τ′). _τ∈ s_o x,|τ|=t _F [t](k)(s_o,τ) _x _k _τ ∈ s_o x ,|τ |=t _F [t](k)(s_o,τ )= _τ∈ s_o x,|τ|=tp_F(s_o,τ) _x _k _τ ∈ s_o x ,|τ |=tp_F(s_o,τ ). The rest proceeds as in Proposition 5.9. In fact, as ℙκF(k)[Tk=t]=∑x∈k∑τ∈so↝xκF(k)(so,τ)=1hk(so)∑x∈k∑τ∈so↝xpF(so,τ)P_ _F [t](k) [T_k=t ]= _x _k _τ∈ s_o x _F [t](k)(s_o,τ)= 1h_k(s_o) _x _k _τ∈ s_o xp_F(s_o,τ) and hk(so)=ℙκF[Tk<∞|so]=∑x∈x∑τ∈so↝xpF(so,τ),h_k(s_o)=P_ _F [T_k<∞|s_o ]= _x _x _τ∈ s_o xp_F(s_o,τ), we deduce that ℙκF(k)[STk(k)=x]=∑1≤t≤L∑τ∈so↝x,|τ|=tpF(so,τ)∑x∈x∑τ∈so↝xpF(so,τ).P_ _F [t](k) [S_T_k [t](k)=x ]= _1≤ t≤ L _τ∈ s_o x,|τ|=tp_F(s_o,τ) _x _x _τ∈ s_o xp_F(s_o,τ). When the GFlowNet abides by a balance condition, this expression can be rewritten as ℙκF(k)[STk(k)=x]=R(x)∑x∈kR(x).P_ _F [t](k) [S_T_k [t](k)=x ]= R(x) _x _kR(x). This demonstrates the second statement in Proposition 5.11. H.8 Proof of Proposition B.1 We show that, if our mixture model satisfies the defined SubTB condition, then it samples from the target distribution. We then show write F(s,dγ)F(s,dγ) in terms of the well-known quantities Z(dγ)Z(dγ) and R(x,dγ)R(x,dγ). The first statement can be derived directly from the fact that, since the condition is satisfied for all trajectories τ between reachable pairs s,s′s,s , it is in particular satisfied by complete trajectories going from sos_o to x∈x . This implies the trajectory balance is satisfied and, by (Malkin et al., 2022, Theorem 1), the marginal distribution of pM(dγ)pF(γ)p_M(dγ)p_F^(γ) over x matches R(x,dγ)R(x,dγ). By integrating γ out, we conclude that Equation 2 is satisfied. To see that Z(dγ)≔∑x∈R(x,dγ)Z(dγ) _x R(x,dγ), we notice that F(so,dγ)pF(γ)(so,τ)=R(x,dγ)pB(γ)(x,τ)F(s_o,dγ)p_F^(γ)(s_o,τ)=R(x,dγ)p_B^(γ)(x,τ) for all x∈x and all trajectories τ∈so↝xτ∈ s_o x. In particular, marginalizing out both τ and x (in sequence), we infer (as in Proposition 3.3) that F(so,dγ)=∑x∈R(x,dγ)≔Z(dγ).F(s_o,dγ)= _x R(x,dγ) Z(dγ). In a similar way, as F(s,dγ)pF(γ)(s,τ)=R(x,dγ)pB(γ)(x,τ)F(s,dγ)p_F^(γ)(s,τ)=R(x,dγ)p_B^(γ)(x,τ), we directly infer that F(s,dγ)=∑x∈∑τ∈s↝xR(x,dγ)pB(γ)(x,τ)F(s,dγ)= _x _τ∈ s xR(x,dγ)p_B^(γ)(x,τ) with the convention that ∑τ∈∅ητ=0 _τ∈ _τ=0 for any trajectory-indexed real sequence ητ\ _τ\_τ. This is exactly Proposition B.1’s second statement. H.9 Proof of Corollary C.2 We derive the relationship between the prior and posterior mixing distributions when they both satisfy the SubTB condition. It is clear that, since pMp_M, qMq_M, pF(γ)p_F [t](γ), and pB(γ)p_B^(γ) abide by SubTB, they also abide by TB. In particular, pM(γ)pF(γ)(so,τ)=R(x)ZqM(γ|x)pB(γ)(x,τ).p_M(γ)p_F^(γ)(s_o,τ)= R(x)Zq_M(γ|x)p_B^(γ)(x,τ). As in Proposition 3.3, we marginalize over τ∈so↝xτ∈ s_o x to infer that pM(γ)⋅mF(γ)(x)=R(x)Z⋅q(γ|x);p_M(γ)· m_F^(γ)(x)= R(x)Z· q(γ|x); we then derive the second equation from Corollary C.2 by isolating q(γ|x)q(γ|x). Similarly, the first equation can be obtained by marginalizing over x∈x and recalling mF(γ)(x)m_F^(γ)(x) is a probability distribution over X, i.e., pM(γ)=pM(γ)∑x∈mF(γ)(x)=∑x∈R(x)ZqM(γ|x).p_M(γ)=p_M(γ) _x m_F^(γ)(x)= _x R(x)Zq_M(γ|x). H.10 Proof of Proposition C.5 Our proof has two steps. We first show that, if a model achieves an error of at most ϵε in terms of TVTV distance to the true policy (which is unique since we are fixing the backward policy for each γ), then the TVTV between the marginal distribution over X is bounded as a monotonically decreasing and continuous function of ϵε. Then, using Abboud et al. (2021, Theorem 1), which characterizes the universality of GNNs with random features, and the intermediate value theorem for the prior bound, we infer that our sampler can approximate any distribution with high-probability. To start with, we temporarily simplify our notation of pF(γ)p_F^(γ) as p, ommiting the dependence on γ. We also refer to the true solution to the chosen balance equations as p⋆p . Our GNN approximates the distribution p(s,⋅)p(s,·); however, we are interested in the difference |∑τ∈so↝x∏(s,s′)∈τp(s,s′)−∏(s,s′)∈τp⋆(s,s′)| | _τ∈ s_o x _(s,s )∈τp(s,s )- _(s,s )∈τp (s,s ) | for each x∈x . We start with the following lemma. Lemma H.1. Let a1,…,ana_1,…,a_n and b1,…,bnb_1,…,b_n be sequences in [0,1][0,1] such that |ai−bi|≤δ|a_i-b_i|≤δ for some ϵ>0ε>0. Then, |∏1≤i≤nai−∏1≤i≤nbi|≤nϵ. | _1≤ i≤ na_i- _1≤ i≤ nb_i |≤ nε. Proof. We proceed by induction. This clearly holds for n=1n=1. To build intuition, we also show for N=2N=2. Notice that |a1a2−b1b2|=|(a1−b1)a2+b1(a2−b2)|≤a2|a1−b1|+b1|a2−b2|≤2ϵ|a_1a_2-b_1b_2|=|(a_1-b_1)a_2+b_1(a_2-b_2)|≤ a_2|a_1-b_1|+b_1|a_2-b_2|≤ 2ε since a1,a2,b1,b2∈[0,1]a_1,a_2,b_1,b_2∈[0,1]. Assume the identity holds for n. Then, |∏1≤i≤n+1ai−∏1≤i≤n+1bi| | _1≤ i≤ n+1a_i- _1≤ i≤ n+1b_i | =|(an+1−bn+1)∏1≤i≤nai+bn+1⋅(∏1≤i≤nai−∏1≤i≤nbi)| = |(a_n+1-b_n+1) _1≤ i≤ na_i+b_n+1· ( _1≤ i≤ na_i- _1≤ i≤ nb_i ) | =ϵ+nϵ=(n+1)ϵ. =ε+nε=(n+1)ε. by the induction hypothesis. This shows our desired bound. ∎ As there is a realization of our GNN for which TV(p(s,⋅),p⋆(s,⋅))≤ϵTV (p(s,·),p (s,·) )≤ε for any s∈s , we infer that |p(s,s′)−p⋆(s,s′)|≤2ϵ|p(s,s )-p (s,s )|≤ 2ε. Under this condition, we let T be the maximum number of trajectories from sos_o to x for x∈x and L to be the largest trajectory length from sos_o to x∈x . By Abboud et al. (2021, Theorem 1), there are h∈ℋth _t and ξ∈ℋsξ _s for which p(s,⋅)p(s,·) approximates p⋆(s,⋅)p (s,·) with error at most ϵ′=ϵ2||TLε = ε2|X|TL and with probability 1−δ||1- δ|X| over the realization of γ. The above difference in marginals can then be bounded as, with probability 1−δ||1- δ|X| over the realization of γ, |(∑τ∈so↝x∏(s,s′)∈τp(s,s′)−∏(s,s′)∈τp⋆(s,s′))| | ( _τ∈ s_o x _(s,s )∈τp(s,s )- _(s,s )∈τp (s,s ) ) | ≤(∑τ∈so↝x|∏(s,s′)∈τp(s,s′)−∏(s,s′)∈τp⋆(s,s′)|) ≤ ( _τ∈ s_o x | _(s,s )∈τp(s,s )- _(s,s )∈τp (s,s ) | ) ≤(1−δT)(2TL)ϵ2||TL=ϵ||. ≤ (1- δT )(2TL) ε2|X|TL= ε|X|. In other words, denoting by mF(γ)m_F^(γ) and by π(x)π(x) the estimated and true marginals for the learned and target models, |mF(γ)(x)−π(x)|≤ϵ|||m_F^(γ)(x)-π(x)|≤ ε|X| with probability 1−δ||1- δ|X| over γ. Through an union bound55 5 We briefly recall that, if P1,…,PnP_1,…,P_n are events with probability at least 1−δ1-δ, then ∩iPi _iP_i is an event with probability at least ℙ[∩iPi]=1−ℙ[∪iPic]≥1−∑iℙ[Pic]≥1−nδ.P[ _iP_i]=1-P[ _iP_i^c]≥ 1- _iP[P_i^c]≥ 1-nδ. for x∈x , we infer that TV(mF(γ),π)=12∑x∈|mF(γ)(x)−π(x)|≤ϵ2TV (m_F^(γ),π )= 12 _x |m_F^(γ)(x)-π(x) |≤ ε2 with probability 1−δ1-δ over the sampling of γ. This demonstrates that RF GFlowNets are universal approximators of distributions over graphs when using the kind of edge-additive process we described in Proposition C.5. H.11 Proof of Lemma C.6 We first expand ℒTBL_TB around γ=0γ=0 and then take the expectation of the corresponding expansion over γ∼pMγ p_M. To start with, let L(γ)≔ℒ(τ,γ,Γ)L(γ) (τ,γ; g_ ); in this case, L(γ)=L(0)+γT∇γ=0L(γ)+12γTHγ(τ,0)γ+(σ3).L(γ)=L(0)+γ^T _γ=0L(γ)+ 12γ^TH_γ(τ,0)γ+O(σ^3). Since, for any matrix A, γ∼pM[γ]=0 and γ∼pM[γTAγ]=σ2⋅tr(A)E_γ p_M [γ ]=0 and E_γ p_M [γ^TAγ ]=σ^2·tr(A) as γTAγ=tr(γTAγ)=tr(AγγT)γ^TAγ=tr(γ^TAγ)=tr(Aγ^T) for any matrix A and trtr is a linear operator, and in general any odd moment of γ has nulls out, we infer that γ∼pM[L(γ)]=L(0)+12σ2⋅tr(Hγ(τ,0))+(σ4),E_γ p_M [L(γ) ]=L(0)+ 12σ^2·tr (H_γ(τ,0) )+O(σ^4), which is the statement of Lemma C.6 after replacing L with ℒTBL_TB. This characterizes the local behavior of the (population) loss in the learning objective of CI GFlowNets. H.12 Proof of Proposition C.7 We derive from Lemma C.6 that ℒTB(Γ)=ℒTB(o)+12⋅σ2⋅τ∼ρ[tr(Hγ(τ,0))]+(σ4)=ℒTB(o)+12⋅σ2⋅ℒreg(Γ)+(σ4).L_TB( g_ )=L_TB( g_o)+ 12·σ^2·E_τ ρ [tr (H_γ(τ,0) ) ]+O(σ^4)=L_TB( g_o)+ 12·σ^2·L_reg( g_ )+O(σ^4). As taking the derivative is a linear operation, f∈(σ4)f (σ^4) implies ∇θ2f∈(σ4) _θ^2f (σ^4) for any twice differentiable function f:Θ→ℝf on a differentiable manifold Θ . In particular, we infer that, by evaluating the Hessian from both members of the above equation with respect to the parameters θ of the underlying neural network, HTB=Ho+σ22⋅Hreg+(σ4),H_TB=H_o+ σ^22· H_reg+O(σ^4), with HTB=∇θ2ℒTB(Γ)H_TB= _θ^2L_TB( g_ ), Ho=∇θ2ℒo(Γ)H_o= _θ^2L_o( g_ ), and Hreg=∇θ2ℒreg(Γ)H_reg= _θ^2L_reg( g_ ). This in particular shows that the random features act as a form of spectral regularization, shifting the eigenvalues of HoH_o by σ22⋅eig(Hreg) σ^22·eig(H_reg), with eig(Hreg)eig(H_reg) being the vector of eigenvalues of HregH_reg. H.13 Proof of Lemma E.2 We show both functions hkh_k and gkg_k are harmonic on ¯∖k S _k. Let s∈¯∖ks∈ S _k. Then, s′∼κF(s,⋅)[hk(s′)] _s _F(s,·) [h_k(s ) ] =∑s′∈¯κF(s,s′)⋅ℙκF[Tk(f)=mink′Tk′(f)<∞|s′] = _s ∈ S _F(s,s )·P_ _F [T_k^(f)= _k T_k ^(f)<∞|s ] =∑s′∈¯κF(s,s′)ℙκF[Tk(f)<∞|s′] = _s ∈ S _F(s,s )P_ _F [T_k^(f)<∞|s ] =ℙκF[Tk(f)<∞|s]=hk(s). =P_ _F [T_k^(f)<∞|s ]=h_k(s). The first equation follows from the definition of hkh_k; the second and third equations follow from the fact that the distance from s∈ks _k to sos_o in the FHC in Definition E.1 is constant for k∈[K]k∈[K] and, hence, if the Markov chains goes through kS_k it will not go through any k′S_k for k′≠k ≠ k, . A similar argument can be carried out for gk(s)g_k(s) and Tk(b)T_k^(b). H.14 Proof of Lemma E.3 We show that hk(so)h_k(s_o) and gk(x)g_k(x) can be interpreted as probability distributions over k∈0,…,Kk∈\0,…,K\. The intuition is that the Markov chain has to either hit one of kS_k when starting at sos_o, or end up in oX_o, in which case none of kS_k are hit for k∈[K]k∈[K]; similarly, if x∈ox _o, the Markov chain following κB _B and starting at x is absorbed by sos_o without going through any of the kS_k for k∈[K]k∈[K]. We start with hk(so)h_k(s_o). By Definition E.1, the events T1(f)<∞,S1(f)=so,…,TK(f)<∞,S1(f)=s1,Tk(f)=∞ for k∈[K],S1(f)=so \T_1^(f)<∞,S_1^(f)=s_o \,…,\T_K^(f)<∞,S_1^(f)=s_1\,\T_k^(f)=∞ for k∈[K],S_1^(f)=s_o\ are mutually joint. In addition, Tk(f)<∞T_k^(f)<∞ implies Tk′(f)=∞T_k ^(f)=∞ almost surely for k′≠k ≠ k. As a consequence, these events also cover the space of possibilities. Hence, by the law of total probability, (∑1≤k≤KℙκF[Tk(f)<∞|S1(f)=so])+ℙκF[Tk(f)=∞ for k∈[K]|S1(f)=so]=1. ( _1≤ k≤ KP_ _F [T_k^(f)<∞|S_1^(f)=s_o ] )+P_ _F [T_k^(f)=∞ for k∈[K]|S_1^(f)=s_o ]=1. This sum is exactly ho(so)+∑k∈[K]hk(so)h_o(s_o)+ _k∈[K]h_k(s_o), which demonstrates the first statement of Lemma E.3. We proceed similarly for gkg_k. As noticed, if x∈ox _o, then Tk(b)=∞T_k^(b)=∞ for k∈[K]k∈[K]. Otherwise, as sos_o is the only absorbing state of a Markov chain dictated by κB _B in ¯ S has to eventually go through some of the kS_k when starting at x. This means that the events T1(b)<∞,…,TK(b)<∞,Tk=∞ for k∈[K]\T_1^(b)<∞\,…,\T_K^(b)<∞\,\T_k=∞ for k∈[K]\ are mutually disjoint when the Markov chain starts at x∈x and cover the space of possibilities. As with hkh_k, this implies go(x)+∑1≤k≤Kgk(x)=1,g_o(x)+ _1≤ k≤ Kg_k(x)=1, which is the second statement in Lemma E.3. H.15 Proof of Proposition E.4 We show that, if the policies pF(k)p_F^(k) and pB(k)p_B^(k) satisfy the ATB condition from Silva et al. (2025b), then the policies p~F(k) p_F^(k) and p~B(k) p_B^(k) satisfy the MTB condition in Definition 3.2. We first notice that, for any trajectory τ from sos_o to x∈ox _o, p~F(o)(so,τ)=ho(x)ho(so)pF(o)(so,τ)=1ho(so)⋅pF(o)(so,τ), p_F^(o)(s_o,τ)= h_o(x)h_o(s_o)p_F^(o)(s_o,τ)= 1h_o(s_o)· p_F^(o)(s_o,τ), a fact that can be derived from a telescoping product and ho(x)=1h_o(x)=1 since for a Markov chain starting at x∈ox _o and following κF _F it is true that Tk(f)=∞T_k^(f)=∞ (as none of kS_k is reachable from x∈ox _o, which is an absorbing state). By the ATB, pF(o)(so,τ)=R(x)ZpB(o)(x,τ)=R(x)Zp~B(o)(x,τ),p_F^(o)(s_o,τ)= R(x)Zp_B^(o)(x,τ)= R(x)Z p_B^(o)(x,τ), as pB(o)(s,s′)=p~B(o)(s,s′)p_B^(o)(s,s )= p_B^(o)(s,s ) for states s,s′s,s within a distance of d of sos_o; since pBp_B does not increase the distance to sos_o, and x∈ox _o is by definition within a distance d of sos_o, it holds that pF(o)(x,τ)=p~F(o)(x,τ)p_F^(o)(x,τ)= p_F^(o)(x,τ) for any τ∈so↝x∈oτ∈ s_o x _o . Similarly, for any trajectory going from sos_o to x∈kx _k through some s∈ks _k (if the trajectory goes through s∈ks _k, it must eventually end up in kX_k by definition), p~F(k)(so,τ)=pF(o)(so,τ1)⋅hk(s)hk(so)⋅pF(k)(s,τ2)=1hk(so)⋅pF(o)(so,τ1)⋅pF(k)(s,τ2), p_F^(k)(s_o,τ)=p_F^(o)(s_o, _1)· h_k(s)h_k(s_o)· p_F^(k)(s, _2)= 1h_k(s_o)· p_F^(o)(s_o, _1)· p_F^(k)(s, _2), with τ1 _1 and τ2 _2 being the portions of τ before and after s∈ks _k, respectively, and hk(s)=1h_k(s)=1 since s∈ks _k, and the Markov chain can only visit one of kS_k for k∈[K]k∈[K] since there is no positive-probability trajectory between them. Also, by the ATB, pF(o)(so,τ1)pF(k)(s,τ2)=R(x)ZpB(k)(so,τ2)⋅pB(o)(so,τ1).p_F^(o)(s_o, _1)p_F^(k)(s, _2)= R(x)Zp_B^(k)(s_o, _2)· p_B^(o)(s_o, _1). As before, for τ1 _1, it holds that pB(o)(x,τ1)=p~B(o)(x,τ1)p_B^(o)(x, _1)= p_B^(o)(x, _1). For τ2 _2, on the other hand, a telescoping product shows that pB(k)(x,τ)=p~B(k)(x,τ)⋅gk(x)gk(s)=gk(x)⋅p~B(k)(x,τ).p_B^(k)(x,τ)= p_B^(k)(x,τ)· g_k(x)g_k(s)=g_k(x)· p_B^(k)(x,τ). In particular, p~F(so,τ)=R(x)Z⋅gk(x)hk(so)⋅p~B(k)(x,τ), p_F(s_o,τ)= R(x)Z· g_k(x)h_k(s_o)· p_B^(k)(x,τ), i.e., defining Zk=Z⋅hk(so)Z_k=Z· h_k(s_o) and q(k|x)=gk(x)q(k|x)=g_k(x), which is a valid probability distribution by Lemma E.3, we infer that Zkp~F(k)(so,τ)=R(x)q(k|x)p~B(x,τ)Z_k p_F^(k)(s_o,τ)=R(x)q(k|x) p_B(x,τ) for k∈[K]k∈[K], k=0k=0, and every τ from sos_o to x∈kx _k through kS_k. This shows the solution to SAL is also a solution to the MTB condition in Definition 3.2. H.16 Proof of Proposition F.4 We show that the solution kk≥1\ g_k\_k≥ 1 for the Boosted GFlowNets learning problem abides by the MTB under Assumptions F.1 and F.3. We first demonstrate that q(k|x)q(k|x) is a probability distribution over k≥1k≥ 1. To see this, we notice that ∑1≤k≤Kq(k|x)=∑1≤k≤KRk(x)R(x)=∑1≤k≤KR(k)(x)−R(k−1)(x)R(x)=R(K)(x)R(x), _1≤ k≤ Kq(k|x)= _1≤ k≤ K R_k(x)R(x)= _1≤ k≤ K R^(k)(x)-R^(k-1)(x)R(x)= R^(K)(x)R(x), as R(o)(x)=0R^(o)(x)=0 by design. By Assumption F.1, q(k|x)≥0q(k|x)≥ 0; by Assumption F.1, the limit of the above equation as K→∞K→∞ is limK→∞R(K)R(x)=R(x)R(x)=1. _K→∞ R^(K)R(x)= R(x)R(x)=1. This shows q(k|x)q(k|x) corresponds to a probability distribution over k≥1k≥ 1. We now prove that the learned policies abide by the MTB condition in Definition 3.2 having q(k|x)q(k|x) as the posterior mixing distribution. By Assumption F.3, R(k)(x)=R(k)(x,τ)≔R(k−1)(x)+Zk⋅pF(k)(so,τ)pB(k)(x,τ),R^(k)(x)=R^(k)(x,τ) R^(k-1)(x)+ Z_k· p_F^(k)(s_o,τ)p_B^(k)(x,τ), i.e., pB(k)(x,τ)⋅(R(k)(x)−R(k−1)(x))=Zk⋅pF(k)(so,τ).p_B^(k)(x,τ)· (R^(k)(x)-R^(k-1)(x) )=Z_k· p_F^(k)(s_o,τ). Since R(k)(x)−R(k−1)(x)=Rk(x)=R(x)⋅q(k|x)R^(k)(x)-R^(k-1)(x)=R_k(x)=R(x)· q(k|x) by definition, Zk⋅pF(k)(so,τ)=R(x)q(k|x)⋅pB(k)(x,τ).Z_k· p_F^(k)(s_o,τ)=R(x)q(k|x)· p_B^(k)(x,τ). This is Proposition F.4’s second statement. It should also be clear that, since Zk=∑x∈Rk(x)=∑x∈R(x)q(k|x),Z_k= _x R_k(x)= _x R(x)q(k|x), our prior verification shows that ∑k≥1Zk=∑k≥1∑x∈R(x)q(k|x)=∑x∈R(x)∑k≥1q(k|x)=∑x∈R(x)=Z. _k≥ 1Z_k= _k≥ 1 _x R(x)q(k|x)= _x R(x) _k≥ 1q(k|x)= _x R(x)=Z. We can interchange the summations due to the fact that ∑k≥1q(k|x)=1 _k≥ 1q(k|x)=1 and X is finite. These derivations jointly highlight Boosted GFlowNets can be interpreted as DI GFlowNets having countably infinite components. Appendix I Related works Generative Flow Networks have been introduced in (Bengio et al., 2021; Bengio et al., 2023) as generative models for discrete, compositional spaces such as graphs and text. They have since been applied to diverse problems, including phylogenetic inference (Zhou et al., 2024), robust scheduling (Zhang et al., 2023), Bayesian structure learning (Deleu et al., 2022; Deleu et al., 2023), LLM fine-tuning (Hu et al., 2023; Venkatraman et al., 2024), symbolic regression (Li et al., 2023), quantitative finance (Chen et al., 2026), and drug discovery (Bengio et al., 2021; Laajil et al., 2025; Pandey et al., 2024), and have been popularized by publicly released software libraries (Tiapkin et al., 2025; Viviano et al., 2026). From a theoretical perspective, their connection to variational inference (Malkin et al., 2023) and reinforcement learning (Tiapkin et al., 2024) and Markov chains (Deleu and Bengio, 2023) has been well-established, and variants operating on cyclic graphs (Brunswic et al., 2024; Morozov et al., 2025) have also been recently considered. As with SC GFlowNets, recent works have proposed modifying the state space of the GFlowNet for improved learning, e.g., (Boussif et al., 2024; Yu et al., 2026b; Silva et al., 2025b), however, their focus lies either on improving the exploratory policy, an issue which is orthogonal to our investigations, or on developing strictly distributed algorithms requiring the training of independent models. Broadly, mixture models have been widely studied in variational and approximate inference, for instance, in semi-implicit variational inference methods (Yin and Zhou, 2018), path-sampling for Monte Carlo methods (Gelman and Meng, 1998; Liu and Liu, 2001), and in continuously indexed normalizing flows (Cornish et al., 2021; Caterini et al., 2021), the terminology of which we borrow for describing our framework. Correspondingly, recent works have proposed partitioning (binning) the continuous component of the state space of both GFlowNets (Zhou et al., 2024) and diffusion probabilistic models (Gupta et al., 2026), the outcome of which is a coarse mixture approximation to a continuous density.