Paper deep dive
FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching
Suman Cha, Seongchan Lee, Dohyun Ko, Hyunjoong Kim
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Generating mixed-type tabular data requires jointly modeling diverse feature distributions and their complex cross-column dependencies. Variational flow matching handles distinct endpoints via factorized distributions, yet leaves feature-specific processing and cross-column interactions implicit within a shared backbone. We introduce Feature-wise Unified Specialization with cross-column Exchange (FUSE) to explicitly separate these roles. FUSE applies separate adaptive mixture modules to numerical and categorical features, allowing each feature to combine shared specialized subnetworks, while joint attention preserves information exchange across all columns. We also characterize the excess population risk from restricted conditioning contexts and bound the continuous Wasserstein generation error by endpoint-prediction risk. Comprehensive experiments on eight tabular datasets demonstrate that FUSE achieves strong and consistent performance across distributional fidelity and downstream utility metrics.
Tags
Links
- Source: https://arxiv.org/abs/2608.07294v1
- Canonical: https://arxiv.org/abs/2608.07294v1
Trouble viewing inline? Open PDF directly →
Full Text
107,922 characters extracted from source content.
Expand or collapse full text
FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching Suman Cha1, Seongchan Lee2, Dohyun Ko1, Hyunjoong Kim1 Abstract Generating mixed-type tabular data requires jointly modeling diverse feature distributions and their complex cross-column dependencies. Variational flow matching handles distinct endpoints via factorized distributions, yet leaves feature-specific processing and cross-column interactions implicit within a shared backbone. We introduce Feature-wise Unified Specialization with cross-column Exchange (FUSE) to explicitly separate these roles. FUSE applies separate adaptive mixture modules to numerical and categorical features, allowing each feature to combine shared specialized subnetworks, while joint attention preserves information exchange across all columns. We also characterize the excess population risk from restricted conditioning contexts and bound the continuous Wasserstein generation error by endpoint-prediction risk. Comprehensive experiments on eight tabular datasets demonstrate that FUSE achieves strong and consistent performance across distributional fidelity and downstream utility metrics. 1 Introduction Tabular data is one of the most prevalent data formats in machine learning applications such as healthcare, finance, and public policy (Gardner et al. 2023; Holzmüller et al. 2024; Little et al. 2025). The ability to generate synthetic tabular data has become essential for data sharing, mitigating data scarcity, and enabling robust downstream analysis (Zhao et al. 2021; Qian et al. 2023; van Breugel et al. 2023). However, generating synthetic samples remains challenging since it requires capturing the complex joint distributions of mixed-type variables. The challenge extends beyond the distinction between numerical and categorical variables. Columns within the same type can also exhibit different statistical structures (Shi et al. 2025). Numerical columns may vary in skewness, tail behavior, and multimodality, while categorical columns differ in cardinality and frequency concentration. Generative models therefore need to capture these column-specific distributions while preserving dependencies between the features. Recent work has adapted flow matching to this setting (Jolicoeur-Martineau et al. 2024; Guzmán-Cordero et al. 2025). Flow matching learns a time-dependent velocity field whose induced ordinary differential equation transports a tractable source distribution to the data distribution (Liu et al. 2023; Lipman et al. 2023; Albergo and Vanden-Eijnden 2023). Variational Flow Matching (VFM) expresses the velocity through conditional inference over trajectory endpoints, while Mean-Field Variational Flow Matching (MF-VFM) makes the endpoint inference tractable by factorizing the variational endpoint distribution across variables (Eijkelboom et al. 2024). For mixed-type data, Exponential Family Variational Flow Matching (EF-VFM) extends this approach by assigning appropriate exponential family distributions to match the statistical properties of each variable type (Guzmán-Cordero et al. 2025). This factorization decomposes the training objective across columns without restricting the information available to their predictors. Each endpoint factor may condition on all coordinates of the intermediate state. While endpoint factorization aligns the objective with varying data types, the architectural challenge of processing diverse variables remains unresolved. TabbyFlow implements EF-VFM by processing column embeddings through a shared backbone (Guzmán-Cordero et al. 2025). Although this design supports parameter sharing and encodes feature identity through column embeddings, it provides no explicit mechanism for feature-dependent transformations. Motivated by this gap, we propose FUSE, a Feature-wise Unified Specialization with cross-column Exchange architecture for mixed-type tabular flow matching. FUSE applies feature-dependent component recombination with joint self-attention, enabling specialized computation while preserving cross-type information exchange. Within each variable type, adaptive mixture processing forms feature-dependent combinations of shared specialized subnetworks using a differentiable aggregate-transform-recombine operator (Puigcerver et al. 2024a). Joint attention subsequently connects all column representations, preserving information exchange between numerical and categorical variables. FUSE therefore separates feature-specific processing from the conditioning information available to each endpoint predictor. The contributions of this work are summarized as follows. 1. We introduce a mixed-type flow-matching architecture that combines type-specific adaptive mixture processing with joint attention across all columns. 2. We quantify the exact excess population risk induced by restricting the conditioning context. Under regularity conditions, we further bound the continuous-space Wasserstein generation error in terms of endpoint-prediction risk. 3. Comprehensive experiments on eight tabular datasets show that FUSE achieves competitive or superior generation quality across diverse metrics. 2 Flow Matching for Mixed-Type Data We review the flow matching formulation for mixed-type data. Flow matching defines a continuous transport through a time-dependent velocity field, while variational flow matching represents this velocity through conditional endpoint inference. Mean-field and exponential-family extensions yield tractable objectives for numerical and categorical endpoints. 2.1 Flow Matching Let X1num∈ℝdnumX_1^num ^d_num denote the numerical endpoint and let C1(k)∈1,…,KkC_1^(k)∈\1,…,K_k\ denote categorical endpoint k for k=1,…,dcatk=1,…,d_cat. Using the one-hot map ek:1,…,Kk→0,1Kke_k:\1,…,K_k\→\0,1\^K_k, we write X1=(X1num,e1(C1(1)),…,edcat(C1(dcat)))∈ℝD,X_1= (X_1^num,e_1(C_1^(1)),…,e_d_cat(C_1^(d_cat)) ) ^D, where D=dnum+∑k=1dcatKk.D=d_num+ _k=1^d_catK_k. Let X0∼p0=(0,ID)X_0 p_0=N(0,I_D) be independent of X1X_1. The linear conditional path is Xt=(1−t)X0+tX1,t∈[0,1).X_t=(1-t)X_0+tX_1, t∈[0,1). (1) Flow matching estimates the marginal velocity of this path and generates samples by integrating the learned ordinary differential equation (Lipman et al. 2023; Albergo et al. 2025). For a fixed endpoint x1x_1, the conditional velocity is ut(x∣x1)=(x1−x)/(1−t).u_t(x x_1)=(x_1-x)/(1-t). The marginal velocity is its conditional expectation given Xt=xX_t=x ut(x) u_t(x) =[ut(x∣X1)∣Xt=x] =E [u_t(x X_1) X_t=x ] =mt(x)−x1−t,mt(x)=[X1∣Xt=x]. = m_t(x)-x1-t, m_t(x)=E[X_1 X_t=x]. Thus, the endpoint posterior determines the marginal velocity only through its mean mt(x)m_t(x). 2.2 Variational Endpoint Inference Let p1∣t(x1∣x)p_1 t(x_1 x) denote the endpoint posterior at time t. Variational Flow Matching (Eijkelboom et al. 2024) approximates this posterior with qθ,t(x1∣x)q_θ,t(x_1 x) by minimizing ℒVFM(θ)=−[logqθ,t(X1∣Xt)],L_VFM(θ)=-E [ q_θ,t(X_1 X_t) ], where the expectation is taken over t∼Unif(0,1)t (0,1), X0∼p0X_0 p_0, and X1∼p1X_1 p_1. The expectation of the variational endpoint distribution is m^θ,t(x)=qθ,t(⋅∣x)[X1], m_θ,t(x)=E_q_θ,t(· x)[X_1], which induces the vector field vθ,t(x)=m^θ,t(x)−x1−t.v_θ,t(x)= m_θ,t(x)-x1-t. Mean-Field Variational Flow Matching imposes a product form on the variational endpoint distribution. Exponential Family Variational Flow Matching (Guzmán-Cordero et al. 2025) extends this construction to mixed-type data by assigning an appropriate exponential-family distribution to each endpoint column. Combining these two ingredients, we use Gaussian factors for numerical endpoints and categorical factors for categorical endpoints qθ,t(x1∣x) q_θ,t(x_1 x) =∏j=1dnum(x1,jnum∣μθ,j,t(x),νt) = _j=1^d_numN\! (x_1,j^num _θ,j,t(x), _t ) (2) ×∏k=1dcatCat(c1(k)∣πθ,k,t(x)). × _k=1^d_catCat (c_1^(k) _θ,k,t(x) ). where νt>0 _t>0 is a predetermined variance. Under this distribution, the conditional expectation of each numerical endpoint is μθ,j,t(x) _θ,j,t(x). For a one-hot categorical block, its conditional expectation is the corresponding probability vector πθ,k,t(x) _θ,k,t(x). Up to additive terms independent of θ, the negative log-likelihood is then ℒend(θ) _end(θ) =[12νt∥X1num−μθ,t(Xt)∥22 =E [ 12 _t \|X_1^num- _θ,t(X_t) \|_2^2 (3) −∑k=1dcatlogπθ,k,t(Xt)C1(k)]. - _k=1^d_cat _θ,k,t(X_t)_C_1^(k) ]. The mean-field assumption factorizes qθ,t(⋅∣x)q_θ,t(· x) across endpoint columns, but places no restriction on the conditioning argument x. Consequently, each column-wise parameter, including μθ,j,t(x) _θ,j,t(x) and πθ,k,t(x) _θ,k,t(x), may depend on all coordinates of x. The variational objective does not specify how computation should be shared effectively among these column-wise parameter functions. 3 Proposed Architecture Figure 1: Overview of the FUSE architecture. Type-specific adaptive mixture processing transforms numerical and categorical tokens into specialized representations, which then interact via global attention to capture multivariate feature dependencies. While the VFM-based approaches in Section 2 specify the factorized endpoint distributions, their effective neural parameterization remains open. For mixed-type tabular data, a fully shared backbone lacks feature-specific computation, whereas independent networks sacrifice beneficial parameter sharing. To resolve this tradeoff, we propose FUSE, an architecture interleaving type-specific adaptive mixture processing with joint attention, enabling feature specialization while maintaining cross-column dependencies. 3.1 Adaptive Mixture Processing Let Xt=(Xtnum,Xtcat)X_t=(X_t^num,X_t^cat) denote the intermediate state at time t. We project each feature into an initial token hr,j,t(0)∈ℝdhh_r,j,t^(0) ^d_h for type r∈num,catr∈\num,cat\ via feature-specific linear layers, learnable embeddings, and a time embedding ψ(t)ψ(t). This yields embedding matrices Hnum,t(0)∈ℝdnum×dhH_num,t^(0) ^d_num× d_h and Hcat,t(0)∈ℝdcat×dhH_cat,t^(0) ^d_cat× d_h. At layer ℓ , the adaptive mixture module for type r employs MrM_r parallel subnetworks to process Sr=MrPrS_r=M_rP_r latent components. Using the projection Φr(ℓ)∈ℝdh×Sr _r^( ) ^d_h× S_r and tokens H¯r,t(ℓ)=LN(Hr,t(ℓ)) H_r,t^( )=LN(H_r,t^( )), we compute the alignment scores as Rr,t(ℓ)=H¯r,t(ℓ)Φr(ℓ)dh∈ℝnr×Sr.R_r,t^( )= H_r,t^( ) _r^( ) d_h ^n_r× S_r. (4) The score matrix is then normalized along two axes. The aggregation weights Dr,t(ℓ)=softmax(Rr,t(ℓ))D_r,t^( )=softmax(R_r,t^( )) are normalized across feature tokens for each component and determine how the feature representations are aggregated into latent components. The recombination weights Cr,t(ℓ)=softmax(Rr,t(ℓ))C_r,t^( )=softmax(R_r,t^( )) are normalized across components for each feature and determine how the processed components are combined to update each feature representation. These two normalizations control the aggregation and recombination stages of the module. The aggregation weights first form the latent components Ur,t(ℓ)=(Dr(ℓ))⊤H¯r,t(ℓ).U_r,t^( )=(D_r^( )) \; H_r,t^( ). The SrS_r components are partitioned among the MrM_r subnetworks, with PrP_r components assigned to each subnetwork. Let κr(s)=⌈s/Pr⌉ _r(s)= s/P_r denote the subnetwork assigned to component s. Each component is then transformed as Vr,s,t(ℓ)=gr,κr(s)(ℓ)(Ur,s,t(ℓ)),s=1,…,Sr,V_r,s,t^( )=g_r, _r(s)^( ) (U_r,s,t^( ) ), s=1,…,S_r, and the resulting representations are stacked into Vr,t(ℓ)∈ℝSr×dhV_r,t^( ) ^S_r× d_h. The recombination weights distribute these transformed components back to the feature tokens, producing the residual update H~r,t(ℓ)=Hr,t(ℓ)+Cr,t(ℓ)Vr,t(ℓ). H_r,t^( )=H_r,t^( )+C_r,t^( )V_r,t^( ). (5) Since both weights depend on the feature representations, each feature can receive a distinct combination of subnetwork transformations. The subnetworks are also shared across all features of type r, allowing type-specific specialization without introducing a separate subnetwork for every feature. 3.2 Joint Attention The adaptive mixture module processes numerical and categorical tokens through separate sets of subnetworks. This separation supports type-specific transformations, but does not allow by itself information exchange across feature types. To enable joint contextualization, we concatenate the type-specific outputs H~t(ℓ)=Concat(H~num,t(ℓ),H~cat,t(ℓ)) H_t^( )=Concat( H_num,t^( ), H_cat,t^( )) and apply multi-head self-attention (Vaswani et al. 2017): Ht(ℓ+1)=H~t(ℓ)+MHA(ℓ)(LN(H~t(ℓ))).H_t^( +1)= H_t^( )+MHA^( ) (LN ( H_t^( ) ) ). (6) This operation allows each feature token to incorporate information from all numerical and categorical features. The resulting sequence is then partitioned into Hnum,t(ℓ+1)H_num,t^( +1) and Hcat,t(ℓ+1)H_cat,t^( +1) before entering the next layer. Each layer therefore combines type-specific adaptive processing with joint interaction. 3.3 Endpoint Prediction After L blocks, feature-specific output heads map the final numerical and categorical tokens to the parameters of their corresponding endpoint factors μθ,j,t(Xt) _θ,j,t(X_t) =(wμ,j)⊤hnum,j,t(L)+bμ,j, = (w_μ,j ) h_num,j,t^(L)+b_μ,j, πθ,k,t(Xt) _θ,k,t(X_t) =softmax(Wπ,khcat,k,t(L)+bπ,k). =softmax (W_π,kh_cat,k,t^(L)+b_π,k ). These outputs parameterize the factorized endpoint distribution in Equation (2) and are optimized using the endpoint objective in Equation (3). Each head operates on a feature-specific representation containing both adaptive transformations and joint context. Concatenating the predicted numerical means and categorical probabilities yields m^θ,t(x) m_θ,t(x), which defines the vector field vθ,t(x)=(m^θ,t(x)−x)/(1−t).v_θ,t(x)=( m_θ,t(x)-x)/(1-t). 4 Theoretical Analysis The mean-field assumption determines only how the endpoint distribution factorizes, leaving unspecified both the information available to each endpoint function and the representations shared among them. We first use an illustrative construction to examine the approximation error induced by restricting distinct endpoint functions to a single shared representation. We then show that endpoint excess risk controls the Wasserstein error of the generated distribution. All proofs and derivations are provided in the Appendix. Conditioning Penalty. Fix T∈(0,1)T∈(0,1). Let t be independent of (X0,X1)(X_0,X_1) with density ρ on [0,T][0,T], and let ℱ=σ(t,Xt)F=σ(t,X_t) denote the full conditioning σ-algebra, where XtX_t follows the conditional path in Equation (1). Let G be a restricted conditioning σ-algebra satisfying σ(t)⊆ℱ.σ(t) . For any σ-algebra A satisfying σ(t)⊆ℱσ(t) , define the numerical posterior mean m=[X1num∣]m_A=E[X_1^num ] and the categorical posterior probability vector pk,p_k,A for C1(k)C_1^(k). Let ℛT∗()R_T^*(A) denote the minimum endpoint risk over all A-measurable numerical predictors and categorical probability vectors. Proposition 1. Suppose ‖X1num‖22<∞E\|X_1^num\|_2^2<∞ with 0<νt<∞0< _t<∞ almost surely. Then ℛT∗()−ℛT∗(ℱ)= _T^*(G)-R_T^*(F)= [‖mℱ−m‖222νt] \! [ \|m_F-m_G\|_2^22 _t ] +∑k=1dcat[KL(pk,ℱ∥pk,)]≥0. + _k=1^d_catE\! [KL (p_k,F\,\|\,p_k,G ) ]≥ 0. Equality holds if and only if mℱ=m_F=m_G and pk,ℱ=pk,p_k,F=p_k,G for every k, almost surely. The decomposition vanishes when the restricted context preserves both the numerical posterior mean and every categorical posterior probability. A penalty arises only when the omitted features contain residual endpoint information after conditioning on the retained features. Architectural Bottlenecks. Let S be a Rademacher variable that satisfies ℙ(S=1)=ℙ(S=−1)=1/2P(S=1)=P(S=-1)=1/2. Let ξ∼(0,σ2)ξ (0,σ^2) and W,V∼i.i.d.(0,1)W,V .i.d. N(0,1) be mutually independent and independent of S. Define U=δS+ξU=δ S+ξ for δ≠0δ≠ 0, and suppose νt≡ν>0 _t≡ν>0. (i)(i) Conditioning bottleneck. Suppose a numerical endpoint has conditional mean m(1)(U,S)=αU+cSm^(1)(U,S)=α U+cS for c≠0c≠ 0. If its predictor is conditioned on U but not on S, Proposition 1 implies the excess numerical endpoint risk c22ν[Var(S∣U)] c^22ν\,E\! [Var(S U) ] =c22ν[sech2(δUσ2)]>0. = c^22ν\,E [sech^2 ( δ Uσ^2 ) ]>0. The equality follows from [S∣U]=tanh(δU/σ2)E[S U]= (δ U/σ^2). The positive penalty shows that an informative numerical proxy for S need not be sufficient for the endpoint mean when σ2>0σ^2>0. Joint attention allows the categorical representation to enter the numerical feature update and therefore avoids imposing the restricted conditioning used in this example. (ii)(i) Representation bottleneck. Now consider two additional endpoint means m(2)(W,V)=aW,m(3)(W,V)=bV,m^(2)(W,V)=aW, m^(3)(W,V)=bV, where a,b≠0a,b≠ 0. Suppose both endpoint functions are restricted to a single shared scalar representation ℋshared=(θ2h,θ3h):‖h‖L2=1,θ2,θ3∈ℝ. _shared= \( _2h, _3h):\|h\|_L^2=1,\; _2, _3 \. The orthogonality of W with V implies inf(f2,f3)∈ℋshared[m(2)−f22+m(3)−f32] _(f_2,f_3) _sharedE [\m^(2)-f_2\^2+\m^(3)-f_3\^2 ] =mina2,b2>0. = \a^2,b^2\>0. By contrast, the shared component dictionary represents both endpoint means exactly when the features utilize divergent recombination weights, demonstrated in Appendix Section A. This construction shows that feature-dependent recombination can remove the approximation error induced by the specified single-representation restriction while retaining a shared set of transformations. The two examples concern distinct architectural restrictions. Joint attention allows cross-type information, whereas adaptive mixture processing allows different features to form distinct combinations of shared transformations. Their relevance depends respectively on residual cross-type endpoint information and the adequacy of a shared representation. Distributional Error Bound. Let f∗f^* be the unrestricted Bayes predictor and define the excess risk ℰT(f)=ℛT(f)−ℛT(f∗)E_T(f)=R_T(f)-R_T(f^*). The following theorem bounds the Wasserstein error of the generated distribution in terms of ℰT(f)E_T(f). Theorem 1. Fix T∈(0,1)T∈(0,1) and let t have density ρ on [0,T][0,T]. Assume 0<ρ¯<ρ(t)0< ρ<ρ(t), 0<ν¯≤νt≤ν¯<∞0< ν≤ _t≤ ν<∞, and ‖X1num‖22<∞E\|X_1^num\|_2^2<∞. Suppose vθ,tv_θ,t is jointly measurable in (t,x)(t,x), uniformly L-Lipschitz in x over t, and satisfies ∫0T‖vθ,t(0)‖2t<∞. _0^T\|v_θ,t(0)\|_2\,dt<∞. Then its time-T distribution p^Tθ p_T^θ satisfies W2(p^Tθ,p1) W_2( p_T^θ,p_1) ≤Kρ,νΓL(T)1−TℰT(θ) ≤ K_ρ,ν _L(T)1-T E_T(θ) +(1−T)(‖X0−X1‖22)1/2, +(1-T) (E\|X_0-X_1\|_2^2 )^1/2, where Kρ,ν=2maxν¯,1/ρ¯K_ρ,ν=2 \ ν,1\/ ρ and ΓL(T)=(e2LT−1)/(2L) _L(T)=(e^2LT-1)/(2L), with Γ0(T)=T _0(T)=T. The first term converts endpoint excess risk into a Wasserstein error bound, while the second captures truncation at T<1T<1. Hence, larger T reduces truncation error but increases the risk-dependent coefficient. Shape (↑ ) Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank (↓ ) CTGAN 0.821(.001) 0.865(.000) 0.802(.001) 0.777(.001) 0.903(.001) 0.808(.000) 0.695(.005) 0.732(.002) 6.88 TVAE 0.852(.001) 0.897(.001) 0.771(.001) 0.759(.001) 0.921(.002) 0.830(.001) 0.812(.006) 0.857(.002) 6.12 CoDi 0.777(.001) 0.784(.001) 0.801(.001) 0.702(.001) 0.903(.003) 0.687(.001) 0.800(.014) 0.777(.040) 7.25 TabDDPM 0.990(.001) 0.987(.001) 0.977(.001) 0.971(.001) 0.992(.001) 0.187(.000) 0.930(.088) 0.302(.208) 5.12 TabSyn 0.979(.001) 0.954(.001) 0.967(.001) 0.983(.001) 0.989(.001) 0.986(.000) 0.947(.006) 0.959(.003) 4.12 TabbyFlow 0.993(.001) 0.989(.001) 0.991(.001) 0.986(.001) 0.991(.001) 0.975(.000) 0.952(.004) 0.970(.002) 2.50 TabDiff 0.992(.000) 0.987(.001) 0.989(.000) 0.987(.002) 0.992(.001) 0.975(.000) 0.952(.005) 0.938(.004) 2.75 FUSE 0.993(.001) 0.992(.001) 0.992(.001) 0.989(.001) 0.992(.001) 0.988(.001) 0.957(.004) 0.972(.002) 1.25 Trend (↑ ) Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank (↓ ) CTGAN 0.832(.016) 0.839(.001) 0.889(.003) 0.845(.002) 0.888(.005) 0.884(.001) 0.650(.053) 0.784(.004) 7.25 TVAE 0.857(.012) 0.931(.001) 0.841(.003) 0.900(.002) 0.930(.010) 0.932(.002) 0.695(.027) 0.892(.004) 6.25 CoDi 0.810(.001) 0.640(.088) 0.916(.006) 0.914(.001) 0.941(.004) 0.863(.002) 0.822(.039) 0.920(.009) 6.38 TabDDPM 0.981(.001) 0.948(.004) 0.977(.001) 0.956(.004) 0.987(.004) 0.602(.008) 0.897(.147) 0.420(.213) 5.00 TabSyn 0.954(.001) 0.920(.014) 0.955(.003) 0.982(.002) 0.992(.002) 0.983(.002) 0.928(.010) 0.969(.004) 3.75 TabbyFlow 0.987(.001) 0.988(.001) 0.972(.004) 0.983(.002) 0.993(.001) 0.984(.002) 0.936(.009) 0.978(.002) 1.75 TabDiff 0.984(.001) 0.986(.001) 0.967(.003) 0.976(.002) 0.991(.001) 0.941(.002) 0.929(.008) 0.818(.011) 3.62 FUSE 0.988(.001) 0.989(.001) 0.966(.006) 0.988(.002) 0.989(.005) 0.983(.003) 0.966(.005) 0.985(.001) 2.00 C2ST (↑ ) Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank (↓ ) CTGAN 0.568(.004) 0.723(.003) 0.803(.003) 0.644(.009) 0.578(.006) 0.588(.009) 0.143(.007) 0.175(.010) 6.50 TVAE 0.553(.003) 0.561(.003) 0.659(.011) 0.301(.011) 0.729(.004) 0.466(.005) 0.439(.018) 0.642(.015) 6.62 CoDi 0.411(.003) 0.268(.002) 0.632(.002) 0.410(.003) 0.737(.004) 0.081(.001) 0.559(.031) 0.471(.107) 7.00 TabDDPM 0.970(.004) 0.978(.005) 0.970(.003) 0.886(.006) 0.993(.006) 0.000(.000) 0.952(.179) 0.008(.009) 5.12 TabSyn 0.912(.006) 0.847(.004) 0.883(.003) 0.962(.009) 0.991(.007) 0.970(.004) 0.980(.023) 0.851(.058) 4.12 TabbyFlow 0.992(.005) 0.987(.005) 0.991(.005) 0.971(.009) 0.998(.004) 0.922(.004) 0.996(.008) 0.996(.007) 1.75 TabDiff 0.981(.005) 0.964(.003) 0.972(.004) 0.982(.008) 0.995(.004) 0.895(.004) 0.983(.022) 0.635(.027) 3.25 FUSE 0.991(.005) 0.989(.010) 0.996(.004) 0.979(.011) 0.997(.004) 0.973(.008) 0.993(.010) 0.994(.007) 1.62 MLE Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank (↓ ) AUC (↑ ) AUC (↑ ) RMSE (↓ ) AUC (↑ ) AUC (↑ ) RMSE (↓ ) AUC (↑ ) AUC (↑ ) CTGAN 0.889(.002) 0.744(.007) 0.865(.017) 0.851(.009) 0.871(.003) 0.894(.013) 0.489(.105) 0.772(.034) 6.88 TVAE 0.878(.002) 0.747(.005) 0.873(.019) 0.872(.006) 0.890(.003) 0.995(.015) 0.808(.026) 0.880(.008) 6.62 CoDi 0.744(.018) 0.483(.024) 0.770(.029) 0.819(.015) 0.926(.002) 1.771(.161) 0.820(.024) 0.948(.010) 6.00 TabDDPM 0.907(.001) 0.770(.002) 0.608(.018) 0.916(.005) 0.931(.003) 2.510(.890) 0.816(.070) 0.511(.030) 4.50 TabSyn 0.908(.001) 0.757(.009) 0.703(.020) 0.909(.005) 0.932(.003) 0.883(.019) 0.823(.034) 0.940(.012) 3.50 TabbyFlow 0.910(.003) 0.764(.007) 0.541(.013) 0.917(.006) 0.931(.003) 0.872(.009) 0.810(.030) 0.960(.007) 2.75 TabDiff 0.912(.002) 0.768(.007) 0.573(.013) 0.919(.004) 0.930(.002) 0.888(.026) 0.818(.037) 0.929(.006) 3.12 FUSE 0.909(.001) 0.769(.004) 0.610(.004) 0.921(.003) 0.930(.003) 0.841(.003) 0.810(.023) 0.962(.006) 2.62 Table 1: Performance comparison across eight datasets. Results are reported as means with standard deviations in parentheses. Bold and underlined values indicate the best and second-best results, respectively, and the proposed method is shaded. 5 Experiments We evaluate the proposed architecture by comparing it against various baseline models across multiple datasets, assessing both distributional fidelity and downstream machine learning utility. We complement this evaluation with qualitative analyses that visually assess the preservation of marginal distributions and pairwise feature correlations. Finally, we conduct ablation studies to examine the respective contributions of joint attention and adaptive mixture processing. 5.1 Experimental Setup Datasets. We evaluate FUSE on eight tabular datasets: Adult, Default, Beijing, Shoppers, Magic, News, Diabetes, and Fault. Each dataset contains both numerical and categorical features and is associated with either binary classification, multiclass classification, or regression. Dataset sizes range from 691 to 37,581 samples. Detailed dataset profiles are provided in the Appendix. Baselines. We compare FUSE against seven competitive synthetic tabular data generation methods spanning four model families: 1) GAN-based method: CTGAN (Xu et al. 2019); 2) VAE-based method: TVAE (Xu et al. 2019); 3) Diffusion-based methods: CoDi (Lee et al. 2023), TabDDPM (Kotelnikov et al. 2023), TabSyn (Zhang et al. 2024), and TabDiff (Shi et al. 2025); and 4) Flow-based methods: TabbyFlow (Guzmán-Cordero et al. 2025). Metrics. We evaluate generative fidelity and downstream utility using diverse metrics. Shape quantifies marginal distributional similarity using Kolmogorov-Smirnov statistics for numerical features and total variation distances for categorical features. Trend measures the preservation of pairwise feature dependencies by comparing the correlation structures between real and synthetic data. We also report a Classifier Two-sample Test (C2ST) score, for which a higher value indicates that a discriminator is less able to distinguish real from synthetic samples. To assess downstream utility, Machine Learning Efficiency (MLE) trains a predictive model on synthetic data and evaluates its performance on held-out real data. This section reports Shape, Trend, C2ST, and MLE, while results for α-Precision and β-Recall are provided in the Appendix. Implementation Details. Each main result is averaged over 20 random initialization seeds. For C2ST, we use an XGBoost classifier to discriminate real from synthetic samples. Complete implementation details, including preprocessing procedures and hyperparameter settings, are provided in the Appendix. Figure 2: Absolute differences between pairwise correlations from real and synthetic data. Rows correspond to datasets and columns to methods. Values closer to zero indicate more faithful preservation of feature correlation. 5.2 Main Results Table 1 compares FUSE with seven tabular data generation baselines in terms of fidelity, pairwise dependence, and downstream utility. Shape and Trend. FUSE demonstrates the most consistent marginal fidelity among the evaluated methods. It ranks first on six datasets and second on the remaining two in Shape, resulting in an average rank of 1.251.25. The next-best method, TabbyFlow, obtains an average rank of 2.502.50. The improvement is therefore observed across all eight datasets rather than being driven by a small subset of favorable cases. For Trend, FUSE achieves the highest score on five datasets and the highest cross-dataset mean of 0.9820.982, compared with 0.9780.978 for TabbyFlow. The largest improvements over TabbyFlow occur on Diabetes and Fault, where Trend increases from 0.9360.936 to 0.9660.966 and from 0.9780.978 to 0.9850.985, respectively. TabbyFlow nevertheless obtains a slightly better average rank of 1.751.75, compared with 2.002.00 for FUSE, reflecting its stronger relative rankings on Beijing, Magic, and News. These results show that FUSE provides a clear improvement in marginal fidelity while remaining highly competitive in preserving pairwise feature dependencies. C2ST. FUSE achieves the best C2ST average rank of 1.621.62, followed by TabbyFlow at 1.751.75. It ranks first on Default, Beijing, and News and second on each of the remaining five datasets, making it the only method that places among the top two across all eight datasets. Since the C2ST classifier processes all features jointly, this consistency cannot be attributed solely to accurate univariate marginals. Rather, it indicates that FUSE leaves fewer multivariate discrepancies detectable by the selected XGBoost classifier. Machine Learning Efficiency. FUSE also achieves the best average MLE rank of 2.622.62, outperforming TabbyFlow at 2.752.75 and TabDiff at 3.123.12. It obtains the best downstream performance on Shoppers, News, and Fault and the second-best result on Default. The improvement is particularly achieved on News, where FUSE reduces RMSE from 0.8720.872 for TabbyFlow, the strongest competing method on this dataset, to 0.8410.841. This corresponds to a relative reduction of approximately 3.6%3.6\%. The gains are not uniform across all tasks, as TabbyFlow achieves a lower RMSE on Beijing and TabSyn obtains a higher AUC on Diabetes. Nevertheless, the best average rank shows that the improvements in distributional fidelity are accompanied by competitive downstream utility. Overall, FUSE achieves average ranks of 1.251.25, 2.002.00, 1.621.62, and 2.622.62 for Shape, Trend, C2ST, and MLE, respectively. The results show a favorable balance across marginal, pairwise, multivariate, and task-oriented evaluations, without relying on uniformly dominant performance under any single metric. Architecture Fidelity MLE Variant Numerical mixture Categorical mixture Joint attention Shape (↑)( ) Trend (↑)( ) C2ST (↑)( ) α-Prec. (↑)( ) β-Rec. (↑)( ) AUC (↑)( ) RMSE (↓)( ) Dense FFN – – – 0.984 0.960 0.980 0.970 0.344 0.630 0.820 Restricted attention ✓ ✓ – 0.984 0.960 0.990 0.972 0.371 0.644 0.818 Shared processing – – ✓ 0.984 0.976 0.985 0.985 0.482 0.888 0.742 Numerical mixture ✓ – ✓ 0.984 0.973 0.983 0.985 0.548 0.884 0.734 Categorical mixture – ✓ ✓ 0.983 0.970 0.973 0.982 0.507 0.882 0.725 FUSE ✓ ✓ ✓ 0.985 0.977 0.991 0.986 0.545 0.882 0.725 Table 2: Component analysis averaged across eight datasets. Check marks indicate active components. MLE reports classification AUC and regression RMSE. Bold and underlined values denote the best and second-best results based on unrounded means. Figure 3: Real and synthetic marginal distributions for numerical and categorical features. 5.3 Synthetic Data Visualization Figure 3 compares marginal distributions generated by CoDi, TabbyFlow, TabDiff, and FUSE, while Figure 2 visualizes errors in pairwise dependence preservation. For numerical features, kernel density estimates show that FUSE recovers both the global shape and local structure of the empirical distributions. Specifically, FUSE captures the sequence of local modes in the polarity variable on News and Shoppers. For categorical variables, FUSE closely matches the empirical category frequencies across Adult, Beijing, Magic, and News. It preserves both dominant and low-frequency classes in Adult and accurately reproduces the target balance in Magic. FUSE also generates a tighter representation of less frequent weekday categories. These results demonstrate that FUSE effectively preserves both dominant categorical structures and minority probability masses. Figure 2 reports the absolute difference between the pairwise correlation matrices of the real and synthetic data. The block-structured matrices reveal how well each method preserves cross-type dependencies. FUSE achieves consistently low correlation errors across all blocks, with a pronounced advantage on News and Fault. Specifically on News, FUSE yields near-zero errors, whereas CoDi exhibits widespread discrepancies and TabDiff retains localized high errors. Ultimately, these visualizations confirm that FUSE successfully captures both the intra-type correlations and the complex cross-type dependencies inherent to mixed-type tabular data. 5.4 Component Analysis Table 2 evaluates six configurations to separate the contributions of adaptive mixture processing and joint attention. All configurations use identical training objectives, data splits, and five matched random seeds. Dense processing replaces the adaptive mixture modules with feed-forward networks (FFN), while restricted attention limits interactions to features of the same type. The shared-processing configuration combines dense networks with joint attention and serves as the reference for evaluating adaptive mixture processing under unrestricted conditioning. The effect of joint attention is examined through two comparisons. Under dense processing, enabling joint attention preserves Shape at 0.9840.984 while increasing Trend from 0.9600.960 to 0.9760.976 and AUC from 0.6300.630 to 0.8880.888. It also reduces RMSE from 0.8200.820 to 0.7420.742. A similar pattern appears when both adaptive mixture modules are active. Relative to restricted attention, FUSE improves Trend from 0.9600.960 to 0.9770.977, AUC from 0.6440.644 to 0.8820.882, and RMSE from 0.8180.818 to 0.7250.725, while maintaining comparable marginal fidelity. This result shows that cross-type information exchange is particularly important for dependency preservation and downstream utility. It is consistent with Proposition 1, which characterizes the cost of withholding cross-feature information. The contribution of adaptive mixture processing is evaluated under joint attention. Compared with shared processing, FUSE improves Shape from 0.9840.984 to 0.9850.985, Trend from 0.9760.976 to 0.9770.977, C2ST from 0.9850.985 to 0.9910.991, and β-Recall from 0.4820.482 to 0.5450.545. It also reduces RMSE from 0.7420.742 to 0.7250.725, although AUC decreases slightly from 0.8880.888 to 0.8820.882. Consequently, FUSE achieves the best result on four of the five fidelity metrics and the second-best β-Recall. Applying the mixture module to a single feature type produces metric-specific trade-offs. The numerical-only configuration attains the highest β-Recall and second-highest AUC, whereas the categorical-only configuration matches the RMSE of FUSE but performs less favorably on the fidelity metrics. Overall, the results support complementary roles for the two components. Joint attention enables effective cross-type conditioning, while adaptive mixture processing further improves fidelity and coverage through type-specific computation. Their combination provides the strongest aggregate fidelity profile, although no configuration uniformly dominates every metric. 6 Conclusion In this work, we have introduced FUSE, a network architecture for mixed-type variational flow matching that makes the parameter sharing configuration between mixed-type endpoint factors explicit. FUSE combines type-specific adaptive mixture processing with joint attention. The adaptive mixture modules form feature-dependent combinations of shared specialized subnetworks, while joint attention preserves cross-type information exchange and unrestricted conditioning for every endpoint predictor. Our theoretical analysis characterizes the complementary roles of these components. The conditioning information bound quantifies the excess population risk caused by restricted conditioning contexts, while the architectural bottleneck analysis distinguishes information loss from limitations in shared computation. The distributional error bound further relates endpoint-prediction risk to Wasserstein generation error under the stated assumptions. Comprehensive experiments show that FUSE achieves strong aggregate performance across the evaluated metrics. The component analysis provides evidence for joint attention and shows that adaptive mixture processing improves several fidelity and utility measures. References A. Alaa, B. Van Breugel, E. S. Saveliev, and M. van der Schaar (2022) How faithful is your synthetic data? Sample-level metrics for evaluating and auditing generative models. In Proceedings of the 39th International Conference on Machine Learning, p. 290–306. Cited by: §E.1. M. Albergo, N. M. Boffi, and E. Vanden-Eijnden (2025) Stochastic interpolants: a unifying framework for flows and diffusions. Journal of Machine Learning Research 26 (209), p. 1–80. External Links: Link Cited by: Appendix B, §2.1. M. S. Albergo and E. Vanden-Eijnden (2023) Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, Cited by: §1. L. Ambrosio, N. Gigli, and G. Savaré (2008) Gradient flows: in metric spaces and in the space of probability measures. 2 edition, Lectures in Mathematics ETH Zürich, Birkhäuser Basel. External Links: ISBN 978-3-7643-8722-8, Document, Link Cited by: §A.3. H. Bao, W. Wang, L. Dong, Q. Liu, O. K. Mohammed, K. Aggarwal, S. Som, S. Piao, and F. Wei (2022) VLMo: unified vision-language pre-training with mixture-of-modality-experts. In Advances in Neural Information Processing Systems 35, NeurIPS 2022, p. 32897–32912. External Links: Link, Document Cited by: Appendix B. J. I. Choi (2026) TabGeoFlow: a geometric flow matching model for tabular data synthesis. Proceedings of the AAAI Conference on Artificial Intelligence 40 (25), p. 20562–20569. External Links: ISSN 2159-5399, Link, Document Cited by: Appendix B. F. Eijkelboom, G. Bartosh, C. Naesseth, M. Welling, and J. V. D. Meent (2024) Variational flow matching for graph generation. In Advances in Neural Information Processing Systems 37, NeurIPS 2024, p. 11735–11764. External Links: Link, Document Cited by: Appendix B, §1, §2.2. P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, D. Podell, T. Dockhorn, Z. English, and R. Rombach (2024) Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 12606–12633. External Links: Link Cited by: Appendix B. J. Gardner, Z. Popovic, and L. Schmidt (2023) Benchmarking distribution shift in tabular data with tableshift. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, p. 53385–53432. External Links: Link, Document Cited by: §1. A. Guzmán-Cordero, F. Eijkelboom, and J. Van De Meent (2025) Exponential family variational flow matching for tabular data generation. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 21516–21529. External Links: Link Cited by: Appendix B, §1, §1, §2.2, §5.1. D. Holzmüller, L. Grinsztajn, and I. Steinwart (2024) Better by default: strong pre-tuned MLPs and boosted trees on tabular data. In Advances in Neural Information Processing Systems 37, NeurIPS 2024, p. 26577–26658. External Links: Link, Document Cited by: §1. A. Jolicoeur-Martineau, K. Fatras, and T. Kachman (2024) Generating and imputing tabular data via diffusion and flow-based gradient-boosted trees. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 238, p. 1288–1296. External Links: Link Cited by: Appendix B, §1. A. Kotelnikov, D. Baranchuk, I. Rubachev, and A. Babenko (2023) TabDDPM: modelling tabular data with diffusion models. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 17564–17579. External Links: Link Cited by: Appendix B, §5.1. C. Lee, J. Kim, and N. Park (2023) CoDi: co-evolving contrastive diffusion models for mixed-type tabular synthesis. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 18940–18956. External Links: Link Cited by: Appendix B, §5.1. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Appendix B, §1, §2.1. C. Little, R. Allmendinger, and M. Elliot (2025) Synthetic census microdata generation: a comparative study of synthesis methods examining the trade-off between disclosure risk and utility. Journal of Official Statistics 41 (1), p. 255–308. Cited by: §1. X. Liu, C. Gong, and Q. Liu (2023) Flow straight and fast: learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, Cited by: §1. M. Mueller, K. Gruber, and D. Fok (2025) Continuous diffusion for mixed-type tabular data. In International Conference on Learning Representations, Vol. 2025, p. 39186–39225. External Links: Link Cited by: Appendix B. M. Mueller, K. Gruber, and D. Fok (2026) Cascaded flow matching for heterogeneous tabular data with mixed-type features. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: Appendix B. B. I. Nasution, F. Eijkelboom, M. Elliot, R. Allmendinger, and C. A. Naesseth (2026) Flow matching for tabular data synthesis. Transactions on Machine Learning Research. External Links: ISSN 2835-8856, Link Cited by: Appendix B. J. Puigcerver, C. Riquelme Ruiz, B. Mustafa, and N. Houlsby (2024a) From sparse to soft mixtures of experts. In International Conference on Learning Representations, Vol. 2024, p. 28435–28445. External Links: Link Cited by: §1. J. Puigcerver, C. Riquelme Ruiz, B. Mustafa, and N. Houlsby (2024b) From sparse to soft mixtures of experts. In International Conference on Learning Representations, Vol. 2024, p. 28435–28445. External Links: Link Cited by: Appendix B. Z. Qian, R. Davis, and M. van der Schaar (2023) Synthcity: a benchmark framework for diverse use cases of tabular synthetic data. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, p. 3173–3188. External Links: Link, Document Cited by: §1. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. V. Le, G. E. Hinton, and J. Dean (2017) Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings, External Links: Link Cited by: Appendix B. J. Shi, M. Xu, H. Hua, H. Zhang, S. Ermon, and J. Leskovec (2025) TabDiff: a mixed-type diffusion model for tabular data generation. In International Conference on Learning Representations, Vol. 2025, p. 37353–37375. External Links: Link Cited by: Appendix B, §1, §5.1. J. Si, Z. Ou, M. Qu, Z. Xiang, and Y. Li (2026) TabRep: training tabular diffusion models with a simple and effective continuous representation. Transactions on Machine Learning Research. External Links: ISSN 2835-8856, Link Cited by: Appendix B. B. van Breugel, N. Seedat, F. Imrie, and M. van der Schaar (2023) Can you rely on your model evaluation? improving model evaluation with synthetic test data. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, p. 1889–1904. External Links: Link, Document Cited by: §1. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. External Links: Link Cited by: §3.2. L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni (2019) Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems, Vol. 32. External Links: Link Cited by: Appendix B, §5.1. H. Zhang, L. Fang, Q. Wu, and P. S. Yu (2025) TabNAT: a continuous-discrete joint generative framework for tabular data. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 74858–74881. External Links: Link Cited by: Appendix B. H. Zhang, J. Zhang, Z. Shen, B. Srinivasan, X. Qin, C. Faloutsos, H. Rangwala, and G. Karypis (2024) Mixed-type tabular data synthesis with score-based diffusion in latent space. In International Conference on Learning Representations, Vol. 2024, p. 52829–52857. External Links: Link Cited by: Appendix B, §5.1. Z. Zhao, A. Kunar, R. Birke, and L. Y. Chen (2021) CTAB-gan: effective table data synthesizing. In Proceedings of The 13th Asian Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 157, p. 97–112. External Links: Link Cited by: §1. Appendix Appendix A Proofs This section follows the order of the theoretical analysis in the main paper. We first prove Proposition 1, then derive the two architectural bottleneck examples, and finally prove Theorem 1. We conclude by verifying that a fixed FUSE network satisfies the regularity conditions used in Theorem 1. A.1 Proof of Proposition 1 We use the population version of the endpoint objective in Equation (3). For f=(μ,πkk=1dcat)f=(μ,\ _k\_k=1^d_cat), write ℛT(f)= _T(f)= [‖X1num−μt(Xt)‖222νt] \! [ \|X_1^num- _t(X_t)\|_2^22 _t ] −∑k=1dcat[logπk,t(Xt)C1(k)], - _k=1^d_catE\! [ _k,t(X_t)_C_1^(k) ], where the expectation includes t, whose density on [0,T][0,T] is ρ. Each πk,t(Xt) _k,t(X_t) is required to lie in the probability simplex ΔKk _K_k. For a conditioning σ-algebra A, let ℛT∗()R_T^*(A) be the infimum of ℛT(f)R_T(f) over all A-measurable predictors. The assumptions of Proposition 1 do not require νt _t to be bounded away from zero. Consequently, both Bayes risks can be infinite, in which case the expression +∞−(+∞)+∞-(+∞) is not defined. We therefore use a common truncation only to define the difference in this exceptional case. Set a(t)=(2νt)−1a(t)=(2 _t)^-1 and aM(t)=a(t)∧Ma_M(t)=a(t) M, and let ℛT,M∗()R_T,M^*(A) denote the Bayes risk obtained by replacing a with aMa_M in the numerical term. We interpret the risk gap in Proposition 1 as ΔT(,ℱ):= _T(G,F):= supM>0ℛT,M∗()−ℛT,M∗(ℱ). _M>0 \R_T,M^*(G)-R_T,M^*(F) \. (7) The proof below shows that the quantity inside the supremum is nondecreasing in M. Whenever both untruncated Bayes risks are finite, ΔT(,ℱ) _T(G,F) agrees with the ordinary difference ℛT∗()−ℛT∗(ℱ)R_T^*(G)-R_T^*(F). Proof of Proposition 1. For ∈,ℱA∈\G,F\, define m m_A =[X1num∣], =E[X_1^num ], pk,(r) p_k,A(r) =ℙ(C1(k)=r∣). =P(C_1^(k)=r ). Because ‖X1num‖22<∞E\|X_1^num\|_2^2<∞, conditional Jensen’s inequality gives m∈L2m_A∈ L^2. For any finite A-measurable numerical predictor h, the conditional projection identity gives, in [0,∞][0,∞], [‖X1num−h‖22∣]= \! [\|X_1^num-h\|_2^2 ]= [‖X1num−m‖22∣] [\|X_1^num-m_A\|_2^2 ] +‖m−h‖22. +\|m_A-h\|_2^2. The identity for a predictor without an a priori square-integrability assumption follows by truncating h and applying monotone convergence. Since aM(t)a_M(t) is σ(t)σ(t)-measurable and σ(t)⊆σ(t) , multiplication by aM(t)a_M(t) and integration show that m_A minimizes the truncated numerical risk. For the categorical term, let H(p)=−∑rp(r)logp(r)H(p)=- _rp(r) p(r). For any A-measurable probability vector qkq_k, [−logqk(C1(k))∣]= [- q_k(C_1^(k)) ]= H(pk,)+KL(pk,∥qk). H(p_k,A)+KL (p_k,A \|q_k ). Hence the Bayes categorical predictor is pk,p_k,A. Combining the numerical and categorical terms yields ℛT,M∗()= _T,M^*(A)= [aM(t)‖X1num−m‖22]+∑k=1dcat[H(pk,)]. \! [a_M(t)\|X_1^num-m_A\|_2^2 ]+ _k=1^d_catE\! [H(p_k,A) ]. (8) Because ⊆ℱG , the tower property gives m=[mℱ∣]m_G=E[m_F ] and pk,=[pk,ℱ∣]p_k,G=E[p_k,F ] coordinatewise. Applying the nested projection identity to the numerical term in Equation (8) gives [aM(t)‖X1num−m‖22−‖X1num−mℱ‖22] \! [a_M(t) \\|X_1^num-m_G\|_2^2-\|X_1^num-m_F\|_2^2 \ ] =[aM(t)‖mℱ−m‖22]. =E\! [a_M(t)\|m_F-m_G\|_2^2 ]. For each categorical feature, the cross-entropy decomposition gives [H(pk,)−H(pk,ℱ)]=[KL(pk,ℱ∥pk,)]. [H(p_k,G)-H(p_k,F) ]=E [KL (p_k,F \|p_k,G ) ]. Therefore, for every M>0M>0, ℛT,M∗()−ℛT,M∗(ℱ) _T,M^*(G)-R_T,M^*(F) =[aM(t)‖mℱ−m‖22]+∑k=1dcat[KL(pk,ℱ∥pk,)]. =E\! [a_M(t)\|m_F-m_G\|_2^2 ]+ _k=1^d_catE\! [KL (p_k,F \|p_k,G ) ]. The right-hand side is nondecreasing in M. Taking the supremum in Equation (7) and applying monotone convergence gives ΔT(,ℱ)= _T(G,F)= [‖mℱ−m‖222νt] \! [ \|m_F-m_G\|_2^22 _t ] +∑k=1dcat[KL(pk,ℱ∥pk,)]≥0. + _k=1^d_catE\! [KL (p_k,F \|p_k,G ) ]≥ 0. This is the identity stated in Proposition 1, with the common-truncation interpretation used only when ordinary subtraction is undefined. Finally, a(t)>0a(t)>0 almost surely, and every KL divergence is nonnegative. Thus the gap is zero if and only if mℱ=mandpk,ℱ=pk,for every k, m_F=m_G p_k,F=p_k,G every k, almost surely. This proves the equality characterization. ∎ We briefly record when the common-truncation convention reduces to ordinary subtraction. Since each categorical entropy is bounded by logKk K_k, both untruncated Bayes risks are finite whenever [‖X1num‖22νt]<∞. \! [ \|X_1^num\|_2^2 _t ]<∞. Indeed, the zero numerical predictor and uniform categorical predictors then have finite risk. For the variance schedule used in the experiments, νt=1−t2≥0 _t=1-t^2≥ 0 on [0,1][0,1], so the assumed second moment of X1numX_1^num is sufficient. Hence the difference in Proposition 1 is an ordinary finite subtraction for the reported experimental settings. A.2 Derivations for the Two Architectural Bottlenecks Conditioning bottleneck. Recall the construction in the main paper: ℙ(S=1)=ℙ(S=−1)=12,U=δS+ξ,ξ∼(0,σ2), (S=1)=P(S=-1)= 12,\;U=δ S+ξ,\;ξ (0,σ^2), where ξ⟂Sξ S, δ≠0δ≠ 0, and σ2>0σ^2>0. Bayes’ rule gives logℙ(S=1∣U=u)ℙ(S=−1∣U=u) P(S=1 U=u)P(S=-1 U=u) =(u+δ)2−(u−δ)22σ2=2δuσ2. = (u+δ)^2-(u-δ)^22σ^2= 2δ uσ^2. Since S∈−1,1S∈\-1,1\, it follows that [S∣U=u] [S U=u] =tanh(δuσ2), = \! ( δ uσ^2 ), Var(S∣U=u) (S U=u) =sech2(δuσ2). =sech^2\! ( δ uσ^2 ). For the numerical endpoint mean m(1)(U,S)=αU+cSm^(1)(U,S)=α U+cS, the full and restricted posterior means are mℱ(1) m_F^(1) =αU+cS, =α U+cS, m(1) m_G^(1) =αU+c[S∣U], =α U+cE[S U], where ℱ=σ(U,S)F=σ(U,S) and =σ(U)G=σ(U). Proposition 1 therefore gives [mℱ(1)−m(1)22ν] \! [ \m_F^(1)-m_G^(1)\^22ν ] =c22ν[S−[S∣U]2] = c^22νE\! [\S-E[S U]\^2 ] =c22ν[Var(S∣U)] = c^22νE\! [Var(S U) ] =c22ν[sech2(δUσ2)]>0. = c^22νE\! [sech^2\! ( δ Uσ^2 ) ]>0. The strict inequality follows because c≠0c≠ 0, ν>0ν>0, and sech2(z)>0sech^2(z)>0 for every finite z. Representation bottleneck. Let ℋ=L2(PW,V)H=L^2(P_W,V). Since W and V are independent standard normal variables, ⟨W,W⟩ℋ W,W _H =⟨V,V⟩ℋ=1, = V,V _H=1, ⟨W,V⟩ℋ W,V _H =0. =0. Fix h∈ℋh with ‖h‖ℋ=1\|h\|_H=1. The least-squares coefficients are θ2∗ _2^* =⟨aW,h⟩ℋ, = aW,h _H, θ3∗ _3^* =⟨bV,h⟩ℋ, = bV,h _H, and hence infθ2,θ3∈ℝ[(aW−θ2h)2+(bV−θ3h)2] _ _2, _3 E\! [(aW- _2h)^2+(bV- _3h)^2 ] =a2+b2−a2⟨W,h⟩ℋ2−b2⟨V,h⟩ℋ2. =a^2+b^2-a^2 W,h _H^2-b^2 V,h _H^2. By Bessel’s inequality, a2⟨W,h⟩ℋ2+b2⟨V,h⟩ℋ2 a^2 W,h _H^2+b^2 V,h _H^2 ≤maxa2,b2(⟨W,h⟩ℋ2+⟨V,h⟩ℋ2) ≤ \a^2,b^2\ ( W,h _H^2+ V,h _H^2 ) ≤maxa2,b2. ≤ \a^2,b^2\. The bound is attained by h=Wh=W when a2≥b2a^2≥ b^2 and by h=Vh=V otherwise. Therefore, inf(f2,f3)∈ℋshared[m(2)−f22+m(3)−f32] _(f_2,f_3) _sharedE\! [\m^(2)-f_2\^2+\m^(3)-f_3\^2 ] =mina2,b2. = \a^2,b^2\. Two shared latent components are sufficient once the features can use different recombination weights. Define g1 g_1 =3aW−bV2, = 3aW-bV2, g2 g_2 =−aW+3bV2, = -aW+3bV2, c2 c_2 =(3/4,1/4), =(3/4,1/4), c3 c_3 =(1/4,3/4). =(1/4,3/4). Both c2c_2 and c3c_3 are probability vectors, and direct calculation gives c2⊤(g1,g2) c_2 (g_1,g_2) =aW, =aW, c3⊤(g1,g2) c_3 (g_1,g_2) =bV. =bV. Thus the same component dictionary represents both endpoint means exactly, while feature-dependent recombination selects a different mixture for each feature. A.3 Proof of Theorem 1 For t<Tt<T, let pt=ℒ(Xt)p_t=L(X_t), where XtX_t is the interpolation in Equation (1). Define the true and learned embedded endpoint means by mt(x) m_t(x) =[X1∣Xt=x], =E[X_1 X_t=x], m^θ,t(x) m_θ,t(x) =(μθ,t(x),πθ,1,t(x),…,πθ,dcat,t(x))∈ℝD. = ( _θ,t(x), _θ,1,t(x),…, _θ,d_cat,t(x) ) ^D. Each categorical probability vector occupies its corresponding one-hot block. As defined in Section 2, the endpoint predictor induces vθ,t(x) v_θ,t(x) =m^θ,t(x)−x1−t. = m_θ,t(x)-x1-t. Let f∗f^* be the unrestricted Bayes predictor and use the shorthand ℰT(θ) _T(θ) =ℛT(fθ)−ℛT(f∗), =R_T(f_θ)-R_T(f^*), which is the excess risk appearing in Theorem 1. Proof of Theorem 1. Because νt≥ν¯>0 _t≥ ν>0, the zero numerical predictor and uniform categorical predictors give 0≤ℛT(f∗) 0 _T(f^*) ≤‖X1num‖222ν¯+∑k=1dcatlogKk<∞. ≤ E\|X_1^num\|_2^22 ν+ _k=1^d_cat K_k<∞. Thus ℰT(θ)E_T(θ) is well defined in [0,∞][0,∞]. If it is infinite, the claimed bound is immediate, so we assume ℰT(θ)<∞E_T(θ)<∞ below. Moreover, each categorical one-hot block has squared norm one, and therefore ‖X1‖22 \|X_1\|_2^2 =‖X1num‖22+dcat<∞. =E\|X_1^num\|_2^2+d_cat<∞. Hence p0,p1∈2(ℝD)p_0,p_1 _2(R^D). For categorical feature k, let pk,t∗(x) p_k,t^*(x) =ℙ(C1(k)=⋅∣Xt=x). =P(C_1^(k)=· X_t=x). The categorical blocks of mt(x)m_t(x) are precisely pk,t∗(x)p_k,t^*(x). The conditional projection and cross-entropy identities used in the proof of Proposition 1 yield ℰT(θ)= _T(θ)= 12∫0Tρ(t)νt‖μθ,t−mtnum‖L2(pt)2t 12 _0^T ρ(t) _t\| _θ,t-m_t^num\|_L^2(p_t)^2\,dt +∑k=1dcat∫0Tρ(t)∫KL(pk,t∗(x)∥πθ,k,t(x))pt(dx)t. + _k=1^d_cat _0^Tρ(t) (p_k,t^*(x) \| _θ,k,t(x) )p_t(dx)\,dt. Pinsker’s inequality implies ‖pk,t∗(x)−πθ,k,t(x)‖22 \|p_k,t^*(x)- _θ,k,t(x)\|_2^2 ≤‖pk,t∗(x)−πθ,k,t(x)‖12 ≤\|p_k,t^*(x)- _θ,k,t(x)\|_1^2 ≤2KL(pk,t∗(x)∥πθ,k,t(x)). ≤ 2KL (p_k,t^*(x) \| _θ,k,t(x) ). Set bt2 b_t^2 =‖m^θ,t−mt‖L2(pt)2. =\| m_θ,t-m_t\|_L^2(p_t)^2. Jointly measurable versions of the conditional means can be chosen below, so t↦bt2t b_t^2 is measurable. Combining the numerical projection identity with Pinsker’s inequality gives ∫0Tρ(t)bt2t _0^Tρ(t)b_t^2\,dt ≤2maxν¯,1ℰT(θ). ≤ 2 \ ν,1\E_T(θ). (9) This is the only point at which the numerical Gaussian loss and the categorical cross-entropy loss enter the Wasserstein argument. The assumed uniform L-Lipschitz property gives the linear-growth bound ‖vθ,t(x)‖2 \|v_θ,t(x)\|_2 ≤‖vθ,t(0)‖2+L‖x‖2. ≤\|v_θ,t(0)\|_2+L\|x\|_2. Together with joint measurability and ∫0T‖vθ,t(0)‖2t<∞ _0^T\|v_θ,t(0)\|_2\,dt<∞, this gives a unique global Carathéodory flow Φt _t. Gronwall’s inequality yields ‖Φt(x)‖2 \| _t(x)\|_2 ≤eLt(‖x‖2+∫0t‖vθ,s(0)‖2s). ≤ e^Lt (\|x\|_2+ _0^t\|v_θ,s(0)\|_2\,ds ). Since p0∈2(ℝD)p_0 _2(R^D), the learned law p^tθ=(Φt)#p0 p_t^θ=( _t)_\#p_0 also belongs to 2(ℝD)P_2(R^D) for every t≤Tt≤ T. We next construct an oracle velocity whose marginal law is ptp_t. Let γσ _σ denote the density of (0,σ2ID)N(0,σ^2I_D). Since T<1T<1, for t∈[0,T]t∈[0,T] define dt(x) d_t(x) =∫γ1−t(x−ty)p1(dy), = _1-t(x-ty)\,p_1(dy), nt(x) n_t(x) =∫yγ1−t(x−ty)p1(dy), = y _1-t(x-ty)\,p_1(dy), mt(x) m_t(x) =nt(x)dt(x), = n_t(x)d_t(x), ut(x) u_t(x) =mt(x)−x1−t=[X1−X0∣Xt=x]. = m_t(x)-x1-t=E[X_1-X_0 X_t=x]. (10) The denominator dt(x)d_t(x) is strictly positive. Because ‖X1‖2<∞E\|X_1\|_2<∞ and 1−t≥1−T>01-t≥ 1-T>0, parameterized dominated convergence shows that (t,x)↦(dt(x),nt(x))(t,x) (d_t(x),n_t(x)) is Borel measurable. Thus Equation (10) provides a jointly Borel version of both mtm_t and utu_t. For every φ∈Cc1(ℝD) ∈ C_c^1(R^D), differentiating the explicit interpolation in Equation (1) gives, for almost every t, dt∫φ(x)pt(dx) ddt (x)\,p_t(dx) =[∇φ(Xt)⊤(X1−X0)] =E\! [∇ (X_t) (X_1-X_0) ] =∫∇φ(x)⊤ut(x)pt(dx). = ∇ (x) u_t(x)\,p_t(dx). Moreover, conditional Jensen’s inequality gives ∫0T‖ut‖L2(pt)2t _0^T\|u_t\|_L^2(p_t)^2\,dt ≤T‖X1−X0‖22<∞, ≤ TE\|X_1-X_0\|_2^2<∞, and the interpolation coupling gives W2(ps,pt) W_2(p_s,p_t) ≤|s−t|(‖X1−X0‖22)1/2. ≤|s-t| (E\|X_1-X_0\|_2^2 )^1/2. Therefore (pt,ut)(p_t,u_t) satisfies the continuity equation with finite kinetic energy. The superposition principle (Ambrosio et al. 2008, Theorem 8.2.1) provides a process (Γt)t∈[0,T]( _t)_t∈[0,T] on a common probability space such that Γt _t ∼pt, p_t, Γ˙t _t =ut(Γt) =u_t( _t) Initialize the learned flow with the same random variable Γ0 _0 and set Γ^t _t =Φt(Γ0), = _t( _0), Dt D_t =‖Γ^t−Γt‖2, =\| _t- _t\|_2, et e_t =‖vθ,t(Γt)−ut(Γt)‖2. =\|v_θ,t( _t)-u_t( _t)\|_2. By the endpoint-induced form of vθ,tv_θ,t and Equation (10), [et2] [e_t^2] =bt2(1−t)2. = b_t^2(1-t)^2. Equation (9), the lower bound on ρ, and 1−t≥1−T1-t≥ 1-T imply [∫0Tet2t] \! [ _0^Te_t^2\,dt ] ≤2maxν¯,1ρ¯(1−T)2ℰT(θ)<∞. ≤ 2 \ ν,1\ ρ(1-T)^2E_T(θ)<∞. Thus e⋅∈L1(0,T)e_·∈ L^1(0,T) almost surely. Using the L-Lipschitz property of vθ,tv_θ,t, the chain rule gives, for almost every t, Dt′ D_t ≤LDt+et,D0=0. ≤ LD_t+e_t, D_0=0. The same inequality holds at times for which Dt=0D_t=0 by the standard absolute-continuity argument. Gronwall’s inequality therefore gives DT D_T ≤∫0TeL(T−t)ett. ≤ _0^Te^L(T-t)e_t\,dt. Using the coupling (Γ^T,ΓT)( _T, _T), Minkowski’s inequality, Cauchy–Schwarz, and ΓL(T)=∫0Te2L(T−t)t _L(T)= _0^Te^2L(T-t)\,dt, we obtain W2(p^Tθ,pT) W_2( p_T^θ,p_T) ≤‖DT‖L2 ≤\|D_T\|_L^2 ≤∫0TeL(T−t)1−tbtt ≤ _0^T e^L(T-t)1-tb_t\,dt ≤Kρ,νΓL(T)1−TℰT(θ), ≤ K_ρ,ν _L(T)1-T E_T(θ), (11) where Kρ,ν=2maxν¯,1/ρ¯K_ρ,ν=2 \ ν,1\/ ρ, exactly as in Theorem 1. Finally, coupling XT=(1−T)X0+TX1X_T=(1-T)X_0+TX_1 with X1X_1 gives W2(pT,p1) W_2(p_T,p_1) ≤(1−T)(‖X0−X1‖22)1/2. ≤(1-T) (E\|X_0-X_1\|_2^2 )^1/2. (12) The triangle inequality, together with Equations (11) and (12), proves Theorem 1. ∎ Theorem 1 concerns the continuous preprocessed Euclidean state in ℝDR^D, including the one-hot categorical blocks. It therefore bounds the distribution transported by the ODE before categorical argmax and inverse preprocessing. Relation to uniform training on [0,1][0,1]. Theorem 1 allows a general time density ρ on [0,T][0,T]. When training uses t∼Unif(0,1)t (0,1) as in Section 2, let rθ(t)≥0r_θ(t)≥ 0 denote the fixed-time excess endpoint risk. The uniform risk restricted to [0,T][0,T] satisfies ℰTunif(θ) _T^unif(θ) =1T∫0Trθ(t)t≤1T∫01rθ(t)t=1Tℰ1unif(θ). = 1T _0^Tr_θ(t)\,dt≤ 1T _0^1r_θ(t)\,dt= 1TE_1^unif(θ). This relation explains how the population objective trained on the full time interval controls the truncated-time risk used in the theorem. A.4 Regularity of the FUSE Vector Field Lemma 1. Fix T∈(0,1)T∈(0,1), and let θ be a finite parameter. Then there exist constants Bθ,T,Kθ,T<∞B_θ,T,K_θ,T<∞ such that sup0≤t≤T‖m^θ,t(0)‖2 _0≤ t≤ T\| m_θ,t(0)\|_2 ≤Bθ,T, ≤ B_θ,T, ‖m^θ,t(x)−m^θ,t(y)‖2 \| m_θ,t(x)- m_θ,t(y)\|_2 ≤Kθ,T‖x−y‖2 ≤ K_θ,T\|x-y\|_2 (13) for all t∈[0,T]t∈[0,T], x,y∈ℝDx,y ^D. Proof. We use the Euclidean norm for vectors, the Frobenius norm for token matrices, and ∥⋅∥op\|·\|_op for the operator norm. Assuming the schema and architecture are finite, all spaces appearing below are finite-dimensional. All constants may depend on the fixed architecture, the parameter setting θ, and T, but not on the input x. We first establish uniform boundedness and Lipschitz continuity of the normalization operators used in the network. We then propagate these properties through the adaptive-mixture networks, joint-attention, and endpoint heads. Time-conditioned normalization. Let P P =Idh−1dh⊤ =I_d_h- 1d_h11 be the centering matrix. For ϵLN>0 _LN>0, layer normalization has the form LNγ,β(z) _γ,β(z) =γ⊙Pzdh−1‖Pz∥22+ϵLN1/2+β. =γ Pz \d_h^-1\|Pz\|_2^2+ _LN \^1/2+β. Because ‖Pz‖2dh−1‖Pz∥22+ϵLN1/2 \|Pz\|_2 \d_h^-1\|Pz\|_2^2+ _LN \^1/2 ≤dh, ≤ d_h, we obtain supz∈ℝdh‖LNγ,β(z)‖2 _z ^d_h \|LN_γ,β(z) \|_2 ≤dh∥γ∥∞+∥β∥2=:BLN. ≤ d_h\|γ\|_∞+\|β\|_2=:B_LN. The positive value of ϵLN _LN prevents the normalization denominator from approaching zero. The Jacobian of the normalization map is therefore uniformly bounded, and hence ‖LNγ,β(z)−LNγ,β(z′)‖2 \|LN_γ,β(z)-LN_γ,β(z ) \|_2 ≤‖γ‖∞ϵLN‖z−z′‖2 ≤ \|γ\|_∞ _LN\|z-z \|_2 =:KLN∥z−z′∥2. =:K_LN\|z-z \|_2. Each processing or attention branch uses its own normalization. We write a generic such operator as q(t,z) _q(t,z) =aq(t)⊙LNγq,βq(z)+cq(t). =a_q(t) _ _q, _q(z)+c_q(t). For the implementation in Equation (9), aq(t)=1+γq(t)a_q(t)=1+ _q(t) and cq(t)=ζq(t)c_q(t)= _q(t). Since the time embedding and all affine maps are continuous in t, compactness of [0,T][0,T] gives Aq A_q :=sup0≤t≤T‖aq(t)‖∞<∞, := _0≤ t≤ T\|a_q(t)\|_∞<∞, Cq C_q :=sup0≤t≤T‖cq(t)‖2<∞. := _0≤ t≤ T\|c_q(t)\|_2<∞. It follows that sup0≤t≤Tz∈ℝdh‖q(t,z)‖2 _ subarrayc0≤ t≤ T\\ z ^d_h subarray\|N_q(t,z)\|_2 ≤AqBLN,q+Cq=:B,q, ≤ A_qB_LN,q+C_q=:B_N,q, ‖q(t,z)−q(t,z′)‖2 \|N_q(t,z)-N_q(t,z )\|_2 ≤AqKLN,q‖z−z′‖2 ≤ A_qK_LN,q\|z-z \|_2 =:K,q∥z−z′∥2. =:K_N,q\|z-z \|_2. Applying the same normalization to a token matrix preserves Lipschitz continuity in the Frobenius norm. Since each schema has finitely many tokens, there are finite constants B¯,q B_N,q and K¯,q K_N,q satisfying supt,H‖q(t,H)‖F _t,H\|N_q(t,H)\|_F ≤B¯,q, ≤ B_N,q, ‖q(t,H)−q(t,H′)‖F \|N_q(t,H)-N_q(t,H )\|_F ≤K¯,q‖H−H′‖F. ≤ K_N,q\|H-H \|_F. Adaptive-mixture processing. Consider a mixture of networks for feature type r∈num,catr∈\num,cat\ at a fixed layer. To simplify notation, omit the layer index and write H¯r H_r =rmix(t,Hr), =N_r^mix(t,H_r), Rr R_r =H¯rΦrdh, = H_r _r d_h, Dr D_r =softmaxcol(Rr), =softmax_col(R_r), Cr C_r =softmaxrow(Rr). =softmax_row(R_r). The two softmax operations correspond to aggregation across feature tokens and recombination across latent components in Equations (4)–(5) of the main paper. The normalization bounds imply ‖Rr‖F \|R_r\|_F ≤B¯,r‖Φr‖opdh, ≤ B_N,r\| _r\|_op d_h, ‖Rr−Rr′‖F \|R_r-R_r \|_F ≤K¯,r‖Φr‖opdh‖Hr−Hr′‖F. ≤ K_N,r\| _r\|_op d_h\|H_r-H_r \|_F. Softmax on a fixed finite-dimensional axis is bounded and Lipschitz. Therefore, finite constants BD,r,BC,r,KD,rB_D,r,B_C,r,K_D,r, and KC,rK_C,r exist such that ‖Dr‖F \|D_r\|_F ≤BD,r,‖Cr‖F≤BC,r, ≤ B_D,r, \|C_r\|_F≤ B_C,r, ‖Dr−Dr′‖F \|D_r-D_r \|_F ≤KD,r‖Hr−Hr′‖F, ≤ K_D,r\|H_r-H_r \|_F, ‖Cr−Cr′‖F \|C_r-C_r \|_F ≤KC,r‖Hr−Hr′‖F. ≤ K_C,r\|H_r-H_r \|_F. The aggregated components are Ur=Dr⊤H¯r.U_r=D_r H_r. They are uniformly bounded because ‖Ur‖F \|U_r\|_F ≤‖Dr‖F‖H¯r‖F ≤\|D_r\|_F\| H_r\|_F ≤BD,rB¯,r=:BU,r. ≤ B_D,r B_N,r=:B_U,r. For two inputs HrH_r and Hr′H_r , we have Ur−Ur′ U_r-U_r =Dr⊤(H¯r−H¯r′)+(Dr−Dr′)⊤H¯r′. =D_r ( H_r- H_r )+(D_r-D_r ) H_r . Hence, ‖Ur−Ur′‖F \|U_r-U_r \|_F ≤(BD,rK¯,r+KD,rB¯,r)‖Hr−Hr′‖F ≤ (B_D,r K_N,r+K_D,r B_N,r )\|H_r-H_r \|_F =:KU,r∥Hr−Hr′∥F. =:K_U,r\|H_r-H_r \|_F. All components of UrU_r lie in the compact set U,r _U,r =u:‖u‖2≤BU,r. = \u:\|u\|_2≤ B_U,r \. Every specialized subnetwork is a fixed finite composition of affine maps and continuously differentiable activations. Since there are finitely many subnetworks, continuity on U,rK_U,r gives Bg,r B_g,r :=maxmsupu∈U,r‖gr,m(u)‖2<∞, := _m _u _U,r\|g_r,m(u)\|_2<∞, Kg,r K_g,r :=maxmsupu∈U,r‖Dgr,m(u)‖op<∞. := _m _u _U,r\|Dg_r,m(u)\|_op<∞. If VrV_r denotes the matrix obtained by stacking the transformed components, the finite number of components implies that there are constants BV,r,KV,r<∞B_V,r,K_V,r<∞ satisfying ‖Vr‖F \|V_r\|_F ≤BV,r, ≤ B_V,r, ‖Vr−Vr′‖F \|V_r-V_r \|_F ≤KV,r‖Hr−Hr′‖F. ≤ K_V,r\|H_r-H_r \|_F. The adaptive-mixture residual network is ℬrmix(t,Hr) _r^mix(t,H_r) =CrVr. =C_rV_r. Using CrVr−Cr′Vr′ C_rV_r-C_r V_r =Cr(Vr−Vr′)+(Cr−Cr′)Vr′, =C_r(V_r-V_r )+(C_r-C_r )V_r , we obtain ‖ℬrmix(t,Hr)‖F \|B_r^mix(t,H_r)\|_F ≤BC,rBV,r=:brmix, ≤ B_C,rB_V,r=:b_r^mix, ∥ℬrmix(t,Hr)− \|B_r^mix(t,H_r)- ℬrmix(t,Hr′)∥F _r^mix(t,H_r )\|_F ≤(BC,rKV,r+KC,rBV,r)‖Hr−Hr′‖F ≤ (B_C,rK_V,r+K_C,rB_V,r )\|H_r-H_r \|_F =:ℓrmix∥Hr−Hr′∥F. =: _r^mix\|H_r-H_r \|_F. Thus every adaptive-mixture residual network is uniformly bounded and globally Lipschitz. Joint attention. Let H denote the concatenated numerical and categorical tokens entering a joint-attention network, and define Z Z =att(t,H). =N^att(t,H). The normalization bounds provide finite constants BZ,KZB_Z,K_Z such that ‖Z‖F \|Z\|_F ≤BZ, ≤ B_Z, ‖Z−Z′‖F \|Z-Z \|_F ≤KZ‖H−H′‖F. ≤ K_Z\|H-H \|_F. For one attention head, write Q Q =ZWQ+bQ⊤, =ZW_Q+1b_Q , K K =ZWK+bK⊤, =ZW_K+1b_K , V V =ZWV+bV⊤. =ZW_V+1b_V . Because these maps are affine and Z is bounded, there exist finite constants BQ,BK,BV,KQ,K,KVB_Q,B_K,B_V,K_Q,K_K,K_V such that ‖Q‖F≤BQ,‖K‖F \|Q\|_F≤ B_Q,\;\|K\|_F ≤BK,‖V‖F≤BV, ≤ B_K,\;\|V\|_F≤ B_V, ‖Q−Q′‖F \|Q-Q \|_F ≤KQ‖H−H′‖F, ≤ K_Q\|H-H \|_F, ‖K−K′‖F \|K-K \|_F ≤K‖H−H′‖F, ≤ K_K\|H-H \|_F, ‖V−V′‖F \|V-V \|_F ≤KV‖H−H′‖F. ≤ K_V\|H-H \|_F. Let A A =softmaxrow(QK⊤dhead) =softmax_row ( QK d_head ) be the attention-weight matrix. The difference between two attention matrices satisfies QK⊤−Q′K′⊤ QK -Q K =(Q−Q′)K⊤+Q′(K−K′)⊤. =(Q-Q )K +Q (K-K ) . Consequently, ‖QK⊤−Q′K′⊤‖F \|QK -Q K \|_F ≤(KQBK+BQK)‖H−H′‖F. ≤ (K_QB_K+B_QK_K )\|H-H \|_F. Since row-wise softmax is bounded and Lipschitz on a fixed finite-dimensional space, there exist BA,KA<∞B_A,K_A<∞ such that ‖A‖F \|A\|_F ≤BA, ≤ B_A, ‖A−A′‖F \|A-A \|_F ≤KA‖H−H′‖F. ≤ K_A\|H-H \|_F. The head output O=AVO=AV therefore satisfies ‖O‖F \|O\|_F ≤BABV, ≤ B_AB_V, ‖O−O′‖F \|O-O \|_F ≤(BAKV+KABV)‖H−H′‖F. ≤ (B_AK_V+K_AB_V )\|H-H \|_F. A finite concatenation of attention heads followed by a finite affine output projection preserves boundedness and global Lipschitz continuity. Thus the complete multi-head attention network in Equation (6) of the main paper satisfies supt,H‖ℬatt(t,H)‖F _t,H\|B^att(t,H)\|_F ≤batt<∞, ≤ b^att<∞, ‖ℬatt(t,H)−ℬatt(t,H′)‖F \|B^att(t,H)-B^att(t,H )\|_F ≤ℓatt‖H−H′‖F ≤ ^att\|H-H \|_F for finite constants battb^att and ℓatt ^att. Propagation through the residual network. Flatten the finite collection of token matrices into a single Euclidean vector and index all adaptive-mixture and attention residual networks in their computational order by q=1,…,Qq=1,…,Q. Write Hq(t,x) H_q(t,x) =Hq−1(t,x)+ℬq(t,Hq−1(t,x)). =H_q-1(t,x)+B_q (t,H_q-1(t,x) ). The preceding arguments show that finite constants bq,ℓqb_q, _q exist such that supt,H‖ℬq(t,H)‖2 _t,H\|B_q(t,H)\|_2 ≤bq, ≤ b_q, ‖ℬq(t,H)−ℬq(t,H′)‖2 \|B_q(t,H)-B_q(t,H )\|_2 ≤ℓq‖H−H′‖2. ≤ _q\|H-H \|_2. The initial tokenization is affine in x and depends on the time embedding. Compactness of [0,T][0,T] therefore gives B0 B_0 :=sup0≤t≤T‖H0(t,0)‖2<∞, := _0≤ t≤ T\|H_0(t,0)\|_2<∞, K0 K_0 :=sup0≤t≤TLipxH0(t,⋅)<∞. := _0≤ t≤ TLip_xH_0(t,·)<∞. At the origin, ‖Hq(t,0)‖2 \|H_q(t,0)\|_2 ≤‖Hq−1(t,0)‖2+bq. ≤\|H_q-1(t,0)\|_2+b_q. Induction over the finite residual sequence gives sup0≤t≤T‖HQ(t,0)‖2 _0≤ t≤ T\|H_Q(t,0)\|_2 ≤B0+∑q=1Qbq. ≤ B_0+ _q=1^Qb_q. For arbitrary x,y∈ℝDx,y ^D, ∥Hq(t,x) \|H_q(t,x) −Hq(t,y)∥2 -H_q(t,y)\|_2 ≤‖Hq−1(t,x)−Hq−1(t,y)‖2 ≤\|H_q-1(t,x)-H_q-1(t,y)\|_2 +‖ℬq(t,Hq−1(t,x))−ℬq(t,Hq−1(t,y))‖2 + \|B_q(t,H_q-1(t,x))-B_q(t,H_q-1(t,y)) \|_2 ≤(1+ℓq)‖Hq−1(t,x)−Hq−1(t,y)‖2. ≤(1+ _q)\|H_q-1(t,x)-H_q-1(t,y)\|_2. A second induction yields ‖HQ(t,x)−HQ(t,y)‖2 \|H_Q(t,x)-H_Q(t,y)\|_2 ≤K0∏q=1Q(1+ℓq)‖x−y‖2. ≤ K_0 _q=1^Q(1+ _q)\|x-y\|_2. Endpoint heads. The endpoint heads consist of layer normalization with a positive epsilon, finite affine maps, and categorical softmax maps. They therefore define a Lipschitz output map with a finite value at the origin. Hence, there are finite constants BoutB_out and KoutK_out such that ‖m^θ,t(0)‖2 \| m_θ,t(0)\|_2 ≤Bout+Kout‖HQ(t,0)‖2, ≤ B_out+K_out\|H_Q(t,0)\|_2, ‖m^θ,t(x)−m^θ,t(y)‖2 \| m_θ,t(x)- m_θ,t(y)\|_2 ≤Kout‖HQ(t,x)−HQ(t,y)‖2. ≤ K_out\|H_Q(t,x)-H_Q(t,y)\|_2. Therefore, we may take Bθ,T B_θ,T =Bout+Kout(B0+∑q=1Qbq), =B_out+K_out (B_0+ _q=1^Qb_q ), Kθ,T K_θ,T =KoutK0∏q=1Q(1+ℓq). =K_outK_0 _q=1^Q(1+ _q). These constants satisfy sup0≤t≤T‖m^θ,t(0)‖2 _0≤ t≤ T\| m_θ,t(0)\|_2 ≤Bθ,T, ≤ B_θ,T, ‖m^θ,t(x)−m^θ,t(y)‖2 \| m_θ,t(x)- m_θ,t(y)\|_2 ≤Kθ,T‖x−y‖2. ≤ K_θ,T\|x-y\|_2. Every operation used above is continuous in its finite-dimensional arguments, and the time embedding is continuous on [0,T][0,T]. Since FUSE is a finite composition of these operations, (t,x)↦m^θ,t(x)(t,x) m_θ,t(x) is jointly continuous. Finally, the endpoint predictor induces vθ,t(x) v_θ,t(x) =m^θ,t(x)−x1−t. = m_θ,t(x)-x1-t. Since t≤T<1t≤ T<1, ‖vθ,t(x)−vθ,t(y)‖2 \|v_θ,t(x)-v_θ,t(y)\|_2 ≤Kθ,T+11−T‖x−y‖2, ≤ K_θ,T+11-T\|x-y\|_2, ‖vθ,t(0)‖2 \|v_θ,t(0)\|_2 ≤Bθ,T1−T, ≤ B_θ,T1-T, ∫0T‖vθ,t(0)‖2t _0^T\|v_θ,t(0)\|_2\,dt ≤TBθ,T1−T<∞. ≤ TB_θ,T1-T<∞. Joint continuity implies joint Borel measurability. Thus vθ,tv_θ,t is jointly measurable, uniformly Lipschitz in x, and integrable at the origin, as required by Theorem 1. ∎ Appendix B Related Work Mixed-Type Tabular Generation. Early approaches such as CTGAN and TVAE combine type-specific preprocessing with adversarial or variational objectives (Xu et al. 2019). Diffusion-based methods represent mixed-type variables in different ways. TabDDPM and CoDi couple continuous and categorical diffusion processes, while TabDiff uses a joint continuous-time model with feature-wise learnable noise schedules (Kotelnikov et al. 2023; Lee et al. 2023; Shi et al. 2025). TabSyn performs diffusion in a learned continuous latent space, CDTD applies continuous diffusion to both variable types, and TabRep constructs a unified continuous representation (Zhang et al. 2024; Mueller et al. 2025; Si et al. 2026). TabNAT instead combines diffusion-based numerical generation with masked categorical generation (Zhang et al. 2025). FUSE follows EF-VFM in representing categorical variables as one-hot vectors along a Euclidean interpolation, while revising the network structure used to predict the endpoint-factor parameters. Variational Flow Matching. Flow matching learns a velocity field that transports a base distribution along a prescribed probability path (Lipman et al. 2023; Albergo et al. 2025). Variational flow matching expresses this field through a variational approximation to the posterior distribution over endpoints (Eijkelboom et al. 2024). When the conditional velocity field is linear in the endpoint, the resulting field depends only on the marginal endpoint means. This permits a mean-field factorization without requiring the joint posterior covariance. EF-VFM extends this formulation to mixed-type data using exponential-family endpoint factors and moment matching, with TabbyFlow providing its tabular implementation (Guzmán-Cordero et al. 2025). Other tabular flow methods replace neural predictors with gradient-boosted trees, impose geometric constraints, adopt cascaded generation, or examine the effects of probability paths and sampling schemes (Jolicoeur-Martineau et al. 2024; Choi 2026; Mueller et al. 2026; Nasution et al. 2026). FUSE retains the probability path and endpoint-factor family of EF-VFM, but focuses on how type-specific processing and full-state conditioning should be organized. Specialized Neural Processing. Mixture architectures process representations through specialized subnetworks whose contributions depend on the input. Sparse mixture models use learned gates to select a subset of subnetworks, whereas Soft MoE replaces discrete assignment with differentiable dispatch and combination weights over weighted token aggregates (Shazeer et al. 2017; Puigcerver et al. 2024b). A related architectural pattern separates modality-specific feed-forward processing from cross-modal interaction. VLMo combines modality-specific feed-forward modules with shared self-attention, while multimodal diffusion transformers use separate modality-specific parameters while allowing joint attention (Bao et al. 2022; Esser et al. 2024). These architectures show that specialized processing can coexist with unrestricted interaction across input types. Appendix C Implementation Details This section specifies the implementation choices omitted from the main paper. It describes time conditioning, the architectural parameterization, and capacity matching. C.1 Time Conditioning Let dh≥2d_h≥ 2 denote the hidden dimension and let dω=⌊dh/2⌋d_ω= d_h/2 . The time t is embedded as ωk _k =exp−klog(10000)max1,dω,k=0,…,dω−1, = \! \- k (10000) \1,d_ω\ \, k=0,…,d_ω-1, s(t) s(t) =(sin(tωk)k=0dω−1,cos(tωk)k=0dω−1), = (\ (t _k)\_k=0^d_ω-1,\ (t _k)\_k=0^d_ω-1 ), ψ(t) ψ(t) =W2SiLU(W1s(t)+b1)+b2. =W_2SiLU\! (W_1s(t)+b_1 )+b_2. (14) When dhd_h is odd, a zero coordinate is appended to s(t)s(t) so that its dimension equals dhd_h. The feature-specific affine maps described in the main paper add a type-specific projection of ψ(t)ψ(t) to each initial token. Each processing and attention branch uses independent time-conditioned normalization parameters. Let q∈proc,attnq∈\proc,attn\ index the branch. For feature type r∈num,catr∈\num,cat\ and layer ℓ , (ζr,q(ℓ)(t),γr,q(ℓ)(t)) ( _r,q^( )(t), _r,q^( )(t) ) =Wr,q(ℓ)SiLU(ψ(t))+br,q(ℓ), =W_r,q^( )SiLU\! (ψ(t) )+b_r,q^( ), r,q(ℓ)(H;t) _r,q^( )(H;t) =LN(H)⊙1+γr,q(ℓ)(t)+ζr,q(ℓ)(t). =LN(H) \1+ _r,q^( )(t) \+ _r,q^( )(t). (15) Layer normalization is applied without learned affine parameters and uses ϵLN=10−5 _LN=10^-5. The output maps producing ζr,q(ℓ)(t) _r,q^( )(t) and γr,q(ℓ)(t) _r,q^( )(t) are zero-initialized. The normalized representation used by adaptive mixture processing is H¯r,t(ℓ)=r,proc(ℓ)(Hr,t(ℓ);t), H_r,t^( )=N_r,proc^( ) (H_r,t^( );t ), while the attention uses its own normalization parameters. C.2 Architecture Specification Adaptive mixture processing. The implementation computes the alignment score matrix as R~r,t(ℓ) R_r,t^( ) =H¯r,t(ℓ)Φr(ℓ). = H_r,t^( ) _r^( ). (16) Since Φr(ℓ) _r^( ) is learned, this omission changes its parameterization but not the represented family of alignment scores. Each specialized subnetwork contains two linear layers with a GELU activation. No dropout or learned residual gate is used within the processing module. Joint attention. Joint attention is applied without an attention mask. The restricted-attention variant retains the same parameterization but masks the two cross-type attention blocks, allowing interactions only within each feature type. Every configuration contains L=4L=4 blocks, each consisting of one processing module followed by one attention module. Attention dropout is set to zero. Endpoint heads. After the final block, numerical and categorical tokens are normalized using separate affine layer-normalization modules, hr,j,t(L) h_r,j,t^(L) =LNr([Hr,t(L)]j),r∈num,cat,L=4. =LN_r ( [H_r,t^(L) ]_j ), r∈\num,cat\, L=4. Thus, hr,j,t(L)h_r,j,t^(L) in the endpoint-head equations of the main paper denotes the normalized final token rather than the corresponding row of Hr,t(L)H_r,t^(L) before normalization. The numerical head applies no additional output activation, while the categorical heads retain the feature-wise parameterization defined in the main paper. The final normalization uses ϵLN=10−5 _LN=10^-5. For fixed learned parameters, this positive normalization constant ensures bounded final token representations, as required in Lemma 1. α-Precision (↑ ) Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank ↓ CTGAN 0.715(.003) 0.683(.002) 0.916(.003) 0.821(.003) 0.789(.004) 0.974(.002) 0.640(.013) 0.656(.012) 6.63 TVAE 0.923(.003) 0.828(.004) 0.859(.003) 0.567(.004) 0.951(.003) 0.857(.003) 0.774(.018) 0.878(.014) 6.50 CoDi 0.497(.003) 0.710(.001) 0.949(.003) 0.727(.005) 0.842(.005) 0.523(.002) 0.896(.024) 0.924(.033) 6.38 TabDDPM 0.971(.004) 0.994(.002) 0.977(.002) 0.938(.004) 0.992(.003) 0.000(.000) 0.876(.206) 0.003(.012) 5.00 TabSyn 0.974(.003) 0.924(.003) 0.987(.003) 0.978(.006) 0.991(.003) 0.968(.003) 0.955(.025) 0.936(.018) 3.63 TabbyFlow 0.991(.002) 0.990(.002) 0.994(.002) 0.993(.002) 0.989(.004) 0.918(.003) 0.972(.012) 0.955(.015) 2.63 TabDiff 0.979(.003) 0.982(.002) 0.981(.003) 0.992(.002) 0.992(.003) 0.965(.003) 0.926(.029) 0.697(.051) 3.75 FUSE 0.992(.002) 0.993(.002) 0.996(.002) 0.985(.005) 0.994(.003) 0.983(.006) 0.969(.013) 0.976(.012) 1.50 β-Recall (↑ ) Method Adult Default Beijing Shoppers Magic News Diabetes Fault Avg. rank ↓ CTGAN 0.270(.002) 0.169(.002) 0.260(.002) 0.286(.004) 0.082(.003) 0.161(.002) 0.078(.009) 0.007(.002) 7.13 TVAE 0.340(.003) 0.185(.001) 0.170(.002) 0.173(.003) 0.328(.003) 0.309(.004) 0.346(.010) 0.113(.007) 6.50 CoDi 0.229(.003) 0.059(.002) 0.300(.003) 0.305(.004) 0.491(.003) 0.112(.001) 0.688(.021) 0.613(.025) 5.50 TabDDPM 0.479(.002) 0.467(.004) 0.400(.002) 0.527(.006) 0.523(.004) 0.000(.000) 0.713(.173) 0.000(.000) 4.00 TabSyn 0.428(.003) 0.403(.002) 0.307(.002) 0.494(.006) 0.481(.005) 0.443(.002) 0.510(.022) 0.338(.040) 4.38 TabbyFlow 0.548(.003) 0.517(.003) 0.552(.003) 0.600(.005) 0.473(.004) 0.578(.002) 0.622(.028) 0.619(.017) 2.13 TabDiff 0.538(.003) 0.520(.002) 0.472(.002) 0.487(.005) 0.493(.004) 0.387(.003) 0.466(.033) 0.036(.012) 3.63 FUSE 0.497(.004) 0.452(.004) 0.551(.002) 0.526(.005) 0.468(.005) 0.444(.009) 0.767(.027) 0.703(.028) 2.75 Table 3: α-Precision and β-Recall across eight datasets. Results are reported as means with standard deviations in parentheses. Bold and underlined values indicate the best and second-best results, respectively, and the proposed method is shaded. C.3 Capacity Matching For each feature type, the dense FFN baseline uses a two-layer feed-forward module with hidden width dstandard=4096d_standard=4096, equal to the sum of the widths of the four specialized subnetworks. Across the eight datasets, FUSE contains between 20.26420.264 and 20.31720.317 million trainable parameters, while the dense processing variants contain between 20.24220.242 and 20.29520.295 million. The difference is 22,52822,528 parameters, corresponding to less than 0.12%0.12\% of the total parameter count. A leading-order multiply-accumulate comparison provides additional context for the processing modules. Let N=dnum+dcatN=d_num+d_cat denote the total number of numerical and categorical feature tokens. In the evaluated configuration, Snum=Scat=S=8,dff=1024,dstd=4096.S_num=S_cat=S=8, d_f=1024, d_std=4096. The per-observation counts for one processing layer are MACadap _adap =4Sdhdff+3NSdh, =4Sd_hd_f+3NSd_h, MACstd _std =2Ndhdstd. =2Nd_hd_std. At the evaluated widths, the ratio MACstd/MACadapMAC_std/MAC_adap ranges from 2.242.24 to 11.5911.59. These counts include only the leading matrix products in the processing modules. They exclude attention, normalization, activation functions, and other element-wise operations and therefore are not interpreted as end-to-end runtime measurements. Appendix D Experimental Details This section provides additional details for the experiments in the main paper. Dataset Task Train Test Num. Cat. Adult Binary 32,561 16,281 6 9 Default Binary 27,000 3,000 14 10 Beijing Regression 37,581 4,176 7 5 Shoppers Binary 11,097 1,233 10 8 Magic Binary 17,117 1,902 10 1 News Regression 35,679 3,965 46 2 Diabetes Binary 691 77 8 1 Fault Multiclass 1,746 195 27 1 Table 4: Downstream tasks, processed train and test split sizes, and variable counts. Binary and multiclass denote classification tasks. Num. and Cat. denote numerical and categorical variables, respectively. The prediction target is included in its corresponding variable type. Datasets. Table 4 summarizes the profiles of the datasets. Adult, Default, Shoppers, Magic, and Diabetes define binary classification tasks, while Fault defines a multiclass classification task. Beijing and News are regression datasets. Training. All FUSE configurations use hidden dimension dh=256d_h=256, four layers, and four attention heads. For each feature type, adaptive mixture processing uses Mr=4M_r=4 parallel subnetworks, Pr=2P_r=2 latent components per subnetwork, and hidden width dff=1024d_f=1024. Training proceeds using AdamW with learning rate 10−310^-3. The model parameters are maintained using an exponential moving average with decay 0.9970.997. Batch sizes are 10241024 for Adult, Magic, and News, 672672 for Beijing, and 512512 for the remaining datasets. The numerical endpoint variance is νt=1−t2 _t=1-t^2 for t∈[0,1).t∈[0,1). This schedule specifies the variance of the numerical endpoint factor and satisfies νt→0 _t→ 0 as t→1t→ 1. Sampling. Figure 4: Numerical marginal distributions on Default and Diabetes. Gray denotes the real data. Figure 5: Categorical marginals on Default and Diabetes. Gray denotes the real data. Generation begins from X^0∼(0,ID), X_0 (0,I_D), where D is the dimension of the transformed row representation. The resulting ODE is integrated to tmax=0.999t_ =0.999 using an adaptive fifth-order Dormand–Prince solver. Both relative and absolute tolerances are set to 10−510^-5, and the exponential-moving-average parameters are used for generation. Each generated table contains ntrainn_train observations. Figure 6: Absolute differences between pairwise correlations from real and synthetic data (Default and Diabetes). Values closer to zero indicate more faithful preservation of feature correlation. Appendix E Additional Experiments The following experiments extend the main evaluation with support-based metrics, and additional visual comparisons. E.1 α-Precision and β-Recall Table 3 reports α-Precision and β-Recall across all eight datasets. The former evaluates whether synthetic observations remain within high-probability regions of the real distribution, while the latter evaluates how well the synthetic distribution covers the support of the real data (Alaa et al. 2022). Reporting both metrics distinguishes fidelity to the data support from support coverage. FUSE achieves the best average rank for α-Precision at 1.501.50 and the second-best average rank for β-Recall at 2.752.75, following TabbyFlow at 2.132.13. It also obtains the highest β-Recall on Diabetes and Fault. These results show that the strong support precision of FUSE is accompanied by competitive coverage of the real distribution. E.2 Additional Synthetic Data Visualizations Figures 4 and 5 compare the marginal distributions generated by all eight methods on Default and Diabetes, which are not included in the main marginal visualizations. Each numerical plotting range is determined from the corresponding real training data, while every category observed in the real data is retained in the categorical panels. FUSE closely follows the numerical distribution in Default and the unimodal distribution in Diabetes. In both cases, it reproduces the dominant probability mass while retaining the overall tail behavior. These observations extend the marginal comparisons in the main paper to two additional datasets and are consistent with the aggregate Shape results. For categorical variables, FUSE preserves both the dominant and lower-frequency levels in Default and closely reproduces the binary class balance in Diabetes. The categorical panels complement the numerical comparisons by illustrating the two components summarized by the Shape metric. Figure 6 extends the dependence analysis to Default and Diabetes and includes the four baselines omitted from the main heatmap. Unlike Trend, which averages similarities over pairs selected according to their dependence in the real data, the heatmaps display every variable pair. Numerical pairs report one half of the absolute difference between the real and synthetic Pearson correlations. Pairs involving a categorical variable report the total variation distance between the corresponding contingency tables. Numerical variables in mixed-type pairs are discretized into ten common bins before the contingency tables are constructed. The shared color scale is capped at 0.10.1, matching the main heatmap. E.3 Extended Component Analysis ρdep _dep Shape Trend C2ST MLE α-Precision β-Recall 0.00 0.000(.001) 0.002(.008) 0.002(.005) 0.003(.004) 0.001(.009) 0.000(.004) 0.20 0.000(.001) 0.002(.007) 0.000(.007) 0.104(.004) -0.003(.007) 0.000(.004) 0.40 0.001(.002) 0.008(.012) 0.002(.004) 0.239(.020) 0.015(.020) 0.012(.006) 0.60 0.003(.004) 0.015(.019) 0.002(.004) 0.364(.015) 0.016(.016) 0.035(.011) 0.80 0.000(.001) 0.028(.018) 0.002(.004) 0.482(.035) 0.004(.015) 0.102(.005) 0.95 0.003(.002) 0.010(.020) 0.000(.006) 0.508(.015) 0.001(.004) 0.161(.002) Table 5: Effects of joint attention across dependence levels. Entries are means with sample standard deviations over five matched seeds. Positive values favor joint attention. The component analysis in the main paper separates adaptive mixture processing from joint attention across datasets. This section examines joint attention more directly through a controlled cross-type dependence experiment and matched comparisons on the eight datasets. Figure 7: Population conditioning penalty and learned endpoint-risk difference across dependence levels. Stacked bars show the numerical and categorical components of the population penalty. Green markers show the mean learned difference. Positive values favor joint attention. Controlled Cross-Type Dependence. To isolate the effect of cross-type dependence, we construct a mixed-type distribution in which ρdep _dep controls the association between a binary variable and a numerical variable. For ρdep∈0,0.2,0.4,0.6,0.8,0.95 _dep∈\0,0.2,0.4,0.6,0.8,0.95\, let C⋆ C ∼Bernoulli(1/2),ε∼(0,1),ε⟂C⋆, (1/2), (0,1), C , Z⋆ Z =ρdep(2C⋆−1)+1−ρdep2ε. = _dep(2C -1)+ 1- _dep^2\, . (17) This construction gives Corr(Z⋆,2C⋆−1)=ρdepCorr(Z ,2C -1)= _dep. Each observation also contains two independent standard Gaussian variables and one independent categorical variable. The binary variable C⋆C serves as the downstream target, making its association with Z⋆Z the only cross-type dependence. Dataset Shape Trend C2ST MLE α-Precision β-Recall Adult 0.001(.001) 0.030(.001) 0.001(.005) 0.034(.001) 0.004(.001) 0.295(.004) Default 0.000(.000) 0.025(.020) 0.007(.011) 0.019(.007) 0.002(.002) 0.139(.004) Beijing 0.000(.001) 0.032(.003) -0.001(.003) 0.164(.004) 0.007(.001) 0.047(.003) Shoppers -0.001(.001) 0.006(.002) 0.004(.026) 0.270(.033) 0.007(.003) 0.092(.005) Magic 0.000(.000) 0.011(.005) 0.007(.006) 0.428(.015) 0.058(.002) 0.069(.007) News 0.000(.001) 0.003(.000) 0.000(.007) 0.007(.001) 0.001(.003) 0.217(.007) Diabetes 0.002(.002) 0.010(.003) 0.003(.014) 0.277(.044) 0.029(.015) 0.084(.025) Fault 0.000(.002) 0.012(.001) 0.007(.020) 0.462(.034) 0.022(.005) 0.303(.021) Table 6: Paired effects of joint attention on the six evaluation metrics. Entries are means with sample standard deviations over five matched seeds after averaging the adaptive and standard processing contrasts. Positive values favor joint attention. Joint and restricted attention are compared over five matched seeds at each dependence level. Both configurations use adaptive mixture processing and differ only in their attention scope. Each seed contains 20,00020,000 training and 5,0005,000 test observations. The population conditioning penalty is estimated using 10610^6 Monte Carlo samples, while each pair of fitted models is evaluated on a separate sample of size 10510^5. Figure 7 shows that both the population penalty and the learned endpoint-risk reduction are near zero at independence and increase with cross-type dependence. At ρdep=0.95 _dep=0.95, they reach 0.10430.1043 and 0.07150.0715, respectively. All five learned reductions are positive at every nonzero dependence level. Although the learned reductions compare fitted predictors rather than estimate the population penalty, their direction is consistent with Proposition 1. Table 5 shows that the MLE gain increases from 0.0030.003 to 0.5080.508 at ρdep=0.95 _dep=0.95. In contrast, the effects remain at most 0.0030.003 for Shape and 0.0020.002 for C2ST. Joint attention therefore becomes most consequential for downstream utility as task-relevant cross-type dependence strengthens. Cross-Dataset Evidence. The controlled experiment is complemented by evaluations on the eight benchmark datasets. First, let Rj,d,scatR_j,d,s^cat denote the categorical endpoint risk on dataset d under seed s and attention configuration j∈joint,restrictedj∈\joint,restricted\. The relative increase under restricted attention is defined as δd,scat _d,s^cat =100×Rrestricted,d,scat−Rjoint,d,scatRjoint,d,scat. =100× R_restricted,d,s^cat-R_joint,d,s^catR_joint,d,s^cat. (18) Dataset Relative increase (%) Positive pairs Adult 4.650 (0.051) 5/5 Default 15.722 (0.157) 5/5 Beijing 5.009 (0.037) 5/5 Shoppers 2.048 (0.179) 5/5 Magic 9.630 (0.121) 5/5 News 13.562 (0.806) 5/5 Diabetes 10.826 (1.689) 5/5 Fault 51.392 (1.916) 5/5 Table 7: Relative increase in categorical endpoint risk under restricted attention. Entries are means with sample standard deviations over five random seeds, together with the number of seeds yielding a positive paired difference. Both configurations use adaptive mixture processing and are evaluated using common interpolation draws over the same fixed held-out observations. Table 7 reports a positive mean increase on every dataset, with positive paired effects across all evaluations. The endpoint-risk advantage of joint attention therefore holds consistently across the datasets. The second analysis considers all six evaluation metrics under both processing configurations. Let Yp,j,d,s(m)Y_p,j,d,s^(m) denote metric m under processing configuration p∈adaptive,standardp∈\adaptive,standard\ and attention configuration j∈joint,restrictedj∈\joint,restricted\. To ensure that positive values consistently favor joint attention, define ηd(m) _d^(m) =−1,if m is regression MLE,1,otherwise, = cases-1,&if $m$ is regression MLE,\\ 1,&otherwise, cases and average the paired effect equally over the two processing configurations as Δd,s(m) _d,s^(m) =ηd(m)2∑(Yp,joint,d,s(m)−Yp,restricted,d,s(m)). = _d^(m)2Σ (Y_p,joint,d,s^(m)-Y_p,restricted,d,s^(m) ). Joint attention consistently improves Trend, β-Recall, and task-oriented MLE across the matched evaluations. As shown in Table 6, the mean effect on α-Precision is positive for every dataset, whereas the effects on Shape and C2ST remain small and vary in sign. These results indicate that joint attention contributes most consistently to dependence preservation, support coverage, and downstream utility. Together with the controlled experiment, the cross-dataset results show that this benefit persists across heterogeneous benchmark datasets.