Paper deep dive
TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks
Jayoung Kim, Sanghyun Hong, Noseong Park
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Generative modeling for tabular data has advanced to the point where models can synthesize realistic records, recover missing values, and generate samples conditioned on class labels or observed attributes. Despite recent advances, existing approaches either require table-specific training or restrict in-context learning to single-cell prediction. In this work, we formulate these tabular generative tasks as a single problem: modeling the joint conditional distribution of an arbitrary set of target cells given the observed cells of an unseen table. To this end, we turn to diffusion, whose denoising objective is defined jointly over all target cells rather than one cell at a time. Under the pretraining distribution, this objective is minimized by the posterior predictive distribution over the target set. Building on this, we propose TabMosaic, a unified in-context diffusion model for generating mixed-type target cells in unseen, potentially incomplete tables without table-specific training. We further show that its pretraining objective bounds the conditional negative log-likelihood and the divergence from the posterior predictive for arbitrary target sets, extending the Bayesian account of in-context learning from a single-value prediction to joint multi-cell generation. Across 48 real-world tables, TabMosaic ranks first on all evaluated generative tasks against 20 baseline approaches
Tags
Links
- Source: https://arxiv.org/abs/2610.15699v1
- Canonical: https://arxiv.org/abs/2610.15699v1
Trouble viewing inline? Open PDF directly →
Full Text
93,910 characters extracted from source content.
Expand or collapse full text
TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks Jayoung Kim Affiliation: KAIST Sanghyun Hong Affiliation: Oregon State Universityjayoung.kim, noseong@kaist.ac.kr, sanghyun.hong@oregonstate.edu Noseong Park Affiliation: KAIST Abstract Generative modeling for tabular data has advanced to the point where models can synthesize realistic records, recover missing values, and generate samples conditioned on class labels or observed attributes. Despite recent advances, existing approaches either require table-specific training or restrict in-context learning to single-cell prediction. In this work, we formulate these tabular generative tasks as a single problem: modeling the joint conditional distribution of an arbitrary set of target cells given the observed cells of an unseen table. To this end, we turn to diffusion, whose denoising objective is defined jointly over all target cells rather than one cell at a time. Under the pretraining distribution, this objective is minimized by the posterior predictive distribution over the target set. Building on this, we propose TabMosaic, a unified in-context diffusion model for generating mixed-type target cells in unseen, potentially incomplete tables without table-specific training. We further show that its pretraining objective bounds the conditional negative log-likelihood and the divergence from the posterior predictive for arbitrary target sets, extending the Bayesian account of in-context learning from a single-value prediction to joint multi-cell generation. Across 48 real-world tables, TabMosaic ranks first on all evaluated generative tasks against 20 baseline approaches. 1 Introduction In domains such as healthcare, finance and scientific discovery, tabular data is widely used for data-driven decision-making (Borisov et al., 2022a; Van Breugel and Van Der Schaar, 2024). Recent tabular generation models synthesize realistic rows, recover missing values, and generate samples conditioned on classes or attributes (Kotelnikov et al., 2023; Kim et al., 2023; Zhang et al., 2024; Shi et al., 2025; Zhang et al., 2025a; Jarrett et al., 2022; Du et al., 2024; Zhang et al., 2025b). Yet these models are typically dataset-specific: they must be trained or fine-tuned separately for each new table, requiring a new training run whenever the dataset changes. This motivates a natural question: Can a single pretrained model perform diverse tabular generative tasks on unseen tables without table-specific training, using only in-context information? Recent work has been moving toward this goal by transferring tabular models across datasets. Cross-table generative models pretrain across many tables, reducing the need to learn each dataset entirely from scratch, but still require fine-tuning on each new one (Lin et al., 2025; Jiang et al., 2026). In-context tabular models remove this adaptation step and operate directly on unseen tables (Hollmann et al., 2025; Qu et al., 2025). However, these models are trained for individual prediction targets rather than the joint distribution over an arbitrary set of cells, so multi-cell generation and imputation must be assembled from separate predictions rather than modeled directly. No existing work provides a single pretrained model that learns the joint conditional distribution over arbitrary target sets and applies it directly to unseen tables without table-specific adaptation. Figure 1: Overview of our proposed framework. (a) A unified formulation expresses generation, imputation and class-conditional generation as the same conditional query, only differing in the target set S. (b) A shared-time diffusion objective jointly models mixed numerical and categorical targets. (c) Our TabMosaic applies directly to unseen tables without table-specific training. Our contributions. In this work, we formulate tabular generation, imputation and conditional generation as a single problem: learning the joint conditional distribution over arbitrary target sets on unseen tables. This formulation shifts tabular generative modeling from training a new model for each dataset and task to learning a reusable conditional model once across datasets. Different generative tasks are specified simply by which cells are observed and which are queried. To this end, we first formulate a unified learning objective for tabular generative tasks. Generation, imputation, and conditional generation are treated as the same family of conditionals, differing only in which cells are designated as targets. We instantiate this objective with in-context diffusion over arbitrary sets of mixed numerical and categorical target cells, without imposing an autoregressive order. We show that, under the pretraining prior, its population optimum is the posterior predictive distribution. We derive a variational upper bound on the conditional negative log-likelihood. Under the pretraining prior, the bound also controls the KL divergence from the posterior predictive distribution to the model distribution, up to a model-independent constant. Second, we introduce TabMosaic, an in-context diffusion model that realizes this objective on unseen tables. The model jointly captures dependencies among numerical and categorical cells while conditioning on the observed cells and context. To scale TabMosaic to long and incomplete tables, we develop a two-staged pretraining strategy: the model first learns from short, complete tables and is then extended to long, partially observed contexts. In the second stage, multiple corruptions share a single context encoding, reducing computation and gradient variance. Missing context cells are represented explicitly, so the model can condition directly on incomplete tables at inference time. Third, we evaluate TabMosaic on generation, imputation, and class-conditional generation across the 48 datasets from a recent tabular benchmark (Erickson et al., 2026). Against 20 baselines spanning dataset-specific generators and imputers, in-context predictors, and training-free conditional samplers, our single pretrained TabMosaic ranks first across all three tasks. We also compare TabMosaic against the exact posterior predictive distribution on synthetic priors for which the posterior is available in closed form. For imputation, we evaluate performance under different missingness patterns and mechanisms, such as settings where entire columns are missing. We conduct an ablation study to assess the contribution of the staged pretraining strategy. 2 Background and Related Work 2.1 In-context learning as posterior prediction A sequence model trained by maximum likelihood on data drawn from a mixture over latent tasks approximates, at its optimum, the Bayesian posterior predictive under that mixture: conditioning on a prompt plays the role of conditioning on observations, and the model marginalizes the latent task rather than inferring it explicitly. This is the reading of in-context learning as implicit Bayesian inference (Xie et al., 2021), and the basis of a line of work that trains sequence models on synthetic priors and compares them with the Bayes-optimal predictor on tasks whose posterior is available in closed form (Garg et al., 2022; Raventós et al., 2023; Panwar et al., 2024). Tables fit this view, since the rows of a table are exchangeable draws from an unknown mechanism, and prior-data fitted networks (Müller et al., 2022) make it concrete for one cell: a transformer pretrained on synthetic tables drawn from a prior p(ϕ)p(φ) over data-generating mechanisms approximates, for a query row with features x, label y, and context rows D, the posterior predictive distribution under the prior: p(y∣x,D)=∫p(y∣x,ϕ)p(ϕ∣D)ϕp(y x,D)\;=\; p(y x,φ)\;p(φ D)\;dφ (1) The same argument applies to any target set xSx_S, provided that the pretraining objective is the likelihood of xSx_S itself. 2.2 Generative models for tables Per-dataset models. Diffusion models pair a fixed corruption process with a learned reverse process, Gaussian for continuous data (Ho et al., 2020; Song et al., 2021; Karras et al., 2022) and masked for discrete data (Austin et al., 2021; Sahoo et al., 2024); GIDD (von Rütte et al., 2025) interpolates the mask with uniform noise, which yields a self-correcting sampler with an exact ELBO, and we use the EDM (Karras et al., 2022) and GIDD kernels (§3.2). For tables, generators of this kind are trained per dataset, in score-based, latent, mixed-type continuous-time diffusion model forms (Kim et al., 2023; Zhang et al., 2024; Shi et al., 2025) or with gradient-boosted trees in place of a network (Jolicoeur-Martineau et al., 2024); TabNAT pairs a masked transformer for discrete columns with a diffusion loss for continuous ones (Zhang et al., 2025a). Per-dataset imputers follow the same pattern (Van Buuren and Oudshoorn, 1999; Stekhoven and Bühlmann, 2012; Yoon et al., 2018; Jarrett et al., 2022; Du et al., 2024; Zhang et al., 2025b). Each recovers the conditionals over a target set in one of two ways: i) Unconditional models are conditioned after training by inpainting, which fixes the given cells at every reverse step; the result approximates the conditional of the trained joint rather than sampling it, and the released procedures cover a single target column or cells of one type (Zhang et al., 2024; Shi et al., 2025; Jolicoeur-Martineau et al., 2024); TabDDPM instead conditions on one column designated before training, the class label (Kotelnikov et al., 2023). i) TabNAT (Zhang et al., 2025a) and GReaT Borisov et al. (2022b) train conditionals of a random target set given the rest by autoregression over a random order, which imposes an order on unordered set of columns; each order is a different factorization, and no term of the objective makes them agree. Both are trained on a complete table, so the context itself cannot contain missing cells. Pretraining across tables. CTSyn (Lin et al., 2025) pretrains an autoencoder into a shared latent space and a schema-conditioned latent diffusion model across tables, and fine-tunes both on each new table; TabFORGE (Jiang et al., 2026) likewise fine-tunes per table. Both target unconditional generation, and neither answers a query on an unseen table without a gradient update. In-context predictors. Models pretrained to predict single cells also impute and generate without training on the table: missing cells are filled by chaining a point predictor over them, and rows are generated by chaining the predictor over the columns, optionally followed by rounds of masking and refilling (Hollmann et al., 2025; Zhang et al., 2025c). The objective is a sum of per-cell log-likelihoods given the unmasked ones, so no term involves a target set as a whole, and a generated row is a chain in a chosen order. Energy samplers on a classifier. TabEBM (Margeloiu et al., 2024) and TabPFGen (Ma et al., 2024) use the logits of a pretrained tabular classifier as an energy and draw class-conditional rows by Langevin dynamics; TabPFGen also imputes by running the same Langevin dynamics on the missing cells only, with the observes cells held fixed and the label given. The classification objective constrains only the class posterior and leaves the distribution over the cells free (Grathwohl et al., 2019), every query must supply a label, and the model has no likelihood. 3 A Unified Objective for Tabular Generative Tasks Setting. A tabular task instance consists of a context set D=r1,…,rnD=\r_1,…,r_n\ of n rows and a query row x=(x1,…,xF)x=(x_1,…,x_F) over the same F columns, where each ri∈ℝFr_i ^F is a row of the table whose cells may be missing, and the cells of x split into an observed part xobsx_obs and a target set S⊆1,⋯,FS \1,·s,F\ of cell indices of the row, S=Snum∪ScatS=S_num∪ S_cat of numerical and categorical indices. Multiple query rows are handled identically and independently given D, and therefore we describe a single row. We write c=(xobs,D)c=(x_obs,D) for the conditioning information. Note that the tasks we consider are instances of a single distributional object, the conditional joint distribution over the target cells, and they differ only in the choice of S, as shown in Fig. 1 (a). 3.1 One family of tasks: conditional queries over cell sets We pose all of the tabular generative tasks above as learning the family p(xS∣xobs,D)S\\,p(x_S x_obs,D)\,\_S with one model. Under a prior p(ϕ)p(φ) over data-generating mechanisms, where the rows of D and the query row are drawn i.i.d. from p(⋅∣ϕ)p(· φ), the ideal member of this family is the posterior predictive distribution of Eq. 1, generalized from a single value to an arbitrary cell set: p(xS∣xobs,D)=∫p(xS∣xobs,ϕ)p(ϕ∣D)ϕ.p(x_S x_obs,D)\;=\; p(x_S x_obs,φ)\;p(φ D)\;dφ. (2) Learning this family imposes three requirements, which drive our design: i) a valid joint distribution for every mixed-type target set S; i) a single pretrained network that serves every (S,xobs)(S,x_obs) in-context; and i) a training objective that provably targets Eq. 2. 3.2 In-context joint diffusion over cell sets Forward process. The diffusion state at time t∈[0,1]t∈[0,1] is zt=(ztnum,ztcat)∈ℝ|Snum|×∏j∈Scatjz_t=(z_t^num,z_t^cat) ^|S_num|× _j∈ S_catV_j, where jV_j is the class set of column j augmented with a mask token. We corrupt only the target cells; the observed cells and the context remain clean. Given the clean values xSx_S, the forward marginal factorizes over cells on a shared time axis, i.e., a Gaussian (EDM) kernel on numerical cells and a GIDD kernel on categorical cells: qt(zt∣xS)=∏j∈Snum(zt,j,xj,σ(t)2)⋅∏j∈ScatCat(zt,j,αtexj+βtπt),q_t(z_t x_S)= _j∈ S_numN\, (z_t,j;\,x_j,\,σ(t)^2 )\;· _j∈ S_catCat\, (z_t,j;\, _te_x_j+ _t _t ), (3) where σ(t)σ(t) is the EDM noise schedule, exj∈ℝ|j|e_x_j ^|V_j| is the one-hot vector of the clean class xjx_j, which is never mask, βt=1−αt _t=1- _t, and πt _t is the mixing distribution of GIDD (von Rütte et al., 2025), a mixture of mask and uniform noise whose uniform component lets the reverse process revise cells that have already been unmasked (See Appendix B). Reverse process. A single denoiser x^θ(zt,c,t) x_θ(z_t,c;t) reads the entire mixed state together with the conditioning c=(xobs,D)c=(x_obs,D), and outputs a scalar estimate for each numerical target cell and a class distribution for each categorical target cell. On a time grid 1=tN>⋯>t0=01=t_N>…>t_0=0, each reverse step is the posterior of the forward process with the clean target value replaced by the denoiser output, and every factor is conditioned on the full mixed state: pθ(zs∣zt,c)=∏j∈Sq(zs,j∣zt,j,x^θ,j(zt,c,t)).p_θ(z_s z_t,c)= _j∈ Sq (z_s,j z_t,j,\, x_θ,j(z_t,c;t) ). (4) The model is the composition of the reverse steps, started from the terminal distribution p(ztN)p(z_t_N), which is independent of the data (Gaussian noise on numerical cells, MASK on categorical cells): pθ(xS∣c)=∫p(ztN)∏i=N1pθ(zti−1∣zti,c)dzt1:tN.p_θ(x_S c)= p(z_t_N) _i=N^1p_θ (z_t_i-1 z_t_i,c )\,dz_t_1:t_N. (5) Proposition 1 (Validity). For every S, c, and θ, Eq. 5 defines a normalized probability distribution on the mixed product space. The proof is in Appendix A. We highlight three properties of the proposed construction. i) The joint is factorized along the time axis, not along a cell or modality ordering, so no ordering is chosen and the two modalities are coherent by construction (Proposition 1). i) Dependencies between cells, including across modalities, arise from conditioning: each factor of Eq. 4 reads the current state of every other cell, and the composition over steps transports this information. i) With a single step (N=1N=1) the model reduces to the conditionally independent product ∏j∈Spθ(xj∣c) _j∈ Sp_θ(x_j c); the iteration is what creates the coupling. 3.3 Training objective and guarantee Target patterns. During pretraining the target pattern is drawn independently of the cell values and of t: with probability 0.40.4 all query cells are targets; otherwise each cell is a target independently with rate ρ∼(0,1)ρ (0,1). A single t then corrupts all target cells. Drawing S and t independently is valid because the bound of Theorem 1 holds for every target set separately, i.e., S only selects which cells enter it and t ranges over [0,1][0,1] for each S. As a result, every combination of target set and noise level lies inside the training distribution, so a query can start its reverse process at any t and for any fraction of target cells. Loss. Let λ(σ)λ(σ) be the EDM weight and w(zt,j)w(z_t,j) be the state-dependent GIDD weights. Our pretraining loss is defined as follows: ℒ=(x∪D)∼p()Stzt∼qt[1|S|(ℒnum+ℒcat)],L=E_(x∪ D) p(D)\,E_S\,E_t\,E_z_t q_t [\, 1|S| (L_num+L_cat )\, ], (6) ℒnum=∑j∈Snumλ(σ(t))(x^θ,j−xj)2,ℒcat=∑j∈Scatw(zt,j)[DKL,j+DIS,j],L_num=\! _j∈ S_num\!\!λ(σ(t)) ( x_θ,j-x_j )^2, _cat=\! _j∈ S_cat\!\!w(z_t,j) [D_KL,j+D_IS,j ], (7) where DKL,jD_KL,j and DIS,jD_IS,j are the divergence terms of the GIDD kernel. We emphasize that the per-cell sums are not a factorization of the model: they arise since the per-step distributions in Eq. 3 and Eq. 4 factorize given ztz_t, whereas the joint distribution in Eq. 5 does not. Theorem 1 (Conditional ELBO). For fixed (x,S,c)(x,S,c), let ℓθ⋆(x,S,c) _θ(x,S,c) be the inner expectation tzt[⋅]E_t\,E_z_t[·] of Eq. 6 without the normalization by |S||S| and with the numerical weight λ replaced by the likelihood weight wlik(t)=σ′(t)/σ(t)3w_lik(t)=σ (t)/σ(t)^3; the categorical weights are already those of the likelihood bound. Then ℓθ⋆(x,S,c)≥−logpθ(xS∣c)−C _θ(x,S,c)≥- p_θ(x_S c)-C, where C is independent of θ, i.e., −ℓθ⋆−C- _θ-C is an evidence lower bound on logpθ(xS∣c) p_θ(x_S c). The proof is in Appendix A. Corollary 1 (The objective targets the posterior predictive distribution). Let tables be drawn from the prior and S independently of the cell values. Then [KL(p(xS∣xobs,D)∥pθ(xS∣xobs,D))]≤[ℓθ⋆]−C′,E [KL (p(x_S x_obs,D)\, \|\,p_θ(x_S x_obs,D) ) ]\;≤\;E [ _θ ]-C , where p(xS∣xobs,D)p(x_S x_obs,D) is the posterior predictive of Eq. 2 and C′C is independent of θ. Moreover, ℒL and [ℓθ⋆]E[ _θ] are minimized by the same denoiser: x^θ,j=[xj∣zt,c] x_θ,j=E[x_j z_t,c] for numerical cells and the posterior PMF p(xj∣zt,c)p(x_j z_t,c) for categorical cells. The proof is in Appendix A. Corollary 2 (The posterior predictive minimizes the objective). Under the assumptions of Corollary 1, the denoiser of the posterior predictive of Eq. 2, i.e., x^j⋆=[xj∣zt,c] x _j=E[x_j z_t,c\,] for numerical and p(xj∣zt,c)p(x_j z_t,c\,) for categorical cells, minimizes ℒL and [ℓθ⋆]E[ _θ], and with it the model of Eq. 5 equals the posterior predictive as N→∞N→∞. The proof is in Appendix A. Corollary 1 bounds the gap to the posterior predictive by the objective, and Corollary 2 shows that the posterior predictive attains it, i.e., it is the minimizer and not only the target. Together they make the Bayesian reading of in-context learning exact for our objective, with the cross-entropy identity relaxed to an upper bound through the diffusion ELBO. The condition on S holds by construction, since every target pattern we draw is independent of the cell values, which makes the target the posterior predictive rather than a conditional that also depends on which cells are hidden. 4 TabMosaic: A Unified In-Context Diffusion Model 4.1 Architecture The denoiser operates at the resolution of individual cells, so that every cell of the context and the query can be a target, an observation, or missing. Query rows attend to the context only, so they are answered independently and in parallel, as Eq. 2 is defined for one row at a time. Tokenization. Each cell of the context and the query row is one token: a categorical cell by a lookup of its class (or mask), a numerical cell by a linear map of its value. A learned observation-state embedding marks the cell as a target (a cell of xSx_S), observed (a cell of xobsx_obs or of D), or missing in the context (a placeholder value with a missing flag), so the target set can differ across rows within one forward pass. The time embedding is added to target tokens only, so the context representation does not depend on t. Consequently, the keys and values of the context tokens are reused across denoising steps (§4.3) and the multiple corruptions (§4.2). No token carries a column index or a role such as “label”: the label is an ordinary column. Backbone. We follow the two-axis attention of in-context tabular transformers (Hollmann et al., 2025), which alternates between the cells of a row and the rows of a column. We differ in the following aspects: i) Tokens carry no column-specific embedding, neither random per-column embeddings nor positional encodings over columns, so the network is permutation-equivariant in the columns. i) Instead of a single label head, a per-cell head, shared by all cells, returns a scalar for each numerical target and class logits for each categorical target. 4.2 Pretraining: prior, estimator, and stages Synthetic prior. We draw tables from a structural causal model (SCM) prior of TabICL (v2) (Qu et al., 2026) with two changes that follow from §3.1: a label is an ordinary column, whose cardinality follows the same sampling scheme as the categorical features, and half of the tables have a numerical label. Multi-corruption estimator. The loss of Eq. 6 is a nested expectation: over tables and target sets, and, inside, over the corruption zt∼qt(⋅∣xS)z_t q_t(· x_S). One corruption per table estimates the inner expectation without bias but with noise. To reduce this noise, for each table we draw one pair (t,S)(t,S) and K independent corruptions zt(1),…,zt(K)∼qt(⋅∣xS)z_t^(1),…,z_t^(K) q_t(· x_S), and average their losses, ℓ^K(x,S,c)=1K∑k=1K1|S|[ℒnum+ℒcat](zt(k)), _K(x,S,c)= 1K _k=1^K 1|S| [L_num+L_cat ] (z_t^(k) ), (8) which is unbiased for the inner expectation and divides the variance due to the corruption by K. The cost is far below K times that of one corruption: since query rows attend only to the context, the K copies are packed along the row axis and share one context encoding, which is most of the cost of a step in Stage 2, so only the query-side compute grows with K. Averaging the outputs instead of the losses would change the objective (See Appendix C). Two stages. Pretraining proceeds in two warm-started stages, each widening the training distribution and adding a capability. T is the number of rows per table, F is the number of features. 1. Stage 1: joint on short, complete tables. Each table has T=1,024T=1,024 rows and F∼log(1,100)F (1,100) features, and the context fraction is sampled from (0.3,0.9)U(0.3,0.9). Tables have no missing values. We train on 3232M tables with a learning rate of 8×10−48× 10^-4. Multi-corruption is not activated (K=1K=1). 2. Stage 2: long and incomplete contexts. We sample T∼log(400,10,240)T (400,10,240) rows and then F∼log(4,min(200,B/T))F (4, (200,B/T)) features under a cell budget B=250,000B=250,000, since the cost of the denoiser grows with T×FT× F. The context fraction is fixed to 0.80.8. Context cells are hidden under MCAR, MAR, or MNAR at a per-table rate ρ∼(0,ρmax)ρ (0, _ ), where ρmax _ increases linearly from 00 to 0.20.2 over the first quarter of the stage. We train on 2.562.56M tables with a learning rate of 10−410^-4 and K=4K=4. Neither stage changes the minimizer of Eq. 6: missing context cells enter as inputs, never as targets, so with an incomplete context D stands for its observed cells and their missingness pattern (See Appendix D). Optimizer and other constants are in Appendix E. 4.3 Inference: one procedure for every task 1: c=(xobs,D)c=(x_obs,D), S, grid tN>⋯>t0=0t_N>…>t_0=0, M 2: for m=1,…,Mm=1,…,M do 3: ztN,j(m)←mask,j∈Scatσ(tN)εj,j∈Snumz^(m)_t_N,j← cases mask,&j∈ S_cat\\ σ(t_N)\, _j,&j∈ S_num cases 4: for i=N,…,1i=N,…,1 do 5: zti−1(m)∼pθ(⋅∣zti(m),c)z^(m)_t_i-1 p_θ(· z^(m)_t_i,c) ⊳ Eq. 4 6: end for 7: end for 8: return zt0(m)m=1M\z^(m)_t_0\_m=1^M Algorithm 1 Unified conditional-query answering Because every generation task we consider is a conditional query against the same distribution, no task needs a procedure of its own: Algorithm 1 answers all of them with the same pretrained network, and a task is an instantiation of which cells form the target set S, where the reverse chain starts (tstartt_start), how many steps it takes (N), and how many trajectories it draws (M). Generation observes nothing and targets the whole row; imputation observes part of the row and target the rest; class-conditional generation observes the label and targets the rest. Per-task defaults are in Appendix F. Two properties make every instantiation valid and cheap (See Appendix F). i) The chain may start at any tstart≤1t_start≤ 1: since S and t are independent in pretraining, every target set was seen at every noise level, and mask lies in the support of every categorical marginal. i) The context is encoded once: query rows attend to the context but not the reverse, and the t enters only the target cells, so the context keys and values are reused across steps, query rows, and trajectories. 5 Evaluation We evaluate TabMosaic through the following research questions: (RQ1. Generative performance) How does a single pretrained model without per-table training or tuning compare with models trained separately for each table on the three tasks? (RQ2. Posterior predictive fidelity) How closely does the learned joint distribution match the posterior predictive distribution in Eq. 2, and how does this compare with the product of exact per-cell posterior predictives? (RQ3. Target-set performance) How does TabMosaic compare with per-dataset and in-context baselines as the target set varies in missing rate, mechanism, and pattern? (RQ4. Staged pretraining) How does the second pretraining stage affect performance on long and incomplete tables? Figure 2: Elo with 95%95\% bootstrap intervals for generation and imputation on the 4848 tables and class-conditional generation on the 3636 classification tables. Bar color marks default and tuned entries, name color the model family, and hatching († ) a method that did not complete every table, whose missing tables are scored as described in Appendix G. Full results are in Appendix L. 5.1 Experimental setup Datasets. All real-data experiments use 48 tables from TabArena (Erickson et al., 2026) with the official splits; every stochastic method runs with three seeds, and methods are ranked by the official Elo procedure (Appendix G). Metrics. Generation synthesizes as many rows as the training split and is scored by utility (train on synthetic, test on real, relative to training on real), α-precision, β-recall, and authenticity (Alaa et al., 2022). Class-conditional generation conditions on the training split’s own label column and is scored by utility, α-precision and β-recall averaged over classes, and label consistency, i.e., the fraction of rows that a classifier trained on real data assigns to the requested class. Imputation hides 30%30\% of the feature cells that are not already missing, completely at random, and is scored by min–max-scaled RMSE and error rate on the hidden cells and by the utility of the imputed table. Baselines. Per-table models are tuned over ten configurations where their paper tunes and run with their published configuration otherwise. For generation we compare with CTGAN, TVAE, TabDDPM, STaSy, TabSyn, TabDiff, TabNAT, ForestDiffusion, and GReaT; for imputation with a mean/mode imputer, MICE, GAIN, HyperImpute, ReMasker, DiffPuter, TabNAT, and ForestDiffusion; and for class-conditional generation with TabDDPM, CTGAN, GReaT, ForestDiffusion, TabNAT, and the training-free TabEBM and TabPFGen, each conditioning by its own mechanism (See Table 4). TabMosaic is one pretrained model used as is for every task and table. In every task we also enter the in-context predictors TabPFN-3, TabICLv2, and LimiX with their official generation and imputation procedures (Appendix H). 5.2 RQ1. Generative performance Figure 3: NLL of held-out cells against the context size n, for a small model pretrained with Eq. 6 on priors with a closed-form posterior predictive (x: numerical, c: categorical cell; Appendix I). Solid black: exact posterior predictive; dashed: M exact samples under the same KDE as the model, i.e., the floor for a sampler. Shaded: beyond the context sizes seen in pretraining. Figure 4: Time to generate one synthetic table against the number of rows and of columns (median of three runs). ×: over the one-hour limit. Reds: per-table diffusion; oranges: per-table autoregressive; blues: in-context predictors sampled column by column. Generation. TabMosaic ranks first by Elo (10991099), with the best mean rank in utility (4.14.1, against 4.64.6 for TabNAT), i.e., the rows it generates train the best predictors, whereas the per-table models, which are trained on the table, fit its support more tightly. Imputation. The margin is widest here: an Elo of 14581458, against 12011201 for TabICLv2 and 11081108 for tuned ReMasker, and a mean error of 0.1290.129, against 0.1460.146 and 0.1570.157. The gap is largest on categorical cells (0.3140.314, against 0.3750.375 for TabICLv2), where every per-table imputer except TabNAT and ForestDiffusion stays above the mode imputer (0.443) Class-conditional generation. The same model answers this task by observing the label cell. It ranks first by Elo (11631163), ahead of TabPFN-3 (11441144) and TabNAT (11221122), and its rows are about as separable by class as real rows (label consistency 0.9830.983 of the real one), whereas TabPFGen over-separates them (1.0931.093). Inference time. Fig. 4 reports the time to generate one synthetic table as the rows and the columns grow. TabMosaic takes 5353 s at 10,24010,240 rows and 3232 columns, against 66–77 min for TabPFN-3 and LimiX. The gap widens with the columns, since the in-context predictors sample one column at a time and re-encode the context for each, whereas TabMosaic denoises all target cells jointly. At 256256 columns, TabMosaic takes 88 min, TabICLv2 1616 min, and TabPFN-3 and LimiX exceed one hour. The per-table models sample in seconds once trained, but pay the training cost on every table. GReaT exceeds the limit from 10,24010,240 rows. 5.3 RQ2. Posterior predictive fidelity Figure 5: Distance to the exact posterior predictive as the target set S grows on the Dirichlet prior. (Appendix I). Corollary 1 states what the objective targets. Following Müller et al. (2022), we test it on priors whose posterior predictive is available in closed form: a Gaussian Process (GP), a GP with a Dirichlet-distributed categorical column, and a Dirichlet prior over the joint of four categorical columns (See Appendix I). Every panel carries the Monte-Carlo floor, i.e., the statistic between two sets of M exact samples, and Fig. 5 adds a factorized oracle, i.e., the product of the exact per-cell posteriors, which bounds any method that fills the cells independently. The objective reaches the posterior predictive. We pretrain a small model with our objective Eq. 6 on a mixture of these priors and evaluate it. As shown in the Fig. 3, the NLL approaches that of the exact posterior predictive on every prior, e.g., on the GP at n=1,024n=1,024, it moves from −0.53-0.53 to −0.71-0.71 against −0.77-0.77 for the exact one, and on the categorical cells of the GP+Dirichlet prior it reaches 0.1140.114 against 0.1030.103. The same holds beyond the context sizes seen in training (n=2048n=2048). The joint over a target set. We then apply the pretrained TabMosaic, which has never seen this prior, to hidden sets |S|∈1,…,4|S|∈\1,…,4\, i.e., from imputation to generation (Fig. 5). The factorized oracle degrades as |S||S| grows, i.e., the dependence among the hidden cells matters. TabMosaic stays within 1.01.0–1.31.3 times the floor for |S|≥2|S|≥ 2 without training, on par with TabNAT, which is trained on each context table. TabICLv2 is within 1.11.1–2.02.0, and TabPFN-3 and LimiX are farther (3.33.3–5.75.7 and 88–2222, against 6.56.5–10.510.5 for the oracle). TabMosaic obtains this joint in one pass without an ordering, whereas the in-context predictors assemble it column by column (Fig. 4). 5.4 RQ3. Target-set performance TabMosaic TabICLv2 TabNAT DiffPuter MCAR rate 10%10\% 0.622 0.729 0.777 0.901 30%30\% 0.675 0.812 0.860 0.966 50%50\% 0.770 0.921 0.932 0.955 70%70\% 0.843 0.974 0.993 1.005 30%30\% MAR 0.628 0.807 0.887 0.887 MNAR 0.704 0.824 0.871 0.866 columns |S|=1|S|=1 0.523 0.575 0.780 0.826 |S|=2|S|=2 0.600 0.776 0.895 0.822 |S|=4|S|=4 0.767 0.888 0.960 0.912 Table 1: Error on hidden cells relative to a mean/mode imputer (below 11 means better than mean/mode), for the four best of nine methods on the 2424 tables that all nine methods completed. All methods in Table 8. Figure 6: Stage 1 vs. Stage 2: per-table scores (top) and the Stage 2 advantage against table size (bottom). Native missing cells are pre-filled for Stage 1. Algorithm 1 takes any target set without adaptation. We vary the target set along three axes on real tables, with the training split as context and the test rows as queries (the out-of-sample setting of Zhang et al. 2025a): the missing rate 10,30,50,70%\10,30,50,70\%\, the mechanism that selects the hidden cells (MCAR, MAR, MNAR); and the pattern, random cells against |S|∈1,2,4|S|∈\1,2,4\ whole columns hidden in every query row. Table 1 reports the error on the hidden cells relative to a mean/mode imputer. TabMosaic has the lowest error in every setting. Its error grows with the missing rate (0.620.62 to 0.840.84), where the baselines fall back to the imputer (≈1≈ 1), and is flat across the mechanisms. 5.5 RQ4. Staged pretraining We compare the model after Stage 1 with the final one on the same tables and tasks to isolate what Stage 2 adds. Fig. 6 shows that Stage 2 lowers the imputation error on 4545 of the 4848 tables, from a mean of 0.1560.156 to 0.1290.129, including 1010 of the 1111 tables whose context has missing values, which Stage 1 reads only after a mean/mode pre-fill. Generation and class-conditional utility improve on 4040 of 4848 and 2626 of 3636 tables, with the gain concentrated on tables above 1010k rows, i.e., the lengths that Stage 1 never saw. 6 Conclusion We posed tabular generation, imputation, and class-conditional generation as one problem, learning the joint conditional distribution over an arbitrary set of target cells on an unseen table, and showed that a diffusion objective over the target set is the natural fit: it defines a valid joint for every mixed-type target set without an ordering, upper-bounds the conditional NLL, and, under the pretraining prior, is minimized by the posterior predictive distribution. TabMosaic, a single model pretrained with this objective in two stages, ranks first on all three tasks across 4848 datasets against 2020 baselines, approaches the exact posterior predictive on synthetic priors, and serves arbitrary target sets as is. AI use statement We used generative AI tools to discuss the experimental design, to write and run the code for experiments and figures, to check the proofs in Appendix A, and to draft and edit parts of the text. The research questions, the method, and all decisions originated with the authors, who reviewed every AI-assisted output and take full responsibility for the content. References Alaa et al. (2022) A. Alaa, B. Van Breugel, E. S. Saveliev, and M. Van Der Schaar How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International conference on machine learning, p. 290–306. Cited by: Table 5, Appendix G, §5.1. Austin et al. (2021) J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg Structured denoising diffusion models in discrete state-spaces. In NeurIPS, Cited by: Appendix B, §2.2. Borisov et al. (2022a) V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci Deep neural networks and tabular data: a survey. IEEE transactions on neural networks and learning systems 35 (6), p. 7499–7519. Cited by: §1. Borisov et al. (2022b) V. Borisov, K. Seßler, T. Leemann, M. Pawelczyk, and G. Kasneci Language models are realistic tabular data generators. arXiv preprint arXiv:2210.06280. Cited by: Appendix G, §2.2. Du et al. (2024) T. Du, L. Melis, and T. Wang Remasker: imputing tabular data with masked autoencoding. In International Conference on Learning Representations, Vol. 2024, p. 9002–9024. Cited by: Appendix G, §1, §2.2. Erickson et al. (2026) N. Erickson, L. Purucker, A. Tschalzev, D. Holzmüller, P. Desai, D. Salinas, and F. Hutter Tabarena: a living benchmark for machine learning on tabular data. Advances in Neural Information Processing Systems 38. Cited by: Appendix G, §1, §5.1. Garg et al. (2022) S. Garg, D. Tsipras, P. S. Liang, and G. Valiant What can transformers learn in-context? a case study of simple function classes. Advances in neural information processing systems 35, p. 30583–30598. Cited by: §2.1. Grathwohl et al. (2019) W. Grathwohl, K. Wang, J. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263. Cited by: §2.2. Grinsztajn et al. (2026) L. Grinsztajn, K. Flöge, O. Key, F. Birkel, P. Jund, B. Roof, M. Manium, S. B. Hoo, M. Bühler, A. Garg, et al. Tabpfn-3: technical report. arXiv preprint arXiv:2605.13986. Cited by: Appendix G, Appendix G, Appendix H. Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel Denoising diffusion probabilistic models. In NeurIPS, Cited by: §2.2. Hollmann et al. (2025) N. Hollmann, S. Müller, L. Purucker, et al. Accurate predictions on small data with a tabular foundation model. Nature. Cited by: §1, §2.2, §4.1. Jarrett et al. (2022) D. Jarrett, B. C. Cebere, T. Liu, A. Curth, and M. van der Schaar Hyperimpute: generalized iterative imputation with automatic model selection. In International conference on machine learning, p. 9916–9937. Cited by: Appendix G, §1, §2.2. Jiang et al. (2026) X. Jiang, M. Liu, N. Simidjievski, T. Klein, and M. Jamnik Tabular foundation model for generative modelling. arXiv preprint arXiv:2605.09424. Cited by: Appendix G, §1, §2.2. Jolicoeur-Martineau et al. (2024) A. Jolicoeur-Martineau, K. Fatras, and T. Kachman Generating and imputing tabular data via diffusion and flow-based gradient-boosted trees. In International conference on artificial intelligence and statistics, p. 1288–1296. Cited by: Appendix G, Appendix G, §2.2. Karras et al. (2022) T. Karras, M. Aittala, T. Aila, and S. Laine Elucidating the design space of diffusion-based generative models. In NeurIPS, Cited by: Appendix B, Appendix B, §2.2. Kim et al. (2023) J. Kim, C. Lee, and N. Park STaSy: score-based tabular data synthesis. In ICLR, Cited by: Appendix G, §1, §2.2. Kotelnikov et al. (2023) A. Kotelnikov, D. Baranchuk, I. Rubachev, and A. Babenko TabDDPM: modelling tabular data with diffusion models. In ICML, Cited by: Appendix G, §1, §2.2. Lin et al. (2025) X. Lin, C. Xu, M. Yang, and G. Cheng CTSyn: a foundation model for cross tabular data generation. In International Conference on Learning Representations, Vol. 2025, p. 47938–47959. Cited by: Appendix G, §1, §2.2. Ma et al. (2024) J. Ma, A. Dankar, G. Stein, G. Yu, and A. Caterini TabPFGen: tabular data generation with tabPFN. arXiv:2406.05216. Cited by: §2.2. Margeloiu et al. (2024) A. Margeloiu, X. Jiang, N. Simidjievski, and M. Jamnik TabEBM: a tabular data augmentation method with distinct class-specific energy-based models. In NeurIPS, Cited by: §2.2. Müller et al. (2022) S. Müller, N. Hollmann, S. P. Arango, J. Grabocka, and F. Hutter Transformers can do bayesian inference. In ICLR, Cited by: Appendix I, §2.1, §5.3. Panwar et al. (2024) M. Panwar, K. Ahuja, and N. Goyal In-context learning through the bayesian prism. In International Conference on Learning Representations, Vol. 2024, p. 49789–49843. Cited by: §2.1. Qu et al. (2025) J. Qu, D. Holzmüller, G. Varoquaux, and M. Le Morvan TabICL: a tabular foundation model for in-context learning on large data. In ICML, Cited by: §1. Qu et al. (2026) J. Qu, D. Holzmüller, G. Varoquaux, and M. L. Morvan TabICLv2: a better, faster, scalable, and open tabular foundation model. arXiv preprint arXiv:2602.11139. Cited by: Appendix E, Appendix G, Appendix G, Appendix H, §4.2. Raventós et al. (2023) A. Raventós, M. Paul, F. Chen, and S. Ganguli Pretraining task diversity and the emergence of non-bayesian in-context learning for regression. Advances in neural information processing systems 36, p. 14228–14246. Cited by: §2.1. Sahoo et al. (2024) S. S. Sahoo, M. Arriola, Y. Schiff, et al. Simple and effective masked diffusion language models. In NeurIPS, Cited by: Appendix B, §2.2. Shi et al. (2025) J. Shi, M. Xu, H. Hua, H. Zhang, S. Ermon, and J. Leskovec TabDiff: a mixed-type diffusion model for tabular data generation. In ICLR, Cited by: Appendix G, §1, §2.2. Song et al. (2021) Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole Score-based generative modeling through stochastic differential equations. In ICLR, Cited by: §A.4, §2.2. Stekhoven and Bühlmann (2012) D. J. Stekhoven and P. Bühlmann MissForest—non-parametric missing value imputation for mixed-type data. Bioinformatics 28 (1), p. 112–118. Cited by: §2.2. Van Breugel and Van Der Schaar (2024) B. Van Breugel and M. Van Der Schaar Why tabular foundation models should be a research priority. arXiv preprint arXiv:2405.01147. Cited by: §1. Van Buuren and Oudshoorn (1999) S. Van Buuren and K. Oudshoorn Flexible multivariate imputation by mice. Leiden, The Netherlands: TNO Prevention Center, p. 1–20. Cited by: Appendix G, §2.2. von Rütte et al. (2025) D. von Rütte, J. Fluri, Y. Ding, A. Orvieto, B. Schölkopf, and T. Hofmann Generalized interpolating discrete diffusion. In ICML, Cited by: §A.2, §A.4, Appendix B, §2.2, §3.2. Xie et al. (2021) S. M. Xie, A. Raghunathan, P. Liang, and T. Ma An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080. Cited by: §2.1. Yoon et al. (2018) J. Yoon, J. Jordon, and M. Schaar Gain: missing data imputation using generative adversarial nets. In International conference on machine learning, p. 5689–5698. Cited by: Appendix G, §2.2. Zhang et al. (2025a) H. Zhang, L. Fang, Q. Wu, and S. Y. Philip TabNAT: a continuous-discrete joint generative framework for tabular data.. In ICML, Cited by: Appendix G, Appendix G, §1, §2.2, §5.4. Zhang et al. (2025b) H. Zhang, L. Fang, Q. Wu, and P. Yu Diffputer: empowering diffusion models for missing data imputation. In International Conference on Learning Representations, Vol. 2025, p. 63164–63185. Cited by: Appendix J, Appendix G, §1, §2.2. Zhang et al. (2024) H. Zhang, J. Zhang, B. Srinivasan, et al. Mixed-type tabular data synthesis with score-based diffusion in latent space. In ICLR, Cited by: Appendix G, §1, §2.2. Zhang et al. (2025c) X. Zhang, G. Ren, H. Yu, H. Yuan, H. Wang, J. Li, J. Wu, L. Mo, L. Mao, M. Hao, et al. Limix: unleashing structured-data modeling capability for generalist intelligence. arXiv preprint arXiv:2509.03505. Cited by: Appendix G, Appendix G, Appendix H, §2.2. Appendix Table of Contents 1 Introduction 2 Background and Related Work 2.1 In-context learning as posterior prediction 2.2 Generative models for tables 3 A Unified Objective for Tabular Generative Tasks 3.1 One family of tasks: conditional queries over cell sets 3.2 In-context joint diffusion over cell sets 3.3 Training objective and guarantee 4 TabMosaic: A Unified In-Context Diffusion Model 4.1 Architecture 4.2 Pretraining: prior, estimator, and stages 4.3 Inference: one procedure for every task 5 Evaluation 5.1 Experimental setup 5.2 RQ1. Generative performance 5.3 RQ2. Posterior predictive fidelity 5.4 RQ3. Target-set performance 5.5 RQ4. Staged pretraining 6 Conclusion References A Proofs A.1 Proof of Proposition (validity) A.2 Proof of Theorem (conditional ELBO) A.3 Proof of Corollary (the objective targets the posterior predictive distribution) A.4 Proof of Corollary (The posterior predictive minimizes the objective) B Kernel details: EDM and GIDD C Multi-corruption estimator of the objective D Missingness in the in-context set E Pretraining details F Inference details G Evaluation protocol H Generation and imputation with in-context predictors I Distance to the posterior predictive: protocol J Target-set scenarios: protocol K What the second stage adds L Additional results Appendix A Proofs Throughout the appendix, we write c=(xobs,D)c=(x_obs,D) for the conditioning information, S=Snum∪ScatS=S_num∪ S_cat for the target set, and =ℝ|Snum|×∏j∈ScatjZ=R^|S_num|× _j∈ S_catV_j for the mixed state space, where jV_j is the class set of column j augmented with the mask token. A.1 Proof of Proposition 1 (validity) Proof. Each factor of the reverse step in Eq. 4 is a normalized Gaussian density or a normalized categorical PMF in its own cell variable, for any value of the conditioning arguments (zt,c,t)(z_t,c,t) and any parameter θ; hence Eq. 4 is a product probability measure on Z with respect to the product of the Lebesgue and counting measures, i.e., a Markov kernel. The terminal law p(ztN)p(z_t_N) is a fixed normalized product law, i.e., (0,σmax2)N(0, _ ^2) per numerical cell and the mask point mass per categorical cell. A finite composition of Markov kernels applied to a normalized initial law defines a normalized joint law of (ztN,…,zt0)(z_t_N,…,z_t_0), and marginalizing the intermediate states leaves a normalized law of zt0=xSz_t_0=x_S, which is Eq. 5. ∎ We emphasize the two distinct roles of the product structure. The loss decomposes additively over cells because the two step distributions, Eq. 3 and Eq. 4, factorize given ztz_t; the model, Eq. 5, does not factorize across cells or modalities at the endpoint, because the factors of consecutive steps are coupled through the full mixed state. A.2 Proof of Theorem 1 (conditional ELBO) Proof. Fix (x,S,c)(x,S,c) and the time grid 1=tN>⋯>t0=01=t_N>…>t_0=0. Writing q for the forward law of (zt1,…,ztN)(z_t_1,…,z_t_N) given xSx_S and applying Jensen’s inequality to Eq. 5 — the only inequality in the proof — −logpθ(xS∣c)≤q[−logp(ztN)∏i=N1pθ(zti−1∣zti,c)q(zt1,…,ztN∣xS)],- p_θ(x_S c)\;≤\;E_q\! [- p(z_t_N) _i=N^1p_θ(z_t_i-1 z_t_i,c)q(z_t_1,…,z_t_N x_S) ], (9) which rearranges into the standard decomposition ∑i=2NKL(q(zti−1∣zti,xS)∥pθ(zti−1∣zti,c))+KL(q(ztN∣xS)∥p(ztN))+[−logpθ(xS∣zt1,c)]. _i=2^NE\,KL (q(z_t_i-1 z_t_i,x_S)\, \|\,p_θ(z_t_i-1 z_t_i,c) )\;+\;E\,KL (q(z_t_N x_S)\, \|\,p(z_t_N) )\;+\;E [- p_θ(x_S z_t_1,c) ]. (10) The prior term is θ-free and vanishes as σmax _ grows (numerical) and as the terminal mixing completes (categorical); together with the discretization residue of the reconstruction term, it forms the constant C. For each step term, both arguments of the KL factorize over cells: the forward posterior by the cell-wise independence of Eq. 3, and the model by the definition of the step, Eq. 4. The KL divergence between two product measures is the sum of the per-factor KL divergences, so every step term is a sum of per-cell terms, which we group by modality. For a numerical cell j, both factors are Gaussians with the same variance, differing only in the location parameter, which is a linear function of the clean value; the per-step KL is therefore a quadratic κi(x^θ,j(zti,c,ti)−xj)2 _i\,( x_θ,j(z_t_i,c;t_i)-x_j)^2 with a θ-free coefficient κi _i determined by the schedule. Summing over the steps and passing to the continuous-time limit yields twlik(t)(x^θ,j−xj)2E_t\,w_lik(t)\,( x_θ,j-x_j)^2 with the likelihood weight wlik(t)=σ′(t)/σ(t)3w_lik(t)=σ (t)/σ(t)^3; with this weight, the numerical part of the bound is exact up to discretization. Replacing wlikw_lik by the EDM weight λ(σ(t))λ(σ(t)) (Appendix B) multiplies each (t,zt)(t,z_t)-indexed term by a positive factor. Since the minimization over x^θ,j(zt,c,t) x_θ,j(z_t,c;t) is pointwise in (t,zt)(t,z_t), every positive weighting has the same minimizer x^θ,j=[xj∣zt,c] x_θ,j=E[x_j z_t,c]; the reweighting changes the optimization geometry, not the target. For a categorical cell j, the per-step KL divergences under the GIDD kernel telescope, in the continuous-time limit, into the exact weighted form tw(zt,j)[DKL,j+DIS,j]E_t\,w(z_t,j)\,[D_KL,j+D_IS,j] of von Rütte et al. (2025), where the weights and both divergence terms are available in closed form (Appendix B). Collecting the numerical cells into ℒnumL_num and the categorical cells into ℒcatL_cat gives the statement. ∎ A.3 Proof of Corollary 1 (the objective targets the posterior predictive distribution) Proof. Let tables be drawn from the prior, (x∪D)∼p()(x∪ D) p(D), with the target pattern drawn independently of the data, S⟂(x,D)S (x,D). Taking expectations of the pointwise bound of Theorem 1, [ℓθ⋆]+C≥[−logpθ(xS∣xobs,D)].E [ _θ ]+C\;≥\;E [- p_θ(x_S x_obs,D) ]. (11) Conditionally on (S,xobs,D)(S,x_obs,D), the clean target xSx_S is distributed as the posterior predictive p(xS∣xobs,D)p(x_S x_obs,D) of Eq. 2, because the query row and the context rows are i.i.d. given the mechanism ϕφ. The cross-entropy decomposition then gives [−logpθ(xS∣xobs,D)]=[H(p(⋅∣xobs,D))]+[KL(p(⋅∣xobs,D)∥pθ(⋅∣xobs,D))],E [- p_θ(x_S x_obs,D) ]=E [H (p(· x_obs,D) ) ]+E [KL (p(· x_obs,D)\, \|\,p_θ(· x_obs,D) ) ], (12) where the entropy term is independent of θ; the constant C′C of the statement collects C and this entropy. Minimizing the loss therefore minimizes an upper bound on the expected KL divergence to the posterior predictive. That ℒL and [ℓθ⋆]E[ _θ] share their minimizer follows from the proof of Theorem 1: the two differ only in the positive per-(t,zt)(t,z_t) weights of the numerical terms. When S is a single cell and the model class is an explicit classifier, the diffusion ELBO is replaced by the exact log-likelihood and the bound collapses to the cross-entropy identity of §2.1; the corollary is the same statement with the single cell generalized to an arbitrary mixed-type cell set and the equality relaxed by the ELBO. ∎ A.4 Proof of Corollary 2 (The posterior predictive minimizes the objective) Proof. Under the assumptions, (xS,zt)(x_S,z_t) in Eq. 6 is distributed as p(xS∣c)qt(zt∣xS)p(x_S c)\,q_t(z_t x_S). Conditioned on (zt,c)(z_t,c), the numerical term is a squared loss, minimized by the posterior mean, and the categorical term is the GIDD ELBO term, which is minimized by the true posterior PMF (von Rütte et al., 2025). Both minimizers are pointwise in (zt,c,t)(z_t,c,t) and independent of the positive weights, hence they minimize ℒL and [ℓθ⋆]E[ _θ] alike. With this denoiser every factor of Eq. 4 is the true reverse transition of the forward process started from p(xS∣c)p(x_S c), and as N→∞N→∞ the per-step KL terms of the ELBO vanish (Song et al., 2021; von Rütte et al., 2025), so the composition has marginal p(xS∣c)p(x_S c) at t=0t=0. ∎ Appendix B Kernel details: EDM and GIDD Shared time. One time t∼[ϵ,1−ϵ]t [ε,1-ε] with ϵ=10−3ε=10^-3 is drawn per table and shared by the numerical and the categorical cells of its target set. The numerical noise level follows the EDM schedule σ(t)=(σmin1/ρ+t(σmax1/ρ−σmin1/ρ))ρσ(t)= ( _ ^1/ρ+t\,( _ ^1/ρ- _ ^1/ρ) )^ρ with σmin=0.002 _ =0.002, σmax=80 _ =80, and ρ=7ρ=7 (Karras et al., 2022); the categorical mixing coefficients are those of the GIDD schedule below at the same t. Numerical cells (EDM). Numerical cells follow Karras et al. (2022): the forward kernel is qσ(z∣x)=(z,x,σ2)q_σ(z x)=N(z;x,σ^2) and the denoiser uses the EDM preconditioning with data scale σd=1 _d=1, since every numerical column is mapped to a standard normal marginal by a per-column quantile transform fit on the observed context cells (Appendix E). The training weight is the EDM weight λ(σ)=(σ2+σd2)/(σσd)2λ(σ)=(σ^2+ _d^2)/(σ _d)^2, which equalizes the gradient magnitude across noise levels and, as shown in Appendix A.2, shares its minimizer [x∣z,c]E[x z,c] with the likelihood weight at every (z,σ)(z,σ). Categorical cells (GIDD). Categorical cells follow the interpolating family of von Rütte et al. (2025), whose forward marginal is Eq. 3 with a mixing distribution πt _t that blends the mask point mass with uniform noise; we use their mixing rate pu=0.1p_u=0.1 and γ=1γ=1, under which the uniform component vanishes at both ends, so that α0=1 _0=1 (clean) and the terminal law is exactly the mask point mass. The per-cell loss carries the exact continuous-time ELBO weights of their Theorem 3.7, tw(zt,j)[DKL,j+DIS,j]E_t\,w(z_t,j)\,[D_KL,j+D_IS,j], where DKL,jD_KL,j compares the true and the model one-step posteriors, DIS,jD_IS,j is the importance-sampled correction for non-mask intermediate states, and the weight w is available in closed form. Three properties matter for our construction: i) the weights are exact, so the categorical block enters Theorem 1 without any reweighting caveat; i) unlike masked (absorbing) diffusion (Austin et al., 2021; Sahoo et al., 2024), where an unmasked token can never change again, the uniform component lets the reverse process revise an already-decoded token; and i) mask is a legal state at every t and carries no information about the clean value, which is what lets a categorical target cell enter the reverse process at any t (Appendix F). Appendix C Multi-corruption estimator of the objective Theorem 1 and Corollary 1 concern the population objective (θ)=(x∪D)∼p()Stzt∼qt(⋅∣xS)[ℓθ(zt,t;xS,c)],ℓθ=1|S|[ℒnum+ℒcat],J(θ)=E_(x∪ D) p(D)\;E_S\;E_t\;E_z_t q_t(· x_S) [ _θ(z_t,t;x_S,c) ], _θ= 1|S| [L_num+L_cat ], (13) which is ℒL of Eq. 6. Training replaces the inner expectations by samples. Per optimization step, we draw G tables; for table g, we draw one time tgt_g and one target pattern SgS_g, and then K independent corruption realizations of the same (tg,Sg)(t_g,S_g): z(g,k)∼iidqtg(⋅∣xSg),^G,K(θ)=1G∑g=1G1K∑k=1Kℓθ(z(g,k),tg;xSg,cg),z^(g,k) iid q_t_g (· x_S_g ), J_G,K(θ)= 1G _g=1^G 1K _k=1^K _θ (z^(g,k),t_g;\;x_S_g,c_g ), (14) whose inner average is ℓ^K _K of Eq. 8. K=1K=1 is the standard single-corruption estimator; K>1K>1 averages the losses of the K copies, not their outputs. Proposition 2 (Unbiasedness and variance). For every K≥1K≥ 1, [^G,K]=E[ J_G,K]=J and [∇θ^G,K]=∇θE[ _θ J_G,K]= _θJ, and Var(∇θ^G,K)=1G(Vout+VinK),Vout=Var(x,D,S,t)(z[∇θℓθ]),Vin=(x,D,S,t)[Varz(∇θℓθ)].Var ( _θ J_G,K )= 1G (V_out+ V_inK ), aligned V_out&=Var_(x,D,S,t) (E_z[ _θ _θ] ),\\ V_in&=E_(x,D,S,t) [Var_z( _θ _θ) ]. aligned (15) Hence K reduces exactly the corruption-conditional component VinV_in by 1/K1/K, while G reduces both, and the minimizer is unchanged by K. Proof. Each summand of Eq. 14 is an unbiased sample of Eq. 13 by construction, which gives unbiasedness, and, under standard regularity, the same for the gradient. The variance follows from the law of total variance conditioned on (x,D,S,t)(x,D,S,t): the inner mean of K i.i.d. draws has conditional variance Varz(∇θℓθ)/KVar_z( _θ _θ)/K, and the G tables are independent. ∎ Appendix D Missingness in the in-context set The second pretraining stage additionally corrupts the conditioning side: some cells of the context rows are hidden from the model. This changes the conditioning object, not the bound. Masking distribution. Let R∈0,1|D|×FR∈\0,1\^|D|× F be an observation mask over context cells. For each table we draw a rate ρ∼(0,ρmax(s))ρ (0, _ (s)), where s is the training step and ρmax(s)=0.2min(1,s/2,500) _ (s)=0.2 (1,s/2,500); a pattern, i.e., i.i.d. cells (probability 0.50.5), a random subset of columns (0.3) or of rows (0.2) hidden at a higher per-cell rate q∼(0.5,1)q (0.5,1) and included with probability ρ/qρ/q, so that the expected fraction of hidden cells is ρ under every pattern; and a mechanism, i.e., MCAR (probability 0.50.5), MAR (0.25) or MNAR (0.25). Under MAR, the per-cell rate of a row is tilted by the within-column rank of one randomly chosen driver column, which stays fully observed; under MNAR, the rate of a cell is tilted by the rank of its own value (self-masking). The tilt amplitude is drawn uniformly in [0,2min(ρ,1−ρ)][0,2 (ρ,1-ρ)], so the expected rate remains ρ and every cell probability stays in [0,1][0,1]. The label column is masked like any other column. The masked context and the conditioning object are D~ij=Dij,Rij=1,∅,Rij=0,c~=(xobs,D~,R), D_ij= casesD_ij,&R_ij=1,\\ ,&R_ij=0, cases c= (x_obs, D,R ), (16) where ∅ is realized in the network as the mask token (categorical) or the value 00 (numerical) together with the third observation state in target,observed,missing-in-context\target,observed,missing-in-context\, so that the network reads both the observed values and the pattern R. The per-column quantile transform of the numerical columns is fit on the observed context cells only, so c~ c is a function of the observed part of D and of R. Missing cells of the query row need no new state: at inference, they are precisely the cells to be imputed, i.e., they belong to S. The training objective is Eq. 13 with c replaced by c~ c; we denote it s(θ)J_s(θ). Appendix E Pretraining details Synthetic prior. Tables are drawn from the structural-causal-model prior of TabICL v2 (Qu et al., 2026) with the two changes of §4.2: the label is an ordinary column whose cardinality follows the same law as the categorical features (up to 200200 classes), and half of the tables have a numerical label. We use the corrected sampling of categorical cardinalities of its implementation, graphs with 22 to 3232 nodes, and its filters that discard graphs and tables whose label is unpredictable from the features. Every numerical column is mapped to a standard normal marginal by a per-column quantile transform fit on the observed context cells; values are mapped back by the inverse transform at inference. Target pattern. For each table, with probability 0.40.4 all cells of the query rows are targets (generation); otherwise a rate ρ∼(0,1)ρ (0,1) is drawn and each query cell is a target independently with probability ρ. The pattern is drawn independently of the cell values and of t (§3.3). Table shapes. Stage 1 uses T=1,024T=1,024 rows per table, F∼log(1,100)F (1,100) features, and a context fraction drawn from (0.3,0.9)U(0.3,0.9); tables have no missing values. Stage 2 draws the number of rows first, T∼log(400,10,240)T (400,10,240), and then the number of features, F∼log(4,min(200,⌊B/T⌋))F (4, (200, B/T )), with the cell budget B=250,000B=250,000, so that T×F≤BT× F≤ B; the context fraction is 0.80.8. Tables in a micro-batch of four share one shape. Table 2: Pretraining configuration. Both stages use 88 GPUs. Stage 1 Stage 2 Rows T 1,0241,024 log(400,10,240) (400,10,240) Features F log(1,100) (1,100) log(4,min(200,B/T)) (4, (200,B/T)), B=250,000B=250,000 Context fraction (0.3,0.9)U(0.3,0.9) 0.80.8 Context missingness — ρ∼(0,ρmax)ρ (0, _ ), ρmax _ : 0→0.20→ 0.2 over 2,5002,500 steps Corruptions K per table 11 44 Steps / tables seen 125125k / 3232M 1010k / 2.562.56M Global batch (tables) 256256 256256 Peak learning rate 8×10−48× 10^-4 10−410^-4 Warm-up 1%1\% 2%2\% Initialization random stage-1 final Stages. Table 2 summarizes the two-stage recipe. Stage 2 warm-starts from the final stage-1 checkpoint and introduces the long tables, the context-missingness distribution of Appendix D, and the multi-corruption estimator of Appendix C with K=4K=4. The evaluated model is the stage-2 checkpoint at step 10,00010,000; no per-table training, tuning, or selection is applied anywhere in the paper. Architecture and optimization. The denoiser is the column–row attention transformer of §4.1 with 1212 blocks, width 384384, 88 heads, a feed-forward expansion of 22, pre-normalization, and normalized queries and keys (2929M parameters). Each cell token combines a type-specific value embedding (a linear map for numerical cells; a class embedding with a shared vocabulary of 201201 entries, the last of which is mask, for categorical cells), the three-state observation embedding, and, for target cells only, the embedding of t. Optimization uses Muon for the two-dimensional weights and AdamW for the remaining parameters at 0.2×0.2× the learning rate, with β1=0.9 _1=0.9, weight decay 0.010.01, a cosine schedule with linear warm-up decaying to 10−710^-7, gradient clipping at 1010, bf16 precision, and activation checkpointing. Appendix F Inference details All the tasks run Algorithm 1 with the same network; Table 3 lists the settings. The context is the training split of the table, capped at 8,1928,192 rows: a random subset for generation and a label-stratified random subset for imputation. Cells of the context that are missing in the table are presented in the missing-in-context state (Appendix D); the quantile transform is fit on the observed cells. Table 3: Settings per task. tstartt_start is the time at which the reverse process starts, N the number of steps, and M the number of trajectories per query row; the context is encoded once per query batch. Task S xobsx_obs tstartt_start Sampler (num.) / N M; output Generation all query cells — 11 deterministic / 128128 11; one row per trajectory Class-cond. gen. feature cells label cell 11 deterministic / 128128 11; one row per trajectory Imputation missing cells observed cells 0.50.5 ancestral / 1616 88; mean (num.), vote (cat.) Starting the reverse process at tstart<1t_start<1. Since S⟂tS t during pretraining, the reverse process can be entered at any tstart≤1t_start≤ 1. Categorical target cells enter at mask, which lies in the support of the forward marginal at every t and carries no information about the clean value (Appendix B). Numerical target cells enter at ztstart=x^(0)+σ(tstart)εz_t_start= x^(0)+σ(t_start)\, , where x^(0)=x^θ(σmaxε′,σmax,c) x^(0)= x_θ( _ ; _ ,c) is the output of one denoiser evaluation at the terminal noise level, i.e., the conditional mean. Imputation uses tstart=0.5t_start=0.5 with N=16N=16 ancestral steps, which we selected on a development checkpoint: per-cell error metrics saturate at a handful of steps, whereas the distribution-level metrics of generation prefer the full trajectory, so generation starts at tstart=1t_start=1 with N=128N=128. At N=1N=1 a numerical target cell returns the conditional mean and a categorical target cell is drawn from the denoiser PMF, each cell independently of the others; this is property i) of §3.2. The coupling between the cells of S is created by the iteration: with N>1N>1 every step reads the current state of every other target cell (Eq. 4). Point estimates from samples. Imputation is scored by per-cell error metrics (Appendix G), which require one value per cell. We take the mean over the M=8M=8 trajectories for a numerical cell and the majority vote for a categorical cell (ties are broken by the mode of the observed cells of the column, then the lowest class index). Distribution-level metrics use the trajectories as they are, one sample per trajectory. Generation. Generation and class-conditional generation run the full reverse process from the terminal law over all the target cells jointly; for class-conditional generation the label cell of every query row is observed and set to the requested class. The denoiser PMF of a categorical cell is used at temperature 11: lowering it trades diversity for marginal sharpness and degrades distribution-level metrics. Each evaluation seed regenerates the entire table, i.e., seeds are whole-table replicates. Cost. The context is encoded once per query batch and reused across steps, query rows, and trajectories (§4.3); query rows are processed in blocks of 8,1928,192. The median wall-clock of generation is 2525 s per table on one GPU, whereas the per-table generators train on every table, e.g., TabNAT for a median of 1,6961,696 s per table over the 4848 tables and three seeds; the scaling with rows and columns is in Fig. 4. Appendix G Evaluation protocol Benchmark. All the evaluations use the TabArena suite (Erickson et al., 2026) under its single-split (lite) protocol, i.e., the first official outer split of each dataset. Three of the 51 tables have more than 1,000 features; we exclude them from every task and every method, which leaves 4848 tables with 498498 to 100,000100,000 training rows and up to 212212 features. Class-conditional generation uses the 3636 classification tables among them. We report Elo, ranks, and means without statistical significance claims. In the per-size analyses, tables are grouped by training rows into ≤1≤ 1k (1212 tables), 11k–1010k (2121), 1010k–5050k (1111), and >50>50k (44). Scoring predictors. Utility and label consistency are measured with three predictors — LightGBM, CatBoost, and TabPFN — at their default configurations; the score of a table is the mean over the three. We use three predictors so that no single inductive bias decides the ranking; the predictors are never tuned on synthetic data. Generation. Every generator is given the official training split as its training data (per-table models) or as its context (TabMosaic and the in-context generators) and synthesizes ntrainn_train rows. Scores are i) utility: the official metric of the scoring predictors trained on synthetic rows and tested on the real test split, divided by the same metric when trained on the real training rows (the ratio is inverted for the metrics where lower is better, i.e., log loss and RMSE); i) α-precision and β-recall (Alaa et al., 2022); and i) authenticity (Alaa et al., 2022). Column-level statistics — Wasserstein-1 on min-max–scaled numerical columns, total variation on categorical columns, and the mean absolute difference of pairwise correlations — are reported as diagnostics (Appendix L). The pool comprises i) per-table generators (Kotelnikov et al., 2023; Kim et al., 2023; Zhang et al., 2024; Shi et al., 2025; Zhang et al., 2025a; Jolicoeur-Martineau et al., 2024; Borisov et al., 2022b), tuned over 1010 configurations from their published search spaces where their paper tunes and run with their published configuration otherwise (TabNAT, ForestDiffusion, and GReaT publish no search space and are run default-only); i) two reference baselines, a Gaussian mixture over the row and independent column resampling, which fix the scale of the Elo; and i) in-context generators that chain a pretrained predictor over the columns without training on the table (Grinsztajn et al., 2026; Qu et al., 2026; Zhang et al., 2025c), run with their official generation procedures (Appendix H). Every stochastic method is run with 33 seeds; each seed regenerates the whole table, scores are computed per seed, and the scores are averaged. On classification tables, TabDDPM is trained with the label as its conditioning input and samples labels from the empirical class proportions, and CTGAN samples its discrete-column condition vector from the data frequencies; on regression tables both treat the target as a numerical feature, as in their papers. Imputation. Missing cells are injected into the feature cells of the training split completely at random at a 30%30\% rate with 33 mask seeds; the label column is never injected and is passed to every imputer as an ordinary observed column; natural missingness is passed as missing, excluded from injection, and never scored. Each imputer fits on the incomplete table and fills it, and only the injected cells are scored: min-max–scaled RMSE on numerical cells and the misclassification rate on categorical cells, averaged per table; the primary score of a table is the RMSE when the table has numerical cells and the misclassification rate otherwise. The pool comprises a column mean/mode imputer (the reference baseline), MICE (Van Buuren and Oudshoorn, 1999), deep imputers (Yoon et al., 2018; Jarrett et al., 2022; Du et al., 2024; Zhang et al., 2025b), the two per-table generators whose papers give an imputation procedure (Zhang et al., 2025a; Jolicoeur-Martineau et al., 2024), and in-context imputers that fill each missing cell from the observed ones (Grinsztajn et al., 2026; Qu et al., 2026; Zhang et al., 2025c); methods whose paper tunes are also run tuned (marked (t)). TabNAT does not accept missing inputs, so we train it on the table with its missing cells filled by the mean/mode imputer and then impute them with its official procedure. TabMosaic takes the incomplete table as context, with its missing cells in the missing-in-context state, and answers every row with one conditional query over its missing cells (Appendix F). Class-conditional generation. The condition of a table is its training-split label vector, so the synthetic table has ntrainn_train rows with the real class proportions. Scores are utility, class-wise α-precision and β-recall, and label consistency, i.e., the fraction of synthetic rows that the scoring predictors assign to the requested class, relative to held-out real rows. Label agreement, the fraction of rows whose emitted label equals the requested class, is 11 for every method except CTGAN, whose condition is a soft constraint; we score CTGAN’s output as emitted and do not apply rejection sampling, which would be a procedure outside the model. Table 4 lists how each method fixes the condition. Table 4: How each method conditions on the class in class-conditional generation. Mechanism Methods Where the conditional comes from Pretrained conditional TabMosaic label cell observed; the conditional is a training term Trained conditional, per table TabNAT masked training over random orders on the table Class input at training TabDDPM the label is an input of the denoiser; not generated One model per class ForestDiffusion a separate model is trained for each class Discrete-column condition CTGAN condition vector plus a cross-entropy penalty (soft) Prompt GReaT the label appears in the prompt; permutation training Classifier energy TabEBM, TabPFGen Langevin sampling of a pretrained classifier’s energy Chained given the label in-context generators label observed; the columns are chained Elo. We follow the official TabArena procedure: every (table, metric) pair is one task, a method’s score on a task is its rank among the entered methods, and Elo ratings are fit to the pairwise win rates with 200200 bootstrap resamples over tasks for the 95%95\% intervals. Generation tasks are table × utility, α-precision, β-recall, authenticity; imputation tasks are table × RMSE on numerical cells, error rate on categorical cells, downstream utility, where a table enters only the metrics its cells admit; class-conditional tasks are table × utility, class-macro α-precision, class-macro β-recall, label-consistency gap. A method that has not been run on a table receives the reference baseline’s score on that table (column resampling or the mean/mode imputer), or, in class-conditional generation, a score just below the worst method, and is marked † . Excluded methods. TabSyn and TabDiff are not entered for imputation or class-conditional generation, because their papers claim neither task in the form we evaluate: TabSyn’s paper evaluates unconditional generation and imputation of a target column by inpainting, and TabDiff’s imputation experiments impute the target column only and require an auxiliary model trained on that column. TVAE, STaSy, and the reference baselines have no conditioning mechanism. CTSyn and TabFORGE (Lin et al., 2025; Jiang et al., 2026) have no public code at the time of writing and are discussed in §2.2 only. Appendix H Generation and imputation with in-context predictors The in-context generators and imputers of Appendix G are pretrained on one cell at a time: a label cell (Grinsztajn et al., 2026; Qu et al., 2026) or masked cells scored separately given the unmasked ones (Zhang et al., 2025c). Their official procedures obtain a row or a set of cells from these single-cell conditionals. i) Generation chains the predictor over the columns, p(x1∣D)p(x2∣x1,D)⋯p(x_1 D)\,p(x_2 x_1,D)·s, in one or several column orders and averages over the orders; the chain is a valid joint for each order, but each order is a different joint, and the conditionals were never trained together. LimiX follows the chain with rounds of masking a fraction of the generated cells and refilling them. i) Imputation fills the missing cells of a row from the observed cells, column by column with the cells already filled entering the conditioning (TabPFN-3, TabICLv2) or all at once (LimiX), and returns one value per cell, i.e., a point estimate at the mode or the mean of each conditional rather than a sample of the joint; Appendix I measures the distance of such point estimates from the joint. i) The masked objective −logqθ(xπ∣x−π,D)- q_θ(x_π x_-π,D) of LimiX is written over a masked set π, but the network outputs one distribution per masked cell given the unmasked cells, so for |π|>1|π|>1 the objective is the pseudo-likelihood-style product ∏j∈πqθ(xj∣x−π,D) _j∈πq_θ(x_j x_-π,D), which parameterizes no joint over π. In TabMosaic, by contrast, the cells of S are coupled at every reverse step and the objective is a bound on the likelihood of xSx_S itself (Theorem 1). Appendix I Distance to the posterior predictive: protocol Corollary 1 and Corollary 2 are statements about the pretraining prior, so we test them as Müller et al. (2022) test theirs, on priors whose posterior predictive is available in closed form. Two experiments use these priors differently. Fig. 3 pretrains a small model on the priors themselves and follows its checkpoints. Fig. 5 applies the pretrained TabMosaic of §4.2, which has never seen these priors, to the Dirichlet prior, so its distances include a prior-mismatch term that the corollaries do not bound. Priors. x denotes a numerical and c a categorical column. i) GP: x1,x2∼(−2,2)x_1,x_2 (-2,2) and x3=f(x1,x2)+εx_3=f(x_1,x_2)+ with f∼(0,k)f (0,k), k the RBF kernel with output scale 11, and ε∼(0,0.12) (0,0.1^2). The posterior predictive of x3x_3 given the other cells and the context is the GP posterior; the length scale is fixed to 0.60.6, so that the posterior predictive is exact. i) Dirichlet: four categorical columns c1,…,c4c_1,…,c_4 with three classes each, one table-level PMF θ∼Dir(0.3⋅)θ (0.3·1) over the 8181 joint cells, and rows drawn i.i.d. from θ; the posterior predictive of any subset of cells given the rest is Dirichlet–multinomial. i) GP+Dirichlet: one categorical column c with three classes and class proportions π∼Dir(0.3⋅)π (0.3·1), two numerical columns as in i), and x3∼(0,k)x_3 (0,k) with k=RBF(x)+[c=c′]k=RBF(x)+1[c=c ], so that rows of the same class are correlated. The posterior of x3x_3 given (x1,x2,c)(x_1,x_2,c) is the GP posterior with the extended kernel, and that of c given (x1,x2,x3)(x_1,x_2,x_3) follows by enumerating the three classes. Small model (Fig. 3). The model is a 66-block, width-256256 instance of the architecture of Appendix E (6.76.7M parameters), trained with the Stage-1 recipe on a mixture of the three priors with equal probability, context length log-uniform in [8,1,024][8,1,024], 1616 query rows, and the target pattern of Appendix E, for 6060k steps on one GPU; numerical cells are standardized with the mean and the standard deviation of the observed context cells. We evaluate the checkpoints at 1010k, 2020k, 3030k, and 6060k steps on 2020 tables per prior, 1616 query rows per table, context sizes n∈8,…,2,048n∈\8,…,2,048\, and M=512M=512 samples per query. The score is the negative log-likelihood of the true value of the hidden cells under the samples: the empirical frequency of the true cell among the samples for categorical targets and a Gaussian kernel density of the samples for a numerical target. The exact posterior predictive is scored in closed form, and the floor applies the same estimator to M of its own samples. Target sets (Fig. 5). Each query hides a set S of cells of a row of the Dirichlet prior, |S|∈1,…,4|S|∈\1,…,4\, with n=1,024n=1,024 context rows, 2020 tables, 1616 query rows per table, and M=1,024M=1,024 samples per query. The distance to the posterior predictive is the total variation to the exact joint PMF. The figure carries the Monte-Carlo floor, the same statistic between the exact posterior predictive and M of its own samples, and the factorized oracle, the product of the exact per-cell marginals, which is the best any method that fills the cells of S independently can reach. The baselines sample the same cells with their own generation procedures, conditioned on the observed cells: TabNAT is trained on the context table and samples the hidden cells autoregressively in a random order; TabPFN-3 samples them column by column from its predictive distribution at temperature 11, averaging 1010 permutations of the conditioning columns per column; TabICLv2 runs the column-wise chain of its official generation procedure over the hidden columns in the original order at temperature 11; LimiX runs the autoregressive stage of its official procedure over the hidden columns and then three rounds of masking and refilling within S. Their imputation procedures, which return one value per cell, are farther from the posterior predictive than the factorized oracle at every |S||S| and are omitted from the figure. Appendix J Target-set scenarios: protocol Unlike the imputation benchmark of Appendix G, which injects missingness into the training split and fills that split, the scenarios of §5.4 keep the training split as it is, natural missingness included, and hide cells of the test rows, so that the context is fixed and only the target set varies; the hidden values are known and scored. Hidden cells are drawn by the same generator as the benchmark (MCAR at rates 1010–70%70\%, MAR with logistic drivers among the numerical features, MNAR by self-masking; Zhang et al. 2025b), or by hiding |S|∈1,2,4|S|∈\1,2,4\ whole feature columns, chosen at random per table, in every test row. Scores are the per-cell errors of the benchmark, divided by the errors of the mean/mode imputer under the same mask; values below 11 improve on it. The eight baselines are MICE, GAIN, DiffPuter, ForestDiffusion, and TabNAT, each fit on the training split and applied to the test rows, and the in-context imputers TabPFN-3, TabICLv2, and LimiX with the training split as context. TabMosaic runs Algorithm 1 unchanged over the whole grid with the training split as context and the imputation settings of Table 3. All results use one mask seed. TabPFN-3, ForestDiffusion, and DiffPuter did not finish the larger tables within our compute budget, so Table 1 reports the 2424 tables on which all nine methods completed, and Table 8 reports every method on those tables together with the number of tables each method has results on. Appendix K What the second stage adds Stage 2 introduces long tables, the context-missingness distribution, and the multi-corruption estimator (Appendix E). We compare the final Stage-1 checkpoint with the final model on the three tasks of Appendix G, with the same inference settings. The Stage-1 model has no state for a missing context cell, so on the 1111 tables whose context has natural missingness its context is pre-filled (mean/mode for imputation; median and an explicit missing category for generation), whereas the final model reads the incomplete context directly; on these tables the two models therefore see different inputs, which we mark in Fig. 6 and exclude from the size analysis of generation and class-conditional generation. Imputation error is lower after Stage 2 on 4545 of 4848 tables (mean 0.1290.129 against 0.1560.156) and on every size group. Generation utility improves on 4040 of 4848 tables and class-conditional utility on 2626 of 3636, and both gains concentrate on tables above 1010k rows (+0.04+0.04 on 99 of 99 and 44 of 44 tables), i.e., the lengths that Stage 1 never saw. Appendix L Additional results Full results. Tables 5, 6, and 7 report every method on generation, imputation, and class-conditional generation over the 4848 and 3636 tables, including the reference baselines and the in-context methods. Table 5: Generation on the 4848 tables: Elo with 95%95\% bootstrap intervals and per-metric means over the tables each method completed (n). Utility is relative to training on real rows; α-precision, β-recall, and authenticity follow Alaa et al. (2022); W1 (median over tables) and TV are column-level distances and |Δρ|| ρ| the mean absolute difference of pairwise correlations. Time is the median wall-clock per table in seconds (training plus sampling for per-table models; inference for the others), measured on one GPU. The Elo tasks are table × utility, α-precision, β-recall, authenticity. (t) = run with the hyperparameters tuned in the original paper, which provides no default. † : not completed on every table; missing tables are filled with column resampling in the Elo. Method n Elo Utility ↑ α-prec. ↑ β-rec. ↑ Auth. ↑ W1 ↓ TV ↓ |Δρ|| ρ| ↓ Time (s) TabMosaic 48 1099 [1063, 1126] 0.914 0.790 0.408 0.569 0.054 0.038 0.038 25 TabNAT 48 1093 [1057, 1121] 0.899 0.805 0.421 0.559 0.046 0.023 0.030 1696 TabSyn 48 1060 [1030, 1079] 0.864 0.738 0.385 0.590 0.059 0.055 0.037 369 TabDiff (t) 48 1059 [1031, 1086] 0.886 0.730 0.380 0.591 0.060 0.022 0.042 2119 TabPFN-3 (chained) 48 1055 [1027, 1088] 0.891 0.662 0.353 0.632 0.079 0.081 0.059 571 ForestDiffusion† 42 1051 [1023, 1081] 0.886 0.770 0.409 0.568 0.059 0.061 0.030 1181 TabICLv2 (chained) 48 1016 [992, 1042] 0.880 0.590 0.304 0.648 0.169 0.062 0.081 72 TabDDPM (t)† 45 1004 [968, 1046] 0.818 0.660 0.371 0.618 0.263 0.095 0.137 241 GReaT† 41 1001 [980, 1024] 0.795 0.767 0.391 0.594 0.274 0.169 0.109 8106 TVAE (t)† 47 996 [978, 1016] 0.795 0.670 0.320 0.628 0.123 0.123 0.081 529 TabEBM† 32 988 [959, 1018] 0.920 0.641 0.408 0.536 0.050 0.059 0.019 183 TabPFGen† 31 964 [935, 994] 0.851 0.647 0.405 0.549 0.077 0.059 0.042 131 STaSy (t)† 47 961 [928, 989] 0.795 0.530 0.264 0.698 0.192 0.138 0.072 1871 Gaussian mixture† 47 945 [921, 970] 0.777 0.521 0.251 0.689 0.179 0.056 0.028 11 CTGAN (t) 48 937 [912, 960] 0.727 0.480 0.226 0.714 0.161 0.113 0.113 306 LimiX (chained)† 46 889 [846, 928] 0.649 0.345 0.133 0.830 0.683 0.277 0.153 219 Column resampling 48 886 [856, 918] 0.518 0.425 0.197 0.743 0.017 0.017 0.203 0 Table 6: Imputation on the 4848 tables (30%30\% of the feature cells hidden completely at random; 33 mask seeds). The Elo tasks are table × RMSE on numerical cells, error rate on categorical cells, downstream utility (125 tasks); the second Elo uses only the 41 tables every method completed (107 tasks). Error is the primary score of a table (min-max–scaled RMSE where the table has numerical cells, misclassification rate otherwise), averaged over the tables each method completed (n); rank is the mean rank over tables. RMSE and error rate are also reported separately over the tables with numerical and with categorical cells (counts in parentheses); cells with RMSE above 55 (out-of-range outputs) are excluded from the means and counted. Downstream is the utility of a predictor trained on the imputed table relative to one trained on the complete table. (t) = tuned. † : not completed on every table; missing tables are filled with the mean/mode imputer in the first Elo. Method n Elo Elo (common) Error ↓ Rank ↓ RMSE ↓ (num.) Error rate ↓ (cat.) Downstream ↑ TabMosaic 48 1458 [1389, 1521] 1458 [1382, 1539] 0.129 1.88 0.108 (44) 0.314 (33) 0.962 TabICLv2 (per cell) 48 1201 [1168, 1238] 1213 [1167, 1258] 0.146 4.38 0.126 (44) 0.375 (33) 0.839 ReMasker (t) 48 1108 [1071, 1145] 1113 [1075, 1159] 0.157 3.58 0.121 (44) 0.467 (33) 0.829 HyperImpute 48 1073 [1040, 1100] 1071 [1043, 1103] 0.163 5.19 0.129 (44) 0.473 (33) 0.828 ReMasker 48 1066 [1034, 1099] 1072 [1033, 1114] 0.167 4.90 0.133 (44) 0.483 (33) 0.827 DiffPuter 48 1063 [1019, 1104] 1086 [1045, 1125] 0.204 (3 excl.) 7.31 0.170 (44) 0.466 (33) 0.907 ForestDiffusion† 45 1043 [1008, 1087] 1037 [1003, 1078] 0.180 8.80 0.149 (41) 0.428 (31) 0.923 TabNAT 48 1029 [992, 1076] 1023 [978, 1061] 0.177 9.08 0.152 (44) 0.413 (33) 0.910 Mean/mode 48 992 [950, 1034] 994 [942, 1046] 0.195 10.96 0.165 (44) 0.443 (33) 0.904 TabPFN-3 (per cell)† 42 973 [937, 1010] 976 [931, 1018] 0.203 10.62 0.175 (39) 0.440 (30) 0.813 LimiX (per cell) 48 957 [913, 987] 952 [914, 993] 0.175 6.85 0.140 (44) 0.499 (33) 0.816 GAIN (t) 48 821 [789, 856] 823 [783, 860] 0.189 10.33 0.154 (44) 0.522 (33) 0.815 MICE (t) 48 815 [774, 847] 806 [774, 840] 0.194 (1 excl.) 10.54 0.158 (44) 0.530 (33) 0.817 GAIN 48 777 [745, 808] 776 [737, 811] 0.192 11.08 0.157 (44) 0.527 (33) 0.815 MICE 48 631 [580, 678] 597 [533, 656] 0.281 13.60 0.252 (44) 0.555 (33) 0.810 Table 7: Class-conditional generation on the 3636 classification tables: Elo with 95%95\% bootstrap intervals and means over the tables each method completed (n). The Elo tasks are table × utility, class-macro α-precision, class-macro β-recall, label-consistency gap, where the gap is the absolute difference between the label consistency of the synthetic rows and that of held-out real rows. Label consistency is the fraction of synthetic rows assigned to the requested class by the scoring predictors trained on real rows, divided by the same fraction on held-out real rows (11 = as separable as real data; above 11 = classes separated more than in the real data); macro is the unnormalized macro-average over classes; agreement is the fraction of rows whose emitted label equals the requested class. TV and W1 (median) are per-class distances, macro-averaged. (t) = the generation model with the hyperparameters tuned in its original paper. † : not completed on every table; a missing table is scored just below the worst method on that table in the Elo. Method n Elo Utility ↑ α-prec. ↑ β-rec. ↑ Consist. (rel.) Consist. (macro) Agreement TV ↓ W1 ↓ TabMosaic 36 1163 [1135, 1191] 0.926 0.624 0.349 0.983 0.675 1.00 0.064 0.096 TabPFN-3 (chained) 36 1144 [1116, 1170] 0.917 0.591 0.349 0.986 0.676 1.00 0.086 0.103 TabNAT 36 1122 [1091, 1157] 0.881 0.675 0.381 0.972 0.671 1.00 0.048 0.117 ForestDiffusion† 32 1076 [1040, 1109] 0.908 0.638 0.386 1.016 0.738 1.00 0.081 0.093 TabEBM† 32 1053 [1015, 1089] 0.922 0.526 0.363 1.008 0.723 1.00 0.078 0.073 TabICLv2 (chained) 36 1019 [991, 1048] 0.911 0.449 0.267 0.941 0.665 1.00 0.084 0.367 TabDDPM (t)† 33 989 [953, 1033] 0.826 0.570 0.345 0.909 0.669 1.00 0.124 0.546 GReaT† 30 964 [932, 999] 0.849 0.656 0.376 0.953 0.656 1.00 0.168 0.271 TabPFGen† 31 951 [909, 990] 0.851 0.507 0.339 1.093 0.818 1.00 0.088 0.176 CTGAN (t)† 35 775 [732, 810] 0.597 0.415 0.206 0.794 0.466 0.70 0.172 0.292 LimiX (chained)† 34 743 [708, 777] 0.729 0.272 0.122 0.881 0.573 1.00 0.292 0.805 Real rows (held out) 36 – – – – 1.000 0.709 – – – Target-set scenarios, all methods. Table 8 extends Table 1 to the nine methods. Table 8: Per-cell error on held-out rows relative to a mean/mode imputer (median over tables; below 11 = better than mean/mode), as the target set varies: the MCAR rate, the mechanism at 30%30\%, and the number |S||S| of whole hidden columns. 2424 tables on which every method completed (the 1515 smallest and 99 medium-sized ones; 2323 for |S|=4|S|=4). Best per row in bold. TabMosaic TabNAT ForestDiff. DiffPuter GAIN MICE TabPFN-3 TabICLv2 LimiX MCAR rate 10%10\% 0.622 0.777 1.000 0.901 1.001 1.157 0.912 0.729 0.872 30%30\% 0.675 0.860 1.082 0.966 1.028 1.233 1.098 0.812 0.916 50%50\% 0.770 0.932 1.113 0.955 1.141 1.237 1.212 0.921 0.966 70%70\% 0.843 0.993 1.182 1.005 1.290 1.296 1.328 0.974 1.016 30%30\% MAR 0.628 0.887 1.065 0.887 1.064 1.212 1.019 0.807 0.931 MNAR 0.704 0.871 1.012 0.866 1.063 1.143 1.068 0.824 0.910 Columns |S|=1|S|=1 0.523 0.780 1.019 0.826 1.010 1.193 0.785 0.575 1.551 |S|=2|S|=2 0.600 0.895 1.053 0.822 1.001 1.158 0.992 0.776 1.702 |S|=4|S|=4 0.767 0.960 1.054 0.912 1.166 1.235 1.054 0.888 1.655 Tables with results (of 4848) 48 48 36 42 48 48 28 48 48