Paper deep dive
From Data to Program: Fast & Direct Generative Program Inference from Empirical Data
Simon Klüttermann, Xueying Ding, Leman Akoglu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI, a pretrained data-to-program model that infers an explicit, executable generative program in a single forward pass. Pretrained on synthetic datasets paired with their ground-truth programs, PRODiGI accommodates diverse generative families and data dimensionalities through template prediction and non-autoregressive program parameter decoding. Its inferred programs support direct sampling, density and score evaluation, and inspection independently of the pretrained model. We further introduce program-space fine-tuning, which refines differentiable program parameters by matching generated and empirical samples while keeping model parameters intact. Experiments show that PRODiGI achieves lower average density and score MAE than existing pretrained models, while offering multi-fold speedups over its closest competitors. Program-space fine-tuning further reduces generation MMD by 84%. By turning empirical data into explicit, reusable programs, PRODiGI introduces a new direction for fast, interpretable tabular generative modeling.
Tags
Links
- Source: https://arxiv.org/abs/2609.35348v1
- Canonical: https://arxiv.org/abs/2609.35348v1
Trouble viewing inline? Open PDF directly →
Full Text
199,731 characters extracted from source content.
Expand or collapse full text
arXiv preprint FROM DATA TO PROGRAM: FAST & DIRECT GENERA- TIVE PROGRAM INFERENCE FROM EMPIRICAL DATA Simon Kl ̈ uttermann Carnegie Mellon University Xueying Ding Carnegie Mellon University Leman Akoglu Carnegie Mellon University sklutter,xding2,lakoglu@andrew.cmu.edu ABSTRACT Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI 1 , a pretrained data-to- program model that infers an explicit, executable generative program in a sin- gle forward pass. Pretrained on synthetic datasets paired with their ground-truth programs, PRODiGI accommodates diverse generative families and data dimen- sionalities through template prediction and non-autoregressive program parameter decoding. Its inferred programs support direct sampling, density and score evalu- ation, and inspection independently of the pretrained model. We further introduce program-space fine-tuning, which refines differentiable program parameters by matching generated and empirical samples while keeping model parameters in- tact. Experiments show that PRODiGI achieves lower average density and score MAE than existing pretrained models, while offering multi-fold speedups over its closest competitors. Program-space fine-tuning further reduces generation MMD by 84%. By turning empirical data into explicit, reusable programs, PRODiGI introduces a new direction for fast, interpretable tabular generative modeling. 1INTRODUCTION Probability density estimation from finite samples is a fundamental problem in statistical inference that underpins generative modeling. Classical approaches range from parametric estimation, which assumes a distributional family (e.g., Gaussian mixtures), to nonparametric methods such as kernel density estimation (KDE), which make fewer assumptions but may deteriorate in high dimensions. Several classical generative families are highly expressive: Gaussian Mixture Models (GMMs) are universal density approximators (3); Structural Causal Models (SCMs) capture a broad class of data- generating processes (1); and by Sklar’s theorem, Copulas can represent any multivariate distribu- tion through its marginals and dependence structure (34). They offer interpretable, often analytically tractable representations, however at a computational cost: Fitting such models from data may re- quire iterative optimization, combinatorial search, or model selection; e.g., EM (5) and component selection for GMMs, causal graph search for SCMs, and dependence estimation for Copulas. In this work, we take a radically different approach to the estimation of density functions. Capital- izing on the prowess of modern tabular foundation models (TFMs) (18; 12; 13; 32; 38; 28; 7), we pretrain a single encoder-decoder architecture that maps a given dataset (i.e., D = x i n i=1 with i.i.d. samples from a continuous target distribution) directly to its generative program. Our key inspiration is that TFMs are pretrained on synthetic datasets sampled from diverse data priors, where each dataset originates from a known generative process—what we call a program. Existing TFMs discard this program after sampling the dataset and train on downstream tasks such as classification, regression, or outlier detection. In contrast, we train the model to recover the as- sociated serialized program given a sampled dataset. Our proposed model, PRODiGI (PROgram Discovery for Generative Inference), thus amortizes program inference into pretraining: once pre- trained, it infers a generative program from empirical data in a single forward pass (See Figure 1). PRODiGI introduces key technical innovations, including serialized canonical program templates for GMM, SCM and Copula based data priors, a novel non-autoregressive program decoder, and a program-parameter (rather than model-parameter) based fine-tuning, besides being (to the best 1 We open-source all artifacts at https://github.com/psorus/PRODiGI 1 arXiv:2609.35348v1 [cs.LG] 28 Sep 2026 arXiv preprint a PRETRAINING Supervised training w/ ground-truth programs b INFERENCE Single forward pass c FINE-TUNING Program parameters φ tuned Data priors GMMSCMCopula serialize normalize Program P sample DATA Data Encoder K tokens DECODER H Template Predictor ̂ τ GMM (d= 2, M= 3) π= [ ? , ? , ? ] μ m = [ ? , ? ],Σ m = ? slots Non-AR Decoder Parallel slot fill-in PROGRAM ̂ P Reconstruction loss CE: template / discrete; MSE: real Template CE Ground-truth P INPUT:OUTPUT: DATA new normalize ProDiGI θ Program ̂ P φ 0 GenerateSample new data Query Density, score, etc. InspectUnderstand the data Initialize from (b) Program ̂ P φ Structure fixed Trainable φ Loss FT NLL or MMD Model parameters θ fixed DATA new Update φ Figure 1: PRODiGI overview: (a) Pretraining: Data sampled from normalized, serialized pro- grams across diverse priors is encoded for template prediction and non-autoregressive slot filling; (b) Inference: A single forward pass maps input data to a standalone generative program; (c) Fine- tuning: Inferred program’s differentiable parameters are refined, model parameters remain fixed. of our knowledge) the first pretrained model that maps (empirical) data directly to a (generative) program without explicit model fitting, search, optimization or model selection. Classical parametric density estimators fit prespecified distribution families, while neural density estimators (NDEs) use neural networks to represent more flexible densities (10; 24; 8; 23). De- spite this difference, both classes of estimators are dataset-specific: their parameters must be fitted separately to each dataset, typically by optimizing a likelihood-based objective. Closest to our work is DiScoFormer (16), a pretrained Transformer that outputs point-wise density and score vector estimates given input samples, but is not natively generative: it requires an external iterative sampler that repeatedly queries its score estimates. In addition, TabPFN’s (14) pretrained classification and regression models can be repurposed for both density estimation and generation through an autoregressive factorization of the joint density into univariate conditionals which, how- ever, requires repeated model evaluations across features. In contrast to these TFMs, PRODiGI is a plug-in density function estimator; it outputs an explicit generative program in a single forward pass, which supports density evaluation and direct sampling. (See Table 1.) Together, these distinctions position PRODiGI as a new paradigm for tabular generative modeling: As a zero-shot framework that rapidly infers a reusable program that can be directly sampled, queried, and inspected, it bridges modern density estimation with interpretable generative program- ming. Our main contributions are summarized as follows. • TFM for Generative Program Inference – without the Fit: We introduce PRODiGI, the first data-to-program pretrained model. Trained to map empirical samples from diverse data priors to underlying generative programs, PRODiGI requires no fitting or optimization at inference time: it infers a generative program from new empirical data in a single forward pass. • Program Serialization & Normalization: Programs have structure; e.g. GMMs have compo- nents, each with mean and covariance of size related to dimensionality. We represent programs with canonically serialized templates, accommodating (i) heterogeneous priors and (i) varying dimensions. We carefully normalize programs to remove sensitivity to feature location and scale. • Novel Decoder Architecture: PRODiGI consists of a data encoder and a Non-AutoRegressive (N-AR) program decoder. We depart from next-token-at-a-time decoding for (i) program tokens are not always sequentially dependent (e.g., mean vector entries are independent), and (i) AR is prone to cascading errors yielding invalid programs. Instead, PRODiGI employs (1) a template prediction head, and (2) a N-AR reconstruction head to fill-in the template slots in parallel. • Fine-tuning the Program – not the Model: Empirical data can be used for fine-tuning, however, as inference-time data lacks the ground-truth program, instead of traditional fine-tuning of the model’s own parameters, PRODiGI fine-tunes the inferred program’s differentiable parameters, to align its generated samples with the input data using a likelihood or distribution distance loss. • Fast, Effective Generation and Estimation: PRODiGI offers fast generative modeling at an order-of-magnitude speedup over existing approaches. Beyond direct sampling, its inferred pro- grams support inspection and statistical queries, including analytical evaluation for GMM- and Copula-based programs. Experiments demonstrate effective distribution matching in two-sample tests, as well as accurate point-wise and population-wide statistical estimation. 2 arXiv preprint Table 1: Comparing classical, neural, and pretrained approaches to density and generative modeling. NDEs: class of Neural Density Estimators; DDE: data-driven Deep Density Estimation (27); SNO: Score Neural Operator (21); NS: Neural Statistician (9); NNKDE: pretrained adaptive KDE (37). Last three rows describe capabilities after estimator construction; where direct implies no additional estimation procedure, such as an external sampler, numerical marginalization, etc. PropertyKDE GMMNDEsTabPFN DiSco DDE SNO NS NNKDEOurs Zero-shot/No per-dataset fit✗✓ Explicit structure and param.s✓✗✓ Input-free upon construction✗✓✗✓✗✓ Pretrn.-model-free upon constr.✓✗✓ Multiple output families✗✓ Direct generation✓△✓ ∗ ✗✓ Direct density estimation✓△✓ ∗ ✓✗✓ Direct score estimation✓△✗✓✗✓✗✓ Explicit structure refers to kernel, mixture, SCM, or Copula representations, rather than neural weights or latent contexts. Discarding input samples permits retaining a distribution, embedding or latent context. Multiple out- put families refers to the inferred representation, not the diversity of pretraining distributions.△: architecture- dependent; ∗ TabPFN uses autoregressive conditional evaluations, requiring repeated pretrained-model calls. 2RELATED WORK Density estimation and generative modeling have a broad literature, spanning classical statistical estimators and modern neural approaches. Here we review pretrained statistical models and their comparison to classical and neural estimators, while Appdx. A gives detailed related work. Pretrained Models for Generation and Estimation.Pretrained models differ substantially in their formulations of what they infer and which tasks they support directly. DiScoFormer (16) is pretrained to predict point-wise densities and score vectors through cross-attention to an input dataset, while generation requires making iterative model calls for score estimates. TabPFN (14) instead repurposes its pretrained classification and regression models through an autoregressive fac- torization, ˆp(x|D)= Q d j=1 ˆp(x j |x <j ,D). Density estimation combines the conditional predictions, while generation samples features sequentially, with repeated model calls. Other pretrained approaches include DDE (27) and NNKDE (37), which train separate models for each dimensionality. DDE predicts pointwise densities, whereas NNKDE predicts bandwidth ma- trices for a sample-centered kernel mixture supporting density evaluation, score evaluation, and direct sampling. Score Neural Operator (21) maps distribution embeddings to time-dependent score functions and generates through iterative sampling. In contrast, PRODiGI maps an input dataset to an explicit generative program directly; subsequent sampling and statistical queries execute this program without further calls to the pretrained network. Comparing Classical, Neural, and Pretrained Statistical Approaches. Table 1 highlights com- plementary properties of classical, neural, and pretrained approaches. GMMs and conventional NDEs yield input-free estimators but require dataset-specific fitting. KDE and NNKDE provide ex- plicit statistical representations while retaining observations as kernel centers. Among pretrained methods, TabPFN, DiScoFormer, and DDE require the sample context; Neural Statistician and Score Neural Operator can retain summaries instead, but still rely on shared pretrained networks. PRODiGI combines inference without per-dataset optimization, explicit statistical structure, and independence from both the original samples and the pretrained network after construction. It addi- tionally predicts programs across multiple prior families, namely GMMs, SCMs, and Copulas. These properties support fast generation, density estimation, and score-vector estimation: a single forward pass constructs a reusable program that natively supports all three tasks without further pretrained-model evaluations. The distinction concerns both construction and subsequent execution. Fitted GMMs and KDE already support direct sampling and density/score evaluation; PRODiGI combines these capabilities with pretrained inference and an input-free representation. TabPFN requires repeated conditional-model evaluations, while DiScoFormer directly predicts densities and scores but requires an external sampler for generation. Thus, PRODiGI’s computational advantage lies in avoiding per-dataset optimization and repeated pretrained-network calls, with its construction cost amortized over subsequent samples and statistical queries. 3 arXiv preprint 3 PRODiGI FOR ZERO-SHOT DATA TO PROGRAM Overview. PRODiGI maps an empirical tabular dataset directly to an executable generative pro- gram. Figure 1 outlines pretraining, inference, and fine-tuning stages. Given a datasetD =x i n i=1 , x i ∈ R d , PRODiGI first encodesD into a fixed-size representation, predicts the structural template of the underlying program, and then reconstructs its parameters using a non-autoregressive decoder: b P =F θ (D) = D θ D (bτ,E θ E (D)).(1) Here, bτ specifies the program family and structural attributes, while b P is the resulting concrete generative program. Once inferred, b P can be executed independently ofD. 3.1DATA ENCODER As datasets are unordered collections of samples, we use a Set Transformer (19) for data encoder: H = E θ E (D)∈ R K×h ,(2) where K is the number of representation tokens with dimension h. No positional encoding is applied to the input samples, so that the encoder remains permutation-invariant with respect to their ordering. The resulting K tokens provide a fixed-size representation of a dataset with arbitrary sample size. 3.2PROGRAM TEMPLATE PREDICTION A program specifies a complete generative process, including its family, structure, and distributional parameters. Because different program families have different structures and numbers of parameters, we represent program structure separately from its concrete values. We formalize a template as τ = (fam, a), where fam ∈ GMM, SCM, Copula denotes the data prior family and a represents the discrete attributes that specify its structure. The corresponding concrete program is obtained by instantiating the template with its parameter values. Heterogeneous Programs. Unlike previous work (14; 16), PRODiGI is trained with programs from three distinct priors, and tackles the challenge of unifying them through structured serialized programs. GMMs contain varying number of components, each with mean and diagonal covariance, in varying dimensions. SCMs are specified by a directed acyclic graph (DAG) among variables with structural equations specifying causal relationships. We materialize the DAG by generating a random acyclic graph, sample the inputs, edge weights, and noise variables from the standard Normal and Uniform distributions, and set the structural equations as the weighted sum followed by a random activation function. Finally, Copulas consist of marginals from a pool of parametric probability functions such as Uniform, Exponential, Student-t, etc. alongside a Gaussian copula. Appdx. B gives detailed descriptions and configurations for the data priors. Normalization. To remove sensitivity to feature-wise shifts and scaling, we train PRODiGI in standardized coordinates. Crucially, we normalize the generative programs themselves, ensuring consistency between training data and program targets. To this end, we reparameterize each sampled program to target zero mean and unit variance per feature, and then draw training data from it. This uses analytical moments for GMMs and Copula marginals and sample estimates for SCMs. Appdx. C details the transformations to normalize programs from each prior family. During inference, we standardize each input feature using its empirical mean and standard deviation, infer a normalized program with a single forward pass, and use the same statistics to transform the program back to the original feature coordinates. Serialization, Templates, and Prediction. For the priors considered here, a GMM template is identified by the dimensionality and the number of mixture components; an SCM template is iden- tified by the dimensionality and maximum number of parents per node; and a Copula template is identified by the dimensionality alone. For example, in a configuration of 50 dimensions using up to 10 components in GMMs and up to 5 parents per variable in SCMs, these choices produce |T| = 786 templates in total: 500 GMM, 236 SCM, and 50 Copula templates. To ensure a unique structural representation, we canonically serialize each program by imposing a deterministic ordering of its entities and parameters. As a result, equivalent programs differing only in arbitrary ordering (e.g., GMM components or SCM parent variable indices) map to the same template and slot layout. Appdx. D provides details on program templates, canonical serialization, and tokenization, along with concrete examples. 4 arXiv preprint Prior to decoding, a template prediction head operates on the encoded dataset representation to infer which template to instantiate, denoted as bτ = T θ T (H) ,where τ = (fam, a) , and fam∈GMM, SCM, Copula .(3) 3.3NON-AUTOREGRESSIVE (N-AR) PROGRAM DECODER Given a template τ , its serialized representation defines an ordered sequence of typed fill-in slots, S τ = (s 1 ,...,s L τ ),(4) where each slot specifies the semantic role and type of a value to be reconstructed. For instance, slots may represent mixture weights and mean entries in GMMs, parent indices and coefficients in SCMs, or marginal families and dependence parameters in Copulas. Each slot is represented by a learned embedding that encodes its semantic identity and canonical position. The slot representations are processed jointly by the decoder D θ D (·,·) through cross-attending the dataset embedding: R = D θ D (S τ ; H)∈ R L τ ×h .(5) Why N-AR Decoding. PRODiGI radically abandons autoregressive decoding; its Transformer- based decoder removes the causal mask and predicts all template slots in parallel. This is appropriate because unlike natural-language sequences, program parameters do not generally have a left-to-right dependency: for example, entries of a GMM mean vector or the marginal choices in a Copula are independently sampled. Nevertheless, self-attention allows the decoder to capture dependencies among slots when needed, such as the structural relationships within an SCM. In addition, our N-AR decoder avoids the error accumulation of autoregressive decoding, where an early mistake can propagate to subsequent tokens and ultimately produce invalid programs. Because PRODiGI’s decoding is performed against a structurally valid, syntactically typed template, the output remains consistent with the selected program structure. Specifically, for each slot s, a type-specific prediction head maps its decoder representation r s to the corresponding program value: bz s = φ t (W t r s ) ,where s∈S t , and t∈cat, int, cont ,(6) where S cat , S int , and S cont denote categorical, integer, and continuous typed slots, respectively, φ cat = φ int = softmax and φ cont is the identity. This parallel template-based decoding not only eliminates autoregressive error propagation but also substantially reduces program decoding latency. 3.4TRAINING We train all three components of PRODiGI, data encoder, template predictor and program decoder, end-to-end to recover the ground-truth program P from empirical dataD drawn from P . For each training task, we first sample a generative program uniformly at random from a synthetic prior and then draw i.i.d. samples from that program: P ∼ Unif(P) , D =x i n i=1 , x i i.i.d. ∼ P .(7) Oracle-template training. The template predictor and parameter decoder are coupled through the template: different templates can imply different types and number of parameters and thus number of slots to decode. Consequently, supervising the decoder through a predicted template would be ill-defined whenever the template prediction is incorrect. Therefore, we provide the ground-truth template τ ⋆ = τ(P) to the decoder during training: R = D θ D (S τ ⋆ ,E θ E (D)).(8) The template prediction head is trained simultaneously to recover τ ⋆ . During inference, the ground- truth template is not available, for which the predicted bτ is used. Training objective. LettingS cat , S int , andS real denote categorical, integer, and continuous val- ued slots in a template, respectively, we optimize L =L temp +L int +L cat +L cont .(9) The first term is the cross-entropy loss for the template classification head, which directly predicts the index of the ground-truth template among the|T| plausible templates. The remaining terms su- pervise the parameters of the ground-truth template, using cross-entropy for categorical and integer slots and mean-squared error for continuous-valued slots. 5 arXiv preprint InputPRODiGI-FTDisco.-FTNNKDE-FTTabPFNKDEGMM Low High Density Figure 2: Example input data vs. baseline generations, with corresponding marginals. Appdx. F presents further examples and case studies. 3.5FINE-TUNING Conventional supervised fine-tuning adapts pretrained tabular foundation models by updating their weights on labeled downstream data. For unsupervised tasks like density estimation and data gen- eration, however, only empirical samples are available, without labels or ground-truth programs. We introduce program-space fine-tuning, which adapts the parameters of the inferred generative program rather than those of the pretrained model itself. Starting from PRODiGI’s zero-shot prediction, we refine the inferred program’s differentiable pa- rameters φ via gradient descent, while keeping the pretrained model weights θ and the inferred program structure fixed. Formally, b P φ 0 := PRODiGI(X new ;θ), min φ ( − P x∈X new log b P φ (x)for GMMs & Copulas, E ̃ X∼ b P φ h L MMD (X new , ̃ X) i for SCMs (10) Starting from the inferred program parameters φ 0 , we minimize the analytical negative log- likelihood (NLL) of the empirical data for GMMs and Copulas, while for SCMs, we minimize the maximum mean discrepancy (MMD) loss (11) between samples ̃ X generated by b P φ and the empirical samples X new since exact likelihood evaluation is generally intractable. Through this op- timization, fine-tuning achieves dataset-specific adaptation using exclusively empirical samples. Importantly, this fine-tuning paradigm supports refinement across all three program families: com- ponent weights, means and covariances in GMMs, functional mechanism parameters in SCMs, and parametric marginals and dependence parameters in Copulas. Program-space fine-tuning decouples adaptation from the pretrained model. By confining optimiza- tion to the substantially smaller parameter space of the inferred generative program, it reduces gra- dient and optimizer memory and limits the degrees of freedom for dataset-specific tuning. Unlike model-weight fine-tuning, it requires no backpropagation through the pretrained model, while sub- sequent generation and estimation tasks also operate independently of the model. 4EXPERIMENTS Baselines. We compare PRODiGI against existing pretrained models DiScoFormer (16), TabPFN (14), and NNKDE (37). DiScoFormer natively supports zero-shot density and score estimation but not generation for which we employ Langevin dynamics (30). TabPFN is natively a zero-shot regres- sor/classifier; density estimation and sampling are executed auto-regressively, while score vectors are obtained by finite differentiation. NNKDE employs KDE using its zero-shot estimated sample- specific bandwidth matrices. We also fine-tune PRODiGI, DiscoFormer, and NNKDE as prescribed, variants denoted as -FT. Details on task execution mechanisms for pretrained baselines are given in Appdx. E. In addition, we compare these zero-shot models to the classical, also non-training baseline KDE (25) and to the fit-per-dataset GMM (5). For GMM, we select the number of components (up to 10) using BIC; for KDE, we tune the RBF kernel bandwidth by maximizing held-out data likelihood. Both DiScoFormer and NNKDE are trained on one fixed input dimensionality at a time, therefore, we use a separate model for each d∈1, 2, 5, 10, 20, 30, 40, 50 for these baselines and restrict our evaluation to these dimensions. Tasks and Metrics. We evaluate PRODiGI against baselines on three main tasks: (1) Generation quality via distribution match, (2) Density estimation, and (3) Score (vector) estimation. 6 arXiv preprint 10 1 10 0 10 1 10 2 Runtime (s) 2 4 6 8 MMD rank better Generation 10 1 10 0 10 1 10 2 Runtime (s) 2 4 6 MAE rank better Density evaluation 10 1 10 0 10 1 10 2 10 3 Runtime (s) 2 4 6 8 MAE rank better Score evaluation 10 1 10 0 10 1 10 2 Runtime (s) 2 4 6 8 MMD rank better Real-world generation PRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Figure 3: Performance-Inference time trade-off on Generation, Density, and Score estimation tasks on synthetic (all priors) and real-world data. PRODiGI-FT performs best on density and scoring, while PRODiGI offers the fastest generation, remaining competitive on real-world datasets. For generation, we use (1.i) the MMD statistic (11) with bandwidth σ= √ d scaling with the dimen- sion as a distribution match metric, measuring the discrepancy between i.i.d. original samples from D in ∼P and generated samples D gen ∼ b P . We also report the (1.i) rejection rate of a two-sample test as the proportion of datasets where the null hypothesis is rejected. For density estimation, we compute (2.i) MAE dense = 1 n P n i=1 | log b f(x i )− logf(x i )|, (2.i) the negative log likelihood (NLL), and also report (2.i) the Spearman correlation between the ground truth ranking of test samples by density vs. the ranking based on the estimates. Finally for score estimation, we report (3.i) MAE score = 1 n P n i=1 ∥∇ log( b f)(x i )−∇ log(f)(x i )∥ 1 , and (3.i) the cosine similarity to the ground truth vector for directional accuracy. In addition, we report running time for each task on 1000 examples each, as different baselines employ varying procedures per task. Datasets.We evaluate on 96 datasets from GMM, SCM, and Copula priors and 96 real-world datasets from MacrOData (6), balanced across available dimensions and priors. We discard class la- bels and labeled anomalies for unsupervised evaluation, and randomly subsample features exceeding pretraining dimensions. We compute empirical MMD for all datasets, while analytical ground-truth densities and score vectors are available only for GMM and Copula priors. For real data, we derive pseudo ground truth from Oracle GMMs with up to 100 components selected by BIC, retaining 37 datasets with indistinguishable GMM-generated and real samples under a two-sample test. Implementation details. We train PRODiGI using simulated datasets that contain n∈[1024, 8192] observations and dimensions d∈[1, 50], with 1500 batches per epoch and 48 program-reconstruction tasks per batch. Tasks are balanced across the GMM, SCM, and Copula priors and across their cor- responding templates. Encoder E θ E (·) produces K=20 tokens in h=512 dimensions, consists of four Set Transformer blocks with four attention heads each, and contains 12.6M parameters. Tem- plate prediction head T θ T (·) is a one-layer MLP with 7.9M parameters. Decoder D θ D (·,·) consists of three Transformer blocks with four attention heads and contains 14.0M parameters. Overall, PRODiGI has|θ|=32.5M trainable parameters. We train for 750 Epochs on 54M distinct programs, which takes 14 days on a H100 GPU, and evaluate each baseline on a l40s-48 GPU. 4.1MAIN RESULTS Mixed-prior data. Figure 2 compares generated samples on two distinct prior families. Fine-tuning substantially improves PRODiGI’s recovery of clusters and nonlinear structure. DiScoFormer strug- gles to reproduce the input distributions despite fine-tuning, while NNKDE variants generate overly diffuse samples. TabPFN and KDE closely match the inputs; although KDE simply perturbs ob- served samples using its tuned bandwidth, while TabPFN incurs substantially higher runtime. GMM struggles to capture skewed marginals smoothly. Figure 3 and Table 2 show competitive accuracy–runtime tradeoffs for PRODiGI and PRODiGI- FT across generation, density, and score estimation. For generation, PRODiGI achieves lower MMD than DiScoFormer and NNKDE in just 0.06 seconds, approximately 8× faster than the next- fastest method. Fine-tuning reduces MMD by 84% to 0.89 (×10 −3 ), approaching the per-dataset- fit GMM’s 0.78 at one-third of its runtime. TabPFN and GMM lead in mean MMD and average rank, respectively, but are substantially slower than either PRODiGI variant. Two-sample test re- jection rate (%p<0.05) closely mirrors lower MMD scores, with TabPFN (30.2%), GMM (31.3%), and PRODiGI-FT (39.6%) generating statistically indistinguishable distributions for over 60% of datasets. DiscoFormer variants rank lowest despite taking over 80 seconds per generation. 7 arXiv preprint Table 2: Performance comparison across tasks and metrics over all prior families a : (1) Generation distribution match via MMD (↓) where %p<0.05 (↓) denotes the proportion of datasets where the null hypothesis (H 0 : generated and real distributions are identical) is rejected; (2) Density estima- tion via point-wise MAE (↓), population-wide Spearman correlation (↑), and NLL of generated data (↓); (3) Score estimation via MAE (↓) and cosine similarity (↑). Average rank (↓) per metric is across methods over all datasets. Time per task (sec.) is averaged over datasets. MetricPRODiGI-FTDiscoFormer-FTNNKDE-FTTabPFNKDEGMM Generation MMD (↓)×10 3 5.46 ± 1.45 0.89 ± 0.19 41.37 ± 8.20 67.58 ± 16.51 19.51 ± 2.77 10.98 ± 1.96 0.35 ± 0.05 (↓)5.24 ± 0.95 0.78 ± 0.30 Rank (↓)4.29 ± 0.17 2.92 ± 0.13 8.05 ± 0.11 7.93 ± 0.13 6.73 ± 0.13 6.14 ± 0.17 2.39 ± 0.15 4.33 ± 0.16 2.23 ± 0.10 % p<0.05 (↓)57.3%39.6%99.0%97.9%92.7%80.2%30.2%61.5%31.3% Time (↓)0.06 ± 0.01 12.90 ± 1.29 92.70 ± 0.14 93.27 ± 0.14 0.46 ± 0.26 1.20 ± 0.02 78.09 ± 7.52 0.79 ± 0.06 37.24 ± 2.74 Rank (↓) 1.04 ± 0.02 4.98 ± 0.03 7.60 ± 0.05 8.60 ± 0.05 2.15 ± 0.06 3.86 ± 0.05 7.41 ± 0.14 3.02 ± 0.06 6.33 ± 0.05 Density MAE (↓)7.58 ± 1.12 2.19 ± 0.44 5.10 ± 0.60 4.17 ± 0.50 6.44 ± 0.80 6.25 ± 0.80 2.31 ± 0.17 7.14 ± 1.07 3.46 ± 0.60 Rank (↓)6.09 ± 0.35 2.08 ± 0.18 5.72 ± 0.22 5.23 ± 0.25 6.61 ± 0.19 6.02 ± 0.18 4.45 ± 0.41 5.73 ± 0.35 3.06 ± 0.17 Spear. (↑)0.69 ± 0.03 0.91 ± 0.01 0.71 ± 0.02 0.74 ± 0.02 0.80 ± 0.02 0.78 ± 0.02 0.84 ± 0.01 0.78 ± 0.02 0.84 ± 0.02 Rank (↓)5.93 ± 0.37 1.96 ± 0.19 7.57 ± 0.16 6.21 ± 0.21 4.97 ± 0.21 5.21 ± 0.20 4.03 ± 0.34 5.59 ± 0.34 3.54 ± 0.24 NLL (↓)5.00 ± 3.84 -0.22 ± 3.86 - b - b 3.82 ± 3.83 3.60 ± 3.83 - b 4.48 ± 4.00 0.72 ± 3.88 Rank (↓)4.12 ± 0.21 1.67 ± 0.13 - b - b 4.92 ± 0.12 4.09 ± 0.10 - b 4.11 ± 0.20 2.08 ± 0.11 Time (↓)0.31 ± 0.00 12.38 ± 1.63 0.04 ± 0.02 1.17 ± 0.57 0.60 ± 0.37 1.23 ± 0.04 203.25 ± 23.50 0.98 ± 0.10 36.92 ± 3.52 Rank (↓)2.97 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.39 ± 0.08 2.28 ± 0.09 5.80 ± 0.07 8.66 ± 0.10 4.59 ± 0.16 8.22 ± 0.05 Score MAE (↓)34.44 ± 10.96 37.35 ± 22.00 35.04 ± 10.68 34.74 ± 10.47 41.05 ± 11.81 54.56 ± 14.45 55.07 ± 21.60 37.93 ± 11.26 34.06 ± 11.17 Rank (↓)5.45 ± 0.34 2.38 ± 0.29 3.97 ± 0.15 4.09 ± 0.21 6.25 ± 0.27 7.62 ± 0.17 6.56 ± 0.31 5.52 ± 0.23 3.16 ± 0.22 Cos. (↑)0.52 ± 0.04 0.75 ± 0.04 0.63 ± 0.03 0.63 ± 0.03 0.57 ± 0.02 0.44 ± 0.02 0.56 ± 0.02 0.56 ± 0.03 0.73 ± 0.03 Rank (↓) 6.16 ± 0.34 2.70 ± 0.28 4.62 ± 0.20 4.33 ± 0.23 6.11 ± 0.24 7.18 ± 0.18 5.17 ± 0.33 6.06 ± 0.31 2.66 ± 0.19 Time (↓)0.31 ± 0.00 12.38 ± 1.62 0.04 ± 0.02 1.17 ± 0.57 0.55 ± 0.32 1.24 ± 0.04 1806.49 ± 301.72 0.99 ± 0.10 36.96 ± 3.52 Rank (↓)2.98 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.38 ± 0.07 2.27 ± 0.09 5.80 ± 0.07 8.64 ± 0.09 4.59 ± 0.15 8.25 ± 0.05 a Detailed results on individual priors and dimensions are given in Appdx. G. b NLL is not reported for DiscoFormer and TabPFN because their local density estimates are not normalized to integrate to one. In density estimation, PRODiGI-FT leads all reported metrics and average ranks, where fine-tuning reduces MAE from 7.58 to 2.19 and increasing Spearman correlation from 0.69 to 0.91. While TabPFN records a comparable mean MAE (2.31), its higher average rank (4.45 vs. 2.08) indicates lower relative performance across datasets. PRODiGI-FT is also approximately 16× faster than TabPFN and 3× faster than GMM. DiScoFormer offers the fastest density queries but lower accu- racy; moreover, its unnormalized local estimates, like TabPFN’s, preclude NLL comparison. For score estimation, PRODiGI-FT achieves the best MAE rank (2.38) and highest mean cosine similarity (0.75), with a cosine rank close to per-dataset-fit GMM’s (2.70 vs. 2.66) at roughly one- third of runtime. NNKDE-FT improves generation MMD but worsens both score metrics, while DiScoFormer-FT provides little improvement in score estimation. TabPFN’s strong generation per- formance does not extend to scores; it has the highest mean score MAE (55.07) and requires more than 30 minutes, about 146× longer than PRODiGI-FT. In summary, PRODiGI offers the fastest generation, while PRODiGI-FT leads density estimation and achieves score estimation accuracy comparable to GMM at a fraction of runtime. Although TabPFN and GMM achieve better generation quality, PRODiGI-FT better balances metric perfor- mance and execution time across all three evaluated tasks. Real-world data. To probe the boundaries of our data priors, we extend our evaluation to real- world datasets as a stress test. Figure 4 illustrates two examples (additional examples in Fig. 11): PRODiGI and PRODiGI-FT capture the main density structure and the marginals, inferring copula- based programs with Uniform and Kumaraswamy marginals (see Appdx. F.3 for the inferred and fine-tuned programs.) NNKDE and KDE also recover the broad structure but blur sharp boundaries, while NNKDE-FT yields noisier estimates. DiScoFormer variants distort the marginals, TabPFN produces spurious density spikes, and GMM introduces artificial modes. Figure 3 (right) shows that PRODiGI remains the fastest generator at 0.06 seconds, while NNKDE- FT and KDE offer competitive intermediate tradeoffs between PRODiGI’s speed and GMM’s gener- ation quality. Fine-tuning reduces PRODiGI’s MMD by approximately 48% and improves recovery of the underlying geometry, achieving lower mean MMD than TabPFN at over 8× faster runtime. Despite fine-tuning gains, real-world distributions with complex manifolds (Figure 11) substantially challenge all methods (Appdx. Table 8). Even per-dataset-fit GMM, which achieves the best mean MMD and average rank, exhibits a two-sample rejection rate of 78.3% on real-world data, up from 8 arXiv preprint InputPRODiGI-FTDisco.-FTNNKDE-FTTabPFNKDEGMM Low High Density Figure 4: Example real-world data vs. baseline generations, with corresponding marginals for (top) FinancialBeneficiary and (bottom) FinancialTransaction datasets (also see Fig. 11). 31.3% on mixed-prior data. Further broadening pretraining priors and refining fine-tuning strategies to improve real-world performance remain directions for future work. 4.2ABLATION RESULTS AND ANALYSES PRODiGI features a non-autoregressive (N-AR) decoder, and normalized and canonically serialized programs for standardized training and deterministic decoding, respectively. 0 25 Epoch 60% 80% 100% Validity AR NAR ARNAR Method 10ms 0.1s 1s Inference Time/Ds 44x faster NoYes Normalize? 0.0 0.2 0.4 0.6 RW MMD 0100200 Epoch 0.4 0.6 0.8 1.0 Float Loss Ordered No Order Figure 5: Ablations of PRODiGI (on 20-dimensional data); left to right: (1) N-AR vs AR decoding validity and speed, (2) generation MMD w/ and w/out normalization, (3) de- coding loss w/ and w/out canonical program serialization. Figure 5 shows that N-AR decoding ensures syntactically valid programs, as well as faster inference. AR de- coding, in contrast, cannot guarantee validity and is 44× slower. Program normalization during training enables varying input data scales and lowers generation MMD by 68% on real- world (RW) data. Finally, canonical serialization resolves reconstruction ambiguity; the loss plateaus prema- turely without it, as the model strug- gles to determine module sequence. 152050 Dimension 0.0 0.2 0.4 0.6 0.8 1.0 Rejection Rate 152050 Dimension 0 50 100 150 200 250 Generation time (s) PRODiGI GMM PRODiGI-FT NNKDE-FT DiscoFormer-FT KDE TabPFN 10 2 10 3 10 4 Context Samples 0.5 1 2 5 10 20 Avg MMD (×10 3 ) GMM(d10) GMM (d > 10) SCM(d10) SCM (d > 10) Copula Figure 6: (left) Two-sample rejection rate vs. input dimen- sionality d; (middle) Running time vs. d; (right) Average MMD per prior under d ≤ 10 vs. d > 10 as the number of context samples is increased. Finally, we analyze performance and runtime across input dimensionality d. Figure 6 (left) shows that the two- sample rejection rate increases with d for all methods, reflecting harder distribution matching in higher di- mensions. Regarding runtime (Figure 6, middle), PRODiGI and its fine- tuned variant scale gracefully with d. In contrast, TabPFN’s runtime grows significantly due to its autoregressive mechanism, while DiScoFormer is dominated by its iterative sampling algorithm. In addition, Figure 6 (right) shows that PRODiGI’s inference improves with larger context sample sizes, particularly for high-dimensional inputs. This behavior aligns with standard sample complexity theory, in which the required sample size scales with input dimensionality. 5CONCLUSION We introduced a data-to-program paradigm through PRODiGI, a model pretrained on diverse data priors to infer executable generative programs from empirical data in one forward pass. At its core, canonical serialization, program normalization, and non-autoregressive template decoding enable effective and efficient training and inference. Experiments show that PRODiGI achieves lower av- erage density and score errors than existing pretrained models while offering multi-fold speedups. Further, its novel program-space fine-tuning adapts only the inferred program parameters with pre- trained backbone fixed, improving distribution match by 84%. By mapping data directly to explicit statistical representations, PRODiGI presents a new direction for tabular generative modeling. 9 arXiv preprint AI USE STATEMENT In this work, we used generative AI tools to interpret results, implement and debug program code, and edit our research paper to improve readability. The remaining tasks requiring disclosure do not apply to our paper. We have reviewed all AI-assisted work: LLM-generated code was manually verified and tested for correctness. We have made sure that every AI interpretation is accurate and that readability changes do not alter the meaning of our text. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. REPRODUCIBILITY STATEMENT To support reproducibility, we publish both training and inference code, all used datasets, and the trained PRODiGI checkpoint at https://github.com/psorus/PRODiGI. We further pro- vide detailed results for each baseline in Appendix G. REFERENCES [1] Stephan Bongers, Patrick Forr ́ e, Jonas Peters, and Joris M Mooij. Foundations of structural causal models with cycles and latent variables. The Annals of Statistics, 49(5):2885–2915, 2021. [2] Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Lan- guage models are realistic tabular data generators. In The Eleventh International Conference on Learning Representations, 2023. [3] Miguel A. Carreira-Perpinan. Mode-finding for mixtures of gaussian distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1318–1323, 2002. [4] Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. Power-law distributions in empirical data. SIAM review, 51(4):661–703, 2009. [5] Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum likelihood from incom- plete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Method- ological), 39(1):1–38, 1977. [6] Xueying Ding, Simon Kl ̈ uttermann, Haomin Wen, Yilong Chen, and Leman Akoglu. MacrO- Data: New benchmarks of thousands of datasets for tabular outlier detection. arXiv Preprint, 2026. [7] Xueying Ding, Haomin Wen, Simon Kl ̈ uttermann, and Leman Akoglu. From zero to hero: Advancing zero-shot foundation models for tabular outlier detection. In Proceedings of the 43rd International Conference on Machine Learning (ICML). PMLR, 2026. [8] Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural spline flows. In Advances in Neural Information Processing Systems, volume 32, 2019. [9] Harrison Edwards and Amos Storkey. Towards a neural statistician. In International Confer- ence on Learning Representations (ICLR), 2017. [10] Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoen- coder for distribution estimation. In Proceedings of the 32nd International Conference on Machine Learning, pages 881–889, 2015. [11] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch ̈ olkopf, and Alexander J. Smola. A kernel two-sample test. In Journal of Machine Learning Research, volume 13, pages 723–773, 2012. [12] Noah Hollmann, Samuel M ̈ uller, Katharina Eggensperger, and Frank Hutter. TabPFN: A trans- former that solves small tabular classification problems in a second. In The Eleventh Interna- tional Conference on Learning Representations, 2023. 10 arXiv preprint [13] Noah Hollmann, Samuel M ̈ uller, Lennart Purucker, Arjun Krishnakumar, Max K ̈ orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025. [14] Noah Hollmann, Samuel M ̈ uller, Lennart Purucker, Arjun Krishnakumar, Max K ̈ orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025. [15] Regis Houssou, Mihai-Cezar Augustin, Efstratios Rappos, Vivien Bonvin, and Stephan Robert- Nicoud.Generation and simulation of synthetic datasets with copulas.arXiv preprint arXiv:2203.17250, 2022. [16] Vasily Ilin and Peter Sushko. Discoformer: Plug-in density and score estimation with trans- formers. In Proceedings of the International Conference on Machine Learning (ICML), 2026. [17] Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. TabDDPM: Mod- elling tabular data with diffusion models. In Proceedings of the 40th International Confer- ence on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17564–17579. PMLR, 2023. [18] Sergei O. Kurashkin, Vadim S. Tynchenko, Aleksei S. Borodulin, Vladimir A. Nelyub, Niko- lay O. Kalutsky, and Tee Connie. The current generation of tabular foundation models: A critical review. Machine Learning and Knowledge Extraction, 8(8):244, 2026. [19] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3744–3753. PMLR, 09–15 Jun 2019. [20] Daniel Lewandowski, Dorota Kurowicka, and Harry Joe. Generating random correlation matri- ces based on vines and extended onion method. Journal of Multivariate Analysis, 100(9):1989– 2001, 2009. [21] Xinyu Liao, Aoyang Qin, Jacob Seidman, Junqi Wang, Wei Wang, and Paris Perdikaris. Score neural operator: A generative model for learning and generalizing across multiple probability distributions. arXiv preprint arXiv:2410.08549, 2024. [22] Roger B Nelsen. An introduction to copulas. Springer, 2006. [23] George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1–64, 2021. [24] George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for den- sity estimation. In Advances in Neural Information Processing Systems, volume 30, 2017. [25] Emanuel Parzen. On estimation of a probability density function and mode. The Annals of Mathematical Statistics, 33(3):1065–1076, 1962. [26] Karl Pearson. Contributions to the mathematical theory of evolution. Philosophical Transac- tions of the Royal Society of London. A, 185:71–110, 1894. [27] Patrik Puchert, Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. Data-driven deep den- sity estimation. Neural Computing and Applications, 33(23):16773–16807, 2021. [28] Jingang Qu, David Holzm ̈ uller, Ga ̈ el Varoquaux, and Marine Le Morvan. Tabiclv2: A better, faster, scalable, and open tabular foundation model. arXiv preprint arXiv:2602.11139, 2026. [29] Scott Reed, Yutian Chen, Thomas Paine, A ̈ aron van den Oord, S. M. Ali Eslami, Danilo Rezende, Oriol Vinyals, and Nando de Freitas. Few-shot autoregressive density estimation: Towards learning to learn distributions. In International Conference on Learning Representa- tions (ICLR), 2018. 11 arXiv preprint [30] Gareth O. Roberts and Richard L. Tweedie. Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika, 83(1):95–110, 1996. [31] Murray Rosenblatt. Remarks on some nonparametric estimates of a density function. The Annals of Mathematical Statistics, 27(3):832–837, 1956. [32] Yuchen Shen, Haomin Wen, and Leman Akoglu. FoMo-0D: A foundation model for zero-shot tabular outlier detection. Transactions on Machine Learning Research, 2025. [33] Juntong Shi, Minkai Xu, Harper Hua, Hengrui Zhang, Stefano Ermon, and Jure Leskovec. TabDiff: A mixed-type diffusion model for tabular data generation. In The Thirteenth Interna- tional Conference on Learning Representations, 2025. [34] M Sklar. Fonctions de r ́ epartition ` a n dimensions et leurs marges. In Annales de l’ISUP, volume 8, pages 229–231, 1959. [35] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional GAN. In Advances in Neural Information Processing Systems, volume 32, 2019. [36] Hengrui Zhang, Jiani Zhang, Balasubramaniam Srinivasan, Zhengyuan Shen, Xiao Qin, Chris- tos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. In The Twelfth International Conference on Learning Representations, 2024. [37] Ruitong Zhang and Ke Deng. Adaptive kernel density estimation with pre-training. In Pro- ceedings of the STAI-X Conference on Statistics and Trustworthy AI for Cross (X)-Domain Acceleration (STAI-X), 2026. [38] Xiyuan Zhang, Danielle C. Maddix, Junming Yin, Nick Erickson, Abdul Fatir Ansari, Bo- ran Han, Shuai Zhang, Leman Akoglu, Christos Faloutsos, Michael W. Mahoney, Cuixiong Hu, Huzefa Rangwala, George Karypis, and Bernie Wang. Mitra: Mixed synthetic priors for enhancing tabular foundation models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 12 arXiv preprint APPENDIX AEXTENDED RELATED WORK Density Estimation. Density estimation has a long history, spanning classical statistical estimators and modern neural approaches. Gaussian mixture models (GMMs) date back to Pearson (26) and represent densities as weighted combinations of Gaussian components. Kernel density estimation (KDE) instead constructs a nonparametric estimate by placing smoothing kernels around observed samples (31; 25). Neural density estimators (NDEs) learn flexible density representations through neural networks, including autoregressive models that factorize the joint density into conditional distributions (10), and normalizing flows that transform a simple base distribution through invert- ible mappings (23). Conventionally, these approaches construct an estimator separately for each dataset, through parameter fitting or data-dependent smoothing. In contrast, PRODiGI amortizes this construction by mapping empirical samples directly to an explicit generative program. Tabular Generative Modeling. Tabular generative modeling aims to synthesize records that pre- serve feature distributions and dependencies across heterogeneous numerical and categorical vari- ables. Representative neural approaches include CTGAN and TVAE (35), which adapt generative adversarial networks and variational autoencoders to tabular data. More recently, TabDDPM (17) and TabDiff (33) develop diffusion processes for numerical and categorical features, while TabSyn (36) performs diffusion in a learned VAE latent space. Language-model-based approaches such as GReaT (2) instead fine-tune pretrained language models on textual representations of table rows and generate records autoregressively. These methods require dataset-specific training or fine-tuning of a neural generator. In comparison, PRODiGI infers an explicit generative program from a new dataset in a single forward pass, at the same time, its current formulation is restricted to continuous distributions and does not support categorical variables. Pretrained Models for Generation and Estimation. Pretrained tabular models differ substantially in their formulations of what they infer and which operations they support directly. DiScoFormer (16) is trained to predict density values and score vectors at query points through cross-attention to an input dataset; generation requires coupling these predictions with a sampling procedure. TabPFN (14) instead repurposes its pretrained classification and regression models through an autoregressive factorization, ˆp(x | D) = Q d j=1 ˆp(x j | x <j ,D). Density evaluation combines the conditional predictions, while generation samples features sequentially, requiring repeated model evaluations. Other approaches explicitly transfer knowledge across datasets include Neural Statistician (9), which infers a posterior over a dataset-level latent context that conditions a shared neural generator, en- abling direct sampling but requiring latent-variable marginalization for density and score evalua- tion. Few-shot autoregressive image models (29) learn densities conditioned on small support sets and generate samples sequentially. Data-driven deep density estimation (DDE) (27) pretrains on synthetic distributions to predict point-wise densities, using separate models for each dimensional- ity; sampling and score estimation require additional procedures. Score Neural Operator (21) maps distribution embeddings to time-dependent score functions and supports generation through iterative sampling, without directly predicting density values. Pretrained adaptive KDE (37) instead predicts sample-specific bandwidth matrices, while retaining KDE as the nonparametric estimator, yielding an explicit kernel mixture that supports density and score evaluation and direct sampling, while re- taining the observed samples as kernel centers. Like DDE, adaptive KDE requires training a separate model for each dimensionality. Table 3 summarizes the capabilities of pretrained models relevant to three statistical tasks; data generation, density estimation and score vector estimation. 13 arXiv preprint Table 3: Capabilities relevant to generation/samling, density estimation, and score-vector estimation. Additional procedures are distinguished from direct/native model outputs. Method / TaskGenerationDensityScore vector Neural Statistician (9)Direct conditional generation Numerical latent marginalization Numerical marginal-score est. DDE (27)External samplerDirect predictionAdditional estimation Score Neural Operator (21)Iterative samplingAdditional likelihood computation Time-conditioned score prediction b Adaptive NNKDE (37)Direct mixture samplingExplicit kernel densityAnalytic score TabPFN (14)Autoregressive samplingAutoregressive product of conditional densities Not a native output a DiScoFormer (16)External samplerDirect predictionDirect prediction PRODiGIDirect program samplingProgram evaluationProgram evaluation a TabPFN provides conditional probability predictions, not native score vectors; score evaluation requires a separately specified estimation procedure. b Score comparisons must match the target density’s noise level and input space; latent-space scores are not directly comparable to input-space scores. BDETAILS ON DATA PRIORS PRODiGI supports three different prior families, Gaussian Mixtures, Structural Causal Models, and Copulas, which we detail as follows. B.1GAUSSIAN MIXTURES We simulate data by drawing from multivariate GMMs with varying number of components and di- mensions. m-components in d dimensions; with centroids μ (k) j ∼N(0, 5) and diagonal covariance Σ (k) j ∼ U(0.5, 2) for k ∈ [m] and j ∈ [d].Each component is assigned a random weight ω (k) . The weights are sampled such that each weight is at least 1%, and the pairwise difference between any two component weights is also at least 1%. These constraints become increasingly restrictive as the number of components grows. The remaining degrees of freedom are sampled from a Dirichlet distribution with a randomly selected concentration parameter α∈ [0.1, 10.0]. We vary m∈ [1,M] and d∈ [1,D] uniformly as we draw new datasets, with M = 10 and D = 50. Table 4 provides hyperparameter configurations for sampling datasets from the GMM prior. Table 4: Hyperparameters for Gaussian Mixture Model (GMM) Data Prior HyperparameterValuesDescription m[1, 10]Number of mixture components d[1, 50]Dimensionality of data μ (k) j ∼N(0, 5)Component means Σ (k) j (0.5, 2]Diagonal variances ω(0.01, 1.0− 0.01· m·(m+1) 2 ]Component weights B.2STRUCTURAL CAUSAL MODELS Structural causal models (SCMs) represent the generative dependencies among variables through causal graphs and structural equations. An SCM is defined by a directed acyclic graph G = (V,E), where each node j ∈ V corresponds to a variable, and causal mechanisms linking them. For SCM data, we first instantiate a causal graph G as a growing random network. We iteratively add new nodes and connect them to k existing nodes. We draw k uniformly from k ∈ [0, MP] and limit the depth of such generated graphs to MD, where MD is drawn uniformly from MD ∈ [1, 5]. Each edge is assigned a random weight ω i,j ∈ [0.5, 1.5]∪ [−1.5,−0.5] and each node is assigned 14 arXiv preprint 5 0 5 1 0 1 Tanh 5 0 5 Odd (Cauchy score) 5 0 5 Even (Cauchy kernel) Figure 7: SCM activation functions used in this paper. We limit our choice to three functions to make sure their behavior is seperable. Odd represents the Cauchy score functions x 1+x 2 and Even represents the Cauchy kernel ( 1 1+x 2 ). an activation function a j , a random noise factor log(η j ) ∼ U(−1, 0), two bias variables b 0 j ,b 1 j ∼ N(0, 1) and an output scale δ j ∼ U(0.5, 1.5). This allows us to construct a structural equation for each node as X j = δ j a j X i∈Pa(X j ;G) ω i,j X i + s 0 j + ε j + s 1 j , ε j ∼N(0,η j ).(11) where Pa(X j ;G) denotes the set of parent (direct cause) variables of X j in G. Table 5 provides the hyperparameter settings of the SCM priors. We visualize each activation func- tion in Figure 7. Table 5: Hyperparameters for Structural Causal Model (SCM) based Data Priors HyperparameterValuesDescription MD[1, 5]Max. In Degree, Max. number of nodes that directly influence any node MP[1, 5]Max. Path Length, Depth of the graph d[1, 50]Dimensionality of data/Number of nodes w i,j [0.5, 1.5]∪ [−1.5, 0.5]Edge factors η i [0.1, 1.0]Gaussian noise factor besides causal relation. s 0 j ∼N(0, 1)Shift inside the activation function s 1 j ∼N(0, 1)Shift outside the activation function δ j [0.5, 1.5]Node scaling factor a(·)tanh, odd, evenActivation function for structural equations B.3COPULAS Real-world univariate features often exhibit skewed, heavy-tailed behavior (4). While SCMs induce non-Gaussian marginals, they do not provide explicit control over feature-wise distributions; we therefore adopt copula models (22; 15). By Sklar’s theorem (34), any joint CDF of continuous variablesX j d j=1 with marginals F j can be expressed as F(x 1 ,...,x d ) = C(F 1 (x 1 ),...,F d (x d )), enabling independent specification of marginals and dependence structure. We sample each fea- ture’s marginal from a diverse pool of parametric distributions (Gaussian, Uniform, Kumaraswamy, Exponential, Student’s t), and use a Gaussian copula for C. Examples of each marginal func- tion are shown in Figure 8.We further sample a correlation matrix R ∈ R d×d from the Lewandowski–Kurowicka–Joe (LKJ) distribution (20) to guarantee nontrivial correlation structures even in high dimensions. 15 arXiv preprint NormalStudent-tKumar.ExponentialUniform Figure 8: Examples of each copula marginal used in this paper. Given C(R) andF j d j=1 , data samples are generated by drawing u ∼ C(R) on [0, 1] d and trans- forming x j = F −1 j (u j ) for all j ∈ [d]. We widely vary the marginal families, their parameters, and the copula to synthesize diverse tabular datasets. Table 6 provides the hyperparameter settings of the Copula priors. Table 6: Hyperparameters for Copula Data Prior HyperparameterValuesDescription CGaussianCopula parameter family d[1, 50]Dimensionality of data/Number of marginals MarginalUniform, Normal, Exponential,Marginal family for each feature Student t, Kumaraswamy μ/σ[−5, 5]/[0.5, 2.0]Mean/Variance for each marginal ν[2.5, 10]Degrees of freedom for Student’s t distribution a,b[0.5, 5.0]Shape parameters of the Kumaraswamy distribution. CNORMALIZATION To remove variation in feature location and scale during training, we normalize each sampled gen- erative program. Specifically, we transform its parameters to target zero mean and unit variance for each generated feature. This transformation uses analytical moments for GMMs and Copula marginals, and sample estimates for SCMs. Normalizing the program itself ensures that the gener- ated data and the target program share the same coordinate system. During training, we sample training datasets from the normalized programs, which readily generate normalized datasets, and train the model to recover the corresponding normalized program. During inference, we first standardize each input feature using its empirical mean and standard deviation, then infer a normalized program with a single forward pass. Finally, we apply the inverse transformation to the inferred program using the input dataset’s empirical feature means and standard deviations, restoring the original feature locations and scales. We describe the transformations for normalizing each prior family below, assuming finite, strictly positive marginal variances. C.1GAUSSIAN MIXTURES We normalize a GMM using its global mean and feature-wise standard deviations. For component weights ω (k) , means μ (k) , and diagonal covariance matrices Σ (k) , these are μ glob = X k ω (k) μ (k) , σ 2 glob = X k ω (k) h diag Σ (k) + μ (k) −μ glob 2 i , (12) 16 arXiv preprint where diag(·) extracts the diagonal as a vector, and vector squares are element-wise. Let D be the diagonal matrix with diagonal σ glob . The normalized component parameters are μ (k) norm = D −1 μ (k) −μ glob , Σ (k) norm = D −1 Σ (k) D −1 . (13) The mixture weights remain unchanged. C.2STRUCTURAL CAUSAL MODELS For general nonlinear SCMs, feature means and standard deviations are not readily available analyt- ically. We therefore estimate them using 10,000 samples from each sampled SCM, obtaining ˆμ i and ˆσ i for feature i. For structural equations of the form (See Appendix B.2 for notation.) X j = δ j a j X i∈Pa(X j ;G) ω j,i X i + s 0 j + ε j + s 1 j , ε j ∼N(0,η j ) ,(14) Substituting X j = ˆμ j + ˆσ j X j,norm for the child and X i = ˆμ i + ˆσ i X i,norm for each parent i ∈ Pa(X j ;G) yields the normalized parameters: s 1 j,norm = s 1 j − ˆμ j ˆσ j , δ j,norm = δ j ˆσ j , η j,norm = η j ˆσ 2 j ,(15) and ω j,i,norm = ω j,i ˆσ i , s 0 j,norm = s 0 j + X i∈Pa(X j ;G) ω j,i ˆμ i .(16) These updates preserve the causal graph and express the structural equations in the normalized coordinates. Because the moments are estimated, the normalized SCM has approximately zero marginal means and unit marginal variances. C.3COPULAS For a Copula-based joint distribution, we standardize each marginal. Specifically, for each marginal F i , we analytically compute its mean μ i and standard deviation σ i from its sampled parameters, and standardize its quantile function as follows: F −1 i,norm (u) = F −1 i (u)− μ i σ i .(17) These strictly increasing affine transformations preserve the Copula. Uniform, Normal, and Exponential marginals reduce to unique standardized distributions: U(− √ 3, √ 3),N(0, 1), and E−1 with E ∼ Exp(1), respectively. Standardized Student-t marginals retain their degrees of freedom ν > 2.5, while affinely transformed Kumaraswamy marginals retain their two shape parameters. 17 arXiv preprint DPROGRAM SERIALIZATION AND TOKENIZATION D.1WHAT IS A GENERATIVE PROGRAM? A generative program is a complete specification of a data-generating process. It consists of a program family, its structural configuration, and the distributional or functional parameters required to instantiate the generator. We consider three heterogeneous program families: Gaussian Mixture Models (GMMs), Structural Causal Models (SCMs), and Copulas. We represent each program as a structured sequence of tokens, analogous to a domain-specific lan- guage (DSL). This representation preserves the semantic role of each parameter while allowing programs with different structures and dimensionalities to be represented in a common format. D.2PROGRAM SERIALIZATION Each concrete program P is serialized into an ordered sequence S(P) = (z 1 ,...,z L ), where each element corresponds to a structural field or program parameter. The serialization is designed to be human-readable while providing a canonical representation for the program decoder. Example of a GMM Program. [GMM][DIMENSIONALITY][2][NUM_COMPONENTS][4] [START_COMPONENT][0][WEIGHT][0.6647] [INDEX][0][MEAN][-0.3763][DIAG_STD][0.6549] [INDEX][1][MEAN][-0.5755][DIAG_STD][0.3401][END_COMPONENT] [START_COMPONENT][1][WEIGHT][0.2407] [INDEX][0][MEAN][0.6825][DIAG_STD][0.7333] [INDEX][1][MEAN][0.9778][DIAG_STD][0.3844] [END_COMPONENT] [START_COMPONENT][2][WEIGHT][0.0752] [INDEX][0][MEAN][1.6824][DIAG_STD][0.7456] [INDEX][1][MEAN][2.2755][DIAG_STD][0.4177] [END_COMPONENT] [START_COMPONENT][3][WEIGHT][0.0194] [INDEX][0][MEAN][1.6918][DIAG_STD][0.7129] [INDEX][1][MEAN][2.4417][DIAG_STD][0.4465] [END_COMPONENT][END_GMM] The example specifies a 2-dimensional GMM with four components. Each component has its own mean vector, covariance representation, and mixture weight. Thus, the serialization describes a specific generative model, rather than merely identifying the GMM family. 18 arXiv preprint Example of an SCM Program. [SCM][DIMENSIONALITY][6][MAX_PARENTS][3] [START_COMPONENT][INDEX][0][FUNCTION][ODD] [PARENTS][-1][0][ -1][0][ -1][0] [NOISE][0.1571][SHIFT][0.2789][-0.241][SCALE][2.5983][END_COMPONENT] [START_COMPONENT][INDEX][1][FUNCTION][EVEN] [PARENTS][-1][0][ -1][0][ -1][0] [NOISE][0.1355][SHIFT][0.3286][-2.3577][SCALE][3.7092][END_COMPONENT] [START_COMPONENT][INDEX][2][FUNCTION][TANH] [PARENTS][0][-0.5697][4][0.1237][5][0.8932] [NOISE][0.1316][SHIFT][-0.5354][0.3304][SCALE][1.2897][END_COMPONENT] [START_COMPONENT][INDEX][3][FUNCTION][ODD] [PARENTS][1][-0.1624][-1][0][ -1][0] [NOISE][0.9306][SHIFT][0.0881][-0.085][SCALE][1.0584][END_COMPONENT] [START_COMPONENT][INDEX][4][FUNCTION][EVEN] [PARENTS][-1][0][ -1][0][ -1][0] [NOISE][0.1013][SHIFT][0.7026][-2.051][SCALE][3.5272][END_COMPONENT] [START_COMPONENT][INDEX][5][FUNCTION][ODD] [PARENTS][0][-0.4509][1][-0.2481][-1][0] [NOISE][0.219][SHIFT][-0.076][0.1409][SCALE][2.4045][END_COMPONENT] [END_SCM] Here, the variables form a DAG: variables 0 and 1 and 4 are root nodes, variable 3 depends on variable 1, variable 5 depends on variables 0 and 1 and variable 2 depends on variables 0 ,4 and 5. The serialization, therefore, specifies both the causal structure and the structural equations. Example of a Copula Program. [COPULA][DIMENSIONALITY][3] [START_COMPONENT][INDEX][0][MARGINAL][EXPONENTIAL] [PARAMS][0][1][0][0][END_COMPONENT] [START_COMPONENT][INDEX][1][MARGINAL][NORMAL] [PARAMS][0][1][0][0][END_COMPONENT] [START_COMPONENT][INDEX][2][MARGINAL][NORMAL] [PARAMS][0][1][0][0][END_COMPONENT] [CORRELATIONS] [0.0748][0.3018][-0.5349] [END_CORRELATIONS][END_COPULA] This program specifies a 3-dimensional Copula through its dependence structure and three feature- wise marginal distributions. In particular, the marginal family and its parameters can differ across dimensions. D.3TEMPLATES A template captures the structure of a program while leaving its dataset-specific values unspecified. Formally, for a concrete program P , we write τ(P) = (f, a), where f is the program family and a contains the discrete structural attributes that determine the program layout. The template induces a sequence of typed fill-in slots, S τ = (s 1 ,...,s L τ ), where each slot specifies the semantic role and type of a value to be predicted. A concrete program is obtained by filling these slots with the corresponding parameter values. For example, the GMM program above gives rise to a template of the form 19 arXiv preprint Example of a GMM Template. [GMM][DIMENSIONALITY][2][NUM_COMPONENTS][4] [START_COMPONENT][0][WEIGHT][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>][END_COMPONENT] [START_COMPONENT][1][WEIGHT][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [END_COMPONENT] [START_COMPONENT][2][WEIGHT][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [END_COMPONENT] [START_COMPONENT][3][WEIGHT][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [INDEX][<int>][MEAN][<float>][DIAG_STD][<float>] [END_COMPONENT][END_GMM] The template determines the number, ordering, semantic role, and type of the decoder outputs, while the corresponding concrete program supplies their values. D.4TOKENIZATION The serialized program is decomposed into typed tokens.Categorical fields, such as GMM,DIMENSIONALITY, and WEIGHT, are represented as symbolic tokens. Discrete quantities, such as DIM, N COMP, and ID, are represented as integer slots. Vector-valued quantities, such as MEAN, are decomposed into one real-valued slot per dimension. Likewise, SCM parent references are represented as integer-valued slots. We also define special Fill-in tokens to encode templates with missing values. D.5CANONICAL ORDERING Each program family is serialized according to a canonical ordering of its fields and entities. This ensures that each dataset fits only one program in our training distribution and is required to prevent the decoder from regressing to the mean. We order GMM components by their component weights, while ensuring that two component weights differ by at least 1%, and order the parents of an SCM node. For an example of a model without such a canonical order, see Figure 5 right. Positional encoding is used for program-decoder slots, but not for the observations in the input dataset. The latter are treated as an unordered set and are encoded by the permutation-invariant Set Transformer. D.6VOCABULARY CONSTRUCTION We initialize the symbolic vocabulary through a bootstrap procedure. We sample programs from the GMM, SCM, and Copula priors, serialize and tokenize them, and collect the symbols, fields, family names, and entity types that occur. The resulting vocabulary is fixed for training and inference, and each such token is translated into a learnable 512 dimensional vector. 20 arXiv preprint EBASELINE DETAILS E.1DISCOFORMER DiscoFormer (16) is a pretrained model for density and score estimation, designed to leverage the structure of analytically tractable distributions. It is trained on a prior over Gaussian Mixture Models (GMMs), using their analytically known log densities and score functions as supervision. We follow the official implementation of DiscoFormer available at https://github.com/ Vilin97/ensemble-score-matching. Since the authors do not provide pretrained check- points, we train a separate DiscoFormer model for each dimensionality considered in our evaluation, d∈1, 2, 5, 10, 20, 30, 40, 50, using the default hyperparameters. DiscoFormer natively provides both log-density estimates and score vectors. However, it does not enforce the normalization of the estimated densities, i.e., there is no guarantee that they integrate to one. In our experiments, we observe that this normalization condition is not satisfied. Consequently, the negative log-likelihood (NLL) is not a meaningful metric for the resulting density estimates. Since DiscoFormer does not directly support sample generation, we use a Langevin chain (30) to generate samples from the provided density and score estimates. We initialize each chain using context points and run 32 parallel chains with a step size of η = 0.01. We discard the first 20,000 iterations as burn-in and subsequently retain every 10-th generated sample. Furthermore, the original paper describes a fine-tuning procedure for DiscoFormer on a given dataset, which optimizes the model by matching sample scores to those inferred from the corre- sponding density estimates. While the original paper recommends fine-tuning for up to 3 epochs per dataset, we observe modest improvements with additional training, with gains continuing up to approximately 10 epochs. We therefore report results after 10 epochs of fine-tuning for each dataset. E.2TABPFN TabPFN (12) is a pretrained transformer model for tabular prediction that performs in-context learn- ing by conditioning on a set of labeled examples. Rather than fitting a separate predictive model for each dataset, TabPFN is pretrained on a large prior over synthetic tabular datasets and uses the resulting prior to make predictions for previously unseen datasets. While TabPFN is designed pri- marily for supervised tasks such as regression and classification, several unsupervised extensions, including generation and density estimation, have also been proposed. We follow the official implementation of TabPFN available at https://github.com/ PriorLabs/TabPFN and the TabPFN extensions available at https://github.com/ PriorLabs/tabpfn-extensions. Both generation and density estimation operate by treat- ing a subset of features as inputs and predicting either the conditional density or the value of another feature. By iteratively repeating this procedure until all features have been predicted, TabPFN can generate new samples and estimate densities. To obtain score vectors, we rely on the density estimates. Since the score is defined as the gradient of the log-likelihood, we estimate it using finite differences between 2· d density predictions: (∇ log(f)(x)) i ≈ log(f)(x + ε· e i )− log(f)(x− ε· e i ) 2· ε , where e i denotes the i-th unit vector. This introduces the hyperparameter ε, which we set to ε = 0.1. While this is a relatively large value for numerical differentiation, we found in preliminary testing that smaller values of ε result in score estimates dominated by noise in the density predictions. The densities produced by TabPFN also do not integrate to one. Consequently, we do not report negative log-likelihood (NLL) values. TabPFN also supports fine-tuning on individual datasets. However, we found the computational cost of fine-tuning prohibitively high for the scope of our evaluation and therefore use the pretrained model without further fine-tuning. 21 arXiv preprint E.3GMM Gaussian Mixture Models (GMMs) provide a flexible parametric family of density models for which the density, score function, and sampling procedure are all available analytically. We fit a GMM to each dataset and use the resulting model to obtain both density and score estimates. In contrast to the pretrained approaches considered above, the GMM is fitted directly to the data using the EM algorithm (5) and does not involve any pretraining or fine-tuning. To select the number of mixture components, we consider GMMs with between 1 and 10 compo- nents and select the model with the lowest Bayesian Information Criterion (BIC). For each candidate number of components, we use five random initializations and select the resulting model according to the BIC criterion. The selected GMM provides the density and score estimates used in our eval- uation. Since both quantities are available analytically for the fitted mixture model, no numerical approximation is required. For sample generation, we first sample a mixture component according to its fitted mixing weights and then draw a sample from the corresponding Gaussian component. In contrast to the GMM components predicted by our model, this baseline GMM estimates a full covariance matrix rather than restricting the components to diagonal covariance matrices. This increases the number of parameters for a 50-dimensional GMM with 10 components from 1010 to 13260, and thus our baseline GMMs are significantly more expressive. E.4KDE Kernel Density Estimation (KDE) provides a non-parametric approach to density estimation by rep- resenting the density as a mixture of kernel functions centered at the observed data points. We select the KDE bandwidth using 5-fold cross-validation. Specifically, we consider bandwidth scaling factors α ∈ 0.1, 0.2, 0.3, 0.5, 0.75, 1.0, 2.0, 3.0, 5.0, 7.5, 10.0 and select the value that maximizes the average log-likelihood on the held-out folds. The KDE density and score are computed analytically according to the corresponding Gaussian mixture formulation, treating each observed data point as the center of a Gaussian component. Con- sequently, the density and score expressions follow directly from those of a Gaussian mixture model, with the KDE bandwidth determining the covariance of each component. To generate new samples, we first select a data point from the observed data uniformly at random and then add Gaussian noise with the selected KDE bandwidth. Thus, each generated sample corre- sponds to a draw from one of the Gaussian kernels comprising the KDE. E.5NNKDE Neural Network Guided Kernel Density Estimation (NNKDE) (37) is a non-parametric density esti- mation method that uses a neural network to predict a local bandwidth for each observed data point. We use the official code and pretrained NNKDE models corresponding to each dimensionality con- sidered in our evaluation, except for a 40-dimensional model, which we train from scratch using the provided code and hyperparameters. Before applying NNKDE, we whiten each dataset using a ZCA whitening transformation and transform the resulting density and score estimates back to the original data space. While the original paper does not propose such a whitening transformation, we found it necessary to achieve competitive performance with NNKDE. For each training point, the neural network predicts a local covariance matrix, which defines the cor- responding Gaussian kernel. The NNKDE density is then obtained as the mixture of these Gaussian kernels, with each training point serving as a kernel center. The density and score are computed ana- lytically from the resulting Gaussian mixture, following the same formulation as for a GMM with a Gaussian component centered at each training point, but with a distinct, locally adapted covariance matrix for each component. To generate new samples, we first select a training point uniformly at random and then draw Gaus- sian noise using the corresponding local covariance matrix. The generated sample is given by the 22 arXiv preprint selected training point plus this noise, followed by the inverse whitening transformation to map the sample back to the original data space. No fine-tuning is performed. The NNKDE model also supports dataset-specific fine-tuning through a global bandwidth-scale cal- ibration procedure (Eq. 9–10 in the original paper). Specifically, a single global scale parameter is optimized by minimizing the self-exclusive leave-one-out (LOO) negative log-likelihood (NLL) of the target dataset, thereby rescaling the local bandwidths predicted by the pretrained model. We per- form this optimization for 1000 gradient descent steps. This remains computationally inexpensive in our experiments and provides sufficient iterations for the optimization to converge reliably. 23 arXiv preprint FADDITIONAL EXPERIMENTS F.1CASE STUDIES We investigate whether the quality of the reconstructed datasets extends beyond matching the ob- served data distribution to recovering the underlying distribution parameters. Figure 9 presents several examples of prior dataset reconstructions. In addition to closely approximating the over- all dataset distribution, our method accurately recovers the individual GMM components, Copula marginals, and SCM structure. (a) GMM(b) Copula(c) SCM Figure 9: Example inferred programs and generations by PRODiGI for each prior family: (a) A 2-d GMM with 3 components; (b) A 2-d Copula with a uniform and a long-tail marginal, respectively; (c) A 5-d SCM with structural mechanisms odd, even, tanh with (right) ground-truth and (left) inferred parents per node/feature in parentheses (empty implies root variable). We further investigate cases in which PRODiGI predicts an incorrect prior family. While PRODiGI correctly identifies the prior family for the vast majority of in-distribution examples, Figure 10 shows that approximately 0.13% of cases are assigned to a different prior family. We examine one such example in Figure 10, where the input distribution is generated by a two-dimensional SCM, whereas PRODiGI reconstructs it using a Copula. Despite this discrepancy in prior family, the reconstructed distribution and marginals closely match those of the input. This suggests that these cases may arise from overlap among our prior families rather than from a failure to model the underlying distribution accurately. For example, a one-component GMM with a diagonal covariance matrix can represent the same distribution as a Copula with Gaussian marginals and no dependence. Consistent with this hypothesis, 75% of incorrect prior-family predictions occur on one-dimensional datasets, where different prior families admit greater overlap. 24 arXiv preprint GMM Copula SCM Output GMM Copula SCM Input 0.306%0.004% 0.026%0.013% 0.033%0.020% Confusion Matrix 10% 1% 0.1% 0.01% Input SCM Output Copula Figure 10: Predicting the family of the program to be reconstructed is a learned task, PRODiGI may occasionally misclassify the data prior. Such confusion is rare, however, with only 1/744 programs misclassified (left). Despite the misclassification, the model continues to fit the data well (right), suggesting that these errors arise from overlapping data priors rather than priors that are inherently difficult for PRODiGI. F.2REAL-WORLD RESULTS AND EXAMPLES We evaluate PRODiGI and baselines on 96 datasets extracted from OddBench (6). We ignore la- beled anomalies and choose a random subset of features for each dataset to guarantee an equivalent distribution of feature dimensions compared to our in-distribution datasets. A list of all used datasets can be found in Table 7 and in our supplementary material. We further show examples of the data generation of PRODiGI and baselines in Figure on Real- World datasets in Figure 11. To obtain approximate ground-truth densities and scores, we fit a Gaussian mixture model (GMM) as described in Appdx. E.3, using a maximum of 100 components per dataset. We retain only datasets for which we fail to reject the null hypothesis when evaluating newly generated samples. This procedure yields 37 datasets with GMM-approximated density and score ground truth. We quantitatively evaluate our methods on these datasets in Figure 12 and Table 8. Overall, the MMD values are approximately an order of magnitude higher than those obtained on the prior families (Table 2), indicating that real-world datasets are substantially more challenging for all methods. This is also reflected in the rejection rate: for datasets with 30 or more dimensions, no method generates samples sufficiently similar to the input data to fail our p-test. Despite this increased difficulty, PRODiGI remains competitive. It generates samples in the shortest amount of time, while PRODiGI-FT achieves the third-lowest average MMD. Performance on the density and score estimation tasks similarly degrades across methods. Since the ground truth in this setting is derived from a GMM, the GMM baseline has a natural advantage. Nevertheless, its performance is substantially lower than on the prior families, with the Spearman correlation decreasing from 0.84 to 0.67 and the average cosine similarity decreasing from 0.73 to 0.55. For PRODiGI, we disable SCM templates for density and score estimation because these quanti- ties cannot be computed for this prior family. We retain Copula templates despite expecting them to provide a poor fit to the GMM-derived ground truth. Consequently, PRODiGI generally under- performs baselines that either explicitly model the data using GMMs or are trained exclusively on GMM-generated data. Nevertheless, PRODiGI remains competitive, achieving, for example, the fourth-lowest Score MAE and the second-lowest density NLL. We further report the distribution of the predicted prior families in Figure 13. While the prior families are not equally prevalent across real-world datasets, each captures characteristics observed in real-world data. 25 arXiv preprint Figure 11: Example real-world datasets and generations by all methods, including fine-tuned vari- ants. The first six examples are two-dimensional, while the last six are PCA representations of one 5-, 10-, 20-, 30-, 40-, and 50-dimensional dataset each. Real-world manifolds challenge all meth- ods, where PRODiGI-FT notably boosts PRODiGI. 26 arXiv preprint Table 7: OddBench datasets used as Real-World Datasets Dim = 1Dim = 2Dim = 5 DifferentialExpressionAmusementParkSafetyStudentProgrammePerformance TrafficIncidentMunicipalExpenditureAbandonedMineLand HeartMurmurDetectionPlanetaryEmissionsAutoencoderAnomaly AgilePriorityRxJavaPackageRestaurantViolation SpeciesValidationFinancialBeneficiaryMaritimeVessel VotingRecallServiceErrorLocationBusStopPrior CallPerformanceFraudProcessStepsKidneyDisease GameItemPassiveFarmerPaymentsSoftwareDevelopmentEffort FishingAnomaliesFinancialTransactionAnomaliesWaterQualityAssessment TextMessageSpamSystemMessageAnomaliesLabTestAbnormality NetworkTrafficDOSRealEstateBlockTypeHousingConstruction NetworkFirewallLanguageLocationAnomalyNetworkActivity Dim = 10Dim = 20Dim = 30 WindmillAnomalyWaterSourcesTravelWeatherScores QuadrotorNavigationEnvironmentalFluxEconomicLifeExpectancy CodeCommitMetricsIndustrialSensorBaseballPitchingAnomalies BuildingDamageEyeTrackingAnomaliesProsperLoan BiologicalClassificationsCreditRiskAssessmentFootballPlayerPosition ChicagoFoodInspectionMineAccidentChemicalReactionAnalysis TreeUsageSoccerActionsEnvironmentalAnomalies KickstarterProjectsMoldProcessSatelliteAnomaly BarExamPredictionICUPhysiologyWeatherUVIndex BudgetGoalBugSeverityWorkforceDemographics BirdAudioAnomalyBuildingInspectionVolleyballMatch RealEstateIrregularityCorporateBondDefaultLondonCrimeFraud Dim = 40Dim = 50 MiningProjectOutcomeCreativeSchoolCertification GamePositionAnomalySystemHealth BoliviaUnpaidLaborGalaxyAnomaly SentencingFraudGasBenchmark AdRequestFraudEmailSpamDetection RNASequencingAnomalyElectrocaloricAnomalies CodeQualityMetricsCancerGeneExpression DataValidationEntriesFinanceJobCategories RNAAmplificationEphysSoftwareQualityMetrics CodeDefectsBugReportNgramIDF SlowlorisAttackDetectionVehicleNetworkTraffic MassBalanceCheckMachineFailureSensors Both of these results suggest that expanding the set of supported prior families could further improve performance on real-world datasets. 27 arXiv preprint 10 1 10 0 10 1 10 2 Runtime (s) 2 4 6 8 MMD rank better Real World Generation 10 1 10 0 10 1 10 2 Runtime (s) 2 4 6 8 MAE rank better Real World Density evaluation 10 1 10 0 10 1 10 2 10 3 Runtime (s) 2 4 6 8 MAE rank better Real World Score evaluation PRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Figure 12: Performance–inference time trade-off for Generation, Density, and Score Estimation on real-world datasets. PRODiGI ranks among the fastest methods across all tasks, while PRODiGI- FT consistently improves performance on each task. However, the limited coverage of prior families constrains performance on datasets that cannot be adequately described by the available priors. Table 8: Performance comparison across tasks and metrics on real-world datasets a : (1) Genera- tion distribution match via MMD (↓) where %p<0.05 (↓) denotes the proportion of datasets where the null hypothesis (H 0 : generated and real distributions are identical) is rejected; (2) Density esti- mation via point-wise MAE (↓), population-wide Spearman correlation (↑), and NLL of generated data (↓); (3) Score estimation via MAE (↓) and cosine similarity (↑). Average rank (↓) per metric is across methods over all datasets. Time (↓) per task (s) is averaged over datasets. MetricPRODiGI-FTDiscoFormer-FTNNKDE-FTTabPFNKDEGMM Generation MMD (↓)×10 3 70.34 ± 5.86 36.75 ± 4.56 253.32 ± 23.04 269.06 ± 22.76 80.67 ± 9.27 30.95 ± 6.18 45.15 ± 10.94 69.36 ± 13.29 15.59 ± 3.27 Rank (↓)5.95 ± 0.17 4.02 ± 0.17 7.85 ± 0.12 8.22 ± 0.11 6.30 ± 0.13 3.33 ± 0.19 3.28 ± 0.23 3.95 ± 0.21 2.11 ± 0.10 % p<0.05 (↓)100%90.2%100%100%100%83.7%84.8%80.4%78.3% Time (↓)0.06 ± 0.01 9.97 ± 1.03 93.39 ± 0.10 96.05 ± 1.89 0.10 ± 0.01 1.16 ± 0.02 83.39 ± 7.86 1.35 ± 0.11 17.84 ± 1.14 Rank (↓)1.13 ± 0.04 5.07 ± 0.04 7.61 ± 0.05 8.62 ± 0.05 1.89 ± 0.04 3.60 ± 0.06 7.52 ± 0.13 3.41 ± 0.06 6.15 ± 0.06 Density MAE (↓)322 ± 239 322 ± 239 320 ± 239 320 ± 239 320 ± 239 307 ± 237 322 ± 239 323 ± 238 284 ± 210 Rank (↓) 6.49 ± 0.43 5.20 ± 0.43 5.34 ± 0.32 5.86 ± 0.28 5.60 ± 0.33 4.57 ± 0.42 5.57 ± 0.48 4.63 ± 0.43 1.74 ± 0.23 Spear. (↑)0.30 ± 0.04 0.42 ± 0.05 0.45 ± 0.04 0.48 ± 0.05 0.51 ± 0.05 0.57 ± 0.05 0.49 ± 0.05 0.55 ± 0.05 0.67 ± 0.05 Rank (↓)7.29 ± 0.36 5.81 ± 0.39 5.83 ± 0.31 5.17 ± 0.39 4.74 ± 0.27 4.69 ± 0.38 5.21 ± 0.44 3.94 ± 0.39 2.31 ± 0.33 NLL (↓)12.28 ± 2.86 8.48 ± 2.16 - b - b 1.21 ± 1.23 15.24 ± 9.15 - b 9.51 ± 3.13 36.17 ± 28.78 Rank (↓)4.69 ± 0.27 3.71 ± 0.26 - b - b 3.00 ± 0.27 2.91 ± 0.29 - b 3.34 ± 0.24 3.34 ± 0.29 Time (↓)0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 67.14 ± 17.11 0.67 ± 0.08 11.85 ± 0.72 Rank (↓)2.00 ± 0.09 7.14 ± 0.10 1.29 ± 0.13 4.66 ± 0.17 2.77 ± 0.08 6.14 ± 0.12 7.97 ± 0.20 4.57 ± 0.12 8.46 ± 0.12 Score MAE (↓)4387 ± 3018 4405 ± 3017 4629 ± 3011 4543 ± 3013 4729 ± 3012 10385 ± 4405 4376 ± 3017 4368 ± 3014 3615 ± 2357 Rank (↓)5.66 ± 0.38 4.46 ± 0.33 5.91 ± 0.32 5.63 ± 0.24 5.14 ± 0.38 7.86 ± 0.40 4.40 ± 0.39 3.94 ± 0.39 2.00 ± 0.33 Cos. (↑)0.07 ± 0.02 0.17 ± 0.03 0.19 ± 0.04 0.16 ± 0.03 0.23 ± 0.04 0.28 ± 0.05 0.24 ± 0.04 0.30 ± 0.04 0.55 ± 0.05 Rank (↓)7.01 ± 0.33 5.70 ± 0.37 5.60 ± 0.33 5.99 ± 0.37 5.01 ± 0.36 4.79 ± 0.43 5.04 ± 0.45 3.71 ± 0.39 2.14 ± 0.27 Time (↓)0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 272.40 ± 99.60 0.70 ± 0.08 11.90 ± 0.73 Rank (↓)2.00 ± 0.09 7.03 ± 0.09 1.29 ± 0.13 4.60 ± 0.15 2.77 ± 0.08 5.94 ± 0.11 8.29 ± 0.13 4.63 ± 0.13 8.46 ± 0.12 a Detailed results on individual priors and dimensions are given in Appdx. G. b NLL is not reported for DiscoFormer and TabPFN because their local density estimates are not normalized to integrate to one. CopulaSCMGMM 0 20 40 #Prior Predictions Figure 13: Distribution of predicted families on Real-World examples. Each Family is predicted between 14 and 46 times, showing that each of our priors helps to model Real-World data more accurately. 28 arXiv preprint F.3PRODIGI OUTPUTS While PRODiGI can generate programs containing thousands of tokens, we restrict the appendix to the two examples shown in Figure 4 for brevity. Both examples generate compact Copula programs. PRODiGI Output for the FinancialBeneficiary Dataset [COPULA][DIMENSIONALITY][2] [START_COMPONENT] [INDEX][0][MARGINAL][UNIFORM] [PARAMS][-0.0248][1.0000][0.0094][0.0115] [END_COMPONENT] [START_COMPONENT] [INDEX][1][MARGINAL][KUMARASWAMY] [PARAMS][-0.0108][0.8946][0.7638][2.8795] [END_COMPONENT] [CORRELATIONS][0.0026][END_CORRELATIONS] [END_COPULA] PRODiGI-FT Output for the FinancialBeneficiary Dataset [COPULA][DIMENSIONALITY][2] [START_COMPONENT] [INDEX][0][MARGINAL][UNIFORM] [PARAMS][-0.0248][1.0000][0.0094][0.0115] [END_COMPONENT] [START_COMPONENT] [INDEX][1][MARGINAL][KUMARASWAMY] [PARAMS][-0.0108][0.8946][0.0951][1.4659] [END_COMPONENT] [CORRELATIONS][-0.0017][END_CORRELATIONS] [END_COPULA] PRODiGI Output for the FinancialTransactionAnomalies Dataset [COPULA][DIMENSIONALITY][2] [START_COMPONENT] [INDEX][0][MARGINAL][UNIFORM] [PARAMS][-0.0043][1.0083][0.0056][0.0254] [END_COMPONENT] [START_COMPONENT] [INDEX][1][MARGINAL][KUMARASWAMY] [PARAMS][0.0086][0.9578][1.4854][3.6404] [END_COMPONENT] [CORRELATIONS][-0.0012][END_CORRELATIONS] [END_COPULA] PRODiGI-FT Output for the FinancialTransactionAnomalies Dataset [COPULA][DIMENSIONALITY][2] [START_COMPONENT] [INDEX][0][MARGINAL][UNIFORM] [PARAMS][-0.0043][1.0083][0.0056][0.0254] [END_COMPONENT] [START_COMPONENT] [INDEX][1][MARGINAL][KUMARASWAMY] [PARAMS][0.0086][0.9578][1.2654][4.3308] [END_COMPONENT] [CORRELATIONS][-0.0056][END_CORRELATIONS] [END_COPULA] 29 arXiv preprint GDETAILED RESULTS We provide detailed results for each algorithm and metric split across dataset dimensions and prior family in the following tables. Table 9: Distribution match performance via MMD (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MMD (×10 3 )2.81 ± 1.37 0.38 ± 0.18 91.35 ± 42.99 276.79 ± 98.84 8.34 ± 1.25 0.53 ± 0.28 0.41 ± 0.13 0.20 ± 0.13 0.35 ± 0.18 Rank4.75 ± 0.55 3.08 ± 0.34 7.96 ± 0.43 8.33 ± 0.18 7.00 ± 0.29 3.92 ± 0.47 4.21 ± 0.48 2.79 ± 0.47 2.96 ± 0.34 Time0.03 ± 0.00 1.47 ± 0.16 93.28 ± 0.61 93.80 ± 0.61 0.38 ± 0.19 1.17 ± 0.01 1.52 ± 0.02 0.11 ± 0.00 8.71 ± 0.18 Time Rank1.00 ± 0.00 4.83 ± 0.23 8.00 ± 0.00 9.00 ± 0.00 3.25 ± 0.24 4.25 ± 0.17 5.67 ± 0.14 2.00 ± 0.00 7.00 ± 0.00 Dim 2 MMD (×10 3 )1.38 ± 0.90 0.48 ± 0.25 12.26 ± 3.79 18.70 ± 8.47 66.38 ± 8.58 8.47 ± 2.72 0.15 ± 0.07 0.45 ± 0.33 0.09 ± 0.08 Rank3.29 ± 0.38 3.67 ± 0.43 7.42 ± 0.25 6.79 ± 0.48 8.83 ± 0.11 6.50 ± 0.22 3.21 ± 0.39 2.75 ± 0.25 2.54 ± 0.29 Time0.04 ± 0.02 2.16 ± 0.24 92.23 ± 0.04 92.78 ± 0.04 0.16 ± 0.01 1.16 ± 0.00 3.15 ± 0.04 0.40 ± 0.01 11.35 ± 0.54 Time Rank 1.08 ± 0.08 5.08 ± 0.08 8.00 ± 0.00 9.00 ± 0.00 1.92 ± 0.08 4.00 ± 0.00 5.92 ± 0.08 3.00 ± 0.00 7.00 ± 0.00 Dim 5 MMD (×10 3 )11.84 ± 6.19 2.20 ± 0.90 19.35 ± 4.78 18.32 ± 4.74 5.76 ± 1.95 3.21 ± 0.66 0.47 ± 0.13 1.95 ± 0.48 0.48 ± 0.12 Rank 6.00 ± 0.50 3.12 ± 0.41 8.33 ± 0.18 8.25 ± 0.21 5.58 ± 0.34 5.58 ± 0.34 1.96 ± 0.19 4.46 ± 0.45 1.71 ± 0.22 Time0.03 ± 0.00 4.27 ± 0.51 93.27 ± 0.76 93.73 ± 0.72 0.16 ± 0.01 1.16 ± 0.00 13.86 ± 0.20 0.44 ± 0.00 12.43 ± 0.65 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 6.67 ± 0.14 3.00 ± 0.00 6.33 ± 0.14 Dim 10 MMD (×10 3 )13.15 ± 7.96 1.84 ± 0.95 118.72 ± 34.43 148.91 ± 43.87 9.37 ± 2.91 5.09 ± 2.08 0.37 ± 0.12 4.05 ± 1.63 0.60 ± 0.22 Rank 5.42 ± 0.62 2.79 ± 0.45 8.33 ± 0.14 8.50 ± 0.19 6.42 ± 0.18 4.75 ± 0.36 1.96 ± 0.24 4.33 ± 0.41 2.50 ± 0.30 Time0.09 ± 0.05 8.38 ± 0.99 92.72 ± 0.42 93.69 ± 0.51 0.19 ± 0.02 1.29 ± 0.12 36.55 ± 0.74 0.61 ± 0.05 20.83 ± 0.94 Time Rank1.08 ± 0.08 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 1.92 ± 0.08 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 MMD (×10 3 )4.45 ± 2.49 0.37 ± 0.13 23.50 ± 6.79 14.89 ± 4.68 12.41 ± 3.42 13.30 ± 3.64 0.23 ± 0.06 6.57 ± 2.12 0.18 ± 0.07 Rank4.00 ± 0.26 2.79 ± 0.32 8.17 ± 0.33 7.67 ± 0.36 6.92 ± 0.25 6.92 ± 0.32 2.67 ± 0.45 4.50 ± 0.30 1.38 ± 0.13 Time0.05 ± 0.00 14.26 ± 2.10 92.70 ± 0.10 93.22 ± 0.10 0.16 ± 0.01 1.17 ± 0.01 76.69 ± 1.38 0.72 ± 0.00 52.44 ± 3.39 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 MMD (×10 3 )4.31 ± 2.21 0.85 ± 0.28 22.00 ± 8.79 15.86 ± 6.17 17.73 ± 5.81 18.17 ± 5.81 0.53 ± 0.17 11.12 ± 4.16 2.50 ± 1.86 Rank3.83 ± 0.20 2.67 ± 0.29 8.17 ± 0.31 7.25 ± 0.38 6.75 ± 0.27 7.33 ± 0.36 1.83 ± 0.25 5.04 ± 0.46 2.12 ± 0.28 Time0.06 ± 0.01 20.02 ± 2.76 92.40 ± 0.12 92.97 ± 0.10 0.24 ± 0.01 1.17 ± 0.00 120.57 ± 2.02 1.05 ± 0.12 55.21 ± 3.17 Time Rank1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 3.92 ± 0.08 9.00 ± 0.00 3.08 ± 0.08 6.00 ± 0.00 Dim 40 MMD (×10 3 ) 4.61 ± 2.72 0.35 ± 0.13 21.17 ± 12.28 22.51 ± 13.35 22.00 ± 10.53 24.14 ± 11.14 0.33 ± 0.16 9.49 ± 3.78 1.73 ± 1.33 Rank3.46 ± 0.25 2.62 ± 0.35 7.67 ± 0.30 8.33 ± 0.38 6.17 ± 0.33 7.00 ± 0.29 1.50 ± 0.25 5.75 ± 0.27 2.50 ± 0.19 Time0.11 ± 0.03 21.98 ± 3.08 92.42 ± 0.07 92.89 ± 0.04 2.21 ± 2.03 1.35 ± 0.12 167.90 ± 2.66 1.33 ± 0.16 55.51 ± 2.50 Time Rank 1.17 ± 0.11 4.92 ± 0.08 7.00 ± 0.00 8.00 ± 0.00 2.08 ± 0.28 3.75 ± 0.12 9.00 ± 0.00 3.08 ± 0.14 6.00 ± 0.00 Dim 50 MMD (×10 3 ) 1.10 ± 0.39 0.65 ± 0.25 22.64 ± 4.89 24.71 ± 5.90 14.05 ± 4.74 14.95 ± 5.06 0.27 ± 0.09 8.11 ± 2.95 0.34 ± 0.12 Rank3.54 ± 0.27 2.58 ± 0.27 8.33 ± 0.25 8.33 ± 0.18 6.17 ± 0.20 7.08 ± 0.18 1.79 ± 0.29 5.00 ± 0.12 2.17 ± 0.26 Time0.08 ± 0.01 30.67 ± 4.13 92.54 ± 0.26 93.10 ± 0.24 0.22 ± 0.01 1.17 ± 0.00 204.46 ± 3.10 1.69 ± 0.19 81.47 ± 4.01 Time Rank1.00 ± 0.00 5.00 ± 0.00 6.83 ± 0.11 7.83 ± 0.11 2.00 ± 0.00 3.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 6.33 ± 0.22 Overall MMD (×10 3 )5.46 ± 1.45 0.89 ± 0.19 41.37 ± 8.20 67.58 ± 16.51 19.51 ± 2.77 10.98 ± 1.96 0.35 ± 0.05 5.24 ± 0.95 0.78 ± 0.30 Rank4.29 ± 0.17 2.92 ± 0.13 8.05 ± 0.11 7.93 ± 0.13 6.73 ± 0.13 6.14 ± 0.17 2.39 ± 0.15 4.33 ± 0.16 2.23 ± 0.10 Time0.06 ± 0.01 12.90 ± 1.29 92.70 ± 0.14 93.27 ± 0.14 0.46 ± 0.26 1.20 ± 0.02 78.09 ± 7.52 0.79 ± 0.06 37.24 ± 2.74 Time Rank1.04 ± 0.02 4.98 ± 0.03 7.60 ± 0.05 8.60 ± 0.05 2.15 ± 0.06 3.86 ± 0.05 7.41 ± 0.14 3.02 ± 0.06 6.33 ± 0.05 Table 10: Rejection rate of two-sample tests for distribution match w.r.t. MMD over test datasets from ALL prior families. Tests under generated data that successfully match the input data distribu- tion fail to reject the null (i.e., p > 0.05) that the two samples come from the same distribution. DimensionPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 141.7%0.0%91.7%100.0%91.7%25.0%8.3%8.3%8.3% Dim 216.7%16.7%100.0%83.3%100.0%91.7%16.7%8.3%0.0% Dim 566.7%41.7%100.0%100.0%91.7%83.3%33.3%75.0%33.3% Dim 1075.0%50.0%100.0%100.0%100.0%83.3%25.0%66.7%41.7% Dim 2083.3%58.3%100.0%100.0%91.7%91.7%41.7%83.3%16.7% Dim 3066.7%58.3%100.0%100.0%100.0%100.0%41.7%91.7%50.0% Dim 4058.3%50.0%100.0%100.0%83.3%83.3%33.3%83.3%58.3% Dim 5050.0%41.7%100.0%100.0%83.3%83.3%41.7%75.0%41.7% Overall57.3%39.6%99.0%97.9%92.7%80.2%30.2%61.5%31.2% 30 arXiv preprint Table 11: Distribution match performance via MMD (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MMD (×10 3 )5.31 ± 2.70 0.45 ± 0.39 59.23 ± 41.13 76.56 ± 59.00 6.77 ± 0.95 1.22 ± 0.71 0.23 ± 0.11 0.00 ± 0.00 0.66 ± 0.45 Rank6.50 ± 0.25 2.38 ± 0.32 8.25 ± 0.41 8.25 ± 0.22 7.00 ± 0.61 4.50 ± 0.43 3.00 ± 0.35 1.88 ± 0.37 3.25 ± 0.41 Time0.03 ± 0.00 1.12 ± 0.01 92.61 ± 0.07 93.18 ± 0.08 0.18 ± 0.01 1.16 ± 0.00 1.56 ± 0.04 0.11 ± 0.00 8.30 ± 0.05 Time Rank 1.00 ± 0.00 4.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 5.00 ± 0.00 6.00 ± 0.00 2.00 ± 0.00 7.00 ± 0.00 Dim 2 MMD (×10 3 )4.02 ± 2.15 0.41 ± 0.34 15.23 ± 2.69 8.24 ± 2.43 52.62 ± 17.98 8.83 ± 2.49 0.22 ± 0.19 0.28 ± 0.24 0.00 ± 0.00 Rank4.62 ± 0.65 3.38 ± 0.60 8.00 ± 0.35 6.75 ± 0.54 8.75 ± 0.22 6.25 ± 0.22 2.75 ± 0.38 2.50 ± 0.18 2.00 ± 0.31 Time0.03 ± 0.00 1.62 ± 0.06 92.27 ± 0.04 92.82 ± 0.04 0.16 ± 0.02 1.16 ± 0.00 3.06 ± 0.06 0.37 ± 0.00 9.34 ± 0.39 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 3.00 ± 0.00 7.00 ± 0.00 Dim 5 MMD (×10 3 )29.65 ± 14.87 4.13 ± 2.22 27.15 ± 11.28 23.22 ± 9.04 10.44 ± 4.71 3.80 ± 1.08 0.28 ± 0.24 1.35 ± 0.41 0.55 ± 0.24 Rank7.00 ± 1.06 3.62 ± 0.69 7.75 ± 0.22 8.25 ± 0.41 6.25 ± 0.22 5.00 ± 0.61 1.38 ± 0.32 3.38 ± 0.48 2.38 ± 0.21 Time0.03 ± 0.00 2.22 ± 0.14 95.06 ± 2.00 95.46 ± 1.88 0.16 ± 0.03 1.16 ± 0.01 13.80 ± 0.31 0.44 ± 0.00 9.71 ± 0.33 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 10 MMD (×10 3 )33.39 ± 20.24 0.96 ± 0.40 173.84 ± 61.97 241.94 ± 89.56 15.53 ± 5.81 6.94 ± 5.07 0.72 ± 0.22 5.24 ± 3.77 0.76 ± 0.45 Rank7.25 ± 0.54 2.50 ± 0.83 8.25 ± 0.22 8.25 ± 0.41 6.25 ± 0.22 4.00 ± 0.61 2.25 ± 0.41 3.50 ± 0.75 2.75 ± 0.41 Time0.04 ± 0.00 4.05 ± 0.08 92.25 ± 0.07 92.80 ± 0.07 0.19 ± 0.02 1.16 ± 0.01 36.99 ± 1.36 0.53 ± 0.00 16.89 ± 1.00 Time Rank 1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 MMD (×10 3 ) 11.29 ± 6.09 0.15 ± 0.09 41.73 ± 12.68 28.12 ± 10.32 18.74 ± 6.94 19.04 ± 6.47 0.26 ± 0.11 7.11 ± 3.46 0.14 ± 0.11 Rank4.50 ± 0.43 1.75 ± 0.22 8.75 ± 0.22 8.25 ± 0.22 6.25 ± 0.41 6.25 ± 0.22 3.75 ± 0.96 4.00 ± 0.00 1.50 ± 0.25 Time0.05 ± 0.01 6.10 ± 0.05 92.47 ± 0.15 92.94 ± 0.08 0.15 ± 0.01 1.16 ± 0.00 78.79 ± 1.17 0.72 ± 0.01 37.97 ± 3.61 Time Rank 1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 MMD (×10 3 )6.87 ± 5.93 0.46 ± 0.39 18.77 ± 15.32 19.17 ± 16.04 14.02 ± 11.86 14.29 ± 11.83 0.21 ± 0.18 1.13 ± 0.87 5.93 ± 5.14 Rank 4.25 ± 0.41 3.00 ± 0.50 8.75 ± 0.22 8.00 ± 0.35 6.25 ± 0.22 7.00 ± 0.35 1.50 ± 0.18 3.62 ± 0.74 2.62 ± 0.48 Time0.06 ± 0.01 8.32 ± 0.07 92.23 ± 0.10 92.77 ± 0.09 0.27 ± 0.03 1.16 ± 0.00 118.88 ± 1.04 0.93 ± 0.00 45.31 ± 6.14 Time Rank1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 40 MMD (×10 3 )12.56 ± 6.53 0.17 ± 0.10 52.43 ± 31.20 56.60 ± 33.98 44.43 ± 25.41 45.41 ± 26.18 0.62 ± 0.37 13.98 ± 7.15 4.73 ± 3.55 Rank2.88 ± 0.57 1.88 ± 0.62 8.00 ± 0.00 9.00 ± 0.00 6.25 ± 0.22 6.75 ± 0.22 2.25 ± 0.54 5.00 ± 0.00 3.00 ± 0.00 Time0.11 ± 0.05 10.67 ± 0.13 92.42 ± 0.07 92.94 ± 0.06 6.46 ± 5.52 1.61 ± 0.32 161.74 ± 4.17 1.16 ± 0.01 50.63 ± 3.56 Time Rank1.00 ± 0.00 4.75 ± 0.22 7.00 ± 0.00 8.00 ± 0.00 2.75 ± 0.65 3.75 ± 0.22 9.00 ± 0.00 2.75 ± 0.22 6.00 ± 0.00 Dim 50 MMD (×10 3 )0.44 ± 0.33 0.08 ± 0.05 12.62 ± 2.73 11.40 ± 2.44 7.41 ± 2.44 7.87 ± 2.28 0.06 ± 0.03 1.84 ± 0.67 0.04 ± 0.02 Rank3.12 ± 0.72 2.50 ± 0.53 9.00 ± 0.00 8.00 ± 0.00 6.25 ± 0.22 6.75 ± 0.22 2.38 ± 0.60 4.75 ± 0.22 2.25 ± 0.41 Time0.05 ± 0.01 12.46 ± 0.11 92.04 ± 0.08 92.65 ± 0.08 0.19 ± 0.00 1.17 ± 0.00 214.56 ± 3.04 1.49 ± 0.01 84.03 ± 7.46 Time Rank1.00 ± 0.00 5.00 ± 0.00 6.75 ± 0.22 7.75 ± 0.22 2.00 ± 0.00 3.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 6.50 ± 0.43 Overall MMD (×10 3 ) 12.94 ± 3.98 0.85 ± 0.37 50.13 ± 13.66 58.15 ± 19.25 21.25 ± 5.23 13.43 ± 4.43 0.32 ± 0.08 3.87 ± 1.36 1.60 ± 0.88 Rank5.02 ± 0.36 2.62 ± 0.23 8.34 ± 0.11 8.09 ± 0.16 6.66 ± 0.19 5.81 ± 0.24 2.41 ± 0.22 3.58 ± 0.23 2.47 ± 0.15 Time0.05 ± 0.01 5.82 ± 0.71 92.67 ± 0.30 93.20 ± 0.28 0.97 ± 0.78 1.22 ± 0.05 78.67 ± 13.22 0.72 ± 0.08 32.77 ± 4.65 Time Rank1.00 ± 0.00 4.84 ± 0.06 7.59 ± 0.10 8.59 ± 0.10 2.22 ± 0.11 3.97 ± 0.09 7.50 ± 0.22 2.97 ± 0.09 6.31 ± 0.09 31 arXiv preprint Table 12: Distribution match performance via MMD (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the SCM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MMD (×10 3 )3.07 ± 2.47 0.36 ± 0.30 179.43 ± 108.64 538.42 ± 175.53 9.01 ± 3.47 0.00 ± 0.00 0.41 ± 0.22 0.24 ± 0.17 0.00 ± 0.00 Rank5.25 ± 0.65 3.00 ± 0.71 8.25 ± 0.22 8.50 ± 0.43 6.50 ± 0.25 2.75 ± 0.72 5.00 ± 0.94 3.50 ± 0.75 2.25 ± 0.38 Time0.03 ± 0.00 1.55 ± 0.05 92.65 ± 0.15 93.13 ± 0.09 0.17 ± 0.01 1.18 ± 0.02 1.51 ± 0.03 0.10 ± 0.00 9.31 ± 0.35 Time Rank1.00 ± 0.00 5.50 ± 0.25 8.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 4.00 ± 0.00 5.50 ± 0.25 2.00 ± 0.00 7.00 ± 0.00 Dim 2 MMD (×10 3 )0.02 ± 0.02 0.74 ± 0.64 7.50 ± 2.78 25.72 ± 17.60 87.66 ± 4.73 5.60 ± 3.88 0.22 ± 0.07 0.04 ± 0.03 0.01 ± 0.01 Rank 2.62 ± 0.21 2.88 ± 0.72 7.25 ± 0.41 7.25 ± 0.41 9.00 ± 0.00 6.25 ± 0.41 4.50 ± 0.56 2.88 ± 0.37 2.38 ± 0.21 Time0.08 ± 0.05 1.96 ± 0.14 92.26 ± 0.09 92.81 ± 0.09 0.17 ± 0.01 1.16 ± 0.00 3.15 ± 0.05 0.47 ± 0.00 13.08 ± 0.48 Time Rank1.25 ± 0.22 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 1.75 ± 0.22 4.00 ± 0.00 6.00 ± 0.00 3.00 ± 0.00 7.00 ± 0.00 Dim 5 MMD (×10 3 )1.27 ± 0.70 1.35 ± 0.78 15.93 ± 7.27 16.39 ± 9.10 4.15 ± 1.63 1.70 ± 0.56 0.55 ± 0.23 1.31 ± 0.50 0.37 ± 0.20 Rank 5.00 ± 0.61 3.50 ± 0.75 8.75 ± 0.22 8.25 ± 0.22 6.00 ± 0.61 5.25 ± 0.22 2.25 ± 0.22 4.75 ± 0.96 1.25 ± 0.22 Time0.04 ± 0.01 4.25 ± 0.10 92.37 ± 0.17 92.82 ± 0.10 0.14 ± 0.01 1.16 ± 0.01 13.88 ± 0.30 0.44 ± 0.00 14.60 ± 0.49 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 6.25 ± 0.22 3.00 ± 0.00 6.75 ± 0.22 Dim 10 MMD (×10 3 )3.02 ± 2.38 1.04 ± 0.74 106.47 ± 47.35 143.14 ± 62.15 6.87 ± 3.48 4.64 ± 2.29 0.36 ± 0.16 2.47 ± 1.18 0.57 ± 0.40 Rank4.00 ± 1.06 3.00 ± 0.61 8.25 ± 0.22 8.75 ± 0.22 6.75 ± 0.22 5.75 ± 0.22 2.25 ± 0.41 4.50 ± 0.25 1.75 ± 0.22 Time0.20 ± 0.14 9.04 ± 0.34 93.79 ± 1.08 94.65 ± 1.11 0.23 ± 0.06 1.55 ± 0.34 35.98 ± 1.12 0.63 ± 0.08 23.51 ± 0.88 Time Rank 1.25 ± 0.22 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 1.75 ± 0.22 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 MMD (×10 3 )0.70 ± 0.30 0.56 ± 0.27 11.12 ± 5.16 9.05 ± 4.58 10.71 ± 5.50 12.81 ± 7.11 0.25 ± 0.12 6.93 ± 3.99 0.18 ± 0.12 Rank 4.00 ± 0.35 4.00 ± 0.35 8.25 ± 0.41 7.25 ± 0.65 7.50 ± 0.25 7.00 ± 0.61 1.75 ± 0.22 4.00 ± 0.50 1.25 ± 0.22 Time0.04 ± 0.00 14.53 ± 0.15 92.78 ± 0.04 93.32 ± 0.03 0.19 ± 0.04 1.19 ± 0.02 73.96 ± 2.50 0.72 ± 0.01 61.81 ± 2.62 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 MMD (×10 3 )0.78 ± 0.32 0.43 ± 0.20 6.12 ± 2.23 5.76 ± 1.94 12.99 ± 8.88 12.44 ± 8.10 0.27 ± 0.12 8.15 ± 5.11 0.57 ± 0.37 Rank 3.50 ± 0.25 2.38 ± 0.60 7.75 ± 0.54 7.25 ± 0.74 6.75 ± 0.65 7.75 ± 0.41 2.00 ± 0.35 5.50 ± 0.43 2.12 ± 0.57 Time0.04 ± 0.00 20.53 ± 0.04 92.27 ± 0.15 92.94 ± 0.06 0.21 ± 0.02 1.17 ± 0.01 128.63 ± 2.01 0.93 ± 0.01 54.67 ± 0.89 Time Rank1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 40 MMD (×10 3 )1.02 ± 0.35 0.79 ± 0.26 8.91 ± 3.03 8.58 ± 2.80 20.85 ± 10.58 26.23 ± 13.43 0.32 ± 0.22 13.75 ± 6.94 0.36 ± 0.11 Rank3.75 ± 0.22 3.25 ± 0.22 6.75 ± 0.65 7.25 ± 0.89 7.00 ± 0.50 8.00 ± 0.50 1.25 ± 0.22 6.00 ± 0.50 1.75 ± 0.22 Time0.05 ± 0.00 27.00 ± 0.08 92.49 ± 0.14 92.88 ± 0.04 0.08 ± 0.00 1.23 ± 0.02 174.44 ± 3.15 1.14 ± 0.02 53.48 ± 2.78 Time Rank 1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 50 MMD (×10 3 ) 0.37 ± 0.21 0.27 ± 0.16 11.25 ± 4.97 11.81 ± 5.25 5.73 ± 2.87 6.24 ± 3.12 0.21 ± 0.11 3.01 ± 1.50 0.31 ± 0.18 Rank3.75 ± 0.22 2.50 ± 0.56 8.25 ± 0.22 8.75 ± 0.22 5.75 ± 0.22 7.00 ± 0.00 1.50 ± 0.43 5.25 ± 0.22 2.25 ± 0.22 Time0.06 ± 0.00 33.41 ± 0.24 93.12 ± 0.58 93.60 ± 0.53 0.21 ± 0.02 1.17 ± 0.00 194.43 ± 2.99 1.50 ± 0.02 72.68 ± 3.24 Time Rank1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 3.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 Overall MMD (×10 3 )1.28 ± 0.48 0.69 ± 0.18 43.34 ± 18.31 94.86 ± 38.54 19.75 ± 5.07 8.71 ± 2.65 0.32 ± 0.06 4.49 ± 1.45 0.30 ± 0.09 Rank3.98 ± 0.23 3.06 ± 0.23 7.94 ± 0.18 7.91 ± 0.22 6.91 ± 0.22 6.22 ± 0.32 2.56 ± 0.29 4.55 ± 0.26 1.88 ± 0.13 Time0.07 ± 0.02 14.04 ± 1.99 92.72 ± 0.18 93.27 ± 0.19 0.17 ± 0.01 1.23 ± 0.05 78.25 ± 12.94 0.74 ± 0.07 37.89 ± 4.24 Time Rank1.06 ± 0.04 5.06 ± 0.04 7.62 ± 0.09 8.62 ± 0.09 2.06 ± 0.08 3.88 ± 0.06 7.34 ± 0.24 3.00 ± 0.09 6.34 ± 0.08 32 arXiv preprint Table 13: Distribution match performance via MMD (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MMD (×10 3 )0.06 ± 0.03 0.34 ± 0.20 35.40 ± 12.62 215.40 ± 159.87 9.24 ± 0.44 0.38 ± 0.19 0.60 ± 0.27 0.38 ± 0.33 0.40 ± 0.19 Rank 2.50 ± 0.40 3.88 ± 0.37 7.38 ± 1.14 8.25 ± 0.22 7.50 ± 0.43 4.50 ± 0.90 4.62 ± 0.69 3.00 ± 0.98 3.38 ± 0.74 Time0.02 ± 0.00 1.75 ± 0.43 94.58 ± 1.64 95.07 ± 1.65 0.78 ± 0.52 1.16 ± 0.00 1.50 ± 0.03 0.11 ± 0.00 8.53 ± 0.10 Time Rank1.00 ± 0.00 5.00 ± 0.35 8.00 ± 0.00 9.00 ± 0.00 3.75 ± 0.65 3.75 ± 0.22 5.50 ± 0.25 2.00 ± 0.00 7.00 ± 0.00 Dim 2 MMD (×10 3 )0.09 ± 0.05 0.28 ± 0.16 14.03 ± 10.29 22.14 ± 16.97 58.87 ± 11.96 10.98 ± 6.46 0.00 ± 0.00 1.02 ± 0.89 0.25 ± 0.22 Rank 2.62 ± 0.41 4.75 ± 0.54 7.00 ± 0.35 6.38 ± 1.24 8.75 ± 0.22 7.00 ± 0.35 2.38 ± 0.54 2.88 ± 0.62 3.25 ± 0.65 Time0.02 ± 0.00 2.91 ± 0.50 92.17 ± 0.06 92.72 ± 0.06 0.15 ± 0.01 1.17 ± 0.01 3.23 ± 0.06 0.38 ± 0.01 11.62 ± 0.65 Time Rank1.00 ± 0.00 5.25 ± 0.22 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 5.75 ± 0.22 3.00 ± 0.00 7.00 ± 0.00 Dim 5 MMD (×10 3 )4.61 ± 1.75 1.11 ± 0.62 14.97 ± 1.68 15.33 ± 5.38 2.69 ± 0.95 4.12 ± 1.24 0.57 ± 0.19 3.18 ± 1.04 0.51 ± 0.16 Rank6.00 ± 0.50 2.25 ± 0.41 8.50 ± 0.25 8.25 ± 0.41 4.50 ± 0.43 6.50 ± 0.56 2.25 ± 0.22 5.25 ± 0.41 1.50 ± 0.43 Time0.04 ± 0.01 6.35 ± 0.45 92.37 ± 0.03 92.92 ± 0.03 0.18 ± 0.02 1.16 ± 0.00 13.89 ± 0.40 0.44 ± 0.00 12.97 ± 0.60 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 6.75 ± 0.22 3.00 ± 0.00 6.25 ± 0.22 Dim 10 MMD (×10 3 )3.06 ± 1.15 3.51 ± 2.53 75.85 ± 57.70 61.67 ± 36.95 5.70 ± 3.97 3.70 ± 2.59 0.05 ± 0.04 4.45 ± 2.70 0.47 ± 0.25 Rank5.00 ± 0.79 2.88 ± 0.84 8.50 ± 0.25 8.50 ± 0.25 6.25 ± 0.41 4.50 ± 0.56 1.38 ± 0.21 5.00 ± 0.79 3.00 ± 0.61 Time0.04 ± 0.00 12.06 ± 0.65 92.11 ± 0.07 93.61 ± 0.82 0.15 ± 0.01 1.16 ± 0.00 36.68 ± 1.28 0.68 ± 0.12 22.07 ± 0.37 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 MMD (×10 3 )1.34 ± 0.89 0.41 ± 0.25 17.66 ± 9.86 7.48 ± 1.83 7.78 ± 3.23 8.05 ± 3.39 0.20 ± 0.09 5.66 ± 3.50 0.24 ± 0.13 Rank3.50 ± 0.43 2.62 ± 0.32 7.50 ± 0.75 7.50 ± 0.75 7.00 ± 0.35 7.50 ± 0.56 2.50 ± 0.56 5.50 ± 0.43 1.38 ± 0.21 Time0.05 ± 0.00 22.15 ± 2.71 92.85 ± 0.22 93.41 ± 0.23 0.15 ± 0.02 1.16 ± 0.00 77.33 ± 2.52 0.71 ± 0.00 57.53 ± 1.63 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 MMD (×10 3 )5.27 ± 1.91 1.65 ± 0.52 41.10 ± 17.28 22.65 ± 6.48 26.18 ± 7.58 27.77 ± 7.98 1.11 ± 0.32 24.07 ± 7.73 0.99 ± 0.32 Rank3.75 ± 0.22 2.62 ± 0.32 8.00 ± 0.61 6.50 ± 0.56 7.25 ± 0.22 7.25 ± 0.89 2.00 ± 0.61 6.00 ± 0.61 1.62 ± 0.21 Time0.07 ± 0.00 31.21 ± 1.73 92.70 ± 0.24 93.21 ± 0.24 0.23 ± 0.01 1.17 ± 0.00 114.19 ± 2.12 1.31 ± 0.33 65.66 ± 0.27 Time Rank1.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 3.75 ± 0.22 9.00 ± 0.00 3.25 ± 0.22 6.00 ± 0.00 Dim 40 MMD (×10 3 )0.24 ± 0.18 0.09 ± 0.06 2.17 ± 1.25 2.35 ± 1.22 0.72 ± 0.55 0.79 ± 0.62 0.04 ± 0.02 0.75 ± 0.61 0.11 ± 0.07 Rank3.75 ± 0.22 2.75 ± 0.65 8.25 ± 0.22 8.75 ± 0.22 5.25 ± 0.54 6.25 ± 0.22 1.00 ± 0.00 6.25 ± 0.41 2.75 ± 0.22 Time0.15 ± 0.06 28.26 ± 6.11 92.37 ± 0.14 92.86 ± 0.10 0.09 ± 0.01 1.19 ± 0.01 167.51 ± 4.03 1.68 ± 0.44 62.42 ± 4.14 Time Rank1.50 ± 0.25 5.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 1.50 ± 0.25 3.50 ± 0.25 9.00 ± 0.00 3.50 ± 0.25 6.00 ± 0.00 Dim 50 MMD (×10 3 )2.48 ± 0.68 1.59 ± 0.46 44.04 ± 3.39 50.91 ± 4.76 29.02 ± 10.18 30.75 ± 11.01 0.55 ± 0.17 19.46 ± 5.22 0.67 ± 0.20 Rank3.75 ± 0.22 2.75 ± 0.22 7.75 ± 0.54 8.25 ± 0.41 6.50 ± 0.43 7.50 ± 0.43 1.50 ± 0.25 5.00 ± 0.00 2.00 ± 0.61 Time0.12 ± 0.00 46.13 ± 3.04 92.48 ± 0.37 93.04 ± 0.37 0.25 ± 0.03 1.17 ± 0.00 204.38 ± 4.17 2.09 ± 0.51 87.71 ± 6.93 Time Rank1.00 ± 0.00 5.00 ± 0.00 6.75 ± 0.22 7.75 ± 0.22 2.00 ± 0.00 3.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 6.50 ± 0.43 Overall MMD (×10 3 )2.14 ± 0.51 1.13 ± 0.39 30.65 ± 8.81 49.74 ± 23.68 17.52 ± 3.98 10.82 ± 2.78 0.39 ± 0.09 7.37 ± 2.00 0.45 ± 0.09 Rank3.86 ± 0.25 3.06 ± 0.22 7.86 ± 0.23 7.80 ± 0.26 6.62 ± 0.26 6.38 ± 0.30 2.20 ± 0.24 4.86 ± 0.30 2.36 ± 0.22 Time0.07 ± 0.01 18.85 ± 2.78 92.70 ± 0.25 93.36 ± 0.27 0.25 ± 0.07 1.17 ± 0.00 77.34 ± 12.92 0.92 ± 0.15 41.06 ± 5.18 Time Rank1.06 ± 0.04 5.03 ± 0.05 7.59 ± 0.10 8.59 ± 0.10 2.16 ± 0.14 3.75 ± 0.08 7.38 ± 0.25 3.09 ± 0.10 6.34 ± 0.09 33 arXiv preprint Table 14: Density estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE0.34 ± 0.15 0.10 ± 0.03 0.49 ± 0.05 0.43 ± 0.06 0.21 ± 0.03 0.19 ± 0.01 2.38 ± 0.01 0.15 ± 0.03 0.12 ± 0.02 Rank4.25 ± 0.84 1.88 ± 0.36 7.75 ± 0.23 7.00 ± 0.00 4.75 ± 0.15 4.50 ± 0.40 9.00 ± 0.00 3.00 ± 0.71 2.88 ± 0.28 Time0.29 ± 0.00 1.48 ± 0.25 0.03 ± 0.02 0.64 ± 0.06 0.17 ± 0.00 1.15 ± 0.00 3.59 ± 1.48 0.14 ± 0.00 8.42 ± 0.07 Time Rank4.00 ± 0.00 7.38 ± 0.25 1.25 ± 0.23 5.00 ± 0.00 2.75 ± 0.15 6.62 ± 0.17 7.12 ± 0.41 2.00 ± 0.18 8.88 ± 0.12 Dim 2 MAE0.61 ± 0.16 0.24 ± 0.08 0.48 ± 0.07 0.57 ± 0.07 0.65 ± 0.05 0.66 ± 0.02 1.47 ± 0.19 0.24 ± 0.02 0.19 ± 0.04 Rank 5.25 ± 1.06 1.88 ± 0.28 4.62 ± 0.17 5.88 ± 0.21 7.00 ± 0.25 6.50 ± 0.56 8.88 ± 0.12 2.75 ± 0.23 2.25 ± 0.46 Time0.29 ± 0.00 2.26 ± 0.32 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 5.51 ± 0.82 0.43 ± 0.00 10.48 ± 0.55 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.12 ± 0.12 4.00 ± 0.00 8.88 ± 0.12 Dim 5 MAE3.19 ± 1.34 0.62 ± 0.21 1.19 ± 0.29 1.11 ± 0.21 1.08 ± 0.21 0.85 ± 0.11 0.95 ± 0.07 0.83 ± 0.12 0.48 ± 0.08 Rank5.75 ± 1.10 2.62 ± 0.90 6.25 ± 0.55 6.38 ± 0.61 6.12 ± 0.48 4.38 ± 0.47 6.75 ± 0.79 4.50 ± 0.92 2.25 ± 0.34 Time0.30 ± 0.00 5.31 ± 0.79 0.01 ± 0.00 0.57 ± 0.01 0.19 ± 0.01 1.16 ± 0.00 50.08 ± 0.77 0.53 ± 0.01 11.34 ± 0.67 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.88 ± 0.12 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.12 ± 0.12 8.00 ± 0.00 Dim 10 MAE9.37 ± 2.83 1.06 ± 0.32 4.08 ± 0.96 4.21 ± 0.86 2.93 ± 0.49 2.61 ± 0.28 1.35 ± 0.07 2.36 ± 0.23 1.23 ± 0.23 Rank8.00 ± 0.81 2.25 ± 0.79 6.00 ± 0.66 7.00 ± 0.25 4.75 ± 0.39 5.62 ± 0.25 2.50 ± 0.35 6.12 ± 0.86 2.75 ± 0.58 Time0.30 ± 0.00 7.24 ± 1.33 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.16 ± 0.00 99.74 ± 1.61 0.72 ± 0.06 19.48 ± 1.06 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE6.28 ± 0.64 1.40 ± 0.38 7.05 ± 0.75 5.54 ± 0.81 6.21 ± 0.40 6.29 ± 0.46 1.79 ± 0.07 5.77 ± 0.48 2.58 ± 0.49 Rank6.38 ± 0.53 1.62 ± 0.25 7.50 ± 0.53 5.25 ± 0.63 6.62 ± 0.35 6.62 ± 0.30 1.88 ± 0.33 6.50 ± 0.88 2.62 ± 0.25 Time0.31 ± 0.00 14.29 ± 3.16 0.01 ± 0.00 0.57 ± 0.00 0.20 ± 0.00 1.17 ± 0.00 203.30 ± 3.20 0.89 ± 0.02 47.75 ± 3.98 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE11.30 ± 1.48 3.67 ± 1.18 7.88 ± 1.09 5.51 ± 0.97 10.39 ± 0.80 10.13 ± 0.86 2.84 ± 0.20 12.03 ± 1.62 5.61 ± 1.26 Rank7.50 ± 0.47 2.12 ± 0.41 4.75 ± 0.39 2.88 ± 0.37 7.62 ± 0.17 6.88 ± 0.33 2.12 ± 0.45 7.50 ± 0.71 3.62 ± 0.50 Time0.33 ± 0.01 20.18 ± 4.17 0.20 ± 0.18 5.20 ± 4.33 0.39 ± 0.10 1.37 ± 0.18 305.11 ± 6.68 1.34 ± 0.18 55.48 ± 4.73 Time Rank 2.50 ± 0.25 6.88 ± 0.12 1.38 ± 0.35 4.38 ± 0.35 2.25 ± 0.15 5.50 ± 0.18 9.00 ± 0.00 5.12 ± 0.33 8.00 ± 0.00 Dim 40 MAE12.67 ± 1.82 3.69 ± 1.15 11.07 ± 1.40 6.89 ± 1.05 13.68 ± 0.98 13.18 ± 1.05 3.27 ± 0.21 15.37 ± 1.96 8.70 ± 1.76 Rank6.62 ± 0.85 1.88 ± 0.28 5.25 ± 0.15 3.38 ± 0.43 8.00 ± 0.25 6.75 ± 0.34 1.75 ± 0.34 7.25 ± 0.68 4.12 ± 0.54 Time0.33 ± 0.01 19.58 ± 4.56 0.04 ± 0.01 0.61 ± 0.01 3.07 ± 2.77 1.49 ± 0.21 425.10 ± 4.92 1.61 ± 0.26 56.53 ± 3.43 Time Rank2.88 ± 0.12 6.88 ± 0.12 1.00 ± 0.00 3.88 ± 0.12 2.62 ± 0.58 5.25 ± 0.15 9.00 ± 0.00 5.50 ± 0.25 8.00 ± 0.00 Dim 50 MAE 16.90 ± 5.73 6.72 ± 2.16 8.52 ± 1.88 9.12 ± 1.84 16.35 ± 2.08 16.09 ± 2.13 4.39 ± 0.71 20.34 ± 3.61 8.76 ± 2.37 Rank 5.00 ± 1.15 2.38 ± 0.35 3.62 ± 0.25 4.12 ± 0.65 8.00 ± 0.31 6.88 ± 0.21 2.75 ± 0.63 8.25 ± 0.34 4.00 ± 0.31 Time0.34 ± 0.01 28.72 ± 5.76 0.05 ± 0.01 0.62 ± 0.01 0.37 ± 0.02 1.21 ± 0.02 533.58 ± 4.60 2.21 ± 0.30 85.87 ± 5.13 Time Rank2.38 ± 0.17 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.62 ± 0.17 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE7.58 ± 1.12 2.19 ± 0.44 5.10 ± 0.60 4.17 ± 0.50 6.44 ± 0.80 6.25 ± 0.80 2.31 ± 0.17 7.14 ± 1.07 3.46 ± 0.60 Rank6.09 ± 0.35 2.08 ± 0.18 5.72 ± 0.22 5.23 ± 0.25 6.61 ± 0.19 6.02 ± 0.18 4.45 ± 0.41 5.73 ± 0.35 3.06 ± 0.17 Time0.31 ± 0.00 12.38 ± 1.63 0.04 ± 0.02 1.17 ± 0.57 0.60 ± 0.37 1.23 ± 0.04 203.25 ± 23.50 0.98 ± 0.10 36.92 ± 3.52 Time Rank2.97 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.39 ± 0.08 2.28 ± 0.09 5.80 ± 0.07 8.66 ± 0.10 4.59 ± 0.16 8.22 ± 0.05 34 arXiv preprint Table 15: Density estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE0.64 ± 0.23 0.16 ± 0.05 0.54 ± 0.08 0.51 ± 0.07 0.23 ± 0.05 0.18 ± 0.01 2.38 ± 0.01 0.09 ± 0.01 0.12 ± 0.01 Rank6.50 ± 0.43 2.50 ± 0.56 7.50 ± 0.43 7.00 ± 0.00 4.75 ± 0.22 4.00 ± 0.35 9.00 ± 0.00 1.25 ± 0.22 2.50 ± 0.25 Time0.30 ± 0.00 1.16 ± 0.02 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 5.20 ± 2.61 0.13 ± 0.00 8.30 ± 0.05 Time Rank 4.00 ± 0.00 7.00 ± 0.35 1.00 ± 0.00 5.00 ± 0.00 3.00 ± 0.00 6.50 ± 0.25 7.75 ± 0.54 2.00 ± 0.00 8.75 ± 0.22 Dim 2 MAE0.79 ± 0.23 0.22 ± 0.12 0.44 ± 0.09 0.54 ± 0.11 0.61 ± 0.06 0.63 ± 0.03 1.42 ± 0.23 0.19 ± 0.03 0.10 ± 0.03 Rank6.50 ± 1.60 2.25 ± 0.22 4.50 ± 0.25 5.75 ± 0.41 6.50 ± 0.25 6.25 ± 0.74 8.75 ± 0.22 3.00 ± 0.35 1.50 ± 0.43 Time0.29 ± 0.00 1.60 ± 0.05 0.01 ± 0.00 0.56 ± 0.00 0.17 ± 0.01 1.16 ± 0.00 4.68 ± 0.78 0.43 ± 0.00 9.34 ± 0.39 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 MAE5.74 ± 2.00 0.57 ± 0.22 1.65 ± 0.49 1.47 ± 0.34 1.44 ± 0.32 1.02 ± 0.18 0.97 ± 0.13 0.73 ± 0.10 0.35 ± 0.11 Rank7.00 ± 1.73 2.25 ± 0.41 6.75 ± 0.82 6.50 ± 0.56 7.00 ± 0.35 5.00 ± 0.35 5.25 ± 1.14 3.50 ± 0.83 1.75 ± 0.41 Time0.30 ± 0.00 3.16 ± 0.14 0.01 ± 0.00 0.57 ± 0.02 0.18 ± 0.01 1.16 ± 0.00 50.96 ± 1.20 0.51 ± 0.01 9.71 ± 0.33 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 8.00 ± 0.00 Dim 10 MAE16.08 ± 3.04 0.44 ± 0.06 6.26 ± 1.14 6.07 ± 1.08 3.95 ± 0.61 3.16 ± 0.35 1.40 ± 0.13 1.87 ± 0.13 0.66 ± 0.03 Rank9.00 ± 0.00 1.00 ± 0.00 7.75 ± 0.22 7.25 ± 0.22 5.75 ± 0.22 5.25 ± 0.22 3.25 ± 0.22 3.75 ± 0.22 2.00 ± 0.00 Time0.30 ± 0.00 4.23 ± 0.07 0.01 ± 0.00 0.58 ± 0.02 0.19 ± 0.00 1.16 ± 0.00 100.98 ± 2.61 0.64 ± 0.01 16.89 ± 1.00 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE7.21 ± 0.96 0.37 ± 0.01 8.84 ± 0.57 7.41 ± 0.74 7.11 ± 0.41 7.32 ± 0.45 1.73 ± 0.03 4.66 ± 0.07 1.42 ± 0.22 Rank 6.50 ± 0.83 1.00 ± 0.00 8.75 ± 0.22 6.75 ± 0.65 6.00 ± 0.35 7.00 ± 0.35 2.75 ± 0.22 4.00 ± 0.00 2.25 ± 0.22 Time0.30 ± 0.00 6.26 ± 0.06 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.17 ± 0.00 203.66 ± 4.41 0.85 ± 0.00 37.97 ± 3.61 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE9.08 ± 2.10 0.34 ± 0.05 7.38 ± 2.11 4.73 ± 1.84 9.76 ± 1.49 9.47 ± 1.60 2.52 ± 0.24 7.71 ± 0.59 2.27 ± 0.73 Rank 7.50 ± 0.83 1.00 ± 0.00 5.75 ± 0.22 3.75 ± 0.41 8.00 ± 0.00 7.50 ± 0.43 3.25 ± 0.41 6.00 ± 0.94 2.25 ± 0.22 Time0.31 ± 0.01 8.63 ± 0.19 0.01 ± 0.00 0.57 ± 0.01 0.27 ± 0.01 1.19 ± 0.02 299.63 ± 8.63 1.06 ± 0.00 45.31 ± 6.14 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 40 MAE11.47 ± 3.51 0.47 ± 0.07 10.77 ± 2.61 6.18 ± 1.44 12.73 ± 1.67 12.24 ± 1.80 2.84 ± 0.13 10.24 ± 1.17 6.95 ± 3.13 Rank6.50 ± 1.64 1.25 ± 0.22 5.50 ± 0.25 4.25 ± 0.54 8.25 ± 0.41 7.00 ± 0.61 2.50 ± 0.43 5.50 ± 0.56 4.25 ± 1.08 Time0.32 ± 0.01 10.81 ± 0.13 0.05 ± 0.01 0.61 ± 0.01 6.02 ± 5.13 1.71 ± 0.39 423.31 ± 3.37 1.26 ± 0.01 50.63 ± 3.56 Time Rank2.75 ± 0.22 6.75 ± 0.22 1.00 ± 0.00 3.75 ± 0.22 3.25 ± 1.08 5.50 ± 0.25 9.00 ± 0.00 5.00 ± 0.35 8.00 ± 0.00 Dim 50 MAE12.85 ± 10.93 0.95 ± 0.62 5.29 ± 2.61 6.76 ± 0.14 11.93 ± 2.40 11.60 ± 2.51 2.86 ± 0.29 10.65 ± 1.39 2.72 ± 1.41 Rank3.00 ± 1.73 1.75 ± 0.22 4.25 ± 0.22 5.25 ± 0.65 8.50 ± 0.43 7.25 ± 0.22 4.25 ± 0.65 7.50 ± 0.43 3.25 ± 0.22 Time0.31 ± 0.01 12.70 ± 0.19 0.02 ± 0.00 0.62 ± 0.02 0.40 ± 0.02 1.20 ± 0.00 532.32 ± 5.85 1.75 ± 0.02 84.03 ± 7.46 Time Rank2.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE7.98 ± 1.78 0.44 ± 0.09 5.15 ± 0.85 4.21 ± 0.59 5.97 ± 0.95 5.70 ± 0.95 2.01 ± 0.13 4.52 ± 0.78 1.82 ± 0.58 Rank6.56 ± 0.53 1.62 ± 0.14 6.34 ± 0.30 5.81 ± 0.27 6.84 ± 0.25 6.16 ± 0.26 4.88 ± 0.47 4.31 ± 0.37 2.47 ± 0.22 Time0.30 ± 0.00 6.07 ± 0.71 0.01 ± 0.00 0.58 ± 0.01 0.95 ± 0.72 1.24 ± 0.06 202.59 ± 33.01 0.83 ± 0.09 32.77 ± 4.65 Time Rank2.94 ± 0.10 6.97 ± 0.05 1.00 ± 0.00 4.34 ± 0.09 2.44 ± 0.16 5.88 ± 0.09 8.72 ± 0.11 4.50 ± 0.20 8.22 ± 0.07 35 arXiv preprint Table 16: Density estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE0.05 ± 0.02 0.04 ± 0.01 0.45 ± 0.05 0.34 ± 0.06 0.18 ± 0.02 0.19 ± 0.01 2.39 ± 0.02 0.20 ± 0.05 0.13 ± 0.03 Rank2.00 ± 0.31 1.25 ± 0.12 8.00 ± 0.00 7.00 ± 0.00 4.75 ± 0.22 5.00 ± 0.61 9.00 ± 0.00 4.75 ± 0.65 3.25 ± 0.41 Time0.29 ± 0.00 1.79 ± 0.45 0.06 ± 0.04 0.73 ± 0.10 0.17 ± 0.00 1.15 ± 0.00 1.97 ± 0.84 0.15 ± 0.00 8.54 ± 0.10 Time Rank4.00 ± 0.00 7.75 ± 0.22 1.50 ± 0.43 5.00 ± 0.00 2.50 ± 0.25 6.75 ± 0.22 6.50 ± 0.43 2.00 ± 0.35 9.00 ± 0.00 Dim 2 MAE0.42 ± 0.16 0.27 ± 0.11 0.52 ± 0.09 0.59 ± 0.08 0.70 ± 0.08 0.68 ± 0.02 1.53 ± 0.29 0.29 ± 0.02 0.29 ± 0.02 Rank 4.00 ± 1.06 1.50 ± 0.43 4.75 ± 0.22 6.00 ± 0.00 7.50 ± 0.25 6.75 ± 0.82 9.00 ± 0.00 2.50 ± 0.25 3.00 ± 0.61 Time0.30 ± 0.00 2.91 ± 0.43 0.01 ± 0.00 0.55 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 6.33 ± 1.31 0.44 ± 0.00 11.62 ± 0.65 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.25 ± 0.22 4.00 ± 0.00 8.75 ± 0.22 Dim 5 MAE0.65 ± 0.04 0.66 ± 0.37 0.74 ± 0.06 0.76 ± 0.02 0.72 ± 0.05 0.68 ± 0.06 0.94 ± 0.06 0.93 ± 0.21 0.62 ± 0.06 Rank4.50 ± 1.03 3.00 ± 1.73 5.75 ± 0.65 6.25 ± 1.08 5.25 ± 0.65 3.75 ± 0.74 8.25 ± 0.22 5.50 ± 1.48 2.75 ± 0.41 Time0.30 ± 0.00 7.46 ± 0.43 0.01 ± 0.00 0.56 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 49.20 ± 0.75 0.55 ± 0.00 12.97 ± 0.60 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.75 ± 0.22 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.25 ± 0.22 8.00 ± 0.00 Dim 10 MAE2.67 ± 0.61 1.67 ± 0.47 1.91 ± 0.23 2.34 ± 0.24 1.91 ± 0.22 2.06 ± 0.23 1.30 ± 0.07 2.85 ± 0.29 1.80 ± 0.23 Rank7.00 ± 1.46 3.50 ± 1.30 4.25 ± 0.41 6.75 ± 0.41 3.75 ± 0.22 6.00 ± 0.35 1.75 ± 0.41 8.50 ± 0.25 3.50 ± 1.03 Time0.30 ± 0.00 10.25 ± 1.59 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.02 1.17 ± 0.00 98.49 ± 1.67 0.80 ± 0.10 22.07 ± 0.37 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE5.35 ± 0.54 2.43 ± 0.19 5.27 ± 0.59 3.67 ± 0.55 5.31 ± 0.23 5.25 ± 0.30 1.84 ± 0.13 6.88 ± 0.54 3.75 ± 0.49 Rank6.25 ± 0.65 2.25 ± 0.22 6.25 ± 0.54 3.75 ± 0.22 7.25 ± 0.41 6.25 ± 0.41 1.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.35 Time0.31 ± 0.00 22.32 ± 2.75 0.01 ± 0.00 0.56 ± 0.00 0.20 ± 0.00 1.16 ± 0.00 202.94 ± 4.64 0.94 ± 0.00 57.53 ± 1.62 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE13.52 ± 1.37 7.00 ± 0.21 8.37 ± 0.41 6.30 ± 0.27 11.02 ± 0.34 10.80 ± 0.37 3.16 ± 0.22 16.35 ± 0.88 8.94 ± 0.54 Rank7.50 ± 0.43 3.25 ± 0.22 3.75 ± 0.22 2.00 ± 0.00 7.25 ± 0.22 6.25 ± 0.22 1.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 Time0.35 ± 0.02 31.73 ± 1.64 0.39 ± 0.33 9.83 ± 8.02 0.50 ± 0.17 1.55 ± 0.33 310.60 ± 9.44 1.62 ± 0.29 65.66 ± 0.27 Time Rank 2.25 ± 0.41 6.75 ± 0.22 1.75 ± 0.65 4.75 ± 0.65 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 5.25 ± 0.65 8.00 ± 0.00 Dim 40 MAE13.87 ± 0.54 6.91 ± 0.34 11.36 ± 1.00 7.60 ± 1.45 14.63 ± 0.79 14.12 ± 0.83 3.70 ± 0.25 20.49 ± 0.96 10.45 ± 0.99 Rank6.75 ± 0.41 2.50 ± 0.25 5.00 ± 0.00 2.50 ± 0.25 7.75 ± 0.22 6.50 ± 0.25 1.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 Time0.35 ± 0.00 28.34 ± 6.70 0.03 ± 0.00 0.61 ± 0.01 0.12 ± 0.01 1.27 ± 0.03 426.88 ± 9.16 1.96 ± 0.45 62.42 ± 4.14 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Dim 50 MAE 20.94 ± 1.98 12.49 ± 1.24 11.76 ± 1.45 11.47 ± 3.28 20.78 ± 1.30 20.59 ± 1.33 5.93 ± 0.87 30.03 ± 1.83 14.79 ± 1.53 Rank 7.00 ± 0.50 3.00 ± 0.50 3.00 ± 0.00 3.00 ± 0.79 7.50 ± 0.25 6.50 ± 0.25 1.25 ± 0.22 9.00 ± 0.00 4.75 ± 0.22 Time0.36 ± 0.00 44.74 ± 2.11 0.08 ± 0.01 0.62 ± 0.02 0.34 ± 0.01 1.23 ± 0.03 534.85 ± 7.03 2.66 ± 0.50 87.71 ± 6.93 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE7.18 ± 1.35 3.93 ± 0.76 5.05 ± 0.84 4.13 ± 0.80 6.91 ± 1.29 6.80 ± 1.27 2.60 ± 0.30 9.75 ± 1.88 5.10 ± 0.96 Rank5.62 ± 0.43 2.53 ± 0.32 5.09 ± 0.29 4.66 ± 0.38 6.38 ± 0.28 5.88 ± 0.25 4.03 ± 0.65 7.16 ± 0.47 3.66 ± 0.22 Time0.32 ± 0.01 18.69 ± 2.75 0.07 ± 0.05 1.75 ± 1.14 0.24 ± 0.03 1.23 ± 0.05 203.91 ± 33.45 1.14 ± 0.17 41.07 ± 5.18 Time Rank3.00 ± 0.10 7.06 ± 0.06 1.16 ± 0.11 4.44 ± 0.12 2.12 ± 0.06 5.72 ± 0.11 8.59 ± 0.16 4.69 ± 0.24 8.22 ± 0.07 36 arXiv preprint Table 17: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Corr 0.82 ± 0.06 0.95 ± 0.03 0.60 ± 0.08 0.74 ± 0.09 0.90 ± 0.02 0.88 ± 0.03 0.87 ± 0.03 0.94 ± 0.02 0.95 ± 0.02 Rank5.21 ± 1.03 2.79 ± 0.63 9.00 ± 0.00 6.43 ± 0.60 4.57 ± 0.83 5.43 ± 0.67 5.86 ± 0.47 2.71 ± 0.48 3.00 ± 0.61 Time0.29 ± 0.00 1.50 ± 0.29 0.03 ± 0.03 0.65 ± 0.07 0.17 ± 0.00 1.15 ± 0.00 3.40 ± 1.69 0.14 ± 0.00 8.44 ± 0.07 Time Rank4.00 ± 0.00 7.43 ± 0.28 1.29 ± 0.26 5.00 ± 0.00 2.86 ± 0.13 6.71 ± 0.17 7.00 ± 0.45 1.86 ± 0.13 8.86 ± 0.13 Dim 2 Corr0.76 ± 0.08 0.91 ± 0.04 0.80 ± 0.04 0.80 ± 0.05 0.92 ± 0.02 0.79 ± 0.03 0.73 ± 0.07 0.95 ± 0.01 0.93 ± 0.03 Rank5.69 ± 1.11 2.81 ± 0.61 7.00 ± 0.43 6.25 ± 0.58 4.00 ± 0.31 6.50 ± 0.61 7.12 ± 0.82 2.62 ± 0.43 3.00 ± 0.71 Time0.29 ± 0.00 2.26 ± 0.32 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 5.51 ± 0.82 0.43 ± 0.00 10.48 ± 0.55 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.12 ± 0.12 4.00 ± 0.00 8.88 ± 0.12 Dim 5 Corr0.63 ± 0.13 0.90 ± 0.04 0.81 ± 0.04 0.81 ± 0.05 0.88 ± 0.02 0.90 ± 0.01 0.83 ± 0.02 0.92 ± 0.01 0.94 ± 0.02 Rank6.06 ± 1.13 3.06 ± 0.90 7.25 ± 0.46 6.88 ± 0.51 5.12 ± 0.51 4.12 ± 0.45 7.00 ± 0.61 3.12 ± 0.60 2.38 ± 0.39 Time0.30 ± 0.00 5.31 ± 0.79 0.01 ± 0.00 0.57 ± 0.01 0.19 ± 0.01 1.16 ± 0.00 50.08 ± 0.77 0.53 ± 0.01 11.34 ± 0.67 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.88 ± 0.12 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.12 ± 0.12 8.00 ± 0.00 Dim 10 Corr0.53 ± 0.10 0.92 ± 0.03 0.70 ± 0.04 0.69 ± 0.04 0.82 ± 0.02 0.80 ± 0.02 0.87 ± 0.01 0.83 ± 0.03 0.88 ± 0.03 Rank7.88 ± 0.41 1.50 ± 0.18 7.62 ± 0.30 7.62 ± 0.50 3.88 ± 0.33 5.75 ± 0.42 3.25 ± 0.74 4.62 ± 0.43 2.88 ± 0.67 Time0.30 ± 0.00 7.24 ± 1.33 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.16 ± 0.00 99.74 ± 1.61 0.72 ± 0.06 19.48 ± 1.06 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Corr0.69 ± 0.03 0.91 ± 0.03 0.74 ± 0.03 0.76 ± 0.03 0.78 ± 0.02 0.78 ± 0.02 0.87 ± 0.01 0.73 ± 0.04 0.83 ± 0.04 Rank7.75 ± 0.58 1.38 ± 0.17 7.88 ± 0.21 6.00 ± 0.31 4.38 ± 0.30 5.00 ± 0.50 2.00 ± 0.31 7.12 ± 0.72 3.50 ± 0.53 Time0.31 ± 0.00 14.29 ± 3.16 0.01 ± 0.00 0.57 ± 0.00 0.20 ± 0.00 1.17 ± 0.00 203.30 ± 3.20 0.89 ± 0.02 47.75 ± 3.98 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Corr0.64 ± 0.04 0.89 ± 0.04 0.67 ± 0.05 0.69 ± 0.05 0.72 ± 0.04 0.72 ± 0.04 0.84 ± 0.01 0.63 ± 0.04 0.73 ± 0.06 Rank 6.12 ± 0.91 1.38 ± 0.17 6.88 ± 0.57 5.88 ± 0.57 5.38 ± 0.47 4.62 ± 0.43 2.12 ± 0.45 8.25 ± 0.39 4.38 ± 0.68 Time0.33 ± 0.01 20.18 ± 4.17 0.20 ± 0.18 5.20 ± 4.33 0.39 ± 0.10 1.37 ± 0.18 305.11 ± 6.68 1.34 ± 0.18 55.48 ± 4.73 Time Rank2.50 ± 0.25 6.88 ± 0.12 1.38 ± 0.35 4.38 ± 0.35 2.25 ± 0.15 5.50 ± 0.18 9.00 ± 0.00 5.12 ± 0.33 8.00 ± 0.00 Dim 40 Corr0.70 ± 0.05 0.90 ± 0.03 0.66 ± 0.05 0.71 ± 0.05 0.71 ± 0.04 0.72 ± 0.04 0.84 ± 0.01 0.65 ± 0.05 0.74 ± 0.05 Rank5.44 ± 1.03 1.44 ± 0.16 7.75 ± 0.42 5.38 ± 0.58 5.88 ± 0.45 5.00 ± 0.47 2.25 ± 0.55 7.88 ± 0.37 4.00 ± 0.50 Time0.33 ± 0.01 19.58 ± 4.56 0.04 ± 0.01 0.61 ± 0.01 3.07 ± 2.77 1.49 ± 0.21 425.10 ± 4.92 1.61 ± 0.26 56.53 ± 3.43 Time Rank2.88 ± 0.12 6.88 ± 0.12 1.00 ± 0.00 3.88 ± 0.12 2.62 ± 0.58 5.25 ± 0.15 9.00 ± 0.00 5.50 ± 0.25 8.00 ± 0.00 Dim 50 Corr0.74 ± 0.07 0.91 ± 0.04 0.65 ± 0.07 0.70 ± 0.08 0.68 ± 0.06 0.68 ± 0.06 0.84 ± 0.02 0.63 ± 0.07 0.71 ± 0.08 Rank3.19 ± 0.81 1.44 ± 0.14 7.38 ± 0.39 5.25 ± 0.55 6.50 ± 0.68 5.25 ± 0.58 2.88 ± 0.60 8.00 ± 0.35 5.12 ± 0.60 Time0.34 ± 0.01 28.72 ± 5.76 0.05 ± 0.01 0.62 ± 0.01 0.37 ± 0.02 1.21 ± 0.02 533.58 ± 4.60 2.21 ± 0.30 85.87 ± 5.13 Time Rank2.38 ± 0.17 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.62 ± 0.17 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Corr0.69 ± 0.03 0.91 ± 0.01 0.71 ± 0.02 0.74 ± 0.02 0.80 ± 0.02 0.78 ± 0.02 0.84 ± 0.01 0.78 ± 0.02 0.84 ± 0.02 Rank 5.93 ± 0.37 1.96 ± 0.19 7.57 ± 0.16 6.21 ± 0.21 4.97 ± 0.21 5.21 ± 0.20 4.03 ± 0.34 5.59 ± 0.34 3.54 ± 0.24 Time0.31 ± 0.00 12.56 ± 1.64 0.04 ± 0.02 1.18 ± 0.58 0.60 ± 0.37 1.24 ± 0.04 206.40 ± 23.66 1.00 ± 0.10 37.37 ± 3.54 Time Rank2.95 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.38 ± 0.08 2.29 ± 0.09 5.79 ± 0.07 8.67 ± 0.10 4.62 ± 0.16 8.21 ± 0.05 37 arXiv preprint Table 18: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Corr 0.68 ± 0.04 0.91 ± 0.04 0.50 ± 0.11 0.64 ± 0.14 0.92 ± 0.03 0.90 ± 0.02 0.87 ± 0.04 0.96 ± 0.01 0.95 ± 0.02 Rank7.50 ± 0.25 3.25 ± 0.96 9.00 ± 0.00 7.00 ± 0.61 3.00 ± 0.79 5.00 ± 0.61 5.50 ± 0.56 2.00 ± 0.35 2.75 ± 0.54 Time0.30 ± 0.00 1.16 ± 0.02 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 5.20 ± 2.61 0.13 ± 0.00 8.30 ± 0.05 Time Rank4.00 ± 0.00 7.00 ± 0.35 1.00 ± 0.00 5.00 ± 0.00 3.00 ± 0.00 6.50 ± 0.25 7.75 ± 0.54 2.00 ± 0.00 8.75 ± 0.22 Dim 2 Corr0.67 ± 0.14 0.92 ± 0.06 0.89 ± 0.05 0.86 ± 0.07 0.95 ± 0.02 0.82 ± 0.04 0.75 ± 0.07 0.98 ± 0.00 0.99 ± 0.00 Rank7.12 ± 1.62 3.12 ± 0.87 6.00 ± 0.35 5.75 ± 0.82 4.25 ± 0.41 6.25 ± 0.89 8.00 ± 0.35 3.00 ± 0.50 1.50 ± 0.43 Time0.29 ± 0.00 1.60 ± 0.05 0.01 ± 0.00 0.56 ± 0.00 0.17 ± 0.01 1.16 ± 0.00 4.68 ± 0.78 0.43 ± 0.00 9.34 ± 0.39 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 Corr0.40 ± 0.19 0.88 ± 0.06 0.79 ± 0.08 0.78 ± 0.09 0.86 ± 0.04 0.89 ± 0.02 0.84 ± 0.02 0.93 ± 0.01 0.98 ± 0.00 Rank7.12 ± 1.62 3.62 ± 1.19 6.50 ± 0.43 7.00 ± 0.87 5.25 ± 0.82 4.25 ± 0.65 6.00 ± 0.94 3.50 ± 1.03 1.75 ± 0.41 Time0.30 ± 0.00 3.16 ± 0.14 0.01 ± 0.00 0.57 ± 0.02 0.18 ± 0.01 1.16 ± 0.00 50.96 ± 1.20 0.51 ± 0.01 9.71 ± 0.33 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 8.00 ± 0.00 Dim 10 Corr0.30 ± 0.11 0.97 ± 0.01 0.69 ± 0.06 0.68 ± 0.07 0.81 ± 0.02 0.78 ± 0.02 0.88 ± 0.01 0.89 ± 0.01 0.96 ± 0.00 Rank9.00 ± 0.00 1.50 ± 0.25 7.00 ± 0.35 6.50 ± 0.56 4.75 ± 0.22 6.75 ± 0.41 4.50 ± 1.03 3.50 ± 0.25 1.50 ± 0.25 Time0.30 ± 0.00 4.23 ± 0.07 0.01 ± 0.00 0.58 ± 0.02 0.19 ± 0.00 1.16 ± 0.00 100.98 ± 2.61 0.64 ± 0.01 16.89 ± 1.00 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Corr0.69 ± 0.05 0.99 ± 0.00 0.78 ± 0.02 0.80 ± 0.02 0.81 ± 0.02 0.81 ± 0.02 0.88 ± 0.01 0.82 ± 0.02 0.93 ± 0.02 Rank8.75 ± 0.22 1.00 ± 0.00 8.00 ± 0.35 6.25 ± 0.22 4.50 ± 0.25 6.00 ± 0.50 2.75 ± 0.22 5.50 ± 0.83 2.25 ± 0.22 Time0.30 ± 0.00 6.26 ± 0.06 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.17 ± 0.00 203.66 ± 4.41 0.85 ± 0.00 37.97 ± 3.61 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Corr0.65 ± 0.08 0.99 ± 0.00 0.79 ± 0.05 0.81 ± 0.05 0.81 ± 0.02 0.81 ± 0.02 0.87 ± 0.01 0.75 ± 0.03 0.88 ± 0.05 Rank 8.00 ± 0.61 1.00 ± 0.00 5.75 ± 0.74 4.75 ± 0.74 5.75 ± 0.89 5.25 ± 0.74 3.00 ± 0.61 8.00 ± 0.61 3.50 ± 1.03 Time0.31 ± 0.01 8.63 ± 0.19 0.01 ± 0.00 0.57 ± 0.01 0.27 ± 0.01 1.19 ± 0.02 299.63 ± 8.63 1.06 ± 0.00 45.31 ± 6.14 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 40 Corr0.74 ± 0.08 0.99 ± 0.00 0.78 ± 0.04 0.84 ± 0.04 0.81 ± 0.01 0.81 ± 0.01 0.86 ± 0.01 0.79 ± 0.02 0.88 ± 0.03 Rank6.88 ± 1.56 1.12 ± 0.11 7.00 ± 0.61 4.75 ± 0.65 6.25 ± 0.82 5.75 ± 0.74 3.25 ± 0.82 7.25 ± 0.54 2.75 ± 0.22 Time0.32 ± 0.01 10.81 ± 0.13 0.05 ± 0.01 0.61 ± 0.01 6.02 ± 5.13 1.71 ± 0.39 423.31 ± 3.37 1.26 ± 0.01 50.63 ± 3.56 Time Rank2.75 ± 0.22 6.75 ± 0.22 1.00 ± 0.00 3.75 ± 0.22 3.25 ± 1.08 5.50 ± 0.25 9.00 ± 0.00 5.00 ± 0.35 8.00 ± 0.00 Dim 50 Corr0.88 ± 0.10 0.98 ± 0.01 0.83 ± 0.03 0.91 ± 0.04 0.83 ± 0.02 0.83 ± 0.02 0.89 ± 0.01 0.83 ± 0.02 0.92 ± 0.05 Rank3.38 ± 1.62 1.38 ± 0.11 6.50 ± 0.43 3.75 ± 0.22 8.00 ± 0.87 6.50 ± 0.75 4.25 ± 0.65 7.25 ± 0.41 4.00 ± 0.87 Time0.31 ± 0.01 12.70 ± 0.19 0.02 ± 0.00 0.62 ± 0.02 0.40 ± 0.02 1.20 ± 0.00 532.32 ± 5.85 1.75 ± 0.02 84.03 ± 7.46 Time Rank2.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Corr0.63 ± 0.05 0.95 ± 0.01 0.76 ± 0.03 0.79 ± 0.03 0.85 ± 0.01 0.83 ± 0.01 0.86 ± 0.01 0.87 ± 0.02 0.94 ± 0.01 Rank 7.22 ± 0.50 2.00 ± 0.29 6.97 ± 0.24 5.72 ± 0.30 5.22 ± 0.35 5.72 ± 0.28 4.66 ± 0.38 5.00 ± 0.44 2.50 ± 0.25 Time0.30 ± 0.00 6.07 ± 0.71 0.01 ± 0.00 0.58 ± 0.01 0.95 ± 0.72 1.24 ± 0.06 202.59 ± 33.01 0.83 ± 0.09 32.77 ± 4.65 Time Rank2.94 ± 0.10 6.97 ± 0.05 1.00 ± 0.00 4.34 ± 0.09 2.44 ± 0.16 5.88 ± 0.09 8.72 ± 0.11 4.50 ± 0.20 8.22 ± 0.07 38 arXiv preprint Table 19: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Corr 1.00 ± 0.00 1.00 ± 0.00 0.73 ± 0.05 0.88 ± 0.06 0.88 ± 0.03 0.87 ± 0.05 0.88 ± 0.02 0.92 ± 0.04 0.96 ± 0.03 Rank2.17 ± 0.54 2.17 ± 0.54 9.00 ± 0.00 5.67 ± 0.98 6.67 ± 0.27 6.00 ± 1.25 6.33 ± 0.72 3.67 ± 0.72 3.33 ± 1.19 Time0.29 ± 0.00 1.95 ± 0.57 0.07 ± 0.05 0.78 ± 0.12 0.17 ± 0.00 1.15 ± 0.00 1.00 ± 0.04 0.14 ± 0.00 8.64 ± 0.05 Time Rank4.00 ± 0.00 8.00 ± 0.00 1.67 ± 0.54 5.00 ± 0.00 2.67 ± 0.27 7.00 ± 0.00 6.00 ± 0.00 1.67 ± 0.27 9.00 ± 0.00 Dim 2 Corr0.86 ± 0.08 0.90 ± 0.07 0.72 ± 0.04 0.75 ± 0.05 0.89 ± 0.02 0.75 ± 0.03 0.71 ± 0.11 0.92 ± 0.02 0.86 ± 0.02 Rank4.25 ± 1.14 2.50 ± 0.83 8.00 ± 0.35 6.75 ± 0.74 3.75 ± 0.41 6.75 ± 0.82 6.25 ± 1.47 2.25 ± 0.65 4.50 ± 0.83 Time0.30 ± 0.00 2.91 ± 0.43 0.01 ± 0.00 0.55 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 6.33 ± 1.31 0.44 ± 0.00 11.62 ± 0.65 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.25 ± 0.22 4.00 ± 0.00 8.75 ± 0.22 Dim 5 Corr0.87 ± 0.05 0.93 ± 0.05 0.83 ± 0.02 0.85 ± 0.02 0.90 ± 0.02 0.90 ± 0.02 0.82 ± 0.03 0.91 ± 0.02 0.90 ± 0.03 Rank5.00 ± 1.37 2.50 ± 1.30 8.00 ± 0.61 6.75 ± 0.54 5.00 ± 0.61 4.00 ± 0.61 8.00 ± 0.35 2.75 ± 0.54 3.00 ± 0.50 Time0.30 ± 0.00 7.46 ± 0.43 0.01 ± 0.00 0.56 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 49.20 ± 0.75 0.55 ± 0.00 12.97 ± 0.60 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.75 ± 0.22 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.25 ± 0.22 8.00 ± 0.00 Dim 10 Corr0.75 ± 0.04 0.86 ± 0.04 0.71 ± 0.05 0.70 ± 0.05 0.83 ± 0.04 0.81 ± 0.04 0.86 ± 0.02 0.76 ± 0.04 0.81 ± 0.04 Rank6.75 ± 0.22 1.50 ± 0.25 8.25 ± 0.22 8.75 ± 0.22 3.00 ± 0.00 4.75 ± 0.22 2.00 ± 0.61 5.75 ± 0.22 4.25 ± 0.89 Time0.30 ± 0.00 10.25 ± 1.59 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.02 1.17 ± 0.00 98.49 ± 1.67 0.80 ± 0.10 22.07 ± 0.37 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Corr0.68 ± 0.04 0.83 ± 0.03 0.70 ± 0.04 0.72 ± 0.04 0.75 ± 0.04 0.75 ± 0.04 0.85 ± 0.02 0.65 ± 0.04 0.74 ± 0.03 Rank6.75 ± 0.89 1.75 ± 0.22 7.75 ± 0.22 5.75 ± 0.54 4.25 ± 0.54 4.00 ± 0.50 1.25 ± 0.22 8.75 ± 0.22 4.75 ± 0.54 Time0.31 ± 0.00 22.32 ± 2.75 0.01 ± 0.00 0.56 ± 0.00 0.20 ± 0.00 1.16 ± 0.00 202.94 ± 4.64 0.94 ± 0.00 57.53 ± 1.62 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Corr0.63 ± 0.03 0.79 ± 0.01 0.55 ± 0.01 0.57 ± 0.01 0.62 ± 0.02 0.62 ± 0.02 0.81 ± 0.01 0.52 ± 0.02 0.58 ± 0.01 Rank4.25 ± 1.08 1.75 ± 0.22 8.00 ± 0.35 7.00 ± 0.35 5.00 ± 0.00 4.00 ± 0.00 1.25 ± 0.22 8.50 ± 0.43 5.25 ± 0.65 Time0.35 ± 0.02 31.73 ± 1.64 0.39 ± 0.33 9.83 ± 8.02 0.50 ± 0.17 1.55 ± 0.33 310.60 ± 9.44 1.62 ± 0.29 65.66 ± 0.27 Time Rank2.25 ± 0.41 6.75 ± 0.22 1.75 ± 0.65 4.75 ± 0.65 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 5.25 ± 0.65 8.00 ± 0.00 Dim 40 Corr0.66 ± 0.03 0.80 ± 0.01 0.54 ± 0.03 0.59 ± 0.03 0.61 ± 0.03 0.62 ± 0.03 0.82 ± 0.01 0.52 ± 0.02 0.60 ± 0.02 Rank 4.00 ± 0.87 1.75 ± 0.22 8.50 ± 0.25 6.00 ± 0.87 5.50 ± 0.25 4.25 ± 0.22 1.25 ± 0.22 8.50 ± 0.25 5.25 ± 0.41 Time0.35 ± 0.00 28.34 ± 6.70 0.03 ± 0.00 0.61 ± 0.01 0.12 ± 0.01 1.27 ± 0.03 426.88 ± 9.16 1.96 ± 0.45 62.42 ± 4.14 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Dim 50 Corr0.61 ± 0.02 0.84 ± 0.06 0.46 ± 0.04 0.49 ± 0.04 0.53 ± 0.05 0.53 ± 0.05 0.80 ± 0.02 0.43 ± 0.05 0.50 ± 0.04 Rank3.00 ± 0.00 1.50 ± 0.25 8.25 ± 0.22 6.75 ± 0.22 5.00 ± 0.00 4.00 ± 0.00 1.50 ± 0.25 8.75 ± 0.22 6.25 ± 0.22 Time0.36 ± 0.00 44.74 ± 2.11 0.08 ± 0.01 0.62 ± 0.02 0.34 ± 0.01 1.23 ± 0.03 534.85 ± 7.03 2.66 ± 0.50 87.71 ± 6.93 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Corr0.75 ± 0.03 0.86 ± 0.02 0.65 ± 0.02 0.69 ± 0.03 0.75 ± 0.03 0.73 ± 0.03 0.82 ± 0.02 0.70 ± 0.04 0.74 ± 0.03 Rank 4.60 ± 0.42 1.92 ± 0.23 8.19 ± 0.13 6.71 ± 0.27 4.71 ± 0.22 4.68 ± 0.26 3.39 ± 0.53 6.19 ± 0.51 4.61 ± 0.30 Time0.32 ± 0.01 19.25 ± 2.78 0.08 ± 0.05 1.79 ± 1.17 0.24 ± 0.03 1.23 ± 0.05 210.33 ± 33.91 1.17 ± 0.17 42.12 ± 5.23 Time Rank2.97 ± 0.10 7.06 ± 0.06 1.16 ± 0.11 4.42 ± 0.12 2.13 ± 0.06 5.71 ± 0.11 8.61 ± 0.16 4.74 ± 0.24 8.19 ± 0.07 39 arXiv preprint Table 20: Density estimation performance via NLL (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one. DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM Dim 1 NLL0.63 ± 0.26 0.58 ± 0.26 0.66 ± 0.27 0.58 ± 0.26 0.61 ± 0.25 0.58 ± 0.25 Rank 3.81 ± 0.57 2.69 ± 0.35 4.75 ± 0.41 3.62 ± 0.23 3.25 ± 0.57 2.88 ± 0.36 Time0.29 ± 0.00 1.48 ± 0.25 0.17 ± 0.00 1.15 ± 0.00 0.14 ± 0.00 8.42 ± 0.07 Time Rank 3.00 ± 0.00 4.88 ± 0.12 1.88 ± 0.12 4.12 ± 0.12 1.12 ± 0.12 6.00 ± 0.00 Dim 2 NLL1.46 ± 0.60 1.32 ± 0.57 1.55 ± 0.65 1.50 ± 0.64 1.31 ± 0.58 1.29 ± 0.58 Rank3.62 ± 0.57 2.62 ± 0.37 4.50 ± 0.37 4.00 ± 0.22 3.38 ± 0.21 2.88 ± 0.30 Time0.29 ± 0.00 2.26 ± 0.32 0.18 ± 0.01 1.16 ± 0.00 0.43 ± 0.00 10.48 ± 0.55 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 5 NLL 6.36 ± 1.46 4.76 ± 1.34 5.02 ± 1.41 4.81 ± 1.39 4.80 ± 1.47 4.48 ± 1.40 Rank3.75 ± 0.63 2.75 ± 0.52 5.00 ± 0.31 4.25 ± 0.29 3.62 ± 0.47 1.62 ± 0.25 Time0.30 ± 0.00 5.31 ± 0.79 0.19 ± 0.01 1.16 ± 0.00 0.53 ± 0.01 11.34 ± 0.67 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 10 NLL12.86 ± 4.44 7.33 ± 2.71 7.83 ± 2.72 7.73 ± 2.79 7.78 ± 2.90 7.19 ± 2.68 Rank 5.44 ± 0.35 2.81 ± 0.47 3.19 ± 0.33 3.56 ± 0.24 3.81 ± 0.45 2.19 ± 0.31 Time0.30 ± 0.00 7.24 ± 1.33 0.20 ± 0.01 1.16 ± 0.00 0.72 ± 0.06 19.48 ± 1.06 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 NLL8.39 ± 4.31 6.35 ± 3.31 8.26 ± 4.08 8.25 ± 4.08 7.83 ± 3.90 6.47 ± 3.38 Rank3.94 ± 0.29 2.69 ± 0.38 3.94 ± 0.22 3.94 ± 0.34 3.69 ± 0.32 2.81 ± 0.33 Time0.31 ± 0.00 14.29 ± 3.16 0.20 ± 0.00 1.17 ± 0.00 0.89 ± 0.02 47.75 ± 3.98 Time Rank 2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 NLL9.22 ± 4.16 5.35 ± 2.62 8.28 ± 3.66 8.19 ± 3.65 9.62 ± 4.13 6.30 ± 2.97 Rank3.88 ± 0.25 2.38 ± 0.41 4.00 ± 0.22 3.75 ± 0.32 4.38 ± 0.45 2.62 ± 0.33 Time0.33 ± 0.01 20.18 ± 4.17 0.39 ± 0.10 1.37 ± 0.18 1.34 ± 0.18 55.48 ± 4.73 Time Rank1.62 ± 0.17 5.00 ± 0.00 1.38 ± 0.17 3.62 ± 0.17 3.38 ± 0.17 6.00 ± 0.00 Dim 40 NLL38.18 ± 12.25 31.75 ± 11.55 37.94 ± 12.32 37.71 ± 12.26 39.31 ± 13.33 35.45 ± 11.88 Rank4.19 ± 0.34 1.94 ± 0.43 4.31 ± 0.31 3.44 ± 0.14 4.19 ± 0.52 2.94 ± 0.37 Time0.33 ± 0.01 19.58 ± 4.56 3.07 ± 2.77 1.49 ± 0.21 1.61 ± 0.26 56.53 ± 3.43 Time Rank1.88 ± 0.12 4.88 ± 0.12 1.50 ± 0.47 3.25 ± 0.15 3.50 ± 0.25 6.00 ± 0.00 Dim 50 NLL27.38 ± 11.06 24.50 ± 10.27 28.01 ± 11.52 27.89 ± 11.47 31.57 ± 12.33 24.73 ± 10.46 Rank 3.50 ± 0.41 2.50 ± 0.37 4.12 ± 0.30 3.50 ± 0.12 4.62 ± 0.41 2.75 ± 0.32 Time0.34 ± 0.01 28.72 ± 5.76 0.37 ± 0.02 1.21 ± 0.02 2.21 ± 0.30 85.87 ± 5.13 Time Rank1.38 ± 0.17 5.00 ± 0.00 1.62 ± 0.17 3.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 Overall NLL5.00 ± 3.84 -0.22 ± 3.86 3.82 ± 3.83 3.60 ± 3.83 4.48 ± 4.00 0.72 ± 3.88 Rank 4.12 ± 0.21 1.67 ± 0.13 4.92 ± 0.12 4.09 ± 0.10 4.11 ± 0.20 2.08 ± 0.11 Time0.31 ± 0.00 12.38 ± 1.63 0.60 ± 0.37 1.23 ± 0.04 0.98 ± 0.10 36.92 ± 3.52 Time Rank1.98 ± 0.06 4.97 ± 0.02 1.30 ± 0.08 3.75 ± 0.06 3.00 ± 0.11 6.00 ± 0.00 40 arXiv preprint Table 21: Density estimation performance via NLL (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one. DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM Dim 1 NLL0.96 ± 0.45 0.86 ± 0.46 0.91 ± 0.49 0.84 ± 0.46 0.83 ± 0.45 0.83 ± 0.46 Rank 5.12 ± 0.51 3.38 ± 0.21 4.38 ± 0.60 3.38 ± 0.21 2.12 ± 0.57 2.62 ± 0.32 Time0.30 ± 0.00 1.16 ± 0.02 0.18 ± 0.01 1.16 ± 0.00 0.13 ± 0.00 8.30 ± 0.05 Time Rank 3.00 ± 0.00 4.75 ± 0.22 2.00 ± 0.00 4.25 ± 0.22 1.00 ± 0.00 6.00 ± 0.00 Dim 2 NLL2.35 ± 0.88 2.08 ± 0.88 2.41 ± 0.97 2.32 ± 0.97 2.02 ± 0.91 2.00 ± 0.91 Rank4.12 ± 1.04 2.38 ± 0.48 4.88 ± 0.45 4.12 ± 0.27 3.12 ± 0.37 2.38 ± 0.48 Time0.29 ± 0.00 1.60 ± 0.05 0.17 ± 0.01 1.16 ± 0.00 0.43 ± 0.00 9.34 ± 0.39 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 5 NLL 7.26 ± 1.97 3.85 ± 1.77 4.42 ± 1.89 4.00 ± 1.80 3.76 ± 1.80 3.51 ± 1.78 Rank5.00 ± 0.87 3.00 ± 0.00 4.75 ± 0.54 4.00 ± 0.35 2.75 ± 0.41 1.50 ± 0.43 Time0.30 ± 0.00 3.16 ± 0.14 0.18 ± 0.01 1.16 ± 0.00 0.51 ± 0.01 9.71 ± 0.33 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 10 NLL13.92 ± 7.92 3.69 ± 3.20 4.70 ± 3.44 4.41 ± 3.53 3.97 ± 3.44 3.77 ± 3.27 Rank6.00 ± 0.00 2.25 ± 0.41 3.75 ± 0.41 3.75 ± 0.41 2.75 ± 0.22 2.50 ± 0.25 Time0.30 ± 0.00 4.23 ± 0.07 0.19 ± 0.00 1.16 ± 0.00 0.64 ± 0.01 16.89 ± 1.00 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 NLL12.15 ± 7.16 8.58 ± 5.35 11.85 ± 6.62 11.90 ± 6.64 10.78 ± 6.23 8.92 ± 5.53 Rank4.25 ± 0.52 2.25 ± 0.62 4.00 ± 0.31 4.50 ± 0.53 3.25 ± 0.12 2.75 ± 0.38 Time0.30 ± 0.00 6.26 ± 0.06 0.20 ± 0.01 1.17 ± 0.00 0.85 ± 0.00 37.97 ± 3.61 Time Rank 2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 NLL5.51 ± 4.77 1.95 ± 1.69 5.55 ± 4.81 5.57 ± 4.82 4.32 ± 3.74 3.01 ± 2.60 Rank3.62 ± 0.11 2.88 ± 0.54 3.88 ± 0.32 4.12 ± 0.54 3.38 ± 0.11 3.12 ± 0.32 Time0.31 ± 0.01 8.63 ± 0.19 0.27 ± 0.01 1.19 ± 0.02 1.06 ± 0.00 45.31 ± 6.14 Time Rank1.75 ± 0.22 5.00 ± 0.00 1.25 ± 0.22 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 40 NLL17.83 ± 9.25 9.52 ± 5.03 17.12 ± 8.78 17.00 ± 8.72 15.32 ± 7.93 15.04 ± 8.38 Rank4.50 ± 0.53 2.25 ± 0.62 4.25 ± 0.52 3.50 ± 0.18 3.00 ± 0.31 3.50 ± 0.53 Time0.32 ± 0.01 10.81 ± 0.13 6.02 ± 5.13 1.71 ± 0.39 1.26 ± 0.01 50.63 ± 3.56 Time Rank1.75 ± 0.22 4.75 ± 0.22 2.00 ± 0.87 3.50 ± 0.25 3.00 ± 0.35 6.00 ± 0.00 Dim 50 NLL19.66 ± 17.03 19.66 ± 17.03 21.95 ± 19.01 21.83 ± 18.91 21.90 ± 18.97 19.85 ± 17.19 Rank2.88 ± 0.54 3.12 ± 0.32 4.12 ± 0.54 3.62 ± 0.11 3.88 ± 0.32 3.38 ± 0.11 Time0.31 ± 0.01 12.70 ± 0.19 0.40 ± 0.02 1.20 ± 0.00 1.75 ± 0.02 84.03 ± 7.46 Time Rank1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 3.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 Overall NLL-0.55 ± 5.14 -8.15 ± 5.24 -2.47 ± 5.09 -2.78 ± 5.09 -3.98 ± 5.11 -6.80 ± 5.29 Rank 4.62 ± 0.33 1.69 ± 0.16 5.19 ± 0.12 4.38 ± 0.13 3.00 ± 0.20 2.12 ± 0.14 Time0.30 ± 0.00 6.07 ± 0.71 0.95 ± 0.72 1.24 ± 0.06 0.83 ± 0.09 32.77 ± 4.65 Time Rank1.94 ± 0.10 4.94 ± 0.04 1.41 ± 0.14 3.84 ± 0.08 2.88 ± 0.14 6.00 ± 0.00 41 arXiv preprint Table 22: Density estimation performance via NLL (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one. DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM Dim 1 NLL0.30 ± 0.12 0.30 ± 0.12 0.41 ± 0.15 0.33 ± 0.13 0.39 ± 0.13 0.32 ± 0.13 Rank 2.50 ± 0.40 2.00 ± 0.47 5.12 ± 0.51 3.88 ± 0.37 4.38 ± 0.60 3.12 ± 0.62 Time0.29 ± 0.00 1.79 ± 0.45 0.17 ± 0.00 1.15 ± 0.00 0.15 ± 0.00 8.54 ± 0.10 Time Rank 3.00 ± 0.00 5.00 ± 0.00 1.75 ± 0.22 4.00 ± 0.00 1.25 ± 0.22 6.00 ± 0.00 Dim 2 NLL0.57 ± 0.49 0.57 ± 0.49 0.70 ± 0.61 0.69 ± 0.60 0.60 ± 0.52 0.58 ± 0.50 Rank3.12 ± 0.32 2.88 ± 0.54 4.12 ± 0.54 3.88 ± 0.32 3.62 ± 0.11 3.38 ± 0.11 Time0.30 ± 0.00 2.91 ± 0.43 0.19 ± 0.01 1.16 ± 0.00 0.44 ± 0.00 11.62 ± 0.65 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 5 NLL 5.47 ± 2.06 5.67 ± 1.91 5.63 ± 2.05 5.62 ± 2.04 5.84 ± 2.21 5.45 ± 2.06 Rank2.50 ± 0.25 2.50 ± 1.03 5.25 ± 0.22 4.50 ± 0.43 4.50 ± 0.56 1.75 ± 0.22 Time0.30 ± 0.00 7.46 ± 0.43 0.19 ± 0.01 1.16 ± 0.00 0.55 ± 0.00 12.97 ± 0.60 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 10 NLL11.79 ± 3.93 10.97 ± 3.53 10.96 ± 3.59 11.06 ± 3.62 11.60 ± 3.81 10.61 ± 3.51 Rank 4.88 ± 0.57 3.38 ± 0.74 2.62 ± 0.32 3.38 ± 0.21 4.88 ± 0.45 1.88 ± 0.51 Time0.30 ± 0.00 10.25 ± 1.59 0.21 ± 0.02 1.17 ± 0.00 0.80 ± 0.10 22.07 ± 0.37 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 NLL4.62 ± 4.00 4.12 ± 3.57 4.66 ± 4.04 4.60 ± 3.99 4.87 ± 4.22 4.02 ± 3.48 Rank3.62 ± 0.11 3.12 ± 0.32 3.88 ± 0.32 3.38 ± 0.11 4.12 ± 0.54 2.88 ± 0.54 Time0.31 ± 0.00 22.32 ± 2.75 0.20 ± 0.00 1.16 ± 0.00 0.94 ± 0.00 57.53 ± 1.62 Time Rank 2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 30 NLL12.93 ± 6.30 8.75 ± 4.33 11.00 ± 5.18 10.82 ± 5.17 14.92 ± 6.33 9.60 ± 4.81 Rank4.12 ± 0.45 1.88 ± 0.51 4.12 ± 0.27 3.38 ± 0.21 5.38 ± 0.54 2.12 ± 0.45 Time0.35 ± 0.02 31.73 ± 1.64 0.50 ± 0.17 1.55 ± 0.33 1.62 ± 0.29 65.66 ± 0.27 Time Rank1.50 ± 0.25 5.00 ± 0.00 1.50 ± 0.25 3.25 ± 0.22 3.75 ± 0.22 6.00 ± 0.00 Dim 40 NLL58.52 ± 17.55 53.97 ± 16.18 58.76 ± 17.71 58.41 ± 17.62 63.31 ± 18.97 55.86 ± 16.90 Rank3.88 ± 0.37 1.62 ± 0.54 4.38 ± 0.32 3.38 ± 0.21 5.38 ± 0.54 2.38 ± 0.32 Time0.35 ± 0.00 28.34 ± 6.70 0.12 ± 0.01 1.27 ± 0.03 1.96 ± 0.45 62.42 ± 4.14 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 3.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 Dim 50 NLL35.10 ± 13.01 29.33 ± 10.96 34.08 ± 12.27 33.94 ± 12.25 41.25 ± 14.19 29.62 ± 11.39 Rank 4.12 ± 0.45 1.88 ± 0.51 4.12 ± 0.27 3.38 ± 0.21 5.38 ± 0.54 2.12 ± 0.45 Time0.36 ± 0.00 44.74 ± 2.11 0.34 ± 0.01 1.23 ± 0.03 2.66 ± 0.50 87.71 ± 6.93 Time Rank1.75 ± 0.22 5.00 ± 0.00 1.25 ± 0.22 3.00 ± 0.00 4.00 ± 0.00 6.00 ± 0.00 Overall NLL10.54 ± 5.53 7.70 ± 5.31 10.11 ± 5.50 9.97 ± 5.49 12.94 ± 5.77 8.23 ± 5.37 Rank 3.62 ± 0.24 1.66 ± 0.21 4.66 ± 0.20 3.81 ± 0.14 5.22 ± 0.21 2.03 ± 0.17 Time0.32 ± 0.01 18.69 ± 2.75 0.24 ± 0.03 1.23 ± 0.05 1.14 ± 0.17 41.07 ± 5.18 Time Rank2.03 ± 0.08 5.00 ± 0.00 1.19 ± 0.07 3.66 ± 0.08 3.12 ± 0.15 6.00 ± 0.00 42 arXiv preprint Table 23: Score estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE 4.66 ± 2.99 1.00 ± 0.38 3.15 ± 1.11 3.33 ± 1.18 1.86 ± 0.45 18.61 ± 4.97 6.48 ± 1.42 1.84 ± 0.31 1.90 ± 0.57 Rank4.69 ± 1.02 2.44 ± 0.47 5.25 ± 0.23 5.88 ± 0.41 3.38 ± 0.35 9.00 ± 0.00 7.25 ± 0.58 3.62 ± 0.71 3.50 ± 0.79 Time0.29 ± 0.00 1.48 ± 0.25 0.03 ± 0.02 0.64 ± 0.06 0.17 ± 0.00 1.15 ± 0.00 2.05 ± 0.63 0.15 ± 0.00 8.42 ± 0.07 Time Rank4.00 ± 0.00 7.38 ± 0.25 1.25 ± 0.23 5.00 ± 0.00 2.75 ± 0.15 6.50 ± 0.18 7.12 ± 0.33 2.00 ± 0.18 9.00 ± 0.00 Dim 2 MAE19.34 ± 7.79 16.26 ± 7.20 28.45 ± 13.85 27.94 ± 13.06 25.51 ± 12.48 102.51 ± 53.97 53.68 ± 33.29 25.84 ± 12.87 28.64 ± 15.15 Rank 5.19 ± 0.98 2.94 ± 0.70 4.38 ± 0.35 5.12 ± 0.45 3.88 ± 0.67 9.00 ± 0.00 7.38 ± 0.35 3.62 ± 0.47 3.50 ± 0.98 Time0.29 ± 0.00 2.26 ± 0.32 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 7.24 ± 0.28 0.44 ± 0.00 10.48 ± 0.55 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 MAE9.26 ± 5.48 12.98 ± 10.68 4.11 ± 1.40 4.10 ± 1.47 5.44 ± 1.80 6.22 ± 2.24 7.02 ± 2.52 3.92 ± 1.24 2.48 ± 0.81 Rank5.31 ± 1.00 2.31 ± 0.91 4.25 ± 0.55 4.12 ± 0.67 6.75 ± 0.49 7.25 ± 0.34 7.00 ± 0.88 4.88 ± 0.37 3.12 ± 0.78 Time0.30 ± 0.00 5.31 ± 0.79 0.01 ± 0.00 0.57 ± 0.01 0.19 ± 0.01 1.16 ± 0.00 50.91 ± 0.57 0.54 ± 0.01 11.34 ± 0.67 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.75 ± 0.15 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.25 ± 0.15 8.00 ± 0.00 Dim 10 MAE86.74 ± 71.88 176.95 ± 163.39 72.70 ± 61.94 70.32 ± 59.68 79.87 ± 68.64 92.98 ± 77.09 168.90 ± 153.09 76.32 ± 65.34 73.86 ± 66.01 Rank7.62 ± 0.30 3.25 ± 1.20 3.88 ± 0.54 4.38 ± 0.79 4.88 ± 0.33 8.00 ± 0.35 5.38 ± 1.06 5.12 ± 0.48 2.50 ± 0.31 Time0.30 ± 0.00 7.24 ± 1.33 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.16 ± 0.00 199.61 ± 2.93 0.73 ± 0.06 19.48 ± 1.06 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE43.20 ± 15.08 27.48 ± 10.05 65.39 ± 30.02 65.92 ± 30.05 90.35 ± 35.16 94.20 ± 37.18 76.95 ± 32.14 74.59 ± 31.32 62.10 ± 30.06 Rank5.00 ± 0.61 2.12 ± 0.62 2.88 ± 0.21 4.00 ± 0.25 8.12 ± 0.21 8.38 ± 0.30 6.25 ± 0.68 6.25 ± 0.23 2.00 ± 0.31 Time0.31 ± 0.00 14.29 ± 3.16 0.01 ± 0.00 0.57 ± 0.00 0.21 ± 0.00 1.16 ± 0.00 867.92 ± 15.25 0.90 ± 0.02 47.75 ± 3.98 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE41.11 ± 10.51 18.63 ± 8.98 38.59 ± 10.74 38.53 ± 10.76 48.69 ± 10.53 46.91 ± 10.51 45.23 ± 10.07 45.18 ± 10.57 35.23 ± 11.43 Rank6.00 ± 0.94 1.62 ± 0.47 3.62 ± 0.17 3.38 ± 0.47 8.25 ± 0.23 7.12 ± 0.21 5.00 ± 0.98 6.62 ± 0.43 3.38 ± 0.50 Time0.33 ± 0.01 20.18 ± 4.16 0.20 ± 0.18 5.20 ± 4.33 0.39 ± 0.09 1.37 ± 0.18 2113.34 ± 36.52 1.35 ± 0.18 55.60 ± 4.75 Time Rank2.50 ± 0.25 6.88 ± 0.12 1.38 ± 0.35 4.38 ± 0.35 2.25 ± 0.15 5.50 ± 0.18 9.00 ± 0.00 5.12 ± 0.33 8.00 ± 0.00 Dim 40 MAE46.54 ± 36.97 37.13 ± 32.27 48.10 ± 39.88 48.30 ± 40.14 53.46 ± 41.38 52.16 ± 40.85 54.70 ± 41.88 52.88 ± 42.26 49.21 ± 41.09 Rank4.94 ± 0.93 2.06 ± 0.93 3.38 ± 0.30 2.88 ± 0.37 7.75 ± 0.42 6.25 ± 0.39 6.88 ± 0.78 7.00 ± 0.50 3.88 ± 0.37 Time0.34 ± 0.01 19.57 ± 4.56 0.04 ± 0.01 0.61 ± 0.01 2.70 ± 2.42 1.50 ± 0.20 4106.05 ± 69.01 1.61 ± 0.26 56.62 ± 3.42 Time Rank2.88 ± 0.12 6.88 ± 0.12 1.00 ± 0.00 3.88 ± 0.12 2.62 ± 0.58 5.38 ± 0.17 9.00 ± 0.00 5.38 ± 0.25 8.00 ± 0.00 Dim 50 MAE24.64 ± 10.40 8.36 ± 2.83 19.82 ± 8.02 19.45 ± 7.94 23.22 ± 7.71 22.90 ± 7.73 27.64 ± 7.29 22.87 ± 7.80 19.04 ± 8.01 Rank4.81 ± 1.17 2.31 ± 0.90 4.12 ± 0.37 3.00 ± 0.35 7.00 ± 0.31 6.00 ± 0.40 7.38 ± 0.92 7.00 ± 0.43 3.38 ± 0.30 Time0.34 ± 0.01 28.69 ± 5.75 0.05 ± 0.01 0.62 ± 0.01 0.37 ± 0.02 1.23 ± 0.00 7104.80 ± 90.39 2.21 ± 0.30 85.99 ± 5.12 Time Rank2.50 ± 0.18 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.18 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE 34.44 ± 10.96 37.35 ± 22.00 35.04 ± 10.68 34.74 ± 10.47 41.05 ± 11.81 54.56 ± 14.45 55.07 ± 21.60 37.93 ± 11.26 34.06 ± 11.17 Rank5.45 ± 0.34 2.38 ± 0.29 3.97 ± 0.15 4.09 ± 0.21 6.25 ± 0.27 7.62 ± 0.17 6.56 ± 0.31 5.52 ± 0.23 3.16 ± 0.22 Time0.31 ± 0.00 12.38 ± 1.62 0.04 ± 0.02 1.17 ± 0.57 0.55 ± 0.32 1.24 ± 0.04 1806.49 ± 301.72 0.99 ± 0.10 36.96 ± 3.52 Time Rank2.98 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.38 ± 0.07 2.27 ± 0.09 5.80 ± 0.07 8.64 ± 0.09 4.59 ± 0.15 8.25 ± 0.05 43 arXiv preprint Table 24: Score estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE9.05 ± 5.11 1.83 ± 0.46 4.65 ± 1.94 4.98 ± 2.03 2.46 ± 0.77 22.28 ± 9.12 6.89 ± 2.76 1.82 ± 0.57 1.69 ± 0.50 Rank7.50 ± 0.25 3.25 ± 0.74 5.25 ± 0.22 6.25 ± 0.22 3.00 ± 0.50 9.00 ± 0.00 6.50 ± 1.03 1.75 ± 0.41 2.50 ± 0.56 Time0.30 ± 0.00 1.16 ± 0.02 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 2.73 ± 1.14 0.14 ± 0.00 8.30 ± 0.05 Time Rank4.00 ± 0.00 7.00 ± 0.35 1.00 ± 0.00 5.00 ± 0.00 3.00 ± 0.00 6.50 ± 0.25 7.50 ± 0.43 2.00 ± 0.00 9.00 ± 0.00 Dim 2 MAE5.94 ± 2.95 2.52 ± 1.29 3.42 ± 1.68 3.99 ± 1.91 3.03 ± 1.40 18.47 ± 7.87 4.72 ± 1.95 2.54 ± 1.24 1.51 ± 0.76 Rank 6.12 ± 1.35 2.88 ± 0.91 4.00 ± 0.35 5.75 ± 0.54 4.75 ± 0.74 9.00 ± 0.00 7.00 ± 0.61 4.00 ± 0.79 1.50 ± 0.43 Time0.29 ± 0.00 1.60 ± 0.05 0.01 ± 0.00 0.56 ± 0.00 0.17 ± 0.01 1.16 ± 0.00 6.79 ± 0.15 0.43 ± 0.00 9.35 ± 0.39 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 MAE15.68 ± 9.90 1.93 ± 0.88 5.39 ± 2.31 5.73 ± 2.48 6.14 ± 2.60 8.55 ± 3.73 5.45 ± 2.19 4.80 ± 1.97 1.94 ± 0.88 Rank6.88 ± 1.56 1.62 ± 0.21 5.25 ± 0.65 5.75 ± 0.41 6.00 ± 0.79 8.00 ± 0.35 5.25 ± 1.24 4.50 ± 0.56 1.75 ± 0.41 Time0.30 ± 0.00 3.16 ± 0.14 0.01 ± 0.00 0.57 ± 0.02 0.18 ± 0.01 1.16 ± 0.00 52.12 ± 0.72 0.52 ± 0.01 9.71 ± 0.33 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 8.00 ± 0.00 Dim 10 MAE15.16 ± 4.84 1.83 ± 0.64 9.43 ± 3.49 9.50 ± 3.52 9.34 ± 3.48 16.36 ± 6.18 6.70 ± 2.33 9.17 ± 3.59 3.84 ± 1.39 Rank8.25 ± 0.22 1.00 ± 0.00 5.00 ± 0.61 6.00 ± 0.61 5.00 ± 0.35 8.75 ± 0.22 4.00 ± 0.87 5.00 ± 0.61 2.00 ± 0.00 Time0.30 ± 0.00 4.23 ± 0.07 0.01 ± 0.00 0.58 ± 0.02 0.19 ± 0.00 1.16 ± 0.00 200.53 ± 4.80 0.65 ± 0.01 16.90 ± 1.00 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE38.08 ± 26.81 5.88 ± 4.25 29.01 ± 19.96 29.52 ± 20.32 64.12 ± 46.12 72.60 ± 52.75 41.91 ± 31.05 40.31 ± 28.59 22.81 ± 16.94 Rank6.00 ± 0.35 1.00 ± 0.00 3.25 ± 0.22 4.25 ± 0.22 8.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.71 6.50 ± 0.25 2.00 ± 0.00 Time0.31 ± 0.00 6.26 ± 0.06 0.01 ± 0.00 0.57 ± 0.01 0.21 ± 0.01 1.17 ± 0.00 876.52 ± 18.37 0.85 ± 0.00 37.98 ± 3.61 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE41.99 ± 13.63 3.09 ± 0.67 19.69 ± 4.25 19.53 ± 4.22 36.82 ± 8.89 33.70 ± 8.06 33.90 ± 8.52 30.23 ± 6.67 12.84 ± 2.33 Rank7.25 ± 0.89 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 7.75 ± 0.22 7.25 ± 0.22 6.25 ± 0.82 6.50 ± 0.75 2.00 ± 0.00 Time0.31 ± 0.01 8.63 ± 0.19 0.01 ± 0.00 0.57 ± 0.01 0.28 ± 0.01 1.19 ± 0.02 2103.25 ± 57.34 1.06 ± 0.00 45.36 ± 6.13 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 40 MAE8.68 ± 4.35 1.22 ± 0.41 6.00 ± 2.11 5.88 ± 2.08 12.43 ± 3.93 11.20 ± 3.53 13.70 ± 4.28 9.71 ± 2.99 5.67 ± 1.85 Rank5.62 ± 1.38 1.12 ± 0.11 3.75 ± 0.41 3.25 ± 0.41 8.50 ± 0.25 7.00 ± 0.35 6.50 ± 1.35 5.75 ± 0.22 3.50 ± 0.56 Time0.32 ± 0.01 10.81 ± 0.13 0.05 ± 0.01 0.61 ± 0.01 5.28 ± 4.48 1.72 ± 0.38 4154.51 ± 88.32 1.26 ± 0.01 50.75 ± 3.53 Time Rank2.75 ± 0.22 6.75 ± 0.22 1.00 ± 0.00 3.75 ± 0.22 3.25 ± 1.08 5.75 ± 0.22 9.00 ± 0.00 4.75 ± 0.22 8.00 ± 0.00 Dim 50 MAE9.75 ± 8.02 1.30 ± 0.72 5.41 ± 2.31 5.12 ± 2.31 11.17 ± 2.83 10.71 ± 2.97 19.46 ± 3.54 10.20 ± 2.36 4.24 ± 1.68 Rank3.38 ± 1.62 1.38 ± 0.11 4.75 ± 0.22 3.75 ± 0.22 7.50 ± 0.25 6.75 ± 0.41 8.25 ± 0.65 6.50 ± 0.56 2.75 ± 0.22 Time0.31 ± 0.01 12.70 ± 0.19 0.02 ± 0.00 0.62 ± 0.02 0.40 ± 0.03 1.23 ± 0.00 7057.41 ± 142.96 1.75 ± 0.02 84.14 ± 7.44 Time Rank2.25 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.75 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE18.04 ± 4.82 2.45 ± 0.64 10.38 ± 3.05 10.53 ± 3.10 18.19 ± 6.92 24.23 ± 7.75 16.59 ± 4.75 13.60 ± 4.40 6.82 ± 2.50 Rank6.38 ± 0.46 1.66 ± 0.21 4.41 ± 0.19 4.75 ± 0.26 6.31 ± 0.36 8.09 ± 0.18 6.09 ± 0.40 5.06 ± 0.34 2.25 ± 0.17 Time0.30 ± 0.00 6.07 ± 0.71 0.01 ± 0.00 0.58 ± 0.01 0.86 ± 0.63 1.24 ± 0.06 1806.73 ± 425.35 0.83 ± 0.08 32.81 ± 4.66 Time Rank2.97 ± 0.09 6.97 ± 0.05 1.00 ± 0.00 4.34 ± 0.09 2.41 ± 0.16 5.91 ± 0.08 8.69 ± 0.11 4.47 ± 0.20 8.25 ± 0.08 Table 25: Score estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE0.26 ± 0.16 0.17 ± 0.09 1.64 ± 0.18 1.68 ± 0.19 1.26 ± 0.19 14.94 ± 2.95 6.08 ± 0.64 1.85 ± 0.27 2.10 ± 1.00 Rank1.88 ± 0.32 1.62 ± 0.11 5.25 ± 0.41 5.50 ± 0.75 3.75 ± 0.41 9.00 ± 0.00 8.00 ± 0.00 5.50 ± 0.25 4.50 ± 1.30 Time0.29 ± 0.00 1.79 ± 0.45 0.06 ± 0.04 0.73 ± 0.10 0.17 ± 0.00 1.15 ± 0.00 1.38 ± 0.22 0.15 ± 0.00 8.54 ± 0.10 Time Rank4.00 ± 0.00 7.75 ± 0.22 1.50 ± 0.43 5.00 ± 0.00 2.50 ± 0.25 6.50 ± 0.25 6.75 ± 0.41 2.00 ± 0.35 9.00 ± 0.00 Dim 2 MAE32.74 ± 12.00 29.99 ± 10.54 53.48 ± 21.24 51.88 ± 19.80 47.98 ± 19.20 186.55 ± 89.77 102.64 ± 56.84 49.14 ± 19.74 55.76 ± 23.44 Rank4.25 ± 1.24 3.00 ± 1.06 4.75 ± 0.54 4.50 ± 0.56 3.00 ± 0.94 9.00 ± 0.00 7.75 ± 0.22 3.25 ± 0.41 5.50 ± 1.30 Time0.30 ± 0.00 2.92 ± 0.43 0.01 ± 0.00 0.55 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 7.68 ± 0.43 0.45 ± 0.00 11.62 ± 0.65 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 MAE2.85 ± 1.32 24.04 ± 19.86 2.84 ± 1.29 2.48 ± 1.07 4.74 ± 2.44 3.88 ± 1.85 8.58 ± 4.39 3.03 ± 1.38 3.01 ± 1.30 Rank 3.75 ± 0.54 3.00 ± 1.73 3.25 ± 0.54 2.50 ± 0.56 7.50 ± 0.25 6.50 ± 0.25 8.75 ± 0.22 5.25 ± 0.41 4.50 ± 1.15 Time0.30 ± 0.00 7.46 ± 0.43 0.01 ± 0.00 0.56 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 49.70 ± 0.22 0.56 ± 0.00 12.98 ± 0.60 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.50 ± 0.25 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.50 ± 0.25 8.00 ± 0.00 Dim 10 MAE158.33 ± 134.48 352.06 ± 302.42 135.96 ± 115.48 131.14 ± 111.29 150.41 ± 127.84 169.60 ± 144.22 331.09 ± 283.89 143.47 ± 121.69 143.89 ± 122.37 Rank7.00 ± 0.35 5.50 ± 1.79 2.75 ± 0.41 2.75 ± 0.89 4.75 ± 0.54 7.25 ± 0.41 6.75 ± 1.67 5.25 ± 0.74 3.00 ± 0.50 Time0.30 ± 0.00 10.25 ± 1.59 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.01 1.17 ± 0.00 198.69 ± 3.29 0.81 ± 0.10 22.07 ± 0.37 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 MAE48.31 ± 13.31 49.08 ± 12.34 101.76 ± 50.45 102.33 ± 50.37 116.59 ± 49.73 115.81 ± 50.13 111.99 ± 50.53 108.87 ± 50.18 101.40 ± 50.55 Rank4.00 ± 0.94 3.25 ± 0.96 2.50 ± 0.25 3.75 ± 0.41 8.25 ± 0.41 7.75 ± 0.41 7.50 ± 0.75 6.00 ± 0.35 2.00 ± 0.61 Time0.31 ± 0.00 22.32 ± 2.75 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.00 1.16 ± 0.00 859.31 ± 23.56 0.95 ± 0.00 57.53 ± 1.63 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 MAE40.23 ± 15.98 34.16 ± 14.18 57.49 ± 16.27 57.52 ± 16.29 60.56 ± 17.15 60.12 ± 17.01 56.57 ± 16.39 60.13 ± 17.04 57.62 ± 16.33 Rank 4.75 ± 1.39 2.25 ± 0.82 3.25 ± 0.22 3.75 ± 0.89 8.75 ± 0.22 7.00 ± 0.35 3.75 ± 1.56 6.75 ± 0.41 4.75 ± 0.22 Time0.35 ± 0.02 31.72 ± 1.65 0.39 ± 0.33 9.83 ± 8.02 0.49 ± 0.16 1.55 ± 0.33 2123.44 ± 44.66 1.63 ± 0.29 65.84 ± 0.30 Time Rank2.25 ± 0.41 6.75 ± 0.22 1.75 ± 0.65 4.75 ± 0.65 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 5.25 ± 0.65 8.00 ± 0.00 Dim 40 MAE84.41 ± 68.78 73.05 ± 59.33 90.20 ± 73.97 90.72 ± 74.44 94.49 ± 77.42 93.12 ± 76.32 95.70 ± 78.48 96.05 ± 78.77 92.75 ± 76.17 Rank4.25 ± 1.14 3.00 ± 1.73 3.00 ± 0.35 2.50 ± 0.56 7.00 ± 0.61 5.50 ± 0.43 7.25 ± 0.74 8.25 ± 0.41 4.25 ± 0.41 Time0.35 ± 0.00 28.32 ± 6.71 0.03 ± 0.00 0.61 ± 0.01 0.12 ± 0.01 1.29 ± 0.03 4057.58 ± 100.37 1.97 ± 0.45 62.48 ± 4.15 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Dim 50 MAE39.54 ± 16.03 15.42 ± 2.56 34.23 ± 12.16 33.79 ± 12.00 35.26 ± 12.53 35.09 ± 12.47 35.82 ± 12.91 35.55 ± 12.54 33.85 ± 12.02 Rank6.25 ± 1.34 3.25 ± 1.67 3.50 ± 0.56 2.25 ± 0.41 6.50 ± 0.43 5.25 ± 0.41 6.50 ± 1.60 7.50 ± 0.56 4.00 ± 0.35 Time0.36 ± 0.00 44.68 ± 2.10 0.08 ± 0.01 0.62 ± 0.02 0.35 ± 0.01 1.23 ± 0.01 7152.20 ± 105.46 2.68 ± 0.50 87.83 ± 6.91 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall MAE50.84 ± 20.98 72.25 ± 43.13 59.70 ± 20.22 58.94 ± 19.80 63.91 ± 21.85 84.89 ± 26.79 93.56 ± 41.85 62.26 ± 21.23 61.30 ± 21.13 Rank4.52 ± 0.44 3.11 ± 0.51 3.53 ± 0.22 3.44 ± 0.30 6.19 ± 0.40 7.16 ± 0.26 7.03 ± 0.45 5.97 ± 0.30 4.06 ± 0.35 Time0.32 ± 0.01 18.68 ± 2.75 0.07 ± 0.05 1.75 ± 1.14 0.24 ± 0.03 1.23 ± 0.05 1806.25 ± 428.03 1.15 ± 0.17 41.11 ± 5.18 Time Rank3.00 ± 0.10 7.06 ± 0.06 1.16 ± 0.11 4.41 ± 0.12 2.12 ± 0.06 5.69 ± 0.10 8.59 ± 0.15 4.72 ± 0.23 8.25 ± 0.08 44 arXiv preprint Table 26: Score estimation performance via cosine similarity between score vector estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from ALL prior families. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Cosine0.54 ± 0.13 0.68 ± 0.13 0.46 ± 0.11 0.49 ± 0.12 0.59 ± 0.10 0.14 ± 0.02 0.39 ± 0.08 0.65 ± 0.10 0.65 ± 0.12 Rank4.94 ± 0.88 3.12 ± 0.67 6.25 ± 0.46 5.50 ± 0.77 4.38 ± 0.58 8.31 ± 0.46 6.12 ± 0.48 3.38 ± 0.64 3.00 ± 0.47 Time0.29 ± 0.00 1.48 ± 0.25 0.03 ± 0.02 0.64 ± 0.06 0.17 ± 0.00 1.15 ± 0.00 2.05 ± 0.63 0.15 ± 0.00 8.42 ± 0.07 Time Rank4.00 ± 0.00 7.38 ± 0.25 1.25 ± 0.23 5.00 ± 0.00 2.75 ± 0.15 6.50 ± 0.18 7.12 ± 0.33 2.00 ± 0.18 9.00 ± 0.00 Dim 2 Cosine0.59 ± 0.11 0.80 ± 0.08 0.66 ± 0.09 0.63 ± 0.09 0.75 ± 0.06 0.22 ± 0.02 0.54 ± 0.05 0.76 ± 0.06 0.77 ± 0.08 Rank5.69 ± 1.10 3.31 ± 0.82 5.38 ± 0.39 6.00 ± 0.53 3.75 ± 0.66 8.75 ± 0.15 5.88 ± 0.62 3.50 ± 0.40 2.75 ± 0.81 Time0.29 ± 0.00 2.26 ± 0.32 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 7.24 ± 0.28 0.44 ± 0.00 10.48 ± 0.55 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 Cosine0.56 ± 0.10 0.90 ± 0.03 0.72 ± 0.06 0.72 ± 0.06 0.63 ± 0.03 0.62 ± 0.04 0.57 ± 0.07 0.77 ± 0.04 0.84 ± 0.06 Rank5.31 ± 1.22 1.94 ± 0.44 4.75 ± 0.34 4.88 ± 0.69 6.75 ± 0.49 7.12 ± 0.28 6.88 ± 0.98 4.12 ± 0.51 3.25 ± 0.79 Time0.30 ± 0.00 5.31 ± 0.79 0.01 ± 0.00 0.57 ± 0.01 0.19 ± 0.01 1.16 ± 0.00 50.91 ± 0.57 0.54 ± 0.01 11.34 ± 0.67 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.75 ± 0.15 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.25 ± 0.15 8.00 ± 0.00 Dim 10 Cosine0.39 ± 0.04 0.76 ± 0.10 0.60 ± 0.05 0.60 ± 0.05 0.60 ± 0.03 0.51 ± 0.03 0.68 ± 0.06 0.63 ± 0.07 0.80 ± 0.06 Rank8.38 ± 0.39 2.62 ± 1.00 4.88 ± 0.69 4.62 ± 0.66 5.12 ± 0.45 6.88 ± 0.45 4.75 ± 0.88 5.62 ± 0.71 2.12 ± 0.28 Time0.30 ± 0.00 7.24 ± 1.33 0.01 ± 0.00 0.57 ± 0.01 0.20 ± 0.01 1.16 ± 0.00 199.61 ± 2.93 0.73 ± 0.06 19.48 ± 1.06 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Cosine0.54 ± 0.09 0.81 ± 0.07 0.70 ± 0.04 0.70 ± 0.04 0.53 ± 0.03 0.53 ± 0.03 0.65 ± 0.06 0.50 ± 0.08 0.77 ± 0.06 Rank6.62 ± 0.73 1.62 ± 0.39 3.75 ± 0.58 4.25 ± 0.39 7.00 ± 0.35 7.25 ± 0.55 4.88 ± 0.65 7.62 ± 0.56 2.00 ± 0.18 Time0.31 ± 0.00 14.29 ± 3.16 0.01 ± 0.00 0.57 ± 0.00 0.21 ± 0.00 1.16 ± 0.00 867.92 ± 15.25 0.90 ± 0.02 47.75 ± 3.98 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Cosine0.46 ± 0.11 0.68 ± 0.11 0.62 ± 0.08 0.62 ± 0.08 0.48 ± 0.05 0.48 ± 0.06 0.63 ± 0.04 0.39 ± 0.09 0.65 ± 0.09 Rank 6.88 ± 0.51 3.25 ± 0.84 3.50 ± 0.43 3.00 ± 0.25 7.25 ± 0.29 6.38 ± 0.39 3.88 ± 0.96 8.50 ± 0.25 2.38 ± 0.17 Time0.33 ± 0.01 20.18 ± 4.16 0.20 ± 0.18 5.20 ± 4.33 0.39 ± 0.09 1.37 ± 0.18 2113.34 ± 36.52 1.35 ± 0.18 55.60 ± 4.75 Time Rank2.50 ± 0.25 6.88 ± 0.12 1.38 ± 0.35 4.38 ± 0.35 2.25 ± 0.15 5.50 ± 0.18 9.00 ± 0.00 5.12 ± 0.33 8.00 ± 0.00 Dim 40 Cosine0.51 ± 0.11 0.72 ± 0.10 0.63 ± 0.08 0.64 ± 0.08 0.50 ± 0.05 0.51 ± 0.05 0.59 ± 0.06 0.43 ± 0.09 0.66 ± 0.09 Rank6.44 ± 0.82 2.31 ± 0.59 4.00 ± 0.35 3.38 ± 0.35 7.75 ± 0.39 6.62 ± 0.47 3.50 ± 1.02 7.62 ± 0.43 3.38 ± 0.53 Time0.34 ± 0.01 19.57 ± 4.56 0.04 ± 0.01 0.61 ± 0.01 2.70 ± 2.42 1.50 ± 0.20 4106.05 ± 69.01 1.61 ± 0.26 56.62 ± 3.42 Time Rank2.88 ± 0.12 6.88 ± 0.12 1.00 ± 0.00 3.88 ± 0.12 2.62 ± 0.58 5.38 ± 0.17 9.00 ± 0.00 5.38 ± 0.25 8.00 ± 0.00 Dim 50 Cosine0.55 ± 0.13 0.63 ± 0.13 0.63 ± 0.10 0.65 ± 0.10 0.47 ± 0.06 0.48 ± 0.06 0.44 ± 0.04 0.39 ± 0.08 0.66 ± 0.10 Rank5.06 ± 1.00 3.44 ± 0.98 4.50 ± 0.35 3.00 ± 0.40 6.88 ± 0.33 6.12 ± 0.48 5.50 ± 1.06 8.12 ± 0.37 2.38 ± 0.30 Time0.34 ± 0.01 28.69 ± 5.75 0.05 ± 0.01 0.62 ± 0.01 0.37 ± 0.02 1.23 ± 0.00 7104.80 ± 90.39 2.21 ± 0.30 85.99 ± 5.12 Time Rank2.50 ± 0.18 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.18 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Cosine0.52 ± 0.04 0.75 ± 0.04 0.63 ± 0.03 0.63 ± 0.03 0.57 ± 0.02 0.44 ± 0.02 0.56 ± 0.02 0.56 ± 0.03 0.73 ± 0.03 Rank6.16 ± 0.34 2.70 ± 0.28 4.62 ± 0.20 4.33 ± 0.23 6.11 ± 0.24 7.18 ± 0.18 5.17 ± 0.33 6.06 ± 0.31 2.66 ± 0.19 Time0.31 ± 0.00 12.38 ± 1.62 0.04 ± 0.02 1.17 ± 0.57 0.55 ± 0.32 1.24 ± 0.04 1806.49 ± 301.72 0.99 ± 0.10 36.96 ± 3.52 Time Rank2.98 ± 0.07 7.02 ± 0.04 1.08 ± 0.06 4.38 ± 0.07 2.27 ± 0.09 5.80 ± 0.07 8.64 ± 0.09 4.59 ± 0.15 8.25 ± 0.05 45 arXiv preprint Table 27: Score estimation performance via cosine similarity between score vector estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the GMM prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Cosine0.35 ± 0.08 0.62 ± 0.15 0.45 ± 0.10 0.45 ± 0.10 0.70 ± 0.10 0.15 ± 0.02 0.51 ± 0.09 0.76 ± 0.06 0.73 ± 0.08 Rank6.88 ± 0.87 3.25 ± 1.14 6.25 ± 0.41 6.75 ± 0.65 3.00 ± 0.00 8.88 ± 0.11 5.50 ± 0.56 2.00 ± 0.61 2.50 ± 0.43 Time0.30 ± 0.00 1.16 ± 0.02 0.01 ± 0.00 0.56 ± 0.00 0.18 ± 0.01 1.16 ± 0.00 2.73 ± 1.14 0.14 ± 0.00 8.30 ± 0.05 Time Rank4.00 ± 0.00 7.00 ± 0.35 1.00 ± 0.00 5.00 ± 0.00 3.00 ± 0.00 6.50 ± 0.25 7.50 ± 0.43 2.00 ± 0.00 9.00 ± 0.00 Dim 2 Cosine0.50 ± 0.16 0.87 ± 0.09 0.86 ± 0.05 0.81 ± 0.07 0.87 ± 0.02 0.26 ± 0.02 0.61 ± 0.07 0.90 ± 0.01 0.97 ± 0.01 Rank6.88 ± 1.56 3.38 ± 1.08 4.50 ± 0.25 5.25 ± 0.65 4.50 ± 1.03 8.50 ± 0.25 6.50 ± 0.75 4.00 ± 0.61 1.50 ± 0.43 Time0.29 ± 0.00 1.60 ± 0.05 0.01 ± 0.00 0.56 ± 0.00 0.17 ± 0.01 1.16 ± 0.00 6.79 ± 0.15 0.43 ± 0.00 9.35 ± 0.39 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 Cosine0.42 ± 0.17 0.91 ± 0.04 0.73 ± 0.09 0.71 ± 0.10 0.69 ± 0.03 0.62 ± 0.07 0.72 ± 0.08 0.83 ± 0.04 0.97 ± 0.01 Rank7.12 ± 1.62 2.12 ± 0.57 5.25 ± 0.41 6.25 ± 0.41 6.00 ± 0.79 7.75 ± 0.22 4.75 ± 1.24 4.00 ± 0.79 1.75 ± 0.41 Time0.30 ± 0.00 3.16 ± 0.14 0.01 ± 0.00 0.57 ± 0.02 0.18 ± 0.01 1.16 ± 0.00 52.12 ± 0.72 0.52 ± 0.01 9.71 ± 0.33 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.00 ± 0.00 8.00 ± 0.00 Dim 10 Cosine0.41 ± 0.05 0.98 ± 0.01 0.60 ± 0.08 0.60 ± 0.08 0.62 ± 0.04 0.48 ± 0.04 0.83 ± 0.02 0.78 ± 0.04 0.96 ± 0.00 Rank8.25 ± 0.65 1.00 ± 0.00 6.25 ± 0.89 6.25 ± 0.41 6.00 ± 0.50 7.75 ± 0.54 3.75 ± 0.65 3.75 ± 0.22 2.00 ± 0.00 Time0.30 ± 0.00 4.23 ± 0.07 0.01 ± 0.00 0.58 ± 0.02 0.19 ± 0.00 1.16 ± 0.00 200.53 ± 4.80 0.65 ± 0.01 16.90 ± 1.00 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Cosine0.78 ± 0.03 0.99 ± 0.00 0.78 ± 0.01 0.78 ± 0.01 0.59 ± 0.02 0.58 ± 0.02 0.79 ± 0.03 0.70 ± 0.04 0.93 ± 0.01 Rank4.75 ± 0.54 1.00 ± 0.00 4.75 ± 0.74 4.75 ± 0.41 7.75 ± 0.22 8.75 ± 0.22 4.50 ± 0.75 6.75 ± 0.89 2.00 ± 0.00 Time0.31 ± 0.00 6.26 ± 0.06 0.01 ± 0.00 0.57 ± 0.01 0.21 ± 0.01 1.17 ± 0.00 876.52 ± 18.37 0.85 ± 0.00 37.98 ± 3.61 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Cosine0.75 ± 0.04 0.99 ± 0.00 0.83 ± 0.02 0.84 ± 0.02 0.62 ± 0.01 0.63 ± 0.02 0.73 ± 0.04 0.63 ± 0.02 0.90 ± 0.02 Rank 5.50 ± 0.25 1.00 ± 0.00 4.25 ± 0.22 3.25 ± 0.22 7.75 ± 0.41 7.25 ± 0.41 5.75 ± 1.08 8.25 ± 0.41 2.00 ± 0.00 Time0.31 ± 0.01 8.63 ± 0.19 0.01 ± 0.00 0.57 ± 0.01 0.28 ± 0.01 1.19 ± 0.02 2103.25 ± 57.34 1.06 ± 0.00 45.36 ± 6.13 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 40 Cosine0.81 ± 0.07 0.99 ± 0.00 0.84 ± 0.04 0.85 ± 0.04 0.62 ± 0.03 0.64 ± 0.03 0.71 ± 0.08 0.69 ± 0.03 0.91 ± 0.02 Rank4.62 ± 0.99 1.12 ± 0.11 4.25 ± 0.41 3.75 ± 0.22 8.75 ± 0.22 7.75 ± 0.22 5.75 ± 1.24 6.50 ± 0.25 2.50 ± 0.25 Time0.32 ± 0.01 10.81 ± 0.13 0.05 ± 0.01 0.61 ± 0.01 5.28 ± 4.48 1.72 ± 0.38 4154.51 ± 88.32 1.26 ± 0.01 50.75 ± 3.53 Time Rank2.75 ± 0.22 6.75 ± 0.22 1.00 ± 0.00 3.75 ± 0.22 3.25 ± 1.08 5.75 ± 0.22 9.00 ± 0.00 4.75 ± 0.22 8.00 ± 0.00 Dim 50 Cosine0.92 ± 0.07 0.98 ± 0.01 0.89 ± 0.03 0.90 ± 0.03 0.62 ± 0.04 0.63 ± 0.04 0.46 ± 0.07 0.63 ± 0.03 0.92 ± 0.02 Rank2.62 ± 0.97 1.38 ± 0.11 4.75 ± 0.22 3.75 ± 0.22 7.50 ± 0.25 7.00 ± 0.61 8.00 ± 0.87 7.25 ± 0.41 2.75 ± 0.22 Time0.31 ± 0.01 12.70 ± 0.19 0.02 ± 0.00 0.62 ± 0.02 0.40 ± 0.03 1.23 ± 0.00 7057.41 ± 142.96 1.75 ± 0.02 84.14 ± 7.44 Time Rank2.25 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.75 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Cosine0.62 ± 0.05 0.92 ± 0.03 0.75 ± 0.03 0.74 ± 0.03 0.67 ± 0.02 0.50 ± 0.03 0.67 ± 0.03 0.74 ± 0.02 0.91 ± 0.02 Rank5.83 ± 0.47 1.78 ± 0.27 5.03 ± 0.22 5.00 ± 0.27 6.41 ± 0.37 7.95 ± 0.17 5.56 ± 0.39 5.31 ± 0.41 2.12 ± 0.12 Time0.30 ± 0.00 6.07 ± 0.71 0.01 ± 0.00 0.58 ± 0.01 0.86 ± 0.63 1.24 ± 0.06 1806.73 ± 425.35 0.83 ± 0.08 32.81 ± 4.66 Time Rank2.97 ± 0.09 6.97 ± 0.05 1.00 ± 0.00 4.34 ± 0.09 2.41 ± 0.16 5.91 ± 0.08 8.69 ± 0.11 4.47 ± 0.20 8.25 ± 0.08 46 arXiv preprint Table 28: Score estimation performance via cosine similarity between score vector estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) is across methods over test datasets from the Copula prior. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Cosine0.73 ± 0.21 0.73 ± 0.21 0.47 ± 0.20 0.54 ± 0.21 0.48 ± 0.15 0.14 ± 0.04 0.27 ± 0.08 0.53 ± 0.17 0.57 ± 0.21 Rank 3.00 ± 0.68 3.00 ± 0.68 6.25 ± 0.82 4.25 ± 1.08 5.75 ± 0.65 7.75 ± 0.82 6.75 ± 0.65 4.75 ± 0.54 3.50 ± 0.75 Time0.29 ± 0.00 1.79 ± 0.45 0.06 ± 0.04 0.73 ± 0.10 0.17 ± 0.00 1.15 ± 0.00 1.38 ± 0.22 0.15 ± 0.00 8.54 ± 0.10 Time Rank4.00 ± 0.00 7.75 ± 0.22 1.50 ± 0.43 5.00 ± 0.00 2.50 ± 0.25 6.50 ± 0.25 6.75 ± 0.41 2.00 ± 0.35 9.00 ± 0.00 Dim 2 Cosine0.69 ± 0.13 0.73 ± 0.12 0.46 ± 0.10 0.45 ± 0.11 0.62 ± 0.07 0.18 ± 0.03 0.47 ± 0.06 0.62 ± 0.07 0.57 ± 0.09 Rank4.50 ± 1.30 3.25 ± 1.24 6.25 ± 0.41 6.75 ± 0.65 3.00 ± 0.61 9.00 ± 0.00 5.25 ± 0.89 3.00 ± 0.35 4.00 ± 1.27 Time0.30 ± 0.00 2.92 ± 0.43 0.01 ± 0.00 0.55 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 7.68 ± 0.43 0.45 ± 0.00 11.62 ± 0.65 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 Dim 5 Cosine0.70 ± 0.07 0.89 ± 0.04 0.72 ± 0.06 0.74 ± 0.06 0.58 ± 0.04 0.63 ± 0.05 0.42 ± 0.03 0.70 ± 0.05 0.72 ± 0.07 Rank3.50 ± 1.30 1.75 ± 0.65 4.25 ± 0.41 3.50 ± 0.90 7.50 ± 0.25 6.50 ± 0.25 9.00 ± 0.00 4.25 ± 0.65 4.75 ± 1.08 Time0.30 ± 0.00 7.46 ± 0.43 0.01 ± 0.00 0.56 ± 0.00 0.19 ± 0.01 1.16 ± 0.00 49.70 ± 0.22 0.56 ± 0.00 12.98 ± 0.60 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.50 ± 0.25 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 4.50 ± 0.25 8.00 ± 0.00 Dim 10 Cosine0.38 ± 0.08 0.54 ± 0.12 0.60 ± 0.06 0.60 ± 0.06 0.58 ± 0.05 0.54 ± 0.04 0.53 ± 0.07 0.48 ± 0.07 0.64 ± 0.05 Rank8.50 ± 0.43 4.25 ± 1.63 3.50 ± 0.43 3.00 ± 0.50 4.25 ± 0.41 6.00 ± 0.35 5.75 ± 1.47 7.50 ± 0.43 2.25 ± 0.54 Time0.30 ± 0.00 10.25 ± 1.59 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.01 1.17 ± 0.00 198.69 ± 3.29 0.81 ± 0.10 22.07 ± 0.37 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 20 Cosine0.31 ± 0.08 0.62 ± 0.04 0.61 ± 0.04 0.61 ± 0.04 0.47 ± 0.03 0.48 ± 0.03 0.51 ± 0.05 0.31 ± 0.07 0.61 ± 0.04 Rank8.50 ± 0.25 2.25 ± 0.65 2.75 ± 0.54 3.75 ± 0.54 6.25 ± 0.41 5.75 ± 0.22 5.25 ± 1.02 8.50 ± 0.25 2.00 ± 0.35 Time0.31 ± 0.00 22.32 ± 2.75 0.01 ± 0.00 0.56 ± 0.00 0.21 ± 0.00 1.16 ± 0.00 859.31 ± 23.56 0.95 ± 0.00 57.53 ± 1.63 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 30 Cosine0.16 ± 0.04 0.36 ± 0.04 0.41 ± 0.04 0.40 ± 0.04 0.33 ± 0.03 0.33 ± 0.03 0.52 ± 0.03 0.15 ± 0.03 0.40 ± 0.04 Rank 8.25 ± 0.22 5.50 ± 0.56 2.75 ± 0.65 2.75 ± 0.41 6.75 ± 0.22 5.50 ± 0.25 2.00 ± 0.87 8.75 ± 0.22 2.75 ± 0.22 Time0.35 ± 0.02 31.72 ± 1.65 0.39 ± 0.33 9.83 ± 8.02 0.49 ± 0.16 1.55 ± 0.33 2123.44 ± 44.66 1.63 ± 0.29 65.84 ± 0.30 Time Rank2.25 ± 0.41 6.75 ± 0.22 1.75 ± 0.65 4.75 ± 0.65 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 5.25 ± 0.65 8.00 ± 0.00 Dim 40 Cosine0.22 ± 0.02 0.44 ± 0.02 0.42 ± 0.03 0.43 ± 0.03 0.37 ± 0.02 0.38 ± 0.02 0.47 ± 0.02 0.17 ± 0.02 0.41 ± 0.03 Rank8.25 ± 0.22 3.50 ± 0.83 3.75 ± 0.54 3.00 ± 0.61 6.75 ± 0.22 5.50 ± 0.43 1.25 ± 0.22 8.75 ± 0.22 4.25 ± 0.82 Time0.35 ± 0.00 28.32 ± 6.71 0.03 ± 0.00 0.61 ± 0.01 0.12 ± 0.01 1.29 ± 0.03 4057.58 ± 100.37 1.97 ± 0.45 62.48 ± 4.15 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Dim 50 Cosine0.19 ± 0.03 0.28 ± 0.06 0.37 ± 0.06 0.40 ± 0.06 0.33 ± 0.04 0.33 ± 0.04 0.41 ± 0.01 0.16 ± 0.03 0.40 ± 0.05 Rank7.50 ± 0.25 5.50 ± 1.30 4.25 ± 0.65 2.25 ± 0.54 6.25 ± 0.41 5.25 ± 0.41 3.00 ± 0.79 9.00 ± 0.00 2.00 ± 0.50 Time0.36 ± 0.00 44.68 ± 2.10 0.08 ± 0.01 0.62 ± 0.02 0.35 ± 0.01 1.23 ± 0.01 7152.20 ± 105.46 2.68 ± 0.50 87.83 ± 6.91 Time Rank2.75 ± 0.22 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.25 ± 0.22 5.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 8.00 ± 0.00 Overall Cosine0.42 ± 0.05 0.57 ± 0.05 0.51 ± 0.04 0.52 ± 0.04 0.47 ± 0.03 0.38 ± 0.03 0.45 ± 0.02 0.39 ± 0.05 0.54 ± 0.04 Rank6.50 ± 0.47 3.62 ± 0.43 4.22 ± 0.31 3.66 ± 0.34 5.81 ± 0.29 6.41 ± 0.26 4.78 ± 0.52 6.81 ± 0.42 3.19 ± 0.33 Time0.32 ± 0.01 18.68 ± 2.75 0.07 ± 0.05 1.75 ± 1.14 0.24 ± 0.03 1.23 ± 0.05 1806.25 ± 428.03 1.15 ± 0.17 41.11 ± 5.18 Time Rank3.00 ± 0.10 7.06 ± 0.06 1.16 ± 0.11 4.41 ± 0.12 2.12 ± 0.06 5.69 ± 0.10 8.59 ± 0.15 4.72 ± 0.23 8.25 ± 0.08 47 arXiv preprint Table 29: Distribution match performance via MMD (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MMD (×10 3 ) 41.79 ± 15.47 6.39 ± 2.24 184.48 ± 66.92 213.03 ± 81.66 19.72 ± 3.36 6.30 ± 3.40 7.59 ± 3.51 5.03 ± 2.52 5.93 ± 3.09 Rank5.90 ± 0.62 3.85 ± 0.68 8.60 ± 0.15 8.30 ± 0.20 5.80 ± 0.28 3.10 ± 0.54 3.70 ± 0.55 2.70 ± 0.21 3.05 ± 0.34 Time0.04 ± 0.01 1.22 ± 0.06 93.20 ± 0.14 93.87 ± 0.15 0.08 ± 0.00 1.13 ± 0.00 2.12 ± 0.23 0.22 ± 0.02 8.34 ± 0.10 Time Rank1.10 ± 0.09 4.70 ± 0.14 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.14 4.30 ± 0.14 6.00 ± 0.00 2.90 ± 0.09 7.00 ± 0.00 Dim 2 MMD (×10 3 )65.60 ± 12.68 18.19 ± 8.04 74.69 ± 16.12 88.65 ± 23.06 165.03 ± 39.65 8.53 ± 2.75 4.86 ± 1.48 8.73 ± 5.71 2.94 ± 0.84 Rank6.42 ± 0.63 4.42 ± 0.53 6.92 ± 0.30 7.17 ± 0.42 8.25 ± 0.29 3.46 ± 0.51 3.04 ± 0.30 2.38 ± 0.50 2.96 ± 0.34 Time0.04 ± 0.01 1.84 ± 0.15 93.22 ± 0.17 93.93 ± 0.19 0.08 ± 0.00 1.14 ± 0.01 3.42 ± 0.07 0.47 ± 0.05 10.29 ± 0.39 Time Rank1.08 ± 0.08 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 1.92 ± 0.08 4.00 ± 0.00 6.00 ± 0.00 3.00 ± 0.00 7.00 ± 0.00 Dim 5 MMD (×10 3 )79.24 ± 17.49 57.42 ± 16.56 131.40 ± 41.85 171.99 ± 57.91 45.14 ± 11.68 20.41 ± 8.38 33.76 ± 14.23 68.47 ± 41.09 17.68 ± 8.32 Rank 6.82 ± 0.36 4.82 ± 0.74 7.27 ± 0.29 8.27 ± 0.29 5.64 ± 0.23 3.23 ± 0.37 3.41 ± 0.79 3.41 ± 0.65 2.14 ± 0.20 Time0.03 ± 0.00 2.99 ± 0.55 92.96 ± 0.12 93.51 ± 0.12 0.08 ± 0.00 1.13 ± 0.00 15.79 ± 0.29 0.70 ± 0.02 12.84 ± 0.42 Time Rank1.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 6.91 ± 0.09 3.00 ± 0.00 6.09 ± 0.09 Dim 10 MMD (×10 3 )66.04 ± 11.61 24.92 ± 4.79 249.01 ± 44.13 338.56 ± 37.08 44.69 ± 7.08 8.43 ± 2.71 31.43 ± 16.60 12.46 ± 2.91 7.30 ± 2.56 Rank 6.45 ± 0.37 4.55 ± 0.20 8.00 ± 0.18 8.82 ± 0.12 5.91 ± 0.24 2.36 ± 0.32 3.45 ± 0.70 3.45 ± 0.45 2.00 ± 0.26 Time0.06 ± 0.02 5.01 ± 0.72 93.72 ± 0.42 94.28 ± 0.41 0.08 ± 0.00 1.13 ± 0.00 37.96 ± 0.53 0.85 ± 0.05 12.76 ± 0.74 Time Rank1.09 ± 0.09 5.00 ± 0.00 8.00 ± 0.00 9.00 ± 0.00 1.91 ± 0.09 4.00 ± 0.00 7.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 MMD (×10 3 ) 63.03 ± 12.47 29.76 ± 5.59 246.50 ± 29.94 259.38 ± 30.70 38.54 ± 7.24 19.67 ± 6.79 32.82 ± 18.00 61.64 ± 23.32 11.03 ± 4.66 Rank6.08 ± 0.25 4.17 ± 0.26 8.00 ± 0.20 8.75 ± 0.12 5.50 ± 0.34 3.17 ± 0.47 2.50 ± 0.71 4.67 ± 0.59 2.17 ± 0.20 Time0.09 ± 0.03 11.37 ± 1.31 94.23 ± 0.48 109.14 ± 13.87 0.15 ± 0.07 1.28 ± 0.14 84.39 ± 2.80 1.27 ± 0.11 15.58 ± 1.19 Time Rank1.08 ± 0.08 5.08 ± 0.08 7.92 ± 0.08 9.00 ± 0.00 2.00 ± 0.12 3.17 ± 0.11 7.08 ± 0.08 3.75 ± 0.17 5.92 ± 0.08 Dim 30 MMD (×10 3 )74.69 ± 22.46 51.91 ± 19.00 267.89 ± 84.72 239.97 ± 74.38 84.98 ± 27.34 53.01 ± 18.81 71.14 ± 46.89 65.43 ± 29.71 23.92 ± 11.49 Rank5.92 ± 0.42 3.75 ± 0.41 7.92 ± 0.32 7.83 ± 0.35 6.67 ± 0.32 3.83 ± 0.68 3.75 ± 0.69 3.75 ± 0.44 1.58 ± 0.22 Time0.06 ± 0.00 14.86 ± 1.93 93.13 ± 0.14 94.85 ± 1.18 0.09 ± 0.00 1.13 ± 0.00 123.86 ± 4.13 1.79 ± 0.17 22.27 ± 1.77 Time Rank1.17 ± 0.11 5.25 ± 0.12 7.08 ± 0.08 8.08 ± 0.08 1.83 ± 0.11 3.25 ± 0.12 8.83 ± 0.16 3.75 ± 0.12 5.75 ± 0.12 Dim 40 MMD (×10 3 )77.44 ± 11.51 48.79 ± 9.45 464.02 ± 57.58 436.19 ± 56.06 89.70 ± 8.28 31.29 ± 6.35 68.97 ± 21.68 157.63 ± 53.04 14.49 ± 2.32 Rank 5.42 ± 0.43 3.42 ± 0.28 8.33 ± 0.40 8.00 ± 0.24 6.08 ± 0.32 3.42 ± 0.58 3.33 ± 0.58 5.58 ± 0.42 1.42 ± 0.18 Time0.07 ± 0.00 20.63 ± 2.67 93.35 ± 0.15 94.23 ± 0.21 0.14 ± 0.05 1.14 ± 0.01 166.06 ± 2.48 2.47 ± 0.22 26.38 ± 2.01 Time Rank1.25 ± 0.12 5.25 ± 0.12 7.00 ± 0.00 8.00 ± 0.00 1.75 ± 0.12 3.08 ± 0.08 9.00 ± 0.00 3.92 ± 0.08 5.75 ± 0.12 Dim 50 MMD (×10 3 )90.50 ± 20.42 52.28 ± 15.20 386.55 ± 63.32 393.06 ± 63.13 141.46 ± 30.51 93.12 ± 34.95 102.26 ± 54.66 159.97 ± 49.99 39.32 ± 17.83 Rank4.67 ± 0.36 3.25 ± 0.41 7.83 ± 0.37 8.67 ± 0.14 6.42 ± 0.14 3.92 ± 0.56 3.17 ± 0.72 5.33 ± 0.40 1.75 ± 0.21 Time0.07 ± 0.01 19.36 ± 3.71 93.26 ± 0.13 93.88 ± 0.13 0.10 ± 0.01 1.15 ± 0.00 210.54 ± 6.65 2.73 ± 0.35 31.86 ± 5.27 Time Rank1.25 ± 0.12 5.17 ± 0.11 7.00 ± 0.00 8.00 ± 0.00 1.75 ± 0.12 3.17 ± 0.11 9.00 ± 0.00 3.83 ± 0.11 5.83 ± 0.11 Overall MMD (×10 3 )70.34 ± 5.86 36.75 ± 4.56 253.32 ± 23.04 269.06 ± 22.76 80.67 ± 9.27 30.95 ± 6.18 45.15 ± 10.94 69.36 ± 13.29 15.59 ± 3.27 Rank5.95 ± 0.17 4.02 ± 0.17 7.85 ± 0.12 8.22 ± 0.11 6.30 ± 0.13 3.33 ± 0.19 3.28 ± 0.23 3.95 ± 0.21 2.11 ± 0.10 Time0.06 ± 0.01 9.97 ± 1.03 93.39 ± 0.10 96.05 ± 1.89 0.10 ± 0.01 1.16 ± 0.02 83.39 ± 7.86 1.35 ± 0.11 17.84 ± 1.14 Time Rank1.13 ± 0.04 5.07 ± 0.04 7.61 ± 0.05 8.62 ± 0.05 1.89 ± 0.04 3.60 ± 0.06 7.52 ± 0.13 3.41 ± 0.06 6.15 ± 0.06 Table 30: Rejection rate of two-sample tests for distribution match w.r.t. MMD over test datasets over real-world test datasets.. Tests under generated data that successfully match the input data dis- tribution fail to reject the null (i.e., p > 0.05) that the two samples come from the same distribution. DimensionPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1100.0%60.0%100.0%100.0%100.0%50.0%60.0%50.0%40.0% Dim 2100.0%66.7%100.0%100.0%100.0%58.3%66.7%41.7%50.0% Dim 5100.0%90.9%100.0%100.0%100.0%81.8%72.7%63.6%63.6% Dim 10100.0%100.0%100.0%100.0%100.0%72.7%90.9%81.8%72.7% Dim 20100.0%100.0%100.0%100.0%100.0%100.0%83.3%100.0%100.0% Dim 30100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0%91.7% Dim 40100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0% Dim 50100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0%100.0% Overall100.0%90.2%100.0%100.0%100.0%83.7%84.8%80.4%78.3% 48 arXiv preprint Table 31: Density estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE1035 ± 791 1035 ± 791 1037 ± 791 1037 ± 791 1037 ± 791 991 ± 789 1037 ± 790 1045 ± 789 933 ± 695 Rank 6.10 ± 0.68 4.30 ± 0.75 5.70 ± 0.60 6.30 ± 0.45 5.80 ± 0.63 3.70 ± 0.72 6.20 ± 0.88 4.50 ± 0.93 2.40 ± 0.62 Time0.04 ± 0.01 1.22 ± 0.06 0.02 ± 0.01 0.68 ± 0.06 0.08 ± 0.00 1.13 ± 0.00 1.07 ± 0.02 0.27 ± 0.02 8.34 ± 0.10 Time Rank2.00 ± 0.14 7.40 ± 0.32 1.30 ± 0.28 5.30 ± 0.28 2.80 ± 0.13 7.00 ± 0.20 6.30 ± 0.14 3.90 ± 0.09 9.00 ± 0.00 Dim 2 MAE5 ± 1 3 ± 1 3 ± 1 3 ± 1 2 ± 1 3 ± 1 3 ± 1 3 ± 1 2 ± 1 Rank6.78 ± 0.86 5.00 ± 0.83 4.67 ± 0.47 5.33 ± 0.42 5.56 ± 0.69 4.56 ± 0.82 7.22 ± 0.68 4.44 ± 0.86 1.44 ± 0.28 Time0.04 ± 0.02 2.02 ± 0.16 0.04 ± 0.02 0.56 ± 0.00 0.08 ± 0.00 1.15 ± 0.01 4.14 ± 0.08 0.57 ± 0.06 10.38 ± 0.47 Time Rank1.89 ± 0.19 7.00 ± 0.00 1.33 ± 0.22 4.22 ± 0.14 2.78 ± 0.14 6.00 ± 0.00 8.00 ± 0.00 4.78 ± 0.14 9.00 ± 0.00 Dim 5 MAE48 ± 26 48 ± 27 49 ± 27 49 ± 27 49 ± 27 48 ± 28 46 ± 27 46 ± 27 41 ± 25 Rank6.14 ± 1.17 6.43 ± 0.85 6.57 ± 0.28 6.57 ± 0.49 6.14 ± 0.65 4.71 ± 1.10 2.71 ± 0.52 3.86 ± 0.62 1.86 ± 0.37 Time0.03 ± 0.00 2.84 ± 0.82 0.03 ± 0.02 0.59 ± 0.02 0.08 ± 0.00 1.14 ± 0.00 52.27 ± 0.96 0.79 ± 0.04 12.14 ± 0.39 Time Rank1.86 ± 0.13 7.00 ± 0.00 1.29 ± 0.26 4.14 ± 0.13 2.86 ± 0.13 6.00 ± 0.00 9.00 ± 0.00 4.86 ± 0.13 8.00 ± 0.00 Dim 10 MAE111 ± 73 119 ± 83 106 ± 69 107 ± 69 106 ± 71 107 ± 72 103 ± 71 103 ± 69 97 ± 70 Rank7.33 ± 0.98 5.67 ± 1.44 6.33 ± 1.36 6.67 ± 1.19 6.00 ± 0.00 5.67 ± 1.09 3.00 ± 0.82 3.33 ± 0.54 1.00 ± 0.00 Time0.03 ± 0.00 3.98 ± 1.09 0.13 ± 0.10 0.96 ± 0.25 0.08 ± 0.00 1.14 ± 0.00 104.85 ± 0.40 0.88 ± 0.19 11.86 ± 0.86 Time Rank1.67 ± 0.27 7.00 ± 0.00 1.67 ± 0.54 5.00 ± 0.47 2.67 ± 0.27 5.67 ± 0.27 9.00 ± 0.00 4.33 ± 0.27 8.00 ± 0.00 Dim 20 MAE24 ± 6 20 ± 4 17 ± 3 18 ± 2 17 ± 3 17 ± 3 16 ± 4 25 ± 3 4 ± 2 Rank5.00 ± 1.63 4.00 ± 1.63 5.00 ± 0.47 6.33 ± 0.27 5.33 ± 1.19 6.00 ± 1.25 5.67 ± 1.52 6.67 ± 1.19 1.00 ± 0.00 Time0.22 ± 0.09 12.17 ± 2.31 0.13 ± 0.10 58.00 ± 46.89 0.38 ± 0.24 1.71 ± 0.47 210.63 ± 4.12 1.21 ± 0.30 18.03 ± 1.42 Time Rank2.33 ± 0.27 7.00 ± 0.00 1.00 ± 0.00 5.33 ± 1.09 3.00 ± 0.47 5.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.82 7.33 ± 0.54 Dim 30 MAE24 ± 6 17 ± 2 10 ± 2 8 ± 3 11 ± 6 14 ± 8 16 ± 6 17 ± 1 7 ± 5 Rank8.50 ± 0.35 6.50 ± 0.35 3.00 ± 1.41 3.00 ± 0.71 3.00 ± 0.71 5.00 ± 1.41 7.00 ± 0.71 7.00 ± 1.41 2.00 ± 0.71 Time0.07 ± 0.01 14.73 ± 4.38 0.01 ± 0.00 0.56 ± 0.01 0.09 ± 0.00 1.14 ± 0.01 304.96 ± 12.44 1.67 ± 0.58 23.86 ± 2.84 Time Rank2.50 ± 0.35 7.50 ± 0.35 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.35 5.50 ± 0.35 9.00 ± 0.00 5.50 ± 0.35 7.50 ± 0.35 Dim 40 MAE107 ± 0 89 ± 0 72 ± 0 73 ± 0 73 ± 0 73 ± 0 169 ± 0 81 ± 0 0 ± 0 Rank 8.00 ± 0.00 7.00 ± 0.00 2.00 ± 0.00 3.00 ± 0.00 5.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 1.00 ± 0.00 Time0.09 ± 0.00 3.67 ± 0.00 0.01 ± 0.00 0.56 ± 0.00 0.08 ± 0.00 1.13 ± 0.00 379.81 ± 0.00 0.71 ± 0.00 15.48 ± 0.00 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Overall MAE322 ± 239 322 ± 239 320 ± 239 320 ± 239 320 ± 239 307 ± 237 322 ± 239 323 ± 238 284 ± 210 Rank 6.49 ± 0.43 5.20 ± 0.43 5.34 ± 0.32 5.86 ± 0.28 5.60 ± 0.33 4.57 ± 0.42 5.57 ± 0.48 4.63 ± 0.43 1.74 ± 0.23 Time0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 67.14 ± 17.11 0.67 ± 0.08 11.85 ± 0.72 Time Rank 2.00 ± 0.09 7.14 ± 0.10 1.29 ± 0.13 4.66 ± 0.17 2.77 ± 0.08 6.14 ± 0.12 7.97 ± 0.20 4.57 ± 0.12 8.46 ± 0.12 Table 32: Density estimation performance via Spearman correlation between the overall ranking by estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Corr0.24 ± 0.09 0.41 ± 0.10 0.31 ± 0.09 0.37 ± 0.11 0.51 ± 0.11 0.74 ± 0.09 0.43 ± 0.11 0.60 ± 0.13 0.49 ± 0.12 Rank6.65 ± 0.67 5.20 ± 0.64 6.75 ± 0.54 6.15 ± 0.67 4.60 ± 0.38 2.80 ± 0.56 5.30 ± 0.72 3.05 ± 0.70 4.50 ± 0.74 Time0.04 ± 0.01 1.22 ± 0.06 0.02 ± 0.01 0.68 ± 0.06 0.08 ± 0.00 1.13 ± 0.00 1.07 ± 0.02 0.27 ± 0.02 8.34 ± 0.10 Time Rank2.00 ± 0.14 7.40 ± 0.32 1.30 ± 0.28 5.30 ± 0.28 2.80 ± 0.13 7.00 ± 0.20 6.30 ± 0.14 3.90 ± 0.09 9.00 ± 0.00 Dim 2 Corr0.40 ± 0.08 0.59 ± 0.08 0.54 ± 0.06 0.53 ± 0.06 0.54 ± 0.06 0.57 ± 0.06 0.54 ± 0.09 0.64 ± 0.05 0.76 ± 0.07 Rank7.50 ± 0.43 5.22 ± 0.73 5.67 ± 0.65 5.44 ± 0.74 4.78 ± 0.80 5.67 ± 0.65 5.72 ± 0.93 3.33 ± 0.74 1.67 ± 0.27 Time0.04 ± 0.02 2.02 ± 0.16 0.04 ± 0.02 0.56 ± 0.00 0.08 ± 0.00 1.15 ± 0.01 4.14 ± 0.08 0.57 ± 0.06 10.38 ± 0.47 Time Rank1.89 ± 0.19 7.00 ± 0.00 1.33 ± 0.22 4.22 ± 0.14 2.78 ± 0.14 6.00 ± 0.00 8.00 ± 0.00 4.78 ± 0.14 9.00 ± 0.00 Dim 5 Corr0.25 ± 0.07 0.40 ± 0.08 0.44 ± 0.06 0.44 ± 0.06 0.50 ± 0.10 0.43 ± 0.11 0.57 ± 0.10 0.52 ± 0.09 0.67 ± 0.05 Rank 7.79 ± 1.05 6.00 ± 0.64 6.14 ± 0.43 5.71 ± 0.75 5.07 ± 0.59 5.43 ± 0.83 4.00 ± 0.99 3.43 ± 0.57 1.43 ± 0.28 Time0.03 ± 0.00 2.84 ± 0.82 0.03 ± 0.02 0.59 ± 0.02 0.08 ± 0.00 1.14 ± 0.00 52.27 ± 0.96 0.79 ± 0.04 12.14 ± 0.39 Time Rank1.86 ± 0.13 7.00 ± 0.00 1.29 ± 0.26 4.14 ± 0.13 2.86 ± 0.13 6.00 ± 0.00 9.00 ± 0.00 4.86 ± 0.13 8.00 ± 0.00 Dim 10 Corr0.35 ± 0.15 0.23 ± 0.19 0.30 ± 0.17 0.33 ± 0.17 0.32 ± 0.18 0.29 ± 0.20 0.40 ± 0.16 0.26 ± 0.21 0.58 ± 0.10 Rank5.83 ± 1.67 7.50 ± 0.41 5.83 ± 0.76 5.17 ± 0.89 5.50 ± 0.41 4.67 ± 1.36 3.33 ± 1.09 6.17 ± 0.95 1.00 ± 0.00 Time0.03 ± 0.00 3.98 ± 1.09 0.13 ± 0.10 0.96 ± 0.25 0.08 ± 0.00 1.14 ± 0.00 104.85 ± 0.40 0.88 ± 0.19 11.86 ± 0.86 Time Rank1.67 ± 0.27 7.00 ± 0.00 1.67 ± 0.54 5.00 ± 0.47 2.67 ± 0.27 5.67 ± 0.27 9.00 ± 0.00 4.33 ± 0.27 8.00 ± 0.00 Dim 20 Corr0.30 ± 0.08 0.17 ± 0.09 0.66 ± 0.11 0.70 ± 0.10 0.67 ± 0.11 0.66 ± 0.10 0.63 ± 0.08 0.52 ± 0.07 0.95 ± 0.03 Rank8.33 ± 0.27 8.67 ± 0.27 5.00 ± 0.82 3.00 ± 0.82 3.67 ± 0.27 4.67 ± 0.27 5.33 ± 0.98 5.33 ± 1.36 1.00 ± 0.00 Time0.22 ± 0.09 12.17 ± 2.31 0.13 ± 0.10 58.00 ± 46.89 0.38 ± 0.24 1.71 ± 0.47 210.63 ± 4.12 1.21 ± 0.30 18.03 ± 1.42 Time Rank2.33 ± 0.27 7.00 ± 0.00 1.00 ± 0.00 5.33 ± 1.09 3.00 ± 0.47 5.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.82 7.33 ± 0.54 Dim 30 Corr0.34 ± 0.09 0.67 ± 0.08 0.65 ± 0.10 0.74 ± 0.15 0.53 ± 0.17 0.49 ± 0.20 0.29 ± 0.13 0.45 ± 0.14 0.76 ± 0.17 Rank8.00 ± 0.71 2.50 ± 1.06 3.50 ± 0.35 2.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.71 8.50 ± 0.35 6.50 ± 0.35 2.00 ± 0.71 Time0.07 ± 0.01 14.73 ± 4.38 0.01 ± 0.00 0.56 ± 0.01 0.09 ± 0.00 1.14 ± 0.01 304.96 ± 12.44 1.67 ± 0.58 23.86 ± 2.84 Time Rank2.50 ± 0.35 7.50 ± 0.35 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.35 5.50 ± 0.35 9.00 ± 0.00 5.50 ± 0.35 7.50 ± 0.35 Dim 40 Corr0.11 ± 0.00 0.01 ± 0.00 0.56 ± 0.00 0.62 ± 0.00 0.52 ± 0.00 0.50 ± 0.00 0.43 ± 0.00 0.48 ± 0.00 1.00 ± 0.00 Rank8.00 ± 0.00 9.00 ± 0.00 3.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 5.00 ± 0.00 7.00 ± 0.00 6.00 ± 0.00 1.00 ± 0.00 Time0.09 ± 0.00 3.67 ± 0.00 0.01 ± 0.00 0.56 ± 0.00 0.08 ± 0.00 1.13 ± 0.00 379.81 ± 0.00 0.71 ± 0.00 15.48 ± 0.00 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Overall Corr0.30 ± 0.04 0.42 ± 0.05 0.45 ± 0.04 0.48 ± 0.05 0.51 ± 0.05 0.57 ± 0.05 0.49 ± 0.05 0.55 ± 0.05 0.67 ± 0.05 Rank7.29 ± 0.36 5.81 ± 0.39 5.83 ± 0.31 5.17 ± 0.39 4.74 ± 0.27 4.69 ± 0.38 5.21 ± 0.44 3.94 ± 0.39 2.31 ± 0.33 Time0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 67.14 ± 17.11 0.67 ± 0.08 11.85 ± 0.72 Time Rank 2.00 ± 0.09 7.14 ± 0.10 1.29 ± 0.13 4.66 ± 0.17 2.77 ± 0.08 6.14 ± 0.12 7.97 ± 0.20 4.57 ± 0.12 8.46 ± 0.12 49 arXiv preprint Table 33: Density estimation performance via NLL (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. We do not report NLL’s for TabPFN and Disco variants, as their predicted densities do not integrate to one. DimensionMetricPRODiGIPRODiGI-FTNNKDENNKDE-FTKDEGMM Dim 1 NLL1.26 ± 0.12 1.29 ± 0.20 1.14 ± 0.09 49.30 ± 28.98 11.37 ± 8.79 105.05 ± 96.95 Rank 3.50 ± 0.55 2.40 ± 0.40 3.80 ± 0.56 4.40 ± 0.43 2.80 ± 0.56 4.10 ± 0.41 Time0.04 ± 0.01 1.22 ± 0.06 0.08 ± 0.00 1.13 ± 0.00 0.27 ± 0.02 8.34 ± 0.10 Time Rank 1.10 ± 0.09 4.70 ± 0.14 1.90 ± 0.09 4.30 ± 0.14 3.00 ± 0.00 6.00 ± 0.00 Dim 2 NLL4.02 ± 0.81 2.38 ± 0.49 1.90 ± 0.36 1.53 ± 0.39 1.51 ± 0.34 1.48 ± 0.39 Rank5.00 ± 0.47 4.11 ± 0.53 3.78 ± 0.47 2.89 ± 0.37 2.67 ± 0.35 2.56 ± 0.45 Time0.04 ± 0.02 2.02 ± 0.16 0.08 ± 0.00 1.15 ± 0.01 0.57 ± 0.06 10.38 ± 0.47 Time Rank 1.11 ± 0.10 5.00 ± 0.00 1.89 ± 0.10 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 5 NLL12.39 ± 2.69 5.75 ± 1.20 3.28 ± 0.83 4.70 ± 1.94 5.26 ± 1.51 9.06 ± 3.06 Rank 4.71 ± 0.56 3.57 ± 0.57 2.43 ± 0.46 2.71 ± 0.60 3.79 ± 0.28 3.79 ± 0.78 Time0.03 ± 0.00 2.84 ± 0.82 0.08 ± 0.00 1.14 ± 0.00 0.79 ± 0.04 12.14 ± 0.39 Time Rank 1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 10 NLL15.23 ± 0.62 10.29 ± 0.51 3.66 ± 2.99 7.67 ± 3.34 3.69 ± 2.64 7.18 ± 5.86 Rank5.67 ± 0.27 4.33 ± 0.27 2.33 ± 0.36 2.83 ± 0.95 2.50 ± 0.24 3.33 ± 1.11 Time0.03 ± 0.00 3.98 ± 1.09 0.08 ± 0.00 1.14 ± 0.00 0.88 ± 0.19 11.86 ± 0.86 Time Rank1.00 ± 0.00 5.00 ± 0.00 2.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Dim 20 NLL26.51 ± 2.31 21.65 ± 2.84 2.39 ± 1.95 2.62 ± 2.14 20.00 ± 4.43 12.08 ± 5.63 Rank 6.00 ± 0.00 4.67 ± 0.27 1.50 ± 0.24 1.83 ± 0.14 4.33 ± 0.27 2.67 ± 0.27 Time0.22 ± 0.09 12.17 ± 2.31 0.38 ± 0.24 1.71 ± 0.47 1.21 ± 0.30 18.03 ± 1.42 Time Rank1.33 ± 0.27 5.33 ± 0.27 2.00 ± 0.47 3.33 ± 0.27 3.33 ± 0.54 5.67 ± 0.27 Dim 30 NLL42.09 ± 2.61 35.53 ± 4.97 12.64 ± 8.94 12.80 ± 9.05 32.38 ± 5.98 16.11 ± 6.25 Rank5.00 ± 0.71 5.50 ± 0.35 1.75 ± 0.18 2.25 ± 0.53 4.50 ± 0.35 2.00 ± 0.71 Time0.07 ± 0.01 14.73 ± 4.38 0.09 ± 0.00 1.14 ± 0.01 1.67 ± 0.58 23.86 ± 2.84 Time Rank1.50 ± 0.35 5.50 ± 0.35 1.50 ± 0.35 3.50 ± 0.35 3.50 ± 0.35 5.50 ± 0.35 Dim 40 NLL84.90 ± 0.00 58.19 ± 0.00 11.12 ± 0.00 11.92 ± 0.00 44.66 ± 0.00 83.45 ± 0.00 Rank6.00 ± 0.00 4.00 ± 0.00 1.00 ± 0.00 2.00 ± 0.00 3.00 ± 0.00 5.00 ± 0.00 Time0.09 ± 0.00 3.67 ± 0.00 0.08 ± 0.00 1.13 ± 0.00 0.71 ± 0.00 15.48 ± 0.00 Time Rank2.00 ± 0.00 5.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 6.00 ± 0.00 Overall NLL 12.28 ± 2.86 8.48 ± 2.16 1.21 ± 1.23 15.24 ± 9.15 9.51 ± 3.13 36.17 ± 28.78 Rank4.69 ± 0.27 3.71 ± 0.26 3.00 ± 0.27 2.91 ± 0.29 3.34 ± 0.24 3.34 ± 0.29 Time0.06 ± 0.01 3.77 ± 0.77 0.11 ± 0.02 1.19 ± 0.05 0.67 ± 0.08 11.85 ± 0.72 Time Rank1.14 ± 0.06 4.97 ± 0.06 1.89 ± 0.07 4.00 ± 0.07 3.06 ± 0.06 5.94 ± 0.04 50 arXiv preprint Table 34: Score estimation performance via MAE (↓) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 MAE14362 ± 9872 14361 ± 9871 14361 ± 9872 14361 ± 9872 14361 ± 9872 13251 ± 9740 14358 ± 9869 14344 ± 9859 11859 ± 7643 Rank6.30 ± 0.45 4.90 ± 0.50 5.70 ± 0.69 5.70 ± 0.47 4.40 ± 0.86 7.00 ± 0.99 4.70 ± 0.57 3.70 ± 0.85 2.60 ± 0.79 Time0.04 ± 0.01 1.22 ± 0.06 0.02 ± 0.01 0.68 ± 0.06 0.08 ± 0.00 1.13 ± 0.00 1.27 ± 0.06 0.31 ± 0.02 8.35 ± 0.10 Time Rank 2.00 ± 0.14 7.00 ± 0.32 1.30 ± 0.28 5.20 ± 0.19 2.80 ± 0.13 6.40 ± 0.25 7.40 ± 0.15 3.90 ± 0.09 9.00 ± 0.00 Dim 2 MAE162 ± 98 170 ± 98 511 ± 291 320 ± 140 240 ± 105 5297 ± 3254 164 ± 99 162 ± 99 46 ± 16 Rank6.11 ± 0.71 4.78 ± 0.83 5.56 ± 0.52 5.33 ± 0.52 4.22 ± 0.49 8.22 ± 0.73 4.78 ± 0.91 3.89 ± 0.67 2.11 ± 0.62 Time0.04 ± 0.02 2.03 ± 0.16 0.04 ± 0.02 0.56 ± 0.00 0.08 ± 0.00 1.15 ± 0.01 6.84 ± 0.12 0.59 ± 0.07 10.38 ± 0.47 Time Rank1.89 ± 0.19 7.00 ± 0.00 1.33 ± 0.22 4.22 ± 0.14 2.78 ± 0.14 6.00 ± 0.00 8.00 ± 0.00 4.78 ± 0.14 9.00 ± 0.00 Dim 5 MAE 711 ± 382 710 ± 382 852 ± 382 860 ± 378 850 ± 386 5183 ± 2840 707 ± 375 711 ± 382 689 ± 373 Rank5.86 ± 1.06 4.29 ± 0.77 6.00 ± 0.45 6.29 ± 0.48 4.86 ± 0.68 7.43 ± 0.83 3.29 ± 0.87 4.71 ± 1.02 2.29 ± 0.75 Time0.03 ± 0.00 2.84 ± 0.82 0.03 ± 0.02 0.59 ± 0.02 0.08 ± 0.00 1.14 ± 0.00 52.66 ± 2.28 0.81 ± 0.04 12.30 ± 0.38 Time Rank1.86 ± 0.13 7.00 ± 0.00 1.29 ± 0.26 4.00 ± 0.00 2.86 ± 0.13 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 10 MAE1095 ± 825 1286 ± 980 2321 ± 969 1975 ± 832 3804 ± 1720 47594 ± 29449 989 ± 729 939 ± 698 875 ± 669 Rank4.67 ± 1.44 4.00 ± 0.94 5.67 ± 1.09 5.00 ± 0.47 7.00 ± 0.82 9.00 ± 0.00 5.00 ± 0.94 3.67 ± 0.98 1.00 ± 0.00 Time0.03 ± 0.00 3.98 ± 1.09 0.13 ± 0.10 0.96 ± 0.25 0.08 ± 0.00 1.14 ± 0.00 202.13 ± 10.89 0.91 ± 0.20 11.87 ± 0.86 Time Rank1.67 ± 0.27 7.00 ± 0.00 1.67 ± 0.54 5.00 ± 0.47 2.67 ± 0.27 5.33 ± 0.27 9.00 ± 0.00 4.67 ± 0.54 8.00 ± 0.00 Dim 20 MAE27 ± 6 26 ± 5 200 ± 91 123 ± 40 708 ± 354 1299 ± 587 32 ± 9 32 ± 9 14 ± 8 Rank5.00 ± 0.94 3.00 ± 0.47 5.33 ± 1.36 5.00 ± 0.82 7.33 ± 0.54 8.67 ± 0.27 5.67 ± 1.36 4.00 ± 0.82 1.00 ± 0.00 Time0.22 ± 0.09 12.17 ± 2.31 0.13 ± 0.10 58.00 ± 46.89 0.36 ± 0.23 1.71 ± 0.47 841.28 ± 91.00 1.22 ± 0.30 18.20 ± 1.51 Time Rank2.33 ± 0.27 7.00 ± 0.00 1.00 ± 0.00 5.33 ± 1.09 3.00 ± 0.47 5.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.82 7.33 ± 0.54 Dim 30 MAE24 ± 6 27 ± 4 88 ± 5 57 ± 1 83 ± 6 123 ± 19 24 ± 9 25 ± 5 9 ± 6 Rank2.50 ± 0.35 4.50 ± 0.35 8.00 ± 0.00 6.00 ± 0.00 7.00 ± 0.00 9.00 ± 0.00 3.50 ± 1.06 3.50 ± 0.35 1.00 ± 0.00 Time0.07 ± 0.01 14.72 ± 4.38 0.01 ± 0.00 0.56 ± 0.01 0.09 ± 0.00 1.14 ± 0.01 1775.14 ± 271.48 1.67 ± 0.58 23.94 ± 2.87 Time Rank 2.50 ± 0.35 7.50 ± 0.35 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.35 5.50 ± 0.35 9.00 ± 0.00 5.50 ± 0.35 7.50 ± 0.35 Dim 40 MAE56 ± 0 56 ± 0 112 ± 0 81 ± 0 89 ± 0 102 ± 0 34 ± 0 56 ± 0 0 ± 0 Rank5.00 ± 0.00 4.00 ± 0.00 9.00 ± 0.00 6.00 ± 0.00 7.00 ± 0.00 8.00 ± 0.00 2.00 ± 0.00 3.00 ± 0.00 1.00 ± 0.00 Time0.09 ± 0.00 3.67 ± 0.00 0.01 ± 0.00 0.56 ± 0.00 0.08 ± 0.00 1.13 ± 0.00 2410.62 ± 0.00 0.71 ± 0.00 15.50 ± 0.00 Time Rank3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Overall MAE4387 ± 3018 4405 ± 3017 4629 ± 3011 4543 ± 3013 4729 ± 3012 10385 ± 4405 4376 ± 3017 4368 ± 3014 3615 ± 2357 Rank 5.66 ± 0.38 4.46 ± 0.33 5.91 ± 0.32 5.63 ± 0.24 5.14 ± 0.38 7.86 ± 0.40 4.40 ± 0.39 3.94 ± 0.39 2.00 ± 0.33 Time0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 272.40 ± 99.60 0.70 ± 0.08 11.90 ± 0.73 Time Rank2.00 ± 0.09 7.03 ± 0.09 1.29 ± 0.13 4.60 ± 0.15 2.77 ± 0.08 5.94 ± 0.11 8.29 ± 0.13 4.63 ± 0.13 8.46 ± 0.12 Table 35: Score estimation performance via cosine similarity between score vector estimates and ground-truth (↑) and rank, as well as runtime in seconds (↓) and runtime rank for varying data dimensions. Average rank per metric (lower is better) across methods over real-world test datasets. DimensionMetricPRODiGIPRODiGI-FTDiscoFormerDiscoFormer-FTNNKDENNKDE-FTTabPFNKDEGMM Dim 1 Cosine0.04 ± 0.03 0.17 ± 0.07 0.13 ± 0.05 0.04 ± 0.02 0.24 ± 0.07 0.37 ± 0.12 0.29 ± 0.10 0.49 ± 0.10 0.42 ± 0.11 Rank7.05 ± 0.50 6.05 ± 0.46 6.35 ± 0.36 7.35 ± 0.32 4.70 ± 0.65 4.05 ± 0.85 4.90 ± 0.67 2.00 ± 0.60 2.55 ± 0.36 Time0.04 ± 0.01 1.22 ± 0.06 0.02 ± 0.01 0.68 ± 0.06 0.08 ± 0.00 1.13 ± 0.00 1.27 ± 0.06 0.31 ± 0.02 8.35 ± 0.10 Time Rank 2.00 ± 0.14 7.00 ± 0.32 1.30 ± 0.28 5.20 ± 0.19 2.80 ± 0.13 6.40 ± 0.25 7.40 ± 0.15 3.90 ± 0.09 9.00 ± 0.00 Dim 2 Cosine0.10 ± 0.04 0.28 ± 0.05 0.26 ± 0.08 0.25 ± 0.07 0.27 ± 0.07 0.37 ± 0.08 0.23 ± 0.06 0.35 ± 0.07 0.61 ± 0.10 Rank7.28 ± 0.65 4.67 ± 0.72 5.33 ± 0.72 5.50 ± 0.66 5.06 ± 0.86 4.67 ± 0.96 6.72 ± 0.64 3.67 ± 0.54 2.11 ± 0.55 Time0.04 ± 0.02 2.03 ± 0.16 0.04 ± 0.02 0.56 ± 0.00 0.08 ± 0.00 1.15 ± 0.01 6.84 ± 0.12 0.59 ± 0.07 10.38 ± 0.47 Time Rank1.89 ± 0.19 7.00 ± 0.00 1.33 ± 0.22 4.22 ± 0.14 2.78 ± 0.14 6.00 ± 0.00 8.00 ± 0.00 4.78 ± 0.14 9.00 ± 0.00 Dim 5 Cosine0.06 ± 0.03 0.18 ± 0.05 0.18 ± 0.08 0.18 ± 0.08 0.20 ± 0.06 0.20 ± 0.05 0.30 ± 0.05 0.23 ± 0.06 0.31 ± 0.06 Rank8.21 ± 0.26 5.71 ± 0.87 5.21 ± 0.77 5.79 ± 1.03 5.14 ± 0.31 4.71 ± 0.77 3.14 ± 1.06 4.14 ± 0.74 2.93 ± 0.85 Time0.03 ± 0.00 2.84 ± 0.82 0.03 ± 0.02 0.59 ± 0.02 0.08 ± 0.00 1.14 ± 0.00 52.66 ± 2.28 0.81 ± 0.04 12.30 ± 0.38 Time Rank 1.86 ± 0.13 7.00 ± 0.00 1.29 ± 0.26 4.00 ± 0.00 2.86 ± 0.13 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Dim 10 Cosine0.15 ± 0.04 0.09 ± 0.05 0.30 ± 0.17 0.27 ± 0.15 0.34 ± 0.21 0.28 ± 0.16 0.15 ± 0.11 0.12 ± 0.04 0.69 ± 0.08 Rank4.00 ± 1.25 6.33 ± 1.19 5.33 ± 1.19 5.67 ± 0.27 5.00 ± 1.89 6.00 ± 0.82 5.00 ± 1.70 6.33 ± 1.36 1.33 ± 0.27 Time0.03 ± 0.00 3.98 ± 1.09 0.13 ± 0.10 0.96 ± 0.25 0.08 ± 0.00 1.14 ± 0.00 202.13 ± 10.89 0.91 ± 0.20 11.87 ± 0.86 Time Rank1.67 ± 0.27 7.00 ± 0.00 1.67 ± 0.54 5.00 ± 0.47 2.67 ± 0.27 5.33 ± 0.27 9.00 ± 0.00 4.67 ± 0.54 8.00 ± 0.00 Dim 20 Cosine 0.04 ± 0.01 0.07 ± 0.03 0.12 ± 0.08 0.14 ± 0.09 0.10 ± 0.07 0.09 ± 0.07 0.26 ± 0.03 0.14 ± 0.01 0.88 ± 0.07 Rank6.33 ± 1.19 7.00 ± 1.25 5.00 ± 0.94 5.00 ± 1.41 6.00 ± 0.94 7.00 ± 0.94 3.33 ± 1.09 4.33 ± 1.09 1.00 ± 0.00 Time0.22 ± 0.09 12.17 ± 2.31 0.13 ± 0.10 58.00 ± 46.89 0.36 ± 0.23 1.71 ± 0.47 841.28 ± 91.00 1.22 ± 0.30 18.20 ± 1.51 Time Rank2.33 ± 0.27 7.00 ± 0.00 1.00 ± 0.00 5.33 ± 1.09 3.00 ± 0.47 5.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.82 7.33 ± 0.54 Dim 30 Cosine0.02 ± 0.01 0.03 ± 0.02 0.16 ± 0.10 0.22 ± 0.14 0.15 ± 0.09 0.15 ± 0.09 0.03 ± 0.00 0.04 ± 0.00 0.88 ± 0.08 Rank6.50 ± 1.06 5.50 ± 2.47 6.00 ± 2.12 5.00 ± 2.12 5.50 ± 1.06 5.50 ± 0.35 5.50 ± 1.06 4.50 ± 1.06 1.00 ± 0.00 Time0.07 ± 0.01 14.72 ± 4.38 0.01 ± 0.00 0.56 ± 0.01 0.09 ± 0.00 1.14 ± 0.01 1775.14 ± 271.48 1.67 ± 0.58 23.94 ± 2.87 Time Rank2.50 ± 0.35 7.50 ± 0.35 1.00 ± 0.00 4.00 ± 0.00 2.50 ± 0.35 5.50 ± 0.35 9.00 ± 0.00 5.50 ± 0.35 7.50 ± 0.35 Dim 40 Cosine0.01 ± 0.00 0.03 ± 0.00 0.10 ± 0.00 0.11 ± 0.00 0.14 ± 0.00 0.15 ± 0.00 0.00 ± 0.00 0.02 ± 0.00 1.00 ± 0.00 Rank 8.00 ± 0.00 6.00 ± 0.00 5.00 ± 0.00 4.00 ± 0.00 3.00 ± 0.00 2.00 ± 0.00 9.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 Time0.09 ± 0.00 3.67 ± 0.00 0.01 ± 0.00 0.56 ± 0.00 0.08 ± 0.00 1.13 ± 0.00 2410.62 ± 0.00 0.71 ± 0.00 15.50 ± 0.00 Time Rank 3.00 ± 0.00 7.00 ± 0.00 1.00 ± 0.00 4.00 ± 0.00 2.00 ± 0.00 6.00 ± 0.00 9.00 ± 0.00 5.00 ± 0.00 8.00 ± 0.00 Overall Cosine0.07 ± 0.02 0.17 ± 0.03 0.19 ± 0.04 0.16 ± 0.03 0.23 ± 0.04 0.28 ± 0.05 0.24 ± 0.04 0.30 ± 0.04 0.55 ± 0.05 Rank7.01 ± 0.33 5.70 ± 0.37 5.60 ± 0.33 5.99 ± 0.37 5.01 ± 0.36 4.79 ± 0.43 5.04 ± 0.45 3.71 ± 0.39 2.14 ± 0.27 Time0.06 ± 0.01 3.77 ± 0.77 0.05 ± 0.02 5.56 ± 4.85 0.11 ± 0.02 1.19 ± 0.05 272.40 ± 99.60 0.70 ± 0.08 11.90 ± 0.73 Time Rank2.00 ± 0.09 7.03 ± 0.09 1.29 ± 0.13 4.60 ± 0.15 2.77 ± 0.08 5.94 ± 0.11 8.29 ± 0.13 4.63 ± 0.13 8.46 ± 0.12 51