Paper deep dive

CDFM: Towards a General-Purpose Causal Discovery Foundation Model

Jie Qiao, Ruichu Cai, Zijian Li, Weilin Chen, Pengfei Hua, Boyan Xu, Zhengming Chen, Zhifeng Hao, Peng Cui

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 85

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%

Last extracted: 7/18/2026, 2:41:44 PM

Summary

The paper introduces the Causal Discovery Foundation Model (CDFM), a unified framework for zero-shot structural inference that overcomes the limitations of fragmented, dataset-specific causal discovery algorithms. By treating unknown causal mechanisms as latent variables within a variational framework, CDFM pretrains on diverse synthetic structural causal models to internalize complex statistical asymmetries, enabling robust generalization across heterogeneous real-world data distributions.

Entities (19)

CDFM · model · 98%Jie Qiao · person · 95%Ruichu Cai · person · 95%Structural Causal Model · concept · 92%Guangdong University of Technology · organization · 90%Shantou University · organization · 90%Tsinghua University · organization · 90%ANM · algorithm · 88%LiNGAM · algorithm · 88%FCI algorithm · algorithm · 85%NOTEARS · algorithm · 85%PC Algorithm · algorithm · 85%LimiX · model · 82%TabICL · model · 82%TabPFN · model · 82%AVICI · algorithm · 80%CausalPFN · model · 80%CSIvA · algorithm · 80%Do-PFN · model · 80%

Relation Signals (11)

CDFM → isa → Foundation Model

confidence 98% · we formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework

Peng Cui → affiliatedwith → Tsinghua University

confidence 95% · Peng Cui 3 ... 3 Department of Computer Science and Technology, Tsinghua University

Jie Qiao → affiliatedwith → Guangdong University of Technology

confidence 95% · Jie Qiao 1 ... 1 School of Computer Science, Guangdong University of Technology

Ruichu Cai → affiliatedwith → Guangdong University of Technology

confidence 95% · Ruichu Cai 1∗ ... 1 School of Computer Science, Guangdong University of Technology

CDFM → outperforms → traditional algorithms

confidence 95% · CDFM consistently outperforms traditional algorithms

CDFM → pretrainedon → synthetic structural causal models

confidence 92% · By pretraining on a massive, highly diverse space of synthetic structural causal models

CDFM → usesmethod → variational framework

confidence 90% · we formulate a principled variational framework that treats unknown causal mechanisms as latent variables

LiNGAM → assumes → linear non-Gaussianity

confidence 88% · Linear non-Gaussian models (LiNGAM) break symmetry by assuming linear equations with non-Gaussian noise.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Causal discovery, the process of recovering underlying causal structures from observational data, is a fundamental pursuit across scientific disciplines. Over the past decades, numerous algorithms have been developed to tackle this challenge through workflows tailored to the specific causal mechanisms underlying each type of dataset, demonstrating effectiveness across a wide range of applications. However, as the volume and heterogeneity of real-world data continue to grow, this dataset-specific approach inevitably leads to a fragmented, test-driven paradigm that struggles to scale to the demands of modern scientific discovery. To address this, we formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference. To ensure reliable generalization across unknown domains, we first investigate the theoretical boundaries of causal identifiability, revealing the indispensable role of causal prior mechanisms in this process. Building on these insights, we formulate a principled variational framework that treats unknown causal mechanisms as latent variables and mathematically decomposes the intractable marginal likelihood into distinct, tractable learning modules. The variational decomposition provides a conceptual design principle for the architecture design of CDFM, while comprehensive causal knowledge guides the large-scale synthesis of our pretraining data. By pretraining on a massive, highly diverse space of synthetic structural causal models, CDFM successfully internalizes complex statistical asymmetries. Extensive experiments demonstrate that CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

84,013 characters extracted from source content.

Expand or collapse full text

DMIR LAB CDFM: Towards a General-Purpose Causal Discovery Foundation Model Jie Qiao 1 , Ruichu Cai 1∗ , Zijian Li 1 , Weilin Chen 1 , Pengfei Hua 1 , Boyan Xu 1 , Zhengming Chen 2 , Zhifeng Hao 2 , Peng Cui 3 1 School of Computer Science, Guangdong University of Technology, Guangzhou, China 2 College of Mathematics and Computer, Shantou University, Shantou, China 3 Department of Computer Science and Technology, Tsinghua University, Beijing, China qiaojie.chn,cuiruichu,leizigin,chenweilin.chn,hua13662626240,hpakyim,chenzhengming1103@gmail.com, haozhifeng@stu.edu.cn, cuip@tsinghua.edu.cn Abstract Causal discovery, the process of recovering underlying causal structures from observational data, is a fundamental pursuit across scientific disciplines. Over the past decades, numerous algorithms have been developed to tackle this challenge through workflows tailored to the specific causal mechanisms underlying each type of dataset, demonstrating effectiveness across a wide range of applications. However, as the volume and heterogeneity of real-world data continue to grow, this dataset-specific approach inevitably leads to a fragmented, test-driven paradigm that struggles to scale to the demands of modern scientific discovery. To address this, we formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference. To ensure reliable generalization across unknown domains, we first investigate the theoretical boundaries of causal identifiability, revealing the indispensable role of causal prior mechanisms in this process. Building on these insights, we formulate a principled variational framework that treats unknown causal mechanisms as latent variables and mathematically decomposes the intractable marginal likelihood into distinct, tractable learning modules. The variational decomposition provides a conceptual design principle for the architecture design of CDFM, while comprehensive causal knowledge guides the large-scale synthesis of our pretraining data. By pretraining on a massive, highly diverse space of synthetic structural causal models, CDFM successfully internalizes complex statistical asymmetries. Extensive experiments demonstrate that CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model. Correspondence: Ruichu Cai (cairuichu@gmail.com) Code: The code is available at https://github.com/DMIRLAB-Group/CDFM. 1 Introduction Causal discovery, the process of recovering underlying causal structures from observational data, is a funda- mental pursuit across scientific disciplines, with broad applications in fields such as economics [9], biology [34], and social science [5]. Over the past decades, numerous causal discovery algorithms have been developed, demonstrating effectiveness across a wide variety of datasets [8, 20, 33]. 1 arXiv:2607.11508v1 [cs.LG] 13 Jul 2026 (a) Traditional Causal Discovery Workflow 12345 Input Data Data Preprocessing Multiple Algorithms Selecting Assumption Checking PC / FCI LiNGAM ANM LSNM Linearity Non-Gaussianity Heteroskedasticity Causal Sufficiency Passed Tests? No Yes Paradigm Shift (b) Causal Discovery Foundation Model 1 2 4 Input Data Simple Preprocessing Standardize Construct Missing Indicator Pretraining Diverse Mechanisms Causal Structures ...... Foundation Model Learn Causal Structure Directly Select Imputation Method Mean KNN MICE ... Data Cleaning & Transformation Missing ValueDetection Output Output Causal Structure Causal Structure 3 Figure 1. Paradigm shift from traditional causal discovery to CDFM. Instead of selecting algorithms and verifying assumptions for each dataset, CDFM performs lightweight preprocessing and directly infers causal structures using a pretrained foundation model. The model is pretrained on diverse synthetic causal mechanisms and structures, enabling generalized causal reasoning across heterogeneous data distributions. Despite these advancements, the field remains fundamentally bottlenecked by the traditional causal discovery workflow, which is highly fragmented, test-driven, and complex. As shown in Figure 1, existing algorithms are meticulously designed to operate under specific causal mechanisms (e.g., linear non-Gaussianity [36] or additive noise [11], etc.). Consequently, these algorithms excel only when their underlying assumptions strictly hold. This creates isolated approaches that fail to generalize across heterogeneous real-world data, forcing practitioners into an exhaustive loop of manually re-evaluating algorithms and statistical tests. As the volume and heterogeneity of data continue to grow, this manual, hypothesis-matching paradigm becomes increasingly untenable; it cannot scale to the demands of modern, automated scientific discovery. What is needed is not another assumption-specific algorithm, but a unified model that can automatically accommodate—or even infer—the appropriate causal structures and mechanisms directly from data. Hence, we are led to the central question: Is it possible to develop a foundation model that transcends rigid assumptions to enable a general-purpose approach to causal discovery? Recent foundation models for tabular data provide evidence that this strategy is plausible. Models such as TabPFN [10], LimiX [50], and TabICL [30] learn reusable inductive biases from large synthetic or heterogeneous pretraining distributions. Related work has extended this idea to causal effect estimation [2,19,31] and amortized causal discovery [17,40]. However, a general formulation connecting causal identifiability, mechanism uncertainty, architecture design, and pretraining coverage remains underdeveloped. We introduce the Causal Discovery Foundation Model (CDFM), a framework for direct structural inference across heterogeneous and initially unknown causal mechanisms. CDFM is pretrained on diverse synthetic data sampled from a broad space of identifiable causal assumptions, incorporating rich causal knowledge. The resulting model emerges as a general-purpose causal discovery foundation model that is capable of zero-shot structural inference across diverse, unknown mechanisms. The main contributions of this work are summarized as follows: •We formulate a causal discovery foundation model, a paradigm that is specifically tailored for general- 2 purpose causal discovery, shifting the paradigm from specialized algorithm selection to a unified inference approach. To mathematically ground this, we unify foundation models and structural identifiability through a principled variational framework, treating unknown mechanisms as latent variables to translate the marginalization problem into a tractable learning objective. •We design CDFM that aligns mechanism encoding, structure-aware reconstruction, and graph decoding with the theoretical decomposition, allowing the model to robustly infer causal structures directly from observational data. •We introduce a comprehensive, large-scale pretraining driven by diverse causal knowledge. By synthesizing a massive hypothesis space of identifiable causal mechanisms and structures, we enable the model to internalize complex statistical asymmetries and effectively generalize to unknown generating processes, and extensive experiments demonstrate that CDFM achieves zero-shot generalization across diverse data distributions and real-world datasets. 2 Related Work The methodology of causal discovery from observational data has evolved significantly in recent years. In this section, we review the literature, tracing the progression from theoretically constrained algorithms to amortized inference and the generalized foundation models. Constrained causal discovery. The traditional approach to causal discovery relies on algorithms derived under specific constraints about the data-generating process. Typical independence constraint-based methods, such as PC and FCI [38], exploit conditional independencies under the causal Markov and faithfulness assumptions, typically identifying structures only up to a Markov equivalence class (MEC). To resolve MEC ambiguities, additional Functional Causal Models (FCMs) are included for imposing strict structural constraints. Linear non-Gaussian models (LiNGAM) [36,37] break symmetry by assuming linear equations with non-Gaussian noise. Additive noise models (ANM) [11] and post-nonlinear models [49] extend this identifiability to nonlinear regimes. This family is further enriched by causal additive models [3] and causal location-scale noise models [12,48], which leverage dependent noise variance as an additional source of causal asymmetry. For discrete variables, a series of works have addressed ordinal, categorical, and count data [4,21,25,27–29,45,46]. Later advancements transitioned from combinatorial search to continuous optimization (e.g., NOTEARS [51]), utilizing smooth algebraic characterizations of acyclicity. While these methods provide rigorous identifiability guarantees within their respective domains, they require a priori knowledge of the underlying mechanisms, lacking the generalizability to the heterogeneous real-world data where the true data-generating process is unknown. Amortized causal discovery. To overcome the bottleneck of solving dataset-specific optimization problems, amortized inference reframes causal discovery as a supervised learning task. Methods like AVICI [17] and CSIvA [13] train neural networks to infer causal graphs directly from observational and interventional samples. This learning-based framework has been subsequently expanded across various dimensions, including applications to time-series data [18], active causal induction using deep reinforcement learning [1], and scalable conditional independence testing [15]. Further methodological extensions [24,41,44,47] have demonstrated the empirical feasibility of large-scale causal discovery. While these amortized models successfully internalize useful inductive biases from simulated data via empirical heuristics, a systematic theoretical framework for addressing structural identifiability across highly heterogeneous and unknown mechanisms remains unresolved. Foundation models for tabular data. Large-scale pretraining has recently demonstrated remarkable in-context generalization on tabular and structured data tasks. Models such as LimiX [50], TabPFN [10], and TabICL [30] have shown that foundation architectures can effectively model complex, heterogeneous datasets. Within the causal inference domain, efforts like CausalPFN [2] and Do-PFN [31] have successfully adapted these architectures for causal effect estimation. These advancements illustrate the profound potential of foundation models to generalize across disparate data distributions. 3 Toward causal discovery foundation models. The evolution of causal discovery reveals a fundamental paradigm shift: from isolated, assumption-bound algorithms to amortized inference, and ultimately, causal discovery foundation models. In this work, we systematically study the causal discovery foundation model, establishing a principled methodology for general-purpose causal discovery. 3 Background and Preliminaries We represent the data-generating process using a Structural Causal Model (SCM). Let X =X 1 ,X 2 ,...,X D be observed variables, whereDdenotes the number of variables, and L be unobserved latent confounders. The system is governed by structural assignments: V i := f i (PA i ,N i ), for V i ∈ V = X∪ L,(1) wherePA i are direct causal parents,f i is a deterministic mechanism, andN i is exogenous noise. By marginalizing out L, we represent the observed structure as an adjacency matrixG∈0,1 D×D encoding the causal structure over the observed variables after marginalizing out L. In addition, to capture the effects of latent confounding, we augment the graph with bidirected edges, signifying the existence of at least one unobserved common causeL ∈L that directly influences both variables. Note that if bothX i → X j and latent confounding exist, we encode only the directed edge, and we restrict the target graph class to bow-free acyclic directed mixed graphs. Given this formulation, our primary goal is to robustly infer the underlying causal graphGfrom purely observational data. Crucially, unlike traditional approaches that rely on manually specifying a restrictive causal prior mechanismM, we aim to uncover these causal relationships across a massive space of unknown and heterogeneous mechanisms M. 4 Paradigm Shift: From Identifiability Theory to Foundation Models As discussed earlier by Pearl [22], behind any causal conclusion, there must be some causal assumption, untested in observational studies. The role of a foundation model is therefore not to remove assumptions, but to learn and adopt over a broad spectrum of assumption classes. To develop principled guidelines for the design of CDFM, enabling it to generalize across diverse causal mechanisms, we therefore first investigate the theoretical boundaries of causal identifiability and guide the architecture design of CDFM. In causal discovery, underlying assumptions are typically encapsulated implicitly within structural causal models. However, as established by [26], an SCM with independent noise alone is insufficient to identify the causal direction. Theorem 1 (Impossibility of Learning Causal Direction without Constraint). LetXandYbe two random variables. For any joint distributionP X,Y , valid SCMs exist for both causal orderings simultaneously. That is, there exist measurable functions f X ,f Y and real-valued noise variables N X ,N Y such that: Y = f Y (X,N Y ), X ⊥ N Y , X = f X (Y,N X ), Y ⊥ N X . (2) Crucially, because this holds for any joint distribution, the variables are perfectly symmetric. Thus, identifying the true causal graph is mathematically impossible without imposing prior assumptions on the underlying structural equations. Traditional algorithms resolve this by manually asserting a specific, known causal prior mechanism (e.g., linear non-Gaussianity [37], or additive noise model [11]). Given that the true mechanismM ∗ is known, identifiability transforms into a structural model selection problem. The optimal graph minimizes the Kullback-Leibler (KL) divergence between the true distribution and the model distribution. Theorem 2 (Identifiability under Known Mechanism). Let the data X be generated by a true graphG ∗ and a known causal mechanismM ∗ . For any alternative graphG ′ , the difference in expected log-likelihood under the true distribution L(G ∗ ) := E X∼P(·|G ∗ ,M ∗ ) [logP (X| G ∗ ,M ∗ )] is bounded by the KL divergence: L(G ∗ )−L(G ′ ) = D KL P (X|G ∗ ,M ∗ )∥ P (X|G ′ ,M ∗ ) ≥ 0.(3) 4 IfM ∗ belongs to a strictly identifiable class, the inequality is strict for everyG ′ ̸=G ∗ , and equality holds if and only if G ′ = G ∗ . Unlike traditional algorithms, a foundation model drives a paradigm shift in which it does not assume a single, known mechanismM ∗ . Instead, it aims to map an observational dataset X∈ R N×D directly to the causal graphGacross a vast hypothesis space of diverse, unknown mechanismsM. This shift means the mapping inevitably amounts to a Bayesian model selection problem, where the optimal likelihood requires marginalizing over the entire mechanism space M: E X [logP (X,G)] = E X log Z M P (X|M,G)P (G|M )P (M )dM .(4) One may still identify the causal structure using the exact marginal likelihood as long as the space of causal prior mechanismsMsatisfies a strict identifiability property. We say the spaceMis strictly identifiable if no alternative structure and mechanism can perfectly mimic the true generating distribution. Formally, P(X|M ∗ ,G ∗ )̸=P(X|M G ′ ,G ′ ) holds for allG ′ ̸=G ∗ , whereM G ′ =arg max M∈M E X [logP(X|M,G ′ )] represents the optimal mechanism choice underG ′ . If the space possesses this intrinsic mutual exclusivity, the exact marginal likelihood uniquely identifies the true causal graph. Theorem 3 (Identifiability by Marginal Likelihood). Let a dataset X (N) ofNi.i.d. samples be generated by the true graphG ∗ and true mechanismM ∗ ∈ M. If the space of causal prior mechanismsMsatisfies the strict space identifiability property, then the exact marginal likelihood uniquely identifies the true causal graph almost surely: lim N→∞ 1 N logP (X (N) |G ∗ ) > lim N→∞ 1 N logP (X (N) |G ′ ), a.s., ∀G ′ ̸= G ∗ .(5) While Theorem 3 guarantees that the true graph can be identified in principle, directly computing the exact integral in Eq. (4) is intractable. To bridge this gap, we formulate the learning process of CDFM through variational inference. Under uninformative uniform priors ofP(G) andP(M), maximizing the marginal likelihood is mathematically equivalent to maximizing the joint likelihood. We introduce a parameterized variational distributionQ(M |X), which is designed to internalize the mechanism and structural information within the architecture. This yields a tractable Evidence Lower Bound (ELBO) over the joint likelihood (see Appendix A.1 for derivation): logP (G,X)≥ E Q(M|X) logP (X|G,M ) + logP (G|M ) − D KL Q(M|X)∥ P (M ) ,(6) where the final KL divergence term serves as a regularizer over the mechanism space, which acts as a constant shift under a standard uninformative uniform prior. This variational formulation explicitly decomposes the learning process of CDFM by decomposing it into three explicit components that govern the design of CDFM: 1.Mechanism InferenceQ(M|X): The model extracts statistical footprints (e.g., noise properties, non- linearities) to infer a latent encoding of the active mechanism and the structural information. 2.Data Reconstructionlog P(X|G, M): Theoretical identifiability requires the inferred mechanism and graph to perfectly explain the data. This provides a self-supervised gradient signal. 3. Graph Inferencelog P(G|M): The model decodes the structural graph conditioned on the neurally encoded mechanism and its structural information. By this, with sufficient capacity and massive pretraining, the variational gap (D KL between the true and approximate posterior) vanishes at the global optimum. Consequently, optimizing this ELBO implicitly maximizes the exact marginal likelihood. This guarantees that CDFM inherently respects the mathematical boundaries of causal identifiability: Corollary 1 (Identifiability of CDFM). Suppose the foundation model has sufficient capacity to reach the optimum variational ELBO. Let X (N) be any test dataset with unknown causal graphG ∗ . In the inference 5 Guides generation Pretraining Enhances understanding Causal Knowledge Domain knowledge Synthetic Data ... Causal Discovery Foundation Model Input data Learned causal structure Identifiable causal prior mechanisms ◼Physical mechanisms ◼Feasibility constraints ◼Structural priors ◼Empirical patterns Causal direction X Y f vs. Reverse direction XY g ... Additive noise model 푦 = 푓(푥) + 푛 Linear model 푦 = 푎푥 + 푛 Post-nonlinear model 푦=푓 2 푓 1 푥+푛 ... Figure 2. The foundation model paradigm for causal discovery. Causal knowledge guides the generation of diverse synthetic data. Pre-training on this data allows the foundation model to internalize causal asymmetries. Zero-shot inference on real-world data enhances our understanding of unknown mechanisms, which in turn helps us refine and expand the causal knowledge, ultimately driving towards a universally generalizable model. phase, the model predicts the graph by maximizing the expected log-likelihood of the structural decoder: ˆ G = arg max G∈G E Q(M|X (N) ) logP (G| M ) .(7) Under the strict identifiability of the mechanism spaceMand a uniform priorP(G), this structural inference uniquely recovers the true causal graph almost surely, i.e., ˆ G = G ∗ as N →∞. By establishing that our architecture is consistent with constraints via the variational framework, we formalize a fundamental paradigm shift in causal discovery, as illustrated in Figure 2. In traditional frameworks, theoretical identifiability merely served as a prerequisite for the specified causal mechanisms. In CDFM, however, Corollary 1 shifts the core bottleneck from manual algorithm selection to the scalable generation of diverse, identifiable synthetic data. This forms an iterative, self-reinforcing closed loop. Existing causal knowledge guides the generation of diverse synthetic datasets. Pre-training on this vast space allows the model’s parameterized distributions to internalize complex causal asymmetries. Subsequently, zero-shot inference on real-world data evaluates the model’s structural predictions. Insights gained from its performance reveal previously unknown causal mechanisms. This feedback empowers us to refine our theoretical understanding of identifiability, expand the underlying causal knowledge base, and generate even richer synthetic data, ultimately driving the foundation model toward a universally generalizable causal discovery engine. 5 Causal Discovery Foundation Model CDFM is designed to instantiate the intractable marginalization over the mechanism spaceMthrough a deep neural architecture. It takes an observation matrix X∈ R N×D (consisting ofNobservations overDvariables, with potential missing entries) and infers the underlying causal graphG ∈ 0,1 D×D . As formalized in Section 3, the graph would be an adjacency matrix augmented with bidirected edges when latent confounding is present. The architectural design of CDFM explicitly mirrors the three components of the variational ELBO: mechanism inference (Section 5.1), data reconstruction (Section 5.2), and graph inference (Section 5.3). 6 ... N D Q ... ... Missing Value Imputation B × N × D × Q N D E ... ... ... ... N D Q ... ... Graph Inference Induced Attention (2 inducing points for each variables) Cause D × H c Effect D × H c D 0 0.5 1 B × D × D D Induced Attention (M inducing experts) ... N D E ... ... ... 풆 ퟏ 풆 ퟐ 풆 푴 Mixture of Experts Compression to 2 Inducing Points Learned Causal Structure Bilinear Head (푐 푇 푊푒) Row-wise (over D) Col-wise (over N) Causal Mask (used for adaptation) N D E ... ... ... Row-wise Block MHA + FFN Causal Mask M (over variables) Col-wise Block MHA + FFN R 1 C 1 R L C L N D E ... ... ... Alternating Row-Col Attention MLP (Imputation Head) Inner Loop (Adaptation) Current Graph ෠ 퐺 Causal Mask MReconstruction Reconstruction Loss Optimize / Refine Ĝ (a) Column-wise Embedding B × N × D × E N D 2 ... ... ... Linear (2 → E) Set Transformer (ISAB × 3) ⋮ N D E ... ... ... (b)(c) (d) Figure 3. The overall architecture of CDFM. The overview of the architecture of CDFM, as illustrated in Figure 3, mainly consists of three distinct components: 1.Column-Wise Embedding and Alternating Attention (Mechanism Inference): Each scalar is first embedded into anE-dimensional vector via a 2-channel, distribution-aware Set Transformer to capture marginal footprints and missing-value patterns. The embedded tensor is then processed by a stack of 2Lattention blocks that alternate between the sample and variable axes, integrating joint dependencies to parameterize the latent mechanism distribution Q(M|X). 2.Missing Value Imputation (Data Reconstruction): A residual block predictsQconditional quantile values for each masked position, serving as a self-supervised task to satisfy the data reconstruction term P (X|G,M ). 3. Extraction and Graph Inference: Induced self-attention extracts cause and effect summary vectors, from which a Mixture-of-Experts bilinear head produces the final edge logits, which fulfill the structural decoding term P (G|M ). 5.1 Mechanism Inference: Column-Wise Embedding and Alternating Attention To build the variational distributionQ(M|X), CDFM first extracts high-order statistics of each feature from the data by transforming each scalar cell into a dense vector representation. To achieve this while explicitly supporting incomplete real-world datasets and paving the way for the subsequent self-supervised data reconstruction task, CDFM begins with a column-wise embedding module based on a two-channel input encoding. Specifically, each observed scalar valuex ij is paired with a binary missingness indicatorr ij ∈0,1, forming a two-dimensional vector [x ij ,r ij ]. For a missing entry,x ij is replaced by zero after column-wise standardization, whiler ij = 1 explicitly marks its missingness. A linear projection maps this pair into the initial embedding space: u ij = Linear in ([x ij , r ij ]) ∈ R E .(8) This missingness channel allows the model to distinguish the missing entries, aiding downstream imputation by marking which positions require reconstruction. 7 Subsequently, inspired by [30], to encompass the high-order statistics of marginal distributions, we employ a shared Set Transformer [14] that treats each column as an unordered set of cell values. For each column c j =x 1j ,...,x Nj , the projected embeddings are processed by a set transformer composed ofL col = 3 bidirectional induced self-attention blocks (ISAB) [14]. Each ISAB block operates as follows: H = MAB 1 (V I , U, U ) ∈ R K×E , V = MAB 2 (U, H, H) ∈ R N×E , (9) where MAB denotes a multi-head attention block,V I ∈ R K×E areKlearnable inducing vectors, andUrefers to the embedded samples within the column. After the ISAB blocks, two independent linear projections generate per-sample affine transformation parameters: W j = Linear W (V ), B j = Linear B (V ),(10) and the final column embedding is obtained via an element-wise affine transform applied only to the value channel: e ij = W ij ⊙ x ij + B ij ∈ R E .(11) This affine gating mechanism inspects the empirical distribution of the entire column and generates per-cell weights and biases that encode each value’s position relative to the column’s distribution. Empirically, this causes columns with similar distributional properties to cluster in embedding space, effectively capturing the structural footprints of the mechanisms. Building upon these marginal footprints extracted from individual columns, CDFM proceeds to capture the joint dependencies across variables to fully parameterize the variational distributionQ(M|X). This is achieved through a sequence of alternating row- and column-wise attention operations that progressively integrate information across both axes of the data, as illustrated in Figure 3. Each attention step employs a multi-head attention block followed by a feedforward network with GELU activation, synthesizing complex variable-sample interactions into a comprehensive latent mechanism state. 5.2 Data Reconstruction: Causal Masking and Imputation To instantiate the data reconstruction termlogP(X| G,M), CDFM employs a self-supervised missing value imputation task guided by a soft structural masking mechanism. During the forward pass, even-indexed attention blocks incorporate a soft causal mask S∈ R D×D that dictates the information flow between variables. Rather than applying a rigid, non-differentiable penalty, CDFM employs a continuous relaxation parameterized by the continuous edge weightsWinferred from the current graph hypothesis. When variableiattends to variablej(i̸=j), the attention logit is penalized based on the strength of the directed edge j → i: S ij =−β· σ − W ji − τ T (12) whereσis the sigmoid function,τis a decision threshold,Tis a temperature parameter controlling the steepness of the transition, andβis a scaling factor determining the maximum penalty. For self-attention, the diagonal remains unpenalized (S i = 0). This soft inductive bias ensures that ifjis not a likely cause ofi(W ji ≪ τ), the attention mechanism heavily restricts information flow fromjtoi. Conversely, strong causal edges allow unhindered attention. This formulation encourages the model to evaluate the target variable as a function primarily of its direct causes. Following the final alternating block, the model gathers the contextualized embeddingsz (2L) at positions originally marked as missing (r ij = 1) and processes them through a residual block to predict quantile levels ˆy ij ∈ R Q . By evaluating the discrepancy between these predictions and the masked true values, the imputation task provides a continuous gradient signal for structure learning. Crucially, because the mask S is soft and differentiable with respect to the edge weightsW, it seamlessly facilitates an unsupervised inner loop during inference. In this adaptation phase, the model iteratively updates 8 the continuous causal graph hypothesis by minimizing the imputation error. This inner loop allows CDFM to refine its structure predictions on novel datasets without relying on ground-truth causal labels, aligning with the principle that an accurate causal graph should yield minimal observational reconstruction error. 5.3 Graph Inference The final architectural component realizes the graph inference termlogP(G | M), decoding the causal structure from the neurally encoded mechanism representations. To produce a causal graph prediction that remains invariant to the sample sizeN, CDFM aggregates the sample-wise contextual representations into variable-level structural summaries. To learn causal structure from a diverse mechanism spaceM, CDFM first employs an induced self-attention block with 2Klearnable inducing vectors asKexperts, aiming to extract the causal mechanism Mixture-of-Experts (MoE) extraction process. Subsequently, these intermediate expert representations are routed through an attention-based aggregator, condensing the diverse expert outputs down to two global summary vectors for each variablei: a cause representation e c i and an effect representation e e i . This attention-driven compression effectively acts as a learned soft-routing mechanism, adaptively attending to the most relevant expert evidence based on the active underlying data-generating process. These representations are evaluated to obtain the respective cause and effect vectors, yielding u i = W c e c i and v j = W e e e j , where u i , v j ∈ R p . The continuous edge logit is then computed using a temperature-scaled dot product augmented by a global sparsity biasA ij =exp(τ)· u T i v j √ p +b 0 , wherepis the projection dimension,τ is a learned log-temperature parameter governing the sharpness of the edge probabilities, andb 0 is a learned global bias representing the baseline sparsity prior of the causal graphs. The diagonal is strictly masked to −∞to explicitly preclude causal self-loops. The inherent asymmetry of this bilinear evaluation (u T i v j ̸= u T j v i ) naturally accommodates both directed causal relationships and bidirected edges. Moreover, to convert the continuous edge probabilities into a binary graph, we employ a lightweight graph- adaptive threshold calibrator fitted on an independent synthetic calibration set. The calibrator uses ten graph-level features, including the mean and standard deviation, distributional quantiles, the logarithm of the number of variables, the median and maximum gaps between consecutively sorted probabilities, and a Kneedle-inspired knee statistic describing the separation between high-confidence edges and the low-probability tail [35]. The graph-specific threshold is predicted by a logit-linear model whose coefficients are learned once from the oracle thresholds of held-out synthetic graphs and remain fixed during evaluation. 5.4 Causal Prior Construction and Pretraining The generalization capability of CDFM fundamentally depends on the diversity and identifiability of its pretraining data. We construct a massive-scale synthetic data generator that produces (X,G) pairs on the fly, where each training instance is sampled from an independently randomized causal model. The generator spans a broad spectrum of causal scenarios through the following dimensions: Graph structure diversity. Causal graphs are sampled from six distinct random graph models to cover a wide range of topological patterns: Erdős–Rényi (ER), scale-free (SF) and transposed scale-free (SFT), Watts–Strogatz small-world (WS), stochastic block model (SBM), and geometric random graph (GRG). The expected number of edges per variable is uniformly sampled from1,2,3, yielding graphs that range from sparse trees to moderately dense DAGs. The number of observed variables and sample size are drawn uniformly from [2, 100] and [50, 4096], respectively. Mechanism type diversity. The core of our data generator is a broad taxonomy of structural causal mechanisms, each encoding distinct functional-form and distribution. We categorize these mechanisms into several families: 9 Continuous additive mechanisms include linear structural equation models (e.g.,X j = P i∈PA j w ij X i +N j ) with both homoscedastic and heteroscedastic noise [48], random Fourier feature (RFF) models that introduce highly nonlinear dependencies; post-nonlinear causal models [49] where a nonlinear distortion is applied after additive mixing (X j =g 2 (g 1 (PA j ) +N j )), causal additive models [3] where each parent contributes through an independent univariate nonlinearity, and physics-inspired mechanisms. Discrete and mixed-type mechanisms target the prevalent non-continuous data in real-world applications. These include conditional probability tables for purely categorical variables; discrete additive noise models [25] for ordinal discrete data; and ordinal causal discovery formulations [21] that bridge continuous and discrete regimes within a single mechanism. Post-processing transformations simulate measurement and recording imperfections. Independent of the base SCM, each batch may undergo additive measurement error (perturbing a subset of variables with independent Gaussian noise) or discretization (rounding a subset of child variables to coarse bins). These transformations can co-occur within the same batch, producing data that reflects the messy, multi-step nature of real-world measurement. Noise and edge-weight diversity. To ensure that CDFM can disentangle causal signal from exogenous variation, we vary both the noise distribution and the signal-to-noise ratio per mechanism instance. The exogenous noiseN j for each variable is drawn from one of several different distribution families, such as Gaussian, Laplace, Cauchy, Beta, Exponential, Student’s-t, log-normal, and Gaussian mixtures. Latent confounding. Causal sufficiency rarely holds in practice. To equip CDFM with the ability to reason under unobserved confounding, 15% of training batches include latent variables. The generator selects d latent non-leaf nodes as latent, and then generates the full SCM, and finally drops the latent columns. In the resulting ground-truth graph, observed variable pairs that share a latent parent receive bidirected edges X i ↔ X j , while direct causal links retain directed edgesX i → X j . When a pair exhibits both a direct causal effect and a shared latent confounder, the directed edge takes priority. Acyclic constraint is disabled for these batches since bidirected edges inherently form cycles in the adjacency matrix. Missing values. We further introduce random missingness to provide the auxiliary imputation task with a training signal and to prepare the model for incomplete real-world datasets. With 50% probability, a batch undergoes masking where some portion of the entries are missing completely at random (MCAR). The original values are retained exclusively for computing the quantile imputation loss, while the model receives only the masked matrix with an explicit binary mask channel. 5.5 Training Objective CDFM is trained end-to-end using a joint objective function that encompasses standard classification supervision, margin-based regularization, sparsity constraints, and self-supervised reconstruction terms: L total =L BCE + α margin ·L margin + γ F 1 ·L soft−F 1 + β acyc ·L acyc + α impute ·L quantile ,(13) where α margin , γ F 1 , β acyc , and α impute balance the respective loss components. Structural supervision. To ensure reliable numerical convergence and sharp directional confidence, structural supervision combines standard binary cross-entropy (L BCE ) and a margin-based variant (L margin ). The BCE loss establishes baseline probabilistic alignment: L BCE =− 1 |E valid | X (i,j)∈E valid h w pos · G ij logσ(W ij ) + (1− G ij ) log(1− σ(W ij )) i ,(14) wherew pos is a positive-class weight adjusted for graph sparsity. The margin-based component acts as a structural regularizer that explicitly penalizes ambiguous predictions near the classification threshold, utilizing 10 a strict margin parameter m: L margin =− 1 |E valid | X (i,j)∈E valid h w pos · G ij logσ(W ij − m) + (1− G ij ) log(1− σ(W ij + m)) i ,(15) SoftF 1 loss. Because causal adjacency matrices are inherently sparse, standard cross-entropy objectives frequently converge to trivial sparse solutions. We incorporate a differentiable relaxation of the F 1 -score: L soft−F 1 = 1− 2· TP soft 2· TP soft + FP soft + FN soft ,(16) where the soft true positives (TP soft ), false positives (FP soft ), and false negatives (FN soft ) are evaluated continuously over the predicted sigmoid probabilities P = σ(W ) and the ground-truth matrix G: TP soft = X i̸=j P ij G ij ,FP soft = X i̸=j P ij (1− G ij ),FN soft = X i̸=j (1− P ij )G ij .(17) Acyclicity constraint. To enforce the structural validity of directed graphs, CDFM integrates a spectral radius acyclicity penaltyL acyc =max(0,ρ(σ(W))− δ), estimated dynamically via power iteration. This constraint is systematically omitted for batches containing latent confounding, where bidirected edges naturally permit cyclic representations in the observed mixed graph. Quantile imputation loss. Corresponding with the data reconstruction termlogP(X| G,M) defined within the variational lower bound, a pinball loss is evaluated at a set ofQdesignated quantile levels (denoted by q k ∈ (0, 1) for k = 1,...,Q) across the set of missing entries R: L quantile = 1 |R|· Q X (i,j)∈R Q X k=1 2· (x true ij − ˆy q k ij )· q k − 1[x true ij ≤ ˆy q k ij ] .(18) 6 Experiments We organize the experiments around three central questions. First, can a single pretrained CDFM model recover causal structures across heterogeneous structural causal models, graph scales, and sample regimes without dataset-specific optimization? Second, can the causal representations learned from synthetic SCMs transfer to physical and empirical systems in the real-world? Third, does CDFM exhibit behavior consistent with established boundaries of causal identifiability under controlled conditions? We additionally evaluate the missing-value reconstruction head as an auxiliary capability arising from the structure-aware training objective. 6.1 Experimental Setup For CDFM, all benchmark experiments are conducted in a zero-shot setting. The model is directly applied to each test dataset without access to its ground-truth graph and without dataset-specific parameter optimization, and a frozen graph-adaptive calibration is used for the decision boundary. The controlled identifiability studies in Section 6.4 follow a different protocol. For these diagnostic experiments, separately adapted copies of CDFM are trained on deliberately constructed low-dimensional SCM families. These experiments are intended to examine whether the model architecture can learn decision boundaries consistent with known identifiability results; they are not included in the zero-shot benchmark claims. Baselines. We compare CDFM with representative causal discovery approaches covering different method- ological paradigms. These include amortized causal discovery models (AVICI [17], TabCausal [16], Arrow [41]), constraint-based methods (PC [38]), score-based methods (GES [6]), functional causal models (DirectLiNGAM [37]), and continuous optimization methods (NOTEARS [51]). All baselines are evaluated using their standard implementations with recommended hyperparameters. 11 Table 1. Synthetic benchmark AUROC grouped by sample size. Results averaged over 15 mechanism families and ten graph sizes (D ∈10, 15, 20, 25, 30, 40, 50, 70, 100). Method500 samples 1000 samples 2000 samples 3000 samples 4000 samples CDFM0.864 ± 0.109 0.878 ± 0.104 0.885 ± 0.101 0.887 ± 0.100 0.888 ± 0.101 TabCausal0.767 ± 0.166 0.774 ± 0.166 0.782 ± 0.167 0.786 ± 0.164 0.786 ± 0.166 Arrow0.676 ± 0.153 0.669 ± 0.149 0.680 ± 0.154 0.682 ± 0.154 0.685 ± 0.154 AVICI0.756 ± 0.163 0.760 ± 0.165 0.764 ± 0.164 0.761 ± 0.161 0.760 ± 0.163 DirectLiNGAM 0.676 ± 0.150 0.689 ± 0.154 0.713 ± 0.158 0.724 ± 0.162 0.731 ± 0.164 GES0.649 ± 0.105 0.657 ± 0.108 0.663 ± 0.110 0.667 ± 0.110 0.667 ± 0.112 PC0.700 ± 0.122 0.715 ± 0.127 0.728 ± 0.129 0.737 ± 0.132 0.740 ± 0.133 NOTEARS0.563 ± 0.127 0.581 ± 0.124 0.574 ± 0.151 0.583 ± 0.157 0.585 ± 0.167 Metrics. For multivariate causal discovery, we evaluate edge ranking using AUROC and thresholded graph recovery using precision, recall,F 1 score. Results are averaged over independently generated datasets. For bivariate causal direction inference, we report weighted accuracy following the official evaluation protocol of the Tübingen benchmark. 6.2 Synthetic Benchmark The synthetic evaluation provides the primary controlled test of whether one frozen model can operate across a broad range of causal data-generating processes. Benchmark design. We construct a comprehensive synthetic evaluation suite spanning 15 distinct structural causal mechanism families (see Appendix B for detailed descriptions). These families encompass linear and nonlinear structural equations, homoscedastic and heteroscedastic noise regimes, continuous and discrete variable types, post-nonlinear transformations, measurement error perturbations, ordinal observations, rounded discretization, and time-lagged causal mechanisms. The benchmark systematically varies graph size acrossD ∈10,15,20,25,30,40,50,70,100variables and sample size acrossN ∈500,1000,2000,3000,4000observations. For each (D,N, mechanism) configuration, multiple independently sampled datasets are generated using held-out random seeds. Tables 1 and 2 summarize the performance of all methods aggregated over all 15 mechanism families and all graph sizes, grouped by sample size. CDFM outperforms all baselines across every sample regime in both AUROC and F 1 . This behavior indicates that CDFM can effectively utilize additional observational information during inference rather than relying on fixed-size statistical heuristics. In contrast to classical methods that often require manually selected assumptions about the underlying mechanism class, CDFM maintains stable performance across different sample regimes. Performance across sample sizes and graph sizes. Figure 4a visualizes these scaling trends across sample sizes fromN= 500 toN= 4000, and CDFM consistently outperforms all baselines in both regimes, with its advantage widening as the problem scale increases. Figure 4b evaluates performance as the number of variables increases fromD= 10 toD= 100. All methods become less accurate as the structural search space grows, but CDFM degrades more gradually than the baselines. AtD= 100, CDFM retains an AUROC above 0.87, whereas the strongest competing amortized model reaches approximately 0.75. The amortized methods (AVICI, TabCausal, Arrow) show similar scaling trends to CDFM but at consistently lower absolute performance levels. Classical constraint-based and score-based methods (PC, GES) exhibit 12 Table 2. Synthetic benchmarkF 1 score grouped by sample size. Results averaged over 15 mechanism families and ten graph sizes (D ∈10, 15, 20, 25, 30, 40, 50, 70, 100). Bold indicates best. Method500 samples 1000 samples 2000 samples 3000 samples 4000 samples CDFM0.552 ± 0.200 0.578 ± 0.197 0.587 ± 0.201 0.591 ± 0.202 0.591 ± 0.201 TabCausal0.378 ± 0.246 0.396 ± 0.250 0.403 ± 0.258 0.405 ± 0.258 0.407 ± 0.260 Arrow0.202 ± 0.155 0.196 ± 0.149 0.191 ± 0.155 0.188 ± 0.152 0.187 ± 0.153 AVICI0.354 ± 0.260 0.360 ± 0.263 0.357 ± 0.266 0.338 ± 0.257 0.338 ± 0.260 DirectLiNGAM 0.381 ± 0.233 0.404 ± 0.244 0.405 ± 0.230 0.401 ± 0.216 0.392 ± 0.207 GES0.370 ± 0.195 0.383 ± 0.196 0.389 ± 0.196 0.394 ± 0.195 0.395 ± 0.196 PC0.471 ± 0.197 0.493 ± 0.196 0.513 ± 0.196 0.522 ± 0.195 0.526 ± 0.195 NOTEARS0.138 ± 0.113 0.139 ± 0.110 0.141 ± 0.115 0.126 ± 0.110 0.126 ± 0.112 5001000200030004000 Sample Size N 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 AUROC CDFM TabCausal Arrow AVICI DirectLiNGAM GES PC NOTEARS (a) AUROC vs. sample size N. 1015202530405070100 Number of Variables D 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 AUROC CDFM TabCausal Arrow AVICI DirectLiNGAM GES PC NOTEARS (b) AUROC vs. number of variables D. Figure 4. Performance over sample size and the number of variables. (a) AUROC as a function of sample sizeN; (b) AUROC as a function of the number of variables D. Results are averaged over all 15 mechanism families. steeper performance degradation due to the combinatorial explosion of conditional independence tests and score evaluations in high dimensions. Performance across causal mechanisms. To further examine whether the observed performance originates from broad mechanism coverage rather than a small subset of favorable cases, we evaluate performance separately for each mechanism family. Figure 5 provides a visual comparison of CDFM against the best baseline per mechanism. The mechanism heatmap in Figure 6 further reveals that CDFM maintains a broad advantage across the evaluated causal mechanisms. Specifically, as shown in Table 3, CDFM achieves competitive or superior performance across most nonlinear, discrete, and corrupted observation settings. Specialized methods such as DirectLiNGAM remain advantageous in specific linear non-Gaussian scenarios, demonstrating the benefit of strong inductive assumptions when their assumptions are exactly satisfied. However, CDFM provides a more balanced performance profile across heterogeneous environments, suggesting that its advantage comes from mechanism-level generalization rather than optimization toward a single causal model class. 6.3 Real-world Benchmark We next examine whether the learned causal representation transfers beyond this synthetic evaluation environment. We consider a physical multivariate system from Causal Chamber and heterogeneous bivariate datasets from the Tübingen cause–effect benchmark. 13 Table 3. Synthetic robustness across 15 mechanism families. Results are averaged over nine graph sizes (D ∈ 10,15,20,25,30,40,50,70,100) and five sample sizes (N ∈500,1000,2000,3000,4000). Best baseline denotes the strongest non-CDFM method under each metric. Bold indicates best overall. Mechanism abbreviations are explained in Appendix B. MechanismCDFM AUROC↑ Best Baseline AUROC↑ AUROC baseline CDFM F 1 ↑ Best Baseline F 1 ↑ F 1 baseline CAM0.863 ± 0.0850.794 ± 0.116TabCausal0.591 ± 0.1560.508 ± 0.146PC CPT0.767 ± 0.1360.725 ± 0.122PC0.373 ± 0.2120.468 ± 0.177PC Discrete ANM0.881 ± 0.0820.783 ± 0.094PC0.600 ± 0.1710.598 ± 0.150PC Linear0.939 ± 0.0710.929 ± 0.089TabCausal0.668 ± 0.1920.759 ± 0.190DirectLiNGAM Linear-Hetero0.927 ± 0.0740.904 ± 0.098TabCausal0.641 ± 0.1870.729 ± 0.173DirectLiNGAM Linear-Ordinal0.862 ± 0.1130.714 ± 0.150PC0.553 ± 0.1960.477 ± 0.208PC Meas. Error0.882 ± 0.0980.827 ± 0.128TabCausal0.591 ± 0.1930.516 ± 0.191PC Physical0.833 ± 0.0970.763 ± 0.117TabCausal0.522 ± 0.1870.446 ± 0.167PC Physical-Hetero 0.870 ± 0.0810.740 ± 0.145TabCausal0.558 ± 0.1730.442 ± 0.149PC PNL0.901 ± 0.0960.813 ± 0.133TabCausal0.589 ± 0.2110.453 ± 0.218PC RFF0.947 ± 0.0540.918 ± 0.090TabCausal0.680 ± 0.1780.647 ± 0.204TabCausal RFF-Hetero0.938 ± 0.0570.906 ± 0.096AVICI0.654 ± 0.1780.647 ± 0.215AVICI RFF-Ordinal0.860 ± 0.1000.736 ± 0.135PC0.547 ± 0.1880.512 ± 0.176PC Rounded0.881 ± 0.0960.787 ± 0.136TabCausal0.575 ± 0.1930.456 ± 0.196PC Time-Lag0.854 ± 0.1150.757 ± 0.147TabCausal0.558 ± 0.1910.463 ± 0.189PC CAM CPT Disc.ANM Linear Lin-Het Lin-Ord MeasErr Physical Phys-Het PNL RFF RFF-Het RFF-Ord Rounded TimeLag 0.4 0.5 0.6 0.7 0.8 0.9 1.0 AUROC 0.863 0.767 0.881 0.939 0.927 0.862 0.882 0.833 0.870 0.901 0.947 0.938 0.860 0.881 0.854 0.794 0.725 0.783 0.929 0.904 0.714 0.827 0.763 0.740 0.813 0.918 0.906 0.736 0.787 0.757 CDFM Best Baseline Figure 5. Mechanism robustness. AUROC comparison between CDFM and the best-performing baseline for each of the 15 mechanism families. CDFM wins on all 15 mechanisms in AUROC. Causal Chamber. Causal Chamber contains experimentally validated causal discovery tasks generated from automatically controlled physical systems. We evaluate CDFM on the light-tunnel data using Tasks A1 and A2. For Task A1, we use the observational environment and infer the directed causal graph from measured variables. For Task A2, the dataset contains multiple intervention environments. Since CDFM receives a single observational matrix as input and does not explicitly model intervention targets, we introduce an environment-ensemble adaptation where each environment is processed independently, and predicted edge probabilities are averaged before evaluation. Table 4 and Table 5 summarize the results of Task A1 and Task A2, respectively. CDFM achieves strong performance on both tasks, demonstrating that a model pretrained entirely on synthetic causal mechanisms can transfer to real physical systems and demonstrate robustness under diverse causal mechanisms. Tübingen cause-effect pairs. We further evaluate CDFM on the Tübingen cause-effect pairs benchmark, which contains heterogeneous real-world bivariate datasets collected from multiple scientific domains. We use 95 causal pairs for strictly bivariate tasks. 14 CAM CPT Disc.ANM Linear Lin-Het Lin-Ord MeasErr Physical Phys-Het PNL RFF RFF-Het RFF-Ord Rounded TimeLag CDFM TabCausal Arrow AVICI DirectLiNGAM GES PC NOTEARS 0.860.770.880.940.930.860.880.830.870.900.950.940.860.880.85 0.790.500.620.930.900.690.830.760.740.810.920.900.730.790.76 0.700.530.710.700.690.650.700.650.670.630.740.740.680.680.72 0.770.590.660.900.880.620.810.730.690.800.900.910.650.770.73 0.690.610.670.920.900.690.720.640.610.630.740.650.660.670.70 0.660.600.700.730.720.650.660.630.620.650.690.650.650.640.66 0.710.720.780.760.740.710.730.680.690.710.750.730.740.710.69 0.550.630.570.570.580.530.570.570.570.560.600.610.570.560.61 0.5 0.6 0.7 0.8 0.9 1.0 AUROC Figure 6. Method×mechanism AUROC heatmap. Rows correspond to methods, columns to mechanism families. CDFM (top row) achieves the highest AUROC across all 15 mechanisms. Table 4. Causal Chamber Task A1 results for multivariate DAG recovery. MethodAUROC↑ F 1 ↑ Precision↑ Recall↑ SHD↓ CDFM0.952 0.7270.8890.61517 TabCausal0.9300.6030.7920.48725 DirectLiNGAM0.7750.4950.4030.64144 GES0.6990.4260.5910.33335 PC0.5680.2810.3600.23143 Arrow0.8480.2550.4380.18040 NOTEARS0.3800.0950.0570.282159 AVICI0.4230.0000.0000.00041 For each pair, CDFM receives only two observed variables and predicts the causal direction. Table 6 shows that CDFM achieves competitive performance with existing state-of-the-art methods. The results demonstrate that the learned causal representation transfers beyond synthetic environments and remains effective across diverse real-world domains. 6.4 Empirical Validation of Identifiability We next conduct controlled diagnostic experiments to examine whether CDFM can learn to examine whether CDFM behaves consistently with theoretical identifiability boundaries. If the training data spans identifiable causal mechanisms, the model should learn to distinguish causal directions where identifiability holds, and should produce near-uniform predictions where it does not. In this section, we empirically validate these claims through a series of controlled experiments. Moreover, to ensure CDFM has been exposed to the full range of diagnostic tasks, we will fine-tune CDFM on these specific tasks. Functional Asymmetry. We first explore functional asymmetry in identifiability. We consider a family of bivariate additive-noise formulationy=x+b·x 3 +n, whereb∈[−1,1] controls the degree of nonlinearity and q ∈[0.5,2.0] controls the non-Gaussianity of the noise distribution via the transformationv 7→ sign(v)·|v| q applied independently to both the causex∼N(0,1) and the noisen∼N(0,1). Atb= 0,q= 1, the model reduces to a linear Gaussian SCM, which is theoretically non-identifiable in both directions. Our goal is to see whether the foundation also adopts the theoretical results. 15 Table 5. Causal Chamber Task A2 results. The predictions from 20 separately processed environments are averaged. MethodAUROC↑ F 1 ↑ Precision↑ Recall↑ SHD↓ CDFM0.965 0.7270.8890.61518 TabCausal0.9390.6350.8330.51323 GES0.9060.6330.9050.48722 DirectLiNGAM0.8130.3490.2260.76982 PC0.6160.2500.4120.18040 Arrow0.7970.1700.5000.10339 NOTEARS0.4460.1270.0730.513188 AVICI0.5030.0000.0000.00040 Table 6. Tübingen results on 95 bivariate cause-effect pairs. MethodAccuracy↑ CDFM0.671 TabCausal0.638 Arrow0.602 AVICI0.488 DirectLiNGAM0.508 NOTEARS0.401 To ensure CDFM has been exposed to the full range of (b,q) configurations, we fine-tune the pretrained model on bivariate ANM data, samplingb∼U[−1,1] andq ∼U[0.5,2.0] independently per batch, with sample sizes N ∼U [100, 500]. Results. Figure 7 presents the empirical findings on non-Gaussianity and nonlinearity. Figure 7a shows accuracy as a function of the non-Gaussianity parameterqatb= 0 (linear SCM). Atq= 1.0 (Gaussian noise), the model achieves an accuracy around 0.5, corresponding to chance-level performance, confirming that CDFM still obeys the theoretical results in the linear Gaussian case. Asqmoves away from 1, accuracy rises sharply to 1.00 for both sub-Gaussian and super-Gaussian regimes. Figure 7b demonstrates the effect of nonlinearity atq= 1 (Gaussian noise). Atb= 0.0 (purely linear), accuracy again remains at around 0.5, while a slight nonlinearity (|b|≥0.1) suffices for identification (0.97–1.00). These results align with the theoretical results of the nonlinear additive noise model [11]. Identifiability Through V-Structure. Under the causal Markov and faithfulness assumptions, observa- tional conditional-independence relations generally identify a Markov equivalence class rather than a unique DAG, while unshielded colliders (also known as V-structures) provide compelled edge orientations, which offer an additional source of asymmetry beyond the functional asymmetry. Therefore, we further investigate whether CDFM can leverage causal information implied by conditional independence constraints. To do so, we exclude additional directional information from functional asymmetry by conducting this experiment exclusively on linear Gaussian structural equation models. Therefore, any identifiable orientation must arise from conditional-independence constraints. Then, we exhaustively enumerate all non-isomorphic DAGs ond= 5 under sample size = 100 and group them according to their actual number of V-structures. We evaluate theF 1 score over CDFM, PC, and the Theoretical Upper Bound, which uses perfect d-separation on the true DAG, and the undetermined edges will be randomly oriented, providing a fairer comparison. To ensure the theoretical boundary is practically detectable, CDFM is fine-tuned on synthetic datasets generated from the same linear Gaussian model family. 16 0.40.60.81.01.21.41.61.82.0 Non-Gaussianity parameter q 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Direction accuracy Gaussian: n(0, 1) (non-identifiable) Identifiability vs Non-Gaussianity (b = 0, linear SCM) CDFM q=1 (Gaussian) (a) Non-Gaussianity parameterqfor linear SCMs (b= 0). 1.000.750.500.250.000.250.500.751.00 Nonlinearity parameter b 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Direction accuracy Linear: y = x + n (non-identifiable) Identifiability vs Nonlinearity (q = 1, Gaussian noise) CDFM b=0 (linear) (b) Nonlinearity parameterbwith Gaussian noise (q= 1). 0123 Number of V-structures in DAG 0.0 0.2 0.4 0.6 0.8 1.0 F1 Score CDFM PC Theoretical Upper Bound (c) F 1 score vs. number of V-structures (d = 5). 0.00.20.40.60.81.0 Non-Gaussianity noise = (1)(0, 1) +( 3 , 3 ) 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy XY (Confounded by L) XY (Confounded by L) XY (Confounded by L) (d) Latent confounding identifiability. Figure 7. Empirical validation of identifiability. (a) Effect of non-Gaussianity on causal direction identification for linear SCMs. (b) Effect of nonlinearity under Gaussian noise. (c)F 1 score as a function of the number of V-structures, compared against PC and the theoretical upper bound. (d) Three-class accuracy under latent confounding as a function of non-Gaussianity. Results. Figure 7c presents the results ofF 1 against the number of V-structures. CDFM’s orientation performance improves as the number of collider structures increases, while PC exhibits the same qualitative tendency but is additionally affected by finite-sample errors in its conditional-independence tests. Moreover, CDFM approaches the theoretical upper bound, empirically confirming the theoretical identifiability boundary, showing that CDFM remains subject to classical causal identifiability theory. Latent Confounding Identifiability. Finally, we examine whether CDFM can distinguish among three distinct causal scenarios under latent confounding: (i) no direct edge betweenXandY, with only a latent confounderLcausing both (X ← L→ Y); (i)X → Ywith confounding; and (i)Y → Xwith confounding. The model observes only the bivariate pair (X,Y) and must classify the underlying relationship. For synthetic data, we follow the functional asymmetry with linear causal relationshipY=aX+nwhere we control the non-Gaussianity of the noise distribution via varying q by the transformation v 7→ sign(v)·|v| q . Figure 7d presents the 3-class classification accuracy for each scenario as a function ofq. Two notable patterns emerge. First, the no-edge scenario (pure confounding) is best identified in the sub-Gaussian regime (q= 0.5, accuracy 0.86), where the distributional signature of a shared latent cause is most distinct from a direct causal edge. Accuracy degrades asqapproaches 1.0 (Gaussian, accuracy 0.64) and recovers partially for super-Gaussian noise. Second, the directed-edge scenarios (X → YandY → X) achieve their best accuracy at extreme super-Gaussian values (q= 2.0, accuracy 0.66), suggesting that heavy-tailed non-Gaussianity 17 Table 7. Missing value imputation MAE↓. CDFM vs. eight classical imputation methods on continuous (RFF) and discrete (CPT) synthetic data. ContinuousDiscrete CDFM0.452±0.164 0.717±0.102 MissForest0.475±0.188 0.748±0.095 MICE0.500±0.163 0.760±0.097 SoftImpute0.505±0.150 0.834±0.117 KNN (k=5)0.573±0.184 0.780±0.095 MatrixFactorization 0.694±0.198 0.990±0.179 Median0.720±0.069 0.721±0.100 Mean0.727±0.067 0.779±0.093 IterativeSVD0.732±0.302 1.217±0.315 provides the strongest signal for distinguishing the causal direction under confounding. Overall, these results demonstrate that CDFM exploits non-Gaussian distributional signatures to distinguish the three simulated relation types, which is consistent with classic identifiability theory. These experiments collectively suggest that the theoretical boundary of CDFM is consistent with the classical identifiability theory. 6.5 Auxiliary Analysis of Missing Value Imputation CDFM includes a quantile-reconstruction head to provide a self-supervised signal for learning conditional structure. Therefore, as a by-product of this work, we evaluate the imputation capability of CDFM on both continuous and discrete data. We consider two synthetic mechanisms: continuous data generated by the RFF mechanism, which uses nonlinear additive noise with random Fourier features, and discrete data generated by the CPT mechanism, which uses conditional probability tables with values in0,1,2,3. For each mechanism, we use synthetic datasets, spanningD ∈[5,50] variables,N ∈ 500,1000,2000samples, and 10 random seeds for each configuration. Missing values are generated under an MCAR scheme with missing rates in 0.1, 0.2, 0.3. CDFM performs imputation in a single forward pass using its built-in imputation head. Specifically, we extract the median quantile from its 21-level quantile prediction. We compare CDFM against eight standard imputation methods: (1) Mean and (2) Median column-wise imputation; (3) KNN (k=5) [23,42]; (4) MICE with BayesianRidge regression for 10 iterations [43]; (5) MissForest [39] using RandomForest with 50 trees and 5 iterations; (6) SoftImpute, (7) IterativeSVD [32], and (8) MatrixFactorization [32]. All methods are evaluated using mean absolute error (MAE) between the imputed and ground-truth values at the masked positions. Table 7 reports the MAE of all methods on both continuous (RFF) and discrete (CPT) data. CDFM one-shot imputation achieves the best overall performance in both regimes, with an MAE of 0.452 on continuous data and 0.717 on discrete data. Matrix-factorization-based methods (SoftImpute, IterativeSVD, MatrixFactorization) perform poorly on discrete data, as their low-rank assumptions are violated by the categorical structure. Across missing rates, CDFM remains the top method for both regimes, with its advantage over competitors widening as the missing rate increases from 0.1 to 0.3. 7 Conclusions In this work, we introduce CDFM, a causal foundation model that reframes causal discovery as inference over uncertain mechanisms rather than optimization under a single assumed causal model. By integrating mechanism inference, structural prediction, and reconstruction-based representation learning, CDFM learns transferable causal representations from diverse synthetic causal environments. 18 Table 8. Imputation MAE by missing rate. CDFM maintains its advantage across all missing rates on both continuous and discrete data. ContinuousDiscrete Method0.10.20.30.10.20.3 CDFM0.436±0.164 0.450±0.165 0.469±0.1620.712±0.104 0.717±0.101 0.721±0.100 MissForest0.456±0.185 0.474±0.188 0.496±0.1890.735±0.095 0.747±0.093 0.763±0.096 IterativeImputer0.463±0.162 0.502±0.161 0.536±0.158 0.752±0.097 0.754±0.095 0.775±0.098 SoftImpute0.487±0.153 0.505±0.151 0.524±0.1460.823±0.113 0.835±0.117 0.845±0.120 KNN (k=5)0.541±0.192 0.572±0.184 0.605±0.1720.770±0.094 0.782±0.096 0.788±0.095 MatrixFactorization 0.674±0.202 0.689±0.197 0.716±0.193 0.984±0.185 0.990±0.181 0.997±0.172 Median0.718±0.069 0.721±0.069 0.720±0.0680.722±0.101 0.721±0.099 0.720±0.099 Mean0.726±0.068 0.728±0.068 0.728±0.0660.779±0.093 0.780±0.094 0.778±0.092 IterativeSVD0.703±0.302 0.730±0.306 0.762±0.2981.232±0.350 1.212±0.309 1.208±0.283 Extensive experiments demonstrate that CDFM achieves strong generalization across heterogeneous synthetic mechanisms, graph structures, and sample sizes, while maintaining competitive performance on real-world causal benchmarks, including physical causal systems and heterogeneous cause-effect datasets. Controlled identifiability experiments further show that CDFM follows theoretically meaningful causal behavior: it exploits available functional and graphical asymmetries while remaining uncertain in observationally non-identifiable regimes. These findings suggest a promising direction for causal discovery: rather than designing increasingly specialized algorithms for individual assumptions, future causal models may benefit from learning broad representations over families of causal mechanisms. CDFM represents an initial step toward such general-purpose causal reasoning systems. References [1]Y. Annadani, S. Bauer, A. Foster, and P. Tigas. Amortized active causal induction with deep reinforcement learning. InAdvancesinNeuralInformationProcessingSystems(NeurIPS2024), 2024. [2]V. Balazadeh Meresht, H. Kamkari, V. Thomas, J. Ma, B. Li, J. Cresswell, and R. Krishnan. Causalpfn: Amortized causal effect estimation via in-context learning. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors,AdvancesinNeuralInformationProcessingSystems, volume 38, pages 154945–154984. Curran Associates, Inc., 2025. [3]P. Bühlmann, J. Peters, and J. Ernest. CAM: Causal additive models, high-dimensional order search and penalized regression.TheAnnalsofStatistics, 42(6):2526–2556, 2014. [4]R. Cai, J. Qiao, K. Zhang, Z. Zhang, and Z. Hao. Causal discovery from discrete data using hidden compact representation. InAdvancesinNeuralInformationProcessingSystems, volume 31, 2018. [5]R. Cai, Z. Zhang, Z. Hao, and M. Winslett. Understanding social causalities behind human action sequences. IEEETransactionsonNeuralNetworksandLearningSystems, 28(8):1801–1813, 2017. [6]D. M. Chickering. Optimal structure identification with greedy search.JournalofMachineLearningResearch, 3:507–554, 2002. [7] A. E. Gamal and Y.-H. Kim.NetworkInformationTheory. Cambridge University Press, Cambridge, U.K., 2011. [8]J. L. Gamella, J. Peters, and P. Bühlmann. Causal chambers as a real-world physical testbed for ai methodology. NatureMachineIntelligence, 7(1):107–118, 2025. [9] E. Ghysels, J. B. Hill, and K. Motegi. Testing for granger causality with mixed frequency data.Journalof Econometrics, 192(1):207–230, 2016. [10]N. Hollmann, S. Müller, L. Purucker, A. Krishnakumar, M. Körfer, S. B. Hoo, R. T. Schirrmeister, and F. Hutter. Accurate predictions on small data with a tabular foundation model. 637(8045):319–326. 19 [11]P. O. Hoyer, D. Janzing, J. M. Mooij, J. Peters, and B. Schölkopf. Nonlinear causal discovery with additive noise models. InAdvancesinNeuralInformationProcessingSystems, volume 21, pages 689–696, 2009. [12] A. Immer, C. Schultheiss, J. E. Vogt, B. Schölkopf, P. Bühlmann, and A. Marx. On the identifiability and estimation of causal location-scale noise models. InInternationalConferenceonMachineLearning, pages 14316– 14332. PMLR, 2023. [13]N. R. Ke, S. Chiappa, J. Wang, A. Goyal, J. Bornschein, M. Rey, T. Weber, M. Botvinick, M. Mozer, and D. J. Rezende. Learning to induce causal structure. InInternationalConferenceonLearningRepresentations, 2023. [14] J. Lee, Y. Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y. W. Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. InProceedingsofthe36thInternationalConferenceonMachineLearning, volume 97 ofProceedingsofMachineLearningResearch, pages 3744–3753. PMLR, 2019. [15]J. Leiner, B. Manzo, A. Ramdas, and W. Tansey. Scalable causal structure learning via amortized conditional independence testing. InProceedingsoftheFourthConferenceonCausalLearningandReasoning, volume 275 ofProceedingsofMachineLearningResearch, pages 174–200. PMLR, 07–09 May 2025. [16]Z.-R. Li, S.-Y. Liu, T.-Z. Wang, and H.-J. Ye. Tabcausal: Pretraining across causal environments for tabular causal discovery.arXivpreprintarXiv:2605.31156, 2026. [17]L. Lorch, S. Sussex, J. Rothfuss, A. Krause, and B. Schölkopf. Amortized inference for causal structure learning. InAdvancesinNeuralInformationProcessingSystems, volume 35, 2022. [18]S. Löwe, D. Madras, R. Zemel, and M. Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. In B. Schölkopf, C. Uhler, and K. Zhang, editors,ProceedingsoftheFirstConference onCausalLearningandReasoning, volume 177 ofProceedingsofMachineLearningResearch, pages 509–525. PMLR, 11–13 Apr 2022. [19] Y. Ma, D. Frauen, E. Javurek, and S. Feuerriegel. Foundation models for causal inference via prior-data fitted networks. InTheFourteenthInternationalConferenceonLearningRepresentations, 2026. [20] J. M. Mooij, J. Peters, D. Janzing, J. Zscheischler, and B. Schölkopf. Distinguishing cause from effect using observational data: methods and benchmarks.JournalofMachineLearningResearch, 17(32):1–102, 2016. [21] Y. Ni and B. Mallick. Ordinal causal discovery. InProceedingsoftheThirty-EighthConferenceonUncertaintyin ArtificialIntelligence, volume 180 ofProceedingsofMachineLearningResearch, pages 1530–1540. PMLR, 2022. [22] J. Pearl. An introduction to causal inference.Theinternationaljournalofbiostatistics, 6(2):7, 2010. [23] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al. Scikit-learn: Machine learning in Python.JournalofMachineLearningResearch, 12:2825–2830, 2011. [24]B. Peng, S. Chen, J. Tian, Y. Qiao, and C. Lu. Causcale: Neural causal discovery at scale.arXivpreprint arXiv:2602.08629, 2026. [25]J. Peters, D. Janzing, and B. Schölkopf. Identifying cause and effect on discrete data using additive noise models. InProceedingsoftheThirteenthInternationalConferenceonArtificialIntelligenceandStatistics, volume 9 of ProceedingsofMachineLearningResearch, pages 597–604. PMLR, 2010. [26]J. Peters, D. Janzing, and B. Scholkopf.Elementsofcausalinference:foundationsandlearningalgorithms. MIT press, 2017. [27]J. Qiao, R. Cai, S. Wu, Y. Xiang, K. Zhang, and Z. Hao. Structural hawkes processes for learning causal structure from discrete-time event sequences. InProceedingsoftheThirty-SecondInternationalJointConference onArtificialIntelligence, 2023. [28] J. Qiao, Y. Xiang, Z. Chen, R. Cai, and Z. Hao. Causal discovery from poisson branching structural causal model using high-order cumulant with path analysis. InProceedingsoftheThirty-SixthAAAIConferenceonArtificial Intelligence(AAAI2024), 2024. [29] J. Qiao, Z. Zeng, R. Cai, Z. Chen, and Z. Hao. On the Identifiability of Poisson Branching Structural Causal Model Under Latent Confounding. InProceedingsoftheForty-Third International ConferenceonMachine Learning InternationalConferenceonMachineLearning(ICML-2026), 2026. 20 [30]J. Qu, D. Holzmüller, G. Varoquaux, and M. Le Morvan. TabICL: A tabular foundation model for in-context learning on large data. InInternationalConferenceonMachineLearning, pages 50817–50847. PMLR. [31] J. Robertson, A. Reuter, S. Guo, N. Hollmann, F. Hutter, and B. Schölkopf. Do-pfn: In-context learning for causal effect estimation. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors, AdvancesinNeuralInformationProcessingSystems, volume 38, pages 174811–174848. Curran Associates, Inc., 2025. [32]A. Rubinsteyn and S. Feldman. fancyimpute: A variety of matrix completion and imputation algorithms implemented in python. Version 0.7.0. [33] J. Runge, X.-A. Tibau, M. Bruhns, J. Muñoz-Marí, and G. Camps-Valls. The causality for climate competition. InNeurIPS2019CompetitionandDemonstrationTrack, pages 110–120. Pmlr, 2020. [34]K. Sachs, O. Perez, D. Pe’er, D. A. Lauffenburger, and G. P. Nolan. Causal protein-signaling networks derived from multiparameter single-cell data.Science, 308(5721):523–529, 2005. [35] V. Satopaa, J. Albrecht, D. Irwin, and B. Raghavan. Finding a Kneedle in a haystack: Detecting knee points in system behavior. In201131stInternationalConferenceonDistributedComputingSystemsWorkshops, pages 166–171. IEEE, 2011. [36]S. Shimizu, P. O. Hoyer, A. Hyvärinen, and A. Kerminen. A linear non-gaussian acyclic model for causal discovery. JournalofMachineLearningResearch, 7:2003–2030, 2006. [37]S. Shimizu, T. Inazumi, Y. Sogawa, A. Hyvärinen, Y. Kawahara, T. Washio, P. O. Hoyer, and K. Bollen. DirectLiNGAM: A direct method for learning a linear non-gaussian structural equation model.JournalofMachine LearningResearch, 12:1225–1248, 2011. [38]P. Spirtes, C. Glymour, and R. Scheines.Causation,Prediction,andSearch. MIT Press, Cambridge, MA, 2 edition, 2000. [39] D. J. Stekhoven and P. Bühlmann. Missforest—non-parametric missing value imputation for mixed-type data. Bioinformatics, 28(1):112–118, 2012. [40] O. Swelam, L. Purucker, J. Robertson, H. Raum, J. Boedecker, and F. Hutter. Does tabPFN understand causal structures? InEurIPS2025Workshop:AIforTabularData, 2025. [41] R. Thompson, H. Zhao, D. M. Steinberg, and E. V. Bonilla. Arrow: A foundation model for causal discovery. arXivpreprintarXiv:2605.07204, 2026. [42] O. Troyanskaya, M. Cantor, G. Sherlock, P. Brown, T. Hastie, R. Tibshirani, D. Botstein, and R. B. Altman. Missing value estimation methods for DNA microarrays.Bioinformatics, 17(6):520–525, 2001. [43]S. van Buuren and K. Groothuis-Oudshoorn. mice: Multivariate imputation by chained equations in R.Journal ofStatisticalSoftware, 45(3):1–67, 2011. [44]M. Wu, B. Qiang, W. Shi, Y. Song, et al. Sample, estimate, aggregate: A recipe for causal discovery foundation models.TransactionsonMachineLearningResearch, 2025. Published March 2025. [45] Y. Xiang, J. Qiao, Z. Liang, Z. Zeng, R. Cai, and Z. Hao. On the identifiability of poisson branching structural causal model using probability generating function. InAdvancesinNeural Information Processing Systems, volume 37, pages 11664–11699, 2024. [46]R. Yao, T. Verdonck, and J. Raymaekers. Causal discovery in mixed additive noise models. InProceedingsof The28thInternationalConferenceonArtificialIntelligenceandStatistics, volume 258 ofProceedingsofMachine LearningResearch, pages 3088–3096. PMLR, 2025. [47] N. Yin, T. Gao, and Y. Yu. Learning causal graphs at scale: A foundation model approach. 2025. [48] N. Yin, T. Gao, Y. Yu, and Q. Ji. Effective causal discovery under identifiable heteroscedastic noise model. In ProceedingsoftheAAAIConferenceonArtificialIntelligence, volume 38, pages 16486–16494, 2024. [49]K. Zhang and A. Hyvärinen. On the identifiability of the post-nonlinear causal model. InProceedingsofthe Twenty-FifthConferenceonUncertaintyinArtificialIntelligence, pages 647–655. AUAI Press, 2009. 21 [50]X. Zhang, G. Ren, H. Yu, H. Yuan, H. Wang, J. Li, J. Wu, L. Mo, L. Mao, M. Hao, et al. Limix: Unleashing structured-data modeling capability for generalist intelligence.arXivpreprintarXiv:2509.03505, 2025. [51] X. Zheng, B. Aragam, P. K. Ravikumar, and E. P. Xing. DAGs with NO TEARS: Continuous optimization for structure learning. InAdvancesinNeuralInformationProcessingSystems, volume 31, 2018. 22 A Proofs and Derivation A.1 Derivation of the Variational ELBO To approximate the intractable marginal likelihood where the mechanismMis integrated out, we introduce a variational distributionQ(M|X) over the mechanism spaceM. The joint log-probabilitylogP(G,X) can be expanded and bounded as follows: logP (G,X) = log Z M P (G,M,X)dM(19) = log Z M P (G,X|M )P (M )dM(20) = log Z M P (G,X|M ) P (M ) Q(M|X) Q(M|X)dM(21) ≥ E Q(M|X) log P (G,X|M ) P (M ) Q(M|X) (by Jensen’s Inequality)(22) = E Q(M|X) logP (G,X|M ) − E Q(M|X) log Q(M|X) P (M ) (23) = E Q(M|X) logP (X|G,M ) + logP (G|M ) − D KL Q(M|X)∥ P (M ) .(24) This completes the derivation of Eq. (6) in the main text. A.2 Proof of Theorem 1 Proof. To provide an intuitive construction, we first assume that X and Y are continuous random variables. We will demonstrate that a valid Structural Causal Model can always be constructed for the causal direction X → Y . LetF Y|X (y|x) denote the conditional cumulative distribution function ofYgivenX. According to the inverse transform sampling theorem, there exists an auxiliary noise variableN Y ∼U[0,1] that is independent ofX, such thatY=F −1 Y|X (N Y | X). Therefore, by defining the structural function asf Y (X,N Y ) :=F −1 Y|X (N Y | X), we obtainY=f Y (X,N Y ) withN Y ⊥ X. Hence, a valid SCM corresponding to the causal directionX → Y exists. Applying the same argument to the reverse direction, there exists an independent noise variableN X ∼U[0,1] such thatX=F −1 X|Y (N X | Y). Equivalently, definingf X (Y,N X ) :=F −1 X|Y (N X | Y), we obtainX=f X (Y,N X ) whereN X ⊥ Y. Therefore, for any continuous joint distributionP X,Y , valid SCMs can be constructed for both causal directions X → Y and Y → X. For more general cases, including discrete or mixed variables, the Functional Representation Lemma [7] guarantees the existence of such independent uniform noise variables and measurable structural functions, yielding the exact same result. Consequently, without additional assumptions on the causal mechanisms, the causal direction is inherently unidentifiable from observational data alone. A.3 Proof of Theorem 2 Proof. By definition, the expected log-likelihood of a graphGunder the true data-generating distribution is L(G) = E X∼P(X|G ∗ ,M ∗ ) [logP (X|G,M ∗ )]. 23 Therefore, the difference between the expected log-likelihood of the true graph and that of an arbitrary alternative graph G ′ is L(G ∗ )−L(G ′ ) = E X∼P(X|G ∗ ,M ∗ ) [logP (X|G ∗ ,M ∗ )− logP (X|G ′ ,M ∗ )] = E X∼P(X|G ∗ ,M ∗ ) log P (X|G ∗ ,M ∗ ) P (X|G ′ ,M ∗ ) = D KL [P (X|G ∗ ,M ∗ )∥ P (X|G ′ ,M ∗ )]. Since the Kullback–Leibler divergence is always non-negative, we have D KL [P (X|G ∗ ,M ∗ )∥ P (X|G ′ ,M ∗ )]≥ 0, which proves that L(G ∗ )≥L(G ′ ). To prove the strictness of the inequality, suppose that there exists a graphG ′ ̸=G ∗ such thatL(G ∗ ) =L(G ′ ). This implies,P(X|G ∗ ,M ∗ ) =P(X|G ′ ,M ∗ ). Therefore, the two distinct graphsG ∗ andG ′ induce the same observational distribution under the known mechanismM ∗ . However, this contradicts the assumption that M ∗ belongs to a strictly identifiable mechanism class, under which each graph corresponds to a unique induced distribution. Hence, no graph G ′ ̸= G ∗ can achieve the same expected log-likelihood as G ∗ . Therefore, the equality holds if and only if G ′ = G ∗ . A.4 Proof of Theorem 3 To show the identifiability under the broad hypothesis space of causal mechanismsM, we will need to first define the strict identifiability property from the set of causal prior mechanisms. Formally, we say the space of causal prior mechanisms is strictly identifiable ifP(X|M ∗ ,G ∗ )̸=P(X|M G ′ ,G ′ ) holds for all alternative graphsG ′ ̸=G ∗ , whereM ∗ andG ∗ denote the true causal mechanism and true causal structure, respectively, andM G ′ =arg max M∈M E X [logP(X|M,G ′ )] represents the optimal mechanism choice underG ′ . This formulation serves as a direct extension from the traditional known causal mechanism setting, further suggesting that the entire space of causal prior mechanisms must possess the intrinsic property of mutual exclusivity. Under this structural foundation, the exact marginal likelihood uniquely isolates the true causal graph. Proof. Let X (N) =x 1 ,x 2 ,...,x N be a dataset ofNindependent and identically distributed (i.i.d.) samples drawn from the true causal distributionP ∗ (x) =P(x|M ∗ ,G ∗ ). For any candidate graphG, the exact marginal likelihood of the dataset is given by the integral over the mechanism space M: P (X (N) |G) = Z M P (X (N) |M,G)P (M|G)dM = Z M N Y i=1 P (x i |M,G) ! P (M|G)dM.(25) We evaluate the scaled log marginal likelihood asN →∞. Following standard asymptotic Bayesian theory, the integral is asymptotically dominated by the maximum likelihood estimate over the parameter space, and we have: lim N→∞ 1 N logP (X (N) |G) = sup M∈M E X [logP (X|M,G)], a.s.(26) We now evaluate Eq.(26)under two distinct structural scenarios. In case 1, the candidate graph is the true graphG=G ∗ . Since the true data-generating distribution is contained in the model class underG ∗ , the supremum is attained by any mechanism inducing P (X| M ∗ ,G ∗ ). By definition, we have sup M∈M E X [logP (X|M,G ∗ )] = E X [logP (X|M ∗ ,G ∗ )]. 24 In case 2, an alternative candidate graphG=G ′ ̸=G ∗ . LetM G ′ ∈M be the optimal mechanism that the alternative graph topology can possibly choose to fit the data, and we have lim N→∞ 1 N logP (X (N) |G ′ ) = E X [logP (X|M G ′ ,G ′ )].(27) We then examine the difference between the two asymptotic limits: ∆L = E X [logP (X|M ∗ ,G ∗ )]− E X [logP (X|M G ′ ,G ′ )] = E X log P (X|M ∗ ,G ∗ ) P (X|M G ′ ,G ′ ) = D KL P (X|M ∗ ,G ∗ )∥ P (X|M G ′ ,G ′ ) . (28) Since the mechanism class M is strictly identifiable, we have: P (X|M G ′ ,G ′ )̸= P (X|M ∗ ,G ∗ ),(29) guaranteeing that the KL divergence is strictly positive: D KL P (X|M ∗ ,G ∗ )∥ P (X|M G ′ ,G ′ ) > 0.(30) Consequently, we conclude that: lim N→∞ 1 N logP (X (N) |G ∗ ) > lim N→∞ 1 N logP (X (N) |G ′ ), a.s., ∀G ′ ̸= G ∗ .(31) This completes the proof that the exact marginal likelihood uniquely identifies the true causal graphG ∗ almost surely as the sample size approaches infinity. A.5 Proof of Corollary 1 Proof.LetJ(G;X (N) ) :=E Q ∗ (M|X (N) ) logP(G| M) denotes the structural inference in CDFM. We prove that its maximizer converges almost surely to the true graphG ∗ andQ ∗ denotes the optimum variational posterior obtained by pretraining. We begin with the connection between the structural inference and the joint likelihoodlogP(X (N) ,G ), which can be expressed by, logP (X (N) ,G) =L(G,X (N) ;Q) + D KL Q(M | X (N) )∥ P (M | G,X (N) ) , where L(G,X;Q) = E Q(M|X) logP (X| G,M ) + logP (G| M ) − D KL Q(M | X)∥ P (M ) is the fitted ELBO of CDFM given in Eq. (6): Then, given that the foundation model has sufficient capacity to reach the optimum variational ELBO, we obtain logP (X (N) ,G) =L(G,X (N) ;Q ∗ ),(32) where the optimal variational posterior Q ∗ (M | X (N) ) approaches P (M | G,X (N) ). Thus, since logP (X (N) ,G) = logP (X (N) | G) + logP (G),(33) and under a uniform graph prior P (G), we have arg max G logP (X (N) ,G) = arg max G logP (G| X (N) ) = arg max G L(G,X (N) ;Q ∗ ).(34) 25 Subsequently, we establish the connection betweenarg max G J(G;X (N) ) andarg max G L(G,X (N) ;Q ∗ ). First, by Theorem 3, strict identifiability of the mechanism space implies that the exact marginal likelihood uniquely identifies the true graph: lim N→∞ 1 N logP (X (N) | G ∗ ) > lim N→∞ 1 N logP (X (N) | G ′ ), ∀G ′ ̸= G ∗ , a.s.(35) Since the graph prior P (G) is uniform, Bayes’ rule gives P (G| X (N) )∝ P (X (N) | G). Therefore, for any G ′ ̸= G ∗ , P (G ′ | X (N) ) P (G ∗ | X (N) ) = P (X (N) | G ′ ) P (X (N) | G ∗ ) . (36) Taking logarithms and dividing byN, Eq.(36)implies that this ratio converges to zero almost surely. Hence P (G ∗ | X (N) )→ 1, P (G ′ | X (N) )→ 0, ∀G ′ ̸= G ∗ , a.s.(37) Thus, the posterior distribution over graphs converges almost surely to a Dirac measure concentrated atG ∗ . Then, we have P (M | X (N) ) = X G∈G P (M | G,X (N) )P (G| X (N) ) = P (M | G ∗ ,X (N) )P (G ∗ | X (N) ) + X G ′ ̸=G ∗ P (M | G ′ ,X (N) )P (G ′ | X (N) ). (38) By Eq.(37), the first posterior weight converges to one and all remaining posterior weights vanish almost surely, and we obtain P (M | G ∗ ,X (N) ) a.s. ∼ P (M | X (N) ), N →∞.(39) Therefore, since the optimal variational posteriorQ ∗ (M |X (N) ) approach toP(M | G,X (N) ), we have Q ∗ (M |X (N) ) a.s. ∼ P(M |X (N) ). This justifies using the amortized encoderQ(M |X (N) ) in place of the true graph-conditioned mechanism posterior asymptotically. Asymptotically, the structural inference simplifies toJ(G;X (N) ) =E P(M|X (N) ) [logP(G|M)]. By applying Jensen’s inequality, we have: J (G;X (N) ) = E P(M|X (N) ) logP (G| M ) ≤ log E P(M|X (N) ) P (G| M ) = logP (G| X (N) ). For the true graphG ∗ , the posteriorP(G ∗ |X (N) )→1, which implieslogP(G ∗ |X (N) )→0, and thus, J(G ∗ ;X (N) )→0. Thus, forG ∗ , both sides of Jensen’s inequality approach zero, making the bound asymptotically tight: J (G;X (N) ) = logP (G| X (N) ).(40) Since the uniform prior ensuresarg max G logP(G |X (N) ) =arg max G logP(X (N) | G ), and Theorem 3 guarantees the marginal likelihood is strictly optimum at G ∗ , it follows that: ˆ G = arg max G∈G J (G;X (N) ) = G ∗ , a.s. as N →∞. This completes the proof. 26 B Synthetic Mechanism Abbreviations Table A1. Abbreviations of synthetic mechanism families used in Table 3. AbbreviationDescription CAMCausal additive model CPTConditional probability table for discrete variables Discrete ANM Discrete additive noise model LinearLinear structural equation model Linear-HeteroLinear model with heteroscedastic noise Linear-Ordinal Linear model with ordinal or discretized observations Meas. ErrorMechanism with additional measurement error PhysicalPhysics-inspired nonlinear mechanism Physical-Hetero Physical mechanism with heteroscedastic noise PNLPost-nonlinear causal model RFFRandom Fourier feature nonlinear mechanism RFF-HeteroRFF mechanism with heteroscedastic noise RFF-OrdinalRFF mechanism with ordinal or discretized observations RoundedMechanism with rounded observations Time-LagTime-lagged causal mechanism 27

Instant research discovery