Paper deep dive

Symmetries and Causality: Causal Effect Identification Beyond IID Data

Martin Rabel, Jakob Runge

Year: 2026Venue: arXiv preprintArea: math.STType: PreprintEmbeddings: 477

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 9/4/2026, 3:44:30 AM

Summary

This paper proposes a formal mathematical framework for causal reasoning that extends beyond Independent and Identically Distributed (IID) data assumptions. By utilizing measure-theoretic probability kernels and group actions to define symmetries and invariance of causal mechanisms, the authors provide a unified language for identifying causal effects in complex systems, including time-series and multi-context data, without relying on traditional random variable formulations or strict IID constraints.

Entities (10)

Symmetries and Causality: Causal Effect Identification Beyond IID Data · paper · 100%Causal Inference · field · 95%IID Data · concept · 95%Jakob Runge · person · 95%Martin Rabel · person · 95%University of Potsdam · organization · 95%do-calculus · method · 90%Group Actions · concept · 90%Probability Kernels · concept · 90%Structural Causal Models · method · 90%

Relation Signals (9)

Martin Rabel → affiliatedwith → University of Potsdam

confidence 95% · aDepartment of Computer Science, University of Potsdam

Jakob Runge → affiliatedwith → University of Potsdam

confidence 95% · aDepartment of Computer Science, University of Potsdam

Jakob Runge → authorof → Symmetries and Causality: Causal Effect Identification Beyond IID Data

confidence 95% · Martin Rabel Jakob Runge ... Abstract

Martin Rabel → authorof → Symmetries and Causality: Causal Effect Identification Beyond IID Data

confidence 95% · Martin Rabel Jakob Runge ... Abstract

Symmetries and Causality: Causal Effect Identification Beyond IID Data → extends → Causal Inference

confidence 95% · unify and substantially extend the scope of causal reasoning, in going beyond IID data

Symmetries and Causality: Causal Effect Identification Beyond IID Data → generalizes → Structural Causal Models

confidence 90% · reproduces and matches standard theoretical results... going beyond IID data

Symmetries and Causality: Causal Effect Identification Beyond IID Data → proposesmethod → Probability Kernels

confidence 90% · We propose a formal description of statistical systems based on symmetries in data... Invariant mechanisms in our formalism are probability kernels

→ →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:In the natural sciences, symmetries and cause-effect relationships are ubiquitous. Yet for complex machine-learning tasks, like world-modeling in reinforcement learning, they appear difficult to harness. We propose a formal description of statistical systems based on symmetries in data leaving causal mechanisms invariant. The result is an abstract, simple and general mathematical language for causal reasoning. This paper provides formal descriptions of models and queries, setting up this language, and the formal infrastructure and strategies for their mathematically rigorous identification from data within this formalism. This approach reproduces and matches standard theoretical results on IID data and transport of experimental and non-experimental data. But its main purpose is to unify and substantially extend the scope of causal reasoning, in going beyond IID data and in approaching complex causal queries not captured by do- or soft-interventions. This new perspective on causally relevant aspects of data-modeling additionally sheds new light on well-known structures like c-components or hedges but also includes aspects of missing data and is inherently well-suited for the description of transfer and robustness properties.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

473,650 characters extracted from source content.

Expand or collapse full text

Symmetries and Causality: Causal Effect Identification Beyond IID Data Martin Rabel Jakob Runge September 3, 2026 Abstract In the natural sciences, symmetries and cause–effect relationships are ubiquitous. Yet for complex machine-learning tasks, like world-modeling in reinforcement learning, they appear difficult to harness. We propose a formal description of statistical systems based on symmetries in data leaving causal mechanisms invariant. The result is an abstract, simple and general formal language for causal reasoning. This paper provides formal descriptions of models and queries, setting up this language, and the formal infrastructure and strategies for their mathematically rigorous identification from data within this formalism. This approach reproduces and matches standard theoretical results on IID data and transport of experimental and non-experimental data. But its main purpose is to unify and substantially extend the scope of causal reasoning, in going beyond IID data and in approaching complex causal queries not captured by do- or soft-interventions. This new perspective on causally relevant aspects of data-modeling additionally sheds new light on well-known structures like c-components or hedges but also includes aspects of missing data and is inherently well-suited for the description of transfer and robustness properties. aDepartment of Computer Science, University of Potsdam, Potsdam, Germany 1 Introduction The scientific method of exploring and understanding our reality seems to naturally evoke a notion of cause and effect in our reasoning. This alone motivates causal machine learning methods for their interpretability and often claimed robustness. But there is also a much more fundamental argument. The core problem in realizing sophisticated AI is in the requirement to describe, understand, explore and reason about a complex world. There is one well-known approach that seems so far to have been rather successful at this task: Human science. We may not know why causality arises in scientific study, yet this evidence suggests a deep role of causal reasoning in efficient world modeling. This evokes another question: Where are these amazing causality driven AIs? Causality in general, as motivating the above claims, is a rather vaguely defined concept. Certain formalisms, like the do-operator and graphical-models [30] or potential outcomes [46; 18], have had tremendous success in capturing some aspects of causality in some systems. But at the same time, for most more complex real-world problems, it seems at first impossible to cast them into a form where such frameworks apply (usually something close to IID with few but sufficient variables), and then it seems similarly impossible to answer our questions – seemingly causal and intuitive – from the formal output provided (usually something close to do-interventions). The main purpose of this paper is to make progress on these problems: Applicability of causal language to complex problems, and the extraction of answers to complex questions. Improving expressiveness, and thereby bringing the notion of causality actually captured by the formalism closer to the vague concept of causality that motivates its use in the first place, will aspirationally also advance its capabilities towards what has often been envisioned in terms of applied science use-cases and machine learning theory. Many approaches to broaden the scope of causal inference have been made before. From the incorporation of experimental data and transportability between contexts [6; 33] for effect estimation, structure learning on multi-context data [19; 27; 15] or regimes and patterns [52; 39]. Also more complex queries like soft-interventions [11], interventions in time-series [51], program-evaluation [30, §4.4] or mediation questions [32] and many more have been studied. Indeed these approaches have brought many invaluable theoretical and practical insights. But at the same time, they seem to largely happen in isolation. We may be able to predict certain soft-interventions, and predict do-interventions from experimental data, but already the prediction of anything from data obtained doing a "soft-experiment" would require additional technology. We may be able to combine IID data-sets [6; 19; 27], but for time-series data we suddenly need new ideas [15]. Of course any such combination could be solved individually, but this is not a satisfactory answer – at some point, there are simply to many combinations. What is needed is not just expressiveness, but also a shared language, that can incorporate and combine all (or at least many) such ideas uniformly. ⋯·sXXZZYYL1L_1 XXZZYYL1L_1 XXZZYYL1L_1 XXZZYYL1L_1 XXZZYYL1L_1 ⋯·sLearn:XXYYL1L_1X2X_2Z2Z_2L2L_2 X2X_2Z2Z_2L2L_2 X2X_2Z2Z_2L2L_2 X2X_2Z2Z_2L2L_2 X2X_2Z2Z_2L2L_2 ⋯·sX doX_ doZ2Z_2L2L_2 X doX_ doZ2Z_2L2L_2 X doX_ doZ2Z_2L2L_2 X doX_ doZ2Z_2L2L_2 X doX_ doZ2Z_2L2L_2 ⋯·sLearn:Z2Z_2L2L_2Predict:Y initY_ initXXZ2Z_2YYL2L_2L1L_1 XXZ2Z_2YYL2L_2L1L_1 XXZ2Z_2YYL2L_2L1L_1 XXZ2Z_2YYL2L_2L1L_1 Figure 1: Consider three given data-sets (green solid – observed, gray dashed – latent): A three-periodic univariate time-series (top), repeated IID observations (middle), and repeated experiment (bottom) evaluating for example an upgraded (modular) replacement part in a technical system. Labels (capital letters) are by mechanism, not variables. A plausible question is, what happens if we replace the mechanism Z by Z2Z_2 (in the time-series setup), and, starting from some known initial state, have the modified system run for a number, say four, periods to the final value (blue circle). The intuitive solution to this problem is rather clear: there are things that can be learned from the different data-sets, and this knowledge can be pieced together to answer the query. For specific (and simple) examples it might be possible to hand-craft a solution, and maybe even to convince oneself of its soundness, yet – while clearly practically relevant – questions of such flexible type are not captured by conventional formalisms of causality. 1.1 Need for Abstraction One conceptual idea that practically all generalizing approaches seem to share is that of adding definitional structure to model and query: Add selection-variables [6; 33] (often called context-variables) to grow the system into a "meta"-system [27], add new types of interventions, to describe queries that are not do-interventions, add proxies for hidden structure [19; 15]. Combining multiple formalisms will be expensive, already because it requires a compatible rephrasing of added structures. Our approach advances in the opposite direction, toward a more abstract model. Instead of adding yet more definitional structure, the basic question we set out to ask is: What structure can be removed form model and query? Generalization to a specific problem is only required, if there is something in the way to modeling it in the first place. A more abstract formalism applies more generally simply by virtue of having less constraints on what problems can be readily described. It is of course of crucial importance that the simplified structure still retains the ability to describe the causal aspects of the problem. It is certainly always possible to remove any formal structure, if one is willing to give up on the ability to rigorously reason about or automate a problem at hand – indeed it has sometimes been claimed that causal inference in general were unnecessary as it only translates causal problems to statistical ones, a task that could also be achieved manually; it is not this trivial type of removal of structure we aim for. The first removed structural aspect baked into the description by structured causal models (SCMs) or potential outcomes (POs) is IIDness of the data. Complex questions in applied science involve combination of data from, and prediction for, different environments, sub-scale experiments etc., thus inherently cannot be phrased in IID-setups. More generally, the ubiquity of IID setups in statistics is based on the idea that a researcher curates suitable data-sets; so the modeling of how to obtain such data-sets is inherently out of scope for IID approaches. Also from the advanced AI perspective, an ML-agent that requires curated data-sets is not autonomous. Note that for example mz-transport [6; 33] or JCI [27] have to add selection- -variables to remodel a problem by an IID meta-SCM – additional structure that will no longer be required, once IIDness is no longer enforced by the underlying formalism. The second removed structural aspect, rooted similarly deep in existing formalisms, is randomness; SCMs and POs are formulated through random variables. Removing randomness from the model and query does not in any way impede the ability to treat stochastic data or questions. Indeed the perspective is similar to the one commonly taken on time-series, which are given by a random process, but under suitable conditions can be modeled by Markov-kernels – certain probability-kernels, which are purely measure-theoretical (do not reference a basic probability space (Ω,,P)( ,A,P) or random elements). Writing a model in measure-theoretic rather than probability-theoretic terms may seem, at first sight, like technical minutiae much more than conceptual paradigm-shift. Yet this innocently looking detail turns out to be crucial to enable a rigorous, clear and surprisingly simple and intuitive description of causal reasoning. 1.2 Employed Structure We base our model-structure on a concept that is already pervasive in causal study: Invariance of mechanisms. While formally somewhat elusive, invariance of mechanisms has played a pivotal role in causal inference in general [30; 38], for discovering causal structure [19; 27; 15; 37] and for robust prediction or transfer [44; 45; 6; 33]. Invariance of mechanisms is also baked into the SCM and PO frameworks, and while usually not made explicit has certainly played a core role in their motivation; for example Pearl [30, p. 322] states (slightly paraphrased) that "what we call ’causal knowledge’" are the "extra assumptions that identify what in the distribution remains invariant when the specified modification takes place". Invariance is the property of remaining unchanged under a given symmetry, the symmetry Pearl seems to have in mind is the symmetry switching mechanisms of observed and intervened model – indeed this is the form of symmetry we will need to reproduce the results of do-calculus as a special case (§E). Invariant mechanisms in our formalism are probability kernels – combining an ambiguous pair of a structural mapping and a noise-distribution into a single unambiguous object; this is (see above) inspired by the use of Markov-kernels in the process literature (see [22, §7]). Additionally we explicitly attach a symmetry and range of applicability. A single model can describe both the observed world an the query uniformly, indeed it makes precise the idea eluded to above about invariance under switching mechanisms of observed and intervened model: both observed and intervened world share a model, with unintervened mechanisms invariant under the symmetry switching between worlds. Some kernels are considered known a priori (conventionally called "interventions") while others can be extracted, individually or only as part of larger confounded structures, from data by virtue of their symmetry properties. Of course statistics, at its very foundations, is about learning shared stochastic properties by combining many observations; so relating invariant structures built from mechanisms to suitable subsets of data is not fundamentally a new idea – however its consistent, rigorous and explicit execution to capture causality is novel. So we elevate a weak notion of invariance of mechanisms – weak enough to be implicit in SCM and PO frameworks, that is without adding definitional structure – to become the fundamental formal starting point of causal modeling. After establishing a canonical link to probabilistic observations (including randomness), there is nothing more required to capture a meaningful notion of causality – a notion that in the special case of IID data and do-interventional queries reproduces the results of SCM and PO frameworks. 1.3 Contributions and Content An important reason why selection- -variables have been so successful [6; 33; 19; 27] is in their ability to map a new problem to an established setup – thus much of the required technology to work on the resulting mapped problem was already established. The proposed formalism here is rather a simpler (more low-level) language. Many ideas of modeling problems can be compiled into this simpler (and unified) language and thus remain intact; such ideas are among the core contributions of the existing frameworks. However, getting information back out of a modeled system is inherently more difficult as there is less definitional structure to work with. So, for example, identification-strategies like the Id-algorithm [61; 57] or mz-transport [5; 4] or even the do-calculus [30] do not carry over. The primary contribution of this paper is to provide the formal machinery to re-establish the link from modeled problem-statement to identification-strategy. Constructions on simple structures are difficult. For example group-theory is much more difficult than linear algebra. Our models are quite simple, so it should not come as a surprise, that substantial work will be required to obtain results. Importantly we can keep the problem tractable by dividing it into three logically independent parts – largely by virtue of having deleted randomness from the model. These parts are the description and transformation of with knowledge about a model; the extraction of knowledge from observations; and the phrasing of causal queries as questions about what knowledge needs to be computed from what was extracted. After a formal definition of our modeling approach §2, these points also dictate the structure of this paper, with §3 introducing structure, §4 extracting it from data, and §5 finally answering causal queries. We do not discuss selection bias or correlated missingness. We also do not focus on completeness of the proposed strategy – it is complete for IID-data and likely for mz-transport, see §5.4 – instead we formulate transparent conjectures 1, 2, 3, 4 that should clarify what questions concerning completeness remain open. We provide algorithmic descriptions, to elucidate what systematic approach an applied researcher might take to write down a specific identification strategy, and to refine and explain the formal structures by enforcing a certain degree of tractability. While our approach is amenable to automation, immediate implementation in code is not the primary goal of the presented algorithms. 1.4 Related Literature Some of the cornerstone results of causal inference in multi-context setups, like transportability of experimental and non-experiment information [5; 4; 6; 33], causal discovery by context-variable and similar approaches [19; 27] and by invariant prediction [37] were already discussed above. Our results are about what would traditionally be called identification of interventional distributions, thus among these are closest to transportability [4] in terms of goals (comparable results are available in potential outcome language, see [26]). There are also many results for identification of interventional distributions in IID-data [61; 57; 56] building on Pearls’s do-calculus (see [30]). We encounter analogues for many of the structures like c-components [61], c-trees, c-forests [57] and ultimately for do-interventional queries also hedges [57]. A comprehensive comparison to the single-context IID-case [57; 56] and transportability [4] is given in §E. Known results for the IID-case also include counter-factual queries [58; 54]. We do not discuss counter-factuals specifically, but coincidentally our flexible query-formulation captures some mediation questions (natural direct effects) recovering known formulas [31], see §E.8. More abstract formulations for causality have been proposed [29] to clarify the relation of causality to probability. Here, we primarily abstract to gain flexibility for non-IID applications and complex queries which leads to a mostly orthogonal (detached from IID and random elements) result; our primary focus is on re-establishing a pathway to identification of complex queries in non-IID systems from observations, the abstraction is guided by (and subordinate to) this goal. We also want to mention the perspective of [21] of causality as the ability to predict unobserved joint distributions, which is philosophically close to our approach but focused on IID observations and statistical learning, and has very interesting consequences for analyzing finite-sample reliability of CD-methods [13]. We do not investigate the discovery of the structures defining our models. Causal structure discovery (CD) results beyond the IID-case [59; 9; 60] exist from time-series [50; 48] to for example multi-context data [19; 27], proxies of hidden confounders [19; 15], for context-specific structure [16; 40], see also [35; 10; 20], or for learning both context-structure and graphical structure [52; 41; 3; 25; 39], or learning differences only [1]. Learning more flexible structure seems initially more difficult, but such weaker structure is also more likely to exist (which is nice for anything one wants to find). Further, the usefulness of non-IIDness for improving causal discovery [19; 27; 37], especially edge-orientations, suggests that a principled approach focused on invariance in modeling may be helpful to extracting information. Symmetries and invariance also play an important role in modern probability theory (for a great overview, see [23]), the causal questions we ask sometimes touch on trivial cases of deep results like de-Finetti’s theorem (exchangeable is conditional IID; cf. Rmk. E.17 or example D.13), but the intricacies of such characterizations (like conditional IIDness) and other topics commonly studied in probability-theory, while interesting problems of their own right, seem mostly disjoint from the causal questions we are interested in. Also in machine-learning, especially for parameter-sharing, symmetries are of great relevance, for example in convolution-models (image-convolution, but also graph-convolutional networks etc.) or in attention-mechanisms (for example in transformers), see any textbook on the topic, [28]. Their use to guide the transfer to causal queries and the systematic structuring of world-models and -exploration seems to be largely unexplored however, even though toy-models like "causal bandits" [24] have received substantial attention lately. Causal models are believed to take a special role for invariant prediction, as justified for example by results like [43; 45] (and references therein). Finally, our results (Cor. 3.9) have some unexpected connections to simple missing-data problems [47]. A causal perspective on missing-data problems can be found for example in [12] and references therein. 2 Models and Observations We introduce the basic formalism, consisting of models and their relation to statistical observations, including observedness and asymptotic limits. 2.1 Preliminaries We are interested in causal properties, not in technical intricacies of measure-theory, hence assume throughout this paper (see also §G): Assumption 2.1. Measurable spaces (X,ℬ⁡(X))(X,B(X)) are standard Borel (Polish spaces X with their associated Borel σ-algebra ℬ⁡(X)B(X)). Examples for Polish spaces are ^n (with standard topology) and open or closed subsets of ^n (with the induced topology), sets with the discrete topology. We describe causal models based on probability kernels. We give a preliminary definition and notation here and discuss technical details in §G. Definition 2.2 (Kernels, preliminary). Given measurable spaces (S,ℬ⁡(S))(S,B(S)) and (T,ℬ⁡(T))(T,B(T)), a measurable mapping f:S→⁡(ℬ⁡(T))f:S (B(T)) into the probability measures on (T,ℬ⁡(T))(T,B(T)) is called a probability kernel. Symmetry is an abstract concept, which we formally express via group-actions. Group-actions are ubiquitous in the natural sciences and in many cases intuitive. More abstract formal descriptions are possible §G.5. Definition 2.3 (Symmetry). Group-Actions: A (left) group action G↷IG I on a set I is a mapping ⋅:G×I→I,(g,i)↦g⋅i·:G× I→ I,(g,i) g· i such that ∀i∈I:e⋅i=i∀ i∈ I:e· i=i and ∀g,h∈G:∀i∈I:(g∗h)⋅i=g⋅(h⋅i)∀ g,h∈ G:∀ i∈ I:(g*h)· i=g·(h· i). Given a sub-group H⊂GH⊂ G, there is an induced action ⋅H:H×I→I,(h,i)↦h⋅Hi:=iH(h)⋅i·_H:H× I→ I,(h,i) h·_Hi:=i_H(h)· i. Properties: A group action G↷IG I is called effective, if (∀i∈I:g⋅i=i)⇒g=e(∀ i∈ I:g· i=i) g=e, called free if ∀i∈I:(g⋅i=i⇒g=e)∀ i∈ I:(g· i=i g=e), and called transitive if ∀i,i′∈I:∃g∈G:i′=g⋅i∀ i,i ∈ I:∃ g∈ G:i =g· i. Given I0⊂I_0⊂ I we will call an action free on I0I_0 if the corresponding condition holds ∀i,i′∈I0∀ i,i ∈ I_0. An orbit is a set G⋅i=i′∈I|∃g∈G:g⋅i=i′G· i=\i ∈ I|∃ g∈ G:g· i=i \, the set of orbits is I/G IG. Invariance : A mapping f:I→Zf:I→ Z is called G↷IG I invariant, if ∀i∈I:∀g∈G:f⁡(g⋅i)=f⁡(i)∀ i∈ I:∀ g∈ G:f(g· i)=f(i). A mapping f′:I→I′f :I→ I is called (G↷IG I, G↷I′G I ) equivariant, if f′(g⋅i)=g⋅f′(i)f (g· i)=g· f (i). Notation: We fix a group G and a effective group action G↷IG I. A symmetry is denoted as (induced action of) a subgroup H⊂GH⊂ G. 2.2 Model Our models are purely measure-theoretic constructions (no (Ω,,P)( ,A,P) or random elements appear). Other than in the IID-setting, we cannot presuppose structure on the index-set (indexing the data), instead we only ask for: Notation 2.4. We fix a countable index-set I, and measurable spaces (Xi,ℬ⁡(Xi))i∈I\(X_i,B(X_i))\_i∈ I that are standard Borel (Ass. 2.1). Given J⊂IJ⊂ I, we denote XJ:=∏j∈JXjX_J:= _j∈ JX_j. Our models will specify (invariant) mechanisms and their symmetries. Definition 2.5 (Mechanism). A mechanism is a probability kernel f from X =∏k=1κX(k)X = _k=1^κX^(k) (with κ arguments) to Y (both standard Borel, Ass. 2.1) together with a region of applicability (a subset) J⊂IJ⊂ I, a symmetry H, acting freely and transitively on J, and a H-equivariant mapping :J→Iκ :J→ I^κ of relative parents such that: Denoting the kkth parent by k k, require ∀k: k(j)≠j∀ k: k(j)≠ j and k≠k′k≠ k ⇒ ∀j∈J∀ j∈ J: k(j)≠ k′(j) k(j)≠ k (j). Finally, ∀j∈J∀ j∈ J, Y⊂XjY⊂ X_j and ∀k∀ k: X k(j)⊂X(k)X_ k(j)⊂ X^(k). Definition 2.6 (Model). A model ℳ=(fJ,J,HJ,…)J∈M=\(f_J,J,H_J,…)\_J is a collection of mechanisms, which we index by their region of applicability J, such that I is the disjoint union of regions of applicability I=∪J∈JI= _J J and J≠J′∈⇒J∩J′=∅J≠ J J∩ J = . Mechanisms are in one of two (disjoint) categories: Either fJf_J is considered initially unknown fJ∈ℱ obsf_J _ obs or known f~J∈ℱ intervene f_J _ intervene ("interventions" transfer external, fixed knowledge into the model). Every i∈Ii∈ I is by construction contained in exactly one J∈J which we denote by J⁡(i)J(i) and we write fi:=fJ⁡(i)f_i:=f_J(i). Example 2.7 (IID-Models from SCMs). Given an SCM on variables indexed by v∈I varsv∈ I_ vars with parent-sets _v SCM-mechanisms gvg_v and noise-distributions P⁡(ηv)P( _v), there is an associated model ℳM on I=I vars×I sampleI=I_ vars× I_ sample with G=I sampleG= S_I_ sample the permutations of I sampleI_ sample (with π∈I sampleπ∈ S_I_ sample acting as π⋅(v,s)=(v,π⁡(s))π·(v,s)=(v,π(s))) with mechanisms fv( ):=gv( ,⋅)∗P(ηv)f_v( _v):=g_v( _v,·)_*P( _v) (the distribution of V (ω):=gv( ,ηv(ω))V_ _v(ω):=g_v( _v, _v(ω))), Hv=GH_v=G, Jv=v×I sampleJ_v=\v\× I_ sample and for j=(v,s)∈Jvj=(v,s)∈ J_v, (j)= ×s _v(j)= _v×\s\. An intervention on an SCM replaces an SCM-mechanism gvg_v by a known function (for example a constant value for do-interventions), so an interventional SCM M for (X=x) (X=x) (where X⊂I varsX⊂ I_ vars) induces a model ℳ as above, but mechanisms associated to v∈Xv∈ X are known: ∀v∈X:fv∈ℱ intervene∀ v∈ X:f_v _ intervene (for do-interventions these fv∈ℱ intervenef_v _ intervene are singular measures without parents, thus (j)=∅ _v(j)= ). We will not need interventional models however, instead we can just define a unified model on I unified=I⊔I unified=I I (we return here in example 5.1), this dramatically simplifies the description of multi-context or experimental-data settings. See also Fig. 6 or Fig. 8. Further details are given in §E. Example 2.8 (Timeseries-Models). Given a time-series SCM on variables indexed by v∈I varsv∈ I_ vars with parent-sets ⊂I vars×−τ,…,0 _v⊂ I_ vars×\-τ,…,0\, where τ is some maximal lag, and again SCM-mechanisms gvg_v and noise-distributions P⁡(ηv)P( _v), given an initial state ν→ ν (for example a stationary distribution if one exists) for τ+1τ+1 time-steps, there is a model ℳM with I=I vars×TI=I_ vars× T, where T=ℕ0T=N_0, with the first τ steps fixed to ν→∈ℱ intervene ν _ intervene and later ones with time-translation invariant (Hv=G=ℤH_v=G=Z, with a⋅(v,t):=(v,t+a)a·(v,t):=(v,t+a) for a∈Ga∈ G; usually called time-homogeneous [22], sometimes "causally stationary") mechanisms fv( ):=gv( ,⋅)∗P(ηv)f_v( _v):=g_v( _v,·)_*P( _v). Note that HvH_v-equivariance of :J→Iκv _v:J→ I _v amounts to fixing pairs (v,Δt)(v, t) of a variable and a lag, automatically matching the intuition and hand-crafted form of a time-series SCM. The choice of an initial state is made explicit in our formalism. While sometimes obfuscated a little (usually by defaulting to the use of a stationary distribution), this cannot be avoided: In cases where no or multiple stationary distributions exist, this choice cannot be made implicitly. Even in cases where a unique stationary distribution does exist, there seems to be little reason to needlessly restrict the formalism to this particular choice, as other initial states may make practical sense. 2.3 Shallow Distribution and Realization We fix a model ℳM. Via relative parents, ℳM defines a single large graph on I. Definition 2.9 (I-Graph). The I-graph GI G_I has nodes I and an edge i→i′i→ i if i∈ (i′)i∈ _I(i ), where (i′):=i∈I|∃k:i= ⁡(i′)(i′) _I(i ):=\i∈ I|∃ k:i= k_J(i )(i )\. We assume the I-graph is acyclic, as we want independent mechanisms. In principle, indices contained in a finite cycle can be replaced by a single index (consistent with Ass. 2.1, which is closed under finite products). Assumption 2.10 (Acyclic I-Graph). We assume that the I-graph is acyclic, and fix a total order πI _I. For formal simplicity (to avoid issues with non-unique solvability of equations and filtrations of σ-algebras), we assume the modeled past is finite. Assumption 2.11 (Finite Past). For all i∈Ii∈ I, | (i)|<∞| _I(i)|<∞ is finite. Remark 2.12 (Boundary Construction). It is possible, rather generally, to construct a "boundary" ( initial state for a time-series -process) that translates infinite past models to finite past models, see example 2.8. Thus this assumption is primarily a statement about decomposition of the problem, where we discuss the causal aspects and detach the logically distinct question of choice of (non-unique) solutions into the choice of a (non-unique) initial state; see §D.4. Then, the model ℳM defines a distribution, jointly over XiX_i associated to i∈Ii∈ I by composing mechanisms. The result is (using notation from §3) given essentially by Pθ=⊗πIifJ⁡(i)P_θ= _I_if_J(i), we can avoid defining infinite ⊗ -products by the following formulation: Lemma 2.13 (Shallow Distribution). Given acyclicity (Ass. 2.10) and finite past (Ass. 2.11), there is a probability measure PθP_θ on ∏i∈IXi _i∈ IX_i, which we will call the shallow distribution, parametrized by θ=ℱ interveneθ=F_ intervene, such that the marginalization to any finite I′⊂I ⊂ I satisfies for all measurable Bi∈ℬ⁡(Xi)B_i (X_i) the following characterizing property: Pθ((∏i′∈I′Bi′)×(∏i∈I∖I′Xi))=[⊗i′∈ (I′)πIfJ⁡(i′)]((∏i′∈I′Bi′)×(∏i∈ (I′)∖IXi)) .P_θ ( ( _i ∈ I B_i )× ( _i∈ I I X_i ) )= [ _I_i ∈ _I(I )f_J(i ) ] ( ( _i ∈ I B_i )× ( _i∈ _I(I ) IX_i ) ) . While the model is not probabilistic, the observable world it describes is: Definition 2.14 (Observable World). An observable world realizing the model ℳM is a family of random variables Vi:Ω→Xii∈I\ V_i: → X_i\_i∈ I, together with measurable maps fi: (i)×[0,1]→Xif_i:X_ _I(i)×[0,1]→ X_i and jointly independent random variables ηi∼U⁡([0,1]) _i U([0,1]) called noises, such that Vi=fi(V (i),ηi)andP⁡(Vii∈I)=Pθ . V_i=f_i( V_ _I(i), _i) P(\ V_i\_i∈ I)=P_θ . Mandated by the second equality, we will often simply write Pθ(Vii∈I)P_θ(\ V_i\_i∈ I). Remark 2.15 (Shallowness). The reason for calling PθP_θ "shallow" is conceptually important: The way we model our data, for each i∈Ii∈ I, there is at most one observation. Thus each random variable Vi V_i is observed at most once. If we had more observations, we would instead extend I and enhance the model’s symmetry to capture this repetition. So PθP_θ, while formally well-defined, does not not allow for a statistical analysis without additional use of the model’s symmetries. It becomes, in this language, an important aspect of identifiability, to ensure that any statistical analysis of the data must be able to draw from an invariant subset of data that becomes large as N→∞N→∞, thus is "deep" as opposed to "shallow". Lemma 2.16 (Observable World Existence). Given acyclicity (Ass. 2.10) and finite past (Ass. 2.11), an observable world exists, it is unique up to equality in distribution and it is locally Markov, VK| (i) V_i V_K| _I(i) for any K⊂IK⊂ I with K∩ (i)=∅K∩ _I(i)= . Example 2.17 (IID-Models from SCMs). Continuing example 2.7, which given an SCM M constructed an associated model ℳM. For ℳM and ∀s∈I sample∀ s∈ I_ sample, Pθ(Vv,sv∈I vars)=P obsP_θ(\ V_v,s\_v∈ I_ vars)=P obs, where P obsP obs is the observed distribution of M. In particular the random model described by M is an observable world of ℳM. Similarly, for ℳ , Pθ(Vv,sv∈Y)=P⁡(Y| (X=x))P_θ(\ V_v,s\_v∈ Y)=P(Y| (X=x)). 2.4 Observedness and Asymptotics So far, we have no notion of observed vs. hidden variables. We use a simple and flexible definition. Definition 2.18 (Viewport). Given a model, a viewport is given by an infinite subset ⊂ℕN together with a mapping V:→ (I), N↦V⁡(N) V:N→ (I), N V(N) suchthat|V⁡(N)|=N and suchthat| V(N)|=N and N≤N′∈⇒V⁡(N)⊂V⁡(N′) . N≤ N V(N)⊂ V(N ) . We write :=∪∈V(N)⊂I V:= _N V(N)⊂ I. Assumption 2.19 (Independent Missingness). The viewport is not a random object and thus implicitly independent of values taken by variables in the observable world. Example 2.20. Consider an IID model with three variables I vars=x,y,lI_ vars=\x,y,l\, where x and y are always observed, while l is never observed. In this case we can choose :=2ℕN:=2N the even numbers (we could try to say what is observed first, x or y, but we do not have to, this is why ≠ℕN is allowed). Then for N=2n∈N=2n , define V⁡(N)=x,y×1,…,n⊂I vars×ℕ V(N)=\x,y\×\1,…,n\⊂ I_ vars×N (we replaced I sampleI_ sample by the natural numbers here) to capture the described missingness pattern. In the asymptotic limit n→∞n→∞ we observe =,×ℕ⊂ V=\x,y\×N⊂ I. It only remains to combine observable worlds and viewports, and to formally encode the assumption that each variable of an observable world is observed at most once (Rmk. 2.15). Definition 2.21 (Realized World). Given an observable world and a viewport, we call a sample N(ω)=Vi(ω)i∈V⁡(N)D^N(ω)= V_i(ω)_i∈ V(N) a realized data-set and for =∪N(ω)D= _ND^N(ω) define Pθ:=Pθ((Vi)i∈I|(Vi)i∈=)P_θ^D:=P_θ(( V_i)_i∈ I|( V_i)_i∈ V=D) a realized world (distribution). Remark 2.22. If we ask a question about the realized world PθP_θ^D, we ask a question about PθP_θ with each i∈i∈ V fixed to a value. Thus we input exactly one value ("observation") for each element of the viewport (additional to the structure of the shallow distribution provided by the model and its symmetries). So the realized world is a formal means of asking questions under the restriction of only one observation per variable. Intuitively, we cannot generally extrapolate to arguments we have never observed. The choice of the formal assumption, is explained in detail in Rmk. A.6; this assumption is usually not made explicit, but formally necessary and implicitly made in other approaches. Assumption 2.23 (Valid Support for Transfer). We assume that queries and other computations on kernels identified from observations remain within the the observational support in the sense that any disintegration νx _x of a observed joint distribution μ⊗νxμ _x is only transferred to (evaluated in) products ξ⊗νxξ _x where ξ≪μξ μ is dominated by μ ( ∀ measurable B: μ⁡(B)=0⇒ξ⁡(B)=0μ(B)=0 ξ(B)=0), or the result can for practical purposes ( regularity assumption like smoothness of densities) be interpreted as if this were the case. 3 Structured Kernels We need formal objects to capture pieces of knowledge that will be obtained from observations in §4. Further, we need tools to transform them in a mathematically rigorous yet practically applicable calculus. 3.1 Preliminaries Probability kernels are in wide spread use especially in probability theory and time-series statistics. We provide a very brief structure-oriented overview of standard constructs here and a more detailed discussion connecting this language to SCMs in §G.2. μνμ⊗νμ ν⊗ → _→marg← _←(ν∘μ)⊗(μ|ν)(ν μ) (μ|ν)X×Y→Y×X× Y→ Y× X,(x,y)↦(y,x)(x,y) (y,x). Figure 2: Basic kernel operations. Helpful analogues are: μ μ ↔P⁡(X|Z=z) P(X|Z=z) ν ν ↔P⁡(Y|X=x,Z=z) P(Y|X=x,Z=z) μ⊗ν μ ν ↔P⁡(X,Y|Z=z) P(X,Y|Z=z) Some basic operations on kernels (illustrated in Fig. 2) are: ⊗ -products, given kernels μ from Z to X and ν from Z×XZ× X to Y, construct a kernel μ⊗νμ ν from Z to X×YX× Y; note the causal ordering (asymmetry in μ↔νμ ν). Marginalizations (on the right) drop a term at the end (in causal order), disintegrations (on the left) turn a term at the beginning (in the causal order) into an argument. These expressions can sometimes be thought of as conditional distributions (if a suitable joint distribution including Z exists). Reordering terms, for example switching (x,y)↦(y,x)(x,y) (y,x) is a Borel-isomorphism – given P⁡(X,Y)P(X,Y) we know what P⁡(Y,X)P(Y,X) is. We can marginalize after switching to get a composition ν∘μν μ of kernels, "hiding" a cause X of Y does not change the distribution of Y, it still depends on the actual distribution of X. Similar by disintegration after switching we get an "anti-causal" disintegration (μ|ν)(μ|ν). With ξ:=μ⊗νξ:=μ ν being a kernel itself, operations like ξ⊗ϑ=[μ⊗ν]⊗ϑξ =[μ ν] , and thus larger products are immediately defined (and turn out to be associative). It is however not immediately obvious, how reordering terms, marginalizations (on the right) and disintegrations (on the left) commute. In analogy to "P⁡(X,Y)=P⁡(Y|X)P(X)P(X,Y)=P(Y|X)P(X)", which holds for tuples -variate X, Y regardless of causal ordering, arbitrary marginalizations and disintegrations can be defined with these characterizing (defining) properties: Lemma 3.1 (Order Independent Operations). Given μ=μ1⊗…⊗μnμ= _1 … _n together with L⊂1,…,nL⊂\1,…,n\ and C=1,…,n∖LC=\1,…,n\ L, there are unique (Rmk. A.6) kernels margL⁡(μ) _L(μ) and disintC⁡(μ) _C(μ) such that ∀ measurable BkC∈ℬ⁡(Xk)B^C_k (X_k) where BkC=XkB^C_k=X_k if k∈Lk∈ L: margL⁡(μ)(∏k∈CBi) _L(μ)( _k∈ CB_i) =μ⁡(∏k=1nBkC) =μ( _k=1^nB^C_k) margL⁡(μ)⊗disintC⁡(μ) _L(μ) _C(μ) =μ (uptoreordering). =μ (uptoreordering). These operations all produce suitably unique results. Combining finitely many such operations again produces a unique result. We call such (finite) computations with these operations regular (functionals). For generic kernels (without known internal structure like linearity), there do not seem to be any more well-defined (finite; excluding limits and fixed-points) computations, thus, for completeness considerations, we are inclined to consider: Conjecture 1 (Completeness of Regular Computation). If a functional uniquely computes a result from finitely many generic kernels then it is regular (Def. A.8, Rmk. A.11). 3.2 Causality and Sparsity So far, ⊗ -products were defined with shared arguments in Z plus the value-space of the left-hand-side as additional argument to the right-hand-side. To define a product of many terms in "causal order", meaning any arguments that depend on another kernels value are such that this parent is to the left, it is of course possible to trivially extend the argument space to include all ancestors (including all shared ones, but without actual dependence on non-parents); the product can then be defined successively from left to right. However causal reasoning oftentimes relies precisely on the sparsity of parents among the ancestors. As is common practice, both for tensor networks and causal modeling [30], we encode this sparsity in a attached (to the trivially constructed product, see above) graph object. Technical details and proofs are given in §A. We usually need not be concerned by reordered results (in the sense explained above), but we will need a notion of hidden variables. Further (shared) arguments (inputs) for kernels are fundamentally different from their outputs. Definition 3.2 (Structural Graph). A structural graph G is given by a finite set of nodes := inner∪˙ outerN:=N_ inner\, ∪\,N_ outer with inner∩ outer=∅N_ inner _ outer= , together with a set of directed edges ℰ⊂× innerE ×N_ inner, such that edges never point to an element of outerN_ outer. Alignment: A model aligned structural graph is an acyclic (no directed cycles) structural graph G together with a probability kernel μnμ^n for each inner node n∈ innern _ inner and a bijective assignment (cf. G.24) of the κ arguments of μnμ^n to parents of n: :1,2,…,κ→1:1 (n) . ^n:\1,2,…,κ\ 1:1 _G(n) . Value-spaces of parents are subspaces of a respective factor of the domain of their children. The mappings ^n tell us how to "wire up" each kernel: Reorder the (inductively obtained) product over ancestors such that (1),…, (κ) ^n(1),…, ^n(κ) are the κ first terms, then trivially "pad" the kernel μnμ^n (add trivial dependence on all other non-parent ancestors) and write down the standard product. Thereby we obtain a structured (by a model-aligned graph) kernel: Definition 3.3 (Structured Kernel). Given a tuple (G,L)(G,L), called a structural model below, of a model aligned structural graph G together with a subset L⊂ innerL _ inner, we define μ(G,L)x := (⊗n∈ inner wiredμn)x ,μ(G,L)_x := _L( _n _ inner wiredμ^n)_x , where x=(xi)i∈ outerx=(x_i)_i _ outer is the tuple of shared arguments in outerN_ outer. A ℱF-structured kernel is a kernel μx _x that can be written in the form μx=μ(G,L)x _x=μ(G,L)_x with ∀n∈ inner:μn∈ℱ∀ n _ inner:μ^n . Finally, the next subsection will require a formal notion of sub-graphs: Definition 3.4 (Structural Subgraphs). A structural subgraph GA≤GBG^A≤ G^B of a structural graph GBG^B is a structural graph GAG^A such that: [label=()] 1. Node Sets: A⊂BN^A ^B and innerA⊂ innerBN_ inner^A _ inner^B. 2. Edge Sets: edges are exactly those in GBG^B from nodes in AN^A to inner nodes of innerAN_ inner^A. 3. Inner Parents: ∀n∈ innerA∀ n _ inner^A: if p∈ (n)p∈ _G^B(n), then p∈Ap ^A. 4. Alignment: for model-aligned graphs ∀n∈ innerA∀ n _ inner^A: (μA)n=(μB)n(μ^A)^n=(μ^B)^n. 3.3 Graphical Operations WWZZXXYYL2L_2L1L_1↝ -componentsWWXXYYL2L_2L1L_1+ZZWWL1L_1L2L_2↝ -fication+XXL1L_1YYL2L_2↝ -componentsXXL1L_1+YYL2L_2XXL1L_1YYL2L_2X~ soft X_ soft↝ Figure 3: Examples for graphical operations. All shown structured kernels can be computed from Pearl’s "napkin"-graph (top-left) and the (know) soft-intervention X~ soft X_ soft (a single inner node plus a single outer node for its argument). Inner nodes are colored by structural c-component, hidden nodes LiL_i have dashed border. We have seen certain regular computations on kernels, and we have structured kernels by graphs. Next we show that many (maybe all) relevant regular computations on kernels can be represented by simple and intuitive operations performed on model-aligned graphs. The first plausible candidate are sub-graphs. Given μ⁡(G,L)μ(G,L) we can certainly compute μ⁡(G′,L′)μ(G ,L ) for trivial cases where we simply discard hidden structure irrelevant to observed nodes: Lemma 3.5 (Simplify). Given a structured kernel μ⁡(G,L)μ(G,L) and a sub-set B⊂ innerB _ inner with (B)⊂B _G(B)⊂ B, then μ⁡(G′,L∖(B∩L))μ(G ,L (B∩ L)) can be computed as a regular functional of μ⁡(G,L)μ(G,L), where we call the sub-graph G′≤G ≤ G with inner nodes inner′= inner∖BN_ inner =N_ inner B and outer nodes outer′= ( inner′)∖ inner′N_ outer = _G(N_ inner ) _ inner a simplification. Restricting to B⊂LB⊂ L, there is a unique maximal B⊂LB⊂ L and we call the corresponding (minimal) μ⁡(G′,L∖B)μ(G ,L B) simplified. But also a kind of sub-graphs well known from the IID-case [61] appear: c-components. Definition 3.6 (C-Components). Given a structural model (G,L)(G,L), we call l∈Ll∈ L a hidden confounder of y,w∈ inner∖Ly,w _ inner L if there are directed paths γy _y from l to y and γw _w from l to w such that all non-endpoint nodes of γy,γw _y, _w are in L. On inner∖LN_ inner L define a relation y∼w:⇔∃y w: ∃ hidden confounder l of y,wy,w. Define ∼L _L as the equivalence relation generated by ∼ . We call equivalence-classes of ∼L _L on inner∖LN_ inner L c-components. We can consistently extend this notion to L by defining l≈yl≈ y for y∈ inner∖Ly _ inner L if there exists a directed path γy _y from l to y such that all non-endpoint nodes of γy _y are in L. We call a sub-graph Gc≤G^c≤ G with innercN_ inner^c an equivalence-class of ≈L _L and outerc= ( innerc)∖ innercN_ outer^c= _G(N_ inner^c) _ inner^c together with Lc=L∩ innerc=L∩cL^c=L _ inner^c=L ^c a structural c-component. Lemma 3.7 (Computation of C-Components). Given a structural c-component Gc≤G^c≤ G, then there is a regular functional computing μ⁡(Gc,Lc)μ(G^c,L^c) from μ⁡(G,L)μ(G,L). On the other hand, we can regularly compute (larger) products from their (smaller) constituents. A graphical analogue to obtain a larger graph from smaller ones is a gluing operation: Lemma 3.8 (Gluing). Given a structural kernel μ⁡(G,L)μ(G,L), and subgraphs GA,GB≤G^A,G^B≤ G, such that each structural c-component Gc≤G^c≤ G is a sub-graph Gc≤GAG^c≤ G^A or Gc≤GBG^c≤ G^B, then there is a regular functional computing μ⁡(G,L)μ(G,L) from μ⁡(GA,L∩A)μ(G^A,L ^A) and μ⁡(GB,L∩B)μ(G^B,L ^B). Our notion of observed and hidden variables is flexible, and it is possible to end up in situations where structural models on a graphical overlap differ by their latent sets. Almost by accident, we end up with a (non-parametric) causal missing-value imputation technique: Corollary 3.9 (Revealing). Given a structural kernel μ⁡(G,L)μ(G,L), a structural c-component Gc≤G^c≤ G and L′⊂LcL ⊂ L^c, then there is a regular functional computing μ⁡(G,(L∖Lc)∪L′)μ(G,(L L^c)∪ L ) from μ⁡(G,L)μ(G,L) and μ⁡(Gc,L′)μ(G^c,L ). Provided knowledge of a set of structured kernels K, we can systematically find regular computations to learn a (unknown) structured kernel of interest. It will be practically useful to order gluing operations to the end, first decompose elements of K into smaller pieces (which can be done algorithmically, Algo. CGDecomp), then glue the kernel of interest from these pieces (which can also be done algorithmically Algo. SVSearch). Finally, for details on how the results of these operations can be computed in practice, see Rmk. A.18, A.22. It seems plausible, that all regular computations of structured results (see §D.4 and nested queries however) can be achieved by these graphical operations, our main concern are potential generalizations of the revealing operation (Cor. 3.9), which currently is a simple corollary of other operations, but might be possible more generally (see §A.5): Conjecture 2 (Graphical Regular Computation). If a structured kernel μ⁡(G,L)μ(G,L) can be regularly computed from a set K of structured kernels, then there is a finite sequence of graphical operations computing μ⁡(G,L)μ(G,L) from K. 4 Extraction from Data Having introduced a language of structured kernels, we still need to connect its primitives to our model and to analyze what can be learned from data. The reader interested in the reasons behind formal choices made in the definitions of this section may want to analyze the proof of Thm. 1, especially step 1, in §C.4. We focus on the single-level case (cf. §4.6), not to be confused with the single-context case. All results and proofs are given for the multi-level case in the appendix. Multi-level statistics (see [14]) do so far not play a major role in causal literature and the specialization to a single level allows for a simplified notation. 4.1 Families of Embeddings We relate model-aligned graphs to individual local structures in the model (and I-graph) by a suitable notion of embeddings. For statistical reasoning, we need certain (for example sufficiently non-degenerate) repeated observations, thus we study families of embeddings additionally to individual embeddings. In the single-level case we can directly embed structured graphs, we nevertheless call them "local" graphs (and denote them G instead of G) in this section to emphasize that conceptually they are not the same as structured graphs. XXYYXXYYXXYYXXYYXXYYXXYYXXYYXXYYXXYYXXYYXXYYXXYY⋯·s⋯·sOKnot injectivenot (i)not (i)not (i)XXYYYYXXYYYYYYXXYYYY Figure 4: Examples illustrating conditions on local graph embeddings. Alignment is indicated by capital letters, external nodes are drawn as circles. Definition 4.1 (Local Graph Embedding). An embedding of a model aligned local graph G, denoted by ψ:↪Iψ:G I, is an injective mapping ψ:↪Iψ:N I such that the following conditions are satisfied (see Fig. 4): [label=()] 1. Proper-Node I-Parents are Included: If n∈ innern _ inner, then ∀i′∈ (ψ⁡(n))∀ i ∈ _I(ψ(n)), ∃n′∈∃ n such that i′=ψ⁡(n′)i =ψ(n ). 2. Proper Edges equal I-Graph Edges: For n′∈ innern _ inner, there is a proper edge n→n′n→ n in G if and only if ψ⁡(n)∈ (ψ⁡(n′))ψ(n)∈ _I(ψ(n )). 3. Model Alignment: ∀n∈ inner∀ n _ inner: μn=fJ⁡(ψ⁡(n))μ^n=f_J(ψ(n)) is a model mechanism (Def. 2.6). Definition 4.2 (Families of Embeddings). A family of local graph embeddings (J0,ψjj∈J0,,H,y0)(J_0,\ _j\_j∈ J_0,G,H,y_0) is a collection of local graph embeddings ψj:↪I _j:G I of a single model-aligned local graph G. Further there is a fixed element y0∈y_0 , which will be called the anchor(-node), a symmetry H⊂GH⊂ G and a range of applicability J0⊂IJ_0⊂ I. For n∈n denote: ψ∗(n):J0→I, j↦ψj(n) . _*(n):J_0→ I, j _j(n) . Finally, we will require the following conditions to be satisfied: [label=()] 1. Trivial on Anchor: ∀j∈J0∀ j∈ J_0: ψj(y0)=j _j(y_0)=j. 2. Rigidity: ∀n∈∀ n : ψ∗(n) _*(n) is H-equivariant. 3. Freeness: ∀n∈ inner∀ n _ inner: ψ∗(n) _*(n) is injective. Remark 4.3. The intuition behind using families of embeddings is to collect repeated occurrences of the same structure, as a prerequisite of statistical learning. The properties can be understood as follows: 1 simply says that indexing within the family is by occurrences (of the anchor node plus correct neighborhood) in the I-graph. 3 will avoid degeneracy of data-sets (for an injective map, the image contains as many elements as the domain J0J_0; different nodes may overlap, but for finitely many nodes this will not affect limits). 2 is specific to our formal description of symmetries by group-actions. In the presence of latent nodes, non-trivial neighborhoods of a mechanism in the I-graph have to be inspected (see below, . 5), which requires the "absorption" of additional nodes (growing a graph). Rigidity guides these absorption-operations for ensuring (and analyzing choices for) freeness 3 also after the addition of new nodes. Orbit-based based approaches might be more general (§G.5), but for formal analysis group-actions have the nice and intuitive property that absorbing a H′H -symmetric mechanism into a H-symmetric structure yields a H∩H′H∩ H -symmetric structure, of larger structures (as intuitively expected) is reduced to the intersection of the symmetries of their parts. This formal description of symmetries also simplifies encoding (both theoretically and practically) which might make it favorable for structure-learning in future work. Example: One also finds for H1H_1-symmetric X and H2H_2-symmetric YxY_x the product will appear in the I-graph (at best) H1∩H2H_1∩ H_2-symmetric. The causal disintegration is simply YxY_x, thus H2H_2-symmetric, while the anti-causal disintegration (X|Y)y(X|Y)_y is only H1∩H2H_1∩ H_2-symmetric, which provides a more concrete connection to invariance-based edge-orientation in structure-discovery, compare [37; 19; 27]. 4.2 Decorated Families These objects, so far, do not account for hidden variables, and also the well-known "backdoor-paths" [30] from the IID-world will find an analogue in embedded families. Definition 4.4 (Observedness). We call I0⊂I_0⊂ I observed if |I0∩V⁡(N)|→∞|I_0∩ V(N)|→∞ for N→∞N→∞. Given a family of local graph embeddings ψjj∈J0\ _j\_j∈ J_0, we call a subset O⊂O observed if J0obs(N,O) := j∈J0|ψj(O)⊂V⁡(N) , J^obs_0(N,O) := \j∈ J_0| _j(O)⊂ V(N)\ , satisfies|J0obs(N,O)|→∞ forN→∞ . satisfies|J^obs_0(N,O)|→∞ forN→∞ . Definition 4.5 (Minimal Latent Subsets). Given a family of local graph embeddings ψjj∈J0\ _j\_j∈ J_0, we call a subset O⊂O maximal observed if O′⊋O⇒O O O′O is not observed. In this case, we call the complement L=−OL=N-O a minimal latent subset. Similarly, some of the ancestral structure of the I-graph will be relevant for identifiability. Definition 4.6 (Ancestral Structure). Given a local graph G, an ancestral structure A on G is a set of directed edges ↝ (ancestral edges) each starting at an inner node and ending at an outer node, such that G with proper and ancestral edges is acyclic. Validity: An ancestral structure A on G is valid for an embedding ψ:↪Iψ:G I if ∀y∈ inner,x∈ outer∀ y _ inner,x _ outer: If there is directed path γ in GI∖ (ψ) G_I (ψ) (the I-graph with ψ-image of proper edges of G removed) from ψ⁡(y)ψ(y) to ψ⁡(x)ψ(x), then there is an ordering edge y↝xy x in A. See Fig. 5 for examples of valid ancestral structures. We call A valid for a family of embeddings ψjj∈J0\ _j\_j∈ J_0 if ∀j∈J0∀ j∈ J_0 it is valid for ψj _j. Definition 4.7 (Minimal Ancestral Structures). Given a local graph G, an ancestral structure A on G valid for a family of embeddings ψjj∈J0\ _j\_j∈ J_0 is J obs(N)J obs(N)-minimal for this family, if removing any non-empty subset ∅≠e⊂ ≠ e then the largest (sequence of) subsets of J0′(N)⊂J obs(N)J_0 (N)⊂ J obs(N) such that ∖eA e is valid for ψjj∈J0′\ _j\_j∈ J _0 is finite for N→∞N→∞, ∀N:|J0′(N)|≤C<∞∀ N:|J_0 (N)|≤ C<∞. Remark 4.8 (Non-Uniqueness). Neither one of these minimal structures is unique. For example consider a local graph containing the structure X1→Y←X2X_1→ Y← X_2 such that for each j∈J0j∈ J_0 of a associated family of embeddings exactly one of X1X_1 or X2X_2 is observed (and either one asymptotically infinitely often). Then there are two minimal latent subsets: L1=X1L_1=\X_1\ and L2=X2L_2=\X_2\. These attached structures "decorate" a family of embeddings (non-uniquely): Definition 4.9 (Decorated Families). A decorated family of embeddings is a family of embeddings ψjj∈J0\ _j\_j∈ J_0 together with a minimal latent subset L⊂L and a J0 obs(N,O)J obs_0(N,O)-minimal ancestral structure A. For identification, the interaction of backdoor-paths and latents will be important. Definition 4.10 (Backdoor-Freeness). Given a decorated family of embeddings (ψjj∈J0CLOSE(\ _j\_j∈ J_0, L, OPEN)A), we call an ancestral edge l↝x∈l x a backdoor if it starts at l∈Ll∈ L. We call the family backdoor-free, if L∩ outer=∅L _ outer= and there are no backdoors. XXZZYYLLin I-graph:start at X:XXAbs. Pa.LLXXstart at Z:ZZLLstart at Y:YYAbs. Pa.LLYYAbs. Ch.XXLLYY Figure 5: Systematic construction of minimal backdoor-free families. While there are: hidden external nodes (red-dotted boxes) absorb parents; backdoor-paths (green-dotted boxes) absorb children. Note that μ⁡(G,L)μ(G,L) extracted by Thm. 1 will in the end drop the remaining ancestral edge (it depends only on G and L, not on A), and can be further decomposed (cf. Fig. 3). A single gluing-operation then yields the well-known frontdoor formula [30]. There is a simple algorithmic approach (cf. Algo. ExtractCS, §C.5) to find (enough, Lemma C.26, "minimal") backdoor-free families by "growing" embeddings successively. As starting point we use direct embeddings of mechanisms: A mechanism fJf_J (Def. 2.5) with range of applicability J defines an embedded family indexed by J with a single proper node (aligned to fJf_J), one external node per parent and ψ⁡(j,n)=jψ(j,n)=j (for n∈ innern _ inner), ψ⁡(j,k)= k(j)ψ(j,k)= k(j) (for k∈ outerk _ outer). Then, as illustrated in Fig. 5, "absorb" latent external nodes as proper (or pinned) nodes and close backdoors by "absorbing" their starting node (a child of a latent node) as proper node. This process must be repeated until the result is backdoor-free. Here absorbing a node means checking overlaps with direct embeddings (see above) to find a family of embeddings for an enlarged local graph; note however, that the same mechanism can appear in the I-graph with different neighborhoods, so absorbing parents (or children of L) is not unique. There are at least three practical challenges to this approach: (i) Many constructions (like minimal latent-sets or candidates for absorption) are non-unique, so we usually have to deal with sets of results. (i) We can start this construction from any directly embedded mechanism, but even univariate queries will require intermediate results on joint descriptions (products of kernels), so besides starting-points of backdoor-paths, additional children of latents have to be absorbed (depending on the query). (i) Minimal backdoor-free families can, in principle, have arbitrarily many nodes which means algorithms either terminate after a finite number of steps or they are complete, but never both; in the IID case this number is bounded by the number of system variables. In practice large backdoor-free families require the estimation of kernels with many parameters and use (often) small subsets of the full data-set, so if we are interested in plausibly estimable results only, we can bail on large graphs and algorithms do effectively terminate. 4.3 Notions of Identifiability In the IID-case, identifiability is conventionally defined as being uniquely determined by the observational distribution P obsP_ obs [61; 57; 56; 4]. Our closest analogue of P obsP_ obs is the shallow distribution PθP_θ and the observable world. But each variable in the observable world is observed at most once, so we cannot "learn" PθP_θ (at least not immediately as for P obsP_ obs), so uniquely expressing anything relative to PθP_θ does not identify it in any reasonable sense. In fact, a suitable "joint" distribution to learn from will often not exist: Example 4.11 (Random Walk). Let ηt∼⁡(b,1) _t (b,1) be normal distributed and Xn:=∑t=1nηtX_n:= _t=1^n _t a random walk. While the kernel that describes Xn+1X_n+1 relative to the preceding time-step Xn+1(xn)=xn+⁡(b,1)X_n+1(x_n)=x_n+N(b,1) if perfectly well-defined, already the marginal distributions even of the unbiased, b=0b=0, one-dimensional case of the "joint distribution" of (Xn,Xn+1)(X_n,X_n+1) are (by CLT) of the form Xn∼⁡(0,σn2=n)X_n (0, _n^2=n). They not only depends on n, even worse the density pointwise approaches zero for large n. The fundamental idea of statistics is to combine many observations of the same thing. We employ a notion of identifiability that is a rather direct formalization of this idea. This approach helps to separate causal ideas from problems of statistical estimation theory (which is not a main topic of the present paper). Further discussion can be found in §C.3. Definition 4.12 (Data-Set). A data-set is a tuple =(Xj,Yj)j∈J0D=( X_j, Y_j)_j∈ J_0, where Xj X_j and Yj Y_j are tuples of Vi V_i, exist iX,j(1),…,iX,j(n)∈Ii_X,j^(1),…,i_X,j^(n)∈ I, such that Xj=(ViX,j(1),…,ViX,j(n)) X_j=( V_i_X,j^(1),…, V_i_X,j^(n)) and analogously for Yj Y_j. We call the data-set valid if X X and Y Y are observed that is J0 obs(N) := ∈J0 | ∀m:iX,j(m)∈V(N),∀m′:iY,j(m′)∈V(N) J_0 obs(N) := \ j∈ J_0 | ∀ m:i_X,j^(m)∈ V(N),∀ m :i_Y,j^(m )∈ V(N) \ satisfies |J0 obs(N)|→∞|J_0 obs(N)|→∞ as N→∞N→∞ and is non-degenerate j≠j′⇒∀m:iY,j(m)≠iY,j′(m)j≠ j ∀ m:i_Y,j^(m)≠ i_Y,j ^(m) (this last condition is imposed only on Y, not on X; see Rmk. C.22). Example 4.13 (Data-Set of Decorated Families). Given a decorated family of embeddings (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A) with L∩ outer=∅L _ outer= , there is a valid data-set (Xj,Yj)j∈J0( X_j, Y_j)_j∈ J_0 (Def. 4.12) defined for j∈J0j∈ J_0 as the tuples Xj X_j = (Vψj(n))n∈ outer = ( V_ _j(n))_n _ outer Yj Y_j = (Vψj(n))n∈ inner∖L . = ( V_ _j(n))_n _ inner L . Definition 4.14 (Direct Identifiability). Given a valid data-set (Def. 4.12) =(Xj,Yj)j∈J0D=( X_j, Y_j)_j∈ J_0 and kernels Xjj∈J0\X_j\_j∈ J_0, YxY_x, such that for each j individually the shallow distribution PθP_θ (Def. 2.14) satisfies, ∀j∈J0: Pθ(Xj,Yj)=Xj⊗Yx ,∀ j∈ J_0: P_θ( X_j, Y_j)=X_j Y_x , then we call YxY_x directly identifiable. Formally we also consider the known a priori ℱ interveneF_ intervene (Def. 2.6) directly identifiable. Example 4.15. If the data are IID, then Xj≡X_j≡ X do not depend on j and given (asymptotically infinite) data for the joint Pθ(Xj,Yj)=X⊗YxP_θ( X_j, Y_j)=X Y_x, we consider Yx=P⁡(Y|X=x)Y_x=P( Y| X=x) directly identifiable. Indeed in this case our notion of direct identifiability is the same as considering P obsP obs and its conditionals to be known (Lemma E.24). Definition 4.16 (Identifiablility). We call a kernel μ identifiable if μ can be uniquely computed from finitely many directly identifiable kernels. We call a collection of directly identifiable kernels μ1,…,μn _1,…, _n (plus their datasets) together with a regular functional F computing μ=F⁡[μ1,…,μn]μ=F[ _1,…, _n] an identification strategy for μ. 4.4 Identifiability from Embeddings Having fixed a formal notion of identifiability from data-sets, we continue by formal statements on the identifiability of structured kernels from decorated families of embeddings. The main result of this section is that backdoor-free families have identifiable μ⁡(,L)μ(G,L). There is a simple intuition for this: For each j, the image of ψj() _j(G) depends on its I-graph ancestors only through outerN_ outer (by Def. 4.1 1), if there are no backdoor-paths L↝ outerL _ outer, then inner nodes innerN_ inner form a union of c-components of the I-graph, thus ψj() _j(G) is essentially a union of structural c-components of the I-graph and a variant of Lemma 3.7 applies. Theorem 1. Given a backdoor free family of embeddings (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A) (Def. 4.10), then μ(,L) (Def.3.3)isidentifiable(Def.4.16).μ(G,L) (Def.\ def:structured_kernel_simp)isidentifiable(Def.\ def:identification). For the computation of this result in practice, cf. Rmk. C.23. Backdoor-free families of embeddings are also rather generic means of curating data-sets with invariant properties, thus the following seems plausible: Conjecture 3 (Completeness of Extraction). In the absence of selection bias, if a kernel μ is directly identifiable (Def. 4.14) from data, then there is a backdoor-free family of embeddings on (,L)(G,L) with μ=μ⁡(,L)μ=μ(G,L), at least for an orbit-based notion of symmetry, see §G.5 and after including multi-level structure (see appendix, §C). 4.5 Summary: Extracted Knowledge We briefly summarize what we kind of knowledge will be available for the reasoning about causal queries. Definition 4.17 (Extracted Knowledge Set). Let ℬB be the set of all backdoor-free families of embeddings. For (ψ,L,)∈ℬ(ψ,L,A) define a structured model k⁡(ψ,L,):=(,L)k(ψ,L,A):=(G,L). By Thm. 1, μ⁡(,L)μ(G,L) is identifiable (we simply say k is identifiable). For f~∈ℱ intervene f _ intervene, define a structured model k⁡(f~)=(G⁡(f~),L=∅)k( f)=(G( f),L= ), where G⁡(f~)G( f) is a graph with a single inner node y aligned to μy=f~μ^y= f, κ (the number of parents) outer nodes outer=x1,…,xκN_ outer=\x_1,…,x_κ\ and a proper edge from each xk→yx_k→ y. By definition (Def. 2.6 and 4.14), elements of ℱ interveneF_ intervene are considered known, thus k⁡(f~)k( f) is identifiable for all f~∈ℱ intervene f _ intervene. Define the, thus (element-wise) identifiable, single-level basic knowledge set as basic = ⋃(ψ,L,)∈ℬk⁡(ψ,L,) ∪ ⋃f~∈ℱ intervenek⁡(f~) .K_ basic = _(ψ,L,A) k(ψ,L,A) ∪ _ f _ intervenek( f) . 4.6 Extension: Multi-Level Statistics Multi-level statistics [14] models collections of datasets hierarchically (see example below). Our models do not explicitly fix a hierarchy on elements of I (nodes of the I-graph). But, while parent-sets in the I-graph are enforced finite, arbitrary numbers of children are allowed. Thus parts of the I-graph can be described meaningfully as hierarchical systems; in our formalism hierarchy is an emerging phenomenon – it is explicit only in the machinery used for identification of queries, not in the queries or models themselves. Example 4.18. Consider a case, similar to the motivating example of [15], where data was collected on water-levels (and similar properties) at multiple river-sites over time. Some physical mechanisms may be shared, but could depend on site-specific, constant in time, "contextual" properties like slope or form of the riverbed. It is possible to model these mechanisms per site (equivalently, a unique site-id is treated as "context-variable" [27; 15]). There are relevant questions, edge-orientations or interventions at a particular site that rely on such information. However, we might want to introduce a second level of modeling that inter-relates contextual properties: The form ( vs. width) of a riverbed may be driven by slope when comparing different sites. In that case, there should be hidden variables ( ) per site, with infinitely many children (time-points at that site). Such structure is allowed by our models implicitly. The argument above, and choice of hierarchy, can be made dynamically: The same variable may, with regard to some extraction-tasks, take the role of a context, while appearing as an ordinary variable in others. On a first reading, it may be helpful to disregard the additional complexity arising from the emergence of hierarchy and we will give simplified statements for the single-level case in the main text. Finally we want to point out that multi-context systems [19; 27; 15] can be described by a single-level system, as long as the modeling of relations between context-variables is not of relevance to a particular question (see §E.6); for an example where the multi-level aspects are relevant, see §D.5. 5 Queries and Prediction At this point, we have a formalism to describe a model, machinery to extract structured kernels from observations, and intuitive graphical transformations to work with such structured kernels. What is left to do, is to provide a way to phrase causal questions. We do so in two steps: First we define simple formal objects (basic queries), which we make then accessible through a more intuitive graphical language. Finally, the identifiability of queries will be a simple consequence of the results obtained in the previous sections. We will additionally phrase the question of query-identifiability in a slightly more abstract way that facilitates and structures the incorporation of internal structure, for example via instrumental variable arguments, for future work. Finally, we compare to standard formalisms in the IID-case. For reference, consider the following standard-setup (see Fig. 6): WWZZXXYYL2L_2L1L_1 WWZZXXYYL2L_2L1L_1 s2s_2 WWZZXXYYL2L_2L1L_1 s3s_3 WWZZXXYYL2L_2L1L_1 s4s_4 WWZZXXYYL2L_2L1L_1 s5s_5⋯·ss1s_1YYL2L_2X doX doQuery (at s=∗s=*) isalso part of I-graph."observed" i∈i∈ V(solid / green) Figure 6: Illustration of I-graph for do-intervention and IID-data (unified model). Nodes correspond to variables of the observable world, labels (capital letters) indicate mechanisms (Def. 2.5). Example 5.1 (IID Do-Intervention). Given SCM and associated IID model (example 2.7) on I varsI_ vars and X,Y⊂I varsX,Y⊂ I_ vars, extend the index set I=I vars×I sampleI=I_ vars× I_ sample by I vars×∗I_ vars×\*\ such that mechanisms are invariant under permutations of I sample⊔∗I_ sample \*\ on Jv=v×(I sample⊔∗)J_v=\v\×(I_ sample \*\) for v∉Xv∉ X, under permutations of I sampleI_ sample on Jv=v×I sampleJ_v=\v\× I_ sample if v∈Xv∈ X or have single element Jv=v×∗J_v=\v\×\*\ and associated known mechanism δv∈ℱ intervene _v _ intervene (singular at a component (x0)v(x_0)_v of x0x_0, the interventional target for X). This is a unified model description of example 2.7. Note that the I-graph has no edges between nodes at different samples in I sampleI_ sample or between any (v,s)∈I vars×I sample(v,s)∈ I_ vars× I_ sample and (v′,∗)∈I vars×∗(v ,*)∈ I_ vars×\*\, in particular we are in a single-level setup. Define θ=x0θ=x_0 (fully parameterizing ℱ interveneF_ intervene) and Y~:=Y×∗ Y:=Y×\*\, from example 2.17 we find Pθ(Y~)=Pθ(Y~)=P⁡(Y| (X=x0))P_θ^D( Y)=P_θ( Y)=P(Y| (X=x_0)) and the dataset ⁡(ω)D(ω) corresponds directly to the observations from the (unintervened) SCM M. Therefore, identifying Pθ(Y~)P_θ^D( Y) from ⁡(ω)D(ω) is the same as identifying P⁡(Y| (X=x0))P(Y| (X=x_0)) from P obsP obs (the observational distribution). We will formalize questions about PθP_θ^D (like "find Pθ(Y~)P_θ^D( Y)") as "basic queries". Identification of interventional distributions (and thus causal effects etc.) in the IID sense is a special case of such a query (by the construction described above). We show in the present section, that basic queries can, under weak assumptions always satisfied for do-interventions on IID-models, be transformed into graphical queries which can be answered soundly using §3 and §4 (see Thm. 2). This approach is complete in the IID case §E.1. 5.1 Basic Queries A basic query is formulated directly relative to a realized world. Query-identifiability is thus a tight statement; it directly relates simple and abstract structures, without referencing any of the technology like embeddings or structured kernels in its hypothesis or claim. Remark 5.2. A formal machinery can always show results about its own structure. But to justify its relevance, such "internal" results are of no use: Results about a structure are relevant if the structure is relevant. So justifying the relevance of a structure by results about its internals is always a cyclic argument. It is conceptually important that the machinery of embeddings and structured kernels presented in this paper enables us to proof more abstract statements, phrased in a much simpler and more abstract language. Definition 5.3 (Query). A basic query q=(Y~,X~,θ)q=( Y, X,θ) consists of finite subsets Y~,X~⊂I Y, X⊂ I with X~∩Y~=∅ X∩ Y= plus a set of parameters θ=ℱ interveneθ=F_ intervene (of the shallow distribution), with target Pθ(Y~|X~=x) .P_θ^D( Y| X=x) . On (standard) Borel spaces (Ass. 2.1), a basic query has a regular version, that is, there is a unique probability kernel Y~x Y_x such that Pθ(Y~|X~=x)=Y~xP_θ^D( Y| X=x)= Y_x. We call a basic query identifiable, if Y~x Y_x is identifiable (Def. 4.16) from the dataset ⁡(ω)D(ω). Remark: For interventional queries X~=∅ X= , the intervention is parameterized by θ. Other X~ X allow queries for intermediate results (cf. Fig. 3) and are necessary example 4.11, see also example D.11. 5.2 Structured Queries The illustrations in Fig. 1 and 6 of queries really show (embedded) local graphs with a few simple properties. This can be made formally clearer. Definition 5.4 (Structured Query). A structured query is an embedded local graph ψ:↪Iψ:G I together with a set of interest Y⊂ innerY _ inner, such that =∅A= is valid (Def. 4.6). Underlying Query: Given a structured query (ψ,Y)(ψ,Y), there is an underlying basic query q⁡(ψ,Y):=(Y~=ψ⁡(Y),X~=ψ⁡( outer),θ)q(ψ,Y):=( Y=ψ(Y), X=ψ(N_ outer),θ). We call (ψ,Y)(ψ,Y) identifiable, if the underlying basic query is identifiable. For many basic queries there is a canonical structured query constructed by, starting from Y~ Y, adding I-graph parents until a parent is either in X~ X (and added as external node), or no more parents are available. By Ass. 2.11 (finite past), this construction terminates after a finite number of steps with a finite graph. Embed this graph by the identity-mapping ψ= ψ= and identify Y:=Y~Y:= Y with its ψ-image. Lemma 5.5 (Associated Structured Query). Given is a basic query q=(Y~,X~,θ)q=( Y, X,θ) about a single-level statement, that is Y~ Y is disconnected (not reachable by directed or undirected paths in the I-graph) from V. Further, X~ X cannot be bypassed in the sense of: For y∈Y~y∈ Y and w∈ (X~)∖X~w∈ _I( X) X, there is no directed path γ:w↝yγ:w y in the I-graph with γ∩X~=∅γ∩ X= . Then there is a structured query (ψ⁡(q),Y~)(ψ(q), Y) with underlying query q. Remark 5.6 (Assumptions for Structured Representation). "Standard" interventions, as in example 5.1, and even transport-questions (§E.6) can be described as single-level statements in this sense. Further, for interventional formulations (like the do-calculus), X~=∅ X= (see §E), so the "no bypassing" assumption is always trivially satisfied; in other cases (parts of) X~ X can be included into Y~ Y and disintegrated at the end, see §D.4; however nodes might be allowed in X~ X but not in Y~ Y, see example 4.11. In the multi-level case the hypothesis of Lemma 5.5 can be weakened (Lemma D.8), and it becomes an if and only if statement. Structured queries can be identified using the previously introduced machinery, §3 produces regular functionals, §4 produces identifiable knowledge: Theorem 2. Given a structured query (ψ,Y)(ψ,Y), using L:= inner∖YL:=N_ inner Y, if there is a regular functional F computing μ⁡(,L)=F⁡[μ1,…,μn]μ(G,L)=F[ _1,…, _n] with μ1,…,μn _1,…, _n all identifiable, then the underlying query q⁡(ψ,Y)q(ψ,Y) is identifiable. Algorithm EDAIdentify Extact–Decompose–Assemble Identification Input: A query μ⁡(G,L)μ(G,L). Output: A (potentially empty) set of identification strategies. [label=0)] 1. Extract: ℬB := ExtractCS. K := Apply Thm. 1 to elements of ℬB. 2. Decompose: ′K := Apply CGDecomp to elements of K. 3. Assemble: Return SVSearch(′∪f~∈ℱ intervenek(f~)∪i∈k(i)K _ f _ intervenek( f) _i∈ Vk(i), (G,L)(G,L)). ⊳ k⁡(i)k(i) are included here for completeness, they are relevant only to the multi-level case (cf. Def. C.13). A systematic approach to find identification strategies based on graphical operations §3 for a given query is sketched in Algo. EDAIdentify. First, minimal backdoor-free families are constructed. Then μ⁡(,L)μ(G,L) (obtained by Thm. 1) are decomposed into smaller parts and finally assembled by gluing (Lemma 3.8) into the query-graph of interest. This approach is evidently only complete if graphical operations are complete (Conj. 2) and additionally: Conjecture 4 (Decomposition Before Assembly). If a regular computation is possible by graphical operations, then it is possible to arrange all gluing operations to the end. We leave a detailed analysis of algorithms and their properties to future work, we do however give multiple partial results (like Lemmas C.17, C.20, C.26) that clarify why some simplifications in the algorithms presented in the appendix are possible without loss of generality. Summarizing, the main open questions concerning completeness are in the already highlighted conjectures: Conjecture (Completeness). If Conjectures 1, 2, 3, 4 hold true, then in absence of selection-bias, a structured query is identifiable if and only if Algo. EDAIdentify returns a non-empty set. 5.3 Knowledge-Closures and Identification At the end of §4, we summarized the extracted knowledge as a set basicK_ basic (Def. 4.17). Next, we ask, from an abstract perspective, when is a query identifiable relative to basicK_ basic? Algorithmic completeness, relative to extraction-results ( completeness of computation only), can be phrased as the recovery of a closure under a given regularity condition. Definition 5.7 (Closures of Knowledge Sets). Given a knowledge-set K and a class ℛR ("regularity class") of functionals closed under finite compositions, the ℛR-closure of K, denoted ¯ℛ K^R, is the set of elements of knowledge ℛR-identified from K: ¯ℛ := ′ | ∃F∈ℛ,∃k1,…,kn∈:k′=F[k1,…,kn] . K^R := \ k | ∃ F ,∃ k_1,…,k_n :k =F[k_1,…,k_n] \ . If ℛR is the set of (standard-)regular functionals in the sense of Def. A.8, then we simply write ¯ K (dropping the superscript), and μ is identifiable (Def. 4.16) if μ∈¯ basicμ∈ K_ basic. Example 5.8 (Instrumental Variables). Assume there are subsets ℱ mono,ℱ epi⊂ℱ intervene∪ℱ obsF_ mono,F_ epi _ intervene _ obs of left- and right-cancelative kernels respectively ( μ∈ℱ monoμ _ mono iff ν∘μ=ν′∘μν μ=ν μ ⇒ ν=ν′ν=ν for all ν∈ℱ intervene∪ℱ obsν _ intervene _ obs; in other words, ℱ monoF_ mono and ℱ epiF_ epi contain "sufficiently injective " elements). Define ℛ IVR IV as the class of regular functionals (Def. A.8), functionals of the form ν∘μ↦ν μ ν if μ∈ℱ monoμ _ mono and ν∘μ↦μν μ μ if ν∈ℱ epiν _ epi and compositions of such functionals. For example if all elements of ℱ intervene∪ℱ obsF_ intervene _ obs are linear, then ℱ mono=ℱ epi=ℱ intervene∪ℱ obsF_ mono=F_ epi=F_ intervene _ obs and this deconvolution is called an "instrumental variable" argument [30, p. 247]. Formally, the distinction should be made between a priori knowledge of ℱ mono,ℱ epiF_ mono,F_ epi and their existence (since not every fJf_J is identifiable, this may or may not be testable, given fixed assumptions on the form of mechanisms). Since (by definition) regular functionals are in ℛ IVR IV, automatically ¯⊂¯ℛ IV K⊂ K^R IV for all K. 5.4 Relation to the IID-Case Under assumptions where the do-calculus applies, if data is IID, interventions are do-interventions, and missingness is uniform (IID-variables are either always or never observed), the id-algorithm [61] is known to be complete [57]. In §E.1, we show in detail that Algo. EDAIdentify returns an empty set only if there is a "hedge", which by a criterion of [57] immediately implies that our algorithm is also complete, at least for unconditional do-interventions, matching the id-algorithm. For conditional queries, some care should be taken as rule 2 of the do-calculus exploits the internal structure of do-interventions (which we intentionally exclude), but conditional queries (treated in detail by [56] in the IID case) matching this internal structure assumption can readily be formulated as meta-queries (see §D.4) without substantial overhead, see §E.4. We also compare our approach in detail to the mz-transportability setup [34; 6; 4] in §E.7. This setup aims to transport experimental and non-experimental information between IID-contexts (with a number of restrictions, for example all contexts share a graph, and hiddenness is cross-context uniform). It uses "selection-variables" (often called context-variables in other sources), which (in similar form) are at the foundation of many influential multi-context approaches to causal questions [19; 27; 15]. We find that also here Algo. EDAIdentify seems to be complete (by a criterion of [4]), matching the algorithm of [5; 4]. Finally, somewhat surprisingly, we can also readily reproduce the mediation formula for natural direct effects (NDE) (or rather: the formula for underlying conditional probabilities). This is initially somewhat surprising, as we do not actually consider counter-factuals. However, our query-formulation is general enough to capture the question motivating NDE directly, without the detour through counterfactuals, see §E.8. We can readily reproduce many relevant results from the literature, importantly we can do so in a unified approach (and algorithm). While literature on the topic of causal effect-estimation is dominated by IID-setups and do-interventions (and hence so is this comparison-section), the main purpose and strength of our formalism is its applicability far from IID and its flexible query formulation. However, as the comparison above shows, this improved flexibility avoids costs in identification-power. While we do not know if our algorithm is complete, it does not lead to a regression below the state of the art in standard setups. 6 Conclusion We introduced a simple and widely applicable abstract language to describe causal reasoning §2. This weak imposed structure proofs sufficient to learn and reason about a model §3, §4 in order to ultimately identify a notion of causal queries §5, that includes many conventionally considered questions as special cases. While some questions, about selection-bias, correlated missingness, counterfactuals and also about the completeness of our identification strategy in general remain open, we do re-discovery many core structures like c-components, c-trees, c-forests and hedges known to be pivotal to IID do-interventional queries [61; 57] from a new perspective, which not only sheds new light on these objects, but it also validates our results against known special cases. Also the ability to (correctly) reproduce the mediation-formula for natural direct effects, and mz-transport results is quite encouraging. From a larger perspective, it would certainly be interesting to understand how and to what degree the models employed by our formalism can be learned from data, §F.3. However, finding the right structure to learn must necessarily precede finding ways of learning it. Finally, returning to the example from the introduction (Fig. 1), note that the panels highlighted as "learn" can indeed be identified by Thm. 1, and the query can be glued (lemma 3.8) from these partial results and the known Y initY_ init. In a last step, variables not of interest can be marginalized. Acknowledgements J.R. has received funding from the European Research Council (ERC) Starting Grant CausalEarth under the European Union’s Horizon 2020 research and innovation program (Grant Agreement No. 948112). J.R. and M.R. have received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 101003469 (XAIDA). References [1] C. K. Assaad. Causal reasoning in difference graphs. arXiv preprint arXiv:2411.01292, 2024. [2] C. Avin, I. Shpitser, and J. Pearl. Identifiability of path-specific effects. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI’05, page 357–363, San Francisco, CA, USA, 2005. Morgan Kaufmann Publishers Inc. [3] C. Balsells-Rodas, Y. Wang, and Y. Li. On the identifiability of markov switching models, 2023. [4] E. Bareinboim and J. Pearl. Transportability of causal effects: Completeness results. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pages 698–704, 2012. [5] E. Bareinboim and J. Pearl. A general algorithm for deciding transportability of experimental results. Journal of Causal Inference, 1(1):107–134, 2013. [6] E. Bareinboim and J. Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113:7345 – 7352, 2016. [7] S. Bongers, P. Forré, J. Peters, and J. M. Mooij. Foundations of structural causal models with cycles and latent variables. The Annals of Statistics, 49(5):2885–2915, 2021. [8] J. T. Chang and D. Pollard. Conditioning as disintegration. Statistica Neerlandica, 51(3):287–317, 1997. [9] D. Colombo, M. H. Maathuis, et al. Order-independent constraint-based causal structure learning. J. Mach. Learn. Res., 15(1):3741–3782, 2014. [10] J. Corander, A. Hyttinen, J. Kontinen, J. Pensar, and J. Vänänen. A logical approach to context-specific independence. Annals of Pure and Applied Logic, 170(9):975–992, 2019. [11] J. Correa and E. Bareinboim. General transportability of soft interventions: Completeness results. Advances in Neural Information Processing Systems, 33:10902–10912, 2020. [12] J. de Aguas, L. Henckel, J. Pensar, and G. Biele. Causal inference amid missingness-specific independencies and mechanism shifts. arXiv preprint arXiv:2506.15441, 2025. [13] P. M. Faller, L. C. Vankadara, A. A. Mastakouri, F. Locatello, and D. Janzing. Self-compatibility: Evaluating causal discovery without ground truth. In International Conference on Artificial Intelligence and Statistics, pages 4132–4140. PMLR, 2024. [14] A. Gelman and J. Hill. Data analysis using regression and multilevel/hierarchical models. Cambridge university press, 2006. [15] W. Günther, U. Ninad, and J. Runge. Causal discovery for time series from multiple datasets with latent contexts. In Uncertainty in Artificial Intelligence, pages 766–776. PMLR, 2023. [16] W. Günther, O.-I. Popescu, M. Rabel, U. Ninad, A. Gerhardus, and J. Runge. Causal discovery with endogenous context variables. Advances in Neural Information Processing Systems, 37:36243–36284, 2025. arXiv:2412.04981. [17] F. R. Guo and E. Perković. Efficient least squares for estimating total effects under linearity and causal sufficiency. Journal of Machine Learning Research, 23(104):1–41, 2022. [18] P. W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945–960, 1986. ISSN 01621459, 1537274X. [19] B. Huang, K. Zhang, J. Zhang, J. Ramsey, R. Sanchez-Romero, C. Glymour, and B. Schölkopf. Causal discovery from heterogeneous/nonstationary data. The Journal of Machine Learning Research, 21(1):3482–3534, 2020. [20] A. Hyttinen, J. Pensar, J. Kontinen, and J. Corander. Structure learning for bayesian networks over labeled dags. In International conference on probabilistic graphical models, pages 133–144. PMLR, 2018. [21] D. Janzing, P. M. Faller, and L. C. Vankadara. Reinterpreting causal discovery as the task of predicting unobserved joint statistics. arXiv preprint arXiv:2305.06894, 2023. [22] O. Kallenberg. Foundations of modern probability. Springer, 1997. [23] O. Kallenberg. Probabilistic symmetries and invariance principles. Springer, 2005. [24] F. Lattimore, T. Lattimore, and M. D. Reid. Causal bandits: Learning good interventions via causal inference. Advances in neural information processing systems, 29, 2016. [25] S. Mameche, L. Cornanguer, U. Ninad, and J. Vreeken. Spacetime: Causal discovery from non-stationary time series. arXiv preprint arXiv:2501.10235, 2025. [26] A.-U. Margueritte, A. Z. Balcıoğlu, J. Krijthe, D. Zachariah, and F. D. Johansson. Learning plug-in surrogate endpoints for randomized experiments. arXiv preprint arXiv:2605.12051, 2026. [27] J. M. Mooij, S. Magliacane, and T. Claassen. Joint causal inference from multiple contexts. The Journal of Machine Learning Research, 21(1):3919–4026, 2020. [28] K. P. Murphy. Probabilistic machine learning: an introduction. MIT press, 2022. [29] J. Park, S. Buchholz, B. Schölkopf, and K. Muandet. A measure-theoretic axiomatisation of causality. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 28510–28540. Curran Associates, Inc., 2023. doi: 10.52202/075280-1239. [30] J. Pearl. Causality: Models, reasoning and inference. Cambride University Press, 2000. [31] J. Pearl. Direct and indirect effects. Proceedings of the Seventeenth Conference on Uncertainty in Artificial intelligence, 2001. [32] J. Pearl. The causal mediation formula—a guide to the assessment of pathways and mechanisms. Prevention science, 13:426–436, 2012. [33] J. Pearl and E. Bareinboim. External validity: From do-calculus to transportability across populations. Statistical Science, 29(4):579–595, 2014. [34] J. Pearl and E. Bareinboim. External validity: From do-calculus to transportability across populations. In Probabilistic and causal inference: The works of Judea Pearl, pages 451–482. 2022. [35] J. Pensar, H. Nyman, T. Koski, and J. Corander. Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models. Data mining and knowledge discovery, 29:503–533, 2015. [36] E. Perković, J. Textor, M. Kalisch, and M. H. Maathuis. A complete generalized adjustment criterion. arXiv preprint arXiv:1507.01524, 2015. [37] J. Peters, P. Bühlmann, and N. Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78(5):947–1012, 2016. [38] J. Peters, D. Janzing, and B. Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017. [39] M. Rabel and J. Runge. Context-specific causal graph discovery with unobserved contexts: Non-stationarity, regimes and spatio-temporal patterns, 2026. [40] M. Rabel, W. Günther, J. Runge, and A. Gerhardus. Causal modeling in multi-context systems: Distinguishing multiple context-specific causal graphs which account for observational support. arXiv preprint arXiv:2410.20405, 2025. [41] A. Rahmani and P. Frossard. Castor: Causal temporal regime structure learning. 2023. [42] J. M. Robins and T. S. Richardson. Alternative graphical causal models and the identification of direct effects. Causality and psychopathology: Finding the determinants of disorders and their cures, 84:103–158, 2010. [43] C. B. Rodas, R. Tu, and H. Kjellstrom. Causal discovery from conditionally stationary time-series. arXiv preprint arXiv:2110.06257, 2021. [44] M. Rojas-Carulla, B. Schölkopf, R. Turner, and J. Peters. Invariant models for causal transfer learning. The Journal of Machine Learning Research, 19(1):1309–1342, 2018. [45] D. Rothenhäusler, N. Meinshausen, P. Bühlmann, and J. Peters. Anchor regression: Heterogeneous data meet causality. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83(2):215–246, 2021. [46] D. B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol., 66(5):688–701, 1974. ISSN 00220663. doi: 10.1037/h0037350. [47] D. B. Rubin. Inference and missing data. Biometrika, 63(3):581–592, 1976. [48] J. Runge. Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets. In Conference on Uncertainty in Artificial Intelligence, pages 1388–1397. PMLR, 2020. [49] J. Runge. Necessary and sufficient graphical conditions for optimal adjustment sets in causal graphical models with hidden variables. Advances in Neural Information Processing Systems, 34:15762–15773, 2021. [50] J. Runge, P. Nowack, M. Kretschmer, S. Flaxman, and D. Sejdinovic. Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances, 5(11):eaau4996, 2019. [51] J. Runge, A. Gerhardus, G. Varando, V. Eyring, and G. Camps-Valls. Causal inference for time series. Nature Reviews Earth & Environment, pages 1–19, 2023. [52] E. Saggioro, J. de Wiljes, M. Kretschmer, and J. Runge. Reconstructing regime-dependent causal relationships from observational time series. Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(11), 2020. [53] R. D. Shah and J. Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics, 48(3):1514, 2020. [54] I. Shpitser. Counterfactual graphical models for longitudinal mediation analysis with unobserved confounding. Cognitive science, 37(6):1011–1035, 2013. [55] I. Shpitser. When does the id algorithm fail? arXiv preprint arXiv:2307.03750, 2023. [56] I. Shpitser and J. Pearl. Identification of conditional interventional distributions. In Uncertainty in Artificial Intelligence, volume 22, pages 437–444, 2006a. arXiv:1206.6876. [57] I. Shpitser and J. Pearl. Identification of joint interventional distributions in recursive semi-markovian causal models. In AAAI, pages 1219–1226, 2006b. [58] I. Shpitser and J. Pearl. What counterfactuals can be tested. arXiv preprint arXiv:1206.5294, 2012. [59] P. Spirtes and C. Glymour. An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review, 9:62–72, 1991. [60] P. Spirtes, C. Glymour, and R. Scheines. Causation, prediction, and search. MIT press, 2001. [61] J. Tian and J. Pearl. A general identification condition for causal effects. In Aaai/iaai, pages 567–573, 2002. [62] V. Vapnik. Estimation of dependences based on empirical data. In M. Jordan, J. Kleinberg, and B. Schölkopf, editors, Vapnik: Estimation of Dependences Based on Empirical Data, Information Science and Statistics, pages 1–399. Springer Science + Business Media, New York, second edition, 2006. Reprint of 1982 edition. Reading Guide This appendix is rather long and in places very technical, so we start with a brief overview to help the reader orient themselves. A table of contents is included at the very end of the paper. The two technically most interesting and relevant results are the proofs of graphical operations (§3.3) and of Thm. 1 (extraction from backdoor-free families). It should be possible to start reading the respective proofs, then reference back to other material of the appendices where required (or of interest). Graphical operations are proved in §A.3. The main proof technique is a reduction to statements about certain univariate kernels called "atoms" in §A.3. The proof of Thm. 1 in §C.4 relies on results about these "atoms" as well, hence it is recommendable to get an overview of results shown in §A.3 before starting to read the proof of Thm. 1 (this is especially true for proof-steps 2 and 3). Some technical details on the relation of data-sets to models relevant to the proof of Thm. 1 have been detached into §B to make the proof more accessible. We removed multi-level results from the main-text for improving readability. Where differences arise (in §C and §D) an initial sub-section introduces the multi-level setup as a simple modification of the main text. It should also be possible to read the corresponding appendices (especially §C) for the single-level case only, without much confusion (see introduction to §C; for the proof of Thm. 1 all multi-level aspects are contained within a separate proof-step 3). For queries (§D), multi-level aspects are relevant for phrasing more general questions: In the single-level case, after marginalizing to the query Pθ(X~,Y~)=Pθ(X~,Y~)P_θ^D( X, Y)=P_θ( X, Y) and many technical problems vanish. For an intuition why additional non-trivial statements can be made with multi-level structures see the example in notation C.11. Further examples of multi-level queries and their usefulness are given in §D.5. Especially to the reader familiar with results for the IID-case [30; 61; 56], to better understand the query-formulation, it may also be helpful to have a look at §E. By separating the random elements of an observable world from the purely measure-theoretical model our approach can be made mathematically rigorous. To clarify notation and technical details, §G builds the detailed technical connection from citeable text-book results to the notation and computational means used throughout the remainder of this paper. Appendix A Details on Structured Kernels Basic notions, especially concerning kernels, are recalled and explained with examples in SCM language in §G. Here we continue with non-standard constructions on kernels, designed for our use-case of encoding (and computing with) the sparsity-structure of a (causal) structural graph while respecting latent structure. A.1 Structural Graphs Large ⊗ -products of kernels with sparse inter-dependencies, are essentially a type of tensor network. Tensor networks are often represented graphically. Also the representation of causal relationships by graphical models is very common. It therefore is not particularly surprising, that a suitable form of graphical representation will also help substantially in organizing the knowledge we encounter in our reasoning here. We start by defining a precise notion of graphical representation that can encapsulate the causal sparsity information of §G.4 in a more accessible form. First, we give a slightly more detailed version of Def. 3.2 split into a graph-definition A.1 and model-alignment A.3. Definition A.1 (Structural Graph And Causal Orders). A structural graph G is a finite set of nodes := inner∪˙ outerN:=N_ inner\, ∪\,N_ outer with inner∩ outer=∅N_ inner _ outer= , together with a set of directed edges ℰ⊂× inner∖ΔE ×N_ inner , where Δ=(n,n)|n∈ inner =\(n,n)|n _ inner\ is the diagonal, (proper edges) between nodes in innerN_ inner and from nodes in outerN_ outer to nodes in innerN_ inner. We call a structural graph G acyclic, if there are no directed cycles. If G is acyclic, then there is a partial order on (all) nodes N defined by n1≤n2:⇔n_1≤ n_2: there is a directed path from n1n_1 to n2n_2. A G-causal order on N is a total order extending this partial order. Such a total order always exists, but it is in general not unique. Remark A.2. Edges are relevant (only) for parent relation-ships (and properties deduced thereof, like ancestral relation-ships), it thus is irrelevant if more than one edge is allowed between nodes for each orientation and that there are no edges from a node to itself. We did adopt the convention of allowing at most one edge (per orientation) between any two nodes (the definition of edges ℰ⊂× inner∖ΔE ×N_ inner enforces this), which is in line with standard causal analysis [30; 38]. With the same mechanism fYf_Y potentially featuring in different contexts, it may occur in practice, with the same variable acting as multiple parents, (x1,x2)f_Y(x_1,x_2) could be evaluated as x↦fY(x,x)x f_Y(x,x). This can be modeled easily, by defining a diagonal map ΔX:X→X×X,x↦(x,x) _X:X→ X× X,x (x,x) with ΔX∈ℱ intervene _X _ intervene and considering fy∘ΔXf_y _X as a soft-intervention (which is allowed both in observations an queries, which we do not distinguish formally); technically the reader may have noticed, that ΔX _X is not actually a probability-kernel, however we can replace it by one which has singular measure 11 at the value taken by ΔX _X. Note that this interpretation does not seem to affect identifiability: If fYf_Y is identifiable, then the composition with an intervention ΔX∈ℱ intervene _X _ intervene will be considered identifiable (if we know fY(x1,x2)f_Y(x_1,x_2) we can in particular plug in the same argument twice), but knowing (from observations) only fY∘ΔXf_Y _X will only retain this composition ( , observing only fY(x,x)f_Y(x,x) we cannot make predictions for queries that involve the full two-dimensional fY(x1,x2)f_Y(x_1,x_2), at least not without substantial knowledge about internal structure, cf. §5.3). Definition A.3 (Model Alignment). A model aligned structural graph is an acyclic structural graph G together with [label=()] 1. a probability kernel μnμ^n for each inner node n∈ innern _ inner. and a bijective assignment (cf. G.24) of the κ arguments of μnμ^n to proper parents of n: :1,2,…,κ→1:1 (n) . ^n:\1,2,…,κ\ 1:1 _G(n) . 2. μnμ^n is a kernel on κnκ^n arguments, from X1n,…,XκnnX^n_1,…,X^n_κ^n to YnY^n, we require ∀n∀ n: Y (1)⊂X1nY ^n(1)⊂ X^n_1, …, Y (κn)⊂XκnnY ^n(κ^n)⊂ X^n_κ^n. Notation A.4 (Structural Graph Contraction). Given a model aligned acyclic structural graph G together with a G-causal order π and a subset ′⊂ innerN _ inner (which is totally ordered by ≤π _π), we write ⊗n∈′πμn:=⊗n∈′ ∗μn (cf.Def.G.24) ^π_n μ^n:= _n ^*μ^n (cf.\ Def.\ def:contraction) ∏x∈ (′)∖′(∩(n,k)| (k)=xXkn) ∏n∈′Yn , whichisakernelfrom _x∈ _G(N ) ( _\(n,k)| ^n(k)=x\X^n_k ) to _n Y^n , for the product of the μnμ^n in π-order, contracted by the mapping ^n of Def. A.3, notation G.25 if nodes in the graph are named X,Y,Z,… X, Y, Z,… with associated kernels X, Y, Z, then for example Y Y having parents X,Z X, Z will have arguments x,zx,z, Yx,zY_x,z (the order is relevant, by convention we write indices in π-order, that is the kernels formally depends on π, albeit only by reordering its arguments). More formally this means we set wn:= ^n:= ^n in Def. G.24. Next we use these structural graphs to attach sparsity-information to kernels, detailing Def. 3.3 of the main text; the main text additionally uses notation A.7: Definition A.5 (Structured Kernel). A structural model (G,L)(G,L) is a tuple of a model aligned structural graph G together with a subset L⊂ innerL _ inner. Given a structural model (G,L)(G,L) and a G-causal order π, we define (using notation A.4; "in causal order compute the value at node n, by plugging into the k-th argument of μnμ^n, the value of the node (k) ^n(k), where outer nodes x are left open as arguments of the resulting μ(G,L,π)xμ(G,L,π)_x") μ(G,L,π)x := (⊗n∈ innerπμn)x ,μ(G,L,π)_x := _L( _n _ inner^πμ^n)_x , where x=(xi)i∈ outerx=(x_i)_i _ outer summarizes arguments in outerN_ outer. Given a set ℱF of probability kernels (later typically ℱ=ℱ model∪ℱ intervene∪δi|i∈IF=F_ model _ intervene∪\ _i|i∈ I\), a ℱF-structured kernel is a kernel μx _x that can be written in the form μx=μ(G,L,π)x _x=μ(G,L,π)_x, where all nodes n∈ innern _ inner are aligned to an element μn∈ℱμ^n . This definition depends on the G-causal order π, but only up to transposition (Def. G.19), see A.7. It is helpful to briefly recall (slightly more formal) the precise definition of subgraphs introduced in the main text, as it is relevant to multiple graphical operations: Definition 3.4 (Structural Subgraphs) A structural subgraph GA≤GBG^A≤ G^B of a structural graph GBG^B is a structural graph GAG^A such that: [label=()] 1. Node Sets: A⊂BN^A ^B and innerA⊂ innerBN_ inner^A _ inner^B. 2. Edge Sets: ℰA=ℰB∩(A× innerA)E^A=E^B∩(N^A×N_ inner^A), are exactly those in GBG^B from nodes in GAG^A to inner nodes of GAG^A. 3. Inner Parents: ∀n∈ innerA∀ n _ inner^A: if p∈ (n)p∈ _G^B(n), then p∈Ap ^A. 4. Alignment: For model-aligned graphs ∀n∈ innerA∀ n _ inner^A: (μA)n=(μB)n(μ^A)^n=(μ^B)^n. We write GA<GBG^A<G^B, if GA≤GBG^A≤ G^B and innerA⊊ innerBN_ inner^A _ inner^B. A.2 Regularity This subsection is conceptually important to understanding the limits of computation and identifiability within the formalism proposed in this paper and beyond it. However, it is somewhat technical and, on a first reading, may be skipped, as it may disrupt the logical flow of the presentation. Statistical and causal inference fundamentally ask: What questions about a system are "identifiable", that is well-defined, answers that are uniquely determined by observations (at least in principle and asymptotically), and how can they be "estimated", that is how can answers be approximated (from finite data). In this section we clarify what kind of computations are possible with kernels within this philosophy. However, first a few remarks about uniqueness of kernels and the relevance of distributional support to transfer problems are required. Remark A.6 (Support). Disintegrations are typically only "ϑ⊗μ μ-almost everywhere" unique (see Lemma G.14). In statistics, "almost everywhere" is often read essentially as "good enough". From the perspective of learning transferable properties of a model, this hides a very fundamental aspect of empirical science: We can only ever learn things that we actually see (encounter with non-zero-probability). If we observe P⁡(X,Y)P( X, Y) of a causal (say IID) model X→Y X→ Y with mariginal μ=P⁡(X)μ=P( X), then whatever we try to learn about (a regular version of) the conditional distribution νx:=P⁡(Y|X=x) _x:=P( Y| X=x), we can learn at best in a μ-almost everywhere sense. This is not a problem of our formalism, it is a deeply rooted issue of modeling transfer of knowledge (for example to an intervened system). Before we go into any detail, it seems important to point out, that this is (usually) not actually a problem in practice. When Pearl [30] (and others) talk about do-interventions to a value x0x_0 then either X X is categorical (and P⁡(X=x0)≠0P( X=x_0)≠ 0, in which case δ⁡[x0]⊗νxδ[x_0] _x is well-defined, even if νx _x is defined only μ-almost everywhere) or the actual real-world problem formally expressed as " (X=x) ( X=x)" is understood as "once X X is forced to a value very close to x"; this last notion is actually perfectly fine, for example if "very close to x" is a measure dominated by (absolutely continuous w. r. t.) μ. Essentially, the usual argument is "the formal problem is just a proxy for the real-world problem". Despite this, there is the (inherently formal) aspect of mathematical rigor. For (most) practical computations, the interpretation of do-interventions given above makes sense. But what about proofs? A proof-technique that can "prove" both true and false statements does not seem particularly convincing. Overly simplistic notions of conditional probability (like quotients of joints) have long since been known for their pitfalls (see [8]). The problem here is as follows: Consider two IID models M1M_1 and M2M_2 with X→Y X→ Y and X∼⁡(0,1)X (0,1), in M1M_1 the mechanism at Y is fY(x)=x+⁡(0,1)f_Y(x)=x+N(0,1) if x≠0x≠ 0 but fY(x)=42+⁡(0,1)f_Y(x)=42+N(0,1) if x=0x=0, while in M2M_2 the mechanism at Y is simply gY(x)=x+⁡(0,1)g_Y(x)=x+N(0,1). Both models produce the same joint observational distribution P⁡(X,Y)P( X, Y), yet P1(Y| (X=0))≠P2(Y| (X=0))P_1( Y| ( X=0))≠ P_2( Y| ( X=0)). Thus the do-intervention is (formally) not identifiable from observations or even from perfect knowledge of the observational joint distribution. The problem is, that by P⁡(X=0)=0P( X=0)=0, so we will almost never (with probability 00) figure out the difference between both models. Note that p⁡(X)>0p( X)>0, so a positivity-assumtion cannot fix this issue. This may often not be a problem in computations (see above), but obviously, if it is not actually formally true that this singular intervention can be identified (the example above is a counter-example), then any rigorous mathematical approach must see this formal problem. Using a rigorous language for conditional probabilities, on standard Borel spaces (Ass. 2.1) disintegrations are available, the problem actually becomes apparent: The general idea of identifying P(Y| (X=0))P( Y| ( X=0)) is to write down a Markov-factorization (actually a disintegration) P⁡(X,Y)=∫P⁡(X=x)P(Y|X=x) ( X, Y)= P(X=x)P(Y|X=x) , then to replace P⁡(X)P(X) by the intervention δ⁡[0]δ[0] singular at 00, and compute the new joint distribution. Writing μ=P⁡(X)μ=P(X), νx=P⁡(Y|X=x) _x=P(Y|X=x) (a regular version), then P⁡(X,Y)=μ⊗νP( X, Y)=μ ν. However, it also becomes clear (cf. Cor. G.2) that ν (computed form the joint) is unique only μ-almost everywhere. So the second step, replacing μ by ξ (more in line with our symmetry-guided perspective: transfer νx _x from the observational system to the query; for example ξ=δ⁡[0]ξ=δ[0]) produces a well-defined (unique) result ξ⊗νξ ν iff ξ≪μξ μ is dominated by μ ( ∀ measurable B: μ⁡(B)=0⇒ξ⁡(B)=0μ(B)=0 ξ(B)=0). For the example above 0\0\ is a measurable set and μ⁡(0)=P⁡(X∈0)=0μ(\0\)=P( X∈\0\)=0, but ξ⁡(0)=δ⁡[0](0)=1ξ(\0\)=δ[0](\0\)=1, in particular δ⁡[0]≪̸μδ[0] μ and our formalism correctly alerts us to the fact that the transfer is not well-defined. Practical interpretation is often possible by replacing ξ≪μξ μ by a regularity assumption, for example by existence of smooth densities. Given a smooth density, a limit over a Dirac-sequence can be exchanged with integration, thus δ⁡[0]δ[0] can be safely expressed as δ⁡[0]⊗νx δ[0] _x =(limn→∞δn[0])⊗νx = ( _n→∞ _n[0] ) _x =limn→∞[δn[0]⊗νx] = _n→∞[ _n[0] _x] bysmoothness. bysmoothness. So in this case as long as a Dirac-sequence with ∀n:δn[0]≪μ∀ n: _n[0] μ can be chosen, the computation is "safe". The uniqueness "ϑ⊗μ μ"-almost everywhere in our formalism can mostly (for cases like example 4.11 this problem is more complicated, depends on b≠0b≠ 0 or dimension ≥3≥ 3) be traced back to some data-set on which we learned our initial information. So it is a formal means of making this reliance on support in data explicit. On the other hand, reliance on observational support is itself a very complicated topic, and not the main topic of this paper. So we will usually "forget about" observational support and simply write μ=νμ=ν if this is true almost surely on our data. We make this transparent however in Ass. 2.23, and want to emphasize that our approach does provide a formal means of making explicit and analyzing this problem, thus sets the stage for future work on this topic. While often ignored, observational support has been understood to be relevant for learning and interpreting causal structure on multi-context systems [16; 40]. Another, less deep, aspect of our notation so far is that it tracks order of factors even though this order can be trivially changed post-hoc, thus is largely irrelevant (cf. §G.3 for details). We will usually suppress this order in notation: Notation A.7 (Equality and Transposition). We call two structured kernels μ,νμ,ν essentially equivalent μ≈νμ≈ν, if μT=νμ^T=ν (Def. G.19). For example, given two G-causal orders π,π′π,π then μ⁡(G,L,π)≈μ⁡(G,L,π′)μ(G,L,π)≈μ(G,L,π ) by Lemma G.27. We will write μ⁡(G,L)μ(G,L) for the ≈-equivalence class of any (thus all) causal orders. Remark: The reader may think of this as a writing μ⁡(G,L)μ(G,L) for μ⁡(G,L,π)μ(G,L,π) in cases where π is not of interest, with slight abuse of notation (as equality "==" on equivalence classes technically means ≈ on actual kernels). Knowing something about a system, in form of one or multiple structured kernels, we seek to understand what else can be concluded or "computed" from this information. We will revisit this question in §C.3 when discussing identifiability more generally. In the IID case, most treatments [61; 56; 57], allow for "general probability-theoretical transformations" (and the do-calculus, which is itself proved by probability-theoretical transformations employing causally mandated independencies [30]). This is in practice taken to mean "quotients" (or formally more rigorous and more generally applicable: disintegrations [8]) to compute conditional distributions, marginalizations, and certain integrals (corresponding to ⊗ -products in our case). This does, in the IID-case, not seem to lead to any confusion, as in practice there are very few available statements (§E). In the present case, with a purely measure-theoretic model (and correspondingly no troubles from trying to define suitable random elements) we are in a position where we can actually state, in a mathematically rigorous way, what exactly we mean by computation. Indeed, there is not a unique meaningful notion: It is well-understood that for example in the linear case instrumental-variable arguments can improve identification-results. This is one of many examples that can be understood as an extended notion of computation and is discussed in §5.3. We will, for the purposes of the present section, start with what in the extended notion of §5.3 is called standard-regular (computations), make, for now, no assumptions about the internal structure (like linearity) of mechanisms. The following notion of regular computations will in the IID case reproduce the results of [61; 56; 57; 4] (see §E), but applies much more generally: Definition A.8 (Regularity). We call a functional F of probability kernels (standard-)regular if F is the identity or projection to one of its arguments, or if F consists of a finite combination of ⊗ -products, left-disintegrations, right-marginalizations and transpositions. More formally F⁡[μ1,…,μn]F[ _1,…, _n] is regular if ∃F1,…,Fm∃ F^1,…,F^m such that defining μ10:=μ1μ^0_1:= _1, …, μn0=μnμ^0_n= _n, then inductively for i+1=1i+1=1 to m, let k≤n+ik≤ n+i: μki+1=μkiμ^i+1_k=μ^i_k and μn+i+1i+1=Fi+1[μ1i,…,μn+i]μ^i+1_n+i+1=F^i+1[μ^i_1,…,μ^i_n+i] where each FiF^i is either of the form μli⊗μjiμ^i_l μ^i_j or a right-marginalization of an argument μjiμ^i_j or a left-disintegration of an argument μjiμ^i_j or a transposition of an argument μjiμ^i_j and finally F⁡[μ1,…,μn]=μn+mmF[ _1,…, _n]=μ^m_n+m. Example A.9. Any finite composition of regular functionals is thus also regular. Anti-causal disintegrations can be written as a combination of a transposition and a left-disintegration, thus are regular. The general marginalization and disintegration operators can be written as a combination of left-disintegrations, anti-causal disintegrations, transpositions and right-marginalizations, thus are regular. Compositions are marginalizations of products, thus are regular. Remark A.10 (Regularity). We call these functionals regular, without actually fixing a regularity condition. The idea is as follows: In practice, given some assumptions (like smoothness etc) about mechanisms and an estimator (for example of a kernel "conditional density") suitable (converging) under those assumptions, it is usually possible to give an actual regularity condition (like smoothness in some sense) such that convergence of the estimator(s) in the arguments entails convergence of the functional. For the chosen estimator to be applicable with the formalism outlined, this estimator-specific regularity must at least include Def. A.8; in practice, additionally data-support is relevant – we cannot intervene to a range of values we have never seen (without extremely strong parametric assumptions like linearity), cf. Rmk. A.6. On generic kernels ( internal structure) no further computations should be possible, motivating Conjecture 1, a relevant details to this problem is however: Remark A.11 (Fixed-Points and Limits). Given a kernel μx _x such that [μx⊗μx′]x[ _x _x ]_x is well-defined ( Markov-kernel of a time-series), it is reasonable to ask for a limit limn→∞[⊗k=1nμ]x _n→∞[ _k=1^nμ]_x ( stationary distribuition of a time-series). There is no reason why this computation would be regular (in the sense above). However, there is also no reason why such limits would be well-defined (exist uniquely) for generic kernels, so Conjecture 1 still seems plausible, albeit for non-trivial reasons. While this clarifies what computations we consider on general kernels, it does not yet tell us how these computations on kernels, when applied to structured kernels, correspond to operations on graphs of structured kernels. A.3 Graphical Operations In this subsection, we describe how structured kernels, can be transformed and combined. To this end, we study which graphical operations correspond to regular computations (see §A.2) on the described structured kernels. Note that the main text uses notation A.7 to suppress causal orders π in these results. The main technical result is Lemma A.17 1. This is actually a result about "atoms", factors associated to individual observable nodes of a structured kernel. The full power of Lemma A.17 1 is a little obscured by its very simple and very technical claim, but it immediately implies Lemma 3.7 (c-components, cf. Lemma A.17 2), makes Lemma 3.8 (gluing, cf. Lemma A.21) astonishingly easy to state and proof and is a major ingredient in the proof of Thm. 1 (extraction from backdoor-free families, cf. §C.4). Given its central importance, the proof of Lemma A.17 1 is stated with great detail; the reader may want to first go through Rmk. A.15, which explains the general idea behind the structures studied carefully in the proof of Lemma A.17 1. We will start with sub-graphs: Given μ⁡(G,L,π)μ(G,L,π), which sub-graphs G′≤G ≤ G encode a kernel μ⁡(G′,L′,π′)μ(G ,L ,π ) that can be computed from μ⁡(G,L,π)μ(G,L,π)? Besides trivial simplifications that only delete hidden parts of the graph (possibly after first hiding them by marginalization), there is also an analogue of c-components which play an important role in the IID case [61], and their simplifications, which form an analogue of c-trees and c-forests [56; 57]. Sub-graphs (at least as they are formulated here) are local, in the sense that information about the kernel of a sub-graph G′≤G ≤ G is obtained from the kernel of G, without any reference to other possibly known information. The next construction glues two graphs with controlled overlap, that is two "smaller" graphs’ kernels provide information about the kernel associated to a larger graph containing both as sub-graphs. If both smaller graphs agree on which variables are hidden, this operation is semi-local, in the sense that it requires two, but only two, graphs as inputs. Finally, we conclude with a construction that reveals certain hidden nodes in a larger graph, by virtue of using a second smaller graph with smaller latent set. This construction seemingly is also semi-local (as is gluing). A more general revealing operation might be possible (see §A.5 for further discussion), in which case gluing on revealed parts might become truly non-local. We start by a more detailed version of Lemma . 3.5. Lemma A.12 (Simplified Graphs). Given a structural model (G,L)(G,L), then for any L′⊂ innerL _ inner, μ⁡(G,L∪L′)=margL′∖L⁡(μ⁡(G,L))μ(G,L∪ L )= _L L(μ(G,L)) can be regularly computed from μ⁡(G,L)μ(G,L). Further, for a sub-set B⊂LB⊂ L with (B)⊂B _G(B)⊂ B, there is a G-causal order π putting B at the end, that ∀n∈∀b∈B∀ n ∀ b∈ B: (n≥b⇒n∈B)(n≥ b n∈ B). Using this π, the sub-graph G′≤G ≤ G with nodes inner′= inner∖BN_ inner =N_ inner B and outer′= ( inner′)∖ inner′⊂ outerN_ outer = _G(N_ inner ) _ inner _ outer (with B⊂L⊂ innerB⊂ L _ inner removing B only removes inner nodes, however, some of the original arguments, nodes, may only be relevant to nodes in B, in which case we can discard them) satisfies μ⁡(G′,L∖B,π)=μ⁡(G,L,π) .μ(G ,L B,π)=μ(G,L,π) . We call (G′,L∖B)(G ,L B) a direct simplification of (G,L)(G,L). There is a unique maximal such B (in the sense that any B′B satisfying the hypothesis is a subset B′⊂B ⊂ B), and we call the corresponding (minimal) G′G simplified. The maximal B can be constructed in practice as described by the proof. Combining both, given a sub-set B⊂ innerB _ inner with (B)⊂B _G(B)⊂ B, define L′:=B∖L :=B L, and again the sub-graph G′≤G ≤ G with nodes inner′= inner∖BN_ inner =N_ inner B and outer′= ( inner′)∖ inner′⊂ outerN_ outer = _G(N_ inner ) _ inner _ outer, then μ⁡(′,L∖B)μ(G ,L B) can be regularly computed from μ⁡(,L)μ(G,L), we call this construction a simplification. proof The first claim follows directly by definition, the last claim follows from the first two claims, thus we proof the remaining second claim. For the existence of π, first choose a πAπ^A of ∖BN B (compatible with the ancestral partial order) and πBπ^B of B (compatible with the ancestral partial order). Define the total order π ( ≤π _π) as follows: Given arbitrary n,n′∈n,n , if both n,n′∈Bn,n ∈ B, define n≤πn′:⇔n≤πBn′n _πn : n _π^Bn , if both n,n′∉Bn,n ∉ B, define n≤πn′:⇔n≤πAn′n _πn : n _π^An , otherwise (one of n,n′n,n is in B, one is not), define n≤πn′:⇔n′∈Bn _πn : n ∈ B ( nodes not in B before those in B). This total order is compatible with the ancestral partial order on ∖BN B and B because πAπ^A and πBπ^B are, on mixed terms because (B)⊂B _G(B)⊂ B ( n,n′n,n with exactly one of them in B are ancestral-comparable, then they must compare as the one not in B ordered after the one in B). By construction π satisfies ∀n∈∀b∈B∀ n ∀ b∈ B: (n≥πb⇒n∈B)(n _πb n∈ B). We will drop the subscript π on comparison operators for the remainder of this proof. Using N:= inner∪( outer∩L)N:=N_ inner∪(N_ outer∩ L) we compute: μ⁡(G,L,π) μ(G,L,π) =margL(⊗n∈Nπμn) = _L( ^π_n∈ Nμ^n) (bydefinition) (bydefinition) =margL([⊗n∈N∖Bπμn]⊗[⊗n∈Bπμn]) = _L([ ^π_n∈ N Bμ^n] [ ^π_n∈ Bμ^n]) (bypropertyofπ) (bypropertyof$π$) =margL∖B([⊗n∈N∖Bπμn]) = _L B([ ^π_n∈ N Bμ^n]) (trivialmarg.onrhs) (trivialmarg.\ onrhs) =μ⁡(G′,L∖B,π) =μ(G ,L B,π) (bydefinition). (bydefinition). To construct a maximal B, filter L by longest descendant-chains, (0):=n∈L| (n)=∅B^(0):=\n∈ L| (n)= \ and inductively B(k+1):=n∈L| (n)⊂B(k)B^(k+1):=\n∈ L| (n)⊂ B^(k)\ (note that B(k)⊂B(k+1)B^(k)⊂ B^(k+1)). Define B:=B(∞):=∪kB(k)B:=B^(∞):= _kB^(k) (since L is finite, obviously there is a finite number m where this filtration becomes stationary and B(∞)=B(m)B^(∞)=B^(m) is not a "real" limit, just a convenient notation). Let B′B satisfying the hypothesis be arbitrary. We show B′⊂B(∞)B ⊂ B^(∞). Since B′B satisfies the hypothesis B′⊂LB ⊂ L and (B′)⊂B′⊂L (B )⊂ B ⊂ L. Define (B′)(0):=n∈B′| (n)=∅(B )^(0):=\n∈ B | (n)= \, then (B′)(0)⊂B(0)(B )^(0)⊂ B^(0) (because B′⊂LB ⊂ L). Define inductively (B′)(k+1):=n∈B′| (n)⊂(B′)(k)(B )^(k+1):=\n∈ B | (n)⊂(B )^(k)\, then (inductively, (B′)(k)⊂B(k)(B )^(k)⊂ B^(k)) we have for arbitrary n∈(B′)(k+1)n∈(B )^(k+1) that n∈B′⊂Ln∈ B ⊂ L and that (n)⊂(B′)(k)⊂B(k) (n)⊂(B )^(k)⊂ B^(k), thus n∈B(k+1)n∈ B^(k+1), showing (B′)(k+1)⊂B(k+1)(B )^(k+1)⊂ B^(k+1); after a finite number of steps (B′)(∞)=(B′)(m)⊂B(m)⊂B(∞)(B )^(∞)=(B )^(m)⊂ B^(m)⊂ B^(∞). Finally, by (B′)⊂B′ (B )⊂ B (and by finiteness of N) B′=(B′)(∞)B =(B )^(∞). The main technical ingredient for proving the more sophisticated statements below is a decomposition of the product describing μ⁡(G,L,π)μ(G,L,π) in per-observable-node factors or "atoms": For each observed inner node n, we define AnA^n for a causal order π on nodes by disintegrating from the left until we reach n and marginalizing from the right until we reach n (by Lemma G.23 3 the order in which this is executed does not matter). Definition A.13 (Atoms). Given a structural model (G,L)(G,L), a G-causal order π, then for n∈ inner∖Ln _ inner L, we define the atom at n as An(G,L,π):=disintn′∈ inner∖L|n′<πnmargL∪n′∈ inner∖L|n′>πn(μ(G,L,π)) .A^n(G,L,π):= _\n _ inner L|n <_πn\ _L∪\n _ inner L|n >_πn\(μ(G,L,π)) . These atoms by construction are regularly computable for a given structured kernel and turn out to contain all relevant information to reconstruct the structured kernel: Lemma A.14 (Properties of Atoms). Given a structural model (G,L)(G,L) and a G-causal order π then: [label=()] 1. An(G,L,π)A^n(G,L,π) is a regular functional of μ⁡(G,L,π)μ(G,L,π). 2. μ(G,L,π)=⊗n∈ inner∖LπAn(G,L,π) .μ(G,L,π)= ^π_n _ inner LA^n(G,L,π) . 3. μ⁡(G,L,π)μ(G,L,π) is a regular functional of its atoms. 4. Given B⊂ innerB _ inner with (B)⊂B _G(B)⊂ B, and π the G-causal order and G′≤G ≤ G are the results produced by Lemma A.12, then ∀n∈ inner′∖L′∀ n _ inner L : An(G,L,π)=An(G′,L′,π′)A^n(G,L,π)=A^n(G ,L ,π ). proof Part 1: Disintegrations and marginaliztions are regular, so are their compositions. Part 2: Inductively over n∈ inner∖Ln _ inner L along π we show: Using Ln:=L∪n′∈ inner|n′>πnL^n:=L∪\n _ inner|n >_πn\ and ≤n:=n′∈ inner∖L|n′≤πnN^≤ n:=\n _ inner L|n _πn\, μ(G,Ln,π)=⊗n′∈≤nπAn′(G,L,π) .μ(G,L^n,π)= ^π_n ^≤ nA^n (G,L,π) . For the start of induction (the first, in π-order, n∈ inner∖Ln _ inner L): By definition An(G,L,π)=margL([⊗l∈ (n)μl]⊗μn)A^n(G,L,π) = _L([ _l∈ _G(n)μ^l] μ^n), where ancestors of n are in L (by n being the first observed inner node) by trivial marginalization on the right this is μ⁡(G,Ln,π)μ(G,L^n,π). Inductive step (n to "n+1n+1", where n+1n+1 is the next, in π-order, node in inner∖LN_ inner L after n): By the characterizing property of the disintegration (Lemma G.22), μ⁡(G,Ln,π)⊗An+1(G,L,π)=μ⁡(G,Ln+1,π)μ(G,L^n,π) A^n+1(G,L,π)=μ(G,L^n+1,π). By inductive hypothesis μ(G,Ln,π)=⊗n′≤nπAn′(G,L,π)μ(G,L^n,π)= ^π_n N^≤ nA^n (G,L,π), thus proving the claim. Part 3: Immediate by 2 (products are regular). Part 4: Intuitively, since π puts B at the end of the product in 2, and μ⁡(′,L′,π′)=margB∖L⁡(μ⁡(,L,π))μ(G ,L ,π )= _B L(μ(G,L,π)) by Lemma A.12 is given by trivial marginalizations on the right this follows by comparing terms in the representation by 2. Formally, for example use μ⁡(′,L′,π′)=margB∖L⁡(μ⁡(,L,π))μ(G ,L ,π )= _B L(μ(G,L,π)) by Lemma A.12, then Lemma G.21 and Lemma G.23 1 and the definition of atoms. Remark A.15 (Computation of Atoms I). We briefly illustrate how atoms are related to the kernels μnμ^n associated to nodes n of the graph (a more formal version can be found in the proof of Lemma A.17). First, we associate the μnμ^n into groups LiL_i (if they are in L) and YiY_i (otherwise) and write (up to transposition, denoted "≈", notation A.7) ⊗nμn ≈ [L1⊗Y1]⊗…⊗[Lm⊗Ym] _nμ^n ≈ [L_1 Y_1] … [L_m Y_m] We can, without loss of generality, assume that each YiY_i-term is exactly a single node (by injecting empty LiL_i-terms with the understanding that μ⊗∅=μ =μ by convention). We reorder this successively by moving L-terms further to the right. First, L1⊗Y1=(Y1∘L1)⊗(L1|Y1)L_1 Y_1=(Y_1 L_1) (L_1|Y_1). We write L1≤:=L1L_1^≤:=L_1, then this reads L1⊗Y1=(Y1∘L1≤)⊗(L1≤|Y1)L_1 Y_1=(Y_1 L_1^≤) (L_1^≤|Y_1) The next term in the expression for ⊗nμn _nμ^n is L2L_2, we collect this term into L2≤:=L1≤⊗L2L_2^≤:=L_1^≤ L_2 and swap it with Y2Y_2 to get [L1⊗Y1]⊗[L2⊗Y2] [L_1 Y_1] [L_2 Y_2] ≈(Y1∘L1≤)⊗L2≤⊗Y2 ≈(Y_1 L_1^≤) L_2^≤ Y_2 ≈(Y1∘L1≤)⊗(Y2∘L2≤)⊗(L2≤|Y2) . ≈(Y_1 L_1^≤) (Y_2 L_2^≤) (L_2^≤|Y_2) . Then inductively define Lk+1≤:=Lk≤⊗Lk+1L_k+1^≤:=L_k^≤ L_k+1 and swap to get ⊗nμn ≈ (Y1∘L1≤)⊗…⊗(Ym∘Lm≤)⊗Lm+1≤ . _nμ^n ≈ (Y_1 L_1^≤) … (Y_m L_m^≤) L_m+1^≤ . This expression can be trivially marginalized at the right (all elements of L were transposed to the right and were collected into the product Lm+1≤L^≤_m+1) to compute μ(G,L,π)=margL(⊗nμn)≈(Y1∘L1≤)⊗…⊗(Ym∘Lm≤) .μ(G,L,π)= _L( _nμ^n)≈(Y_1 L^≤_1) … (Y_m L^≤_m) . Disintegrating observable nodes from left to right is now a trivial disintegration on the left, thus we can read of An(G,L,π)=Yn∘Ln≤ .A^n(G,L,π)=Y_n L^≤_n . We have not yet considered which arguments these terms actually carry, we will investigate this in the proof of Lemma A.17, and continue in Rmk. A.18. Similar to the IID-case [61], c-components of graphs (in the sense of the next definition) play an important role for computing results, detailing Def. 3.6: Definition A.16 (Local C-Component). Given a structural model (G,L)(G,L), we call l∈Ll∈ L a hidden confounder of y,w∈ inner∖Ly,w _ inner L if there are directed paths γy _y from l to y and γw _w from l to w such that all non-endpoint nodes of γy,γw _y, _w are in L. On inner∖LN_ inner L define a relation y∼w:⇔∃y w: ∃ hidden confounder l of y,wy,w. Define ∼L _L as the equivalence relation generated by ∼ (the "smallest" equivalence relation with y∼w⇒y∼Lwy w y _Lw, since ∼ is already symmetric, this amounts to trivially making ∼ reflective and transitively closing it, y∼Lwy _Lw if y=wy=w or if y∼z1∼…∼zn∼wy z_1 … z_n w). We call equivalence-classes inner∖L/∼L N_ inner L _L c-components of (G,L)(G,L) and say (G,L)(G,L) is c-connected if there is only a single c-component, | inner∖L/∼L|=1| N_ inner L _L|=1. We can extend this notion to innerN_ inner by defining l≈yl≈ y for y∈ inner∖Ly _ inner L if there exists a directed path γy _y from l to y such that all non-endpoint nodes of γy _y are in L. Then each l∈ ( inner∖L)∩Ll∈ _G(N_ inner L)∩ L is equivalent to at least one y, and if it is equivalent via γw _w to another w∈ inner∖Lw _ inner L, then y∼wy w via (γy,γw)( _y, _w). Thus ≈ and ∼ together ( nodes n, n′n are related if n≈n′n≈ n or n∼n′n n ), then generates an equivalence relation ≈L _L on ( inner∖L) _G(N_ inner L) with the same number of equivalence-classes as ∼L _L (indeed the intersections of ≈L _L equivalence-classes of ( inner∖L) _G(N_ inner L) with inner∖LN_ inner L are exactly the ∼L _L equivalence-classes; every c-component thus has a unique associated ≈L _L equivalence class, containing additional hidden ancestors). Finally, we formally make all elements of L∖ ( inner∖L)L _G(N_ inner L) equivalent, this adds zero or one additional classes (depending on whether L∖ ( inner∖L)=∅L _G(N_ inner L)= ) witch we call a trivial c-component. A structural c-component is a subgraph Gc≤G^c≤ G containing as inner nodes innercN_ inner^c the elements of a non-trivial ≈L _L-equivalence class and as outer nodes outerc= ( innerc)∖ innercN_ outer^c= _G(N_ inner^c) _ inner^c. By slight abuse of notation, we also refer to the structural model (Gc,Lc)(G^c,L^c) with Lc:=L∩cL^c:=L ^c as structural c-component; note that by construction of ≈L _L, innercN_ inner^c is closed under latent parents, thus Lc⊂ innercL^c _ inner^c. These c-components are a direct analogue of c-components in the IID-case which are well-understood to be of great importance to the question of identifiability of interventional distributions [61]. Similarly we find also here (note that part (b) is Lemma 3.7): Lemma A.17 (Properties of C-Components). Given a structural model (G,L)(G,L), a G-causal order π and a c-component (Gc,Lc)≤(G,L)(G^c,L^c)≤(G,L) with GcG^c-causal order πc=π|π^c=π| then: [label=()] 1. If n∈ innerc∖Lcn _ inner^c L^c, then An(Gc,Lc,πc)=An(G,L,π)A^n(G^c,L^c,π^c)=A^n(G,L,π). 2. μ⁡(Gc,Lc,πc)μ(G^c,L^c,π^c) is a regular functional of μ⁡(G,L,π)μ(G,L,π). 3. μ⁡(G,L,π)μ(G,L,π) is a regular functional of jointly all non-trivial c-components’ μ⁡(Gc,Lc,πc)μ(G^c,L^c,π^c). proof We follow the idea outlined in Rmk. A.15, but carefully track kernel-arguments. For any fixed structural c-component, we will be able split the L≤L^≤ terms of the remark into terms L≤L^≤ relevant to that particular c-component and terms W≤W^≤ irrelevant to that particular c-component. We write C= innerc∖Lc⊂ inner∖LC=N_ inner^c L^c _ inner L for the c-component with associated structural c-component GcG^c. We may assume there are only non-trivial structural c-components (otherwise simplify, Lemma A.12). We may assume, without loss of generality that the π-ordered inner=1,…,mN_ inner=\1,…,m\. Thus μ⁡(G,L,π)=margL⁡(μ1⊗…⊗μm)μ(G,L,π)= _L(μ^1 … μ^m). To simplify notation, we define "empty" kernels L0≤=∅L_0^≤= and W0≤=∅W_0^≤= (with the understanding that ∀μ∀μ: ∅⊗μ=μ⊗∅=μ μ=μ =μ and μ∘∅=μ =μ; this is merely for notational convenience), then we inductively for k=1,…,mk=1,…,m construct Lk≤,Wk≤L_k^≤,W_k^≤ with the properties (to be shown below): [label=()] 1. Lk≤L_k^≤ and Wk≤W_k^≤ have arguments only in observed nodes (nodes not in L). 2. Lk≤L_k^≤ and Wk≤W_k^≤ contain exactly those factors associated to nodes n≤kn≤ k with n∈Lcn∈ L^c and n∈L∖Lcn∈ L L^c respectively ( transposition, Def. G.19, T in (c) is such that it puts these terms to the right, see also proof below). 3. (⊗i≤kμi)T=[⊗i∉L,i≤k(μi∘Zi−1≤)]⊗Lk≤⊗Wk≤( _i≤ kμ^i)^T=[ _i∉ L,i≤ k(μ^i Z_i-1^≤)] L_k^≤ W_k^≤, where Zi−1≤=Li−1≤Z_i-1^≤=L_i-1^≤ if i∈Ci∈ C, and Zi−1≤=Wi−1≤Z_i-1^≤=W_i-1^≤ otherwise. For the inductive start at k=0k=0, define L0≤=∅L^≤_0= , W0≤=∅W^≤_0= , then (a,b) are trivial, defining the empty ⊗ -product as ∅ also (c) is trivial (and this will be consistent, as for k=1k=1, the product containing only μ1μ^1 by definition is μ1=∅⊗μ1μ^1= μ^1). For the inductive step (k−1↦k≥1k-1 k≥ 1), we note that for each k=1,…,mk=1,…,m we are in exactly one of the following cases: [label=0)] 1. Case k∈Lck∈ L^c: Define Wk≤:=Wk−1≤W_k^≤:=W_k-1^≤ and Lk≤:=Lk−1≤⊗μkL_k^≤:=L_k-1^≤ μ^k. Arguments of Lk≤L_k^≤ are those of Lk−1≤L_k-1^≤ plus those of μkμ^k, (k) (k) (parents of inner nodes in GcG^c agree with those in G, cf. Def. 3.4). Note that (k)∩L⊂Lc (k)∩ L⊂ L^c: Let p∈ (k)∩Lp∈ (k)∩ L be arbitrary. Since k∈Lc⊂ innerck∈ L^c _ inner^c, k≈Lyk _Ly for some y∈Cy∈ C, is a path γ:k↝yγ:k y with non-endpoints in L. Prefixing this path with the edge p→kp→ k we get p≈Lyp _Ly, thus p∈ innercp _ inner^c. Since also p∈Lp∈ L, and Lc=L∩ innercL^c=L _ inner^c by definition, p∈Lcp∈ L^c. We show (a): Arguments of Wk≤W_k^≤ and Lk−1≤L_k-1^≤ already satisfy (a) by inductive hypothesis (a). Given p∈ (k)∩Lp∈ (k)∩ L, it remains to show p is not an argument of Lk≤L_k^≤, is contracted with Lk−1≤L_k-1^≤. Clearly parents are ancestors, thus π-before k, <πkp<_πk, thus this parent p≤kp≤ k with p∈Lcp∈ L^c is contracted into Lk−1≤L_k-1^≤ by inductive hypothesis (b). We show (b): By inductive hypothesis (b) this is true for n<πkn<_πk. We absorbed μkμ^k into Lk≤:=Lk−1≤⊗μkL_k^≤:=L_k-1^≤ μ^k, thus it remains true for n≤πkn _πk. We show (c): Plugging in the inductive hypothesis (c), (⊗i≤kμi)T=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗Wk−1≤⊗μk( _i≤ kμ^i)^T=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ W_k-1^≤ μ^k. The new term μkμ^k has only observed arguments and arguments in LcL^c (a; shown above) while Wk≤W_k^≤ contains only hidden node factors in L∖LcL L^c (b; shown above), so Wk−1≤⊗μk=μk⊗Wk−1≤W_k-1^≤ μ^k=μ^k W_k-1^≤ by sparsity, Lemma G.27. Thus (c) follows with the definition of Lk≤=Lk−1≤⊗μkL_k^≤=L_k-1^≤ μ^k. 2. Case k∈Ck∈ C: Define Wk≤:=Wk−1≤W_k^≤:=W_k-1^≤ and Lk≤:=(Lk−1≤|μk)L_k^≤:=(L_k-1^≤|μ^k). We show (a): By inductive hypothesis arguments of Wk≤W_k^≤ and Lk−1≤L_k-1^≤ are in observed nodes. Lk≤:=(Lk−1≤|μk)L_k^≤:=(L_k-1^≤|μ^k) has additional arguments k and (k)∖Lc (k) L^c (from the disintegration, Lemma G.27 4 and using the inductive hypothesis (b)). k itself is observed by case-hypothesis. (k) (k) are either observed or in LcL^c by definition of GcG^c, so (k)∖Lc (k) L^c are observed. We show (b): Since k is observed by case-hypothesis the claim of (b) is already given by the inductive hypothesis (b). We show (c): Plugging in the inductive hypothesis (c), (⊗i≤kμi)T=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗Wk−1≤⊗μk( _i≤ kμ^i)^T=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ W_k-1^≤ μ^k. All hidden arguments of the new term μkμ^k are in LcL^c (by definition of ≈L _L and cG^c) while Wk≤W_k^≤ contains only hidden node factors in L∖LcL L^c (b, shown above), so Wk−1≤⊗μk=μk⊗Wk≤W_k-1^≤ μ^k=μ^k W_k^≤ by sparsity, Lemma G.27. By anti-causal disintegration [⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗μk⊗Wk−1≤=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗(μk∘Lk−1≤)⊗(Lk−1≤|μk)⊗Wk−1≤[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ μ^k W_k-1^≤=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] (μ^k L_k-1^≤) (L_k-1^≤|μ^k) W_k-1^≤. Thus (c) follows with the definition of Lk≤L_k^≤. 3. Case k∈L∖Lck∈ L L^c, define Wk≤:=Wk−1≤⊗μkW_k^≤:=W_k-1^≤ μ^k and Lk≤=Lk−1≤L_k^≤=L_k-1^≤. Arguments of Wk≤W_k^≤ are those of Wk−1≤W_k-1^≤ plus those of μkμ^k, (k) (k). Note that (k)∩Lc=∅ (k)∩ L^c= : By contradiction. If there were a p∈ (k)∩Lcp∈ (k)∩ L^c, then there is a latent path γy:p↝y _y:p y for some observed y∈Cy∈ C. Since k∉ innerck _ inner^c is in a non-trivial ( simplify, see above) structural c-component associated to k∈C′≠Ck∈ C ≠ C, there is a latent path γw:k↝w _w:k w for some observed w∈C′∖Lw∈ C L. Prefix this path with the edge p→kp→ k to obtain the latent path γw′:p↝w _w :p w. Using (γy,γw′)( _y, _w ), thus y∼Lwy _Lw and C′=C =C contradicting C′≠C ≠ C. We show (a): Arguments of Wk−1≤W_k-1^≤ and Lk≤L_k^≤ are observed by inductive hypothesis (a). Clearly parents are ancestors, thus π-before k, <πkp<_πk and if p∈Lp∈ L, then p∉Lcp∉ L^c (see above), thus such arguments are contracted into Wk−1≤W_k-1^≤. We show (b): For factors n<kn<k this is true by inductive hypothesis (b). For n=kn=k it remains true by definition of Wk≤W_k^≤. We show (c): Property (c) holds immediately by inductive hypothesis and definition of Wk≤W_k^≤. 4. Case k∉C∪Lk∉ C∪ L: Define Wk≤:=(Wk−1≤|μk)W_k^≤:=(W_k-1^≤|μ^k) and Lk≤:=Lk−1≤L_k^≤:=L_k-1^≤. We show (a): By inductive hypothesis arguments of Wk≤W_k^≤ and Lk−1≤L_k-1^≤ are observed nodes. Wk≤:=(Wk−1≤|μk)W_k^≤:=(W_k-1^≤|μ^k) has a additional arguments k and (k)∖(L∖Lc) (k) (L L^c) (from the disintegration, Lemma G.27 4 and using the inductive hypothesis (b)). k itself is observed by case-hypothesis. (k) (k) are either observed or in L∖LcL L^c by definition of GcG^c, so (k)∖(L∖Lc) (k) (L L^c) are observed. We show (b): Since k is observed by case-hypothesis the claim of (b) is already given by the inductive hypothesis (b). We show (c): Plugging in the inductive hypothesis (c), (⊗i≤kμi)T=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗Wk−1≤⊗μk( _i≤ kμ^i)^T=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ W_k-1^≤ μ^k. By anti-causal disintegration [⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗Wk−1≤⊗μk=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk−1≤⊗(μk∘Wk−1≤)⊗(Wk−1≤|μk)=[⊗i∉L,i≤k−1(μi∘Zi−1≤)]⊗Lk≤⊗(μk∘Wk−1≤)⊗Wk≤[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ W_k-1^≤ μ^k=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k-1^≤ (μ^k W_k-1^≤) (W_k-1^≤|μ^k)=[ _i∉ L,i≤ k-1(μ^i Z_i-1^≤)] L_k^≤ (μ^k W_k-1^≤) W_k^≤ by definitions of Lk≤L_k^≤ and Wk≤W_k^≤. Wk−1≤W_k-1^≤ has no hidden arguments (a; shown above) and all hidden arguments of the new term μkμ^k are in L∖LcL L^c (by definition of ≈L _L and cG^c) thus all hidden arguments of μk∘Wk−1≤μ^k W_k-1^≤ are in L∖LcL L^c. On the other hand Lk≤L_k^≤ contains only hidden node factors in LcL^c (b, shown above), so Lk≤⊗(μk∘Wk−1≤)=(μk∘Wk−1≤)⊗Lk≤L_k^≤ (μ^k W_k-1^≤)=(μ^k W_k-1^≤) L_k^≤ by sparsity, Lemma G.27. Thus (c) follows. In the end, (after m-steps), we obtain by property (c): ⊗iμi=[⊗i∉L(μi∘Zi−1≤)]⊗Lm≤⊗Wm≤ _iμ^i=[ _i∉ L(μ^i Z_i-1^≤)] L_m^≤ W_m^≤, where Zi−1≤=Li−1≤Z_i-1^≤=L_i-1^≤ if i∈Ci∈ C, and Zi−1≤=Wi−1≤Z_i-1^≤=W_i-1^≤ otherwise. By property (b), Lm≤⊗Wm≤L_m^≤ W_m^≤ contains (as factors) exactly the term in L, thus μ(G,L,π)=margL(⊗iμi)=⊗i∉L(μi∘Zi−1≤) .μ(G,L,π)= _L( _iμ^i)= _i∉ L(μ^i Z_i-1^≤) . By trivially marginalizing on the right, then trivially disintegrating from the left we get: Ai(G,L,π)=(μi∘Zi−1≤) .A^i(G,L,π)=(μ^i Z_i-1^≤) . (∗*) Next we apply the same process to GcG^c instead of G. By definition, GcG^c contains exactly the k∈ck ^c, thus cases 1) and 2) are invoked exactly as before, we obtain the same Lk≤L_k^≤, while steps 3) and 4) cannot occur, thus all Wk≤W_k^≤ are trivial. We thereby obtain: μ(Gc,Lc,πc)=margLc(⊗i∈C∪Lcμi)=⊗i∈C(μi∘Zi−1≤) .μ(G^c,L^c,π^c)= _L^c( _i∈ C∪ L^cμ^i)= _i∈ C(μ^i Z_i-1^≤) . By trivially marginalizing on the right, then trivially disintegrating from the left we get for i∈Ci∈ C: Ai(Gc,Lc,πc)=(μi∘Li−1≤)=(∗),i∈CAi(G,L,π) .A^i(G^c,L^c,π^c)=(μ^i L_i-1^≤) (*),i∈ C=A^i(G,L,π) . This proves part (a) of the lemma. Parts (b) and (c) follow immediately with Lemma A.14. Remark A.18 (Computation of Atoms I). Fundamentally, atoms are computable essentially by definition via disintegrations from the left and marginalizations from the right. Marginalizations from the right are trivial in practice: They just drop the last factor. Disintegrations on the left can be complicated (they are like estimating conditional probabilities). If we want to compute an atom An(,L,π)A^n(G,L,π) in practice, then Lemma A.17 1 and its proof tells us something about the actual arguments. That is, we drop terms strictly π-after n, then disintegrate those strictly π-before n, but since we know that the result is actually a function (or kernel) in arguments C∪ (C)C∪ (C) (where C is the c-component containing n), we can simply ignore all other disintegrated nodes and shared arguments. Thus the kernel An(,L,π)A^n(G,L,π) in practice gets harder to estimate if the c-component is large, but at least it depends only on the c-component (and its parents). Estimating each atom in C relative to its ancestors in C and parents of C is of course very closely related to estimating the joint distribution of nodes in C by the (not simplifiable in this case, cf. [61]) Markov-factorization of P(C| (C)∖C)P(C| (C) C). Example: For the napkin graph (cf. Fig. 6) in the IID case, with causal order π=(L1,L2,W,Z,X,Y)π=(L_1,L_2,W,Z,X,Y) the atom AwA^w is simply P⁡(W)=W[l1,l2]∘[L1⊗L2]P(W)=W_[l_1,l_2] [L_1 L_2], the atom AzA^z is simply P⁡(Z|W=w)=ZwP(Z|W=w)=Z_w, the atom AxA^x is P⁡(X|W=w,Z=z)P(X|W=w,Z=z) (because Z is a parent and W is an ancestor in the same c-component), but note that P⁡(X|W=w,Z=z)=(Xz,[l1]∘(L1|W)w)w,zP(X|W=w,Z=z)= (X_z,[l_1] (L_1|W)_w )_w,z actually contains an anti-causal term! The atom at AyA^y is P⁡(Y|W=w,Z=z,X=x)P(Y|W=w,Z=z,X=x) (because W,XW,X are ancestors in the same c-component, Z is additionally a parent of the c-component). The dependence is non-trivial in Z, because Y Z|W,XY Z|W,X, due to the colliders at W,XW,X there is an open path Z→X↔W↔YZ→ X W Y. To estimate AyA^y from data, we only have to know its arguments (see above), yet the question remains: What is AyA^y as expressed by model-kernels? Tracing Rmk. A.15 and the proof of the lemma we find (cf. π given above) L1≤=L1L_1^≤=L_1, L2≤=L1⊗L2L_2^≤=L_1 L_2, from W∈CW∈ C at index 33 we get L3≤=(L2≤|W)w=(L1⊗L2|W)wL_3^≤=(L_2^≤|W)_w=(L_1 L_2|W)_w from Z∉CZ∉ C at index 44 simply L4≤=L3≤L_4^≤=L_3^≤, from X∈CX∈ C at index 55 we get and L5≤=((L4≤)w|Xz,[l1])w,z,x=((L1⊗L2|W)w|Xz,[l1])w,z,xL_5^≤=((L_4^≤)_w|X_z,[l_1])_w,z,x=((L_1 L_2|W)_w|X_z,[l_1])_w,z,x so finally: P⁡(Y|W=w,Z=z,X=x)=Yx,[l2]∘L5≤=Yx,[l2]∘((L1⊗L2|W)w|Xz,[l1])w,z,x .P(Y|W=w,Z=z,X=x)=Y_x,[l_2] L_5^≤=Y_x,[l_2] ((L_1 L_2|W)_w|X_z,[l_1])_w,z,x . Computing μ⁡(c,Lc,π)μ(G^c,L^c,π) for the c-component containing W,X,YW,X,Y from these atoms in the form Aw⊗Ax⊗AyA^w A^x A^y, through the definition of the ⊗ -product (Def. G.8) contains an integral over W (if we then marginalize W, this simply "integrates out" W) which corresponds to the well-known integrating out of the adjustment set W\W\ for the effect P(X,Y| (Z=z))P( X, Y| ( Z=z)) (we study an IID example after all). In case the reader was wondering, why we bothered introducing graphical operations, they might take note that in Fig. 3 it is almost immediately evident that the c-subgraph marginalizing W from cG^c has two c-components and can be split further, actually can compute the much simpler Yx,[l2]∘L2Y_x,[l_2] L_2. Indeed as pointed out above, we know the arguments of atoms, thus we may estimate Ax=P⁡(X|W=w,Z=z)A^x=P(X|W=w,Z=z) and Ay=P⁡(Y|W=w,Z=z,X=x)A^y=P(Y|W=w,Z=z,X=x), then marginalizing W means computing the integral Pz′(X,Y)=∫P⁡(W=w)P(X|W=w,Z=z)P(Y|W=w,Z=z,X=x)P _z(X,Y)= P(W=w)P(X|W=w,Z=z)P(Y|W=w,Z=z,X=x), finally we know the argument of the (new) atoms, and (Ay)′(A^y) has argument X (cf. Fig. 3), thus we can estimate (Ay)′=Pz′(Y|X=x)(A^y) =P _z(Y|X=x) for any z. In practice to use data optimally we will probably want to integrate z over the observational distribution P⁡(Z)P(Z). Note that from Fig. 3 it is also clear that we can compute P(Y| (X=x))P(Y| (X=x)) by gluing a δ⁡[x]∈ℱ interveneδ[x] _ intervene (which, cf. proof of Lemma A.21 and Rmk. A.22, is simply a product with AyA^y), the product with δ⁡[x]δ[x] in turn is the same as plugging x in for X, thus (if we are willing to write the disintegration as a quotient, which is common but may not be a good idea in general [8]): P⁡(Y| (X=x))=∫P⁡(W=w)P(X|W=w,Z=z)P(Y|W=w,Z=z,X=x) ∫P⁡(W=w)P(X|W=w,Z=z) (Y| (X=x))= P(W=w)P(X|W=w,Z=z)P(Y|W=w,Z=z,X=x) P(W=w)P(X|W=w,Z=z) This is of course the same result commonly obtained using do-calculus. As illustrated in Fig. 3, simplifications of c-components (and more generally repeated identification of c-components and their simplifications) is often relevant: Definition A.19 (c-Subgraphs). Given a structural model (G,L)(G,L), and a c-component (Gc,Lc)≤(G,L)(G^c,L^c)≤(G,L), we call a simplification (Def. A.12) (G′,Lc∖B)(G ,L^c B) of (Gc,Lc)(G^c,L^c) a c-subgraph. Corollary A.20. Given a structural model (G,L)(G,L), and a c-subgraph (G′,L′)≤(G,L)(G ,L )≤(G,L), then μ⁡(G′,L′,π′)μ(G ,L ,π ) is a regular functional of μ⁡(G,L,π)μ(G,L,π). proof Apply Lemma A.17 2, then Lemma A.12. Given suitable overlap, two graphs (with know kernels) can be glued to a larger graph with regularly computable kernel. The next result is Lemma 3.8: Lemma A.21 (Gluing). Given a structural model (G,L)(G,L), a G-causal order π (with restrictions πAπ^A, πBπ^B respectively) and subgraphs GA,GB≤G^A,G^B≤ G, such that each structural c-component GcG^c is contained in at least one subgraph as Gc≤GAG^c≤ G^A or Gc≤GBG^c≤ G^B then there is a regular functional F∪[μ⁡(GA,L∩A,πA),μ⁡(GB,L∩B,πB)]F_∪[μ(G^A,L ^A,π^A),μ(G^B,L ^B,π^B)] which we denote as GA∪(G,L)GBG^A _(G,L)G^B computing μ⁡(G,L,π)μ(G,L,π), μ(G,L,π) = ∪(G,L)GB .μ(G,L,π) = G^A _(G,L)G^B . proof Let n∈ inner∖Ln _ inner L be arbitrary. Let Gc≤G^c≤ G be the structural c-component associated to the c-component of n ( unique structural c-component with n∈ innercn _ inner^c). By Lemma A.17 1, An(G,L,π)=An(Gc,Lc,πc)A^n(G,L,π)=A^n(G^c,L^c,π^c). By hypothesis GcG^c is contained in at least one of GAG^A or GBG^B. If Gc≤GAG^c≤ G^A, then Lemma A.17 1 applies to GAG^A and yields An(GA,LA,πA)=An(Gc,Lc,πc)A^n(G^A,L^A,π^A)=A^n(G^c,L^c,π^c). By Lemma A.14 1, An(G,L,π)A^n(G,L,π) is thus a regular functional of μ⁡(GA,LA,πA)μ(G^A,L^A,π^A). If Gc≤GBG^c≤ G^B, the same argument yields An(G,L,π)A^n(G,L,π) as a regular functional of μ⁡(GB,LB,πB)μ(G^B,L^B,π^B). Since n∈ inner∖Ln _ inner L was arbitrary, we thus have regular functionals computing all the (finitely many) An(G,L,π)A^n(G,L,π) from μ⁡(GA,LA,πA)μ(G^A,L^A,π^A) or μ⁡(GB,LB,πB)μ(G^B,L^B,π^B). By Lemma A.14 3, μ⁡(G,L,π)μ(G,L,π) is a regular functional of these An(G,L,π)A^n(G,L,π). Since composition of regular functionals is regular, thus μ⁡(G,L,π)μ(G,L,π) is a regular functional of μ⁡(GA,L∩A,πA)μ(G^A,L ^A,π^A) and μ⁡(GB,L∩B,πB)μ(G^B,L ^B,π^B). Remark A.22 (Computation of Gluings). As pointed out for the example in Rmk. A.18, the gluing lemma is not only easy to proof once we have atoms of c-components Lemma A.17 1, but are also easy to compute given atoms: For each (non-trivial) structural c-component cG^c, by hypothesis either c≤AG^c ^A or c≤BG^c ^B, compute atoms in C= innerc∖LcC=N_ inner^c L^c in this respective graph as described in Rmk. A.18. Once we have all atoms (from all c-components), use Lemma A.14 2, μ⁡(G,L,π)μ(G,L,π) as a product over atoms. The product is essentially a Markov factorization of the joint distribution. The revealing corollary (Cor. 3.9) follows immediately: Corollary A.23 (Revealing). Given a structural kernel μ⁡(G,L)μ(G,L), a structural c-component Gc≤G^c≤ G and L′⊂LcL ⊂ L^c, then there is a regular functional computing μ⁡(G,(L∖Lc)∪L′)μ(G,(L L^c)∪ L ) from μ⁡(G,L)μ(G,L) and μ⁡(Gc,L′)μ(G^c,L ). proof Use Lemma A.17 2 to extract all (other) c-components of (G,L)(G,L), then glue (by repeated application of Lemma A.21) (G,(L∖Lc)∪L′)(G,(L L^c)∪ L ) from these c-components and (Gc,L′)(G^c,L ). The question as to whether such a revealing operation can be computed more generally is quite subtle and discussed in A.5. A.4 Algorithmic Construction We give a "simplest viable search" algorithm (Algo. SVSearch) that constructs, by a regular computation, a structural kernel of interest by gluing elements from a set of known structural kernels. We do not know if this algorithm is complete (cf. also A.5). But first decomposing knowledge, then gluing (only growing the result) makes the approach rather simple, while remaining complete in the IID case. This is a "search" in the sense that other than for the local operations of the last sub-section, we have to search a knowledge-set K for pieces to glue. See also §F.4. Algorithm SVSearch simplest_viable_search Input: A structural model (G,L)(G,L) to construct, a set structured kernels =μ⁡(k,Lk,πk)K=\μ(G^k,L^k,π^k)\, Input: optionally (G′,L′)≤(G,L)(G ,L )≤(G,L) and a regular functional F′F computing μ⁡(G′,L′,π′)μ(G ,L ,π ) Input: from finitely many elements of K. Output: A (possibly empty) set of regular functionals F computing μ⁡(G,L,π)μ(G,L,π) from finitely Output: many elements of K. R:=∅R:= . for k∈k , (G∪,L∪)(G^∪,L^∪) with ( ( matching(G∪,L∪)_(G^∪,L^∪)(k,(G,L)(G,L)) and gluable(G∪,L∪)_(G^∪,L^∪)(k, (′,L′)(G ,L )) ) ) do R:=R:=R ∪ SVS( (G,L)(G,L), K, (G∪,L∪)(G^∪,L^∪), F∪[F′,k]F^∪[F ,k] ). ⊳ Use Lemma 3.8. end for if (G′,L′)(G ,L )==(G,L)(G,L) then return F′\F \. else return R. end if Remark: G∪G^∪ with G,Gk≤G∪G,G^k≤ G^∪ is used to describe where and how these graphs overlap. If no (′,L′),F′(G ,L ),F are given, then gluable is true and F∪[F,k]:=Id⁡[k]F^∪[F,k]:= [k]. The "opposite" of searching for glued structures, namely an algorithmic decomposition into c-subgraphs, is in many ways much simpler (it is "local" in the sense of not having to search over a set K of potential building-blocks). It is described in Algo. CGDecomp and returns the set of smaller sub-graphs plus regular functionals computing their structured kernels. Algorithm CGDecomp csubgraph_decomposition Input: A structured kernel μ=μ⁡(G,L,π)μ=μ(G,L,π). Output: A set of pairs of a structural model (′,L′)(G ,L ) and a regular functional F′F computing μ⁡(′,L′,π′)=F′[μ]μ(G ,L ,π )=F [μ], such that the projection to the first component is a bijection to the set of c-subgraphs of (G,L)(G,L). [label=0)] 1. C-Components: Construct structural c-components (as equivalence-classes of ≈L _L). For each structural c-component (c,Lc)(G^c,L^c), denote by FcF^c the regular functional from Lemma A.17 2 computing μ⁡(c,Lc,πc)=Fc[μ]μ(G^c,L^c,π^c)=F^c[μ] and perform the steps below. 2. Latent-Sets: Enumerate L′⊊ innerL _ inner with Lc⊂L′L^c⊂ L . For each L′L , denote by FmF^m the (regular) marginalization of L′∖LcL L^c computing μ⁡(c,L′,πc)=Fm[μ⁡(c,Lc,πc)]=Fm∘Fc[μ]μ(G^c,L ,π^c)=F^m[μ(G^c,L^c,π^c)]=F^m F^c[μ], and perform the steps below. 3. Simplify: Construct the maximal B for L′L as described in the proof of Lemma A.12. Denote the B-simplification of (c,L′)(G^c,L ) by (′,L′∖B)(G ,L B), denote by FsF^s the regular functional (actually Fs=IdF^s= ) from Lemma A.12, computing μ⁡(′,L′∖B,π′)=Fs[μ⁡(c,L′,πc)]=Fs∘Fm∘Fc[μ]μ(G ,L B,π )=F^s[μ(G^c,L ,π^c)]=F^s F^m F^c[μ]. 4. Finalize or Iterate: If (′,L′∖B)(G ,L B) is c-connected: Yield ((′,L′∖B),Fs∘Fm∘Fc)((G ,L B),F^s F^m F^c). Else: Repeat from 1). Remark: Yield adds the given expression to the output-set. For an example, see Fig. 3 (ignoring the last gluing operation, which is not part of decomposition). Importantly, also the trivial, L′=LcL =L^c, simplification is in the output, so for example the first "large" c-component in Fig. 3 (which cannot be glued from its simplifications!) is in the output. A.5 Revealing Operations The gluing operation (Lemma A.21) was formulated enforcing the same missingness: nodes are latent in one subgraph, if and only if they are latent in the other one. To account for different missingness from different sub-graphs, we formulated a revealing operation (Cor. A.23) – this was however a simple combinations of other graphical operations. Intuitively it seems plausible that a more elementary (and more general) revealing operation, c-subgraphs (instead of c-components) or by combining gluing and revealing might be possible; a careful analysis of the situation suggests however that this might be an illusion (at least if one first decomposes into c-connected pieces). The difficulties are quite subtle, so we illustrate them on a simple example. Example A.24 (C-Subgraph Revealing). Consider we are given information about structured kernels with the following structural graphs (mechanisms in square brackets are hidden), where we want to "reveal" some of the hidden structure on the left graph from one on the right: (i)XXYYZZ[L1][L_1][L2][L_2](i)ZZL2L_2(i′)[L1][L_1]YYZZL2L_2(i)XXYYL1L_1[L2][L_2] we want to compute one of the following: (A)XXYYZZ[L1][L_1]L2L_2(B)XXYYZZL1L_1[L2][L_2] First, we simply try to apply the gluing operation (Lemma A.21) with (i) plus one of the other structures. This works trivially ( marginalizing L1L_1 or L2L_2 in any of (i,i′, i) to restore the same missingness hypothesis again, but also only reproduces (i) then. We could first simplify (i) by Lemma A.12: We can drop Z\Z\ (still enforcing L2L_2 as latent as above) we cannot drop L2,Z\L_2,Z\ (the descendants of L2L_2 contain Y), we can drop Y,Z,L2\Y,Z,L_2\, but then it no longer contains the (left-hand side) structural c-component cG^c of (A) as subgraph (on X,Y,[L1]X,Y,[L_1], with an outer node instead of L2L_2; this is also a hypothesis of gluing), and neither of (i,i′) do either (for (i)′ note that sub-graphs must not contain outer nodes of the containing graph as inner nodes by definition 3.4 1). Neither the simplification nor (i) even contain Z, so the best we could possibly reproduce would be (i) itself. Next we try to apply the gluing operation with (i) or (i)′ and (i), but for (i)′ and (i), we can only glue (i) (we have to marginalize to restore same missingess), and for (i) and (i), we have to marginalize L2L_2 (can at most glue (B)), but neither (i) nor (i) contain the right-hand side structural c-component. It seems on first sight, that one could disintegrate observed nodes other than L1L_1 (similar for (i), (i)′) and directly "imputate" the result in (i), but note that in (i), Z provides information about L2L_2, which restricts (via knowledge of Y) the plausible origin in the (multivariate) L1L_1–L2L_2–argument-plane of Y, thus it restricts L1L_1 – information that cannot possibly be extracted from (i) where nothing is known about where in the argument plane of Y (other than the "level-lines" of the preimage of Y) we are. Another indication of this problem is L1 Z|YL_1 Z|Y, because the path L1→Y↔ZL_1→ Y Z is open; after realizing the kernel by Lemma 2.16 the independence-statement makes sense, the question we are asking is however not really one of dependence vs. independence. Similarly, it seems on first sight plausible to obtain B from (i) and (i) by disintegrating the left-hand side (X and Y), with the idea that using Lemma G.22 the kernel μB _B of (B) can be reconstructed from margZ⁡(μB)=μ′ _Z( _B)=μ and a disintegration, however this disintegration has an argument L1L_1 which for similar reasons as above cannot be reconstructed from (i). The problem that we could not find a more general "revealing" operation does of cause not prove that there is none. However, the situation is clearly much more complicated than it may initially seem. If we inspect the proofs of the other operations, then they are rather simple consequences of the properties of "atoms", and the problem encountered here does not seem to be amenable to this approach directly either, at least not beyond the statement of Cor. A.23. Appendix B Details on Models and Observations This section provides proofs for results (like the existence of model-realizations as "observable worlds") in the main text, plus some simple properties of observable and realized worlds that will be helpful for filling in the formal details when identifying model-properties from observations (§C.4). Remark B.1 (Subspace-Relations). In Def. 2.5 and A.5, we ask for certain subspace-relations between XiX_i for different i∈Ii∈ I. Formally, we should point out that notation 2.4 should include a suitable notion of intersections; for example we could fix a shared topological value space X¯ X and continuous inclusions ji:Xi↪X¯j_i:X_i X, then intersect in X¯ X. In practice it is usually clear what it means that the output of X is in the domain of YxY_x so X⊗YxX Y_x makes sense. B.1 Results in the Main Text First note, that by acyclicity and finite past assumptions, causally ordered index sets are essentially I=ℕI=N (or finite). Remark B.2 (Simplified Form of Index-Sets). By acyclicity (Ass. 2.10), a causal order πI _I exists, thus ⊂ℤI . By finite past (Ass. 2.11), a πI _I-first element exists and actually =ℕI=N or I=1,…,NI=\1,…,N\ (if I is finite) as ordered set (obviously the countable I is has this property as a set anyway). The shallow distribution was defined as follows: plain Lemma 2.13 (Shallow Distribution) Given acyclicity (Ass. 2.10) and finite past (Ass. 2.11), there is a probability measure PθP_θ on ∏i∈IXi _i∈ IX_i, which we will call the shallow distribution, parametrized by θ=ℱ interveneθ=F_ intervene, such that the marginalization to any finite I′⊂I ⊂ I satisfies for all measurable Bi∈ℬ⁡(Xi)B_i (X_i) the following characterizing property: Pθ((∏i′∈I′Bi′)×(∏i∈I∖I′Xi))=[⊗i′∈ (I′)πIfJ⁡(i′)]((∏i′∈I′Bi′)×(∏i∈ (I′)∖IXi)) .P_θ ( ( _i ∈ I B_i )× ( _i∈ I I X_i ) )= [ _I_i ∈ _I(I )f_J(i ) ] ( ( _i ∈ I B_i )× ( _i∈ _I(I ) IX_i ) ) . proof We use the notation from Rmk. B.2. First, we construct PθnP_θ^n for n∈ℕ=In =I inductively over n as Pθn:=⊗i=1nfJ⁡(i)P_θ^n:= _i=1^nf_J(i) (cf. §G.4 for technical details on extending domains from parents to ancestors). This is well-defined for any finite n and satisfies the claimed equation by construction. It remains to show, that this is actually well-defined and characterizing, there is a unique probability-measure with this property. This follows from the definition of products of σ-algebras, for a standard-result see [22, Prop. 2.2 (p. 25)]. Let B∈ℬ⁡(∏iXi)B ( _iX_i) be arbitrary. By definition of the product sigma-algebra, a basis of measurable sets in ℬ⁡(∏iXi)B( _iX_i) is given by "rectangles" (∏i∈IBBi)×(∏i∈I∖IBXi)( _i∈ I_BB_i)×( _i∈ I I_BX_i) with a finite IB⊂I_B⊂ I and Bi∈ℬ⁡(Xi)B_i (X_i). It is enough to fix PθP_θ on a basis, which the property does. We start with the existence of realizations by random elements. The non-trivial result at the core of the argument is provided by [22, Lemma 2.22 (p. 34)], a standard result in probability-theory. definition Definition 2.14 (Observable World) An observable world realizing the model ℳM is a family of random variables Vi:Ω→Xii∈I\ V_i: → X_i\_i∈ I, together with measurable maps fi: (i)×[0,1]→Xif_i:X_ _I(i)×[0,1]→ X_i and jointly independent random variables ηi∼U⁡([0,1]) _i U([0,1]) called noises, such that Vi=fi(V (i),ηi)andP⁡(Vii∈I)=Pθ . V_i=f_i( V_ _I(i), _i) P(\ V_i\_i∈ I)=P_θ . Mandated by the second equality, we will often simply write Pθ(Vii∈I)P_θ(\ V_i\_i∈ I). plain Lemma 2.16 (Observable World Existence) Given acyclicity (Ass. 2.10) and finite past (Ass. 2.11), an observable world exists, it is unique up to equality in distribution and it is locally Markov, VK| (i) V_i V_K| _I(i) for any K⊂IK⊂ I with K∩ (i)=∅K∩ _I(i)= . proof Existence: We use notation from Rmk. B.2. Even though I may be infinite, by definition of products of sigma-algebras, only finitely many factors of a measurable set in ∏i∈IXi _i∈ IX_i are not the full space, thus in order to proof P⁡(Vii∈I)=PθP(\ V_i\_i∈ I)=P_θ it suffices to proof this statement for all finite subsets I′⊂I ⊂ I (see [22, Prop. 2.2 (p. 25)]). By finite past (Ass. 2.11) we can replace I′I by the (also finite) I′:= (I′)I := _I(I ), so ′= (I′)=1,…,m⊂ℕ=I = _I(I )=\1,…,m\ =I. Our construction will be inductive over n∈In∈ I, show: ∀n∈I∀ n∈ I ∃V1,…,Vn∃ V_1,…, V_n together with fi,ηif_i, _i such that (i) ∀i≤n:Vi=fi(V (i),ηi)∀ i≤ n: V_i=f_i( V_ _I(i), _i) and (i) P⁡(V1,…,Vn)=Pθ(V1,…,Vn)P( V_1,…, V_n)=P_θ( V_1,…, V_n). Inductive start (n=0n=0): There is nothing to show; the case n=0n=0 suffices as inductive start, see use of the inductive hypothesis in the inductive step. Inductive step (n↦n+1≥1n n+1≥ 1): We are given V1,…,Vn V_1,…, V_n with the required properties by inductive hypothesis. Since πI _I is a causal order, (n+1)⊂1,…,n _I(n+1)⊂\1,…,n\. If n+1=1n+1=1, then (n+1)=∅ _I(n+1)= and even though the inductive hypothesis – in this case constructed as the inductive start – is trivial and does not give us any random variables the construction below works. By [22, Lemma 2.22 (p. 34)], there is a measurable map fn+1:X (n+1)×[0,1]→Xn+1f_n+1:X_ _I(n+1)×[0,1]→ X_n+1 and a random variable ηn+1∼ ([0,1]) _n+1 ([0,1]) with ηn+1 (η1,…,ηn) _n+1 ( _1,…, _n), such that ∀ +1∈X (n+1)∀ _n+1∈ X_ _I(n+1) we have fn+1( +1,ηn+1)∼fJ⁡(n+1)( +1)f_n+1( _n+1, _n+1) f_J(n+1)( _n+1); we write fJ⁡(i)f_J(i) to unambiguously distinguish the mechanism at i (Def. 2.6) from the mapping fif_i constructed in this proof. Define Vn+1:=fn+1(V (n+1),ηn+1) V_n+1:=f_n+1( V_ _I(n+1), _n+1). Property (i) is thus immediately satisfied. For (i), by definition of PθP_θ, Pθ(V1,…,Vn+1)=Pθ(V1,…,Vn)⊗(fJ⁡(n+1)) (n+1)P_θ( V_1,…, V_n+1)=P_θ( V_1,…, V_n) (f_J(n+1))_ _I(n+1). Since fn+1( +1,ηn+1)∼fJ⁡(n+1)( +1)f_n+1( _n+1, _n+1) f_J(n+1)( _n+1) (see above) P⁡(Vn+1|V (n+1)= +1)=fJ⁡(n+1)( +1)P( V_n+1| V_ _I(n+1)= _n+1)=f_J(n+1)( _n+1). Thus for all measurable B=B≤n×Bn+1B=B_≤ n× B_n+1 P⁡(V1,…,Vn+1∈B) P( V_1,…, V_n+1∈ B) =∫P⁡(dvn+1|V (n+1)= +1)P(dv1,…,dvn)1B(v1,…,vn,vn+1)v→ = P(dv_n+1| V_ _I(n+1)= _n+1)P(dv_1,…,dv_n)1_B(v_1,…,v_n,v_n+1)d v =∫fJ⁡(n+1)( +1,dvn+1)Pθ(dv1,…,dvn)1B(v1,…,vn,vn+1)v→ = f_J(n+1)( _n+1,dv_n+1)P_θ(dv_1,…,dv_n)1_B(v_1,…,v_n,v_n+1)d v =[Pθ(V1,…,Vn)⊗(fJ⁡(n+1)) (n+1)](B) . = [P_θ( V_1,…, V_n) (f_J(n+1))_ _I(n+1) ](B) . Using that products form a basis of the σ-algebra ℬ⁡(X1×…×Xn+1)B(X_1×…× X_n+1), this shows claim (i). Note that the model (Def. 2.6) by definition ensures X (j)⊂X _ _J(j)⊂ X_ _J and XJ⊂XjX_J⊂ X_j, so fn+1f_n+1 is defined on all values the parents might take and produces values in Xn+1X_n+1. Properties: Uniqueness up to equality in distribution is clear by definition (as any observed world has distribution PθP_θ). For the local Markov property, first note that by construction Vn+1:=fn+1(V (n+1)CLOSE, V_n+1:=f_n+1( V_ _I(n+1), OPENηn+1) _n+1), so it inductively follows immediately that Vn+1:Ω→Xn+1 V_n+1: → X_n+1 factors through η (n+1) _ _I(n+1), is a measurable mapping gn+1:[0,1]| (n+1)|→Xn+1g_n+1:[0,1]^| _I(n+1)|→ X_n+1 with Vn+1(ω) V_n+1(ω) =gn+1(η (n+1))=g_n+1( _ _I(n+1)). Also by Vn+1:=fn+1(V (n+1)CLOSE V_n+1:=f_n+1( V_ _I(n+1), OPENηn+1) _n+1) we have P⁡(Vn+1| (n+1)= +1)=fn+1( +1,ηn+1)P( V_n+1| _I(n+1)= _n+1)=f_n+1( _n+1, _n+1). So for K⊂IK⊂ I with K∩ (n+1)=∅K∩ _I(n+1)= , P⁡(Vn+1,VK| (n+1)= +1)=P⁡(fn+1( +1,ηn+1),gK(η (K)))P( V_n+1, V_K| _I(n+1)= _n+1)=P (f_n+1( _n+1, _n+1),g_K( _ _I(K)) ). Using ηn+1 (η1,…,ηn) _n+1 ( _1,…, _n) (see above) repeatedly ηn+1 η (K) _n+1 _ _I(K); by K∩ (n+1)=∅K∩ _I(n+1)= we know n+1∉ (K)n+1∉ _I(K). Measurable transformations of independent variables are independent variables, thus the claim follows. B.2 Properties of Probabilistic Worlds The following property is essentially a direct consequence of definitions, but it will be helpful for making the connection to identifiability in §C.4 clearer. A similar result connecting structured queries to the realized world will be given in Lemma D.9. Lemma B.3 (Properties). Given an observable world Vii∈I\ V_i\_i∈ I and a model-aligned structural graph G with inner=⊂IN_ inner=N⊂ I and ℰ=(p,c)∈× inner|p∈ (c)E=$p,c) ×N_ inner|p∈ _I(c)\. [label=()] 1. Single-Level: If ()⊂ _I(N) and ∀n∈:μn=fJ⁡(n)∀ n :μ^n=f_J(n), then ∀L⊂∀ L : Pθ(V∖L) = μ(G,L) .P_θ( V_N L) = μ(G,L) . 2. Multi-Level: Given N0⊂N_0 with (∖N0)⊂ _I(N N_0) and ∀n∈∖N0:μn=fJ⁡(n)∀ n N_0:μ^n=f_J(n) and ∀n∈N0∀ n∈ N_0: μn=δ⁡[xn′]μ^n=δ[x _n], then there is a structural graph G′G with ⊂′N , ∀n∈∖ (N0):∀ n _I(N_0): (μ′)n=μn(μ )^n=μ^n, ∀n∈∖ (N0):∀ n _I(N_0): ′(n)= (n) (n)= _I(n), such that Pθ(V∖L|VN0=x′) = μ(G′,L′)[x′] ,P_θ( V_N L| V_N_0=x ) = μ(G ,L )[x ] , where μ⁡(G′,L′)[x′]μ(G ,L )[x ] is a functional of x′x through alignment to μn=δ⁡[xn′]μ^n=δ[x _n] at n∈N0n∈ N_0. Further, given Y⊂Y with Y∩ (N0)=∅Y∩ _I(N_0)= and such that it satisfies the graphical property (a "latent path" here means a directed path with all nodes except for its last one in L′L , γ∩L′=γ∖bγ∩ L =γ \b$: (i) Given a latent path γ in G′G starting at a∈Y∖N0′a∈ Y N_0 to a node b∉L′b∉ L , then b∈Y∖L′b∈ Y L . Then G′G can be chosen to also satisfy the graphical property (i’) with a∈N0a∈ N_0 allowed: (i’) Given a latent path γ in G′G starting at a∈Ya∈ Y to a node b∉L′b∉ L , then b∈Y∖L′b∈ Y L . Remark: This graphical property will make more sense after reading the proof of Thm. 1 in §C.4. Essentially (i) appears (also named (i)) in step 1 of that proof, for step 3 (the multi-level case) it has to be modified to (i’), the statement above essentially says this can always be achieved by proper choice of G′G . proof Using the notation I=ℕI=N from Rmk. B.2, let m:=max⁡()m:= (N). By definition Pθ(V1,…,Vm)=⊗n=1mfm .P_θ( V_1,…, V_m)= _n=1^mf_m . Part 1: In the single-level case, by ()⊂ _I(N) we can define B=1,…,m∖B=\1,…,m\ with B= (B)B= _G(B) and G is a simplification (Lemma A.12) of a similar (using the same edges as the I-graph, aligned to fif_i) G+G^+ with +=1,…,mN^+=\1,…,m\. By Lemma A.12 also μ⁡(G,L)=μ⁡(G+,L∪B)μ(G,L)=μ(G^+,L∪ B). By definition, μ(G+,L∪B)=margL∪B(⊗n=1mfm)=margL∪B(Pθ(V1,…,Vm))=Pθ(V∖L) .μ(G^+,L∪ B)= _L∪ B( _n=1^mf_m)= _L∪ B(P_θ( V_1,…, V_m))=P_θ( V_N L) . Part (b): For the multi-level case, first define A:= (N0)A:= _I(N_0). By exploiting sparsity (Lemma G.27) we can nodes (replace π) such that A=1,…,m0A=\1,…,m_0\. The conditional distribution Pθ(V1,…,m∖A|VA=a)P_θ( V_\1,…,m\ A| V_A=a) has a regular version (by Ass. 2.1, [22, Thm. 5.3 (p. 84)]) and satisfies Pθ(VN0)⊗Pθ(V1,…,m∖N0|VN0=x′)=Pθ(V1,…,Vm)P_θ( V_N_0) P_θ( V_\1,…,m\ N_0| V_N_0=x )=P_θ( V_1,…, V_m); so by uniqueness of disintegrations (Lemma G.22), this is the same as disintegrating Pθ(V1,…,m)=⊗n=1mfnP_θ( V_\1,…,m\)= _n=1^mf_n (Def. 2.13) by N0N_0 as: Pθ(V1,…,m∖N0|VN0=x′) = 0(⊗n=1mfn)x′ .P_θ( V_\1,…,m\ N_0| V_N_0=x ) = _N_0( _n=1^mf_n)_x . This disintegration can be computed (by uniqueness, Lemma G.22) inductively over a1<a2<…∈N0a_1<a_2<…∈ N_0 moving a1a_1 (then a2a_2 etc.) to the first (then second etc.) position in ⊗n=1mfn _n=1^mf_n by anti-causal disintegrations (Def. G.16); this replaces terms to the left of a1a_1 producing ⊗n=1mfn = (fa1∘[⊗n<a1fn])⊗(⊗n<a1fn|fa1)⊗… . _n=1^mf_n = (f_a_1 [ _n<a_1f_n]) ( _n<a_1f_n|f_a_1) … . Before we continue with a2a_2, we note that this only modifies terms ≤a1≤ a_1, and a1∈N0⊂Aa_1∈ N_0⊂ A, thus a1≤m0a_1≤ m_0. In particular, there are kernels WnW^n (we can move a1a_1 one position at a time to avoid confusion) such that (up to transposition, Def. G.19, denoted "≈" following notation A.7) ⊗n=1mfn ≈ [⊗n∈N0Wn]⊗[⊗n∈A∖N0Wn]⊗[⊗n>m0fn] . _n=1^mf_n ≈ [ _n∈ N_0W^n] [ _n∈ A N_0W^n] [ _n>m_0f_n] . The kernels WnW^n may have additional arguments among their ancestors (Lemma G.27 4); call the new parent-sets ′(n) (n). The disintegration of N0N_0 is now (by uniqueness) a simple disintegration from the left: disintN0(⊗n=1mfn) = [⊗n∈A∖N0Wn]⊗[⊗n>m0fn] . _N_0( _n=1^mf_n) = [ _n∈ A N_0W^n] [ _n>m_0f_n] . We can already define a graph G~ G on ~=~ inner=1,…,m N= N_ inner=\1,…,m\, with edges prescribed by _I (for n>m0n>m_0) and ′ (for n≤m0n≤ m_0), and aligned to WnW^n for n≤m0n≤ m_0, to δ⁡[xn′]δ[x _n] for n∈N0n∈ N_0 and to fnf_n for n∈n>m0=∖An∈\n>m_0\=N A. Using δ[X=x0]⊗μx=μx=x0δ[X=x_0] _x= _x=x_0 we readily get the first part of our result (using again B=1,…,m∖(∪A)B=\1,…,m\ (N∪ A), as in the proof of part (a); note that (∪A)⊂(∪A) _I(N∪ A)⊂(N∪ A) by closedness under parents, cf. hypothesis): μ(G~,L∪B)n0=x′=Pθ(V1,…,m∖N0|VN0=x′) .μ( G,L∪ B)_n_0=x =P_θ( V_\1,…,m\ N_0| V_N_0=x ) . However, we cannot yet satisfy the graphical property: the nodes in N0N_0 now seem to confound nodes in ∖AN A with nodes in A. (In the proof of Thm. 1, we need a sub-graph of the I-graph that is a structural c-component, if N0N_0 [which will be pinned latents] confounds Y [nodes of the subgraph] with other ancestors, then those ancestors are also in the c-component. The graphical property is a formal means of ensuring that ancestors are "disconnected" from the c-component Y after the shuffling procedure above.) The solution is surprisingly simple. In this case we actually do know something about the internal structure of the confounder. For singular (taking only a single value) kernels there is no actual information in the value taken so they right-distribute: ⟦Zx⊗Yz,x⟧[x]∘δ[X=x0] Z_x Y_z,x _[x] δ[X=x_0] = Zx=x0⊗Yz,x=x0 = Z_x=x_0 Y_z,x=x_0 (∗*) = (Zx∘δ[X=x0])⊗(Yz,[x]∘δ[X=x0])z = (Z_x δ[X=x_0]) (Y_z,[x] δ[X=x_0])_z , while generally, copying a node in a structural graph (adding a node with the same alignment and distributing children across both copies) changes the represented structured kernel, we can copy nodes aligned to singular kernels. In particular, we can modify G~ G into G′G with additional nodes copyN_ copy (added to L) to ensure the graphical criterion as follows: Define copy := (Y)∩N0 ,N_ copy := _I(Y)∩ N_0 , and N0′:=N0⊔ copyN_0 :=N_0 _ copy as the set adding a disjoint copy (we will write a∈N0a∈ N_0 and a′∈ copya _ copy if the distinction is relevant). Define G′G on nodes ′= inner′=⊔ copyN =N_ inner =N _ copy and L′=L⊔ copyL =L _ copy, with edges as in G~ G except for edges out of N0N_0: Here, keep edges out of (Y)∩N0 _I(Y)∩ N_0 into Y and out of N0∖ (Y)N_0 _I(Y) as is, edges out of p∈ (Y)∩N0p∈ _I(Y)∩ N_0 to w∈∖Yw Y are removed and instead added as p′→wp → w, from copyN_ copy. By equation (∗)(*), we still have μ(G′,L′∪B)n0=x′=Pθ(V1,…,m∖N0|VN0=x′) .μ(G ,L ∪ B)_n_0=x =P_θ( V_\1,…,m\ N_0| V_N_0=x ) . Given Y⊂Y with Y∩ (N0)=∅Y∩ _I(N_0)= and such that (i) given a latent path γ in G′G starting at a∈Y∖N0′a∈ Y N_0 to a node b∉L′b∉ L , then b∈Y∖L′b∈ Y L . Let γ be an arbitrary latent path in G′G starting at a∈Ya∈ Y to a node b∉L′b∉ L . If a∉N0⊔ copya∉ N_0 _ copy, then by (i), b∈Y∖L′b∈ Y L . If a∈(N0⊔ copy)∩Y=N0a∈(N_0 _ copy)∩ Y=N_0, then the child c of a which is the first node on γ after a is, by construction of G′G , in Y (edges out of N0N_0 into Y were kept in the construction of G′G , while edges out of N0N_0 into ∖YN Y were moved to copyN_ copy). If c is still in N0N_0, repeat the last argument, otherwise apply (i) to the restriction of γ starting at c, to obtain b∈Y∖L′b∈ Y L . Appendix C Details on the Extraction of Structured Kernels This section provides the technical details about the connection from observations to the identification of model properties. Before proving statements from the main text, we give definitions for the multi-level case in parallel to the main text. The reader not (currently) interested in the multi-level case may skip the first subsection, and only needs to know that in the single-level case "=G obs=G0G=G obs=G^0", inner= properN_ inner=N_ proper, outer= externN_ outer=N_ extern (while pinned=∅N_ pinned= ). The phrasing "μ is δi _i-identifiable" is equivalent to "μ is identifiable" in the single-level case. C.1 Multi-Level Modifications We need some smaller modifications to families of embeddings and to data-sets -definitions for the multi-level case. Families of Embeddings: We formulate the machinery of embeddings on local graphs, which are closely related to structural graphs (and structured kernels); in the single-level case, pinned=∅N_ pinned= and G, G0G^0, G obsG obs (see below) agree. Starting from a slightly different kind of object allows to keep both aspects of the formalism reasonably simple. Definition C.1 (Local Graph). A local graph G is a finite set of nodes which is a disjoint union of proper, pinned (new in multi-level) and external nodes = proper ∪˙ pinned ∪˙ externN = _ proper ∪ _ pinned ∪ _ extern together with a parent structure ⊂× properP ×N_ proper, a set of directed edges (which we will call proper edges) ending at a node in properN_ proper. Alignment: There is an underlying structural graph G0()G^0(G), with 0:=N^0:=N, inner0:= properN_ inner^0:=N_ proper and ℰ0:=E^0:=P. A model-aligned local graph is a local graph together with indices ix∈Ii^x∈ I for x∈ pinnedx _ pinned (new in multi-level) and kernels μnμ^n plus argument-assignments ^n for n∈ propern _ proper such that G0G^0 is model-aligned. The embeddings from Def. 4.1 are slightly modified, the only difference is that model-alignment now includes additionally the statement "∀x∈ pinned∀ x _ pinned: ix=ψ⁡(n)i^x=ψ(n)." Definition C.2 (Local Graph Embedding). An embedding of a model aligned local graph G, denoted by ψ:↪Iψ:G I, is an injective mapping ψ:↪Iψ:N I such that the following conditions are satisfied (see Fig. 4): [label=()] 1. Proper-Node I-Parents are Included: If n∈ propern _ proper, then ∀i′∈ (ψ⁡(n))∀ i ∈ _I(ψ(n)), ∃n′∈∃ n such that i′=ψ⁡(n′)i =ψ(n ). 2. Proper Edges equal I-Graph Edges: For n′∈ propern _ proper, there is a proper edge n→n′n→ n in G if and only if ψ⁡(n)∈ (ψ⁡(n′))ψ(n)∈ _I(ψ(n )). 3. Model Alignment: ∀n∈ proper∀ n _ proper: μn=fJ⁡(ψ⁡(n))μ^n=f_J(ψ(n)) is a model mechanism (Def. 2.6) and new in multi-level: ∀x∈ pinned∀ x _ pinned: ix=ψ⁡(n)i^x=ψ(n). Families of embeddings can be defined as before (Def. 4.2), only the notation for inner↦ properN_ inner _ proper changes: Definition C.3 (Families of Embeddings). A family of local graph embeddings (J0,ψjj∈J0,,H,y0)(J_0,\ _j\_j∈ J_0,G,H,y_0) is a collection of local graph embeddings ψj:↪I _j:G I of a single model-aligned local graph G. Further there is a fixed element y0∈y_0 , which will be called the anchor(-node), a symmetry H⊂GH⊂ G and a range of applicability J0⊂IJ_0⊂ I. For n∈n denote: ψ∗(n):J0→I, j↦ψj(n) . _*(n):J_0→ I, j _j(n) . Finally, we will require the following conditions to be satisfied: [label=()] 1. Trivial on Anchor: ∀j∈J0∀ j∈ J_0: ψj(y0)=j _j(y_0)=j. 2. Rigidity: ∀n∈∀ n : ψ∗(n) _*(n) is H-equivariant. 3. Freeness: ∀n∈ proper∀ n _ proper (multi-level: inner nodes become proper nodes): ψ∗(n) _*(n) is injective. While our models allow only a fixed and finite number of parents in the I-graph, nodes can have an arbitrary number of children. This allows for example for multi-level statics. Definition C.4 (Singular Nodes). This has no analogue in the single-level case: Given a family ψjj∈J0\ _j\_j∈ J_0 of local graph embeddings, we call a node x∈x singular embedded at i∈Ii∈ I if ∀j∈J0∀ j∈ J_0: ψj(x)=i _j(x)=i. Remark: For |J0|>1|J_0|>1, proper nodes cannot be singular by freeness. For pinned alignment Def. 4.13, all pinned nodes pinnedN_ pinned must be singular embedded. Conversely singular embedded nodes x∈ externx _ extern can always be pinned at ix=ψj(x)i^x= _j(x), is a valid family of embeddings with x∈ pinnedx _ pinned (this can render the family backdoor-free, cf. 4.10) For attached structure (decorated graphs) not much new happens for the multi-level case. Some general remarks on attached structure can be found in the next subsection. Some smaller remarks are in place however and the definition of ancestral structure should use the nomenclature of external and proper (rather than inner and outer) nodes: Definition C.5 (Ancestral Structure). Given a local graph G, an ancestral structure A on G is a set of directed edges ↝ (ancestral edges) each starting at a proper node and ending at an external node, such that G with proper and ancestral edges is acyclic. Remark C.6 (Validity of Ancestral Sets in the Multi-Level Case). In the multi-level case, a valid ancestral structure remains essentially unchanged (compared to the single-level case), so it tracks only ancestral paths to externN_ extern, not to nodes pinnedN_ pinned. This will suffice for asymptotic results as there are only finitely many j∈J0j∈ J_0 with ancestral paths to pinnedN_ pinned (see proof of Thm. 1, step 3, choice of J0 invalidJ_0 invalid). Finally, also in backdoor-freeness (Def. 4.10), replace outer↦ pinnedN_ outer _ pinned. This means pinned∩L≠∅N_ pinned∩ L≠ is explicitly allowed, and is indeed typically (excluding trivial cases) the case. Definition C.7 (Backdoor-Freeness). Given a decorated family of embeddings (ψjj∈J0CLOSE(\ _j\_j∈ J_0, L, OPEN)A), we call an ancestral edge l↝x∈l x a backdoor if it starts at l∈Ll∈ L. We call the family backdoor-free, if L∩ extern=∅L _ extern= (multi-level: outerN_ outer is replaced by externN_ extern) and there are no backdoors. Data-Sets: We slightly extend the definition of data-sets (Def. 4.12), note that a data-set in the sense of the main text is a dataset with X′=∅ X = . Similar to backdoor-freeness, "pinned nodes" (here X′ X ) are typically not observed; they are however j-independent. Definition C.8 (Data-Set). A data-set is a tuple =((XjCLOSECLOSED=(( X_j, OPENYj)j∈J0 Y_j)_j∈ J_0, OPENX′) X ), where Xj X_j, Yj Y_j and X′ X (multi-level: the inclusion of a X′ X is new) are tuples of Vi V_i, exist iX,j(1),…,iX,j(n)∈Ii_X,j^(1),…,i_X,j^(n)∈ I, such that Xj=(ViX,j(1),…,ViX,j(n)) X_j=( V_i_X,j^(1),…, V_i_X,j^(n)) and analogously for Yj Y_j and X′ X . We call the data-set valid if X and Y are observed that is J0 obs(N) := ∈J0 | ∀m:iX,j(m)∈V(N),∀m′:iY,j(m′)∈V(N) J_0 obs(N) := \ j∈ J_0 | ∀ m:i_X,j^(m)∈ V(N),∀ m :i_Y,j^(m )∈ V(N) \ satisfies |J0 obs(N)|→∞|J_0 obs(N)|→∞ as N→∞N→∞ and is non-degenerate j≠j′⇒∀m:iY,j(m)≠iY,j′(m)j≠ j ∀ m:i_Y,j^(m)≠ i_Y,j ^(m) (this last condition is imposed only on Y, not on X; see Rmk. A.6). As before decorated families of embeddings induce data-sets (note that even though X′ X is not observed, its j-independent existence is an important condition). Example C.9 (Data-Set of Decorated Families). Given a decorated family of embeddings (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A) with L∩ extern=∅L _ extern= , there is a valid data-set ((Xj,Yj)j∈J0CLOSE(( X_j, Y_j)_j∈ J_0 , OPENX′) X ) (Def. 4.12) defined for j∈J0j∈ J_0 as the tuples Xj X_j = (Vψj(n))n∈ extern∪( pinned∖L) (multi-level: add obs. pinned n) = ( V_ _j(n))_n _ extern∪(N_ pinned L) (multi-level: add obs.\ pinned $n$) Yj Y_j = (Vψj(n))n∈ proper∖L = ( V_ _j(n))_n _ proper L X′ X = (Vin)n∈ pinned∩L (multi-level only) . = ( V_i^n)_n _ pinned∩ L (multi-level only) . Identifiability: Direct identifiability (Def. 4.14) must respect the shared across j value of X′ X : Definition C.10 (Direct Identifiability). Given a valid data-set (Def. 4.12) =((Xj,Yj)j∈J0CLOSE,D=(( X_j, Y_j)_j∈ J_0, OPENX′) X ) and kernels Xjj∈J0\X_j\_j∈ J_0, Yx[x′]Y_x[x ] (a functional of x′x ), such that for each j individually the shallow distribution PθP_θ (Def. 2.14) satisfies, ∀j∈J0: θ(Xj,Yj|X′=x′)=Xj⊗Yx[x′] (multi-level: x′ is new) ,∀ j∈ J_0: P_θ( X_j, Y_j| X =x )=X_j Y_x[x ] (multi-level: $x $ is new) , then we call Yx[x′]Y_x[x ] directly identifiable. Formally we also consider the known a priori ℱ interveneF_ intervene (Def. 2.6) and the immediately observed Vi V_i for i∈i∈ V (thus also δ[Vi=xi′]δ[ V_i=x _i]; multi-level only) directly identifiable. Remark: We usually will not know or observe the value x′x , we only know that x′x does not depend on j. For the multi-level case, we use the following notation. Notation C.11 (Pinned Kernels). Tracking in computations which arguments of a structured kernel correspond to pinned nodes would be arduous. Instead we adjoin generic kernels ℱ I=δii∈IF_ I=\ _i\_i∈ I with zero parents (similar to how, for example, polynomial rings adjoin a generic variable to an algebraic structure) and consider ℱ=ℱ model∪ℱ intervene∪ℱ IF=F_ model _ intervene _ I-structured kernels. We will then call an expression μ⁡[δi1,…,δim]μ[ _i^1,…, _i^m], δi _i-identifiable, if there is an identifiable kernel ν such that (writing δ⁡[Vi]δ[ V_i] for the random measure singular at the value taken by Vi V_i): μ[δ[Vi1],…,δ[Vim]]= a.s.ν (almostsurelyw.r.t.Pθ).μ[δ[ V_i^1],…,δ[ V_i^m]] a.\,s.=ν (almostsurelyw.\,r.\,t.\ P_θ^D). The intuitive idea is that, in each realization, hidden contexts (like slope in example 4.18) do take a value C=cC=c. We do not know this value, but given observations of many children we gain partial information: Yc=⁡(c2,1)Y_c=N(c^2,1) (in SCM notation this could be Y:=C2+ηY:=C^2+ _Y), from many observations of Y we gain information about c2c^2. So from observations, we know Pθ(C∈±E⁡[Y])=1P_θ^D(C∈\± E[Y]\)=1. Indeed, we gain precisely the information about C needed to compute Y; we can learn YcY_c, up to sign of c, as Y′=⁡(c2,1)Y =N(c^2,1). Taken together: Pθ(YC=Y′)=1P_θ^D(Y_C=Y )=1, where the capital index YCY_C means we plug in the random-variable C (or equivalently: compose YcY_c to YC=Yc∘δ⁡[C]Y_C=Y_c δ[C]). We have to associate structural graphs to local graphs (the embedded objects) for identification statements. Definition C.12 (Associated Structural Graphs). We are given a model-aligned local graph (,L)(G,L), with L∩ extern=∅L _ extern= . Def. C.1 introduced its underlying structural model (G0,L)(G^0,L) with 0:=N^0:=N, inner0:= properN_ inner^0:=N_ proper and ℰ0:=E^0:=P. We additionally define G obs()G obs(G) with nodes obs=N obs=N split as inner obs:= proper∪( pinned∩L) , outer obs:= extern∪( pinned∖L)N_ inner obs:=N_ proper∪(N_ pinned∩ L) , _ outer obs:=N_ extern∪(N_ pinned L) and the edges ℰ obs=E obs=P. Then G obsG obs is model-aligned with μnμ^n for n∈ propern _ proper and μx:=δix∈ℱ Iμ^x:= _i^x _ I for x∈ pinned∩Lx _ pinned∩ L. Thm. 1 will be formulated as δi _i-identifying (notation C.11) μ⁡(G obs,L)μ(G obs,L), the formal details are given below at Theorem . Knowledge Sets: We modify Def. 4.17 to include immediately observed variables as (basic) knowledge: Definition C.13 (Extracted Knowledge Set). Let ℬB be the set of all backdoor-free families of embeddings. For (ψ,L,)∈ℬ(ψ,L,A) define a structured model k⁡(ψ,L,):=(G obs,L)k(ψ,L,A):=(G obs,L). By Thm. 1 μ⁡(G obs,L)μ(G obs,L) is δi _i-identifiable (we simply say k is identifiable). For f~∈ℱ intervene f _ intervene, define a structured model k⁡(f~)=(G⁡(f~),L=∅)k( f)=(G( f),L= ), where G⁡(f~)G( f) is a graph with a single inner node y aligned to μy=f~μ^y= f, κ (the number of parents) outer nodes outer=x1,…,xκN_ outer=\x_1,…,x_κ\ and a proper edge from each xk→yx_k→ y. By definition (Def. 2.6 and 4.14), elements of ℱ interveneF_ intervene are considered known, thus k⁡(f~)k( f) is identifiable for all f~∈ℱ intervene f _ intervene. Multi-Level only: Finally, what is immediately observed is known (Def. 4.14), so for i∈i∈ V, define k⁡(i)k(i) as the structured graph with a single inner node aligned to δ⁡[Vi]δ[ V_i]. Define the, thus (element-wise) δi _i-identifiable (δi _i added in multi-level), basic knowledge set as (multi-level: add last union) basic = ⋃(ψ,L,)∈ℬk⁡(ψ,L,) ∪ ⋃f~∈ℱ intervenek⁡(f~) ∪ ⋃i∈k⁡(i) .K_ basic = _(ψ,L,A) k(ψ,L,A) ∪ _ f _ intervenek( f) ∪ _i∈ Vk(i) . C.2 Embedding-Families and Attached Structure We add some remarks and simple properties, beyond what was stated in the main text, that are useful for later reference and for the practical interpretation of these definitions. Remark C.14 (Change of Anchor). Given a family of embeddings ψjj∈J0\ _j\_j∈ J_0 anchored on x∈x and a fixed choice y∈y with ψ∗(y) _*(y) injective ( proper node by freeness 4.2 3). Define J0′:= (ψ∗(y))⊂IJ_0 := ( _*(y))⊂ I, clearly when restricting the target to the image ψ∗(y):J0↠J0′ _*(y):J_0 J_0 is also surjective, thus ψ∗(y) _*(y) is bijective (to its image J0′J_0 ); we will write ψy−1(j′):=(ψ∗(y))−1(j′) _y^-1(j ):=( _*(y))^-1(j ) for its inverse at j′∈J0′j ∈ J_0 . Define a family of local graph embeddings anchored on y indexed by J0′J_0 as ψj′:↪I,n↦ψy−1(j)(n) .ψ _j:N I, n _ _y^-1(j)(n) . Note that given an anchor (here x∈x ), even though freeness 3 is given only at proper nodes, the mapping ψ∗(x) _*(x) at the original anchor is of course also injective (by 1). Remark C.15 (Relevant Latent Subsets). In practice, for the finite sample case, the definition of ψ L_ψ can easily be adapted as follows: A L⊂L is relevant, if there is no L′⊊L L such that (at least) the same number of data-points is available |J0 obs(N)|≤|(J0 obs)′(N)||J_0 obs(N)|≤|(J_0 obs) (N)| (cf. Def. 4.4) and the number |J0 obs(N)||J_0 obs(N)| of available data-points is larger some (possibly ψ-dependent; estimator and precision-target specific) minimum number (if precision-measures are available for a specific estimator, those can of course used directly instead of a minimum number of samples, this may involve expensive evaluations on many potentially relevant branches however). It may however make sense to track not only minimal latent subsets, as trading additional latents for additional samples might be favorable. It will be helpful to have a notion of sub-graphs and sub-families (in loose analogy to Def. 3.4). Definition C.16 (Local Subgraphs). A local subgraph A≤BG^A ^B of a local graph BG^B is a local graph AG^A such that: [label=()] 1. Node Sets: A⊂BN^A ^B, properA⊂ properBN_ proper^A _ proper^B and pinnedA⊂ pinnedBN_ pinned^A _ pinned^B. 2. Edge Sets: B∩(A× properA)P^B∩(N^A×N_ proper^A), are exactly those in GBG^B from nodes in GAG^A to inner nodes of GAG^A. 3. Proper Parents: ∀n∈ properA∀ n _ proper^A: if p∈ (n)p∈ _G^B(n), then p∈Ap ^A. 4. Alignment: For model-aligned graphs ∀n∈ properA∀ n _ proper^A (inner nodes of G0(A)G^0(G^A)): (μA)n=(μB)n(μ^A)^n=(μ^B)^n. A sub-family of embeddings ψjAJ0A≤ψjBJ0B\ψ^A_j\_J_0^A≤\ψ^B_j\_J_0^B of a family of embeddings ψjBJ0B\ψ^B_j\_J_0^B is a family of embeddings ψjAJ0A\ψ^A_j\_J_0^A, such that [label=()] 1. Graphs: A≤BG^A ^B. 2. Applicability: ψjAJ0A\ψ^A_j\_J_0^A and ψjBJ0B\ψ^B_j\_J_0^B have the same anchor n0∈An_0 ^A, J0B⊂J0AJ_0^B⊂ J_0^A (note that larger graphs have worse symmetry, cf. Rmk. 4.3) and ∀j∈J0B:ψjA=ψjB|A∀ j∈ J_0^B: _j^A= _j^B|_N^A. A decorated sub-family (ψjAJ0A,LA,A)≤(ψjBJ0B,LB,B)(\ψ^A_j\_J_0^A,L^A,A^A)≤(\ψ^B_j\_J_0^B,L^B,A^B) of a decorated family (ψjBJ0BCLOSE,(\ψ^B_j\_J_0^B, LB,L^B, OPENB)A^B) is a decorated family (ψjAJ0ACLOSE,(\ψ^A_j\_J_0^A, LA,L^A, OPENA)A^A), such that [label=()] 1. Underlying Families: ψjAJ0A≤ψjBJ0B\ψ^A_j\_J_0^A≤\ψ^B_j\_J_0^B. 2. Latents: LA⊂LB∩AL^A⊂ L^B ^A. 3. Ancestral Structure: A⊂PB∖AA^A⊂ P_B A, where PB∖AP_B A contains exactly those pairs (y,x)∈ properA× externA(y,x) _ proper^A×N_ extern^A, such that there is directed path in BG^B exclusively along edges in B∪(B∖A)A^B∪(P^B ^A) from y to x. Larger graphs have larger latent sets in the following sense, which will make algorithmic search simpler: Lemma C.17 (Latent Monotonicity). Given a sub-family of embeddings ψjAJ0A≤ψjBJ0B\ψ^A_j\_J_0^A≤\ψ^B_j\_J_0^B, then ∀LB∈B: ∃LA∈A:LA⊂LB∩A .∀ L^B∈ L^B: ∃ L^A∈ L^A:L^A⊂ L^B ^A . proof Let LB∈BL^B∈ L^B be arbitrary. We first show: OA:=A∖(LB∩A)O^A:=N^A (L^B ^A) is ψAψ^A-observed. Since LBBL^B L^B, OB=B∖LBO^B=N^B L^B is ψBψ^B-observed, thus (by definition, 4.4) |J0B,obs(N,OB)|→∞|J^B,obs_0(N,O^B)|→∞ for N→∞N→∞, where J0B,obs(N,OB)⊂J0B⊂J0AJ^B,obs_0(N,O^B)⊂ J_0^B⊂ J_0^A by definition. Since OA=A∖(LB∩A)⊂OBO^A=N^A (L^B ^A)⊂ O^B, we have J0B,obs(N,OB)⊂J0A,obs(N,OA)J^B,obs_0(N,O^B)⊂ J^A,obs_0(N,O^A). So also |J0A,obs(N,OA)|→∞|J^A,obs_0(N,O^A)|→∞, and by definition 4.4 OAO^A is ψAψ^A-observed. Finally, ∃LA∈A∃ L^A∈ L^A with LA⊂LB∩AL^A⊂ L^B ^A, by definition of A L^A and OAO^A being observed. Remark C.18 (Data Availability). Given a family of embeddings ψjj∈J0\ _j\_j∈ J_0 with a minimal latent-set L then (by definition) O=∖LO=N L is observed. By Def. 4.4, thus |J0obs(N,O)|→∞|J^obs_0(N,O)|→∞ for N→∞N→∞, where J0obs(N,O)=j∈J0|ψj(O)⊂V⁡(N)J^obs_0(N,O)=\j∈ J_0| _j(O)⊂ V(N)\. So there is, asymptotically, infinite data available; this is implicit in the definition of decorated families of embeddings. Note, that freeness at inner nodes (Def. 4.23), together with | inner|<∞|N_ inner|<∞, protects inner nodes sufficiently from degeneracies in this data-set; see also Lemma C.21. Remark C.19 (Relevant Sparse Families). In analogy to Rmk. C.15, we call a decorated family of embeddings (ψjj∈J0CLOSE(\ _j\_j∈ J_0, OPENL,)L,A) weakly maximal if for any (ψj′j∈J0′CLOSE(\ψ _j\_j∈ J _0, OPENL′,′)L ,A ) that satisfies all requirements for (ψj′j∈J0′CLOSE(\ψ _j\_j∈ J _0, OPENL′,′)≤(ψjj∈J0CLOSEL ,A )≤(\ _j\_j∈ J_0, OPENL,)L,A) other than J0′⊂J0J _0⊂ J_0, also J0′⊂J0J _0⊂ J_0 holds. Larger graphs have (essentially) larger ancestral structure, thus together with Lemma C.17 all attached structure is monotonic. Lemma C.20 (Ancestral Monotonicity). Given a family of embeddings ψjAJ0A\ψ^A_j\_J_0^A and a decorated family (ψjBJ0B,LB,B)(\ψ^B_j\_J_0^B,L^B,A^B) with ψjAJ0A≤ψjBJ0B\ψ^A_j\_J_0^A≤\ψ^B_j\_J_0^B, then ∃LA∈A∃ validA suchthat(ψjAJ0A,LA,A)≤(ψjBJ0B,LB,B) .∃ L^A∈ L^A∃ validA^A suchthat(\ψ^A_j\_J_0^A,L^A,A^A)≤(\ψ^B_j\_J_0^B,L^B,A^B) . proof A suitable LA⊂LB∩AL^A⊂ L^B ^A exists by Lemma C.17. We may shrink J0AJ_0^A as long as J0A⊂J0BJ_0^A⊂ J_0^B, thus it is enough to check j∈J0Bj∈ J_0^B for ancestral paths. Let γ be an ancestral path, directed path in the I-graph from ψjA(y)ψ^A_j(y) to ψjB(x)ψ^B_j(x) (see Def. 4.6) for y∈ properAy _ proper^A to x∈ externAx _ extern^A along I-graph edges covered by edges in AG^A. Then either γ is covered by edges in BG^B (and thus (y,x)∈PB∖A(y,x)∈ P_B A) or γ leaves BG^B. If γ leaves BG^B, since BP^B contains only edges pointing to properBN_ proper^B, the last covered edge points at ψjB(y′)ψ^B_j(y ) for y′∈ properBy _ proper^B. To end up at x∈Ax ^A, γ must at some point (possibly at x) reenter BN^B, by Def. 4.1 1 and 2 this can only happen at ψjB(x′)ψ^B_j(x ) for x′∈ pinnedBx _ pinned^B. The restriction of γ to ψjB(y′)↝ψjB(x′) _j^B(y ) _j^B(x ) is an ancestral path, so by validity of BA^B we must have (y′,x′)∈B(y ,x ) ^B. Doing this whenever γ leaves BG^B again, we get a path in B∪(B∖A)A^B∪(P^B ^A) (there are no edges in AP^A as γ is outside of AG^A as ancestral path for AG^A). This shows (y,x)∈PB∖A(y,x)∈ P_B A. C.3 Identifiability We extend on the remarks made in §4.3 about meaningful definitions of identifiablility. Statistical Estimation: We intentionally separate the question of having data for a problem (our notion of identifiability) from the question of the existence of statistical estimators and notions of estimator consistency. Already (absolute) density-estimation is usually an ill-posed problem [62, §2.5 (p. 36f)], also problems like the hardness of conditional independence testing arise from conditional density estimation [53, p. 1517]. Many approaches to such problems, restoring meaningful notions of consistency and of the actually estimable, are known. Influential and informative approaches include VC-theory [62] or Skorohod-embeddings [22, §12 (p. 220f)] (which includes for example functional CLT formulations like Donsker’s theorem, cf. [22, Thm. 12.9]). While learning conditional distributions is not generally possible in a naive sense, these (or often also much simpler) ideas still allow to learn meaningful and practically useful result. For example, in practice one is often interested in causal effects (expectation-values), optimal choices of interventional parameters or approximations by specific function-classes. These problems often can be resolved in a satisfactory way, and provide statistical estimators that provide interpretable results given enough (and suitably curated) data. We focus on the aspect of "suitably curating" data for sub-problems, without regard to what kind of statistical estimator is used in the end. That is, we focus on the causal and symmetry aspects of the problem. The kernels "directly identified" in our notion contain all information about that particular (combination of) model-mechanisms, queries etc., thus any estimator learning some property of that kernel should be applicable to the curated data-set if the actual model-property in question is compatible with estimator-assumptions (like linearity etc.). Stationarity: The random walk example 4.11 may raise, for the time-series case, the question about connections to stationarity. In some cases, it may be possible to restore an IID-like notion of identifiability via a stationarity (or similar) condition. But is such a condition necessary, or even helpful? Fundamentally, the properties of the model and its predictions (like effects) do not seem to have anything to do with stationarity, so unsurprisingly it turns out to be unnecessary in a systematic approach via symmetry (see main text). Concerning the actual helpfulness of such assumptions, there are two cases that should be distinguished: Stationarity, as an a-priori assumption about the form of a specific model(-realization), can be helpful, see §D.4, §5.3. On the other hand stationarity as a means of structuring the problem, besides conceptual doubts (see above), does not seem to solve, but rather to move and obfuscate the difficulty. C.4 Identification from Families of Embeddings The main technical content of §4 – besides finding workable definitions for what embeddings should look like and how additional structure like latent-sets or ancestral information can be encoded – is in the actual identification of structured kernels. We first need a simple result about valid data-sets provided by families of embeddings. Lemma C.21 (Valid Data-Sets). Given a decorated family of embeddings (ψjj∈J0,L)(\ _j\_j∈ J_0,L) and NY⊂ proper∖LN_Y _ proper L, NX⊂∖(NY∪L)N_X (N_Y∪ L), and N0⊂ pinned∩LN_0 _ pinned∩ L then the data-set =((Xj,Yj)j∈J0D=(( X_j, Y_j)_j∈ J_0, X′) X ) defined for j∈J0j∈ J_0 by the tuples Xj X_j = (Vψj(n))n∈NX = ( V_ _j(n))_n∈ N_X Yj Y_j = (Vψj(n))n∈NY = ( V_ _j(n))_n∈ N_Y X′ X = (Vin)n∈N0 . = ( V_i^n)_n∈ N_0 . is valid (Def. 4.12). Remark C.22. Our definition of valid datasets (Def. 4.12, Def. C.8) does not require any guarantee on the non-degeneracy of Xj X_j. In particular, it is always legal to move nodes from X′ X to Xj X_j if they are not hidden (or do not overlap G obsG obs in the proof of Thm. 1 below). This choice should be understood in the context of Ass. 2.23 and Rmk. A.6: If Xj X_j is degenerate (the same random variable Vi V_i appears asymptotically infinitely often), then the directly identified kernel (Def. 4.14) is learned on an observational support which has singular points of non-zero probability. This is no different from other problems related to observational support. If all j lead to the same value, there is no observational support elsewhere and Ass. 2.23 severely restricts allowed conclusions about transfer. But similar problems can occur for non-X-degenerate data-sets as well. It seems conceptually more meaningful to treat this problem as an issue of observational support, rather than as a problem of multi-level statistics, see also §E.7. proof By definition of decorated families, L is a minimal latent subset, thus O:=∖LO:=N L is observed (Def. 4.4), J0obs(N,O) := j∈J0|ψj(O)⊂V⁡(N) , J^obs_0(N,O) := \j∈ J_0| _j(O)⊂ V(N)\ , satisfies|J0obs(N,O)|→∞ forN→∞ . satisfies|J^obs_0(N,O)|→∞ forN→∞ . The sets J0 obs(N)J_0 obs(N) defined in Def. 4.12 as J0 obs(N) := ∈J0 | ∀m:iX,j(m)∈V(N),∀m′:iY,j(m′)∈V(N) ,J_0 obs(N) := \ j∈ J_0 | ∀ m:i_X,j^(m)∈ V(N),∀ m :i_Y,j^(m )∈ V(N) \ , by NX∪NY⊂ON_X∪ N_Y⊂ O, satisfy J0obs(N,O)⊂J0 obs(N)J^obs_0(N,O)⊂ J_0 obs(N), in particular |J0 obs(N)|→∞|J_0 obs(N)|→∞ for N→∞N→∞. The non-degeneracy of elements in NYN_Y, j≠j′⇒∀m:iY,j(m)≠iY,j′(m)j≠ j ∀ m:i_Y,j^(m)≠ i_Y,j ^(m), is satisfied by Def. 4.2 3 and NY⊂ properN_Y _ proper. Next, we show that backdoor free embedded families allow for the identification of structured kernels. The intuition is that ψj(G obs) _j(G obs) behaves like a structural c-component of the I-graph. We need to find suitable sub-structures that are both j-independent and have suitable data-sets attached to them, so that they become directly identifiable. plain Theorem 1 Given a backdoor free family of embeddings (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A) (Def. C.7), then μ(G (),L) (Def.A.3)isδi−identifiable(NotationC.11).μ(G obs(G),L) (Def.\ def:model_alignment_and_kernel)is _i-identifiable(Notation def:delta_i_identified). proof We first focus on the single-level case, the multi-level case is then studied in step 3. Let j∈J0j∈ J_0 be arbitrary. We build a model-aligned structural graph GjG^j with only inner nodes. To this end, set j= innerj= (ψj())N^j=N_ inner^j= _I( _j(N)) and define an edge-set ℰj=(p,c)∈j× innerj|p∈ (c)E^j=\(p,c) ^j×N_ inner^j|p∈ _I(c)\ the same as in the I-graph. Align (inner) nodes to the model-mechanisms, μi:=fiμ^i:=f_i for i∈ innerji _ inner^j. Then define Lj:=L∪(j∖)L^j:=L∪(N^j ). With ψj:↪I _j:N I having (ψj)⊂j ( _j) ^j by construction of jN^j, we can identify nodes in G obsG obs with their images in jN^j (by injectivity of ψj _j, Def. 4.1) making it a sub-graph G obs≤GjG obs≤ G^j (the alignment of nodes agrees by Def. 4.1 3). Step 1: We show, by backdoor freeness, G obs≤GjG obs≤ G^j is a union of structural c-components ( structural c-component Gc≤GjG^c≤ G^j touching, in the sense of non-empty overlap of inner nodes, G obs≤GjG obs≤ G^j is also a structural c-component of G obsG obs and every structural c-component of Gc≤G obsG^c≤ G obs is also a structural c-component of GjG^j). We write n≈jn′n _jn for n≈Ln′n _Ln in (Gj,Lj)(G^j,L^j) and n≈ ′n _ obsn for n≈Ln′n _Ln in (G obs,L)(G obs,L). We have to show: (a) Given n,n′∈ inner obsn,n _ inner obs with n≈ ′n _ obsn , then n≈jn′n _jn and (b) given n∈ inner obsn _ inner obs, n′∈ innerjn _ inner^j with n≈jn′n _jn , then n′∈ inner obsn _ inner obs and n≈ ′n _ obsn . (a) Let n,n′∈ inner obsn,n _ inner obs with n≈ ′n _ obsn be arbitrary. By definition of ≈ _ obs, there are y,w∈ inner obs∖Ly,w _ inner obs L (possibly equal to n or n′n ), latent (all nodes on γ other than y are in L) paths γy:n↝y _y:n y and γw:n′↝w _w:n w in G obsG obs (possibly trivial), an element l∈Ll∈ L and latent paths γy′:l↝yγ _y:l y and γw′:l↝wγ _w:l w in G obsG obs. By L⊂LjL⊂ L^j non-endpoint nodes on these paths are also latent in GjG^j. Edges on these paths are in ℰjE^j by Def. 4.1 2 and the definition of ℰjE^j as I-graph edges. Thus these paths are latent paths in GjG^j and thus n≈jn′n _jn . (b) Let n∈ inner obsn _ inner obs, n′∈ innerjn _ inner^j with n≈jn′n _jn be arbitrary. By definition of ≈j _j, there are y,w∈ innerj∖Ljy,w _ inner^j L^j (possibly equal to n or n′n ), latent paths γy:n↝y _y:n y and γw:n′↝w _w:n w in GjG^j (possibly trivial), an element l∈Ljl∈ L^j and latent paths γy′:l↝yγ _y:l y and γw′:l↝wγ _w:l w in GjG^j. We show: (i) Given a latent path γ in GjG^j starting at a∈ inner obsa _ inner obs to a node b∉Ljb∉ L^j, then b∈ inner obs∖Lb _ inner obs L. Proof of (i): Recall that Lj=L∪(j∖ obs)L^j=L∪(N^j obs), thus b∈ obs∖Lb obs L. By contradiction, assume it were b∈ outer obsb _ outer obs. With a∈ inner obsa _ inner obs, we have a≠ba≠ b and γ is non-trivial (contains more than one node). By definition (Def. A.1), there are no (proper) edges in G obsG obs ending at b, thus the last node before b on γ is not in obsN obs (using Def. 4.1 2). The starting point a of γy _y is in obsN obs, thus there is a last (along γy _y) node l obs∈ obs∩Lj= obs∩Ll obs obs∩ L^j=N obs∩ L. Restrict γ to l obs↝bl obs b. This is a path in the I-graph (by construction of GjG^j) not through G obsG obs. By validity of the ancestral structure A (Def. 4.6), there is l obs↝bl obs b in A. This is a contradiction to backdoor freeness of (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A). Next we show: (i) Given a latent path γ in GjG^j ending at b∈ inner obs∖Lb _ inner obs L, then γ is a latent path in G obsG obs. Proof of (i): By contradiction. If γ is not a latent path in G obsG obs, then by L⊂LjL⊂ L^j, there is a last (along γ) node a with a∈Lj∖ inner obsa∈ L^j _ inner obs. Clearly a≠ba≠ b, because b∈ inner obsb _ inner obs. By outer obs= extern∪( pinned∖L)N_ outer obs=N_ extern∪(N_ pinned L) by definition of G obsG obs and extern∩L=∅N_ extern∩ L= by backdoor freeness, we have outer obs∩L=∅N_ outer obs∩ L= . By definition of LjL^j also outer obs∩Lj=∅N_ outer obs∩ L^j= . In particular (∗)(*) a∉ obsa obs. But a∈ (b′)a∈ _I(b ) where b′b is the node directly after a (on γ). Note that b′∈ inner obsb _ inner obs, because a by definition was the last node (along γ) not in inner obsN_ inner obs. By Def. 4.1 1 also a∈ obsa obs, contradicting the previous result (∗)(*). Finalizing the proof of (b): By (i) applied to γy _y: y∈ inner obs∖Ly _ inner obs L. Thus by (i), γy _y and γy′ _y are latent paths in G obsG obs. By (i), applied to γw _w, w∈ inner obs∖Lw _ inner obs L. Thus by (i), γw _w and γw′ _w are latent paths in G obsG obs. In particular n′∈ inner obsn _ inner obs. Finally the latent paths γy _y, γy′ _y and γw _w, γw′ _w in G obsG obs show n≈ ′n _ obsn . Step 2: We show the single-level ( pinned=∅N_ pinned= ) case. For n∈ inner obs∖Ln _ inner obs L, contained in the structural c-component Gc≤G obs,GjG^c≤ G obs,G^j (by step 1), we have An(G obs,L)=An(Gc,Lc)=An(Gj,Lj)A^n(G obs,L)=A^n(G^c,L^c)=A^n(G^j,L^j) by applying Lemma A.17 1 twice. Crucially An(G obs,L)A^n(G obs,L) (and thus An(Gj,Lj)A^n(G^j,L^j)) does not depend on j. Further, by Lemma A.14 2, μ⁡(Gj,Lj)=∏n′∈ innerj∖LAn′(Gj,Lj)μ(G^j,L^j)= _n _ inner^j LA^n (G^j,L^j) (note that only terms n′n also in inner obs∖LN_ inner obs L do not depend on j), thus for n∈ inner obs∖Ln _ inner obs L: margn′>n⁡(μ⁡(Gj,Lj))=[∏n′∈ innerj∖Lj,n′<nAn′(Gj,Lj)]⊗An(G obs,L) . _n >n(μ(G^j,L^j))= [ _n _ inner^j L^j,n <nA^n (G^j,L^j) ] A^n(G obs,L) . This is already in the correct form for the right-hand-side in Def. 4.14. For n∈ inner obs∖Ln _ inner obs L, by Lemma C.21, there is further a valid data-set nD_n for NXn=n′∈ innerj∖Lj|n′<nN_X^n=\n _ inner^j L^j|n <n\ (by innerj∖Lj= obs∖LN_ inner^j L^j=N obs L Lemma C.21 applies), NYn=nN_Y^n=\n\ and N0n=∅N_0^n= . Finally, by Lemma B.3 1 Pθ(Xj,Yj)=margn′>n⁡(μ⁡(Gj,Lj)) ,P_θ( X_j, Y_j)= _n >n(μ(G^j,L^j)) , so Def. 4.14 applies and An(G obs,L)A^n(G obs,L) is directly identifiable. By Lemma A.14 2, μ⁡(G obs,L)μ(G obs,L) is identifiable (Def. 4.16). Step 3: We show the multi-level case. Define the structural graph GjG^j as before. Applying Lemma B.3 2 (cf. below for satisfiability of the hypothesis) with N0=ψj( pinned)N_0= _j(N_ pinned) provides a (Gj)′(G^j) ; the statement of the lemma ensures G [δi=δ[i]]≤(Gj)′G obs[ _i=δ[D_i]]≤(G^j) . Note that, if inner obs∩ ( pinned)=∅N_ inner obs∩ _I(N_ pinned)= , its graphical property (see Lemma B.3 2) applies for Y= inner obsY=N_ inner obs by result (i) of step 1 above and then replaces (i) in the remainder of step 1. Then step 1 produces the same result as before. The condition inner obs∩ ( pinned)=∅N_ inner obs∩ _I(N_ pinned)= in the hypothesis of Lemma B.3 2 may not hold true for all j∈J0j∈ J_0. It fails however only for finitely many, and thus we can replace J0J_0 by a (still infinite) J0′:=J0∖J0 invalidJ_0 :=J_0 J_0 invalid. Removing a finite (N-independent) subset does not affect asymptotic properties: if the used data-set D is valid (using an infinite subset of J0J_0), then the removal of any intersection with J0 invalidJ_0 invalid does not change this validity (infinitude of the subset of J0J_0). The subset we want to remove is: J0 invalid := j∈J0 | ψj( proper)∩ (ψj( pinned))≠∅ .J_0 invalid := \ j∈ J_0 | _j(N_ proper)∩ _I( _j(N_ pinned))≠ \ . This set is finite: By the finite past assumption (Ass. 2.11), (ψj( pinned)) ( _j(N_ pinned)) is finite. For any n∈ propern _ proper, by freeness Def. 4.23, ψ∗(n) _*(n) is injective, thus ψ∗(n)−1( (ψj( pinned)))⊂J0 _*(n)^-1( ( _j(N_ pinned))) ⊂ J_0 is finite. The number of inner nodes | proper|<∞|N_ proper|<∞ is also finite (by Def. C.1) thus ∪n∈ properψ∗(n)−1( (ψj( pinned))) _n _ proper _*(n)^-1( ( _j(N_ pinned))) is finite. Finally we show J0 invalid⊂∪n∈ properψ∗(n)−1( (ψj( pinned)))J_0 invalid⊂ _n _ proper _*(n)^-1( ( _j(N_ pinned))) (thus J0 invalidJ_0 invalid is finite as subset of a finite set): Let j∈J0 invalidj∈ J_0 invalid be arbitrary. By construction, ψj( proper)∩ (ψj( pinned))≠∅ _j(N_ proper)∩ _I( _j(N_ pinned))≠ , ∃n∈ proper:ψj(n)∈ (ψj( pinned))∃ n _ proper: _j(n)∈ _I( _j(N_ pinned)). Therefore j∈ψ∗(n)−1( (ψj( pinned)))j∈ _*(n)^-1( ( _j(N_ pinned))), in particular j∈∪n∈ properψ∗(n)−1( (ψj( pinned))) .j∈ _n _ proper _*(n)^-1( ( _j(N_ pinned))) . Returning to step 3, with step 1 established, it remains to modify step 2. μ⁡(G obs,L)[δi]μ(G obs,L)[ _i] now is a functional of δi _i for i∈ψj( pinned)i∈ _j(N_ pinned), but step 2 works essentially as before, Lemma C.21 is applied with NXn=n′∈ inner′∖(L′∪N0)|n′<nN_X^n=\n _ inner (L ∪ N_0)|n <n\ NYn=nN_Y^n=\n\ and N0n=N0N_0^n=N_0 to obtain a data-set nD_n on which Lemma B.3 2 yields Pθ(Xj,Yj|X′=x′)=margn′>n⁡(μ⁡(G′,L′)[x′]) ,P_θ( X_j, Y_j| X =x )= _n >n(μ(G ,L )[x ]) , thus Def. 4.14 applies and An(G obs[δN0[x′]]CLOSEA^n(G obs[ _N_0[x ]] is directly identifiable, where X′=x′ X =x is the value taken by nodes in (the j-independent) N0N_0 in the data-set n⊂⁡(ω)D_n (ω), ′=Vψj( pinned)(ω)x = V_ _j(N_ pinned)(ω). The claim of the Lemma is that μ⁡(G obs,L)μ(G obs,L) is δi _i-identifiable, remains to show, that μ⁡(G obs,L)[δ⁡[Vψj( pinned)(ω)]]= a.s.μ(G obs,L)[δ⁡[Vψj( pinned)]] .μ(G obs,L)[δ[ V_ _j(N_ pinned)(ω)]] a.\,s.=μ(G obs,L)[δ[ V_ _j(N_ pinned)]] . The left-hand-side is a constant (not random), thus we have to show PθP_θ^D a. s. the right-hand-side takes this value. Let ω′∈Ωω ∈ with ⁡(ω′)=⁡(ω)D(ω )=D(ω) be arbitrary (by definition Pθ(⋅)=Pθ(⋅|Vii∈=(ω))P_θ^D(·)=P_θ(·|\ V_i\_i∈ V=D(ω)), so we know ⁡(ω′)=⁡(ω)D(ω )=D(ω) almost surely). By the above result μ⁡(G obs,L)[δ⁡[Vψj( pinned)(ω′)]]μ(G obs,L)[δ[ V_ _j(N_ pinned)(ω )]] is (uniquely) identifiable from ⁡(ω′)=⁡(ω)D(ω )=D(ω), thus μ⁡(G obs,L)[δ⁡[Vψj( pinned)(ω′)]]=μ⁡(G obs,L)[δ⁡[Vψj( pinned)(ω)]] .μ(G obs,L)[δ[ V_ _j(N_ pinned)(ω )]]=μ(G obs,L)[δ[ V_ _j(N_ pinned)(ω)]] . Remark C.23 (Practical Computation of Extraction). Inspecting step 2 of the proof of Thm. Theorem , it becomes evident that we first have to learn (j-independent) atoms AnA^n for n∈ inner∖Ln _ inner L. This can be achieved in practice as described in Rmk. A.18: We know the relevant arguments and data-set, thus we simply apply our favorite estimator. Knowing all the atoms of (G obs,L)(G obs,L), by Lemma A.14 2, we can compute μ⁡(G obs,L)μ(G obs,L) as their product. We have thus seen, that backdoor-free families of embeddings allow for the identification of associated kernels from data. On the other hand, backdoor-free families of embeddings are a rather generic means of curating data-sets with invariant properties, thus it seems plausible that Conj. 3 might be true. Remark C.24. It seems that internal structure (like linearity) can be employed post-hoc as a computational means (see §5.3) without the need to modify this claim. It is however conceivable that using internal structure might allow to effectively improve symmetry locally, in which case the above conjecture 3 could hold at most in the generic case. C.5 Algorithmic Construction of Backdoor-Free Families Algorithm ExtractCS-R shows how to extend a given decorated family (ψ,L,)(ψ,L,A) to obtain (non-unique) c-connected backdoor-free families of embeddings. Definition C.25 (C-Connected Backdoor-Free Families). We call a backdoor-free family of embeddings (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A) c-connected, if G obs(,L)G obs(G,L) is c-connected. It is enough to find all c-connected backdoor-free families: Lemma C.26. Let ℬ CB_ C be the set of all c-connected backdoor-free families of embeddings and k′=(G′,L′)∈⋃(ψ,L,)∈ℬk⁡(ψ,L,)k =(G ,L )∈ _(ψ,L,A) k(ψ,L,A) (cf. Def. 4.17), then ∃(ψ1,L1,1),…,(ψm,Lm,m)∈ℬ C∃(ψ^1,L^1,A^1),…,(ψ^m,L^m,A^m) _ C such that by repeated application of Lemma 3.8 μ⁡(G′,L′)μ(G ,L ) can be regularly computed from μ⁡(G obs1,L1),…,μ⁡(G obsm,Lm)μ(G obs_1,L^1),…,μ(G obs_m,L^m). In particular by Thm. 1, k′∈ basick _ basic is identifiable from only c-connected backdoor-free families. proof Let Gc≤G′=G obs(′,L′)G^c≤ G =G obs(G ,L ) be a structural c-component. Define a local graph ′G with inner nodes innerc∖( pinned′∩L)N_ inner^c (N_ pinned ∩ L), pinned nodes innerc∩( pinned′∩L)N_ inner^c∩(N_ pinned ∩ L) and external nodes outercN_ outer^c. Then the restrictions of ψj′|′ψ _j|_N to ′G are again a family of embeddings. The choices L′=L′∩′L =L and ′=′∩( proper′× pinned′)A =A ∩(N_ proper ×N_ pinned ) are such that ′∖L′N L is observed and ′A is valid, but they may not be minimal anymore. If a smaller (minimal) L′⊂L′L ⊂ L exists, repeat the previous proof-steps with (potentially smaller) c-components of ((G obs)′,L′)((G obs) ,L ); note: smaller ′A will result in G obsG obs having more simplifications (Lemma A.12), which is needed to get smaller pieces, but is not needed to recover the glued G′G . Lemma 3.8 automatically applies with the c-components (which each contain themselves). If the above steps were repeated with smaller L′L , the gluing target (larger graph) is the one with L=∪cL′cL= _cL _c (where Lc′L _c is L′L of c-component c) a finial marginalization will connected this result to the original G′G . Algorithm ExtractCS-R extract_cstructures_relative Input: A decorated family of embeddings FB=(ψjBj∈J0B,LB,B)F^B=(\ _j^B\_j∈ J_0^B,L^B,A^B). Output: The set of backdoor-free families of embeddings F=(ψjj∈J0,L,)F=(\ _j\_j∈ J_0,L,A) c-connected and with FB≤F^B≤ F (Def. C.16). [label=0)] 1. Attach monotonic structure: L:⁣=∀L ∀:= minimal_hidden(ψ,L)(ψ,L). ⊳ No-op in first iteration. :⁣=∀A ∀:= minimal_ancestral(ψ,L,)(ψ,L,A). ⊳ Relative choice by Lemma C.20. 2. Absorb hidden parent-chains: ψ:⁣=∀ψ ∀:= absorb_parents (ψ,L∩( outer∖ pinned))(ψ,L∩(N_ outer _ pinned)). Repeat (1–2) until L∩( outer∖ pinned)=∅ .L∩(N_ outer _ pinned)= . (∗*) 3. Absorb starting nodes of backdoor paths: ψ,w:⁣=∀ψ,w ∀:= absorb_child_ancestral(ψ,L,)(ψ,L,A). Repeat (1–3) until (ψjj∈J0,L,) isbackdoor−free.(\ _j\_j∈ J_0,L,A) isbackdoor-free. (∗⁣∗**) 4. Collect branch into output set: Yield (ψjj∈J0,L,)(\ _j\_j∈ J_0,L,A). Notation: For set-returning sub-algorithms, we write :⁣=∀ ∀:= short for iterate over all elements of. Yield returns to iterating those other elements, appending the yielded result to the output set. Details on subroutines are given in F.4. Algorithm ExtractCS extract_cstructures Input: Implicitly the model and viewport. Output: The set of c-connected backdoor-free families of embeddings F=(ψjj∈J0,L,)F=(\ _j\_j∈ J_0,L,A). [label=0)] 1. Initialize: k:=0k:=0. R0:=∅R_0:= . For each J∈⁡(M)J (M): Add FCS-R( direct_embbed(J) ) to R0R_0. 2. Step: Rk+1=RkR_k+1=R_k. k:=k+1k:=k+1. For each F∈RkF∈ R_k and For each χ∈Chrelevant⁡(F)χ∈ (F): Add FCS-R( absorb_child(F,χ)(F,χ) ) to RkR_k. Repeat 2 until Rk=Rk−1R_k=R_k-1 or limit k maxk_ max reached. 3. Finalize: Return RkR_k. Remark: We separate Algo. ExtractCS from Algo. ExtractCS-R because, for most practical applications, for example for use as input to Algo. SVSearch,, searching all relevant children in step 2 is not necessary, rather the match (and even gluable) filters of SVSearch can be integrated directly with Algo. ExtractCS-R to only enumerate useful families of embeddings. direct_embbed(J) returns the embedding of Lemma F.1. In the algorithms ExtractCS-R and ExtractCS we sketch the general logic for the systematic construction of c-connected backdoor-free families of embeddings (absorbing operations are given in §F.4). Important to note are the many non-unique partial constructions, discernible from the :⁣=∀ ∀:= operators in ExtractCS-R and for-each statements in ExtractCS. These constructions in the IID case (§E) are usually unique (Lemma E.22). Both algorithms are intentionally written such that ignoring these non-uniqueness challenges, they are easy to read and make sense for the IID case. Indeed the construction of c-components – leaving non-uniqueness aside (each mechanism can occur locally in many different c-components) – in the IID case is very similar. There are however two notable differences: First, in enumerating all (or all matching etc.) c-components ExtractCS always starts from embedding a single variable (in fcs-r(( direct_embed(J))(J))), then adds children (ancestral or relevant) relative to this embedded variable. There is a simple reason for this approach: Embedding disconnected graphs does not usually make sense in our formalism (any form of "rigidity", equivariance relative to an anchor in another component, must be introduced in some ad-hoc way). Second, and maybe more importantly, we cannot, in general, start from maximal size c-components and go to c-subgraphs as we can in the IID case. The "largest" local c-structure is also the local c-structure with worst symmetry (Rmk. 4.3), it may therefore happen that smaller c-structures (proper sub-graphs of this largest one) are observed (have smaller than the trivial intersection minimal latent set L), while the largest one is not. Thus it may happen that these smaller sub-graphs lead to better identification results. It seems usually not advisable to eagerly discover all elements of ℬ CB_ C. Rather, given a query, only parents and children matching the structure of the query have to be inspected (see Algo. SVSearch). These matching results can be discovered lazily (see §F.4). Appendix D Details on Query Identification The formal claims of §5 fall in two main categories: The representation of basic queries by structured queries and the relation of underlying basic queries, in particular their target in the realized distribution, to the material of §3 and §4. These aspects are discussed separately in the two subsections §D.2 and §D.3. As in §C, we start by stating some small modifications that allow to account for the multi-context case. Finally, we briefly outline how our work fits into a larger formal context and can be extended in future work, for example to include instrumental variable arguments or stationarity considerations for time-series. D.1 Multi-Level Modifications Proofs are given in the sub-sections below, here we only provide the correct statements to be proved for the multi-level case. In the multi-level case, we need to identify "contexts", Vi V_i shared by the observations and a query. In example 4.18, a query about an intervention in a specific context ( -site), has relevant contexts ( ) as auxiliary index (see below). A query about an unknown (generic) context on the other hand instead contains a copy (a separate node sharing the mechanism but not observed descendants) of this context ( is a Baysian estimate integrating out the prior over, for example, slopes; see §D.5). Definition D.1 (Auxiliaries). This has no single-level analogue: Given an embedded local graph ψ:↪Iψ:G I, we call I aux(ψ):= (ψ⁡())∩ ()I aux(ψ):= _I(ψ(N))∩ _I( V) ψ-auxiliary indices. While it is not immediately obviously at a first glance, the basic query implicitly accounts for multi-level structure: this is because it is formulated on PθP_θ^D, not on PθP_θ. The hierarchical structure emerges in embeddings and structural queries from these basic queries, despite the absence of an explicit hierarchy in either the basic query or the model. Compared to Def. 5.4, structured queries in the multi-level case have to additionally account for auxiliary nodes: Definition D.2 (Structured Query). A structured query is an embedded local graph ψ:↪Iψ:G I together with a set of interest Y⊂ properY _ proper, such that =∅A= is valid (Def. 4.6) and (multi-level case only:) ψ⁡( pinned)=I aux(ψ)∩ψ⁡(∖ extern)ψ(N_ pinned)=I aux(ψ)∩ψ(N _ extern). Underlying Query: Given a structured query (ψ,Y)(ψ,Y), there is an underlying basic query q⁡(ψ,Y):=(Y~=ψ⁡(Y),X~=ψ⁡( extern),θ)q(ψ,Y):=( Y=ψ(Y), X=ψ(N_ extern),θ). We call (ψ,Y)(ψ,Y) identifiable, if the underlying basic query is identifiable. Before connecting basic queries to structured queries in the multi-level case, we first need to make a small modification to associated structural graphs (for reasons explained below in Rmk. D.4). In §C, for the multi-level extraction, we used an associated structural graph G obsG obs (Def. C.12), the next definition is almost identical, except for the treatment it gives to observed pinned nodes: Definition D.3 (Associated Structural Query-Graphs). We are given a model-aligned local graph (,L)(G,L), with L∩ extern=∅L _ extern= . We define a model-aligned structural graph G query()G query(G) with nodes query=N query=N split as inner query:= proper∪ pinned , outer query:= externN_ inner query:=N_ proper _ pinned , _ outer query:=N_ extern and the edges ℰ query=E query=P. Then G queryG query is model-aligned with μnμ^n for n∈ propern _ proper and μx:=δix∈ℱ Iμ^x:= _i^x _ I for x∈ pinnedx _ pinned. Remark D.4 (Comparison of Associated Structural Graphs). Extraction and queries take a different perspective on observed pinned nodes. Extraction: We want to learn the most informative object from data. Given an observed pinned node, we can treat it as an outer node with the understanding that we know the associated δi=δ⁡[Vi] _i=δ[ V_i] since (Vi V_i is observed), thus we can plug in the value of Vi V_i in case we want to recover the context-specific result, or we may use the structured kernel like any other kernel. We will not have actually useful observational support (Rmk. A.6) for doing much else (other than plugging in the actual value of Vi V_i) with it, but conceptually, since we are working under Ass. 2.23, it seems more logical to retain the general object rather than suddenly mixing support-arguments into our otherwise support-agnostic treatment. Query: We have to produce a specific target kernel (Def. 5.3), and must fill in all information required to compute that target. The value of an observed Vi V_i is fixed in PθP_θ^D, thus if it affects the query (is in its structural query), the information about the (known) value of Vi V_i must be used, and the structural graph of the query cannot leave "open" additional external nodes. Thus and inner node must be aligned to δi _i. Identification: Note that for identification, we typically have to glue (Lemma 3.8) the structural query graph from smaller (extracted, §4) pieces. These pieces can contain a composition with δi _i (if they are extracted with pinned nodes, cf. Thm. Theorem ), but it is also legal to extract a (standard, no δi _i) kernel with the pinned query-node as outer-node and glue δi _i (as a structured kernel whose graph has a single node, cf. k⁡(i)k(i) in Def. C.13), which is (directly) identifiable if i∈i∈ V, Vi V_i is observed (cf. Def. C.10). This more sophisticated structured query-formulation can capture additional basic queries, hypothesis in Lemma 5.5 can be weakened (we still exclude "queries causing observations" by enforcing Y~∩ ()=∅ Y∩ _I( V)= , cf. Rmk. D.6, but this is a much weaker condition than disconnecting Y~ Y from V in the I-graph): Lemma D.5 (Associated Structured Query). Given an abstract query q=(Y~,X~,θ)q=( Y, X,θ), then there is a structured query (ψ⁡(q),Y~)(ψ(q), Y) with underlying query q, if and only if Y~∩ ()=∅ Y∩ _I( V)= (replacing the stronger disconnectivity-hypothesis of the single-level case) and X~ X cannot be bypassed in the sense of: For y∈Y~y∈ Y and w∈ (X~)∖(X~∪ψ⁡( pinned))w∈ _I( X) ( X∪ψ(N_ pinned)), there is no directed path γ:w↝yγ:w y in the I-graph with γ∩(X~∪ψ⁡( pinned))=∅γ∩( X∪ψ(N_ pinned))= (multi-level: γ now also does not pass through the previously empty ψ⁡( pinned)ψ(N_ pinned)). Remark D.6 (Assumptions for Structured Representation). The condition Y~∩ ()=∅ Y∩ _I( V)= intuitively speaking means the query does not cause the observations. For "interventional" formulations (like the do-calculus) this is always true; one plausible use-case violating this assumption would be missing-value imputation, the problem we avoid by this simplification is to give a formal interpretation of what "imputation" should mean precisely (for example, should it use downstream observations to narrow down imputed distributions?). Finally, in Lemma Thm. 2 the hypothesis has to be (and can be) weakened to δi _i-identifiable inputs: Lemma D.7. Given a structured query (ψ,Y)(ψ,Y), using L:=( proper∖Y)∪ pinnedL:=(N_ proper Y) _ pinned (multi-level: includes pinnedN_ pinned), if there is a regular functional F computing μ⁡(G query(),L)=F⁡[μ1,…,μn]μ(G query(G),L)=F[ _1,…, _n] with μ1,…,μn _1,…, _n all δi _i-identifiable (multi-level: replace identifiable by δi _i-identifiable), then the underlying query q⁡(ψ,Y)q(ψ,Y) is identifiable. D.2 Structured Representation of Basic Queries The representation of basic queries by structured queries in the main text (Lemma 5.5) was phrased as a single-level simplification of (and immediately follows from) the following Lemma D.8 (Associated Structured Query). Given an abstract query q=(Y~,X~,θ)q=( Y, X,θ), then there is a structured query (ψ⁡(q),Y~)(ψ(q), Y) with underlying query q, if and only if Y~∩ ()=∅ Y∩ _I( V)= and X~ X cannot be bypassed in the sense of: For y∈Y~y∈ Y and w∈ (X~)∖(X~∪ψ⁡( pinned))w∈ _I( X) ( X∪ψ(N_ pinned)), there is no directed path γ:w↝yγ:w y in the I-graph with γ∩(X~∪ψ⁡( pinned))=∅γ∩( X∪ψ(N_ pinned))= . proof By slight abuse of notation, we will use injectivity of ψ (Def. C.2) to identify N with ψ⁡()ψ(N). "⇐ ": We start by constructing a local graph G. First, the node-set is built inductively as a subset of I as follows: We start from proper0:=0:=Y~N_ proper^0:=N^0:= Y. Then k+1 ^k+1 :=k∪ ( properk) , :=N^k∪ _I(N_ proper^k) , externk+1 _ extern^k+1 :=k+1∩X~ , :=N^k+1∩ X , pinnedk+1 _ pinned^k+1 :=(k+1∖ externk+1)∩ () , :=(N^k+1 _ extern^k+1)∩ _I( V) , properk+1 _ proper^k+1 :=k+1∖( externk+1∪ pinnedk+1) . :=N^k+1 (N_ extern^k+1 _ pinned^k+1) . By finiteness of Y~ Y (Def. 5.3) and finite past (Ass. 2.11), | (Y~)|<∞| _I( Y)|<∞, so we may define the finite sets :=∞:=∪kN:=N^∞:= _kN^k etc., and obtain a graph G by adding edges from the I-graph, :=(p,c)∈× proper|p∈ (c)P:=\(p,c) ×N_ proper|p∈ _I(c)\. Then define ψ:↪I,n↦nψ:N I,n n as the (restricted) identity mapping, align to model-mechanisms μi=fJ⁡(i)μ^i=f_J(i) and pin x∈ pinnedx _ pinned at ix=xi^x=x, finally define the node-set of interest as Y:=Y~Y:= Y. Then by construction the underlying query is q. We have to check that (a) Y⊂ properY _ proper, (b) =∅A= is valid, (c) ψ⁡( pinned)=I aux(ψ)∩ψ⁡()ψ(N_ pinned)=I aux(ψ)∩ψ(N) and that (d) ψ:↪Iψ:G I is indeed an embedded local graph (satisfies Def. 4.1). Part (a): By Def. 5.3 X~∩Y~=∅ X∩ Y= , thus extern∩Y=∅N_ extern∩ Y= . By hypothesis, Y~∩ ()=∅ Y∩ _I( V)= thus pinned∩Y=∅N_ pinned∩ Y= . So, indeed, Y⊂ properY _ proper. Part (b): We have to check Def. 4.6. Let w∈ properw _ proper and x∈ externx _ extern be arbitrary. By contradiction. Assume there were a directed path γ:w↝xγ:w x in the I-graph not through G. In particular w∈ (X~)∖(X~∪ψ⁡( pinned))w∈ _I( X) ( X∪ψ(N_ pinned)). By w∈ properw _ proper and by construction of properN_ proper there is path η:w↝yη:w y to y∈Y~y∈ Y with η⊂ properη _ proper (the chain of parents of y that inductively were included in subsequent properkN_ proper^k). But η also is a path as described in the statement of the lemma (it starts at w∈ (X~)∖(X~∪ψ⁡( pinned))w∈ _I( X) ( X∪ψ(N_ pinned)) and ends at y∈Y~y∈ Y), thus by hypothesis η∩(X~∪ψ⁡( pinned))≠∅η∩( X∪ψ(N_ pinned))≠ , which contradicts η⊂ properη _ proper. Part (c): By construction pinned=(∖ extern)∩ ()N_ pinned=(N _ extern)∩ _I( V) and thus pinned=I aux(ψ)∩(∖ extern)N_ pinned=I aux(ψ)∩(N _ extern) by definition (Def. D.1). Part (d): First note that G is indeed a local graph, point only at proper nodes by construction. For Def. 4.1, ψ is the identity mapping, thus injective. (i) ( proper)⊂ _I(N_ proper) by construction. (i) Edges equal I-graph edges by construction. (i) Model-alignment is to fJ⁡(i)f_J(i) and to ix=x=ψ⁡(x)i^x=x=ψ(x) by construction. "⇒ ": Part 1 (ψ⁡(Y)∩ ()=∅ψ(Y)∩ _I( V)= ): Y⊂ properY _ proper, but ()∩ψ⁡()⊂ (ψ)∩ψ⁡() _I( V)∩ψ(N)⊂ I aux(ψ)∩ψ(N) (by Def. D.1) and I aux(ψ)∩ψ⁡()⊂ψ⁡( extern∪ pinned)I aux(ψ)∩ψ(N)⊂ψ(N_ extern _ pinned) (by Def. 5.4), so by proper∩( extern∪ pinned)=∅N_ proper∩(N_ extern _ pinned)= (by Def. C.1) ψ⁡(Y)∩ ()=∅ψ(Y)∩ _I( V)= . Part 2 (no bypassing of X~ X): By contradiction. Assume there were y∈Y~y∈ Y and w∈ (X~)∖(X~∪ψ⁡( pinned))w∈ _I( X) ( X∪ψ(N_ pinned)), and a directed path γ:w↝yγ:w y in the I-graph with γ∩(X~∪ψ⁡( pinned))=∅γ∩( X∪ψ(N_ pinned))= . With proper parents being included Def. C.2 1, and γ∩(X~∪ψ⁡( pinned))=∅γ∩( X∪ψ(N_ pinned))= , all nodes on γ (going backwards from y step by step) must be in Y~ Y, in particular w∈Y~w∈ Y, contradicting validity of A. D.3 Identification of Queries Finally, for the identification of structured queries, we connect their structured kernels to the realized world, then confirm that regular computation works on δi _i-identifiable objects as is, and then assemble both to a proof of Thm. 2. Having fixed a structured graph associated to the structured query (Def. D.3, note that in the single-level case G query=G query=G), we start by associating it to the realized world distribution; this is actually already the main technical step of the query-formalism. Lemma D.9. Given a structured query (,Y)(G,Y), then, using L:=( proper∖Y)∪ pinnedL:=(N_ proper Y) _ pinned, Pθ(Vψ⁡(Y)|Vψ⁡( pinned),Vψ⁡( extern)=x~) = (μ(G ,L)[δi=δ[Vi]])x=x~ .P_θ^D( V_ψ(Y)| V_ψ(N_ pinned), V_ψ(N_ extern)= x) = (μ(G query,L)[ _i=δ[ V_i]] )_x= x . If ∃ν∃ν a probability kernel with μ(G ,L)[δi=δ[Vi]]=νμ(G query,L)[ _i=δ[ V_i]]=ν almost surely PθP_θ^D, then Pθ(Vψ⁡(Y)|Vψ⁡( extern)=x) = νx .P_θ^D( V_ψ(Y)| V_ψ(N_ extern)=x) = _x . Remark: The last equation, by our notation of equality of kernels (cf. discussion in and around Rmk. A.6) is an equality almost everywhere, so the "almost surely" part did not magically vanish, it was just absorbed into our notation of kernel equality . proof We will write Y~:=ψ⁡(Y) Y:=ψ(Y) and X~:=ψ⁡( extern) X:=ψ(N_ extern). We start by using an embedded local graph analogue of simplification (Lemma A.12) which allows us to assume ψ⁡()⊂ (ψ⁡(Y))ψ(N)⊂ _I(ψ(Y)). Applying Lemma A.12 with the unique maximal B for L=( proper∖Y)∪ pinnedL=(N_ proper Y) _ pinned (see statement of the Lemma) we get (G query)′(G query) with inner′⊂ (Y)N_ inner ⊂ _G(Y), and μ⁡(G query,L)=μ⁡((G query)′,L′)μ(G query,L)=μ((G query) ,L ). The restriction of ψ to (G query)′(G query) is still a structured query. The hypothesis of the lemma is thus still satisfied, and the claimed result is the same, thus we may proof the Lemma for this new structured query instead. This new structured query satisfies ψ⁡()⊂ (ψ⁡(Y))ψ(N)⊂ _I(ψ(Y)). By finite past (Ass. 2.11), (ψ(Y)) _I(ψ(Y)) is finite. We use Rmk. B.2 to write I=ℕI=N as totally ordered set and we may assume (ψ⁡(Y))=1,…,m _I(ψ(Y))=\1,…,m\. Next, we show that by =∅A= being valid the query-graph is not "interlaced" with any of its ancestors, that is we can further assume ψ⁡( proper)=m0,m0+1,…,m−1,mψ(N_ proper)=\m_0,m_0+1,…,m-1,m\, can put ψ⁡( proper)ψ(N_ proper) a the π-end of (ψ(Y)) _I(ψ(Y)). To see this, note that by validity of =∅A= and inclusion of proper parents Def. 4.1 1, nodes in (ψ⁡( proper))= (ψ⁡()) _I(ψ(N_ proper))= _I(ψ(N)) cannot have arguments in any ψ⁡( proper)ψ(N_ proper), so by sparsity (Lemma G.27) we can move them all to the left of ψ⁡( proper)ψ(N_ proper). By definition of PθP_θ (Def. 2.13): Pθ(V (Y~))=[⊗k=1m0−1fk]⊗[⊗k=m0mfk]P_θ( V_ _I( Y))=[ _k=1^m_0-1f_k] [ _k=m_0^mf_k] By uniqueness of disintegrations, with Pθ(Vm0,…,m|V1,…,m0−1)P_θ( V_m_0,…,m| V_1,…,m_0-1) satisfying the characterizing property of Lemma G.22, and disintegrating m0−1m_0-1 times from the left: Pθ(Vm0,…,m|V1,…,m0−1)=[⊗k=m0mfk] .P_θ( V_m_0,…,m| V_1,…,m_0-1)=[ _k=m_0^mf_k] . Combining this with the definition of structured kernels (and model-alignment of ψ, Def. 4.1 3) using the extraction-associated graph G obs(,L∖ pinned)G obs(G,L _ pinned), which has outer nodes extern∪ pinnedN_ extern _ pinned (denoted with value x′=(x~,d)x =( x,d), where x~ x is associated to externN_ extern and thus X~ X, while d the pinned part related to the dataset), we thus already find μ:=μ⁡(G obs(,L∖ pinned)x′=(x~,d) μ:=μ(G obs(G,L _ pinned)_x =( x,d) = ([⊗k=m0mfk]) = _L([ _k=m_0^mf_k]) = Pθ(VY~|V1,…,m0−1=(x~,d)) . = P_θ( V_ Y| V_1,…,m_0-1=( x,d)) . Importantly, if we treat the model-aligned multi-level kernel μ⁡(G query,L)[δi]μ(G query,L)[ _i] as a functional of its place-holder kernels δi _i, then with all proper nodes aligned the same in μ⁡(G query,L)μ(G query,L) and μ⁡(G obs(,L∖ pinned)CLOSEμ(G obs(G,L _ pinned), μ:=μ(G (,L∖ pinned),L)x′=(x~,d) = (μ(G ,L)[δi=δ[di]])x′=(x~) ,μ:=μ(G obs(G,L _ pinned),L)_x =( x,d) = (μ(G query,L)[ _i=δ[d_i]] )_x =( x) , (∗*) and it suffices to compute μ, for di=Vid_i= V_i ( the random variable Vi V_i into the argument did_i; this is the right-hand side of the claim of the Lemma). For structured queries ψ⁡(Y)∩ ()=∅ψ(Y)∩ _I( V)= (Lemma D.8 is an if and only if statement). So by applying the Markov-property (Lemma 2.16): Pθ(VY~|V1,…,m0−1) = Pθ(VY~|V1,…,m0−1,V) . P_θ( V_ Y| V_1,…,m_0-1) = P_θ( V_ Y| V_1,…,m_0-1, V_ V) . On the other hand μ⁡(G obs,L∖ pinned)μ(G obs,L _ pinned) has arguments only in ψ⁡( extern∪ pinned)ψ(N_ extern _ pinned); by Def. 4.1 1 and 2 this is also true for ⊗k=m0mfk _k=m_0^mf_k in the definition of PθP_θ. Thus by successive application of the Markov-property (Lemma 2.16) Pθ(VY~|V1,…,m0−1,V) = Pθ(VY~|VX~∪ψ⁡( pinned),V) . P_θ( V_ Y| V_1,…,m_0-1, V_ V) = P_θ( V_ Y| V_ X∪ψ(N_ pinned), V_ V) . Putting the previous results together: μ = Pθ(VY~|VX~∪ψ⁡( pinned)=(x,d),V=) .μ = P_θ( V_ Y| V_ X∪ψ(N_ pinned)=(x,d), V_ V=D) . The right-hand-side, by Def. 2.21, is Pθ(VY~|VX~=x)P_θ^D( V_ Y| V_ X=x), thus, plugging in (∗)(*), we have proved the first claim of the lemma. (Note that, writing P⁡(Y|X=x)|x=XP(Y|X=x)|_x=X for the random measure obtained by plugging the random variable X into a regular version of P⁡(Y|X=x)P(Y|X=x), we have P⁡(Y|X=x)|x=X=P⁡(Y|X)P(Y|X=x)|_x=X=P(Y|X); cf. [22, Thm. 5.3 (p. 84)]). For the second part, we have Pθ(Vψ⁡(Y)|Vψ⁡( pinned),Vψ⁡( extern)=x)=νxP_θ^D( V_ψ(Y)| V_ψ(N_ pinned), V_ψ(N_ extern)=x)= _x almost surely PθP_θ^D, where the right-hand side is a probability kernel (not a random measure or kernel) by hypothesis. The left-hand side is (see above, cf. [22, Thm. 5.3 (p. 84)]) Pθ(Vψ⁡(Y)|Vψ⁡( pinned),Vψ⁡( extern)=x) P_θ^D( V_ψ(Y)| V_ψ(N_ pinned), V_ψ(N_ extern)=x) =Pθ(Vψ⁡(Y)|Vψ⁡( pinned)=d,Vψ⁡( extern)=x)|d=Vψ⁡( pinned)(ω) . =P_θ^D( V_ψ(Y)| V_ψ(N_ pinned)=d, V_ψ(N_ extern)=x)|_d= V_ψ(N_ pinned)(ω) . In particular Pθ(Vψ⁡(Y)|Vψ⁡( pinned)=t,Vψ⁡( extern)=x)=νxP_θ^D( V_ψ(Y)| V_ψ(N_ pinned)=t, V_ψ(N_ extern)=x)= _x for Pθ(Vψ⁡( pinned))P_θ^D( V_ψ(N_ pinned)) almost all t. Plugging this into Pθ(Vψ⁡(Y)|Vψ⁡( extern)=x) P_θ^D( V_ψ(Y)| V_ψ(N_ extern)=x) =∫Pθ(Vψ⁡(Y)|Vψ⁡( pinned)=t,Vψ⁡( extern)=x)Pθ(Vψ⁡( pinned)∈ ) = P_θ^D( V_ψ(Y)| V_ψ(N_ pinned)=t, V_ψ(N_ extern)=x)P_θ^D( V_ψ(N_ pinned)∈ ) =νx∫Pθ(Vψ⁡( pinned)∈ ) = _x P_θ^D( V_ψ(N_ pinned)∈ ) =νx , = _x , we obtain the claimed result; we denoted the integral of a measurable f⁡(t)f(t) in the variable t over the measure Pθ(Vψ⁡( pinned))P_θ^D( V_ψ(N_ pinned)) as ∫f⁡(t)Pθ(Vψ⁡( pinned)∈ ) f(t)P_θ^D( V_ψ(N_ pinned)∈ ). Lemma D.10. Given a regular functional computing μ⁡[δi]=F⁡[μ1,…,μn]μ[ _i]=F[ _1,…, _n] with μ1,…,μn _1,…, _n all δi _i-identifiable, then μ is δi _i-identifiable. Given representing ν1,…,νn _1,…, _n with νi=μi _i= _i almost surely PθP_θ^D, then ν=F⁡[ν1,…,νn]ν=F[ _1,…, _n] represents μ almost surely PθP_θ^D. proof By definition (C.11), there are ν1,…,νn _1,…, _n with νi=μi _i= _i almost surely PθP_θ^D after plugging in δi(ω)=δ⁡[Vi(ω)] _i(ω)=δ[ V_i(ω)]. Let ω∈Ωω∈ be arbitrary. With probability 11, the finitely many νi=μi(ω) _i= _i(ω) simultaneously. Thus μ⁡(ω)=F⁡[μ1(ω),…,μn(ω)]=F⁡[ν1,…,νn]μ(ω)=F[ _1(ω),…, _n(ω)]=F[ _1,…, _n] with probability 11. In particular ∃ν:=F⁡[ν1,…,νn]∃ν:=F[ _1,…, _n] with μ=νμ=ν almost surely PθP_θ^D after plugging in δi(ω)=δ⁡[Vi(ω)] _i(ω)=δ[ V_i(ω)] and μ is δi _i-identified by definition (C.11). plain Theorem 2 Given a structured query (ψ,Y)(ψ,Y), using L:=( proper∖Y)∪ pinnedL:=(N_ proper Y) _ pinned, if there is a regular functional computing μ⁡(G query(),L)=F⁡[μ1,…,μn]μ(G query(G),L)=F[ _1,…, _n] with μ1,…,μn _1,…, _n all δi _i-identifiable, then μ⁡(G query,L)μ(G query,L) and the underlying query q⁡(ψ,Y)q(ψ,Y) are identifiable. proof We have to identify the target Pθ(Y~|X~=x)P_θ^D( Y| X=x) of the underlying query. We apply Lemma D.9, using Lemma D.10 to apply the second part, to obtain Pθ(Vψ⁡(Y)|Vψ⁡( extern)=x) = νx ,P_θ^D( V_ψ(Y)| V_ψ(N_ extern)=x) = _x , where ν=F⁡[ν1,…,νn]ν=F[ _1,…, _n] with ν1,…,νn _1,…, _n identifiable representatives of μ1,…,μn _1,…, _n (see Lemma D.10). In particular ν is identifiable, and hence so is Pθ(Y~|X~=x)P_θ^D( Y| X=x) (using that by definition of the underlying query, X~=ψ⁡( extern) X=ψ(N_ extern), Y~=ψ⁡(Y) Y=ψ(Y)). D.4 Beyond Basic Queries Basic queries contain conventional do- or soft-interventions as special cases, but are substantially more flexible and by their simple form easily extensible. Meta Queries: Our approach is intentionally kept simple, describing only causal relations, no other preconceptions about data. By "meta-queries" we mean queries that stack other kinds of reasoning on top of our queries. This includes for example compiling the language of IID do-interventions to basic queries as described in example 5.1, or accounting for internal structure of singular interventions to reproduce "rule 2" conditioned interventions (§E.4) and thus the results of [56]. More generally, assumptions like stationarity of time-series can be leveraged: To ensure Ass. 2.11 (finite past) on a stationary k-Markov time-series (with not too complicated missingness structure), estimate the stationary distribution jointly over k time-steps from data and prefix the observed time-interval in the I-graph by a "known" (in ℱ intervene=θF_ intervene=θ) k-step block (without ancestors), then in identification-formulas of queries, plug in the estimated stationary distribution for these "known" kernels (where they appear). Similarly, if a query Y~x Y_x can also be formulated in form of a Markov-kernel ( ~ X is a "past" and Y~ Y a "present" in the sense that Y~y0⊗Y~y1⊗Y~y2⊗… Y_y_0 Y_y_1 Y_y_2 … is a valid expression), one may post-hoc want to know what stationary distributions of this process (with Markov-kernel Y~x Y_x) exist. is (from a formal perspective) readily possible to ask for example for the new stationary distribution(s) after an intervention; our formalism provides a kernel suitable to assess this fixed-point question. Consider also the following example (which also illustrates the relevance of the set X~ X in basic queries): Example D.11 (Query Argument). Consider a simple time-homogeneous ( is only one model-variable fXf_X), Markov (single time-lag) univariate time-series …→Xt−1→Xt→Xt+1→…→ X_t-1→ X_t→ X_t+1→… without latents ( ⁡(N) V(N) contains X1,…,XNX_1,…,X_N). For time-series, different meaningful notions of interventions compete. For example we could replace XtX_t by a do intervention X _ (without parents) for a single t. At least if Y~ Y contains only elements Xt′X_t with t′>t >t, we can obviously extract the kernel Xx pastX_x past – note that it does at this point not matter in our formalism if the time-series is stationary (up to support problems), the joint "P⁡(Xt−1,Xt)P(X_t-1,X_t)" nor the marginals "P⁡(Xt)P(X_t)" must be defined, cf. example 4.11, Rmk. A.6 – then compute any finite future sub-sequence (thus elements of Y~ Y jointly) as X ⊗Xx past⊗Xx past⊗…Xx past .X_ X_x past X_x past … X_x past . Now, what about soft interventions? Or even intervention-free queries (we have extracted Xx pastX_x past, but how do we ask for it as a query)? If f~x f_x replacing XtX_t has a single parent (its immediate past), then even asking for future values of Y~ Y makes no sense if the time-series is not stationary (indeed this is also not a finite past query, Ass. 2.11). The definition of query we employ allows in such cases to ask the (actually meaningful) question about Xt−1′⊗f~x⊗Xx past⊗Xx past⊗…Xx past withX~=Xt−1′ .X _t-1 f_x X_x past X_x past … X_x past with X=\X _t-1\ . We can reason about the effect of this soft intervention on future values Y~ Y, if we can reason about Xt−1′∈ℱ interveneX _t-1 _ intervene. For example if we believe our intervention happens in a stationary time-series, if our observations are also stationary simply intervene to the distribution over observations; the last point need not be true however: we could extract Xx pastX_x past also from a non-stationary time-series ( Xx pastX_x past and another kernel two-periodically). In this case, while we can reason about Xt−1′∈ℱ interveneX _t-1 _ intervene, we may not actually care about any specific choice of Xt−1′X _t-1, and in this case can use X~=Xt−1′ X=\X _t-1\ instead to query for a "partial" result Xx pastX_x past. Then Xt−1′X _t-1 should be an invariant state of this (partial) result Xx pastX_x past, in particular it can (at least in principle) be estimated from this first query. In a second query (the one from before), we can then use this estimated Xt−1′X _t-1 as an intervention. Nested Queries: We focused on queries (and graphical operations) with structured kernels as results. The authors are not aware of queries outside of this category having been considered in the literature, however, such queries exist, can be identifiable, and might make sense. It should, with the same graphical operations plus "disintegration nesting" (align nodes to disintegrations of other queries), be possible to approach problems like this: Example D.12. We are given system A which describes an ecosystem of micro-organisms in a Petri-dish (with multiple causally interrelated biological variables). System B is an experimental protocol that leads to samples (full ecosystems/Petri-dishes) being discarded or kept for future test-generations based on some of the measured properties of A. If we want to describe the long term result of repeatedly generating time-series from A, then selecting to initial states from B and again propagating by A many times, our query has disintegrated (from selection) sub-systems of A as sub-graphs (or "sub-structured" nodes). D.5 Examples for Multi-Level Queries A multi-level description allows for very fine-grained query specification. We start from a modification of example 4.18. C sedimentC_ sedimentC geomC_ geom XXYY XXYY XXYY XXYY ⋯·sX′X Y1Y_1C geomC_ geomX′X Y2Y_2C sedimentC_ sedimentC geomC_ geomX′X Y3Y_3 Figure 7: Different multi-level queries for the same system, see example D.13. Green (solid outline, rounded corner rectangles) are observations a single site (other observed sites are not shown). Example D.13. We observe many river sites (water-levels, throughput, …), each over many time-steps. Each river-site has a context C sedimentC_ sediment "sediment type". This context is extremely complex (there are a lot of aspects to the actual sediment that affect to form of a resulting river-bed geometry and other relevant parameters) and we thus consider C sedimentC_ sediment unobserved. However, C sedimentC_ sediment will not change over time (at least not within realistic time-frames of observation). There is a distribution P sedimentP_ sediment that describes how likely different types of sediment among the river-sites we observe are; we do not know P sedimentP_ sediment, but we fix notation here for later reference and interpretation. Further we assume there is a context C geomC_ geom describing the "geometry" of a riverbed. This context is extremely complex (there are a lot of aspects to the actual geometry that are relevant to water-flow, like the distribution over cross-sections encountered but also the relation between cross-sections along the river, roughness measures and many more) and we thus consider C geomC_ geom unobserved. However, C geomC_ geom will not change over time (at least not within realistic time-frames of observation). There is a distribution P geom| sedimentP_ geom| sediment, over geometries encountered given a sediment-type; this is a probability-kernel from sediment-types to geometries. Again, we do not know this kernel. For each river-site, there are time-dependent variables like water-throughput X and water-level Y. For the sake of argument, assume that water-throughput X is driven externally (throughput must match influx over time-scales where rising water-levels and "water-capacity" at the site are irrelevant; for simplicity let’s assume samples far enough apart in time, that they are effectively IID for each site) and that water-levels Y depends on X and C geomC_ geom only, in particular the kernel Yx,gY_x,g is assumed to be the same across all river-sites (site-dependent aspects are captured by C geomC_ geom in good enough approximation); X may vary by site (this will not matter for the purpose of this example). We ask a question about Y, when changing X (for example by diverting part of the water-influx to a channel or large-scale reservoir). Indeed we ask not one question, but rather three questions that are similar, but practically important to distinguish. [label=0)] 1. Enforcing influx X′X at a particular site s, how will Y change at that site s? 2. Enforcing influx X′X at a particular previously unobserved site s′s that is geographically very close to an observed site s and can plausibly be assumed to share geological sediment properties C sedimentC_ sediment with site s, how will Y change at that site s′s ? 3. Enforcing influx X′X at a particular previously unobserved site s′s , how will Y change at that site s′s ? The actual question is always the same, so we should be more precise as to what we mean by "how will Y change at that site s?". Indeed we will typically want to know, what is the "most narrow" (smallest noise) prediction we can make. There are both a finite-sample aspect and an aspect of the precise formulation of the question (and infinite-sample answer) to this question. Figure 7 illustrates these three different queries for Y1,Y2,Y3Y_1,Y_2,Y_3 (in this order); the mechanism at each YiY_i is Yx,gY_x,g as in the observations, the additional index is just for book-keeping. Note, that these queries are identifiable under different conditions: For Y1Y_1, we need many time-points in context s to learn the structural query where C geomC_ geom is pinned; for Y2Y_2, we either need to observe C geomC_ geom (as variable, with distribution C geom∘C sedimentC_ geom C_ sediment) across many sites or we need to observe many sites (not just s, s′s ) with "shared" C sedimentC_ sediment (while not shown in Fig. 7 this could be read off the I-graph, where a single C sedimentC_ sediment occurrence may have many C geomC_ geom-children) to learn Yx,[g]∘C geomY_x,[g] C_ geom with C sedimentC_ sediment pinned (the argument of C geomC_ geom); for Y3Y_3 we need to learn Yx,[g]∘C geom∘C sedimentY_x,[g] C_ geom C_ sediment, which requires many sites (note that our "freeness" condition means we use only one sample for Y for each site, which is likely not a good strategy in the finite sample case; however one has to ensure to suitably reweigh different sites if they have different numbers of time-points). In principle, staying more local is more precise: For example the variance of Y1Y_1 is essentially the variance of noise at Y, while the variance of Y2Y_2 (or Y3Y_3) additionally captures variance from P geom| sedimentP_ geom| sediment (or from both P sedimentP_ sediment and P geom| sedimentP_ geom| sediment); are relevant to the interpretation of resulting distributions. However, as pointed out above, conditions for identification are different. Further, in the finite-sample case, the actual variance of an estimator depends on both the variance of the true predicted distribution but also on finite-sample error – and the number of available samples favors the opposite order (Y3Y_3 can use, somewhat over-simplified, more data-points than Y2Y_2 than Y1Y_1), so these non-trivial trade-offs have to be considered in practice. Finally the mathematically inclined reader may have noticed an interesting detail: If we observe multiple Y (time-points) at the new site, then they are IID for query Y1Y_1, but only exchangeable (conditionally IID given C geomC_ geom for that particular site) for queries Y2Y_2, Y3Y_3. θ((Y2)t0,…,(Y2)tn)P_θ^D((Y_2)_t_0,…,(Y_2)_t_n) is not IID (but exchangeable), while Pθ((Y1)t0,…,(Y1)tn)P_θ^D((Y_1)_t_0,…,(Y_1)_t_n) is IID. This subtlety is correctly captured by our query and identification formalism: The query (for (Y2)t0,…,(Y2)tn(Y_2)_t_0,…,(Y_2)_t_n) in this case has a single C geomC_ geom node ic∈Ii_c∈ I (for the new site s′s ) which is a child of a C sedimentC_ sediment node shared with s (but s has a different C geomC_ geom node). This single node ici_c in turn has n+1n+1 children (Y2)t0,…,(Y2)tn(Y_2)_t_0,…,(Y_2)_t_n. Above we sketched two possible identification-strategies for Y2Y_2: (1) "observe" (somehow) C geomC_ geom to learn ∙→C geom → C_ geom (where ∙ is a pinned node for the shared C sedimentC_ sediment) and ∘→Y←∘ → Y← (with two outer external nodes for X and the now assumed observed C geomC_ geom). The query (see above) can immediately be glued from these: In practice this means, to jointly draw from Pθ((Y2)t0,…,(Y2)tn)P_θ^D((Y_2)_t_0,…,(Y_2)_t_n), first draw from ∙→C geom → C_ geom once, then plug the result into ∘→Y←∘ → Y← for all n+1n+1 observations. To draw again from Pθ((Y2)t0,…,(Y2)tn)P_θ^D((Y_2)_t_0,…,(Y_2)_t_n), draw a new value from ∙→C geom → C_ geom (once) and so on. (2) given many sites sharing C sedimentC_ sediment, learn ∙→[C geom]→Y←∘ →[C_ geom]→ Y← (where C geomC_ geom is hidden, indicated by square-brackets), for a single Y, plug in X′X . However, this second strategy does not apply for the joint Pθ((Y2)t0,…,(Y2)tn)P_θ^D((Y_2)_t_0,…,(Y_2)_t_n), because the single node for C geomC_ geom in the query would enforce an overlap which is not of the form allowed by Lemma 3.8 (gluing; formally we work on G obsG obs with the pinned node aligned to some δi _i, however the query has a c-component containing the hidden C geomC_ geom and its children (Y2)t0,…,(Y2)tn(Y_2)_t_0,…,(Y_2)_t_n, none of our learned structures contains this full c-component as subgraph). Indeed from ∙→[C geom]→Y←∘ →[C_ geom]→ Y← we only know how to draw Y once before redrawing C geomC_ geom. Small issues like the last one are often difficult to even spot. In our graphical operations and query perspective, they become surprisingly evident. As this example shows, there are subtly different, but meaningful, queries that are formally distinguished by our formalism. These queries differ in how the query-part of the I-graph is attached to the observational part of the I-graph, in particular this distinction cannot be made in the single-level case (where query and observational part of the I-graph are disconnected, cf. Lemma 5.5). Appendix E The IID Case We compare our results to well-established standard results for the IID case. This enables us to validate that answers obtained from our formalism are indeed sound. Further, while we do not show that our algorithms are complete in the general case, they turn out to be complete in IID settings. E.1 Standard Results for the Single-Context Case Some of the assumptions implicit in this notation become clearer in the next subsection §E.2 where they appear as explicit constraints on the form of the model (in the sense of Def. 2.6). We start from the standard formulation via structural causal models (SCM) [30; 38]: Notation E.1 (Variables). For some finite index set I vars×I sampleI_ vars× I_ sample (typically I sample=ℕI_ sample=N), I varsI_ vars is a finite set, fix a set of endogenous random variables Xv,s(v,s)∈I vars×I sample\ X_v,s\_(v,s)∈ I_ vars× I_ sample, taking values in Xv X_v, that is measurable mappings Xv,s:Ω→ Xv X_v,s: → X_v. We assume these are IID (in I sampleI_ sample), ∀s,s′∈I sample∀ s,s ∈ I_ sample: (∗)(*) (Xv,s)v∈I vars= d(Xv,s′)v∈I vars( X_v,s)_v∈ I_ vars d=( X_v,s )_v∈ I_ vars are (jointly) equal in distribution and for different s∈I samples∈ I_ sample are (jointly) independent, P⁡(Xv,s(v,s)∈I vars×I sample) P(\ X_v,s\_(v,s)∈ I_ vars× I_ sample) = ∏s∈I sampleP⁡(Xv,sv∈I vars) = _s∈ I_ sampleP(\ X_v,s\_v∈ I_ vars) =(∗) ∏s∈I sampleP⁡(Xvv∈I vars) , (*)= _s∈ I_ sampleP(\ X_v\_v∈ I_ vars) , by slight abuse of notation (we just drop the index s mandated by (∗)(*)); as is common in the literature we will (justified by IIDness) talk about variables Xvv∈I vars\ X_v\_v∈ I_ vars (omitting an index s∈I samples∈ I_ sample). We will sometimes use indices v∈I varsv∈ I_ vars and their associated random variable Xv X_v interchangeably, this should not lead to confusion (it is equivalent to saying the I varsI_ vars, which as a finite set is defined essentially by the number of its elements is the set containing as elements these random variables). Definition E.2 (SCM). An SCM M consists of the following information: Fix an index-set I vars×I sampleI_ vars× I_ sample and an observed subset ⊆I varsO I_ vars with complement L:=I vars∖L:=I_ vars . Using notation E.1, there are (IID) variables Xvv∈I vars\ X_v\_v∈ I_ vars and (IID) exogenous noises (hidden) ηvv∈I vars\ _v\_v∈ I_ vars, taking values in Nv N_v. Noises are jointly independent also within a sample, additionally (to IID) have P⁡(ηvv∈I vars)=∏v∈I varsP⁡(ηv)P(\ _v\_v∈ I_ vars)= _v∈ I_ varsP( _v). Further, for each v∈I varsv∈ I_ vars there is a set of parents ⊂I vars∖v _v⊂ I_ vars \v\ and a mechanism (a measurable mapping) fv: X × Nv→ Xvf_v: X_ _v× N_v→ X_v such that the endogenous variables satisfy the structural equations Xv:=fv(X ,ηi) X_v:=f_v( X_ _i, _i) relative to their parents and noise-term. Parent-sets are assumed to satisfy a suitable minimality condition, [7, Def. 2.6], that is fvf_v are not trivial in any argument. We assume SCMs are uniquely solvable (for example acyclic), meaning the distribution of endogenous variables is fixed uniquely by noise-terms and mechanisms. We assume (in the IID case, this is essentially w. l. o. g., cf. Rmk. E.3) each l∈Ll∈ L has no parents =∅ _l= and at most two children (usually we start from M where there are always exactly two children, but after a hard-intervention, see below, there might be less children), and its children are observed. Remark E.3 (Latent Projections). The form of latent-structure in assumed in the previous definition is standard in the literature, see [30, Def. 2.6.1 (p. 52)], because it simplifies graphical representation and logic, and a model can also be transformed into this form, see [30, Thm. 2.6.2 (p. 52)]. This is however primarily a consequence of the strong restrictions IIDness puts on symmetries of the model. In general, there does not seem to be any simple "standard form" in a comparable sense. Note that our approach makes latents always explicit, also in IID models, for example in Fig. 1 it is very clear that (Z2)x,[l2]∘L2(Z_2)_x,[l_2] L_2 (the second blue-box labeled "learn") does depend on L2L_2 and can only be transferred to a query, where the latent has the same distribution. For transfer between contexts, it is important to be transparent about such requirements. Definition E.4 (Interventions). Interventions (or "actions") are defined as follows: Given an SCM M, an intervened on X~⊂ X SCM M′M is an SCM with the same exogenous noises, and parent-sets ′ _v, mechanisms gvg_v such that for v∈I vars∖X~:v∈ I_ vars X: ′= _v= _v and gv=fvg_v=f_v. An intervention is called hard if for v∈X~:v∈ X: ′=∅ _v= . A hard intervention is a do-intervention, if for v∈X~:v∈ X: gv=xvg_v=x_v is a constant. We denote a do-intervention on X~ X by Pearl’s do-operator (X~=x) ( X=x), for example the interventional distribution of Y~⊂ Y conditioned on ("in a given context" in [56]) Z~⊂ Z P⁡(Y~| (X~=x),Z~) := P(Xv′v∈Y~|Xv′v∈Z~) ,P( Y| ( X=x), Z) := P(\ X _v\_v∈ Y|\ X _v\_v∈ Z) , where Xv′ X _v are the endogenous variables of the model M′M (since we assumed SCMs are uniquely solvable, this is well-defined). We also write M (X~=x)M ( X=x) or simply M (if clear from context) in this case. We will assume that the intervened value is in the value-space x∈ X~x∈ X_ X ( do not intervene variables to arbitrary real values). Definition E.5 (Causal Graph). The causal graph including latents G^L of a SCM M is defined as the directed graph with nodes = N^L=I_ vars and a directed edge x→yx→ y in E^L if x∈ ∈ _y. The corresponding graph of the do-interventional model M is denoted G_ do^L, and can also be obtained graphically by "amputating" all incoming edges to nodes in X~ X. Define G as the graph with nodes = N=O and a directed edge x→yx→ y in E if x∈ ∈ _y and a bi-directed edge x↔yx y in E if L∩ ∩ ≠∅L∩ _x∩ _y≠ . Causal graphs including latents G^L will typcially be DAGs (directed acyclic graphs) below, in this case the graph G^L is called Markovian and G is called semi-Markovian. For this section we adapt: Definition E.6 (Causal Effect Identifiability). Quoting [56, Def. 2 (p 3)] with only notation adapted and comments in square brackets added: The causal effect of an action (X~)=x ( X)=x on a set of variables Y~ Y in a given context Z~=z Z=z such that X~,Y~,Z~ X, Y, Z are disjoint [subsets of O] is said to be identifiable from P [P is the joint distribution P⁡(Xvv∈)P(\ X_v\_v )] in G, if P(Y~| (X~=x),Z~)P( Y| ( X=x), Z) is (uniquely) computable from P in any causal model which induces G. Remark E.7. Here computable (seems to) mean "uniquely determined", the causal graph G and any observed distribution P, then any two SCMs that produce the same P and have causal graph G will induce the same interventional distribution. Here "any SCM" puts us in the generic case, structure (like linearity) cannot be used (as there could still be a non-linear SCM with the same graph and observed distribution). Non-identifiablity (and thus completeness) is then proved by counter-example ( some property of G, show there are two SCMs with the same observational distribution and different interventional distribution). This last point is relevant: We are talking about SCMs compatible with G and producing the same, but any observed distribution, not the observed distribution. [56; 57] implicitly11 1 Unfortunately neither of both references seem to define this notion, however in [57], in the example given in the paragraph immediately above Thm. 4 (p. 1223) it becomes evident that this is what their definitions assume. use the following definition of "subgraphs" Definition E.8. A edge-subgraph ′ G G is a graph on the same nodes ′= N = N but with a subset of edges ′⊂ E ⊂ E. We call a edge-subgraph of a (node-)subgraph a sparse subgraph. Remark E.9. We use, throughout this paper, the convention that ancestors (X) (X) of X include X as an element X∈ (X)X∈ (X). The definitions below and in [56; 57] also follow this convention. However some care has to be taken, as in places (for example the definition of a root set [57, p. 1220, left column] and [56, p. 2, right column]) the given references deviate from this convention. Definition E.10 (C-Component). [57, Def. 3 (p. 1221)], see also [61]: Given a semi-Markovian graph G such that a subset of its bidirected arcs forms a spanning tree over all vertices in G. Then G is called a c-component. Remark: This is analogous (and the inspiration for) our definition of c-connectivity (on G^L), a "c-component" is a graph with a single c-component in our notation. Where this would lead to confusion, we clarify what is meant; usually the wording " G is a c-component" is used, which seems rather unambiguous in either nomenclature. Definition E.11 (C-Tree). [57, Def. 4 (p. 1221)]: Given a semi-Markovian graph G such that G is a c-component, all observable nodes have at most one child, and there is a node Y∈Y∈ N such that (Y)= _ G(Y)= N. Then G is a Y-rooted c-tree. Remark: Usually we are only interested in whether there is a edge-subgraph that is a c-tree. Definition E.12 (C-Forest). [57, Def. 5 (p. 1222)]: Given a semi-Markovian graph G and R⊂R⊂ N (called the root set) such that (R)= _ G(R)= N. Then G is a R-rooted c-forest, if G is a c-component, and all observable nodes have at most one child. Remark: Usually we are only interested in whether there is a edge-subgraph that is a c-forest. Definition E.13 (Hedge). [57, Def. 6 (p. 1223)]: Given a semi-Markovian graph G. Let X,Y⊂X,Y⊂ N with X∩Y=∅X∩ Y= . Let F,F′F,F be R-rooted c-forests [both are sparse subgraphs of G] such that (for their node-sets which are subsets of the node-set of G) F∩X≠∅F∩ X≠ , F′∩X=∅F ∩ X= , F′F is a sparse subgraph of F, and R⊂ (Y)R⊂ _ G_ do(Y). Then F and F′F form a hedge for (X,Y)(X,Y) in G. Lemma E.14 (Hedge Criterion). We use the formulation from [55, Prop. 1 (p. 7)], where the authors correct a small erratum in their statement of [57, Cor. 3 (p. 1225)]: An interventional distribution P(Y~| (X~=x))P( Y| ( X=x)), is identifiable (Def. E.6), if and only if there is no hedge for (X~,Y~)( X, Y) in G. Lemma E.15 (Completeness of ID-Algorithm). [57, Thm. 8 (p. 1226)]: If the id-algorithm (see [61; 57]) fails to identify P(Y~| (X~=x))P( Y| ( X=x)), then it outputs a hedge (F,F′)(F,F ) for (X~,Y~)( X, Y). As a corollary by using the hedge-criterion [57, Cor. 5 (p. 1226)] thus: The id-algorithm is complete. E.2 Translating the Standard IID-Setup All IID models can be described with the following strongly restricted form of symmetry and viewport. See also examples 2.7, 2.17 and 5.1. Definition E.16 (IID-Observations with Uniform Missingness). We call a model IID (see Rmk. E.17), if it can be written in the following form: I=I dataset×I vars×I sampleI=I_ dataset× I_ vars× I_ sample, where |I sample|=∞|I_ sample|=∞, while |I vars|<∞|I_ vars|<∞ and |I dataset|<∞|I_ dataset|<∞ with symmetry-group G=I dataset×I sampleG= S_I_ dataset× I_ sample (the permutations group acting trivially on the middle factor I varsI_ vars of I and mixing datasets and samples), and variables J(C,v)=C×v×I sampleJ_(C,v)=C×\v\× I_ sample with symmetry Hv=C×I sampleH_v= S_C× I_ sample for some subset C⊂I datasetC⊂ I_ dataset (but disjoint, c is in exaclty one C of some J(C,v)J_(C,v), denoted C⁡(c)C(c)) and C⊂I dataset S_C⊂ S_I_ dataset embedded as subgroup by extending as the identity on elements outside of C. We assume I dataset=I dataset obs⊔I dataset queryI_ dataset=I_ dataset obs I_ dataset query is a disjoint union of observed data-sets (see viewport below) and pseudo-datasets I dataset queryI_ dataset query to encode queries (in the single context case, both are one-element sets: I dataset obs=αI_ dataset obs=\α\ and I dataset query=βI_ dataset query=\β\). By slight abuse of notation we will pretend that I sampleI_ sample depends on the "context" c∈I datasetc∈ I_ dataset (as only the viewport, see below, really matters this is purely notational). There is a subset ⊂I varsO⊂ I_ vars with complement L=I vars∖L=I_ vars and the viewport V⁡(N) V(N) is such that ∀(c,v,s)∈I dataset obs×I sample∀(c,v,s)∈ I_ dataset obs×O× I_ sample ∃N0∈∃ N_0 such that ∀N≥N0∀ N≥ N_0: (c,v,s)∈V⁡(N)(c,v,s)∈ V(N), while for ∀(c,v,s)∈(I dataset obs×L×I sample)∪(I dataset query×I vars×I sample)∀(c,v,s)∈(I_ dataset obs× L× I_ sample)∪(I_ dataset query× I_ vars× I_ sample) ( the complement in I) ∀N∈∀ N : (c,v,s)∉V⁡(N)(c,v,s)∉ V(N). In words: All variables in O are asymptotically observed for infinitely many samples in all observed contexts I dataset obsI_ dataset obs. A typical viewport of this form, for I sample=ℕI_ sample=N, is given by =|I dataset obs×|ℕN=|I_ dataset obs×O|N (we observe samples of size M:=|I dataset obs×|M:=|I_ dataset obs×O|, thus the viewport V⁡(N) V(N) is defined for N that are multiples of this number), and for N=nM∈N=nM observed indices are V⁡(N)=I dataset obs×1,…,n V(N)=I_ dataset obs×O×\1,…,n\. Note that by equivariance of parent-sets, the I-graph decomposes into disjoint (no mutual edges) subgraphs on nodes c×I vars×s\c\× I_ vars×\s\, and further by I sample⊂HJ S_I_ sample⊂ H_J (and acting as permutations on I sampleI_ sample), the equivariant parent sets cannot change between samples s (only between contexts c; here lower indices c on graphs will denote contexts, elsewhere in this paper upper indices c on graphs denote structural c-components, this should not lead to confusion), thus there are local graphs cG_c (with only proper nodes) and families of embeddings (anchored at any v0∈I varsv_0∈ I_ vars) ψj=(c,v0,s)cs∈I sample\ψ^c_j=(c,v_0,s)\_s∈ I_ sample embedding cG_c with image c×I vars×s\c\× I_ vars×\s\ into the I-graph; we can identify cG_c with Gc=G obs(c)G_c=G obs(G_c) (there are no external or pinned nodes; see also Def. E.20). Remark E.17 (Exchangability). Exchangability refers to the property of a distribution being invariant under permutations. We explicitly keep track of multiple datasets via I=I dataset×I vars×I sampleI=I_ dataset× I_ vars× I_ sample and given the data-set index we are essentially in a simple case of de Finetti’s theorem (see [23]): If the samples are exchangeable, than they are conditionally IID. In our case: conditional on the data-set index. Thus even though we only assume a symmetry, by explicitly tracking data-set association the models defined above are effectively IID (not just exchangeable) and the naming-convention makes sense. Model mechanisms in the sense of Def. 2.5 under these symmetry-restrictions are then push-forwards of noise-distributions. Lemma E.18 (Translating Mechanisms). Given a measurable map f: X× N→ Yf: X× N→ Y and a probability measure PηP_η on N N, define a probability-kernel μ⁡(f,Pη)μ(f,P_η) from X X to Y Y given by x↦f(x,⋅)∗Pηx f(x,·)_*P_η (see Rmk. G.5) or equivalently for x∈ Xx∈ X and B∈ℬ⁡( Y)B ( Y): μ⁡(f,Pη)(x,B) := ∫1B(f⁡(x,u))Pη( ) .μ(f,P_η)(x,B) := 1_B (f(x,u) )P_η( ) . If a random variable Y Y satisfies P⁡(Y|X=x)=f⁡(x,η)P( Y| X=x)=f(x,η) for a random variable η Xη X with law P⁡(η)=PηP(η)=P_η, then P⁡(X,Y)=Pη⊗μ⁡(f,Pη)P( X, Y)=P_η μ(f,P_η). In particular for an SCM M, there are probability-kernels μv:=μ⁡(fv,P⁡(ηv)) _v:=μ(f_v,P( _v)) for v∈I varsv∈ I_ vars such that aligning G=G= G^L (with only inner nodes) to these μv _v has a structured kernel μ⁡(G,L,π)=P⁡(Xvv∈)μ(G,L,π)=P(\ X_v\_v ). Remark: Indeed μ(fy,P⁡(ηy)) =P⁡(Y| = )μ(f_y,P( _y))_ _y=P( Y| _y= _y), however it seems conceptually clearer to define it directly relative mechanisms and noise-distributions, rather than through intermediate realizations as random variables. proof See examples in §G. Commonly, [30; 38; 61; 57; 56; 4], the following assumption is implicit in the assumption of "generic" mechanisms: Assumption E.19 (Unique Mechanisms). There are no two different variables with the same mechanism: Given an SCM M, then ∀v≠v′∈I vars:μ⁡(fv,P⁡(ηv))≠μ⁡(fv′,P⁡(ηv′))∀ v≠ v ∈ I_ vars:μ(f_v,P( _v))≠μ(f_v ,P( _v )). We are now able to "translate" standard SCMs to (restricted symmetry) models in the sense of Def. 2.6. Lemma E.20 (Translated Model). Given an SCM M with variables I varsI_ vars and observed variables ⊂I varsO⊂ I_ vars, and an intervention P(Y~| (X~=x))P( Y| ( X=x)), there is an IID model ℳ⁡(M)M(M) (Def. 2.6) given by I dataset obs=αI_ dataset obs=\α\, I dataset query=βI_ dataset query=\β\, variables as required for being an IID model (Def. E.16), (C,v)=C×v×I sampleJ_(C,v)=C×\v\× I_ sample with symmetry Hv=C×I sampleH_v= S_C× I_ sample. Here C is C=α,βC=\α,β\ if v∉X~v∉ X, if v∈X~v∈ X then there are two variables J(α,v)J_(\α\,v) and J(β,v)J_(\β\,v) associated to v. If α∈Cα∈ C, then the mechanisms are fJ(C,v)=μ⁡(fv,P⁡(ηv))f_J_(C,v)=μ(f_v,P( _v)) (Def. E.18) and (equivariant) parent-mappings at j=(v,s)∈Jvj=(v,s)∈ J_v take values kJv(j)=(pk,s) k_J_v(j)=(p_k,s), where pkp_k is the k-th (in order of arguments of fvf_v) parent of v, a set (j=(v,s))= ×s _J_v(j=(v,s))= _v×\s\. Value- and parent-value-spaces are the value-spaces Xv X_v of the SCM. If C=βC=\β\ (and thus v∈X~v∈ X), then mechanisms are known fv ∈ℱ intervenef _v _ intervene. Indeed for a do-intervention, they are singular at value xvx_v (the v-component of the tuple x to which X~ X was intervened) without parents. We will default to the value-space Xv X_v (assuming xv∈ Xvx_v∈ X_v as required in Def. E.4). The viewport is as required by Def. E.16, typically V⁡(N)=I dataset obs×1,…,n V(N)=I_ dataset obs×O×\1,…,n\. This model ℳ⁡(M)M(M) has the following properties. The I-graph of ℳ⁡(M)M(M) is the disjoint union of its restrictions to α×I vars×s\α\× I_ vars×\s\ which are of the form G^L (the causal graph including latents of M) and its restriction to β×I vars×∗\β\× I_ vars×\*\ which is of the form G_ do^L, families of embeddings of the last paragraph of Def. E.16 embed Gα=α=G_α=G_α= G^L and Gβ=β= G_β=G_β= G_ do^L as local graphs with only proper nodes and aligned to μ⁡(fv,P⁡(ηv))μ(f_v,P( _v)). As an immediate consequence ∀s∈I sample∀ s∈ I_ sample: P=P⁡(Xvv∈) P=P(\ X_v\_v ) = Pθ(Vii∈α×s) = P_θ(\ V_i\_i∈\α\×O×\s\) P(Y~| (X~=x)) P( Y| ( X=x)) = Pθ(Vii∈β×Y~×∗) , = P_θ(\ V_i\_i∈\β\× Y×\*\) , where ∗* is the single (relevant) "sample" of the query. proof The claims about the I-graph are true by construction. By Lemma E.18 thus for each s∈I samples∈ I_ sample: μ⁡(α,L,π)=P⁡(Xvv∈I vars)=Pμ(G_α,L,π)=P(\ X_v\_v∈ I_ vars)=P. On the other hand, by Pθ(Vii∈α×s)=μ⁡(α,L,π)P_θ(\ V_i\_i∈\α\×O×\s\)=μ(G_α,L,π) by Def. 2.13. Similarly μ⁡(β,L,π)=P⁡(Xv v∈I vars)=P⁡(Y~| (X~=x))μ(G_β,L,π)=P(\ X _v\_v∈ I_ vars)=P( Y| ( X=x)) and Pθ(Vii∈β×Y~×∗)=μ⁡(β,L,π)P_θ(\ V_i\_i∈\β\× Y×\*\)=μ(G_β,L,π). The commonly considered do-interventions are a special case of structured queries. We already built them into the translated model as a context β, and the last result, P⁡(Y~| (X~=x))=Pθ(Vii∈β×Y~×∗)P( Y| ( X=x))=P_θ(\ V_i\_i∈\β\× Y×\*\) suggests a rather obvious choice of query (Def. 5.3): Definition E.21 (Translated Query). In ℳ⁡(M)M(M), construct a query as follows: q=(Yq,Xq,θ)q=(Y_q,X_q,θ) with Xq=∅X_q= and Yq=β×Y~×∗Y_q=\β\× Y×\*\ and θ=x=(xk)k∈X~θ=x=(x_k)_k∈ X fully parameterizes ℱ intervene=δ⁡[xk]k∈X~F_ intervene=\δ[x_k]\_k∈ X. The structured query associated by Lemma 5.5 is | (~) G_ do^L|_ _ ^L( Y) (cf. proof of Lemma 5.5 or its explanation in the main text), using the identification Gβ= G_β= G_ do^L and restricting this graph to (Y~):= (Y~) _ ^L( Y):= _ G_ do^L( Y). Note that I aux=∅I aux= (the view-port and the query are in disconnected graph-components of the I-graph); thus (by Markov-property, Lemma 2.16), the underlying query is Pθ(Vii∈β×Y~×∗)=Pθ(Vii∈β×Y~×∗)=P⁡(Y~| (X~=x)) .P_θ^D(\ V_i\_i∈\β\× Y×\*\)=P_θ(\ V_i\_i∈\β\× Y×\*\)=P( Y| ( X=x)) . In the IID-case, no difficulties arise from non-uniqueness (of minimal latent-sets, ancestral structures or absorbed parents ), which allows for a very simple classification of families of embeddings: Lemma E.22 (IID Classification of Embeddings). For an IID-model with =(,) G^L=( N^L, E^L), decorated families of embeddings with maximal J0J_0 are of the form: J0=C×I sampleJ_0=C× I_ sample, where ∅≠C⊂I dataset obs ≠ C⊂ I_ dataset obs are the relevant contexts (for the present sub-section C=αC=\α\ is the only valid choice), ⊂= N⊂ N^L=I_ vars, pinned=∅N_ pinned= and ψj=(c,s)(n)=(c,n,s)∈I dataset×I vars×I sample=I _j=(c,s)(n)=(c,n,s)∈ I_ dataset× I_ vars× I_ sample=I. There is always a unique minimal latent subset given by L∩LN^L∩ L. Edges are fixed by the subset of E^L (by Def. 4.1). Remark: Some care should be taken, because ancestral structure is part of decorated families of embeddings (but hidden in this notation), in a multi-context setup, there might be different choices for the relevant contexts C⊂I datasetC⊂ I_ dataset differing only by validity of respective ancestral structures. proof As has been noted in Def. E.16, there are families of embeddings of cG_c to c×I vars×s\c\× I_ vars×\s\ indexed by s∈J0=I samples∈ J_0=I_ sample. We can also embed subgraphs c′G _c of these cG_c. The index set J0′J _0 can be potentially enlarged to J0C=C×v0×I sampleJ_0^C=C×\v_0\× I_ sample, where C is the largest subset of I dataset obsI_ dataset obs such that model-alignment is still valid. However by Ass. E.19 (unique mechanisms) each mechanism μv _v appears exactly once for each (c,s)∈I dataset obs×I sample(c,s)∈ I_ dataset obs× I_ sample so these are indeed the largest possible J0J_0. By the form of the viewport, hidden-ness of i=(c,v,s)i=(c,v,s) depends only on v∈Lv∈ L and all contexts are observed (asymptotically) infinitely often, so the claims minimal latent sets are immediate. Notation E.23. The classification-lemma E.22 justifies denoting IID embeddings ψ as a triple (C, proper,)(C,N_ proper,N) with C⊂I datasetC⊂ I_ dataset, proper⊂I varsN_ proper ⊂ I_ vars. In the single context case, there is only one observed context, so C=αC=\α\, thus until §E.6 we will simply write ψ=( proper,)ψ=(N_ proper,N) for statements about extraction (the set of minimal latents is implicitly given by L∩L ). Indeed, as long as contexts individually have infinitely many samples (asymptotically), the size of C is actually irrelevant (as long as it is non-empty). Lemma E.24 (Direct Identification). In the IID case, the claim of Thm. 1 (extraction from backdoor-free families of embeddings) holds true with the notion of "direct identifiability" from Def. 4.14 replaced by "knowledge of the observational distribution P=Pθ(Vii∈α×s)P=P_θ(\ V_i\_i∈\α\×O×\s\) and its conditional distributions". Remark: This results will become much clearer in the light of the analysis of the ED-phase in §E.3. proof The proof of Thm. 1 only uses data-sets induced via Lemma C.21, which are of the form of marginalizations of the observational distribution. Direct identification than claims that if for each sample P⁡(Xs,Ys)=μs⊗νxP(X_s,Y_s)= _s _x for s-independent ν (the disintegration νx=P⁡(Y|X=x) _x=P(Y|X=x) always independent of s by IIDness), ν is "directly identifiable", and νx=P⁡(Y|X=x) _x=P(Y|X=x) is, indeed, a conditional distribution of P. Corollary E.25 (Identification). Regular functionals are given uniquely by their argument, thus identifiability in the sense of Def. 4.16 implies in the IID case identifiability in the sense of Def. E.6. If we can show that our algorithm identifies a query in the sense of Def. 4.16, then this implies our algorithm identifies this query in the sense of Def. E.6. Finally, identifying the structured query | (~) G_ do^L|_ _ ^L( Y) identifies P(Y~| (X~=x))P( Y| ( X=x)) (cf. E.21) in the sense of Def. E.6. proof Regular functionals are composed of finitely many operations, which individual produce a well-defined (unique) result (§G). Thus regular functionals compute a unique result. The other claims follow directly from the definitions E.3 Single-Context with Empty Conditioning Set We first analyze what results are produced by the extract–decompose (ED)-phase(s) in Algo. EDAIdentify, steps 1 and 2. Then we separately analyze the assembly (A)-phase in Algo. EDAIdentify, step 3. ED-Phase: We discus the "ED-phase" combined, as it will turn out that in the IID-case the extraction (E)-phase can be replaced by the claim "μ⁡(,)=μ( G^L,L)=P (the observational distribution) is identifiable" (which also makes it clearer why Lemma E.24 was essentially trivial to prove). To illustrate this, we start with a (not finite-context IID) example where the E-phase is clearly necessary. Example E.26 (Non-Trivial Extraction). Consider a setup (inspired by JPCMCI [15]) given as follows: There are time-series for time-points T at river-sites S for multiple variables. Some of the mechanisms vary between sites s∈Ss∈ S, others over time t∈Tt∈ T or are shared. We assume there are many sites and time-points, relevant asymptotic limit has both |S|→∞|S|→∞ and |T|→∞|T|→∞ as N→∞N→∞. Lets consider a very simple such model of the form X→Zs→YtX→ Z_s→ Y_t, where X is the same mechanism for all s∈Ss∈ S and t∈Tt∈ T, ZsZ_s is the same for all t∈Tt∈ T, but depends on s∈Ss∈ S, while YtY_t is the same for all s∈Ss∈ S, but depends on t∈Tt∈ T. Most frameworks, like JCI [27], mz-transport [4] or JPCMCI [15] would inject "context-variables" for S and T. We want to predict an intervention on site s and at time t on for (X=x) (X=x). Conventional wisdom tells us this effect is identifiable and given simply by P⁡(Y|X=x,S=s,T=t)P(Y|X=x,S=s,T=t). This is correct in IID-terms. It is also, unfortunately, not helpful: We have exactly one data-point at S=sS=s and T=tT=t. So what could we do? The E-phase of our approach would actually search for smallest (best symmetry) backdoor-free graphs. It would find the following: ∘→Zs → Z_s is identifiable on the (asymptotically infinite) data-set T×s×(x,z)T×\s\×\(x,z)\. ∘→Yt → Y_t is identifiable on the (asymptotically infinite) data-set t×S×(z,y)\t\× S×\(z,y)\. Then the A-phase simply glues these partial results. In this example the E-phase does essentially two things: (1) It validates that only local graphs with sufficient available data are taken into account. Thus it avoids, even if Z is unobserved, dubious identification claims and rather fails gracefully. (2) It enables the A-phase to find the (actually useful) "frontdoor-like" identification strategy, even though it would conventionally be entirely unclear what makes this strategy better than any other plausible one. The point here is, that smaller local graphs have better symmetry, thus larger available data-sets. This is not just a finite sample issue, as the example above illustrates, taking the wrong local graphs, we can end up with finite data-sets (or even literally a single data-point) to learn from, which also breaks asymptotic identification results; this problem only gets more severe once latents are added. As already remarked 4.3, the symmetry of a local graph containing two subgraphs A and B of symmetry HAH_A and HBH_B has symmetry HA∩HBH_A∩ H_B. In some cases, we can simply intersect all symmetries of all variables J to get ∩J∈HJ _J H_J. We then get a "fundamental-domain" local graph containing "everything" about the model. In the IID case, all mechanisms are at least I sample S_I_ sample-invariant, and essentially the fundamental domain would be G (or in the finite-context -transport case: a disjoint union of causal and interventional graphs, one for each context and experiment, cf. Fig. 8). Another example: A time-series with m- and k-periodic mechanisms has fundamental domain τ-periodic, where τ is the smallest common multiple of m and k (translation symmetries here are mℤmZ and kℤkZ with mℤ∩kℤ=τℤmZ∩ kZ= ). As the example above shows, generally the fundamental domain may not be identifiable (here: (S×0)∩(e×ℤT)=(e,0)( S_S×\0\)∩(\e\×Z_T)=\(e,0)\ is the trivial sub-group). This is the deeper reason why the E-phase is necessary in general, but could be skipped for the IID-case. The result of the combined ED-phase is the same as applying only th D-phase (Algo. CGDecomp) to μ⁡(,)μ( G^L,L); see also Fig. 3 for an illustration. Lemma E.27 (ED-Phase). The ED-phase (the first two steps of Algo. EDAIdentify) produces after a finite number of steps the finite set of all c-connected structured kernels (and associated identification strategies) which can be obtained from μ⁡(,)μ( G^L,L) by repeated decomposition into c-components (Lemma 3.7) and simplification (Lemma A.12); this includes intermediate c-connected results (cf. Algo. CGDecomp). Notation: We call c-connected sub-graphs obtained by repeated decomposition into c-components (Lemma 3.7) and simplification (Lemma A.12), c-fragments. The claim above then reads: The ED-phase outputs all c-fragments of μ⁡(,)μ( G^L,L). proof The E-phase produces all minimal (c-connected) backdoor-free families, which are, by classification of embeddings (Lemma E.22), are all c-subgraphs of G^L. E-phase produces the decomposition of G^L into c-components (Lemma 3.7) and simplifications (Lemma A.12). The D-phase then further iterates these steps. Since all non-trivial decomposition-steps (c-components or simplifications) reduce the size of the graph (the produced pieces are smaller than the original one) and Algo. CGDecomp yields on trivial ones, the sizes of graphs in step m is at most |I vars|−m|I_ vars|-m nodes (because G^L has |I vars||I_ vars| nodes), and the algorithm terminates after a finite number of iterations. There is a finite number of forks (sub-iterations, -components, simplifications) in each iteration (there are only finitely many c-components and subsets of L in finite graphs). A-Phase: We finally discuss the assembly (A)-phase in Algo. EDAIdentify, step 3. We start by relating c-fragments (cf. E.27) to c-forests (Def. E.12, cf. [57]). Lemma E.28 (C-Fragments and C-Forests). Some properties of c-structures are: [label=()] 1. Given a c-fragment Gc≤(G,L)G^c≤(G,L), then there is a c-forest in G on innerc∖LN_ inner^c L rooted on R := y∈ innerc∖L | (y)∩ innerc=∅ .R := \ y _ inner^c L | _G(y) _ inner^c= \ . 2. A c-forest F in G_ do is a c-forest in G. 3. Given a R-rooted c-forest F′F in G such that there is no c-fragment Gc≤(,)G^c≤( G^L,L), with innerc∖L=F′N_ inner^c L=F , then there is a R-rooted c-forest F⊋F′F F in G. proof Part 1: We first show: C-fragments contain an edge-subgraph that is a forest. By definition of R, every node y∈ innerc∖(L∪R)y _ inner^c (L∪ R) has a child c∈ innercc _ inner^c. Using the special form of hidden structure assumed in the IID-case (cf. Def. E.2, Rmk. E.3), hidden nodes have no parents, thus c∈ innerc∖Lc _ inner^c L. Pick one of these children and call it c⁡(y)c(y). For y∈Ry∈ R. Then define a forest F rooted on R with nodes innercN_ inner^c and an edges y→c⁡(y)y→ c(y) for each y∈ innerc∖(L∪R)y _ inner^c (L∪ R). C-fragments are c-connected (a c-component in the sense of [61; 57]) by definition, thus this is a c-forest. Part 2: A c-forest in G_ do is an edge-subgraph F⊂ F⊂ G_ do with a single c-component. Also ⊂ G_ do⊂ G is an edge-subgraph, so F⊂F⊂ G is an edge-subgraph. Since the nodes of F are in a single c-component of G_ do, there is a path of bi-directed arrows between any two nodes, but again by ⊂ G_ do⊂ G being an edge-subgraph, this path of bi-directed arrows is also in G. Part 3: By definition the c-forest F′F is a single c-component, its nodes are in the same c-component F′⊂C⊂F ⊂ C⊂ N of G. Call the associated structural c-component (with innerc∖L=CN_ inner^c L=C) Gc≤G^c≤ G^L. This GcG^c need not be rooted on R. So we define a simplification G′G of GcG^c as follows: Define R0:=R_0:=R, then Rk+1:=Rk∪( innerc∩ (Rk)) .R_k+1:=R_k∪ (N_ inner^c∩ _ G(R_k) ) . After a finite number of steps Rn+1=RnR_n+1=R_n. Finally define B:= innerc∖(Rn∪L)B:=N_ inner^c (R_n∪ L) and add latents that are not a parent (or ancestor by form of hidden structure, cf. Def. E.2, Rmk. E.3) of any node in innercN_ inner^c not in B to get BL:=B∪(L∖ ( innerc∖B))B^L:=B∪(L _ G^L(N_ inner^c B)). By construction B= (B)B= _G^c(B) and there is a B-simplification G′G of GcG^c. As a simplification of the structural c-component GcG^c, G′G is a c-fragment of G^L. Thus by hypothesis inner′∖L≠F′N_ inner L≠ F . Note that initially F⊂ innercF _ inner^c, and the simplification cannot remove nodes in F (because they are ancestors of the observable R), thus inner′∖L⊊F′N_ inner L F . By 1, there is a c-forest F~ F on the nodes inner′∖LN_ inner L. Note that F~ F is rooted on R, by construction of RnR_n. Define F on the nodes inner′∖LN_ inner L containing F′F as a subgraph as follows: For nodes y∈F′⊂F~y∈ F ⊂ F insert all edges (there is actually at most one) out of y in F′F , for nodes y∈F~∖Fy∈ F F, insert all edges (there is actually exactly one) out of y in F~ F. Lemma E.29 (Identification of Do-Interventions). Applying algorithm EDAIdentify to the structured query (Def. E.21) of the intervention P(Y| (X)=x)P( Y| ( X)=x) returns after a finite number of steps. It outputs a non-empty set (thus at least one sound identification strategy) if and only if there is a no hedge (F,F′)(F,F ) for P(Y~| (X~=x))P( Y| ( X=x)). proof We equivalently proof: The output of Algo. EDAIdentify is empty ⇔ there is a hedge. Recall that G query= | ()G query= G_ do^L|_ _ ^L(Y) (Def. E.21). Further, we note that by construction structural c-components Gc≤ | ()G^c≤ G_ do^L|_ _ ^L(Y) of | () G_ do^L|_ _ ^L(Y) are either such that innerc∩X~=∅N_ inner^c∩ X= or innerc=xN_ inner^c=\x\ for a single node x∈X~x∈ X. This is because hard-interventions do not have parents, thus also no hidden parents. "⇒ ": Given, the output of Algo. EDAIdentify is empty. Step 1: ∃Gc≤ | ()∃ G^c≤ G_ do^L|_ _ ^L(Y), a structural c-component of | () G_ do^L|_ _ ^L(Y), where GcG^c is not one of the single-node c-components with inner node in X~ X, such that Gc≤G^c≤ G^L is not a c-fragment of G^L. Proof of Step 1: By contradiction. Let Gc≤ | ()G^c≤ G_ do^L|_ _ ^L(Y) be an arbitrary structural c-component of | () G_ do^L|_ _ ^L(Y). If GcG^c is one of the single-node c-components with inner node in X~ X, then GcG^c is identified by definition of ℱ interveneF_ intervene and added to the output of the ED-phase (in step 3 of Algo. EDAIdentify). Otherwise, Gc≤G^c≤ G^L is a c-fragment of G^L (this is the negation of the claim considered in the proof by contradiction). Thus by Lemma E.27, GcG^c is identifiable and in the output of the ED-phase. By repeated application of gluing (Lemma 3.8), thus | () G_ do^L|_ (Y) is identifiable. The A-phase (Algo. SVSearch) searches all such gluings on the output of the ED-phase, thus it finds (and returns) this identification-strategy. This is a contradiction to the output of Algo. EDAIdentify being empty. Step 2: There is a hedge for P(Y| (X)=x)P( Y| ( X)=x) in G^L. Proof of Step 2: Let Gc≤ | ()G^c≤ G_ do^L|_ _ ^L(Y) be the structural c-component from step 1. By applying Lemma E.28 1 (with G= G= G_ do^L and using that c-components are trivially c-fragments), then E.28 2 there is a c-forest F′F in G on nodes innerc∖LN_ inner^c L rooted on (some) R⊂ (Y)R⊂ _ G(Y) (by Gc≤ | ()G^c≤ G_ do^L|_ (Y) we have innerc⊂ (Y)⊂ (Y)N_ inner^c⊂ _ ^L(Y)⊂ _ G^L(Y), with (Y)= (Y)∖L _ G(Y)= _ G^L(Y) L, thus innerc∖L⊂ (Y)N_ inner^c L⊂ _ G(Y)). Since GcG^c is not one of the single-node c-components with inner node in X~ X, this c-forest satisfies innerc∩X~=∅N_ inner^c∩ X= . Also by step 1, Gc≤G^c≤ G^L is not a c-fragment. Thus, by Lemma E.28 3, there is a c-forest F⊋F′F F in G^L rooted on R. Let x∈F∖F′x∈ F F be arbitrary. By F being a c-forest there is y∈F′y∈ F , x↔yx y in G ( ∃l∈∃ l∈ G^L: x←l→yx← l→ y in G^L). Since x∉ innerc=F′x _ inner^c=F , one of the two edges x←l→yx← l→ y is not in G_ do^L (otherwise x would be in the c-component innercN_ inner^c). The only edges in G^L not in G_ do^L, by definition, are edges into X~ X. Since y∈F′y∈ F and F′∩X~=∅F ∩ X= , thus x∈X~x∈ X. In particular F∩X~≠∅F∩ X≠ . Thus (F,F′)(F,F ) is a hedge for P(Y| (X)=x)P( Y| ( X)=x) in G^L. "⇐ ": This direction follows indirectly by soundness and the hedge-criterion [57] (Lemma E.14). Alternatively, note that the proof of the other direction can (essentially) be traced backward, very briefly: Since F′F is c-connected and F′∩X~=∅F ∩ X= , there is a c-component GcG^c in G_ do containing F′F in its inner nodes. By F∩X~≠∅F∩ X≠ and F being rooted on R there is a chain of children in F from x∈X~∩Fx∈ X∩ F to F′F ; or from the perspective of its endpoint y in F′F : there is a chain of (observable) parents in F to x. Parents of observables cannot be simplified (Lemma A.12) away, so they must remain in any c-fragment of G. Finite number of steps: The query-graph is finite, and gluing grows the intermediate result by at least one node. Further there are finitely many elements in K to choose from (for gluing), thus Algo. SVSearch terminates after a finite number of steps. For the other algorithms, see Lemma E.27. Corollary E.30 (IID Completeness, single context, empty conditioning set). By the hedge-criterion (Lemma E.14; cf. [57; 55]), algorithm EDAIdentify is complete in the (single context) IID case for do-interventions with empty conditioning set. Remark: This result is analogous to the completeness of the id-algorithm (Lemma. E.15). E.4 Single-Context with Non-Empty Conditioning Set In the conditional case, the reader may notice that seemingly our criterion and the one given by [56] disagree. Consider the following example: Example E.31. Given observations and a simple intervention X ∈ℱ intervene X_ _ intervene. Observations:X XY YM M Query:X X_ Y YM M We want to know P⁡(Y| (X=x),M=m)P( Y| ( X=x), M=m). • According to [56] this query is identifiable. • According to our logic it is not. As it turns out, both assessments are correct. This happens, because we do not assume any (known ) internal structure on mechanisms (observed or intervened) while [56] implicitly does through rule 2 of the do-calculus. We first illustrate the actual problem: Rule 2 of the do-calculus allows the replacement P⁡(Y| (X=x),M=m)=P⁡(Y| (X=x,M=m)) .P( Y| ( X=x), M=m)=P( Y| ( X=x, M=m)) . This replacement is correct, if P⁡(X )P( X_ ) is singular (only takes a single value), which rule 2 assumes but we do not (it is an internal structure of the intervened mechanism X X_ ). If the intervention were for example chosen as X ∼⁡(0,1) X_ (0,1), then conditioning on M generally will introduce selection-bias on X X_ . For estimation of Pθ(Y|M=m)P_θ( Y| M=m) (the intervention X X_ is now encapsulated as part of θ), we have to estimate the joint distribution Pθ(Y,M)P_θ( Y, M) first – which is not identifiable from our observations (due to the confounder between X and M). The reader may convince themselves, that this assessment is correct, on basis of the confounder preventing an unbiased estimation of the selection-bias P⁡(X |M)P( X_ | M). So in general, the conditional effect discussed above is not identifiable, but under the constraint of X X_ taking only a single value it is. Both conclusions in example E.31 make sense, depending on the question we initially intended to ask. The evident next question to ask is: Can we, in our formulation, recover the results of do-calculus? we draw conclusions under assumptions on the internal structure of X X_ ? As it turns out, there are actually two different ways to achieve this: [label=()] 1. Conventionally, expressions like P⁡(Y| (X=x),X=x)P( Y| ( X=x), X=x) are not considered. If the intervention on X X is singular, there is no further information provided by its value. However, our formalism allows to ask for Pθ(Y|M=m,X =x)P_θ( Y| M=m, X_ =x), and this does make sense if X X_ is not singular. Interestingly this query is identifiable (because Yx,mY_x,m is identifiable), so one can directly recover the notion of conditional queries as interpreted by the do-calculus (and [56]) by conditioning on intervened variables (including them in X~ X for the basic query). This approach does not require any additional theory and works as is. It does, however, not provide any insights about other internal structure affecting selection-bias. 2. Similar to, for example, instrumental variables (example 5.8), we can extend our notion of regularity : The rule we need is that for some kernels in ℱ special⊂ℱ intervene∪ℱ modelF_ special _ intervene _ model (containing for example all singular interventions in queries or experimental data) selection-bias is trivial, for a regular functional (μ|ν)↦μ(μ|ν) μ if μ∈ℱ specialμ _ special. Such additional classification of kernels (similar to instrumental variables) can be useful more generally. For example if X X is multi-dimensional or categorical of the form X∈1,…,m×1,…,n X∈\1,…,m\×\1,…,n\ and M M depending only on one coordinate-projection π1 _1, Y Y on the other coordinate-projection π2 _2, then if P⁡(X)P( X) is a product ( components relevant to M M and Y Y are independent X1 X2 X_1 X_2), the query of example E.31 is identifiable. This approach is more complicated to implement, as it requires a custom notion of regularity, but if one is interested in complex problems with categorical or multi-dimensional variables, it may provide substantial insights. In conclusion, the confusion seems to arise from the do-calculus conventionally being thought of as a set of causal inference rules. However, through rule 2, it allows additionally for conclusions based on the internal structure of interventions (and unrelated to causality). To reproduce the same results from a causal rule-set, one might have to add such an assumption about internal structure separately. This is possible for our approach in different ways, simply by conditioning on interventions, or by clarifying the rules of allowed computations for interventions known to satisfy suitable constraints. E.5 Standard Results for the Multi-Context, Experimental Case In a seminal series of papers [5; 4; 6] (see also [34] for an overview) Bareinboim and Pearl studied the use of "selection-variables" for combining observations and experiments from multiple related contexts. This idea was also employed and further developed ( the name "context-variables") for causal discovery for example in [19; 27; 15] and has formed the technical and conceptual basis for most available results concerning causality in multi-context systems. Many of the problems encountered and insights made possible by these approaches provided substantial motivation for the present paper. It hence seems natural assess how our formalism describes their "mz-transportability" setups. We primarily discuss the topic along the lines of [4] in this subsection, but reproduce its contents in language consistent with §E.1. The setup from before is modified two-fold: Instead of one system, multiple systems (with known relations) are available, with further potentially different experiments (observations from intervened, by do-interventions, models). We start with two contexts π and π∗π^* (and add experiments later). The reader may want to have a look at Fig. 8, which illustrates many of the encountered concepts on an example. Definition E.32 (Selection Diagrams). [4, Def. 1 (p. 4)] (slightly rephrased to make it clearer on which nodes D is defined): Let (M,M∗)(M,M^*) be a pair of SCMs relative to domains (π,π∗)(π,π^*) [domains are "labels" for the contexts] sharing a causal graph G. (M,M∗)(M,M^*) is said to induce a selection diagram D if D is constructed as follows: D has nodes ∪ N∪ S, where S contains a "selection variable" SvS_v for each v∈v with fv≠fv∗f_v≠ f_v^* or P⁡(ηv)≠P⁡(ηv∗)P( _v)≠ P( _v^*), every edge in G is also an edge in D; D contains an extra edge Sv→vS_v→ v for each Sv∈S_v∈ S. Remark E.33 (Changing Latents). This definition is not compatible with changing latent variables, but can immediately be interpreted in that case by adding a selection variable SvS_v also if a latent parent l∈ ∩Ll∈ _v∩ L has changing mechanism fl≠fl∗f_l≠ f_l^* or changing noise P⁡(ηl)≠P⁡(ηl∗)P( _l)≠ P( _l^*). Definition E.34 (mz-Transportability). [4, Def. 2 (p. 4)]: Let =(),…,() D=\D^(1),…,D^(n)\ be a collection of selection diagrams relative to source domains Π=π1,…,πn =\ _1,…, _n\ and a target domain π∗π^*, respectively [ are SCMs M∗M^* and M1,…,MnM_1,…,M_n and D(k)D^(k) is a selection-diagram for (Mk,M∗)(M_k,M^*), in particular they all share a causal graph G], and ZkZ_k (and Z∗Z^*) [subsets of I varsI_ vars] be the variables in which experiments can be conducted in domain πk _k (and π∗π^*). Let (Pk,Izk)(P^k,I_z^k) be the pair of observational and interventional distributions of πk _k [on subsets of ZkZ_k] [ =P⁡(Xvkv∈)P^k=P(\ X^k_v\_v ), where Xvk X^k_v are the endogenous variables of MkM_k, and Izk=∪Z′⊂ZkP(Xvkv∈| (Z′=z′))I^k_z= _Z ⊂ Z_kP(\ X^k_v\_v | (Z =z )), where the union-symbol ’∪’ means the collection of these distributions is considered], and in an analogous manner, (P∗,Pz∗)(P^*,P^*_z) be the observational and interventional distributions of π∗π^*. [Identifiability:] The causal effect R=P∗(Y| (X=x))R=P^*(Y| (X=x)) is said to be mz-transportable, from Π to π∗π^* in D if P∗(Y| (X=x))P^*(Y| (X=x)) is uniquely computable from ∪k(Pk,Pzk)∪(P∗,Pz∗) _k(P^k,P^k_z)∪(P^*,P^*_z) in any model [any collection of SCMs M∗,M1,…,MnM^*,M_1,…,M_n] that induces D. Remark: This notion of identifiability used is conceptual very similar to Def. E.6. Definition E.35 (mz*-shedge). [4, Def. 5 (p. 6)]: Let =(),…,() D=\D^(1),…,D^(n)\ be a collection of selection diagrams relative to source domains Π=π1,…,πn =\ _1,…, _n\ and a target domain π∗π^*, respectively, SkS_k represents the collection of S-variables in the selection-diagram D(i)D^(i) and let D(∗)=∗D^(*)= G^* be the causal diagram [graph] of π∗π^*. Let (Pk,Izk)(P^k,I_z^k) be the pairs of observational and interventional [on subsets of ZkZ_k] distributions of πk _k [as before], similar for (P∗,Iz∗)(P^*,I_z^*). Consider an R-rooted hedge (F,F′)(F,F ) (Def. E.13). We say that the induced collection of pairs of R-rooted c-forests over each diagram ((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)), is an mz-shedge for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n) if they are all hedges and one of the following conditions hold for each domain πk _k (k∈∗,1,…,nk∈\*,1,…,n\): [label=0.] 1. There exists at least one variable s∈Sks∈ S_k pointing to the induced diagram Fk′F _k, or 2. (Fk∖Fk′)∩Zk=∅(F_k F _k)∩ Z_k= , or 3. The collection of pairs of c-forests induced over diagrams, ((F1,F1′),…,(Fk∖Zk∗,Fk′)CLOSE,((F_1,F _1),…,(F_k Z_k^*,F _k), …,…, OPEN(Fn,Fn′))(F_n,F _n)), is also an mz-shedge relative to (Iz∗,Iz1,…,Iz∖zk∗,…,Izn)(I^*_z,I_z^1,…,I_z z^*_k,…,I_z^n), where Zk∗=(Fk∖Fk′)∩ZkZ_k^*=(F_k F _k)∩ Z_k. Furthermore, we call mz*-shedge the mz-shedge in which there exist[s] one directed path from R∖(R∩ (X))R (R∩ _F(X)) to R∩ (X)R∩ _F(X) not passing through X. Remark E.36. Unfortunately the quoted definition is in places slightly unclear. The appendix to [4] containing proof-details also seems to not be available online (anymore?). (a) The path from R∖(R∩ (X))R (R∩ _F(X)) to R∩ (X)R∩ _F(X): For the distinction between mz*-shedges and mz-shedges [4, p. 6f] give the following example (lhs): [4, p. 6f]:XXYYZZ Extended Example:XXYYZZY2Y2 As [4, p. 6f] correctly point out P(Y| (X=x))P(Y| (X=x)) is identifiable, while P(Y,Z| (X=x))P(Y,Z| (X=x)) is not. The requirement for the existence of a path (last sentence of the definition) is motivated as follows: F=X,Y,ZF=\X,Y,Z\ and F′=Z,YF =\Z,Y\ are c-forests rooted at R=Z,YR=\Z,Y\ and the directed path "from R∖(R∩ (X))=ZR (R∩ _F(X))=\Z\ to R∩ (X)=YR∩ _F(X)=\Y\ not passing through X" does not exist. However, the definition of a hedge already asks for R⊂ (Y~)R⊂ _ ^L( Y) (where Y~ Y is the set of target-nodes). In particular (F,F′)(F,F ) is a hedge for Y~=Y,Z Y=\Y,Z\ (consistent with P(Y,Z| (X=x))P(Y,Z| (X=x)) being not identifiable), but it is not a hedge for Y~=Y Y=\Y\, because Z is not an G_ do-ancestor of Y. Indeed for Y~=Y Y=\Y\ no hedge exists (removing Y from F,F′F,F would break them being c-forests). Note, that directed paths "not passing through X" are the same as paths in G_ do (if the path does not start at a node in X). So the definition seems to attempt to move the ancestral requirement from G_ do-ancestors of Y "up" to G_ do-ancestors of (R∩ (X))(R∩ _F(X)) (there is also a small additional technical problem here: if one of the sets R∖(R∩ (X))R (R∩ _F(X)) or R∩ (X)R∩ _F(X) is empty – which occurs especially for simple examples quite often – there obviously never exists such a path; the intended statements seems to be one about ancestral relationships in G_ do). However, the weaker requirement of R⊂ (Y~)R⊂ _ ^L( Y) already seems to prevent identification. Consider the second, extended, example above: If we want to identify (from a single context without experiments) P⁡(Y2| (X=x))P(Y_2| (X=x)), then we notice that (F,F′)(F,F ) is a hedge as R⊂ (Y2)R⊂ _ ^L(\Y_2\). Yet there still exists no directed path from R∖(R∩ (X))=ZR (R∩ _F(X))=\Z\ to R∩ (X)=YR∩ _F(X)=\Y\. So for theorems 2 and 3 in [4, p. 7] to hold, one cannot insist on the existence of such a path (otherwise the extended example above is a simple counter-example). (b) The "inducedness" and point 3: The authors say a mz-shedge is "induced" from a pair (F,F′)(F,F ). Since all contexts in Π share a graph this makes sense. However, point 3 then modifies a single FiF_i which clearly breaks the "inducedness" property, so clearly the result cannot "also [be] a mz-shedge". It appears that the statement intended by the authors was something like: "((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)) has the mz-shedging property, if 1–3 hold" (where in 3 it is only required that the replacement "also has the mz-shedging property"); this can be written inductively in a non-cyclic form. Then an mz-shedge is a pair (F,F′)(F,F ) such that the induced ((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)) has the shedging-property. Our attempt at a slightly clearer phrasing is the following: Definition E.37 (mz-shedge revisited). Interpreting [4, Def. 5 (p. 6)]: Let =(),…, D=\D^(1),…, D(n)D^(n)\ be a collection of selection diagrams relative to source domains Π=π1,…,πn =\ _1,…, _n\ and a target domain π∗π^*, respectively, SkS_k represents the collection of S-variables in the selection-diagram D(i)D^(i) and let D(∗)=∗D^(*)= G^* be the causal diagram [graph] of π∗π^*. Let (Pk,Izk)(P^k,I_z^k) be the pairs of observational and interventional [on subsets of ZkZ_k] distributions of πk _k [as before], similar for (P∗,Iz∗)(P^*,I_z^*). We say that a collection of pairs of R-rooted c-forests, one for each diagram, ((F1,F1′),…CLOSE,((F_1,F _1),…, OPEN(Fn,Fn′))(F_n,F _n)), has the mz-shedging0 property for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n) if they are all hedges (Def. E.13) and one of the following conditions holds for each domain πk _k (k∈∗,1,…,nk∈\*,1,…,n\): [label=0.] 1. There exists at least one variable s∈Sks∈ S_k pointing to the induced diagram Fk′F _k, or 2. (Fk∖Fk′)∩Zk=∅(F_k F _k)∩ Z_k= . Inductively, ((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)), has the mz-shedgingm+1 property for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n) if they are all hedges (Def. E.13) and one of the following conditions holds for each domain πk _k (k∈∗,1,…,nk∈\*,1,…,n\): [label=0.] 1. There exists at least one variable s∈Sks∈ S_k pointing to the induced diagram Fk′F _k, or 2. The collection of pairs of c-forests induced over diagrams, ((F1,F1′),…CLOSE,((F_1,F _1),…, (Fk∖Zk∗,Fk′),(F_k Z_k^*,F _k), …,…, OPEN(Fn,Fn′))(F_n,F _n)), is has the mz-shedgingm property relative to (Iz∗,Iz1,…,Iz∖zk∗,…,Izn)(I^*_z,I_z^1,…,I_z z^*_k,…,I_z^n), where Zk∗=(Fk∖Fk′)∩ZkZ_k^*=(F_k F _k)∩ Z_k. ((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)), has the mz-shedging property for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n) if ∃m∈ℕ0∃ m _0 such that it has the mz-shedgingm property for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n). Consider an R-rooted hedge (F,F′)(F,F ) (Def. E.13). (F,F′)(F,F ) is a mz-shedge, for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n), if the induced ((F1,F1′),…,(Fn,Fn′))((F_1,F _1),…,(F_n,F _n)) has the mz-shedging property for P∗(Y| (X=x))P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I^*_z,I_z^1,…,I_z^n). Lemma E.38 (Shedge-Criterion). Interpreting [4, Thm. 3 (p. 7)]: Let =(),…, D=\D^(1),…, D(n)D^(n)\ be a collection of selection diagrams relative to source domains Π=π1,…,πn =\ _1,…, _n\ and a target domain π∗π^*, respectively, and Izi\I_z^i\ for i=∗,1,…,ni=\*,1,…,n\ defined appropriately. If there is a mz*-shedge for the effect R=P∗(Y| (X=x))R=P^*(Y| (X=x)) relative to experiments (Iz∗,Iz1,…,Izn)(I_z^*,I_z^1,…,I_z^n) in D, R is not mz-transportable (Def. E.34) from Π to π∗π^* in D. Lemma E.39 (Completeness of TRmz-Algorithm). [4, Thm. 4 (p. 8)]: If the TRmz-algorithm ([4, Fig. 3 (p. 7)]) fails to identify P(Y~| (X~=x))P( Y| ( X=x)), then it outputs a graph-pair that contains as edge-subgraphs c-forests that span a mz*-hshedge for P(Y~| (X~=x))P( Y| ( X=x)). [To our understanding, the same problem as with [57, Cor. 3 (p. 1225)] appears, see [55, Prop. 1 (p. 7)] and Lemma E.14]. As a corollary by using the shedge-criterion thus [4, Thm. 5 (p. 8)]: The TRmz-algorithm is complete. E.6 Translating the Transportability Setup It is possible to translate the mz-transport setup in two equivalent ways: as different systems sharing properties or as copies of the same system with selection-variables providing a multi-level structure (where the upper level is trivial exclusively of nodes without edges connecting them). Viewpoint – Different Systems Sharing Properties: As long as every context c∈I dataset obsc∈ I_ dataset obs has an infinite number of observations, it is only relevant what properties are shared with the query (which is always in π∗π^*). For clarity we nevertheless give the full model (and modified Def. E.16): Definition E.40 (IID-Model, Shared Properties). We call a model IID (shared properties), if it can be written in the following form: I=I dataset×I vars×I sampleI=I_ dataset× I_ vars× I_ sample, where |I sample|=∞|I_ sample|=∞, with symmetry-group G=I dataset×I sampleG= S_I_ dataset× I_ sample (acting trivially on the middle factor I varsI_ vars of I), and variables J(C,v)=C×v×I sampleJ_(C,v)=C×\v\× I_ sample (where C⊂I datasetC⊂ I_ dataset) with symmetry HC,v=C×I sampleH_C,v= S_C× I_ sample, where C⊂I dataset S_C⊂ S_I_ dataset is embedded as the permutations on elements of C extended by the identity on other elements of I datasetI_ dataset. Lemma E.41. Given a mz-transport setup M mzM mz Def. E.34, and an intervention P(Y~| (X~=x))P( Y| ( X=x)), then there is an IID (shared properties) model (Def. 2.6) ℳ⁡(M mz)M(M mz) with |I dataset obs|=∑k∈∗,1,…,n(|Izk|+1)|I_ dataset obs|= _k∈\*,1,…,n$|I_z^k|+1), call its single elements (πk,Z′)( _k,Z ) (where Z′Z is the actual intervention, cf. Def. E.34), |I dataset query|=1|I_ dataset query|=1, call its single element β, such that ∀c=(πk,Z′)∈I dataset obs,s∈I sample∀ c=( _k,Z )∈ I_ dataset obs,s∈ I_ sample P=P⁡(Xvπkv∈∖Z′| (Z′=z′)) P=P(\ X _k_v\_v Z | (Z =z )) = Pθ(Vii∈c×(∖Z′)×s) = P_θ(\ V_i\_i∈\c\×(O Z )×\s$ P(Y~| (X~=x)) P( Y| ( X=x)) = Pθ(Vii∈β×Y~×∗) , = P_θ(\ V_i\_i∈\β\× Y×\*\) , where Xvπk X _k_v are the variables of SCM MkM_k (corresponding to context πk _k) and ∗* is the single (relevant) "sample" of the query. This model is such that for i=(β,x~,∗)i=(β, x,*) with x~∈X~ x∈ X and for i=((πk,Z′),z~′,s)i=(( _k,Z ), z ,s) with z~′∈Z′ z ∈ Z the corresponding mechanism fi∈ℱ intervenef_i _ intervene is known: It is singular at value x or z′z respectively. The I-graph of ℳ⁡(M mz)M(M mz) is the disjoint union of its restrictions to c×I vars×s\c\× I_ vars×\s\ which are, for c=(πk,Z′)c=( _k,Z ) of the form G^L (the shared causal graph including latents) if Z′=∅Z = and ,′ G_ do^L,Z for the intervention on Z′Z and its restriction to β×I vars×∗\β\× I_ vars×\*\ which is of the form G_ do^L (for the intervention on X~ X). The query graph (embedded into β×I vars×∗\β\× I_ vars×\*\, which is as before) can be defined as in the single-context case (Def. E.21). Similarly the classification of embeddings and results on definitions of identification carry over. Viewpoint – Multi Level Structure: This is the viewpoint taken for example by [27] (even though the connection to multi-level statistics [14] is not made explicit). Definition E.42 (IID-Model, Shared Properties). We call a model IID (shared model), if it can be written in the form Def. E.16 after extending the system as follows: Let S:=I varsS:=I_ vars be a disjoint copy of I varsI_ vars. To avoid issues with observational support (Rmk. A.6) we follow the philosophy of [15]: We think of "context" as a hidden property of the system, and the (know) context πk _k (the dataset of origin) as a proxy for this property (this also clarifies how to define context- -variables as random elements): They are not realized per-sample but rather (at most) per data-set, more precisely: Add for all v∈I varsv∈ I_ vars a variable sv0s^0_v to S, and add (the single node in the I-graph) sv0s^0_v as a parent to (the infinitely many nodes in the I-graph) of ((π∗,Z′),v,s)((π^*,Z ),v,s) for all Z′⊂Z∗Z ⊂ Z_* and for all s∈I samples∈ I_ sample an of (β,v,∗)(β,v,*) (making it auxiliary for the query-graph). If DkD_k contains a selection-variable SvS_v, then add a index svks^k_v to S, and add (the single node in the I-graph) svks^k_v as a parent to (the infinitely many nodes in the I-graph) of ((πk,Z′),v,s)(( _k,Z ),v,s) for all Z′⊂ZkZ ⊂ Z_k and for all s∈I samples∈ I_ sample. If DkD_k does not contain a selection-variable SvS_v, instead add sv0s^0_v as a parent of these variables. Finally add S to I, I by I′=I⊔SI =I S (a disjoint union). Further we assume the viewport is such that S∩V⁡(N)=∅S∩ V(N)= for all N. E.7 Multi-Context and Transportability π∗π^* (target) π1π^1 π2π^2 Selection- Diagram DiD_i XXZ1Z_1Z2Z_2Y XXZ1Z_1Z2Z_2Y Observations XXZ1Z_1Z2Z_2Y XXZ1′Z_1 Z2′Z_2 Y XXZ1Z_1Z2Z_2Y′Y Intervention Query:X doX doZ2Z_2YYL2L_2L3L_3 Experiment:XXZ1′Z_1 Z2 doZ_2 doYYL2L_2 Experiment:XXZ1 doZ_1 doZ2Z_2Y′Y L1L_1L2L_2L3L_3 Extract X do∈ℱ knownX do known, glue query from known and extracted. ∘ 3L_3 ∘ 2Z_2L2L_2 Figure 8: Example from [4, Fig. 1 c, d (p. 3)] (first row), filled squares are the selection-variables for each selection-diagram, experiments are possible in Z2Z_2 (for π1π^1) and in Z1Z_1 (for π2π^2) respectively (gray circle nodes). Below (rows 2 and 3), are observations, observed interventions (experiments) and the query-graph. This is the data-available and the problem to solve (query). Finally the last line shows relevant extracted c-subgraphs (in this case both non-trivially extracted kernels are obtained from the respective experiment). Mechanisms with a prime (dark orange) differ compared to π∗π^*. Viewpoint – Different Systems Sharing Properties: Lemma E.43 (ED-Phase). The ED-phase (the first two steps of Algo. EDAIdentify) outputs, after a finite number of steps, all c-fragments (cf. Lemma E.27) in one of: π G _k (the context πk _k affects which kernels each node is aligned to) or π,′ G_ do _k,Z for Z′⊂ZkZ ⊂ Z_k. together with identification formulas. proof As for Lemma E.27, note that different contexts correspond to disconnected graph-components in the I-graph. Lemma E.44 (Identification of Do-Interventions). Applying algorithm EDAIdentify to a query P(Y| (X)=x)P( Y| ( X)=x) returns after a finite number of steps. It outputs a non-empty set (thus at least one sound identification strategy) if and only if there is a no mz-shedge (F,F′)(F,F ) for P(Y~| (X~=x))P( Y| ( X=x)). proof [Proof Sketch] Formal details about c-forests etc. are similar as in the proof of Lemma E.29. Additionally we use the following information: We can use an extracted c-fragment for the mz-transport query, if its nodes are aligned to the same kernels as in the query-graph (as in π∗π^*), this explains the condition "1." in the mz*-shedge definition (Def. E.35). We extract c-forests of π,′ G_ do _k,Z , which have no edges into Z′Z (the targets of interventions in experimental data-sets), in particular nodes in Z′Z are their own c-components (aligned to ℱ interveneF_ intervene however), but also they thus cannot be part of any other c-fragment, which explains the appearance of conditions "2." and "3." in the mz*-shedge definition (Def. E.35). Viewpoint – Multi Level Structure: This time, we will extract many results with pinned nodes: the hidden (potentially very complex) system properties encoded by selection-variables are realized once for one or multiple systems, potentially including π∗π^* and thus our query. Definition E.45 (Multi-Level Query-Graph). The query-graph from Def. E.21 needs some small modification. We add, for each node in v∈ proper queryv _ proper query, a pinned node aligned to the index i=sv0∈S⊂Ii=s_v^0∈ S⊂ I. The underlying query thus contains a kernel δsv0 _s_v^0 in this place and we ask for the target Pθ(Y~)P_θ^D( Y) to be δi _i-identified. Condition "1." in the mz*-shedge definition (Def. E.35) now arises from agreement (or disagreement) of aligned indices at pinned nodes. In gluing G obs(G query)G obs(G query) this structural graph has aligned kernels δsv0 _s_v^0 in π∗π^* and δsv0 _s_v^0 or δsvk _s_v^k in πkπ^k which must be matched (as before), see Algo. SVSearch. As before, experimental knowledge is the reason for the appearance of "2." and "3.". E.8 Mediation and Counterfactuals Observations G: XXMMYY Query-Graph: X _ ^BX _ ^AMMYY Figure 9: Query graph yielding the mediation-formula. Lemma E.46. On IID-observations for G in Fig. 9, extraction yields kernels X=P⁡(X)X=P(X), Mx=P⁡(M|X=x)M_x=P(M|X=x) and Yx,m=P⁡(Y|X=x,M=m)Y_x,m=P(Y|X=x,M=m) plus X ,X ∈ℱ interveneX_ ^A,X_ ^B _ intervene, singular at x~a x^a and x~b x^b. The structured query in Fig. 9 with nodes of interest Y\Y\ is identifiable (and identified by Algo. EDAIdentify) as for all B∈ℬ⁡( Y)B ( Y): Y[m,xb]∘[X ⊗(M[xa]∘X )](B) Y_[m,x^b] [X_ ^B (M_[x^a] X_ ^A)](B) =∫X (dxb)∫X (dxa)∫Mxa( )∫Yxb,m( )1B(y) = X_ ^B(dx^b) X_ ^A(dx^a) M_x^a( ) Y_x^b,m( )1_B(y) =∫Mx~a( )∫Yx~b,m( )1B(y) = M_ x^a( ) Y_ x^b,m( )1_B(y) =∫P⁡(M=m|X=x~a)(∫P⁡(Y=y|X=x~b,M=m)1B(y) ) . = P(M=m|X= x^a) ( P(Y=y|X= x^b,M=m)1_B(y) ) . Taking the expectation of Y we get E⁡[Y[m,xb]∘[X ⊗(M[xa]∘X )]] E [Y_[m,x^b] [X_ ^B (M_[x^a] X_ ^A)] ] =∫P(M=m|X=x~a)E[Y=y|X=x~b,M=m] . = P(M=m|X= x^a)E[Y=y|X= x^b,M=m] . our algorithm finds the mediation-formula for natural direct effects (see [31, Eq. 8, (p. 6)]; usually natural direct effects are defined as a difference of expectations, but taking the expectation in Y computes both terms of the difference immediately). With additional observed confounders Z1,…Z_1,…, extraction yields kernels with more arguments, the same confounders appear in the query, but are not in the variables of interest, thus get integrated out as in [31, Eq. 8, (p. 6)]. Considering that we never introduced counterfactuals, it may seem quite surprising, that we can describe natural direct effects, which conventionally are formalized via counterfactuals. However, maybe more accurately, the idea behind natural direct effects can be directly articulated with our more flexible query-language, thus counterfactuals are not actually needed to phrase the questions we wanted to ask in the first place. Many issues arising for counter-factuals, see [42], seem related to formulating practical questions via counterfactuals, rather than from the original questions. From the formal perspective, counterfactuals are usually seen as fundamentally different from interventional queries, Pearl [30] even elevates them to a separate rank on his causal ladder. This special place is usually justified by the observation that there are counterfactuals that cannot be identified even given full experimental knowledge [2, Lemma 2 (p 5)]. However such counter-examples usually rely on scm-mechanisms fi( ,ηi)f_i( _i, _i) which are not injective in their respective noise-term argument. This type of counter-example seems impossible with mechanisms formulated as probability-kernels, as pushing forward along non-injective mappings removes precisely the ambiguity in the model that allows examples like [2, Lemma 2 (p 5)]. This raises the question if this kind of counterfactual is meaningful in the first place, which in turn puts the special role of counterfactuals in doubt. E.9 Conclusion We focused our comparison to literature on the IID case and do-interventions (both in queries and experimental data). The primary reason for this decision is, that most results in the literature are in these setups or build on these setups. The relation to do-calculus and selection- -variables seems of central importance to understand – and validate – our approach in the light of well-understood problem-statements. While we can generalize some of the results immediately within the space of questions available in standard causal language (for example to soft-interventions; such individual problems have often also been studied, see [11]), the purpose of our formalism is not primarily on the extension to a single setup, rather it is in unifying many apparently completely different causal questions (like mediation, transportability, etc.) into a simple shared language and to expand the scope of accessible data-structures and accessible queries – not just algorithmically, but also in terms of what assumptions about data and what queries can be formulated and be made formally available in the first place. Appendix F Future Work Our approach is very flexible by design, therefore it is impossible to fully explore all its consequences in a single paper. We summarize some insights and observations relevant to future work here. This should also hope clarify why the scope of the present paper was chosen as presented. F.1 Statistical Considerations Our approach will usually extract small compounds of mechanisms, then compose (possibly many) such smaller pieces (via Lemma 3.8). A main reason for doing so, is the high symmetry of small compounds (Rmk. 4.3), besides improving asymptotic results, these can, in the finite data case, be learned on a larger part of the data. They might (having better symmetry) also be more robust under structure miss-specification. Further, the results of [17] for linear models suggest that at least for simple (low parameter-count per learned kernel) estimators such a local approach via many small pieces can actually be statistically very efficient. It is also clear however, that for very complex estimators, or very large queries, statistical properties can also be adversely affected by this approach. The extraction phase, and especially the query-construction phase, should therefore potentially take statistical finite-sample properties into account. Our identification results are phrased relative to probability kernels, similar to how classical identification results are often phrased relative to conditional distributions [61; 56; 36; 49]. Depending on the actual quantity of interest, one will want to choose one’s estimator wisely, cf. C.3. It is in practice of course quite complex to do so, as suitable choices of estimators have to be propagated through regular computations in a correct, but also statistically efficient way. An instructive example for constructing non-trivial estimators for such a compound problem can be found [26]. F.2 Selection Bias and Correlated Missingness Correlated missingness refers to statistical dependence between what is observed and the values taken by the observed (see [47] and the many subsequent works inspired by it). We could model this by allowing the viewport V V itself to be random, for example by adding a (always observed) binary variable Bi B_i for each i∈Ii∈ I such that Vi V_i is observed if and only if Bi=1 B_i=1. This problem is closely related with selection bias: Selecting for Vi∈Ai V_i∈ A_i essentially amounts to Bi=1 B_i=1 if Vi∈Ai V_i∈ A_i and Bi=0 B_i=0 otherwise. Bi B_i has in this case a single parent (Vi V_i), and selection-bias is modeled as a special case of correlated missingness. Allowing for correlated missingness in this sense ( ) is evidently quite non-trivial: Already finding a suitable notion of what an assymptotic limit is supposed to be is not obvious anymore. Providing a comprehensive treatment is beyond the scope of this paper. Special cases on the other hand should be rather accessible. Nested queries, §D.4, already allow for phrasing questions for systems under selection-bias. In the extraction phase, selection bias from a descendant can be for example included by "growing" a backdoor-free graph to envelop the selecting descendant, a disintegration by the selected nodes of the joint distribution of the enlarged G obsG obs should be identifiable from data. In this case a computation in the sense of §3 can no longer be complete (as nested graphs produced by the extraction-phase must be considered for regular computations). F.3 Structure Discovery This paper describes models and how to use them for the identification of queries from data. The separate topic of discovery of such structure is left to future work. We want to briefly remark, that the model-structure as described has is not explicitly bounded in its complexity, meaning it can become more complicated with more data. In practice one does of course require some assumptions about the prior distribution of the model-structure that might appear. As a special case, the prior over model-structures can be chosen to allow only IID-models (with fixed product structure on I) or only time-series etc., in these cases conventional causal discovery ideas apply. If the result of query-estimation does not depend on choice of representative in a discovered Markov equivalence-class, it is identifiable. Generally, it is well-understood that employing some notion of invariant mechanism can substantially aid structure discovery [19; 27; 15; 37]. Considering that in our approach it is formally very clear what this means, holds promise to better understand the discovery of causal structures from data. F.4 Algorithmic Approach We briefly explain what "absorbing" nodes in the main text should mean formally. Then we provide some clarification about some of the algorithms. The algorithmic approach is, as pointed out repeatedly in the main text, considered future work. We provide sketches for algorithms, because these seem quite helpful to better understand the concepts of the main text. Constructing Families: Families of embeddings can be constructed from individual mechanisms by successive "absorbtion" of neighbors. This sub-section briefly explains what this means concretely. Lemma F.1 (Direct Embedding of Model Mechanisms). Given a mechanism (fJ,J,HJ, )(f_J,J,H_J, _J) (Def. 2.5) of the model M (Def. 2.6), define the local graph G consisting of a single inner node y and external nodes x1x_1, …, xκx_κ (where κ is the number of arguments of the mechanisms kernel) with exactly one proper edge from each external to the single inner node. Model-alignment is to μy:=fJμ^y:=f_J (and assigning arguments by (i)=xi ^y(i)=x_i in the structured kernel, cf. A.5). This local graph is embedded by the mappings ψj(y)=j _j(y)=j and ψj(xk)= kJ(j) _j(x_k)= k_J(j). The collection ψjj∈J\ _j\_j∈ J is a family of local graph embeddings anchored at y. proof We unravel definitions carefully. First G is an model-aligned local graph (Def. C.1) by construction. Next we check, that ψj _j is a local graph embedding (Def. 4.1): Injectivity of ψj _j: By Def. 2.5, ∀k: k(j)≠j∀ k: k(j)≠ j and k≠k′k≠ k ⇒ ∀j∈J∀ j∈ J: k(j)≠ k′(j) k(j)≠ k (j). Thus ψj _j is injective. Proper node I-parents included 1: The only proper node is y, by definition of the I-graph (Def. 2.9), ψj(y) _j(y) has parents (ψj(y))=i∈I|∃k:i= (ψj(y))(ψj(y)) _I( _j(y))=\i∈ I|∃ k:i= k_J( _j(y))( _j(y))\. By construction, ψj(y)=j∈J _j(y)=j∈ J the (region of applicability of) the mechanism we are working on, so by definition J(ψj(y))=J( _j(y))=J, also by ψj(y)=j _j(y)=j, thus (ψj(y))(ψj(y))= (j)=ψj(xk) k_J( _j(y))( _j(y))\= k_J(j)= _j(x_k). In particular i∈ (ψj(y))i∈ _I( _j(y)) ⇒ ∃k∃ k: i=ψj(xk)i= _j(x_k). Proper edges equal I-graph edges 2: The only proper node is y, by the previous point (and injectivity), there are exactly κ parents ψj(xk) _j(x_k) in the I-graph, these are also precisely the proper parents in the local graph. Model Alignment 3: The only proper node y is aligned to fJ(ψj(y))=fJf_J( _j(y))=f_J (see above) by construction. There are no pinned nodes. We check that the collection ψjj∈J\ _j\_j∈ J is a family of local graph embeddings (Def. 4.2). Trivial on Anchor 1: ψj(y)=j _j(y)=j by construction. Rigidity 2: Using the notation ψ∗(n):J0→I,j↦ψj(n) _*(n):J_0→ I,j _j(n). Let h∈H=HJh∈ H=H_J (by construction), and j′=h⋅j =h· j with both j,j′∈Jj,j ∈ J, we have to show ψh⋅j(n)=h⋅ψj(n) _h· j(n)=h· _j(n) (cf. Def. 2.3, which explicitly defined equivariance restricted to subsets of I in this way). Case 1 (n=yn=y): ψh⋅j(y)=h⋅j=ψh⋅j(y) _h· j(y)=h· j= _h· j(y) by contstruction. Case 2 (n=xkn=x_k): ψh⋅j(xk)= kJ(h⋅j)=h⋅ kJ(j)=h⋅ψj(xk) _h· j(x_k)= k_J(h· j)=h· k_J(j)=h· _j(x_k) by HJH_J-equivariance of kJ k_J (Def. 2.5). Freeness 3: For the (only) proper node y, we have to show ψ∗(y) _*(y) (see last point) is injective. This is the identity mapping :J→J,j↦j _J:J→ J,j j and clearly injective. Definition F.2 (Absorbed Parents). Given a family of local graph embeddings ψjj∈J0\ _j\_j∈ J_0 and an external node x∈ externx _ extern, we say a family of local graph embeddings ψj′j∈J0′\ψ _j\_j∈ J _0, on J0′⊂J0J _0⊂ J_0, is an x-parent extension of ψjj∈J0\ _j\_j∈ J_0, if exactly one of inner′= inner∪x and pinned′= pinned ,or inner= inner and pinned′= pinned∪x . casesN_ inner =N_ inner∪\x\& andN_ pinned =N_ pinned ,or\\ N_ inner=N_ inner& andN_ pinned =N_ pinned∪\x\ . cases A unique minimal set of outer nodes, and edges, are determined by Def. 4.2. We assume J0′J _0 is chosen maximal with the above properties. We write for the set of x-parent extensions AbsorbPa⁡(ψ,x) := ψj′j∈J0′ | ψj′j∈J0′ isanx−parentextensionofψjj∈J0 . (ψ,x) := \ \ψ _j\_j∈ J _0 | \ψ _j\_j∈ J _0 isanx-parentextensionof\ _j\_j∈ J_0 \ . We say an x-parent extension introduces the mechanism J∈J (Def. 2.6, Def. 2.5), if x∈ inner′x _ inner and μx=fJμ^x=f_J. For a mechanism J∈J , we write AbsorbPa⁡(ψ,x,J) (ψ,x,J) for the subset of x-parent extensions introducing J and AbsorbPa(ψ,x,∗) (ψ,x,*) for the subset of x-parent extensions with x∈ extern′x _ extern . Remark F.3 (Freeness and Non-Uniqueness). The original family ψjj∈J0\ _j\_j∈ J_0, on the external node x, may be degenerate. In order for ψj′j∈J0′\ψ _j\_j∈ J _0, to be free (Def. 4.2 3), if x is absorbed as proper node, it might be necessary to shrink J0J_0, and the resulting J0′J _0 need not be unique. In practice the best choice for J0′J_0 will also depend on latent- and ancestral-structure, see §F.4. Definition F.4 (Absorbed Children). Given a family of local graph embeddings ψjj∈J0\ _j\_j∈ J_0 and an inner node l∈ innerl _ inner, we say a family of local graph embeddings ψj′j∈J0′\ψ _j\_j∈ J _0, on J0′⊂J0J _0⊂ J_0, is an l-child extension of ψjj∈J0\ _j\_j∈ J_0, if inner′= inner∪wN_ inner =N_ inner∪\w\ for l∈ ′(w)l∈ _G (w) and pinned= pinnedN_ pinned=N_ pinned. A unique minimal set of outer nodes, and edges, are determined by Def. 4.2. We assume J0′J _0 is chosen maximal with the above properties. We write for the set of l-child extensions AbsorbCh⁡(ψ,l) := ψj′j∈J0′ | ψj′j∈J0′ isanx−parentextensionofψjj∈J0 . (ψ,l) := \ \ψ _j\_j∈ J _0 | \ψ _j\_j∈ J _0 isanx-parentextensionof\ _j\_j∈ J_0 \ . We say an l-child extension introduces (J,k)(J,k) if μw=fJμ^w=f_J and l is the kkth parent of w. We call the l-child extension ancestral, if every valid ancestral structure A for ψ′ψ contains w↝xw x to some x∈ externx _ extern. Finally, we call a tuple χ=(l,J,k)χ=(l,J,k) a relevant child, if an l-child extension introducing (J,k)(J,k) with J0≠∅J_0≠ exists. We write Chrelevant⁡(ψ) (ψ) for the set of relevant children of ψ. Constructing C-Structures: We briefly provide some context on how theses definitions tie into the algorithms ExtractCS, ExtractCS-R. Definition F.5 (Sub-Algorithms of Algo. ExtractCS). The following sub-routines are involved: • minimal_hidden(ψ,L)(ψ,L) for a family of embeddings and L⊂L returns the set of minimal latent subsets containing L, L′⊂ψ|L⊂L′\L ⊂ L_ψ|L⊂ L \. Remark: By Lemma C.17 this recursive relative search finds all L. • minimal_ancestral_structure(ψ,L,)(ψ,L,A) for a family of embeddings and L∈ψL∈ L_ψ returns the set of minimal valid ancestral structures subsets containing A (in the sense of Def. C.16). Remark: By Lemma C.20 this recursive relative search finds all A. • absorb_parents(ψ,P)(ψ,P) for a family of embeddings and P⊂ outerP _ outer returns the set of local graph embeddings AbsorbPa⁡(ψ,P) (ψ,P) (absorbing parents one-by-one in any order). • absorb_children_ancestral(ψ,C) for a family of embeddings and C⊂C returns the set of tuples (ψ′,J,k)(ψ ,J,k) of ancestral local graph embeddings ψ′ψ introducing (J,k)(J,k), ⁡(ψ,C) (ψ,C) such that no additional ancestral child can be absorbed ( until no more ancestral absorbtions are possible, discard "incomplete" intermediate results). • absorb_child(ψ,χ) for a family of embeddings and a relevant child χ (cf. Def. F.4) returns the set of tuples (ψ′,J,k)(ψ ,J,k) of ancestral local graph embeddings ψ′ψ introducing χ. Practical Considerations: From a practical perspective, the main challenge to an algorithmic realization seems to be in issues of non-uniqueness and ensued problems in structuring intermediate results. It is also clear, that in very simple cases, like IID or mutli-context IID cases an algorithmic realization, in light of classifications of embedded families (Lemma E.22), should be rather easy to attain. It may be helpful to restrict the setup initially. There are some obvious practical modifications that should be made: Instead of infinitude of data-sets, it only matters if they are large enough to give an estimate suitable for the problem at hand. Also, for most cases Algo. EDAIdentify should probably not be evaluated eagerly, however the "matching" filters of Algo. SVSearch can effectively be propagated to absorbtion-operations, so these can be evaluated lazily with constraints making them much more attainable in practice. It also seems that instead of inflating absorbtion-results to ensure freeness (Rmk. F.3) eagerly, it may be more practical to instead track possibly degnerate data-sets, then account for degeneracy in the very end. This not only simplifies absorbtion-operations in practice, but likely estimators can, in the finite sample case, benefit from additional data, even if it is to some degree degenerate (enforcing freeness corresponds to what in the missing data-literature seems to usually be called "deletion"). Appendix G Mathematical Basics and Notation This section revisits standard concepts from measure theory and probability theory, fixing notation for the remainder of the paper. We start by some standard results from measure-theory, in particular disintegrations. The focus is on writing statements as found in literature, under Ass. 2.1 (spaces are standard Borel) in a simplified language suitable for our purposes. Next we recall probability kernels and some commonly employed operations. Afterwards results presented in §G.3 and §G.4 are still elementary, but tailored more specificially to the needs of the present discussion, which makes them hard to find in this form in the literature (we provide proofs for these results). Finally, even more customized, and to our knowledge novel in this form, results are presented separately in §A where kernels are systematically "structured" using certain graphs. G.1 Measure-Theory We recall some standard definitions and translate results from literature, specifically from [22; 8], into a more uniform language – we will only consider standard Borel spaces (Ass. 2.1; see below) which will allow for a number of simplifications. A topological space (X,)(X,O) has an induced Borel σ-algebra ℬ=σ⁡()B=σ(O); the pair (X,ℬ)(X,B) is then a measurable space, a measure on this measurable space is called a Borel measure on X. A measurable space (X,ℬ)(X,B) is called Borel space (there are different conventions, we follow [22, p. 20]) if it is Borel-isomorphic (a Borel-morphism is a measurable map, an isomorphism is a morphism f:X→Yf:X→ Y for which an inverse morphism g – fg= = _Y and gf= = _X – exists) to ([0,1],ℬ[0,1])([0,1],B_[0,1]), where ℬ[0,1]B_[0,1] is the Borel σ-algebra induced by the standard (subspace in ℝR) topology on [0,1][0,1]. A Polish space X is a separable (there exists a dense countable subset) topological space (X,)(X,O) such that there exists a metric d inducing the topology O which makes (X,d)(X,d) into a complete (Cauchy-sequences converge) metric space. Polish spaces, endowed with their Borel σ-algebra are Borel spaces [22, Thm. A1.6]. Given a metric space X, a Borel measure λ on X is called a Radon measure [8, p. 293], if for all compact sets K, λK<∞λ K<∞ (compact subsets of Hausdorff-spaces – and metric spaces are Hausdorff – are closed, thus in the Borel σ-algebra), and ∀B∈ℬ:λB=supK⊂BλK∀ B :λ B= _K⊂ Bλ K (the supremum is over all compact K⊂BK⊂ B). A finite Borel-measure (thus in particular any probability measure) on a Polish space is Radon (see [8, p. 293] and references therein). A measure λ on (X,ℱ)(X,F) is called σ-finite if there is a countable union X=∪iXiX= _iX_i of Xi∈ℱX_i such that ∀i:λXi<∞∀ i:λ X_i<∞. In particular every finite measure, and thus every probability-measure, is also σ-finite. Finally note, that since metric spaces are Hausdorff and in Hausdorff-spaces points (subsets t\t\ containing a single point) are closed, they are in the Borel σ-algebra t∈ℬ\t\ . Further separable metric spaces are second countable, is a countable basis of the topology O, thus of the induced Borel σ-algebra. In particular both is the case for Polish spaces with their Borel σ-algebra ( standard Borel spaces): individual points t\t\ are measurable and the σ-algebra is countably generated. We will later specialize to P=X×YP=X× Y and π=πX:P→Xπ= _X:P→ X the coordinate-projection, but first note the following general result. Definition G.1 (Disintegration). See [8, Def. 1 (p. 292)]: Let π:(P,)→(X,ℬ)π:(P,A)→(X,B) be a measurable map, λ a σ-finite measure on A, μ a σ-finite measure on ℬB, then: We say λ has a (π,μ)(π,μ)-disintegration λxx∈X\ _x\_x∈ X, if [label=()] 1. λx _x is a sigma-finite measure on A, concentrated on π(p)=x\π(p)=x\ ( λxπ(p)≠x=0 _x\π(p)≠ x\=0 for μ-almost all t). 2. ∀ non-negative measurable f:P→ ≥0f:P→ _≥ 0: x↦∫λx( )f(p)x _x( )f(p) is measurable. 3. ∀ non-negative measurable f:P→ ≥0f:P→ _≥ 0: ∫λ⁡( )f(p)=∫μ⁡( )∫λx( )f(p) λ( )f(p)= μ( ) _x( )f(p). Theorem 3 (Existence and Uniqueness of Disintegrations). See [8, Thm. 1 (p. 293)]: Let λ be a σ-finite Radon measure on a metric space P (with its Borel σ-algebra), and let π:P→(X,ℬ)π:P→(X,B) be measurable. Let μ be a σ-finite measure on ℬB that dominates π∗λ _*λ ( π∗λB=0⇒μB=0 _*λ B=0 μ B=0). If ℬB is countably generated and contains all singleton sets, ∀x∈X:x∈ℬ∀ x∈ X:\x\ , then λ has a (π,μ)(π,μ)-disintegration. Uniqueness: If λx\ _x\, λx′\λ _x\ are two (π,μ)(π,μ)-disintegrations of λ, then μ⁡x∈X|λx≠λx′=0μ\x∈ X| _x≠ _x \=0, disintegration is μ-almost everywhere unique. Corollary G.2. Let X and Y be standard Borel spaces and λ a probability-measure on X×YX× Y. Denote by πX:X×Y→X _X:X× Y→ X the projection to the first coordinate. Set μ=(πX)∗λμ=( _X)_*λ. Then there is a μ-almost everywhere unique probability-kernel ν from X to Y, such that λ=μ⊗νλ=μ ν (cf. Def. G.8). proof Both λ and μ are probability measures. Apply Thm. 3 to get λx _x with properties (i–i) of Def. G.1. Define for B∈ℬ⁡(Y)B (Y) νx(B):=λx(x×B) . _x(B):= _x(\x\× B) . For all B′∈ℬ⁡(X×Y)B (X× Y) and for all x∈Xx∈ X by property (i): λx(B′) _x(B ) =λx(B′∩x×Y)+λx(B′∩(X−x)×Y) = _x(B ∩\x\× Y)+ _x(B ∩(X-\x\)× Y) (∗*) = (i)λx(B′∩x×Y) μ−almosteverywhere (i)= _x(B ∩\x\× Y) $μ$-almosteverywhere =νx(πY(B′∩x×Y)) = _x( _Y(B ∩\x\× Y)) We first use this relation to immediately conclude from properties (i) and (i) (which holds in particular for indicator functions) that νx _x is a probability kernel (for normalization, see [8, Thm. 2 (i) (p. 294)]). Second, we use this relation to rewrite property (i) with f⁡(p)=1B′(p)=1πX(B′)(πX(p))1πY(B′∩πX(p)×Y)(πY(p))f(p)=1_B (p)=1_ _X(B )( _X(p))1_ _Y(B ∩\ _X(p)\× Y)( _Y(p)) as follows: λ⁡(B′) λ(B ) =∫λ⁡( )1B′(p) = λ( )1_B (p) = (iii)∫μ⁡( )∫λx( )1B′(p) (i)= μ( ) _x( )1_B (p) =∫μ⁡( )1πX(B′)(x)∫λx( )1πY(B′∩πX(p)×Y)(πY(p)) = μ( )1_ _X(B )(x) _x( )1_ _Y(B ∩\ _X(p)\× Y)( _Y(p)) ((∗),plusμ=(πX)∗λ,sodefinitionallyμ( )=λ⁡(πX−1( ))) ($(*)$,plus$μ=( _X)_*λ$,sodefinitionally$μ( )=λ( _X^-1( ))$) =∫μ⁡( )1πX(B′)(x)λ(B′∩πX(p)×Y) = μ( )1_ _X(B )(x)λ(B ∩\ _X(p)\× Y) =∫μ⁡( )1πX(B′)(x)νx(πY(B′∩πX(p)×Y)) = μ( )1_ _X(B )(x) _x( _Y(B ∩\ _X(p)\× Y)) =∫μ⁡( )1πX(B′)(x)∫νx( )1πY(B′∩πX(p)×Y)(y) = μ( )1_ _X(B )(x) _x( )1_ _Y(B ∩\ _X(p)\× Y)(y) =∫μ( )1πX(B′)(x)∫νx( )1B′(p)=(μ⊗ν)(B′) . = μ( )1_ _X(B )(x) _x( )1_B (p) =(μ ν)(B ) . This demonstrates existence. For uniqueness, assume νx _x and νx′ν _x are two probability-kernels with λ=μ⊗ν=μ⊗ν′λ=μ ν=μ ν . Then both λx(B′):=νx(πY(B′∩x×Y)) _x(B ):= _x( _Y(B ∩\x\× Y)) and λx′(B′):=νx′(πY(B′∩x×Y))λ _x(B ):=ν _x( _Y(B ∩\x\× Y)) are disintegrations (by reverting the arguments given above; the only one that is not evidently an equivalence is that our definition of kernel requires measurability only on indicator functions, but general integrals are limits over such indicator-function integrals, thus not more general in this case, . G.1i follows from Def. G.4i) and thus μ⁡x∈X|λx≠λx′=0μ\x∈ X| _x≠ _x \=0 by the uniqueness-part of Thm. 3. In particular, by equation (∗)(*) also νx=νx′ _x=ν _x, μ-almost everywhere. Corollary G.3. Let S, X and Y be standard Borel spaces and λs _s a probability-kernel from S to X×YX× Y. Denote by πX:X×Y→X _X:X× Y→ X the projection to the first coordinate. Set μs=(πX)∗λs _s=( _X)_* _s. Fix a supporting probability-measure ϑ on S. Then there is a probability-kernel νs,x _s,x from S×XS× X to Y, such that λs=(μ⊗ν)s _s=(μ ν)_s and this kernel is ϑ⊗μ μ-almost everywhere unique in the sense that for and any two such ν and ν′ν : (ϑ⊗μ)(s,x)|νs,x≠νs,x′=0( μ)\(s,x)| _s,x≠ν _s,x\=0 proof Apply Cor. G.2 to ϑ⊗λ λ with π1:(S×X)×Y→S×X _1:(S× X)× Y→ S× X. G.2 Kernels and Basic Properties There are multiple equivalent ways to define kernels, we use the following definition; the preliminary definition form the main text can be made mathematically rigorous and then agrees with this notion (Rmk. G.5). Definition G.4 (Kernels). See [22, p. 19]: Given measurable spaces (S,)(S,S) and (T,)(T,T), a mapping μ:S×→ ¯+μ:S×T→ _+ where ¯+= ≥0∪∞ _+= _≥ 0∪\∞\ are the non-negative real numbers plus an element at infinity, then μ is called a (probability) kernel from (S,)(S,S) to (T,)(T,T) if [label=()] 1. ∀B∈∀ B : μsB:S→ ¯+ ,s↦μ⁡(s,B) _sB:S→ _+ , s μ(s,B) is S-measurable. 2. ∀s∈S∀ s∈ S: μs:→ ¯+ ,B↦μ⁡(s,B) _s:T→ _+ , B μ(s,B) is a (probability) measure. Remark G.5. Given a measurable space (S,)(S,S), on the set ℳ⁡()M(S) of the σ-finite measures on S, one defines the σ-algebra induced by mappings of the form πB:ℳ⁡()→ ¯+,μ↦μB _B:M(S)→ _+,μ μ B ( smallest σ-algebra for which all these projects are measurable w. r. t. the Borel σ-algebra on the reals; see [22, p. 18]). In particular the probability measures ⁡()=πS−1(1)P(S)= _S^-1(\1\) are a measurable subset ⁡()⊂ℳ⁡()P(S) (S), thus possess an induced σ-algebra. Then μ is a probability-kernel from (S,)(S,S) to (T,)(T,T) according to Def. G.4, if and only if S→⁡(),s↦μsS (T),s _s is measurable [22, Lemma 1.37 (p. 19)]. intuition that probability kernels are measure-valued mappings is formally correct in this sense and the preliminary Def. 2.2 can be made precise by fixing this σ-algebra. Example G.6. Given a probability-kernel μ from (S,)(S,S) to (T,)(T,T), then there is a mapping f:S×[0,1]→Tf:S×[0,1]→ T and a uniform random variable η∼U⁡([0,1])η U([0,1]) such that ∀s∈S:f⁡(s,η)∀ s∈ S:f(s,η) has distribution μs _s [22, Lemma 2.22 (p. 34)]. The reader accustomed to SCMs in the notation given a noise-term η and a mapping f may thus think about probability-kernels as the analogue of the law of a random variable for causal mechanisms. The domain S corresponds to the value-space of the parents. Remark: Given random variables ϑ1,…,ϑn _1,…, _n, the new random variable η can be chosen independent η (ϑ1,…,ϑn)η ( _1,…, _n). Conversely, given a measurable mapping f:S×N→Tf:S× N→ T and a random element η∈Nη∈ N, then mapping to the push-forward s↦f(s,η)∗P=f(s,⋅)∗η∗P∈⁡(T)s f(s,η)_*P=f(s,·)_* _*P (T) is measurable, thus defines a probability kernel (see remark G.5). Remark G.7. There are in general many choices for f and η in the example G.6 above. This ambiguity goes far beyond simply transforming η to a universal uniform. Indeed even after standardizing on uniform noise, besides flipping it to 1−η1-η, without continuity-requirements on f we can arbitrarily tear apart and reassemble the unit interval (and adapt f accordingly) or choose non-injective f and so on. The corresponding probability-kernel may then be thought of as a means of representing the equivalence-class of indistinguishable pairs (f,P⁡(η))(f,P(η)). Kernels can easily be arranged together to describe more complex joint distributions : Definition G.8 (Product of Kernels). See [22, p. 19]. Given measurable spaces (S,)(S,S), (T,)(T,T), (U,)(U,U) and (probability-)kernels μ:S×→ℝ≥0μ:S×T _≥ 0 and ν:S×T×→ℝ≥0ν:S× T×U _≥ 0, define the product μ⊗ν:S×(⊗)→ℝ≥0μ ν:S×(T ) _≥ 0 as the (probability-)kernel given by ∀B∈⊗:[μ⊗ν](s,B)=∫μ⁡(s, )∫ν⁡(s,t, )1B(t,u) .∀ B : [μ ν](s,B)= μ(s, ) ν(s,t, )1_B(t,u) . Example G.9. Given probability-kernels μ from ∗\*\ (the one-element set) to S, and ν from ∗×S=S\*\× S=S to T, then by example G.6, there are mappings g:∗→Sg:\*\→ S, f:S×[0,1]→Tf:S×[0,1]→ T and independent uniform random variables ηx,ηy∼U⁡([0,1]) _x, _y U([0,1]) such that X:=g⁡(ηx) X:=g( _x) has distribution μ and ∀s∈S:f⁡(s,ηy)∀ s∈ S:f(s, _y) has distribution νs _s. If we define Y:=f⁡(X,ηy) Y:=f( X, _y), then the pair (X,Y)( X, Y) has joint distribution μ⊗νμ ν, as can be verified by a simple computation. The reader accustomed to SCMs in the notation given an noise-terms η and a mappings f may thus think about products of probability-kernels as a generalization (μ need not, in general, be defined on the trivial space ∗\*\) of the law of the joint distribution of a causal model X→Y X→ Y. Concerning the choice of domain for μ, note that in the case of ν above (which does have a non-trivial domain S), ν encodes P⁡(Y|X)P( Y| X), and its domain corresponds to the value-space of X X. Similarly, the domain of μ relates to conditioning on causal ancestors, if a suitable joint distribution to be conditioned on exists (already for non-stationary time-series, this is typically not the case, see example 4.11). Lemma G.10 (Properties of Products). Products have the following properties: [label=()] 1. Associativity: ξ⊗[μ⊗ν]=[ξ⊗μ]⊗νξ [μ ν]=[ξ μ] ν 2. Trivial marginalization on the right (U∈U is the full space): ∀B∈:[μ⊗ν](B×U)=μ⁡(B)∀ B : [μ ν](B× U)=μ(B) We will write marg←⁡(μ⊗ν)=μ _←(μ ν)=μ for this operator, uniquely computing μ from μ⊗νμ ν. proof (i) Use πuw:T×U×W→U×W _uw:T× U× W→ U× W and set Bt:=πuw(B∩t×U×W)B_t:= _uw(B∩\t\× U× W), then by definition ∀B∈⊗=ℬ⁡(T×U×W)∀ B =B(T× U× W) (using Ass. 2.1): [ξ⊗[μ⊗ν]](s,B) [ξ [μ ν]](s,B) =∫ξ⁡(s, )∫[μ⊗ν]((s,t), )1B(t,u,w) = ξ(s, ) [μ ν]((s,t), )1_B(t,u,w) =∫ξ⁡(s, )∫[μ⊗ν]((s,t), )1Bt(u,w) = ξ(s, ) [μ ν]((s,t), )1_B_t(u,w) =∫ξ⁡(s, )[μ⊗ν]((s,t),Bt) = ξ(s, )[μ ν]((s,t),B_t) =∫ξ⁡(s, )∫μ⁡((s,t), )∫ν⁡((s,t,u), )1Bt(u,w) = ξ(s, ) μ((s,t), ) ν((s,t,u), )1_B_t(u,w) =∫ξ⁡(s, )∫μ⁡((s,t), )∫ν⁡((s,t,u), )1B(t,u,w) . = ξ(s, ) μ((s,t), ) ν((s,t,u), )1_B(t,u,w) . A similar argument expands [ξ⊗μ]⊗ν[ξ μ] ν into the same ("un-associated") expression. (i) By Ass. 2.1, ⊗=ℬ⁡(T×U)T =B(T× U), in particular ∀B∈∀ B , also B×U∈⊗B× U is measurable (so the expression is well-defined). By definition, [μ⊗ν](s,B×U) [μ ν](s,B× U) =∫μ⁡(s, )∫ν⁡(s,t, )1B×U(t,u) = μ(s, ) ν(s,t, )1_B× U(t,u) =∫μ⁡(s, )∫ν⁡(s,t, )1B(t) = μ(s, ) ν(s,t, )1_B(t) =∫μ⁡(s, )1B(t)∫ν⁡(s,t, ) = μ(s, )1_B(t) ν(s,t, ) =∫μ⁡(s, )1B(t)=μ⁡(s,B) . = μ(s, )1_B(t)=μ(s,B) . The last line is obtained by ν being a probability-kernel (normalized to 11) and definition of the integral. Definition G.11 (Composition of Kernels). Given measurable spaces (S,)(S,S), (T,)(T,T), (U,)(U,U) and (probability-)kernels μ:S×→ℝ≥0μ:S×T _≥ 0 and ν:S×T×→ℝ≥0ν:S× T×U _≥ 0, define the composition ν∘μ:S×→ℝ≥0ν μ:S×U _≥ 0 as the (probability-)kernel given by ∀B∈:(ν∘μ)(B):=[μ⊗ν](T×B).∀ B : (ν μ)(B):=[μ ν](T× B). Example G.12. Extending example G.9, note that the composition ν∘μν μ (ν∘μ)(B) (ν μ)(B) =[μ⊗ν](T×B) =[μ ν](T× B) =∫μ⁡( )∫ν⁡(t, )1B(s) = μ( ) ν(t, )1_B(s) simply integrates out μ corresponding to X X in example G.9. The result is thus simply P⁡(Y)P( Y). Why is this a composition? Since Y Y causally depends on X X, it also "sees" the noise ηX _X injected at X X, and the distribution of Y Y is only fully described from a suitable combination of the mechanisms μ at X X and ν at Y Y. Lemma G.13 (Properties of Compositions). Compositions are associative: (ν∘μ)∘ξ=ν∘(μ∘ξ)(ν μ) ξ=ν (μ ξ) Warning: The emerging ring-like algebraic structure does not right-distribute, generally [μ⊗ν]∘ξ≠(μ∘ξ)⊗(ν∘ξ) ,seeLemmaG.18ii,G.27iii.[μ ν] ξ ≠ (μ ξ) (ν ξ) ,seeLemma lemma:kernel_anticausal_propertiesi, lemma:causal_transformationsi. proof We again make use of ⊗=ℬ⁡(T×U×W)T =B(T× U× W) (by Ass. 2.1). By definition ∀B∈∀ B , ((ν∘μ)∘ξ)(s,B) ((ν μ) ξ )(s,B) =[ξ⊗(ν∘μ)](s,T×B) =[ξ (ν μ)](s,T× B) =∫ξ⁡(s, )∫(ν∘μ)((s,t), )1T×B(t,w) = ξ(s, ) (ν μ)((s,t), )1_T× B(t,w) =∫ξ⁡(s, )∫(ν∘μ)((s,t), )1B(w) = ξ(s, ) (ν μ)((s,t), )1_B(w) =∫ξ⁡(s, )(ν∘μ)((s,t),B) = ξ(s, )(ν μ)((s,t),B) =∫ξ⁡(s, )[μ⊗ν]((s,t),U×B) = ξ(s, )[μ ν]((s,t),U× B) =∫ξ⁡(s, )∫μ⁡((s,t), )∫ν⁡((s,t,u), )1U×B(u,w) = ξ(s, ) μ((s,t), ) ν((s,t,u), )1_U× B(u,w) =∫ξ⁡(s, )∫μ⁡((s,t), )∫ν⁡((s,t,u), )1B(w) = ξ(s, ) μ((s,t), ) ν((s,t,u), )1_B(w) Again, the same "un-associated" form can similarly be obtained for ν∘(μ∘ξ)ν (μ ξ). Knowing μ and ν, we can obviously obtain μ⊗νμ ν, interestingly the opposite is also true (beware of the warning in the previous lemma G.13 however) via disintegrations (see [8] for a very nice overview): Lemma G.14 (Disintegration). Let (S,ℬ⁡(S))(S,B(S)), (X,ℬ⁡(X))(X,B(X)) and (Y,ℬ⁡(Y))(Y,B(Y)) be standard Borel spaces (Ass. 2.1) and λs _s a probability-kernel from S to X×YX× Y. Denote by πX:X×Y→X _X:X× Y→ X the projection to the first coordinate and set μs=(πX)∗λs _s=( _X)_* _s (then μs(B)=λs(B×Y) _s(B)= _s(B× Y), so this marginalizes the Y-coordinate away). Then there is a probability-kernel ν from S×XS× X to Y, such that λ=μ⊗νλ=μ ν. This kernel is unique almost everywhere relative to (any fixed choice of) a supporting probability measure ϑ on S, in the sense that for any two such ν and ν′ν : [ϑ⊗μ](s,x)|νs,x≠νs,x′=0[ μ]$s,x)| _s,x≠ν _s,x\=0 We will write disint→⁡(μ⊗ν)=ν _→(μ ν)=ν for this operator, uniquely (in the sense defined above; see also Rmk. A.6) computing ν from μ⊗νμ ν. proof This was already proofed as Cor. G.3, as a consequence of standard results [8] in §G.1 above. Example G.15. If we think of λ as a joint distribution P⁡(X,Y)P( X, Y), then μ (by marginalization from the right, lemma G.10i) contains precisely the information about P⁡(X)P( X) (see example G.9). By example G.6 the constructed kernel ν (for trivial S=∗S=\*$ corresponds to P⁡(Y|X)P( Y| X), so disintegration states in this language that the (regular version [22, Thm. 5.3 (p. 84)] of) the conditional distribution P⁡(Y|X)P( Y| X) is unique and thus can be reconstructed from knowledge of the joint distribution; see also [22, Thm. 5.4 (p. 85)]. It does not generally make sense to talk about commutativity of ⊗ as domains of the involved kernels are not generally suitable for exchanging the order, we will come back to this in Lemma G.27i. For now, we make the following observation: Definition G.16 (Anti-Causal Disintegration). Given are standard Borel spaces (S,ℬ⁡(S))(S,B(S)), (T,ℬ⁡(T))(T,B(T)), (U,ℬ⁡(U))(U,B(U)) and (probability-)kernels μ:S×ℬ⁡(T)→ℝ≥0μ:S×B(T) _≥ 0 and ν:S×T×ℬ⁡(U)→ℝ≥0ν:S× T×B(U) _≥ 0. For B⊂X×YB⊂ X× Y denote by Bt=(y,x)∈Y×X|(x,y)∈BB^t=\(y,x)∈ Y× X|(x,y)∈ B\ the "transposed" set22 2 Note that ℬ⁡(X×Y)=ℬ⁡(X)×ℬ⁡(Y)B(X× Y)=B(X)×B(Y), so transposing in this sense maps measurable sets to measurable sets, induces a well-defined mapping (⋅)t:ℬ⁡(X×Y)→ℬ⁡(Y×X)(·)^t:B(X× Y) (Y× X); indeed the element-wise transposition (x,y)↦(y,x)(x,y) (y,x) is a Borel isomorphism on the Borel product.. For a kernel λ from S to X×YX× Y denote ∀B∈ℬ⁡(Y×X):λst(B):=λs(Bt) .∀ B (Y× X): _s^t(B):= _s(B^t) . If λ=μ⊗νλ=μ ν, then lemma G.14 produces a disintegration of λt=ξ⊗ϑλ^t=ξ . We will denote ξ:=marg←⁡(λt)ξ:= _←(λ^t) by ν∘μν μ (as will be justified by Lemma G.18i) and ϑ:=disint→⁡(λt) := _→(λ^t) by (μ|ν)(μ|ν), thus (definitionally) [μ⊗ν]t=(ν∘μ)⊗(μ|ν) ,[μ ν]^t=(ν μ) (μ|ν) , where ν∘μν μ is a probability-kernel from S to U and (μ|ν)(μ|ν) is a probability-kernel from S×US× U to T. Example G.17. Given a joint distribution for (X,Y)( X, Y), from a causal model X→Y X→ Y, there is a "causal" conditioning P⁡(Y|X)P( Y| X) which is given directly by the causal mechanism ν alone. But there is of course also an anti-causal P⁡(X|Y)P( X| Y) that is uniquely (in a suitable sense) determined by the joint distribution. The joint distribution is P⁡(X,Y)=P⁡(X|Y)P(Y)P( X, Y)=P( X| Y)P( Y) – however P⁡(Y)P( Y) is not trivially given by the mechanism on Y Y, rather by the composition ν∘μν μ (cf. example G.12). Similarly the anti-causal disintegration is a property of the joint distribution and depends on both μ and ν. Lemma G.18 (Properties of Anti-Causal Disintegrations). The kernels ξ and ϑ=:(μ|ν) =:(μ|ν) in Def. G.16 have the following properties: [label=()] 1. ξ=ν∘μξ=ν μ (justifying the notation used in Def. G.16). 2. (μ|ν)∘(ν∘μ)=μ(μ|ν) (ν μ)=μ and (μ∘ξ)⊗(ν∘(ξ|μ))=⟦μ⊗ν⟧∘ξ(μ ξ) (ν (ξ|μ))= μ ν ξ. proof (i) This follows immediately from Lemma G.10 (i) and the definition of composition (using again Ass. 2.1): ξ⁡(s,B) ξ(s,B) =[ξ⊗ϑ](s,B×T) =[ξ ](s,B× T) =[μ⊗ν]t(s,B×T) =[μ ν]^t(s,B× T) =[μ⊗ν](s,T×B) =[μ ν](s,T× B) =(ν∘μ)(s,B) . =(ν μ)(s,B) . (i) By definition of composition and the anti-causal disintegration, and Lemma G.10 (i): ((μ|ν)∘(ν∘μ))(s,B) ((μ|ν) (ν μ) )(s,B) =[(ν∘μ)⊗(μ|ν)](s,U×B) =[(ν μ) (μ|ν)](s,U× B) =[μ⊗ν]t(s,U×B) =[μ ν]^t(s,U× B) =[μ⊗ν](s,B×U) =[μ ν](s,B× U) =μ⁡(s,B) =μ(s,B) Given that compositions are defined as marginalized ⊗ -products, it should not surprise us, that also the second statement works (by using associativity, Lemma G.10): (μ∘ξ)⊗(ν∘(ξ|μ))(s,B×B′) (μ ξ) (ν (ξ|μ))(s,B× B ) =(μ∘ξ)⊗[(ξ|μ)⊗ν](s,B×T×B′) =(μ ξ) [(ξ|μ) ν](s,B× T× B ) =[(μ∘ξ)⊗(ξ|μ)]⊗ν⁡(s,B×T×B′) =[(μ ξ) (ξ|μ)] ν(s,B× T× B ) =[ξ⊗μ]t⊗ν⁡(s,B×T×B′) =[ξ μ]^t ν(s,B× T× B ) =[ξ⊗μ]⊗ν⁡(s,T×B×B′) =[ξ μ] ν(s,T× B× B ) =ξ⊗[μ⊗ν](s,T×B×B′) =ξ [μ ν](s,T× B× B ) =⟦μ⊗ν⟧∘ξ⁡(s,B×B′) = μ ν ξ(s,B× B ) Note, that by definition (ν∘μ)⊗(μ|ν)=[μ⊗ν]t(ν μ) (μ|ν)=[μ ν]^t, so this statement also tells us what happens if we replace the ⊗ -product in the anti-causal disintegration by a confounded one. Recall, that we said earlier, that disintegrating confounded terms is non-trivial – it does not simply produce the right-hand side kernel; this latest statement also tells us what the non-trivial result actually is: The disintegration of the right-hand side is of course the same as the disintegration of the (equivalent) expression on the left-hand side – thus we have also found, that disintegrating ⟦μ⊗ν⟧∘ξ μ ν ξ will yield (ν∘(ξ|μ))(s,t)(ν (ξ|μ))(s,t). This does make intuitive sense: Conditioning on μ selects values of ξ, namely (ξ|μ)(ξ|μ), and the effect of confounding is precisely that conditioning ν on μ will see this selection of its (other, marginalized) parent ξ on top of the causal effect. G.3 Order Independent Operators In general the order of variables in a joint distribution does not matter – yet properties like the factorization of a joint distribution into a marginal and a conditional remain untouched – and similarly we can reorder ⊗ -products in the same way we have already seen for anti-causal disintegrations (Def. G.16). Definition G.19 (Transposition). Given a ⊗ -product μ1⊗…⊗μn _1 … _n with n factors from X to Y1×…×YnY_1×…× Y_n and a permutation T∈nT∈ S_n, there is a Borel-isomorphism TB:YT⁡(1)×…×YT⁡(n)→Y1×…×Yn,(y1,…,yn)↦(yT−1(1),…,yT−1(n))T_B:Y_T(1)×…× Y_T(n)→ Y_1×…× Y_n,(y_1,…,y_n) (y_T^-1(1),…,y_T^-1(n)) and we write (μ1⊗…⊗μn)T(x,B):=(μ1⊗…⊗μn)(x,TB(B))( _1 … _n)^T(x,B):=( _1 … _n)(x,T_B(B)), which is a kernel from X to YT⁡(1)×…×YT⁡(n)Y_T(1)×…× Y_T(n). We defined marginalizations on the right Lemma G.10 2 and disintegrations on the left Lemma G.14 however in the light of anti-causal disintegrations and more generally reordering by transposition, it is unclear how exactly the order in which multiple marginalizations or disintegrations on different orders of the variables are executed matters. The present section shows, that there is a suitable order-independent notion, that clarifies this behavior. First note, that marginalization more generally can immediately be defined as follows: Definition G.20 (Marginalizing Out). Given a product of kernels μ1⊗…⊗μn _1 … _n (with respective target XkX_k) and a subset L⊂1,…,nL⊂\1,…,n\ we define a marginalization of L, denoted margL⁡(μ1⊗…⊗μn) _L( _1 … _n) on ∏k∉LBk∈ℬ⁡(∏k∉LXk) _k∉ LB_k ( _k∉ LX_k) as margL⁡(μ1⊗…⊗μn)(s,∏k∉LBk) _L( _1 … _n)(s, _k∉ LB_k) := [μ1⊗…⊗μn](s,∏Wk) := [ _1 … _n](s,Π W_k) whereWk whereW_k =Xk ifk∈LBk ifk∉L . = casesX_k& ifk∈ L\\ B_k& ifk∉ L . cases Lemma G.21 (Properties of Marginalization). If L=L1∪L2L=L_1∪ L_2, then margL⁡(μ1⊗…⊗μn) _L( _1 … _n) =margL2∖L1⁡(margL1⁡(μ1⊗…⊗μn)) = _L_2 L_1( _L_1( _1 … _n)) =margL1∖L2⁡(margL2⁡(μ1⊗…⊗μn)) . = _L_1 L_2( _L_2( _1 … _n)) . proof This follows immediately from plugging in the definitions. Lemma G.22 (Joint from Marginal and Conditional). Given a product μ=μ1⊗…⊗μmμ= _1 … _m of kernels and a subset C⊂1,…,mC⊂\1,…,m\, label its complement as L:=1,…,m∖CL:=\1,…,m\ C, then there is a unique (in the sense of Lemma G.14) kernel ∃!disintC(μ)∃! _C(μ), such that margL⁡(μ)⊗disintC⁡(μ)=μT . _L(μ) _C(μ)=μ^T . proof Part 1 of Lemma G.23 is proved independently of this result, and will be used here as result (∗)(*). The (non-cyclic) logical proof order is thus: Show Lemma G.23 1, then show the present Lemma G.22, then show Lemma G.23 2 and 3. Inductively over |C||C|. Start of induction (C=∅C= ): If C=∅C= , then define disint∅⁡(μ)=μ _ (μ)=μ. Further this can of course be written as a product of m−|C|=m-|C|=m terms and the last m ( ) factors remain the same. The identity functional is regular by definition. Inductive step (|C|=k↦|C|=k+1|C|=k |C|=k+1, k≥0k≥ 0): We can write C=c1,…,ck+1C=\c_1,…,c_k+1\ with c1<c2<…<ck+1c_1<c_2<…<c_k+1. Define C′:=c1,…,ckC :=\c_1,…,c_k\ and set c:=ck+1c:=c_k+1. By inductive hypothesis, there is disintC′⁡(μ) _C (μ) with margL′⁡(μ)⊗disintC′⁡(μ)=μ _L (μ) _C (μ)=μ, where L′=L∪cL =L∪\c\ is the complement of C′C . Further disintC′⁡(μ)=ν1⊗…⊗νck−k⊗μck+1⊗…⊗μm _C (μ)= _1 … _c_k-k _c_k+1 … _m. With ck+1≥ck+1c_k+1≥ c_k+1 the corresponding term μck+1 _c_k+1 is in the second part. We use anti-causal disintegration to write this as disintC′⁡(μ) _C (μ) = μck+1∘[ν1⊗…⊗νck−k⊗μck+1⊗…⊗μck+1−1] = _c_k+1 [ _1 … _c_k-k _c_k+1 … _c_k+1-1] ⊗[(ν1⊗…⊗νck−k⊗μck+1⊗…⊗μck+1−1|μck+1)⊗μck+1+1⊗…⊗μm] . [( _1 … _c_k-k _c_k+1 … _c_k+1-1| _c_k+1) _c_k+1+1 … _m] . This can be elementarily disintegrated (from the left; Lemma G.14), a regular functional we obtain: disintC⁡(μ) := (ν1⊗…⊗νck−k⊗μck+1⊗…⊗μck+1−1|μck+1)⊗μck+1+1⊗…⊗μm . _C(μ) := ( _1 … _c_k-k _c_k+1 … _c_k+1-1| _c_k+1) _c_k+1+1 … _m . This is regular and has the required product-form, it remains to check, that indeed margL⁡(μ)⊗disintC⁡(μ)=μ _L(μ) _C(μ)=μ. First, note that μck+1∘[ν1⊗…⊗νck−k⊗μck+1⊗…⊗μck+1−1]=margL⁡(disintC′⁡(μ)) _c_k+1 [ _1 … _c_k-k _c_k+1 … _c_k+1-1]= _L( _C (μ)). μ μ =margL′⁡(μ)⊗disintC′⁡(μ) = _L (μ) _C (μ) =margL′⁡(μ)⊗margL⁡(disintC′⁡(μ))⊗disintC⁡(μ) = _L (μ) _L( _C (μ)) _C(μ) =margL⁡(margL′⁡(μ)⊗disintC′⁡(μ))⊗disintC⁡(μ) = _L( _L (μ) _C (μ)) _C(μ) use(∗)withLA=∅,LB=L use$(*)$with$L^A= $,$L^B=L$ =margL⁡(μ)⊗disintC⁡(μ) . = _L(μ) _C(μ) . Remark: In the line applying (∗)(*) (cf. initial paragraph of this proof), we use the same indexing convention explained in detail in the proof of Lemma G.23 3. Finally, uniqueness follows from the uniqueness in Lemma G.14 applied to μTμ^T (a kernel on XC×XLX_C× X_L) factorizing it into the marginalization margL⁡(μ) _L(μ) on XCX_C (marginalizing L leaves its the value-space components of its complement C), and a unique second factor. Any kernel satisfying the characterizing property can take the role of this second factor, thus by its uniqueness is always the same as the kernel constructed above. Lemma G.23 (Properties). These operators have the following properties: [label=()] 1. Given products of kernels μA=μ1A⊗…⊗μmAμ^A=μ^A_1 … μ^A_m and μB=μm+1B⊗…⊗μnBμ^B=μ^B_m+1 … μ^B_n of kernels and subsets LA⊂1,…,mL^A⊂\1,…,m\, LB⊂m+1,…,nL^B⊂\m+1,…,n\, such that arguments of μBμ^B are not in LAL^A, then margLA⁡(μA)⊗margLB⁡(μB) = margLA∪LB⁡(μA⊗μB) . _L^A(μ^A) _L^B(μ^B)\\ = _L^A∪ L^B(μ^A μ^B) . 2. Given products of kernels μA=μ1A⊗…⊗μmAμ^A=μ^A_1 … μ^A_m and μB=μm+1B⊗…⊗μnBμ^B=μ^B_m+1 … μ^B_n of kernels and subsets CA⊂1,…,mC^A⊂\1,…,m\, CB⊂m+1,…,nC^B⊂\m+1,…,n\, such that arguments of μBμ^B are in CAC^A or shared (with μAμ^A, in 1,…,m∖CA\1,…,m\ C^A), then disintCA⁡(μA)⊗disintCB⁡(μB) = disintCA∪CB⁡(μA⊗μB) . _C^A(μ^A) _C^B(μ^B)\\ = _C^A∪ C^B(μ^A μ^B) . 3. Given a product μ=μ1⊗…⊗μmμ= _1 … _m of kernels and subsets C,L⊂1,…,mC,L⊂\1,…,m\, with C∩L=∅C∩ L= , then disintC⁡(margL⁡(μ))=margL⁡(disintC⁡(μ)) . _C ( _L(μ) )= _L ( _C(μ) ) . proof Part 1: With arguments of μBμ^B in 1,…,m∖LA\1,…,m\ L^A (by hypothesis), the left-hand side is defined. By definition, for Bi∈ℬ⁡(Xi)B_i (X_i) with Bi=XiB_i=X_i for i∈LA,LBi∈ L^A,L^B, [margLA⁡(μA)⊗margLB⁡(μB)](∏i=1,i∉LA∪LBnBi) [ _L^A(μ^A) _L^B(μ^B)]( _i=1,i∉ L^A∪ L^B^nB_i) =[μA⊗μB](∏i=1nBi) =[μ^A μ^B]( _i=1^nB_i) =margLA∪LB⁡(μA⊗μB)(∏i=1,i∉LA∪LBnBi) . = _L^A∪ L^B(μ^A μ^B)( _i=1,i∉ L^A∪ L^B^nB_i) . These products form a basis of the product sigma-algebra, thus this proves the claim. Part 2: Define LA:=1,…,m∖CAL^A:=\1,…,m\ C^A, LB:=m+1,…,n∖CBL^B:=\m+1,…,n\ C^B. The right-hand side is (by definition) the unique kernel with the universal property margLA∪LB⁡(μA⊗μB)⊗disintCA∪CB⁡(μA⊗μB)=μA⊗μB _L^A∪ L^B(μ^A μ^B) _C^A∪ C^B(μ^A μ^B)=μ^A μ^B, thus it suffices to show, that the left-hand side also has this characterizing property. To this end, we compute (using the hypothesis that arguments of μBμ^B are in CAC^A in the second and third line): margLA∪LB⁡(μA⊗μB)⊗[disintCA⁡(μA)⊗disintCB⁡(μB)] _L^A∪ L^B(μ^A μ^B) [ _C^A(μ^A) _C^B(μ^B)] = = margLA⁡(μA)⊗margLB⁡(μB)⊗disintCA⁡(μA)⊗disintCB⁡(μB) _L^A(μ^A) _L^B(μ^B) _C^A(μ^A) _C^B(μ^B) by1 by lemma:product_operator_properties:marg_prod = = [margLA⁡(μA)⊗disintCA⁡(μA)]⊗[margLB⁡(μB)⊗disintCB⁡(μB)] [ _L^A(μ^A) _C^A(μ^A)] [ _L^B(μ^B) _C^B(μ^B)] bysparsityG.27 bysparsity lemma:causal_transformations = = μA⊗μB μ^A μ^B bydefinition . bydefinition . The claim follows by uniqueness of disintegrations. Part 3: Define L′:=1,…,m∖CL :=\1,…,m\ C. By L∩C=∅L∩ C= , we have L⊂L′L⊂ L . We will again use the universal property of disintegrations, for example margL′⁡(μ)⊗disintC⁡(μ)=μ _L (μ) _C(μ)=μ. To apply previous parts, we first have to clarify our indexing convention, as terms μi _i in μ appear twice, once in the first term, once in the second term. It does not matter which one of these remains ( to the right-hand side), only that we get each factor exactly once and in the right order. We retain μA=μ1⊗…⊗μmμ^A= _1 … _m as given, and define μB:=μm+1⊗…⊗μnμ^B:= _m+1 … _n, where n=2mn=2m and μm+i:=μiμm+i:= _i, in particular μ=μA=μBμ=μ^A=μ^B (as kernels, ignoring indexing). For subsets of 1,…,m\1,…,m\ we write L+mL+m etc. for the respective shifted versions. We use the same trick as in 2, and notice that the left-hand side is characterized by the property margL′∖L⁡(margL⁡(μ))⊗disintC⁡margL⁡(μ)=margL⁡(μ) _L L( _L(μ)) _C _L(μ)= _L(μ). Thus we compute: margL′∖L⁡(margL⁡(μ))⊗margL⁡(disintC⁡μ) _L L( _L(μ)) _L( _Cμ) =margL′∖L⁡(margL⁡(μA))⊗margL+m⁡(disintC+m⁡μB) = _L L( _L(μ^A)) _L+m( _C+mμ^B) byμ=μA=μB by$μ=μ^A=μ^B$ =marg∅⁡(margL′∖L⁡(margL⁡(μA)))⊗margL+m⁡(disintC+m⁡μB) = _ ( _L L( _L(μ^A))) _L+m( _C+mμ^B) =margL+m⁡(margL′⁡(μA)⊗disintC+m⁡μB) = _L+m( _L (μ^A) _C+mμ^B) by1andG.21 by lemma:product_operator_properties:marg_prodand lemma:kernel_marginalization_safe =margL⁡(μ) = _L(μ) byμ=μA=μBandDef. by$μ=μ^A=μ^B$andDef. The claim follows by uniqueness of disintegrations. G.4 Sparsity and Contraction In principle the constructions introduced so far already describe acyclic causal models, in the sense that a suitable joint distribution can be described by a product over mechanisms in causal order: Each node can be interpreted to depend on all its causal-order predecessors, albeit the dependence on predecessors that are not direct parents is trivial. However, for causal reasoning, precisely this sparsity in arguments, of each node depending only on its parents, is of central importance. Thus a formal description that "forgets" about this structure may be simple, but is unlikely to be useful. We approach the problem in two steps: First, in the present sub-section, we formally define what we mean by "sparsity" in arguments. This notion is unfortunately rather unwieldy to work with, so §A.1 instead encapsulates it in a graphical description. For most practical purposes, the reader may want to continue with §A.1, as it seems rather evident that the graphical definitions given there can be expressed directly in a formal language. Nevertheless, in case the reader really wants to know, these ideas can be expressed formally as follows (we borrow language from tensor-networks, and call the wiring of input-arguments to sources contractions): Definition G.24 (Contraction). Given finite sets IS,IT,IU,IWI_S,I_T,I_U,I_W, given a kernel μ from S:=∏i∈ISXiS:= _i∈ I_SX_i to T:=∏i∈ITXiT:= _i∈ I_TX_i, a kernel ν from U:=∏i∈IUXiU:= _i∈ I_UX_i to W:=∏i∈IWXiW:= _i∈ I_WX_i, and a mapping w:IU→IS∪ITw:I_U→ I_S∪ I_T such that Xw⁡(iu)⊂XiuX_w(i_u)⊂ X_i_u, define a kernel μ⊗wνμ _wν from S to T×WT× W by ∀s=(xi)i∈IS∈S, ∀(Bi)i∈IT∪IW∈ℬ⁡(T×W): ∀ s=(x_i)_i∈ I_S∈ S, ∀(B_i)_i∈ I_T∪ I_W (T× W): μ⊗wν⁡((xi)i∈IS,(Bi)i∈IT∪IW) μ _wν((x_i)_i∈ I_S,(B_i)_i∈ I_T∪ I_W) := ∫μ⁡(s,( )i∈IT)∫ν⁡((xw⁡(i))i∈IU,( )i∈IW)1B((xi)i∈IT∪IU) . := μ(s,( _i)_i∈ I_T) ν((x_w(i))_i∈ I_U,( _i)_i∈ I_W)1_B((x_i)_i∈ I_T∪ I_U) . Given a set μnn∈\μ^n\_n of kernels indexed by a finite totally ordered (by ≤) set N ( =1,…,nN=\1,…,n\ to simplify notation), from ∏i∈InXi _i∈ I^nX_i to XnX_n (for finite InI^n), a finite set ISI_S (and S:=∏i∈ISXiS:= _i∈ I_SX_i) and wn:In→IS∪m∈|m<nw^n:I^n→ I_S∪\m |m<n\, such that Xwn(i)⊂XiX_w_n(i)⊂ X_i then there is a kernel ⊗n∈w∗μn := μ~1⊗w2μ2⊗w3…⊗wnμn ∏n∈Xn , _n ^w^*μ^n := μ^1 _w^2μ^2 _w^3… _w^nμ^n fromSto _n X_n , where μ~1(s=(xi)i∈I1,B):=μ1((xw1(i))i∈I1,B) μ^1(s=(x_i)_i∈ I^1,B):=μ^1((x_w^1(i))_i∈ I^1,B). If one or multiple of the μkμ^k have no arguments (are measures on the one-element set ∗\*\), replace "⊗wk _w^k" by "⊗ ", if μ1μ^1 has no arguments, then μ~1=μ1 μ^1=μ^1. Notation G.25 (Causal Wiring). In explicit computations, we denote kernels by capital roman letters, . If the same letter appears as a lowercase roman letter in an index, YxY_x, then this argument is set to the integration-variable taking the value of their capitalized version, mediator plus direct effect (a) vs. a chain (mediator only; b) then read: (a) (a) [X⊗Mx⊗Yx,m](B) [X M_x Y_x,m](B) =∫X⁡( )∫Mx( )∫Yx,m( )1B(x,m,y) = X( ) M_x( ) Y_x,m( )1_B(x,m,y) (b) (b) [X⊗Mx⊗Ym](B) [X M_x Y_m](B) =∫X⁡( )∫Mx( )∫Ym 1B(x,m,y) . = X( ) M_x( ) Y_m 1_B(x,m,y) . For readability, we repeat inputs that come from "outside" of a bracket around kernels [X⊗Mx]⊗Yx,m=X⊗[Mx⊗Yx,m]x[X M_x] Y_x,m=X [M_x Y_x,m]_x and put composed upon arguments in square brackets (m↦[m]m [m]) (Yx,[m]∘Mx)x .(Y_x,[m] M_x)_x . In light of Lemma G.18i (see also the warning on Lemma. G.13), we will say for kernels Xs,tX_s,t and Yx,s,uY_x,s,u that "overlap" in at least one composed with argument (here: s), that their product is confounded (see example G.26) and change the notation of the brackets to ⟦Xs,t⊗Yx,s,u⟧[s],t,u∘S X_s,t Y_x,s,u _[s],t,u S Note that, while this seems to occur less rarely, similar problems arise with "confounded" compositions (which are defined through products), and we adopt a similar notation: (⦇Yx,[m]∘Mx⦈[x]∘X)(B) ( Y_x,[m] M_x _[x] X )(B) =[X⊗[Mx⊗Yx,m]x](X×M×B) =[X [M_x Y_x,m]_x](X× M× B) =(⟦Mx⊗Yx,m⟧[x]∘X)(M×B) = ( M_x Y_x,m _[x] X )(M× B) Example G.26. For a causal graph X←Z→YX← Z→ Y, where X X and Y Y are confounded by Z Z, we have (see example G.9) kernels Z, XzX_z and YzY_z, thus for example: [Xz⊗Yz]z and⟦Xz⊗Yz⟧[z]∘Z[X_z Y_z]_z and X_z Y_z _[z] Z Note, that composition by definition is taking the joint distribution Z⊗Xz⊗YzZ X_z Y_z (an observed confounder) and integrates out Z (producing "hidden" confounding) ⟦Xz⊗Yz⟧[z]∘Z X_z Y_z _[z] Z, while re-associating as Z⊗[Xz⊗Yx,z]zZ [X_z Y_x,z]_z makes the conditioned (on the observed confounder) joint P⁡(X,Y|Z)P( X, Y| Z) given by [Xz⊗Yz]z[X_z Y_z]_z explicit (but retains information of on Z Z). We can, from the distribution of Z Z and the conditioned joint distribution [Xz⊗Yz]z[X_z Y_z]_z reproduce the full Z⊗Xz⊗Yx,zZ X_z Y_x,z (by associativity), but from ⟦Xz⊗Yz⟧[z]∘Z X_z Y_z _[z] Z we can neither decompose (deconvolute) in the sense of undoing the composition with Z (thus cannot get back [Xz⊗Yx,z]z[X_z Y_x,z]_z) nor can we decompose (disintegrate) in the sense of obtaining factors to get XzX_z, YzY_z or Y[z]∘ZY_[z] Z (only X[z]∘ZX_[z] Z by marginalizing from the right). Lemma G.27 (Causality Mandated Transformations). Sparsity enables further operations, we write s, t etc. for tuples of arguments, and summarize combined arguments as s∪ts∪ t and shared arguments as s∩ts∩ t in the evident way, similarly X and Y are allowed to be products of kernels (in the sense of having distinguishable factors that might partially overlap s or t): [label=()] 1. Products without parent–child relation (Y has no index ’x’) commute up to transposition: Ift∩x=∅=s∩ythen [Xs⊗Yt]s∪t=[Yt⊗Xs]s∪t Ift∩ x= =s∩ ythen [X_s Y_t]_s∪ t=[Y_t X_s]_s∪ t^t 2. Compositions without parent–child relation are trivial: Ifx∩t=∅,then (Yt∘Xs)s∪t=Yt Ifx∩ t= ,then (Y_t X_s)_s∪ t=Y_t 3. Composition right-distributes over unconfounded products (cf. (i)): Ift∩z=∅,then [Xs⊗Yt]s∪(t∖x)∘Zu If$t∩ z= $,then [X_s Y_t]_s∪(t x) Z_u = [(Xs∘Zu)(s∖z)∪u⊗Yt](s∖z)∪u∪(t∖x) = [(X_s Z_u)_(s z)∪ u Y_t]_(s z)∪ u∪(t x) Ifs∩z=∅,then [Xs⊗Yt]s∪(t∖x)∘Zu If$s∩ z= $,then [X_s Y_t]_s∪(t x) Z_u = [Xs⊗(Yt∘Zu)(t∖z)∪u](t∖z∖x)∪u∪s = [X_s (Y_t Z_u)_(t z)∪ u]_(t z x)∪ u∪ s 4. Anti-causal disintegrations carry arguments and entries of the joint distribution excluding visible terms: ForXs,Yt: (Xs|Yt)y∪s∪(t∖x) . ForX_s,Y_t: (X_s|Y_t)_y∪ s∪(t x) . proof Part 1: Note that the argument [Xs⊗t]s∪(t∖x)[X_s _t]_s∪(t x) by t∩x=∅t∩ x= is indeed s∪(t∖x)=s∪ts∪(t x)=s∪ t. By definition of the ⊗ -product (Def. G.8), using that B=BX×BYB=B_X× B_Y form a basis of ℬ⁡(X)⊗ℬ⁡(Y)B(X) (Y): ∀BX×BY∈ℬ⁡(X)⊗ℬ⁡(Y) ∀ B_X× B_Y (X) (Y) : : [Xs⊗Yt](s∪t,BX×BY) [X_s Y_t](s∪ t,B_X× B_Y) =∫X⁡(s, )∫Y⁡(t, )1BX×BY(x,y) = X(s, ) Y(t, )1_B_X× B_Y(x,y) =∫X⁡(s, )1BX(x)∫Y⁡(t, )1BY(y) = X(s, )1_B_X(x) Y(t, )1_B_Y(y) byt∩x=∅ by$t∩ x= $ =X⁡(s,BX)Y(t,BY) =X(s,B_X)Y(t,B_Y) .of∫ def.\ of$ $ The problem is symmetric under exchange of X↔YX Y, thus the same computation shows [Yt⊗Xs](s∪t,BY×BX)=[Yt⊗Xs]t(s∪t,BX×BY)=X⁡(s,BX)Y(t,BY)[Y_t X_s](s∪ t,B_Y× B_X)=[Y_t X_s]^t(s∪ t,B_X× B_Y)=X(s,B_X)Y(t,B_Y). Part 2: By definition of composition (Def. G.11): ∀BY∈ℬ⁡(Y) ∀ B_Y (Y) : : Yt∘Xs(s∪t,BY) Y_t X_s(s∪ t,B_Y) =∫X⁡(s, )∫Y⁡(t, )1X×BY(x,y) = X(s, ) Y(t, )1_X× B_Y(x,y) =∫Y⁡(t, )1BY(y)∫X⁡(s, ) = Y(t, )1_B_Y(y) X(s, ) byt∩x=∅ by$t∩ x= $ =Y⁡(t,BY) =Y(t,B_Y) ∫X=1;def.of∫ $ X=1$;def.\ of$ $ Part 3: By definition of composition (Def. G.11), using that B=BX×BYB=B_X× B_Y form a basis of ℬ⁡(X)⊗ℬ⁡(Y)B(X) (Y) for the first case (t∩z=∅t∩ z= ): ∀BX×BY∈ℬ⁡(X)⊗ℬ⁡(Y): ∀ B_X× B_Y (X) (Y): [Xs⊗Yt]∘Zu((s∖z)∪(t∖x),BX×BY) [X_s Y_t] Z_u((s z)∪(t x),B_X× B_Y) =∫Z⁡(u, )∫X⁡(s, )∫Y⁡(t, )1Z×BX×BY(z,x,y) = Z(u, ) X(s, ) Y(t, )1_Z× B_X× B_Y(z,x,y) =∫Z⁡(u, )∫X⁡(s, )1BX(x)∫Y⁡(t, )1BY(y) = Z(u, ) X(s, )1_B_X(x) Y(t, )1_B_Y(y) Also by definition of composition (Def. G.11) and definition of integrals: ∀BX∈ℬ⁡(X) ∀ B_X (X) ∫(Xs∘Zu)((s∖u)∪t, )1BX(x) (X_s Z_u)((s u)∪ t, )1_B_X(x) =Xs∘Zu((s∖u)∪t,BX) =X_s Z_u((s u)∪ t,B_X) =∫Z⁡(u, )∫X⁡(s, )1Z×BX(z,x) = Z(u, ) X(s, )1_Z× B_X(z,x) =∫Z⁡(u, )∫X⁡(s, )1BX(x) = Z(u, ) X(s, )1_B_X(x) By definition of integrals, for any measurable mapping f:X→[0,1]f:X→[0,1] thus: ∫(Xs∘Zu)((s∖u)∪t, )f(x)=∫Z⁡(u, )∫X⁡(s, )f(x) (X_s Z_u)((s u)∪ t, )f(x)= Z(u, ) X(s, )f(x) The choice ft(x):=1BX(x)∫Y⁡(t, )1BY(y)f_t(x):=1_B_X(x) Y(t, )1_B_Y(y) is valid for any fixed t and we can fix a t, because by hypothesis, t∩z=∅t∩ z= , so plugging this integral-operator into the previous equation we get for all t: ∀BX×BY∈ℬ⁡(X)⊗ℬ⁡(Y): ∀ B_X× B_Y (X) (Y): [Xs⊗Yt]∘Zu((s∖z)∪(t∖x),BX×BY) [X_s Y_t] Z_u((s z)∪(t x),B_X× B_Y) =∫(Xs∘Zu)((s∖u)∪t, )1BX(x)∫Y⁡(t, )1BY(y) = (X_s Z_u)((s u)∪ t, )1_B_X(x) Y(t, )1_B_Y(y) =[(Xs∘Zu)⊗Yt]((s∖z)∪u∪(t∖x),BX×BY) = [(X_s Z_u) Y_t ] ((s z)∪ u∪(t x),B_X× B_Y ) The second case works similar (but is actually easier, because we can move the integral over Z inside the integral over X). Part 4: The joint distribution [Xs⊗Yt]s∪(t∖x)t[X_s Y_t]^t_s∪(t x) carries arguments s∪(t∖x)s∪(t x) by construction (Def. G.8). Disintegration adds an argument y for Yt∘XsY_t X_s by definition (Def. G.14), so in general (Xs|Yt)y∪s∪(t∖x)(X_s|Y_t)_y∪ s∪(t x). Remark: We only need (in the proof of Lemma A.17) that (Xs|Yt)(X_s|Y_t) does not have more than these arguments. Considering that these are basically all plausible arguments that could be added this is result is not actually surprising. It is, however, in general not possible to remove any of these arguments, which further emphasizes that anti-causal (as opposed to causal-direction) disintegrations are really properties of the joint distribution and in general as complicated as joint distributions from the causal perspective. G.5 Symmetries As the attentive reader may have noticed, we do not really rely on symmetries described as group-actions, rather on the orbit-sets of these group-actions. Indeed we could easily reformulate our formalism as follows. Change Def. 2.5 to: Definition G.28 (Mechanism, Alternative Version). A mechanism is a probability kernel f from X =∏k=1κX(k)X = _k=1^κX^(k) (with κ arguments) to Y (both standard Borel, Ass. 2.1) together with a region of applicability J⊂IJ⊂ I, and a mapping :J→Iκ :J→ I^κ of relative parents such that (no sub-group or equivariance-condition): Denoting the kkth parent by k k, require ∀k: k(j)≠j∀ k: k(j)≠ j and k≠k′k≠ k ⇒ ∀j∈J∀ j∈ J: k(j)≠ k′(j) k(j)≠ k (j). Additionally: ∀k k∀ k\> k is injective. Finally, ∀j∈J∀ j∈ J, Y⊂XjY⊂ X_j and ∀k∀ k: X k(j)⊂X(k)X_ k(j)⊂ X^(k). Remark G.29. A mechanism in the sense of Def. 2.5 is a mechanism in the sense of Def. G.28. The additional condition immediately restores freeness of the direct embedding (Lemma F.1), so if we delete the equivariance ("rigitidty", 2) in Def. 4.2: Definition G.30 (Families of Embeddings, Alternative Version). A family of local graph embeddings (J0,ψjj∈J0,,y0)(J_0,\ _j\_j∈ J_0,G,y_0) is a collection of local graph embeddings ψj:↪I _j:G I of a single model-aligned local graph G. Further there is a fixed element y0∈y_0 , which will be called the anchor(-node) and a range of applicability J0⊂IJ_0⊂ I. For n∈n denote: ψ∗(n):J0→I, j↦ψj(n) . _*(n):J_0→ I, j _j(n) . Finally, we will require the following conditions to be satisfied: [label=()] 1. Trivial on Anchor: ∀j∈J0∀ j∈ J_0: ψj(y0)=j _j(y_0)=j. 2. (no rigidity condition) 3. Freeness: ∀n∈ proper∀ n _ proper: ψ∗(n) _*(n) is injective. This actually leaves us in a place where Thm. 1 still applies! Indeed the proof of Thm. 1, while painstakingly going through all the properties of the definitions, does never reference rigidity. So everything else works exactely as before. Motivation: Considering Rmk. G.29 this alternative formulation is strictly more general, while at the same time producing the same output. There are, however very good reasons for an approach by group-actions. • Group-actions are the standard approach to symmetries in physics and other natural sciences. Thinking about a model for example as time-translation invariant (or λ-periodic etc.) is much more intuitive than thinking about symmetries in terms of their induced orbit-spaces. Also, the gain in generality only matters, if an application would specify a symmetry that is not induced by Rmk. G.29. • Group-actions allow for an elegant structure. For example observations as in Rmk. 4.3 are intuitive and immediate in the language of group-theory. More generally, having a sub-group hierarchy paralleling constructions has been of great relevance in many topics, from (classical) Galois-theory to classifications of covering-maps. • Group-actions allow for efficient encoding. Going forward, at some point, we will certainly want to find models from data in some form of structure-discovery (§F.3). Clearly searching all possible structures is not a viable option. Here group-actions should help to efficiently encode both models and assumptions about priors over models. • This paper is targeted at an audience in machine-learning and applied sciences, and for that purpose is already on the rather abstract side. Making it yet more abstract without substantial benefit seems unwise. The pure mathematician reading this is of course invited to investigate simpler and more abstract ideas. Contents 1 Introduction 1.1 Need for Abstraction 1.2 Employed Structure 1.3 Contributions and Content 1.4 Related Literature 2 Models and Observations 2.1 Preliminaries 2.2 Model 2.3 Shallow Distribution and Realization 2.4 Observedness and Asymptotics 3 Structured Kernels 3.1 Preliminaries 3.2 Causality and Sparsity 3.3 Graphical Operations 4 Extraction from Data 4.1 Families of Embeddings 4.2 Decorated Families 4.3 Notions of Identifiability 4.4 Identifiability from Embeddings 4.5 Summary: Extracted Knowledge 4.6 Extension: Multi-Level Statistics 5 Queries and Prediction 5.1 Basic Queries 5.2 Structured Queries 5.3 Knowledge-Closures and Identification 5.4 Relation to the IID-Case 6 Conclusion References Reading Guide A Details on Structured Kernels A.1 Structural Graphs A.2 Regularity A.3 Graphical Operations A.4 Algorithmic Construction A.5 Revealing Operations B Details on Models and Observations B.1 Results in the Main Text B.2 Properties of Probabilistic Worlds C Details on the Extraction of Structured Kernels C.1 Multi-Level Modifications C.2 Embedding-Families and Attached Structure C.3 Identifiability C.4 Identification from Families of Embeddings C.5 Algorithmic Construction of Backdoor-Free Families D Details on Query Identification D.1 Multi-Level Modifications D.2 Structured Representation of Basic Queries D.3 Identification of Queries D.4 Beyond Basic Queries D.5 Examples for Multi-Level Queries E The IID Case E.1 Standard Results for the Single-Context Case E.2 Translating the Standard IID-Setup E.3 Single-Context with Empty Conditioning Set E.4 Single-Context with Non-Empty Conditioning Set E.5 Standard Results for the Multi-Context, Experimental Case E.6 Translating the Transportability Setup E.7 Multi-Context and Transportability E.8 Mediation and Counterfactuals E.9 Conclusion F Future Work F.1 Statistical Considerations F.2 Selection Bias and Correlated Missingness F.3 Structure Discovery F.4 Algorithmic Approach G Mathematical Basics and Notation G.1 Measure-Theory G.2 Kernels and Basic Properties G.3 Order Independent Operators G.4 Sparsity and Contraction G.5 Symmetries \@add@PDF@RDFa@triples

Instant research discovery