Paper deep dive
Embedded Conditional Independence Tests for Large Language Model Generated Text with an Application to German Parliament Speeches
Marco Simnacher, Georg Keilbar, Benjamin König, Christoph Lippert, Sonja Greven
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/5/2026, 3:46:54 AM
Summary
This paper introduces embedded Conditional Independence Tests (eCITs) to detect bias in Large Language Model (LLM) generated text. The authors propose embedding both the source text (Z) and the LLM output (X) into feature representations to apply existing CITs, which are typically limited to low-dimensional data. They theoretically justify that if the embedding of the source text is sufficient, the null hypothesis of conditional independence transfers from the raw texts to their representations. The method is applied to German Parliament speeches, revealing that LLM summaries contain information about the speaker's faction and gender beyond what is present in the original speech, indicating potential bias.
Entities (6)
Relation Signals (4)
Embedded Conditional Independence Tests → appliedto → German Parliament Speeches
confidence 95% · Applying the eCITs to German Parliament speeches, we find for all combinations of embedding maps considered that the summaries of two LLMs contain information about the speaker’s faction and gender beyond the speech they were generated from.
Embedded Conditional Independence Tests → detects → Bias
confidence 95% · We define bias as additional information in the LLM output X about the protected attribute Y beyond the information already contained in the source text Z. This directly corresponds to the alternative hypothesis of a CIT, so bias can be tested...
Large Language Models → generates → summaries
confidence 90% · summaries of two LLMs contain information about the speaker’s faction and gender beyond the speech they were generated from.
Embedded Conditional Independence Tests → uses → Projected Covariance Measure
confidence 85% · we adapt the projected covariance measure (Lundborg et al., 2024, PCM;), the conditional mean independence test (CMIT) we apply
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Conditional independence tests (CITs) test for conditional dependence between two random objects $X$ and $Y$ given a third random object $Z$. Existing CITs have limited applicability to high-dimensional data, especially multimodal data like text. However, we show that such tests are of interest for large language model (LLM) outputs, where we test whether an output $X$ generated from a source text $Z$ carries information about an attribute $Y$ beyond $Z$ itself. For this purpose, we propose embedded CITs (eCITs), which embed $X$ and $Z$ and apply an existing CIT to the resulting representations and to $Y$. We show that, provided the embedding of $Z$ is sufficient, i.e. retains the information $Z$ carries about either $Y$ or the representation of $X$, the null hypothesis transfers from $X$ and $Z$ to their representations, so that a CIT valid for the embedded hypothesis is valid for the original one. We further give conditions for equivalence of the two hypotheses, and show that sufficiency weakens to mean sufficiency when the embedded test targets conditional mean independence. We propose a semi-synthetic simulation design to assess type I error (T1E) control and power of the eCITs for given embedding maps on a specific dataset and task, and use it to evaluate them on our application. Applying the eCITs to German Parliament speeches, we find for all combinations of embedding maps considered that the summaries of two LLMs contain information about the speaker's faction and gender beyond the speech they were generated from.
Tags
Links
- Source: https://arxiv.org/abs/2609.00946v1
- Canonical: https://arxiv.org/abs/2609.00946v1
Trouble viewing inline? Open PDF directly →
Full Text
133,786 characters extracted from source content.
Expand or collapse full text
Marco Simnacher, Georg Keilbar, Benjamin König, Christoph Lippert, Sonja Greven Embedded Conditional Independence Tests for Large Language Model Generated Text with an Application to German Parliament Speeches Marco Simnacher marco.simnacher@hu-berlin.de Affiliation: Georg Keilbar georg.keilbar@hu-berlin.de Affiliation: Benjamin König b.koenig@hu-berlin.de Affiliation: Chair of Statistics Affiliation: Humboldt-Universität zu Berlin Affiliation: Spandauer Str. 1, 10178 Berlin, Germany Christoph Lippert Christoph.Lippert@hpi.de Affiliation: Chair Digital Health & Machine Learning Affiliation: Hasso-Plattner-Institut for Digital Engineering Affiliation: Prof.-Dr.-Helmert-Straße 2-3, 14482 Potsdam, Germany Sonja Greven sonja.greven@hu-berlin.de Affiliation: Chair of Statistics Affiliation: Humboldt-Universität zu Berlin Affiliation: Spandauer Str. 1, 10178 Berlin, Germany Abstract Conditional independence tests (CITs) test for conditional dependence between two random objects X and Y given a third random object Z. Existing CITs have limited applicability to high-dimensional data, especially multimodal data like text. However, we show that such tests are of interest for large language model (LLM) outputs, where we test whether an output X generated from a source text Z carries information about an attribute Y beyond Z itself. For this purpose, we propose embedded CITs (eCITs), which embed X and Z and apply an existing CIT to the resulting representations and to Y. We show that, provided the embedding of Z is sufficient, i.e. retains the information Z carries about either Y or the representation of X, the null hypothesis transfers from X and Z to their representations, so that a CIT valid for the embedded hypothesis is valid for the original one. We further give conditions for equivalence of the two hypotheses, and show that sufficiency weakens to mean sufficiency when the embedded test targets conditional mean independence. We propose a semi-synthetic simulation design to assess type I error (T1E) control and power of the eCITs for given embedding maps on a specific dataset and task, and use it to evaluate them on our application. Applying the eCITs to German Parliament speeches, we find for all combinations of embedding maps considered that the summaries of two LLMs contain information about the speaker’s faction and gender beyond the speech they were generated from. †heading: 23 2026 1- 1/21; Revised 5/22 9/22 21-0000†shortheadings: CI Testing for LLMs / Simnacher, Keilbar, König, Lippert, Greven†firstpage: 1†editor: My editor keywords conditional independence testing, embedding maps, sufficient representation, large language model 1 Introduction A conditional independence test (CIT) is a statistical test for the null hypothesis of conditional independence (CI), H0:X⟂Y|ZH_0:X \!\!\! Y|Z, between two random variables, X and Y, given a third variable, Z, against the alternative hypothesis of conditional dependence, H1:X⟂Y|ZH_1:X \! \!\!\! Y|Z. X and Y are conditionally dependent, given Z, if X (or Y) provides additional information about Y (or X), given Z. A CIT should control the type I error (T1E), i.e. the probability of falsely rejecting the null hypothesis, while simultaneously achieving high power, i.e. a high probability of correctly rejecting the null hypothesis of CI. One setting in which such conditional independencies are of interest is the evaluation of large language models (LLMs). Consider a source text Z, e.g. a parliament speech, observed together with a protected attribute Y, e.g. the speaker’s gender or faction, and an LLM-generated summary X of the source text. Such summaries can contain biases regarding the protected attribute, framing e.g. the speeches of one faction more critically than those of another, although the content of the speeches does not differ. Due to the widespread use of LLMs across different tasks, such biases are widely studied (Gallegos et al., 2024). However, the complexity of LLMs and their task-specific behavior make general statements about the bias of a specific LLM challenging (Anthis et al., 2024), and existing bias metrics are of limited use in this setting. Intrinsic metrics based on embeddings or token probabilities often do not correlate with the bias observed in downstream tasks (Goldfarb-Tarrant et al., 2020; Delobelle et al., 2022). Extrinsic metrics are typically bound to benchmark datasets derived from sentence completion or occupation masking, so that their results need not translate to domain-specific tasks (Li et al., 2023). LLM-based evaluators are applicable to arbitrary tasks, but can themselves be biased (Sun et al., 2022; Gao et al., 2025), and their scores do not come with statistical guarantees. We instead define bias as additional information in the LLM output X about the protected attribute Y beyond the information already contained in the source text Z. This directly corresponds to the alternative hypothesis of a CIT, so bias can be tested on a given task and dataset with a T1E guarantee. Since the summary is generated solely from the source text, any such additional information cannot come from the observed input itself, but must have entered through the model, for example via memorization of the source texts through the training data the model was fitted on. The definition is reference-free in the sense that no gold-standard output or counterfactually generated output has to be constructed, since the source text itself supplies the reference. While we focus on summarization as one instance, the definition applies to any task in which the output is generated from a source text. In our empirical application with a categorical variable Y, the definition is a conditional version of the partisanship measure of Gentzkow et al. (2019), who quantify how strongly congressional speech reveals a speaker’s faction. We ask the same of the summaries, treating the speech as the conditioning variable: whether the summary reveals the speaker’s faction beyond the speech it was generated from. Testing this hypothesis, however, is not straightforward when X and Z are texts. Existing CITs focus on low-dimensional, vector-valued variables (Li and Fan, 2020), and both Z and X cause problems through their structure and high-dimensionality. The structure of Z affects the T1E of the CITs. Their null hypothesis comparisons require an object constructed from Z: the conditional distribution of X or of Y given Z for resampling-based comparisons (Candes et al., 2018; Berrett et al., 2020), a kernel or a partition on the support of Z for kernel- and neighborhood-based comparisons (Zhang et al., 2011; Strobl et al., 2019), or a regression on Z for residual-based comparisons (Shah and Peters, 2020; Lundborg et al., 2024). None of these objects can be constructed on raw texts with the guarantees the tests require, whether because the construction itself is unavailable or because the rate conditions on which validity rests are not. The tests thus become applicable only on a feature representation of Z and condition on that representation rather than on Z, as in the application of algorithm-agnostic significance tests to multimodal data (Kook and Lundborg, 2024). The text structure of X affects both whether a CIT can be applied at all and, once it can, its power. Simnacher et al. (2026) address this for structured, high-dimensional X by embedding it onto a feature representation and applying a CIT to the representation instead. Provided the embedding map is learned as they prescribe, e.g. on external data or through sample splitting, this leaves the T1E unaffected in their setting, where Z is not embedded, and the choice of representation affects only the power, which depends on whether the representation retains the conditional association between X and Y given Z. The representation, however, remains high-dimensional, so that only a few CITs are applicable to it. Neither line of work addresses the representation of Z, on which the T1E of the resulting test depends. We therefore require CITs that are applicable to texts X and Z, control the T1E for the hypothesis stated on the level of the texts, and have power against relevant alternatives. To meet these requirements, we propose embedded CITs (eCITs), which map both Z and X onto feature representations and apply an existing CIT for conditional dependence of the representation of X and Y, given the representation of Z, making the test applicable to structured, high-dimensional inputs. Since the CIT conditions on the representation of Z rather than on the source text, it is enough that that representation retain the information in Z about either Y or the representation of X. Any information it drops plays the role of an omitted conditioning variable and can inflate the T1E. We call embedding maps ωZ _Z yielding such a representation sufficient and characterize them in Subsection 3.1. The representation of X needs no sufficiency condition of its own and mainly affects the power, since the test has power only if it retains the conditional association between X and Y given Z in a form the CIT can detect. The two choices nevertheless interact, since the sufficiency of the embedding of Z is stated relative to the representation of X, unlike in Simnacher et al. (2026), where the conditioning variable is not embedded and the choice of representation affects the power alone. We discuss this further in Subsection 3.1 and Section 4. The CIT contributes its guarantees at the level of the representations, so whether they transfer to the texts depends on the embedding maps and the dataset at hand. Our contributions to the existing literature are as follows. First, we justify CI testing as a bias criterion for LLM outputs and discuss when the null and the alternative can hold for a fixed LLM (Section 2). Second, we propose eCITs, which combine embedding maps for X and Z with existing CITs, making CI testing applicable when both X and Z are texts (Section 3). Third, we characterize sufficient embedding maps for Z, under which the null hypothesis stated on the texts transfers to the null hypothesis stated on the representations, so that a valid CIT for the embedded hypothesis controls the T1E for the textual one. We also give conditions under which the two hypotheses are equivalent, and show that the requirement weakens to mean sufficiency when the embedded test targets conditional mean independence (Subsection 3.1). Fourth, we adapt the projected covariance measure (Lundborg et al., 2024, PCM;), the conditional mean independence test (CMIT) we apply, to the categorical Y of our application, with a precision-weighted projection direction that we show to be optimal, and record the conditions under which the validity argument of Lundborg et al. (2024) carries over (Appendix D). Fifth, we show empirically how insufficient embeddings of Z inflate the T1E, and provide a simulation design that can be used to assess the eCITs on a given dataset before applying them (Subsection 4.2). On speech-summary pairs, eCITs with a sufficient embedding of Z are calibrated at all sample sizes considered for the categorical outcomes, also for high-dimensional representations, while for insufficient ones the inflation ranges from negligible to growing steeply with the sample size, depending on how much of the dropped information the remaining embedding maps recover. Finally, we apply the eCITs to German Parliament speeches and find, for both summarizers and all combinations of embedding maps considered, that the summaries of two LLMs contain information about the speaker’s faction and gender beyond the speech (Subsection 4.3). 2 Conditional (In)dependence for LLM-Generated Summaries Along with a protected scalar variable Y and a source text Z, we generate LLM outputs X from the source text Z. We define bias as additional information in the output X about a protected attribute Y given the source text Z. Specifically, we test for bias in an output by testing the hypothesis H0:Y⟂X|Zvs.H1:Y⟂X|Z. H_0:Y \!\!\! X Z . H_1:Y \! \!\!\! X Z. (1) Under the null, the output does not add any information about the protected attribute beyond the information already available in the source text. Under the alternative, the output provides additional information about the protected attribute that was not available in the source. We distinguish between the text corpus DLLMD_LLM used to train the LLM and the test dataset DCIT=(Xi,Yi,Zi)i=1nD_CIT=(X_i,Y_i,Z_i)_i=1^n used in the CIT. To generate XiX_i, we prompt the LLM with a task constant across all i and a source text ZiZ_i, cf. Subsection 4.1 and Appendix B.1 for details. If the text corpus DLLMD_LLM and (Z,Y)∈DCIT(Z,Y)∈ D_CIT are independent, we are trivially under the null hypothesis in (1), since the output X is a measurable function of Z, DLLMD_LLM and, under stochastic decoding, of sampling noise drawn independently of (Y,Z)(Y,Z), and hence conditionally independent of Y given Z. If Z,YZ,Y or texts Z~ Z with the same Y are contained in the text corpus, then the null or alternative hypothesis can be true. A plausible mechanism for such a conditional dependence is memorization. This is also the situation we expect in our application in Section 4, where the source texts are German Parliament speeches and the two LLMs used to summarize the speeches are pre-trained on web-scale German corpora that plausibly include Bundestag plenary protocols. If the LLM has seen a source text ZiZ_i, or other texts by the same speaker, during training, its weights can carry information about the associated attribute YiY_i that ZiZ_i alone does not determine. When generating XiX_i from ZiZ_i, the model may then reproduce phrasing or framing tied to that particular observation rather than to the content of ZiZ_i, so that X reveals Y beyond Z. The following example isolates this mechanism in a setting in which the null and the alternative can be constructed by design. Figure 1: Rejection rates of the PCM testing conditional mean independence of X and Y given Z with default parameters applied to data generated with Z∼N(010,I10)Z N(0_10,I_10), Yi=Zi,1+εi,εi∼iidN(0,1)Y_i=Z_i,1+ _i, _i iid N(0,1), and Xi=fθ^(Zi)X_i=f_ θ(Z_i), where fθ^f_ θ is a knn model predicting Y from Z with k=2,5,10k=2,5,10. Under the null, the training dataset Dtr=(Zi,Yi)i=1ND_tr=(Z_i,Y_i)_i=1^N is independent of the test dataset Dtest=(Xi,Yi,Zi)i=1nD_test=(X_i,Y_i,Z_i)_i=1^n, under the alternative Dtest⊂DtrD_test⊂ D_tr. Example 1. We generate Dtr=(Zi,Yi)i=1N,D_tr=(Z_i,Y_i)_i=1^N, with independent copies Zi∼N(010,I10)Z_i N(0_10,I_10) and Yi=Zi,1+εi,εi∼iidN(0,1)Y_i=Z_i,1+ _i,\ _i iid N(0,1). We fit a k-nearest neighbor model fθf_θ on Dtr=(Zi,Yi)D_tr=(Z_i,Y_i), and afterwards fix the model fθ^f_ θ. Then, we predict YiY_i from ZiZ_i using (Zi,Yi)i=1n⊂Dtest(Z_i,Y_i)_i=1^n⊂ D_test. Under the null hypothesis, (Zi,Yi)i=1n(Z_i,Y_i)_i=1^n from the test dataset Dtest=(Xi,Yi,Zi)i=1nD_test=(X_i,Y_i,Z_i)_i=1^n is sampled independently of DtrD_tr, Xi=fθ^(Zi)X_i=f_ θ(Z_i) are out-of-sample predictions, and thus, X⟂Y|ZX \!\!\! Y Z holds on DtestD_test. Under the alternative, Dtest⊂DtrD_test⊂ D_tr, the predictions Xi=fθ^(Zi)X_i=f_ θ(Z_i) are in-sample, and thus, X⟂Y|ZX \! \!\!\! Y Z holds on DtestD_test. Figure 1 shows the rejection rates of the PCM with default parameters testing for conditional dependence for three k-nearest neighbor (knn) models (k=2,5,10) under the null and alternative. Under the null, the T1E of the PCM is within one Monte Carlo standard error of the nominal level of 0.05. The power under the alternative depends on the number of nearest neighbors. For a specific YiY_i, this YiY_i contributes more to the model’s predictions for a model with smaller k, thus leading to larger conditional dependence and higher rejection rates. In the example, DtrD_tr and DtestD_test mimic DLLMD_LLM and DCITD_CIT. We observe that a model with a stronger memorization of individual observations leads to stronger conditional dependence, detectable by the PCM. 3 Embedded Conditional Independence Tests Let (,ℱ),(,ℱ),(,ℱ)(X,F_X),(Y,F_Y),(Z,F_Z) be measurable spaces and X,Y,ZX,Y,Z be (,ℱ)(X,F_X)-, (,ℱ)(Y,F_Y)-, (,ℱ)(Z,F_Z)-valued random variables with joint distribution ℙX,Y,ZP^X,Y,Z. Moreover, let β β and γ γ be (ℬ,ℱℬ)(B,F_B)- and (,ℱ)(C,F_C)-valued random variables representing estimated parameters of embedding maps ωX:×ℬ→,(X,β^)↦ωX(X,β^) _X:X×B→ X,(X, β) _X(X, β) and ωZ:×→ℨ,(Z,γ^)↦ωZ(Z,γ^) _Z:Z×C→ Z,(Z, γ) _Z(Z, γ). We denote the resulting random representations by XωX=ωX(X,β^)X _X= _X(X, β) and ZωZ=ωZ(Z,γ^)Z _Z= _Z(Z, γ), which are (,ℱ)( X,F_ X)- and (ℨ,ℱℨ)( Z,F_ Z)-valued random variables. The focus of this paper is on X and Z as spaces of text, ⊆ℝY or =1,…,LY=\1,…,L\ as a univariate continuous or categorical variable, and ⊆ℝq X ^q and ℨ⊆ℝp Z ^p as the spaces of feature representations. Nevertheless, the theoretical results apply also to CITs applicable to vector-valued Y, as well as general X and Z for which representations can be obtained. The notation and the assumptions on β β follow Simnacher et al. (2026), where Z is not embedded; we extend the setup by embedding Z and by conditioning on γ γ. We assume that (Xi,Yi,Zi),i=1,…,n(X_i,Y_i,Z_i),i=1,…,n are n independently and identically distributed (i.i.d.) copies of (X,Y,Z)(X,Y,Z). We define XiωX=ωX(Xi,β^)X _X_i= _X(X_i, β), and write Xn=(X1,…,Xn)X^n=(X_1,…,X_n), (X,Y)n=(Xi,Yi)i=1n(X,Y)^n=(X_i,Y_i)_i=1^n, and analogously for all other combinations of XiωX,X_i _X, ZiωZ_i _Z, XiX_i, Yi,Y_i, Zi,Z_i, i=1,…,ni=1,…,n. For the whole sample with text and their feature representations, we write S=(X,Y,Z)nS=(X,Y,Z)^n and Sω=(XωX,Y,ZωZ)nS^ω=(X _X,Y,Z _Z)^n, SωX=(XωX,Y,Z)nS _X=(X _X,Y,Z)^n, SωZ=(X,Y,ZωZ)nS _Z=(X,Y,Z _Z)^n respectively. Furthermore, let the underlying probability spaces for the samples be (,ℱ,ℙS)(S,F_S,P^S) and analogously (ω,ℱω,ℙSω)(S^ω,F_S^ω,P^S^ω). We call ×ℨ X×Y× Z the embedded observation space, in contrast to the embedded sample space ωXS _X and ωS^ω, and denote the collections of null and alternative distributions of (1) by 0=(ℙX,Y,Z)⊗n:X⟂Y∣ZunderℙX,Y,Z,P_0= \(P^X,Y,Z) n\ :\ X \!\!\! Y Z\ under\ P^X,Y,Z \, and 1=(PX,Y,Z)⊗n:X⟂Y∣ZunderℙX,Y,Z,P_1= \(P^X,Y,Z) n\ :\ X \! \!\!\! Y Z\ under\ P^X,Y,Z \, respectively. On the embedded sample spaces we define the corresponding null classes on ωS^ω and ωXS _X as 0ω ^ω_0 :=Q probability measure on (ω,ℱω):XωX,n⟂Yn∣ZωZ,n under Q, := \Q probability measure on (S^ω,F_S^ω):\ X _X,n \!\!\! Y^n Z _Z,n under Q \, (2) 0ωX _X_0 :=Q probability measure on (ωX,ℱωX):XωX,n⟂Yn∣Zn under Q. := \Q probability measure on (S _X,F_S _X):\ X _X,n \!\!\! Y^n Z^n under Q \. (3) Both classes refer only to the embedded sample space and carry no reference to β β, γ γ or ℙX,Y,ZP^X,Y,Z. Their elements are not required to be product measures: this is needed because the conditional law of SωS^ω given the embedding parameters is in general not a product measure when the parameters are estimated in sample. Both classes depend on n, which we suppress in the notation. Throughout, we assume that (,ℱ)(X,F_X), (,ℱ)(Y,F_Y), (,ℱ)(Z,F_Z) and (,ℱ)( X,F_ X), (ℨ,ℱℨ)( Z,F_ Z) are standard Borel. Since finite products of standard Borel spaces are standard Borel, this carries over to the sample spaces S, ωXS _X and ωS^ω, and guarantees the existence of the regular conditional distributions used below. The assumption is satisfied in our setting: X and Z are the countable sets of finite strings over a finite alphabet equipped with the discrete σ-field, while ⊆ℝY or =1,…,LY=\1,…,L\ and ⊆ℝq X ^q, ℨ⊆ℝp Z ^p carry their Borel σ-fields. No such assumption is placed on the parameter spaces (ℬ,ℱℬ)(B,F_B) and (,ℱ)(C,F_C), which may be arbitrary measurable spaces. To test hypothesis (1), we embed the texts with the embedding maps ωX(⋅,β^) _X(·, β) and ωZ(⋅,γ^) _Z(·, γ), and test instead H0ω:ℙSω|θ^∈0ωℙθ^-a.s.vs.H1ω:ℙθ^(ℙSω|θ^∈0ω)<1,H^ω_0:P^S^ω θ ^ω_0\ P θ-a.s. . H^ω_1:P θ (P^S^ω θ ^ω_0 )<1, (4) where θ^=(β^,γ^) θ=( β, γ) and ℙSω|θ^P^S^ω θ denotes a regular conditional distribution of the embedded sample given the embedding parameters. We propose the eCIT as the measurable function φω:×ℬ×[0,1]→0,1,(s,b,c,u)↦ψ(ω¯(s,b,c),u), _ω:S×B×C×[0,1]→\0,1\, (s,b,c,u) ψ ( ω(s,b,c),u ), (5) where ω¯ ω takes the whole sample and the estimators β^,γ β, γ as input and maps it onto ωS^ω through the embedding maps, so that ω¯(S,β^,γ^)=Sω ω(S, β, γ)=S^ω, and where ψ:ω×[0,1]→0,1ψ:S^ω×[0,1]→\0,1\ is a CIT testing (4), with φω=0 _ω=0 and φω=1 _ω=1 defining decisions for H0H_0 and H1H_1 in (1). The last argument is reserved for a random variable U∼Unif[0,1]U [0,1], independent of (S,β^,γ^)(S, β, γ), which carries any randomization employed by the CIT, such as a sample split (Shah and Peters, 2020, cf. e.g.,). Since a randomized test may always ignore this argument, deterministic CITs are included. We suppress U from the notation, writing φω(S) _ω(S) and ψ(Sω)ψ(S^ω), and let all probabilities of rejection be taken under the joint law of (S,β^,γ^,U)(S, β, γ,U); analogously, Q(ψ=1)Q(ψ=1) abbreviates (Q⊗Unif[0,1])(ψ=1)(Q [0,1])(ψ=1) for Q a probability measure on (ω,ℱω)(S^ω,F_S^ω). The test statistic of the eCIT is defined analogously as Tφω(s,b,c,u)=Tψ(ω¯(s,b,c),u)T_ _ω(s,b,c,u)=T_ψ ( ω(s,b,c),u ) for a test statistic Tψ:ω×[0,1]→ℝT_ψ:S^ω×[0,1] of the CIT ψ. Throughout, the estimation procedure for the embedding parameters is fixed: it is a Markov kernel from (,ℱ)(S,F_S) to (ℬ×,ℱℬ⊗ℱ)(B×C,F_B _C), specified for every s∈s and not depending on ℙSP^S. The parameters remain random, while their conditional distribution given S=sS=s is held fixed across the null class. If θ θ is a measurable function of the sample, as for the in-sample construction of ωZ _Z by regressing XωX,nX _X,n on ZnZ^n, its conditional distribution given S=sS=s places probability one on the corresponding value. If θ θ is instead learned on data independent of S, its conditional distribution does not depend on s and carries the randomness of that fit. In general the conditional distribution both varies with s and is non-degenerate given S=sS=s, as when the sample is combined with external data or the fit is randomly initialized. Since U is independent of (S,β^,γ^)(S, β, γ), the joint law of (S,β^,γ^,U)(S, β, γ,U) is obtained from ℙSP^S, this kernel and the law of U, and is therefore a function of ℙSP^S alone for a fixed estimation procedure. We write ℙS(φω=1)P^S( _ω=1) for the rejection probability under it. Both embedding maps ωX _X and ωZ _Z have to be chosen in the eCIT. For the former, we follow the approaches discussed in Simnacher et al. (2026). This choice affects only the power of the test as long as the learning approaches characterized in Simnacher et al. (2026, Corollary 3), e.g. sample splitting or externally trained embedding maps, are used. The choice of ωZ _Z affects the T1E and the power of the eCIT. We will characterize sufficient embedding maps ωZ _Z in the next subsection. Furthermore, we evaluate the validity and power of the corresponding eCITs on semi-synthetic data in Subsection 4.2. 3.1 Sufficient Embedding Maps In this subsection, we take the embedding map ωX _X as given, so that a valid test for (6) is already a valid test for (1), and characterize the embedding maps ωZ _Z for which we may test (4) instead. We first restate both hypotheses as CI statements conditioning on the estimated parameters β β and γ γ. We then give conditions under which H0ωXH_0 _X implies H0ωH_0^ω, so that a valid test of the embedded hypothesis controls the T1E for the hypothesis on the texts: γ γ must not introduce conditional dependence (Assumption 3.1), and ZωZ _Z must retain all information in Z about either Y or XωX _X (Theorem 3). We call such ωZ _Z sufficient. Finally, we give conditions for the converse implication (Theorem 8) and for equivalence of the two hypotheses (Corollary 9). We can either embed X or Z first. We embed X first, because this simplifies embedding Z, as we will see in the following sufficiency characterizations for the embedding map of Z. Thus, we assume throughout this subsection that X was embedded onto XωX _X. Then, to test (1), we test instead H0ωX:ℙSωX|β^∈0ωXℙβ^-a.s.vs.H1ωX:ℙβ^(ℙSωX|β^∈0ωX)<1, H_0 _X:P^S _X| β _0 _X\;\;P β-a.s. vs. H_1 _X:P β(P^S _X| β _0 _X)<1, (6) where ℙSωX|β^P^S _X| β denotes the conditional distributions of the sample after embedding X given the parameter β β of the embedding map ωX _X. We assume that ωX _X was obtained as specified in Simnacher et al. (2026, Theorem 1, Corollary 3), e.g. learned on external data, through sample splitting or in an unsupervised manner. This ensures that a valid test for (6) is a valid test for (1), although the embedding map can affect the power of the test. Given such a XωX _X, we are interested in embedding maps ωZ _Z to test (4) instead of (6). To do this, we will assume throughout that γ γ does not introduce conditional dependence. assumption Either OPENa)γ^⟂Yn|((XωX,Z)n,β^)a)\; γ \!\!\! Y^n ((X _X,Z)^n, β) or OPENb)γ^⟂XωX,n|((Y,Z)n,β^)b)\; γ \!\!\! X _X,n ((Y,Z)^n, β). This is similar to the assumption placed on the embedding map parameters β β here and in the theoretical results of Simnacher et al. (2026). Furthermore, it holds trivially for γ γ learned via sample splitting, when S denotes the holdout split, or on data independent of the test sample S, but has to be ensured for γ γ learned from S. In particular, branch a) holds by construction whenever γ γ is a measurable function of (XωX,Z)n(X _X,Z)^n, since γ γ is then measurable with respect to the conditioning σ-field. This covers the in-sample construction of ωZ _Z by regressing XωX,nX _X,n on ZnZ^n, which is the natural use of the representation XωX _X obtained in the first step. In contrast to β β, this assumption is not sufficient for a valid test for (4) to yield a valid test for (6) and (1). Specifically, we need also that Z does not contain information about Y or XωX _X beyond ZωZ _Z, which we ensure below. First, we show that the hypothesis H0ωH_0^ω in (4), formulated as an almost-sure statement of the conditional distribution ℙSω|γ^,β^P^S^ω γ, β, is equivalent to a single conditional independence statement in which γ γ and β β are added to the conditioning set. cit˙for˙llms-pratenddefaultcategory.tex Lemma 2. Let ℙSω|γ^,β^P^S^ω γ, β denote a regular conditional distribution of SωS^ω given (γ^,β^)( γ, β). Then XωX,n⟂Yn∣(ZωZ,n,γ^,β^)⟺ℙSω|γ^,β^∈0ωℙγ^,β^-a.s.X _X,n \!\!\! Y^n (Z _Z,n, γ, β) ^S^ω γ, β _0^ω γ, β-a.s. The analogous equivalence holds for SωXS _X, β β, 0ωXP_0 _X and ZnZ^n in place of Sω,(γ^,β^),0ωS^ω,( γ, β),P_0^ω and ZωZ,nZ _Z,n. All proofs are provided in Appendix A.cit˙for˙llms-pratenddefaultcategory.tex Both instances follow from the general Lemma 14 in Appendix A. Note that no assumption is placed on the spaces ℬ,B,C in which β^,γ β, γ take their values, and that 0ωP_0^ω is not required to consist of product measures, so that Lemma 2 applies unchanged when γ γ is estimated in sample and the rows of SωS^ω are conditionally dependent. Next, we show that for an embedding map ωZ _Z containing all information within Z of either, Y or XωX _X, the null hypothesis (6) including texts Z implies the null hypothesis (4) including only the corresponding embeddings ZωZ _Z. Theorem 3 (Z-text null implies Z-embedded null). Let γ γ satisfy Assumption 3.1, and (i)XωX,n⟂Zn|(γ^,β^,ZωZ,n)or(ii)Yn⟂Zn|(γ^,β^,ZωZ,n). (i)\;X _X,n \!\!\! Z^n ( γ, β,Z _Z,n) (i)\;Y^n \!\!\! Z^n ( γ, β,Z _Z,n). Then H0ωX⟹H0ωH_0 _X H_0^ω. cit˙for˙llms-pratenddefaultcategory.tex The conditions on β β, γ γ and ωZ _Z used above are statements about the joint law of (S,β^,γ^)(S, β, γ) and hence properties of ℙSP^S, so we may collect the null distributions satisfying them: 0suff:=ℙS∈0: _0^suff:= \P^S _0:\ β^⟂Yn|(X,Z)n,Assumption 3.1 holds, and β \!\!\! Y^n (X,Z)^n,\ Assumption \ ass:gamma_no_cond_dep holds, and (i) or (i) of Theorem 3 holds. (i) or (i) of Theorem \ thm:text0_implies_emb0_Z holds \. The class depends on the estimation procedure, ωX _X, ωZ _Z and n, which we suppress in the notation. Definition 4 (Validity). Let α∈(0,1)α∈(0,1) and let ⊆0P _0. A test φω _ω is valid for P at level α if supℙS∈ℙS(φω=1)≤α. _P^S P^S ( _ω=1 )≤α. Analogously, for a class Q of probability measures on (ω,ℱω)(S^ω,F_S^ω), the test ψ is valid for Q at level α if supQ∈Q(ψ=1)≤α _Q Q(ψ=1)≤α. Theorem 5 (Validity of the eCIT). Let α∈(0,1)α∈(0,1). If the CIT ψ is valid for 0ωP_0^ω at level α, then the eCIT φω=ψ∘ω¯ _ω=ψ ω is valid for 0suffP_0^suff at level α. cit˙for˙llms-pratenddefaultcategory.tex Remark 6. Unlike in Simnacher et al. (2026, Theorem 2), the embedded null cannot be stated for the marginal law of SωS^ω. There, the assumption on β β allows β β to be removed from the conditioning set by contraction and decomposition, because XωX,nX _X,n appears on the left of the conditioning bar and β β is a free element of the conditioning set. Here ZωZ,nZ _Z,n is σ(Zn,γ^)σ(Z^n, γ)-measurable and appears on the right, so γ γ cannot be removed: deleting it changes the σ-field relative to which the conditioning variable is defined. Marginalizing instead averages over γ γ, and a mixture of conditionally independent laws need not be conditionally independent. The almost-sure formulation (4) and the conditional argument in Step 3 of the proof of Theorem 5 are therefore needed. Theorem 5 requires ψ to be valid over all of 0ωP_0^ω, whose elements are not required to be product measures. Two regimes have to be distinguished. If γ γ is estimated on data independent of S — externally, or on a training split disjoint from the test sample — then conditionally on θ θ the rows of SωS^ω remain i.i.d., and validity over the i.i.d. null class suffices. Let 0ω,⊗P_0^ω, denote the n-fold products M⊗nM n of probability measures M on the embedded observation space under which XωX⟂Y|ZωZX _X \!\!\! Y Z _Z, which by Lemma 16 implies 0ω,⊗⊆0ωP_0^ω, _0^ω. Corollary 7 (Independently estimated embedding parameters). Let α∈(0,1)α∈(0,1) and let ψ be valid for 0ω,⊗P^ω, _0 at level α. Then φω _ω is valid at level α for ℙS∈0suff:θ^⟂S\P^S ^suff_0: θ \!\!\! S\. cit˙for˙llms-pratenddefaultcategory.tex If instead γ γ is estimated in sample, Lemma 16 no longer applies at the embedded level: the conditional laws are in general not product measures, and ψ must be valid over the strictly larger class 0ωP_0^ω. We therefore state Theorem 3 and Theorem 5 for general γ γ, since the hypothesis-transfer argument requires no product structure, but restrict γ γ to embedding maps trained on a held-out corpus in the application in Subsection 4.3, so that Corollary 7 applies and the conditional null class is the i.i.d. one. While Theorem 3 shows that a sufficient embedding map ωZ _Z transfers the null hypothesis for texts Z to the null hypothesis for embedded texts ZωZ _Z, it does not establish equivalence of the two hypotheses. Thus, the embedded null (4) could be true while the text null (6) is false. For example, ZωZ _Z could contain Y, so that conditioning on ZωZ,nZ _Z,n makes YnY^n measurable with respect to the conditioning set and (4) holds trivially. For the converse implication we have to exclude such cases, and we additionally need that γ γ may be removed from the conditioning set. assumption Either a) γ^⟂Yn|(Zn,β^) γ \!\!\! Y^n (Z^n, β) or b) γ^⟂XωX,n|(Zn,β^) γ \!\!\! X _X,n (Z^n, β). Assumption 3.1 holds trivially whenever γ γ is learned on data independent of S, and whenever γ γ is a measurable function of ZnZ^n alone, for instance for unsupervised embedding maps of the source texts. Theorem 8 (Embedded null implies text null). Let (i)XωX,n⟂Zn|(β^,γ^,ZωZ,n,Yn)and Assumption 3.1a),or (i)\;X _X,n \!\!\! Z^n ( β, γ,Z _Z,n,Y^n) Assumption \ ass:gamma_conversea), (ii)Yn⟂Zn|(β^,γ^,ZωZ,n,XωX,n)and Assumption 3.1b). (i)\;Y^n \!\!\! Z^n ( β, γ,Z _Z,n,X _X,n) Assumption \ ass:gamma_converseb). Then H0ω⟹H0ωXH_0^ω H_0 _X. cit˙for˙llms-pratenddefaultcategory.tex Finally, a joint CI assumption gives the equivalence: Corollary 9. Let Assumptions 3.1 and 3.1 hold and let (∗):(XωX,n,Yn)⟂Zn|(β^,γ^,ZωZ,n).(*): (X _X,n,Y^n) \!\!\! Z^n ( β, γ,Z _Z,n). Then H0ω⇔H0ωXH_0^ω H_0 _X. cit˙for˙llms-pratenddefaultcategory.tex Corollary 9 is a statement about the hypotheses, not about the power of a specific test: under (∗)(*), the embedding map cannot map an alternative in 1ωXP_1 _X into the embedded null, so no alternative becomes undetectable in principle. It does not imply that the eCIT has the same power as a test of (6), since the embedding may reduce the effect size and project onto alternatives not detectable by the CIT applied to test (4). Throughout this subsection we treated β β as already learned, e.g. on an external dataset, and considered only the estimation of γ γ. This weakens the requirement on ωZ _Z. Since XωX,nX _X,n is σ(Xn,β^)σ(X^n, β)-measurable, we have σ(XωX,n)⊆σ(Xn,β^)σ(X _X,n) σ(X^n, β), and condition (i) of Theorem 3 requires only XωX,n⟂Zn|(β^,γ^,ZωZ,n)instead ofXn⟂Zn|(β^,γ^,ZωZ,n),X _X,n\; \!\!\! \;Z^n ( β, γ,Z _Z,n) of X^n\; \!\!\! \;Z^n ( β, γ,Z _Z,n), so that ZωZ _Z need only retain the information in Z about the representation XωX _X, and not about the whole text X. The coarser the embedding ωX _X, the weaker this condition becomes. However, if ωX _X does not represent the conditional association between X and Y in a form the CIT can detect, the test loses power (Simnacher et al., 2026). As X is embedded in any case to apply Simnacher et al. (2026), the choice is not whether to embed X but in which order: embedding X first weakens the sufficiency requirement on ωZ _Z without any additional loss of power, and may thereby facilitate T1E control. 3.2 Conditional Mean Independence Throughout the paper, we focus on the projected covariance measure (Lundborg et al., 2024, PCM;) to test hypothesis (4), an algorithm-agnostic CMIT which performs well for low-dimensional, vector-valued Z and embedded images X (Simnacher et al., 2026). However, other conditional (mean) independence tests (Strobl et al., 2019; Dai et al., 2022; Williamson et al., 2023; Cai et al., 2025, e.g.,) can be applied to test (4). Throughout this subsection we take Y to be ℝKR^K-valued, so that conditional means of Y are defined: K=1K=1 in the continuous case ⊆ℝY , and K=L−1K=L-1 in the categorical case =1,…,LY=\1,…,L\, where Y is represented by its vector of indicators of the non-baseline classes (cf. Remark 13 and Appendix D). We assume throughout that ‖Y1‖<∞E\|Y_1\|<∞, where ∥⋅∥\|·\| is any norm on ℝKR^K, all of which are equivalent, and which reduces to the absolute value for K=1K=1. The assumption holds automatically in the categorical case and, since the rows of S are i.i.d., gives ‖Yi‖<∞E\|Y_i\|<∞ for all i≤ni≤ n, so that the conditional means of YiY_i appearing below are well defined. We test (4) by applying the PCM, a CMIT, which tests instead H0ω,mean:ℙSω|θ^∈0ω,meanℙθ^-a.s.vs.H1ω,mean:ℙθ^(ℙSω|θ^∈0ω,mean)<1,H_0^ω,mean:\ P^S^ω θ _0^ω,mean\ \ P θ-a.s. . H_1^ω,mean:\ P θ (P^S^ω θ _0^ω,mean )<1, (7) where 0ω,mean:= _0^ω,mean:= \ Q probability measure on (ω,ℱω):Q‖Yi‖<∞ and Q probability measure on (S^ω,F_S^ω):E_Q\|Y_i\|<∞ and Q[Yi∣XωX,n,ZωZ,n]=Q[Yi∣ZωZ,n]Q-a.s. for all i≤n. _Q [Y_i X _X,n,Z _Z,n ]=E_Q [Y_i Z _Z,n ]\ Q-a.s. for all i≤ n \. Since CI implies conditional mean independence and YiY_i is a measurable function of YnY^n, every integrable Q∈0ωQ _0^ω lies in 0ω,meanP_0^ω,mean. Integrability is not implied by 0ωP_0^ω and has to be required separately, but it is not a restriction here: the embedding maps act on X and Z only, so the standing assumption ‖Y1‖<∞E\|Y_1\|<∞ carries over to the embedded sample space and the conditional law ℙSω|θ^P^S^ω θ is integrable almost surely. Whenever this conditional law lies in 0ωP_0^ω it therefore lies in 0ω,meanP_0^ω,mean, and since Step 3 of the proof of Theorem 5 uses only that it lies almost surely in the class over which ψ is valid, Theorem 5 applies unchanged. However, power against alternatives in 0ω,mean∖0ωP_0^ω,mean _0^ω can be lost (Lundborg et al., 2024), i.e. those in which the summary carries information about Y beyond the speech without shifting its conditional mean. As in the previous subsection, we embed X first and characterize afterwards mean sufficiency for the embedding of Z. Thus, we define H0ωX,mean:ℙSωX|β^∈0ωX,meanℙβ^-a.s.vs.H1ωX,mean:ℙβ^(ℙSωX|β^∈0ωX,mean)<1,H_0 _X,mean:\ P^S _X β _0 _X,mean\ \ P β-a.s. . H_1 _X,mean:\ P β (P^S _X β _0 _X,mean )<1, (8) where 0ωX,mean:= _0 _X,mean:= \ Q probability measure on (ωX,ℱωX):Q‖Yi‖<∞ and Q probability measure on (S _X,F_S _X):E_Q\|Y_i\|<∞ and Q[Yi∣XωX,n,Zn]=Q[Yi∣Zn]Q-a.s. for all i≤n. _Q [Y_i X _X,n,Z^n ]=E_Q [Y_i Z^n ]\ Q-a.s. for all i≤ n \. Because (8) is stated only for the conditional mean, the assumption β^⟂Yn|(X,Z)n β \!\!\! Y^n (X,Z)^n under which Simnacher et al. (2026) transfer H0H_0 to H0ωXH_0 _X can be weakened. Specifically, β β is only required to introduce no conditional mean dependence: assumption [Yi∣Xn,Zn,β^]=[Yi∣Xn,Zn]E [Y_i X^n,Z^n, β ]=E [Y_i X^n,Z^n ] a.s. for all i≤ni≤ n. Assumption 3.2 is implied by β^⟂Yn|(X,Z)n β \!\!\! Y^n (X,Z)^n and holds in particular whenever β β is a measurable function of (X,Z)n(X,Z)^n or is learned on data independent of S (Simnacher et al., 2026, Corollary 3). Next, we show that this assumption is sufficient to transfer the null hypothesis (1) including X and Z to the null hypothesis H0ωX,meanH_0 _X,mean including only a representation of X. Theorem 10 (Text null implies X-embedded mean null). Let Assumption 3.2 hold. Then H0⟹H0ωX,meanH_0 H_0 _X,mean. cit˙for˙llms-pratenddefaultcategory.tex In addition, we can weaken the sufficiency condition on the embedding of Z if only the mean-level null has to be transferred. Write mi:=[Yi∣Zn,β^],i=1,…,n,m_i:=E [Y_i Z^n, β ], i=1,…,n, for the regression of YiY_i on the source texts and the embedding parameter β β. When β β is deterministic or learned on data independent of S, it is independent of (Yi,Zn)(Y_i,Z^n) and mi=[Yi∣Zn]m_i=E[Y_i Z^n] is the regression of YiY_i on the source texts alone; β β is retained in general because an in-sample β β is a function of (X,Z)n(X,Z)^n and can then carry information about YiY_i beyond ZnZ^n. We first state a mean-level version of Assumption 3.1a), which restricts γ γ to introduce no conditional mean dependence and holds in particular whenever γ γ is learned independently of S or is a measurable function of (XωX,Z)n(X _X,Z)^n. We then characterize when H0ωX,meanH_0 _X,mean transfers to H0ω,meanH_0^ω,mean, and show that it does so whenever mim_i is σ(ZωZ,n,θ^)σ(Z _Z,n, θ)-measurable, that is, whenever Z carries no information about the conditional mean of Y beyond ZωZ _Z. This replaces the requirement that ZωZ _Z preserve the entire conditional law of Y or XωX _X given Z by a requirement on a single regression function. assumption [Yi∣XωX,n,Zn,θ^]=[Yi∣XωX,n,Zn,β^]E[Y_i X _X,n,Z^n, θ]=E[Y_i X _X,n,Z^n, β] a.s. for all i≤ni≤ n. Assumption 3.1a) implies Assumption 3.2, which restricts γ γ in the same way as Assumption 3.2 restricts β β. There is no analogue of Assumption 3.1b), because conditional mean independence is not symmetric in its two arguments. Theorem 11 (Z-text mean null implies Z-embedded mean null). Let Assumption 3.2 hold, and let (i′)mi=[mi∣ZωZ,n,θ^]a.s. for all i≤n,(i$ $) m_i=E [m_i Z _Z,n, θ ] .s. for all i≤ n, that is, let each mim_i be almost surely equal to a σ(ZωZ,n,θ^)σ(Z _Z,n, θ)-measurable random variable. Then H0ωX,mean⟹H0ω,meanH_0 _X,mean H_0^ω,mean. cit˙for˙llms-pratenddefaultcategory.tex The proof of Theorem 11 shows that [Yi∣ZωZ,n,θ^]=[mi∣ZωZ,n,θ^]E[Y_i Z _Z,n, θ]=E[m_i Z _Z,n, θ], so (i′) holds if and only if mi=[Yi∣ZωZ,n,θ^]m_i=E[Y_i Z _Z,n, θ] a.s. for all i≤ni≤ n, that is, if and only if the regression of YiY_i on the embedded source texts coincides with its regression on the source texts. Mean sufficiency of ωZ _Z is thus a condition on the regression of Y on Z alone and can be assessed without reference to XωX _X. Condition (i′) is an exact measurability requirement and is not plausible for embedding maps learned from data, which approximate e.g. the regression of Y on the source texts rather than reproducing it. What matters in practice is whether the residual mi−[mi∣ZωZ,n,θ^]m_i-E[m_i Z _Z,n, θ] is negligible relative to n−1/2n^-1/2, since a fixed nonzero residual enters the test statistic as a location shift growing with n n. It is nevertheless approachable, because mim_i need not depend on all of ZiZ_i: if mi=g(t(Zi))m_i=g(t(Z_i)) for a map t and a function g, then (i′) holds for every ωZ _Z for which t(Zi)t(Z_i) is σ(ZωZ,n,θ^)σ(Z _Z,n, θ)-measurable. In the simulation study of Subsection 4.2, t is a linear score in the embedding used to generate Y, so every eCIT whose ZωZ _Z retains that embedding satisfies (i′) exactly; in the application in Subsection 4.3, t is unknown and we approximate it by a learned representation of the speech, the masked mean of the last hidden layer of a model predicting Y from Z. Appending further embedding maps enlarges σ(ZωZ,n)σ(Z _Z,n) and can therefore only reduce the residual, which motivates the concatenated conditioning sets used in both. Whether the residual is negligible at a given sample size depends on the dataset and the embedding maps, and we propose to assess it with the simulation design of Subsection 4.2. The null class 0ω,meanP_0^ω,mean constrains the conditional mean of YiY_i given the whole embedded sample, whereas the PCM fits regressions of YiY_i only on (XiωX,ZiωZ)(X _X_i,Z _Z_i) and on ZiωZ _Z_i. The two agree whenever the rows of SωS^ω are i.i.d. conditionally on θ θ, which is e.g. the case when the embedding parameters are estimated independently of S, as in our application. Let 0ω,mean,⊗P_0^ω,mean, denote the n-fold products M⊗nM n of probability measures M on the embedded observation space (×ℨ)( X×Y× Z) under which M‖Y‖<∞E_M\|Y\|<∞ and M[Y∣XωX,ZωZ]=M[Y∣ZωZ]E_M[Y X _X,Z _Z]=E_M[Y Z _Z], which by Lemma 17 satisfies 0ω,mean,⊗⊆0ω,meanP_0^ω,mean, _0^ω,mean. Then, we collect the null distributions satisfying the conditions used above in 0suff,mean:=ℙS∈0:Assumptions 3.2 and 3.2 hold, and (i′) of Theorem 11 holds, _0^suff,mean:= \P^S _0:\ Assumptions \ ass:beta-mean and ass:mean-gamma hold, and (i$ $) of Theorem \ thm:mean-transfer holds \, which depends on the estimation procedure, ωX _X, ωZ _Z and n, suppressed in the notation as for 0suffP_0^suff. Step 3 of the proof of Theorem 5 uses only that ℙSω|θ^P^S^ω θ lies almost surely in the class over which ψ is valid, so a ψ valid for 0ω,meanP_0^ω,mean yields an eCIT valid for 0suff,meanP_0^suff,mean. Then, we can derive the analog of Corollary 7 for the conditional mean independence hypothesis. Corollary 12 (Independently estimated embedding parameters, mean version). Let α∈(0,1)α∈(0,1) and let ψ be valid for 0ω,mean,⊗P_0^ω,mean, at level α. Then φω _ω is valid at level α for ℙS∈0suff,mean:θ^⟂S\P^S _0^suff,mean: θ \!\!\! S\. cit˙for˙llms-pratenddefaultcategory.tex Remark 13 (Categorical Y). For Y finite and Y represented by its vector of category indicators, Q[Yi∣XωX,n,ZωZ,n]=Q[Yi∣ZωZ,n]E_Q[Y_i X _X,n,Z _Z,n]=E_Q[Y_i Z _Z,n] determines the conditional law of YiY_i and is therefore equivalent to XωX,n⟂Yi|ZωZ,nX _X,n \!\!\! Y_i Z _Z,n, for each i≤ni≤ n. For product measures these row-wise statements lift to the joint one by Lemma 16, so 0ω,mean,⊗=0ω,⊗P_0^ω,mean, =P_0^ω, , and (i′) is the row-wise counterpart of condition (i), the two coinciding for product measures for the same reason. For general Q the mean class is the larger one, since row-wise conditional independence need not imply XωX,n⟂Yn|ZωZ,nX _X,n \!\!\! Y^n Z _Z,n. In our application θ^⟂S θ \!\!\! S, so Corollary 12 applies with the product class and nothing is lost by using a CMIT. Finally, we note that the PCM guarantees only asymptotic uniform validity, so the eCIT is asymptotically valid for mean sufficient embedding maps. In our application the embedding parameters are estimated independently of S, so Corollary 12 is the relevant statement and the null class is 0ω,mean,⊗P_0^ω,mean, , whose elements are the i.i.d. laws over which the PCM is asymptotically uniformly valid. The conditional argument in Step 3 of the proof of Theorem 5 transfers uniform asymptotic validity, since the bound holds simultaneously for all Q∈0ω,mean,⊗Q _0^ω,mean, and hence for the realized conditional law ℙSω|θ^P^S^ω θ; pointwise asymptotic validity would not suffice, as the conditioning value θ θ varies with n. For γ γ estimated in sample the conditional laws need not be products, and validity over the larger class 0ω,meanP_0^ω,mean would be required, which the PCM is not known to provide. 4 Application We apply the eCITs to German Parliament speeches and their summaries. First, we provide a description of the data in Subsection 4.1. Then, we evaluate the performance of the eCITs on semi-synthetic data based on the speech-summary pairs in Subsection 4.2. While our simulation study evaluates the T1E control and power of the eCITs on this specific dataset and for the LLM summaries, the simulation design can be used to evaluate if the eCITs can control T1E and have power against alternatives of interest for other datasets with Z, X pairs before applying the eCITs. On other datasets and their underlying data generating processes, the embedding maps used, their collinearity within and between Z and X, and the corresponding separation for categorical Y can affect the performance of the eCITs and particularly the T1E control with respect to the sufficiency of ZωZ _Z and the convergence of the nuisance regressions required by the PCM or other CITs. Finally, we apply in Subsection 4.3 the eCITs to the full dataset, including speech-summary pairs together with the protected attributes of the speakers’ gender and faction. 4.1 Data For the source texts, we use the German Parliament speeches provided in Richter et al. (2020). We clean and merge speeches as described in Appendix B.1. The resulting corpus comprises 274,598274,598 merged speeches from electoral terms 11–2020, dated 12 September 1949–20 May 2022. Roles are dominated by Member of Parliament (MP; 185,009185,009) and Presidium of Parliament (34,39234,392), plus government speakers (Secretary of State 28,54728,547; Minister 19,89219,892; Chancellor 1,5591,559), guests (5,1845,184), and 15 speeches with unlabeled role. Merged speech length has median 382382 words (mean 605605; IQR 122122–794794), and there are 4,0304,030 unique speaker identifiers. For the analysis, we further restrict this corpus to the 274,579274,579 speeches for which a Gemma 4 summary was generated, with 19 dropped speeches because they did not produce a summary. Among merged MP speeches, the main parliamentary groups are CDU/CSU (59,04159,041), SPD (54,75254,752), FDP (27,67727,677), Grüne (21,36921,369), DIE LINKE. (10,00610,006), PDS (4,3384,338), and AfD (2,7422,742), plus smaller and historical groups (KPD, DP, GB/BHE, Zentrum, and others) and speakers without faction (Fraktionslos). We drop unlabeled factions and keep only factions at least as frequent as AfD, yielding seven factions (CDU/CSU, SPD, FDP, Grüne, DIE LINKE., PDS, AfD; n=179,925n=179,925). For the analysis including the speakers’ faction, we randomly draw a held-out test set of size ntest=53,978n_test=53,978 for the eCITs while the remainder (ntrain=125,947n_train=125,947) is used to train embedding maps. For the second attribute we use the speaker’s gender as recorded in the corpus, which takes two values (L=2)(L=2). Here, we keep electoral terms 1010–2020 with non-missing speaker gender (n=169,463n=169,463; ntrain=118,624n_train=118,624, ntest=50,839n_test=50,839), of which around 71%71\% are by male and 29%29\% by female speakers. For the semi-synthetic simulation study, we use the intersection of the faction and gender train/test splits (ntrain=61,807n_train=61,807 to train embeddings, and ntest=11,357n_test=11,357 to apply the eCITs). 4.2 Simulation We examine the empirical performance of the eCIT on simulated data. To evaluate whether the eCIT controls the T1E and has power against relevant alternatives on the dataset of interest, we resample Z,XZ,X pairs and generate Y from a pre-specified embedding of Z (under the null hypothesis) or from prespecified embeddings of Z and X (under the alternative hypothesis). Since we apply the PCM, a CMIT, and the embedding maps are trained on a holdout set disjoint from the test sample, so that θ^⟂S θ \!\!\! S, the relevant statements are Theorem 11 and Corollary 12, and the condition on ωZ _Z is mean sufficiency, i.e. (i′). For the binary and multinomial outcomes this coincides with sufficiency by Remark 13, and we write sufficient for mean sufficient below. 4.2.1 Design We compare the proposed eCITs across several combinations of fixed and learned embedding maps for X and Z. Summaries are generated with Gemma 4 (Gemma Team, 2026) throughout, and additionally with Qwen 3.6 (Qwen Team, 2026) for the application in Subsection 4.3 (Appendix B.1). Data generating mechanisms (DGMs): We consider 12 DGMs across three outcome types, two generating embedders, under the null and alternative hypothesis. For all, we vary the sample size over n=1,000,5,000,10,000n=1,000,5,000,10,000. These DGMs extend the simulation studies of Simnacher et al. (2026), which evaluate CITs for low-dimensional Z and images X, to high-dimensional texts Z and X. For the source texts Z, we resample repeatedly n speeches from the test set (cf. Subsection 4.1). We obtain the corresponding summaries X using the Gemma 4 LLM. This provides speech-summary pairs (Zi,Xi)(Z_i,X_i) as used in the real-world application. We embed each ZiZ_i and XiX_i with the Jina embedder for text classification (Akram et al., 2026) and the Qwen embedder for general tasks (Zhang et al., 2025), truncated with their Matryoshka representations (Kusupati et al., 2022) to d=32d=32 dimensions. Depending on the DGM, we generate Y from one of the two embedders, whose maps for the speech and the summary we denote by ω~Z ω_Z and ω~X ω_X, i.e. ω~Z,ω~X∈Jina, Qwen ω_Z, ω_X∈\Jina, Qwen\, so that ziω~Z,xiω~X∈ℝ32z_i ω_Z,x_i ω_X ^32. We draw two directions BZ,BX∼N(032,I32)B_Z,B_X N(0_32,I_32) once and compute the unnormalized scores s~Z,i=(ziω~Z)⊤BZ,s~X,i=(xiω~X−A⊤ziω~Z)⊤BX, s_Z,i= (z_i ω_Z ) B_Z, s_X,i= (x_i ω_X-A z_i ω_Z ) B_X, where A is the coefficient matrix of the regression of xω~Xx ω_X on zω~Zz ω_Z, fitted on the training split. Then, the speech and summary scores used in the DGMs are sZ,i=s~Z,i/sd^(s~Z)s_Z,i= s_Z,i/ sd( s_Z) and sX,i=s~X,i/sd^(s~X)s_X,i= s_X,i/ sd( s_X), so that both have unit variance. The column standardization of the embeddings and the two standard deviations are likewise computed on the training split, so that all of these are held fixed across replications. Thus sZs_Z is a one-dimensional projection of the speech representation zω~Zz ω_Z, and sXs_X a one-dimensional projection of the part of the summary representation xω~Xx ω_X that the speech representation does not explain. Given sZs_Z and sXs_X, we generate Y under c=0c=0 (CI) and c=1c=1 (no CI) according to the outcome type: continuous: Yi=sZ,i+csX,i+εi,εi∼N(0,σ2), Y_i=s_Z,i+c\,s_X,i+ _i, _i N(0,σ^2), binary: Yi∼Bernoulli(expit(ηi)),ηi=αZsZ,i+cαXsX,i+b, Y_i (expit( _i) ), _i= _Zs_Z,i+c\, _Xs_X,i+b, (9) multinomial: Yi∼Categorical(softmax(ηi)),Yi∈0,…,L−1, Y_i (softmax( _i) ), Y_i∈\0,…,L-1\, where in the multinomial case the reference class has ηi,0=0 _i,0=0 and the remaining L−1L-1 logits are fixed linear combinations of sZ,is_Z,i and csX,ic\,s_X,i, chosen so that the two scores load on the classes in different directions, with the softmax taken over all L logits. We set L=5L=5. The coefficients αZ,αX _Z, _X, the class weights, the intercept b, and the noise variance σ2σ^2 of the continuous outcome are given in Appendix B.3. Under the null, Y depends on the speech only through sZ,is_Z,i, and hence only through zω~Zz ω_Z. Embeddings in eCITs: First, we apply the eCITs with only the embedding maps used to generate Y (oracle). Next, we apply the eCITs with additional embedding maps, including the BGE embedder (Chen et al., 2024), the Llama Nemotron embedder (Moreira et al., 2024; NVIDIA, 2025), the Jina or Qwen embedder, depending on which was not used in the corresponding Y generation, and learned embeddings. We refer to the eCITs combining the generating embedding with further embedding maps as diluted, and contrast them with those omitting the generating embedding. We selected the four pre-trained embedding maps because they cover embedding models based on different architectures, they allow for the long context of the speeches, and they performed best in a recent meta study of embedding maps for text across several tasks (Gjorgjevikj et al., 2026). For the trained embedding maps, we predict Y separately from the speech Z and from the summary X using a text prediction model with 134 million parameters and classification or regression head (Wunderle et al., 2025) on a large holdout set of 61,80761,807 speech-summary pairs together with the generated Y in the corresponding DGM. Target and performance measures: Performance is measured in terms of the T1E and power. T1E and power are estimated from the rejection rates RR^=1nsim∑l=1nsimpl≤α R= 1n_sim _l=1^n_sim 1\p_l≤α\ under the null (c=0c=0) and alternative (c=1c=1) hypothesis, respectively, for a significance level of α=0.05α=0.05, over l=1,…,nsiml=1,…,n_sim random data generations and using p-values plp_l for each DGM. We choose nsim=50n_sim=50 for computational reasons, resulting at level α=0.05α=0.05 in a Monte Carlo (MC) standard error (Morris et al., 2019, sec. 5.3) of MC SE(RR^)=RR^(1−RR^)nsim≈0.05∗0.9550≈0.031. SE( R)= R(1- R)n_sim≈ 0.05^*0.9550≈ 0.031. 4.2.2 Results Figure 2 shows the rejection rates for binary and multinomial Y generated from the Jina embeddings. The oracle eCIT, which uses the embedding maps ω~Z,ω~X ω_Z, ω_X that generate Y, satisfies condition (i′) of Theorem 11 by construction, since under the null mim_i depends on ZiZ_i only through sZ,is_Z,i, which is a linear function of ziJinaz_i^Jina. It therefore isolates the behavior of the PCM on this DGM, that is, whether the test controls the T1E and has power for the embedded hypothesis, and the results show that it holds the nominal level and rejects in every replication under the alternative already at n=1,000n=1,000. It thus serves as the reference for the remaining eCITs: deviations from it are attributable to the embedding maps and the PCM applied to the larger dimension of the corresponding combined embeddings. Figure 2: The rejection rates of the eCIT for binary and multinomial Y generated under the null (first row) and alternative (second row) from the Jina embeddings ZJinaZ^Jina and XJinaX^Jina. The eCIT is applied with the Jina embeddings (oracle), the specified diluted embeddings, including additionally learned, BGE, Qwen, and Llama embeddings, as well as the same combinations excluding the Jina embedding. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation set unpenalized; for categorical Y the regressions of Y are multinomial logistic and those of the fitted probabilities and of f f are least squares (Appendix D.1). The dashed line marks the nominal level α=0.05α=0.05, the shaded area marking two MC standard errors around it. The continuous outcome and the corresponding Q-plots are shown in Figures 3 and 4. Under the null and the Jina DGM, all eCITs stay within or close to two MC standard errors of α=0.05α=0.05 for binary and multinomial Y, with a maximum rejection rate of around 13%13\% at n=10,000n=10,000. This holds also for the eCITs that omit the generating embedding ZJinaZ^Jina entirely, i.e. for conditioning sets for which sufficiency does not hold by construction, although for some of them, e.g. ZωZ=BGE+Llama+QwenZ _Z=BGE+Llama+Qwen and ZωZ=learnedZ _Z=learned, the rejection rate increases mildly over the sample sizes considered. The Q-plots in Figure 4 show the same picture: for the categorical outcomes the p-values track the uniform diagonal closely at n=1,000n=1,000, with departures for individual eCITs at n=10,000n=10,000 that remain small. In general, level control does not require ZωZ _Z to contain the generating embedding, only that it retain enough of sZs_Z for the transfer of the null hypothesis on the level of the texts to the one on the level of the embedded texts (Theorem 11) to hold approximately. Here, the part of sZs_Z that is lost by omitting ZJinaZ^Jina is still negligible at n n scale at the sample sizes considered. Under the alternative and the Jina DGM, the combination of the embeddings of XωX _X and ZωZ _Z affects the power of the corresponding eCITs. The ordering of the eCITs follows from how much of sXs_X is retained in XωX _X. At n=1,000n=1,000 no non-oracle arm exceeds around 10%10\%. At n=5,000n=5,000 the arms that contain the generating embeddings ZJina,XJinaZ^Jina,X^Jina reach 5555–97%97\% for binary and 5656–90%90\% for multinomial Y, whereas the eCITs omitting the generating embeddings have lower or similar power. The eCIT with ZωZ=learned+BGE+Qwen+LlamaZ _Z=learned+BGE+Qwen+Llama and XωX=learned+Qwen+LlamaX _X=learned+Qwen+Llama achieves the lowest power for binary and multinomial Y. Here, the BGE embedding in ZωZ _Z is not included in XωX _X, lowering the detectable signal. Adding embeddings to XωX _X can thus recover power, depending on how well the available maps span the conditional dependence of the DGM. Finally, eCITs for multinomial Y are less powerful than for binary Y at the largest sample size. The Qwen DGM separates including and omitting the generating embedding ZQwenZ^Qwen, and the corresponding (in)sufficiency of the embedding of Z, more clearly (Figure 5). The eCITs with sufficient embeddings containing ZQwenZ^Qwen, i.e. the oracle and the three diluted arms, stay within or close to two MC standard errors of α=0.05α=0.05 at every sample size, with rejection rates of at most 12%12\% for binary and 11%11\% for multinomial Y at n=10,000n=10,000, and their power reaches at least 90%90\% at n=10,000n=10,000. Diluting the generating embedding with further embedding maps therefore still leads to tests controlling T1E, and only raises the sample size at which full power is reached, as under the Jina DGM. This also indicates that the PCM performs well on this dataset and the DGMs, also in the higher-dimensional conditioning sets, as long as the embedding of Z remains sufficient. The T1E inflation of eCITs that omit ZQwenZ^Qwen increases with the sample size. For example, for binary Y and ZωZ=BGE+Jina+LlamaZ _Z=BGE+Jina+Llama the T1E rises from 6%6\% at n=1,000n=1,000 to 48%48\% at n=5,000n=5,000 and 86%86\% at n=10,000n=10,000. Other DGMs show the same pattern at somewhat smaller magnitudes. Enlarging the conditioning set within this group reduces the inflation, with ZωZ=learned+BGE+Jina+LlamaZ _Z=learned+BGE+Jina+Llama rejecting in 38%38\% of the replications at n=10,000n=10,000. For the Qwen DGM, concatenating further embedding maps thus recovers part, but not all, of the information in Z about Y that ZQwenZ^Qwen carries. For the corresponding eCITs, we cannot interpret their power, because they do not control the T1E. For continuous Y the results separate the sample size behavior of the arms by the sufficiency of ZωZ _Z (Figures 3 and 5). The oracle eCIT stays within two MC standard errors of α=0.05α=0.05 at every sample size considered, as for the categorical outcomes. The diluted arms, whose ZωZ _Z retains the generating embedding and is therefore sufficient by construction, are inflated at n=1,000n=1,000 but calibrated at n=10,000n=10,000, so that the nuisance regressions of the PCM converge more slowly here than in the categorical case. The arms omitting the generating embedding remain inflated at n=10,000n=10,000. Under the Jina DGM the rejection rates of some of these arms are not monotone in n, consistent with the convergence of the nuisance regressions and the effect of the omitted information acting simultaneously. Information in Z about Y that ZωZ _Z does not represent acts as an omitted conditioning variable, which condition (i′) of Theorem 11 excludes. It enters as conditional dependence between the embeddings, so the embedded null is false although the null on the texts holds. In addition, the nuisance regressions of the PCM, which are linear in the supplied embeddings, need no longer be correctly specified once the generating embedding is omitted. Both may contribute to the observed inflation, and the present simulation does not separate their contributions. Enlarging ZωZ _Z then reduces the T1E inflation (cf. the T1E of the eCITs with ZωZ=learned+BGE+Jina+LlamaZ _Z=learned+BGE+Jina+Llama against ZωZ=learnedZ _Z=learned and ZωZ=BGE+Jina+LlamaZ _Z=BGE+Jina+Llama under the Qwen DGM in Figure 5), but does not guarantee that the remaining inflation is negligible at a given n. Both DGMs omit the generating embedding in some arms, but under the Jina DGM the induced conditional dependence stays negligible at n n scale for the categorical outcomes over the sample sizes considered. Whether an insufficient ZωZ _Z matters at a given n is thus a property of the DGM and the embeddings. The simulation design assesses this for the dataset at hand before the eCITs are applied. In our simulation, the arms retaining the generating embedding are calibrated throughout for the categorical outcomes, and for the continuous one the oracle is calibrated throughout while the diluted arms are calibrated from n=10,000n=10,000, and the inflation of the remaining arms decreases as ZωZ _Z is enlarged, motivating the full concatenation in Subsection 4.3. 4.3 Real-world case study We apply the eCITs to the test sets described in Subsection 4.1, with Y the speaker’s faction (L=7L=7) or gender (L=2L=2), Z the cleaned speeches and X its summary, for the Gemma 4 and Qwen 3.6 summarizers (cf. Appendix B.1 for additional details). Since Y is categorical, testing conditional mean independence with the PCM is equivalent to testing CI (Remark 13), so nothing is lost by using a CMIT in our application. The specifically learned embedding maps are obtained from classifiers fitted on the training set, which is disjoint from the test set (see also Appendix B.2), while the pre-trained embedding maps were fitted on large corpora that may overlap with the German Parliament speeches. Treating all embedding maps as independent of the test set and the observations as i.i.d., we have θ^=(β^,γ^)⟂S θ=( β, γ) \!\!\! S, so the conditions on β β (Assumption 3.2) and γ γ (Assumption 3.2) in 0suff,meanP_0^suff,mean hold and Corollary 12 applies with the i.i.d. null class, over which the PCM is asymptotically valid. Neither assumption is guaranteed here, and we return to both below, together with the sufficiency of ZωZ _Z, which cannot be verified and is instead examined through the simulation study in Subsection 4.2 and by varying ZωZ _Z across the settings reported here. Table 1: Test statistics T and p-values of the eCITs on the test sets of Subsection 4.1, for the speaker’s gender and faction and the summaries of both summarizers. Gemma 4: gender n=50,839n=50,839, faction n=53,978n=53,978. Qwen 3.6: gender n=50,796n=50,796, faction n=53,855n=53,855; the sample sizes differ slightly because a few summary generations were empty. ∗ indicates p<0.05p<0.05. ZωZ _Z and XωX _X give the embeddings of the speech and of the summary; pred-hidden denotes the masked mean of the last hidden layer of the classifier predicting Y, and Qwen the Qwen3 embedder, as distinct from the Qwen 3.6 summarizer. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation dataset unpenalized, the regressions of Y being multinomial logistic and those of the fitted probabilities and of f f least squares, with the ridge constant c c in the projection direction tuned on D1D_1 (Appendix D.1). Outcome ZωZ _Z XωX _X Gemma 4 T Gemma 4 p Qwen 3.6 T Qwen 3.6 p gender BGE+Jina+Qwen+Llama BGE+Jina+Qwen+Llama 24.68724.687 <10−15<10^-15∗ 34.14634.146 <10−15<10^-15∗ faction BGE+Jina+Qwen+Llama BGE+Jina+Qwen+Llama 34.70534.705 <10−15<10^-15∗ 51.31751.317 <10−15<10^-15∗ gender pred-hidden+BGE+Jina+Qwen+Llama pred-hidden+BGE+Jina+Qwen+Llama 4.1944.194 1.37×10−51.37× 10^-5∗ 5.5055.505 1.85×10−81.85× 10^-8∗ faction pred-hidden+BGE+Jina+Qwen+Llama pred-hidden+BGE+Jina+Qwen+Llama 11.85911.859 <10−15<10^-15∗ 18.33418.334 <10−15<10^-15∗ gender pred-hidden+BGE+Jina+Qwen+Llama pred-hidden 3.5133.513 2.21×10−42.21× 10^-4∗ 5.3845.384 3.64×10−83.64× 10^-8∗ faction pred-hidden+BGE+Jina+Qwen+Llama pred-hidden 10.66410.664 <10−15<10^-15∗ 17.49517.495 <10−15<10^-15∗ gender pred-hidden pred-hidden 4.6384.638 1.76×10−61.76× 10^-6∗ 5.9805.980 1.12×10−91.12× 10^-9∗ faction pred-hidden pred-hidden 12.68312.683 <10−15<10^-15∗ 20.01020.010 <10−15<10^-15∗ We report four settings (Table 1), which differ in how much of the speech and the summary the embeddings are able to represent. First, we concatenate the four pre-trained embedders (BGE, Jina, Llama, Qwen) for both Z and X, giving q=p=1120q=p=1120, leading to eCITs with diverse pre-trained embeddings but without specifically trained ones. Second, we additionally include the learned embeddings for X and Z, giving q=p=1888q=p=1888. This setting includes all considered embeddings, and aims to minimize residual insufficiency of the embedding of Z. Specifically, the learned representation of the speech is trained to extract the information in Z about Y, and the pre-trained embeddings are general representations of Z, which together aim to satisfy condition (i′), i.e. retaining the conditional mean information of Z about Y. Third, we omit the pre-trained embeddings of the second setting from XωX _X, giving p=1888p=1888 and q=768q=768, evaluating a potential power decrease under a less representative embedding of X, and addressing potential violations of Assumption 3.2. Fourth, we use only the learned embeddings for X and Z, giving q=p=768q=p=768. This setting drops the embedders whose training corpora may overlap with the German Parliament speeches, addressing potential violations of Assumption 3.2 and Assumption 3.2. All sixteen tests reject at α=0.05α=0.05. In the first setting the test statistics are large, at T=24.7T=24.7 (gender) and T=34.7T=34.7 (faction) for the Gemma summaries, and even larger for Qwen (Table 1). Adding the learned representations to ZωZ _Z and XωX _X in the second setting reduces the test statistics strongly. The two settings differ only by the learned representations, so the reduction shows that the learned representation of the speech carries information about Y that the pre-trained embeddings do not, and hence that the first conditioning set is insufficient and at least part of the conditional association detected in this setting is attributable to that insufficiency. Reducing XωX _X to the learned representation in the third setting lowers the statistics slightly further, in line with the smaller signal retained by a less representative embedding of the summary. Dropping the pre-trained embedders from ZωZ _Z in the fourth setting increases the statistics only slightly, which suggests that these embedders carry little about Y beyond what the learned representations already supply. The test decision does not change across the settings: the smallest statistic across all settings, outcomes and summarizers is T=3.5T=3.5, corresponding to p=2.2×10−4p=2.2× 10^-4, and the faction statistics remain above 1010 throughout. We therefore find that the summaries of both LLMs contain information about the speaker’s faction and gender beyond the speech they were generated from. The statistics are larger for Qwen than for Gemma in all eight comparisons, but T depends on the sufficiency of the embedding maps and the nuisance fits as well as on the strength of the conditional dependence, so we do not read this as a comparison of the two summarizers. We discuss four explanations for the test decision other than conditional dependence. First, the embedding of Z could be insufficient: exact sufficiency is not plausible for embedding maps learned from data, and by Theorem 11 any information in Z about the conditional mean of Y that ZωZ _Z drops can act as an omitted conditioning variable (cf. also the Qwen DGM results in Subsection 4.2). However, the eCITs with a large ZωZ _Z consisting of concatenated pre-trained and learned embeddings, and thus covering both general and Y-specific information in Z, still reject (second setting). Furthermore, adding the four pre-trained embedders to a ZωZ _Z containing only learned embeddings, while holding XωX=pred-hiddenX _X=pred-hidden fixed, changes the test statistics only slightly and leaves every statistic well above the critical value. The pre-trained embedders thus carry little information about Y beyond what the learned representation of the speech already supplies, which suggests that the bias from residual insufficiency is not large here. The corresponding simulation arms are calibrated for categorical Y up to n=10,000n=10,000, but this is limited evidence at the sample sizes used here, since the inflation caused by an insufficient ZωZ _Z grows with n. We can therefore not exclude residual insufficiency, although it would have to concern information that neither the pre-trained embedders nor a classifier trained for the task recovers from the speech. Second, the nuisance regressions on which the validity of the PCM rests could be estimated inaccurately. The dimension of the conditioning set, between p=768p=768 and p=1888p=1888, is large relative to the n2≈26,000n_2≈ 26,000 observations in the evaluation half. However, this is probably not the main driver of the observed statistics, since they do not increase with p, whereas the corresponding bias term does. In particular, the largest statistics are obtained at p=1120p=1120, while the settings with the highest and the lowest dimension, p=1888p=1888 and p=768p=768, give smaller values of comparable size. Furthermore, the simulation study uses conditioning sets of comparable dimensions at the smaller sample size n=10,000n=10,000, hence at a less favorable ratio of p to n2n_2 than here, and the arms with sufficient ZωZ _Z remain calibrated there. Third, the detected conditional dependence could run through the embedding maps rather than the summarizers. The pre-trained embedders are fitted on external corpora, which may themselves contain German Parliament speeches, and the classifier supplying the learned representations is fine-tuned from a pre-trained encoder. The split is drawn at the level of speeches, so the training and test samples share speakers, topics and electoral terms, which can induce dependence between train and test sample. Thus, β β and γ γ might not be independent of the test sample, and both Assumption 3.2 and Assumption 3.2 in the definition of 0suff,meanP_0^suff,mean can fail. The embeddings could then contain information about Y from the embedder rather than the summarizer, so that a rejection need not reflect conditional dependence at the level of the texts. For this to produce a rejection, the effect has to differ between the speech and its summary, since information that the embedder attaches to both is conditioned on through ZωZ _Z. Fine-tuning the learned maps on a holdout split removes the dependence on the test speeches themselves, but neither this nor the choice of split removes the dependence induced by the pre-training corpus of the encoder, so the fourth setting reduces but does not remove this dependence. The agreement across the two summarizers does not speak against this explanation, as the same embedders are used for both. Fourth, the observations could be dependent rather than i.i.d. Speakers recur across speeches and speeches are dependent through shared topics and across time, so the rows of S are not i.i.d. and the attributes Y are constant within speaker. Lemmas 16 and 17 and the asymptotic validity of the PCM all rely on the i.i.d. assumption, and positive dependence between observations can make the PCM anti-conservative. We can therefore not exclude that this dependence contributes to the observed magnitude of the statistics; extending the eCITs to dependent samples is left for future work. The result is consistent with the mechanism of Example 1. Both summarizers were pre-trained on web-scale corpora that plausibly include German Parliament plenary protocols, so speeches in the test set, speeches by the same speakers, or other texts associated with them may have been seen during training. Memorization of these texts then allows the summary to contain information about speaker attributes which is not in the speech. Under all four combinations of ZωZ _Z and XωX _X the eCITs reject for both outcomes and both summarizers, and the changes in the test statistics across the settings follow the expected direction, which weakens some of the alternative explanations for the test decisions. The test thus certifies that the summaries carry additional information about the protected attribute on this task and dataset, with a T1E guarantee under the stated conditions. It does not guarantee sufficiency of ZωZ _Z, localize the information, quantify its magnitude on an interpretable scale, or attribute it to a specific mechanism, including the memorization discussed here. 5 Conclusion We introduced embedded conditional independence tests (eCITs) to test for conditional dependence between a text and a scalar given a text, a setting in which existing CITs cannot be applied with T1E guarantees. We first justified CI testing as a bias criterion for LLM outputs and characterized when the null and alternative can hold for a fixed LLM, illustrating that memorization of individual observations can induce conditional dependence. Existing bias metrics do not provide such guarantees: intrinsic metrics need not correlate with downstream behavior, extrinsic metrics are bound to benchmark datasets with specific characteristics, and LLM-based evaluators can themselves be biased. In contrast, our criterion requires neither a gold-standard output nor an annotation of what an unbiased output would be: the source text supplies the baseline against which additional information is measured. We then characterized sufficient embedding maps for the conditioned text, under which a valid CIT for the embedded hypothesis is a valid CIT for the textual hypothesis, and gave conditions under which the two hypotheses are equivalent. We further showed that the requirement on the embedding maps can be weakened to mean sufficiency when the embedded test targets conditional mean independence, and that for categorical Y, as in our application, the two nulls coincide for the i.i.d. laws considered here. We further proposed a simulation design to assess T1E control and power of the eCITs on a given dataset before applying them. On German Parliament speeches, the design exhibits both regimes it is meant to detect: the eCITs are calibrated for some combinations of embedding maps, while others show T1E inflation that need not vanish, and can grow, with the sample size. Since validity depends on the sufficiency of the embedding map under a given data generating process, the design serves to assess sufficiency for the dataset and embedding maps at hand. Applying the eCITs to the speeches and the summaries of two LLMs, we reject the null for both the speaker’s faction and gender. Four alternative explanations cannot be excluded, although the calibration of comparable arms in the simulation and the changes in test statistics with consistent test decisions both speak against any of them accounting for the rejections on its own. The main open challenge is exact (mean) sufficiency, which is unlikely to hold for pre-trained or learned embedding maps, and even small amounts of residual information can affect the T1E at larger sample sizes, as the simulation suggests. A diagnostic quantifying how much relevant information about Y or XωX _X is left in the source text, rather than certifying that none is, would be of considerable interest. Two further directions remain. We studied only embedding maps learned on a holdout set; (unsupervised) in-sample learning and the dependence it introduces in the test sample deserve theoretical and empirical study. And while we used the PCM, other CITs could be combined with sufficient embedding maps in the same way. eCITs make conditional independence testing available for multimodal data and thereby give statistical guarantees for bias claims about LLM outputs on a specified task and dataset. While our focus has been on summarization and text, our eCITs apply to any setting in which the conditioning variable and the tested variable admit representations satisfying the (mean) sufficiency conditions. acknowledgments-disclosure-of-funding.Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 459422098. Use of AI Tools The LLMs analysed in this work are described in Appendix B; their outputs are the object of study rather than part of the manuscript. In preparing the manuscript, the authors additionally used LLMs to assist with drafting and language editing. The authors take full responsibility for the contents of the paper, including all derivations, results and references. Appendix A Proofs The proofs are based on applying the lemmata from Dawid (1979). For readability, we refer to them as specified in Table 2. Dawid (1979) Statement Name Lemma 4.1 A⟂B|C⇔A⟂(B,C)|CA \!\!\! B C A \!\!\! (B,C) C redundancy Lemma 4.2 (i) A⟂B|C⟹A⟂f(B)|CA \!\!\! B C A \!\!\! f(B) C decomposition Lemma 4.2 (i) A⟂B|C and D=f(B)⟹A⟂B|(C,D)A \!\!\! B C and D=f(B) A \!\!\! B (C,D) weak union Lemma 4.3 A⟂B|C and D⟂B|(A,C)⟹(A,D)⟂B|CA \!\!\! B C and D \!\!\! B (A,C) (A,D) \!\!\! B C contraction Table 2: f denotes any measurable function of B. The general form of Lemma 2 is given in the following: Lemma 14 (Disintegration of conditional independence). Let (EA,ℰA)(E_A,E_A), (EB,ℰB)(E_B,E_B), (EC,ℰC)(E_C,E_C) be standard Borel spaces and write (E,ℰ)=(EA×EB×EC,ℰA⊗ℰB⊗ℰC)(E,E)=(E_A× E_B× E_C,\,E_A _B _C), with coordinate projections A,B,CA,B,C. Let 0:=μ probability measure on (E,ℰ):A⟂B∣C under μ.P_0:= \\,μ probability measure on (E,E)\;:\;A \!\!\! B C under μ\, \. Let V be an E-valued and W a (,)(W,G)-valued random variable on (Ω,ℱ,ℙ)( ,F,P), where (,)(W,G) is an arbitrary measurable space, and let ℙV|WP^V W be a regular conditional distribution of V given W. Then w∈:ℙV|W=w∈0∈\w :P^V W=w _0\ and A⟂B∣(C,W)⟺ℙV|W∈0ℙW-a.s.A \!\!\! B (C,W) ^V W _0 ^W-a.s. Proof. For a bounded measurable h:E→ℝh:E choose, by the Doob–Dynkin lemma, a jointly measurable Φh:×EC→ℝ _h:W× E_C with [h(V)∣C,W]=Φh(W,C)E[h(V) C,W]= _h(W,C) ℙP-a.s.; this requires no assumption on (,)(W,G). Step 1 (two-stage conditioning). We show that for ℙWP^W-a.e. w, the map Φh(w,⋅) _h(w,·) is a version of ℙV|W=w[h∣C]E_P^V W=w[h C]. Fix D∈ℰCD _C and set ψD(w):=∫(Φh(w,c)−h(v))D(c)ℙV|W=w(v),v=(a,b,c), _D(w):= ( _h(w,c)-h(v) )1_D(c)\,P^V W=w(dv), v=(a,b,c), which is measurable in w by joint measurability of Φh _h and of w↦ℙV|W=w(F)w ^V W=w(F), F∈ℰF . By the defining property of the regular conditional distribution, ψD(W)=[(Φh(W,C)−h(V))D(C)∣W] _D(W)=E [( _h(W,C)-h(V))1_D(C) W ] a.s. For every G∈G the variable D(C)G(W)1_D(C)1_G(W) is σ(C,W)σ(C,W)-measurable, whence [(Φh(W,C)−h(V))D(C)G(W)]=0E[( _h(W,C)-h(V))1_D(C)1_G(W)]=0 and therefore [ψD(W)G(W)]=0E[ _D(W)1_G(W)]=0. As ψD(W) _D(W) is σ(W)σ(W)-measurable, ψD(W)=0 _D(W)=0 a.s. Since ECE_C is standard Borel, ℰCE_C is countably generated; applying the above to a countable generating algebra, discarding the union of the corresponding null sets and extending by a monotone class argument for each fixed w yields the claim. Step 2 (reduction to a countable family). Let Akk∈ℕ\A_k\_k and Bll∈ℕ\B_l\_l be countable generating algebras of ℰAE_A and ℰBE_B, and abbreviate fk:=Ak∘Af_k:=1_A_k A, gl:=Bl∘Bg_l:=1_B_l B. By a monotone class argument, A⟂B|(C,W)A \!\!\! B (C,W) holds if and only if Φfkgl=ΦfkΦgl _f_kg_l= _f_k\, _g_l (10) holds at (W,C)(W,C) ℙP-a.s. for all k,lk,l; and, by Step 1, for ℙWP^W-a.e. w the measure ℙV|W=wP^V W=w lies in 0P_0 if and only if (10) holds at (w,C)(w,C) ℙV|W=wP^V W=w-a.s. for all k,lk,l. Step 3 (transfer). Let Nk,l:=(w,v)∈×E:Φfkgl(w,c)≠Φfk(w,c)Φgl(w,c)∈⊗ℰ,N_k,l:= \(w,v) × E\;:\; _f_kg_l(w,c)≠ _f_k(w,c) _g_l(w,c) \ , with sections (Nk,l)w(N_k,l)_w. The regular conditional distribution gives ℙ((W,V)∈Nk,l)=∫ℙV|W=w((Nk,l)w)ℙW(w),P ((W,V)∈ N_k,l )= ^V W=w ((N_k,l)_w )\,P^W(dw), so ℙ((W,V)∈Nk,l)=0P((W,V)∈ N_k,l)=0 if and only if ℙV|W=w((Nk,l)w)=0P^V W=w((N_k,l)_w)=0 for ℙWP^W-a.e. w. Taking the union over the countably many pairs (k,l)(k,l) and combining with Step 2 proves the equivalence. Finally, w:ℙV|W=w∈0=⋂k,lw:ℙV|W=w((Nk,l)w)=0∈, \w:P^V W=w _0 \= _k,l \w:P^V W=w ((N_k,l)_w )=0 \ , since w↦ℙV|W=w((Nk,l)w)w ^V W=w((N_k,l)_w) is measurable for each (k,l)(k,l) by Fubini’s theorem for kernels. ∎ Lemma 15 (Disintegration of conditional mean independence). Let (EA,ℰA),(EB,ℰB),(EC,ℰC)(E_A,E_A),(E_B,E_B),(E_C,E_C) be standard Borel with EB⊆ℝdE_B ^d Borel, and write (E,ℰ)=(EA×EB×EC,ℰA⊗ℰB⊗ℰC)(E,E)=(E_A× E_B× E_C,E_A _B _C) with coordinate projections A,B,CA,B,C. Let 0mean:=μ probability measure on (E,ℰ):∫∥b∥μ(dv)<∞,μ[B∣A,C]=μ[B∣C]μ-a.s..P_0^mean:= \μ probability measure on (E,E): \|b\|\,μ(dv)<∞,\ E_μ[B A,C]=E_μ[B C]\ μ-a.s. \. Let V be an E-valued and W a (,)(W,G)-valued random variable on (Ω,ℱ,ℙ)( ,F,P) with ‖B(V)‖<∞E\|B(V)\|<∞, where (,)(W,G) is an arbitrary measurable space, and let ℙV|WP^V W be a regular conditional distribution of V given W. Then w∈:ℙV|W=w∈0mean∈\w :P^V W=w _0^mean\ and [B∣A,C,W]=[B∣C,W]a.s.⇔ℙV|W∈0meanℙW-a.s.E[B A,C,W]=E[B C,W]\ a.s. ^V W _0^mean\ \ P^W-a.s. Proof. It suffices to treat d=1d=1, applying the result to each coordinate of B and intersecting the countably many resulting null sets. By the Doob–Dynkin lemma choose jointly measurable Φ1:×EA×EC→ℝ _1:W× E_A× E_C and Φ2:×EC→ℝ _2:W× E_C with [B∣A,C,W]=Φ1(W,A,C)E[B A,C,W]= _1(W,A,C) and [B∣C,W]=Φ2(W,C)E[B C,W]= _2(W,C) ℙP-a.s.; this requires no assumption on (,)(W,G). Step 1. Step 1 of Lemma 14 applies verbatim to integrable rather than bounded h: by Fubini’s theorem for kernels, ‖B(V)‖<∞E\|B(V)\|<∞ gives ∫‖b‖ℙV|W=w(v)<∞ \|b\|\,P^V W=w(dv)<∞ for ℙWP^W-a.e. w, and dominated convergence replaces boundedness in the monotone class step. Applying it once with conditioning field σ(A,C)σ(A,C) and once with σ(C)σ(C) yields that, for ℙWP^W-a.e. w, the maps Φ1(w,⋅,⋅) _1(w,·,·) and Φ2(w,⋅) _2(w,·) are versions of ℙV|W=w[B∣A,C]E_P^V W=w[B A,C] and ℙV|W=w[B∣C]E_P^V W=w[B C], respectively. In particular ℙV|W=w∈0meanP^V W=w _0^mean if and only if Φ1(w,A,C)=Φ2(w,C) _1(w,A,C)= _2(w,C) holds ℙV|W=wP^V W=w-a.s. Step 2. Let N:=(w,v)∈×E:Φ1(w,a,c)≠Φ2(w,c)∈⊗ℰN:=\(w,v) × E: _1(w,a,c)≠ _2(w,c)\ , with sections NwN_w. The regular conditional distribution gives ℙ((W,V)∈N)=∫ℙV|W=w(Nw)ℙW(w),P ((W,V)∈ N )= ^V W=w(N_w)\,P^W(dw), so ℙ((W,V)∈N)=0P((W,V)∈ N)=0 if and only if ℙV|W=w(Nw)=0P^V W=w(N_w)=0 for ℙWP^W-a.e. w. Combined with Step 1 this is the asserted equivalence. Finally w:ℙV|W=w∈0mean=w:ℙV|W=w(Nw)=0∩w:∫‖b‖ℙV|W=w(v)<∞∈\w:P^V W=w _0^mean\=\w:P^V W=w(N_w)=0\∩\w: \|b\|\,P^V W=w(dv)<∞\ , since w↦ℙV|W=w(Nw)w ^V W=w(N_w) and w↦∫‖b‖ℙV|W=w(v)w \|b\|\,P^V W=w(dv) are measurable by Fubini’s theorem for kernels. ∎ Lemma 16 (Lifting to the sample level). Let (Xi,Yi,Zi)i=1n(X_i,Y_i,Z_i)_i=1^n be i.i.d. copies of (X,Y,Z)(X,Y,Z) with values in the standard Borel spaces (,ℱ)(X,F_X), (,ℱ)(Y,F_Y), (,ℱ)(Z,F_Z). Then X⟂Y∣Z⟺Xn⟂Yn∣Zn.X \!\!\! Y Z X^n \!\!\! Y^n Z^n. Proof of Lemma 16. “⇒ ”. Since the rows (Xi,Yi,Zi)(X_i,Y_i,Z_i) are independent, for bounded measurable hi:×→ℝh_i:X×Y we have [∏i=1nhi(Xi,Yi)|Zn]=∏i=1n[hi(Xi,Yi)∣Zi]a.s.E [ _i=1^nh_i(X_i,Y_i)\, |\,Z^n ]= _i=1^nE [h_i(X_i,Y_i) Z_i ] .s. (11) Indeed, the right-hand side is σ(Zn)σ(Z^n)-measurable, and for ϕ(Zn)=∏iϕi(Zi)φ(Z^n)= _i _i(Z_i) with bounded measurable ϕi _i both sides have the same expectation against ϕ(Zn)φ(Z^n) by independence across i; such ϕφ generate a π-system generating σ(Zn)σ(Z^n), so (11) follows by a functional monotone class argument. Now let fi,gif_i,g_i be bounded and measurable and apply (11) with hi(x,y)=fi(x)gi(y)h_i(x,y)=f_i(x)g_i(y). By X⟂Y|ZX \!\!\! Y Z applied row-wise, [fi(Xi)gi(Yi)∣Zi]=[fi(Xi)∣Zi][gi(Yi)∣Zi]E[f_i(X_i)g_i(Y_i) Z_i]=E[f_i(X_i) Z_i]\,E[g_i(Y_i) Z_i], so regrouping the product and applying (11) twice more, once with gi≡1g_i≡ 1 and once with fi≡1f_i≡ 1, gives [∏ifi(Xi)∏igi(Yi)|Zn]=[∏ifi(Xi)|Zn][∏igi(Yi)|Zn].E [ _if_i(X_i) _ig_i(Y_i)\, |\,Z^n ]=E [ _if_i(X_i)\, |\,Z^n ]\,E [ _ig_i(Y_i)\, |\,Z^n ]. Functions of the form ∏ifi _if_i with fi=Aif_i=1_A_i, Ai∈ℱA_i _X, form a π-system generating ℱ⊗nF_X n, and analogously for YnY^n; a functional monotone class argument in each argument separately extends the display to all bounded ℱ⊗nF_X n- and ℱ⊗nF_Y n-measurable functions, which is Xn⟂Yn|ZnX^n \!\!\! Y^n Z^n. “⇐ ”. Decomposition applied twice gives X1⟂Y1|ZnX_1 \!\!\! Y_1 Z^n. Since (X1,Y1,Z1)⟂Z2:n(X_1,Y_1,Z_1) \!\!\! Z_2:n, redundancy and contraction give X1⟂Y1|Z1X_1 \!\!\! Y_1 Z_1, and (X1,Y1,Z1)=(X,Y,Z)(X_1,Y_1,Z_1) d=(X,Y,Z). ∎ Lemma 17 (Lifting to the sample level, mean version). Let (Xi,Yi,Zi)i=1n(X_i,Y_i,Z_i)_i=1^n be i.i.d. copies of (X,Y,Z)(X,Y,Z) with values in the standard Borel spaces (,ℱ)(X,F_X), (,ℱ)(Y,F_Y), (,ℱ)(Z,F_Z), with ⊆ℝKY ^K Borel and ‖Y‖<∞E\|Y\|<∞. Then [Y∣X,Z]=[Y∣Z]a.s.⇔[Yi∣Xn,Zn]=[Yi∣Zn]a.s. for all i≤n.E[Y X,Z]=E[Y Z]\ a.s. [Y_i X^n,Z^n]=E[Y_i Z^n]\ a.s. for all i≤ n. Proof. Fix i≤ni≤ n. Since (Xi,Yi,Zi)(X_i,Y_i,Z_i) is independent of (X,Z)−in(X,Z)^n_-i, weak union applied with the measurable function (Xi,Zi)(X_i,Z_i) of (Xi,Yi,Zi)(X_i,Y_i,Z_i), followed by decomposition, gives Yi⟂(X,Z)−in|(Xi,Zi)Y_i \!\!\! (X,Z)^n_-i (X_i,Z_i) and Yi⟂Z−in|ZiY_i \!\!\! Z^n_-i Z_i. Hence [Yi∣Xn,Zn]=[Yi∣Xi,Zi],[Yi∣Zn]=[Yi∣Zi]a.s.E[Y_i X^n,Z^n]=E[Y_i X_i,Z_i], [Y_i Z^n]=E[Y_i Z_i] .s. (12) Both conditional expectations are well defined since ‖Y‖<∞E\|Y\|<∞. The two right-hand sides agree a.s. for every i if and only if [Y∣X,Z]=[Y∣Z]E[Y X,Z]=E[Y Z] a.s., because (Xi,Yi,Zi)=d(X,Y,Z)(X_i,Y_i,Z_i) d=(X,Y,Z); by (12) this is the asserted equivalence. ∎ Appendix B Application B.1 Source Texts, LLMs, and Summary Generation We clean the German Parliament speeches from Richter et al. (2020) by removing parenthetical text such as applause or laughter, deleting common procedural phrases (e.g., “Das Wort hat”), stripping out text between dashes and extraneous whitespace, and normalizing quotation marks. Furthermore, we consider speeches as valid after filtering by minimum word count (50) and minimum sentences (2), and excluding speeches containing procedural indicators. For each speaker, we merge consecutive speeches if no valid speech from another speaker intervenes. We treat invalid interruptions from other speakers as [UNTERBRECHUNG/REDE:…] appended to the main speech. We randomly sample cleaned speeches and manually inspect for remaining procedural or invalid content. Next, we summarize the cleaned speeches Z with the LLM Gemma 4 12B (Gemma Team, 2026), used throughout the simulation study (Subsection 4.2) and the real-world application (Subsection 4.3). We allow for a maximum of 256 generated tokens per summary (3–4 sentences) and use greedy decoding so that the summaries are reproducible. For the real-world application (Subsection 4.3), we also summarize the source texts Z with the LLM Qwen 3.6 27B (Qwen Team, 2026). We again allow for a maximum of 256 generated tokens per summary (3–4 sentences) and sample with temperature 0.70.7, top-p 0.80.8, and top-k 2020, the recommended decoding parameters for the model’s non-thinking (Instruct) mode. The summaries X are generated by prompting the LLM with Du bist ein erfahrener politischer Analyst. Deine Aufgabe ist es, Reden aus dem deutschen Bundestag prägnant und objektiv in einem Fließtext zusammenzufassen. Fokussiere auf die Hauptargumente und politischen Positionen. Die mit ’(\ +)(\ d+\)’ gekennzeichneten Bemerkungen im Text beziehen sich auf mögliche Zwischenrufe oder kleinere Beiträge von anderen Abgeordneten. Die Rede kann außerdem [REDE] und [UNTERBRECHUNG] Marker enthalten. [REDE] markiert Teile der Hauptrede. [UNTERBRECHUNG] markiert Zwischenkommentare oder Fragen. WICHTIG: Gib NUR die Zusammenfassung zurück, ohne einleitende Phrasen wie ’Hier ist eine Zusammenfassung’ oder Ähnliches. B.2 Embedding Maps We embed speeches Z and summaries X with four multilingual models, using the same encoders in the simulation study (Subsection 4.2) and the real-world application (Subsection 4.3): BGE-M3 (Chen et al., 2024) at its native 10241024 dimensions; jina-embeddings-v5-text-small (Akram et al., 2026) with the classification adapter, truncated to 3232 dimensions; Llama Nemotron Embed 1B v2 (NVIDIA, 2025), truncated to 3232 dimensions; and Qwen3-Embedding-8B (Zhang et al., 2025), likewise truncated to 3232 dimensions. The three 3232-dimensional representations use Matryoshka truncation of the native embedding (Kusupati et al., 2022). Each embedding is ℓ2 _2-normalized. We then column-standardize the coordinates (train-fitted mean and standard deviation) so that features enter the simulation DGP for Y on a comparable scale, and because ℓ2 _2-penalized PCM nuisances are sensitive to the scale of X and Z. As additional representations we use learned embedding maps: we fit the ModernGBERT 134M (Wunderle et al., 2025), a German encoder with 768768-dimensional hidden states, to predict Y from the raw speech or summary, and take the masked mean of the last hidden layer. In the real-world application these models predict faction or gender on the training split, and the 768768-dimensional hidden states on the test split are supplied to the CIT, either alone or concatenated with the pretrained embeddings. In the simulation study the same architecture is trained to predict the simulated outcome, and the resulting hidden states are used in the simulation. In the simulation study we generate Y from a single 3232-dimensional embedding of the Gemma summaries (Jina or Qwen3) and supply the CIT with the following concatenations of ZωZ _Z and XωX _X. Oracle uses only that DGP pair: Zω~Z,Xω~X∈ℝ32Z ω_Z,X ω_X ^32. The diluted eCITs prepend the generating embedder to three further concatenations: BGE with the two remaining 3232-dimensional models (ZωZ,XωX∈ℝ1120Z _Z,X _X ^1120); the 768768-dimensional learned speech hidden state, BGE, and those two models as ZωZ∈ℝ1888Z _Z ^1888, and the learned summary hidden state with those two models as XωX∈ℝ864X _X ^864; and the learned hidden states alone (ZωZ,XωX∈ℝ800Z _Z,X _X ^800). The eCITs omitting the generating embedder use the same three stacks without the DGP embedder: BGE with the two remaining 3232-dimensional models (ZωZ,XωX∈ℝ1088Z _Z,X _X ^1088); the learned speech hidden state, BGE, and those two models as ZωZ∈ℝ1856Z _Z ^1856, and the learned summary hidden state with those two models as XωX∈ℝ832X _X ^832; and the learned hidden states alone (ZωZ,XωX∈ℝ768Z _Z,X _X ^768). B.3 Outcome Generation in the Simulation Study We set αZ=αX=1 _Z= _X=1 throughout. For the continuous outcome, the noise variance is σ2=(1−r2)/r2σ^2=(1-r^2)/r^2 with r2=0.4r^2=0.4, so that sZs_Z explains 40% of the variance of Y under the null; for the binary and multinomial outcomes the noise is determined by the link, so that the three outcome types are not on a common signal scale. Write s¯Z s_Z for the training-split mean of the speech score. The binary intercept is b=−αZs¯Zb=- _Z s_Z. For the multinomial outcome, the reference class has logit ηi,0=0 _i,0=0 and the remaining logits are ηi,k=cos(2πkL)αZ(sZ,i−s¯Z)+csin(2πkL)αXsX,i,k=1,…,L−1, _i,k= \! ( 2π kL ) _Z(s_Z,i- s_Z)+c\, \! ( 2π kL ) _Xs_X,i, k=1,…,L-1, so that the speech and the summary score load on the classes in orthogonal directions. The logits are clipped to [−10,10][-10,10] before the softmax, and the resulting probabilities are floored at 10−610^-6 and renormalized. Appendix C Detailed Simulation Results C.1 Jina Embedding Data Generating Mechanism Figure 3: The rejection rates of the eCIT for binary, multinomial and continuous Y generated under the null (first row) and alternative (second row) from the Jina embeddings ZJinaZ^Jina and XJinaX^Jina. The eCIT is applied with the Jina embeddings (oracle), the specified diluted embeddings, including additionally learned, BGE, Qwen, and Llama embeddings, as well as the same combinations excluding the Jina embedding. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation set unpenalized; for categorical Y the regressions of Y are multinomial logistic and those of the fitted probabilities and of f f are least squares (Appendix D.1). The dashed line marks the nominal level α=0.05α=0.05, the shaded area marking two MC standard errors around it. Figure 4: Q-plots of the observed p-values against the theoretical quantiles of a uniform distribution on the interval [0,1][0,1] for eCITs with the specified embeddings. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation set unpenalized; for categorical Y the regressions of Y are multinomial logistic and those of the fitted probabilities and of f f are least squares (Appendix D.1). Y is generated under the null hypothesis from the Jina embedding ZJinaZ^Jina at selected sample sizes (indicated in the panel titles). The diagonal black lines y=xy=x serve as references for the theoretical quantiles of the uniform distribution on [0,1][0,1]. If the p-values are uniformly distributed, they should align along this line. C.2 Qwen Embedding Data Generating Mechanism Figure 5: The rejection rates of the eCIT for binary, multinomial and continuous Y generated under the null (first row) and alternative (second row) from the Qwen embeddings ZQwenZ^Qwen and XQwenX^Qwen. The eCIT is applied with the Qwen embeddings (oracle), the specified diluted embeddings, including additionally learned, BGE, Jina, and Llama embeddings, as well as the same combinations excluding the Qwen embedding. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation set unpenalized; for categorical Y the regressions of Y are multinomial logistic and those of the fitted probabilities and of f f are least squares (Appendix D.1). The dashed line marks the nominal level α=0.05α=0.05, the shaded area marking two MC standard errors around it. Figure 6: Q-plots of the observed p-values against the theoretical quantiles of a uniform distribution on the interval [0,1][0,1] for eCITs with the specified embeddings. For the PCM, the nuisance regressions on the training dataset are L2L_2-penalized and those on the evaluation set unpenalized; for categorical Y the regressions of Y are multinomial logistic and those of the fitted probabilities and of f f are least squares (Appendix D.1). Y is generated under the null hypothesis from the Qwen embedding ZQwenZ^Qwen at selected sample sizes (indicated in the panel titles). The diagonal black lines y=xy=x serve as references for the theoretical quantiles of the uniform distribution on [0,1][0,1]. If the p-values are uniformly distributed, they should align along this line. Appendix D The PCM for Categorical Y We suppress the embedding maps throughout this appendix and write X∈⊆ℝqX ^q, Z∈⊆ℝpZ ^p for XωX _X, ZωZ _Z, so that p=dim()p= (Z). Let Y take L values, drop a baseline class and let Y~∈0,1K Y∈\0,1\^K, K=L−1K=L-1, be the reduced vector of class indicators of Remark 13. Write g(x,z)=[Y~∣X=x,Z=z],mY(z)=[Y~∣Z=z],h=g−mY,g(x,z)=E[ Y X=x,Z=z], m_Y(z)=E[ Y Z=z], h=g-m_Y, so that g is the reduced conditional probability mass function of Y and mYm_Y is the population version of the regression mim_i of Subsection 3.2, and let Σ(x,z)=Var(Y~∣X=x,Z=z)=diagg(x,z)−g(x,z)g(x,z)⊤ (x,z)=Var( Y X=x,Z=z)=diag\g(x,z)\-g(x,z)g(x,z) . D.1 The Categorical PCM Following Lundborg et al. (2024), we split the sample into halves D1D_1, D2D_2 of sizes n1n_1, n2n_2. On D1D_1 we fit g g, a multinomial (L2L_2-penalized) logistic regression of Y on (X,Z)(X,Z), and write g^(x,z)∈ℝK g(x,z) ^K for its vector of fitted probabilities with the baseline class dropped, so that g g estimates g. Then we fit m~ m, a column-wise (L2L_2-penalized) least-squares regression of the fitted probabilities g g on Z, and set h^=g^−m~ h= g- m and f^(x,z)=(Σ^(x,z)+c^IK)−1h^(x,z)∈ℝK, f(x,z)= ( (x,z)+ c\,I_K )^-1 h(x,z) ^K, (13) where Σ is Σ evaluated at g^(x,z) g(x,z), the entries of the full fitted probability vector being floored at ε=10−6 =10^-6 and renormalized before the baseline class is dropped. The ridge penalty c c is selected on D1D_1. It is needed because class probabilities close to 00 or 11, as under quasi-complete separation, leave Σ nearly singular and the inverse in (13) unstable. We therefore split D1D_1 into D1aD_1a and D1bD_1b, fit g g and m~ m on D1aD_1a, and for each candidate value of c on a logarithmically spaced grid in [10−6,1][10^-6,1] form the direction (13) and the corresponding scores ℓi _i of (14) on D1bD_1b, with the nuisance regressions m^Y m_Y and m^f m_f fitted on D1bD_1b as well. We select the candidate maximizing ℓ¯/σ^ℓ / σ_ on D1bD_1b and refit g g, m~ m and the resulting direction on the whole of D1D_1 at the selected c c. Since c c is a function of D1D_1 alone, f f remains D1D_1-measurable and the selection affects the power of the test but not the argument for its level. On D2D_2 we fit m^Y m_Y, an unpenalized multinomial logistic regression of Y on Z, again reduced to its K non-baseline fitted probabilities, and m^f m_f, an unpenalized least-squares regression of f f on Z, and form r^iY=Y~i−m^Y(Zi) r^Y_i= Y_i- m_Y(Z_i) and r^if=f^(Xi,Zi)−m^f(Zi) r^f_i= f(X_i,Z_i)- m_f(Z_i). The test statistic is T=n2ℓ¯σ^ℓ,ℓi=⟨r^iY,r^if⟩,ℓ¯=1n2∑i∈D2ℓi,σ^ℓ2=1n2∑i∈D2(ℓi−ℓ¯)2,T= n_2\, σ_ , _i= r^Y_i, r^f_i , = 1n_2 _i∈ D_2 _i, σ_ ^2= 1n_2 _i∈ D_2( _i- )^2, (14) and we reject at level α when T>z1−αT>z_1-α. This is Algorithm 1 of Lundborg et al. (2024), as implemented for continuous Y in the R package comets (Kook and Lundborg, 2024), with their scalar direction replaced by (13) and their product r^iYr^if r^Y_i r^f_i by an inner product. The ridge constant c c has no counterpart there, and is required here to invert Σ stably. D.2 Validity The summands ℓi _i in (14) are scalar, since the K-vector residuals are projected onto one dimension by the inner product before averaging, so the numerator remains an average of real-valued summands that are conditionally independent and, under the null, conditionally mean-zero given D1D_1. The argument of Lundborg et al. (2024, Theorem 4) is therefore available with their product ϵiξi _i _i replaced by ⟨ϵi,ξi⟩ _i, _i , provided the conditions it places on the nuisance estimates are read in vector form. We record these conditions below; a detailed verification, including vector restatements of the supporting lemmas, is left to future work. assumption Let ϵ=Y~−g(X,Z)ε= Y-g(X,Z), mf(z)=[f^(X,Z)∣Z=z,D1]m_f(z)=E[ f(X,Z) Z=z,D_1], σ2=Var(⟨ϵ,f^(X,Z)−mf(Z)⟩∣D1)σ^2=Var( ε, f(X,Z)-m_f(Z) D_1), E1=[∥m^Y(Z)−mY(Z)∥22∣D1]E_1=E[ m_Y(Z)-m_Y(Z) _2^2 D_1] and E2=σ−2[∥m^f(Z)−mf(Z)∥22∣D1]E_2=σ^-2\,E[ m_f(Z)-m_f(Z) _2^2 D_1]. Assume (a) E1E2=oℙ(n2−1)E_1E_2=o_P(n_2^-1); (b) σ−(2+δ)[∥f^(X,Z)−mf(Z)∥22+δ∣D1]=Oℙ(1)σ^-(2+δ)\,E[ f(X,Z)-m_f(Z) _2^2+δ D_1]=O_P(1) for some δ>0δ>0; and (c) λmin(Σ(X,Z))≥c0 _ ( (X,Z))≥ c_0 almost surely for some c0>0c_0>0. Two features of the categorical case are worth noting. First, by Remark 13 the conditional mean of Y~ Y is the conditional distribution of Y, so conditional mean independence of Y~ Y and X⟂Y|ZX \!\!\! Y Z are the same statement and the mean-level null class coincides with the CI null class for the i.i.d. laws considered in Corollary 12. The PCM therefore tests the CI hypothesis itself here, and nothing is lost by using a CMIT. Second, Y~ Y is bounded, which makes the moment condition (b) a statement about ∥f^−mf∥2 f-m_f _2 alone. Condition (c), by contrast, requires every class probability to be bounded away from 00 and 11, which is a substantive restriction when L is large and some classes are rare. We do not claim uniform asymptotic level over a nonparametric class for this variant. Null calibration is instead assessed by simulation in Subsection 4.2 and Appendix C. D.3 The Precision-Weighted Direction The weighting in (13) is motivated by the following, which identifies the direction maximizing the population signal-to-noise ratio. At K=1K=1 it gives f∝h/vf h/v with v=Var(Y~∣X,Z)v=Var( Y X,Z). Proposition 18. Suppose Σ(X,Z) (X,Z) is almost surely positive definite with 0<[h⊤Σ−1h]<∞0<E[h ^-1h]<∞, and let ℱF be the set of measurable f:×→ℝKf:X×Z ^K with 0<[f⊤Σf]<∞0<E[f f]<∞, all functions being evaluated at (X,Z)(X,Z). Then supf∈ℱ(⟨h,f⟩)2[f⊤Σf]=[h⊤Σ−1h], _f (E h,f )^2E [f f ]=E [h ^-1h ], (15) and the supremum is attained at f∈ℱf if and only if f=κΣ−1hf=κ\, ^-1h almost surely for some κ>0κ>0. Proof. Let Σ1/2 ^1/2 be the symmetric positive definite square root of Σ and put u=Σ−1/2hu= ^-1/2h, w=Σ1/2fw= ^1/2f, which are measurable since A↦A±1/2A A^± 1/2 is continuous on the open cone of positive definite matrices, and satisfy ∥u∥22=[h⊤Σ−1h]E u _2^2=E[h ^-1h] and ∥w∥22=[f⊤Σf]E w _2^2=E[f f], both in (0,∞)(0,∞). Pointwise, ⟨h,f⟩=⟨u,w⟩≤∥u∥2∥w∥2 h,f = u,w ≤ u _2 w _2 almost surely, so taking expectations and applying Cauchy–Schwarz in L2(ℙ)L^2(P), ⟨h,f⟩≤[∥u∥2∥w∥2]≤([h⊤Σ−1h])1/2([f⊤Σf])1/2.E h,f \;≤\;E [ u _2 w _2 ]\;≤\; (E[h ^-1h] )^1/2 (E[f f] )^1/2. (16) Since f and −f-f share the denominator in (15), we may assume ⟨h,f⟩>0E h,f >0; squaring and dividing bounds the left-hand side of (15) by [h⊤Σ−1h]E[h ^-1h], with equality at f=Σ−1h∈ℱf= ^-1h . If f∈ℱf attains the supremum, both inequalities in (16) are equalities. The first forces ⟨u,w⟩=∥u∥2∥w∥2 u,w = u _2 w _2 almost surely, the second ∥w∥2=κ∥u∥2 w _2=κ u _2 almost surely for some κ∈[0,∞)κ∈[0,∞), with κ>0κ>0 since ∥w∥22>0E w _2^2>0. On u≠0\u≠ 0\ the equality case in ℝKR^K gives w=κuw=κ u, and on u=0\u=0\ we have w=0w=0, so Σ1/2f=κΣ−1/2h ^1/2f=κ ^-1/2h almost surely. The converse is the computation above. ∎ Remark 19. The constant c c in (13) serves only to invert Σ stably when fitted probabilities are close to the boundary, and has no counterpart in Algorithm 1 of Lundborg et al. (2024), whose Step 2(i) calibrates a constant of the same name against a variance identity. We apply no sign(ρ)sign(ρ) correction. Since Σ^+c^IK + cI_K is positive definite, ⟨h^i,f^i⟩≥0 h_i, f_i ≥ 0 by construction. D.4 Nuisance Estimation on the Evaluation Half The evaluation-half nuisances are fitted unpenalized and in sample, which yields an exact finite-sample orthogonality. Let W∈ℝn2×(p+1)W ^n_2×(p+1) have rows Wi=(1,Zi⊤)W_i=(1,Z_i ) and collect the residuals in R^Y,R^f∈ℝn2×K R^Y, R^f ^n_2× K. The score equations of the multinomial logistic fit give W⊤R^Y=0W R^Y=0 and the normal equations of the least-squares fit give W⊤R^f=0W R^f=0. Since m^f(Zi)=WiB m_f(Z_i)=W_i B for some B^∈ℝ(p+1)×K B ^(p+1)× K, the first identity yields ∑i∈D2⟨r^iY,r^if⟩=∑i∈D2⟨r^iY,f^(Xi,Zi)⟩, _i∈ D_2 r^Y_i, r^f_i = _i∈ D_2 r^Y_i, f(X_i,Z_i) , (17) so the regression of f f on Z does not contribute to the numerator. Under the null, writing r^iY=ϵi−Mi r^Y_i= _i-M_i with Mi=m^Y(Zi)−mY(Zi)M_i= m_Y(Z_i)-m_Y(Z_i), the numerator splits as ∑i∈D2⟨ϵi,r^if⟩−∑i∈D2⟨Mi,r^if⟩ _i∈ D_2 _i, r^f_i - _i∈ D_2 M_i, r^f_i , in which the first term is conditionally mean-zero given (X,Z)n2(X,Z)^n_2 and D1D_1, because r^f r^f does not depend on Y on D2D_2, leaving the second as the only bias term. A Cauchy–Schwarz bound on that term is of order n2(E1E2)1/2 n_2\,(E_1E_2)^1/2, which is negligible under Assumption D.2(a) and requires the two nuisance errors to be small relative to n2−1/2n_2^-1/2 jointly. Each evaluation-half nuisance is an unpenalized fit with K(p+1)K(p+1) free parameters, and in Subsection 4.3 the conditioning sets consist of high-dimensional embeddings with p between 768768 and 18881888 at n2n_2 of roughly 25,00025,000 to 27,00027,000, so p is not small relative to n2 n_2 and we cannot appeal to this bound there. It is sufficient rather than necessary, and conservative in that it ignores the orthogonality (17). The simulations in Subsection 4.2 use conditioning sets of comparable dimension at smaller n2n_2, hence at a less favorable ratio, and the arms with sufficient ZωZ _Z remain calibrated there, which suggests that the bound overstates the requirement at these dimensions. Misspecification of mYm_Y on Z is the more serious concern, since it is not addressed by the orthogonality. The simulation does not separate it from mean insufficiency: in the arms retaining the generating embedder the outcome is a link applied to a linear score in that embedding, so the linear nuisances are close to correctly specified by construction, while in the arms omitting it the conditional mean given the supplied embedding is in general no longer linear, so misspecification and mean insufficiency act together and their contributions to the observed inflation cannot be attributed separately. References Akram et al. (2026) M. K. Akram, S. Sturua, N. Havriushenko, Q. Herreros, M. Günther, M. Werk, and H. Xiao Jina-embeddings-v5-text: task-targeted embedding distillation. External Links: 2602.15547, Link Cited by: §B.2, §4.2.1. Anthis et al. (2024) J. Anthis, K. Lum, M. Ekstrand, A. Feller, A. D’Amour, and C. Tan The impossibility of fair llms. arXiv preprint arXiv:2406.03198. Cited by: §1. Berrett et al. (2020) T. Berrett, Y. Wang, R. Barber, and R. Samworth The conditional permutation test for independence while controlling for confounders. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82 (1), p. 175–197. Cited by: §1. Cai et al. (2025) L. Cai, X. Guo, and W. Zhong Test and measure for partial mean dependence based on machine learning methods. Journal of the American Statistical Association 120 (550), p. 833–845. Cited by: §3.2. Candes et al. (2018) E. Candes, Y. Fan, L. Janson, and J. Lv Panning for gold: ‘Model-x’knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society Series B: Statistical Methodology 80 (3), p. 551–577. Cited by: §1. Chen et al. (2024) J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu M3-embedding: multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the association for computational linguistics: ACL 2024, p. 2318–2335. Cited by: §B.2, §4.2.1. Dai et al. (2022) B. Dai, X. Shen, and W. Pan Significance tests of feature relevance for a black-box learner. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §3.2. Dawid (1979) P. Dawid Conditional independence in statistical theory. Journal of the Royal Statistical Society: Series B (Methodological) 41 (1), p. 1–15. Cited by: Table 2, Appendix A. Delobelle et al. (2022) P. Delobelle, E. K. Tokpo, T. Calders, and B. Berendt Measuring fairness with biased rulers: a comparative study on bias metrics for pre-trained language models. In Proceedings of the 2022 Conference of the North American chapter of the association for computational linguistics, p. 1693–1706. Cited by: §1. Gallegos et al. (2024) I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, S. Kim, F. Dernoncourt, T. Yu, R. Zhang, and N. K. Ahmed Bias and fairness in large language models: a survey. Computational Linguistics, p. 1–79. Cited by: §1. Gao et al. (2025) M. Gao, X. Hu, X. Yin, J. Ruan, X. Pu, and X. Wan Llm-based nlg evaluation: current status and challenges. Computational Linguistics, p. 1–28. Cited by: §1. Gemma Team (2026) Gemma Team Gemma 4 technical report. External Links: 2607.02770, Link Cited by: §B.1, §4.2.1. Gentzkow et al. (2019) M. Gentzkow, J. M. Shapiro, and M. Taddy Measuring group differences in high-dimensional choices: method and application to congressional speech. Econometrica 87 (4), p. 1307–1340. Cited by: §1. Gjorgjevikj et al. (2026) A. Gjorgjevikj, B. K. Seljak, and T. Eftimov On the robustness of multilingual text embedding rankings across learning tasks, languages, and benchmark datasets. arXiv preprint arXiv:2605.31142. Cited by: §4.2.1. Goldfarb-Tarrant et al. (2020) S. Goldfarb-Tarrant, R. Marchant, R. M. Sánchez, M. Pandya, and A. Lopez Intrinsic bias metrics do not correlate with application bias. arXiv preprint arXiv:2012.15859. Cited by: §1. Kook and Lundborg (2024) L. Kook and A. R. Lundborg Algorithm-agnostic significance testing in supervised learning with multimodal data. Briefings in Bioinformatics 25 (6), p. bbae475. Cited by: §D.1, §1. Kusupati et al. (2022) A. Kusupati, G. Bhatt, A. Rege, M. Wallingford, A. Sinha, V. Ramanujan, W. Howard-Snyder, K. Chen, S. Kakade, P. Jain, and A. Farhadi Matryoshka representation learning. In Advances in Neural Information Processing Systems, Cited by: §B.2, §4.2.1. Li and Fan (2020) C. Li and X. Fan On nonparametric conditional independence tests for continuous variables. Wiley Interdisciplinary Reviews: Computational Statistics 12 (3), p. e1489. Cited by: §1. Li et al. (2023) Y. Li, M. Du, R. Song, X. Wang, and Y. Wang A survey on fairness in large language models. arXiv preprint arXiv:2308.10149. Cited by: §1. Lundborg et al. (2024) A. R. Lundborg, I. Kim, R. D. Shah, and R. J. Samworth The projected covariance measure for assumption-lean variable significance testing. The Annals of Statistics 52 (6), p. 2851–2878. Cited by: §D.1, §D.1, §D.2, §1, §1, §3.2, §3.2, Remark 19. Moreira et al. (2024) G. d. S. P. Moreira, R. Osmulski, M. Xu, R. Ak, B. Schifferer, and E. Oldridge NV-retriever: improving text embedding models with effective hard-negative mining. arXiv preprint arXiv:2407.15831. Cited by: §4.2.1. Morris et al. (2019) T. P. Morris, I. R. White, and M. J. Crowther Using simulation studies to evaluate statistical methods. Statistics in medicine 38 (11), p. 2074–2102. Cited by: §4.2.1. NVIDIA (2025) NVIDIA Llama nemotron embedding 1b v2. Note: Hugging Face model repositoryRevision 113abe4acafa848e77ead9c0623205e511932348, accessed 2026-08-11 External Links: Link Cited by: §B.2, §4.2.1. Qwen Team (2026) Qwen Team Qwen3.6-27B: flagship-level coding in a 27B dense model. External Links: Link Cited by: §B.1, §4.2.1. Richter et al. (2020) F. Richter, P. Koch, O. Franke, J. Kraus, F. Kuruc, A. Thiem, J. Högerl, S. Heine, and K. Schöps Open Discourse. Harvard Dataverse. External Links: Document, Link Cited by: §B.1, §4.1. Shah and Peters (2020) R. Shah and J. Peters The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics 48 (3), p. 1514–1538. Cited by: §1, §3. Simnacher et al. (2026) M. Simnacher, X. Xu, H. Park, C. Lippert, and S. Greven Deep nonparametric conditional independence tests for images. Journal of Machine Learning Research 27 (96), p. 1–73. Cited by: §1, §1, §3.1, §3.1, §3.1, §3.2, §3.2, §3, §3, §4.2.1, Remark 6. Strobl et al. (2019) E. Strobl, K. Zhang, and S. Visweswaran Approximate kernel-based conditional independence tests for fast non-parametric causal discovery. Journal of Causal Inference 7 (1). Cited by: §1, §3.2. Sun et al. (2022) T. Sun, J. He, X. Qiu, and X. Huang BERTScore is unfair: on social bias in language model-based metrics for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 3726–3739. Cited by: §1. Williamson et al. (2023) B. D. Williamson, P. B. Gilbert, N. R. Simon, and M. Carone A general framework for inference on algorithm-agnostic variable importance. Journal of the American Statistical Association 118 (543), p. 1645–1658. Cited by: §3.2. Wunderle et al. (2025) J. Wunderle, A. Ehrmanntraut, J. Pfister, F. Jannidis, and A. Hotho New encoders for german trained from scratch: comparing moderngbert with converted llm2vec models. arXiv preprint arXiv:2505.13136. Cited by: §B.2, §4.2.1. Zhang et al. (2011) K. Zhang, J. Peters, D. Janzing, and B. Schölkopf Kernel-based conditional independence test and application in causal discovery. In Uncertainty in Artificial Intelligence (UAI 2011), p. 804–813. Cited by: §1. Zhang et al. (2025) Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §B.2, §4.2.1.