Paper deep dive
Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities
Yichen Xu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 6/21/2026, 7:35:13 AM
Summary
The paper identifies a critical pitfall in using fully generative models (like GANs and LLMs) for causal inference: high predictive fidelity (TSTR) does not guarantee the preservation of causal estimands like the Average Treatment Effect (ATE). The authors formalize this through sensitivity analysis, showing that joint generative objectives often dilute the importance of the outcome mechanism. To remedy this, they propose a 'hybrid synthetic-data framework' that generates covariates separately from the treatment and outcome mechanisms. This approach significantly improves ATE preservation and provides a simulation engine for benchmarking causal estimators (OR, IPW, AIPW, TMLE). Additionally, the paper explores targeted synthetic augmentation to address practical positivity problems by improving conditional-effect estimation in sparse covariate regions.
Entities (10)
Relation Signals (4)
Hybrid Synthetic-Data Framework → uses → Distance-to-Closest-Record (DCR)
confidence 100% · using distance-to-closest-record diagnostics to monitor covariate synthesis
LLM-based Models → candistort → Average Treatment Effect (ATE)
confidence 95% · LLM-based models... can achieve strong TSTR performance while substantially distorting causal estimands such as the average treatment effect (ATE).
Hybrid Synthetic-Data Framework → improves → Average Treatment Effect (ATE)
confidence 95% · Across experiments, hybrid synthetic data substantially improve ATE preservation relative to fully generative baselines
Targeted Synthetic Augmentation → addresses → Positivity Problems
confidence 90% · We further study targeted synthetic augmentation for practical positivity problems
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Synthetic data offers a promising tool for privacy-preserving data release, augmentation, and simulation, but its use in causal inference requires preserving more than predictive fidelity. We show that fully generative tabular synthesizers, including GAN- and LLM-based models, can achieve strong train-on-synthetic-test-on-real performance while substantially distorting causal estimands such as the average treatment effect (ATE). We formalize this failure through sensitivity and tradeoff results showing that ATE preservation requires control of both the generated covariate law and the treatment-effect contrast in the outcome regression. Motivated by this observation, we propose a hybrid synthetic-data framework that generates covariates separately from the treatment and outcome mechanisms, using distance-to-closest-record diagnostics to monitor covariate synthesis and separately learned nuisance models to construct (W, A, Y) triplets. We further study targeted synthetic augmentation for practical positivity problems and characterize when added overlap support helps by improving conditional-effect estimation more than it shifts the covariate distribution. Finally, we develop a synthetic simulation engine for pre-analysis estimator evaluation, enabling finite-sample comparison of OR, IPW, AIPW, and TMLE under realistic covariate structure. Across experiments, hybrid synthetic data substantially improve ATE preservation relative to fully generative baselines and provide a practical diagnostic tool for robust causal analysis.
Tags
Links
- Source: https://arxiv.org/abs/2604.23904v1
- Canonical: https://arxiv.org/abs/2604.23904v1
Trouble viewing inline? Open PDF directly →
Full Text
56,800 characters extracted from source content.
Expand or collapse full text
Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities Yichen Xu University of California, Berkeley yichen_xu@berkeley.edu Abstract Synthetic data offers a promising tool for privacy-preserving data release, augmentation, and simulation, but its use in causal inference requires preserving more than predictive fidelity. We show that fully generative tabular synthesizers, including GAN- and LLM-based models, can achieve strong train-on-synthetic-test-on-real performance while substantially distorting causal estimands such as the average treatment effect (ATE). We formalize this failure through sensitivity and tradeoff results showing that ATE preservation requires control of both the generated covariate law and the treatment-effect contrast in the outcome regression. Motivated by this observation, we propose a hybrid synthetic-data framework that generates covariates separately from the treatment and outcome mechanisms, using distance-to-closest-record diagnostics to monitor covariate synthesis and separately learned nuisance models to construct (W,A,Y)(W,A,Y) triplets. We further study targeted synthetic augmentation for practical positivity problems and characterize when added overlap support helps by improving conditional-effect estimation more than it shifts the covariate distribution. Finally, we develop a synthetic simulation engine for pre-analysis estimator evaluation, enabling finite-sample comparison of OR, IPW, AIPW, and TMLE under realistic covariate structure. Across experiments, hybrid synthetic data substantially improve ATE preservation relative to fully generative baselines and provide a practical diagnostic tool for robust causal analysis. 1 Introduction Synthetic data is increasingly used for data sharing, privacy protection, augmentation, and simulation, but its role in causal inference remains underdeveloped. This gap is important because causal analysis requires more than reproducing the observed data distribution. For the average treatment effect (ATE), the key object is the conditional contrast Q(1,W)−Q(0,W),Q(a,W)=[Y∣A=a,W],Q(1,W)-Q(0,W), Q(a,W)=E[Y A=a,W], together with the covariate law of W. A synthetic dataset can therefore look realistic, achieve strong train-on-synthetic-test-on-real (TSTR) performance, or remain close to the original data under privacy-distance diagnostics, while still distorting the causal estimand. We identify and formalize this pitfall through sensitivity and tradeoff results showing that predictive or reconstruction fidelity does not imply preservation of causal estimands. Large language models (LLMs) create new opportunities for causal synthetic-data pipelines because tabular rows can be serialized and modeled flexibly. However, a fully joint generator of (W,A,Y)(W,A,Y) is not automatically aligned with causal estimation. In high-dimensional tabular rows, a row-level reconstruction or next-token objective can be dominated by reconstruction of the covariates, while the outcome mechanism receives comparatively little weight. Motivated by this observation, we propose a hybrid generation strategy: instead of generating the full triplet (W,A,Y)(W,A,Y) from one model, we use a generative model for covariates W, monitor the synthetic covariates using distance-to-closest-record diagnostics, and fit the treatment and outcome mechanisms separately. This separation allows flexible covariate synthesis while giving direct modeling attention to the nuisance components that determine the ATE. Empirically, hybrid generation substantially improves causal fidelity for both LLM- and GAN-based generators, often reducing ATE MSE by an order of magnitude while keeping privacy-distance diagnostics comparable. Synthetic data can also support causal estimation under limited overlap or practical positivity problems. In rare treatment-covariate regimes, outcome regression Q(A,W)Q(A,W) can be unstable and weighting estimators can suffer from extreme propensities. We introduce a targeted synthetic augmentation strategy that pairs extreme-propensity observations with nearby synthetic covariates to improve balance in sparse but plausible regions of the covariate space. This use should not be interpreted as solving structural positivity violations; synthetic augmentation remains extrapolative when the treatment truly has no support in a region. Instead, we characterize the tradeoff between improved conditional-effect estimation and covariate distribution shift, and show empirically that targeted synthetic pairing can reduce error when the gain in estimating the conditional effect outweighs the shift introduced by augmentation. Finally, synthetic data can provide a realistic simulation engine for estimator evaluation. Real datasets rarely offer repeated experiments with known ground truth, while hand-crafted simulations often miss the complexity of real covariate distributions. We develop a hybrid synthetic simulation engine that generates repeated finite-sample datasets from a learned data-generating environment, enabling practical comparison of estimator bias, variance, RMSE, and MSE before the final real-data analysis. We use this engine to benchmark OR, IPW, AIPW, and TMLE, showing that well-aligned hybrid synthetic data can reproduce realistic finite-sample estimator behavior more faithfully than simpler or poorly aligned generative baselines. Taken together, these results position synthetic data not as a purely generative artifact, but as a structured component of the causal inference workflow whose value depends on causal validity, estimation stability, and diagnostic usefulness. 2 Related works Few studies have investigated the intersection of synthetic data generation and causal inference. Our work builds on advances in synthetic data generation, causal effect estimation, and simulation-based estimator evaluation. Nakada et al. (2025) study LLM-based synthetic oversampling and augmentation for imbalanced classification and spurious correlation. They develop theoretical guarantees showing how synthetic samples reduce group-specific risk under imbalance, derive scaling laws for adding synthetic data across groups, and analyze transformer-based generators as sources of high-quality synthetic samples. Their work is closest to ours in using LLM-generated tabular data to address imbalance, but their focus is prediction under class imbalance and spurious correlation, whereas we study preservation of causal estimands and synthetic-data-based causal estimator evaluation. Lyu et al. (2026) study synthetic data augmentation for imbalanced learning and show that naively treating synthetic minority samples as real data can introduce systematic bias when the synthetic distribution differs from the true minority distribution. They propose a bias-corrected augmentation procedure that estimates synthetic-data bias using majority-class information and adjusts the empirical loss accordingly, with theoretical guarantees for imbalanced classification, multi-task learning, and ATE estimation. Recent work has also studied LLMs as synthetic data generators. Long et al. (2024) survey LLM-driven synthetic data generation for NLP, organizing the literature around generation, curation, and evaluation, and emphasizing the need to balance faithfulness and diversity. Li et al. (2023) study synthetic data generation for text classification, finding that few-shot generation guided by real examples improves over zero-shot generation, but models trained on LLM-generated data generally still underperform models trained on real data. They also show that synthetic data is more effective for low-subjectivity tasks such as news topic classification and spam detection, while performance degrades more for subjective tasks such as humor, sarcasm, and emotion classification. For tabular data, CTGAN (Xu et al., 2019) pioneered conditional GANs with mode-specific normalization and training-by-sampling, and GReaT (Borisov et al., 2023) showed that language models can generate realistic tabular data by serializing tables as text. These methods focus primarily on distributional or predictive fidelity; in contrast, we ask whether synthetic tabular data preserve causal structure, especially the treatment-effect contrast that determines the ATE. Several studies have begun exploring synthetic data and foundation models for causal inference. De Bartolomeis et al. (2024) use foundation models for randomized experiments, focusing on experimental efficiency rather than positivity violations or causal-parameter preservation. Liu et al. (2025) survey LLM applications in causal inference, but do not address whether synthetic data preserve causal estimands. Our work fills this gap by showing that predictive fidelity can coexist with poor ATE preservation and by proposing a hybrid generation strategy that explicitly models the causal nuisance components. For positivity violations, traditional approaches such as trimming extreme propensity scores can reduce variance but may also change the target population or introduce bias (Gruber et al., 2022; Xu et al., 2026a). Many works diagnose or mitigate practical positivity problems in IPW-based estimators by truncating extreme fitted propensity scores or filtering samples with extreme scores (Zhou et al., 2020; Kang and Schafer, 2007; Freedman and Berk, 2008; Cole and Hernán, 2008). Other methods use overlap weights to emphasize units with stronger treatment overlap (Cheng et al., 2022; Matsouaka and Zhou, 2024). Our approach is complementary: we use synthetic covariate support and paired augmentation to stabilize learning in rare but plausible (A,W)(A,W) regimes, while explicitly acknowledging the distribution-shift tradeoff introduced by augmentation. A large literature studies estimators for the average treatment effect from observational data. Outcome regression (OR), or G-computation, estimates the conditional outcome model Q(a,W)=[Y∣A=a,W]Q(a,W)=E[Y A=a,W] and plugs fitted counterfactual means into the ATE functional (Robins, 1986); flexible regressors such as BART are often used for this step (Chipman et al., 2010). Inverse probability weighting (IPW), closely related to the Horvitz–Thompson estimator, uses the propensity score to reweight observations and emulate a randomized experiment (Horvitz and Thompson, 1952). Augmented inverse probability weighting (AIPW) combines outcome regression and propensity score models, yielding a doubly robust estimator that can remain consistent if either nuisance component is correctly specified (Robins et al., 1994). Targeted maximum likelihood estimation (TMLE) updates an initial outcome regression along a targeted fluctuation submodel so that the estimator solves the efficient influence-function equation, yielding a doubly robust and locally efficient substitution estimator (van der Laan and Rubin, 2006; van der Laan and Rose, 2011, 2018). Because these estimators have different finite-sample and limited-overlap failure modes, we use OR, IPW, AIPW, and TMLE as benchmark estimators throughout our synthetic-data evaluation. Evaluating causal estimators typically relies on simplified simulations that fail to capture real-world complexity. The closest related idea is the generative modeling-based bootstrap of Tran et al. (2026), which uses a learned generative model to resample synthetic datasets for statistical inference. Their goal is inferential: to approximate the sampling distribution of a fixed estimator and construct valid confidence intervals, including in settings where the classical bootstrap can fail. Our use of synthetic data serves a different function. We use hybrid synthetic data as a pre-analysis simulation engine: before applying an estimator to the real dataset, the analyst can repeatedly sample from a learned, realistic data-generating environment and diagnose how competing causal estimators behave. Thus, our synthetic engine is not a bootstrap for one fixed estimator, but a diagnostic benchmark for estimator selection. By preserving both covariate complexity and learned causal mechanisms, it provides a testbed for comparing the bias, variance, RMSE, and MSE of OR, IPW, AIPW, and TMLE under realistic finite-sample regimes before the final real-data analysis is performed. 3 Failure of Generative Models to Preserve Causal Parameters and a Hybrid Generation Approach Generative model-based data encryption using synthetic data generation has gained increasing attention, with approaches based on GANs Xu et al. (2019) and LLMs Borisov et al. (2023). While metrics such as train-on-synthetic-test-on-real (TSTR) are commonly used to evaluate predictive quality, there is limited focus on whether synthetic data preserve causal parameters. This distinction is important because strong predictive fidelity does not necessarily imply causal fidelity. In particular, the average treatment effect (ATE), [Y1−Y0],E[Y^1-Y^0], depends on how the outcome changes with treatment conditional on covariates, rather than only on how well a synthetic dataset reproduces marginal or predictive patterns. We focus on the tuple (W,A,Y)(W,A,Y), where W denotes covariates, A∈0,1A∈\0,1\ is a binary treatment, and Y is an outcome. Under standard causal assumptions, including consistency, ignorability, and positivity, the ATE is identified as Ψ(P)=PW[QP(1,W)−QP(0,W)],QP(a,w):=P[Y∣A=a,W=w]. (P)=E_P_W [Q_P(1,W)-Q_P(0,W) ], Q_P(a,w):=E_P[Y A=a,W=w]. Thus causal preservation requires preserving both the covariate law PWP_W and the treatment-effect contrast ΔQ(w):=Q(1,w)−Q(0,w). _Q(w):=Q(1,w)-Q(0,w). The following sensitivity bound makes this point explicit. Proposition 3.1 (L2 sensitivity bound for synthetic ATE). Let W∈[0,1]dW∈[0,1]^d, and suppose PWP_W and PW⋆P_W admit densities pWp_W and pW⋆p_W with respect to Lebesgue measure. Let Y∈[0,1]Y∈[0,1], and define ΔQ(w):=Q(1,w)−Q(0,w),Δ⋆(w):=Q⋆(1,w)−Q⋆(0,w). _Q(w):=Q(1,w)-Q(0,w), (w):=Q (1,w)-Q (0,w). Then |Ψ(PW,Q)−Ψ(PW⋆,Q⋆)|≤‖pW−pW⋆‖L2+‖ΔQ−Δ⋆‖L2(PW⋆).| (P_W,Q)- (P_W ,Q )|≤\|p_W-p_W \|_L_2+\| _Q- \|_L_2(P_W ). Proposition 3.1 separates synthetic ATE error into two pieces: an error in the generated covariate law and an error in the outcome contrast. The first term, ‖pW−pW⋆‖L2,\|p_W-p_W \|_L_2, measures whether the synthetic covariates resemble the target covariate distribution. The second term, ‖ΔQ−Δ⋆‖L2(PW⋆),\| _Q- \|_L_2(P_W ), measures whether the synthetic outcome mechanism preserves the treatment-effect contrast. This second term is directly relevant for causal estimation, but it is not necessarily prioritized by standard row-level generative objectives. The next result formalizes this misalignment. A joint generative model often optimizes a row-level reconstruction or next-token loss over all entries in the row. When there are d covariate coordinates but only one outcome component, the outcome loss receives only a 1/(d+1)1/(d+1) share of the objective. Hence small joint reconstruction loss need not imply small error in Y∣A,WY A,W. Theorem 3.1 (Joint reconstruction induces an ATE-relevant tradeoff). Let W=(W1,…,Wd)W=(W_1,…,W_d) denote the d covariate coordinates in a tabular row and let Y∈0,1Y∈\0,1\. For a model f, define Qf(a,w):=ℙf(Y=1∣A=a,W=w),Δf(w):=Qf(1,w)−Qf(0,w),Q_f(a,w):=P_f(Y=1 A=a,W=w), _f(w):=Q_f(1,w)-Q_f(0,w), and let Q⋆,Δ⋆Q , denote the corresponding truth. Suppose the row-level reconstruction loss gives equal weight to the d covariate coordinates and one outcome component, so that Ljoint(f)=LW(f)+1d+1LY(f),L_joint(f)=L_W(f)+ 1d+1L_Y(f), where LWL_W is the averaged covariate-reconstruction loss and LYL_Y is the balanced conditional outcome loss for Y∣A,WY A,W. Then, for any reference model g, LY(f)−LY(g)=(d+1)[Ljoint(f)−Ljoint(g)+LW(g)−LW(f)].L_Y(f)-L_Y(g)=(d+1) [L_joint(f)-L_joint(g)+L_W(g)-L_W(f) ]. Thus, outcome-loss improvement inside a joint reconstruction objective is traded against covariate-reconstruction loss with a factor d+1d+1. Assume further that LYL_Y is the balanced Bernoulli KL loss and infhLY(h)=0 _hL_Y(h)=0. Then ‖Δf−Δ⋆‖L2(PW⋆)≤2LY(f).\| _f- \|_L_2(P_W )≤ 2 L_Y(f). Consequently, if Ljoint(f)≤εL_joint(f)≤ and LW(f)≥0L_W(f)≥ 0, then ‖Δf−Δ⋆‖L2(PW⋆)≤2(d+1)ε.\| _f- \|_L_2(P_W )≤ 2 (d+1) . By contrast, if a hybrid outcome model fhybf_hyb is fit directly to LYL_Y and satisfies LY(fhyb)≤εY,L_Y(f_hyb)≤ _Y, then ‖Δfhyb−Δ⋆‖L2(PW⋆)≤2εY.\| _f_hyb- \|_L_2(P_W )≤ 2 _Y. Theorem 3.1 explains why high reconstruction fidelity can be misleading for causal inference. A joint generator may achieve small row-level loss primarily by reconstructing the many covariate coordinates well, while the outcome mechanism receives only a diluted share of the objective. However, Proposition 3.1 shows that the ATE remains first-order sensitive to the contrast error ‖ΔQ−Δ⋆‖L2(PW⋆).\| _Q- \|_L_2(P_W ). This creates a tradeoff: a fully generative model fixes the relative weight between covariate reconstruction and outcome modeling, whereas a hybrid construction can tune these components separately. In particular, the hybrid approach can keep the covariate-generation term comparable while allocating additional modeling effort to Q(Y∣A,W)Q(Y A,W), thereby improving the ATE-relevant contrast. Motivated by this observation, we propose a hybrid data generation approach. As shown in Algorithm 1, we first train a generative model on the seed dataset seedD_seed to synthesize covariates W~ W, while monitoring privacy and distributional similarity using distance to closest records (DCR) Borisov et al. (2023). We then fit a propensity model g^(A∣W) g(A W) and an outcome model Q^(A,W) Q(A,W) directly on seedD_seed. Finally, for each synthetic covariate W~i W_i, we sample treatment A~i A_i from g^(⋅∣W~i) g(· W_i) and generate outcome Y~i Y_i from Q^(A~i,W~i) Q( A_i, W_i). The separation between the outcome model and the covariate generator also enables safe transfer methods, such as REFINE Xu et al. (2026b), to incorporate external information when learning the outcome mechanism. Algorithm 1 Hybrid synthetic data generation 1:Input: Seed data seed=(Wi,Ai,Yi)i=1mD_seed=\(W_i,A_i,Y_i)\_i=1^m and target size n 2:Construct P^W P_W from Wii=1m\W_i\_i=1^m ⊳ e.g. a generative model, monitored by DCR 3:Construct g^(a∣w) g(a w) and Q^(a,w) Q(a,w) ⊳ trained on seedD_seed 4:for i=1,…,ni=1,…,n do 5: Sample W~i∼P^W W_i P_W 6: Sample A~i∼g^(⋅∣W~i) A_i g(· W_i) 7: Set Y~i←Q^(A~i,W~i) Y_i← Q( A_i, W_i) 8:end for 9:Return: ~=(W~i,A~i,Y~i)i=1n D=\( W_i, A_i, Y_i)\_i=1^n We benchmark synthetic data generated by large language models (LLMs), generative adversarial networks (GANs), and their corresponding hybrid variants. In the fully synthetic setting, denoted as “Syn Full,” both treatment A~ A and outcome Y~ Y are generated directly from the generative model pθp_θ. In the hybrid setting, only W~ W is generated by the synthetic covariate model, while A~ A and Y~ Y are produced using separately fitted nuisance models. Our findings show that current generative AI methods for tabular data synthesis can fail to preserve key causal parameters. As shown in Figure 1, LLM-generated data can achieve strong TSTR performance, indicating useful predictive fidelity, while still producing substantially biased ATE estimates. This confirms the theoretical point above: predictive or reconstruction quality alone does not guarantee preservation of the treatment-effect contrast. At the same time, the hybrid constructions substantially improve causal fidelity. In Figure 1, hybrid synthetic data achieve much lower ATE MSE than fully generative synthetic data. For instance, with true ATE 0.41830.4183, the LLM IPW estimate improves from 0.47510.4751 to 0.41720.4172 and its MSE drops from 0.00470.0047 to 0.00040.0004; for TMLE, MSE drops from 0.01080.0108 to 0.00050.0005. For GAN, IPW MSE drops from 0.18540.1854 to 0.01140.0114, and TMLE MSE drops from 0.20270.2027 to 0.00870.0087. The DCR values remain comparable to those of the corresponding full generators, and in some cases become slightly smaller, indicating that the hybrid procedure does not simply improve ATE estimation by moving far away from the original data distribution. Rather, it improves the estimand-relevant component of the synthetic data: the conditional outcome mechanism. These results support the tradeoff suggested by Proposition 3.1 and Theorem 3.1: the covariate-generation term can remain well controlled, while direct modeling of Y∣A,WY A,W substantially improves preservation of the ATE. Figure 1: Privacy and causal-fidelity diagnostics for synthetic data. Left: predictive utility (TSTR AUC) and privacy distance (mean DCR) across synthetic datasets. Right: ATE MSE across estimators, with the vertical axis zoomed to emphasize the comparison among the better-performing methods. Hybrid constructions improve causal fidelity substantially relative to fully generative synthetic data, while DCR and TSTR alone do not fully determine causal usefulness. 4 A potential way to address positivity violation with LLM synthetic data Positivity violations make causal estimation unstable because some treatment values are rarely observed in parts of the covariate space. In these rare (A,W)(A,W) regimes, the outcome regression Q(A,W)Q(A,W) can be poorly learned, and weighting-based estimators can become unstable due to extreme propensity scores. Synthetic data may help by providing plausible covariate scaffolds in sparse regions and by assigning treatments to improve balance over (W,A)(W,A). This does not solve structural positivity violations, but it can help when the observed data are imbalanced while the target population still contains plausible underrepresented regimes. The following proposition formalizes the tradeoff. Synthetic augmentation can reduce error by improving the conditional treatment-effect estimate, but it may also introduce error by shifting the covariate distribution. Proposition 4.1 (When synthetic overlap support helps). Let ψ0=∫τ0(w)μ0(w),τ0(w)=Q0(1,w)−Q0(0,w), _0= _0(w)\,d _0(w), _0(w)=Q_0(1,w)-Q_0(0,w), be the target ATE under the target covariate law μ0 _0. Let τorig(w)=Qorig(1,w)−Qorig(0,w),τaug(w)=Qaug(1,w)−Qaug(0,w), _orig(w)=Q_orig(1,w)-Q_orig(0,w), _aug(w)=Q_aug(1,w)-Q_aug(0,w), and define ψorig=∫τorig(w)μ0(w),ψaug=∫τaug(w)μaug(w), _orig= _orig(w)\,d _0(w), _aug= _aug(w)\,d _aug(w), where μaug _aug is the covariate law induced by synthetic augmentation. Then ψorig−ψ0=∫(τorig−τ0)μ0, _orig- _0= ( _orig- _0)\,d _0, and ψaug−ψ0=∫(τaug−τ0)μaug+∫τ0d(μaug−μ0). _aug- _0= ( _aug- _0)\,d _aug+ _0\,d( _aug- _0). Therefore, synthetic overlap support improves absolute error, |ψaug−ψ0|<|ψorig−ψ0|,| _aug- _0|<| _orig- _0|, if and only if |∫(τaug−τ0)μaug+∫τ0d(μaug−μ0)|<|∫(τorig−τ0)μ0|. | ( _aug- _0)\,d _aug+ _0\,d( _aug- _0) |< | ( _orig- _0)\,d _0 |. In particular, the key comparison is between two terms: augmentation can reduce error by improving the conditional effect estimate τ, but it can also introduce a distribution-shift term through μaug−μ0 _aug- _0. Motivated by Proposition 4.1, we use synthetic data to support the poorly represented parts of the covariate-treatment space. We define samples with propensity scores pi<1nlognp_i< 1 n n as extreme-propensity samples. For these samples, we search for synthetic counterparts that are close in covariate space, using Euclidean distance, and pair them to improve treatment balance. The goal is to stabilize estimation of Q(A,W)Q(A,W) in rare regimes while keeping the induced covariate shift limited. Scenario IPW AIPW OR TMLE Original 0.0082 0.0232 0.0259 0.0104 Pair Hybrid GAN 0.0017 0.0052 0.0072 0.0040 Pair Hybrid LLM 0.0013 0.0022 0.0033 0.0043 Pair Self-Supervised GAN 0.0153 0.0276 0.0330 0.0093 Pair Self-Supervised LLM 0.0218 0.0364 0.0409 0.0193 Pair Hybrid Flip 5% GAN 0.0038 0.0082 0.0103 0.0047 Pair Hybrid Flip 10% GAN 0.0058 0.0106 0.0131 0.0053 Pair Hybrid Flip 20% GAN 0.0130 0.0255 0.0297 0.0100 Pair Hybrid Flip 5% LLM 0.0031 0.0071 0.0088 0.0056 Pair Hybrid Flip 10% LLM 0.0033 0.0059 0.0079 0.0062 Pair Hybrid Flip 20% LLM 0.0196 0.0327 0.0364 0.0159 Table 1: MSE across positivity experiments. Lower is better. The results in Table 1 support this tradeoff. Pair Hybrid LLM reduces IPW MSE from 0.00820.0082 to 0.00130.0013, AIPW MSE from 0.02320.0232 to 0.00220.0022, and OR MSE from 0.02590.0259 to 0.00330.0033. Pair Hybrid GAN also improves over the original data, reducing IPW MSE to 0.00170.0017 and TMLE MSE to 0.00400.0040. In contrast, self-supervised pairing, i.e., training Q only on the current seed data and then predicting outcomes over synthetic covariates W, is less reliable and can perform worse than the original data. This suggests that synthetic covariates alone do not resolve the positivity problem: without additional outcome information or improved transfer for learning Q(A,W)Q(A,W) in rare regimes, augmentation can fail to improve causal estimation. The flip experiments show that the benefit does not require a perfect synthetic outcome model. With 10%10\% flips, Pair Hybrid LLM still improves over the original data for IPW, AIPW, and OR, with MSEs 0.00330.0033, 0.00590.0059, and 0.00790.0079, respectively. When the flip rate reaches 20%20\%, the advantage weakens, consistent with Proposition 4.1: augmentation helps only when the improvement in τ outweighs the covariate shift or outcome noise introduced by the synthetic samples. 5 Synthetic data engine for estimator evaluation Another use of hybrid synthetic data is estimator evaluation. In practice, analysts often need guidance before committing to a final causal analysis: which estimator is likely to be stable, whether bias or variance will dominate the error, whether a sample size such as 1000 is adequate, and whether OR, IPW, AIPW, or TMLE is safer in the observed regime. Real datasets rarely provide repeated experiments with known ground truth, while hand-crafted simulations often miss the complexity of real covariate distributions. A synthetic simulation engine offers a middle ground: it turns one observed dataset into a realistic finite-sample forecasting environment. We build this engine by learning a realistic covariate generator and separately learning treatment and outcome mechanisms. Once a large synthetic population is generated, we estimate a high-precision reference effect using a large synthetic sample and then repeatedly subsample smaller datasets to evaluate finite-sample bias, variance, RMSE, and MSE. In our experiment, the large synthetic reference sample has size 50,000, and each finite-sample replication uses 1,000 observations. Table 2 compares the finite-sample behavior induced by synthetic data against the corresponding real-data benchmark. The LLM-based hybrid simulator tracks the real benchmark closely: the signs of bias are correct for all estimators, and the synthetic biases remain close to the real biases. For example, IPW has real bias −0.0012-0.0012 and synthetic bias −0.0029-0.0029, while TMLE has real bias 0.02430.0243 and synthetic bias 0.02020.0202. AIPW and OR also show close agreement, with synthetic biases −0.0302-0.0302 and −0.0375-0.0375 compared with real biases −0.0263-0.0263 and −0.0343-0.0343. The GAN-based simulator is less faithful: GAN synthetic IPW has bias −0.1025-0.1025 compared with real bias −0.0012-0.0012, GAN synthetic AIPW has bias −0.1275-0.1275 compared with real bias −0.0263-0.0263, and TMLE even has the wrong bias sign. These discrepancies are also reflected in RMSE and MSE, where GAN synthetic errors are much larger than the real-data benchmarks for several estimators. Overall, the results suggest that hybrid synthetic data can serve as a practical diagnostic tool for estimator selection, but the quality of the covariate generator matters. The LLM-based simulator more closely reproduces the real finite-sample ranking and error decomposition, making it more useful for anticipating estimator behavior before the final analysis. The goal is not to replace real-data analysis, but to provide a realistic stress test that helps analysts identify likely estimator failure modes in advance. Source Estimator Sign correct Real bias Syn. bias Real var Syn. var Real RMSE Syn. RMSE Real MSE Syn. MSE LLM IPW Yes -0.0012 -0.0029 0.000279 0.000660 0.0163 0.0252 0.000267 0.000635 LLM TMLE Yes 0.0243 0.0202 0.000379 0.000762 0.0308 0.0337 0.000948 0.001133 LLM AIPW Yes -0.0263 -0.0302 0.000314 0.000621 0.0315 0.0388 0.000989 0.001504 LLM OR Yes -0.0343 -0.0375 0.000315 0.000630 0.0384 0.0447 0.001478 0.002002 GAN IPW Yes -0.0012 -0.1025 0.000279 0.000530 0.0163 0.1050 0.000267 0.011015 GAN TMLE No 0.0243 -0.0808 0.000379 0.000892 0.0308 0.0859 0.000948 0.007380 GAN AIPW Yes -0.0263 -0.1275 0.000314 0.000589 0.0315 0.1297 0.000989 0.016822 GAN OR Yes -0.0343 -0.1343 0.000315 0.000578 0.0384 0.1363 0.001478 0.018582 Table 2: Real-data fidelity diagnostic summary. Synthetic performance is evaluated by how closely synthetic-data finite-sample behavior matches the real-data benchmark, using synthetic-versus-real bias, variance, RMSE, and MSE. 6 Real-World Applications and Limitations We next evaluate our framework on the ACTG175 dataset, a real-world HIV randomized trial with a continuous outcome. Unlike the earlier privacy experiment, where the target effect is known by construction, the ACTG setting is intended to assess whether synthetic data can support two realistic goals simultaneously: preserving useful predictive and causal structure, and serving as a credible simulation environment for benchmarking estimators under complex covariates. We construct four synthetic ACTG datasets: LLM Full, LLM Hybrid, CTGAN Full, and CTGAN Hybrid. For each synthetic dataset, we report three diagnostics: train-on-synthetic-test-on-real (TSTR) AUC, mean distance to the closest record (DCR), and repeated treatment-effect estimates based on subsamples of size 1000. The resulting summary is shown in Table 3. On predictive fidelity, both hybrid constructions improve substantially over their fully generative counterparts: TSTR AUC increases from 0.74220.7422 to 0.81020.8102 for the LLM generator and from 0.44440.4444 to 0.81240.8124 for CTGAN. The DCR summaries remain similar within each generator family, with mean DCR near 1.051.05 for LLM Full and LLM Hybrid and around 2.092.09 and 2.002.00 for CTGAN Full and CTGAN Hybrid. The treatment-effect estimates also change after hybridization: for both LLM and CTGAN, the hybrid datasets produce more consistently negative ATE estimates across IPW, AIPW, and TMLE. Together with the improved TSTR AUC, this pattern suggests that hybridization may reduce distortion in the outcome-relevant structure. Dataset TSTR AUC Mean DCR IPW ATE AIPW ATE TMLE ATE LLM Full 0.7422 1.0479 -3.1670 -10.0869 -2.9390 LLM Hybrid 0.8102 1.0461 -20.4772 -14.6497 -23.9624 CTGAN Full 0.4444 2.0900 -2.4600 -18.7035 -1.4010 CTGAN Hybrid 0.8124 1.9969 -13.5488 -27.0269 -17.5316 Table 3: ACTG synthetic-data diagnostics and effect estimates. TSTR reports predictive utility on the real ACTG benchmark, DCR summarizes privacy distance, and ATE columns report mean estimated treatment effects over repeated subsamples of size 1000. Beyond point estimates, the main value of the ACTG experiment is to assess whether hybrid synthetic data can function as a realistic simulation engine for finite-sample benchmarking. To do so, we generate large hybrid pools and repeatedly subsample from them at sample sizes ranging from 100 to 1000. For each sample size, we evaluate IPW, AIPW, outcome regression, and TMLE relative to a large-sample synthetic TMLE reference within the same hybrid environment. The resulting bias, variance, MSE, and RMSE curves are shown in Figures 2 and A.1. Several consistent patterns emerge. First, IPW has relatively small bias but the largest variance, reflecting the instability of weight-based estimators under estimated propensities. Second, as sample size grows, the biases of all estimators decrease relative to the large-sample TMLE reference, consistent with the expectation that regular asymptotically linear estimators such as AIPW and TMLE become increasingly close in large samples. Third, AIPW generally improves upon outcome regression in both MSE and bias, highlighting the benefit of doubly robust correction over a purely outcome-model-based plug-in estimator. The CTGAN-based simulator shows broadly similar qualitative trends, but with more irregular bias and MSE behavior than the LLM-based simulator. Together with the earlier diagnostics, this suggests that CTGAN may provide a less ideal simulation environment than the LLM-based hybrid generator in this setting. Overall, these results indicate that hybrid synthetic data can be used not only to generate plausible real-world datasets, but also to forecast estimator behavior in realistic regimes where practitioners need guidance about finite sample bias-variance tradeoffs before conducting a final analysis. Figure 2: Finite-sample benchmarking results under the LLM-based hybrid ACTG simulator. Panels report bias, variance, MSE, and RMSE as functions of sample size for IPW, AIPW, outcome regression, and TMLE. These ACTG results illustrate both the promise and the limitation of our framework. The promise is that hybrid synthetic data can convert a single real dataset into a realistic benchmarking environment that reveals estimator-specific failure modes before a final analysis is committed. The limitation is that the usefulness of this benchmark still depends critically on the quality of the learned synthetic covariate generator. In our experiments, the LLM-based hybrid simulator appears substantially more faithful than the CTGAN-based one, suggesting that not all synthetic generators are equally suitable as foundations for causal simulation. Thus, while hybrid synthetic data can support estimator evaluation in complex real-world settings, careful validation of the synthetic data generator remains essential. 7 Conclusion We have shown that synthetic data for causal inference must be evaluated by more than predictive or distributional fidelity: fully generative tabular synthesizers can reproduce useful observational patterns while substantially distorting causal estimands such as the ATE. Motivated by this failure mode, we proposed a hybrid synthetic-data framework that separates covariate generation from treatment and outcome modeling, improving preservation of the treatment-effect contrast, supporting targeted augmentation for practical positivity problems, and enabling a synthetic simulation engine for pre-analysis estimator evaluation. Several directions remain important for future work. First, the outcome-model component of the hybrid pipeline can be strengthened further, for example by studying plug-in outcome models enhanced by safe transfer methods such as REFINE and quantifying when externally augmented outcome modeling improves synthetic-data quality. Second, the framework should be extended beyond static tabular data to more complex formats, including longitudinal data, time-varying treatments, and multimodal settings where images, text, or other modalities are paired with tabular covariates. Third, future work should investigate synthetic-data validity for causal estimands beyond the ATE, such as conditional average treatment effects, heterogeneous treatment effects, policy values, mediation effects, and longitudinal causal parameters. Fourth, better auditing tools are needed: while we use DCR to monitor covariate similarity and privacy distance, causal usefulness also depends on the learned outcome contrast, especially in rare (A,W)(A,W) regimes under limited overlap. Developing diagnostics that audit synthetic-data quality for causal validity, overlap support, and rare-regime outcome reliability is therefore a key step toward certifying synthetic data pipelines for high-stakes causal analysis. References V. Borisov, K. Sessler, T. Leemann, M. Pawelczyk, and G. Kasneci (2023) Language models are realistic tabular data generators. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Additional experiments and reproducibility details, §2, §3, §3. C. Cheng, F. Li, L. E. Thomas, and F. (. Li (2022) Addressing extreme propensity scores in estimating counterfactual survival functions via the overlap weights. American Journal of Epidemiology 191 (6), p. 1140–1151. External Links: ISSN 0002-9262, Document, Link, https://academic.oup.com/aje/article-pdf/191/6/1140/43830708/kwac043.pdf Cited by: §2. H. Chipman, E. George, and R. McCulloch (2010) BART: bayesian additive regression trees. The Annals of Applied Statistics 4, p. . External Links: Document Cited by: §2. S. R. Cole and M. A. Hernán (2008) Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology 168 (6), p. 656–664. Note: Epub 2008 Aug 5 External Links: Document Cited by: §2. P. De Bartolomeis, J. Abad, G. Wang, K. Donhauser, R. M. Duch, F. Yang, and I. J. Dahabreh (2024) Efficient randomized experiments using foundation models. arXiv preprint arXiv:2502.04262. Cited by: §2. D. A. Freedman and R. A. Berk (2008) Weighting regressions by propensity scores. Evaluation Review 32 (4), p. 392–409. External Links: Document Cited by: §2. S. Gruber, R. V. Phillips, H. Lee, and M. J. van der Laan (2022) Data-adaptive selection of the propensity score truncation level for inverse-probability–weighted and targeted maximum likelihood estimators of marginal point treatment effects. American Journal of Epidemiology 191 (9), p. 1640–1651. External Links: ISSN 0002-9262, Document, Link, https://academic.oup.com/aje/article-pdf/191/9/1640/45508605/kwac087.pdf Cited by: §2. D. G. Horvitz and D. J. Thompson (1952) A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association 47 (260), p. 663–685. External Links: Document, Link, https://doi.org/10.1080/01621459.1952.10483446 Cited by: §2. J. D. Y. Kang and J. L. Schafer (2007) Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. Statistical Science 22 (4), p. 523 – 539. External Links: Document, Link Cited by: §2. Z. Li, H. Zhu, Z. Lu, and M. Yin (2023) Synthetic data generation with large language models for text classification: potential and limitations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, p. 10443–10461. External Links: Link, Document Cited by: §2. X. Liu, P. Xu, J. Wu, J. Yuan, Y. Yang, Y. Zhou, F. Liu, T. Guan, H. Wang, T. Yu, J. McAuley, W. Ai, and F. Huang (2025) Large language models and causal inference in collaboration: a survey. arXiv preprint arXiv:2403.09606. Cited by: §2. L. Long, R. Wang, R. Xiao, J. Zhao, X. Ding, G. Chen, and H. Wang (2024) On LLMs-driven synthetic data generation, curation, and evaluation: a survey. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 11065–11082. External Links: Link, Document Cited by: §2. P. Lyu, Z. Ma, L. Zhang, and A. R. Zhang (2026) Bias-corrected data synthesis for imbalanced learning. External Links: 2510.26046, Link Cited by: §2. R. A. Matsouaka and Y. Zhou (2024) Causal inference in the absence of positivity: the role of overlap weights. Biometrical Journal 66 (4), p. e2300156. External Links: Document Cited by: §2. R. Nakada, Y. Xu, L. Li, and L. Zhang (2025) Synthetic oversampling: theory and a practical approach using llms to address data imbalance. arXiv preprint arXiv:2406.03628. Cited by: §2. J. M. Robins, A. Rotnitzky, and L. P. Zhao (1994) Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association 89 (427), p. 846–866. External Links: Document, Link, https://doi.org/10.1080/01621459.1994.10476818 Cited by: §2. J. Robins (1986) A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling 7 (9), p. 1393–1512. External Links: ISSN 0270-0255, Document, Link Cited by: §2. L. Tran, T. Ye, P. Ding, and F. Han (2026) Generative modeling for the bootstrap. External Links: 2602.17052, Link Cited by: §2. M. J. van der Laan and S. Rose (2011) Targeted learning: causal inference for observational and experimental data. 1 edition, Springer Series in Statistics, Springer New York, NY, New York, NY. External Links: ISBN 978-1-4419-9781-4, Document Cited by: §2. M. J. van der Laan and S. Rose (2018) Targeted learning in data science: causal inference for complex longitudinal studies. Springer Series in Statistics, Springer Cham. External Links: ISBN 978-3-319-65303-7, Document, Link Cited by: §2. M. J. van der Laan and D. Rubin (2006) . The International Journal of Biostatistics 2 (1). External Links: Link, Document Cited by: §2. L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni (2019) Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32, p. . External Links: Link Cited by: §2, §3. Y. Xu, S. Gruber, and M. J. van der Laan (2026a) Investigating targeting strategies and truncation in tmle for the average treatment effect under practical positivity violations. External Links: 2604.20059, Link Cited by: §2. Y. Xu, R. Nakada, L. Zhang, and L. Li (2026b) Residual feature integration is sufficient to prevent negative transfer. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §3. Y. Zhou, R. A. Matsouaka, and L. Thomas (2020) Propensity score weighting under limited overlap and model misspecification. Statistical Methods in Medical Research 29 (12), p. 3721–3756. Note: Epub 2020 Jul 21 External Links: Document Cited by: §2. Appendix Additional experiments and reproducibility details Figure A.1 reports the corresponding simulation-engine results when CTGAN is used as the synthetic covariate generator. Several qualitative patterns remain consistent with the earlier benchmark. First, IPW again exhibits the largest variance across sample sizes, reflecting the instability of purely weight-based estimators under estimated propensities. Second, AIPW consistently improves upon outcome regression in both MSE and RMSE, indicating that doubly robust correction remains beneficial in this setting. Third, TMLE remains the estimator that is most closely aligned with the large-sample synthetic reference, with the most pronounced bias reduction as sample size increases. At the same time, the CTGAN-based simulator also shows some distinct differences in finite-sample behavior. Although IPW often has smaller bias than AIPW and outcome regression, its substantially larger variance can lead to less favorable overall error at some sample sizes, so that AIPW may achieve smaller MSE despite having higher bias. More generally, the bias, MSE, and RMSE curves display more irregular finite-sample fluctuations across estimators, illustrating that the synthetic benchmark can preserve broad qualitative rankings while still inducing estimator-specific differences in error decomposition. Figure A.1: Finite-sample benchmarking results under the CTGAN-based hybrid ACTG simulator. Panels report bias, variance, MSE, and RMSE as functions of sample size for IPW, AIPW, outcome regression, and TMLE. To support reproducibility, we describe the simulation design and the LLM-based synthetic tabular generation pipeline in implementation terms. In the main synthetic-data experiments, the data-generating process contains six covariates, one binary treatment, and one binary outcome: W1,W2∼Bernoulli(0.5),ℙ(W3=1∣W1,W2)=0.3+0.35(W1+W2)/2, W_1,W_2 (5), (W_3=1 W_1,W_2)=3+35(W_1+W_2)/2, W4,W5∼N(0,1),W6=0.5W4+0.5W5+ϵ,ϵ∼N(0,1), W_4,W_5 N(0,1), W_6=5W_4+5W_5+ε, ε N(0,1), ℙ(A=1∣W)=expit−30+16W1−24W2+12W3+6W4−10W5+16W6, (A=1 W)=expit\-0+6W_1-4W_2+2W_3+6W_4-0W_5+6W_6\, τ(W)=2.0+0.5sin(W1)+0.3log(|W2|+1)−0.2W32+0.1exp(W4)−0.3tanh(W5)+0.2cos(W6), τ(W)=0+5 (W_1)+3 (|W_2|+1)-2W_3^2+1 (W_4)-3 (W_5)+2 (W_6), ℙ(Y=1∣A,W)=expit−0.5+τ(W)A+0.5W1+W2−W3+0.2W4−0.3W5+0.1W6. (Y=1 A,W)=expit\-5+τ(W)A+5W_1+W_2-W_3+2W_4-3W_5+1W_6\. In the randomized setting, we replace the observational treatment rule by A∼Bernoulli(0.5)A (0.5). The theoretical ATE is computed from a large randomized population by averaging the difference between the treated and control outcome probabilities. For the privacy and causal-fidelity experiments, we use a randomized seed dataset of size 10001000 and a randomized test dataset of size 10001000. For the positivity experiments, we additionally generate an observational dataset of size 200200, where treatment assignment follows the propensity model above. For LLM-based tabular synthesis, we used the GReaT framework Borisov et al. (2023) with GPT-2 as the underlying language model. Each row of the training table was serialized into a textual template that explicitly encoded covariates, treatment, and outcome, for example W1=1, W2=0, W3=1, W4=0, W5=1, W6=0 => A=1, Y=0. Following the GReaT methodology, we randomized the order of features during serialization to reduce positional artifacts and improve robustness to different conditioning orders. This representation allowed the model to treat each tabular row as a sequence while preserving the dependence structure among W, A, and Y. We fine-tuned GPT-2 using the GReaT Python package. In the main experiments, the model was trained for 50 epochs with batch size 32, and checkpoints were saved every 400,000 training steps to allow checkpoint recovery and exact reruns. After training, we reloaded the saved checkpoint and generated 50,000 synthetic samples using autoregressive sampling. Sampling was performed with randomized feature order and a maximum generation length between 1024 and 2000 tokens. The resulting outputs were then converted back into structured tabular format for downstream causal analyses. The GReaT framework is especially useful in this setting because it avoids lossy one-hot preprocessing and supports conditioning on arbitrary subsets of variables through textual prompts. This flexibility is valuable for causal inference tasks, where preserving conditional structure is often more important than merely matching marginal distributions. To evaluate whether the synthetic data remained close to the original data manifold without collapsing into memorization, we monitored distance to the closest record (DCR). In our experiments, DCR diagnostics showed that the generated samples remained close to the observed distribution while still preserving enough variability to avoid trivial duplication of training rows. The full pipeline is reproducible through fixed data-generating seeds, fixed training configurations, saved checkpoints, deterministic sampling settings, and systematic file naming for all intermediate and final outputs. Across repeated runs, fine-tuning and sampling typically required between 15 and 20 minutes depending on hardware and system load. We provide the full implementation details and code in the supplementary repository so that all synthetic-data generation and downstream evaluation steps can be rerun directly. Theoretical analysis Proof for Proposition 3.1. Write Ψ(PW,Q)=PW[ΔQ(W)],Ψ(PW⋆,Q⋆)=PW⋆[Δ⋆(W)]. (P_W,Q)=E_P_W[ _Q(W)], (P_W ,Q )=E_P_W [ (W)]. Add and subtract PW⋆[ΔQ(W)]E_P_W [ _Q(W)]: Ψ(PW,Q)−Ψ(PW⋆,Q⋆) (P_W,Q)- (P_W ,Q ) =PW[ΔQ(W)]−PW⋆[ΔQ(W)] = \E_P_W[ _Q(W)]-E_P_W [ _Q(W)] \ +PW⋆[ΔQ(W)−Δ⋆(W)]. +E_P_W [ _Q(W)- (W)]. Taking absolute values and applying the triangle inequality gives |Ψ(PW,Q)−Ψ(PW⋆,Q⋆)| | (P_W,Q)- (P_W ,Q )| ≤|∫ΔQ(w)pW(w)−pW⋆(w)w| ≤ | _Q(w)\p_W(w)-p_W (w)\\,dw | +|PW⋆[ΔQ(W)−Δ⋆(W)]|. + |E_P_W [ _Q(W)- (W)] |. Since Y∈[0,1]Y∈[0,1], we have |ΔQ(w)|≤1| _Q(w)|≤ 1. Hence |∫ΔQ(w)pW(w)−pW⋆(w)w|≤∫|pW(w)−pW⋆(w)|w. | _Q(w)\p_W(w)-p_W (w)\\,dw |≤ |p_W(w)-p_W (w)|\,dw. Because [0,1]d[0,1]^d has Lebesgue measure one, ∫|pW(w)−pW⋆(w)|w≤‖pW−pW⋆‖L2. |p_W(w)-p_W (w)|\,dw≤\|p_W-p_W \|_L_2. For the second term, by Cauchy–Schwarz, |PW⋆[ΔQ(W)−Δ⋆(W)]|≤‖ΔQ−Δ⋆‖L2(PW⋆). |E_P_W [ _Q(W)- (W)] |≤\| _Q- \|_L_2(P_W ). Combining the two displays gives |Ψ(PW,Q)−Ψ(PW⋆,Q⋆)|≤‖pW−pW⋆‖L2+‖ΔQ−Δ⋆‖L2(PW⋆).| (P_W,Q)- (P_W ,Q )|≤\|p_W-p_W \|_L_2+\| _Q- \|_L_2(P_W ). ∎ Proof for Theorem 3.1. First, the equal-weight reconstruction assumption gives Ljoint(f)=LW(f)+1d+1LY(f).L_joint(f)=L_W(f)+ 1d+1L_Y(f). Therefore, for any two models f and g, Ljoint(f)−Ljoint(g)=LW(f)−LW(g)+1d+1LY(f)−LY(g).L_joint(f)-L_joint(g)=\L_W(f)-L_W(g)\+ 1d+1\L_Y(f)-L_Y(g)\. Rearranging, 1d+1LY(f)−LY(g)=Ljoint(f)−Ljoint(g)−LW(f)−LW(g). 1d+1\L_Y(f)-L_Y(g)\=L_joint(f)-L_joint(g)-\L_W(f)-L_W(g)\. Multiplying both sides by d+1d+1 yields LY(f)−LY(g)=(d+1)[Ljoint(f)−Ljoint(g)+LW(g)−LW(f)].L_Y(f)-L_Y(g)=(d+1) [L_joint(f)-L_joint(g)+L_W(g)-L_W(f) ]. This is the exact tradeoff identity. It shows that a decrease in covariate-reconstruction loss can compensate for an increase in outcome loss inside the joint objective, and the conversion from joint-loss control to outcome-loss control costs a factor d+1d+1. As a special case, if LW(f)≥0L_W(f)≥ 0 and Ljoint(f)≤ε,L_joint(f)≤ , then 1d+1LY(f)≤Ljoint(f)≤ε. 1d+1L_Y(f)≤ L_joint(f)≤ . Hence LY(f)≤(d+1)ε.L_Y(f)≤(d+1) . Now assume LYL_Y is the balanced Bernoulli KL loss: LY(f)=PW⋆∑a∈0,1KL(Bern(Q⋆(a,W))∥Bern(Qf(a,W))).L_Y(f)=E_P_W _a∈\0,1\KL\! (Bern(Q (a,W))\, \|\,Bern(Q_f(a,W)) ). By Pinsker’s inequality, for each a∈0,1a∈\0,1\, |Qf(a,w)−Q⋆(a,w)|2≤12KL(Bern(Q⋆(a,w))∥Bern(Qf(a,w))).|Q_f(a,w)-Q (a,w)|^2≤ 12KL\! (Bern(Q (a,w))\, \|\,Bern(Q_f(a,w)) ). Integrating over PW⋆P_W gives ‖Qf(a,⋅)−Q⋆(a,⋅)‖L2(PW⋆)≤LY(f)2.\|Q_f(a,·)-Q (a,·)\|_L_2(P_W )≤ L_Y(f)2. Because Δf−Δ⋆=Qf(1,⋅)−Q⋆(1,⋅)−Qf(0,⋅)−Q⋆(0,⋅), _f- =\Q_f(1,·)-Q (1,·)\-\Q_f(0,·)-Q (0,·)\, the triangle inequality gives ‖Δf−Δ⋆‖L2(PW⋆)≤‖Qf(1,⋅)−Q⋆(1,⋅)‖L2(PW⋆)+‖Qf(0,⋅)−Q⋆(0,⋅)‖L2(PW⋆).\| _f- \|_L_2(P_W )≤\|Q_f(1,·)-Q (1,·)\|_L_2(P_W )+\|Q_f(0,·)-Q (0,·)\|_L_2(P_W ). Therefore, ‖Δf−Δ⋆‖L2(PW⋆)≤2LY(f).\| _f- \|_L_2(P_W )≤ 2 L_Y(f). The constant is loose but sufficient. Combining this with LY(f)≤(d+1)εL_Y(f)≤(d+1) gives ‖Δf−Δ⋆‖L2(PW⋆)≤2(d+1)ε.\| _f- \|_L_2(P_W )≤ 2 (d+1) . For the hybrid outcome model, the assumption LY(fhyb)≤εYL_Y(f_hyb)≤ _Y is a direct conditional-outcome loss certificate. Applying the same Pinsker and triangle-inequality argument gives ‖Δfhyb−Δ⋆‖L2(PW⋆)≤2εY.\| _f_hyb- \|_L_2(P_W )≤ 2 _Y. Thus the tradeoff is explicit: joint reconstruction controls the ATE-relevant outcome contrast only through a (d+1)(d+1)-inflated certificate, while direct hybrid outcome fitting controls it without that dilution. ∎ Proof for Proposition 4.1. Start from the definitions of the augmented estimator and the target: ψaug−ψ0 _aug- _0 =∫τaug(w)μaug(w)−∫τ0(w)μ0(w). = _aug(w)\,d _aug(w)- _0(w)\,d _0(w). Insert and subtract the same intermediate quantity ∫τ0μaug _0\,d _aug: ψaug−ψ0 _aug- _0 =(∫τaugμaug−∫τ0μaug)+(∫τ0μaug−∫τ0μ0). = ( _aug\,d _aug- _0\,d _aug )+ ( _0\,d _aug- _0\,d _0 ). By linearity of integration, ∫τaugμaug−∫τ0μaug _aug\,d _aug- _0\,d _aug =∫(τaug−τ0)μaug, = ( _aug- _0)\,d _aug, and, by the definition of integration against a signed measure, ∫τ0μaug−∫τ0μ0 _0\,d _aug- _0\,d _0 =∫τ0d(μaug−μ0). = _0\,d( _aug- _0). Therefore, ψaug−ψ0=∫(τaug−τ0)μaug+∫τ0d(μaug−μ0). _aug- _0= ( _aug- _0)\,d _aug+ _0\,d( _aug- _0). For the original estimator, ψorig−ψ0 _orig- _0 =∫τorigμ0−∫τ0μ0 = _orig\,d _0- _0\,d _0 =∫(τorig−τ0)μ0. = ( _orig- _0)\,d _0. Now synthetic overlap support is beneficial exactly when |ψaug−ψ0|<|ψorig−ψ0|. | _aug- _0|<| _orig- _0|. Substituting the two exact identities above yields |∫(τaug−τ0)μaug+∫τ0d(μaug−μ0)|<|∫(τorig−τ0)μ0|. | ( _aug- _0)\,d _aug+ _0\,d( _aug- _0) |< | ( _orig- _0)\,d _0 |. This is precisely the claimed equivalence. The proof is complete. ∎