Paper deep dive
Separating Covariate Shift from Mechanism Change with Two Discriminators: CJSD, a Conditional Discrepancy with an Exact Covariate-Concept Decomposition
Kentaro Oda
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/21/2026, 3:50:13 AM
Summary
The paper introduces the Conditional Jensen-Shannon Discrepancy (CJSD), a metric that decomposes dataset discrepancy into covariate shift and mechanism change (functional axis) using two discriminators. It proves theoretical properties including covariate-nullity and one-sided misspecification control, demonstrating superior performance over existing methods in separating concept drift from covariate shift across various benchmarks.
Entities (12)
Relation Signals (8)
Kentaro Oda → affiliatedwith → Kagoshima University
confidence 95% · Kentaro Oda Affiliation: Center for Management of Information Technologies, Kagoshima University
CJSD → decomposes → mechanism change
confidence 95% · CJSD: a conditional discrepancy with an exact covariate–concept decomposition
CJSD → decomposes → Covariate Shift
confidence 95% · CJSD: a conditional discrepancy with an exact covariate–concept decomposition
CLS → conflates → Covariate Shift
confidence 90% · exchange-based scores... conflate covariate shift with mechanism change
MMD → isblindto → mechanism change
confidence 90% · input-distribution distances (MMD, Wasserstein) are blind to changes of P(Y∣X)
CJSD → uses → T2
confidence 90% · DCJS^=CE(Z∣X)−CE(Z∣X,Y) on held-out data... Two ordinary classifiers predict the dataset indicator Z, one from x, one from (x,y)
CJSD → uses → T1
confidence 90% · DCJS^=CE(Z∣X)−CE(Z∣X,Y) on held-out data... Two ordinary classifiers predict the dataset indicator Z, one from x, one from (x,y)
CJSD → outperforms → KNN
confidence 85% · the kNN plug-in fails from d=64 while the discriminator route holds to d=256
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Streaming systems that maintain a pool of expert models must repeatedly decide whether to reuse an existing expert for arriving data, spawn a new one, or defer. We present a decision layer that makes all three outcomes statistically meaningful. Reuse and spawn are posed as one-sided sequential hypotheses on a conditional (mechanism-level) discrepancy, separated by an indifference zone; defer is exactly the state in which neither betting e-process has accumulated sufficient evidence. We prove finite-time anytime validity for the observable surrogate discrepancy of a predictable discriminator sequence, and an unconditional one-sided transfer to the population quantity in which each side's slack is the excess risk of a single discriminator; an empirically observed downward-bias regularity makes the spawn side exactly conservative. Recency without sacrificing the guarantee is obtained by a restarted e-detector: a bank of unwindowed betting supermartingales at geometrically spaced restart times (O(log t) memory), with the error budget spent over restart instances, which preserves lifetime anytime validity; spending over expert-creation order likewise controls multiplicity for unboundedly many experts. On synthetic multi-concept streams, Electricity, Covertype, and the recurrence-heavy INSECTS benchmark, the instance-accounted restarted bank achieves zero false spawns and zero false reuses after switches and matches or exceeds the retired windowed heuristic (INSECTS-reoccurring accuracy 0.675), making the deployed algorithm and the guaranteed algorithm one and the same.
Tags
Links
- Source: https://arxiv.org/abs/2608.19885v1
- Canonical: https://arxiv.org/abs/2608.19885v1
Trouble viewing inline? Open PDF directly →
Full Text
37,122 characters extracted from source content.
Expand or collapse full text
Separating Covariate Shift from Mechanism Change with Two Discriminators CJSD: a conditional discrepancy with an exact covariate–concept decomposition Kentaro Oda Affiliation: Center for Management of Information Technologies, Kagoshima University Email: odaken@c.kagoshima-u.ac.jp Abstract After the inputs X are known, how much additional information does the label Y carry about which dataset a sample came from? That single quantity—estimable as a difference of two discriminators’ held-out cross-entropies, DCJS=CE(Z∣X)−CE(Z∣X,Y)D_CJS=CE(Z X)-CE(Z X,Y)—is exactly the part of a dataset difference that covariate shift cannot explain. Deciding whether two supervised learning problems share the same input–output mechanism is the core primitive behind expert reuse in continual learning, drift-type diagnosis, and category discovery. Existing task comparisons fall into two families with complementary blind spots: input-distribution distances (MMD, Wasserstein) are blind to changes of P(Y∣X)P(Y X), while exchange-based scores (cross-evaluating models trained on each task, e.g. the Cross-Learning Score) conflate covariate shift with mechanism change because a model evaluated off its training support incurs extrapolation error. We propose the Conditional Jensen–Shannon Discrepancy (CJSD): with a task indicator Z, the chain rule I(Z,X,Y)=I(Z,X)+I(Z;Y∣X)I(Z;X,Y)=I(Z;X)+I(Z;Y X) splits total task discrepancy exactly into a covariate axis and a functional axis, and the functional axis is estimable as the difference of the held-out cross-entropies of two discriminators—one seeing x, one seeing (x,y)(x,y)—without training task-specific predictors, generative models, or bootstrap surrogates. We prove a covariate-null property (the functional axis is exactly zero under pure covariate shift, however severe), a drift-mass law (DCJS/ln2D_CJS/ 2 equals the mass of the disagreement region for deterministic labels), a one-sided misspecification-control inequality (the loss-gap estimand overshoots DCJSD_CJS by at most the excess risk of the x-discriminator and undershoots by at most that of the (x,y)(x,y)-discriminator, so each one-sided decision rests on a single excess risk), and a fixed-measure metrization: conditional distributions are identifiable from the discriminators via a likelihood-ratio identity, yielding a true metric between mechanisms under a fixed reference measure (the pair-dependent quantity itself is provably non-metric). Empirically, on a ten-measure battery over synthetic, Electricity, and Covertype pair families (202 pairs; separate experiments cover INSECTS, MNIST, and CIFAR-10), only the two conditional-information estimators—CJSD and a neighborhood plug-in CMI—separate concept from covariate shift with AUC 1.01.0 (all others 0.00.0–0.900.90), at equal drift-mass sensitivity; the case for CJSD is the estimator: under controlled dimensionality scaling the kNN plug-in fails from d=64d=64 while the discriminator route holds to d=256d=256 with a swappable classifier, and it alone yields paired per-point confidence intervals and sequential extensions from the same learned object. The same estimator audits the conditional fidelity of synthetic-data generators that marginal and joint QA metrics pass, detects annotation-guideline changes invisible to any input-space monitor, and supports null-calibrated fairness audits of conditional demographic disparity. 1 Introduction Online systems that maintain a pool of predictive models must repeatedly answer one question: is the data now arriving governed by the same input–output mechanism as the data an existing model was trained on? Answering it wrongly in one direction wastes capacity (spawning experts for data an existing expert already explains); wrongly in the other direction corrupts experts (absorbing data whose labeling mechanism has changed). The question also underlies drift-type diagnosis (should we retrain, or reweight?), on-the-fly category discovery (is this a new category, or a new appearance of an old one?), and data-pipeline quality control (did the annotation guideline change?). Two families of task-comparison measures dominate. Input-distribution distances (MMD, optimal transport) compare P(X)P(X) and are constitutionally blind to mechanism change. Exchange-based scores train a predictor per task and cross-evaluate: the recently proposed Cross-Learning Score (CLS) [1] symmetrizes the excess risk of swapped predictors and is, at the population level, identical to the reciprocal-regret quantity we call CPD. Exchange-based scores do respond to mechanism change, but we show they carry a structural confound: under pure covariate shift (identical P(Y∣X)P(Y X), shifted P(X)P(X)), the swapped model is evaluated off its training support, and its extrapolation error masquerades as mechanism change. On a two-dimensional benchmark the exchange score inflates from 0.0010.001 to 0.3460.346 as the supports separate, with no change of mechanism whatsoever; sharing a deep encoder does not remove the effect (0.800.80 at 90∘90 input rotation). We take a different route. Pool the two datasets with a task indicator Z and consider how identifiable Z is. The mutual-information chain rule I(Z,X,Y)=I(Z,X)⏟covariate axis Ix+I(Z;Y∣X)⏟functional axis DCJSI(Z;X,Y)\;=\; I(Z;X)_covariate axis I_x\;+\; I(Z;Y X)_functional axis D_CJS (1) splits total discrepancy into what the inputs explain and what only the input–output relation explains. Both terms are estimable from two discriminators: T1:x↦ZT_1:x Z and T2:(x,y)↦ZT_2:(x,y) Z, via DCJS^=CE(Z∣X)−CE(Z∣X,Y) D_CJS=CE(Z X)-CE(Z X,Y) on held-out data (Fig. 1). No task-specific predictor is trained, so nothing is ever evaluated off-support; no generative model or nearest-neighbor bootstrap is needed, unlike classifier-based CMI estimators [2] and conditional independence tests [3]. dataset A(x,y)(x,y), Z=0Z=0dataset B(x,y)(x,y), Z=1Z=1pool +cross-fitT1:x→ZT_1:\ x→ ZT2:(x,y)→ZT_2:\ (x,y)→ ZI^x=ln2−CE1 I_x= 2-CE_1covariate axisD^CJS=CE1−CE2 D_CJS=CE_1-CE_2functional axisheld-out CE1CE_1held-out CE2CE_2 Figure 1: The whole estimator. Two ordinary classifiers predict the dataset indicator Z, one from x, one from (x,y)(x,y); subtracting their held-out cross-entropies yields the functional axis—the information about Z that Y adds after X is known. Nothing is ever evaluated off its training support. Contributions. (1) We define CJSD and derive its basic representation (weighted conditional Jensen–Shannon divergence) and the exact two-axis decomposition (1). (2) We prove four properties that make it suitable as a decision primitive: a covariate-null theorem (Prop. 1), a drift-mass law (Prop. 2), a one-sided misspecification-control inequality (Prop. 4: each direction of error is bounded by the excess risk of a single discriminator, with no assumption on the other), and a fixed-measure metrization with an identifiability lemma recovering both conditionals from the discriminators (Prop. 3, Lemma 2). (3) We give a simple cross-fitted estimator with paired confidence intervals, robust to sample-size asymmetry (balanced by construction), whose misspecification bias was moreover consistently downward in all main benchmark configurations (an empirical refinement that sharpens, but does not carry, the one-sided guarantees; Appendix A exhibits an engineered counterexample in which the sign reverses, within the proven slack). (4) Across ten competing measures on three pair families (with further datasets in dedicated experiments), the two conditional-information estimators (CJSD and a kNN CMI plug-in) alone separate concept from covariate shift cleanly (AUC 1.01.0 vs 0.00.0–0.900.90) at equal drift-mass sensitivity; CJSD’s advantage over the plug-in is practicality, not the estimand (Sec. 5); we demonstrate downstream value in synthetic-data fidelity auditing, drift-type diagnosis, annotation-drift detection, and fairness auditing. 2 Related work Exchange-based and supervised task similarity. CLS [1] symmetrizes swapped excess risks; Taskonomy, LEEP, LogME estimate directed transferability; OTDD [6] transports jointly over features and labels, and Task2Vec [7] embeds tasks via Fisher information — all target transfer or global similarity rather than conditional-mechanism equality on the observable region. These quantify reuse risk of a trained model, which mixes mechanism change with support mismatch; our experiments quantify the confound. CJSD instead compares the mechanisms themselves on the region where both are observable. Input-distribution and representation distances. MMD, OT/W2, CKA compare P(X)P(X) or internal representations; they are blind to P(Y∣X)P(Y X) (AUC 0.00.0 in our concept-vs-covariate task). Conditional two-sample testing and CMI estimation. Conditional equality of P(Y∣X)P(Y X) can be tested via density-ratio reductions or conformal devices [4, 5]; CMI can be estimated with classifiers plus generative/bootstrap surrogates [2, 3]. Our estimator needs neither surrogate, and our aim differs: not a p-value but a bounded, normalized, decomposable discrepancy (metrizable in its fixed-measure form) that can gate online decisions. Drift detection. Error-stream detectors (ADWIN, DDM) implicitly assume mechanism change; input-space detectors (D3) see only covariate shift. CJSD’s two axes diagnose the type (Sec. 6). 3 The conditional Jensen–Shannon discrepancy Let tasks A,BA,B have laws PT=PTX⊗ηT(⋅∣x)P_T=P_T^X _T(· x) on ×X×Y, ||=K|Y|=K. Draw Z∼Bern(1/2)Z (1/2) and (X,Y)|Z=T∼PT(X,Y) Z=T P_T; let μ~=12(PAX+PBX) μ= 12(P_A^X+P_B^X) and w(x)=P(Z=A∣X=x)w(x)=P(Z=A X=x). Lemma 1 (Representation). I(Z,X)=JS(PAX,PBX)I(Z;X)=JS(P_A^X,P_B^X) and DCJS:=I(Z;Y∣X)=x∼μ~[JSw(x)(ηA(⋅∣x),ηB(⋅∣x))]D_CJS:=I(Z;Y X)=E_x μ\! [JS_w(x) ( _A(· x), _B(· x) ) ], where JSw(p,q)=H(wp+(1−w)q)−wH(p)−(1−w)H(q)JS_w(p,q)=H(wp+(1-w)q)-wH(p)-(1-w)H(q). Moreover (1) holds, and 0≤DCJS≤ln20≤ D_CJS≤ 2. Proposition 1 (Covariate null). If ηA(⋅∣x)=ηB(⋅∣x) _A(· x)= _B(· x) for μ~ μ-a.e. x then DCJS(A,B)=0D_CJS(A,B)=0, for arbitrary PAX,PBXP_A^X,P_B^X (including disjoint supports). Conversely DCJS=0D_CJS=0 implies ηA=ηB _A= _B at μ~ μ-a.e. x with w(x)∈(0,1)w(x)∈(0,1). Proposition 2 (Drift-mass law). If PAX=PBX=PXP_A^X=P_B^X=P_X and both labels are deterministic, ηT(⋅∣x)=δfT(x) _T(· x)= _f_T(x), then DCJS(A,B)=ln2⋅PX(fA(X)≠fB(X))D_CJS(A,B)= 2· P_X (f_A(X)≠ f_B(X) ). Lemma 2 (Reconstruction). Wherever w(x)∈(0,1)w(x)∈(0,1), with qy(x)=P(Z=A∣X=x,Y=y)q_y(x)=P(Z=A X=x,Y=y) and pooled conditional m(y∣x)m(y x), ρy(x):=ηA(y∣x)ηB(y∣x)=qy1−qy⋅1−w _y(x):= _A(y x) _B(y x)= q_y1-q_y· 1-ww, and ηB(y∣x)=m(y∣x)w(x)ρy(x)+1−w(x) _B(y x)= m(y x)w(x) _y(x)+1-w(x), ηA=ρyηB _A= _y\, _B. Hence both conditionals are identifiable from (T1,T2)(T_1,T_2) plus one pooled label model, for any finite K. Proposition 3 (Fixed-measure metrization). For a fixed reference measure μ, dμ(A,B):=x∼μJS1/2(ηA(⋅∣x),ηB(⋅∣x))d_μ(A,B):= E_x μ\,JS_1/2( _A(· x), _B(· x)) is a metric on conditionals modulo μ-null sets. With pair-dependent mixtures in place of μ, the triangle inequality fails (numerical counterexamples in 11% of random triples with heterogeneous supports). Anti-coupling of the axes. Since I(Z,X,Y)≤H(Z)=ln2I(Z;X,Y)≤ H(Z)= 2, the chain rule forces DCJS≤ln2−IxD_CJS≤ 2-I_x: severe covariate separability caps the observable functional signal. The axes are additive, not orthogonal; a large IxI_x means a small DCJSD_CJS is inconclusive, and the decision layer must defer rather than conclude “no mechanism change.” Proofs are in Appendix A. Together the propositions delimit exactly what a conditional discrepancy can honestly claim: differences are measured where both mechanisms are observable (w∈(0,1)w∈(0,1)); outside that region CJSD reports zero, not a hallucinated difference—while the covariate axis IxI_x reports how separable the inputs are, which by the anti-coupling inequality caps the functional signal that can be observed at all; the decision layer routes such cases to defer. 4 Estimation Assumptions. Throughout: (A1) predicted probabilities are clipped to [ϵ,1−ϵ][ε,1-ε], making per-point losses bounded; (A2) estimates are cross-fitted, so each held-out loss is computed by a model not trained on that point; (A3) the task prior is Bern(1/2)Bern(1/2), enforced by balanced subsampling. Under (A1)–(A3), if the two discriminators are log-loss risk-consistent (CE(T1)→H(Z∣X)CE(T_1)→ H(Z X), CE(T2)→H(Z∣X,Y)CE(T_2)→ H(Z X,Y) in probability), then D^CJS→DCJS D_CJS→ D_CJS; with fixed nuisances the paired difference obeys a CLT, which is what the reported intervals track (we do not claim finite-sample coverage under model selection). Train T1T_1 on (xi,zi)\(x_i,z_i)\ and T2T_2 on ((xi,yi),zi)\((x_i,y_i),z_i)\ with any probabilistic classifier (cross-fitted); on held-out points compute per-point log-losses ℓi(1),ℓi(2) ^(1)_i, ^(2)_i and set D^CJS=ℓ(1)−ℓ(2)¯ D_CJS= ^(1)- ^(2), I^x=ln2−ℓ(1)¯ I_x= 2- ^(1), with the paired empirical variance of ℓi(1)−ℓi(2) ^(1)_i- ^(2)_i giving confidence intervals; unequal task sizes are balanced by subsampling so that Z∼Bern(1/2)Z (1/2) holds by construction and the ln2 2 normalization remains exact; a one-hot interaction map [x,oh(y),x⊗oh(y)][x,oh(y),x (y)] suffices for linear discriminators. Three practical properties (validated in Sec. 5): (i) no capacity-asymmetry confound: unlike exchange scores no per-task model exists, so unequal task sample sizes (250250 vs 80008000) leave the estimate stable (Fig. 6, left); (i) one-sided misspecification control: with achievable risks R1=H(Z∣X)+ϵ1R_1=H(Z X)+ _1, R2=H(Z∣X,Y)+ϵ2R_2=H(Z X,Y)+ _2 (ϵ1,ϵ2≥0 _1, _2≥ 0 the excess log-losses of the two learned discriminators) the estimand of the loss gap is D~=DCJS+ϵ1−ϵ2 D=D_CJS+ _1- _2, whose sign is not controlled in general—but each direction of error is controlled by a single discriminator (Prop. 4 below); in all main benchmark configurations, with T1T_1’s inputs nested in T2T_2’s and matched architectures, the observed bias was moreover consistently downward (ϵ1≤ϵ2 _1≤ _2, shrinkage toward zero), which we report as an empirical refinement rather than an assumption (the engineered exception is in Appendix A); (i) negative empirical or surrogate values occur, through misspecification asymmetry (ϵ2−ϵ1>DCJS _2- _1>D_CJS makes even the population-level D~ D negative) as well as finite-sample noise; we never clip and instead use intervals or a null calibration (Sec. 6). Proposition 4 (One-sided misspecification control). For any fixed discriminator pair (T1,T2)(T_1,T_2), however misspecified or overfit, the population loss gap D~=R1−R2 D=R_1-R_2 satisfies DCJS−ϵ2≤D~≤DCJS+ϵ1.D_CJS- _2\;≤\; D\;≤\;D_CJS+ _1. Consequently: (a) overshoot is bounded by the simple discriminator alone—D~>τ D>τ implies DCJS>τ−ϵ1D_CJS>τ- _1 for any T2T_2; (b) undershoot is bounded by the joint discriminator alone—D~<τ D<τ implies DCJS<τ+ϵ2D_CJS<τ+ _2 for any T1T_1; (c) |D~−DCJS|≤max(ϵ1,ϵ2)| D-D_CJS|≤ ( _1, _2). For empirical-risk minimization over a class ℱ1F_1 with the clipped log-loss bounded by B, standard uniform convergence gives, with probability at least 1−δ1-δ, ϵ1≤A1+O(ℜn(ℓ∘ℱ1)+Bln(1/δ)/n) _1≤ A_1+O ( R_n( _1)+B (1/δ)/n ), where A1A_1 is the approximation error of ℱ1F_1 for the target P(Z∣x)P(Z x) and ℜn(ℓ∘ℱ1) R_n( _1) the Rademacher complexity of the clipped-loss-composed class [8] (constants absorbed in O(⋅)O(·); a Lipschitz contraction converts this to the raw class)—a bound that involves only the x-discriminator, whose lower-dimensional x-only target was empirically the easier of the two to estimate. The downward-bias regularity ϵ1≤ϵ2 _1≤ _2 is exactly the statement that the overshoot slack in (a) vanishes; none of (a)–(c) requires it. The practical reading: an alarm (large D^CJS D_CJS, feeding spawn or drift decisions) can only be inflated by ϵ1 _1, the excess risk of the marginal discriminator—the quantity held-out model selection already minimizes—no matter how badly T2T_2 behaves; a clearance (small D^CJS D_CJS, feeding reuse decisions) can only be deflated by ϵ2 _2. Each one-sided decision therefore rests on the quality of one learned object, and the two sides can be audited separately (a matched-null calibration empirically compensates the residual offset ϵ1−ϵ2 _1- _2, Sec. 6). In one sentence: surrogate-level decisions are exact; population-level readings inherit one-sided, discriminator-specific slacks unconditionally, and exact zero-slack conservativeness is the special case obtained either by threshold correction with a valid excess-risk bound (holding on that bound’s 1−δ1-δ event, so failure budgets compose additively) or under the empirically observed downward-bias regularity. Appendix A reports direct measurements of (ϵ1,ϵ2)( _1, _2) on analytic mixtures: the downward direction held in every adequately capacitated cell, and an engineered misspecified-marginal counterexample produced an upward false signal that stayed within the ϵ1 _1 slack and disappeared under a flexible discriminator. 5 Experiments Figure 2: Ten-measure comparison over 202 pairs across three data families. Left: concept-vs-covariate discrimination AUC (the two conditional-information estimators, CJSD and kNN-CMI, reach 1.01.0). Middle: false signal under pure covariate shift, normalized by the concept signal (00 is ideal). Right: drift-mass sensitivity is preserved. Identities. On a rotation family with known η, the estimator tracks the population value; the drift-mass law holds exactly in population and within estimator shrinkage in finite samples; D^CJS D_CJS violated the triangle inequality in 0/2000/200 random triples under a fixed measure (Fig. 3). Figure 3: Identity checks. Left: estimator consistency against the population value. Middle: the drift-mass law DCJS/ln2=ρD_CJS/ 2=ρ. Right: the covariate null—the exchange-based score inflates with support separation while D^CJS D_CJS stays at zero. The covariate null, empirically. Under pure support shift the exchange score inflates (0.0510.051 at s=2s=2, 0.3460.346 at s=4s=4) while D^CJS∈[−0.002,0.002] D_CJS∈[-0.002,0.002] throughout; on MNIST/CIFAR rotations the same holds with a frozen shared encoder, where deep-CLS head exchange inflates to 0.800.80. Ten-measure comparison. Across synthetic, Electricity and Covertype pair batteries (202 pairs), concept-vs-covariate AUC: MMD/SW2 0.00.0; disagreement, −-LEEP, one-way regret, CPD0-1^0-1(=CLS), CPDlog 0.780.78–0.820.82; a CCIT-style local-permutation classifier 0.900.90; CJSD 1.001.00 at equal drift-mass rank correlation (ρ≈0.9ρ≈ 0.9) (Fig. 2). A neighborhood plug-in CMI baseline also attains 1.001.00 on this (≤54≤ 54-dimensional) tabular battery—as expected, since it estimates the same population quantity. The contribution of CJSD is therefore not the discrimination ability of the estimand but the estimator: no nearest-neighbor geometry, paired per-point confidence intervals and sequential e-process extensions from the same learned object, and exact [0,ln2][0, 2] normalization. The next experiment makes the geometry claim concrete. Dimensionality scaling. We embed the same task (2 informative dimensions, drift mass ρ=0.3ρ=0.3, n=6000n=6000 per side) in d total dimensions, d=8d=8 to 512512 (8 seeds each; the covariate condition shifts all d coordinates). Concept-vs-covariate AUC (Fig. 4): the kNN plug-in is perfect through d=32d=32, degrades at d=64d=64 (0.720.72), and is at chance from d=256d=256—distance concentration destroys the local neighborhoods it depends on, and it has no tunable remedy. The CCIT-style permutation classifier collapses immediately (0.410.41 at d=16d=16), since its local permutations also rest on kNN geometry. The discriminator route holds AUC 1.001.00 through d=128d=128 with the same random-forest discriminators used everywhere else in this paper, and—the practical point—when the forest finally loses the signal at d=256d=256, swapping the discriminator (gradient boosting, one line) restores 1.001.00 at d=256d=256. Estimates shrink toward zero as d grows (downward, never inflating; Fig. 4, right), consistent with the misspecification analysis of Sec. 4. Figure 4: Dimensionality scaling (2 informative dims ++ noise, ρ=0.3ρ=0.3, 8 seeds). Left: concept-vs-covariate AUC; kNN geometry fails from d=64d=64, the discriminator route holds to d=128d=128 (RF) / d=256d=256 (HGB). Right: mean estimate under concept drift; shrinkage is downward. The n×dn× d phase diagram: geometry limits vs. sample limits. Whether a failure boundary is geometric or merely a matter of sample size is the question that decides which estimator to trust in embedding spaces. Extending the grid to n∈1,3,6,12kn∈\1,3,6,12\k per side (6 seeds; Fig. 5) separates the two failure modes cleanly. For the discriminator route, more data moves the boundary: the RF frontier advances from d=64d=64 at n=1kn=1k to d=256d=256 at n=12kn=12k (0.920.92 at 512512), and the HGB discriminator reaches AUC 1.001.00 on the entire grid at n=12kn=12k, including d=512d=512. For the kNN plug-in, more data did not help over the tested range: it never establishes reliable discrimination beyond d=64d=64 at any n tried (0.220.22–0.830.83 across the d≥128d≥ 128 cells, non-monotone in n). The distinction matters because it turns the earlier scaling curve into guidance: a discriminator-based estimate that fails at the current sample size can be rescued by data or a stronger classifier; over the tested range, a neighborhood-based estimate at high d had no such lever. (One implementation note we found the hard way: sklearn’s gradient boosting silently enables early stopping above 10410^4 samples, which destroys the subtle T2T_2 signal; estimator configurations must be held fixed across n.) Figure 5: n×dn× d phase diagram of concept-vs-covariate AUC (6 seeds per cell). Left: the kNN plug-in is geometry-limited—no row reaches reliable discrimination past d=64d=64. Middle/right: the discriminator route is sample-limited—the boundary moves outward with n, and HGB discriminators clear the whole grid at n=12kn=12k. Figure 6: Left: robustness to sample-size asymmetry (nA=8000n_A=8000 fixed). Middle/right: over the s×θs×θ grid each estimated axis responds to its own factor only (additive decomposition; note the feasible region is anti-coupled, DCJS≤ln2−IxD_CJS≤ 2-I_x). Real drifts with documented change points. On INSECTS the two axes decompose each documented drift into covariate and functional components (Ix∈[0.26,0.31]I_x∈[0.26,0.31], DCJS∈[0.13,0.24]D_CJS∈[0.13,0.24]) and recover, without segment supervision, the recurrence structure of the temperature cycle: the recurring segment pairs (0,2),(0,5),(2,5)(0,2),(0,5),(2,5) fall at DCJS≤0.03D_CJS≤ 0.03, clustering with the within-segment nulls. The full anatomy figure and its use for drift-type monitoring appear in the companion diagnosis paper; we cite the numbers here rather than reproduce its figure. Figure 7: Real-data pair batteries in the CJSD plane: the four constructed shift types occupy the four quadrants; pure covariate pairs sit at D^CJS≈0 D_CJS≈ 0 (the exchange score places them at 0.040.04–0.100.10). Decisions. The two-axis gate turns these estimates into reuse/spawn/defer decisions with near-perfect quadrant accuracy; system-level results (streaming, anytime-valid e-process gating, expert pools) are developed in a companion paper. 6 Applications Companion-paper applications. The two-axis statistic is the decision core of a drift-type diagnosis monitor (real vs. virtual vs. incomparable alarms, benchmarked against standard detectors and WATCH) and of an embedding-space novelty separator for deep expert pools; both are developed and evaluated in their own companion papers, and we do not reproduce their results here. Annotation-guideline drift. Swapping two confusable Covertype classes on a boundary region (22% mass) is invisible to every input monitor (IxI_x, MMD ≡0≡ 0) yet detected at 9σ9σ. On CIFAR-10H the same audit detects a 10%10\% label-corruption positive control at z=3.3z=3.3, while real annotator-population splits lie below the calibrated detection floor (|ΔD|<10−3| D|<10^-3)—a sensitivity-floor statement, not a separation claim (Appendix). Conditional-fidelity audit of synthetic data (new, exclusive to this paper). A tabular generator is judged by whether it preserves P(Y∣X)P(Y X). We built three generators over the same bootstrap X-generator (disjoint source/evaluation halves, so all X-marginal statistics coincide by construction): faithful (y∼P^(y∣x)y P(y x)), shuffled (y∼P^(y∣x′)y P(y x ) for a random other row—X- and y-marginals exactly preserved, joint destroyed), and subtle (conditional flipped on a top-decile region). On Adult and Covertype, mean per-feature KS and the marginal-y gap are blind to both corruptions, and joint MMD moves at noise level (0.00020.0002–0.00140.0014); D^CJS D_CJS reads ≤0.0002≤ 0.0002 for the faithful generator, 0.0410.041–0.0740.074 for shuffled, and 0.0080.008–0.0380.038 for subtle (z≈20z≈ 20 on the hardest case that every marginal metric passes). Conditional fidelity is precisely the axis marginal QA suites do not measure. Fairness auditing is a further application (with Z a protected attribute, DCJSD_CJS is conditional demographic disparity I(Z;Y∣X)I(Z;Y X)); because its interpretation raises normative questions orthogonal to the estimator, we develop it in Appendix B only. 7 Limitations CJSD gates no-adaptation reuse; it does not predict fine-tuning transferability (LEEP correlates ρ=0.86ρ=0.86 with fine-tuned accuracy, CJSD does not, by design: differences outside the overlap are honestly reported as zero and adaptation changes the representation). Estimates depend on discriminator calibration (in our experiments under-trained discriminators shrank the signal; the sign of the bias is uncontrolled in general, but each direction of error is bounded by a single excess risk, Prop. 4); the weighted-JS form attenuates differences confined to low-overlap regions, which is the identifiability boundary made visible. Reproducibility All experiments (46 scripts, checkpointed JSON results, figures) are included in the supplementary package. References [1] S. Sun, H. H. Zhang, J. C. Watkins. Quantifying data similarity using cross learning. arXiv:2510.10866, 2025. [2] S. Mukherjee, H. Asnani, S. Kannan. CCMI: Classifier based conditional mutual information estimation. UAI 2019. [3] R. Sen et al. Model-powered conditional independence test. NeurIPS 2017. [4] S. Lee, S. Cha, I. Kim. General frameworks for conditional two-sample testing. arXiv:2410.16636, 2024. [5] X. Hu, J. Lei. A two-sample conditional distribution test using conformal prediction. arXiv:2010.07147, 2020. [6] D. Alvarez-Melis, N. Fusi. Geometric dataset distances via optimal transport. NeurIPS 2020. [7] A. Achille et al. Task2Vec: task embedding for meta-learning. ICCV 2019. [8] P. L. Bartlett, S. Mendelson. Rademacher and Gaussian complexities: risk bounds and structural results. JMLR, 2002. [9] D. M. Endres, J. E. Schindelin. A new metric for probability distributions. IEEE Trans. IT, 2003. Appendix A Proofs Proof of Lemma 1. I(Z,X)=H(Z)−H(Z∣X)=ln2−μ~[H(w(X))]I(Z;X)=H(Z)-H(Z X)= 2-E_ μ[H(w(X))], which is the mixture representation of JS(PAX,PBX)JS(P_A^X,P_B^X). Conditionally on X=xX=x we have Z∼Bern(w(x))Z (w(x)) and Y∣Z=T∼ηT(⋅∣x)Y Z=T _T(· x), so I(Z;Y∣X=x)=H(Y∣X=x)−H(Y∣X=x,Z)=H(wηA+(1−w)ηB)−wH(ηA)−(1−w)H(ηB)=JSw(ηA,ηB)I(Z;Y X=x)=H(Y X=x)-H(Y X=x,Z)=H(w _A+(1-w) _B)-wH( _A)-(1-w)H( _B)=JS_w( _A, _B). Taking μ~E_ μ gives the claim; the chain rule is the standard information identity, and 0≤JSw≤ln20 _w≤ 2 gives the bounds. ∎ Proof of Proposition 1. JSw(p,p)=0JS_w(p,p)=0 for every w, so the first claim follows pointwise from Lemma 1. For the converse, if w(x)∈(0,1)w(x)∈(0,1) then strict concavity of H makes JSw(p,q)=0JS_w(p,q)=0 iff p=qp=q. At points with w∈0,1w∈\0,1\, JSw≡0JS_w≡ 0 carries no information: this is the identifiability boundary, and DCJSD_CJS reports zero rather than an unverifiable difference. ∎ Proof of Proposition 2. With w≡1/2w≡ 1/2: where fA(x)=fB(x)f_A(x)=f_B(x), JS1/2(δ,δ)=0JS_1/2(δ,δ)=0; where they differ, JS1/2(δa,δb)=H(12δa+12δb)=ln2JS_1/2( _a, _b)=H( 12 _a+ 12 _b)= 2. Integrate over PXP_X. ∎ Proof of Lemma 2. Bayes: qy=wηA(y)wηA(y)+(1−w)ηB(y)q_y= w\, _A(y)w\, _A(y)+(1-w) _B(y), hence qy1−qy=w1−w⋅ηA(y)ηB(y) q_y1-q_y= w1-w· _A(y) _B(y), giving ρy _y. Substituting ηA=ρyηB _A= _y _B into m=wηA+(1−w)ηB=(wρy+1−w)ηBm=w _A+(1-w) _B=(w _y+1-w) _B solves for ηB _B. ∎ Proof of Proposition 3. δ(x)=JS1/2(ηA(x),ηB(x))δ(x)= JS_1/2( _A(x), _B(x)) is, for each x, a metric between the conditional laws (Endres–Schindelin). Then dμ(A,C)=‖δAC‖L2(μ)≤‖δAB+δBC‖L2(μ)≤‖δAB‖L2(μ)+‖δBC‖L2(μ)d_μ(A,C)=\| _AC\|_L^2(μ)≤\| _AB+ _BC\|_L^2(μ)≤\| _AB\|_L^2(μ)+\| _BC\|_L^2(μ) by the pointwise triangle inequality and Minkowski. Identity of indiscernibles follows from dμ=0⇔JS1/2=0d_μ=0 _1/2=0 μ-a.e. For the pair-dependent variant we exhibit numerical violations (11% of 300 random heterogeneous-support triples, worst slack −0.19-0.19). ∎ Proof of Proposition 4. By definition D~=R1−R2=(H(Z∣X)+ϵ1)−(H(Z∣X,Y)+ϵ2)=DCJS+ϵ1−ϵ2 D=R_1-R_2=(H(Z X)+ _1)-(H(Z X,Y)+ _2)=D_CJS+ _1- _2. The population log-loss of any predictor is at least the conditional entropy of its target (Gibbs’ inequality), so ϵ1≥0 _1≥ 0 and ϵ2≥0 _2≥ 0 hold unconditionally, and the two-sided sandwich follows by dropping one nonnegative term at a time; (a)–(c) are immediate. The high-probability excess-risk bound for ϵ1 _1 is the standard symmetrization argument for bounded losses applied to ℓ∘ℱ1 _1 [8], combined with A1=inff∈ℱ1R1(f)−H(Z∣X)A_1= _f _1R_1(f)-H(Z X). ∎ Appendix B Additional experimental details Ten-measure battery (202 pairs). Synthetic rotation family (s×θs×θ and localized drift), Electricity and Covertype constructed pairs (null / covariate via PC-biased sampling / concept via region-restricted cyclic remap with recorded realized drift mass / both). Per-family AUCs: concept-vs-null 1.01.0 for every function-based measure; concept-vs-covariate: CJSD 1.000/1.000/1.0001.000/1.000/1.000 (synthetic/Electricity/Covertype), kNN-CMI plug-in likewise 1.0001.000 on this ≤54≤ 54-dimensional battery, CCIT-style local-permutation classifier 0.900.90, best remaining competitor 0.6670.667–1.0001.000 with mean 0.790.79. Covariate inflation (cov−-null)/(con−-null): CJSD −0.011-0.011–0.0100.010; disagreement 0.390.39; LEEP 0.640.64; one-way 0.420.42; CPD0-1^0-1 0.440.44; CPDlog 0.360.36. Direct measurement of the excess risks (Prop. 4). On analytic two-task Gaussian mixtures where H(Z∣X)H(Z X), H(Z∣X,Y)H(Z X,Y), and DCJSD_CJS are computable from the true posteriors, we measured (ϵ1,ϵ2)( _1, _2) of fitted pairs directly (six scenarios × linear-logistic, HGB × five seeds; 80008000 fitting points, population risks on 2×1052× 10^5 Monte-Carlo points). The sandwich of Prop. 4 held in all 6060 cells (as it must; the identity D~−DCJS=ϵ1−ϵ2 D-D_CJS= _1- _2 was machine-exact), and the downward direction ϵ1≤ϵ2 _1≤ _2 held in every cell with adequate capacity—nulls with and without covariate shift, concept drifts of two masses, a label flip—with one engineered exception: a scale-only covariate shift whose T1T_1 target is quadratic in x (unreachable for a linear-logistic ℱ1F_1) while y proxies exactly that quadratic statistic, producing ϵ1=0.1492>ϵ2=0.1255 _1=0.1492> _2=0.1255 and a false conditional signal D~=+0.0237 D=+0.0237 at DCJS=0D_CJS=0—within the ϵ1 _1 slack, as the proposition requires. Replacing both discriminators with HGB restored the downward direction in the same scenario (D~=−0.003 D=-0.003). The upward failure mode is therefore visible: it requires a poor marginal discriminator—and held-out CE1CE_1 model selection directly targets exactly this quantity among candidate discriminators—while a matched-null calibration empirically compensates the residual offset. Sequential monitoring. Betting e-processes on the per-point discriminator loss differences yield zero false alarms with +20%+20\% delay over (invalid) repeated CIs; an indifference zone [τ,3τ][τ,3τ] is required for the reuse-side test to be well posed. Bounded-memory recency with a lifetime guarantee is obtained by the companion system paper’s restarted e-detector construction. Fairness audit (Appendix B). Because a flexible T2T_2 can be finitely biased, we audit against a fair control (labels resampled from a pooled model). Null-calibrated z-scores: COMPAS-sex 6.26.2, Adult-race 9.39.3, Adult-sex 3.93.9, COMPAS-race 1.71.7 (n.s. —dependence below detection given the other features, sharpening rather than settling the conditional-vs-marginal debate; the feature set remains a normative choice). CIFAR-10H annotator drift. With sufficient-statistic features (model softmax + one-hot label) the audit detects a 10%10\% random-corruption positive control at z=3.3z=3.3, while real annotator-population splits (fast/slow, low/high accuracy, human vs. ground truth) lie below the calibrated detection floor (|ΔD|<10−3| D|<10^-3); pair construction must use disjoint inputs (duplicated inputs create twin-copy memorization leakage). Negative estimates and calibration. D^CJS<0 D_CJS<0 arises from the extra features of T2T_2; we report intervals or null-calibrated differences and never clip. In our experiments, under-trained discriminators shrank D^CJS D_CJS toward zero in every case we ran (the sign is uncontrolled in general, Sec. 4) (e.g. CIFAR encoder at 60%60\% accuracy yields slope 0.550.55 against realized drift mass); this direction is an empirical regularity under nested discriminator inputs, not a theorem (see Sec. 4).