Paper deep dive
Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation
Cedric Caruzzo, Donggeun Yoo, Tae Soo Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/22/2026, 3:03:42 AM
Summary
This paper investigates routing divergence in Mixture-of-Experts (MoE) models during same-weight self-distillation, where a teacher and student share weights but route tokens differently due to conditioning. The authors propose an exact blockwise decomposition separating routing mismatch (A) from content shift (B). Across seven checkpoints, they find that routing mismatch magnitude is stable, but its exposure to the residual stream varies significantly based on the backbone share. Crucially, causal interventions show that routing mismatch alone has limited behavioral influence compared to content shifts, suggesting router movement is not sufficient evidence of behavioral change.
Entities (14)
Relation Signals (9)
Cedric Caruzzo → authored → Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation
confidence 100% · Title: Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation Cedric Caruzzo
Content shift term B → ispartof → Total routed output change
confidence 95% · B is the dense-like remainder... Δ_routed = A + B.
Routing mismatch term A → ispartof → Total routed output change
confidence 95% · The change in the routed output between the two passes, Δ_routed, splits by adding and subtracting... A + B = Δ_routed.
Backbone rescaling → affects → Routing coherence
confidence 90% · Scaling the always-on backbone... moves exposure monotonically... supports a mass-and-coherence mechanism
Backbone share → determines → Residual exposure
confidence 90% · Exposure is ordered by the routed block's share of the residual... exposure is the routing factor diluted by backbone share
OLMoE → hashigh → Residual exposure
confidence 90% · OLMoE shows the reverse (0.124 and 0.056)... OLMoE most exposed
Routing mismatch term A → haslimited → Behavioral influence
confidence 90% · Router movement alone is therefore not evidence of behavioral influence... the full routing term moves outputs by less than half the natural context effect
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Two Mixture-of-Experts (MoE) forward passes can share every weight yet route the same token through different experts. This creates a possible blind spot in same-weight self-distillation, where a demonstration-conditioned teacher supervises a query-only student. We study this mismatch in its single-step form, with frozen weights rather than as a proxy for a full training trajectory. An exact blockwise decomposition separates a routing term, which changes gates at fixed content, from a dense-like content term. Across seven open-weight checkpoints and two domains, the routing term spans only $1.6\times$ as a fraction of block output, while its residual-stream exposure spans $3.2\times$. Exposure is ordered by the routed block's share of the residual. Scaling the always-on backbone in two confirmatory models moves exposure monotonically; common-mode controls support a mass-and-coherence mechanism rather than denominator dilution alone. Preregistered PubMedQA patches on three models show that the full routing term moves outputs by less than half the natural context effect and is largely reproduced by matched-norm noise, whereas the content term is strongly direction-specific. Scale and merged-expert probes show that the narrow block-level range is not universal, although exposure remains small at the tested boundaries. Router movement alone is therefore not evidence of behavioral influence: measure exposure first, and use a behavioral intervention when the decision matters.
Tags
Links
- Source: https://arxiv.org/abs/2608.15787v1
- Canonical: https://arxiv.org/abs/2608.15787v1
Trouble viewing inline? Open PDF directly →
Full Text
53,961 characters extracted from source content.
Expand or collapse full text
Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation Cedric Caruzzo Affiliation: Lunit Email: cedric.caruzzo@lunit.io Donggeun Yoo Affiliation: Lunit Email: dgyoo@lunit.io Tae Soo Kim Thanks: Corresponding author. Affiliation: Lunit Email: taesoo.kim@lunit.io Abstract Two Mixture-of-Experts (MoE) forward passes can share every weight yet route the same token through different experts. This creates a possible blind spot in same-weight self-distillation, where a demonstration-conditioned teacher supervises a query-only student. We study this mismatch in its single-step form, with frozen weights rather than as a proxy for a full training trajectory. An exact blockwise decomposition separates a routing term, which changes gates at fixed content, from a dense-like content term. Across seven open-weight checkpoints and two domains, the routing term spans only 1.6×1.6× as a fraction of block output, while its residual-stream exposure spans 3.2×3.2×. Exposure is ordered by the routed block’s share of the residual. Scaling the always-on backbone in two confirmatory models moves exposure monotonically; common-mode controls support a mass-and-coherence mechanism rather than denominator dilution alone. Preregistered PubMedQA patches on three models show that the full routing term moves outputs by less than half the natural context effect and is largely reproduced by matched-norm noise, whereas the content term is strongly direction-specific. Scale and merged-expert probes show that the narrow block-level range is not universal, although exposure remains small at the tested boundaries. Router movement alone is therefore not evidence of behavioral influence: measure exposure first, and use a behavioral intervention when the decision matters. 1 Introduction Post-training a Mixture-of-Experts model introduces a complication absent from dense models: two forward passes can share every parameter yet execute different sparse computations. In same-weight self-distillation, a demonstration-conditioned teacher and a query-only student can route the same token to different experts. Existing work measures or suppresses router drift (Dai et al. 2022; Ma et al. 2025; Xie et al. 2026), but gate divergence alone does not say how much of the induced perturbation reaches the residual stream or the model output. We ask that downstream question in the single-step regime the method actually constructs. We hold weights fixed and compare the teacher and student passes before any update. An exact blockwise decomposition separates the change caused by moving the gates at fixed content from the dense-like change caused by shifting content at fixed gates. This lets us trace routing mismatch from the routed block to the residual stream and, through causal patches, to output behavior. Drift accumulated over many updates, or mismatch between separately parameterized teachers and students, is outside this study. The natural prediction is that a router that moves more creates more downstream risk. Our measurements invert it. Across seven checkpoints and two domains, the routing term is relatively stable as a share of block output, but its exposure to later layers varies with the routed block’s share of the residual. The most fine-grained router is among the least exposed. Backbone rescaling on two confirmatory checkpoints supports the causal account: always-on mass contains the mismatch chiefly by maintaining routing coherence. Output patches then show that the routing term is bounded and mostly magnitude-generic, while the dense-like content term is direction-specific. The claim is deliberately narrow: routing can matter, but router movement is not itself a measure of downstream influence. Our contributions are: • An exact A/BA/B decomposition and reconstruction gate that distinguish routing mismatch from content shift, together with residual exposure as the downstream quantity of interest. • A seven-checkpoint, two-domain measurement showing that exposure follows backbone share, and a two-model intervention showing that always-on mass preserves routing coherence. • Three-model causal A/BA/B patches that connect the decomposition to output behavior, plus scale and merged-expert probes that delimit rather than universalize the observed range. Relation to companion work. This report asks how much same-weight, conditioning-induced routing mismatch survives from an MoE block into the residual stream and what effect that term has when patched causally. A companion study analyzes how moved gate mass aligns with expert-output directions; a related challenge formulation asks how to allocate a limited budget of behavioral interventions. The works share an experimental program, but their primary estimands are distinct: residual exposure here, correspondence geometry there, and audit allocation in the challenge. 2 Preliminaries and the decomposition teacher passdemonstration ++ querystudent passquery onlyMoE blockshared weights θ → experts fjf_jgT,hTg^T,\,h^TgS,hSg^S,\,h^SA=∑j(gjT−gjS)fj(hS)A= _j(g^T_j-g^S_j)\,f_j(h^S)routing mismatch: same content, different gatesB=∑jgjT(fj(hT)−fj(hS))B= _jg^T_j (f_j(h^T)-f_j(h^S) )dense-like: same gates, different contentroutingdivergencedecompositionA,BA,\;Brouting factor rA/blkr_A/blk× backbone share rblk/resr_blk/resexposurerA/resr_A/resoutputinfluenceraw divergence does not set exposure; backbone share does Figure 1: The decomposition and its causal chain. Top: two forward passes of the same model with the same weights, one with a demonstration in context (teacher) and one without (student), produce different gate vectors and hidden states at each MoE block. The change in the routed output Δrouted=A+B _routed=A+B splits into a routing-mismatch term A (the same content hSh^S sent through the two passes’ different gates) and a dense-like remainder B (the teacher routing applied to the content shift). This paper measures A and its share of the residual stream (Eq. 4). Bottom: the routing factor varies less than exposure in the primary screen, while backbone share maps the block-level perturbation into residual exposure. Neither the observed range nor exposure alone is a behavioral guarantee. Setup. We compare two forward passes of the same model with the same weights (Figure 1). In the teacher pass the input carries a demonstration (the context a self-distillation teacher sees (Shenfeld et al. 2026)); in the student pass it does not. Superscripts T and S mark the pass. At an MoE block, for a given token, let gjg_j be the realized gate weight on expert j (the value applied to the expert output, which need not sum to one across the selected experts; see Σg g below), let fj(h)f_j(h) be expert j’s output on hidden state h, and let the routed output be routed=∑jgjfj(h)routed= _jg_jf_j(h). The block adds an optional shared-expert output and returns y; the residual stream carries resres into the block. Decomposition. The change in the routed output between the two passes, Δrouted=routedT−routedS _routed=routed^T-routed^S, splits by adding and subtracting ∑jgjTfj(hS) _jg^T_jf_j(h^S): A A =∑j(gjT−gjS)fj(hS), = _j(g^T_j-g^S_j)\,f_j(h^S), (1) B B =∑jgjT(fj(hT)−fj(hS)), = _jg^T_j\, (f_j(h^T)-f_j(h^S) ), (2) Δrouted _routed =A+B. =A+B. (3) A is the routing-mismatch term: the same hidden state hSh^S sent through the two passes’ different gate vectors. B is the dense-like remainder: the teacher routing applied to the content shift fj(hT)−fj(hS)f_j(h^T)-f_j(h^S), the term a dense model would also incur. The anchor choice (evaluating the expert functions at hSh^S in A) is one of two symmetric options; we use hSh^S throughout and note the alternative in Appendix A.11 1 The identity (3) is exact; we verify it per model with a reconstruction gate, ∥(A+B)−Δrouted∥/∥Δrouted∥<0.02 (A+B)- _routed / _routed <0.02. Quantized expert execution does not reproduce the identity and is excluded; robustness checks are in Appendix C. Three ratios. We report A’s magnitude relative to three denominators, all per token per layer: rA/blk=∥A∥y∥⏟routing factor,rblk/res=∥y∥res∥⏟backbone share,rA/res=∥A∥res∥⏟exposure=rA/blk⋅rblk/res. r_A/blk= A y _routing factor, r_blk/res= y _backbone share, r_A/res= A _exposure=r_A/blk· r_blk/res. (4) The routing factor is how much the routing mismatch perturbs the block’s own output. The exposure is how much it perturbs the residual stream that the rest of the network reads. Equation (4) (the wedge identity) says exposure is the routing factor diluted by backbone share: the smaller the routed block’s share of the residual, the more a fixed routing perturbation is diluted. We also report rA/rt=∥A∥/∥routed∥r_A/rt= A / (the shared expert removed from the denominator) and the context term rB/res=∥B∥/∥res∥r_B/res= B / . Glossary. A: routing-mismatch term, same content through different gates (Eq. 1). B: dense-like remainder, same gates on shifted content (Eq. 2). rA/blkr_A/blk (routing factor) =∥A∥/∥y∥= A / y , the mismatch’s share of the block output. rblk/resr_blk/res (backbone share) =∥y∥/∥res∥= y / , the routed block’s share of the residual. rA/resr_A/res (exposure) =∥A∥/∥res∥=rA/blk⋅rblk/res= A / =r_A/blk· r_blk/res, the mismatch’s share of the residual, the quantity that reaches the rest of the network. Σg g: sum of applied gate weights (convention scale). Residual convention. We measure exposure at two points. The screen (Section 4) uses the pre-block residual, the input to the post-attention layernorm. The intervention (Section 5) uses the post-block residual, res+yres+y, which is the well-posed local target for scaling y. The two differ by the block’s own output y at each token and are reported separately; every table states which is used. Gate convention and a magnitude bound. Because A uses the applied gate weights, its raw scale depends on whether a router renormalizes, is sub-stochastic, or applies a scaled convention. Dividing by the block output makes rA/blkr_A/blk less sensitive to that convention. Writing Fmax=maxj∥fj(hS)∥F_ = _j f_j(h^S) , Eq. 1 gives the exact triangle bound ∥A∥≤Fmax∥Δg∥1 A ≤ F_ g _1. For renormalized gate distributions this becomes ∥A∥≤2ΣgFmaxTV A ≤ 2 gF_ TV, making the convention scale explicit. We use the decomposition and reconstruction gate for measurement; the bound is an envelope, not an estimator. What the decomposition buys. The split (3) is an exact identity, so it is not a claim on its own; its value is that A isolates a quantity that can be measured in isolation, the change in the block output that comes purely from the gates moving, with hidden state and expert weights held fixed. That is not the same as the routed output changing, which also absorbs the content shift B, and it is not recoverable from a routing-distance metric such as Jaccard or TV, which see the gate vectors but not what the experts do with them. The rest of the paper asks how large A is relative to the block and residual, what controls that exposure, and what happens when the recorded term is injected causally at the output. Scope. Our instrument measures conditioning-induced routing mismatch between two forward passes of the same model: teacher and student share parameters and differ only in conditioning. This is the setting of self-distillation methods in which the teacher is the current model under additional conditioning (Shenfeld et al. 2026; Hübotter et al. 2026); for these methods, conditioning-induced mismatch is not a simplification of the general fine-tuning case, it is the whole of the routing mismatch present in the training signal. Pipelines that distill from a separate teacher checkpoint (Agarwal et al. 2024; Qwen Team 2025; GLM Team 2025) additionally introduce weight-induced routing mismatch between distinct routers, which is outside the scope of the present work. 3 Measurement methodology Models. We measure seven open-weight MoE checkpoints chosen to vary the architecture axes that a containment account should care about (Table 1): expert count and top-k (8/2 for Mixtral (Jiang et al. 2024) up to 128/8 for Qwen3 (Qwen Team 2025)), presence of an always-on shared expert (DeepSeek-V2-Lite (DeepSeek-AI 2024), Gravity-16B-A3B (Trillion Labs 2026), and Qwen1.5-MoE-A2.7B (Qwen Team 2024) have one; OLMoE (Muennighoff et al. 2024), Phi-3.5-MoE (Abdin et al. 2024), and the others do not), and gate convention (Σg g). All seven reproduce the decomposition identity. Larger Qwen3 and DeepSeek-V2 checkpoints enter only as boundary probes and do not define the observed range. Probes. Two domains carry the analysis, PubMedQA (Jin et al. 2019) (medical, n=200n=200) and GSM8K (Cobbe et al. 2021) (mathematical reasoning, n=200n=200). Each probe pair is a question presented with and without a worked demonstration; the demonstration is the only difference between the teacher and student inputs, so A isolates the routing response to the demonstration. Inputs use each model’s chat template where present. We aggregate per-layer-first: within a layer we sum numerator and denominator norms across tokens and divide once, then summarize over layers. The alternative (averaging per-token ratios) overweights small-residual tokens and inflates the headline. We report the per-layer-first value throughout; aggregation and prompt-format checks are in Appendix C. Uncertainty. We bootstrap over the n=200n=200 probe prompts (2000 resamples), recomputing the per-layer-first ratio under each resample, to place a 95%95\% confidence interval on every per-model headline. The intervals are tight: across the seven checkpoints and both domains, the per-model half-widths are at or below 0.0030.003 for the routing factor and 0.00120.0012 for the exposure, at most 5.5%5.5\% and 2.8%2.8\% of the widths of the corresponding two-domain ranges (Figure 2, Appendix B). The intervals establish that the seven point estimates are stable, not that the underlying quantity must cluster. Routing metrics and controls. The backbone intervention reports top-k overlap (Jaccard) and total variation (TV) on the renormalized gate distributions. Confound and format controls are summarized in Appendix C. Table 1: The seven primary checkpoints. Values are per-layer-first body aggregates, shown as PubMedQA/GSM8K. Backbone share is computed directly as ∥y∥/∥res∥ y / before averaging across layers; exposure uses the pre-block residual. Prompt-bootstrap intervals are in Appendix B. Model experts/k shared path rA/blkr_A/blk rblk/resr_blk/res rA/resr_A/res Qwen3-30B-A3B 128/8 – 0.133/0.139 0.174/0.178 0.019/0.021 Mixtral-8x7B 8/2 – 0.094/0.110 0.271/0.283 0.022/0.028 OLMoE-1B-7B 64/8 – 0.124/0.122 0.481/0.515 0.056/0.061 DeepSeek-V2-Lite 64/6 always-on 0.090/0.102 0.356/0.364 0.027/0.033 Gravity-16B-A3B 64/8 always-on 0.098/0.108 0.470/0.511 0.044/0.053 Qwen1.5-MoE-A2.7B 60/4 input-gated 0.085/0.102 0.464/0.500 0.036/0.048 Phi-3.5-MoE 16/2 – 0.106/0.110 0.246/0.262 0.023/0.027 range 0.085–0.139 0.174–0.515 0.019–0.061 4 Results I: backbone share orders exposure The routing factor varies less than exposure. Across the seven checkpoints and both domains, the per-layer-first routing factor rA/blkr_A/blk lies between 0.0850.085 and 0.1390.139, a 1.6×1.6× span (Figure 2). Exposure rA/resr_A/res spans 0.0190.019 to 0.0610.061, or 3.2×3.2×, while directly aggregated backbone share spans 0.1740.174 to 0.5150.515. Prompt-bootstrap intervals are narrow relative to these ranges (Appendix B), so the checkpoint estimates are precise. The range is nevertheless an empirical regularity of seven family-correlated checkpoints, not a population law. Backbone share tracks the inversion. The checkpoints vary from top-22-of-88 to top-88-of-128128 routing and include shared and shared-free blocks. Yet Qwen3-30B, the most fine-grained router, combines one of the largest routing factors (0.1330.133 on PubMedQA) with the smallest exposure (0.0190.019). OLMoE shows the reverse (0.1240.124 and 0.0560.056). Directly measuring ∥y∥/∥res∥ y / preserves this ordering (Figure 3); over the seven checkpoints its Spearman association with exposure is 0.960.96 on PubMedQA and 1.001.00 on GSM8K. The dense-like term B is also larger than A on every checkpoint (1.61.6–4.1×4.1×), motivating the output intervention in Section 5. The narrow range has a scale boundary. We hold Qwen3 expert count and top-k fixed while moving from 30B to 235B. The routing factor rises to 0.188/0.2000.188/0.200 on PubMedQA/GSM8K, above the observed range, while exposure remains 0.0300.030. The same pattern does not repeat in a second probe: DeepSeek-V2 at 236B remains at or inside the range (0.114/0.1390.114/0.139) and has exposure 0.012/0.0190.012/0.019. The two probes establish that the block-level range is not scale-universal, but they do not isolate a scaling law. Exposure remains within or below the primary observed range in both. Figure 2: Routing factor rA/blkr_A/blk for the seven primary checkpoints on PubMedQA (circles) and GSM8K (squares). Whiskers are 95%95\% confidence intervals from a bootstrap over the 200200 prompts. The precise checkpoint estimates occupy the observed 0.0850.085–0.1390.139 range despite substantial differences in expert count, top-k, shared paths, and gate conventions. The shaded range is a description of these checkpoints, not a universal confidence band. Figure 3: Exposure rA/resr_A/res against directly aggregated backbone share rblk/resr_blk/res on PubMedQA (circles) and GSM8K (squares). Each model’s domain points are connected. The plot uses meanℓ(∑∥y∥/∑∥res∥)mean_ (Σ y /Σ ) rather than deriving the horizontal coordinate from exposure, so the association is descriptive rather than an algebraic restatement of Eq. 4. Qwen3 is least exposed and OLMoE most exposed; the causal test of the backbone lever is Section 5. 5 Results I: the causal intervention From observation to manipulation. The wedge is an identity, not a mechanism. We therefore scale the always-on shared output directly in two confirmatory checkpoints, DeepSeek-V2-Lite and Gravity. At each block we multiply that output by α∈1,0.75,0.5,0.25,0α∈\1,0.75,0.5,0.25,0\, propagate the full forward pass, and remeasure on the fixed base trajectory. The router is not directly modified; it responds only to the resulting downstream drift. The α=1α=1 no-op reproduces the baseline exactly. Exposure here is measured post-block, and we interpret the graded response before treating full removal as in-distribution behavior. Always-on mass contains routing mismatch through coherence. Removing backbone mass raises post-block exposure monotonically: from 0.0240.024 to 0.0400.040 on DeepSeek-V2-Lite and from 0.0400.040 to 0.0770.077 on Gravity. A frozen-numerator null, which changes only the denominator, is nearly flat. Instead, ∥A∥ A rises to 1.57×1.57× and 1.69×1.69× its baseline as routing coherence falls. Thus the causal response is not pure denominator dilution. The always-on path contains mismatch mainly by keeping the two passes’ routes coherent. Preserved mass contributes beyond content. A valid control must change the shared path in common mode across teacher and student; independent perturbations create an artificial between-pass mismatch. At matched input drift, replacing shared content while preserving its mass produces less routing decoherence than a matched-norm off-manifold perturbation, which in turn produces less than removing the mass (Figure 4). A descriptive layer-point bootstrap preserves the three-way ordering in 96%96\% and 98%98\% of resamples; because layers recur across doses, this is a stability diagnostic rather than a prompt-level interval. The content-preserving arm is near the no-op on DeepSeek-V2-Lite but reacts on Gravity, so we do not claim complete content irrelevance. The supported claim is narrower: preserved mass contributes beyond content, and its dominant measured effect is routing coherence. Scope and replication. Full removal substantially changes routing, so the mechanistic claim rests on the graded trajectory and common-mode ladder, not on treating the knockout as a natural operating point. Repeating the same n=100n=100 protocol on GSM8K reproduces the result on both checkpoints: exposure rises against a nearly flat null (72%72\% on DeepSeek-V2-Lite and 106%106\% on Gravity), and the same control ordering holds. Detailed trajectories and routing-health measurements are in Appendix D. Causal A/BA/B patches connect the instrument to behavior. In a preregistered test on OLMoE, DeepSeek-V2-Lite, and Qwen3-30B (n=200n=200), we inject the recorded student-anchored term A at every MoE block and compare it with matched-norm random directions and the natural context effect. Injected A flips 3.13.1–4.0%4.0\% of greedy tokens, 4040–45%45\% of the context reference, while matched-norm noise reproduces 5858–74%74\% of that flip rate. The routing term therefore has a real but bounded effect that is largely generic to perturbation magnitude. A GSM8K replication gives the same qualitative result. The B-side arm closes the attribution: at its own norm, the content term’s premium over random direction is 5.65.6–80×80×, against 1.541.54–2.05×2.05× for A on the same checkpoints and domain. The natural context effect is carried much more strongly by B’s direction. Behavioral effect does not order cleanly by exposure across three checkpoints, so exposure is a screening coordinate, not a behavioral threshold. Finally, the exact decomposition permits a symmetric teacher anchor; its norms are similar but its direction was not patched, so the causal claim is specific to the student-anchored A defined in Eq. 1. Figure 4: Backbone rescaling on the two confirmatory checkpoints: DeepSeek-V2-Lite (top) and Gravity (bottom). Left: post-block exposure against the frozen-numerator null. Center: routing-coherence response under the mass-removal, matched-norm, and content-replacement controls at matched input drift. Right: top-k Jaccard and gate TV across the dose trajectory. Full removal changes routing substantially; the claim rests on the graded response and control ordering. 6 Related work Sparse MoE layers route each token to a small subset of experts (Shazeer et al. 2017). This discrete assignment motivates work on load balance and router stability (Lepikhin et al. 2021; Fedus et al. 2022; Dai et al. 2022). Later studies track router drift during continual tuning or suppress it through consistency, replay, and compatible expert selection (Xie et al. 2026; Hou et al. 2026; Guo et al. 2026; Liu et al. 2026; Omi et al. 2025). In reinforcement learning, rollout-to-update routing mismatch can destabilize training (Ma et al. 2025; Dong et al. 2026). Those studies address assignment stability, sample efficiency, forgetting, or trajectory-level optimization. We address a different estimand: how much a realized same-weight conditioning mismatch propagates into the residual stream and output. Containment here does not imply that trajectory-level router instability is harmless. Shared-expert architectures send every token through always-on computation while routed experts specialize (Dai et al. 2024; DeepSeek-AI 2024). Sparse upcycling similarly anchors experts to a dense checkpoint (Komatsuzaki et al. 2023; He et al. 2024; Zhang et al. 2024). These designs suggest that common computation can stabilize behavior, but do not measure the residual exposure of an actual routing perturbation. Our backbone intervention supplies that measurement and separates a small denominator-only effect from the larger coherence response. Measurement work interprets experts, separates routing inputs from ignored content, and evaluates counterfactual route changes (Herbst et al. 2026; Yang et al. 2025; Ye et al. 2026; Yoon et al. 2026). The last of these asks how much an optimized alternative route can move fragile tokens; our quantity is the realized conditioning-induced route difference. The two are complementary: a large effect available somewhere in route space does not imply that the route change produced by a particular training example has that effect. Same-weight self-distillation uses a model under additional conditioning as its own teacher (Yang et al. 2024; Shenfeld et al. 2026). That is exactly the two-pass construction measured here. Distillation from a separate checkpoint (Agarwal et al. 2024; Qwen Team 2025; GLM Team 2025) introduces an additional weight-induced mismatch, and self-teacher reinforcement learning (Hübotter et al. 2026) introduces a trajectory; both lie outside our single-step scope. We also do not expect the observed range to transfer unchanged to expert-choice, metadata-routed, or independently merged experts (Zhou et al. 2022; Gururangan et al. 2022; Li et al. 2022; Wortsman et al. 2022; Ilharco et al. 2023; Sukhbaatar et al. 2024). The merged-expert probe below tests one such boundary. 7 Discussion and limitations From motion to influence. Routing motion and routing influence are different quantities. Gate overlap describes whether routes changed; the decomposition measures the block-output perturbation those changes produced; exposure measures how much reached the residual stream. The screen links exposure to backbone share, the rescaling intervention shows that always-on mass also preserves route coherence, and the patches show that the routing term can change outputs but is smaller and less direction-specific than the dense-like content term. The result is containment, not harmlessness. What the observed range means. Normalizing by the routed block output reduces sensitivity to gate convention and raw expert-output scale, but it does not derive the observed width. The two scale probes behave differently, and each changes several capacity or training axes at once. Together with the merged-expert boundary, they rule out a universal range without localizing a scaling law. Exposure remains within or below the primary observed range in both scale probes, but that observation is not a behavioral threshold. A boundary that fails in the informative direction. A merged-expert checkpoint tests a design outside the jointly trained family. Beyonder-4x7B-v3 is a mergekit-built MoE (Goddard et al. 2024; Labonne 2024) whose router was not jointly trained with its four experts. Its routing factor falls to 0.0140.014 on PubMedQA and 0.0110.011 on GSM8K, far below the primary range, while exposure is 0.0050.005–0.0060.006. We predicted that the range would break but expected an upward break because the design lacks an always-on backbone. The direction was wrong. Despite ordinary gate movement, reallocating mass changes the block little, consistent with high similarity among experts derived from the same dense base. Backbone mass can contain a perturbation after it forms; sufficiently similar expert outputs can suppress it at the source. Use as a diagnostic, not a threshold. A checkpoint-level audit should follow the evidence. First verify that the decomposition reconstructs the routed-output difference. Then measure exposure rather than inferring influence from router overlap. If the downstream decision matters, patch the recorded term or run another behavioral intervention. Exposure within the observed range is evidence of residual containment in this regime, not proof that behavior is unchanged. The larger and more direction-specific B term also means that routing-only mitigation should not be assumed to address the dominant context-induced change. Limits of the evidence. The seven open-weight checkpoints form an opportunistic, family-correlated sample rather than a factorial architecture sweep. Backbone rescaling is confirmatory only for DeepSeek-V2-Lite and Gravity, whose always-on branches provide a clean scalar handle. The merged boundary is one checkpoint. The output patches cover three checkpoints, and exposure does not order their behavioral effects. All-layer patches can also compound recorded local terms differently from the natural context trajectory, so the matched-norm A/BA/B comparison establishes relative direction specificity rather than an exact partition of the final output change. The reported behaviors are next-token quantities, not task-level generation quality. The causal patch uses the student anchor from Eq. 1; the alternate anchor has similar norm but a different vector. The instrument currently requires reconstruction-valid, full-precision expert execution. Most importantly, the claims concern one same-weight conditioning mismatch. They do not cover accumulated optimization, separately parameterized teachers, reinforcement-learning trajectories, or forgetting. 8 Conclusion Same weights do not imply the same computation in an MoE: conditioning can change the route. But route change is not the same as downstream influence. Our decomposition follows the mismatch from the gates, through the routed block, into the residual stream. Across the tested checkpoints, backbone share orders exposure; causal rescaling shows that always-on mass contains mismatch chiefly by preserving routing coherence; and output patches reveal a bounded, mostly magnitude-generic routing effect beside a more direction-specific content effect. The narrow block-level range is not universal; scale and merged-expert designs break it. The durable practical lesson is narrower: use router divergence to locate change, residual exposure to quantify propagation, and behavioral intervention to establish consequence. Reproducibility statement Analysis scripts and stochastic controls use fixed seeds; the frozen analysis-ready captures support deterministic reduction. Fresh model acquisition is a scientific replication, not a guaranteed bitwise replay. Code and captures underlying the reported figures and tables are available at https://github.com/CedricCaruzzo/routing-divergence-moe. Public model release names are listed in Appendix B. Author Contributions Contributor roles are described using the CRediT taxonomy.22 2 Contributor Roles Taxonomy (CRediT), an ANSI/NISO standard: https://credit.niso.org. C.C.: Conceptualization, Methodology, Software, Formal analysis, Investigation, Validation, Visualization, Writing – original draft. D.Y.: Funding acquisition, Resources. T.S.K.: Funding acquisition, Resources. References Abdin et al. [2024] Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024. URL https://arxiv.org/abs/2404.14219. Agarwal et al. [2024] Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2306.13649. Austin et al. [2021] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. URL https://arxiv.org/abs/2108.07732. Cobbe et al. [2021] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. URL https://arxiv.org/abs/2110.14168. Dai et al. [2022] Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, and Furu Wei. Stablemoe: Stable routing strategy for mixture of experts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022. URL https://arxiv.org/abs/2204.08396. Dai et al. [2024] Damai Dai, Chengqi Deng, Chenggang Zhao, R.X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, et al. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL https://arxiv.org/abs/2401.06066. DeepSeek-AI [2024] DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024. URL https://arxiv.org/abs/2405.04434. Dong et al. [2026] Daize Dong, Junlin Chen, Haolong Jia, Jiang Liu, Jiawei Wu, Huanwei Di, Jialian Wu, Zhengzhong Liu, Zicheng Liu, Emad Barsoum, Dimitris N. Metaxas, and Hongyi Wang. PR2: Predictive routing replay for moe-based llm reinforcement learning. arXiv preprint arXiv:2606.00395, 2026. URL https://arxiv.org/abs/2606.00395. Fedus et al. [2022] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1–39, 2022. URL https://arxiv.org/abs/2101.03961. GLM Team [2025] GLM Team. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471, 2025. URL https://arxiv.org/abs/2508.06471. Goddard et al. [2024] Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee’s MergeKit: A toolkit for merging large language models. arXiv preprint arXiv:2403.13257, 2024. URL https://arxiv.org/abs/2403.13257. Guo et al. [2026] Zirui Guo, Quan Cheng, Da-Wei Zhou, and Lijun Zhang. Stable routing for mixture-of-experts in class-incremental learning. arXiv preprint arXiv:2605.17571, 2026. URL https://arxiv.org/abs/2605.17571. Gururangan et al. [2022] Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A. Smith, and Luke Zettlemoyer. Demix layers: Disentangling domains for modular language modeling. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 5557–5576, 2022. URL https://aclanthology.org/2022.naacl-main.407/. He et al. [2024] Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Korthikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, and Bryan Catanzaro. Upcycling large language models into mixture of experts. arXiv preprint arXiv:2410.07524, 2024. URL https://arxiv.org/abs/2410.07524. Herbst et al. [2026] Jeremy Herbst, Stefan Wermter, and Jae Hee Lee. The expert strikes back: Interpreting mixture-of-experts language models at expert level. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026. URL https://arxiv.org/abs/2604.02178. Hou et al. [2026] Zhiyan Hou, Haiyun Guo, Haokai Ma, Yandu Sun, Yonghui Yang, and Jinqiao Wang. Pass-moe: Mitigating misaligned co-drift among router and experts via pathway activation subspaces for continual learning. arXiv preprint arXiv:2601.13020, 2026. URL https://arxiv.org/abs/2601.13020. Hübotter et al. [2026] Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. Reinforcement learning via self-distillation. arXiv preprint arXiv:2601.20802, 2026. URL https://arxiv.org/abs/2601.20802. Ilharco et al. [2023] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2212.04089. Jiang et al. [2024] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. URL https://arxiv.org/abs/2401.04088. Jin et al. [2019] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567–2577, 2019. doi: 10.18653/v1/D19-1259. URL https://aclanthology.org/D19-1259/. Komatsuzaki et al. [2023] Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2212.05055. Labonne [2024] Maxime Labonne. Beyonder-4x7b-v3. Hugging Face model card, 2024. URL https://huggingface.co/mlabonne/Beyonder-4x7B-v3. Lepikhin et al. [2021] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. In International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/2006.16668. Li et al. [2022] Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, and Luke Zettlemoyer. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306, 2022. URL https://arxiv.org/abs/2208.03306. Liu et al. [2026] Yang Liu, Toan Nguyen, and Flora D. Salim. Cp-moe: Consistency-preserving mixture-of-experts for continual learning. arXiv preprint arXiv:2605.20247, 2026. URL https://arxiv.org/abs/2605.20247. Ma et al. [2025] Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, and Fuli Luo. Stabilizing moe reinforcement learning by aligning training and inference routers. arXiv preprint arXiv:2510.11370, 2025. URL https://arxiv.org/abs/2510.11370. Muennighoff et al. [2024] Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, et al. Olmoe: Open mixture-of-experts language models. arXiv preprint arXiv:2409.02060, 2024. URL https://arxiv.org/abs/2409.02060. Omi et al. [2025] Nabil Omi, Siddhartha Sen, and Ali Farhadi. Load balancing mixture of experts with similarity preserving routers. arXiv preprint arXiv:2506.14038, 2025. URL https://arxiv.org/abs/2506.14038. Qwen Team [2024] Qwen Team. Qwen1.5-moe: Matching 7b model performance with 1/3 activated parameters. Qwen blog, 2024. URL https://qwenlm.github.io/blog/qwen-moe/. Accessed 2026-07-22. Qwen Team [2025] Qwen Team. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. URL https://arxiv.org/abs/2505.09388. Shazeer et al. [2017] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations (ICLR), 2017. URL https://arxiv.org/abs/1701.06538. Shenfeld et al. [2026] Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning. arXiv preprint arXiv:2601.19897, 2026. URL https://arxiv.org/abs/2601.19897. Sukhbaatar et al. [2024] Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, and Xian Li. Branch-train-mix: Mixing expert llms into a mixture-of-experts llm. arXiv preprint arXiv:2403.07816, 2024. URL https://arxiv.org/abs/2403.07816. Trillion Labs [2026] Trillion Labs. Gravity-16b-a3b-preview. Hugging Face model card, 2026. URL https://huggingface.co/trillionlabs/Gravity-16B-A3B-Preview. Wortsman et al. [2022] Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the 39th International Conference on Machine Learning (ICML), 2022. URL https://proceedings.mlr.press/v162/wortsman22a.html. Xie et al. [2026] Zhen-Hao Xie, Jun-Tao Tang, Yu-Cheng Shi, Han-Jia Ye, De-Chuan Zhan, and Da-Wei Zhou. Same: Stabilized mixture-of-experts for multimodal continual instruction tuning. arXiv preprint arXiv:2602.01990, 2026. URL https://arxiv.org/abs/2602.01990. Yang et al. [2025] Xingyi Yang, Constantin Venhoff, Ashkan Khakzar, Christian Schroeder de Witt, Puneet K. Dokania, Adel Bibi, and Philip Torr. Mixture of experts made intrinsically interpretable. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025. URL https://arxiv.org/abs/2503.07639. Yang et al. [2024] Zhaorui Yang, Qian Liu, Tianyu Pang, Han Wang, Haozhe Feng, Minfeng Zhu, and Wei Chen. Self-distillation bridges distribution gap in language model fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL https://arxiv.org/abs/2402.13669. Ye et al. [2026] Charles Ye, Bo Yuan, and Lee Sharkey. Polysemantic experts, monosemantic paths: Routing as control in moes. arXiv preprint arXiv:2604.17837, 2026. URL https://arxiv.org/abs/2604.17837. Yoon et al. [2026] Youngsik Yoon, Siwei Wang, Wei Chen, and Jungseul Ok. When are experts misrouted? counterfactual routing analysis in mixture-of-experts language models. arXiv preprint arXiv:2605.07260, 2026. URL https://arxiv.org/abs/2605.07260. Zhang et al. [2024] Qizhen Zhang, Nikolas Gritsch, Dwaraknath Gnaneshwar, Simon Guo, David Cairuz, Bharat Venkitesh, Jakob Foerster, Phil Blunsom, Sebastian Ruder, Ahmet Ustun, and Acyr Locatelli. Bam! just like that: Simple and efficient parameter upcycling for mixture of experts. arXiv preprint arXiv:2408.08274, 2024. URL https://arxiv.org/abs/2408.08274. Zhou et al. [2022] Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M. Dai, Zhifeng Chen, Quoc V. Le, and James Laudon. Mixture-of-experts with expert choice routing. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2202.09368. Appendix A Derivation and anchor choice Starting from Δrouted=∑jgjTfj(hT)−∑jgjSfj(hS) _routed= _jg^T_jf_j(h^T)- _jg^S_jf_j(h^S), add and subtract ∑jgjTfj(hS) _jg^T_jf_j(h^S): Δrouted=∑j(gjT−gjS)fj(hS)⏟A+∑jgjT(fj(hT)−fj(hS))⏟B. _routed= _j(g^T_j-g^S_j)f_j(h^S)_A+ _jg^T_j (f_j(h^T)-f_j(h^S) )_B. The split is not unique: adding and subtracting ∑jgjSfj(hT) _jg^S_jf_j(h^T) instead gives A′=∑j(gjT−gjS)fj(hT)A = _j(g^T_j-g^S_j)f_j(h^T) and B′=∑jgjS(fj(hT)−fj(hS))B = _jg^S_j(f_j(h^T)-f_j(h^S)), evaluating the gate difference against the teacher hidden state. We use the hSh^S anchor throughout (the routing difference applied to the state the student produced). The two anchors agree exactly when fj(hT)=fj(hS)f_j(h^T)=f_j(h^S); their difference, A−A′=−∑j(gjT−gjS)(fj(hT)−fj(hS))A-A =- _j(g^T_j-g^S_j) (f_j(h^T)-f_j(h^S) ), is the gate-difference-weighted content shift. An instrumented re-run of the PubMedQA screen quantifies it: the alternate-anchor magnitude agrees with the reported one to within a few percent per layer (median per-layer ∥A′∥/∥A∥ A / A of 0.960.96 to 0.990.99 across the seven models), and the routing factor recomputed under A′A moves by at most 4.6%4.6\% per model (range 0.0840.084–0.1270.127 against 0.0850.085–0.1330.133, same ordering), so the norm-based quantities we report are anchor-robust. The difference tensor is not small (median per-layer ∥A−A′∥/∥A∥ A-A / A of 0.360.36 to 0.650.65): the anchors agree in magnitude, not as vectors, so a directional analysis of A would need to fix the convention. A second-domain check (DeepSeek-V2-Lite, GSM8K) reproduces the picture (medians 1.001.00 and 0.410.41), and a held-out scale probe remains within the same norm-sensitivity range. (Both splits are exact identities, so reconstruction closure does not distinguish them.) All reported quantities use the hSh^S anchor consistently. A uses the raw realized gates g, so the reconstruction is exact only when the applied gate convention is reproduced. Quantized expert execution that fails this gate is excluded. Appendix B Uncertainty and boundary checkpoints The public release names for the primary screen are Qwen3-30B-A3B, Mixtral-8x7B-Instruct-v0.1, OLMoE-1B-7B-0924-Instruct, DeepSeek-V2-Lite-Chat, Gravity-16B-A3B-Preview, Qwen1.5-MoE-A2.7B-Chat, and Phi-3.5-MoE-instruct. Boundary probes use Qwen3-235B-A22B, DeepSeek-V2-Chat, and Beyonder-4x7B-v3. Table 2 gives the prompt-level 95%95\% bootstrap CIs behind the Figure 2 whiskers and the Table 1 footnote. We resample the n=200n=200 probe prompts (20002000 resamples), recompute the per-layer-first ratio on each resample, and take the 2.52.5 and 97.597.5 percentiles; the full-data reduction reproduces the committed headline exactly. The per-model intervals pin each checkpoint tightly relative to the observed two-domain range. They quantify sampling uncertainty on each checkpoint, not uncertainty over a population of architectures. Table 2: Prompt-level 95%95\% bootstrap CIs (resampling the n=200n=200 probe prompts, 20002000 resamples) for the two headline ratios, both domains. These intervals quantify prompt-sampling uncertainty for each checkpoint rather than variation across checkpoints. PubMedQA GSM8K Model rA/blkr_A/blk CI rA/resr_A/res CI rA/blkr_A/blk CI rA/resr_A/res CI Qwen3-30B-A3B [0.131, 0.135] [0.019, 0.020] [0.137, 0.141] [0.021, 0.021] Mixtral-8x7B [0.091, 0.096] [0.021, 0.023] [0.108, 0.111] [0.028, 0.029] OLMoE-1B-7B [0.121, 0.127] [0.055, 0.058] [0.120, 0.125] [0.060, 0.062] DeepSeek-V2-Lite [0.088, 0.092] [0.026, 0.027] [0.100, 0.104] [0.032, 0.033] Gravity-16B-A3B [0.097, 0.099] [0.044, 0.045] [0.107, 0.109] [0.052, 0.053] Qwen1.5-MoE-A2.7B [0.084, 0.087] [0.035, 0.037] [0.100, 0.104] [0.047, 0.049] Phi-3.5-MoE [0.104, 0.109] [0.022, 0.024] [0.108, 0.112] [0.026, 0.027] Scale probe (outside the primary range): Qwen3-235B-A22B [0.186, 0.191] [0.029, 0.030] [0.197, 0.203] [0.030, 0.031] Table 3: Boundary checkpoints, shown as PubMedQA/GSM8K. Family pairs are descriptive scale probes; the merged checkpoint tests transfer outside jointly trained expert families. Exposure uses the pre-block screen convention. Checkpoint rA/blkr_A/blk rA/resr_A/res Qwen3-30B-A3B 0.133/0.139 0.019/0.021 Qwen3-235B-A22B 0.188/0.200 0.030/0.030 DeepSeek-V2-Lite 0.090/0.102 0.027/0.033 DeepSeek-V2 (236B) 0.114/0.139 0.012/0.019 Beyonder-4x7B-v3 0.014/0.011 0.006/0.005 Appendix C Robustness and exclusions Aggregation order. Averaging tokenwise ratios overweights tokens with small residual norms. Relative to the per-layer-first estimator, it inflates exposure by 2020–38%38\% on OLMoE and DeepSeek-V2-Lite across both domains. It also widens the seven-checkpoint ranges to 0.0890.089–0.1500.150 for the routing factor and 0.0200.020–0.0730.073 for exposure, without changing the central ordering. We therefore use the preregistered per-layer-first reduction throughout. Reconstruction validity. All seven primary checkpoints pass the identity-closure gate. A quantized checkpoint produced reconstruction error 1.061.06 and is excluded from every result that uses A or B. This exclusion defines the operating envelope of the instrument, not additional evidence for the containment claim. Prompt format and held-out domain. Removing the chat template moved the routing factor by 0.0070.007 on Mixtral and 0.0170.017 on Qwen3-30B, both with valid reconstruction. This bounds format sensitivity on two checkpoints but does not prove format invariance. A held-out MBPP code screen [Austin et al. 2021] did not define the primary range: six of seven checkpoints remained inside it, while OLMoE was marginally below (0.0820.082 against the 0.0850.085 floor) with reconstruction error 0.0200.020 at the gate. We draw no further inference from that third-domain screen. Appendix D Intervention details and stopped extension Tables 4 and 5 report the PubMedQA dose trajectory and matched-drift controls for the two confirmatory checkpoints. Exposure is post-block, matching the intervention target; the screen’s pre-block convention is not mixed into these tables. Table 4: PubMedQA backbone-rescaling dose-response. The null freezes the routing numerator; nA is ∥A(α)∥/∥A(1)∥ A(α) / A(1) ; Jaccard is routing overlap with α=1α=1. DeepSeek-V2-Lite uses n=200n=200 probe pairs and Gravity n=100n=100. Model α rA/resr_A/res post null nA Jaccard DeepSeek-V2-Lite 1.00 0.0245 0.0245 1.000 1.000 0.75 0.0278 0.0247 1.095 0.813 0.50 0.0315 0.0249 1.209 0.690 0.25 0.0375 0.0250 1.407 0.578 0.00 0.0395 0.0250 1.567 0.438 Gravity 1.00 0.0403 0.0403 1.000 1.000 0.75 0.0451 0.0406 1.068 0.878 0.50 0.0522 0.0408 1.174 0.789 0.25 0.0610 0.0408 1.320 0.702 0.00 0.0769 0.0406 1.690 0.595 Table 5: PubMedQA matched-drift control ladder. Values are mean routing-coherence response nA by input-drift bin, pooled over intervening doses. At high drift, preserving shared-path mass while replacing content produces a smaller response than matched-norm noise or mass removal. Model arm [0.00,0.15) [0.15,0.30) [0.30,0.50) [0.50,0.90) DeepSeek-V2-Lite content replaced 1.018 0.938 0.863 0.958 matched noise 1.043 1.109 1.124 1.387 mass removed 1.059 1.129 1.226 1.546 Gravity content replaced 0.990 1.034 1.099 1.233 matched noise 0.992 1.060 1.138 1.384 mass removed 0.994 1.069 1.186 1.480 Second domain. The same n=100n=100 protocol on GSM8K reproduces the monotone response against a nearly flat null and the high-drift control ordering on both confirmatory checkpoints. Table 6: GSM8K dose-response and the matched-drift ladder’s highest bin. The ladder entries are content replacement / matched noise / mass removal. Model α rA/resr_A/res post null nA Jaccard ladder [0.50,0.90) DeepSeek-V2-Lite 1.00 0.0303 0.0303 1.000 1.000 0.75 0.0364 0.0307 1.153 0.803 0.50 0.0412 0.0309 1.307 0.678 1.11 / 1.52 / 1.62 0.25 0.0502 0.0310 1.532 0.566 0.00 0.0521 0.0310 1.715 0.420 Gravity 1.00 0.0495 0.0495 1.000 1.000 0.75 0.0571 0.0501 1.089 0.855 0.50 0.0668 0.0505 1.207 0.755 1.24 / 1.39 / 1.44 0.25 0.0805 0.0507 1.408 0.657 0.00 0.1020 0.0506 1.874 0.537 Stopped descriptive extension. Moonlight-16B-A3B was preregistered as a third always-on-branch intervention with an all-dose monotonicity gate and a parking rule. Its screened routing factor was below the primary range on both domains. Exposure then turned over at full removal on both domains, so the stop gate fired. Accordingly, Moonlight is excluded from the confirmatory mechanism claim. We report the stopped trajectory only to document that decision; no conclusion rests on its healthy-dose behavior. Table 7: Moonlight stopped extension. The screen factor is pre-block (n=200n=200); the dose trajectory is post-block exposure (n=100n=100). Domain screen factor α=1α=1 0.75 0.50 0.25 0 PubMedQA 0.052 0.0241 0.0297 0.0443 0.0819 0.0520 GSM8K 0.059 0.0280 0.0356 0.0583 0.1156 0.0850 Appendix E Wedge consistency The wedge identity (4) is exact per layer per token by construction. As an internal check we recompute the per-layer-first rA/resr_A/res as the mean over layers of the per-layer ratio and compare it to the stored headline value; the two agree to within 10−1610^-16 across all models and both domains. Figure 3 instead plots backbone share computed directly from the same stored sums as meanℓ(∑∥y∥/∑∥res∥)mean_ (Σ y /Σ ). Because this is a different aggregate reduction, its product with the plotted routing factor need not reproduce the aggregate exposure exactly. Identity-implied shares are 33–17%17\% below the direct shares, but the model ordering is unchanged. Figure 3 is therefore a descriptive cross-check; the causal evidence for the backbone lever comes from the rescaling intervention.