Paper deep dive

Uncovering Latent Reasoning Strategies in Language Models

Awni Altabaa, John Lafferty

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 104

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/21/2026, 5:26:06 AM

Summary

This paper addresses the challenge of uncovering implicit reasoning strategies in pretrained language models by decomposing their response distribution into a structured, strategy-conditioned representation. The authors propose a router-generator factorization where a router maps inputs to latent strategies and a generator produces responses conditioned on these strategies. To prevent posterior collapse—a common issue where the latent variable becomes inert because the base model already captures the response distribution—they introduce a variational objective that measures fractional information gain relative to the base model's response loss. This objective concentrates reconstruction pressure on tokens with high base model surprisal, encouraging the latent variable to encode strategy-relevant variation. The method is evaluated on a benchmark of multi-strategy algorithmic tasks, demonstrating that it successfully recovers latent codes aligned with distinct reference strategies while preserving the base model's response distribution.

Entities (8)

Awni Altabaa · author · 99%John Lafferty · author · 99%Router-Generator Factorization · method · 95%Posterior Collapse · problem · 92%Transformer · architecture · 90%Variational Objective · method · 90%Base Model Surprisal · metric · 88%Algorithmic Tasks · dataset · 85%

Relation Signals (6)

Awni Altabaa → affiliatedwith → Yale University

confidence 99% · Awni Altabaa Yale University

John Lafferty → affiliatedwith → Yale University

confidence 99% · John Lafferty Yale University

Variational Objective → uses → Base Model Surprisal

confidence 92% · concentrates reconstruction pressure on tokens with high base model surprisal

Router-Generator Factorization → addresses → Posterior Collapse

confidence 90% · To address this, we propose a variational objective... To address this problem, we propose a training objective... The ELBO failure above shows that marginal fidelity is not enough... We address this selection problem with a model-directed reconstruction term.

Router-Generator Factorization → implementedusing → Transformer

confidence 90% · Autoregressive router-generator Transformer architecture

Router-Generator Factorization → evaluatedon → Algorithmic Tasks

confidence 85% · We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:A language model $p_\theta(y \mid x)$ trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the model's response distribution. We study the problem of decomposing the response distribution of a given pretrained language model into a structured, strategy-conditioned representation. Specifically, we learn a latent-variable factorization $p_\theta(y \mid x) \leadsto (r_\phi(z \mid x), g_\phi(y \mid x,z))$, where a router $r$ maps each input to a distribution over latent strategies $z$ and a generator $g$ produces the response conditioned on that strategy. A key challenge is that the generator, initialized from the base model, already represents $p_\theta(y \mid x)$ without using $z$. Standard variational inference therefore gives the model no incentive to route information through $z$ and can yield a severe form of posterior collapse. To address this, we propose a variational objective that measures fractional information gain relative to the base model's response loss and concentrates reconstruction pressure on tokens with high base model surprisal, encouraging $z$ to encode strategy-relevant response variation. We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies while preserving the base model's response distribution.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

102,572 characters extracted from source content.

Expand or collapse full text

Uncovering Latent Reasoning Strategies in Language Models Awni Altabaa Yale University awni.altabaa@yale.edu John Lafferty Yale University john.lafferty@yale.edu Abstract A language modelp θ (y|x)trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the model’s response distribution. We study the problem of decomposing the response distribution of a given pretrained language model into a structured, strategy-conditioned representation. Specifically, we learn a latent-variable fac- torizationp θ (y|x)⇝(r φ (z|x), g φ (y|x,z)), where a routerrmaps each input to a distribution over latent strategieszand a generatorgproduces the response conditioned on that strategy. A key challenge is that the generator, initialized from the base model, already representsp θ (y|x)without usingz. Standard variational inference therefore gives the model no incentive to route information throughz and can yield a severe form of posterior collapse. To address this, we propose a variational objective that measures fractional information gain relative to the base model’s response loss and concentrates reconstruction pressure on tokens with high base model surprisal, encouragingzto encode strategy-relevant response variation. We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies while preserving the base model’s response distribution. Code:github.com/Awni00/latent-strategies-in-lms 1 Introduction Language models trained on reasoning tasks can solve the same problem using multiple distinct strategies. For example, a model asked to prove a theorem may produce a proof by contradiction, induction, or construction, while a model asked to solve a programming problem may formulate it as dynamic programming, graph search, or a greedy algorithm. These strategies can recur across many problems, but are represented only implicitly. The model’s response distributionp θ (y|x)is an entangled mixture over distinct strategies, with no separate variable for identifying or controlling the strategy used during generation. We study how to uncover these strategies by decomposing the response distribution of a pretrained language model into a structured, strategy-conditioned representation. Concretely, we seek arouter- generator factorization(r φ (z|x), g φ (y|x,z))of the base modelp θ (y|x), where a router maps each input to a distribution over a latent strategy variablez, and a strategy-conditioned generator produces responses given the input and the sampled strategy. The goal is to find a factorization that preserves the observable response distribution, so that R r φ (z|x)g φ (y|x,z)dz=:p φ (y|x)≈p θ (y|x), while makingzinformative and strategy-aligned: varyingzshould change which strategy the generator uses, and the same latent value should carry the same strategy meaning across related inputs. This problem is distinct from standard latent variable generative modeling [KW13; SLY15]. In VAE-style methods, the latent variable model is trained from data, and the target distribution is not already represented at initialization. Here, the pretrained generator already capturesp θ (y|x)without any latent variable, creating a pathological form of posterior collapse. A factorization that ignores Preprint. arXiv:2607.17674v1 [cs.LG] 20 Jul 2026 p θ (y|x) Output SpaceY Basen= 2: 4 even. Supposek 2 . . . Induct onn. Assume (n−2) 2 . . . Supposenodd,n= 2k+ 1. Then. . . We show:nodd⇒n 2 odd. . . x:prove that ifnis even, thenn 2 is even misaligned latentz Output SpaceY strategy-aligned latentz Output SpaceY Proof by Induction Proof by Contradiction Proof by Contrapositive Algebraic Manipulation Figure 1:Latent strategy structure in output space.A pretrained LM’s response distribution mixes strategies implicitly (left). A latent variable model initialized to match it leaveszunused (center). Training alignszwith reference strategies, disentangling them in latent space while preserving the base model’s response distribution (right). z –by reproducing the base model’s response distribution for everyz–is available at initialization and is globally optimal for objectives that only fit the observable response distribution. Standard variational training, including the conditional ELBO, is therefore subject to this failure mode [Bow+16; He+19; Fu+19; HJ16]: it can preserve distributional fidelity while leavingzinert, because it does not require zto explain strategy-level variation. To address this problem, we propose a training objective that replaces marginal-fit pressure with model-directed reconstruction pressure. The fixed base modelp θ serves as a reference for what is already predictable fromxalone. The reconstruction term is normalized by the base model’s response loss and weighted at the token level by base model surprisal, concentrating pressure on positions where the base model leaves multiple continuations plausible, which tend to be the branch points where strategy choices are made. Together, these mechanisms encouragezto encode strategy-relevant response variation that is not already determined byx, rather than reconstructing what the base model already captures. The broader promise of this problem is to expose latent capabilities that large language models already possess but do not reliably express under default sampling. By recovering a strategy- aligned factorization, we can turn implicit strategy structure into an explicit object of analysis and intervention. This has applications across model interpretability [Koh+20; Hub+24], controllability [Kes+19; L21], and exploration [Wan+23; Yao+23]. In particular, strategy-level interventions could support directed exploration across high-level solution strategies during reinforcement-learning post- training, beyond passive on-policy sampling from the base model’s response distribution [Yue+25; Tuy+25]. Such broader exploration could enable the discovery and improvement of previously under-sampled reasoning strategies and support their transfer or recombination across inputs. We leave these downstream applications to future work. We summarize the contributions of this paper as follows: 1.Problem formulation.We formulate the problem of learning a router-generator factorization of a pretrained language model and identify the core challenge that makes this setting different from standard latent variable modeling. 2.Methodology.We propose a variational objective with model-directed reconstruction pressure that uses the base model to normalize reconstruction in base model reference units and to direct token-level pressure toward positions of high base model residual surprisal, encouraging the latent to encode strategy-relevant variation. 3. Benchmark and empirical evaluation.We introduce a controlled benchmark of multi-strategy algorithmic tasks with ground-truth strategy annotations, withheld during training and used only for evaluation. We evaluate our method on both pretrained and randomly initialized language models, characterizing recovery conditions, ablation effects, and failure modes. 4.Theoretical analysis.We initiate a theoretical analysis of the strategy-structure recovery problem, studying the conditions under which a useful factorization can be identified. 2 Our experiments show that strategy information is decodable from the base model’s hidden states before any factorization is trained, indicating that the target structure exists. Standard variational objectives nevertheless fail to expose this information through the latent variable. Model-directed reconstruction pressure recovers latents that are aligned with reference solution strategies and seman- tically consistent across inputs. As a result, the latent variable transfers the same strategy from one problem to another, while preserving the base model’s response distribution. This opens possibilities for intervening and more efficiently searching over solutions that are guided by high-level strategies. 2 Problem Statement: Latent-Variable Factorization Letp θ (y|x)denote the response distribution of a pretrained base model, wherexis the input andy is the response sequence. We consider reasoning tasks for whichp θ (y|x)mixes multiple high-level strategies for producing valid responses. The goal of this paper is to expose such variation as an explicit latent variable while preserving the observable response distribution that the base model already represents. We formalize this goal as alatent-variable router-generator factorizationof p θ (y|x): the routerr(z|x)maps each input to a distribution over latent strategy valuesz, and the strategy-conditioned generatorg(y|x,z)produces responses conditioned on bothxandz. Problem Statement: Latent-variable factorization of a pretrained model Letp θ (y|x)be the fixed response distribution of a pretrained base model. A router-generator factorizationp θ (y|x)⇝p φ = (r φ ,g φ )introduces a latent variablez∈Z, a routerr φ (z|x), and a strategy-conditioned generatorg φ (y|x,z), inducing p φ (y|x) = Z Z r φ (z|x)g φ (y|x,z)dμ(z). The goal is to select a factorization that preserves the observable response distribution,p φ (y|x)≈ p θ (y|x), while makingzinformative aboutybeyondxand strategy-aligned: variation inzshould expose high-level, reusable strategy variation in responses rather than arbitrary predictive surface details. The objective cannot be merely to fit the latent marginalp φ (y|x)to the base model distribution. For expressive router-generator classes, many factorizations can induce nearly the same distribution overyafter integrating outz. One extreme ignoresz, withg φ (y|x,z)reproducing the base model distribution for allz. More generally, a marginal-preserving factorization may usezfor input- specific partitions, surface form, or other predictive details rather than reusable strategy structure. The problem is therefore not ordinary latent-variable modeling ofp θ (y|x), but selection among marginal-preserving factorizations for one whose latent variable is informative and strategy-aligned. The desired latent must first be informative:zshould encode information aboutybeyond what is already determined byx. This is nontrivial for autoregressive language models, where the base model already defines a stochastic response distribution without an explicit latent; makingzinformative therefore requires it to encode structured variation across responses. However, informativeness alone is too weak: a latent can help predictyby encoding surface form, input-specific partitions, or arbitrary details. We seek an informative latent that is strategy-aligned, where a strategy is a high-level, reusable attribute of a response trace: the broad procedure used to generate a solution, not incidental wording, formatting, or token-level variation. A strategy-aligned factorization is one in which changingzchanges the generator along this strategy-level axis, while marginalizing overz remains faithful to the base model. There are two distinct requirements: local strategy separation and stable cross-input latent semantics. Within-input strategy separation asks whether varyingzchanges the strategy used for a fixed input x. This rules out an inert latent, but not an input-local code whose interpretation changes across problems. The stronger requirement is cross-input semantic consistency: a latent value or region should retain its strategy meaning across related inputs. Only with this stability doeszbehave like a reusable strategy variable, rather than an input-specific response index. To measure whether a factorization succeeds, we use controlled experiments in which response traces are parseable and their reference strategy can be classified. These strategy labels are withheld during factorization training and used only to evaluate whether the learned latent achieves within-input 3 x 1 · x t x <lat> E(z) </lat> y 1 · y t y Adapted Transformer backboneφ h x 1 · h x t x h lat ⟨ h lat z h lat ⟩ h y 1 · h y t y · μ φ ,logσ 2 φ p(x 2 |·) · p(<lat>|·) r φ (z|x) p(</lat>|·)p(y 1 |x,z)p(y 2 |·) · p(EOS|·) samplez∼r φ (z|x), project asE(z) yattends toE(z) x 1:t x , y 1:t y Backboneξ h xy μ ξ ,logσ 2 ξ q ξ (z|x,y) Training-only posterior training:z∼q ξ (z|x,y)substituted atE(z) Figure 2:Autoregressive router-generator Transformer architecture.The adapted Transformerφ implements both the routerr φ (z|x), which reads the prompt and produces a latent distribution, and the strategy-conditioned generatorg φ (y|x,z), which attends toE(z)while generating the response. During variational training,q ξ (z|x,y)inferszfrom the full input-response pair. strategy separation and cross-input semantic consistency. The next section develops a procedure for learning a router-generator factorization whose latent variable is informative and strategy-aligned. 3 Learning Strategy-Aligned Latent-Variable Factorizations This section describes our methodology for learning a latent-variable router-generator factorization of a pretrained base model, with the goal of recovering an informative and strategy-aligned latent variable. We first describe the architecture and parameterization of the router-generator model, then identify the under-specification and latent collapse issues that arise when applying standard variational inference to this setting, and finally introduce a modified training objective that addresses these issues by using the base model to direct reconstruction pressure. Throughout,p θ (y|x)denotes the fixed pretrained base model whose response distribution we seek to factorize,φdenotes the adapted factorized router-generator model’s parameters, andξdenotes the training-time posterior model parameters. 3.1 A Transformer Architecture for Router-Generator Factorization We implement the latent-variable router-generator factorization byadaptinga pretrained Transformer language model into the three roles shown in Figure 2. The routerr φ (z|x)reads the input and produces a distribution over latent strategy values. Sampling from the router corresponds to selecting a strategy for generating a response. The strategy-conditioned generatorg φ (y|x,z)generates the response autoregressively while conditioning onxand an embeddingE(z)of the latent strategy. The training-time posteriorq ξ (z|x,y)reads the full input-response pair and produces a distribution over latent values, which is used to provide variational training signal. This parameterization requires only small architectural additions to the base model. Leth (L) φ (x) denote the final-layer representation produced by the adapted Transformer after reading the input prefixx. The router adds two learned projection heads on top of this representation, producing the mean and log-variance of a diagonal Gaussian over the continuous latent: r φ (z|x) =N W μ h (L) φ (x),diag exp(W σ h (L) φ (x)) . During autoregressive generation, a latent samplezis projected to an embeddingE φ (z)and inserted as a pseudo-token at the embedding layer, so response tokens can attend to it while the generator pre- dictsy. The training-time posteriorq ξ (z|x,y)is implemented similarly to the router, parameterizing a Gaussian distribution overzfrom the final-layer input representation after reading(x,y). The router and generator are a single autoregressive model with shared parametersφ. The posterior is a separate model with parametersξ, and is used only during variational training. Both models are initialized from the base parametersθand trained as light adaptations using LoRA-style low-rank updates [Hu+22], together with the small Gaussian heads and latent embedding projection. 4 Becausep θ (y|x)is already represented by the base model, lightweight adaptation lets the generator inherit the base model’s response distribution rather than relearn it. However, because the base gener- ator realizes the target response distribution, a factorization that ignoreszis available at initialization, biasing optimization toward it when the objective only measures marginal fit. The training objective must therefore choose a factorization wherezexplains meaningful strategy variation. 3.2 ELBO Under-Specification and Latent Collapse The standard negative conditional ELBO objective for this router-generator model is J cELBO (x,y;φ,ξ) =E z∼q ξ (·|x,y) [−logg φ (y|x,z)] + KL(q ξ (·|x,y)∥r φ (·|x)).(1) Letp φ (z|x,y) =r φ (z|x)g φ (y|x,z)/p φ (y|x)denote the posterior induced by the router-generator factorization. This objective satisfies the identity J cELBO (x,y;φ,ξ) =−logp φ (y|x) + KL(q ξ (·|x,y)∥p φ (·|x,y)).(2) Thusthestandardvariationalobjectiveaskstheinducedmarginalp φ (y|x)= R Z r φ (z|x)g φ (y|x,z)dμ(z)to match the observed response distribution, and asksq ξ to match the posterior of whichever factorization produced that marginal. It does not ask whetherzexplains variation inybeyondx, nor whether the resulting latent is strategy-aligned. This is an objective-level under-specification problem. If the generator class can represent the base model while ignoringz, then the factorization g φ (y|x,z) =p θ (y|x)for allz, q ξ (z|x,y) =r φ (z|x) 0.00.20.40.60.81.0 Training progress 0.00 0.25 0.50 0.75 1.00 Metric value Distributional Fidelity Strategy Alignment KL / initial KL Figure 3:ELBO training can preserve fi- delity while leaving the latent unused.Dis- tributional fidelity rises, posterior-router KL collapses relative to initialization, and strat- egy alignment remains low. preserves the base model’s response distribution and attains the objective’s optimum. The latent variable is then inert even thoughp φ (y|x) =p θ (y|x). In our setting, with expressive generator classes built from pretrained language models, thisz-ignoring factor- ization is not only available, but also close to initial- ization, biasing optimization toward it. In particular, the generator can implementg φ (y|x,z) =p θ (y|x) for allzby simply learning to ignorez. This is a latent collapse driven by both the objective’s under- specification and the proximity of the inert solution to initialization. 3.3 Model-Directed Reconstruction for Informative, Strategy-Aligned Latents The ELBO failure above shows that marginal fidelity is not enough: a factorized model can preserve p θ (y|x)while leavingzunused. We address this selection problem with amodel-directed reconstruc- tion term. The base modelp θ is used to concentrate reconstruction pressure on response tokens where the base model is uncertain, so the latent is encouraged to explain variation inythat the base model does not already explain throughxand the autoregressive prefix. The fixed base model is used first to define the response distribution being factorized. Training examples are drawn as x∼D X , y∼p θ (·|x), z∼q ξ (·|x,y). Thusyis sampled from the base model, whilezis inferred by the training-time posterior for that sampled input-response pair. No strategy annotations are available during training. The core methodological idea is to use the base modelp θ toadaptivelyrescale the reconstruction loss and direct reconstruction pressure towards tokens where the base model is uncertain. We instantiate this idea with token-level base model surprisal. Define b θ,t (x,y) =−logp θ (y t |x,y <t ), b θ (x,y) = 1 T y T y X t=1 b θ,t (x,y), as thez-free base model surprisal. This score is computed by the latent-free base model.b θ,t (x,y) measures how much uncertainty remains abouty t after conditioning onxandy <t , and therefore gives a local estimate of wherezcan explain variation beyond thez-free prefix model. 5 x y 1 y 2 y 3 y 4 y 5 y 6 b t y 1 y 2 y 3 y 4 y 5 y 6 w t Figure 4:Token-levelp θ surprisal weighting.The left panel shows a response path with branch points, where several next tokens remain plausible, and determined positions, where the next token is mostly fixed byxand earlier response tokens. The right panel shows z-free base model surprisal b θ,t (x,y)above each token and the corresponding token weight below it; higher-surprisal positions receive more reconstruction pressure ina θ,t (x,y). This token-level view matters because autoregressive responses quickly become partially self- predictable. After an early commitment, many later tokens may be determined byxandy <t , even if the commitment itself reflects a strategy choice. Our reconstruction term therefore replaces uniform token pressure withp θ -surprisal-directed pressure, assigning more weight to branch points and high-surprisal positions where the base model leaves multiple continuations plausible. The proposed objective is J θ (x,y;φ,ξ) =R θ (x,y;φ,ξ) +βKL(q ξ (·|x,y)∥r φ (·|x)), R θ (x,y;φ,ξ) = 1 c θ E z∼q ξ (·|x,y)   T y X t=1 a θ,t (x,y) −logg φ (y t |x,z,y <t )   , a θ,t (x,y) =α 1 T y + (1−α)w (γ) θ,t (x,y), c θ =E x∼D X , y∼p θ (·|x) [b θ (x,y)]. (3) Quantities with aθsubscript reflect values that depend on the base modelp θ . The global scalec θ expresses reconstruction in base model reference units, so the reconstruction term is normalized across different data distributions and remains comparable to the KL term even when the base model already has lowz-free loss. The interpolation parameterα∈[0,1]mixes uniform response-token pressure with surprisal-directed pressure. The token weightsw (γ) θ,t (x,y)are defined in terms of the base model surprisal as follows: ew (γ) θ,t (x,y) = b θ,t (x,y) γ P T y s=1 b θ,s (x,y) γ , w (γ) θ,t (x,y) = ew (γ) θ,t (x,y) κ (γ) θ (x,y) ,(4) whereκ (γ) θ (x,y) = 1 b θ (x,y) P T y s=1 ew (γ) θ,s (x,y)b θ,s (x,y)is a normalization factor that preserves the z-free base scale of the reconstruction term. The parameterγ≥0controls concentration of the token-level weighting. The reconstruction termR θ is designed so thatE x,y [R θ (x,y;φ,ξ)] = 1 wheng φ (y|x,z,y <t ) =p θ (y t |x,y <t ), and thus can be interpreted as a (weighted) fraction of the base model response loss that remains after conditioning on the latentz. The objective therefore balances the KL penalty against thefractionof the base model’s response loss explained byz, giving reconstruction improvement a fractional information-gain interpretation rather than measuring it in absolute log-loss units. 4 Experiments 4.1 Benchmark and evaluation protocol We evaluate on a controlled multi-strategy algorithmic benchmark designed to make strategy align- ment measurable. The task structure is parseable, allowing generated solution traces to be validated and matched to compatible benchmark strategies. This enables ground-truth evaluation of whether a learned latent variable tracks strategy-level variation, a property that is not directly observable in open-ended language modeling. Each example consists of a problem instancexand a solution tracey. The benchmark covers six task families: list summation, sorting algorithms, grid pathfinding, linear equation solving, base 6 conversion, and multidigit addition. Solution traces instantiate task-specific procedures for reaching the same valid answer, such as different traversal orders, algebraic manipulation orders, or algorithms. Details of the benchmark design are provided in Appendix C. We use a two-stage experimental protocol. First, to construct a controlled factorization target with known reference strategies, we fit a language model to the benchmark distribution, producing a base modelp θ (y|x)whose response distribution mixes multiple strategies. We then train a router-generator factorization on samples from this fixed response distribution, (x,y)∼Dfor base model fitting, x∼D X , y∼p θ (·|x)for factorization training. We report two primary evaluation axes. Distributional Fidelity measures whether the factorized model preserves the observable base model behavior, in particular whether it continues to produce valid solution traces. Strategy Alignment measures whether the learned latent is aligned with the reference strategy structure, in particular whether it achieves cross-input semantic consistency in assigning the same latent value to the same strategy across different problems. For each method class, we report the best run over the swept KL-to-reconstruction weightβ; sweep details are in Appendix E. 4.2 Recovering Strategy-Aligned Latents Implicit strategy information.We first ask whether the base model already contains information about the reference strategy before any router-generator factorization is trained. For each task and token position, we train a lightweight linear probe on the base model’s hidden states to predict the reference strategy annotation. 0.00.20.40.60.81.0 Position Summation Addition Base conv. Linear eqs. Sorting Grid paths Task 0.00 0.25 0.50 0.75 1.00 Probe accuracy Figure 5:Strategy labels are decodable from base model hidden states.Rows are benchmark tasks, columns are normalized token positions, and color gives linear-probe accuracy for predicting the ground-truth strategy from the base model’s hidden states. Figure 5 shows that strategy information is broadly decodable across the algorithmic bench- mark, especially at positions where strategy- specific computation is expressed in the se- quence. This provides evidence that the tar- get strategy structure is represented in the base model’s hidden states, but it is not exposed as a reusable latent variable that can be sampled, transferred, or compared across inputs. ELBO failure.The presence of implicit strat- egy information does not imply that a standard variational objective will identify it. The condi- tional ELBO is under-specified for this purpose: it rewards matching the observable response distribution, but does not requirezto explain strategy-level variation. Empirically, Figure 3 and Figure 6 show that ELBO-style baselines can preserve Distributional Fidelity while leaving the latent inert or strategy-unaligned. The proposed model-directed reconstruction objective changes which marginal-preserving factoriza- tion is learned. Figure 6 depicts the distributional fidelity and strategy alignment of factorizations of pretrained language models induced by different objectives. The model-directed reconstruc- tion objective and its variants recover strategy-aligned latents while preserving the base model’s response distribution, whereas ELBO-style baselines preserve the distribution but fail to recover strategy-aligned latents. The latent dynamics in Figure 7 provide complementary geometric evidence: posterior latents that begin as an overlapping cloud separate into task- and strategy-structured regions over training. In the multi-task setting, this organization is task-conditioned rather than a single global strategy codebook: the same latent regions can carry different strategy meanings for different task families. This latent multiplexing is visible in linear diagnostics, as strategies pooled across all tasks are not linearly separable fromzalone, but become linearly separable when task identity is provided. Together, these results indicate that model-directed reconstruction pressure can expose the strategy structure that was implicit in the base model’s response distribution. Cross-input semantic consistency.A latent variable can separate strategies locally for each input while failing to giveZa stable strategy meaning across inputs. We therefore evaluate a stronger notion of Strategy Alignment with Analogical Consistency: sample a source inputX, draw Z∼r φ (·|X), reuse the sameZon a related target inputX ′ , and check whetherY∼g φ (·|X,Z)and 7 global + token token weighting global scale token inverse ELBO β-ELBO 0.0 0.2 0.4 0.6 0.8 1.0 Metric value Qwen2.5 0.5B global + token token weighting global scale token inverse ELBO β-ELBO Qwen2.5 1.5B Distributional FidelityStrategy AlignmentBaselineAblationtask Figure 6:Method comparison on pretrained Qwen2.5 runs.For each objective variant and model size, bars show Distributional Fidelity and Strategy Alignment, with open points giving per-task values. Strategy Alignment denotes cross-input latent strategy alignment measured by Analogical Consistency. Hatching marks baselines and ablations. 2%6%10%20%50%100% Base conv. Grid paths Linear eqs. Summation Addition Sorting Figure 7:Posterior latent geometry over training.Each panel is a PCA projection of posterior mean latents at the indicated fraction of training. Points are colored by task and strategy; the initially overlapping cloud separates into task- and strategy-specific regions over training. Y ′ ∼g φ (·|X ′ ,Z)use the same inferred strategy. In the main figures, including Figure 6, “Strategy Alignment” denotes this stronger cross-input metric, which we write asAnalogicalConsistency = Pr[strat(Y) = strat(Y ′ )] . High Strategy Alignment therefore means thatZcarries the same strategy meaning across inputs; Figure 6 shows that our objective recovers this property. Ablation:inverse-surprisal reconstruction pressure.The proposed objective puts more re- construction pressure on high-surprisal tokens underp θ , where strategy-branching information is expected to concentrate. We test this hypothesis through an ablation in which the reconstruction loss is weighted byinversesurprisal, so that low-surprisal tokens receive more emphasis. This ablation weakens Strategy Alignment, supporting the view that the model-directed reconstruction helps by pressuringzto encode strategy-relevant response variation. Details are in Appendix E. 4.3 How Model-Directed Reconstruction Pressure Drives Strategy Alignment The preceding results show that the proposed objective recovers strategy-aligned latents. We next ask which part of the training signal explains the alignment gains across proposed methods and variational baselines. If alignment comes from capturing high-surprisal response variation inz, runs with lower token-weighted reconstruction loss should have higher Strategy Alignment. Figure 8 supports this mechanism-level interpretation. The phase-space dynamics show that, when the objective is underspecified, the model can move toward high Distributional Fidelity without moving toward high Strategy Alignment. The scatter plot shows that token-weighted reconstruction loss tracks Strategy Alignment across all methods. This suggests strategy alignment improves when zis pressured to explain strategy-relevant residual surprisal underp θ . 5 Toward a Theory of Strategy-Structure Recovery This section outlines a line of analysis that can provide theoretical insight into the methods introduced in the paper. The goal is to understand when the variational approach can identify a latent variableZ that aligns with a target strategy structureS ⋆ implicit in the pretrained model. We provide a preview of the main results here; the full formal treatment is given in Appendix F. Consider finite setsXandY, an input lawμ, and a fixed pretrained response distributionp(y|x). For an encoderq(z|x,y), routerr(z|x), and generatorg(y|x,z), letQ q be the joint law obtained 8 0.600.650.700.750.800.850.900.951.00 Distributional Fidelity 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Strategy-Alignment ELBO β-ELBO global scale token weighting global + token token inverse 10 −3 10 −2 10 −1 10 0 Token-Weighted Reconstruction 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Strategy-Alignment ELBO baseline global scale token weighting global + token token inverse ELBO baseline global scale token weighting global + token token inverse Figure 8:Left:training trajectories in fidelity-alignment phase space for different objective variants. Right:token-weighted reconstruction loss versus Strategy Alignment at training endpoints; outlined points show method centroids and translucent points show individual runs. by samplingX∼μ,Y∼p(· |X), andZ∼q(· |X,Y). We study, forβ∈[0,1)andλ≥0, the population complexity-penalized variational objective L β,λ (q,r,g) :=E Q q [−logg(Y|X,Z)] +βE X,Y KL(q(·|X,Y)∥r(·|X)) +λC(r,g), (5) whereC(r,g)is a theoretical proxy for architecture, parameterization, and optimization biases that favor simple router-generator factorizations. The appendix shows the decomposition L β,λ (q,r,g) =H(Y|X)−(1−β)I q (Y;Z|X) + DecGap(q,g) +βPriorGap(q,r) +λC(r,g), (6) whereDecGapandPriorGapare the generator and router KL gaps. For a fixed encoderq, define A β,λ (q) :=inf r∈R, g∈G [DecGap(q,g) +βPriorGap(q,r) +λC(r,g)],(7) We show that minimizingL β,λ is equivalent to maximizing the following profiled score J prof β,λ (q) :=I q (Y;Z|X)− A β,λ (q) 1−β .(8) The first termI q (Y;Z|X)is the information pressure that encouragesZto capture information aboutYbeyond what is already predictable fromX, while the second termA β,λ (q)is an effective approximation-complexity penalty that measures whether the induced distribution is close to the target while being simple, as measured by the complexity termC(r,g). LetS ⋆ =f(X,Y)be the targeted strategy, and letq ⋆ (z|x,y) =1z=S ⋆ . Letd(q,S ⋆ )be a recovery discrepancy betweenqandS ⋆ . Our primary result gives sufficient conditions under which the complexity-penalized variational objective selects a latent factorization aligned withS ⋆ . Theoretical analysis of latent-variable disentanglement Given a discrepancy measured(q,S ⋆ ), define ∆ prof β,λ (δ) =J prof β,λ (q ⋆ )−sup q∈Q:d(q,S ⋆ )≥δ J prof β,λ (q).(9) If∆ prof β,λ (δ)>0, then every population minimizer(bq,br,bg)∈arg min q,r,g L β,λ (q,r,g)satisfies d(bq,S ⋆ )< δ. Moreover, if the empirical objective b L β,λ,n satisfies, with probability at least1−η, sup q,r,g b L β,λ,n (q,r,g)−L β,λ (q,r,g) ≤ (1−β)∆ prof β,λ (δ) 4 , 9 then every empirical minimizer(bq n ,br n ,bg n )∈arg min q,r,g b L β,λ,n (q,r,g)satisfiesd(bq n ,S ⋆ )< δ. For finiteQ,R,Gwith sub-Gaussian losses, n=O log(|Q||R||G|) + log(1/η) (1−β) 2 (∆ prof β,λ (δ)) 2 ! is sufficient to ensure this condition. 6 Conclusion This paper studies the problem of recovering explicit latent variable representations of reasoning strategies from pretrained language models. The central observation is that a pretrained model’s response distribution is an entangled mixture over high-level strategies, and that exposing this structure requires more than fitting a latent variable model to observable outputs using previously explored methods. In particular, because the base generator already realizes this response distribution, a factorization that ignores the latent variable is immediately available and is globally optimal under objectives that only measure marginal fit. We show that standard variational training can preserve the observable response distribution while leaving the latent variable unused. To address this, we propose a model-directed reconstruction objective that uses the base model itself as a reference. In our approach, reconstruction is normalized by the base model’s response loss and weighted by the uncertainty seen in the base model at the token level, allowing the loss to concentrate on positions where strategy choices are most consequential. On a controlled benchmark of multi-strategy algorithmic tasks, this objective recovers latent structure that is aligned with reference solution strategies and is semantically consistent across inputs. The benchmark design makes strategy recovery measurable, but it limits the scope of the conclusions. The benchmarks we introduce cover synthetic algorithmic tasks with ground-truth strategies; how well the methodology extends to open-ended reasoning tasks remains an open question. Moreover, the theoretical analysis is preliminary, and the connection between the oracle selection criterion and actual training dynamics is not yet formally characterized. Further discussion of related work, broader implications, and future directions appears in Appendix A. References [AMR09]Elizabeth S. Allman, Catherine Matias, and John A. Rhodes. “Identifiability of Param- eters in Latent Structure Models with Many Observed Variables”. In:The Annals of Statistics(2009) (cited on pages 13, 14). [Bao+19] Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xin-yu Dai, and Jiajun Chen. “Generating Sentences from Disentangled Syntactic and Semantic Spaces”. In:Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019 (cited on pages 13, 14). [Bow+16]Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, and Samy Bengio. “Generating Sentences from a Continuous Space”. In:Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. 2016 (cited on pages 2, 13, 14). [Bur+18]Christopher P. Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. “Understanding disentangling inβ-VAE”. 2018. arXiv:1804.03599 [stat.ML](cited on pages 13, 14). [Che+16]Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. “InfoGAN: Interpretable Representation Learning by Information Maximizing Genera- tive Adversarial Nets”. In:Advances in Neural Information Processing Systems. 2016 (cited on pages 13, 14). [Fu+19]Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. “Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing”. In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. 2019 (cited on pages 2, 14). [Gal+25]Andrey Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y. Rogov, Elena Tutubalina, and Ivan Oseledets. “I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders”. 2025. arXiv: 2503.18878 [cs.CL](cited on pages 13, 14). 10 [Gu+23]Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Weihong Zhong, and Bing Qin. “Controllable Text Generation via Probability Density Estimation in the Latent Space”. In:Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023 (cited on page 14). [He+19]Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg-Kirkpatrick. “Lagging Inference Networks and Posterior Collapse in Variational Autoencoders”. In:Interna- tional Conference on Learning Representations. 2019 (cited on pages 2, 14). [Hig+17]Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. “beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework”. In:International Conference on Learning Representations. 2017 (cited on pages 13, 14). [HJ16]Matthew D. Hoffman and Matthew J. Johnson. “ELBO Surgery: Yet Another Way to Carve up the Variational Evidence Lower Bound”. 2016 (cited on pages 2, 14). [HST19]Aapo Hyvärinen, Hiroaki Sasaki, and Richard E. Turner. “Nonlinear ICA Using Aux- iliary Variables and Generalized Contrastive Learning”. In:Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics. 2019 (cited on pages 13, 14). [Hu+22]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. “LoRA: Low-Rank Adaptation of Large Language Models”. In:International Conference on Learning Representations. 2022 (cited on page 4). [Hub+24]Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. “Sparse Autoencoders Find Highly Interpretable Features in Language Models”. In: International Conference on Learning Representations. 2024 (cited on page 2). [JT99]Wenxin Jiang and Martin A. Tanner. “Hierarchical Mixtures-of-Experts for Exponential Family Regression Models: Approximation and Maximum Likelihood Estimation”. In: The Annals of Statistics(1999) (cited on pages 13, 14). [Kes+19]Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. “CTRL: A Conditional Transformer Language Model for Controllable Genera- tion”. 2019. arXiv:1909.05858 [cs.CL](cited on page 2). [Khe+20] Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, and Aapo Hyvärinen. “Variational Autoencoders and Nonlinear ICA: A Unifying Framework”. In:Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics. 2020 (cited on pages 13, 14). [KM18]Hyunjik Kim and Andriy Mnih. “Disentangling by Factorising”. In:Proceedings of the 35th International Conference on Machine Learning. 2018 (cited on pages 13, 14). [Koh+20]Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. “Concept Bottleneck Models”. In:Proceedings of the 37th International Conference on Machine Learning. 2020 (cited on page 2). [Kon+25]Deqian Kong et al. “Latent Thought Models with Variational Bayes Inference-Time Com- putation”. In:Proceedings of the 42nd International Conference on Machine Learning. 2025 (cited on page 14). [KW13]Diederik P. Kingma and Max Welling. “Auto-Encoding Variational Bayes”. 2013. arXiv: 1312.6114 [stat.ML](cited on pages 1, 13, 14). [Li+20]Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, and Jianfeng Gao. “Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space”. In:Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, 2020 (cited on pages 13, 14). [L21]Xiang Lisa Li and Percy Liang. “Prefix-Tuning: Optimizing Continuous Prompts for Generation”. In:Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021 (cited on page 2). [MF21] Giangiacomo Mercatali and André Freitas. “Disentangling Generative Factors in Natural Language with Discrete Variational Autoencoders”. In:Findings of the Association for Computational Linguistics: EMNLP 2021. 2021 (cited on pages 13, 14). [PL21] Seongmin Park and Jihwa Lee. “Finetuning Pretrained Transformers into Variational Autoencoders”. In:Proceedings of the Second Workshop on Insights from Negative Results in NLP. 2021 (cited on pages 13, 14). 11 [Qwe24]Qwen Team. “Qwen2.5 Technical Report”. 2024. arXiv:2412.15115 [cs.CL](cited on page 20). [Ser+17]Iulian V. Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. “A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues”. In:Proceedings of the AAAI Conference on Artificial Intelligence. 2017 (cited on page 14). [Sha+17]Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer”. In:International Conference on Learning Representations. 2017 (cited on page 13). [SLY15] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. “Learning Structured Output Representa- tion using Deep Conditional Generative Models”. In:Advances in Neural Information Processing Systems. 2015 (cited on pages 1, 13, 14). [Tan+24]Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, and Huan Liu. “Interpreting Pretrained Language Models via Concept Bottlenecks”. In:Advances in Knowledge Discovery and Data Mining. 2024 (cited on pages 13, 14). [Tei63]Henry Teicher. “Identifiability of Finite Mixtures”. In:The Annals of Mathematical Statistics(1963) (cited on pages 13, 14). [Tuy+25]Jens Tuyls, Dylan J. Foster, Akshay Krishnamurthy, and Jordan T. Ash. “Representation- Based Exploration for Language Models: From Test-Time to Post-Training”. 2025. arXiv:2510.11686 [cs.LG](cited on pages 2, 15). [Vas+22]Jake Vasilakes, Chrysoula Zerva, Makoto Miwa, and Sophia Ananiadou. “Learning Disentangled Representations of Negation and Uncertainty”. In:Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022 (cited on pages 13, 14). [Wan+23]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. “Self-Consistency Improves Chain of Thought Reasoning in Language Models”. In:International Conference on Learning Representa- tions. 2023 (cited on page 2). [Wu+26] Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, and Yejin Choi. “The Invisible Leash: Why RLVR May or May Not Escape Its Origin”. 2026. arXiv:2507.14843 [cs.LG](cited on page 15). [Yao+23]Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. “Tree of Thoughts: Deliberate Problem Solving with Large Language Models”. In:Advances in Neural Information Processing Systems. 2023 (cited on page 2). [YS68]Sidney Yakowitz and John D. Spragins. “On the Identifiability of Finite Mixtures”. In: The Annals of Mathematical Statistics(1968) (cited on pages 13, 14). [Yue+25]Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?” 2025. arXiv:2504.13837 [cs.AI](cited on pages 2, 15). [Zha+24]Yingji Zhang, Marco Valentino, Danilo Carvalho, Ian Pratt-Hartmann, and Andre Freitas. “Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEn- coders”. In:Findings of the Association for Computational Linguistics: NAACL 2024. 2024 (cited on pages 13, 14). [ZSE19]Shengjia Zhao, Jiaming Song, and Stefano Ermon. “InfoVAE: Balancing Learning and Inference in Variational Autoencoders”. In:Proceedings of the AAAI Conference on Artificial Intelligence. 2019 (cited on page 14). [ZZE17] Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. “Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders”. In:Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017 (cited on pages 13, 14). 12 A Extended Discussion This paper studies the problem of recovering explicit latent variable representations of reasoning strategies from pretrained language models. The central observation is that a pretrained model’s response distribution is an entangled mixture over high-level strategies, and that exposing this structure requires more than fitting a latent variable model to observable outputs using previously explored methods. In particular, because the base generator already realizes this response distribution, a factorization that ignores the latent variable is immediately available and is globally optimal under objectives that only measure marginal fit. We show that standard variational training can preserve the observable response distribution while leaving the latent variable unused. To address this, we propose a model-directed reconstruction objective that uses the base model itself as a reference. In our approach, reconstruction is normalized by the base model’s response loss and weighted by the uncertainty seen in the base model at the token level, allowing the loss to concentrate on positions where strategy choices are most consequential. On a controlled benchmark of multi-strategy algorithmic tasks, this objective recovers latent structure that is well-aligned with reference solution strategies, while also being semantically consistent across inputs. In this way, the latent variable transfers the same strategy from one problem to another. A.1 Related Work There is a rich and extensive literature on latent variable methods for generative models of text. This work includes VAEs [KW13] and CVAEs [SLY15], text and sequence VAEs [Bow+16; ZZE17], and disentanglement-oriented variants such as theβ-VAE [Hig+17; Bur+18], FactorVAE [KM18], and InfoGAN [Che+16], as well as pretrained-Transformer VAEs such as OPTIMUS [Li+20] and the Transformer VAE of Park and Lee [PL21]. Adjacent language-VAE work also uses structured latent spaces to separate syntax from semantics, represent discrete linguistic factors, and control semantic attributes such as negation or uncertainty [Bao+19; MF21; Vas+22; Zha+24]. The problem we study in this work is not directly addressed by this prior work. The key distinction is that these methods learn a latent variable model where the target distribution is not represented by the model at initialization. In our setting the base model already realizesp θ (y|x), and it would be inefficient, if not impossible, to retrain the model from scratch to expose the strategy. Mixture-of-experts language models also learn routing over specialized generators [Sha+17; JT99], but such models are optimized for capacity or efficiency, not for correspondence to an interpretable strategy variable, and training is typically carried out from scratch. Mechanistic interpretability methods, including sparse autoencoders [Gal+25] and concept-bottleneck approaches [Tan+24], extract or steer internal representations post-hoc; our goal is instead a generative factorization that supports sampling and intervention. While there is also a rich theory for latent variable methods, classical identifiability theory for mixture models [Tei63; YS68; AMR09] and nonlinear ICA [HST19; Khe+20] achieve uniqueness by restricting the component family or adding auxiliary observations. When using an expressive pretrained generator, classical identifiability conditions generally fail, and the central question becomes how to select a useful factorization among the many that preserve the marginal distribution. An extended discussion of related work with fuller citations is given in the supplementary material. A.2 Implications and Future Directions The ideas and methods presented here open up several directions for future work; here we briefly mention potential connections to interpretability, directed exploration, and safety. First, because a strategy-aligned factorization exposes the strategies implicit in a pretrained model’s response distribution and makes them comparable across related inputs, it can provide a higher-level and more interpretable description of model behavior than token-level analysis. Moreover, intervening onz at generation time can direct the model toward a specific strategy; shifting the router distribution r φ (z|x)adjusts the relative weight of different strategies without retraining. This has the potential to provide a structured prior for exploring the strategy space, which could improve coverage in settings where the base model under-samples certain valid approaches. Finally, because strategy-level control offers a more interpretable abstraction for constraining or auditing generation compared with token-level steering, the methods introduced in this paper may support applications where certain reasoning paths need to be permitted or excluded, presenting possible benefits for safety. 13 The scope of our conclusions is limited in two respects. The benchmarks we introduce cover synthetic algorithmic tasks with ground-truth strategies; how well the methodology extends to open-ended reasoning tasks remains an open question. Moreover, the theoretical analysis is preliminary, and the connection between the oracle selection criterion and actual training dynamics is not yet formally characterized. B Extended Related Work Conditional latent-variable models and language VAEs.Latent-variable generative models provide the formal background for our router-generator factorization. The VAE and conditional VAE literature learns a latent-variable model from observed data, using an encoder or recognition model and an ELBO to train a generator with latent variation [KW13; SLY15]. Sequence and dialogue VAEs extend this idea to autoregressive generation, where the latent is intended to capture sentence-level, discourse-level, or response-level variation beyond the input context [Bow+16; Ser+17; ZZE17]. Pretrained language VAEs and latent-thought models scale related ideas with pretrained Transformer components, continuous sentence spaces, or inference-time latent optimization [Li+20; PL21; Kon+25]. Another line of language-VAE work explicitly structures the latent space around controllable or disentangled linguistic factors. Prior work separates syntax and semantics in sentence VAEs [Bao+19], extends syntactic-semantic separation to Transformer-based VAEs [Zha+24], uses discrete latent variables to capture natural-language generative factors [MF21], and disentangles semantic attributes such as negation and uncertainty from content [Vas+22]. Latent-space control methods further model control distributions directly in the VAE latent space [Gu+23]. Our setting differs in the learning target: rather than fitting a latent-variable generator to data, we start from a fixed or pretrained response distributionp θ (y|x)and learn a factorization R r φ (z|x)g φ (y|x,z)dz whose latent variable exposes strategy-level variation already implicit in that distribution. Posterior collapse, latent usage, and disentanglement objectives.Autoregressive decoders can ignore latent variables, giving rise to posterior-collapse and KL-vanishing pathologies in sequence VAEs [Bow+16; He+19; Fu+19]. Several objective analyses separate the standard ELBO regularizer into latent information and aggregate-prior matching terms, motivating variants that preserve latent usage while keeping generation well regularized [HJ16; ZSE19]. Disentanglement-oriented objectives such asβ-VAE, capacity-controlledβ-VAE, FactorVAE, and InfoGAN also show that useful latent structure usually requires explicit inductive bias or information pressure, not merely the presence of a latent code [Hig+17; Bur+18; KM18; Che+16]. These works motivate our use of objective terms that pressurezto carry information. However, our desideratum is more specific than latent activity or visual-factor disentanglement:zshould be informative aboutybeyondx, aligned with reference solution strategies, and semantically consistent across related inputs. The collapse issue is also sharper in our adaptation setting because the initialized generator already realizes the observable conditional behavior without consultingz. Identifiability, conditional mixtures, and routing.Classical finite-mixture theory studies when a marginal distribution uniquely determines its mixture components, often through restrictive com- ponent families, linear-independence conditions, or multi-view structure [Tei63; YS68; AMR09]. Conditional mixture and mixture-of-experts models introduce input-dependent routing or gating, but their classical guarantees usually concern approximation, prediction, or consistency under structured expert families rather than recovery of a semantically named latent variable [JT99]. Identifiable latent- variable and nonlinear-ICA results restore stronger recovery guarantees by adding auxiliary variables or conditional-prior structure that breaks otherwise unavoidable symmetries [HST19; Khe+20]. These literatures clarify why latent-variable factorization is not identified by marginal fit alone. In our setting, the router and generator are expressive neural language models, so many factorizations can preservep θ (y|x). The problem is therefore not only estimation of the observable conditional law, but selection of a useful factorization whose latent variable captures reusable strategy structure. Interpretability and reasoning features.A separate line of work exposes internal model structure through supervised concept bottlenecks or post-hoc feature decompositions. Concept-bottleneck methods for pretrained language models insert human-defined concept variables and train the model to predict through them, providing an explicit supervised interpretability interface [Tan+24]. Sparse- autoencoder analyses of reasoning models instead discover internal features associated with reasoning motifs and can steer model behavior by intervening on feature directions [Gal+25]. These approaches are closely related to our interpretability motivation because they aim to make latent reasoning structure visible and manipulable. The distinction is that we learn a generative router-generator 14 factorization ofp θ (y|x), so the learned latent variable participates directly in generation rather than serving only as a supervised concept layer or a post-hoc activation feature. Exploration and RLVR motivation.Recent analyses of reinforcement learning with verifiable rewards suggest that on-policy post-training often sharpens solutions already accessible under the base model rather than broadly expanding the set of reachable reasoning strategies [Yue+25; Wu+26]. Representation-based exploration methods respond to this limitation by adding novelty or diversity pressure in model representation space [Tuy+25]. This literature motivates strategy-level intervention as a possible future use of latent-variable factorization: a reusable strategy-alignedzcould provide a higher-level handle for directed exploration than token-space sampling alone. We do not evaluate RLVR or downstream exploration in this paper; the connection is a motivation for why exposing abstract strategies in a pretrained model may be useful beyond the controlled benchmark. C Benchmark and Data Generation The controlled multi-strategy algorithmic benchmark provides problem instances, solution traces, and reference strategy labels in a setting where strategy alignment can be measured directly. Each task family is designed so that the same problem can be solved by multiple valid procedures. The procedures differ in the intermediate solution trace while preserving the same final answer. This gives an evaluation reference for asking whether a learned latent variable corresponds to strategy-level variation in generated reasoning traces. C.1 Task families Table 1 summarizes the benchmark tasks used in the paper. Within each task family, strategies are sampled uniformly from the listed strategy set. The table lists the sampled strategies, not every procedure that could in principle solve the task. The task families probe complementary forms of strategy variation: reduction order for summation, state-transition algorithms for sorting, move-order policies for grid paths, algebraic manipulation order for linear equations, conversion procedures for base conversion, and arithmetic decomposition for multidigit addition. In each case, the strategies alter the intermediate trace while preserving the same problem-answer relation. Table 1:Controlled multi-strategy algorithmic benchmark.Each task family admits multiple valid solution procedures for the same problem instance. The benchmark uses the listed sampled strategies as reference labels for evaluating strategy-aligned latent variables. Task familyProblem domainSampled strategies List summationFour integers from0to9left-to-right; right-to-left; pairwise Sorting algorithmsFive integers from0to9bubble sort; insertion sort; merge sort Grid pathfindingMonotone shortest paths on a6×6 grid right-first; down-first; alternating Linear equation solvingInteger equationsax+b=c, with a= 2,...,9and integer-safe solutions and offsets in[−9,9] subtract-then-divide; divide-then-subtract; inverse-ops Base conversionIntegern= 1,...,255, base in 2,4,8,16 repeated division; via binary; decomposition Multidigit additionTwo three-digit nonnegative integersright-to-left carry; left-to-right partials; rounding decomposition C.2 Data generation The benchmark data distribution samples a task family, a problem instance for that task, and a reference strategy for solving that instance: T∼Unif(task families), X∼D T , S∼Unif(S T ), Y= Trace T (X,S). 15 HereD T is the task-specific input distribution andS T is the listed strategy set for taskT. The rendering functionTrace T produces a solution trace that follows strategySand reaches the task’s correct final answer. The resulting example consists of a problem instanceX, a solution traceY, and a reference strategy labelSthat is used for evaluation but not training. C.3 Reference strategy identification The benchmark is designed to enable parsing solution trajectories generated by a model and identify- ing the reference strategy. Given a problem instance and a generated solution trace, the task parser checks whether the trace is a valid solution and identifies which benchmark strategy it expresses. This makes it possible to evaluate whether generated outputs preserve valid task behavior and whether a learned latent variable is aligned with the benchmark’s reference strategy structure. The reference labels are used as an evaluation target for the controlled benchmark. C.4 Representative examples To illustrate the benchmark, the examples below show a sample problem from each task with two sample solutions demonstrating two different strategies. Some traces are abbreviated with ellipses where the omitted steps continue the same procedure. List summation.For the problem instance “3, 7, 2, 8”, the benchmark can render traces such as: StrategyTrace left-to-right3+7=10 ; 10+2=12 ; 12+8=20 pairwise3+7=10 ; 2+8=10 ; 10+10=20 Sorting algorithms.For the problem instance “[5,2,8,1,4]”, the benchmark can render traces such as: StrategyTrace bubble sort[2,5,8,1,4] ; [2,5,1,8,4] ; ... ; [1,2,4,5,8] merge sort[2,5,8,1,4] ; [2,5,1,8,4] ; [1,2,5,8,4] ; [1,2,4,5,8] Grid pathfinding.For the problem instance “grid=6 ; start=(0,0) ; goal=(2,3)”, the benchmark can render traces such as: StrategyTrace right-firstR:(0,1) ; R:(0,2) ; R:(0,3) ; D:(1,3) ; D:(2,3) alternatingR:(0,1) ; D:(1,1) ; R:(1,2) ; D:(2,2) ; R:(2,3) Linear equation solving.For the problem instance “3x+6=21”, the benchmark can render traces such as: StrategyTrace subtract-then-divide3x+6=21 ; 3x=15 ; x=5 divide-then-subtract3x+6=21 ; x+2=7 ; x=7-2 ; x=5 Base conversion.For the problem instance “n=45, base=4”, the benchmark can render traces such as: StrategyTrace repeated divisionDIV:45/4=11,r:1 ; DIV:11/4=2,r:3 ; DIV:2/4=0,r:2 ; OUT:231 decompositionTERM:2*4^2=32 ; TERM:3*4^1=12 ; TERM:1*4^0=1 ; OUT:231 Multidigit addition.For the problem instance “348+274”, the benchmark can render traces such as: StrategyTrace right-to-left carryd2:8+4+0=12->w2,c1 ; d1:4+7+1=12->w2,c1 ; d0:3+2+1=6->w6,c0 ; sum=622 left-to-right partialsp100:300+200=500 ; p10:40+70=110 ; p1:8+4=12 ; sum=500+110+12=622 16 Algorithm 1:Model-directed training of a router-generator factorization Input:Input distributionD X ; frozen base modelp θ ; objective parametersβ,α,γ. Output:Routerr φ (z|x)and strategy-conditioned generatorg φ (y|x,z). 1Initializeφfromθ, adding router heads and a latent embedding projection 2Initializeξfromθ, adding posterior heads 3Freezep θ as both the response distribution to factorize and the reconstruction reference 4Estimatec θ =E x∼D X , y∼p θ (·|x) [b θ (x,y)]using frozen-base samples 5foreach optimization stepdo 6Sample a minibatchx i ∼D X , fori= 1,...,B 7Sample responsesy i ∼p θ (·|x i ) 8Compute frozen-base surprisalsb θ,t (x i ,y i )and weightsa θ,t (x i ,y i ) 9Sample or reparameterizez i ∼q ξ (·|x i ,y i ) 10Form b J θ =B −1 P B i=1 J θ (x i ,y i ;φ,ξ) 11Updateφandξby a stochastic gradient step on b J θ , leavingθfixed 12end D Details of Training Methodology Section 3 describes the objective-level motivation for model-directed reconstruction. This appendix gives the corresponding training procedure and the implementation details needed to reproduce the factorization objective. Throughout this section,p θ is a frozen base model,φdenotes the adapted router-generator parameters, andξdenotes the training-time posterior parameters. The training distribution is generated by the frozen base model itself: inputs are drawn fromD X and responses are sampled fromp θ (·|x). D.1 Architecture Parameterization The router and generator are implemented as two roles of one adapted autoregressive Transformer. The router reads the input prefix and produces a Gaussian distribution over continuous latent values. Leth (L) φ (x)be the final-layer representation produced by the adapted Transformer after readingx. The router distribution is r φ (z|x) =N W μ h (L) φ (x),diag exp(W σ h (L) φ (x)) . During autoregressive generation, a latent samplezis projected to an embeddingE φ (z)and inserted as a pseudo-token at the embedding layer. The generator then predicts each response token while attending tox, the latent pseudo-token, and the previous response tokens. The training-time posterior uses the same construction, but reads the full input-response sequence. Leth (L) ξ (x,y)be the final-layer representation after reading(x,y). The posterior distribution is q ξ (z|x,y) =N U μ h (L) ξ (x,y),diag exp(U σ h (L) ξ (x,y)) . The posterior is used only during training to provide variational signal. At generation time, latents are sampled from the router. Both adapted models are initialized from the frozen base model parametersθ. In the experiments, the trainable parameters consist of LoRA-style low-rank updates to the Transformer backbone together with the Gaussian heads and latent embedding projection. This keeps the factorization close to the base model while allowing the router and generator to assign a nontrivial role toz. D.2 Objective and Normalization Details For a sampled responsey= (y 1 ,...,y T y ), define the frozen-base token surprisal and response- average surprisal as b θ,t (x,y) =−logp θ (y t |x,y <t ), b θ (x,y) = 1 T y T y X t=1 b θ,t (x,y). 17 The global reference scale is c θ =E x∼D X , y∼p θ (·|x) [b θ (x,y)]. Theθsubscript marks dependence on the frozen base model. Token-level reconstruction weights are derived from the base model surprisals. Forγ≥0, define ew (γ) θ,t (x,y) = b θ,t (x,y) γ P T y s=1 b θ,s (x,y) γ , κ (γ) θ (x,y) = P T y s=1 ew (γ) θ,s (x,y)b θ,s (x,y) b θ (x,y) , and w (γ) θ,t (x,y) = ew (γ) θ,t (x,y) κ (γ) θ (x,y) , a θ,t (x,y) =α 1 T y + (1−α)w (γ) θ,t (x,y). The parameterα∈[0,1]mixes uniform response-token pressure with token-levelp θ -surprisal- directed pressure. The parameterγcontrols how strongly the token weights concentrate on high- surprisal positions. The base model reference scores, token weights, and normalization constants defined above are treated as constants when updatingφandξ. The resulting reconstruction term and training objective are R θ (x,y;φ,ξ) = 1 c θ E z∼q ξ (·|x,y)   T y X t=1 a θ,t (x,y) [−logg φ (y t |x,z,y <t )]   , and J θ (x,y;φ,ξ) =R θ (x,y;φ,ξ) +βKL(q ξ (·|x,y)∥r φ (·|x)). The normalization byκ (γ) θ preserves the frozen-base response scale under token weighting: T y X t=1 w (γ) θ,t (x,y)b θ,t (x,y) =b θ (x,y). Since the uniform component also satisfies T y X t=1 1 T y b θ,t (x,y) =b θ (x,y), the mixed weights satisfy T y X t=1 a θ,t (x,y)b θ,t (x,y) =b θ (x,y). Consequently, if the generator matches the frozen base model token distribution, so that g φ (y t |x,z,y <t ) =p θ (y t |x,y <t ), thenR θ (x,y;φ,ξ) =b θ (x,y)/c θ . Taking expectation over x∼ D X andy∼p θ (·|x)givesE[R θ ] = 1. ThusR θ can be read as a base-normalized frac- tion of the frozen-base response loss that remains after conditioning on the latent. D.3 Inference And Evaluation Interface The posteriorq ξ (z|x,y)is a training-time object. It provides latent samples for reconstruction and a variational bridge between the generator and router, but it is not needed to sample from the learned factorization. At inference time, the model first samplesz∼r φ (·|x)and then generatesy autoregressively fromg φ (·|x,z). Strategy annotations are not used in the optimization loop. They enter only after training, when we evaluate whether the learned latent is informative and strategy-aligned, separates strategies within each input, and assigns semantically consistent latent regions across inputs E Experimental Details E.1 Evaluation protocol and metrics We evaluate trained router-generator factorizations using the parseable structure of the benchmark described in Appendix C. Generated solution traces can be parsed and assigned to the benchmark’s reference strategy structure, enabling direct evaluation of whether the learned latent variable preserves task behavior and has stable strategy meaning across related inputs. Strategy labels are used only for evaluation. 18 Table 2:Objective variants used in the experimental figures.All model-directed variants train on samplesx∼D X , y∼p θ (·|x)and keepp θ frozen as the reference model. Method labelReconstruction termRole in comparison ELBO baselineStandard token reconstruction, with- out base-relative normalization Tests whether ordinary variational train- ing identifies strategy-aligned latents. Global scaleBase-relative reconstruction with uni- form token weights Tests the effect of putting reconstruction in frozen-base reference units. Token weightingBase-relative reconstruction with high- surprisal token weights Tests model-directed pressure toward re- sponse regions wherep θ has residual sur- prisal. Global + tokenConvex mixture of uniform base- relative and high-surprisal token- weighted reconstruction Combines global normalization with token-level model-directed pressure. Token inverseBase-relative reconstruction with inverse-surprisal token weights Directional negative control emphasizing low-surprisal tokens. Distributional Fidelity.Distributional Fidelity measures whether samples from the factorized model preserve the base model response behavior at the level of valid benchmark solutions. Opera- tionally, we sampleZ∼r φ (·|X), generateY∼g φ (·|X,Z), parse the generated trace, and report the fraction of samples that remain strategy-compatible task solutions. Strategy Alignment.The main-paper Strategy Alignment metric is Analogical Consistency, as defined in Section 4.2. We sample a source inputX, drawZ∼r φ (·|X), generateY∼g φ (·|X,Z), reuse the sameZon a related target inputX ′ , and generateY ′ ∼g φ (·|X ′ ,Z). The score is AnalogicalConsistency = Pr[strat(Y) = strat(Y ′ )], whereX ′ is sampled from the same task family asX. This tests whether the latent value carries the same strategy meaning across different inputs. Supporting diagnostics.Router Strategy Decodability is a linear-probe diagnostic measuring whether reference strategy information is linearly recoverable from router-sampled latentsZ∼ r φ (·|X) . It is useful for diagnosing latent information, but it is weaker than Analogical Consistency because it does not test whether the same latent value has reusable cross-input meaning. For mechanism analysis, we also report base-relative reconstructionR θ , token-weighted reconstruction, and the posterior-router prior KL term from the training objective. E.2 Objective variants and inverse-surprisal ablation Appendix D.2 gives the full model-directed objective. The experiments compare variants of J(x,y;φ,ξ) =R(x,y;φ,ξ) +βKL(q ξ (·|x,y)∥r φ (·|x)), where the variants differ in the reconstruction termR. The standard ELBO uses the unnormalized autoregressive reconstruction loss, while the proposed variants use the frozen base modelp θ to normalize and direct reconstruction pressure. For uniform base-relative reconstruction, the token weights area θ,t = 1/T y . For surprisal- proportionate token weighting,a θ,t is the concentration-corrected token weight defined from the frozen-base surprisalb θ,t (x,y)in Appendix D.2. The global+token variants use convex mixtures of the uniform base-relative term and the high-surprisal token-weighted term. Table 2 maps the paper-facing method labels to the objective components used in the figures. Inverse-surprisal ablation.The inverse-surprisal ablation tests whether the direction of token-level pressure matters. Let ̃ b θ,t (x,y) = maxb θ,t (x,y),ε, and define inverse-surprisal weights ev θ,t (x,y) = ̃ b θ,t (x,y) −1 P T y s=1 ̃ b θ,s (x,y) −1 . As with the high-surprisal token-weighted term, we use a per-example scale correction so that the z-free base model reconstruction remains on the same base-relative scale: v θ,t (x,y) = ev θ,t (x,y) η θ (x,y) , η θ (x,y) = P T y s=1 ev θ,s (x,y)b θ,s (x,y) b θ (x,y) . 19 Replacinga θ,t byv θ,t emphasizes tokens already well predicted byp θ , while preserving the same z-free reconstruction scale in expectation. The ablation therefore tests the mechanism-level hypothesis that alignment gains come from pressure on high-surprisal, strategy-relevant response regions rather than from nonuniform token weighting alone. E.3 Training settings and hyperparameter sweeps The default experimental setting is multi-task factorization: each run trains one router-generator model jointly across all six benchmark task families. This setting better matches the heterogeneous training regime of language models, where a single model must represent many task distributions. Additional single-task runs are used for diagnostic sweeps and mechanism plots where per-task variation is useful. We construct the benchmark base models from two initializations: pretrained Qwen2.5 checkpoints and random weights. In both settings, we first fit the model to the benchmark task distribution; we then freeze and factorize the resulting base modelp θ . The randomly initialized diagnostic grid contains 504 complete LoRA adaptation runs: six single-task settings plus the multi-task setting, twelve reconstruction-family settings including ablations, three KL weightsβ∈1,0.1,0.01, and two schedules forβ(constant and linear warmup over training). The pretrained method-comparison runs factorize Qwen2.5 0.5B and 1.5B base models [Qwe24] in the multi-task setting with con- tinuous latent dimension 64. We use the Hugging Face checkpointsQwen/Qwen2.5-0.5Band Qwen/Qwen2.5-1.5B, whose model cards list the Apache 2.0 license. E.4 Additional diagnostics The main text focuses on the headline pretrained method comparison and compact mechanism diagnostics. Here we include additional views that support the same claims: latent geometry over training, role of KL weights, metric associations, and phase-space training dynamics. F A Theory of Variational Approaches to the Strategy-Structure Recovery Problem This appendix gives the formal treatment behind the preview theorem in the main text. The main result studies the population ELBO over independently parameterized expressive encoder, router, and generator classes. F.1 Setup and the Strategy Structure Recovery Problem LetXandYbe finite sets (e.g., space of token sequences). Letμbe a fixed input law onX, and let p(y|x)be a fixed conditional law onY. All expectations, entropies, and mutual informations are computed under X∼μ, Y∼p(·|X), unless stated otherwise. Because all sample spaces are finite, measurability is automatic; the only analytic issues are support, finite-value, and minimizer-attainment issues, stated explicitly below. Fix a latent cardinalityK, and writeZ= [K]. LetQ,R,Gbe classes of encodersq(z|x,y), routers r(z|x), and generatorsg(y|x,z), respectively. Forq∈Q, define the joint law Q q (x,y,z) =μ(x)p(y|x)q(z|x,y). UnderQ q , define the encoder-induced router q(z|x) := X y p(y|x)q(z|x,y), and the encoder-induced branch conditional, wheneverq(z|x)>0, by p q (y|x,z) := p(y|x)q(z|x,y) q(z|x) . The conditionalp q (y|x,z)may be defined arbitrarily on zero-mass branches; all KL and expectation terms below weight such branches byq(z|x). LetI q (Y;Z|X)denote conditional mutual information underQ q . 20 2%6%10%20%50%100% Base conv. repeated-division via-binary decomposition Grid paths right-first down-first alternating Linear eqs. subtract-then-divide divide-then-subtract inverse-ops Summation left-to-right right-to-left pairwise Addition right-to-left-carry left-to-right-partials rounding-decomposition Sorting bubble-sort selection-sort insertion-sort merge-sort heap-sort Figure 9:Task-faceted posterior latent dynamics.Each row shows one benchmark task for the selected multi-task token-weighting run, and columns show checkpoints over training. Points are posterior mean latents projected with task-local PCA and colored by reference strategy. The view complements Figure 7 by showing that strategy organization emerges within each task family. LetS ⋆ =σ(X,Y)∈[K]denote the reference strategy. This hard-strategy assumption says that, once the completed trace is observed, the strategy is recognizable as an attribute of(X,Y). Letq ⋆ be the hard reference encoder q ⋆ (z|x,y) :=1z=σ(x,y). We assumeq ⋆ ∈Qwhen stating recovery theorems. Because latent labels are arbitrary, recovery can only be defined up to relabeling. The next definition separates relabelings that may vary with the input from a single label system shared across inputs. Definition 1: Weak and strong recovery An encoderq∈Qweakly recoversS ⋆ if, forμ-almost everyx, there exists a permutationπ x of [K]such that Z=π x (S ⋆ )Q q -almost surely conditional onX=x. It strongly recoversS ⋆ if there exists one global permutationπof[K]such thatZ=π(S ⋆ ) Q q -almost surely. Null labels are ignored; any bijection between positive-probability labels may be extended arbitrarily to a permutation of[K]. We measure approximate recovery by the corresponding variation-of-information distances under Q q : d w (q,S ⋆ ) :=H q (Z|X,S ⋆ ) +H q (S ⋆ |X,Z), and d s (q,S ⋆ ) :=H q (Z|S ⋆ ) +H q (S ⋆ |Z). 21 10.10.01 β 0.0 0.2 0.4 0.6 0.8 1.0 Fidelity-Alignment Constant β 10.10.01 β Linear β warmup ELBO baselineglobal scaletoken weightingglobal + tokentoken inverse Figure 10:Method success acrossβsettings.Plots show the product of Distributional Fidelity and Strategy Alignment across different KL weightsβfor the multi-task randomly initialized diagnostic grid. Standard ELBO-style baselines remain weak in fidelity-alignment acrossβ, while model- directed objectives succeed robustly onceβis small enough to enable informative latents. 10 −3 10 −2 10 −1 10 0 10 1 Base-Normalized Reconstruction 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Strategy-Alignment ELBO baseline global scale token weighting global + token token inverse ELBO baseline global scale token weighting global + token token inverse 0.920.940.960.981.001.02 Distributional Fidelity 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Strategy-Alignment ELBO baseline global scale token weighting global + token token inverse ELBO baseline global scale token weighting global + token token inverse Figure 11:Endpoint metric associations in single-task diagnostic runs. Left:base-relative reconstruction versus Strategy Alignment. There is a general positive association between recon- struction measured in base model units and strategy alignment; however, unlike the token-weighted reconstruction shown in the main text, the association is weaker for the pure token-weighted variant. This is likely caused by too little reconstruction pressure on non high-surprisal tokens, which causes some loss on these tokens. This motivates settingα >0in the global+token variant to preserve some uniform reconstruction pressure.Right:Distributional Fidelity versus Strategy Alignment. Larger outlined points show method centroids and smaller translucent points show individual runs; these diagnostics illustrate that it is possible to have high distributional fidelity without strategy alignment. In general distributional fidelity is easy, especially because the factorization is initialized near the pretrained model’s response distribution, but strategy alignment is harder and requires the right objective design. The weak distance conditions onX, so it compares strategy semantics separately within each input. The strong distance does not condition onX, so it also penalizes input-dependent relabeling. The following lemma verifies that these distances are exactly the right zero-error notions for the two definitions. 22 10 −1 10 0 10 1 0.75 0.80 0.85 0.90 0.95 1.00 Qwen2.5 0.5B Strategy-Alignment  θ = 1 Baseline reconstruction 10 −1 10 0 0.75 0.80 0.85 0.90 0.95 1.00  θ = 1 Token-weighted reconstruction 0.80.91.0 0.75 0.80 0.85 0.90 0.95 1.00 Distributional Fidelity 10 −1 10 1 10 3 10 5 0.75 0.80 0.85 0.90 0.95 1.00 Prior KL 10 −1 10 0 10 1 Baseline recon. 0.75 0.80 0.85 0.90 0.95 1.00 Qwen2.5 1.5B Strategy-Alignment  θ = 1 10 −1 10 0 Token recon. 0.75 0.80 0.85 0.90 0.95 1.00  θ = 1 0.80.91.0 Fidelity 0.75 0.80 0.85 0.90 0.95 1.00 10 −1 10 1 10 3 Prior KL 0.75 0.80 0.85 0.90 0.95 1.00 β-ELBO global scaletoken weightingglobal + tokentoken inverse Figure 12:Phase-space training dynamics of pretrained Qwen model factorization.Phase- space dynamics diagrams of Qwen2.5 0.5B and 1.5B multi-task runs are shown across base-relative reconstruction, token-weighted reconstruction, Distributional Fidelity, and Prior KL, each plotted against Strategy Alignment. Trajectories show how each method moves through these diagnostic phase spaces over training, from initialization to the final checkpoint. Lemma 1: Recovery distances For everyq∈Q: 1.d w (q,S ⋆ ) = 0if and only ifqweakly recoversS ⋆ . 2.d s (q,S ⋆ ) = 0if and only ifqstrongly recoversS ⋆ . 3.d w (q,S ⋆ )≤d s (q,S ⋆ ). Proof.For the weak statement,d w (q,S ⋆ ) = 0holds if and only if H q (Z|X,S ⋆ ) = 0, H q (S ⋆ |X,Z) = 0. Thus, forμ-almost everyx,Zis a deterministic function ofS ⋆ underQ q (· |X=x), andS ⋆ is a deterministic function ofZunder the same conditional law. The two variables therefore induce the same partition of positive-probability labels at that input, up to a bijection between the labels that occur. Since both label spaces are subsets of[K], the bijection extends to a permutationπ x . This is weak recovery. The converse is immediate becauseZ=π x (S ⋆ )makes the two conditional entropies vanish. The strong statement is the same argument without conditioning onX. The conditiond s (q,S ⋆ ) = 0 is equivalent toZandS ⋆ determining each other as global random variables underQ q , which is strong recovery up to null labels. Finally, conditioning reduces entropy: H q (Z|X,S ⋆ )≤H q (Z|S ⋆ ), H q (S ⋆ |X,Z)≤H q (S ⋆ |Z). Adding these inequalities givesd w (q,S ⋆ )≤d s (q,S ⋆ ). Thus any strong recovery guarantee also implies weak recovery, but not conversely. This distinction becomes important when different inputs can reuse the same latent labels with different meanings. 23 F.2 ELBO Decomposition with Router-Generator Complexity We study the population complexity-penalized beta-ELBO, withβ≥0andλ≥0, L β,λ (q,r,g) :=E Q q [−logg(Y|X,Z)] +βE X,Y KL(q(·|X,Y)∥r(·|X)) +λC(r,g). The functionalC(r,g)∈[0,∞]is a theoretical proxy for architecture, parameterization, and opti- mization biases that favor simple router-generator factorizations. We assume it is invariant under global relabeling of latent states. All KL terms use the usual extended-real convention. Define DecGap(q,g) :=E X,Z KL(p q (·|X,Z)∥g(·|X,Z)), and PriorGap(q,r) :=E X KL(q(·|X)∥r(·|X)). The following lemma decomposes the variational objective into four terms: the irreducible conditional entropyH(Y|X), an information term involvingI q (Y;Z|X), the router and decoder KL gaps, and the complexity penaltyC(r,g). This form makes clear how the objective trades off information in the latent variable against the fit and complexity of the router-generator factorization. Lemma 2: ELBO information-complexity decomposition For every(q,r,g), L β,λ (q,r,g) =H(Y|X) + (β−1)I q (Y;Z|X) + DecGap(q,g) +βPriorGap(q,r) +λC(r,g). Proof.First decompose the reconstruction term. UnderQ q , E[−logg(Y|X,Z)] =E X,Z X y p q (y|X,Z)(−logg(y|X,Z)). For each(x,z), cross-entropy equals entropy plus KL: X y p q (y|x,z)(−logg(y|x,z)) =H(p q (·|x,z)) + KL(p q (·|x,z)∥g(·|x,z)). Averaging gives E[−logg(Y|X,Z)] =H q (Y|X,Z) + DecGap(q,g). Because the(X,Y)marginal ofQ q isμ(x)p(y|x), H q (Y|X) =H(Y|X). Thus H q (Y|X,Z) =H(Y|X)−I q (Y;Z|X), and E[−logg(Y|X,Z)] =H(Y|X)−I q (Y;Z|X) + DecGap(q,g). Next decompose the KL-to-prior term: E X,Y KL(q(·|X,Y)∥r(·|X)) =E Q q log q(Z|X,Y) r(Z|X) . Insertq(Z|X): log q(Z|X,Y) r(Z|X) = log q(Z|X,Y) q(Z|X) + log q(Z|X) r(Z|X) . The first expectation isI q (Y;Z|X), and the second isPriorGap(q,r). Therefore E X,Y KL(q(·|X,Y)∥r(·|X)) =I q (Y;Z|X) + PriorGap(q,r). Combining the reconstruction decomposition, the KL decomposition, and the complexity term gives the identity. The coefficient ofI q (Y;Z|X)isβ−1. Therefore the objective is information-seeking only when β <1. Atβ= 1, selection can still occur through the profiled cost below, but the ELBO no longer rewards conditional information after the variational gaps have been profiled out. Forβ >1, the objective penalizes information inZ. 24 F.3 Profiled Router-Generator Objective For a fixed encoderq, define the profiled router-generator cost A β,λ (q) :=inf r∈R, g∈G [DecGap(q,g) +βPriorGap(q,r) +λC(r,g)]∈[0,∞]. Also define the profiled population loss L β,λ (q) :=inf r∈R, g∈G L β,λ (q,r,g). Profiling asks how good an encoder can look after choosing the best available router and generator for it. The next proposition rewrites that profiled loss as an encoder-only score. Proposition 1: Profiled ELBO score For everyq∈Q, L β,λ (q) =H(Y|X)−(1−β)I q (Y;Z|X) +A β,λ (q). Ifβ <1, define J prof β,λ (q) :=I q (Y;Z|X)− A β,λ (q) 1−β , Then L β,λ (q) =H(Y|X)−(1−β)J prof β,λ (q). Proof. Take the infimum overr∈Randg∈Gin Lemma 2. The termsH(Y|X)andI q (Y;Z|X) do not depend onrorg, and the remaining infimum is exactlyA β,λ (q). The final identity is a rearrangement whenβ <1. Thus the full expressive ELBO selects encoders by an information-minus-profiled approximation- complexity score. The quantityA β,λ (q), which involves the router-generator complexityC(r,g), measures whether the encoder-induced routing law and branch conditionals can be approximated with low profiled cost by a simple router-generator pair. F.4 Metric Recovery Margins Fix a notion of recoveryρ∈w,sandδ >0. Define the bad encoder set B ρ (δ) :=q∈Q:d ρ (q,S ⋆ )≥δ. Forβ <1andA β,λ (q ⋆ )<∞, define the profiled metric recovery margin ∆ ρ,prof β,λ (δ) :=J prof β,λ (q ⋆ )−sup q∈B ρ (δ) J prof β,λ (q), with the convention that the supremum of an empty set is−∞. Equivalently, ∆ ρ,prof β,λ (δ) =inf q∈B ρ (δ) I ⋆ −I(q) + A β,λ (q)−A β,λ (q ⋆ ) 1−β , whereI(q) :=I q (Y;Z|X)andI ⋆ :=I q ⋆ (Y;Z|X). The following theorem states the population strategy-structure recovery criterion. When the profiled margin at levelδis positive, every population ELBO minimizer must recover the reference strategy up to recovery distanceδ. 25 Theorem 1: Population strategy-structure recovery for the expressive ELBO Assumeβ <1,q ⋆ ∈Q,A β,λ (q ⋆ )<∞, and∆ ρ,prof β,λ (δ)>0. If a profiled minimizer exists and bq∈arg min q∈Q L β,λ (q), then d ρ (bq,S ⋆ )< δ. Consequently, if a full ELBO minimizer exists and (bq,br,bg)∈arg min q∈Q, r∈R, g∈G L β,λ (q,r,g), thend ρ (bq,S ⋆ )< δ. Proof.By Proposition 1, minimizingL β,λ is equivalent to maximizingJ prof β,λ whenβ <1. Hence any profiled population minimizerbqsatisfies J prof β,λ (bq)≥J prof β,λ (q ⋆ ). Ifd ρ (bq,S ⋆ )≥δ, thenbq∈B ρ (δ), so J prof β,λ (bq)≤sup q∈B ρ (δ) J prof β,λ (q) =J prof β,λ (q ⋆ )−∆ ρ,prof β,λ (δ)< J prof β,λ (q ⋆ ), contradicting optimality. Therefored ρ (bq,S ⋆ )< δ. For the final claim, let(bq,br,bg)be a full ELBO minimizer and let V:=inf q∈Q,r∈R,g∈G L β,λ (q,r,g) =L β,λ (bq,br,bg). For everyq,V≤ L β,λ (q), soV≤inf q L β,λ (q). Conversely, inf q L β,λ (q)≤L β,λ (bq)≤L β,λ (bq,br,bg) =V. Thusbqminimizes the profiled loss overQ, and the first part applies. This theorem reduces population recovery to a score-separation condition. It states that semantic recovery of the reference strategy is controlled by the profiled-score gap∆ ρ,prof β,λ (δ) : if this gap is positive, then optimizing the complexity-penalized variational objective selects only encoders inside the desiredδ-recovery neighborhood. Remark 1: Sufficient conditions for minimizer existence The above theorem is conditional on minimizers existing. One sufficient route, in these finite ambient spaces, is thatQ,R,Gare compact subsets of the relevant probability simplexes and Cis lower semicontinuous, with support restrictions or closure conventions that make the KL terms lower semicontinuous and avoid undefined∞−∞expressions. The substantive content of this theorem is the identification of the profiled score that must separate the target encoder from misaligned encoders. The margin definition is sometimes hard to check directly because it ranges over all bad encoders. The following corollary gives a more interpretable sufficient condition in terms of an information advantage and a profiled-cost advantage. It says that if encoders that fail to recover the strategy don’t have much more conditional information than the target, and they pay a large enough profiled cost penalty, then the selected encoder will recover the strategy. 26 Corollary 1: A sufficient information-complexity separation condition Assume the hypotheses of Theorem 1 except for the positivity of the margin. Suppose there exist γ I ∈Randγ A >0such that, for everyq∈B ρ (δ), I q (Y;Z|X)−I q ⋆ (Y;Z|X)≤γ I , and A β,λ (q)−A β,λ (q ⋆ )≥γ A . Ifγ A >(1−β)γ I , then ∆ ρ,prof β,λ (δ)≥ γ A 1−β −γ I >0. Proof.For every badq, J prof β,λ (q ⋆ )−J prof β,λ (q) =I q ⋆ (Y;Z|X)−I q (Y;Z|X) + A β,λ (q)−A β,λ (q ⋆ ) 1−β ≥−γ I + γ A 1−β . Taking the infimum overB ρ (δ)proves the claim. This corollary makes the tradeoff explicit: a bad encoder may have more conditional information, but only up toγ I . The target still wins if the bad encoder pays a large enough additional router-generator cost. We now return to the distinction between weak and strong recovery. Strong recovery requires more than recovering the correct partition separately for each input; it also requires label alignmentacross inputs. The next proposition shows that input-dependent relabelings can preserve weak recovery and conditional information exactly, while still preventing strong recovery unless the profiled cost favors a globally consistent label system. Proposition 2: Input-dependent relabelings and strong recovery For a family of permutationsπ=π x :x∈X, define the hard encoder q π (z|x,y) :=1z=π x (σ(x,y)). Assumeβ <1. Ifq π ∈Q, thend w (q π ,S ⋆ ) = 0and I q π (Y;Z|X) =I q ⋆ (Y;Z|X). If no single global permutation agrees withπ x on the positive-probability support ofS ⋆ |X=x forμ-almost everyx, thend s (q π ,S ⋆ )>0. In that case,q ⋆ beats this input-dependent relabeling in the profiled score if and only if A β,λ (q π )> A β,λ (q ⋆ ). Consequently, strong recovery requires a positive profiled-cost gap against such non-globally aligned relabelings; if some suchq π hasA β,λ (q π ) =A β,λ (q ⋆ ), then the strong margin cannot be positive for everyδ≤d s (q π ,S ⋆ ). Proof.Conditional onX=x, the mapS ⋆ 7→π x (S ⋆ )is a bijection on labels, so weak recovery holds and I q π (Y;Z|X) =H q π (Z|X) =H(S ⋆ |X) =I q ⋆ (Y;Z|X). If no global permutation agrees with the familyπ x on the positive supports, thenq π does not strongly recoverS ⋆ ; by Lemma 1,d s (q π ,S ⋆ )>0. Since the conditional information terms are equal, the profiled-score comparison betweenq ⋆ andq π is determined exactly by the profiled costs. If the profiled costs tie, the two scores tie, so no positive strong margin can separateq ⋆ fromq π at any scale includingq π in the bad set. This shows why strong recovery requires more than predictive information. The model class or complexity proxy must also prefer a globally aligned label system over equally informative local relabelings. 27 The recovery theorem above gives approximate recovery at a chosen radius with respect to the corresponding recovery distance. When the candidate class has a positive distance gap around the target equivalence class, that approximate statement upgrades to exact recovery. Corollary 2: Exact recovery from metric recovery Let∼ ρ denote weak equivalence whenρ= wand strong equivalence whenρ= s. Suppose m ρ := inf q̸∼ ρ q ⋆ d ρ (q,S ⋆ )>0. If Theorem 1 holds for someδ≤m ρ , then every population ELBO minimizer exactly recovers S ⋆ in the corresponding weak or strong sense. Proof.The theorem givesd ρ (bq,S ⋆ )< δ≤m ρ . By definition ofm ρ , every non-recovering encoder has distance at leastm ρ . Thereforebq∼ ρ q ⋆ . The separation assumptionm ρ >0is natural for finite or otherwise discrete encoder classes. Example 1: Positive margin from semantic branch sharing Suppose the target strategy yields branch conditionals that can be represented by shared low- complexity generator templates, while every encoder at recovery scaleδeither has at mostγ I more conditional information or requires at leastγ A additional profiled router-generator cost. Ifγ A >(1−β)γ I , Corollary 1 gives a positive recovery margin. This is the intended regime: semantic strategies win not merely because they are informative, but because their factorization is cheaper to realize. Example 2: Failure from a superficial high-information attribute Ifλ= 0andR,Gare expressive enough thatA β,λ (q) = 0for all candidate deterministic encoders, then the profiled score is justI q (Y;Z|X). A length bucket, formatting marker, or balanced nuisance partition with larger conditional information thanS ⋆ will beat the semantic strategy. The theorem then correctly predicts non-recovery. Example 3: Failure of strong recovery under free local relabeling If the available router-generator class and complexity proxy are invariant not only to global relabeling but also to input-dependent relabeling, then the encodersq π in Proposition 2 can tieq ⋆ in profiled score. Weak recovery may still hold, but the strong margin cannot be positive unless the objective assigns positive cost to such local label misalignment. F.5 Marginal Fidelity and the Small-Complexity Limit For a router-generator pair, define the marginal conditional law m r,g (y|x) := X z r(z|x)g(y|x,z). The next proposition connects the profiled variational gaps to the observable marginal fit of this router-generator pair. It clarifies how much fidelity can be lost when the profiled optimizer trades exact factorization for lower complexity. 28 Proposition 3: Variational gaps control marginal fidelity For every(q,r,g), E X KL(p(·|X)∥m r,g (·|X))≤PriorGap(q,r) + DecGap(q,g). Proof.Fixx∈X. Define two joint distributions onY ×Z: P x (y,z) :=p(y|x)q(z|x,y), M x (y,z) :=r(z|x)g(y|x,z). The distributionP x is the joint law over(Y,Z)induced by first samplingY∼p(· |x)and then Z∼q(·|x,Y). By Bayes’ rule, it can also be written as P x (y,z) =q(z|x)p q (y|x,z), with the usual convention that zero-mass branches do not affect the KL terms. Now compare theY-marginals of these two joint distributions. ForP x , X z P x (y,z) = X z p(y|x)q(z|x,y) =p(y|x). ForM x , X z M x (y,z) = X z r(z|x)g(y|x,z) =m r,g (y|x). Thusp(· |x)andm r,g (· |x)are obtained fromP x andM x , respectively, by applying the same projection map(y,z)7→y. By data processing for KL divergence, marginalizing outZcannot increase KL. Therefore KL(p(·|x)∥m r,g (·|x))≤KL(P x ∥M x ). It remains to expand the joint KL on the right. Using the factorizationsP x (y,z) =q(z|x)p q (y|x,z) andM x (y,z) =r(z|x)g(y|x,z), KL(P x ∥M x ) = X z,y q(z|x)p q (y|x,z) log q(z|x)p q (y|x,z) r(z|x)g(y|x,z) = X z q(z|x) log q(z|x) r(z|x) + X z q(z|x) X y p q (y|x,z) log p q (y|x,z) g(y|x,z) = KL(q(·|x)∥r(·|x)) +E Z∼q(·|x) KL(p q (·|x,Z)∥g(·|x,Z)). Combining the previous two displays gives, for each fixedx, KL(p(·|x)∥m r,g (·|x))≤KL(q(·|x)∥r(·|x)) +E Z∼q(·|x) KL(p q (·|x,Z)∥g(·|x,Z)). Finally, average both sides overX∼μ. The two terms on the right becomePriorGap(q,r)and DecGap(q,g), respectively, which proves E X KL(p(·|X)∥m r,g (·|X))≤PriorGap(q,r) + DecGap(q,g). This proposition makes the fidelity tradeoff explicit. For finiteλ, the profiled optimizer can prefer a simpler(r,g)with nonzero variational gaps, and therefore need not exactly factorizep(y|x). The next result records the high-fidelity limit. 29 Proposition 4: Small-λfidelity for a fixed encoder Fixq∈Qandβ >0. Suppose the exact induced pair r q (z|x) :=q(z|x), g q (y|x,z) :=p q (y|x,z) belongs toR×G, andC(r q ,g q )<∞. If(r λ ,g λ )minimizesA β,λ (q), for eachλ >0, then DecGap(q,g λ )→0,PriorGap(q,r λ )→0 asλ↓0. Consequently, E X KL(p(·|X)∥m r λ ,g λ (·|X))→0. Proof.Fixq∈Q. By assumption, the exact induced router-generator pair r q (z|x) :=q(z|x), g q (y|x,z) :=p q (y|x,z) belongs toR×Gand has finite complexity. Forq’s induced router-generator pair(r q ,g q ), the prior gap and decoder gap are both zero. Since (r λ ,g λ )minimizesA β,λ (q), its objective value is no larger than the value achieved by the exact induced pair. Therefore DecGap(q,g λ ) +βPriorGap(q,r λ ) +λC(r λ ,g λ ) ≤DecGap(q,g q ) +βPriorGap(q,r q ) +λC(r q ,g q ) =λC(r q ,g q ). All three terms on the left are nonnegative. Hence DecGap(q,g λ )≤λC(r q ,g q ), and, sinceβ >0, PriorGap(q,r λ )≤ λC(r q ,g q ) β . BecauseC(r q ,g q )<∞, both upper bounds go to zero asλ↓0. Thus DecGap(q,g λ )→0,PriorGap(q,r λ )→0. Finally, Proposition 3 gives E X KL(p(·|X)∥m r λ ,g λ (·|X))≤PriorGap(q,r λ ) + DecGap(q,g λ ). The right-hand side tends to zero, so the marginal KL also tends to zero. The limitλ↓0restores high-fidelity factorization under exact realizability. It also weakens the selection role of complexity: the profiled encoder score approaches maximum conditional information, which can favor superficial high-information partitions unless complexity still breaks the relevant near-ties. F.6 Empirical Recovery The population theorem gives a profiled-score margin∆ ρ,prof β,λ (δ). Empirical recovery only requires one additional ingredient: the empirical profiled ELBO must approximate the population profiled ELBO at a scale smaller than the same margin measured in loss units. Since L β,λ (q) =H(Y|X)−(1−β)J prof β,λ (q), the loss separation corresponding to∆ ρ,prof β,λ (δ)is(1−β)∆ ρ,prof β,λ (δ). Equivalently, for every bad encoderq∈B ρ (δ), L β,λ (q)−L β,λ (q ⋆ ) =A β,λ (q)−A β,λ (q ⋆ ) −(1−β)I q (Y;Z|X)−I q ⋆ (Y;Z|X). Thus empirical recovery is possible when finite samples preserve this information-complexity loss gap. 30 Let(X i ,Y i ) n i=1 be i.i.d. samples fromμ(x)p(y|x). We use the analytic expectation overz∼q(·| X i ,Y i )and define b L β,λ,n (q,r,g) := 1 n n X i=1 X z q(z|X i ,Y i ) −logg(Y i |X i ,z) +βlog q(z|X i ,Y i ) r(z|X i ) +λC(r,g), with the usual extended-real conventions for the logarithmic terms. The empirical profiled loss is b L β,λ,n (q) :=inf r∈R, g∈G b L β,λ,n (q,r,g). Theorem 2: Empirical recovery for the profiled ELBO Assumeβ <1,q ⋆ ∈Q, L β,λ (q ⋆ )<∞, and ∆ ρ,prof β,λ (δ)>0. Suppose an empirical profiled minimizer exists, bq n ∈arg min q∈Q b L β,λ,n (q), and the empirical profiled objective satisfies a uniform convergence bound: sup q∈Q b L β,λ,n (q)−L β,λ (q) < (1−β)∆ ρ,prof β,λ (δ) 2 . Then every empirical profiled minimizer recoversS ⋆ at scaleδ: d ρ (bq n ,S ⋆ )< δ. Proof.Let ε n := sup q∈Q b L β,λ,n (q)−L β,λ (q) . By empirical optimality, b L β,λ,n (bq n )≤ b L β,λ,n (q ⋆ ). Therefore L β,λ (bq n )≤ b L β,λ,n (bq n ) +ε n ≤ b L β,λ,n (q ⋆ ) +ε n ≤ L β,λ (q ⋆ ) + 2ε n < L β,λ (q ⋆ ) + (1−β)∆ ρ,prof β,λ (δ). Ifd ρ (bq n ,S ⋆ )≥δ, thenbq n ∈B ρ (δ). Since L β,λ (q) =H(Y|X)−(1−β)J prof β,λ (q), the definition of∆ ρ,prof β,λ (δ)implies L β,λ (bq n )≥L β,λ (q ⋆ ) + (1−β)∆ ρ,prof β,λ (δ), a contradiction. Henced ρ (bq n ,S ⋆ )< δ. This theorem is deterministic conditional on uniform convergence. The remaining question is how largenmust be for sup q∈Q b L β,λ,n (q)−L β,λ (q) < (1−β)∆ ρ,prof β,λ (δ) 2 31 to hold with high probability. This is a statistical question about the profiled log-loss class. In what follows, we give two concrete sufficient conditions for this uniform convergence. The first corollary assumes a finite encoder class and a fixed concentration bound for each encoder. The second derives a concrete condition under which such concentration holds by assuming a support floor for the decoder and router probabilities such that the log losses are bounded. Corollary 3: Finite-class sample complexity Suppose|Q|=N <∞and for every fixedq∈Q, the empirical profiled loss b L β,λ,n (q)satisfies P b L β,λ,n (q)−L β,λ (q) ≥t ≤2 exp − nt 2 2σ 2 . If n≥ 8σ 2 (1−β) 2 (∆ ρ,prof β,λ (δ)) 2 log 2N η , then with probability at least1−η, every empirical profiled-ELBO minimizer satisfies d ρ (bq n ,S ⋆ )< δ. Proof.For each fixedq∈Q, apply the assumed concentration inequality. A union bound over theN encoders gives P sup q∈Q b L β,λ,n (q)−L β,λ (q) ≥t ≤2Nexp − nt 2 2σ 2 . Set t= (1−β)∆ ρ,prof β,λ (δ) 2 . The displayed lower bound onnmakes the right-hand side at mostη. On the complementary event, sup q∈Q b L β,λ,n (q)−L β,λ (q) < (1−β)∆ ρ,prof β,λ (δ) 2 . The empirical recovery theorem then impliesd ρ (bq n ,S ⋆ )< δfor every empirical profiled-ELBO minimizer. FiniteQalone does not make the preceding concentration automatic. The log terms −logg(y|x,z),log q(z|x,y) r(z|x) can be unbounded or infinite wheng(y|x,z)orr(z|x)is near zero. A simple sufficient condition is to assume finiteQ,R,G, finiteC(r,g), and a support floorτ∈(0,1]such that r(z|x)≥τ, g(y|x,z)≥τ for all admissibler,gand allx,y,z. This is stated in the following corollary. 32 Corollary 4: Finite classes with bounded log loss SupposeQ,R,Gare finite,C(r,g)<∞for all(r,g)∈R×G, and there isτ∈(0,1]such that r(z|x)≥τ, g(y|x,z)≥τ for allr∈R,g∈G, and allx,y,z. If n≥ 2(1 +β) 2 log 2 (1/τ) (1−β) 2 (∆ ρ,prof β,λ (δ)) 2 log 2|Q||R||G| η , then with probability at least1−η, every empirical profiled-ELBO minimizer satisfies d ρ (bq n ,S ⋆ )< δ. Proof.For a fixed triple(q,r,g), the data-dependent part of the single-sample loss is X z q(z|x,y) −logg(y|x,z) +βlog q(z|x,y) r(z|x) . The support floor gives−logg(y|x,z)≤log(1/τ), and X z q(z|x,y) log q(z|x,y) r(z|x) = KL(q(·|x,y)∥r(·|x))≤log 1 τ . Thus the data-dependent part is bounded between0and(1 +β) log(1/τ). The complexity term λC(r,g)is deterministic and cancels in empirical-population deviations. Hoeffding’s inequality and a union bound overQ×R×Gimply P sup q,r,g b L β,λ,n (q,r,g)−L β,λ (q,r,g) ≥t ≤2|Q||R||G|exp − 2nt 2 (1 +β) 2 log 2 (1/τ) . Set t= (1−β)∆ ρ,prof β,λ (δ) 2 . The displayed lower bound onnmakes the right-hand side at mostη. On the complementary event, profiling cannot increase the deviation: for everyq, inf r,g b L β,λ,n (q,r,g)−inf r,g L β,λ (q,r,g) ≤sup r,g b L β,λ,n (q,r,g)−L β,λ (q,r,g) . Therefore sup q∈Q b L β,λ,n (q)−L β,λ (q) < (1−β)∆ ρ,prof β,λ (δ) 2 . The empirical recovery theorem then givesd ρ (bq n ,S ⋆ )< δfor every empirical profiled-ELBO minimizer. Support floors are only one transparent sufficient condition. Other standard routes are to clip predictive probabilities, assume direct concentration of the profiled losses, prove a high-probability envelope or sub-exponential tail bound for the relevant log losses, or use entropy, bracketing, or Rademacher control for the likelihood class. The recovery argument itself only needs uniform accuracy at scale (1−β)∆ ρ,prof β,λ (δ); support or tail assumptions are one way to obtain that event. 33

Instant research discovery