Paper deep dive
What Pretraining and Midtraining Make Learnable from Rewards?
Chiwun Yang, Xiaoyu Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/1/2026, 8:18:49 AM
Summary
This paper investigates how pretraining and midtraining (source prediction) supply the necessary information and computation to make reward adaptation effective in language models. The authors argue that terminal rewards alone are ambiguous and cannot distinguish between different underlying mechanisms that yield the same training outcomes. By introducing task-independent source observations (such as first-operation supervision or memory retrieval), models acquire executable computations (execution or retrieval) before reward-based fine-tuning. Experiments on Qwen2.5 checkpoints demonstrate that this two-stage approach significantly improves success rates in sequential state computation and contextual memory tasks compared to models trained with rewards alone or private-random sources.
Entities (9)
Relation Signals (7)
Sequential models → achievessuccessrate → 82.61%
confidence 95% · Sequential models trained with correct source and first-operation supervision reach 82.61% success
Qwen2.5 → usedin → experiments
confidence 95% · Experiments with pretrained Qwen2.5 checkpoints test this division of labor.
Sequential models → outperforms → Private-random source control
confidence 93% · Sequential models trained with correct source and first-operation supervision reach 82.61% success, versus 44.15% for a private-random source control.
Source prediction → enables → Reward adaptation
confidence 92% · We study how pretraining and midtraining supply the information and computation that make reward adaptation effective.
Memory replay → preserves → Retrieval
confidence 90% · Memory replay preserves retrieval during reward adaptation
Source prediction → acquires → Execution
confidence 88% · the sequential learner acquires execution and selects the task binding
Source prediction → acquires → Retrieval
confidence 88% · the contextual learner acquires retrieval and an input-dependent task rule
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A reward can identify a correct answer while leaving the computation needed for new inputs undetermined. We study how pretraining and midtraining supply the information and computation that make reward adaptation effective. In sequential state computation and contextual memory, we characterize mechanisms that agree on every training reward yet demand different held-out answers. Task-independent source observations resolve this ambiguity. We construct finite sampled Adam paths from specified random initializations through source prediction and reward adaptation in the same parameters, proving how prediction acquires execution or retrieval and rewards learn their task-specific use. Experiments with pretrained Qwen2.5 checkpoints test this division of labor. Across eight worlds, Sequential models trained with correct source and first-operation supervision reach 82.61% success, versus 44.15% for a private-random source control. Memory replay preserves retrieval during reward adaptation, and an independent eight-world confirmation achieves 75.32% task success versus 49.86% after matched alternative-retrieval training. GSM8K and HotpotQA separate accuracy at reward entry, subsequent gain and final performance. Together, these results connect information acquisition, executable computation and reward-guided task learning.
Tags
Links
- Source: https://arxiv.org/abs/2609.38446v1
- Canonical: https://arxiv.org/abs/2609.38446v1
Trouble viewing inline? Open PDF directly →
Full Text
550,573 characters extracted from source content.
Expand or collapse full text
What Pretraining and Midtraining Make Learnable from Rewards? Chiwun Yang †thanks: City University of Hong Kong. christiannyang37@gmail.com. Xiaoyu Li †thanks: University of New South Wales. 7.xiaoyu.li@gmail.com. Abstract A reward can identify a correct answer while leaving the computation needed for new inputs undetermined. We study how task-independent prediction supplies the missing information and turns it into computation that rewards can use. In sequential state computation and contextual memory, we characterize exactly which mechanism–rule worlds agree on every training reward yet demand opposite held-out answers. Source observations expose the mechanism before its effects cancel in training rewards. Finite sampled Adam realizes this division of labor from random initialization in the same parameters: the sequential learner acquires execution and selects the task binding in two reward updates, reaching held-out success at least 7/87/8; the contextual learner acquires retrieval and an input-dependent task rule within explicit finite budgets, reaching 19/2019/20, while randomly initialized frozen attention stays near chance. Across eight worlds, source midtraining from pretrained Qwen2.5 checkpoints supplies executable state predictions and Memory responses sensitive to stored values. With task-independent first-operation supervision, Sequential models reach 82.61% success against 44.15% for a private-random source control, with a 67.57-point task-selectivity advantage. Memory replay retains 92.21% original/value-flip pair accuracy during reward adaptation, compared with 0% under pure reward training. An independent eight-world Memory confirmation reaches 75.32% task success versus 49.86% after matched alternative-retrieval training. GSM8K and HotpotQA separate accuracy at reward entry, subsequent gain and final performance. The results connect three steps in learning from rewards: identifying a mechanism, acquiring its computation and learning its task-specific use. 1 Introduction Reinforcement learning can substantially improve reasoning in pretrained language models, as demonstrated by DeepSeek-R1 (DeepSeek-AI and others, 2025). What makes a model ready to learn from rewards? Earlier prediction stages shape downstream performance: domain- and task-adaptive training improve classification (Gururangan et al., 2020), and agent-native midtraining strengthens subsequent software-engineering training (Zeng et al., 2026). Learning-dynamics analyses connect optimization to generalization in transformers (Yang, 2025); prefix-learning theory gives a complementary account of adaptation through trainable context (Liang et al., 2025). To choose training data and interpret these gains, we need to understand what each stage contributes. Prediction may supply information absent from rewards, build a computation that uses it, or already teach part of the task. These possibilities lead to different explanations of the same final score. We ask which information and computations make a new task rule learnable from rewards. A terminal reward evaluates an answer without directly observing its intermediate mechanism. Consider a learner that composes unknown operations on a hidden state. Two transition rules can agree on every training reward yet demand opposite held-out answers: their difference cancels along training sequences and reappears on test sequences. More reward queries restricted to that support cannot distinguish them; observing individual transitions can. We study such source prediction during initial pretraining or midtraining of an existing model, followed by reward adaptation of the same parameters (Figure 2). A world pairs a reusable mechanism with a task rule; our theoretical source laws reveal the former while leaving the latter unspecified. Source prediction trains composition or retrieval; reward training then learns a task-specific use of that computation. Source observations s0→u1s1→u2s2s_0 u_1s_1 u_2s_2 (k1,v1)(k2,v2)(k_1,v_1) (k_2,v_2) Task rule hidden Reusable computation Compose: u2∘u1u_2 u_1 Retrieve: q=k2⇒v2q=k_2\; \;v_2 One shared checkpoint Reward A → task-A policy Reward B → task-B policy Source predictionHeld-out evaluationNew combinations or memoriesMeasure: information → acquired computation → task selection Figure 1: Source prediction prepares a shared computation for reward adaptation. States sis_i and operations uiu_i illustrate composition; keys kik_i, values viv_i and query q illustrate retrieval. Rewards select task-specific use. Figure 2: Measured Sequential parameter slice: task success (left) and source loss (right). Colors evaluate models in a common SVD plane; paths project actual checkpoints and need not share their scores. Star: shared start; dashed squares: source; solid circles: RL; diamonds: endpoints. White contours interpolate source loss. Evaluation and reconstruction checks: Sections 6.3 and 15. Exactly what rewards leave open. Prior theory explains how RL exploits rare task demonstrations in prediction data (Tsilivis et al., 2025); controlled experiments also show RL composing pretrained primitives into reusable strategies (Abdulsalam et al., 2026). We characterize what source observations must add when training rewards leave the mechanism ambiguous. In our Sequential family, a two-parameter family of transformations preserves all training rewards, while half reverse every held-out answer (Theorem 4.1). Local source observations distinguish these possibilities. Contextual memory gives a second exact characterization: after finitely many numerical source records, unobserved contexts impose an exact error floor that additional rewards cannot lower (Theorem 5.1). A finite learning path through the same parameters. From specified random initializations, we construct finite sampled Adam paths through both stages with high-probability guarantees. Sequential source prediction teaches a transformer to execute transitions on its own generated prefixes; two REINFORCE updates using 2BR2B_R terminal verifications then select the initial operation, giving held-out success at least 7/87/8 (Theorem 4.2). In contextual memory, source Adam acquires retrieval and reward Adam learns an input-dependent rule, reaching at least 19/2019/20 within explicit budgets; freezing randomly initialized query/key attention under the same schedule keeps success at most 1/641/64 above chance (Theorem 5.2). Widths, batches, step sizes and budgets are chosen without test information. Measuring acquisition and task-specific use. Our Qwen2.5 experiments apply source midtraining to pretrained checkpoints and measure acquisition, retention and task-specific use separately. Across eight matched worlds, correct-source models acquire Sequential execution and Memory retrieval; action-target Sequential training reaches 82.61% success versus 44.15% for a private-random source control (Figures 3 and 4). Source-checkpoint probes reveal the computation acquired before rewards; paired reward branches test its use for opposite tasks. Memory replay preserves retrieval; an independent eight-world confirmation reaches 75.32% task success versus 49.86% with alternative retrieval (Section A.2). An independent single-world comparison shows correct-source preparation enabling rapid reward learning from zero task success at reward entry (Figure 15). The measured parameter slice in Figure 2 complements these actual-checkpoint curves by displaying source loss and task success across a common plane. On GSM8K and HotpotQA, accuracy at reward entry, reward-stage gain and final accuracy reveal how preparation changes subsequent learning. Contributions. • Exact information characterizations: which mechanisms training rewards leave indistinguishable, and how source observations resolve the ambiguity (Sections 4 and 5). • Finite sampled-Adam paths that turn source observations into execution or attention and then into task learning, with a frozen-attention comparison isolating the role of acquired retrieval (Theorems 4.2 and 5.2). • Measurements that distinguish acquiring a computation, retaining it and learning its task-specific use in Qwen2.5, together with the contribution of each training stage to public-task accuracy (Section 6). Section 2 connects the results to prior work; Section 7 develops their implications. 2 Related Work We organize prior work around three questions: what information each training stage supplies, how optimization turns that information into computation, and how rewards use the acquired computation. 2.1 Information supplied by observations and feedback Identification with different observation channels. Learning an unknown mechanism depends on which queries and observations are permitted. Membership and equivalence queries support regular-language identification (Angluin, 1987); related methods learn weighted automata from queries and counterexamples (Weiss et al., 2019). Bounded-width branching programs and statistical-query learning of semiautomata provide further settings in which the observation model determines learnability (Ergün et al., 1995; Giapitzakis et al., 2025). Pairwise measurements with side information illustrate a complementary distinction: edge observations constrain relative labels, while additional vertex information helps recover the labels themselves (Foster et al., 2018). In attention inversion, Deng et al. (2023) study input recovery from supplied attention weights and outputs. The specified observations likewise determine which reconstruction problem the learner faces. Our information results identify exactly which transition and task transformations leave all training rewards unchanged, and which source observations distinguish their held-out predictions. The resulting obstruction concerns indistinguishable observations even with unlimited reward queries; it is different from an optimizer’s difficulty in finding an informative trajectory. What a verifier supplies. The feedback interface matters beyond a binary reward’s numerical range. Learning a chain-of-thought verifier online uses an expert label stream and has distinct soundness and completeness objectives (Balcan et al., 2026). Machine teaching can also choose among demonstrations and other feedback modalities (Larian et al., 2026). These settings supply different information from the terminal correctness feedback in our sequential task. In the contextual binary-action task, correctness together with the chosen action does reveal the binary label, so supervised adaptation is a valid alternative there. We account for this difference when comparing learning routes. Specifications, safety, and external structure. Reach–avoid control specifies dynamics, target sets, and constraints (Fisac et al., 2015). Shielding uses a supplied specification and model information to restrict actions, including under partial observability (Alshiekh et al., 2018; Carr et al., 2023); almost-sure reachability in partially observable Markov decision processes (POMDPs) studies the corresponding structural conditions (Junges et al., 2021). Automata-conditioned reinforcement learning likewise uses an explicit automaton interface (Yalcinkaya et al., 2025). These comparisons identify what is supplied to a learner or controller; our sequential result concerns task utility and public-coordinate violations within a specified finite grammar. Other reliability objectives require different observations and success events. Noninterference compares executions with different secret inputs (Volpano et al., 1996); selective classification couples conditional error with coverage (Geifman and El-Yaniv, 2017); randomized smoothing certifies predictions within a specified perturbation set (Cohen et al., 2019). Safe exploration also distinguishes zero-violation guarantees under suitable structural or startup information from bounds on accumulated constraint violation (Turchetta et al., 2016; Amani et al., 2021; Wei et al., 2024). Copyright-protection objectives can instead constrain optimization itself: Chu et al. (2024) formulate a softmax-regression method for avoiding protected outputs. These objectives each specify their own success event and verification requirement, distinct from terminal task success. 2.2 Learning across source and adaptation stages Intermediate training and task information. Supervised training on intermediate labeled-data tasks (STILTs) places a supervised intermediate task between language-model pretraining and downstream adaptation (Phang et al., 2018). Domain- and task-adaptive pretraining continue prediction on unlabeled text from a relevant distribution (Gururangan et al., 2020). Agent-native midtraining instead uses trajectories preserving agent workflows, including observations from executed tools and tests (Zeng et al., 2026). Their common use of an intermediate stage does not make their information interfaces identical: supervised examples may reveal the downstream choice, whereas our source laws leave the task binding or contextual decision rule unspecified. The public-task experiments use domain adaptation as an empirical comparison; the controlled tasks separate source mechanism information from downstream task information. From source computation to reward learning. Tsilivis et al. (2025, Section 4, Theorems 1–2 and Appendices D.3–D.4) analyze next-token prediction followed by reinforcement learning on mixtures of short and long demonstrations of a single parity task. Their autoregressive linear model is trained with finite source stochastic gradient descent and self-taught reasoner (STaR) rounds. Their finite learning theorem uses a reward that checks the complete chain of thought, and explains how post-training increases the probability of long, correct computations. In our sequential construction, source observations describe local transitions without revealing the downstream task binding, and feedback checks the terminal answer. This changes both the missing information and what the feedback stage must learn. Wei and Kim (2026, Sections 3.2–3.3 and 4.3–4.4) study a related world-model-to-reward sequence for graph backtracking. They establish a pretraining convergence result for a linear-softmax bigram model, then analyze post-training from a model assumed arbitrarily close to the world law. Their edge-state bigram analysis uses an outcome reward with a length penalty. Its gradient-flow and sign-policy-gradient-flow analysis compares outcome-reward learning with shortest-path supervised fine-tuning (SFT) through inference-time efficiency. Our sequential theorem follows finite sampled updates across both stages, from random initialization, and evaluates held-out operation words. Reusable modules and compositional curricula provide further accounts of what stages contribute (Kong et al., 2026; Rajaraman et al., 2026). Empirically, Abdulsalam et al. (2026) study primitive rewrite pretraining followed by final-answer reinforcement learning (RL) and trace how valid composed procedures develop. Their distinction between exposure to primitives and usable procedures is particularly relevant to our computation interventions. Staged training can also change where a computation occurs: Huang et al. (2026a) analyze a structured curriculum for internalizing explicit chain-of-thought steps. Our question is the acquisition and downstream use of reusable computation across the two stages. 2.3 Acquisition and reuse of representations Transferable prediction structure. Language-modeling performance can imply useful downstream features when the source and target tasks satisfy an appropriate relationship (Saunshi et al., 2020). Using linear attention and preconditioned projected proximal updates for sparse contextual bigrams, Ren et al. (2024) show that a correlated pretrained transition matrix supplies a signal that allows their learning algorithm to bypass its initial sample-intensive stage. Our contextual result instead derives the source-trained attention and head-factor distribution and follows it into learning a new reward-dependent rule with sampled Adam, including its two moment states. Another adaptation route trains an auxiliary prefix: Liang et al. (2025) establish convergence in a stylized neural-tangent-kernel setting and approximate ultra-long prefix attention using a compact parameterization. Representation reuse also depends on later updates: full fine-tuning can distort pretrained features and worsen out-of-distribution performance (Kumar et al., 2022). We therefore track both source acquisition and retention of the information needed during feedback. Representing and learning sequential computation. Graph algorithms, parallel computation, and automata shortcuts characterize what transformer architectures can express and how depth or intermediate computation changes their resources (Sanford et al., 2024a; Sanford et al., 2024b; Liu et al., 2022). Huang et al. (2025b) analyze gradient-based learning and length generalization in structured state-tracking tasks. These results motivate explicit operation and history interfaces. Our sequential source theorem additionally establishes execution on generated prefixes outside the source trajectories, which is what makes the learned local computation available during reward rollouts. Optimization, generalization and numerical resources. Yang (2025) connects transformer learning dynamics to generalization through a kernel approximation and separate optimization and statistical regimes. For binary-weight networks, Daliri et al. (2025) analyze width-dependent kernel behavior, convergence and the discrepancy from full-precision networks. These analyses make the model, training dynamics and resource scaling explicit. Our finite-training results track a further object: a source-acquired computation whose role changes when a new reward objective is introduced. Learning attention and memory. Margin-based token selection, induction-head dynamics, and latent causal structure provide different accounts of attention acquisition (Tarzanagh et al., 2023; Chen et al., 2024; Nichani et al., 2024a). Random-feature, sparse-parity, regular-language, mean-field, and Bayesian copy-head analyses study complementary representations and training regimes (Fu et al., 2023; Han and Ghoshdastidar, 2026; Huang et al., 2025a; Huan and Yuan, 2026; Liu and Herty, 2026; Lavie et al., 2026). Ke et al. (2025) identify asymmetric feature learning in attention-based time-series forecasting, including difficulties on sign-inconsistent inputs. Sparse-attention approximation asks a complementary question: Deng et al. (2024) analyze when retaining large attention entries approximates the full attention output. Normalization can change gradient scales at initialization (Xiong et al., 2020). In our contextual model, the relevant interaction occurs across stages: attention pooling changes correlation and signal scale, and the normalization family allows useful reward learning with different directions of attention-contrast change. Memory is not a single observation interface. Associative-memory analyses study parametric factual storage (Nichani et al., 2024b); contextual recall combines pretraining knowledge with implicit attribute inference after fine-tuning (Vasudeva et al., 2026). Statistical in-context learning and abstract-symbol reasoning address other forms of adaptation and generalization (Bai et al., 2023; Boix-Adsera et al., 2023). Titans changes writable neural memory, while Verifiable Memory trains an agent’s operations over long-term memory, active context, and episodic history using multiple verifier signals (Behrouz et al., 2024; Sun et al., 2026). At inference time, ParallelComp uses parallel chunk processing and KV-cache eviction to extend the context handled by a fixed language model (Xiong et al., 2025). This changes access to contextual information through compression. Our contextual theorem concerns retrieval and a context–cue rule in a supplied numerical interface, and follows their acquisition through both training stages. 2.4 Reward learning, empirical comparisons, and capability interfaces What rewards change. Outcome-reward learning can acquire a graph traversal strategy when the training distribution has sufficient mass on simple instances (Ran-Milo et al., 2026). That analysis follows expected-gradient flow of attention matrices with fixed values and prescribed initial structure, and also gives a discrete extension. Implicit curricula and learned search have also been analyzed through RL dynamics (Huang et al., 2026b; Yang et al., 2026), while Lyu et al. (2026) compare RL and supervised Boolean-policy learning with a supplied dependency mask and process rewards. Zhou et al. (2025) give a reference-policy-dependent resource analysis for Kullback–Leibler (KL)-regularized distributional value learning. These works clarify different entry conditions and learning objectives. Our paired constructions derive source-acquired computation and then study how feedback uses it, alongside the information that feedback alone cannot identify. Empirical comparisons require the same distinctions. Yue et al. (2025) report improved small-budget success after RL without an expansion of large-budget coverage in their tested settings. Our equivalence result is of a different kind: it identifies held-out targets that rewards on the training distribution cannot reliably distinguish across reward-equivalent worlds. Self-correction training supplies a sequence of attempts and rewards (Kumar et al., 2024); thought-action models study when initial policies make internal computation useful (Hanna and Corrado, 2025); logit-bias adaptation studies a different route for using a fixed model (Cohen et al., 2026). Goal-conditioned control also depends on whether success means arrival at a deadline, first arrival, or occupancy (Modirshanechi et al., 2026). Our experiments distinguish access to successful trajectories, acquisition of the required computation, and reward-dependent adaptation, rather than treating any one of these measurements as all three. Task-specific computation and verification. Several capability families require additional resources that are explicit in their original settings. SATNet incorporates a differentiable satisfiability solver (Wang et al., 2019); DeepCoder uses learned guidance for program search (Balog et al., 2016); LeanDojo provides an executable theorem-proving and retrieval environment (Yang et al., 2023). Toolformer and ReAct study tool calls and environment interaction (Schick et al., 2023; Yao et al., 2022). These are distinct sources of computation and feedback from our finite transition and contextual tasks. Plan-verification theory, rule-based inference attacks, and attention interventions similarly specify particular symbolic or architectural interfaces (Sarrof et al., 2026; Xue et al., 2024; Guardieiro et al., 2025). Instruction-hierarchy training and external enforcement address different parts of agent reliability (Wallace et al., 2024; Guo et al., 2026; Debenedetti et al., 2025). Transfer across theory-of-mind tasks must also be measured rather than inferred from in-distribution success (Sarangi and Salam, 2025). 3 Framework and Preliminaries Two stages, two kinds of information. A world is a fixed but unknown pair of a reusable mechanism and a task rule. The mechanism maps inputs to intermediate computations; the rule specifies which use of that computation earns a reward. The learner first observes task-independent source records, whose distribution we call the source law, and trains a predictor. It then adapts the same parameters w with terminal correctness rewards on training prompts. Evaluation uses held-out prompts and no further rewards. In the sequential family below, the mechanism is a transition table θ and the task rule is an initial operation η; in the contextual family of Section 5, they are a retrieval map and a context–cue decision rule. Information and choice order. The world and the training/test split are fixed before data sampling; the learner and initialization are chosen independently of the world. Source observations reveal samples of the mechanism but carry no information about the task rule. Each feedback query supplies a fresh training prompt and scores one generated candidate; the learner cannot choose or repeat prompts except where a theorem states otherwise. A symbolic learner may choose its candidate, whereas the neural learner samples it from its current policy. Neither receives test feedback or intermediate correctness labels. Throughout, ℝR denotes the reals, iid abbreviates independent and identically distributed, and E, Pr are taken over the randomness specified locally. Adam across the two stages. The neural theorems use bias-corrected Adam (Kingma and Ba, 2015). At stage-local step t, let gtg_t be the minibatch gradient of the source loss (descent) or of the sampled reward score (ascent). For coordinate j, mt,j m_t,j =β1mt−1,j+(1−β1)gt,j, = _1m_t-1,j+(1- _1)g_t,j, vt,j v_t,j =β2vt−1,j+(1−β2)gt,j2, = _2v_t-1,j+(1- _2)g_t,j^2, (3.1) m^t,j m_t,j =mt,j/(1−β1t), =m_t,j/(1- _1^t), v^t,j v_t,j =vt,j/(1−β2t), =v_t,j/(1- _2^t), (3.2) wt+1,j w_t+1,j =wt,j∓γt,jm^t,jv^t,j+ϵj, =w_t,j∓ _t,j m_t,j v_t,j+ _j, (3.3) with decay coefficients β1,β2 _1, _2, coordinate step γt,j>0 _t,j>0 and offset ϵj>0 _j>0; the sign is −- for source descent and ++ for reward ascent. Moments start at zero. At the stage boundary every parameter is retained, both moments reset, and the step counter restarts. Each theorem fixes its source objective, stopping rule and blockwise steps and offsets. The SGD variant is in Section H.1. Sequential states and operations. Let 2=0,1F_2=\0,1\ with arithmetic modulo two. Fix r≥1r≥ 1, put Z=22r+1Z=F_2^2r+1, m=|Z|=22r+1m=|Z|=2^2r+1 and q=2mq=2m. States and controls form disjoint q-symbol alphabets S and U, indexed by (s,z)∈2×Z(s,z) _2× Z and (b,u)∈2×Z(b,u) _2× Z; with PAD and EOS the vocabulary V has 2q+22q+2 symbols. An unknown array θ∈2m×mθ _2^m× m defines the transition ϕ(b,u)(s,z)=(s+b+θu,z,z+u). _(b,u)(s,z)=(s+b+ _u,z,\ z+u). (3.4) We call z and u the public coordinates: they follow a known update and are readable from any token through a known index map. The state bit s is the private coordinate: its evolution depends on the m2m^2 unknown entries of θ. Every learner, neural or symbolic, has the index map and the grammar; neural token embeddings are initialized independently of them. Each operation is a permutation, and uniform averaging over controls sends any state to the uniform law Π . Write u=(x,y,t)u=(x,y,t) with x,y∈2rx,y _2^r, t∈2t _2, and define the tag ℓ(u)=t+x⋅y∈2 (u)=t+x· y _2, which is balanced under a uniform control. Let H be the number of executed operations and L=H−1L=H-1 the suffix length. A sequence of controls is a word. For a displayed suffix U2,…,UHU_2,…,U_H with Ui=(bi,ui)U_i=(b_i,u_i), training prompts are uniform conditional on ∑i=2Hℓ(ui)=0 _i=2^H (u_i)=0 and test prompts on the disjoint class with sum one; the split is fixed before source data, initialization and feedback. Equivalently, Csuffix=(−1)∑i=2Hℓ(ui),±=[⋅∣Csuffix=±1].C_ suffix=(-1) _i=2^H (u_i), _±=E[\,· C_ suffix=± 1]. (3.5) For prompt functions these expectations include the uniform initial state and suffix; for source functions they also include the independent first control and transition noise. 0E_0 denotes the source law without tag conditioning. Source records and prediction. A source record has N=2H+2N=2H+2 symbols, S0,U2,…,UH,U1,S1,…,SH,EOS,S_0,U_2,…,U_H,U_1,S_1,…,S_H,EOS, (3.6) with S0S_0 uniform, U1U_1 independent uniform, a training-tag suffix, and states evolving in execution order U1,…,UHU_1,…,U_H under the kernel TU=aPU+(1−a)ΠT_U=aP_U+(1-a) , where PUP_U is the permutation kernel of ϕU _U and 0<a≤10<a≤ 1 is the probability of applying the mechanism rather than a uniform reset. Records are independent, unfiltered by downstream success and carry no success labels. The task binding η=(bη,uη)η=(b_η,u_η), the unknown initial control that attaches the task to the mechanism, has no effect on the source law; its token may occur in records, but its downstream role is hidden. The source objective averages next-token cross-entropy (CE) over every position of every record in a minibatch. Random controls and noisy states give a nonzero conditional entropy; the constructions use a=1−c/Ha=1-c/H with reset scale c=1/64c=1/64 for Adam (0<c≤1/640<c≤ 1/64 for SGD) and bound the excess risk above this entropy. All prefix tokens are retained in the N-token context, left-padded. Terminal feedback and success measurements. A prompt is X=(S0,U2,…,UH)X=(S_0,U_2,…,U_H) with target y∗(X)=ϕUH∘⋯∘ϕU2∘ϕη(S0).y^*(X)= _U_H ·s _U_2 _η(S_0). (3.7) The learner generates A,Y1,…,YH,EOSA,Y_1,…,Y_H,EOS in n=H+2n=H+2 decisions, where A is the chosen first control and YiY_i the iith state. The verifier returns one exactly when A is a control, every YiY_i is a state, EOS is placed correctly, the public coordinates follow A and then the displayed suffix, and YH=y∗(X)Y_H=y^*(X). It does not require A=ηA=η or a faithful private history: an incorrect state-bit history that repairs itself is rewarded. For parameters w and a prompt X, under free generation from the induced policy, let E(w,X)E(w,X) be the probability of grammar-respecting endpoint success, S(w,X)S(w,X) the probability of endpoint success obtained with the correct first control A=ηA=η (task utility), M(w,X)M(w,X) the probability that A is a control and Y1=ϕη(S0)Y_1= _η(S_0), and V(w,X)V(w,X) the probability of a grammar or public-coordinate violation. Task utility implies endpoint success; an incomplete execution has zero utility. The memory metric M is a delayed-retention task: prompts differing only in the state bit of S0S_0 share a long suffix but have opposite first-state targets, so the model must carry the initial information through the prefix until it generates Y1Y_1. Learning resources and probability statements. The resource vector B=(Npre,Qverifier,Ngenerated,Nupdates)B=(N_ pre,Q_ verifier,N_ generated,N_ updates) counts observed source tokens, training verifier queries (abbreviated Q), generated tokens and optimizer updates. The verifier is the function that returns the terminal reward; one call evaluates H private transition entries and H public checks, so the private table is an oracle resource. High-probability bounds hold for each fixed world (θ,η)(θ,η) over initialization, source data and feedback sampling, simultaneously for every test prompt on that event. 4 Information and a finite neural learning path 4.1 What training feedback cannot identify The first question concerns every possible reward, before any optimizer is chosen. Once uηu_η is known, call two worlds reward-equivalent if every grammar-valid training candidate receives the same reward in both. Theorem 4.1 (Mechanisms with identical training rewards; informal version of Theorem C.1). Let H≥3H≥ 3 and fix uηu_η. Two worlds are reward-equivalent if and only if, for some κ,d∈2κ,d _2, θu,z′=θu,z+κ+dℓ(u),bη′=bη+(Hmod2)κ+dℓ(uη).θ _u,z= _u,z+κ+d\, (u), b _η=b_η+(H 2)\,κ+d\, (u_η). (4.1) Their targets differ on every test prompt exactly when d=1d=1. A three-step example. Take r=1r=1, H=3H=3, θu,z=0 _u,z=0, and η=(0,(0,0,0))η=(0,(0,0,0)). Compare it with θu,z′=ℓ(u)θ _u,z= (u) and the same binding, the case κ=0,d=1κ=0,d=1 in (4.1). Starting with private bit zero and using controls with b=0b=0, a training suffix with tags (1,1)(1,1) leaves the original private bit at zero throughout, while the changed mechanism produces private bits 0,1,00,1,0 after the three operations. The terminal target is identical. A test suffix with tags (1,0)(1,0) instead produces 0,1,10,1,1 under the changed mechanism, so the terminal targets differ. Public coordinates agree throughout. Source observations expose the intermediate distinction: on the same input with s=0s=0 and a tag-one control, the probability that the next private bit is one is (1−a)/2(1-a)/2 in the original source law and (1+a)/2(1+a)/2 in the changed law. Thus training rewards conceal a difference that a single-step source distribution reveals. The theorem shows that these tag shifts and the constant shift κ exhaust the ambiguity. Its proof compares short paths padded with zero controls, forcing all tag-zero differences to agree and all tag-one differences to agree (Lemma C.3, including the verifier that accepts self-repairing private histories). Two reward-equivalent worlds with d=1d=1 demand opposite answers on every test prompt, so any learner that sees only training rewards has average test success at most 1/21/2 on that pair, after any number of queries. Source records break the tie: they expose single transitions before their effects cancel along a training word. This is the information that source observations add. Theorem C.2 gives the symbolic, query-optimal learner that uses it at logarithmic horizon; Theorem 4.2 below gives the neural one. 4.2 A complete neural learner in the same family Theorem 4.1 identifies what source observations must supply. We now show that a trainable network acquires the reusable execution from them and that rewards then teach the missing binding, along one finite optimization path. For a full padded prefix with token embedding ele_l at lag l, the network uses R causal attention heads and a bilinear feed-forward block: αh(l) _h(l) =softmaxl(bh(l)tatt+⟨Qhe0,Khel⟩dh), =softmax_l\! ( b_h(l)t_ att+ Q_he_0,K_he_l d_h ), (4.2) zatt z_ att =e0+R−1/2∑hOh∑lαh(l)Vhel, =e_0+R^-1/2 _hO_h _l _h(l)V_he_l, (4.3) fw(X) f_w(X) =W[zatt+U((Bzatt+b)⊙(Czatt+c))+bout]. =W [z_ att+U ((Bz_ att+b) (Cz_ att+c) )+b_ out ]. (4.4) Position l=0l=0 is the current token and h indexes heads; bh(l)b_h(l) is a learned positional bias, tattt_ att the attention temperature, dhd_h the query/key width, and Qh,Kh,Vh,OhQ_h,K_h,V_h,O_h the query, key, value and output maps. The residual attention feature zattz_ att feeds a feed-forward/readout block with matrices U,B,C,WU,B,C,W and biases b,c,boutb,c,b_ out; ⊙ is elementwise multiplication and fwf_w the logit vector. Every matrix, embedding and bias trains. Widths, head count, temperature and Gaussian scales are prescribed independently of θ,ηθ,η (Sections F.5 and E). Writing uffu_ f for the bracketed feature in (4.4), a second member of the class replaces it by g⊙2uff/1+∥uff∥2/12g 2u_ f/ 1+ u_ f ^2/12 with a trainable gain g initialized to ones; Section E.3 handles the changed function and derivative. The source objective is the average CE over all targets plus λ∥w∥2/2λ w ^2/2 with λ>0λ>0, a coupled L2L_2 penalty on every parameter. Adam runs with β1=0.9 _1=0.9, β2=0.999 _2=0.999, bias correction and separate positive readout and backbone rates; the source stage stops when the fresh-batch head gradient falls below a prescribed norm threshold, a criterion computed from raw source data alone. At feedback entry the regularizer is removed, every parameter is retained and both moments reset. Two REINFORCE policy-gradient updates with Adam then use fresh on-policy trajectories, terminal reward and baseline zero; the moments of the first update enter the second, control-output rows use a faster rate, and zero-reward trajectories, EOS scores and malformed outputs all stay in the batch. Let εlogit>0 _ logit>0 be the prescribed uniform logit-approximation tolerance (denoted h in (F.97)). Theorem 4.2 (Finite source learning and sampled Adam improvement; informal version of Theorem F.1). Fix q=22r+2q=2^2r+2 with r≥1r≥ 1, c=1/64c=1/64 and δ∈(0,1)δ∈(0,1). Run the source algorithm on either network member in (4.2)–(4.4), with the horizon, dimensions and training choices of Section F.5, then take exactly two feedback Adam updates, each on BRB_R fresh sampled trajectories. For every fixed world, with probability at least 1−δ1-δ, source stopping succeeds and, simultaneously for every held-out prompt X, minE(wf,X),M(wf,X),S(wf,X) \E(w_f,X),M(w_f,X),S(w_f,X)\ ≥7/8, ≥ 7/8, (4.5) maxE(ws,X),M(ws,X),S(ws,X) \E(w_s,X),M(w_s,X),S(w_s,X)\ ≤1/q+2nεlogit, ≤ 1/q+2n _ logit, (4.6) minJ∈E,M,S[J(wf,X)−J(ws,X)] _J∈\E,M,S\ [J(w_f,X)-J(w_s,X) ] ≥3/4, ≥ 3/4, (4.7) V(wf,X) V(w_f,X) ≤9c/8+1/128, ≤ 9c/8+1/128, (4.8) Here wsw_s is the source stopping state, wfw_f the second feedback iterate and n=H+2n=H+2. Feedback uses Q=2BRQ=2B_R queries; the complete source-token and update counts are in Theorem F.1. Reading the theorem. The dimensions, precisions, batch sizes, update bound and inverse rates are polynomial in q,H,δ−1q,H,δ^-1 up to logarithms (Section F.5); feedback uses no checkpoint selection. The three metrics move together: a source model already executes, but its success is of order 1/q1/q because it guesses the binding; two reward updates raise every metric by at least 3/43/4 while keeping format violations at the reset-noise level. Source loss must control the complete policy. Accurate conditional transition predictions do not by themselves make a model emit a state token. In the canonical coefficient class of Lemma F.3, an unregularized population near-minimizer can keep exact conditional transitions and an arbitrarily small source gradient yet emit EOS with probability close to one on a chosen suffix (Proposition F.38). Low source loss alone therefore does not deliver a usable executor; the regularizer does. In the canonical class the ridge objective has a unique optimum, which an invariant-gradient calculation identifies as class/PAD terms plus two state–operation interactions. The source excess decomposition controls transition errors and token-class masses (control, state, EOS, PAD) together, including the opposing EOS term; signed contraction bounds the effect of conditioning on the training word class; strong convexity carries the optimum to the random feature coordinates; and ridge bounds every readout coordinate along the Adam trajectory. A momentum argument along the full trajectory, with fresh-record concentration, then forces the observable stopping criterion within a finite budget. How that computation changes feedback updates. The source model executes any supplied operation but chooses the binding nearly uniformly; its task success is of order 1/q1/q because the source law leaves the binding unspecified. Forcing a candidate first control makes the model generate its learned transition sequence from there, so the reward stage chooses among computations that already exist: the correct binding earns reward near one, the opposite private bit earns a small reward from self-repairing histories, and wrong public coordinates earn zero. The choice is global and transfers to held-out suffixes because the same learned transitions execute them. Adam’s denominator is correlated with the sampled gradient and depends on the previous update. We prove a bound on the feature operator that is uniform over all bounded second-moment histories; Gaussian reference dynamics (the population updates around which the sampled path is controlled) then reaches correct-binding probability 15/1615/16 in two updates, with a small first step and a larger second step retaining both histories. A comparison of the full sampled path with its stopped reference absorbs sampling, non-choice scores, EOS, wrong-token-class events and all backbone motion into the batch-size requirement of Theorem 4.2. An SGD learning path. The same family also admits finite source minibatch SGD followed by sampled policy-gradient updates, with the same 7/87/8 final success and 3/43/4 improvement for all three metrics, using unregularized source training, a predetermined source endpoint and separately chosen horizons, block rates and budgets (Theorem H.1; proofs in Sections H.2, E and H.4). In the sequential model, execution is learned through rich initial features while every backbone block trains at a small positive rate, and feedback selects one global initial operation. The contextual construction next brings the representation itself into the learning problem: attention forms during source training and carries a task rule that varies with the input. From exact ambiguity to finite exposure. Initial source observations and inherited checkpoint weights can be treated jointly as an observation O about the world. A reward transcript can only resolve ambiguity that its allowed prompts expose. For the known two-world gauge experiment in Proposition C.16, with initial-law overlap α and odd training prompts appearing independently with probability ε , the optimal equal-prior endpoint error is α(1−ε)Q/2α(1- )^Q/2. The full raw-source formula is in Corollary C.21. These are information bounds for the specified pair; the finite neural path above supplies a separate optimization guarantee. Appendix D compares it with quotient recovery from few source records and table recovery followed by binding search. 5 Learning the representation that rewards use The sequential model shows how learned transitions support a new global binding. The harder case is a task rule that varies with the input, because rewards can only teach such a rule through a representation that exposes the relevant input. A contextual memory task separates the two ingredients: a hidden retrieval rule, which source training must acquire, and a decision made with the retrieved information, which only rewards can teach. We characterize what each observation channel identifies, then follow finite Adam updates from random factors through retrieval acquisition to reward-based task learning. 5.1 A contextual memory task Let nctx≥1n_ ctx≥ 1 be the number of contexts, let the content dimension p≥17p≥ 17 be odd, and put k=p−2k=p-2. A fixed unknown world consists of U=diag(U1,…,Unctx),Uℓ∈I2,Pswap,s∈±1nctx,τ∈±1k.U=diag(U_1,…,U_n_ ctx),U_ ∈\I_2,P_ swap\,s∈\± 1\^n_ ctx,τ∈\± 1\^k. Here I2I_2 is the two-dimensional identity, PswapP_ swap exchanges the two slots, and k is the cue dimension. The map UℓU_ specifies which of two slots a query selects; sℓs_ is a contextual sign; and τ specifies a Boolean cue function shared across contexts. The world is fixed before initialization and training data are drawn. The width dwidthd_ width counts factor rows. There are dwidthd_ width query/key rows qi,ki∈ℝ2nctxq_i,k_i ^2n_ ctx and dwidthd_ width head rows ai,bi∈ℝnctx+pa_i,b_i ^n_ ctx+p, oi∈ℝo_i . Every coordinate is initially an independent standard Gaussian. Define the attention and head coefficient matrices (called cores below): M=1dwidth∑i=1dwidthqiki⊤,C=1dwidth∑i=1dwidthoiaibi⊤.M= 1d_ width _i=1^d_ widthq_ik_i , C= 1d_ width _i=1^d_ widtho_ia_ib_i . (5.1) For query x, keys κ1,κ2 _1, _2 and values V1,V2V_1,V_2, the network computes πj=ex⊤Mκj∑j′=12ex⊤Mκj′,h=∑j=12πjVj,g=hψ(∥h∥2),f=g⊤Cg. _j= e^x M _j _j =1^2e^x M _j , h= _j=1^2 _jV_j, g=h ψ( h ^2), f=g Cg. (5.2) The weights πj _j pool the memory values into h, g is the normalized pooled representation, and f is a scalar logit. The binary policy takes action +1+1 with probability σ(f)=(1+e−f)−1σ(f)=(1+e^-f)^-1. Its readout is a trainable bilinear feed-forward block, since f=dwidth−1∑ioi(ai⊤g)(bi⊤g)f=d_ width^-1 _io_i(a_i g)(b_i g). The head receives only the pooled values. All dwidth(6nctx+2p+1)d_ width(6n_ ctx+2p+1) supplied parameters are trainable and retained across both stages. The normalizer is any fixed finite mixture ψ(u)=∑j=1mnormωj(u+ζj)−αj,ωj>0,∑jωj=1,0≤αj≤1,0≤ζj≤Zmax<∞.ψ(u)= _j=1^m_ norm _j(u+ _j)^- _j, _j>0, _j _j=1, 0≤ _j≤ 1, 0≤ _j≤ Z_ <∞. (5.3) Here mnormm_ norm counts mixture components, ωj _j are their weights, ζj _j their offsets, αj _j their powers, and ZmaxZ_ bounds the offsets; u is a squared norm. The mixture constants specify the architecture and remain fixed. Define g=0g=0 and its selected derivative as zero at an exceptional zero pool; the proof events avoid this point. The class takes context features and numerical query/key/value inputs, and varies normalization at nonvanishing scale while retaining the attention and factorized head. Here O(2)O(2) is the group of two-dimensional orthogonal matrices, eie_i is a standard basis vector, ⊗ is the tensor product, 0nctx0_n_ ctx is a length-nctxn_ ctx zero vector, and N(0,Ip)N(0,I_p) is the standard p-dimensional Gaussian. The sign map signsign codes the future bit as −1-1 or +1+1. A source record draws a uniform context ℓ , an independent uniformly random orthogonal frame R∈O(2)R∈ O(2), and a uniform selected slot J∈1,2J∈\1,2\. It supplies κj=eℓ⊗Rej,x=UκJ,Vj=(0nctx,Gj),G1,G2∼iidN(0,Ip), _j=e_ Re_j, x=U _J, V_j=(0_n_ ctx,G_j), G_1,G_2 iid N(0,I_p), followed by the raw future bit y=sign(GJ,1GJ,2)y=sign(G_J,1G_J,2). Source matching is the correct-slot probability πJ _J; its uniform bounds range over the source context, frame and query. Only the future-bit position contributes the cross-entropy log(1+e−yf) (1+e^-yf). The source law is independent of s,τs,τ and unfiltered by downstream success, so next-bit prediction teaches retrieval without touching the downstream Boolean rule. A training prompt draws independent uniform ℓ , query index q∈1,2q∈\1,2\ and cue z∈±1kz∈\± 1\^k. Keys are eℓ⊗e1,eℓ⊗e2e_ e_1,e_ e_2, the query is eℓ⊗eqe_ e_q, and values are (eℓ,0,0,z),(eℓ,0,0,−z).(e_ ,0,0,z), (e_ ,0,0,-z). The selected slot j⋆∈1,2j_ ∈\1,2\ is defined by Uℓej⋆=eqU_ e_j_ =e_q. Its cue is (−1)j⋆−1z(-1)^j_ -1z, so the correct action is sℓ(−1)j⋆−1sign(τ⊤z)s_ (-1)^j_ -1sign(τ z). The policy samples an action A, receives only the correctness reward Rrew∈0,1R_ rew∈\0,1\, and uses Rrew∇logPr(A)R_ rew∇ (A) with baseline zero. Every sample, including zero reward, remains in the minibatch denominator. Let cminc_ be the attention-contrast threshold, mmemm_ mem the finite amplitude resolution and ρmem _ mem the smaller test amplitude. Set cmin=1k,mmem=⌈log2(2k)⌉,ρmem=1−2−mmem.c_ = 1 k, m_ mem= _2(2 k) , _ mem=1-2^-m_ mem. Test values have cues u1z,u2zu_1z,u_2z, with independent uniform u1∈±1u_1∈\± 1\ and u2∈±ρmemu_2∈\± _ mem\. All four amplitude pairs differ from the training pair (1,−1)(1,-1). Their label is ytest=sℓsign(uselected)sign(τ⊤z).y_ test=s_ sign(u_ selected)sign(τ z). (5.4) The two memories share a cue direction. Testing keeps the context identities and changes their memory configurations. Let JtestJ_ test denote average success of one sampled action. For an individual test input the corresponding success is σ(ytestf)σ(y_ testf). 5.2 What the two observation channels identify Write ξℓ=+1 _ =+1 for the identity orientation and ξℓ=−1 _ =-1 for the swap, and put v1=1,v2=−1v_1=1,v_2=-1. Training labels factor as ytrain=sℓξℓvqsign(τ⊤z).y_ train=s_ _ v_qsign(τ z). Rewards identify the products sℓξℓs_ _ and the cue vector τ up to a common sign, and nothing more; source geometry identifies each orientation ξℓ _ . This factorization yields a complete information characterization for the very task that the attention model learns. Theorem 5.1 (Information supplied by contextual source observations; informal version of Theorem C.26). Two worlds are reward-equivalent on every training input if and only if τ′=ετ,sℓ′ξℓ′=εsℓξℓ(ℓ=1,…,nctx),ε∈±1.τ = τ, s _ ξ _ = s_ _ ( =1,…,n_ ctx), ∈\± 1\. (5.5) The nctxn_ ctx orientations may vary independently; the global sign leaves all predictions unchanged. Flipping an orientation reverses exactly the equal-sign test-amplitude labels in that context. For the exact numerical source, let NobsN_ obs count contextual source records, and let ℛNobsR_N_ obs be the infimum, over all world-independent learners and finite fixed iid feedback-query budgets, of worst-world expected test error after NobsN_ obs iid source records. Then ℛNobs=14(1−1/nctx)Nobs.R_N_ obs= 14(1-1/n_ ctx)^N_ obs. (5.6) For nctx=1n_ ctx=1, the factor is one at Nobs=0N_ obs=0 and zero at Nobs≥1N_ obs≥ 1. With source observations alone, the minimax error is 1/21/2 for every NobsN_ obs. The first Fourier coefficients of the cue function identify τ up to sign, which gives necessity in (5.5); one source record identifies the orientation of its context almost surely, which gives (5.6): a test context missed by all NobsN_ obs records, an event of probability (1−1/nctx)Nobs(1-1/n_ ctx)^N_ obs, keeps its orientation hidden. The orientation changes labels on the equal-sign half of test configurations, where indistinguishability forces error 1/21/2, giving 12⋅12=14 12· 12= 14. A finite feedback-label table approaches this floor; Theorem C.26 gives the explicit query count and tolerance (proof in Section C.5). The theorem separates two needs that a task score merges. Source observations remove the orientation ambiguity that rewards preserve; rewards identify a task rule absent from the source law. The next result shows how ordinary training acquires the representation through which that rule becomes learnable. (The optimization result also covers a finite coordinate alphabet; (5.6) uses exact numerical observations.) 5.3 Finite acquisition and feedback learning We use bias-corrected Adam with first- and second-moment decay parameters 0≤β1<10≤ _1<1, 0<β2<10< _2<1, and β12<β2 _1^2< _2. Source minibatch descent starts with zero moments. At its fixed endpoint all parameters are retained, both moments are reset once, and feedback minibatch ascent begins. The dimension-normalized coordinates are those in (5.1). Physical query/key row parameters are Qi=qi/dwidth,Ki=ki/dwidthQ_i=q_i/ d_ width,K_i=k_i/ d_ width; physical head rows are ai,bi,oia_i,b_i,o_i. A normalized step ηt _t and positive denominator offset ϵε are implemented using query/key step ηt/dwidth _t/ d_ width and offset ϵ/dwidthε/ d_ width, and head step ηt _t and offset ϵ/dwidthε/d_ width. Thus every normalized coordinate has the same nonzero rate and offset. These physical step and offset choices implement the common normalized update. Theorem 5.2 (Acquisition, contextual transfer and intervention; informal version of Theorem G.1). Fix the task dimensions, an architecture (5.3), Adam parameters (β1,β2)( _1, _2) as above, a world (U,s,τ)(U,s,τ) and δ∈(0,1)δ∈(0,1). Choose width, batches, steps, offsets and input precision as in Sections G.7 and G.8; they are explicit and finite. With probability at least 1−δ1-δ over initialization and all training samples, jointly: 1. Acquisition. Initial source matching is at most 0.510.51; after source training it is at least 0.90.9 uniformly over contexts, frames and queries, and greedy source-bit accuracy exceeds 0.90.9. 2. Transfer. At the final feedback iterate, sampled success is at least 19/2019/20 on every held-out context, query, cue and amplitude pair, an improvement of at least 2/52/5 over the source checkpoint. 3. Intervention. The same schedules and budgets with the initial query/key parameters frozen in both stages, and every head factor trained, end with sampled test success at most 1/2+1/641/2+1/64; the paired gap exceeds 2/52/5. The conclusions hold for the finite coordinate alphabet of Section G.8 with supplied numerical decoding and real-valued optimizer states. At fixed nctx,p,Zmax,mnormn_ ctx,p,Z_ ,m_ norm the budget formulas may be shared across mixture members, with probability evaluated for each fixed member. The learned policy is input-dependent: reversing the cue sign reverses the correct action at the same context and query, and a constant action has success exactly 1/21/2. The learned coefficient shares τ across contexts and combines it with the contextual sign sℓs_ . A single shared step and offset per stage carries the matching statistic from at most 0.510.51 to at least 0.90.9. The frozen-attention run is a complete training run under the same schedule; it keeps the small random initial attention and whatever the trained head can amplify from it, and remains within 1/641/64 of chance. Acquired attention exposes the task information that makes the reward stage effective. Throughout, a reference is the deterministic population dynamics around which the sampled Adam path is controlled. The reference source state has attention core M=βUM=β U with β≥0β≥ 0 and correct-slot attention weight (matching) σ(β)σ(β). Pooling two Gaussian values changes both signal correlation and scale, which can compete under normalization. Let L(t,β)L(t,β) be the source cross-entropy at the symmetric reference whose logit is th1h2ψ(‖h‖2)t\,h_1h_2ψ(\|h\|^2), where h1,h2h_1,h_2 are the two content coordinates that determine the source bit and t is the coefficient on their product. The Gaussian identity in Equation G.3 gives ∂βL(t,β)<0(t>0, 0≤β<∞). _βL(t,β)<0 (t>0,\ 0≤β<∞). (5.7) Increasing matching therefore improves the source objective throughout the normalization family once the head has acquired positive parity alignment. The head’s initial source direction creates that alignment, and a uniform drift bound carries attention to reliable matching. The appendix derives the identity by separating conditional label uncertainty from the correctly signed loss. The source stage also controls the distribution of the trained factors: inactive context and cue factors retain their Gaussian reference law, independently of the evolved source-active coordinates, while the output factor stays symmetric. The proof bounds the deviation of the trained state from this law. This retained distribution supplies the starting state of the feedback argument. The block entry CℓiC_ i below pairs context coordinate ℓ with cue coordinate i (full-vector index nctx+2+in_ ctx+2+i); CiℓC_i denotes the transposed block entry. Let B denote the scalar strength of the learned context–cue interaction. Under the feedback reference law, its coefficients take the form Cℓi+Ciℓ=Bsℓτi.C_ i+C_i =Bs_ _i. The pooled training cue is a signed multiple czcz, where c=tanh(λ/2)c= (λ/2) is the learned attention contrast and λ is the contrast coefficient of the reference attention block (Section G.3). The policy’s population success becomes J(B,c)=σ(Bw(c)|τ⊤z|),w(c)=cψ(1+kc2).J(B,c)=Eσ\! (Bw(c)|τ z| ), w(c)=cψ(1+kc^2). Here w(c)w(c) is the effective cue-signal scale. Uniform retrieval gives c=0c=0 and erases the cue in training. Acquired retrieval gives a positive direction for B under the specified coordinatewise update. For all class members, c≥cminc≥ c_ is a lower boundary preserved by the reference dynamics, because dcc(1+kc2+ζ)α=1+ζ+(1−2α)kc2(1+kc2+ζ)α+1≥0at c=cmin. ddc c(1+kc^2+ζ)^α= 1+ζ+(1-2α)kc^2(1+kc^2+ζ)^α+1≥ 0 c=c_ . Here α,ζα,ζ denote a single mixture component’s power and offset. This is the structural condition that survives the class variation. When α=1,ζ=0α=1,ζ=0, further sharpening above this boundary can reduce w; useful feedback is therefore compatible with decreasing attention contrast. Every held-out selected cue still has a positive pooled signed amplitude, so a sufficiently large learned B gives a uniform test margin. We compare sampled Adam with a reference evolving each row by fϵ(v)=v/(|v|+ϵ)f_ε(v)=v/(|v|+ε) applied to a scalar coordinate v of its current scaled population gradient. The proof controls sampled, bias-corrected Adam around it using a deterministic coordinate bound |m^tv^t+ϵ|≤KA:=1−β1(1−β2)(1−β12/β2). | m_t v_t+ε |≤ K_A:= 1- _1 (1- _2)(1- _1^2/ _2). Here m^t,v^t m_t, v_t are Adam’s bias-corrected first and second gradient moments, and KAK_A bounds the normalized update. History comparisons retain old momentum and the correlation between each sample gradient and its denominator. Because the head moment map is cubic, its Gaussian rows do not admit a uniform sensitivity bound. We solve the rowwise sensitivity inequality first and only then average, using a finite exponential Gaussian moment. This yields an explicit joint path bound for the trained and frozen branches. Source errors persist across the switch; only optimizer moments reset. 5.4 Training resources For one run, let BmbB_ mb be the per-update batch size and Ts,TfT_s,T_f the source and feedback update counts. Write Ns=BmbTsN_s=B_ mbT_s for source records and Nf=BmbTfN_f=B_ mbT_f for sampled feedback actions. The resource coordinates count source records, verifier queries, sampled actions and optimizer updates: (Nsourcerecords,Qverifier,Nactions,Nupdates)=(Ns,Nf,Nf,Ts+Tf).(N_ source\ records,Q_ verifier,N_ actions,N_ updates)=(N_s,N_f,N_f,T_s+T_f). Each source record contributes one predicted bit and the numerical coordinates counted at the precision in Section G.8; the learned and frozen runs receive the same budget. Together with the minimax error in Theorem 5.1, these budgets give both the information available for learning and a finite algorithm that uses it. Attention contrast and useful reward signal. Retrieval exposes the task cue, but its effective update scale also depends on normalization. For the member ψ(u)=u−1ψ(u)=u^-1, w(c)=c/(1+kc2)w(c)=c/(1+kc^2) increases up to c=1/kc=1/ k and decreases above it. Proposition G.16 derives this dependence within the stated normalization model. The finite theorem combines a preserved sufficient contrast bound with control of the sampled-path error. 6 Experiments We measure source acquisition, reward-task selection and retention separately on Qwen2.5 Base models (Qwen et al., 2024). The intervention is full-parameter source midtraining from pretrained checkpoints before terminal-reward adaptation. Greedy evaluation uses one highest-probability completion per input; reward training samples at temperature one in the original experiments, with calibrated sampling in the independent Memory confirmation (Section A.2). A world–training-seed pair is one independent unit. We average A/B branches within world before computing 95% Student-t intervals; effects use paired world differences. Methods and complete comparisons are in Sections A.1 and A.2; earlier experimental designs are described separately (Sections A.4 and A.13). 6.1 What computation does source training acquire? Eight fresh worlds per module compare correct source with a matched structural control, using Qwen2.5-0.5B. A task-independent format stage precedes source training, with no later format stage. Sequential observes 2222^22 source tokens; its control preserves public states and randomizes private bits. We cross source structure with two objectives: predict states and END/EOS, or additionally predict the first operation already present in each record. This uniformly sampled operation is independent of the downstream task binding. The records and observed-token budgets are unchanged; the added operation positions increase the supervised targets. Memory observes 2242^24 tokens with an all-token objective, comparing correct with wrong retrieval on matched inputs. With the first operation supplied at evaluation, action-objective Sequential source models generate 87.74% strictly complete trajectories versus 1.93% for the private-random control (Figure 3a). Correct-source Memory first-field accuracy is 99.76%, versus 51.95% under wrong retrieval; joint original/value-flip accuracy is 99.41% versus 5.81%. The latter probe requires both answers to be correct, exposing input-sensitive source predictions without supplying a downstream task rule. These tests distinguish mechanism-dependent responses from output syntax or a constant bit. (a) Sequential source (b) Memory source (c) Memory retention Figure 3: Acquiring and retaining source computation. (a) Strict trajectories on 512 source probes per world with first operation supplied, using the action objective. (b) Correct first field on 256 Memory records. Points and gray lines show paired worlds; diamonds give means and 95% intervals. (c) Both answers correct on 256 original/value-flip pairs during reward training, for correct/wrong source with pure RL or replay. A/B branches are averaged within each of eight worlds; bands are untruncated 95% intervals. Replay adds 131,072 source targets and 128 CE updates per branch. 6.2 Do rewards select and retain the computation? The original source checkpoints branch into opposite tasks A/B for 8,192 reward queries, with fresh Adam moments and a fixed learning rate 3×10−63× 10^-6. Both verifiers score each output on the same 1,024 held-out evaluation records per world. If JXYJ_XY is success of the task-X branch under verifier Y, task selectivity is Δtask=(JAA−JAB)+(JBB−JBA)/2 _ task=\(J_A-J_AB)+(J_B-J_BA)\/2. Shared ancestry fixes what the two branches acquired; their cross-scored outputs measure whether different rewards select different task behavior. Sequential: source computation supports task selection. With first-operation supervision, correct-source success rises from 4.83% to 82.61%, compared with a control endpoint of 44.15% (Figure 4a). The paired final-score advantage is 38.46 p, with 95% interval [19.78,57.15][19.78,57.15]. The predeclared task-selectivity advantage is 67.57 p, interval [37.04,98.10][37.04,98.10], with two-test Holm-adjusted exact p=.015625p=.015625 (Figure 13b). All sixteen correct-source and all sixteen control branches make 128 nonzero reward updates. Thus the source comparison is made between actively adapting models with the same reward budget. The first-operation target trains an output position needed to begin a free rollout; it supplies no preferred task binding. Both action-objective routes can therefore initiate reward-bearing trajectories, while their source records differ in private-state structure. The resulting selectivity contrast tests the contribution of that structure under the action objective. Comparing action and state objectives is a separate, predeclared secondary analysis: the source-by-objective selectivity interaction is 42.59 p with t interval [0.82,84.36][0.82,84.36], but exact p=.08594p=.08594 (Table 4). The exact test leaves this secondary interaction unresolved. (a) Sequential (b) Memory confirmation Figure 4: Source preparation and reward learning. (a) Sequential action-objective routes, worlds 500–507. (b) Independent Memory confirmation, worlds 508–515: selected versus alternative retrieval after a shared correct-source start. Success uses 1,024 fixed held-out records; A/B are averaged within each world, with 95% intervals across eight worlds. Source costs differ between panels (Sections A.1 and A.2). Memory: source replay preserves acquired behavior. From the same source checkpoints, Memory compares pure reward training with a source CE step after each reward update. Replay uses 1,024 targets from that route’s own previously observed source records, including wrong labels for the wrong-retrieval control. Both kinds of update share the branch’s Adam moments and learning rate. After 8,192 queries, correct-source replay retains 92.21% joint original/flip-pair accuracy versus 0% under pure RL (Figure 3c); first-field accuracy is 96.24% versus 49.66%. All branches perform 128 nonzero RL updates; replay additionally uses 131,072 source targets in 128 CE updates. Correct-source final task accuracy is numerically higher than wrong retrieval by 4.40 p under pure RL and 5.24 p with replay, with paired intervals [−11.38,20.18][-11.38,20.18] and [−8.10,18.58][-8.10,18.58]. The predeclared correct-minus-control task-selectivity effect under replay is 0.67 p, interval [−1.22,2.57][-1.22,2.57] (Tables 3 and 2). The source probes establish that replay preserves responses to stored values; the task intervals span zero. Measuring both distinguishes retaining the source computation from selecting it for the reward task. Memory: task-relevant retrieval supports reward learning. In an independent confirmation on eight new Memory worlds, task-relevant retrieval reaches 75.32% success versus 49.86% for alternative retrieval, from A/B-averaged entry scores of 50.00% (Figure 4b). The paired accuracy advantage is 25.46 p (95% CI 20.10–30.81), positive in every world. The predeclared selectivity advantage is 50.92 p (CI 40.20–61.63; exact p=.0078125p=.0078125), a separate single primary test. Both routes inherit the same correct 2242^24-token source checkpoint, then receive matched training on public-projection and retrieval-vector targets. Reward adaptation uses calibrated sampling, vector/syntax replay without supervising source answer signs, and guarded steps. This comparison shows task-relevant retrieval supporting reward learning under this recipe; full methods, added source costs and all 32 endpoints appear in Section A.2. Stage order and computation retention. Format learning and computation retention are distinct: an earlier study found that learning termination could coincide with a loss of source-probe accuracy (Figures 25 and A.14). This observation motivates format preparation before source acquisition. The resulting sequence lets probes distinguish what source training acquires, what adaptation retains, and which task the rewards select. Initialization and budget comparisons appear in Section A.13. 6.3 How does source training change the subsequent reward trajectory? A separate, prespecified Sequential world (508, Task A) compares three Qwen2.5-0.5B routes from the same format checkpoint: direct RL, correct source followed by RL, and public-preserving/private-random source followed by RL. The source routes each observe 2222^22 tokens with first-operation supervision; all routes use 8,192 reward queries. Original-checkpoint evaluation on 1,024 fixed held-out inputs shows correct-source success rising from 0% at reward entry to 100% after 1,024 queries, remaining there through 8,192 queries. Direct RL finishes at 51.76% and the private-random route at 51.27%; all three make 128 nonzero RL updates (Figure 15). Thus, in this world, correct source prepares a model for rapid reward learning without already solving the downstream task at reward entry. This independently specified world complements the eight-world A/B task-selection experiment with a direct-RL comparison. Figure 2 evaluates a common two-dimensional parameter slice at 81 grid models: Task A success on 256 held-out inputs and source cross-entropy on 128 correct-source records. Correct and private-random source move the parameters in different projected directions, while direct RL remains closer to the shared origin in this projection. The plane retains 91.96% of aggregate squared parameter displacement. Its colors describe the evaluated slice models; projected checkpoints can have different task scores from their originals, so training performance is read from the actual-checkpoint curves in Figure 15. This separates two views of preparation: the measured slice displays how source loss and task success vary across parameters, and the original trajectories show how each starting point responds to rewards. Section A.3 gives the plane construction, route-wise residuals and paired reconstruction measurements. An additional source stage and final public-task performance. Five paired Qwen2.5-1.5B seeds compare direct RL, domain-source training followed by RL, and shuffled-source training followed by RL. Source data are raw OpenWebMath documents (Paster et al., 2023) for GSM8K arithmetic (Cobbe et al., 2021), or title-disjoint Wikipedia passages for HotpotQA multi-hop questions (Yang et al., 2018). Source training receives no downstream answer or supporting-fact labels; it uses 2222^22 prediction tokens, and reward adaptation uses 8,192 queries. The shuffled control permutes tokens within 64-token document blocks. This comparison asks how continued prediction changes reward-entry performance and subsequent reward gains. For these public-task comparisons, let Ja,QJ_a,Q be route a’s greedy success after Q reward queries. Its within-seed gain and paired difference from direct RL are Ga=Ja,8192−Ja,0,D=Gsource−Gdirect.G_a=J_a,8192-J_a,0, D=G_source-G_direct. (6.1) On identical before/after sets of 1,319 GSM8K or 1,024 HotpotQA questions, GSM8K domain-source training moves feedback-entry accuracy from 8.57% to 2.32%, and source+RL then reaches 62.05% against 62.32% for direct RL: the source route’s paired reward-stage gain is 5.97±3.665.97± 3.66 points larger. HotpotQA source training raises the starting score from 23.93% to 35.10%, with final scores of 45.59% and 47.60%: here the source route’s paired gain is 13.18±1.5013.18± 1.50 points smaller (Table 2; full figure and learning curves in Figures 21 and 23). Final accuracy is entry accuracy plus reward gain, and a source stage changes the starting point and the subsequent gain in different directions, so none of the three quantities can stand in for the others. A 2202^20-token source route on the same seeds reaches 61.76% on GSM8K and 46.23% on HotpotQA; the seed-paired endpoint differences from the 2222^22 route are −0.29±5.64-0.29± 5.64 and 0.64±1.120.64± 1.12 points (Tables 29 and 30). Table 1: Task accuracy (%) in the original eight-world comparison (500–507), after 8,192 queries. Means ± 95% Student-t half-widths after A/B averaging, on 1,024 records; Δ is paired correct minus control (p). Table 2: Public-task accuracy (%) at entry and after 8,192 queries; gain is final minus entry (p). Means ± 95% Student-t half-widths over five paired seeds, on 1,319 GSM8K / 1,024 HotpotQA records. None denotes direct RL; its fixed-checkpoint entries have ±0± 0. Setting Correct Control Δ Sequential State 27.9927.99 ±30.48± 30.48 12.4112.41 ±15.57± 15.57 +15.58+15.58 ±19.42± 19.42 Action 82.6182.61 ±16.80± 16.80 44.1544.15 ±9.71± 9.71 +38.46+38.46 ±18.69± 18.69 Memory Pure RL 48.1648.16 ±3.61± 3.61 43.7643.76 ±14.75± 14.75 +4.40+4.40 ±15.78± 15.78 Replay 47.5447.54 ±3.44± 3.44 42.3042.30 ±11.84± 11.84 +5.24+5.24 ±13.34± 13.34 Source Entry Gain Final GSM8K None 8.578.57 ±0± 0 53.7553.75 ±4.14± 4.14 62.3262.32 ±4.14± 4.14 Domain 2.322.32 ±0.39± 0.39 59.7359.73 ±3.16± 3.16 62.0562.05 ±2.93± 2.93 Shuffled 8.588.58 ±1.31± 1.31 41.1241.12 ±28.34± 28.34 49.7049.70 ±29.29± 29.29 HotpotQA None 23.9323.93 ±0± 0 23.6723.67 ±1.02± 1.02 47.6047.60 ±1.02± 1.02 Domain 35.1035.10 ±0.98± 0.98 10.4910.49 ±1.74± 1.74 45.5945.59 ±0.86± 0.86 Shuffled 5.625.62 ±0.96± 0.96 38.8338.83 ±1.91± 1.91 44.4544.45 ±1.13± 1.13 Public controls and transfer. Public answer-only SFT is weaker than RL on GSM8K (12.36%) and stronger on HotpotQA (53.74%) (Table 28). Secondary measurements show that the routes differ in answer style, not only in correctness: GSM8K source routes generate longer solutions and the shuffled route’s wide interval coincides with format failures (17.8±47.5%17.8± 47.5\%) and truncated outputs (17.7±47.6%17.7± 47.6\%), while HotpotQA answers are short and well formed on every route (Table 11); the HotpotQA direct-RL lead over the domain source, about two points, holds on both bridge and comparison questions (Table 12). Transfer depends on the evaluation task: after GSM8K training, GSM-Symbolic success is 53.0% for source+RL against 54.5% for direct RL, and BIG-Bench Hard (BBH) logic and object-tracking results depend on the training task and route (Tables 13 and 27). Public-task best-of-16 selection measures candidate coverage under test-time verification, a separate resource from reward training (Tables 13 and A.13). 7 Discussion Rewards select among computations that exist. The information results and training constructions explain complementary parts of the same mechanism. Source observations distinguish mechanisms that training rewards leave indistinguishable. The Adam constructions turn this information into execution or retrieval, then use sampled rewards to learn its task-specific use: two updates in the sequential construction and an explicit finite budget in the contextual one. The acquired computation applies to held-out inputs, allowing the task rule to transfer. This gives a concrete question to ask of a reward-stage gain: which acquired computation supports the new task rule? The measurements in Section 6 address each part of this question. Three interventions, three questions. The no-source bound in contextual memory concerns world-independent learners with the specified observations. The frozen-attention run holds the training schedule fixed and freezes the initial random query/key factors in both stages while training the head. Direct language-model RL omits additional source midtraining and starts from inherited pretrained weights. Each intervention isolates one question: information access, representation learning, or continued source training. Appendices D and 32 compare the neural path, which reaches the binding through the model’s own sampled trajectories at 2BR2B_R verifier calls, with an explicit controller that needs at most q−1q-1 verifications, and with supervised and test-time alternatives. Initial information and the stage of source prediction. The information results specify what the learner observes. An inherited checkpoint that already reveals the hidden mechanism can distinguish worlds even when their training rewards coincide; the no-source bounds assume a world-independent learner and initialization. Likewise, the source-only bound uses source laws that leave the downstream rule unspecified. The finite Adam constructions start from the stated random distributions, whose initial geometry lets the proofs establish head alignment and control the training trajectory. The Qwen experiments study continued source prediction from inherited weights, making source midtraining the intervention. The constructions and experiments thus study acquisition from two distinct starting points: specified random factors and inherited language-model weights. On public tasks, domain-related source text without downstream labels is a different information interface from the rule-independent source laws of the controlled families. From acquired computation to task use. Source-checkpoint probes, retention measurements and paired reward branches separate acquired computation from its task-specific use. The historical output decompositions illustrate this distinction: a valid public path or output format can coexist with chance-level private-task behavior (Sections A.12 and A.8). The original eight-world Sequential comparison shows reward-task selection after execution is acquired; its Memory comparison shows that replay preserves intervention-sensitive retrieval (Section A.1). The independent Memory confirmation connects retrieval preparation to task learning: under a matched adaptation recipe, task-relevant retrieval reaches 75.32% success versus 49.86% for alternative retrieval (Section A.2). Both confirmation routes inherit correct source information and receive the same added training budget. The original comparison uses a two-test Holm family; the confirmation has a separate single primary test and additional source preparation. Together, these experiments show how stage-specific measurements connect the computation acquired from source data to its retention and reward-dependent use. Entry accuracy, reward gain and final accuracy. Final accuracy combines two effects of source training: accuracy at reward entry and gain during reward adaptation. Our public-task comparisons show that these effects can move in different directions. Reporting all three scores locates the improvement in the training sequence; the controlled-task probes identify the computation available at each stage. This connects public-task evaluation to the paper’s central question: which computations does source prediction make available for rewards to use? 8 Conclusion Pretraining and midtraining can make a task learnable by revealing a mechanism that training rewards leave ambiguous and building the computation needed to use it. Our finite families connect this information gain to execution, retrieval and task learning along sampled-Adam paths from random initialization. With first-operation supervision, Sequential experiments demonstrate reward-task selection; Memory experiments connect retrieval retention to task use. Together with public-task comparisons, these results show why understanding learning from rewards requires tracking both the computation a model brings to the reward stage and how that stage uses it. References Abdulsalam et al. (2026) A. Abdulsalam, N. Patel, and A. Saxe RL Post-Training Builds Compositional Reasoning Strategies. Note: arXiv:2607.07646v1 External Links: Link Cited by: §1, §2.2. Alshiekh et al. (2018) M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu Safe Reinforcement Learning via Shielding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32. External Links: Document, Link Cited by: §2.1. Amani et al. (2021) S. Amani, C. Thrampoulidis, and L. Yang Safe Reinforcement Learning with Linear Function Approximation. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 243–253. External Links: Link Cited by: §2.1. Angluin (1987) D. Angluin Learning regular sets from queries and counterexamples. Information and Computation 75 (2), p. 87–106. External Links: Link Cited by: §2.1. Bai et al. (2023) Y. Bai, F. Chen, H. Wang, C. Xiong, and S. Mei Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection. In Advances in Neural Information Processing Systems, Vol. 36, p. 57125–57211. External Links: Document, Link Cited by: §2.3. Balcan et al. (2026) M. Balcan, A. Blum, K. Fragkia, Z. Li, and D. Sharma Online Learnability of Chain-of-Thought Verifiers: Soundness and Completeness Trade-offs. Note: arXiv:2603.03538v4 External Links: Link Cited by: §2.1. Balog et al. (2016) M. Balog, A. L. Gaunt, M. Brockschmidt, S. Nowozin, and D. Tarlow DeepCoder: Learning to Write Programs. Note: arXiv:1611.01989 External Links: Link Cited by: §2.4. Behrouz et al. (2024) A. Behrouz, P. Zhong, and V. Mirrokni Titans: Learning to Memorize at Test Time. Note: arXiv:2501.00663v1 External Links: Link Cited by: §2.3. Boix-Adsera et al. (2023) E. Boix-Adsera, O. Saremi, E. Abbe, S. Bengio, E. Littwin, and J. Susskind When can transformers reason with abstract symbols?. Note: arXiv:2310.09753v2 External Links: Link Cited by: §2.3. Carr et al. (2023) S. Carr, N. Jansen, S. Junges, and U. Topcu Safe Reinforcement Learning via Shielding under Partial Observability. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, p. 14748–14756. External Links: Document, Link Cited by: §2.1. Chen et al. (2024) S. Chen, H. Sheen, T. Wang, and Z. Yang Unveiling induction heads: provable training dynamics and feature learning in transformers. Note: arXiv:2409.10559v1 External Links: Link Cited by: §2.3. Chu et al. (2024) T. Chu, Z. Song, and C. Yang How to protect copyright data in optimization of large language models?. Proceedings of the AAAI Conference on Artificial Intelligence 38 (16), p. 17871–17879. External Links: Document, Link Cited by: §2.1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training Verifiers to Solve Math Word Problems. Note: arXiv:2110.14168 External Links: Link Cited by: §6.3. Cohen et al. (2019) J. Cohen, E. Rosenfeld, and Z. Kolter Certified Adversarial Robustness via Randomized Smoothing. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, p. 1310–1320. External Links: Link Cited by: §2.1. Cohen et al. (2026) O. I. Cohen, L. Shani, A. Rosenberg, A. Samanta, T. Wagner, and Y. Efroni Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias. Note: arXiv:2607.22837v1 External Links: Link Cited by: §2.4. Daliri et al. (2025) M. Daliri, Z. Song, and C. Yang Unlock the theory behind scaling 1-bit neural networks. In Conference on Parsimony and Learning, Proceedings of Machine Learning Research, Vol. 280, p. 545–598. External Links: Link Cited by: §2.3. Debenedetti et al. (2025) E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr Defeating Prompt Injections by Design. Note: arXiv:2503.18813v2 External Links: Link Cited by: §2.4. DeepSeek-AI et al. (2025) DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Note: arXiv:2501.12948v1 External Links: Link Cited by: §1. Deng et al. (2023) Y. Deng, Z. Song, S. Xie, and C. Yang Unmasking transformers: a theoretical approach to data recovery via attention weights. External Links: 2310.12462, Link Cited by: §2.1. Deng et al. (2024) Y. Deng, Z. Song, J. Xiong, and C. Yang How sparse attention approximates exact attention? your attention is naturally nCn^C-sparse. External Links: 2404.02690, Link Cited by: §2.3. Ergün et al. (1995) F. Ergün, S. R. Kumar, and R. Rubinfeld On learning bounded-width branching programs. In Proceedings of the Annual Conference on Computational Learning Theory, External Links: Link Cited by: §2.1. Fisac et al. (2015) J. F. Fisac, M. Chen, C. J. Tomlin, and S. S. Sastry Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraints. In Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control, External Links: Document, Link Cited by: §2.1. Foster et al. (2018) D. Foster, D. Reichman, and K. Sridharan Inference in sparse graphs with pairwise measurements and side information. In Proceedings of the International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 84, p. 1810–1818. External Links: Link Cited by: §2.1. Fu et al. (2023) H. Fu, T. Guo, Y. Bai, and S. Mei What can a Single Attention Layer Learn? A Study Through the Random Features Lens. Note: arXiv:2307.11353 External Links: Link Cited by: §2.3. Geifman and El-Yaniv (2017) Y. Geifman and R. El-Yaniv Selective Classification for Deep Neural Networks. In Advances in Neural Information Processing Systems, Vol. 30. External Links: Link Cited by: §2.1. Giapitzakis et al. (2025) G. Giapitzakis, K. Fountoulakis, E. Nichani, and J. D. Lee On the Statistical Query Complexity of Learning Semiautomata: a Random Walk Approach. Note: arXiv:2510.04115v1 External Links: Link Cited by: §2.1. Guardieiro et al. (2025) V. Guardieiro, A. Khare, A. Stein, and E. Wong Instruction Following by Principled Boosting Attention of Large Language Models. Note: arXiv:2506.13734 External Links: Link Cited by: §2.4. Guo et al. (2026) C. Guo, J. F. C. Uribe, S. Zhu, C. A. Choquette-Choo, S. Lin, N. Kandpal, M. Nasr, Rai, S. Toyer, M. Wang, Y. Yu, A. Beutel, and K. Xiao IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs. Note: arXiv:2603.10521v1 External Links: Link Cited by: §2.4. Gururangan et al. (2020) S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p. 8342–8360. External Links: Document, Link Cited by: §1, §2.2. Han and Ghoshdastidar (2026) Y. Han and D. Ghoshdastidar Transformers provably learn sparse XOR with polylogarithmic parameters. Note: arXiv:2502.07553v2 External Links: Link Cited by: §2.3. Hanna and Corrado (2025) J. P. Hanna and N. E. Corrado When Can Model-Free Reinforcement Learning be Enough for Thinking?. Note: arXiv:2506.17124v1 External Links: Link Cited by: §2.4. Huan and Yuan (2026) C. Huan and H. Yuan A mean-field analysis of multi-head self-attention under cross-entropy training. Note: arXiv:2606.10469v1 External Links: Link Cited by: §2.3. Huang et al. (2025a) R. Huang, Y. Liang, and J. Yang How transformers learn regular language recognition: a theoretical study on training dynamics and implicit bias. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 25516–25549. External Links: Link Cited by: §2.3. Huang et al. (2026a) Y. Huang, H. Zhu, Z. Wang, J. Jiao, S. Russell, S. Sojoudi, and S. Mei Transformers Provably Learn to Internalize Chain-of-Thought. Note: arXiv:2605.28600v1 External Links: Link Cited by: §2.2. Huang et al. (2026b) Y. Huang, Z. Wen, Y. Chi, Y. Wei, A. Singh, Y. Liang, and Y. Chen On the Emergence of Implicit Curriculum in RLVR Learning Dynamics. Note: arXiv:2602.14872v3 External Links: Link Cited by: §2.4. Huang et al. (2025b) Y. Huang, Z. Wen, A. Singh, Y. Chi, and Y. Chen Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §2.3. Junges et al. (2021) S. Junges, N. Jansen, and S. A. Seshia Enforcing Almost-Sure Reachability in POMDPs. In Computer Aided Verification, Lecture Notes in Computer Science, Vol. 12760, p. 602–625. External Links: Document, Link Cited by: §2.1. Ke et al. (2025) Y. Ke, Y. Liang, Z. Shi, Z. Song, and C. Yang Curse of attention: a kernel-based perspective for why transformers fail to generalize on time series forecasting and beyond. In Conference on Parsimony and Learning, Proceedings of Machine Learning Research, Vol. 280, p. 675–738. External Links: Link Cited by: §2.3. Kingma and Ba (2015) D. P. Kingma and J. Ba Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations, External Links: Link Cited by: §3. Kong et al. (2026) L. Kong, X. Liu, G. Chen, M. Q. Ma, X. Song, Y. Sun, M. Yurochkin, T. W. Killian, R. Salakhutdinov, K. Zhang, E. P. Xing, and Z. Liu From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning. Note: arXiv:2606.18089v2 External Links: Link Cited by: §2.2. Kumar et al. (2022) A. Kumar, A. Raghunathan, R. Jones, T. Ma, and P. Liang Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. Note: arXiv:2202.10054 External Links: Link Cited by: §2.3. Kumar et al. (2024) A. Kumar, V. Zhuang, R. Agarwal, Y. Su, J. D. Co-Reyes, A. Singh, K. Baumli, S. Iqbal, C. Bishop, R. Roelofs, L. M. Zhang, K. McKinney, D. Shrivastava, C. Paduraru, G. Tucker, D. Precup, F. Behbahani, and A. Faust Training Language Models to Self-Correct via Reinforcement Learning. Note: arXiv:2409.12917 External Links: Link Cited by: §2.4. Larian et al. (2026) A. Larian, Q. Lin, C. Z. Wu, and D. S. Brown Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning. Note: arXiv:2607.08647v1 External Links: Link Cited by: §2.1. Lavie et al. (2026) I. Lavie, K. Fischer, A. Lekov, F. Van Maele, Z. Ringel, and M. Helias Phase transitions in attention: a bayesian theory of copy head emergence. Note: arXiv:2606.12058v1 External Links: Link Cited by: §2.3. Liang et al. (2025) Y. Liang, Z. Shi, Z. Song, and C. Yang Towards infinite-long prefix in transformer. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 11127–11191. External Links: Document, Link Cited by: §1, §2.3. Liu et al. (2022) B. Liu, J. T. Ash, S. Goel, A. Krishnamurthy, and C. Zhang Transformers Learn Shortcuts to Automata. Note: arXiv:2210.10749 External Links: Link Cited by: §2.3. Liu and Herty (2026) H. Liu and M. Herty A mean-field theory of transformers: well-posedness of the coupled data–parameter dynamics and global convergence of training. Note: arXiv:2608.25055v1 External Links: Link Cited by: §2.3. Lyu et al. (2026) B. Lyu, Y. Jia, X. Cai, and Z. Zhu Transformers with RL or SFT provably learn sparse boolean functions, but differently. Note: arXiv:2511.17852v3ICML 2026 final version External Links: Link Cited by: §2.4. Modirshanechi et al. (2026) A. Modirshanechi, B. Eysenbach, P. Dayan, and E. Schulz Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization. Note: arXiv:2605.06145v1 External Links: Link Cited by: §2.4. Nichani et al. (2024a) E. Nichani, A. Damian, and J. D. Lee How Transformers Learn Causal Structure with Gradient Descent. Note: arXiv:2402.14735v2 External Links: Link Cited by: §2.3. Nichani et al. (2024b) E. Nichani, J. D. Lee, and A. Bietti Understanding Factual Recall in Transformers via Associative Memories. Note: arXiv:2412.06538v1 External Links: Link Cited by: §2.3. Paster et al. (2023) K. Paster, M. D. Santos, Z. Azerbayev, and J. Ba OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text. Note: arXiv:2310.06786 External Links: Link Cited by: §6.3. Phang et al. (2018) J. Phang, T. Févry, and S. R. Bowman Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks. Note: arXiv:1811.01088 External Links: Link Cited by: §2.2. Qwen et al. (2024) Qwen, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. External Links: Link Cited by: §6. Rajaraman et al. (2026) N. Rajaraman, A. Huang, M. Dudik, R. Schapire, D. Foster, and A. Krishnamurthy Learning to Reason with Curriculum I: Compositional Generalization. Note: arXiv:2606.27721v1 External Links: Link Cited by: §2.2. Ran-Milo et al. (2026) Y. Ran-Milo, Y. Alexander, S. Mendel, and N. Cohen Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Data. Note: arXiv:2601.15158v4 External Links: Link Cited by: §2.4. Ren et al. (2024) Y. Ren, Z. Wang, and J. D. Lee Learning and transferring sparse contextual bigrams with linear transformers. Note: arXiv:2410.23438NeurIPS 2024 External Links: Link Cited by: §2.3. Sanford et al. (2024a) C. Sanford, B. Fatemi, E. Hall, A. Tsitsulin, M. Kazemi, J. Halcrow, B. Perozzi, and V. Mirrokni Understanding Transformer Reasoning Capabilities via Graph Algorithms. Note: arXiv:2405.18512v1 External Links: Link Cited by: §2.3. Sanford et al. (2024b) C. Sanford, D. Hsu, and M. Telgarsky Transformers, parallel computation, and logarithmic depth. Note: arXiv:2402.09268v1 External Links: Link Cited by: §2.3. Sarangi and Salam (2025) S. Sarangi and H. Salam Small LLMs Do Not Learn a Generalizable Theory of Mind via Reinforcement Learning. Note: arXiv:2507.15788v1 External Links: Link Cited by: §2.4. Sarrof et al. (2026) Y. Sarrof, Y. Du, K. Stein, A. Koller, S. Thiébaux, and M. Hahn On the Ability of Transformers to Verify Plans. Note: arXiv:2603.19954v2 External Links: Link Cited by: §2.4. Saunshi et al. (2020) N. Saunshi, S. Malladi, and S. Arora A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. Note: arXiv:2010.03648 External Links: Link Cited by: §2.3. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: Language Models Can Teach Themselves to Use Tools. Note: arXiv:2302.04761 External Links: Link Cited by: §2.4. Sun et al. (2026) X. Sun, Q. Wang, H. Li, and L. Chen Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents. Note: arXiv:2608.03137v1 External Links: Link Cited by: §2.3. Tarzanagh et al. (2023) D. A. Tarzanagh, Y. Li, C. Thrampoulidis, and S. Oymak Transformers as Support Vector Machines. Note: arXiv:2308.16898 External Links: Link Cited by: §2.3. Tsilivis et al. (2025) N. Tsilivis, E. Malach, K. Ullrich, and J. Kempe How Reinforcement Learning After Next-Token Prediction Facilitates Learning. Note: arXiv:2510.11495v2 External Links: Link Cited by: §1, §2.2. Turchetta et al. (2016) M. Turchetta, F. Berkenkamp, and A. Krause Safe Exploration in Finite Markov Decision Processes with Gaussian Processes. In Advances in Neural Information Processing Systems, Vol. 29. External Links: Link Cited by: §2.1. Vasudeva et al. (2026) B. Vasudeva, P. Deora, A. Bietti, V. Sharan, and C. Thrampoulidis Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge. Note: arXiv:2603.20969v1 External Links: Link Cited by: §2.3. Volpano et al. (1996) D. Volpano, C. Irvine, and G. Smith A Sound Type System for Secure Flow Analysis. Journal of Computer Security 4. External Links: Document, Link Cited by: §2.1. Wallace et al. (2024) E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. Note: arXiv:2404.13208 External Links: Link Cited by: §2.4. Wang et al. (2019) P. Wang, P. Donti, B. Wilder, and Z. Kolter SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, p. 6545–6554. External Links: Link Cited by: §2.4. Wei et al. (2024) H. Wei, X. Liu, and L. Ying Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 21708–21716. External Links: Document, Link Cited by: §2.1. Wei and Kim (2026) S. Wei and J. Kim Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently. Note: arXiv:2606.22938v1 External Links: Link Cited by: §2.2. Weiss et al. (2019) G. Weiss, Y. Goldberg, and E. Yahav Learning deterministic weighted automata with queries and counterexamples. In Advances in Neural Information Processing Systems, Vol. 32. External Links: Link Cited by: §2.1. Xiong et al. (2025) J. Xiong, J. Shen, C. Zheng, Z. Wan, C. Zhao, C. Yang, F. Ye, H. Yang, L. Kong, and N. Wong ParallelComp: parallel long-context compressor for length extrapolation. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 68998–69016. External Links: Link Cited by: §2.3. Xiong et al. (2020) R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T. Liu On Layer Normalization in the Transformer Architecture. Note: arXiv:2002.04745 External Links: Link Cited by: §2.3. Xue et al. (2024) A. Xue, A. Khare, R. Alur, S. Goel, and E. Wong Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference. Note: arXiv:2407.00075 External Links: Link Cited by: §2.4. Yalcinkaya et al. (2025) B. Yalcinkaya, N. Lauffer, M. Vazquez-Chanlatte, and S. A. Seshia Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learning. Note: arXiv:2503.05042v2 External Links: Link Cited by: §2.1. Yang (2025) C. Yang Unifying learning dynamics and generalization in transformers scaling law. Note: Version 3, revised June 2026 External Links: 2512.22088, Link Cited by: §1, §2.3. Yang et al. (2023) K. Yang, A. M. Swope, A. Gu, R. Chalamala, P. Song, S. Yu, S. Godil, R. Prenger, and A. Anandkumar LeanDojo: Theorem Proving with Retrieval-Augmented Language Models. Note: arXiv:2306.15626 External Links: Link Cited by: §2.4. Yang et al. (2026) T. Yang, Y. Huang, Y. Liang, and Y. Chi Agentic Transformers Provably Learn to Search via Reinforcement Learning. Note: arXiv:2606.00183v1 External Links: Link Cited by: §2.4. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. Note: arXiv:1809.09600 External Links: Link Cited by: §6.3. Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: Synergizing Reasoning and Acting in Language Models. Note: arXiv:2210.03629 External Links: Link Cited by: §2.4. Yue et al. (2025) Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, Y. Yue, S. Song, and G. Huang Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?. Note: arXiv:2504.13837 External Links: Link Cited by: §2.4. Zeng et al. (2026) J. Zeng, D. Fu, T. Mi, Y. Zhuang, Y. Huang, X. Li, L. Ye, M. Xie, Q. Hua, Z. Huang, M. Jiang, H. Wang, J. Lin, Y. Xiao, J. Sun, Y. Wu, and P. Liu daVinci-Dev: agent-native midtraining for software engineering. arXiv preprint arXiv:2601.18418. External Links: Link Cited by: §1, §2.2. Zhou et al. (2025) J. P. Zhou, K. Wang, J. Chang, Z. Gao, N. Kallus, K. Q. Weinberger, K. Brantley, and W. Sun Q♯Q : Provably Optimal Distributional RL for LLM Post-Training. Note: arXiv:2502.20548v2 External Links: Link Cited by: §2.4. Appendix Appendix A Experimental details and supplementary results World/training seeds are the independent units throughout; shared source trajectories, evaluation prompts and checkpoints are repeated measurements within those units. Item-level results, configurations and analysis code accompany the paper. A.1 Output supervision and source replay We use eight fresh worlds (seeds 500–507) for each module, Qwen2.5-0.5B, and a shared task-independent format stage followed by source training and reward adaptation. The format stage has 128 updates for Sequential and 32 for Memory, at learning rate 10−510^-5. Source training also uses 10−510^-5: Sequential receives 2222^22 observed tokens in 512 updates, and discrete Memory receives 2242^24 in 2,048 updates. There is no format stage after source training. Sequential crosses correct versus public-preserving/private-randomized source with state/END/EOS targets versus first-operation/state/END/EOS targets. The new target is an already observed random operation, independent of the private task binding; initial-state and suffix inputs are not prediction targets. Memory crosses correct versus wrong-retrieval source with pure reward adaptation versus source replay. Both retrieval routes use the all-token source objective. Replay samples only complete records previously observed by that route, with an independent random stream, and supplies exactly 1,024 source prediction targets after each reward update. It shares the branch’s Adam state and learning rate, 3×10−63× 10^-6, with reward training. Each replay branch thus uses an additional 131,072 source targets and 128 cross-entropy updates. Each source checkpoint initializes two reward-trained branches, one for task A and one for task B. Relative to A, B flips the Sequential binding’s private bit, or negates all Memory context signs; mechanism and prompt distribution remain fixed. Every branch receives 8,192 verifier queries in 128 batches of 64, at fixed learning rate 3×10−63× 10^-6. We score unmodified generated outputs with the strict verifier. At 0, 1,024, 2,048, 4,096 and 8,192 queries, all routes and both task-trained models use the same fixed 1,024 held-out evaluation records within a world. Source probes are repeated at these points. There are 16 format stages, 48 source stages and 128 reward branches; shared ancestors do not create additional independent samples. Across both modules the reward-training budget is 1,048,576 queries. The two primary paired effects compare correct and matched-control task selectivity under first-operation supervision for Sequential and under source replay for Memory. We first average the task-A and task-B contributions within each world, then report the eight world-level paired differences, a 95% Student-t interval, and a two-sided exact sign-flip test. Exact sign-flip inference assumes sign symmetry of paired world effects under the null. Holm correction applies jointly to these two primary tests. The target-objective and replay interactions are secondary comparisons. These samples and this test family are separate from the earlier eight-world acquisition study. Resources, implementation and evaluation units. Sequential state-only and action-plus-state objectives share observed source records, but adding operation targets increases optimized target positions. Observed tokens, optimized prediction targets and optimizer calls are therefore distinct resources. Memory pure/replay A/B branches reuse the same route-specific source ancestor. Replay uses that ancestor’s recorded labels, excludes partial boundary and held-out records, and preserves the reward-prompt random stream. A replay branch has 128 RL calls and 128 source CE calls, versus 128 RL calls for pure training. Both modules use BF16 model evaluation with FP32 master parameters and Adam moments in this eight-world comparison. Sampled completions use the generation backend; response log probabilities for updates are recomputed by the trainer. The constructions establish finite learning paths in the specified architectures and schedules; these Qwen experiments measure acquisition and reward-dependent use in pretrained language models. The 1,024 evaluation items in a world are fixed across routes, tasks and checkpoints. Sampling can repeat a prompt: Sequential worlds 501 and 502 each contain 1,023 distinct prompt texts among 1,024 item IDs; the remaining worlds contain 1,024. These repeats are identical across paired routes and are retained with their original sample multiplicities. The independent statistical unit is the world, not an item, A/B branch or checkpoint. Source probes use 512 Sequential records with first operation supplied, and 256 Memory records with matched relevant-value interventions. These probes measure execution of the source computation on held-out records. Primary and secondary effects. Table 3 reports the complete predeclared two-test family; Table 4 reports every predeclared secondary contrast. The Sequential primary effect compares correct and private-random source within the action objective. The source-by-objective interaction measures how adding action supervision changes the source-control selectivity advantage. Its Student-t interval is positive, while the unadjusted exact sign-flip test gives p=.08594p=.08594 over the eight world effects. Table 3: Predeclared source-control effects on task selectivity (percentage points), eight paired worlds per module. Two-sided exact sign-flip tests; Holm adjustment over both primary tests. Module Effect 95% CI Exact p Holm p Sequential 67.57 [37.04, 98.10] 0.007812 0.015625 Memory 0.67 [-1.22, 2.57] 0.640625 0.640625 Table 4: All predeclared secondary contrasts at 8,192 queries (percentage points). Eight paired world differences; 95% Student-t intervals and unadjusted exact sign-flip tests. Module Comparison Metric Effect 95% CI Exact p Seq. Correct: action–state Select. 43.55 [1.47, 85.64] 0.08594 Seq. Correct: action–state Success 54.62 [23.75, 85.50] 0.01562 Seq. Control: action–state Select. 0.96 [-0.09, 2.02] 0.04688 Seq. Control: action–state Success 31.74 [13.15, 50.32] 0.01562 Seq. Source interaction Select. 42.59 [0.82, 84.36] 0.08594 Seq. Source interaction Success 22.88 [-1.44, 47.20] 0.08594 Mem. Correct: replay–pure Select. -0.06 [-0.52, 0.40] 1.00000 Mem. Correct: replay–pure Success -0.62 [-5.54, 4.31] 0.62500 Mem. Control: replay–pure Select. -0.59 [-2.45, 1.27] 1.00000 Mem. Control: replay–pure Success -1.45 [-6.13, 3.22] 0.56250 Mem. Source interaction Select. 0.52 [-1.40, 2.45] 0.87500 Mem. Source interaction Success 0.84 [-6.20, 7.87] 0.68750 Table 5: Original eight-world task success: reward entry, within-world reward gain, and final score at 8,192 queries. A/B branches are averaged within each world; means and 95% Student-t half-widths over eight worlds. Entry and final are percentages; gain is percentage points. Module Route Entry Gain Final Sequential Correct/state 0.20±0.460.20\,± 0.46 27.80±30.3727.80\,± 30.37 27.99±30.4827.99\,± 30.48 Sequential Control/state 0.00±0.000.00\,± 0.00 12.41±15.5712.41\,± 15.57 12.41±15.5712.41\,± 15.57 Sequential Correct/action 4.83±6.724.83\,± 6.72 77.78±17.7277.78\,± 17.72 82.61±16.8082.61\,± 16.80 Sequential Control/action 2.32±3.392.32\,± 3.39 41.83±11.1241.83\,± 11.12 44.15±9.7144.15\,± 9.71 Memory Correct/pure 49.70±0.7149.70\,± 0.71 −1.54±3.80-1.54\,± 3.80 48.16±3.6148.16\,± 3.61 Memory Control/pure 41.41±11.9341.41\,± 11.93 2.35±9.532.35\,± 9.53 43.76±14.7543.76\,± 14.75 Memory Correct/replay 49.70±0.7149.70\,± 0.71 −2.16±2.82-2.16\,± 2.82 47.54±3.4447.54\,± 3.44 Memory Control/replay 41.41±11.9341.41\,± 11.93 0.90±5.180.90\,± 5.18 42.30±11.8442.30\,± 11.84 Absolute success and task selection. Correct-source Sequential action models finish at 78.09% and 87.13% on tasks A/B, versus 38.39% and 49.90% under private-random source. Averaging A/B within world gives 82.61% versus 44.15%, a paired difference of 38.46 p (95% interval [19.78,57.15][19.78,57.15]). These absolute-success summaries complement the task-selectivity primary test. Here the source advantage is measured against the private-random source control. Correct-source Memory replay retains the acquired source response: original/flip-pair accuracy is 92.21%, compared with 0% under pure RL, with a paired retention-difference interval of [87.93,96.50][87.93,96.50] p. First-field accuracy is 96.24% versus 49.66%. Correct source exceeds wrong retrieval in final task accuracy by 4.40 p under pure RL (95% interval [−11.38,20.18][-11.38,20.18]) and 5.24 p with replay ([−8.10,18.58][-8.10,18.58]). These endpoint intervals and the primary task-selectivity result leave the task advantage unresolved at this budget. Replay preserves the measured source computation, while its task-specific use remains a separate outcome. Figure 5: Correct and matched-control task selectivity at 8,192 reward queries: (a) Sequential action objective; (b) Memory source replay. Small dots are eight worlds, gray lines pair worlds, and diamonds give route means with untruncated 95% intervals. Primary inference uses paired differences, not overlap of route intervals. Figure 6: Sequential learning curves for (a) task A and (b) task B, with correct/private-random source crossed with state/action-plus-state targets. Each point averages eight worlds on 1,024 fixed evaluation items per world; bands are untruncated 95% Student-t intervals. Figure 7: Memory learning curves for (a) task A and (b) task B. Correct/wrong retrieval is crossed with pure reward training/source replay; the same ancestors and evaluation items are paired. Means and untruncated 95% intervals over eight worlds. Figure 8: (a) Legal first-operation output and (b) complete syntax on the fixed held-out reward inputs. Four source routes, A/B averaged within world, with 95% intervals across eight worlds. Correct syntax and task correctness are separate metrics. Figure 9: (a) Strict complete source trajectory, (b) required state-prefix content and (c) termination, on 512 fixed source probes per world with the first operation supplied. A/B are averaged within world; bands are 95% intervals over eight worlds. Figure 10: (a) Correct first field on 256 source records and (b) joint correctness on 256 original/relevant-flip pairs. A/B are averaged within each of eight worlds; bands are untruncated 95% intervals. Pure/replay branches share ancestors; replay contributes 131,072 additional source targets. Figure 11: Sequential training diagnostics: (a) positive training rewards, (b) nonzero RL updates and (c) nonzero source CE updates (zero for these routes). Cumulative counts average A/B within world, then eight worlds, with 95% intervals. Logged calls are distinguished from measured nonzero updates. Figure 12: Memory training diagnostics: (a) positive training rewards, (b) nonzero RL updates and (c) nonzero source CE updates. A/B are averaged within world, with 95% intervals over eight worlds. CE calls do not increment the verifier-query count. (a) Reward learning(b) Source-selectivity effect Figure 13: Complete original Sequential comparison. (a) State-only and action-plus-state source objectives, correct and private-random source, on 1,024 fixed held-out records. (b) Action-objective source-selectivity effects for all eight worlds, with their mean and 95% interval. A/B are averaged within world. World-level results correspond to Tables 3 and 4. A.2 Independent Memory confirmation: retrieval that rewards can use We test task-relevant retrieval on eight new worlds, 508–515, using Qwen2.5-0.5B and the discrete Memory representation. Each world has four reward branches: selected/other retrieval crossed with opposite tasks A/B. The two interventions start from the same correct-source checkpoint and receive matched additional training; A/B then branch from the same intervention endpoint. The experiment asks whether preparing the relevant retrieval computation supports reward-task selection under a fixed adaptation recipe. Source interventions. After format preparation and 2242^24 correct-source tokens, each route observes 131,072 additional records in 2,048 batches of 64. The selected route predicts the sign of the sum of the 15 content coordinates in the task-relevant memory slot; the other route predicts the same public projection of the other slot. Prompts are identical across routes. These targets depend on the retrieval mechanism, without the downstream contextual signs or cue-rule coefficients. Both routes therefore inherit correct source information; the comparison changes which retrieval computation the additional stage trains. A second source stage uses 65,536 record exposures in 1,024 batches of 64, adding a retrieval-vector objective to next-token CE. For context c∈1,…,4c∈\1,…,4\, the target z∈ℝ60z ^60 places the retrieved 15-coordinate value in block c and sets the other blocks to zero. The route is inferred from observed source input/bit pairs. With final-prefix hidden state h, a fixed, data-independent 60×89660× 896 orthonormal-row projection P (NumPy seed 970506) predicts P(h/RMS(h))P(h/RMS(h)); the auxiliary loss is ‖P(h/RMS(h))−z‖2/15\|P(h/RMS(h))-z\|^2/15, averaged over records and added to CE with coefficient one. Here RMS(h)=max(‖h‖2/d,10−12)RMS(h)= (\|h\|^2/d,10^-12). The projection is a training buffer, absent from generated answers. These stages train all model parameters at nominal learning rate 10−510^-5; the vector stage uses the step safeguard below. Reward adaptation and representation retention. Each branch uses 8,192 fresh on-policy reward queries: 128 batches of 64, REINFORCE baseline 1/21/2, and nominal Adam learning rate 3×10−63× 10^-6. Sampling and scoring use the same full-vocabulary autoregressive policy. After each reward update, 256 previously observed prefixes supply the same retrieval-vector loss and a syntax objective. Each prefix is paired with both supplied signs; CE supervises only the three public suffix tokens after each sign. Thus this step uses 1,536 CE targets and 15,360 vector components, without CE supervision of the answer sign. Adam moments are reset at stage boundaries and shared by reward and retention updates within each branch. At the intervention endpoint, temperature is calibrated using 256 unlabeled task-training prompts to mean conditional sign entropy 0.5 bit, then fixed throughout adaptation. If the original next-token probabilities of the two sign tokens are p+,p−p_+,p_-, write m=p++p−m=p_++p_- and d=log(p+/p−)d= (p_+/p_-). The processed probabilities are p+′=mσ(d/T),p−′=m[1−σ(d/T)],p _+=m\,σ(d/T), p _-=m\,[1-σ(d/T)], with all other token probabilities unchanged. Only the first sign position is transformed. The temperatures range from 3.7381 to 4.8753; training samples, sequence log probabilities and greedy evaluation use the same processing. Calibration uses no reward labels or held-out examples. For each guarded Adam proposal, the displacement is scaled by the first accepted value in 1,10−1,10−2,10−3,10−4,0\1,10^-1,10^-2,10^-3,10^-4,0\. Acceptance requires finite values, loss at most its pre-update value plus 10−710^-7, and mean first-position full-vocabulary DKL(πold∥πnew)≤.01D_ KL( _ old\| _ new)≤.01 on the update prompts. The guard scales parameter displacement; Adam’s updated moments are retained. Every branch makes 128 actual reward updates; 121–128 of its 128 retention attempts change parameters. These settings define the complete recipe used for both source interventions. Budgets. The shared warm start uses 16,777,216 observed source tokens. Per intervention, the public-projection stage adds approximately 20.56–20.60 million observed tokens in 131,072 records; the vector stage adds 65,536 record exposures. Per reward branch, retention adds 196,608 CE targets and 1,966,080 vector components in 128 attempts, separately from 8,192 reward queries. The 32 branches use 262,144 new training queries in total. These costs are additional to, and separately recorded from, the original experiment in Section A.1. The contrast evaluates task-relevant versus alternative retrieval under the complete recipe; individual components are not separately ablated. Evaluation and paired inference. Each branch is evaluated greedily at 0, 1,024, 2,048, 4,096 and 8,192 reward queries on 1,024 fixed held-out records and 256 separate training-distribution records. Both task verifiers score each output. A/B scores are averaged within world before taking world means or differences; the eight worlds are the independent units. The single predeclared primary effect is selected-minus-other task selectivity Δtask _ task as defined in Section 6. Its exact two-sided sign-flip test enumerates all 282^8 sign assignments and remains separate from the earlier two-test Holm family. Intervals use Student’s t with seven degrees of freedom. Selected retrieval reaches 75.32% held-out success, compared with 49.86% for other retrieval; both routes enter reward training at 50.00% after averaging A/B within world. The paired accuracy difference is 25.46 p (95% CI 20.10–30.81). All eight world differences are positive. The primary selectivity difference is 50.92 p (95% CI 40.20–61.63), exact p=2/256=.0078125p=2/256=.0078125. All 32 branches have 100% format accuracy on their final evaluation records. With valid binary answers and opposite task labels, JAB=1−JAAJ_AB=1-J_A and JBA=1−JBBJ_BA=1-J_B, so the selectivity difference is exactly twice the accuracy difference. Figures 14 and 6 display every world and both task branches. (a) Separate A/B branches (b) Selected minus other Figure 14: Independent Memory confirmation. (a) Separate task-A and task-B success curves for selected and other retrieval; means and 95% Student-t bands over eight worlds, using 1,024 held-out records per branch. (b) Each point is a world’s selected-minus-other task-selectivity effect after averaging A/B; black diamond and error bar give the mean and 95% interval. All fixed evaluation points and all worlds are retained. Table 6: Independent Memory confirmation: all 32 branch endpoints after 8,192 reward queries. Success is a percentage of 1,024 fixed held-out records per branch. Differences first average A/B within world; all eight worlds are included. World Selected A Selected B Other A Other B Accuracy Δ Selectivity Δ 508 73.34 75.10 52.44 50.68 22.66 45.31 509 83.69 91.31 49.32 49.32 38.18 76.37 510 80.66 81.84 49.80 49.41 31.64 63.28 511 70.90 73.05 47.56 48.63 23.88 47.75 512 73.54 71.09 50.10 50.59 21.97 43.95 513 73.83 72.27 48.83 50.00 23.63 47.27 514 73.83 66.31 52.54 52.05 17.77 35.55 515 71.68 72.66 48.73 47.75 23.93 47.85 A.3 Independent Sequential parameter-plane measurement This descriptive comparison uses the prespecified world 508 and Task A, separately from the formal worlds 500–507. Three routes share a Qwen2.5-0.5B checkpoint after 128 task-independent format updates. Direct RL skips added source training. Correct source and the public-preserving/private-random control each use 2222^22 observed tokens and 512 source updates, predicting the record’s first operation, subsequent states and END/EOS. The input initial state and suffix operations are not supervised targets under this objective. Each reward branch uses learning rate 3×10−63× 10^-6, batch size 64 and 128 updates, totaling 8,192 verifier queries. The three branches make 128 nonzero updates each and receive 3,874, 7,668 and 3,786 positive training rewards, respectively. Additional source cost is separate from the common inherited checkpoint and equal reward budgets. Actual learning and evaluation units. Figure 15 reports greedy strict success on 1,024 fixed held-out records, balanced between suffix lengths four and six, at 0, 1,024, 2,048, 4,096 and 8,192 reward queries. Source-endpoint task success is distinct from a source-computation probe: the correct-source model has 0% task success before RL and 100% after 1,024 queries. Final advantages are 48.24 percentage points over direct RL and 48.73 over the private-random source control. The bands use 2,000 item-bootstrap resamples stratified by suffix length, with shared indices across routes and checkpoints. These intervals describe item uncertainty conditional on the prespecified world and trained run; they collapse at all-correct or all-incorrect endpoints. This single-world comparison is analyzed separately from the eight-world tests. Figure 15: Actual reward learning and projection fidelity in world 508. Left: original-checkpoint success on 1,024 fixed records; bands are conditional 95% item-bootstrap intervals. Right: original versus reconstructed checkpoint success on the same 256 slice-evaluation inputs. Squares mark source checkpoints and circles mark RL checkpoints; overlapping points are retained. The diagonal denotes equal scores. The source routes receive 2222^22 additional tokens before RL; direct RL skips this stage. All three routes share format preparation and use 8,192 reward queries. One common plane and complete measurement grid. Let ϑ0 _0 denote the shared format checkpoint. We use 17 distinct checkpoints: the origin, both source routes at 2202^20 and 2222^22 tokens, and all three reward routes at 1,024, 2,048, 4,096 and 8,192 queries. Shared source-to-reward boundaries use the same weights and are counted once. An uncentered rank-two SVD of displacements ϑi−ϑ0 _i- _0 across all these states gives orthonormal directions v1,v2v_1,v_2. A checkpoint is projected to coordinates αij=⟨ϑi−ϑ0,vj⟩ _ij= _i- _0,v_j and reconstructed as ϑ~i=ϑ0+αi1v1+αi2v2 _i= _0+ _i1v_1+ _i2v_2. Grid models use the same affine plane and are cast to the evaluation model’s dtype. We evaluate the 81 points of the fixed 9×99× 9 grid, all 17 original checkpoints and their 17 reconstructions. Every measurement uses the same 256 held-out inputs (128 of each suffix length) and 128 fixed correct-source records. Source loss is the summed negative log-likelihood over 3,856 supervised first-operation/state/END/EOS targets divided by that target count, in nats per target. The colored cells in Figure 2 are measured grid values; only the white source-loss contours interpolate between grid values. Geometric coverage and behavioral fidelity. The plane retains 91.96% of ∑i‖ϑi−ϑ0‖2 _i\| _i- _0\|^2. For the direct-RL endpoint alone, the relative residual ‖ϑi−ϑ~i‖/‖ϑi−ϑ0‖\| _i- _i\|/\| _i- _0\| is 89.63%, leaving only 19.67% of its squared displacement in the plane. Aggregate coverage thus masks route-wise reconstruction loss. On the common 256 inputs, actual/reconstructed success is 0.00%/100.00% at the correct-source endpoint, 0.00%/52.73% at the private-random source endpoint, and 53.91%/19.14% at the direct-RL endpoint. The two source-plus-RL endpoints agree at 100.00% and 52.73%, respectively. These denominators differ from the full 1,024-record learning curves. Because the basis includes later RL checkpoints, reconstructing a source checkpoint introduces directions learned during reward adaptation. The grid describes the behavior of reconstructed models in this shared plane; the original-checkpoint curves measure the actual training behavior. Together they relate the measured slice to learning on task A, while the paired A/B experiments measure task selectivity. A.4 Eight-world source acquisition and reward adaptation This comparison separates learning a source computation from using it under a new reward. Each module uses eight independently generated worlds (seeds 300–307), two source routes and two reward tasks A/B per route. Sequential uses its original representation, comparing the correct source with a source that preserves every public state and randomizes the private bits. Memory uses discrete signed-permutation keys and ±1± 1 values, comparing correct retrieval with wrong retrieval under the same all-token objective. Inputs and source exposure are matched within each world; A/B branches share the same source checkpoint. Training and evaluation. All runs use Qwen2.5-0.5B Base and full-parameter Adam. A task-independent format stage precedes source training: 128 updates for Sequential and 32 for Memory, batch 64, learning rate 10−510^-5. Sequential then observes 2222^22 source prediction tokens in 512 updates, optimizing state tokens and the existing END/EOS targets; Memory observes 2242^24 tokens in 2,048 updates, optimizing all tokens. Source learning rate is 10−510^-5 for both. No subsequent format stage is applied. Each source checkpoint branches directly into tasks A/B, with fresh Adam moments, fixed learning rate 3×10−63× 10^-6 and 8,192 verifier queries (128 batches of 64 sampled completions). The strict reward and output parser are unchanged. Greedy evaluation uses 1,024 fixed held-out prompts at 0, 1,024, 2,048, 4,096 and 8,192 queries; both task verifiers score every output. All 64 reward branches finish the full budget, totaling 524,288 training queries. Source computation. The source-stage diagnostics in Figure 16 use 512 Sequential prompts per world, with the first operation given and horizons five and seven equally represented. A complete answer contains all correct states and END in exactly the required positions. Memory uses 256 original source records and 256 corresponding relevant-value interventions per world; a pair is correct only when both first-field answers are correct. These are source-distribution probes without the downstream task binding. Across eight worlds, correct minus control differences are 87.5087.50 percentage points for Sequential complete trajectories (95% paired t interval [62.89,112.11][62.89,112.11]), 44.4844.48 for Memory original records ([31.92,57.05][31.92,57.05]), and 88.2888.28 for Memory intervention pairs ([60.57,115.99][60.57,115.99]). These source diagnostics are distinct from the two primary reward-task comparisons below. The t intervals are untruncated; their endpoints can exceed the bounded metric range. Figure 16: Historical source acquisition, worlds 300–307. (a) Sequential complete trajectories on 512 probes with first operation supplied; (b) Memory first-field correctness and (c) original/value-flip pair correctness, each on 256 records or pairs. Correct source and matched structural controls are paired within world; points are worlds, diamonds means with untruncated 95% Student-t intervals. This protocol is separate from worlds 500–507 in Section A.1. Task selection. For each route and world, selectivity is 12[(JAA−JAB)+(JBB−JBA)] 12[(J_A-J_AB)+(J_B-J_BA)], where JXYJ_XY is the success of the task-X branch under verifier Y. The two primary effects subtract the matched-control selectivity from correct-source selectivity at 8,192 queries. We average eight world-level differences, use Student-t intervals with seven degrees of freedom, enumerate all 282^8 sign flips for two-sided tests, and apply Holm correction to these two hypotheses. This two-test family is separate from the earlier comparisons. Neither interval excludes zero (Tables 7 and 18). Sequential world effects are [0,0,50,0,−1.123,0,0,−1.172][0,0,50,0,-1.123,0,0,-1.172] percentage points, so its positive mean is concentrated in one world. Full learning curves appear in Figure 17. Table 7: Correct-source minus matched-control task selectivity after 8,192 reward queries, eight worlds per module. Effects and 95% paired Student-t intervals are in percentage points; p is the exact two-sided sign-flip value and pHp_H applies Holm correction to the two comparisons. Module Effect 95% interval p pHp_H Sequential 5.965.96 [−8.92,20.85][-8.92,20.85] 1.0001.000 1.0001.000 Memory −0.87-0.87 [−2.10,0.35][-2.10,0.35] 0.1250.125 0.2500.250 (a) Sequential(b) Memory Figure 17: Held-out success during reward adaptation, eight worlds per route. Colors distinguish correct source and the matched control (random private states for Sequential; wrong retrieval for Memory); solid and dashed lines denote tasks A and B. Markers are world means on 1,024 prompts per world at each query budget, with 95% Student-t bands. Training samples at temperature one; evaluation is greedy. Intervals are shown without clipping to the success-rate range. (a) Sequential(b) Memory Figure 18: Correct-source minus matched-control task selectivity at 8,192 queries. Each dot is one complete world-level A/B contrast; worlds 1–8 correspond to seeds 300–307. The blue line is their mean, the shaded band its 95% Student-t interval, and the gray line marks zero. Units are percentage points. Both comparisons include all eight worlds; vertical scales differ between panels. Computation during reward adaptation. Figure 19 repeats the fixed source probes through the reward stage. A/B probe scores are averaged within each world before forming the eight-world mean and interval. Correct-source Sequential complete trajectories change from 89.60% to 88.94%, with END placement at 100% throughout. Twenty-six of the 32 Sequential reward branches receive no positive reward and make no parameter updates, so most branches retain the source checkpoint unchanged. Correct-source Memory first-field accuracy changes from 94.38% to 50.34% and relevant-pair accuracy from 89.01% to 0%, while its source-probe public-format accuracy stays at 100%. All 32 Memory branches make 128 nonzero updates. Thus computation acquired on source inputs and its expression after reward adaptation are separately observable properties. (a) Sequential trajectory(b) Memory record(c) Memory flip pair Figure 19: Source-computation probes through reward adaptation, using the same metrics and matched controls as Figure 16. Lines join the five measured query budgets; shaded bands are 95% Student-t intervals over eight worlds after averaging A/B within each world. Shared ancestors are not counted as additional independent samples. Sequential probes supply the first operation; Memory pairs require both the original and relevant-value-flipped bit to be correct. The following subsections describe the earlier controlled-task and public-task comparisons; the eight-world source-acquisition design above has its own source objectives and budgets. A.5 Earlier execution, task-selection and interface comparisons These earlier comparisons identify which part of learning changes with source preparation. They use their own objectives, budgets and checkpoints, described below. Execution and task choice. The endpoint decomposition separates a valid public trajectory from its private answer, while opposite-reward branches test task-specific selection (Figures 20, A.12 and A.8). This distinguishes learning to execute from learning which execution the task requires. Figure 20: Historical Sequential execution and private-task adaptation, using 2202^20 source prediction tokens. (a) Greedy success on 1,024 held-out prompts per world: eight worlds for correct source, wrong source and direct RL, and three for format control. (b) Final private-bit accuracy among outputs with valid public execution and the correct public endpoint, computed within each world before averaging; direct and format routes have no eligible outputs. (c) Shared-source A/B branches in three worlds: paired task selectivity Δtask _ task in percentage points (p). Small translucent dots show individual worlds; connected markers in (a) and large markers in (b,c) show world means. Shading and error bars are 95% Student-t intervals across worlds; horizontal jitter separates overlapping points. Dotted lines mark 50% private accuracy in (b) and zero task selectivity in (c). Acquisition and stage order. Memory source probes test the retrieved-value rule directly (Section A.8). The separate two-world study measures state content, termination and value-flip responses before and after format training (Section A.14). Together these diagnostics locate the computation available at each stage. Initialization, budgets and learning routes. Section A.13 compares source dose, inherited versus random initialization, model scale, continued prediction, answer supervision and test-time selection. Section A.12 gives the output-level decomposition and sampled-decoding comparison, with the full denominators and uncertainty. A.6 Task definitions and observation permissions For sequential execution, a state is (s,z)∈0,1×23(s,z)∈\0,1\×F_2^3, and an operation (b,u)(b,u) maps it to (s⊕b⊕θ(u,z),z⊕u)(s b θ(u,z),z u). Here ⊕ is addition modulo two. Writing u=(t,x,y)u=(t,x,y), let a(u)=t⊕xya(u)=t xy. Displayed suffixes have length L∈4,6L∈\4,6\; the actual execution horizon is H=L+1H=L+1, including the hidden initial operation. Training and validation draw independently from suffixes with ⨁ja(uj)=0 _ja(u_j)=0; held-out suffixes have parity one. The paired mechanism replaces θ(u,z)θ(u,z) by θ(u,z)⊕a(u)θ(u,z) a(u) and compensates the hidden initial binding. The terminal training-reward function is unchanged. Evaluation has 512 prompts at each length. A source record serializes the initial state, the operation suffix, the independently uniform first operation, and the resulting states. At each transition a reset occurs with probability (1/64)/H(1/64)/H, replacing the state uniformly. Every source token is a next-token prediction target, including random fields and EOS. In textual outputs, END is an explicit field terminator, distinct from the tokenizer’s end-of-sequence marker. The verifier checks output syntax, the public z trajectory, and the final state; correct intermediate answer bits are not required. Source data do not contain the downstream hidden binding. The contextual memory experiment uses four hidden orientations and a 15-dimensional shared cue rule, as in the contextual model. Source records contain random keys, a transformed query, Gaussian values, and the raw future bit, with no task-dependent signs. Target memory amplitudes separate training from four equally weighted test configurations (±1,±0.875)(± 1,± 0.875). Equal-sign and opposite-sign subsets are reported separately. Language-model inputs use textual KEY, QUERY, VALUE, and BIT fields, with numeric values rendered to eight significant digits. The language-model objective predicts all serialized fields; the contextual theorem trains on the future-bit position of typed numerical input. The information panel uses eight paired worlds and nested source record counts 0,1,2,4,8,16,32,640,1,2,4,8,16,32,64. The contextual curve 14(34)N 14( 34)^N is the analytic expected error floor; the measured coverage statistic is the number of contexts actually observed. The paired-world construction is verified by enumeration. A.7 Models, training and evaluation The models are Qwen2.5-0.5B Base and Qwen2.5-1.5B Base, with all parameters trainable; Table 8 lists the comparisons and their independent seeds. Source training uses 8,192 prediction targets per update, with saved states at 218,219,2202^18,2^19,2^20 targets. Feedback uses one fresh batch of 64 sampled completions per REINFORCE update, for 8,192 verifications in total. Its loss is the negative batch mean of reward times the sum of sampled completion log probabilities, including EOS, with baseline zero; zero-reward batches still advance Adam moments. There is no reward filtering, Kullback–Leibler (KL) penalty, weight decay or gradient clipping. Parameters and Adam moments are stored in bfloat16 with 32-bit residual, normalization and probability computations; sampled completions are produced by SGLang, with trainer-side response log probabilities recomputed for the update. Contexts are limited to 2,048 tokens and continuations to 128 new tokens. Sampling uses temperature one without top-k or top-p truncation. Feedback begins from the source parameters with reset moments. Learning rates are selected on two pilot worlds disjoint from the evaluated seeds, by source validation cross-entropy over 10−5,3⋅10−5,10−4\10^-5,3· 10^-5,10^-4\ and then by validation feedback on the allowed training support over 10−6,3⋅10−6,10−5\10^-6,3· 10^-6,10^-5\, with smaller rates breaking ties. The selected rates (Table 9) are shared by all routes within a task, model and initialization. Evaluation occurs at 0,1024,2048,4096,81920,1024,2048,4096,8192 training queries. Each checkpoint has 1,024 fixed paired test prompts, evaluated with both greedy and sampled decoding; raw outputs, success, public-path and format errors, and truncation are saved. BBH uses every item from the two three-object tasks, a zero-shot option-only prompt, greedy decoding, and a strict option parser at the base, source, and final reward checkpoints. Table 8: Experimental comparisons. Source-only measurements reuse source checkpoints; the fixed external base model contributes one checkpoint. Comparison Independent seeds and scope Direct RL, correct source, paired wrong source 8 sequential and 5 contextual, 0.5B Random initialization and 1.5B transfer 3 sequential seeds per three-arm panel Source dose 5 sequential seeds, 0.5B, shared source Alternative learning routes 3 sequential LM seeds Alternate downstream task choice 3 sequential seeds, shared source states Format-only source then RL 3 sequential seeds BBH checkpoint evaluation Base, source, feedback; 3 source/feedback seeds Table 9: Adam learning rates used in the historical controlled and public-task comparisons, selected on pilot worlds by source validation cross-entropy and training-support validation feedback. Source and reward stages reset optimizer moments. Task / model / initialization Source rate Reward rate Contextual memory / 0.5B / pretrained 3×10−53× 10^-5 10−510^-5 Sequential / 0.5B / pretrained 3×10−53× 10^-5 10−610^-6 Sequential / 0.5B / random 10−410^-4 10−610^-6 Sequential / 1.5B / pretrained 3×10−53× 10^-5 10−610^-6 GSM8K / 1.5B / pretrained 10−510^-5 10−510^-5 HotpotQA / 1.5B / pretrained 3×10−53× 10^-5 10−510^-5 A.8 Alternative routes and controls Continued source prediction consumes the next 2202^20 targets with the source optimizer state continued. Interleaved source updates add one 1,024-target source cross-entropy update after each reward update, drawing only from observed source records. Answer-only SFT generates its own prefix and supervises just the terminal answer and END; it receives no expert intermediate states, and answer labels are counted separately from verifications. Best-of-N reuses nested sets of N=1,2,4,8N=1,2,4,8 candidates from a fixed source model with test-time verification. The classical comparator estimates transitions from the same complete observed source records and searches the hidden binding with allowed training feedback. Source balancing changes sampling over initial public contexts at the same token budget. Format exposure without a shared transition rule. The format control uses three sequential Qwen2.5-0.5B worlds. It reuses the source input law: the initial state, suffix, independent first control, vocabulary, field positions, sequence lengths and END marker. Every output-state token is instead an independent uniform element of the legal state space. The generator reads neither the target transition table nor the downstream binding. Thus it preserves the format and the state marginal distribution while removing any persistent state–operation relation. The mechanism-sensitive contrast is Gsource−GformatG_source-G_format. Shared-source, opposite-task feedback branches. For three sequential worlds, write the hidden first operation as η=(b,u)η=(b,u). Task A uses the original η and task B uses (b⊕1,u)(b 1,u), with identical θ. The source generator does not read η and the displayed prompts omit it. Every public coordinate is unchanged, while the terminal private bit flips under the same suffix, so the two success sets are disjoint. Task-B direct RL and correct-source RL branches are paired with the task-A branches; correct-source branches share the exact source checkpoint. Both tasks’ scoring functions are applied offline to the same outputs at zero and final queries; Table 19 reports the cross-scored rates and Table 20 the exact eligible counts C=NA+NBC=N_A+N_B behind them. The paired selectivity is −0.10-0.10 percentage points with interval [−0.42,0.22][-0.42,0.22] (Figures 20, 19 and 20). Together with the conditional private accuracy in Section A.12, this identifies shared public execution without a measurable preference for the branch’s reward task. Memory-rule acquisition probes. Per-update answer frequencies locate input-independent behavior, and training label frequencies distinguish it from a majority-label task. Two matched source runs, with the same source token budget, learning rate and Adam schedule as the historical continuous-numeric Memory runs, are probed before and after source training: 32 fixed held-out source records each generate three input domains and three value interventions, giving 288 observations per checkpoint. The domains use the original random key frames, basis keys, or basis keys with discrete nuisance coordinates; the first two content coordinates retain the source parity rule. Negating the selected slot’s first coordinate reverses the future bit, whereas the same intervention on the unselected slot preserves it. For every input we record the log-likelihoods of the two valid bit completions (including the common terminator), their difference, two-candidate accuracy, and loss by source field. A correct relevant-value response together with invariance to irrelevant values indicates rule acquisition. Table 10: Source-rule diagnostics from two matched memory source runs. Intervention triples share an original record. Fractions report correct cases over the displayed denominator; CE denotes cross-entropy. Measurement Run A Run B Two-candidate completion accuracy 46/96 49/96 Original records correct 14/32 17/32 Prediction reverses on relevant-value flip 1/32 0/32 Original and relevant-flip pair both correct 0/32 0/32 Query-field CE, before 0.2045 0.2114 Query-field CE, after 0.0430 0.0397 Bit-field CE per token, after 0.3483 0.3869 Each group of 96 cases consists of 32 records and their relevant and irrelevant interventions. The answer field contains two predicted tokens; normalizing the two completion likelihoods gives binary cross-entropies of 0.6949 and 0.7705. The lower query-field loss shows learning of predictable source content, but the pair test shows that these models do not reliably express the composed retrieval-and-sign-product response. After feedback, all five correct-source runs produce a constant valid action, with 49.18% greedy correctness on balanced task labels; wrong-source runs share this endpoint (Table 21). Comparing source and feedback checkpoints separates predictable-field learning from input-dependent task responses. A.9 Uncertainty Means and 95% Student-t intervals are computed across world/training seed means, with individual seed values retained; paired differences use the same seeds. The sampled sequential correct-source-versus-direct and correct-source-versus-wrong-source endpoint contrasts additionally use exact two-sided sign-flip tests with Holm correction at 0.05 (Table 24). The sign-flip tests assume sign symmetry of paired effects under the null. Holm adjustment uses a fixed family size of three, with p=1p=1 assigned to the remaining slot; the adjusted values for the two reported contrasts are 0.0234 and 0.672. Secondary comparisons report effect sizes and intervals. Separately, 5,000 paired item-bootstrap resamples quantify uncertainty conditional on the trained models. A.10 Public-task adaptation and feedback Public-task source midtraining uses domain-related text without downstream labels; it does not instantiate the exact rule-independence assumption of the theoretical source laws. Reported budgets cover additional source tokens and reward queries, excluding inherited pretraining. The public-task panel uses five paired Qwen2.5-1.5B seeds for each of GSM8K and HotpotQA, with three reward routes: direct, domain-source and shuffled-source learning. Each source trajectory predicts 2222^22 tokens and retains 2202^20 and 2222^22 checkpoints. Shuffling acts within each document’s 64-token blocks, preserving the target-token multiset. Source and feedback learning rates use the same grids and selection rule as the controlled experiments, with two pilot seeds per task; selection uses source validation cross-entropy and training-distribution validation rewards. The numeric answer parser compares final GSM8K answers; HotpotQA uses normalized exact match for reward and official answer F1 as an additional measurement. Mathematical continuations have at most 512 new tokens, and question-answering continuations at most 256. HotpotQA prompts are admitted by tokenized length at most 4,096; all 7,405 development candidates qualify, and a fixed 1,024-question subset is evaluated. All 1,319 GSM8K test questions are evaluated. Source article titles are disjoint from downstream training and evaluation article titles, and question and document identities, dataset versions and tokenization are fixed before training. Reward queries are evaluated at 0,1024,2048,4096,81920,1024,2048,4096,8192. The three intermediate points use 512 paired prompts; the pre-RL and final evaluations use the full 1,319 or 1,024 items, and learning curves use the common 512 prompts at every checkpoint. Each task also has three answer-only SFT seeds initialized from the corresponding domain-source state, supervising only the answer and EOS with 8,192 answer labels and the same update batch. Fixed-model best-of-16 uses the same 256 test prompts for base and domain-source models. Public transfer uses 100 GSM-Symbolic templates with ten fixed variants each, and every item in the two three-object BBH tasks; symbolic variants are repeated observations of a template. Public results show every seed, means and 95% Student-t intervals over seed-level outcomes and paired gains; conditional item-bootstrap intervals preserve item pairing, and GSM-Symbolic resamples templates. Figure 21 shows the entry scores, endpoints and paired gain differences summarized in Table 2, and Figure 23 the greedy learning curves on the common 512 prompts, which locate where along the 8,192 queries each route’s gain accrues. Figure 21: Public-task starting scores and reward gains, five paired training seeds. (a) Open and filled markers show mean greedy success before and after RL; segments connect the same route. (b) Within-seed reward gain minus direct RL’s gain, in percentage points: small translucent dots are seeds, large markers are means, and the dotted line is zero. Horizontal bars are 95% Student-t intervals. Each seed and route uses the same full endpoint sets: 1,319 GSM8K and 1,024 HotpotQA prompts. Direct starting scores evaluate one fixed pretrained checkpoint. A.11 Secondary public-task metrics and external transfer Success, numeric correctness and F1 are higher-is-better; format errors and truncation are lower-is-better. Uncertainty describes training-seed variation. Table 11: Public-task secondary greedy measurements. Percentages apply to correctness, F1 and error rates; generated length is tokens. RL uses five seeds and answer-only SFT three. F1 is answer token-overlap F1; format and truncation rates use all generated answers. Entries are seed means with 95% Student-t half-widths. Task Method Decoding Metric Mean and 95% CI GSM8K Direct RL greedy numeric correct 62.3±4.162.3\, ± 4.1 GSM8K Direct RL greedy Format error 0.5±0.20.5\, ± 0.2 GSM8K Direct RL greedy Length (tokens) 108.1±8.8108.1\, ± 8.8 GSM8K Direct RL greedy truncated 0.3±0.10.3\, ± 0.1 GSM8K Correct source greedy numeric correct 62.0±2.962.0\, ± 2.9 GSM8K Correct source greedy Format error 2.2±4.22.2\, ± 4.2 GSM8K Correct source greedy Length (tokens) 149.8±101.5149.8\, ± 101.5 GSM8K Correct source greedy truncated 1.6±3.41.6\, ± 3.4 GSM8K Shuffled source greedy numeric correct 49.7±29.349.7\, ± 29.3 GSM8K Shuffled source greedy Format error 17.8±47.517.8\, ± 47.5 GSM8K Shuffled source greedy Length (tokens) 177.8±210.9177.8\, ± 210.9 GSM8K Shuffled source greedy truncated 17.7±47.617.7\, ± 47.6 GSM8K Answer SFT greedy numeric correct 12.4±0.412.4\, ± 0.4 GSM8K Answer SFT greedy Format error 0.0±0.00.0\, ± 0.0 GSM8K Answer SFT greedy Length (tokens) 4.5±0.04.5\, ± 0.0 GSM8K Answer SFT greedy truncated 0.0±0.00.0\, ± 0.0 HotpotQA Direct RL greedy Answer F1 60.4±1.660.4\, ± 1.6 HotpotQA Direct RL greedy Format error 0.0±0.00.0\, ± 0.0 HotpotQA Direct RL greedy Length (tokens) 5.3±0.35.3\, ± 0.3 HotpotQA Direct RL greedy truncated 0.0±0.00.0\, ± 0.0 HotpotQA Correct source greedy Answer F1 58.4±1.158.4\, ± 1.1 HotpotQA Correct source greedy Format error 0.0±0.00.0\, ± 0.0 HotpotQA Correct source greedy Length (tokens) 5.6±0.25.6\, ± 0.2 HotpotQA Correct source greedy truncated 0.0±0.00.0\, ± 0.0 HotpotQA Shuffled source greedy Answer F1 57.4±1.457.4\, ± 1.4 HotpotQA Shuffled source greedy Format error 0.0±0.00.0\, ± 0.0 HotpotQA Shuffled source greedy Length (tokens) 5.5±0.45.5\, ± 0.4 HotpotQA Shuffled source greedy truncated 0.0±0.00.0\, ± 0.0 HotpotQA Answer SFT greedy Answer F1 69.0±0.569.0\, ± 0.5 HotpotQA Answer SFT greedy Format error 0.0±0.00.0\, ± 0.0 HotpotQA Answer SFT greedy Length (tokens) 4.9±0.24.9\, ± 0.2 HotpotQA Answer SFT greedy truncated 0.0±0.00.0\, ± 0.0 Table 11: Public-task secondary greedy measurements. Percentages apply to correctness, F1 and error rates; generated length is tokens. (continued) Table 12: HotpotQA greedy results within each indicated question-type subset. Success is normalized exact match; rates are percentages with seed-level 95% Student-t half-widths. Method Decoding Type Metric Mean and 95% CI Direct RL greedy bridge Answer F1 60.5±1.160.5\, ± 1.1 Direct RL greedy bridge success 46.2±0.446.2\, ± 0.4 Direct RL greedy comparison Answer F1 60.2±3.560.2\, ± 3.5 Direct RL greedy comparison success 52.9±3.752.9\, ± 3.7 Correct source greedy bridge Answer F1 58.3±1.158.3\, ± 1.1 Correct source greedy bridge success 44.2±0.944.2\, ± 0.9 Correct source greedy comparison Answer F1 58.9±2.458.9\, ± 2.4 Correct source greedy comparison success 51.2±2.651.2\, ± 2.6 Shuffled source greedy bridge Answer F1 57.3±1.357.3\, ± 1.3 Shuffled source greedy bridge success 43.0±1.043.0\, ± 1.0 Shuffled source greedy comparison Answer F1 57.7±2.457.7\, ± 2.4 Shuffled source greedy comparison success 50.0±2.450.0\, ± 2.4 Answer SFT greedy bridge Answer F1 70.0±1.170.0\, ± 1.1 Answer SFT greedy bridge success 52.9±2.252.9\, ± 2.2 Answer SFT greedy comparison Answer F1 65.0±3.265.0\, ± 3.2 Answer SFT greedy comparison success 56.9±1.856.9\, ± 1.8 Table 12: HotpotQA greedy results within each indicated question-type subset. Success is normalized exact match; rates are percentages with seed-level 95% Student-t half-widths. (continued) Table 13: External-transfer and verification-selection results. Bo16 denotes best-of-16 selection with test-time verification. ± gives training-seed 95% Student-t half-widths; brackets give item/template-bootstrap intervals for a fixed model. BBH logic/tracking are three-object logical deduction/object tracking. Training Method Evaluation Checkpoint Seeds Success (%) GSM8K Base/Bo16 GSM8K base 1 44.5[38.3,50.8]†44.5\, [38.3,50.8] GSM8K Correct source/Bo16 GSM8K source 5 22.1±4.922.1\, ± 4.9 GSM8K Direct RL BBH logic feedback 5 40.1±1.640.1\, ± 1.6 GSM8K Direct RL GSM-Symbolic feedback 5 54.5±3.254.5\, ± 3.2 GSM8K Direct RL BBH tracking feedback 5 11.4±17.611.4\, ± 17.6 GSM8K Correct source BBH logic feedback 5 30.9±21.030.9\, ± 21.0 GSM8K Correct source GSM-Symbolic feedback 5 53.0±3.553.0\, ± 3.5 GSM8K Correct source BBH tracking feedback 5 13.2±22.313.2\, ± 22.3 GSM8K Correct source BBH logic source 5 40.7±1.240.7\, ± 1.2 GSM8K Correct source GSM-Symbolic source 5 2.1±0.42.1\, ± 0.4 GSM8K Correct source BBH tracking source 5 30.8±3.330.8\, ± 3.3 GSM8K Shuffled source BBH logic feedback 5 27.1±21.327.1\, ± 21.3 GSM8K Shuffled source GSM-Symbolic feedback 5 43.6±26.543.6\, ± 26.5 GSM8K Shuffled source BBH tracking feedback 5 11.4±18.511.4\, ± 18.5 GSM8K Shuffled source BBH logic source 5 39.5±2.939.5\, ± 2.9 GSM8K Shuffled source GSM-Symbolic source 5 6.3±0.86.3\, ± 0.8 GSM8K Shuffled source BBH tracking source 5 32.8±2.632.8\, ± 2.6 HotpotQA Base/Bo16 HotpotQA base 1 45.7[39.5,52.0]†45.7\, [39.5,52.0] HotpotQA Correct source/Bo16 HotpotQA source 5 50.7±1.350.7\, ± 1.3 HotpotQA Direct RL BBH logic feedback 5 41.5±1.241.5\, ± 1.2 HotpotQA Direct RL BBH tracking feedback 5 35.5±0.835.5\, ± 0.8 HotpotQA Correct source BBH logic feedback 5 41.0±1.641.0\, ± 1.6 HotpotQA Correct source BBH tracking feedback 5 36.1±1.136.1\, ± 1.1 HotpotQA Correct source BBH logic source 5 39.0±1.839.0\, ± 1.8 HotpotQA Correct source BBH tracking source 5 38.2±1.338.2\, ± 1.3 HotpotQA Shuffled source BBH logic feedback 5 26.8±10.726.8\, ± 10.7 HotpotQA Shuffled source BBH tracking feedback 5 34.0±2.834.0\, ± 2.8 HotpotQA Shuffled source BBH logic source 5 5.8±2.35.8\, ± 2.3 HotpotQA Shuffled source BBH tracking source 5 35.2±2.035.2\, ± 2.0 Shared Base GSM-Symbolic base 1 5.8[3.5,8.5]5.8\, [3.5,8.5] Shared Base BBH logic base 1 41.6[35.6,48.0]41.6\, [35.6,48.0] Shared Base BBH tracking base 1 34.4[28.4,40.4]34.4\, [28.4,40.4] Table 13: External-transfer and verification-selection results. Bo16 denotes best-of-16 selection with test-time verification. ± gives training-seed 95% Student-t half-widths; brackets give item/template-bootstrap intervals for a fixed model. BBH logic/tracking are three-object logical deduction/object tracking. (continued) A dagger marks a fixed-base best-of-16 estimate. Bracketed base intervals bootstrap prompts (or GSM-Symbolic templates) conditional on the one fixed model; GSM-Symbolic uses equal template means over ten variants per template. A.12 Endpoint decomposition and complete controlled-task results For the sequential decomposition, let N be all evaluated prompts, P syntactically valid outputs with a consistent public path, C those outputs that also have the correct terminal public coordinate, and K those in C with the correct private bit. We report P/NP/N, C/NC/N and K/CK/C; the last quantity is undefined when C=0C=0. Confidence intervals use per-seed ratios, and exact counts are shown. Cross-scoring yields disjoint task-verifier counts NA+NB=CN_A+N_B=C. Memory validity is the fraction producing a legal bit and terminator; its conditional correctness uses only valid completions. All gains are computed from the two feedback evaluations on identical prompts. Greedy fixed-base scores repeat across training seeds because they evaluate one inherited checkpoint. Public execution and private-task accuracy. Sequential evaluation uses held-out suffixes of lengths four and six. Each source route receives 2202^20 prediction tokens, then 8,192 reward queries: 128 updates of 64 sampled completions. Eight paired worlds compare correct source, coherently wrong source and direct RL. The wrong source is a consistent source law with the same public transition rule but a private transition table different from the target mechanism. Correct-source models obtain 242–584 positive training rewards per world, whereas direct RL and a format control that randomizes every output state obtain none. Consistent source structure, not exposure to the output format, is what gives rewards successful trajectories to reinforce. Correct-source greedy success rises from 3.30% to 41.67% and wrong-source success from 3.94% to 38.15% (Table 23). The endpoint decomposition, which splits greedy success into a valid public path, a correct public endpoint and a correct private bit, says what this shared improvement is made of. Correct-source outputs have valid public paths on 96.22% of prompts and correct public endpoints on 83.25%, while private-bit accuracy conditional on correct public execution is 50.12% for the correct source and 49.87% for the wrong source (Tables 17 and 20). The gain is acquired public execution. The error partition over all prompts (Table 22) identifies the private bit as the largest correct-source error category, 41.58% of all prompts, against 12.96% for a wrong public endpoint and under 4% for malformed or path errors, so the decomposition locates the remaining error in private-state computation and its task-specific use. A classical transition-table learner given the same source records and training feedback reaches 100% in three tested worlds (Table 16), so the observations carry the private information, and the decomposition measures how much of it a neural route has turned into computation. Format and reward-bearing trajectories. In the historical Sequential direct-RL runs, all 1,024 greedy outputs per world are malformed. The format-source control instead produces syntactically valid answers on 97.75% of prompts on average, but none has a consistent public trajectory (Table 22). Thus a zero public-path rate can reflect either invalid syntax or invalid execution. Historical Memory direct-RL outputs likewise contain no valid bit-and-terminator answer. Across these eight Sequential direct, three format-source and five Memory direct runs, each full 8,192-query training budget contains zero positive rewards and all 128 reward-loss values are zero. The zero endpoints therefore accompany an absence of reward-weighted learning signal. Tables reporting decomposition and cross-task counts reuse these outputs; they are not additional independent runs. Table 14: Greedy evaluation on identical before/after prompts. Start and end are success percentages; gain is percentage points. Entries are means with 95% Student-t half-widths across paired seeds. Start is the feedback-entry evaluation on the same prompts; direct public-task starts evaluate one fixed base checkpoint. Training route Seeds Before (%) After (%) Gain (p) Sequential Correct source 8 3.30±3.273.30\, ± 3.27 41.67±5.5641.67\, ± 5.56 38.38±5.2138.38\, ± 5.21 Direct RL 8 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Wrong source 8 3.94±3.483.94\, ± 3.48 38.15±10.6838.15\, ± 10.68 34.20±9.8734.20\, ± 9.87 Format source 3 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Memory Correct source 5 51.02±2.5351.02\, ± 2.53 49.18±1.4349.18\, ± 1.43 −1.84±3.15-1.84\, ± 3.15 Direct RL 5 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Wrong source 5 49.90±1.4649.90\, ± 1.46 49.18±1.4349.18\, ± 1.43 −0.72±1.35-0.72\, ± 1.35 GSM8K Direct RL 5 8.57±0.008.57\, ± 0.00 62.32±4.1462.32\, ± 4.14 53.75±4.1453.75\, ± 4.14 Domain source 5 2.32±0.392.32\, ± 0.39 62.05±2.9362.05\, ± 2.93 59.73±3.1659.73\, ± 3.16 Shuffled source 5 8.58±1.318.58\, ± 1.31 49.70±29.2949.70\, ± 29.29 41.12±28.3441.12\, ± 28.34 HotpotQA Direct RL 5 23.93±0.0023.93\, ± 0.00 47.60±1.0247.60\, ± 1.02 23.67±1.0223.67\, ± 1.02 Domain source 5 35.10±0.9835.10\, ± 0.98 45.59±0.8645.59\, ± 0.86 10.49±1.7410.49\, ± 1.74 Shuffled source 5 5.62±0.965.62\, ± 0.96 44.45±1.1344.45\, ± 1.13 38.83±1.9138.83\, ± 1.91 Table 15: Sequential source dose on five matched worlds. Greedy success after 8,192 reward queries; values are percentages with 95% Student-t half-widths. Source checkpoints share their training trajectories. Source prediction tokens Worlds Final success (%) 2182^18 5 0.00±0.000.00\, ± 0.00 2192^19 5 0.00±0.000.00\, ± 0.00 2202^20 5 41.99±8.9241.99\, ± 8.92 Table 16: Greedy sequential endpoints for initialization, scale and alternative-route controls after 8,192 reward queries. Entries are seed means with 95% Student-t half-widths; the classical learner uses the same source records and permitted training feedback. Setting Route Worlds Success (%) 0.5B, pretrained, 2202^20 source tokens Balanced source 3 39.71±32.8039.71\, ± 32.80 Interleaved source updates 3 38.57±24.0538.57\, ± 24.05 Classical table learner 3 100.00±0.00100.00\, ± 0.00 0.5B, random initialization, 2202^20 source tokens Correct source 3 0.00±0.000.00\, ± 0.00 Wrong source 3 0.00±0.000.00\, ± 0.00 Direct RL 3 0.00±0.000.00\, ± 0.00 1.5B, pretrained, 2202^20 source tokens Correct source 3 26.43±35.4426.43\, ± 35.44 Wrong source 3 22.27±30.7622.27\, ± 30.76 Direct RL 3 0.00±0.000.00\, ± 0.00 Table 17: Sequential greedy endpoints. Public path requires valid syntax and a consistent public trajectory. Correct public additionally requires the target terminal public coordinate. Private accuracy conditions on this latter event; – denotes a zero denominator in every seed. The first two rates use all prompts; the last is the within-seed conditional ratio. Entries are seed means and 95% Student-t half-widths. Route Seeds Public path (%) Correct public (%) Private ∣ public (%) Correct source 8 96.22±8.5696.22\, ± 8.56 83.25±11.3883.25\, ± 11.38 50.12±1.8150.12\, ± 1.81 Wrong source 8 98.13±2.8798.13\, ± 2.87 76.77±21.7976.77\, ± 21.79 49.87±1.0949.87\, ± 1.09 Direct RL 8 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 – Format source 3 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 – Table 18: Exact denominators for the sequential greedy decomposition. N is all prompts, P valid public paths, C correct public endpoints, and K correct private bits among C. The conditional accuracy is K/CK/C separately in each seed; C=0C=0 is missing. Route Seed N P C K Correct source 0 1024 1012 695 345 Correct source 1 1024 1024 1021 515 Correct source 2 1024 1024 824 381 Correct source 3 1024 1024 773 412 Correct source 4 1024 1024 1017 497 Correct source 5 1024 1024 771 405 Correct source 6 1024 726 715 358 Correct source 7 1024 1024 1004 501 Wrong source 0 1024 957 772 388 Wrong source 1 1024 1024 1024 524 Wrong source 2 1024 1024 610 299 Wrong source 3 1024 1022 274 141 Wrong source 4 1024 1024 1021 502 Wrong source 5 1024 1024 646 329 Wrong source 6 1024 940 918 438 Wrong source 7 1024 1024 1024 504 Direct RL 0 1024 0 0 0 Direct RL 1 1024 0 0 0 Direct RL 2 1024 0 0 0 Direct RL 3 1024 0 0 0 Direct RL 4 1024 0 0 0 Direct RL 5 1024 0 0 0 Direct RL 6 1024 0 0 0 Direct RL 7 1024 0 0 0 Format source 3 1024 0 0 0 Format source 4 1024 0 0 0 Format source 5 1024 0 0 0 Table 19: Greedy cross-scoring of the same saved outputs, three paired worlds. A/B success sets are disjoint. The conditional A score uses the union of the two sets as denominator; it is undefined for direct runs with no eligible outputs. Entries are seed means with 95% Student-t half-widths. Route Reward task Test A (%) Test B (%) A ∣ A or B (%) Correct source A 42.77±12.4242.77\, ± 12.42 40.59±21.9340.59\, ± 21.93 51.57±5.8851.57\, ± 5.88 Correct source B 41.83±14.4941.83\, ± 14.49 39.45±22.7939.45\, ± 22.79 51.73±6.3451.73\, ± 6.34 Direct RL A 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 – Direct RL B 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 – Table 20: Exact cross-task greedy counts. C=NA+NBC=N_A+N_B is the common eligible denominator: valid public execution with the correct public endpoint. Each world uses 1,024 prompts shared across its A/B branches and scorers; task B changes only the required private bit. Route Seed N C NAN_A NBN_B Correct source 3 1024 773 412 361 Correct source 4 1024 1017 497 520 Correct source 5 1024 771 405 366 Source / task B 3 1024 824 439 385 Source / task B 4 1024 988 482 506 Source / task B 5 1024 685 364 321 Direct RL 3 1024 0 0 0 Direct RL 4 1024 0 0 0 Direct RL 5 1024 0 0 0 Direct / task B 3 1024 0 0 0 Direct / task B 4 1024 0 0 0 Direct / task B 5 1024 0 0 0 Table 21: Contextual greedy endpoints. Correctness conditional on a valid bit is undefined if no valid answer is produced. Constant counts runs producing a single valid action on all 1,024 prompts. A balanced input-independent bit rule has 50% population accuracy. Entries are seed means with 95% Student-t half-widths. Route Seeds Valid (%) Correct (%) Correct ∣ valid (%) Constant runs Correct source 5 100.00±0.00100.00\, ± 0.00 49.18±1.4349.18\, ± 1.43 49.18±1.4349.18\, ± 1.43 5 Wrong source 5 100.00±0.00100.00\, ± 0.00 49.18±1.4349.18\, ± 1.43 49.18±1.4349.18\, ± 1.43 5 Direct RL 5 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 – 0 Table 22: Sequential greedy endpoint error partition (percent of all prompts). Malformed, invalid public path, wrong public endpoint, wrong private bit and success are mutually exclusive and exhaustive. Private-bit failures here use all prompts as denominator; conditional accuracy uses only correct-public outputs. Entries are seed means with 95% Student-t half-widths. Route Malformed Path error Public error Private error Success Correct source 3.72±8.583.72\, ± 8.58 0.06±0.140.06\, ± 0.14 12.96±11.0012.96\, ± 11.00 41.58±6.1641.58\, ± 6.16 41.67±5.5641.67\, ± 5.56 Wrong source 1.81±2.831.81\, ± 2.83 0.06±0.100.06\, ± 0.10 21.36±22.3721.36\, ± 22.37 38.62±11.1838.62\, ± 11.18 38.15±10.6838.15\, ± 10.68 Direct RL 100.00±0.00100.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Format source 2.25±5.252.25\, ± 5.25 97.75±5.2597.75\, ± 5.25 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Table 23: Paired differences of reward-stage gains: correct source minus the indicated route. Only shared seeds enter each contrast; entries are seed-paired mean differences in percentage points with 95% Student-t half-widths. Task Evaluation Comparator Seeds Difference (p) sequential greedy Direct RL 8 38.38±5.2138.38\, ± 5.21 sequential greedy Wrong source 8 4.17±8.954.17\, ± 8.95 sequential greedy Format source 3 41.96±10.1641.96\, ± 10.16 contextual greedy Direct RL 5 −1.84±3.15-1.84\, ± 3.15 contextual greedy Wrong source 5 −1.11±2.06-1.11\, ± 2.06 GSM8K greedy Direct RL 5 5.97±3.665.97\, ± 3.66 GSM8K greedy Shuffled source 5 18.61±30.1618.61\, ± 30.16 HotpotQA greedy Direct RL 5 −13.18±1.50-13.18\, ± 1.50 HotpotQA greedy Shuffled source 5 −28.34±2.42-28.34\, ± 2.42 Sampled decoding. Under sampled decoding of the same checkpoints (one sampled answer per prompt), source-trained sequential models also outperform direct RL, by 5.48±1.685.48± 1.68 points at the endpoint, while the correct-source reward-stage gain is 0.31±0.550.31± 0.55 percentage points against 38.38±5.2138.38± 5.21 under greedy decoding. The output decompositions in this subsection use greedy decoding (Figures 22 and 24). Table 24: Sampled-decoding robustness check on the same checkpoints and prompts as the greedy results (one sampled answer per prompt at temperature one). Start and end are success percentages; gain is percentage points; entries are seed means with 95% Student-t half-widths. The two sequential endpoint contrasts use exact two-sided sign-flip tests; adjusted p-values use a fixed three-hypothesis Holm family, of which the two measured endpoint contrasts are shown and the third hypothesis is entered conservatively at p=1p=1. Training route Seeds Before (%) After (%) Gain (p) Sequential Correct source 8 5.18±1.715.18\, ± 1.71 5.48±1.685.48\, ± 1.68 0.31±0.550.31\, ± 0.55 Direct RL 8 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Wrong source 8 5.16±1.595.16\, ± 1.59 5.73±1.725.73\, ± 1.72 0.56±0.770.56\, ± 0.77 Format source 3 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Memory Correct source 5 48.26±1.1648.26\, ± 1.16 49.18±1.4349.18\, ± 1.43 0.92±1.410.92\, ± 1.41 Direct RL 5 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 0.00±0.000.00\, ± 0.00 Wrong source 5 48.16±2.6648.16\, ± 2.66 49.18±1.4349.18\, ± 1.43 1.02±2.391.02\, ± 2.39 GSM8K Direct RL 5 5.13±0.225.13\, ± 0.22 45.75±3.9645.75\, ± 3.96 40.62±4.0440.62\, ± 4.04 Domain source 5 1.62±0.251.62\, ± 0.25 45.17±5.1945.17\, ± 5.19 43.55±4.9943.55\, ± 4.99 Shuffled source 5 2.68±0.912.68\, ± 0.91 40.89±13.8740.89\, ± 13.87 38.21±13.8738.21\, ± 13.87 HotpotQA Direct RL 5 13.59±0.9013.59\, ± 0.90 45.27±1.0245.27\, ± 1.02 31.68±0.8331.68\, ± 0.83 Domain source 5 19.55±1.9719.55\, ± 1.97 43.71±1.4843.71\, ± 1.48 24.16±1.8224.16\, ± 1.82 Shuffled source 5 2.99±0.712.99\, ± 0.71 42.23±1.0542.23\, ± 1.05 39.24±1.1339.24\, ± 1.13 Sequential endpoint contrasts (difference in p, p, pHolmp_ Holm) Correct source −- Direct RL 5.48±1.685.48\, ± 1.68; p=0.0078p=0.0078; pHolm=0.0234p_ Holm=0.0234 Correct source −- Wrong source −0.24±0.51-0.24\, ± 0.51; p=0.336p=0.336; pHolm=0.672p_ Holm=0.672 Figure 22: Sampled evaluation of (a) Sequential, eight worlds, and (b) Memory, five worlds. Each checkpoint uses one answer per prompt at temperature one on 1,024 prompts per world. Curves and large markers show world means, small translucent dots individual worlds, and shaded bands 95% Student-t intervals. The dotted 50% line in (b) marks binary chance. Figures 20, 14 and 17 report greedy evaluation of these historical checkpoints. Figure 23: Greedy learning curves for (a) GSM8K and (b) HotpotQA on the same 512 prompts at every checkpoint, five training seeds. Curves and large markers are seed means, small translucent dots are individual seeds, and bands are 95% Student-t intervals. The full endpoint comparison in Figures 21 and 2 uses the larger matched evaluation sets. Figure 24: Two effects of source observations on distinct horizontal scales. (a) Remaining contextual ambiguity versus the number N of source records in eight worlds; the dashed curve is the analytic expectation 14(34)N 14( 34)^N. (b) Greedy Sequential success after 8,192 reward queries versus source prediction tokens, five matched worlds. Solid curves and large markers are world means; small translucent dots are individual worlds. Shading in (a) and error bars in (b) are 95% Student-t intervals. A.13 Alternative routes, selection and external transfer Source dose and access to reward. The budget comparison evaluates the 2182^18, 2192^19 and 2202^20 checkpoints of five source trajectories. At 2182^18 and 2192^19 source tokens, no feedback run receives a positive reward; at 2202^20 every run does, and greedy success reaches 41.99% (Tables 15 and 24). Access to successful trajectories therefore switches on within a factor of two in source budget. Initialization, model scale and adaptation. In the three-world comparison at 2202^20 source tokens, no random-initialized route attains sequential success. The contrast with pretrained initialization shows that inherited weights matter under this budget and protocol. At 1.5B parameters, correct-source and wrong-source routes reach 26.43% and 22.27% while direct RL stays at zero; source sampling balanced over initial public contexts, at the same token budget, reaches 39.71±32.80%39.71± 32.80\% over three worlds, with substantial between-world uncertainty (Table 16). Continued prediction and answer-only supervised fine-tuning (SFT) do not reproduce the reward-stage improvement, whereas interleaving source updates with RL retains it at 38.57% (Tables 25 and 16). Fixed-source best-of-N selection raises success from 4.95% at N=1N=1 to 26.73% at N=8N=8 by spending test-time verifier queries instead of training queries (Table 26). Source-only, doubled-source and answer-only SFT scores on the same three worlds are 3.55%, 1.40% and zero (Table 25); their separate source and answer-label costs are listed with the endpoints. Public SFT yields 12.36% on GSM8K and 53.74% on HotpotQA over three seeds (Table 28). Transfer to GSM-Symbolic gives 53.0% after source+RL versus 54.5% after direct RL; BBH logic and object-tracking results vary by task and route (Table 13). Controlled sequential checkpoints change little on the same external tasks (Table 27). On separate 256-question sets, best-of-16 selection scores 22.1% after domain source versus 44.5% for the fixed base on GSM8K, and 50.7% versus 45.7% on HotpotQA (Table 13). Table 25: Greedy sequential source-only and alternative-adaptation endpoints. Route Seeds Source tokens Answer labels Success (%) Source only, all worlds 8 1,048,576 0 3.30±3.273.30\, ± 3.27 Source only, paired worlds 3 1,048,576 0 3.55±3.763.55\, ± 3.76 Continue source prediction 3 2,097,152 0 1.40±3.401.40\, ± 3.40 Answer-only adaptation 3 1,048,576 8,192 0.00±0.000.00\, ± 0.00 All evaluations use 1,024 identical prompts within a world. The paired-world rows use three matched worlds. All routes have zero reward-training queries. Answer-only adaptation additionally uses its recorded answer-prediction targets, separately from source observations. Intervals are 95% Student-t intervals across seed means. Table 26: Verification-selected sampling from the fixed sequential source checkpoint. Uncertainty is the 95% Student-t half-width across training seeds. Candidates Seeds Prompts Test queries/seed Success (%) 1 3 1,024 1,024 4.95±6.294.95\, ± 6.29 2 3 1,024 2,048 8.72±9.868.72\, ± 9.86 4 3 1,024 4,096 15.92±16.0115.92\, ± 16.01 8 3 1,024 8,192 26.73±24.5726.73\, ± 24.57 Each row selects a correct answer, when present, among the first N sampled candidates. All rows use nested prefixes of the same eight candidates per prompt, with no reward-stage parameter updates. Intervals summarize three seed-level means. Table 27: Greedy external transfer after controlled sequential training of Qwen2.5-0.5B. Checkpoint Independent units Prompts Success (%) Logical deduction (250 prompts) Base checkpoint 1 fixed model 250 30.40[24.80,36.00]30.40\,[24.80,36.00] After source prediction 3 training seeds 250 34.40±2.6334.40\, ± 2.63 After source + reward 3 training seeds 250 34.27±3.1934.27\, ± 3.19 Object tracking (250 prompts) Base checkpoint 1 fixed model 250 21.20[16.40,26.40]21.20\,[16.40,26.40] After source prediction 3 training seeds 250 28.13±14.1028.13\, ± 14.10 After source + reward 3 training seeds 250 27.60±12.6927.60\, ± 12.69 Source and feedback uncertainty is a 95% Student-t interval over three training seeds. Base brackets give the 95% item-bootstrap interval for the fixed pretrained model. Table 28: Greedy answer-only supervised adaptation on public tasks. Task Seeds Prompts/seed Answer labels/seed Success (%) GSM8K 3 1,319 8,192 12.36±0.3812.36\, ± 0.38 HotpotQA 3 1,024 8,192 53.74±1.3853.74\, ± 1.38 Each Qwen2.5-1.5B run starts from its domain-source checkpoint and receives 8,192 terminal-answer labels, with no reward-training queries. Source training uses 2222^22 prediction tokens. Success is exact answer correctness; intervals are 95% Student-t intervals over three training seeds, not prompt intervals. A.14 Source-acquisition probes and public source dose This subsection gives the two-world format-stage study discussed in Section 6 and the 2202^20-token public source-dose comparison. Public source-dose comparison. The 2202^20-token source route uses Qwen2.5-1.5B, the same source learning rates as the 2222^22 route and 8,192 training verifications per feedback run. Within each seed, the direct, 2202^20 and 2222^22 routes share identical evaluation prompts and the same ordered sequence of 8,192 training prompts. Tables 29 and 30 report start, endpoint, gain and seed-paired contrasts. Table 29: Public source-dose endpoints. Five paired training seeds; the same 1,319 GSM8K or 1,024 HotpotQA prompts at both endpoints and across routes. Rates are percentages; gains are percentage points. Uncertainty is a seed-level 95% Student-t half-width. Direct starting scores evaluate one fixed pretrained checkpoint. Source tokens Before RL (%) After RL (%) Gain (p) GSM8K; 8,192 reward queries None (direct) 8.57±0.008.57\, $± 0.00$ 62.32±4.1462.32\, $± 4.14$ 53.75±4.1453.75\, $± 4.14$ 2202^20 3.11±0.573.11\, $± 0.57$ 61.76±3.1561.76\, $± 3.15$ 58.65±3.6758.65\, $± 3.67$ 2222^22 2.32±0.392.32\, $± 0.39$ 62.05±2.9362.05\, $± 2.93$ 59.73±3.1659.73\, $± 3.16$ HotpotQA; 8,192 reward queries None (direct) 23.93±0.0023.93\, $± 0.00$ 47.60±1.0247.60\, $± 1.02$ 23.67±1.0223.67\, $± 1.02$ 2202^20 35.55±1.4435.55\, $± 1.44$ 46.23±1.2846.23\, $± 1.28$ 10.68±2.6110.68\, $± 2.61$ 2222^22 35.10±0.9835.10\, $± 0.98$ 45.59±0.8645.59\, $± 0.86$ 10.49±1.7410.49\, $± 1.74$ Table 30: Seed-paired public source-dose contrasts, in percentage points (p). Each effect is calculated within seed before forming the five-seed mean and 95% Student-t half-width. Contrast Endpoint difference (p) Gain difference (p) GSM8K 2202^20 minus direct −0.56±4.87-0.56\, $± 4.87$ 4.90±5.284.90\, $± 5.28$ 2222^22 minus direct −0.27±3.63-0.27\, $± 3.63$ 5.97±3.665.97\, $± 3.66$ 2202^20 minus 2222^22 −0.29±5.64-0.29\, $± 5.64$ −1.08±6.38-1.08\, $± 6.38$ HotpotQA 2202^20 minus direct −1.37±0.37-1.37\, $± 0.37$ −12.99±1.67-12.99\, $± 1.67$ 2222^22 minus direct −2.01±0.88-2.01\, $± 0.88$ −13.18±1.50-13.18\, $± 1.50$ 2202^20 minus 2222^22 0.64±1.120.64\, $± 1.12$ 0.20±1.820.20\, $± 1.82$ Figure 25: Acquired computation before and after the answer-format stage, two worlds at 2242^24 source tokens. (a) Sequential, original representation with state targets: correct first-H state prefixes and correct END placement, out of 512 probes with the first operation supplied. (b) Memory, discrete representation with the all-token objective: correct first-bit answers on original records and on both members of record/value-flip pairs, each out of 256. Circles and squares are the two worlds, slightly offset where they coincide. Colors identify metrics; lines join the same world and metric across stages. Acquisition design. Two source worlds per representation compare learning rates 10−510^-5 and 3×10−53× 10^-5 at 2222^22 prediction tokens, and the better rate per representation is trained to 2242^24 tokens with matched objective controls. Sequential compares state targets with private-bit-weighted state targets (private weight four, other state targets weight one). Memory compares all-token prediction with future-bit prediction, the latter excluding END and end-of-sequence (EOS) from its source loss. Each source update observes 8,192 prediction-token positions. Alongside the original representations, an alternative Sequential representation adds an explicit length boundary and a local-to-complete-trajectory curriculum, and an alternative Memory representation uses discrete keys and values; alternative tasks are reported separately from the original continuous-numeric Memory task. An answer-format stage precedes and follows source training. Its loss supervises complete public-format answers, including termination, on records whose private fields are independent random labels, so it reveals neither the target mechanism nor the task binding. Sequential uses 128 format updates per stage and Memory 32, with batch 64 and learning rate 10−510^-5; stage transitions reset Adam moments. Separating state content from termination. Each Sequential endpoint has 512 source-validation prompts, split equally between horizons H=5H=5 and H=7H=7, with the random first operation supplied and no task binding. The complete-answer metric requires exactly the expected states followed by END; a content diagnostic instead checks the first H generated states without modifying the output. Seven of the eight original-representation 2222^22 endpoints already generate at least 511/512 correct state prefixes. At 2242^24, all four original source models and three of four curriculum models generate every state prefix correctly, continuing to the 128-token limit rather than emitting END. After the format stage, every 2242^24 Sequential model places END correctly on all 512 prompts, while state-prefix counts fall to 6–14/512 (Table 31). The two metrics separate termination, learned in the format stage, from state-prefix accuracy, which falls sharply after this stage. Figure 26: Two-world acquisition probes before and after the format stage at 2242^24 source tokens. Left: exact Sequential state prefixes, excluding termination, on 512 probes. Right: both members correct in each of 256 Memory original/relevant-flip pairs. Open blue markers are source endpoints; filled purple markers follow the format stage. Circles and squares denote the two worlds, and links pair the same world and representation. Legend abbreviations: “Orig.” is the original representation and “Course” the Sequential curriculum representation; “state” uses state-token targets and “weighted” upweights private state targets; “all” and “bit” are the Memory all-token and future-bit objectives. The Sequential diagnostic uses source-validation support with the first operation given. Figure 25 shows the original-representation Sequential state-target and discrete Memory all-token conditions, adding termination and first-bit accuracy to the two diagnostics shown here. Memory interventions and the format interface. Each Memory world has 256 original source records and paired relevant-value flips; a successful pair requires the correct first bit on both inputs. At 2242^24 tokens, original-representation all-token source accuracy averages 49.61% with no successful pairs, and future-bit accuracy averages 75.20% with 50.39% successful pairs. Discrete all-token source models answer all originals and relevant pairs correctly and already satisfy the public format checks. After the format stage, these means fall to 52.54% and 3.91%; discrete future-bit means change from 85.35%/68.16% to 64.45%/33.79%. For a binary private field predicted with probability p, independent uniform labels give expected cross-entropy −12logp−12log(1−p)- 12 p- 12 (1-p), minimized at p=1/2p=1/2: supervising random private content pulls the model toward an input-independent private prediction, which is consistent with the observed loss of intervention sensitivity after the format stage and motivates placing format preparation before source acquisition and excluding such random-label supervision afterwards. Table 31: Acquisition-probe counts at 2242^24 source tokens. Sequential columns count correct first-H state prefixes out of 512; Memory columns count correct original bits and relevant-flip pairs out of 256. Each cell shows before/after format. Before the format stage no Sequential model emits END; after it, every model places END correctly on all 512 prompts. Representation Objective World Content before Content after Pair before/after Sequential; content /512 Original State 1 512 11 – Original State 2 512 12 – Original Weighted 1 512 6 – Original Weighted 2 512 14 – Curriculum State 1 512 6 – Curriculum State 2 12 12 – Curriculum Weighted 1 512 5 – Curriculum Weighted 2 512 14 – Memory; content and pairs /256 Original All-token 1 130 130 0/0 Original All-token 2 124 132 0/0 Original Future-bit 1 192 145 128/39 Original Future-bit 2 193 182 130/105 Discrete All-token 1 256 138 256/20 Discrete All-token 2 256 131 256/0 Discrete Future-bit 1 220 152 170/69 Discrete Future-bit 2 217 178 179/104 Appendix B Reading map and theorem interfaces The information results first specify what observations can distinguish. The optimization results then construct finite training paths under their stated architectures and schedules. Complete statements appear next to their proofs. Shared network facts are proved once; each optimizer checks its own parameter choices and failure allocations. Question Complete statement and proof Sequential reward equivalence Theorems C.1 and C.3 Contextual source information Theorems C.26 and C.5 Shared initialization and network bounds Appendix E Sequential finite Adam path Theorems F.1 and F.5 Contextual finite Adam path Theorems G.1 and G.7 SGD alternative Theorems H.1 and H Information and computation costs Appendix D Experimental protocols and results Appendix A Here q=2mq=2m counts sequential states and controls, H is the operation horizon, and N=2H+2N=2H+2 counts tokens in a source record. Within shared network bounds, N is the padded context length, R the attention-head count, khead=d/Rk_ head=d/R its width, and DfeatD_ feat the feature dimension. Optimizer sections explicitly instantiate these quantities; their local rate and radius symbols do not change the task binding. Contextual dimensions are nctxn_ ctx and k=p−2k=p-2. A source record, prediction target, verifier query and optimizer update are distinct resources. Appendix C Information supplied by source and reward observations In either family a world fixes a reusable mechanism and a downstream rule. Initial information O includes any source sample and inherited weights; reward adaptation produces a transcript of prompts, sampled outputs and terminal feedback. The decision is an output on a held-out input. In Sequential the mechanism is θ, the rule is η, and source reveals noisy transitions; in Contextual they are U and (s,τ)(s,τ), and source reveals selected-value predictions. Reward equivalence concerns the complete permitted feedback interface, while learnability by the specified network additionally requires the finite optimization arguments below. World-independent initialization is part of each no-source lower-bound experiment; arbitrary pretrained weights must instead be counted in O. C.1 Mechanisms with identical training rewards Use the sequential state space, operation tags, training and held-out supports, and terminal verifier of Section 3. A world fixes the transition table θ and initial operation η=(bη,uη)η=(b_η,u_η). All sums below are in 2F_2. Theorem C.1 (Mechanisms with identical training rewards). Let H≥3H≥ 3 and fix uηu_η. Two worlds are reward-equivalent if and only if, for some κ,d∈2κ,d _2, θu,z′=θu,z+κ+dℓ(u),bη′=bη+(Hmod2)κ+dℓ(uη).θ _u,z= _u,z+κ+d\, (u), b _η=b_η+(H 2)\,κ+d\, (u_η). (C.1) Their targets differ on every test prompt exactly when d=1d=1. The proof is given in Lemma C.3. C.2 Exact identification from source and terminal observations This appendix proves the statistical results. Throughout, L=H−1L=H-1 is the suffix length, m=|Z|m=|Z|, and all additions of bits or coordinates are in 2F_2. Matrix norms and inner products are over the reals. The matrix arguments are analysis of a binary-equation learner; they do not grant it an additional oracle. The iid-prompt restriction makes recovery quantitative. A symbolic learner first tests public binding coordinates with random endpoint bits. Once that coordinate is identified, one terminal reply reveals one exact binary equation in the unknown transition table and binary binding coordinate. After solving these equations, the learner uses its stored source records to recover κ,dκ,d. Canonical execution starts with η and applies the true transition at every step; δ below is the allowed failure probability. Theorem C.2 (A matched low-source query regime). Let L=H−1≥⌈216ln(2m)⌉L=H-1≥ 2^16 (2m) and 0<δ<10<δ<1. The explicit fresh-iid learner with budgets (C.18) and (C.20) has, for each fixed world, probability at least 1−δ1-δ of correct canonical execution on every test prompt. Its endpoint success, target first-state retention and task utility then equal one. It uses Npre N_ pre ≤32a−2ln(6/δ)+2H+2, ≤ 32a^-2 (6/δ)+2H+2, (C.2) Q Q =O(m2+(m+m2L)ln(m/δ)+mln(1/δ)), =O\! (m^2+ (m+ m^2L ) (m/δ)+m (1/δ) ), (C.3) with nQnQ generated decisions, HQHQ prompt tokens and zero gradient updates. At fixed δ=1/16δ=1/16, fixed c∈(0,1/64]c∈(0,1/64], a=1−c/Ha=1-c/H and L=⌈216ln(2m)⌉L= 2^16 (2m) , this is O(logm)O( m) raw source tokens and O(m2)O(m^2) queries. For m≥512m≥ 512, any learner with nrecn_ rec source records and worst-case expected endpoint risk at most 1/161/16 must satisfy Q≥5896m2−nrecH−1,Q≥ 5896m^2-n_ recH-1, (C.4) even when granted the binding and adaptive chosen training prompts. Hence the query coordinate is minimax-optimal in this fixed-confidence, logarithmic-source-budget and logarithmic-horizon regime. The low-error structure is the key to the upper bound. For the binary entrywise difference f between two transition tables, form a signed transition matrix A(z,z′)=(−1)f(z′+z,z)/mA(z,z )=(-1)^f(z +z,z)/m, indexed by public states z,z′z,z and its tagged counterpart. Their powers describe the probability of agreement along a random suffix. Writing ε for training-endpoint disagreement probability, near-perfect agreement forces one matrix close to a signed rank-one form. After sign rounding, only O(εm2/L)O( m^2/L) transition defects and O(εm)O( m) vertex-sign choices remain at error ε . The resulting small hypothesis count makes O(m2)O(m^2) iid equations sufficient when L=Θ(logm)L= ( m). A separate short-witness argument rules out arbitrarily small nonzero error before applying this rounding estimate. The binding prefix has its own sampling law. Retaining its factor throughout the suffix approximation controls accuracy on this distinguished row. Finally, bounded source sign statistics recover the two residual bits. The source is independent of the feedback calculation because it is stored but not used during that calculation. The lower bound packs 2Θ(m2)2 (m^2) separated mechanisms and counts at most nrecH+Qn_ recH+Q information bits. The proofs and complete computation/storage charges are in Section C.2. The logarithmic source-token charge includes the length of a mandatory whole record. At fixed confidence and fixed noise level, decoding the two ambiguity parameters needs only a constant-order number of informative transitions. Thus record length and the information needed for the two residual bits make distinct contributions to the token count. Lemma C.3 (The complete two-bit kernel). In the sequential model of Section 3, fix uηu_η and let H≥3H≥ 3. The two worlds have equal endpoints on all training prompts if and only if their difference has the form (C.1). Such a difference changes every test endpoint by the bit d. Equality of these endpoints is equivalent to equality of the full terminal verification oracle. Proof. Fix the public binding coordinate uηu_η. Write f(u,z)=θu,z′+θu,zf(u,z)=θ _u,z+ _u,z and β=bη′+bηβ=b _η+b_η. At suffix-start coordinate z, equality of all training endpoints is equivalent to β+f(uη,z+uη)+∑j=1Lf(vj,zj−1)=0,zj=zj−1+vj,∑jℓ(vj)=0.β+f(u_η,z+u_η)+ _j=1^Lf(v_j,z_j-1)=0, z_j=z_j-1+v_j, _j (v_j)=0. (C.5) Consequently every admissible length-L path starting at z has the same cost. Its all-zero path has cost Lf(0,z)Lf(0,z). For ℓ(u)=0 (u)=0, compare (u,0,…,0)(u,0,…,0) with (0,u,0,…,0)(0,u,0,…,0). This gives f(0,z)=f(0,z+u)f(0,z)=f(0,z+u). The set ℓ−1(0) ^-1(0) spans Z: it contains the coordinate vectors in the x and y components and t+x1+y1t+x_1+y_1, whose tag is zero. Thus f(0,z)=κf(0,z)=κ is constant. Comparing a single tag-zero control followed by zeros with the all-zero path gives f(u,z)=κf(u,z)=κ for every tag-zero control. For tag-one controls u,vu,v, the path (u,v,0,…,0)(u,v,0,…,0) has tag zero. Comparing with the all-zero path gives f(u,z)=f(v,z+u)f(u,z)=f(v,z+u). At any fixed vertex all tag-one outgoing costs therefore agree; call their value a(z)a(z). The same equality gives a(z)=a(z+u)a(z)=a(z+u) for each tag-one u. The tag-one controls contain t,t+xi,t+yit,t+x_i,t+y_i and span Z, so a(z)=λa(z)=λ is constant. Hence f(u,z)=κ+dℓ(u),d=κ+λ.f(u,z)=κ+d (u), d=κ+λ. (C.6) Substitution into (C.5) forces β=Hκ+dℓ(uη)β=Hκ+d (u_η). Conversely these differences cancel on every tag-zero suffix and produce the difference d on every tag-one suffix. This argument requires only L≥2L≥ 2. Endpoint equality is equivalent to equality of the full binary verification oracle once uηu_η agrees. Invalid grammar or public-coordinate paths are rejected in both worlds. On any given prompt one can instead choose a valid public-coordinate path and either final state bit. Exactly the correct endpoint is accepted, independently of the intermediate state bits. These two candidates distinguish unequal endpoints. This proves the claimed equivalence for the actual verifier, including repaired trajectories. ∎ Proof of Theorem C.1. With L=H−1≥2L=H-1≥ 2, Lemma C.3 gives both directions of (C.1) and equivalence for the actual verifier. On every test prompt the remaining endpoint difference is d, so all test targets differ exactly when d=1d=1. ∎ Lemma C.4 (Binding-coordinate identification and exact equations). For 0<δ<10<δ<1, Qid=m⌈log2(3/δ)⌉Q_ id=m _2(3/δ) fresh iid queries identify uηu_η with failure probability at most δ/3δ/3. Conditional on successful identification, each subsequent fresh prompt and binary reply yield the exact iid binary equation (C.7) in m2+1m^2+1 unknown bits. Proof. First identify uηu_η. For each of its m possible values, use at most k=⌈log2(3/δ)⌉k= _2(3/δ) fresh iid training prompts. Output a grammar-compliant candidate with the required public coordinates for that coordinate and a fresh fair final state bit. A wrong coordinate never succeeds; the correct coordinate succeeds with probability 1/21/2 independently on each trial. Stop at the first success, or declare failure after the fixed budget. The failure probability is at most 2−k≤δ/32^-k≤δ/3, and the query cost is at most Qid=mkQ_ id=mk. Thereafter choose A=(0,uη)A=(0,u_η) and a compliant trajectory with final state bit zero. Its binary reward reveals the true endpoint bit: reward one means bit zero, and reward zero means bit one. Subtract the known initial state bit and the displayed control bits. The remaining label is bη+θuη,z0+∑i=2Hθui,zi−1,z1=z0+uη,zi=zi−1+ui.b_η+ _u_η,z_0+ _i=2^H _u_i,z_i-1, z_1=z_0+u_η, z_i=z_i-1+u_i. (C.7) Each observation is a known binary row in m2+1m^2+1 unknown bits. The rows are iid because prompts remain fresh. Gaussian elimination returns any consistent representative. It remains to show that a controlled number of these particular path rows eliminates every difference outside the two-bit kernel. Counting the number of unknown bits alone does not establish that assertion. ∎ Lemma C.5 (Tagged convolution norm). For the sequential tag ℓ(t,x,y)=t+x⋅y (t,x,y)=t+x· y, the matrix Mℓ(z,w)=m−1(−1)ℓ(w+z)M_ (z,w)=m^-1(-1) (w+z) has operator norm ρ=2−r≤1/2ρ=2^-r≤ 1/2. Proof. Define the signed convolution Mℓ(z,w)=m−1(−1)ℓ(w+z).M_ (z,w)=m^-1(-1) (w+z). (C.8) At Walsh frequency (α,β,τ)(α,β,τ) its eigenvalue is the average of (−1)t+x⋅y+α⋅x+β⋅y+τt(-1)^t+x· y+α· x+β· y+τ t. It vanishes when τ=0τ=0. When τ=1τ=1, the sum over y forces x=βx=β and leaves 2−r(−1)α⋅β2^-r(-1)^α·β. Thus ∥Mℓ∥=ρ=2−r≤1/2 M_ =ρ=2^-r≤ 1/2. ∎ Lemma C.6 (Short witness or vertex potential). Every binary transition cost f either has two length-nine control blocks with the same start, end and tag but opposite costs, or admits (C.9) for a vertex potential h:Z→2h:Z _2 and bits κ,dκ,d. Proof. For any cost f, either two length-nine blocks have the same start, end and tag but opposite costs, or f(u,z)=κ+h(z)+h(z+u)+dℓ(u).f(u,z)=κ+h(z)+h(z+u)+d (u). (C.9) To see this, comparisons (0,u)(0,u) versus (u,0)(u,0) detect nonconstant self-loop costs. Without such a witness, subtract their constant κ. Comparisons (u,u)(u,u) versus (0,0)(0,0) then detect unequal forward and reverse costs. If those also agree, use the undirected lifted graph on (z,e)∈Z×2(z,e)∈ Z×F_2, whose u-edge leads to (z+u,e+ℓ(u))(z+u,e+ (u)). A direct edge gives any public displacement; the three-edge loop (x1,y1,x1+y1)(x_1,y_1,x_1+y_1) has zero displacement and odd tag. The graph therefore has diameter at most four. A breadth-first spanning tree has depth at most four. A non-tree edge violating a potential produces a fundamental closed walk of length at most nine with odd cost. Pad this walk and the zero walk to length nine. If no such violation occurs, the graph has a potential G(z,e)G(z,e). Since costs do not depend on e, the difference G(z,e+1)+G(z,e)G(z,e+1)+G(z,e) is constant along edges, hence is a constant d. Writing G(z,e)=h(z)+deG(z,e)=h(z)+de proves (C.9). All earlier short witnesses can likewise be padded because their endpoints agree. ∎ Lemma C.7 (Coarse separation of nonkernel differences). For L≥10L≥ 10, every binary difference hypothesis outside the kernel in Lemma C.3 has training-endpoint disagreement probability at least m−10m^-10. Proof. Use the alternatives in Lemma C.6. Place either witness block at the beginning of the suffix and fix its public starting vertex. At least one suffix control remains when L≥10L≥ 10. Conditioning the entire suffix tag to zero leaves the probability of each fixed block and starting vertex exactly m−10m^-10. Couple the remaining controls identically. The endpoint-difference parities of the paired words disagree, so at least one is wrong. The error is at least m−10m^-10; the two block probabilities already pay for the pairing, with no additional factor 1/21/2. For (C.9), the full endpoint difference is β+Hκ+h(z0)+h(zH)+dℓ(uη).β+Hκ+h(z_0)+h(z_H)+d (u_η). (C.10) The conditional final public-coordinate law is m−1+MℓL(z0+uη,zH)m^-1+M_ ^L(z_0+u_η,z_H). Every entry is at least m−1−2−rL≥(2m)−1m^-1-2^-rL≥(2m)^-1 for L≥10L≥ 10. A nonconstant h therefore gives error at least (2m)−1(2m)^-1. A constant h gives a kernel difference or error one. In particular every nonkernel hypothesis has training disagreement at least m−10m^-10. ∎ Lemma C.8 (Low-error count and refined separation). Assume L≥⌈216ln(2m)⌉L≥ 2^16 (2m) . For 0<ε≤1/8000< ≤ 1/800, the number of difference hypotheses with error at most ε is bounded by 16exp160ε(m+m2L)ln(m+1).16 \! \160 (m+ m^2L ) (m+1) \. (C.11) Moreover every nonkernel difference has error at least ε∗=minL24m2,152m,1800. _*= \ L24m^2, 152m, 1800 \. (C.12) Proof. Here is the quantitative argument. Put A(z,w)=(−1)f(w+z,z)/mA(z,w)=(-1)^f(w+z,z)/m and Ac(z,w)=(−1)ℓ(w+z)A(z,w)A_c(z,w)=(-1) (w+z)A(z,w). Both have Frobenius norm one. Keep the binding factor v(z)=(−1)β+f(uη,z+uη)v(z)=(-1)^β+f(u_η,z+u_η). Then 1−2ε=m−1⟨v,(AL+AcL)⟩.1-2 =m^-1 v,(A^L+A_c^L)1 . (C.13) Select the branch of larger operator norm a0a_0. It satisfies a0L≥(1−2ε)/2≥1/4a_0^L≥(1-2 )/2≥ 1/4, so δA=1−a02≤2ln4/L _A=1-a_0^2≤ 2 4/L. Its best rank-one singular approximation has squared residual δA _A. Round its left and right singular vectors to signs h,kh,k. Each incorrectly rounded matrix entry contributes at least 1/m21/m^2 to this residual. The rounded matrix P(z,w)=h(z)k(w)/mP(z,w)=h(z)k(w)/m thus differs in at most δAm2 _Am^2 signs and in Frobenius norm by at most 2δA2 _A. The complementary tagged branch has norm at most ρ+2δA≤5/8ρ+2 _A≤ 5/8. Consequently a0L≥1−2ε−(5/8)La_0^L≥ 1-2 -(5/8)^L. For a nonkernel difference the preceding coarse bound gives ε≥m−10 ≥ m^-10, whereas (5/8)L≤m−20(5/8)^L≤ m^-20. Taking logarithms yields δA≤12ε/L,#f≠f0≤12εm2/L, _A≤ 12 /L, \#\f≠ f_0\≤ 12 m^2/L, (C.14) where f0f_0 is the rounded separable cost, with the chosen tag branch restored. Zero singular-vector coordinates cause no problem: any wrong sign still has the stated residual cost. Kernel differences may be counted separately below the coarse separation. Retain the original factor v and replace only the suffix costs by f0f_0. Before tag conditioning every suffix edge is uniform on m2m^2 pairs; conditioning costs at most a factor two. A union bound over L edges and (C.14) bounds the change of suffix parity by 24ε24 . The auxiliary predictor therefore has error at most 25ε25 . This replacement does not require a bound on the distinguished binding row. Write g=khg=kh, with pointwise multiplication. The exact rank-one identity is PL=hkh¯L−1k¯.P^L1=h\, kh^\,L-1 k. (C.15) The complementary signed matrix of P has norm ρ. Thus auxiliary agreement is at most |k¯||g¯|L−1+ρL| k|\,| g|^L-1+ρ^L. Since ρL≤ερ^L≤ , the elementary logarithmic bounds from agreement at least 1−50ε1-50 imply minority(k)≤26εm,minority(g)≤102εm/L.minority(k)≤ 26 m, (g)≤ 102 m/L. (C.16) Choose the branch, the two global signs and β in at most 1616 ways. The remaining choices are bounded by 16 16 [∑i≤⌊26εm⌋(mi)][∑j≤⌊102εm/L⌋(mj)][∑d≤⌊12εm2/L⌋(m2d)]. [ _i≤ 26 m mi ] [ _j≤ 102 m/L mj ] [ _d≤ 12 m^2/L m^2d ]. (C.17) Using ∑j≤t(nj)≤(n+1)t _j≤ t nj≤(n+1)^t for integral t≥0t≥ 0 gives (C.11); the coefficient 160160 is conservative. If ε<ε∗ < _*, all three integer radii are zero. Then k,gk,g are constant, there are no defects, and f is a constant plus a tag. Its binding bit gives either the kernel or error one. This proves (C.12). ∎ Lemma C.9 (Finite iid schedule for coset recovery). Under the horizon assumption of Lemma C.8, the iid binary-row learner of Lemma C.4 with QrankQ_ rank in (C.18) recovers the correct observable coset with conditional failure probability at most γ=δ/3γ=δ/3. Proof. Let D=m+m2/LD=m+m^2/L, B=2+⌈log2(52m2)⌉B=2+ _2(52m^2) and γ=δ/3γ=δ/3. Use Qrank=⌈max800[(m2+1)ln2+ln(2/γ)],640Dln(m+1)+ε∗−1ln(32B/γ)⌉.Q_ rank= \ aligned &800 [(m^2+1) 2+ (2/γ) ],\\ &640D (m+1)+ _*^-1 (32B/γ) aligned \ . (C.18) The 2m2+12^m^2+1 hypotheses of error at least 1/8001/800, including equality, survive with total probability at most γ/2γ/2. Partition the remaining interval [ε∗,1/800)[ _*,1/800) into at most B dyadic bins with lower endpoints eje_j, truncating the final bin. Equation (C.11) bounds each bin’s size by 16exp[320ejDln(m+1)]16 [320e_jD (m+1)]; survival is at most exp(−Qrankej) (-Q_ ranke_j). Equation (C.18) makes the contribution at most γ/(2B)γ/(2B) per bin. If ε∗=1/800 _*=1/800, the low-error interval is empty. Thus elimination recovers the correct coset with failure at most γ. ∎ Lemma C.10 (Recovering both gauges from retained source records). Let a>0a>0, L=H−1L=H-1, and suppose the coordinate/rank procedure has recovered the correct coset without using the independently stored source records. With nrecn_ rec in (C.20), both residual gauge bits are recovered with conditional failure at most δ/3δ/3. Correcting the representative then gives the true binding bit and transition table, hence canonical execution and the target first state on every test prompt. Proof. Store raw source records before feedback but do not use them in the coordinate/rank procedure. Conditional on a fixed world and the feedback representative θ0θ^0, raw source transitions remain independent of that procedure. For an observed transition from (s,z)(s,z) under (b,u)(b,u) to a state with state bit s′s , set R=s′+s+b+θu,z0,Xg=ℓ(u)=g(−1)R,g∈0,1.R=s +s+b+θ^0_u,z, X_g=1\ (u)=g\(-1)^R, g∈\0,1\. (C.19) On coset recovery, R=κ+dℓ(u)R=κ+d (u) in the true-transition component, and it is a fair bit in the reset component, even when the public coordinate is wrong. For a sequentially uniform control, [Xg∣past]=(a/2)(−1)κ+dgE[X_g ]=(a/2)(-1)^κ+dg. Use the first H−1H-1 execution transitions in each record. The control U1U_1 is independent uniform, and U2,…,UH−1U_2,…,U_H-1 are a strict subset of the tag-conditioned suffix and remain sequentially uniform. Marginalize the unused final control. This is an execution-order proof filtration, not a claim that the raw serialization reveals variables in that order. For nrecn_ rec records, bounded martingale differences give a two-sign error probability at most 2exp[−nrec(H−1)a2/8]2 [-n_ rec(H-1)a^2/8]. Taking nrec=⌈8a−2ln(6/δ)L⌉n_ rec= 8a^-2 (6/δ)L (C.20) makes this at most δ/3δ/3. Both gauges, including the endpoint-irrelevant common gauge, are recovered. Correcting θ0θ^0 and the binding bit therefore gives the true first-state target and the full canonical execution on every test prompt. Coordinate, rank and source failure allowances sum to δ. ∎ Lemma C.11 (Complete low-source learner and its resources). Combining Lemmas C.4, C.9 and C.10 with their three failure allocations gives simultaneous canonical execution on every test prompt with probability at least 1−δ1-δ. Its source tokens are at most 32a−2ln(6/δ)+2H+232a^-2 (6/δ)+2H+2, and its query, generation, computation and storage costs are the quantities below. Proof. The three failure allowances sum to δ, and on their common success event all test executions are correct by Lemma C.10. The raw token cost is Npre=(2H+2)nrec≤32a−2ln(6/δ)+2H+2.N_ pre=(2H+2)n_ rec≤ 32a^-2 (6/δ)+2H+2. (C.21) The query cost is Q=Qid+QrankQ=Q_ id+Q_ rank, generation costs (H+2)Q(H+2)Q decisions, and prompts contribute HQHQ tokens. There are zero gradient updates and no checkpoint or test selection. Dense binary elimination costs conservatively O(Q(Hlogq+m4)+Nprelogq)O(Q(H q+m^4)+N_ pre q) bit operations, with O(m4+Hlogq+Nprelogq)O(m^4+H q+N_ pre q) stored bits when raw source is retained. The verifier separately performs H trusted transitions and H public-coordinate checks per query. Deployment uses H table lookups and O(H)O(H) generated symbols; no external shield is inserted. ∎ Lemma C.12 (Packing separated endpoint functions). For m≥512m≥ 512 and H≥5H≥ 5, restrict to mechanisms with θ0,v≠θ0,0 _0,v≠ _0,0 for a fixed v≠0v≠ 0. There exists a packing of Kpack=2m2/128K_ pack=2^m^2/128 mechanisms with pairwise endpoint disagreement at least 7/167/16, even with a fixed known binding. Proof. We give the elementary packing calculation to specify the stronger information permissions of the lower bound. For a random pair of such tables, its difference matrix has independent fair signs except for one tied entry. For the signed matrices A,AcA,A_c above, couple each to an iid sign matrix; changing the tied entry costs at most 2/m2/m in operator norm. Two 1/41/4-nets of size 9m9^m and scalar Hoeffding bounds yield Pr[max(∥A∥,∥Ac∥)>12]≤4exp[2mln9−(m−4)232]≤e−m2/64. \! [ ( A , A_c )> 12 ]≤ 4 \! [2m 9- (m-4)^232 ]≤ e^-m^2/64. (C.22) A union bound over fewer than Kpack2K_ pack^2 pairs proves existence. The fixed initial binding contributes the diagonal sign matrix Dη(z,z)=(−1)f(uη,z)D_η(z,z)=(-1)^f(u_η,z) and permutation Pu(z,z′)=z′=z+uP_u(z,z )=1\z =z+u\, both orthogonal. The signed test agreement is m−1⊤DηPuη(AH−1−AcH−1),m^-11 D_ηP_u_η(A^H-1-A_c^H-1)1, (C.23) whose magnitude is at most 22−H2^2-H. Thus disagreement is at least (1−22−H)/2≥7/16(1-2^2-H)/2≥ 7/16. ∎ Lemma C.13 (Mixed source-and-feedback information lower bound). For m≥512m≥ 512 and H≥5H≥ 5, every learner with nrecn_ rec source records and at most Q binary verification replies satisfies (C.24), even with known binding and adaptively chosen training prompts. In particular, worst-world expected endpoint risk at most 1/161/16 requires (C.4). Proof. Use the packing of Lemma C.12. Let J be a uniformly chosen packing index. Reveal the learner’s independent random seed at the start of the transcript and condition on it; subsequent randomized choices are then deterministic functions of that seed and preceding observations. Raw controls and resets convey no mechanism information. Conditional public-coordinate transitions have a mechanism-independent law; each next state bit conveys at most one bit. Hence nrecn_ rec source records convey at most nrecHn_ recH bits. Any adaptive binary verification replies add at most Q bits, even when the learner may choose its training prompts. Conditional on this seed and the preceding transcript, the chosen queries supply no additional mechanism information. Therefore I(J,transcript)≤nrecH+QI(J;transcript)≤ n_ recH+Q bits. Decode the closest packing endpoint function from the learned predictor. A mistaken index forces endpoint error at least 7/327/32 by the triangle inequality. For a randomized predictor use its conditional output probabilities as a randomized endpoint function; malformed outputs only add error. The entropy encoding bound H(J∣transcript)≤1+perrlog2KpackH(J )≤ 1+p_ err _2K_ pack gives supθ,η[endpoint risk]≥732[1−128(nrecH+Q+1)m2]+. _θ,ηE[endpoint risk]≥ 732 [1- 128(n_ recH+Q+1)m^2 ]_+. (C.24) For worst-case risk at most 1/161/16, this implies Q≥5m2/896−nrecH−1Q≥ 5m^2/896-n_ recH-1. ∎ Lemma C.14 (Matched query order at logarithmic source budget). For δ=1/16δ=1/16, fixed c∈(0,1/64]c∈(0,1/64], a=1−c/Ha=1-c/H, and L=⌈216ln(2m)⌉L= 2^16 (2m) , the sequential learner has Npre=O(logm)N_ pre=O( m) and Q=O(m2)Q=O(m^2), while every learner at this source budget with worst-world expected endpoint risk at most 1/161/16 has Q=Ω(m2)Q= (m^2) as m→∞m→∞. Proof. At fixed confidence δ=1/16δ=1/16, fixed c∈(0,1/64]c∈(0,1/64], a=1−c/Ha=1-c/H and L=⌈216ln(2m)⌉L= 2^16 (2m) , the upper has Npre=O(logm)N_ pre=O( m) and Q=O(m2)Q=O(m^2). Its perfect-recovery event implies the required expected-risk upper. Equation (C.24) gives Q=Ω(m2)Q= (m^2) at that source budget. This matches the query coordinate in this regime, while retaining the whole-record cost in the source coordinate. ∎ Proof of Theorem C.2. Lemma C.11 supplies the stated success event and all operational costs. By (C.12), ε∗−1=max24m2/L,52m,800 _*^-1= \24m^2/L,52m,800\, and the horizon condition implies L≥1L≥ 1. Substitution into (C.18), together with Qid=m⌈log2(3/δ)⌉Q_ id=m _2(3/δ) , yields Q=O(m2+(m+m2/L)ln(m/δ)+mln(1/δ))Q=O(m^2+(m+m^2/L) (m/δ)+m (1/δ)); the factor lnB=O(lnm) B=O( m) is absorbed here. Exact canonical execution gives endpoint success, target first-state retention and task utility one on the same event. Lemma C.13 supplies (C.4), and Lemma C.14 supplies the fixed-confidence specialization and query optimality. ∎ C.3 A finite-budget boundary for a known pair of worlds We isolate the gauge ambiguity from the cost of learning an unknown transition table. The following experiment supplies the learner with two candidate worlds and asks which one generated its observations. It does not supply the true member. This restriction permits an exact boundary; it is not an upper bound for the full unknown-table class. Definition C.15 (Known-pair sequential observation experiment). Fix H≥3H≥ 3 and a sequential world (θ0,η0)(θ^0,η^0). Let θu,z1=θu,z0+ℓ(u)θ^1_u,z=θ^0_u,z+ (u), uη1=uη0u_η^1=u_η^0 and bη1=bη0+ℓ(uη0)b_η^1=b_η^0+ (u_η^0), with all additions in 2F_2. Both tables and both bindings are known candidates. The unknown index is W∈0,1W∈\0,1\. Let D0,D1D_0,D_1 be the common prompt laws conditional on even and odd suffix tag parity, respectively. An initial observation O has law PwP_w in world w and includes all source records, inherited checkpoint weights and other side information jointly. Take its observation space to be standard Borel. After O, at most Q∈ℕ0Q _0 verifier queries receive fresh exogenous iid prompts from Dε=(1−ε)D0+εD1D_ =(1- )D_0+ D_1, 0≤ε≤10≤ ≤ 1. The prompt sequence is independent of (W,O)(W,O); every presented prompt costs a query, including a rejection. At a prompt, the learner chooses a candidate trajectory adaptively and receives the deterministic endpoint verifier’s reply. There is no uncharged prompt selection. The final test prompt has law D1D_1 independently of training and W. The loss is endpoint failure, including malformed outputs. Procedures may randomize and have no computational restriction. Write μ=P0+P1μ=P_0+P_1, pw=dPw/dμp_w=dP_w/dμ, and α=∫min(p0,p1)μ=1−TV(P0,P1)α= (p_0,p_1)\,dμ=1-TV(P_0,P_1). Proposition C.16 (Exact pair risk under rare informative prompts). In Definition C.15, for a prior Pr(W=0)=π (W=0)=π, the optimal Bayes risk is Rπ∗(Q)=(1−ε)Q∫minπp0,(1−π)p1μ.R_π^*(Q)=(1- )^Q \π p_0,(1-π)p_1\\,dμ. (C.25) Here (1−ε)0=1(1- )^0=1, including at ε=1 =1. If rmm(P0,P1)r_ m(P_0,P_1) is the infimum, over randomized tests based on O, of their maximum error in the two worlds, then Rmm∗(Q)=(1−ε)Qrmm(P0,P1).R_ m^*(Q)=(1- )^Qr_ m(P_0,P_1). (C.26) For the equal prior, R1/2∗(Q)=12(1−ε)QαR_1/2^*(Q)= 12(1- )^Qα. This also equals the minimax risk if a measurable involution of O exchanges P0P_0 and P1P_1. Lemma C.17 (Parity equivalence and an identifying query). In Definition C.15, all verifier replies coincide in the two worlds on an even prompt. On an odd prompt the canonical trajectory of candidate 00 receives reply 1−W1-W, identifying W exactly. Proof. Public state coordinates are the same in the two worlds. Their private endpoint difference at a fixed prompt is (bη1+bη0)+(θuη,z01+θuη,z00)+∑i=2H(θui,zi−11+θui,zi−10)=∑i=2Hℓ(ui).(b_η^1+b_η^0)+(θ^1_u_η,z_0+θ^0_u_η,z_0)+ _i=2^H(θ^1_u_i,z_i-1+θ^0_u_i,z_i-1)= _i=2^H (u_i). The first two terms are both ℓ(uη) (u_η) and cancel. On an even prompt, the endpoint targets and all verifier replies coincide: malformed strings fail in both worlds; admissibility checks the common public path, and the final bit is the same. Intermediate private bits are not verified. On an odd prompt, submit the canonical trajectory of candidate world 00. Its public path is legal in either world, while its endpoint is correct precisely when W=0W=0. The reply is therefore 1−W1-W. One odd prompt identifies the candidate world exactly. ∎ Lemma C.18 (No information gain on the all-even event). In Definition C.15, every adaptive procedure has Bayes risk at least the right-hand side of (C.25) and maximum-world risk at least the right-hand side of (C.26). Proof. If (1−ε)Q=0(1- )^Q=0, both proposed lower bounds are zero and follow from nonnegativity. Otherwise the following conditioning is well defined. Presample Q prompts even if a procedure stops early. Let E be the event that all are even; Pr(E∣W,O)=(1−ε)Q (E W,O)=(1- )^Q. Conditional on E, the prompt law is world independent. At query j, a common history and the procedure’s same random seed induce the same submitted string; Lemma C.17 gives the same reply. Induction over j=1,…,Qj=1,…,Q shows that the full training transcript conditional on E is a common Markov kernel applied to O. The independent test prompt supplies only additional world-independent randomness. From any output, define a test choosing world 00 exactly when the output passes its endpoint verifier, and choosing 11 otherwise. Since the two odd-prompt targets are disjoint, this test’s error is at most the output’s endpoint failure in each world. Thus, conditional on E, no procedure improves on a randomized test based on O. For prior π, pointwise choice between the two weighted densities gives the error integral in (C.25). Dropping nonnegative errors on EcE^c proves its lower bound. For the minimax claim, the same induced test has two conditional errors e0,e1e_0,e_1, with maxwew≥rmm _we_w≥ r_ m. The unconditional maximum error is at least (1−ε)Qmaxwew(1- )^Q _we_w, proving the lower bound in (C.26). ∎ Lemma C.19 (Attaining pair risk and symmetrizing tests). In Definition C.15, the Bayes lower bound in (C.25) is attained, and the minimax lower bound in (C.26) is an attainable infimum. If a measurable involution exchanges P0,P1P_0,P_1, their minimax testing risk equals their equal-prior Bayes testing risk. Proof. Use the separating candidate at the first odd prompt. If there is one, execute the identified world at test time. If none occurs, use a Bayes-optimal test of O, or an arbitrarily near-optimal minimax test, and execute the chosen world. At an odd test prompt the output is correct exactly when this choice is correct. These procedures attain the Bayes bound and approach the minimax infimum. For an involution T exchanging the two laws, transform a test f into 1−f∘T1-f T; its two errors are those of f in reverse order. A fair mixture has equal errors, each the equal-prior average error of f. Applying this to a Bayes-optimal test proves that minimax equals equal-prior Bayes risk under symmetry. Without this assumption, the two quantities need not agree. ∎ Proof of Proposition C.16. If ε=1 =1 and Q≥1Q≥ 1, every prompt is odd; Lemma C.17 attains zero risk. In all other cases Lemmas C.18 and C.19 give the matching lower and upper values. For equal prior, the weighted-density integral is α/2α/2 by Definition C.15. The symmetry assertion follows from Lemma C.19. The convention at Q=0Q=0 corresponds to no prompt being observed. ∎ Definition C.20 (Independent-record specialization). Suppose O consists of n independent raw source records from the usual even-suffix source law, together with an independent, world-independent initializer. The initial state is uniform, U1U_1 is uniform, and the suffix is uniform conditional on even tag parity. Each state transition uses aPU+(1−a)ΠaP_U+(1-a) with 0≤a≤10≤ a≤ 1 and fresh transition noise. The endpoint values a=0,1a=0,1 below concern this information experiment; the finite-training results keep their own prescribed a=1−c/Ha=1-c/H. Put GH(z) G_H(z) =(1+z)[(1+z)H−1+(1−z)H−1]2H, = (1+z) [(1+z)^H-1+(1-z)^H-1 ]2^H, (C.27) h h =a+2(1−a)q,r=a+(1−a)/qh,ck=[zk]GH(z)n. =a+ 2(1-a)q, r= a+(1-a)/qh, c_k=[z^k]G_H(z)^n. (C.28) For j≥0j≥ 0 define the majority error with a fair tie break by bj(r)=∑t=0⌊(j−1)/2⌋(jt)rt(1−r)j−t+j even2(j/2)[r(1−r)]j/2.b_j(r)= _t=0 (j-1)/2 jtr^t(1-r)^j-t+ 1_\j even\2 jj/2[r(1-r)]^j/2. (C.29) For odd j, the tie term is absent; b0(r)=1/2b_0(r)=1/2 and an empty sum is zero. Corollary C.21 (Source records and reward queries for the pair). Under Definition C.20, the equal-prior Bayes and minimax risks both equal (1−ε)Q∑k=0nHck∑j=0k(kj)hj(1−h)k−jbj(r).(1- )^Q _k=0^nHc_k _j=0^k kjh^j(1-h)^k-jb_j(r). (C.30) In the noiseless case a=1a=1, this reduces to R∗(n,Q)=12 2−n(H−1)(1−ε)Q.R^*(n,Q)= 12\,2^-n(H-1)(1- )^Q. (C.31) Consequently, in the noiseless case a=1a=1, for 0<δ<1/20<δ<1/2 and 0≤ε<10≤ <1, error at most δ is possible exactly when n(H−1)log2+Q[−log(1−ε)]≥log12δ.n(H-1) 2+Q[- (1- )]≥ 12δ. (C.32) Lemma C.22 (Symmetry of complete source records). Under Definition C.20, there is a measurable involution of all observed records and the independent initializer exchanging P0P_0 and P1P_1. Hence equal-prior Bayes and minimax risks agree. Proof. For a record with states (sj,zj)(s_j,z_j) and controls (bj,uj)(b_j,u_j), leave controls and public states fixed and replace each private state by sj+∑i=1jℓ(ui)s_j+ _i=1^j (u_i), leaving s0s_0, PAD and EOS fixed. Applying this map twice is the identity. A transition following θ0θ^0 becomes a transition following θ1θ^1; a uniform reset remains uniform. The control law is unchanged, so the map exchanges the two complete-record laws. Apply it separately to all records and leave the independent initializer fixed. The resulting involution exchanges P0,P1P_0,P_1; Proposition C.16 therefore equates Bayes and minimax risk. ∎ Lemma C.23 (The source innovation channel). Under Definition C.20, conditional on the controls, the sequential innovation transformation displayed below is invertible and has a product likelihood. A tag-zero position is uninformative. Each tag-one position independently gives an erasure with probability 1−h1-h or a binary observation equal to W with probability r, conditional on not being erased, with h,rh,r from (C.28). Proof. Given the controls and initial state, define innovations relative to candidate 00: ejz=zj−zj−1−uj,ejs=sj−sj−1−bj−θuj,zj−10.e_j^z=z_j-z_j-1-u_j, e_j^s=s_j-s_j-1-b_j-θ^0_u_j,z_j-1. This is an invertible sequential transformation of the observed states. When ℓ(uj)=0 (u_j)=0, the two conditional innovation laws coincide. When ℓ(uj)=1 (u_j)=1, the innovation (ejz,ejs)=(0,w)(e_j^z,e_j^s)=(0,w) has probability a+(1−a)/qa+(1-a)/q in world w; each of the other q−1q-1 innovations has probability (1−a)/q(1-a)/q. These probabilities do not depend on previous states. Iterating the conditional factorization shows that, given the controls, the innovation likelihood factors over positions. An innovation with ejz≠0e_j^z≠ 0 has world-independent likelihood. The event ejz=0e_j^z=0 has probability h in either world; conditional on it, the private innovation equals W with probability r. Thus each tag-one position gives either an erasure or an independent binary observation with correctness r≥1/2r≥ 1/2. Tag-zero positions give no information. Because the erasure and control laws do not depend on W, they supply no additional likelihood factor. ∎ Lemma C.24 (Tag counts and majority error). Under Definition C.20, the total number K of tag-one positions has law Pr(K=k)=ck (K=k)=c_k. Given K=kK=k, the number J of nonerased observations is Bin(k,h)Bin(k,h). For J=jJ=j, the optimal equal-prior classification error is bj(r)b_j(r) in (C.29), including j=0j=0 and r=1/2,1r=1/2,1. Proof. The balanced tag ℓ(u)=t+x⋅y (u)=t+x· y is a fair bit under uniform u. For one record, its first tag is an independent fair bit, while its H−1H-1 suffix tags are uniform over even-parity bit strings. The generating function of the latter count is [(1+z)H−1+(1−z)H−1]/2H−1[(1+z)^H-1+(1-z)^H-1]/2^H-1. Multiplication by (1+z)/2(1+z)/2 gives (C.27). Independent records give Pr(K=k)=ck (K=k)=c_k for the total tag-one count. Given K=kK=k, the number J of nonerased observations is Bin(k,h)Bin(k,h). Given J=jJ=j, the number of correct binary observations is Bin(j,r)Bin(j,r). For 1/2<r<11/2<r<1, the likelihood ratio increases with the number voting for world 11, so majority with a fair tie break is Bayes-optimal. At r=1/2r=1/2 all choices have average error 1/21/2, and at r=1r=1 any nonempty sample identifies W; the same formula includes both endpoints. Its error is exactly (C.29). Averaging first over J, then K, and multiplying by the all-even reward probability from Proposition C.16 proves (C.30). ∎ Proof of Corollary C.21. Lemma C.22 gives symmetry, Lemma C.23 gives the channel likelihood, and Lemma C.24 gives its count distribution and optimal classification error. Averaging bj(r)b_j(r) first against Bin(k,h)Bin(k,h) and then against ckc_k, and applying Proposition C.16, proves (C.30) for both Bayes and minimax risk. At a=1a=1, h=r=1h=r=1. Any tag-one transition identifies the world, so only K=0K=0 contributes error 1/21/2. Since GH(0)=2−(H−1)G_H(0)=2^-(H-1), its probability is 2−n(H−1)2^-n(H-1). This proves (C.31). Taking logarithms of its comparison to δ gives (C.32), including n=0n=0 or Q=0Q=0. ∎ Corollary C.25 (Limiting observation regimes). Under Definition C.20, for n=0n=0 or a=0a=0, the risk is (1−ε)Q/2(1- )^Q/2. For ε=0 =0, reward cannot improve the source-only risk; for ε=1 =1 and Q≥1Q≥ 1 it is zero. A checkpoint that already distinguishes the worlds has disjoint initial laws and zero risk even with Q=0Q=0. Proof. For n=0n=0 the initial laws coincide. At a=0a=0 all transitions are uniform resets and the two source laws again coincide. In both cases α=1α=1 and symmetry gives the displayed risk by Proposition C.16. The values at ε=0 =0 and ε=1,Q≥1 =1,Q≥ 1 follow from its factor (1−ε)Q(1- )^Q. Disjoint initial laws give α=0α=0 and identify the world from O alone. ∎ Proposition C.16 accounts for checkpoint information through its joint law with source, including their dependence. This is an observation-based identification result; the optimization results use their specified random initializations. The factor (1−ε)Q(1- )^Q relies on charged, exogenous prompts: freely choosing an odd prompt reduces the identification cost to one query. The source channel and likelihood test are available because both candidate tables are known. Learning an arbitrary unknown table still incurs the separate identification costs in Theorem C.2; the pair formula does not remove them. C.4 Information supplied by contextual source observations Use the contextual observation model of Section 5.1, with nctx≥1n_ ctx≥ 1, odd cue dimension k=p−2k=p-2, independent uniform contexts and exact numerical source observations. A world fixes the slot orientations ξ, contextual signs s and cue vector τ. The error below averages the four held-out amplitude pairs, as well as contexts, queries and cues. Theorem C.26 (Information supplied by contextual source observations). Two worlds are reward-equivalent on every training input if and only if τ′=ετ,sℓ′ξℓ′=εsℓξℓ(ℓ=1,…,nctx),ε∈±1.τ = τ, s _ ξ _ = s_ _ ( =1,…,n_ ctx), ∈\± 1\. (C.33) The nctxn_ ctx orientations may vary independently; the global sign leaves all predictions unchanged. Flipping an orientation reverses exactly the equal-sign test-amplitude labels in that context. For the exact numerical source, let NobsN_ obs count contextual source records, and let ℛNobsR_N_ obs be the infimum, over all world-independent learners and finite fixed iid feedback-query budgets, of worst-world expected test error after NobsN_ obs iid source records. Then ℛNobs=14(1−1/nctx)Nobs.R_N_ obs= 14(1-1/n_ ctx)^N_ obs. (C.34) For nctx=1n_ ctx=1, the factor is one at Nobs=0N_ obs=0 and zero at Nobs≥1N_ obs≥ 1. For approximation tolerance 0<ϵ<10<ε<1 and K=nctx2kK=n_ ctx2^k context–cue types, ⌈Klog(K/ϵ)⌉ K (K/ε) iid reward queries attain error at most ℛNobs+ϵ/2R_N_ obs+ε/2 for every fixed world. With source observations alone, the minimax error is 1/21/2 for every NobsN_ obs. The proof is given in Section C.5. C.5 The information supplied by source observations We prove Theorem C.26 for the contextual task in Section 5.1. The proof also applies to any fixed 0<ρmem<10< _ mem<1 and odd k≥1k≥ 1. The count NobsN_ obs in this appendix is the number of observed source records. Definition C.27 (Exact contextual source observation). A source record reveals its context ℓ , the two columns κj=Rej _j=Re_j of an independent Haar frame in O(2)O(2), and x=UℓκJx=U_ _J for independent uniform J∈1,2J∈\1,2\. In the full input these keys and query are embedded in the observed context block. The numerical fields G1,G2G_1,G_2 are independent N(0,Ik+2)N(0,I_k+2) vectors, independent of the world, frame, context and index. The predicted bit is sign(GJ,1GJ,2)sign(G_J,1G_J,2). In particular, the joint source law in one context contains no information about another context’s orientation or about s,τs,τ. Lemma C.28 (First Fourier coefficients of the cue rule). Let k≥1k≥ 1 be odd, τ∈±1kτ∈\± 1\^k, and Z uniform on ±1k\± 1\^k. Set aℓ=sℓξℓa_ =s_ _ and Mτ(z)=sign(τ⊤z)M_τ(z)=sign(τ z). For a uniform sign vector Z, [Mτ(Z)Zi]=μkτi,μk=2−(k−1)(k−1(k−1)/2)>0.E[M_τ(Z)Z_i]= _k _i, _k=2^-(k-1) k-1(k-1)/2>0. (C.35) Proof. To derive this identity, absorb τj _j into each ZjZ_j and condition on the other k−1k-1 signs. The conditional average of sign(Zi+∑j≠iZj)Zisign(Z_i+ _j≠ iZ_j)Z_i is one when the even sum is zero and zero otherwise. ∎ Lemma C.29 (Contextual reward equivalence and test separation). Under the contextual training and test distributions of Section 5.1, two worlds have identical correctness rewards for every training input and action if and only if (C.33) holds. Within this equivalence class, reversing a context orientation reverses precisely that context’s equal-sign test-amplitude labels. The common sign transformation preserves every prediction. Proof. Equality of rewards for every action implies equality of the training labels. Taking their first Fourier coefficients at q=1q=1 using Lemma C.28 gives aℓτi=aℓ′τi′a_ _i=a _ τ _i for every ℓ,i ,i. Since all entries are signs, fixing one coordinate gives a common ε with τ′=ετ = τ and a′=εa = a. These relations also suffice, proving (5.5). For equal-sign test amplitudes with common sign d, the label is sℓdMτ(z)s_ dM_τ(z). Changing ξℓ _ while preserving the training equivalence changes sℓs_ by −ε- , reversing this label. For opposite-sign amplitudes with first sign d, the label is aℓvqdMτ(z)a_ v_qdM_τ(z) and is invariant. The global transformation (s,τ)↦(εs,ετ)(s,τ) ( s, τ) preserves both training and test predictions. Any changed orientation has equal-sign test witnesses, so this exhausts the prediction-inert transformations after test labels are included. ∎ Definition C.30 (Finite-query contextual minimax risk). Precisely, define ℛNobs=infQ∈ℕ0inf:at most Q iid feedback queriessupξ,s,τ[err()].R_N_ obs= _Q _0\ _A:\,at most Q iid feedback queries\ _ξ,s,τE[err(A)]. The algorithm and initialization are chosen independently of the world; observations may determine all subsequent computation. Source count NobsN_ obs and the feedback cap are fixed. Risk averages test inputs, training observations and algorithmic randomness. Lemma C.31 (Error contributed by an unobserved orientation). For Nobs∈ℕ0N_ obs _0 exact source records and every finite feedback-query budget, the worst-world expected test error is at least 14(1−1/nctx)Nobs 14(1-1/n_ ctx)^N_ obs. This bound continues to hold for adaptive actions and chosen training prompts. At nctx=1,Nobs=0n_ ctx=1,N_ obs=0, the missing-context probability is interpreted as one. Proof. For the lower bound, fix a,τa,τ, choose independent uniform orientations ξℓ _ , and set sℓ=aℓξℓs_ =a_ _ . All training labels are independent of these orientations. If a context has no source record, its orientation remains uniform conditional on the entire transcript: every observed source record belongs to another context, and every training reward has the same value under its flip. This also holds for adaptive actions or chosen training prompts, since each conditional reply distribution is unchanged. Equal-sign tests have probability 1/21/2 and a fair hidden label, giving conditional error at least 1/41/4 on a missing context. Its missing probability is (1−1/nctx)Nobs(1-1/n_ ctx)^N_ obs. Averaging over the uniform test context, then comparing the worst world to this prior, proves ℛNobs≥14(1−1/nctx)Nobs.R_N_ obs≥ 14(1-1/n_ ctx)^N_ obs. ∎ Lemma C.32 (Orientation recovery from exact observations). Under Definition C.27, one record identifies its context orientation almost surely. For any finite number of records, all visited orientations are therefore recovered simultaneously almost surely. Proof. A visited orientation is identifiable almost surely from exact numerical observations. Its query belongs to κ1,κ2\ _1, _2\ for the identity and to Pκ1,Pκ2\P _1,P _2\ for the swap. These sets intersect only at finitely many angles in each component of O(2)O(2): a column must be fixed by P or mapped to the other column. These angles have Haar probability zero. Finite NobsN_ obs therefore permits exact recovery of every visited orientation, even though J is hidden. ∎ Lemma C.33 (Finite feedback-table attainment). Let K=nctx2kK=n_ ctx2^k, 0<ϵ<10<ε<1, and FNobs=14(1−1/nctx)NobsF_N_ obs= 14(1-1/n_ ctx)^N_ obs. Under Definition C.27, a learner with ⌈Klog(K/ϵ)⌉ K (K/ε) fresh iid binary correctness queries has expected test error at most FNobs+ϵ/2F_N_ obs+ε/2 for every fixed world. It uses one binary action and one verification per query, zero optimizer updates, and the finite storage specified below. Proof. For the upper bound, reduce a feedback prompt (ℓ,q,z)( ,q,z) to (ℓ,vqz)( ,v_qz). These reduced cells are uniform over K=nctx2kK=n_ ctx2^k possibilities, and the label is aℓMτ(vqz)a_ M_τ(v_qz). Choosing action +1+1 turns its binary correctness reply into the exact label. After Q iid queries, Pr[some cell missing]≤Ke−Q/K. [some cell missing]≤ Ke^-Q/K. For 0<ϵ<10<ε<1, take Q=⌈Klog(K/ϵ)⌉Q= K (K/ε) . Recover visited orientations by Lemma C.32. On full coverage, the learner knows gℓ(z)=aℓMτ(z)g_ (z)=a_ M_τ(z). It predicts vqdgℓ(z)v_qdg_ (z) for an opposite-sign test pair with first sign d. For an equal-sign pair it predicts ξℓdgℓ(z) _ dg_ (z) when the source context was observed, and an independent fair sign otherwise. Its conditional expected error is exactly FNobs=14(1−1/nctx)NobsF_N_ obs= 14(1-1/n_ ctx)^N_ obs. Full feedback coverage is independent of source coverage. If feedback coverage fails, fair prediction gives error 1/21/2, so the unconditional error is FNobs+(1/2−FNobs)Pr[some cell missing]≤FNobs+ϵ/2.F_N_ obs+(1/2-F_N_ obs) [some cell missing]≤ F_N_ obs+ε/2. This bound holds for every fixed world. Taking ϵ↓0ε 0 proves the risk infimum; no fixed finite budget is asserted to attain it exactly. Each query generates one binary action and makes one verification. There are no optimizer updates, test queries or selection queries. The construction stores K labels, K occupancy bits, O(nctx)O(n_ ctx) orientation and context flags, and O(log(Q+1))O( (Q+1)) counting bits, in addition to its numerical input and working memory. This finite table establishes information attainability; its exponential dependence on k is separate from the neural learning budgets. ∎ Lemma C.34 (Source-only error). In the contextual observation model, for every source count NobsN_ obs, the source-only minimax expected test error is 1/21/2. Proof. Finally, fix ξ,τξ,τ and randomize the independent context signs sℓs_ . Source observations have the same distribution for all signs, while every test label changes with its context’s sign. The prior-average source-only error is 1/21/2, and fair prediction attains that value. This proves the last assertion of Theorem 5.1. ∎ Proof of Theorem C.26. Lemma C.29 proves the reward-equivalence and test-separation assertions, using the cue identity of Lemma C.28. For the risk in Definition C.30, Lemma C.31 gives the lower bound for every finite query cap. Lemma C.33 gives a world-uniform upper bound FNobs+ϵ/2F_N_ obs+ε/2 for each 0<ϵ<10<ε<1 with a finite fixed cap. Taking the infimum over those caps and then ϵ↓0ε 0 gives equality. A single context is missing with probability one when no source record is observed and probability zero after any positive source count, giving the stated convention at nctx=1n_ ctx=1. Lemma C.34 proves the final assertion. ∎ Appendix D Learning routes and resource comparisons D.1 What additional source data can replace The sequential family admits two resource endpoints. With little source, feedback learns the observable transition class and source resolves its remaining ambiguity. With abundant source, the mechanism can be learned first, leaving binding search. The formal guarantees and their exact budget substitutions are given below. At failure probability δtab _ tab and source correlation a>0a>0, a classical table learner uses O(q2a−2log(q/δtab)+H)O(q^2a^-2 (q/ _ tab)+H) raw source tokens and at most q−1q-1 fresh-prompt verifications, storing q2q^2 successor indices. The additive H pays for the final whole source record. Table 32: What source learning leaves for reward queries. The first two routes compare information resources at fixed confidence and the stated noise regimes; the last gives sufficient observation and query budgets for a finite Adam learning path. Here q is the number of states and controls and H is the execution horizon. Route and remaining reward problem Observed source tokens Training verifier queries Information resources: identify a mechanism and its task binding Few source observations; reward queries recover the mechanism class O(logq)O( q) Θ(q2) (q^2) Logarithmic H; a=1−c/Ha=1-c/H with fixed c; Theorem C.2. Learn a transition table first; rewards identify only the initial operation O(q2logq+H)O(q^2 q+H) At most q−1q-1 Neural training: acquire execution, then adapt the same parameters Source Adam and two sampled reward updates At most NBsTsNB_sT_s 2BR2B_R For the Adam route, N is tokens per source record, BsB_s records per source batch, TsT_s the maximum source batches inspected, and BRB_R completions per reward batch. There are at most Ts−1T_s-1 source updates. Each of the two reward updates uses BRB_R trajectories, totaling 2BR2B_R verifier queries. Its horizon and rates are those of Theorem 4.2. Each verifier query additionally processes H prompt tokens and H+2H+2 generated decisions; verifier computation is separate. The query optimum in Theorem C.2 applies to its fixed-confidence, logarithmic-source and logarithmic-horizon regime. The neural theorem provides another finite training route with its own horizon and block rates. In the contextual task, Theorem 5.1 instead gives an exact source-dependent error floor, approached by a finite label-table learner. Together these results distinguish the cost of identifying the task from the cost of learning a neural computation. Corollary D.1 (The resource routes in the comparison table). Under the sequential observation model, the low-source row of Table 32 holds for the fixed-confidence regime of Theorem C.2. For H≥5H≥ 5, under the independent-triple table learner of Proposition F.37 with a>0a>0 and 0<δtab<10< _ tab<1, the table-first row uses at most 6M0+2H+26M_0+2H+2 raw source tokens, where M0=⌈8q2a−2log(2q3/δtab)⌉M_0= 8q^2a^-2 (2q^3/ _ tab) , stores q2q^2 successor indices, and uses at most q−1q-1 fresh-prompt verifications. Its exact-execution event has probability at least 1−δtab1- _ tab. The Adam row has the budgets and success guarantee of Theorem 4.2. Proof. The low-source row takes δ=1/16δ=1/16, m=q/2≥512m=q/2≥ 512, fixed c∈(0,1/64]c∈(0,1/64], a=1−c/Ha=1-c/H and H−1=⌈216log(2m)⌉H-1= 2^16 (2m) ; its upper bound is fresh-iid and its matching lower bound even permits chosen prompts. It solves binary equations for the observable class and then estimates two source sign statistics. The table-first route uses the retained independent triples in Proposition F.37: M0=⌈8q2a−2log(2q3/δtab)⌉M_0= 8q^2a^-2 (2q^3/ _ tab) and at most 6M0+2H+26M_0+2H+2 raw tokens. It stores a transition table and enumerates bindings. Its guarantee is exact execution on the table-recovery event, whose probability is at least 1−δtab1- _ tab. The Adam row instead gives the stated high-probability sampled-success bound under its prescribed horizon, model widths and real-arithmetic updates. ∎ D.2 What remains for feedback and its alternatives Proposition D.2 (Task parameters absent from source observations). In the sequential family, a decoder with binding-independent initialization using only task-independent source data and source replay has endpoint success at most 1/q1/q averaged over an independent uniform binding. In the contextual family its source-only minimax error is 1/21/2. Proof. Source-only prediction and source replay contain no task-binding information in the sequential family. Under a uniform independent binding, any such decoder averages endpoint success at most 1/q1/q. In the contextual family, the corresponding source-only minimax error is 1/21/2. More source prediction cannot identify those task parameters. For the sequential claim, condition on the mechanism, source observations, decoder randomness and a test prompt. The binding remains uniform on q operations. Applied to the initial state, these q operations give q distinct states: the public component distinguishes different u, and the private bit distinguishes the two values of b. Every subsequent true transition is a permutation, so the q possible endpoints are distinct. A decoder output can match at most one endpoint, giving success at most 1/q1/q; malformed outputs cannot increase it. Averaging the conditioned variables preserves this bound. The contextual assertion is Lemma C.34. ∎ Task labels or expert supervised fine-tuning (SFT) can supply the missing information. Binary correctness in the contextual model already identifies a binary label once the action is known, making supervised adaptation a legitimate comparison. Definition D.3 (Reward-weighted sampled likelihood). For batch item b, let RbR_b be its terminal reward, Yb,jY_b,j its jjth sampled token, and πw(Yb,j∣prefixb,j) _w(Y_b,j _b,j) the next-token probability given the preceding tokens. On the exact baseline-zero REINFORCE batch, the reward-weighted likelihood loss −1BR∑b=1BRRb∑j=1nlogπw(Yb,j∣prefixb,j)- 1B_R _b=1^B_RR_b _j=1^n _w(Y_b,j _b,j) defines the reward-weighted likelihood objective, with sampled tokens and rewards held fixed. Proposition D.4 (Equality of single-batch updates). The objective in Definition D.3 gives the same update as baseline-zero REINFORCE on that exact batch when optimizer settings and state agree. Proof. Differentiation with the sampled tokens and rewards fixed gives the negative batch average of Rb∑j=1n∇wlogπw(Yb,j∣prefixb,j)R_b _j=1^n _w _w(Y_b,j _b,j), which is precisely the negative baseline-zero REINFORCE score estimate. Thus both routes supply the same gradient to the same optimizer state and settings, yielding the same parameter and optimizer-state update. ∎ Additional off-policy passes or normalization by accepted samples define different updates. Verification-based selection at deployment introduces test-verifier access and its generation cost. Corollary D.5 (Frozen source-informed execution by binding search). On the source event in Proposition F.36, a frozen greedy controller identifies the sequential binding in at most q−1q-1 fresh-prompt verifications and then executes every test prompt exactly. In addition to its source budget it uses at most n(q−1)n(q-1) generated symbols and (H+1)(q−1)(H+1)(q-1) forward evaluations, with zero further gradient updates. Deployment uses H+1H+1 forwards and n symbols without verification. Proof. There is also a concrete source-informed controller in the sequential task. On the source event, local state and EOS argmaxes are correct. Freeze this model, choose a candidate first control, and greedily generate the rest. Enumerating at most q−1q-1 candidates on fresh training prompts identifies the binding and gives exact deployment execution. In addition to the source budget, this costs at most q−1q-1 verifications, n(q−1)n(q-1) generated symbols and (H+1)(q−1)(H+1)(q-1) forward evaluations, with zero additional gradient updates. Deployment uses H+1H+1 forwards and n symbols without verification; retained context and workspace are charged. Proposition F.36 proves the margins and identification property. ∎ The controller explicitly searches bindings; the Adam construction adapts the source model’s own parameters using sampled trajectories. Both use the same acquired transition information, with different query and computation costs. Appendix E Shared Gaussian network foundations The following initialization and derivative statements are used by both sequential optimizers. Their distinct objectives, rates, stopping rules and probability allocations are verified in the separate applications. E.1 Finite Gaussian initialization Let N be the full padded context length, V the vocabulary size, R the head count, d the residual width, kheadk_head the head width, and MffM_f the feed-forward width. The local head width kheadk_head equals dh=d/Rd_h=d/R in the source budget. The public-state cardinality remains m=q/2m=q/2; write kμ=∥μ∥2k_μ= μ ^2 for the kernel-mean quantity in the feedback reference, to distinguish it from the head width. Definition E.1 (Prescribed Gaussian initialization and lag weights). For the complete block in (E.22), initialize all the following entries independently: Eij E_ij ∼(0,d−1), (0,d^-1), (Qh)ij,(Kh)ij,(Vh)ij (Q_h)_ij,(K_h)_ij,(V_h)_ij ∼(0,khead−1), (0,k_head^-1), (Oh)ij (O_h)_ij ∼(0,d−1), (0,d^-1), bh(ℓ) b_h( ) ∼(0,1), (0,1), Bij,Cij,bj,cj B_ij,C_ij,b_j,c_j ∼(0,1), (0,1), Uij U_ij ∼(0,(Mffd)−1), (0,(M_fd)^-1), (bout)i (b_out)_i ∼(0,d−1), (0,d^-1), (Wout)yi (W_out)_yi ∼(0,d−1). (0,d^-1). (E.1) The attention-head sum has scale R−1/2R^-1/2. Its fixed temperature is chosen from these dimensions and the confidence level, without using the unknown mechanism. Every displayed parameter trains from these prescribed width-dependent Gaussian scales. Let ℓh _h be the lag with maximum relative bias in head h. The winners depend only on the independent relative biases. Conditional on coverage nℓ:=#h:ℓh=ℓ≥R/(2N)n_ :=\#\h: _h= \≥ R/(2N), put ωℓ=nℓR,wℓ=ωℓ+ℓ=0,p=NV,Dw=diagℓ(wℓIV). _ = n_ R, w_ = _ +1_\ =0\, p=NV, D_w=diag_ (w_ I_V). (E.2) Lemma E.2 (Hard-lag encoder frame). Under Definition E.1, conditional on its coverage event, the hard-lag reference has zhard(x)=∑ℓ=0N−1Aℓexℓ,Aℓ=ℓ=0E+R−1/2∑h:ℓh=ℓOhVhE.z_hard(x)= _ =0^N-1A_ e_x_ , A_ =1_\ =0\E+R^-1/2 _h: _h= O_hV_hE. (E.3) Here exℓe_x_ is a categorical basis vector, not a supplied semantic state representation. Set =[A0⋯AN−1]A=[A_0\ ·s\ A_N-1] and T=Dw−1/2T=AD_w^-1/2. For 0<ξ≤10<ξ≤ 1, the sufficient widths d d ≥2048ξ−2[plog9+log8δenc], ≥ 2048ξ^-2 [p 9+ 8 _enc ], khead k_head ≥max1,⌈4096NRξ2[Vlog9+log8Nδenc]⌉ ≥ \1, 4096NRξ^2 [V 9+ 8N _enc ] \ (E.4) give ∥T⊤T−Ip∥op≤ξ T T-I_p _op≤ξ with conditional probability at least 1−δenc1- _enc, uniformly over winner allocations satisfying coverage. Proof. Write Dw−1/2=Cres+OZAD_w^-1/2=C_res+OZ, where O concatenates the OhO_h and is independent of E and all VhV_h. Explicitly, Cres=[E/w0,0,…,0]C_ res=[E/ w_0,0,…,0] and the head/lag block of Z is Zh,ℓ=ℓh=ℓVhE/RwℓZ_h, =1_\ _h= \V_hE/ Rw_ . With ε=ξ/8 =ξ/8, a Gaussian 1/41/4-net bound gives ∥E⊤E−IV∥op≤ε E E-I_V _op≤ . Conditional on E, the stacked value maps in lag group ℓ have nℓkheadn_ k_head independent Gaussian rows. Set HE=E⊤EH_E=E E and stack the lag-group maps VhEV_hE as YℓY_ . On the encoder event HEH_E is positive definite. Conditional on E, Gℓ=kheadYℓHE−1/2G_ = k_ headY_ H_E^-1/2 has nℓkheadn_ k_ head independent standard Gaussian rows. Their normalized Gram matrices are Wishart; the net bound and a union over all N groups give ‖Gℓ⊤Gℓnℓkhead−I‖≤ε,‖Yℓ⊤Yℓnℓ−HE‖≤‖HE‖ε≤(1+ε)ε. \| G_ G_ n_ k_ head-I \|≤ , \| Y_ Y_ n_ -H_E \|≤\|H_E\| ≤(1+ ) . The second inequality follows by multiplying the first error on both sides by HE1/2H_E^1/2. Thus ∥Cres⊤Cres+Z⊤Z−I∥op C_res C_res+Z Z-I _op ≤ε+(1+ε)ε, ≤ +(1+ ) , ∥Cres∥op2 C_res _op^2 ≤1+ε, ≤ 1+ , ∥Z∥op2 Z _op^2 ≤(1+ε)2. ≤(1+ )^2. (E.5) Conditional on E and the value maps, Gaussian subspace isometry for O on range(Z)range(Z) bounds its Gram error by ε∥Z∥op2 Z _op^2. The residual cross term is retained. With orthonormal bases UC,UZU_C,U_Z of the two fixed column spaces, UC⊤OUZU_C OU_Z has independent Gaussian entries of variance 1/d1/d. Two 1/41/4-nets under (E.4) bound its norm by ε , giving ∥Cres⊤OZ∥op≤ε(1+ε)3/2 C_res OZ _op≤ (1+ )^3/2. The four failure allocations are δenc/4 _enc/4 each. The total error is at most ε+(1+ε)ε+ε(1+ε)2+2ε(1+ε)3/2≤6ε<ξ. +(1+ ) + (1+ )^2+2 (1+ )^3/2≤ 6 <ξ. (E.6) The elementary Gaussian net estimate used here states that an M×sM× s standard Gaussian matrix G satisfies ∥G⊤G/M−I∥op≤ε G G/M-I _op≤ , except with probability η, whenever M≥32ε−2[slog9+log2η].M≥ 32 ^-2 [s 9+ 2η ]. Fixed-direction chi-square tails followed by a 1/41/4-net prove it. Gaussian scalar tails and two nets give the residual cross bound. Content-score events are combined afterwards by a union bound; conditioning on them first would invalidate the Gaussian independence. ∎ Definition E.3 (Augmented tensor features). Define r0(x) r_0(x) =Dw1/2concatℓ(exℓ), =D_w^1/2concat_ (e_x_ ), ∥r0(x)∥2 r_0(x) ^2 =∑ℓwℓ=2, = _ w_ =2, kaug k_aug =p+1, =p+1, ssym s_sym =kaug(kaug+1)2, = k_aug(k_aug+1)2, Dfeat D_feat =p+ssym+1, =p+s_sym+1, Φ(x) (x) =[r0(x)symvec([r0(x)1][r0(x)1]⊤)1]. = bmatrixr_0(x)\\ symvec\! ( bmatrixr_0(x)\\ 1 bmatrix bmatrixr_0(x)\\ 1 bmatrix^\! )\\ 1 bmatrix. (E.7) Symmetric vectorization is orthonormal for the Frobenius inner product, so off-diagonal coordinates carry a factor 2 2. Lemma E.4 (Exact coefficient representation). Under Definitions E.1 and E.3, there exists a realized coefficient matrix CallC_all for which zhard=Tr0z_hard=Tr_0, and ∥Φ(x)∥2=2+9+1=12,hhard(x)=CallΦ(x). (x) ^2=2+9+1=12, h_hard(x)=C_all (x). (E.8) Proof. The latter identity is exact, obtained by expanding the actual bilinear feed-forward block and its biases; it assumes no output-value matching. Explicitly, substitute zhard=Tr0z_ hard=Tr_0 in (E.22). Each factor BjTr0+bjB_jTr_0+b_j or CjTr0+cjC_jTr_0+c_j is a linear functional of (r0,1)(r_0,1). Their product is the Frobenius inner product of (r0,1)(r0,1)⊤(r_0,1)(r_0,1) with the symmetric outer product of the two coefficient vectors. Orthonormal symmetric vectorization gives its coefficient row; multiplication by U, addition of Tr0Tr_0, and addition of boutb_ out assemble CallC_ all. Finally ‖r0‖2=2\|r_0\|^2=2, the squared Frobenius norm of (r0,1)(r0,1)⊤(r_0,1)(r_0,1) is 32=93^2=9, and the last coordinate contributes one, proving the norm identity. ∎ Lemma E.5 (Realized coefficient-operator concentration). Under Definitions E.1 and E.3, suppose 0<γ≤10<γ≤ 1 and ∥T⊤T−I∥op≤γ/16 T T-I _op≤γ/16. The additional conditions d d ≥4096γ−2[Dfeatlog9+log8δffn], ≥ 4096γ^-2 [D_feat 9+ 8 _ffn ], (E.9) J J =ssymlog9+log8δffn, =s_sym 9+ 8 _ffn, A0 A_0 =1+γ−2J, =1+γ^-2J, L0 L_0 =logeA0kaugδffnγ, = eA_0k_aug _ffnγ, Mff M_f =⌈250A02(kaug+L0)8⌉ = 2^50A_0^2(k_aug+L_0)^8 (E.10) give ∥Call⊤Call−IDfeat∥op≤γ C_all C_all-I_D_feat _op≤γ with conditional probability at least 1−δffn1- _ffn. In particular d≥Dfeatd≥ D_feat. Proof. The residual Gram error is a principal block of the full Gram error, and must therefore meet the desired total accuracy. Let Haug=diag(T⊤T,1)H_aug=diag(T T,1). A neuron’s two augmented coefficient vectors are independent (0,Haug)N(0,H_aug). Whitening gives independent a,b∼(0,Ikaug)a,b (0,I_k_aug) and v=symvec(ab⊤+ba⊤2).v=symvec\! ( ab +ba 2 ). For each symmetric S, (a⊤Sb)2=∥S∥F2,vv⊤=Isym.E(a Sb)^2= S _F^2, =I_sym. (E.11) Furthermore, ℙ∥v∥2>(2kaug+3t)2 \ v ^2>(2k_aug+3t)^2\ ≤2exp(−t), ≤ 2 (-t), ∥v∥4 v ^4 ≤[kaug(kaug+2)]2. ≤[k_aug(k_aug+2)]^2. (E.12) Truncation at squared norm B0=(2kaug+3t)2B_0=(2k_aug+3t)^2 has covariance bias at most 5kaug2exp(−t/2)5k_aug^2 (-t/2). Set ε=γ/16 =γ/16. It suffices that t t ≥max1,log8Mffδffn,2log10kaug2ε, ≥ \1, 8M_f _ffn,2 10k_aug^2 \, Mff M_f ≥8B02ε−2[ssymlog9+log8δffn]. ≥ 8B_0^2 ^-2 [s_sym 9+ 8 _ffn ]. (E.13) Scalar Hoeffding on a 1/41/4-net, together with the truncation bias and the event of any realized truncation, bounds the actual whitened sample-covariance error by ε , with failure probability δffn/2 _ffn/2. The explicit choice (E.10) satisfies (E.13) with t=128(kaug+L0)t=128(k_aug+L_0); no implicit logMff M_f condition remains. Transforming back through Haug1/2H_aug^1/2, the actual product-feature matrix F satisfies ∥F⊤F/Mff−Isym∥op≤(1+γ/16)2ε+2γ/16+(γ/16)2≤γ/4. F F/M_f-I_sym _op≤(1+γ/16)^2 +2γ/16+(γ/16)^2≤γ/4. (E.14) Conditional on T,FT,F, the realized Z=[UF,bout]Z=[UF,b_out] has distribution XQ1/2/dXQ^1/2/ d, where Q=diag(F⊤F/Mff,1)Q=diag(F F/M_f,1) and X is standard Gaussian. This representation is independent of the now-fixed T. Gaussian nets and (E.9) imply, simultaneously, ∥X⊤X/d−I∥op≤γ/8,∥T⊤X/d∥op≤γ/4 X X/d-I _op≤γ/8, T X/ d _op≤γ/4 (E.15) except with probability δffn/2 _ffn/2. Hence ∥Z⊤Z−I∥op≤13γ/32,∥T⊤Z∥op≤5γ/8. Z Z-I _op≤ 13γ/32, T Z _op≤ 5\,γ/8. Since Call=[T,Z]C_all=[T,Z], the total block error is at most maxγ/16,13γ/32+5γ/8<γ. \γ/16,13γ/32\+ 5\,γ/8<γ. This proves concentration of the realized coefficient operator. ∎ Lemma E.6 (Tensor kernel and source coefficients). For the features in Definition E.3, the ideal kernel is ⟨Φ(x),Φ(x′)⟩=kbase(x,x′)2+3kbase(x,x′)+2,kbase(x,x′)=∑ℓwℓxℓ=x′ℓ. (x), (x ) =k_base(x,x )^2+3k_base(x,x )+2, k_base(x,x )= _ w_ 1_\x_ =x _ \. (E.16) Thus Kmax=12K_ =12, and on coverage 3wℓ+wℓ2≥32N,k0H=2w0wH≥1N.3w_ +w_ ^2≥ 32N, k_0H=2w_0w_H≥ 1N. (E.17) These singleton weights and pair coefficients are those used in the source reference. Proof. Write b=⟨r0(x),r0(x′)⟩=kbase(x,x′)b= r_0(x),r_0(x ) =k_ base(x,x ). The tensor-feature inner product is (b+1)2(b+1)^2, because symmetric vectorization preserves the Frobenius inner product. Adding the first and last feature blocks gives b+(b+1)2+1=b2+3b+2b+(b+1)^2+1=b^2+3b+2. Since 0≤b≤20≤ b≤ 2, its maximum is 1212. Expanding b2b^2 gives the singleton coefficient wℓ2w_ ^2 and the pair coefficient 2w0wH2w_0w_H. On coverage wℓ≥1/(2N)w_ ≥ 1/(2N) and w0≥1w_0≥ 1, so the two displayed lower bounds follow. ∎ Finite soft attention and nonzero initial readout. Lemma E.7 (Coverage and separated bias winners). For N≥2N≥ 2 and 0<ℓ<N−10< <N-1, the independent bias event has coverage and every pairwise gap at least g, with probability at least 1−δbias1- _bias, when R≥8Nlog2Nδbias,g=δbiasRN2.R≥ 8N 2N _bias, g= _biasRN^2. (E.18) Proof. For coverage, uniformly distributed winning lags give failure at most Nexp(−R/(8N))≤δbias/2N (-R/(8N))≤ _ bias/2. A pairwise difference of independent unit Gaussians has density at most 1/(2π)1/(2 π); the union over at most RN2/2RN^2/2 pairs gives gap failure at most RN2g/(2π)<δbias/2RN^2g/(2 π)< _ bias/2. ∎ Lemma E.8 (Finite-temperature approximation of hard lag selection). On the bias event of Lemma E.7 and a block-norm event with bound MinitM_ init, choose 0<ℓ<N−10< <N-1 and a fixed positive temperature satisfying tatt≤g/(4Minit4)t_ att≤ g/(4M_ init^4) and (E.19). Then each head has off-winner attention mass at most ℓ , and the soft and hard initial features obey (E.20) uniformly over all input strings. Proof. A separate Gaussian norm event supplies a deterministic MinitM_init bounding every initialized block. Content scores then have magnitude at most Minit4M_init^4. Taking tatt≤g/(4Minit4)t_att≤ g/(4M_init^4) keeps the winner gap, in unscaled score units, at least g/2g/2. If also tatt≤g2log((N−1)/ℓ),t_att≤ g2 ((N-1)/ ), (E.19) each head’s off-winner mass is at most ℓ . With Z0=Minit+RMinit3Z_0=M_init+ R\,M_init^3, supx∥hsoft,0(x)−hhard,0(x)∥≤2RMinit3ℓ[1+2Minit3(Z0+1)]. _x h_soft,0(x)-h_hard,0(x) ≤ 2 R\,M_init^3 [1+2M_init^3(Z_0+1)]. (E.20) To obtain the feature estimate, the difference of an attention-weighted average from its winning value has norm at most 2ℓMinit22 M_ init^2 before multiplication by OhO_h. Summing the scaled heads gives a raw residual difference at most 2RMinit3ℓ2 RM_ init^3 . Both residual vectors have norm at most Z0Z_0. Along their connecting segment, the derivative of the bilinear feed-forward map with respect to its residual input has norm at most 1+2Minit3(Z0+1)1+2M_ init^3(Z_0+1). Multiplying these bounds proves (E.20). This holds uniformly over all strings, including repeated padding and wrong generated tokens. The derivatives are bounded in Section E.2; subsequent motion is controlled separately in Sections F.5 and H.14. No infinite-width or zero-temperature limit is interchanged. ∎ Lemma E.9 (A simultaneous initialization norm event). For the Gaussian law in Definition E.1, let P be its number of initialized scalar parameters and let 0<δnorm<10< _ norm<1. With probability at least 1−δnorm1- _ norm, every initialized block Frobenius, spectral or vector norm is at most the deterministic MinitM_ init in (E.21). Proof. For an explicit coarse norm event, let P be the number of initialized scalar parameters, all with variance at most one in (E.1). With failure probability at most δnorm _norm, every block Frobenius, spectral or vector norm is bounded by Minit=max1,2Plog2Pδnorm.M_init= \1, 2P 2P _norm \. (E.21) For each scalar Gaussian of variance at most one, the probability that its absolute value exceeds 2log(2P/δnorm) 2 (2P/ _ norm) is at most δnorm/P _ norm/P. A union bound over the P scalars and the inequality between spectral and Frobenius norms give (E.21). Bound every scalar Gaussian and then each block’s Frobenius norm. This conservative bound is polynomial in the prescribed dimensions. It permits the temperature and all positive rates to be selected before drawing initialization or source data. ∎ E.2 Uniform derivatives of the complete block Definition E.10 (Complete raw attention and bilinear block). Use R heads, N retained positions, relative biases bh(ℓ)b_h( ) and fixed temperature tatt>0t_att>0. Define eℓ e_ =E(xℓ), =E(x_ ), αh(ℓ) _h( ) =softmaxℓ[bh(ℓ)tatt+⟨Qhe0,Kheℓ⟩khead], =softmax_ \! [ b_h( )t_att+ Q_he_0,K_he_ k_head ], z z =e0+R−1/2∑h=1ROh∑ℓ=0N−1αh(ℓ)Vheℓ, =e_0+R^-1/2 _h=1^RO_h _ =0^N-1 _h( )V_he_ , hψ h_ψ =z+U[(Bz+b)⊙(Cz+c)]+bout. =z+U[(Bz+b) (Cz+c)]+b_out. (E.22) Every displayed matrix, embedding and bias is included in ψ with positive rates specified separately in the Adam and SGD applications. Causal attention sees every real token in the padded context; the displayed equations specify the complete raw architecture. Lemma E.11 (Uniform raw feature and derivative bounds). For Definition E.10, if every block matrix spectral norm, bias-vector norm and embedding-column norm is at most M≥1M≥ 1, put Z Z =M+RM3, =M+ R\,M^3, Araw A_raw =Z+M3(Z+1)2+M, =Z+M^3(Z+1)^2+M, Dz D_z =1+R[3M2+M3N(4M3+tatt−1)], =1+ R[3M^2+M^3N(4M^3+t_att^-1)], D D =Dz[1+2M3(Z+1)]+3M2(Z+1)2+1. =D_z[1+2M^3(Z+1)]+3M^2(Z+1)^2+1. (E.23) Then ∥hψ∥≤Araw h_ψ ≤ A_raw and ∥Dψhψ∥op≤D D_ψh_ψ _op≤ D in the concatenated Euclidean/Frobenius metric. Proof. For a global unit parameter direction, each individual matrix or bias direction has norm at most one. The attention-score derivative is at most 4M3+tatt−14M^3+t_att^-1, while ∥α˙∥1≤N∥score˙∥∞. α _1≤ N score _∞. Differentiating the O,V,eO,V,e paths gives DzD_z. The feature bound follows from ‖z‖≤Z\|z\|≤ Z and the block norm bounds in the displayed architecture. Both feed-forward factors have norm at most M(Z+1)M(Z+1), yielding the displayed direct-factor and through-z terms. No block is omitted. ∎ E.3 The nonlinear normalized member Definition E.12 (Normalized block and trainable gain). Retain every raw attention and bilinear block, replacing only the final output feature by hnorm=g⊙ℛ(hraw),ℛ(x)=2x1+∥x∥2/K,K=12.h_norm=g (h_raw), (x)= 2\,x 1+ x ^2/K, K=12. (E.24) The gain g∈ℝdg ^d is trainable and starts at the task-independent vector of ones. Every original random parameter retains its prescribed Gaussian law. Each original backbone block and the gain update at the same positive backbone rate; the readout uses the specified faster rate. The gain initializer is thus deterministic. This nonlinear final RMS-type normalization has a fixed positive denominator offset and trainable coordinate gain. It changes the feature and its derivatives away from the constant-norm ideal feature manifold. Lemma E.13 (Normalized initial-feature approximation). Use Definition E.12. The normalization has global derivative norm at most 2 2 and global output norm at most 2K 2K. On the actual hard-feature event, ∥Call⊤Call−I∥op≤γ C_all C_all-I _op≤γ and ∥Φ∥2=K ^2=K. For γ≤1/4γ≤ 1/4, ∥ℛ(CallΦ)−CallΦ∥≤4γ. (C_all )-C_all ≤ 4γ. (E.25) If the raw soft/hard features differ by at most ζraw _raw, then ∥hnorm,0−CallΦ∥≤2ζraw+4γ. h_norm,0-C_all ≤ 2\, _raw+4γ. (E.26) Proof. The derivative of ℛR has tangential eigenvalue 2(1+∥x∥2/K)−1/2 2(1+\|x\|^2/K)^-1/2 and radial eigenvalue 2(1+∥x∥2/K)−3/2 2(1+\|x\|^2/K)^-3/2, both at most 2 2; its output norm is at most 2K 2K. To see this, write ∥CallΦ∥2/K=1+t C_all ^2/K=1+t, where |t|≤γ|t|≤γ. The factor 2/2+t 2/ 2+t differs from one by at most γ, and ∥CallΦ∥<4 C_all <4. Thus the estimate concerns the realized random operator. The Lipschitz bound and triangle inequality give the soft/hard estimate in the statement, because g0g_0 is the vector of ones. ∎ Lemma E.14 (Normalized feature and derivative envelopes). Use Definition E.12 and the norm event in Lemma E.9. On a radius-one ball around initialization in all backbone parameters, ∥g∥∞≤2 g _∞≤ 2. Therefore ∥hnorm∥≤22K<12=A. h_norm ≤ 2 2K<12=A. (E.27) Let DrawD_raw be the raw derivative bound (E.23), evaluated at Minit+1M_init+1. The concatenated derivative norm is bounded by Dnorm=22Draw+2K.D_norm=2 2\,D_raw+ 2K. (E.28) Proof. A radius-one displacement from g0=g_0=1 gives ‖g‖∞≤2\|g\|_∞≤ 2. Apply the global output and derivative bounds of Lemma E.13 and the raw derivative bound of Lemma E.11. The first term differentiates every original attention, embedding and feed-forward parameter through ℛR. The second differentiates the coordinate gain, using ∥ℛ(hraw)∥∞≤2K (h_raw) _∞≤ 2K. Both the gain and all original blocks are included in the motion bound. ∎ Appendix F Sequential acquisition and sampled Adam adaptation F.1 Finite source learning and sampled Adam improvement Use the two architectures and source-to-feedback update rule in Section 4: all parameters train, source cross-entropy has the coupled positive ridge penalty, and bias-corrected Adam uses (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999). At the source stopping state, retain every parameter, remove the penalty and reset both moments. Feedback uses fresh on-policy trajectories, baseline zero, and retains every trajectory and every output score. The complete choices of horizon, width, Gaussian scales, positive block rates, stopping threshold and batch sizes are those in Section F.5. Write εlogit:=h _ logit:=h for the tolerance in (F.97). The metrics E,M,S,VE,M,S,V are defined in Section 3. Theorem F.1 (Finite source learning and sampled Adam improvement). Fix q=22r+2q=2^2r+2 with r≥1r≥ 1, c=1/64c=1/64 and δ∈(0,1)δ∈(0,1). For either network member in (4.2)–(4.4), with the explicit horizon, dimensions and training choices of Section F.5, the source algorithm uses at most TsT_s batches of BsB_s records and makes at most Ts−1T_s-1 updates; it is followed by exactly two feedback Adam updates, each on BRB_R sampled trajectories. For every fixed world, with probability at least 1−δ1-δ, source stopping succeeds and, simultaneously for every held-out prompt X, minE(wf,X),M(wf,X),S(wf,X) \E(w_f,X),M(w_f,X),S(w_f,X)\ ≥7/8, ≥ 7/8, (F.1) maxE(ws,X),M(ws,X),S(ws,X) \E(w_s,X),M(w_s,X),S(w_s,X)\ ≤1/q+2nεlogit, ≤ 1/q+2n _ logit, (F.2) minJ∈E,M,S[J(wf,X)−J(ws,X)] _J∈\E,M,S\ [J(w_f,X)-J(w_s,X) ] ≥3/4, ≥ 3/4, (F.3) V(wf,X) V(w_f,X) ≤9c/8+1/128, ≤ 9c/8+1/128, (F.4) where wsw_s is the source stopping state and wfw_f the second feedback iterate. With N=2H+2N=2H+2 source tokens per record and n=H+2n=H+2 decisions per trajectory, Npre N_ pre ≤NBsTs, ≤ NB_sT_s, Q Q =2BR, =2B_R, (F.5) Ngenerated N_ generated =2nBR, =2nB_R, Nupdates,source N_ updates,source ≤Ts−1,Nupdates,feedback=2. ≤ T_s-1, N_ updates,feedback=2. (F.6) The complete proof closes Section F.5, following its parameter construction. F.2 Applying the shared bounds to Adam Proposition F.2 (Adam substitution into the shared initialization bounds). Use the dimensions, temperature and probability allocations prescribed in Section F.5, with γ=d−1/8γ=d^-1/8. These satisfy the hypotheses of the shared encoder, coefficient, bias, norm, soft-attention and derivative lemmas. Their four initialization events have total failure at most δ/4δ/4. For the normalized member, the initial feature discrepancy is at most 3εsrc/83 _ src/8. The raw or normalized derivative envelopes apply throughout every radius-one backbone ball controlled by the Adam displacement bounds. Proof. The substitutions into Lemmas E.2 and E.5 are N=2H+2N=2H+2, V=2q+2V=2q+2, R=RheadR=R_ head, khead=d/Rheadk_ head=d/R_ head, Dfeat=kD_ feat=k, and ξenc=γ/16 _ enc=γ/16 from Section F.5. In particular, 2048ξenc−2=219γ−22048 _ enc^-2=2^19γ^-2 and 4096ξenc−2=220γ−24096 _ enc^-2=2^20γ^-2. Thus the encoder and head premises are exactly the first and third terms in DΓγ−2D_ γ^-2; the second term supplies the output Gram premise. Since γ=d−1/8γ=d^-1/8, d≥DΓ4/3d≥ D_ ^4/3 implies all three. Rounding d to a multiple of RheadR_ head preserves them and makes the head width integral. The FFN width is the shared lemma’s width with (J,A0,L0)=(Jff,Aff,Lff)(J,A_0,L_0)=(J_ f,A_ f,L_ f). The bias, encoder, FFN and global-norm events each use δ/16δ/16; the encoder and FFN conditional bounds are combined by the tower property, then with the other events by a union bound, without conditioning the Gaussian laws on content scores. The temperature satisfies both inequalities in Equation E.19 and the preceding gap condition; its choice of ℓ makes the raw feature error at most zrawz_ raw. For the normalized member, γ≤εsrc/16γ≤ _ src/16 and zraw≤εsrc/(82)z_ raw≤ _ src/(8 2) give 2zraw+4γ≤3εsrc/8 2z_ raw+4γ≤ 3 _ src/8. The separate row, readout, geometry, batch and reward events keep their allocations in (F.95); none is replaced by the SGD allocation. Evaluate the displayed derivative bound at M=Minit+1M=M_ init+1 for the whole radius-one parameter ball. The source and feedback displacement bounds in Section F.5 keep every supplied backbone parameter in that ball. The actual initialization has bounded features, and its smaller source tube gives the feature envelope Asrc=12A_ src=12. The amplitude-free coordinate Adam bound pays motion even when coupled regularizer gradients are large. The gain is included in the backbone parameter count and in every source and feedback update. The source feature tolerance and subsequent motion are selected in Section F.5; their errors include the initial 4γ4γ normalization discrepancy. The proof uses the same raw coefficient operator as an analysis reference and makes no rotation-equivariance claim for the trained normalized network. ∎ F.3 Finite source acquisition by coupled-regularized Adam The source stage learns a transition function and output format before any terminal reward is observed. We first identify an invariant population ridge optimum, transfer it to the realized hard-feature head, and then show that the actual sampled Adam algorithm reaches a source-observable stopping condition. The finite initializer and full-model derivative bounds are in Appendix E; Section F.5 chooses the joint horizon, widths, source precision, and failure allocations. Throughout this section, N=2H+2N=2H+2, H≥5H≥ 5, V=2q+2V=2q+2, and K=12K=12. We write Ts,Bs,ηsT_s,B_s, _s for the source inspection limit, complete-record batch size, and effective head step. The physical head step is ηs/d _s/ d. Let F0(Θ)F_0( ) and F+(Θ)F_+( ) denote the unconditioned and tag-plus population source cross-entropies, averaged over all N raw next-token targets. In the unconditioned law, S0S_0 and all controls are independent uniform. Conditional on the controls, the state transitions use aPU+(1−a)ΠaP_U+(1-a) . The canonical feature map has kernel ⟨Φ(x),Φ(x′)⟩=kbase(x,x′)2+3kbase(x,x′)+2,kbase(x,x′)=∑l=0N−1wlxl=xl′. (x), (x ) =k_ base(x,x )^2+3k_ base(x,x )+2, k_ base(x,x )= _l=0^N-1w_l1\x_l=x _l\. (F.7) Lemma F.3 (Canonical feature geometry). For the canonical kernel (F.7) on the encoder event of Appendix E, the squared feature norm is K=12K=12, the singleton weights are at least 3/(2N)3/(2N), and the lag-pair weight k0Hk_0H is at least 1/N1/N. Its categorical and tensor representations are isometric on their spans. Proof. Its squared feature norm is K, its singleton coefficients are at least kmin=3/(2N)k_ =3/(2N), and its lag-pair coefficient satisfies k0H≥1/Nk_0H≥ 1/N. The categorical singleton/pair representation and the tensor representation in Appendix E have the same kernel. Mapping finite linear combinations of one representation to the other preserves their Gram quadratic form, and hence gives an isometry of their spans. Projecting any comparator onto that span cannot increase its norm. ∎ Definition F.4 (Semantic source coordinates). For a state-output row y, define h1,y(x) h_1,y(x) =x0∈,xH∈(y=ϕx0(xH)−q−1), =1\x_0 ,x_H \ (1\y= _x_0(x_H)\-q^-1 ), (F.8) h2,y(x) h_2,y(x) =x0∈,xH∈(y=ϕxH(x0)−q−1), =1\x_0 ,x_H \ (1\y= _x_H(x_0)\-q^-1 ), and set both functions to zero on nonstate rows. Their pair supports are disjoint. The active lag pair is (U1,S0)(U_1,S_0) at the first-state prediction, (Si−1,Ui)(S_i-1,U_i) at subsequent state predictions, and (SH,U1)(S_H,U_1) at EOS. Both functions vanish at the initial control-choice prediction because lag H is PAD. Lemma F.5 (Invariant population source dynamics). Under the unconditioned source law above, Euclidean gradient descent on F0F_0 preserves the mask/class space plus the semantic span in Definition F.4. Its two semantic coordinates satisfy (F.10), with bjb_j defined by (F.9). These identities hold on every legal class/PAD pattern, including generated state histories. Proof. The gradient of F0F_0 preserves the sum of mask/class logits and α1h1+α2h2 _1h_1+ _2h_2. Here αj _j are semantic logit amplitudes, distinct from Adam’s moment parameters. Put bj=exp(αj)−1exp(αj)+q−1.b_j= ( _j)-1 ( _j)+q-1. (F.9) If viv_i is the correct state-class mass at state target i and vEv_E is the incorrect state-class mass at EOS, the semantic coordinates of an auxiliary Euclidean descent step are α1+−α1 _1^+- _1 =ηk0HNq2(a−v1b1), = η k_0HNq^2(a-v_1b_1), (F.10) α2+−α2 _2^+- _2 =ηk0HNq2[∑i=2H(a−vib2)−vEb2]. = η k_0HNq^2 [ _i=2^H(a-v_ib_2)-v_Eb_2 ]. The EOS contribution is essential because h2h_2 is active on the incorrect state class there. To verify closure, center in both interaction arguments. At the first-state query, independence leaves only the pair (U1,S0)(U_1,S_0). At a repeated-state query, averaging the current control eliminates coordinates omitting it; a surviving coordinate pairs that control with the immediately preceding state. At EOS, integrating uniform S0S_0 makes (U2,…,UH,S1,…,SH)(U_2,…,U_H,S_1,…,S_H) independent of U1U_1. Coordinates omitting both U1U_1 and S0S_0 vanish by averaging U1U_1. Among coordinates retaining U1U_1, an earlier state leaves a later control to average, and a control leaves uniform S0S_0; only (U1,SH)(U_1,S_H) survives. Among coordinates retaining S0S_0 but not U1U_1, an earlier state leaves a later control, SHS_H leaves a suffix average followed by U1U_1, and a suffix control leaves another suffix control when H≥3H≥ 3. The surviving semantic coefficient has factor q−2q^-2. Distinct individual record variables are pairwise independent uniform, which also preserves the mask/class space. These categorical identities hold on every prefix with the legal class/PAD pattern, including incorrect generated states. ∎ Lemma F.6 (Finite source comparator). Under the unconditioned source law with 0<a<10<a<1, its target-averaged entropy is H0H_0 in (F.11). For every 0<ε≤10< ≤ 1, the invariant feature class contains a finite comparator with excess cross-entropy at most ε and squared norm at most D∗(ε)D^*( ) in (F.14). Proof. Let hq(a)h_q(a) be the entropy of a point mass mixed with the uniform q-state law at weights a,1−a,1-a. The unconditioned entropy per optimized target is H0=(H+1)logq+Hhq(a)N.H_0= (H+1) q+Hh_q(a)N. (F.11) For ε>0 >0, set α∗=log1+(q−1)a1−a,M=α∗+log(V/ε),Pl=xl=PAD.α^*= 1+(q-1)a1-a, M=α^*+ (V/ ), P_l=1\x_l=PAD\. (F.12) Use the class logits cS c_S =M(P0−PH+P2H), =M(P_0-P_H+P_2H), cU c_U =M(PH−P0), =M(P_H-P_0), (F.13) cE c_E =M(1−P2H), =M(1-P_2H), cPAD c_ PAD =0, =0, and add α∗(h1+h2)α^*(h_1+h_2). They select the correct class at every raw target; the extra α∗α^* in M pays for the active incorrect state partition at EOS. The constant term of cEc_E can be represented by all lag-zero indicators. This comparator has excess CE at most ε and squared norm bounded by D∗(ε)=(5q+V+1)M2kmin+2q(q−1)(α∗)2k0H.D^*( )= (5q+V+1)M^2k_ + 2q(q-1)(α^*)^2k_0H. (F.14) For deterministic algorithmic choices, we use the upper envelope obtained by substituting kmin=3/(2N)k_ =3/(2N) and k0H=1/Nk_0H=1/N in this expression. ∎ Lemma F.7 (Excess loss controls execution). For an invariant reference from Lemma F.5 with excess E=F0(Θ)−H0E=F_0( )-H_0, the decomposition (F.15) and bounds (F.16) hold. In particular, at a=1−c/Ha=1-c/H, E≤c2(q−1)2/(32q2H2N)E≤ c^2(q-1)^2/(32q^2H^2N) implies |bj−a|≤c/(8H)|b_j-a|≤ c/(8H) for j=1,2j=1,2. Proof. Let KLKL denote Kullback–Leibler divergence. For any invariant reference, let Dj=KL(Ta∥Tbj)D_j=KL(T_a\|T_b_j), where TbT_b denotes the relevant true transition permutation mixed with uniform at weights b,1−b,1-b. The exact excess decomposition is D1+(H−1)D2+∑all stages s[−logvs]=N[F0(Θ)−H0].D_1+(H-1)D_2+ _all stages s[- v_s]=N [F_0( )-H_0 ]. (F.15) Consequently, if the excess is E, then |b1−a|≤q−1NE/2,|b2−a|≤q−1NE2(H−1).|b_1-a|≤ qq-1 NE/2, |b_2-a|≤ qq-1 NE2(H-1). (F.16) For a=1−c/Ha=1-c/H, the condition E≤c2(q−1)2/(32q2H2N)E≤ c^2(q-1)^2/(32q^2H^2N) gives |bj−a|≤c/(8H)|b_j-a|≤ c/(8H) for both transitions. The first transition is observed at one position per record, which determines the stronger precision requirement. ∎ Lemma F.8 (Tag contraction and source-gradient transfer). For the Sequential source law, let Ctag=∏i=2H(−1)ℓ(Ui)C_ tag= _i=2^H(-1) (U_i) and ρc=2−r _c=2^-r. A bounded observable retaining rSr_S states and rtagr_ tag suffix controls satisfies (F.17). At each invariant reference of Lemma F.5, the gradient discrepancy is at most μF _F in (F.18), uniformly over the unknown mechanism. Proof. Write Ctag=∏i=2H(−1)ℓ(Ui)C_ tag= _i=2^H(-1) (U_i) and ρc=2−r _c=2^-r. The signed contraction holds on the full state space for every unknown mechanism. Indeed, in the Koopman convention PAf=f∘ϕAP_Af=f _A, decompose f(s,z)=f+(z)+(−1)sf−(z)f(s,z)=f_+(z)+(-1)^sf_-(z). In Mc=q−1∑b,u(−1)ℓ(u)P(b,u)M_c=q^-1 _b,u(-1) (u)P_(b,u), averaging the control bit b annihilates the odd component. The even component reduces to normalized convolution by (−1)t+x⋅y(-1)^t+x· y on the public Z coordinates, independently of θ. Its Walsh eigenvalues are zero at dual t-frequency zero and 2−r(−1)α⋅β2^-r(-1)^α·β at dual t-frequency one. Thus ‖Mc‖op=2−r\|M_c\|_ op=2^-r. For the noisy kernels TA=aPA+(1−a)ΠT_A=aP_A+(1-a) , balanced tags annihilate the Π term, giving signed operator norm aρca _c. For a bounded observable G retaining rSr_S states and rtagr_ tag suffix controls, the signed-coordinate estimate is |0[CtagG]|≤qrS(aρc)H−1−rtag∥G∥∞. |E_0[C_ tagG] |≤ q^r_S(a _c)^H-1-r_ tag G _∞. (F.17) Insert diagonal projectors for retained states into the Markov product. Each unretained tagged control contributes an operator of norm aρca _c, while a retained control has norm at most one. Summing retained state values costs at most qrSq^r_S. The control U1U_1 supplies no signed contraction. At an invariant reference, the probability and label terms of each gradient coordinate retain at most three tagged controls and three states. The sum of squared categorical feature weights is at most V2KV^2K. Summing over V output rows and taking a square root therefore gives ∥∇F+(Θ)−∇F0(Θ)∥F≤μF,μF=2q3(aρc)H−4V3/2K. ∇ F_+( )-∇ F_0( ) _F≤ _F, _F=2q^3(a _c)^H-4V^3/2 K. (F.18) This bound will be used only at the invariant ridge optimum. ∎ Lemma F.9 (Invariant ridge optimum). For λ>0λ>0, the unique minimizer Θλ _λ of J0=F0+λ‖Θ‖F2/2J_0=F_0+λ\| \|_F^2/2 belongs to the invariant space of Lemma F.5 and satisfies (F.19). Under (F.20), its excess is at most e, its norm at most RλR_λ, and every correct output-class mass on a legal-pattern prefix is at least exp(−Ne) (-Ne). Proof. Define J0(Θ)=F0(Θ)+λ∥Θ∥F2/2J_0( )=F_0( )+λ _F^2/2. Its unique minimizer Θλ _λ belongs to the invariant space above: auxiliary zero-start Euclidean GD with step at most 1/(K+λ)1/(K+λ) stays in that space by (F.10), ridge scales its coefficients, and strong convexity gives convergence to the unique optimum. This auxiliary sequence only characterizes the optimum; the source algorithm below is Adam. The comparator yields F0(Θλ)−H0 F_0( _λ)-H_0 ≤ε+λD∗(ε)/2, ≤ +λ D^*( )/2, (F.19) ∥Θλ∥F2 _λ _F^2 ≤D∗(ε)+2ε/λ. ≤ D^*( )+2 /λ. Choose ε=e/2,D¯=maxD∗(e/2),e,λ=e/D¯≤1,Rλ=2D¯. =e/2, D= \D^*(e/2),e\, λ=e/ D≤ 1, R_λ= 2 D. (F.20) Then the excess is at most e and ∥Θλ∥F≤Rλ _λ _F≤ R_λ. Class masses depend only on the legal class/PAD pattern. Equation (F.15) therefore gives vs≥exp(−Ne)v_s≥ (-Ne) on every legal-pattern prefix, including unseen words and incorrect state histories. ∎ Lemma F.10 (Transfer to the realized physical head). Let Θλ _λ be as in Lemma F.9, and assume ‖C⊤C−I‖op≤γ<1\|C C-I\|_ op≤γ<1. The minimizer W∗W^* of (F.21) satisfies (F.22) and the uniform logit bound (F.23). A total logit error at most h relative to ΘλΦ _λ gives correct class mass at least exp(−Ne−2h) (-Ne-2h) and leakage at most Ne+2hNe+2h. Proof. Let C be the realized hard-feature operator, GC=C⊤CG_C=C C, and ∥GC−I∥op≤γ<1 G_C-I _op≤γ<1. The fixed-feature physical head objective JC(W)=F+(WC)+λ∥W∥F2/2J_C(W)=F_+(WC)+λ W _F^2/2 (F.21) is λ-strongly convex in the complete physical head, including its nullspace components. At W~=ΘλC⊤ W= _λC , ∇JC(W~) ∇ J_C( W) =[∇F+(ΘλGC)+λΘλ]C⊤, =[∇ F_+( _λG_C)+λ _λ]C , (F.22) r:=∥∇JC(W~)∥F r:= ∇ J_C( W) _F ≤∥C∥op(μF+KγRλ),∥W∗−W~∥F≤r/λ, ≤ C _op( _F+Kγ R_λ), W^*- W _F≤ r/λ, where W∗W^* minimizes JCJ_C. Thus, uniformly over all input strings, ∥(W∗C−Θλ)Φ(x)∥∞≤K[γRλ+(1+γ)(μF+KγRλ)λ]. (W^*C- _λ) (x) _∞≤ K [γ R_λ+ (1+γ)( _F+Kγ R_λ)λ ]. (F.23) The tag-conditioned source perturbation is paid once through strong convexity. A total uniform logit error at most h relative to ΘλΦ _λ gives class mass at least exp(−Ne−2h) (-Ne-2h) and leakage at most Ne+2hNe+2h. ∎ Lemma F.11 (Descent with a current-sample adaptive denominator). We use an inexact first-moment estimate that allows the preconditioner to depend on all current samples. Let F be globally L-smooth and bounded below by F∗F^*. Write gt=∇F(xt)g_t=∇ F(x_t), zt=gt+ξtz_t=g_t+ _t, ∥ξt∥≤b _t ≤ b, and let m^t(z) m_t(z) be the bias-corrected first moment with parameter β∈[0,1)β∈[0,1) and zero initial moment. Suppose xt+1=xt−ηPtm^t(z)+rt,∥rt∥≤ηbr,λpI⪯Pt⪯ΛpI.x_t+1=x_t-η P_t m_t(z)+r_t, r_t ≤η b_r, _pI P_t _pI. (F.24) Set eerr=Λpb+bre_ err= _pb+b_r and u=1−βu=1-β. If η≤λpu3/28LΛp2,η≤ _pu^3/28L _p^2, (F.25) then every finite prefix satisfies F(xT+1)+ηλp2∑t=1T∥gt∥2≤F(x1)+3ηTeerr22λp.F(x_T+1)+ η _p2 _t=1^T g_t ^2≤ F(x_1)+ 3η Te_ err^22 _p. (F.26) In particular, with Δ=F(x1)−F∗ =F(x_1)-F^*, ∑t∥gt∥2≤2Δ/(ηλp)+3Teerr2/λp2 _t g_t ^2≤ 2 /(η _p)+3Te_ err^2/ _p^2. Proof. For the proof, write ht=m^t(g)h_t= m_t(g) and absorb the corrected noise into an update error of norm at most eerre_ err. Let Sg=∑t∥gt∥2S_g= _t g_t ^2 and Sh=∑t∥ht∥2S_h= _t h_t ^2. Corrected-weight Jensen gives Sh≤Sg/uS_h≤ S_g/u, and the first-moment lag identity gives ∑t∥ht−gt∥2≤LηΛpβu3/2Sg+LηeerrβuT. _t h_t-g_t ^2≤ Lη _pβu^3/2 S_g+ Lη e_ errβu T. (F.27) Smoothness and the squared-step bound imply F(xT+1) F(x_T+1) ≤F(x1)−3ηλp4Sg+ηDmeerrTSg+Lη2Teerr2, ≤ F(x_1)- 3η _p4S_g+η D_me_ err TS_g+Lη^2Te_ err^2, (F.28) Dm D_m =1+LηΛpβ/u≤9/8. =1+Lη _pβ/u≤ 9/8. The lag and squared-step losses cost at most λp(β+u)/8≤λp/4 _p(β+ u)/8≤ _p/4 in the descent coefficient. Young’s inequality absorbs the mixed term using Dm2/λp+Lη≤89/(64λp)<3/(2λp)D_m^2/ _p+Lη≤ 89/(64 _p)<3/(2 _p), proving (F.26). ∎ Corollary F.12 (Finite stopping and a closed parameter tube). Under Lemma F.11, the choices (F.29) give an inspected gradient of norm at most r. If in addition F≥μ‖x‖2/2F≥μ\|x\|^2/2 and F(x1)≤FinitF(x_1)≤ F_ init, the prefix-radius bound below holds. With the stated headroom B0B_0, radius and step cap, (F.30) closes the stopped tube by first exit. Proof. For a target r>0r>0, the conditions eerr≤λpr/4,T≥max1,⌈4Δηλpr2⌉e_ err≤ _pr/4, T≥ \1, 4 η _pr^2 \ (F.29) ensure ∥gt∥≤r g_t ≤ r at some inspected iterate. If F≥μ∥x∥2/2F≥μ x ^2/2 and F(x1)≤FinitF(x_1)≤ F_ init, the squared radius of every prefix is at most 2Finit/μ+3ηTeerr2/(μλp)2F_ init/μ+3η Te_ err^2/(μ _p). To close a stopped tube without a circular radius, choose a headroom B0>0B_0>0, radius squared 2(2Finit+B0)/μ2(2F_ init+B_0)/μ, impose also η≤B0/(λpr2)η≤ B_0/( _pr^2), and use T=max1,⌈4Finit/(ηλpr2)⌉T= \1, 4F_ init/(η _pr^2) \. Then ηTλpr2≤4Finit+B0,F(xt)≤Finit+3Finit/8+3B0/32<2Finit+B0.η T _pr^2≤ 4F_ init+B_0, F(x_t)≤ F_ init+3F_ init/8+3B_0/32<2F_ init+B_0. (F.30) Bounds established inside this predetermined tube therefore close by first exit. ∎ Lemma F.13 (Coordinate confinement under coupled ridge Adam). Assume 0≤β1<10≤ _1<1, 0<β2<10< _2<1, β12<β2 _1^2< _2, λ>0λ>0, and ϵ0>0 _0>0. Suppose each head-gradient coordinate is gt,j=ct,j+λwt,jg_t,j=c_t,j+λ w_t,j with |ct,j|≤Ch/d|c_t,j|≤ C_h/ d. Use physical step ηs/d _s/ d, ηs≤1 _s≤ 1, and physical denominator offset ϵ0/d _0/ d. Let the moments start at zero and put Kmom=[(1−β2)(1−β12/β2)]−1/2,Mc=maxM0,Chλ+ηsKmom1−β1.K_ mom= [(1- _2)(1- _1^2/ _2) ]^-1/2, M_c= \M_0, C_hλ+ _sK_ mom1- _1 \. (F.31) If |w1,j|≤M0/d|w_1,j|≤ M_0/ d, then |wt,j|≤Mc/d|w_t,j|≤ M_c/ d at every finite step. Proof. Indeed, weighted Cauchy–Schwarz for the complete corrected histories gives |m^t,j|≤Kmomv^t,j| m_t,j|≤ K_ mom v_t,j, so each coordinate step is at most δw=ηsKmom/d _w= _sK_ mom/ d. The corrected first-moment weights have mean age ∑sats(t−s)=β11−β1−tβ1t1−β1t≤β11−β1. _sa_ts(t-s)= _11- _1- t _1^t1- _1^t≤ _11- _1. (F.32) Consequently, |m^t,j−λwt,j|≤Ch/d+λδwβ1/(1−β1).| m_t,j-λ w_t,j|≤ C_h/ d+λ _w _1/(1- _1). (F.33) Outside the interval of radius [Ch/λ+ηsKmomβ1/(1−β1)]/d[C_h/λ+ _sK_ mom _1/(1- _1)]/ d, momentum has the coordinate’s sign, so the step points inward. Inside it, one step stays within Mc/dM_c/ d; an overshoot across zero is at most the step bound. This proves confinement by induction. Define Gc=Ch+λmaxM0,Ch/λ+Kmom/(1−β1).G_c=C_h+λ \M_0,C_h/λ+K_ mom/(1- _1) \. (F.34) Then |gt,j|≤Gc/d|g_t,j|≤ G_c/ d, and the actual update is exactly Wt+1=Wt−ηsPtm^t,Pt=diagj1dv^t,j+ϵ0,IGc+ϵ0⪯Pt⪯Iϵ0.W_t+1=W_t- _sP_t m_t, P_t=diag_j 1 d v_t,j+ _0, IG_c+ _0 P_t I _0. (F.35) The ridge coefficient is constant throughout the moment history. These are spectral bounds on the realized denominator for coupled regularization. ∎ Proposition F.14 (Finite sampled source stopping). Use the source law and architecture of Sections F.3 and E. Choose the precision, Gram tolerance, envelopes, positive rates and batches by (F.36)–(F.43), with (F.37) and (F.40) satisfied. Conditional on the stated initialization, row and Gram events, the all-parameter source Adam procedure below stops within TsT_s inspections with probability at least 1−δbatch1- _ batch. Its uniform legal-pattern source-logit error relative to ΘλΦ _λ is less than h/2h/2, including unseen words and generated state histories. It uses at most BsTsB_sT_s fresh records, NBsTsNB_sT_s targets and Ts−1T_s-1 updates, with no verifier queries. Proof. We now use β1=0.9 _1=0.9, β2=0.999 _2=0.999, u=1−β1u=1- _1, and ϵ0=1 _0=1. Every supplied parameter, including the backbone and any normalization gain, is optimized with the complete source loss plus λ/2λ/2 times the squared norm of all supplied parameters. In the fixed-head comparison (F.21), the constant backbone penalty is omitted; it remains in the actual backbone gradients. All moments start at zero and remain carried throughout source training. Choose h∈(0,1)h∈(0,1) and 0<e≤minh/N,c2(q−1)232q2H2N,γ=d−1/8≤1/8,0<e≤ \h/N, c^2(q-1)^232q^2H^2N \, γ=d^-1/8≤ 1/8, (F.36) and use (F.20). Require the realized hard Gram event and ∥C⊤C−I∥op≤γ,μF≤λh64K,γ≤λh64KK(Rλ+1). C C-I _op≤γ, _F≤ λ h64 K, γ≤ λ h64 KK(R_λ+1). (F.37) Then (F.23) is less than h/8h/8. Here the source feature envelope is As=12A_s=12, distinct from the canonical norm K K. For an initialization failure allocation δ0 _0, put M0 M_0 =2log(2Vd/δ0), = 2 (2Vd/ _0), Wmax W_ =VM0, = VM_0, (F.38) Crow C_ row =22k+4log(2d/δ0), =2 2k+4 (2d/ _0), Ch C_h =2KCrow+3, =2 KC_ row+3, where k is the canonical feature dimension. The row and readout events are maxid∥Ci∥≤Crow _i d C_i ≤ C_ row and maxy,i|W0,yi|≤M0/d _y,i|W_0,yi|≤ M_0/ d. Use (F.34) and define λp _p =(Gc+1)−1, =(G_c+1)^-1, Λp _p =1, =1, L L =2K+1, =2K+1, (F.39) Finit F_ init =logV+2AsWmax+λWmax2/2, = V+2A_sW_ +λ W_ ^2/2, R R =2(2Finit+1)/λ, = 2(2F_ init+1)/λ, ρ ρ =λh/(64K), =λ h/(64 K), rgoal r_ goal =ρ/4, =ρ/4, ηs _s =min1,λpu3/2/(8L),1/(λprgoal2), = \1, _pu^3/2/(8L),1/( _pr_ goal^2)\, b b =λpρ/32, = _pρ/32, Ts T_s =max1,⌈4Finitηsλprgoal2⌉, = \1, 4F_ init _s _pr_ goal^2 \, εfeat _ feat =min1,b/[2(AsR+2)],h/(8R). = \1,b/[2(A_sR+ 2)],h/(8R)\. In addition require γ≤εfeat/16.γ≤ _ feat/16. (F.40) Choose the initial raw soft/hard error using the finite temperature construction in Appendix E to be at most zraw=minεfeat/(82),1/(4d).z_ raw= \ _ feat/(8 2),1/(4 d)\. (F.41) The normalized member then has initial discrepancy at most 2zraw+4γ≤3εfeat/8 2z_ raw+4γ≤ 3 _ feat/8; the base member has a smaller bound. Both initial coordinate envelopes are at most (Ch−1)/d(C_h-1)/ d. Let D be the complete feature-derivative bound on the radius-one backbone ball and let PψP_ψ count all supplied backbone coordinates. Set rψ=min1,1/D,εfeat/(2D),1/(Dd),ρback=min1,rψd2ηsTsKmomPψ>0.r_ψ= \1,1/D, _ feat/(2D),1/(D d)\, _ back= \1, r_ψ d2 _sT_sK_ mom P_ψ \>0. (F.42) Every backbone coordinate uses physical step ηsρback/d _s _ back/ d. The Adam coordinate-step bound gives total displacement at most rψ/2r_ψ/2, independently of the batches and including the coupled regularizer gradients. Thus all feature coordinates remain bounded by Ch/dC_h/ d and the full-feature discrepancy stays at most εfeat _ feat. This also includes the normalization gain when present. With Phead=VdP_ head=Vd, choose Bs=⌈8PheadAs2b−2log2PheadTsδbatch⌉.B_s= 8P_ headA_s^2b^-2 2P_ headT_s _ batch . (F.43) At each current source iterate, draw BsB_s fresh complete records and compute the averaged all-target gradient, including the coupled regularizer. If the head gradient’s Frobenius norm is at most ρ/2ρ/2, stop before any update. Otherwise update all blocks using actual bias-corrected Adam, unless this was the TsT_sth check; a failed final check returns failure without another update. A stopping batch is not inserted into a later moment. If the final check fails, mark source acquisition as failed and pass the final inspected parameter state to the two prescribed feedback batches; take no further source update. This algorithm uses at most TsT_s batches and Ts−1T_s-1 updates on every outcome. Its selection statistic uses only source observations. Explicitly, for each performed update and every supplied coordinate, its actual minibatch gradient gtg_t enters mt=β1mt−1+(1−β1)gt,vt=β2vt−1+(1−β2)gt2,m^t=mt1−β1t,v^t=vt1−β2t.m_t= _1m_t-1+(1- _1)g_t, v_t= _2v_t-1+(1- _2)g_t^2, m_t= m_t1- _1^t, v_t= v_t1- _2^t. (F.44) The coordinate subtracts its prescribed physical step times m^t/(v^t+1/d) m_t/( v_t+1/ d). All blocks update simultaneously, with their strictly positive head or backbone rates. Within ∥W∥F≤R W _F≤ R, the difference between actual-feature and hard-reference population head gradients is at most (AsR+2)εfeat≤b/2(A_sR+ 2) _ feat≤ b/2. Each coordinate of a complete-record CE gradient lies in [−As,As][-A_s,A_s]. Conditional scalar Hoeffding and a union bound over the PheadP_ head coordinates and TsT_s predictable inspections give a batch/mean Frobenius error at most b/2b/2, with failure at most δbatch _ batch. Thus every inspected actual gradient differs from ∇JC(W)∇ J_C(W) by at most b inside the stopped tube. Coordinate confinement makes the effective preconditioner in (F.35) lie between λpI _pI and I. Apply (F.26) with η=ηsη= _s, noise bound b, no additive update error, and target rgoalr_ goal. The tube construction uses μ=λμ=λ and headroom one. If the final check fails, extend its already computed gradient by one counterfactual update only in this argument. The bound then gives some inspected gradient norm at most rgoalr_ goal, while every prefix loss remains below 2Finit+12F_ init+1. First exit closes ∥W∥F<R W _F<R. Since b≤ρ/32b≤ρ/32, such a reference gradient necessarily triggers the observable threshold ρ/2ρ/2. Hence failure cannot occur on the good event. Conversely, a triggering gradient gives ∥∇JC(W)∥F≤ρ/2+b<ρ ∇ J_C(W) _F≤ρ/2+b<ρ. Strong convexity now yields ∥W−W∗∥F≤ρ/λ,K∥C∥opρ/λ<h/32,Rεfeat≤h/8. W-W^* _F≤ρ/λ, K C _opρ/λ<h/32, R _ feat≤ h/8. (F.45) Together with (F.37), these bounds give uniform source-logit discrepancy less than h/2h/2 from ΘλΦ _λ , including unseen legal-pattern words and incorrect generated state histories. The canonical class masses are at least exp(−Ne) (-Ne) and |bj−a|≤c/(8H)|b_j-a|≤ c/(8H). The actual source policy inherits the corresponding explicit leakage and transition perturbation bounds, and its initial control choice is close to uniform because the canonical control rows agree. Its worst-case consumption is BsTsB_sT_s fresh records, NBsTsNB_sT_s raw next-token targets, at most Ts−1T_s-1 updates, and TsT_s gradient-norm checks, with zero verifier calls. ∎ Lemma F.15 (Width requirements without self-reference). For the source parameters in Proposition F.14, the envelopes (F.46) are independent of d. The explicit bounds (F.48) ensure the feature condition (F.40) and the Gram-accuracy condition in (F.37). With γ=d−1/8γ=d^-1/8, an initialization requirement d≥DΓγ−2d≥ D_ γ^-2 is satisfied by d≥DΓ4/3d≥ D_ ^4/3. Proof. Write td=1+logdt_d=1+ d and define quantities independent of d: m¯ m =2max1,log(2V/δ0), = 2 \1, (2V/ _0)\, (F.46) C¯ C =22k+4log(2/δ0)+4,c¯h=2KC¯+3, =2 2k+4 (2/ _0)+4, c_h=2 K\, C+3, g¯ g =1+c¯h+maxλm¯,c¯h+λKmom/u, =1+ c_h+ \λ m, c_h+λ K_ mom/u\, f¯ f =logV+2AsVm¯+(λV/2)m¯2,r¯=2(2f¯+1)/λ. = V+2A_s V\, m+(λ V/2) m^2, r= 2(2 f+1)/λ. Then M0≤m¯tdM_0≤ m t_d, Ch≤c¯htdC_h≤ c_h t_d, Gc+1≤g¯tdG_c+1≤ g t_d, Finit≤f¯tdF_ init≤ ft_d, and R≤r¯tdR≤ r t_d. Put Bdim=max16,1024g¯(Asr¯+2)/ρ,128r¯/h.B_ dim= \16,1024 g(A_s r+ 2)/ρ,128 r/h\. (F.47) The prescribed feature tolerance satisfies εfeat≥16/[Bdim(1+logd)] _ feat≥ 16/[B_ dim(1+ d)]. Since logd≤16d1/16/exp(1) d≤ 16d^1/16/ (1), the explicit requirements d≥[Bdim(1+16/exp(1))]16,d≥[64KK(Rλ+1)λh]8d≥[B_ dim(1+16/ (1))]^16, d≥ [ 64 KK(R_λ+1)λ h ]^8 (F.48) imply (F.40) and the Gram-accuracy inequality in (F.37). An initializer requirement d≥DΓγ−2d≥ D_ γ^-2 becomes d≥DΓ4/3d≥ D_ ^4/3 under γ=d−1/8γ=d^-1/8. The feature and head widths can then be instantiated without self-reference. The separate signed-source inequality in (F.37), and any feedback-driven tightening of εfeat _ feat, are handled in Section F.5. ∎ Lemma F.16 (Realized row envelope). Condition on coverage and the actual Gram event ‖C⊤C−I‖op≤γ≤1/8\|C C-I\|_ op≤γ≤ 1/8. If (F.50) holds, then maxid‖Ci‖≤Crow _i d\|C_i\|≤ C_ row, with CrowC_ row in (F.38), except with conditional probability at most δ0 _0. Proof. For completeness, the row event used above follows from left invariance conditional on coverage and the actual Gram event. In distribution, C=ZdHZ−1/2RC,HZ=Z⊤Z/d,RC=(C⊤C)1/2,C= Z dH_Z^-1/2R_C, H_Z=Z Z/d, R_C=(C C)^1/2, (F.49) where Z has independent standard Gaussian entries and is independent of RCR_C. If d≥128[klog9+log(4/δ0)],d≥ 128[k 9+ (4/ _0)], (F.50) then ∥HZ−I∥op≤1/2 H_Z-I _op≤ 1/2 except with probability δ0/2 _0/2. On this event, ∥HZ−1/2RC∥op≤3/2<2 H_Z^-1/2R_C _op≤ 3/2<2 because γ≤1/8γ≤ 1/8. The Gaussian moment-generating function at 1/41/4 gives Pr∥zi∥2>2k+4log(2d/δ0)≤δ0/(2d). \ z_i ^2>2k+4 (2d/ _0)\≤ _0/(2d). (F.51) A row union bound therefore supplies maxid∥Ci∥≤Crow _i d C_i ≤ C_ row with conditional failure at most δ0 _0. This does not assert independence of the realized feature rows. The readout-coordinate event has a separate failure allocation δ0 _0; coverage, full-Gram, parameter-envelope, and source-batch failures are added in Section F.5. ∎ F.4 Two sampled Adam updates after source training The source model constructed in Appendix F.3 supplies a uniform approximation to a canonical executor. We show that two complete, on-policy feedback batches identify the downstream binding while every supplied parameter continues to train. The argument first controls the actual Adam histories, then compares the correlated finite feature operator with a Gaussian operator uniformly over adaptive histories, and finally couples complete generated trajectories. Appendix F.5 selects the dimensions and precisions simultaneously and combines the probability events. Throughout this appendix, k denotes the canonical feature dimension, K=12K=12 is its squared feature norm, and A=KA= K. The hard feature map from Appendix E is hhard(x)=CΦ(x)h_ hard(x)=C (x), with C∈ℝd×kC ^d× k and ∥Φ(x)∥2=K (x) ^2=K. We write ∥⋅∥F · _F for the Frobenius norm. The source and feedback stages use β1=0.9 _1=0.9 and β2=0.999 _2=0.999. At feedback entry the parameters are retained, the source regularizer is removed, and all first and second moments are reset. Feedback ascent uses the complete terminal-reward score, baseline zero, and the original batch denominator. We record a bound that remains valid when the gradients and denominators are dependent. For a stage-local history g1,…,gtg_1,…,g_t, initialized with zero moments, define at,s a_t,s =(1−β1)β1t−s1−β1t, = (1- _1) _1^t-s1- _1^t, bt,s b_t,s =(1−β2)β2t−s1−β2t, = (1- _2) _2^t-s1- _2^t, (F.52) m^t(g) m_t(g) =∑s=1tat,sgs, = _s=1^ta_t,sg_s, v^t(g) v_t(g) =∑s=1tbt,sgs⊙2. = _s=1^tb_t,sg_s 2. (F.53) Both weight sequences sum to one. For 0≤β1<10≤ _1<1, 0<β2<10< _2<1, and β12<β2 _1^2< _2, put Kmom=[(1−β2)(1−β12/β2)]−1/2.K_ mom= [(1- _2)(1- _1^2/ _2) ]^-1/2. (F.54) Lemma F.17 (Magnitude and history perturbation). For every coordinate and every finite history, |m^t(g)|≤Kmomv^t(g)| m_t(g)|≤ K_ mom v_t(g). For ϵ>0ε>0, let t(g)=m^t(g)/(v^t(g)+ϵ)D_t(g)= m_t(g)/( v_t(g)+ε), coordinatewise. Two histories with the same zero initialization satisfy ∥t(g)−t(h)∥∞≤1+Kmomϵmaxs≤t∥gs−hs∥∞. _t(g)-D_t(h) _∞≤ 1+K_ momε _s≤ t g_s-h_s _∞. (F.55) In particular, every Adam coordinate direction has magnitude at most KmomK_ mom, without an upper gradient bound or a positive lower second moment. Proof. For one coordinate, weighted Cauchy–Schwarz gives |m^t(g)|2 | m_t(g)|^2 ≤Kt2v^t(g), ≤ K_t^2 v_t(g), (F.56) Kt2=∑s=1tat,s2bt,s K_t^2= _s=1^t a_t,s^2b_t,s =(1−β1)2(1−β1t)21−β2t1−β2∑j=0t−1(β12/β2)j≤Kmom2. = (1- _1)^2(1- _1^t)^2 1- _2^t1- _2 _j=0^t-1( _1^2/ _2)^j≤ K_ mom^2. (F.57) Here 1−β1t≥1−β11- _1^t≥ 1- _1; when β1=0 _1=0, the first-moment weight is concentrated at the newest gradient and the zeroth power is one. Write rg=(∑sbt,sgs2)1/2r_g=( _sb_t,sg_s^2)^1/2 and define rhr_h similarly. If dt=maxs≤t|gs−hs|d_t= _s≤ t|g_s-h_s|, then |m^t(g)−m^t(h)|≤dt,|rg−rh|≤dt| m_t(g)- m_t(h)|≤ d_t, |r_g-r_h|≤ d_t by the reverse triangle inequality in the weighted Euclidean norm. The identity mgrg+ϵ−mhrh+ϵ=mg−mhrg+ϵ+mhrh+ϵrh−rgrg+ϵ m_gr_g+ε- m_hr_h+ε= m_g-m_hr_g+ε+ m_hr_h+ε r_h-r_gr_g+ε now proves (F.55), including coordinates with zero second moment. ∎ The bound uses the complete history since the common zero initialization. Source moments are retained during source training and reset once at the stage boundary; both feedback histories enter the second update. The finite coefficient rows are correlated. Their relevant symmetry follows from the complete initialization in Appendix E. Lemma F.18 (Initial left invariance). Conditional on any relative-bias realization satisfying lag coverage, the base member’s actual hard coefficient operator obeys C=lawQC law=QC for every deterministic Q∈O(d)Q∈ O(d). Its canonical feature coordinates and lag weights are unchanged by this transformation. Proof. Distinguish the FFN matrices Bff,Cff,UffB_ f,C_ f,U_ f from the coefficient operator. Apply the deterministic transformation E E ↦QE, QE, Oh O_h ↦QOh, QO_h, Qh Q_h ↦QhQ⊤, Q_hQ , Kh K_h ↦KhQ⊤, K_hQ , Vh V_h ↦VhQ⊤, V_hQ , Bff B_ f ↦BffQ⊤, B_ fQ , Cff C_ f ↦CffQ⊤, C_ fQ , Uff U_ f ↦QUff, QU_ f, bout b_ out ↦Qbout, Qb_ out, Wout W_ out ↦WoutQ⊤. W_ outQ . Leave the relative biases and FFN input biases unchanged. Each Gaussian block has its original law under its indicated left or right orthogonal transform; the transformations also preserve joint independence. This remains true conditional on the unchanged relative biases. The query, key, and value vectors are unchanged, so all soft attention scores and weights are unchanged. The residual feature transforms as z↦Qz Qz; the FFN preactivations are unchanged and its output transforms by Q. For the hard reference, the lag blocks satisfy Aℓ′=QAℓA_ =QA_ and T′=QT =QT, since the lag-weight matrix is unchanged and invertible on coverage. The augmented FFN coefficient rows [BffT,bff][B_ fT,b_ f] and [CffT,cff][C_ fT,c_ f] are unchanged, hence so is their symmetric-product coefficient matrix F. The actual expansion therefore gives C′=[QT,QUffF,Qbout]=QC,(C′)⊤C′=C⊤C.C =[QT,QU_ fF,Qb_ out]=QC, (C ) C =C C. This proves the identity for the coefficient columns themselves, including their constant and linear terms, rather than only on sampled inputs. ∎ This lemma concerns the raw initial operator. For the normalized member the same operator is used as a reference, and the normalization and trainable gain are paid through the feature approximation. No trajectory equivariance of coordinatewise Adam is required. For rows ci⊤c_i of C, define the finite and Gaussian operators C(S) _C(S) =∑i=1dcici⊤dci⊤Sci+ϵ, = _i=1^d c_ic_i d\,c_i Sc_i+ε, (S) (S) =z∼N(0,Ik)zz⊤z⊤Sz+ϵ. =E_z N(0,I_k) z z Sz+ε. (F.58) The next estimate is uniform over the second-moment matrices, which permits their dependence on the realized features and training batches. Lemma F.19 (Uniform correlated adaptive geometry). Let k≥1k≥ 1, d≥kd≥ k, G≥0G≥ 0, ϵ>0ε>0, 0<τ≤10<τ≤ 1, and 0<δgeo<10< _ geo<1. Define K3 K_3 =8[k(k+2)]3/4δgeo, = 8[k(k+2)]^3/4 _ geo, Lpert L_ pert =6kϵ+GK3ϵ2, = 6kε+ GK_3ε^2, (F.59) γgeo _ geo =min18,τ16Lpert, = \ 18, τ16L_ pert \, ageo a_ geo =τ2, = τ2, (F.60) Ttr2 T_ tr^2 =max1,64k(k+2)ϵageoδgeo, = \1, 64k(k+2)ε a_ geo _ geo \, hnet h_ net =(ageoϵ216Ttr3)2, = ( a_ geoε^216T_ tr^3 )^2, (F.61) Dsym D_ sym =k(k+1)2, = k(k+1)2, Nnet N_ net =(1+2kG2hnet)Dsym. = (1+ 2 kG^2h_ net )^D_ sym. (F.62) Suppose C has a left-orthogonally invariant law and the event ℰC=∥C⊤C−I∥op≤γgeoE_C=\ C C-I _op≤ _ geo\ has positive probability. Set dgeo=⌈max32γgeo−2[klog9+log16δgeo],32k2Ttr4ϵ2ageo2log16k2Nnetδgeo⌉.d_ geo= \ aligned &32 _ geo^-2 [k 9+ 16 _ geo ],\\ & 32k^2T_ tr^4ε^2a_ geo^2 16k^2N_ net _ geo aligned \ . (F.63) If d≥dgeod≥ d_ geo, then, conditional on ℰCE_C, with probability at least 1−δgeo1- _ geo, supS⪰0,∥S∥op≤G2∥C(S)−(S)∥op≤τ. _S 0,\, S _op≤ G^2 _C(S)-A(S) _op≤τ. (F.64) The unconditional failure probability is at most Pr(ℰCc)+δgeo (E_C^c)+ _ geo. Proof. On ℰCE_C, let R=(C⊤C)1/2R=(C C)^1/2, which is positive definite and satisfies ∥R−I∥op≤γgeo R-I _op≤ _ geo. Conditional on R, left invariance makes CR−1CR^-1 a uniform Stiefel matrix. Averaging an independent uniform left rotation identifies this conditional law and its independence from R. Thus, conditional on ℰCE_C, we may represent C=d−1/2ZM,M=HZ−1/2R,HZ=Z⊤Z/d,C=d^-1/2ZM, M=H_Z^-1/2R, H_Z=Z Z/d, (F.65) where Z has independent standard Gaussian entries and is independent of R. This representation does not assert independence of the rows of C. For a fixed unit vector v, the chi-square moment generating function gives Pr(|v⊤(HZ−I)v|>b)≤2exp(−db2/8),0<b<1. (|v (H_Z-I)v|>b )≤ 2 (-db^2/8), 0<b<1. A 1/41/4-net of size at most 9k9^k and the symmetric-matrix net inequality give Pr(∥HZ−I∥op>γgeo)≤2expklog9−dγgeo2/32≤δgeo/8. ( H_Z-I _op> _ geo )≤ 2 \k 9-d _ geo^2/32\≤ _ geo/8. On the complementary event, ∥HZ−1/2−I∥op H_Z^-1/2-I _op ≤2γgeo, ≤ 2 _ geo, ∥R∥op R _op ≤1+γgeo, ≤ 1+ _ geo, (F.66) ∥M−I∥op M-I _op ≤4γgeo≤12, ≤ 4 _ geo≤ 12, d−1∑i∥zi∥2 d^-1 _i z_i ^2 ≤2k. ≤ 2k. Since ∥z∥3≤[k(k+2)]3/4E z ^3≤[k(k+2)]^3/4, Markov’s inequality also gives d−1∑i∥zi∥3≤K3d^-1 _i z_i ^3≤ K_3 except with probability δgeo/8 _ geo/8. Write Z(S)=d−1∑izizi⊤/(zi⊤Szi+ϵ)A_Z(S)=d^-1 _iz_iz_i /( z_i Sz_i+ε). Superscript tr tr denotes truncation to ∥z∥≤Ttr z ≤ T_ tr. Uniformly over S, ∥(S)−tr(S)∥op≤[∥z∥2∥z∥>Ttr]ϵ≤k(k+2)ϵTtr2≤ageoδgeo64. (S)-A tr(S) _op≤ E[ z ^21_\ z >T_ tr\]ε≤ k(k+2)ε T_ tr^2≤ a_ geo _ geo64. (F.67) The corresponding empirical tail is at most ageo/8a_ geo/8 except with probability δgeo/8 _ geo/8, by Markov’s inequality. The positive semidefinite (PSD) matrices of operator norm at most G2G^2 lie in a Frobenius ball of radius kG2 kG^2 in dimension DsymD_ sym. A packing argument supplies an hneth_ net-net, with centers in that set, of size at most NnetN_ net. Each truncated matrix entry has magnitude at most Ttr2/ϵT_ tr^2/ε. Scalar Hoeffding at entry tolerance ageo/(4k)a_ geo/(4k), unioned over the net and k2k^2 entries, has failure probability at most 2k2Nnetexp−dageo2ϵ232k2Ttr4≤δgeo/8.2k^2N_ net \- da_ geo^2ε^232k^2T_ tr^4 \≤ _ geo/8. (F.68) At every net point the operator error is at most ageo/4a_ geo/4, since an entrywise bound b gives operator norm at most kbkb. For two PSD matrices, |z⊤Sz−z⊤S′z|≤∥z∥∥S−S′∥op.| z Sz- z S z|≤ z S-S _op. Consequently each truncated kernel changes by at most Ttr3hnet/ϵ2T_ tr^3 h_ net/ε^2 within a net ball. The empirical and population extension costs together are at most 2Ttr3hnet/ϵ2=ageo/82T_ tr^3 h_ net/ε^2=a_ geo/8. Combining both tails, the net bound, and its extension gives supS∥Z(S)−(S)∥op≤ageo8+ageo64+ageo4+ageo8=33ageo64<ageo. _S _Z(S)-A(S) _op≤ a_ geo8+ a_ geo64+ a_ geo4+ a_ geo8= 33a_ geo64<a_ geo. (F.69) It remains to transfer through M. For S(x)=xx⊤/(x⊤Sx+ϵ)K_S(x)=x /( x Sx+ε) and κ=∥M−I∥op≤1/2κ= M-I _op≤ 1/2, direct subtraction gives ∥S(M⊤y)−S(y)∥op≤κ[(2+κ)∥y∥2ϵ+G∥y∥3ϵ2]. _S(M y)-K_S(y) _op≤κ [ (2+κ) y ^2ε+ G y ^3ε^2 ]. (F.70) Here the numerator difference is bounded by (∥M⊤y∥+∥y∥)∥(M⊤−I)y∥( M y + y ) (M -I)y ; both denominators are at least ϵε, and their difference is at most G∥(M⊤−I)y∥G (M -I)y . Averaging (F.70) and using the second- and third-moment bounds gives supS∥C(S)−Z(S)∥op≤4γgeoLpert≤τ/4. _S _C(S)-A_Z(S) _op≤ 4 _ geoL_ pert≤τ/4. Together with (F.69), this is at most 3τ/4≤τ3τ/4≤τ. The Gaussian auxiliary failures total at most δgeo/2 _ geo/2, uniformly over the admissible R; integrating its conditional law proves the lemma. ∎ Corollary F.20 (Adaptive histories and exact coefficient scaling). On the event of Lemma F.19, every coefficient history with ‖us‖≤G\|u_s\|≤ G satisfies the moment, subspace-leakage and Gaussian lower bounds below. With physical gradients CusCu_s, denominator ϵ/dε/ d and rate η/dη/ d, the coefficient increment is exactly (F.72), including the current adaptive denominator. Proof. For complete coefficient histories usu_s, set mt=∑sat,sus,St=∑sbt,susus⊤.m_t= _sa_t,su_s, S_t= _sb_t,su_su_s . (F.71) If ∥us∥≤G u_s ≤ G, then ∥mt∥≤G m_t ≤ G and ∥St∥op≤G2 S_t _op≤ G^2. Equation (F.64) therefore applies to histories chosen using C and any training information. For histories in a subspace H0H_0, Gaussian reflection gives PH0⟂(St)mt=0P_H_0 A(S_t)m_t=0 and hence ∥PH0⟂C(St)mt∥≤τG P_H_0 A_C(S_t)m_t ≤τ G. The same Gaussian operator satisfies (S)⪰I2(G2(k+2)+ϵ).A(S) I2(G 2(k+2)+ε). Indeed, retain ∥z∥≤2(k+2) z ≤ 2(k+2) and use [(v⊤z)2∥z∥2]=k+2E[(v z)^2 z ^2]=k+2 for each unit v. This lower bound concerns action on the momentum vector; current gradients need not align with previous momentum. For fixed features, a physical gradient CusCu_s and coefficient coordinate ϑ=C⊤w =C w yield the exact ascent identity ϵphysical=ϵ/d,ηphysical=η/d⟹Δϑ=ηC(St)mt. _ physical=ε/ d, _ physical=η/ d = _C(S_t)m_t. (F.72) The current denominator remains inside CA_C. ∎ Lemma F.21 (Rewards of the learned executor). Let q≥16q≥ 16, c=1/64c=1/64, H≥5H≥ 5, and |bj−(1−c/H)|≤c/(8H)|b_j-(1-c/H)|≤ c/(8H) for j=1,2j=1,2. Conditional on correct output classes, the executor rewards satisfy (F.74) and (F.75), including trajectories with repaired hidden-bit errors. In particular, r∗−r¯≥1/2r_*- r≥ 1/2 and r¯<r∗/q r<r_*/q. Proof. Let q≥16q≥ 16, c=1/64c=1/64, and let the source conditional transition parameters satisfy |bj−(1−c/H)|≤c/(8H)|b_j-(1-c/H)|≤ c/(8H) for j=1,2j=1,2, as proved in Appendix F.3. Define Bexec=b1b2H−1,dj=bj+2(1−bj)/q,Dexec=d1d2H−1.B_ exec=b_1b_2^H-1, d_j=b_j+2(1-b_j)/q, D_ exec=d_1d_2^H-1. (F.73) At a valid transition of type j, the correct and hidden-bit-flipped states have probabilities αj=bj+(1−bj)/q _j=b_j+(1-b_j)/q and γj=(1−bj)/q _j=(1-b_j)/q. Their sum is djd_j and difference is bjb_j. Summing all even and odd hidden-error histories gives, conditional on the choice and correct output classes, r∗=Dexec+Bexec2,r¯=Dexec−Bexec2,ra=0for the other public arms.r_*= D_ exec+B_ exec2, r= D_ exec-B_ exec2, r_a=0 the other public arms. (F.74) Here r∗r_* is the reward of the bound operation and r¯ r that of its hidden-bit opposite. Other public operations cannot reach the required public endpoint while satisfying the grammar and public-coordinate checks. Repaired hidden-bit histories are included in (F.74). Conditional on the correct output classes, the public-coordinate checks pass with probability DexecD_ exec, independently of the selected control, and r∗≥Bexec≥1−9c/8,0≤r¯≤∑jγj≤9c8q,1−Dexec≤9c/8.r_*≥ B_ exec≥ 1-9c/8, 0≤ r≤ _j _j≤ 9c8q, 1-D_ exec≤ 9c/8. (F.75) The sum over j counts one first transition and H−1H-1 repeated transitions. Thus Δ=r∗−r¯≥1/2 =r_*- r≥ 1/2 and r¯<r∗/q r<r_*/q. ∎ Lemma F.22 (Prompt-independent canonical choice geometry). For the canonical source feature map on a legal choice prompt, with H≥5H≥ 5, let μ be its mean under the training prompt law. Then (F.76) holds on every training and held-out choice prompt. Proof. Let μ be the mean canonical feature of a training choice prompt. A degree-two coordinate retains at most two controls, so H≥5H≥ 5 leaves an unused balanced suffix tag. Its expectation is therefore the same in either tag class. Against each fixed legal choice prompt, averaging the kernel’s matching indicators also gives the same constant. Consequently μ⊤Φ(x)=∥μ∥2=:κ∈[2,K]for every training and held-out choice prompt.μ (x)= μ ^2=:κ∈[2,K] every training and held-out choice prompt. (F.76) This identity keeps the reference choice probabilities prompt-independent. ∎ Definition F.23 (Two-step Gaussian reference). The Gaussian reference starts with zero coefficient increments Y0=0Y_0=0 and uniform choice probabilities. Its row gradient is ut,a=ct,aμ,ct,a=pt−1,a(ra−Jt−1),Jt−1=∑apt−1,ara.u_t,a=c_t,aμ, c_t,a=p_t-1,a(r_a-J_t-1), J_t-1= _ap_t-1,ar_a. (F.77) Each row uses its own corrected moments and update ηt(St,a)mt,a _tA(S_t,a)m_t,a. Set ϵf _f =164q, = 164q, ρ0 _0 =2e−22π, = 2e^-2 2π, C0 C_0 =2(1−β1)ρ0642+1, = 2(1- _1) _064 2+1, (F.78) η1 _1 =1100KmomK, = 1100K_ mom K, η2 _2 =log(15(q−1))C0, = (15(q-1))C_0, Sstep S_ step =η1+η2. = _1+ _2. (F.79) Lemma F.24 (Two Gaussian reference updates). With (F.79), the reference chooses the bound operation with probability at least 15/1615/16 after the second update, on every training and held-out choice prompt. Its total Frobenius path length is at most Rref=SstepCmove,Cmove=Kmom+32K.R_ ref=S_ stepC_ move, C_ move=K_ mom+32 K. (F.80) Proof. For a row history, write mt,a=c¯t,aμm_t,a= c_t,aμ and St,a=vt,aμμ⊤S_t,a=v_t,aμ . Gaussian symmetry gives (vμμ⊤)μ=μℬ(v),ℬ(v)=Z∼N(0,1)Z2κv|Z|+ϵf>0.A(vμ )μ= (v), (v)=E_Z N(0,1) Z^2 κ v|Z|+ _f>0. (F.81) Every prompt logit increment is ηtκc¯t,aℬ(vt,a) _tκ c_t,aB(v_t,a). Whenever pt−1,∗≥1/qp_t-1,*≥ 1/q, Jt−1≥r∗/q>r¯J_t-1≥ r_*/q> r: the correct row has positive gradient and momentum, and every wrong row has negative gradient and momentum. Thus the correct probability is nondecreasing, and prompt independence follows from (F.76). Before pt−1,∗p_t-1,* reaches 15/1615/16, ct,∗ c_t,* ≥Δpt−1,∗(1−pt−1,∗)≥Δ/(16q), ≥ p_t-1,*(1-p_t-1,*)≥ /(16q), (F.82) cs,∗ c_s,* ≤r∗ps−1,∗≤r∗pt−1,∗≤(16r∗/Δ)ct,∗≤32ct,∗(s≤t). ≤ r_*p_s-1,*≤ r_*p_t-1,*≤(16r_*/ )c_t,*≤ 32c_t,* (s≤ t). (F.83) The newest first-moment weight is at least 1−β11- _1, so c¯t,∗≥(1−β1)ct,∗ c_t,*≥(1- _1)c_t,* and vt,∗≤32ct,∗ v_t,*≤ 32c_t,*. The event 1≤|Z|≤21≤|Z|≤ 2 has probability at least ρ0 _0. Therefore ℬ(vt,∗)≥ρ02κvt,∗+ϵf.B(v_t,*)≥ _02 κ v_t,*+ _f. Writing χ=16qϵf/Δ≤1χ=16q _f/ ≤ 1, the correct logit increment is at least ηt(1−β1)ρ0κ64κ+χ≥ηtC0. _t(1- _1) _0 κ64 κ+χ≥ _tC_0. (F.84) The fraction increases for κ≥2κ≥ 2. Wrong logits decrease. If the target has not already been reached, the second step alone therefore raises every correct-versus-wrong logit gap by at least log(15(q−1)) (15(q-1)). Since the gaps started at zero and increased on the first step, p2,∗≥15/16p_2,*≥ 15/16. For the path bound, Lemma F.17 and Gaussian dual Cauchy–Schwarz bound each row field by KmomK_ mom. The first step thus changes each logit by at most 1/1001/100. At both gradient evaluations, pt−1,a≤e0.02/q≤50/(49q),t=1,2.p_t-1,a≤ e^0.02/q≤ 50/(49q), t=1,2. Using Jt−1≤(3/2)(50/49)r∗/qJ_t-1≤(3/2)(50/49)r_*/q and negativity of wrong coefficients gives |ct,a|≤2r∗/q2|c_t,a|≤ 2r_*/q^2 for wrong rows. Their corrected first moments have norm at most 2r∗K/q22r_* K/q^2. Since ∥(S)∥op≤1/ϵf (S) _op≤ 1/ _f, each wrong row field has norm at most 128r∗K/q128r_* K/q. Their aggregate norm is at most 128K/q≤32K128 K/ q≤ 32 K, while the correct row contributes at most KmomK_ mom. Summing the two step lengths proves (F.80). ∎ Definition F.25 (Sampled feedback and coefficient increments). Generate exactly n=H+2n=H+2 full-vocabulary decisions in the designated choice, state, and EOS slots. The verifier checks grammar, public-coordinate consistency, and the terminal endpoint. Every sampled trajectory, including a zero-reward one, contributes through the original full batch denominator. Define Xt=(Wfast,t−Wfast,s)C,X0=0,X_t=(W_ fast,t-W_ fast,s)C, X_0=0, where Wfast,sW_ fast,s is the source control-row readout. The fixed-executor choice-only population gradient, with the actual training-prompt distribution, is denoted U(X)U(X). Lemma F.26 (Choice gradients and two-step history fields). Under Definition F.25, the fixed-executor choice gradient is globally 6K6K-Lipschitz as in (F.85). For histories of length at most two, the Gaussian row fields are Lipschitz in the maximum Frobenius history error, with constant DAD_A in (F.86). Proof. Bounded rewards and ∥Φ∥≤A ≤ A give the global bound ∥U(X)−U(Y)∥F≤Lc∥X−Y∥F,Lc=6K. U(X)-U(Y) _F≤ L_c X-Y _F, L_c=6K. (F.85) At the reference iterates this gradient is (F.77). For two histories, the Gaussian row fields together are Lipschitz in the maximum Frobenius history error, with constant DA=2(1+Kmom)ϵf.D_A= 2(1+K_ mom) _f. (F.86) To see this, apply (F.55) to each scalar history z⊤usz u_s. The squared scalar maximum is bounded by the sum of its two squared errors. Summing over output rows and using dual Cauchy–Schwarz against a unit Frobenius test matrix gives (F.86); Gaussian second moments replace each scalar square by its coefficient norm squared. There is no factor depending on the number of canonical coordinates or rows. The physical-coordinate version uses epsilon ϵf/d _f/ d. ∎ Lemma F.27 (Complete-trajectory population comparison). Under Definition F.25, set G=n2AG=n 2A and impose the uniform geometry event. While the coefficient discrepancy from Definition F.23 is at most E∗E_* in (F.87), assume source, slow-head and feature-motion logit error at most 2h2h, canonical class leakage at most h≤1h≤ 1, and feature error at most efe_f. Then the actual reward-weighted coefficient score satisfies (F.88), with CpopC_ pop as displayed there. Proof. Set G=n2AG=n 2A. Apply Lemma F.19 with this G, ϵ=ϵfε= _f, and tolerance τ. A complete coefficient score has norm at most G, so the same event covers all actual row histories. Compare the iterates until their coefficient error exceeds E∗=(1024A)−1,Bexp=A(Rref+1).E_*=(1024A)^-1, B_ =A(R_ ref+1). (F.87) Within this region ∥Xt∥F≤Rref+1 X_t _F≤ R_ ref+1. Let h≤1h≤ 1 be the source logit tolerance. The source entry and slow-rate choices in Appendices F.3 and F.5 supply three uniform bounds: the source hard-logit error plus slow-head and feature-motion errors is at most 2h2h; the actual feature error is at most efe_f; and the canonical source class leakage is at most h, using Ne≤hNe≤ h. On each matching legal-class history, the actual incorrect-class mass is at most ℓ=hexp(2Bexp+4) =h (2B_ +4). The factor pays the change in class odds caused by arbitrary fast-row coefficients in the stopped region. Once an incorrect-class token is generated the coupling has already failed, so no class-mass bound is required on its malformed continuation. Conditional state laws only change through the 2h2h perturbation of state rows; fast control rows do not change the within-state normalization. Conditional choice and state laws differ from the fixed-executor laws by at most 8h8h in total variation. Let gcoeff,tg_ coeff,t denote the complete reward-weighted score expressed using Φ , with the probabilities and trajectories of the actual policy. Maximal conditional couplings on matching histories give ∥[gcoeff,t∣ℱt−1]−U(Xt−1)∥F≤Cpoph,Cpop=256n2Aexp(2Bexp+4). [g_ coeff,t _t-1]-U(X_t-1) _F≤ C_ poph, C_ pop=256n^2A (2B_ +4). (F.88) Here the update indexed t is evaluated at iterate t−1t-1. Indeed the path total variation is at most n(8h+ℓ)n(8h+ ), and both complete coefficient score vectors have norm at most n2Kn 2K. On a matched valid path, each non-choice fast-control score has norm at most 2Kℓ 2K , whereas the reference fast score is zero there. The choice fast-score difference is at most 82K(h+ℓ)8 2K(h+ ). The same deterministic verifier returns the same reward on matched paths, including histories whose hidden errors repair. Hence the expectation error is at most 2n2Kn(8h+ℓ)+8n2K(h+ℓ)≤Cpoph.2n 2K\,n(8h+ )+8n 2K(h+ )≤ C_ poph. This accounts for non-choice prefixes, EOS, distribution change, and malformed trajectories without a teacher-prefix substitution. ∎ Lemma F.28 (Concentration for the two fresh feedback batches). For the two predictable iterates in Definition F.25, let each fresh batch have size (F.89). With probability at least 1−δR1- _R, both sampled coefficient gradients differ from their conditional means in Frobenius norm by at most εg _g. Independence is required within each fresh batch, conditional on its past. Proof. For a desired sampling error εg _g, take BR=⌈8qkn2A2εg−2log4qkδR⌉.B_R= 8qkn^2A^2 _g^-2 4qk _R . (F.89) Each complete-trajectory coefficient entry lies in [−nA,nA][-nA,nA]. Conditional Hoeffding and a union over qkqk coordinates and the two predictable iterates give both Frobenius errors at most εg _g with probability at least 1−δR1- _R; the smaller threshold εg/2 _g/2 already meets this allocation. Only trajectories within each fresh batch are independent. ∎ Lemma F.29 (History-aware moving-feature comparison). On the events and within the stopped region of Lemmas F.27 and F.28, suppose the actual feature error is at most efe_f and ‖C‖op≤2\|C\|_ op≤ 2. The complete physical Adam updates, including both moment histories, satisfy the coefficient-error recurrence (F.90) for t=1,2t=1,2. Proof. The actual physical fast gradient differs from gcoeff,tC⊤g_ coeff,tC by at most n2efn 2e_f. Apply the physical history bound before projection: for two histories the squared sum of coordinatewise maximal errors is at most the sum of the two squared Frobenius errors. Multiplying the physical rate ηt/d _t/ d by the reciprocal physical epsilon d/ϵf d/ _f cancels d. Postmultiplication by C, with ∥C∥op≤2 C _op≤ 2, therefore costs at most 2ηtDAn2ef2 _tD_An 2e_f. The fixed-feature update is exactly (F.72); its uniform operator error costs at most ηtτG _tτ G across all rows. Writing Et=max0≤s≤t∥Xs−Ys∥FE_t= _0≤ s≤ t X_s-Y_s _F, Gaussian history comparison and (F.85) give, for t=1,2t=1,2, Et≤(1+ηtDALc)Et−1+ηt[DA(Cpoph+εg)+τG+2DAn2ef].E_t≤(1+ _tD_AL_c)E_t-1+ _t [D_A(C_ poph+ _g)+τ G+2D_An 2e_f ]. (F.90) Both the moment numerator and the current random denominator are retained. The equality X0=Y0=0X_0=Y_0=0 concerns increments from the learned source; its nonzero source function error enters (F.88). ∎ Proposition F.30 (Two sampled updates preserve the reference margin). Use Definition F.25, the source guarantee of Proposition F.14, and the reference of Definition F.23. On the geometry and sampling events, choose tolerances by (F.91) and (F.93), and impose the source-error and slow-motion bounds below. Then maxt≤2‖Xt−Yt‖F≤E∗/2 _t≤ 2\|X_t-Y_t\|_F≤ E_*/2. The final choice-logit discrepancy is at most AE∗+2hAE_*+2h, and the remaining complete-path discrepancy is at most n(8h+hexp(2Bexp+4))n(8h+h (2B_ +4)). The procedure uses exactly two updates and 2BR2B_R verifier queries. Proof. Define Qamp=(1+η1DALc)(1+η2DALc),ξ=E∗8SstepQamp.Q_ amp=(1+ _1D_AL_c)(1+ _2D_AL_c), ξ= E_*8S_ stepQ_ amp. (F.91) Choose h h ≤min1220nexp(2Bexp+4),ξDACpop, ≤ \ 12^20n (2B_ +4), ξD_AC_ pop \, εg _g ≤ξ/DA, ≤ξ/D_A, (F.92) τ τ ≤ξ/G, ≤ξ/G, ef e_f ≤minξ2DAn2,h32RW. ≤ \ ξ2D_An 2, h32R_W \. (F.93) Here a deterministic bound for the complete physical readout is RW=R+SstepKmomq+1R_W=R+S_ stepK_ mom q+1, where R is the source head bound. The forcing in (F.90) is at most 4ξ4ξ, so E2≤4SstepQampξ=E∗/2E_2≤ 4S_ stepQ_ ampξ=E_*/2 and the stopped region cannot be exited. This argument controls the large second update from its gradient evaluation inside the region. The source feature error is included in the total efe_f allowance, rather than reset at feedback entry. It suffices to cap it at ef/2e_f/2, to require slow readout movement at most h/(32A)h/(32A), and to bound additional backbone movement by min12,ef2D,h32DRW \ 12, e_f2D, h32DR_W \ (F.94) inside the remaining derivative ball. Appendix F.5 uses the stronger source allowance ef/4e_f/4, reserves the radius remaining after source training, and specifies a positive common slow multiplier. The bound on each Adam coordinate from Lemma F.17 makes these movement bounds deterministic. They include every backbone parameter and the normalized member’s gain. The full RWR_W also includes physical readout nullspace components and pays the head/backbone cross term through the feature error. At the final iterate the choice logit error from X2−Y2X_2-Y_2 is at most AE∗AE_*, with an additional base error at most 2h2h. The remaining complete-path error is bounded by n(8h+hexp(2Bexp+4))n(8h+h (2B_ +4)). The reference task utility and first-state retention are at least (15/16)(1−9c/8)(15/16)(1-9c/8), and its grammar-or-public-coordinate violation probability is at most 9c/89c/8. Appendix F.5 combines these margins with the source baseline, the finite-width events, and (F.93) to obtain the stated capability and gain bounds. The two feedback batches use exactly 2BR2B_R fresh training prompts, 2BR2B_R binary verifier calls, 2nBR2nB_R unrestricted generated decisions, and two optimizer updates. The 2HBR2HB_R state, 2BR2B_R control, and 2BR2B_R EOS counts refer to designated positions; the actual emitted token classes may differ. There is no feedback checkpoint selection or test-time verifier use. ∎ F.5 An explicit joint source and feedback budget We give an explicit order for choosing parameters for the Gaussian architecture of Appendix E, the sampled ridge-Adam source procedure of Appendix F.3, and the two sampled feedback updates of Appendix F.4. All logarithms are natural unless their base is displayed. Fix q=22r+2≥16,V=2q+2,c=164,K=12,β1=0.9,β2=0.999.q=2^2r+2≥ 16, V=2q+2, c= 164, K=12, _1=0.9, _2=0.999. The source estimates use Asrc=12A_src=12, whereas feedback uses Afb=KA_fb= K. These two constants have different roles. Here S is SstepS_ step in Appendix F.4, and εf _f below is its feature tolerance efe_f. For a requested failure probability 0<δ<10<δ<1, allocate δbias=δenc=δffn=δnorm _bias= _enc= _ffn= _norm =δ/16, =δ/16, δ0 _0 =δ/16, =δ/16, δgeo=δbatch=δR _geo= _batch= _R =δ/8. =δ/8. (F.95) The allocation δ0 _0 is used separately for the row envelope and the readout-coordinate event. Lemma F.31 (An admissible horizon and source precision). Fix the task and optimizer parameters above. The ordered choices (F.96) and (F.97) satisfy the signed-source requirement μF≤λh/(64K) _F≤λ h/(64 K) of Proposition F.14 and both feedback bounds on h in (F.93). Every choice depends only on the declared task parameters; the minimum allowed horizon is determined before H is chosen. Proof. First define constants that depend only on q and the fixed optimizer parameters: u u =1−β1, =1- _1, Kmom K_mom =[(1−β2)(1−β12/β2)]−1/2, = [(1- _2)(1- _1^2/ _2) ]^-1/2, ϵf _f =164q, = 164q, ρ0 _0 =2exp(−2)2π, = 2 (-2) 2π, C0 C_0 =2(1−β1)ρ0642+1, = 2(1- _1) _064 2+1, η1 _1 =1100KmomK, = 1100K_mom K, η2 _2 =log(15(q−1))C0, = (15(q-1))C_0, S S =η1+η2, = _1+ _2, Cmove C_move =Kmom+32K, =K_mom+32 K, Rref R_ref =SCmove, =SC_move, Bexp B_exp =Afb(Rref+1), =A_fb(R_ref+1), Eexp E_exp =exp(2Bexp+4), = (2B_exp+4), DA D_A =2(1+Kmom)ϵf, = 2(1+K_mom) _f, Lc L_c =6K, =6K, E∗ E_* =11024Afb, = 11024A_fb, Qamp Q_amp =(1+η1DALc)(1+η2DALc), =(1+ _1D_AL_c)(1+ _2D_AL_c), ξ ξ =E∗8SQamp. = E_*8SQ_amp. Set h0 h_0 =min1220Eexp,ξ256AfbDAEexp, = \ 12^20E_exp, ξ256A_fbD_AE_exp \, Cq C_q =23(5q+V+1)+2q(q−1), = 23(5q+V+1)+2q(q-1), L0 L_0 =max1,log24qVch0, = \1, 24qVch_0 \, BH B_H =7372800KCqq3V3/2L02h02. = 7372800KC_qq^3V^3/2L_0^2h_0^2. Choose any integer horizon satisfying H≥⌈max128, 2log2BH⌉,n=H+2,N=2H+2.H≥ \128,\,2 _2B_H\ , n=H+2, N=2H+2. (F.96) For the minimum prescribed horizon, the right side determines H directly from q; larger prescribed horizons are also allowed. Set the tolerances exactly as follows: h=h0(H+2)2,e=minhN,c2(q−1)232q2H2N,D¯=maxD∗(e/2),e,λ=eD¯,Rλ=2D¯. gatheredh= h_0(H+2)^2, e= \ hN, c^2(q-1)^232q^2H^2N \,\\ D= \D_*(e/2),e\, λ= e D, R_λ= 2 D. gathered (F.97) Here D∗D_* is the deterministic comparator envelope from Appendix F.3: its kernel weights are replaced by kmin=3/(2N)k_ =3/(2N) and k0H=1/Nk_0H=1/N. Explicitly, with a=1−c/H,β∗=log1+(q−1)a1−a,M=β∗+log2Ve,a=1-c/H, _*= 1+(q-1)a1-a, M= _*+ 2Ve, the value used in (F.97) is D∗(e/2)=2N3(5q+V+1)M2+2Nq(q−1)β∗2.D_*(e/2)= 2N3(5q+V+1)M^2+2Nq(q-1) _*^2. Thus these choices use neither realized lag frequencies nor the hidden mechanism nor any test score. We check the horizon substitution explicitly. For H≥5H≥ 5, the above choices give h h ≥h04H2, ≥ h_04H^2, e e ≥h012H3, ≥ h_012H^3, β∗ _* ≤log(qH/c), ≤ (qH/c), D¯ D ≤3CqH(L0+4logH)2≤75CqL02H3, ≤ 3C_qH(L_0+4 H)^2≤ 75C_qL_0^2H^3, λh λ h ≥h023600CqL02H8. ≥ h_0^23600C_qL_0^2H^8. In the source-tag estimate, ρc≤1/2 _c≤ 1/2 and μF=2q3(aρc)H−4V3/2K,64KμF≤2048Kq3V3/22−H. _F=2q^3(a _c)^H-4V^3/2 K, 64 K\, _F≤ 2048Kq^3V^3/22^-H. For H≥128H≥ 128, H8≤2H/2H^8≤ 2^H/2; moreover, (F.96) gives BH≤2H/2B_H≤ 2^H/2. Hence 2H≥BHH82^H≥ B_HH^8, which proves μF≤λh64K. _F≤ λ h64 K. The constant BHB_H contains no H. In particular, this step does not require a source tolerance exponentially small in the horizon. The equality choice of h also gives both feedback requirements h≤1220nEexp,h≤ξDACpop,Cpop=256n2AfbEexp.h≤ 12^20nE_exp, h≤ ξD_AC_pop, C_pop=256n^2A_fbE_exp. ∎ Lemma F.32 (Joint finite widths). Use the horizon and tolerances of Lemma F.31 and the failure allocations (F.95). The dimensions selected below, culminating in (F.99), meet the encoder, coefficient-Gram, row-envelope and uniform-geometry requirements of Appendices E, F.16 and F.19. They also satisfy γ≤εsrc/16γ≤ _ src/16, γ≤γgeoγ≤ _ geo and the Gram part of (F.37). All dimension choices are made without self-reference. Proof. Define the feature dimensions and feedback precisions before selecting the residual width: p p =NV, =NV, kaug k_aug =p+1, =p+1, ssym s_sym =kaug(kaug+1)/2, =k_aug(k_aug+1)/2, k k =p+ssym+1, =p+s_sym+1, G G =n2Afb, =n 2A_fb, εg _g =ξ/DA, =ξ/D_A, τ τ =ξ/G. =ξ/G. For completeness, instantiate Lemma F.19 with k,G,ϵf,τk,G, _f,τ, and δgeo _geo. Its explicit constants are K3 K_3 =8[k(k+2)]3/4δgeo, = 8[k(k+2)]^3/4 _geo, Lpert L_pert =6kϵf+GK3ϵf2, = 6k _f+ GK_3 _f^2, γgeo _geo =min1/8,τ/(16Lpert), = \1/8,τ/(16L_pert)\, ageo a_geo =τ/2, =τ/2, Tgeo2 T_geo^2 =max1,64k(k+2)ϵfageoδgeo, = \1, 64k(k+2) _fa_geo _geo \, hgeo h_geo =(ageoϵf216Tgeo3)2, = ( a_geo _f^216T_geo^3 )^2, Dgeo D_geo =k(k+1)/2, =k(k+1)/2, geo _geo =(1+2kG2hgeo)Dgeo. = (1+ 2 kG^2h_geo )^D_geo. The sufficient width is dgeo=⌈max32γgeo−2[klog9+log16δgeo],32k2Tgeo4ϵf2ageo2log16k2geoδgeo⌉.d_geo= \32 _geo^-2 [k 9+ 16 _geo ],\, 32k^2T_geo^4 _f^2a_geo^2 16k^2N_geo _geo \ . (F.98) All these quantities are fixed before d. We next recall the dimension-independent source envelopes, retaining Asrc=12A_src=12: m¯ m =2max1,log(2V/δ0), = 2 \1, (2V/ _0)\, C¯ C =22k+4log(2/δ0)+4, =2 2k+4 (2/ _0)+4, c¯h c_h =2KC¯+3, =2 K\, C+3, g¯ g =1+c¯h+maxλm¯,c¯h+λKmom/u, =1+ c_h+ \λ m, c_h+λ K_mom/u\, f¯ f =logV+2AsrcVm¯+λV2m¯2, = V+2A_src V\, m+ λ V2 m^2, r¯ r =2(2f¯+1)λ, = 2(2 f+1)λ, ρ ρ =λh64K, = λ h64 K, Bdim B_dim =max16,1024g¯(Asrcr¯+2)ρ,128r¯h. = \16, 1024 g(A_src r+ 2)ρ, 128 rh \. The source formulas below satisfy, for t=1+logdt=1+ d, R(d)≤r¯t,εfeat,src(d)≥16Bdimt.R(d)≤ r t, _feat,src(d)≥ 16B_dimt. Put RW(d) R_W(d) =R(d)+SKmomq+1, =R(d)+SK_mom q+1, r¯W r_W =r¯+SKmomq+1, = r+SK_mom q+1, εf,0 _f,0 =ξ2DAn2, = ξ2D_An 2, ε¯f _f =minεf,0,h/(32r¯W), = \ _f,0,h/(32 r_W)\, εf(d) _f(d) =minεf,0,h/(32RW(d)), = \ _f,0,h/(32R_W(d))\, εsrc(d) _src(d) =minεfeat,src(d),εf(d)/4, = \ _feat,src(d), _f(d)/4\, Bjoint B_joint =maxBdim,64/ε¯f. = \B_dim,64/ _f\. Here ϵf _f denotes Adam’s feedback denominator constant, whereas εf _f denotes feature approximation error. Since RW(d)≤r¯WtR_W(d)≤ r_W t, we have εf(d)≥ε¯f/t,εsrc(d)≥16Bjointt. _f(d)≥ _f/ t, _src(d)≥ 16B_jointt. Consequently, using logd≤16d1/16/e d≤ 16d^1/16/e, the bound d≥[Bjoint(1+16/e)]16d≥[B_joint(1+16/e)]^16 ensures d−1/8≤εsrc(d)/16d^-1/8≤ _src(d)/16. Choose the head count and concentration constants by Rhead R_head =⌈8Nlog2Nδbias⌉, = 8N 2N _bias , Dhead D_head =220N[Vlog9+log8Nδenc], =2^20N [V 9+ 8N _enc ], DΓ D_ =max219[plog9+log8δenc], 4096[klog9+log8δffn],Dhead. = \2^19 [p 9+ 8 _enc ],\,4096 [k 9+ 8 _ffn ],\,D_head \. Define dreq=max[Bjoint(1+16/e)]16,[64KK(Rλ+1)λh]8,γgeo−8,dgeo,128[klog9+log4δ0],DΓ4/3,d=Rhead⌈dreqRhead⌉,γ=d−1/8. splitd_req= \&[B_joint(1+16/e)]^16,\, [ 64 K\,K(R_λ+1)λ h ]^8,\, _geo^-8,\,d_geo,\\ &128 [k 9+ 4 _0 ],\,D_ ^4/3 \, d=R_head d_reqR_head , γ=d^-1/8. split (F.99) Every quantity on the right of dreqd_req is independent of d. The condition d≥DΓ4/3d≥ D_ ^4/3 is precisely the substitution of γ=d−1/8γ=d^-1/8 into d≥DΓγ−2d≥ D_ γ^-2. It supplies the actual encoder and full-feature Gram bounds with encoder tolerance γ/16γ/16. In particular the DheadD_head term ensures the head-width bound while preserving the architecture’s integer relation khead=d/Rhead.k_head=d/R_head. After fixing d, choose the FFN width by Jff J_f =ssymlog9+log(8/δffn), =s_sym 9+ (8/ _ffn), Aff A_f =1+γ−2Jff, =1+γ^-2J_f, Lff L_f =logeAffkaugδffnγ, = eA_fk_aug _ffnγ, mffn m_ffn =⌈250Aff2(kaug+Lff)8⌉. = 2^50A_f^2(k_aug+L_f)^8 . ∎ Lemma F.33 (Positive rates and sampled implementation). At the dimensions of Lemma F.32, the finite temperature, batch sizes and strictly positive block rates below meet the hypotheses of Propositions F.14 and F.30. On their good events, source stopping succeeds, the total backbone displacement across both stages is at most one, and maxt≤2‖Xt−Yt‖F≤E∗/2 _t≤ 2\|X_t-Y_t\|_F≤ E_*/2. Source and feedback retain the exact bias-corrected moment and stopping rules specified below. Proof. At the now chosen width, the source constants of Appendix F.3 are explicitly M0 M_0 =2log(2Vd/δ0), = 2 (2Vd/ _0), Wmax W_ =VM0, = VM_0, Crow C_row =22k+4log(2d/δ0), =2 2k+4 (2d/ _0), Ch C_h =2KCrow+3, =2 KC_row+3, Gc G_c =Ch+λmaxM0,Ch/λ+Kmom/u, =C_h+λ \M_0,C_h/λ+K_mom/u\, λp _p =(Gc+1)−1, =(G_c+1)^-1, Ls L_s =2K+1, =2K+1, Finit F_init =logV+2AsrcWmax+λWmax2/2, = V+2A_srcW_ +λ W_ ^2/2, R R =2(2Finit+1)/λ, = 2(2F_init+1)/λ, rgoal r_goal =ρ/4, =ρ/4, ηs _s =min1,λpu3/28Ls,1λprgoal2, = \1, _pu^3/28L_s, 1 _pr_goal^2 \, b b =λpρ/32, = _pρ/32, Ts T_s =max1,⌈4Finitηsλprgoal2⌉, = \1, 4F_init _s _pr_goal^2 \, εfeat,src _feat,src =min1,b2(AsrcR+2),h8R, = \1, b2(A_srcR+ 2), h8R \, Bs B_s =⌈8VdAsrc2b2log2VdTsδbatch⌉. = 8VdA_src^2b^2 2VdT_s _batch . These formulas also define the width-dependent functions used above; their dimension-independent envelopes were fixed first. Source Adam starts with fresh zero moments, uses physical denominator offset 1/d1/ d, and uses head rate ηs/d _s/ d. At every inspected iterate it computes the complete-record minibatch head gradient, including the coupled term λWλ W. It stops before an update when that gradient’s Frobenius norm is at most ρ/2ρ/2. A failed TsT_sth check declares failure without another update. On the joint good event, the source result proves that this failure cannot occur and that the output has the required uniform source logit accuracy. The norm criterion uses only source observations. All dimensions are fixed before drawing initialization. Let PtotalP_total be the number of initialized scalars and PψP_ψ the number of supplied backbone scalars. Define Minit M_init =max1,2Ptotallog(2Ptotal/δnorm), = \1, 2P_total (2P_total/ _norm) \, Z0 Z_0 =Minit+RheadMinit3, =M_init+ R_headM_init^3, zraw z_raw =minεsrc82,14d, = \ _src8 2, 14 d \, ℓ =min12N,zraw2RheadMinit3[1+2Minit3(Z0+1)], = \ 12N,\, z_raw2 R_headM_init^3[1+2M_init^3(Z_0+1)] \, gbias g_bias =δbiasRheadN2, = _biasR_headN^2, tatt t_att =mingbias4Minit4,gbias2log((N−1)/ℓ)>0. = \ g_bias4M_init^4, g_bias2 ((N-1)/ ) \>0. Thus the implemented attention has a finite positive temperature. For the radius-one derivative bound, write M1=Minit+1M_1=M_init+1 and set Z1 Z_1 =M1+RheadM13, =M_1+ R_headM_1^3, Dz D_z =1+Rhead[3M12+M13N(4M13+tatt−1)], =1+ R_head[3M_1^2+M_1^3N(4M_1^3+t_att^-1)], Draw D_raw =Dz[1+2M13(Z1+1)]+3M12(Z1+1)2+1, =D_z[1+2M_1^3(Z_1+1)]+3M_1^2(Z_1+1)^2+1, D D =Draw,base member,22Draw+2K,normalized gain member. = casesD_raw,&base member,\\ 2 2D_raw+ 2K,&normalized gain member. cases The Gaussian parameter envelope and gain initialization are included in this radius-one bound. In the source precision choices use the capped εsrc _src, and put rψ r_ψ =min1,1D,εsrc2D,1Dd, = \1, 1D, _src2D, 1D d \, ρback _back =min1,rψd2ηsTsKmomPψ>0. = \1, r_ψ d2 _sT_sK_mom P_ψ \>0. Every source backbone coordinate uses the physical rate ηsρback/d _s _back/ d. Its total source displacement is at most rψ/2≤1/2r_ψ/2≤ 1/2, and the source feature error is at most εsrc≤εf/4 _src≤ _f/4. At feedback, remove the source regularizer and reset all first and second moments. Choose the positive common slow multiplier ρR=min1,h32AfbSKmomV,d/PψSKmommin12,εf2D,h32DRW. _R= \1,\, h32A_fbSK_mom V,\, d/P_ψSK_mom \ 12, _f2D, h32DR_W \ \. (F.100) Use physical rate ηt/d _t/ d for fast control-output rows and ηtρR/d _t _R/ d for every slow supplied parameter, at t=1,2t=1,2. Every feedback block uses physical denominator offset ϵf/d _f/ d. Indeed, (F.100) gives ρRSKmomV _RSK_mom V ≤h/(32Afb), ≤ h/(32A_fb), ρRSKmomPψ/d _RSK_mom P_ψ/d ≤min1/2,εf/(2D),h/(32DRW). ≤ \1/2, _f/(2D),h/(32DR_W)\. Source and feedback backbone displacements total at most one, so the same D remains valid during both phases. The bound RWR_W includes physical readout components in the nullspace of the feature operator. These choices supply total feature error at most εf _f and base, slow-row, and feature-motion logit discrepancy at most 2h2h; fast coefficients are controlled by the feedback comparison. Each of the two feedback updates averages complete on-policy trajectories using batch size BR=⌈8qkn2Afb2εg2log4qkδR⌉.B_R= 8qkn^2A_fb^2 _g^2 4qk _R . (F.101) Zero-reward trajectories and all token-score terms are retained. The baseline is zero. The history-sensitive comparison in Appendix F.4 has per-step forcing at most 4ξ4ξ and therefore maxt≤2‖Xt−Yt‖F≤4SQampξ=E∗/2. _t≤ 2\|X_t-Y_t\|_F≤ 4SQ_ampξ=E_*/2. Both nonzero Adam histories participate in the second update. The width and precision choices control the current random denominator before coefficient projection, through the uniform geometry of Lemma F.19. ∎ Lemma F.34 (Joint probability and held-out capability). For every fixed unknown world, the construction of Lemmas F.31, F.32 and F.33 succeeds with probability at least 1−δ1-δ. On this event, simultaneously for all held-out prompts, each source metric E,M,SE,M,S is at most 1/q+2nh1/q+2nh, each final metric exceeds 7/87/8, and each gain exceeds 3/43/4. The final grammar-or-public-coordinate violation probability is at most 9c/8+1/1289c/8+1/128. Proof. The bias, encoder, FFN/full-Gram, and parameter-envelope failures total 4δ/164δ/16. The row and readout-coordinate failures total 2δ/162δ/16. Uniform geometry, source batches, and the two feedback batches contribute 3δ/83δ/8. Thus the union bound is 4δ/16+2δ/16+3δ/8=3δ/4<δ.4δ/16+2δ/16+3δ/8=3δ/4<δ. The encoder’s four internal allocations total δenc _enc and the FFN’s two internal allocations total δffn _ffn. The row envelope is conditional on coverage and the actual Gram event; the geometry event is uniform over histories. These facts suffice for the union bound without assuming independent realized feature rows. Gaussian initialization is not conditioned on later content-score events when its independence lemmas are applied. For every fixed unknown world, fresh batches obey the stopped conditional concentration estimates. This is not a simultaneous guarantee for one dataset across all unknown worlds. The canonical source selects a uniform operation. Sharp transitivity makes its first state uniform, and subsequent doubly stochastic noisy-permutation kernels keep its endpoint uniform. Its valid-class endpoint success and first-state retention are consequently 1/q1/q, and its task utility is at most 1/q1/q. Full-vocabulary and actual-logit coupling bounds each of the three actual source metrics by 1/q+2nh.1/q+2nh. Writing roppr_opp for the opposite arm’s reward, the separate exact identity is r∗+ropp=Dexec≤1r_*+r_opp=D_exec≤ 1; the source uniformity conclusion uses the task structure, rather than the individual reward upper bounds. After the two reference updates, the correct choice has probability at least 15/1615/16. Reference task utility and first-state retention are at least 1516(1−9c8)=75458192. 1516 (1- 9c8 )= 75458192. The coefficient comparison contributes choice-logit error at most AfbE∗=1/1024A_fbE_*=1/1024. The base, format, and state contribution is n(8h+hEexp)≤9/220,n(8h+hE_exp)≤ 9/2^20, and the choice-distribution contribution is at most 1/2561/256. Their total deployment-law discrepancy is strictly less than 1/1281/128. Hence all three actual metrics E,S,ME,S,M exceed 7/87/8, and each exceeds its corresponding source value by more than 3/43/4. The probability of a grammar or public-coordinate violation is at most 9c8+1128. 9c8+ 1128. The bound uses the original verifier, which checks grammar, public-coordinate consistency and the endpoint. In the reference, only the correct arm and its hidden-bit opposite can receive reward, including histories whose private-state errors repair. ∎ Proposition F.35 (Polynomial implementation and exact query accounting). For the ordered construction of Lemmas F.31, F.32 and F.33, the dimensions, batch sizes, iteration bound, inverse temperature and reciprocals of all positive physical rates have polynomial real-arithmetic bounds in expanded q,H,δ−1q,H,δ^-1, up to logarithms. Source consumes at most BsTsB_sT_s records and NBsTsNB_sT_s targets, with at most Ts−1T_s-1 updates and no verifier queries. Feedback consumes exactly 2BR2B_R fresh prompts and terminal queries, generates 2nBR2nB_R decisions, and makes two updates. Proof. We make the resource dependence explicit, including the late choices of temperature and positive rates. With fixed optimizer parameters, η1 _1 is constant, η2=O(logq) _2=O( q), and Rref=O(logq)R_ref=O( q). In fact Eexp=exp(2Afb(Cmoveη1+1)+4)[15(q−1)]2AfbCmove/C0.E_exp= \! (2A_fb(C_move _1+1)+4 )[15(q-1)]^2A_fbC_move/C_0. This is a fixed power of q. Moreover DA=O(q)D_A=O(q), Qamp=O(q2logq)Q_amp=O(q^2 q), and ξ−1=O(q2(logq)2)ξ^-1=O(q^2( q)^2), so h0−1h_0^-1 is polynomial in q up to logarithms. Thus BHB_H is explicit and the minimum horizon in (F.96) is finite. For any admitted H, h−1=(H+2)2h0,e−1≤12H3h0,λ−1≤900CqL02H6h0.h^-1= (H+2)^2h_0, e^-1≤ 12H^3h_0, λ^-1≤ 900C_qL_0^2H^6h_0. These bounds make the dimension-independent source envelopes polynomial in the expanded variables q,H,δ−1q,H,δ^-1, up to logarithms. The geometry net does not introduce an exponential width: it enters only through loggeo=Dgeolog(1+2kG2hgeo). _geo=D_geo (1+ 2 kG^2h_geo ). All other geometry constants are powers, products, or positive reciprocals of already chosen quantities. Thus dgeod_geo, BjointB_joint, and dreqd_req are polynomial in those expanded variables. Rounding adds less than RheadR_head to dreqd_req. The number of heads is polynomial, and khead=d/Rheadk_head=d/R_head is integral. Since γ−2=d1/4γ^-2=d^1/4, Aff=1+d1/4Jff,mffn=O((1+d1/4Jff)2[kaug+logd+log(δffn−1)]8),A_f=1+d^1/4J_f, m_ffn=O\! ((1+d^1/4J_f)^2[k_aug+ d+ ( _ffn^-1)]^8 ), with the explicit multiplier 2502^50 given above. The complete architecture of Appendix E has polynomially many scalars in these dimensions, hence PtotalP_total, PψP_ψ, and MinitM_init are polynomial up to logarithms. The source cap gives εsrc−1≤Bjoint(1+logd)/16 _src^-1≤ B_joint(1+ d)/16. Therefore zraw−1z_raw^-1 and ℓ−1 ^-1 are polynomial as well. In particular tatt−1=max4Minit4gbias,2log((N−1)/ℓ)gbiast_att^-1= \ 4M_init^4g_bias, 2 ((N-1)/ )g_bias \ is polynomial. The displayed derivative formula is a fixed-degree polynomial in M1M_1, N, Rhead R_head, and tatt−1t_att^-1; consequently D is polynomial for both members. Finally, the displayed formulas make ηs−1 _s^-1, TsT_s, BsB_s, and BRB_R polynomial. For the late backbone rate, rψ−1=max1,D,2D/εsrc,Dd,ρback−1=max1,2ηsTsKmomPψ/drψ,r_ψ^-1= \1,D,2D/ _src,D d\, _back^-1= \1, 2 _sT_sK_mom P_ψ/dr_ψ \, and (F.100) similarly gives a polynomial bound for ρR−1 _R^-1. Multiplying by d d and the reciprocals of ηs,η1,η2 _s, _1, _2 proves the same statement for every positive physical learning-rate reciprocal. Neither D nor these late rates occurs on the right side of the residual-width choice. This ordered substitution gives a polynomial real-arithmetic bound in expanded q,H,δ−1q,H,δ^-1, up to logarithms, with the absolute exponents specified above. The source phase uses at most BsTsB_sT_s fresh complete raw records and NBsTsNB_sT_s predicted next-token observations. On every outcome it performs at most Ts−1T_s-1 optimizer updates and TsT_s gradient-norm checks, and makes zero task-verifier calls. The feedback phase uses exactly 2BR2B_R fresh prompts, containing 2HBR2HB_R prompt tokens, and exactly 2BR2B_R terminal verifier calls. It generates 2nBR2nB_R unrestricted tokens and performs exactly two optimizer updates. There are 2HBR2HB_R designated state positions, 2BR2B_R designated control positions, and 2BR2B_R designated EOS positions. These are position counts: the full-vocabulary sampler can emit an incorrect type at any position, so each emitted-type count is bounded by 2nBR2nB_R unless measured from the actual sequence. There is no test-time verifier, reward filtering, source-success labeling, or feedback checkpoint selection. Appendix D compares the corresponding learning routes and resources. ∎ Proposition F.36 (Frozen-source binding identification). On the source event established in Proposition F.14, with the joint tolerances (F.97), externally fixing the first control and greedily decoding the frozen source produces its exact trajectory and EOS. Binding enumeration therefore uses at most q−1q-1 verifier queries and no additional gradient updates. Its generation, forward-evaluation, padding and storage costs are as detailed below. Proof. On the same source event, the canonical correct-state probability at every legal prefix is at least (1−h)(1−9c/(8H))(1-h)(1-9c/(8H)). The actual source logit error is below h/2h/2, so its probability is at least (1−h)(1−9c/(8H))−h>1/2(1-h)(1-9c/(8H))-h>1/2 under (F.97). EOS also has probability above 1/21/2. Thus externally forcing any candidate first control and greedily decoding the frozen source model yields its exact state trajectory and EOS, including on unseen words. By sharp transitivity, a training query accepts exactly when that candidate equals the unknown binding. Enumerate at most q−1q-1 candidates on fresh training prompts and infer the last if all reject. This uses at most q−1q-1 verifier calls, n(q−1)n(q-1) generated symbols, H(q−1)H(q-1) prompt tokens, and (H+1)(q−1)(H+1)(q-1) neural forward evaluations, with zero further gradient updates. Store the binding index and an enumeration counter in addition to the model and its retained context. Deployment requires H+1H+1 forwards and n generated symbols with no verifier. This comparator retains the original source cost. Padding is separate from observed information. Each source record starts with N PAD tokens; separate full-prefix evaluation of its N targets uses N2N^2 context-token occurrences, including N(N+1)/2N(N+1)/2 PAD occurrences. Each feedback trajectory uses nNnN context-token occurrences and H prompt symbols. Each verifier performs H private transitions and H public-coordinate checks. Source stopping uses an already-computed head gradient norm, without extra data or verifier access. The model stores PtotalP_total real parameters and the corresponding optimizer moments. These costs use real arithmetic and real-valued parameter storage. ∎ Proposition F.37 (Opaque-table recovery and binding selection). Under the raw Sequential source law with H≥5H≥ 5 and 0<a<10<a<1 and 0<δtab<10< _ tab<1, the estimator using (F.102) recovers all q2q^2 opaque transition-table rows with probability at least 1−δtab1- _ tab. Its charged source-token count is at most 6M0+2H+26M_0+2H+2. On recovery, at most q−1q-1 fresh-prompt verifier queries identify the binding and yield perfect deployment utility and public-coordinate compliance, with no gradient updates or test-time oracle. Proof. Under the raw source law, execution controls U1,…,UH−1U_1,…,U_H-1 are jointly iid uniform: U1U_1 is independent, and U2,…,UH−1U_2,…,U_H-1 form a strict subset of the tag-conditioned suffix. Retain odd execution transitions through H−1H-1 and average the intervening even controls. Their inputs become independent uniform states because q−1∑U∈TU=Πq^-1 _U T_U= . Thus a complete record supplies ⌊H/2⌋ H/2 iid triples in this marginal retained experiment. Take M0=⌈8q2a−2log2q3δtab⌉,nrec=⌈M0⌊H/2⌋⌉.M_0= 8q^2a^-2 2q^3 _ tab , n_ rec= M_0 H/2 . (F.102) The charged raw-token count satisfies Nnrec=(2H+2)nrec≤6M0+2H+2.Nn_ rec=(2H+2)n_ rec≤ 6M_0+2H+2. The empirical mode for every opaque state/control row recovers all q2q^2 successors with probability at least 1−δtab1- _ tab. This estimator requires neither a coordinate chart nor downstream goal labels. For completeness, let M=nrec⌊H/2⌋M=n_ rec H/2 be the number of retained iid triples and put t=log(2q3/δtab)t= (2q^3/ _ tab). Each row count has mean μ=M/q2≥8a−2tμ=M/q^2≥ 8a^-2t. Binomial lower-tail concentration bounds the probability of fewer than μ/2μ/2 observations by e−μ/8e^-μ/8. Conditional on n observations of a row, true-minus-competitor indicator differences lie in [−1,1][-1,1] and have mean a. Their centered log moment generating function is at most λ2/2λ^2/2, so an incorrect empirical mode, including a tie, has probability at most e−na2/2e^-na^2/2. Union bounding the q2q^2 row counts and q2(q−1)q^2(q-1) comparisons gives failure probability at most δtab _ tab. This proves recovery with full- record rounding. It does not assume independence of all H transitions within a record. On recovery, enumerate candidate bindings on separately drawn fresh uniform training prompts. Generate each candidate’s exact path using the recovered table. The binary terminal reward equals one exactly when that candidate is η: suffix invertibility and sharp transitivity of the first transition prove the equivalence. Test at most q−1q-1 candidates, stopping at success or inferring the last binding if all reject. No canonical-prompt query selection is needed. The additional costs are at most q−1q-1 verifier calls, (H+2)(q−1)(H+2)(q-1) generated symbols, and H table lookups per candidate or deployment. There are zero gradient updates and no test-time oracle. Final storage is q2q^2 successor indices and one binding index; training may use q3q^3 counts. Deployment has perfect utility and public-coordinate compliance on the recovery event. This comparator uses the same observations with a table-based computation, separating information requirements from the neural training route. The 2H+22H+2 granularity term and every unused raw token remain charged. ∎ Proof of Theorem F.1. Fix q=22r+2q=2^2r+2, r≥1r≥ 1, c=1/64c=1/64 and δ∈(0,1)δ∈(0,1) as in the theorem. Apply Lemma F.31 to select H and (h,e,λ)(h,e,λ), then Lemma F.32 to choose the integer head, residual and FFN dimensions. These choices satisfy the source precision, signed-gradient, Gram and geometry inequalities before initialization is drawn. For either architecture member, Lemma F.33 supplies the appropriate derivative bound, finite temperature and positive physical rates. The coupled ridge term and source moments are retained throughout the source stage; the penalty is removed and both moments are reset only at the feedback boundary. By Proposition F.14, on the source events the observable stopping test succeeds within TsT_s inspections and at most Ts−1T_s-1 actual updates. Its uniform logit bound applies to the legal-pattern prefixes needed for freely generated trajectories. The same parameter choices satisfy the sampling, motion and history assumptions of Proposition F.30, so the two actual sampled updates stay within E∗/2E_*/2 of the reference in Lemma F.24. This comparison uses full reward-weighted trajectories and the current adaptive denominators. The union bound and metric conversion are exactly those in Lemma F.34: the allocated failures sum to 3δ/4<δ3δ/4<δ, and, on the resulting event, the source upper bound, final 7/87/8 lower bounds, 3/43/4 gains and violation bound hold simultaneously over all held-out prompts. Substituting εlogit=h _ logit=h gives the four displayed assertions of the theorem. Finally, Proposition F.35 gives Npre≤NBsTsN_ pre≤ NB_sT_s, Q=2BRQ=2B_R, Ngenerated=2nBRN_ generated=2nB_R, at most Ts−1T_s-1 source updates and exactly two feedback updates, with N=2H+2N=2H+2 and n=H+2n=H+2. This establishes every stated resource count for the same training construction. ∎ F.6 Output format and regularized source learning The source theorem uses a regularizer to select a structured optimum. The following obstruction explains why accurate conditional execution and a small unregularized source gradient would not suffice. It concerns population risk in the canonical coefficient class, rather than the trajectory selected by neural Adam. Proposition F.38 (A source near-minimizer with incorrect held-out format). Use the raw source law with H≥5H≥ 5 and 0<a<10<a<1. Let fΘ(x)=ΘΦ(x)f_ (x)= (x) contain all positively weighted categorical singleton and pair features, with ∥Φ(x)∥=K (x) = K, K=12K=12, and let F+F_+ be population source CE. Fix a forbidden suffix w=(w2,…,wH)w=(w_2,…,w_H). For every ε>0 >0 and δf∈(0,1) _f∈(0,1), there is a finite Θ such that F+(Θ)−infΘ′F+(Θ′)≤ε,∥∇F+(Θ)∥F≤ε,F_+( )- _ F_+( )≤ , ∇ F_+( ) _F≤ , all within-state conditional predictions equal the noisy true transitions, but EOS has probability at least 1−δf1- _f at the first-state prediction on w, uniformly over the initial state and preceding chosen control. Proof. Let Pl=xl=PADP_l=1\x_l=PAD\. The polynomial D1=PH+1−PHD_1=P_H+1-P_H is one exactly when the raw prefix length is H+1H+1, the first-state target, and zero at every other source target. At that stage UjU_j lies at lag H+1−jH+1-j. Define dw(x)=∑j=2H(1−xH+1−j=wj),ΔfEOS(x)=M(12−dw(x))D1(x).d_w(x)= _j=2^H (1-1\x_H+1-j=w_j\ ), f_ EOS(x)=M ( 12-d_w(x) )D_1(x). (F.103) This is a degree-two categorical function. Every training suffix differs from w, so dw≥1d_w≥ 1 and the change suppresses the incorrect EOS logit by at least M/2M/2 wherever it acts. On w it raises that logit by M/2M/2. Choose finite base logits with exact noisy state kernels, uniform initial-state and control conditionals, and a format separator whose leakage is at most ζ. The separator in (F.13), with sufficiently large finite amplitude, provides this property even at the actual EOS, where the state interaction remains active. The resulting within-class population objective has zero gradient in every coefficient. State predictions equal their true conditional laws; the initial state and early controls are uniform. At the last suffix control UHU_H, the total tag constrains one bit, but any degree-two prefix feature retains at most two of the H−2≥3H-2≥ 3 preceding suffix controls. An unused independent balanced tag annihilates the target-feature correlation. Uniform control logits therefore have zero population gradient at this target as well. Convexity implies the within-class minimum H0=(H+1)logq+Hhq(a)N.H_0= (H+1) q+Hh_q(a)N. Full CE is that objective plus nonnegative class loss, and finite separators approach zero class loss. Hence infF+=H0 F_+=H_0. The unrestricted conditional entropy is lower by log2/N 2/N; the coefficient-class infimum is not being identified with it. The perturbation changes no correct-class conditional law and only reduces source class leakage. Consequently F+(Θ)≤H0−log(1−ζ),∥∇F+(Θ)∥F≤2Kζ,F_+( )≤ H_0- (1-ζ), ∇ F_+( ) _F≤ 2 K\,ζ, independently of M. The second bound follows because the full probability vector differs from its within-class counterpart by at most 2ζ2ζ in ℓ1 _1; average the residual outer products over all targets. Taking ζ=min1/2,1−e−ε,ε/(2K)ζ= \1/2,1-e^- , /(2 K)\ supplies the source bounds. At a forbidden first-state prefix, the EOS probability is pEOS=11+R(x)e−M/2,R(x)=∑y≠EOSefy(x)−fEOS(x).p_ EOS= 11+R(x)e^-M/2, R(x)= _y e^f_y(x)-f_ EOS(x). The base logits and the set of initial-state/control combinations are finite, so Rmax<∞R_ <∞. Choose M≥max0,2log[Rmax(1−δf)/δf]M≥ \0,2 [R_ (1- _f)/ _f]\. This proves the uniform EOS bound. Under the fixed-length sampler an EOS token in a state slot is a grammar failure, regardless of subsequent tokens. ∎ The perturbation increases the coefficient norm with M and targets one forbidden suffix. It isolates the role of regularization in extending source prediction to the required output format. Appendix G Contextual acquisition and sampled Adam adaptation G.1 Acquisition, contextual transfer and intervention Use the task and attention–bilinear architecture in Section 5.1, with nctx≥1n_ ctx≥ 1 and odd p≥17p≥ 17. The fixed normalizer is the finite positive mixture in (5.3), with weights summing to one, powers in [0,1][0,1] and offsets in [0,Zmax][0,Z_ ]. Initialize every factor coordinate independently from a standard Gaussian. Use 0≤β1<10≤ _1<1, 0<β2<10< _2<1, β12<β2 _1^2< _2, and the physical step sizes and denominator offsets in Section 5.3, which implement a common normalized update. Retain all parameters between stages and reset only the two Adam moments. Theorem G.1 (Acquisition, contextual transfer and intervention). Fix the task dimensions, an architecture (5.3), Adam parameters (β1,β2)( _1, _2) as above, a world (U,s,τ)(U,s,τ) and δ∈(0,1)δ∈(0,1). Choose width, batches, steps, offsets and input precision as in Sections G.7 and G.8; they are explicit and finite. With probability at least 1−δ1-δ over initialization and all training samples, jointly: 1. Acquisition. Initial source matching is at most 0.510.51; after source training it is at least 0.90.9 uniformly over contexts, frames and queries, and greedy source-bit accuracy exceeds 0.90.9. 2. Transfer. At the final feedback iterate, sampled success is at least 19/2019/20 on every held-out context, query, cue and amplitude pair, an improvement of at least 2/52/5 over the source checkpoint. 3. Intervention. The same schedules and budgets with the initial query/key parameters frozen in both stages, and every head factor trained, end with sampled test success at most 1/2+1/641/2+1/64; the paired gap exceeds 2/52/5. The conclusions hold for the finite coordinate alphabet of Section G.8 with supplied numerical decoding and real-valued optimizer states. At fixed nctx,p,Zmax,mnormn_ ctx,p,Z_ ,m_ norm the budget formulas may be shared across mixture members, with probability evaluated for each fixed member. The complete proof is in Section G.8, using the budgets of Section G.7. G.2 Source learning in the radial class We prove Theorem 5.2 with explicit constants. All core norms below are Frobenius norms, combined by the Euclidean direct sum. The row coordinates are the normalized parameters of Section 5.1. Time derivatives in this and the next section describe a deterministic reference; Section G.6 constructs that reference and couples the actual finite Adam runs to this analytical reference. Lemma G.2 (Source symmetry and Gaussian reduction). Under the source experiment of Section 5.1, consider the unique self-consistent reference from independent standard-Gaussian factor rows, with coordinatewise update FϵF_ε as defined below. Its attention core has form M=βUM=β U, and its head has only C12=C21=t/2C_12=C_21=t/2 in the source-active content block. For β≥0β≥ 0, the pooled content has law h=SGh= SG and conditional label mean mρ(G1)mρ(G2)m_ρ(G_1)m_ρ(G_2), with S,ρ,mρS,ρ,m_ρ defined in the proof. The population head gradient has no entries outside the same two positions. Existence and uniqueness of this reference are supplied by Lemma G.26. Proof. At a source core M=βUM=β U, β≥0β≥ 0, write a=σ(β),S=a2+(1−a)2,ϱ=a2/S,ρ=ϱ.a=σ(β), S=a^2+(1-a)^2, =a^2/S, ρ= . Then Sβ/S=2(ϱ−a)S_β/S=2( -a) and ϱβ=2ϱ(1−ϱ) _β=2 (1- ). Gaussian conditioning gives a pooled content vector h=SGh= SG, G∼N(0,Ip)G N(0,I_p), and [y∣G]=mρ(G1)mρ(G2),mρ(z)=2Φ(ρz1−ρ2)−1.E[y G]=m_ρ(G_1)m_ρ(G_2), m_ρ(z)=2 \! ( ρ z 1-ρ^2 )-1. Here Φ and ϕφ denote the standard Gaussian cumulative distribution function (CDF) and density. Source sign and permutation symmetries force the reference head core to have only C12=C21=t/2C_12=C_21=t/2 in its p-dimensional content block. The indices in this subsection refer to that block. Flipping an inactive value coordinate and its two factor coordinates preserves the source law and kills entries with exactly one such index. Negating o together with both first-channel factors preserves source alignment and kills diagonal and inactive–inactive mean entries. Exchanging a,ba,b equates the remaining two entries. Context-factor sign flips apply because source context inputs vanish. These are signed parameter permutations, so uniqueness of the coordinatewise reference preserves the row law. At the resulting core, the population head gradient itself has only its 12,2112,21 entries: the same Gaussian sign symmetries kill the other derivatives individually. This stronger fact will preserve the inactive row factors. ∎ Lemma G.3 (Strict source drift throughout the normalization class). Use the source law and symmetric core of Lemma G.2, with p≥17p≥ 17 and the finite positive mixture (5.3). For t>0t>0 and finite β≥0β≥ 0, the source loss satisfies Lβ<0L_β<0. For each component with α>0α>0, the uncertainty term D has the integral representation (G.2) and the logarithmic derivative (G.3); for α=0α=0, D=(1−ϱ)/πD=(1- )/π and that logarithmic derivative equals 2a2a. Proof. The source logit is f=th1h2ψ(∥h∥2)f=th_1h_2ψ( h ^2), and its loss has the exact decomposition L=log(1+e−|f|)+12[|f|1−ysign(G1G2)].L=E (1+e^-|f|)+ 12E [|f|\1-ysign(G_1G_2)\ ]. (G.1) For one component (α,ζ)(α,ζ) with α>0α>0, put v=ζ/Sv=ζ/S and define D=12[|G1G2|(∥G∥2+v)α1−mρ(G1)mρ(G2)sign(G1G2)].D= 12E [ |G_1G_2|( G ^2+v)^α\1-m_ρ(G_1)m_ρ(G_2)sign(G_1G_2)\ ]. The Gamma integral and elementary Gaussian integration yield D=1−ϱπΓ(α)∫0∞uα−1e−vu(1+2u)−p/21+2u(1−ϱ)u.D= 1- π (α) _0^∞ u^α-1e^-vu(1+2u)^-p/21+2u(1- )\,du. (G.2) Indeed, [Ze−uZ2mρ(Z)]=2/πρ(1+2u)1+2u(1−ρ2).E[Ze^-uZ^2m_ρ(Z)]= 2/πρ(1+2u) 1+2u(1-ρ^2). Multiply this identity for the two active Gaussian coordinates and by (1+2u)−(p−2)/2(1+2u)^-(p-2)/2 for the rest, then subtract from the perfect-sign expression. The Gamma representation follows by substitution in its defining integral; nonnegative sign-adjusted integrands justify Tonelli. Let wE_w integrate under the normalized positive integrand in (G.2), and write Bu=1+2u(1−ϱ)B_u=1+2u(1- ). Integration of the derivative of u times that integrand gives wBu−1=1−α+pwu1+2u+vwu.E_wB_u^-1=1-α+pE_w u1+2u+vE_wu. The origin boundary vanishes for α>0α>0, and the infinity boundary and derivatives are integrable for p≥17p≥ 17, including v=0v=0. Differentiating with vβ=−2(ϱ−a)v_β=-2( -a)v gives −∂β(S1−αD)S1−αD=2[(1−α)a+ϱpwu1+2u+avwu]>0.- _β(S^1-αD)S^1-αD=2 [(1-α)a+ pE_w u1+2u+avE_wu ]>0. (G.3) For α=0α=0 the direct formula is D=(1−ϱ)/πD=(1- )/π, and the ratio is 2a2a; no Γ(0) (0) is used. For every component, the derivative with respect to S of its margin factor is ∂S(S∥G∥2+ζ)α=ζ+(1−α)S∥G∥2(S∥G∥2+ζ)α+1≥0. _S S(S G ^2+ζ)^α= ζ+(1-α)S G ^2(S G ^2+ζ)^α+1≥ 0. The first term of (G.1) therefore cannot increase with β. Its second term is t times the positive weighted sum of S1−αDS^1-αD. Equation (G.3) proves Lβ<0L_β<0 whenever t>0t>0 and β is finite. The argument averages the noise terms componentwise; it does not replace the logistic loss of a mixture by a mixture of logistic losses. ∎ Definition G.4 (Uniform source constants). Set a⋆=19/20a_ =19/20, ϱ⋆=a⋆2/[a⋆2+(1−a⋆)2] _ =a_ ^2/[a_ ^2+(1-a_ )^2], ρ⋆=ϱ⋆ _ = _ , s⋆=ρ⋆/1−ρ⋆2s_ = _ / 1- _ ^2, and PE P_E =ϕ(2)2Pr(|Z|≤1)p−2, =φ(2)^2 (|Z|≤ 1)^p-2, m0 m_0 =2ϕ(1), =2φ(1), m⋆ m_ =2Φ(2s⋆)−1, =2 (2s_ )-1, Amin A_ =PEm022(p+6+2Zmax), = P_Em_0^22(p+6+2Z_ ), Dmin D_ =PE(1−m⋆2)2(p+6+2Zmax), = P_E(1-m_ ^2)2(p+6+2Z_ ), γs _s =minDmin2,p(1−ϱ⋆)e−4Zmax5−p/2−23π. = \ D_ 2, p(1- _ )e^-4Z_ 5^-p/2-23π \. (G.4) Lemma G.5 (Uniform drift and head warmup bounds). Under the assumptions of Lemma G.3, the constants in Definition G.4 are positive. For a≤a⋆a≤ a_ and t≥0t≥ 0, −Lβ≥tγs/2-L_β≥ t _s/2. Put tmin=Amin/8t_ =A_ /8 and ds=3Amin/8d_s=3A_ /8. For all β≥0β≥ 0, Ltt<1L_t<1 and −Lt≥ds-L_t≥ d_s whenever 0≤t≤tmin0≤ t≤ t_ . Proof. These are explicit positive real numbers. On G1,G2∈[1,2]G_1,G_2∈[1,2] with all other coordinates in [−1,1][-1,1], the probability is at least PEP_E and ∥G∥2+ζ/S≤p+6+2Zmax G ^2+ζ/S≤ p+6+2Z_ . Its correlation and remaining uncertainty give Amin,DminA_ ,D_ uniformly until a⋆a_ . For α≤1/2α≤ 1/2, the first term of (G.3) lower bounds the unscaled decrease by Dmin/2D_ /2. For α≥1/2α≥ 1/2, restrict the integral to u∈[1,2]u∈[1,2], and use Γ(α)≤3 (α)≤ 3, v≤2Zmaxv≤ 2Z_ and 1+2u≤51+2u≤ 5 to obtain the second term in γs _s. Since S1−α≥1/2S^1-α≥ 1/2, −Lβ≥tγs/2(a≤a⋆).-L_β≥ t _s/2 (a≤ a_ ). (G.5) The coefficient of t in f is bounded by |G1G2|+1/2|G_1G_2|+1/2, whence Ltt<1L_t<1. At t=0t=0, −Lt≥Amin/2-L_t≥ A_ /2. Thus, with tmin=Amin/8,ds=3Amin/8,t_ =A_ /8, d_s=3A_ /8, we have −Lt≥ds-L_t≥ d_s on 0≤t≤tmin0≤ t≤ t_ . The correlation-event bound also holds beyond a⋆a_ whenever β≥0β≥ 0, which will make tmint_ an invariant lower boundary. ∎ Proposition G.6 (Finite-time source acquisition by the reference). Start the source reference from the Gaussian row law in Lemma G.2. With the constants of Definition G.4 and the offset and horizon in (G.7), its parity coefficient reaches and stays above tmint_ after time SwarmS_ warm. Its correct-slot matching reaches a⋆=19/20a_ =19/20 by SsS_s and remains at least a⋆a_ thereafter. Its attention gradient is (G.6), and every normalized coordinate has speed at most one. Proof. Let Fϵ(z)=z/(|z|+ϵ)F_ε(z)=z/(|z|+ε) act coordinatewise on each row’s scaled population gradient. Align k′=Uk =Uk. The signed-coordinate source symmetries give M=βUM=β U and the head form above. Haar averaging in the active block and uniform contexts give the full gradient ∇ML=Lβ2nctxU. _ML= L_β2n_ ctxU. (G.6) The head gradient has no extra 1/nctx1/n_ ctx: each context has the same head loss. Each aligned query/key pair obeys q˙=Fϵ(χk′),k˙′=Fϵ(χq),χ=−Lβ/(2nctx). q=F_ε(χ k ), k =F_ε(χ q), χ=-L_β/(2n_ ctx). Its product derivative is nonnegative for χ≥0χ≥ 0. Initially q,k′∈[1,2]q,k ∈[1,2] has probability at least ϕ(2)2φ(2)^2; there both remain at least one, and their product derivative is at least one when χ≥ϵχ≥ε. The negative event gives the same contribution. Consequently β˙≥p0:=e−4/π=2ϕ(2)2(χ≥ϵ). β≥ p_0:=e^-4/π=2φ(2)^2 (χ≥ε). The active head coordinates (a1,a2,b1,b2,o)(a_1,a_2,b_1,b_2,o) have alignment t=[o(a1b2+a2b1)]t=E[o(a_1b_2+a_2b_1)]. With d=−Ltd=-L_t, their equations are a˙1 a_1 =Fϵ(dob2), =F_ε(dob_2), a˙2 a_2 =Fϵ(dob1), =F_ε(dob_1), b˙1 b_1 =Fϵ(doa2), =F_ε(doa_2), b˙2 b_2 =Fϵ(doa1), =F_ε(doa_1), o˙ o =Fϵ(d(a1b2+a2b1)). =F_ε (d(a_1b_2+a_2b_1) ). Every alignment contribution has the form uFϵ(du)uF_ε(du). On the event that all five initial entries are in [1,2][1,2], with probability at least p5=ϕ(2)5p_5=φ(2)^5, all remain there or above during warmup, and t˙≥6p5d+ϵ. t≥ 6p_5 dd+ε. At t=0t=0 the direction is positive; at t=tmint=t_ it is strictly inward. Thus t reaches and then stays above tmint_ , even if its derivative later changes sign. Choose ϵs=tminγs4nctx,Swarm=tmin(ds+ϵs)6p5ds,Ss=Swarm+log19p0+1. _s= t_ _s4n_ ctx, S_ warm= t_ (d_s+ _s)6p_5d_s, S_s=S_ warm+ 19p_0+1. (G.7) After warmup, (G.5) implies χ≥ϵsχ≥ _s until a⋆a_ , so matching reaches a⋆a_ by SsS_s. Thereafter β cannot decrease because Lβ<0L_β<0 and t>0t>0. Every block remains trainable, and every normalized coordinate has speed at most one. ∎ Lemma G.7 (Inactive factors and the output sign at the source endpoint). For the reference of Proposition G.6, cross-context attention moments vanish. At the source switch, the nctxn_ ctx context head coordinates and k inactive cue coordinates retain their independent standard-Gaussian law, independently of the active five-coordinate evolution. The output factor o has a symmetric absolutely continuous law and Pr(o≥0)=1/2 (o≥ 0)=1/2. Proof. Sparse context support supplies the block decomposition. Simultaneous query/key sign flips in each block kill cross-context reference moments already at initialization. Their finite deviations remain in the full-core norm of Section G.6; acquired matching is within the supplied blocks. At the switch, the nctxn_ ctx context head coordinates and k inactive cue coordinates retain their independent standard-Gaussian reference law, independent of the active five-coordinate evolution. Their population source gradients vanish; source context sample gradients vanish identically. The transformation (o,a1,b1)↦(−o,−a1,−b1)(o,a_1,b_1) (-o,-a_1,-b_1) preserves source alignment and the coordinatewise flow. Hence o has a symmetric continuous law. Finite-time locally Lipschitz flow preserves absolute continuity, so Pr(o≥0)=1/2 (o≥ 0)=1/2. This entire source-produced law is retained in the feedback proof. ∎ G.3 Contextual feedback from the acquired state Let βs _s be the source contrast at SsS_s. The two source-active content coordinates are zero on all target inputs. Thus the reference source checkpoint has zero target logit without replacing any parameter. The starting law of this stage is exactly the law constructed in Proposition G.6. Lemma G.8 (Invariant feedback cores and population gradients). Run the feedback reference from the source endpoint in Propositions G.6 and G.7, under the contextual task in Section 5.1. The target-active head has form (G.8); the attention blocks have a common contrast parameter λ and fixed sum coefficient βs _s. With c=tanh(λ/2)c= (λ/2), its population success and scalar coefficients are (G.9), and its full core gradients are (G.10). Proof. The target-active head core has the invariant form Cℓi+Ciℓ=Bsℓτi,Ccontext,context=Ccue,cue=0,C_ i+C_i =Bs_ _i, C_ context,context=C_ cue,cue=0, (G.8) where i ranges over cue coordinates. To see this for arbitrary s, conjugate context factors by the diagonal signs s and replace the uniform cue in context ℓ by sℓzs_ z. The quadratic logit is invariant under an overall input sign, reducing to all-positive s. The inactive source factors retain their Gaussian law under these transformations. Context permutations and signed cue permutations equate the aligned cross coefficients. Finally, negating o, both first source-channel factors and every cue factor preserves the source law; on target inputs it maps f(z)f(z) to −f(−z)-f(-z). Labels also change sign with the cue, so this symmetry kills even target blocks and preserves the cross block. All these transformations are signed permutations, under which FϵF_ε is equivariant. Let P−P_- project onto (1,−1)(1,-1) and P+=I−P−P_+=I-P_-. The reference attention blocks are Mℓℓ=(βsP++λP−)Uℓ,c=tanh(λ/2),M_ =( _sP_++λ P_-)U_ , c= (λ/2), with common λ and zero cross-context blocks. A training pool has cue cvzcvz, where v∈±1v∈\± 1\ is the selected slot sign. Define T T =τ⊤z, =τ z, w(c) w(c) =cψ(1+kc2), =cψ(1+kc^2), J(B,c) J(B,c) =σ(Bw(c)|T|), =Eσ(Bw(c)|T|), ET E_T =[|T|σ′(Bw(c)|T|)], =E[|T|σ (Bw(c)|T|)], df d_f =w(c)ET/k, =w(c)E_T/k, gf g_f =(1−c2)Bw′(c)ET/2. =(1-c^2)Bw (c)E_T/2. (G.9) Direct differentiation gives the full, unsymmetrized core gradients (∇CJ)ℓi=(∇CJ)iℓ=dfnctxsℓτi,(∇MJ)ℓℓ=gfnctxUℓP−.( _CJ)_ i=( _CJ)_i = d_fn_ ctxs_ _i, ( _MJ)_ = g_fn_ ctxU_ P_-. (G.10) Cue symmetry makes its aligned gradient proportional to τ; the scalar product with τ is w(c)ETw(c)E_T, giving 1/k1/k. Uniform contexts give 1/nctx1/n_ ctx. The attention factor follows from two-slot softmax, query averaging and dc/dλ=(1−c2)/2dc/dλ=(1-c^2)/2. There is no additional source factor 1/21/2 in gfg_f. ∎ Lemma G.9 (Coordinate dynamics and positive task-rule drift). Under Lemma G.8, the signed head coordinates satisfy (G.11). As long as df>0d_f>0, all contributions to B˙ B are nonnegative and the positive-probability event inherited from Lemma G.7 yields (G.12), with pg=ϕ(2)4/2p_g=φ(2)^4/2. Proof. For a head row define Ac=s⊤acontextnctx,Dc=s⊤bcontextnctx,Xc=τ⊤acuek,Yc=τ⊤bcuek.A_c= s a_ context n_ ctx, D_c= s b_ context n_ ctx, X_c= τ a_ cue k, Y_c= τ b_ cue k. For the output factor, contracting the core gradient with its row feature gives the normalized row gradient go g_o :=⟨∇CJ,ab⊤⟩F := _CJ,ab _F =dfnctx[(s⊤acontext)(τ⊤bcue)+(τ⊤acue)(s⊤bcontext)] = d_fn_ ctx [(s a_ context)(τ b_ cue)+(τ a_ cue)(s b_ context) ] =dfknctx(AcYc+XcDc). =d_f kn_ ctx(A_cY_c+X_cD_c). Summing the coordinatewise equations with their signs therefore yields A˙c A_c =nctxFϵ(dfknctxoYc), = n_ ctxF_ε\! ( d_f kn_ ctxoY_c ), D˙c D_c =nctxFϵ(dfknctxoXc), = n_ ctxF_ε\! ( d_f kn_ ctxoX_c ), X˙c X_c =kFϵ(dfnctxoDc), = kF_ε\! ( d_f n_ ctxoD_c ), Y˙c Y_c =kFϵ(dfnctxoAc), = kF_ε\! ( d_f n_ ctxoA_c ), o˙ o =Fϵ(dfknctx(AcYc+XcDc)), =F_ε\! (d_f kn_ ctx(A_cY_c+X_cD_c) ), B B =[o(AcYc+XcDc)]nctxk. = E[o(A_cY_c+X_cD_c)] n_ ctxk. (G.11) These equations follow by summing the original coordinatewise updates. All contributions to B˙ B have sign dfd_f. At the switch, Ac,Dc,Xc,YcA_c,D_c,X_c,Y_c are independent standard Gaussians, independent of the symmetric sign of o. The event that these four variables are at least one and o≥0o≥ 0 has probability at least pg=ϕ(2)4/2p_g=φ(2)^4/2. These inequalities persist while df>0d_f>0. The o contribution alone implies B˙≥2pgnctxk2dfk/nctx2dfk/nctx+ϵf. B≥ 2p_g n_ ctxk 2d_f k/n_ ctx2d_f k/n_ ctx+ _f. (G.12) The acquired contrast makes the right side positive; with c=0c=0 the training cue disappears. ∎ Lemma G.10 (Feedback attention dynamics). Under Lemma G.8, the aligned query/key contrasts obey (G.13). The sign of λ˙ λ is the sign of gfg_f, the sum coordinates remain fixed, and the attention-core form in Lemma G.8 is preserved. Proof. For attention align k′=Uk =Uk and set u=q1−q2u=q_1-q_2, v=k1′−k2′v=k _1-k _2 within a block. Then u˙ u =2Fϵ(gfv/(2nctx)), =2F_ε(g_fv/(2n_ ctx)), v˙ v =2Fϵ(gfu/(2nctx)), =2F_ε(g_fu/(2n_ ctx)), λ λ =[uv]/2, =E[uv]/2, λ˙ λ =[vFϵ(gfv/(2nctx))+uFϵ(gfu/(2nctx))]. =E [vF_ε(g_fv/(2n_ ctx))+uF_ε(g_fu/(2n_ ctx)) ]. (G.13) Thus λ˙ λ has the sign of gfg_f. Sum coordinates are pointwise constant. Simultaneous coordinate exchange preserves vanishing mixed sum/contrast moments of the source law; their independence is unnecessary. Block sign symmetries and context permutations retain the complete attention-core form. ∎ Lemma G.11 (An invariant contrast region). For the positive mixture (5.3), each component satisfies (G.14) and has nonnegative derivative at cmin=1/kc_ =1/ k. Along the feedback reference started at the acquired source endpoint, the region B≥0B≥ 0, c≥cminc≥ c_ is invariant. Proof. For every component, dcc(1+kc2+ζ)α=1+ζ+(1−2α)kc2(1+kc2+ζ)α+1. ddc c(1+kc^2+ζ)^α= 1+ζ+(1-2α)kc^2(1+kc^2+ζ)^α+1. (G.14) It is nonnegative at cmin=1/kc_ =1/ k. Equations (G.12) and (G.13), the source bound c≥0.9c≥ 0.9, and uniqueness give joint invariance of B≥0B≥ 0 and c≥cminc≥ c_ . Some members decrease contrast above that boundary; actual Adam deviations from the reference barrier will be paid separately. ∎ Lemma G.12 (Finite time to the target coefficient). In the invariant region of Lemma G.11, let wmin,B⋆,μk,dmin,f,ϵf,Sfw_ ,B_ , _k,d_ ,f, _f,S_f be as in (G.15) and its preceding sentence. Then wmin≤w(c)≤1w_ ≤ w(c)≤ 1, |T|/k=μkE|T|/k= _k, and df≥dmin,fd_f≥ d_ ,f until B=B⋆B=B_ . With offset ϵf _f, the reference reaches B≥B⋆B≥ B_ by SfS_f and remains there. Proof. In this region wmin≤w(c)≤1w_ ≤ w(c)≤ 1, where wmin=cmin/(1+k+Zmax)w_ =c_ /(1+k+Z_ ). Put L⋆ L_ =log19+1, = 19+1, B⋆ B_ =2(1+k+Zmax)L⋆cmin, = 2(1+k+Z_ )L_ c_ , μk _k =(k−1(k−1)/2)2k−1, = k-1(k-1)/22^k-1, dmin,f d_ ,f =wminμke−B⋆k/4, =w_ _ke^-B_ k/4, ϵf _f =dmin,fk/nctx, =d_ ,f k/n_ ctx, Sf S_f =3nctxkB⋆4pg+1. = 3 n_ ctxkB_ 4p_g+1. (G.15) Conditioning one sign in the odd Boolean sum, or cancelling paired binomial terms, gives |T|/k=μkE|T|/k= _k. Since |T|≤k|T|≤ k and σ′(z)≥e−z/4σ (z)≥ e^-z/4 for z≥0z≥ 0, we have df≥dmin,fd_f≥ d_ ,f until B reaches B⋆B_ . Then (G.12) gives B˙≥4pg/(3nctxk) B≥ 4p_g/(3 n_ ctxk). Hence B≥B⋆B≥ B_ by SfS_f and remains so. ∎ Lemma G.13 (Held-out margin and pooled-cue sensitivity). At the endpoint supplied by Lemma G.12, every context, query, Boolean cue and pair of amplitudes from ±1,±ρmem\± 1,± _ mem\ has signed reference logit at least L⋆=log19+1L_ = 19+1. For |z|≤1|z|≤ 1, the positive normalizer mixture satisfies the derivative bound (G.16). Proof. For any two amplitudes from ±1,±ρmem\± 1,± _ mem\, let the correct weight be a=(1+c)/2a=(1+c)/2 and the pooled cue multiplier be z0=auselected+(1−a)uotherz_0=au_ selected+(1-a)u_ other. Then sign(uselected)z0 (u_ selected)z_0 ≥aρmem−(1−a)=c−1+c22−mmem≥cmin/2, ≥ a _ mem-(1-a)=c- 1+c22^-m_ mem≥ c_ /2, |z0| |z_0| ≤1. ≤ 1. The signed logit is at least Bcmin/[2(1+k+Zmax)]Bc_ /[2(1+k+Z_ )], since |T|≥1|T|≥ 1. The reference therefore has margin at least L⋆L_ on all sixteen such pairs, including the four disjoint test pairs. The bound |dzzψ(1+kz2)|≤1 | ddz\zψ(1+kz^2)\ |≤ 1 (G.16) holds componentwise and hence for the positive mixture, and will control finite core error. ∎ G.4 The frozen intervention and source-information bound Proposition G.14 (Stationary target prediction under frozen reference attention). Use the same Gaussian initialization law, source and feedback schedules and moment reset as in Theorem G.1, but set query/key updates to zero in both stages. Its population reference has M=0M=0; at the source switch its target-active head coefficients vanish. During feedback every reference head row is stationary, and its test logit remains zero on every amplitude pair. Proof. The frozen branch shares initialization and schedules with the main run but sets query/key updates to zero in both stages. Its reference M=0M=0 is an expectation identity; the actual finite initialization is retained in the coupling. Source head symmetry still leaves only the parity-channel coefficients at uniform retrieval. No source-acquisition conclusion is needed for this branch. On target training records its reference pool is exactly eℓe_ . Conditional labels are balanced, so yσ(yCℓℓ)=1/2,∇CJ=0.E_yσ(yC_ )=1/2, _CJ=0. All target-active reference coefficients vanish at the switch; the target logit is zero for every attention core there, and ∇MJ=0 _MJ=0 as well. After the moment reset, every reference head row is stationary during feedback. Its logit stays zero on every test pair. Section G.7 bounds the finite branch with its random initial attention and trained head. ∎ Proposition G.15 (The error contributed by an unobserved context). In the exact numerical contextual experiment of Theorem 5.1, for NpreN_ pre iid source observations (or a worst-case cap of that many observations), every world-independent learner has worst-world expected test error at least 14(1−1/nctx)Npre 14(1-1/n_ ctx)^N_ pre, regardless of its number of permitted adaptive feedback queries. A missing context contributes conditional error at least 1/41/4 under the independent uniform prior on orientation flips. Proof. For the lower-bound direction of Theorem 5.1, pair worlds by independently flipping any subset of (Uℓ,sℓ)⟼(UℓPswap,−sℓ).(U_ ,s_ ) (U_ P_ swap,-s_ ). Every permitted antipodal training prompt has the same reward for each action in paired worlds. This pointwise equality survives adaptive prompts, randomized policies, parameter updates, arbitrary computation and any number of permitted feedback queries. Source records from other contexts are also unaffected. If context ℓ is absent from NpreN_ pre iid source records, its flip remains uniform under an independent uniform prior on the nctxn_ ctx flips, conditional on the full transcript and supplied τ. Equal-sign test amplitude pairs have opposite paired labels, whereas opposite-sign pairs agree. Equal-sign mass is 1/21/2, so a missing context has conditional test error at least 1/41/4. It is missing with probability (1−1/nctx)Npre(1-1/n_ ctx)^N_ pre. Averaging over uniform test contexts and then bounding the supremum by the prior mean proves the lower bound. Receiving fewer than a worst-case cap of iid observations cannot improve on receiving the full cap. An expected stopping budget is different: a coupon-collector rule can wait until every context appears. Chosen-context source queries also change the experiment. The information result concerns unobserved orientations; the frozen-attention comparison concerns the specified neural training path (Section 7). ∎ G.5 A sufficient contrast region and its signal scale The contextual reference path stays in c≥1/kc≥ 1/ k, and the sampled-path error budget controls its deviation. The reciprocal normalization below separates this sufficient invariant region from the presence of a learning signal. Proposition G.16 (Nonmonotone signal within the reciprocal member). Let k>1k>1, 0≤c≤10≤ c≤ 1, and ψ(u)=u−1ψ(u)=u^-1, corresponding to a single mixture component with weight one, exponent one and shift zero. Then w(c)=cψ(1+kc2)=c/(1+kc2)w(c)=cψ(1+kc^2)=c/(1+kc^2) increases strictly on [0,1/k][0,1/ k], decreases strictly on [1/k,1][1/ k,1], and has maximum 1/(2k)1/(2 k). It is positive at every c>0c>0. Proof. The denominator is positive throughout the interval. The quotient rule gives w′(c)=(1+kc2)−2kc2(1+kc2)2=1−kc2(1+kc2)2.w (c)= (1+kc^2)-2kc^2(1+kc^2)^2= 1-kc^2(1+kc^2)^2. Its sign is positive for c<1/kc<1/ k, zero at that point and negative above it. Integrating its strict sign between distinct points on either side proves the two monotonicity statements. Substitution gives w(1/k)=(1/k)/(1+1)=1/(2k)w(1/ k)=(1/ k)/(1+1)=1/(2 k). For c>0c>0, numerator and denominator are positive. Thus this member has a positive signal below 1/k1/ k, while the convergence argument uses the invariant region above it. ∎ G.6 Actual sampled Adam and the cubic moment map We now justify the reference and control both finite runs. Lemma G.17 (Physical Adam and normalized coordinates). Use the physical query/key and head step sizes and offsets of Theorem G.1, and reset the moments at each stage. The actual bias-corrected Adam update in every normalized coordinate is exactly ∓ηtDt∓ _tD_t, with minus sign for source and plus sign for feedback, and DtD_t given by the moment recurrence below. Proof. Within each stage, scalar gradients gtg_t generate mt m_t =β1mt−1+(1−β1)gt, = _1m_t-1+(1- _1)g_t, vt v_t =β2vt−1+(1−β2)gt2, = _2v_t-1+(1- _2)g_t^2, m^t m_t =mt/(1−β1t), =m_t/(1- _1^t), v^t v_t =vt/(1−β2t),Dt=m^tv^t+ϵ. =v_t/(1- _2^t), D_t= m_t v_t+ε. Source subtracts ηtDt _tD_t; feedback adds it. If Hi=Dφi⊤H_i=D _i times the batch core gradient, physical query/key and head gradients are Hi/dwidthH_i/ d_ width and Hi/dwidthH_i/d_ width, respectively. Homogeneity of both moments with the prescribed physical offsets yields exactly ηtDt _tD_t in each normalized coordinate. This uses the physical first and second moments; it does not replace them by a population gradient. ∎ Definition G.18 (History constants). Define KA=1−β1(1−β2)(1−β12/β2),Λ=β11−β1+β2(1+β2)1−β2.K_A= 1- _1 (1- _2)(1- _1^2/ _2), = _11- _1+ _2(1+ _2)1- _2. (G.17) Lemma G.19 (A uniform bound for every Adam history). For 0≤β1<10≤ _1<1, 0<β2<10< _2<1, β12<β2 _1^2< _2 and ϵ>0ε>0, every scalar gradient history satisfies |Dt|≤KA|D_t|≤ K_A, where KAK_A is defined in Definition G.18; moreover KA≥1K_A≥ 1. Proof. Weighted Cauchy–Schwarz gives |Dt|≤KA|D_t|≤ K_A for every coordinate history. Before adding ϵε, the squared ratio is bounded by (1−β1)21−β21−β2t(1−β1t)21−(β12/β2)t1−β12/β2. (1- _1)^21- _2 1- _2^t(1- _1^t)^2 1-( _1^2/ _2)^t1- _1^2/ _2. The inequality (1−x)(1−y)≤(1−xy)2(1-x)(1-y)≤(1- xy)^2 bounds this by KA2K_A^2, and also shows KA≥1K_A≥ 1. A positive offset can only decrease the ratio. Thus all-zero rewards and a zero current gradient with nonzero old momentum are covered. ∎ Lemma G.20 (Gaussian row envelopes and core moments). Use the stage horizons from Propositions G.6 and G.12 and the history bound of Lemma G.19. With the row dimensions, radii and moments defined below, every actual and reference row remains in its radius-ρi _i ball up to H. The quadratic and cubic moment maps satisfy (G.18). For 0<δc<10< _c<1, event (G.19) has failure probability at most δc/4 _c/4 and bounds the actual, empirical-reference and population cores by B3B_3, and their speeds by V=3v0B3V=3v_0B_3. The same bounds hold for the frozen branch. Proof. Put d1 d_1 =4nctx, =4n_ ctx, d2 d_2 =2(nctx+p)+1, =2(n_ ctx+p)+1, H H =Ss+Sf+2, =S_s+S_f+2, v0 v_0 =KAmax(d1,d2), =K_A (d_1,d_2), a0 a_0 =1+v0H, =1+v_0H, ρi _i =a0+∥Zi∥, =a_0+ Z_i , where ZiZ_i is a row’s initial Gaussian vector. Every actual or reference row lies in its radius-ρi _i ball through time H. For dimension d define μ3(d) _3(d) =4[a03+23/2Γ((d+3)/2)Γ(d/2)], =4 [a_0^3+2^3/2 ((d+3)/2) (d/2) ], μ6(d) _6(d) =32[a06+d(d+2)(d+4)]. =32[a_0^6+d(d+2)(d+4)]. The two row moment maps, embedded in the direct-sum core, are φ1(q,k)=qk⊤,φ2(a,b,o)=oab⊤. _1(q,k)=qk , _2(a,b,o)=oab . Since ρi≥1 _i≥ 1, both satisfy ∥φi∥≤ρi3,∥Dφi∥≤3ρi2,∥D2φi∥≤6ρi. _i ≤ _i^3, D _i ≤ 3 _i^2, D^2 _i ≤ 6 _i. (G.18) For an allocation δc _c set B3=4δcμ3(d1)+μ3(d2).B_3= 4 _c\ _3(d_1)+ _3(d_2)\. Markov’s inequality gives, with failure at most δc/4 _c/4, ∑groups1dwidth∑iρi3≤B3. _ groups 1d_ width _i _i^3≤ B_3. (G.19) It bounds both actual cores, their coupled empirical reference cores and their speeds by V=3v0B3V=3v_0B_3. Deterministic population cores satisfy the same envelope. Freezing query/key velocities only reduces it. ∎ Lemma G.21 (Normalizer and sample-logit derivatives). Let ψ satisfy (5.3), let the source pool be nonzero, and restrict the head core to ‖C‖≤B3\|C\|≤ B_3. With Y=pWY= pW, X=Y/‖h‖X=Y/\|h\| and W the largest of the 2p2p Gaussian magnitudes, the normalizer, pooled feature and sample logit satisfy the derivative bounds below; in particular (G.20) holds with C1=64(B3+1)C_1=64(B_3+1) and C2=4096(B3+1)C_2=4096(B_3+1). Proof. For u>0u>0, the mixture obeys ψ(u)≤max(1,u−1),|uψ′(u)|≤ψ(u),u2ψ′(u)≤2ψ(u).ψ(u)≤ (1,u^-1), |uψ (u)|≤ψ(u), u^2ψ (u)≤ 2ψ(u). For A(dwidth0)=ψ(dwidth02)A(d_ width_0)= ψ(d_ width_0^2), differentiation gives |A′|≤A/dwidth0|A |≤ A/d_ width_0 and |A′|≤6A/dwidth02|A |≤ 6A/d_ width_0^2. Hence ∥Dhg∥≤2A D_hg ≤ 2A and ∥Dh2g∥≤9A/dwidth0 D_h^2g ≤ 9A/d_ width_0; the latter follows by bounding dwidth0|A′|+3|A′|d_ width_0|A |+3|A | in the derivative of A(dwidth0)hA(d_ width_0)h. On source records let W be the maximum of the 2p2p Gaussian magnitudes, Y=pWY= pW, and X=Y/∥h∥X=Y/ h . Unit keys/query and softmax give ∥h∥≤Y,∥DMh∥≤2Y,∥DM2h∥≤8Y. h ≤ Y, D_Mh ≤ 2Y, D_M^2h ≤ 8Y. Therefore ∥g∥ g ≤1+Y, ≤ 1+Y, ∥DMg∥ D_Mg ≤4(Y+X), ≤ 4(Y+X), ∥DM2g∥ D_M^2g ≤16(Y+X)+36(YX+X2). ≤ 16(Y+X)+36(YX+X^2). On ∥C∥≤B3 C ≤ B_3, conservative joint-core bounds are ∥Df∥≤C1(1+Y2+X2),∥D2f∥≤C2(1+Y3+X3), Df ≤ C_1(1+Y^2+X^2), D^2f ≤ C_2(1+Y^3+X^3), (G.20) where C1=64(B3+1)C_1=64(B_3+1) and C2=4096(B3+1)C_2=4096(B_3+1). ∎ Lemma G.22 (Uniform source inverse-moment bounds). For p≥17p≥ 17 and any deterministic attention core, the source variables Y,XY,X of Lemma G.21 satisfy Yj≤YjEY^j≤ Y_j and Xj≤XjEX^j≤ X_j for 1≤j≤81≤ j≤ 8, with the explicit constants in (G.21). These estimates require no independence between W and h. Proof. For 1≤j≤161≤ j≤ 16 define mj=2j/2Γ((j+1)/2)/πm_j=2^j/2 ((j+1)/2)/ π. For 1≤j≤81≤ j≤ 8 put Yj=pj/2(2p)mj,Xj=pj/2(2p)m2j2j∏ℓ=1j(p−2ℓ).Y_j=p^j/2(2p)m_j, X_j=p^j/2 (2p)m_2j2^j _ =1^j(p-2 ). (G.21) These bound YjEY^j and XjEX^j uniformly over attention cores. Conditional on the keys/query, h is isotropic Gaussian with variance Sπ=∑jπj2≥1/2S_π= _j _j^2≥ 1/2. Cauchy–Schwarz bounds the product Wj∥h∥−jW^j h ^-j using its 2j2j moments; inverse chi-square integration gives the denominator. Independence of W and h is not used. The strongest denominator remains positive at p=17p=17. ∎ Lemma G.23 (Population source regularity at zero shifts). On the core region bounded by B3B_3, the source population gradient norm, its Lipschitz constant and the sample-gradient second moment are bounded respectively by Gs,Ls,σs2G_s,L_s, _s^2 in (G.22). The expected source loss is C2C^2 on compact core sets, including mixture members with zero architectural shifts. Proof. Consequently, the population source gradient bound, its Lipschitz constant, and the sample-gradient second moment are bounded respectively by Gs G_s =C1(1+Y2+X2), =C_1(1+Y_2+X_2), σs2 _s^2 =3C12(1+Y4+X4), =3C_1^2(1+Y_4+X_4), Ls L_s =σs2/4+C2(1+Y3+X3). = _s^2/4+C_2(1+Y_3+X_3). (G.22) To justify derivatives at zero architectural shifts, first add a common auxiliary positive squared shift. The envelopes above are uniform in that shift. Along deterministic convergent core/shift sequences the limiting pool is nonzero almost surely, so sample derivatives converge. The eighth moments bound squared Hessians and give uniform integrability. Vitali convergence establishes continuity of expected derivatives and the C2C^2 limit on compact core sets. No sample-adaptive supremum of inverse radii is asserted; the algorithm uses the prescribed shifts. ∎ Lemma G.24 (Actual reward-gradient regularity). On the same core region, every target pool has norm in [1,k+1][1, k+1]. The actual terminal-reward sample gradient has norm at most AfA_f and conditional expectation yσ′(f)Dfyσ (f)Df. Its second moment and the population feedback gradient norm and Lipschitz constant are bounded by σf2,Gf,Lf _f^2,G_f,L_f in (G.23). Proof. Target pools have norm between one and k+1 k+1 for arbitrary M. Define Af A_f =64(B3+1)(k+1)2, =64(B_3+1)(k+1)^2, Hf H_f =4096(B3+1)(k+1)3, =4096(B_3+1)(k+1)^3, Gf G_f =Af/4, =A_f/4, Lf L_f =(Af2+Hf)/4, =(A_f^2+H_f)/4, σf2 _f^2 =Af2. =A_f^2. (G.23) The actual sample gradient is RrewAσ(−Af)DfR_ rewAσ(-Af)Df. Its norm is at most AfA_f; its conditional action expectation is yσ′(f)Dfyσ (f)Df, exactly the derivative of sampled success. Thus the stated variance bound applies to actual terminal rewards. ∎ Definition G.25 (Joint regularity constants). Set G=max(Gs,Gf),L=max(Ls,Lf),σ2=max(σs2,σf2),ϵmin=min(ϵs,ϵf).G= (G_s,G_f), L= (L_s,L_f), σ^2= ( _s^2, _f^2), _ = ( _s, _f). Lemma G.26 (Existence and uniqueness of the self-consistent reference). For the source and feedback objectives of Section 5.1, use the Gaussian row initialization, offsets and horizons specified above. The population row equations admit a unique self-consistent reference on both stages, retaining row states at the switch. A frozen branch has the same property after projecting query/key velocities to zero. For prescribed bounded core paths the row gradients satisfy (G.24), and their averaged finite-time sensitivity is controlled by the finite Gaussian moment [ρ4e6GHρ/ϵmin]E[ρ^4e^6GHρ/ _ ]. Proof. Write Ψ=(M,C) =(M,C), and let F=LF=L in the source stage and F=JF=J in feedback. For a prescribed continuous core path, each reference row solves the locally Lipschitz ordinary differential equation (ODE) with velocity ±Fϵ(Dφi⊤∇F)± F_ε(D _i ∇ F); the sign depends on the stage. A frozen row projects its query/key velocities to zero. Bounded coordinate speed prevents finite-time divergence. For two row/core pairs, (G.18) gives ∥Hi(X)−Hi(Y)∥≤6Gρi∥Xi−Yi∥+3Lρi2∥ΨX−ΨY∥. H_i(X)-H_i(Y) ≤ 6G _i X_i-Y_i +3L _i^2 _X- _Y . (G.24) Solving this inequality for each row before averaging gives a finite Volterra Lipschitz constant proportional to [ρ4e6GHρ/ϵmin]E[ρ^4e^6GHρ/ _ ]. The map from a prescribed core path to the expectation of the evolved row moment is therefore a contraction on sufficiently short time intervals and maps the bounded core region into itself. Iterating these intervals constructs the unique self-consistent reference on both stages. The same inequality proves uniqueness; the switch concatenates the paths with their row states retained. This justifies the symmetry arguments above. ∎ Lemma G.27 (An averaged exponential-radius event). With λ0=6GH/ϵmin _0=6GH/ _ and E4,B4E_4,B_4 from (G.25), [ρ4eλ0ρ]≤E4(d)E[ρ^4e _0ρ]≤ E_4(d) in dimension d. Event (G.26) has failure probability at most δc/4 _c/4. Proof. Put λ0=6GH/ϵmin _0=6GH/ _ and E4(d) E_4(d) =8 2d/2[a04+4d(d+2)]eλ0a0+λ02, =8\,2^d/2[a_0^4+4d(d+2)]e _0a_0+ _0^2, B4 B_4 =4δcE4(d1)+E4(d2). = 4 _c\E_4(d_1)+E_4(d_2)\. (G.25) The inequality eλ0∥Z∥≤eλ02+∥Z∥2/4e _0 Z ≤ e _0^2+ Z ^2/4, a Gaussian tilt and (a+dwidth0)4≤8(a4+dwidth04)(a+d_ width_0)^4≤ 8(a^4+d_ width_0^4) give this moment bound. A second unconditional Markov event, with failure at most δc/4 _c/4, gives ∑groups1dwidth∑iρi4eλ0ρi≤B4. _ groups 1d_ width _i _i^4e _0 _i≤ B_4. (G.26) The averaged radius moment controls the cubic row maps without a width-dependent maximum-radius bound. ∎ Lemma G.28 (Comparing a history with a current gradient). For any vector h0h_0 and bias-corrected Adam history with positive offset ϵε, inequality (G.27) holds. The normalized first-moment mean lag is at most β1/(1−β1) _1/(1- _1) and the normalized second-moment squared lag is at most β2(1+β2)/(1−β2)2 _2(1+ _2)/(1- _2)^2. Proof. For any current vector h0h_0 and bias-corrected history, let btsb_ts be the normalized nonnegative second-moment weights. Subtracting fractions and using weighted reverse triangle inequalities gives ∥Dt−Fϵ(h0)∥≤∥m^t−h0∥+∑sbts∥gs−h0∥2ϵ. D_t-F_ε(h_0) ≤ m_t-h_0 + _sb_ts g_s-h_0 ^2ε. (G.27) The normalized first-moment lag mean is at most β1/(1−β1) _1/(1- _1), and the second-moment lag second moment is at most β2(1+β2)/(1−β2)2 _2(1+ _2)/(1- _2)^2, producing Λ in (G.17). ∎ Lemma G.29 (Sample, lag and Euler errors within a stage). On event (G.19), suppose all deterministic steps are at most η and each fresh core batch gradient differs from its conditional population gradient by at most ν. With Ai=6Gv0ρi+3LVρi2A_i=6Gv_0 _i+3LV _i^2, the actual direction error is bounded by (G.28), and the reference Euler local error is at most ηt2Ai/(2ϵmin) _t^2A_i/(2 _ ). These estimates apply within each stage after its moment reset, with accumulated parameter error retained. Proof. Let η bound all deterministic steps. On (G.19), population row gradients at two indices in the same stage differ by at most η times their index distance times Ai=6Gv0ρi+3LVρi2.A_i=6Gv_0 _i+3LV _i^2. If the fresh core batch error is at most ν, its row error is at most 3ρi2ν3 _i^2ν. Equation (G.27) then bounds the actual direction error relative to the normalized current population gradient by ηΛAi+6ρi2νϵmin. η A_i+6 _i^2ν _ . (G.28) The reference Euler local error is at most ηt2Ai/(2ϵmin) _t^2A_i/(2 _ ). Both moments reset at the boundary, so history estimates stay within their objective; accumulated parameter error remains. Shortening the final step improves these lag and Euler bounds. ∎ Proposition G.30 (Joint finite-Adam core coupling). Couple each actual row to the reference of Lemma G.26 using its initial Gaussian row. Assume (G.19), (G.26), the batch-error condition of Lemma G.29, and empirical-reference core error at most w at each deterministic grid time. Then the actual-to-population core errors satisfy (G.29) and (G.30), including the stage switch. The same bounds apply to the frozen branch using its own reference and the shared radius events. Proof. Couple actual and reference rows using the same initial Gaussian row. Let ei,ℓe_i, be row discrepancy and Δℓ _ the actual core error relative to its population reference. Then ei,ℓ+1≤ e_i, +1≤ (1+ηℓ6Gρiϵmin)ei,ℓ+ηℓ3Lρi2ϵminΔℓ (1+ _ 6G _i _ )e_i, + _ 3L _i^2 _ _ +ηℓϵminη(Λ+1/2)Ai+6ρi2ν. + _ _ \η( +1/2)A_i+6 _i^2ν \. Solve rowwise, multiply by 3ρi23 _i^2, and average using (G.26). If the reference empirical-core discrepancy at all deterministic grid times is at most w, the result is Δℓ _ ≤w+∑j<ℓηj(KHΔj+RH), ≤ w+ _j< _j(K_H _j+R_H), KH K_H =9LB4/ϵmin, =9LB_4/ _ , RH R_H =B4ϵminη(Λ+1/2)(18Gv0+9LV)+18ν. = B_4 _ \η( +1/2)(18Gv_0+9LV)+18ν \. (G.29) Discrete Gronwall gives, including at the stage boundary, Δℓ≤(w+HRH)eKHH. _ ≤(w+HR_H)e^K_HH. (G.30) The same argument controls the frozen branch with its own reference. Shared Gaussian radius events suffice. Its finite initial attention core is included in w; actual trained rows are not asserted to remain independent. ∎ G.7 Explicit budgets and functional outcomes Definition G.31 (Explicit continuous-input training budgets). Fix the task, architecture and Adam coefficients first. Equations (G.7) and (G.15) determine ϵs,Ss,ϵf,Sf _s,S_s, _f,S_f, hence H,ϵminH, _ . Set δc=δ/4 _c=δ/4, and form all constants in Section G.6. They do not depend on width, step count, or a sampled maximum row norm. Define cp=Γ(p/2)/[πΓ((p−1)/2)]c_p= (p/2)/[ π ((p-1)/2)] and κ κ =min1256π,tmin160000cp2,cmin8,132(k+1+2kB3), = \ 1256π, t_ 160000c_p^2, c_ 8, 132(k+1+ 2kB_3) \, τcore _ core =κ/4,E0=eKHH,w=τcore/(4E0), =κ/4, E_0=e^K_HH, w= _ core/(4E_0), ν ν =ϵminτcore144HB4E0, = _ _ core144HB_4E_0, η η =min1,ϵminτcore8HB4E0(Λ+1/2)(18Gv0+9LV). = \1, _ _ core8HB_4E_0( +1/2)(18Gv_0+9LV) \. (G.31) Put Ts=⌈Ss/η⌉T_s= S_s/η , Tf=⌈Sf/η⌉T_f= S_f/η , and T=Ts+TfT=T_s+T_f. Use step η, shortening the final step of each stage to make its total time exactly SsS_s or SfS_f. Every step is positive. Choose integers, rounded up and at least one, dwidth≥8(T+1)μ6(d1)+μ6(d2)δcw2,Bmb≥8Tσ2δcν2.d_ width≥ 8(T+1)\ _6(d_1)+ _6(d_2)\ _cw^2, B_ mb≥ 8Tσ^2 _cν^2. (G.32) These choices make the width and sample budget explicit for every fixed member. Lemma G.32 (A joint high-probability core event). For the budgets in Definition G.31, with probability at least 1−δc1- _c, both trained and frozen branches have core discrepancy at most τcore/2 _ core/2, hence at most κ/4κ/4, at every deterministic grid time. The event jointly includes the two Gaussian moment events, empirical-reference concentration and fresh-batch concentration; it does not require independence between branches. Proof. At a deterministic reference time the core is an average of iid row moments with total variance at most μ6(d1)+μ6(d2)/dwidth\ _6(d_1)+ _6(d_2)\/d_ width. Chebyshev and the union over T+1T+1 times give error at most w with failure at most δc/8 _c/8 per branch. We apply this concentration bound before intersecting with the radius events, preserving row independence. Given each actual past, a fresh batch has variance at most σ2/Bmbσ^2/B_ mb while (G.19) holds. Chebyshev and the update union bound give batch error at most ν, failing with probability at most δc/8 _c/8 per branch. The branches may share samples; no interbranch independence is needed. The two moment events cost δc/2 _c/2, the two reference-grid events δc/4 _c/4, and the two batch events δc/4 _c/4. On their intersection, (G.30) and (G.31) give Δ≤τcore/2 ≤ _ core/2, in particular Δ≤κ/4 ≤κ/4, for both branches throughout the grid. Source parameter error is carried into feedback. ∎ Lemma G.33 (Continuous-input source acquisition). On the joint event of Lemma G.32, initial source matching is below 0.510.51, and source-endpoint matching is at least 0.90.9 uniformly over source contexts, frames and queries. The endpoint greedy source-bit error is less than 0.10.1. Proof. Initially ∥M∥≤w M ≤ w, so correct source attention is at most e2w/2<0.51e^2w/2<0.51. At the source endpoint the reference odds are at least 1919; core error at most κ gives actual odds at least 19e−2κ>919e^-2κ>9. Matching is therefore at least 0.90.9 uniformly over unit source frames and queries. Conditional on a fixed frame/query and trained parameters, each relevant pooled Gaussian channel has correlation at least 9/829/ 82 with its selected channel. Gaussian planar angles bound the parity-sign error by 2arctan(1/9)/π2 (1/9)/π. Coefficient error changes this sign only if |h1h2|/∥h∥2≤κ/tmin.|h_1h_2|/ h ^2≤κ/t_ . The direction h/∥h∥h/ h is uniform on the (p−1)(p-1)-sphere; each of its first two coordinate densities is at most cpc_p. A union bound gives probability at most 4cpκ/tmin≤0.014c_p κ/t_ ≤ 0.01. The radial multiplier is positive. The total source error is below 0.10.1. ∎ Lemma G.34 (Continuous-input transfer and entry comparison). On the joint event of Lemma G.32, each source-checkpoint target logit has magnitude at most 1/321/32. At the feedback endpoint, every held-out signed logit exceeds log19 19, so sampled success exceeds 19/2019/20. Source-checkpoint success is at most 1/2+1/1281/2+1/128, and the gain exceeds 2/52/5. Proof. At the source checkpoint the reference target-active coefficients vanish, giving actual target logit magnitude at most (k+1)κ(k+1)κ. At the final endpoint the reference signed logit is at least log19+1 19+1. A core attention error ΔM _M changes a softmax weight by at most ΔM/2 _M/2 and the pooled scalar by at most ΔM _M. Equation (G.16) bounds the resulting logit error by kBfinalΔMkB_ final _M. Moreover, ∥Cref∥F2≥nctxkBfinal2/2,kBfinal≤2k/nctxB3≤2kB3. C_ ref _F^2≥n_ ctxkB_ final^2/2, kB_ final≤ 2k/n_ ctxB_3≤ 2kB_3. Coefficient error contributes at most (k+1)ΔC(k+1) _C. The choice of κ makes the total final error and source-entry magnitude at most 1/321/32. Hence final success exceeds 19/2019/20, entry success is at most 1/2+1/1281/2+1/128, and their difference exceeds 2/52/5. ∎ Corollary G.35 (Continuous-input frozen-attention gap). On the same event, the frozen branch has sampled test success at most 1/2+1/641/2+1/64 on every held-out input, and its gap below the trained branch exceeds 2/52/5. Proof. For the frozen branch, the reference head uses absent source channels and has zero target logit for every M. Its actual test logit, even on unequal-amplitude memories, has magnitude at most (k+1)ΔC≤1/16(k+1) _C≤ 1/16. The 1/41/4-Lipschitz sigmoid gives sampled success at most 1/2+1/641/2+1/64, proving the paired gap. This completes the continuous-input part of Theorem 5.2. ∎ G.8 Finite coordinate alphabets We fix all continuous budgets first, then specify a deterministic observation encoding. Write Ns=BmbTsN_s=B_ mbT_s and Nf=BmbTfN_f=B_ mbT_f. Both branches can share source records, so a single source-data event suffices. Optimizer states remain real-valued. Definition G.36 (Source pool separation constants). Set δv=δ/8 _v=δ/8 and s0=p−1e(δv4Ns)1/(p−1),Lv=2p+log(2Ns/δv),mv=s02/Lv.s_0= p-1e ( _v4N_s )^1/(p-1), L_v=2 p+ (2N_s/ _v), m_v=s_0^2/L_v. (G.33) Lemma G.37 (A uniform lower bound for all source attention pools). For the NsN_s iid two-column Gaussian source value matrices and the constants in Definition G.36, event (G.34) holds with probability at least 1−δv1- _v. On that event every simplex pool has norm at least mv/2m_v/ 2, and an operator-norm value perturbation at most mv/2m_v/2 leaves every interpolation pool at norm at least mv/(22)m_v/(2 2). Proof. For each p-by-two Gaussian value matrix, a chi-square Chernoff bound allocates probability at most δv/(4Ns) _v/(4N_s) to each of two events: its first-column norm is below s0s_0, or the distance of its second column from the first-column span is below s0s_0. The first dimension is p≥p−1p≥ p-1, so the latter bound suffices for both. Their union costs δv/(2Ns) _v/(2N_s). The Frobenius upper-tail event above LvL_v also costs at most δv/(2Ns) _v/(2N_s). The QR determinant identity therefore gives σmin≥mv _ ≥ m_v. Union over the NsN_s records yields, with failure at most δv _v, ∥V∥F≤Lv,σmin(V)≥mv,∥Vπ∥≥mv/2for every simplex weight π. V _F≤ L_v, _ (V)≥ m_v, Vπ ≥ m_v/ 2 every simplex weight π. (G.34) In the first expression, ∥V∥F V _F denotes the Frobenius norm of the two-column value matrix. The event controls adaptive weights without selecting or rejecting any record. A value perturbation of operator norm at most mv/2m_v/2 leaves every interpolation pool at norm at least mv/(22)m_v/(2 2). ∎ Definition G.38 (Sign-preserving observation encoding). Use an odd, sign-preserving saturating quantizer with spacing γ=2−qγ=2^-q. It rounds each nonzero magnitude upward to the next grid point and leaves zero unchanged, with clipping level ⌈Lclip/γ⌉γ L_ clip/γ γ. Error within the clipping range is at most γ. It encodes source queries, keys and values, commutes with U, and preserves the parity bit. Embedded zeros remain exact. Rounded frames need not be orthonormal; we couple their paths to the continuous ones. Definition G.39 (Finite computation-graph constants). Let P=dwidth(6nctx+2p+1)P=d_ width(6n_ ctx+2p+1), dh=nctx+pd_h=n_ ctx+p, ϵdec=1/100 _ dec=1/100, and define R0 R_0 =2log(16P/δ), = 2 (16P/δ), Rbd R_ bd =1+R0+KAH, =1+R_0+K_AH, Lclip L_ clip =maxLv,2,2log(8p/ϵdec), = \L_v,2, 2 (8p/ _ dec)\, h0 h_0 =min1,mv/(22), = \1,m_v/(2 2)\, Vbd V_ bd =Lclip+2, =L_ clip+2, Fmax F_ =dh2Rbd3Vbd2(1+h0−1)2, =d_h^2R_ bd^3V_ bd^2(1+h_0^-1)^2, Bprim B_ prim =exp(100[1+dwidthFmax+dwidthRbd2+dhVbd2+h0−2+Zmax+mnorm]), = \! (100[1+d_ widthF_ +d_ widthR_ bd^2+d_hV_ bd^2+h_0^-2+Z_ +m_ norm] ), Nop N_ op =100dwidth(3nctx+p+mnorm+1)2, =100d_ width(3n_ ctx+p+m_ norm+1)^2, D D =P+8nctx+2p+20, =P+8n_ ctx+2p+20, Kgraph K_ graph =(4Bprim)4Nop, =(4B_ prim)^4N_ op, Lgraph L_ graph =dwidthDKgraph, =d_ widthDK_ graph, Lstate L_ state =(7nctx+3p)(Rbd+1)2. =(7n_ ctx+3p)(R_ bd+1)^2. (G.35) Lemma G.40 (Finite-graph bounds for encoded inputs). Use the budgets in Definition G.31, the constants in Definition G.39 and the quantizer of Definition G.38. The Gaussian coordinate event has failure probability at most δ/8δ/8 and keeps both continuous and encoded paths within RbdR_ bd. On that event and (G.34), restrict interpolation pools to norms at least h0h_0. The scalar computation graph has derivatives through order two bounded by KgraphK_ graph; LgraphL_ graph bounds normalized row-gradient state/input Lipschitz constants and logit differences in maximum norm, and LstateL_ state bounds core differences caused by maximum-norm parameter perturbations. Proof. A Gaussian maximum event, failing with probability at most δ/8δ/8, bounds all initialized coordinates by R0R_0. The Adam coordinate bound then keeps both continuous and encoded paths within RbdR_ bd. Scores have magnitude at most 16Rbd216R_ bd^2, since an embedded query/key has only two nonzero coordinates. The graph may compute the full 2nctx2n_ ctx-by-2nctx2n_ ctx core, softmax, pool and head. The operation count covers scalar additions, products, reciprocals, exponentials and logarithms. Unaveraged core/head sums cost at most dwidthRbd2d_ widthR_ bd^2 or dwidthFmaxd_ widthF_ . Evaluate a radial component as exp[−αjlog(u+ζj)] [- _j (u+ _j)] and its mixture square root as the exponential of half its logarithm. On the interpolation domain, h02≤u≤dhVbd2,11+dhVbd2+Zmax≤ψ(u)≤max(1,h0−2).h_0^2≤ u≤ d_hV_ bd^2, 11+d_hV_ bd^2+Z_ ≤ψ(u)≤ (1,h_0^-2). Thus no inverse small mixture weight is needed. The bound BprimB_ prim covers every primitive’s value and first two derivatives, including softmax/logistic denominators and the normalizer logarithms. Induction over unary and binary nodes bounds scalar graph derivatives through order two by KgraphK_ graph. Normalized row gradients equal dwidthd_ width times literal normalized-coordinate derivatives. Hence LgraphL_ graph bounds their state/input Lipschitz constants in maximum norm and also logit differences. Batch averaging preserves this bound. Subtracting quadratic and cubic moment entries and taking Frobenius norms gives the core map’s maximum-norm state bound LstateL_ state. ∎ Lemma G.41 (Lipschitz comparison of actual Adam histories). For two same-length actual Adam gradient histories with the same decay coefficients and offset ϵ>0ε>0, their directions satisfy (G.36), where KAK_A is the history constant of Definition G.18. Proof. For any two actual Adam histories, subtracting moment fractions and applying weighted reverse triangle inequalities yields ∥Dt(g)−Dt(g′)∥∞≤1+KAϵmaxs≤t∥gs−gs′∥∞. D_t(g)-D_t(g ) _∞≤ 1+K_Aε _s≤ t g_s-g _s _∞. (G.36) ∎ Lemma G.42 (Parameter coupling before the first action mismatch). Under the graph conditions of Lemma G.40, couple continuous and encoded runs with the same initial parameters and samples. While their sampled actions agree, the running maximum-norm parameter discrepancy obeys (G.37), with A0=(1+KA)/ϵminA_0=(1+K_A)/ _ . The bound retains accumulated parameter error across the moment reset. Proof. Let EtE_t be the running parameter discrepancy while all sampled actions in a paired continuous/encoded history agree. With A0=(1+KA)/ϵminA_0=(1+K_A)/ _ , Et+1 E_t+1 ≤(1+ηtA0Lgraph)Et+ηtA0Lgraphγ, ≤(1+ _tA_0L_ graph)E_t+ _tA_0L_ graphγ, Efinal E_ final ≤γeA0LgraphH−1. ≤γ\e^A_0L_ graphH-1\. (G.37) Both paths reset moments at the switch while carrying accumulated parameter error. ∎ Definition G.43 (Precision and permitted parameter error). Define E⋆=min1,κ8Lstate,δ32NfLgraph.E_ = \1, κ8L_ state, δ32N_fL_ graph \. Choose an integer q≥mmemq≥ m_ mem such that γ=2−qγ=2^-q is no larger than each of 14,mv22p,κ100(Rbd+1)2,πϵdec296p3/2Lclip,E⋆2e−A0LgraphH. 14, m_v2 2p, κ100(R_ bd+1)^2, π _ dec^296p^3/2L_ clip, E_ 2e^-A_0L_ graphH. (G.38) Proposition G.44 (On-policy encoding coupling and endpoint margins). Choose precision as in Definition G.43. On the Gaussian coordinate and source-pool events, the grid represents all target inputs exactly, source perturbations satisfy the pool interpolation bound, and parameter discrepancy before an action mismatch is at most E⋆/2E_ /2. Across both branches the probability of any action mismatch, conditional on the preceding bounds up to that mismatch, is at most δ/128δ/128. Together with the continuous core event, all required events fail with probability at most 65δ/128<δ65δ/128<δ. On their intersection the combined core discrepancy is at most 5κ/165κ/16; the matching and target endpoint conclusions in Lemmas G.33, G.34 and G.35 retain their stated margins. Fresh encoded source-bit accuracy is established separately in Lemma G.46. Proof. This is a finite, explicit choice after width and sample counts. Target amplitudes 0,±1,±ρmem0,± 1,± _ mem are exact on the grid, source perturbations satisfy the interpolation condition, and parameter discrepancy is at most E⋆/2E_ /2. Couple actual policy actions by shared uniform random variables and stop at the first differing action. Until that event, target inputs coincide; the sigmoid Lipschitz bound makes its conditional mismatch probability at most LgraphE⋆/8L_ graphE_ /8. Across both branches’ 2Nf2N_f samples, the union costs at most δ/128δ/128. On the complementary event all sampled labels/rewards coincide and (G.37) applies to the actual algorithms. Deterministic path Lipschitzness alone would not justify this on-policy step. The continuous two-branch proof costs δ/4δ/4, the maximum event δ/8δ/8, the QR event δ/8δ/8, and mismatches at most δ/128δ/128. Their sum is 65δ/128<δ65δ/128<δ. Combined core discrepancy is at most κ/4+κ/16=5κ/16κ/4+κ/16=5κ/16. Rounded queries and keys alter a source score by at most 9Rbd2γ9R_ bd^2γ, within the remaining matching slack. Target inputs are exact, so the endpoint margins in Section G.7 still apply. ∎ Lemma G.45 (Quadratic anti-concentration and rounding error). Condition on trained parameters and fresh rounded source keys/query. Let C be the nonzero symmetric part of the p×p× p restriction of the head to the source content coordinates, rescaled to Frobenius norm one, and let h be its p-dimensional Gaussian content pool. Pooled values with variance Sπ≥1/2S_π≥ 1/2 satisfy Pr(|h⊤Ch−z0|≤u)≤2πp1/4u/Sπ (|h Ch-z_0|≤ u)≤ 2 πp^1/4 u/S_π for every fixed z0z_0 and u≥0u≥ 0. Rounding with spacing γ and clipping threshold at least MclipM_ clip changes the quadratic sign with probability at most 4pe−Mclip2/2+22/πp3/4γ(2Mclip+γ).4pe^-M_ clip^2/2+2 2/π\,p^3/4 γ(2M_ clip+γ). Proof. Source-bit accuracy on a fresh encoded record requires an additional sign argument. Condition on the trained head and fresh rounded keys/query; its attention weights are independent of fresh Gaussian values. Normalize a nonzero symmetric quadratic coefficient to Frobenius norm one. Some eigenvalue has magnitude at least 1/p1/ p. Conditioning on the other Gaussian coordinates and using the squared-Gaussian density gives, for every fixed center z0z_0, Pr(|h⊤Ch−z0|≤u)≤2πp1/4u/Sπ,Sπ≥1/2. (|h Ch-z_0|≤ u)≤ 2 πp^1/4 u/S_π, S_π≥ 1/2. An interval of fixed length has maximal squared-Gaussian mass at zero. On all 2p2p magnitudes being at most MclipM_ clip, rounding changes a quadratic score by at most pγ(2Mclip+γ)pγ(2M_ clip+γ). Thus its sign-change probability is at most 4pe−Mclip2/2+22/πp3/4γ(2Mclip+γ).4pe^-M_ clip^2/2+2 2/π\,p^3/4 γ(2M_ clip+γ). ∎ Lemma G.46 (Greedy accuracy on a fresh encoded source record). Under Proposition G.44 and precision Definition G.43, rounding a fresh source record adds at most ϵdec=1/100 _ dec=1/100 to its Gaussian-proxy greedy source-bit error. The resulting error remains less than 0.10.1. Proof. The reference source-content block has norm t/2≥tmin/2t/ 2≥ t_ / 2. The core bounds in (G.31) and Proposition G.44 keep the trained restriction within κ<tmin/2κ<t_ / 2, and restriction and symmetrization cannot increase Frobenius error, so its symmetric part is nonzero and Lemma G.45 applies. Take Mclip=2log(8p/ϵdec)M_ clip= 2 (8p/ _ dec). Equation (G.38) makes this at most ϵdec _ dec. A positive radial multiplier preserves the greedy sign, including the zero-pool convention. Adding this error to the Gaussian-proxy source error still gives less than 0.10.1. This uniform argument is specific to quadratic heads; unrestricted finite-grid decoders are not covered. ∎ Proposition G.47 (Observation and computation costs). For the quantizer and precision of Definitions G.38 and G.43, a coordinate needs bcoordb_ coord bits as specified below. A literal source record has 8nctx+2p8n_ ctx+2p coordinate fields and one predicted bit; sparse serialization has 2p+62p+6 coordinate fields, a context index and that bit. One run uses (Ns,Nf,Nf,Ts+Tf)(N_s,N_f,N_f,T_s+T_f) source records, generated actions, verifier queries and updates, with no validation or selection queries, P parameters and mnormm_ norm normalizer components per pool. The encoding preserves the missing-context lower-bound direction of Proposition G.15. Proof. Each coordinate has at most 2⌈Lclip/γ⌉+12 L_ clip/γ +1 symbols, requiring bcoord=⌈log2[2⌈Lclip/γ⌉+1]⌉b_ coord= _2[2 L_ clip/γ +1] bits in a fixed-length encoding. A literal source record contains 8nctx+2p8n_ ctx+2p coordinate fields and one predicted bit. Public sparse serialization uses 2p+62p+6 coordinate fields, a context index and the next bit; it does not reveal the selected-slot index. Numerical decoding is part of the interface and is not learned by a token embedding. The trained and frozen neural runs use this same alphabet. Deterministic encoding preserves the lower-bound direction of Theorem 5.1, since it cannot reveal an unvisited orientation. The equality and its attainability proof in Section C.5 use exact numerical source observations. For one algorithm run, the resources are NsN_s source records and predicted bits, NfN_f generated binary actions, NfN_f verifier queries, and Ts+TfT_s+T_f updates, with zero validation or selection queries. Coordinate bits are additionally charged by the chosen serialization. The normalizer evaluates mnormm_ norm components per pool and the network has P parameters. Optimization uses real-valued states; the resource counts additionally specify observation precision and normalizer evaluation. All conclusions of Theorem 5.2 now follow. ∎ Proof of Theorem G.1. Fix the task, world, normalizer member, Adam coefficients and δ as in the theorem. The normalized implementation is Lemma G.17. The reference exists uniquely by Lemma G.26; Propositions G.6 and G.7 supply its acquired matching and switch law. The target-core and coordinate results in Lemmas G.8, G.9 and G.10 imply the invariant region and finite hitting time in Lemmas G.11 and G.12; Lemma G.13 then supplies the held-out margin. The frozen reference is Proposition G.14. Set δc=δ/4 _c=δ/4 and choose the deterministic budgets of Definition G.31. The row, gradient and history bounds in Lemmas G.20, G.23, G.24, G.27, G.28 and G.29 verify the hypotheses of Proposition G.30; Lemma G.32 realizes them jointly for both branches. Thus Lemmas G.33, G.34 and G.35 give acquisition, transfer and intervention for continuous observations. Finally choose the encoding and precision of Definitions G.38 and G.43. The uniform-pool and graph bounds in Lemmas G.37 and G.40, together with the actual-history comparison Lemma G.41, verify the parameter coupling Lemma G.42. Proposition G.44 carries the matching and target margins to the actual encoded, on-policy runs with total failure probability at most 65δ/128<δ65δ/128<δ. Lemma G.46 gives the remaining fresh source-bit accuracy. These jointly prove all three conclusions on an event of probability at least 1−δ1-δ. Proposition G.47 supplies the observation and computation costs, with real-valued optimizer states. Every budget constant uses the fixed dimensions, Adam coefficients, ZmaxZ_ and mnormm_ norm rather than a sampled row maximum; the mixture bounds used above are uniform over the stated members, so the same formulas apply at the fixed dimensions with probability evaluated separately for each fixed member. ∎ Appendix H The supplementary SGD path H.1 Supplementary SGD learning path The sequential task and the two architecture members in (4.2) also admit a finite stochastic gradient descent (SGD) learning path. This supplementary result uses an unregularized source objective, a predetermined source endpoint and its own horizon and rate choices. Its probability and resource statements apply to this schedule; the Adam schedule is specified separately in Theorem 4.2. The following subsections provide the complete source, optimizer-specific coupling and feedback proofs, using the common initialization in Appendix E. Source SGD uses a readout rate and a smaller strictly positive rate for every other block. Feedback uses the actual complete-trajectory terminal REINFORCE score with baseline zero and fresh minibatches. Control-output rows have predictable rate ηt=η0/pmax,t _t= _0/p_ ,t, where pmax,tp_ ,t is the maximum conditional control probability on one fixed training prompt. It is inspected before the fresh batch and uses no reward or test information. Here η0 _0 is the base feedback step and ρR>0 _R>0 the relative step multiplier for non-control blocks, which have rate ηtρR _t _R. These rates are distinct from the task binding η. Batch denominators are unchanged even when all rewards are zero. Write ws,wfw_s,w_f for the source-trained and final parameters, Bsrc,BRB_ src,B_R for source-record and feedback-trajectory batch sizes, and Tsrc,T_ src,T for their update counts. In the bounds below, h is the prescribed per-decision approximation tolerance, ν the feedback gradient-noise tolerance, and A the feature-norm envelope (6 for the base architecture and 12 for its normalized variant), distinct from the generated control A in the task definition. Theorem H.1 (Finite source acquisition and feedback improvement). Fix q=22r+2q=2^2r+2, r≥1r≥ 1, 0<c≤1/640<c≤ 1/64 and 0<δ<10<δ<1. For either architecture member, choose the explicit nonempty horizon range (H.22), widths, initialization and rates of Sections H.2, E and H.4. After TsrcT_ src raw next-token SGD updates and T sampled REINFORCE updates of the same parameter vector, for every fixed world, with probability at least 1−δ1-δ, simultaneously for every test prompt X, minE(wf,X),M(wf,X),S(wf,X) \E(w_f,X),M(w_f,X),S(w_f,X)\ ≥7/8, ≥ 7/8, (H.1) maxE(ws,X),M(ws,X),S(ws,X) \E(w_s,X),M(w_s,X),S(w_s,X)\ ≤1/q+nh, ≤ 1/q+nh, (H.2) minJ∈E,M,S[J(wf,X)−J(ws,X)] _J∈\E,M,S\ [J(w_f,X)-J(w_s,X) ] ≥3/4, ≥ 3/4, (H.3) V(wf,X) V(w_f,X) ≤9c/8+4nh. ≤ 9c/8+4nh. (H.4) Here wsw_s is the predetermined source checkpoint, and h=ν/(128n2A2)h=ν/(128n^2A^2) with all constants given in the appendices. The budgets are Npre=NBsrcTsrc,Q=BRT,Ngenerated=nQ,Nupdates=Tsrc+T.N_ pre=NB_ srcT_ src, Q=B_RT, N_ generated=nQ, N_ updates=T_ src+T. (H.5) There are zero source verifier calls and zero checkpoint/test selection. Rate inspections add T inference-only evaluations and HTHT prompt-token appearances; padding and full-context costs are listed in Proposition H.46. Under real arithmetic, the finite budgets are polynomial in expanded q,H,c−1,δ−1q,H,c^-1,δ^-1 up to logarithms; the explicit exponents are given in the appendices. The complete proof closes Section H.4, using the source event in Proposition H.10 and the final-metric conversion in Proposition H.39. H.2 Finite neural source acquisition We give the complete source construction and its finite budgets. The feedback constants in Lemma H.30 and (H.83) are chosen first and depend only on q: K=12K=12, η0 _0, T, s, Λ , ν, B0B_0 and BexpB_ . The horizon and source accuracy are chosen next, followed by widths, temperature, derivative envelopes and strictly positive slow rates. This order prevents a width–accuracy circularity. In this and the neural appendices, N=2H+2N=2H+2 is both the record length and padded context length; n=H+2n=H+2 is the number of generated decisions. Lemma H.2 (Permutation family and signed contraction). For the sequential family of Theorem H.1, every ϕ(b,u) _(b,u) is a permutation and the uniform control average is Π . There are 2m22^m^2 labeled tables, including 2m2−12^m^2-1 with θ0,0≠θ0,v _0,0≠ _0,v for a fixed v≠0v≠ 0, which contain the noncommuting pair ϕ(0,0),ϕ(0,v) _(0,0), _(0,v). For the Koopman signed operator McM_c defined below, ‖Mc‖=ρc=2−r\|M_c\|= _c=2^-r; the noisy source operator has signed norm aρca _c. Proof. Each ϕ(b,u) _(b,u) is a permutation: recover z=z′+uz=z +u and then s=s′+b+θu,zs=s +b+ _u,z. For any fixed initial and desired final state there is exactly one control attaining that final state. Consequently q−1∑b,uP(b,u)=Π,q^-1 _b,uP_(b,u)= , (H.6) where Π is uniform averaging on the state alphabet. The family has 2m22^m^2 distinct labeled tables and needs no group-closure assumption. The subfamily θ0,0≠θ0,v _0,0≠ _0,v for fixed v≠0v≠ 0 has 2m2−12^m^2-1 elements and contains noncommuting operations: compare ϕ(0,0) _(0,0) and ϕ(0,v) _(0,v) at public coordinate zero. For the Koopman convention PUf=f∘ϕUP_Uf=f _U for a full control U=(b,u)U=(b,u), decompose f(s,z)=f+(z)+(−1)sf−(z)f(s,z)=f_+(z)+(-1)^sf_-(z). Averaging over b in Mc=q−1∑b,u(−1)ℓ(u)P(b,u)M_c=q^-1 _b,u(-1) (u)P_(b,u) kills the odd state-bit subspace. On the even subspace it is precisely the signed XOR convolution in Lemma C.5. Hence ∥Mc∥=2−r=ρc M_c =2^-r= _c for every unknown table. With source noise aP+(1−a)ΠaP+(1-a) , the signed norm is aρca _c. Transposing to forward distributions preserves the norm. ∎ Lemma H.3 (Invariant all-target source dynamics). Under the unconditioned source law, the diagonal-lag kernel in Equation E.16 has K=12K=12, kmin≥3/(2N)k_ ≥ 3/(2N) and k0H≥1/Nk_0H≥ 1/N. For H≥3H≥ 3, zero-start population CE descent is invariant in the mask/class span plus β1h1+β2h2 _1h_1+ _2h_2, with h1,h2h_1,h_2 defined below. Its semantic coefficients satisfy (H.10) on every prefix with the prescribed class/PAD pattern, including incorrect state histories. Proof. Consider first the unconditioned source distribution, with all controls independent uniform. The diagonal-lag quadratic kernel realized by the Gaussian feature construction in Equation E.16 is K(x,x′)=kbase(x,x′)2+3kbase(x,x′)+2,kbase=∑l=0N−1wlxl=xl′.K(x,x )=k_ base(x,x )^2+3k_ base(x,x )+2, k_ base= _l=0^N-1w_l1\x_l=x _l\. (H.7) It has diagonal K=12K=12, singleton coefficients at least 3/(2N)3/(2N) and the lag-pair coefficient k0H≥1/Nk_0H≥ 1/N. Here xlx_l denotes the token at lag l from the prediction position, with leading PAD tokens as needed. Define the two centered state-output functions h1,y(x) h_1,y(x) =x0∈,xH∈(y=ϕx0(xH)−1q), =1\x_0 ,x_H \ (1\y= _x_0(x_H)\- 1q ), (H.8) h2,y(x) h_2,y(x) =x0∈,xH∈(y=ϕxH(x0)−1q), =1\x_0 ,x_H \ (1\y= _x_H(x_0)\- 1q ), (H.9) and set them to zero on nonstate output rows. Their categorical supports are disjoint. Zero-initialized population CE descent stays in the sum of mask/class logits and β1h1+β2h2 _1h_1+ _2h_2. Write bj=(eβj−1)/(eβj+q−1)b_j=(e _j-1)/(e _j+q-1), let viv_i be the correct state-class mass at state prediction i, and let vEv_E be the incorrect state-class mass at EOS. The simultaneous updates are exactly β1+−β1 _1^+- _1 =ηk0HNq2(a−v1b1), = η k_0HNq^2(a-v_1b_1), (H.10) β2+−β2 _2^+- _2 =ηk0HNq2[∑i=2H(a−vib2)−vEb2]. = η k_0HNq^2 [ _i=2^H(a-v_ib_2)-v_Eb_2 ]. (H.11) In particular EOS contributes a negative term; there is no source grammar mask. For completeness, the categorical-coordinate argument checks more than pairwise independence. At the first-state query, all input states and controls are jointly independent; centering both arguments retains only the lag pair (0,H)(0,H). At a repeated-state query, averaging its current control eliminates coordinates omitting that control; a surviving coordinate must pair it with the current state. At EOS, integrating the uniform S0S_0 makes (U2,…,UH,S1,…,SH)(U_2,…,U_H,S_1,…,S_H) independent of U1U_1. Coordinates omitting both U1,S0U_1,S_0 vanish by averaging U1U_1. Among coordinates containing U1U_1, only its pair with SHS_H survives: an earlier state leaves a later control to average, and a control leaves uniform S0S_0. Among coordinates containing S0S_0 but not U1U_1, an earlier state leaves a later control, SHS_H leaves a suffix average followed by U1U_1, and a suffix control leaves another suffix control when H≥3H≥ 3. Thus the only semantic EOS contribution is (SH,U1)(S_H,U_1), with factor q−2q^-2. Mask/class residuals remain in their span because distinct individual record variables are pairwise independent uniform. The resulting identities are functions of categorical prefixes and therefore also hold on incorrect generated state histories with the same class/PAD pattern. ∎ Lemma H.4 (Source comparator and transition precision). For the invariant reference of Lemma H.3, let 0<a<10<a<1 and 0<ϵ≤10<ε≤ 1, as in the prescribed source accuracy. The comparator below has squared norm bounded by (H.14) and CE excess at most ϵε. At η≤1/Kη≤ 1/K and Tsrc≥D∗/(2ηϵ)T_ src≥ D^*/(2ηε), the final excess satisfies ℰ≤2ϵE≤ 2ε. Its first and repeated transition errors obey the Pinsker bounds below, with denominators 11 and H−1H-1, respectively. Proof. Let hq(a)h_q(a) be the entropy of a point mass mixed with the uniform q-state law at weights a,1−a,1-a. The irreducible unconditioned entropy per optimized token is [(H+1)logq+Hhq(a)]/N[(H+1) q+Hh_q(a)]/N. It is positive. Define β∗=log1+(q−1)a1−a,M=β∗+log(V/ϵ),Pl=xl=PAD.β^*= 1+(q-1)a1-a, M=β^*+ (V/ε), P_l=1\x_l=PAD\. (H.12) The class logits cS=M(P0−PH+P2H),cU=M(PH−P0),cE=M(1−P2H),cPAD=0c_S=M(P_0-P_H+P_2H), c_U=M(P_H-P_0), c_E=M(1-P_2H), c_ PAD=0 (H.13) select the correct class at every raw target. Add β∗(h1+h2)β^*(h_1+h_2). The extra β∗β^* in M pays for the active wrong-class state partition at EOS. A squared comparator-norm bound is D∗≤(5q+V+1)M2kmin+2q(q−1)(β∗)2k0H,kmin≥32N.D^*≤ (5q+V+1)M^2k_ + 2q(q-1)(β^*)^2k_0H, k_ ≥ 32N. (H.14) The constant part of cEc_E uses all lag-zero indicators. The comparator’s CE excess is at most ϵε. Convex smooth descent at η≤1/Kη≤ 1/K and Tsrc≥D∗/(2ηϵ)T_ src≥ D^*/(2ηε) gives last-iterate excess ℰ≤2ϵE≤ 2ε. For any invariant reference, if Dj=KL(Ta∥Tbj)D_j=KL(T_a\|T_b_j), the exact decomposition is D1+(H−1)D2+∑stages−log(correct-class mass)=Nℰ.D_1+(H-1)D_2+ _ stages- (correct-class mass)=NE. (H.15) Pinsker’s inequality gives |b1−a|≤q−1Nℰ/2|b_1-a|≤ qq-1 NE/2 and |b2−a|≤q−1Nℰ/[2(H−1)]|b_2-a|≤ qq-1 NE/[2(H-1)]. The first-state transition has only one observation position per record and requires the stronger accuracy. ∎ Lemma H.5 (Conditioning error for source gradients). Assume H≥5H≥ 5 and the source family of Lemma H.2. A bounded observable retaining rSr_S states and rtagr_ tag tagged controls has signed expectation at most qrS(aρc)H−1−rtag‖F‖∞q^r_S(a _c)^H-1-r_ tag\|F\|_∞. For the reference CE gradient this gives coordinate error μsrc=2q3(aρc)H−4 _ src=2q^3(a _c)^H-4, gradient error at most μsrcV3/2K _ srcV^3/2 K, and uniform logit error at source time ssrcs_ src at most ssrcμsrcV3/2Ks_ src _ srcV^3/2K. Proof. For a signed low-coordinate observable F, retaining rSr_S state values and rtagr_ tag of the tagged controls U2,…,UHU_2,…,U_H, Markov-product contraction gives |0[CsuffixF]|≤qrS(aρc)H−1−rtag∥F∥∞.|E_0[C_ suffixF]|≤ q^r_S(a _c)^H-1-r_ tag F _∞. (H.16) Indeed insert coordinate projectors for retained states. A fixed control has transition norm at most one, each averaged tagged control has norm aρca _c, and U1U_1 supplies no signed contraction. Summing retained state values gives the factor qrSq^r_S. Reference CE-gradient coordinates retain conservatively at most three tagged controls and three states. Their discrepancy is at most μsrc=2q3(aρc)H−4. _ src=2q^3(a _c)^H-4. (H.17) The squared sum of categorical feature weights is at most V2KV^2K, so the gradient discrepancy is at most μsrcV3/2K _ srcV^3/2 K. Nonexpansiveness of the conditional population CE map couples it to the unconditioned reference, giving uniform logit error at most ssrcμsrcV3/2Ks_ src _ srcV^3/2K. Take H≥5H≥ 5 to retain an unused balanced tag also in operation-label terms. The reference risk is unchanged by conditioning, but the conditional Bayes entropy is lower by log2/N 2/N. The comparison therefore retains the conditional reference approximation gap of log2/N 2/N. ∎ Definition H.6 (SGD source accuracy and horizon). Let V=2q+2V=2q+2, α=−logρcα=- _c and 0<c≤1/640<c≤ 1/64. Use A=6A=6 for the base architecture and A=12A=12 for the normalized member. With the feedback constants chosen first as stated above, put Cϵ C_ε =minc2/1024,νe−Bexp/(214A2), = \c^2/1024,\ ν e^-B_ /(2^14A^2)\, Ce C_e =ν13824A2(1+3As), = ν13824A^2(1+3As), (H.18) CM C_M =logqVcCϵ+4, = qVcC_ε+4, CB C_B =log(q/c)+1, = (q/c)+1, (H.19) CD C_D =2(5q+V+1)CM2+6q(q−1)CB2, =2(5q+V+1)C_M^2+6q(q-1)C_B^2, Cs C_s =CD/(2Cϵ)+1, =C_D/(2C_ε)+1, (H.20) Kcond K_ cond =2Csq3ρc−4V3/2K/Ce. =2C_sq^3 _c^-4V^3/2K/C_e. (H.21) Choose any integer H≥⌈max5,(18/α)2,2max(0,logKcond)/α⌉.H≥ \5,(18/α)^2,2 (0, K_ cond)/α \ . (H.22) Set N=2H+2N=2H+2, n=H+2n=H+2, a=1−c/Ha=1-c/H, and ϵsrc _ src =Cϵ/H3, =C_ε/H^3, (H.23) β∗ β^* =log(qH/c−q+1), = (qH/c-q+1), (H.24) M M =β∗+log(V/ϵsrc), =β^*+ (V/ _ src), (H.25) D∗ D^* =(2N/3)(5q+V+1)M2+2Nq(q−1)(β∗)2, =(2N/3)(5q+V+1)M^2+2Nq(q-1)(β^*)^2, (H.26) ηsrc _ src =1/A2, =1/A^2, (H.27) Tsrc T_ src =⌈D∗/(2ηsrcϵsrc)⌉, = D^*/(2 _ src _ src) , (H.28) ssrc s_ src =ηsrcTsrc, = _ srcT_ src, (H.29) h h =ν128n2A2, = ν128n^2A^2, (H.30) e0 e_0 =h48(1+2nAs). = h48(1+ 2nAs). (H.31) Lemma H.7 (Nonempty source horizon and error budgets). The choices in Definition H.6 give, for both transition types, |bj−a|<c/(8H)|b_j-a|<c/(8H), correct-class leakage at most he−Bexp/2he^-B_ /2, and conditioning logit error at most e0e_0. Moreover ssrc≤CsH6s_ src≤ C_sH^6 and e0≥CeH−3e_0≥ C_eH^-3. The source endpoint is chosen before initialization or sampling. Proof. All checkpoints are fixed before data and initialization. The reference satisfies |bj−a|≤23Cϵ/H<c/(8H)|b_j-a|≤ 2 3C_ε/H<c/(8H) for both j. Correct-class leakage is at most Nℰ≤6Cϵ/H2≤he−Bexp/2NE≤ 6C_ε/H^2≤ he^-B_ /2. Using β∗≤CBHβ^*≤ C_BH, M≤CMHM≤ C_MH, D∗≤CDH3D^*≤ C_DH^3 gives ssrc≤CsH6s_ src≤ C_sH^6; also e0≥CeH−3e_0≥ C_eH^-3. Thus the conditioning discrepancy is at most e0e_0 provided KcondH9e−αH≤1K_ condH^9e^-α H≤ 1. Equation (H.22) ensures this because 9logH≤9H≤αH/29 H≤ 9 H≤α H/2 and logKcond≤αH/2 K_ cond≤α H/2. ∎ Definition H.8 (SGD widths, temperature, and positive block rates). Let δj=δ/20 _j=δ/20 for each of seven source estimates, and choose Bsrc B_ src =⌈16ssrc2K2log(2Tsrc/δj)/e02⌉, = 16s_ src^2K^2 (2T_ src/ _j)/e_0^2 , (H.32) WH W_H =V+2log(1/δj), = V+ 2 (1/ _j), Rsrc R_ src =WH+2Assrc, =W_H+ 2As_ src, (H.33) Rtot R_ tot =Rsrc+2nAs, =R_ src+ 2nAs, ζtot _ tot =min1,e0Assrc(ARsrc+2)+Rsrc. = \1, e_0As_ src(AR_ src+ 2)+R_ src \. (H.34) For the base member choose γ=min1/16,e0/(25ssrc),hγ= \1/16,e_0/(25s_ src),h\ and ζraw=ζtot _ raw= _ tot. For the normalized member include ζtot/8 _ tot/8 in that minimum and set ζraw=ζtot/(22) _ raw= _ tot/(2 2). These choices precede every width. Put ξenc=γ/16 _ enc=γ/16, R=⌈8Nlog(2N/δj)⌉R= 8N (2N/ _j) , p=NVp=NV, kaug=p+1k_ aug=p+1, ssym=kaug(kaug+1)/2s_ sym=k_ aug(k_ aug+1)/2, and Dfeat=p+ssym+1D_ feat=p+s_ sym+1. Choose the smallest d divisible by R, with head width dh=d/Rd_h=d/R, satisfying d d ≥2048ξenc−2[plog9+log(8/δj)], ≥ 2048 _ enc^-2[p 9+ (8/ _j)], (H.35) dh d_h ≥max1,⌈4096NRξenc2[Vlog9+log(8N/δj)]⌉, ≥ \1, 4096NR _ enc^2[V 9+ (8N/ _j)] \, (H.36) d d ≥4096γ−2[Dfeatlog9+log(8/δj)], ≥ 4096γ^-2[D_ feat 9+ (8/ _j)], (H.37) d d ≥50[VDfeat+2log(1/δj)]2/e02. ≥ 50[ VD_ feat+ 2 (1/ _j)]^2/e_0^2. (H.38) With J=ssymlog9+log(8/δj)J=s_ sym 9+ (8/ _j), A0=1+γ−2JA_0=1+γ^-2J, and L0=log(eA0kaug/(δjγ))L_0= (eA_0k_ aug/( _jγ)), set Mff=⌈250A02(kaug+L0)8⌉.M_ f= 2^50A_0^2(k_ aug+L_0)^8 . (H.39) The parameter count is P=2dV+4Rdhd+3dMff+2Mff+d+RNP=2dV+4Rd_hd+3dM_ f+2M_ f+d+RN, plus d for the normalized gain. Define Minit M_ init =max1,2Plog(2P/δj), = \1, 2P (2P/ _j)\, Z0 Z_0 =Minit+RMinit3, =M_ init+ RM_ init^3, (H.40) Latt L_ att =2RMinit3[1+2Minit3(Z0+1)], =2 RM_ init^3[1+2M_ init^3(Z_0+1)], (H.41) ℓatt _ att =min1/2,ζraw/Latt, = \1/2, _ raw/L_ att\, ggap g_ gap =δj/(RN2), = _j/(RN^2), (H.42) tatt t_ att =minggap4Minit4,ggap2log((N−1)/ℓatt). = \ g_ gap4M_ init^4, g_ gap2 ((N-1)/ _ att) \. (H.43) At M∗=Minit+1M_*=M_ init+1, put Z∗ Z_* =M∗+RM∗3, =M_*+ RM_*^3, (H.44) Dz D_z =1+R[3M∗2+M∗3N(4M∗3+1/tatt)], =1+ R[3M_*^2+M_*^3N(4M_*^3+1/t_ att)], (H.45) Draw D_ raw =Dz[1+2M∗3(Z∗+1)]+3M∗2(Z∗+1)2+1. =D_z[1+2M_*^3(Z_*+1)]+3M_*^2(Z_*+1)^2+1. (H.46) Use D=DrawD=D_ raw for the base member or D=22Draw+24D=2 2D_ raw+ 24 for the normalized member, and rball=min(1,1/D)r_ ball= (1,1/D). The source readout rate is ηsrc _ src and every other supplied parameter has rate ηsrcρsrc _ src _ src, with ρsrc=min1,rball22DRsrcssrc,e0D2[ARsrc(ARsrc+2)ssrc2/2+2Rsrc2ssrc]. _ src= \1, r_ ball2 2DR_ srcs_ src, e_0D^2 [AR_ src(AR_ src+ 2)s_ src^2/ 2+ 2R_ src^2s_ src ] \. (H.47) Every rate is strictly positive. There is no momentum state, reinitialization or source-score selection. The feedback rate schedule is separately prescribed in Equation H.83. Checking the shared bounds for SGD. Lemma H.9 (Shared-foundation and moving-feature premises for SGD). For the choices in Definitions H.6 and H.8, the shared encoder and coefficient concentration premises hold. The initial coefficient-readout bound is e0/5e_0/5, the initial soft or normalized feature error is at most ζtot _ tot, and Proposition H.14 applies with Wmax=WHW_ =W_H, RW=RsrcR_W=R_ src, s=ssrcs=s_ src and r=rballr=r_ ball. Its logit error is at most e0e_0 and its backbone displacement at most rball/2r_ ball/2. Proof. Here ξenc=γ/16 _ enc=γ/16, Dfeat=p+ssym+1D_ feat=p+s_ sym+1 and δbias=δenc=δffn=δnorm=δj _ bias= _ enc= _ ffn= _ norm= _j. The first three width inequalities above are precisely the premises of Lemmas E.2 and E.5; the displayed FFN width supplies its remaining premise. The fourth width inequality and 1+γ≤21+γ≤ 2 give initial coefficient-readout norm at most e0/5e_0/5 in (H.49). The temperature bounds preserve the bias winner gap and make the soft/hard error at most ζraw _ raw. For the normalized member, γ≤ζtot/8γ≤ _ tot/8 and ζraw=ζtot/(22) _ raw= _ tot/(2 2) give 2ζraw+4γ≤ζtot 2 _ raw+4γ≤ _ tot; for the base member the raw bound already equals ζtot _ tot. Apply Proposition H.14 with Wmax=WHW_ =W_H, RW=RsrcR_W=R_ src, s=ssrcs=s_ src, r=rballr=r_ ball, η=ηsrcη= _ src and the chosen A (66 or 1212). The second term defining ρsrc _ src implies 2ρsrcDRsrcssrc≤rball/2 2 _ srcDR_ srcs_ src≤ r_ ball/2; the third bounds (H.53) by e0e_0. Thus both the ball and error premises hold for the actual common-batch path. Conditional encoder and coefficient bounds are integrated before taking the union; the seven source failure allocations remain δj=δ/20 _j=δ/20 each. ∎ Proposition H.10 (Actual SGD source event). For either architecture, use Definitions H.6 and H.8. For each fixed world, with failure probability at most 7δ/207δ/20, the actual source endpoint has uniform logit error at most 6e0≤h6e_0≤ h relative to the invariant reference, feature error at most 2e0/Rsrc≤h/[12(Rtot+1)]2e_0/R_ src≤ h/[12(R_ tot+1)], readout norm at most RsrcR_ src and backbone displacement at most rball/2r_ ball/2. These bounds hold on all legal-pattern prefixes, including incorrect generated histories. Together with a conditional feedback failure allowance δ/2δ/2, the total failure is at most 17δ/2017δ/20. Proof. The seven estimates proved in Appendices E, H.3 and H.25 concern bias coverage/gaps, encoder frame, FFN/output Gram matrix, whole-initialization norms, initial coefficient readout, the full readout norm and fresh complete-record sampling. Their union failure is at most 7δ/207δ/20. The population reference, conditioned population, sampled ideal coefficients, actual hard features, soft or normalized fixed features, and moving all-parameter network are compared on the same batches. The total uniform source logit error is at most 6e0≤h6e_0≤ h; feature error is at most 2e0/Rsrc2e_0/R_ src, readout norm at most RsrcR_ src, and backbone displacement at most rball/2r_ ball/2. Since Rsrc≥1R_ src≥ 1 and Rtot+1≤2Rsrc(1+2nAs)R_ tot+1≤ 2R_ src(1+ 2nAs), 2e0/Rsrc≤h/[12(Rtot+1)]<h/[4(Rtot+1)].2e_0/R_ src≤ h/[12(R_ tot+1)]<h/[4(R_ tot+1)]. (H.48) These estimates provide all entrance conditions used by the finite feedback proof. In particular the checkpoint was produced by the actual source algorithm, and the uniform prefix bound includes incorrect generated histories. Combined with the target failure allowance δ/2δ/2, the joint failure is at most 17δ/20<δ17δ/20<δ for each fixed unknown world. ∎ Corollary H.11 (Source-stage metric bounds). On the event in Proposition H.10, the actual ancestral source sampler obeys maxE(ws,X),M(ws,X),S(ws,X)≤1/q+nh \E(w_s,X),M(w_s,X),S(w_s,X)\≤ 1/q+nh simultaneously for every test prompt X. Proof. The source reference selects its first control uniformly. Averaging the first transition over all controls produces a uniform state, and subsequent noisy permutations preserve uniformity. Class masses depend only on the mask. Thus its endpoint, first-state and task-utility probabilities are at most 1/q1/q on every fixed prompt. Sequential coupling gives at most 1/q+nh1/q+nh for the actual source sampler. Arbitrary decoders require the separate average-over-binding information bound in Proposition H.40. ∎ Proposition H.12 (Source resource accounting). The source schedule in Definitions H.6 and H.8 uses BsrcTsrcB_ srcT_ src records, NBsrcTsrcNB_ srcT_ src raw next-token targets, TsrcT_ src updates and zero verifier queries. All budgets are finite and polynomial in expanded q,H,c−1,δ−1q,H,c^-1,δ^-1 up to logarithms under real arithmetic, with the padding and context costs below. Proof. The source uses TsrcBsrcT_ srcB_ src records, NBsrcTsrcNB_ srcT_ src raw next-token targets, NBsrcTsrcNB_ srcT_ src prepended PADs, zero verifier calls and TsrcT_ src updates. Separate target evaluation processes N2N^2 context-token appearances per record, with N(N+1)/2N(N+1)/2 PAD appearances. Final feedback and rate-inspection accounting appears in Proposition H.46. The explicit bounds are finite polynomials in the expanded variables q,H,c−1,δ−1q,H,c^-1,δ^-1 up to logarithms, with the exponents specified in the budget formulas. The task alphabet has finite-precision symbols; the neural optimizer is analyzed in real arithmetic. ∎ H.3 SGD readout and moving-feature bounds Use the shared initializer and network in Appendix E, including the same Gaussian laws, encoder frame, realized coefficient operator, soft/hard approximation and derivative bounds. The remaining estimates below concern the SGD schedule. Initial readout events. Lemma H.13 (Gaussian readout events). Conditional on the shared coefficient-operator event and its independence from the Gaussian readout W0W_0, the coefficient-readout bound (H.49) fails with probability at most δhead _ head. The separate full-readout norm bound below has its own failure probability at most δhead _ head. Proof. The readout W0W_0 is independent of CallC_all, with entry variance 1/d1/d. Conditional on the coefficient-operator event, Gaussian norm concentration gives ∥W0Call∥F≤1+γd[VDfeat+2log(1/δhead)] W_0C_all _F≤ 1+γd [ VD_feat+ 2 (1/ _head) ] (H.49) except with probability δhead _head. Increasing d thus gives a small but nonzero initial coefficient error, without setting the initial function to zero. The separate full-readout Gaussian tail gives ∥W0∥F≤V+2log(1/δhead)d, W_0 _F≤ V+ 2 (1/ _head)d, with its own failure allocation. ∎ Proposition H.14 (Deterministic shared-batch coupling). Write the complete logits as Whψ(x)Wh_ψ(x), where ψ contains all trainable blocks except the readout W. On the Euclidean/Frobenius ball ∥ψ−ψ0∥≤r ψ- _0 ≤ r, suppose, uniformly over allowed prefixes, ∥hψ(x)∥≤A,∥Dψhψ(x)∥op≤D. h_ψ(x) ≤ A, D_ψh_ψ(x) _op≤ D. (H.50) Let ∥W0∥F≤Wmax W_0 _F≤ W_ , η≤A−2η≤ A^-2, T<∞T<∞, s=ηT>0s=η T>0, and RW=Wmax+2AsR_W=W_ + 2\,As. The actual minibatch cross-entropy algorithm uses rate η on W and ηρηρ on every block in ψ, with ρ>0ρ>0. Each complete-record loss is the arithmetic mean over its N prescribed next-token targets, and the minibatch loss averages complete-record losses. The reference uses the same batch sequence and W0W_0, holding features at hψ0h_ _0. Writing the batch loss as ℒtL_t, the simultaneous updates are Wt+1 W_t+1 =Wt−η∇Wℒt(Wt,ψt), =W_t-η _WL_t(W_t, _t), ψt+1 _t+1 =ψt−ηρ∇ψℒt(Wt,ψt), = _t-ηρ _ψL_t(W_t, _t), Vt+1 V_t+1 =Vt−η∇Wℒt(Vt,ψ0),V0=W0. =V_t-η _WL_t(V_t, _0), V_0=W_0. (H.51) The reference is an analysis process, not the actual algorithm. If 2ρDRWs≤r/2, 2\,ρ DR_Ws≤ r/2, (H.52) the actual path stays in the radius-r ball and satisfies supx∥WThψT(x)−VThψ0(x)∥2 _x W_Th_ _T(x)-V_Th_ _0(x) _2 ≤ρD2[ARW(ARW+2)s22+2RW2s]. ≤ρ D^2 [ AR_W(AR_W+ 2)s^2 2+ 2\,R_W^2s ]. (H.53) This holds deterministically for every shared batch sequence, without an additional source-sampling or test-word-distribution premise. Proof. The softmax residual has norm at most 2 2, so ∥∇Wℒt∥F≤2A _WL_t _F≤ 2\,A, and both readouts have norm at most RWR_W. The full non-readout gradient satisfies ∥∇ψℒt∥≤2DRW _ψL_t ≤ 2\,DR_W. Therefore ∥ψt−ψ0∥≤2ρDRWηt. _t- _0 ≤ 2\,ρ DR_Wη t. (H.54) Induction with (H.52) prevents leaving the ball. A fixed-feature batch objective is convex and A2A^2-smooth in W, so its step is nonexpansive at the stated rate. At the same readout, ∥∇Wℒt(W,ψ)−∇Wℒt(W,ψ0)∥F≤D∥ψ−ψ0∥(ARW+2). _WL_t(W,ψ)- _WL_t(W, _0) _F≤ D ψ- _0 (AR_W+ 2). (H.55) Add and subtract the two residual–feature products and use softmax Lipschitzness to obtain this inequality. Summing the nonexpansive comparison gives ∥WT−VT∥F≤ρD2RW(ARW+2)s22. W_T-V_T _F≤ ρ D^2R_W(AR_W+ 2)s^2 2. (H.56) The final logit difference is at most A∥WT−VT∥F+RWD∥ψT−ψ0∥A W_T-V_T _F+R_WD _T- _0 , proving (H.53). Its feature-motion term contains D2D^2: the parameter gradient and the conversion of parameter motion to feature motion each supply one factor of D. ∎ Raw-network radius and rates. Corollary H.15 (A positive slow rate on the raw-network ball). Suppose the initial block norms are bounded by MinitM_ init, the initial feature envelope is AinitA_ init, and D bounds the feature derivative on the radius-one block ball. The choices (H.57) and (H.58) satisfy the premises of Proposition H.14 and yield logit error at most εnet>0 _ net>0. Proof. If initial block norms are at most MinitM_init, evaluate D at Minit+1M_init+1. If the realized initializer satisfies supx∥hψ0(x)∥≤Ainit _x h_ _0(x) ≤ A_init, choose r≤min1,D−1,A=Ainit+1.r≤ \1,D^-1\, A=A_init+1. (H.57) The mean-value bound gives ∥hψ∥≤A h_ψ ≤ A throughout this ball, without using the much looser ArawA_raw for the readout rate. Any prescribed εnet>0 _net>0 is attained by the explicit positive choice 0<ρ≤min1,r22DRWs,εnetD2[ARW(ARW+2)s2/2+2RW2s].0<ρ≤ \1,\ r2 2\,DR_Ws,\ _netD^2[AR_W(AR_W+ 2)s^2/ 2+ 2\,R_W^2s] \. (H.58) The initialization and source-budget construction specifies every quantity in this rate choice. The backbone-to-readout rate ratio ρ may be any positive value satisfying the displayed bound, while readout learning proceeds on the η time scale. ∎ Normalized readout and accuracy choices. A dimension-independent initial readout bound. Lemma H.16 (Width-independent full-readout envelope). The independent Gaussian readout obeys (H.59) with probability at least 1−δheadnorm1- _ headnorm. Consequently the deterministic source readout envelope RW=WH+2AssrcR_W=W_H+ 2As_ src can be chosen before the widths. Proof. Use a separate Gaussian norm event for the complete initial readout: ∥W0∥F≤WH:=V+2log(1/δheadnorm). W_0 _F≤ W_H:= V+ 2 (1/ _headnorm). (H.59) The failure probability is at most δheadnorm _headnorm, since the sharper tail is V+2log(1/δheadnorm)/d V+ 2 (1/ _headnorm)/d. The bound is independent of d. Set RW=WH+2AssrcR_W=W_H+ 2\,As_src. The deterministic source/update couplings need this full-readout bound, including all nullspace components, rather than the larger all-parameter envelope MinitM_init. Using MinitM_init in RWR_W would introduce a circular choice: normalization error depends on the coefficient-Gram accuracy, while MinitM_init grows with the widths selected to improve that accuracy. Equation (H.59) avoids this problem. The raw derivative and attention-temperature bounds still use MinitM_init. ∎ Proposition H.17 (Normalized-member precision choices). Use the normalized architecture, the accuracy and horizon in Definition H.6, and the source reference of Lemma H.4. For A=12A=12, the accuracy choices (H.60), made with Lemma H.16, bound the initial normalized feature error by ζtotal _ total. With the normalized derivative envelope and radius specified below, the source schedule (H.61) uses the same comparator and satisfies ssrc≤CsH6s_ src≤ C_sH^6. Proof. Given source time ssrcs_src and a per-error budget e0e_0, set A=12A=12 and ζtotal _total =min1,e0Assrc(ARW+2)+RW, = \1,\ e_0As_src(AR_W+ 2)+R_W \, γ γ ≤min116,e025ssrc,ζtotal8, ≤ \ 116, e_025s_src, _total8 \, ζraw _raw ≤ζtotal22. ≤ _total2 2. (H.60) Equation (E.26) is then at most ζtotal _total. These quantities precede d,Mffd,M_f, since RWR_W uses (H.59). The same actual coefficient-operator concentration proof applies to CallC_all at the smaller γ. Choose the temperature from the raw attention bound using ζraw _raw. The fixed-feature coupling handles the actual normalized map as a uniformly bounded perturbation, not as an exactly polynomial map. Choose rball=min1,Dnorm−1r_ball= \1,D_norm^-1\ and use DnormD_norm in every source/feedback rate and stopping inequality. Take ηsrc=A−2=1144,Tsrc=⌈D∗2ηsrcε⌉. _src=A^-2= 1144, T_src= D^*2 _src . (H.61) At the prescribed source precision, the explicit construction gives ssrc≤CsH6s_src≤ C_sH^6. The reference kernel and comparator remain at K=12K=12. All other source choices use the same formulas with A,Dnorm,WH,ζtotalA,D_norm,W_H, _total substituted as indicated. No normalization second-derivative assertion is required: the finite feedback proof uses a fixed-backbone readout Hessian and pays the entire backbone update through a first-derivative trajectory bound. ∎ Semantic functions and reference geometry. Definition H.18 (Source semantic reference and entropy). We use the same h1,h2h_1,h_2, D∗D^*, μsrc _ src and ssrcs_ src as the source construction. Here the reference CE excess is denoted ℰE, while E(w,X)E(w,X) remains the endpoint metric. For a state-output index y, define h1,y(x) h_1,y(x) =x0∈,xH∈(y=ϕx0(xH)−1q), =1_\x_0 ,\ x_H \ (1_\y= _x_0(x_H)\- 1q ), h2,y(x) h_2,y(x) =x0∈,xH∈(y=ϕxH(x0)−1q). =1_\x_0 ,\ x_H \ (1_\y= _x_H(x_0)\- 1q ). (H.62) They are zero for nonstate outputs. At the first-state prediction, lag zero is U1U_1 and lag H is S0S_0. At prediction i≥2i≥ 2, these lags contain Si−1S_i-1 and UiU_i; at the end-of-sequence prediction they contain SHS_H and U1U_1. Both functions vanish at the initial choice prediction because lag H is padding. The reference is its mask/class component plus β1h1+β2h2 _1h_1+ _2h_2. Its unconditioned Bayes entropy per optimized token is (H+1)logq+Hhq(a)N,N=2H+2, (H+1) q+Hh_q(a)N, N=2H+2, (H.63) where hq(a)h_q(a) is the entropy of a point mass mixed with uniform at weights a and 1−a1-a. Meaningful state histories, including inconsistent histories, share the same class/padding layout. The source coefficient identity controls their logits as functions, not merely on teacher prefixes. Lemma H.19 (Isometric source-reference representations). The categorical singleton/pair feature map and tensor map Φ are isometric on their spans. Comparator norms and zero-start coefficient descent therefore transfer between these representations. Proof. The categorical singleton/pair map and tensor map Φ have the same kernel. For any finite linear combination of feature vectors, sending one representation to the other preserves its Gram quadratic form. The map is consequently well-defined and isometric between their spans. Project a comparator into the span if needed; this cannot increase its norm. This transfers the source zero-start coefficient descent and comparator bounds without equating arbitrary ambient parameter norms. ∎ Lemma H.20 (Last-iterate source excess). In the unconditioned reference, use the comparator of Lemma H.4, rate 0<η≤1/K0<η≤ 1/K, and the predetermined budget T≥D∗/(2ηεsrc)T≥ D^*/(2η _ src). Then the final CE excess satisfies (H.64). Proof. The unconditioned reference readout cross-entropy is convex and K-smooth, since logits are linear in Φ and ∥Φ∥2=K ^2=K. For η≤1/Kη≤ 1/K, descent is nonexpansive and decreases risk. The squared-distance telescope to any comparator gives average excess over that comparator at most D∗/(2ηT)D^*/(2η T). Monotonicity gives the last-iterate version. The comparator’s own excess above irreducible entropy is at most εsrc _src, so the prescribed source horizon yields ℰ≤2εsrc.E≤ 2 _src. (H.64) ∎ Conditioning and complete-record sampling. Lemma H.21 (Conditional-population trajectory comparison). For H≥5H≥ 5, the source choices in Definition H.6 give the gradient-coordinate bound (H.65) and conditional versus unconditioned reference logit error (H.66). Proof. For a signed low-coordinate observable, write its Markov transition product and insert a diagonal coordinate projector for each retained state. Fixed controls have norm at most one; each averaged tagged control has norm aρca _c. The independent U1U_1 has norm at most one and supplies no signed contraction. Summing retained state values contributes the conservative factor qrSq^r_S. After multiplying a singleton/pair feature, probability and label terms in a reference categorical gradient coordinate touch at most three retained controls and three states. Every normalized coordinate discrepancy is therefore at most μsrc=2q3(aρc)H−4. _ src=2q^3(a _c)^H-4. (H.65) The squared sum of categorical feature weights is at most V2KV^2K; there are V output coordinates. Thus the Frobenius gradient discrepancy is at most μsrcV3/2K _ srcV^3/2 K. Apply nonexpansiveness of the conditional population cross-entropy map to the unconditioned reference at every step to obtain uniform logit discrepancy at most ssrcμsrcV3/2K≤e0.s_src _ srcV^3/2K≤ e_0. (H.66) This controls local observables of the reference trajectory, without asserting a full-history density-ratio bound. ∎ Lemma H.22 (Fresh complete-record sampling). At predictable source iterates with independent complete-record minibatches of size BsrcB_ src from Definition H.8, all TsrcT_ src coefficient-gradient errors satisfy (H.67) except with probability δj _j. On this event, the sampled versus population uniform logit error is at most e0e_0 as in (H.68). Proof. A complete-record coefficient gradient has norm at most 2K 2K, regardless of correlations among its N target positions. For each fresh independent record batch, a Hilbert-space bounded-difference estimate and a union bound over predictable TsrcT_src iterates give gradient noise at most 4Klog(2Tsrc/δj)Bsrc.4 K (2T_src/ _j)B_src. (H.67) The same conditional population map is nonexpansive, so the sampling logit discrepancy is at most 4ssrcKlog(2Tsrc/δj)Bsrc≤e0.4s_srcK (2T_src/ _j)B_src≤ e_0. (H.68) Independent batches consist of complete records; individual within-record tokens are not assumed independent. ∎ Transport to realized and moving features. Lemma H.23 (Realized hard-feature coefficient comparison). Let hhard=CallΦh_ hard=C_ all with ‖Call⊤Call−I‖≤γ≤1/16\|C_ all C_ all-I\|≤γ≤ 1/16. Couple the realized and ideal readout updates on the same batches, starting from W0W_0 and zero respectively. At η≤[(1+γ)K]−1η≤[(1+γ)K]^-1, the coefficient discrepancy satisfies (H.71). Under Definition H.8, it is at most 2e02e_0. Proof. For hhard=CallΦh_hard=C_all , put G=Call⊤CallG=C_all C_all. The same-batch readout step induces Z=WCall,Z+=Z−η∇ℒbatch(Z)G.Z=WC_all, Z^+=Z-η _batch(Z)\,G. (H.69) Ideal coefficient descent uses G=IG=I and starts at zero. For η≤[(1+γ)K]−1η≤[(1+γ)K]^-1, the first map is nonexpansive in ∥D∥G−12=tr(DG−1D⊤), D _G^-1^2=tr(DG^-1D ), (H.70) by convex smoothness after the invertible coordinate change. Each batch gradient has norm at most 2K 2K, so supx∥(ZT−Zideal,T)Φ(x)∥ _x (Z_T-Z_ideal,T) (x) ≤K1+γ1−γ[∥W0Call∥F+ssrc2Kγ] ≤ K 1+γ1-γ [ W_0C_all _F+s_src 2K\,γ ] ≤5∥W0Call∥F+25ssrcγ≤2e0. ≤ 5 W_0C_all _F+25s_srcγ≤ 2e_0. (H.71) ∎ Lemma H.24 (Fixed and moving feature comparisons). Use the feature envelopes, source schedule and positive slow rate of Definition H.8. The fixed soft or normalized feature comparison has logit error at most e0e_0 in (H.72); the moving-feature comparison adds at most e0e_0. Final feature error is at most 2e0/Rsrc2e_0/R_ src and backbone displacement at most rball/2r_ ball/2. Proof. For two fixed feature maps within uniform distance ζtotal _total, starting from the same full W0W_0, the gradients at a fixed readout differ by at most ζtotal(ARsrc+2) _total(AR_src+ 2). Their fixed-feature step is nonexpansive for η≤A−2η≤ A^-2. Hence the fixed soft/normalized versus hard logit discrepancy is at most ζtotal[Assrc(ARsrc+2)+Rsrc]≤e0. _total[As_src(AR_src+ 2)+R_src]≤ e_0. (H.72) Proposition H.14 supplies the final moving-feature error at most e0e_0, and backbone displacement at most rball/2r_ball/2. The error denominator in (H.58) also ensures feature motion at most e0/Rsrce_0/R_src. Initial soft/normalized feature error is at most ζtotal≤e0/Rsrc _total≤ e_0/R_src, so the final feature error is at most 2e0/Rsrc2e_0/R_src. The full W0W_0 norm, including nullspace components, is retained. ∎ The joint probability event. Lemma H.25 (Joint source and feedback probability). Allocate δj=δ/20 _j=δ/20 to each of the seven source estimates and δ/2δ/2 to feedback sampling conditionally on the source event. With the widths and bias gap in Definition H.8, their joint failure probability is at most 17δ/2017δ/20, without assuming independence among the resulting events. Proof. Combine the seven source-event allocations by a union bound, without requiring mutual independence after conditioning. Maxima of iid continuous biases select each lag uniformly. A binomial Chernoff bound gives coverage failure at most Nexp(−R8N)≤δj/2.N \! (- R8N )≤ _j/2. (H.73) A difference of two (0,1)N(0,1) biases has density at most 1/(2π)1/(2 π). The probability that any of at most RN2/2RN^2/2 pairwise gaps has magnitude below g is therefore at most RN2g2≤δj/2. RN^2g2≤ _j/2. (H.74) The encoder/feed-forward concentration proofs condition only on the independent previously drawn blocks specified above. Each of the seven source events fails with probability at most δj=δ/20 _j=δ/20. Conditional on all source information on their joint event, fresh target sampling fails with probability at most δ/2δ/2. Thus total failure is at most 7δ/20+δ/2=17δ/20.7δ/20+δ/2=17δ/20. (H.75) Target concentration conditions on pre-batch history, never on future adaptive rates. ∎ H.4 Finite neural feedback and information comparisons This appendix continues the same parameter vector produced by the source analysis. Throughout, N=2H+2N=2H+2 is the source-record and context length, n=H+2n=H+2 is the number of generated decisions, and L=H−1L=H-1 retains its meaning as the displayed suffix length. The fixed kernel bound is K=12K=12. We write A for the feature-norm envelope (A=6A=6 for the base member and A=12A=12 for the normalized member); an operation selected by the policy is identified explicitly as a control token below. Lemma H.26 (Exact reward of the fixed source executor). Let 0<c≤1/640<c≤ 1/64 and |bj−(1−c/H)|≤c/(8H)|b_j-(1-c/H)|≤ c/(8H) for j=1,2j=1,2. For the fixed source reference, conditional on the selected control and correct output classes, the expected rewards are exactly (H.76) and satisfy (H.77). Its public-coordinate checks pass with probability DexecD_ exec independently of the selected control. Proof. Let 0<c≤1/640<c≤ 1/64 and suppose the source reference has |bj−(1−cH)|≤c8H,j∈1,2. |b_j- (1- cH ) |≤ c8H, j∈\1,2\. Set Bexec=b1b2H−1,dj=bj+2(1−bj)q,Dexec=d1d2H−1.B_ exec=b_1b_2^H-1, d_j=b_j+ 2(1-b_j)q, D_ exec=d_1d_2^H-1. At a valid state step of type j, the probabilities of the correct local successor and its state-bit flip are respectively αj=bj+1−bjq,γj=1−bjq. _j=b_j+ 1-b_jq, _j= 1-b_jq. Their sum is djd_j and their difference is bjb_j. Multiplying the two-state even/odd transition matrices shows that all even and odd state-bit-error histories have total masses (Dexec+Bexec)/2(D_ exec+B_ exec)/2 and (Dexec−Bexec)/2(D_ exec-B_ exec)/2. Thus, conditional on the selected control and correct output classes, the exact conditional expected rewards are rη=Dexec+Bexec2,rη¯=Dexec−Bexec2,ra=0(a∉η,η¯),r_η= D_ exec+B_ exec2, r_ η= D_ exec-B_ exec2, r_a=0 (a∉\η, η\), (H.76) where η¯ η flips the binary control coordinate of η and preserves its public coordinate. A different public control coordinate cannot reach the target public endpoint while satisfying the public-coordinate consistency checks. The opposite state-bit choice can be repaired; its reward is explicitly included in (H.76). Conditional on the correct output classes, the public-coordinate checks pass with probability DexecD_ exec, independently of the selected control in this fixed reference. Product and union bounds give rη≥Bexec≥1−9c8,rη¯≤γ1+(H−1)γ2≤9c8q,1−Dexec≤9c8.r_η≥ B_ exec≥ 1- 9c8, r_ η≤ _1+(H-1) _2≤ 9c8q, 1-D_ exec≤ 9c8. (H.77) The transfer argument below separately pays class leakage, EOS, and feature/parameter drift. These fixed-executor equalities are not asserted to hold exactly for the moving network. ∎ Lemma H.27 (Prompt-uniform choice-feature mean). For the ideal choice feature of the SGD source reference, q≥16q≥ 16 and H≥5H≥ 5, the training and test means coincide: +Φ=−Φ=μE_+ =E_- =μ, and ⟨μ,Φ(x)⟩=‖μ‖2=kμ∈[2,K] μ, (x) =\|μ\|^2=k_μ∈[2,K] on every valid choice prompt. Also ‖Φ(x)‖2=K=12\| (x)\|^2=K=12. Proof. Write ∗=η*=η and η¯ η for the two possibly rewarded choices. For q≥16q≥ 16, the ideal choice feature obeys ∥Φ(x)∥2=K,+Φ=−Φ=μ,⟨μ,Φ(x)⟩=∥μ∥2=kμ∈[2,K] (x) ^2=K, _+ =E_- =μ, μ, (x) = μ ^2=k_μ∈[2,K] at every valid choice prompt. To obtain these identities, average the categorical singleton and pair features over the uniform initial state and the H−1H-1 suffix controls. Since H≥5H≥ 5, every coordinate leaves an unused balanced suffix tag. At degree at most two, all non-PAD coordinates are independent and uniform within their prescribed classes, so the kernel mean against any fixed valid choice prompt is constant. The constant kernel term gives kμ≥2k_μ≥ 2. This mean identity concerns choice prompts, not arbitrary later prefixes. ∎ Definition H.28 (Reference reward path and predictable rate). Starting from equal conditional control logits, define the reference coefficient increments by Δt+1=Δt+ηtg(Δt),ga=pa(ra−J)μ,J=∑apara,Δ0=0. _t+1= _t+ _tg( _t), g_a=p_a(r_a-J)μ, J= _ap_ar_a, _0=0. (H.78) Here p=softmax(ΔtΦ(x))p=softmax( _t (x)) on the control class. Use exactly the same predictable rates as the actual network; do not differentiate the rate rule. Let ηt=η0pmax,actual,t,η0=164K. _t= _0p_ ,actual,t, _0= 164K. The maximum is over conditional control probabilities on one fixed canonical training prompt, inspected before the fresh batch at step t. This inspection uses no reward, binding identity, or test information. Lemma H.29 (Actual and reference rate comparison). For the predictable rate in Definition H.28, ηt≤η0q _t≤ _0q pathwise. If the actual and reference conditional control logits differ uniformly by at most 1/161/16 and the correct reference control is maximal, then (H.79) holds. Proof. Pathwise, ηt≤η0q _t≤ _0q. Within a uniform actual/reference control-logit discrepancy of at most 1/161/16, and provided the correct reference choice is maximal, e−1/8η0≤ηtp∗ref≤e1/8η0.e^-1/8 _0≤ _tp_* ref≤ e^1/8 _0. (H.79) ∎ Lemma H.30 (Finite reference reward improvement). Use Definition H.28 with the fixed rewards of Lemma H.26 and feature mean of Lemma H.27. If the actual/reference discrepancy remains at most 1/161/16, the correct control stays maximal and nondecreasing from its uniform start. The predetermined T in (H.80) gives p∗T≥15/16p_*^T≥ 15/16, and cumulative coefficient displacement is at most B0B_0 in (H.81). Proof. Put Δgap=1−9c8(1+1q)≥3132,Lq=log(15(q−1)). _ gap=1- 9c8 (1+ 1q )≥ 3132, L_q= (15(q-1)). At every reference step with p∗≥1/qp_*≥ 1/q, J≥r∗q>rη¯,za+−za=ηtkμpa(ra−J).J≥ r_*q>r_ η, z_a^+-z_a= _tk_μp_a(r_a-J). The correct logit strictly increases and every wrong logit decreases. Consequently each correct-versus-wrong gap increases, and p∗p_* remains maximal and nondecreasing. Until p∗≥15/16p_*≥ 15/16, r∗−J≥Δgap/16r_*-J≥ _ gap/16, so every such gap increases by at least kμη0e−1/8Δgap/16k_μ _0e^-1/8 _ gap/16. Therefore the predetermined number of updates T=⌈640KLq⌉T= 640KL_q (H.80) suffices for p∗T≥15/16p_*^T≥ 15/16 for every kμ∈[2,K]k_μ∈[2,K]. Indeed, a sharper sufficient bound is ⌈16e1/8Lqkμη0Δgap⌉. 16e^1/8L_qk_μ _0 _ gap . The threshold persists after crossing. Only the correct gradient coefficient is positive, and all coefficients sum to zero. Hence ∥g∥F≤2kμp∗(r∗−J)≤2Kp∗. g _F≤ 2k_μ\,p_*(r_*-J)≤ 2K\,p_*. Using (H.79), a conservative bound on cumulative reference displacement is B0=2Kη0T.B_0=2 K\, _0T. (H.81) It is O(logq)O( q) because K=12K=12 is fixed. ∎ Lemma H.31 (Local reward stability for the realized rate). Under Lemmas H.26, H.27 and H.29, fix the realized predictable ηt _t. In the convex tube with control-logit distance at most 1/161/16 from the reference, the coefficient reward Hessian obeys ‖∇U2F‖≤7Kp∗ref\| _U^2F\|≤ 7Kp_* ref and the fixed-rate update has Lipschitz factor less than 9/89/8, as in (H.82). Proof. For an arbitrary coefficient array U, define F(U)=+∑apa(UΦ(x))ra.F(U)=E_+ _ap_a(U (x))r_a. The source control-class logits are equal and cancel in this conditional softmax. With va=pa(ra−J)v_a=p_a(r_a-J), the exact logit Hessian is ∇z2J=diag(v)−vp⊤−pv⊤. _z^2J=diag(v)-vp -pv . Nonnegative rewards imply ∥v∥1≤2J v _1≤ 2J and ∥∇z2J∥≤6J _z^2J ≤ 6J. In the convex tube where every control-logit vector differs from ΔtΦ _t by at most 1/161/16, J(U,x)≤p∗(U,x)+rη¯≤(e1/8+9c8)p∗ref,∥∇U2F∥≤7Kp∗ref.J(U,x)≤ p_*(U,x)+r_ η≤ (e^1/8+ 9c8 )p_* ref, _U^2F ≤ 7Kp_* ref. For the same fixed ηt _t, the update map is therefore locally Lipschitz with factor at most 1+7Kηtp∗ref≤1+7e1/864<98.1+7K _tp_* ref≤ 1+ 7e^1/864< 98. (H.82) No derivative of pmaxp_ is taken. The bound allows arbitrary prompt-dependent coefficient perturbations. The reference path may itself depend on previous actual batches through the predictable rate sequence; the displayed inequalities hold pathwise for the realized sequence. Concentration below conditions only on the pre-batch filtration. Conditioning on the entire future adaptive sequence would be invalid. The fixed bounds K=12K=12 and kμ≥2k_μ≥ 2 keep the horizon O(logq)O( q) and the amplification polynomial in q. ∎ Definition H.32 (Feedback tolerances). Set s s =Tη0q,Λ=(9/8)T,ν=14096K(1+s)Λ, =T _0q, =(9/8)^T, ν= 14096 K(1+s) , (H.83) h h =ν128n2A2,Bexp=2K(B0+1)+1. = ν128n^2A^2, B_ =2 K(B_0+1)+1. Corollary H.33 (Source entrance bounds for feedback). The event in Proposition H.10, with the accuracy and widths of Definitions H.6 and H.8, provides the five entrance bounds below for either architecture. Here Rtot=Rsrc+2nAsR_ tot=R_ src+ 2nAs and s,h,Bexps,h,B_ are defined in Definition H.32. Proof. The source analysis, starting from initialization and raw minibatch SGD, supplies the following actual source event: • An invariant reference fsrcf_ src with |bj−(1−c/H)|≤c/(8H)|b_j-(1-c/H)|≤ c/(8H) for j=1,2j=1,2. • Correct-class leakage at every valid source/generation mask at most he−Bexp/2he^-B_ /2. • Uniform actual-source/reference logit discrepancy at most h on all legal-prefix strings, including incorrect state histories. • A representation hsource=CΦ+esourceh_ source=C +e_ source with ∥C⊤C−I∥≤γ≤h C C-I ≤γ≤ h and sup∥esource∥≤h/[4(Rtot+1)] e_ source ≤ h/[4(R_ tot+1)]. • ∥Wsource∥≤Rsrc W_ source ≤ R_ src, a feature/backbone derivative envelope D on the remaining radius rball/2r_ ball/2, and actual feature norm at most A throughout the initialization ball. Here C is the initialized hard-feature analysis operator CallC_ all, not the bilinear feed-forward factor with the same architectural name. It is never supplied to the learner. The deterministic radii satisfy Rtot=Rsrc+2nAs.R_ tot=R_ src+ 2\,nAs. The source proof derives these bounds from initialization in the order specified in Section H.2. ∎ Definition H.34 (All-parameter feedback schedule). Actual REINFORCE uses fresh independent training-suffix prompts and rollouts, the single binary terminal reward, baseline zero, and the complete generated-token score sum, including EOS. Control-output readout rows have rate ηt _t. Every other readout row and every backbone/gain parameter have the strictly positive rate ηtρR _t _R. All rows and blocks update simultaneously. Let Gslow=2n(A+DRtot)G_ slow= 2\,n(A+DR_ tot) and prescribe ρR=min1,rball2(1+s)Gslow,h4(1+s)Gslow(A+DRtot),h4D(Rtot+1)(1+s)Gslow. _R= \1, r_ ball2(1+s)G_ slow, h4(1+s)G_ slow(A+DR_ tot), h4D(R_ tot+1)(1+s)G_ slow \. (H.84) Lemma H.35 (Feedback slow-block and projection bounds). Starting from Corollary H.33, the actual schedule of Definition H.34 stays within the initialization ball. Throughout feedback, ‖Wt‖≤Rtot\|W_t\|≤ R_ tot, slow-block displacement is at most ρRsGslow _RsG_ slow, and its direct noncontrol/backbone logit effect is at most h/4h/4. With projected fast increment UtU_t defined below, the complete logit error relative to fsrc+control_rows(UtΦ)f_ src+control\_rows(U_t ) is at most 3h3h and ‖ht⊤C−Φ⊤‖≤6h\|h_t C- \|≤ 6h. Proof. The source has already used at most rball/2r_ ball/2 of the backbone ball. A stopped pathwise induction bounds target slow-block displacement by ρRsGslow _RsG_ slow, using at most the remaining half-radius. Its direct noncontrol-logit and backbone-logit effects total at most h/4h/4; feature displacement is at most h/[4(Rtot+1)]h/[4(R_ tot+1)]. Every head update has norm at most ηt2nA _t 2\,nA, because all block rates are at most ηt _t. Consequently ∥Wt∥≤Rtot W_t ≤ R_ tot even without a favorable reward batch. Define the projected actual fast-row increment Ut=(Wcontrol,t−Wcontrol,source)C.U_t=(W_ control,t-W_ control, source)C. On each legal prefix the actual logits differ from fsrc+control_rows(UtΦ)f_ src+control\_rows(U_t ) by at most 3h3h. This pays the source discrepancy, the fast increment times esourcee_ source, and the total slow/backbone effect. Moreover, ∥ht⊤C−Φ⊤∥≤γK+2h≤6h. h_t C- ≤γ K+2h≤ 6h. The projection does not discard head nullspace: its contribution is paid by the ambient head norm times the feature error. ∎ Lemma H.36 (Transport of the complete terminal score). Under Corollaries H.33 and H.35, suppose ‖Ut−Δt‖≤1/(128K)\|U_t- _t\|≤ 1/(128 K). For the complete-trajectory terminal-reward score, conditional on the pre-batch history, the projected expected fast gradient differs from ∇F(Ut)∇ F(U_t) by at most 32n2A2h≤ν/432n^2A^2h≤ν/4, as in (H.85). Proof. Conditional on the past, the expected-gradient comparison is ∥[gactual,fastC∣past]−∇F(Ut)∥≤32n2A2h≤ν4. [g_ actual, fastC ]-∇ F(U_t) ≤ 32n^2A^2h≤ ν4. (H.85) We give the full transport argument. As long as ∥Ut−Δt∥≤1/(128K) U_t- _t ≤ 1/(128 K), put B=K(B0+1)B= K(B_0+1). Every added control logit has magnitude at most B. With source class leakage ℓ0≤he−Bexp/2 _0≤ he^-B_ /2 and the paid 3h3h logit discrepancy, the actual wrong-class probability satisfies ℓactual _ actual ≤ℓ01−ℓ0exp(2B+6h) ≤ _01- _0 (2B+6h) ≤hexp(−1+6h)1−he−Bexp≤h, ≤ h (-1+6h)1-he^-B_ ≤ h, using h≤1/12h≤ 1/12 and Bexp=2B+1B_ =2B+1. Thus both the odds denominator and all source/slow logit errors remain in the calculation. Couple actual generation to the conditional control softmax UtΦU_t , the fixed b1,b2b_1,b_2 conditional state executor, and forced EOS. The conditional-softmax error is at most 3h3h per decision, and class exits add at most h, so sequential coupling fails with probability at most 4nh4nh. The projected complete actual fast score has norm at most 2nA2nA; the ideal choice score has norm at most 2K≤2A 2K≤ 2A. On a matched correct-class path the choice-score error is at most (62+4K)h(6 2+4 K)h. Feature-projection error supplies its first term; conditional probability and the control-class deficit supply its second. Every later state/EOS fast score is at most 2Ah2Ah. The expected-gradient difference is therefore at most [8n(n+1)A+62+4K+2(n−1)A]h [8n(n+1)A+6 2+4 K+2(n-1)A]h ≤20An2h≤32n2A2h, 56.9055pt≤ 20An^2h≤ 32n^2A^2h, which proves (H.85). Matched complete trajectories receive the same actual terminal bit, including repaired hidden-state histories. On unmatched trajectories the score envelopes pay the reward change. No hidden correct-trace constraint is inserted. In particular, the nullspace term is explicitly bounded by ∥ΔW∥≤2Rtot,∥ΔWesource∥≤h/2; W ≤ 2R_ tot, We_ source ≤ h/2; source and slow effects then total less than 3h3h. An ambient head norm is never identified with its projection. ∎ Lemma H.37 (Uniform fresh-feedback batch concentration). Under the same source event and the stopped feedback path in Lemma H.36, use independent fresh batches of size (H.86). With probability at least 1−δR1- _R, the projected fast-score batch noise is at most ν simultaneously for all T predictable iterates. The original batch denominator is retained, including for zero-reward batches. Proof. For a target failure allowance δR _R, take BR=⌈256n2A2log(2T/δR)ν2⌉.B_R= 256n^2A^2 (2T/ _R)ν^2 . (H.86) Conditional on the pre-batch filtration, the independent reward-weighted scores have norm at most Mscore=2nAM_ score=2nA, and their batch mean obeys ∥batchmean−[score]∥≤MscoreBR.E \ mean-E[score] ≤ M_ score B_R. Bounded differences gives deviation at most Mscore[1+2log(T/δR)]BR<ν M_ score[1+ 2 (T/ _R)] B_R<ν with failure probability at most δR/T _R/T. A union bound gives simultaneous batch noise at most ν with probability at least 1−δR1- _R. Batches with zero rewards are included. The denominator is the original BRB_R, without reward filtering. No independence between an iterate or its rate and earlier batches is asserted. ∎ Lemma H.38 (The actual feedback path stays in the reference tube). On the source event and the batch event of Lemma H.37, use Definitions H.32 and H.34. The actual and reference projected increments satisfy (H.87) for every 0≤t≤T0≤ t≤ T. Neither the coefficient nor parameter stopping boundary is reached, and the rate comparison holds throughout. Proof. Compare actual and population updates using the same ηt _t and put Et=∥Ut−Δt∥E_t= U_t- _t . Inside the stopped tube, (H.82), (H.85), and the batch event imply conservatively Et+1≤98Et+8ηtν,E0=0,Et≤8sΛν<1512K.E_t+1≤ 98E_t+8 _tν, E_0=0, E_t≤ 8s ν< 1512 K. (H.87) This is stricter than the stopping threshold 1/(128K)1/(128 K). Together with the 3h3h source/soft/slow discrepancy, it makes the actual/reference conditional choice-logit difference less than 1/161/16, closing the rate comparison used in (H.79) and (H.82). The same induction closes coefficient and parameter exits. Rates are predictable before each batch. Since K=12K=12 and T=O(logq)T=O( q), Λ is a fixed power of q up to absolute constants. ∎ Proposition H.39 (Final SGD metrics and source-to-reward gain). On the events in Propositions H.10 and H.37, at the predetermined source and reward endpoints, simultaneously on every test prompt, minE(wf,X),M(wf,X),S(wf,X)>7/8 \E(w_f,X),M(w_f,X),S(w_f,X)\>7/8, maxE(ws,X),M(ws,X),S(ws,X)≤1/q+nh \E(w_s,X),M(w_s,X),S(w_s,X)\≤ 1/q+nh, each final-minus-source gain exceeds 3/43/4, and V(wf,X)≤9c/8+4nhV(w_f,X)≤ 9c/8+4nh. Proof. At the predetermined final iterate the actual conditional correct-choice probability differs by at most KET+3h KE_T+3h from a reference probability of at least 15/1615/16, on every valid choice prompt, including held-out suffixes. In particular the uniform choice-logit discrepancy is less than 1/2561/256, so the comparison choice probability is at least 239/256239/256. The fixed executor has expected reward conditional on the correct initial control at least 1−9c/8≥503/5121-9c/8≥ 503/512 and correct first-state probability at least the same lower bound. Final trajectory coupling costs at most 4nh<1/10244nh<1/1024. Thus task utility and first-state memory success exceed 239256503512−11024>78. 239256 503512- 11024> 78. (H.88) Endpoint success dominates task utility. In the source ancestral sampler, uniform choice makes the first state uniform. Every subsequent transition kernel preserves uniformity, while class masses depend only on position. Therefore the reference has Esource≤1/qE_ source≤ 1/q, Ssource≤EsourceS_ source≤ E_ source, and Msource≤1/qM_ source≤ 1/q. Actual source coupling bounds all three by 1/q+nh1/q+nh. Since q≥16q≥ 16 and nh<1/4096nh<1/4096, each final-minus-source gain exceeds 3/43/4. The fixed executor satisfies the public-coordinate checks with probability DexecD_ exec independently of its choice, so actual grammar-or-public-coordinate violation risk is at most 1−Dexec+4nh≤9c8+4nh.1-D_ exec+4nh≤ 9c8+4nh. ∎ Arbitrary source-only decoders. Proposition H.40 (Source-only decoding with an independent binding). If the binding η is uniform over the q controls and independent of source observations, initialization, learner randomness and evaluation prompts, every source-only decoder has binding-averaged endpoint success at most 1/q1/q. Proof. Let η be uniform over the q controls, independently of all source observations, initialization, learner randomness, and evaluation prompts. For each fixed prompt the map η⟼ϕUH⋯ϕU2ϕη(S0)η _U_H·s _U_2 _η(S_0) is bijective. Condition on every source-only observation and algorithmic random choice. Any endpoint equals exactly one of these q targets, so averaged endpoint success is at most 1/q1/q. Malformed output only reduces success. This includes arbitrary source-only computation, decoding, and calibration. It is an average-over-binding or minimax statement, not a pointwise assertion for every binding. ∎ Indistinguishable direct-feedback worlds. Proposition H.41 (Paired direct-feedback indistinguishability). With no source observations and only the prescribed terminal training bit, the involution below pairs worlds whose adaptive training transcripts have identical laws for any number of queries. On a common tag-one test prompt, their average endpoint success is at most 1/21/2. Proof. Let ℓ(u) (u) be the public tag bit. Pair the worlds by θu,z′=θu,z⊕ℓ(u),η′=(bη⊕ℓ(uη),uη).θ _u,z= _u,z (u), η =(b_η (u_η),u_η). The adjusted first transition is identical: ϕη′=ϕηφ _η = _η. The central state-bit flip B commutes with every transition map. Each suffix operation adds Bℓ(u)B (u), so paired target endpoints differ by B raised to the suffix tag parity. They agree on every tag-zero training suffix and are opposite on every tag-one test suffix. Grammar and the public z-consistency checks are identical for every candidate trace in both worlds, including hidden errors that repair later. Hence their entire terminal training oracles are pointwise identical. Without source observations, couple the learner’s random tape. Every adaptive training transcript remains identical, even with unlimited training queries. On a common test prompt the identical output law has summed endpoint success at most one against the two opposite targets; average paired success is at most 1/21/2. The pairing is an involution and preserves uniform η. It does not imply a first-state memory lower bound, because the paired first targets agree. Hidden-mechanism intermediate verification would invalidate this pointwise oracle equivalence and is outside the information model. ∎ Singleton-query resource bound. Proposition H.42 (Singleton-query lower bound). Let the binding be uniform and independent of learner randomness and test prompts. Even if θ is given, a learner with Q adaptive pre-test candidate-verification queries and no binding-dependent prompt information obeys (H.89). Hence binding-averaged endpoint success at least 7/87/8 requires Q≥7q/8−1Q≥ 7q/8-1. Proof. Even grant θ for free. Endpoint bijectivity implies that any fixed prompt/candidate terminal query accepts for at most one η; malformed or public-inconsistent candidates accept for none. Fix learner randomness and follow the all-zero transcript. At most Q adaptive queries name at most Q distinct bindings. Every world producing a positive response belongs to these at most Q values, irrespective of subsequent behavior. All remaining bindings yield the same all-zero transcript. On each independent test prompt a final output succeeds for at most one remaining value. Therefore η[test endpoint success]≤min1,Q+1q.E_η[test endpoint success]≤ \1, Q+1q \. (H.89) In particular, averaged success at least 7/87/8 requires Q≥7q/8−1Q≥ 7q/8-1. Each sampled candidate verification counts separately. Expected-reward oracles, intermediate labels, binding-dependent prompt side information, and test-time verification are excluded. This is a membership-query law, not a logarithmic entropy argument; the neural upper need not match this bound. ∎ Packed-mechanism information bound. The separate proof in Lemma C.13 establishes, for m≥512m≥ 512, H≥5H≥ 5, independent reset source records, and at most Q adaptive binary pre-test replies, supθ,η[endpoint error]≥732[1−128(nrecH+Q+1)m2]+. _θ,ηE[endpoint error]≥ 732 [1- 128(n_ recH+Q+1)m^2 ]_+. It permits a fixed known binding and strengthens query permissions to adaptive prompt choice. The quantity nrecH+Qn_ recH+Q counts information bits, not heterogeneous monetary or computation costs. It supplements the independent-binding singleton bound and the source-free paired-world ambiguity; its proof is not duplicated here. Proposition H.43 (Opaque transition-table recovery). Under the sequential raw source law, let 0<δtab<10< _ tab<1 and use the complete-record budget (H.90). The rowwise empirical-mode estimator recovers all q2q^2 opaque successors with probability at least 1−δtab1- _ tab, using at most 6M0+2H+26M_0+2H+2 raw tokens and no downstream goal labels. Proof. Under the raw source law, execution controls U1,…,UH−1U_1,…,U_H-1 are jointly iid uniform: U1U_1 is independent, and U2,…,UH−1U_2,…,U_H-1 form a strict subset of the tag-conditioned suffix. Retain odd execution transitions through H−1H-1 and average the intervening even controls. Their inputs become independent uniform states because q−1∑U∈TU=Πq^-1 _U T_U= . Thus a complete record supplies ⌊H/2⌋ H/2 iid triples in this marginal retained experiment. Take M0=⌈8q2a−2log2q3δtab⌉,nrec=⌈M0⌊H/2⌋⌉.M_0= 8q^2a^-2 2q^3 _ tab , n_ rec= M_0 H/2 . (H.90) The charged raw-token count satisfies Nnrec=(2H+2)nrec≤6M0+2H+2.Nn_ rec=(2H+2)n_ rec≤ 6M_0+2H+2. The empirical mode for every opaque state/control row recovers all q2q^2 successors with probability at least 1−δtab1- _ tab. This estimator requires neither a coordinate chart nor downstream goal labels. For completeness, let M=nrec⌊H/2⌋M=n_ rec H/2 be the number of retained iid triples and put t=log(2q3/δtab)t= (2q^3/ _ tab). Each row count has mean μ=M/q2≥8a−2tμ=M/q^2≥ 8a^-2t. Binomial lower-tail concentration bounds the probability of fewer than μ/2μ/2 observations by e−μ/8e^-μ/8. Conditional on n observations of a row, true-minus-competitor indicator differences lie in [−1,1][-1,1] and have mean a. Their centered log moment generating function is at most λ2/2λ^2/2, so an incorrect empirical mode, including a tie, has probability at most e−na2/2e^-na^2/2. Union bounding the q2q^2 row counts and q2(q−1)q^2(q-1) comparisons gives failure probability at most δtab _ tab. This proves recovery with full- record rounding. It does not assume independence of all H transitions within a record. ∎ Corollary H.44 (Binding selection from a recovered table). On the recovery event of Proposition H.43, at most q−1q-1 exact-path terminal verifications on fresh uniform training prompts identify the binding and give perfect test utility and public-coordinate compliance. Additional costs, storage and computation are as stated below. Proof. On recovery, enumerate candidate bindings on separately drawn fresh uniform training prompts. Generate each candidate’s exact path using the recovered table. The binary terminal reward equals one exactly when that candidate is η: suffix invertibility and sharp transitivity of the first transition prove the equivalence. Test at most q−1q-1 candidates, stopping at success or inferring the last binding if all reject. No canonical-prompt query selection is needed. The additional costs are at most q−1q-1 verifier calls, (H+2)(q−1)(H+2)(q-1) generated symbols, and H table lookups per candidate or deployment. There are zero gradient updates and no test-time oracle. Final storage is q2q^2 successor indices and one binding index; training may use q3q^3 counts. Deployment has perfect utility and public-coordinate compliance on the recovery event. This comparator uses the same observations with a table-based computation, separating information requirements from the neural training route. The 2H+22H+2 granularity term and every unused raw token remain charged. ∎ Proposition H.45 (Greedy source execution with an external binding controller). On Proposition H.10, externally choosing any valid first control and greedily decoding thereafter yields its exact trajectory and EOS. At most q−1q-1 fresh-prompt verifications identify the binding, after which every test prompt has the correct first state and endpoint. This holds with probability at least 1−7δ/201-7δ/20 for each fixed world and uses the additional resources enumerated below. Proof. On the proved source event, the conditional transition parameters obey bj≥1−9c/(8H)b_j≥ 1-9c/(8H), the wrong-class mass is at most ℓ0=he−Bexp/2≤h/2 _0=he^-B_ /2≤ h/2, and actual source logits differ from the reference by at most h uniformly over legal-pattern prefixes. An individual softmax probability changes by at most h under this logit discrepancy: integrate its coordinate gradient, whose ℓ1 _1 norm is 2p(1−p)≤1/22p(1-p)≤ 1/2. The correct state token therefore has actual probability at least (1−ℓ0)bj−h≥1−9c8H−3h2>12.(1- _0)b_j-h≥ 1- 9c8H- 3h2> 12. The correct EOS probability is at least 1−ℓ0−h>1/21- _0-h>1/2. The strict inequalities follow from c≤1/64c≤ 1/64, H≥5H≥ 5 and nh<1/4096nh<1/4096. After externally selecting any valid first control, greedy decoding consequently follows its unique exact state trajectory and then EOS, by induction over legal-pattern prefixes. This uses the uniform source function bound, not an assumption about teacher-forced visits. That candidate satisfies the public-coordinate consistency checks. Suffix invertibility and sharp transitivity make its endpoint equal the target if and only if its first control equals η. At most q−1q-1 fresh-prompt verifications therefore identify the binding, with the last candidate inferred if all others reject. The same corrected choice gives the target first state and exact execution on every test prompt. Success is deterministic on the source event, whose probability is at least 1−7δ/201-7δ/20 for each fixed world. The comparator pays the original source tokens and updates. Additional costs are at most q−1q-1 verifier queries, n(q−1)n(q-1) generated symbols including the externally supplied control, H(q−1)H(q-1) prompt tokens and (H+1)(q−1)(H+1)(q-1) model forward evaluations, each retaining N context tokens. There are no additional optimizer updates. The binding index and enumeration counter are additional persistent controller state; model parameters, inference context and forward workspace are separate. Each verification still performs H trusted transitions and public checks. Deployment uses H+1H+1 forwards and n output symbols, with no verification. This comparator combines the frozen source model with an external binding search. ∎ Proposition H.46 (Complete SGD resource accounting). For Definitions H.6, H.8, H.32 and H.34, there are Q=BRTQ=B_RT feedback queries, nQnQ generated decisions, Tsrc+T_ src+T optimizer updates, and T additional inference-only rate inspections. Padding, context, prompt, parameter-storage and verifier costs are the counts below; no test score chooses a checkpoint. Proof. No model updates occur after the fixed source and target budgets, and no test score selects a checkpoint. A canonical rate-inspection prompt may use a fixed state and H−1H-1 copies of the publicly tag-positive zero control. Before each fresh rollout batch, read its conditional control probabilities without generation or verification. This charges T inference-only evaluations, HTHT prompt-token appearances, NTNT processed context-token appearances, and nTnT PAD appearances, separately from zero data-selection queries and from optimizer updates. Each source record prepends N PADs and trains on N non-PAD targets. If predictions are separately evaluated, it processes N2N^2 context-token occurrences with N(N+1)/2N(N+1)/2 PAD appearances. Each feedback rollout starts its N-token context with n=N−Hn=N-H PADs and H real prompt tokens. Its n decisions process nNnN context-token occurrences, including n(n+1)/2n(n+1)/2 PAD appearances; no real token is truncated. Across Q=BRTQ=B_RT rollouts, charge nQprepended PADs,nNQcontext-token occurrences, nQ\ prepended PADs, nNQ\ context-token occurrences, n(n+1)Q2attended PAD occurrences, n(n+1)Q2\ attended PAD occurrences, HQprompt tokens,nQgenerated decisions. HQ\ prompt tokens, nQ\ generated decisions. Canonical inspections are additional. Optimization uses real arithmetic and stores P real parameter scalars. Each verifier call evaluates H unknown transition-table entries for the target endpoint and H public-coordinate comparisons. The verifier reveals the prescribed bit; its private θ table remains part of the verifier’s resources. ∎ Proposition H.47 (Pathwise equivalence to on-policy reward-weighted likelihood). For one fixed complete sampled batch and current model, baseline-zero terminal REINFORCE equals the negative gradient of (H.91), holding sampled tokens and rewards fixed. With the same simultaneous block rates, inspection rule and fresh coupled batches, the two procedures have identical parameter paths; independently sampled fresh batches give the same law. Proof. For a fixed complete sampled batch and current model, baseline-zero REINFORCE ascent is exactly the negative gradient of −1BR∑bRb∑j=1nlogπw(Yb,j∣prefixb,j).- 1B_R _bR_b _j=1^n _w(Y_b,j _b,j). (H.91) Hold the realized rewards and trajectory tokens fixed for this single differentiation. With identical simultaneous block rates and the same canonical inspection rule, reward-weighted likelihood makes exactly the same update. Fresh coupled batches preserve pathwise equality; independently generated fresh batches give equality in law. This does not identify arbitrary expert SFT, filtered-denominator training, or repeated off-policy replay with this algorithm. Source-only continuation remains independent of the binding, so the arbitrary-decoder average bound still applies. Test-time best-of-N scored by the task verifier adds post-test-prompt oracle calls outside the pre-test lower bounds and cannot be counted as free. ∎ Proof of Theorem H.1. Choose the feedback constants in Definitions H.28 and H.32 first, and then the horizon, accuracy, widths, temperature and positive rates in Definitions H.6 and H.8. The nonempty horizon and its error inequalities follow from Lemma H.7; Lemma H.9 verifies the shared-foundation and moving-feature premises for each architecture. By Proposition H.10, the prescribed actual source endpoint satisfies all five feedback entrance conditions in Corollary H.33 except on an event of probability at most 7δ/207δ/20. Set δR=δ/2 _R=δ/2 in (H.86). Conditional on the source event, Lemma H.37 gives the fresh-batch event with failure probability at most δ/2δ/2. Their union failure is 17δ/20<δ17δ/20<δ by Lemma H.25; no independence between these events is required. On their intersection, Lemmas H.35, H.36 and H.38 compare the same-parameter actual training path with the fixed-executor reference, and Lemma H.30 supplies its final correct-choice probability. Proposition H.39 then gives all four displayed metric conclusions, simultaneously for every test prompt. In particular it combines 239/256239/256 correct-choice probability, executor success at least 503/512503/512, and trajectory-coupling loss below 1/10241/1024 to exceed 7/87/8; the source bound 1/q+nh1/q+nh, with q≥16q≥ 16 and nh<1/4096nh<1/4096, gives the 3/43/4 gain. The same coupling yields the violation bound 9c/8+4nh9c/8+4nh. Finally Propositions H.12 and H.46 give NBsrcTsrcNB_ srcT_ src source targets, BRTB_RT reward queries, nBRTnB_RT generated decisions and Tsrc+T_ src+T updates, plus the stated rate-inspection and context costs. The formulas in the cited definitions establish the finite polynomial resource dependence under real arithmetic. The endpoints are predetermined, so neither checkpoint nor test selection adds a query. ∎