Paper deep dive
Many Steps into One: Reasoning Contraction and the Limits of Safe Memory Reuse for LLM Agents
Jianghong Zhou, Hao Rong, Chen Lin, Daoan Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Thought templates, skill libraries, agent memories and caches turn many reasoning steps into one. We call this reasoning contraction. A contracted memory saves reasoning steps and errors until its facts change, then quietly answers wrong. Most systems trust memories by counting successes and failures. We prove that such counters are badly tuned change detectors: a symmetric counter notices a partial change only after a power of its false-alarm horizon, and an unclipped one may never notice it. On five LLMs answering multi-hop questions over edited facts, ExpeL- and ACE-style counters serve 7.0-13.5x more stale answers than certified memories given the same information, and on code tasks with an exact verifier 35-73x more. Safety also has a price. When facts change and each memory is judged only by its own verified outcomes, one number, the certification load, sets how much reasoning any safe memory can skip, and under Zipf demand it yields a scaling law. Our recipe weighs each memory's failures by its own likelihood ratio, certifies its utility against live reasoning, and restarts the certificate when facts change, with the likelihood-ratio detector as the safety net for unannounced changes; for large edits it also invalidates the memory.
Tags
Links
- Source: https://arxiv.org/abs/2610.17544v1
- Canonical: https://arxiv.org/abs/2610.17544v1
Trouble viewing inline? Open PDF directly →
Full Text
299,435 characters extracted from source content.
Expand or collapse full text
Many Steps into One: Reasoning Contraction and the Limits of Safe Memory Reuse for LLM Agents Jianghong Zhou Affiliation: ByteDance Hao Rong Affiliation: Scale AI Chen Lin Affiliation: ByteDance Daoan Zhang Affiliation: ByteDance Abstract Thought templates, skill libraries, agent memories and caches turn many reasoning steps into one. We call this reasoning contraction. A contracted memory saves reasoning steps and errors until its facts change, then quietly answers wrong. Most systems trust memories by counting successes and failures. We prove that such counters are badly tuned change detectors: a symmetric counter notices a partial change only after a power of its false-alarm horizon, and an unclipped one may never notice it. On five LLMs answering multi-hop questions over edited facts, ExpeL- and ACE-style counters serve 7.07.0–13.5×13.5× more stale answers than certified memories given the same information, and on code tasks with an exact verifier 3535–73×73× more. Safety also has a price. When facts change and each memory is judged only by its own verified outcomes, one number, the certification load, sets how much reasoning any safe memory can skip, and under Zipf demand it yields a scaling law. Our recipe weighs each memory’s failures by its own likelihood ratio, certifies its utility against live reasoning, and restarts the certificate when facts change, with the likelihood-ratio detector as the safety net for unannounced changes; for large edits it also invalidates the memory. 1 Introduction Figure 1: Trusting a contracted memory is a sequential test. (a) Top: the lifecycle of a contracted memory (a macro). In shadow mode a betting certificate scores the macro’s verified answers and folds it (one-step answers) once its wealth reaches 1/δ1/δ; a likelihood-ratio CUSUM (LR-CUSUM) alarm or an edit notice retires it, and the next candidate restarts the test. Middle: both tests on one real macro (Qwen3-8B, KQA-Stream; per candidate, no notices). Its log-wealth reaches ln20 20 (δ=0.05δ=0.05) after 33 labels; an edit then drops its reliability from 1.001.00 to 0.370.37, below break-even (r⋆=0.49r =0.49), and the LR-CUSUM crosses h=7h=7 on the fourth label. Bottom: its timeline (hatched: certification); ExpeL- and ACE-style counters have not retired it by stream end (2,427 stale answers, against 29). (b) Exact detection delay D against uses to a false retirement A for three clipped counters (score per failure/success), failure rate 0.05→0.250.05→ 0.25: the symmetric ±1± 1 rule is slow by a power of A; likelihood-ratio weights (+7/−1+7/-1) track the optimum. q~ q: the failure rate a rule implicitly tests (Remark 1). (c) The exact coverage gap M (share not answered in one step) of a restarted test for 2,520 settings (dots) lies on the limit Fπ(Θ)F_π( ) (lines). Everyone builds reasoning memories. Thought templates (Yang et al., 2024a), reasoning and workflow memories (Ouyang et al., 2026; Wang et al., 2025b; Suzgun et al., 2026), behavior handbooks (Didolkar et al., 2025), skill libraries (Wang et al., 2024a; Wang et al., 2025a), answer caches (Bang, 2023; Schroeder et al., 2026), and the memories that agents share (Zhang et al., 2025a) or rewrite themselves from (Zhang et al., 2026a) all do one thing: they compile a derivation of m≥2m≥ 2 steps into a one-step contracted memory for later instances of the problem. We call this reasoning contraction. Trusting such a memory is a sequential test, bound by the limits of change detection and certification. Live reasoning is costly and fragile. LLM reasoners buy accuracy with computation: voting over sampled chains (Wang et al., 2023), tree search (Yao et al., 2023a), tool calls (Yao et al., 2023b) and self-critique (Shinn et al., 2023; Madaan et al., 2023). Every step is another chance to err. On the unedited facts of our multi-hop benchmark, Qwen3-8B’s live accuracy falls from 0.630.63 at two hops to 0.310.31 at three and 0.190.19 at four (Appendix D). A verified contracted memory answers in one step; on our edited streams it raises Qwen3-8B’s accuracy from 0.3790.379 to 0.6480.648 with fewer calls (Table 3). The price is a new failure mode shared by all these systems: when the facts behind a memory change, it keeps answering, wrongly. This is the utility problem of macro learning (Minton, 1990), now for LLM memories over changing facts. Memories are trusted by counters, which are badly tuned change detectors. ExpeL and ACE vote memories up and down (Zhao et al., 2024; Zhang et al., 2026b), and validation-gated memories clip a Hebbian strength (Singh and Yu, 2025). Every clipped counter is a cumulative-sum (CUSUM) change detector in disguise, testing an implied change that nobody chose (Section 3). A symmetric counter assumes that a stale memory is as often wrong as a fresh one was right, so counting votes equally is provably the wrong detector for partial staleness: when an edit leaves a memory failing less than half the time, the counter’s evidence drifts the wrong way, and its delay grows as a power of its false-alarm horizon, not its logarithm (Theorem 1, Figure 1b). Unclipped counters are worse: credit banked while the memory was healthy must be spent before retirement. On KQA-Stream, multi-hop questions over controlled knowledge-base edits, ExpeL- and ACE-style counters leave about half of the same 156156 stale memories in service at stream end, against 2%2\% for a likelihood-ratio rule, and need about 60×60× more uses to retire them (restricted mean; Figure 2). On five open LLMs they serve 7.07.0–13.5×13.5× more stale answers than our certificate given the same information (no notices; Table 2); on code, larger still (Section 6). Safety has a price. Can a memory be safe and still skip most reasoning? One number decides: the certification load Θ=νln(1/δ)/(pKL) =ν (1/δ)/(pKL), which combines the rate of change (ν), the risk tolerated (δ), the audit rate (p) and how much one verified outcome reveals (KLKL). Whatever its algorithm, every safe memory must, in the worst case and as δ→0δ→ 0, leave a share Fπ(Θ)F_π( ) of queries to multi-step reasoning, and a restarted test achieves it (Theorem 3, Figure 1c). Under Zipf demand it scales as κsΘ1−1/s _s ^1-1/s (Corollary 4): popular problems can be contracted; rare ones change before they can be trusted. The recipe. Three changes turn a counter into a certificate, a rule that lets a memory answer in one step only while the risk of trusting a bad one stays below δ (Section 5). (i) Weigh each memory’s failures against its successes by its own likelihood ratio, set by its healthy and stale failure rates; on KQA-Stream this detects staleness in 1.31.3–5.9×5.9× fewer uses than the best tested fixed weighting. (i) Certify utility, not accuracy, by betting on the paired difference with live reasoning on the same query, with no estimate of live accuracy. (i) Restart a memory’s certificate whenever its answers or the live reasoning it competes with change, so every trust decision rests on current evidence. Where edits are substantial, our deployment also invalidates memories an edit announces, and the likelihood-ratio detector is the safety net for unannounced edits; it cuts stale answers 99–12×12× against invalidation alone. On five models it trusts under one below-break-even memory per stream (with notices; counters: 111111–595595), and, scoring answers alone, makes no false certification at δ≤0.1δ≤ 0.1. Agent memories and skill libraries with a verifier should therefore adopt these changes, and memory benchmarks should edit facts. Contributions and what is new. Our results apply when facts can change and each memory can be judged only by its own verified outcomes. (1) A concept: reasoning contraction, one lens on templates, skills, memories and caches, whose two risky decisions, trust and retirement, are sequential tests bound by the limits of change detection and certification (Figure 1a); done safely, KQA-Stream’s symbolic macros add 1414–3333 points over live reasoning. (2) Two theorems: the phase diagram of counter maintenance (equal votes: delay ≍A0.373 A^0.373, not lnA A, at failure rate 0.05→0.250.05→ 0.25), with the learning rates that fix it (Theorems 1 and 2), and the certification-load limit with its scaling law κsΘ1−1/s _s ^1-1/s, matched within 0.9450.945–1.025×1.025× by a restarted test (Theorems 3 and 4). (3) A certificate: a per-template certify–retire–restart lifecycle from standard anytime-valid tools (Yu et al., 2026b), tested on KQA-Stream (with invalidation, no stale answers under notices; 7.07.0–13.5×13.5× fewer than counters without), natural Wikidata edits and code. Where memory systems vote and anytime-valid routing certifies one global threshold, we test each memory’s trust over its lifecycle: memory systems with a verifier should give each memory its own weight and certificate, and price skipped reasoning by Θ . 2 Contraction as a memory lifecycle Queries, templates, macros. A reasoner serves a stream of queries. Each query instantiates a template i, a canonical derivation skeleton such as a relation path in a knowledge base, a lemma chain, or a tool-call sequence. Template i is drawn with probability πi _i and has a live derivation of mi≥2m_i≥ 2 steps (LLM calls). A macro for template i is a one-step operator formed from a verified derivation: a materialized composed relation, a stored lemma, or a distilled direct-answer note. Its reliability r is the probability that it answers a random instance of i correctly. On each query the system answers live (cost m, correct with probability a), folded (cost 11, answer from the trusted macro), or in shadow mode (live answer, macro evaluated alongside). A verifier (exact match, execution, or a unit test) labels the macro’s answer on audited queries, a fraction p∈(0,1]p∈(0,1] of the template’s shadow or folded queries. With error price L (in steps), a folded query costs 1+L(1−r)1+L(1-r) and a live one m+L(1−a)m+L(1-a), so folding pays if and only if r>r⋆≔a−(m−1)/L.r>r a-(m-1)/L. (1) The lifecycle. Contraction is a memory lifecycle (Figure 1a): form a candidate from a verified trace, certify it in shadow mode, fold it, monitor it, retire it when it goes stale, and re-form. Two decisions carry all the risk: when may a candidate be trusted (certification) and when has a trusted macro gone stale (retirement). Verifier outcomes Y∈0,1Y∈\0,1\ of a macro are conditionally Bernoulli given the past, with mean equal to its current reliability; this allows adaptive usage and drift. A macro is stale once an edit has raised its failure rate: healthy macros fail with probability q0q_0, stale ones with q1>q0q_1>q_0. Maintenance rules in current systems. ExpeL gives a new insight importance 22, adds 11 per up-vote or edit, subtracts 11 per down-vote and deletes it at 00 (Zhao et al., 2024). ACE keeps unbounded helpful/harmful counts, and its released code flags a bullet when harmful ≥ helpful and harmful >0>0 (Zhang et al., 2026b); our ACE-style baseline treats a flag as retirement. We idealize their LLM-issued votes as one vote per verified use. Validation-gated Hebbian memories clip a strength to [0,1][0,1] with time decay (Singh and Yu, 2025); we analyze the symmetric variant c←min1,c+η+c← \1,c+ _+\ on success and c←c−η−c← c- _- on failure, retiring below τ. 3 Memory counters are change detectors A score rule assigns a bounded score f(Y)f(Y) to each verifier outcome, keeps Sn=max0,Sn−1+f(Yn),S_n= \0,\,S_n-1+f(Y_n)\, (2) and retires the macro at the first n with Sn≥hS_n≥ h. Clipped Hebbian confidence has this form. With S=(1−c)/η−S=(1-c)/ _- the score is +1+1 on a failure and −ρ-ρ on a success, ρ=η+/η−ρ= _+/ _-, and the clip at full confidence is the reflection at 00 (Appendix A). Let P0,P1P_0,P_1 be the outcome laws of a healthy and a stale macro, with 0f<0<P0(f>0)E_0f<0<P_0(f>0), and let θj>0 _j>0 solve jeθjf=1E_je _jf=1 (the Cramér–Lundberg adjustment coefficient under PjP_j). We measure a rule by A, the mean number of healthy uses before it falsely retires a macro, and by D, its worst-case number of uses to detect a change (Lorden, 1971). Remark 1 (Implied alternative). Every score rule (2) is exactly Page’s CUSUM (Page, 1954) for the change P0→P~0≔eθ0fP0P_0→ P_0 e _0fP_0 and for no other change, as for quality-control charts (Munford, 1980; Brook and Evans, 1972). The symmetric Hebbian rule (ρ=1ρ=1) thus tests failure rate q0q_0 against its mirror 1−q01-q_0: it assumes that a stale memory is as wrong as a fresh one was right (reliability 0.95→0.050.95→ 0.05), whereas a fact edit may corrupt only part of a template’s instances (reliability 0.95→0.750.95→ 0.75). Theorem 1 (Phase diagram of maintenance rules). Let μ1=1f _1=E_1f and γ=θ1/θ0γ= _1/ _0. As A→∞A→∞, μ1>0: _1>0: D=lnAθ0μ1+O(1); D= A _0 _1+O(1); μ1=0: _1=0: D=(lnA)2(θ0σ1)2+O(lnA),σ12=Var1f; D= ( A)^2( _0 _1)^2+O( A),\ _1^2=Var_1f; μ1<0: _1<0: D≍Aγ if P1(f>0)>0, D A^γ\ if P_1(f>0)>0, where γ<1γ<1 if and only if 1eθ0f>1E_1e _0f>1, e.g. whenever q1>q0q_1>q_0 for Bernoulli rules. In the first regime the efficiency relative to the optimal delay lnA/KL(P1∥P0) A/KL(P_1\|P_0) (Lorden, 1971) is Eff=θ0μ1KL(P1∥P0)=1−KL(P1∥P~0)KL(P1∥P0),Eff= _0 _1KL(P_1\|P_0)=1- KL(P_1\| P_0)KL(P_1\|P_0), (3) so Eff≤1Eff≤ 1, with equality if and only if f is proportional to the log-likelihood ratio. When it applies: any clipped counter with a bounded score per verified outcome, when outcomes are i.i.d. before and after one change (constants in Appendix A). Why three regimes. After a change the score drifts at rate μ1 _1. With positive drift it climbs to the threshold h≈lnA/θ0h≈ A/ _0 in linear time, hence lnA A. With zero drift it diffuses and needs time h2h^2. With negative drift it must make a rare excursion, like a false alarm but at the post-change exponent θ1<θ0 _1< _0, hence eθ1h=Aγe _1h=A^γ. The sign is that of θ0μ1=KL(P1∥P0)−KL(P1∥P~0) _0 _1=KL(P_1\|P_0)-KL(P_1\| P_0): a rule detects a change quickly only if the change moves the outcome law closer to the rule’s implied alternative than to the healthy law. For the symmetric rule this requires q1>12q_1> 12. Partial staleness (reliability 0.95→0.750.95→ 0.75) gives γ=ln3/ln19=0.373γ= 3/ 19=0.373 (Figure 1b). The logarithmic regime is known (Section 7); the power-law regime and the unified diagram are new. Why it matters: under partial staleness, equal votes wait AγA^γ uses when clipped, not lnA A, and with positive probability forever when unclipped (Remark 2). Corollary 2 (Likelihood-ratio learning rates). A clipped Hebbian rule is the likelihood-ratio CUSUM for q0→q1q_0→ q_1, optimal at the false-alarm horizons its thresholds attain, if and only if η+η−=ρ(q0,q1)≔ln1−q01−q1/lnq1q0. _+ _-=ρ(q_0,q_1) 1-q_01-q_1 / q_1q_0. (4) When it applies: a memory whose healthy and stale failure rates q0<q1q_0<q_1 can be estimated (if only an interval for q1q_1 is known, tuning at its lower end is first-order minimax; Appendix A). We use the Bernoulli likelihood-ratio CUSUM (LR-CUSUM) of Reynolds and Stoumbos (1999) to set memory learning rates. Under partial staleness (q0=0.05q_0=0.05, q1=0.25q_1=0.25), ρ=0.147ρ=0.147: a failure must weigh about 77 successes, not 11 as in ExpeL and ACE. Scores +7/−1+7/-1 detect in 4141 and 5454 uses at A≈1.6×105A≈ 1.6× 10^5 and 3×1063× 10^6, 5.7×5.7× and 13×13× faster than the symmetric rule (Figure 1b). Why it matters: weighting each memory by its own ρ makes it the optimal detector of its q0→q1q_0→ q_1; if memories’ ρ differ, no global weight is optimal for all. Remark 2 (Unclipped counters). An unclipped counter such as ExpeL’s or ACE’s (Section 2) rarely retires a healthy memory (A=∞A=∞) but has no finite worst-case delay: credit banked before a change must be spent first, at rate 2q1−12q_1-1. Under partial staleness (q1<12q_1< 12) the stale memory is never retired with probability at least 1−2q11-2q_1 (ACE) or 1−(q1/(1−q1))21-(q_1/(1-q_1))^2 (ExpeL), i.e. 0.500.50 and 0.890.89 for reliability 0.95→0.750.95→ 0.75 (Proposition 9 in Appendix A). 4 The price of safety: how far can reasoning contract? How large a share of queries can any safe system answer in one step when facts keep changing? Model. Queries of template i arrive at rate πi _i (∑iπi=1 _i _i=1), and each is audited with probability p; an audit reveals one verifier outcome of the candidate it evaluates. Each template’s facts change at rate ν. After a change the current macro is invalid, and a candidate must earn trust in the new epoch. A candidate is good if its reliability is ≥r1≥ r_1 and bad if it is ≤r0<r1≤ r_0<r_1 (e.g. r0=r⋆r_0=r of Equation 1). A fold event is a time when the system starts answering a template in one step with some candidate, formed in this or an earlier epoch. The coverage gap M is the long-run fraction of queries not answered in one step by a valid, certified macro. Definition 1 (δ-safety). A policy is δ-safe if, for every epoch, every history before it and every outcome law within it, the probability that the epoch contains a fold event of a candidate that is bad in that epoch is at most δ. Safety is per epoch, not per candidate: folding each of many candidates blindly with probability δ is safe per candidate but eventually folds a bad one. A sequential probability ratio test (SPRT), Λn=∑k≤nlnPr1(Yk)Pr0(Yk) _n= _k≤ n P_r_1(Y_k)P_r_0(Y_k), that folds when Λn≥ln(1/δ) _n≥ (1/δ) and restarts at each change is δ-safe by Ville’s inequality (Wald, 1945; Ville, 1939). Theorem 3 (Fundamental limit of safe contraction). Let KL=KL(r1∥r0)KL=KL(r_1\|r_0) (Bernoulli), define the certification load Θ≔νln(1/δ)pKL, ν\, (1/δ)p\,KL, (5) and let Fπ(Θ)≔∑iπi(1−e−Θ/πi)F_π( ) _i _i (1-e^- / _i ). (i) No safe policy does better: every δ-safe policy has M≥(1−εδ)Fπ(Θ)M≥(1- _δ)F_π( ), where εδ→0 _δ→ 0 as δ→0δ→ 0, whether or not changes are announced and however many candidates it forms. (i) A restarted test achieves it: the SPRT restarted at each change is δ-safe and has M≤(1+ln(r1/r0)/ln(1/δ))Fπ(Θ)M≤(1+ (r_1/r_0)/ (1/δ))\,F_π( ). When it applies: facts change at random times, each audit reveals one outcome, only audited outcomes are informative (shadow agreement excluded; Theorem 22), and good memories have reliability only r1r_1 (the worst case; exact model in Appendix B). Why Fπ(Θ)F_π( ). A good candidate cannot be told apart from a bad one before its log-likelihood ratio reaches ln(1/δ) (1/δ), which takes n⋆=ln(1/δ)/KLn = (1/δ)/KL audited uses, i.e. time n⋆/(pπi)n /(p _i). A query is folded only if its epoch is older than that. Epoch ages are exponential with rate ν, so this happens with probability e−Θ/πie^- / _i. Popular templates certify quickly and are almost always covered; templates with πi≲Θ _i change before certification. A change of measure makes this rigorous (Lai, 1981; Tartakovsky et al., 2014; Kaufmann et al., 2016). Under Zipf demand it becomes a scaling law: Corollary 4 (Scaling law of one-step reasoning). If πi=i−s/ζ(s) _i=i^-s/ζ(s) with s>1s>1, then for all Θ>0 >0 κsΘ1−1/s−Θ≤Fπ(Θ)≤κsΘ1−1/s, _s ^1-1/s- \ ≤\ F_π( )\ ≤\ _s ^1-1/s, (6) with κs=Γ(1/s)/((s−1)ζ(s)1/s) _s= (1/s)/((s-1)ζ(s)^1/s) (κ2≈1.38 _2≈ 1.38). Hence the optimal gap is κsΘ1−1/s(1+o(1)) _s ^1-1/s(1+o(1)) for small δ and small load. Reading the law. Drift, risk, audit rate, reliability gap and verifier noise (through the KLKL of the observed channel) enter only through Θ , and the exponent 1−1/s1-1/s is set by the demand tail alone. Why it matters: in this model no safe algorithm, candidate supply or notice beats the limit as δ→0δ→ 0; for given demand only a lower load contracts more. Figure 1c checks the law exactly: the gap of the restarted SPRT, computed by dynamic programming for 2,520 settings of reliabilities, verifier noise, δ∈[10−32,0.03]δ∈[10^-32,0.03], audit and drift rates, has MSPRT/Fπ(Θ)∈[0.945,1.025]M_ SPRT/F_π( )∈[0.945,1.025], and the fitted log-log slopes are within 0.0080.008 of 1−1/s1-1/s (finite-δ bounds: Appendix B). 5 The recipe: a certificate that needs no live accuracy The SPRT of Theorem 3 tests r≤r⋆r≤ r , but r⋆r involves the unknown live accuracy a (Equation 1), and plugging in an estimate breaks the guarantee. Proposition 5 (Plug-in tests are not level δ). If the SPRT uses r0=a^−m−1Lr_0= a- m-1L and r1=a^r_1= a with a a estimated from nan_a live runs, its false-fold probability at break-even can exceed δ. At a=0.9a=0.9 and m−1L=δ=0.05 m-1L=δ=0.05 it is 0.340.34, 0.250.25 and 0.080.08 for na=20n_a=20, 100100 and 10001000 (exact; Proposition 18). Following testing by betting (Shafer, 2021; Waudby-Smith and Ramdas, 2024), with inverse-propensity weights as in B-PAC (Yu et al., 2026b), we bet on the paired utility of each shadow evaluation, with live reasoning on the same instance as control: Ut=CtL−CtM+L(YtM−YtL)U_t=C^L_t-C^M_t+L(Y^M_t-Y^L_t), the steps saved plus L times the correctness gained, where CtL,CtMC^L_t,C^M_t are the costs and YtL,YtM∈0,1Y^L_t,Y^M_t∈\0,1\ the verified correctness of the live and macro answers; Zt=Ut/B≥−1Z_t=U_t/B≥-1 for a known bound B. Theorem 6 (Betting certificate). Fold when the wealth of predictable bets on the inverse-propensity-weighted ZtZ_t reaches 1/δ1/δ. (i) Validity. A macro that is never more useful than live reasoning is folded with probability at most δ, whatever the live accuracy and however live and macro errors are correlated. (i) Rate. If the unweighted ZtZ_t are i.i.d. with law PZP_Z and audits have rate p, certification takes (1+o(1))ln(1/δ)/(pKLinf(PZ))(1+o(1)) (1/δ)/(p\,KL_ (P_Z)) uses, and no δ-safe rule that sees only audited utilities is faster (Kaufmann et al., 2016; Garivier and Kaufmann, 2016). Here KLinf(P)≔infKL(P∥Q):QZ≤0KL_ (P) \KL(P\|Q):E_QZ≤ 0\ over laws Q on the same bounded range: the divergence from PZP_Z to the nearest useless macro. When it applies: audits have known probabilities, bounded below, independent of the hidden outcome (exact conditions in Appendix C). Under the null, reaching 1/δ1/δ is rare (Ville’s inequality); a useful macro grows log-wealth by KLinf(PZ)KL_ (P_Z) per audited pair (Honda and Takemura, 2010). For a certificate restarted at each change, Theorem 3 holds with KLinf(PZ)KL_ (P_Z) for KLKL and unknown a (Corollary 20). Unlike a discordant-pair test (Shawn, 2026), it can certify macros less accurate than live but above break-even (Theorem 19). From certificate to deployment. The restart-only certificate (Table 1) runs a template’s j-th test at level δj=6δ/(π2j2) _j=6δ/(π^2j^2), so that all its tests together spend at most δ; it restarts, at the next level, at each notice that the template’s answers changed, and retires a folded macro by its own LR-CUSUM (Corollary 2). Our recommended deployment (“ours”) also invalidates and re-forms folded macros at each notice, so the LR-CUSUM is the safety net for missed notices. When it applies: both are δ-safe per epoch for their own null, paired utility ≤0≤ 0, when notices cover every change of that null, including changes of live reasoning (Proposition 23). So one δ bounds false certification, a fold of a candidate whose expected paired utility is ≤0≤ 0 at the fold, in every epoch, however many successive candidates and edits; later staleness is left to retirement (Section 3). Certificate Needs Guarantee Restart only: ledger δj _j, restarts notices† δ-safe per epoch (Proposition 23); as run, growth KLinf/(1.03CLOSEKL_ /(1.03–OPEN1.07)1.07) on the laws checked, plus 2lnj+O(1)2 j+O(1) nats (Section C.2) Ours (recommended): + invalidate on notice notices†, some missed as above; a missed notice adds the carried-over term of Proposition 23(i) No notice: discard at own alarm none δ-safe for folds on post-change evidence (Proposition 25); not per epoch if αir0>δ _ir_0>δ (Proposition 24) Per candidate: level δ each none per candidate only (Figure 1a, code) Table 1: Restarting at each notice makes certification safe per epoch. Bold: ours. All retire folded macros by LR-CUSUM (Corollary 2). The optimum, an SPRT or grid bet restarted at known change times, attains Fπ(Θ)F_π( ) (Theorems 3 and 20); as run is a rule’s own growth rate; measured rates are in Section 6. †Of every change of the null, including live reasoning’s (Section D.5.1). 6 Experiments Benchmark. KQA-Stream is built from the KQA Pro knowledge base (Cao et al., 2022): 300 templates of 2–4 relation hops with 60 instances each (18k LLM-verbalized questions) and three sequences of 24 fact edits (Appendix D). The live reasoner is a hop-by-hop agent that chooses one relation per LLM call. A macro materializes a verified trace’s relation path as an anchor-to-answer table, reached through one routing call; it goes stale when edits change the facts it cached. A compiled plan stores the path and re-executes it on the current knowledge base. Every LLM call is recorded once per model, with exact selective re-runs after edits, and policies are replayed on 200k-query Zipf streams (s=1.2s=1.2; 10 seeds × 3 edit sequences; 10% random audits), so all policies see identical queries, edits and audit coins. We use five open models: Qwen3-8B and Qwen3-4B (Yang et al., 2025) (thinking disabled), Granite-3.3-8B (IBM Granite Team, 2025), Mistral-7B (Jiang et al., 2023) and OLMo-3-7B (Team OLMo, 2025). Prompts, seeds and scripts are in our code release. Policies. Fold once folds a verified candidate at formation and never retires it, like a semantic cache. ExpeL- and ACE-style baselines transplant each system’s retention rule onto our macros, one vote per audited outcome. Detector baselines certify with the plug-in SPRT and retire with an LR-CUSUM; the optimistic one folds at formation. The certificates of Table 1 run at δ=0.05δ=0.05; a notice is an edit that changes the gold answer of a query routed to the template; a vCache-style cache re-implements Schroeder et al. (2026) (Section D.2.4; more variants: Table 5). Metrics: stale answers are wrong one-step answers that were right when the macro was formed; a bad fold trusts a candidate below break-even; a stale episode runs from the edit that makes a folded macro stale to its retirement or the stream end; J20J_20 is LLM calls per query plus 20m¯20 m times the error rate (m¯ m: mean live cost; L=20m¯L=20 m in Equation 1; lower is better). Audits query the knowledge base at no LLM-call cost. stale per 10410^4 queries unretired at stream end bad folds Model ExpeL ACE ours∘ ExpeL ACE LR counters ours Qwen3-8B 967 946 72 53% 53% 3% 284–400 0.0 Qwen3-4B 795 783 85 57% 51% 0% 111–148 0.1 Granite-8B 657 650 82 22% 22% 0% 282–369 0.4 Mistral-7B 298 294 42 25% 25% 0% 486–595 0.6 OLMo-3-7B 480 445 51 64% 55% 9% 172–211 0.3 Table 2: Counters serve 7.07.0–13.5×13.5× more stale answers on every model, even when ours gets no edit notice. ∘Ours with every notice missed, i.e. the counters’ information; with notices it serves none. Unretired: share of the stale episodes of Figure 2 (66, 53, 18, 8 and 11) still in service at stream end. Bad folds: per stream (ours: with notices). Bold: ours. Counters leave stale memories in service. As Remark 2 predicts, counters serve 7.07.0–13.5×13.5× more stale answers than ours on every model, even with every notice missed (Table 2). To isolate detection, we replay every macro that restart only folds, which only detection retires, under each rule with the same audits: 156156 stale episodes (Section D.3.2). Counters leave 50%50\% and 47%47\% in service at stream end against 2%2\% for the LR-CUSUM, with restricted mean delays of 4,0694,069 and 3,9493,949 uses against 6767 (Figure 2); each episode costs counters about 1,3501,350 stale answers, against 38.538.5. Loss hides this: their J20J_20 is close to ours because, skipping certification, they answer 2828–76%76\% of queries in one step (ours: 2020–53%53\%), so benchmarks must count stale answers. Figure 2: Counters leave half the stale memories in service at stream end; likelihood ratios retire them about 60×60× sooner (restricted mean). Kaplan–Meier share of the 156 stale memories not yet retired (same audits for all rules; counters start at the fold without credit). Legend: share still in service when the stream ends (open episodes, so it differs from the curve’s tail at 10410^4 uses) / restricted mean delay (≤104≤ 10^4 uses). Bands: 95% template bootstrap. Detection follows the phase diagram. A false retirement (FR) retires a macro still above break-even. Staleness here is nearly total (mean q1=0.75q_1=0.75–0.900.90, q0=0.04q_0=0.04–0.240.24), and Theorems 1 and 2 predict three things, all observed. (i) The symmetric rule detects, but slowly: with q1>12q_1> 12 its delay is logarithmic, but the LR-CUSUM needs 3.73.7–6.1×6.1× fewer uses at 0.80.8–55 FR per 10410^4 healthy uses (Figure 6). (i) Per-memory weights beat the best tested global one, as q0q_0 varies: 1.31.3–5.9×5.9× fewer uses (significant on three of five models; Section D.3.6); that best ratio is itself small (below 1:11:1 on four models; predicted 1/ρ1/ρ: 0.670.67–2.142.14; Figure 5). (i) Partial staleness moves the optimum up: on 217217 real partially stale episodes (predicted 1/ρ=4.81/ρ=4.8) the best tested weight is 55–77, and the symmetric rule needs 1.21.2–1.5×1.5× more uses (Section D.3.5). The theory thus ranks the rules on real LLM errors. Restarted certificates are safe in every epoch. Scoring answers alone, as the certificate’s paired test does (Section D.4.1), ours makes no false certification (Section 5) at δ≤0.1δ≤ 0.1 (00 of 5,7795,779 at-risk epochs at δ=0.05δ=0.05) and stays far below δ up to 0.50.5, whether “bad” means null at the fold or below break-even (Figure 3); counting routing errors as wrong, 0.26%0.26\% of at-risk epochs have a bad fold at δ=0.05δ=0.05 (Section D.4.2). Restarts remove the 1111 bad folds that candidates good at formation carry across edits without them (standard scoring; Section D.3.3). Although 16%16\% of template-versions change live reasoning unannounced, none of the 9,8359,835 answer-scored folds (22 of 8,2948,294 under standard scoring) rests on a test spanning such a change (Section D.5.1). δ controls which memories are trusted; how often a trusted memory errs is set by r⋆r (1010–11%11\% of one-step answers, mostly from macros above break-even). Figure 3: False certifications stay far below δ. At-risk epochs with a fold that violates (a) the certificate’s own null, the event δ bounds, or (b) break-even r⋆r ; answers scored alone; five models. Without notices, folds on null evidence stay at zero up to δ=0.1δ=0.1, and all folds more than 10×10× below δ. Bars: 95% template bootstrap. Policy Acc Calls Stale Bad J20J_20 no memory live always .379 3.26 0 0 43.8 no retire fold once .534 1.45 1290 34 31.8 compiled plan, fold once .639 1.45 95 34 24.8 vCache-style, δc=.02 _c=.02 .377 2.86 30 – 43.5 counters ExpeL-style counter .613 1.75 967 284 27.0 ACE-style counter .618 1.86 946 400 26.7 detectors SPRT + LR-CUSUM .622 2.64 54 0.7 27.3 optimistic + LR-CUSUM .632 1.58 573 82 25.5 certified ours (recommended)⋆ .648 2.56 0 0.0 25.5 restart only .642 2.57 66 0.0 26.0 no notice (own detector) .650 2.52 70 0.1 25.3 compiled plan + betting .668 2.46 16 0.0 24.1 Table 3: Certification removes bad folds and invalidation removes stale answers, with fewer calls than live reasoning (Qwen3-8B, 30 streams of 200k queries). Calls: LLM calls per query. Stale: per 10410^4 queries. Bad: bad folds per stream (–: no macros). J20J_20: loss, lower is better (m¯=3.26 m=3.26). Bold: ours. ⋆Perfect notices; with every notice missed it serves 7272 stale answers per 10410^4 queries. Compiled plans (with restarts) gain +2.6+2.6 points [1.71.7, 3.83.8] over restart-only macros (paired 95% template bootstrap). Certified contraction pays. On Qwen3-8B, ours raises accuracy from 0.3790.379 (live) to 0.6480.648 with 21%21\% fewer LLM calls, and certified compiled plans to 0.6680.668 (Table 3), because a macro executes a verified path that live reasoning re-derives with errors. Uncertified reuse (fold once) serves 1,2901,290 stale answers per 10410^4 queries; certification cuts this 20×20×, invalidation to zero. Across models ours adds 1414–3333 points over live reasoning (Table 6). The detector matters exactly where notices fail. With perfect notices ours serves no stale answers and matches restart only in accuracy (+0.24+0.24 points [−0.24-0.24, +0.81+0.81], pooled). With 80%80\%, 50%50\% or no edits announced, invalidation alone serves 125125, 325325 and 810810 stale answers per 10410^4 queries, ours 1313, 2828 and 6666 (Section D.4.2), 99–12×12× fewer: invalidation needs the detector wherever notices can be missed. Natural edits: safety holds, and mild staleness shows its price. On a 1992–2016 timeline of Wikidata temporal qualifiers (four models), where changes are small and partial (median 3.3%3.3\% of a template’s answers per year), ours keeps bad-fold epochs below δ at every level, the LR-CUSUM detects in 1.11.1–1.6×1.6× fewer uses than the best tested fixed weight, and counters leave 84%84\% of the same stale memories in service against 37%37\% (Sections D.4.4 and D.5.3). The price of safety (Section 4) shows: counters make 149149–207207 bad folds per stream against 0.10.1 and serve 3×3× more stale answers than restart only, yet are 3.53.5 points more accurate because folding pays when staleness is mild; for the same reason invalidation costs ours 1.91.9 points against restart only. On code, counters serve 3535–73×73× more stale answers. On 521521 EvalPlus tasks (Liu et al., 2023) (five models), live reasoning writes, tests and repairs a program; a memory is a verified program rerun on its task’s later queries; cost counts steps, an execution being one. Memories go stale under announced requirement revisions and unannounced Python API changes, which also change live reasoning and so exercise the safety net. Ratios are per query, against our certificate given the counters’ information (per candidate, no notices), on all five models (3838–86×86× in LLM-call accounting). Here most stale memories never failed before (q0=0q_0=0, so ρ→0ρ→ 0), and the per-memory LR-CUSUM leaves them folded for 10×10× fewer uses than the best fixed weight at matched false retirements (7.67.6–11.4×11.4× per model, weights up to 5000:15000:1; Section D.6). Ours folds a below-break-even candidate in 1212 of 946,678946,678 epochs under test (counters: 4.84.8–5.4%5.4\%) and gains 8.78.7 accuracy points [7.17.1, 10.510.5] over live reasoning, with lower J20J_20 in both units: it uses 23%23\% fewer steps, while its shadow tests add 17%17\% LLM calls. Zero-utility ties (8484–86%86\% of audited pairs in LLM-call accounting) make audits nearly uninformative and halve one-step coverage (0.61→0.320.61→ 0.32), as Theorem 6(i) predicts: ties shrink KLinfKL_ and slow certification. 7 Related work and what is new Detection. A mismatched CUSUM with positive drift, our first regime and (3), has the known delay lnA/(θ0μ1) A/( _0 _1) (Molloy and Ford, 2017; Cooper and Meyn, 2024). We add the zero- and negative-drift regimes, including the finite power law Aθ1/θ0A _1/ _0 where Wu and Yang (2016, Thm. 2) state an infinite delay (counterexample: Appendix A), read deployed counters’ implied alternative and set learning rates by it. Limit. Ours extends crawler freshness (Azar et al., 2018), where one refresh restores a page, to items whose trust needs ln(1/δ)/KL (1/δ)/KL audited uses; the Zipf law is an occupancy sum (Karlin, 1967) at the load Θ . Certificate. With standard betting, inverse-propensity weights and anytime validity, B-PAC (Yu et al., 2026b) certifies one global routing threshold; new is a per-template lifecycle that certifies, retires and restarts. Concurrent work extends reasoning contraction to self-modification (Zhou et al., 2026a) and shared multi-agent memory (Zhou et al., 2026b). More: Appendix F. 8 Conclusion Reusing a reasoning memory safely is a sequential test, and vote counters (ExpeL, ACE) are change detectors tuned to the wrong change. Three rules follow. (1) Weigh a memory’s failures by its own likelihood ratio: at failure rate 0.05→0.250.05→ 0.25 a failure weighs about 77 successes, not 11. (2) Certify utility against live reasoning; the load Θ prices skipped reasoning in the worst case. (3) Restart at every notice (00 false certifications in 5,7795,779 at-risk epochs at δ=0.05δ=0.05, answers scored alone) and, for substantial edits, invalidate too, with the detector as safety net (99–12×12× fewer stale answers when notices are missed). Memory benchmarks should edit facts and count stale answers. Limitations Scope. Our results concern memories whose answers an exact verifier checks, such as knowledge-base execution or unit tests. The evidence comes from three domains and five open models of up to 8B parameters: multi-hop reasoning over a knowledge graph with controlled fact edits, a natural timeline of Wikidata temporal qualifiers (four models; Section D.4.4), and EvalPlus code tasks, whose memories go stale under requirement revisions and re-enacted Python API changes (Section D.6). Each instance–version is decoded once; seed variance is measured on a subset (Section D.4.3). The theorems assume that each audit reveals one verified outcome whose law is set by the memory’s current reliability, and that facts change at random times; Appendices A, B and C state the exact conditions and Appendix D the replay setting. Per-epoch safety needs change notices that cover every change of the certificate’s paired-utility null, including changes of live reasoning; our notices cover answer changes (Section D.5.1). Some change information is necessary on every template whose epochs contain an audit with probability above δ/r0δ/r_0 (Proposition 24); without an edit log, restarts triggered by the memory’s own detector keep every decision that rests on post-change evidence safe (Proposition 25). Future work. Five directions extend this scope. (i) A formation-aware limit: on real streams, coverage also depends on how often the live agent produces a verified trace from which a candidate can form, and a limit that models formation would predict real-system coverage. (i) Noisy and LLM-judge verifiers, which would bring text memories into reach; the certification load already prices noise through the observed channel, and a certificate built for noisy paired labels is the next step (Section D.2.3). (i) Longer and denser natural edit histories, such as full knowledge-base revision logs, where change notices arrive with their own delays and errors, and real library drift on benchmarks with version-specific APIs, where staleness reaches more programs than re-enacted deprecations do. (iv) Larger models. (v) Open problems in the theory: a square-root law for adaptive audit budgets, a joint detection–certification limit that prices stale answers, the exact optimal certifier for one template, near-optimal retirement for an unknown post-change rate by parallel or GLR rules, and the agreement-aware version of the limit. Ethical considerations This work analyzes when stored reasoning may be trusted, and its main practical implication is safety-positive: it bounds how often a memory-augmented reasoner serves stale or spurious shortcuts, and it shows that common counter rules may never retire a stale memory. The datasets we use (KQA Pro, C BY-SA 4.0; MuSiQue, C BY 4.0; Wikidata, C0; EvalPlus, Apache-2.0, with MBPP, C BY 4.0, and HumanEval, MIT) are public and contain no personal data beyond public facts about notable entities; our synthetic knowledge-base edits, natural-edit timeline and code requirement revisions are released as edit logs for research (under C BY-SA 4.0 where derived from KQA Pro; our code is MIT-licensed). All questions and tasks are in English or Python. All models are open-weight and were run locally without paid APIs. AI assistants were used to draft proofs, code and text and to run experiments, and independent AI referee agents checked them; the authors take responsibility for all content. The guarantees hold only under the stated assumptions, and deploying certified memory in high-stakes settings still requires a verifier whose errors are understood. References Aggarwal et al. (2024) Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, and Mausam. 2024. AutoMix: Automatically mixing language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37. Anderson et al. (2004) John R. Anderson, Daniel Bothell, Michael D. Byrne, Scott Douglass, Christian Lebiere, and Yulin Qin. 2004. An integrated theory of the mind. Psychological Review, 111(4):1036–1060. Angelopoulos et al. (2025) Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. 2025. Learn Then Test: Calibrating predictive algorithms to achieve risk control. The Annals of Applied Statistics, 19(2):1641–1662. Asadolahi et al. (2026) Mohammad Asadolahi, Amir Amini, Samira Talebi, Amirfarhad Farhadi, and Azadeh Zamanifar. 2026. Memory reward inflation in self-improving LLM agents. arXiv preprint arXiv:2608.00017. Asmussen (2003) Søren Asmussen. 2003. Applied Probability and Queues, 2 edition. Stochastic Modelling and Applied Probability. Springer, New York. Austin et al. (2021) Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Azar et al. (2018) Yossi Azar, Eric Horvitz, Eyal Lubetzky, Yuval Peres, and Dafna Shahaf. 2018. Tractable near-optimal policies for crawling. Proceedings of the National Academy of Sciences, 115(32):8099–8103. Banerjee and Veeravalli (2013) Taposh Banerjee and Venugopal V. Veeravalli. 2013. Data-efficient quickest change detection in minimax settings. IEEE Transactions on Information Theory, 59(10):6917–6931. Bang (2023) Fu Bang. 2023. GPTCache: An open-source semantic cache for LLM applications enabling faster answers and cost savings. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 212–218. Association for Computational Linguistics. Bastopcu and Ulukus (2021) Melih Bastopcu and Sennur Ulukus. 2021. Information freshness in cache updating systems. IEEE Transactions on Wireless Communications, 20(3):1861–1874. Brook and Evans (1972) D. Brook and D. A. Evans. 1972. An approach to the probability distribution of CUSUM run length. Biometrika, 59(3):539–549. Cai et al. (2026) Qianshu Cai, Yonggang Zhang, Jun Nie, Maohao Ran, Huajiang Zheng, Jun Song, Xinmei Tian, Yike Guo, and Wei Xue. 2026. A theory of reliable self-evolution for agent harnesses. arXiv preprint arXiv:2609.08175. Version 1 (8 Sep 2026) titled “Safe Harness Self-Evolution: A Theoretical Analysis of Feasibility and Limits”. Cai et al. (2024) Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. 2024. Large language models as tool makers. In International Conference on Learning Representations. ArXiv:2305.17126. Cao et al. (2022) Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, and Hanwang Zhang. 2022. KQA Pro: A dataset with explicit compositional programs for complex question answering over knowledge base. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6101–6119, Dublin, Ireland. Association for Computational Linguistics. Chandak et al. (2020) Yash Chandak, Scott M. Jordan, Georgios Theocharous, Martha White, and Philip S. Thomas. 2020. Towards safe policy improvement for non-stationary MDPs. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Che et al. (2002) Hao Che, Ye Tung, and Zhijun Wang. 2002. Hierarchical Web caching systems: Modeling, design and experimental results. IEEE Journal on Selected Areas in Communications, 20(7):1305–1314. Chen et al. (2024a) Lingjiao Chen, Matei Zaharia, and James Zou. 2024a. FrugalGPT: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research. Chen et al. (2024b) Lingjiao Chen, Matei Zaharia, and James Zou. 2024b. How is ChatGPT’s behavior changing over time? Harvard Data Science Review, 6(2). Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Chen et al. (2026) Qijia Chen, Andrea Bellucci, Zhida Sun, and Giulio Jacucci. 2026. SkillDroid: Compile once, reuse forever. arXiv preprint arXiv:2604.14872. Chhikara et al. (2025) Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Cho and Garcia-Molina (2003) Junghoo Cho and Hector Garcia-Molina. 2003. Effective page refresh policies for Web crawlers. ACM Transactions on Database Systems, 28(4):390–426. Choungmo Fofack et al. (2014) Nicaise Choungmo Fofack, Philippe Nain, Giovanni Neglia, and Don Towsley. 2014. Performance evaluation of hierarchical TTL-based cache networks. Computer Networks, 65:212–231. Cohen and Shenker (2002) Edith Cohen and Scott Shenker. 2002. Replication strategies in unstructured peer-to-peer networks. In Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), pages 177–190. ACM. Cooper and Meyn (2024) Austin Cooper and Sean Meyn. 2024. Quickest change detection using mismatched CUSUM (extended abstract). In Proceedings of the 60th Annual Allerton Conference on Communication, Control, and Computing, pages 1–7. IEEE. Extended version: arXiv:2409.07948. Didolkar et al. (2025) Aniket Didolkar, Nicolas Ballas, Sanjeev Arora, and Anirudh Goyal. 2025. Metacognitive reuse: Turning recurring LLM reasoning into concise behaviors. arXiv preprint arXiv:2509.13237. Dong et al. (2025) Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. 2025. Memory injection attacks on LLM agents via query-only interaction. arXiv preprint arXiv:2503.03704. Ellis et al. (2021) Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sablé-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B. Tenenbaum. 2021. DreamCoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI), pages 835–850. ACM. Fagin (1977) Ronald Fagin. 1977. Asymptotic miss ratios over independent references. Journal of Computer and System Sciences, 14(2):222–250. Feng et al. (2021) Jean Feng, Scott Emerson, and Noah Simon. 2021. Approval policies for modifications to machine learning-based software as a medical device: A study of bio-creep. Biometrics, 77(1):31–44. Fikes et al. (1972) Richard E. Fikes, Peter E. Hart, and Nils J. Nilsson. 1972. Learning and executing generalized robot plans. Artificial Intelligence, 3:251–288. Forouzandeh et al. (2026) Saman Forouzandeh, Wei Peng, Parham Moradi, Xinghuo Yu, and Mahdi Jalili. 2026. Learning hierarchical procedural memory for LLM agents through Bayesian selection and contrastive refinement. In Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Gallo et al. (2014) Massimo Gallo, Bruno Kauffmann, Luca Muscariello, Alain Simonian, and Christian Tanguy. 2014. Performance evaluation of the random replacement policy for networks of caches. Performance Evaluation, 72:16–36. Gao et al. (2026) Haowen Gao, Haoran Chen, Can Wang, Shasha Guo, Liang Pang, Zhaoyang Liu, Huawei Shen, and Xueqi Cheng. 2026. SelfEvoSkill: Paired execution audits for skill revision without external outcome supervision. arXiv preprint arXiv:2606.14239. Version 1 (12 Jun 2026) titled “SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing”. Garivier and Kaufmann (2016) Aurélien Garivier and Emilie Kaufmann. 2016. Optimal best arm identification with fixed confidence. In Proceedings of the 29th Annual Conference on Learning Theory (COLT), volume 49 of Proceedings of Machine Learning Research, pages 998–1027. PMLR. Gnedin et al. (2007) Alexander Gnedin, Ben Hansen, and Jim Pitman. 2007. Notes on the occupancy problem with infinitely many boxes: General asymptotics and power laws. Probability Surveys, 4:146–171. Good (1953) I. J. Good. 1953. The population frequencies of species and the estimation of population parameters. Biometrika, 40(3–4):237–264. Good and Toulmin (1956) I. J. Good and G. H. Toulmin. 1956. The number of new species, and the increase in population coverage, when a sample is increased. Biometrika, 43(1–2):45–63. Gratch and DeJong (1992) Jonathan Gratch and Gerald DeJong. 1992. COMPOSER: A probabilistic solution to the utility problem in speed-up learning. In Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI), pages 235–240. AAAI Press. Greiner (1996) Russell Greiner. 1996. PALO: A probabilistic hill-climbing algorithm. Artificial Intelligence, 84(1–2):177–208. Hall and Heyde (1980) Peter Hall and Christopher C. Heyde. 1980. Martingale Limit Theory and Its Application. Academic Press, New York. Hao et al. (2025) Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2025. Training large language models to reason in a continuous latent space. In Conference on Language Modeling (COLM). Heathcote et al. (2000) Andrew Heathcote, Scott Brown, and D. J. K. Mewhort. 2000. The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7(2):185–207. Honda and Takemura (2010) Junya Honda and Akimichi Takemura. 2010. An asymptotically optimal bandit algorithm for bounded support models. In Proceedings of the 23rd Conference on Learning Theory (COLT), pages 67–79. Howard et al. (2021) Steven R. Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. 2021. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2):1055–1080. Huang et al. (2026) Jianguo Huang, Hao Zeng, Bingyi Jing, Hongxin Wei, and Bo An. 2026. Conditional performance guarantee for large reasoning models. arXiv preprint arXiv:2601.22790. Hutter (2021) Marcus Hutter. 2021. Learning curve theory. arXiv preprint arXiv:2102.04074. IBM Granite Team (2025) IBM Granite Team. 2025. Granite 3.3 8B instruct. Model card. Iglehart (1972) Donald L. Iglehart. 1972. Extreme values in the GI/G/1 queue. The Annals of Mathematical Statistics, 43(2):627–635. Jelenković (1999) Predrag R. Jelenković. 1999. Asymptotic approximation of the move-to-front search cost distribution and least-recently used caching fault probabilities. The Annals of Applied Probability, 9(2):430–464. Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. Preprint, arXiv:2310.06825. Karlin (1967) Samuel Karlin. 1967. Central limit theorems for certain infinite urn schemes. Journal of Mathematics and Mechanics, 17(4):373–401. Kasai et al. (2023) Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2023. RealTime QA: What’s the answer right now? In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, volume 36. Kaufmann et al. (2016) Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. 2016. On the complexity of best-arm identification in multi-armed bandit models. Journal of Machine Learning Research, 17(1):1–42. Khosravi and Huo (2026) Hamed Khosravi and Xiaoming Huo. 2026. Conformal selective acting: Anytime-valid risk control for RLVR-trained LLMs. arXiv preprint arXiv:2605.20270. Kim et al. (2025) Elliot Kim, Avi Garg, Kenny Peng, and Nikhil Garg. 2025. Correlated errors in large language models. In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pages 30038–30066. PMLR. Kolobov et al. (2019) Andrey Kolobov, Yuval Peres, Cheng Lu, and Eric J. Horvitz. 2019. Staying up to date with online content changes using reinforcement learning for scheduling. In Advances in Neural Information Processing Systems (NeurIPS), volume 32. Korf (1985) Richard E. Korf. 1985. Macro-operators: A weak method for learning. Artificial Intelligence, 26(1):35–77. Kwon et al. (2023) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), pages 611–626. Lai (1981) Tze Leung Lai. 1981. Asymptotic optimality of invariant sequential probability ratio tests. The Annals of Statistics, 9(2):318–333. Lai (1998) Tze Leung Lai. 1998. Information bounds and quick detection of parameter changes in stochastic systems. IEEE Transactions on Information Theory, 44(7):2917–2929. Laird et al. (1986) John E. Laird, Paul S. Rosenbloom, and Allen Newell. 1986. Chunking in Soar: The anatomy of a general learning mechanism. Machine Learning, 1(1):11–46. Lee et al. (2025) Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, and Hyun-Hwan Jeong. 2025. ConSol: Sequential probability ratio testing to find consistent LLM reasoning paths efficiently. arXiv preprint arXiv:2503.17587. Li et al. (2025) Zhiyu Li, Chenyang Xi, Chunyu Li, Ding Chen, Boyu Chen, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Chen Tang, Qingchen Yu, Jihao Zhao, Yezhaohui Wang, Peng Liu, Zehao Lin, Pengyuan Wang, Jiahao Huo, Tianyi Chen, Kai Chen, and 20 others. 2025. MemOS: A memory OS for AI system. arXiv preprint arXiv:2507.03724. Liška et al. (2022) Adam Liška, Tomáš Kočiský, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien de Masson d’Autume, Tim Scholtes, Manzil Zaheer, Susannah Young, Ellen Gilsenan-McMahon, Sophia Austin, Phil Blunsom, and Angeliki Lazaridou. 2022. StreamingQA: A benchmark for adaptation to new knowledge over time in question answering models. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pages 13604–13622. Liu et al. (2023) Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems. ArXiv:2305.01210. Liu et al. (2024) Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2024. Can language models learn to skip steps? In Advances in Neural Information Processing Systems (NeurIPS), volume 37. Liu et al. (2026) Xutong Liu, Baran Atalar, Xiangxiang Dai, Jinhang Zuo, Siwei Wang, John C. S. Lui, Wei Chen, and Carlee Joe-Wong. 2026. Semantic caching for low-cost LLM serving: From offline learning to online adaptation. In Proceedings of IEEE INFOCOM 2026 – IEEE Conference on Computer Communications. Logan (1988) Gordon D. Logan. 1988. Toward an instance theory of automatization. Psychological Review, 95(4):492–527. Lorden (1971) G. Lorden. 1971. Procedures for reacting to a change in distribution. The Annals of Mathematical Statistics, 42(6):1897–1908. Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems. ArXiv:2303.17651. Mansoor et al. (2026) Muhammad Mansoor, Tahir Ahmad, and Yeo-Chan Yoon. 2026. Risk-constrained freshness-aware semantic caching for open-web retrieval-augmented LLMs. arXiv preprint arXiv:2607.04281. Martina et al. (2014) Valentina Martina, Michele Garetto, and Emilio Leonardi. 2014. A unified approach to the performance analysis of caching systems. In Proceedings of IEEE INFOCOM 2014 – IEEE Conference on Computer Communications, pages 2040–2048. McAllester and Schapire (2000) David A. McAllester and Robert E. Schapire. 2000. On the convergence rate of Good-Turing estimators. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory (COLT), pages 1–6. Michaud et al. (2023) Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark. 2023. The quantization model of neural scaling. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 28699–28722. Minton (1990) Steven Minton. 1990. Quantitative results concerning the utility of explanation-based learning. Artificial Intelligence, 42(2–3):363–391. Mitchell et al. (1986) Tom M. Mitchell, Richard M. Keller, and Smadar T. Kedar-Cabelli. 1986. Explanation-based generalization: A unifying view. Machine Learning, 1(1):47–80. Mo et al. (2026) Yunxiang Mo, Donghao Zhao, and Hejia Geng. 2026. Stable answers, unfinished reasoning: Why self-consensus is not a safe early-exit signal. arXiv preprint arXiv:2609.09989. Molloy (2021) Timothy L. Molloy. 2021. Misspecified and asymptotically minimax robust quickest change diagnosis. IEEE Transactions on Automatic Control, 66(2):857–864. Molloy and Ford (2017) Timothy L. Molloy and Jason J. Ford. 2017. Misspecified and asymptotically minimax robust quickest change detection. IEEE Transactions on Signal Processing, 65(21):5730–5742. Moustakides (1986) George V. Moustakides. 1986. Optimal stopping times for detecting changes in distributions. The Annals of Statistics, 14(4):1379–1387. Munford (1980) A. G. Munford. 1980. A control chart based on cumulative scores. Journal of the Royal Statistical Society. Series C (Applied Statistics), 29(3):252–258. Newell and Rosenbloom (1981) Allen Newell and Paul S. Rosenbloom. 1981. Mechanisms of skill acquisition and the law of practice. In John R. Anderson, editor, Cognitive Skills and Their Acquisition, pages 1–55. Lawrence Erlbaum Associates, Hillsdale, NJ. Ohannessian and Dahleh (2012) Mesrob I. Ohannessian and Munther A. Dahleh. 2012. Rare probability estimation under regularly varying heavy tails. In Proceedings of the 25th Annual Conference on Learning Theory (COLT), volume 23 of Proceedings of Machine Learning Research, pages 21.1–21.24. PMLR. Orlitsky et al. (2016) Alon Orlitsky, Ananda Theertha Suresh, and Yihong Wu. 2016. Optimal prediction of the number of unseen species. Proceedings of the National Academy of Sciences, 113(47):13283–13288. Ouyang et al. (2026) Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2026. ReasoningBank: Scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations (ICLR). Page (1954) E. S. Page. 1954. Continuous inspection schemes. Biometrika, 41(1–2):100–115. Park et al. (2023) Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST). ACM. Podkopaev and Ramdas (2022) Aleksandr Podkopaev and Aaditya Ramdas. 2022. Tracking the risk of a deployed model and detecting harmful distribution shifts. In International Conference on Learning Representations (ICLR). Pu et al. (2026) Hongji Pu, Xinyuan Song, and Liang Zhao. 2026. SkillOps: Managing LLM agent skill libraries as self-maintaining software ecosystems. arXiv preprint arXiv:2605.13716. Ramdas et al. (2023) Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. 2023. Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):576–601. Reynolds and Stoumbos (1999) Marion R. Reynolds, Jr. and Zachary G. Stoumbos. 1999. A CUSUM chart for monitoring a proportion when inspecting continuously. Journal of Quality Technology, 31(1):87–108. Richter et al. (2025) Leo Richter, Xuanli He, Pasquale Minervini, and Matt J. Kusner. 2025. An auditing test to detect behavioral shift in language models. In International Conference on Learning Representations (ICLR). Sadhuka et al. (2025) Shuvom Sadhuka, Drew Prinster, Clara Fannjiang, Gabriele Scalia, Bonnie Berger, Aviv Regev, and Hanchen Wang. 2025. E-valuator: Reliable agent verifiers with sequential hypothesis testing. arXiv preprint arXiv:2512.03109. Schroeder et al. (2026) Luis Gaspar Schroeder, Aditya Desai, Alejandro Cuadron, Kyle Chu, Shu Liu, Mark Zhao, Stephan Krusche, Alfons Kemper, Matei Zaharia, and Joseph E. Gonzalez. 2026. vCache: Verified semantic prompt caching. In International Conference on Learning Representations (ICLR). Schuster et al. (2022) Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, and Donald Metzler. 2022. Confident adaptive language modeling. In Advances in Neural Information Processing Systems (NeurIPS), volume 35. Scrivens (2026) Arsenios Scrivens. 2026. Information-theoretic limits of safety verification for self-improving systems. arXiv preprint arXiv:2603.28650. Shafer (2021) Glenn Shafer. 2021. Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(2):407–431. Shawn (2026) Zayx Shawn. 2026. PACE: Anytime-valid acceptance tests for self-evolving agents. arXiv preprint arXiv:2606.08106. Shin et al. (2024) Jaehyeok Shin, Aaditya Ramdas, and Alessandro Rinaldo. 2024. E-detectors: A nonparametric framework for sequential change detection. The New England Journal of Statistics in Data Science, 2(2):229–260. Shinn et al. (2023) Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems. ArXiv:2303.11366. Siegmund (1985) David Siegmund. 1985. Sequential Analysis: Tests and Confidence Intervals. Springer Series in Statistics. Springer, New York. Simsek (2026) Baris Simsek. 2026. When to forget: A memory governance primitive. arXiv preprint arXiv:2604.12007. Singh and Yu (2025) Pragya Singh and Stanley Yu. 2025. Validation-gated Hebbian learning for adaptive agent memory. In Proceedings of the First Workshop on Knowledge Graphs and Agentic Systems Interplay (NORA 2025), co-located with NeurIPS 2025, volume 4162 of CEUR Workshop Proceedings, pages 42–54. Sui et al. (2025) Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. 2025. Stop overthinking: A survey on efficient reasoning for large language models. Transactions on Machine Learning Research. Sun et al. (2025) Changzhi Sun, Xiangyu Chen, Jixiang Luo, Dell Zhang, and Xuelong Li. 2025. Beyond heuristics: A decision-theoretic framework for agent memory management. arXiv preprint arXiv:2512.21567. Suzgun et al. (2026) Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. 2026. Dynamic Cheatsheet: Test-time learning with adaptive memory. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7080–7106. Association for Computational Linguistics. Taatgen and Lee (2003) Niels A. Taatgen and Frank J. Lee. 2003. Production compilation: A simple mechanism to model complex skill acquisition. Human Factors, 45(1):61–76. Tadepalli and Natarajan (1996) Prasad Tadepalli and Balas K. Natarajan. 1996. A formal framework for speedup learning from problems and solutions. Journal of Artificial Intelligence Research, 4:445–475. Tartakovsky et al. (2014) Alexander Tartakovsky, Igor Nikiforov, and Michèle Basseville. 2014. Sequential Analysis: Hypothesis Testing and Changepoint Detection. Chapman and Hall/CRC. Team OLMo (2025) Team OLMo. 2025. OLMo 3. Allen Institute for AI. Model card: https://huggingface.co/allenai/Olmo-3-7B-Instruct. Thomas et al. (2015) Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. 2015. High confidence policy improvement. In Proceedings of the 32nd International Conference on Machine Learning (ICML), volume 37 of Proceedings of Machine Learning Research, pages 2380–2388. PMLR. Thomas et al. (2019) Philip S. Thomas, Bruno Castro da Silva, Andrew G. Barto, Stephen Giguere, Yuriy Brun, and Emma Brunskill. 2019. Preventing undesirable behavior of intelligent machines. Science, 366(6468):999–1004. Trivedi et al. (2022) Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. Ville (1939) Jean Ville. 1939. Étude critique de la notion de collectif. Gauthier-Villars, Paris. Wald (1945) Abraham Wald. 1945. Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics, 16(2):117–186. Wald and Wolfowitz (1948) Abraham Wald and Jacob Wolfowitz. 1948. Optimum character of the sequential probability ratio test. The Annals of Mathematical Statistics, 19(3):326–339. Wang et al. (2024a) Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024a. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research. Wang et al. (2024b) Haiming Wang, Huajian Xin, Chuanyang Zheng, Lin Li, Zhengying Liu, Qingxing Cao, Yinya Huang, Jing Xiong, Han Shi, Enze Xie, Jian Yin, Zhenguo Li, Heng Liao, and Xiaodan Liang. 2024b. LEGO-Prover: Neural theorem proving with growing libraries. In International Conference on Learning Representations (ICLR). Wang et al. (2026) Xi Wang, Anushri Suresh, Alvin Zhang, Rishi More, William Jurayj, Benjamin Van Durme, Mehrdad Farajtabar, Daniel Khashabi, and Eric Nalisnick. 2026. Conformal Thinking: Risk control for reasoning on a compute budget. In Proceedings of the 43rd International Conference on Machine Learning (ICML). Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations. ArXiv:2203.11171. Wang et al. (2024c) Zhiruo Wang, Graham Neubig, and Daniel Fried. 2024c. TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pages 51177–51191. PMLR. Wang et al. (2025a) Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, and Daniel Fried. 2025a. Inducing programmatic skills for agentic tasks. arXiv preprint arXiv:2504.06821. Wang et al. (2025b) Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2025b. Agent Workflow Memory. In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pages 63897–63911. PMLR. Waudby-Smith and Ramdas (2024) Ian Waudby-Smith and Aaditya Ramdas. 2024. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(1):1–27. Wu and Yang (2016) Jingxian Wu and Jing Yang. 2016. Quickest change detection with mismatched post-change models. arXiv preprint arXiv:1601.06868. Wu and Canedo (2026) Michael Wu and Arquimedes Canedo. 2026. Invalidation contracts for cross-episode agent memory. arXiv preprint arXiv:2609.00243. Wu et al. (2025) Xuening Wu, Shenqin Yin, Yanlan Kang, Xinhang Zhang, Qianya Xu, Zeping Chen, and Wenqiang Zhang. 2025. SGM: A statistical Gödel machine for risk-controlled recursive self-modification. arXiv preprint arXiv:2510.10232. Xia et al. (2025) Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. 2025. TokenSkip: Controllable chain-of-thought compression in LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3351–3363. Association for Computational Linguistics. Xu et al. (2025) Haoran Xu, Jiacong Hu, Ke Zhang, Lei Yu, Yuxin Tang, Xinyuan Song, Yiqun Duan, Lynn Ai, and Bill Shi. 2025. SEDM: Scalable self-evolving distributed memory for agents. arXiv preprint arXiv:2509.09498. Xu et al. (2021) Qunzhi Xu, Yajun Mei, and George V. Moustakides. 2021. Optimum multi-stream sequential change-point detection with sampling control. IEEE Transactions on Information Theory, 67(11):7627–7636. Xu et al. (2024) Ziyu Xu, Nikos Karampatziakis, and Paul Mineiro. 2024. Active, anytime-valid risk controlling prediction sets. In Advances in Neural Information Processing Systems (NeurIPS), volume 37. Yang et al. (2025) An Yang, Anfeng Li, Baosong Yang, et al. 2025. Qwen3 technical report. arXiv:2505.09388. Yang et al. (2024a) Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, and Bin Cui. 2024a. Buffer of Thoughts: Thought-augmented reasoning with large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37. Yang et al. (2024b) Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. 2024b. Do large language models latently perform multi-hop reasoning? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10210–10229. Association for Computational Linguistics. Yao et al. (2023a) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems. ArXiv:2305.10601. Yao et al. (2023b) Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023b. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations. ArXiv:2210.03629. Yates et al. (2017) Roy D. Yates, Philippe Ciblat, Aylin Yener, and Michèle Wigger. 2017. Age-optimal constrained cache updating. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), pages 141–145. Yu et al. (2026a) Chengxin Yu, Zhaoxin Fan, Faguo Wu, Hongwei Zheng, Yun Zhou, and Zhiyu Li. 2026a. CoMem: Collective-individual memory synergy for evolutionary multi-agent systems. arXiv preprint arXiv:2609.15009. Yu et al. (2026b) Chengyao Yu, Hao Zeng, Youxin Zhu, Jianguo Huang, Huajun Zeng, and Bingyi Jing. 2026b. Anytime safe PAC efficient reasoning. In Proceedings of the 43rd International Conference on Machine Learning (ICML). Yu et al. (2024) Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. 2024. Distilling System 2 into System 1. arXiv preprint arXiv:2407.06023. Yuan et al. (2024) Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R. Fung, Hao Peng, and Heng Ji. 2024. CRAFT: Customizing LLMs by creating and retrieving from specialized toolsets. In International Conference on Learning Representations. ArXiv:2309.17428. Zeng et al. (2025) Hao Zeng, Jianguo Huang, Bingyi Jing, Hongxin Wei, and Bo An. 2025. On the provable performance guarantee of efficient reasoning models. arXiv preprint arXiv:2510.09133. Zhang et al. (2025a) Guibin Zhang, Muxin Fu, Kun Wang, Guancheng Wan, Miao Yu, and Shuicheng Yan. 2025a. G-Memory: Tracing hierarchical memory for Multi-Agent systems. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, pages 14587–14617. Zhang et al. (2026a) Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. 2026a. Darwin Gödel machine: Open-ended evolution of self-improving agents. In International Conference on Learning Representations (ICLR). Zhang et al. (2025b) Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025b. AdaptThink: Reasoning models can learn when to think. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3716–3730. Association for Computational Linguistics. Zhang et al. (2026b) Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. 2026b. Agentic Context Engineering: Evolving contexts for self-improving language models. In International Conference on Learning Representations (ICLR). Code: https://github.com/ace-agent/ace. Zhang et al. (2025c) Qizheng Zhang, Michael Wornow, Gerry Wan, and Kunle Olukotun. 2025c. Agentic Plan Caching: Test-time memory for fast and cost-efficient LLM agents. In Advances in Neural Information Processing Systems (NeurIPS), volume 38. Zhang et al. (2026c) Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Zhuo Li, Yujie Zheng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, and Muning Wen. 2026c. MemRL: Self-evolving agents via runtime reinforcement learning on episodic memory. arXiv preprint arXiv:2601.03192. Zhang et al. (2026d) Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. 2026d. Ratchet: How reliable must an LLM judge be to retire a skill? arXiv preprint arXiv:2605.22148. Zhang et al. (2026e) Yaolun Zhang, Yiran Wu, Yijiong Yu, Qingyun Wu, and Huazheng Wang. 2026e. Live-Evo: Online evolution of agentic memory from continuous feedback. arXiv preprint arXiv:2602.02369. Zhao et al. (2024) Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19632–19642. Zhong et al. (2024) Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. MemoryBank: Enhancing large language models with long-term memory. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19724–19731. Zhong et al. (2023) Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. 2023. MQuAKE: Assessing knowledge editing in language models via multi-hop questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 15686–15702. Zhou et al. (2026a) Jianghong Zhou, Hao Rong, and Chen Lin. 2026a. Certification as a rate limit on self-improving LLM agents. Under review. Zhou et al. (2026b) Jianghong Zhou, Hao Rong, Chen Lin, and Daoan Zhang. 2026b. More verifiers are not always better: Robust shared memory for multi-agent LLM systems. Under review. Zhu et al. (2023) Banghua Zhu, Ying Sheng, Lianmin Zheng, Clark Barrett, Michael I. Jordan, and Jiantao Jiao. 2023. Towards optimal caching and model selection for large model inference. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Zhu et al. (2026) Jinchang Zhu, Jindong Li, Cheng Zhang, Jiahong Liu, and Menglin Yang. 2026. HeLa-Mem: Hebbian learning and associative memory for LLM agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13757–13769. Association for Computational Linguistics. Zrnic and Candès (2024) Tijana Zrnic and Emmanuel J. Candès. 2024. Active statistical inference. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pages 62993–63010. PMLR. Appendix A Proofs for Section 3 Setting. Outcomes Y1,Y2,…Y_1,Y_2,… are independent, with law P0P_0 before an unknown change time tct_c and P1P_1 from tct_c on. A score rule uses a bounded score X=f(Y)X=f(Y) with x¯=esssupX x=ess\,supX and b¯=esssup(−X) b=ess\,sup(-X), and runs S0=0S_0=0, Sn=max0,Sn−1+XnS_n= \0,S_n-1+X_n\, τh=infn:Sn≥h _h= \n:S_n≥ h\. We assume μ0≔0X<0<P0(X>0) _0 _0X<0<P_0(X>0). The ARL is A(h)=∞τhA(h)=E_∞ _h. Lorden’s worst-case delay D¯(h)=suptcesssuptc[(τh−tc+1)+∣Y<tc] D(h)= _t_cess\,supE_t_c[( _h-t_c+1)^+ Y_<t_c] equals the zero-start delay D(h)=1τhD(h)=E_1 _h. The reason is that s↦Sn(s)s S_n(s) is monotone, so the worst pre-change state is S=0S=0. We write Wn=∑k≤nXkW_n= _k≤ nX_k, φj(θ)=lnjeθX _j(θ)= _je^θ X, and θj>0 _j>0 for the positive root of φj _j. Hebbian rules and the exact level. Start from c0=1c_0=1, update c←clip[0,1](c+η+Y=1−η−Y=0)c _[0,1](c+ _+1\Y=1\- _-1\Y=0\), and retire at c<τc<τ. With S=(1−c)/η−S=(1-c)/ _- and ρ=η+/η−ρ= _+/ _-, the upper clip is the reflection at 00. The lower clip can bind only on a step that retires under both descriptions. So the rule is (2) with score +1+1 on failure, −ρ-ρ on success, and retirement at S>(1−τ)/η−S>(1-τ)/ _-. For η+=η−=η _+= _-=η (ρ=1ρ=1), S is integer and the retirement time is τH _H with H=⌊(1−τ)/η⌋+1H= (1-τ)/η +1. A first-passage recursion (t0=1/qt_0=1/q, tk=1/q+λtk−1t_k=1/q+λ t_k-1, λ=(1−q)/qλ=(1-q)/q) gives the exact run length qτH=1q(λ−1)[λ(λH−1)λ−1−H]E_q _H= 1q(λ-1) [ λ(λ^H-1)λ-1-H ] for q≠12q≠ 12, and 1/2τH=H(H+1)E_1/2 _H=H(H+1). This agrees with an exact Markov-chain solve to 10−4610^-46 relative error. Floating-point implementations of “c<τc<τ” can retire one failure early (e.g. η=0.05,τ=0.9η=0.05,τ=0.9), so we recommend an integer counter. Proof of Remark 1. By convexity, φ0 _0 has a unique positive root θ0 _0 and φ0′(θ0)>0 _0 ( _0)>0. Then P~0(dy)=eθ0f(y)P0(dy) P_0(dy)=e _0f(y)P_0(dy) is a probability measure with ln(dP~0/dP0)=θ0f (d P_0/dP_0)= _0f. Multiplying by θ0>0 _0>0 commutes with max0,⋅ \0,·\, so θ0Sn _0S_n is Page’s log-likelihood CUSUM for P0→P~0P_0→ P_0 with threshold θ0h _0h. Its Lorden optimality for that change is due to Moustakides (1986). It is the likelihood-ratio CUSUM of no other alternative because ln(dQ/dP0)=κf (dQ/dP_0)=κ f with E0eκf=1E_0e^κ f=1 forces κ=θ0κ= _0. For the ±1± 1 rule eθ0=(1−q0)/q0e _0=(1-q_0)/q_0, so q~=q0eθ0=1−q0 q=q_0e _0=1-q_0. More generally, the rule “+a+a on failure, −b-b on success” is the exact CUSUM along the curve (q0,q1):ρ(q0,q1)=b/a\(q_0,q_1):ρ(q_0,q_1)=b/a\. Lemma 7 (Page cycles and Lundberg bounds). Let Nh=infn≥1:Wn≤0 or Wn≥hN_h= \n≥ 1:W_n≤ 0 or W_n≥ h\, ph=ℙ(WNh≥h)p_h=P(W_N_h≥ h) and N−=infn≥1:Wn≤0N_-= \n≥ 1:W_n≤ 0\. (a) If Nh<∞EN_h<∞ and ph>0p_h>0, then τh=Nh/phE _h=EN_h/p_h. (b) If X<0EX<0 with adjustment coefficient θ, and cθ=[1−eθX;X<0]c_θ=E[1-e^θ X;X<0], then cθe−θ(h+x¯)≤ph≤e−θhc_θe^-θ(h+ x)≤ p_h≤ e^-θ h and ℙ(supnWn≥x)≤e−θxP( _nW_n≥ x)≤ e^-θ x. (c) If X<0EX<0, then N−=1/ℙ(supnWn=0)<∞EN_-=1/P( _nW_n=0)<∞. Proof. (a) Until NhN_h the walk and S coincide. On a downward exit S restarts at 00, so τh _h is a geometric(php_h) sum of i.i.d. cycle lengths, and Wald’s identity applies (Page, 1954; Siegmund, 1985). (b) eθWne^θ W_n is a martingale bounded up to NhN_h, so 1=eθWNh1=Ee^θ W_N_h. Split into upward exits (value in [eθh,eθ(h+x¯))[e^θ h,e^θ(h+ x))) and downward exits (value ≤1≤1, and =eθX1=e^θ X_1 on X1<0\X_1<0\). The maximal inequality follows from optional stopping at the first passage above x. (c) Duality: W1>0,…,Wn>0\W_1>0,…,W_n>0\ is the event that n is a strict ascending ladder epoch of the reversed walk. The number of such epochs is geometric with mean 1/ℙ(supW=0)1/P( W=0). ∎ Proof of Theorem 1. ARL. By Lemma 7(a)–(c) under P0P_0, eθ0h≤A(h)≤(0N−/c0)eθ0(h+x¯)e _0h≤ A(h)≤(E_0N_-/c_0)e _0(h+ x), so θ0h≤lnA≤θ0h+Δ0 _0h≤ A≤ _0h+ _0 with an explicit constant Δ0 _0. (i) μ1>0 _1>0. Since Sn≥WnS_n≥ W_n, τh _h is at most the first passage of W above h, and Wald gives D≤(h+x¯)/μ1D≤(h+ x)/ _1. Conversely μ1D=1Wτh≥h−1|infnWn|≥h−1/β1 _1D=E_1W_ _h≥ h-E_1| _nW_n|≥ h-1/ _1, where β1>0 _1>0 solves 1e−β1X=1E_1e^- _1X=1 (maximal inequality for −W-W). Substituting h=(lnA+O(1))/θ0h=( A+O(1))/ _0 gives D=lnA/(θ0μ1)+O(1)D= A/( _0 _1)+O(1). (i) μ1=0 _1=0. Sn2−nσ12S_n^2-n _1^2 is a supermartingale, because [max(0,s+X)2]≤s2+σ12E[ (0,s+X)^2]≤ s^2+ _1^2. Hence D≥h2/σ12D≥ h^2/ _1^2. Wald’s second identity on a Page cycle, together with ph≥1X−/(h+x¯)p_h _1X^-/(h+ x), gives D≤[(h+x¯)2+b¯2(h+x¯)/1X−]/σ12D≤[(h+ x)^2+ b^2(h+ x)/E_1X^-]/ _1^2. (i) μ1<0 _1<0. Apply the ARL argument under P1P_1 with θ1 _1: eθ1h≤D≤(1N−/c1)eθ1(h+x¯)e _1h≤ D≤(E_1N_-/c_1)e _1(h+ x). Since eθ1h=(eθ0h)γe _1h=(e _0h)^γ, this gives D≍AγD A^γ. Moreover γ<1γ<1 iff φ1(θ0)>0 _1( _0)>0, which holds when the score law under P1P_1 stochastically dominates that under P0P_0. For Bernoulli rules this is equivalent to q1>q0q_1>q_0. If P1(X>0)=0P_1(X>0)=0 the rule never alarms after the change. Efficiency. Because ln(dP~0/dP0)=θ0f (d P_0/dP_0)= _0f, θ0μ1 _0 _1 =1lndP~0dP0 =E_1 d P_0dP_0 =1lndP1dP0−1lndP1dP~0 =E_1 dP_1dP_0-E_1 dP_1d P_0 =KL(P1∥P0)−KL(P1∥P~0). =KL(P_1\|P_0)-KL(P_1\| P_0). Dividing the optimal delay lnA/KL(P1∥P0) A/KL(P_1\|P_0) (Lorden, 1971) by lnA/(θ0μ1) A/( _0 _1) gives (3). Equality holds iff P~0=P1 P_0=P_1, i.e. θ0f=lndP1/dP0 _0f= dP_1/dP_0. ∎ Proof of Corollary 2. The Bernoulli log-likelihood ratio is ln(q1/q0)>0 (q_1/q_0)>0 on a failure and ln1−q11−q0<0 1-q_11-q_0<0 on a success. A rule with scores (+1,−ρ)(+1,-ρ) is proportional to it iff ρ=ρ(q0,q1)ρ=ρ(q_0,q_1), and the retirement threshold in log-likelihood units is ln(q1/q0)(1−τ)/η− (q_1/q_0)\,(1-τ)/ _-. The symmetric rule is optimal iff q1=1−q0q_1=1-q_0. Unknown degradation. Proposition 8 (First-order minimax tuning). Let q1∈[q−,q+]q_1∈[q_-,q_+] with q0<q−q_0<q_-. For every detector with ARL ≥A≥ A, the worst-case delay over this interval is at least (1+o(1))lnA/KL(q−∥q0)(1+o(1)) A/KL(q_-\|q_0), and the likelihood-ratio rule tuned at q−q_- attains it. Proof. The lower bound is Lorden’s bound at q1=q−q_1=q_- (Lorden, 1971). The rule tuned at q−q_- has scores equal to the log-likelihood ratio of q−q_- against q0q_0, so θ0=1 _0=1, and its post-change drift μ1(q1) _1(q_1) increases in q1q_1 because the failure score is positive and the success score negative. By Theorem 1(i) its delay is lnA/μ1(q1)+O(1)≤lnA/μ1(q−)+O(1) A/ _1(q_1)+O(1)≤ A/ _1(q_-)+O(1), and μ1(q−)=KL(q−∥q0) _1(q_-)=KL(q_-\|q_0). ∎ Proposition 9 (Unclipped counters; restates Remark 2). Consider an unclipped counter Cn=Cn−1+Yn=1−Yn=0C_n=C_n-1+1\Y_n=1\-1\Y_n=0\ started at c0c_0 and deleted at C≤0C≤ 0 (ExpeL: c0=2c_0=2), or the majority rule “flag when harmful ≥ helpful” (ACE’s released code). (a) A=∞A=∞, since false deletion has probability (q0/(1−q0))c0<1(q_0/(1-q_0))^c_0<1, but no worst-case delay is finite: for q1>12q_1> 12 the delay after a change equals the credit banked before it divided by 2q1−12q_1-1. (b) Under partial staleness (q1<12q_1< 12) the stale macro is never flagged (ACE) or deleted (ExpeL) with probability at least 1−2q11-2q_1 (majority rule) or 1−(q1/(1−q1))c01-(q_1/(1-q_1))^c_0, for every change time (averaged over the pre-change history). Proof of Proposition 9. (a) Without a cap, a ±1± 1 walk with up-probability 1−q0>121-q_0> 12 started at c0≥1c_0≥ 1 hits 00 with probability λ0−c0<1 _0^-c_0<1, so A=∞A=∞. Given the pre-change history on τ≥tc\τ≥ t_c\, the post-change walk starts at Ctc−1C_t_c-1 with drift −(2q1−1)-(2q_1-1). Wald’s identity gives the conditional delay Ctc−1/(2q1−1)C_t_c-1/(2q_1-1), and the all-success history makes Ctc−1=c0+tc−1C_t_c-1=c_0+t_c-1 arbitrarily large, so D¯=∞ D=∞. (b) For q1<12q_1< 12, the probability that a walk from c never hits 00 is h1(c)=1−λ1−ch_1(c)=1- _1^-c. The process h1(Cn)τ>nh_1(C_n)1\τ>n\ is a P∞P_∞-submartingale, so ℙtc(τ=∞)P_t_c(τ=∞) is nondecreasing in tct_c. Its value at tc=1t_c=1 is 1−λ1−c01- _1^-c_0, and for the majority rule it is (1−q1)h1(1)=1−2q1(1-q_1)h_1(1)=1-2q_1. Exact dynamic programming reproduces these values to 8 digits. Time decay is a time-to-live. Suppose confidence also decays as c←(1−λ)c←(1-λ)c at every query, and the template is used with probability ϕφ per query. Then any run of k⋆=⌊lnτ/ln(1−λ)⌋+1k = τ/ (1-λ) +1 consecutive queries that do not use the template retires the macro, whatever its reliability. The expected waiting time for such a run is ((1−ϕ)−k⋆−1)/ϕ((1-φ)^-k -1)/φ queries, i.e. (1−ϕ)−k⋆−1(1-φ)^-k -1 uses. For λ=10−3λ=10^-3 and τ=0.85τ=0.85 we get k⋆=163k =163, so a perfectly reliable macro used by 1%1\% of queries is retired after at most 4.154.15 uses on average. The negative-drift regime in prior work: a correction. Wu and Yang (2016) study Page’s CUSUM run with a mismatched post-change density g. Their eq. (26) defines it by the restarted recursion C~n+1=max(1,C~n)Λ~n+1 C_n+1= (1, C_n)\, _n+1, i.e. (2) with f=ln(g/f0)f= (g/f_0), for which θ0=1 _0=1 and P~0=g P_0=g. Their Thm. 2 states that if KL(P1∥P0)<KL(P1∥g)KL(P_1\|P_0)<KL(P_1\|g) (their D10<D~11D_10< D_11), the average detection delay is infinite for every false-alarm level below one, and they stress that the statement is not asymptotic. This is our regime (i), and there the delay is finite whenever P1(f>0)>0P_1(f>0)>0: D(h)≤(1N−/c1)eθ1(h+x¯)D(h)≤(E_1N_-/c_1)e _1(h+ x) (proof of Theorem 1). The proof of their Thm. 2 writes the CUSUM stopping time as a minimum over restart times, τ~c=minvτv(A) τ_c= _v _v(A) (their eq. (77)), where τv(A) _v(A) is the one-sided test started at v and never restarted, and their eq. (78) then equates k(minvτv(A)−k)+E_k( _v _v(A)-k)^+ with minvk(τv(A)−k)+ _vE_k( _v(A)-k)^+. Their Lemma 6 is correct: each τv(A) _v(A) never stops with positive probability, so its mean delay is infinite. But in general only min≤minE ≤ holds, and here the inequality is strict: the CUSUM alarms at the first success of a geometric number of Page cycles (Lemma 7(a)), each of finite mean length. Counterexample. The symmetric rule with q0=0.05q_0=0.05 is the CUSUM with g the failure rate 0.950.95. At q1=0.25q_1=0.25, KL(P1∥P0)=0.225<KL(P1∥g)=1.697KL(P_1\|P_0)=0.225<KL(P_1\|g)=1.697, so their condition holds; their regularity conditions (their (5) and (47)) hold for i.i.d. data by the strong law. A single test τv(A) _v(A) with threshold h=4h=4 never stops with probability 1−3−4=0.9881-3^-4=0.988, yet the CUSUM’s exact Lorden delays are D(4)=232D(4)=232 and D(6)=2172D(6)=2172 (zero start is the worst case), and Monte Carlo gives 231.9231.9 and 21602160 (2×1052× 10^5 and 4×1044× 10^4 runs). Under their own Bayesian criterion, with the change at a geometric(0.10.1) time, [τ~c−ν∣τ~c≥ν]=230.8E[ τ_c-ν τ_c≥ν]=230.8 and 2170.82170.8. What their theorem does capture is the first-order statement D/lnA→∞D/ A→∞ in this regime; Theorem 1(i) gives its rate, D≍AγD A^γ (checks/wuyang_check.py in our code release). Appendix B Proofs for Section 4 Assumptions. • (A1) Demand. Queries of template i form independent Poisson processes of rate πi _i, with π1≥π2≥⋯ _1≥ _2≥·s and ∑iπi=1 _i _i=1. • (A2) Audits. Each query is audited independently with probability p. An audit yields one verifier outcome, of the candidate that the policy chose to evaluate on that query, revealed after answering. Nothing else about the candidates’ correctness is observed; in particular, the policy does not see whether an unaudited shadow answer agrees with the live one. (Under exact-match scoring such agreement is informative, and Theorem 22 shows that it can speed up certification.) • (A3) Changes. Facts change at the points of an independent Poisson process of rate ν per template. Changes may or may not be announced to the policy. A macro formed in an earlier epoch never counts as coverage. • (A4) Outcomes. In environment r E_r the audited outcomes of the current epoch are i.i.d. Bernoulli(r)(r), whichever candidate they evaluate, and independent of everything else, denoted G (arrivals, audits, change times, other templates and epochs, internal randomization). • (A5) Policies. Policies are non-anticipating: a decision at time t depends only on G-information up to t and on outcomes revealed before t. Precise form of Theorem 3. Assume (A1)–(A5) and that in every epoch the audited outcomes are i.i.d. Bernoulli(r1)(r_1), whichever candidate they evaluate (environment r1 E_r_1, the least-favorable good case). (i) For every demand law π and all ν,p,δν,p,δ, every policy that is δ-safe in the sense of Definition 1 has lim infM≥(1−εδ)Fπ(Θ) M≥(1- _δ)F_π( ) almost surely, whether or not changes are announced and however many candidates it forms, where εδ=O(lnln(1/δ)/ln(1/δ)) _δ=O ( (1/δ)/ (1/δ) ) depends only on (δ,r0,r1)(δ,r_0,r_1). (i) With announced changes, the SPRT that discards all evidence and restarts at each change is δ-safe and has M≤(1+ln(r1/r0)/ln(1/δ))Fπ(Θ)M≤(1+ (r_1/r_0)/ (1/δ))F_π( ). Hence M⋆=(1+o(1))Fπ(Θ)M =(1+o(1))F_π( ) as δ→0δ→ 0, uniformly in π,ν,pπ,ν,p. The main text calls the environment “good memories are only barely good”; the converse needs no other restriction on the policy than non-anticipation (A5) and the information structure (A2). Notation: x=ν/px=ν/p, ρi=πi/(πi+x) _i= _i/( _i+x), a±a_± the log-likelihood-ratio increments on success and failure, w=a++a−w=a_++a_-, KL=KL(r1∥r0)KL=KL(r_1\|r_0), L=ln(1/δ)L= (1/δ), n⋆=L/KLn =L/KL, Θ=xn⋆ =xn and β=1−1/sβ=1-1/s. Define Mfix(n,x)=∑iπi(1−ρin)M_ fix(n,x)= _i _i(1- _i^n). Lemma 10 (Exact stationary gap). If the policy folds when the NeN_e-th outcome of epoch e is observed, with (Ne)(N_e) i.i.d. copies of N independent of arrivals, audits and changes, then the long-run unfolded fraction of template i is ui=1−[ρiN]u_i=1-E[ _i^N] a.s. and in mean, and M=∑iπiuiM= _i _iu_i. Proof. Epochs are i.i.d. renewal cycles of mean length 1/ν1/ν with πi/ν _i/ν queries on average. A query at time t of an epoch is unfolded iff fewer than N audited queries preceded it. By the Mecke formula, Ue=πi∫0ℓℙ(Poi(pπit)<N)tEU_e= _iE _0 P(Poi(p _it)<N)\,dt with ℓ∼Exp(ν) (ν). The count Poi(pπiE)Poi(p _iE) with E∼Exp(ν)E (ν) is geometric with ℙ(=k)=(1−ρi)ρikP(=k)=(1- _i) _i^k, so ℙ(Poi(pπiE)<N)=1−ρiNP(Poi(p _iE)<N)=1-E _i^N. Apply renewal–reward per template and sum, using Fatou for the lower bound and the superposition property for the upper bound. ∎ Lemma 11 (Zipf sums). For πi=i−s/ζ(s) _i=i^-s/ζ(s), s>1s>1, real n≥1n≥ 1 and x>0x>0, Ks,nxβ−nx≤Mfix(n,x)≤Ks,nxβ,K_s,nx^β-nx≤ M_ fix(n,x)≤ K_s,nx^β, where Ks,n=Γ(1s)Cs,nK_s,n= ( 1s)\,C_s,n and Cs,n=ζ(s)−1/sΓ(n+1−1s)/((s−1)Γ(n))C_s,n=ζ(s)^-1/s (n+1- 1s)/((s-1) (n)). For integer n the latter equals the occupancy form ζ(s)−1/s∑k<nΓ(k+1−1s)/k! ζ(s)^-1/ss _k<n (k+1- 1s)/k!. Likewise κsΘβ−Θ≤Fπ(Θ)≤κsΘβ _s ^β- ≤ F_π( )≤ _s ^β with κs=Γ(1s)/((s−1)ζ(s)1/s) _s= ( 1s)/((s-1)ζ(s)^1/s). Proof. The summand g(π)=π(1−(π/(π+x))n)g(π)=π(1-(π/(π+x))^n) is nondecreasing in π and bounded by nxnx. Hence the sum lies between ∫1∞ _1^∞ and ∫0∞ _0^∞ of g(t−s/ζ(s))dtg(t^-s/ζ(s))\,dt, and the two differ by at most nxnx. Substituting y=xζ(s)tsy=xζ(s)t^s and integrating by parts turns ∫0∞ _0^∞ into a Beta integral, which gives Ks,nxβK_s,nx^β. The identity ∑k<nΓ(k+a)/k!=Γ(n+a)/(aΓ(n)) _k<n (k+a)/k!= (n+a)/(a (n)) follows by induction. The bounds on FπF_π follow in the same way with g(π)=π(1−e−Θ/π)≤Θg(π)=π(1-e^- /π)≤ . ∎ Lemma 12 (From Poisson to exponential). For n≥1n≥ 1 and y≥0y≥ 0, (1−C∗/n)(1−e−ny)≤1−(1+y)−n≤1−e−ny(1-C_ /n)(1-e^-ny)≤ 1-(1+y)^-n≤ 1-e^-ny with C∗=1.1047C_ =1.1047. Consequently (1−C∗/n)Fπ(nx)≤Mfix(n,x)≤Fπ(nx)(1-C_ /n)F_π(nx)≤ M_ fix(n,x)≤ F_π(nx). The function FπF_π is concave and nondecreasing with Fπ(0)=0F_π(0)=0. For random N, 1−ρN≤1−ρN1-Eρ^N≤ 1-ρ^EN by Jensen’s inequality. Proof of Theorem 3: converse. Fix template i and let ℋeH_e be the history up to the start of epoch e. We bound the expected unfolded count of one epoch given ℋeH_e, and then average over epochs with a martingale strong law. One epoch. Condition on ℋeH_e. Let T1<T2<⋯T_1<T_2<·s be the audit times of template i from the epoch start (continued past its end), YkY_k the audited outcomes, σ the first fold event of the epoch, of a candidate formed in any epoch (σ=∞σ=∞ if there is none), and N=#k:Tk<σN=\#\k:T_k<σ\. By (A5), N is a stopping time for ℱm=σ(ℋe,,Y1,…,Ym)F_m=σ(H_e,G,Y_1,…,Y_m), and σ is ℱNF_N-measurable. By (A4), dℙr1/dℙr0|ℱm=eWmdP_r_1/dP_r_0|_F_m=e^W_m, where WmW_m is the log-likelihood-ratio walk; this holds whichever candidates the policy forms or audits, because every audit reveals one outcome with the same law. Fix an integer n≤n⋆n≤ n and c≥0c≥ 0. Let Q<nQ^<n be the number of the epoch’s queries preceded by fewer than n of its audits, and let R be the number of those that arrive after the first fold event σ. Every query answered by a candidate of the epoch arrives after σ, so the unfolded count satisfies U≥Q<n−RU≥ Q^<n-R. Moreover, R is a function of σ and G, hence ℱNF_N-measurable even if the policy later unfolds and refolds, and R>0R>0 implies N≤n−1N≤ n-1. • Low evidence. Wald’s likelihood-ratio identity for the ℱNF_N-measurable R≥0R≥ 0 gives r1[R;WN≤c,N<∞]≤ecr0[R]E_r_1[R;W_N≤ c,N<∞]≤ e^cE_r_0[R]. Under r0r_0 all candidates are bad, so by Definition 1 the epoch has a fold event with probability at most δ given ℋeH_e. Given ℱσF_σ, the strong Markov property of the Poisson arrival, audit and change processes bounds the mean number of queries between σ and the earlier of the n-th audit and the epoch end by min(n/p,πi/ν) (n/p, _i/ν). Hence r0[R∣ℋe]≤δmin(n/p,πi/ν)E_r_0[R _e]≤δ (n/p, _i/ν). • High evidence. On WN>c,R>0\W_N>c,R>0\ we have maxk≤n−1Wk>c _k≤ n-1W_k>c. Since R≤Q<nR≤ Q^<n and Q<nQ^<n is G-measurable, r1[R;WN>c∣ℋe]≤Q<n⋅βn−1(c)E_r_1[R;W_N>c _e] ^<n· _n-1(c), where βm(c)=ℙr1(maxk≤mWk≥c) _m(c)=P_r_1( _k≤ mW_k≥ c). Since Q<n=πiν(1−ρin)EQ^<n= _iν(1- _i^n) by the computation in Lemma 10, the unfolded count UeU_e of epoch e satisfies [Ue∣ℋe]≥πiνbiE[U_e _e]≥ _iν\,b_i, where bi≔(1−βn−1(c))(1−ρin)−ecδmin(1,nνpπi).b_i (1- _n-1(c))(1- _i^n)-e^cδ (1, nνp _i ). Many epochs. Let QeQ_e be the number of queries of template i in epoch e. The QeQ_e are i.i.d. geometric with mean πi/ν _i/ν whatever the policy does, so Qe2<∞EQ_e^2<∞. The differences De=Ue−[Ue∣ℋe]D_e=U_e-E[U_e _e] are martingale differences with De2≤Q12ED_e^2 _1^2, because 0≤Ue≤Qe0≤ U_e≤ Q_e. Hence ∑eDe/e _eD_e/e converges a.s. by the L2L^2 martingale convergence theorem, and Kronecker’s lemma gives 1K∑e≤KDe→0 1K _e≤ KD_e→ 0 a.s. (Hall and Heyde, 1980, Thm. 2.18). With the ordinary strong law for (Qe)(Q_e), lim infK∑e≤KUe/∑e≤KQe≥bi _K _e≤ KU_e/ _e≤ KQ_e≥ b_i a.s. The last, incomplete epoch contributes a vanishing share because QK/K→0Q_K/K→ 0 a.s., so the long-run unfolded fraction satisfies lim infui≥bi u_i≥ b_i a.s. Choice of n and c. Take n−1=⌊(1−ε)n⋆⌋n-1= (1- )n and c=(1−ε/2)Lc=(1- /2)L, with ε→0 → 0 and εL/lnL→∞ L/ L→∞. Then βn−1(c)→0 _n-1(c)→ 0 by Azuma’s maximal inequality for W (the mean of Wn−1W_n-1 is at most (1−ε)L(1- )L), and ecδ=δε/2→0e^cδ=δ /2→ 0. Lemma 12 and concavity give 1−ρin≥(1−C∗/n)(1−ε)(1−e−Θ/πi)1- _i^n≥(1-C_ /n)(1- )(1-e^- / _i), and min(1,Θ/πi)≤e−1(1−e−Θ/πi) (1, / _i)≤ ee-1(1-e^- / _i). Hence lim infui≥(1−εδ)(1−e−Θ/πi) u_i≥(1- _δ)(1-e^- / _i) with εδ=O(lnL/L) _δ=O( L/L), uniformly in i, π, ν and p. Summing over templates with Fatou’s lemma for series gives lim infM≥(1−εδ)Fπ(Θ) M≥(1- _δ)F_π( ) a.s. Announcements and candidates. The argument used announcements only through ℱmF_m, which contains the change times through G. A policy that does not observe changes is non-anticipating for a smaller filtration, so the bound holds for it a fortiori. The number of candidates entered only through the likelihood ratio, which (A2) and (A4) make independent of which candidate is audited. ∎ Proof of Theorem 3: achievability. At each announced change the policy discards all candidates’ evidence and restarts one SPRT, so every fold event of an epoch is a crossing of that epoch’s test. Under r≤r0r≤ r_0, eZ≤1Ee^Z≤ 1 for each increment of the SPRT statistic. So eΛne _n is a nonnegative supermartingale, and Ville’s inequality gives ℙ(supnΛn≥L)≤δP( _n _n≥ L)≤δ under arbitrary drift below r0r_0 and adaptive audit schedules. Under r≥r1r≥ r_1, Wald’s identity with overshoot <a+<a_+ gives N≤(L+a+)/KLEN≤(L+a_+)/KL. By Lemma 10, Jensen’s inequality and Lemma 12, MSPRT≤Fπ(xN)≤(1+a+/L)Fπ(Θ)M_ SPRT≤ F_π(xEN)≤(1+a_+/L)F_π( ), where the last step uses concavity of FπF_π. ∎ The per-template gap of the SPRT can be bracketed exactly, which sharpens the Jensen step above and makes the limit quantitative at practical δ. Lemma 13 (Finite-δ bracket). Let Z be the SPRT increment, a+=ln(r1/r0)a_+= (r_1/r_0) its value on a success, and let ξi>0 _i>0 solve r1eξiZ=1+ν/(pπi).E_r_1e _iZ=1+ν/(p _i). (7) If every epoch runs the SPRT at level δ, the unfolded fraction uiu_i of template i (so M=∑iπiuiM= _i _iu_i) satisfies 1−δξi≤ui≤1−δξie−ξia+.1-δ _i≤ u_i≤ 1-δ _ie^- _ia_+. Every per-epoch δ-safe certifier that folds at a stopping time of the audited outcomes has ui≥1−δξi/(1+ξi)u_i≥ 1-δ _i/(1+ _i). As ν/(pπi)→0ν/(p _i)→ 0 with Θ/πi / _i fixed, ξiln(1/δ)→Θ/πi _i (1/δ)→ / _i, and all three bounds tend to 1−e−Θ/πi1-e^- / _i. Proof of Lemma 13. Write L=ln(1/δ)L= (1/δ) and σ2=Varr1Zσ^2=Var_r_1Z. The right side of Equation 7 is 1/ρi1/ _i, and N is the SPRT sample size in audited outcomes. By Lemma 10, ui=1−ρiNu_i=1-E _i^N, since audited outcomes of template i arrive at rate pπip _i and the epoch ends at rate ν. The function ψ(ξ)=lnr1eξZψ(ξ)= _r_1e^ξ Z is convex with ψ(0)=0ψ(0)=0, ψ′(0)=KL>0ψ (0)=KL>0 and ψ(ξ)→∞ψ(ξ)→∞, so Equation 7 has a unique root. Hence Mn=ρineξiΛnM_n= _i^ne _i _n is a mean-one ℙr1P_r_1-martingale, as in Wald’s fundamental identity (Siegmund, 1985). Bracket. Before the crossing Mn≤eξiLM_n≤ e _iL, and at the crossing L≤ΛN<L+a+L≤ _N<L+a_+. Since N<∞N<∞ a.s., dominated convergence gives MN=1EM_N=1, and the bounds on ΛN _N give δξie−ξia+≤ρiN≤δξiδ _ie^- _ia_+ _i^N≤δ _i. Converse. Write ρiN=MN1/(1+ξi)(ρiNe−ΛN)ξi/(1+ξi) _i^N=M_N^1/(1+ _i)( _i^Ne^- _N) _i/(1+ _i). By Hölder’s inequality with exponents 1+ξi1+ _i and 1+ξiξi 1+ _i _i, r1[ρiN;N<∞]E_r_1[ _i^N;N<∞] is at most r1[MN;N<∞]1/(1+ξi)E_r_1[M_N;N<∞]^1/(1+ _i) times r1[ρiNe−ΛN;N<∞]ξi/(1+ξi)E_r_1[ _i^Ne^- _N;N<∞] _i/(1+ _i). The first factor is ≤1≤ 1 by Fatou’s lemma. By change of measure the second expectation is r0[ρiN;N<∞]E_r_0[ _i^N;N<∞], the probability of folding a bad candidate within the epoch, which is ≤δ≤δ. Limit. Expanding ψ(ξ)=KLξ+σ2ξ2/2+O(ξ3)ψ(ξ)=KLξ+σ^2ξ^2/2+O(ξ^3) and ln(1+y)=y+O(y2) (1+y)=y+O(y^2) in Equation 7 gives ξiL=Θπi(1+O(νpπi(1+σ2KL2))) _iL= _i (1+O( νp _i(1+ σ^2KL^2)) ), and ξia+→0 _ia_+→ 0. ∎ The bracket is two-sided and sharper than the Jensen bound ui≤1−ρiNu_i≤ 1- _i^EN used for Theorem 3. For (r0,r1)=(0.85,0.95)(r_0,r_1)=(0.85,0.95), δ=10−2δ=10^-2 and Θ/πi=0.908 / _i=0.908, exact dynamic programming gives ui=0.5610∈[0.5557,0.5643]u_i=0.5610∈[0.5557,0.5643]. Here ξi=0.1762 _i=0.1762, the converse bound is 0.49840.4984, the first-order value 1−e−Θ/πi1-e^- / _i is 0.59660.5966, and the Jensen bound is 0.59980.5998. Universality. Suppose G(θ)=∑i:πi≤θπiG(θ)= _i: _i≤θ _i is regularly varying with index 1−α∈(0,1)1-α∈(0,1). Writing Fπ(Θ)=1−e−Θ+∫Θ∞G(Θ/y)e−yyF_π( )=1-e^- + _ ^∞G( /y)e^-ydy and applying Potter’s bounds with dominated convergence gives Fπ(Θ)∼Γ(α)G(Θ)F_π( ) (α)G( ) as Θ→0 → 0. For Zipf demand, α=1/sα=1/s. A finite-size law for cold starts. The steady state of Theorem 3 is reached only after about 1/ν1/ν queries. The following law interpolates between a cold start and the drift floor. It is stated for a fixed certification sample size n, not for sequentially tested (safe) folding; extending it to safe folding is open. Theorem 14 (One law for cold start and drift). Let demand be Zipf(s)(s), start all templates cold at time 00, let facts change at rate ν, and fold after n audited uses. The probability that the query at time T is not answered in one step satisfies M(T,ν)=(pT)−(1−1s)Φs,n(νT)(1+o(1))M(T,ν)=(pT)^-(1- 1s)\, _s,n(ν T)\,(1+o(1)) with relative error O(m1/sln(1/m))O(m^1/s (1/m)), m=max(1/(pT),ν/p)m= (1/(pT),ν/p), where Φs,n(y)=Cs,n[e−y+y1−1sγinc(1s,y)] _s,n(y)=C_s,n [e^-y+y^1- 1s _ inc( 1s,y) ], Cs,nC_s,n is as in Lemma 11, and γinc _ inc is the lower incomplete gamma function. At y=νT=0y=ν T=0 this is the static amortization law M(T)∼Cs,n(pT)−(1−1/s)M(T) C_s,n(pT)^-(1-1/s), a corollary of Karlin (1967) and Ohannessian and Dahleh (2012, Prop. 7) summed over r<nr<n. It is a power law of practice (Newell and Rosenbloom, 1981; Heathcote et al., 2000) whose exponent 1−1/s1-1/s, shared by memorization learning curves (Hutter, 2021; Michaud et al., 2023), is set by the task distribution. As y→∞y→∞ it tends to Γ(1s)Cs,n(ν/p)1−1/s=κsΘ1−1/s(1+O(1/n)) ( 1s)C_s,n(ν/p)^1-1/s= _s ^1-1/s(1+O(1/n)) with Θ=nν/p =nν/p, the steady state of Corollary 4. Section D.1 tests the static branch on natural template streams. Proof of Theorem 14. With changes at rate ν from a cold start at 00, the age of the current epoch at time T is AT=min(T,Eν)A_T= (T,E_ν), and the audited count of template i given ATA_T is Poi(pπiAT)Poi(p _iA_T). For audited exposure a (i.e. a=pATa=pA_T), the sum Sn(a)=∑iπiℙ(Poi(πia)<n)S_n(a)= _i _iP(Poi( _ia)<n) satisfies |Sn(a)−Cs,na−β|≤2n/a|S_n(a)-C_s,na^-β|≤ 2n/a. The summand is unimodal in i with maximum at most n/an/a, and its integral is Cs,na−βC_s,na^-β. Averaging (pAT)−β(pA_T)^-β gives p−βAT−β=(pT)−β[e−y+yβγinc(1−β,y)]p^-βEA_T^-β=(pT)^-β[e^-y+y^β _ inc(1-β,y)] with y=νTy=ν T, which is the stated Φs,n _s,n. The error bound O(m1/sln(1/m))O(m^1/s (1/m)) with m=max(1/(pT),ν/p)m= (1/(pT),ν/p) follows by integrating min(2n/a,1) (2n/a,1) against the law of pATpA_T. ∎ Appendix C Certifying utility by betting The SPRT of Section 4 tests reliability against r0=r⋆r_0=r (Equation 1), which depends on the live accuracy a. Here we certify the macro’s net utility relative to live on paired shadow evaluations instead, so a never enters the null. The tools are testing by betting (Shafer, 2021; Ramdas et al., 2023; Waudby-Smith and Ramdas, 2024) with inverse-propensity audits, as in Xu et al. (2024) and Yu et al. (2026b). Setup. Fix a template and a frozen candidate. At shadow evaluation t=1,2,…t=1,2,… both paths run on the same instance. Live has cost CtLC^L_t and correctness YtL∈0,1Y^L_t∈\0,1\; the macro has CtMC^M_t and YtMY^M_t. The paired utility of folding is Ut=CtL−CtM+L(YtM−YtL).U_t=C^L_t-C^M_t+L\,(Y^M_t-Y^L_t). (8) Given a known B with Ut≥−BU_t≥-B, put Zt=Ut/B∈[−1,M]Z_t=U_t/B∈[-1,M]. In the paired Bernoulli model (CL≡mC^L≡ m, CM≡1C^M≡ 1), let η=(m−1)/L<1η=(m-1)/L<1, Dt=YtM−YtLD_t=Y^M_t-Y^L_t and B=L(1−η)B=L(1-η). Then Zt=(η+Dt)/(1−η)Z_t=(η+D_t)/(1-η) and Zt=(r−r⋆)/(1−η)EZ_t=(r-r )/(1-η). Correctness is revealed only on audited instances (At=1A_t=1). Let ℱt−1F_t-1 be everything that happened before t, observed or not, and let WtW_t be pre-audit side information (costs, both answers, a judge score). Put ℋt=ℱt−1∨σ(Wt)H_t=F_t-1 σ(W_t), and let t−1⊆ℱt−1G_t-1 _t-1 be the system’s records. We assume: (A1) the audit probability ptp_t is ℋtH_t-measurable and computable, and pt≥πtp_t≥ _t on pt>0\p_t>0\ for a t−1G_t-1-measurable floor πt∈(0,1] _t∈(0,1]; (A2) ℙ(At=1∣ℋt,Zt)=ptP(A_t=1 _t,Z_t)=p_t; (A3) a computable surrogate gt∈[−1,G¯]g_t∈[-1, G] satisfies Zt=gtZ_t=g_t on pt=0\p_t=0\. The estimator Z^t=gt+At(Zt−gt)/pt Z_t=g_t+A_t(Z_t-g_t)/p_t (with 0/0≔00/0 0) is observable. Taking g≡0g≡ 0 gives IPW, and p≡1p≡ 1 gives full information. The candidate is inadmissible under H0:[Zt∣ℱt−1]≤0 a.s. for all t.H_0:\ E[Z_t _t-1]≤ 0\ a.s.\ for all t. (9) H0H_0 leaves arbitrary the joint law of (CL,CM,YL,YM)(C^L,C^M,Y^L,Y^M), including the live–macro error dependence, and its dependence on the past. Drifting live accuracy, shared difficulty and adaptive query selection are therefore all allowed. Test. Let κt=(1+G¯(1−πt))/πt _t=(1+ G(1- _t))/ _t, which is 1/πt1/ _t for IPW. Take t−1G_t-1-measurable bets λt(j)∈[0,1/κt]λ^(j)_t∈[0,1/ _t] and weights wj≥0w_j≥ 0 with ∑jwj≤1 _jw_j≤ 1. Fold at τ=infn:Kn≥1/δτ= \n:K_n≥ 1/δ\, where Kn=∑jwj∏t≤n(1+λt(j)Z^t).K_n= _jw_j _t≤ n (1+λ^(j)_t Z_t ). (10) The grid mixture at audit rate p uses λ(j)=pcjλ^(j)=p\,c_j, cj=(1+ε)−jc_j=(1+ )^-j, wj=1/(j(j+1))w_j=1/(j(j+1)) and ε=1/ln(1/δ) =1/ (1/δ). Theorem 15 (Anytime validity). Under (A1)–(A3) and H0H_0, (Kn)(K_n) is a nonnegative supermartingale with K0≤1K_0≤ 1. Hence ℙ(∃n:Kn≥1/δ)≤δP(∃ n:K_n≥ 1/δ)≤δ for every schedule of shadow evaluations and every stopping or abandonment rule. Proof. Positivity. If At=0A_t=0, then Z^t=gt≥−1≥−κt Z_t=g_t≥-1≥- _t. If At=1A_t=1, then Z^t=gt(1−1/pt)+Zt/pt≥G¯−(1+G¯)/pt Z_t=g_t(1-1/p_t)+Z_t/p_t≥ G-(1+ G)/p_t. This bound increases in ptp_t, so it is ≥−κt≥- _t because pt≥πtp_t≥ _t. Hence every factor is ≥1−λt(j)κt≥0≥ 1-λ^(j)_t _t≥ 0. Unbiasedness. On pt>0\p_t>0\, (A2) gives [Z^t∣ℋt,Zt]=ZtE[ Z_t _t,Z_t]=Z_t. On pt=0\p_t=0\, Z^t=gt=Zt Z_t=g_t=Z_t by (A3). Since ℱt−1⊆ℋtF_t-1 _t, [Z^t∣ℱt−1]=[Zt∣ℱt−1]≤0E[ Z_t _t-1]=E[Z_t _t-1]≤ 0. Supermartingale. Because λt(j)λ^(j)_t is ℱt−1F_t-1-measurable, each product in Equation 10 is a nonnegative supermartingale, and so is their weighted sum, by monotone convergence. Ville. Let σ=infn:Kn≥1/δσ= \n:K_n≥ 1/δ\. Optional stopping at σ∧nσ n gives 1≥Kσ∧n≥δ−1ℙ(σ≤n)1 _σ n≥δ^-1P(σ≤ n). Let n→∞n→∞ (Ville, 1939). ∎ Remark 3. Validity has three limits. • Bets may not use WtW_t. • (A2) excludes outcome-dependent labelling, such as labels that arrive through user complaints. • Drift is covered only inside H0H_0, so folded macros must still be monitored. Audits can also be triggered by disagreement. Audit only when the two answers differ, and set gt=(CtL−CtM)/Bg_t=(C^L_t-C^M_t)/B on agreement, which (A3) allows only under exact-match scoring (condition (E) of Section C.1). Then pt∈0,1p_t∈\0,1\ and κt=1 _t=1, so the test keeps full-information power at a label cost of ℙ(answers differ)τP(answers differ)\,Eτ. Unlike a sign test on discordant pairs (Shawn, 2026), concordant pairs here carry the deterministic gain η/(1−η)η/(1-η). A macro slightly less accurate than live is therefore still certifiable when r>r⋆r>r . The information constant. For a law P of Z on [−1,M][-1,M] with mean μ, let gP(c)=Pln(1+cZ)g_P(c)=E_P (1+cZ) and g∗(P)=maxc∈[0,1]gP(c)g^*(P)= _c∈[0,1]g_P(c), attained at c∗c^*. Let KLinf(P)=infKL(P∥Q):QZ≤0KL_ (P)= \KL(P\|Q):E_QZ≤ 0\ over laws Q on [−1,M][-1,M]. Lemma 16 (Growth equals information). (The dual form of KLinfKL_ is due to Honda and Takemura (2010); we include a short proof for bounded Z.) g∗(P)=KLinf(P)g^*(P)=KL_ (P), and g∗>0g^*>0 iff μ>0μ>0. If μ>0μ>0, the infimum is attained by Q∗(dz)=P(dz)/(1+c∗z)+wδ−1(dz)Q^*(dz)=P(dz)/(1+c^*z)+w\, _-1(dz) with w=1−P[(1+c∗Z)−1]≥0w=1-E_P[(1+c^*Z)^-1]≥ 0. Proof. (≤). Let QZ≤0E_QZ≤ 0 and ℓ=dP/dQ =dP/dQ. For every c∈[0,1]c∈[0,1], Jensen gives Pln1+cZℓ≤lnQ[(1+cZ)ℓ>0]≤ln(1+cQZ)≤0E_P 1+cZ ≤ _Q[(1+cZ)1_ >0]≤ (1+c\,E_QZ)≤ 0. (≥). If μ≤0μ≤ 0, both sides are 00. Otherwise c∗>0c^*>0. If c∗<1c^*<1, differentiation under E is legitimate because |Z/(1+cZ)|≤max(1c,11−c)|Z/(1+cZ)|≤ ( 1c, 11-c). The first-order condition P[Z/(1+c∗Z)]=0E_P[Z/(1+c^*Z)]=0 then shows that w=0w=0 and that Q∗Q^* has mass 11 and mean 00. If c∗=1c^*=1, then P(−1)=0P(\-1\)=0, and the left derivative P[Z/(1+Z)]=wE_P[Z/(1+Z)]=w is ≥0≥ 0, so Q∗Q^* again has mass 11 and mean 00. In both cases KL(P∥Q∗)=Pln(1+c∗Z)=g∗KL(P\|Q^*)=E_P (1+c^*Z)=g^*. ∎ Since Q∗Q^* lives on suppP∪−1suppP∪\-1\, in the paired Bernoulli model KLinfKL_ equals the information of the three-point model. Locally, KLinf=μ22σ2(1+O(μ))KL_ = μ^22σ^2(1+O(μ)) with σ2=Var(Z)σ^2=Var(Z). Observation models. A lower bound on certification time depends on what the rule observes. Let U be an independent randomization of the rule, and let ξ1,ξ2,… _1, _2,… be i.i.d. uniform audit coins, independent of U and of all shadow outcomes. (O1) Audited utilities only. After t shadow evaluations the rule’s records are ℛt=σ(U,(As,AsZs)s≤t)R_t=σ(U,(A_s,A_sZ_s)_s≤ t), and At=ξt≤ptA_t=1\ _t≤ p_t\, where ptp_t is a fixed (law-independent) function of the records ℛt−1R_t-1. Unaudited shadow outputs, including whether the two answers agree, are neither used nor allowed to trigger audits. (O2) Audited utilities and agreement. The records also contain the agreement indicators GsG_s, and At=ξt≤pt(Gt)A_t=1\ _t≤ p_t(G_t)\, where pt(⋅)p_t(·) is a fixed function of the records before t; audits may thus depend on GtG_t but not on ZtZ_t (Section C.1). The IPW certificate (g≡0g≡ 0) with audits that do not depend on WtW_t is an (O1) rule. So is our restart-only certificate, which updates its wealth only on audited pairs. Theorem 17 (Certification time and optimality). Let ZtZ_t be i.i.d. P with μ>0μ>0. (a) Suppose audits are i.i.d. Bernoulli(p)(p) and independent of (Zt)(Z_t). Under the grid mixture, the label count NpN_p has the law of the full-information stopping time τ1 _1, τp=τ1/pE _p=E _1/p, and Np≤ln1δ+2lnln1δ+O(1)KLinf(P),EN_p≤ 1δ+2 1δ+O(1)KL_ (P), where O(1)O(1) depends only on c∗c^* and M. (b) Consider a rule of type (O1) with pt≤p¯p_t≤ p, folding at a stopping time τ of (ℛt)(R_t). Let ℙQ(τ<∞)≤δP_Q(τ<∞)≤δ for every i.i.d. law Q of (Zt)(Z_t) on [−1,M][-1,M] with QZ≤0E_QZ≤ 0, and let ℙP(τ<∞)=1P_P(τ<∞)=1. Then P∑t≤τAt≥ln(1/δ)/KLinf(P), _P _t≤τA_t≥ (1/δ)/KL_ (P), Pτ≥ln(1/δ)/(p¯KLinf(P)). _Pτ≥ (1/δ)/ ( p\,KL_ (P) ). Hence, among (O1) rules, τp=(1+o(1))ln(1/δ)/(pKLinf(P))E _p=(1+o(1)) (1/δ)/(p\,KL_ (P)) is optimal as δ→0δ→ 0. Here P is the law of the unweighted ZtZ_t. The audit rate enters only through the factor 1/p1/p (proof of (a)); expressing the rate through the law of the weighted increment and dividing by p again would count p twice. Part (b) does not extend beyond (O1): a rule that also sees whether the answers agree can be faster (Propositions 21 and 22). Proof. (a) With λ(j)=pcjλ^(j)=pc_j, the factor 1+cjAtZt1+c_jA_tZ_t equals 11 on unaudited steps. The wealth therefore moves only at audits, whose outcomes are i.i.d. P and independent of the Geometric(p)(p) gaps. So Np=τ1N_p d= _1, and Wald’s identity gives τp=Np/pE _p=EN_p/p. Pick j∗j^* with cj∗∈[c∗/(1+ε),c∗]c_j^*∈[c^*/(1+ ),c^*]. By concavity, g(cj∗)≥g∗/(1+ε)g(c_j^*)≥ g^*/(1+ ). Since Kn≥wj∗∏t≤n(1+cj∗Zt)K_n≥ w_j^* _t≤ n(1+c_j^*Z_t), Wald’s identity with overshoot ≤ln(1+M)≤ (1+M) gives τ1≤ _1≤ 1+εg∗[ln1δ+ln(j∗(j∗+1)) 1+ g^* [ 1δ+ (j^*(j^*+1) ) +ln(1+M)]. + (1+M) ]. Finally, j∗≤1+2ln(1/c∗)ln(1/δ)j^*≤ 1+2 (1/c^*) (1/δ). (b) Take Q∗Q^* from Lemma 16, so that dP/dQ∗=1+c∗zdP/dQ^*=1+c^*z on suppPsuppP, and put ℓ=ln(1+c∗Z) = (1+c^*Z). Under an i.i.d. law Q of (Zt)(Z_t) the records ℛnR_n have, with respect to the law of U times a fixed dominating measure, the density ∏t≤nptAt(1−pt)1−Atq(Zt)At _t≤ np_t^A_t(1-p_t)^1-A_t\,q(Z_t)^A_t, where q is the density of Q. Under (O1) the audit factors are the same functions of the records under P and Q∗Q^*, because the coins ξt _t and the map ℛt−1↦ptR_t-1 p_t do not depend on the law; they cancel. The likelihood ratio of ℛnR_n is therefore exp∑t≤nAtℓ(Zt) _t≤ nA_t (Z_t). (This is the step that fails outside (O1): if the records contain WtW_t, or audits depend on WtW_t, the ratio also contains the likelihood ratio of WtW_t, whose law may differ between P and every null.) Since ZtZ_t and ξt _t are independent of ℛt−1R_t-1 and of each other, P[Atℓ(Zt)∣ℛt−1]=ptg∗E_P[A_t (Z_t) _t-1]=p_tg^*. By optional stopping at τ∧nτ n, the KL divergence between the laws of the records up to τ∧nτ n is g∗P∑t≤τ∧nAtg^*E_P _t≤τ nA_t. Data processing on τ≤n\τ≤ n\, whose Q∗Q^*-probability is ≤δ≤δ, together with kl(x,y)≥xln1y−h(x)kl(x,y)≥ x 1y-h(x), bounds this below by xnln1δ−h(xn)x_n 1δ-h(x_n), where xn=ℙP(τ≤n)→1x_n=P_P(τ≤ n)→ 1. Now let n→∞n→∞ (Kaufmann et al., 2016; Garivier and Kaufmann, 2016). Finally, P∑t≤τAt=P∑t≤τpt≤p¯PτE_P _t≤τA_t=E_P _t≤τp_t≤ p\,E_Pτ. ∎ Why the plug-in reliability test fails. The SPRT is a Kelly bet: eΛn=∏k≤n(1+b(Yk−r0))e _n= _k≤ n(1+b(Y_k-r_0)) with b=r1−r0r0(1−r0)b= r_1-r_0r_0(1-r_0). With r0=r⋆r_0=r it is the betting test on (YM−r⋆)/r⋆(Y^M-r )/r , and it is valid only if a is known. Proposition 18 (Plug-in tests are not level δ; restates Proposition 5). Let a^=k/na a=k/n_a with k∼Bin(na,a)k (n_a,a), estimated from independent live runs. Run the one-sided SPRT of Section 4 with r0=a^−ηr_0= a-η and r1=a^r_1= a on i.i.d. macro outcomes at the break-even r=r⋆r=r . Let E=η<a^<1E=\η< a<1\ and rmid=ln1−r01−r1/lnr1(1−r0)r0(1−r1)∈(r0,r1).r_ mid= 1-r_01-r_1 / r_1(1-r_0)r_0(1-r_1)∈(r_0,r_1). Then: (i) ℙ(fold)≥ℙ(E,rmid<r⋆)≥ℙ(η<a^≤r⋆)P(fold) (E,r_ mid<r ) (η< a≤ r ); (i) ℙ(fold)≥[Er0<r⋆(δr0/r1)θ]P(fold) [1_E1_\r_0<r \(δ r_0/r_1)^θ ]. In (i), θ=0θ=0 if r⋆≥rmidr ≥ r_ mid. Otherwise θ∈(0,1)θ∈(0,1) solves r⋆(r1/r0)θ+(1−r⋆)(1−r11−r0)θ=1r (r_1/r_0)^θ+(1-r )( 1-r_11-r_0)^θ=1. For large nan_a, the middle term of (i) is ≈Φ(−η2na/(a(1−a)))≈ (- η2 n_a/(a(1-a)) ). Proof. Condition on a^∈E a∈ E. The increments XkX_k of Λ are i.i.d. with mean KL(r⋆∥r0)−KL(r⋆∥r1)KL(r \|r_0)-KL(r \|r_1). This mean is affine in r⋆r and vanishes at rmidr_ mid. If r⋆≥rmidr ≥ r_ mid, the drift is nonnegative and the steps are non-degenerate, so lim supΛn=∞ _n=∞ and the test folds a.s. On E, a^≤r⋆ a≤ r implies rmid<r1≤r⋆r_ mid<r_1≤ r , which proves (i). For (i), let r0<r⋆<rmidr_0<r <r_ mid. The map θ↦eθXθ ^θ X is strictly convex. It equals 11 at 00 with negative slope and equals 1+b(r⋆−r0)>11+b(r -r_0)>1 at 11, so the root θ exists and eθΛne^θ _n is a martingale. At the fold time σ, Λσ≤ln1δ+lnr1r0 _σ≤ 1δ+ r_1r_0; on σ=∞\σ=∞\, Λn→−∞ _n→-∞. Bounded convergence therefore gives 1=[eθΛσ;σ<∞]≤(r1/(δr0))θℙ(σ<∞∣a^)1=E[e^θ _σ;σ<∞]≤(r_1/(δ r_0))^θ\,P(σ<∞ a). ∎ nan_a 10 20 50 100 200 500 1000 δ=0.05δ=0.05 .28 .34 .30 .25 .19 .11 .081 δ=0.01δ=0.01 .27 .33 .26 .20 .13 .058 .030 bound (i) .26 .32 .23 .20 .10 .025 .003 Table 4: Exact false-fold probability of the plug-in SPRT for a macro at break-even, computed by dynamic programming (a=0.9a=0.9, (m−1)/L=0.05(m-1)/L=0.05, horizon 2000 uses). Table 4 shows that the excess over δ persists long after the middle term of (i) drops below δ. At na=1000n_a=1000 and δ=0.01δ=0.01, bound (i) gives 0.0350.035 and the exact value is 0.0340.034. Macros slightly below break-even behave similarly. At shortfall 0.040.04 with na≤20n_a≤ 20 live labels, the rate is 0.10.1–0.40.4 across a∈[0.5,0.9]a∈[0.5,0.9] and η∈[0.05,0.2]η∈[0.05,0.2]. On the full KQA-Stream tables (Qwen3-8B, 30 streams) both certificates stay far below δ on below-break-even candidates. The plug-in SPRT folds 0.32%0.32\% of them (477 of 147,349 encounters) and the betting test 0.017%0.017\% (33 of 190,187), while counter rules fold all of them. The failure of the plug-in test is a boundary phenomenon: it needs candidates just below break-even and live accuracy estimated from few labels. Most spurious candidates in our streams are far below break-even: the ones that counters fold fall short by a median of 0.680.68. Pairing. Write P1≤cxP2P_1 _ cxP_2 if ϕ(Z1)≤ϕ(Z2)Eφ(Z_1) φ(Z_2) for all convex ϕφ. Theorem 19 (Pairing gain). If P1≤cxP2P_1 _ cxP_2 on [−1,M][-1,M], then KLinf(P1)≥KLinf(P2)KL_ (P_1) _ (P_2), strictly if P1≠P2P_1≠ P_2 and μ>0μ>0. Now take the paired Bernoulli model with marginals (a,r)(a,r) fixed and r>r⋆r>r . Then KLinfKL_ is strictly increasing in γ=Cov(YL,YM)γ=Cov(Y^L,Y^M). Pairing therefore certifies faster than an unpaired design (live outcome from an independent instance) iff γ>0γ>0, that is, when live and macro fail on the same instances. Proof. For c<1c<1, z↦ln(1+cz)z (1+cz) is concave and bounded, so gP1(c)≥gP2(c)g_P_1(c)≥ g_P_2(c). Let c↑1c 1 and maximize. For strictness, Strassen’s theorem gives a coupling Z2=Z1+ξZ_2=Z_1+ξ with [ξ∣Z1]=0E[ξ Z_1]=0 and ξ≠0ξ≠ 0 with positive probability. At c=c∗(P2)c=c^*(P_2) the map ln(1+cz) (1+cz) is strictly concave where finite, so conditional Jensen is strict. Raising γ by Δ moves mass Δ from each of D=±1D=± 1 to D=0D=0. This changes ϕ(D)Eφ(D) by −Δ[ϕ(1)+ϕ(−1)−2ϕ(0)]≤0- [φ(1)+φ(-1)-2φ(0)]≤ 0 for convex ϕφ, and Z is affine in D. ∎ At a=r=0.9a=r=0.9 and η=0.05η=0.05, pairing raises KLinfKL_ by 1.85×1.85× at correlation 0.50.5 and by 7.56×7.56× at correlation 11. Near break-even, it beats even the SPRT that knows a iff the correlation exceeds a(1−a)/(2r⋆(1−r⋆)) a(1-a)/(2 r (1-r )), which is 0.420.42 here. The scaling law with KLinfKL_ . Corollary 20 (Coverage gap with KLinfKL_ ). Assume (A1)–(A3) and (A5) of Appendix B, and replace (A4) by: in every epoch the normalized paired utilities Zt=Ut/BZ_t=U_t/B of the audited shadow evaluations are i.i.d. from a finitely supported law PZP_Z on [−1,M][-1,M] with positive mean, whichever candidate they evaluate, and independent of G. Let ΘU≔νln(1/δ)pKLinf(PZ). _U ν\, (1/δ)p\,KL_ (P_Z). (i) Every policy of type (O1) that is δ-safe in the sense of Definition 1, with “bad” meaning Z≤0EZ≤ 0, has lim infM≥(1−εδ)Fπ(ΘU) M≥(1- _δ)F_π( _U) almost surely, with εδ=O(lnln(1/δ)/ln(1/δ)) _δ=O ( (1/δ)/ (1/δ) ) depending only on (δ,PZ)(δ,P_Z). (i) With announced changes, the grid-mixture betting test of Theorem 17(a), restarted at each change with all earlier evidence discarded, is δ-safe and has M≤(1+(2lnln1δ+O(1))/ln1δ)Fπ(ΘU)M≤ (1+(2 1δ+O(1))/ 1δ )F_π( _U). Thus Theorem 3 holds with KLinf(PZ)KL_ (P_Z) in place of KL(r1∥r0)KL(r_1\|r_0), and the live accuracy a need not be known. Proof. (i) Repeat the converse proof of Theorem 3 with three substitutions. The null environment. Let Q∗Q^* be the least-favorable null of Lemma 16; it has mean 00, so in the environment where the epoch’s audited utilities are i.i.d. Q∗Q^* every candidate is bad, and δ-safety bounds the probability of a fold event in the epoch by δ given ℋeH_e. The likelihood ratio. By Lemma 16, dPZ/dQ∗=1+c∗z>0dP_Z/dQ^*=1+c^*z>0 on suppPZsuppP_Z, so PZ≪Q∗P_Z Q^*. Under (O1) the records add to the audited utilities only the audit coins and the internal randomization, whose laws do not depend on the environment, exactly as in the proof of Theorem 17(b). Hence dℙPZ/dℙQ∗dP_P_Z/dP_Q^* restricted to ℱmF_m is eWme^W_m with Wm=∑k≤mℓ(Zk)W_m= _k≤ m (Z_k) and ℓ(z)=ln(1+c∗z) (z)= (1+c^*z), whichever candidates the policy forms or audits. Wald’s likelihood-ratio identity then gives the low-evidence bound PZ[R;WN≤c,N<∞]≤ecQ∗[R]E_P_Z[R;W_N≤ c,N<∞]≤ e^cE_Q^*[R] as before. The drift. Under PZP_Z the increments ℓ(Zk) (Z_k) are i.i.d., bounded (the support is finite and 1+c∗z>01+c^*z>0 on it) and have mean g∗=KLinf(PZ)g^*=KL_ (P_Z). With n⋆=ln(1/δ)/KLinf(PZ)n = (1/δ)/KL_ (P_Z), n−1=⌊(1−ε)n⋆⌋n-1= (1- )n and c=(1−ε/2)ln(1/δ)c=(1- /2) (1/δ), Azuma’s maximal inequality gives βn−1(c)→0 _n-1(c)→ 0, and the rest of the proof (one epoch, many epochs, choice of n and c) applies verbatim with Θ replaced by ΘU _U. (i) After each change the test restarts from wealth 11 on the epoch’s own evidence, so Theorem 15 and Ville’s inequality bound the probability of a fold of a null candidate in the epoch by δ, which is Definition 1. The wealth moves only at audits (Theorem 17(a)), so the test folds at the NeN_e-th audited outcome of epoch e, where the NeN_e are i.i.d., independent of arrivals, audits and changes, and have the law of NpN_p in Theorem 17(a). Lemma 10 gives ui=1−ρiNu_i=1-E _i^N, and Lemma 12 gives ui≤1−ρiN≤1−e−xN/πiu_i≤ 1- _i^EN≤ 1-e^-xEN/ _i with x=ν/px=ν/p. By Theorem 17(a), xN≤(1+ηδ)ΘUxEN≤(1+ _δ) _U with ηδ=(2lnln1δ+O(1))/ln1δ _δ=(2 1δ+O(1))/ 1δ. Since FπF_π is concave and nondecreasing with Fπ(0)=0F_π(0)=0, Fπ((1+η)ΘU)≤(1+η)Fπ(ΘU)F_π((1+η) _U)≤(1+η)F_π( _U); summing πiui _iu_i gives the bound. ∎ Pairing, disagreement-triggered audits and cost-aware utilities thus all act through one constant, KLinf(PZ)KL_ (P_Z), in the certification load. Under Zipf demand, a pairing gain ρg=KLinfpair/KLinfunpair _g=KL_ pair/KL_ unpair lowers the coverage gap by the factor ρg1−1/s _g^1-1/s. C.1 Agreement-aware certification Shadow mode runs both paths, so the system sees both answers before any audit. When the answers agree, their correctness labels agree, provided scoring depends on the answer alone: (E) Exact-match scoring. There are Gt∈0,1G_t∈\0,1\ and z0,tz_0,t, observed before the audit decision (functions of WtW_t), such that Gt=1G_t=1 implies Zt=z0,tZ_t=z_0,t. (E) holds if correctness is a deterministic function of the instance and the final answer, Y=ϕt(answer)Y= _t(answer) (exact match to a gold answer, or execution against fixed tests), if Gt=1G_t=1 exactly when the two final answers are identical as inputs to ϕt _t, and if both costs are observed; then Gt=1G_t=1 forces YtM=YtLY^M_t=Y^L_t, and z0,t=(CtL−CtM)/Bz_0,t=(C^L_t-C^M_t)/B. It fails for stochastic or rubric-based judges, for judges that read more than the final answer, and whenever a path’s score depends on anything other than its answer. KQA-Stream is such a case: a macro answer counts as correct only if the query was also routed to the right template, so on a misrouted query whose macro answer equals a correct live answer, Gt=1G_t=1 but YtM=0≠YtL=1Y^M_t=0≠ Y^L_t=1. The certificates of this subsection are therefore not valid for the KQA-Stream labels as defined; they are valid for a variant that scores a macro by its answer alone. Proposition 21 (Agreement breaks the (O1) bound). Assume (E) in the paired model with constant costs, so z0=η/(1−η)z_0=η/(1-η). Let Vt=z0V_t=z_0 if Gt=1G_t=1 and Vt=−1V_t=-1 otherwise. For every λ∈[0,1]λ∈[0,1], ∏s≤t(1+λVs) _s≤ t(1+λ V_s) is a test supermartingale under H0H_0, for any audit mechanism. If GtG_t is i.i.d. with ℙ(Gt=1)=gP(G_t=1)=g, the best constant bet grows the log-wealth by kl(g∥11+z0)kl (g\,\|\, 11+z_0 ) per shadow run if g>11+z0g> 11+z_0, and by 00 otherwise. If g=1g=1, the bet λ=1λ=1 folds after ⌈ln(1/δ)/ln(1+z0)⌉ (1/δ)/ (1+z_0) shadow runs and uses no labels, whereas every (O1) rule needs Pτ≥ln(1/δ)/(pln(1+z0))E_Pτ≥ (1/δ)/(p (1+z_0)). Proof. By (E), Vt=ZtV_t=Z_t if Gt=1G_t=1, and Vt=−1≤ZtV_t=-1≤ Z_t otherwise. So Vt≤ZtV_t≤ Z_t pointwise, and [Vt∣ℱt−1]≤[Zt∣ℱt−1]≤0E[V_t _t-1] [Z_t _t-1]≤ 0 whatever the audits do. The factors satisfy 1+λVt≥1−λ≥01+λ V_t≥ 1-λ≥ 0, and VtV_t is observed without a label. For i.i.d. GtG_t, VtV_t has the two-point law gδz0+(1−g)δ−1g _z_0+(1-g) _-1. By Lemma 16 the best constant bet grows at rate KLinfKL_ of this law, and its least-favorable null lives on z0,−1\z_0,-1\. A null law qδz0+(1−q)δ−1q _z_0+(1-q) _-1 has mean qz0−(1−q)≤0qz_0-(1-q)≤ 0 iff q≤1/(1+z0)q≤ 1/(1+z_0), and kl(g∥q)kl(g\|q) decreases in q on [0,g][0,g], which gives the rate. If g=1g=1, the wealth is (1+z0)n(1+z_0)^n. For (O1) rules, PZ=δz0P_Z= _z_0 and KLinf(δz0)=ln(1+z0)KL_ ( _z_0)= (1+z_0), so Theorem 17(b) applies. ∎ At η=δ=0.05η=δ=0.05 and p=0.1p=0.1 the agreement bet folds after 5959 shadow runs, while (O1) rules need at least 584584 (the grid-mixture IPW certificate needs 910910). Under the least-favorable null q=1/(1+z0)q=1/(1+z_0) with Z=−1Z=-1 on disagreement, the bet λ=1λ=1 folds with probability q59=0.048≤δq^59=0.048≤δ. The rule is powerless unless agreement is nearly certain (g>0.95g>0.95 here), because an unaudited disagreement is charged the worst case −1-1. The certificate below also bets on audited disagreements, with inverse-propensity weights. The agreement-aware certificate. Assume (E), (A1), (A2), and pt≥πt>0p_t≥ _t>0 on Gt=0\G_t=0\ (agreements need no audits). Let t=(λ,u):λ≥0,u≥−1,λ+u≤πt(1+u).C_t=\(λ,u):λ≥ 0,u≥-1,λ+u≤ _t(1+u)\. For ℱt−1F_t-1-measurable (λt,ut)∈t( _t,u_t) _t, the agreement-aware factor is Ft=1+λtz0,t,Gt=1,1+ut+At(λtZt−ut)/pt,Gt=0.F_t= cases1+ _tz_0,t,&G_t=1,\\ 1+u_t+A_t( _tZ_t-u_t)/p_t,&G_t=0. cases Equivalently, Ft=1+λtXtF_t=1+ _tX_t with Xt=z0,tX_t=z_0,t on agreement and Xt=vt+At(Zt−vt)/ptX_t=v_t+A_t(Z_t-v_t)/p_t on disagreement, where vt=ut/λtv_t=u_t/ _t (for λt>0 _t>0): an inverse-propensity estimator whose surrogate is exact on agreement and the constant vtv_t on disagreement. The conservative increment Xt0=AtZt+(1−At)[Gtz0,t−(1−Gt)]X^0_t=A_tZ_t+(1-A_t) [G_tz_0,t-(1-G_t) ] (audited: ZtZ_t; unaudited agreement: z0,tz_0,t; unaudited disagreement: −1-1, or any known lower bound on ZtZ_t) is bet on with λ∈[0,1]λ∈[0,1] and no weights. As before, Kn=∑kwk∏t≤nFt(k)K_n= _kw_k _t≤ nF^(k)_t with ∑kwk≤1 _kw_k≤ 1, and the certificate folds when Kn≥1/δK_n≥ 1/δ. For the rate, take the i.i.d. model: (Gt,Zt)(G_t,Z_t) are i.i.d. with law P satisfying (E) with a constant z0≥0z_0≥ 0, g=P(G=1)g=P(G=1), Z given G=0G=0 has a finitely supported law PdisP_ dis on [−1,M][-1,M], and audits are i.i.d. Bernoulli(p)(p), independent of everything else. A law Q of (G,Z)(G,Z) that satisfies (E) with the same z0z_0 is structural; write q=Q(G=1)q=Q(G=1) and R for the law of Z given G=0G=0. Define D∗(P;p)=infkl(g∥q)+p(1−g)KL(Pdis∥R),D^*(P;p)= \kl(g\|q)+p(1-g)KL(P_ dis\|R) \, over structural Q with qz0+(1−q)RZ≤0qz_0+(1-q)E_RZ≤ 0. The first term is the information in the agreement frequency, the second the information in the audited disagreements. Let Γ(λ,u)=PlnF (λ,u)=E_P F be the growth rate of the constant bet (λ,u)∈(λ,u) (with πt=p _t=p; for p=1p=1, where Γ does not depend on u, we also require u≤0u≤ 0, so that C is compact), and call (λ,u)(λ,u) strictly feasible if λ>0λ>0, u>−1u>-1 and λ+u<p(1+u)λ+u<p(1+u). Theorem 22 (Agreement-aware certificate). (i) Validity. Under (E), (A1), (A2), pt≥πt>0p_t≥ _t>0 on Gt=0\G_t=0\ and H0H_0, (Kn)(K_n) is a test supermartingale, so the certificate folds with probability at most δ. The conservative certificate is a test supermartingale under (E) and H0H_0 alone, for any audit mechanism, including outcome-dependent audits. (i) Information. In the i.i.d. model, D∗(P,p)=maxΓD^*(P;p)= _C ; D∗>0D^*>0 iff PZ>0E_PZ>0; some maximizer has u∗≤0u^*≤ 0; and D∗≥max D^*≥ \ pKLinf(PZ),KLinf(PX0), p\,KL_ (P_Z),\ KL_ (P_X^0), kl(g∥11+z0)g>11+z0. (g\,\|\, 11+z_0 )1_\g> 11+z_0\ \. (i) Converse. Consider a rule of type (O2) with pt(0)≤p_t(0)≤ p that folds at a stopping time τ of its records. Suppose ℙQ(τ<∞)≤δP_Q(τ<∞)≤δ for every i.i.d. structural Q with QZ≤0E_QZ≤ 0, and ℙP(τ<∞)=1P_P(τ<∞)=1. Then Pτ≥ln(1/δ)/D∗(P,p)E_Pτ≥ (1/δ)/D^*(P;p). (iv) Achievability. For any finite set of strictly feasible (λk,uk)( _k,u_k) with uk≤0u_k≤ 0 and weights wkw_k, the mixture has Pτ≤mink:Γk>0ln1δ+ln1wk+ln1+MpΓ(λk,uk).E_Pτ≤ _k: _k>0 1δ+ 1w_k+ 1+Mp ( _k,u_k). With grids refined as δ→0δ→ 0 and ln(1/wk)=o(ln1δ) (1/w_k)=o( 1δ), Pτ=(1+o(1))ln(1/δ)/D∗(P,p)E_Pτ=(1+o(1)) (1/δ)/D^*(P;p). The conservative certificate with the grid mixture of Theorem 17(a) (with p=1p=1) has Pτ≤(ln1δ+2lnln1δ+O(1))/KLinf(PX0)E_Pτ≤( 1δ+2 1δ+O(1))/KL_ (P_X^0). Proof. (i) Nonnegativity. Since πt≤1 _t≤ 1 and 1+u≥01+u≥ 0, λ≤πt(1+u)−u≤1λ≤ _t(1+u)-u≤ 1, so 1+λz0,t≥1−λ≥01+λ z_0,t≥ 1-λ≥ 0 on agreement. On disagreement without audit, Ft=1+ut≥0F_t=1+u_t≥ 0. With audit, Zt≥−1Z_t≥-1 gives Ft≥1+ut−(λt+ut)/ptF_t≥ 1+u_t-( _t+u_t)/p_t. This is ≥1+ut≥0≥ 1+u_t≥ 0 if λt+ut≤0 _t+u_t≤ 0; otherwise it increases in ptp_t and is ≥1+ut−(λt+ut)/πt≥0≥ 1+u_t-( _t+u_t)/ _t≥ 0 because (λt,ut)∈t( _t,u_t) _t. Mean. On Gt=1\G_t=1\, (E) gives Ft=1+λtZtF_t=1+ _tZ_t. On Gt=0\G_t=0\, (A2) and pt>0p_t>0 give [Ft∣ℋt,Zt]=1+ut+λtZt−ut=1+λtZtE[F_t _t,Z_t]=1+u_t+ _tZ_t-u_t=1+ _tZ_t. Since λt,ut _t,u_t are ℱt−1F_t-1-measurable and λt≥0 _t≥ 0, [Ft∣ℱt−1]=1+λt[Zt∣ℱt−1]≤1E[F_t _t-1]=1+ _tE[Z_t _t-1]≤ 1. The mixture and Ville’s inequality follow as in Theorem 15. For the conservative certificate, Xt0=ZtX^0_t=Z_t if At=1A_t=1, Xt0=ZtX^0_t=Z_t if At=0A_t=0 and Gt=1G_t=1 (by (E)), and Xt0=−1≤ZtX^0_t=-1≤ Z_t otherwise. So Xt0≤ZtX^0_t≤ Z_t pointwise, whatever the audits depend on, Xt0X^0_t is observable, and 1+λXt0≥1−λ≥01+λ X^0_t≥ 1-λ≥ 0. (i) Let O=(G,A,AZ)O=(G,A,AZ). Because A is independent Bernoulli(p)(p) under P and under every structural Q, and AZ=Az0AZ=Az_0 on G=1\G=1\, the chain rule gives KL(PO∥QO)=kl(g∥q)+p(1−g)KL(Pdis∥R)KL(P_O\|Q_O)=kl(g\|q)+p(1-g)KL(P_ dis\|R). So D∗D^* is the infimum of KL(PO∥QO)KL(P_O\|Q_O) over structural nulls. Weak duality. The computation in (i) with pt=p_t=p gives QF=1+λQZ≤1E_QF=1+ _QZ≤ 1 for every (λ,u)∈(λ,u) and structural null Q. As in Lemma 16, Jensen then gives Γ(λ,u)=PlnF≤KL(PO∥QO) (λ,u)=E_P F (P_O\|Q_O). Hence supΓ≤D∗ _C ≤ D^*. If PZ≤0E_PZ≤ 0, P is itself a structural null, so D∗=0=Γ(0,0)D^*=0= (0,0). Let PZ>0E_PZ>0. The set C is a compact polygon, and Γ is concave and upper semicontinuous with values in [−∞,∞)[-∞,∞), so it has a maximizer (λ∗,u∗)(λ^*,u^*). For λ≤pλ≤ p, (λ,0)∈(λ,0) and dλΓ(λ,0)|λ=0=PZ>0 ddλ (λ,0)|_λ=0=E_PZ>0, so Γ∗≔Γ(λ∗,u∗)>0 ^* (λ^*,u^*)>0 and λ∗>0λ^*>0. Case g=1g=1. Then Γ=ln(1+λz0) = (1+λ z_0), which is maximal at (1,−1)∈(1,-1) with value ln(1+z0) (1+z_0). The structural law with q=1/(1+z0)q=1/(1+z_0) and R=δ−1R= _-1 has mean 00 and KL(PO∥QO)=ln(1+z0)KL(P_O\|Q_O)= (1+z_0), so D∗=Γ∗D^*= ^* and u∗=−1u^*=-1. Case g<1g<1. Since Γ∗ ^* is finite and P has finite support, all factors are positive at (λ∗,u∗)(λ^*,u^*) on the support of POP_O, and Γ is differentiable nearby, with Fd(z)≔1+u+(λz−u)/pF_d(z) 1+u+(λ z-u)/p, ∂λΓ _λ =gz01+λz0+(1−g)PdisZFd, = gz_01+λ z_0+(1-g)\,E_P_ dis ZF_d, ∂uΓ _u =(1−g)(1−p)[11+u−Pdis1Fd]. =(1-g)(1-p) [ 11+u-E_P_ dis 1F_d ]. The constraint λ≥0λ≥ 0 is inactive. Write the others as c1=1+u≥0c_1=1+u≥ 0 and c2=p(1+u)−u−λ≥0c_2=p(1+u)-u-λ≥ 0. The KKT conditions for a concave function on a polytope give ν1,ν2≥0 _1, _2≥ 0 with νici=0 _ic_i=0, ∂λΓ=ν2 _λ = _2 and ∂uΓ=ν2(1−p)−ν1 _u = _2(1-p)- _1. If p<1p<1, Γ=−∞ =-∞ at u=−1u=-1, so ν1=0 _1=0. If p=1p=1, ∂uΓ≡0 _u ≡ 0; take the maximizer with u∗=0u^*=0, where c1>0c_1>0, so ν1=0 _1=0 and the constraint u≤0u≤ 0 gets multiplier 00. If ν2>0 _2>0, then c2=0c_2=0, so Fd(−1)=0F_d(-1)=0 and Pdis(−1)=0P_ dis(\-1\)=0. Define a measure QO∗Q^*_O on the values of O: mass g/(1+λ∗z0)g/(1+λ^*z_0) on agreement, (1−g)(1−p)/(1+u∗)(1-g)(1-p)/(1+u^*) on an unaudited disagreement, and (1−g)pPdis(z)/Fd(z)+pν2z=−1(1-g)pP_ dis(z)/F_d(z)+p _21_\z=-1\ on an audited disagreement with Z=zZ=z. On the support of POP_O, dPO/dQO∗=F∗dP_O/dQ^*_O=F^*, and the added atom lies outside it. Mass. Since F−1F-1 is linear in (λ,u)(λ,u), λ∂λΓ+u∂uΓ=P[1−1/F]=1−(QO∗(all)−pν2)λ _λ +u _u =E_P[1-1/F]=1-(Q^*_O(all)-p _2). By the KKT conditions the left side equals ν2(λ∗+u∗(1−p)) _2(λ^*+u^*(1-p)), and this is pν2p _2 because ν2>0 _2>0 forces λ∗=p(1+u∗)−u∗λ^*=p(1+u^*)-u^*. So QO∗Q^*_O has mass 11. Structure. For p<1p<1, the u-condition with ν1=0 _1=0 reads (1−g)/(1+u∗)−(1−g)Pdis[1/Fd]=ν2(1-g)/(1+u^*)-(1-g)E_P_ dis[1/F_d]= _2, that is, QO∗(unaudited disagreement)/(1−p)=QO∗(audited disagreement)/p≕1−q∗Q^*_O(unaudited disagreement)/(1-p)=Q^*_O(audited disagreement)/p 1-q^*. With the mass identity, the agreement mass is q∗q^*. So QO∗Q^*_O is the observation law (the audit coin is Bernoulli(p)(p) under both laws) of the structural Q∗Q^* with Q∗(G=1)=q∗Q^*(G=1)=q^* and R∗=QO∗(audited disagreement,⋅)/(p(1−q∗))R^*=Q^*_O(audited disagreement,·)/(p(1-q^*)). For p=1p=1 there are no unaudited disagreements, and 1−q∗1-q^* is the audited disagreement mass. Mean. Q∗Z=q∗z0+(1−q∗)R∗Z=∂λΓ−ν2=0E_Q^*Z=q^*z_0+(1-q^*)E_R^*Z= _λ - _2=0. So Q∗Q^* is a structural null and D∗≤KL(PO∥QO∗)=PlnF∗=Γ∗≤D∗D^* (P_O\|Q^*_O)=E_P F^*= ^*≤ D^*. Sign of u∗u^*. For p<1p<1, the structure identity gives 1+u∗=(1−g)/(1−q∗)1+u^*=(1-g)/(1-q^*), and q∗=g/(1+λ∗z0)≤gq^*=g/(1+λ^*z_0)≤ g because z0≥0z_0≥ 0. Hence u∗≤0u^*≤ 0. For p=1p=1, u does not enter Γ , and u∗=0u^*=0 is a maximizer. Lower bounds. Under a structural null Q, the pair (A,AZ)(A,AZ) is a function of O with law Bernoulli(p)⊗QZ(p) Q_Z and QZ≤0E_QZ≤ 0, so data processing gives KL(PO∥QO)≥pKL(PZ∥QZ)≥pKLinf(PZ)KL(P_O\|Q_O)≥ pKL(P_Z\|Q_Z)≥ pKL_ (P_Z). Likewise X0X^0 is a function of O with QX0≤QZ≤0E_QX^0 _QZ≤ 0, which gives KLinf(PX0)KL_ (P_X^0). Finally G is a function of O, and RZ≥−1E_RZ≥-1 forces q≤1/(1+z0)q≤ 1/(1+z_0), which gives the klkl term. Take the infimum over Q. (i) Let Q∗Q^* be the structural null from (i). The records after n runs are (U,(Gt,At,AtZt)t≤n)(U,(G_t,A_t,A_tZ_t)_t≤ n). Their density under an i.i.d. structural law is a product of Q(Gt)Q(G_t), of the audit factors pt(Gt)At(1−pt(Gt))1−Atp_t(G_t)^A_t(1-p_t(G_t))^1-A_t, and of the density of R at ZtZ_t when At=1A_t=1 and Gt=0G_t=0; on Gt=1\G_t=1\, AtZt=Atz0A_tZ_t=A_tz_0 carries no further information. The audit factors are the same under P and Q∗Q^*. The log-likelihood ratio is therefore ∑t≤nℓt _t≤ n _t with ℓt=lnP(Gt)Q∗(Gt)+At(1−Gt)lndPdisdR∗(Zt), _t= P(G_t)Q^*(G_t)+A_t(1-G_t) dP_ disdR^*(Z_t), which is bounded because the supports are finite and R∗>0R^*>0 on suppPdissuppP_ dis. Its conditional mean under P given the records before t is kl(g∥q∗)+(1−g)pt(0)KL(Pdis∥R∗)≤D∗kl(g\|q^*)+(1-g)p_t(0)KL(P_ dis\|R^*)≤ D^*. Optional stopping at τ∧nτ n and data processing on τ≤n\τ≤ n\ give D∗Pτ≥xnln(1/δ)−h(xn)D^*E_Pτ≥ x_n (1/δ)-h(x_n) with xn=ℙP(τ≤n)→1x_n=P_P(τ≤ n)→ 1, as in Theorem 17(b). (iv) For a strictly feasible (λk,uk)( _k,u_k) with uk≤0u_k≤ 0, the factors lie between a positive constant and (1+M)/p(1+M)/p, since 1+uk(1−1/p)+λkM/p≤(1+M)/p1+u_k(1-1/p)+ _kM/p≤(1+M)/p. The log-wealth SnS_n of this bet has i.i.d. bounded increments of mean Γk>0 _k>0. Since Kn≥wkeSnK_n≥ w_ke^S_n, τ is at most the first n with Sn≥ln(1/(δwk))S_n≥ (1/(δ w_k)), and Wald’s identity with overshoot at most ln((1+M)/p) ((1+M)/p) gives the bound. For the limit, take θ∈(0,1)θ∈(0,1) and a maximizer with u∗≤0u^*≤ 0. The point θ(λ∗,u∗)θ(λ^*,u^*) has λ=θλ∗>0λ=θλ^*>0, u=θu∗∈(−1,0]u=θ u^*∈(-1,0] and c2=(1−θ)p+θc2(λ∗,u∗)≥(1−θ)p>0c_2=(1-θ)p+θ c_2(λ^*,u^*)≥(1-θ)p>0, so it is strictly feasible, and Γ(θλ∗,θu∗)≥θΓ∗+(1−θ)Γ(0,0)=θΓ∗ (θλ^*,θ u^*)≥θ ^*+(1-θ) (0,0)=θ ^* by concavity. Γ is continuous on the strictly feasible set, so fine grids contain points with Γ≥θ2Γ∗ ≥θ^2 ^*; let θ→1θ→ 1. The conservative certificate is Theorem 17(a) applied to the i.i.d. Xt0∈[−1,M]X^0_t∈[-1,M] with p=1p=1. ∎ Implementation. A replay policy implements the certificate as follows. The agreement indicator is Gt=1G_t=1 iff both final answers are non-empty and identical under the scorer’s own comparison, and (E) must hold for the benchmark’s labels. On a shadow run with live cost ctc_t, put z0,t=(ct−1)/Bz_0,t=(c_t-1)/B (clipped to [−1,1][-1,1] as for ZtZ_t), and on an audited disagreement compute ZtZ_t as in the IPW certificate. The bets are λ=c(π(1−s)+s)λ=c(π(1-s)+s) and u=−su=-s, where π is the audit floor, c runs over the grid cj=(1+ε)−jc_j=(1+ )^-j of Theorem 17(a) with weights 1/(j(j+1))1/(j(j+1)) (truncated at cj≥1/200c_j≥ 1/200), and s runs over 0,0.01,0.03,0.1,0.3,0.6,0.9,0.97,0.99\0,0.01,0.03,0.1,0.3,0.6,0.9,0.97,0.99\ with weight 1/21/2 for s=0s=0 and 1/161/16 for each other value. Every grid point is strictly feasible, because λ+u−π(1+u)=−(1−c)(π(1−s)+s)<0λ+u-π(1+u)=-(1-c)(π(1-s)+s)<0. The factors are 1+λz0,t1+λ z_0,t on agreement (audited or not), 1−s1-s on an unaudited disagreement, and 1−s+(λZt+s)/pt1-s+(λ Z_t+s)/p_t on an audited disagreement. The wealth is updated on every shadow run, not only on audited ones. The slice s=0s=0 is the IPW certificate with the exact surrogate z0,tz_0,t on agreement. By concavity, gln(1+cpz0)≥pgln(1+cz0)g (1+cpz_0)≥ pg (1+cz_0), so the growth of its bet c is at least that of the IPW bet c. The bound of Theorem 17(a) therefore holds for the mixture with an extra ln2 2 in the numerator. Numerical checks. On 274 random structural laws (p∈0.05,0.1,0.3,0.7,1p∈\0.05,0.1,0.3,0.7,1\), maxΓ _C and a direct convex program over structural nulls agree to 3⋅10−123· 10^-12. The constructed Q∗Q^* has mass 11, mean 00 and the structural ratio to within 5⋅10−85· 10^-8, u∗≤10−8u^*≤ 10^-8, and D∗D^* never falls below the three lower bounds in (i). The median of D∗/(pKLinf(PZ))D^*/(pKL_ (P_Z)) is only 1.021.02: agreement adds little information unless it is frequent. In the paired model with p=0.1p=0.1 the ratio is 1.001.00 for live accuracy 0.380.38 and macro reliability 0.970.97 (as on KQA-Stream), 1.181.18 for a=r=0.9a=r=0.9 with correlated errors, and 5.25.2 when the answers agree with probability 0.990.99. In 200200 simulated runs per setting, the grid above certified in 1.071.07, 1.031.03 and 0.290.29 times the time of the IPW grid mixture at δ=10−2δ=10^-2, and in 1.021.02, 0.880.88 and 0.230.23 times at δ=10−6δ=10^-6. The conservative certificate has rate 00 in the first two settings and 0.0270.027 (against D∗=0.030D^*=0.030) in the third. Under five structural nulls, including the least-favorable one, the certificate folded in 2.42.4–3.3%3.3\% of 4,0004,000 runs at δ=0.05δ=0.05 (horizon 4,0004,000). C.2 Changes, carried-over evidence and the lifetime ledger Definition 1 counts every fold event of a bad candidate in an epoch, including candidates formed in earlier epochs. We separate what is proved for the idealized certificate from what holds for the two implemented ledgers: without restarts and with restarts (the restart-only certificate of Section 5). Idealized. Changes are announced, all certification evidence is discarded at each change, and one test per template (SPRT, or the grid mixture of Theorem 17) restarts in each epoch at level δ; if several candidates are tested within an epoch, they share δ, e.g. through a per-epoch ledger. Every fold in an epoch then rests on that epoch’s evidence alone, Ville’s inequality gives Definition 1 for every fold event, and Theorem 3(i) and Corollary 20 describe this certificate. Ledger with restarts (restart only). The replay notifies the policy of every edit that changes the gold answer of some instance routed to a template; epochs of a template are the intervals between such edits, within which the macro’s reliability is constant. At each notification, a candidate under test discards its wealth and its shadow statistics and restarts its test with the next ledger index, so the lifetime ledger still sums to δ. Folded macros are not reset; they are monitored by the LR-CUSUM. By Proposition 23(iv) this certificate satisfies Definition 1 for these epochs if its own null is constant within them. The null Z≤0EZ≤ 0 also depends on live reasoning, whose accuracy and cost can change without any answer change; across such a silent change only Proposition 23(i) applies (Section D.5.1 measures how often this happens and evaluates a variant that also restarts on live-trace changes). It needs to know which templates an edit touches; a coarser change log that also flags edits without effect on a template’s answers triggers extra restarts, which cost coverage but not safety, because every restart consumes a fresh index. Ledger without restarts. Here the policy is not told when an edit occurs. A candidate keeps its wealth (or its SPRT statistic) across edits; it is neither reset nor discarded. The j-th candidate of a template, counted over the whole stream, is tested at δj=6δ/(π2j2) _j=6δ/(π^2j^2) with seven fixed bets. For a candidate j and its shadow evaluations t=1,2,…t=1,2,…, let ζj _j be the first t with [Zt∣ℱt−1]>0E[Z_t _t-1]>0 (ζj=∞ _j=∞ if there is none). Call a fold of candidate j at evaluation t<ζjt< _j a fold on null evidence: the candidate has been at or below break-even at every evaluation so far. Proposition 23 (What the lifetime ledger guarantees). Let each candidate’s wealth be a mixture of test supermartingales, as in Theorem 15, started at 11 at its formation and compared with 1/δj1/ _j. (i) Lifetime. With probability at least 1−δ1-δ, over the whole stream no candidate of the template folds on null evidence. The expected number of such folds per template is at most δ. (i) Fresh candidates. Given the history at the start of an epoch, the candidates formed from then on fold on null evidence with total probability at most ∑j>Jδj≤δ _j>J _j≤δ, where J is the number of earlier candidates. (i) Carried-over candidates. Let candidate j be unfolded at a stopping time c with wealth KcK_c, and let e≥ce≥ c be a stopping time such that the null holds at its evaluations in [c,e)[c,e). Then it folds in [c,e)[c,e) with conditional probability at most min1,δjKc \1, _jK_c\ given ℱcF_c. Equality holds (with e=∞e=∞) if, after c, its wealth is a martingale that tends to 00 unless it reaches 1/δj1/ _j, and it reaches 1/δj1/ _j without overshoot. (iv) Per epoch. If the library holds at most one candidate per template, the probability that an epoch contains a fold of a candidate that is bad in that epoch is at most ∑j>Jδj+δJKc≤δ+δJ(Kc−1)+ _j>J _j+ _JK_c≤δ+ _J(K_c-1)^+, where candidate J is carried over with wealth KcK_c at the epoch start (δJKc≔0 _JK_c 0 if there is none, in particular if J=0J=0). The ledger satisfies Definition 1 if, at each change, carried-over candidates are discarded or restart their wealth at 11 with the next ledger index (a restarted test then counts as a fresh candidate in (i)); otherwise it does not in general. Proof. (i) Fix j and condition on the history at the formation of candidate j. Since [Zt∣ℱt−1]E[Z_t _t-1] is ℱt−1F_t-1-measurable, ζj>t∈ℱt−1\ _j>t\ _t-1. Let K~t K_t be the wealth with factors replaced by 11 from evaluation ζj _j on. Then [K~t∣ℱt−1]=K~t−1(ζj>t[Ft∣ℱt−1]+ζj≤t)≤K~t−1E[ K_t _t-1]= K_t-1(1_\ _j>t\E[F_t _t-1]+1_\ _j≤ t\)≤ K_t-1, because [Ft∣ℱt−1]≤1E[F_t _t-1]≤ 1 whenever the null holds at t (Theorem 15). A fold on null evidence implies suptK~t≥1/δj _t K_t≥ 1/ _j, which has conditional probability at most δj _j by Ville’s inequality. Sum over j; ∑jδj=δ _j _j=δ. Each candidate folds at most once, which gives the expectation bound. (i) The same argument, conditioned on the history at the epoch start, for j>Jj>J. (i) Stop the factors at e and at the first non-null evaluation, as in (i). Given ℱcF_c, the stopped wealth (Kt∧e)t≥c(K_t e)_t≥ c is a nonnegative supermartingale started at KcK_c; apply Ville’s inequality to Kt∧e/KcK_t e/K_c. In the equality case, the martingale stopped at 1/δj1/ _j is bounded by 1/δj1/ _j, so bounded convergence gives Kc=[K∞∣ℱc]=δj−1ℙ(fold after c∣ℱc)K_c=E[K_∞ _c]= _j^-1P(fold after c _c). (iv) A fold in the epoch of a candidate that is bad in the epoch is either a fold on null evidence of a candidate formed in the epoch, covered by (i), or a fold of the carried-over candidate, covered by (i) with c the epoch start and e its end. Previously folded macros cause no fold event. Add the bounds and use ∑j≥Jδj≤δ _j≥ J _j≤δ. ∎ In the paired model the utility null Z≤0EZ≤ 0 is the reliability condition r≤r⋆r≤ r , so “bad” in Definition 1 (with r0=r⋆r_0=r ) and “null” coincide, up to the online estimates of L and m that the certificate uses; since r⋆=a−(m−1)/Lr =a-(m-1)/L moves with live reasoning, both can change inside an epoch of answer changes. Numerical check of (i). We simulated a candidate with the deployed seven-bet mixture (p=0.1p=0.1, j=1j=1, δ=0.05δ=0.05) that sees a good utility law for 6060 shadow runs and then a null law with mean exactly 00. Of the candidates still unfolded at the change (mean wealth 4.44.4), 11.1%11.1\% folded after it, against the bound min(1,δ1Kc)=13.4%E (1, _1K_c)=13.4\% and δ=5%δ=5\%. After 120120 good runs the numbers are 22.8%22.8\% and 28.1%28.1\%. So carried-over evidence can defeat per-epoch safety, as (iv) allows. Restarts without change information. The restarts above use the edit log. The next result shows when per-epoch safety is out of reach without change information; the one after it gives the guarantee of restarts triggered by the memory’s own change detector, which needs none. Under (A1)–(A3) of Appendix B, audits of template i form a Poisson process of rate λi≔pπi _i p _i, independent of the changes. Let αi≔λiλi+ν _i _i _i+ν be the probability that an epoch of template i contains at least one audit. By memorylessness this holds given any history at the epoch start, and the number of audits in an epoch is geometric, ℙ(K≥k)=αikP(K≥ k)= _i^k. In the notation of Lemma 10, αi=ρi _i= _i. Proposition 24 (Per-epoch safety without change information). Let the audited outcomes of an epoch be i.i.d. from the epoch’s law, whichever candidate they evaluate, and let a policy observe neither change times nor any signal of them, and fold only at audited outcomes of template i. (i) Upper bound. For every epoch and every history before it, the probability that the epoch contains a fold of template i is at most αi _i. So if αi≤δ _i≤δ, Definition 1 holds for template i whatever the policy. (i) Lower bound. Suppose that, with positive probability, the policy reaches a history h after which its next audited outcome of template i, whenever it arrives, folds a candidate if it lies in a set F. Then for every bad law Q, given that an epoch starts after h and before that outcome, the probability that this epoch contains a fold of a bad candidate is at least αiQ(F) _iQ(F), and at least Q(F)Q(F) given in addition that the epoch contains an audit. The policy is therefore not δ-safe if αiQ(F)>δ _iQ(F)>δ. Consequently, without change information, among certificates whose fold decisions for template i are deterministic functions of its own audited outcomes, (a) every reliability certificate that folds with positive probability violates Definition 1 at every level δ<αir0δ< _ir_0; (b) every paired-utility certificate with Z taking finitely many values in [−1,M][-1,M] that folds with positive probability violates it at every level δ<αi/(1+M)δ< _i/(1+M). Proof. (i) A fold of template i in the epoch requires an audit of template i in the epoch. Given the history at the epoch start s, the audits after s form a Poisson process of rate λi _i and the epoch length is exponential with rate ν, independent of each other and of the history ((A1)–(A3)), so the first audit after s precedes the epoch end with probability λi/(λi+ν)=αi _i/( _i+ν)= _i. (i) The policy’s action at the next audited outcome is the same function of h and of that outcome whether or not a change occurred, because it observes neither. Changes are the points of a Poisson process independent of the history, so an epoch starts between h and the next audited outcome with positive probability, and Definition 1 must hold for the histories at the start of such epochs; these histories consist of h and the elapsed time. Given such a history, the next audited outcome is the first audit of the epoch, and it falls in the epoch with probability αi _i, by the argument of (i). Under Q every candidate is bad in the epoch, and this outcome lies in F with probability Q(F)Q(F), independently of its arrival time and of the history; on the intersection of the two events a bad candidate folds in the epoch. For (a) and (b), outcomes take finitely many values, so some history h just before the policy’s first fold has positive probability; take F the set of outcomes on which the policy folds after h, which is not empty and, by determinism, does not depend on when the next audit arrives. (a) If 1∈F1∈ F, take Q=Bernoulli(r0)Q=Bernoulli(r_0), so Q(F)≥r0Q(F)≥ r_0; otherwise 0∈F0∈ F and Q=Bernoulli(0)Q=Bernoulli(0) gives Q(F)=1Q(F)=1. (b) Take z∈Fz∈ F. If z≤0z≤ 0, the point mass at z is bad and Q(F)=1Q(F)=1. If z>0z>0, the law with mass 1/(1+z)1/(1+z) at z and z/(1+z)z/(1+z) at −1-1 has mean 00, so it is bad, and Q(F)≥1/(1+z)≥1/(1+M)Q(F)≥ 1/(1+z)≥ 1/(1+M). ∎ So, for a reliability certificate without change information, per-epoch safety is impossible on every template that is likely enough to be audited in an epoch, αir0>δ _ir_0>δ, i.e. with more than δ/(r0−δ)δ/(r_0-δ) audits per epoch on average, and automatic on every template with αi≤δ _i≤δ; in between it depends on the rule. With M=1M=1 (our clipped utilities) or r0=12r_0= 12, and δ=0.05δ=0.05, the impossibility covers every template with more than one audit per nine epochs on average. The templates it leaves out are barely coverable even with change information: by Lemma 10, the restarted SPRT of Theorem 3(i), which needs at least nmin=⌈ln(1/δ)/ln(r1/r0)⌉≥1n_ = (1/δ)/ (r_1/r_0) ≥ 1 audits in an epoch to fold, answers in one step a share at most ρiN≤αinmin≤(δ/r0)nminE _i^N≤ _i^n_ ≤(δ/r_0)^n_ of the queries of a template with αir0≤δ _ir_0≤δ. What remains attainable without change information is safety for every fold that rests on evidence collected after the change (Proposition 25). Exact value for a stationary rule. The lower bound of (i) is attained by a rule whose only chance to fold is the next audited outcome. A rule that, from the epoch start on, folds at the first audited outcome in F, whenever it comes, has an exact bad-fold probability: outcomes are i.i.d. within the epoch and independent of the audit times, so the audits whose outcome lies in F form a Poisson process of rate λiQ(F) _iQ(F), and the first of them precedes the epoch end with probability βi(F)=λiQ(F)λiQ(F)+ν∈[αiQ(F),αi]. _i(F)= _iQ(F) _iQ(F)+ν∈ [ _iQ(F),\, _i ]. For fold-at-first-success under the bad law Bernoulli(r0)Bernoulli(r_0) this is λir0/(λir0+ν) _ir_0/( _ir_0+ν), so this rule is δ-safe if and only if λi/ν≤δ/(r0(1−δ)) _i/ν≤δ/(r_0(1-δ)), i.e. 0.06580.0658 at r0=0.8r_0=0.8 and δ=0.05δ=0.05, slightly below the threshold δ/(r0−δ)=0.0667δ/(r_0-δ)=0.0667 beyond which (a) excludes every rule. The two bounds of the proposition bracket the exact value, and (a) cannot be sharpened to βi _i for every rule, because a rule may stop folding after an audited failure. Numerical check. At λi=0.01 _i=0.01, ν=1ν=1, δ=0.05δ=0.05 and r0=0.8r_0=0.8, αi=0.0099<δ _i=0.0099<δ, so every audit-driven rule is δ-safe there; the next-audit rule folds a bad candidate with probability αir0=0.0079 _ir_0=0.0079, and fold-at-first-success with λir0/(λir0+ν)=0.0079 _ir_0/( _ir_0+ν)=0.0079 (they differ in the fifth decimal). Monte Carlo over 2×1062× 10^6 epochs per setting (Poisson audits and changes, bad law Bernoulli(r0)Bernoulli(r_0)) reproduces the exact value of fold-at-first-success at 4949 settings with r0∈0.5,0.8,0.95,1r_0∈\0.5,0.8,0.95,1\ and λi/ν _i/ν from 10−310^-3 to 100100: 4747 of the 4949 95% intervals contain it, the largest relative deviation is 2.1%2.1\%, and an event-driven simulator (2×1052× 10^5 epochs) agrees on all 1010 rows it was run on. Every exact value lies between αir0 _ir_0 and αi _i, with equality to αi _i at r0=1r_0=1, and the δ-safety boundary falls where predicted: at r0=0.8r_0=0.8 the probability is 0.02340.0234 at λi/ν=0.03 _i/ν=0.03 and 0.05000.0500 at 0.06580.0658. For the next-audit rule, Monte Carlo reproduces αir0 _ir_0 to within 3⋅10−43· 10^-4 at five settings with αi _i from 0.00990.0099 to 0.990.99. For an SPRT without restarts (r1=0.95r_1=0.95, r0=0.5r_0=0.5, δ=0.05δ=0.05), bad epochs that start one success from the threshold contain a bad fold with frequency αir0 _ir_0 up to Monte Carlo error (more for αi≥0.9 _i≥ 0.9, through later audits), which exceeds δ from αi=0.2 _i=0.2 on; in a stream of epochs the frequency stays below αi _i at every tested αi∈[0.02,0.98] _i∈[0.02,0.98] (checks/prop24_check.py in our code release). Proposition 25 (Detector-triggered restarts). Let the library hold at most one candidate per template, and let each candidate restart its test (wealth 11, the next ledger index) at stopping times of the system’s own records, for instance at the alarms of an LR-CUSUM on its audited shadow outcomes, or be discarded then. Call the interval from a candidate’s last restart (or its formation) to its decision its test segment. Then: (i) statement (i) of Proposition 23 holds; (i) for every epoch and every history before it, the probability that the epoch contains a fold of a candidate that is bad in the epoch and whose test segment began in the epoch is at most ∑j>Jδj≤δ _j>J _j≤δ, where J is the number of ledger indices used before the epoch; (i) the segment running at the epoch start, with index J and wealth KcK_c, folds a candidate that is bad in the epoch before its next restart with probability at most min1,δJKc \1, _JK_c\. Proof. Restart times are stopping times and every restart starts a fresh mixture of test supermartingales at 11 with a fresh index, so each test segment is a candidate in the sense of Proposition 23. Apply (i) of that proposition to all segments, (i) to the segments that begin in the epoch, and (i) to the segment running at its start, with e the earlier of its next restart and the epoch end. ∎ A restart costs coverage, never safety, because it consumes a fresh index. The carried-over risk of (i) lasts until the detector’s next alarm, so the detector bounds its duration by its detection delay when the change raises the candidate’s failure rate. A change that leaves the macro’s outcomes unchanged but improves live reasoning is invisible to a detector on macro outcomes; it can make a candidate useless without making it wrong. Section D.3.4 measures both effects on KQA-Stream. In a simulation of discard-on-alarm with the lifetime ledger (Bernoulli likelihood-ratio wealth, r1=0.95r_1=0.95, bad law at r0=0.5r_0=0.5, δ=0.05δ=0.05, αi∈0.5,0.9,0.97 _i∈\0.5,0.9,0.97\), fresh segments fold in at most 0.00030.0003 of bad epochs (mean bound ∑j>Jδj=0.001 _j>J _j=0.001–0.0090.009), and carried-over segments in 0.0090.009–0.0140.014 (mean bound min1,δJKc=0.015E \1, _JK_c\=0.015–0.0520.052), as (i) and (i) require. What the restart-only certificate attains. For the j-th test of a template (restarts included) the threshold is ln(1/δj)=ln(1/δ)+2lnj+ln(π2/6) (1/ _j)= (1/δ)+2 j+ (π^2/6). With i.i.d. audited utilities of law P and the seven bets ℬB with weights 1/71/7, the best bet in ℬB and Wald’s identity (factors in [0.1,1.9][0.1,1.9] because the code clips Z to [−1,1][-1,1]) give an expected label count of at most ln1δ+2lnj+lnπ26+ln7+ln1.9gℬ(P) 1δ+2 j+ π^26+ 7+ 1.9g_B(P) before the fold threshold is reached (ignoring the futility test), where gℬ(P)=maxb∈ℬPln(1+bZ)g_B(P)= _b E_P (1+bZ) is the growth of the best bet, and gℬ≤KLinf(P)g_B _ (P), with equality only if c∗∈ℬc^* . The restart-only certificate therefore does not attain the limit of Theorem 3 or Corollary 20 as δ→0δ→ 0. It is a finite-candidate, finite-grid version of the idealized certificate, whose extra cost is the additive 2lnj+ln(π2/6)+ln7+ln1.92 j+ (π^2/6)+ 7+ 1.9 and the factor KLinf/gℬKL_ /g_B (1.031.03–1.071.07 on the three paired laws we checked). Appendix D Experimental details Compute and software. All LLM runs used one NVIDIA RTX 4080 SUPER GPU (16 GB). Models were served with vLLM 0.30 (Kwon et al., 2023) (FlashAttention backend) and PyTorch 2.13 under Ubuntu 22.04. The 8B models were served with FP8 weights: Qwen3-8B-FP8 natively (Yang et al., 2025), and Granite-3.3-8B-Instruct (IBM Granite Team, 2025), Mistral-7B-Instruct-v0.3 (Jiang et al., 2023) and OLMo-3-7B-Instruct (Team OLMo, 2025) with on-the-fly FP8 quantization. Qwen3-4B ran in BF16. The natural-edit timeline used 4.754.75 GPU-hours and the code domain 17.217.2 GPU-hours of wall time. Mistral has no system role, so its system message is prepended to the first user turn. Granite-3.3’s chat template embeds the current date, so its prompts change with the calendar day; all Granite prompts were rendered with the date pinned to 2026-09-25, and prompt hashes confirm that this reproduces the main-run prompts exactly. Qwen3 “thinking” was disabled. Main-table decoding used T=0.7T=0.7, top-p 0.80.8, top-k 2020, with a deterministic per-request seed. Every LLM request carries a content hash. Results are appended to checkpoint shards and skipped on restart, so all runs are resumable. Tables and figures are generated by script from the raw logs. Mechanism computations. Run lengths of integer-score rules were computed by solving the Markov chain of the reflected walk. The gap of SPRT certification uses Lemma 10 with the exact sample-size distribution of the one-sided SPRT, obtained by dynamic programming on the lattice of log-likelihood-ratio values. Sums over templates run to 10710^7 with rigorous tail brackets. The collapse grid crossed (r0,r1)∈(0.85,0.95),(0.6,0.9),(0.3,0.7)(r_0,r_1)∈\(0.85,0.95),(0.6,0.9),(0.3,0.7)\ plus random pairs, verifier flip rates 0,0.05,0.1,0.2\0,0.05,0.1,0.2\, δ∈[10−32,0.03]δ∈[10^-32,0.03], audit rates p∈[0.03,1]p∈[0.03,1] (tabulated settings: p=1p=1, Θ∈10−5,…,10−1 ∈\10^-5,…,10^-1\), 10 drift rates for the random settings and s∈1.2,1.5,2,3s∈\1.2,1.5,2,3\. Plotted against x=ν/px=ν/p alone, the same points spread over two orders of magnitude at fixed x because of the ln(1/δ)/KL (1/δ)/KL factor. The literal-stream check simulated Poisson arrivals, audits with p∈0.2,0.3,0.5,1p∈\0.2,0.3,0.5,1\, Poisson changes and verifier flips, for 500500–10410^4 templates, and matched the exact formula within 0.51%0.51\%. Natural template streams. KQA Pro templates are the programs of the train and validation sets (106,173 questions) with entity names, strings, numbers and dates abstracted. Function, relation, attribute and concept names are kept, giving 45,544 templates. MuSiQue templates abstract each single-hop sub-question of the answerable training set (19,938 questions; 10,686 templates). We abstract the relation phrase after “≫ ” when it is present, and otherwise the question with bridge entities replaced by placeholders. Exponents were estimated from rank-frequency data (Sibuya-type estimator on counts ≤10≤10), from the Clauset–Shalizi–Newman tail fit, and from Heaps’ law. The Sibuya and Heaps estimators give s∈[1.15,1.26]s∈[1.15,1.26] for the fine-grained templates. The CSN tail fit gives s^≈0.85 s≈ 0.85–0.880.88 with a poor goodness of fit on MuSiQue (p<0.01p<0.01), so we do not use it. Coverage curves average 100 random stream orders. KQA-Stream construction. We index the KQA Pro knowledge base: 16,960 entities, 794 concepts, 363 relations (726 relation–direction types) and 415,702 directed edges. Our path executor agrees with the official KoPL engine on 600 sampled one- and two-hop programs (2,187 steps). A template is an anchor concept plus a relation path of 2–4 hops. It is kept if at least 60 anchors have 1–3 answers of a consistent concept, the path does not immediately backtrack, and no single answer covers more than half of the anchors. From 2,250 qualifying distinct paths we selected 300 templates (120/120/60 with 2/3/4 hops), covering 17 anchor concepts, 51 answer concepts and 114 relations, and sampled 60 anchors each (18,000 instances). A random pattern verbalizes each instance question; 296 of the 300 templates have three LLM-written patterns with a single entity slot (the rest have fewer valid patterns). An edit event replaces the object of a relation for a fraction f∈0.25,0.6f∈\0.25,0.6\ of the relevant subjects with a random entity of the same concept, keeping inverse edges consistent. Each of three sequences has 24 events. An event changes the gold answer of 223 (f=0.25f=0.25) or 386 (f=0.6f=0.6) instances on average and touches 3.9%3.9\% of templates, which become 32%32\% or 59%59\% stale respectively. Templates that share an edited relation go stale together. The per-template change rate grows with derivation length (events that hit a 2/3/4-hop template: 2.7/4.2/5.5%2.7/4.2/5.5\%; Spearman 0.340.34), which makes staleness itself a multi-step-reasoning phenomenon. LLM paths. The live agent answers hop by hop. At each step its prompt contains the question, the linked anchor, the path so far, the current frontier (up to 5 entity names) and a numbered list of the (relation, direction) pairs available at the frontier (up to 60, sorted by frequency). It outputs JSON that either takes one hop or answers, with at most 5 hops, and may filter the final frontier by one of its concepts. The answer is the resulting entity set, and it is correct if it equals the gold set of the current knowledge-base version. For Qwen3-8B a live query uses 3.243.24 calls with 1,4861,486 prompt and 6262 completion tokens on average; on the unedited knowledge base its live accuracy is 0.6250.625, 0.3050.305 and 0.1900.190 at 2, 3 and 4 hops (0.4100.410 over all 18,000 instances). Routing, the one LLM step of a macro, retrieves the top-5 templates by BM25 over their verbalized patterns and asks the LLM for a template id or NONE. It runs against the full, fixed template catalog, so its outcome does not depend on the policy. Its accuracy is 0.9390.939 for Qwen3-8B, and the correct template is among the top 5 for all but 40 of the 18,000 instances. The direct baselines use one call: a parametric answer without the knowledge base, or whole-path generation from the question and the BM25 top-60 relation directions, executed on the current version. For each call we log prompt and completion tokens and latency. After an edit, live traces are re-run only for instances whose earlier trace or gold path touches an edited edge; all other traces are reused unchanged, which is exact because the agent’s inputs are unchanged. A brute-force re-run of all instances at versions 0,8,16,240,8,16,24 of every sequence matched in all 216,000216,000 comparisons. Baselines and statistics. The ExpeL-style counter starts at importance 22, moves by ±1± 1 per audited success or failure and retires the macro at 00. The ACE-style counter keeps helpful and harmful tallies from audited outcomes and retires the macro when it would be flagged (harmful ≥ helpful, harmful >0>0). Fold once folds a verified candidate at formation and never retires it. A semantic cache keyed on question embeddings reduces to this policy behind our router. vCache’s per-request bound assumes i.i.d. prompts and has no retirement; Section D.2.4 evaluates a re-implementation, which avoids staleness mainly by rarely answering from cache. No policy was tuned on the evaluation streams. The validation-gated Hebbian rule uses its published defaults (η=0.1η=0.1, λ=10−3λ=10^-3, τ=0.85τ=0.85, c0=0.5c_0=0.5, fusion bonus 0.050.05). Our certificates use δ=β=0.05δ=β=0.05 and a likelihood-ratio CUSUM with threshold h=7h=7 and post-change failure rate max(1−r^⋆,q^0+0.15) (1- r , q_0+0.15); its in-control rate q^0 q_0 is estimated from the candidate’s shadow labels. The betting certificate mixes seven fixed bets λ∈0.05,0.1,0.2,0.3,0.5,0.7,0.9λ∈\0.05,0.1,0.2,0.3,0.5,0.7,0.9\ with equal weights. Any fixed mixture is valid; the grid mixture of Appendix C is needed only for the rate result. All defaults were fixed on synthetic tables before any real-LLM run. Risk ledger. The library holds at most one candidate or macro per template. The lifetime ledger tests the j-th test of a template, counted over the whole stream, at δj=6δ/(π2j2) _j=6δ/(π^2j^2). The restart-only certificate adds restarts: the replay notifies it of every edit that changes the gold answer of an instance routed to a template, and a candidate under test then discards its wealth and shadow statistics and restarts with the next ledger index (Section C.2). By Proposition 23(iv) this satisfies Definition 1 when the certificate’s null changes only at notices (Section D.5.1). The recommended deployment (“ours” in the main text) also drops the notified template’s folded macro or candidate and re-forms it under the ledger (invalidate + LR, Section D.4.2). Without restarts, the engine never tells a policy that an edit has occurred, and a candidate keeps its betting wealth (or SPRT statistic) across edits. The ledger then still bounds by δ the probability that a template ever folds a candidate that was at or below break-even at all its evaluations, and, given the history at the start of any epoch, the probability that a candidate formed in that epoch folds while bad. A candidate carried across an edit with wealth w is covered only by the conditional bound δjw _jw, so this variant does not satisfy Definition 1 in general. The per-candidate variants test each new candidate at a fresh level δ=0.05δ=0.05. This gives per-candidate conditional validity only, because a template’s total fold risk grows with its number of candidates. Compiled plans were run with the restart-only certificate (restarts), with the ledger without restarts, and per candidate. The SpF columns count every fold event of a candidate whose reliability at the fold’s knowledge-base version is below the break-even r⋆r of its template at that version (L=20m¯L=20 m), including candidates carried across edits (Section D.3.3). Same information and tuning. Every policy sees the same queries, edits, audit coins and verifier labels; all certificates share δ, the bet grid and h, and no threshold was tuned on the evaluation streams. Only our certificates receive edit notices: restart only uses them to restart candidates under test, ours also to invalidate, and Table 2 reports ours with every notice missed; Section D.4.2 gives the notices to baselines that invalidate on them, and Sections D.4.1 and D.4.3 report the answer-only safety replays and the decoding-variance subset. On Qwen3-8B the ledger without restarts lowers accuracy by 0.350.35 and 0.410.41 points relative to per-candidate testing and raises J20J_20 by about 1%1\% (betting: 0.646→0.6430.646→ 0.643, J20J_20 25.6→25.925.6→ 25.9; compiled + betting: 0.677→0.6730.677→ 0.673, J20J_20 23.4→23.723.4→ 23.7). On the other four models the change in accuracy ranges from −0.59-0.59 to +0.14+0.14 points. Restarts cost another 0.10.1–0.40.4 points on every model (Tables 6 and 13). Paired differences use a 95% bootstrap clustered by template (the 300 templates are resampled) over the 10 seeds × 3 edit sequences. Each instance–version is decoded once per model, so these intervals do not include LLM sampling variance. Replay setting. The replays fix the following setting. Audits reveal the gold answer of the current knowledge-base version at no LLM cost. Templates are given by the benchmark, and each has up to three verbalization patterns; routing errors enter every metric as measured losses (a misrouted macro answer counts as wrong) and are not modeled by the theory. A macro executes a verified relation path exactly, which is the main source of its accuracy gain over live reasoning; text notes and distilled models would need a costlier, noisier verifier. Each instance–version is decoded once per model, so intervals resample templates, streams and edit sequences, not decoding randomness (measured separately in Section D.4.3), and calls and latencies are reconstructed from the recorded requests. The main edits are synthetic but structured: correlated across templates that share a relation, and more frequent for longer derivations; Section D.4.4 adds a natural timeline and Section D.6 a code domain with an exact verifier. Edit notices list the templates whose gold answers change; changes of live reasoning are not announced (Section D.5.1). Two comparisons are deliberately narrow. Compiled plans re-execute a stored path on the current facts; when such re-execution is cheap it can dominate caching the result, and Table 3 reports this baseline. The vCache-style cache stores unverified live answers, so on this benchmark it inherits live accuracy (Section D.2.4); a verified exact-repeat cache is evaluated in Section D.4.2. The fundamental limit concerns candidates informed by the outcomes of their own template; transfer across correlated templates, or a macro that a change does not affect, lies outside its model. Completed coverage and validation. The final analysis includes all five models, three edit sequences and ten seeds, with 57,750 distinct configuration–seed checkpoints after deduplicating identical effective settings. Every checkpoint and its trace-derived summary passed completion and version checks. Recomputing the 80 primary model–policy cells reproduces the exported mean accuracy, calls and J20J_20. Paired differences use per-candidate betting + LR-CUSUM as the reference, except the price of restarts (Table 13), which is paired against the same ledger without restarts. All policies and all models. Table 5 adds to Table 3 one-call path generation, the validation-gated Hebbian rule with time decay, and compiled plans certified by the plug-in SPRT. Hebbian decay almost never folds, because its time-to-live starves all but the most popular templates (Appendix A). Table 6 gives accuracy, calls and J20J_20 for all five models. Policy (Qwen3-8B) Acc Calls Stale SpF J20J_20 direct (path generation) .186 1.00 0 0 54.1 Hebbian + decay .444 3.80 0.4 0.3 40.1 compiled plan + SPRT .650 2.51 10 0.3 25.3 loss-tuned .631 1.62 659 118 25.7 SPRT + symmetric .604 2.84 20 0.7 28.7 betting, no restarts .643 2.55 72 0.1 25.9 betting, per candidate .646 2.50 76 0.2 25.6 plan, ledger, no restarts .673 2.42 14 0.0 23.7 plan, per candidate .677 2.38 12 0.03 23.4 invalidate on notice (no LR) .651 2.42 0 0.0 25.2 Table 5: Further Qwen3-8B rows for Table 3 (same setting and columns; SpF counts the bad folds of the main text). Loss-tuned: fold and retire thresholds set from a renewal-reward approximation of the loss. Per candidate: a fresh level δ for each candidate. The last row drops and re-forms every macro an edit announces, with perfect notices (Section D.4.2). Bold: ours. live ours (recommended) macros + betting restart only / no restarts / per-cand. compiled plan + betting restarts / no restarts / per-cand. Model Acc Calls J20J_20 Acc Calls J20J_20 Acc Calls Stale J20J_20 Acc Calls Stale J20J_20 Qwen3-8B .379 3.26 43.8 .648 2.56 25.5 .642/.643/.646 2.57 66/72/76 26.0 .668/.673/.677 2.46 16/14/12 24.1 Qwen3-4B .226 4.48 74.0 .556 3.10 43.0 .551/.554/.553 3.08 82/85/91 43.5 .591/.596/.596 2.90 10/12/12 39.7 Granite-3.3-8B .152 4.47 80.3 .439 3.57 53.8 .438/.441/.447 3.52 79/82/84 53.8 .469/.472/.477 3.39 13/15/16 50.9 Mistral-7B .097 4.96 94.6 .241 5.00 80.5 .245/.247/.250 4.94 38/42/43 80.0 .257/.259/.258 4.88 9/9/10 78.7 OLMo-3-7B .076 2.39 46.4 .280 2.68 36.7 .276/.280/.283 2.68 51/51/51 36.9 .294/.297/.299 2.64 11/12/12 36.0 Table 6: All five models in the main setting (30 streams of 200k queries each). Stale: stale wrong answers per 10410^4 queries. Ours: invalidate on notice + LR-CUSUM, perfect notices (it serves no stale answers). Calls and J20J_20 of the other groups are for the first variant listed: the restart-only certificate with restarts (restart only), for macros and for compiled plans (compiled plan, Section D.4.2). Entries are descriptive means; calls are reconstructed from recorded requests. Bold: ours. Where the accuracy comes from. On Qwen3-8B (Table 3), reuse without certification (fold once) adds 15.515.5 points over live reasoning for macros and 26.126.1 for compiled plans. Certification with retirement and restarts adds 10.710.7 and 2.92.9 more and cuts stale answers 20×20× and 6×6×; over the counters, which reuse and retire too, it adds 2.32.3–2.92.9. Reuse helps this much because a macro executes a verified relation path symbolically, whereas the live agent re-derives it with a noisy LLM at 38%38\% accuracy. For text or distilled macros this advantage need not hold, so we regard the accuracy gain as specific to this benchmark and the staleness result as the transferable one. On OLMo-3-7B the certificate uses more calls than live reasoning (2.39→2.682.39→ 2.68; Table 6). Stale counts also depend on how much a policy folds: SPRT + symmetric serves 2020 stale answers per 10410^4 queries against restart only’s 6666 on Qwen3-8B, but answers 43%43\% of queries in one step against 51%51\%. This is why the main text compares retirement rules on a matched population (Section D.3.2). D.1 Natural template streams We extracted derivation templates from 106k KQA Pro programs (Cao et al., 2022) and 19.9k MuSiQue decompositions (Trivedi et al., 2022), with entities and values abstracted (details above). Both are heavy-tailed: the Sibuya-type estimator gives s^=1.26 s=1.26 and Heaps’ law s^=1.15 s=1.15 on both corpora. The static branch of Theorem 14 therefore predicts that coverage improves only as T−(0.13–0.21)T^-(0.13--0.21), as observed (Figure 4): slopes at n0=1n_0=1 are −0.16-0.16 (KQA Pro) and −0.18-0.18 (MuSiQue), steepening to −0.19-0.19 and −0.21-0.21 late in the stream. The Good–Turing estimator of the rare mass (Good, 1953; Orlitsky et al., 2016), M^(T)=∑k≤n0kKk(T)/T M(T)= _k≤ n_0kK_k(T)/T with KkK_k the number of templates seen k times, predicts the gap online with a median relative error of 0.40.4–3.4%3.4\% (2.12.1–3.4%3.4\% at n0=1n_0=1), without knowing s. Raising the certification sample size n0n_0 slows contraction by Cs,n0/Cs,1≈n01−1/s/Γ(2−1s)C_s,n_0/C_s,1≈ n_0^1-1/s/ (2- 1s). These streams test the demand side of the theory without an LLM in the loop. Figure 4: MuSiQue templates: measured coverage gap (markers; mean of 100 stream orders), fitted Zipf law (dashed) and online Good–Turing forecast (solid) for certification sample sizes n0n_0. D.2 Learning rates, counters, noisy verifiers and a semantic cache These experiments replay the recorded KQA-Stream outcome tables of the five models; no LLM is called. They use the main setting (200k queries, Zipf s=1.2s=1.2, Poisson edits at rate 28/T28/T so that nearly all 24 edit events of a sequence occur, uniform audits p=0.1p=0.1, L=20L=20) with three edit sequences and ten seeds, i.e. 30 runs per cell. Intervals are 95% bootstraps that resample the 300 templates (1000 resamples), pooled over sequences and seeds, and paired differences use the same template bootstrap as the main text. Here ours is betting + LR-CUSUM at the per-candidate level; it reproduces the per-candidate row of Table 3 exactly (Qwen3-8B: accuracy 0.6460.646, J20J_20 25.5825.58, 7676 stale answers per 10410^4). Definitions. A false retirement (FR) is the retirement of a macro whose true reliability is still ≥r⋆≥ r ; we count them per 10410^4 folded uses of such healthy macros. The detection delay of a stale episode is the number of folded uses from its onset, the version at which a macro folded while healthy drops below r⋆r , to its retirement. Episodes still open at the end of a stream contribute their uses so far, so delays are lower bounds; Section D.3.1 gives censoring-aware estimates. At a matched FR, each rule’s delay and stale answers are interpolated linearly in log FR along its threshold grid, at the FR∗ of our LR-CUSUM at its default h=7h=7. Matching uses the true reliabilities. This makes the comparison of detectors fair, but it is not a deployable tuning procedure. D.2.1 Learning-rate sweep Rules. A fixed-ratio rule keeps our betting certificate and retires with the clipped score +1+1 per audited failure and −1/k-1/k per audited success, retiring at S>hS>h, for k∈0.1,0.25,0.5,0.75,1,2,3,5,7,10,15,25k∈\0.1,0.25,0.5,0.75,1,2,3,5,7,10,15,25\ and 1515 thresholds h∈[0.5,80]h∈[0.5,80]. Ours is the same except that the score is the log-likelihood ratio of the macro’s own estimated q^0 q_0 against max(1−r^⋆,q^0+0.15) (1- r , q_0+0.15). The failure rates q0q_0 and q1q_1 of Table 7 are the true failure rates (1−1-reliability, including routing errors) of the stale episodes of our policy, q0q_0 at the fold version and q1q_1 at the onset version. The prediction 1/ρ(q¯0,q¯1)1/ρ( q_0, q_1) uses their means. empirical optimum delay at matched FR (uses) stale per 10410^4 Model episodes q¯0 q_0 q¯1 q_1 q1>12q_1> 12 predicted 1/ρ1/ρ [95% CI] k P(argmin)P( ) k=1k=1 k=7k=7 best k ours best k ours Qwen3-8B 81 .068 .746 .86 1.84 [1.36, 2.47] 25 .24 853 482 468 81 185 76 Qwen3-4B 65 .044 .746 .88 2.14 [1.52, 3.32] 0.75 .34 330 398 317 54 231 91 Granite-3.3-8B 32 .161 .863 .97 0.93 [0.57, 1.50] 0.25 .40 169 317 126 63 173 84 Mistral-7B 17 .237 .896 .94 0.67 [0.39, 1.11] 0.1 .33 141 252 106 84 78 43 OLMo-3-7B 17 .135 .821 .94 1.14 [0.59, 2.83] 0.25 .23 131 227 115 57 88 51 Table 7: Learning-rate sweep on KQA-Stream. Episodes: stale episodes over 30 runs. q1>12q_1> 12: share of stale episodes with q1>0.5q_1>0.5. P(argmin)P( ): bootstrap frequency of the empirical optimum. Best k: the fixed ratio with the lowest delay at matched FR over all tested k, and with the fewest stale answers over k≥1k≥ 1. Ours: per-macro LR-CUSUM (per-candidate certificate). Delays are naive means over censored episodes. Bold: ours. Results. Staleness on these streams is nearly total, not partial: 8686–97%97\% of stale episodes have q1>0.5q_1>0.5 (Table 7). The predicted weights 1/ρ1/ρ are therefore 0.670.67–2.142.14, not 77. On four of five models the delay at matched FR is smallest below k=1k=1 (k=0.1k=0.1–0.750.75), consistent with the ordering of the prediction, and it grows with k above 11; k=7k=7 is 1.31.3–2.5×2.5× slower than the best tested ratio on these models (Figure 5). Below 11 the curves are flat, so the optimum there is not sharply identified. Qwen3-8B is the exception: its curve is flat for k≥5k≥ 5 and worse at k≤2k≤ 2, and the bootstrap spreads its optimum over k=25k=25, 55 and 1010. The larger effect is not the tuning of k. The per-macro LR-CUSUM detects 1.31.3–5.9×5.9× faster than each model’s best tested fixed ratio (paired intervals exclude 11 on Qwen3-8B, Qwen3-4B and OLMo-3-7B; Section D.3.6) and serves 1.71.7–2.5×2.5× fewer stale answers than the fixed ratio k≥1k≥ 1 with the fewest. Every fixed ratio at k=7k=7 is significantly slower than ours, and at k=1k=1 this holds on three of five models (paired intervals exclude 00). Healthy failure rates are heterogeneous: their medians (00–0.210.21) lie far below their means. We interpret this, without a direct test, as the reason why one global ratio cannot be likelihood-ratio tuned for all macros at once, whereas the per-macro rule adapts q^0 q_0 to each macro. The first-order delay curve follows the rising trend on Granite, Mistral and OLMo but not on Qwen3-8B. At k=1k=1 it underestimates the measured delays by 2.32.3–17×17×, and at larger k it leaves the logarithmic regime on three models. With the episode-median in place of the mean (q0,q1)(q_0,q_1) the predicted weights are 0.510.51–3.013.01. With only 1717–8181 stale episodes per model the intervals are wide, and only the large differences are resolved. Figure 5: Per-memory likelihood ratios beat every tested global counter ratio. Mean uses to detect staleness for a clipped counter that weighs a failure k times a success (orange; circle: best tested k) and for the per-memory LR-CUSUM (blue dotted), at the false-retirement rate of the latter (1.91.9–3.43.4 per 10410^4 healthy uses). Purple: the weight 1/ρ1/ρ that Corollary 2 predicts from the mean failure rates (q¯0,q¯1)( q_0, q_1); dashed: its first-order delay, an optimum-level rate that the measured delays exceed (by 2.32.3–17×17× at k=1k=1; see the results below). On four of five models the delay is flat or lowest up to the predicted weight and grows beyond it; the per-memory gain is significant on Qwen3-8B, Qwen3-4B and OLMo-3-7B. 30 streams per model, per-candidate certificate; bands: 95% template bootstrap; naive means, which favour the slower rules (Section D.2.1). D.2.2 Counters in the main setting ExpeL- and ACE-style counters serve 294294–967967 stale answers per 10410^4 queries (77–13×13× ours), fold 111111–595595 spurious macros per run (ours: at most 1.71.7), and have 1010–3535 false retirements per 10410^4 (ours: 1.81.8–3.43.4) with naive mean delays of 307307–478478 uses (ours: 5454–8484; lower bounds, see Section D.3.1). Their J20J_20, however, is not significantly different from ours on any model, and their accuracy is significantly lower only on Qwen3-8B (Table 8). The reason is coverage, as the price of safety of Section 4 suggests: the counters skip certification and answer 2828–76%76\% of queries in one step, against 2121–53%53\% for the restart-only certificate and 2020–53%53\% for ours (recommended), so they make fewer LLM calls (e.g. 1.751.75 and 1.861.86 against 2.572.57 on Qwen3-8B, Table 3), and in J20J_20 this saving roughly offsets their stale and spurious answers. ExpeL-style counter ACE-style counter Model Δ ΔJ20 J_20 Δ ΔJ20 J_20 Qwen3-8B −.033-.033 [−.061-.061, −.004-.004] +1.38+1.38 [−0.56-0.56, +3.20+3.20] −.028-.028 [−.055-.055, −.001-.001] +1.15+1.15 [−0.72-0.72, +2.95+2.95] Qwen3-4B +.006+.006 [−.020-.020, +.035+.035] −1.18-1.18 [−3.93-3.93, +1.34+1.34] +.002+.002 [−.026-.026, +.032+.032] −0.68-0.68 [−3.61-3.61, +1.99+1.99] Granite-3.3-8B −.014-.014 [−.034-.034, +.009+.009] +0.87+0.87 [−1.27-1.27, +2.79+2.79] −.014-.014 [−.035-.035, +.009+.009] +1.05+1.05 [−1.14-1.14, +3.01+3.01] Mistral-7B −.008-.008 [−.023-.023, +.006+.006] +0.30+0.30 [−1.17-1.17, +1.79+1.79] −.008-.008 [−.022-.022, +.006+.006] +0.49+0.49 [−0.94-0.94, +1.93+1.93] OLMo-3-7B +.007+.007 [−.019-.019, +.037+.037] −0.47-0.47 [−2.13-2.13, +0.90+0.90] −.000-.000 [−.028-.028, +.032+.032] −0.09-0.09 [−1.82-1.82, +1.36+1.36] Table 8: Main setting: paired differences of the counters against ours (betting + LR-CUSUM, per-candidate level), with 95% template-bootstrap intervals; positive ΔJ20 J_20 is worse. Only the counters’ accuracy on Qwen3-8B differs significantly. D.2.3 A noise-aware certificate Setup. Noisy verifiers are the next step named in the Limitations; this experiment measures where the current certificate stands. Each audited judgement (of the live answer, the macro and any other evaluated answer) is flipped independently with probability ε . The noisy live verdict also decides which traces count as verified when candidates are formed. The noise-aware variant bets on the paired utility of the observed channel, X=(1−2ε)(m−1)+L(Y~M−Y~L)X=(1-2 )(m-1)+L( Y^M- Y^L), and maps the CUSUM’s alternative into the observed channel, q1=max(ε+(1−2ε)(1−r⋆),q0+0.15(1−2ε))q_1= ( +(1-2 )(1-r ),\,q_0+0.15(1-2 )), with live accuracy corrected as a=(aobs−ε)/(1−2ε)a=(a_ obs- )/(1-2 ). It uses either the true ε or an estimate from a simulated clean calibration subset of 200200 items, drawn per sequence and seed (mean estimates 0.0540.054, 0.1030.103 and 0.2010.201). It corrects the certificate and the CUSUM, not the formation of candidates. The labels-to-fold slowdown is the mean over templates of the number of audited shadow labels a candidate needs before it folds, relative to the uncorrected certificate at ε=0 =0. ε certificate Acc Stale SpF slowdown (1−2ε)−2(1-2 )^-2 0 uncorrected .436 69 0.71 1.00 1.00 .05 uncorrected .370 63 0.21 1.88 1.23 known .370 61 0.21 2.00 estimated .369 61 0.21 1.99 .1 uncorrected .326 61 0.17 2.56 1.56 known .328 59 0.12 2.82 estimated .327 59 0.14 2.70 .2 uncorrected .279 53 0.27 3.67 2.78 known .286 50 0.21 3.36 estimated .286 49 0.22 3.21 Table 9: Noisy verifier, per-candidate betting, means over the five models (30 streams each; at ε=0 =0 the mean of the per-candidate column of Table 6). Stale: per 10410^4 queries. SpF: spurious folds per run. Slowdown: audited labels a candidate needs before it folds, relative to ε=0 =0. Bold: ours (all rows). Results (negative). Certification slows faster than (1−2ε)−2(1-2 )^-2 (Table 9): the labels-to-fold slowdown is 1.881.88, 2.562.56 and 3.673.67 at ε=0.05 =0.05, 0.10.1 and 0.20.2, against 1.231.23, 1.561.56 and 2.782.78 predicted, and the slowdown in key-queries is larger still (2.152.15, 3.203.20, 5.055.05). The prediction is for a single-label test; our certificate bets on the difference of two noisy labels. The noise-aware variant, with known or estimated ε , is indistinguishable from the uncorrected certificate on accuracy, stale answers and spurious folds, and nearly every per-model accuracy interval contains 00. Noise does not inflate spurious folds (0.710.71 per run clean, 0.120.12–0.270.27 noisy), because far fewer candidates fold at all. What dominates is the loss of accuracy for every variant (0.4360.436 to 0.2790.279 at ε=0.2 =0.2), driven by slower certification and by macros formed from falsely verified live traces, neither of which the correction addresses. Counter rules, which fold nearly everything, keep their accuracy under noise (Qwen3-8B, 15 streams: .592→.575.592→.575 and .597→.580.597→.580 at ε=0.2 =0.2) but fold more spurious macros (ExpeL 271→385271→ 385, ACE 377→601377→ 601 per stream). A certificate built for the noisy paired channel is left to future work. D.2.4 A vCache-style semantic cache Re-implementation. We re-implemented vCache from Algorithms 1–2 of its paper (Schroeder et al., 2026); the following details are our design choices. A query is looked up against the nearest cached question by cosine similarity (ties go to the newest entry). Each entry fits a sigmoid of similarity by ridge-penalized logistic maximum likelihood (λ=10−3λ=10^-3), with a delta-method upper confidence bound on its threshold. The exploration probability is the minimum of the paper’s Eq. 11 over a grid of 49 values of its error parameter, and 11 while an entry has no observations. An exploration answers live, evaluates the cached answer in shadow at no cost, and adds the query to the cache only if the cached answer was judged wrong; an exploitation returns the cached answer at no LLM call, and embedding compute is not charged. Labels are vCache’s own (agree: cached answer equals the live answer) or gold correctness on audited queries (audit). There is no retirement. The plug-in variant uses the point estimate without the confidence band, which is optimistic. The vCache paper used GTE-large, E5-large-v2 and OpenAI embeddings. None was available offline, so we use all-MiniLM-L6-v2 (384 dimensions), with TF-IDF as a sensitivity check; the median nearest-neighbour similarity over the 17,927 unique questions is 0.810.81 (MiniLM) and 0.660.66 (TF-IDF). A faithful replication would need a 1024-dimensional embedding and the authors’ code to confirm the confidence band. A cache hit on a different question counts as stale if the cached answer would have been correct for that query when the entry was formed. vCache, δc=0.02 _c=0.02 plug-in, δc=0.02 _c=0.02 Model live Acc hits Acc ΔJ20 J_20 hits Acc Stale Qwen3-8B .379 .124 .377 +17.9+17.9 .847 .309 536 Qwen3-4B .226 .119 .225 +30.3+30.3 .848 .187 241 Granite-3.3-8B .152 .110 .151 +26.9+26.9 .841 .126 203 Mistral-7B .097 .111 .097 +14.7+14.7 .847 .082 115 OLMo-3-7B .076 .119 .076 +9.5+9.5 .835 .063 100 Table 10: vCache-style cache (MiniLM, agree labels). Hits: share of queries served from cache. ΔJ20 J_20: against ours (per-candidate betting); all intervals exclude 00. Stale: per 10410^4 queries. Our accuracy is .646.646, .553.553, .447.447, .250.250 and .283.283. Results. With its confidence band, vCache serves only 66–26%26\% of queries from cache for δc=0.01 _c=0.01–0.050.05, and 3%3\% with audited labels. Its accuracy equals live accuracy, and its J20J_20 is 9.29.2–30.730.7 worse than ours on every model (Table 10). TF-IDF embeddings give similar results. The reason is that this cache stores earlier live answers without verifying them, so here it inherits live accuracy (0.080.08–0.380.38), whereas a certified macro re-executes a verified derivation (our accuracy 0.250.25–0.650.65). A cache that stores only verified answers was tested for exact repeats (with notice invalidation; Section D.4.2): it serves almost no stale answers but stays at live accuracy, because it answers only 13%13\% of queries in one step; a verified semantic cache remains untested. Removing the confidence band raises the hit rate to about 85%85\%, but accuracy falls below live and stale answers rise to 100100–536536 per 10410^4. vCache’s δc _c bounds disagreement with the LLM, not error against gold: 6262–92%92\% of its hits are wrong against gold, mirroring live accuracy. An exact-question answer cache beats vCache on J20J_20 but is still 7.77.7–27.027.0 worse than ours. Cache hits are charged no LLM call, which favours the caches. D.3 Censoring, per-epoch safety and per-memory weights These replays reuse the recorded outcome tables of the five models in the main setting (30 streams per model and policy); no LLM is called. Intervals are 95% template-clustered bootstraps (1000 resamples); pooled rows resample templates jointly across models. D.3.1 Censoring-aware detection delays A stale episode starts when a macro that was above break-even at its fold drops below r⋆r after an edit; its onset is the first query of that version. Retirement is the event, and the end of the 200k-query stream censors the episode. The delay is the number of folded uses from onset to retirement, as in Section D.2. We estimate the survival function by Kaplan–Meier. The restricted mean delay within a window w is min(T,w)E (T,w), and “unretired at stream end” is the censored share. This closed-loop analysis covers our LR-CUSUM retirement with the per-candidate certificate and with the SPRT; each rule has its own population of stale episodes. Section D.3.2 compares the rules on one matched population, including the restart-only certificate. Rule (pooled) episodes cens. unretired at end RM delay, 10410^4 median LR (per-cand. betting) 212 13 6.1 [2.5, 10.8]% 76 [55, 107] 45 [40, 55] LR (SPRT) 222 14 6.3 [2.4, 10.6]% 103 [64, 176] 50 [44, 64] ExpeL-style 1,954 1,241 63.5 [57.5, 69.8]% 3,014 [2,204, 3,895] 375 [216, 682] ACE-style 1,865 999 53.6 [47.6, 59.9]% 2,569 [1,882, 3,332] 211 [122, 373] Table 11: Censoring-aware detection, pooled over five models (150 streams per rule). RM delay: restricted mean delay within 10410^4 folded uses; median: Kaplan–Meier median (folded uses). Bold: ours. On the audited-use clock the restricted means within 10310^3 audited uses are 7.67.6 (betting), 10.410.4 (SPRT), 299.9299.9 (ExpeL-style) and 257.6257.6 (ACE-style), with medians of 55, 55, 3737 and 2121 audited uses. The censored episodes of the LR-CUSUM are onsets near the end of a stream (at most 192192 folded uses of follow-up), whereas the counters’ censored episodes include some with 39,46739,467 uses. Per model the counters leave 2828–69%69\% of their own stale episodes unretired at stream end (Table 2 gives the matched population). The naive mean delay of Figure 6 counts an open episode with its uses so far. It is therefore a lower bound: 363363 and 366366 uses for the counters, 77–8×8× below their restricted means. Figure 6: Detection frontier on real streams (five models, 30 streams each; SPRT certification): naive mean delay after an edit against false retirements of healthy macros, as frontiers over each rule’s thresholds. The naive mean counts an episode that is still open at the end of the stream with its uses so far, so it is a lower bound; the counters’ censoring-aware delays are 77–8×8× longer (Table 11). Legend: share of each counter’s stale episodes not retired by stream end. D.3.2 Detection on a matched population The comparison above uses different populations: the counters fold every candidate, so their 1,9541,954 and 1,8651,865 stale episodes include many macros that a certificate would never fold, whereas the LR-CUSUM rows contain 212212. Here all rules see the same macros and the same audits. Every macro that a certified policy folded (the restart-only certificate, or per-candidate betting) is replayed open loop from its fold to the end of the stream, on its own audited label stream (the same audit coins; labels of its build instance withheld), under each retirement rule started at the fold: its own LR-CUSUM (reproducing the closed-loop retirement of all 7,3857,385 and 9,4489,448 folded macros exactly), the ExpeL-style rule (importance 22 at the fold) and the ACE-style rule. Counters therefore bank no credit from the shadow phase, which favours them. Stale onset is defined without retirement; spurious folds are excluded. Population. The restart-only certificate makes 7,3857,385 folds over the 150150 streams; the 4343 spurious ones (below r⋆r at the fold version) are not stale episodes. A matched stale episode is a fold above r⋆r whose macro is below r⋆r at a later version before the stream ends: there are 221221. Some rule retired 6565 of them before onset (LR-CUSUM 4747, ExpeL 2727, ACE 4141; 2424 by the LR-CUSUM alone and 1111 by ACE alone), and an episode retired before it began cannot be compared, so the common population is the remaining 156156 (4949 templates, 2525 edited relations, 3434 edit events, 33 sequences). Code: paper_figs/acl3_matched.py. unretired at stream end RM delay within 10410^4 uses Model eps. ExpeL ACE LR ExpeL ACE LR Qwen3-8B 66 53% 53% 3% 4,501 4,507 92 Qwen3-4B 53 57% 51% 0% 4,391 4,139 46 Granite-3.3-8B 18 22% 22% 0% 3,278 3,251 44 Mistral-7B 8 25% 25% 0% 2,110 2,095 61 OLMo-3-7B 11 64% 55% 9% 5,071 4,507 942 pooled 156 50% 47% 2% 4,069 3,949 67 per-cand., pooled 177 52% 46% 5% 4,115 3,773 72 Table 12: Matched-population detection: stale episodes of the macros folded by the restart-only certificate (last row: by per-candidate betting), replayed open loop under each retirement rule with identical audits; 30 streams per model. The OLMo LR-CUSUM value comes from one censored episode with long follow-up. Bold: ours. Results. Pooled, the counters leave 50.0%50.0\% [38.238.2, 62.862.8] and 47.4%47.4\% [36.336.3, 60.260.2] of the episodes unretired at stream end, against 1.9%1.9\% [0.00.0, 4.74.7] for the LR-CUSUM (Table 12, Figure 2). Their restricted mean delays are 4,0694,069 [2,9242,924, 5,4485,448] and 3,9493,949 [2,8302,830, 5,2965,296] uses against 6767 [4646, 9999], ratios 60.860.8 [44.744.7, 87.187.1] and 59.159.1 [43.743.7, 83.783.7] (paired template bootstrap, 2,0002,000 resamples; the ratio is computed in each resample); KM medians are 1,7501,750, 1,3031,303 and 4343 uses, and on the audited clock the restricted means within 10310^3 audits are 403403, 391391 and 6.66.6. Each episode costs 1,3591,359 and 1,3461,346 stale wrong answers under the counters, against 38.538.5. The ratio interval excludes 11 for Qwen3-8B (49×49×), Qwen3-4B (95×95×, 90×90×) and Granite (75×75×, 74×74×); Mistral (88 episodes) and OLMo (1111) are not resolved. The matched population thus confirms the closed-loop picture of Table 11, with a larger gap in delay. Robustness of the 60×60×. Episodes of one template, relation or edit event are correlated, so we also resample these clusters. The ExpeL/LR ratio is 60.860.8 with 95% intervals [44.744.7, 87.187.1] clustered by template, [44.544.5, 91.091.0] by edited relation and [41.141.1, 90.090.0] by edit event (ACE/LR: 59.159.1; [43.743.7, 83.783.7], [43.143.1, 89.389.3], [40.140.1, 86.686.6]). With only three edit sequences, sequence clustering is coarse ([31.231.2, 93.493.4]); the ratio is 9393, 3131 and 89×89× on sequences 101, 202 and 303 alone and 4646–92×92× when one sequence is left out. Without the exclusion, i.e. on all 221221 matched episodes with pre-onset retirements counted as zero delay, the ratio is 63.563.5 [46.946.9, 88.488.4] (58.958.9 for ACE), with 40.7%40.7\%, 37.6%37.6\% and 3.2%3.2\% unretired. D.3.3 Per-epoch safety A template epoch is a run, a routed template and an interval between edits that change the gold answer of some instance routed to that template. A false fold is a fold of a candidate whose true reliability (over all instances routed to the template, at stream popularity) is below r⋆r at the fold version; this is the SpF event of Table 2. An epoch is at risk if a candidate that is below break-even at some version of the epoch is under test in it, and the per-epoch false-fold frequency is the share of at-risk epochs with at least one false fold. A false fold is carried over if the candidate formed in an earlier epoch. This subsection uses the standard scoring; Section D.4.1 repeats the analysis with answer-only scoring, which the main text reports (Figure 3). Figure 7: Standard scoring (routing errors count as wrong answers): share of at-risk epochs with a task-level bad fold (reliability below r⋆r ) for five certificates; the dotted line y=δy=δ is a reference line, not a guarantee for these curves. Proposition 23(iv) bounds by δ only folds that violate a certificate’s own null (paired utility ≤0≤ 0; Figure 3), and per epoch only for the restarted certificate (blue), when notices cover every change of that null; the per-candidate variants are valid per candidate, the ledger without restarts over a template’s lifetime, and the no-notice variant for folds on post-change evidence (Proposition 25). Five models, 150 streams per policy and δ, 95% template-bootstrap intervals. Results. At δ=0.05δ=0.05 the per-epoch false-fold frequency is 0.0070.007 [0.0050.005, 0.0100.010] for the SPRT and 0.0060.006 [0.0030.003, 0.0100.010] for betting, both at a fresh level per candidate, 0.0050.005 [0.0030.003, 0.0070.007] for betting with the lifetime ledger and 0.0020.002 [0.0010.001, 0.0040.004] for the restart-only certificate with restarts; all stay below δ for every δ≤0.5δ≤ 0.5 (Figure 7). Carried-over folds are 4040 of 106106 false folds for per-candidate betting (1616 by candidates that were good at formation), 3232 of 7070 with the ledger (1111), 6262 of 8787 for the SPRT (5050), and 66 of 4343 with restarts (00). Table 13 gives the price of restarts per model. Model Δ ΔJ20 J_20 stale SpF folds restarts Qwen3-8B −.001-.001 [−.002-.002, .000.000] +0.08+0.08 [−0.02-0.02, 0.180.18] 72→ 66 0.10→ 0.00 60.0→ 53.8 149 Qwen3-4B −.003-.003 [−.007-.007, −.001-.001] +0.30+0.30 [0.070.07, 0.620.62] 85→ 82 0.20→ 0.07 72.9→ 66.5 110 Granite-3.3-8B −.003-.003 [−.006-.006, −.002-.002] +0.33+0.33 [0.170.17, 0.570.57] 82→ 79 0.67→ 0.47 69.4→ 62.3 100 Mistral-7B −.002-.002 [−.004-.004, .000.000] +0.27+0.27 [0.050.05, 0.480.48] 42→ 38 0.97→ 0.60 36.1→ 28.1 138 OLMo-3-7B −.004-.004 [−.006-.006, −.002-.002] +0.20+0.20 [0.120.12, 0.320.32] 51→ 51 0.40→ 0.30 41.1→ 35.5 79 Table 13: The price of restarts: the restart-only certificate against the same lifetime ledger without restarts (paired differences with 95% template-bootstrap intervals; stale answers per 10410^4 queries, SpF, folds and restarts per stream). The scoring convention behind most false folds. KQA-Stream scores a macro answer as correct only if the query was also routed to the right template, and a template’s true reliability averages over every query routed to it, including misrouted ones. The certificate instead tests the paired utility of its own shadow evaluations. The two notions of “bad” disagree for some candidates: at δ=0.05δ=0.05, 9191 of the 106106 false folds of per-candidate betting and all 4343 of the restart-only certificate have a positive exact paired-utility drift, so by the test’s own criterion these candidates were useful. Among candidates that are bad under the test’s own null at every version after formation, none folds (00 of 8,4348,434 for per-candidate betting and 00 of 8,0848,084 for the restart-only certificate). Scoring macro answers by the answer alone removes this convention. With answer-only scoring, every remaining false fold of per-candidate betting (1515) is carried over by a candidate that was good at formation, which is exactly the event that restarts remove. D.3.4 Restarts without edit notices The restart-only certificate restarts a candidate when the replay reports an answer-changing edit of its template. The no-notice certificate (discard on alarm; “no notice” in Tables 1 and 3) receives no edit information. It runs, on every candidate under test, the retirement detector of its folded macros: a Bernoulli LR-CUSUM on the candidate’s audited shadow outcomes with q^0 q_0 from the current test segment (estimated before each label), q1=max(1−r^⋆,q^0+0.15)q_1= (1- r , q_0+0.15) and h=7h=7. On an alarm it discards the candidate, and the template’s next candidate forms from a later verified live trace. A retired macro is removed, so its successor starts a fresh test. Proposition 25 covers this rule; a fold is spanning if its test segment began before the edit that opened the fold’s epoch, the event that (i) of that proposition bounds. Results. Discard-on-alarm has bad folds in 0.00080.0008 [0.00030.0003, 0.00140.0014] of at-risk epochs at δ=0.05δ=0.05 (99 of 11,37511,375), against 0.00250.0025 [0.00110.0011, 0.00430.0043] for the restart-only certificate (2929 of 11,67011,670), and raises accuracy by 1.31.3 points [0.80.8, 2.02.0] and lowers J20J_20 by 1.061.06 [0.630.63, 1.601.60] relative to it (Table 14). All 99 of its bad folds are spanning folds of candidates that were good at formation, and all 99 are bad under the test’s own null, so they are exactly the residual risk of Proposition 25(i); no fold rests on post-change evidence of a bad candidate (00 of 13,93513,935 candidates that are bad under the test’s own null at every version after formation fold). The restart-only certificate’s 2929 epochs come from the scoring convention above (all with a positive test drift). Spanning folds are frequent (1,1741,174 of 7,7667,766 folds), but the edit left all but 99 of these candidates good. Retirement is unchanged: 195195 stale episodes with 5.1%5.1\% censored and a mean delay of 68.068.0 uses, against 174174, 4.0%4.0\% and 68.068.0. The accuracy gain is positive on every model, because an alarm replaces a candidate whose cached answers broke by one formed on the current facts, while a restarted candidate keeps its stale answers. At every tested level δ∈0.01,0.05,0.2,0.5δ∈\0.01,0.05,0.2,0.5\, discard-on-alarm has fewer bad-fold epochs than the restart-only certificate (0.00030.0003–0.00400.0040 against 0.00180.0018–0.00540.0054; Figure 7) and 1.11.1–1.31.3 more accuracy points. Model Δ ΔJ20 J_20 bad-fold epochs alarms stale Qwen3-8B +.009+.009 [.004.004, .015.015] −0.62-0.62 [−1.09-1.09, −0.26-0.26] .0009.0009 / .0000.0000 68.3 70← 66 Qwen3-4B +.016+.016 [.003.003, .039.039] −1.62-1.62 [−3.86-3.86, −0.26-0.26] .0023.0023 / .0011.0011 61.0 83← 82 Granite-3.3-8B +.009+.009 [.004.004, .017.017] −0.86-0.86 [−1.60-1.60, −0.33-0.33] .0006.0006 / .0064.0064 91.8 81← 79 Mistral-7B +.012+.012 [.006.006, .022.022] −1.32-1.32 [−2.27-2.27, −0.62-0.62] .0003.0003 / .0026.0026 224.2 42← 38 OLMo-3-7B +.017+.017 [.004.004, .042.042] −0.89-0.89 [−2.22-2.22, −0.18-0.18] .0000.0000 / .0053.0053 99.7 49← 51 Table 14: Discard-on-alarm (no edit notices) against the restart-only certificate (edit-notified restarts), δ=0.05δ=0.05: paired differences with 95% template-bootstrap intervals; share of at-risk epochs with a bad fold (no notice / restart only); detector alarms per stream; stale answers per 10410^4 queries (no notice ← restart only). D.3.5 Partial staleness on real streams Half of the edits of each sequence change 25%25\% of the affected subjects (f=0.25f=0.25), and some of them leave a macro partially stale. We test the phase diagram on these episodes open loop: every macro that per-candidate betting folds is re-scored on its full audited label stream from its fold to the end of the stream (9,4489,448 macros over the five models). Ground truth is the macro’s exact failure rate qv=1−Rvq_v=1-R_v at each knowledge-base version. An episode starts at the first version after the fold at which q rises by at least 0.10.1 over its value q0q_0 at the fold, has failure rate q1q_1 there, and ends at the next version at which q moves by more than 0.020.02, or at the stream end. We keep episodes with q0∈[0.02,0.15]q_0∈[0.02,0.15], since with q0=0q_0=0 every rule can alarm at the first failure without false alarms, and call them partial if q1∈[0.15,0.5]q_1∈[0.15,0.5] and near-total if q1>0.5q_1>0.5. Each rule restarts at the change (Lorden’s worst case for a CUSUM); its delay is the number of folded uses from the change to its alarm, censored at the episode end, summarized by the Kaplan–Meier mean restricted to 1,0001,000 uses. Thresholds are matched to 11, 33 and 1010 false alarms per 10410^4 healthy uses, measured on the uses between fold and change of all folded macros with q0q_0 in the same band. Intervals are 95% template-clustered bootstraps with thresholds re-matched in every resample. Results. Of the 2,0762,076 single-change episodes, 217217 are partial episodes in the band (6363 templates; 3232–5353 per model) with mean q0=0.067q_0=0.067 and q1=0.331q_1=0.331, for which Corollary 2 predicts the failure weight 1/ρ=4.81/ρ=4.8. At 11 false alarm per 10410^4 healthy uses the best tested weight is 77 (195195 uses [152152, 246246]); the delay is U-shaped in the weight, as the efficiency (3) predicts: the symmetric rule is 1.49×1.49× [1.181.18, 1.821.82] slower and detects 90%90\% of the episodes within 1,0001,000 uses (weight 77: 99.8%99.8\%), and weight 2525 is 2.09×2.09× [1.711.71, 2.652.65] slower (Figure 8). At 33 and 1010 false alarms the best weight is 55, and the symmetric rule is 1.22×1.22× [1.031.03, 1.501.50] and 1.29×1.29× [1.041.04, 1.451.45] slower. In the bootstrap the best weight is 55 or 77 in 98.6%98.6\%, 93.2%93.2\% and 46.3%46.3\% of resamples at the three rates (33: 52.5%52.5\% at the highest rate). On the 170170 near-total episodes of the same band (mean q1=0.690q_1=0.690, predicted 1/ρ=2.01/ρ=2.0) the optimum moves down to 33–55 and the symmetric rule is within 1.041.04–1.19×1.19× of it (intervals contain 11), so the advantage of failure-heavy weights is specific to partial staleness, as Theorem 1 predicts. The per-memory LR-CUSUM tuned at the low end q1=q^0+0.15q_1= q_0+0.15 (Proposition 8) is within 1.21×1.21× [0.900.90, 1.641.64] of the best fixed weight; the deployed LR-CUSUM, tuned for a drop below break-even (q1=max(1−r^⋆,q^0+0.15)q_1= (1- r , q_0+0.15)), is 1.59×1.59× [1.201.20, 2.162.16] slower on these smaller changes, for which it is not tuned. Figure 8: Partial staleness on real streams: delay (Kaplan–Meier mean within 1,0001,000 uses) of fixed-weight counters against the failure weight k at 11 false alarm per 10410^4 healthy uses (95% template-bootstrap bands), with the weight 1/ρ1/ρ predicted by Corollary 2 from the mean failure rates. Left: 217217 partial episodes (q1∈[0.15,0.5]q_1∈[0.15,0.5]); right: 170170 near-total episodes; both with q0∈[0.02,0.15]q_0∈[0.02,0.15]. D.3.6 Per-memory weights against the best fixed ratio The learning-rate sweep of Section D.2.1 compares the per-memory LR-CUSUM with fixed-ratio counters on the same certificate, closed loop, at matched false retirements (Table 7). Delays here are naive means that count open episodes at their length so far (as in Section D.2); slower rules are censored more often, so this bias favours the fixed-ratio rules over the LR-CUSUM. With k down to 0.10.1, the closed-loop optimum lies below 11 on four of five models (k=0.75k=0.75, 0.250.25, 0.10.1 and 0.250.25; 2525 for Qwen3-8B), consistent with the predicted 1/ρ1/ρ of 0.670.67–2.142.14 under near-total staleness. The LR-CUSUM remains 5.85.8, 5.95.9, 2.02.0, 1.31.3 and 2.0×2.0× faster than the best tested ratio. Paired intervals of this ratio at each model’s best k exclude 11 for Qwen3-8B (5.85.8 [3.23.2, 9.49.4]), Qwen3-4B (5.95.9 [2.32.3, 11.211.2]) and OLMo-3-7B (2.02.0 [1.11.1, 4.64.6]), but not for Granite (2.02.0 [0.90.9, 3.43.4]) or Mistral (1.31.3 [0.90.9, 3.43.4]) (paper_figs/acl3_bestk.py). The weights the LR-CUSUM assigns differ across memories and models: the median per-memory weight ratio is 1.901.90, 1.331.33, 0.920.92, 0.500.50 and 0.670.67 (Qwen3-8B to OLMo-3-7B). D.4 Aligned safety, same-information baselines, decoding variance and natural edits The replays of Sections D.4.1 and D.4.2 reuse the recorded outcome tables of the five models in the main setting (30 streams per model and policy; no shard is missing). Intervals are 95% template-clustered bootstraps; pooled rows resample templates jointly across models. D.4.1 Safety scored as the certificate tests it Scoring. Under the standard scoring of KQA-Stream, a macro answer counts as correct only if the query was also routed to the right template, so a template’s reliability averages over misrouted queries, which the certificate’s paired test does not see (Section D.3.3). Here a one-step answer is correct if and only if it equals the gold answer (answer-only scoring). A candidate is then bad for the task if its answer-scored reliability is below r⋆=a−(m−1)/Lr =a-(m-1)/L (L=20m¯L=20 m), and bad under its own null if the exact drift ZEZ of its paired utility on its routed queries is ≤0≤ 0 (with the certificate’s own L, B and clipping; build instance excluded). An epoch is an interval between answer-changing edits of a template, and it is at risk if a candidate that is bad at some version of it is under test. We count three events: (i) a false certification, a fold of a candidate that is bad under its own null at the fold; it is on null evidence if the null held at every version of the fold’s test segment (the event bounded by Proposition 23(i)–(i) and Proposition 25(i)); (i) a task-level bad fold, below break-even; (i) a wrong answer among the one-step answers served. δ (i) own null on null ev. (i) task (i) wrong stale restart only .01 0/6704 0 0/6693 11.2% 69 .05 0/6657 0 0/6645 11.6% 75 .1 0/6620 0 0/6611 11.7% 78 .2 2/6554 2 2/6546 11.8% 82 .5 12/6401 12 10/6396 12.8% 103 no notice .01 6/6471 0 5/6458 10.7% 70 .05 12/6411 0 13/6396 11.2% 80 .1 13/6379 0 15/6365 11.4% 85 .2 26/6319 2 29/6304 11.7% 92 .5 58/6122 19 59/6109 12.5% 116 Table 15: Safety under answer-only scoring (pooled, 150 streams per row). (i), (i): epochs with the event / at-risk epochs; on null ev.: epochs with a false certification on null evidence; (i): wrong answers among served one-step answers; stale: stale answers per 10410^4 queries. Restart only: the restart-only certificate; no notice: discard on own alarm; ours (the recommended deployment): Section D.5.2. Bold: ours (both rows). Results. The restart-only certificate makes no false certification at δ≤0.1δ≤ 0.1 on any model (00 of 6,6576,657 at-risk epochs at δ=0.05δ=0.05; Table 15, Figure 3); at δ=0.2δ=0.2 and 0.50.5 the shares are 0.00030.0003 [00, 0.00090.0009] and 0.00190.0019 [0.00060.0006, 0.00360.0036], all on null evidence and in fresh test segments, as Proposition 23(i) allows. Intervals clustered by relation or by edit sequence are comparable. The two notions of “bad” nearly coincide: at δ=0.05δ=0.05, of 27,33927,339 candidates, 3,6113,611 are bad for the task, 3,6193,619 under their own null and 3,5713,571 under both; the task-level counts in Table 15 differ from the own-null counts by at most three epochs. Without notices, no false certification rests on null evidence at δ≤0.1δ≤ 0.1; every task-level bad fold at these levels (55, 1313 and 1515) spans an edit, i.e. its test segment began before the change, which is the residual risk that Proposition 25(i) bounds. Every point estimate of both certificates is below δ. False certification is not the error rate. At every δ, 11.211.2–12.8%12.8\% of the one-step answers that the restart-only certificate serves are wrong (7.67.6–8.7%8.7\% on Qwen3-8B, 13.213.2–17.1%17.1\% on Mistral-7B). Stale answers are only 1414–17%17\% of them; the rest are errors of macros that were above break-even when folded, which a fold accepts by design (r>r⋆r>r allows r<1r<1). δ therefore controls whether a bad memory is trusted, not how often a trusted memory errs; the second is set by r⋆r and by the detection delay of Section 3. Standard scoring. With routing errors counted as wrong answers, the restart-only certificate has bad folds in 0.00250.0025 [0.00110.0011, 0.00430.0043] of at-risk epochs at δ=0.05δ=0.05 (2929 of 11,67011,670), all of them candidates with a positive paired-utility drift, and discard-on-alarm has 0.00080.0008 [0.00030.0003, 0.00140.0014] (99 of 11,37511,375; Sections D.3.3 and D.3.4, Figure 7). The order of the two certificates flips between the scorings because the routing convention creates most of the restart-only certificate’s standard-scoring bad folds, whereas the no-notice certificate’s bad folds are carried across edits under both. Both scorings stay below δ at every level. D.4.2 Same-information baselines The restart-only certificate receives an edit notice: the templates whose gold answer an edit changes. It uses the notice only to restart candidates under test; folded macros are retired by the LR-CUSUM alone. The baselines below receive the same notices, on the same streams, audits and edits, with the same certificate parameters (δ=0.05δ=0.05, seven bets, h=7h=7) where they certify. Invalidate on notice drops the folded macro or the candidate under test of a notified template; a later verified live trace forms a fresh candidate, tested under the lifetime ledger, and folded macros are retired by notices only or also by their LR-CUSUM. Because carried-over candidates are discarded at every announced change, Proposition 23(iv) gives per-epoch safety for those epochs. Drop only sends the template back to live reasoning for the rest of the stream. The answer caches evict notified entries: a verified-answer cache, which stores a question’s live answer only when that run was audited and verified correct and returns it on an exact repeat, and the vCache-style cache of Section D.2.4. To model missed notices, each answer-changing edit is announced independently with probability f (the same draws for all policies). The compiled plan row is the restart-only certificate on compiled plans (with restarts). Policy (f=1f=1) Acc Calls Stale Δ vs restart only (points) restart only (restart-only certificate) .430 3.36 63.3 – invalidate .430 3.20 0.0 −0.06-0.06 [−0.94-0.94, +0.80+0.80] ours: invalidate + LR .433 3.38 0.0 +0.24+0.24 [−0.24-0.24, +0.81+0.81] drop only .332 3.84 0.0 −9.8-9.8 [−12.9-12.9, −7.0-7.0] verified-answer cache .188 3.43 0.0 −24.3-24.3 [−28.1-28.1, −20.3-20.3] vCache-style .186 3.50 0.0 −24.5-24.5 [−28.3-28.3, −20.5-20.5] compiled plan, no restarts .461 3.20 12.3 +3.1+3.1 [+2.5+2.5, +3.8+3.8] compiled plan .456 3.25 11.8 +2.5+2.5 [+2.0+2.0, +3.2+3.2] Stale per 10410^4 queries at notice fraction f 1 0.8 0.5 0 restart only (restart-only certificate) 63.3 65.3 67.8 66.3 invalidate 0.0 124.5 324.5 809.6 ours: invalidate + LR (recommended) 0.0 13.3 28.2 66.3 compiled plan 11.8 12.0 12.4 12.3 Table 16: Same-information baselines, pooled over five models (150 streams per row; standard scoring). Top: perfect notices, paired differences with 95% template-bootstrap intervals. Bottom: missed notices. Bold: ours. Results. With perfect notices, invalidation matches the restart-only certificate’s accuracy and serves no stale answers (Table 16); its J20J_20 differs by −0.12-0.12 [−0.82-0.82, +0.51+0.51]. Dropping without re-forming loses 9.89.8 points, and the answer caches stay at live accuracy (0.190.19, the pooled live accuracy) because they answer only 1010–13%13\% of queries in one step. With missed notices, invalidation alone degrades fast: at f=0f=0 every one of its 218218 stale episodes is still in service at stream end, and its accuracy falls by 4.44.4 points [2.22.2, 6.96.9]. Adding the LR-CUSUM keeps stale answers at or below the restart-only certificate’s at every f, and at f=0f=0 the two policies coincide. The share of at-risk epochs with a bad fold rises as notices are missed, because carried-over evidence is no longer discarded (Proposition 23(iv)): 0.00260.0026, 0.00300.0030, 0.00360.0036 and 0.00470.0047 for invalidate + LR at f=1f=1, 0.80.8, 0.50.5 and 00 (restart-only certificate: 0.00250.0025–0.00470.0047), all below δ. On Qwen3-8B, invalidate + LR adds 0.60.6 points [0.150.15, 1.11.1] over the restart-only certificate (Table 3). Compiled plans with restarts. With restarts, compiled plans keep +2.5+2.5 points over macros and 11.811.8 stale answers per 10410^4 queries, and they halve the bad folds of plans without restarts: 0.00320.0032 against 0.00640.0064 of at-risk epochs (2929 of 9,1799,179 against 5959 of 9,1729,172) and 0.290.29 against 0.630.63 bad folds per stream, for 0.60.6 accuracy points. Per model the gain over macros is 1.21.2–4.04.0 points (Table 6). D.4.3 Decoding variance Each instance–version of the main runs is decoded once, so the intervals elsewhere resample templates, streams and edit sequences, not decoding randomness. To measure the latter we re-decoded a Qwen3-8B subset (60 templates × 10 instances, edit sequence 101, knowledge-base versions 0, 8, 16 and 24) with three decoding seeds. Live accuracy per version is 0.368/0.343/0.322/0.3120.368/0.343/0.322/0.312, 0.367/0.338/0.320/0.3120.367/0.338/0.320/0.312 and 0.368/0.335/0.313/0.3050.368/0.335/0.313/0.305 for the three seeds: the standard deviation across seeds is at most 0.00440.0044 per version, against a template-bootstrap 95% half-width of 0.090.09–0.100.10 for one seed. Two seeds give the same final answer on 91.9%91.9\% of instance–versions and the same correctness on 97.6%97.6\%. Replays of each seed’s tables (20 streams each) rank the six policies (live, fold once, ExpeL- and ACE-style counters, the restart-only certificate and its no-notice variant) in the same order of accuracy for every seed. The restart-only certificate’s accuracy is 0.4470.447–0.4520.452 and its stale answers 3232–3333 per 10410^4 queries, against 698698–771771 for fold once and the counters. On the matched population of its folded macros, the LR-CUSUM leaves 0%0\% unretired and the counters 5757–75%75\%, a delay ratio of 131131–155×155× on this small subset (32–35 episodes). Decoding noise is thus two orders of magnitude below the template variation that our intervals already include, and no conclusion changes on the subset. D.4.4 Natural edits from Wikidata temporal qualifiers Timeline. Of the 207,851207,851 facts of the knowledge base, 15,29015,290 (7.4%7.4\%) carry time information (7,0177,017 start or end dates and 8,2738,273 dated events). The 300300 main templates are too sparse for a natural timeline: only 321321 of their instances change answer between 1992 and 2016 while staying answerable. We therefore built a natural subset with the construction rules of Appendix D: of 2,2502,250 candidate paths, 1,7691,769 are valid at every date, 8888 change at least 5%5\% of their answers, and 7272 templates (6868 new, 44 from the main set; 4,3044,304 instances) remain after the diversity caps. The stream applies the 1992 state at query 00 and one knowledge-base version per year, 1993–2016, at uniform stream times (8,0008,000 queries per year). Real changes are small and partial: 43.643.6 answer changes and 8.88.8 templates touched per year; in the median template-year only 3.3%3.3\% of the template’s answers change, and no change is total (main synthetic edits: median 40%40\%, 3%3\% total; Figure 9b). We ran four models (Qwen3-8B, Qwen3-4B, Granite-3.3-8B, Mistral-7B); OLMo-3-7B was not run on this timeline. The offline exact check reproduced all 111,904111,904 traces at all 2626 versions. Policies and settings are those of the main replays (3030 streams of 200200k queries per model, δ=0.05δ=0.05). Figure 9: Natural-edit transfer (Wikidata temporal qualifiers, 1992–2016; four models, 30 streams each). (a) Real fact changes per year. (b) Share of a template’s instances whose answer changes in one edit: half of the real changes touch under 5%5\% of the template, most synthetic ones 2020–75%75\%. (c) Stale answers per 10410^4 queries: counters serve 3.0×3.0× more than the restart-only certificate. (d) The same 7070 stale memories under three retirement rules (Kaplan–Meier). (e) Delay to detect at 33 false alarms per 10410^4 healthy uses on 1,3221,322 real episodes: fixed failure weights against the per-memory LR-CUSUM. (f) Share of at-risk epochs with a bad fold against δ (“answers”: answer-only scoring; no notice: discard at the own alarm). Bands and bars: 95% template bootstrap. Policy (pooled) Acc Calls Stale Bad/stream J20J_20 live always .175 4.47 0 0 79.3 fold once .340 2.60 231 8.5 62.6 ExpeL-style counter .391 3.39 241 149 59.0 ACE-style counter .391 3.52 238 207 59.2 restart only (restart-only certificate) .356 4.03 80.5 0.10 62.8 ours (recommended) .337 4.22 0 0.11 64.8 no notice (own detector) .384 3.84 99.5 0.03 60.2 compiled plan .374 3.95 2.8 0.10 61.2 Table 17: Natural edits, pooled over four models (30 streams of 200k queries each; standard scoring). Stale: per 10410^4 queries. Bold: ours. Results. Counters. Counters serve 3.0×3.0× [2.22.2, 4.04.0] more stale answers than the restart-only certificate (4.04.0, 2.82.8, 2.22.2 and 3.2×3.2× per model) and make 149149–207207 bad folds per stream, against 0.100.10. Replayed on the same macros as in Section D.3.2, 7070 of the 142142 stale episodes remain after removing those that some rule retired before the change; the counters leave 84.3%84.3\% [46.346.3, 100100] in service at stream end and the LR-CUSUM 37.1%37.1\% [4.54.5, 55.355.3], with a restricted-mean delay ratio of 20.5×20.5× [13.213.2, 72.972.9] and 405405 against 3030 stale answers per episode. The LR-CUSUM leaves more episodes open here than on synthetic edits because real staleness is mostly partial and 2626 of its 7070 episodes are cut off by the end of the stream; Section D.5.3 reports the sensitivity to the event set and to leaving out one year or one relation. Weights. On 1,3221,322 single-change episodes (3535 templates; mean q0=0.121→q1=0.330q_0=0.121→ q_1=0.330, optimal weight 1/ρ=3.71/ρ=3.7), the per-memory LR-CUSUM is faster than the best fixed weight at every matched false-retirement rate: 1.08×1.08× [1.011.01, 1.321.32], 1.56×1.56× [1.341.34, 1.851.85] and 1.14×1.14× [0.930.93, 1.561.56] at 11, 33 and 1010 false alarms per 10410^4 healthy uses (1.72×1.72× on the 907907 partial episodes at 33). The best fixed weight is about 22, not the 55–77 of the synthetic partial episodes (Section D.3.5), and on the 170170 near-total episodes the symmetric rule is best, as Corollary 2 predicts (1/ρ=0.731/ρ=0.73). Safety. Bad-fold epochs of the restart-only certificate are 0.00080.0008, 0.00130.0013, 0.00210.0021 and 0.00620.0062 of at-risk epochs at δ=0.01δ=0.01, 0.050.05, 0.20.2 and 0.50.5 (4/4,7904/4,790, 6/4,7706/4,770, 10/4,74410/4,744, 29/4,68629/4,686), and 00 up to δ=0.2δ=0.2 under answer-only scoring. Every point estimate is below δ; one per-model upper bound exceeds it (Qwen3-4B at δ=0.01δ=0.01, 22 of 690690 epochs). Accuracy and cost. The restart-only certificate adds 18.118.1 points [14.714.7, 21.721.7] over live reasoning with 10%10\% fewer calls (on Mistral-7B it uses slightly more, 5.315.31 against 5.045.04), and certified compiled plans serve 2.82.8 stale answers per 10410^4 queries. The counters are 3.53.5 points more accurate than the restart-only certificate (5.45.4 than the recommended deployment, Section D.5.3), with lower loss, because when real staleness is this mild, folding everything without certification pays; this is the price of safety of Section 4, bought here with 149149–207207 bad folds per stream. The synthetic ratios of the main text (7.07.0–13.5×13.5×, 60×60×, a best weight of 55–77) are not claimed for natural edits. D.5 Silent changes of the null, the recommended deployment, natural-edit sensitivity D.5.1 When the null changes without a notice What triggers a notice. In the replays, the notice of version v lists the templates k for which the gold answer of some instance routed to k differs between versions v−1v-1 and v (notice_tables); nothing else restarts a test or invalidates a memory. Epochs are the intervals between such notices. The certificate’s null, however, is the paired-utility drift Z≤0EZ≤ 0, which depends on live reasoning as well: with CL−CM=2C^L-C^M=2, L=50L=50 and a macro of reliability 0.90.9, a live accuracy of 0.80.8 gives U=+7EU=+7, and 0.950.95 gives U=−0.5EU=-0.5. Live reasoning can thus make a memory useless without any change of its answers or of the gold answers. Proposition 23(iv) therefore gives per-epoch safety only if notices cover every change of the null, including changes of live reasoning; across a change without a notice, the evidence banked before it is covered only by the carried-over bound min1,δjKc \1, _jK_c\ of Proposition 23(i). Two events must be kept apart: a candidate that is null at the fold, and a test segment that was null throughout (“on null evidence”). Only the second is bounded by δ without further assumptions. How often it happens. Over the three edit sequences and five models, 16,82716,827 of 107,496107,496 template-versions (15.7%15.7\%) change some routed instance’s live trace (answer, correctness or call count) without a notice, 1,8281,828 change a live correctness and 1,7151,715 the routed live accuracy; 5,1795,179 carry a notice (Table 18, top). Within the replays (δ=0.05δ=0.05, 150 streams per row), we tracked the exact drift of every candidate’s own test at every version of its life. Table 18 (bottom) counts the candidates whose null flips without a notice while they are under test, and the folds whose test segment spans such a flip. Flips are rare (0.30.3–0.7%0.7\% of candidates), and of 7,3857,385–9,8359,835 folds per row at most 22 span one, all of candidates that are useful at the fold; no fold of a candidate that is null at the fold occurs at δ=0.05δ=0.05. So on these streams the per-epoch statement holds as measured, but the assumption it needs is not automatic. Model template-versions notice silent live-trace change silent live-accuracy change Qwen3-8B 21,528 884 1,392 286 Qwen3-4B 21,600 867 3,124 363 Granite-3.3-8B 21,456 966 4,719 448 Mistral-7B 21,384 1,333 5,328 385 OLMo-3-7B 21,528 1,129 2,264 233 Policy scoring candidates with flip to null / useful folds span a flip null at fold restart only standard 31,593 226 97 / 155 7,385 2 0 restart only answer 27,339 192 82 / 133 8,632 0 0 ours standard 42,306 132 68 / 80 8,294 2 0 ours answer 37,928 114 58 / 75 9,835 0 0 Table 18: Silent changes of the null. Top: template-versions (three sequences, routed templates) with a notice and without one but with a change of live reasoning. Bottom: candidates whose own null flips without a notice while under test (flip events by direction), and folds whose test segment spans such a flip; δ=0.05δ=0.05, 150 streams per row, pooled over five models. Bold: ours (all policies). D.5.2 The recommended deployment The main tables and Figure 3 report the recommended deployment (“ours”: the restart-only certificate plus invalidation on notice, invalidate + LR of Section D.4.2); restart only (the restart-only certificate) is the comparison row. Under answer-only scoring, ours makes 00 own-null false certifications at δ≤0.1δ≤ 0.1 (0/5,8140/5,814, 0/5,7790/5,779, 0/5,7580/5,758 at-risk epochs at δ=0.01δ=0.01, 0.050.05, 0.10.1), 1/5,7181/5,718 at δ=0.2δ=0.2 and 11/5,61411/5,614 (0.00200.0020 [0.00060.0006, 0.00380.0038]) at δ=0.5δ=0.5, all on null evidence; task-level bad folds are 00, 00, 00, 11 and 88. Of the one-step answers it serves, 9.69.6–10.6%10.6\% are wrong (restart only: 11.211.2–12.8%12.8\%), and none is stale. In Table 2 its stale answers are reported with every notice missed (the counters’ information; then it coincides with the ledger certificate without restarts); its bad folds per stream are those with perfect notices (0.000.00, 0.070.07, 0.430.43, 0.630.63, 0.300.30). The matched retirement comparison (Figure 2) uses the macros that restart only folds, because their folded macros, like ours after a missed notice, are retired by detection alone. D.5.3 Natural edits: sensitivity All numbers below reuse the natural replays of Section D.4.4; the recommended deployment was replayed in addition (same streams). Recommended deployment. It serves no stale answers, has bad folds in 0.00070.0007, 0.00130.0013, 0.00290.0029 and 0.00600.0060 of at-risk epochs at δ=0.01δ=0.01, 0.050.05, 0.20.2, 0.50.5 (4/5,4934/5,493, 7/5,4987/5,498, 16/5,49216/5,492, 33/5,48733/5,487), and 0.110.11 bad folds per stream, but it is 1.91.9 points [1.01.0, 2.82.8] less accurate than restart only (0.3370.337 against 0.3560.356): when an edit changes 3%3\% of a template’s answers, invalidation discards a memory that is still mostly right. Event set. The matched retirement comparison uses the 7070 of 142142 stale episodes that no rule retired before onset. On all 142142, counting a pre-onset retirement as detection at 00 uses, the counters leave 70.4%70.4\% (ExpeL) and 62.7%62.7\% (ACE) in service at stream end against 26.8%26.8\% for the LR-CUSUM, with a delay ratio of 13.6×13.6× [5.45.4, 88.788.7]; each rule on its own episodes gives 90.1%90.1\%, 89.0%89.0\% and 45.2%45.2\% unretired and a ratio of 10.3×10.3×. Leave one out. Leaving out the episodes whose onset falls in one year, the counters leave 57.757.7–100%100\% and the LR-CUSUM 3.83.8–45.6%45.6\% in service, and the delay ratio is 18.518.5–32.0×32.0×; leaving out the templates that use one changing relation (21 relations), 56.056.0–100%100\% against 21.221.2–45.6%45.6\% and 17.217.2–38.1×38.1×. Most common episodes (4444 of 7070) start in the last year, 2016, and are censored early; without them the shares are 58%58\% against 4%4\%, so the unretired shares depend on censoring, while the delay ratio is stable. Leaving out one relation, the counters serve 2.52.5–3.2×3.2× more stale answers than restart only (2.32.3–2.9×2.9× than the ledger without restarts, i.e. ours with every notice missed), every certificate’s bad-fold share stays below δ at every level (largest: 0.00800.0080 at δ=0.5δ=0.5), and the per-memory LR-CUSUM stays faster than the best fixed weight at 11, 33 and 1010 false alarms per 10410^4 healthy uses (1.011.01–1.17×1.17×, 1.331.33–1.74×1.74×, 1.041.04–1.29×1.29×). The weight comparison uses the 1,3221,322 single-change episodes; episodes with several changes do not have one change point and are not used. D.5.4 Which variant for which goal Variant Use when Evidence Recommended: invalidate on notice + certify with restarts + LR-CUSUM false deployments must be bounded and notices may be incomplete 0 stale with notices; 13–66 per 10410^4 when 20–100% are missed (Table 16); no false certification at δ≤0.1δ≤ 0.1; code: 2.02.0 stale per 10410^4 with only requirement notices, bad folds in 1.3⋅10−51.3· 10^-5 of epochs under test at δ=0.05δ=0.05 (Section D.6) Restart only changes are mild and partial, notices reliable +1.9+1.9 points over invalidation on natural edits (Section D.5.3) Compiled plans + certificate the derivation can be re-executed cheaply on current facts +2.5+2.5 points over restart-only macros, 1212 stale per 10410^4 (Table 16) Verified answer cache exact repeats, reliable notices live accuracy; 13%13\% of queries in one step Counters (ExpeL/ACE) staleness is mild and folding everything pays; never when stale answers are costly natural edits: +3.5+3.5 points, 149149–207207 bad folds per stream Table 19: Decision guide (pooled evidence; main text: Section 6). Bold: ours. D.6 A code domain with an exact verifier: CodeStream CodeStream tests the theory on a second real domain, chosen so that the verifier is exact and memories are what tool-building agents store: a verified program, reused as a tool (Wang et al., 2024a; Cai et al., 2024; Wang et al., 2024c; Yuan et al., 2024). It writes the same potential-outcome tables as KQA-Stream, so every policy, metric and replay setting of Appendix D runs on it unchanged; only the domain, the stream length and the cost unit differ. The setting was fixed after a two-model pilot, before the full run. Tasks, queries and the verifier. We use 521 EvalPlus tasks (Liu et al., 2023): 359 from MBPP+ (v0.2.0; Austin et al., 2021) and 162 from HumanEval+ (v0.1.10; Chen et al., 2021). We exclude 21 tasks: 17 with a special oracle or a failing canonical solution, and 4 with fewer than 10 usable hidden inputs. A query is a task plus one hidden plus-test input, 20 per task (10,42010,420 queries); its answer is the function’s output. The verifier is EvalPlus’s own check against the reference output of the task version in force: exact equality, or closeness within the task’s tolerance for floating-point outputs. Each (program, version, input set) runs once and is cached, so every policy sees the same verdicts; re-executing the 6,3306,330 sandbox runs of the table build in fresh processes changes 5757 of 126,600126,600 verdicts (0.05%0.05\%), from programs whose output is not deterministic (e.g. set order). Live reasoning and memories. The live reasoner is a solve–test–repair agent. It writes the function from the task prompt (temperature 0.70.7, at most 768 new tokens; Qwen3 with thinking disabled), runs it on the visible tests (the EvalPlus base inputs) in a sandbox of the current version, and, if a test fails, gets the failing call, the expected output and the output or exception it produced. It makes at most three attempts; the final program is run on the query input. Each task has 20 samples, one per query, so live outcomes of different queries are independent samples. A memory is the final program of a live trace whose answer was audited correct; folded, it is executed directly on later queries of the same task, in the current environment. Memories are thus compiled plans in the sense of Section 6: an environment change can break them, and a requirement revision changes the answers they are checked against. Some programs pass the visible tests but fail other hidden inputs, EvalPlus’s own finding that base tests under-specify tasks; they are the domain’s candidates that are bad at formation, so certification has real work to do. Edit catalogue. Each timeline has 14 edits (15 versions) at uniform stream times, in the order A1, S1, A2, A3, S2, A4, S3, A5, A6, S4, A7, S5, A8, S6. Requirement revisions S1–S6 are the main source of staleness. Each batch revises 22 tasks (6%6\% of the revisable tasks; 132 per timeline, each at most once) by appending one natural post-processing change to the prompt as “Requirement update: …”: sort, deduplicate, lowercase, reduce modulo 109+710^9+7, clamp negatives to 00, round to two decimals, return ’Yes’/’No’, or return a list instead of a tuple. The reference output changes exactly on the inputs whose output the change affects, so staleness is partial or total by construction from the real input distribution (in the design-time catalogue the share of a revised task’s queries whose answer changes ranged from 0.050.05 to 11, median about 0.60.6). The three timelines (seeds 101, 202, 303) revise different tasks. API edits A1–A8 re-enact documented Python breaking changes in the sandbox (Table 20); they change no reference output, and a memory goes stale only if its own program changes behavior. Error messages carry the migration hint, as real deprecations do, so live reasoning can repair from test feedback. Id Edit (re-enacted) Precedent Programs broken A1 typing.List etc. removed PEP 585 deprecation 55 / 68, total A2 re.sub/split: count, flags keyword-only deprecated in 3.13 0 / 0 A3 str patterns ASCII-only by default Python 2/3 default 0 / 0 A4 int–str conversion capped at 640 digits 3.11 (CVE-2020-10735) 1 / 1, partial A5 round half away from zero Python 2 2 / 2, partial A6 math.floor/ceil return float Python 2 1 / 1, total A7 Counter.most_common tie order dict order before 3.7 0 / 0 A8 zip strict by default PEP 618 option 3 / 3, partial Table 20: API edits of CodeStream. Programs broken: distinct verified programs whose behavior the edit changes, out of 1,4941,494 (Qwen3-4B) / 1,3801,380 (Qwen3-8B) in the two-model pilot. Notices. A requirement revision bumps the task’s version: this is the answer-changing notice of the main experiments. API edits are unannounced: they are the domain’s unannounced changes, and for them the LR-CUSUM is the only safety net. They also change live reasoning (in the pilot, after A1 the mean number of attempts on the 210 affected Qwen3-4B traces rose from 1.211.21 to 2.132.13) without changing any answer, a silent change of the certificate’s null in the sense of Section D.5.1. The changelog variant additionally invalidates, at each API edit, every memory whose source mentions a changed symbol, a static match that real maintainers would make: it is exact for removals but flags many unaffected programs for semantic edits. Streams, policies and statistics. Streams have T=106T=10^6 queries, Zipf demand over tasks (s=1.2s=1.2), audits with p=0.1p=0.1, L=20m¯L=20 m and δ=0.05δ=0.05; the certificates, the bet grid and the LR-CUSUM threshold h=7h=7 are those of the main experiments. Each model is replayed on 10 stream seeds per timeline. All memory policies store programs. Labels: fold once; ExpeL- and ACE-style counters; per candidate, the per-candidate certificate of Table 1, without notices or restarts, i.e. the counters’ information; restart only, the restart-only certificate, which restarts at requirement notices; ours, the recommended deployment (invalidate on notice and re-form, ledger, restarts, LR-CUSUM), as in the main text; and ours+changelog. A stale answer is a folded answer that is wrong although the same memory answered this query correctly when it was formed. 95% intervals resample templates (clusters) with 2,000 bootstrap draws, pooled over seeds and timelines; pooled intervals resample (model, template) clusters. Models and compute. All five models ran all three timelines, so pooled numbers rest on 150 streams, 30 per model; every stream has the same weight. Two cost accountings. Writing, testing and repairing a program costs more than running a stored one, and the domain admits two units. Step accounting, our main setting and faithful for tool libraries, charges a live query its attempts (LLM calls) plus one execution step, and a folded memory one execution step; live reasoning costs 2.682.68 steps per query. Call accounting, the unit of KQA-Stream, charges live reasoning its LLM calls (1.681.68 per query) and a folded memory one call. J20J_20 is computed in each unit with L=20m¯L=20 m, as in the main text. Every cost statement below names its unit. Figure 10: CodeStream (EvalPlus; five models, 150 streams of 10610^6 queries). (a) Stale answers per 10410^4 queries (step accounting; dots: models); ours+CL: ours with changelog invalidation. (b) Share of each policy’s own stale memories not retired by stream end (numbers: stale episodes). (c) Stale exposure at matched false retirements: folded uses while stale against false retirements, both per stream, for the per-memory LR-CUSUM (threshold sweep; star: h=7h=7) and fixed failure:success weights, all on the same certificate (weights 1:11:1–5000:15000:1 were run; above 25:125:1 the curves coincide with 5000:15000:1, the no-forgetting limit). (d) Accuracy gain over live reasoning (points) against cost per query relative to live reasoning, in steps (filled) and in LLM calls (hollow). Bars: 95% template-clustered bootstrap; in (d) on the filled markers only, since Δ in LLM calls is within 0.10.1 points of it. step accounting call accounting Policy Acc Δ (points) steps/q J20J_20 one-step calls/q J20J_20 one-step Stale Unretired live always .762 00 2.68 16.0 0 1.68 10.21 0 0 – fold once .765 +0.3+0.3 [−1.9-1.9, 2.52.5] 1.24 14.1 .94 1.17 9.35 .94 1,043 100% ExpeL-style counter .822 +6.0+6.0 [4.24.2, 7.97.9] 1.37 11.1 .90 1.27 7.44 .90 518 59% ACE-style counter .823 +6.2+6.2 [4.34.3, 8.18.1] 1.40 11.0 .89 1.29 7.40 .89 510 57% per candidate (no notices) .850 +8.9+8.9 [7.27.2, 10.610.6] 2.01 10.2 .63 1.95 7.19 .33 10.1 10% restart only .849 +8.7+8.7 [7.17.1, 10.410.4] 2.06 10.4 .61 1.97 7.26 .32 11.0 9% ours (recommended) .849 +8.7+8.7 [7.17.1, 10.510.5] 2.06 10.3 .61 1.97 7.27 .32 2.0 6% ours + changelog .849 +8.7+8.7 [7.07.0, 10.410.4] 2.08 10.4 .60 1.98 7.30 .32 0.01 – Table 21: CodeStream, pooled over five models (150 streams of 10610^6 queries). Accuracy and Δ (paired, template-clustered 95% intervals) in step accounting; in call accounting every Δ is within 0.10.1 points of these. One-step: share of queries answered by a folded memory. Stale: per 10410^4 queries (step accounting). Unretired: share of the policy’s own stale episodes still in service at stream end (step accounting; 14,82614,826, 14,70414,704, 1,4561,456, 1,3281,328 and 9696 episodes for the counters, per candidate, restart only and ours; fold once never retires). Bold: ours. (a) Counters leave stale memories in service. The counters serve 518518 [471471, 567567] (ExpeL) and 510510 [463463, 561561] (ACE) stale answers per 10410^4 queries, against 10.110.1 [8.88.8, 11.411.4] for the certificate with the counters’ information (per candidate): 51×51× pooled (95% intervals within [4343, 6060]) and 3535–73×73× on every model, with every per-model lower confidence bound at least 2525. Per folded answer, i.e. after dividing each policy’s stale answers by its one-step coverage, the ratio is still 36×36× pooled (2727–48×48× per model), so the gap is not an artifact of the counters’ higher coverage. Against restart only (11.011.0 [9.09.0, 13.213.2]) the ratio is 47×47× [3939, 5858] (ExpeL) and 47×47× [3838, 5757] (ACE); against ours (2.02.0 [1.41.4, 2.72.7]) it is 258×258× [191191, 363363] (ExpeL). Ours with changelog notices serves 0.010.01 per 10410^4 (all on Mistral-7B). Each policy’s own stale memories remain in service at stream end in 59%59\% [5656, 6161] (ExpeL) and 57%57\% [5454, 5959] (ACE) of cases, against 10%10\% [77, 1212] for per candidate; fold once retires none. These shares compare each rule on its own folded memories, not on a matched population as in Figure 2; the matched comparison is claim (b). In call accounting the stale answers of the certified policies are 9.29.2 (per candidate), 12.312.3 (restart only) and 2.02.0 (ours) per 10410^4; the counters serve 5555–56×56× more than per candidate pooled (3838–86×86× per model), and the counter/restart-only ratio is 42×42× [3232, 5656] (ExpeL). (b) Per-memory weights at matched false retirements. All rules run on the same per-candidate certificate and differ only in how they retire a folded memory. The per-memory LR-CUSUM uses its default threshold h=7h=7. Each fixed failure:success weight k:1k:1 (score +1+1 per audited failure, −1/k-1/k per success), for k∈1,2,7,25,50,100,200,500,1000,5000k∈\1,2,7,25,50,100,200,500,1000,5000\, gets its best threshold (between 11 and 2,5842,584 failure units; a finer grid from 3434 to 8989 for 7:17:1 and 5000:15000:1) among those with no more false retirements per stream than the LR-CUSUM (a retirement of a memory at or above break-even), a choice that favors the fixed rules. We measure stale exposure: folded uses of a memory while it is stale, per stream. Pooled, the LR-CUSUM has 702702 stale uses per stream at 33.533.5 false retirements; the best fixed weight (5000:15000:1, threshold 7878, 31.731.7 false retirements) has 7,3007,300, i.e. 10.4×10.4× [7.77.7, 13.713.7] more, and the symmetric 1:11:1 rule 29.1×29.1× [14.614.6, 51.751.7] more (Table 22). On every model the best fixed weight is 7.67.6–11.4×11.4× worse, with every lower confidence bound at least 3.53.5. On the shared threshold grid, exposure falls from 25:125:1 to 100:1100:1 and is flat from 100:1100:1 to 1000:11000:1 (within 3%3\%): there a success barely discounts a failure, and all weights converge to one rule, a failure count that never forgets; 5000:15000:1, this limit tuned on a finer threshold grid, is lower still. The optimum is interior on Mistral-7B (7:17:1) and this limit on the other four models. The theory predicts both the limit and the gap. In 86%86\% of the 1,4561,456 stale episodes the memory was perfect before the change (q0=0q_0=0): a failure cannot occur before the change, so the optimal weight 1/ρ(q0,q1)1/ρ(q_0,q_1) of Equation 4 is infinite, and the best global weight moves to the no-forgetting limit. The other episodes want the opposite: 1/ρ=01/ρ=0 when q1=1q_1=1 (9%9\%) and a median of 1.01.0 otherwise (5%5\%). No single weight fits both groups, and the limit is still 10×10× worse because a global rule also needs a global threshold. To spare the healthy memories that do fail now and then, it must wait for about 7878 failures. The per-memory LR-CUSUM instead charges each failure ln(q1/q^0) (q_1/ q_0) of its own memory, which is large when the memory never failed on its shadow labels. Among retired episodes of perfect memories, it retires after a median of 3030 stale uses, about three audits; the best fixed rule needs 874874. This is also why the gap is far larger than on KQA-Stream (1.31.3–5.9×5.9× in detection delay) and on natural edits (1.11.1–1.6×1.6×), where few memories are perfect before the change. 42%42\% of the episodes are partial (q1<0.95q_1<0.95; median q1=1q_1=1). LR-CUSUM (h=7h=7) best fixed weight (1:11:1–5000:15000:1) Model FR stale uses episodes weight (h) × LR [95% CI] Qwen3-8B 38.8 968 196 5000:15000:1 (84) 7.67.6 [3.53.5, 21.721.7] Qwen3-4B 29.8 656 234 5000:15000:1 (84) 9.89.8 [5.85.8, 16.716.7] Granite-3.3-8B 47.5 526 281 5000:15000:1 (42) 8.28.2 [6.46.4, 10.910.9] Mistral-7B 29.7 813 468 7:17:1 (84) 11.411.4 [9.39.3, 13.913.9] OLMo-3-7B 21.8 550 277 5000:15000:1 (46) 9.69.6 [7.57.5, 12.012.0] pooled 33.5 702 1,456 5000:15000:1 (78) 10.410.4 [7.77.7, 13.713.7] Table 22: CodeStream claim (b), step accounting. FR: false retirements per stream; stale uses: folded uses while stale per stream; ×LR: the fixed rule’s stale uses over the LR-CUSUM’s at no more false retirements. 5000:15000:1 is the no-forgetting limit, tuned on a finer threshold grid; on the shared grid exposure is flat from 100:1100:1 to 1000:11000:1. Bold: ours. (c) Per-epoch false certification. An epoch under test is a (task, version) pair in which a candidate is under test, from its formation to its fold or discard; a bad fold folds a candidate whose reliability at that version is below break-even, the task-level event of Figure 3b. Ours has bad folds in 66 of 965,730965,730 epochs at δ=0.01δ=0.01 (6.2⋅10−66.2· 10^-6), 1212 of 946,678946,678 at δ=0.05δ=0.05 (1.3⋅10−51.3· 10^-5; Clopper–Pearson upper bound 2.2⋅10−52.2· 10^-5), 1616 of 934,466934,466 at δ=0.1δ=0.1 (1.7⋅10−51.7· 10^-5) and 2121 of 917,079917,079 at δ=0.2δ=0.2 (2.3⋅10−52.3· 10^-5): the rate rises with δ and stays more than 1,000×1,000× below it. Per model, every rate is at least 250×250× below δ; the closest to δ is Mistral-7B’s at δ=0.01δ=0.01 (66 of 161,262161,262). With changelog notices it is 00 of 941,085941,085 at δ=0.05δ=0.05, and in call accounting 1010 of 968,421968,421. The counters fold a below-break-even candidate in 4.8%4.8\% (ExpeL, 4,625/97,1724,625/97,172) and 5.4%5.4\% (ACE, 5,733/106,4245,733/106,424) of their epochs under test, fold once in 3.7%3.7\%; these policies fold at formation, so their epochs under test are formation epochs. The guarantee of Proposition 23(iv) does not cover the API edits, which change live reasoning silently; the measured rates show that, here too, this boundary is not where certification fails. (d) Accuracy and cost. Live reasoning is right on 76.2%76.2\% of queries (57.557.5–83.1%83.1\% per model). Ours gains 8.78.7 points [7.17.1, 10.510.5]: 2.42.4 (Qwen3-4B), 3.13.1 (Qwen3-8B), 7.27.2 (OLMo-3-7B), 10.010.0 (Granite-3.3-8B) and 21.021.0 (Mistral-7B) per model, because a certified program has passed hidden inputs that the live agent’s visible tests do not cover. The cost depends on the unit. In steps ours costs 0.620.62 fewer per query [0.520.52, 0.710.71] (−23%-23\%) and lowers J20J_20 by 5.35.3 [4.44.4, 6.36.3]; in LLM calls it costs 0.290.29 more per query [0.200.20, 0.390.39] (+17%+17\%), because shadow mode pays for live reasoning while a candidate is under test, and still lowers J20J_20 by 2.62.6 [2.02.0, 3.33.3]. The certified policies have the highest accuracy and the lowest J20J_20 in both units. Counters gain 6.06.0–6.26.2 points at a lower cost in both units (on Qwen3-8B and Qwen3-4B they are 0.40.4–0.80.8 points below live reasoning) while serving 47×47× more stale answers than restart only; fold once is within noise of live reasoning (+0.3+0.3 points [−1.9-1.9, 2.52.5]). Ties: the certification load on a real domain. The cost unit changes how much certified reuse the stream allows, and Theorem 6(i) says why. Audited memory–live pairs of the certified policies agree on the outcome in 9494–97%97\% of cases (8484–99%99\% per model), and live reasoning re-derives the memory’s exact program in 4949–59%59\% (2121–79%79\% per model; 2121–33%33\% for Mistral-7B and OLMo-3-7B). In call accounting, a tie in which live reasoning succeeds at its first attempt has paired utility exactly 00; this holds for 8484–86%86\% of audited pairs (6767–89%89\% per model). The utility law then has a large atom at zero, so KLinf(PZ)KL_ (P_Z) is small and certification is slow, and one-step coverage drops to 0.320.32, against 0.610.61 in step accounting, where every tie is positive evidence because live reasoning costs at least one step more than the memory. For ours, the drop tracks the share of zero-utility pairs across models: Qwen3-8B and Qwen3-4B (8888–89%89\%) keep 3434–36%36\% of their step-accounting coverage, Granite-3.3-8B and OLMo-3-7B (8686–87%87\%) 5252–53%53\%, and Mistral-7B (68%68\%) 81%81\%. This is the certification load Θ∝1/(pKLinf) 1/(pKL_ ) of Sections 4 and 5 observed on a real domain: an exact verifier and a strong live reasoner make audits nearly uninformative, and safe contraction must wait for the queries on which the two differ. Scope. API edits break few programs: in the pilot, A1 breaks 3.73.7–4.9%4.9\% of the distinct verified programs and the other seven edits together at most 77 programs per model, because MBPP and HumanEval programs mostly use builtins. They test the unannounced channel and changelog notices, but we claim no staleness magnitudes for real library drift; a benchmark with version-specific APIs would measure them. Requirement revisions are controlled changes of real tasks, as the fact edits of KQA-Stream are. Appendix E Worked examples from KQA-Stream This appendix follows real Qwen3-8B macros through the replays of Table 3 (main setting). Questions, unedited facts, LLM outputs, routing decisions and audit outcomes come from the logs; post-edit facts are counterfactual (Appendix D). Cases are named by edit sequence, stream seed and policy; t is the query index in the stream. Every replay used here was re-run with an instrumented recorder and matched the stored run exactly. The values r, a, m and r⋆r are exact popularity-weighted values over the instances routed to the template, with L=20m¯L=20 m (m¯ m: the stream’s mean live calls). A stale answer is a folded answer that was correct at formation and is wrong now. “Betting” and “SPRT” denote betting + LR-CUSUM (per-candidate level δ, fold threshold ln20 20, no ledger; the “per candidate” row of Table 5) and SPRT + LR-CUSUM. These are mechanism examples: the main-text results use the restart-only ledger certificate and the recommended deployment, whose first threshold is ln20+ln(π2/6)=3.49 20+ (π^2/6)=3.49 (so the 3333-label fold of Table 23, at log-wealth 3.433.43, is a per-candidate fold). E.1 One macro from formation to re-formation t Stage Real evidence (Qwen3-8B, template T118, edit sequence 303, stream seed 6) Statistic 44 form Audited live trace of “What individuals have their residence in the state where Louisiana Tech University is situated?” (3 calls, 1,300/59 tokens): hop located in the administrative territorial entity (fwd) → Louisiana; hop residence (bwd) → Elmore Leonard; answer type human. Correct. r=1.00r=1.00, r⋆=0.772r =0.772 44–10,769 certify 33 audited shadow labels: 26 both correct (U=+2U=+2), 7 live wrong and macro correct (U≈+70U≈+70), 0 macro failures lnW+:0→3.43≥ln20 W^+\!:0→ 3.43≥ 20 10,769 fold routing call "type": 1 + lookup in a 272-anchor table frozen at v0v_0 LR-CUSUM q0=.015q_0=.015, q1=.294q_1=.294, h=7h=7 → 93,194 reuse 2,982 one-call answers before the edit, all correct (live: .798 on the same queries); 5,566 LLM calls (64.8%) saved over the macro’s 3,026 folded uses 93,194 edit v16v_16 located in the administrative territorial entity, f=0.6f=0.6 (451 facts); e.g. Cardiff University: Cardiff ⇒ Hong Kong, gold John Williams ⇒ Bruce Lee, Ip Man, Stephen Chow; the macro still answers John Williams r: 1.00→0.367<r⋆=0.4911.00→ 0.367<r =0.491 93,227–94,247 retire audited folded labels 0,1,0,0 (Univ. of Basel, Univ. of Windsor, East Tennessee State Univ., Cardiff Univ.); 1,053 queries, 43 folded uses and 29 stale answers after the edit S: 2.99, 2.66, 5.66, 8.65>7>7 94,378–98,104 re-form trace (valid at v16v_16) of “Who lives in the state where University of Freiburg is located?”; 12 labels (7 discordant); then 3,694 folded uses to the end, 0 wrong (0 of 334 audited labels fail) lnW+=3.00 W^+=3.00 93,194–200k other rules ExpeL-style, ACE-style and fold once do not retire the macro before the stream ends: 3,863 folded uses and 2,427 stale answers after the edit. SPRT: fold at 20,978, then the same retirement (29 stale answers). Compiled plan + betting: 0 stale answers, 0 of 346 post-edit labels fail ExpeL: 333→249333→ 249 ACE: 331/0→462/215331/0→ 462/215 Table 23: Lifecycle of one real macro under betting + LR-CUSUM, and the same macro under other rules (last row). t: query index in the 200k-query stream. Stale answers include the retiring query. Post-edit facts are counterfactual by construction. One real macro passes through every stage of the lifecycle and illustrates Equations 1, 6, 2 and 9. Template T118 maps a university to the people living in its state (edit sequence 303, seed 6). It is the fifth most popular template in this stream (π=0.036π=0.036). Formation and certification. The macro forms at t=44t=44 from the first audited T118 query (Table 23). At its second step the live agent outputs "action": "hop", "relation": "residence (backward)", "direction": "backward". Both one-call baselines fail; the parametric one outputs "answers": ["State of Louisiana"]. At v0v_0 the macro is always correct, while live has a=0.800a=0.800 at m=2.85m=2.85 calls, so r⋆=0.772r =0.772 with L=66.98L=66.98. The 26 concordant pairs add 0.010–0.019 each to lnW+ W^+; the 7 pairs where only live fails add 0.33–0.52 each. The SPRT plugs in r0=0.756r_0=0.756 and r1=0.788r_1=0.788 (true: 0.772 and 0.800) and needs 72 labels instead of 33. Retirement and re-formation. Edit v16v_16 lowers the macro’s reliability to 0.367<r⋆=0.4910.367<r =0.491, while live re-derives answers on the edited facts. The LR-CUSUM was set at the fold to q0=0.0147q_0=0.0147 and q1=1−r^⋆=0.294q_1=1- r =0.294. It scores +2.995+2.995 per failure and −0.333-0.333 per success. One failure thus weighs about nine successes (ρ=0.111ρ=0.111 in Equation 4), and three failures in four labels retire the macro. The SPRT retires it on the same labels at the same t. The re-formed candidate needs only 12 labels, 7 of them discordant, because live accuracy has fallen to 0.517. Counters and plans. The ExpeL-style, ACE-style and fold-once policies do not retire the macro before the stream ends (last row of Table 23). Its 331 successful audits before the edit bank an ExpeL importance of 333, and Proposition 9(a) predicts a delay of 333/(2⋅0.633−1)=1,249333/(2· 0.633-1)=1,249 labels; only 346 remain. The compiled plan, which re-executes the path on current facts, has no failure in 677 audited labels. How typical is this case? Among betting’s 81 stale-after-edit onsets in the 30 streams, this one is at the median in folded uses to retirement (43); their median stale count is 30 (IQR 21–47). The counter side is extreme (98.5th percentile of ExpeL onsets): T118 is popular, and 106,806 queries remain. In seeds 1 and 2 the counters serve 358 and 273 stale answers after this edit, and betting 52 and 40. Takeaway. Certification took 33 labels, and a likelihood-ratio detector limited a real edit to 29 stale answers. Counters with banked credit kept the stale macro to the end of the stream. E.2 Counters under partial staleness A at in-control rate Rule at edit retires at label uses / stale τEτ q0=.05q_0=.05 q0=.013q_0=.013 ExpeL-style importance (±1± 1) 92 not by end (239) 3,283 / 894 ∞ ∞ ∞ ACE-style helpful/harmful 90/0 not by end (330/93) 3,283 / 894 ∞ ∞ ∞ symmetric, H=2H=2 (SPRT + symmetric)† 0 19 169 / 47 17.8 420 6.2×1036.2×10^3 symmetric, H=4H=4 0 152 1,460 / 374 180.6 1.5×1051.5×10^5 3.7×1073.7×10^7 LR +7/−1+7/-1, H=43H=43 0 44 389 / 103 37.1 2.0×1052.0×10^5 9.7×1099.7×10^9 LR-CUSUM as run (SPRT), h=7h=7† 0 19 169 / 47 13.7 249 2.0×1042.0×10^4 Table 24: Retention rules on the same 333 real post-edit labels of the T048 macro (edit sequence 303, seed 0). Uses / stale: folded uses before retirement / stale answers incl. the retiring query (to the stream’s end for “not by end”). τEτ: exact mean delay (labels) at q1=0.267q_1=0.267. A: exact mean labels to a false retirement; .013 is the deployed LR-CUSUM’s in-control estimate. †A real replay retired the macro at label 19. Bold: ours. On one real sequence of audit outcomes, this example compares all retention rules and checks Propositions 9, 1 and 2. The T048 macro maps a film to Academy Awards via producer and nominated for (edit sequence 303, seed 0). It forms at t=738t=738 from the trace for “What Academy Awards did the producer of Man of Steel get nominated for?” and passes all 90 audits before the edit. Edit v5v_5 gives a new producer to 25% of the relevant subjects. Goodfellas’ producer Irwin Winkler, for example, becomes Eminem, yet the macro still answers Best Picture. Its reliability falls to 0.733<r⋆=0.7680.733<r =0.768, so q1=0.267q_1=0.267: partial staleness. The 333 post-edit labels contain 93 failures. Table 24 replays every rule on these labels. By Proposition 9(b), ExpeL (credit 92) and ACE (90/0) are ever retired with probability at most 10−40.410^-40.4 and 10−39.510^-39.5. The symmetric rule is in the negative-drift regime of Theorem 1 (μ1=−0.467 _1=-0.467, γ=0.344γ=0.344). At a similar A, the +7/−1+7/-1 rule of Corollary 2 retires at label 44 instead of 152. Every realized delay lies between the 57th and 81st percentile of its exact law. On the longest real onset (T164, sequence 101, seed 5; 4,004 labels), the counters serve 9,997 stale answers. There the symmetric rule with H=4H=4 needs 610 labels and the LR-CUSUM 8. Over the 30 streams, ExpeL has not retired 95.5% of its 287 partial-staleness onsets by the end of the stream, and ACE 80.8% of its 312. The bounds of Proposition 9(b), computed from each onset’s real credit, average 0.89 and 0.76. The LR-CUSUM detectors leave 1 of 24 (SPRT) and 0 of 11 (betting) unretired. Takeaway. Under partial staleness, ±1± 1 counters with banked credit are practically never retired: the probability of ever retiring them is small (Proposition 9(b)), and on real streams they stay in service until the stream ends. A failure must weigh 1/ρ(q0,q1)1/ρ(q_0,q_1) successes, about 6.5 here. E.3 Certifying utility instead of reliability # t instance (T010) (YM,YL)(Y^M,Y^L) UtU_t ZtZ_t lnW+ W^+ lnW− W^- plug-in LLR 1 3179 Q2656 (1,1) +2+2 +.027+.027 .011 −.011-.011 .096 11 27863 Q30002 Chengdu (0,0) +2+2 +.027+.027 .121 −.115-.115 .910 17 34168 Q25416 (1,1) +2+2 +.027+.027 .190 −.174-.174 1.342 18 35098 Q48320 Kōbe (stale) (0,1) −64.4-64.4 −.877-.877 −.290-.290 .096 1.295 19 35271 Q1467 Cebu City (stale) (0,1) −64.4-64.4 −.877-.877 −.599-.599 .402 1.248 35 59956 Q1467 (0,1) −64.4-64.4 −.877-.877 −1.573-1.573 3.226 discard 1.781 94 187133 Q123709 (1,1) – – – – 3.007 fold Table 25: Real certificates for the T010 candidate (edit sequence 202, seed 1). #: audited label, shared by betting and the SPRT; labels 1–11 precede edit v7v_7. Threshold ln20=2.996 20=2.996 for lnW+ W^+ (fold), lnW− W^- (discard) and the plug-in LLR (fold). Template (rank) πi _i audits ni⋆n _i ni⋆/(pπi)n _i/(p _i) epoch e−Θi/πie^- _i/ _i one-step labels to fold in stream (queries) (queries) predicted observed SPRT / betting T132 (1) .258 5,042 642 24.9k 200k .883 .994 19 / 9 T048 (8) .0214 430 210 98.2k 100ka .375 .738 38 / 106 T027 (10) .0165 323 213 129k 200k .524 .906 38 / 106 T110 (73) .00154 31 213 1.38M 100kb <10−5<10^-5 0 not in stream (8 of 39b) / not in stream T115 (204) .00043 6 208 4.87M 200k <10−10<10^-10 0 not in stream (5 of 38) / not in stream T014 (294) .00025 3 171 6.92M 200k <10−15<10^-15 0 not in stream (2 of 35) / not in stream Table 26: Six templates of one stream (edit sequence 101, seed 0). πi _i: share of foldable queries. Epoch: 1/(νsi+1/T)1/(ν s_i+1/T), sis_i: fraction of harmful edits. One-step: share of the template’s queries answered by a folded macro, stale ones included. Bold: ours. aEdit v19v_19 (t=149,426t=149,426); retired at t=161,145t=161,145, re-certified after 77 labels at t=194,254t=194,254. bEdit v7v_7 (t=55,476t=55,476) drops the candidate’s reliability from 1.0 to .733 after 8 of 39 labels. Two real candidates show why Theorem 6 bets on the paired utility and how a plug-in break-even misleads the SPRT (Proposition 5). The replays bet on Zt=Ut/(L+7)Z_t=U_t/(L+7), clipped to [−1,1][-1,1], with Ut=(mt−1)+L(YtM−YtL)U_t=(m_t-1)+L(Y^M_t-Y^L_t) and a uniform mixture of seven fixed bets. This differs from the grid mixture of Appendix C but keeps validity; mirrored bets (W−W^-) decide discards. (a) Less accurate than live, yet worth folding. Template T096 maps a record label to its time zone and has rank 2 in seed 9 of sequence 101 (π=0.109π=0.109). Its macro forms at t=100t=100 from the trace for “Which time zone is associated with Imperial Records?”. It follows the gold path, and no edit changes its answers. Still r=0.997<a=0.998r=0.997<a=0.998, because 21 misrouted instances (mass 0.003) count as macro failures. For example, “What time zone does the headquarters of Fox Searchlight Pictures observe?” belongs to T281, but the router outputs "type": 3 and picks T096. Live, which does not route, answers correctly, so the pair enters with Ut=−65.3U_t=-65.3. The macro is still above break-even (r⋆=0.968r =0.968) and worth +1.93+1.93 steps per query. Betting folds it at t=22,459t=22,459 after 249 labels: 247 pairs with both correct, one routing loss and one joint failure. No discordant pair favors the macro, so a sign test on discordant pairs would never fold it. Betting folds on the concordant gain η/(1−η)η/(1-η) of Appendix C. All three numbers below are for the per-candidate certificate at level δ (fold threshold ln(1/δ)=ln20 (1/δ)= 20, no ledger term), the version this example ran. Lemma 16 gives KLinf(PU)=0.0244KL_ (P_U)=0.0244, i.e. ln20/KLinf=123 20/KL_ =123 labels to first order (the optimum level of Section 5); the implemented seven-bet mixture adds ln7 7 nats to the threshold and grows at the rate gℬ=0.0223g_B=0.0223 of its best bet (0.90.9), which predicts (ln20+ln7)/gℬ=221( 20+ 7)/g_B=221 (as run: the implementation loss; the overshoot term ln1.9 1.9 of Section C.2 is not included); the 249 labels it took are the measured level, 13%13\% above the as-run prediction through finite-sample effects. The deployed ledger’s j-th test adds 2lnj+ln(π2/6)2 j+ (π^2/6) nats; for j=1j=1 the first two levels become (ln20+lnπ26)/KLinf=143( 20+ π^26)/KL_ =143 and (ln20+ln7+lnπ26)/gℬ=244( 20+ 7+ π^26)/g_B=244 labels. The SPRT folds earlier (t=6,537t=6,537, 67 labels); its plug-in error is harmless here because the macro is good. (b) A plug-in break-even folds a stale macro. The T010 candidate (sequence 202, seed 1) forms at t=651t=651 from “Which time zone is associated with the administrative twin of Nottingham?”. Before the edit, r=0.965>r⋆=0.947r=0.965>r =0.947: a thin margin that needs about 686 labels, while only 11 arrive. Edit v7v_7 then gives a new time zone to 25% of the relevant subjects. Seattle, for example, moves from Pacific Time Zone to East Africa Time. The cached macro drops to r=0.597r=0.597, while live stays at a=0.977a=0.977. Betting sees 9 stale pairs with Ut=−64.4U_t=-64.4 and discards the candidate at t=59,956t=59,956 (Table 25). The SPRT froze a^=0.345 a=0.345 from one shrunk key label, so r0=0.313r_0=0.313 and r1=0.345r_1=0.345. Its zero-drift point, 0.329, lies below the true r in every phase. As in the proof of Proposition 18, it therefore folds almost surely. It folds at t=187,133t=187,133, 0.51 below break-even, and serves 45 stale answers. Across the 30 streams, the SPRT makes 21 below-break-even folds and betting 6 (SpF in Tables 3 and 5). In all 21 SPRT cases the plug-in r0r_0 lay below the true r⋆r , and betting folded none of those candidates. Takeaway. Pair the macro with live on the same instance. Routing errors and live mistakes then enter the certificate directly, and a need not be estimated. E.4 Why rare templates stay uncovered Six templates of one stream show the mechanism of Theorem 3: a template is covered only if its epoch outlasts ni⋆/(pπi)n _i/(p _i). We use edit sequence 101 and seed 0, with SPRT certification. T048, T027, T110 and T115 have matched difficulty: live solves 59 of 60 instances, the gold-path macro has r=1.000r=1.000, and ni⋆n _i is 208–213. Only their popularity ranks differ (Table 26). The head certifies within 38 labels; the tail never does. T110 forms at t=5,301t=5,301 from a correct trace for “What record labels are connected to graduates of Radcliffe College?”. It gets 8 of the 39 labels it needs before edit v7v_7 lowers its reliability to 0.733<r⋆=0.9520.733<r =0.952. T115 and T014 get 5 and 2 labels in the whole stream. In the 30 main-setting streams, the SPRT needs a median of 39 labels per fold (all 1,201 folds), fewer than the first-order ni⋆n _i (208–213 for the templates above), which assumes the least favorable candidate. Ranks 101–300 get 7.5–14.5 audits per template-stream on average, and none of their 5,771 template-streams was ever folded. Takeaway. With a fixed audit budget, templates rarer than the load are not certified between changes; coverage comes from popular templates. Appendix F Extended related work Reasoning memory and skill libraries. LLM systems increasingly store reusable reasoning: thought templates (Yang et al., 2024a), strategies and insights (Suzgun et al., 2026; Zhao et al., 2024; Ouyang et al., 2026; Zhang et al., 2026b; Didolkar et al., 2025), workflows and programmatic skills (Wang et al., 2025b; Wang et al., 2025a; Wang et al., 2024a; Wang et al., 2024c), verified lemmas (Wang et al., 2024b), and long-term memories (Zhong et al., 2024; Park et al., 2023; Chhikara et al., 2025; Li et al., 2025). Many never retire an item (Yang et al., 2024a; Ouyang et al., 2026; Wang et al., 2025b). The rest use hand-set rules. ExpeL (Zhao et al., 2024) keeps an integer importance count that starts at 2, moves by ±1± 1 on up- and down-votes and deletes the item at 0. ACE (Zhang et al., 2026b) keeps unbounded helpful/harmful tallies. Other rules include clipped Hebbian strengths with time decay (Singh and Yu, 2025; Zhu et al., 2026), lifetime failure rates thresholded at 1/21/2 (Chen et al., 2026), EWMA utilities (Zhang et al., 2026c), Beta counts (Forouzandeh et al., 2026), usage-frequency pruning (Wang et al., 2024c), time-to-live (Li et al., 2025), sliding-window utilities (Pu et al., 2026) and paired with/without-memory audits (Zhang et al., 2026e; Gao et al., 2026). MQuAKE (Zhong et al., 2023), StreamingQA and RealTime QA (Liška et al., 2022; Kasai et al., 2023) evaluate a model after knowledge edits; KQA-Stream evaluates a memory lifecycle over a stream with recurring templates and interleaved edits. The statistical analyses we know of either prove stationary consistency of a two-counter “memory worth” (Simsek, 2026) or prove that an EMA converges (Zhang et al., 2026c). Sun et al. (2025) recast memory management in decision-theoretic terms but give no tests. Asadolahi et al. (2026) show that self-graded memory scores are inflated unless verifier errors are independent. Sequential tests enter LLM pipelines only to audit trajectories (Sadhuka et al., 2025) or to stop sampling within one query (Lee et al., 2025). We show that these maintenance rules are change detectors: clipped counters are Page CUSUMs with mis-specified scores, and unclipped tallies have no renewal. We quantify their detection delay and replace them with certified folding and likelihood-ratio retirement. Step skipping versus system-level contraction. Model-level methods train an LLM to skip or internalize steps (Liu et al., 2024; Yu et al., 2024; Hao et al., 2025), to compress chains of thought (Xia et al., 2025), or to decide when to think (Zhang et al., 2025b); see Sui et al. (2025) for a survey. These methods change what one forward pass can compute. They choose training data offline. At inference time nothing decides whether a template’s one-step answer can be trusted, or notices when it has gone stale. The problem is real. Latent multi-hop composition is unreliable (Yang et al., 2024b), and consensus-based early exit is unsafe (Mo et al., 2026). Contraction is complementary. It treats any distilled routine as a macro and decides per template, over time and with explicit error control, when to serve that macro and when to retire it. Caching, routing and cascades with guarantees. Semantic caches reuse responses for similar prompts (Bang, 2023), and agent systems cache plan templates (Zhang et al., 2025c). Caching theory gives optimal content under stationary demand (Zhu et al., 2023; Liu et al., 2026). Cascades and routers decide per query (Chen et al., 2024a; Aggarwal et al., 2024). Where these systems carry guarantees, the guarantees are fixed-sample risk control under exchangeability. Examples are CALM (Schuster et al., 2022), which uses Learn-then-Test (Angelopoulos et al., 2025), PAC and group-conditional reasoning (Zeng et al., 2025; Huang et al., 2026), and conformal early stopping (Wang et al., 2026). Two works are closer. • B-PAC (Yu et al., 2026b) certifies a global thinking/non-thinking router anytime, with inverse-propensity-weighted test supermartingales. Our certificate uses the same tools (betting, inverse-propensity weights, Ville’s inequality); what differs is the object: one certificate per template, restarted at changes and followed by retirement, over a memory’s whole lifecycle. • vCache (Schroeder et al., 2026) bounds the per-request error of cache hits using randomized LLM verification. Its bound assumes i.i.d. prompts and a correct similarity model, and exploration can stop entirely. Staleness has been studied only empirically (Mansoor et al., 2026; Wu and Canedo, 2026), although model behavior does drift (Chen et al., 2024b). Anytime-valid monitors watch whole models, not reusable items (Podkopaev and Ramdas, 2022; Richter et al., 2025; Khosravi and Huo, 2026). We certify a reusable derivation with a test that stays valid at all times, under adaptive shadowing and arbitrary drift below the null. The indifference zone comes from the loss. Certification is followed by quickest-detection retirement, and the result yields population-level coverage laws. Sequential testing and change detection. Certified folding rests on the SPRT (Wald, 1945; Wald and Wolfowitz, 1948), Ville’s inequality (Ville, 1939), and testing by betting (Shafer, 2021; Ramdas et al., 2023; Howard et al., 2021; Waudby-Smith and Ramdas, 2024). Retirement rests on Page’s CUSUM (Page, 1954) and its optimality (Lorden, 1971; Moustakides, 1986; Lai, 1998). Two of our results are instances of older work. The counter identity matches cumulative-score and integer CUSUM charts (Munford, 1980; Brook and Evans, 1972). Our likelihood-ratio learning rates are the Bernoulli CUSUM (Reynolds and Stoumbos, 1999). Analyses of mis-specified CUSUM assume positive post-change drift (Molloy and Ford, 2017; Cooper and Meyn, 2024), or note only qualitatively that delay can exceed the false-alarm time (Molloy, 2021). Wu and Yang (2016, Thm. 2) state that the delay is infinite in the negative-drift regime; their proof exchanges an expectation and a minimum over the CUSUM’s restarts. Observation-controlled detection (Banerjee and Veeravalli, 2013; Xu et al., 2021) and e-detectors (Shin et al., 2024) are the natural comparisons for audited and composite settings. Using Cramér–Lundberg asymptotics (Iglehart, 1972; Siegmund, 1985; Asmussen, 2003), we make explicit the full delay–ARL phase diagram for additive-score detectors. It includes the power law D≍Aθ1/θ0D A _1/ _0 under negative drift, where Wu and Yang (2016) state an infinite delay; Appendix A locates the step of their proof that fails and gives a counterexample. Occupancy, freshness and scaling laws. Our static amortization law is a corollary of occupancy asymptotics (Karlin, 1967; Gnedin et al., 2007). Its constant follows from rare-mass results (Ohannessian and Dahleh, 2012), and Good–Turing theory estimates and forecasts it (Good, 1953; Good and Toulmin, 1956; McAllester and Schapire, 2000; Orlitsky et al., 2016). For n0=1n_0=1 it is the cold-start or characteristic-time miss ratio of caching (Fagin, 1977; Che et al., 2002; Jelenković, 1999). Admission after k hits generalizes this case (Martina et al., 2014), and memorization learning curves share the exponent (Hutter, 2021; Michaud et al., 2023). Under drift, the n0=1n_0=1 steady state is the freshness objective of crawling (Azar et al., 2018) and the hit law of TTL and random-replacement caches (Choungmo Fofack et al., 2014; Gallo et al., 2014). Square-root refresh and replication rules are classical (Yates et al., 2017; Cohen and Shenker, 2002; Kolobov et al., 2019), and so is abandoning fast-changing items (Cho and Garcia-Molina, 2003; Bastopcu and Ulukus, 2021). Square-root allocations also appear in label-efficient inference (Xu et al., 2024; Zrnic and Candès, 2024). The law of practice has been explained by chunk recurrence and by averaging over items (Newell and Rosenbloom, 1981; Logan, 1988; Heathcote et al., 2000). We couple sequential-test sample complexity with occupancy. This gives the fundamental limit of safe one-step coverage under drift, M⋆∼κsΘ1−1/sM _s ^1-1/s, with a first-order matching converse for every δ-safe policy. Safe self-improvement and multi-agent memory. Self-modifying agents accept a change when a benchmark score rises (Zhang et al., 2026a). Per-decision betting gates (Shawn, 2026) and trajectory-level error spending (Wu et al., 2025) control false commits. Earlier antecedents are high-confidence and Seldonian improvement (Thomas et al., 2015; Thomas et al., 2019; Chandak et al., 2020) and the “bio-creep” study of approval policies (Feng et al., 2021). Recent work proves limits for classifier-style gates (Scrivens, 2026) and near the reward ceiling (Cai et al., 2026). Shared multi-agent memories admit items through judges, one-shot A/B replay or count-and-mean rules (Zhang et al., 2025a; Xu et al., 2025; Yu et al., 2026a). Yet LLM errors are correlated (Kim et al., 2025), judges are noisy (Zhang et al., 2026d), and shared memory is a poisoning channel (Dong et al., 2025). These gates decide one modification or one shared item at a time. We study a single memory whose items are certified per template from verifier feedback, and leave shared and self-modifying memories to future work. Speed-up learning and the utility problem. Symbolic AI compiled multi-step reasoning into macro-operators (Fikes et al., 1972; Korf, 1985), explanation-based rules (Mitchell et al., 1986), chunks (Laird et al., 1986) and compiled productions (Anderson et al., 2004; Taatgen and Lee, 2003). Program synthesis grows libraries in the same way (Ellis et al., 2021). The utility problem (Minton, 1990) led to statistically gated adoption (Gratch and DeJong, 1992; Greiner, 1996) and PAC analyses of speed-up learning (Tadepalli and Natarajan, 1996). These analyses assume a fixed problem distribution and never un-adopt a rule. We add four things: validity at all times under adaptive schedules and drift, an indifference zone derived from the loss, retirement with optimal detection delay, and closed-form population laws for how much reasoning can be contracted.