Paper deep dive
Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents
Nadeem Shaikh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/26/2026, 5:36:07 AM
Summary
This paper introduces 'Bayesian Self-Escalation,' a framework for hierarchical LLM agents that decides mid-generation whether to delegate to a stronger model. It formulates this as a Bayesian optimal-stopping problem over a learned 'competence posterior' (an estimate of task success), rather than relying on raw entropy. The authors derive a myopic escalation threshold, prove the belief is a martingale, and establish regret bounds linking performance to posterior calibration. Empirical validation on a Qwen2.5-Coder cascade confirms that this method dominates post-hoc routing at equal cost.
Entities (6)
Relation Signals (5)
Bayesian Self-Escalation → uses → Competence Posterior
confidence 95% · We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior
Qwen2.5-Coder → validatedwith → Bayesian Self-Escalation
confidence 95% · real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade... confirming two of three pre-registered predictions
Competence Posterior → hasproperty → Martingale
confidence 93% · Theorem 1 (Martingale property). Under Assumption 1, (Bt,Ft)t≤T is a martingale
Bayesian Self-Escalation → outperforms → Post-hoc Routing
confidence 92% · the escalation frontier dominates post-hoc routing at equal cost
Competence Posterior → islearnedfrom → Labelled Trajectories
confidence 90% · sufficient statistics are learned from labelled trajectories, not read off raw entropy
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior -- an online estimate of the agent's eventual task success whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove that the optimal policy is a time-varying threshold with no shape assumption on the raw signal. We further prove exponential separation of the oracle belief at the Chernoff-information rate of the signal, a regret bound governed by the calibration of the posterior, and a finite-sample guarantee: with n labelled calibration trajectories the deployed plug-in policy's regret decays as 1/sqrt(n). A controlled simulation study confirms each prediction of the theory, including the predicted 1/sqrt(n) rate. We additionally report a real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade (MBPP, 257 tasks), confirming two of three pre-registered predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief's discrimination rises over generation.
Tags
Links
- Source: https://arxiv.org/abs/2608.24087v1
- Canonical: https://arxiv.org/abs/2608.24087v1
Trouble viewing inline? Open PDF directly →
Full Text
68,955 characters extracted from source content.
Expand or collapse full text
Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM AgentsThanks: Version 1.1. Version 1 (DOI: 10.5281/zenodo.21330788) reported no real-model results and pre-registered an evaluation protocol in their place. This version adds a first execution of that protocol on a Qwen2.5-Coder 1.51.5B→ 77B code cascade (Section 8), confirming two of its three predictions; the theoretical results are unchanged, and the constant-threshold instantiation tested does not exercise the optimal-stopping dynamic program. Nadeem Shaikh Affiliation: Independent Researcher Affiliation: Melbourne, Australia Email: nadeem@nadeemshaikh.net August 25, 2026 Abstract Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. Our contribution is not the observation that agents can defer to other agents—that is well established—but a decision-theoretic formulation of intra-generation delegation as an optimal-stopping problem over an online estimate of the agent’s eventual task success. The junior agent maintains a competence posterior: an abstract Bayesian state whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. It escalates when the expected cost of continuing exceeds the expected cost of deferral. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove—using monotonicity alone, without any concavity argument—that under a conditional-independence signal model the optimal policy is a time-varying threshold on the competence posterior, with no monotone-likelihood or other shape assumption on the raw signal. We further prove that the oracle belief separates at the Chernoff-information rate of the signal, and give a finite-sample guarantee: with n labelled calibration trajectories of length T, the deployed plug-in policy’s regret over the oracle is O(LtlogK/(nT))O (Lt K/(nT) ) with high probability. Our central message is a regret bound showing that excess cost is controlled by the calibration of that posterior: better calibration matters more than a smarter router, and “confidently wrong” predictions are the binding failure mode. A simulation study with a known data-generating process confirms each prediction of the theory. A pre-registered protocol yields falsifiable predictions; in this revised version we report a first execution of that protocol on a Qwen2.5-Coder 1.51.5B→ 77B code cascade, confirming two of its three predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief’s discrimination rises over the course of generation. The theoretical results are unchanged. Keywords: LLM agents; hierarchical inference; model cascades; optimal stopping; uncertainty quantification; calibration; selective prediction; learning to defer. 1 Introduction The economics of large language model (LLM) deployment increasingly favour hierarchies: a small, fast model handles the bulk of traffic while a larger, more capable—and considerably more expensive—model is invoked only when needed. Model cascades and routers realise this idea and can cut inference cost by large factors without sacrificing quality [1, 19, 10]. Almost all such systems, however, make the routing decision either before inference (a router inspects the query and picks a model) or after a full generation (a verifier scores the completed output and may retry). Neither option lets the working model notice, partway through a difficult chain of reasoning, that it has left the region of competence and should hand off. Many systems already let an agent hand work to another agent or to itself: self-reflection and critic/debate models, verifier-guided generation and best-of-n reranking, speculative decoding with fallback, and adaptive-compute and test-time-scaling methods all revisit or reallocate computation. What is missing across these is a principled, sequential account of when, mid-generation, to stop and defer. We do not claim to introduce agent delegation. We introduce a decision-theoretic formulation of intra-generation delegation in which escalation is an optimal-stopping problem over an online estimate of eventual task success. We refer to the mechanism as self-escalation. A point we make central rather than incidental: the framework does not assume that raw uncertainty metrics are calibrated. High entropy is not failure and low entropy is not correctness—a model can be confidently wrong. The Bayesian state is therefore an abstract competence posterior whose sufficient statistics (the mapping from a signal history to a probability of success) are learned from labelled trajectories. Per-token entropy or the next-token margin serve only as raw evidence into that learned map, never as the posterior itself. This distinction is what makes the theory meaningful, and it turns out to be the crux: our main result is that the method’s cost is governed by how well that posterior is calibrated. Contributions. 1. Formulation. We cast intra-generation self-escalation as a Bayesian optimal-stopping problem over the junior agent’s eventual success (Section 3), separating a learned competence-posterior component from the decision component. 2. Policy and theory. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove (i) that the competence posterior is a martingale, (i) that the optimal policy is a time-varying threshold—proved via monotonicity alone, with no concavity argument and no monotone-likelihood assumption on the raw signal—(i) a regret bound linking excess cost to posterior miscalibration, (iv) exponential separation of the oracle belief at the Chernoff-information rate of the signal, and (v) a finite-sample regret guarantee for the plug-in policy, decaying as 1/n1/ n in the number of labelled calibration trajectories (Sections 4–5). 3. Calibration as the binding constraint. We elevate the regret bound to a design principle: for self-escalation, improving the calibration of the competence posterior dominates improving the decision rule (Section 5.4). 4. Algorithm. We give a streaming inference algorithm and an offline backward-induction procedure for the threshold schedule, with a text-level (not hidden-state) context handoff (Section 6). 5. Simulation, protocol, and real-model validation. On a model with a known data-generating process we confirm each prediction of the theory (Section 7); we pre-register a falsifiable protocol (Section 9); and we execute a first instance of it on a Qwen2.5-Coder code cascade, confirming two of its predictions (Section 8). 2 Related Work Selective prediction and learning to defer. The option to abstain rather than predict has a long history as selective prediction or classification with a reject option [2, 5, 6]. Learning to defer generalises abstention by routing the rejected input to an external expert and learning the predictor and the deferral rule jointly [17, 18, 24]. Our setting is an instance of deferral in which the “expert” is a stronger model, but with two differences that existing formulations do not address: the deferral decision is made sequentially within a single generation rather than once per input, and the object of the posterior is the junior model’s own eventual success rather than a label. Cascades and routing for LLMs. FrugalGPT introduced learned LLM cascades that query models in increasing order of cost and stop when a scorer judges the answer adequate [1]. RouteLLM learns a router from preference data [19]; agreement- and consistency-based cascades escalate when cheap models disagree [10, 23]; and Jitkrittum et al. [8] analyse when confidence-based deferral in a cascade is and is not sufficient. These methods act at the granularity of a whole response and typically require at least one completed cheap generation (and often several samples) before deciding. Self-escalation instead consumes the token-level signal as it streams and can stop mid-generation, trading a small amount of bookkeeping for the ability to abort a doomed trajectory early. Uncertainty quantification in LLMs. A large literature estimates LLM uncertainty from output distributions. Semantic entropy clusters sampled generations by meaning and measures entropy over clusters, giving a strong hallucination signal [13, 3]; semantic-entropy probes approximate this from a single forward pass’s hidden states [11]. Most of this work targets post-hoc detection on a completed generation. We use such signals as the evidence stream of a sequential decision, and our framework is agnostic to which signal is used: token entropy, next-token margin, a probe output, or semantic entropy can all serve as the per-step observation ete_t. Self-revision and adaptive computation. A separate line of work reallocates or revisits computation within a single system. Adaptive computation time lets a network learn how many internal steps to spend per input [7]; early-exit decoding such as CALM stops a token’s computation when intermediate confidence suffices [21]; speculative decoding drafts with a cheap model and verifies with an expensive one, falling back on disagreement [14]; self-refinement iterates on a model’s own output with self-feedback [16]; and test-time-scaling studies how to allocate extra inference compute optimally [22]. These share our motivation—spend more effort only when needed—but they revise, rerank, or extend the same model’s computation rather than framing a sequential decision to hand off to a stronger model, and they generally lack an explicit stopping rule tied to a calibrated estimate of eventual task success. Relatedly, Kadavath et al. [9] show LLMs’ self-evaluations of correctness carry real signal but are imperfectly calibrated—precisely the regime where Proposition 3 says the gains of self-escalation are won or lost. Sequential testing. The oracle decision core of our problem is a finite-horizon, cost-asymmetric relative of Wald’s sequential probability ratio test, which continues sampling until the likelihood ratio exits an interval [25] and is optimal for the symmetric testing problem [26]. We flag this plainly so the reader can locate the classical core: the stopping mathematics is not new. What the classical theory does not supply—and where this paper works—is the formulation (deferral to a stronger model as the stopping action, with compute and error costs), the analysis when the likelihood ratio is estimated (Sections 5.4–5.5), and the systems instantiation. Optimal stopping. The decision component is an optimal-stopping problem [4, 20]: at each step, stop-and-defer or continue-and-observe. We use standard monotone-stopping arguments to establish the threshold structure of the optimal policy. Our framing—an optimal-stopping rule over a learned online estimate of the working model’s own eventual correctness, used to trigger escalation to a stronger model—combines these ingredients in a way we have not seen made explicit for LLM hierarchies, though we make no claim to the underlying stopping theory itself. 3 Problem Formulation Consider a junior agent J and a senior agent S. Given a query x, agent J generates a response over T steps (tokens or reasoning steps). Let Y∈0,1Y∈\0,1\ be the latent event that J’s completed answer would be correct; write π=ℙ(Y=1)π=P(Y=1) for the base competence of J on the query population. At each step t∈1,…,Tt∈\1,…,T\ the agent emits a competence-evidence signal et∈ℰe_t (a raw statistic such as normalised token entropy or the next-token margin). Let ℱt=σ(e1,…,et)F_t=σ(e_1,…,e_t) and define the competence posterior Bt=ℙ(Y=1∣ℱt).B_t\;=\;P(Y=1 _t). (1) Remark 1 (The posterior is learned, not read off entropy). We stress that ete_t is evidence, not the posterior. A raw uncertainty metric is not itself a likelihood ratio: high entropy is not failure and low entropy is not correctness, since a model can be confidently wrong. The mapping from a signal history to BtB_t—the sufficient statistics below—is learned from labelled trajectories and post-hoc calibrated (Section 6). Nothing in the theory requires the raw signal to be calibrated; it requires the fitted posterior to be. Proposition 3 makes the cost of failing this precise. Assumption 1 (Conditional independence). Conditional on Y=yY=y, the signals (et)t≤T(e_t)_t≤ T are i.i.d. with density fyf_y, and f0,f1f_0,f_1 have common support with finite log-likelihood ratio λ(e)=logf1(e)f0(e)λ(e)= f_1(e)f_0(e). In practice λ is the learned statistic of Remark 1, not a closed-form entropy transform. Assumption 1 is a modelling idealisation; real token signals are correlated and non-stationary. We adopt it to obtain a transparent update and discuss its relaxation in Section 10. Under Assumption 1 the belief evolves as a simple log-odds recursion. Lemma 1 (Belief update). Let ℓt=logBt1−Bt _t= B_t1-B_t and ℓ0=logπ1−π _0= π1-π. Then ℓt=ℓ0+∑s=1tλ(es) _t= _0+ _s=1^tλ(e_s) and Bt=σ(ℓt)B_t=σ( _t) with σ(z)=(1+e−z)−1σ(z)=(1+e^-z)^-1. Proof. By Bayes’ rule and conditional independence, ℙ(Y=1∣ℱt)ℙ(Y=0∣ℱt)=π1−π∏s=1tf1(es)f0(es) P(Y=1 _t)P(Y=0 _t)= π1-π _s=1^t f_1(e_s)f_0(e_s); taking logs gives the recursion, and inverting the log-odds gives Bt=σ(ℓt)B_t=σ( _t). ∎ Costs and actions. Let κ>0κ>0 be the compute cost per junior step, γ>0γ>0 the one-off cost of escalation (senior compute plus added latency), and L>0L>0 the cost of a wrong final answer. The senior returns a correct answer with probability q∈(0,1)q∈(0,1), assumed (for the base model) independent of ℱtF_t. At each step the agent chooses an action at∈continue,escalatea_t∈\ continue, escalate\. Escalation is terminal and hands S the query together with a distilled context; if the agent never escalates it answers locally at step T. A policy ρ maps ℱtF_t to actions; we seek ρ minimising the expected total cost (ρ)=[κτstop⏟junior compute+γ 1escalate⏟deferral+L 1final answer wrong⏟error],C(ρ)=E\! [ κ\, _stop_junior compute+ γ\,1\escalate\_deferral+ L\,1\final answer wrong\_error ], (2) where τstop _stop is the number of junior steps taken before stopping. 4 The Escalation Policy 4.1 Myopic threshold Consider the decision at step t with belief b=Btb=B_t, comparing immediate escalation against finishing locally. The forward cost of escalating is γ+(1−q)Lγ+(1-q)L (sunk junior steps are ignored). The forward cost of continuing to completion is (T−t)κ+(1−b)L(T-t)κ+(1-b)L. Escalation is preferred iff γ+(1−q)L<(T−t)κ+(1−b)L.γ+(1-q)L\;<\;(T-t)κ+(1-b)L. (3) Rearranging isolates a belief threshold. Proposition 1 (Myopic escalation threshold). Under the “finish locally” comparison, escalate at step t iff Bt<τtmyoB_t< _t^myo, where τtmyo=q−γL+(T−t)κL. _t^myo\;=\;q- γL+ (T-t)κL. (4) In particular τTmyo=q−γ/L _T^myo=q-γ/L, and τtmyo _t^myo decreases in t. Equation (4) is interpretable: one escalates more readily when the senior is reliable (large q), when errors are costly (large L), and when escalation is cheap (small γ). The term (T−t)κ/L(T-t)κ/L says that with many tokens still to pay for, local completion looks relatively expensive, nudging the threshold up. 4.2 Optimal stopping The myopic rule ignores the option value of continuing: another token yields fresh evidence that may resolve the uncertainty without paying for escalation. The optimal policy is the solution of the dynamic program with value function Vt(b)V_t(b) equal to the minimal expected cost-to-go at step t with belief b: VT(b) V_T(b) =minγ+(1−q)L,(1−b)L, = \γ+(1-q)L,\;(1-b)L \, (5) Vt(b) V_t(b) =minγ+(1−q)L⏟escalate,κ+[Vt+1(Bt+1)|Bt=b]⏟continue,t<T, = \\; γ+(1-q)L_escalate,\; κ+E [V_t+1(B_t+1)\, |\,B_t=b ]_continue\; \, t<T, (6) where the transition is the Bayes update of Lemma 1 driven by the posterior-predictive signal et+1∼bf1+(1−b)f0e_t+1 b\,f_1+(1-b)f_0. The optimal policy escalates at t iff the escalate branch attains the minimum in (6). 5 Theoretical Analysis 5.1 The belief is a martingale Theorem 1 (Martingale property). Under Assumption 1, (Bt,ℱt)t≤T(B_t,F_t)_t≤ T is a martingale: [Bt+1∣ℱt]=BtE[B_t+1 _t]=B_t. Proof. Bt=ℙ(Y=1∣ℱt)=[Y=1∣ℱt]B_t=P(Y=1 _t)=E[1\Y=1\ _t]. By the tower property, [Bt+1∣ℱt]=[[Y=1∣ℱt+1]∣ℱt]=[Y=1∣ℱt]=BtE[B_t+1 _t]=E[E[1\Y=1\ _t+1] _t]=E[1\Y=1\ _t]=B_t. ∎ Theorem 1 is often misread. It does not say waiting is worthless. Because the belief is a martingale, waiting does not increase the expected belief—but it can increase expected decision quality by revealing information, and the value of that information comes precisely from the nonlinearity of the future value [V(Bt+1)]E[V(B_t+1)]. A stock price is a martingale yet options on it have value; likewise here, the option to continue has value even though [Bt+1∣ℱt]=BtE[B_t+1 _t]=B_t. What the martingale property does give us is a clean structural handle for the monotone-stopping argument below. 5.2 Consistency and separation rate How quickly does the oracle belief become decisive? The answer is governed by a single scalar property of the signal pair: its Chernoff information. Proposition 2 (Exponential belief separation). Assume a finite signal alphabet and let C=−logmin∑ks∈[0,1]f0(k)sf1(k)1−sC\;=\;- \ _s∈[0,1]\ _kf_0(k)^sf_1(k)^1-s (7) be the Chernoff information of (f0,f1)(f_0,f_1), with C>0C>0 iff f0≠f1f_0≠ f_1. Then for every fixed τ∈(0,1)τ∈(0,1) there is a constant Aτ<∞A_τ<∞, independent of t, such that ℙ(Bt≤τ∣Y=1)≤Aτe−tC,ℙ(Bt≥τ∣Y=0)≤Aτe−tC.P (B_t≤τ Y=1 )\;≤\;A_τ\,e^-tC, (B_t≥τ Y=0 )\;≤\;A_τ\,e^-tC. Consequently Bt→Y=1B_t 1\Y=1\ almost surely, and the oracle’s step-t thresholded decision errs with probability at most Aτe−tCA_τe^-tC. Proof. Bt≤τB_t≤τ iff ∑s≤tλ(es)≤c _s≤ tλ(e_s)≤ c with c=logτ1−τ−logπ1−πc= τ1-τ- π1-π. For any s>0s>0, the Chernoff bound gives ℙ(∑λ≤c∣Y=1)≤esc(f1[e−sλ])t=esc(∑kf1(k)1−sf0(k)s)tP (Σλ≤ c Y=1 )≤ e^sc (E_f_1[e^-sλ] )^t=e^sc ( _kf_1(k)^1-sf_0(k)^s )^t. Minimising the base over s∈[0,1]s∈[0,1] yields e−Ce^-C per step, with Aτ=es⋆cA_τ=e^s c at the minimiser s⋆s . The Y=0Y=0 side is symmetric with f0[esλ]=∑kf0(k)1−sf1(k)sE_f_0[e^sλ]= _kf_0(k)^1-sf_1(k)^s, whose minimum over s∈[0,1]s∈[0,1] is the same e−Ce^-C. Almost-sure convergence follows from Borel–Cantelli. ∎ The practical reading: the number of tokens the oracle needs before its escalation decision is reliable at level α is t≳log(Aτ/α)/Ct (A_τ/α)/C. “How informative is this uncertainty signal?” is thus answered by one number, computable from the fitted class-conditionals, and comparable across candidate signals (entropy vs. margin vs. probe output) before any policy is built. 5.3 Optimality of a threshold policy The key structural fact is that a larger current belief makes the next belief stochastically larger. This is the step a careful reader will demand a proof of, so we give one. Notably, no monotone-likelihood-ratio or other shape assumption on the raw signal is needed: the Bayes update depends on e only through its log-likelihood ratio λ(e)λ(e), and the law of the likelihood ratio under f1f_1 stochastically dominates its law under f0f_0 for any pair of densities. Related belief-monotonicity results appear in the partially observed MDP literature [15, 12]. Lemma 2 (FOSD-monotone belief transition). Under Assumption 1 alone, the map b↦Law(Bt+1∣Bt=b)b (B_t+1 B_t=b) is non-decreasing in the first-order stochastic dominance (FOSD) order: for b≤b′b≤ b and every non-decreasing h:[0,1]→ℝh:[0,1] , [h(Bt+1)∣Bt=b]≤[h(Bt+1)∣Bt=b′]E[h(B_t+1) B_t=b] [h(B_t+1) B_t=b ]. Proof. Write the update as Bt+1=σ(logb1−b+Λ)B_t+1=σ\! ( b1-b+ ) where Λ=λ(e) =λ(e) and the incoming signal has posterior-predictive density mb=bf1+(1−b)f0m_b=bf_1+(1-b)f_0. Three steps. (i) The law of Λ under f1f_1 FOSD-dominates its law under f0f_0. Let R=f1(e)/f0(e)R=f_1(e)/f_0(e), so Λ=logR = R and it suffices to prove the claim for R. For any c≥1c≥ 1: ℙf1(R≥c)=f0[R 1R≥c]≥cℙf0(R≥c)≥ℙf0(R≥c)P_f_1(R≥ c)=E_f_0[R\,1\R≥ c\]≥ c\,P_f_0(R≥ c) _f_0(R≥ c). For any c<1c<1: ℙf1(R<c)=f0[R 1R<c]≤cℙf0(R<c)≤ℙf0(R<c)P_f_1(R<c)=E_f_0[R\,1\R<c\]≤ c\,P_f_0(R<c) _f_0(R<c), hence again ℙf1(R≥c)≥ℙf0(R≥c)P_f_1(R≥ c) _f_0(R≥ c). No assumption on f0,f1f_0,f_1 beyond common support was used. (i) The law of Λ under mbm_b is FOSD-non-decreasing in b. For non-decreasing u, mb[u(Λ)]=bf1[u(Λ)]+(1−b)f0[u(Λ)]E_m_b[u( )]=b\,E_f_1[u( )]+(1-b)\,E_f_0[u( )] is affine in b with slope f1[u]−f0[u]≥0E_f_1[u]-E_f_0[u]≥ 0 by (i). (i) Chaining. Bt+1B_t+1 is non-decreasing in b for fixed Λ and non-decreasing in Λ for fixed b. For non-decreasing h, the map Λ↦h(σ(logb1−b+Λ)) h (σ( b1-b+ ) ) is non-decreasing, so mb′[h(Bt+1(b′,Λ))]≥mb′[h(Bt+1(b,Λ))]≥mb[h(Bt+1(b,Λ))],E_m_b \! [h(B_t+1(b , )) ]\;≥\;E_m_b \! [h(B_t+1(b, )) ]\;≥\;E_m_b\! [h(B_t+1(b, )) ], the first inequality by pointwise monotonicity in b, the second by (i). ∎ Remark 2 (Signal orientation). For interpretability one typically wants λ monotone in e (lower entropy ⇒ evidence for success), and our simulations use such signals; but the theory does not require it, since only the induced law of λ(e)λ(e) enters the update. Theorem 2 (Threshold structure). Under Assumption 1, for each t there exists a threshold τt⋆∈[0,1] _t ∈[0,1] such that the optimal policy escalates at step t iff Bt≤τt⋆B_t≤ _t . At the horizon, τT⋆=q−γ/L _T =q-γ/L. Proof. The argument uses monotonicity only; we never invoke concavity, so no concavity-preservation step is required. We show by backward induction that each VtV_t is non-increasing in b. Base case. VT(b)=(1−b)LV_T(b)=(1-b)L is non-increasing. Inductive step. Suppose Vt+1V_t+1 is non-increasing. The escalate value γ+(1−q)Lγ+(1-q)L is a constant, hence non-increasing. For the continue value Ct(b)=κ+[Vt+1(Bt+1)∣Bt=b]C_t(b)=κ+E[V_t+1(B_t+1) B_t=b], Lemma 2 gives that b↦Law(Bt+1∣b)b (B_t+1 b) is non-decreasing in the FOSD order. Since Vt+1V_t+1 is non-increasing, its expectation against an FOSD-larger law is smaller, so CtC_t is non-increasing in b. As the pointwise minimum of two non-increasing functions, Vt=minγ+(1−q)L,CtV_t= \γ+(1-q)L,C_t\ is non-increasing, closing the induction. Threshold structure. Escalation is optimal at b iff γ+(1−q)L≤Ct(b)γ+(1-q)L≤ C_t(b). Because CtC_t is non-increasing, this set is a lower interval [0,τt⋆][0, _t ] with τt⋆=supb:γ+(1−q)L≤Ct(b) _t = \b:γ+(1-q)L≤ C_t(b)\ (and ∅ read as τt⋆=0 _t =0), which is exactly a threshold rule. Terminal threshold. At t=Tt=T there is no continuation: the choice is escalate at cost γ+(1−q)Lγ+(1-q)L or answer at expected cost (1−b)L(1-b)L (Eq. (5)). Escalation is optimal iff γ+(1−q)L≤(1−b)Lγ+(1-q)L≤(1-b)L, i.e. b≤q−γ/Lb≤ q-γ/L, matching the myopic value of Proposition 1. ∎ Remark 3 (Why monotonicity, not concavity). An earlier route argues VtV_t is concave and reads off the threshold from concavity. That route is defensible—the pointwise minimum of concave functions is in fact concave—but the concavity-preservation step under the Bayesian update is delicate and invites attack. The monotonicity proof above needs strictly less (only Lemma 2) and yields the same threshold conclusion, so we prefer it. Remark 4 (Thresholds need not be monotone in t). An earlier draft claimed τ1⋆≤⋯≤τT⋆ _1 ≤…≤ _T . That claim is false in general, and we retract it. Counterexample: take κ>γ+(1−q)Lκ>γ+(1-q)L, so a single further token costs more than full escalation. Then continuing is never optimal before the horizon and τt⋆=1 _t =1 for all t<Tt<T, while τT⋆=q−γ/L<1 _T =q-γ/L<1: thresholds decrease. The direction of the schedule reflects a tug-of-war between two forces—remaining-token costs (which favour escalating early, pushing early thresholds up) and the option value of information (which favours continuing early, pushing early thresholds down)—and which force wins depends on (κ,γ,L,q)(κ,γ,L,q) and the informativeness of the signal. In the small-κ regime of our simulation the option value dominates and the computed interior schedule is low and gently increasing (Section 7); in that regime the myopic rule, whose threshold (4) is largest early, over-escalates at the start of generation (Table 1 quantifies this). Nor is non-monotonicity confined to the κ>γ+(1−q)Lκ>γ+(1-q)L boundary: re-running the backward induction of Section 7 with κ=0.02κ=0.02 (all else unchanged) yields a schedule of ≈1.0≈ 1.0 for t≤20t≤ 20, dipping to ≈0.20≈ 0.20 near t=37t=37 and rising again to ≈0.30≈ 0.30 by t=39t=39—grossly non-monotone in the interior. 5.4 Calibration is the binding constraint We regard the following as the paper’s central practical message. The decision rule of Theorem 2 is optimal given the posterior, but in deployment the posterior is estimated, inducing beliefs B^t B_t that may differ from the true BtB_t. The next bound shows the excess cost is controlled entirely by that gap—so, for self-escalation, effort spent improving the calibration of the competence posterior dominates effort spent on a more elaborate router or decision rule. Proposition 3 (Miscalibration regret). Fix step t and a threshold τ. Let the realised decision (continue to completion vs. escalate) use B^t B_t and the oracle decision use the true BtB_t, both thresholding at τ. Let gt(b)=(T−t)κ+(1−b)L−γ−(1−q)Lg_t(b)=(T-t)κ+(1-b)L-γ-(1-q)L denote the forward-cost difference (continue minus escalate) at true belief b, and let D=Bt,B^t on opposite sides of τD=\B_t, B_t on opposite sides of τ\ be the disagreement event. Then Regrett≤L[|Bt−B^t|]+|gt(τ)|ℙ(D).Regret_t\;≤\;L\,E [\,|B_t- B_t|\, ]\;+\;|g_t(τ)|\;P(D). (8) In particular, at the myopic threshold τ=τtmyoτ= _t^myo of Proposition 1, where gtg_t vanishes, Regrett≤L[|Bt−B^t|].Regret_t\;≤\;L\,E [\,|B_t- B_t|\, ]. (9) Proof. Excess cost is incurred only on D, where it equals |gt(Bt)||g_t(B_t)|. The function gtg_t is affine with slope −L-L, so |gt(Bt)|≤|gt(Bt)−gt(τ)|+|gt(τ)|=L|Bt−τ|+|gt(τ)||g_t(B_t)|≤|g_t(B_t)-g_t(τ)|+|g_t(τ)|=L|B_t-τ|+|g_t(τ)|. On D the threshold τ lies between BtB_t and B^t B_t, hence |Bt−τ|≤|Bt−B^t||B_t-τ|≤|B_t- B_t|. Taking expectations over D and bounding [D|Bt−B^t|]≤|Bt−B^t|E[1_D|B_t- B_t|] |B_t- B_t| gives (8); gt(τtmyo)=0g_t( _t^myo)=0 gives (9). ∎ Remark 5 (The threshold restriction is essential). An earlier draft asserted the bound (9) for arbitrary τ; that claim is false, and we retract it. Counterexample (at the horizon, with the costs of Section 7): take τ=0.5τ=0.5, Bt=0.49B_t=0.49, B^t=0.51 B_t=0.51. The decisions disagree and the realised regret is gT(0.49)=0.51−0.25=0.26g_T(0.49)=0.51-0.25=0.26, while L|Bt−B^t|=0.02L|B_t- B_t|=0.02. The extra |gt(τ)|ℙ(D)|g_t(τ)|\,P(D) term in (8) is exactly the price of operating at a threshold where the two actions are not cost-indifferent; it vanishes at τtmyo _t^myo and is small near it. Proposition 3 formalises the “confidently wrong” failure mode: a query with Y=0Y=0 whose signals mimic success drives B^t B_t high while the true BtB_t is low, producing an O(L)O(L) regret event precisely when it is most costly. Two consequences are worth stating plainly. First, confidently wrong predictions are the central failure mode of self-escalation: no threshold policy on a miscalibrated signal can recover the lost escalations. Second, the bound is a directive for practitioners—measure and minimise |Bt−B^t|E|B_t- B_t| (via reliability diagrams and post-hoc calibration) before tuning thresholds, because thresholds cannot compensate for a miscalibrated posterior. Section 7 measures this dependence directly. The bound as stated involves the unobservable oracle belief BtB_t. It can be connected to measurable calibration quantities in both directions. Corollary 1 (Excess Brier score controls regret). Let BS(Z)=[(Y−Z)2]BS(Z)=E[(Y-Z)^2] denote the Brier score of a [0,1][0,1]-valued, ℱtF_t-measurable predictor Z. At the myopic threshold τ=τtmyoτ= _t^myo, Regrett≤LBS(B^t)−BS(Bt).Regret_t\;≤\;L\, BS( B_t)-BS(B_t). (10) Proof. Since Bt=[Y∣ℱt]B_t=E[Y _t] and B^t B_t is ℱtF_t-measurable, the cross term vanishes in [(Y−B^t)2]=[(Y−Bt)2]+[(Bt−B^t)2]E[(Y- B_t)^2]=E[(Y-B_t)^2]+E[(B_t- B_t)^2] (the calibration–refinement decomposition), so [(Bt−B^t)2]=BS(B^t)−BS(Bt)E[(B_t- B_t)^2]=BS( B_t)-BS(B_t). Combine with Proposition 3 via |Bt−B^t|≤[(Bt−B^t)2]E|B_t- B_t|≤ E[(B_t- B_t)^2] (Jensen). ∎ Corollary 1 turns the abstract bound into a training objective: BS(Bt)BS(B_t) is a fixed property of the signal, so minimising the Brier score of the fitted posterior directly minimises the regret bound. This is why the protocol in Section 9 reports Brier score as a primary metric rather than a diagnostic afterthought. Remark 6 (ECE is necessary but not sufficient). The (L1L_1) expected calibration error satisfies ECE(B^t)=|[Y∣B^t]−B^t|=|[Bt−B^t∣B^t]|≤|Bt−B^t|ECE( B_t)=E |E[Y B_t]- B_t |=E |E[B_t- B_t B_t] | |B_t- B_t| by the tower property and Jensen. So ECE lower-bounds the quantity that drives regret: a large ECE certifies a problem, but a small ECE does not certify safety, because B^t B_t can be perfectly calibrated on average while ignoring information in ℱtF_t (poor refinement). Brier score, which penalises both calibration and refinement, is the right target; ECE is the right alarm. 5.5 Sample complexity of calibrated self-escalation The bounds above take the fitted posterior as given. We now close the loop: how much labelled calibration data buys how much regret? The following gives a complete, finite-sample answer for the plug-in estimator on a discretised signal (discretisation is standard in implementations; K is the number of bins). Theorem 3 (Finite-sample regret of the plug-in policy). Assume a finite signal alphabet of size K with fy(k)≥εf_y(k)≥ for all k,yk,y, and a labelled calibration set containing mym_y token observations of class y; let m=min(m0,m1)m= (m_0,m_1) and suppose m≥(2/ε2)log(4K/δ)m≥(2/ ^2) (4K/δ). Let λ λ be the plug-in log-likelihood ratio of the empirical bin frequencies, with the prior π known, and let B^t B_t be the resulting beliefs. Then with probability at least 1−δ1-δ over the calibration set, simultaneously for all t≤Tt≤ T, [|B^t−Bt|]≤tεlog(4K/δ)2m,and hence, at the myopic threshold,Regrett≤Ltεlog(4K/δ)2m.E [\,| B_t-B_t|\, ]\;≤\; t (4K/δ)2m, hence, at the myopic threshold, _t\;≤\; L\,t (4K/δ)2m. (11) Proof. Hoeffding’s inequality gives ℙ(|f^y(k)−fy(k)|≥u)≤2e−2myu2P (| f_y(k)-f_y(k)|≥ u )≤ 2e^-2m_yu^2 per bin and class; a union bound over the 2K2K pairs with u=log(4K/δ)/(2m)u= (4K/δ)/(2m) leaves failure probability at most δ. On the success event, the hypothesis on m gives u≤ε/2u≤ /2, so f^y(k)≥ε/2 f_y(k)≥ /2 and, since x↦logx x is (2/ε)(2/ )-Lipschitz on [ε/2,∞)[ /2,∞), |logf^y(k)−logfy(k)|≤2u/ε| f_y(k)- f_y(k)|≤ 2u/ for each class, hence ‖λ^−λ‖∞≤4u/ε\| λ-λ\|_∞≤ 4u/ . The log-odds error after t updates is at most t⋅4u/εt· 4u/ (π known), and σ is 14 14-Lipschitz, so |B^t−Bt|≤tu/ε| B_t-B_t|≤ tu/ pointwise on the success event, hence also in expectation over trajectories. Combining with Proposition 3 at the myopic threshold gives the regret bound. ∎ Three remarks, in decreasing order of comfort. First, the rate: regret decays as O(LtlogK/(nT)/ε)O (Lt K/(nT)\,/ ) when the calibration set consists of n trajectories of length T (so m≈nTmin(π,1−π)m≈ nT (π,1-π)); every trajectory contributes T token observations, which is why modest labelled sets suffice in practice. Second, the linear-in-t compounding is a worst case of the plug-in construction; recalibrating B^t B_t directly at each t (e.g. isotonic regression per step) targets |B^t−Bt|E| B_t-B_t| without the compounding and is what we recommend in deployment. Third, the 1/ε1/ dependence is pessimistic: it charges for accuracy on low-mass bins that belief trajectories near the threshold rarely visit; a margin-weighted refinement is left to future work. Section 7 verifies the theorem’s driver empirically: the belief error decays at the predicted 1/n1/ n rate, while the realised cost gap sits far below the bound. 6 Algorithm Algorithm 1 is the streaming inference procedure: a single junior forward pass, an O(1)O(1) belief update per token, and an early exit when the belief crosses the schedule. Algorithm 2 computes the threshold schedule τt⋆\ _t \ once, offline, by backward induction on a discretised belief grid, using the calibrated likelihood ratio to Monte-Carlo the transition. Context handoff is text, not activations. The junior and senior are in general different models—different architectures, tokenizers, and KV-cache layouts—so the senior cannot ingest the junior’s hidden state. DistillContext therefore produces text-level artifacts: the partial reasoning trace generated so far, any scratchpad or intermediate results, the tool-call history and their returns, and retrieved evidence. This is a prompt the senior can consume directly, and it keeps the handoff model-agnostic. It also means the escalation cost γ should include the tokens re-read by the senior, which the protocol in Section 9 measures. Algorithm 1 Bayesian Self-Escalation (inference time) 1: query x; junior J; senior S; thresholds τt\ _t\; ratio λ; prior log-odds ℓ0 _0 2: ℓ←ℓ0 ← _0 3: for t=1t=1 to T do 4: (tokent,et)←J.step(x)(token_t,e_t)← J. step(x) ⊳ ete_t: token entropy / margin / probe output 5: ℓ←ℓ+λ(et) ← +λ(e_t); B←σ(ℓ)B←σ( ) ⊳ O(1)O(1) belief update 6: if B<τtB< _t then 7: c←DistillContext(trace1:t,tool_state,evidence)c← DistillContext(trace_1:t,\ tool\_state,\ evidence) 8: return S.solve(x,c)S. solve(x,c) ⊳ escalate; hand off text-level artifacts, not hidden state 9: end if 10: end for 11: return J.finalize()J. finalize() ⊳ answer locally Algorithm 2 Offline threshold schedule (backward induction) 1: calibrated λ; costs (κ,γ,L,q)(κ,γ,L,q); grid bjj=1m\b_j\_j=1^m; samples e(k)\e^(k)\ 2: V(bj)←minγ+(1−q)L,(1−bj)LV(b_j)← \γ+(1-q)L,\ (1-b_j)L\ for all j ⊳ terminal value, Eq. (5) 3: for t=T−1t=T-1 down to 11 do 4: for each grid point bjb_j do 5: propagate ℓj=logbj1−bj _j= b_j1-b_j by λ(e(k))λ(e^(k)) under both classes; form next beliefs b′(k)b (k) 6: C(bj)←κ+bjV(bY=1′(k))¯+(1−bj)V(bY=0′(k))¯C(b_j)←κ+b_j\, V(b (k)_Y=1)+(1-b_j)\, V(b (k)_Y=0) 7: V(bj)←minγ+(1−q)L,C(bj)V(b_j)← \γ+(1-q)L,\;C(b_j)\ 8: end for 9: τt⋆←maxbj:γ+(1−q)L≤C(bj) _t ← \b_j:γ+(1-q)L≤ C(b_j)\ 10: end for 11: return τt⋆\ _t \ Fitting the competence posterior. The likelihood ratio λ (or, equivalently, a direct map from signal history to BtB_t) is fit offline on a labelled development set: run J on queries with known correctness Y, collect signal trajectories e1:Te_1:T, and either (i) estimate the class-conditional densities f0,f1f_0,f_1 parametrically or by kernel methods and take their log-ratio, or (i) fit a logistic model mapping cumulative signal features (running mean entropy, margin trend, spike counts) to ℙ(Y=1)P(Y=1), which sidesteps density estimation. Either way the resulting beliefs are then calibrated post hoc (isotonic regression or temperature scaling), targeting the Brier score per Corollary 1. Reliability diagrams of B^t B_t against empirical success at several t are the basic sanity check. Why a posterior plus thresholds, not a directly learned policy? One could instead train a classifier that maps signals straight to escalate/continue. We prefer the factored design for three reasons. First, modularity under changing costs: the fitted posterior depends only on the model and signal, while (κ,γ,L,q)(κ,γ,L,q) enter only through Algorithm 2; when prices, latency budgets, or the senior model change, one reruns a cheap backward induction instead of recollecting labels and retraining. Second, auditability: B^t B_t is an interpretable monitoring statistic (“the agent currently believes it has a 22% chance of being right”), useful for logging and human oversight independent of the routing decision. Third, statistical efficiency: the posterior is learned from all trajectories, whereas a direct policy gradient sees the cost signal only at decision boundaries. The price is model misspecification risk in the belief update, which Section 7 probes directly. Theory versus production instantiation. The explicit Bayesian filter is the analyzable idealisation; in production we expect a learned success predictor B^t=fθ(signal history) B_t=f_θ(signal history) to replace it while preserving the decision structure. A learned predictor handles correlated, non-stationary signals that violate Assumption 1 and can ingest hidden-state features directly. The division of labour under this swap is clean: the regret analysis (Section 5.4) is filter-agnostic and becomes the contract the learned predictor must satisfy—minimise Brier score, verify calibration, then trust the thresholds—while the threshold-structure guarantee (Theorem 2) is what is formally lost, since an arbitrary learned predictor need not inherit the FOSD transition. Pragmatically one thresholds anyway, computing the schedule by running Algorithm 2 on empirical belief transitions from development trajectories rather than the analytic ones. We caution against going one step further and learning the escalate/continue policy end-to-end: that forfeits cost modularity (price changes then require retraining rather than a cheap backward induction) and the auditability of B^t B_t as a monitoring statistic. Overhead and task adaptivity. At inference the update is an O(1)O(1) table lookup (or tiny MLP evaluation) per token, negligible next to a transformer forward pass; all expensive work (fitting, calibration, backward induction) is offline. The framework also adapts across task types without refitting the signal model: per-domain priors πd _d and per-domain costs yield per-domain threshold schedules from the same fitted λ, again via Algorithm 2 alone. 7 Simulation Study We call this a simulation study rather than an experiment, and we are candid about what it can and cannot show. Because we specify the data-generating process, this is a world in which the modelling assumptions hold by construction; it is a check that the derived policy behaves as the theory predicts, and a diagnostic of how it degrades when an assumption is violated. It is emphatically not evidence about real LLM token dynamics—that is the role of the protocol in Section 9. All numbers come from a single reproducible script (fixed seed). Setup. We draw Y∼Bernoulli(π)Y (π) with π=0.60π=0.60 and, per token (T=40T=40), a signal et∼Beta(2,4)e_t (2,4) if Y=1Y=1 and et∼Beta(4,2)e_t (4,2) if Y=0Y=0 (the signal enters the update only through its likelihood ratio, so no shape assumption is needed; Lemma 2). The senior succeeds with q=0.90q=0.90. Costs are L=1L=1, κ=0.002κ=0.002 per token, γ=0.15γ=0.15. We evaluate on N=40,000N=40,000 queries. We compare: junior-only; senior-only; fixed-rule (escalate at the first token with et>θe_t>θ); selective (generate fully, escalate if final confidence <τ<τ, a post-hoc baseline that always pays full local generation); Bayesian myopic schedule (the literal, parameter-free rule of Eq. (4)); Bayesian constant threshold (escalate at the first token with Bt<τB_t<τ for a constant τ, swept); and Bayesian optimal-stopping (Algorithm 2, parameter-free). Cost per query counts junior tokens plus escalation; accuracy is the fraction of correct final answers. Disclosure: the fixed-rule, selective, and constant-threshold policies each have one free parameter, which is swept and reported at the compute-matched point; the myopic-schedule and optimal-stopping rows involve no tuning. Results. Figure 1 shows the cost–accuracy frontier. The Bayesian frontier dominates both baselines: for any compute budget it attains higher accuracy, and the optimal-stopping operating point (star) sits above and to the left of unconditional escalation. Table 1 reports a matched-compute slice at ≈0.11≈ 0.11 cost/query. The Bayesian policy reaches 96.0%96.0\% accuracy while escalating on only 40%40\% of queries, versus 91.0%91.0\% for the fixed rule and 90.1%90.1\% for always escalating to the senior; it also matches the accuracy of the post-hoc selective baseline at lower cost, because it can abort early rather than always completing the local generation. That the number nominally exceeds senior-only accuracy should be given no weight: it is an artifact of the constant-q assumption (the senior’s q=0.90q=0.90 does not degrade with query difficulty, so keeping the junior’s confidently-correct easy cases mechanically lifts the mixture). With a realistic difficulty-dependent q(x)q(x) the effect shrinks and can vanish. Throughout, the meaningful comparison is the ordering and spacing of policies at matched cost, not any absolute margin over the senior; Section 10 returns to this. Two further honest readings of Table 1. First, the literal myopic schedule performs poorly: its first-token threshold τ1myo=0.828 _1^myo=0.828 exceeds the prior π=0.60π=0.60, so it escalates 62%62\% of queries at the very first token (68%68\% overall), reaching only 0.9340.934 accuracy at higher cost (0.1290.129)—a concrete demonstration of the early over-escalation predicted by Remark 4. The strong “Bayesian, constant threshold” row is a tuned rule, not Proposition 1; the untuned policy that performs well is the optimal-stopping schedule. Second, the matched-compute slice flatters the Bayesian–fixed-rule gap: at slightly higher compute the fixed rule nearly catches up (θ=0.85θ=0.85 gives 0.9560.956 accuracy at cost 0.1180.118). The robust claim is that the Bayesian frontier weakly dominates everywhere (Figure 1); the size of the point gap depends on where the budget lands on the fixed rule’s steep region. Table 1: Matched-compute comparison (≈0.11≈ 0.11 cost/query) on the simulation model. Accuracy is fraction correct; “esc.” is escalation rate. Rows marked “tuned” sweep one operating parameter and are reported at the compute-matched point; the myopic-schedule and optimal-stopping rows are parameter-free. Numbers are produced by the accompanying simulation. Policy Accuracy Compute/query Esc. rate Junior only 0.6010.601 0.0800.080 0.000.00 Senior only 0.9010.901 0.1500.150 1.001.00 Fixed-rule (entropy, tuned) 0.9100.910 0.1140.114 — Selective (post-hoc, tuned) 0.9590.959 0.1400.140 — Bayesian, myopic schedule (Eq. (4)) 0.9340.934 0.1290.129 0.680.68 Bayesian, constant threshold (tuned) 0.9580.958 0.1110.111 — Bayesian, optimal-stopping 0.9600.960 0.1110.111 0.400.40 Figure 1: Cost–accuracy frontier on the simulation model. The Bayesian policies (teal) dominate the fixed-rule (orange) and post-hoc selective (grey) baselines; the optimal-stopping point (red star) beats unconditional escalation on both axes. Belief dynamics and the threshold schedule. Figure 2 plots posterior trajectories: beliefs for eventual successes drift up and for eventual failures drift down, so a threshold cleanly separates them within a few tokens. The computed schedule illustrates Remark 4’s small-κ regime: interior thresholds stay low (rising gently from 0.020.02 to 0.080.08 across the generation)—because escalating one step later costs only κ=0.002κ=0.002 while buying another observation, the option value of waiting keeps interior escalation conservative—and the threshold jumps to the analytic terminal value q−γ/L=0.75q-γ/L=0.75 exactly at the horizon, where no further information can arrive. The myopic schedule (4), by contrast, is highest early; in this regime it over-escalates at the start of generation. Figure 2: Posterior success-belief trajectories BtB_t for twelve queries. Eventual successes (teal) separate from eventual failures (orange) within a few tokens; the dashed line is an illustrative threshold. Calibration sensitivity. To probe Proposition 3 we contaminate the stream with “confidently wrong” queries: a fraction of Y=0Y=0 cases whose signals are drawn from the success distribution. Figure 3 shows accuracy falling from 95.2%95.2\% at 0%0\% contamination to 85.7%85.7\% at 30%30\%, while the escalation rate drops (from 0.470.47 to 0.370.37) because the contaminated cases look confident and are wrongly kept local. This is exactly the O(L)O(L) regret event of Proposition 3 and underlines that belief calibration, not the decision rule, is the binding constraint. Figure 3: Sensitivity to signal miscalibration. As confidently-wrong contamination grows, accuracy degrades and—counter-productively—the escalation rate falls, because miscalibrated confidence suppresses the very escalations that are needed. Sample-complexity check. To test Theorem 3’s driver, we discretise the signal into K=20K=20 bins, fit the plug-in λ λ from n labelled trajectories (add-one smoothing), and compare against the exact binned oracle on a fixed evaluation set, averaging over 3030 calibration resamples per n. The belief error |B^10−B10|E| B_10-B_10| falls from 2.9×10−32.9× 10^-3 at n=25n=25 to 2.1×10−42.1× 10^-4 at n=3200n=3200, with log–log slope −0.53-0.53—matching the predicted 1/n1/ n rate. The realised end-to-end cost gap is already below 0.00160.0016 (about 1%1\% of total cost) at n=25n=25 and sits far under the bound at every n: with a well-separated signal, beliefs rarely linger near the threshold, so estimation errors rarely flip decisions. This is the theorem’s pessimism working as intended—the bound is a worst-case guarantee, and the practical message is that modest labelled sets suffice when the signal is informative. Robustness across observation models. To check that the conclusions are not an artifact of the Beta observation family, we repeat the comparison under two further signal models (Table 2): (i) Gaussian class-conditionals with equal variance, conditionally i.i.d.; and (i) an AR(1)-correlated Gaussian model with the same marginals but lag-one correlation φ=0.6 =0.6, evaluated while the belief update still assumes independence—a deliberate violation of Assumption 1 that makes the update overconfident (it double-counts correlated evidence). For each model we sweep each policy’s operating parameter and report its best total-cost point. The Bayesian rule attains lower total cost than the tuned fixed rule in all three models. Under misspecification its edge narrows but does not invert (total cost 0.1580.158 vs. 0.1660.166), and accuracy degrades gracefully (95.6%95.6\% vs. 95.8%95.8\% in the matched i.i.d. model): correlation costs performance, consistent with the calibration analysis, but does not break the method. Table 2: Robustness across observation models. Each policy is tuned to its best total-cost operating point per model; total cost == compute ++ expected error cost. AR(1) uses the i.i.d. belief update on correlated signals (misspecified). Numbers produced by the accompanying robustness script. Observation model Policy Accuracy Compute Total cost Beta, i.i.d. (baseline) Fixed-rule 0.9560.956 0.1170.117 0.1610.161 Bayesian myopic 0.9600.960 0.1110.111 0.1510.151 Gaussian, i.i.d. Fixed-rule 0.9580.958 0.1170.117 0.1600.160 Bayesian myopic 0.9580.958 0.1110.111 0.1530.153 Gaussian AR(1), φ=0.6 =0.6 Fixed-rule 0.9540.954 0.1200.120 0.1660.166 (misspecified update) Bayesian myopic 0.9560.956 0.1140.114 0.1580.158 8 Real-Model Validation The simulation of Section 7 verifies the derived policy in a world where the modelling assumptions hold by construction. Version 1 of this paper deferred any real-model evaluation, reporting only the pre-registered protocol of Section 9. This section, added in Version 1.1, takes a first, deliberately narrow step toward that protocol and runs the framework on a real hierarchical system. We are candid about scope: this is a single code-generation cascade, one model pair, greedy decoding, a single seed, and—importantly—a constant-threshold instantiation with a belief fit from running signal summaries rather than the recursive filter of Lemma 1. It is evidence that the competence signal and the escalation frontier behave as the theory predicts; it is not a test of the optimal-stopping dynamic program, which we leave to future work (Section 10). All numbers come from reproducible scripts with fixed seeds. Setup. The junior is Qwen2.5-Coder-1.5B-Instruct and the senior Qwen2.5-Coder-7B-Instruct, both served locally with token-level log-probabilities. We evaluate on the sanitized MBPP test split (257257 tasks), defining Y by execution against the task’s unit tests—an unambiguous correctness label. For each greedy junior generation we log per-step token entropy, next-token log-probability, and top-2 margin. The competence posterior BtB_t is a logistic regression on the running (cumulative) means of these three signals, evaluated with 55-fold cross-validation so that every reported belief is out-of-fold. The senior attempts every task once (greedy), giving the counterfactual needed to price escalation. Capability gap and escalation ceiling. The junior solves 62.3%62.3\% of tasks and the senior 80.9%80.9\%. Of the 9797 junior failures the senior rescues 5454 (55.7%55.7\%); the remaining 4343 (44.3%44.3\%) are failed by both models and form an irreducible floor no routing policy can cross. The escalation ceiling— junior successes plus every failure escalated—is therefore 214/257=83.3%214/257=83.3\%, not the senior’s marginal 80.9%80.9\%. This shared-failure floor is the real-model face of the constant-q caveat in Section 7: senior reliability is difficulty-correlated, so the achievable gain is bounded well below perfect rescue. The competence signal is informative but imperfect. The cross-validated posterior attains AUROC 0.7580.758 against eventual success. The per-step signal separates the classes early—mean entropy on eventual failures exceeds that on successes from ∼5% \!5\% of the generation—but non-monotonically: through the 2525–55%55\% band the instantaneous gap collapses and briefly inverts, precisely where confidently-wrong failures (low-entropy, incorrect) coincide with hard-but-correct successes (high-entropy, correct). This is the mechanism of Remark 1 and Proposition 3 observed directly: raw entropy is not the posterior, and confident errors are the binding failure mode. Cumulative belief discrimination rises in t (prediction (b)). Although the instantaneous signal is non-monotonic, the cumulative posterior BtB_t is not: its discrimination increases near-monotonically over the generation (Spearman ρ=0.93ρ=0.93 between generation fraction and AUROC of BtB_t; AUROC rising from 0.510.51 to 0.760.76). Through the middle band where the instantaneous signal collapses, BtB_t plateaus rather than declining— confidently-wrong tokens stop contributing fresh evidence, but the belief retains what it accumulated earlier—and resumes rising thereafter (Figure 5). This confirms pre-registered prediction (b) of Section 9 and is direct support for the paper’s central design choice: accumulate a calibrated belief rather than react to per-step uncertainty. We note the corresponding tension for early action: discrimination peaks at the horizon (t=Tt=T), so any policy that stops early necessarily acts on a weaker-than-terminal belief. The escalation frontier dominates post-hoc routing (prediction (a)). We compare a streaming policy—escalate at the first step where the running belief falls below a threshold τ, aborting the remaining junior generation—against post-hoc routing, which runs the junior to completion and escalates the least-confident fraction (the confidence-cascade family the protocol names as a baseline). Sweeping each policy’s operating parameter traces the cost–accuracy frontier of Figure 4. Streaming dominates post-hoc across the frontier: to reach 75%75\% accuracy it uses 30.2%30.2\% less total compute (14,84114,841 vs. 21,27321,273 generated tokens). More strikingly, streaming reaches 75%75\% accuracy—+12.7+12.7 points over the junior alone—at essentially the junior’s own compute (0.98×0.98× junior-only tokens): the tokens saved by aborting doomed generations offset the senior calls added. At τ=0.5τ=0.5 the policy escalates 37%37\% of tasks, catching them at a mean of 29%29\% of the way through generation, for 74.7%74.7\% accuracy. The advantage over post-hoc is structural: post-hoc pays every junior generation in full before it can route, whereas streaming stops paying for a generation the moment the belief turns against it. Table 3: Real-model comparison on MBPP (sanitized test, 257257 tasks; Qwen2.5-Coder 1.51.5B→ 77B). Compute is total generated tokens, normalised to junior-only. Post-hoc and streaming are reported at the operating point reaching 75%75\% accuracy; the belief is cross-validated. Numbers produced by the accompanying harvest and analysis scripts. Policy Accuracy Compute (× junior) Esc. rate Junior only 0.6230.623 1.001.00 0.000.00 Senior only 0.8090.809 — 1.001.00 Post-hoc routing (tuned) 0.7500.750 1.411.41 — Streaming (tuned τ) 0.7470.747 0.980.98 0.370.37 Figure 4: Real-model cost–accuracy frontier. Streaming escalation (purple) dominates post-hoc routing (teal) everywhere: it climbs from junior to senior accuracy at near-constant compute, because aborting doomed generations offsets the added senior calls. Dotted lines mark junior-only and senior-only accuracy. Figure 5: Discrimination of the cumulative belief BtB_t versus generation fraction (AUROC of BtB_t against eventual success, cross-validated). The curve rises near-monotonically (Spearman ρ=0.93ρ=0.93) and plateaus—rather than declining—through the shaded 2525–55%55\% band where the instantaneous signal collapses, confirming prediction (b). What this does and does not establish. Two of the three pre-registered predictions are supported on real data: the Bayesian frontier dominates the post-hoc baseline at equal cost (a), and the cumulative belief’s AUROC rises in t (b). Prediction (c)—that the accuracy gap over baselines shrinks as calibration error grows—we do not test here; the simulation’s calibration- sensitivity study (Figure 3) is its controlled analogue. Three limits bound the reading. First, the policy evaluated is a constant threshold, not the optimal-stopping schedule of Eqs. (5)–(6); by Theorem 2 a threshold rule is the right form, but the option-value machinery—the contribution over classical sequential testing—is not exercised, and the empirical margin of the dynamic program over the best constant threshold remains to be measured. Second, the belief is a running-mean proxy for the recursive filter of Lemma 1, as anticipated in Section 6. Third, this is one benchmark, one model pair, greedy decoding, and a single seed; the protocol’s reasoning and commonsense datasets, and the sampling-based baselines, remain open. Within those limits, the framework transfers: the signal is informative, the cumulative belief behaves as predicted, and confidence-routed escalation is markedly more compute-efficient than routing after the fact. 9 Protocol for Real LLM Systems We pre-registered the full evaluation below to make the empirical test falsifiable; it was specified in Version 1 of this paper as a plan, before any real-model run. Section 8, added in Version 1.1, reports its first execution on a code-generation cascade—keeping prediction and test separated in time—and finds two of the three predictions confirmed. Systems. Junior: a small instruction/coding model served with token-level logit access (so that entropy and next-token margin are available at each step). Senior: a substantially stronger reasoning/coding model. Both served through an inference stack that exposes per-token log-probabilities. Signals. Per-step ete_t candidates: token entropy, next-token probability margin, and a single-pass semantic-entropy probe [11]. Each is calibrated separately so that comparisons isolate the signal’s quality. Belief fitting and calibration. On a labelled development split, run the junior, record (e1:T,Y)(e_1:T,Y), fit λ, and calibrate BtB_t by isotonic regression. Report reliability diagrams and expected calibration error (ECE) for BtB_t at several t. Baselines. (i) junior-only; (i) senior-only; (i) query-level router [19]; (iv) confidence cascade [8]; (v) sampling-based semantic-entropy deferral [3]; (vi) our myopic and optimal-stopping policies. Datasets. A reasoning set (e.g. multi-hop QA), a commonsense set with natural easy/hard structure, and a code set with executable unit tests to define Y unambiguously; plus an in-domain deployment set. Metrics. Cost–accuracy Pareto frontier (primary); escalation precision/recall; calibration (ECE, Brier, AUROC of BtB_t vs. Y); and decision/end-to-end latency, including the cost of computing ete_t. Falsifiable predictions. If the framework transfers, then (a) the Bayesian policy’s frontier should dominate the query-level router and the confidence cascade at equal cost; (b) AUROC of BtB_t should rise monotonically in t; and (c) the accuracy gap over baselines should shrink as calibration error grows, per Proposition 3. Failure of (a)–(c) would falsify the central claims. 10 Limitations Modelling assumptions. Assumption 1 (conditional-i.i.d. signals) is false for real token streams, which are correlated and non-stationary; the update then becomes an approximation and λ should be replaced by a sequence model of the signal. (No shape assumption on the signal densities is needed for the threshold structure itself; Lemma 2 holds for any density pair.) Confidently-wrong predictions. As Proposition 3 and Figure 3 show, the method inherits the calibration of its signal. Where the junior is confidently wrong, no threshold policy on that signal can help; combining epistemic signals with lightweight cross-model agreement [10] is a natural remedy. Logit access. Token-level signals require an inference stack that exposes log-probabilities; many managed APIs return them only after generation, or not at all, restricting deployment to self-hosted or logprob-exposing endpoints. Senior independence. We treat q as constant; in practice senior success correlates with query difficulty, and a difficulty-conditioned q(x)q(x) would tighten the decision. Scope of the real-model study. Version 1 flagged the absence of real-model results as its central limitation; Version 1.1 partially closes it. Section 8 validates the framework on a single code-generation cascade, one model pair, greedy decoding, and a single seed. Three limits bound it: the policy evaluated is a constant threshold, not the optimal-stopping schedule of Eqs. (5)–(6), so the option-value contribution over classical sequential testing is not yet exercised empirically; the belief is a running-mean proxy for the recursive filter of Lemma 1; and prediction (c) of Section 9 remains untested on real models. The remaining datasets and baselines of the protocol are open. 11 Conclusion We framed the question of when an agent should ask for help as Bayesian self-escalation: a junior model tracks an online posterior over its own eventual success from the uncertainty signals it emits and defers to a stronger model when the expected utility of deferral wins. The framework yields a closed-form myopic threshold, an optimal-stopping characterisation with a proven threshold structure, and a regret bound that pins the method’s success to belief calibration. A controlled simulation study confirms the predicted behaviour, including the myopic–optimal threshold gap and the calibration-driven failure mode. The decisive next step is the real-system protocol of Section 9; whether token-level signals on current LLMs are calibrated enough to realise these gains is, in the end, an empirical question this paper is designed to make testable. Reproducibility. The simulation study (Section 7) is generated by three self-contained scripts (main simulation, robustness study, and sample-complexity experiment); the real-model validation (Section 8) by a harvest script and three post-hoc analysis scripts (separation and frontier, streaming escalation, and the BtB_t AUROC test). All are released with this paper. The paper and simulation scripts are permanently archived at DOI: 10.5281/zenodo.21330787; the full code, including the real-model pipeline, is at github.com/nadeem-shaikh/llm-self-escalation. References [1] L. Chen, M. Zaharia, and J. Zou. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv:2305.05176, 2023. [2] C. K. Chow. On optimum recognition error and reject tradeoff. IEEE Trans. Information Theory, 16(1):41–46, 1970. [3] S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630:625–630, 2024. [4] T. S. Ferguson. Optimal Stopping and Applications. Electronic text, UCLA, 2006. [5] Y. Geifman and R. El-Yaniv. Selective classification for deep neural networks. In NeurIPS, 2017. [6] Y. Geifman and R. El-Yaniv. SelectiveNet: A deep neural network with an integrated reject option. In ICML, 2019. [7] A. Graves. Adaptive computation time for recurrent neural networks. arXiv:1603.08983, 2016. [8] W. Jitkrittum, N. Gupta, A. K. Menon, H. Narasimhan, A. Rawat, and S. Kumar. When does confidence-based cascade deferral suffice? In NeurIPS, 2023. [9] S. Kadavath, T. Conerly, A. Askell, et al. Language models (mostly) know what they know. arXiv:2207.05221, 2022. [10] S. Kolawole, D. Dennis, A. Talwalkar, and V. Smith. Agreement-based cascading for efficient inference. Transactions on Machine Learning Research, 2025. [11] J. Kossen, J. Han, M. Razzak, L. Schut, S. Malik, and Y. Gal. Semantic entropy probes: Robust and cheap hallucination detection in LLMs. arXiv:2406.15927, 2024. [12] V. Krishnamurthy. Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. Cambridge University Press, 2016. [13] L. Kuhn, Y. Gal, and S. Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In ICLR, 2023. [14] Y. Leviathan, M. Kalman, and Y. Matias. Fast inference from transformers via speculative decoding. In ICML, 2023. [15] W. S. Lovejoy. Some monotonicity results for partially observed Markov decision processes. Operations Research, 35(5):736–743, 1987. [16] A. Madaan, N. Tandon, P. Gupta, et al. Self-Refine: Iterative refinement with self-feedback. In NeurIPS, 2023. [17] D. Madras, T. Pitassi, and R. Zemel. Predict responsibly: Improving fairness and accuracy by learning to defer. In NeurIPS, 2018. [18] H. Mozannar and D. Sontag. Consistent estimators for learning to defer to an expert. In ICML, 2020. [19] I. Ong, A. Almahairi, V. Wu, W.-L. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica. RouteLLM: Learning to route LLMs with preference data. arXiv:2406.18665, 2024. [20] G. Peskir and A. Shiryaev. Optimal Stopping and Free-Boundary Problems. Birkhäuser, 2006. [21] T. Schuster, A. Fisch, J. Gupta, M. Dehghani, D. Bahri, V. Q. Tran, Y. Tay, and D. Metzler. Confident adaptive language modeling. In NeurIPS, 2022. [22] C. Snell, J. Lee, K. Xu, and A. Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv:2408.03314, 2024. [23] D. Soiffer, S. Kolawole, and V. Smith. Semantic agreement enables efficient open-ended LLM cascades. In EMNLP (Industry Track), 2025. [24] R. Verma and E. Nalisnick. Calibrated learning to defer with one-vs-all classifiers. In ICML, 2022. [25] A. Wald. Sequential tests of statistical hypotheses. Annals of Mathematical Statistics, 16(2):117–186, 1945. [26] A. Wald and J. Wolfowitz. Optimum character of the sequential probability ratio test. Annals of Mathematical Statistics, 19(3):326–339, 1948.